2024-04-14T08:21:11.195 INFO:root:teuthology version: 0.0.1.dev265+gd7d6e60 2024-04-14T08:21:11.195 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_teuthology@teuthology --name teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} -- /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369/orig.config.yaml 2024-04-14T08:21:11.242 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T08:21:11.368 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369 branch: main description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} email: ceph-qa@ceph.com first_in_suite: false job_id: '7638369' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: d7e176b41636f150fe82aaedea82def3e9af0203 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d7e176b41636f150fe82aaedea82def3e9af0203 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: b3cf8ca51f5d8edaa7cbce710f0de3a66b6f6be9 owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1489 sha1: d7e176b41636f150fe82aaedea82def3e9af0203 sleep_before_teardown: 0 subset: 22/512 suite: fs suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_b3cf8ca51f5d8edaa7cbce710f0de3a66b6f6be9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b3cf8ca51f5d8edaa7cbce710f0de3a66b6f6be9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLf2A6OlxKixkzXPIy/I95aPoML/ifPAeBpNg+k0+A/BYkM6XAvMsOlkDx7+/5WvDC3MQjg0rXwnu5LDckBmdq0= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF3YkGOI+hKcHUDd0JnNpKEdi2eUoiseAifR6sFT8/i8WM/z1HIrPCzlwlYhh05ID4cTD0ylLad0oCa3thhmXfQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: d7d6e60867cadf19fbfd7655a551ae7d58060757 timestamp: 2024-04-03_20:24:15 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-14T08:21:11.368 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b3cf8ca51f5d8edaa7cbce710f0de3a66b6f6be9/qa; will attempt to use it 2024-04-14T08:21:11.369 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b3cf8ca51f5d8edaa7cbce710f0de3a66b6f6be9/qa/tasks 2024-04-14T08:21:11.369 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-14T08:21:11.371 INFO:teuthology.task.internal:Checking packages... 2024-04-14T08:21:11.395 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd7e176b41636f150fe82aaedea82def3e9af0203' 2024-04-14T08:21:11.395 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-14T08:21:11.395 INFO:teuthology.packaging:ref: None 2024-04-14T08:21:11.395 INFO:teuthology.packaging:tag: None 2024-04-14T08:21:11.395 INFO:teuthology.packaging:branch: main 2024-04-14T08:21:11.396 INFO:teuthology.packaging:sha1: d7e176b41636f150fe82aaedea82def3e9af0203 2024-04-14T08:21:11.396 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-04-14T08:21:11.635 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3080.gc2d8c315 2024-04-14T08:21:11.637 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-14T08:21:11.644 INFO:teuthology.task.internal:no buildpackages task found 2024-04-14T08:21:11.644 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-14T08:21:11.658 INFO:teuthology.task.internal:Saving configuration 2024-04-14T08:21:11.678 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-14T08:21:11.684 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-14T08:21:11.714 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-14 08:15:50.089990', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLf2A6OlxKixkzXPIy/I95aPoML/ifPAeBpNg+k0+A/BYkM6XAvMsOlkDx7+/5WvDC3MQjg0rXwnu5LDckBmdq0='} 2024-04-14T08:21:11.735 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-14 08:15:50.091053', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF3YkGOI+hKcHUDd0JnNpKEdi2eUoiseAifR6sFT8/i8WM/z1HIrPCzlwlYhh05ID4cTD0ylLad0oCa3thhmXfQ='} 2024-04-14T08:21:11.736 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-14T08:21:11.743 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-14T08:21:11.743 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-14T08:21:11.743 INFO:teuthology.run_tasks:Running task console_log... 2024-04-14T08:21:11.860 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f53d71ea0d0>, signals=[15]) 2024-04-14T08:21:11.860 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-14T08:21:11.869 INFO:teuthology.task.internal:Opening connections... 2024-04-14T08:21:11.869 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:21:11.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T08:21:11.967 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:21:11.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T08:21:12.056 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-14T08:21:12.063 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2024-04-14T08:21:12.082 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2024-04-14T08:21:12.082 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-04-14T08:21:12.138 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2024-04-14T08:21:12.139 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2024-04-14T08:21:12.139 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2024-04-14T08:21:12.139 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2024-04-14T08:21:12.139 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2024-04-14T08:21:12.139 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2024-04-14T08:21:12.139 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T08:21:12.139 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2024-04-14T08:21:12.139 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2024-04-14T08:21:12.139 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T08:21:12.139 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2024-04-14T08:21:12.139 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T08:21:12.139 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T08:21:12.140 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T08:21:12.140 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-04-14T08:21:12.163 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2024-04-14T08:21:12.181 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2024-04-14T08:21:12.181 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2024-04-14T08:21:12.237 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2024-04-14T08:21:12.238 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="9" 2024-04-14T08:21:12.238 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2024-04-14T08:21:12.238 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2024-04-14T08:21:12.238 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="9" 2024-04-14T08:21:12.239 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el9" 2024-04-14T08:21:12.239 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T08:21:12.239 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2024-04-14T08:21:12.239 INFO:teuthology.orchestra.run.smithi177.stdout:LOGO="fedora-logo-icon" 2024-04-14T08:21:12.239 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T08:21:12.239 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2024-04-14T08:21:12.239 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T08:21:12.239 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T08:21:12.239 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T08:21:12.239 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2024-04-14T08:21:12.255 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-14T08:21:12.266 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-14T08:21:12.341 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-14T08:21:12.341 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2024-04-14T08:21:12.344 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2024-04-14T08:21:12.360 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-14T08:21:12.365 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-14T08:21:12.366 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2024-04-14T08:21:12.401 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2024-04-14T08:21:12.501 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-14T08:21:12.557 INFO:teuthology.run_tasks:Running task kernel... 2024-04-14T08:21:12.575 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-14T08:21:12.576 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-14T08:21:12.576 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-14T08:21:12.576 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-14T08:21:12.577 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-14T08:21:12.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:21:12.593 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2024-04-14T08:21:12.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:21:12.594 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-04-14T08:21:12.650 INFO:teuthology.orchestra.run.smithi172.stdout:5.14.0-361.el9.x86_64 2024-04-14T08:21:12.651 INFO:teuthology.task.kernel:Running kernel on smithi172: 5.14.0-361.el9.x86_64 2024-04-14T08:21:12.651 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum install -y kernel 2024-04-14T08:21:12.652 INFO:teuthology.orchestra.run.smithi177.stdout:5.14.0-361.el9.x86_64 2024-04-14T08:21:12.652 INFO:teuthology.task.kernel:Running kernel on smithi177: 5.14.0-361.el9.x86_64 2024-04-14T08:21:12.652 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum install -y kernel 2024-04-14T08:21:15.007 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-04-14T08:21:15.028 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - AppStream 11 MB/s | 8.1 MB 00:00 2024-04-14T08:21:16.670 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - BaseOS 7.7 MB/s | 2.5 MB 00:00 2024-04-14T08:21:16.691 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.5 MB 00:00 2024-04-14T08:21:17.665 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-14T08:21:21.326 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.0 MB 00:04 2024-04-14T08:21:23.169 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-14T08:21:28.220 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.2 MB 00:01 2024-04-14T08:21:28.481 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 1.9 MB/s | 19 MB 00:09 2024-04-14T08:21:30.620 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-14T08:21:32.499 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-14T08:21:33.492 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.2 MB 00:01 2024-04-14T08:21:36.149 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-04-14T08:21:37.108 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-04-14T08:21:37.621 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-14T08:21:39.869 INFO:teuthology.orchestra.run.smithi177.stderr:Modular dependency problems: 2024-04-14T08:21:39.869 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-04-14T08:21:39.869 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 1: conflicting requests 2024-04-14T08:21:39.869 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.869 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 2: conflicting requests 2024-04-14T08:21:39.869 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.869 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 3: conflicting requests 2024-04-14T08:21:39.869 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.869 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 4: conflicting requests 2024-04-14T08:21:39.870 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.870 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 5: conflicting requests 2024-04-14T08:21:39.870 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.870 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 6: conflicting requests 2024-04-14T08:21:39.870 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.870 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 7: conflicting requests 2024-04-14T08:21:39.870 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.870 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 8: conflicting requests 2024-04-14T08:21:39.870 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.870 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 9: conflicting requests 2024-04-14T08:21:39.870 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.870 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 10: conflicting requests 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 11: conflicting requests 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 12: conflicting requests 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 13: conflicting requests 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 14: conflicting requests 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 15: conflicting requests 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 16: conflicting requests 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 17: conflicting requests 2024-04-14T08:21:39.871 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 18: conflicting requests 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 19: conflicting requests 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 20: conflicting requests 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 21: conflicting requests 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 22: conflicting requests 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 23: conflicting requests 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 24: conflicting requests 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.872 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 25: conflicting requests 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 26: conflicting requests 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 27: conflicting requests 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 28: conflicting requests 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 29: conflicting requests 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 30: conflicting requests 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 31: conflicting requests 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 32: conflicting requests 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.873 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 33: conflicting requests 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 34: conflicting requests 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 35: conflicting requests 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 36: conflicting requests 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 37: conflicting requests 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T08:21:39.874 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 39: conflicting requests 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.875 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.876 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.877 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.877 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.877 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T08:21:39.877 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.877 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.877 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.877 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.877 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T08:21:39.877 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.877 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.877 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T08:21:39.877 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T08:21:40.254 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-14T08:21:40.254 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-14T08:21:40.359 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-04-14T08:21:40.370 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-04-14T08:21:40.370 INFO:teuthology.orchestra.run.smithi177.stdout: Package Architecture Version Repository Size 2024-04-14T08:21:40.371 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-04-14T08:21:40.371 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-04-14T08:21:40.371 INFO:teuthology.orchestra.run.smithi177.stdout: kernel x86_64 5.14.0-435.el9 baseos 214 k 2024-04-14T08:21:40.371 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-04-14T08:21:40.371 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-core x86_64 5.14.0-435.el9 baseos 15 M 2024-04-14T08:21:40.371 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules x86_64 5.14.0-435.el9 baseos 34 M 2024-04-14T08:21:40.371 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-core x86_64 5.14.0-435.el9 baseos 28 M 2024-04-14T08:21:40.371 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:21:40.371 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-04-14T08:21:40.371 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2024-04-14T08:21:40.371 INFO:teuthology.orchestra.run.smithi177.stdout:Install 4 Packages 2024-04-14T08:21:40.371 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:21:40.372 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 77 M 2024-04-14T08:21:40.372 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 123 M 2024-04-14T08:21:40.372 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-04-14T08:21:41.031 INFO:teuthology.orchestra.run.smithi177.stdout:(1/4): kernel-5.14.0-435.el9.x86_64.rpm 412 kB/s | 214 kB 00:00 2024-04-14T08:21:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-14T08:21:44.468 INFO:teuthology.orchestra.run.smithi172.stderr:Modular dependency problems: 2024-04-14T08:21:44.468 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:21:44.468 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 1: conflicting requests 2024-04-14T08:21:44.468 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.468 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 2: conflicting requests 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 3: conflicting requests 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 4: conflicting requests 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 5: conflicting requests 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 6: conflicting requests 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 7: conflicting requests 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 8: conflicting requests 2024-04-14T08:21:44.469 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 9: conflicting requests 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 10: conflicting requests 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 11: conflicting requests 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 12: conflicting requests 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 13: conflicting requests 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 14: conflicting requests 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 15: conflicting requests 2024-04-14T08:21:44.470 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 16: conflicting requests 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 17: conflicting requests 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 18: conflicting requests 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 19: conflicting requests 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 20: conflicting requests 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 21: conflicting requests 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 22: conflicting requests 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.471 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 23: conflicting requests 2024-04-14T08:21:44.472 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.472 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 24: conflicting requests 2024-04-14T08:21:44.472 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.472 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 25: conflicting requests 2024-04-14T08:21:44.472 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.472 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 26: conflicting requests 2024-04-14T08:21:44.472 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.472 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 27: conflicting requests 2024-04-14T08:21:44.472 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.472 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 28: conflicting requests 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 29: conflicting requests 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 30: conflicting requests 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 31: conflicting requests 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 32: conflicting requests 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 33: conflicting requests 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 34: conflicting requests 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 35: conflicting requests 2024-04-14T08:21:44.473 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.474 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 36: conflicting requests 2024-04-14T08:21:44.474 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.474 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 37: conflicting requests 2024-04-14T08:21:44.474 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.474 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.474 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.474 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T08:21:44.474 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.474 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.474 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.474 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.474 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 39: conflicting requests 2024-04-14T08:21:44.475 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.475 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.475 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.475 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.475 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.475 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.475 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T08:21:44.475 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.476 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.477 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T08:21:44.907 INFO:teuthology.orchestra.run.smithi172.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-14T08:21:44.907 INFO:teuthology.orchestra.run.smithi172.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-14T08:21:45.005 INFO:teuthology.orchestra.run.smithi177.stdout:(2/4): kernel-modules-5.14.0-435.el9.x86_64.rpm 7.5 MB/s | 34 MB 00:04 2024-04-14T08:21:45.023 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-04-14T08:21:45.032 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-04-14T08:21:45.032 INFO:teuthology.orchestra.run.smithi172.stdout: Package Architecture Version Repository Size 2024-04-14T08:21:45.032 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-04-14T08:21:45.032 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-04-14T08:21:45.032 INFO:teuthology.orchestra.run.smithi172.stdout: kernel x86_64 5.14.0-435.el9 baseos 214 k 2024-04-14T08:21:45.032 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-04-14T08:21:45.032 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-core x86_64 5.14.0-435.el9 baseos 15 M 2024-04-14T08:21:45.032 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules x86_64 5.14.0-435.el9 baseos 34 M 2024-04-14T08:21:45.032 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-core x86_64 5.14.0-435.el9 baseos 28 M 2024-04-14T08:21:45.032 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:21:45.033 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-04-14T08:21:45.033 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-04-14T08:21:45.033 INFO:teuthology.orchestra.run.smithi172.stdout:Install 4 Packages 2024-04-14T08:21:45.033 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:21:45.033 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 77 M 2024-04-14T08:21:45.033 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 123 M 2024-04-14T08:21:45.033 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-04-14T08:21:45.243 INFO:teuthology.orchestra.run.smithi172.stdout:(1/4): kernel-5.14.0-435.el9.x86_64.rpm 1.4 MB/s | 214 kB 00:00 2024-04-14T08:21:45.297 INFO:teuthology.orchestra.run.smithi177.stdout:(3/4): kernel-modules-core-5.14.0-435.el9.x86_6 6.5 MB/s | 28 MB 00:04 2024-04-14T08:21:45.448 INFO:teuthology.orchestra.run.smithi177.stdout:(4/4): kernel-core-5.14.0-435.el9.x86_64.rpm 3.1 MB/s | 15 MB 00:04 2024-04-14T08:21:45.448 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-04-14T08:21:45.448 INFO:teuthology.orchestra.run.smithi177.stdout:Total 15 MB/s | 77 MB 00:05 2024-04-14T08:21:45.551 INFO:teuthology.orchestra.run.smithi172.stdout:(2/4): kernel-core-5.14.0-435.el9.x86_64.rpm 33 MB/s | 15 MB 00:00 2024-04-14T08:21:46.018 INFO:teuthology.orchestra.run.smithi172.stdout:(3/4): kernel-modules-5.14.0-435.el9.x86_64.rpm 36 MB/s | 34 MB 00:00 2024-04-14T08:21:46.119 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-04-14T08:21:46.419 INFO:teuthology.orchestra.run.smithi172.stdout:(4/4): kernel-modules-core-5.14.0-435.el9.x86_6 24 MB/s | 28 MB 00:01 2024-04-14T08:21:46.419 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-04-14T08:21:46.419 INFO:teuthology.orchestra.run.smithi172.stdout:Total 55 MB/s | 77 MB 00:01 2024-04-14T08:21:46.507 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-04-14T08:21:46.507 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-04-14T08:21:47.092 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-04-14T08:21:47.438 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-04-14T08:21:47.438 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-04-14T08:21:47.511 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-04-14T08:21:47.511 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-04-14T08:21:48.459 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-04-14T08:21:48.459 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-04-14T08:21:49.964 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-04-14T08:21:50.574 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-modules-core-5.14.0-435.el9.x86_64 1/4 2024-04-14T08:21:50.657 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-14T08:21:51.038 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-04-14T08:21:51.621 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-modules-core-5.14.0-435.el9.x86_64 1/4 2024-04-14T08:21:51.711 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-14T08:21:53.430 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-14T08:21:53.568 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-14T08:21:54.494 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-14T08:21:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-14T08:21:58.300 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-14T08:21:58.343 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-5.14.0-435.el9.x86_64 4/4 2024-04-14T08:21:59.330 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-14T08:21:59.433 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-5.14.0-435.el9.x86_64 4/4 2024-04-14T08:22:03.084 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-core-5.14.0-435.el9.x86_64 4/4 2024-04-14T08:22:04.217 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-core-5.14.0-435.el9.x86_64 4/4 2024-04-14T08:22:32.403 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-core-5.14.0-435.el9.x86_64 4/4 2024-04-14T08:22:32.404 INFO:teuthology.orchestra.run.smithi177.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-14T08:22:32.404 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:22:32.449 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-5.14.0-435.el9.x86_64 4/4 2024-04-14T08:22:32.850 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-5.14.0-435.el9.x86_64 4/4 2024-04-14T08:22:32.850 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-5.14.0-435.el9.x86_64 1/4 2024-04-14T08:22:32.850 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-14T08:22:32.851 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-14T08:22:33.267 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-modules-core-5.14.0-435.el9.x86_64 4/4 2024-04-14T08:22:33.267 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:22:33.267 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-04-14T08:22:33.267 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-5.14.0-435.el9.x86_64 2024-04-14T08:22:33.268 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-core-5.14.0-435.el9.x86_64 2024-04-14T08:22:33.268 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-5.14.0-435.el9.x86_64 2024-04-14T08:22:33.268 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-core-5.14.0-435.el9.x86_64 2024-04-14T08:22:33.268 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:22:33.268 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-04-14T08:22:33.596 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2024-04-14T08:22:33.630 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-core-5.14.0-435.el9.x86_64 4/4 2024-04-14T08:22:33.630 INFO:teuthology.orchestra.run.smithi172.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-14T08:22:33.630 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:22:33.675 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-5.14.0-435.el9.x86_64 4/4 2024-04-14T08:22:33.738 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-5.14.0-435.el9.x86_64 2024-04-14T08:22:33.738 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-435.el9.x86_64 2024-04-14T08:22:33.738 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-435.el9.x86_64 2024-04-14T08:22:33.738 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-14T08:22:33.739 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-14T08:22:33.739 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-04-14T08:22:33.739 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi177.front.sepia.ceph.com, path=None, version=distro) 2024-04-14T08:22:33.739 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2024-04-14T08:22:33.819 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-5.14.0-435.el9.x86_64 2024-04-14T08:22:33.820 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-435.el9.x86_64 2024-04-14T08:22:33.820 DEBUG:teuthology.orchestra.run.smithi177:> sudo rpm -qi grub2-tools 2024-04-14T08:22:33.854 INFO:teuthology.orchestra.run.smithi177.stdout:Name : grub2-tools 2024-04-14T08:22:33.854 INFO:teuthology.orchestra.run.smithi177.stdout:Epoch : 1 2024-04-14T08:22:33.854 INFO:teuthology.orchestra.run.smithi177.stdout:Version : 2.06 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:Release : 68.el9 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:Architecture: x86_64 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:Group : Unspecified 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:Size : 8264254 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:License : GPLv3+ 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:Packager : builder@centos.org 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:Vendor : CentOS 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-14T08:22:33.855 INFO:teuthology.orchestra.run.smithi177.stdout:Summary : Support tools for GRUB. 2024-04-14T08:22:33.856 INFO:teuthology.orchestra.run.smithi177.stdout:Description : 2024-04-14T08:22:33.856 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:22:33.856 INFO:teuthology.orchestra.run.smithi177.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-14T08:22:33.856 INFO:teuthology.orchestra.run.smithi177.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-14T08:22:33.856 INFO:teuthology.orchestra.run.smithi177.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-14T08:22:33.856 INFO:teuthology.orchestra.run.smithi177.stdout:hardware devices. 2024-04-14T08:22:33.856 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:22:33.856 INFO:teuthology.orchestra.run.smithi177.stdout:This subpackage provides tools for support of all platforms. 2024-04-14T08:22:33.857 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-14T08:22:33.857 INFO:teuthology.task.kernel:Updating grub on smithi177 to boot 5.14.0-435.el9.x86_64 2024-04-14T08:22:33.857 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-14T08:22:33.996 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-5.14.0-435.el9.x86_64 4/4 2024-04-14T08:22:33.996 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-5.14.0-435.el9.x86_64 1/4 2024-04-14T08:22:33.996 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-14T08:22:33.997 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-14T08:22:34.465 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-modules-core-5.14.0-435.el9.x86_64 4/4 2024-04-14T08:22:34.465 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:22:34.465 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-04-14T08:22:34.465 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-5.14.0-435.el9.x86_64 2024-04-14T08:22:34.465 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-core-5.14.0-435.el9.x86_64 2024-04-14T08:22:34.465 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-5.14.0-435.el9.x86_64 2024-04-14T08:22:34.466 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-core-5.14.0-435.el9.x86_64 2024-04-14T08:22:34.466 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:22:34.466 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-04-14T08:22:34.485 INFO:teuthology.orchestra.run.smithi177.stderr:Generating grub configuration file ... 2024-04-14T08:22:34.833 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q kernel | sort -rV | head -n 1 2024-04-14T08:22:34.939 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-5.14.0-435.el9.x86_64 2024-04-14T08:22:34.940 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-435.el9.x86_64 2024-04-14T08:22:34.940 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-435.el9.x86_64 2024-04-14T08:22:34.940 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-14T08:22:34.940 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-14T08:22:34.940 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-04-14T08:22:34.941 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi172.front.sepia.ceph.com, path=None, version=distro) 2024-04-14T08:22:34.941 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q kernel | sort -rV | head -n 1 2024-04-14T08:22:35.024 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-5.14.0-435.el9.x86_64 2024-04-14T08:22:35.028 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-435.el9.x86_64 2024-04-14T08:22:35.028 DEBUG:teuthology.orchestra.run.smithi172:> sudo rpm -qi grub2-tools 2024-04-14T08:22:35.062 INFO:teuthology.orchestra.run.smithi172.stdout:Name : grub2-tools 2024-04-14T08:22:35.062 INFO:teuthology.orchestra.run.smithi172.stdout:Epoch : 1 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:Version : 2.06 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:Release : 68.el9 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:Architecture: x86_64 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:Group : Unspecified 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:Size : 8264254 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:License : GPLv3+ 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:Packager : builder@centos.org 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:Vendor : CentOS 2024-04-14T08:22:35.063 INFO:teuthology.orchestra.run.smithi172.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-14T08:22:35.064 INFO:teuthology.orchestra.run.smithi172.stdout:Summary : Support tools for GRUB. 2024-04-14T08:22:35.064 INFO:teuthology.orchestra.run.smithi172.stdout:Description : 2024-04-14T08:22:35.064 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:22:35.064 INFO:teuthology.orchestra.run.smithi172.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-14T08:22:35.064 INFO:teuthology.orchestra.run.smithi172.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-14T08:22:35.064 INFO:teuthology.orchestra.run.smithi172.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-14T08:22:35.064 INFO:teuthology.orchestra.run.smithi172.stdout:hardware devices. 2024-04-14T08:22:35.064 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:22:35.064 INFO:teuthology.orchestra.run.smithi172.stdout:This subpackage provides tools for support of all platforms. 2024-04-14T08:22:35.065 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-14T08:22:35.065 INFO:teuthology.task.kernel:Updating grub on smithi172 to boot 5.14.0-435.el9.x86_64 2024-04-14T08:22:35.066 DEBUG:teuthology.orchestra.run.smithi172:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-14T08:22:35.669 INFO:teuthology.orchestra.run.smithi172.stderr:Generating grub configuration file ... 2024-04-14T08:22:35.788 INFO:teuthology.orchestra.run.smithi177.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-14T08:22:35.822 INFO:teuthology.orchestra.run.smithi177.stderr:done 2024-04-14T08:22:35.824 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-04-14T08:22:35.839 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.7ci7a5sGcN 2024-04-14T08:22:35.839 DEBUG:teuthology.orchestra.run.smithi177:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7ci7a5sGcN 2024-04-14T08:22:35.908 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.7ci7a5sGcN 2024-04-14T08:22:36.043 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.7ci7a5sGcN is 6KB 2024-04-14T08:22:36.091 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.7ci7a5sGcN 2024-04-14T08:22:36.106 DEBUG:teuthology.orchestra.run.smithi177:> sudo /bin/ls /boot/loader/entries || true 2024-04-14T08:22:36.170 INFO:teuthology.orchestra.run.smithi177.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-14T08:22:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-14T08:22:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-14T08:22:36.171 INFO:teuthology.orchestra.run.smithi177.stdout:15949e3c88704300882482eddfd7baae-5.14.0-435.el9.x86_64.conf 2024-04-14T08:22:36.172 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-435.el9.x86_64 2024-04-14T08:22:36.343 DEBUG:teuthology.orchestra.run.smithi177:> sudo shutdown -r now 2024-04-14T08:22:37.006 INFO:teuthology.orchestra.run.smithi172.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-14T08:22:37.043 INFO:teuthology.orchestra.run.smithi172.stderr:done 2024-04-14T08:22:37.046 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-04-14T08:22:37.062 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.d8vQCYTjqn 2024-04-14T08:22:37.062 DEBUG:teuthology.orchestra.run.smithi172:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.d8vQCYTjqn 2024-04-14T08:22:37.131 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.d8vQCYTjqn 2024-04-14T08:22:37.271 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.d8vQCYTjqn is 6KB 2024-04-14T08:22:37.320 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.d8vQCYTjqn 2024-04-14T08:22:37.334 DEBUG:teuthology.orchestra.run.smithi172:> sudo /bin/ls /boot/loader/entries || true 2024-04-14T08:22:37.399 INFO:teuthology.orchestra.run.smithi172.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-14T08:22:37.399 INFO:teuthology.orchestra.run.smithi172.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-14T08:22:37.399 INFO:teuthology.orchestra.run.smithi172.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-14T08:22:37.399 INFO:teuthology.orchestra.run.smithi172.stdout:15949e3c88704300882482eddfd7baae-5.14.0-435.el9.x86_64.conf 2024-04-14T08:22:37.400 DEBUG:teuthology.orchestra.run.smithi172:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-435.el9.x86_64 2024-04-14T08:22:37.538 DEBUG:teuthology.orchestra.run.smithi172:> sudo shutdown -r now 2024-04-14T08:22:37.624 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-04-14T08:23:06.375 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-14T08:23:06.375 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-04-14T08:23:06.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T08:23:07.541 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-14T08:23:07.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-14T08:23:07.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T08:23:24.703 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-04-14T08:23:25.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-14T08:23:33.710 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-04-14T08:23:33.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T08:23:34.984 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-14T08:23:34.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T08:23:36.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-04-14T08:23:38.046 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-14T08:23:48.818 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-04-14T08:23:48.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T08:23:50.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-14T08:23:50.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T08:24:20.619 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-04-14T08:24:21.008 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-04-14T08:24:21.008 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-435.el9.x86_64"... 2024-04-14T08:24:21.008 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2024-04-14T08:24:21.023 INFO:teuthology.orchestra.run.smithi177.stdout:5.14.0-435.el9.x86_64 2024-04-14T08:24:21.023 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-435.el9.x86_64 vs 5.14.0-435.el9.x86_64 2024-04-14T08:24:21.023 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-14T08:24:21.023 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-14T08:24:22.024 DEBUG:teuthology.parallel:result is None 2024-04-14T08:24:50.052 DEBUG:teuthology.orchestra.remote:timed out 2024-04-14T08:25:05.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-14T08:25:05.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T08:25:05.438 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-04-14T08:25:05.852 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-14T08:25:05.852 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-435.el9.x86_64"... 2024-04-14T08:25:05.853 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2024-04-14T08:25:05.868 INFO:teuthology.orchestra.run.smithi172.stdout:5.14.0-435.el9.x86_64 2024-04-14T08:25:05.869 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-435.el9.x86_64 vs 5.14.0-435.el9.x86_64 2024-04-14T08:25:05.869 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-14T08:25:05.869 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-14T08:25:06.870 DEBUG:teuthology.parallel:result is None 2024-04-14T08:25:06.871 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-14T08:25:06.878 INFO:teuthology.task.internal:Creating test directory... 2024-04-14T08:25:06.879 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-14T08:25:06.881 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-14T08:25:06.902 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-14T08:25:06.909 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-14T08:25:06.918 INFO:teuthology.task.internal:Creating archive directory... 2024-04-14T08:25:06.919 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-14T08:25:06.940 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-14T08:25:06.982 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-14T08:25:06.990 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-14T08:25:06.990 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-14T08:25:07.017 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-14T08:25:07.098 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-14T08:25:07.118 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-14T08:25:07.137 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-14T08:25:07.162 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-14T08:25:07.164 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-14T08:25:07.173 INFO:teuthology.task.internal:Configuring sudo... 2024-04-14T08:25:07.173 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-14T08:25:07.176 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-14T08:25:07.205 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-14T08:25:07.215 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-14T08:25:07.215 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-14T08:25:07.247 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-14T08:25:07.263 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-14T08:25:07.346 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-14T08:25:07.398 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-14T08:25:07.482 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-14T08:25:07.526 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:25:07.526 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-14T08:25:07.593 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-14T08:25:07.636 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-14T08:25:07.691 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-14T08:25:07.772 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-14T08:25:07.822 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:25:07.822 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-14T08:25:07.892 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2024-04-14T08:25:07.894 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2024-04-14T08:25:07.958 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-14T08:25:07.986 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-14T08:25:08.311 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-14T08:25:08.319 INFO:teuthology.task.internal:Starting timer... 2024-04-14T08:25:08.319 INFO:teuthology.run_tasks:Running task pcp... 2024-04-14T08:25:08.401 INFO:teuthology.run_tasks:Running task selinux... 2024-04-14T08:25:08.410 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-14T08:25:08.410 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2024-04-14T08:25:08.473 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2024-04-14T08:25:08.474 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2024-04-14T08:25:08.532 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2024-04-14T08:25:08.534 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-14T08:25:08.534 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2024-04-14T08:25:08.562 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2024-04-14T08:25:08.563 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2024-04-14T08:25:08.595 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2024-04-14T08:25:08.595 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi172.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi177.front.sepia.ceph.com': 'permissive'} 2024-04-14T08:25:08.595 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-14T08:25:08.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:25:08.633 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-14T08:25:08.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:25:08.663 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-14T08:25:08.663 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2024-04-14T08:25:08.706 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2024-04-14T08:25:08.739 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-14T08:25:08.749 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-14T08:25:08.765 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-14T08:25:08.767 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi172.front.sepia.ceph.com,smithi177.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-14T08:29:56.416 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi172.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com')] 2024-04-14T08:29:56.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-14T08:29:56.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T08:29:56.509 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-04-14T08:29:56.595 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-14T08:29:56.595 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-04-14T08:29:56.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T08:29:56.684 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-04-14T08:29:56.766 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-04-14T08:29:56.767 INFO:teuthology.run_tasks:Running task clock... 2024-04-14T08:29:56.779 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-14T08:29:56.779 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-14T08:29:56.780 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-14T08:29:56.782 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-14T08:29:56.782 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-14T08:29:56.814 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-14T08:29:56.831 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-14T08:29:56.837 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-14T08:29:56.853 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-14T08:29:56.861 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2024-04-14T08:29:56.875 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2024-04-14T08:29:56.882 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2024-04-14T08:29:56.893 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-14T08:29:56.896 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2024-04-14T08:29:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-14T08:29:56.912 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-14T08:29:56.929 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-14T08:29:56.963 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-04-14T08:29:56.966 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-14T08:29:56.967 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-04-14T08:29:56.967 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T08:29:56.967 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T08:29:56.967 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T08:29:56.967 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T08:29:56.978 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-04-14T08:29:56.982 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-14T08:29:56.982 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-04-14T08:29:56.982 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T08:29:56.982 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T08:29:56.982 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T08:29:56.982 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T08:29:56.982 INFO:teuthology.run_tasks:Running task install... 2024-04-14T08:29:56.991 DEBUG:teuthology.task.install:project ceph 2024-04-14T08:29:56.991 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd7e176b41636f150fe82aaedea82def3e9af0203'}} 2024-04-14T08:29:56.991 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd7e176b41636f150fe82aaedea82def3e9af0203'} 2024-04-14T08:29:56.991 INFO:teuthology.task.install:Using flavor: default 2024-04-14T08:29:56.996 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-14T08:29:56.997 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-14T08:29:56.997 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd7e176b41636f150fe82aaedea82def3e9af0203', 'tag': None, 'wait_for_package': False} 2024-04-14T08:29:56.998 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d7e176b41636f150fe82aaedea82def3e9af0203 2024-04-14T08:29:57.000 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd7e176b41636f150fe82aaedea82def3e9af0203', 'tag': None, 'wait_for_package': False} 2024-04-14T08:29:57.000 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d7e176b41636f150fe82aaedea82def3e9af0203 2024-04-14T08:29:57.150 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/d7e176b41636f150fe82aaedea82def3e9af0203/centos/9/flavors/default/ 2024-04-14T08:29:57.150 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2592.gd7e176b4 2024-04-14T08:29:57.223 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/d7e176b41636f150fe82aaedea82def3e9af0203/centos/9/flavors/default/ 2024-04-14T08:29:57.223 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2592.gd7e176b4 2024-04-14T08:29:57.477 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/d7e176b41636f150fe82aaedea82def3e9af0203/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/d7e176b41636f150fe82aaedea82def3e9af0203/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/d7e176b41636f150fe82aaedea82def3e9af0203/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-14T08:29:57.477 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:29:57.478 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-14T08:29:57.507 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-14T08:29:57.508 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d7e176b41636f150fe82aaedea82def3e9af0203/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-14T08:29:57.542 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/d7e176b41636f150fe82aaedea82def3e9af0203/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/d7e176b41636f150fe82aaedea82def3e9af0203/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/d7e176b41636f150fe82aaedea82def3e9af0203/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-14T08:29:57.542 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:29:57.542 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-14T08:29:57.572 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-14T08:29:57.573 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d7e176b41636f150fe82aaedea82def3e9af0203/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-14T08:29:57.576 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-14T08:29:57.640 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-14T08:29:57.657 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-14T08:29:57.722 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-14T08:29:57.724 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2024-04-14T08:29:57.725 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2024-04-14T08:29:57.788 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2024-04-14T08:29:57.790 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2024-04-14T08:29:57.982 INFO:teuthology.orchestra.run.smithi172.stdout:74 files removed 2024-04-14T08:29:58.010 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-14T08:29:58.029 INFO:teuthology.orchestra.run.smithi177.stdout:74 files removed 2024-04-14T08:29:58.056 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-14T08:29:58.909 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 151 kB/s | 83 kB 00:00 2024-04-14T08:29:58.953 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 129 kB/s | 83 kB 00:00 2024-04-14T08:29:59.401 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 26 kB/s | 11 kB 00:00 2024-04-14T08:29:59.712 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 16 kB/s | 11 kB 00:00 2024-04-14T08:30:00.252 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 2.3 kB/s | 1.9 kB 00:00 2024-04-14T08:30:00.421 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-04-14T08:30:01.029 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - AppStream 11 MB/s | 8.1 MB 00:00 2024-04-14T08:30:01.282 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS-9 - AppStream 9.9 MB/s | 8.1 MB 00:00 2024-04-14T08:30:02.654 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS-9 - BaseOS 8.3 MB/s | 2.5 MB 00:00 2024-04-14T08:30:02.941 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS-9 - BaseOS 8.5 MB/s | 2.5 MB 00:00 2024-04-14T08:30:03.545 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-04-14T08:30:03.915 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-14T08:30:06.674 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 19 MB 00:02 2024-04-14T08:30:10.764 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-14T08:30:12.140 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 19 MB 00:07 2024-04-14T08:30:12.564 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-14T08:30:14.718 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-14T08:30:16.807 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 6.2 MB 00:01 2024-04-14T08:30:18.600 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-14T08:30:19.857 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-14T08:30:20.637 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-14T08:30:22.450 INFO:teuthology.orchestra.run.smithi177.stderr:Modular dependency problems: 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 1: conflicting requests 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 2: conflicting requests 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 3: conflicting requests 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 4: conflicting requests 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 5: conflicting requests 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 6: conflicting requests 2024-04-14T08:30:22.451 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 7: conflicting requests 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 8: conflicting requests 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 9: conflicting requests 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 10: conflicting requests 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 11: conflicting requests 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 12: conflicting requests 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.452 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 13: conflicting requests 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 14: conflicting requests 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 15: conflicting requests 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 16: conflicting requests 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 17: conflicting requests 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 18: conflicting requests 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 19: conflicting requests 2024-04-14T08:30:22.453 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.454 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 20: conflicting requests 2024-04-14T08:30:22.454 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.454 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 21: conflicting requests 2024-04-14T08:30:22.454 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.454 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 22: conflicting requests 2024-04-14T08:30:22.454 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.454 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 23: conflicting requests 2024-04-14T08:30:22.454 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.454 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 24: conflicting requests 2024-04-14T08:30:22.454 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.454 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 25: conflicting requests 2024-04-14T08:30:22.454 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 26: conflicting requests 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 27: conflicting requests 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 28: conflicting requests 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 29: conflicting requests 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 30: conflicting requests 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 31: conflicting requests 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 32: conflicting requests 2024-04-14T08:30:22.455 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 33: conflicting requests 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 34: conflicting requests 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 35: conflicting requests 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 36: conflicting requests 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 37: conflicting requests 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T08:30:22.456 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 39: conflicting requests 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.457 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T08:30:22.458 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.459 INFO:teuthology.orchestra.run.smithi177.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T08:30:22.905 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-14T08:30:22.906 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-14T08:30:22.919 INFO:teuthology.orchestra.run.smithi177.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-14T08:30:22.920 INFO:teuthology.orchestra.run.smithi177.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-14T08:30:22.921 INFO:teuthology.orchestra.run.smithi177.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-14T08:30:22.922 INFO:teuthology.orchestra.run.smithi177.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-14T08:30:22.924 INFO:teuthology.orchestra.run.smithi177.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-14T08:30:22.927 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-14T08:30:22.928 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-14T08:30:23.019 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout:========================================================================================= 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout:========================================================================================= 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 6.6 k 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 5.4 M 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 890 k 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 145 k 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 1.5 M 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 141 k 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 3.5 M 2024-04-14T08:30:23.039 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 7.4 M 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 49 k 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 9.8 M 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 48 M 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 264 k 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 762 k 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 225 k 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-shell x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 27 k 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 18 k 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 32 k 2024-04-14T08:30:23.040 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 746 k 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 127 k 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 162 k 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 321 k 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 298 k 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 99 k 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 85 k 2024-04-14T08:30:23.041 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 3.1 M 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 171 k 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager x86_64 1:1.47.4-1.el9 baseos 2.3 M 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 baseos 1.8 M 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 baseos 42 k 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 baseos 249 k 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-14T08:30:23.042 INFO:teuthology.orchestra.run.smithi177.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 3.4 M 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 3.1 M 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-14T08:30:23.043 INFO:teuthology.orchestra.run.smithi177.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 21 M 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 34 k 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 2.4 M 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 249 k 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 4.7 M 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 17 M 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 16 k 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 25 k 2024-04-14T08:30:23.044 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 163 k 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-14T08:30:23.045 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 501 k 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 5.1 M 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-14T08:30:23.046 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 45 k 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 134 k 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-14T08:30:23.047 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-14T08:30:23.048 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-14T08:30:23.049 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-14T08:30:23.050 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-14T08:30:23.051 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout:Installing weak dependencies: 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout:========================================================================================= 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout:Install 150 Packages 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 24 Packages 2024-04-14T08:30:23.052 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:30:23.053 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 228 M 2024-04-14T08:30:23.053 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-04-14T08:30:24.042 INFO:teuthology.orchestra.run.smithi177.stdout:(1/174): ceph-19.0.0-2592.gd7e176b4.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-04-14T08:30:24.234 INFO:teuthology.orchestra.run.smithi177.stdout:(2/174): ceph-fuse-19.0.0-2592.gd7e176b4.el9.x8 4.5 MB/s | 890 kB 00:00 2024-04-14T08:30:24.317 INFO:teuthology.orchestra.run.smithi177.stdout:(3/174): ceph-immutable-object-cache-19.0.0-259 1.7 MB/s | 145 kB 00:00 2024-04-14T08:30:24.976 INFO:teuthology.orchestra.run.smithi177.stdout:(4/174): ceph-mds-19.0.0-2592.gd7e176b4.el9.x86 3.6 MB/s | 2.4 MB 00:00 2024-04-14T08:30:25.401 INFO:teuthology.orchestra.run.smithi177.stdout:(5/174): ceph-common-19.0.0-2592.gd7e176b4.el9. 13 MB/s | 21 MB 00:01 2024-04-14T08:30:25.493 INFO:teuthology.orchestra.run.smithi177.stdout:(6/174): ceph-base-19.0.0-2592.gd7e176b4.el9.x8 3.3 MB/s | 5.4 MB 00:01 2024-04-14T08:30:25.537 INFO:teuthology.orchestra.run.smithi177.stdout:(7/174): ceph-mgr-19.0.0-2592.gd7e176b4.el9.x86 2.7 MB/s | 1.5 MB 00:00 2024-04-14T08:30:25.696 INFO:teuthology.orchestra.run.smithi177.stdout:(8/174): ceph-mon-19.0.0-2592.gd7e176b4.el9.x86 16 MB/s | 4.7 MB 00:00 2024-04-14T08:30:25.763 INFO:teuthology.orchestra.run.smithi177.stdout:(9/174): ceph-selinux-19.0.0-2592.gd7e176b4.el9 379 kB/s | 25 kB 00:00 2024-04-14T08:30:25.818 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-14T08:30:27.329 INFO:teuthology.orchestra.run.smithi177.stdout:(10/174): ceph-radosgw-19.0.0-2592.gd7e176b4.el 5.5 MB/s | 9.8 MB 00:01 2024-04-14T08:30:27.771 INFO:teuthology.orchestra.run.smithi177.stdout:(11/174): ceph-osd-19.0.0-2592.gd7e176b4.el9.x8 7.4 MB/s | 17 MB 00:02 2024-04-14T08:30:27.796 INFO:teuthology.orchestra.run.smithi177.stdout:(12/174): cephfs-mirror-19.0.0-2592.gd7e176b4.e 482 kB/s | 225 kB 00:00 2024-04-14T08:30:27.830 INFO:teuthology.orchestra.run.smithi177.stdout:(13/174): cephfs-shell-19.0.0-2592.gd7e176b4.el 464 kB/s | 27 kB 00:00 2024-04-14T08:30:27.847 INFO:teuthology.orchestra.run.smithi177.stdout:(14/174): libcephfs-devel-19.0.0-2592.gd7e176b4 633 kB/s | 32 kB 00:00 2024-04-14T08:30:27.930 INFO:teuthology.orchestra.run.smithi177.stdout:(15/174): libcephsqlite-19.0.0-2592.gd7e176b4.e 1.9 MB/s | 163 kB 00:00 2024-04-14T08:30:27.955 INFO:teuthology.orchestra.run.smithi177.stdout:(16/174): libcephfs2-19.0.0-2592.gd7e176b4.el9. 5.8 MB/s | 746 kB 00:00 2024-04-14T08:30:27.989 INFO:teuthology.orchestra.run.smithi177.stdout:(17/174): librados-devel-19.0.0-2592.gd7e176b4. 2.1 MB/s | 127 kB 00:00 2024-04-14T08:30:28.040 INFO:teuthology.orchestra.run.smithi177.stdout:(18/174): libradosstriper1-19.0.0-2592.gd7e176b 5.8 MB/s | 501 kB 00:00 2024-04-14T08:30:28.107 INFO:teuthology.orchestra.run.smithi177.stdout:(19/174): python3-ceph-argparse-19.0.0-2592.gd7 678 kB/s | 45 kB 00:00 2024-04-14T08:30:28.182 INFO:teuthology.orchestra.run.smithi177.stdout:(20/174): python3-ceph-common-19.0.0-2592.gd7e1 1.8 MB/s | 134 kB 00:00 2024-04-14T08:30:28.265 INFO:teuthology.orchestra.run.smithi177.stdout:(21/174): python3-cephfs-19.0.0-2592.gd7e176b4. 1.9 MB/s | 162 kB 00:00 2024-04-14T08:30:28.349 INFO:teuthology.orchestra.run.smithi177.stdout:(22/174): python3-rados-19.0.0-2592.gd7e176b4.e 3.8 MB/s | 321 kB 00:00 2024-04-14T08:30:28.402 INFO:teuthology.orchestra.run.smithi172.stderr:Modular dependency problems: 2024-04-14T08:30:28.402 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:30:28.402 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 1: conflicting requests 2024-04-14T08:30:28.402 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.402 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 2: conflicting requests 2024-04-14T08:30:28.402 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.402 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 3: conflicting requests 2024-04-14T08:30:28.402 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.402 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 4: conflicting requests 2024-04-14T08:30:28.402 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.403 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 5: conflicting requests 2024-04-14T08:30:28.403 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.403 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 6: conflicting requests 2024-04-14T08:30:28.403 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.403 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 7: conflicting requests 2024-04-14T08:30:28.403 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.403 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 8: conflicting requests 2024-04-14T08:30:28.403 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.403 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 9: conflicting requests 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 10: conflicting requests 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 11: conflicting requests 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 12: conflicting requests 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 13: conflicting requests 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 14: conflicting requests 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 15: conflicting requests 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.404 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 16: conflicting requests 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 17: conflicting requests 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 18: conflicting requests 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 19: conflicting requests 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 20: conflicting requests 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 21: conflicting requests 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 22: conflicting requests 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.405 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 23: conflicting requests 2024-04-14T08:30:28.406 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.406 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 24: conflicting requests 2024-04-14T08:30:28.406 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.406 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 25: conflicting requests 2024-04-14T08:30:28.406 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.406 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 26: conflicting requests 2024-04-14T08:30:28.406 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.406 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 27: conflicting requests 2024-04-14T08:30:28.406 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.406 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 28: conflicting requests 2024-04-14T08:30:28.406 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.406 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 29: conflicting requests 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 30: conflicting requests 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 31: conflicting requests 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 32: conflicting requests 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 33: conflicting requests 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 34: conflicting requests 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 35: conflicting requests 2024-04-14T08:30:28.407 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.408 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 36: conflicting requests 2024-04-14T08:30:28.408 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.408 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 37: conflicting requests 2024-04-14T08:30:28.408 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.408 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.408 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.408 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T08:30:28.408 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.408 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.408 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.408 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.408 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 39: conflicting requests 2024-04-14T08:30:28.409 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.409 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.409 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.410 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.410 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.410 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.410 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T08:30:28.410 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.410 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.410 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.410 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.410 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T08:30:28.410 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.411 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.412 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T08:30:28.412 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.412 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.412 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.412 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.412 INFO:teuthology.orchestra.run.smithi172.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T08:30:28.412 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.412 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.412 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.412 INFO:teuthology.orchestra.run.smithi172.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T08:30:28.432 INFO:teuthology.orchestra.run.smithi177.stdout:(23/174): python3-rbd-19.0.0-2592.gd7e176b4.el9 3.5 MB/s | 298 kB 00:00 2024-04-14T08:30:28.508 INFO:teuthology.orchestra.run.smithi177.stdout:(24/174): python3-rgw-19.0.0-2592.gd7e176b4.el9 1.3 MB/s | 99 kB 00:00 2024-04-14T08:30:28.583 INFO:teuthology.orchestra.run.smithi177.stdout:(25/174): rbd-fuse-19.0.0-2592.gd7e176b4.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-14T08:30:28.864 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-14T08:30:28.865 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-14T08:30:28.878 INFO:teuthology.orchestra.run.smithi172.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-14T08:30:28.879 INFO:teuthology.orchestra.run.smithi172.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-14T08:30:28.880 INFO:teuthology.orchestra.run.smithi172.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-14T08:30:28.881 INFO:teuthology.orchestra.run.smithi172.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-14T08:30:28.883 INFO:teuthology.orchestra.run.smithi172.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-14T08:30:28.885 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-14T08:30:28.886 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-14T08:30:28.925 INFO:teuthology.orchestra.run.smithi177.stdout:(26/174): rbd-mirror-19.0.0-2592.gd7e176b4.el9. 9.0 MB/s | 3.1 MB 00:00 2024-04-14T08:30:28.979 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-04-14T08:30:29.002 INFO:teuthology.orchestra.run.smithi172.stdout:========================================================================================= 2024-04-14T08:30:29.002 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-04-14T08:30:29.002 INFO:teuthology.orchestra.run.smithi172.stdout:========================================================================================= 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 6.6 k 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 5.4 M 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 890 k 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 145 k 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 1.5 M 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 141 k 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 3.5 M 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 7.4 M 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 49 k 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 9.8 M 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 48 M 2024-04-14T08:30:29.003 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 264 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 762 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 225 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 27 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 18 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 32 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 746 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 127 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-14T08:30:29.004 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 162 k 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 321 k 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 298 k 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 99 k 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 85 k 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 3.1 M 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 171 k 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager x86_64 1:1.47.4-1.el9 baseos 2.3 M 2024-04-14T08:30:29.005 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 baseos 1.8 M 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 baseos 42 k 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 baseos 249 k 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 3.4 M 2024-04-14T08:30:29.006 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 3.1 M 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-14T08:30:29.007 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 21 M 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 34 k 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 2.4 M 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 249 k 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 4.7 M 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 17 M 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2592.gd7e176b4.el9 ceph-noarch 16 k 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 25 k 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-14T08:30:29.008 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 163 k 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 501 k 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 5.1 M 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-14T08:30:29.009 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-14T08:30:29.010 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-14T08:30:29.010 INFO:teuthology.orchestra.run.smithi172.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-14T08:30:29.010 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-14T08:30:29.010 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-14T08:30:29.010 INFO:teuthology.orchestra.run.smithi172.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-14T08:30:29.010 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-14T08:30:29.010 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-14T08:30:29.010 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-14T08:30:29.010 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-14T08:30:29.010 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-14T08:30:29.010 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-14T08:30:29.010 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 45 k 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:19.0.0-2592.gd7e176b4.el9 ceph 134 k 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-14T08:30:29.011 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-14T08:30:29.012 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-14T08:30:29.013 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-14T08:30:29.014 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-14T08:30:29.015 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-04-14T08:30:29.016 INFO:teuthology.orchestra.run.smithi172.stdout:Installing weak dependencies: 2024-04-14T08:30:29.016 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-14T08:30:29.016 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:30:29.016 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-04-14T08:30:29.016 INFO:teuthology.orchestra.run.smithi172.stdout:========================================================================================= 2024-04-14T08:30:29.016 INFO:teuthology.orchestra.run.smithi172.stdout:Install 150 Packages 2024-04-14T08:30:29.016 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 24 Packages 2024-04-14T08:30:29.016 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:30:29.017 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 228 M 2024-04-14T08:30:29.017 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-04-14T08:30:29.359 INFO:teuthology.orchestra.run.smithi177.stdout:(27/174): ceph-test-19.0.0-2592.gd7e176b4.el9.x 13 MB/s | 48 MB 00:03 2024-04-14T08:30:29.384 INFO:teuthology.orchestra.run.smithi177.stdout:(28/174): rbd-nbd-19.0.0-2592.gd7e176b4.el9.x86 372 kB/s | 171 kB 00:00 2024-04-14T08:30:29.401 INFO:teuthology.orchestra.run.smithi177.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2592.g 820 kB/s | 34 kB 00:00 2024-04-14T08:30:29.434 INFO:teuthology.orchestra.run.smithi177.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2592.gd7e176b 2.8 MB/s | 141 kB 00:00 2024-04-14T08:30:29.626 INFO:teuthology.orchestra.run.smithi177.stdout:(31/174): librgw2-19.0.0-2592.gd7e176b4.el9.x86 3.1 MB/s | 5.1 MB 00:01 2024-04-14T08:30:29.710 INFO:teuthology.orchestra.run.smithi177.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2592.gd7e17 11 MB/s | 3.5 MB 00:00 2024-04-14T08:30:29.752 INFO:teuthology.orchestra.run.smithi177.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2592.gd7 1.9 MB/s | 249 kB 00:00 2024-04-14T08:30:29.777 INFO:teuthology.orchestra.run.smithi177.stdout:(34/174): ceph-mgr-rook-19.0.0-2592.gd7e176b4.e 740 kB/s | 49 kB 00:00 2024-04-14T08:30:29.802 INFO:teuthology.orchestra.run.smithi177.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2592.gd 328 kB/s | 16 kB 00:00 2024-04-14T08:30:29.827 INFO:teuthology.orchestra.run.smithi172.stdout:(1/174): ceph-19.0.0-2592.gd7e176b4.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-04-14T08:30:29.836 INFO:teuthology.orchestra.run.smithi177.stdout:(36/174): ceph-volume-19.0.0-2592.gd7e176b4.el9 4.4 MB/s | 264 kB 00:00 2024-04-14T08:30:29.894 INFO:teuthology.orchestra.run.smithi177.stdout:(37/174): cephfs-top-19.0.0-2592.gd7e176b4.el9. 313 kB/s | 18 kB 00:00 2024-04-14T08:30:29.986 INFO:teuthology.orchestra.run.smithi177.stdout:(38/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 192 kB/s | 18 kB 00:00 2024-04-14T08:30:30.020 INFO:teuthology.orchestra.run.smithi177.stdout:(39/174): cephadm-19.0.0-2592.gd7e176b4.el9.noa 3.4 MB/s | 762 kB 00:00 2024-04-14T08:30:30.115 INFO:teuthology.orchestra.run.smithi177.stdout:(40/174): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-14T08:30:30.161 INFO:teuthology.orchestra.run.smithi172.stdout:(2/174): ceph-fuse-19.0.0-2592.gd7e176b4.el9.x8 2.6 MB/s | 890 kB 00:00 2024-04-14T08:30:30.196 INFO:teuthology.orchestra.run.smithi177.stdout:(41/174): fuse-2.9.9-15.el9.x86_64.rpm 384 kB/s | 80 kB 00:00 2024-04-14T08:30:30.221 INFO:teuthology.orchestra.run.smithi177.stdout:(42/174): jq-1.6-16.el9.x86_64.rpm 934 kB/s | 187 kB 00:00 2024-04-14T08:30:30.236 INFO:teuthology.orchestra.run.smithi172.stdout:(3/174): ceph-immutable-object-cache-19.0.0-259 1.9 MB/s | 145 kB 00:00 2024-04-14T08:30:30.263 INFO:teuthology.orchestra.run.smithi177.stdout:(43/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-14T08:30:30.305 INFO:teuthology.orchestra.run.smithi177.stdout:(44/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.5 MB/s | 218 kB 00:00 2024-04-14T08:30:30.338 INFO:teuthology.orchestra.run.smithi177.stdout:(45/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-04-14T08:30:30.364 INFO:teuthology.orchestra.run.smithi177.stdout:(46/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-04-14T08:30:30.389 INFO:teuthology.orchestra.run.smithi177.stdout:(47/174): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-14T08:30:30.414 INFO:teuthology.orchestra.run.smithi177.stdout:(48/174): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-14T08:30:30.439 INFO:teuthology.orchestra.run.smithi177.stdout:(49/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-14T08:30:30.465 INFO:teuthology.orchestra.run.smithi177.stdout:(50/174): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-14T08:30:30.478 INFO:teuthology.orchestra.run.smithi172.stdout:(4/174): ceph-base-19.0.0-2592.gd7e176b4.el9.x8 6.4 MB/s | 5.4 MB 00:00 2024-04-14T08:30:30.498 INFO:teuthology.orchestra.run.smithi177.stdout:(51/174): python3-cryptography-36.0.1-4.el9.x86 6.4 MB/s | 1.2 MB 00:00 2024-04-14T08:30:30.615 INFO:teuthology.orchestra.run.smithi177.stdout:(52/174): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-04-14T08:30:30.628 INFO:teuthology.orchestra.run.smithi172.stdout:(5/174): ceph-mgr-19.0.0-2592.gd7e176b4.el9.x86 10 MB/s | 1.5 MB 00:00 2024-04-14T08:30:30.640 INFO:teuthology.orchestra.run.smithi177.stdout:(53/174): boost-program-options-1.75.0-8.el9.x8 755 kB/s | 107 kB 00:00 2024-04-14T08:30:30.657 INFO:teuthology.orchestra.run.smithi177.stdout:(54/174): elfutils-libelf-devel-0.190-2.el9.x86 516 kB/s | 21 kB 00:00 2024-04-14T08:30:30.670 INFO:teuthology.orchestra.run.smithi172.stdout:(6/174): ceph-mds-19.0.0-2592.gd7e176b4.el9.x86 5.4 MB/s | 2.4 MB 00:00 2024-04-14T08:30:30.691 INFO:teuthology.orchestra.run.smithi177.stdout:(55/174): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-04-14T08:30:30.708 INFO:teuthology.orchestra.run.smithi177.stdout:(56/174): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-04-14T08:30:30.750 INFO:teuthology.orchestra.run.smithi177.stdout:(57/174): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-14T08:30:30.767 INFO:teuthology.orchestra.run.smithi177.stdout:(58/174): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-04-14T08:30:30.792 INFO:teuthology.orchestra.run.smithi177.stdout:(59/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-14T08:30:30.809 INFO:teuthology.orchestra.run.smithi177.stdout:(60/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-14T08:30:30.876 INFO:teuthology.orchestra.run.smithi177.stdout:(61/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-04-14T08:30:30.901 INFO:teuthology.orchestra.run.smithi177.stdout:(62/174): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-04-14T08:30:30.918 INFO:teuthology.orchestra.run.smithi177.stdout:(63/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 613 kB/s | 25 kB 00:00 2024-04-14T08:30:30.935 INFO:teuthology.orchestra.run.smithi177.stdout:(64/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-14T08:30:30.970 INFO:teuthology.orchestra.run.smithi172.stdout:(7/174): ceph-mon-19.0.0-2592.gd7e176b4.el9.x86 14 MB/s | 4.7 MB 00:00 2024-04-14T08:30:30.977 INFO:teuthology.orchestra.run.smithi177.stdout:(65/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-14T08:30:31.002 INFO:teuthology.orchestra.run.smithi177.stdout:(66/174): libtool-2.4.6-45.el9.x86_64.rpm 3.0 MB/s | 582 kB 00:00 2024-04-14T08:30:31.027 INFO:teuthology.orchestra.run.smithi177.stdout:(67/174): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-14T08:30:31.044 INFO:teuthology.orchestra.run.smithi177.stdout:(68/174): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-04-14T08:30:31.111 INFO:teuthology.orchestra.run.smithi177.stdout:(69/174): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-04-14T08:30:31.178 INFO:teuthology.orchestra.run.smithi177.stdout:(70/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-14T08:30:31.246 INFO:teuthology.orchestra.run.smithi177.stdout:(71/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 19 MB/s | 4.1 MB 00:00 2024-04-14T08:30:31.271 INFO:teuthology.orchestra.run.smithi177.stdout:(72/174): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-04-14T08:30:31.296 INFO:teuthology.orchestra.run.smithi177.stdout:(73/174): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-14T08:30:31.338 INFO:teuthology.orchestra.run.smithi177.stdout:(74/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-14T08:30:31.363 INFO:teuthology.orchestra.run.smithi177.stdout:(75/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-14T08:30:31.405 INFO:teuthology.orchestra.run.smithi177.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 5.8 MB/s | 245 kB 00:00 2024-04-14T08:30:31.503 INFO:teuthology.orchestra.run.smithi177.stdout:(77/174): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-04-14T08:30:31.530 INFO:teuthology.orchestra.run.smithi177.stdout:(78/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-04-14T08:30:31.556 INFO:teuthology.orchestra.run.smithi177.stdout:(79/174): python3-jsonpatch-1.21-16.el9.noarch. 492 kB/s | 26 kB 00:00 2024-04-14T08:30:31.581 INFO:teuthology.orchestra.run.smithi177.stdout:(80/174): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-14T08:30:31.606 INFO:teuthology.orchestra.run.smithi177.stdout:(81/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-14T08:30:31.623 INFO:teuthology.orchestra.run.smithi177.stdout:(82/174): python3-markupsafe-1.1.1-12.el9.x86_6 832 kB/s | 35 kB 00:00 2024-04-14T08:30:31.670 INFO:teuthology.orchestra.run.smithi172.stdout:(8/174): ceph-radosgw-19.0.0-2592.gd7e176b4.el9 14 MB/s | 9.8 MB 00:00 2024-04-14T08:30:31.698 INFO:teuthology.orchestra.run.smithi177.stdout:(83/174): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-14T08:30:31.723 INFO:teuthology.orchestra.run.smithi177.stdout:(84/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-14T08:30:31.749 INFO:teuthology.orchestra.run.smithi177.stdout:(85/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-14T08:30:31.754 INFO:teuthology.orchestra.run.smithi172.stdout:(9/174): ceph-selinux-19.0.0-2592.gd7e176b4.el9 303 kB/s | 25 kB 00:00 2024-04-14T08:30:31.774 INFO:teuthology.orchestra.run.smithi177.stdout:(86/174): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-14T08:30:31.791 INFO:teuthology.orchestra.run.smithi177.stdout:(87/174): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-14T08:30:31.816 INFO:teuthology.orchestra.run.smithi177.stdout:(88/174): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-04-14T08:30:31.842 INFO:teuthology.orchestra.run.smithi177.stdout:(89/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-14T08:30:31.867 INFO:teuthology.orchestra.run.smithi177.stdout:(90/174): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-14T08:30:31.884 INFO:teuthology.orchestra.run.smithi177.stdout:(91/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-14T08:30:31.901 INFO:teuthology.orchestra.run.smithi177.stdout:(92/174): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-04-14T08:30:31.984 INFO:teuthology.orchestra.run.smithi177.stdout:(93/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-14T08:30:32.001 INFO:teuthology.orchestra.run.smithi177.stdout:(94/174): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-04-14T08:30:32.018 INFO:teuthology.orchestra.run.smithi177.stdout:(95/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-04-14T08:30:32.044 INFO:teuthology.orchestra.run.smithi177.stdout:(96/174): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-14T08:30:32.060 INFO:teuthology.orchestra.run.smithi177.stdout:(97/174): qatlib-service-24.02.0-1.el9.x86_64.r 859 kB/s | 36 kB 00:00 2024-04-14T08:30:32.077 INFO:teuthology.orchestra.run.smithi177.stdout:(98/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-14T08:30:32.103 INFO:teuthology.orchestra.run.smithi177.stdout:(99/174): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 305 kB 00:00 2024-04-14T08:30:32.119 INFO:teuthology.orchestra.run.smithi177.stdout:(100/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-04-14T08:30:32.145 INFO:teuthology.orchestra.run.smithi177.stdout:(101/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-04-14T08:30:32.170 INFO:teuthology.orchestra.run.smithi177.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-14T08:30:32.187 INFO:teuthology.orchestra.run.smithi177.stdout:(103/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-04-14T08:30:32.404 INFO:teuthology.orchestra.run.smithi177.stdout:(104/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-04-14T08:30:32.421 INFO:teuthology.orchestra.run.smithi177.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-04-14T08:30:32.546 INFO:teuthology.orchestra.run.smithi177.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-04-14T08:30:32.596 INFO:teuthology.orchestra.run.smithi172.stdout:(10/174): ceph-osd-19.0.0-2592.gd7e176b4.el9.x8 8.7 MB/s | 17 MB 00:01 2024-04-14T08:30:32.605 INFO:teuthology.orchestra.run.smithi177.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 336 kB/s | 19 kB 00:00 2024-04-14T08:30:32.639 INFO:teuthology.orchestra.run.smithi177.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.4 MB/s | 309 kB 00:00 2024-04-14T08:30:32.654 INFO:teuthology.orchestra.run.smithi172.stdout:(11/174): cephfs-mirror-19.0.0-2592.gd7e176b4.e 3.8 MB/s | 225 kB 00:00 2024-04-14T08:30:32.672 INFO:teuthology.orchestra.run.smithi177.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-04-14T08:30:32.689 INFO:teuthology.orchestra.run.smithi177.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-04-14T08:30:32.706 INFO:teuthology.orchestra.run.smithi177.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-04-14T08:30:32.713 INFO:teuthology.orchestra.run.smithi172.stdout:(12/174): cephfs-shell-19.0.0-2592.gd7e176b4.el 464 kB/s | 27 kB 00:00 2024-04-14T08:30:32.731 INFO:teuthology.orchestra.run.smithi177.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-14T08:30:32.773 INFO:teuthology.orchestra.run.smithi177.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-14T08:30:32.780 INFO:teuthology.orchestra.run.smithi172.stdout:(13/174): libcephfs-devel-19.0.0-2592.gd7e176b4 476 kB/s | 32 kB 00:00 2024-04-14T08:30:32.898 INFO:teuthology.orchestra.run.smithi177.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-14T08:30:32.930 INFO:teuthology.orchestra.run.smithi172.stdout:(14/174): libcephfs2-19.0.0-2592.gd7e176b4.el9. 4.9 MB/s | 746 kB 00:00 2024-04-14T08:30:32.940 INFO:teuthology.orchestra.run.smithi177.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-04-14T08:30:32.982 INFO:teuthology.orchestra.run.smithi177.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 776 kB/s | 32 kB 00:00 2024-04-14T08:30:32.997 INFO:teuthology.orchestra.run.smithi172.stdout:(15/174): libcephsqlite-19.0.0-2592.gd7e176b4.e 2.4 MB/s | 163 kB 00:00 2024-04-14T08:30:33.016 INFO:teuthology.orchestra.run.smithi177.stdout:(117/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 838 kB 00:00 2024-04-14T08:30:33.041 INFO:teuthology.orchestra.run.smithi177.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-14T08:30:33.064 INFO:teuthology.orchestra.run.smithi172.stdout:(16/174): librados-devel-19.0.0-2592.gd7e176b4. 1.9 MB/s | 127 kB 00:00 2024-04-14T08:30:33.083 INFO:teuthology.orchestra.run.smithi177.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-14T08:30:33.150 INFO:teuthology.orchestra.run.smithi177.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-14T08:30:33.156 INFO:teuthology.orchestra.run.smithi172.stdout:(17/174): libradosstriper1-19.0.0-2592.gd7e176b 5.3 MB/s | 501 kB 00:00 2024-04-14T08:30:33.184 INFO:teuthology.orchestra.run.smithi177.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-14T08:30:33.334 INFO:teuthology.orchestra.run.smithi177.stdout:(122/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-04-14T08:30:33.742 INFO:teuthology.orchestra.run.smithi177.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 60 kB/s | 35 kB 00:00 2024-04-14T08:30:33.815 INFO:teuthology.orchestra.run.smithi172.stdout:(18/174): librgw2-19.0.0-2592.gd7e176b4.el9.x86 7.8 MB/s | 5.1 MB 00:00 2024-04-14T08:30:33.882 INFO:teuthology.orchestra.run.smithi172.stdout:(19/174): python3-ceph-argparse-19.0.0-2592.gd7 680 kB/s | 45 kB 00:00 2024-04-14T08:30:33.918 INFO:teuthology.orchestra.run.smithi177.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 276 kB/s | 203 kB 00:00 2024-04-14T08:30:33.949 INFO:teuthology.orchestra.run.smithi172.stdout:(20/174): python3-ceph-common-19.0.0-2592.gd7e1 2.0 MB/s | 134 kB 00:00 2024-04-14T08:30:34.032 INFO:teuthology.orchestra.run.smithi172.stdout:(21/174): python3-cephfs-19.0.0-2592.gd7e176b4. 1.9 MB/s | 162 kB 00:00 2024-04-14T08:30:34.068 INFO:teuthology.orchestra.run.smithi177.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.8 MB/s | 2.0 MB 00:00 2024-04-14T08:30:34.110 INFO:teuthology.orchestra.run.smithi177.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 425 kB/s | 18 kB 00:00 2024-04-14T08:30:34.116 INFO:teuthology.orchestra.run.smithi172.stdout:(22/174): python3-rados-19.0.0-2592.gd7e176b4.e 3.8 MB/s | 321 kB 00:00 2024-04-14T08:30:34.127 INFO:teuthology.orchestra.run.smithi177.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 51 kB/s | 11 kB 00:00 2024-04-14T08:30:34.160 INFO:teuthology.orchestra.run.smithi177.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 585 kB/s | 19 kB 00:00 2024-04-14T08:30:34.185 INFO:teuthology.orchestra.run.smithi177.stdout:(129/174): python3-grpcio-tools-1.46.7-10.el9.x 326 kB/s | 144 kB 00:00 2024-04-14T08:30:34.199 INFO:teuthology.orchestra.run.smithi172.stdout:(23/174): python3-rbd-19.0.0-2592.gd7e176b4.el9 3.5 MB/s | 298 kB 00:00 2024-04-14T08:30:34.202 INFO:teuthology.orchestra.run.smithi177.stdout:(130/174): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-04-14T08:30:34.219 INFO:teuthology.orchestra.run.smithi177.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-04-14T08:30:34.244 INFO:teuthology.orchestra.run.smithi177.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-14T08:30:34.270 INFO:teuthology.orchestra.run.smithi177.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-14T08:30:34.312 INFO:teuthology.orchestra.run.smithi177.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-14T08:30:34.329 INFO:teuthology.orchestra.run.smithi177.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 687 kB/s | 58 kB 00:00 2024-04-14T08:30:34.346 INFO:teuthology.orchestra.run.smithi177.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 494 kB/s | 16 kB 00:00 2024-04-14T08:30:34.371 INFO:teuthology.orchestra.run.smithi177.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-14T08:30:34.388 INFO:teuthology.orchestra.run.smithi177.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 675 kB/s | 28 kB 00:00 2024-04-14T08:30:34.413 INFO:teuthology.orchestra.run.smithi177.stdout:(139/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-14T08:30:34.430 INFO:teuthology.orchestra.run.smithi177.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-04-14T08:30:34.447 INFO:teuthology.orchestra.run.smithi177.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-04-14T08:30:34.464 INFO:teuthology.orchestra.run.smithi177.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-04-14T08:30:34.481 INFO:teuthology.orchestra.run.smithi177.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 881 kB/s | 59 kB 00:00 2024-04-14T08:30:34.498 INFO:teuthology.orchestra.run.smithi177.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-14T08:30:34.523 INFO:teuthology.orchestra.run.smithi177.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-04-14T08:30:34.525 INFO:teuthology.orchestra.run.smithi172.stdout:(24/174): ceph-common-19.0.0-2592.gd7e176b4.el9 4.2 MB/s | 21 MB 00:04 2024-04-14T08:30:34.548 INFO:teuthology.orchestra.run.smithi177.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-14T08:30:34.550 INFO:teuthology.orchestra.run.smithi172.stdout:(25/174): python3-rgw-19.0.0-2592.gd7e176b4.el9 283 kB/s | 99 kB 00:00 2024-04-14T08:30:34.565 INFO:teuthology.orchestra.run.smithi177.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-04-14T08:30:34.599 INFO:teuthology.orchestra.run.smithi177.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-04-14T08:30:34.649 INFO:teuthology.orchestra.run.smithi177.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-04-14T08:30:34.674 INFO:teuthology.orchestra.run.smithi177.stdout:(150/174): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-04-14T08:30:34.908 INFO:teuthology.orchestra.run.smithi177.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-04-14T08:30:34.976 INFO:teuthology.orchestra.run.smithi172.stdout:(26/174): ceph-test-19.0.0-2592.gd7e176b4.el9.x 15 MB/s | 48 MB 00:03 2024-04-14T08:30:34.983 INFO:teuthology.orchestra.run.smithi177.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-04-14T08:30:35.009 INFO:teuthology.orchestra.run.smithi172.stdout:(27/174): rbd-fuse-19.0.0-2592.gd7e176b4.el9.x8 176 kB/s | 85 kB 00:00 2024-04-14T08:30:35.016 INFO:teuthology.orchestra.run.smithi177.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 42 kB 00:00 2024-04-14T08:30:35.058 INFO:teuthology.orchestra.run.smithi172.stdout:(28/174): rbd-nbd-19.0.0-2592.gd7e176b4.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-14T08:30:35.067 INFO:teuthology.orchestra.run.smithi177.stdout:(154/174): librados2-19.0.0-2592.gd7e176b4.el9. 7.3 MB/s | 3.4 MB 00:00 2024-04-14T08:30:35.084 INFO:teuthology.orchestra.run.smithi172.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2592.g 451 kB/s | 34 kB 00:00 2024-04-14T08:30:35.092 INFO:teuthology.orchestra.run.smithi177.stdout:(155/174): NetworkManager-tui-1.47.4-1.el9.x86_ 3.2 MB/s | 249 kB 00:00 2024-04-14T08:30:35.143 INFO:teuthology.orchestra.run.smithi177.stdout:(156/174): librbd1-19.0.0-2592.gd7e176b4.el9.x8 6.4 MB/s | 3.1 MB 00:00 2024-04-14T08:30:35.143 INFO:teuthology.orchestra.run.smithi172.stdout:(30/174): rbd-mirror-19.0.0-2592.gd7e176b4.el9. 5.2 MB/s | 3.1 MB 00:00 2024-04-14T08:30:35.168 INFO:teuthology.orchestra.run.smithi177.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 484 kB/s | 36 kB 00:00 2024-04-14T08:30:35.168 INFO:teuthology.orchestra.run.smithi172.stdout:(31/174): ceph-mgr-cephadm-19.0.0-2592.gd7e176b 1.3 MB/s | 141 kB 00:00 2024-04-14T08:30:35.213 INFO:teuthology.orchestra.run.smithi177.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 490 kB/s | 71 kB 00:00 2024-04-14T08:30:35.236 INFO:teuthology.orchestra.run.smithi172.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2592.gd7 3.6 MB/s | 249 kB 00:00 2024-04-14T08:30:35.243 INFO:teuthology.orchestra.run.smithi177.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.3 MB/s | 256 kB 00:00 2024-04-14T08:30:35.268 INFO:teuthology.orchestra.run.smithi177.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 418 kB/s | 23 kB 00:00 2024-04-14T08:30:35.294 INFO:teuthology.orchestra.run.smithi177.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-04-14T08:30:35.319 INFO:teuthology.orchestra.run.smithi172.stdout:(33/174): ceph-mgr-rook-19.0.0-2592.gd7e176b4.e 594 kB/s | 49 kB 00:00 2024-04-14T08:30:35.319 INFO:teuthology.orchestra.run.smithi177.stdout:(162/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.1 MB/s | 192 kB 00:00 2024-04-14T08:30:35.344 INFO:teuthology.orchestra.run.smithi177.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-04-14T08:30:35.369 INFO:teuthology.orchestra.run.smithi177.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-04-14T08:30:35.394 INFO:teuthology.orchestra.run.smithi172.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2592.gd 220 kB/s | 16 kB 00:00 2024-04-14T08:30:35.403 INFO:teuthology.orchestra.run.smithi177.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 762 kB/s | 64 kB 00:00 2024-04-14T08:30:35.429 INFO:teuthology.orchestra.run.smithi177.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 489 kB/s | 29 kB 00:00 2024-04-14T08:30:35.470 INFO:teuthology.orchestra.run.smithi172.stdout:(35/174): ceph-volume-19.0.0-2592.gd7e176b4.el9 3.4 MB/s | 264 kB 00:00 2024-04-14T08:30:35.487 INFO:teuthology.orchestra.run.smithi177.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 10 MB/s | 1.2 MB 00:00 2024-04-14T08:30:35.537 INFO:teuthology.orchestra.run.smithi177.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 21 MB/s | 2.3 MB 00:00 2024-04-14T08:30:35.563 INFO:teuthology.orchestra.run.smithi177.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.0 MB/s | 465 kB 00:00 2024-04-14T08:30:35.588 INFO:teuthology.orchestra.run.smithi177.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 622 kB/s | 31 kB 00:00 2024-04-14T08:30:35.613 INFO:teuthology.orchestra.run.smithi177.stdout:(171/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2024-04-14T08:30:35.637 INFO:teuthology.orchestra.run.smithi172.stdout:(36/174): ceph-mgr-dashboard-19.0.0-2592.gd7e17 6.4 MB/s | 3.5 MB 00:00 2024-04-14T08:30:35.663 INFO:teuthology.orchestra.run.smithi177.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 8.3 MB/s | 2.2 MB 00:00 2024-04-14T08:30:35.697 INFO:teuthology.orchestra.run.smithi177.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.9 MB/s | 1.1 MB 00:00 2024-04-14T08:30:35.720 INFO:teuthology.orchestra.run.smithi172.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-04-14T08:30:35.739 INFO:teuthology.orchestra.run.smithi177.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 593 kB/s | 74 kB 00:00 2024-04-14T08:30:35.742 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-04-14T08:30:35.742 INFO:teuthology.orchestra.run.smithi177.stdout:Total 18 MB/s | 228 MB 00:12 2024-04-14T08:30:35.745 INFO:teuthology.orchestra.run.smithi172.stdout:(38/174): cephfs-top-19.0.0-2592.gd7e176b4.el9. 168 kB/s | 18 kB 00:00 2024-04-14T08:30:35.771 INFO:teuthology.orchestra.run.smithi172.stdout:(39/174): cephadm-19.0.0-2592.gd7e176b4.el9.noa 2.5 MB/s | 762 kB 00:00 2024-04-14T08:30:35.846 INFO:teuthology.orchestra.run.smithi172.stdout:(40/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 140 kB/s | 18 kB 00:00 2024-04-14T08:30:36.121 INFO:teuthology.orchestra.run.smithi172.stdout:(41/174): fuse-2.9.9-15.el9.x86_64.rpm 214 kB/s | 80 kB 00:00 2024-04-14T08:30:36.271 INFO:teuthology.orchestra.run.smithi172.stdout:(42/174): jq-1.6-16.el9.x86_64.rpm 374 kB/s | 187 kB 00:00 2024-04-14T08:30:36.305 INFO:teuthology.orchestra.run.smithi172.stdout:(43/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-14T08:30:36.463 INFO:teuthology.orchestra.run.smithi172.stdout:(44/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.1 MB/s | 218 kB 00:00 2024-04-14T08:30:36.489 INFO:teuthology.orchestra.run.smithi172.stdout:(45/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-04-14T08:30:36.622 INFO:teuthology.orchestra.run.smithi172.stdout:(46/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-14T08:30:36.648 INFO:teuthology.orchestra.run.smithi172.stdout:(47/174): python3-ply-3.11-14.el9.noarch.rpm 672 kB/s | 106 kB 00:00 2024-04-14T08:30:36.723 INFO:teuthology.orchestra.run.smithi172.stdout:(48/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-14T08:30:36.756 INFO:teuthology.orchestra.run.smithi172.stdout:(49/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-14T08:30:36.832 INFO:teuthology.orchestra.run.smithi172.stdout:(50/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-14T08:30:36.873 INFO:teuthology.orchestra.run.smithi172.stdout:(51/174): python3-cryptography-36.0.1-4.el9.x86 3.0 MB/s | 1.2 MB 00:00 2024-04-14T08:30:36.982 INFO:teuthology.orchestra.run.smithi172.stdout:(52/174): boost-program-options-1.75.0-8.el9.x8 986 kB/s | 107 kB 00:00 2024-04-14T08:30:37.024 INFO:teuthology.orchestra.run.smithi172.stdout:(53/174): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-04-14T08:30:37.049 INFO:teuthology.orchestra.run.smithi172.stdout:(54/174): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-04-14T08:30:37.072 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-04-14T08:30:37.075 INFO:teuthology.orchestra.run.smithi172.stdout:(55/174): elfutils-libelf-devel-0.190-2.el9.x86 232 kB/s | 21 kB 00:00 2024-04-14T08:30:37.133 INFO:teuthology.orchestra.run.smithi172.stdout:(56/174): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-14T08:30:37.150 INFO:teuthology.orchestra.run.smithi172.stdout:(57/174): flexiblas-3.0.4-8.el9.x86_64.rpm 324 kB/s | 33 kB 00:00 2024-04-14T08:30:37.226 INFO:teuthology.orchestra.run.smithi172.stdout:(58/174): flexiblas-openblas-openmp-3.0.4-8.el9 198 kB/s | 18 kB 00:00 2024-04-14T08:30:37.260 INFO:teuthology.orchestra.run.smithi172.stdout:(59/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-14T08:30:37.326 INFO:teuthology.orchestra.run.smithi172.stdout:(60/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-04-14T08:30:37.356 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-04-14T08:30:37.356 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-04-14T08:30:37.385 INFO:teuthology.orchestra.run.smithi172.stdout:(61/174): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-04-14T08:30:37.427 INFO:teuthology.orchestra.run.smithi172.stdout:(62/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.5 MB/s | 3.0 MB 00:00 2024-04-14T08:30:37.452 INFO:teuthology.orchestra.run.smithi172.stdout:(63/174): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-04-14T08:30:37.485 INFO:teuthology.orchestra.run.smithi172.stdout:(64/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-04-14T08:30:37.536 INFO:teuthology.orchestra.run.smithi172.stdout:(65/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-04-14T08:30:37.561 INFO:teuthology.orchestra.run.smithi172.stdout:(66/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-14T08:30:37.595 INFO:teuthology.orchestra.run.smithi172.stdout:(67/174): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-14T08:30:37.637 INFO:teuthology.orchestra.run.smithi172.stdout:(68/174): openblas-0.3.21-2.el9.x86_64.rpm 350 kB/s | 35 kB 00:00 2024-04-14T08:30:38.054 INFO:teuthology.orchestra.run.smithi172.stdout:(69/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 9.0 MB/s | 4.1 MB 00:00 2024-04-14T08:30:38.179 INFO:teuthology.orchestra.run.smithi172.stdout:(70/174): postgresql-13.14-1.el9.x86_64.rpm 2.9 MB/s | 1.6 MB 00:00 2024-04-14T08:30:38.454 INFO:teuthology.orchestra.run.smithi172.stdout:(71/174): postgresql-private-libs-13.14-1.el9.x 484 kB/s | 133 kB 00:00 2024-04-14T08:30:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:(72/174): postgresql-contrib-13.14-1.el9.x86_64 724 kB/s | 911 kB 00:01 2024-04-14T08:30:39.516 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-04-14T08:30:39.516 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-04-14T08:30:40.279 INFO:teuthology.orchestra.run.smithi172.stdout:(73/174): openblas-openmp-0.3.21-2.el9.x86_64.r 1.7 MB/s | 4.7 MB 00:02 2024-04-14T08:30:40.480 INFO:teuthology.orchestra.run.smithi172.stdout:(74/174): python3-attrs-20.3.0-7.el9.noarch.rpm 437 kB/s | 87 kB 00:00 2024-04-14T08:30:40.872 INFO:teuthology.orchestra.run.smithi172.stdout:(75/174): protobuf-3.14.0-13.el9.x86_64.rpm 663 kB/s | 1.0 MB 00:01 2024-04-14T08:30:41.238 INFO:teuthology.orchestra.run.smithi172.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 669 kB/s | 245 kB 00:00 2024-04-14T08:30:41.597 INFO:teuthology.orchestra.run.smithi172.stdout:(77/174): python3-jinja2-2.11.3-5.el9.noarch.rp 692 kB/s | 248 kB 00:00 2024-04-14T08:30:41.705 INFO:teuthology.orchestra.run.smithi172.stdout:(78/174): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-04-14T08:30:41.848 INFO:teuthology.orchestra.run.smithi172.stdout:(79/174): python3-jsonpointer-2.0-4.el9.noarch. 135 kB/s | 19 kB 00:00 2024-04-14T08:30:41.939 INFO:teuthology.orchestra.run.smithi172.stdout:(80/174): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-04-14T08:30:42.048 INFO:teuthology.orchestra.run.smithi172.stdout:(81/174): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-04-14T08:30:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:(82/174): python3-mako-1.1.4-6.el9.noarch.rpm 543 kB/s | 172 kB 00:00 2024-04-14T08:30:42.365 INFO:teuthology.orchestra.run.smithi172.stdout:(83/174): postgresql-server-13.14-1.el9.x86_64. 1.5 MB/s | 5.8 MB 00:03 2024-04-14T08:30:42.432 INFO:teuthology.orchestra.run.smithi172.stdout:(84/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-14T08:30:42.482 INFO:teuthology.orchestra.run.smithi172.stdout:(85/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-04-14T08:30:42.525 INFO:teuthology.orchestra.run.smithi172.stdout:(86/174): python3-packaging-20.9-5.el9.noarch.r 837 kB/s | 77 kB 00:00 2024-04-14T08:30:42.558 INFO:teuthology.orchestra.run.smithi172.stdout:(87/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 638 kB/s | 250 kB 00:00 2024-04-14T08:30:42.583 INFO:teuthology.orchestra.run.smithi172.stdout:(88/174): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-04-14T08:30:42.633 INFO:teuthology.orchestra.run.smithi172.stdout:(89/174): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-04-14T08:30:42.692 INFO:teuthology.orchestra.run.smithi172.stdout:(90/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-14T08:30:42.726 INFO:teuthology.orchestra.run.smithi172.stdout:(91/174): python3-pytz-2021.1-5.el9.noarch.rpm 555 kB/s | 51 kB 00:00 2024-04-14T08:30:42.792 INFO:teuthology.orchestra.run.smithi172.stdout:(92/174): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-04-14T08:30:42.834 INFO:teuthology.orchestra.run.smithi172.stdout:(93/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 575 kB/s | 159 kB 00:00 2024-04-14T08:30:42.893 INFO:teuthology.orchestra.run.smithi172.stdout:(94/174): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-04-14T08:30:42.935 INFO:teuthology.orchestra.run.smithi172.stdout:(95/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-04-14T08:30:43.011 INFO:teuthology.orchestra.run.smithi172.stdout:(96/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-04-14T08:30:43.044 INFO:teuthology.orchestra.run.smithi172.stdout:(97/174): qatlib-service-24.02.0-1.el9.x86_64.r 330 kB/s | 36 kB 00:00 2024-04-14T08:30:43.111 INFO:teuthology.orchestra.run.smithi172.stdout:(98/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-04-14T08:30:43.220 INFO:teuthology.orchestra.run.smithi172.stdout:(99/174): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-04-14T08:30:43.386 INFO:teuthology.orchestra.run.smithi172.stdout:(100/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-04-14T08:30:43.411 INFO:teuthology.orchestra.run.smithi172.stdout:(101/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 488 kB/s | 94 kB 00:00 2024-04-14T08:30:43.487 INFO:teuthology.orchestra.run.smithi172.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 637 kB/s | 64 kB 00:00 2024-04-14T08:30:43.512 INFO:teuthology.orchestra.run.smithi172.stdout:(103/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-04-14T08:30:43.554 INFO:teuthology.orchestra.run.smithi172.stdout:(104/174): socat-1.7.4.1-5.el9.x86_64.rpm 598 kB/s | 305 kB 00:00 2024-04-14T08:30:43.680 INFO:teuthology.orchestra.run.smithi172.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 116 kB/s | 22 kB 00:00 2024-04-14T08:30:43.971 INFO:teuthology.orchestra.run.smithi172.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-04-14T08:30:44.071 INFO:teuthology.orchestra.run.smithi172.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-14T08:30:44.097 INFO:teuthology.orchestra.run.smithi172.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 741 kB/s | 309 kB 00:00 2024-04-14T08:30:44.172 INFO:teuthology.orchestra.run.smithi172.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 351 kB/s | 26 kB 00:00 2024-04-14T08:30:44.230 INFO:teuthology.orchestra.run.smithi172.stdout:(110/174): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-14T08:30:44.256 INFO:teuthology.orchestra.run.smithi172.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-04-14T08:30:44.381 INFO:teuthology.orchestra.run.smithi172.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-14T08:30:44.406 INFO:teuthology.orchestra.run.smithi172.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 384 kB/s | 67 kB 00:00 2024-04-14T08:30:44.432 INFO:teuthology.orchestra.run.smithi172.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 862 kB/s | 151 kB 00:00 2024-04-14T08:30:44.473 INFO:teuthology.orchestra.run.smithi172.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-04-14T08:30:44.507 INFO:teuthology.orchestra.run.smithi172.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 576 kB/s | 43 kB 00:00 2024-04-14T08:30:44.549 INFO:teuthology.orchestra.run.smithi172.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 426 kB/s | 32 kB 00:00 2024-04-14T08:30:44.583 INFO:teuthology.orchestra.run.smithi172.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-14T08:30:44.625 INFO:teuthology.orchestra.run.smithi172.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-14T08:30:44.650 INFO:teuthology.orchestra.run.smithi172.stdout:(120/174): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-04-14T08:30:44.692 INFO:teuthology.orchestra.run.smithi172.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-14T08:30:44.717 INFO:teuthology.orchestra.run.smithi172.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-14T08:30:44.742 INFO:teuthology.orchestra.run.smithi172.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 383 kB/s | 35 kB 00:00 2024-04-14T08:30:44.776 INFO:teuthology.orchestra.run.smithi172.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-14T08:30:44.835 INFO:teuthology.orchestra.run.smithi172.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-04-14T08:30:44.860 INFO:teuthology.orchestra.run.smithi172.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-14T08:30:44.885 INFO:teuthology.orchestra.run.smithi172.stdout:(127/174): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-14T08:30:44.911 INFO:teuthology.orchestra.run.smithi172.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-14T08:30:44.936 INFO:teuthology.orchestra.run.smithi172.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-14T08:30:44.961 INFO:teuthology.orchestra.run.smithi172.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-14T08:30:44.986 INFO:teuthology.orchestra.run.smithi172.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-14T08:30:45.037 INFO:teuthology.orchestra.run.smithi172.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 616 kB/s | 46 kB 00:00 2024-04-14T08:30:45.062 INFO:teuthology.orchestra.run.smithi172.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-14T08:30:45.096 INFO:teuthology.orchestra.run.smithi172.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-04-14T08:30:45.121 INFO:teuthology.orchestra.run.smithi172.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-04-14T08:30:45.146 INFO:teuthology.orchestra.run.smithi172.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-14T08:30:45.163 INFO:teuthology.orchestra.run.smithi172.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 244 kB/s | 16 kB 00:00 2024-04-14T08:30:45.188 INFO:teuthology.orchestra.run.smithi172.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-14T08:30:45.205 INFO:teuthology.orchestra.run.smithi172.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 480 kB/s | 28 kB 00:00 2024-04-14T08:30:45.222 INFO:teuthology.orchestra.run.smithi172.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 522 kB/s | 31 kB 00:00 2024-04-14T08:30:45.272 INFO:teuthology.orchestra.run.smithi172.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-14T08:30:45.298 INFO:teuthology.orchestra.run.smithi172.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-14T08:30:45.323 INFO:teuthology.orchestra.run.smithi172.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 504 kB/s | 59 kB 00:00 2024-04-14T08:30:45.348 INFO:teuthology.orchestra.run.smithi172.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-04-14T08:30:45.373 INFO:teuthology.orchestra.run.smithi172.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-14T08:30:45.398 INFO:teuthology.orchestra.run.smithi172.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-14T08:30:45.449 INFO:teuthology.orchestra.run.smithi172.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-14T08:30:45.474 INFO:teuthology.orchestra.run.smithi172.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-14T08:30:45.500 INFO:teuthology.orchestra.run.smithi172.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-14T08:30:45.550 INFO:teuthology.orchestra.run.smithi172.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-14T08:30:45.775 INFO:teuthology.orchestra.run.smithi172.stdout:(151/174): librados2-19.0.0-2592.gd7e176b4.el9. 11 MB/s | 3.4 MB 00:00 2024-04-14T08:30:45.909 INFO:teuthology.orchestra.run.smithi172.stdout:(152/174): librbd1-19.0.0-2592.gd7e176b4.el9.x8 7.7 MB/s | 3.1 MB 00:00 2024-04-14T08:30:46.092 INFO:teuthology.orchestra.run.smithi172.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 228 kB/s | 42 kB 00:00 2024-04-14T08:30:46.334 INFO:teuthology.orchestra.run.smithi172.stdout:(154/174): NetworkManager-libnm-1.47.4-1.el9.x8 3.3 MB/s | 1.8 MB 00:00 2024-04-14T08:30:46.359 INFO:teuthology.orchestra.run.smithi172.stdout:(155/174): NetworkManager-tui-1.47.4-1.el9.x86_ 932 kB/s | 249 kB 00:00 2024-04-14T08:30:46.401 INFO:teuthology.orchestra.run.smithi172.stdout:(156/174): NetworkManager-1.47.4-1.el9.x86_64.r 2.7 MB/s | 2.3 MB 00:00 2024-04-14T08:30:46.443 INFO:teuthology.orchestra.run.smithi172.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 656 kB/s | 71 kB 00:00 2024-04-14T08:30:46.468 INFO:teuthology.orchestra.run.smithi172.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 335 kB/s | 36 kB 00:00 2024-04-14T08:30:46.510 INFO:teuthology.orchestra.run.smithi172.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-04-14T08:30:46.552 INFO:teuthology.orchestra.run.smithi172.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-04-14T08:30:46.578 INFO:teuthology.orchestra.run.smithi172.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 211 kB/s | 23 kB 00:00 2024-04-14T08:30:46.604 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-04-14T08:30:46.611 INFO:teuthology.orchestra.run.smithi172.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 108 kB 00:00 2024-04-14T08:30:46.653 INFO:teuthology.orchestra.run.smithi172.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-04-14T08:30:46.678 INFO:teuthology.orchestra.run.smithi172.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-04-14T08:30:46.701 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-14T08:30:46.712 INFO:teuthology.orchestra.run.smithi172.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 634 kB/s | 64 kB 00:00 2024-04-14T08:30:46.736 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-14T08:30:46.754 INFO:teuthology.orchestra.run.smithi172.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 285 kB/s | 29 kB 00:00 2024-04-14T08:30:46.807 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-14T08:30:46.824 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-14T08:30:46.963 INFO:teuthology.orchestra.run.smithi172.stdout:(167/174): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.3 MB 00:00 2024-04-14T08:30:46.996 INFO:teuthology.orchestra.run.smithi172.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.8 MB/s | 1.2 MB 00:00 2024-04-14T08:30:47.008 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-14T08:30:47.071 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-14T08:30:47.088 INFO:teuthology.orchestra.run.smithi172.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.6 MB/s | 465 kB 00:00 2024-04-14T08:30:47.111 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-14T08:30:47.113 INFO:teuthology.orchestra.run.smithi172.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 266 kB/s | 31 kB 00:00 2024-04-14T08:30:47.136 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-14T08:30:47.189 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-14T08:30:47.222 INFO:teuthology.orchestra.run.smithi172.stdout:(171/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.4 MB/s | 323 kB 00:00 2024-04-14T08:30:47.264 INFO:teuthology.orchestra.run.smithi172.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.1 MB/s | 1.1 MB 00:00 2024-04-14T08:30:47.314 INFO:teuthology.orchestra.run.smithi172.stdout:(173/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.6 MB/s | 2.2 MB 00:00 2024-04-14T08:30:47.339 INFO:teuthology.orchestra.run.smithi172.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 635 kB/s | 74 kB 00:00 2024-04-14T08:30:47.346 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-04-14T08:30:47.346 INFO:teuthology.orchestra.run.smithi172.stdout:Total 12 MB/s | 228 MB 00:18 2024-04-14T08:30:47.527 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-14T08:30:47.580 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:19.0.0-2592.gd7e176b4.el9.x86_64 10/198 2024-04-14T08:30:47.690 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:19.0.0-2592.gd7e176b4.el9.x86_64 10/198 2024-04-14T08:30:47.714 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:19.0.0-2592.gd7e176b4.el9.x86_64 11/198 2024-04-14T08:30:47.789 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:19.0.0-2592.gd7e176b4.el9.x86_64 11/198 2024-04-14T08:30:47.819 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:19.0.0-2592.gd7e176b4.el9.x86_64 12/198 2024-04-14T08:30:47.860 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-14T08:30:48.049 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-14T08:30:48.108 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-14T08:30:48.128 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:19.0.0-2592.gd7e176b4.el9.x86_64 16/198 2024-04-14T08:30:48.231 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2592.gd7e176b4.el9.x86_64 16/198 2024-04-14T08:30:48.253 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:19.0.0-2592.gd7e176b4.el9.x86 17/198 2024-04-14T08:30:48.302 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2592.gd7e176b4.el9.x86 17/198 2024-04-14T08:30:48.349 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-14T08:30:48.372 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-14T08:30:48.387 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-14T08:30:48.580 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-14T08:30:48.611 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-14T08:30:48.643 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-14T08:30:48.724 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-04-14T08:30:48.827 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-14T08:30:49.003 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-14T08:30:49.010 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-04-14T08:30:49.010 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-04-14T08:30:49.064 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-14T08:30:49.178 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-14T08:30:49.231 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-14T08:30:49.450 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-14T08:30:49.565 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-14T08:30:49.657 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-14T08:30:49.686 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-14T08:30:49.721 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-14T08:30:49.785 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-04-14T08:30:49.828 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-04-14T08:30:49.872 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-04-14T08:30:49.905 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-04-14T08:30:49.932 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-04-14T08:30:50.004 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-04-14T08:30:50.053 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-04-14T08:30:50.118 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-04-14T08:30:50.161 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-04-14T08:30:50.238 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-04-14T08:30:50.281 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:19.0.0-2592.gd7e176b4.el9. 39/198 2024-04-14T08:30:50.325 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:19.0.0-2592.gd7e176b4.el 40/198 2024-04-14T08:30:50.353 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:19.0.0-2592.gd7e176b4.el9.x86_6 41/198 2024-04-14T08:30:50.423 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-14T08:30:50.484 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-14T08:30:50.559 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-14T08:30:50.690 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-14T08:30:50.736 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-14T08:30:51.221 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-04-14T08:30:51.229 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-04-14T08:30:51.261 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-14T08:30:51.338 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-14T08:30:51.419 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-14T08:30:51.468 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-14T08:30:51.477 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-14T08:30:51.509 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-14T08:30:51.558 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-14T08:30:51.595 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:19.0.0-2592.gd7e176b4.el9.x86_6 53/198 2024-04-14T08:30:51.625 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-14T08:30:51.914 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-14T08:30:52.294 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-14T08:30:52.369 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-14T08:30:52.414 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-14T08:30:52.559 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-14T08:30:52.615 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-14T08:30:52.669 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-14T08:30:52.734 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-14T08:30:52.766 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-14T08:30:52.802 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-14T08:30:52.849 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-14T08:30:52.884 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-14T08:30:52.928 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-14T08:30:52.970 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-14T08:30:53.111 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-14T08:30:53.159 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-14T08:30:53.191 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-14T08:30:53.233 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-14T08:30:53.327 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-14T08:30:53.366 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-14T08:30:53.406 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-14T08:30:53.432 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-14T08:30:53.456 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-14T08:30:53.535 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-14T08:30:53.666 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-14T08:30:53.705 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-14T08:30:53.732 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-04-14T08:30:53.792 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-04-14T08:30:53.832 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-04-14T08:30:53.863 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-04-14T08:30:53.892 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-04-14T08:30:53.927 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-04-14T08:30:53.960 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-04-14T08:30:54.002 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-04-14T08:30:54.037 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-04-14T08:30:54.123 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-04-14T08:30:54.166 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-04-14T08:30:54.206 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-04-14T08:30:54.254 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-04-14T08:30:54.351 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-04-14T08:30:54.408 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-04-14T08:30:55.130 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-04-14T08:30:55.180 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-04-14T08:30:55.213 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-04-14T08:30:55.321 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-04-14T08:30:55.379 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-04-14T08:30:55.458 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-04-14T08:30:55.490 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-04-14T08:30:55.668 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-04-14T08:30:55.696 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-04-14T08:30:55.774 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-04-14T08:30:56.418 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-04-14T08:30:56.536 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-04-14T08:30:57.771 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-04-14T08:30:57.858 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-04-14T08:30:57.885 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-04-14T08:30:58.198 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-04-14T08:30:58.223 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:19.0.0-2592.gd7e176b4.el9.x86_64 112/198 2024-04-14T08:30:58.314 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:19.0.0-2592.gd7e176b4.el9.x86_64 112/198 2024-04-14T08:30:58.345 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:19.0.0-2592.gd7e176b4.el9.x86_64 113/198 2024-04-14T08:30:58.381 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-04-14T08:30:58.447 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-04-14T08:30:58.544 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-14T08:30:58.587 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-14T08:30:58.633 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-14T08:30:58.644 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-14T08:30:58.690 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-14T08:30:58.745 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-14T08:30:58.778 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-14T08:30:58.804 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-14T08:30:58.842 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-14T08:30:58.856 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-14T08:30:58.868 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:19.0.0-2592.gd7e176b4.el9.x86_64 116/198 2024-04-14T08:30:58.940 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:19.0.0-2592.gd7e176b4.el9.x86_64 116/198 2024-04-14T08:30:58.969 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 117/198 2024-04-14T08:30:59.185 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-14T08:30:59.243 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:19.0.0-2592.gd7e176b4.el9.x86_64 10/198 2024-04-14T08:30:59.365 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:19.0.0-2592.gd7e176b4.el9.x86_64 10/198 2024-04-14T08:30:59.394 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:19.0.0-2592.gd7e176b4.el9.x86_64 11/198 2024-04-14T08:30:59.503 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:19.0.0-2592.gd7e176b4.el9.x86_64 11/198 2024-04-14T08:30:59.532 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:19.0.0-2592.gd7e176b4.el9.x86_64 12/198 2024-04-14T08:30:59.574 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-14T08:30:59.759 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-14T08:30:59.821 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-14T08:30:59.849 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:19.0.0-2592.gd7e176b4.el9.x86_64 16/198 2024-04-14T08:30:59.953 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2592.gd7e176b4.el9.x86_64 16/198 2024-04-14T08:30:59.983 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:19.0.0-2592.gd7e176b4.el9.x86 17/198 2024-04-14T08:31:00.049 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2592.gd7e176b4.el9.x86 17/198 2024-04-14T08:31:00.088 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-14T08:31:00.110 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-14T08:31:00.127 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-14T08:31:00.310 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-14T08:31:00.349 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-14T08:31:00.382 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-14T08:31:00.691 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-14T08:31:00.985 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-14T08:31:01.037 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-14T08:31:01.152 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-14T08:31:01.215 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-14T08:31:01.416 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-14T08:31:01.520 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-14T08:31:01.598 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.0.0-2592.gd7e176b4.el9.x86_64 118/198 2024-04-14T08:31:01.624 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-14T08:31:01.632 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:19.0.0-2592.gd7e176b4.el9.x86_64 118/198 2024-04-14T08:31:01.653 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-14T08:31:01.687 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-14T08:31:01.751 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-04-14T08:31:01.787 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-04-14T08:31:01.830 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-04-14T08:31:01.862 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-04-14T08:31:01.890 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-04-14T08:31:01.967 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-04-14T08:31:02.020 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-04-14T08:31:02.060 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-04-14T08:31:02.104 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-04-14T08:31:02.171 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.0.0-2592.gd7e176b4.el9.x86_64 118/198 2024-04-14T08:31:02.178 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-04-14T08:31:02.210 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:19.0.0-2592.gd7e176b4.el9.x86_64 119/198 2024-04-14T08:31:02.214 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:19.0.0-2592.gd7e176b4.el9. 39/198 2024-04-14T08:31:02.258 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:19.0.0-2592.gd7e176b4.el 40/198 2024-04-14T08:31:02.294 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:19.0.0-2592.gd7e176b4.el9.x86_64 119/198 2024-04-14T08:31:02.294 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-14T08:31:02.294 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-14T08:31:02.295 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:02.303 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:19.0.0-2592.gd7e176b4.el9.x86_6 41/198 2024-04-14T08:31:02.329 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:19.0.0-2592.gd7e176b4.el9.x86_64 120/198 2024-04-14T08:31:02.371 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-14T08:31:02.417 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-14T08:31:02.490 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-14T08:31:02.642 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-14T08:31:02.702 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-14T08:31:03.224 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-14T08:31:03.306 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-14T08:31:03.396 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-14T08:31:03.443 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-14T08:31:03.453 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-14T08:31:03.494 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-14T08:31:03.532 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-14T08:31:03.570 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:19.0.0-2592.gd7e176b4.el9.x86_6 53/198 2024-04-14T08:31:03.693 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-14T08:31:04.017 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-14T08:31:04.198 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-14T08:31:04.285 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-14T08:31:04.329 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-14T08:31:04.465 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-14T08:31:04.515 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-14T08:31:04.578 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-14T08:31:04.650 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-14T08:31:04.682 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-14T08:31:04.710 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-14T08:31:04.757 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-14T08:31:04.793 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-14T08:31:04.836 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-14T08:31:04.878 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-14T08:31:05.010 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-14T08:31:05.059 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-14T08:31:05.089 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-14T08:31:05.130 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-14T08:31:05.224 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-14T08:31:05.263 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-14T08:31:05.304 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-14T08:31:05.329 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-14T08:31:05.354 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-14T08:31:05.441 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-14T08:31:05.563 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-14T08:31:05.611 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-14T08:31:05.639 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-04-14T08:31:05.673 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-04-14T08:31:05.712 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-04-14T08:31:05.744 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-04-14T08:31:05.774 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-04-14T08:31:05.808 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-04-14T08:31:05.842 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-04-14T08:31:05.882 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-04-14T08:31:05.917 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-04-14T08:31:06.004 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-04-14T08:31:06.056 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-04-14T08:31:06.088 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-04-14T08:31:06.134 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-04-14T08:31:06.232 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-04-14T08:31:06.280 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-04-14T08:31:07.017 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-04-14T08:31:07.061 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-04-14T08:31:07.094 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-04-14T08:31:07.192 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-04-14T08:31:07.244 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-04-14T08:31:07.333 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-04-14T08:31:07.379 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-04-14T08:31:07.557 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-04-14T08:31:07.586 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-04-14T08:31:07.664 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-04-14T08:31:08.308 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-04-14T08:31:08.416 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-04-14T08:31:09.662 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-04-14T08:31:09.746 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-04-14T08:31:09.773 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-04-14T08:31:10.084 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-04-14T08:31:10.119 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:19.0.0-2592.gd7e176b4.el9.x86_64 112/198 2024-04-14T08:31:10.194 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:19.0.0-2592.gd7e176b4.el9.x86_64 112/198 2024-04-14T08:31:10.224 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:19.0.0-2592.gd7e176b4.el9.x86_64 113/198 2024-04-14T08:31:10.261 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-04-14T08:31:10.720 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-14T08:31:10.747 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:19.0.0-2592.gd7e176b4.el9.x86_64 116/198 2024-04-14T08:31:10.819 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:19.0.0-2592.gd7e176b4.el9.x86_64 116/198 2024-04-14T08:31:10.849 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 117/198 2024-04-14T08:31:13.084 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.0.0-2592.gd7e176b4.el9.x86_64 118/198 2024-04-14T08:31:13.120 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:19.0.0-2592.gd7e176b4.el9.x86_64 118/198 2024-04-14T08:31:13.662 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.0.0-2592.gd7e176b4.el9.x86_64 118/198 2024-04-14T08:31:13.698 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:19.0.0-2592.gd7e176b4.el9.x86_64 119/198 2024-04-14T08:31:13.800 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:19.0.0-2592.gd7e176b4.el9.x86_64 119/198 2024-04-14T08:31:13.800 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-14T08:31:13.800 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-14T08:31:13.800 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:13.841 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:19.0.0-2592.gd7e176b4.el9.x86_64 120/198 2024-04-14T08:31:14.096 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2592.gd7e176b4.el9.x86_64 120/198 2024-04-14T08:31:14.096 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2024-04-14T08:31:14.096 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2024-04-14T08:31:14.096 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2024-04-14T08:31:14.097 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2024-04-14T08:31:14.097 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2024-04-14T08:31:14.097 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /root 2024-04-14T08:31:14.097 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2024-04-14T08:31:14.097 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:14.273 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:19.0.0-2592.gd7e176b4.el9.noarch 121/198 2024-04-14T08:31:14.290 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:19.0.0-2592.gd7e176b4.el9.noarch 121/198 2024-04-14T08:31:14.290 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:14.290 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-14T08:31:14.290 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:15.801 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:19.0.0-2592.gd7e176b4.el9.x86_64 122/198 2024-04-14T08:31:15.827 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:19.0.0-2592.gd7e176b4.el9.x86_64 122/198 2024-04-14T08:31:15.827 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:15.827 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-14T08:31:15.827 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-14T08:31:15.827 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-14T08:31:15.827 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:16.060 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:19.0.0-2592.gd7e176b4.el9.x86_64 123/198 2024-04-14T08:31:16.083 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:19.0.0-2592.gd7e176b4.el9.x86_64 123/198 2024-04-14T08:31:16.083 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:16.083 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-14T08:31:16.083 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-14T08:31:16.083 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-14T08:31:16.083 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:16.535 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:19.0.0-2592.gd7e176b4.el9.x86_64 124/198 2024-04-14T08:31:16.558 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:19.0.0-2592.gd7e176b4.el9.x86_64 124/198 2024-04-14T08:31:16.558 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:16.558 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-14T08:31:16.558 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-14T08:31:16.558 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-14T08:31:16.558 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:16.674 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-04-14T08:31:16.723 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-04-14T08:31:16.779 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-04-14T08:31:16.971 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-04-14T08:31:17.019 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-04-14T08:31:17.097 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-04-14T08:31:17.185 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-04-14T08:31:17.325 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-04-14T08:31:17.361 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 133/198 2024-04-14T08:31:17.393 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-04-14T08:31:17.425 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-04-14T08:31:17.461 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-04-14T08:31:17.488 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-04-14T08:31:17.886 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:19.0.0-2592.gd7e176b4.el9.noarch 138/198 2024-04-14T08:31:17.912 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:19.0.0-2592.gd7e176b4.el9.noarch 138/198 2024-04-14T08:31:17.944 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2592.gd7e176b4.e 139/198 2024-04-14T08:31:18.005 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2592.gd7e176b4. 140/198 2024-04-14T08:31:18.035 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2592.gd7e176b4.el9.noa 141/198 2024-04-14T08:31:19.078 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2592.gd7e176b4.el9.noa 141/198 2024-04-14T08:31:19.112 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2592.gd7e176b4.el9.n 142/198 2024-04-14T08:31:20.079 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2592.gd7e176b4.el9.n 142/198 2024-04-14T08:31:20.104 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2592.gd7e 143/198 2024-04-14T08:31:20.205 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2592.gd7e 143/198 2024-04-14T08:31:20.366 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2592.gd7e176b4.el 144/198 2024-04-14T08:31:20.406 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:19.0.0-2592.gd7e176b4.el9.x86_64 145/198 2024-04-14T08:31:20.429 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2592.gd7e176b4.el9.x86_64 145/198 2024-04-14T08:31:20.430 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:20.430 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-14T08:31:20.430 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-14T08:31:20.430 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-14T08:31:20.430 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:20.473 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:19.0.0-2592.gd7e176b4.el9.noarch 146/198 2024-04-14T08:31:20.484 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2592.gd7e176b4.el9.noarch 146/198 2024-04-14T08:31:20.581 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:19.0.0-2592.gd7e176b4.el9.x86_64 147/198 2024-04-14T08:31:25.016 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:19.0.0-2592.gd7e176b4.el9.x86_64 148/198 2024-04-14T08:31:25.084 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:19.0.0-2592.gd7e176b4.el9.x86_64 149/198 2024-04-14T08:31:25.449 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2592.gd7e176b4.el9.x86_64 120/198 2024-04-14T08:31:25.450 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2024-04-14T08:31:25.450 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2024-04-14T08:31:25.450 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2024-04-14T08:31:25.450 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2024-04-14T08:31:25.450 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2024-04-14T08:31:25.450 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /root 2024-04-14T08:31:25.450 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2024-04-14T08:31:25.450 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:25.609 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:19.0.0-2592.gd7e176b4.el9.noarch 121/198 2024-04-14T08:31:25.624 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:19.0.0-2592.gd7e176b4.el9.noarch 121/198 2024-04-14T08:31:25.624 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:25.625 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-14T08:31:25.625 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:25.954 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-04-14T08:31:25.980 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 151/198 2024-04-14T08:31:26.002 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 151/198 2024-04-14T08:31:26.003 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:26.003 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-14T08:31:26.003 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-14T08:31:26.003 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-14T08:31:26.003 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:26.047 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2592.gd7e17 152/198 2024-04-14T08:31:26.069 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2592.gd7e17 152/198 2024-04-14T08:31:26.069 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:26.069 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-14T08:31:26.069 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:26.114 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 153/198 2024-04-14T08:31:26.136 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 153/198 2024-04-14T08:31:26.136 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:26.136 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-14T08:31:26.136 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-14T08:31:26.136 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-14T08:31:26.136 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:26.448 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 154/198 2024-04-14T08:31:26.470 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 154/198 2024-04-14T08:31:26.471 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:26.471 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-14T08:31:26.471 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-14T08:31:26.471 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-14T08:31:26.471 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:26.523 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:19.0.0-2592.gd7e176b4.el9.x86_64 155/198 2024-04-14T08:31:26.553 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:19.0.0-2592.gd7e176b4.el9.x86_64 156/198 2024-04-14T08:31:26.580 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-04-14T08:31:26.766 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-shell-2:19.0.0-2592.gd7e176b4.el9.x86_64 158/198 2024-04-14T08:31:26.828 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 159/198 2024-04-14T08:31:26.888 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-14T08:31:26.936 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-14T08:31:26.979 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-14T08:31:27.012 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:19.0.0-2592.gd7e176b4.el9.x86_ 163/198 2024-04-14T08:31:27.044 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-14T08:31:27.074 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-14T08:31:27.111 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:19.0.0-2592.gd7e176b4.el9.x86_64 122/198 2024-04-14T08:31:27.137 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:19.0.0-2592.gd7e176b4.el9.x86_64 122/198 2024-04-14T08:31:27.138 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:27.138 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-14T08:31:27.138 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-14T08:31:27.138 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-14T08:31:27.138 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:27.149 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-14T08:31:27.212 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-14T08:31:27.239 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-14T08:31:27.379 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:19.0.0-2592.gd7e176b4.el9.x86_64 123/198 2024-04-14T08:31:27.402 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:19.0.0-2592.gd7e176b4.el9.x86_64 123/198 2024-04-14T08:31:27.403 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:27.403 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-14T08:31:27.403 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-14T08:31:27.403 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-14T08:31:27.403 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:27.855 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-14T08:31:27.856 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:19.0.0-2592.gd7e176b4.el9.x86_64 124/198 2024-04-14T08:31:27.880 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:19.0.0-2592.gd7e176b4.el9.x86_64 124/198 2024-04-14T08:31:27.880 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:27.880 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-14T08:31:27.880 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-14T08:31:27.880 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-14T08:31:27.880 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:27.896 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-14T08:31:27.916 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-14T08:31:27.986 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-04-14T08:31:28.035 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-04-14T08:31:28.090 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-04-14T08:31:28.273 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-04-14T08:31:28.323 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-04-14T08:31:28.401 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-04-14T08:31:28.490 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-04-14T08:31:28.630 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-04-14T08:31:28.683 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 133/198 2024-04-14T08:31:28.720 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-top-2:19.0.0-2592.gd7e176b4.el9.noarch 170/198 2024-04-14T08:31:28.723 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-04-14T08:31:28.754 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-04-14T08:31:28.790 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-04-14T08:31:28.817 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-04-14T08:31:28.823 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-14T08:31:28.909 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-04-14T08:31:28.943 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-04-14T08:31:28.977 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-04-14T08:31:29.002 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-14T08:31:29.027 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-14T08:31:29.054 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-14T08:31:29.054 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-14T08:31:29.141 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-14T08:31:29.171 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-14T08:31:29.171 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-14T08:31:29.241 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:19.0.0-2592.gd7e176b4.el9.noarch 138/198 2024-04-14T08:31:29.247 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-14T08:31:29.274 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-14T08:31:29.274 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:19.0.0-2592.gd7e176b4.el9.noarch 138/198 2024-04-14T08:31:29.306 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2592.gd7e176b4.e 139/198 2024-04-14T08:31:29.316 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-14T08:31:29.340 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-14T08:31:29.365 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-14T08:31:29.367 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2592.gd7e176b4. 140/198 2024-04-14T08:31:29.397 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2592.gd7e176b4.el9.noa 141/198 2024-04-14T08:31:29.400 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-14T08:31:29.438 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-14T08:31:29.479 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-14T08:31:29.504 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-14T08:31:29.529 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-14T08:31:29.555 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-14T08:31:29.580 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-14T08:31:29.587 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-14T08:31:29.587 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-14T08:31:29.774 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-14T08:31:29.774 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-14T08:31:29.841 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-14T08:31:29.865 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-14T08:31:29.898 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-14T08:31:29.924 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-14T08:31:29.958 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-14T08:31:30.024 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-14T08:31:30.441 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2592.gd7e176b4.el9.noa 141/198 2024-04-14T08:31:30.474 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2592.gd7e176b4.el9.n 142/198 2024-04-14T08:31:31.443 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2592.gd7e176b4.el9.n 142/198 2024-04-14T08:31:31.466 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2592.gd7e 143/198 2024-04-14T08:31:31.567 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2592.gd7e 143/198 2024-04-14T08:31:31.728 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2592.gd7e176b4.el 144/198 2024-04-14T08:31:31.759 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:19.0.0-2592.gd7e176b4.el9.x86_64 145/198 2024-04-14T08:31:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2592.gd7e176b4.el9.x86_64 145/198 2024-04-14T08:31:31.783 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:31.783 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-14T08:31:31.783 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-14T08:31:31.783 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-14T08:31:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:31.818 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:19.0.0-2592.gd7e176b4.el9.noarch 146/198 2024-04-14T08:31:31.830 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2592.gd7e176b4.el9.noarch 146/198 2024-04-14T08:31:31.918 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:19.0.0-2592.gd7e176b4.el9.x86_64 147/198 2024-04-14T08:31:35.493 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-14T08:31:35.493 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:19.0.0-2592.gd7e176b4.el9.x86_64 1/198 2024-04-14T08:31:35.493 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:19.0.0-2592.gd7e176b4.el9.x86_64 2/198 2024-04-14T08:31:35.493 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:19.0.0-2592.gd7e176b4.el9.x86_64 3/198 2024-04-14T08:31:35.493 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:19.0.0-2592.gd7e176b4.el9.x86_64 4/198 2024-04-14T08:31:35.493 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2592.gd7e17 5/198 2024-04-14T08:31:35.493 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:19.0.0-2592.gd7e176b4.el9.x86_64 6/198 2024-04-14T08:31:35.493 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:19.0.0-2592.gd7e176b4.el9.x86_64 7/198 2024-04-14T08:31:35.494 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:19.0.0-2592.gd7e176b4.el9.x86_64 8/198 2024-04-14T08:31:35.494 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:19.0.0-2592.gd7e176b4.el9.x86_64 9/198 2024-04-14T08:31:35.494 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 10/198 2024-04-14T08:31:35.494 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:19.0.0-2592.gd7e176b4.el9.x86_64 11/198 2024-04-14T08:31:35.494 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:19.0.0-2592.gd7e176b4.el9.x86_64 12/198 2024-04-14T08:31:35.495 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 13/198 2024-04-14T08:31:35.495 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-shell-2:19.0.0-2592.gd7e176b4.el9.x86_64 14/198 2024-04-14T08:31:35.495 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:19.0.0-2592.gd7e176b4.el9.x86_ 15/198 2024-04-14T08:31:35.495 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:19.0.0-2592.gd7e176b4.el9.x86_64 16/198 2024-04-14T08:31:35.495 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:19.0.0-2592.gd7e176b4.el9.x86_64 17/198 2024-04-14T08:31:35.495 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:19.0.0-2592.gd7e176b4.el9.x86_6 18/198 2024-04-14T08:31:35.495 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:19.0.0-2592.gd7e176b4.el9.x86 19/198 2024-04-14T08:31:35.496 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:19.0.0-2592.gd7e176b4.el9.x86_64 20/198 2024-04-14T08:31:35.496 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2592.gd7e176b4.el 21/198 2024-04-14T08:31:35.496 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:19.0.0-2592.gd7e176b4.el9. 22/198 2024-04-14T08:31:35.496 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:19.0.0-2592.gd7e176b4.el9.x86_6 23/198 2024-04-14T08:31:35.496 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:19.0.0-2592.gd7e176b4.el9.x86_64 24/198 2024-04-14T08:31:35.496 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:19.0.0-2592.gd7e176b4.el9.x86_64 25/198 2024-04-14T08:31:35.496 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 26/198 2024-04-14T08:31:35.496 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:19.0.0-2592.gd7e176b4.el9.x86_64 27/198 2024-04-14T08:31:35.496 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 28/198 2024-04-14T08:31:35.497 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:19.0.0-2592.gd7e176b4.el9.x86_64 29/198 2024-04-14T08:31:35.517 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2592.gd7e176b4. 30/198 2024-04-14T08:31:35.517 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2592.gd7e176b4.el9.noa 31/198 2024-04-14T08:31:35.517 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2592.gd7e176b4.el9.n 32/198 2024-04-14T08:31:35.517 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2592.gd7e 33/198 2024-04-14T08:31:35.517 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2592.gd7e176b4.el 34/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2592.gd7e176b4.el9.noarch 35/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2592.gd7e176b4.e 36/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:19.0.0-2592.gd7e176b4.el9.noarch 37/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:19.0.0-2592.gd7e176b4.el9.noarch 38/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-top-2:19.0.0-2592.gd7e176b4.el9.noarch 39/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-04-14T08:31:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-04-14T08:31:35.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-04-14T08:31:35.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-04-14T08:31:35.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-04-14T08:31:35.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-04-14T08:31:35.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-04-14T08:31:35.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-04-14T08:31:35.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-04-14T08:31:35.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-04-14T08:31:35.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-04-14T08:31:35.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-04-14T08:31:35.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-04-14T08:31:35.519 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-04-14T08:31:35.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-04-14T08:31:35.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-04-14T08:31:35.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-04-14T08:31:35.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-04-14T08:31:35.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-04-14T08:31:35.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-04-14T08:31:35.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-04-14T08:31:35.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-04-14T08:31:35.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-04-14T08:31:35.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-04-14T08:31:35.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-04-14T08:31:35.520 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-13.14-1.el9.x86_64 71/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 72/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 73/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 74/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-04-14T08:31:35.521 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-04-14T08:31:35.522 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-04-14T08:31:35.522 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-04-14T08:31:35.522 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-04-14T08:31:35.522 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-04-14T08:31:35.522 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-04-14T08:31:35.522 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-04-14T08:31:35.522 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-04-14T08:31:35.522 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-14T08:31:35.523 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-14T08:31:35.524 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-14T08:31:35.524 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-14T08:31:35.524 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-14T08:31:35.524 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-14T08:31:35.524 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-14T08:31:35.524 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-14T08:31:35.524 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-14T08:31:35.524 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-14T08:31:35.524 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-14T08:31:35.525 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-14T08:31:35.526 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-14T08:31:35.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-14T08:31:35.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-14T08:31:35.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-14T08:31:35.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-14T08:31:35.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-14T08:31:35.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-14T08:31:35.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-14T08:31:35.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-14T08:31:35.527 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:19.0.0-2592.gd7e176b4.el9.x86_64 151/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:19.0.0-2592.gd7e176b4.el9.x86_64 153/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-04-14T08:31:35.528 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-04-14T08:31:35.529 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-04-14T08:31:35.530 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-04-14T08:31:35.531 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-04-14T08:31:35.531 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-14T08:31:35.531 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-14T08:31:35.531 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-14T08:31:36.341 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:19.0.0-2592.gd7e176b4.el9.x86_64 148/198 2024-04-14T08:31:36.414 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:19.0.0-2592.gd7e176b4.el9.x86_64 149/198 2024-04-14T08:31:37.286 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-04-14T08:31:37.317 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 151/198 2024-04-14T08:31:37.340 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 151/198 2024-04-14T08:31:37.341 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:37.341 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-14T08:31:37.341 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-14T08:31:37.341 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-14T08:31:37.341 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:37.392 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2592.gd7e17 152/198 2024-04-14T08:31:37.415 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2592.gd7e17 152/198 2024-04-14T08:31:37.415 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:37.415 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-14T08:31:37.415 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:37.460 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 153/198 2024-04-14T08:31:37.483 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 153/198 2024-04-14T08:31:37.483 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:37.483 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-14T08:31:37.483 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-14T08:31:37.483 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-14T08:31:37.483 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:37.786 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 154/198 2024-04-14T08:31:37.809 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 154/198 2024-04-14T08:31:37.809 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T08:31:37.809 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-14T08:31:37.809 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-14T08:31:37.809 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-14T08:31:37.809 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:37.859 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:19.0.0-2592.gd7e176b4.el9.x86_64 155/198 2024-04-14T08:31:37.890 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:19.0.0-2592.gd7e176b4.el9.x86_64 156/198 2024-04-14T08:31:37.909 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-04-14T08:31:38.087 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-shell-2:19.0.0-2592.gd7e176b4.el9.x86_64 158/198 2024-04-14T08:31:38.139 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 159/198 2024-04-14T08:31:38.198 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-14T08:31:38.256 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-14T08:31:38.299 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-14T08:31:38.324 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:19.0.0-2592.gd7e176b4.el9.x86_ 163/198 2024-04-14T08:31:38.364 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-14T08:31:38.393 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-14T08:31:38.459 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-14T08:31:38.518 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-14T08:31:38.584 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-14T08:31:38.681 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-14T08:31:38.681 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:38.681 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2024-04-14T08:31:38.681 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-14T08:31:38.681 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-14T08:31:38.681 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-14T08:31:38.681 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-14T08:31:38.681 INFO:teuthology.orchestra.run.smithi177.stdout: acl-2.3.1-4.el9.x86_64 2024-04-14T08:31:38.681 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-14T08:31:38.682 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: automake-1.16.2-8.el9.noarch 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: bison-3.7.4-5.el9.x86_64 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.683 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:38.684 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-shell-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: flex-2.6.4-9.el9.x86_64 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-14T08:31:38.685 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: jq-1.6-16.el9.x86_64 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-14T08:31:38.686 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-14T08:31:38.687 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.687 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.687 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-14T08:31:38.687 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.687 INFO:teuthology.orchestra.run.smithi177.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-14T08:31:38.687 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-14T08:31:38.687 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-14T08:31:38.687 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-14T08:31:38.687 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-14T08:31:38.687 INFO:teuthology.orchestra.run.smithi177.stdout: lua-5.4.4-4.el9.x86_64 2024-04-14T08:31:38.687 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-14T08:31:38.687 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-13.14-1.el9.x86_64 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-14T08:31:38.688 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-14T08:31:38.689 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-14T08:31:38.690 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-14T08:31:38.691 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-14T08:31:38.691 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-14T08:31:38.691 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-14T08:31:38.691 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-14T08:31:38.691 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-14T08:31:38.691 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-14T08:31:38.691 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-14T08:31:38.691 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply-3.11-14.el9.noarch 2024-04-14T08:31:38.691 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-14T08:31:38.691 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.692 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-14T08:31:38.693 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-14T08:31:38.694 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-14T08:31:38.694 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.694 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.694 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:38.694 INFO:teuthology.orchestra.run.smithi177.stdout: re2-1:20211101-3.el9.x86_64 2024-04-14T08:31:38.694 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-14T08:31:38.694 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-14T08:31:38.694 INFO:teuthology.orchestra.run.smithi177.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-14T08:31:38.694 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-14T08:31:38.694 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-14T08:31:38.695 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-14T08:31:38.695 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:38.695 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-04-14T08:31:39.211 DEBUG:teuthology.parallel:result is None 2024-04-14T08:31:39.226 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-14T08:31:39.273 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-14T08:31:39.294 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-14T08:31:40.110 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-top-2:19.0.0-2592.gd7e176b4.el9.noarch 170/198 2024-04-14T08:31:40.211 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-14T08:31:40.297 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-04-14T08:31:40.331 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-04-14T08:31:40.356 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-04-14T08:31:40.382 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-14T08:31:40.407 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-14T08:31:40.433 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-14T08:31:40.433 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-14T08:31:40.521 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-14T08:31:40.551 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-14T08:31:40.551 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-14T08:31:40.653 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-14T08:31:40.679 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-14T08:31:40.712 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-14T08:31:40.737 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-14T08:31:40.762 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-14T08:31:40.796 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-14T08:31:40.843 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-14T08:31:40.875 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-14T08:31:40.909 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-14T08:31:40.934 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-14T08:31:40.959 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-14T08:31:40.984 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-14T08:31:40.992 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-14T08:31:40.992 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-14T08:31:41.187 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-14T08:31:41.187 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-14T08:31:41.270 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-14T08:31:41.295 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-14T08:31:41.326 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-14T08:31:41.354 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-14T08:31:41.379 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-14T08:31:41.445 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-14T08:31:47.187 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-14T08:31:47.188 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:19.0.0-2592.gd7e176b4.el9.x86_64 1/198 2024-04-14T08:31:47.188 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:19.0.0-2592.gd7e176b4.el9.x86_64 2/198 2024-04-14T08:31:47.188 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:19.0.0-2592.gd7e176b4.el9.x86_64 3/198 2024-04-14T08:31:47.188 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:19.0.0-2592.gd7e176b4.el9.x86_64 4/198 2024-04-14T08:31:47.188 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2592.gd7e17 5/198 2024-04-14T08:31:47.188 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:19.0.0-2592.gd7e176b4.el9.x86_64 6/198 2024-04-14T08:31:47.188 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:19.0.0-2592.gd7e176b4.el9.x86_64 7/198 2024-04-14T08:31:47.188 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:19.0.0-2592.gd7e176b4.el9.x86_64 8/198 2024-04-14T08:31:47.188 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:19.0.0-2592.gd7e176b4.el9.x86_64 9/198 2024-04-14T08:31:47.188 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 10/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:19.0.0-2592.gd7e176b4.el9.x86_64 11/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:19.0.0-2592.gd7e176b4.el9.x86_64 12/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 13/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-shell-2:19.0.0-2592.gd7e176b4.el9.x86_64 14/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:19.0.0-2592.gd7e176b4.el9.x86_ 15/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:19.0.0-2592.gd7e176b4.el9.x86_64 16/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:19.0.0-2592.gd7e176b4.el9.x86_64 17/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:19.0.0-2592.gd7e176b4.el9.x86_6 18/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:19.0.0-2592.gd7e176b4.el9.x86 19/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:19.0.0-2592.gd7e176b4.el9.x86_64 20/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2592.gd7e176b4.el 21/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:19.0.0-2592.gd7e176b4.el9. 22/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:19.0.0-2592.gd7e176b4.el9.x86_6 23/198 2024-04-14T08:31:47.189 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:19.0.0-2592.gd7e176b4.el9.x86_64 24/198 2024-04-14T08:31:47.190 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:19.0.0-2592.gd7e176b4.el9.x86_64 25/198 2024-04-14T08:31:47.190 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 26/198 2024-04-14T08:31:47.190 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:19.0.0-2592.gd7e176b4.el9.x86_64 27/198 2024-04-14T08:31:47.190 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 28/198 2024-04-14T08:31:47.190 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:19.0.0-2592.gd7e176b4.el9.x86_64 29/198 2024-04-14T08:31:47.205 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2592.gd7e176b4. 30/198 2024-04-14T08:31:47.205 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2592.gd7e176b4.el9.noa 31/198 2024-04-14T08:31:47.205 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2592.gd7e176b4.el9.n 32/198 2024-04-14T08:31:47.205 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2592.gd7e 33/198 2024-04-14T08:31:47.205 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2592.gd7e176b4.el 34/198 2024-04-14T08:31:47.205 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2592.gd7e176b4.el9.noarch 35/198 2024-04-14T08:31:47.205 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2592.gd7e176b4.e 36/198 2024-04-14T08:31:47.205 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:19.0.0-2592.gd7e176b4.el9.noarch 37/198 2024-04-14T08:31:47.205 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:19.0.0-2592.gd7e176b4.el9.noarch 38/198 2024-04-14T08:31:47.206 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-top-2:19.0.0-2592.gd7e176b4.el9.noarch 39/198 2024-04-14T08:31:47.206 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-04-14T08:31:47.206 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-04-14T08:31:47.206 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-04-14T08:31:47.206 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-04-14T08:31:47.206 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-04-14T08:31:47.206 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-04-14T08:31:47.206 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-04-14T08:31:47.206 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-04-14T08:31:47.206 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-04-14T08:31:47.206 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-04-14T08:31:47.206 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-04-14T08:31:47.207 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-13.14-1.el9.x86_64 71/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 72/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 73/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 74/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-04-14T08:31:47.208 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-04-14T08:31:47.209 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-04-14T08:31:47.210 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-14T08:31:47.211 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-14T08:31:47.212 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-14T08:31:47.213 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:19.0.0-2592.gd7e176b4.el9.x86_64 151/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:19.0.0-2592.gd7e176b4.el9.x86_64 153/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-14T08:31:47.214 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-04-14T08:31:47.215 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-04-14T08:31:47.216 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-04-14T08:31:47.217 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-04-14T08:31:47.217 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-04-14T08:31:47.217 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-04-14T08:31:47.217 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-04-14T08:31:47.217 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-04-14T08:31:47.217 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-14T08:31:47.217 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-14T08:31:47.217 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-14T08:31:50.397 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-14T08:31:50.397 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:50.398 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2024-04-14T08:31:50.398 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-14T08:31:50.398 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-14T08:31:50.398 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-14T08:31:50.398 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-14T08:31:50.399 INFO:teuthology.orchestra.run.smithi172.stdout: acl-2.3.1-4.el9.x86_64 2024-04-14T08:31:50.399 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-14T08:31:50.399 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-14T08:31:50.399 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-14T08:31:50.399 INFO:teuthology.orchestra.run.smithi172.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-14T08:31:50.399 INFO:teuthology.orchestra.run.smithi172.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-14T08:31:50.399 INFO:teuthology.orchestra.run.smithi172.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-14T08:31:50.399 INFO:teuthology.orchestra.run.smithi172.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-14T08:31:50.399 INFO:teuthology.orchestra.run.smithi172.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-14T08:31:50.399 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-14T08:31:50.400 INFO:teuthology.orchestra.run.smithi172.stdout: automake-1.16.2-8.el9.noarch 2024-04-14T08:31:50.401 INFO:teuthology.orchestra.run.smithi172.stdout: bison-3.7.4-5.el9.x86_64 2024-04-14T08:31:50.401 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-14T08:31:50.401 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.401 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.401 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.401 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.401 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:50.401 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.401 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.401 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.401 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:50.402 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:50.402 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:50.402 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:50.402 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:50.402 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.402 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.402 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:50.402 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.402 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.402 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.402 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:50.402 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:50.403 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.403 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.403 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top-2:19.0.0-2592.gd7e176b4.el9.noarch 2024-04-14T08:31:50.403 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-14T08:31:50.403 INFO:teuthology.orchestra.run.smithi172.stdout: flex-2.6.4-9.el9.x86_64 2024-04-14T08:31:50.403 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-14T08:31:50.403 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-14T08:31:50.403 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-14T08:31:50.403 INFO:teuthology.orchestra.run.smithi172.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-14T08:31:50.403 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-14T08:31:50.403 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-14T08:31:50.403 INFO:teuthology.orchestra.run.smithi172.stdout: jq-1.6-16.el9.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.404 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: lua-5.4.4-4.el9.x86_64 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-14T08:31:50.405 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-14T08:31:50.406 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-14T08:31:50.406 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-13.14-1.el9.x86_64 2024-04-14T08:31:50.406 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-14T08:31:50.406 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-14T08:31:50.406 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-14T08:31:50.406 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-14T08:31:50.406 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-14T08:31:50.406 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-14T08:31:50.406 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-14T08:31:50.406 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-14T08:31:50.407 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-14T08:31:50.407 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-14T08:31:50.407 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.407 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.407 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.407 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-14T08:31:50.407 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-14T08:31:50.407 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-14T08:31:50.407 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-14T08:31:50.407 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-14T08:31:50.408 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-14T08:31:50.408 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-14T08:31:50.408 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-14T08:31:50.408 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-14T08:31:50.408 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-14T08:31:50.408 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-14T08:31:50.408 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-14T08:31:50.408 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-14T08:31:50.408 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-14T08:31:50.408 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-14T08:31:50.408 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-14T08:31:50.408 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-14T08:31:50.409 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-14T08:31:50.409 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-14T08:31:50.409 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-14T08:31:50.409 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-14T08:31:50.409 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-14T08:31:50.409 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-14T08:31:50.409 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-14T08:31:50.409 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-14T08:31:50.409 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-14T08:31:50.409 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-14T08:31:50.409 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-14T08:31:50.410 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-14T08:31:50.410 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-14T08:31:50.410 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply-3.11-14.el9.noarch 2024-04-14T08:31:50.410 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-14T08:31:50.410 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-14T08:31:50.410 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-14T08:31:50.410 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-14T08:31:50.410 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-14T08:31:50.410 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-14T08:31:50.410 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-14T08:31:50.410 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-14T08:31:50.411 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-14T08:31:50.412 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-14T08:31:50.412 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-14T08:31:50.412 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-14T08:31:50.412 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-14T08:31:50.412 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-14T08:31:50.412 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-14T08:31:50.412 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-14T08:31:50.412 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-14T08:31:50.412 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-14T08:31:50.412 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.412 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.412 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:19.0.0-2592.gd7e176b4.el9.x86_64 2024-04-14T08:31:50.413 INFO:teuthology.orchestra.run.smithi172.stdout: re2-1:20211101-3.el9.x86_64 2024-04-14T08:31:50.413 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-14T08:31:50.413 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-14T08:31:50.413 INFO:teuthology.orchestra.run.smithi172.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-14T08:31:50.413 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-14T08:31:50.413 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-14T08:31:50.413 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-14T08:31:50.413 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:50.414 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-04-14T08:31:50.934 DEBUG:teuthology.parallel:result is None 2024-04-14T08:31:50.935 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d7e176b41636f150fe82aaedea82def3e9af0203 2024-04-14T08:31:51.167 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-14T08:31:51.189 INFO:teuthology.orchestra.run.smithi172.stdout:19.0.0-2592.gd7e176b4.el9 2024-04-14T08:31:51.190 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2592.gd7e176b4.el9 2024-04-14T08:31:51.190 INFO:teuthology.task.install:The correct ceph version 19.0.0-2592.gd7e176b4 is installed. 2024-04-14T08:31:51.192 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d7e176b41636f150fe82aaedea82def3e9af0203 2024-04-14T08:31:51.345 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-14T08:31:51.370 INFO:teuthology.orchestra.run.smithi177.stdout:19.0.0-2592.gd7e176b4.el9 2024-04-14T08:31:51.371 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2592.gd7e176b4.el9 2024-04-14T08:31:51.371 INFO:teuthology.task.install:The correct ceph version 19.0.0-2592.gd7e176b4 is installed. 2024-04-14T08:31:51.372 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-14T08:31:51.373 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:31:51.373 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-14T08:31:51.404 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:31:51.404 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-14T08:31:51.443 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-14T08:31:51.444 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:31:51.444 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2024-04-14T08:31:51.472 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-14T08:31:51.540 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:31:51.540 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2024-04-14T08:31:51.570 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-14T08:31:51.639 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-14T08:31:51.640 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:31:51.640 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-14T08:31:51.667 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-14T08:31:51.734 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:31:51.734 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-14T08:31:51.763 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-14T08:31:51.832 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-14T08:31:51.832 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:31:51.832 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2024-04-14T08:31:51.859 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-14T08:31:51.926 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:31:51.926 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2024-04-14T08:31:51.953 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-14T08:31:52.022 INFO:teuthology.run_tasks:Running task ceph... 2024-04-14T08:31:52.116 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-14T08:31:52.116 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /var/log/ceph 2024-04-14T08:31:52.118 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /var/log/ceph 2024-04-14T08:31:52.146 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-14T08:31:52.146 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-14T08:31:52.186 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-14T08:31:52.215 INFO:tasks.ceph:Creating extra log directories... 2024-04-14T08:31:52.215 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-14T08:31:52.252 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-14T08:31:52.288 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-14T08:31:52.288 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-14T08:31:52.288 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-14T08:31:52.289 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:31:52.289 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-14T08:31:52.332 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-14T08:31:52.448 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-14T08:31:52.475 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:31:52.475 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-14T08:31:52.489 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-14T08:31:52.608 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-14T08:31:52.640 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-14T08:31:52.640 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-14T08:31:52.640 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369', 'branch': 'main', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7638369', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'd7e176b41636f150fe82aaedea82def3e9af0203'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd7e176b41636f150fe82aaedea82def3e9af0203'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'b3cf8ca51f5d8edaa7cbce710f0de3a66b6f6be9'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1489, 'sha1': 'd7e176b41636f150fe82aaedea82def3e9af0203', 'sleep_before_teardown': 0, 'subset': '22/512', 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b3cf8ca51f5d8edaa7cbce710f0de3a66b6f6be9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b3cf8ca51f5d8edaa7cbce710f0de3a66b6f6be9', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi172.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLf2A6OlxKixkzXPIy/I95aPoML/ifPAeBpNg+k0+A/BYkM6XAvMsOlkDx7+/5WvDC3MQjg0rXwnu5LDckBmdq0=', 'smithi177.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF3YkGOI+hKcHUDd0JnNpKEdi2eUoiseAifR6sFT8/i8WM/z1HIrPCzlwlYhh05ID4cTD0ylLad0oCa3thhmXfQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd7e176b41636f150fe82aaedea82def3e9af0203'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'd7e176b41636f150fe82aaedea82def3e9af0203', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'd7d6e60867cadf19fbfd7655a551ae7d58060757', 'timestamp': '2024-04-03_20:24:15', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-14T08:31:52.641 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-14T08:31:52.644 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-14T08:31:52.691 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/run/ceph 2024-04-14T08:31:52.698 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/run/ceph 2024-04-14T08:31:52.758 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:31:52.758 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2024-04-14T08:31:52.778 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-14T08:31:52.779 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2024-04-14T08:31:52.834 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-14T08:31:52.834 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T08:31:52.834 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-04-14T08:31:52.834 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T08:31:52.834 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T08:31:52.834 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-14 08:31:41.804909439 +0000 2024-04-14T08:31:52.834 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-14 08:29:13.294197703 +0000 2024-04-14T08:31:52.835 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-14 08:29:13.294197703 +0000 2024-04-14T08:31:52.835 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-14 08:29:13.294197703 +0000 2024-04-14T08:31:52.835 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-14T08:31:52.899 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-14T08:31:52.899 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-14T08:31:52.899 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000242816 s, 2.1 MB/s 2024-04-14T08:31:52.900 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-14T08:31:52.956 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2024-04-14T08:31:53.010 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-14T08:31:53.011 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T08:31:53.011 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-14T08:31:53.011 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T08:31:53.011 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T08:31:53.011 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-14 08:31:41.805909432 +0000 2024-04-14T08:31:53.011 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-14 08:29:13.635195505 +0000 2024-04-14T08:31:53.011 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-14 08:29:13.635195505 +0000 2024-04-14T08:31:53.011 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-14 08:29:13.635195505 +0000 2024-04-14T08:31:53.011 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-14T08:31:53.076 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-14T08:31:53.076 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-14T08:31:53.076 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000241457 s, 2.1 MB/s 2024-04-14T08:31:53.078 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-14T08:31:53.135 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2024-04-14T08:31:53.190 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-14T08:31:53.190 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T08:31:53.190 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-04-14T08:31:53.190 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T08:31:53.191 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T08:31:53.191 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-14 08:31:41.805909432 +0000 2024-04-14T08:31:53.191 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-14 08:29:13.985193250 +0000 2024-04-14T08:31:53.191 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-14 08:29:13.985193250 +0000 2024-04-14T08:31:53.191 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-14 08:29:13.985193250 +0000 2024-04-14T08:31:53.191 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-14T08:31:53.255 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-14T08:31:53.255 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-14T08:31:53.255 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000190349 s, 2.7 MB/s 2024-04-14T08:31:53.256 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-14T08:31:53.313 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2024-04-14T08:31:53.367 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-14T08:31:53.368 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T08:31:53.368 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-14T08:31:53.368 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T08:31:53.368 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T08:31:53.368 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-14 08:31:41.806909425 +0000 2024-04-14T08:31:53.368 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-14 08:29:14.324191065 +0000 2024-04-14T08:31:53.368 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-14 08:29:14.324191065 +0000 2024-04-14T08:31:53.368 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-14 08:29:14.324191065 +0000 2024-04-14T08:31:53.368 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-14T08:31:53.433 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-14T08:31:53.434 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-14T08:31:53.434 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000229552 s, 2.2 MB/s 2024-04-14T08:31:53.435 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-14T08:31:53.490 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-14T08:31:53.491 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:31:53.491 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2024-04-14T08:31:53.507 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-14T08:31:53.507 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2024-04-14T08:31:53.561 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-14T08:31:53.562 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T08:31:53.562 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-04-14T08:31:53.562 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T08:31:53.562 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T08:31:53.562 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-04-14 08:31:30.365191301 +0000 2024-04-14T08:31:53.562 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-04-14 08:29:04.124596476 +0000 2024-04-14T08:31:53.562 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-04-14 08:29:04.124596476 +0000 2024-04-14T08:31:53.562 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-04-14 08:29:04.124596476 +0000 2024-04-14T08:31:53.563 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-14T08:31:53.627 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-04-14T08:31:53.628 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-04-14T08:31:53.628 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000178976 s, 2.9 MB/s 2024-04-14T08:31:53.629 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-14T08:31:53.684 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2024-04-14T08:31:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-14T08:31:53.740 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T08:31:53.740 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-14T08:31:53.740 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T08:31:53.740 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T08:31:53.740 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-04-14 08:31:30.366191274 +0000 2024-04-14T08:31:53.741 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-04-14 08:29:04.468586934 +0000 2024-04-14T08:31:53.741 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-04-14 08:29:04.468586934 +0000 2024-04-14T08:31:53.741 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-04-14 08:29:04.468586934 +0000 2024-04-14T08:31:53.741 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-14T08:31:53.805 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-04-14T08:31:53.805 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-04-14T08:31:53.805 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000220893 s, 2.3 MB/s 2024-04-14T08:31:53.807 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-14T08:31:53.864 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2024-04-14T08:31:53.919 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-14T08:31:53.919 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T08:31:53.919 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-14T08:31:53.919 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T08:31:53.919 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T08:31:53.919 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-04-14 08:31:30.366191274 +0000 2024-04-14T08:31:53.919 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-04-14 08:29:04.811577419 +0000 2024-04-14T08:31:53.919 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-04-14 08:29:04.811577419 +0000 2024-04-14T08:31:53.919 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-04-14 08:29:04.811577419 +0000 2024-04-14T08:31:53.920 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-14T08:31:53.984 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-04-14T08:31:53.984 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-04-14T08:31:53.984 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000208937 s, 2.5 MB/s 2024-04-14T08:31:53.985 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-14T08:31:54.041 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2024-04-14T08:31:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-14T08:31:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T08:31:54.096 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-14T08:31:54.096 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T08:31:54.096 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T08:31:54.096 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-04-14 08:31:30.366191274 +0000 2024-04-14T08:31:54.096 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-04-14 08:29:05.148568071 +0000 2024-04-14T08:31:54.096 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-04-14 08:29:05.148568071 +0000 2024-04-14T08:31:54.096 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-04-14 08:29:05.148568071 +0000 2024-04-14T08:31:54.096 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-14T08:31:54.161 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-04-14T08:31:54.161 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-04-14T08:31:54.161 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000227048 s, 2.3 MB/s 2024-04-14T08:31:54.162 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-14T08:31:54.218 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-14T08:31:54.218 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-14T08:31:54.218 INFO:tasks.ceph:Generating config... 2024-04-14T08:31:54.219 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-14T08:31:54.220 INFO:tasks.ceph:[client] debug client = 20 2024-04-14T08:31:54.220 INFO:tasks.ceph:[client] debug ms = 1 2024-04-14T08:31:54.220 INFO:tasks.ceph:[client] pid file = 2024-04-14T08:31:54.220 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-14T08:31:54.220 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-14T08:31:54.220 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-14T08:31:54.220 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-14T08:31:54.220 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-14T08:31:54.220 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-14T08:31:54.220 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-14T08:31:54.220 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-14T08:31:54.220 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-14T08:31:54.220 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-14T08:31:54.221 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-14T08:31:54.221 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-14T08:31:54.221 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-14T08:31:54.221 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-14T08:31:54.221 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-14T08:31:54.221 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-14T08:31:54.221 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-14T08:31:54.221 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-14T08:31:54.221 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-14T08:31:54.221 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-14T08:31:54.221 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-14T08:31:54.221 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-14T08:31:54.221 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-14T08:31:54.221 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-14T08:31:54.221 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-14T08:31:54.222 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-14T08:31:54.222 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-14T08:31:54.222 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-14T08:31:54.222 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-14T08:31:54.222 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-14T08:31:54.222 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-14T08:31:54.222 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-14T08:31:54.222 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-14T08:31:54.222 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-14T08:31:54.222 INFO:tasks.ceph:Setting up mon.a... 2024-04-14T08:31:54.222 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-14T08:31:54.268 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.keyring 2024-04-14T08:31:54.270 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-14T08:31:54.315 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-14T08:31:54.382 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.172'), ('mon.b', '172.21.15.177'), ('mon.c', '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]')] 2024-04-14T08:31:54.383 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.172,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-14T08:31:54.383 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:31:54.384 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-14T08:31:54.437 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.172 --add b 172.21.15.177 --addv c '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:31:54.522 INFO:teuthology.orchestra.run.smithi172.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:31:54.522 INFO:teuthology.orchestra.run.smithi172.stdout:monmaptool: generated fsid 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:31:54.522 INFO:teuthology.orchestra.run.smithi172.stdout:setting min_mon_release = quincy 2024-04-14T08:31:54.522 INFO:teuthology.orchestra.run.smithi172.stdout:epoch 0 2024-04-14T08:31:54.522 INFO:teuthology.orchestra.run.smithi172.stdout:fsid 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:31:54.522 INFO:teuthology.orchestra.run.smithi172.stdout:last_changed 2024-04-14T08:31:54.521994+0000 2024-04-14T08:31:54.522 INFO:teuthology.orchestra.run.smithi172.stdout:created 2024-04-14T08:31:54.521994+0000 2024-04-14T08:31:54.522 INFO:teuthology.orchestra.run.smithi172.stdout:min_mon_release 17 (quincy) 2024-04-14T08:31:54.522 INFO:teuthology.orchestra.run.smithi172.stdout:election_strategy: 1 2024-04-14T08:31:54.522 INFO:teuthology.orchestra.run.smithi172.stdout:0: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.a 2024-04-14T08:31:54.523 INFO:teuthology.orchestra.run.smithi172.stdout:1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2024-04-14T08:31:54.523 INFO:teuthology.orchestra.run.smithi172.stdout:2: [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] mon.c 2024-04-14T08:31:54.523 INFO:teuthology.orchestra.run.smithi172.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-14T08:31:54.524 DEBUG:teuthology.orchestra.run.smithi172:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-14T08:31:54.538 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8e7e3a2f-e38f-4857-b139-5df1f7a099dc... 2024-04-14T08:31:54.539 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-14T08:31:54.580 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-14T08:31:54.629 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-04-14T08:31:54.629 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2024-04-14T08:31:54.629 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-14T08:31:54.629 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2024-04-14T08:31:54.629 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.629 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2024-04-14T08:31:54.629 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.629 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2024-04-14T08:31:54.629 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.629 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2024-04-14T08:31:54.629 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.630 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.631 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.172,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2024-04-14T08:31:54.632 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2024-04-14T08:31:54.633 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.634 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2024-04-14T08:31:54.634 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.634 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2024-04-14T08:31:54.634 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.634 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2024-04-14T08:31:54.634 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.634 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2024-04-14T08:31:54.634 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2024-04-14T08:31:54.634 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.635 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2024-04-14T08:31:54.635 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2024-04-14T08:31:54.635 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2024-04-14T08:31:54.635 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2024-04-14T08:31:54.635 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 20 2024-04-14T08:31:54.635 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 20 2024-04-14T08:31:54.635 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:31:54.635 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2024-04-14T08:31:54.635 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 10 2024-04-14T08:31:54.635 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2024-04-14T08:31:54.635 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2024-04-14T08:31:54.635 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-14T08:31:54.636 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.637 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: mon down mkfs grace = 300 2024-04-14T08:31:54.638 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2024-04-14T08:31:54.639 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = True 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = True 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-14T08:31:54.640 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-14T08:31:54.641 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-14T08:31:54.641 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2024-04-14T08:31:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2024-04-14T08:31:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-14T08:31:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2024-04-14T08:31:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.641 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2024-04-14T08:31:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2024-04-14T08:31:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2024-04-14T08:31:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2024-04-14T08:31:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2024-04-14T08:31:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2024-04-14T08:31:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.642 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2024-04-14T08:31:54.643 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2024-04-14T08:31:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-14T08:31:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-14T08:31:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2024-04-14T08:31:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-14T08:31:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2024-04-14T08:31:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2024-04-14T08:31:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2024-04-14T08:31:54.644 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2024-04-14T08:31:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2024-04-14T08:31:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.172,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2024-04-14T08:31:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on skipped message = False 2024-04-14T08:31:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:31:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.645 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2024-04-14T08:31:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2024-04-14T08:31:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2024-04-14T08:31:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2024-04-14T08:31:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2024-04-14T08:31:54.645 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug trim objects = true 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.646 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 20 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 20 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:31:54.647 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 10 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-04-14T08:31:54.648 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2024-04-14T08:31:54.649 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2024-04-14T08:31:54.650 INFO:teuthology.orchestra.run.smithi177.stdout: mon down mkfs grace = 300 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: mon op complaint time = 120 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 600 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = "" 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2024-04-14T08:31:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2024-04-14T08:31:54.652 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2024-04-14T08:31:54.652 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2024-04-14T08:31:54.652 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2024-04-14T08:31:54.652 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2024-04-14T08:31:54.652 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2024-04-14T08:31:54.652 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2024-04-14T08:31:54.652 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2024-04-14T08:31:54.652 INFO:teuthology.orchestra.run.smithi177.stdout:[mds] 2024-04-14T08:31:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds = 20 2024-04-14T08:31:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds balancer = 20 2024-04-14T08:31:54.652 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:31:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug frag = True 2024-04-14T08:31:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug scatterstat = True 2024-04-14T08:31:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: mds op complaint time = 180 2024-04-14T08:31:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: mds verify scatter = True 2024-04-14T08:31:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2024-04-14T08:31:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2024-04-14T08:31:54.653 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2024-04-14T08:31:54.655 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-14T08:31:54.655 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-14T08:31:54.751 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-14T08:31:54.751 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:31:54.751 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-14T08:31:54.766 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:31:54.766 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-14T08:31:54.821 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:31:54.821 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:31:54.821 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-14T08:31:54.821 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-14T08:31:54.900 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:31:54.900 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:31:54.955 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:31:54.956 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:31:54.956 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-14T08:31:54.956 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-14T08:31:54.996 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:31:54.996 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:31:55.050 INFO:tasks.ceph:Setting up mon nodes... 2024-04-14T08:31:55.051 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-14T08:31:55.051 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-14T08:31:55.106 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-14T08:31:55.108 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-14T08:31:55.169 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-14T08:31:55.172 INFO:tasks.ceph:Setting up mds nodes... 2024-04-14T08:31:55.172 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-14T08:31:55.227 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-14T08:31:55.229 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-14T08:31:55.256 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-14T08:31:55.351 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-14T08:31:55.353 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-14T08:31:55.380 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-14T08:31:55.435 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-14T08:31:55.438 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-14T08:31:55.466 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-14T08:31:55.561 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-14T08:31:55.563 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-14T08:31:55.589 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-14T08:31:55.590 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-14T08:31:55.632 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-14T08:31:55.647 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-14T08:31:55.729 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-14T08:31:55.744 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-14T08:31:55.786 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-14T08:31:55.801 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-14T08:31:55.883 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-14T08:31:55.898 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-14T08:31:55.898 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-14T08:31:55.898 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-14T08:31:55.925 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-14T08:31:55.925 INFO:tasks.ceph:role: osd.0 2024-04-14T08:31:55.925 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:31:55.925 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-14T08:31:55.999 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-14T08:31:55.999 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-14T08:31:55.999 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-14T08:31:55.999 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-14T08:31:55.999 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-14T08:31:55.999 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-04-14T08:31:56.000 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-14T08:31:56.000 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-14T08:31:56.000 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-14T08:31:56.000 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-14T08:31:56.509 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-04-14T08:31:56.511 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-04-14T08:31:56.511 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-14T08:31:56.661 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-14T08:31:56.697 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-14T08:31:56.766 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-14T08:31:56.766 INFO:tasks.ceph:role: osd.1 2024-04-14T08:31:56.766 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:31:56.766 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-14T08:31:56.841 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-14T08:31:56.841 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-14T08:31:56.841 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-14T08:31:56.841 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-14T08:31:56.841 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-14T08:31:56.842 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-04-14T08:31:56.842 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-14T08:31:56.842 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-14T08:31:56.842 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-14T08:31:56.842 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-14T08:31:57.289 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-04-14T08:31:57.291 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-04-14T08:31:57.291 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-14T08:31:57.379 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-14T08:31:57.413 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-14T08:31:57.481 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-14T08:31:57.481 INFO:tasks.ceph:role: osd.2 2024-04-14T08:31:57.481 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:31:57.481 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-14T08:31:57.556 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-14T08:31:57.556 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-14T08:31:57.556 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-14T08:31:57.556 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-14T08:31:57.557 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-14T08:31:57.557 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-04-14T08:31:57.557 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-14T08:31:57.557 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-14T08:31:57.557 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-14T08:31:57.557 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-14T08:31:58.047 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-04-14T08:31:58.048 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-04-14T08:31:58.049 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-14T08:31:58.208 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-14T08:31:58.240 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-14T08:31:58.307 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-14T08:31:58.307 INFO:tasks.ceph:role: osd.3 2024-04-14T08:31:58.307 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:31:58.307 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-14T08:31:58.382 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-14T08:31:58.382 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-14T08:31:58.382 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-14T08:31:58.382 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-14T08:31:58.382 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-14T08:31:58.382 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-04-14T08:31:58.382 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-14T08:31:58.383 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-14T08:31:58.383 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-14T08:31:58.383 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-14T08:31:58.879 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-04-14T08:31:58.881 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-04-14T08:31:58.882 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-14T08:31:59.028 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-14T08:31:59.064 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:31:59.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:31:59.160+0000 7f2120993640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-14T08:31:59.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:31:59.161+0000 7f2120993640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-14T08:31:59.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:31:59.161+0000 7f2120993640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-14T08:31:59.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:31:59.161+0000 7f2120993640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-14T08:31:59.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:31:59.161+0000 7f2120993640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-14T08:31:59.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:31:59.161+0000 7f2120993640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-14T08:32:02.006 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-14T08:32:02.034 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:32:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:02.121+0000 7f63bfd36640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-14T08:32:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:02.121+0000 7f63bfd36640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-14T08:32:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:02.121+0000 7f63bfd36640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-14T08:32:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:02.121+0000 7f63bfd36640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-14T08:32:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:02.121+0000 7f63bfd36640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-14T08:32:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:02.121+0000 7f63bfd36640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-14T08:32:04.970 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-14T08:32:05.002 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:32:05.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:05.092+0000 7f0c06d21640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-14T08:32:05.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:05.092+0000 7f0c06d21640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-14T08:32:05.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:05.092+0000 7f0c06d21640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-14T08:32:05.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:05.092+0000 7f0c06d21640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-14T08:32:05.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:05.092+0000 7f0c06d21640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-14T08:32:05.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:05.093+0000 7f0c06d21640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-14T08:32:07.952 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-14T08:32:07.981 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:32:08.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:08.068+0000 7f647a40d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-14T08:32:08.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:08.068+0000 7f647a40d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-14T08:32:08.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:08.068+0000 7f647a40d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-14T08:32:08.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:08.068+0000 7f647a40d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-14T08:32:08.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:08.068+0000 7f647a40d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-14T08:32:08.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:08.069+0000 7f647a40d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-14T08:32:10.913 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-14T08:32:10.944 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-14T08:32:10.976 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-14T08:32:10.976 INFO:tasks.ceph:role: osd.4 2024-04-14T08:32:10.977 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:32:10.977 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-14T08:32:11.054 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-14T08:32:11.055 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-14T08:32:11.055 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-14T08:32:11.055 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-14T08:32:11.055 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-14T08:32:11.055 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-04-14T08:32:11.055 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-14T08:32:11.055 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-14T08:32:11.055 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-14T08:32:11.055 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-14T08:32:11.556 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-04-14T08:32:11.559 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-04-14T08:32:11.559 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-14T08:32:11.668 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-14T08:32:11.705 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-14T08:32:11.775 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-14T08:32:11.775 INFO:tasks.ceph:role: osd.5 2024-04-14T08:32:11.775 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:32:11.775 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-14T08:32:11.851 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-14T08:32:11.851 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-14T08:32:11.851 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-14T08:32:11.851 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-14T08:32:11.851 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-14T08:32:11.851 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-04-14T08:32:11.851 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-14T08:32:11.851 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-14T08:32:11.851 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-14T08:32:11.851 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-14T08:32:12.302 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-04-14T08:32:12.304 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-04-14T08:32:12.304 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-14T08:32:12.397 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-14T08:32:12.430 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-14T08:32:12.498 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-14T08:32:12.498 INFO:tasks.ceph:role: osd.6 2024-04-14T08:32:12.499 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:32:12.499 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-14T08:32:12.574 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-14T08:32:12.574 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-14T08:32:12.575 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-14T08:32:12.575 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-14T08:32:12.575 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-14T08:32:12.575 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-04-14T08:32:12.575 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-14T08:32:12.575 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-14T08:32:12.575 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-14T08:32:12.575 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-14T08:32:13.053 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-04-14T08:32:13.055 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-04-14T08:32:13.055 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-14T08:32:13.209 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-14T08:32:13.242 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-14T08:32:13.312 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-14T08:32:13.312 INFO:tasks.ceph:role: osd.7 2024-04-14T08:32:13.312 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:32:13.312 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-14T08:32:13.387 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-14T08:32:13.387 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-14T08:32:13.388 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-14T08:32:13.388 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-14T08:32:13.388 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-14T08:32:13.388 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2024-04-14T08:32:13.388 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-14T08:32:13.388 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-14T08:32:13.388 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-14T08:32:13.388 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-14T08:32:13.896 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2024-04-14T08:32:13.898 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2024-04-14T08:32:13.898 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-14T08:32:14.044 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-14T08:32:14.081 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:32:14.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:14.182+0000 7f3959e11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-14T08:32:14.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:14.183+0000 7f3959e11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-14T08:32:14.184 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:14.183+0000 7f3959e11640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-14T08:32:14.185 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:14.183+0000 7f3959e11640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-14T08:32:14.185 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:14.183+0000 7f3959e11640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-14T08:32:14.185 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:14.183+0000 7f3959e11640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-14T08:32:17.047 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-14T08:32:17.078 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:32:17.169 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:17.167+0000 7fc68f4c1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-14T08:32:17.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:17.168+0000 7fc68f4c1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-14T08:32:17.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:17.168+0000 7fc68f4c1640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-14T08:32:17.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:17.168+0000 7fc68f4c1640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-14T08:32:17.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:17.168+0000 7fc68f4c1640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-14T08:32:17.170 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:17.168+0000 7fc68f4c1640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-14T08:32:20.015 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-14T08:32:20.044 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:32:20.134 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:20.132+0000 7ffba1a01640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-14T08:32:20.134 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:20.132+0000 7ffba1a01640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-14T08:32:20.134 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:20.132+0000 7ffba1a01640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-14T08:32:20.134 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:20.132+0000 7ffba1a01640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-14T08:32:20.134 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:20.132+0000 7ffba1a01640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-14T08:32:20.134 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:20.133+0000 7ffba1a01640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-14T08:32:22.642 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:32:22.646 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:32:22.980 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-14T08:32:23.010 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:32:23.098 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:23.097+0000 7fcf43193640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-14T08:32:23.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:23.097+0000 7fcf43193640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-14T08:32:23.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:23.097+0000 7fcf43193640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-14T08:32:23.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:23.097+0000 7fcf43193640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-14T08:32:23.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:23.097+0000 7fcf43193640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-14T08:32:23.099 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:23.098+0000 7fcf43193640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-14T08:32:25.943 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-14T08:32:25.971 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-14T08:32:25.971 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:25.971 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-14T08:32:26.003 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:26.004 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-14T08:32:26.073 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:26.074 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-14T08:32:26.143 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:26.143 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-14T08:32:26.212 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:26.212 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-14T08:32:26.281 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:26.281 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-14T08:32:26.349 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:26.349 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-14T08:32:26.417 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:26.417 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-14T08:32:26.445 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:26.446 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-14T08:32:26.515 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:26.515 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-14T08:32:26.583 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:26.583 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-14T08:32:26.651 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:26.651 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-14T08:32:26.718 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:26.718 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-14T08:32:26.785 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:26.786 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-14T08:32:26.853 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:26.853 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-14T08:32:26.870 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:26.870 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-14T08:32:26.927 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:26.927 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-14T08:32:26.942 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:26.943 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-14T08:32:27.000 INFO:tasks.ceph:Adding keys to all mons... 2024-04-14T08:32:27.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-14T08:32:27.003 DEBUG:teuthology.orchestra.run.smithi177:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-14T08:32:27.044 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.y] 2024-04-14T08:32:27.044 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmfddYBhAAh8zgf4W3WWmNYWNAA7WuRw== 2024-04-14T08:32:27.044 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-04-14T08:32:27.044 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmnNCRDRAAz7aU3JD7iiaTQH0cNotfaw== 2024-04-14T08:32:27.044 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-04-14T08:32:27.044 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmb7z0FBAAGVu3+TWshDGvd7/fLbgJ5A== 2024-04-14T08:32:27.044 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.0] 2024-04-14T08:32:27.044 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB/lBtmnsWuCRAAqcuQhsrzpGVfrlgwTnFsOw== 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.1] 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCClBtme9lPBxAAFB1E3g8qwZxPCdHiFogp5Q== 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.2] 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCFlBtm966YBRAAWXZGq/0TuUMZTsyeR4xWww== 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.3] 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCIlBtmHpktBBAAUbddNHESwyNvvp8UE/SaCw== 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.x] 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmm5IbChAAeVcVs3Zt6+lU0wghjCcwnQ== 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtm1on4GRAAms1he+O4jWu01z0ZLLgL0A== 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmEEx0IRAAXiFiOK1RE/0bfyu7KI/3ng== 2024-04-14T08:32:27.045 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.4] 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCOlBtm+//7ChAA7rApbKll2vBGcDlW8q+kyg== 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.5] 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCRlBtmVU4YChAAysdUosQ+TnHtjnRl1FOISA== 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.6] 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCUlBtmnE7+BxAAqBn8YMwecA2NcZz63eBm7Q== 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.7] 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQCXlBtmN8DhBRAA6CHQ943t6qJCL77jcbOv4w== 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA== 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout:[client.2] 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmJoV9KxAAp6zlRX+gvhASQBHjpF9ewQ== 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout:[client.1] 2024-04-14T08:32:27.046 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmp4nkLhAAhYKm8VcykVbPSohvK11I9w== 2024-04-14T08:32:27.047 INFO:teuthology.orchestra.run.smithi172.stdout:[client.3] 2024-04-14T08:32:27.047 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmeC+pNBAASi7W4jlo6imMMzNEwLrPDA== 2024-04-14T08:32:27.066 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.y] 2024-04-14T08:32:27.066 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB7lBtmfddYBhAAh8zgf4W3WWmNYWNAA7WuRw== 2024-04-14T08:32:27.066 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB7lBtmnNCRDRAAz7aU3JD7iiaTQH0cNotfaw== 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB7lBtmb7z0FBAAGVu3+TWshDGvd7/fLbgJ5A== 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.0] 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB/lBtmnsWuCRAAqcuQhsrzpGVfrlgwTnFsOw== 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.1] 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCClBtme9lPBxAAFB1E3g8qwZxPCdHiFogp5Q== 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.2] 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCFlBtm966YBRAAWXZGq/0TuUMZTsyeR4xWww== 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.3] 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCIlBtmHpktBBAAUbddNHESwyNvvp8UE/SaCw== 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.x] 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB7lBtmm5IbChAAeVcVs3Zt6+lU0wghjCcwnQ== 2024-04-14T08:32:27.067 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB7lBtm1on4GRAAms1he+O4jWu01z0ZLLgL0A== 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB7lBtmEEx0IRAAXiFiOK1RE/0bfyu7KI/3ng== 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.4] 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCOlBtm+//7ChAA7rApbKll2vBGcDlW8q+kyg== 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.5] 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCRlBtmVU4YChAAysdUosQ+TnHtjnRl1FOISA== 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.6] 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCUlBtmnE7+BxAAqBn8YMwecA2NcZz63eBm7Q== 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.7] 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCXlBtmN8DhBRAA6CHQ943t6qJCL77jcbOv4w== 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout:[client.0] 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA== 2024-04-14T08:32:27.068 INFO:teuthology.orchestra.run.smithi177.stdout:[client.2] 2024-04-14T08:32:27.069 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB7lBtmJoV9KxAAp6zlRX+gvhASQBHjpF9ewQ== 2024-04-14T08:32:27.069 INFO:teuthology.orchestra.run.smithi177.stdout:[client.1] 2024-04-14T08:32:27.069 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB7lBtmp4nkLhAAhYKm8VcykVbPSohvK11I9w== 2024-04-14T08:32:27.069 INFO:teuthology.orchestra.run.smithi177.stdout:[client.3] 2024-04-14T08:32:27.069 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB7lBtmeC+pNBAASi7W4jlo6imMMzNEwLrPDA== 2024-04-14T08:32:27.070 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-14T08:32:27.091 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-14T08:32:27.162 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-14T08:32:27.165 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-14T08:32:27.220 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-14T08:32:27.222 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-14T08:32:27.277 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.279 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.331 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.333 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.386 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.389 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.440 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.442 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.495 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-14T08:32:27.498 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-14T08:32:27.549 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-14T08:32:27.552 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-14T08:32:27.608 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-14T08:32:27.611 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-14T08:32:27.666 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.668 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.720 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.722 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.773 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.776 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.827 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.829 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-14T08:32:27.881 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-14T08:32:27.885 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-14T08:32:27.941 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-14T08:32:27.943 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-14T08:32:27.998 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-14T08:32:28.000 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-14T08:32:28.055 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-14T08:32:28.057 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-14T08:32:28.112 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-14T08:32:28.112 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-14T08:32:28.139 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-14T08:32:28.237 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-14T08:32:28.264 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-14T08:32:28.291 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-14T08:32:28.389 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-14T08:32:28.415 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-14T08:32:28.483 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-14T08:32:28.581 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-14T08:32:28.608 DEBUG:teuthology.orchestra.run.smithi172:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:32:28.610 DEBUG:teuthology.orchestra.run.smithi177:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-14T08:32:28.663 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-14T08:32:28.663 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-14T08:32:28.664 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-14T08:32:28.666 INFO:tasks.ceph.mon.a:Started 2024-04-14T08:32:28.666 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-14T08:32:28.666 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-14T08:32:28.705 INFO:tasks.ceph.mon.b:Started 2024-04-14T08:32:28.705 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-14T08:32:28.706 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-14T08:32:28.707 INFO:tasks.ceph.mon.c:Started 2024-04-14T08:32:28.708 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-14T08:32:28.708 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-14T08:32:28.708 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-14T08:32:28.711 INFO:tasks.ceph.mgr.y:Started 2024-04-14T08:32:28.711 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-14T08:32:28.711 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-14T08:32:28.713 INFO:tasks.ceph.mgr.x:Started 2024-04-14T08:32:28.713 INFO:tasks.ceph:Setting crush tunables to default 2024-04-14T08:32:28.713 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd crush tunables default 2024-04-14T08:32:28.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.814+0000 7f1de331e640 1 Processor -- start 2024-04-14T08:32:28.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.814+0000 7f1de331e640 1 -- start start 2024-04-14T08:32:28.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.814+0000 7f1de331e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ddc15b2c0 0x7f1ddc0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:28.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.814+0000 7f1de331e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1ddc0cb780 0x7f1ddc0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.814+0000 7f1de331e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 0x7f1ddc0722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.814+0000 7f1de331e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f1ddc0cfae0 con 0x7f1ddc15b2c0 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.814+0000 7f1de331e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f1ddc0cfdc0 con 0x7f1ddc0cb780 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.814+0000 7f1de331e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1ddc0d00a0 con 0x7f1ddc0cc3c0 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.814+0000 7f1de10ba640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ddc15b2c0 0x7f1ddc0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de10ba640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ddc15b2c0 0x7f1ddc0cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54878/0 (socket says 172.21.15.172:54878) 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de10ba640 1 -- 172.21.15.172:0/1927257859 learned_addr learned my addr 172.21.15.172:0/1927257859 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de18bb640 1 --2- 172.21.15.172:0/1927257859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 0x7f1ddc0722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de08b9640 1 --2- 172.21.15.172:0/1927257859 >> v2:172.21.15.177:3300/0 conn(0x7f1ddc0cb780 0x7f1ddc0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de10ba640 1 -- 172.21.15.172:0/1927257859 >> v2:172.21.15.172:3300/0 conn(0x7f1ddc15b2c0 msgr2=0x7f1ddc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de10ba640 1 -- 172.21.15.172:0/1927257859 >> v2:172.21.15.172:3300/0 conn(0x7f1ddc15b2c0 msgr2=0x7f1ddc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de10ba640 1 --2- 172.21.15.172:0/1927257859 >> v2:172.21.15.172:3300/0 conn(0x7f1ddc15b2c0 0x7f1ddc0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-14T08:32:28.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de10ba640 1 --2- 172.21.15.172:0/1927257859 >> v2:172.21.15.172:3300/0 conn(0x7f1ddc15b2c0 0x7f1ddc0cf4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-14T08:32:28.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de18bb640 1 -- 172.21.15.172:0/1927257859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 msgr2=0x7f1ddc0722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-14T08:32:28.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de18bb640 1 -- 172.21.15.172:0/1927257859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 msgr2=0x7f1ddc0722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-14T08:32:28.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de18bb640 1 --2- 172.21.15.172:0/1927257859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 0x7f1ddc0722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-14T08:32:28.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de08b9640 1 -- 172.21.15.172:0/1927257859 >> v2:172.21.15.177:3300/0 conn(0x7f1ddc0cb780 msgr2=0x7f1ddc0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-14T08:32:28.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de08b9640 1 -- 172.21.15.172:0/1927257859 >> v2:172.21.15.177:3300/0 conn(0x7f1ddc0cb780 msgr2=0x7f1ddc0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-14T08:32:28.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de08b9640 1 --2- 172.21.15.172:0/1927257859 >> v2:172.21.15.177:3300/0 conn(0x7f1ddc0cb780 0x7f1ddc0cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-14T08:32:28.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de08b9640 1 --2- 172.21.15.172:0/1927257859 >> v2:172.21.15.177:3300/0 conn(0x7f1ddc0cb780 0x7f1ddc0cbe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-14T08:32:28.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:28.815+0000 7f1de18bb640 1 --2- 172.21.15.172:0/1927257859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 0x7f1ddc0722f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-14T08:32:29.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.016+0000 7f1de10ba640 1 --2- 172.21.15.172:0/1927257859 >> v2:172.21.15.172:3300/0 conn(0x7f1ddc15b2c0 0x7f1ddc0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.016+0000 7f1de08b9640 1 --2- 172.21.15.172:0/1927257859 >> v2:172.21.15.177:3300/0 conn(0x7f1ddc0cb780 0x7f1ddc0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.016+0000 7f1de18bb640 1 --2- 172.21.15.172:0/1927257859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 0x7f1ddc0722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.016+0000 7f1de10ba640 1 -- 172.21.15.172:0/1927257859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 msgr2=0x7f1ddc0722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:29.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.016+0000 7f1de10ba640 1 --2- 172.21.15.172:0/1927257859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 0x7f1ddc0722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.016+0000 7f1de10ba640 1 -- 172.21.15.172:0/1927257859 >> v2:172.21.15.177:3300/0 conn(0x7f1ddc0cb780 msgr2=0x7f1ddc0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:29.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.016+0000 7f1de10ba640 1 --2- 172.21.15.172:0/1927257859 >> v2:172.21.15.177:3300/0 conn(0x7f1ddc0cb780 0x7f1ddc0cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.016+0000 7f1de10ba640 1 -- 172.21.15.172:0/1927257859 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ddc0d0380 con 0x7f1ddc15b2c0 2024-04-14T08:32:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.017+0000 7f1de10ba640 1 --2- 172.21.15.172:0/1927257859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ddc15b2c0 0x7f1ddc0cf4d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0c4730 tx=0x7f1dcc02bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb34d1e204e5b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.017+0000 7f1dd3fff640 1 -- 172.21.15.172:0/1927257859 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc039070 con 0x7f1ddc15b2c0 2024-04-14T08:32:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.017+0000 7f1dd3fff640 1 -- 172.21.15.172:0/1927257859 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dcc02ca90 con 0x7f1ddc15b2c0 2024-04-14T08:32:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.017+0000 7f1dd3fff640 1 -- 172.21.15.172:0/1927257859 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00f040 con 0x7f1ddc15b2c0 2024-04-14T08:32:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.018+0000 7f1de331e640 1 -- 172.21.15.172:0/1927257859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ddc15b2c0 msgr2=0x7f1ddc0cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.018+0000 7f1de331e640 1 --2- 172.21.15.172:0/1927257859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ddc15b2c0 0x7f1ddc0cf4d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0c4730 tx=0x7f1dcc02bce0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.018+0000 7f1de331e640 1 -- 172.21.15.172:0/1927257859 shutdown_connections 2024-04-14T08:32:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.018+0000 7f1de331e640 1 --2- 172.21.15.172:0/1927257859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 0x7f1ddc0722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.018+0000 7f1de331e640 1 --2- 172.21.15.172:0/1927257859 >> v2:172.21.15.177:3300/0 conn(0x7f1ddc0cb780 0x7f1ddc0cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.018+0000 7f1de331e640 1 --2- 172.21.15.172:0/1927257859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ddc15b2c0 0x7f1ddc0cf4d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.018+0000 7f1de331e640 1 -- 172.21.15.172:0/1927257859 >> 172.21.15.172:0/1927257859 conn(0x7f1ddc0d0f20 msgr2=0x7f1ddc0d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.018+0000 7f1de331e640 1 -- 172.21.15.172:0/1927257859 shutdown_connections 2024-04-14T08:32:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.018+0000 7f1de331e640 1 -- 172.21.15.172:0/1927257859 wait complete. 2024-04-14T08:32:29.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.019+0000 7f1de331e640 1 Processor -- start 2024-04-14T08:32:29.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.019+0000 7f1de331e640 1 -- start start 2024-04-14T08:32:29.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de331e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ddc0cb780 0x7f1ddc158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:29.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de331e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 0x7f1ddc09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:29.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de331e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ddc15b2c0 0x7f1ddc06ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:29.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de331e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ddc15b6a0 con 0x7f1ddc0cb780 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de331e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ddc0734c0 con 0x7f1ddc15b2c0 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de331e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1ddc15b820 con 0x7f1ddc0cc3c0 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de10ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ddc0cb780 0x7f1ddc158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de10ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ddc0cb780 0x7f1ddc158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54910/0 (socket says 172.21.15.172:54910) 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de10ba640 1 -- 172.21.15.172:0/4276486641 learned_addr learned my addr 172.21.15.172:0/4276486641 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de08b9640 1 --2- 172.21.15.172:0/4276486641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 0x7f1ddc09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de18bb640 1 --2- 172.21.15.172:0/4276486641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ddc15b2c0 0x7f1ddc06ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de10ba640 1 -- 172.21.15.172:0/4276486641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 msgr2=0x7f1ddc09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de10ba640 1 --2- 172.21.15.172:0/4276486641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 0x7f1ddc09aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de10ba640 1 -- 172.21.15.172:0/4276486641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ddc15b2c0 msgr2=0x7f1ddc06ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de10ba640 1 --2- 172.21.15.172:0/4276486641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ddc15b2c0 0x7f1ddc06ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de10ba640 1 -- 172.21.15.172:0/4276486641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ddc158580 con 0x7f1ddc0cb780 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de10ba640 1 --2- 172.21.15.172:0/4276486641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ddc0cb780 0x7f1ddc158040 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc037040 tx=0x7f1dcc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:29.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1dd1ffb640 1 -- 172.21.15.172:0/4276486641 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc044020 con 0x7f1ddc0cb780 2024-04-14T08:32:29.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1dd1ffb640 1 -- 172.21.15.172:0/4276486641 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dcc003610 con 0x7f1ddc0cb780 2024-04-14T08:32:29.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1de331e640 1 -- 172.21.15.172:0/4276486641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ddc0c8b80 con 0x7f1ddc0cb780 2024-04-14T08:32:29.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.020+0000 7f1dd1ffb640 1 -- 172.21.15.172:0/4276486641 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc039040 con 0x7f1ddc0cb780 2024-04-14T08:32:29.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.021+0000 7f1de331e640 1 -- 172.21.15.172:0/4276486641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ddc0c9140 con 0x7f1ddc0cb780 2024-04-14T08:32:29.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.021+0000 7f1dd1ffb640 1 -- 172.21.15.172:0/4276486641 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1dcc047020 con 0x7f1ddc0cb780 2024-04-14T08:32:29.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.021+0000 7f1dd1ffb640 1 -- 172.21.15.172:0/4276486641 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1dcc03fbf0 con 0x7f1ddc0cb780 2024-04-14T08:32:29.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.022+0000 7f1de331e640 1 -- 172.21.15.172:0/4276486641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ddc075490 con 0x7f1ddc0cb780 2024-04-14T08:32:29.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.024+0000 7f1dd1ffb640 1 -- 172.21.15.172:0/4276486641 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1dcc04e030 con 0x7f1ddc0cb780 2024-04-14T08:32:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.090+0000 7f1de331e640 1 -- 172.21.15.172:0/4276486641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1ddc0c78b0 con 0x7f1ddc0cb780 2024-04-14T08:32:29.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.091+0000 7f1dd1ffb640 1 -- 172.21.15.172:0/4276486641 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f1dcc03f470 con 0x7f1ddc0cb780 2024-04-14T08:32:29.093 INFO:teuthology.orchestra.run.smithi172.stderr:adjusted tunables profile to default 2024-04-14T08:32:29.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.092+0000 7f1de331e640 1 -- 172.21.15.172:0/4276486641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ddc0cb780 msgr2=0x7f1ddc158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.092+0000 7f1de331e640 1 --2- 172.21.15.172:0/4276486641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ddc0cb780 0x7f1ddc158040 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc037040 tx=0x7f1dcc004730 comp rx=0 tx=0).stop 2024-04-14T08:32:29.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.092+0000 7f1de331e640 1 -- 172.21.15.172:0/4276486641 shutdown_connections 2024-04-14T08:32:29.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.092+0000 7f1de331e640 1 --2- 172.21.15.172:0/4276486641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ddc15b2c0 0x7f1ddc06ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.092+0000 7f1de331e640 1 --2- 172.21.15.172:0/4276486641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ddc0cc3c0 0x7f1ddc09aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.092+0000 7f1de331e640 1 --2- 172.21.15.172:0/4276486641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ddc0cb780 0x7f1ddc158040 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.092+0000 7f1de331e640 1 -- 172.21.15.172:0/4276486641 >> 172.21.15.172:0/4276486641 conn(0x7f1ddc0d0f20 msgr2=0x7f1ddc0b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:29.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.092+0000 7f1de331e640 1 -- 172.21.15.172:0/4276486641 shutdown_connections 2024-04-14T08:32:29.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:29.092+0000 7f1de331e640 1 -- 172.21.15.172:0/4276486641 wait complete. 2024-04-14T08:32:29.105 INFO:tasks.ceph:check_enable_crimson: False 2024-04-14T08:32:29.105 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-14T08:32:29.105 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:29.105 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-14T08:32:29.132 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:29.132 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-14T08:32:29.199 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:29.199 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-14T08:32:29.266 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:32:29.266 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-14T08:32:29.334 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:29.334 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-14T08:32:29.361 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:29.361 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-14T08:32:29.428 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:29.428 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-14T08:32:29.440 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:29.438+0000 7fc72122b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-14T08:32:29.465 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:32:29.466 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-14T08:32:29.480 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:29.479+0000 7fa8b5430040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-14T08:32:29.495 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:29.494+0000 7fc72122b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-14T08:32:29.522 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new d6f1b9c1-f5ac-4793-a0f4-925d05e6981c 0 2024-04-14T08:32:29.533 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:29.532+0000 7fa8b5430040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-14T08:32:29.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.662+0000 7fdb4b99c640 1 Processor -- start 2024-04-14T08:32:29.663 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.662+0000 7fdb4b99c640 1 -- start start 2024-04-14T08:32:29.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.663+0000 7fdb4b99c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdb440ceab0 0x7fdb4406e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:29.664 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.663+0000 7fdb4b99c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb440cc000 con 0x7fdb4406e670 2024-04-14T08:32:29.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.663+0000 7fdb4b99c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb440cc2e0 con 0x7fdb4415afd0 2024-04-14T08:32:29.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.663+0000 7fdb4b99c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fdb440cbdd0 con 0x7fdb440ceab0 2024-04-14T08:32:29.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.663+0000 7fdb48f37640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdb440ceab0 0x7fdb4406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.663+0000 7fdb49738640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fdb4415afd0 0x7fdb440cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:37828/0 (socket says 172.21.15.177:37828) 2024-04-14T08:32:29.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.663+0000 7fdb48f37640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdb440ceab0 0x7fdb4406e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41492/0 (socket says 172.21.15.177:41492) 2024-04-14T08:32:29.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.663+0000 7fdb49738640 1 -- 172.21.15.177:0/2382934232 learned_addr learned my addr 172.21.15.177:0/2382934232 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-04-14T08:32:29.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.663+0000 7fdb48f37640 1 -- 172.21.15.177:0/2382934232 >> v1:172.21.15.177:6789/0 conn(0x7fdb4415afd0 legacy=0x7fdb440cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.663+0000 7fdb48f37640 1 -- 172.21.15.177:0/2382934232 >> v1:172.21.15.172:6789/0 conn(0x7fdb4406e670 legacy=0x7fdb440723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.665 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.663+0000 7fdb48f37640 1 -- 172.21.15.177:0/2382934232 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb440cc5c0 con 0x7fdb440ceab0 2024-04-14T08:32:29.666 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.664+0000 7fdb48f37640 1 --2- 172.21.15.177:0/2382934232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb440ceab0 0x7fdb4406e130 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdb340097a0 tx=0x7fdb3402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d56544af816f149c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:29.666 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.664+0000 7fdb3bfff640 1 -- 172.21.15.177:0/2382934232 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34034070 con 0x7fdb440ceab0 2024-04-14T08:32:29.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.664+0000 7fdb3bfff640 1 -- 172.21.15.177:0/2382934232 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb34030610 con 0x7fdb440ceab0 2024-04-14T08:32:29.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.664+0000 7fdb3bfff640 1 -- 172.21.15.177:0/2382934232 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb340309e0 con 0x7fdb440ceab0 2024-04-14T08:32:29.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.665+0000 7fdb4b99c640 1 -- 172.21.15.177:0/2382934232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb440ceab0 msgr2=0x7fdb4406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.665+0000 7fdb4b99c640 1 --2- 172.21.15.177:0/2382934232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb440ceab0 0x7fdb4406e130 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdb340097a0 tx=0x7fdb3402b960 comp rx=0 tx=0).stop 2024-04-14T08:32:29.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.665+0000 7fdb4b99c640 1 -- 172.21.15.177:0/2382934232 shutdown_connections 2024-04-14T08:32:29.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.665+0000 7fdb4b99c640 1 --2- 172.21.15.177:0/2382934232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb440ceab0 0x7fdb4406e130 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.665+0000 7fdb4b99c640 1 -- 172.21.15.177:0/2382934232 >> 172.21.15.177:0/2382934232 conn(0x7fdb440cd960 msgr2=0x7fdb440cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:29.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.665+0000 7fdb4b99c640 1 -- 172.21.15.177:0/2382934232 shutdown_connections 2024-04-14T08:32:29.667 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.665+0000 7fdb4b99c640 1 -- 172.21.15.177:0/2382934232 wait complete. 2024-04-14T08:32:29.669 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.666+0000 7fdb4b99c640 1 Processor -- start 2024-04-14T08:32:29.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.666+0000 7fdb4b99c640 1 -- start start 2024-04-14T08:32:29.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb4b99c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb440ceab0 0x7fdb441e6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:29.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb4b99c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb4415afd0 0x7fdb4406b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:29.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb4b99c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb441f1480 0x7fdb441f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:29.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb4b99c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4415b3b0 con 0x7fdb441f1480 2024-04-14T08:32:29.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb4b99c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdb440d01e0 con 0x7fdb440ceab0 2024-04-14T08:32:29.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb4b99c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4415b530 con 0x7fdb4415afd0 2024-04-14T08:32:29.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb48f37640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb4415afd0 0x7fdb4406b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb48f37640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb4415afd0 0x7fdb4406b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:46846/0 (socket says 172.21.15.177:46846) 2024-04-14T08:32:29.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb48f37640 1 -- 172.21.15.177:0/4269537803 learned_addr learned my addr 172.21.15.177:0/4269537803 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-14T08:32:29.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb49738640 1 --2- 172.21.15.177:0/4269537803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb440ceab0 0x7fdb441e6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.670 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb49f39640 1 --2- 172.21.15.177:0/4269537803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb441f1480 0x7fdb441f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb48f37640 1 -- 172.21.15.177:0/4269537803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb440ceab0 msgr2=0x7fdb441e6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb48f37640 1 --2- 172.21.15.177:0/4269537803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb440ceab0 0x7fdb441e6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb48f37640 1 -- 172.21.15.177:0/4269537803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb441f1480 msgr2=0x7fdb441f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb48f37640 1 --2- 172.21.15.177:0/4269537803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb441f1480 0x7fdb441f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb48f37640 1 -- 172.21.15.177:0/4269537803 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb441f5460 con 0x7fdb4415afd0 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb48f37640 1 --2- 172.21.15.177:0/4269537803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb4415afd0 0x7fdb4406b790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdb3402bf70 tx=0x7fdb340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb39ffb640 1 -- 172.21.15.177:0/4269537803 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34034040 con 0x7fdb4415afd0 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb39ffb640 1 -- 172.21.15.177:0/4269537803 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb34039a10 con 0x7fdb4415afd0 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb4b99c640 1 -- 172.21.15.177:0/4269537803 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb441f5180 con 0x7fdb4415afd0 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.667+0000 7fdb39ffb640 1 -- 172.21.15.177:0/4269537803 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34039e20 con 0x7fdb4415afd0 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.668+0000 7fdb4b99c640 1 -- 172.21.15.177:0/4269537803 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb441e5e40 con 0x7fdb4415afd0 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.668+0000 7fdb39ffb640 1 -- 172.21.15.177:0/4269537803 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdb34037070 con 0x7fdb4415afd0 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.668+0000 7fdb39ffb640 1 -- 172.21.15.177:0/4269537803 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fdb34042730 con 0x7fdb4415afd0 2024-04-14T08:32:29.671 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.669+0000 7fdb4b99c640 1 -- 172.21.15.177:0/4269537803 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb440cb730 con 0x7fdb4415afd0 2024-04-14T08:32:29.672 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.671+0000 7fdb39ffb640 1 -- 172.21.15.177:0/4269537803 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdb440cb730 con 0x7fdb4415afd0 2024-04-14T08:32:29.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.743+0000 7fdb4b99c640 1 -- 172.21.15.177:0/4269537803 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d6f1b9c1-f5ac-4793-a0f4-925d05e6981c", "id": 0} v 0) -- 0x7fdb44073020 con 0x7fdb4415afd0 2024-04-14T08:32:29.747 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.746+0000 7fdb39ffb640 1 -- 172.21.15.177:0/4269537803 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d6f1b9c1-f5ac-4793-a0f4-925d05e6981c", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fdb3404b400 con 0x7fdb4415afd0 2024-04-14T08:32:29.747 INFO:teuthology.orchestra.run.smithi177.stdout:0 2024-04-14T08:32:29.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.747+0000 7fdb4b99c640 1 -- 172.21.15.177:0/4269537803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb4415afd0 msgr2=0x7fdb4406b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.747+0000 7fdb4b99c640 1 --2- 172.21.15.177:0/4269537803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb4415afd0 0x7fdb4406b790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdb3402bf70 tx=0x7fdb340046c0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.747+0000 7fdb4b99c640 1 -- 172.21.15.177:0/4269537803 shutdown_connections 2024-04-14T08:32:29.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.747+0000 7fdb4b99c640 1 --2- 172.21.15.177:0/4269537803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb441f1480 0x7fdb441f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.747+0000 7fdb4b99c640 1 --2- 172.21.15.177:0/4269537803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb4415afd0 0x7fdb4406b790 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.747+0000 7fdb4b99c640 1 --2- 172.21.15.177:0/4269537803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb440ceab0 0x7fdb441e6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.747+0000 7fdb4b99c640 1 -- 172.21.15.177:0/4269537803 >> 172.21.15.177:0/4269537803 conn(0x7fdb440cd960 msgr2=0x7fdb440ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:29.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.747+0000 7fdb4b99c640 1 -- 172.21.15.177:0/4269537803 shutdown_connections 2024-04-14T08:32:29.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.747+0000 7fdb4b99c640 1 -- 172.21.15.177:0/4269537803 wait complete. 2024-04-14T08:32:29.758 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 689180ef-9f72-41c1-b646-a24328801759 1 2024-04-14T08:32:29.859 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.858+0000 7f3ddbba3640 1 Processor -- start 2024-04-14T08:32:29.860 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.858+0000 7f3ddbba3640 1 -- start start 2024-04-14T08:32:29.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.859+0000 7f3ddbba3640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3dd40b3790 0x7f3dd40b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:29.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.859+0000 7f3ddbba3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd40d0480 con 0x7f3dd415e690 2024-04-14T08:32:29.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.859+0000 7f3ddbba3640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd406b700 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.859+0000 7f3ddbba3640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd406b9e0 con 0x7f3dd40b3790 2024-04-14T08:32:29.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.859+0000 7f3dd913e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3dd40b3790 0x7f3dd40b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.859+0000 7f3dda140640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f3dd40ad8e0 0x7f3dd40adcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:37830/0 (socket says 172.21.15.177:37830) 2024-04-14T08:32:29.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.859+0000 7f3dda140640 1 -- 172.21.15.177:0/2789948059 learned_addr learned my addr 172.21.15.177:0/2789948059 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-04-14T08:32:29.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.859+0000 7f3dd913e640 1 -- 172.21.15.177:0/2789948059 >> v1:172.21.15.177:6789/0 conn(0x7f3dd40ad8e0 legacy=0x7f3dd40adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.859+0000 7f3dd913e640 1 -- 172.21.15.177:0/2789948059 >> v1:172.21.15.172:6789/0 conn(0x7f3dd415e690 legacy=0x7f3dd40cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.861 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.859+0000 7f3dd913e640 1 -- 172.21.15.177:0/2789948059 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd406bcc0 con 0x7f3dd40b3790 2024-04-14T08:32:29.862 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.860+0000 7f3dd913e640 1 --2- 172.21.15.177:0/2789948059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd40b3790 0x7f3dd40b3c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40097a0 tx=0x7f3dc402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3990af37adb2e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:29.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.860+0000 7f3dd893d640 1 -- 172.21.15.177:0/2789948059 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4035020 con 0x7f3dd40b3790 2024-04-14T08:32:29.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.860+0000 7f3dd893d640 1 -- 172.21.15.177:0/2789948059 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc4031830 con 0x7f3dd40b3790 2024-04-14T08:32:29.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.860+0000 7f3dd893d640 1 -- 172.21.15.177:0/2789948059 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4031c00 con 0x7f3dd40b3790 2024-04-14T08:32:29.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.861+0000 7f3ddbba3640 1 -- 172.21.15.177:0/2789948059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd40b3790 msgr2=0x7f3dd40b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.861+0000 7f3ddbba3640 1 --2- 172.21.15.177:0/2789948059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd40b3790 0x7f3dd40b3c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40097a0 tx=0x7f3dc402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.861+0000 7f3ddbba3640 1 -- 172.21.15.177:0/2789948059 shutdown_connections 2024-04-14T08:32:29.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.861+0000 7f3ddbba3640 1 --2- 172.21.15.177:0/2789948059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd40b3790 0x7f3dd40b3c20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.861+0000 7f3ddbba3640 1 -- 172.21.15.177:0/2789948059 >> 172.21.15.177:0/2789948059 conn(0x7f3dd40d0b20 msgr2=0x7f3dd40d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:29.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.861+0000 7f3ddbba3640 1 -- 172.21.15.177:0/2789948059 shutdown_connections 2024-04-14T08:32:29.863 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.861+0000 7f3ddbba3640 1 -- 172.21.15.177:0/2789948059 wait complete. 2024-04-14T08:32:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.862+0000 7f3ddbba3640 1 Processor -- start 2024-04-14T08:32:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.862+0000 7f3ddbba3640 1 -- start start 2024-04-14T08:32:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3ddbba3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd40ad8e0 0x7f3dd4151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3ddbba3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3dd40b3790 0x7f3dd406eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3ddbba3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3dd415e690 0x7f3dd41e9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3ddbba3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd40b5940 con 0x7f3dd415e690 2024-04-14T08:32:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3ddbba3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd406c280 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3ddbba3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd40ae400 con 0x7f3dd40b3790 2024-04-14T08:32:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3dd993f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd40ad8e0 0x7f3dd4151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.866 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3dd913e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3dd40b3790 0x7f3dd406eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3dd993f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd40ad8e0 0x7f3dd4151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41512/0 (socket says 172.21.15.177:41512) 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3dd993f640 1 -- 172.21.15.177:0/1538551847 learned_addr learned my addr 172.21.15.177:0/1538551847 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3dda140640 1 --2- 172.21.15.177:0/1538551847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3dd415e690 0x7f3dd41e9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3dd993f640 1 -- 172.21.15.177:0/1538551847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3dd40b3790 msgr2=0x7f3dd406eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3dd993f640 1 --2- 172.21.15.177:0/1538551847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3dd40b3790 0x7f3dd406eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3dd993f640 1 -- 172.21.15.177:0/1538551847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3dd415e690 msgr2=0x7f3dd41e9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3dd993f640 1 --2- 172.21.15.177:0/1538551847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3dd415e690 0x7f3dd41e9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.863+0000 7f3dd993f640 1 -- 172.21.15.177:0/1538551847 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd41e8ef0 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.864+0000 7f3dd993f640 1 --2- 172.21.15.177:0/1538551847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd40ad8e0 0x7f3dd4151b10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00c760 tx=0x7f3dcc00cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.864+0000 7f3dca7fc640 1 -- 172.21.15.177:0/1538551847 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc016020 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.864+0000 7f3dca7fc640 1 -- 172.21.15.177:0/1538551847 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dcc006d50 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.864+0000 7f3ddbba3640 1 -- 172.21.15.177:0/1538551847 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd41e91d0 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.864+0000 7f3dca7fc640 1 -- 172.21.15.177:0/1538551847 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc003bb0 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.864+0000 7f3ddbba3640 1 -- 172.21.15.177:0/1538551847 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd41e9790 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.867 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.864+0000 7f3dca7fc640 1 -- 172.21.15.177:0/1538551847 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3dcc00abe0 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.864+0000 7f3dca7fc640 1 -- 172.21.15.177:0/1538551847 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f3dcc0124b0 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.865+0000 7f3ddbba3640 1 -- 172.21.15.177:0/1538551847 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd40ccaa0 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.868 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.867+0000 7f3dca7fc640 1 -- 172.21.15.177:0/1538551847 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3dd40ccaa0 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.941 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.939+0000 7f3ddbba3640 1 -- 172.21.15.177:0/1538551847 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "689180ef-9f72-41c1-b646-a24328801759", "id": 1} v 0) -- 0x7f3dd4075490 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.945+0000 7f3dca7fc640 1 -- 172.21.15.177:0/1538551847 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "689180ef-9f72-41c1-b646-a24328801759", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f3dcc020020 con 0x7f3dd40ad8e0 2024-04-14T08:32:29.946 INFO:teuthology.orchestra.run.smithi177.stdout:1 2024-04-14T08:32:29.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.946+0000 7f3ddbba3640 1 -- 172.21.15.177:0/1538551847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd40ad8e0 msgr2=0x7f3dd4151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:29.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.946+0000 7f3ddbba3640 1 --2- 172.21.15.177:0/1538551847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd40ad8e0 0x7f3dd4151b10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00c760 tx=0x7f3dcc00cc20 comp rx=0 tx=0).stop 2024-04-14T08:32:29.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.946+0000 7f3ddbba3640 1 -- 172.21.15.177:0/1538551847 shutdown_connections 2024-04-14T08:32:29.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.946+0000 7f3ddbba3640 1 --2- 172.21.15.177:0/1538551847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3dd415e690 0x7f3dd41e9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.946+0000 7f3ddbba3640 1 --2- 172.21.15.177:0/1538551847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3dd40b3790 0x7f3dd406eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.946+0000 7f3ddbba3640 1 --2- 172.21.15.177:0/1538551847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd40ad8e0 0x7f3dd4151b10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:29.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.946+0000 7f3ddbba3640 1 -- 172.21.15.177:0/1538551847 >> 172.21.15.177:0/1538551847 conn(0x7f3dd40d0b20 msgr2=0x7f3dd4071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:29.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.946+0000 7f3ddbba3640 1 -- 172.21.15.177:0/1538551847 shutdown_connections 2024-04-14T08:32:29.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:29.946+0000 7f3ddbba3640 1 -- 172.21.15.177:0/1538551847 wait complete. 2024-04-14T08:32:29.958 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 245caafe-6315-445d-8e4d-5ef247dfe476 2 2024-04-14T08:32:30.058 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.057+0000 7fc03c7cf640 1 Processor -- start 2024-04-14T08:32:30.058 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.057+0000 7fc03c7cf640 1 -- start start 2024-04-14T08:32:30.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc03c7cf640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340cfbf0 0x7fc0340cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc03c7cf640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc0340d05e0 0x7fc0340d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc03c7cf640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0340ccc30 con 0x7fc03415f8b0 2024-04-14T08:32:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc03c7cf640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc0340d0f90 con 0x7fc0340d05e0 2024-04-14T08:32:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc03c7cf640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc03415fc90 con 0x7fc0340cfbf0 2024-04-14T08:32:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc039d6a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340cfbf0 0x7fc0340cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc03ad6c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc0340d05e0 0x7fc0340d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc03ad6c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc0340d05e0 0x7fc0340d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41526/0 (socket says 172.21.15.177:41526) 2024-04-14T08:32:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc039d6a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340cfbf0 0x7fc0340cd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:46872/0 (socket says 172.21.15.177:46872) 2024-04-14T08:32:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc03ad6c640 1 -- 172.21.15.177:0/1060438426 learned_addr learned my addr 172.21.15.177:0/1060438426 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-14T08:32:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc03ad6c640 1 -- 172.21.15.177:0/1060438426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340cfbf0 msgr2=0x7fc0340cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc03ad6c640 1 --2- 172.21.15.177:0/1060438426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340cfbf0 0x7fc0340cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.060 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc03ad6c640 1 -- 172.21.15.177:0/1060438426 >> v1:172.21.15.172:6789/0 conn(0x7fc03415f8b0 legacy=0x7fc0340cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.061 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.058+0000 7fc03ad6c640 1 -- 172.21.15.177:0/1060438426 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0340d01b0 con 0x7fc0340d05e0 2024-04-14T08:32:30.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.059+0000 7fc03ad6c640 1 --2- 172.21.15.177:0/1060438426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340d05e0 0x7fc0340d0a50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009e50 tx=0x7fc02c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fc4e80231a51cef server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:30.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.059+0000 7fc039569640 1 -- 172.21.15.177:0/1060438426 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c02cb00 con 0x7fc0340d05e0 2024-04-14T08:32:30.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.059+0000 7fc039569640 1 -- 172.21.15.177:0/1060438426 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c02cca0 con 0x7fc0340d05e0 2024-04-14T08:32:30.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.059+0000 7fc039569640 1 -- 172.21.15.177:0/1060438426 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c034610 con 0x7fc0340d05e0 2024-04-14T08:32:30.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.060+0000 7fc03c7cf640 1 -- 172.21.15.177:0/1060438426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340d05e0 msgr2=0x7fc0340d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.060+0000 7fc03c7cf640 1 --2- 172.21.15.177:0/1060438426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340d05e0 0x7fc0340d0a50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009e50 tx=0x7fc02c02bf60 comp rx=0 tx=0).stop 2024-04-14T08:32:30.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.060+0000 7fc03c7cf640 1 -- 172.21.15.177:0/1060438426 shutdown_connections 2024-04-14T08:32:30.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.060+0000 7fc03c7cf640 1 --2- 172.21.15.177:0/1060438426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340d05e0 0x7fc0340d0a50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.060+0000 7fc03c7cf640 1 --2- 172.21.15.177:0/1060438426 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340cfbf0 0x7fc0340cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.062 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.060+0000 7fc03c7cf640 1 -- 172.21.15.177:0/1060438426 >> 172.21.15.177:0/1060438426 conn(0x7fc0340d2020 msgr2=0x7fc0340d2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:30.063 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.060+0000 7fc03c7cf640 1 -- 172.21.15.177:0/1060438426 shutdown_connections 2024-04-14T08:32:30.063 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.060+0000 7fc03c7cf640 1 -- 172.21.15.177:0/1060438426 wait complete. 2024-04-14T08:32:30.065 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.061+0000 7fc03c7cf640 1 Processor -- start 2024-04-14T08:32:30.065 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.061+0000 7fc03c7cf640 1 -- start start 2024-04-14T08:32:30.065 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03c7cf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340cfbf0 0x7fc03415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.065 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03c7cf640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340d05e0 0x7fc0341e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.065 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03c7cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03415f8b0 0x7fc0341e8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03c7cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0340cd490 con 0x7fc03415f8b0 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03c7cf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0340cd1b0 con 0x7fc0340cfbf0 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03c7cf640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc0340cdef0 con 0x7fc0340d05e0 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03a56b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340cfbf0 0x7fc03415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc039d6a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340d05e0 0x7fc0341e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc039d6a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340d05e0 0x7fc0341e7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:46888/0 (socket says 172.21.15.177:46888) 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03a56b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340cfbf0 0x7fc03415b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41534/0 (socket says 172.21.15.177:41534) 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc039d6a640 1 -- 172.21.15.177:0/2173450964 learned_addr learned my addr 172.21.15.177:0/2173450964 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03ad6c640 1 --2- 172.21.15.177:0/2173450964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03415f8b0 0x7fc0341e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03a56b640 1 -- 172.21.15.177:0/2173450964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340d05e0 msgr2=0x7fc0341e7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03a56b640 1 --2- 172.21.15.177:0/2173450964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340d05e0 0x7fc0341e7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03a56b640 1 -- 172.21.15.177:0/2173450964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03415f8b0 msgr2=0x7fc0341e8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.066 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03a56b640 1 --2- 172.21.15.177:0/2173450964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03415f8b0 0x7fc0341e8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03a56b640 1 -- 172.21.15.177:0/2173450964 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0340b67e0 con 0x7fc0340cfbf0 2024-04-14T08:32:30.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc039d6a640 1 --2- 172.21.15.177:0/2173450964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340d05e0 0x7fc0341e7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:30.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc03a56b640 1 --2- 172.21.15.177:0/2173450964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340cfbf0 0x7fc03415b100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc03000c340 tx=0x7fc03000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:30.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.062+0000 7fc02affd640 1 -- 172.21.15.177:0/2173450964 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0300052d0 con 0x7fc0340cfbf0 2024-04-14T08:32:30.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.063+0000 7fc02affd640 1 -- 172.21.15.177:0/2173450964 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc030005490 con 0x7fc0340cfbf0 2024-04-14T08:32:30.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.063+0000 7fc02affd640 1 -- 172.21.15.177:0/2173450964 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030007260 con 0x7fc0340cfbf0 2024-04-14T08:32:30.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.063+0000 7fc03c7cf640 1 -- 172.21.15.177:0/2173450964 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0340b5c60 con 0x7fc0340cfbf0 2024-04-14T08:32:30.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.063+0000 7fc03c7cf640 1 -- 172.21.15.177:0/2173450964 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0340b6220 con 0x7fc0340cfbf0 2024-04-14T08:32:30.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.063+0000 7fc02affd640 1 -- 172.21.15.177:0/2173450964 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc030007400 con 0x7fc0340cfbf0 2024-04-14T08:32:30.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.063+0000 7fc02affd640 1 -- 172.21.15.177:0/2173450964 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fc030011590 con 0x7fc0340cfbf0 2024-04-14T08:32:30.067 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.064+0000 7fc03c7cf640 1 -- 172.21.15.177:0/2173450964 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0340cc7c0 con 0x7fc0340cfbf0 2024-04-14T08:32:30.068 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:30.065+0000 7fa8b5430040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-14T08:32:30.068 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.066+0000 7fc02affd640 1 -- 172.21.15.177:0/2173450964 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc0340cc7c0 con 0x7fc0340cfbf0 2024-04-14T08:32:30.070 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:30.069+0000 7fc72122b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-14T08:32:30.119 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:30.119+0000 7fa8b5430040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-14T08:32:30.124 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:30.123+0000 7fc72122b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-14T08:32:30.138 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.137+0000 7fc03c7cf640 1 -- 172.21.15.177:0/2173450964 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "245caafe-6315-445d-8e4d-5ef247dfe476", "id": 2} v 0) -- 0x7fc034075490 con 0x7fc0340cfbf0 2024-04-14T08:32:30.142 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.140+0000 7fc02affd640 1 -- 172.21.15.177:0/2173450964 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "245caafe-6315-445d-8e4d-5ef247dfe476", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fc030003a50 con 0x7fc0340cfbf0 2024-04-14T08:32:30.142 INFO:teuthology.orchestra.run.smithi177.stdout:2 2024-04-14T08:32:30.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.141+0000 7fc03c7cf640 1 -- 172.21.15.177:0/2173450964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340cfbf0 msgr2=0x7fc03415b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.141+0000 7fc03c7cf640 1 --2- 172.21.15.177:0/2173450964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340cfbf0 0x7fc03415b100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc03000c340 tx=0x7fc03000c800 comp rx=0 tx=0).stop 2024-04-14T08:32:30.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.141+0000 7fc03c7cf640 1 -- 172.21.15.177:0/2173450964 shutdown_connections 2024-04-14T08:32:30.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.141+0000 7fc03c7cf640 1 --2- 172.21.15.177:0/2173450964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03415f8b0 0x7fc0341e8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.141+0000 7fc03c7cf640 1 --2- 172.21.15.177:0/2173450964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0340d05e0 0x7fc0341e7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.141+0000 7fc03c7cf640 1 --2- 172.21.15.177:0/2173450964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0340cfbf0 0x7fc03415b100 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.141+0000 7fc03c7cf640 1 -- 172.21.15.177:0/2173450964 >> 172.21.15.177:0/2173450964 conn(0x7fc0340d2020 msgr2=0x7fc0340b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:30.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.141+0000 7fc03c7cf640 1 -- 172.21.15.177:0/2173450964 shutdown_connections 2024-04-14T08:32:30.143 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.142+0000 7fc03c7cf640 1 -- 172.21.15.177:0/2173450964 wait complete. 2024-04-14T08:32:30.153 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 0889e14a-47cc-4585-ab53-9d78a1bdf5ad 3 2024-04-14T08:32:30.164 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:30.164+0000 7fa8b5430040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-14T08:32:30.171 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:30.170+0000 7fc72122b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-14T08:32:30.216 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:30.216+0000 7fa8b5430040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-14T08:32:30.227 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:30.226+0000 7fc72122b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-14T08:32:30.254 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.253+0000 7fedbb3c8640 1 Processor -- start 2024-04-14T08:32:30.254 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.253+0000 7fedbb3c8640 1 -- start start 2024-04-14T08:32:30.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.253+0000 7fedbb3c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fedb415c4e0 0x7fedb40cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedbb3c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb40d09d0 0x7fedb40b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedbb3c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedb40d2df0 con 0x7fedb40cd650 2024-04-14T08:32:30.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedbb3c8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fedb40cdb50 con 0x7fedb415c4e0 2024-04-14T08:32:30.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedbb3c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fedb40cde30 con 0x7fedb40d09d0 2024-04-14T08:32:30.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedb9965640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb40d09d0 0x7fedb40b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedb8963640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fedb40cd650 0x7fedb40d02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:37840/0 (socket says 172.21.15.177:37840) 2024-04-14T08:32:30.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedb8963640 1 -- 172.21.15.177:0/2331846572 learned_addr learned my addr 172.21.15.177:0/2331846572 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-04-14T08:32:30.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedb9965640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb40d09d0 0x7fedb40b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:46902/0 (socket says 172.21.15.177:46902) 2024-04-14T08:32:30.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedb9164640 1 --2- 172.21.15.177:0/2331846572 >> v2:172.21.15.172:3300/0 conn(0x7fedb415c4e0 0x7fedb40cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedb9965640 1 -- 172.21.15.177:0/2331846572 >> v2:172.21.15.172:3300/0 conn(0x7fedb415c4e0 msgr2=0x7fedb40cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:30.257 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedb9965640 1 --2- 172.21.15.177:0/2331846572 >> v2:172.21.15.172:3300/0 conn(0x7fedb415c4e0 0x7fedb40cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.258 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedb9965640 1 -- 172.21.15.177:0/2331846572 >> v1:172.21.15.177:6789/0 conn(0x7fedb40cd650 legacy=0x7fedb40d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.258 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedabfff640 1 -- 172.21.15.177:0/2331846572 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2666290482 0 0) 0x7fedb40d2df0 con 0x7fedb40cd650 2024-04-14T08:32:30.258 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.254+0000 7fedb9965640 1 -- 172.21.15.177:0/2331846572 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb40b1990 con 0x7fedb40d09d0 2024-04-14T08:32:30.258 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.255+0000 7fedb9965640 1 --2- 172.21.15.177:0/2331846572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb40d09d0 0x7fedb40b2470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fedac009d90 tx=0x7fedac02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b50c7286b436f6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:30.258 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.255+0000 7fedabfff640 1 -- 172.21.15.177:0/2331846572 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac039070 con 0x7fedb40d09d0 2024-04-14T08:32:30.258 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.255+0000 7fedabfff640 1 -- 172.21.15.177:0/2331846572 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedac02cc50 con 0x7fedb40d09d0 2024-04-14T08:32:30.258 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.255+0000 7fedabfff640 1 -- 172.21.15.177:0/2331846572 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac034440 con 0x7fedb40d09d0 2024-04-14T08:32:30.258 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.255+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2331846572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb40d09d0 msgr2=0x7fedb40b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.259 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.255+0000 7fedbb3c8640 1 --2- 172.21.15.177:0/2331846572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb40d09d0 0x7fedb40b2470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fedac009d90 tx=0x7fedac02bea0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.259 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.255+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2331846572 shutdown_connections 2024-04-14T08:32:30.259 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.255+0000 7fedbb3c8640 1 --2- 172.21.15.177:0/2331846572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb40d09d0 0x7fedb40b2470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.259 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.255+0000 7fedbb3c8640 1 --2- 172.21.15.177:0/2331846572 >> v2:172.21.15.172:3300/0 conn(0x7fedb415c4e0 0x7fedb40cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.259 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.255+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2331846572 >> 172.21.15.177:0/2331846572 conn(0x7fedb40d2030 msgr2=0x7fedb40d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:30.259 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.256+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2331846572 shutdown_connections 2024-04-14T08:32:30.259 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.256+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2331846572 wait complete. 2024-04-14T08:32:30.260 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:30.257+0000 7fa8b5430040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-14T08:32:30.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.257+0000 7fedbb3c8640 1 Processor -- start 2024-04-14T08:32:30.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.257+0000 7fedbb3c8640 1 -- start start 2024-04-14T08:32:30.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.257+0000 7fedbb3c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb40cd650 0x7fedb41f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.257+0000 7fedbb3c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedb40d09d0 0x7fedb40c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.257+0000 7fedbb3c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb415c4e0 0x7fedb415e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.262 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.257+0000 7fedbb3c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fedb40b2b30 con 0x7fedb40d09d0 2024-04-14T08:32:30.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.257+0000 7fedbb3c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fedb40b29b0 con 0x7fedb40cd650 2024-04-14T08:32:30.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.257+0000 7fedbb3c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fedb40b2cb0 con 0x7fedb415c4e0 2024-04-14T08:32:30.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb9965640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb415c4e0 0x7fedb415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb9164640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb40cd650 0x7fedb41f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb9965640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb415c4e0 0x7fedb415e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:46914/0 (socket says 172.21.15.177:46914) 2024-04-14T08:32:30.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb9965640 1 -- 172.21.15.177:0/2948575241 learned_addr learned my addr 172.21.15.177:0/2948575241 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-14T08:32:30.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb9164640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb40cd650 0x7fedb41f2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41548/0 (socket says 172.21.15.177:41548) 2024-04-14T08:32:30.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb8963640 1 --2- 172.21.15.177:0/2948575241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedb40d09d0 0x7fedb40c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb9965640 1 -- 172.21.15.177:0/2948575241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb40cd650 msgr2=0x7fedb41f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb9965640 1 --2- 172.21.15.177:0/2948575241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb40cd650 0x7fedb41f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.263 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb9965640 1 -- 172.21.15.177:0/2948575241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedb40d09d0 msgr2=0x7fedb40c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb9965640 1 --2- 172.21.15.177:0/2948575241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedb40d09d0 0x7fedb40c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb9965640 1 -- 172.21.15.177:0/2948575241 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb40c92a0 con 0x7fedb415c4e0 2024-04-14T08:32:30.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb9164640 1 --2- 172.21.15.177:0/2948575241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb40cd650 0x7fedb41f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:30.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedb9965640 1 --2- 172.21.15.177:0/2948575241 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb415c4e0 0x7fedb415e680 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fedac037040 tx=0x7fedac004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:30.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7feda9ffb640 1 -- 172.21.15.177:0/2948575241 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac039050 con 0x7fedb415c4e0 2024-04-14T08:32:30.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7feda9ffb640 1 -- 172.21.15.177:0/2948575241 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedac045070 con 0x7fedb415c4e0 2024-04-14T08:32:30.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2948575241 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb40c9580 con 0x7fedb415c4e0 2024-04-14T08:32:30.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7feda9ffb640 1 -- 172.21.15.177:0/2948575241 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac009450 con 0x7fedb415c4e0 2024-04-14T08:32:30.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.258+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2948575241 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb40c9b40 con 0x7fedb415c4e0 2024-04-14T08:32:30.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.259+0000 7feda9ffb640 1 -- 172.21.15.177:0/2948575241 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fedac03e610 con 0x7fedb415c4e0 2024-04-14T08:32:30.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.259+0000 7feda9ffb640 1 -- 172.21.15.177:0/2948575241 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fedac04acb0 con 0x7fedb415c4e0 2024-04-14T08:32:30.264 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.259+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2948575241 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb4075490 con 0x7fedb415c4e0 2024-04-14T08:32:30.265 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.261+0000 7feda9ffb640 1 -- 172.21.15.177:0/2948575241 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fedac04a820 con 0x7fedb415c4e0 2024-04-14T08:32:30.272 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:30.271+0000 7fc72122b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-14T08:32:30.334 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.333+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2948575241 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0889e14a-47cc-4585-ab53-9d78a1bdf5ad", "id": 3} v 0) -- 0x7fedb41f3420 con 0x7fedb415c4e0 2024-04-14T08:32:30.337 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.336+0000 7feda9ffb640 1 -- 172.21.15.177:0/2948575241 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0889e14a-47cc-4585-ab53-9d78a1bdf5ad", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fedac04aa80 con 0x7fedb415c4e0 2024-04-14T08:32:30.338 INFO:teuthology.orchestra.run.smithi177.stdout:3 2024-04-14T08:32:30.339 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.337+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2948575241 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb415c4e0 msgr2=0x7fedb415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.339 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.337+0000 7fedbb3c8640 1 --2- 172.21.15.177:0/2948575241 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb415c4e0 0x7fedb415e680 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fedac037040 tx=0x7fedac004a40 comp rx=0 tx=0).stop 2024-04-14T08:32:30.339 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.337+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2948575241 shutdown_connections 2024-04-14T08:32:30.339 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.337+0000 7fedbb3c8640 1 --2- 172.21.15.177:0/2948575241 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb415c4e0 0x7fedb415e680 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.339 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.337+0000 7fedbb3c8640 1 --2- 172.21.15.177:0/2948575241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedb40d09d0 0x7fedb40c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.339 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.337+0000 7fedbb3c8640 1 --2- 172.21.15.177:0/2948575241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb40cd650 0x7fedb41f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.339 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.337+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2948575241 >> 172.21.15.177:0/2948575241 conn(0x7fedb40d2030 msgr2=0x7fedb40b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:30.339 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.337+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2948575241 shutdown_connections 2024-04-14T08:32:30.339 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.337+0000 7fedbb3c8640 1 -- 172.21.15.177:0/2948575241 wait complete. 2024-04-14T08:32:30.348 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 39793065-1412-483e-a616-b20f815c4a05 4 2024-04-14T08:32:30.374 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:30.374+0000 7fa8b5430040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-14T08:32:30.391 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:30.390+0000 7fc72122b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-14T08:32:30.443 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:30.443+0000 7fa8b5430040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-14T08:32:30.464 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:30.463+0000 7fc72122b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-14T08:32:30.488 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:30.488+0000 7fa8b5430040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-14T08:32:30.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.487+0000 7fc8f9ede640 1 Processor -- start 2024-04-14T08:32:30.489 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.488+0000 7fc8f9ede640 1 -- start start 2024-04-14T08:32:30.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.488+0000 7fc8f9ede640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8f40cf6f0 0x7fc8f40ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.488+0000 7fc8f9ede640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc8f40ceea0 0x7fc8f415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.488+0000 7fc8f9ede640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8f40cc7b0 con 0x7fc8f40b7a00 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.488+0000 7fc8f9ede640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f40d1690 con 0x7fc8f40cf6f0 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.488+0000 7fc8f9ede640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f40cd0e0 con 0x7fc8f40ceea0 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.488+0000 7fc8f3fff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc8f40ceea0 0x7fc8f415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.488+0000 7fc8f37fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fc8f40b7a00 0x7fc8f40d0d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:37854/0 (socket says 172.21.15.177:37854) 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.488+0000 7fc8f37fe640 1 -- 172.21.15.177:0/697305304 learned_addr learned my addr 172.21.15.177:0/697305304 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.489+0000 7fc8f2ffd640 1 --2- 172.21.15.177:0/697305304 >> v2:172.21.15.172:3300/0 conn(0x7fc8f40cf6f0 0x7fc8f40ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.489+0000 7fc8f3fff640 1 -- 172.21.15.177:0/697305304 >> v2:172.21.15.172:3300/0 conn(0x7fc8f40cf6f0 msgr2=0x7fc8f40ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.489+0000 7fc8f3fff640 1 --2- 172.21.15.177:0/697305304 >> v2:172.21.15.172:3300/0 conn(0x7fc8f40cf6f0 0x7fc8f40ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.489+0000 7fc8f3fff640 1 -- 172.21.15.177:0/697305304 >> v1:172.21.15.177:6789/0 conn(0x7fc8f40b7a00 legacy=0x7fc8f40d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.489+0000 7fc8f27fc640 1 -- 172.21.15.177:0/697305304 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2693888773 0 0) 0x7fc8f40cc7b0 con 0x7fc8f40b7a00 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.489+0000 7fc8f3fff640 1 -- 172.21.15.177:0/697305304 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f40cca90 con 0x7fc8f40ceea0 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.490+0000 7fc8f3fff640 1 --2- 172.21.15.177:0/697305304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40ceea0 0x7fc8f415dca0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4073010 tx=0x7fc8ec02be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=433ff717abefbedf server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.490+0000 7fc8f27fc640 1 -- 172.21.15.177:0/697305304 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec039070 con 0x7fc8f40ceea0 2024-04-14T08:32:30.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.490+0000 7fc8f27fc640 1 -- 172.21.15.177:0/697305304 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8ec02cbf0 con 0x7fc8f40ceea0 2024-04-14T08:32:30.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.490+0000 7fc8f27fc640 1 -- 172.21.15.177:0/697305304 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec034440 con 0x7fc8f40ceea0 2024-04-14T08:32:30.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.491+0000 7fc8f9ede640 1 -- 172.21.15.177:0/697305304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40ceea0 msgr2=0x7fc8f415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.491+0000 7fc8f9ede640 1 --2- 172.21.15.177:0/697305304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40ceea0 0x7fc8f415dca0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4073010 tx=0x7fc8ec02be40 comp rx=0 tx=0).stop 2024-04-14T08:32:30.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.491+0000 7fc8f9ede640 1 -- 172.21.15.177:0/697305304 shutdown_connections 2024-04-14T08:32:30.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.491+0000 7fc8f9ede640 1 --2- 172.21.15.177:0/697305304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40ceea0 0x7fc8f415dca0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.491+0000 7fc8f9ede640 1 --2- 172.21.15.177:0/697305304 >> v2:172.21.15.172:3300/0 conn(0x7fc8f40cf6f0 0x7fc8f40ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.491+0000 7fc8f9ede640 1 -- 172.21.15.177:0/697305304 >> 172.21.15.177:0/697305304 conn(0x7fc8f40d2030 msgr2=0x7fc8f40d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:30.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.491+0000 7fc8f9ede640 1 -- 172.21.15.177:0/697305304 shutdown_connections 2024-04-14T08:32:30.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.491+0000 7fc8f9ede640 1 -- 172.21.15.177:0/697305304 wait complete. 2024-04-14T08:32:30.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.492+0000 7fc8f9ede640 1 Processor -- start 2024-04-14T08:32:30.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.492+0000 7fc8f9ede640 1 -- start start 2024-04-14T08:32:30.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f9ede640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40b7a00 0x7fc8f415d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.496 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f9ede640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8f40ceea0 0x7fc8f41577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f9ede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f40cf6f0 0x7fc8f4158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f9ede640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f40b4e80 con 0x7fc8f40ceea0 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f9ede640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f40b7de0 con 0x7fc8f40b7a00 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f9ede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f40b5000 con 0x7fc8f40cf6f0 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f37fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40b7a00 0x7fc8f415d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f37fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40b7a00 0x7fc8f415d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41560/0 (socket says 172.21.15.177:41560) 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f37fe640 1 -- 172.21.15.177:0/828960798 learned_addr learned my addr 172.21.15.177:0/828960798 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f3fff640 1 --2- 172.21.15.177:0/828960798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f40cf6f0 0x7fc8f4158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f2ffd640 1 --2- 172.21.15.177:0/828960798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8f40ceea0 0x7fc8f41577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f37fe640 1 -- 172.21.15.177:0/828960798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f40cf6f0 msgr2=0x7fc8f4158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f37fe640 1 --2- 172.21.15.177:0/828960798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f40cf6f0 0x7fc8f4158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f37fe640 1 -- 172.21.15.177:0/828960798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8f40ceea0 msgr2=0x7fc8f41577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f37fe640 1 --2- 172.21.15.177:0/828960798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8f40ceea0 0x7fc8f41577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f37fe640 1 -- 172.21.15.177:0/828960798 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f41596c0 con 0x7fc8f40b7a00 2024-04-14T08:32:30.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f3fff640 1 --2- 172.21.15.177:0/828960798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f40cf6f0 0x7fc8f4158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:30.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f2ffd640 1 --2- 172.21.15.177:0/828960798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8f40ceea0 0x7fc8f41577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:30.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8f37fe640 1 --2- 172.21.15.177:0/828960798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40b7a00 0x7fc8f415d600 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c430 tx=0x7fc8e400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:30.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8dbfff640 1 -- 172.21.15.177:0/828960798 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40074e0 con 0x7fc8f40b7a00 2024-04-14T08:32:30.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.493+0000 7fc8dbfff640 1 -- 172.21.15.177:0/828960798 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e40047e0 con 0x7fc8f40b7a00 2024-04-14T08:32:30.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.494+0000 7fc8f9ede640 1 -- 172.21.15.177:0/828960798 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f41599a0 con 0x7fc8f40b7a00 2024-04-14T08:32:30.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.494+0000 7fc8dbfff640 1 -- 172.21.15.177:0/828960798 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4004bd0 con 0x7fc8f40b7a00 2024-04-14T08:32:30.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.494+0000 7fc8f9ede640 1 -- 172.21.15.177:0/828960798 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f4159f60 con 0x7fc8f40b7a00 2024-04-14T08:32:30.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.494+0000 7fc8dbfff640 1 -- 172.21.15.177:0/828960798 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc8e4011070 con 0x7fc8f40b7a00 2024-04-14T08:32:30.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.494+0000 7fc8dbfff640 1 -- 172.21.15.177:0/828960798 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fc8e40138d0 con 0x7fc8f40b7a00 2024-04-14T08:32:30.498 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.494+0000 7fc8f9ede640 1 -- 172.21.15.177:0/828960798 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f4075490 con 0x7fc8f40b7a00 2024-04-14T08:32:30.499 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.496+0000 7fc8dbfff640 1 -- 172.21.15.177:0/828960798 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc8e4004dc0 con 0x7fc8f40b7a00 2024-04-14T08:32:30.513 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:30.512+0000 7fc72122b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-14T08:32:30.539 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:30.539+0000 7fa8b5430040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-14T08:32:30.566 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:30.565+0000 7fc72122b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-14T08:32:30.570 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.569+0000 7fc8f9ede640 1 -- 172.21.15.177:0/828960798 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "39793065-1412-483e-a616-b20f815c4a05", "id": 4} v 0) -- 0x7fc8f40cf4a0 con 0x7fc8f40b7a00 2024-04-14T08:32:30.576 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.575+0000 7fc8dbfff640 1 -- 172.21.15.177:0/828960798 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "39793065-1412-483e-a616-b20f815c4a05", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fc8e4013c80 con 0x7fc8f40b7a00 2024-04-14T08:32:30.576 INFO:teuthology.orchestra.run.smithi177.stdout:4 2024-04-14T08:32:30.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.575+0000 7fc8f9ede640 1 -- 172.21.15.177:0/828960798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40b7a00 msgr2=0x7fc8f415d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.575+0000 7fc8f9ede640 1 --2- 172.21.15.177:0/828960798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40b7a00 0x7fc8f415d600 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c430 tx=0x7fc8e400c8f0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.575+0000 7fc8f9ede640 1 -- 172.21.15.177:0/828960798 shutdown_connections 2024-04-14T08:32:30.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.575+0000 7fc8f9ede640 1 --2- 172.21.15.177:0/828960798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8f40cf6f0 0x7fc8f4158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.575+0000 7fc8f9ede640 1 --2- 172.21.15.177:0/828960798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8f40ceea0 0x7fc8f41577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.575+0000 7fc8f9ede640 1 --2- 172.21.15.177:0/828960798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8f40b7a00 0x7fc8f415d600 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.575+0000 7fc8f9ede640 1 -- 172.21.15.177:0/828960798 >> 172.21.15.177:0/828960798 conn(0x7fc8f40d2030 msgr2=0x7fc8f40b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:30.577 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.576+0000 7fc8f9ede640 1 -- 172.21.15.177:0/828960798 shutdown_connections 2024-04-14T08:32:30.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.576+0000 7fc8f9ede640 1 -- 172.21.15.177:0/828960798 wait complete. 2024-04-14T08:32:30.586 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:30.586+0000 7fa8b5430040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-14T08:32:30.587 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new f593b8e5-fd62-4917-bc8d-9c695c04b8ab 5 2024-04-14T08:32:30.615 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:30.614+0000 7fc72122b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-14T08:32:30.715 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.713+0000 7f8050b08640 1 Processor -- start 2024-04-14T08:32:30.715 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.713+0000 7f8050b08640 1 -- start start 2024-04-14T08:32:30.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.714+0000 7f8050b08640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f804c0cfbf0 0x7f804c0cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.714+0000 7f8050b08640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f804c0ccc30 con 0x7f804c0d05e0 2024-04-14T08:32:30.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.714+0000 7f8050b08640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f804c0d01b0 con 0x7f804c15f8b0 2024-04-14T08:32:30.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.714+0000 7f8050b08640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f804c15fc90 con 0x7f804c0cfbf0 2024-04-14T08:32:30.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.714+0000 7f804a59c640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f804c15f8b0 0x7f804c0cf400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:37860/0 (socket says 172.21.15.177:37860) 2024-04-14T08:32:30.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.714+0000 7f804a59c640 1 -- 172.21.15.177:0/4133811421 learned_addr learned my addr 172.21.15.177:0/4133811421 (peer_addr_for_me v1:172.21.15.177:0/0) 2024-04-14T08:32:30.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.714+0000 7f8043fff640 1 --2- 172.21.15.177:0/4133811421 >> v2:172.21.15.172:3300/0 conn(0x7f804c0cfbf0 0x7f804c0cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.715+0000 7f8049d9b640 1 -- 172.21.15.177:0/4133811421 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3168820200 0 0) 0x7f804c0d01b0 con 0x7f804c15f8b0 2024-04-14T08:32:30.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.715+0000 7f8049d9b640 1 -- 172.21.15.177:0/4133811421 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8038003520 con 0x7f804c15f8b0 2024-04-14T08:32:30.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.715+0000 7f8043fff640 1 -- 172.21.15.177:0/4133811421 >> v1:172.21.15.177:6789/0 conn(0x7f804c15f8b0 legacy=0x7f804c0cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.715+0000 7f8043fff640 1 -- 172.21.15.177:0/4133811421 >> v1:172.21.15.172:6789/0 conn(0x7f804c0d05e0 legacy=0x7f804c0d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.717 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.715+0000 7f8049d9b640 1 -- 172.21.15.177:0/4133811421 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1602513894 0 0) 0x7f804c0ccc30 con 0x7f804c0d05e0 2024-04-14T08:32:30.718 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.715+0000 7f8043fff640 1 -- 172.21.15.177:0/4133811421 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c0cced0 con 0x7f804c0cfbf0 2024-04-14T08:32:30.718 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.715+0000 7f8043fff640 1 --2- 172.21.15.177:0/4133811421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c0cfbf0 0x7f804c0cd9b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f80340097a0 tx=0x7f803402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc6b25834b43bd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:30.718 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.716+0000 7f8049d9b640 1 -- 172.21.15.177:0/4133811421 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034034070 con 0x7f804c0cfbf0 2024-04-14T08:32:30.718 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.716+0000 7f8049d9b640 1 -- 172.21.15.177:0/4133811421 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8034030610 con 0x7f804c0cfbf0 2024-04-14T08:32:30.718 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.716+0000 7f8049d9b640 1 -- 172.21.15.177:0/4133811421 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80340309e0 con 0x7f804c0cfbf0 2024-04-14T08:32:30.719 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.716+0000 7f8050b08640 1 -- 172.21.15.177:0/4133811421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c0cfbf0 msgr2=0x7f804c0cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.719 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.716+0000 7f8050b08640 1 --2- 172.21.15.177:0/4133811421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c0cfbf0 0x7f804c0cd9b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f80340097a0 tx=0x7f803402b960 comp rx=0 tx=0).stop 2024-04-14T08:32:30.719 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.716+0000 7f8050b08640 1 -- 172.21.15.177:0/4133811421 shutdown_connections 2024-04-14T08:32:30.719 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.716+0000 7f8050b08640 1 --2- 172.21.15.177:0/4133811421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c0cfbf0 0x7f804c0cd9b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.719 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.716+0000 7f8050b08640 1 -- 172.21.15.177:0/4133811421 >> 172.21.15.177:0/4133811421 conn(0x7f804c0d2020 msgr2=0x7f804c0d2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:30.719 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.716+0000 7f8050b08640 1 -- 172.21.15.177:0/4133811421 shutdown_connections 2024-04-14T08:32:30.719 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.716+0000 7f8050b08640 1 -- 172.21.15.177:0/4133811421 wait complete. 2024-04-14T08:32:30.719 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f8050b08640 1 Processor -- start 2024-04-14T08:32:30.719 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f8050b08640 1 -- start start 2024-04-14T08:32:30.721 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f8050b08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804c0cfbf0 0x7f804c0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.721 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f8050b08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f804c0d05e0 0x7f804c0b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f8050b08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c15f8b0 0x7f804c156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f8050b08640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f804c0cdef0 con 0x7f804c15f8b0 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f8050b08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f804c0cd490 con 0x7f804c0cfbf0 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f8050b08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f804c0b70f0 con 0x7f804c0d05e0 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f804a59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804c0cfbf0 0x7f804c0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f8043fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f804c0d05e0 0x7f804c0b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f804a59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804c0cfbf0 0x7f804c0c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41572/0 (socket says 172.21.15.177:41572) 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f804a59c640 1 -- 172.21.15.177:0/2939853336 learned_addr learned my addr 172.21.15.177:0/2939853336 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.718+0000 7f804ad9d640 1 --2- 172.21.15.177:0/2939853336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c15f8b0 0x7f804c156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8043fff640 1 -- 172.21.15.177:0/2939853336 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804c0cfbf0 msgr2=0x7f804c0c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8043fff640 1 --2- 172.21.15.177:0/2939853336 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804c0cfbf0 0x7f804c0c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8043fff640 1 -- 172.21.15.177:0/2939853336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c15f8b0 msgr2=0x7f804c156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.722 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8043fff640 1 --2- 172.21.15.177:0/2939853336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c15f8b0 0x7f804c156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8043fff640 1 -- 172.21.15.177:0/2939853336 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c0c8880 con 0x7f804c0d05e0 2024-04-14T08:32:30.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f804ad9d640 1 --2- 172.21.15.177:0/2939853336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c15f8b0 0x7f804c156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:30.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f804a59c640 1 --2- 172.21.15.177:0/2939853336 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804c0cfbf0 0x7f804c0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:30.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8043fff640 1 --2- 172.21.15.177:0/2939853336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f804c0d05e0 0x7f804c0b6bb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8034033000 tx=0x7f80340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:30.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8042ffd640 1 -- 172.21.15.177:0/2939853336 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034049070 con 0x7f804c0d05e0 2024-04-14T08:32:30.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8042ffd640 1 -- 172.21.15.177:0/2939853336 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8034030410 con 0x7f804c0d05e0 2024-04-14T08:32:30.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8050b08640 1 -- 172.21.15.177:0/2939853336 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804c0b4f80 con 0x7f804c0d05e0 2024-04-14T08:32:30.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8042ffd640 1 -- 172.21.15.177:0/2939853336 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034034040 con 0x7f804c0d05e0 2024-04-14T08:32:30.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8050b08640 1 -- 172.21.15.177:0/2939853336 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f804c0b5540 con 0x7f804c0d05e0 2024-04-14T08:32:30.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8042ffd640 1 -- 172.21.15.177:0/2939853336 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f803404c070 con 0x7f804c0d05e0 2024-04-14T08:32:30.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.719+0000 7f8042ffd640 1 -- 172.21.15.177:0/2939853336 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f80340449a0 con 0x7f804c0d05e0 2024-04-14T08:32:30.723 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.720+0000 7f8050b08640 1 -- 172.21.15.177:0/2939853336 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804c075490 con 0x7f804c0d05e0 2024-04-14T08:32:30.724 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.722+0000 7f8042ffd640 1 -- 172.21.15.177:0/2939853336 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f80340305c0 con 0x7f804c0d05e0 2024-04-14T08:32:30.795 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.794+0000 7f8050b08640 1 -- 172.21.15.177:0/2939853336 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f593b8e5-fd62-4917-bc8d-9c695c04b8ab", "id": 5} v 0) -- 0x7f804c0b1710 con 0x7f804c0d05e0 2024-04-14T08:32:30.800 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.799+0000 7f8042ffd640 1 -- 172.21.15.177:0/2939853336 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f593b8e5-fd62-4917-bc8d-9c695c04b8ab", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f8034044420 con 0x7f804c0d05e0 2024-04-14T08:32:30.800 INFO:teuthology.orchestra.run.smithi177.stdout:5 2024-04-14T08:32:30.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.800+0000 7f8050b08640 1 -- 172.21.15.177:0/2939853336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f804c0d05e0 msgr2=0x7f804c0b6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.800+0000 7f8050b08640 1 --2- 172.21.15.177:0/2939853336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f804c0d05e0 0x7f804c0b6bb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8034033000 tx=0x7f80340046c0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.801 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.800+0000 7f8050b08640 1 -- 172.21.15.177:0/2939853336 shutdown_connections 2024-04-14T08:32:30.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.800+0000 7f8050b08640 1 --2- 172.21.15.177:0/2939853336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c15f8b0 0x7f804c156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.800+0000 7f8050b08640 1 --2- 172.21.15.177:0/2939853336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f804c0d05e0 0x7f804c0b6bb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.800+0000 7f8050b08640 1 --2- 172.21.15.177:0/2939853336 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f804c0cfbf0 0x7f804c0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.800+0000 7f8050b08640 1 -- 172.21.15.177:0/2939853336 >> 172.21.15.177:0/2939853336 conn(0x7f804c0d2020 msgr2=0x7f804c0b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:30.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.800+0000 7f8050b08640 1 -- 172.21.15.177:0/2939853336 shutdown_connections 2024-04-14T08:32:30.802 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.800+0000 7f8050b08640 1 -- 172.21.15.177:0/2939853336 wait complete. 2024-04-14T08:32:30.812 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 3075e364-a72d-4130-9bc9-2fcd91b77c1a 6 2024-04-14T08:32:30.913 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.912+0000 7f71e20b7640 1 Processor -- start 2024-04-14T08:32:30.914 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.912+0000 7f71e20b7640 1 -- start start 2024-04-14T08:32:30.915 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71e20b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc0c68c0 0x7f71dc0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71e20b7640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f71dc072290 0x7f71dc0cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71e20b7640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71dc06fe80 con 0x7f71dc073290 2024-04-14T08:32:30.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71e20b7640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f71dc06f9f0 con 0x7f71dc072290 2024-04-14T08:32:30.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71e20b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f71dc0c71e0 con 0x7f71dc0c68c0 2024-04-14T08:32:30.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71db7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc0c68c0 0x7f71dc0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71db7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc0c68c0 0x7f71dc0c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:46942/0 (socket says 172.21.15.177:46942) 2024-04-14T08:32:30.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71db7fe640 1 -- 172.21.15.177:0/4018199604 learned_addr learned my addr 172.21.15.177:0/4018199604 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-14T08:32:30.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71dbfff640 1 --2- 172.21.15.177:0/4018199604 >> v2:172.21.15.172:3300/0 conn(0x7f71dc072290 0x7f71dc0cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71db7fe640 1 -- 172.21.15.177:0/4018199604 >> v2:172.21.15.172:3300/0 conn(0x7f71dc072290 msgr2=0x7f71dc0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:30.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71db7fe640 1 --2- 172.21.15.177:0/4018199604 >> v2:172.21.15.172:3300/0 conn(0x7f71dc072290 0x7f71dc0cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.916 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71db7fe640 1 -- 172.21.15.177:0/4018199604 >> v1:172.21.15.172:6789/0 conn(0x7f71dc073290 legacy=0x7f71dc0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.913+0000 7f71db7fe640 1 -- 172.21.15.177:0/4018199604 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc0b0710 con 0x7f71dc0c68c0 2024-04-14T08:32:30.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.914+0000 7f71db7fe640 1 --2- 172.21.15.177:0/4018199604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc0c68c0 0x7f71dc0c6ca0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00bb10 tx=0x7f71cc02e0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6280eff47410e5d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:30.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.914+0000 7f71daffd640 1 -- 172.21.15.177:0/4018199604 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc03b070 con 0x7f71dc0c68c0 2024-04-14T08:32:30.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.914+0000 7f71daffd640 1 -- 172.21.15.177:0/4018199604 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71cc036440 con 0x7f71dc0c68c0 2024-04-14T08:32:30.917 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.914+0000 7f71daffd640 1 -- 172.21.15.177:0/4018199604 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc036810 con 0x7f71dc0c68c0 2024-04-14T08:32:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.915+0000 7f71e20b7640 1 -- 172.21.15.177:0/4018199604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc0c68c0 msgr2=0x7f71dc0c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.915+0000 7f71e20b7640 1 --2- 172.21.15.177:0/4018199604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc0c68c0 0x7f71dc0c6ca0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00bb10 tx=0x7f71cc02e0c0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.915+0000 7f71e20b7640 1 -- 172.21.15.177:0/4018199604 shutdown_connections 2024-04-14T08:32:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.915+0000 7f71e20b7640 1 --2- 172.21.15.177:0/4018199604 >> v2:172.21.15.172:3300/0 conn(0x7f71dc072290 0x7f71dc0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.915+0000 7f71e20b7640 1 --2- 172.21.15.177:0/4018199604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc0c68c0 0x7f71dc0c6ca0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.915+0000 7f71e20b7640 1 -- 172.21.15.177:0/4018199604 >> 172.21.15.177:0/4018199604 conn(0x7f71dc0d0f10 msgr2=0x7f71dc0d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.915+0000 7f71e20b7640 1 -- 172.21.15.177:0/4018199604 shutdown_connections 2024-04-14T08:32:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.915+0000 7f71e20b7640 1 -- 172.21.15.177:0/4018199604 wait complete. 2024-04-14T08:32:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.916+0000 7f71e20b7640 1 Processor -- start 2024-04-14T08:32:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.916+0000 7f71e20b7640 1 -- start start 2024-04-14T08:32:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71e20b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc072290 0x7f71dc0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.920 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71e20b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71dc073290 0x7f71dc153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71e20b7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71dc0c68c0 0x7f71dc159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71e20b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc06f0c0 con 0x7f71dc073290 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71e20b7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc06ede0 con 0x7f71dc0c68c0 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71e20b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f71dc06f3a0 con 0x7f71dc072290 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71db7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc072290 0x7f71dc0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71db7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc072290 0x7f71dc0b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:46956/0 (socket says 172.21.15.177:46956) 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71dbfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71dc0c68c0 0x7f71dc159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71db7fe640 1 -- 172.21.15.177:0/718740812 learned_addr learned my addr 172.21.15.177:0/718740812 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71d3fff640 1 --2- 172.21.15.177:0/718740812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71dc073290 0x7f71dc153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71db7fe640 1 -- 172.21.15.177:0/718740812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71dc0c68c0 msgr2=0x7f71dc159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71db7fe640 1 --2- 172.21.15.177:0/718740812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71dc0c68c0 0x7f71dc159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71db7fe640 1 -- 172.21.15.177:0/718740812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71dc073290 msgr2=0x7f71dc153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71db7fe640 1 --2- 172.21.15.177:0/718740812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71dc073290 0x7f71dc153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:30.921 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.917+0000 7f71db7fe640 1 -- 172.21.15.177:0/718740812 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc0c8180 con 0x7f71dc072290 2024-04-14T08:32:30.922 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.918+0000 7f71db7fe640 1 --2- 172.21.15.177:0/718740812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc072290 0x7f71dc0b3c30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00b2c0 tx=0x7f71cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:30.922 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.918+0000 7f71d8ff9640 1 -- 172.21.15.177:0/718740812 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc045030 con 0x7f71dc072290 2024-04-14T08:32:30.922 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.918+0000 7f71d8ff9640 1 -- 172.21.15.177:0/718740812 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71cc034440 con 0x7f71dc072290 2024-04-14T08:32:30.922 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.918+0000 7f71e20b7640 1 -- 172.21.15.177:0/718740812 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71dc0c8460 con 0x7f71dc072290 2024-04-14T08:32:30.922 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.918+0000 7f71d8ff9640 1 -- 172.21.15.177:0/718740812 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc03b040 con 0x7f71dc072290 2024-04-14T08:32:30.922 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.918+0000 7f71e20b7640 1 -- 172.21.15.177:0/718740812 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71dc0c8a20 con 0x7f71dc072290 2024-04-14T08:32:30.922 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.918+0000 7f71d8ff9640 1 -- 172.21.15.177:0/718740812 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f71cc048070 con 0x7f71dc072290 2024-04-14T08:32:30.922 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.918+0000 7f71d8ff9640 1 -- 172.21.15.177:0/718740812 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f71cc041bf0 con 0x7f71dc072290 2024-04-14T08:32:30.922 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.919+0000 7f71e20b7640 1 -- 172.21.15.177:0/718740812 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71dc075490 con 0x7f71dc072290 2024-04-14T08:32:30.923 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.922+0000 7f71d8ff9640 1 -- 172.21.15.177:0/718740812 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f71cc041600 con 0x7f71dc072290 2024-04-14T08:32:30.965 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:30.965+0000 7fa8b5430040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-14T08:32:30.998 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:30.997+0000 7f71e20b7640 1 -- 172.21.15.177:0/718740812 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3075e364-a72d-4130-9bc9-2fcd91b77c1a", "id": 6} v 0) -- 0x7f71dc06f670 con 0x7f71dc072290 2024-04-14T08:32:31.004 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.003+0000 7f71d8ff9640 1 -- 172.21.15.177:0/718740812 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3075e364-a72d-4130-9bc9-2fcd91b77c1a", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f71cc0417e0 con 0x7f71dc072290 2024-04-14T08:32:31.004 INFO:teuthology.orchestra.run.smithi177.stdout:6 2024-04-14T08:32:31.006 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.004+0000 7f71e20b7640 1 -- 172.21.15.177:0/718740812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc072290 msgr2=0x7f71dc0b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:31.006 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.004+0000 7f71e20b7640 1 --2- 172.21.15.177:0/718740812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc072290 0x7f71dc0b3c30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00b2c0 tx=0x7f71cc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.006 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.004+0000 7f71e20b7640 1 -- 172.21.15.177:0/718740812 shutdown_connections 2024-04-14T08:32:31.006 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.004+0000 7f71e20b7640 1 --2- 172.21.15.177:0/718740812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71dc0c68c0 0x7f71dc159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.006 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.004+0000 7f71e20b7640 1 --2- 172.21.15.177:0/718740812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71dc073290 0x7f71dc153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.006 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.004+0000 7f71e20b7640 1 --2- 172.21.15.177:0/718740812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71dc072290 0x7f71dc0b3c30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.006 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.004+0000 7f71e20b7640 1 -- 172.21.15.177:0/718740812 >> 172.21.15.177:0/718740812 conn(0x7f71dc0d0f10 msgr2=0x7f71dc15c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:31.006 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.004+0000 7f71e20b7640 1 -- 172.21.15.177:0/718740812 shutdown_connections 2024-04-14T08:32:31.006 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.004+0000 7f71e20b7640 1 -- 172.21.15.177:0/718740812 wait complete. 2024-04-14T08:32:31.016 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 69516eae-c9e6-4c79-a191-bad7981ec9ab 7 2024-04-14T08:32:31.020 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:31.019+0000 7fc72122b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-14T08:32:31.101 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:31.100+0000 7fa8b5430040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-14T08:32:31.117 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.116+0000 7f05756a2640 1 Processor -- start 2024-04-14T08:32:31.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.116+0000 7f05756a2640 1 -- start start 2024-04-14T08:32:31.119 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.117+0000 7f05756a2640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f057006fea0 0x7f0570070280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.117+0000 7f05756a2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f057006d800 0x7f057006dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.117+0000 7f05756a2640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05700707c0 con 0x7f05700aed50 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.117+0000 7f05756a2640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f057006c760 con 0x7f057006d800 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.117+0000 7f05756a2640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f057006ca40 con 0x7f057006fea0 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.117+0000 7f056effd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f057006fea0 0x7f0570070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.117+0000 7f056effd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f057006fea0 0x7f0570070280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41590/0 (socket says 172.21.15.177:41590) 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.117+0000 7f056effd640 1 -- 172.21.15.177:0/3255377388 learned_addr learned my addr 172.21.15.177:0/3255377388 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.117+0000 7f056f7fe640 1 --2- 172.21.15.177:0/3255377388 >> v2:172.21.15.172:3300/0 conn(0x7f057006d800 0x7f057006dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.118+0000 7f056effd640 1 -- 172.21.15.177:0/3255377388 >> v2:172.21.15.172:3300/0 conn(0x7f057006d800 msgr2=0x7f057006dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.118+0000 7f056effd640 1 --2- 172.21.15.177:0/3255377388 >> v2:172.21.15.172:3300/0 conn(0x7f057006d800 0x7f057006dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.118+0000 7f056effd640 1 -- 172.21.15.177:0/3255377388 >> v1:172.21.15.172:6789/0 conn(0x7f05700aed50 legacy=0x7f05700d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.118+0000 7f056effd640 1 -- 172.21.15.177:0/3255377388 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057006cd20 con 0x7f057006fea0 2024-04-14T08:32:31.120 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.118+0000 7f056effd640 1 --2- 172.21.15.177:0/3255377388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f057006fea0 0x7f0570070280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f057006c420 tx=0x7f056402b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7149766d570f64b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:31.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.118+0000 7f056dffb640 1 -- 172.21.15.177:0/3255377388 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056402e070 con 0x7f057006fea0 2024-04-14T08:32:31.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.118+0000 7f056dffb640 1 -- 172.21.15.177:0/3255377388 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0564037070 con 0x7f057006fea0 2024-04-14T08:32:31.121 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.118+0000 7f056dffb640 1 -- 172.21.15.177:0/3255377388 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564031a00 con 0x7f057006fea0 2024-04-14T08:32:31.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.119+0000 7f05756a2640 1 -- 172.21.15.177:0/3255377388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f057006fea0 msgr2=0x7f0570070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:31.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.119+0000 7f05756a2640 1 --2- 172.21.15.177:0/3255377388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f057006fea0 0x7f0570070280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f057006c420 tx=0x7f056402b880 comp rx=0 tx=0).stop 2024-04-14T08:32:31.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.119+0000 7f05756a2640 1 -- 172.21.15.177:0/3255377388 shutdown_connections 2024-04-14T08:32:31.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.119+0000 7f05756a2640 1 --2- 172.21.15.177:0/3255377388 >> v2:172.21.15.172:3300/0 conn(0x7f057006d800 0x7f057006dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.119+0000 7f05756a2640 1 --2- 172.21.15.177:0/3255377388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f057006fea0 0x7f0570070280 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.119+0000 7f05756a2640 1 -- 172.21.15.177:0/3255377388 >> 172.21.15.177:0/3255377388 conn(0x7f05700d2020 msgr2=0x7f05700d2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:31.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.119+0000 7f05756a2640 1 -- 172.21.15.177:0/3255377388 shutdown_connections 2024-04-14T08:32:31.124 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.119+0000 7f05756a2640 1 -- 172.21.15.177:0/3255377388 wait complete. 2024-04-14T08:32:31.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.120+0000 7f05756a2640 1 Processor -- start 2024-04-14T08:32:31.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.121+0000 7f05756a2640 1 -- start start 2024-04-14T08:32:31.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.121+0000 7f05756a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f057006d800 0x7f05701f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:31.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.121+0000 7f05756a2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f057006fea0 0x7f05701f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:31.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.121+0000 7f05756a2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05700aed50 0x7f05700c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:31.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.121+0000 7f05756a2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f05700d1570 con 0x7f057006d800 2024-04-14T08:32:31.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.121+0000 7f05756a2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f057006d2e0 con 0x7f05700aed50 2024-04-14T08:32:31.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.121+0000 7f05756a2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f05700d16f0 con 0x7f057006fea0 2024-04-14T08:32:31.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.121+0000 7f056e7fc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f057006fea0 0x7f05701f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:31.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.121+0000 7f056f7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05700aed50 0x7f05700c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:31.125 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.121+0000 7f056e7fc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f057006fea0 0x7f05701f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:46970/0 (socket says 172.21.15.177:46970) 2024-04-14T08:32:31.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.121+0000 7f056e7fc640 1 -- 172.21.15.177:0/2167740122 learned_addr learned my addr 172.21.15.177:0/2167740122 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-14T08:32:31.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.121+0000 7f056effd640 1 --2- 172.21.15.177:0/2167740122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f057006d800 0x7f05701f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:31.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f056f7fe640 1 -- 172.21.15.177:0/2167740122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f057006fea0 msgr2=0x7f05701f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:31.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f056f7fe640 1 --2- 172.21.15.177:0/2167740122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f057006fea0 0x7f05701f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f056f7fe640 1 -- 172.21.15.177:0/2167740122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f057006d800 msgr2=0x7f05701f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:31.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f056f7fe640 1 --2- 172.21.15.177:0/2167740122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f057006d800 0x7f05701f7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f056f7fe640 1 -- 172.21.15.177:0/2167740122 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05701f8520 con 0x7f05700aed50 2024-04-14T08:32:31.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f056f7fe640 1 --2- 172.21.15.177:0/2167740122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05700aed50 0x7f05700c6f40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f056800ebd0 tx=0x7f056800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:31.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f05577fe640 1 -- 172.21.15.177:0/2167740122 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568018020 con 0x7f05700aed50 2024-04-14T08:32:31.126 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f05577fe640 1 -- 172.21.15.177:0/2167740122 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056800ccc0 con 0x7f05700aed50 2024-04-14T08:32:31.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f05577fe640 1 -- 172.21.15.177:0/2167740122 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05680146c0 con 0x7f05700aed50 2024-04-14T08:32:31.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f05756a2640 1 -- 172.21.15.177:0/2167740122 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05701f8800 con 0x7f05700aed50 2024-04-14T08:32:31.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f05756a2640 1 -- 172.21.15.177:0/2167740122 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05701f4d80 con 0x7f05700aed50 2024-04-14T08:32:31.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f05577fe640 1 -- 172.21.15.177:0/2167740122 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0568005d10 con 0x7f05700aed50 2024-04-14T08:32:31.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.122+0000 7f05577fe640 1 -- 172.21.15.177:0/2167740122 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f056801dd90 con 0x7f05700aed50 2024-04-14T08:32:31.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.123+0000 7f05756a2640 1 -- 172.21.15.177:0/2167740122 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05700d2ba0 con 0x7f05700aed50 2024-04-14T08:32:31.127 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.125+0000 7f05577fe640 1 -- 172.21.15.177:0/2167740122 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f05700d2ba0 con 0x7f05700aed50 2024-04-14T08:32:31.161 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:31.160+0000 7fc72122b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-14T08:32:31.199 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.198+0000 7f05756a2640 1 -- 172.21.15.177:0/2167740122 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "69516eae-c9e6-4c79-a191-bad7981ec9ab", "id": 7} v 0) -- 0x7f05700705e0 con 0x7f05700aed50 2024-04-14T08:32:31.205 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.204+0000 7f05577fe640 1 -- 172.21.15.177:0/2167740122 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "69516eae-c9e6-4c79-a191-bad7981ec9ab", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f05700705e0 con 0x7f05700aed50 2024-04-14T08:32:31.205 INFO:teuthology.orchestra.run.smithi177.stdout:7 2024-04-14T08:32:31.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.205+0000 7f05756a2640 1 -- 172.21.15.177:0/2167740122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05700aed50 msgr2=0x7f05700c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:31.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.205+0000 7f05756a2640 1 --2- 172.21.15.177:0/2167740122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05700aed50 0x7f05700c6f40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f056800ebd0 tx=0x7f056800c490 comp rx=0 tx=0).stop 2024-04-14T08:32:31.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.205+0000 7f05756a2640 1 -- 172.21.15.177:0/2167740122 shutdown_connections 2024-04-14T08:32:31.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.205+0000 7f05756a2640 1 --2- 172.21.15.177:0/2167740122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05700aed50 0x7f05700c6f40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.206 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.205+0000 7f05756a2640 1 --2- 172.21.15.177:0/2167740122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f057006fea0 0x7f05701f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.205+0000 7f05756a2640 1 --2- 172.21.15.177:0/2167740122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f057006d800 0x7f05701f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.205+0000 7f05756a2640 1 -- 172.21.15.177:0/2167740122 >> 172.21.15.177:0/2167740122 conn(0x7f05700d2020 msgr2=0x7f057015d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:31.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.205+0000 7f05756a2640 1 -- 172.21.15.177:0/2167740122 shutdown_connections 2024-04-14T08:32:31.207 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:32:31.205+0000 7f05756a2640 1 -- 172.21.15.177:0/2167740122 wait complete. 2024-04-14T08:32:31.207 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:31.207+0000 7fa8b5430040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-14T08:32:31.216 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-14T08:32:31.217 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-14T08:32:31.219 INFO:tasks.ceph.osd.0:Started 2024-04-14T08:32:31.219 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-14T08:32:31.219 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-14T08:32:31.224 INFO:tasks.ceph.osd.1:Started 2024-04-14T08:32:31.224 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-14T08:32:31.224 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-14T08:32:31.226 INFO:tasks.ceph.osd.2:Started 2024-04-14T08:32:31.226 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-14T08:32:31.226 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-14T08:32:31.228 INFO:tasks.ceph.osd.3:Started 2024-04-14T08:32:31.228 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-14T08:32:31.229 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-14T08:32:31.259 INFO:tasks.ceph.osd.4:Started 2024-04-14T08:32:31.259 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-14T08:32:31.259 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-14T08:32:31.261 INFO:tasks.ceph.osd.5:Started 2024-04-14T08:32:31.261 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-14T08:32:31.261 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-14T08:32:31.262 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:31.261+0000 7fa8b5430040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-14T08:32:31.264 INFO:tasks.ceph.osd.6:Started 2024-04-14T08:32:31.264 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-14T08:32:31.264 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-14T08:32:31.266 INFO:tasks.ceph.osd.7:Started 2024-04-14T08:32:31.267 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:32:31.285 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:31.284+0000 7fc72122b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-14T08:32:31.337 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:31.336+0000 7fc72122b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-14T08:32:31.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.384+0000 7f177b7cc640 1 Processor -- start 2024-04-14T08:32:31.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.385+0000 7f177b7cc640 1 -- start start 2024-04-14T08:32:31.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f177b7cc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f17740cde10 0x7f17740ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:31.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f177b7cc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f17740ced80 0x7f17740afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:31.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f177b7cc640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17740aef60 con 0x7f177415c710 2024-04-14T08:32:31.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f177b7cc640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f17740cf1f0 con 0x7f17740cde10 2024-04-14T08:32:31.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f177b7cc640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f17740aed10 con 0x7f17740ced80 2024-04-14T08:32:31.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f1778d67640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f17740cde10 0x7f17740ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:31.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f1778d67640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f17740cde10 0x7f17740ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36940/0 (socket says 172.21.15.172:36940) 2024-04-14T08:32:31.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f1778d67640 1 -- 172.21.15.172:0/3604503739 learned_addr learned my addr 172.21.15.172:0/3604503739 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:31.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f1779d69640 1 --2- 172.21.15.172:0/3604503739 >> v2:172.21.15.177:3300/0 conn(0x7f17740ced80 0x7f17740afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:31.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f1778d67640 1 -- 172.21.15.172:0/3604503739 >> v2:172.21.15.177:3300/0 conn(0x7f17740ced80 msgr2=0x7f17740afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:31.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f1778d67640 1 --2- 172.21.15.172:0/3604503739 >> v2:172.21.15.177:3300/0 conn(0x7f17740ced80 0x7f17740afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f1778d67640 1 -- 172.21.15.172:0/3604503739 >> v1:172.21.15.177:6789/0 conn(0x7f177415c710 legacy=0x7f17740d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:31.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.386+0000 7f1778d67640 1 -- 172.21.15.172:0/3604503739 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17740af240 con 0x7f17740cde10 2024-04-14T08:32:31.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.387+0000 7f1778d67640 1 --2- 172.21.15.172:0/3604503739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17740cde10 0x7f17740ce840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1774069190 tx=0x7f177002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e55178aefa6a899 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:31.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.387+0000 7f176bfff640 1 -- 172.21.15.172:0/3604503739 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770034070 con 0x7f17740cde10 2024-04-14T08:32:31.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.387+0000 7f176bfff640 1 -- 172.21.15.172:0/3604503739 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1770030860 con 0x7f17740cde10 2024-04-14T08:32:31.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.387+0000 7f176bfff640 1 -- 172.21.15.172:0/3604503739 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770030c30 con 0x7f17740cde10 2024-04-14T08:32:31.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.387+0000 7f177b7cc640 1 -- 172.21.15.172:0/3604503739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17740cde10 msgr2=0x7f17740ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.387+0000 7f177b7cc640 1 --2- 172.21.15.172:0/3604503739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17740cde10 0x7f17740ce840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1774069190 tx=0x7f177002b9a0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.387+0000 7f177b7cc640 1 -- 172.21.15.172:0/3604503739 shutdown_connections 2024-04-14T08:32:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.387+0000 7f177b7cc640 1 --2- 172.21.15.172:0/3604503739 >> v2:172.21.15.177:3300/0 conn(0x7f17740ced80 0x7f17740afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.387+0000 7f177b7cc640 1 --2- 172.21.15.172:0/3604503739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17740cde10 0x7f17740ce840 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.387+0000 7f177b7cc640 1 -- 172.21.15.172:0/3604503739 >> 172.21.15.172:0/3604503739 conn(0x7f177408c000 msgr2=0x7f17740d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.388+0000 7f177b7cc640 1 -- 172.21.15.172:0/3604503739 shutdown_connections 2024-04-14T08:32:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.388+0000 7f177b7cc640 1 -- 172.21.15.172:0/3604503739 wait complete. 2024-04-14T08:32:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.389+0000 7f177b7cc640 1 Processor -- start 2024-04-14T08:32:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.389+0000 7f177b7cc640 1 -- start start 2024-04-14T08:32:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.389+0000 7f177b7cc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17740cde10 0x7f17741580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.389+0000 7f177b7cc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17740ced80 0x7f17740c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:31.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.389+0000 7f177b7cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f177415c710 0x7f177415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.389+0000 7f177b7cc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f177415caf0 con 0x7f177415c710 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.389+0000 7f177b7cc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f17740af890 con 0x7f17740ced80 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.389+0000 7f177b7cc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f17740b0260 con 0x7f17740cde10 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.389+0000 7f1779d69640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f177415c710 0x7f177415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1779d69640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f177415c710 0x7f177415ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36942/0 (socket says 172.21.15.172:36942) 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1779d69640 1 -- 172.21.15.172:0/3326637534 learned_addr learned my addr 172.21.15.172:0/3326637534 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1778d67640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17740ced80 0x7f17740c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1779568640 1 --2- 172.21.15.172:0/3326637534 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17740cde10 0x7f17741580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1779d69640 1 -- 172.21.15.172:0/3326637534 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17740cde10 msgr2=0x7f17741580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1779d69640 1 --2- 172.21.15.172:0/3326637534 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17740cde10 0x7f17741580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1779d69640 1 -- 172.21.15.172:0/3326637534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17740ced80 msgr2=0x7f17740c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1779d69640 1 --2- 172.21.15.172:0/3326637534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17740ced80 0x7f17740c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1779d69640 1 -- 172.21.15.172:0/3326637534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17740b7330 con 0x7f177415c710 2024-04-14T08:32:31.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1778d67640 1 --2- 172.21.15.172:0/3326637534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17740ced80 0x7f17740c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:31.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1779d69640 1 --2- 172.21.15.172:0/3326637534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f177415c710 0x7f177415ee40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f176c009dd0 tx=0x7f176c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:31.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1769ffb640 1 -- 172.21.15.172:0/3326637534 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c018020 con 0x7f177415c710 2024-04-14T08:32:31.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1769ffb640 1 -- 172.21.15.172:0/3326637534 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f176c014480 con 0x7f177415c710 2024-04-14T08:32:31.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1769ffb640 1 -- 172.21.15.172:0/3326637534 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c014870 con 0x7f177415c710 2024-04-14T08:32:31.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f177b7cc640 1 -- 172.21.15.172:0/3326637534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17740b75b0 con 0x7f177415c710 2024-04-14T08:32:31.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f177b7cc640 1 -- 172.21.15.172:0/3326637534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17740b7b70 con 0x7f177415c710 2024-04-14T08:32:31.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.390+0000 7f1769ffb640 1 -- 172.21.15.172:0/3326637534 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f176c005d10 con 0x7f177415c710 2024-04-14T08:32:31.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.391+0000 7f1769ffb640 1 -- 172.21.15.172:0/3326637534 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f176c014da0 con 0x7f177415c710 2024-04-14T08:32:31.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.391+0000 7f177b7cc640 1 -- 172.21.15.172:0/3326637534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1774075490 con 0x7f177415c710 2024-04-14T08:32:31.395 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:31.395+0000 7fa8b5430040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-14T08:32:31.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.395+0000 7f1769ffb640 1 -- 172.21.15.172:0/3326637534 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f176c022070 con 0x7f177415c710 2024-04-14T08:32:31.453 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:31.452+0000 7fc72122b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-14T08:32:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.456+0000 7f177b7cc640 1 -- 172.21.15.172:0/3326637534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17740d03e0 con 0x7f177415c710 2024-04-14T08:32:31.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.457+0000 7f1769ffb640 1 -- 172.21.15.172:0/3326637534 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f17740b7330 con 0x7f177415c710 2024-04-14T08:32:31.458 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:32:31.458 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":10,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:31.201031+0000","last_up_change":"0.000000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:32:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.458+0000 7f177b7cc640 1 -- 172.21.15.172:0/3326637534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f177415c710 msgr2=0x7f177415ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.458+0000 7f177b7cc640 1 --2- 172.21.15.172:0/3326637534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f177415c710 0x7f177415ee40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f176c009dd0 tx=0x7f176c00c490 comp rx=0 tx=0).stop 2024-04-14T08:32:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.458+0000 7f177b7cc640 1 -- 172.21.15.172:0/3326637534 shutdown_connections 2024-04-14T08:32:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.458+0000 7f177b7cc640 1 --2- 172.21.15.172:0/3326637534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f177415c710 0x7f177415ee40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.458+0000 7f177b7cc640 1 --2- 172.21.15.172:0/3326637534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17740ced80 0x7f17740c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.458+0000 7f177b7cc640 1 --2- 172.21.15.172:0/3326637534 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17740cde10 0x7f17741580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.458+0000 7f177b7cc640 1 -- 172.21.15.172:0/3326637534 >> 172.21.15.172:0/3326637534 conn(0x7f177408c000 msgr2=0x7f177406fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.458+0000 7f177b7cc640 1 -- 172.21.15.172:0/3326637534 shutdown_connections 2024-04-14T08:32:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:31.458+0000 7f177b7cc640 1 -- 172.21.15.172:0/3326637534 wait complete. 2024-04-14T08:32:31.470 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-14T08:32:31.470 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-14T08:32:31.494 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:31.494+0000 7fa8b5430040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-14T08:32:31.546 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:31.544+0000 7fc72122b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-14T08:32:31.649 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:31.648+0000 7fa8b5430040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-14T08:32:31.690 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:31.690+0000 7fa8b5430040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-14T08:32:31.698 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:31.697+0000 7fc72122b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-14T08:32:31.739 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:31.738+0000 7fc72122b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-14T08:32:31.827 INFO:tasks.ceph.osd.0.smithi172.stderr:2024-04-14T08:32:31.826+0000 7fcee9024640 -1 Falling back to public interface 2024-04-14T08:32:31.844 INFO:tasks.ceph.osd.3.smithi172.stderr:2024-04-14T08:32:31.844+0000 7fa7e139e640 -1 Falling back to public interface 2024-04-14T08:32:31.845 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:31.845+0000 7fa8b5430040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-14T08:32:31.852 INFO:tasks.ceph.osd.2.smithi172.stderr:2024-04-14T08:32:31.852+0000 7fa7fe20d640 -1 Falling back to public interface 2024-04-14T08:32:31.855 INFO:tasks.ceph.osd.1.smithi172.stderr:2024-04-14T08:32:31.854+0000 7f268f6f7640 -1 Falling back to public interface 2024-04-14T08:32:31.861 INFO:tasks.ceph.osd.6.smithi177.stderr:2024-04-14T08:32:31.860+0000 7f771cb9e640 -1 Falling back to public interface 2024-04-14T08:32:31.876 INFO:tasks.ceph.osd.7.smithi177.stderr:2024-04-14T08:32:31.875+0000 7f4daa9fa640 -1 Falling back to public interface 2024-04-14T08:32:31.888 INFO:tasks.ceph.osd.5.smithi177.stderr:2024-04-14T08:32:31.887+0000 7fcbcfe0f640 -1 Falling back to public interface 2024-04-14T08:32:31.889 INFO:tasks.ceph.osd.4.smithi177.stderr:2024-04-14T08:32:31.888+0000 7fa90b32a640 -1 Falling back to public interface 2024-04-14T08:32:31.894 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:31.893+0000 7fc72122b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-14T08:32:32.276 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-14T08:32:32.366 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.366+0000 7fbbf87cf640 1 Processor -- start 2024-04-14T08:32:32.367 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.366+0000 7fbbf87cf640 1 -- start start 2024-04-14T08:32:32.368 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf87cf640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbf0155700 0x7fbbf00afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:32.368 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf87cf640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbbf00b5310 0x7fbbf00cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:32.368 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf87cf640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf015b5c0 con 0x7fbbf00676e0 2024-04-14T08:32:32.368 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf87cf640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fbbf015c5f0 con 0x7fbbf00b5310 2024-04-14T08:32:32.368 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf87cf640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf00cdf60 con 0x7fbbf0155700 2024-04-14T08:32:32.368 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf656b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbf0155700 0x7fbbf00afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:32.368 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf6d6c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbbf00b5310 0x7fbbf00cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:32.368 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf5d6a640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fbbf00676e0 0x7fbbf00b4c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:55258/0 (socket says 172.21.15.172:55258) 2024-04-14T08:32:32.369 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf5d6a640 1 -- 172.21.15.172:0/1268742029 learned_addr learned my addr 172.21.15.172:0/1268742029 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:32:32.369 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf656b640 1 -- 172.21.15.172:0/1268742029 >> v2:172.21.15.177:3300/0 conn(0x7fbbf00b5310 msgr2=0x7fbbf00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:32.369 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf656b640 1 --2- 172.21.15.172:0/1268742029 >> v2:172.21.15.177:3300/0 conn(0x7fbbf00b5310 0x7fbbf00cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:32.369 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf656b640 1 -- 172.21.15.172:0/1268742029 >> v1:172.21.15.177:6789/0 conn(0x7fbbf00676e0 legacy=0x7fbbf00b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:32.369 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf656b640 1 -- 172.21.15.172:0/1268742029 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf0155ae0 con 0x7fbbf0155700 2024-04-14T08:32:32.369 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.367+0000 7fbbf6d6c640 1 --2- 172.21.15.172:0/1268742029 >> v2:172.21.15.177:3300/0 conn(0x7fbbf00b5310 0x7fbbf00cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:32.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.368+0000 7fbbf656b640 1 --2- 172.21.15.172:0/1268742029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbf0155700 0x7fbbf00afd30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0097a0 tx=0x7fbbec02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e0ac575b229dd2c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:32.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.368+0000 7fbbf5569640 1 -- 172.21.15.172:0/1268742029 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec0315a0 con 0x7fbbf0155700 2024-04-14T08:32:32.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.368+0000 7fbbf5569640 1 -- 172.21.15.172:0/1268742029 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbec037070 con 0x7fbbf0155700 2024-04-14T08:32:32.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.368+0000 7fbbf5569640 1 -- 172.21.15.172:0/1268742029 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec031960 con 0x7fbbf0155700 2024-04-14T08:32:32.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.369+0000 7fbbf87cf640 1 -- 172.21.15.172:0/1268742029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbf0155700 msgr2=0x7fbbf00afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:32.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.369+0000 7fbbf87cf640 1 --2- 172.21.15.172:0/1268742029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbf0155700 0x7fbbf00afd30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0097a0 tx=0x7fbbec02b840 comp rx=0 tx=0).stop 2024-04-14T08:32:32.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.369+0000 7fbbf87cf640 1 -- 172.21.15.172:0/1268742029 shutdown_connections 2024-04-14T08:32:32.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.369+0000 7fbbf87cf640 1 --2- 172.21.15.172:0/1268742029 >> v2:172.21.15.177:3300/0 conn(0x7fbbf00b5310 0x7fbbf00cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:32.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.369+0000 7fbbf87cf640 1 --2- 172.21.15.172:0/1268742029 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbf0155700 0x7fbbf00afd30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:32.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.369+0000 7fbbf87cf640 1 -- 172.21.15.172:0/1268742029 >> 172.21.15.172:0/1268742029 conn(0x7fbbf0072b50 msgr2=0x7fbbf006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:32.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.369+0000 7fbbf87cf640 1 -- 172.21.15.172:0/1268742029 shutdown_connections 2024-04-14T08:32:32.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.369+0000 7fbbf87cf640 1 -- 172.21.15.172:0/1268742029 wait complete. 2024-04-14T08:32:32.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.370+0000 7fbbf87cf640 1 Processor -- start 2024-04-14T08:32:32.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.370+0000 7fbbf87cf640 1 -- start start 2024-04-14T08:32:32.373 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf87cf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbf00676e0 0x7fbbf0159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf87cf640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbf00b5310 0x7fbbf014e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf87cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbf0155700 0x7fbbf006b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf87cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf015bb10 con 0x7fbbf0155700 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf87cf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf00b0270 con 0x7fbbf00676e0 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf87cf640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf015bdf0 con 0x7fbbf00b5310 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf6d6c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbf0155700 0x7fbbf006b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf656b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbf00676e0 0x7fbbf0159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf6d6c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbf0155700 0x7fbbf006b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36954/0 (socket says 172.21.15.172:36954) 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf6d6c640 1 -- 172.21.15.172:0/2777268981 learned_addr learned my addr 172.21.15.172:0/2777268981 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf5d6a640 1 --2- 172.21.15.172:0/2777268981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbf00b5310 0x7fbbf014e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf5d6a640 1 -- 172.21.15.172:0/2777268981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbf00676e0 msgr2=0x7fbbf0159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf5d6a640 1 --2- 172.21.15.172:0/2777268981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbf00676e0 0x7fbbf0159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:32.374 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf5d6a640 1 -- 172.21.15.172:0/2777268981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbf0155700 msgr2=0x7fbbf006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf5d6a640 1 --2- 172.21.15.172:0/2777268981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbf0155700 0x7fbbf006b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf5d6a640 1 -- 172.21.15.172:0/2777268981 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf0158090 con 0x7fbbf00b5310 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf656b640 1 --2- 172.21.15.172:0/2777268981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbf00676e0 0x7fbbf0159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.371+0000 7fbbf6d6c640 1 --2- 172.21.15.172:0/2777268981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbf0155700 0x7fbbf006b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.372+0000 7fbbf5d6a640 1 --2- 172.21.15.172:0/2777268981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbf00b5310 0x7fbbf014e500 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c370 tx=0x7fbbe000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.372+0000 7fbbe6ffd640 1 -- 172.21.15.172:0/2777268981 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0017070 con 0x7fbbf00b5310 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.372+0000 7fbbe6ffd640 1 -- 172.21.15.172:0/2777268981 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe0005ce0 con 0x7fbbf00b5310 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.372+0000 7fbbf87cf640 1 -- 172.21.15.172:0/2777268981 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf0158370 con 0x7fbbf00b5310 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.372+0000 7fbbe6ffd640 1 -- 172.21.15.172:0/2777268981 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0007310 con 0x7fbbf00b5310 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.372+0000 7fbbf87cf640 1 -- 172.21.15.172:0/2777268981 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf0158930 con 0x7fbbf00b5310 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.372+0000 7fbbe6ffd640 1 -- 172.21.15.172:0/2777268981 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbbe001a070 con 0x7fbbf00b5310 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.372+0000 7fbbe6ffd640 1 -- 172.21.15.172:0/2777268981 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fbbe0012ad0 con 0x7fbbf00b5310 2024-04-14T08:32:32.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.373+0000 7fbbf87cf640 1 -- 172.21.15.172:0/2777268981 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf0068e50 con 0x7fbbf00b5310 2024-04-14T08:32:32.376 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.375+0000 7fbbe6ffd640 1 -- 172.21.15.172:0/2777268981 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbbe0021020 con 0x7fbbf00b5310 2024-04-14T08:32:32.440 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.439+0000 7fbbf87cf640 1 -- 172.21.15.172:0/2777268981 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbf0151930 con 0x7fbbf00b5310 2024-04-14T08:32:32.440 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.440+0000 7fbbe6ffd640 1 -- 172.21.15.172:0/2777268981 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fbbe00074b0 con 0x7fbbf00b5310 2024-04-14T08:32:32.441 INFO:teuthology.misc.health.smithi172.stdout: 2024-04-14T08:32:32.441 INFO:teuthology.misc.health.smithi172.stdout:{"epoch":10,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:31.201031+0000","last_up_change":"0.000000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:32:32.442 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.441+0000 7fbbf87cf640 1 -- 172.21.15.172:0/2777268981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbf00b5310 msgr2=0x7fbbf014e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:32.442 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.441+0000 7fbbf87cf640 1 --2- 172.21.15.172:0/2777268981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbf00b5310 0x7fbbf014e500 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c370 tx=0x7fbbe000c830 comp rx=0 tx=0).stop 2024-04-14T08:32:32.442 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.441+0000 7fbbf87cf640 1 -- 172.21.15.172:0/2777268981 shutdown_connections 2024-04-14T08:32:32.442 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.441+0000 7fbbf87cf640 1 --2- 172.21.15.172:0/2777268981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbf0155700 0x7fbbf006b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:32.442 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.441+0000 7fbbf87cf640 1 --2- 172.21.15.172:0/2777268981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbbf00b5310 0x7fbbf014e500 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:32.442 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.441+0000 7fbbf87cf640 1 --2- 172.21.15.172:0/2777268981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbbf00676e0 0x7fbbf0159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:32.442 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.441+0000 7fbbf87cf640 1 -- 172.21.15.172:0/2777268981 >> 172.21.15.172:0/2777268981 conn(0x7fbbf0072b50 msgr2=0x7fbbf015d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:32.442 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.441+0000 7fbbf87cf640 1 -- 172.21.15.172:0/2777268981 shutdown_connections 2024-04-14T08:32:32.442 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:32.441+0000 7fbbf87cf640 1 -- 172.21.15.172:0/2777268981 wait complete. 2024-04-14T08:32:32.451 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-14T08:32:32.667 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:32.667+0000 7fa8b5430040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-14T08:32:32.718 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:32.716+0000 7fc72122b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-14T08:32:32.942 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:32.942+0000 7fa8b5430040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-14T08:32:32.991 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:32.990+0000 7fc72122b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-14T08:32:32.995 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:32.995+0000 7fa8b5430040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-14T08:32:33.043 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:33.042+0000 7fc72122b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-14T08:32:33.109 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:33.109+0000 7fa8b5430040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-14T08:32:33.162 INFO:tasks.ceph.mgr.x.smithi177.stderr:2024-04-14T08:32:33.160+0000 7fc72122b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-14T08:32:34.124 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:34.124+0000 7fa8557bf640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-14T08:32:34.124 INFO:tasks.ceph.mgr.y.smithi172.stderr:2024-04-14T08:32:34.124+0000 7fa8557bf640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-14T08:32:35.296 INFO:tasks.ceph.osd.0.smithi172.stderr:2024-04-14T08:32:35.296+0000 7fcee9024640 -1 osd.0 0 log_to_monitors true 2024-04-14T08:32:35.307 INFO:tasks.ceph.osd.6.smithi177.stderr:2024-04-14T08:32:35.306+0000 7f771cb9e640 -1 osd.6 0 log_to_monitors true 2024-04-14T08:32:35.338 INFO:tasks.ceph.osd.3.smithi172.stderr:2024-04-14T08:32:35.338+0000 7fa7e139e640 -1 osd.3 0 log_to_monitors true 2024-04-14T08:32:35.370 INFO:tasks.ceph.osd.4.smithi177.stderr:2024-04-14T08:32:35.369+0000 7fa90b32a640 -1 osd.4 0 log_to_monitors true 2024-04-14T08:32:35.373 INFO:tasks.ceph.osd.7.smithi177.stderr:2024-04-14T08:32:35.372+0000 7f4daa9fa640 -1 osd.7 0 log_to_monitors true 2024-04-14T08:32:35.378 INFO:tasks.ceph.osd.2.smithi172.stderr:2024-04-14T08:32:35.378+0000 7fa7fe20d640 -1 osd.2 0 log_to_monitors true 2024-04-14T08:32:35.397 INFO:tasks.ceph.osd.5.smithi177.stderr:2024-04-14T08:32:35.396+0000 7fcbcfe0f640 -1 osd.5 0 log_to_monitors true 2024-04-14T08:32:35.402 INFO:tasks.ceph.osd.1.smithi172.stderr:2024-04-14T08:32:35.402+0000 7f268f6f7640 -1 osd.1 0 log_to_monitors true 2024-04-14T08:32:39.121 INFO:tasks.ceph.osd.7.smithi177.stderr:2024-04-14T08:32:39.120+0000 7f4da65f7640 -1 osd.7 0 waiting for initial osdmap 2024-04-14T08:32:39.138 INFO:tasks.ceph.osd.7.smithi177.stderr:2024-04-14T08:32:39.138+0000 7f4d9e480640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T08:32:39.154 INFO:tasks.ceph.osd.5.smithi177.stderr:2024-04-14T08:32:39.154+0000 7fcbcbbf8640 -1 osd.5 0 waiting for initial osdmap 2024-04-14T08:32:39.163 INFO:tasks.ceph.osd.0.smithi172.stderr:2024-04-14T08:32:39.162+0000 7fcee45f7640 -1 osd.0 0 waiting for initial osdmap 2024-04-14T08:32:39.165 INFO:tasks.ceph.osd.5.smithi177.stderr:2024-04-14T08:32:39.165+0000 7fcbc3280640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T08:32:39.198 INFO:tasks.ceph.osd.6.smithi177.stderr:2024-04-14T08:32:39.197+0000 7f77187f7640 -1 osd.6 0 waiting for initial osdmap 2024-04-14T08:32:39.204 INFO:tasks.ceph.osd.0.smithi172.stderr:2024-04-14T08:32:39.204+0000 7fcedc480640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T08:32:39.208 INFO:tasks.ceph.osd.6.smithi177.stderr:2024-04-14T08:32:39.208+0000 7f7710680640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T08:32:39.226 INFO:tasks.ceph.osd.1.smithi172.stderr:2024-04-14T08:32:39.225+0000 7f268b3f7640 -1 osd.1 0 waiting for initial osdmap 2024-04-14T08:32:39.230 INFO:tasks.ceph.osd.4.smithi177.stderr:2024-04-14T08:32:39.230+0000 7fa906ff7640 -1 osd.4 0 waiting for initial osdmap 2024-04-14T08:32:39.241 INFO:tasks.ceph.osd.2.smithi172.stderr:2024-04-14T08:32:39.241+0000 7fa7f97f7640 -1 osd.2 0 waiting for initial osdmap 2024-04-14T08:32:39.245 INFO:tasks.ceph.osd.4.smithi177.stderr:2024-04-14T08:32:39.245+0000 7fa8fee80640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T08:32:39.257 INFO:tasks.ceph.osd.1.smithi172.stderr:2024-04-14T08:32:39.257+0000 7f2683280640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T08:32:39.258 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-14T08:32:39.282 INFO:tasks.ceph.osd.2.smithi172.stderr:2024-04-14T08:32:39.282+0000 7fa7f1680640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T08:32:39.361 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.360+0000 7f9f04a80640 1 Processor -- start 2024-04-14T08:32:39.361 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.360+0000 7f9f04a80640 1 -- start start 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.361+0000 7f9f04a80640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f001520e0 0x7f9f00072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.361+0000 7f9f04a80640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9f00070910 0x7f9f000b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.361+0000 7f9f04a80640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f000b4280 con 0x7f9f000b5530 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.361+0000 7f9f04a80640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f9f000b4030 con 0x7f9f001520e0 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.361+0000 7f9f04a80640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f9f000b5eb0 con 0x7f9f00070910 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.361+0000 7f9efe59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f001520e0 0x7f9f00072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.361+0000 7f9efe59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f001520e0 0x7f9f00072ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37052/0 (socket says 172.21.15.172:37052) 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.361+0000 7f9efe59c640 1 -- 172.21.15.172:0/3920285805 learned_addr learned my addr 172.21.15.172:0/3920285805 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.361+0000 7f9ef7fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9f000b5530 0x7f9f0015eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:51512/0 (socket says 172.21.15.172:51512) 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.361+0000 7f9efed9d640 1 --2- 172.21.15.172:0/3920285805 >> v2:172.21.15.177:3300/0 conn(0x7f9f00070910 0x7f9f000b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.362+0000 7f9efe59c640 1 -- 172.21.15.172:0/3920285805 >> v2:172.21.15.177:3300/0 conn(0x7f9f00070910 msgr2=0x7f9f000b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.362+0000 7f9efe59c640 1 --2- 172.21.15.172:0/3920285805 >> v2:172.21.15.177:3300/0 conn(0x7f9f00070910 0x7f9f000b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:39.363 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.362+0000 7f9efe59c640 1 -- 172.21.15.172:0/3920285805 >> v1:172.21.15.172:6789/0 conn(0x7f9f000b5530 legacy=0x7f9f0015eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:39.364 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.362+0000 7f9efdd9b640 1 -- 172.21.15.172:0/3920285805 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1444930231 0 0) 0x7f9f000b4280 con 0x7f9f000b5530 2024-04-14T08:32:39.364 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.362+0000 7f9efe59c640 1 -- 172.21.15.172:0/3920285805 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f000b4560 con 0x7f9f001520e0 2024-04-14T08:32:39.364 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.362+0000 7f9efed9d640 1 --2- 172.21.15.172:0/3920285805 >> v2:172.21.15.177:3300/0 conn(0x7f9f00070910 0x7f9f000b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:32:39.365 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.363+0000 7f9efe59c640 1 --2- 172.21.15.172:0/3920285805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f001520e0 0x7f9f00072ab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9f000d0f30 tx=0x7f9ef002e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7082fa778dd1bf server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:39.365 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.363+0000 7f9efdd9b640 1 -- 172.21.15.172:0/3920285805 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef003b070 con 0x7f9f001520e0 2024-04-14T08:32:39.365 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.363+0000 7f9efdd9b640 1 -- 172.21.15.172:0/3920285805 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef0036440 con 0x7f9f001520e0 2024-04-14T08:32:39.365 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.363+0000 7f9efdd9b640 1 -- 172.21.15.172:0/3920285805 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0036810 con 0x7f9f001520e0 2024-04-14T08:32:39.365 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.363+0000 7f9f04a80640 1 -- 172.21.15.172:0/3920285805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f001520e0 msgr2=0x7f9f00072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:39.365 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.363+0000 7f9f04a80640 1 --2- 172.21.15.172:0/3920285805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f001520e0 0x7f9f00072ab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9f000d0f30 tx=0x7f9ef002e170 comp rx=0 tx=0).stop 2024-04-14T08:32:39.365 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.364+0000 7f9f04a80640 1 -- 172.21.15.172:0/3920285805 shutdown_connections 2024-04-14T08:32:39.366 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.364+0000 7f9f04a80640 1 --2- 172.21.15.172:0/3920285805 >> v2:172.21.15.177:3300/0 conn(0x7f9f00070910 0x7f9f000b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:39.366 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.364+0000 7f9f04a80640 1 --2- 172.21.15.172:0/3920285805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f001520e0 0x7f9f00072ab0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:39.366 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.364+0000 7f9f04a80640 1 -- 172.21.15.172:0/3920285805 >> 172.21.15.172:0/3920285805 conn(0x7f9f000d0f80 msgr2=0x7f9f000d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:39.366 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.364+0000 7f9f04a80640 1 -- 172.21.15.172:0/3920285805 shutdown_connections 2024-04-14T08:32:39.366 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.364+0000 7f9f04a80640 1 -- 172.21.15.172:0/3920285805 wait complete. 2024-04-14T08:32:39.367 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.366+0000 7f9f04a80640 1 Processor -- start 2024-04-14T08:32:39.367 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.366+0000 7f9f04a80640 1 -- start start 2024-04-14T08:32:39.369 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9f04a80640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f00070910 0x7f9f001579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:39.369 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9f04a80640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f000b5530 0x7f9f001f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:39.369 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9f04a80640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f001520e0 0x7f9f00153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:39.369 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9f04a80640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9f00072ff0 con 0x7f9f00070910 2024-04-14T08:32:39.369 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9f04a80640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9f000b4b20 con 0x7f9f000b5530 2024-04-14T08:32:39.369 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9f04a80640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9f000726b0 con 0x7f9f001520e0 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9efe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f00070910 0x7f9f001579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9efe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f00070910 0x7f9f001579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37058/0 (socket says 172.21.15.172:37058) 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9efe59c640 1 -- 172.21.15.172:0/1013572894 learned_addr learned my addr 172.21.15.172:0/1013572894 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9ef7fff640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f000b5530 0x7f9f001f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9efe59c640 1 -- 172.21.15.172:0/1013572894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f001520e0 msgr2=0x7f9f00153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9efe59c640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f001520e0 0x7f9f00153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9efe59c640 1 -- 172.21.15.172:0/1013572894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f000b5530 msgr2=0x7f9f001f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9efe59c640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f000b5530 0x7f9f001f3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9efe59c640 1 -- 172.21.15.172:0/1013572894 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f001f47e0 con 0x7f9f00070910 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.367+0000 7f9efe59c640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f00070910 0x7f9f001579b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0000c00 tx=0x7f9ef00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.368+0000 7f9ef6ffd640 1 -- 172.21.15.172:0/1013572894 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0046070 con 0x7f9f00070910 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.368+0000 7f9ef6ffd640 1 -- 172.21.15.172:0/1013572894 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef003d050 con 0x7f9f00070910 2024-04-14T08:32:39.370 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.368+0000 7f9ef6ffd640 1 -- 172.21.15.172:0/1013572894 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef003b030 con 0x7f9f00070910 2024-04-14T08:32:39.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.368+0000 7f9f04a80640 1 -- 172.21.15.172:0/1013572894 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f001f4ac0 con 0x7f9f00070910 2024-04-14T08:32:39.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.368+0000 7f9f04a80640 1 -- 172.21.15.172:0/1013572894 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f00155fa0 con 0x7f9f00070910 2024-04-14T08:32:39.371 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.369+0000 7f9f04a80640 1 -- 172.21.15.172:0/1013572894 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f00068e50 con 0x7f9f00070910 2024-04-14T08:32:39.372 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.369+0000 7f9ef6ffd640 1 -- 172.21.15.172:0/1013572894 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f9ef0049070 con 0x7f9f00070910 2024-04-14T08:32:39.372 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.370+0000 7f9ef6ffd640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9ee006c860 0x7f9ee006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:39.372 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.370+0000 7f9ef6ffd640 1 -- 172.21.15.172:0/1013572894 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f9ef00ac770 con 0x7f9f00070910 2024-04-14T08:32:39.372 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.370+0000 7f9ef7fff640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9ee006c860 0x7f9ee006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:39.372 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.370+0000 7f9ef7fff640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9ee006c860 0x7f9ee006ed20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9f000b2d00 tx=0x7f9ee8007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:39.375 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.375+0000 7f9ef6ffd640 1 -- 172.21.15.172:0/1013572894 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9ef007b450 con 0x7f9f00070910 2024-04-14T08:32:39.543 INFO:tasks.ceph.osd.3.smithi172.stderr:2024-04-14T08:32:39.542+0000 7fa7dcff7640 -1 osd.3 0 waiting for initial osdmap 2024-04-14T08:32:39.552 INFO:tasks.ceph.osd.3.smithi172.stderr:2024-04-14T08:32:39.552+0000 7fa7d4e80640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T08:32:39.569 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.569+0000 7f9f04a80640 1 -- 172.21.15.172:0/1013572894 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f000b5310 con 0x7f9f00070910 2024-04-14T08:32:39.570 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.569+0000 7f9ef6ffd640 1 -- 172.21.15.172:0/1013572894 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f9ef007f750 con 0x7f9f00070910 2024-04-14T08:32:39.570 INFO:teuthology.misc.health.smithi172.stdout: 2024-04-14T08:32:39.570 INFO:teuthology.misc.health.smithi172.stdout:{"epoch":12,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:37.125673+0000","last_up_change":"0.000000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:32:39.572 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.571+0000 7f9f04a80640 1 -- 172.21.15.172:0/1013572894 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9ee006c860 msgr2=0x7f9ee006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:39.572 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.571+0000 7f9f04a80640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9ee006c860 0x7f9ee006ed20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9f000b2d00 tx=0x7f9ee8007d50 comp rx=0 tx=0).stop 2024-04-14T08:32:39.572 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.571+0000 7f9f04a80640 1 -- 172.21.15.172:0/1013572894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f00070910 msgr2=0x7f9f001579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:39.572 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.571+0000 7f9f04a80640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f00070910 0x7f9f001579b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0000c00 tx=0x7f9ef00046c0 comp rx=0 tx=0).stop 2024-04-14T08:32:39.572 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.571+0000 7f9f04a80640 1 -- 172.21.15.172:0/1013572894 shutdown_connections 2024-04-14T08:32:39.572 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.571+0000 7f9f04a80640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9ee006c860 0x7f9ee006ed20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:39.572 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.571+0000 7f9f04a80640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f001520e0 0x7f9f00153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:39.572 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.571+0000 7f9f04a80640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f000b5530 0x7f9f001f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:39.572 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.571+0000 7f9f04a80640 1 --2- 172.21.15.172:0/1013572894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f00070910 0x7f9f001579b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:39.572 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.571+0000 7f9f04a80640 1 -- 172.21.15.172:0/1013572894 >> 172.21.15.172:0/1013572894 conn(0x7f9f000d0f80 msgr2=0x7f9f000d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:39.573 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.571+0000 7f9f04a80640 1 -- 172.21.15.172:0/1013572894 shutdown_connections 2024-04-14T08:32:39.573 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:39.571+0000 7f9f04a80640 1 -- 172.21.15.172:0/1013572894 wait complete. 2024-04-14T08:32:39.581 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-14T08:32:46.388 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-14T08:32:46.489 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.489+0000 7fa3e8a88640 1 Processor -- start 2024-04-14T08:32:46.489 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.489+0000 7fa3e8a88640 1 -- start start 2024-04-14T08:32:46.491 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e8a88640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa3e415c260 0x7fa3e415fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.491 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e8a88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3e4072da0 0x7fa3e40731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.491 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e8a88640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa3e40b1c20 0x7fa3e40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e8a88640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e40b25b0 con 0x7fa3e40b1c20 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e8a88640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e415c640 con 0x7fa3e415c260 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e8a88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e40cd970 con 0x7fa3e4072da0 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e2d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa3e40b1c20 0x7fa3e40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e2d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa3e40b1c20 0x7fa3e40b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60816/0 (socket says 172.21.15.172:60816) 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e2d9d640 1 -- 172.21.15.172:0/1729027531 learned_addr learned my addr 172.21.15.172:0/1729027531 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e1d9b640 1 --2- 172.21.15.172:0/1729027531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3e4072da0 0x7fa3e40731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e259c640 1 --2- 172.21.15.172:0/1729027531 >> v2:172.21.15.177:3300/0 conn(0x7fa3e415c260 0x7fa3e415fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e2d9d640 1 -- 172.21.15.172:0/1729027531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3e4072da0 msgr2=0x7fa3e40731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e2d9d640 1 --2- 172.21.15.172:0/1729027531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3e4072da0 0x7fa3e40731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e2d9d640 1 -- 172.21.15.172:0/1729027531 >> v2:172.21.15.177:3300/0 conn(0x7fa3e415c260 msgr2=0x7fa3e415fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e2d9d640 1 --2- 172.21.15.172:0/1729027531 >> v2:172.21.15.177:3300/0 conn(0x7fa3e415c260 0x7fa3e415fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.490+0000 7fa3e2d9d640 1 -- 172.21.15.172:0/1729027531 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e40cdc50 con 0x7fa3e40b1c20 2024-04-14T08:32:46.492 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.491+0000 7fa3e2d9d640 1 --2- 172.21.15.172:0/1729027531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e40b1c20 0x7fa3e40b2070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40b5570 tx=0x7fa3dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f14a0ba8a3895c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:46.493 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.491+0000 7fa3e159a640 1 -- 172.21.15.172:0/1729027531 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc02ca70 con 0x7fa3e40b1c20 2024-04-14T08:32:46.493 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.491+0000 7fa3e159a640 1 -- 172.21.15.172:0/1729027531 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3dc02cc10 con 0x7fa3e40b1c20 2024-04-14T08:32:46.493 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.491+0000 7fa3e159a640 1 -- 172.21.15.172:0/1729027531 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0343f0 con 0x7fa3e40b1c20 2024-04-14T08:32:46.494 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.492+0000 7fa3e8a88640 1 -- 172.21.15.172:0/1729027531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e40b1c20 msgr2=0x7fa3e40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:46.494 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.492+0000 7fa3e8a88640 1 --2- 172.21.15.172:0/1729027531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e40b1c20 0x7fa3e40b2070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40b5570 tx=0x7fa3dc02bed0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.494 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.492+0000 7fa3e8a88640 1 -- 172.21.15.172:0/1729027531 shutdown_connections 2024-04-14T08:32:46.494 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.492+0000 7fa3e8a88640 1 --2- 172.21.15.172:0/1729027531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e40b1c20 0x7fa3e40b2070 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.494 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.492+0000 7fa3e8a88640 1 --2- 172.21.15.172:0/1729027531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3e4072da0 0x7fa3e40731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.494 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.492+0000 7fa3e8a88640 1 --2- 172.21.15.172:0/1729027531 >> v2:172.21.15.177:3300/0 conn(0x7fa3e415c260 0x7fa3e415fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.494 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.492+0000 7fa3e8a88640 1 -- 172.21.15.172:0/1729027531 >> 172.21.15.172:0/1729027531 conn(0x7fa3e406ebe0 msgr2=0x7fa3e406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:46.494 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.492+0000 7fa3e8a88640 1 -- 172.21.15.172:0/1729027531 shutdown_connections 2024-04-14T08:32:46.494 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.492+0000 7fa3e8a88640 1 -- 172.21.15.172:0/1729027531 wait complete. 2024-04-14T08:32:46.494 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.493+0000 7fa3e8a88640 1 Processor -- start 2024-04-14T08:32:46.494 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.493+0000 7fa3e8a88640 1 -- start start 2024-04-14T08:32:46.497 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e8a88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3e4072da0 0x7fa3e414ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.497 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e8a88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3e40b1c20 0x7fa3e40aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.497 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e8a88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e415c260 0x7fa3e414f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.497 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e8a88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e40ce4f0 con 0x7fa3e415c260 2024-04-14T08:32:46.497 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e8a88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e40ce210 con 0x7fa3e4072da0 2024-04-14T08:32:46.497 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e8a88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e41602d0 con 0x7fa3e40b1c20 2024-04-14T08:32:46.497 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e415c260 0x7fa3e414f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.497 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e415c260 0x7fa3e414f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60822/0 (socket says 172.21.15.172:60822) 2024-04-14T08:32:46.497 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e2d9d640 1 -- 172.21.15.172:0/2339220832 learned_addr learned my addr 172.21.15.172:0/2339220832 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:46.497 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e259c640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3e4072da0 0x7fa3e414ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.497 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e1d9b640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3e40b1c20 0x7fa3e40aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.497 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e2d9d640 1 -- 172.21.15.172:0/2339220832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3e40b1c20 msgr2=0x7fa3e40aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e2d9d640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3e40b1c20 0x7fa3e40aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e2d9d640 1 -- 172.21.15.172:0/2339220832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3e4072da0 msgr2=0x7fa3e414ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e2d9d640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3e4072da0 0x7fa3e414ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e2d9d640 1 -- 172.21.15.172:0/2339220832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e4151a00 con 0x7fa3e415c260 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.494+0000 7fa3e2d9d640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e415c260 0x7fa3e414f590 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009f40 tx=0x7fa3dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.495+0000 7fa3d2ffd640 1 -- 172.21.15.172:0/2339220832 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc043020 con 0x7fa3e415c260 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.495+0000 7fa3d2ffd640 1 -- 172.21.15.172:0/2339220832 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3dc03e450 con 0x7fa3e415c260 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.495+0000 7fa3e8a88640 1 -- 172.21.15.172:0/2339220832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e4151ce0 con 0x7fa3e415c260 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.495+0000 7fa3d2ffd640 1 -- 172.21.15.172:0/2339220832 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc009510 con 0x7fa3e415c260 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.495+0000 7fa3e8a88640 1 -- 172.21.15.172:0/2339220832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e41522a0 con 0x7fa3e415c260 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.496+0000 7fa3e8a88640 1 -- 172.21.15.172:0/2339220832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e4068e50 con 0x7fa3e415c260 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.496+0000 7fa3d2ffd640 1 -- 172.21.15.172:0/2339220832 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3dc046020 con 0x7fa3e415c260 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.496+0000 7fa3d2ffd640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa3c406c8f0 0x7fa3c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.498 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.496+0000 7fa3e259c640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa3c406c8f0 0x7fa3c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.499 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.496+0000 7fa3d2ffd640 1 -- 172.21.15.172:0/2339220832 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fa3dc0ade90 con 0x7fa3e415c260 2024-04-14T08:32:46.499 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.497+0000 7fa3e259c640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa3c406c8f0 0x7fa3c406edb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40079a0 tx=0x7fa3d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:46.502 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.501+0000 7fa3d2ffd640 1 -- 172.21.15.172:0/2339220832 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa3dc04d030 con 0x7fa3e415c260 2024-04-14T08:32:46.677 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.676+0000 7fa3e8a88640 1 -- 172.21.15.172:0/2339220832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3e4156090 con 0x7fa3e415c260 2024-04-14T08:32:46.677 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.677+0000 7fa3d2ffd640 1 -- 172.21.15.172:0/2339220832 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12944 (secure 0 0 0) 0x7fa3dc07bfc0 con 0x7fa3e415c260 2024-04-14T08:32:46.678 INFO:teuthology.misc.health.smithi172.stdout: 2024-04-14T08:32:46.678 INFO:teuthology.misc.health.smithi172.stdout:{"epoch":17,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:44.140937+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:32:46.680 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.679+0000 7fa3e8a88640 1 -- 172.21.15.172:0/2339220832 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa3c406c8f0 msgr2=0x7fa3c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:46.680 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.679+0000 7fa3e8a88640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa3c406c8f0 0x7fa3c406edb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40079a0 tx=0x7fa3d4008040 comp rx=0 tx=0).stop 2024-04-14T08:32:46.680 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.679+0000 7fa3e8a88640 1 -- 172.21.15.172:0/2339220832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e415c260 msgr2=0x7fa3e414f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:46.680 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.679+0000 7fa3e8a88640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e415c260 0x7fa3e414f590 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009f40 tx=0x7fa3dc004a40 comp rx=0 tx=0).stop 2024-04-14T08:32:46.680 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.679+0000 7fa3e8a88640 1 -- 172.21.15.172:0/2339220832 shutdown_connections 2024-04-14T08:32:46.680 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.679+0000 7fa3e8a88640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa3c406c8f0 0x7fa3c406edb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.680 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.679+0000 7fa3e8a88640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3e415c260 0x7fa3e414f590 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.681 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.679+0000 7fa3e8a88640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3e40b1c20 0x7fa3e40aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.681 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.679+0000 7fa3e8a88640 1 --2- 172.21.15.172:0/2339220832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3e4072da0 0x7fa3e414ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.681 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.679+0000 7fa3e8a88640 1 -- 172.21.15.172:0/2339220832 >> 172.21.15.172:0/2339220832 conn(0x7fa3e406ebe0 msgr2=0x7fa3e40b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:46.681 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.679+0000 7fa3e8a88640 1 -- 172.21.15.172:0/2339220832 shutdown_connections 2024-04-14T08:32:46.681 INFO:teuthology.misc.health.smithi172.stderr:2024-04-14T08:32:46.679+0000 7fa3e8a88640 1 -- 172.21.15.172:0/2339220832 wait complete. 2024-04-14T08:32:46.690 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-14T08:32:46.690 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-14T08:32:46.690 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-14T08:32:46.690 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-14T08:32:46.692 INFO:tasks.ceph.mds.a:Started 2024-04-14T08:32:46.693 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-14T08:32:46.693 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-14T08:32:46.695 INFO:tasks.ceph.mds.c:Started 2024-04-14T08:32:46.695 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-14T08:32:46.695 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-14T08:32:46.698 INFO:tasks.ceph.mds.b:Started 2024-04-14T08:32:46.698 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-14T08:32:46.698 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-14T08:32:46.700 INFO:tasks.ceph.mds.d:Started 2024-04-14T08:32:46.700 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-14T08:32:46.700 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:32:46.769 INFO:tasks.ceph.mds.a.smithi172.stdout:starting mds.a at 2024-04-14T08:32:46.772 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:32:46.784 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-04-14T08:32:46.784 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-04-14T08:32:46.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.806+0000 7f0d5000f640 1 Processor -- start 2024-04-14T08:32:46.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.807+0000 7f0d5000f640 1 -- start start 2024-04-14T08:32:46.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.807+0000 7f0d5000f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d4815b250 0x7f0d48072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.807+0000 7f0d5000f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0d480c6470 0x7f0d4806f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.807+0000 7f0d5000f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d4806ec50 con 0x7f0d480c69c0 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.807+0000 7f0d5000f640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0d4806fee0 con 0x7f0d480c6470 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.807+0000 7f0d5000f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0d4806eec0 con 0x7f0d4815b250 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.807+0000 7f0d4ddab640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d4815b250 0x7f0d48072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.808+0000 7f0d4d5aa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0d480c6470 0x7f0d4806f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.808+0000 7f0d4ddab640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d4815b250 0x7f0d48072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:41224/0 (socket says 172.21.15.172:41224) 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.808+0000 7f0d4ddab640 1 -- 172.21.15.172:0/4008298771 learned_addr learned my addr 172.21.15.172:0/4008298771 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.808+0000 7f0d4d5aa640 1 -- 172.21.15.172:0/4008298771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d4815b250 msgr2=0x7f0d48072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.808+0000 7f0d4d5aa640 1 --2- 172.21.15.172:0/4008298771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d4815b250 0x7f0d48072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.808+0000 7f0d4d5aa640 1 -- 172.21.15.172:0/4008298771 >> v1:172.21.15.172:6789/0 conn(0x7f0d480c69c0 legacy=0x7f0d480cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.808+0000 7f0d4d5aa640 1 -- 172.21.15.172:0/4008298771 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4806f1a0 con 0x7f0d480c6470 2024-04-14T08:32:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.808+0000 7f0d4cda9640 1 -- 172.21.15.172:0/4008298771 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3728854441 0 0) 0x7f0d4806ec50 con 0x7f0d480c69c0 2024-04-14T08:32:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.809+0000 7f0d4d5aa640 1 --2- 172.21.15.172:0/4008298771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d480c6470 0x7f0d4806f9a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0d380097a0 tx=0x7f0d3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7c87a20b697573 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.809+0000 7f0d4cda9640 1 -- 172.21.15.172:0/4008298771 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38035020 con 0x7f0d480c6470 2024-04-14T08:32:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.809+0000 7f0d4cda9640 1 -- 172.21.15.172:0/4008298771 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d38031830 con 0x7f0d480c6470 2024-04-14T08:32:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.809+0000 7f0d4cda9640 1 -- 172.21.15.172:0/4008298771 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38031c00 con 0x7f0d480c6470 2024-04-14T08:32:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.809+0000 7f0d5000f640 1 -- 172.21.15.172:0/4008298771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d480c6470 msgr2=0x7f0d4806f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.809+0000 7f0d5000f640 1 --2- 172.21.15.172:0/4008298771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d480c6470 0x7f0d4806f9a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0d380097a0 tx=0x7f0d3802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.809+0000 7f0d5000f640 1 -- 172.21.15.172:0/4008298771 shutdown_connections 2024-04-14T08:32:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.809+0000 7f0d5000f640 1 --2- 172.21.15.172:0/4008298771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d480c6470 0x7f0d4806f9a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.809+0000 7f0d5000f640 1 --2- 172.21.15.172:0/4008298771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d4815b250 0x7f0d48072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.809+0000 7f0d5000f640 1 -- 172.21.15.172:0/4008298771 >> 172.21.15.172:0/4008298771 conn(0x7f0d480d0f30 msgr2=0x7f0d480d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.809+0000 7f0d5000f640 1 -- 172.21.15.172:0/4008298771 shutdown_connections 2024-04-14T08:32:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.809+0000 7f0d5000f640 1 -- 172.21.15.172:0/4008298771 wait complete. 2024-04-14T08:32:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.810+0000 7f0d5000f640 1 Processor -- start 2024-04-14T08:32:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.810+0000 7f0d5000f640 1 -- start start 2024-04-14T08:32:46.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d5000f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d480c6470 0x7f0d48154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d5000f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d480c69c0 0x7f0d480b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d5000f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d4815b250 0x7f0d480b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d5000f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48073390 con 0x7f0d480c69c0 2024-04-14T08:32:46.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d5000f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d480c5fd0 con 0x7f0d4815b250 2024-04-14T08:32:46.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d5000f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0d4815b630 con 0x7f0d480c6470 2024-04-14T08:32:46.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d4d5aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d480c69c0 0x7f0d480b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d4d5aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d480c69c0 0x7f0d480b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60856/0 (socket says 172.21.15.172:60856) 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d4d5aa640 1 -- 172.21.15.172:0/56207069 learned_addr learned my addr 172.21.15.172:0/56207069 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d4e5ac640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d4815b250 0x7f0d480b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d4ddab640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d480c6470 0x7f0d48154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d4d5aa640 1 -- 172.21.15.172:0/56207069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d480c6470 msgr2=0x7f0d48154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d4d5aa640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d480c6470 0x7f0d48154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.811+0000 7f0d4d5aa640 1 -- 172.21.15.172:0/56207069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d4815b250 msgr2=0x7f0d480b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.812+0000 7f0d4d5aa640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d4815b250 0x7f0d480b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.812+0000 7f0d4d5aa640 1 -- 172.21.15.172:0/56207069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d481f4640 con 0x7f0d480c69c0 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.812+0000 7f0d4d5aa640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d480c69c0 0x7f0d480b5a90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000c00 tx=0x7f0d380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.812+0000 7f0d3e7fc640 1 -- 172.21.15.172:0/56207069 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38035070 con 0x7f0d480c69c0 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.812+0000 7f0d3e7fc640 1 -- 172.21.15.172:0/56207069 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d38004230 con 0x7f0d480c69c0 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.812+0000 7f0d5000f640 1 -- 172.21.15.172:0/56207069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d481f05e0 con 0x7f0d480c69c0 2024-04-14T08:32:46.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.812+0000 7f0d3e7fc640 1 -- 172.21.15.172:0/56207069 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3803a630 con 0x7f0d480c69c0 2024-04-14T08:32:46.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.812+0000 7f0d5000f640 1 -- 172.21.15.172:0/56207069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d481f0ba0 con 0x7f0d480c69c0 2024-04-14T08:32:46.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.813+0000 7f0d5000f640 1 -- 172.21.15.172:0/56207069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d481f03e0 con 0x7f0d480c69c0 2024-04-14T08:32:46.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.813+0000 7f0d3e7fc640 1 -- 172.21.15.172:0/56207069 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d38038020 con 0x7f0d480c69c0 2024-04-14T08:32:46.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.814+0000 7f0d3e7fc640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0d2c06c8f0 0x7f0d2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:46.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.814+0000 7f0d3e7fc640 1 -- 172.21.15.172:0/56207069 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f0d380ad3b0 con 0x7f0d480c69c0 2024-04-14T08:32:46.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.814+0000 7f0d4ddab640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0d2c06c8f0 0x7f0d2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:46.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.814+0000 7f0d4ddab640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0d2c06c8f0 0x7f0d2c06edb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d4809bc90 tx=0x7f0d4400b450 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.818+0000 7f0d3e7fc640 1 -- 172.21.15.172:0/56207069 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0d38040070 con 0x7f0d480c69c0 2024-04-14T08:32:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.995+0000 7f0d5000f640 1 -- 172.21.15.172:0/56207069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d48075490 con 0x7f0d480c69c0 2024-04-14T08:32:46.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.996+0000 7f0d3e7fc640 1 -- 172.21.15.172:0/56207069 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12944 (secure 0 0 0) 0x7f0d3807bbc0 con 0x7f0d480c69c0 2024-04-14T08:32:46.997 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:32:46.997 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":17,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:44.140937+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:32:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.998+0000 7f0d5000f640 1 -- 172.21.15.172:0/56207069 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0d2c06c8f0 msgr2=0x7f0d2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.998+0000 7f0d5000f640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0d2c06c8f0 0x7f0d2c06edb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d4809bc90 tx=0x7f0d4400b450 comp rx=0 tx=0).stop 2024-04-14T08:32:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.998+0000 7f0d5000f640 1 -- 172.21.15.172:0/56207069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d480c69c0 msgr2=0x7f0d480b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.998+0000 7f0d5000f640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d480c69c0 0x7f0d480b5a90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000c00 tx=0x7f0d380046c0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.998+0000 7f0d5000f640 1 -- 172.21.15.172:0/56207069 shutdown_connections 2024-04-14T08:32:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.998+0000 7f0d5000f640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0d2c06c8f0 0x7f0d2c06edb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.998+0000 7f0d5000f640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d4815b250 0x7f0d480b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.998+0000 7f0d5000f640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d480c69c0 0x7f0d480b5a90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.998+0000 7f0d5000f640 1 --2- 172.21.15.172:0/56207069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d480c6470 0x7f0d48154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.998+0000 7f0d5000f640 1 -- 172.21.15.172:0/56207069 >> 172.21.15.172:0/56207069 conn(0x7f0d480d0f30 msgr2=0x7f0d480cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.998+0000 7f0d5000f640 1 -- 172.21.15.172:0/56207069 shutdown_connections 2024-04-14T08:32:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:46.998+0000 7f0d5000f640 1 -- 172.21.15.172:0/56207069 wait complete. 2024-04-14T08:32:47.010 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:32:47.010 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:32:47.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.113+0000 7fc94071e640 1 Processor -- start 2024-04-14T08:32:47.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.113+0000 7fc94071e640 1 -- start start 2024-04-14T08:32:47.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.114+0000 7fc94071e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc9380d1a10 0x7fc93815eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.114+0000 7fc94071e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9380d0860 con 0x7fc9380ccdc0 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.114+0000 7fc94071e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9380d0b40 con 0x7fc93815f020 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.114+0000 7fc94071e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc9380d0610 con 0x7fc9380d1a10 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.114+0000 7fc93e4ba640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc9380ccdc0 0x7fc9380cd1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:39916/0 (socket says 172.21.15.172:39916) 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.114+0000 7fc93e4ba640 1 -- 172.21.15.172:0/1055950752 learned_addr learned my addr 172.21.15.172:0/1055950752 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.114+0000 7fc93dcb9640 1 --2- 172.21.15.172:0/1055950752 >> v2:172.21.15.177:3300/0 conn(0x7fc9380d1a10 0x7fc93815eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.114+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325115866 0 0) 0x7fc9380d0860 con 0x7fc9380ccdc0 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc924003520 con 0x7fc9380ccdc0 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3398598804 0 0) 0x7fc924003520 con 0x7fc9380ccdc0 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 >> v2:172.21.15.177:3300/0 conn(0x7fc9380d1a10 msgr2=0x7fc93815eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 --2- 172.21.15.172:0/1055950752 >> v2:172.21.15.177:3300/0 conn(0x7fc9380d1a10 0x7fc93815eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 >> v1:172.21.15.177:6789/0 conn(0x7fc93815f020 legacy=0x7fc9380cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9380cb730 con 0x7fc9380ccdc0 2024-04-14T08:32:47.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fc934001670 con 0x7fc9380ccdc0 2024-04-14T08:32:47.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93dcb9640 1 --2- 172.21.15.172:0/1055950752 >> v2:172.21.15.177:3300/0 conn(0x7fc9380d1a10 0x7fc93815eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:47.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 >> v1:172.21.15.172:6789/0 conn(0x7fc9380ccdc0 legacy=0x7fc9380cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:47.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 --2- 172.21.15.172:0/1055950752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9240039b0 0x7fc924003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 --2- 172.21.15.172:0/1055950752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc924004800 0x7fc924004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 --2- 172.21.15.172:0/1055950752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9240051c0 0x7fc92400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc9380d0610 con 0x7fc9240039b0 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc92400c160 con 0x7fc924004800 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc92400c360 con 0x7fc9240051c0 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93e4ba640 1 --2- 172.21.15.172:0/1055950752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9240039b0 0x7fc924003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93dcb9640 1 --2- 172.21.15.172:0/1055950752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc924004800 0x7fc924004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93ecbb640 1 --2- 172.21.15.172:0/1055950752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9240051c0 0x7fc92400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93e4ba640 1 -- 172.21.15.172:0/1055950752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9240051c0 msgr2=0x7fc92400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93e4ba640 1 --2- 172.21.15.172:0/1055950752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9240051c0 0x7fc92400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93e4ba640 1 -- 172.21.15.172:0/1055950752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc924004800 msgr2=0x7fc924004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93e4ba640 1 --2- 172.21.15.172:0/1055950752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc924004800 0x7fc924004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93e4ba640 1 -- 172.21.15.172:0/1055950752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc924003520 con 0x7fc9240039b0 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93ecbb640 1 --2- 172.21.15.172:0/1055950752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9240051c0 0x7fc92400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.115+0000 7fc93ecbb640 1 -- 172.21.15.172:0/1055950752 reap_dead start 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.116+0000 7fc93e4ba640 1 --2- 172.21.15.172:0/1055950752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9240039b0 0x7fc924003dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c210 tx=0x7fc924005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e618704684884609 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:47.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.116+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934007d30 con 0x7fc9240039b0 2024-04-14T08:32:47.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.116+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc934030070 con 0x7fc9240039b0 2024-04-14T08:32:47.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.116+0000 7fc93d4b8640 1 -- 172.21.15.172:0/1055950752 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93402cda0 con 0x7fc9240039b0 2024-04-14T08:32:47.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.117+0000 7fc94071e640 1 -- 172.21.15.172:0/1055950752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9240039b0 msgr2=0x7fc924003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:47.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.117+0000 7fc94071e640 1 --2- 172.21.15.172:0/1055950752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9240039b0 0x7fc924003dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c210 tx=0x7fc924005830 comp rx=0 tx=0).stop 2024-04-14T08:32:47.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.117+0000 7fc94071e640 1 -- 172.21.15.172:0/1055950752 shutdown_connections 2024-04-14T08:32:47.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.117+0000 7fc94071e640 1 --2- 172.21.15.172:0/1055950752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9240039b0 0x7fc924003dd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.117+0000 7fc94071e640 1 -- 172.21.15.172:0/1055950752 >> 172.21.15.172:0/1055950752 conn(0x7fc9380d0f20 msgr2=0x7fc9380d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:47.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.117+0000 7fc94071e640 1 -- 172.21.15.172:0/1055950752 shutdown_connections 2024-04-14T08:32:47.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.117+0000 7fc94071e640 1 -- 172.21.15.172:0/1055950752 wait complete. 2024-04-14T08:32:47.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.118+0000 7fc94071e640 1 Processor -- start 2024-04-14T08:32:47.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.118+0000 7fc94071e640 1 -- start start 2024-04-14T08:32:47.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc94071e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9240039b0 0x7fc9380af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc94071e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9380a1e80 0x7fc93815d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc94071e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9380c9fd0 0x7fc9380ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc94071e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc9380d0340 con 0x7fc9380a1e80 2024-04-14T08:32:47.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc94071e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9380d01c0 con 0x7fc9380c9fd0 2024-04-14T08:32:47.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc94071e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc9380ced00 con 0x7fc9240039b0 2024-04-14T08:32:47.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc93dcb9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9380a1e80 0x7fc93815d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc93dcb9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9380a1e80 0x7fc93815d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60862/0 (socket says 172.21.15.172:60862) 2024-04-14T08:32:47.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc93dcb9640 1 -- 172.21.15.172:0/1831734392 learned_addr learned my addr 172.21.15.172:0/1831734392 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:47.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc93e4ba640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9240039b0 0x7fc9380af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc93ecbb640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9380c9fd0 0x7fc9380ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc93dcb9640 1 -- 172.21.15.172:0/1831734392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9240039b0 msgr2=0x7fc9380af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:47.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc93dcb9640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9240039b0 0x7fc9380af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc93dcb9640 1 -- 172.21.15.172:0/1831734392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9380c9fd0 msgr2=0x7fc9380ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:47.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc93dcb9640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9380c9fd0 0x7fc9380ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc93dcb9640 1 -- 172.21.15.172:0/1831734392 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9380c8aa0 con 0x7fc9380a1e80 2024-04-14T08:32:47.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc93dcb9640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9380a1e80 0x7fc93815d1b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc928011cd0 tx=0x7fc92800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:47.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc92effd640 1 -- 172.21.15.172:0/1831734392 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92800fcc0 con 0x7fc9380a1e80 2024-04-14T08:32:47.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc92effd640 1 -- 172.21.15.172:0/1831734392 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92800fe60 con 0x7fc9380a1e80 2024-04-14T08:32:47.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc94071e640 1 -- 172.21.15.172:0/1831734392 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9380c8d80 con 0x7fc9380a1e80 2024-04-14T08:32:47.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc92effd640 1 -- 172.21.15.172:0/1831734392 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928014680 con 0x7fc9380a1e80 2024-04-14T08:32:47.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.119+0000 7fc94071e640 1 -- 172.21.15.172:0/1831734392 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9380c9340 con 0x7fc9380a1e80 2024-04-14T08:32:47.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.120+0000 7fc94071e640 1 -- 172.21.15.172:0/1831734392 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc93815e6c0 con 0x7fc9380a1e80 2024-04-14T08:32:47.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.121+0000 7fc92effd640 1 -- 172.21.15.172:0/1831734392 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc928012040 con 0x7fc9380a1e80 2024-04-14T08:32:47.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.121+0000 7fc92effd640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc91c06c920 0x7fc91c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.121+0000 7fc92effd640 1 -- 172.21.15.172:0/1831734392 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fc92808bd60 con 0x7fc9380a1e80 2024-04-14T08:32:47.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.121+0000 7fc93e4ba640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc91c06c920 0x7fc91c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.121+0000 7fc93e4ba640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc91c06c920 0x7fc91c06ede0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc934034000 tx=0x7fc934009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:47.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.126+0000 7fc92effd640 1 -- 172.21.15.172:0/1831734392 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc93815e6c0 con 0x7fc9380a1e80 2024-04-14T08:32:47.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.301+0000 7fc94071e640 1 -- 172.21.15.172:0/1831734392 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc938075490 con 0x7fc9380a1e80 2024-04-14T08:32:47.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.302+0000 7fc92effd640 1 -- 172.21.15.172:0/1831734392 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fc928019030 con 0x7fc9380a1e80 2024-04-14T08:32:47.303 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:32:47.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.304+0000 7fc94071e640 1 -- 172.21.15.172:0/1831734392 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc91c06c920 msgr2=0x7fc91c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:47.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.304+0000 7fc94071e640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc91c06c920 0x7fc91c06ede0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc934034000 tx=0x7fc934009030 comp rx=0 tx=0).stop 2024-04-14T08:32:47.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.304+0000 7fc94071e640 1 -- 172.21.15.172:0/1831734392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9380a1e80 msgr2=0x7fc93815d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:47.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.304+0000 7fc94071e640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9380a1e80 0x7fc93815d1b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc928011cd0 tx=0x7fc92800f610 comp rx=0 tx=0).stop 2024-04-14T08:32:47.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.304+0000 7fc94071e640 1 -- 172.21.15.172:0/1831734392 shutdown_connections 2024-04-14T08:32:47.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.304+0000 7fc94071e640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc91c06c920 0x7fc91c06ede0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.304+0000 7fc94071e640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9380c9fd0 0x7fc9380ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.304+0000 7fc94071e640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9380a1e80 0x7fc93815d1b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.304+0000 7fc94071e640 1 --2- 172.21.15.172:0/1831734392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9240039b0 0x7fc9380af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.304+0000 7fc94071e640 1 -- 172.21.15.172:0/1831734392 >> 172.21.15.172:0/1831734392 conn(0x7fc9380d0f20 msgr2=0x7fc9380b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:47.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.304+0000 7fc94071e640 1 -- 172.21.15.172:0/1831734392 shutdown_connections 2024-04-14T08:32:47.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.304+0000 7fc94071e640 1 -- 172.21.15.172:0/1831734392 wait complete. 2024-04-14T08:32:47.315 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:32:47.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.416+0000 7fda9371e640 1 Processor -- start 2024-04-14T08:32:47.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.416+0000 7fda9371e640 1 -- start start 2024-04-14T08:32:47.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.417+0000 7fda9371e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fda8c15e390 0x7fda8c0d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.417+0000 7fda9371e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fda8c15f3a0 0x7fda8c0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.417+0000 7fda9371e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda8c15e770 con 0x7fda8c0b4260 2024-04-14T08:32:47.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.417+0000 7fda9371e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fda8c0d0080 con 0x7fda8c15f3a0 2024-04-14T08:32:47.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.417+0000 7fda9371e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fda8c06d640 con 0x7fda8c15e390 2024-04-14T08:32:47.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.417+0000 7fda90cb9640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fda8c15f3a0 0x7fda8c0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.417+0000 7fda90cb9640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fda8c15f3a0 0x7fda8c0b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60868/0 (socket says 172.21.15.172:60868) 2024-04-14T08:32:47.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.417+0000 7fda90cb9640 1 -- 172.21.15.172:0/2591158818 learned_addr learned my addr 172.21.15.172:0/2591158818 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:47.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.417+0000 7fda90cb9640 1 -- 172.21.15.172:0/2591158818 >> v2:172.21.15.177:3300/0 conn(0x7fda8c15e390 msgr2=0x7fda8c0d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:47.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.417+0000 7fda90cb9640 1 --2- 172.21.15.172:0/2591158818 >> v2:172.21.15.177:3300/0 conn(0x7fda8c15e390 0x7fda8c0d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.417+0000 7fda90cb9640 1 -- 172.21.15.172:0/2591158818 >> v1:172.21.15.177:6789/0 conn(0x7fda8c0b4260 legacy=0x7fda8c161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:47.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.417+0000 7fda90cb9640 1 -- 172.21.15.172:0/2591158818 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c06dba0 con 0x7fda8c15f3a0 2024-04-14T08:32:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.418+0000 7fda90cb9640 1 --2- 172.21.15.172:0/2591158818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda8c15f3a0 0x7fda8c0b3d20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda8c070690 tx=0x7fda7c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3c80d8156cc825 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.418+0000 7fda83fff640 1 -- 172.21.15.172:0/2591158818 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c02e070 con 0x7fda8c15f3a0 2024-04-14T08:32:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.418+0000 7fda83fff640 1 -- 172.21.15.172:0/2591158818 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7c037070 con 0x7fda8c15f3a0 2024-04-14T08:32:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.418+0000 7fda83fff640 1 -- 172.21.15.172:0/2591158818 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c031a00 con 0x7fda8c15f3a0 2024-04-14T08:32:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.419+0000 7fda9371e640 1 -- 172.21.15.172:0/2591158818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda8c15f3a0 msgr2=0x7fda8c0b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:47.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.419+0000 7fda9371e640 1 --2- 172.21.15.172:0/2591158818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda8c15f3a0 0x7fda8c0b3d20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda8c070690 tx=0x7fda7c02b880 comp rx=0 tx=0).stop 2024-04-14T08:32:47.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.419+0000 7fda9371e640 1 -- 172.21.15.172:0/2591158818 shutdown_connections 2024-04-14T08:32:47.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.419+0000 7fda9371e640 1 --2- 172.21.15.172:0/2591158818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda8c15f3a0 0x7fda8c0b3d20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.419+0000 7fda9371e640 1 --2- 172.21.15.172:0/2591158818 >> v2:172.21.15.177:3300/0 conn(0x7fda8c15e390 0x7fda8c0d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.419+0000 7fda9371e640 1 -- 172.21.15.172:0/2591158818 >> 172.21.15.172:0/2591158818 conn(0x7fda8c06f820 msgr2=0x7fda8c06fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:47.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.419+0000 7fda9371e640 1 -- 172.21.15.172:0/2591158818 shutdown_connections 2024-04-14T08:32:47.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.419+0000 7fda9371e640 1 -- 172.21.15.172:0/2591158818 wait complete. 2024-04-14T08:32:47.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.420+0000 7fda9371e640 1 Processor -- start 2024-04-14T08:32:47.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.420+0000 7fda9371e640 1 -- start start 2024-04-14T08:32:47.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda9371e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda8c0b4260 0x7fda8c1531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda9371e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda8c15e390 0x7fda8c155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda9371e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda8c15f3a0 0x7fda8c154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda9371e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c15f800 con 0x7fda8c0b4260 2024-04-14T08:32:47.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda9371e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c0b4690 con 0x7fda8c15f3a0 2024-04-14T08:32:47.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda9371e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fda8c0d0510 con 0x7fda8c15e390 2024-04-14T08:32:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda914ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda8c0b4260 0x7fda8c1531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda914ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda8c0b4260 0x7fda8c1531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60882/0 (socket says 172.21.15.172:60882) 2024-04-14T08:32:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda914ba640 1 -- 172.21.15.172:0/4198152200 learned_addr learned my addr 172.21.15.172:0/4198152200 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda914ba640 1 -- 172.21.15.172:0/4198152200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda8c15e390 msgr2=0x7fda8c155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda90cb9640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda8c15e390 0x7fda8c155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda91cbb640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda8c15f3a0 0x7fda8c154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda914ba640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda8c15e390 0x7fda8c155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda914ba640 1 -- 172.21.15.172:0/4198152200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda8c15f3a0 msgr2=0x7fda8c154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:47.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda914ba640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda8c15f3a0 0x7fda8c154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda914ba640 1 -- 172.21.15.172:0/4198152200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c155790 con 0x7fda8c0b4260 2024-04-14T08:32:47.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda90cb9640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda8c15e390 0x7fda8c155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:32:47.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda91cbb640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda8c15f3a0 0x7fda8c154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:32:47.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda914ba640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda8c0b4260 0x7fda8c1531f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda8800b410 tx=0x7fda8800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:47.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda81ffb640 1 -- 172.21.15.172:0/4198152200 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88015020 con 0x7fda8c0b4260 2024-04-14T08:32:47.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda81ffb640 1 -- 172.21.15.172:0/4198152200 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda88011850 con 0x7fda8c0b4260 2024-04-14T08:32:47.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.421+0000 7fda9371e640 1 -- 172.21.15.172:0/4198152200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8c152280 con 0x7fda8c0b4260 2024-04-14T08:32:47.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.422+0000 7fda81ffb640 1 -- 172.21.15.172:0/4198152200 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88011c40 con 0x7fda8c0b4260 2024-04-14T08:32:47.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.422+0000 7fda9371e640 1 -- 172.21.15.172:0/4198152200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8c155a30 con 0x7fda8c0b4260 2024-04-14T08:32:47.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.423+0000 7fda9371e640 1 -- 172.21.15.172:0/4198152200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8c0d1680 con 0x7fda8c0b4260 2024-04-14T08:32:47.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.423+0000 7fda81ffb640 1 -- 172.21.15.172:0/4198152200 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda88018020 con 0x7fda8c0b4260 2024-04-14T08:32:47.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.423+0000 7fda81ffb640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda7006c8f0 0x7fda7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:47.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.423+0000 7fda90cb9640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda7006c8f0 0x7fda7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:47.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.424+0000 7fda81ffb640 1 -- 172.21.15.172:0/4198152200 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fda8808b630 con 0x7fda8c0b4260 2024-04-14T08:32:47.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.424+0000 7fda90cb9640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda7006c8f0 0x7fda7006edb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fda7c036000 tx=0x7fda7c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:47.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.428+0000 7fda81ffb640 1 -- 172.21.15.172:0/4198152200 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fda8c0d1680 con 0x7fda8c0b4260 2024-04-14T08:32:47.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.639+0000 7fda9371e640 1 -- 172.21.15.172:0/4198152200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda8c068520 con 0x7fda8c0b4260 2024-04-14T08:32:47.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.639+0000 7fda81ffb640 1 -- 172.21.15.172:0/4198152200 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7fda8801f020 con 0x7fda8c0b4260 2024-04-14T08:32:47.640 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:32:47.640 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3053874257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3053874257},{"type":"v1","addr":"172.21.15.172:6835","nonce":3053874257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-14T08:32:47.640 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 2 2024-04-14T08:32:47.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.641+0000 7fda9371e640 1 -- 172.21.15.172:0/4198152200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda7006c8f0 msgr2=0x7fda7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:47.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.641+0000 7fda9371e640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda7006c8f0 0x7fda7006edb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fda7c036000 tx=0x7fda7c008030 comp rx=0 tx=0).stop 2024-04-14T08:32:47.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.641+0000 7fda9371e640 1 -- 172.21.15.172:0/4198152200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda8c0b4260 msgr2=0x7fda8c1531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:47.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.641+0000 7fda9371e640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda8c0b4260 0x7fda8c1531f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda8800b410 tx=0x7fda8800b8d0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.641+0000 7fda9371e640 1 -- 172.21.15.172:0/4198152200 shutdown_connections 2024-04-14T08:32:47.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.641+0000 7fda9371e640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda7006c8f0 0x7fda7006edb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.641+0000 7fda9371e640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda8c15f3a0 0x7fda8c154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.641+0000 7fda9371e640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda8c15e390 0x7fda8c155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.641+0000 7fda9371e640 1 --2- 172.21.15.172:0/4198152200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda8c0b4260 0x7fda8c1531f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:47.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.641+0000 7fda9371e640 1 -- 172.21.15.172:0/4198152200 >> 172.21.15.172:0/4198152200 conn(0x7fda8c06f820 msgr2=0x7fda8c159710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:47.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.641+0000 7fda9371e640 1 -- 172.21.15.172:0/4198152200 shutdown_connections 2024-04-14T08:32:47.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:47.641+0000 7fda9371e640 1 -- 172.21.15.172:0/4198152200 wait complete. 2024-04-14T08:32:49.653 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:32:49.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.764+0000 7ffaf25e0640 1 Processor -- start 2024-04-14T08:32:49.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.764+0000 7ffaf25e0640 1 -- start start 2024-04-14T08:32:49.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.765+0000 7ffaf25e0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ffaec0d1a70 0x7ffaec0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:49.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.765+0000 7ffaf25e0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaec0d1ed0 con 0x7ffaec15e390 2024-04-14T08:32:49.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.765+0000 7ffaf25e0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaec15e770 con 0x7ffaec0b4260 2024-04-14T08:32:49.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.765+0000 7ffaf25e0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ffaec0d0080 con 0x7ffaec0d1a70 2024-04-14T08:32:49.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.765+0000 7ffaebfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ffaec15e390 0x7ffaec15f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34812/0 (socket says 172.21.15.172:34812) 2024-04-14T08:32:49.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.765+0000 7ffaebfff640 1 -- 172.21.15.172:0/772578855 learned_addr learned my addr 172.21.15.172:0/772578855 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.765+0000 7ffaeb7fe640 1 --2- 172.21.15.172:0/772578855 >> v2:172.21.15.177:3300/0 conn(0x7ffaec0d1a70 0x7ffaec0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.766+0000 7ffaeaffd640 1 -- 172.21.15.172:0/772578855 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3891834387 0 0) 0x7ffadc001110 con 0x7ffaec15e390 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.766+0000 7ffaeaffd640 1 -- 172.21.15.172:0/772578855 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffad4003520 con 0x7ffaec15e390 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.766+0000 7ffaeaffd640 1 -- 172.21.15.172:0/772578855 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1124810307 0 0) 0x7ffaec15e770 con 0x7ffaec0b4260 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.766+0000 7ffaeaffd640 1 -- 172.21.15.172:0/772578855 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffadc001110 con 0x7ffaec0b4260 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.766+0000 7ffaeb7fe640 1 -- 172.21.15.172:0/772578855 >> v1:172.21.15.177:6789/0 conn(0x7ffaec0b4260 legacy=0x7ffaec06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.766+0000 7ffaeb7fe640 1 -- 172.21.15.172:0/772578855 >> v1:172.21.15.172:6789/0 conn(0x7ffaec15e390 legacy=0x7ffaec15f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.766+0000 7ffaeaffd640 1 -- 172.21.15.172:0/772578855 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2519858387 0 0) 0x7ffad4003520 con 0x7ffaec15e390 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.766+0000 7ffaeb7fe640 1 -- 172.21.15.172:0/772578855 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec06dba0 con 0x7ffaec0d1a70 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.767+0000 7ffaeb7fe640 1 --2- 172.21.15.172:0/772578855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaec0d1a70 0x7ffaec0b3d20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffae00097a0 tx=0x7ffae002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85a861a9cca84c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.767+0000 7ffaeaffd640 1 -- 172.21.15.172:0/772578855 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0037070 con 0x7ffaec0d1a70 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.767+0000 7ffaeaffd640 1 -- 172.21.15.172:0/772578855 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae0030510 con 0x7ffaec0d1a70 2024-04-14T08:32:49.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.767+0000 7ffaeaffd640 1 -- 172.21.15.172:0/772578855 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc001840 con 0x7ffaec0d1a70 2024-04-14T08:32:49.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.767+0000 7ffaf25e0640 1 -- 172.21.15.172:0/772578855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaec0d1a70 msgr2=0x7ffaec0b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.767+0000 7ffaf25e0640 1 --2- 172.21.15.172:0/772578855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaec0d1a70 0x7ffaec0b3d20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffae00097a0 tx=0x7ffae002b790 comp rx=0 tx=0).stop 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.767+0000 7ffaf25e0640 1 -- 172.21.15.172:0/772578855 shutdown_connections 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.767+0000 7ffaf25e0640 1 --2- 172.21.15.172:0/772578855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaec0d1a70 0x7ffaec0b3d20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.767+0000 7ffaf25e0640 1 -- 172.21.15.172:0/772578855 >> 172.21.15.172:0/772578855 conn(0x7ffaec06f820 msgr2=0x7ffaec06fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.767+0000 7ffaf25e0640 1 -- 172.21.15.172:0/772578855 shutdown_connections 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.767+0000 7ffaf25e0640 1 -- 172.21.15.172:0/772578855 wait complete. 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.769+0000 7ffaf25e0640 1 Processor -- start 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.769+0000 7ffaf25e0640 1 -- start start 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.769+0000 7ffaf25e0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffaec0b4260 0x7ffaec153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.769+0000 7ffaf25e0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaec0d1a70 0x7ffaec15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.769+0000 7ffaf25e0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffaec15e390 0x7ffaec15a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.769+0000 7ffaf25e0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec0d0510 con 0x7ffaec15e390 2024-04-14T08:32:49.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.769+0000 7ffaf25e0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec0b4690 con 0x7ffaec0d1a70 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.769+0000 7ffaf25e0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ffaec06cd40 con 0x7ffaec0b4260 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaf0b7d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffaec15e390 0x7ffaec15a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaf0b7d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffaec15e390 0x7ffaec15a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58042/0 (socket says 172.21.15.172:58042) 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaf0b7d640 1 -- 172.21.15.172:0/781915122 learned_addr learned my addr 172.21.15.172:0/781915122 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaebfff640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffaec0b4260 0x7ffaec153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaeb7fe640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaec0d1a70 0x7ffaec15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaf0b7d640 1 -- 172.21.15.172:0/781915122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffaec0b4260 msgr2=0x7ffaec153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaf0b7d640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffaec0b4260 0x7ffaec153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaf0b7d640 1 -- 172.21.15.172:0/781915122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaec0d1a70 msgr2=0x7ffaec15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaf0b7d640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaec0d1a70 0x7ffaec15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaf0b7d640 1 -- 172.21.15.172:0/781915122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec15b1a0 con 0x7ffaec15e390 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaf0b7d640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffaec15e390 0x7ffaec15a720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffae400c7c0 tx=0x7ffae400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:49.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffae8ff9640 1 -- 172.21.15.172:0/781915122 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4003b70 con 0x7ffaec15e390 2024-04-14T08:32:49.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffae8ff9640 1 -- 172.21.15.172:0/781915122 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae4003d10 con 0x7ffaec15e390 2024-04-14T08:32:49.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaf25e0640 1 -- 172.21.15.172:0/781915122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaec15b480 con 0x7ffaec15e390 2024-04-14T08:32:49.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffae8ff9640 1 -- 172.21.15.172:0/781915122 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae400aa50 con 0x7ffaec15e390 2024-04-14T08:32:49.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.770+0000 7ffaf25e0640 1 -- 172.21.15.172:0/781915122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaec15ba40 con 0x7ffaec15e390 2024-04-14T08:32:49.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.771+0000 7ffaf25e0640 1 -- 172.21.15.172:0/781915122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaec068520 con 0x7ffaec15e390 2024-04-14T08:32:49.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.772+0000 7ffae8ff9640 1 -- 172.21.15.172:0/781915122 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffae400abf0 con 0x7ffaec15e390 2024-04-14T08:32:49.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.772+0000 7ffae8ff9640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffad806c920 0x7ffad806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:49.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.772+0000 7ffaebfff640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffad806c920 0x7ffad806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:49.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.772+0000 7ffae8ff9640 1 -- 172.21.15.172:0/781915122 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ffae408b630 con 0x7ffaec15e390 2024-04-14T08:32:49.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.773+0000 7ffaebfff640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffad806c920 0x7ffad806ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0013b0 tx=0x7ffadc001dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:49.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.777+0000 7ffae8ff9640 1 -- 172.21.15.172:0/781915122 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ffae4059760 con 0x7ffaec15e390 2024-04-14T08:32:49.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.989+0000 7ffaf25e0640 1 -- 172.21.15.172:0/781915122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffaec0b3a20 con 0x7ffaec15e390 2024-04-14T08:32:49.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.989+0000 7ffae8ff9640 1 -- 172.21.15.172:0/781915122 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3873 (secure 0 0 0) 0x7ffae405da60 con 0x7ffaec15e390 2024-04-14T08:32:49.990 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:32:49.990 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3053874257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3053874257},{"type":"v1","addr":"172.21.15.172:6835","nonce":3053874257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2790588849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2790588849},{"type":"v1","addr":"172.21.15.172:6837","nonce":2790588849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3130776573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3130776573},{"type":"v1","addr":"172.21.15.177:6834","nonce":3130776573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/517963877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":517963877},{"type":"v1","addr":"172.21.15.177:6835","nonce":517963877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-14T08:32:49.990 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 3 2024-04-14T08:32:49.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.991+0000 7ffaf25e0640 1 -- 172.21.15.172:0/781915122 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffad806c920 msgr2=0x7ffad806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:49.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.991+0000 7ffaf25e0640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffad806c920 0x7ffad806ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0013b0 tx=0x7ffadc001dd0 comp rx=0 tx=0).stop 2024-04-14T08:32:49.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.991+0000 7ffaf25e0640 1 -- 172.21.15.172:0/781915122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffaec15e390 msgr2=0x7ffaec15a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:49.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.991+0000 7ffaf25e0640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffaec15e390 0x7ffaec15a720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffae400c7c0 tx=0x7ffae400cc80 comp rx=0 tx=0).stop 2024-04-14T08:32:49.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.991+0000 7ffaf25e0640 1 -- 172.21.15.172:0/781915122 shutdown_connections 2024-04-14T08:32:49.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.991+0000 7ffaf25e0640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffad806c920 0x7ffad806ede0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:49.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.991+0000 7ffaf25e0640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffaec15e390 0x7ffaec15a720 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:49.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.991+0000 7ffaf25e0640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaec0d1a70 0x7ffaec15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:49.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.991+0000 7ffaf25e0640 1 --2- 172.21.15.172:0/781915122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffaec0b4260 0x7ffaec153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:49.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.991+0000 7ffaf25e0640 1 -- 172.21.15.172:0/781915122 >> 172.21.15.172:0/781915122 conn(0x7ffaec06f820 msgr2=0x7ffaec0c7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:49.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.991+0000 7ffaf25e0640 1 -- 172.21.15.172:0/781915122 shutdown_connections 2024-04-14T08:32:49.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:49.992+0000 7ffaf25e0640 1 -- 172.21.15.172:0/781915122 wait complete. 2024-04-14T08:32:50.004 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:32:50.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.105+0000 7f2e99915640 1 Processor -- start 2024-04-14T08:32:50.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.105+0000 7f2e99915640 1 -- start start 2024-04-14T08:32:50.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e99915640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2e940d2830 0x7f2e940d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e99915640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2e9415e890 0x7f2e940d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e99915640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e940d3150 con 0x7f2e940cedd0 2024-04-14T08:32:50.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e99915640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2e940cff30 con 0x7f2e9415e890 2024-04-14T08:32:50.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e99915640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2e940d2350 con 0x7f2e940d2830 2024-04-14T08:32:50.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e927fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2e9415e890 0x7f2e940d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e927fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2e9415e890 0x7f2e940d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58050/0 (socket says 172.21.15.172:58050) 2024-04-14T08:32:50.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e927fc640 1 -- 172.21.15.172:0/2858976778 learned_addr learned my addr 172.21.15.172:0/2858976778 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:50.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e927fc640 1 -- 172.21.15.172:0/2858976778 >> v2:172.21.15.177:3300/0 conn(0x7f2e940d2830 msgr2=0x7f2e940d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:50.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e92ffd640 1 --2- 172.21.15.172:0/2858976778 >> v2:172.21.15.177:3300/0 conn(0x7f2e940d2830 0x7f2e940d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e927fc640 1 --2- 172.21.15.172:0/2858976778 >> v2:172.21.15.177:3300/0 conn(0x7f2e940d2830 0x7f2e940d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e927fc640 1 -- 172.21.15.172:0/2858976778 >> v1:172.21.15.177:6789/0 conn(0x7f2e940cedd0 legacy=0x7f2e941621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:50.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e927fc640 1 -- 172.21.15.172:0/2858976778 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e940cf320 con 0x7f2e9415e890 2024-04-14T08:32:50.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.106+0000 7f2e92ffd640 1 --2- 172.21.15.172:0/2858976778 >> v2:172.21.15.177:3300/0 conn(0x7f2e940d2830 0x7f2e940d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:32:50.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.107+0000 7f2e927fc640 1 --2- 172.21.15.172:0/2858976778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e9415e890 0x7f2e940d1300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e9408c830 tx=0x7f2e8802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efb289e9a638cce server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:50.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.107+0000 7f2e91ffb640 1 -- 172.21.15.172:0/2858976778 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88034070 con 0x7f2e9415e890 2024-04-14T08:32:50.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.107+0000 7f2e91ffb640 1 -- 172.21.15.172:0/2858976778 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e88030860 con 0x7f2e9415e890 2024-04-14T08:32:50.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.107+0000 7f2e91ffb640 1 -- 172.21.15.172:0/2858976778 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88030c30 con 0x7f2e9415e890 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.108+0000 7f2e99915640 1 -- 172.21.15.172:0/2858976778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e9415e890 msgr2=0x7f2e940d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.108+0000 7f2e99915640 1 --2- 172.21.15.172:0/2858976778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e9415e890 0x7f2e940d1300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e9408c830 tx=0x7f2e8802b9a0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.108+0000 7f2e99915640 1 -- 172.21.15.172:0/2858976778 shutdown_connections 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.108+0000 7f2e99915640 1 --2- 172.21.15.172:0/2858976778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e9415e890 0x7f2e940d1300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.108+0000 7f2e99915640 1 --2- 172.21.15.172:0/2858976778 >> v2:172.21.15.177:3300/0 conn(0x7f2e940d2830 0x7f2e940d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.108+0000 7f2e99915640 1 -- 172.21.15.172:0/2858976778 >> 172.21.15.172:0/2858976778 conn(0x7f2e9409d590 msgr2=0x7f2e940d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.108+0000 7f2e99915640 1 -- 172.21.15.172:0/2858976778 shutdown_connections 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.108+0000 7f2e99915640 1 -- 172.21.15.172:0/2858976778 wait complete. 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.109+0000 7f2e99915640 1 Processor -- start 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.109+0000 7f2e99915640 1 -- start start 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e99915640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e940cedd0 0x7f2e9414f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e99915640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e940d2830 0x7f2e940b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e99915640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e9415e890 0x7f2e940c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e99915640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940cfbc0 con 0x7f2e940cedd0 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e99915640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940cf8e0 con 0x7f2e9415e890 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e99915640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2e940d1840 con 0x7f2e940d2830 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e92ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e940cedd0 0x7f2e9414f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e92ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e940cedd0 0x7f2e9414f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58066/0 (socket says 172.21.15.172:58066) 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e92ffd640 1 -- 172.21.15.172:0/857109760 learned_addr learned my addr 172.21.15.172:0/857109760 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e927fc640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e940d2830 0x7f2e940b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e937fe640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e9415e890 0x7f2e940c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e92ffd640 1 -- 172.21.15.172:0/857109760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e940d2830 msgr2=0x7f2e940b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e92ffd640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e940d2830 0x7f2e940b6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e92ffd640 1 -- 172.21.15.172:0/857109760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e9415e890 msgr2=0x7f2e940c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e92ffd640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e9415e890 0x7f2e940c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e92ffd640 1 -- 172.21.15.172:0/857109760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9414fbe0 con 0x7f2e940cedd0 2024-04-14T08:32:50.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e92ffd640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e940cedd0 0x7f2e9414f6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400c7f0 tx=0x7f2e8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:50.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e7bfff640 1 -- 172.21.15.172:0/857109760 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84015070 con 0x7f2e940cedd0 2024-04-14T08:32:50.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e7bfff640 1 -- 172.21.15.172:0/857109760 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e84007820 con 0x7f2e940cedd0 2024-04-14T08:32:50.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e99915640 1 -- 172.21.15.172:0/857109760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9415fe00 con 0x7f2e940cedd0 2024-04-14T08:32:50.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e7bfff640 1 -- 172.21.15.172:0/857109760 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84007bd0 con 0x7f2e940cedd0 2024-04-14T08:32:50.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.110+0000 7f2e99915640 1 -- 172.21.15.172:0/857109760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e941603c0 con 0x7f2e940cedd0 2024-04-14T08:32:50.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.111+0000 7f2e99915640 1 -- 172.21.15.172:0/857109760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e94068520 con 0x7f2e940cedd0 2024-04-14T08:32:50.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.112+0000 7f2e7bfff640 1 -- 172.21.15.172:0/857109760 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e84018070 con 0x7f2e940cedd0 2024-04-14T08:32:50.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.112+0000 7f2e7bfff640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2e8006c8f0 0x7f2e8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.112+0000 7f2e927fc640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2e8006c8f0 0x7f2e8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.112+0000 7f2e7bfff640 1 -- 172.21.15.172:0/857109760 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f2e8408b910 con 0x7f2e940cedd0 2024-04-14T08:32:50.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.113+0000 7f2e927fc640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2e8006c8f0 0x7f2e8006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2e88036000 tx=0x7f2e8803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:50.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.117+0000 7f2e7bfff640 1 -- 172.21.15.172:0/857109760 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2e8405a090 con 0x7f2e940cedd0 2024-04-14T08:32:50.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.295+0000 7f2e99915640 1 -- 172.21.15.172:0/857109760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e940d2610 con 0x7f2e940cedd0 2024-04-14T08:32:50.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.296+0000 7f2e7bfff640 1 -- 172.21.15.172:0/857109760 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12944 (secure 0 0 0) 0x7f2e84059990 con 0x7f2e940cedd0 2024-04-14T08:32:50.296 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:32:50.297 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":17,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:44.140937+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:32:50.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.298+0000 7f2e99915640 1 -- 172.21.15.172:0/857109760 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2e8006c8f0 msgr2=0x7f2e8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.298+0000 7f2e99915640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2e8006c8f0 0x7f2e8006edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2e88036000 tx=0x7f2e8803b040 comp rx=0 tx=0).stop 2024-04-14T08:32:50.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.298+0000 7f2e99915640 1 -- 172.21.15.172:0/857109760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e940cedd0 msgr2=0x7f2e9414f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.298+0000 7f2e99915640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e940cedd0 0x7f2e9414f6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400c7f0 tx=0x7f2e8400ccb0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.298+0000 7f2e99915640 1 -- 172.21.15.172:0/857109760 shutdown_connections 2024-04-14T08:32:50.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.298+0000 7f2e99915640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2e8006c8f0 0x7f2e8006edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.298+0000 7f2e99915640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e9415e890 0x7f2e940c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.298+0000 7f2e99915640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e940d2830 0x7f2e940b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.298+0000 7f2e99915640 1 --2- 172.21.15.172:0/857109760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e940cedd0 0x7f2e9414f6a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.298+0000 7f2e99915640 1 -- 172.21.15.172:0/857109760 >> 172.21.15.172:0/857109760 conn(0x7f2e9409d590 msgr2=0x7f2e940d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:50.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.298+0000 7f2e99915640 1 -- 172.21.15.172:0/857109760 shutdown_connections 2024-04-14T08:32:50.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.298+0000 7f2e99915640 1 -- 172.21.15.172:0/857109760 wait complete. 2024-04-14T08:32:50.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:32:50.310 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:32:50.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.413+0000 7fbd39098640 1 Processor -- start 2024-04-14T08:32:50.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.413+0000 7fbd39098640 1 -- start start 2024-04-14T08:32:50.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.414+0000 7fbd39098640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbd340ce550 0x7fbd3406d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.414+0000 7fbd39098640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd340cc780 con 0x7fbd3406d580 2024-04-14T08:32:50.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.414+0000 7fbd39098640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd340cca60 con 0x7fbd3415e3c0 2024-04-14T08:32:50.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.414+0000 7fbd39098640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fbd3406ce40 con 0x7fbd340ce550 2024-04-14T08:32:50.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.414+0000 7fbd3259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbd340ce550 0x7fbd3406d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.414+0000 7fbd3359e640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fbd3406d580 0x7fbd340d0240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34826/0 (socket says 172.21.15.172:34826) 2024-04-14T08:32:50.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.414+0000 7fbd3259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbd340ce550 0x7fbd3406d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58068/0 (socket says 172.21.15.172:58068) 2024-04-14T08:32:50.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.414+0000 7fbd3359e640 1 -- 172.21.15.172:0/3946225195 learned_addr learned my addr 172.21.15.172:0/3946225195 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:32:50.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.414+0000 7fbd3259c640 1 -- 172.21.15.172:0/3946225195 >> v1:172.21.15.177:6789/0 conn(0x7fbd3415e3c0 legacy=0x7fbd3406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.414+0000 7fbd3259c640 1 -- 172.21.15.172:0/3946225195 >> v1:172.21.15.172:6789/0 conn(0x7fbd3406d580 legacy=0x7fbd340d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.414+0000 7fbd3259c640 1 -- 172.21.15.172:0/3946225195 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd340ccd40 con 0x7fbd340ce550 2024-04-14T08:32:50.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.414+0000 7fbd31d9b640 1 -- 172.21.15.172:0/3946225195 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1861663546 0 0) 0x7fbd340cc780 con 0x7fbd3406d580 2024-04-14T08:32:50.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.415+0000 7fbd3259c640 1 --2- 172.21.15.172:0/3946225195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd340ce550 0x7fbd3406d040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbd280097a0 tx=0x7fbd2802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca5d803657e10cd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:50.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.415+0000 7fbd31d9b640 1 -- 172.21.15.172:0/3946225195 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28034070 con 0x7fbd340ce550 2024-04-14T08:32:50.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.415+0000 7fbd31d9b640 1 -- 172.21.15.172:0/3946225195 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd28030610 con 0x7fbd340ce550 2024-04-14T08:32:50.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.415+0000 7fbd31d9b640 1 -- 172.21.15.172:0/3946225195 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd280309e0 con 0x7fbd340ce550 2024-04-14T08:32:50.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.416+0000 7fbd39098640 1 -- 172.21.15.172:0/3946225195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd340ce550 msgr2=0x7fbd3406d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.416+0000 7fbd39098640 1 --2- 172.21.15.172:0/3946225195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd340ce550 0x7fbd3406d040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbd280097a0 tx=0x7fbd2802b960 comp rx=0 tx=0).stop 2024-04-14T08:32:50.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.416+0000 7fbd39098640 1 -- 172.21.15.172:0/3946225195 shutdown_connections 2024-04-14T08:32:50.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.416+0000 7fbd39098640 1 --2- 172.21.15.172:0/3946225195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd340ce550 0x7fbd3406d040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.416+0000 7fbd39098640 1 -- 172.21.15.172:0/3946225195 >> 172.21.15.172:0/3946225195 conn(0x7fbd340b3af0 msgr2=0x7fbd340b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:50.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.416+0000 7fbd39098640 1 -- 172.21.15.172:0/3946225195 shutdown_connections 2024-04-14T08:32:50.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.416+0000 7fbd39098640 1 -- 172.21.15.172:0/3946225195 wait complete. 2024-04-14T08:32:50.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.417+0000 7fbd39098640 1 Processor -- start 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.417+0000 7fbd39098640 1 -- start start 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd39098640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd3406d580 0x7fbd34158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd39098640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd340ce550 0x7fbd3415f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd39098640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3415e3c0 0x7fbd34070180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd39098640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd340d0950 con 0x7fbd3406d580 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd39098640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3415e7a0 con 0x7fbd340ce550 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd39098640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbd340d0ad0 con 0x7fbd3415e3c0 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd32d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd3406d580 0x7fbd34158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd32d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd3406d580 0x7fbd34158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58078/0 (socket says 172.21.15.172:58078) 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd32d9d640 1 -- 172.21.15.172:0/595325207 learned_addr learned my addr 172.21.15.172:0/595325207 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd3259c640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd340ce550 0x7fbd3415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd3359e640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3415e3c0 0x7fbd34070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd32d9d640 1 -- 172.21.15.172:0/595325207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3415e3c0 msgr2=0x7fbd34070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd32d9d640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3415e3c0 0x7fbd34070180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd32d9d640 1 -- 172.21.15.172:0/595325207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd340ce550 msgr2=0x7fbd3415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd32d9d640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd340ce550 0x7fbd3415f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd32d9d640 1 -- 172.21.15.172:0/595325207 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd34161810 con 0x7fbd3406d580 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd3259c640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd340ce550 0x7fbd3415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd3359e640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3415e3c0 0x7fbd34070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.418+0000 7fbd32d9d640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd3406d580 0x7fbd34158500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c400 tx=0x7fbd2400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.419+0000 7fbd1b7fe640 1 -- 172.21.15.172:0/595325207 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24017070 con 0x7fbd3406d580 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.419+0000 7fbd1b7fe640 1 -- 172.21.15.172:0/595325207 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd240052d0 con 0x7fbd3406d580 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.419+0000 7fbd39098640 1 -- 172.21.15.172:0/595325207 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd34160770 con 0x7fbd3406d580 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.419+0000 7fbd1b7fe640 1 -- 172.21.15.172:0/595325207 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24003830 con 0x7fbd3406d580 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.419+0000 7fbd39098640 1 -- 172.21.15.172:0/595325207 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd34161010 con 0x7fbd3406d580 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.420+0000 7fbd39098640 1 -- 172.21.15.172:0/595325207 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd34068590 con 0x7fbd3406d580 2024-04-14T08:32:50.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.420+0000 7fbd1b7fe640 1 -- 172.21.15.172:0/595325207 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd2401a070 con 0x7fbd3406d580 2024-04-14T08:32:50.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.420+0000 7fbd1b7fe640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbd2006c8f0 0x7fbd2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.420+0000 7fbd1b7fe640 1 -- 172.21.15.172:0/595325207 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fbd2408aad0 con 0x7fbd3406d580 2024-04-14T08:32:50.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.420+0000 7fbd3259c640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbd2006c8f0 0x7fbd2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.421+0000 7fbd3259c640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbd2006c8f0 0x7fbd2006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbd28036000 tx=0x7fbd2803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:50.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.425+0000 7fbd1b7fe640 1 -- 172.21.15.172:0/595325207 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbd240592e0 con 0x7fbd3406d580 2024-04-14T08:32:50.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.598+0000 7fbd39098640 1 -- 172.21.15.172:0/595325207 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbd34160b00 con 0x7fbd3406d580 2024-04-14T08:32:50.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.599+0000 7fbd1b7fe640 1 -- 172.21.15.172:0/595325207 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fbd24058be0 con 0x7fbd3406d580 2024-04-14T08:32:50.599 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:32:50.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.600+0000 7fbd39098640 1 -- 172.21.15.172:0/595325207 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbd2006c8f0 msgr2=0x7fbd2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.600+0000 7fbd39098640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbd2006c8f0 0x7fbd2006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbd28036000 tx=0x7fbd2803b040 comp rx=0 tx=0).stop 2024-04-14T08:32:50.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.600+0000 7fbd39098640 1 -- 172.21.15.172:0/595325207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd3406d580 msgr2=0x7fbd34158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.600+0000 7fbd39098640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd3406d580 0x7fbd34158500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c400 tx=0x7fbd2400c8c0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.601+0000 7fbd39098640 1 -- 172.21.15.172:0/595325207 shutdown_connections 2024-04-14T08:32:50.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.601+0000 7fbd39098640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbd2006c8f0 0x7fbd2006edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.601+0000 7fbd39098640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd3415e3c0 0x7fbd34070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.601+0000 7fbd39098640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd340ce550 0x7fbd3415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.601+0000 7fbd39098640 1 --2- 172.21.15.172:0/595325207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd3406d580 0x7fbd34158500 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.601+0000 7fbd39098640 1 -- 172.21.15.172:0/595325207 >> 172.21.15.172:0/595325207 conn(0x7fbd340b3af0 msgr2=0x7fbd3406c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:50.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.601+0000 7fbd39098640 1 -- 172.21.15.172:0/595325207 shutdown_connections 2024-04-14T08:32:50.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.601+0000 7fbd39098640 1 -- 172.21.15.172:0/595325207 wait complete. 2024-04-14T08:32:50.612 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-14T08:32:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.712+0000 7f9fa67dc640 1 Processor -- start 2024-04-14T08:32:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.712+0000 7f9fa67dc640 1 -- start start 2024-04-14T08:32:50.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.713+0000 7f9fa67dc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9fa015e390 0x7f9fa00d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.713+0000 7f9fa67dc640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa015e770 con 0x7f9fa00b4430 2024-04-14T08:32:50.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.713+0000 7f9fa67dc640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa006dba0 con 0x7f9fa015f3a0 2024-04-14T08:32:50.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.713+0000 7f9fa67dc640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa00d0080 con 0x7f9fa015e390 2024-04-14T08:32:50.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.713+0000 7f9f9ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9fa015e390 0x7f9fa00d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.713+0000 7f9fa4d79640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9fa00b4430 0x7f9fa006c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34832/0 (socket says 172.21.15.172:34832) 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.713+0000 7f9f9ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9fa015e390 0x7f9fa00d1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58090/0 (socket says 172.21.15.172:58090) 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.713+0000 7f9fa4d79640 1 -- 172.21.15.172:0/2915783888 learned_addr learned my addr 172.21.15.172:0/2915783888 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.713+0000 7f9f9ffff640 1 -- 172.21.15.172:0/2915783888 >> v1:172.21.15.177:6789/0 conn(0x7f9fa015f3a0 legacy=0x7f9fa00b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.713+0000 7f9f9ffff640 1 -- 172.21.15.172:0/2915783888 >> v1:172.21.15.172:6789/0 conn(0x7f9fa00b4430 legacy=0x7f9fa006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.713+0000 7f9f9ffff640 1 -- 172.21.15.172:0/2915783888 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa006de80 con 0x7f9fa015e390 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.713+0000 7f9f9effd640 1 -- 172.21.15.172:0/2915783888 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 694534244 0 0) 0x7f9fa015e770 con 0x7f9fa00b4430 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.714+0000 7f9f9ffff640 1 --2- 172.21.15.172:0/2915783888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa015e390 0x7f9fa00d1a70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f900097a0 tx=0x7f9f9002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f86a05000f4afb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.714+0000 7f9f9effd640 1 -- 172.21.15.172:0/2915783888 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f900315a0 con 0x7f9fa015e390 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.714+0000 7f9f9effd640 1 -- 172.21.15.172:0/2915783888 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f90037070 con 0x7f9fa015e390 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.714+0000 7f9f9effd640 1 -- 172.21.15.172:0/2915783888 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90031960 con 0x7f9fa015e390 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.715+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2915783888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa015e390 msgr2=0x7f9fa00d1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.715+0000 7f9fa67dc640 1 --2- 172.21.15.172:0/2915783888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa015e390 0x7f9fa00d1a70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f900097a0 tx=0x7f9f9002b840 comp rx=0 tx=0).stop 2024-04-14T08:32:50.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.715+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2915783888 shutdown_connections 2024-04-14T08:32:50.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.715+0000 7f9fa67dc640 1 --2- 172.21.15.172:0/2915783888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa015e390 0x7f9fa00d1a70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.715+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2915783888 >> 172.21.15.172:0/2915783888 conn(0x7f9fa006f820 msgr2=0x7f9fa006fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:50.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.715+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2915783888 shutdown_connections 2024-04-14T08:32:50.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.715+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2915783888 wait complete. 2024-04-14T08:32:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.716+0000 7f9fa67dc640 1 Processor -- start 2024-04-14T08:32:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.716+0000 7f9fa67dc640 1 -- start start 2024-04-14T08:32:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa67dc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fa00b4430 0x7f9fa00ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa67dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fa015e390 0x7f9fa00b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa67dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa015f3a0 0x7f9fa0150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa67dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa006cd40 con 0x7f9fa015f3a0 2024-04-14T08:32:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa67dc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa00d0510 con 0x7f9fa00b4430 2024-04-14T08:32:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa67dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa006cec0 con 0x7f9fa015e390 2024-04-14T08:32:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa4d79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa015f3a0 0x7f9fa0150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa4d79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa015f3a0 0x7f9fa0150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58098/0 (socket says 172.21.15.172:58098) 2024-04-14T08:32:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa4d79640 1 -- 172.21.15.172:0/2796822828 learned_addr learned my addr 172.21.15.172:0/2796822828 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9f9ffff640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fa00b4430 0x7f9fa00ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9f9f7fe640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fa015e390 0x7f9fa00b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa4d79640 1 -- 172.21.15.172:0/2796822828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fa015e390 msgr2=0x7f9fa00b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa4d79640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fa015e390 0x7f9fa00b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa4d79640 1 -- 172.21.15.172:0/2796822828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fa00b4430 msgr2=0x7f9fa00ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa4d79640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fa00b4430 0x7f9fa00ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa4d79640 1 -- 172.21.15.172:0/2796822828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa0150bd0 con 0x7f9fa015f3a0 2024-04-14T08:32:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9f9ffff640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fa00b4430 0x7f9fa00ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:32:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9f9f7fe640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fa015e390 0x7f9fa00b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:32:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa4d79640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa015f3a0 0x7f9fa0150690 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800cac0 tx=0x7f9f9800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9f9cff9640 1 -- 172.21.15.172:0/2796822828 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9800ac30 con 0x7f9fa015f3a0 2024-04-14T08:32:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9f9cff9640 1 -- 172.21.15.172:0/2796822828 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9800add0 con 0x7f9fa015f3a0 2024-04-14T08:32:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2796822828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa00adf80 con 0x7f9fa015f3a0 2024-04-14T08:32:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9f9cff9640 1 -- 172.21.15.172:0/2796822828 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98012640 con 0x7f9fa015f3a0 2024-04-14T08:32:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.717+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2796822828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa00ae540 con 0x7f9fa015f3a0 2024-04-14T08:32:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.718+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2796822828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa0068520 con 0x7f9fa015f3a0 2024-04-14T08:32:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.719+0000 7f9f9cff9640 1 -- 172.21.15.172:0/2796822828 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f980127e0 con 0x7f9fa015f3a0 2024-04-14T08:32:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.719+0000 7f9f9cff9640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f8c06c920 0x7f9f8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.719+0000 7f9f9cff9640 1 -- 172.21.15.172:0/2796822828 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f9f9808beb0 con 0x7f9fa015f3a0 2024-04-14T08:32:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.719+0000 7f9f9ffff640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f8c06c920 0x7f9f8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.720+0000 7f9f9ffff640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f8c06c920 0x7f9f8c06ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9f90036000 tx=0x7f9f90008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:50.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.723+0000 7f9f9cff9640 1 -- 172.21.15.172:0/2796822828 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9f9805a5a0 con 0x7f9fa015f3a0 2024-04-14T08:32:50.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.901+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2796822828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9fa00b21c0 con 0x7f9fa015f3a0 2024-04-14T08:32:50.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.902+0000 7f9f9cff9640 1 -- 172.21.15.172:0/2796822828 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f9f98059ea0 con 0x7f9fa015f3a0 2024-04-14T08:32:50.902 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:32:50.902 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-04-14T08:32:50.902 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:32:50.903 INFO:teuthology.orchestra.run.smithi172.stdout: "poolnum": 1, 2024-04-14T08:32:50.903 INFO:teuthology.orchestra.run.smithi172.stdout: "poolname": ".mgr" 2024-04-14T08:32:50.903 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:32:50.903 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-04-14T08:32:50.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.904+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2796822828 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f8c06c920 msgr2=0x7f9f8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.904+0000 7f9fa67dc640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f8c06c920 0x7f9f8c06ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9f90036000 tx=0x7f9f90008030 comp rx=0 tx=0).stop 2024-04-14T08:32:50.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.904+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2796822828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa015f3a0 msgr2=0x7f9fa0150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:50.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.904+0000 7f9fa67dc640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa015f3a0 0x7f9fa0150690 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800cac0 tx=0x7f9f9800cf80 comp rx=0 tx=0).stop 2024-04-14T08:32:50.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.904+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2796822828 shutdown_connections 2024-04-14T08:32:50.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.904+0000 7f9fa67dc640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f8c06c920 0x7f9f8c06ede0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.904+0000 7f9fa67dc640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fa015f3a0 0x7f9fa0150690 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.904+0000 7f9fa67dc640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fa015e390 0x7f9fa00b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.904+0000 7f9fa67dc640 1 --2- 172.21.15.172:0/2796822828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fa00b4430 0x7f9fa00ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:50.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.904+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2796822828 >> 172.21.15.172:0/2796822828 conn(0x7f9fa006f820 msgr2=0x7f9fa00b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:50.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.904+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2796822828 shutdown_connections 2024-04-14T08:32:50.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:50.904+0000 7f9fa67dc640 1 -- 172.21.15.172:0/2796822828 wait complete. 2024-04-14T08:32:50.915 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-14T08:32:50.915 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-14T08:32:51.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.019+0000 7f7c1815e640 1 Processor -- start 2024-04-14T08:32:51.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.019+0000 7f7c1815e640 1 -- start start 2024-04-14T08:32:51.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c1815e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c1009b180 0x7f7c1015d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:51.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c1815e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7c100cfc30 0x7f7c100ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:51.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c1815e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c100d01a0 con 0x7f7c1015c5a0 2024-04-14T08:32:51.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c1815e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f7c1015d990 con 0x7f7c100cfc30 2024-04-14T08:32:51.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c1815e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7c100d0db0 con 0x7f7c1009b180 2024-04-14T08:32:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c166fb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7c100cfc30 0x7f7c100ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c166fb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7c100cfc30 0x7f7c100ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58106/0 (socket says 172.21.15.172:58106) 2024-04-14T08:32:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c15efa640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7c1015c5a0 0x7f7c1009aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34842/0 (socket says 172.21.15.172:34842) 2024-04-14T08:32:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c166fb640 1 -- 172.21.15.172:0/831042845 learned_addr learned my addr 172.21.15.172:0/831042845 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c156f9640 1 --2- 172.21.15.172:0/831042845 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c1009b180 0x7f7c1015d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c166fb640 1 -- 172.21.15.172:0/831042845 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c1009b180 msgr2=0x7f7c1015d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c166fb640 1 --2- 172.21.15.172:0/831042845 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c1009b180 0x7f7c1015d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c166fb640 1 -- 172.21.15.172:0/831042845 >> v1:172.21.15.172:6789/0 conn(0x7f7c1015c5a0 legacy=0x7f7c1009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c166fb640 1 -- 172.21.15.172:0/831042845 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c100d0480 con 0x7f7c100cfc30 2024-04-14T08:32:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c14ef8640 1 -- 172.21.15.172:0/831042845 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2278948502 0 0) 0x7f7c100d01a0 con 0x7f7c1015c5a0 2024-04-14T08:32:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.020+0000 7f7c156f9640 1 --2- 172.21.15.172:0/831042845 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c1009b180 0x7f7c1015d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:32:51.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.021+0000 7f7c166fb640 1 --2- 172.21.15.172:0/831042845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c100cfc30 0x7f7c100ccce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009e50 tx=0x7f7c0802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1ac8f5aa956d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:51.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.021+0000 7f7c14ef8640 1 -- 172.21.15.172:0/831042845 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0802cb00 con 0x7f7c100cfc30 2024-04-14T08:32:51.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.021+0000 7f7c14ef8640 1 -- 172.21.15.172:0/831042845 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0802cca0 con 0x7f7c100cfc30 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.021+0000 7f7c14ef8640 1 -- 172.21.15.172:0/831042845 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08034610 con 0x7f7c100cfc30 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.022+0000 7f7c1815e640 1 -- 172.21.15.172:0/831042845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c100cfc30 msgr2=0x7f7c100ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.022+0000 7f7c1815e640 1 --2- 172.21.15.172:0/831042845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c100cfc30 0x7f7c100ccce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009e50 tx=0x7f7c0802bf60 comp rx=0 tx=0).stop 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.022+0000 7f7c1815e640 1 -- 172.21.15.172:0/831042845 shutdown_connections 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.022+0000 7f7c1815e640 1 --2- 172.21.15.172:0/831042845 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c1009b180 0x7f7c1015d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.022+0000 7f7c1815e640 1 --2- 172.21.15.172:0/831042845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c100cfc30 0x7f7c100ccce0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.022+0000 7f7c1815e640 1 -- 172.21.15.172:0/831042845 >> 172.21.15.172:0/831042845 conn(0x7f7c100b3b50 msgr2=0x7f7c100b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.022+0000 7f7c1815e640 1 -- 172.21.15.172:0/831042845 shutdown_connections 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.022+0000 7f7c1815e640 1 -- 172.21.15.172:0/831042845 wait complete. 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.023+0000 7f7c1815e640 1 Processor -- start 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.023+0000 7f7c1815e640 1 -- start start 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c1815e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c1009b180 0x7f7c10159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c1815e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c100cfc30 0x7f7c100c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c1815e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1015c5a0 0x7f7c100c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c1815e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7c100cd220 con 0x7f7c1015c5a0 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c1815e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7c100d0a40 con 0x7f7c1009b180 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c1815e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7c100cd3a0 con 0x7f7c100cfc30 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c166fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1015c5a0 0x7f7c100c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c166fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1015c5a0 0x7f7c100c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58122/0 (socket says 172.21.15.172:58122) 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c166fb640 1 -- 172.21.15.172:0/306856979 learned_addr learned my addr 172.21.15.172:0/306856979 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c156f9640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c100cfc30 0x7f7c100c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c15efa640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c1009b180 0x7f7c10159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c166fb640 1 -- 172.21.15.172:0/306856979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c100cfc30 msgr2=0x7f7c100c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c166fb640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c100cfc30 0x7f7c100c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c166fb640 1 -- 172.21.15.172:0/306856979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c1009b180 msgr2=0x7f7c10159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c166fb640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c1009b180 0x7f7c10159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c166fb640 1 -- 172.21.15.172:0/306856979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c100c7b50 con 0x7f7c1015c5a0 2024-04-14T08:32:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c166fb640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1015c5a0 0x7f7c100c6610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009e50 tx=0x7f7c08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.024+0000 7f7c067fc640 1 -- 172.21.15.172:0/306856979 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08043020 con 0x7f7c1015c5a0 2024-04-14T08:32:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.025+0000 7f7c067fc640 1 -- 172.21.15.172:0/306856979 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0802ce50 con 0x7f7c1015c5a0 2024-04-14T08:32:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.025+0000 7f7c1815e640 1 -- 172.21.15.172:0/306856979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c100c5b30 con 0x7f7c1015c5a0 2024-04-14T08:32:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.025+0000 7f7c067fc640 1 -- 172.21.15.172:0/306856979 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08009510 con 0x7f7c1015c5a0 2024-04-14T08:32:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.025+0000 7f7c1815e640 1 -- 172.21.15.172:0/306856979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c10159dd0 con 0x7f7c1015c5a0 2024-04-14T08:32:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.026+0000 7f7c1815e640 1 -- 172.21.15.172:0/306856979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c100686a0 con 0x7f7c1015c5a0 2024-04-14T08:32:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.026+0000 7f7c067fc640 1 -- 172.21.15.172:0/306856979 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c08046020 con 0x7f7c1015c5a0 2024-04-14T08:32:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.026+0000 7f7c067fc640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7bf406c8f0 0x7f7bf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.026+0000 7f7c067fc640 1 -- 172.21.15.172:0/306856979 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f7c080addc0 con 0x7f7c1015c5a0 2024-04-14T08:32:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.026+0000 7f7c15efa640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7bf406c8f0 0x7f7bf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.027+0000 7f7c15efa640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7bf406c8f0 0x7f7bf406edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c007fd0 tx=0x7f7c0c007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:51.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.031+0000 7f7c067fc640 1 -- 172.21.15.172:0/306856979 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7c0804d030 con 0x7f7c1015c5a0 2024-04-14T08:32:51.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:51.211+0000 7f7c1815e640 1 -- 172.21.15.172:0/306856979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7c100ce8c0 con 0x7f7c1015c5a0 2024-04-14T08:32:52.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.170+0000 7f7c067fc640 1 -- 172.21.15.172:0/306856979 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f7c0807bef0 con 0x7f7c1015c5a0 2024-04-14T08:32:52.171 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' created 2024-04-14T08:32:52.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.172+0000 7f7c1815e640 1 -- 172.21.15.172:0/306856979 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7bf406c8f0 msgr2=0x7f7bf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:52.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.172+0000 7f7c1815e640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7bf406c8f0 0x7f7bf406edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c007fd0 tx=0x7f7c0c007de0 comp rx=0 tx=0).stop 2024-04-14T08:32:52.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.172+0000 7f7c1815e640 1 -- 172.21.15.172:0/306856979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1015c5a0 msgr2=0x7f7c100c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:52.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.172+0000 7f7c1815e640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1015c5a0 0x7f7c100c6610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009e50 tx=0x7f7c08004a40 comp rx=0 tx=0).stop 2024-04-14T08:32:52.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.172+0000 7f7c1815e640 1 -- 172.21.15.172:0/306856979 shutdown_connections 2024-04-14T08:32:52.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.172+0000 7f7c1815e640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7bf406c8f0 0x7f7bf406edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:52.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.172+0000 7f7c1815e640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1015c5a0 0x7f7c100c6610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:52.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.172+0000 7f7c1815e640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c100cfc30 0x7f7c100c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:52.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.172+0000 7f7c1815e640 1 --2- 172.21.15.172:0/306856979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c1009b180 0x7f7c10159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:52.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.172+0000 7f7c1815e640 1 -- 172.21.15.172:0/306856979 >> 172.21.15.172:0/306856979 conn(0x7f7c100b3b50 msgr2=0x7f7c100b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:52.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.173+0000 7f7c1815e640 1 -- 172.21.15.172:0/306856979 shutdown_connections 2024-04-14T08:32:52.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.173+0000 7f7c1815e640 1 -- 172.21.15.172:0/306856979 wait complete. 2024-04-14T08:32:52.190 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-14T08:32:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.301+0000 7fd47718c640 1 Processor -- start 2024-04-14T08:32:52.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.301+0000 7fd47718c640 1 -- start start 2024-04-14T08:32:52.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd47718c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700ce550 0x7fd47006d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:52.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd47718c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4700cc780 con 0x7fd47015e3c0 2024-04-14T08:32:52.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd47718c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4700cca60 con 0x7fd47006d580 2024-04-14T08:32:52.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd47718c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd47006ce40 con 0x7fd4700ce550 2024-04-14T08:32:52.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd474f28640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd47015e3c0 0x7fd47006c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34854/0 (socket says 172.21.15.172:34854) 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd474f28640 1 -- 172.21.15.172:0/2339908584 learned_addr learned my addr 172.21.15.172:0/2339908584 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2625103434 0 0) 0x7fd4700cc780 con 0x7fd47015e3c0 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd458003520 con 0x7fd47015e3c0 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd46ffff640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700ce550 0x7fd47006d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1666660536 0 0) 0x7fd458003520 con 0x7fd47015e3c0 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700ce550 msgr2=0x7fd47006d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd46f7fe640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700ce550 0x7fd47006d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 >> v1:172.21.15.177:6789/0 conn(0x7fd47006d580 legacy=0x7fd4700d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4700ccd40 con 0x7fd47015e3c0 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fd460001670 con 0x7fd47015e3c0 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.302+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 >> v1:172.21.15.172:6789/0 conn(0x7fd47015e3c0 legacy=0x7fd47006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd46f7fe640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4580039e0 0x7fd458003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd46f7fe640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd458004800 0x7fd458004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd46f7fe640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700ce550 0x7fd4580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd47006ce40 con 0x7fd458004800 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd45800ba70 con 0x7fd4580039e0 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd45800bc70 con 0x7fd4700ce550 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd46ffff640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd458004800 0x7fd458004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd474f28640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4580039e0 0x7fd458003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd475729640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700ce550 0x7fd4580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd46ffff640 1 -- 172.21.15.172:0/2339908584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700ce550 msgr2=0x7fd4580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd46ffff640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700ce550 0x7fd4580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd46ffff640 1 -- 172.21.15.172:0/2339908584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4580039e0 msgr2=0x7fd458003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd46ffff640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4580039e0 0x7fd458003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd46ffff640 1 -- 172.21.15.172:0/2339908584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd458003520 con 0x7fd458004800 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.303+0000 7fd475729640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700ce550 0x7fd4580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:32:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.304+0000 7fd46ffff640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd458004800 0x7fd458004c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd464009a90 tx=0x7fd458005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e895eab9f276c87a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:52.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.304+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464039070 con 0x7fd458004800 2024-04-14T08:32:52.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.304+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd46402c680 con 0x7fd458004800 2024-04-14T08:32:52.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.304+0000 7fd46f7fe640 1 -- 172.21.15.172:0/2339908584 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464009690 con 0x7fd458004800 2024-04-14T08:32:52.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.304+0000 7fd47718c640 1 -- 172.21.15.172:0/2339908584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd458004800 msgr2=0x7fd458004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:52.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.304+0000 7fd47718c640 1 --2- 172.21.15.172:0/2339908584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd458004800 0x7fd458004c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd464009a90 tx=0x7fd458005830 comp rx=0 tx=0).stop 2024-04-14T08:32:52.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.304+0000 7fd475729640 1 -- 172.21.15.172:0/2339908584 reap_dead start 2024-04-14T08:32:52.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.304+0000 7fd47718c640 1 -- 172.21.15.172:0/2339908584 shutdown_connections 2024-04-14T08:32:52.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.304+0000 7fd47718c640 1 -- 172.21.15.172:0/2339908584 >> 172.21.15.172:0/2339908584 conn(0x7fd4700b3af0 msgr2=0x7fd4700b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:52.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.304+0000 7fd47718c640 1 -- 172.21.15.172:0/2339908584 shutdown_connections 2024-04-14T08:32:52.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.304+0000 7fd47718c640 1 -- 172.21.15.172:0/2339908584 wait complete. 2024-04-14T08:32:52.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.305+0000 7fd47718c640 1 Processor -- start 2024-04-14T08:32:52.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.305+0000 7fd47718c640 1 -- start start 2024-04-14T08:32:52.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.306+0000 7fd47718c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd458004800 0x7fd4700cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:52.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.306+0000 7fd47718c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4700cb130 0x7fd4700ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:52.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.306+0000 7fd47718c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700cef30 0x7fd4700c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:52.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.306+0000 7fd47718c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd47006d040 con 0x7fd458004800 2024-04-14T08:32:52.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.306+0000 7fd47718c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd47015e7a0 con 0x7fd4700cb130 2024-04-14T08:32:52.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.306+0000 7fd47718c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd47006d1c0 con 0x7fd4700cef30 2024-04-14T08:32:52.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.306+0000 7fd474f28640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd458004800 0x7fd4700cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:52.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.306+0000 7fd474f28640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd458004800 0x7fd4700cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58138/0 (socket says 172.21.15.172:58138) 2024-04-14T08:32:52.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.306+0000 7fd474f28640 1 -- 172.21.15.172:0/2022434520 learned_addr learned my addr 172.21.15.172:0/2022434520 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.307+0000 7fd474f28640 1 -- 172.21.15.172:0/2022434520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700cef30 msgr2=0x7fd4700c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.307+0000 7fd474f28640 1 --2- 172.21.15.172:0/2022434520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700cef30 0x7fd4700c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.307+0000 7fd474f28640 1 -- 172.21.15.172:0/2022434520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4700cb130 msgr2=0x7fd4700ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.307+0000 7fd474f28640 1 --2- 172.21.15.172:0/2022434520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4700cb130 0x7fd4700ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.307+0000 7fd474f28640 1 -- 172.21.15.172:0/2022434520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4700c9de0 con 0x7fd458004800 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.307+0000 7fd474f28640 1 --2- 172.21.15.172:0/2022434520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd458004800 0x7fd4700cabf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd460007290 tx=0x7fd460007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.307+0000 7fd46d7fa640 1 -- 172.21.15.172:0/2022434520 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46000f960 con 0x7fd458004800 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.307+0000 7fd46d7fa640 1 -- 172.21.15.172:0/2022434520 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd460018070 con 0x7fd458004800 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.307+0000 7fd47718c640 1 -- 172.21.15.172:0/2022434520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4700ca0c0 con 0x7fd458004800 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.307+0000 7fd46d7fa640 1 -- 172.21.15.172:0/2022434520 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46000fd60 con 0x7fd458004800 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.307+0000 7fd47718c640 1 -- 172.21.15.172:0/2022434520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4700c9610 con 0x7fd458004800 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.308+0000 7fd47718c640 1 -- 172.21.15.172:0/2022434520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd470068590 con 0x7fd458004800 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.308+0000 7fd46d7fa640 1 -- 172.21.15.172:0/2022434520 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd46001d460 con 0x7fd458004800 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.309+0000 7fd46d7fa640 1 --2- 172.21.15.172:0/2022434520 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd45c06c8f0 0x7fd45c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:52.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.309+0000 7fd46ffff640 1 --2- 172.21.15.172:0/2022434520 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd45c06c8f0 0x7fd45c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:52.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.309+0000 7fd46d7fa640 1 -- 172.21.15.172:0/2022434520 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fd46008bae0 con 0x7fd458004800 2024-04-14T08:32:52.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.309+0000 7fd46ffff640 1 --2- 172.21.15.172:0/2022434520 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd45c06c8f0 0x7fd45c06edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd464036000 tx=0x7fd46402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.313+0000 7fd46d7fa640 1 -- 172.21.15.172:0/2022434520 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd46005a0f0 con 0x7fd458004800 2024-04-14T08:32:52.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:52.489+0000 7fd47718c640 1 -- 172.21.15.172:0/2022434520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fd4700b5680 con 0x7fd458004800 2024-04-14T08:32:52.728 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:32:52.730 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:32:53.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.177+0000 7fd46d7fa640 1 -- 172.21.15.172:0/2022434520 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7fd4600599f0 con 0x7fd458004800 2024-04-14T08:32:53.178 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' created 2024-04-14T08:32:53.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.179+0000 7fd47718c640 1 -- 172.21.15.172:0/2022434520 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd45c06c8f0 msgr2=0x7fd45c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:53.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.179+0000 7fd47718c640 1 --2- 172.21.15.172:0/2022434520 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd45c06c8f0 0x7fd45c06edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd464036000 tx=0x7fd46402e000 comp rx=0 tx=0).stop 2024-04-14T08:32:53.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.179+0000 7fd47718c640 1 -- 172.21.15.172:0/2022434520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd458004800 msgr2=0x7fd4700cabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.179+0000 7fd47718c640 1 --2- 172.21.15.172:0/2022434520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd458004800 0x7fd4700cabf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd460007290 tx=0x7fd460007750 comp rx=0 tx=0).stop 2024-04-14T08:32:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.180+0000 7fd47718c640 1 -- 172.21.15.172:0/2022434520 shutdown_connections 2024-04-14T08:32:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.180+0000 7fd47718c640 1 --2- 172.21.15.172:0/2022434520 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd45c06c8f0 0x7fd45c06edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.180+0000 7fd47718c640 1 --2- 172.21.15.172:0/2022434520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700cef30 0x7fd4700c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.180+0000 7fd47718c640 1 --2- 172.21.15.172:0/2022434520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4700cb130 0x7fd4700ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.180+0000 7fd47718c640 1 --2- 172.21.15.172:0/2022434520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd458004800 0x7fd4700cabf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.180+0000 7fd47718c640 1 -- 172.21.15.172:0/2022434520 >> 172.21.15.172:0/2022434520 conn(0x7fd4700b3af0 msgr2=0x7fd470160660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.180+0000 7fd47718c640 1 -- 172.21.15.172:0/2022434520 shutdown_connections 2024-04-14T08:32:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.180+0000 7fd47718c640 1 -- 172.21.15.172:0/2022434520 wait complete. 2024-04-14T08:32:53.205 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-14T08:32:53.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.310+0000 7fa8c71aa640 1 Processor -- start 2024-04-14T08:32:53.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.310+0000 7fa8c71aa640 1 -- start start 2024-04-14T08:32:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.310+0000 7fa8c71aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8c00d1a20 0x7fa8c006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.310+0000 7fa8c71aa640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa8c006d000 0x7fa8c006d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.310+0000 7fa8c71aa640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c00701b0 con 0x7fa8c0071250 2024-04-14T08:32:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.310+0000 7fa8c71aa640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa8c006d9d0 con 0x7fa8c006d000 2024-04-14T08:32:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.310+0000 7fa8c71aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c0070dc0 con 0x7fa8c00d1a20 2024-04-14T08:32:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.311+0000 7fa8c4f46640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8c00d1a20 0x7fa8c006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.311+0000 7fa8c5747640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa8c006d000 0x7fa8c006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.311+0000 7fa8c4f46640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8c00d1a20 0x7fa8c006fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:51818/0 (socket says 172.21.15.172:51818) 2024-04-14T08:32:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.311+0000 7fa8bffff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fa8c0071250 0x7fa8c00cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:40492/0 (socket says 172.21.15.172:40492) 2024-04-14T08:32:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.311+0000 7fa8c5747640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa8c006d000 0x7fa8c006d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:53490/0 (socket says 172.21.15.172:53490) 2024-04-14T08:32:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.311+0000 7fa8c4f46640 1 -- 172.21.15.172:0/3603267308 learned_addr learned my addr 172.21.15.172:0/3603267308 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.311+0000 7fa8c4f46640 1 -- 172.21.15.172:0/3603267308 >> v2:172.21.15.177:3300/0 conn(0x7fa8c006d000 msgr2=0x7fa8c006d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.311+0000 7fa8c4f46640 1 --2- 172.21.15.172:0/3603267308 >> v2:172.21.15.177:3300/0 conn(0x7fa8c006d000 0x7fa8c006d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.311+0000 7fa8c4f46640 1 -- 172.21.15.172:0/3603267308 >> v1:172.21.15.177:6789/0 conn(0x7fa8c0071250 legacy=0x7fa8c00cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.311+0000 7fa8c4f46640 1 -- 172.21.15.172:0/3603267308 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c0070490 con 0x7fa8c00d1a20 2024-04-14T08:32:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.311+0000 7fa8c5747640 1 --2- 172.21.15.172:0/3603267308 >> v2:172.21.15.177:3300/0 conn(0x7fa8c006d000 0x7fa8c006d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.312+0000 7fa8c4f46640 1 --2- 172.21.15.172:0/3603267308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8c00d1a20 0x7fa8c006fb80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009870 tx=0x7fa8a802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd297772e620360f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.312+0000 7fa8bf7fe640 1 -- 172.21.15.172:0/3603267308 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8035020 con 0x7fa8c00d1a20 2024-04-14T08:32:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.312+0000 7fa8bf7fe640 1 -- 172.21.15.172:0/3603267308 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a8031640 con 0x7fa8c00d1a20 2024-04-14T08:32:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.312+0000 7fa8bf7fe640 1 -- 172.21.15.172:0/3603267308 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a802d040 con 0x7fa8c00d1a20 2024-04-14T08:32:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.312+0000 7fa8c71aa640 1 -- 172.21.15.172:0/3603267308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8c00d1a20 msgr2=0x7fa8c006fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.312+0000 7fa8c71aa640 1 --2- 172.21.15.172:0/3603267308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8c00d1a20 0x7fa8c006fb80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009870 tx=0x7fa8a802b8f0 comp rx=0 tx=0).stop 2024-04-14T08:32:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.312+0000 7fa8c71aa640 1 -- 172.21.15.172:0/3603267308 shutdown_connections 2024-04-14T08:32:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.312+0000 7fa8c71aa640 1 --2- 172.21.15.172:0/3603267308 >> v2:172.21.15.177:3300/0 conn(0x7fa8c006d000 0x7fa8c006d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.312+0000 7fa8c71aa640 1 --2- 172.21.15.172:0/3603267308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8c00d1a20 0x7fa8c006fb80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.312+0000 7fa8c71aa640 1 -- 172.21.15.172:0/3603267308 >> 172.21.15.172:0/3603267308 conn(0x7fa8c008c000 msgr2=0x7fa8c00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.313+0000 7fa8c71aa640 1 -- 172.21.15.172:0/3603267308 shutdown_connections 2024-04-14T08:32:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.313+0000 7fa8c71aa640 1 -- 172.21.15.172:0/3603267308 wait complete. 2024-04-14T08:32:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.314+0000 7fa8c71aa640 1 Processor -- start 2024-04-14T08:32:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.314+0000 7fa8c71aa640 1 -- start start 2024-04-14T08:32:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c71aa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c006d000 0x7fa8c00b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c71aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8c0071250 0x7fa8c01f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c71aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8c00d1a20 0x7fa8c00b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c71aa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c00d1e00 con 0x7fa8c00d1a20 2024-04-14T08:32:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c71aa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c0070a50 con 0x7fa8c006d000 2024-04-14T08:32:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c71aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c01f8160 con 0x7fa8c0071250 2024-04-14T08:32:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c5747640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8c00d1a20 0x7fa8c00b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c5747640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8c00d1a20 0x7fa8c00b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58140/0 (socket says 172.21.15.172:58140) 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c5747640 1 -- 172.21.15.172:0/1894274282 learned_addr learned my addr 172.21.15.172:0/1894274282 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8bffff640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8c0071250 0x7fa8c01f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c4f46640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c006d000 0x7fa8c00b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c5747640 1 -- 172.21.15.172:0/1894274282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8c0071250 msgr2=0x7fa8c01f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c5747640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8c0071250 0x7fa8c01f85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c5747640 1 -- 172.21.15.172:0/1894274282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c006d000 msgr2=0x7fa8c00b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c5747640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c006d000 0x7fa8c00b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c5747640 1 -- 172.21.15.172:0/1894274282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c01f4fe0 con 0x7fa8c00d1a20 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c5747640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8c00d1a20 0x7fa8c00b37f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800efc0 tx=0x7fa8b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8bd7fa640 1 -- 172.21.15.172:0/1894274282 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8018030 con 0x7fa8c00d1a20 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8bd7fa640 1 -- 172.21.15.172:0/1894274282 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b800cdf0 con 0x7fa8c00d1a20 2024-04-14T08:32:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c71aa640 1 -- 172.21.15.172:0/1894274282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c01f52c0 con 0x7fa8c00d1a20 2024-04-14T08:32:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8bd7fa640 1 -- 172.21.15.172:0/1894274282 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8003bb0 con 0x7fa8c00d1a20 2024-04-14T08:32:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.315+0000 7fa8c71aa640 1 -- 172.21.15.172:0/1894274282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c01f8b30 con 0x7fa8c00d1a20 2024-04-14T08:32:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.316+0000 7fa8c71aa640 1 -- 172.21.15.172:0/1894274282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c0075490 con 0x7fa8c00d1a20 2024-04-14T08:32:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.317+0000 7fa8bd7fa640 1 -- 172.21.15.172:0/1894274282 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8b8005d10 con 0x7fa8c00d1a20 2024-04-14T08:32:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.317+0000 7fa8bd7fa640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8a006c8f0 0x7fa8a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.317+0000 7fa8bd7fa640 1 -- 172.21.15.172:0/1894274282 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7fa8b808c2d0 con 0x7fa8c00d1a20 2024-04-14T08:32:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.317+0000 7fa8c4f46640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8a006c8f0 0x7fa8a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.318+0000 7fa8c4f46640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8a006c8f0 0x7fa8a006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8036000 tx=0x7fa8a803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.322+0000 7fa8bd7fa640 1 -- 172.21.15.172:0/1894274282 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa8b8022070 con 0x7fa8c00d1a20 2024-04-14T08:32:53.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:53.534+0000 7fa8c71aa640 1 -- 172.21.15.172:0/1894274282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa8c00cca80 con 0x7fa8c00d1a20 2024-04-14T08:32:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.180+0000 7fa8bd7fa640 1 -- 172.21.15.172:0/1894274282 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fa8b8010040 con 0x7fa8c00d1a20 2024-04-14T08:32:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:new fs with metadata pool 2 and data pool 3 2024-04-14T08:32:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.184+0000 7fa8c71aa640 1 -- 172.21.15.172:0/1894274282 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8a006c8f0 msgr2=0x7fa8a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.184+0000 7fa8c71aa640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8a006c8f0 0x7fa8a006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8036000 tx=0x7fa8a803c040 comp rx=0 tx=0).stop 2024-04-14T08:32:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.184+0000 7fa8c71aa640 1 -- 172.21.15.172:0/1894274282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8c00d1a20 msgr2=0x7fa8c00b37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.184+0000 7fa8c71aa640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8c00d1a20 0x7fa8c00b37f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800efc0 tx=0x7fa8b800c490 comp rx=0 tx=0).stop 2024-04-14T08:32:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.184+0000 7fa8c71aa640 1 -- 172.21.15.172:0/1894274282 shutdown_connections 2024-04-14T08:32:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.184+0000 7fa8c71aa640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8a006c8f0 0x7fa8a006edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.184+0000 7fa8c71aa640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8c00d1a20 0x7fa8c00b37f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.184+0000 7fa8c71aa640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8c0071250 0x7fa8c01f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.184+0000 7fa8c71aa640 1 --2- 172.21.15.172:0/1894274282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c006d000 0x7fa8c00b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.184+0000 7fa8c71aa640 1 -- 172.21.15.172:0/1894274282 >> 172.21.15.172:0/1894274282 conn(0x7fa8c008c000 msgr2=0x7fa8c01e5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.184+0000 7fa8c71aa640 1 -- 172.21.15.172:0/1894274282 shutdown_connections 2024-04-14T08:32:54.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.184+0000 7fa8c71aa640 1 -- 172.21.15.172:0/1894274282 wait complete. 2024-04-14T08:32:54.205 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-14T08:32:54.205 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-14T08:32:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.311+0000 7f0a7b71e640 1 Processor -- start 2024-04-14T08:32:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.312+0000 7f0a7b71e640 1 -- start start 2024-04-14T08:32:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.312+0000 7f0a7b71e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a740cc340 0x7f0a740ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.312+0000 7f0a7b71e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a740cdb90 con 0x7f0a7415b350 2024-04-14T08:32:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.312+0000 7f0a7b71e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a740cde70 con 0x7f0a740cecc0 2024-04-14T08:32:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.312+0000 7f0a7b71e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0a740cf150 con 0x7f0a740cc340 2024-04-14T08:32:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.312+0000 7f0a794ba640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f0a7415b350 0x7f0a740cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34858/0 (socket says 172.21.15.172:34858) 2024-04-14T08:32:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.312+0000 7f0a794ba640 1 -- 172.21.15.172:0/3551729201 learned_addr learned my addr 172.21.15.172:0/3551729201 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:32:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a78cb9640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a740cc340 0x7f0a740ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 305962318 0 0) 0x7f0a740cdb90 con 0x7f0a7415b350 2024-04-14T08:32:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a60003520 con 0x7f0a7415b350 2024-04-14T08:32:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 439524834 0 0) 0x7f0a60003520 con 0x7f0a7415b350 2024-04-14T08:32:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a740cc340 msgr2=0x7f0a740ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a740cc340 0x7f0a740ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 >> v1:172.21.15.177:6789/0 conn(0x7f0a740cecc0 legacy=0x7f0a740c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a740ce150 con 0x7f0a7415b350 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f0a700019f0 con 0x7f0a7415b350 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 >> v1:172.21.15.172:6789/0 conn(0x7f0a7415b350 legacy=0x7f0a740cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a740cc340 0x7f0a600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a60003f60 0x7f0a60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a60004d40 0x7f0a600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a740cf150 con 0x7f0a60004d40 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6000dd60 con 0x7f0a60003f60 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6000df60 con 0x7f0a740cc340 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a79cbb640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a60004d40 0x7f0a600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a794ba640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a740cc340 0x7f0a600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a78cb9640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a60003f60 0x7f0a60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a79cbb640 1 -- 172.21.15.172:0/3551729201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a740cc340 msgr2=0x7f0a600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a79cbb640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a740cc340 0x7f0a600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a79cbb640 1 -- 172.21.15.172:0/3551729201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a60003f60 msgr2=0x7f0a60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a79cbb640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a60003f60 0x7f0a60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a79cbb640 1 -- 172.21.15.172:0/3551729201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a60003520 con 0x7f0a60004d40 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.313+0000 7f0a794ba640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a740cc340 0x7f0a600039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.314+0000 7f0a79cbb640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a60004d40 0x7f0a600051c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c001750 tx=0x7f0a60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e6d0a4c97f1fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.314+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c001370 con 0x7f0a60004d40 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.314+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a6c0052d0 con 0x7f0a60004d40 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.314+0000 7f0a6bfff640 1 -- 172.21.15.172:0/3551729201 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c003830 con 0x7f0a60004d40 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.315+0000 7f0a7b71e640 1 -- 172.21.15.172:0/3551729201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a60004d40 msgr2=0x7f0a600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.315+0000 7f0a7b71e640 1 --2- 172.21.15.172:0/3551729201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a60004d40 0x7f0a600051c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c001750 tx=0x7f0a60005830 comp rx=0 tx=0).stop 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.315+0000 7f0a79cbb640 1 -- 172.21.15.172:0/3551729201 reap_dead start 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.315+0000 7f0a7b71e640 1 -- 172.21.15.172:0/3551729201 shutdown_connections 2024-04-14T08:32:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.315+0000 7f0a7b71e640 1 -- 172.21.15.172:0/3551729201 >> 172.21.15.172:0/3551729201 conn(0x7f0a740b1a50 msgr2=0x7f0a740b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.315+0000 7f0a7b71e640 1 -- 172.21.15.172:0/3551729201 shutdown_connections 2024-04-14T08:32:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.315+0000 7f0a7b71e640 1 -- 172.21.15.172:0/3551729201 wait complete. 2024-04-14T08:32:54.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.316+0000 7f0a7b71e640 1 Processor -- start 2024-04-14T08:32:54.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.316+0000 7f0a7b71e640 1 -- start start 2024-04-14T08:32:54.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a7b71e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a60004d40 0x7f0a7415b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:54.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a7b71e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a7409c070 0x7f0a7414db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:54.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a7b71e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a7409d610 0x7f0a74159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:54.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a7b71e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a740ce4c0 con 0x7f0a60004d40 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a7b71e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0a740cd940 con 0x7f0a7409d610 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a7b71e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0a740ce780 con 0x7f0a7409c070 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a794ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a60004d40 0x7f0a7415b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a794ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a60004d40 0x7f0a7415b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58162/0 (socket says 172.21.15.172:58162) 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a794ba640 1 -- 172.21.15.172:0/485911896 learned_addr learned my addr 172.21.15.172:0/485911896 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a78cb9640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a7409c070 0x7f0a7414db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a79cbb640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a7409d610 0x7f0a74159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a794ba640 1 -- 172.21.15.172:0/485911896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a7409c070 msgr2=0x7f0a7414db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a794ba640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a7409c070 0x7f0a7414db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a794ba640 1 -- 172.21.15.172:0/485911896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a7409d610 msgr2=0x7f0a74159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a794ba640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a7409d610 0x7f0a74159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a794ba640 1 -- 172.21.15.172:0/485911896 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7409c570 con 0x7f0a60004d40 2024-04-14T08:32:54.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a79cbb640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a7409d610 0x7f0a74159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:32:54.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a794ba640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a60004d40 0x7f0a7415b7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0a70007b10 tx=0x7f0a70007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:54.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a69ffb640 1 -- 172.21.15.172:0/485911896 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70013420 con 0x7f0a60004d40 2024-04-14T08:32:54.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a69ffb640 1 -- 172.21.15.172:0/485911896 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a70018070 con 0x7f0a60004d40 2024-04-14T08:32:54.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a7b71e640 1 -- 172.21.15.172:0/485911896 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7409c850 con 0x7f0a60004d40 2024-04-14T08:32:54.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a69ffb640 1 -- 172.21.15.172:0/485911896 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a700137c0 con 0x7f0a60004d40 2024-04-14T08:32:54.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.317+0000 7f0a7b71e640 1 -- 172.21.15.172:0/485911896 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7409d0f0 con 0x7f0a60004d40 2024-04-14T08:32:54.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.318+0000 7f0a7b71e640 1 -- 172.21.15.172:0/485911896 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a740686a0 con 0x7f0a60004d40 2024-04-14T08:32:54.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.319+0000 7f0a69ffb640 1 -- 172.21.15.172:0/485911896 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a70013960 con 0x7f0a60004d40 2024-04-14T08:32:54.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.319+0000 7f0a69ffb640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a5806c8f0 0x7f0a5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:54.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.319+0000 7f0a69ffb640 1 -- 172.21.15.172:0/485911896 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0a7008c3f0 con 0x7f0a60004d40 2024-04-14T08:32:54.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.319+0000 7f0a78cb9640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a5806c8f0 0x7f0a5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:54.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.319+0000 7f0a78cb9640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a5806c8f0 0x7f0a5806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a740b6220 tx=0x7f0a64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:54.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.324+0000 7f0a69ffb640 1 -- 172.21.15.172:0/485911896 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0a7005a790 con 0x7f0a60004d40 2024-04-14T08:32:54.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:54.507+0000 7f0a7b71e640 1 -- 172.21.15.172:0/485911896 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f0a74073410 con 0x7f0a60004d40 2024-04-14T08:32:55.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.201+0000 7f0a69ffb640 1 -- 172.21.15.172:0/485911896 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7f0a7005a090 con 0x7f0a60004d40 2024-04-14T08:32:55.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.205+0000 7f0a7b71e640 1 -- 172.21.15.172:0/485911896 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a5806c8f0 msgr2=0x7f0a5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:55.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.205+0000 7f0a7b71e640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a5806c8f0 0x7f0a5806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a740b6220 tx=0x7f0a64008040 comp rx=0 tx=0).stop 2024-04-14T08:32:55.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.205+0000 7f0a7b71e640 1 -- 172.21.15.172:0/485911896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a60004d40 msgr2=0x7f0a7415b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:55.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.205+0000 7f0a7b71e640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a60004d40 0x7f0a7415b7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0a70007b10 tx=0x7f0a70007c00 comp rx=0 tx=0).stop 2024-04-14T08:32:55.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.205+0000 7f0a7b71e640 1 -- 172.21.15.172:0/485911896 shutdown_connections 2024-04-14T08:32:55.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.205+0000 7f0a7b71e640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a5806c8f0 0x7f0a5806edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:55.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.205+0000 7f0a7b71e640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a7409d610 0x7f0a74159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:55.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.205+0000 7f0a7b71e640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a7409c070 0x7f0a7414db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:55.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.205+0000 7f0a7b71e640 1 --2- 172.21.15.172:0/485911896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a60004d40 0x7f0a7415b7a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:55.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.205+0000 7f0a7b71e640 1 -- 172.21.15.172:0/485911896 >> 172.21.15.172:0/485911896 conn(0x7f0a740b1a50 msgr2=0x7f0a740ae140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:55.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.205+0000 7f0a7b71e640 1 -- 172.21.15.172:0/485911896 shutdown_connections 2024-04-14T08:32:55.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.205+0000 7f0a7b71e640 1 -- 172.21.15.172:0/485911896 wait complete. 2024-04-14T08:32:55.219 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-14T08:32:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.334+0000 7f206fd55640 1 Processor -- start 2024-04-14T08:32:55.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.335+0000 7f206fd55640 1 -- start start 2024-04-14T08:32:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.335+0000 7f206fd55640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20680b1440 0x7f20680b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.335+0000 7f206fd55640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f20680b23b0 0x7f20680d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.335+0000 7f206fd55640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f206806d370 con 0x7f20680b2810 2024-04-14T08:32:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.335+0000 7f206fd55640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f20680b1960 con 0x7f20680b23b0 2024-04-14T08:32:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.335+0000 7f206fd55640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f206806d650 con 0x7f20680b1440 2024-04-14T08:32:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.335+0000 7f206e2f2640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f20680b2810 0x7f2068162420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34860/0 (socket says 172.21.15.172:34860) 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.336+0000 7f206e2f2640 1 -- 172.21.15.172:0/1540068528 learned_addr learned my addr 172.21.15.172:0/1540068528 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.336+0000 7f206daf1640 1 --2- 172.21.15.172:0/1540068528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20680b1440 0x7f20680b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.336+0000 7f206d2f0640 1 --2- 172.21.15.172:0/1540068528 >> v2:172.21.15.177:3300/0 conn(0x7f20680b23b0 0x7f20680d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.336+0000 7f206caef640 1 -- 172.21.15.172:0/1540068528 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 773634796 0 0) 0x7f206806d370 con 0x7f20680b2810 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.336+0000 7f206caef640 1 -- 172.21.15.172:0/1540068528 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2054003520 con 0x7f20680b2810 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.336+0000 7f206d2f0640 1 -- 172.21.15.172:0/1540068528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20680b1440 msgr2=0x7f20680b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.336+0000 7f206d2f0640 1 --2- 172.21.15.172:0/1540068528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20680b1440 0x7f20680b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.336+0000 7f206d2f0640 1 -- 172.21.15.172:0/1540068528 >> v1:172.21.15.172:6789/0 conn(0x7f20680b2810 legacy=0x7f2068162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.336+0000 7f206d2f0640 1 -- 172.21.15.172:0/1540068528 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206806d930 con 0x7f20680b23b0 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.336+0000 7f206daf1640 1 --2- 172.21.15.172:0/1540068528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20680b1440 0x7f20680b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.337+0000 7f206d2f0640 1 --2- 172.21.15.172:0/1540068528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20680b23b0 0x7f20680d0350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f20580097a0 tx=0x7f205802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1316512beffd4ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.337+0000 7f206caef640 1 -- 172.21.15.172:0/1540068528 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20580315a0 con 0x7f20680b23b0 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.337+0000 7f206caef640 1 -- 172.21.15.172:0/1540068528 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2058037070 con 0x7f20680b23b0 2024-04-14T08:32:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.337+0000 7f206caef640 1 -- 172.21.15.172:0/1540068528 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058031940 con 0x7f20680b23b0 2024-04-14T08:32:55.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.337+0000 7f206fd55640 1 -- 172.21.15.172:0/1540068528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20680b23b0 msgr2=0x7f20680d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:55.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.337+0000 7f206fd55640 1 --2- 172.21.15.172:0/1540068528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20680b23b0 0x7f20680d0350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f20580097a0 tx=0x7f205802b840 comp rx=0 tx=0).stop 2024-04-14T08:32:55.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.337+0000 7f206fd55640 1 -- 172.21.15.172:0/1540068528 shutdown_connections 2024-04-14T08:32:55.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.337+0000 7f206fd55640 1 --2- 172.21.15.172:0/1540068528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20680b23b0 0x7f20680d0350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:55.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.337+0000 7f206fd55640 1 --2- 172.21.15.172:0/1540068528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20680b1440 0x7f20680b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:55.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.337+0000 7f206fd55640 1 -- 172.21.15.172:0/1540068528 >> 172.21.15.172:0/1540068528 conn(0x7f20680b3b10 msgr2=0x7f20680b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:55.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.337+0000 7f206fd55640 1 -- 172.21.15.172:0/1540068528 shutdown_connections 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.337+0000 7f206fd55640 1 -- 172.21.15.172:0/1540068528 wait complete. 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206fd55640 1 Processor -- start 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206fd55640 1 -- start start 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206fd55640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20680b1440 0x7f20680cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206fd55640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20680b23b0 0x7f20680c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206fd55640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20680b2810 0x7f20680b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206fd55640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f20680d0890 con 0x7f20680b1440 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206fd55640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f20680b1e90 con 0x7f20680b2810 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206fd55640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f20680d0a10 con 0x7f20680b23b0 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206daf1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20680b1440 0x7f20680cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206daf1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20680b1440 0x7f20680cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58176/0 (socket says 172.21.15.172:58176) 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206daf1640 1 -- 172.21.15.172:0/1601630533 learned_addr learned my addr 172.21.15.172:0/1601630533 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:55.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206d2f0640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20680b23b0 0x7f20680c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:55.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.339+0000 7f206e2f2640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20680b2810 0x7f20680b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:55.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.340+0000 7f206daf1640 1 -- 172.21.15.172:0/1601630533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20680b23b0 msgr2=0x7f20680c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:55.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.340+0000 7f206daf1640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20680b23b0 0x7f20680c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:55.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.340+0000 7f206daf1640 1 -- 172.21.15.172:0/1601630533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20680b2810 msgr2=0x7f20680b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:55.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.340+0000 7f206daf1640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20680b2810 0x7f20680b74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:55.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.340+0000 7f206daf1640 1 -- 172.21.15.172:0/1601630533 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2068152450 con 0x7f20680b1440 2024-04-14T08:32:55.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.340+0000 7f206daf1640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20680b1440 0x7f20680cb8c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f206400ebe0 tx=0x7f206400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:55.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.340+0000 7f205e7fc640 1 -- 172.21.15.172:0/1601630533 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064017070 con 0x7f20680b1440 2024-04-14T08:32:55.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.340+0000 7f205e7fc640 1 -- 172.21.15.172:0/1601630533 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f206400cc30 con 0x7f20680b1440 2024-04-14T08:32:55.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.340+0000 7f206fd55640 1 -- 172.21.15.172:0/1601630533 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20680c6df0 con 0x7f20680b1440 2024-04-14T08:32:55.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.340+0000 7f205e7fc640 1 -- 172.21.15.172:0/1601630533 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064013470 con 0x7f20680b1440 2024-04-14T08:32:55.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.340+0000 7f206fd55640 1 -- 172.21.15.172:0/1601630533 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20680c73b0 con 0x7f20680b1440 2024-04-14T08:32:55.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.341+0000 7f206fd55640 1 -- 172.21.15.172:0/1601630533 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2068068520 con 0x7f20680b1440 2024-04-14T08:32:55.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.342+0000 7f205e7fc640 1 -- 172.21.15.172:0/1601630533 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f206401a070 con 0x7f20680b1440 2024-04-14T08:32:55.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.342+0000 7f205e7fc640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f204c06c8f0 0x7f204c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:55.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.342+0000 7f205e7fc640 1 -- 172.21.15.172:0/1601630533 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7f206408bb80 con 0x7f20680b1440 2024-04-14T08:32:55.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.342+0000 7f206d2f0640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f204c06c8f0 0x7f204c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:55.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.342+0000 7f206d2f0640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f204c06c8f0 0x7f204c06edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2058036000 tx=0x7f205803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:55.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.347+0000 7f205e7fc640 1 -- 172.21.15.172:0/1601630533 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2064059830 con 0x7f20680b1440 2024-04-14T08:32:55.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:55.520+0000 7f206fd55640 1 -- 172.21.15.172:0/1601630533 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f206806c1c0 con 0x7f20680b1440 2024-04-14T08:32:57.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.216+0000 7f205e7fc640 1 -- 172.21.15.172:0/1601630533 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7f206405db30 con 0x7f20680b1440 2024-04-14T08:32:57.217 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data_ec' created 2024-04-14T08:32:57.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.220+0000 7f206fd55640 1 -- 172.21.15.172:0/1601630533 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f204c06c8f0 msgr2=0x7f204c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:57.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.220+0000 7f206fd55640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f204c06c8f0 0x7f204c06edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2058036000 tx=0x7f205803d040 comp rx=0 tx=0).stop 2024-04-14T08:32:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.220+0000 7f206fd55640 1 -- 172.21.15.172:0/1601630533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20680b1440 msgr2=0x7f20680cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.220+0000 7f206fd55640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20680b1440 0x7f20680cb8c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f206400ebe0 tx=0x7f206400c520 comp rx=0 tx=0).stop 2024-04-14T08:32:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.221+0000 7f206fd55640 1 -- 172.21.15.172:0/1601630533 shutdown_connections 2024-04-14T08:32:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.221+0000 7f206fd55640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f204c06c8f0 0x7f204c06edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.221+0000 7f206fd55640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20680b2810 0x7f20680b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.221+0000 7f206fd55640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20680b23b0 0x7f20680c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.221+0000 7f206fd55640 1 --2- 172.21.15.172:0/1601630533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20680b1440 0x7f20680cb8c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.221+0000 7f206fd55640 1 -- 172.21.15.172:0/1601630533 >> 172.21.15.172:0/1601630533 conn(0x7f20680b3b10 msgr2=0x7f2068070ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.221+0000 7f206fd55640 1 -- 172.21.15.172:0/1601630533 shutdown_connections 2024-04-14T08:32:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.221+0000 7f206fd55640 1 -- 172.21.15.172:0/1601630533 wait complete. 2024-04-14T08:32:57.240 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-14T08:32:57.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.348+0000 7f801ded5640 1 Processor -- start 2024-04-14T08:32:57.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.348+0000 7f801ded5640 1 -- start start 2024-04-14T08:32:57.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.349+0000 7f801ded5640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f801815e3b0 0x7f8018162a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:57.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.349+0000 7f801ded5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f801806bc30 0x7f80180ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:57.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.349+0000 7f801ded5640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8018162020 con 0x7f80180b5660 2024-04-14T08:32:57.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.349+0000 7f801ded5640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f80180b00e0 con 0x7f801815e3b0 2024-04-14T08:32:57.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.349+0000 7f801ded5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f80180aecd0 con 0x7f801806bc30 2024-04-14T08:32:57.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.349+0000 7f80177fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f801815e3b0 0x7f8018162a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:57.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.349+0000 7f80177fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f801815e3b0 0x7f8018162a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58184/0 (socket says 172.21.15.172:58184) 2024-04-14T08:32:57.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.349+0000 7f80177fe640 1 -- 172.21.15.172:0/2060870202 learned_addr learned my addr 172.21.15.172:0/2060870202 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:57.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.350+0000 7f8017fff640 1 --2- 172.21.15.172:0/2060870202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f801806bc30 0x7f80180ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:57.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.350+0000 7f80177fe640 1 -- 172.21.15.172:0/2060870202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f801806bc30 msgr2=0x7f80180ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:57.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.350+0000 7f80177fe640 1 --2- 172.21.15.172:0/2060870202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f801806bc30 0x7f80180ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:57.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.350+0000 7f80177fe640 1 -- 172.21.15.172:0/2060870202 >> v1:172.21.15.177:6789/0 conn(0x7f80180b5660 legacy=0x7f801806b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:57.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.350+0000 7f80177fe640 1 -- 172.21.15.172:0/2060870202 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80180b0310 con 0x7f801815e3b0 2024-04-14T08:32:57.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.350+0000 7f80177fe640 1 --2- 172.21.15.172:0/2060870202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f801815e3b0 0x7f8018162a90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8008009870 tx=0x7f800802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeec574900fe56b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:57.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.351+0000 7f8016ffd640 1 -- 172.21.15.172:0/2060870202 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008034070 con 0x7f801815e3b0 2024-04-14T08:32:57.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.351+0000 7f8016ffd640 1 -- 172.21.15.172:0/2060870202 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8008030840 con 0x7f801815e3b0 2024-04-14T08:32:57.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.351+0000 7f8016ffd640 1 -- 172.21.15.172:0/2060870202 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008030c10 con 0x7f801815e3b0 2024-04-14T08:32:57.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.351+0000 7f801ded5640 1 -- 172.21.15.172:0/2060870202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f801815e3b0 msgr2=0x7f8018162a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:57.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.351+0000 7f801ded5640 1 --2- 172.21.15.172:0/2060870202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f801815e3b0 0x7f8018162a90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8008009870 tx=0x7f800802b980 comp rx=0 tx=0).stop 2024-04-14T08:32:57.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.351+0000 7f801ded5640 1 -- 172.21.15.172:0/2060870202 shutdown_connections 2024-04-14T08:32:57.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.351+0000 7f801ded5640 1 --2- 172.21.15.172:0/2060870202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f801806bc30 0x7f80180ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:57.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.351+0000 7f801ded5640 1 --2- 172.21.15.172:0/2060870202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f801815e3b0 0x7f8018162a90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.351+0000 7f801ded5640 1 -- 172.21.15.172:0/2060870202 >> 172.21.15.172:0/2060870202 conn(0x7f80180b2c40 msgr2=0x7f80180b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.351+0000 7f801ded5640 1 -- 172.21.15.172:0/2060870202 shutdown_connections 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.351+0000 7f801ded5640 1 -- 172.21.15.172:0/2060870202 wait complete. 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.352+0000 7f801ded5640 1 Processor -- start 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.352+0000 7f801ded5640 1 -- start start 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.353+0000 7f801ded5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f801806bc30 0x7f80181f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.353+0000 7f801ded5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80180b5660 0x7f8018155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.353+0000 7f801ded5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f801815e3b0 0x7f8018150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.353+0000 7f801ded5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8018162570 con 0x7f80180b5660 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.353+0000 7f801ded5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f80180b5b10 con 0x7f801815e3b0 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.353+0000 7f801ded5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f801815e790 con 0x7f801806bc30 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.353+0000 7f800ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80180b5660 0x7f8018155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.353+0000 7f800ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80180b5660 0x7f8018155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58196/0 (socket says 172.21.15.172:58196) 2024-04-14T08:32:57.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.353+0000 7f800ffff640 1 -- 172.21.15.172:0/1492750155 learned_addr learned my addr 172.21.15.172:0/1492750155 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.353+0000 7f80177fe640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f801806bc30 0x7f80181f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.353+0000 7f8017fff640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f801815e3b0 0x7f8018150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.354+0000 7f800ffff640 1 -- 172.21.15.172:0/1492750155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f801806bc30 msgr2=0x7f80181f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.354+0000 7f800ffff640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f801806bc30 0x7f80181f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.354+0000 7f800ffff640 1 -- 172.21.15.172:0/1492750155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f801815e3b0 msgr2=0x7f8018150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.354+0000 7f800ffff640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f801815e3b0 0x7f8018150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.354+0000 7f800ffff640 1 -- 172.21.15.172:0/1492750155 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8018152030 con 0x7f80180b5660 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.354+0000 7f800ffff640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80180b5660 0x7f8018155320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f800000c480 tx=0x7f800000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.354+0000 7f8014ff9640 1 -- 172.21.15.172:0/1492750155 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000052d0 con 0x7f80180b5660 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.354+0000 7f8014ff9640 1 -- 172.21.15.172:0/1492750155 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8000005470 con 0x7f80180b5660 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.354+0000 7f8014ff9640 1 -- 172.21.15.172:0/1492750155 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000012870 con 0x7f80180b5660 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.354+0000 7f801ded5640 1 -- 172.21.15.172:0/1492750155 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80181522d0 con 0x7f80180b5660 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.354+0000 7f801ded5640 1 -- 172.21.15.172:0/1492750155 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8018152890 con 0x7f80180b5660 2024-04-14T08:32:57.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.355+0000 7f801ded5640 1 -- 172.21.15.172:0/1492750155 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80180686a0 con 0x7f80180b5660 2024-04-14T08:32:57.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.355+0000 7f8014ff9640 1 -- 172.21.15.172:0/1492750155 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8000012a10 con 0x7f80180b5660 2024-04-14T08:32:57.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.356+0000 7f8014ff9640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7ff806c8f0 0x7f7ff806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:57.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.356+0000 7f8014ff9640 1 -- 172.21.15.172:0/1492750155 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f800008b950 con 0x7f80180b5660 2024-04-14T08:32:57.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.356+0000 7f80177fe640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7ff806c8f0 0x7f7ff806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:57.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.356+0000 7f80177fe640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7ff806c8f0 0x7f7ff806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8008036000 tx=0x7f800803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:57.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.360+0000 7f8014ff9640 1 -- 172.21.15.172:0/1492750155 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f80000594a0 con 0x7f80180b5660 2024-04-14T08:32:57.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:57.534+0000 7f801ded5640 1 -- 172.21.15.172:0/1492750155 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f8018070930 con 0x7f80180b5660 2024-04-14T08:32:57.787 INFO:tasks.ceph.mds.d.smithi177.stderr:2024-04-14T08:32:57.786+0000 7f369c3f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:32:58.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.221+0000 7f8014ff9640 1 -- 172.21.15.172:0/1492750155 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f800005d7a0 con 0x7f80180b5660 2024-04-14T08:32:58.221 INFO:teuthology.orchestra.run.smithi172.stderr:set pool 4 allow_ec_overwrites to true 2024-04-14T08:32:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.224+0000 7f801ded5640 1 -- 172.21.15.172:0/1492750155 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7ff806c8f0 msgr2=0x7f7ff806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.224+0000 7f801ded5640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7ff806c8f0 0x7f7ff806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8008036000 tx=0x7f800803b040 comp rx=0 tx=0).stop 2024-04-14T08:32:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.224+0000 7f801ded5640 1 -- 172.21.15.172:0/1492750155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80180b5660 msgr2=0x7f8018155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.224+0000 7f801ded5640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80180b5660 0x7f8018155320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f800000c480 tx=0x7f800000c940 comp rx=0 tx=0).stop 2024-04-14T08:32:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.224+0000 7f801ded5640 1 -- 172.21.15.172:0/1492750155 shutdown_connections 2024-04-14T08:32:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.224+0000 7f801ded5640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7ff806c8f0 0x7f7ff806edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.224+0000 7f801ded5640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f801815e3b0 0x7f8018150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:58.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.224+0000 7f801ded5640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80180b5660 0x7f8018155320 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:58.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.224+0000 7f801ded5640 1 --2- 172.21.15.172:0/1492750155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f801806bc30 0x7f80181f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:58.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.224+0000 7f801ded5640 1 -- 172.21.15.172:0/1492750155 >> 172.21.15.172:0/1492750155 conn(0x7f80180b2c40 msgr2=0x7f801806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:58.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.225+0000 7f801ded5640 1 -- 172.21.15.172:0/1492750155 shutdown_connections 2024-04-14T08:32:58.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.225+0000 7f801ded5640 1 -- 172.21.15.172:0/1492750155 wait complete. 2024-04-14T08:32:58.256 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-14T08:32:58.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.360+0000 7f73b0315640 1 Processor -- start 2024-04-14T08:32:58.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.360+0000 7f73b0315640 1 -- start start 2024-04-14T08:32:58.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.361+0000 7f73b0315640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f73a80d1a20 0x7f73a806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:58.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.361+0000 7f73b0315640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f73a806d000 0x7f73a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:58.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.361+0000 7f73b0315640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73a80701b0 con 0x7f73a8071250 2024-04-14T08:32:58.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.361+0000 7f73b0315640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f73a8070dc0 con 0x7f73a806d000 2024-04-14T08:32:58.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.361+0000 7f73b0315640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f73a806d990 con 0x7f73a80d1a20 2024-04-14T08:32:58.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.361+0000 7f73ae8b2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f73a806d000 0x7f73a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:58.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.361+0000 7f73ae8b2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f73a806d000 0x7f73a806d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58206/0 (socket says 172.21.15.172:58206) 2024-04-14T08:32:58.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.361+0000 7f73ae8b2640 1 -- 172.21.15.172:0/4013435783 learned_addr learned my addr 172.21.15.172:0/4013435783 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:58.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.361+0000 7f73ae0b1640 1 --2- 172.21.15.172:0/4013435783 >> v2:172.21.15.177:3300/0 conn(0x7f73a80d1a20 0x7f73a806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:58.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.362+0000 7f73ae8b2640 1 -- 172.21.15.172:0/4013435783 >> v2:172.21.15.177:3300/0 conn(0x7f73a80d1a20 msgr2=0x7f73a806fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:58.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.362+0000 7f73ae8b2640 1 --2- 172.21.15.172:0/4013435783 >> v2:172.21.15.177:3300/0 conn(0x7f73a80d1a20 0x7f73a806fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:58.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.362+0000 7f73ae8b2640 1 -- 172.21.15.172:0/4013435783 >> v1:172.21.15.177:6789/0 conn(0x7f73a8071250 legacy=0x7f73a80cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:58.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.362+0000 7f73ae8b2640 1 -- 172.21.15.172:0/4013435783 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a8070490 con 0x7f73a806d000 2024-04-14T08:32:58.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.362+0000 7f73ae8b2640 1 --2- 172.21.15.172:0/4013435783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73a806d000 0x7f73a806d450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f73a80740b0 tx=0x7f73a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04af38bd04dcc7e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:58.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.363+0000 7f73ad0af640 1 -- 172.21.15.172:0/4013435783 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0035020 con 0x7f73a806d000 2024-04-14T08:32:58.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.363+0000 7f73ad0af640 1 -- 172.21.15.172:0/4013435783 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a0031870 con 0x7f73a806d000 2024-04-14T08:32:58.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.363+0000 7f73ad0af640 1 -- 172.21.15.172:0/4013435783 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0031c40 con 0x7f73a806d000 2024-04-14T08:32:58.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.363+0000 7f73b0315640 1 -- 172.21.15.172:0/4013435783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73a806d000 msgr2=0x7f73a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:58.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.363+0000 7f73b0315640 1 --2- 172.21.15.172:0/4013435783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73a806d000 0x7f73a806d450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f73a80740b0 tx=0x7f73a002b910 comp rx=0 tx=0).stop 2024-04-14T08:32:58.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.363+0000 7f73b0315640 1 -- 172.21.15.172:0/4013435783 shutdown_connections 2024-04-14T08:32:58.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.363+0000 7f73b0315640 1 --2- 172.21.15.172:0/4013435783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73a806d000 0x7f73a806d450 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:58.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.363+0000 7f73b0315640 1 --2- 172.21.15.172:0/4013435783 >> v2:172.21.15.177:3300/0 conn(0x7f73a80d1a20 0x7f73a806fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:58.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.363+0000 7f73b0315640 1 -- 172.21.15.172:0/4013435783 >> 172.21.15.172:0/4013435783 conn(0x7f73a808c000 msgr2=0x7f73a80d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:58.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.363+0000 7f73b0315640 1 -- 172.21.15.172:0/4013435783 shutdown_connections 2024-04-14T08:32:58.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.363+0000 7f73b0315640 1 -- 172.21.15.172:0/4013435783 wait complete. 2024-04-14T08:32:58.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.364+0000 7f73b0315640 1 Processor -- start 2024-04-14T08:32:58.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.364+0000 7f73b0315640 1 -- start start 2024-04-14T08:32:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73b0315640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73a806d000 0x7f73a80b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73b0315640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73a8071250 0x7f73a80b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73b0315640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73a80d1a20 0x7f73a80b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73b0315640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f73a80d1e00 con 0x7f73a806d000 2024-04-14T08:32:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73b0315640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8070a50 con 0x7f73a8071250 2024-04-14T08:32:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73b0315640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f73a81589d0 con 0x7f73a80d1a20 2024-04-14T08:32:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73ae0b1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73a806d000 0x7f73a80b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73ae0b1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73a806d000 0x7f73a80b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58214/0 (socket says 172.21.15.172:58214) 2024-04-14T08:32:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73ae0b1640 1 -- 172.21.15.172:0/1789080031 learned_addr learned my addr 172.21.15.172:0/1789080031 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73ad8b0640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73a8071250 0x7f73a80b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73ae8b2640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73a80d1a20 0x7f73a80b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73ae0b1640 1 -- 172.21.15.172:0/1789080031 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73a80d1a20 msgr2=0x7f73a80b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73ae0b1640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73a80d1a20 0x7f73a80b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73ae0b1640 1 -- 172.21.15.172:0/1789080031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73a8071250 msgr2=0x7f73a80b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73ae0b1640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73a8071250 0x7f73a80b7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.365+0000 7f73ae0b1640 1 -- 172.21.15.172:0/1789080031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a80cb1f0 con 0x7f73a806d000 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.366+0000 7f73ae0b1640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73a806d000 0x7f73a80b0940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009d10 tx=0x7f73a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.366+0000 7f739effd640 1 -- 172.21.15.172:0/1789080031 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4018020 con 0x7f73a806d000 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.366+0000 7f739effd640 1 -- 172.21.15.172:0/1789080031 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a4014480 con 0x7f73a806d000 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.366+0000 7f739effd640 1 -- 172.21.15.172:0/1789080031 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4014830 con 0x7f73a806d000 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.366+0000 7f73b0315640 1 -- 172.21.15.172:0/1789080031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a8156400 con 0x7f73a806d000 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.366+0000 7f73b0315640 1 -- 172.21.15.172:0/1789080031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a8156ca0 con 0x7f73a806d000 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.367+0000 7f73b0315640 1 -- 172.21.15.172:0/1789080031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a8075490 con 0x7f73a806d000 2024-04-14T08:32:58.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.367+0000 7f739effd640 1 -- 172.21.15.172:0/1789080031 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73a4005d10 con 0x7f73a806d000 2024-04-14T08:32:58.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.368+0000 7f739effd640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f738c06c8f0 0x7f738c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:58.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.368+0000 7f739effd640 1 -- 172.21.15.172:0/1789080031 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f73a408c6f0 con 0x7f73a806d000 2024-04-14T08:32:58.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.368+0000 7f73ad8b0640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f738c06c8f0 0x7f738c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:58.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.368+0000 7f73ad8b0640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f738c06c8f0 0x7f738c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73a8161b10 tx=0x7f7398009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:58.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.372+0000 7f739effd640 1 -- 172.21.15.172:0/1789080031 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f73a4022070 con 0x7f73a806d000 2024-04-14T08:32:58.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:58.585+0000 7f73b0315640 1 -- 172.21.15.172:0/1789080031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f73a80b7f90 con 0x7f73a806d000 2024-04-14T08:32:59.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.226+0000 7f739effd640 1 -- 172.21.15.172:0/1789080031 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f73a4010040 con 0x7f73a806d000 2024-04-14T08:32:59.227 INFO:teuthology.orchestra.run.smithi172.stderr:added data pool 4 to fsmap 2024-04-14T08:32:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.228+0000 7f73b0315640 1 -- 172.21.15.172:0/1789080031 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f738c06c8f0 msgr2=0x7f738c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.228+0000 7f73b0315640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f738c06c8f0 0x7f738c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73a8161b10 tx=0x7f7398009040 comp rx=0 tx=0).stop 2024-04-14T08:32:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.228+0000 7f73b0315640 1 -- 172.21.15.172:0/1789080031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73a806d000 msgr2=0x7f73a80b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.228+0000 7f73b0315640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73a806d000 0x7f73a80b0940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009d10 tx=0x7f73a400c490 comp rx=0 tx=0).stop 2024-04-14T08:32:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.228+0000 7f73b0315640 1 -- 172.21.15.172:0/1789080031 shutdown_connections 2024-04-14T08:32:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.228+0000 7f73b0315640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f738c06c8f0 0x7f738c06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.228+0000 7f73b0315640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f73a80d1a20 0x7f73a80b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.228+0000 7f73b0315640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73a8071250 0x7f73a80b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.228+0000 7f73b0315640 1 --2- 172.21.15.172:0/1789080031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73a806d000 0x7f73a80b0940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.228+0000 7f73b0315640 1 -- 172.21.15.172:0/1789080031 >> 172.21.15.172:0/1789080031 conn(0x7f73a808c000 msgr2=0x7f73a80d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:59.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.228+0000 7f73b0315640 1 -- 172.21.15.172:0/1789080031 shutdown_connections 2024-04-14T08:32:59.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.228+0000 7f73b0315640 1 -- 172.21.15.172:0/1789080031 wait complete. 2024-04-14T08:32:59.240 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:32:59.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.341+0000 7f9df6ddc640 1 Processor -- start 2024-04-14T08:32:59.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.341+0000 7f9df6ddc640 1 -- start start 2024-04-14T08:32:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9df6ddc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9df006fdf0 0x7f9df00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9df6ddc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9df015e890 0x7f9df015ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9df6ddc640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9df0070270 con 0x7f9df006b520 2024-04-14T08:32:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9df6ddc640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f9df00b4310 con 0x7f9df006fdf0 2024-04-14T08:32:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9df6ddc640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f9df0072d80 con 0x7f9df015e890 2024-04-14T08:32:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9deffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9df006fdf0 0x7f9df00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9df4b78640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9df006b520 0x7f9df006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34864/0 (socket says 172.21.15.172:34864) 2024-04-14T08:32:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9df4b78640 1 -- 172.21.15.172:0/3349538211 learned_addr learned my addr 172.21.15.172:0/3349538211 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:32:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9df5379640 1 --2- 172.21.15.172:0/3349538211 >> v2:172.21.15.177:3300/0 conn(0x7f9df015e890 0x7f9df015ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9deffff640 1 -- 172.21.15.172:0/3349538211 >> v2:172.21.15.177:3300/0 conn(0x7f9df015e890 msgr2=0x7f9df015ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:59.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9deffff640 1 --2- 172.21.15.172:0/3349538211 >> v2:172.21.15.177:3300/0 conn(0x7f9df015e890 0x7f9df015ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9deffff640 1 -- 172.21.15.172:0/3349538211 >> v1:172.21.15.172:6789/0 conn(0x7f9df006b520 legacy=0x7f9df006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9deffff640 1 -- 172.21.15.172:0/3349538211 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df00b4540 con 0x7f9df006fdf0 2024-04-14T08:32:59.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.342+0000 7f9def7fe640 1 -- 172.21.15.172:0/3349538211 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934918612 0 0) 0x7f9df0070270 con 0x7f9df006b520 2024-04-14T08:32:59.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.343+0000 7f9deffff640 1 --2- 172.21.15.172:0/3349538211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9df006fdf0 0x7f9df00b3dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9df006c0e0 tx=0x7f9ddc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc5065283336de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:59.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.343+0000 7f9def7fe640 1 -- 172.21.15.172:0/3349538211 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc035020 con 0x7f9df006fdf0 2024-04-14T08:32:59.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.343+0000 7f9def7fe640 1 -- 172.21.15.172:0/3349538211 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ddc031830 con 0x7f9df006fdf0 2024-04-14T08:32:59.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.343+0000 7f9def7fe640 1 -- 172.21.15.172:0/3349538211 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc031c00 con 0x7f9df006fdf0 2024-04-14T08:32:59.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.344+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3349538211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9df006fdf0 msgr2=0x7f9df00b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.344+0000 7f9df6ddc640 1 --2- 172.21.15.172:0/3349538211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9df006fdf0 0x7f9df00b3dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9df006c0e0 tx=0x7f9ddc02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.344+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3349538211 shutdown_connections 2024-04-14T08:32:59.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.344+0000 7f9df6ddc640 1 --2- 172.21.15.172:0/3349538211 >> v2:172.21.15.177:3300/0 conn(0x7f9df015e890 0x7f9df015ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.344+0000 7f9df6ddc640 1 --2- 172.21.15.172:0/3349538211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9df006fdf0 0x7f9df00b3dd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.344+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3349538211 >> 172.21.15.172:0/3349538211 conn(0x7f9df00d0f20 msgr2=0x7f9df00d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:59.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.344+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3349538211 shutdown_connections 2024-04-14T08:32:59.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.344+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3349538211 wait complete. 2024-04-14T08:32:59.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.345+0000 7f9df6ddc640 1 Processor -- start 2024-04-14T08:32:59.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.345+0000 7f9df6ddc640 1 -- start start 2024-04-14T08:32:59.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df6ddc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9df006b520 0x7f9df013f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df6ddc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9df006fdf0 0x7f9df01435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df6ddc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9df015e890 0x7f9df0161600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df6ddc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9df0072a10 con 0x7f9df006b520 2024-04-14T08:32:59.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df6ddc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9df0072730 con 0x7f9df015e890 2024-04-14T08:32:59.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df6ddc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9df0073210 con 0x7f9df006fdf0 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df4b78640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9df006b520 0x7f9df013f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df4b78640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9df006b520 0x7f9df013f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58232/0 (socket says 172.21.15.172:58232) 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df4b78640 1 -- 172.21.15.172:0/3995209122 learned_addr learned my addr 172.21.15.172:0/3995209122 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9deffff640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9df006fdf0 0x7f9df01435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df5379640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9df015e890 0x7f9df0161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df4b78640 1 -- 172.21.15.172:0/3995209122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9df006fdf0 msgr2=0x7f9df01435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df4b78640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9df006fdf0 0x7f9df01435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df4b78640 1 -- 172.21.15.172:0/3995209122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9df015e890 msgr2=0x7f9df0161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df4b78640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9df015e890 0x7f9df0161600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df4b78640 1 -- 172.21.15.172:0/3995209122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df015f920 con 0x7f9df006b520 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df4b78640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9df006b520 0x7f9df013f970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c460 tx=0x7f9de400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9ded7fa640 1 -- 172.21.15.172:0/3995209122 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4018070 con 0x7f9df006b520 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9ded7fa640 1 -- 172.21.15.172:0/3995209122 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de4007500 con 0x7f9df006b520 2024-04-14T08:32:59.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3995209122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df015fc00 con 0x7f9df006b520 2024-04-14T08:32:59.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9ded7fa640 1 -- 172.21.15.172:0/3995209122 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4004a20 con 0x7f9df006b520 2024-04-14T08:32:59.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.346+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3995209122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df01601c0 con 0x7f9df006b520 2024-04-14T08:32:59.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.347+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3995209122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df0068520 con 0x7f9df006b520 2024-04-14T08:32:59.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.348+0000 7f9ded7fa640 1 -- 172.21.15.172:0/3995209122 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9de4004bc0 con 0x7f9df006b520 2024-04-14T08:32:59.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.348+0000 7f9ded7fa640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9dd406c8f0 0x7f9dd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.348+0000 7f9deffff640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9dd406c8f0 0x7f9dd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.348+0000 7f9ded7fa640 1 -- 172.21.15.172:0/3995209122 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9de408b140 con 0x7f9df006b520 2024-04-14T08:32:59.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.349+0000 7f9deffff640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9dd406c8f0 0x7f9dd406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc036000 tx=0x7f9ddc03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:59.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.353+0000 7f9ded7fa640 1 -- 172.21.15.172:0/3995209122 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9de4058b60 con 0x7f9df006b520 2024-04-14T08:32:59.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.564+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3995209122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9df006c3b0 con 0x7f9df006b520 2024-04-14T08:32:59.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.564+0000 7f9ded7fa640 1 -- 172.21.15.172:0/3995209122 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5186 (secure 0 0 0) 0x7f9de405ce60 con 0x7f9df006b520 2024-04-14T08:32:59.565 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:32:59.565 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3053874257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3053874257},{"type":"v1","addr":"172.21.15.172:6835","nonce":3053874257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2790588849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2790588849},{"type":"v1","addr":"172.21.15.172:6837","nonce":2790588849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3130776573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3130776573},{"type":"v1","addr":"172.21.15.177:6834","nonce":3130776573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:32:53.535483+0000","modified":"2024-04-14T08:32:58.586363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/517963877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":517963877},{"type":"v1","addr":"172.21.15.177:6835","nonce":517963877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4309,"qdb_cluster":[4309]},"id":1}]} 2024-04-14T08:32:59.565 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 7 2024-04-14T08:32:59.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.566+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3995209122 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9dd406c8f0 msgr2=0x7f9dd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.566+0000 7f9df6ddc640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9dd406c8f0 0x7f9dd406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc036000 tx=0x7f9ddc03c040 comp rx=0 tx=0).stop 2024-04-14T08:32:59.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.566+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3995209122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9df006b520 msgr2=0x7f9df013f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.566+0000 7f9df6ddc640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9df006b520 0x7f9df013f970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c460 tx=0x7f9de400c920 comp rx=0 tx=0).stop 2024-04-14T08:32:59.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.566+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3995209122 shutdown_connections 2024-04-14T08:32:59.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.566+0000 7f9df6ddc640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9dd406c8f0 0x7f9dd406edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.566+0000 7f9df6ddc640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9df015e890 0x7f9df0161600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.566+0000 7f9df6ddc640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9df006fdf0 0x7f9df01435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.566+0000 7f9df6ddc640 1 --2- 172.21.15.172:0/3995209122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9df006b520 0x7f9df013f970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.566+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3995209122 >> 172.21.15.172:0/3995209122 conn(0x7f9df00d0f20 msgr2=0x7f9df00b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:59.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.567+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3995209122 shutdown_connections 2024-04-14T08:32:59.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.567+0000 7f9df6ddc640 1 -- 172.21.15.172:0/3995209122 wait complete. 2024-04-14T08:32:59.578 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:32:59.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.677+0000 7fa373555640 1 Processor -- start 2024-04-14T08:32:59.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.677+0000 7fa373555640 1 -- start start 2024-04-14T08:32:59.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa373555640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa36c06fdf0 0x7fa36c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa373555640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36c15e890 0x7fa36c15ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa373555640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa36c070270 con 0x7fa36c06b520 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa373555640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa36c0b4310 con 0x7fa36c06fdf0 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa373555640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa36c072d80 con 0x7fa36c15e890 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa371af2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36c15e890 0x7fa36c15ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa370af0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa36c06fdf0 0x7fa36c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa3712f1640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fa36c06b520 0x7fa36c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:54118/0 (socket says 172.21.15.172:54118) 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa3712f1640 1 -- 172.21.15.172:0/3800275928 learned_addr learned my addr 172.21.15.172:0/3800275928 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa363fff640 1 -- 172.21.15.172:0/3800275928 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1265661690 0 0) 0x7fa36c070270 con 0x7fa36c06b520 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa363fff640 1 -- 172.21.15.172:0/3800275928 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa358003520 con 0x7fa36c06b520 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa371af2640 1 -- 172.21.15.172:0/3800275928 >> v2:172.21.15.177:3300/0 conn(0x7fa36c06fdf0 msgr2=0x7fa36c0b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa371af2640 1 --2- 172.21.15.172:0/3800275928 >> v2:172.21.15.177:3300/0 conn(0x7fa36c06fdf0 0x7fa36c0b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa371af2640 1 -- 172.21.15.172:0/3800275928 >> v1:172.21.15.177:6789/0 conn(0x7fa36c06b520 legacy=0x7fa36c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa371af2640 1 -- 172.21.15.172:0/3800275928 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36c0b4540 con 0x7fa36c15e890 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.678+0000 7fa370af0640 1 --2- 172.21.15.172:0/3800275928 >> v2:172.21.15.177:3300/0 conn(0x7fa36c06fdf0 0x7fa36c0b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:59.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.679+0000 7fa371af2640 1 --2- 172.21.15.172:0/3800275928 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36c15e890 0x7fa36c15ed20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa364009d90 tx=0x7fa36402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d0235d5042fedfa server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:59.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.679+0000 7fa363fff640 1 -- 172.21.15.172:0/3800275928 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364039070 con 0x7fa36c15e890 2024-04-14T08:32:59.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.679+0000 7fa363fff640 1 -- 172.21.15.172:0/3800275928 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa36402cc50 con 0x7fa36c15e890 2024-04-14T08:32:59.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.679+0000 7fa363fff640 1 -- 172.21.15.172:0/3800275928 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364034440 con 0x7fa36c15e890 2024-04-14T08:32:59.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.680+0000 7fa373555640 1 -- 172.21.15.172:0/3800275928 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36c15e890 msgr2=0x7fa36c15ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.680+0000 7fa373555640 1 --2- 172.21.15.172:0/3800275928 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36c15e890 0x7fa36c15ed20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa364009d90 tx=0x7fa36402bea0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.680+0000 7fa373555640 1 -- 172.21.15.172:0/3800275928 shutdown_connections 2024-04-14T08:32:59.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.680+0000 7fa373555640 1 --2- 172.21.15.172:0/3800275928 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36c15e890 0x7fa36c15ed20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.680+0000 7fa373555640 1 --2- 172.21.15.172:0/3800275928 >> v2:172.21.15.177:3300/0 conn(0x7fa36c06fdf0 0x7fa36c0b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.680+0000 7fa373555640 1 -- 172.21.15.172:0/3800275928 >> 172.21.15.172:0/3800275928 conn(0x7fa36c0d0f20 msgr2=0x7fa36c0d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.680+0000 7fa373555640 1 -- 172.21.15.172:0/3800275928 shutdown_connections 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.680+0000 7fa373555640 1 -- 172.21.15.172:0/3800275928 wait complete. 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.681+0000 7fa373555640 1 Processor -- start 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.681+0000 7fa373555640 1 -- start start 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa373555640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa36c06b520 0x7fa36c155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa373555640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36c06fdf0 0x7fa36c15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa373555640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa36c15e890 0x7fa36c15b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa373555640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c072a10 con 0x7fa36c06b520 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa373555640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c072730 con 0x7fa36c15e890 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa373555640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa36c073210 con 0x7fa36c06fdf0 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa3712f1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa36c06b520 0x7fa36c155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa3712f1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa36c06b520 0x7fa36c155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38790/0 (socket says 172.21.15.172:38790) 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa370af0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36c06fdf0 0x7fa36c15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa3712f1640 1 -- 172.21.15.172:0/3735876228 learned_addr learned my addr 172.21.15.172:0/3735876228 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:59.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa371af2640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa36c15e890 0x7fa36c15b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa3712f1640 1 -- 172.21.15.172:0/3735876228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36c06fdf0 msgr2=0x7fa36c15a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa3712f1640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36c06fdf0 0x7fa36c15a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa3712f1640 1 -- 172.21.15.172:0/3735876228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa36c15e890 msgr2=0x7fa36c15b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa3712f1640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa36c15e890 0x7fa36c15b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa3712f1640 1 -- 172.21.15.172:0/3735876228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36c155e40 con 0x7fa36c06b520 2024-04-14T08:32:59.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa3712f1640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa36c06b520 0x7fa36c155900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa36800c390 tx=0x7fa36800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:59.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa361ffb640 1 -- 172.21.15.172:0/3735876228 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368017070 con 0x7fa36c06b520 2024-04-14T08:32:59.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa361ffb640 1 -- 172.21.15.172:0/3735876228 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa368005ce0 con 0x7fa36c06b520 2024-04-14T08:32:59.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa373555640 1 -- 172.21.15.172:0/3735876228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36c159150 con 0x7fa36c06b520 2024-04-14T08:32:59.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.682+0000 7fa361ffb640 1 -- 172.21.15.172:0/3735876228 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3680072f0 con 0x7fa36c06b520 2024-04-14T08:32:59.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.683+0000 7fa373555640 1 -- 172.21.15.172:0/3735876228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36c159710 con 0x7fa36c06b520 2024-04-14T08:32:59.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.683+0000 7fa373555640 1 -- 172.21.15.172:0/3735876228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa36c068520 con 0x7fa36c06b520 2024-04-14T08:32:59.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.684+0000 7fa361ffb640 1 -- 172.21.15.172:0/3735876228 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa36801a070 con 0x7fa36c06b520 2024-04-14T08:32:59.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.684+0000 7fa361ffb640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa35006c8f0 0x7fa35006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.684+0000 7fa370af0640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa35006c8f0 0x7fa35006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.684+0000 7fa361ffb640 1 -- 172.21.15.172:0/3735876228 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa36808b280 con 0x7fa36c06b520 2024-04-14T08:32:59.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.684+0000 7fa370af0640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa35006c8f0 0x7fa35006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009560 tx=0x7fa35c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:59.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.688+0000 7fa361ffb640 1 -- 172.21.15.172:0/3735876228 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa368058ca0 con 0x7fa36c06b520 2024-04-14T08:32:59.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.863+0000 7fa373555640 1 -- 172.21.15.172:0/3735876228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa36c06c3b0 con 0x7fa36c06b520 2024-04-14T08:32:59.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.864+0000 7fa361ffb640 1 -- 172.21.15.172:0/3735876228 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7fa36805cfa0 con 0x7fa36c06b520 2024-04-14T08:32:59.864 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:32:59.865 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:32:59.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.866+0000 7fa373555640 1 -- 172.21.15.172:0/3735876228 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa35006c8f0 msgr2=0x7fa35006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.866+0000 7fa373555640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa35006c8f0 0x7fa35006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009560 tx=0x7fa35c009190 comp rx=0 tx=0).stop 2024-04-14T08:32:59.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.866+0000 7fa373555640 1 -- 172.21.15.172:0/3735876228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa36c06b520 msgr2=0x7fa36c155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.866+0000 7fa373555640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa36c06b520 0x7fa36c155900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa36800c390 tx=0x7fa36800c850 comp rx=0 tx=0).stop 2024-04-14T08:32:59.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.866+0000 7fa373555640 1 -- 172.21.15.172:0/3735876228 shutdown_connections 2024-04-14T08:32:59.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.866+0000 7fa373555640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa35006c8f0 0x7fa35006edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.866+0000 7fa373555640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa36c15e890 0x7fa36c15b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.866+0000 7fa373555640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36c06fdf0 0x7fa36c15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.866+0000 7fa373555640 1 --2- 172.21.15.172:0/3735876228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa36c06b520 0x7fa36c155900 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.866+0000 7fa373555640 1 -- 172.21.15.172:0/3735876228 >> 172.21.15.172:0/3735876228 conn(0x7fa36c0d0f20 msgr2=0x7fa36c0b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:59.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.866+0000 7fa373555640 1 -- 172.21.15.172:0/3735876228 shutdown_connections 2024-04-14T08:32:59.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.866+0000 7fa373555640 1 -- 172.21.15.172:0/3735876228 wait complete. 2024-04-14T08:32:59.878 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:32:59.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.977+0000 7fba26c88640 1 Processor -- start 2024-04-14T08:32:59.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.977+0000 7fba26c88640 1 -- start start 2024-04-14T08:32:59.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba26c88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba2009b180 0x7fba2006f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba26c88640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fba200d2ba0 0x7fba2006f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba26c88640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba200d1050 con 0x7fba200ce120 2024-04-14T08:32:59.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba26c88640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fba200d1610 con 0x7fba200d2ba0 2024-04-14T08:32:59.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba26c88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fba200d1330 con 0x7fba2009b180 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba25225640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fba200d2ba0 0x7fba2006f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba25225640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fba200d2ba0 0x7fba2006f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38802/0 (socket says 172.21.15.172:38802) 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba25225640 1 -- 172.21.15.172:0/2150358679 learned_addr learned my addr 172.21.15.172:0/2150358679 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba1ffff640 1 --2- 172.21.15.172:0/2150358679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba2009b180 0x7fba2006f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba25225640 1 -- 172.21.15.172:0/2150358679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba2009b180 msgr2=0x7fba2006f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba25225640 1 --2- 172.21.15.172:0/2150358679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba2009b180 0x7fba2006f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba25225640 1 -- 172.21.15.172:0/2150358679 >> v1:172.21.15.177:6789/0 conn(0x7fba200ce120 legacy=0x7fba2009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba1f7fe640 1 -- 172.21.15.172:0/2150358679 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 104838551 0 0) 0x7fba200d1050 con 0x7fba200ce120 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.978+0000 7fba25225640 1 -- 172.21.15.172:0/2150358679 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2006c7d0 con 0x7fba200d2ba0 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.979+0000 7fba25225640 1 --2- 172.21.15.172:0/2150358679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba200d2ba0 0x7fba2006f8d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba18009e50 tx=0x7fba1802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4abb7681881092b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.979+0000 7fba1f7fe640 1 -- 172.21.15.172:0/2150358679 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1802cb00 con 0x7fba200d2ba0 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.979+0000 7fba1f7fe640 1 -- 172.21.15.172:0/2150358679 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1802cca0 con 0x7fba200d2ba0 2024-04-14T08:32:59.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.979+0000 7fba1f7fe640 1 -- 172.21.15.172:0/2150358679 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18034610 con 0x7fba200d2ba0 2024-04-14T08:32:59.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.979+0000 7fba26c88640 1 -- 172.21.15.172:0/2150358679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba200d2ba0 msgr2=0x7fba2006f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.979+0000 7fba26c88640 1 --2- 172.21.15.172:0/2150358679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba200d2ba0 0x7fba2006f8d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba18009e50 tx=0x7fba1802bf60 comp rx=0 tx=0).stop 2024-04-14T08:32:59.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.979+0000 7fba26c88640 1 -- 172.21.15.172:0/2150358679 shutdown_connections 2024-04-14T08:32:59.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.979+0000 7fba26c88640 1 --2- 172.21.15.172:0/2150358679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba200d2ba0 0x7fba2006f8d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.979+0000 7fba26c88640 1 --2- 172.21.15.172:0/2150358679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba2009b180 0x7fba2006f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.979+0000 7fba26c88640 1 -- 172.21.15.172:0/2150358679 >> 172.21.15.172:0/2150358679 conn(0x7fba200d2080 msgr2=0x7fba200d2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.979+0000 7fba26c88640 1 -- 172.21.15.172:0/2150358679 shutdown_connections 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.979+0000 7fba26c88640 1 -- 172.21.15.172:0/2150358679 wait complete. 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.980+0000 7fba26c88640 1 Processor -- start 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.981+0000 7fba26c88640 1 -- start start 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.981+0000 7fba26c88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba2009b180 0x7fba2009f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.981+0000 7fba26c88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba200ce120 0x7fba2006f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.981+0000 7fba26c88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba200d2ba0 0x7fba2006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.981+0000 7fba26c88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fba200d0ce0 con 0x7fba200d2ba0 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.981+0000 7fba26c88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fba200ce500 con 0x7fba200ce120 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.981+0000 7fba26c88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fba2006dfe0 con 0x7fba2009b180 2024-04-14T08:32:59.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.981+0000 7fba25225640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba200d2ba0 0x7fba2006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba24a24640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba2009b180 0x7fba2009f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba25225640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba200d2ba0 0x7fba2006fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38810/0 (socket says 172.21.15.172:38810) 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba24a24640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba2009b180 0x7fba2009f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:58146/0 (socket says 172.21.15.172:58146) 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba25225640 1 -- 172.21.15.172:0/1697751420 learned_addr learned my addr 172.21.15.172:0/1697751420 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba1ffff640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba200ce120 0x7fba2006f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba24a24640 1 -- 172.21.15.172:0/1697751420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba200ce120 msgr2=0x7fba2006f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba24a24640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba200ce120 0x7fba2006f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba24a24640 1 -- 172.21.15.172:0/1697751420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba200d2ba0 msgr2=0x7fba2006fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba24a24640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba200d2ba0 0x7fba2006fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba24a24640 1 -- 172.21.15.172:0/1697751420 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba200c68e0 con 0x7fba2009b180 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba25225640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba200d2ba0 0x7fba2006fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba24a24640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba2009b180 0x7fba2009f890 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fba1400c450 tx=0x7fba1400c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba1d7fa640 1 -- 172.21.15.172:0/1697751420 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14017070 con 0x7fba2009b180 2024-04-14T08:32:59.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba1d7fa640 1 -- 172.21.15.172:0/1697751420 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba140035f0 con 0x7fba2009b180 2024-04-14T08:32:59.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba26c88640 1 -- 172.21.15.172:0/1697751420 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba200c6bc0 con 0x7fba2009b180 2024-04-14T08:32:59.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba1d7fa640 1 -- 172.21.15.172:0/1697751420 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba140039e0 con 0x7fba2009b180 2024-04-14T08:32:59.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.982+0000 7fba26c88640 1 -- 172.21.15.172:0/1697751420 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2009fdd0 con 0x7fba2009b180 2024-04-14T08:32:59.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.983+0000 7fba26c88640 1 -- 172.21.15.172:0/1697751420 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba20067dd0 con 0x7fba2009b180 2024-04-14T08:32:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.984+0000 7fba1d7fa640 1 -- 172.21.15.172:0/1697751420 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba1401a070 con 0x7fba2009b180 2024-04-14T08:32:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.984+0000 7fba1d7fa640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fba0406c8f0 0x7fba0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:32:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.984+0000 7fba1ffff640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fba0406c8f0 0x7fba0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:32:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.984+0000 7fba1d7fa640 1 -- 172.21.15.172:0/1697751420 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fba1408b190 con 0x7fba2009b180 2024-04-14T08:32:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.984+0000 7fba1ffff640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fba0406c8f0 0x7fba0406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fba201f74b0 tx=0x7fba0c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:32:59.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:32:59.989+0000 7fba1d7fa640 1 -- 172.21.15.172:0/1697751420 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fba20067dd0 con 0x7fba2009b180 2024-04-14T08:33:00.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.163+0000 7fba26c88640 1 -- 172.21.15.172:0/1697751420 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba20068520 con 0x7fba2009b180 2024-04-14T08:33:00.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.165+0000 7fba1d7fa640 1 -- 172.21.15.172:0/1697751420 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7fba14058ca0 con 0x7fba2009b180 2024-04-14T08:33:00.165 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:00.165 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:00.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.167+0000 7fba26c88640 1 -- 172.21.15.172:0/1697751420 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fba0406c8f0 msgr2=0x7fba0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.167+0000 7fba26c88640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fba0406c8f0 0x7fba0406edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fba201f74b0 tx=0x7fba0c009190 comp rx=0 tx=0).stop 2024-04-14T08:33:00.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.167+0000 7fba26c88640 1 -- 172.21.15.172:0/1697751420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba2009b180 msgr2=0x7fba2009f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.167+0000 7fba26c88640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba2009b180 0x7fba2009f890 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fba1400c450 tx=0x7fba1400c910 comp rx=0 tx=0).stop 2024-04-14T08:33:00.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.167+0000 7fba26c88640 1 -- 172.21.15.172:0/1697751420 shutdown_connections 2024-04-14T08:33:00.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.167+0000 7fba26c88640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fba0406c8f0 0x7fba0406edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.167+0000 7fba26c88640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba200d2ba0 0x7fba2006fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.167+0000 7fba26c88640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba200ce120 0x7fba2006f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.167+0000 7fba26c88640 1 --2- 172.21.15.172:0/1697751420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fba2009b180 0x7fba2009f890 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.167+0000 7fba26c88640 1 -- 172.21.15.172:0/1697751420 >> 172.21.15.172:0/1697751420 conn(0x7fba200d2080 msgr2=0x7fba20161420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:00.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.167+0000 7fba26c88640 1 -- 172.21.15.172:0/1697751420 shutdown_connections 2024-04-14T08:33:00.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.167+0000 7fba26c88640 1 -- 172.21.15.172:0/1697751420 wait complete. 2024-04-14T08:33:00.193 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:33:00.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.293+0000 7f2754b1e640 1 Processor -- start 2024-04-14T08:33:00.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.293+0000 7f2754b1e640 1 -- start start 2024-04-14T08:33:00.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.294+0000 7f2754b1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f27500d1a10 0x7f2750161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:00.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.294+0000 7f2754b1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2750162530 0x7f27500d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:00.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.294+0000 7f2754b1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27500d0200 con 0x7f275015c150 2024-04-14T08:33:00.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.294+0000 7f2754b1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2750162980 con 0x7f27500d1a10 2024-04-14T08:33:00.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.294+0000 7f2754b1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f27500cef10 con 0x7f2750162530 2024-04-14T08:33:00.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.294+0000 7f274dd9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f27500d1a10 0x7f2750161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:00.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.294+0000 7f274dd9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f27500d1a10 0x7f2750161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38820/0 (socket says 172.21.15.172:38820) 2024-04-14T08:33:00.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.294+0000 7f274dd9b640 1 -- 172.21.15.172:0/397421336 learned_addr learned my addr 172.21.15.172:0/397421336 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:00.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.294+0000 7f274ed9d640 1 --2- 172.21.15.172:0/397421336 >> v2:172.21.15.177:3300/0 conn(0x7f2750162530 0x7f27500d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:00.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.295+0000 7f274dd9b640 1 -- 172.21.15.172:0/397421336 >> v2:172.21.15.177:3300/0 conn(0x7f2750162530 msgr2=0x7f27500d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:00.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.295+0000 7f274dd9b640 1 --2- 172.21.15.172:0/397421336 >> v2:172.21.15.177:3300/0 conn(0x7f2750162530 0x7f27500d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.295+0000 7f274dd9b640 1 -- 172.21.15.172:0/397421336 >> v1:172.21.15.177:6789/0 conn(0x7f275015c150 legacy=0x7f27500d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.295+0000 7f274dd9b640 1 -- 172.21.15.172:0/397421336 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27500cf160 con 0x7f27500d1a10 2024-04-14T08:33:00.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.295+0000 7f274dd9b640 1 --2- 172.21.15.172:0/397421336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27500d1a10 0x7f2750161ff0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f27500c7ad0 tx=0x7f274402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b339e6536f9c13ab server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:00.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.296+0000 7f274d59a640 1 -- 172.21.15.172:0/397421336 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744034070 con 0x7f27500d1a10 2024-04-14T08:33:00.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.296+0000 7f274d59a640 1 -- 172.21.15.172:0/397421336 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2744030860 con 0x7f27500d1a10 2024-04-14T08:33:00.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.296+0000 7f274d59a640 1 -- 172.21.15.172:0/397421336 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744030c30 con 0x7f27500d1a10 2024-04-14T08:33:00.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.296+0000 7f2754b1e640 1 -- 172.21.15.172:0/397421336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27500d1a10 msgr2=0x7f2750161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.296+0000 7f2754b1e640 1 --2- 172.21.15.172:0/397421336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27500d1a10 0x7f2750161ff0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f27500c7ad0 tx=0x7f274402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.296+0000 7f2754b1e640 1 -- 172.21.15.172:0/397421336 shutdown_connections 2024-04-14T08:33:00.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.296+0000 7f2754b1e640 1 --2- 172.21.15.172:0/397421336 >> v2:172.21.15.177:3300/0 conn(0x7f2750162530 0x7f27500d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.296+0000 7f2754b1e640 1 --2- 172.21.15.172:0/397421336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27500d1a10 0x7f2750161ff0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.296+0000 7f2754b1e640 1 -- 172.21.15.172:0/397421336 >> 172.21.15.172:0/397421336 conn(0x7f275009d580 msgr2=0x7f27500d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:00.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.296+0000 7f2754b1e640 1 -- 172.21.15.172:0/397421336 shutdown_connections 2024-04-14T08:33:00.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.296+0000 7f2754b1e640 1 -- 172.21.15.172:0/397421336 wait complete. 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.297+0000 7f2754b1e640 1 Processor -- start 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.298+0000 7f2754b1e640 1 -- start start 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.298+0000 7f2754b1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27500d1a10 0x7f27500b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.298+0000 7f2754b1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275015c150 0x7f275014a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.298+0000 7f2754b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2750162530 0x7f275014ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.298+0000 7f2754b1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27500cfa00 con 0x7f275015c150 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.298+0000 7f2754b1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f27500cf720 con 0x7f27500d1a10 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.298+0000 7f2754b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f27500cfce0 con 0x7f2750162530 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.298+0000 7f274dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275015c150 0x7f275014a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.298+0000 7f274dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275015c150 0x7f275014a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38828/0 (socket says 172.21.15.172:38828) 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.298+0000 7f274dd9b640 1 -- 172.21.15.172:0/1300381428 learned_addr learned my addr 172.21.15.172:0/1300381428 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.298+0000 7f274ed9d640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2750162530 0x7f275014ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.298+0000 7f274e59c640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27500d1a10 0x7f27500b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:00.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.299+0000 7f274dd9b640 1 -- 172.21.15.172:0/1300381428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2750162530 msgr2=0x7f275014ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.299+0000 7f274dd9b640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2750162530 0x7f275014ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.299+0000 7f274dd9b640 1 -- 172.21.15.172:0/1300381428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27500d1a10 msgr2=0x7f27500b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.299+0000 7f274dd9b640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27500d1a10 0x7f27500b61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.299+0000 7f274dd9b640 1 -- 172.21.15.172:0/1300381428 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275015c910 con 0x7f275015c150 2024-04-14T08:33:00.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.299+0000 7f274dd9b640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275015c150 0x7f275014a5f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2744000c00 tx=0x7f2744004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:00.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.299+0000 7f273effd640 1 -- 172.21.15.172:0/1300381428 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744048070 con 0x7f275015c150 2024-04-14T08:33:00.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.299+0000 7f273effd640 1 -- 172.21.15.172:0/1300381428 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2744030660 con 0x7f275015c150 2024-04-14T08:33:00.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.299+0000 7f2754b1e640 1 -- 172.21.15.172:0/1300381428 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275015cbf0 con 0x7f275015c150 2024-04-14T08:33:00.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.299+0000 7f273effd640 1 -- 172.21.15.172:0/1300381428 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744034040 con 0x7f275015c150 2024-04-14T08:33:00.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.299+0000 7f2754b1e640 1 -- 172.21.15.172:0/1300381428 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27500b59f0 con 0x7f275015c150 2024-04-14T08:33:00.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.300+0000 7f2754b1e640 1 -- 172.21.15.172:0/1300381428 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2750075490 con 0x7f275015c150 2024-04-14T08:33:00.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.300+0000 7f273effd640 1 -- 172.21.15.172:0/1300381428 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f274404b070 con 0x7f275015c150 2024-04-14T08:33:00.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.301+0000 7f273effd640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f273006c8f0 0x7f273006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:00.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.301+0000 7f273effd640 1 -- 172.21.15.172:0/1300381428 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f27440adec0 con 0x7f275015c150 2024-04-14T08:33:00.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.301+0000 7f274e59c640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f273006c8f0 0x7f273006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:00.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.301+0000 7f274e59c640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f273006c8f0 0x7f273006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2740001f10 tx=0x7f2740009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:00.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.305+0000 7f273effd640 1 -- 172.21.15.172:0/1300381428 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f274407b8d0 con 0x7f275015c150 2024-04-14T08:33:00.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.514+0000 7f2754b1e640 1 -- 172.21.15.172:0/1300381428 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f275015bcc0 con 0x7f275015c150 2024-04-14T08:33:00.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.514+0000 7f273effd640 1 -- 172.21.15.172:0/1300381428 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5186 (secure 0 0 0) 0x7f274407fbd0 con 0x7f275015c150 2024-04-14T08:33:00.515 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:00.515 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3053874257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3053874257},{"type":"v1","addr":"172.21.15.172:6835","nonce":3053874257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2790588849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2790588849},{"type":"v1","addr":"172.21.15.172:6837","nonce":2790588849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3130776573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3130776573},{"type":"v1","addr":"172.21.15.177:6834","nonce":3130776573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:32:53.535483+0000","modified":"2024-04-14T08:32:58.586363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/517963877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":517963877},{"type":"v1","addr":"172.21.15.177:6835","nonce":517963877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4309,"qdb_cluster":[4309]},"id":1}]} 2024-04-14T08:33:00.516 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 7 2024-04-14T08:33:00.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.516+0000 7f2754b1e640 1 -- 172.21.15.172:0/1300381428 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f273006c8f0 msgr2=0x7f273006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.516+0000 7f2754b1e640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f273006c8f0 0x7f273006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2740001f10 tx=0x7f2740009040 comp rx=0 tx=0).stop 2024-04-14T08:33:00.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.516+0000 7f2754b1e640 1 -- 172.21.15.172:0/1300381428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275015c150 msgr2=0x7f275014a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.516+0000 7f2754b1e640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275015c150 0x7f275014a5f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2744000c00 tx=0x7f2744004730 comp rx=0 tx=0).stop 2024-04-14T08:33:00.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.516+0000 7f2754b1e640 1 -- 172.21.15.172:0/1300381428 shutdown_connections 2024-04-14T08:33:00.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.516+0000 7f2754b1e640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f273006c8f0 0x7f273006edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.516+0000 7f2754b1e640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2750162530 0x7f275014ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.516+0000 7f2754b1e640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f275015c150 0x7f275014a5f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.516+0000 7f2754b1e640 1 --2- 172.21.15.172:0/1300381428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27500d1a10 0x7f27500b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.516+0000 7f2754b1e640 1 -- 172.21.15.172:0/1300381428 >> 172.21.15.172:0/1300381428 conn(0x7f275009d580 msgr2=0x7f27500b8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:00.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.516+0000 7f2754b1e640 1 -- 172.21.15.172:0/1300381428 shutdown_connections 2024-04-14T08:33:00.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.517+0000 7f2754b1e640 1 -- 172.21.15.172:0/1300381428 wait complete. 2024-04-14T08:33:00.528 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:00.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.629+0000 7fcca78de640 1 Processor -- start 2024-04-14T08:33:00.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.629+0000 7fcca78de640 1 -- start start 2024-04-14T08:33:00.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcca78de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca015e790 0x7fcca00d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:00.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcca78de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcca00d17a0 con 0x7fcca00b2830 2024-04-14T08:33:00.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcca78de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcca015eb70 con 0x7fcca00d2680 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcca78de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcca00d02a0 con 0x7fcca015e790 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcca5e7b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fcca00b2830 0x7fcca00d1090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40214/0 (socket says 172.21.15.172:40214) 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcca5e7b640 1 -- 172.21.15.172:0/1075858111 learned_addr learned my addr 172.21.15.172:0/1075858111 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcca567a640 1 --2- 172.21.15.172:0/1075858111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca015e790 0x7fcca00d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcc97fff640 1 -- 172.21.15.172:0/1075858111 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 823373018 0 0) 0x7fcca00d17a0 con 0x7fcca00b2830 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcc97fff640 1 -- 172.21.15.172:0/1075858111 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc8c003520 con 0x7fcca00b2830 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcc97fff640 1 -- 172.21.15.172:0/1075858111 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3028606376 0 0) 0x7fcca015eb70 con 0x7fcca00d2680 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcc97fff640 1 -- 172.21.15.172:0/1075858111 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcca00d17a0 con 0x7fcca00d2680 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcca567a640 1 -- 172.21.15.172:0/1075858111 >> v1:172.21.15.177:6789/0 conn(0x7fcca00d2680 legacy=0x7fcca00d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcca567a640 1 -- 172.21.15.172:0/1075858111 >> v1:172.21.15.172:6789/0 conn(0x7fcca00b2830 legacy=0x7fcca00d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.630+0000 7fcca567a640 1 -- 172.21.15.172:0/1075858111 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca00cfce0 con 0x7fcca015e790 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.631+0000 7fcca567a640 1 --2- 172.21.15.172:0/1075858111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca015e790 0x7fcca00d4b70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0097d0 tx=0x7fcc9c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0503366017fd61f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.631+0000 7fcc97fff640 1 -- 172.21.15.172:0/1075858111 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c035020 con 0x7fcca015e790 2024-04-14T08:33:00.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.631+0000 7fcc97fff640 1 -- 172.21.15.172:0/1075858111 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c031890 con 0x7fcca015e790 2024-04-14T08:33:00.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.631+0000 7fcc97fff640 1 -- 172.21.15.172:0/1075858111 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c031c60 con 0x7fcca015e790 2024-04-14T08:33:00.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.632+0000 7fcca78de640 1 -- 172.21.15.172:0/1075858111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca015e790 msgr2=0x7fcca00d4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.632+0000 7fcca78de640 1 --2- 172.21.15.172:0/1075858111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca015e790 0x7fcca00d4b70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0097d0 tx=0x7fcc9c02b930 comp rx=0 tx=0).stop 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.632+0000 7fcca78de640 1 -- 172.21.15.172:0/1075858111 shutdown_connections 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.632+0000 7fcca78de640 1 --2- 172.21.15.172:0/1075858111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca015e790 0x7fcca00d4b70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.632+0000 7fcca78de640 1 -- 172.21.15.172:0/1075858111 >> 172.21.15.172:0/1075858111 conn(0x7fcca009d590 msgr2=0x7fcca00d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.632+0000 7fcca78de640 1 -- 172.21.15.172:0/1075858111 shutdown_connections 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.632+0000 7fcca78de640 1 -- 172.21.15.172:0/1075858111 wait complete. 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.633+0000 7fcca78de640 1 Processor -- start 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.633+0000 7fcca78de640 1 -- start start 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca78de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcca00b2830 0x7fcca00cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca78de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca00d2680 0x7fcca00ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca78de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca015e790 0x7fcca00cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca78de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcca00b1020 con 0x7fcca00b2830 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca78de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcca00b1390 con 0x7fcca015e790 2024-04-14T08:33:00.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca78de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcca00d50b0 con 0x7fcca00d2680 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca567a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcca00b2830 0x7fcca00cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca567a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcca00b2830 0x7fcca00cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38842/0 (socket says 172.21.15.172:38842) 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca567a640 1 -- 172.21.15.172:0/2238196793 learned_addr learned my addr 172.21.15.172:0/2238196793 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca5e7b640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca015e790 0x7fcca00cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca4e79640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca00d2680 0x7fcca00ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca567a640 1 -- 172.21.15.172:0/2238196793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca00d2680 msgr2=0x7fcca00ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca567a640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca00d2680 0x7fcca00ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca567a640 1 -- 172.21.15.172:0/2238196793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca015e790 msgr2=0x7fcca00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca567a640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca015e790 0x7fcca00cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca567a640 1 -- 172.21.15.172:0/2238196793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca00ca5e0 con 0x7fcca00b2830 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca567a640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcca00b2830 0x7fcca00cb320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0098f0 tx=0x7fcc9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcc95ffb640 1 -- 172.21.15.172:0/2238196793 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c035040 con 0x7fcca00b2830 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcc95ffb640 1 -- 172.21.15.172:0/2238196793 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c030d20 con 0x7fcca00b2830 2024-04-14T08:33:00.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca78de640 1 -- 172.21.15.172:0/2238196793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca00cb9c0 con 0x7fcca00b2830 2024-04-14T08:33:00.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcc95ffb640 1 -- 172.21.15.172:0/2238196793 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c006e10 con 0x7fcca00b2830 2024-04-14T08:33:00.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.634+0000 7fcca78de640 1 -- 172.21.15.172:0/2238196793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca00cbf80 con 0x7fcca00b2830 2024-04-14T08:33:00.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.635+0000 7fcca78de640 1 -- 172.21.15.172:0/2238196793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca00b6bf0 con 0x7fcca00b2830 2024-04-14T08:33:00.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.636+0000 7fcc95ffb640 1 -- 172.21.15.172:0/2238196793 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc9c038020 con 0x7fcca00b2830 2024-04-14T08:33:00.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.636+0000 7fcc95ffb640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcc8406c8f0 0x7fcc8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:00.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.636+0000 7fcc95ffb640 1 -- 172.21.15.172:0/2238196793 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcc9c0ae410 con 0x7fcca00b2830 2024-04-14T08:33:00.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.636+0000 7fcca4e79640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcc8406c8f0 0x7fcc8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:00.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.637+0000 7fcca4e79640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcc8406c8f0 0x7fcc8406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc90007fd0 tx=0x7fcc90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:00.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.641+0000 7fcc95ffb640 1 -- 172.21.15.172:0/2238196793 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcca00b6bf0 con 0x7fcca00b2830 2024-04-14T08:33:00.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.817+0000 7fcca78de640 1 -- 172.21.15.172:0/2238196793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcca015f580 con 0x7fcca00b2830 2024-04-14T08:33:00.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.818+0000 7fcc95ffb640 1 -- 172.21.15.172:0/2238196793 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7fcc9c02d040 con 0x7fcca00b2830 2024-04-14T08:33:00.819 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:00.819 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:00.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.820+0000 7fcca78de640 1 -- 172.21.15.172:0/2238196793 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcc8406c8f0 msgr2=0x7fcc8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.820+0000 7fcca78de640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcc8406c8f0 0x7fcc8406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc90007fd0 tx=0x7fcc90009040 comp rx=0 tx=0).stop 2024-04-14T08:33:00.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.820+0000 7fcca78de640 1 -- 172.21.15.172:0/2238196793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcca00b2830 msgr2=0x7fcca00cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:00.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.820+0000 7fcca78de640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcca00b2830 0x7fcca00cb320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0098f0 tx=0x7fcc9c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.820+0000 7fcca78de640 1 -- 172.21.15.172:0/2238196793 shutdown_connections 2024-04-14T08:33:00.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.820+0000 7fcca78de640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcc8406c8f0 0x7fcc8406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.820+0000 7fcca78de640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcca015e790 0x7fcca00cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.820+0000 7fcca78de640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcca00d2680 0x7fcca00ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.820+0000 7fcca78de640 1 --2- 172.21.15.172:0/2238196793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcca00b2830 0x7fcca00cb320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:00.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.820+0000 7fcca78de640 1 -- 172.21.15.172:0/2238196793 >> 172.21.15.172:0/2238196793 conn(0x7fcca009d590 msgr2=0x7fcca015ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:00.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.820+0000 7fcca78de640 1 -- 172.21.15.172:0/2238196793 shutdown_connections 2024-04-14T08:33:00.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:00.820+0000 7fcca78de640 1 -- 172.21.15.172:0/2238196793 wait complete. 2024-04-14T08:33:01.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:32:53.535483+0000', 'modified': '2024-04-14T08:32:58.586363+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4309}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4309': {'gid': 4309, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6835/517963877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6833', 'nonce': 517963877}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 517963877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4309, 'qdb_cluster': [4309]} 2024-04-14T08:33:01.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:33:01.235 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-14T08:33:01.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.337+0000 7fa82e02d640 1 Processor -- start 2024-04-14T08:33:01.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.337+0000 7fa82e02d640 1 -- start start 2024-04-14T08:33:01.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa82e02d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa82815e590 0x7fa8281629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa82e02d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa8280cea60 0x7fa8280b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa82e02d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8280b3750 con 0x7fa8280b63d0 2024-04-14T08:33:01.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa82e02d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa8280cef20 con 0x7fa8280cea60 2024-04-14T08:33:01.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa82e02d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa8280cf200 con 0x7fa82815e590 2024-04-14T08:33:01.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa826ffd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa8280cea60 0x7fa8280b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa8277fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa82815e590 0x7fa8281629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa827fff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fa8280b63d0 0x7fa82806c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:54164/0 (socket says 172.21.15.172:54164) 2024-04-14T08:33:01.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa827fff640 1 -- 172.21.15.172:0/4239847973 learned_addr learned my addr 172.21.15.172:0/4239847973 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:01.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa8277fe640 1 -- 172.21.15.172:0/4239847973 >> v2:172.21.15.177:3300/0 conn(0x7fa8280cea60 msgr2=0x7fa8280b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:01.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa8277fe640 1 --2- 172.21.15.172:0/4239847973 >> v2:172.21.15.177:3300/0 conn(0x7fa8280cea60 0x7fa8280b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa8277fe640 1 -- 172.21.15.172:0/4239847973 >> v1:172.21.15.177:6789/0 conn(0x7fa8280b63d0 legacy=0x7fa82806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa8277fe640 1 -- 172.21.15.172:0/4239847973 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8280cf4e0 con 0x7fa82815e590 2024-04-14T08:33:01.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.338+0000 7fa826ffd640 1 --2- 172.21.15.172:0/4239847973 >> v2:172.21.15.177:3300/0 conn(0x7fa8280cea60 0x7fa8280b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:01.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.339+0000 7fa8277fe640 1 --2- 172.21.15.172:0/4239847973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa82815e590 0x7fa8281629c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa8180097a0 tx=0x7fa81802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74b1aaac25b319a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.339+0000 7fa8267fc640 1 -- 172.21.15.172:0/4239847973 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818035020 con 0x7fa82815e590 2024-04-14T08:33:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.339+0000 7fa8267fc640 1 -- 172.21.15.172:0/4239847973 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa818031830 con 0x7fa82815e590 2024-04-14T08:33:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.339+0000 7fa8267fc640 1 -- 172.21.15.172:0/4239847973 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818031c00 con 0x7fa82815e590 2024-04-14T08:33:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.340+0000 7fa82e02d640 1 -- 172.21.15.172:0/4239847973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa82815e590 msgr2=0x7fa8281629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.340+0000 7fa82e02d640 1 --2- 172.21.15.172:0/4239847973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa82815e590 0x7fa8281629c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa8180097a0 tx=0x7fa81802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.340+0000 7fa82e02d640 1 -- 172.21.15.172:0/4239847973 shutdown_connections 2024-04-14T08:33:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.340+0000 7fa82e02d640 1 --2- 172.21.15.172:0/4239847973 >> v2:172.21.15.177:3300/0 conn(0x7fa8280cea60 0x7fa8280b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.340+0000 7fa82e02d640 1 --2- 172.21.15.172:0/4239847973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa82815e590 0x7fa8281629c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.340+0000 7fa82e02d640 1 -- 172.21.15.172:0/4239847973 >> 172.21.15.172:0/4239847973 conn(0x7fa8280d0e90 msgr2=0x7fa8280d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.340+0000 7fa82e02d640 1 -- 172.21.15.172:0/4239847973 shutdown_connections 2024-04-14T08:33:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.340+0000 7fa82e02d640 1 -- 172.21.15.172:0/4239847973 wait complete. 2024-04-14T08:33:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.341+0000 7fa82e02d640 1 Processor -- start 2024-04-14T08:33:01.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.341+0000 7fa82e02d640 1 -- start start 2024-04-14T08:33:01.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.342+0000 7fa82e02d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8280b63d0 0x7fa82815f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.342+0000 7fa82e02d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8280cea60 0x7fa82814ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.342+0000 7fa82e02d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa82815e590 0x7fa828160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.342+0000 7fa82e02d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8280d38f0 con 0x7fa8280b63d0 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.342+0000 7fa82e02d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8280d3770 con 0x7fa8280cea60 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.342+0000 7fa82e02d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa8280d3a70 con 0x7fa82815e590 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.342+0000 7fa8277fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8280b63d0 0x7fa82815f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.342+0000 7fa8277fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8280b63d0 0x7fa82815f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38852/0 (socket says 172.21.15.172:38852) 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.342+0000 7fa8277fe640 1 -- 172.21.15.172:0/1918615098 learned_addr learned my addr 172.21.15.172:0/1918615098 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.342+0000 7fa826ffd640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8280cea60 0x7fa82814ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.342+0000 7fa827fff640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa82815e590 0x7fa828160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.343+0000 7fa8277fe640 1 -- 172.21.15.172:0/1918615098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa82815e590 msgr2=0x7fa828160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.343+0000 7fa8277fe640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa82815e590 0x7fa828160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.343+0000 7fa8277fe640 1 -- 172.21.15.172:0/1918615098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8280cea60 msgr2=0x7fa82814ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.343+0000 7fa8277fe640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8280cea60 0x7fa82814ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.343+0000 7fa8277fe640 1 -- 172.21.15.172:0/1918615098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8280c9500 con 0x7fa8280b63d0 2024-04-14T08:33:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.343+0000 7fa8277fe640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8280b63d0 0x7fa82815f410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa818000c00 tx=0x7fa8180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.343+0000 7fa80ffff640 1 -- 172.21.15.172:0/1918615098 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818040070 con 0x7fa8280b63d0 2024-04-14T08:33:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.343+0000 7fa80ffff640 1 -- 172.21.15.172:0/1918615098 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81803a850 con 0x7fa8280b63d0 2024-04-14T08:33:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.343+0000 7fa82e02d640 1 -- 172.21.15.172:0/1918615098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8280c97e0 con 0x7fa8280b63d0 2024-04-14T08:33:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.343+0000 7fa80ffff640 1 -- 172.21.15.172:0/1918615098 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818035040 con 0x7fa8280b63d0 2024-04-14T08:33:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.343+0000 7fa82e02d640 1 -- 172.21.15.172:0/1918615098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8280ca080 con 0x7fa8280b63d0 2024-04-14T08:33:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.344+0000 7fa80ffff640 1 -- 172.21.15.172:0/1918615098 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa81804b060 con 0x7fa8280b63d0 2024-04-14T08:33:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.345+0000 7fa80ffff640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa81406c8f0 0x7fa81406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.345+0000 7fa80ffff640 1 -- 172.21.15.172:0/1918615098 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa8180ad650 con 0x7fa8280b63d0 2024-04-14T08:33:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.345+0000 7fa826ffd640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa81406c8f0 0x7fa81406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.345+0000 7fa826ffd640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa81406c8f0 0x7fa81406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009560 tx=0x7fa81c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.346+0000 7fa830e0f740 1 Processor -- start 2024-04-14T08:33:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.346+0000 7fa830e0f740 1 -- start start 2024-04-14T08:33:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.346+0000 7fa830e0f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc14deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.346+0000 7fa830e0f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc150e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.346+0000 7fa830e0f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc14df040 0x563bc150ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa830e0f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563bc1448030 con 0x563bc14df040 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa830e0f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563bc13f0620 con 0x563bc150bf40 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa830e0f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563bc1443fe0 con 0x563bc150bb60 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa827fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc14df040 0x563bc150ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa8277fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc14deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa827fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc14df040 0x563bc150ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38866/0 (socket says 172.21.15.172:38866) 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa827fff640 1 -- 172.21.15.172:0/743949692 learned_addr learned my addr 172.21.15.172:0/743949692 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa826ffd640 1 --2- 172.21.15.172:0/743949692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc150e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa8277fe640 1 -- 172.21.15.172:0/743949692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 msgr2=0x563bc150e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa8277fe640 1 --2- 172.21.15.172:0/743949692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc150e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa8277fe640 1 -- 172.21.15.172:0/743949692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc14df040 msgr2=0x563bc150ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa8277fe640 1 --2- 172.21.15.172:0/743949692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc14df040 0x563bc150ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa8277fe640 1 -- 172.21.15.172:0/743949692 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc115c2d0 con 0x563bc150bb60 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa8277fe640 1 --2- 172.21.15.172:0/743949692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc14deb00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa8180ab810 tx=0x7fa818031800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9abe483499f66898 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa80d7fa640 1 -- 172.21.15.172:0/743949692 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818083c20 con 0x563bc150bb60 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa80d7fa640 1 -- 172.21.15.172:0/743949692 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa818079360 con 0x563bc150bb60 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.347+0000 7fa80d7fa640 1 -- 172.21.15.172:0/743949692 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81803ac40 con 0x563bc150bb60 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.348+0000 7fa830e0f740 1 -- 172.21.15.172:0/743949692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 msgr2=0x563bc14deb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.348+0000 7fa830e0f740 1 --2- 172.21.15.172:0/743949692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc14deb00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa8180ab810 tx=0x7fa818031800 comp rx=0 tx=0).stop 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.348+0000 7fa830e0f740 1 -- 172.21.15.172:0/743949692 shutdown_connections 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.348+0000 7fa830e0f740 1 --2- 172.21.15.172:0/743949692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc14df040 0x563bc150ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.348+0000 7fa830e0f740 1 --2- 172.21.15.172:0/743949692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc150e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.348+0000 7fa830e0f740 1 --2- 172.21.15.172:0/743949692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc14deb00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.348+0000 7fa830e0f740 1 -- 172.21.15.172:0/743949692 >> 172.21.15.172:0/743949692 conn(0x563bc14e0d70 msgr2=0x563bc15298f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.348+0000 7fa830e0f740 1 -- 172.21.15.172:0/743949692 shutdown_connections 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.348+0000 7fa830e0f740 1 -- 172.21.15.172:0/743949692 wait complete. 2024-04-14T08:33:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.348+0000 7fa830e0f740 1 -- start start 2024-04-14T08:33:01.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa830e0f740 1 Processor -- start 2024-04-14T08:33:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa830e0f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc1511d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa830e0f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc1509630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa830e0f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc1529ea0 0x563bc1514d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa830e0f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563bc1448030 con 0x563bc1529ea0 2024-04-14T08:33:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa830e0f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563bc14039e0 con 0x563bc150bf40 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa830e0f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563bc1443fe0 con 0x563bc150bb60 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa830e0f740 10 client.0 _ll_register_callbacks cb 0x563bc1440080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa827fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc1529ea0 0x563bc1514d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa8277fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc1511d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa8277fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc1511d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:58206/0 (socket says 172.21.15.172:58206) 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa826ffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc1509630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.349+0000 7fa8277fe640 1 -- 172.21.15.172:0/2834752635 learned_addr learned my addr 172.21.15.172:0/2834752635 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa827fff640 1 -- 172.21.15.172:0/2834752635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 msgr2=0x563bc1511d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa827fff640 1 --2- 172.21.15.172:0/2834752635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc1511d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa827fff640 1 -- 172.21.15.172:0/2834752635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 msgr2=0x563bc1509630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa827fff640 1 --2- 172.21.15.172:0/2834752635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc1509630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa827fff640 1 -- 172.21.15.172:0/2834752635 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc12266b0 con 0x563bc1529ea0 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa827fff640 1 --2- 172.21.15.172:0/2834752635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc1529ea0 0x563bc1514d60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x563bc14e1d00 tx=0x7fa820012e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcc6f4dee0e8fe server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa8067fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa8067fc640 1 -- 172.21.15.172:0/2834752635 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820019020 con 0x563bc1529ea0 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa8067fc640 1 -- 172.21.15.172:0/2834752635 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa820005df0 con 0x563bc1529ea0 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa830e0f740 1 -- 172.21.15.172:0/2834752635 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa818079360 con 0x563bc1529ea0 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa8067fc640 1 -- 172.21.15.172:0/2834752635 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa820023050 con 0x563bc1529ea0 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa830e0f740 10 client.4428 fetch_fsmap learned FSMap version 7 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa830e0f740 1 -- 172.21.15.172:0/2834752635 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x563bc1514530 con 0x563bc1529ea0 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa8067fc640 1 -- 172.21.15.172:0/2834752635 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa818079360 con 0x563bc1529ea0 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa8067fc640 1 -- 172.21.15.172:0/2834752635 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fa820026020 con 0x563bc1529ea0 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa830e0f740 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa830e0f740 10 client.4428 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa830e0f740 20 client.4428 populate_metadata read hostname 'smithi172' 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa830e0f740 1 --2- 172.21.15.172:0/2834752635 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x563bc153d560 0x563bc15104c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa830e0f740 4 client.4428 mds_command: new command op to 4309 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.350+0000 7fa830e0f740 1 -- 172.21.15.172:0/2834752635 --> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bc13efcc0 con 0x563bc153d560 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.351+0000 7fa8277fe640 1 --2- 172.21.15.172:0/2834752635 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x563bc153d560 0x563bc15104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.351+0000 7fa8067fc640 10 client.4428 ms_handle_connect on v2:172.21.15.177:6833/517963877 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.351+0000 7fa8277fe640 1 --2- 172.21.15.172:0/2834752635 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x563bc153d560 0x563bc15104c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f5d725882870af7 server_cookie=9a5a6aff62e6163a in_seq=0 out_seq=0 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa8067fc640 1 -- 172.21.15.172:0/2834752635 <== mds.0 v2:172.21.15.177:6833/517963877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563bc13efcc0 con 0x563bc153d560 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa8067fc640 10 client.4428 handle_command_reply: tid=0 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa830e0f740 1 client.4428 shutdown 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa830e0f740 1 -- 172.21.15.172:0/2834752635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc1529ea0 msgr2=0x563bc1514d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa830e0f740 1 --2- 172.21.15.172:0/2834752635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc1529ea0 0x563bc1514d60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x563bc14e1d00 tx=0x7fa820012e30 comp rx=0 tx=0).stop 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa830e0f740 1 -- 172.21.15.172:0/2834752635 shutdown_connections 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa830e0f740 1 --2- 172.21.15.172:0/2834752635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc1529ea0 0x563bc1514d60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa830e0f740 1 --2- 172.21.15.172:0/2834752635 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x563bc153d560 0x563bc15104c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa830e0f740 1 --2- 172.21.15.172:0/2834752635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc1509630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa830e0f740 1 --2- 172.21.15.172:0/2834752635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc1511d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa830e0f740 1 -- 172.21.15.172:0/2834752635 >> 172.21.15.172:0/2834752635 conn(0x563bc14df040 msgr2=0x563bc14e2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:01.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa8067fc640 0 client.4428 ms_handle_reset on v2:172.21.15.177:6833/517963877 2024-04-14T08:33:01.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa830e0f740 1 -- 172.21.15.172:0/2834752635 shutdown_connections 2024-04-14T08:33:01.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa830e0f740 1 -- 172.21.15.172:0/2834752635 wait complete. 2024-04-14T08:33:01.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.352+0000 7fa830e0f740 20 client.4428 trim_cache size 0 max 16384 2024-04-14T08:33:01.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.370+0000 7fa830e0f740 1 Processor -- start 2024-04-14T08:33:01.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.370+0000 7fa830e0f740 1 -- start start 2024-04-14T08:33:01.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa830e0f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc1511cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa830e0f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bf40 0x563bc154d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa830e0f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc1529ea0 0x563bc154d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa830e0f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa828200fe0 con 0x563bc150bf40 2024-04-14T08:33:01.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa830e0f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa82814e4e0 con 0x563bc1529ea0 2024-04-14T08:33:01.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa830e0f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa828200990 con 0x563bc150bb60 2024-04-14T08:33:01.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa826ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bf40 0x563bc154d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa826ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bf40 0x563bc154d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38890/0 (socket says 172.21.15.172:38890) 2024-04-14T08:33:01.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa826ffd640 1 -- 172.21.15.172:0/3342217285 learned_addr learned my addr 172.21.15.172:0/3342217285 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:01.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa8277fe640 1 --2- 172.21.15.172:0/3342217285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc1511cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa827fff640 1 --2- 172.21.15.172:0/3342217285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc1529ea0 0x563bc154d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa826ffd640 1 -- 172.21.15.172:0/3342217285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 msgr2=0x563bc1511cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa826ffd640 1 --2- 172.21.15.172:0/3342217285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc1511cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa826ffd640 1 -- 172.21.15.172:0/3342217285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc1529ea0 msgr2=0x563bc154d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa826ffd640 1 --2- 172.21.15.172:0/3342217285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc1529ea0 0x563bc154d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa826ffd640 1 -- 172.21.15.172:0/3342217285 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc1515ff0 con 0x563bc150bf40 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa827fff640 1 --2- 172.21.15.172:0/3342217285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc1529ea0 0x563bc154d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa826ffd640 1 --2- 172.21.15.172:0/3342217285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bf40 0x563bc154d1d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa81c004090 tx=0x7fa81c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53aab98d0c4e262d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa80cff9640 1 -- 172.21.15.172:0/3342217285 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c003a20 con 0x563bc150bf40 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa80cff9640 1 -- 172.21.15.172:0/3342217285 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c028070 con 0x563bc150bf40 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.371+0000 7fa80cff9640 1 -- 172.21.15.172:0/3342217285 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c003e00 con 0x563bc150bf40 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.372+0000 7fa830e0f740 1 -- 172.21.15.172:0/3342217285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bf40 msgr2=0x563bc154d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.372+0000 7fa830e0f740 1 --2- 172.21.15.172:0/3342217285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bf40 0x563bc154d1d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa81c004090 tx=0x7fa81c014000 comp rx=0 tx=0).stop 2024-04-14T08:33:01.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.372+0000 7fa830e0f740 1 -- 172.21.15.172:0/3342217285 shutdown_connections 2024-04-14T08:33:01.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.372+0000 7fa830e0f740 1 --2- 172.21.15.172:0/3342217285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc1529ea0 0x563bc154d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.372+0000 7fa830e0f740 1 --2- 172.21.15.172:0/3342217285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bf40 0x563bc154d1d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.372+0000 7fa830e0f740 1 --2- 172.21.15.172:0/3342217285 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc150bb60 0x563bc1511cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.372+0000 7fa830e0f740 1 -- 172.21.15.172:0/3342217285 >> 172.21.15.172:0/3342217285 conn(0x563bc14df040 msgr2=0x563bc150e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:01.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.372+0000 7fa830e0f740 1 -- 172.21.15.172:0/3342217285 shutdown_connections 2024-04-14T08:33:01.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.372+0000 7fa830e0f740 1 -- 172.21.15.172:0/3342217285 wait complete. 2024-04-14T08:33:01.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.372+0000 7fa830e0f740 1 -- start start 2024-04-14T08:33:01.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa830e0f740 1 Processor -- start 2024-04-14T08:33:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa830e0f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bb60 0x563bc1511cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa830e0f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc159da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa830e0f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc1529ea0 0x563bc14dd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa830e0f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563bc153d3e0 con 0x563bc150bb60 2024-04-14T08:33:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa830e0f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563bc1510ba0 con 0x563bc150bf40 2024-04-14T08:33:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa830e0f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563bc153da30 con 0x563bc1529ea0 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa830e0f740 10 client.0 _ll_register_callbacks cb 0x7fa820001ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa8277fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bb60 0x563bc1511cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa8277fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bb60 0x563bc1511cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38900/0 (socket says 172.21.15.172:38900) 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa8277fe640 1 -- 172.21.15.172:0/454017185 learned_addr learned my addr 172.21.15.172:0/454017185 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa827fff640 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc1529ea0 0x563bc14dd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.373+0000 7fa826ffd640 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc159da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa8277fe640 1 -- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc1529ea0 msgr2=0x563bc14dd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa8277fe640 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc1529ea0 0x563bc14dd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa8277fe640 1 -- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 msgr2=0x563bc159da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa8277fe640 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc159da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa8277fe640 1 -- 172.21.15.172:0/454017185 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc1529850 con 0x563bc150bb60 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa826ffd640 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc159da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa8277fe640 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bb60 0x563bc1511cb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa81802bee0 tx=0x7fa81803c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa04275ee55cd10d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa8067fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa8067fc640 1 -- 172.21.15.172:0/454017185 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818079360 con 0x563bc150bb60 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa8067fc640 1 -- 172.21.15.172:0/454017185 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa8180776d0 con 0x563bc150bb60 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa830e0f740 1 -- 172.21.15.172:0/454017185 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa81c028070 con 0x563bc150bb60 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa8067fc640 1 -- 172.21.15.172:0/454017185 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa81803ac40 con 0x563bc150bb60 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa830e0f740 10 client.4434 fetch_fsmap learned FSMap version 7 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa830e0f740 1 -- 172.21.15.172:0/454017185 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x563bc1529850 con 0x563bc150bb60 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa8067fc640 1 -- 172.21.15.172:0/454017185 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c028070 con 0x563bc150bb60 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa8067fc640 1 -- 172.21.15.172:0/454017185 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fa8180a9030 con 0x563bc150bb60 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa830e0f740 10 client.4434 fetch_fsmap finished waiting for FSMap version 7 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa830e0f740 10 client.4434 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa830e0f740 20 client.4434 populate_metadata read hostname 'smithi172' 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa830e0f740 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x563bc153ba50 0x563bc153be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa830e0f740 4 client.4434 mds_command: new command op to 4309 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa830e0f740 1 -- 172.21.15.172:0/454017185 --> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] -- command(tid 0: {"prefix": "status"}) -- 0x563bc1455570 con 0x563bc153ba50 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.374+0000 7fa826ffd640 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x563bc153ba50 0x563bc153be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa8067fc640 10 client.4434 ms_handle_connect on v2:172.21.15.177:6833/517963877 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa826ffd640 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x563bc153ba50 0x563bc153be30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d478725f0dd007 server_cookie=2a5107813243dcfe in_seq=0 out_seq=0 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa8067fc640 1 -- 172.21.15.172:0/454017185 <== mds.0 v2:172.21.15.177:6833/517963877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x563bc1455570 con 0x563bc153ba50 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa8067fc640 10 client.4434 handle_command_reply: tid=0 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa830e0f740 1 client.4434 shutdown 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa830e0f740 1 -- 172.21.15.172:0/454017185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bb60 msgr2=0x563bc1511cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa830e0f740 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bb60 0x563bc1511cb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa81802bee0 tx=0x7fa81803c040 comp rx=0 tx=0).stop 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa830e0f740 1 -- 172.21.15.172:0/454017185 shutdown_connections 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa830e0f740 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563bc1529ea0 0x563bc14dd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa830e0f740 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x563bc153ba50 0x563bc153be30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa830e0f740 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bc150bf40 0x563bc159da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa8067fc640 0 client.4434 ms_handle_reset on v2:172.21.15.177:6833/517963877 2024-04-14T08:33:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa830e0f740 1 --2- 172.21.15.172:0/454017185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bc150bb60 0x563bc1511cb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa830e0f740 1 -- 172.21.15.172:0/454017185 >> 172.21.15.172:0/454017185 conn(0x563bc14df040 msgr2=0x563bc1513520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa830e0f740 1 -- 172.21.15.172:0/454017185 shutdown_connections 2024-04-14T08:33:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa830e0f740 1 -- 172.21.15.172:0/454017185 wait complete. 2024-04-14T08:33:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.375+0000 7fa830e0f740 20 client.4434 trim_cache size 0 max 16384 2024-04-14T08:33:01.383 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:33:01.383 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:33:01.383 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:33:01.383 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 4309, 2024-04-14T08:33:01.383 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:33:01.384 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:33:01.384 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:33:01.384 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 7.1934373579999997, 2024-04-14T08:33:01.384 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 7, 2024-04-14T08:33:01.384 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 25, 2024-04-14T08:33:01.384 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 25, 2024-04-14T08:33:01.384 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 14.590064765999999 2024-04-14T08:33:01.384 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:33:01.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.376+0000 7fa82e02d640 1 -- 172.21.15.172:0/1918615098 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa81406c8f0 msgr2=0x7fa81406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.376+0000 7fa82e02d640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa81406c8f0 0x7fa81406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009560 tx=0x7fa81c009190 comp rx=0 tx=0).stop 2024-04-14T08:33:01.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.376+0000 7fa82e02d640 1 -- 172.21.15.172:0/1918615098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8280b63d0 msgr2=0x7fa82815f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.376+0000 7fa82e02d640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8280b63d0 0x7fa82815f410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa818000c00 tx=0x7fa8180046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.376+0000 7fa82e02d640 1 -- 172.21.15.172:0/1918615098 shutdown_connections 2024-04-14T08:33:01.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.376+0000 7fa82e02d640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa81406c8f0 0x7fa81406edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.376+0000 7fa82e02d640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa82815e590 0x7fa828160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.376+0000 7fa82e02d640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8280cea60 0x7fa82814ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.376+0000 7fa82e02d640 1 --2- 172.21.15.172:0/1918615098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8280b63d0 0x7fa82815f410 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.376+0000 7fa82e02d640 1 -- 172.21.15.172:0/1918615098 >> 172.21.15.172:0/1918615098 conn(0x7fa8280d0e90 msgr2=0x7fa8280b5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:01.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.376+0000 7fa82e02d640 1 -- 172.21.15.172:0/1918615098 shutdown_connections 2024-04-14T08:33:01.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.376+0000 7fa82e02d640 1 -- 172.21.15.172:0/1918615098 wait complete. 2024-04-14T08:33:01.387 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-14T08:33:01.387 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-14T08:33:01.387 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:33:01.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.487+0000 7efe01888640 1 Processor -- start 2024-04-14T08:33:01.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.487+0000 7efe01888640 1 -- start start 2024-04-14T08:33:01.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efe01888640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efdfc1530d0 0x7efdfc0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efe01888640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdfc160780 0x7efdfc0d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efe01888640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdfc15c780 con 0x7efdfc0cc8f0 2024-04-14T08:33:01.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efe01888640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7efdfc0d20a0 con 0x7efdfc1530d0 2024-04-14T08:33:01.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efe01888640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efdfc15ca00 con 0x7efdfc160780 2024-04-14T08:33:01.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efdf3fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7efdfc0cc8f0 0x7efdfc0b3d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40222/0 (socket says 172.21.15.172:40222) 2024-04-14T08:33:01.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efdf3fff640 1 -- 172.21.15.172:0/3626495012 learned_addr learned my addr 172.21.15.172:0/3626495012 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:01.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efdfaffd640 1 --2- 172.21.15.172:0/3626495012 >> v2:172.21.15.177:3300/0 conn(0x7efdfc1530d0 0x7efdfc0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efdfb7fe640 1 --2- 172.21.15.172:0/3626495012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdfc160780 0x7efdfc0d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efdfa7fc640 1 -- 172.21.15.172:0/3626495012 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4198577275 0 0) 0x7efde4001110 con 0x7efdfc0cc8f0 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efdfa7fc640 1 -- 172.21.15.172:0/3626495012 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efde8003520 con 0x7efdfc0cc8f0 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efdfaffd640 1 -- 172.21.15.172:0/3626495012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdfc160780 msgr2=0x7efdfc0d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efdfaffd640 1 --2- 172.21.15.172:0/3626495012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdfc160780 0x7efdfc0d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efdfaffd640 1 -- 172.21.15.172:0/3626495012 >> v1:172.21.15.172:6789/0 conn(0x7efdfc0cc8f0 legacy=0x7efdfc0b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.488+0000 7efdfaffd640 1 -- 172.21.15.172:0/3626495012 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc15cce0 con 0x7efdfc1530d0 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.489+0000 7efdfaffd640 1 --2- 172.21.15.172:0/3626495012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdfc1530d0 0x7efdfc0cfe70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efdec0097a0 tx=0x7efdec02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd43c5f7a3e89524 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.489+0000 7efdfa7fc640 1 -- 172.21.15.172:0/3626495012 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec035020 con 0x7efdfc1530d0 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.489+0000 7efdfa7fc640 1 -- 172.21.15.172:0/3626495012 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdec031830 con 0x7efdfc1530d0 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.489+0000 7efdfa7fc640 1 -- 172.21.15.172:0/3626495012 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec031c00 con 0x7efdfc1530d0 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.490+0000 7efe01888640 1 -- 172.21.15.172:0/3626495012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdfc1530d0 msgr2=0x7efdfc0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.490+0000 7efe01888640 1 --2- 172.21.15.172:0/3626495012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdfc1530d0 0x7efdfc0cfe70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efdec0097a0 tx=0x7efdec02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.490+0000 7efe01888640 1 -- 172.21.15.172:0/3626495012 shutdown_connections 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.490+0000 7efe01888640 1 --2- 172.21.15.172:0/3626495012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdfc160780 0x7efdfc0d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.490+0000 7efe01888640 1 --2- 172.21.15.172:0/3626495012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdfc1530d0 0x7efdfc0cfe70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.490+0000 7efe01888640 1 -- 172.21.15.172:0/3626495012 >> 172.21.15.172:0/3626495012 conn(0x7efdfc06f820 msgr2=0x7efdfc06fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:01.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.490+0000 7efe01888640 1 -- 172.21.15.172:0/3626495012 shutdown_connections 2024-04-14T08:33:01.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.490+0000 7efe01888640 1 -- 172.21.15.172:0/3626495012 wait complete. 2024-04-14T08:33:01.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.491+0000 7efe01888640 1 Processor -- start 2024-04-14T08:33:01.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.491+0000 7efe01888640 1 -- start start 2024-04-14T08:33:01.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efe01888640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdfc0cc8f0 0x7efdfc14ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efe01888640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdfc1530d0 0x7efdfc1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efe01888640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdfc160780 0x7efdfc157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efe01888640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc0d0c10 con 0x7efdfc1530d0 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efe01888640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc15d2a0 con 0x7efdfc0cc8f0 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efe01888640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efdfc1534b0 con 0x7efdfc160780 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdf3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdfc1530d0 0x7efdfc1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdf3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdfc1530d0 0x7efdfc1573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38908/0 (socket says 172.21.15.172:38908) 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdf3fff640 1 -- 172.21.15.172:0/1389123547 learned_addr learned my addr 172.21.15.172:0/1389123547 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdfb7fe640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdfc160780 0x7efdfc157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdfaffd640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdfc0cc8f0 0x7efdfc14ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdf3fff640 1 -- 172.21.15.172:0/1389123547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdfc160780 msgr2=0x7efdfc157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdf3fff640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdfc160780 0x7efdfc157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdf3fff640 1 -- 172.21.15.172:0/1389123547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdfc0cc8f0 msgr2=0x7efdfc14ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdf3fff640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdfc0cc8f0 0x7efdfc14ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdf3fff640 1 -- 172.21.15.172:0/1389123547 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc0ca540 con 0x7efdfc1530d0 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdfaffd640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdfc0cc8f0 0x7efdfc14ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdf3fff640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdfc1530d0 0x7efdfc1573e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efde400c3a0 tx=0x7efde400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.492+0000 7efdf8ff9640 1 -- 172.21.15.172:0/1389123547 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4017070 con 0x7efdfc1530d0 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.493+0000 7efdf8ff9640 1 -- 172.21.15.172:0/1389123547 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efde4005ce0 con 0x7efdfc1530d0 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.493+0000 7efe01888640 1 -- 172.21.15.172:0/1389123547 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdfc0b43e0 con 0x7efdfc1530d0 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.493+0000 7efdf8ff9640 1 -- 172.21.15.172:0/1389123547 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4007310 con 0x7efdfc1530d0 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.493+0000 7efe01888640 1 -- 172.21.15.172:0/1389123547 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdfc0b49a0 con 0x7efdfc1530d0 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.494+0000 7efe01888640 1 -- 172.21.15.172:0/1389123547 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdfc068520 con 0x7efdfc1530d0 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.494+0000 7efdf8ff9640 1 -- 172.21.15.172:0/1389123547 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efde401a070 con 0x7efdfc1530d0 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.494+0000 7efdf8ff9640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efddc06c920 0x7efddc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.495+0000 7efdfaffd640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efddc06c920 0x7efddc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.495+0000 7efdf8ff9640 1 -- 172.21.15.172:0/1389123547 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efde408b250 con 0x7efdfc1530d0 2024-04-14T08:33:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.495+0000 7efdfaffd640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efddc06c920 0x7efddc06ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efdec036000 tx=0x7efdec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:01.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.499+0000 7efdf8ff9640 1 -- 172.21.15.172:0/1389123547 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efdfc15c780 con 0x7efdfc1530d0 2024-04-14T08:33:01.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.709+0000 7efe01888640 1 -- 172.21.15.172:0/1389123547 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efdfc071840 con 0x7efdfc1530d0 2024-04-14T08:33:01.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.709+0000 7efdf8ff9640 1 -- 172.21.15.172:0/1389123547 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7efde4058c60 con 0x7efdfc1530d0 2024-04-14T08:33:01.710 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-14T08:33:01.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.711+0000 7efe01888640 1 -- 172.21.15.172:0/1389123547 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efddc06c920 msgr2=0x7efddc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.711+0000 7efe01888640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efddc06c920 0x7efddc06ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efdec036000 tx=0x7efdec008030 comp rx=0 tx=0).stop 2024-04-14T08:33:01.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.711+0000 7efe01888640 1 -- 172.21.15.172:0/1389123547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdfc1530d0 msgr2=0x7efdfc1573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:01.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.711+0000 7efe01888640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdfc1530d0 0x7efdfc1573e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efde400c3a0 tx=0x7efde400c860 comp rx=0 tx=0).stop 2024-04-14T08:33:01.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.711+0000 7efe01888640 1 -- 172.21.15.172:0/1389123547 shutdown_connections 2024-04-14T08:33:01.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.711+0000 7efe01888640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efddc06c920 0x7efddc06ede0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.711+0000 7efe01888640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdfc160780 0x7efdfc157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.711+0000 7efe01888640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdfc1530d0 0x7efdfc1573e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.711+0000 7efe01888640 1 --2- 172.21.15.172:0/1389123547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdfc0cc8f0 0x7efdfc14ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:01.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.711+0000 7efe01888640 1 -- 172.21.15.172:0/1389123547 >> 172.21.15.172:0/1389123547 conn(0x7efdfc06f820 msgr2=0x7efdfc0ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:01.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.711+0000 7efe01888640 1 -- 172.21.15.172:0/1389123547 shutdown_connections 2024-04-14T08:33:01.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01.711+0000 7efe01888640 1 -- 172.21.15.172:0/1389123547 wait complete. 2024-04-14T08:33:01.723 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:33:01.723 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:33:01.723 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:33:01.723 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-14T08:33:01.723 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-14T08:33:01.723 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:33:01.723 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-14T08:33:01.723 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:33:01.723 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-14T08:33:01.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:01.724 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:33:01.739 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:33:01.739 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:33:01.739 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:33:01.739 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:01.739 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:33:01.739 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:01.739 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:01.739 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:01.739 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:33:01.739 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:01.740 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:01.740 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:33:01.740 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:01.740 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:01.740 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:33:01.740 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:01.740 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:33:01.740 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42687sec preferred_lft 42687sec 2024-04-14T08:33:01.740 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:33:01.740 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:01.741 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-14T08:33:01.741 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:01.741 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:01.741 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add name ceph-brx type bridge 2024-04-14T08:33:01.741 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr flush dev ceph-brx 2024-04-14T08:33:01.741 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set ceph-brx up 2024-04-14T08:33:01.741 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-14T08:33:01.741 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:01.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:01.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:01.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:01.931 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:33:01.958 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:33:01.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:01.960 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:33:02.026 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:33:02.027 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:33:02.027 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:33:02.027 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:33:02.027 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:33:02.027 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:02.027 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:02.028 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:33:02.028 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:33:02.028 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:33:02.028 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:02.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:02.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:02.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:02.323 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:33:02.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:02.339 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2024-04-14T08:33:02.395 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:02.395 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:02.395 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-14T08:33:02.395 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-14T08:33:02.395 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:02.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:02.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:02.518 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-14T08:33:02.518 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:02.518 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:02.518 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-14T08:33:02.518 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-14T08:33:02.518 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-14T08:33:02.518 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-14T08:33:02.519 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-14T08:33:02.519 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:02.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:02.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:02.658 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:02.658 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:02.658 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:33:02.658 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.0 master ceph-brx 2024-04-14T08:33:02.658 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:02.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:02.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:02.739 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-14T08:33:02.739 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:33:02.739 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-14T08:33:02.754 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-14T08:33:02.754 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:33:02.754 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-14T08:33:02.809 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2024-04-14T08:33:02.877 DEBUG:teuthology.orchestra.run.smithi172:> cat /proc/self/mounts | awk '{print $2}' 2024-04-14T08:33:02.933 INFO:teuthology.orchestra.run.smithi172.stdout:/proc 2024-04-14T08:33:02.933 INFO:teuthology.orchestra.run.smithi172.stdout:/sys 2024-04-14T08:33:02.933 INFO:teuthology.orchestra.run.smithi172.stdout:/dev 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/security 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/shm 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/pts 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/run 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/cgroup 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/pstore 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/bpf 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/ 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/selinux 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/proc/sys/fs/binfmt_misc 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/mqueue 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/hugepages 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/debug 2024-04-14T08:33:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/tracing 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/fuse/connections 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/config 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-sysctl.service 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/nfs/rpc_pipefs 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/run/user/1000 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph/osd/ceph-0 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph/osd/ceph-1 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph/osd/ceph-2 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph/osd/ceph-3 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns 2024-04-14T08:33:02.935 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-14T08:33:02.936 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-14T08:33:02.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:02.936 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2024-04-14T08:33:02.988 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-14T08:33:02.988 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-14T08:33:03.031 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2024-04-14T08:33:03.057 DEBUG:teuthology.orchestra.run.smithi172:> cat /proc/self/mounts | awk '{print $2}' 2024-04-14T08:33:03.114 INFO:teuthology.orchestra.run.smithi172.stdout:/proc 2024-04-14T08:33:03.114 INFO:teuthology.orchestra.run.smithi172.stdout:/sys 2024-04-14T08:33:03.114 INFO:teuthology.orchestra.run.smithi172.stdout:/dev 2024-04-14T08:33:03.114 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/security 2024-04-14T08:33:03.114 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/shm 2024-04-14T08:33:03.114 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/pts 2024-04-14T08:33:03.114 INFO:teuthology.orchestra.run.smithi172.stdout:/run 2024-04-14T08:33:03.115 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/cgroup 2024-04-14T08:33:03.115 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/pstore 2024-04-14T08:33:03.115 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/bpf 2024-04-14T08:33:03.115 INFO:teuthology.orchestra.run.smithi172.stdout:/ 2024-04-14T08:33:03.115 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/selinux 2024-04-14T08:33:03.115 INFO:teuthology.orchestra.run.smithi172.stdout:/proc/sys/fs/binfmt_misc 2024-04-14T08:33:03.115 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/mqueue 2024-04-14T08:33:03.115 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/hugepages 2024-04-14T08:33:03.115 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/debug 2024-04-14T08:33:03.115 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/tracing 2024-04-14T08:33:03.115 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/fuse/connections 2024-04-14T08:33:03.115 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/config 2024-04-14T08:33:03.116 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-sysctl.service 2024-04-14T08:33:03.116 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-14T08:33:03.116 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-14T08:33:03.116 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/nfs/rpc_pipefs 2024-04-14T08:33:03.116 INFO:teuthology.orchestra.run.smithi172.stdout:/run/user/1000 2024-04-14T08:33:03.116 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph 2024-04-14T08:33:03.116 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph/osd/ceph-0 2024-04-14T08:33:03.116 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph/osd/ceph-1 2024-04-14T08:33:03.116 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph/osd/ceph-2 2024-04-14T08:33:03.116 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph/osd/ceph-3 2024-04-14T08:33:03.116 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns 2024-04-14T08:33:03.117 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-14T08:33:03.117 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-14T08:33:03.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:03.117 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2024-04-14T08:33:03.118 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi172.stderr:2024-04-14T08:33:03.117+0000 7f16aeeb7480 -1 init, newargv = 0x555e44d83bb0 newargc=15 2024-04-14T08:33:03.118 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi172.stderr:ceph-fuse[44451]: starting ceph client 2024-04-14T08:33:03.123 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi172.stderr:ceph-fuse[44451]: starting fuse 2024-04-14T08:33:03.132 INFO:teuthology.orchestra.run.smithi172.stdout:43 2024-04-14T08:33:03.132 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-14T08:33:03.132 DEBUG:teuthology.orchestra.run.smithi172:> sudo stdin-killer -- python3 -c ' 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> import glob 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> import re 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> import subprocess 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> def _find_admin_socket(client_name): 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> files = glob.glob(asok_path) 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> # Given a non-glob path, it better be there 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> if "*" not in asok_path: 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> assert(len(files) == 1) 2024-04-14T08:33:03.133 DEBUG:teuthology.orchestra.run.smithi172:> return files[0] 2024-04-14T08:33:03.134 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:33:03.134 DEBUG:teuthology.orchestra.run.smithi172:> for f in files: 2024-04-14T08:33:03.134 DEBUG:teuthology.orchestra.run.smithi172:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-14T08:33:03.134 DEBUG:teuthology.orchestra.run.smithi172:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-14T08:33:03.134 DEBUG:teuthology.orchestra.run.smithi172:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-14T08:33:03.134 DEBUG:teuthology.orchestra.run.smithi172:> contents = proc_f.read() 2024-04-14T08:33:03.134 DEBUG:teuthology.orchestra.run.smithi172:> if mountpoint in contents: 2024-04-14T08:33:03.134 DEBUG:teuthology.orchestra.run.smithi172:> return f 2024-04-14T08:33:03.134 DEBUG:teuthology.orchestra.run.smithi172:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-14T08:33:03.134 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:33:03.134 DEBUG:teuthology.orchestra.run.smithi172:> print(_find_admin_socket("client.admin")) 2024-04-14T08:33:03.134 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:33:03.253 INFO:teuthology.orchestra.run.smithi172.stdout:/var/run/ceph/ceph-client.admin.44451.asok 2024-04-14T08:33:03.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:03.263 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44451.asok 2024-04-14T08:33:03.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:03.263 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44451.asok status 2024-04-14T08:33:03.400 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:33:03.400 INFO:teuthology.orchestra.run.smithi172.stdout: "metadata": { 2024-04-14T08:33:03.400 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_sha1": "d7e176b41636f150fe82aaedea82def3e9af0203", 2024-04-14T08:33:03.400 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 19.0.0-2592-gd7e176b4 (d7e176b41636f150fe82aaedea82def3e9af0203) squid (dev)", 2024-04-14T08:33:03.400 INFO:teuthology.orchestra.run.smithi172.stdout: "entity_id": "admin", 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: "hostname": "smithi172", 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: "pid": "44451", 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: "root": "/" 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_count": 0, 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_pinned_count": 0, 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 4340, 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: "inst": { 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: "name": { 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "client", 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: "num": 4340 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:03.401 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2024-04-14T08:33:03.402 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2024-04-14T08:33:03.402 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "172.21.15.172:0", 2024-04-14T08:33:03.402 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 4033979893 2024-04-14T08:33:03.402 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:03.402 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:03.402 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2024-04-14T08:33:03.402 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2024-04-14T08:33:03.402 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "172.21.15.172:0", 2024-04-14T08:33:03.402 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 4033979893 2024-04-14T08:33:03.402 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:03.402 INFO:teuthology.orchestra.run.smithi172.stdout: "inst_str": "client.4340 172.21.15.172:0/4033979893", 2024-04-14T08:33:03.402 INFO:teuthology.orchestra.run.smithi172.stdout: "addr_str": "172.21.15.172:0/4033979893", 2024-04-14T08:33:03.403 INFO:teuthology.orchestra.run.smithi172.stdout: "inode_count": 1, 2024-04-14T08:33:03.403 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_epoch": 7, 2024-04-14T08:33:03.403 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch": 25, 2024-04-14T08:33:03.403 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch_barrier": 0, 2024-04-14T08:33:03.403 INFO:teuthology.orchestra.run.smithi172.stdout: "blocklisted": false, 2024-04-14T08:33:03.403 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs" 2024-04-14T08:33:03.403 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:33:03.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:03.408 DEBUG:teuthology.orchestra.run.smithi172:> stat --file-system '--printf=%T 2024-04-14T08:33:03.409 DEBUG:teuthology.orchestra.run.smithi172:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-14T08:33:03.424 INFO:teuthology.orchestra.run.smithi172.stdout:fuseblk 2024-04-14T08:33:03.424 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-14T08:33:03.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:03.424 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-14T08:33:03.492 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-14T08:33:03.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:03.603 INFO:teuthology.orchestra.run.smithi172.stdout:# file: . 2024-04-14T08:33:03.604 INFO:teuthology.orchestra.run.smithi172.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-14T08:33:03.604 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:03.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:03.610 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:33:03.610 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:03.626 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:03.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.766+0000 7f3407c0f640 1 Processor -- start 2024-04-14T08:33:03.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.766+0000 7f3407c0f640 1 -- start start 2024-04-14T08:33:03.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.767+0000 7f3407c0f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f34000cc8f0 0x7f34000d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:03.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.767+0000 7f3407c0f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f340015c780 con 0x7f34001530d0 2024-04-14T08:33:03.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.767+0000 7f3407c0f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f340015ca00 con 0x7f34000d0580 2024-04-14T08:33:03.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.767+0000 7f3407c0f640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f340015cce0 con 0x7f34000cc8f0 2024-04-14T08:33:03.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.767+0000 7f34059ab640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f34001530d0 0x7f34000cfe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40230/0 (socket says 172.21.15.172:40230) 2024-04-14T08:33:03.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.767+0000 7f34059ab640 1 -- 172.21.15.172:0/1085730980 learned_addr learned my addr 172.21.15.172:0/1085730980 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:03.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.767+0000 7f34061ac640 1 --2- 172.21.15.172:0/1085730980 >> v2:172.21.15.177:3300/0 conn(0x7f34000cc8f0 0x7f34000d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:03.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.767+0000 7f34049a9640 1 -- 172.21.15.172:0/1085730980 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 699019543 0 0) 0x7f33fc001110 con 0x7f34001530d0 2024-04-14T08:33:03.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.767+0000 7f34049a9640 1 -- 172.21.15.172:0/1085730980 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33ec003520 con 0x7f34001530d0 2024-04-14T08:33:03.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.767+0000 7f34061ac640 1 -- 172.21.15.172:0/1085730980 >> v1:172.21.15.177:6789/0 conn(0x7f34000d0580 legacy=0x7f34000d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:03.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.767+0000 7f34061ac640 1 -- 172.21.15.172:0/1085730980 >> v1:172.21.15.172:6789/0 conn(0x7f34001530d0 legacy=0x7f34000cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:03.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.767+0000 7f34061ac640 1 -- 172.21.15.172:0/1085730980 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340015cfc0 con 0x7f34000cc8f0 2024-04-14T08:33:03.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.768+0000 7f34061ac640 1 --2- 172.21.15.172:0/1085730980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34000cc8f0 0x7f34000d17a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009d90 tx=0x7f33f802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a89e6ffb511c89b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:03.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.768+0000 7f34049a9640 1 -- 172.21.15.172:0/1085730980 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8039070 con 0x7f34000cc8f0 2024-04-14T08:33:03.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.768+0000 7f34049a9640 1 -- 172.21.15.172:0/1085730980 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f802cd00 con 0x7f34000cc8f0 2024-04-14T08:33:03.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.768+0000 7f34049a9640 1 -- 172.21.15.172:0/1085730980 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8034660 con 0x7f34000cc8f0 2024-04-14T08:33:03.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.769+0000 7f3407c0f640 1 -- 172.21.15.172:0/1085730980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34000cc8f0 msgr2=0x7f34000d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:03.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.769+0000 7f3407c0f640 1 --2- 172.21.15.172:0/1085730980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34000cc8f0 0x7f34000d17a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009d90 tx=0x7f33f802bf50 comp rx=0 tx=0).stop 2024-04-14T08:33:03.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.769+0000 7f3407c0f640 1 -- 172.21.15.172:0/1085730980 shutdown_connections 2024-04-14T08:33:03.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.769+0000 7f3407c0f640 1 --2- 172.21.15.172:0/1085730980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34000cc8f0 0x7f34000d17a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:03.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.769+0000 7f3407c0f640 1 -- 172.21.15.172:0/1085730980 >> 172.21.15.172:0/1085730980 conn(0x7f340006f820 msgr2=0x7f340006fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:03.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.769+0000 7f3407c0f640 1 -- 172.21.15.172:0/1085730980 shutdown_connections 2024-04-14T08:33:03.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.769+0000 7f3407c0f640 1 -- 172.21.15.172:0/1085730980 wait complete. 2024-04-14T08:33:03.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.770+0000 7f3407c0f640 1 Processor -- start 2024-04-14T08:33:03.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.770+0000 7f3407c0f640 1 -- start start 2024-04-14T08:33:03.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f3407c0f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34000cc8f0 0x7f34000b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:03.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f3407c0f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34000d0580 0x7f340014e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:03.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f3407c0f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34001530d0 0x7f34000cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:03.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f3407c0f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34001534b0 con 0x7f34000cc8f0 2024-04-14T08:33:03.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f3407c0f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f34000d0c10 con 0x7f34000d0580 2024-04-14T08:33:03.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f3407c0f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3400153630 con 0x7f34001530d0 2024-04-14T08:33:03.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f34059ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34000cc8f0 0x7f34000b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:03.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f34059ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34000cc8f0 0x7f34000b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38916/0 (socket says 172.21.15.172:38916) 2024-04-14T08:33:03.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f34051aa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34000d0580 0x7f340014e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:03.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f34059ab640 1 -- 172.21.15.172:0/3491699634 learned_addr learned my addr 172.21.15.172:0/3491699634 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:03.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f34061ac640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34001530d0 0x7f34000cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:03.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f34059ab640 1 -- 172.21.15.172:0/3491699634 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34001530d0 msgr2=0x7f34000cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:03.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f34059ab640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34001530d0 0x7f34000cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:03.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f34059ab640 1 -- 172.21.15.172:0/3491699634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34000d0580 msgr2=0x7f340014e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:03.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f34059ab640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34000d0580 0x7f340014e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:03.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f34059ab640 1 -- 172.21.15.172:0/3491699634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34000cc190 con 0x7f34000cc8f0 2024-04-14T08:33:03.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f34059ab640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34000cc8f0 0x7f34000b4110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00c430 tx=0x7f33fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:03.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f33f67fc640 1 -- 172.21.15.172:0/3491699634 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc017070 con 0x7f34000cc8f0 2024-04-14T08:33:03.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f33f67fc640 1 -- 172.21.15.172:0/3491699634 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33fc005ce0 con 0x7f34000cc8f0 2024-04-14T08:33:03.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f3407c0f640 1 -- 172.21.15.172:0/3491699634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34000b3350 con 0x7f34000cc8f0 2024-04-14T08:33:03.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f33f67fc640 1 -- 172.21.15.172:0/3491699634 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0072d0 con 0x7f34000cc8f0 2024-04-14T08:33:03.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.771+0000 7f3407c0f640 1 -- 172.21.15.172:0/3491699634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34000b3910 con 0x7f34000cc8f0 2024-04-14T08:33:03.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.772+0000 7f3407c0f640 1 -- 172.21.15.172:0/3491699634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3400068520 con 0x7f34000cc8f0 2024-04-14T08:33:03.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.773+0000 7f33f67fc640 1 -- 172.21.15.172:0/3491699634 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33fc01a070 con 0x7f34000cc8f0 2024-04-14T08:33:03.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.773+0000 7f33f67fc640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33e406c8f0 0x7f33e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:03.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.773+0000 7f33f67fc640 1 -- 172.21.15.172:0/3491699634 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f33fc08b100 con 0x7f34000cc8f0 2024-04-14T08:33:03.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.773+0000 7f34051aa640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33e406c8f0 0x7f33e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:03.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.774+0000 7f34051aa640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33e406c8f0 0x7f33e406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3400150e00 tx=0x7f33f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:03.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.778+0000 7f33f67fc640 1 -- 172.21.15.172:0/3491699634 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f340015c780 con 0x7f34000cc8f0 2024-04-14T08:33:03.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.956+0000 7f3407c0f640 1 -- 172.21.15.172:0/3491699634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3400071840 con 0x7f34000cc8f0 2024-04-14T08:33:03.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.957+0000 7f33f67fc640 1 -- 172.21.15.172:0/3491699634 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7f33fc058ba0 con 0x7f34000cc8f0 2024-04-14T08:33:03.958 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:03.958 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.959+0000 7f3407c0f640 1 -- 172.21.15.172:0/3491699634 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33e406c8f0 msgr2=0x7f33e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.959+0000 7f3407c0f640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33e406c8f0 0x7f33e406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3400150e00 tx=0x7f33f0009040 comp rx=0 tx=0).stop 2024-04-14T08:33:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.959+0000 7f3407c0f640 1 -- 172.21.15.172:0/3491699634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34000cc8f0 msgr2=0x7f34000b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.959+0000 7f3407c0f640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34000cc8f0 0x7f34000b4110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00c430 tx=0x7f33fc00c8f0 comp rx=0 tx=0).stop 2024-04-14T08:33:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.959+0000 7f3407c0f640 1 -- 172.21.15.172:0/3491699634 shutdown_connections 2024-04-14T08:33:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.959+0000 7f3407c0f640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33e406c8f0 0x7f33e406edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.959+0000 7f3407c0f640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34001530d0 0x7f34000cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.959+0000 7f3407c0f640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34000d0580 0x7f340014e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.959+0000 7f3407c0f640 1 --2- 172.21.15.172:0/3491699634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34000cc8f0 0x7f34000b4110 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.959+0000 7f3407c0f640 1 -- 172.21.15.172:0/3491699634 >> 172.21.15.172:0/3491699634 conn(0x7f340006f820 msgr2=0x7f3400151fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.959+0000 7f3407c0f640 1 -- 172.21.15.172:0/3491699634 shutdown_connections 2024-04-14T08:33:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:03.960+0000 7f3407c0f640 1 -- 172.21.15.172:0/3491699634 wait complete. 2024-04-14T08:33:03.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:03.972 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:04.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.074+0000 7fc912988640 1 Processor -- start 2024-04-14T08:33:04.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.074+0000 7fc912988640 1 -- start start 2024-04-14T08:33:04.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.075+0000 7fc912988640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc90c15c5f0 0x7fc90c0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.075+0000 7fc912988640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90c0cb570 con 0x7fc90c06b520 2024-04-14T08:33:04.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.075+0000 7fc912988640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90c0cb810 con 0x7fc90c070910 2024-04-14T08:33:04.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.075+0000 7fc912988640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc90c0cbaf0 con 0x7fc90c15c5f0 2024-04-14T08:33:04.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.075+0000 7fc90b7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc90c15c5f0 0x7fc90c0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.075+0000 7fc90bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc90c06b520 0x7fc90c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40246/0 (socket says 172.21.15.172:40246) 2024-04-14T08:33:04.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.075+0000 7fc90b7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc90c15c5f0 0x7fc90c0703d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38922/0 (socket says 172.21.15.172:38922) 2024-04-14T08:33:04.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.075+0000 7fc90bfff640 1 -- 172.21.15.172:0/4229203682 learned_addr learned my addr 172.21.15.172:0/4229203682 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:04.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.076+0000 7fc90b7fe640 1 -- 172.21.15.172:0/4229203682 >> v1:172.21.15.177:6789/0 conn(0x7fc90c070910 legacy=0x7fc90c0cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.076+0000 7fc90b7fe640 1 -- 172.21.15.172:0/4229203682 >> v1:172.21.15.172:6789/0 conn(0x7fc90c06b520 legacy=0x7fc90c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.076+0000 7fc90affd640 1 -- 172.21.15.172:0/4229203682 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2579780580 0 0) 0x7fc90c0cb810 con 0x7fc90c070910 2024-04-14T08:33:04.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.076+0000 7fc90b7fe640 1 -- 172.21.15.172:0/4229203682 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c0cbdd0 con 0x7fc90c15c5f0 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.076+0000 7fc90b7fe640 1 --2- 172.21.15.172:0/4229203682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc90c15c5f0 0x7fc90c0703d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc9000097a0 tx=0x7fc90002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08b6a0c9f947384 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.076+0000 7fc90affd640 1 -- 172.21.15.172:0/4229203682 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900037070 con 0x7fc90c15c5f0 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.077+0000 7fc90affd640 1 -- 172.21.15.172:0/4229203682 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc900030710 con 0x7fc90c15c5f0 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.077+0000 7fc90affd640 1 -- 172.21.15.172:0/4229203682 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900030ae0 con 0x7fc90c15c5f0 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.077+0000 7fc912988640 1 -- 172.21.15.172:0/4229203682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc90c15c5f0 msgr2=0x7fc90c0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.077+0000 7fc912988640 1 --2- 172.21.15.172:0/4229203682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc90c15c5f0 0x7fc90c0703d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc9000097a0 tx=0x7fc90002b790 comp rx=0 tx=0).stop 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.077+0000 7fc912988640 1 -- 172.21.15.172:0/4229203682 shutdown_connections 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.077+0000 7fc912988640 1 --2- 172.21.15.172:0/4229203682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc90c15c5f0 0x7fc90c0703d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.077+0000 7fc912988640 1 -- 172.21.15.172:0/4229203682 >> 172.21.15.172:0/4229203682 conn(0x7fc90c0d0f20 msgr2=0x7fc90c0d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.077+0000 7fc912988640 1 -- 172.21.15.172:0/4229203682 shutdown_connections 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.077+0000 7fc912988640 1 -- 172.21.15.172:0/4229203682 wait complete. 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.078+0000 7fc912988640 1 Processor -- start 2024-04-14T08:33:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.079+0000 7fc912988640 1 -- start start 2024-04-14T08:33:04.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.079+0000 7fc912988640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90c06b520 0x7fc90c0b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.079+0000 7fc912988640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc90c070910 0x7fc90c159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.079+0000 7fc912988640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90c15c5f0 0x7fc90c1f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.079+0000 7fc912988640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c0ccce0 con 0x7fc90c070910 2024-04-14T08:33:04.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.079+0000 7fc912988640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c06e2a0 con 0x7fc90c15c5f0 2024-04-14T08:33:04.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.079+0000 7fc912988640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc90c0cce60 con 0x7fc90c06b520 2024-04-14T08:33:04.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.079+0000 7fc90b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc90c070910 0x7fc90c159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.079+0000 7fc90b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc90c070910 0x7fc90c159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38936/0 (socket says 172.21.15.172:38936) 2024-04-14T08:33:04.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.079+0000 7fc90b7fe640 1 -- 172.21.15.172:0/3171390099 learned_addr learned my addr 172.21.15.172:0/3171390099 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:04.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.079+0000 7fc90bfff640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90c06b520 0x7fc90c0b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.079+0000 7fc910f25640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90c15c5f0 0x7fc90c1f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.080+0000 7fc90b7fe640 1 -- 172.21.15.172:0/3171390099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90c06b520 msgr2=0x7fc90c0b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.080+0000 7fc90b7fe640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90c06b520 0x7fc90c0b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.080+0000 7fc90b7fe640 1 -- 172.21.15.172:0/3171390099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90c15c5f0 msgr2=0x7fc90c1f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.080+0000 7fc90b7fe640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90c15c5f0 0x7fc90c1f1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.080+0000 7fc90b7fe640 1 -- 172.21.15.172:0/3171390099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c0c9af0 con 0x7fc90c070910 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.080+0000 7fc90b7fe640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc90c070910 0x7fc90c159340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc9000096f0 tx=0x7fc9000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.080+0000 7fc908ff9640 1 -- 172.21.15.172:0/3171390099 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900042070 con 0x7fc90c070910 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.080+0000 7fc908ff9640 1 -- 172.21.15.172:0/3171390099 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc900032a40 con 0x7fc90c070910 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.080+0000 7fc908ff9640 1 -- 172.21.15.172:0/3171390099 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900037040 con 0x7fc90c070910 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.080+0000 7fc912988640 1 -- 172.21.15.172:0/3171390099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c159880 con 0x7fc90c070910 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.080+0000 7fc912988640 1 -- 172.21.15.172:0/3171390099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c0c9d90 con 0x7fc90c070910 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.081+0000 7fc912988640 1 -- 172.21.15.172:0/3171390099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90c075490 con 0x7fc90c070910 2024-04-14T08:33:04.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.081+0000 7fc908ff9640 1 -- 172.21.15.172:0/3171390099 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc900045070 con 0x7fc90c070910 2024-04-14T08:33:04.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.082+0000 7fc908ff9640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc8f806c920 0x7fc8f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.082+0000 7fc908ff9640 1 -- 172.21.15.172:0/3171390099 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc9000ada40 con 0x7fc90c070910 2024-04-14T08:33:04.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.082+0000 7fc90bfff640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc8f806c920 0x7fc8f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.082+0000 7fc90bfff640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc8f806c920 0x7fc8f806ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000f60 tx=0x7fc8fc007e20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:04.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.086+0000 7fc908ff9640 1 -- 172.21.15.172:0/3171390099 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc90007b570 con 0x7fc90c070910 2024-04-14T08:33:04.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.265+0000 7fc912988640 1 -- 172.21.15.172:0/3171390099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc90c0c4420 con 0x7fc90c070910 2024-04-14T08:33:04.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.265+0000 7fc908ff9640 1 -- 172.21.15.172:0/3171390099 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fc90007f870 con 0x7fc90c070910 2024-04-14T08:33:04.267 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:04.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.267+0000 7fc912988640 1 -- 172.21.15.172:0/3171390099 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc8f806c920 msgr2=0x7fc8f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.267+0000 7fc912988640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc8f806c920 0x7fc8f806ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000f60 tx=0x7fc8fc007e20 comp rx=0 tx=0).stop 2024-04-14T08:33:04.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.267+0000 7fc912988640 1 -- 172.21.15.172:0/3171390099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc90c070910 msgr2=0x7fc90c159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.267+0000 7fc912988640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc90c070910 0x7fc90c159340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc9000096f0 tx=0x7fc9000046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.267+0000 7fc912988640 1 -- 172.21.15.172:0/3171390099 shutdown_connections 2024-04-14T08:33:04.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.267+0000 7fc912988640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc8f806c920 0x7fc8f806ede0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.267+0000 7fc912988640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc90c15c5f0 0x7fc90c1f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.267+0000 7fc912988640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc90c070910 0x7fc90c159340 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.267+0000 7fc912988640 1 --2- 172.21.15.172:0/3171390099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc90c06b520 0x7fc90c0b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.267+0000 7fc912988640 1 -- 172.21.15.172:0/3171390099 >> 172.21.15.172:0/3171390099 conn(0x7fc90c0d0f20 msgr2=0x7fc90c0b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:04.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.268+0000 7fc912988640 1 -- 172.21.15.172:0/3171390099 shutdown_connections 2024-04-14T08:33:04.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.268+0000 7fc912988640 1 -- 172.21.15.172:0/3171390099 wait complete. 2024-04-14T08:33:04.279 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:04.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.382+0000 7f99958de640 1 Processor -- start 2024-04-14T08:33:04.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.382+0000 7f99958de640 1 -- start start 2024-04-14T08:33:04.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f99958de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999015e430 0x7f9990162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f99958de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9990162540 0x7f99901629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f99958de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99900b3a30 0x7f99900b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f99958de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f99900cefb0 con 0x7f99900b3a30 2024-04-14T08:33:04.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f99958de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f99900cf290 con 0x7f9990162540 2024-04-14T08:33:04.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f99958de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f99900cf570 con 0x7f999015e430 2024-04-14T08:33:04.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f998f7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99900b3a30 0x7f99900b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f998f7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99900b3a30 0x7f99900b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38952/0 (socket says 172.21.15.172:38952) 2024-04-14T08:33:04.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f998f7fe640 1 -- 172.21.15.172:0/1382738853 learned_addr learned my addr 172.21.15.172:0/1382738853 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:04.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f998e7fc640 1 --2- 172.21.15.172:0/1382738853 >> v2:172.21.15.177:3300/0 conn(0x7f9990162540 0x7f99901629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f998effd640 1 --2- 172.21.15.172:0/1382738853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999015e430 0x7f9990162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f998f7fe640 1 -- 172.21.15.172:0/1382738853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999015e430 msgr2=0x7f9990162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:04.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f998f7fe640 1 --2- 172.21.15.172:0/1382738853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999015e430 0x7f9990162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f998f7fe640 1 -- 172.21.15.172:0/1382738853 >> v2:172.21.15.177:3300/0 conn(0x7f9990162540 msgr2=0x7f99901629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:04.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f998f7fe640 1 --2- 172.21.15.172:0/1382738853 >> v2:172.21.15.177:3300/0 conn(0x7f9990162540 0x7f99901629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f998f7fe640 1 -- 172.21.15.172:0/1382738853 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99900cf850 con 0x7f99900b3a30 2024-04-14T08:33:04.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.383+0000 7f998e7fc640 1 --2- 172.21.15.172:0/1382738853 >> v2:172.21.15.177:3300/0 conn(0x7f9990162540 0x7f99901629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:04.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.384+0000 7f998f7fe640 1 --2- 172.21.15.172:0/1382738853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99900b3a30 0x7f99900b3e60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f999008c720 tx=0x7f998802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96414694d62ae5a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:04.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.384+0000 7f998dffb640 1 -- 172.21.15.172:0/1382738853 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998802ca70 con 0x7f99900b3a30 2024-04-14T08:33:04.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.384+0000 7f998dffb640 1 -- 172.21.15.172:0/1382738853 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f998802cc10 con 0x7f99900b3a30 2024-04-14T08:33:04.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.384+0000 7f998dffb640 1 -- 172.21.15.172:0/1382738853 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99880343f0 con 0x7f99900b3a30 2024-04-14T08:33:04.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.385+0000 7f99958de640 1 -- 172.21.15.172:0/1382738853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99900b3a30 msgr2=0x7f99900b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.385+0000 7f99958de640 1 --2- 172.21.15.172:0/1382738853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99900b3a30 0x7f99900b3e60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f999008c720 tx=0x7f998802bed0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.385+0000 7f99958de640 1 -- 172.21.15.172:0/1382738853 shutdown_connections 2024-04-14T08:33:04.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.385+0000 7f99958de640 1 --2- 172.21.15.172:0/1382738853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99900b3a30 0x7f99900b3e60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.385+0000 7f99958de640 1 --2- 172.21.15.172:0/1382738853 >> v2:172.21.15.177:3300/0 conn(0x7f9990162540 0x7f99901629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.385+0000 7f99958de640 1 --2- 172.21.15.172:0/1382738853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999015e430 0x7f9990162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.385+0000 7f99958de640 1 -- 172.21.15.172:0/1382738853 >> 172.21.15.172:0/1382738853 conn(0x7f999009d560 msgr2=0x7f99900d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:04.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.385+0000 7f99958de640 1 -- 172.21.15.172:0/1382738853 shutdown_connections 2024-04-14T08:33:04.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.385+0000 7f99958de640 1 -- 172.21.15.172:0/1382738853 wait complete. 2024-04-14T08:33:04.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.386+0000 7f99958de640 1 Processor -- start 2024-04-14T08:33:04.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.386+0000 7f99958de640 1 -- start start 2024-04-14T08:33:04.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f99958de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99900b3a30 0x7f99900cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f99958de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999015e430 0x7f99901574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f99958de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9990162540 0x7f99900c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f99958de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f99900d4e20 con 0x7f9990162540 2024-04-14T08:33:04.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f99958de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f999015e810 con 0x7f99900b3a30 2024-04-14T08:33:04.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f99958de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f99900d0050 con 0x7f999015e430 2024-04-14T08:33:04.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f998f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9990162540 0x7f99900c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f998f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9990162540 0x7f99900c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38958/0 (socket says 172.21.15.172:38958) 2024-04-14T08:33:04.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f998f7fe640 1 -- 172.21.15.172:0/2827248627 learned_addr learned my addr 172.21.15.172:0/2827248627 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:04.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f998e7fc640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999015e430 0x7f99901574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f998effd640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99900b3a30 0x7f99900cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f998f7fe640 1 -- 172.21.15.172:0/2827248627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999015e430 msgr2=0x7f99901574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f998f7fe640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999015e430 0x7f99901574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f998f7fe640 1 -- 172.21.15.172:0/2827248627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99900b3a30 msgr2=0x7f99900cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f998f7fe640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99900b3a30 0x7f99900cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f998f7fe640 1 -- 172.21.15.172:0/2827248627 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99900b8e60 con 0x7f9990162540 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f998f7fe640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9990162540 0x7f99900c8220 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9988036040 tx=0x7f9988004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f9977fff640 1 -- 172.21.15.172:0/2827248627 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988043020 con 0x7f9990162540 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f9977fff640 1 -- 172.21.15.172:0/2827248627 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f998803e450 con 0x7f9990162540 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.387+0000 7f99958de640 1 -- 172.21.15.172:0/2827248627 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99900b9140 con 0x7f9990162540 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.388+0000 7f9977fff640 1 -- 172.21.15.172:0/2827248627 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988009510 con 0x7f9990162540 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.388+0000 7f99958de640 1 -- 172.21.15.172:0/2827248627 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99900c96d0 con 0x7f9990162540 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.388+0000 7f99958de640 1 -- 172.21.15.172:0/2827248627 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9990068590 con 0x7f9990162540 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.389+0000 7f9977fff640 1 -- 172.21.15.172:0/2827248627 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9988046020 con 0x7f9990162540 2024-04-14T08:33:04.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.389+0000 7f9977fff640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f997c06c8f0 0x7f997c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.389+0000 7f9977fff640 1 -- 172.21.15.172:0/2827248627 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f99880ae430 con 0x7f9990162540 2024-04-14T08:33:04.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.389+0000 7f998effd640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f997c06c8f0 0x7f997c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.390+0000 7f998effd640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f997c06c8f0 0x7f997c06edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f99900d2de0 tx=0x7f9980009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:04.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.394+0000 7f9977fff640 1 -- 172.21.15.172:0/2827248627 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f998804d030 con 0x7f9990162540 2024-04-14T08:33:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.570+0000 7f99958de640 1 -- 172.21.15.172:0/2827248627 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f99900c98a0 con 0x7f9990162540 2024-04-14T08:33:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.571+0000 7f9977fff640 1 -- 172.21.15.172:0/2827248627 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f998807bed0 con 0x7f9990162540 2024-04-14T08:33:04.571 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.573+0000 7f99958de640 1 -- 172.21.15.172:0/2827248627 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f997c06c8f0 msgr2=0x7f997c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.573+0000 7f99958de640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f997c06c8f0 0x7f997c06edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f99900d2de0 tx=0x7f9980009190 comp rx=0 tx=0).stop 2024-04-14T08:33:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.573+0000 7f99958de640 1 -- 172.21.15.172:0/2827248627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9990162540 msgr2=0x7f99900c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.573+0000 7f99958de640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9990162540 0x7f99900c8220 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9988036040 tx=0x7f9988004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.573+0000 7f99958de640 1 -- 172.21.15.172:0/2827248627 shutdown_connections 2024-04-14T08:33:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.573+0000 7f99958de640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f997c06c8f0 0x7f997c06edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.573+0000 7f99958de640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9990162540 0x7f99900c8220 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.573+0000 7f99958de640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999015e430 0x7f99901574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.573+0000 7f99958de640 1 --2- 172.21.15.172:0/2827248627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99900b3a30 0x7f99900cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.573+0000 7f99958de640 1 -- 172.21.15.172:0/2827248627 >> 172.21.15.172:0/2827248627 conn(0x7f999009d560 msgr2=0x7f999006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.573+0000 7f99958de640 1 -- 172.21.15.172:0/2827248627 shutdown_connections 2024-04-14T08:33:04.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.573+0000 7f99958de640 1 -- 172.21.15.172:0/2827248627 wait complete. 2024-04-14T08:33:04.585 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:04.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.687+0000 7fc8c7b1e640 1 Processor -- start 2024-04-14T08:33:04.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.687+0000 7fc8c7b1e640 1 -- start start 2024-04-14T08:33:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.688+0000 7fc8c7b1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8c00ccca0 0x7fc8c006fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.688+0000 7fc8c7b1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc8c0073090 0x7fc8c00b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.688+0000 7fc8c7b1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c00ceee0 con 0x7fc8c006b520 2024-04-14T08:33:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.688+0000 7fc8c7b1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c0073520 con 0x7fc8c00ccca0 2024-04-14T08:33:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.688+0000 7fc8c7b1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c0070150 con 0x7fc8c0073090 2024-04-14T08:33:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.688+0000 7fc8c58ba640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8c00ccca0 0x7fc8c006fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.688+0000 7fc8c58ba640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8c00ccca0 0x7fc8c006fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38960/0 (socket says 172.21.15.172:38960) 2024-04-14T08:33:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.688+0000 7fc8c58ba640 1 -- 172.21.15.172:0/2342556065 learned_addr learned my addr 172.21.15.172:0/2342556065 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.688+0000 7fc8c50b9640 1 --2- 172.21.15.172:0/2342556065 >> v2:172.21.15.177:3300/0 conn(0x7fc8c0073090 0x7fc8c00b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.689+0000 7fc8c58ba640 1 -- 172.21.15.172:0/2342556065 >> v2:172.21.15.177:3300/0 conn(0x7fc8c0073090 msgr2=0x7fc8c00b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.689+0000 7fc8c58ba640 1 --2- 172.21.15.172:0/2342556065 >> v2:172.21.15.177:3300/0 conn(0x7fc8c0073090 0x7fc8c00b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.689+0000 7fc8c58ba640 1 -- 172.21.15.172:0/2342556065 >> v1:172.21.15.177:6789/0 conn(0x7fc8c006b520 legacy=0x7fc8c006b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.689+0000 7fc8c58ba640 1 -- 172.21.15.172:0/2342556065 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c00722d0 con 0x7fc8c00ccca0 2024-04-14T08:33:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.689+0000 7fc8c50b9640 1 --2- 172.21.15.172:0/2342556065 >> v2:172.21.15.177:3300/0 conn(0x7fc8c0073090 0x7fc8c00b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.689+0000 7fc8c58ba640 1 --2- 172.21.15.172:0/2342556065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c00ccca0 0x7fc8c006fb70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0068f70 tx=0x7fc8bc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc686059dfa3b11 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:04.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.689+0000 7fc8c48b8640 1 -- 172.21.15.172:0/2342556065 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc02ca80 con 0x7fc8c00ccca0 2024-04-14T08:33:04.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.689+0000 7fc8c48b8640 1 -- 172.21.15.172:0/2342556065 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8bc02cc20 con 0x7fc8c00ccca0 2024-04-14T08:33:04.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.689+0000 7fc8c48b8640 1 -- 172.21.15.172:0/2342556065 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc0343f0 con 0x7fc8c00ccca0 2024-04-14T08:33:04.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.690+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/2342556065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c00ccca0 msgr2=0x7fc8c006fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.690+0000 7fc8c7b1e640 1 --2- 172.21.15.172:0/2342556065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c00ccca0 0x7fc8c006fb70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0068f70 tx=0x7fc8bc02bee0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.690+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/2342556065 shutdown_connections 2024-04-14T08:33:04.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.690+0000 7fc8c7b1e640 1 --2- 172.21.15.172:0/2342556065 >> v2:172.21.15.177:3300/0 conn(0x7fc8c0073090 0x7fc8c00b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.690+0000 7fc8c7b1e640 1 --2- 172.21.15.172:0/2342556065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c00ccca0 0x7fc8c006fb70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.690+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/2342556065 >> 172.21.15.172:0/2342556065 conn(0x7fc8c00cda00 msgr2=0x7fc8c00cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:04.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.690+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/2342556065 shutdown_connections 2024-04-14T08:33:04.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.690+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/2342556065 wait complete. 2024-04-14T08:33:04.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.691+0000 7fc8c7b1e640 1 Processor -- start 2024-04-14T08:33:04.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.691+0000 7fc8c7b1e640 1 -- start start 2024-04-14T08:33:04.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c7b1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8c006b520 0x7fc8c01554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c7b1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c0073090 0x7fc8c00d0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c7b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8c00ccca0 0x7fc8c00d0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c7b1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c00cd080 con 0x7fc8c0073090 2024-04-14T08:33:04.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c7b1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c0072890 con 0x7fc8c006b520 2024-04-14T08:33:04.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c7b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c006e150 con 0x7fc8c00ccca0 2024-04-14T08:33:04.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c50b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c0073090 0x7fc8c00d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c50b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c0073090 0x7fc8c00d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38966/0 (socket says 172.21.15.172:38966) 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c50b9640 1 -- 172.21.15.172:0/1547524535 learned_addr learned my addr 172.21.15.172:0/1547524535 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c58ba640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8c006b520 0x7fc8c01554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c60bb640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8c00ccca0 0x7fc8c00d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c50b9640 1 -- 172.21.15.172:0/1547524535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8c00ccca0 msgr2=0x7fc8c00d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c50b9640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8c00ccca0 0x7fc8c00d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c50b9640 1 -- 172.21.15.172:0/1547524535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8c006b520 msgr2=0x7fc8c01554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c50b9640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8c006b520 0x7fc8c01554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.692+0000 7fc8c50b9640 1 -- 172.21.15.172:0/1547524535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c00b18a0 con 0x7fc8c0073090 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.693+0000 7fc8c50b9640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c0073090 0x7fc8c00d0340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ebe0 tx=0x7fc8b000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.693+0000 7fc8b67fc640 1 -- 172.21.15.172:0/1547524535 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0017070 con 0x7fc8c0073090 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.693+0000 7fc8b67fc640 1 -- 172.21.15.172:0/1547524535 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b000cc30 con 0x7fc8c0073090 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.693+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/1547524535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c00b1b80 con 0x7fc8c0073090 2024-04-14T08:33:04.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.693+0000 7fc8b67fc640 1 -- 172.21.15.172:0/1547524535 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0013470 con 0x7fc8c0073090 2024-04-14T08:33:04.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.693+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/1547524535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c00b0b50 con 0x7fc8c0073090 2024-04-14T08:33:04.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.694+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/1547524535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c00b20a0 con 0x7fc8c0073090 2024-04-14T08:33:04.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.694+0000 7fc8b67fc640 1 -- 172.21.15.172:0/1547524535 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8b001a070 con 0x7fc8c0073090 2024-04-14T08:33:04.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.695+0000 7fc8b67fc640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc8a406c8f0 0x7fc8a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.695+0000 7fc8b67fc640 1 -- 172.21.15.172:0/1547524535 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc8b008bdd0 con 0x7fc8c0073090 2024-04-14T08:33:04.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.695+0000 7fc8c58ba640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc8a406c8f0 0x7fc8a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.695+0000 7fc8c58ba640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc8a406c8f0 0x7fc8a406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc035000 tx=0x7fc8bc030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:04.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.699+0000 7fc8b67fc640 1 -- 172.21.15.172:0/1547524535 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc8b00597f0 con 0x7fc8c0073090 2024-04-14T08:33:04.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.873+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/1547524535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc8c0068590 con 0x7fc8c0073090 2024-04-14T08:33:04.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.874+0000 7fc8b67fc640 1 -- 172.21.15.172:0/1547524535 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fc8b005daf0 con 0x7fc8c0073090 2024-04-14T08:33:04.874 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:04.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.876+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/1547524535 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc8a406c8f0 msgr2=0x7fc8a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.876+0000 7fc8c7b1e640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc8a406c8f0 0x7fc8a406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc035000 tx=0x7fc8bc030000 comp rx=0 tx=0).stop 2024-04-14T08:33:04.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.876+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/1547524535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c0073090 msgr2=0x7fc8c00d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.876+0000 7fc8c7b1e640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c0073090 0x7fc8c00d0340 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ebe0 tx=0x7fc8b000c520 comp rx=0 tx=0).stop 2024-04-14T08:33:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.876+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/1547524535 shutdown_connections 2024-04-14T08:33:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.876+0000 7fc8c7b1e640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc8a406c8f0 0x7fc8a406edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.876+0000 7fc8c7b1e640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8c00ccca0 0x7fc8c00d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.876+0000 7fc8c7b1e640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c0073090 0x7fc8c00d0340 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.876+0000 7fc8c7b1e640 1 --2- 172.21.15.172:0/1547524535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8c006b520 0x7fc8c01554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.876+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/1547524535 >> 172.21.15.172:0/1547524535 conn(0x7fc8c00cda00 msgr2=0x7fc8c0160390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.876+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/1547524535 shutdown_connections 2024-04-14T08:33:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.876+0000 7fc8c7b1e640 1 -- 172.21.15.172:0/1547524535 wait complete. 2024-04-14T08:33:04.887 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:04.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.990+0000 7fe57dc1c640 1 Processor -- start 2024-04-14T08:33:04.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.990+0000 7fe57dc1c640 1 -- start start 2024-04-14T08:33:04.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe57dc1c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5780ccca0 0x7fe57806fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe57dc1c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe578073090 0x7fe5780b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe57dc1c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe57806b520 0x7fe57806b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe57dc1c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe57806be90 con 0x7fe57806b520 2024-04-14T08:33:04.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe57dc1c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe5780734f0 con 0x7fe578073090 2024-04-14T08:33:04.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe57dc1c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe5780ceee0 con 0x7fe5780ccca0 2024-04-14T08:33:04.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe577fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe57806b520 0x7fe57806b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe577fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe57806b520 0x7fe57806b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38974/0 (socket says 172.21.15.172:38974) 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe577fff640 1 -- 172.21.15.172:0/196445601 learned_addr learned my addr 172.21.15.172:0/196445601 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe5777fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5780ccca0 0x7fe57806fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe56effd640 1 --2- 172.21.15.172:0/196445601 >> v2:172.21.15.177:3300/0 conn(0x7fe578073090 0x7fe5780b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe577fff640 1 -- 172.21.15.172:0/196445601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5780ccca0 msgr2=0x7fe57806fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe577fff640 1 --2- 172.21.15.172:0/196445601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5780ccca0 0x7fe57806fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe577fff640 1 -- 172.21.15.172:0/196445601 >> v2:172.21.15.177:3300/0 conn(0x7fe578073090 msgr2=0x7fe5780b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe577fff640 1 --2- 172.21.15.172:0/196445601 >> v2:172.21.15.177:3300/0 conn(0x7fe578073090 0x7fe5780b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.991+0000 7fe577fff640 1 -- 172.21.15.172:0/196445601 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe578070150 con 0x7fe57806b520 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.992+0000 7fe577fff640 1 --2- 172.21.15.172:0/196445601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe57806b520 0x7fe57806b950 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe578073fd0 tx=0x7fe57002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ec90b5466f6c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.992+0000 7fe576ffd640 1 -- 172.21.15.172:0/196445601 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57002c9e0 con 0x7fe57806b520 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.992+0000 7fe576ffd640 1 -- 172.21.15.172:0/196445601 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57002cb80 con 0x7fe57806b520 2024-04-14T08:33:04.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.992+0000 7fe576ffd640 1 -- 172.21.15.172:0/196445601 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5700343f0 con 0x7fe57806b520 2024-04-14T08:33:04.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.992+0000 7fe57dc1c640 1 -- 172.21.15.172:0/196445601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe57806b520 msgr2=0x7fe57806b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.992+0000 7fe57dc1c640 1 --2- 172.21.15.172:0/196445601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe57806b520 0x7fe57806b950 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe578073fd0 tx=0x7fe57002be40 comp rx=0 tx=0).stop 2024-04-14T08:33:04.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.993+0000 7fe57dc1c640 1 -- 172.21.15.172:0/196445601 shutdown_connections 2024-04-14T08:33:04.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.993+0000 7fe57dc1c640 1 --2- 172.21.15.172:0/196445601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe57806b520 0x7fe57806b950 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.993+0000 7fe57dc1c640 1 --2- 172.21.15.172:0/196445601 >> v2:172.21.15.177:3300/0 conn(0x7fe578073090 0x7fe5780b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.993+0000 7fe57dc1c640 1 --2- 172.21.15.172:0/196445601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5780ccca0 0x7fe57806fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.993+0000 7fe57dc1c640 1 -- 172.21.15.172:0/196445601 >> 172.21.15.172:0/196445601 conn(0x7fe5780cda00 msgr2=0x7fe5780cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:04.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.993+0000 7fe57dc1c640 1 -- 172.21.15.172:0/196445601 shutdown_connections 2024-04-14T08:33:04.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.993+0000 7fe57dc1c640 1 -- 172.21.15.172:0/196445601 wait complete. 2024-04-14T08:33:04.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.994+0000 7fe57dc1c640 1 Processor -- start 2024-04-14T08:33:04.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.994+0000 7fe57dc1c640 1 -- start start 2024-04-14T08:33:04.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe57dc1c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe57806b520 0x7fe57814e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe57dc1c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe578073090 0x7fe5780b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe57dc1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5780ccca0 0x7fe5780b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:04.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe57dc1c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe578072890 con 0x7fe5780ccca0 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe57dc1c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5780725b0 con 0x7fe578073090 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe57dc1c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe5780cd080 con 0x7fe57806b520 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe577fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5780ccca0 0x7fe5780b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe577fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5780ccca0 0x7fe5780b65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38986/0 (socket says 172.21.15.172:38986) 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe577fff640 1 -- 172.21.15.172:0/1036605563 learned_addr learned my addr 172.21.15.172:0/1036605563 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe5777fe640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe57806b520 0x7fe57814e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe56effd640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe578073090 0x7fe5780b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe577fff640 1 -- 172.21.15.172:0/1036605563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe57806b520 msgr2=0x7fe57814e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe577fff640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe57806b520 0x7fe57814e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe577fff640 1 -- 172.21.15.172:0/1036605563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe578073090 msgr2=0x7fe5780b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe577fff640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe578073090 0x7fe5780b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe577fff640 1 -- 172.21.15.172:0/1036605563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5780c8d10 con 0x7fe5780ccca0 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe577fff640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5780ccca0 0x7fe5780b65b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe570036040 tx=0x7fe570004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:04.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe574ff9640 1 -- 172.21.15.172:0/1036605563 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe570043020 con 0x7fe5780ccca0 2024-04-14T08:33:05.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe574ff9640 1 -- 172.21.15.172:0/1036605563 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57003e450 con 0x7fe5780ccca0 2024-04-14T08:33:05.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe57dc1c640 1 -- 172.21.15.172:0/1036605563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5780c8ff0 con 0x7fe5780ccca0 2024-04-14T08:33:05.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe574ff9640 1 -- 172.21.15.172:0/1036605563 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe570009510 con 0x7fe5780ccca0 2024-04-14T08:33:05.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.995+0000 7fe57dc1c640 1 -- 172.21.15.172:0/1036605563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57814f2e0 con 0x7fe5780ccca0 2024-04-14T08:33:05.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.996+0000 7fe57dc1c640 1 -- 172.21.15.172:0/1036605563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe578068590 con 0x7fe5780ccca0 2024-04-14T08:33:05.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.997+0000 7fe574ff9640 1 -- 172.21.15.172:0/1036605563 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe570046020 con 0x7fe5780ccca0 2024-04-14T08:33:05.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.997+0000 7fe574ff9640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe55806c920 0x7fe55806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.997+0000 7fe574ff9640 1 -- 172.21.15.172:0/1036605563 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe5700ae3c0 con 0x7fe5780ccca0 2024-04-14T08:33:05.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.997+0000 7fe5777fe640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe55806c920 0x7fe55806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:04.998+0000 7fe5777fe640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe55806c920 0x7fe55806ede0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe568009aa0 tx=0x7fe568009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:05.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.002+0000 7fe574ff9640 1 -- 172.21.15.172:0/1036605563 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe57004d030 con 0x7fe5780ccca0 2024-04-14T08:33:05.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.177+0000 7fe57dc1c640 1 -- 172.21.15.172:0/1036605563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe57806c120 con 0x7fe5780ccca0 2024-04-14T08:33:05.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.177+0000 7fe574ff9640 1 -- 172.21.15.172:0/1036605563 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fe57007bef0 con 0x7fe5780ccca0 2024-04-14T08:33:05.178 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:05.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.179+0000 7fe57dc1c640 1 -- 172.21.15.172:0/1036605563 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe55806c920 msgr2=0x7fe55806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.179+0000 7fe57dc1c640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe55806c920 0x7fe55806ede0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe568009aa0 tx=0x7fe568009190 comp rx=0 tx=0).stop 2024-04-14T08:33:05.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.179+0000 7fe57dc1c640 1 -- 172.21.15.172:0/1036605563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5780ccca0 msgr2=0x7fe5780b65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.179+0000 7fe57dc1c640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5780ccca0 0x7fe5780b65b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe570036040 tx=0x7fe570004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.179+0000 7fe57dc1c640 1 -- 172.21.15.172:0/1036605563 shutdown_connections 2024-04-14T08:33:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.179+0000 7fe57dc1c640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe55806c920 0x7fe55806ede0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.179+0000 7fe57dc1c640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5780ccca0 0x7fe5780b65b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.179+0000 7fe57dc1c640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe578073090 0x7fe5780b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.179+0000 7fe57dc1c640 1 --2- 172.21.15.172:0/1036605563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe57806b520 0x7fe57814e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.179+0000 7fe57dc1c640 1 -- 172.21.15.172:0/1036605563 >> 172.21.15.172:0/1036605563 conn(0x7fe5780cda00 msgr2=0x7fe5780b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.179+0000 7fe57dc1c640 1 -- 172.21.15.172:0/1036605563 shutdown_connections 2024-04-14T08:33:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.179+0000 7fe57dc1c640 1 -- 172.21.15.172:0/1036605563 wait complete. 2024-04-14T08:33:05.191 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:33:05.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.294+0000 7ff962508640 1 Processor -- start 2024-04-14T08:33:05.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.294+0000 7ff962508640 1 -- start start 2024-04-14T08:33:05.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff962508640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff95c0ce200 0x7ff95c1603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff962508640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff95c160930 0x7ff95c072450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff962508640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff95c15cf10 con 0x7ff95c159c20 2024-04-14T08:33:05.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff962508640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff95c15fc80 con 0x7ff95c0ce200 2024-04-14T08:33:05.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff962508640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff95c15ff60 con 0x7ff95c160930 2024-04-14T08:33:05.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff95b7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff95c0ce200 0x7ff95c1603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff95b7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff95c0ce200 0x7ff95c1603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38996/0 (socket says 172.21.15.172:38996) 2024-04-14T08:33:05.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff95b7fe640 1 -- 172.21.15.172:0/3720370251 learned_addr learned my addr 172.21.15.172:0/3720370251 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:05.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff960aa5640 1 --2- 172.21.15.172:0/3720370251 >> v2:172.21.15.177:3300/0 conn(0x7ff95c160930 0x7ff95c072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff95b7fe640 1 -- 172.21.15.172:0/3720370251 >> v2:172.21.15.177:3300/0 conn(0x7ff95c160930 msgr2=0x7ff95c072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:05.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff95b7fe640 1 --2- 172.21.15.172:0/3720370251 >> v2:172.21.15.177:3300/0 conn(0x7ff95c160930 0x7ff95c072450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff95b7fe640 1 -- 172.21.15.172:0/3720370251 >> v1:172.21.15.177:6789/0 conn(0x7ff95c159c20 legacy=0x7ff95c15c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.295+0000 7ff95b7fe640 1 -- 172.21.15.172:0/3720370251 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c15d1f0 con 0x7ff95c0ce200 2024-04-14T08:33:05.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.296+0000 7ff95b7fe640 1 --2- 172.21.15.172:0/3720370251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c0ce200 0x7ff95c1603f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0ccfd0 tx=0x7ff95002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac022ad7548f666c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:05.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.296+0000 7ff95affd640 1 -- 172.21.15.172:0/3720370251 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950034070 con 0x7ff95c0ce200 2024-04-14T08:33:05.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.296+0000 7ff95affd640 1 -- 172.21.15.172:0/3720370251 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff950030860 con 0x7ff95c0ce200 2024-04-14T08:33:05.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.296+0000 7ff95affd640 1 -- 172.21.15.172:0/3720370251 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950030c30 con 0x7ff95c0ce200 2024-04-14T08:33:05.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.297+0000 7ff962508640 1 -- 172.21.15.172:0/3720370251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c0ce200 msgr2=0x7ff95c1603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.297+0000 7ff962508640 1 --2- 172.21.15.172:0/3720370251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c0ce200 0x7ff95c1603f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0ccfd0 tx=0x7ff95002b9a0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.297+0000 7ff962508640 1 -- 172.21.15.172:0/3720370251 shutdown_connections 2024-04-14T08:33:05.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.297+0000 7ff962508640 1 --2- 172.21.15.172:0/3720370251 >> v2:172.21.15.177:3300/0 conn(0x7ff95c160930 0x7ff95c072450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.297+0000 7ff962508640 1 --2- 172.21.15.172:0/3720370251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c0ce200 0x7ff95c1603f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.297+0000 7ff962508640 1 -- 172.21.15.172:0/3720370251 >> 172.21.15.172:0/3720370251 conn(0x7ff95c06b520 msgr2=0x7ff95c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:05.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.297+0000 7ff962508640 1 -- 172.21.15.172:0/3720370251 shutdown_connections 2024-04-14T08:33:05.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.297+0000 7ff962508640 1 -- 172.21.15.172:0/3720370251 wait complete. 2024-04-14T08:33:05.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.298+0000 7ff962508640 1 Processor -- start 2024-04-14T08:33:05.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.298+0000 7ff962508640 1 -- start start 2024-04-14T08:33:05.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff962508640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c0ce200 0x7ff95c1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff962508640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff95c159c20 0x7ff95c15b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff962508640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff95c150df0 0x7ff95c151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff962508640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c072990 con 0x7ff95c0ce200 2024-04-14T08:33:05.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff962508640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c15a000 con 0x7ff95c150df0 2024-04-14T08:33:05.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff962508640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff95c072b10 con 0x7ff95c159c20 2024-04-14T08:33:05.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff95bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c0ce200 0x7ff95c1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff95bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c0ce200 0x7ff95c1549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38998/0 (socket says 172.21.15.172:38998) 2024-04-14T08:33:05.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff95bfff640 1 -- 172.21.15.172:0/2510946020 learned_addr learned my addr 172.21.15.172:0/2510946020 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:05.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff960aa5640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff95c150df0 0x7ff95c151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff95b7fe640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff95c159c20 0x7ff95c15b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff95bfff640 1 -- 172.21.15.172:0/2510946020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff95c159c20 msgr2=0x7ff95c15b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff95bfff640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff95c159c20 0x7ff95c15b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff95bfff640 1 -- 172.21.15.172:0/2510946020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff95c150df0 msgr2=0x7ff95c151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff95bfff640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff95c150df0 0x7ff95c151250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff95bfff640 1 -- 172.21.15.172:0/2510946020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c140130 con 0x7ff95c0ce200 2024-04-14T08:33:05.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff95bfff640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c0ce200 0x7ff95c1549b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00c460 tx=0x7ff94c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:05.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.299+0000 7ff958ff9640 1 -- 172.21.15.172:0/2510946020 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c0052d0 con 0x7ff95c0ce200 2024-04-14T08:33:05.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.300+0000 7ff958ff9640 1 -- 172.21.15.172:0/2510946020 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff94c012630 con 0x7ff95c0ce200 2024-04-14T08:33:05.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.300+0000 7ff962508640 1 -- 172.21.15.172:0/2510946020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95c140410 con 0x7ff95c0ce200 2024-04-14T08:33:05.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.300+0000 7ff958ff9640 1 -- 172.21.15.172:0/2510946020 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c0054b0 con 0x7ff95c0ce200 2024-04-14T08:33:05.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.300+0000 7ff962508640 1 -- 172.21.15.172:0/2510946020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95c1409d0 con 0x7ff95c0ce200 2024-04-14T08:33:05.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.300+0000 7ff962508640 1 -- 172.21.15.172:0/2510946020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff95c068590 con 0x7ff95c0ce200 2024-04-14T08:33:05.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.301+0000 7ff958ff9640 1 -- 172.21.15.172:0/2510946020 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff94c012a20 con 0x7ff95c0ce200 2024-04-14T08:33:05.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.301+0000 7ff958ff9640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff94806c8f0 0x7ff94806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.301+0000 7ff958ff9640 1 -- 172.21.15.172:0/2510946020 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff94c08b920 con 0x7ff95c0ce200 2024-04-14T08:33:05.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.301+0000 7ff95b7fe640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff94806c8f0 0x7ff94806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.302+0000 7ff95b7fe640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff94806c8f0 0x7ff94806edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff950036000 tx=0x7ff950008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.306+0000 7ff958ff9640 1 -- 172.21.15.172:0/2510946020 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff94c0593c0 con 0x7ff95c0ce200 2024-04-14T08:33:05.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.492+0000 7ff962508640 1 -- 172.21.15.172:0/2510946020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff95c1593d0 con 0x7ff95c0ce200 2024-04-14T08:33:05.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.493+0000 7ff958ff9640 1 -- 172.21.15.172:0/2510946020 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7ff94c05d6c0 con 0x7ff95c0ce200 2024-04-14T08:33:05.493 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:33:05.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.495+0000 7ff962508640 1 -- 172.21.15.172:0/2510946020 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff94806c8f0 msgr2=0x7ff94806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.495+0000 7ff962508640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff94806c8f0 0x7ff94806edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff950036000 tx=0x7ff950008030 comp rx=0 tx=0).stop 2024-04-14T08:33:05.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.495+0000 7ff962508640 1 -- 172.21.15.172:0/2510946020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c0ce200 msgr2=0x7ff95c1549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.495+0000 7ff962508640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c0ce200 0x7ff95c1549b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00c460 tx=0x7ff94c00c920 comp rx=0 tx=0).stop 2024-04-14T08:33:05.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.495+0000 7ff962508640 1 -- 172.21.15.172:0/2510946020 shutdown_connections 2024-04-14T08:33:05.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.495+0000 7ff962508640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff94806c8f0 0x7ff94806edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.495+0000 7ff962508640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff95c150df0 0x7ff95c151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.495+0000 7ff962508640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff95c159c20 0x7ff95c15b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.495+0000 7ff962508640 1 --2- 172.21.15.172:0/2510946020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff95c0ce200 0x7ff95c1549b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.495+0000 7ff962508640 1 -- 172.21.15.172:0/2510946020 >> 172.21.15.172:0/2510946020 conn(0x7ff95c06b520 msgr2=0x7ff95c158990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:05.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.495+0000 7ff962508640 1 -- 172.21.15.172:0/2510946020 shutdown_connections 2024-04-14T08:33:05.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.495+0000 7ff962508640 1 -- 172.21.15.172:0/2510946020 wait complete. 2024-04-14T08:33:05.506 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:33:05.506 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:05.522 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:05.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.663+0000 7f5905c36640 1 Processor -- start 2024-04-14T08:33:05.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.664+0000 7f5905c36640 1 -- start start 2024-04-14T08:33:05.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.664+0000 7f5905c36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f590015c710 0x7f59000d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.664+0000 7f5905c36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59000cde10 0x7f59000ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.664+0000 7f5905c36640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59000aef60 con 0x7f59000ced80 2024-04-14T08:33:05.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.664+0000 7f5905c36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f59000aed10 con 0x7f590015c710 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.664+0000 7f5905c36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f59000af890 con 0x7f59000cde10 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.664+0000 7f58ff7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f590015c710 0x7f59000d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.664+0000 7f58fffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f59000ced80 0x7f59000ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40254/0 (socket says 172.21.15.172:40254) 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.665+0000 7f58fffff640 1 -- 172.21.15.172:0/4193046079 learned_addr learned my addr 172.21.15.172:0/4193046079 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.665+0000 7f58f6ffd640 1 --2- 172.21.15.172:0/4193046079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59000cde10 0x7f59000ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.665+0000 7f58f6ffd640 1 -- 172.21.15.172:0/4193046079 >> v2:172.21.15.172:3300/0 conn(0x7f590015c710 msgr2=0x7f59000d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.665+0000 7f58f6ffd640 1 --2- 172.21.15.172:0/4193046079 >> v2:172.21.15.172:3300/0 conn(0x7f590015c710 0x7f59000d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.665+0000 7f58f6ffd640 1 -- 172.21.15.172:0/4193046079 >> v1:172.21.15.172:6789/0 conn(0x7f59000ced80 legacy=0x7f59000ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.665+0000 7f58feffd640 1 -- 172.21.15.172:0/4193046079 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2347206068 0 0) 0x7f59000aef60 con 0x7f59000ced80 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.665+0000 7f58f6ffd640 1 -- 172.21.15.172:0/4193046079 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59000af240 con 0x7f59000cde10 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.665+0000 7f58ff7fe640 1 --2- 172.21.15.172:0/4193046079 >> v2:172.21.15.172:3300/0 conn(0x7f590015c710 0x7f59000d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.666+0000 7f58f6ffd640 1 --2- 172.21.15.172:0/4193046079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59000cde10 0x7f59000ce840 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f58e80097a0 tx=0x7f58e802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63ee3558c82963d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.666+0000 7f58feffd640 1 -- 172.21.15.172:0/4193046079 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8035020 con 0x7f59000cde10 2024-04-14T08:33:05.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.666+0000 7f58feffd640 1 -- 172.21.15.172:0/4193046079 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e8031830 con 0x7f59000cde10 2024-04-14T08:33:05.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.666+0000 7f58feffd640 1 -- 172.21.15.172:0/4193046079 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8031c00 con 0x7f59000cde10 2024-04-14T08:33:05.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.666+0000 7f5905c36640 1 -- 172.21.15.172:0/4193046079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59000cde10 msgr2=0x7f59000ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.666+0000 7f5905c36640 1 --2- 172.21.15.172:0/4193046079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59000cde10 0x7f59000ce840 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f58e80097a0 tx=0x7f58e802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.666+0000 7f5905c36640 1 -- 172.21.15.172:0/4193046079 shutdown_connections 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.666+0000 7f5905c36640 1 --2- 172.21.15.172:0/4193046079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59000cde10 0x7f59000ce840 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.666+0000 7f5905c36640 1 --2- 172.21.15.172:0/4193046079 >> v2:172.21.15.172:3300/0 conn(0x7f590015c710 0x7f59000d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.666+0000 7f5905c36640 1 -- 172.21.15.172:0/4193046079 >> 172.21.15.172:0/4193046079 conn(0x7f590008c000 msgr2=0x7f59000d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.666+0000 7f5905c36640 1 -- 172.21.15.172:0/4193046079 shutdown_connections 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.666+0000 7f5905c36640 1 -- 172.21.15.172:0/4193046079 wait complete. 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.667+0000 7f5905c36640 1 Processor -- start 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.668+0000 7f5905c36640 1 -- start start 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.668+0000 7f5905c36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59000cde10 0x7f5900158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.668+0000 7f5905c36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59000ced80 0x7f59000b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.668+0000 7f5905c36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f590015c710 0x7f5900159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.668+0000 7f5905c36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f59000d0e90 con 0x7f59000cde10 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.668+0000 7f5905c36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f590015caf0 con 0x7f590015c710 2024-04-14T08:33:05.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.668+0000 7f5905c36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f59000d1010 con 0x7f59000ced80 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59000cde10 0x7f5900158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58f6ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59000ced80 0x7f59000b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58fffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f590015c710 0x7f5900159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59000cde10 0x7f5900158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39018/0 (socket says 172.21.15.172:39018) 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58ff7fe640 1 -- 172.21.15.172:0/2554984143 learned_addr learned my addr 172.21.15.172:0/2554984143 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58ff7fe640 1 -- 172.21.15.172:0/2554984143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59000ced80 msgr2=0x7f59000b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58ff7fe640 1 --2- 172.21.15.172:0/2554984143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59000ced80 0x7f59000b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58ff7fe640 1 -- 172.21.15.172:0/2554984143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f590015c710 msgr2=0x7f5900159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58ff7fe640 1 --2- 172.21.15.172:0/2554984143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f590015c710 0x7f5900159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58ff7fe640 1 -- 172.21.15.172:0/2554984143 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59000b5fe0 con 0x7f59000cde10 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58ff7fe640 1 --2- 172.21.15.172:0/2554984143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59000cde10 0x7f5900158750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58f000e960 tx=0x7f58f000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58fcff9640 1 -- 172.21.15.172:0/2554984143 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0017070 con 0x7f59000cde10 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58fcff9640 1 -- 172.21.15.172:0/2554984143 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f000ca00 con 0x7f59000cde10 2024-04-14T08:33:05.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f5905c36640 1 -- 172.21.15.172:0/2554984143 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59000b62c0 con 0x7f59000cde10 2024-04-14T08:33:05.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f58fcff9640 1 -- 172.21.15.172:0/2554984143 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f000cdd0 con 0x7f59000cde10 2024-04-14T08:33:05.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.669+0000 7f5905c36640 1 -- 172.21.15.172:0/2554984143 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59000b6880 con 0x7f59000cde10 2024-04-14T08:33:05.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.670+0000 7f5905c36640 1 -- 172.21.15.172:0/2554984143 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5900075490 con 0x7f59000cde10 2024-04-14T08:33:05.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.671+0000 7f58fcff9640 1 -- 172.21.15.172:0/2554984143 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58f001a070 con 0x7f59000cde10 2024-04-14T08:33:05.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.671+0000 7f58fcff9640 1 --2- 172.21.15.172:0/2554984143 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58e006c8f0 0x7f58e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.671+0000 7f58f6ffd640 1 --2- 172.21.15.172:0/2554984143 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58e006c8f0 0x7f58e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.671+0000 7f58fcff9640 1 -- 172.21.15.172:0/2554984143 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f58f008c050 con 0x7f59000cde10 2024-04-14T08:33:05.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.672+0000 7f58f6ffd640 1 --2- 172.21.15.172:0/2554984143 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58e006c8f0 0x7f58e006edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58e8036000 tx=0x7f58e803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:05.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.676+0000 7f58fcff9640 1 -- 172.21.15.172:0/2554984143 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f58f0059af0 con 0x7f59000cde10 2024-04-14T08:33:05.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.851+0000 7f5905c36640 1 -- 172.21.15.172:0/2554984143 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59000b31a0 con 0x7f59000cde10 2024-04-14T08:33:05.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.852+0000 7f58fcff9640 1 -- 172.21.15.172:0/2554984143 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7f58f005ddf0 con 0x7f59000cde10 2024-04-14T08:33:05.853 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:05.853 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:05.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.854+0000 7f5905c36640 1 -- 172.21.15.172:0/2554984143 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58e006c8f0 msgr2=0x7f58e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.854+0000 7f5905c36640 1 --2- 172.21.15.172:0/2554984143 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58e006c8f0 0x7f58e006edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58e8036000 tx=0x7f58e803d040 comp rx=0 tx=0).stop 2024-04-14T08:33:05.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.854+0000 7f5905c36640 1 -- 172.21.15.172:0/2554984143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59000cde10 msgr2=0x7f5900158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.854+0000 7f5905c36640 1 --2- 172.21.15.172:0/2554984143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59000cde10 0x7f5900158750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58f000e960 tx=0x7f58f000ee20 comp rx=0 tx=0).stop 2024-04-14T08:33:05.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.854+0000 7f5905c36640 1 -- 172.21.15.172:0/2554984143 shutdown_connections 2024-04-14T08:33:05.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.854+0000 7f5905c36640 1 --2- 172.21.15.172:0/2554984143 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58e006c8f0 0x7f58e006edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.854+0000 7f5905c36640 1 --2- 172.21.15.172:0/2554984143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f590015c710 0x7f5900159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.854+0000 7f5905c36640 1 --2- 172.21.15.172:0/2554984143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59000ced80 0x7f59000b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.854+0000 7f5905c36640 1 --2- 172.21.15.172:0/2554984143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59000cde10 0x7f5900158750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.854+0000 7f5905c36640 1 -- 172.21.15.172:0/2554984143 >> 172.21.15.172:0/2554984143 conn(0x7f590008c000 msgr2=0x7f59000704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:05.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.855+0000 7f5905c36640 1 -- 172.21.15.172:0/2554984143 shutdown_connections 2024-04-14T08:33:05.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.855+0000 7f5905c36640 1 -- 172.21.15.172:0/2554984143 wait complete. 2024-04-14T08:33:05.866 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:05.867 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:05.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.969+0000 7f8398ad1640 1 Processor -- start 2024-04-14T08:33:05.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.969+0000 7f8398ad1640 1 -- start start 2024-04-14T08:33:05.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f8398ad1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f83940cfc40 0x7f8394166db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f8398ad1640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83940d0ea0 con 0x7f839415c1f0 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f8398ad1640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83940cda90 con 0x7f83940cd5f0 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f8398ad1640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f83940b4e50 con 0x7f83940cfc40 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f839259c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f839415c1f0 0x7f83940d0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40256/0 (socket says 172.21.15.172:40256) 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f839259c640 1 -- 172.21.15.172:0/3806309670 learned_addr learned my addr 172.21.15.172:0/3806309670 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f8392d9d640 1 --2- 172.21.15.172:0/3806309670 >> v2:172.21.15.172:3300/0 conn(0x7f83940cfc40 0x7f8394166db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f839159a640 1 -- 172.21.15.172:0/3806309670 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1652059425 0 0) 0x7f83940d0ea0 con 0x7f839415c1f0 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f839159a640 1 -- 172.21.15.172:0/3806309670 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f837c003520 con 0x7f839415c1f0 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f839159a640 1 -- 172.21.15.172:0/3806309670 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2133826376 0 0) 0x7f83940cda90 con 0x7f83940cd5f0 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f839159a640 1 -- 172.21.15.172:0/3806309670 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83940d0ea0 con 0x7f83940cd5f0 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f8392d9d640 1 -- 172.21.15.172:0/3806309670 >> v1:172.21.15.177:6789/0 conn(0x7f83940cd5f0 legacy=0x7f83940d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f8392d9d640 1 -- 172.21.15.172:0/3806309670 >> v1:172.21.15.172:6789/0 conn(0x7f839415c1f0 legacy=0x7f83940d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.970+0000 7f8392d9d640 1 -- 172.21.15.172:0/3806309670 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83940cdd70 con 0x7f83940cfc40 2024-04-14T08:33:05.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.971+0000 7f8392d9d640 1 --2- 172.21.15.172:0/3806309670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83940cfc40 0x7f8394166db0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f83840097a0 tx=0x7f838402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2da35ce5ca0046f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:05.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.971+0000 7f839159a640 1 -- 172.21.15.172:0/3806309670 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384035020 con 0x7f83940cfc40 2024-04-14T08:33:05.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.971+0000 7f839159a640 1 -- 172.21.15.172:0/3806309670 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8384031830 con 0x7f83940cfc40 2024-04-14T08:33:05.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.971+0000 7f839159a640 1 -- 172.21.15.172:0/3806309670 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384031c00 con 0x7f83940cfc40 2024-04-14T08:33:05.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.972+0000 7f8398ad1640 1 -- 172.21.15.172:0/3806309670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83940cfc40 msgr2=0x7f8394166db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.972+0000 7f8398ad1640 1 --2- 172.21.15.172:0/3806309670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83940cfc40 0x7f8394166db0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f83840097a0 tx=0x7f838402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.972+0000 7f8398ad1640 1 -- 172.21.15.172:0/3806309670 shutdown_connections 2024-04-14T08:33:05.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.972+0000 7f8398ad1640 1 --2- 172.21.15.172:0/3806309670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83940cfc40 0x7f8394166db0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.972+0000 7f8398ad1640 1 -- 172.21.15.172:0/3806309670 >> 172.21.15.172:0/3806309670 conn(0x7f83940b3af0 msgr2=0x7f83940b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:05.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.972+0000 7f8398ad1640 1 -- 172.21.15.172:0/3806309670 shutdown_connections 2024-04-14T08:33:05.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.972+0000 7f8398ad1640 1 -- 172.21.15.172:0/3806309670 wait complete. 2024-04-14T08:33:05.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.973+0000 7f8398ad1640 1 Processor -- start 2024-04-14T08:33:05.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.973+0000 7f8398ad1640 1 -- start start 2024-04-14T08:33:05.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8398ad1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83940cd5f0 0x7f8394159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8398ad1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83940cfc40 0x7f83940cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8398ad1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f839415c1f0 0x7f839415e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8398ad1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f839415c5d0 con 0x7f83940cfc40 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8398ad1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f83940b5380 con 0x7f83940cd5f0 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8398ad1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f83940b68b0 con 0x7f839415c1f0 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8391d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83940cfc40 0x7f83940cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8391d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83940cfc40 0x7f83940cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39034/0 (socket says 172.21.15.172:39034) 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8391d9b640 1 -- 172.21.15.172:0/2283226001 learned_addr learned my addr 172.21.15.172:0/2283226001 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f839259c640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83940cd5f0 0x7f8394159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8392d9d640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f839415c1f0 0x7f839415e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8391d9b640 1 -- 172.21.15.172:0/2283226001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f839415c1f0 msgr2=0x7f839415e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8391d9b640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f839415c1f0 0x7f839415e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8391d9b640 1 -- 172.21.15.172:0/2283226001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83940cd5f0 msgr2=0x7f8394159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8391d9b640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83940cd5f0 0x7f8394159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.974+0000 7f8391d9b640 1 -- 172.21.15.172:0/2283226001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83940ca470 con 0x7f83940cfc40 2024-04-14T08:33:05.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.975+0000 7f8391d9b640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83940cfc40 0x7f83940cb960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f838800c500 tx=0x7f838800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:05.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.975+0000 7f8382ffd640 1 -- 172.21.15.172:0/2283226001 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83880052d0 con 0x7f83940cfc40 2024-04-14T08:33:05.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.975+0000 7f8382ffd640 1 -- 172.21.15.172:0/2283226001 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8388005490 con 0x7f83940cfc40 2024-04-14T08:33:05.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.975+0000 7f8382ffd640 1 -- 172.21.15.172:0/2283226001 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388012870 con 0x7f83940cfc40 2024-04-14T08:33:05.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.975+0000 7f8398ad1640 1 -- 172.21.15.172:0/2283226001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83940ca750 con 0x7f83940cfc40 2024-04-14T08:33:05.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.975+0000 7f8398ad1640 1 -- 172.21.15.172:0/2283226001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83940cad10 con 0x7f83940cfc40 2024-04-14T08:33:05.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.976+0000 7f8398ad1640 1 -- 172.21.15.172:0/2283226001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8394067dd0 con 0x7f83940cfc40 2024-04-14T08:33:05.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.976+0000 7f8382ffd640 1 -- 172.21.15.172:0/2283226001 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8388012a10 con 0x7f83940cfc40 2024-04-14T08:33:05.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.977+0000 7f8382ffd640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f837406c920 0x7f837406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:05.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.977+0000 7f8382ffd640 1 -- 172.21.15.172:0/2283226001 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f838808b9a0 con 0x7f83940cfc40 2024-04-14T08:33:05.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.977+0000 7f839259c640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f837406c920 0x7f837406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:05.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.977+0000 7f839259c640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f837406c920 0x7f837406ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f838c001560 tx=0x7f838c0019f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:05.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:05.981+0000 7f8382ffd640 1 -- 172.21.15.172:0/2283226001 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8394067dd0 con 0x7f83940cfc40 2024-04-14T08:33:06.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.154+0000 7f8398ad1640 1 -- 172.21.15.172:0/2283226001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8394068590 con 0x7f83940cfc40 2024-04-14T08:33:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.155+0000 7f8382ffd640 1 -- 172.21.15.172:0/2283226001 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f8388059450 con 0x7f83940cfc40 2024-04-14T08:33:06.156 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.157+0000 7f8398ad1640 1 -- 172.21.15.172:0/2283226001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f837406c920 msgr2=0x7f837406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.157+0000 7f8398ad1640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f837406c920 0x7f837406ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f838c001560 tx=0x7f838c0019f0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.157+0000 7f8398ad1640 1 -- 172.21.15.172:0/2283226001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83940cfc40 msgr2=0x7f83940cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.157+0000 7f8398ad1640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83940cfc40 0x7f83940cb960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f838800c500 tx=0x7f838800c9c0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.157+0000 7f8398ad1640 1 -- 172.21.15.172:0/2283226001 shutdown_connections 2024-04-14T08:33:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.157+0000 7f8398ad1640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f837406c920 0x7f837406ede0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.157+0000 7f8398ad1640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f839415c1f0 0x7f839415e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.157+0000 7f8398ad1640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83940cfc40 0x7f83940cb960 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.157+0000 7f8398ad1640 1 --2- 172.21.15.172:0/2283226001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83940cd5f0 0x7f8394159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.157+0000 7f8398ad1640 1 -- 172.21.15.172:0/2283226001 >> 172.21.15.172:0/2283226001 conn(0x7f83940b3af0 msgr2=0x7f83940b75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.157+0000 7f8398ad1640 1 -- 172.21.15.172:0/2283226001 shutdown_connections 2024-04-14T08:33:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.157+0000 7f8398ad1640 1 -- 172.21.15.172:0/2283226001 wait complete. 2024-04-14T08:33:06.168 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:06.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.276+0000 7fe03f37b640 1 Processor -- start 2024-04-14T08:33:06.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.276+0000 7fe03f37b640 1 -- start start 2024-04-14T08:33:06.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03f37b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe03806d040 0x7fe0380b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03f37b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe03815e820 0x7fe03806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03f37b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe0380d0d30 0x7fe03809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03f37b640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe03806cb90 con 0x7fe03806d040 2024-04-14T08:33:06.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03f37b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe0380cc660 con 0x7fe0380d0d30 2024-04-14T08:33:06.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03f37b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe0380ccc20 con 0x7fe03815e820 2024-04-14T08:33:06.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03d117640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe03806d040 0x7fe0380b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03d117640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe03806d040 0x7fe0380b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39046/0 (socket says 172.21.15.172:39046) 2024-04-14T08:33:06.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03d117640 1 -- 172.21.15.172:0/466751394 learned_addr learned my addr 172.21.15.172:0/466751394 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:06.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03c916640 1 --2- 172.21.15.172:0/466751394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe03815e820 0x7fe03806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03d918640 1 --2- 172.21.15.172:0/466751394 >> v2:172.21.15.177:3300/0 conn(0x7fe0380d0d30 0x7fe03809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03d117640 1 -- 172.21.15.172:0/466751394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe03815e820 msgr2=0x7fe03806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:06.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03d117640 1 --2- 172.21.15.172:0/466751394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe03815e820 0x7fe03806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03d117640 1 -- 172.21.15.172:0/466751394 >> v2:172.21.15.177:3300/0 conn(0x7fe0380d0d30 msgr2=0x7fe03809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:06.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03d117640 1 --2- 172.21.15.172:0/466751394 >> v2:172.21.15.177:3300/0 conn(0x7fe0380d0d30 0x7fe03809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.277+0000 7fe03d117640 1 -- 172.21.15.172:0/466751394 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0380d12e0 con 0x7fe03806d040 2024-04-14T08:33:06.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.278+0000 7fe03d117640 1 --2- 172.21.15.172:0/466751394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe03806d040 0x7fe0380b4620 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe038073410 tx=0x7fe03402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774fecbee341eed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:06.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.278+0000 7fe02ffff640 1 -- 172.21.15.172:0/466751394 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034035020 con 0x7fe03806d040 2024-04-14T08:33:06.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.278+0000 7fe02ffff640 1 -- 172.21.15.172:0/466751394 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe034031870 con 0x7fe03806d040 2024-04-14T08:33:06.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.278+0000 7fe02ffff640 1 -- 172.21.15.172:0/466751394 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034031c40 con 0x7fe03806d040 2024-04-14T08:33:06.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.278+0000 7fe03f37b640 1 -- 172.21.15.172:0/466751394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe03806d040 msgr2=0x7fe0380b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.278+0000 7fe03f37b640 1 --2- 172.21.15.172:0/466751394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe03806d040 0x7fe0380b4620 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe038073410 tx=0x7fe03402b910 comp rx=0 tx=0).stop 2024-04-14T08:33:06.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.278+0000 7fe03f37b640 1 -- 172.21.15.172:0/466751394 shutdown_connections 2024-04-14T08:33:06.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.278+0000 7fe03f37b640 1 --2- 172.21.15.172:0/466751394 >> v2:172.21.15.177:3300/0 conn(0x7fe0380d0d30 0x7fe03809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.278+0000 7fe03f37b640 1 --2- 172.21.15.172:0/466751394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe03815e820 0x7fe03806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.278+0000 7fe03f37b640 1 --2- 172.21.15.172:0/466751394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe03806d040 0x7fe0380b4620 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.278+0000 7fe03f37b640 1 -- 172.21.15.172:0/466751394 >> 172.21.15.172:0/466751394 conn(0x7fe0380b3b00 msgr2=0x7fe0380b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:06.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.279+0000 7fe03f37b640 1 -- 172.21.15.172:0/466751394 shutdown_connections 2024-04-14T08:33:06.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.279+0000 7fe03f37b640 1 -- 172.21.15.172:0/466751394 wait complete. 2024-04-14T08:33:06.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03f37b640 1 Processor -- start 2024-04-14T08:33:06.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03f37b640 1 -- start start 2024-04-14T08:33:06.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03f37b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe03806d040 0x7fe03809d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03f37b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0380d0d30 0x7fe03809f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03f37b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe03815e820 0x7fe0381eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03f37b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe0380cc8b0 con 0x7fe0380d0d30 2024-04-14T08:33:06.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03f37b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe0380d1580 con 0x7fe03815e820 2024-04-14T08:33:06.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03f37b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe03809afb0 con 0x7fe03806d040 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03c916640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0380d0d30 0x7fe03809f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03c916640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0380d0d30 0x7fe03809f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39062/0 (socket says 172.21.15.172:39062) 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03c916640 1 -- 172.21.15.172:0/1259985342 learned_addr learned my addr 172.21.15.172:0/1259985342 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03d117640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe03806d040 0x7fe03809d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.280+0000 7fe03d918640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe03815e820 0x7fe0381eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.281+0000 7fe03c916640 1 -- 172.21.15.172:0/1259985342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe03806d040 msgr2=0x7fe03809d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.281+0000 7fe03c916640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe03806d040 0x7fe03809d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.281+0000 7fe03c916640 1 -- 172.21.15.172:0/1259985342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe03815e820 msgr2=0x7fe0381eabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.281+0000 7fe03c916640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe03815e820 0x7fe0381eabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.281+0000 7fe03c916640 1 -- 172.21.15.172:0/1259985342 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0381f8760 con 0x7fe0380d0d30 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.281+0000 7fe03c916640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0380d0d30 0x7fe03809f7f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe02800e870 tx=0x7fe02800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.281+0000 7fe02dffb640 1 -- 172.21.15.172:0/1259985342 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028017070 con 0x7fe0380d0d30 2024-04-14T08:33:06.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.281+0000 7fe02dffb640 1 -- 172.21.15.172:0/1259985342 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe02800c940 con 0x7fe0380d0d30 2024-04-14T08:33:06.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.281+0000 7fe03f37b640 1 -- 172.21.15.172:0/1259985342 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0380b7c50 con 0x7fe0380d0d30 2024-04-14T08:33:06.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.281+0000 7fe02dffb640 1 -- 172.21.15.172:0/1259985342 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02800cd30 con 0x7fe0380d0d30 2024-04-14T08:33:06.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.281+0000 7fe03f37b640 1 -- 172.21.15.172:0/1259985342 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03809cfc0 con 0x7fe0380d0d30 2024-04-14T08:33:06.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.282+0000 7fe03f37b640 1 -- 172.21.15.172:0/1259985342 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe038068590 con 0x7fe0380d0d30 2024-04-14T08:33:06.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.283+0000 7fe02dffb640 1 -- 172.21.15.172:0/1259985342 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe02801a070 con 0x7fe0380d0d30 2024-04-14T08:33:06.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.283+0000 7fe02dffb640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe01c06c8f0 0x7fe01c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.283+0000 7fe02dffb640 1 -- 172.21.15.172:0/1259985342 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe02808bcb0 con 0x7fe0380d0d30 2024-04-14T08:33:06.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.283+0000 7fe03d117640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe01c06c8f0 0x7fe01c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.284+0000 7fe03d117640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe01c06c8f0 0x7fe01c06edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe034036000 tx=0x7fe03403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:06.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.288+0000 7fe02dffb640 1 -- 172.21.15.172:0/1259985342 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe028059750 con 0x7fe0380d0d30 2024-04-14T08:33:06.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.464+0000 7fe03f37b640 1 -- 172.21.15.172:0/1259985342 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe03809fed0 con 0x7fe0380d0d30 2024-04-14T08:33:06.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.464+0000 7fe02dffb640 1 -- 172.21.15.172:0/1259985342 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fe02805da50 con 0x7fe0380d0d30 2024-04-14T08:33:06.465 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:06.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.466+0000 7fe03f37b640 1 -- 172.21.15.172:0/1259985342 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe01c06c8f0 msgr2=0x7fe01c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.466+0000 7fe03f37b640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe01c06c8f0 0x7fe01c06edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe034036000 tx=0x7fe03403c040 comp rx=0 tx=0).stop 2024-04-14T08:33:06.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.466+0000 7fe03f37b640 1 -- 172.21.15.172:0/1259985342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0380d0d30 msgr2=0x7fe03809f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.466+0000 7fe03f37b640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0380d0d30 0x7fe03809f7f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe02800e870 tx=0x7fe02800ed30 comp rx=0 tx=0).stop 2024-04-14T08:33:06.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.466+0000 7fe03f37b640 1 -- 172.21.15.172:0/1259985342 shutdown_connections 2024-04-14T08:33:06.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.466+0000 7fe03f37b640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe01c06c8f0 0x7fe01c06edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.466+0000 7fe03f37b640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe03815e820 0x7fe0381eabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.466+0000 7fe03f37b640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0380d0d30 0x7fe03809f7f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.466+0000 7fe03f37b640 1 --2- 172.21.15.172:0/1259985342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe03806d040 0x7fe03809d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.466+0000 7fe03f37b640 1 -- 172.21.15.172:0/1259985342 >> 172.21.15.172:0/1259985342 conn(0x7fe0380b3b00 msgr2=0x7fe03815ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:06.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.467+0000 7fe03f37b640 1 -- 172.21.15.172:0/1259985342 shutdown_connections 2024-04-14T08:33:06.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.467+0000 7fe03f37b640 1 -- 172.21.15.172:0/1259985342 wait complete. 2024-04-14T08:33:06.478 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:06.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.582+0000 7f30bd37b640 1 Processor -- start 2024-04-14T08:33:06.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.582+0000 7f30bd37b640 1 -- start start 2024-04-14T08:33:06.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.582+0000 7f30bd37b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30b815e430 0x7f30b8162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.582+0000 7f30bd37b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f30b80d2e00 0x7f30b81629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.582+0000 7f30bd37b640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30b80cefb0 con 0x7f30b8162540 2024-04-14T08:33:06.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.582+0000 7f30bd37b640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f30b80cf290 con 0x7f30b80d2e00 2024-04-14T08:33:06.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.582+0000 7f30bd37b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f30b80cf570 con 0x7f30b815e430 2024-04-14T08:33:06.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.583+0000 7f30b67fc640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f30b8162540 0x7f30b80b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40266/0 (socket says 172.21.15.172:40266) 2024-04-14T08:33:06.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.583+0000 7f30b67fc640 1 -- 172.21.15.172:0/861856703 learned_addr learned my addr 172.21.15.172:0/861856703 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:06.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.583+0000 7f30b77fe640 1 --2- 172.21.15.172:0/861856703 >> v2:172.21.15.172:3300/0 conn(0x7f30b80d2e00 0x7f30b81629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.583+0000 7f30b6ffd640 1 --2- 172.21.15.172:0/861856703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30b815e430 0x7f30b8162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.583+0000 7f30b77fe640 1 -- 172.21.15.172:0/861856703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30b815e430 msgr2=0x7f30b8162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:06.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.583+0000 7f30b77fe640 1 --2- 172.21.15.172:0/861856703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30b815e430 0x7f30b8162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.583+0000 7f30b77fe640 1 -- 172.21.15.172:0/861856703 >> v1:172.21.15.172:6789/0 conn(0x7f30b8162540 legacy=0x7f30b80b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.583+0000 7f30b77fe640 1 -- 172.21.15.172:0/861856703 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b80cf850 con 0x7f30b80d2e00 2024-04-14T08:33:06.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.584+0000 7f30b77fe640 1 --2- 172.21.15.172:0/861856703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b80d2e00 0x7f30b81629a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f30b0009d90 tx=0x7f30b002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6c0269d076ba90 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:06.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.584+0000 7f30b5ffb640 1 -- 172.21.15.172:0/861856703 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0039070 con 0x7f30b80d2e00 2024-04-14T08:33:06.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.584+0000 7f30b5ffb640 1 -- 172.21.15.172:0/861856703 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b002cc50 con 0x7f30b80d2e00 2024-04-14T08:33:06.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.584+0000 7f30b5ffb640 1 -- 172.21.15.172:0/861856703 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0034440 con 0x7f30b80d2e00 2024-04-14T08:33:06.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.584+0000 7f30bd37b640 1 -- 172.21.15.172:0/861856703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b80d2e00 msgr2=0x7f30b81629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.584+0000 7f30bd37b640 1 --2- 172.21.15.172:0/861856703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b80d2e00 0x7f30b81629a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f30b0009d90 tx=0x7f30b002bea0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.584+0000 7f30bd37b640 1 -- 172.21.15.172:0/861856703 shutdown_connections 2024-04-14T08:33:06.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.584+0000 7f30bd37b640 1 --2- 172.21.15.172:0/861856703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b80d2e00 0x7f30b81629a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.584+0000 7f30bd37b640 1 --2- 172.21.15.172:0/861856703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30b815e430 0x7f30b8162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.584+0000 7f30bd37b640 1 -- 172.21.15.172:0/861856703 >> 172.21.15.172:0/861856703 conn(0x7f30b809d560 msgr2=0x7f30b80d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:06.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.584+0000 7f30bd37b640 1 -- 172.21.15.172:0/861856703 shutdown_connections 2024-04-14T08:33:06.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.584+0000 7f30bd37b640 1 -- 172.21.15.172:0/861856703 wait complete. 2024-04-14T08:33:06.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.586+0000 7f30bd37b640 1 Processor -- start 2024-04-14T08:33:06.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.586+0000 7f30bd37b640 1 -- start start 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.586+0000 7f30bd37b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b80d2e00 0x7f30b8158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.586+0000 7f30bd37b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30b815e430 0x7f30b80b9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.586+0000 7f30bd37b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30b8162540 0x7f30b80b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.586+0000 7f30bd37b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30b80d4e20 con 0x7f30b80d2e00 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.586+0000 7f30bd37b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f30b815e810 con 0x7f30b8162540 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.586+0000 7f30bd37b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f30b80d0050 con 0x7f30b815e430 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.586+0000 7f30b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b80d2e00 0x7f30b8158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.586+0000 7f30b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b80d2e00 0x7f30b8158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39078/0 (socket says 172.21.15.172:39078) 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.586+0000 7f30b6ffd640 1 -- 172.21.15.172:0/457279474 learned_addr learned my addr 172.21.15.172:0/457279474 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.586+0000 7f30b67fc640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30b815e430 0x7f30b80b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.587+0000 7f30b77fe640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30b8162540 0x7f30b80b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.587+0000 7f30b6ffd640 1 -- 172.21.15.172:0/457279474 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30b815e430 msgr2=0x7f30b80b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.587+0000 7f30b6ffd640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30b815e430 0x7f30b80b9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.587+0000 7f30b6ffd640 1 -- 172.21.15.172:0/457279474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30b8162540 msgr2=0x7f30b80b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.587+0000 7f30b6ffd640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30b8162540 0x7f30b80b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.587+0000 7f30b6ffd640 1 -- 172.21.15.172:0/457279474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b80b7a20 con 0x7f30b80d2e00 2024-04-14T08:33:06.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.587+0000 7f30b6ffd640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b80d2e00 0x7f30b8158d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f30a800e9b0 tx=0x7f30a800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:06.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.587+0000 7f309f7fe640 1 -- 172.21.15.172:0/457279474 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8017070 con 0x7f30b80d2e00 2024-04-14T08:33:06.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.587+0000 7f309f7fe640 1 -- 172.21.15.172:0/457279474 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30a800c9a0 con 0x7f30b80d2e00 2024-04-14T08:33:06.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.587+0000 7f30bd37b640 1 -- 172.21.15.172:0/457279474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b80b7d00 con 0x7f30b80d2e00 2024-04-14T08:33:06.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.587+0000 7f309f7fe640 1 -- 172.21.15.172:0/457279474 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a800cd90 con 0x7f30b80d2e00 2024-04-14T08:33:06.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.587+0000 7f30bd37b640 1 -- 172.21.15.172:0/457279474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b80b9110 con 0x7f30b80d2e00 2024-04-14T08:33:06.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.588+0000 7f30bd37b640 1 -- 172.21.15.172:0/457279474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b8068590 con 0x7f30b80d2e00 2024-04-14T08:33:06.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.588+0000 7f309f7fe640 1 -- 172.21.15.172:0/457279474 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30a801a070 con 0x7f30b80d2e00 2024-04-14T08:33:06.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.589+0000 7f309f7fe640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30a406c8f0 0x7f30a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.589+0000 7f309f7fe640 1 -- 172.21.15.172:0/457279474 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f30a808c1b0 con 0x7f30b80d2e00 2024-04-14T08:33:06.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.589+0000 7f30b67fc640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30a406c8f0 0x7f30a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.589+0000 7f30b67fc640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30a406c8f0 0x7f30a406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f30b80b6710 tx=0x7f30ac007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:06.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.593+0000 7f309f7fe640 1 -- 172.21.15.172:0/457279474 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f30a8059c50 con 0x7f30b80d2e00 2024-04-14T08:33:06.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.770+0000 7f30bd37b640 1 -- 172.21.15.172:0/457279474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f30b80d18a0 con 0x7f30b80d2e00 2024-04-14T08:33:06.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.770+0000 7f309f7fe640 1 -- 172.21.15.172:0/457279474 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f30a805df50 con 0x7f30b80d2e00 2024-04-14T08:33:06.771 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:06.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.772+0000 7f30bd37b640 1 -- 172.21.15.172:0/457279474 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30a406c8f0 msgr2=0x7f30a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.772+0000 7f30bd37b640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30a406c8f0 0x7f30a406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f30b80b6710 tx=0x7f30ac007e80 comp rx=0 tx=0).stop 2024-04-14T08:33:06.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.772+0000 7f30bd37b640 1 -- 172.21.15.172:0/457279474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b80d2e00 msgr2=0x7f30b8158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.772+0000 7f30bd37b640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b80d2e00 0x7f30b8158d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f30a800e9b0 tx=0x7f30a800ee70 comp rx=0 tx=0).stop 2024-04-14T08:33:06.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.772+0000 7f30bd37b640 1 -- 172.21.15.172:0/457279474 shutdown_connections 2024-04-14T08:33:06.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.772+0000 7f30bd37b640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30a406c8f0 0x7f30a406edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.772+0000 7f30bd37b640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30b8162540 0x7f30b80b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.772+0000 7f30bd37b640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30b815e430 0x7f30b80b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.772+0000 7f30bd37b640 1 --2- 172.21.15.172:0/457279474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30b80d2e00 0x7f30b8158d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.772+0000 7f30bd37b640 1 -- 172.21.15.172:0/457279474 >> 172.21.15.172:0/457279474 conn(0x7f30b809d560 msgr2=0x7f30b80b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:06.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.772+0000 7f30bd37b640 1 -- 172.21.15.172:0/457279474 shutdown_connections 2024-04-14T08:33:06.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.772+0000 7f30bd37b640 1 -- 172.21.15.172:0/457279474 wait complete. 2024-04-14T08:33:06.784 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:06.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.887+0000 7f9afb884640 1 Processor -- start 2024-04-14T08:33:06.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.888+0000 7f9afb884640 1 -- start start 2024-04-14T08:33:06.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.888+0000 7f9afb884640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9af415e430 0x7f9af4162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.888+0000 7f9afb884640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9af4162540 0x7f9af41629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.888+0000 7f9afb884640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af40b3a30 0x7f9af40b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.888+0000 7f9afb884640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f9af40cefb0 con 0x7f9af415e430 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.888+0000 7f9afb884640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f9af40cf290 con 0x7f9af4162540 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.888+0000 7f9afb884640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9af40cf570 con 0x7f9af40b3a30 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.889+0000 7f9af9620640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9af415e430 0x7f9af4162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.889+0000 7f9af8e1f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9af4162540 0x7f9af41629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.889+0000 7f9af9620640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9af415e430 0x7f9af4162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39082/0 (socket says 172.21.15.172:39082) 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.889+0000 7f9af9620640 1 -- 172.21.15.172:0/1349721236 learned_addr learned my addr 172.21.15.172:0/1349721236 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.889+0000 7f9af9e21640 1 --2- 172.21.15.172:0/1349721236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af40b3a30 0x7f9af40b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.889+0000 7f9af9620640 1 -- 172.21.15.172:0/1349721236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af40b3a30 msgr2=0x7f9af40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.889+0000 7f9af9620640 1 --2- 172.21.15.172:0/1349721236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af40b3a30 0x7f9af40b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.889+0000 7f9af9620640 1 -- 172.21.15.172:0/1349721236 >> v2:172.21.15.177:3300/0 conn(0x7f9af4162540 msgr2=0x7f9af41629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:06.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.889+0000 7f9af9620640 1 --2- 172.21.15.172:0/1349721236 >> v2:172.21.15.177:3300/0 conn(0x7f9af4162540 0x7f9af41629d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.889+0000 7f9af9620640 1 -- 172.21.15.172:0/1349721236 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af40cf850 con 0x7f9af415e430 2024-04-14T08:33:06.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.889+0000 7f9af8e1f640 1 --2- 172.21.15.172:0/1349721236 >> v2:172.21.15.177:3300/0 conn(0x7f9af4162540 0x7f9af41629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:06.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9af9620640 1 --2- 172.21.15.172:0/1349721236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af415e430 0x7f9af4162000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9af408dae0 tx=0x7f9af002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93803d6f42b9073 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:06.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9aebfff640 1 -- 172.21.15.172:0/1349721236 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0035020 con 0x7f9af415e430 2024-04-14T08:33:06.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9aebfff640 1 -- 172.21.15.172:0/1349721236 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af0031870 con 0x7f9af415e430 2024-04-14T08:33:06.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9aebfff640 1 -- 172.21.15.172:0/1349721236 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0031c40 con 0x7f9af415e430 2024-04-14T08:33:06.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9afb884640 1 -- 172.21.15.172:0/1349721236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af415e430 msgr2=0x7f9af4162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9afb884640 1 --2- 172.21.15.172:0/1349721236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af415e430 0x7f9af4162000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9af408dae0 tx=0x7f9af002b910 comp rx=0 tx=0).stop 2024-04-14T08:33:06.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9afb884640 1 -- 172.21.15.172:0/1349721236 shutdown_connections 2024-04-14T08:33:06.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9afb884640 1 --2- 172.21.15.172:0/1349721236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af40b3a30 0x7f9af40b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9afb884640 1 --2- 172.21.15.172:0/1349721236 >> v2:172.21.15.177:3300/0 conn(0x7f9af4162540 0x7f9af41629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9afb884640 1 --2- 172.21.15.172:0/1349721236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af415e430 0x7f9af4162000 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9afb884640 1 -- 172.21.15.172:0/1349721236 >> 172.21.15.172:0/1349721236 conn(0x7f9af409d560 msgr2=0x7f9af40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:06.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9afb884640 1 -- 172.21.15.172:0/1349721236 shutdown_connections 2024-04-14T08:33:06.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.890+0000 7f9afb884640 1 -- 172.21.15.172:0/1349721236 wait complete. 2024-04-14T08:33:06.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9afb884640 1 Processor -- start 2024-04-14T08:33:06.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9afb884640 1 -- start start 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9afb884640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af40b3a30 0x7f9af4151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9afb884640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af415e430 0x7f9af40b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9afb884640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9af4162540 0x7f9af40b9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9afb884640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9af40d4e20 con 0x7f9af40b3a30 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9afb884640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9af415e810 con 0x7f9af4162540 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9afb884640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9af40d0050 con 0x7f9af415e430 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9af9620640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af40b3a30 0x7f9af4151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9af9620640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af40b3a30 0x7f9af4151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39094/0 (socket says 172.21.15.172:39094) 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9af9620640 1 -- 172.21.15.172:0/1338380263 learned_addr learned my addr 172.21.15.172:0/1338380263 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9af8e1f640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af415e430 0x7f9af40b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.892+0000 7f9af9e21640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9af4162540 0x7f9af40b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.893+0000 7f9af9620640 1 -- 172.21.15.172:0/1338380263 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af415e430 msgr2=0x7f9af40b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.893+0000 7f9af9620640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af415e430 0x7f9af40b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.893+0000 7f9af9620640 1 -- 172.21.15.172:0/1338380263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9af4162540 msgr2=0x7f9af40b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.893+0000 7f9af9620640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9af4162540 0x7f9af40b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.893+0000 7f9af9620640 1 -- 172.21.15.172:0/1338380263 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af4151d60 con 0x7f9af40b3a30 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.893+0000 7f9af9620640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af40b3a30 0x7f9af4151820 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9af002b950 tx=0x7f9af00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.893+0000 7f9ae9ffb640 1 -- 172.21.15.172:0/1338380263 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0040070 con 0x7f9af40b3a30 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.893+0000 7f9ae9ffb640 1 -- 172.21.15.172:0/1338380263 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af0031e10 con 0x7f9af40b3a30 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.893+0000 7f9ae9ffb640 1 -- 172.21.15.172:0/1338380263 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0035040 con 0x7f9af40b3a30 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.893+0000 7f9afb884640 1 -- 172.21.15.172:0/1338380263 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af40b7750 con 0x7f9af40b3a30 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.893+0000 7f9afb884640 1 -- 172.21.15.172:0/1338380263 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af40b7d10 con 0x7f9af40b3a30 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.894+0000 7f9afb884640 1 -- 172.21.15.172:0/1338380263 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af4068590 con 0x7f9af40b3a30 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.895+0000 7f9ae9ffb640 1 -- 172.21.15.172:0/1338380263 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9af004b060 con 0x7f9af40b3a30 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.895+0000 7f9ae9ffb640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9ad806c920 0x7f9ad806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.895+0000 7f9ae9ffb640 1 -- 172.21.15.172:0/1338380263 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9af00ad900 con 0x7f9af40b3a30 2024-04-14T08:33:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.895+0000 7f9af8e1f640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9ad806c920 0x7f9ad806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:06.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.895+0000 7f9af8e1f640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9ad806c920 0x7f9ad806ede0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9af415de10 tx=0x7f9ae4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:06.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:06.899+0000 7f9ae9ffb640 1 -- 172.21.15.172:0/1338380263 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9af007b3a0 con 0x7f9af40b3a30 2024-04-14T08:33:07.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.074+0000 7f9afb884640 1 -- 172.21.15.172:0/1338380263 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9af40d18a0 con 0x7f9af40b3a30 2024-04-14T08:33:07.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.075+0000 7f9ae9ffb640 1 -- 172.21.15.172:0/1338380263 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f9af007f6a0 con 0x7f9af40b3a30 2024-04-14T08:33:07.075 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:07.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.077+0000 7f9afb884640 1 -- 172.21.15.172:0/1338380263 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9ad806c920 msgr2=0x7f9ad806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.077+0000 7f9afb884640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9ad806c920 0x7f9ad806ede0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9af415de10 tx=0x7f9ae4009190 comp rx=0 tx=0).stop 2024-04-14T08:33:07.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.077+0000 7f9afb884640 1 -- 172.21.15.172:0/1338380263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af40b3a30 msgr2=0x7f9af4151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.077+0000 7f9afb884640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af40b3a30 0x7f9af4151820 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9af002b950 tx=0x7f9af00046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.077+0000 7f9afb884640 1 -- 172.21.15.172:0/1338380263 shutdown_connections 2024-04-14T08:33:07.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.077+0000 7f9afb884640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9ad806c920 0x7f9ad806ede0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.077+0000 7f9afb884640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9af4162540 0x7f9af40b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.077+0000 7f9afb884640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9af415e430 0x7f9af40b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.077+0000 7f9afb884640 1 --2- 172.21.15.172:0/1338380263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af40b3a30 0x7f9af4151820 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.077+0000 7f9afb884640 1 -- 172.21.15.172:0/1338380263 >> 172.21.15.172:0/1338380263 conn(0x7f9af409d560 msgr2=0x7f9af40b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:07.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.077+0000 7f9afb884640 1 -- 172.21.15.172:0/1338380263 shutdown_connections 2024-04-14T08:33:07.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.077+0000 7f9afb884640 1 -- 172.21.15.172:0/1338380263 wait complete. 2024-04-14T08:33:07.089 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:33:07.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.191+0000 7f33f1c88640 1 Processor -- start 2024-04-14T08:33:07.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.191+0000 7f33f1c88640 1 -- start start 2024-04-14T08:33:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33f1c88640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f33ec06d040 0x7f33ec0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33f1c88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33ec15e820 0x7f33ec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33f1c88640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f33ec0d0d30 0x7f33ec09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33f1c88640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f33ec06cb90 con 0x7f33ec0d0d30 2024-04-14T08:33:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33f1c88640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f33ec0cc660 con 0x7f33ec06d040 2024-04-14T08:33:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33f1c88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f33ec0ccc20 con 0x7f33ec15e820 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33ebfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f33ec0d0d30 0x7f33ec09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33ebfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f33ec0d0d30 0x7f33ec09aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39110/0 (socket says 172.21.15.172:39110) 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33ebfff640 1 -- 172.21.15.172:0/3859713679 learned_addr learned my addr 172.21.15.172:0/3859713679 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33eb7fe640 1 --2- 172.21.15.172:0/3859713679 >> v2:172.21.15.177:3300/0 conn(0x7f33ec06d040 0x7f33ec0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33eaffd640 1 --2- 172.21.15.172:0/3859713679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33ec15e820 0x7f33ec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33ebfff640 1 -- 172.21.15.172:0/3859713679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33ec15e820 msgr2=0x7f33ec06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33ebfff640 1 --2- 172.21.15.172:0/3859713679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33ec15e820 0x7f33ec06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33ebfff640 1 -- 172.21.15.172:0/3859713679 >> v2:172.21.15.177:3300/0 conn(0x7f33ec06d040 msgr2=0x7f33ec0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33ebfff640 1 --2- 172.21.15.172:0/3859713679 >> v2:172.21.15.177:3300/0 conn(0x7f33ec06d040 0x7f33ec0b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33ebfff640 1 -- 172.21.15.172:0/3859713679 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ec0d12e0 con 0x7f33ec0d0d30 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.192+0000 7f33eb7fe640 1 --2- 172.21.15.172:0/3859713679 >> v2:172.21.15.177:3300/0 conn(0x7f33ec06d040 0x7f33ec0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.193+0000 7f33ebfff640 1 --2- 172.21.15.172:0/3859713679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33ec0d0d30 0x7f33ec09aa70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33ec068210 tx=0x7f33e402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5001b96882616a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:07.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.193+0000 7f33ea7fc640 1 -- 172.21.15.172:0/3859713679 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e402ca70 con 0x7f33ec0d0d30 2024-04-14T08:33:07.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.193+0000 7f33ea7fc640 1 -- 172.21.15.172:0/3859713679 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e402cc10 con 0x7f33ec0d0d30 2024-04-14T08:33:07.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.193+0000 7f33ea7fc640 1 -- 172.21.15.172:0/3859713679 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e40343f0 con 0x7f33ec0d0d30 2024-04-14T08:33:07.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.193+0000 7f33f1c88640 1 -- 172.21.15.172:0/3859713679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33ec0d0d30 msgr2=0x7f33ec09aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.193+0000 7f33f1c88640 1 --2- 172.21.15.172:0/3859713679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33ec0d0d30 0x7f33ec09aa70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33ec068210 tx=0x7f33e402bed0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.193+0000 7f33f1c88640 1 -- 172.21.15.172:0/3859713679 shutdown_connections 2024-04-14T08:33:07.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.193+0000 7f33f1c88640 1 --2- 172.21.15.172:0/3859713679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33ec0d0d30 0x7f33ec09aa70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.193+0000 7f33f1c88640 1 --2- 172.21.15.172:0/3859713679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33ec15e820 0x7f33ec06d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.193+0000 7f33f1c88640 1 --2- 172.21.15.172:0/3859713679 >> v2:172.21.15.177:3300/0 conn(0x7f33ec06d040 0x7f33ec0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.193+0000 7f33f1c88640 1 -- 172.21.15.172:0/3859713679 >> 172.21.15.172:0/3859713679 conn(0x7f33ec0b3b00 msgr2=0x7f33ec0b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:07.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.194+0000 7f33f1c88640 1 -- 172.21.15.172:0/3859713679 shutdown_connections 2024-04-14T08:33:07.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.194+0000 7f33f1c88640 1 -- 172.21.15.172:0/3859713679 wait complete. 2024-04-14T08:33:07.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.195+0000 7f33f1c88640 1 Processor -- start 2024-04-14T08:33:07.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.195+0000 7f33f1c88640 1 -- start start 2024-04-14T08:33:07.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.195+0000 7f33f1c88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33ec06d040 0x7f33ec0b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33f1c88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33ec0d0d30 0x7f33ec1543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33f1c88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33ec15e820 0x7f33ec0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33f1c88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec0cc8b0 con 0x7f33ec06d040 2024-04-14T08:33:07.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33f1c88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec0d1580 con 0x7f33ec0d0d30 2024-04-14T08:33:07.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33f1c88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f33ec09afb0 con 0x7f33ec15e820 2024-04-14T08:33:07.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33eb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33ec06d040 0x7f33ec0b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33eb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33ec06d040 0x7f33ec0b79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39124/0 (socket says 172.21.15.172:39124) 2024-04-14T08:33:07.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33eb7fe640 1 -- 172.21.15.172:0/1432999132 learned_addr learned my addr 172.21.15.172:0/1432999132 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:07.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33eaffd640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33ec0d0d30 0x7f33ec1543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33ebfff640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33ec15e820 0x7f33ec0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33eb7fe640 1 -- 172.21.15.172:0/1432999132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33ec15e820 msgr2=0x7f33ec0cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33eb7fe640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33ec15e820 0x7f33ec0cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33eb7fe640 1 -- 172.21.15.172:0/1432999132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33ec0d0d30 msgr2=0x7f33ec1543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33eb7fe640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33ec0d0d30 0x7f33ec1543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33eb7fe640 1 -- 172.21.15.172:0/1432999132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ec0b6c70 con 0x7f33ec06d040 2024-04-14T08:33:07.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33eb7fe640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33ec06d040 0x7f33ec0b79f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00e980 tx=0x7f33dc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:07.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33d3fff640 1 -- 172.21.15.172:0/1432999132 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc017070 con 0x7f33ec06d040 2024-04-14T08:33:07.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33d3fff640 1 -- 172.21.15.172:0/1432999132 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33dc00ca00 con 0x7f33ec06d040 2024-04-14T08:33:07.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33f1c88640 1 -- 172.21.15.172:0/1432999132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ec0b6f10 con 0x7f33ec06d040 2024-04-14T08:33:07.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33d3fff640 1 -- 172.21.15.172:0/1432999132 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc00cdd0 con 0x7f33ec06d040 2024-04-14T08:33:07.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.196+0000 7f33f1c88640 1 -- 172.21.15.172:0/1432999132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ec0cbbd0 con 0x7f33ec06d040 2024-04-14T08:33:07.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.197+0000 7f33f1c88640 1 -- 172.21.15.172:0/1432999132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33ec068590 con 0x7f33ec06d040 2024-04-14T08:33:07.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.198+0000 7f33d3fff640 1 -- 172.21.15.172:0/1432999132 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33dc01a070 con 0x7f33ec06d040 2024-04-14T08:33:07.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.198+0000 7f33d3fff640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33d806c8f0 0x7f33d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.198+0000 7f33d3fff640 1 -- 172.21.15.172:0/1432999132 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f33dc08bc90 con 0x7f33ec06d040 2024-04-14T08:33:07.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.198+0000 7f33eaffd640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33d806c8f0 0x7f33d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.199+0000 7f33eaffd640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33d806c8f0 0x7f33d806edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33ec1f8aa0 tx=0x7f33e0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:07.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.203+0000 7f33d3fff640 1 -- 172.21.15.172:0/1432999132 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f33dc059730 con 0x7f33ec06d040 2024-04-14T08:33:07.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.395+0000 7f33f1c88640 1 -- 172.21.15.172:0/1432999132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f33ec06ce20 con 0x7f33ec06d040 2024-04-14T08:33:07.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.395+0000 7f33d3fff640 1 -- 172.21.15.172:0/1432999132 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f33dc05da30 con 0x7f33ec06d040 2024-04-14T08:33:07.396 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:33:07.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.397+0000 7f33f1c88640 1 -- 172.21.15.172:0/1432999132 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33d806c8f0 msgr2=0x7f33d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.397+0000 7f33f1c88640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33d806c8f0 0x7f33d806edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33ec1f8aa0 tx=0x7f33e0009190 comp rx=0 tx=0).stop 2024-04-14T08:33:07.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.397+0000 7f33f1c88640 1 -- 172.21.15.172:0/1432999132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33ec06d040 msgr2=0x7f33ec0b79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.397+0000 7f33f1c88640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33ec06d040 0x7f33ec0b79f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00e980 tx=0x7f33dc00ee40 comp rx=0 tx=0).stop 2024-04-14T08:33:07.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.397+0000 7f33f1c88640 1 -- 172.21.15.172:0/1432999132 shutdown_connections 2024-04-14T08:33:07.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.397+0000 7f33f1c88640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33d806c8f0 0x7f33d806edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.397+0000 7f33f1c88640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33ec15e820 0x7f33ec0cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.397+0000 7f33f1c88640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33ec0d0d30 0x7f33ec1543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.397+0000 7f33f1c88640 1 --2- 172.21.15.172:0/1432999132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33ec06d040 0x7f33ec0b79f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.397+0000 7f33f1c88640 1 -- 172.21.15.172:0/1432999132 >> 172.21.15.172:0/1432999132 conn(0x7f33ec0b3b00 msgr2=0x7f33ec0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.397+0000 7f33f1c88640 1 -- 172.21.15.172:0/1432999132 shutdown_connections 2024-04-14T08:33:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.397+0000 7f33f1c88640 1 -- 172.21.15.172:0/1432999132 wait complete. 2024-04-14T08:33:07.408 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi172.front.sepia.ceph.com... 2024-04-14T08:33:07.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:07.409 DEBUG:teuthology.orchestra.run.smithi172:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-14T08:33:07.434 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi172.stderr:ceph-fuse[44451]: fuse finished with error 0 and tester_r 0 2024-04-14T08:33:07.436 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:33:07.510 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:33:07.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:07.510 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-14T08:33:07.531 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:07.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.672+0000 7f3c2a39c640 1 Processor -- start 2024-04-14T08:33:07.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.672+0000 7f3c2a39c640 1 -- start start 2024-04-14T08:33:07.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.673+0000 7f3c2a39c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3c240af340 0x7f3c240af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.673+0000 7f3c2a39c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c240cf920 con 0x7f3c2406dee0 2024-04-14T08:33:07.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.673+0000 7f3c2a39c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2415f9e0 con 0x7f3c240cc450 2024-04-14T08:33:07.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.673+0000 7f3c2a39c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f3c240cdea0 con 0x7f3c240af340 2024-04-14T08:33:07.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.673+0000 7f3c237fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3c240af340 0x7f3c240af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.673+0000 7f3c28939640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3c2406dee0 0x7f3c240ccbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40278/0 (socket says 172.21.15.172:40278) 2024-04-14T08:33:07.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.673+0000 7f3c28939640 1 -- 172.21.15.172:0/3909208880 learned_addr learned my addr 172.21.15.172:0/3909208880 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:07.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.673+0000 7f3c237fe640 1 -- 172.21.15.172:0/3909208880 >> v1:172.21.15.177:6789/0 conn(0x7f3c240cc450 legacy=0x7f3c2406d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.673+0000 7f3c237fe640 1 -- 172.21.15.172:0/3909208880 >> v1:172.21.15.172:6789/0 conn(0x7f3c2406dee0 legacy=0x7f3c240ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.673+0000 7f3c237fe640 1 -- 172.21.15.172:0/3909208880 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c240ce0f0 con 0x7f3c240af340 2024-04-14T08:33:07.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.673+0000 7f3c22ffd640 1 -- 172.21.15.172:0/3909208880 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2907896995 0 0) 0x7f3c1c001490 con 0x7f3c2406dee0 2024-04-14T08:33:07.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.674+0000 7f3c237fe640 1 --2- 172.21.15.172:0/3909208880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c240af340 0x7f3c240af790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3c180097a0 tx=0x7f3c1802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60c31da3d0cb51f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:07.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.674+0000 7f3c22ffd640 1 -- 172.21.15.172:0/3909208880 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18034070 con 0x7f3c240af340 2024-04-14T08:33:07.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.674+0000 7f3c22ffd640 1 -- 172.21.15.172:0/3909208880 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c18030610 con 0x7f3c240af340 2024-04-14T08:33:07.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.674+0000 7f3c22ffd640 1 -- 172.21.15.172:0/3909208880 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c180309e0 con 0x7f3c240af340 2024-04-14T08:33:07.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.675+0000 7f3c2a39c640 1 -- 172.21.15.172:0/3909208880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c240af340 msgr2=0x7f3c240af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.675+0000 7f3c2a39c640 1 --2- 172.21.15.172:0/3909208880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c240af340 0x7f3c240af790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3c180097a0 tx=0x7f3c1802b960 comp rx=0 tx=0).stop 2024-04-14T08:33:07.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.675+0000 7f3c2a39c640 1 -- 172.21.15.172:0/3909208880 shutdown_connections 2024-04-14T08:33:07.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.675+0000 7f3c2a39c640 1 --2- 172.21.15.172:0/3909208880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c240af340 0x7f3c240af790 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.675+0000 7f3c2a39c640 1 -- 172.21.15.172:0/3909208880 >> 172.21.15.172:0/3909208880 conn(0x7f3c24072b20 msgr2=0x7f3c24072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:07.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.675+0000 7f3c2a39c640 1 -- 172.21.15.172:0/3909208880 shutdown_connections 2024-04-14T08:33:07.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.675+0000 7f3c2a39c640 1 -- 172.21.15.172:0/3909208880 wait complete. 2024-04-14T08:33:07.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.676+0000 7f3c2a39c640 1 Processor -- start 2024-04-14T08:33:07.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.676+0000 7f3c2a39c640 1 -- start start 2024-04-14T08:33:07.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c2a39c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c2406dee0 0x7f3c24159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c2a39c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c240af340 0x7f3c241595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c2a39c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c240cc450 0x7f3c240b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c2a39c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c240ce3d0 con 0x7f3c240af340 2024-04-14T08:33:07.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c2a39c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2406f7a0 con 0x7f3c2406dee0 2024-04-14T08:33:07.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c2a39c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2415f600 con 0x7f3c240cc450 2024-04-14T08:33:07.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c240af340 0x7f3c241595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c240af340 0x7f3c241595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39136/0 (socket says 172.21.15.172:39136) 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c237fe640 1 -- 172.21.15.172:0/499979555 learned_addr learned my addr 172.21.15.172:0/499979555 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c28939640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c240cc450 0x7f3c240b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c23fff640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c2406dee0 0x7f3c24159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c237fe640 1 -- 172.21.15.172:0/499979555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c240cc450 msgr2=0x7f3c240b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c237fe640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c240cc450 0x7f3c240b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c237fe640 1 -- 172.21.15.172:0/499979555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c2406dee0 msgr2=0x7f3c24159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c237fe640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c2406dee0 0x7f3c24159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.677+0000 7f3c237fe640 1 -- 172.21.15.172:0/499979555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2415a440 con 0x7f3c240af340 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.678+0000 7f3c237fe640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c240af340 0x7f3c241595a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c180098c0 tx=0x7f3c180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.678+0000 7f3c20ff9640 1 -- 172.21.15.172:0/499979555 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18034040 con 0x7f3c240af340 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.678+0000 7f3c20ff9640 1 -- 172.21.15.172:0/499979555 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c18039a10 con 0x7f3c240af340 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.678+0000 7f3c20ff9640 1 -- 172.21.15.172:0/499979555 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18039de0 con 0x7f3c240af340 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.678+0000 7f3c2a39c640 1 -- 172.21.15.172:0/499979555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c241e8bb0 con 0x7f3c240af340 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.678+0000 7f3c2a39c640 1 -- 172.21.15.172:0/499979555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c241e9130 con 0x7f3c240af340 2024-04-14T08:33:07.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.679+0000 7f3c2a39c640 1 -- 172.21.15.172:0/499979555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c24068520 con 0x7f3c240af340 2024-04-14T08:33:07.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.679+0000 7f3c20ff9640 1 -- 172.21.15.172:0/499979555 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c18037070 con 0x7f3c240af340 2024-04-14T08:33:07.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.679+0000 7f3c20ff9640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3c1006c920 0x7f3c1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.680+0000 7f3c20ff9640 1 -- 172.21.15.172:0/499979555 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3c180adde0 con 0x7f3c240af340 2024-04-14T08:33:07.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.680+0000 7f3c23fff640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3c1006c920 0x7f3c1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.680+0000 7f3c23fff640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3c1006c920 0x7f3c1006ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3c14000f60 tx=0x7f3c14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:07.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.684+0000 7f3c20ff9640 1 -- 172.21.15.172:0/499979555 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3c2415a440 con 0x7f3c240af340 2024-04-14T08:33:07.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.861+0000 7f3c2a39c640 1 -- 172.21.15.172:0/499979555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c24072340 con 0x7f3c240af340 2024-04-14T08:33:07.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.862+0000 7f3c20ff9640 1 -- 172.21.15.172:0/499979555 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7f3c1807b910 con 0x7f3c240af340 2024-04-14T08:33:07.863 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:07.863 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:07.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.864+0000 7f3c2a39c640 1 -- 172.21.15.172:0/499979555 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3c1006c920 msgr2=0x7f3c1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.864+0000 7f3c2a39c640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3c1006c920 0x7f3c1006ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3c14000f60 tx=0x7f3c14009040 comp rx=0 tx=0).stop 2024-04-14T08:33:07.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.864+0000 7f3c2a39c640 1 -- 172.21.15.172:0/499979555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c240af340 msgr2=0x7f3c241595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.864+0000 7f3c2a39c640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c240af340 0x7f3c241595a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c180098c0 tx=0x7f3c180046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.864+0000 7f3c2a39c640 1 -- 172.21.15.172:0/499979555 shutdown_connections 2024-04-14T08:33:07.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.864+0000 7f3c2a39c640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3c1006c920 0x7f3c1006ede0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.864+0000 7f3c2a39c640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c240cc450 0x7f3c240b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.864+0000 7f3c2a39c640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c240af340 0x7f3c241595a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.864+0000 7f3c2a39c640 1 --2- 172.21.15.172:0/499979555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c2406dee0 0x7f3c24159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.864+0000 7f3c2a39c640 1 -- 172.21.15.172:0/499979555 >> 172.21.15.172:0/499979555 conn(0x7f3c24072b20 msgr2=0x7f3c240afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:07.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.864+0000 7f3c2a39c640 1 -- 172.21.15.172:0/499979555 shutdown_connections 2024-04-14T08:33:07.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.864+0000 7f3c2a39c640 1 -- 172.21.15.172:0/499979555 wait complete. 2024-04-14T08:33:07.877 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:07.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.978+0000 7fd744b1e640 1 Processor -- start 2024-04-14T08:33:07.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.979+0000 7fd744b1e640 1 -- start start 2024-04-14T08:33:07.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.979+0000 7fd744b1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd74015b250 0x7fd740072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.979+0000 7fd744b1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd7400c6470 0x7fd74006f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.979+0000 7fd744b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7400c69c0 0x7fd7400c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd744b1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd74006fee0 con 0x7fd7400c6470 2024-04-14T08:33:07.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd744b1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fd7400c7350 con 0x7fd74015b250 2024-04-14T08:33:07.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd744b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd74006ec50 con 0x7fd7400c69c0 2024-04-14T08:33:07.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd73dd9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd7400c6470 0x7fd74006f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd73e59c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd74015b250 0x7fd740072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd73dd9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd7400c6470 0x7fd74006f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39138/0 (socket says 172.21.15.172:39138) 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd73ed9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7400c69c0 0x7fd7400c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd73dd9b640 1 -- 172.21.15.172:0/1547437942 learned_addr learned my addr 172.21.15.172:0/1547437942 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd73e59c640 1 -- 172.21.15.172:0/1547437942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7400c69c0 msgr2=0x7fd7400c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd73e59c640 1 --2- 172.21.15.172:0/1547437942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7400c69c0 0x7fd7400c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd73e59c640 1 -- 172.21.15.172:0/1547437942 >> v2:172.21.15.172:3300/0 conn(0x7fd7400c6470 msgr2=0x7fd74006f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd73e59c640 1 --2- 172.21.15.172:0/1547437942 >> v2:172.21.15.172:3300/0 conn(0x7fd7400c6470 0x7fd74006f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd73e59c640 1 -- 172.21.15.172:0/1547437942 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74006eec0 con 0x7fd74015b250 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.980+0000 7fd73dd9b640 1 --2- 172.21.15.172:0/1547437942 >> v2:172.21.15.172:3300/0 conn(0x7fd7400c6470 0x7fd74006f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.981+0000 7fd73e59c640 1 --2- 172.21.15.172:0/1547437942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd74015b250 0x7fd740072d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7400cea30 tx=0x7fd73802e010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dea3bbde247a9eae server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.981+0000 7fd73d59a640 1 -- 172.21.15.172:0/1547437942 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73803b070 con 0x7fd74015b250 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.981+0000 7fd73d59a640 1 -- 172.21.15.172:0/1547437942 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73802edc0 con 0x7fd74015b250 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.981+0000 7fd73d59a640 1 -- 172.21.15.172:0/1547437942 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738036660 con 0x7fd74015b250 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.981+0000 7fd744b1e640 1 -- 172.21.15.172:0/1547437942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd74015b250 msgr2=0x7fd740072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.981+0000 7fd744b1e640 1 --2- 172.21.15.172:0/1547437942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd74015b250 0x7fd740072d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7400cea30 tx=0x7fd73802e010 comp rx=0 tx=0).stop 2024-04-14T08:33:07.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.982+0000 7fd744b1e640 1 -- 172.21.15.172:0/1547437942 shutdown_connections 2024-04-14T08:33:07.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.982+0000 7fd744b1e640 1 --2- 172.21.15.172:0/1547437942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7400c69c0 0x7fd7400c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.982+0000 7fd744b1e640 1 --2- 172.21.15.172:0/1547437942 >> v2:172.21.15.172:3300/0 conn(0x7fd7400c6470 0x7fd74006f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.982+0000 7fd744b1e640 1 --2- 172.21.15.172:0/1547437942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd74015b250 0x7fd740072d00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.982+0000 7fd744b1e640 1 -- 172.21.15.172:0/1547437942 >> 172.21.15.172:0/1547437942 conn(0x7fd7400d0f30 msgr2=0x7fd7400d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:07.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.982+0000 7fd744b1e640 1 -- 172.21.15.172:0/1547437942 shutdown_connections 2024-04-14T08:33:07.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.982+0000 7fd744b1e640 1 -- 172.21.15.172:0/1547437942 wait complete. 2024-04-14T08:33:07.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.983+0000 7fd744b1e640 1 Processor -- start 2024-04-14T08:33:07.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.983+0000 7fd744b1e640 1 -- start start 2024-04-14T08:33:07.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd744b1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7400c6470 0x7fd740153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd744b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7400c69c0 0x7fd7400cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd744b1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd74015b250 0x7fd740154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd744b1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd7400c5fd0 con 0x7fd7400c6470 2024-04-14T08:33:07.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd744b1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd74006f480 con 0x7fd74015b250 2024-04-14T08:33:07.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd744b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd740073390 con 0x7fd7400c69c0 2024-04-14T08:33:07.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd73e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7400c6470 0x7fd740153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd73e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7400c6470 0x7fd740153a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39148/0 (socket says 172.21.15.172:39148) 2024-04-14T08:33:07.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd73e59c640 1 -- 172.21.15.172:0/2594369002 learned_addr learned my addr 172.21.15.172:0/2594369002 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd73dd9b640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7400c69c0 0x7fd7400cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd73ed9d640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd74015b250 0x7fd740154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd73e59c640 1 -- 172.21.15.172:0/2594369002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7400c69c0 msgr2=0x7fd7400cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd73e59c640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7400c69c0 0x7fd7400cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd73e59c640 1 -- 172.21.15.172:0/2594369002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd74015b250 msgr2=0x7fd740154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd73e59c640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd74015b250 0x7fd740154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd73e59c640 1 -- 172.21.15.172:0/2594369002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd740155e80 con 0x7fd7400c6470 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd73e59c640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7400c6470 0x7fd740153a50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd73802e050 tx=0x7fd7380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd736ffd640 1 -- 172.21.15.172:0/2594369002 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73803b040 con 0x7fd7400c6470 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd736ffd640 1 -- 172.21.15.172:0/2594369002 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd738044070 con 0x7fd7400c6470 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd744b1e640 1 -- 172.21.15.172:0/2594369002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7400adfc0 con 0x7fd7400c6470 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd736ffd640 1 -- 172.21.15.172:0/2594369002 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738006e10 con 0x7fd7400c6470 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.984+0000 7fd744b1e640 1 -- 172.21.15.172:0/2594369002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7400ae540 con 0x7fd7400c6470 2024-04-14T08:33:07.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.985+0000 7fd744b1e640 1 -- 172.21.15.172:0/2594369002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd740075490 con 0x7fd7400c6470 2024-04-14T08:33:07.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.986+0000 7fd736ffd640 1 -- 172.21.15.172:0/2594369002 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd73803f680 con 0x7fd7400c6470 2024-04-14T08:33:07.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.986+0000 7fd736ffd640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd72006c8f0 0x7fd72006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:07.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.986+0000 7fd736ffd640 1 -- 172.21.15.172:0/2594369002 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd7380adbc0 con 0x7fd7400c6470 2024-04-14T08:33:07.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.986+0000 7fd73dd9b640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd72006c8f0 0x7fd72006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:07.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.987+0000 7fd73dd9b640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd72006c8f0 0x7fd72006edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd74009ee80 tx=0x7fd728009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:07.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:07.991+0000 7fd736ffd640 1 -- 172.21.15.172:0/2594369002 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd73807bdd0 con 0x7fd7400c6470 2024-04-14T08:33:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.166+0000 7fd744b1e640 1 -- 172.21.15.172:0/2594369002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7400ce840 con 0x7fd7400c6470 2024-04-14T08:33:08.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.167+0000 7fd736ffd640 1 -- 172.21.15.172:0/2594369002 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7fd73807b6d0 con 0x7fd7400c6470 2024-04-14T08:33:08.168 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:08.168 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.169+0000 7fd744b1e640 1 -- 172.21.15.172:0/2594369002 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd72006c8f0 msgr2=0x7fd72006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.169+0000 7fd744b1e640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd72006c8f0 0x7fd72006edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd74009ee80 tx=0x7fd728009240 comp rx=0 tx=0).stop 2024-04-14T08:33:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.169+0000 7fd744b1e640 1 -- 172.21.15.172:0/2594369002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7400c6470 msgr2=0x7fd740153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.169+0000 7fd744b1e640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7400c6470 0x7fd740153a50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd73802e050 tx=0x7fd7380046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.169+0000 7fd744b1e640 1 -- 172.21.15.172:0/2594369002 shutdown_connections 2024-04-14T08:33:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.169+0000 7fd744b1e640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd72006c8f0 0x7fd72006edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.169+0000 7fd744b1e640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd74015b250 0x7fd740154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.169+0000 7fd744b1e640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7400c69c0 0x7fd7400cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:08.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.169+0000 7fd744b1e640 1 --2- 172.21.15.172:0/2594369002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7400c6470 0x7fd740153a50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:08.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.169+0000 7fd744b1e640 1 -- 172.21.15.172:0/2594369002 >> 172.21.15.172:0/2594369002 conn(0x7fd7400d0f30 msgr2=0x7fd7400ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:08.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.169+0000 7fd744b1e640 1 -- 172.21.15.172:0/2594369002 shutdown_connections 2024-04-14T08:33:08.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.169+0000 7fd744b1e640 1 -- 172.21.15.172:0/2594369002 wait complete. 2024-04-14T08:33:08.181 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-14T08:33:08.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.286+0000 7f99a36de640 1 Processor -- start 2024-04-14T08:33:08.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.286+0000 7f99a36de640 1 -- start start 2024-04-14T08:33:08.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a36de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f999c0cd100 0x7f999c0cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:08.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a36de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999c0cf4b0 0x7f999c0cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:08.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a36de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f999c15ffd0 con 0x7f999c0afa60 2024-04-14T08:33:08.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a36de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f999c1602b0 con 0x7f999c0cd100 2024-04-14T08:33:08.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a36de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f999c160590 con 0x7f999c0cf4b0 2024-04-14T08:33:08.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a147a640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f999c0afa60 0x7f999c15d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:54218/0 (socket says 172.21.15.172:54218) 2024-04-14T08:33:08.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a147a640 1 -- 172.21.15.172:0/3674112039 learned_addr learned my addr 172.21.15.172:0/3674112039 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:08.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a0c79640 1 --2- 172.21.15.172:0/3674112039 >> v2:172.21.15.177:3300/0 conn(0x7f999c0cd100 0x7f999c0cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:08.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a1c7b640 1 --2- 172.21.15.172:0/3674112039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999c0cf4b0 0x7f999c0cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:08.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f9993fff640 1 -- 172.21.15.172:0/3674112039 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3926865329 0 0) 0x7f999c15ffd0 con 0x7f999c0afa60 2024-04-14T08:33:08.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f9993fff640 1 -- 172.21.15.172:0/3674112039 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9988003520 con 0x7f999c0afa60 2024-04-14T08:33:08.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a0c79640 1 -- 172.21.15.172:0/3674112039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999c0cf4b0 msgr2=0x7f999c0cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:08.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a0c79640 1 --2- 172.21.15.172:0/3674112039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999c0cf4b0 0x7f999c0cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:08.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a0c79640 1 -- 172.21.15.172:0/3674112039 >> v1:172.21.15.177:6789/0 conn(0x7f999c0afa60 legacy=0x7f999c15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:08.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a0c79640 1 -- 172.21.15.172:0/3674112039 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c160870 con 0x7f999c0cd100 2024-04-14T08:33:08.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.287+0000 7f99a1c7b640 1 --2- 172.21.15.172:0/3674112039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999c0cf4b0 0x7f999c0cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:08.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.288+0000 7f99a0c79640 1 --2- 172.21.15.172:0/3674112039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999c0cd100 0x7f999c0cd590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f998c0097a0 tx=0x7f998c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=619303626f3bcca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:08.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.288+0000 7f9993fff640 1 -- 172.21.15.172:0/3674112039 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c0315a0 con 0x7f999c0cd100 2024-04-14T08:33:08.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.288+0000 7f9993fff640 1 -- 172.21.15.172:0/3674112039 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f998c037070 con 0x7f999c0cd100 2024-04-14T08:33:08.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.288+0000 7f9993fff640 1 -- 172.21.15.172:0/3674112039 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c031940 con 0x7f999c0cd100 2024-04-14T08:33:08.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.289+0000 7f99a36de640 1 -- 172.21.15.172:0/3674112039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999c0cd100 msgr2=0x7f999c0cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.289+0000 7f99a36de640 1 --2- 172.21.15.172:0/3674112039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999c0cd100 0x7f999c0cd590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f998c0097a0 tx=0x7f998c02b840 comp rx=0 tx=0).stop 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.289+0000 7f99a36de640 1 -- 172.21.15.172:0/3674112039 shutdown_connections 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.289+0000 7f99a36de640 1 --2- 172.21.15.172:0/3674112039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999c0cf4b0 0x7f999c0cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.289+0000 7f99a36de640 1 --2- 172.21.15.172:0/3674112039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999c0cd100 0x7f999c0cd590 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.289+0000 7f99a36de640 1 -- 172.21.15.172:0/3674112039 >> 172.21.15.172:0/3674112039 conn(0x7f999c0b29b0 msgr2=0x7f999c0d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.289+0000 7f99a36de640 1 -- 172.21.15.172:0/3674112039 shutdown_connections 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.289+0000 7f99a36de640 1 -- 172.21.15.172:0/3674112039 wait complete. 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.290+0000 7f99a36de640 1 Processor -- start 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.290+0000 7f99a36de640 1 -- start start 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a36de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999c0afa60 0x7f999c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a36de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f999c0cd100 0x7f999c15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a36de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999c0cf4b0 0x7f999c1e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:08.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a36de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f999c0afe40 con 0x7f999c0cd100 2024-04-14T08:33:08.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a36de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f999c15fca0 con 0x7f999c0cf4b0 2024-04-14T08:33:08.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a36de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f999c06b520 con 0x7f999c0afa60 2024-04-14T08:33:08.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a0c79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f999c0cd100 0x7f999c15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:08.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a147a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999c0afa60 0x7f999c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:08.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a1c7b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999c0cf4b0 0x7f999c1e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:08.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a0c79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f999c0cd100 0x7f999c15b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39162/0 (socket says 172.21.15.172:39162) 2024-04-14T08:33:08.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a0c79640 1 -- 172.21.15.172:0/4170373778 learned_addr learned my addr 172.21.15.172:0/4170373778 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:08.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a0c79640 1 -- 172.21.15.172:0/4170373778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999c0afa60 msgr2=0x7f999c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:08.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a0c79640 1 --2- 172.21.15.172:0/4170373778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999c0afa60 0x7f999c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:08.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a0c79640 1 -- 172.21.15.172:0/4170373778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999c0cf4b0 msgr2=0x7f999c1e78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:08.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a0c79640 1 --2- 172.21.15.172:0/4170373778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999c0cf4b0 0x7f999c1e78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:08.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a0c79640 1 -- 172.21.15.172:0/4170373778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c1403b0 con 0x7f999c0cd100 2024-04-14T08:33:08.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a0c79640 1 --2- 172.21.15.172:0/4170373778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f999c0cd100 0x7f999c15b3f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f998c034040 tx=0x7f998c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:08.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f9991ffb640 1 -- 172.21.15.172:0/4170373778 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c032630 con 0x7f999c0cd100 2024-04-14T08:33:08.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f9991ffb640 1 -- 172.21.15.172:0/4170373778 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f998c0327f0 con 0x7f999c0cd100 2024-04-14T08:33:08.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f9991ffb640 1 -- 172.21.15.172:0/4170373778 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c032be0 con 0x7f999c0cd100 2024-04-14T08:33:08.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.291+0000 7f99a36de640 1 -- 172.21.15.172:0/4170373778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c1e5b00 con 0x7f999c0cd100 2024-04-14T08:33:08.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.292+0000 7f99a36de640 1 -- 172.21.15.172:0/4170373778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c1e6340 con 0x7f999c0cd100 2024-04-14T08:33:08.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.293+0000 7f99a36de640 1 -- 172.21.15.172:0/4170373778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999c068520 con 0x7f999c0cd100 2024-04-14T08:33:08.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.293+0000 7f9991ffb640 1 -- 172.21.15.172:0/4170373778 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f998c04d3f0 con 0x7f999c0cd100 2024-04-14T08:33:08.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.293+0000 7f9991ffb640 1 --2- 172.21.15.172:0/4170373778 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f998006c8f0 0x7f998006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:08.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.293+0000 7f9991ffb640 1 -- 172.21.15.172:0/4170373778 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f998c02f030 con 0x7f999c0cd100 2024-04-14T08:33:08.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.293+0000 7f99a147a640 1 --2- 172.21.15.172:0/4170373778 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f998006c8f0 0x7f998006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:08.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.294+0000 7f99a147a640 1 --2- 172.21.15.172:0/4170373778 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f998006c8f0 0x7f998006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9998001200 tx=0x7f9998000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:08.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.298+0000 7f9991ffb640 1 -- 172.21.15.172:0/4170373778 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f999c1e5b00 con 0x7f999c0cd100 2024-04-14T08:33:08.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:08.511+0000 7f99a36de640 1 -- 172.21.15.172:0/4170373778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f999c145c50 con 0x7f999c0cd100 2024-04-14T08:33:09.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.256+0000 7f9991ffb640 1 -- 172.21.15.172:0/4170373778 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f998c040070 con 0x7f999c0cd100 2024-04-14T08:33:09.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.259+0000 7f99a36de640 1 -- 172.21.15.172:0/4170373778 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f998006c8f0 msgr2=0x7f998006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:09.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.259+0000 7f99a36de640 1 --2- 172.21.15.172:0/4170373778 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f998006c8f0 0x7f998006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9998001200 tx=0x7f9998000bc0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.259+0000 7f99a36de640 1 -- 172.21.15.172:0/4170373778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f999c0cd100 msgr2=0x7f999c15b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:09.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.259+0000 7f99a36de640 1 --2- 172.21.15.172:0/4170373778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f999c0cd100 0x7f999c15b3f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f998c034040 tx=0x7f998c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.259+0000 7f99a36de640 1 -- 172.21.15.172:0/4170373778 shutdown_connections 2024-04-14T08:33:09.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.259+0000 7f99a36de640 1 --2- 172.21.15.172:0/4170373778 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f998006c8f0 0x7f998006edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.259+0000 7f99a36de640 1 --2- 172.21.15.172:0/4170373778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f999c0cf4b0 0x7f999c1e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.259+0000 7f99a36de640 1 --2- 172.21.15.172:0/4170373778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f999c0cd100 0x7f999c15b3f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.259+0000 7f99a36de640 1 --2- 172.21.15.172:0/4170373778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f999c0afa60 0x7f999c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.259+0000 7f99a36de640 1 -- 172.21.15.172:0/4170373778 >> 172.21.15.172:0/4170373778 conn(0x7f999c0b29b0 msgr2=0x7f999c06e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:09.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.259+0000 7f99a36de640 1 -- 172.21.15.172:0/4170373778 shutdown_connections 2024-04-14T08:33:09.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.259+0000 7f99a36de640 1 -- 172.21.15.172:0/4170373778 wait complete. 2024-04-14T08:33:09.271 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-14T08:33:09.272 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-14T08:33:09.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.378+0000 7f49f8d1e640 1 Processor -- start 2024-04-14T08:33:09.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.378+0000 7f49f8d1e640 1 -- start start 2024-04-14T08:33:09.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.379+0000 7f49f8d1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49f40b7a00 0x7f49f40d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:09.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.379+0000 7f49f8d1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f49f40cf6f0 0x7f49f40ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:09.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.379+0000 7f49f8d1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f49f40ceea0 0x7f49f415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:09.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.379+0000 7f49f8d1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f49f40d1690 con 0x7f49f40ceea0 2024-04-14T08:33:09.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.379+0000 7f49f8d1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f49f40cd0e0 con 0x7f49f40cf6f0 2024-04-14T08:33:09.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.379+0000 7f49f8d1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f49f40cc7b0 con 0x7f49f40b7a00 2024-04-14T08:33:09.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.379+0000 7f49f2d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f49f40ceea0 0x7f49f415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:09.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.379+0000 7f49f2d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f49f40ceea0 0x7f49f415dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39174/0 (socket says 172.21.15.172:39174) 2024-04-14T08:33:09.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.379+0000 7f49f2d9d640 1 -- 172.21.15.172:0/3358981106 learned_addr learned my addr 172.21.15.172:0/3358981106 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:09.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.379+0000 7f49f259c640 1 --2- 172.21.15.172:0/3358981106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49f40b7a00 0x7f49f40d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:09.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.379+0000 7f49e9d9b640 1 --2- 172.21.15.172:0/3358981106 >> v2:172.21.15.177:3300/0 conn(0x7f49f40cf6f0 0x7f49f40ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:09.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.380+0000 7f49f2d9d640 1 -- 172.21.15.172:0/3358981106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49f40b7a00 msgr2=0x7f49f40d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:09.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.380+0000 7f49f2d9d640 1 --2- 172.21.15.172:0/3358981106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49f40b7a00 0x7f49f40d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.380+0000 7f49f2d9d640 1 -- 172.21.15.172:0/3358981106 >> v2:172.21.15.177:3300/0 conn(0x7f49f40cf6f0 msgr2=0x7f49f40ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:09.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.380+0000 7f49f2d9d640 1 --2- 172.21.15.172:0/3358981106 >> v2:172.21.15.177:3300/0 conn(0x7f49f40cf6f0 0x7f49f40ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.380+0000 7f49f2d9d640 1 -- 172.21.15.172:0/3358981106 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f40cca90 con 0x7f49f40ceea0 2024-04-14T08:33:09.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.380+0000 7f49f2d9d640 1 --2- 172.21.15.172:0/3358981106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f40ceea0 0x7f49f415dca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f49f406ed80 tx=0x7f49ec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cddca1bd68b2c8b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:09.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.381+0000 7f49f1d9b640 1 -- 172.21.15.172:0/3358981106 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec02c9e0 con 0x7f49f40ceea0 2024-04-14T08:33:09.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.381+0000 7f49f1d9b640 1 -- 172.21.15.172:0/3358981106 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ec02cb80 con 0x7f49f40ceea0 2024-04-14T08:33:09.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.381+0000 7f49f1d9b640 1 -- 172.21.15.172:0/3358981106 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0343f0 con 0x7f49f40ceea0 2024-04-14T08:33:09.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.381+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3358981106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f40ceea0 msgr2=0x7f49f415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:09.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.381+0000 7f49f8d1e640 1 --2- 172.21.15.172:0/3358981106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f40ceea0 0x7f49f415dca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f49f406ed80 tx=0x7f49ec02be40 comp rx=0 tx=0).stop 2024-04-14T08:33:09.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.381+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3358981106 shutdown_connections 2024-04-14T08:33:09.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.381+0000 7f49f8d1e640 1 --2- 172.21.15.172:0/3358981106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f40ceea0 0x7f49f415dca0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.381+0000 7f49f8d1e640 1 --2- 172.21.15.172:0/3358981106 >> v2:172.21.15.177:3300/0 conn(0x7f49f40cf6f0 0x7f49f40ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.381+0000 7f49f8d1e640 1 --2- 172.21.15.172:0/3358981106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49f40b7a00 0x7f49f40d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.381+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3358981106 >> 172.21.15.172:0/3358981106 conn(0x7f49f40d2030 msgr2=0x7f49f40d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:09.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.381+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3358981106 shutdown_connections 2024-04-14T08:33:09.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.381+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3358981106 wait complete. 2024-04-14T08:33:09.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49f8d1e640 1 Processor -- start 2024-04-14T08:33:09.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49f8d1e640 1 -- start start 2024-04-14T08:33:09.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49f8d1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49f40b7a00 0x7f49f41f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49f8d1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49f40ceea0 0x7f49f409e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49f8d1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f40cf6f0 0x7f49f409ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49f8d1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49f40d12c0 con 0x7f49f40cf6f0 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49f8d1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f49f40b7de0 con 0x7f49f40b7a00 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49f8d1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f49f40d1440 con 0x7f49f40ceea0 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49f2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f40cf6f0 0x7f49f409ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49f259c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49f40b7a00 0x7f49f41f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49f2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f40cf6f0 0x7f49f409ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39184/0 (socket says 172.21.15.172:39184) 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49f2d9d640 1 -- 172.21.15.172:0/3916447041 learned_addr learned my addr 172.21.15.172:0/3916447041 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.383+0000 7f49e9d9b640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49f40ceea0 0x7f49f409e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.384+0000 7f49f2d9d640 1 -- 172.21.15.172:0/3916447041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49f40ceea0 msgr2=0x7f49f409e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.384+0000 7f49f2d9d640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49f40ceea0 0x7f49f409e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.384+0000 7f49f2d9d640 1 -- 172.21.15.172:0/3916447041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49f40b7a00 msgr2=0x7f49f41f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.384+0000 7f49f2d9d640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49f40b7a00 0x7f49f41f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:09.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.384+0000 7f49f2d9d640 1 -- 172.21.15.172:0/3916447041 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f4156070 con 0x7f49f40cf6f0 2024-04-14T08:33:09.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.384+0000 7f49f259c640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49f40b7a00 0x7f49f41f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:09.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.384+0000 7f49f2d9d640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f40cf6f0 0x7f49f409ceb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f49ec036040 tx=0x7f49ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:09.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.384+0000 7f49eb7fe640 1 -- 172.21.15.172:0/3916447041 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec043020 con 0x7f49f40cf6f0 2024-04-14T08:33:09.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.384+0000 7f49eb7fe640 1 -- 172.21.15.172:0/3916447041 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ec03e450 con 0x7f49f40cf6f0 2024-04-14T08:33:09.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.384+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3916447041 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f409d900 con 0x7f49f40cf6f0 2024-04-14T08:33:09.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.384+0000 7f49eb7fe640 1 -- 172.21.15.172:0/3916447041 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec009510 con 0x7f49f40cf6f0 2024-04-14T08:33:09.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.384+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3916447041 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f409e1a0 con 0x7f49f40cf6f0 2024-04-14T08:33:09.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.385+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3916447041 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f4075490 con 0x7f49f40cf6f0 2024-04-14T08:33:09.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.386+0000 7f49eb7fe640 1 -- 172.21.15.172:0/3916447041 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49ec046020 con 0x7f49f40cf6f0 2024-04-14T08:33:09.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.386+0000 7f49eb7fe640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f49d406c8f0 0x7f49d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:09.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.386+0000 7f49eb7fe640 1 -- 172.21.15.172:0/3916447041 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f49ec0ae4a0 con 0x7f49f40cf6f0 2024-04-14T08:33:09.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.386+0000 7f49f259c640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f49d406c8f0 0x7f49d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:09.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.387+0000 7f49f259c640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f49d406c8f0 0x7f49d406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009e90 tx=0x7f49e4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:09.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.391+0000 7f49eb7fe640 1 -- 172.21.15.172:0/3916447041 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f49ec04d030 con 0x7f49f40cf6f0 2024-04-14T08:33:09.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:09.603+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3916447041 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f49f406f530 con 0x7f49f40cf6f0 2024-04-14T08:33:10.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.265+0000 7f49eb7fe640 1 -- 172.21.15.172:0/3916447041 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f49ec07bf40 con 0x7f49f40cf6f0 2024-04-14T08:33:10.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.268+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3916447041 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f49d406c8f0 msgr2=0x7f49d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.268+0000 7f49f8d1e640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f49d406c8f0 0x7f49d406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009e90 tx=0x7f49e4009190 comp rx=0 tx=0).stop 2024-04-14T08:33:10.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.268+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3916447041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f40cf6f0 msgr2=0x7f49f409ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.268+0000 7f49f8d1e640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f40cf6f0 0x7f49f409ceb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f49ec036040 tx=0x7f49ec004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:10.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.268+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3916447041 shutdown_connections 2024-04-14T08:33:10.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.268+0000 7f49f8d1e640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f49d406c8f0 0x7f49d406edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.268+0000 7f49f8d1e640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f40cf6f0 0x7f49f409ceb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.268+0000 7f49f8d1e640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49f40ceea0 0x7f49f409e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.268+0000 7f49f8d1e640 1 --2- 172.21.15.172:0/3916447041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49f40b7a00 0x7f49f41f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.268+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3916447041 >> 172.21.15.172:0/3916447041 conn(0x7f49f40d2030 msgr2=0x7f49f406cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:10.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.268+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3916447041 shutdown_connections 2024-04-14T08:33:10.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.268+0000 7f49f8d1e640 1 -- 172.21.15.172:0/3916447041 wait complete. 2024-04-14T08:33:10.282 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-14T08:33:10.283 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:33:10.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.388+0000 7f1e3362d640 1 Processor -- start 2024-04-14T08:33:10.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.389+0000 7f1e3362d640 1 -- start start 2024-04-14T08:33:10.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.389+0000 7f1e3362d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e2c0b1df0 0x7f1e2c0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.389+0000 7f1e3362d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1e2c0b21d0 0x7f1e2c15c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.389+0000 7f1e3362d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e2c0b3430 con 0x7f1e2c15c5a0 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.389+0000 7f1e3362d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f1e2c0b3700 con 0x7f1e2c0b21d0 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.389+0000 7f1e3362d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1e2c0b39e0 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.390+0000 7f1e313c9640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e2c0b1df0 0x7f1e2c0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.390+0000 7f1e30bc8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1e2c0b21d0 0x7f1e2c15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.390+0000 7f1e31bca640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f1e2c15c5a0 0x7f1e2c0b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42612/0 (socket says 172.21.15.172:42612) 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.390+0000 7f1e31bca640 1 -- 172.21.15.172:0/1844749869 learned_addr learned my addr 172.21.15.172:0/1844749869 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.390+0000 7f1e313c9640 1 -- 172.21.15.172:0/1844749869 >> v2:172.21.15.177:3300/0 conn(0x7f1e2c0b21d0 msgr2=0x7f1e2c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.390+0000 7f1e313c9640 1 --2- 172.21.15.172:0/1844749869 >> v2:172.21.15.177:3300/0 conn(0x7f1e2c0b21d0 0x7f1e2c15c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.390+0000 7f1e313c9640 1 -- 172.21.15.172:0/1844749869 >> v1:172.21.15.177:6789/0 conn(0x7f1e2c15c5a0 legacy=0x7f1e2c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.390+0000 7f1e313c9640 1 -- 172.21.15.172:0/1844749869 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2c0b3cc0 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.390+0000 7f1e30bc8640 1 --2- 172.21.15.172:0/1844749869 >> v2:172.21.15.177:3300/0 conn(0x7f1e2c0b21d0 0x7f1e2c15c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:10.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.391+0000 7f1e313c9640 1 --2- 172.21.15.172:0/1844749869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e2c0b1df0 0x7f1e2c0b2ef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0097a0 tx=0x7f1e1c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fddcaa434d029fc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:10.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.391+0000 7f1e23fff640 1 -- 172.21.15.172:0/1844749869 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c035020 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.391+0000 7f1e23fff640 1 -- 172.21.15.172:0/1844749869 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1c031830 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.391+0000 7f1e23fff640 1 -- 172.21.15.172:0/1844749869 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c031c00 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.391+0000 7f1e3362d640 1 -- 172.21.15.172:0/1844749869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e2c0b1df0 msgr2=0x7f1e2c0b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.391+0000 7f1e3362d640 1 --2- 172.21.15.172:0/1844749869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e2c0b1df0 0x7f1e2c0b2ef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0097a0 tx=0x7f1e1c02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.391+0000 7f1e3362d640 1 -- 172.21.15.172:0/1844749869 shutdown_connections 2024-04-14T08:33:10.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.391+0000 7f1e3362d640 1 --2- 172.21.15.172:0/1844749869 >> v2:172.21.15.177:3300/0 conn(0x7f1e2c0b21d0 0x7f1e2c15c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.391+0000 7f1e3362d640 1 --2- 172.21.15.172:0/1844749869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e2c0b1df0 0x7f1e2c0b2ef0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.391+0000 7f1e3362d640 1 -- 172.21.15.172:0/1844749869 >> 172.21.15.172:0/1844749869 conn(0x7f1e2c06ec30 msgr2=0x7f1e2c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:10.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.391+0000 7f1e3362d640 1 -- 172.21.15.172:0/1844749869 shutdown_connections 2024-04-14T08:33:10.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.391+0000 7f1e3362d640 1 -- 172.21.15.172:0/1844749869 wait complete. 2024-04-14T08:33:10.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.393+0000 7f1e3362d640 1 Processor -- start 2024-04-14T08:33:10.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.393+0000 7f1e3362d640 1 -- start start 2024-04-14T08:33:10.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.393+0000 7f1e3362d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e2c0b1df0 0x7f1e2c1572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:10.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.393+0000 7f1e3362d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e2c0b21d0 0x7f1e2c138810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:10.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.393+0000 7f1e3362d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e2c13b9d0 0x7f1e2c13bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:10.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.393+0000 7f1e3362d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c0b4750 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.393+0000 7f1e3362d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c15b7f0 con 0x7f1e2c0b21d0 2024-04-14T08:33:10.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.393+0000 7f1e3362d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1e2c0b48d0 con 0x7f1e2c13b9d0 2024-04-14T08:33:10.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.393+0000 7f1e313c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e2c0b1df0 0x7f1e2c1572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:10.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.393+0000 7f1e313c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e2c0b1df0 0x7f1e2c1572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49288/0 (socket says 172.21.15.172:49288) 2024-04-14T08:33:10.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.393+0000 7f1e313c9640 1 -- 172.21.15.172:0/3919779060 learned_addr learned my addr 172.21.15.172:0/3919779060 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:10.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.393+0000 7f1e30bc8640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e2c0b21d0 0x7f1e2c138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.394+0000 7f1e31bca640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e2c13b9d0 0x7f1e2c13bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.394+0000 7f1e313c9640 1 -- 172.21.15.172:0/3919779060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e2c13b9d0 msgr2=0x7f1e2c13bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.394+0000 7f1e313c9640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e2c13b9d0 0x7f1e2c13bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.394+0000 7f1e313c9640 1 -- 172.21.15.172:0/3919779060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e2c0b21d0 msgr2=0x7f1e2c138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.394+0000 7f1e313c9640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e2c0b21d0 0x7f1e2c138810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.394+0000 7f1e313c9640 1 -- 172.21.15.172:0/3919779060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2c157950 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.394+0000 7f1e313c9640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e2c0b1df0 0x7f1e2c1572c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c034040 tx=0x7f1e1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.394+0000 7f1e21ffb640 1 -- 172.21.15.172:0/3919779060 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c040070 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.394+0000 7f1e21ffb640 1 -- 172.21.15.172:0/3919779060 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1c03a910 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.394+0000 7f1e21ffb640 1 -- 172.21.15.172:0/3919779060 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c035040 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.394+0000 7f1e3362d640 1 -- 172.21.15.172:0/3919779060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2c0ce920 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.394+0000 7f1e3362d640 1 -- 172.21.15.172:0/3919779060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2c0ceee0 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.395+0000 7f1e3362d640 1 -- 172.21.15.172:0/3919779060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2c0ccdc0 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.396+0000 7f1e21ffb640 1 -- 172.21.15.172:0/3919779060 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e1c038020 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.396+0000 7f1e21ffb640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1e1006c8f0 0x7f1e1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:10.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.396+0000 7f1e21ffb640 1 -- 172.21.15.172:0/3919779060 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1e1c0ad6f0 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.396+0000 7f1e30bc8640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1e1006c8f0 0x7f1e1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:10.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.397+0000 7f1e30bc8640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1e1006c8f0 0x7f1e1006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009560 tx=0x7f1e28009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:10.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.400+0000 7f1e21ffb640 1 -- 172.21.15.172:0/3919779060 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1e2c0ccdc0 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.613+0000 7f1e3362d640 1 -- 172.21.15.172:0/3919779060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e2c068520 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.614+0000 7f1e21ffb640 1 -- 172.21.15.172:0/3919779060 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f1e1c031800 con 0x7f1e2c0b1df0 2024-04-14T08:33:10.615 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:10.615 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3053874257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3053874257},{"type":"v1","addr":"172.21.15.172:6835","nonce":3053874257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2790588849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2790588849},{"type":"v1","addr":"172.21.15.172:6837","nonce":2790588849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3130776573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3130776573},{"type":"v1","addr":"172.21.15.177:6834","nonce":3130776573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:32:53.535483+0000","modified":"2024-04-14T08:33:09.604218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/517963877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":517963877},{"type":"v1","addr":"172.21.15.177:6835","nonce":517963877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4309,"qdb_cluster":[4309]},"id":1}]} 2024-04-14T08:33:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 9 2024-04-14T08:33:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.616+0000 7f1e3362d640 1 -- 172.21.15.172:0/3919779060 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1e1006c8f0 msgr2=0x7f1e1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.616+0000 7f1e3362d640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1e1006c8f0 0x7f1e1006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009560 tx=0x7f1e28009190 comp rx=0 tx=0).stop 2024-04-14T08:33:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.616+0000 7f1e3362d640 1 -- 172.21.15.172:0/3919779060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e2c0b1df0 msgr2=0x7f1e2c1572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.616+0000 7f1e3362d640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e2c0b1df0 0x7f1e2c1572c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c034040 tx=0x7f1e1c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.616+0000 7f1e3362d640 1 -- 172.21.15.172:0/3919779060 shutdown_connections 2024-04-14T08:33:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.616+0000 7f1e3362d640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1e1006c8f0 0x7f1e1006edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.616+0000 7f1e3362d640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e2c13b9d0 0x7f1e2c13bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.616+0000 7f1e3362d640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e2c0b21d0 0x7f1e2c138810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.616+0000 7f1e3362d640 1 --2- 172.21.15.172:0/3919779060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e2c0b1df0 0x7f1e2c1572c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.616+0000 7f1e3362d640 1 -- 172.21.15.172:0/3919779060 >> 172.21.15.172:0/3919779060 conn(0x7f1e2c06ec30 msgr2=0x7f1e2c0b4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.616+0000 7f1e3362d640 1 -- 172.21.15.172:0/3919779060 shutdown_connections 2024-04-14T08:33:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.616+0000 7f1e3362d640 1 -- 172.21.15.172:0/3919779060 wait complete. 2024-04-14T08:33:10.628 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:10.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.727+0000 7fc947c36640 1 Processor -- start 2024-04-14T08:33:10.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.727+0000 7fc947c36640 1 -- start start 2024-04-14T08:33:10.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.728+0000 7fc947c36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc9400b7a60 0x7fc9400ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:10.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.728+0000 7fc947c36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc94008c4c0 0x7fc9400cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:10.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.728+0000 7fc947c36640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9400d2b10 con 0x7fc9400cd410 2024-04-14T08:33:10.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.728+0000 7fc947c36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc9400d0ad0 con 0x7fc9400b7a60 2024-04-14T08:33:10.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.728+0000 7fc947c36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc9400d0db0 con 0x7fc94008c4c0 2024-04-14T08:33:10.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.728+0000 7fc9459d2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc9400b7a60 0x7fc9400ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:10.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.728+0000 7fc9461d3640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc9400cd410 0x7fc9400cd880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35578/0 (socket says 172.21.15.172:35578) 2024-04-14T08:33:10.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.728+0000 7fc9461d3640 1 -- 172.21.15.172:0/3127856670 learned_addr learned my addr 172.21.15.172:0/3127856670 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:10.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.728+0000 7fc9451d1640 1 --2- 172.21.15.172:0/3127856670 >> v2:172.21.15.177:3300/0 conn(0x7fc94008c4c0 0x7fc9400cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:10.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.729+0000 7fc9459d2640 1 -- 172.21.15.172:0/3127856670 >> v2:172.21.15.177:3300/0 conn(0x7fc94008c4c0 msgr2=0x7fc9400cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:10.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.729+0000 7fc9459d2640 1 --2- 172.21.15.172:0/3127856670 >> v2:172.21.15.177:3300/0 conn(0x7fc94008c4c0 0x7fc9400cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.729+0000 7fc9459d2640 1 -- 172.21.15.172:0/3127856670 >> v1:172.21.15.172:6789/0 conn(0x7fc9400cd410 legacy=0x7fc9400cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.729+0000 7fc9459d2640 1 -- 172.21.15.172:0/3127856670 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9400d1090 con 0x7fc9400b7a60 2024-04-14T08:33:10.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.729+0000 7fc9449d0640 1 -- 172.21.15.172:0/3127856670 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1090572644 0 0) 0x7fc9400d2b10 con 0x7fc9400cd410 2024-04-14T08:33:10.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.729+0000 7fc9451d1640 1 --2- 172.21.15.172:0/3127856670 >> v2:172.21.15.177:3300/0 conn(0x7fc94008c4c0 0x7fc9400cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:10.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.729+0000 7fc9459d2640 1 --2- 172.21.15.172:0/3127856670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9400b7a60 0x7fc9400ce8c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc940069090 tx=0x7fc93c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3963b3c39bd000d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:10.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.730+0000 7fc9449d0640 1 -- 172.21.15.172:0/3127856670 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c039070 con 0x7fc9400b7a60 2024-04-14T08:33:10.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.730+0000 7fc9449d0640 1 -- 172.21.15.172:0/3127856670 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc93c02cbf0 con 0x7fc9400b7a60 2024-04-14T08:33:10.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.730+0000 7fc9449d0640 1 -- 172.21.15.172:0/3127856670 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c034440 con 0x7fc9400b7a60 2024-04-14T08:33:10.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.730+0000 7fc947c36640 1 -- 172.21.15.172:0/3127856670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9400b7a60 msgr2=0x7fc9400ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.730+0000 7fc947c36640 1 --2- 172.21.15.172:0/3127856670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9400b7a60 0x7fc9400ce8c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc940069090 tx=0x7fc93c02be40 comp rx=0 tx=0).stop 2024-04-14T08:33:10.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.730+0000 7fc947c36640 1 -- 172.21.15.172:0/3127856670 shutdown_connections 2024-04-14T08:33:10.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.730+0000 7fc947c36640 1 --2- 172.21.15.172:0/3127856670 >> v2:172.21.15.177:3300/0 conn(0x7fc94008c4c0 0x7fc9400cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.730+0000 7fc947c36640 1 --2- 172.21.15.172:0/3127856670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9400b7a60 0x7fc9400ce8c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.730+0000 7fc947c36640 1 -- 172.21.15.172:0/3127856670 >> 172.21.15.172:0/3127856670 conn(0x7fc94008c000 msgr2=0x7fc9400d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.730+0000 7fc947c36640 1 -- 172.21.15.172:0/3127856670 shutdown_connections 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.730+0000 7fc947c36640 1 -- 172.21.15.172:0/3127856670 wait complete. 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.731+0000 7fc947c36640 1 Processor -- start 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.731+0000 7fc947c36640 1 -- start start 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.732+0000 7fc947c36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc94008c4c0 0x7fc9400af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.732+0000 7fc947c36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400b7a60 0x7fc940161c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.732+0000 7fc947c36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9400cd410 0x7fc9400b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.732+0000 7fc947c36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc9400d2750 con 0x7fc9400cd410 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.732+0000 7fc947c36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9400b7e40 con 0x7fc9400b7a60 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.732+0000 7fc947c36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc9400d28d0 con 0x7fc94008c4c0 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.732+0000 7fc9461d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9400cd410 0x7fc9400b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:10.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.732+0000 7fc9459d2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc94008c4c0 0x7fc9400af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.732+0000 7fc9459d2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc94008c4c0 0x7fc9400af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:42926/0 (socket says 172.21.15.172:42926) 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.732+0000 7fc9461d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9400cd410 0x7fc9400b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49306/0 (socket says 172.21.15.172:49306) 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.732+0000 7fc9459d2640 1 -- 172.21.15.172:0/3539950174 learned_addr learned my addr 172.21.15.172:0/3539950174 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.732+0000 7fc9451d1640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400b7a60 0x7fc940161c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc9459d2640 1 -- 172.21.15.172:0/3539950174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400b7a60 msgr2=0x7fc940161c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc9459d2640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400b7a60 0x7fc940161c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc9459d2640 1 -- 172.21.15.172:0/3539950174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9400cd410 msgr2=0x7fc9400b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc9459d2640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9400cd410 0x7fc9400b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc9459d2640 1 -- 172.21.15.172:0/3539950174 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9400afde0 con 0x7fc94008c4c0 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc9451d1640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400b7a60 0x7fc940161c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc9461d3640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9400cd410 0x7fc9400b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc9459d2640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc94008c4c0 0x7fc9400af720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc93c037040 tx=0x7fc93c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc9367fc640 1 -- 172.21.15.172:0/3539950174 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c039050 con 0x7fc94008c4c0 2024-04-14T08:33:10.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc9367fc640 1 -- 172.21.15.172:0/3539950174 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc93c045070 con 0x7fc94008c4c0 2024-04-14T08:33:10.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc947c36640 1 -- 172.21.15.172:0/3539950174 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9400b00c0 con 0x7fc94008c4c0 2024-04-14T08:33:10.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc9367fc640 1 -- 172.21.15.172:0/3539950174 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c03e850 con 0x7fc94008c4c0 2024-04-14T08:33:10.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.733+0000 7fc947c36640 1 -- 172.21.15.172:0/3539950174 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9400b10f0 con 0x7fc94008c4c0 2024-04-14T08:33:10.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.734+0000 7fc947c36640 1 -- 172.21.15.172:0/3539950174 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc940075490 con 0x7fc94008c4c0 2024-04-14T08:33:10.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.734+0000 7fc9367fc640 1 -- 172.21.15.172:0/3539950174 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc93c03e9f0 con 0x7fc94008c4c0 2024-04-14T08:33:10.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.735+0000 7fc9367fc640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc92406c920 0x7fc92406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:10.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.735+0000 7fc9367fc640 1 -- 172.21.15.172:0/3539950174 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc93c0ae410 con 0x7fc94008c4c0 2024-04-14T08:33:10.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.735+0000 7fc9451d1640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc92406c920 0x7fc92406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:10.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.735+0000 7fc9451d1640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc92406c920 0x7fc92406ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc94006d620 tx=0x7fc930009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:10.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.740+0000 7fc9367fc640 1 -- 172.21.15.172:0/3539950174 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc93c04e020 con 0x7fc94008c4c0 2024-04-14T08:33:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.916+0000 7fc947c36640 1 -- 172.21.15.172:0/3539950174 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc94015f100 con 0x7fc94008c4c0 2024-04-14T08:33:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.917+0000 7fc9367fc640 1 -- 172.21.15.172:0/3539950174 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7fc93c07bf20 con 0x7fc94008c4c0 2024-04-14T08:33:10.918 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:10.918 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.919+0000 7fc947c36640 1 -- 172.21.15.172:0/3539950174 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc92406c920 msgr2=0x7fc92406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.919+0000 7fc947c36640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc92406c920 0x7fc92406ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc94006d620 tx=0x7fc930009190 comp rx=0 tx=0).stop 2024-04-14T08:33:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.919+0000 7fc947c36640 1 -- 172.21.15.172:0/3539950174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc94008c4c0 msgr2=0x7fc9400af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.919+0000 7fc947c36640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc94008c4c0 0x7fc9400af720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc93c037040 tx=0x7fc93c004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.919+0000 7fc947c36640 1 -- 172.21.15.172:0/3539950174 shutdown_connections 2024-04-14T08:33:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.919+0000 7fc947c36640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc92406c920 0x7fc92406ede0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.919+0000 7fc947c36640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9400cd410 0x7fc9400b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.919+0000 7fc947c36640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9400b7a60 0x7fc940161c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.920+0000 7fc947c36640 1 --2- 172.21.15.172:0/3539950174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc94008c4c0 0x7fc9400af720 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.920+0000 7fc947c36640 1 -- 172.21.15.172:0/3539950174 >> 172.21.15.172:0/3539950174 conn(0x7fc94008c000 msgr2=0x7fc94006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.920+0000 7fc947c36640 1 -- 172.21.15.172:0/3539950174 shutdown_connections 2024-04-14T08:33:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:10.920+0000 7fc947c36640 1 -- 172.21.15.172:0/3539950174 wait complete. 2024-04-14T08:33:10.931 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-14T08:33:10.932 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:33:11.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.032+0000 7fe9f535e640 1 Processor -- start 2024-04-14T08:33:11.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.032+0000 7fe9f535e640 1 -- start start 2024-04-14T08:33:11.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.033+0000 7fe9f535e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f00cefd0 0x7fe9f006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.033+0000 7fe9f535e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe9f008c4c0 0x7fe9f006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.033+0000 7fe9f535e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9f00d2b10 con 0x7fe9f00b7a60 2024-04-14T08:33:11.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.033+0000 7fe9f535e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe9f00ce420 con 0x7fe9f008c4c0 2024-04-14T08:33:11.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.033+0000 7fe9f535e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe9f00d0ad0 con 0x7fe9f00cefd0 2024-04-14T08:33:11.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.033+0000 7fe9ee7fc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f00cefd0 0x7fe9f006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.033+0000 7fe9eeffd640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fe9f00b7a60 0x7fe9f00ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42620/0 (socket says 172.21.15.172:42620) 2024-04-14T08:33:11.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.033+0000 7fe9ef7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe9f008c4c0 0x7fe9f006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.033+0000 7fe9eeffd640 1 -- 172.21.15.172:0/2889159788 learned_addr learned my addr 172.21.15.172:0/2889159788 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:11.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.034+0000 7fe9ee7fc640 1 -- 172.21.15.172:0/2889159788 >> v2:172.21.15.177:3300/0 conn(0x7fe9f008c4c0 msgr2=0x7fe9f006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:11.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.034+0000 7fe9ee7fc640 1 --2- 172.21.15.172:0/2889159788 >> v2:172.21.15.177:3300/0 conn(0x7fe9f008c4c0 0x7fe9f006d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.034+0000 7fe9ee7fc640 1 -- 172.21.15.172:0/2889159788 >> v1:172.21.15.177:6789/0 conn(0x7fe9f00b7a60 legacy=0x7fe9f00ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.034+0000 7fe9ee7fc640 1 -- 172.21.15.172:0/2889159788 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f00d0db0 con 0x7fe9f00cefd0 2024-04-14T08:33:11.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.034+0000 7fe9edffb640 1 -- 172.21.15.172:0/2889159788 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367009383 0 0) 0x7fe9f00d2b10 con 0x7fe9f00b7a60 2024-04-14T08:33:11.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.034+0000 7fe9ef7fe640 1 --2- 172.21.15.172:0/2889159788 >> v2:172.21.15.177:3300/0 conn(0x7fe9f008c4c0 0x7fe9f006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:11.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.034+0000 7fe9ee7fc640 1 --2- 172.21.15.172:0/2889159788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f00cefd0 0x7fe9f006cc30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40097a0 tx=0x7fe9e402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7dfc37d47e4e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:11.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.035+0000 7fe9edffb640 1 -- 172.21.15.172:0/2889159788 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40315a0 con 0x7fe9f00cefd0 2024-04-14T08:33:11.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.035+0000 7fe9edffb640 1 -- 172.21.15.172:0/2889159788 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e4037070 con 0x7fe9f00cefd0 2024-04-14T08:33:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.035+0000 7fe9edffb640 1 -- 172.21.15.172:0/2889159788 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4031960 con 0x7fe9f00cefd0 2024-04-14T08:33:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.035+0000 7fe9f535e640 1 -- 172.21.15.172:0/2889159788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f00cefd0 msgr2=0x7fe9f006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.035+0000 7fe9f535e640 1 --2- 172.21.15.172:0/2889159788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f00cefd0 0x7fe9f006cc30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40097a0 tx=0x7fe9e402b840 comp rx=0 tx=0).stop 2024-04-14T08:33:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.035+0000 7fe9f535e640 1 -- 172.21.15.172:0/2889159788 shutdown_connections 2024-04-14T08:33:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.035+0000 7fe9f535e640 1 --2- 172.21.15.172:0/2889159788 >> v2:172.21.15.177:3300/0 conn(0x7fe9f008c4c0 0x7fe9f006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.035+0000 7fe9f535e640 1 --2- 172.21.15.172:0/2889159788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f00cefd0 0x7fe9f006cc30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.035+0000 7fe9f535e640 1 -- 172.21.15.172:0/2889159788 >> 172.21.15.172:0/2889159788 conn(0x7fe9f008c000 msgr2=0x7fe9f00d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.035+0000 7fe9f535e640 1 -- 172.21.15.172:0/2889159788 shutdown_connections 2024-04-14T08:33:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.035+0000 7fe9f535e640 1 -- 172.21.15.172:0/2889159788 wait complete. 2024-04-14T08:33:11.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.036+0000 7fe9f535e640 1 Processor -- start 2024-04-14T08:33:11.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.037+0000 7fe9f535e640 1 -- start start 2024-04-14T08:33:11.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.037+0000 7fe9f535e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9f008c4c0 0x7fe9f0160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.037+0000 7fe9f535e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9f00b7a60 0x7fe9f00c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.037+0000 7fe9f535e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f00cefd0 0x7fe9f00c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.037+0000 7fe9f535e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f00b7e40 con 0x7fe9f00b7a60 2024-04-14T08:33:11.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.037+0000 7fe9f535e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f00b4e90 con 0x7fe9f008c4c0 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.037+0000 7fe9f535e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe9f00d2750 con 0x7fe9f00cefd0 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.037+0000 7fe9ee7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9f00b7a60 0x7fe9f00c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.037+0000 7fe9ee7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9f00b7a60 0x7fe9f00c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49314/0 (socket says 172.21.15.172:49314) 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.037+0000 7fe9eeffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9f008c4c0 0x7fe9f0160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.037+0000 7fe9ee7fc640 1 -- 172.21.15.172:0/366817472 learned_addr learned my addr 172.21.15.172:0/366817472 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.037+0000 7fe9ef7fe640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f00cefd0 0x7fe9f00c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.038+0000 7fe9eeffd640 1 -- 172.21.15.172:0/366817472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f00cefd0 msgr2=0x7fe9f00c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.038+0000 7fe9eeffd640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f00cefd0 0x7fe9f00c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.038+0000 7fe9eeffd640 1 -- 172.21.15.172:0/366817472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9f00b7a60 msgr2=0x7fe9f00c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.038+0000 7fe9eeffd640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9f00b7a60 0x7fe9f00c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.038+0000 7fe9eeffd640 1 -- 172.21.15.172:0/366817472 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f00c5dc0 con 0x7fe9f008c4c0 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.038+0000 7fe9ee7fc640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9f00b7a60 0x7fe9f00c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.038+0000 7fe9eeffd640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9f008c4c0 0x7fe9f0160ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000c780 tx=0x7fe9e000cb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.038+0000 7fe9d77fe640 1 -- 172.21.15.172:0/366817472 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0018070 con 0x7fe9f008c4c0 2024-04-14T08:33:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.038+0000 7fe9d77fe640 1 -- 172.21.15.172:0/366817472 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e0004d80 con 0x7fe9f008c4c0 2024-04-14T08:33:11.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.038+0000 7fe9f535e640 1 -- 172.21.15.172:0/366817472 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f00c60a0 con 0x7fe9f008c4c0 2024-04-14T08:33:11.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.038+0000 7fe9d77fe640 1 -- 172.21.15.172:0/366817472 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000a820 con 0x7fe9f008c4c0 2024-04-14T08:33:11.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.038+0000 7fe9f535e640 1 -- 172.21.15.172:0/366817472 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f00c6660 con 0x7fe9f008c4c0 2024-04-14T08:33:11.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.039+0000 7fe9f535e640 1 -- 172.21.15.172:0/366817472 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f0075490 con 0x7fe9f008c4c0 2024-04-14T08:33:11.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.039+0000 7fe9d77fe640 1 -- 172.21.15.172:0/366817472 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9e001b070 con 0x7fe9f008c4c0 2024-04-14T08:33:11.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.040+0000 7fe9d77fe640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe9dc06c8f0 0x7fe9dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.040+0000 7fe9d77fe640 1 -- 172.21.15.172:0/366817472 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe9e008b300 con 0x7fe9f008c4c0 2024-04-14T08:33:11.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.040+0000 7fe9ee7fc640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe9dc06c8f0 0x7fe9dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.040+0000 7fe9ee7fc640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe9dc06c8f0 0x7fe9dc06edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe9e402b880 tx=0x7fe9e403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:11.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.044+0000 7fe9d77fe640 1 -- 172.21.15.172:0/366817472 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe9e0058d00 con 0x7fe9f008c4c0 2024-04-14T08:33:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.218+0000 7fe9f535e640 1 -- 172.21.15.172:0/366817472 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9f00741a0 con 0x7fe9dc06c8f0 2024-04-14T08:33:11.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.226+0000 7fe9d77fe640 1 -- 172.21.15.172:0/366817472 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fe9f00741a0 con 0x7fe9dc06c8f0 2024-04-14T08:33:11.231 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:11.233 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:33:11.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.230+0000 7fe9f535e640 1 -- 172.21.15.172:0/366817472 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe9dc06c8f0 msgr2=0x7fe9dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.230+0000 7fe9f535e640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe9dc06c8f0 0x7fe9dc06edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe9e402b880 tx=0x7fe9e403d040 comp rx=0 tx=0).stop 2024-04-14T08:33:11.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.230+0000 7fe9f535e640 1 -- 172.21.15.172:0/366817472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9f008c4c0 msgr2=0x7fe9f0160ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.230+0000 7fe9f535e640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9f008c4c0 0x7fe9f0160ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000c780 tx=0x7fe9e000cb80 comp rx=0 tx=0).stop 2024-04-14T08:33:11.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.230+0000 7fe9f535e640 1 -- 172.21.15.172:0/366817472 shutdown_connections 2024-04-14T08:33:11.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.230+0000 7fe9f535e640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe9dc06c8f0 0x7fe9dc06edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.230+0000 7fe9f535e640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9f00cefd0 0x7fe9f00c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.230+0000 7fe9f535e640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9f00b7a60 0x7fe9f00c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.230+0000 7fe9f535e640 1 --2- 172.21.15.172:0/366817472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9f008c4c0 0x7fe9f0160ea0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.230+0000 7fe9f535e640 1 -- 172.21.15.172:0/366817472 >> 172.21.15.172:0/366817472 conn(0x7fe9f008c000 msgr2=0x7fe9f0161760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:11.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.230+0000 7fe9f535e640 1 -- 172.21.15.172:0/366817472 shutdown_connections 2024-04-14T08:33:11.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.230+0000 7fe9f535e640 1 -- 172.21.15.172:0/366817472 wait complete. 2024-04-14T08:33:11.241 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-14T08:33:11.124560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219480,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755240,"statfs":{"total":773094113280,"available":772869365760,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.901572"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219633+0000","last_change":"2024-04-14T08:32:53.196809+0000","last_active":"2024-04-14T08:33:01.219633+0000","last_peered":"2024-04-14T08:33:01.219633+0000","last_clean":"2024-04-14T08:33:01.219633+0000","last_became_active":"2024-04-14T08:32:53.196554+0000","last_became_peered":"2024-04-14T08:32:53.196554+0000","last_unstale":"2024-04-14T08:33:01.219633+0000","last_undegraded":"2024-04-14T08:33:01.219633+0000","last_fullsized":"2024-04-14T08:33:01.219633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:43:41.734066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225244+0000","last_change":"2024-04-14T08:32:54.204652+0000","last_active":"2024-04-14T08:32:59.225244+0000","last_peered":"2024-04-14T08:32:59.225244+0000","last_clean":"2024-04-14T08:32:59.225244+0000","last_became_active":"2024-04-14T08:32:54.204361+0000","last_became_peered":"2024-04-14T08:32:54.204361+0000","last_unstale":"2024-04-14T08:32:59.225244+0000","last_undegraded":"2024-04-14T08:32:59.225244+0000","last_fullsized":"2024-04-14T08:32:59.225244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224356+0000","last_change":"2024-04-14T08:32:58.244817+0000","last_active":"2024-04-14T08:32:59.224356+0000","last_peered":"2024-04-14T08:32:59.224356+0000","last_clean":"2024-04-14T08:32:59.224356+0000","last_became_active":"2024-04-14T08:32:58.243913+0000","last_became_peered":"2024-04-14T08:32:58.243913+0000","last_unstale":"2024-04-14T08:32:59.224356+0000","last_undegraded":"2024-04-14T08:32:59.224356+0000","last_fullsized":"2024-04-14T08:32:59.224356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:53.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225340+0000","last_change":"2024-04-14T08:32:53.202277+0000","last_active":"2024-04-14T08:32:59.225340+0000","last_peered":"2024-04-14T08:32:59.225340+0000","last_clean":"2024-04-14T08:32:59.225340+0000","last_became_active":"2024-04-14T08:32:53.201980+0000","last_became_peered":"2024-04-14T08:32:53.201980+0000","last_unstale":"2024-04-14T08:32:59.225340+0000","last_undegraded":"2024-04-14T08:32:59.225340+0000","last_fullsized":"2024-04-14T08:32:59.225340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:22:15.886932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220320+0000","last_change":"2024-04-14T08:32:54.211674+0000","last_active":"2024-04-14T08:33:01.220320+0000","last_peered":"2024-04-14T08:33:01.220320+0000","last_clean":"2024-04-14T08:33:01.220320+0000","last_became_active":"2024-04-14T08:32:54.211323+0000","last_became_peered":"2024-04-14T08:32:54.211323+0000","last_unstale":"2024-04-14T08:33:01.220320+0000","last_undegraded":"2024-04-14T08:33:01.220320+0000","last_fullsized":"2024-04-14T08:33:01.220320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224678+0000","last_change":"2024-04-14T08:32:58.242594+0000","last_active":"2024-04-14T08:32:59.224678+0000","last_peered":"2024-04-14T08:32:59.224678+0000","last_clean":"2024-04-14T08:32:59.224678+0000","last_became_active":"2024-04-14T08:32:58.242238+0000","last_became_peered":"2024-04-14T08:32:58.242238+0000","last_unstale":"2024-04-14T08:32:59.224678+0000","last_undegraded":"2024-04-14T08:32:59.224678+0000","last_fullsized":"2024-04-14T08:32:59.224678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:45.716305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226152+0000","last_change":"2024-04-14T08:32:53.200133+0000","last_active":"2024-04-14T08:32:59.226152+0000","last_peered":"2024-04-14T08:32:59.226152+0000","last_clean":"2024-04-14T08:32:59.226152+0000","last_became_active":"2024-04-14T08:32:53.199380+0000","last_became_peered":"2024-04-14T08:32:53.199380+0000","last_unstale":"2024-04-14T08:32:59.226152+0000","last_undegraded":"2024-04-14T08:32:59.226152+0000","last_fullsized":"2024-04-14T08:32:59.226152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405017+0000","last_change":"2024-04-14T08:32:54.208922+0000","last_active":"2024-04-14T08:32:59.405017+0000","last_peered":"2024-04-14T08:32:59.405017+0000","last_clean":"2024-04-14T08:32:59.405017+0000","last_became_active":"2024-04-14T08:32:54.206845+0000","last_became_peered":"2024-04-14T08:32:54.206845+0000","last_unstale":"2024-04-14T08:32:59.405017+0000","last_undegraded":"2024-04-14T08:32:59.405017+0000","last_fullsized":"2024-04-14T08:32:59.405017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225258+0000","last_change":"2024-04-14T08:32:58.242683+0000","last_active":"2024-04-14T08:32:59.225258+0000","last_peered":"2024-04-14T08:32:59.225258+0000","last_clean":"2024-04-14T08:32:59.225258+0000","last_became_active":"2024-04-14T08:32:58.242189+0000","last_became_peered":"2024-04-14T08:32:58.242189+0000","last_unstale":"2024-04-14T08:32:59.225258+0000","last_undegraded":"2024-04-14T08:32:59.225258+0000","last_fullsized":"2024-04-14T08:32:59.225258+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:52.084133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225875+0000","last_change":"2024-04-14T08:32:53.200880+0000","last_active":"2024-04-14T08:32:59.225875+0000","last_peered":"2024-04-14T08:32:59.225875+0000","last_clean":"2024-04-14T08:32:59.225875+0000","last_became_active":"2024-04-14T08:32:53.200615+0000","last_became_peered":"2024-04-14T08:32:53.200615+0000","last_unstale":"2024-04-14T08:32:59.225875+0000","last_undegraded":"2024-04-14T08:32:59.225875+0000","last_fullsized":"2024-04-14T08:32:59.225875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:40:24.515179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405104+0000","last_change":"2024-04-14T08:32:54.207091+0000","last_active":"2024-04-14T08:32:59.405104+0000","last_peered":"2024-04-14T08:32:59.405104+0000","last_clean":"2024-04-14T08:32:59.405104+0000","last_became_active":"2024-04-14T08:32:54.206685+0000","last_became_peered":"2024-04-14T08:32:54.206685+0000","last_unstale":"2024-04-14T08:32:59.405104+0000","last_undegraded":"2024-04-14T08:32:59.405104+0000","last_fullsized":"2024-04-14T08:32:59.405104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:41:14.768103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224936+0000","last_change":"2024-04-14T08:32:58.237435+0000","last_active":"2024-04-14T08:32:59.224936+0000","last_peered":"2024-04-14T08:32:59.224936+0000","last_clean":"2024-04-14T08:32:59.224936+0000","last_became_active":"2024-04-14T08:32:58.237056+0000","last_became_peered":"2024-04-14T08:32:58.237056+0000","last_unstale":"2024-04-14T08:32:59.224936+0000","last_undegraded":"2024-04-14T08:32:59.224936+0000","last_fullsized":"2024-04-14T08:32:59.224936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:20.577556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115125+0000","last_change":"2024-04-14T08:32:54.202442+0000","last_active":"2024-04-14T08:33:01.115125+0000","last_peered":"2024-04-14T08:33:01.115125+0000","last_clean":"2024-04-14T08:33:01.115125+0000","last_became_active":"2024-04-14T08:32:54.201860+0000","last_became_peered":"2024-04-14T08:32:54.201860+0000","last_unstale":"2024-04-14T08:33:01.115125+0000","last_undegraded":"2024-04-14T08:33:01.115125+0000","last_fullsized":"2024-04-14T08:33:01.115125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226036+0000","last_change":"2024-04-14T08:32:53.194232+0000","last_active":"2024-04-14T08:32:59.226036+0000","last_peered":"2024-04-14T08:32:59.226036+0000","last_clean":"2024-04-14T08:32:59.226036+0000","last_became_active":"2024-04-14T08:32:53.193961+0000","last_became_peered":"2024-04-14T08:32:53.193961+0000","last_unstale":"2024-04-14T08:32:59.226036+0000","last_undegraded":"2024-04-14T08:32:59.226036+0000","last_fullsized":"2024-04-14T08:32:59.226036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:32.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226274+0000","last_change":"2024-04-14T08:32:58.239778+0000","last_active":"2024-04-14T08:32:59.226274+0000","last_peered":"2024-04-14T08:32:59.226274+0000","last_clean":"2024-04-14T08:32:59.226274+0000","last_became_active":"2024-04-14T08:32:58.239445+0000","last_became_peered":"2024-04-14T08:32:58.239445+0000","last_unstale":"2024-04-14T08:32:59.226274+0000","last_undegraded":"2024-04-14T08:32:59.226274+0000","last_fullsized":"2024-04-14T08:32:59.226274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:53.001768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224657+0000","last_change":"2024-04-14T08:32:54.202322+0000","last_active":"2024-04-14T08:32:59.224657+0000","last_peered":"2024-04-14T08:32:59.224657+0000","last_clean":"2024-04-14T08:32:59.224657+0000","last_became_active":"2024-04-14T08:32:54.202065+0000","last_became_peered":"2024-04-14T08:32:54.202065+0000","last_unstale":"2024-04-14T08:32:59.224657+0000","last_undegraded":"2024-04-14T08:32:59.224657+0000","last_fullsized":"2024-04-14T08:32:59.224657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610551+0000","last_change":"2024-04-14T08:32:53.195288+0000","last_active":"2024-04-14T08:33:00.610551+0000","last_peered":"2024-04-14T08:33:00.610551+0000","last_clean":"2024-04-14T08:33:00.610551+0000","last_became_active":"2024-04-14T08:32:53.195009+0000","last_became_peered":"2024-04-14T08:32:53.195009+0000","last_unstale":"2024-04-14T08:33:00.610551+0000","last_undegraded":"2024-04-14T08:33:00.610551+0000","last_fullsized":"2024-04-14T08:33:00.610551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225775+0000","last_change":"2024-04-14T08:32:58.238192+0000","last_active":"2024-04-14T08:32:59.225775+0000","last_peered":"2024-04-14T08:32:59.225775+0000","last_clean":"2024-04-14T08:32:59.225775+0000","last_became_active":"2024-04-14T08:32:58.237917+0000","last_became_peered":"2024-04-14T08:32:58.237917+0000","last_unstale":"2024-04-14T08:32:59.225775+0000","last_undegraded":"2024-04-14T08:32:59.225775+0000","last_fullsized":"2024-04-14T08:32:59.225775+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:41:18.809183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224457+0000","last_change":"2024-04-14T08:32:54.195485+0000","last_active":"2024-04-14T08:32:59.224457+0000","last_peered":"2024-04-14T08:32:59.224457+0000","last_clean":"2024-04-14T08:32:59.224457+0000","last_became_active":"2024-04-14T08:32:54.195192+0000","last_became_peered":"2024-04-14T08:32:54.195192+0000","last_unstale":"2024-04-14T08:32:59.224457+0000","last_undegraded":"2024-04-14T08:32:59.224457+0000","last_fullsized":"2024-04-14T08:32:59.224457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610087+0000","last_change":"2024-04-14T08:32:53.193700+0000","last_active":"2024-04-14T08:33:00.610087+0000","last_peered":"2024-04-14T08:33:00.610087+0000","last_clean":"2024-04-14T08:33:00.610087+0000","last_became_active":"2024-04-14T08:32:53.193412+0000","last_became_peered":"2024-04-14T08:32:53.193412+0000","last_unstale":"2024-04-14T08:33:00.610087+0000","last_undegraded":"2024-04-14T08:33:00.610087+0000","last_fullsized":"2024-04-14T08:33:00.610087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610793+0000","last_change":"2024-04-14T08:32:58.246138+0000","last_active":"2024-04-14T08:33:00.610793+0000","last_peered":"2024-04-14T08:33:00.610793+0000","last_clean":"2024-04-14T08:33:00.610793+0000","last_became_active":"2024-04-14T08:32:58.245731+0000","last_became_peered":"2024-04-14T08:32:58.245731+0000","last_unstale":"2024-04-14T08:33:00.610793+0000","last_undegraded":"2024-04-14T08:33:00.610793+0000","last_fullsized":"2024-04-14T08:33:00.610793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:54.763380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224951+0000","last_change":"2024-04-14T08:32:54.203712+0000","last_active":"2024-04-14T08:32:59.224951+0000","last_peered":"2024-04-14T08:32:59.224951+0000","last_clean":"2024-04-14T08:32:59.224951+0000","last_became_active":"2024-04-14T08:32:54.203433+0000","last_became_peered":"2024-04-14T08:32:54.203433+0000","last_unstale":"2024-04-14T08:32:59.224951+0000","last_undegraded":"2024-04-14T08:32:59.224951+0000","last_fullsized":"2024-04-14T08:32:59.224951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:15.534364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224734+0000","last_change":"2024-04-14T08:32:53.197850+0000","last_active":"2024-04-14T08:32:59.224734+0000","last_peered":"2024-04-14T08:32:59.224734+0000","last_clean":"2024-04-14T08:32:59.224734+0000","last_became_active":"2024-04-14T08:32:53.197575+0000","last_became_peered":"2024-04-14T08:32:53.197575+0000","last_unstale":"2024-04-14T08:32:59.224734+0000","last_undegraded":"2024-04-14T08:32:59.224734+0000","last_fullsized":"2024-04-14T08:32:59.224734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:04:36.870813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225529+0000","last_change":"2024-04-14T08:32:58.243185+0000","last_active":"2024-04-14T08:32:59.225529+0000","last_peered":"2024-04-14T08:32:59.225529+0000","last_clean":"2024-04-14T08:32:59.225529+0000","last_became_active":"2024-04-14T08:32:58.242880+0000","last_became_peered":"2024-04-14T08:32:58.242880+0000","last_unstale":"2024-04-14T08:32:59.225529+0000","last_undegraded":"2024-04-14T08:32:59.225529+0000","last_fullsized":"2024-04-14T08:32:59.225529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:47.641204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220461+0000","last_change":"2024-04-14T08:32:54.194367+0000","last_active":"2024-04-14T08:33:01.220461+0000","last_peered":"2024-04-14T08:33:01.220461+0000","last_clean":"2024-04-14T08:33:01.220461+0000","last_became_active":"2024-04-14T08:32:54.194111+0000","last_became_peered":"2024-04-14T08:32:54.194111+0000","last_unstale":"2024-04-14T08:33:01.220461+0000","last_undegraded":"2024-04-14T08:33:01.220461+0000","last_fullsized":"2024-04-14T08:33:01.220461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:05:03.511225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115632+0000","last_change":"2024-04-14T08:32:53.194969+0000","last_active":"2024-04-14T08:33:01.115632+0000","last_peered":"2024-04-14T08:33:01.115632+0000","last_clean":"2024-04-14T08:33:01.115632+0000","last_became_active":"2024-04-14T08:32:53.194665+0000","last_became_peered":"2024-04-14T08:32:53.194665+0000","last_unstale":"2024-04-14T08:33:01.115632+0000","last_undegraded":"2024-04-14T08:33:01.115632+0000","last_fullsized":"2024-04-14T08:33:01.115632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224778+0000","last_change":"2024-04-14T08:32:58.237006+0000","last_active":"2024-04-14T08:32:59.224778+0000","last_peered":"2024-04-14T08:32:59.224778+0000","last_clean":"2024-04-14T08:32:59.224778+0000","last_became_active":"2024-04-14T08:32:58.236366+0000","last_became_peered":"2024-04-14T08:32:58.236366+0000","last_unstale":"2024-04-14T08:32:59.224778+0000","last_undegraded":"2024-04-14T08:32:59.224778+0000","last_fullsized":"2024-04-14T08:32:59.224778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:49.446712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225123+0000","last_change":"2024-04-14T08:32:54.205064+0000","last_active":"2024-04-14T08:32:59.225123+0000","last_peered":"2024-04-14T08:32:59.225123+0000","last_clean":"2024-04-14T08:32:59.225123+0000","last_became_active":"2024-04-14T08:32:54.203946+0000","last_became_peered":"2024-04-14T08:32:54.203946+0000","last_unstale":"2024-04-14T08:32:59.225123+0000","last_undegraded":"2024-04-14T08:32:59.225123+0000","last_fullsized":"2024-04-14T08:32:59.225123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:16.536477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610666+0000","last_change":"2024-04-14T08:32:53.198148+0000","last_active":"2024-04-14T08:33:00.610666+0000","last_peered":"2024-04-14T08:33:00.610666+0000","last_clean":"2024-04-14T08:33:00.610666+0000","last_became_active":"2024-04-14T08:32:53.197878+0000","last_became_peered":"2024-04-14T08:32:53.197878+0000","last_unstale":"2024-04-14T08:33:00.610666+0000","last_undegraded":"2024-04-14T08:33:00.610666+0000","last_fullsized":"2024-04-14T08:33:00.610666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224235+0000","last_change":"2024-04-14T08:32:58.246772+0000","last_active":"2024-04-14T08:32:59.224235+0000","last_peered":"2024-04-14T08:32:59.224235+0000","last_clean":"2024-04-14T08:32:59.224235+0000","last_became_active":"2024-04-14T08:32:58.246388+0000","last_became_peered":"2024-04-14T08:32:58.246388+0000","last_unstale":"2024-04-14T08:32:59.224235+0000","last_undegraded":"2024-04-14T08:32:59.224235+0000","last_fullsized":"2024-04-14T08:32:59.224235+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:54.763380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610409+0000","last_change":"2024-04-14T08:32:54.194202+0000","last_active":"2024-04-14T08:33:00.610409+0000","last_peered":"2024-04-14T08:33:00.610409+0000","last_clean":"2024-04-14T08:33:00.610409+0000","last_became_active":"2024-04-14T08:32:54.193938+0000","last_became_peered":"2024-04-14T08:32:54.193938+0000","last_unstale":"2024-04-14T08:33:00.610409+0000","last_undegraded":"2024-04-14T08:33:00.610409+0000","last_fullsized":"2024-04-14T08:33:00.610409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226263+0000","last_change":"2024-04-14T08:32:53.202270+0000","last_active":"2024-04-14T08:32:59.226263+0000","last_peered":"2024-04-14T08:32:59.226263+0000","last_clean":"2024-04-14T08:32:59.226263+0000","last_became_active":"2024-04-14T08:32:53.201945+0000","last_became_peered":"2024-04-14T08:32:53.201945+0000","last_unstale":"2024-04-14T08:32:59.226263+0000","last_undegraded":"2024-04-14T08:32:59.226263+0000","last_fullsized":"2024-04-14T08:32:59.226263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219955+0000","last_change":"2024-04-14T08:32:58.248277+0000","last_active":"2024-04-14T08:33:01.219955+0000","last_peered":"2024-04-14T08:33:01.219955+0000","last_clean":"2024-04-14T08:33:01.219955+0000","last_became_active":"2024-04-14T08:32:58.247535+0000","last_became_peered":"2024-04-14T08:32:58.247535+0000","last_unstale":"2024-04-14T08:33:01.219955+0000","last_undegraded":"2024-04-14T08:33:01.219955+0000","last_fullsized":"2024-04-14T08:33:01.219955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:05.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405198+0000","last_change":"2024-04-14T08:32:54.203448+0000","last_active":"2024-04-14T08:32:59.405198+0000","last_peered":"2024-04-14T08:32:59.405198+0000","last_clean":"2024-04-14T08:32:59.405198+0000","last_became_active":"2024-04-14T08:32:54.203170+0000","last_became_peered":"2024-04-14T08:32:54.203170+0000","last_unstale":"2024-04-14T08:32:59.405198+0000","last_undegraded":"2024-04-14T08:32:59.405198+0000","last_fullsized":"2024-04-14T08:32:59.405198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:01.152061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610433+0000","last_change":"2024-04-14T08:32:53.187816+0000","last_active":"2024-04-14T08:33:00.610433+0000","last_peered":"2024-04-14T08:33:00.610433+0000","last_clean":"2024-04-14T08:33:00.610433+0000","last_became_active":"2024-04-14T08:32:53.187519+0000","last_became_peered":"2024-04-14T08:32:53.187519+0000","last_unstale":"2024-04-14T08:33:00.610433+0000","last_undegraded":"2024-04-14T08:33:00.610433+0000","last_fullsized":"2024-04-14T08:33:00.610433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610230+0000","last_change":"2024-04-14T08:32:58.246145+0000","last_active":"2024-04-14T08:33:00.610230+0000","last_peered":"2024-04-14T08:33:00.610230+0000","last_clean":"2024-04-14T08:33:00.610230+0000","last_became_active":"2024-04-14T08:32:58.245792+0000","last_became_peered":"2024-04-14T08:32:58.245792+0000","last_unstale":"2024-04-14T08:33:00.610230+0000","last_undegraded":"2024-04-14T08:33:00.610230+0000","last_fullsized":"2024-04-14T08:33:00.610230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:05.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225311+0000","last_change":"2024-04-14T08:32:54.192476+0000","last_active":"2024-04-14T08:32:59.225311+0000","last_peered":"2024-04-14T08:32:59.225311+0000","last_clean":"2024-04-14T08:32:59.225311+0000","last_became_active":"2024-04-14T08:32:54.192182+0000","last_became_peered":"2024-04-14T08:32:54.192182+0000","last_unstale":"2024-04-14T08:32:59.225311+0000","last_undegraded":"2024-04-14T08:32:59.225311+0000","last_fullsized":"2024-04-14T08:32:59.225311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:05.453493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405000+0000","last_change":"2024-04-14T08:32:53.201969+0000","last_active":"2024-04-14T08:32:59.405000+0000","last_peered":"2024-04-14T08:32:59.405000+0000","last_clean":"2024-04-14T08:32:59.405000+0000","last_became_active":"2024-04-14T08:32:53.201611+0000","last_became_peered":"2024-04-14T08:32:53.201611+0000","last_unstale":"2024-04-14T08:32:59.405000+0000","last_undegraded":"2024-04-14T08:32:59.405000+0000","last_fullsized":"2024-04-14T08:32:59.405000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610320+0000","last_change":"2024-04-14T08:32:58.249518+0000","last_active":"2024-04-14T08:33:00.610320+0000","last_peered":"2024-04-14T08:33:00.610320+0000","last_clean":"2024-04-14T08:33:00.610320+0000","last_became_active":"2024-04-14T08:32:58.249079+0000","last_became_peered":"2024-04-14T08:32:58.249079+0000","last_unstale":"2024-04-14T08:33:00.610320+0000","last_undegraded":"2024-04-14T08:33:00.610320+0000","last_fullsized":"2024-04-14T08:33:00.610320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:16.724895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610081+0000","last_change":"2024-04-14T08:32:54.194141+0000","last_active":"2024-04-14T08:33:00.610081+0000","last_peered":"2024-04-14T08:33:00.610081+0000","last_clean":"2024-04-14T08:33:00.610081+0000","last_became_active":"2024-04-14T08:32:54.193884+0000","last_became_peered":"2024-04-14T08:32:54.193884+0000","last_unstale":"2024-04-14T08:33:00.610081+0000","last_undegraded":"2024-04-14T08:33:00.610081+0000","last_fullsized":"2024-04-14T08:33:00.610081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278559+0000","last_change":"2024-04-14T08:32:53.196690+0000","last_active":"2024-04-14T08:33:01.278559+0000","last_peered":"2024-04-14T08:33:01.278559+0000","last_clean":"2024-04-14T08:33:01.278559+0000","last_became_active":"2024-04-14T08:32:53.195977+0000","last_became_peered":"2024-04-14T08:32:53.195977+0000","last_unstale":"2024-04-14T08:33:01.278559+0000","last_undegraded":"2024-04-14T08:33:01.278559+0000","last_fullsized":"2024-04-14T08:33:01.278559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610859+0000","last_change":"2024-04-14T08:32:58.249862+0000","last_active":"2024-04-14T08:33:00.610859+0000","last_peered":"2024-04-14T08:33:00.610859+0000","last_clean":"2024-04-14T08:33:00.610859+0000","last_became_active":"2024-04-14T08:32:58.248928+0000","last_became_peered":"2024-04-14T08:32:58.248928+0000","last_unstale":"2024-04-14T08:33:00.610859+0000","last_undegraded":"2024-04-14T08:33:00.610859+0000","last_fullsized":"2024-04-14T08:33:00.610859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:53.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219816+0000","last_change":"2024-04-14T08:32:54.206032+0000","last_active":"2024-04-14T08:33:01.219816+0000","last_peered":"2024-04-14T08:33:01.219816+0000","last_clean":"2024-04-14T08:33:01.219816+0000","last_became_active":"2024-04-14T08:32:54.205781+0000","last_became_peered":"2024-04-14T08:32:54.205781+0000","last_unstale":"2024-04-14T08:33:01.219816+0000","last_undegraded":"2024-04-14T08:33:01.219816+0000","last_fullsized":"2024-04-14T08:33:01.219816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:25:28.934928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224825+0000","last_change":"2024-04-14T08:32:53.195805+0000","last_active":"2024-04-14T08:32:59.224825+0000","last_peered":"2024-04-14T08:32:59.224825+0000","last_clean":"2024-04-14T08:32:59.224825+0000","last_became_active":"2024-04-14T08:32:53.195442+0000","last_became_peered":"2024-04-14T08:32:53.195442+0000","last_unstale":"2024-04-14T08:32:59.224825+0000","last_undegraded":"2024-04-14T08:32:59.224825+0000","last_fullsized":"2024-04-14T08:32:59.224825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:48.807150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220403+0000","last_change":"2024-04-14T08:32:58.247104+0000","last_active":"2024-04-14T08:33:01.220403+0000","last_peered":"2024-04-14T08:33:01.220403+0000","last_clean":"2024-04-14T08:33:01.220403+0000","last_became_active":"2024-04-14T08:32:58.246758+0000","last_became_peered":"2024-04-14T08:32:58.246758+0000","last_unstale":"2024-04-14T08:33:01.220403+0000","last_undegraded":"2024-04-14T08:33:01.220403+0000","last_fullsized":"2024-04-14T08:33:01.220403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:53.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224440+0000","last_change":"2024-04-14T08:32:54.209924+0000","last_active":"2024-04-14T08:32:59.224440+0000","last_peered":"2024-04-14T08:32:59.224440+0000","last_clean":"2024-04-14T08:32:59.224440+0000","last_became_active":"2024-04-14T08:32:54.209703+0000","last_became_peered":"2024-04-14T08:32:54.209703+0000","last_unstale":"2024-04-14T08:32:59.224440+0000","last_undegraded":"2024-04-14T08:32:59.224440+0000","last_fullsized":"2024-04-14T08:32:59.224440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404884+0000","last_change":"2024-04-14T08:32:53.197701+0000","last_active":"2024-04-14T08:32:59.404884+0000","last_peered":"2024-04-14T08:32:59.404884+0000","last_clean":"2024-04-14T08:32:59.404884+0000","last_became_active":"2024-04-14T08:32:53.197291+0000","last_became_peered":"2024-04-14T08:32:53.197291+0000","last_unstale":"2024-04-14T08:32:59.404884+0000","last_undegraded":"2024-04-14T08:32:59.404884+0000","last_fullsized":"2024-04-14T08:32:59.404884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:54:34.240052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224914+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:32:59.224914+0000","last_peered":"2024-04-14T08:32:59.224914+0000","last_clean":"2024-04-14T08:32:59.224914+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:32:59.224914+0000","last_undegraded":"2024-04-14T08:32:59.224914+0000","last_fullsized":"2024-04-14T08:32:59.224914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115413+0000","last_change":"2024-04-14T08:32:58.237249+0000","last_active":"2024-04-14T08:33:01.115413+0000","last_peered":"2024-04-14T08:33:01.115413+0000","last_clean":"2024-04-14T08:33:01.115413+0000","last_became_active":"2024-04-14T08:32:58.236861+0000","last_became_peered":"2024-04-14T08:32:58.236861+0000","last_unstale":"2024-04-14T08:33:01.115413+0000","last_undegraded":"2024-04-14T08:33:01.115413+0000","last_fullsized":"2024-04-14T08:33:01.115413+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:05.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226018+0000","last_change":"2024-04-14T08:32:54.211527+0000","last_active":"2024-04-14T08:32:59.226018+0000","last_peered":"2024-04-14T08:32:59.226018+0000","last_clean":"2024-04-14T08:32:59.226018+0000","last_became_active":"2024-04-14T08:32:54.211268+0000","last_became_peered":"2024-04-14T08:32:54.211268+0000","last_unstale":"2024-04-14T08:32:59.226018+0000","last_undegraded":"2024-04-14T08:32:59.226018+0000","last_fullsized":"2024-04-14T08:32:59.226018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:41:14.768103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278564+0000","last_change":"2024-04-14T08:32:53.198176+0000","last_active":"2024-04-14T08:33:01.278564+0000","last_peered":"2024-04-14T08:33:01.278564+0000","last_clean":"2024-04-14T08:33:01.278564+0000","last_became_active":"2024-04-14T08:32:53.197626+0000","last_became_peered":"2024-04-14T08:32:53.197626+0000","last_unstale":"2024-04-14T08:33:01.278564+0000","last_undegraded":"2024-04-14T08:33:01.278564+0000","last_fullsized":"2024-04-14T08:33:01.278564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:10:49.016747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115290+0000","last_change":"2024-04-14T08:32:58.240943+0000","last_active":"2024-04-14T08:33:01.115290+0000","last_peered":"2024-04-14T08:33:01.115290+0000","last_clean":"2024-04-14T08:33:01.115290+0000","last_became_active":"2024-04-14T08:32:58.240555+0000","last_became_peered":"2024-04-14T08:32:58.240555+0000","last_unstale":"2024-04-14T08:33:01.115290+0000","last_undegraded":"2024-04-14T08:33:01.115290+0000","last_fullsized":"2024-04-14T08:33:01.115290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:53.001768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405061+0000","last_change":"2024-04-14T08:32:54.205351+0000","last_active":"2024-04-14T08:32:59.405061+0000","last_peered":"2024-04-14T08:32:59.405061+0000","last_clean":"2024-04-14T08:32:59.405061+0000","last_became_active":"2024-04-14T08:32:54.205086+0000","last_became_peered":"2024-04-14T08:32:54.205086+0000","last_unstale":"2024-04-14T08:32:59.405061+0000","last_undegraded":"2024-04-14T08:32:59.405061+0000","last_fullsized":"2024-04-14T08:32:59.405061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:43.600124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610474+0000","last_change":"2024-04-14T08:32:53.194785+0000","last_active":"2024-04-14T08:33:00.610474+0000","last_peered":"2024-04-14T08:33:00.610474+0000","last_clean":"2024-04-14T08:33:00.610474+0000","last_became_active":"2024-04-14T08:32:53.194535+0000","last_became_peered":"2024-04-14T08:32:53.194535+0000","last_unstale":"2024-04-14T08:33:00.610474+0000","last_undegraded":"2024-04-14T08:33:00.610474+0000","last_fullsized":"2024-04-14T08:33:00.610474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:48.807150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220190+0000","last_change":"2024-04-14T08:32:58.246826+0000","last_active":"2024-04-14T08:33:01.220190+0000","last_peered":"2024-04-14T08:33:01.220190+0000","last_clean":"2024-04-14T08:33:01.220190+0000","last_became_active":"2024-04-14T08:32:58.246174+0000","last_became_peered":"2024-04-14T08:32:58.246174+0000","last_unstale":"2024-04-14T08:33:01.220190+0000","last_undegraded":"2024-04-14T08:33:01.220190+0000","last_fullsized":"2024-04-14T08:33:01.220190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:57.127403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226211+0000","last_change":"2024-04-14T08:32:54.200509+0000","last_active":"2024-04-14T08:32:59.226211+0000","last_peered":"2024-04-14T08:32:59.226211+0000","last_clean":"2024-04-14T08:32:59.226211+0000","last_became_active":"2024-04-14T08:32:54.199748+0000","last_became_peered":"2024-04-14T08:32:54.199748+0000","last_unstale":"2024-04-14T08:32:59.226211+0000","last_undegraded":"2024-04-14T08:32:59.226211+0000","last_fullsized":"2024-04-14T08:32:59.226211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:49.939225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278584+0000","last_change":"2024-04-14T08:32:53.196334+0000","last_active":"2024-04-14T08:33:01.278584+0000","last_peered":"2024-04-14T08:33:01.278584+0000","last_clean":"2024-04-14T08:33:01.278584+0000","last_became_active":"2024-04-14T08:32:53.196036+0000","last_became_peered":"2024-04-14T08:32:53.196036+0000","last_unstale":"2024-04-14T08:33:01.278584+0000","last_undegraded":"2024-04-14T08:33:01.278584+0000","last_fullsized":"2024-04-14T08:33:01.278584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404693+0000","last_change":"2024-04-14T08:32:58.239865+0000","last_active":"2024-04-14T08:32:59.404693+0000","last_peered":"2024-04-14T08:32:59.404693+0000","last_clean":"2024-04-14T08:32:59.404693+0000","last_became_active":"2024-04-14T08:32:58.239510+0000","last_became_peered":"2024-04-14T08:32:58.239510+0000","last_unstale":"2024-04-14T08:32:59.404693+0000","last_undegraded":"2024-04-14T08:32:59.404693+0000","last_fullsized":"2024-04-14T08:32:59.404693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:37:10.427564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226121+0000","last_change":"2024-04-14T08:32:54.200777+0000","last_active":"2024-04-14T08:32:59.226121+0000","last_peered":"2024-04-14T08:32:59.226121+0000","last_clean":"2024-04-14T08:32:59.226121+0000","last_became_active":"2024-04-14T08:32:54.200284+0000","last_became_peered":"2024-04-14T08:32:54.200284+0000","last_unstale":"2024-04-14T08:32:59.226121+0000","last_undegraded":"2024-04-14T08:32:59.226121+0000","last_fullsized":"2024-04-14T08:32:59.226121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:53.086323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225655+0000","last_change":"2024-04-14T08:32:53.184799+0000","last_active":"2024-04-14T08:32:59.225655+0000","last_peered":"2024-04-14T08:32:59.225655+0000","last_clean":"2024-04-14T08:32:59.225655+0000","last_became_active":"2024-04-14T08:32:53.184483+0000","last_became_peered":"2024-04-14T08:32:53.184483+0000","last_unstale":"2024-04-14T08:32:59.225655+0000","last_undegraded":"2024-04-14T08:32:59.225655+0000","last_fullsized":"2024-04-14T08:32:59.225655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:03:30.482046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224689+0000","last_change":"2024-04-14T08:32:58.243067+0000","last_active":"2024-04-14T08:32:59.224689+0000","last_peered":"2024-04-14T08:32:59.224689+0000","last_clean":"2024-04-14T08:32:59.224689+0000","last_became_active":"2024-04-14T08:32:58.242726+0000","last_became_peered":"2024-04-14T08:32:58.242726+0000","last_unstale":"2024-04-14T08:32:59.224689+0000","last_undegraded":"2024-04-14T08:32:59.224689+0000","last_fullsized":"2024-04-14T08:32:59.224689+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:09.583268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225539+0000","last_change":"2024-04-14T08:32:54.204697+0000","last_active":"2024-04-14T08:32:59.225539+0000","last_peered":"2024-04-14T08:32:59.225539+0000","last_clean":"2024-04-14T08:32:59.225539+0000","last_became_active":"2024-04-14T08:32:54.204357+0000","last_became_peered":"2024-04-14T08:32:54.204357+0000","last_unstale":"2024-04-14T08:32:59.225539+0000","last_undegraded":"2024-04-14T08:32:59.225539+0000","last_fullsized":"2024-04-14T08:32:59.225539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:01.152061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278559+0000","last_change":"2024-04-14T08:32:53.187972+0000","last_active":"2024-04-14T08:33:01.278559+0000","last_peered":"2024-04-14T08:33:01.278559+0000","last_clean":"2024-04-14T08:33:01.278559+0000","last_became_active":"2024-04-14T08:32:53.187667+0000","last_became_peered":"2024-04-14T08:32:53.187667+0000","last_unstale":"2024-04-14T08:33:01.278559+0000","last_undegraded":"2024-04-14T08:33:01.278559+0000","last_fullsized":"2024-04-14T08:33:01.278559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:54:34.240052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220316+0000","last_change":"2024-04-14T08:32:58.246500+0000","last_active":"2024-04-14T08:33:01.220316+0000","last_peered":"2024-04-14T08:33:01.220316+0000","last_clean":"2024-04-14T08:33:01.220316+0000","last_became_active":"2024-04-14T08:32:58.245970+0000","last_became_peered":"2024-04-14T08:32:58.245970+0000","last_unstale":"2024-04-14T08:33:01.220316+0000","last_undegraded":"2024-04-14T08:33:01.220316+0000","last_fullsized":"2024-04-14T08:33:01.220316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:54.763380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115215+0000","last_change":"2024-04-14T08:32:54.201957+0000","last_active":"2024-04-14T08:33:01.115215+0000","last_peered":"2024-04-14T08:33:01.115215+0000","last_clean":"2024-04-14T08:33:01.115215+0000","last_became_active":"2024-04-14T08:32:54.201618+0000","last_became_peered":"2024-04-14T08:32:54.201618+0000","last_unstale":"2024-04-14T08:33:01.115215+0000","last_undegraded":"2024-04-14T08:33:01.115215+0000","last_fullsized":"2024-04-14T08:33:01.115215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115228+0000","last_change":"2024-04-14T08:32:53.195817+0000","last_active":"2024-04-14T08:33:01.115228+0000","last_peered":"2024-04-14T08:33:01.115228+0000","last_clean":"2024-04-14T08:33:01.115228+0000","last_became_active":"2024-04-14T08:32:53.195547+0000","last_became_peered":"2024-04-14T08:32:53.195547+0000","last_unstale":"2024-04-14T08:33:01.115228+0000","last_undegraded":"2024-04-14T08:33:01.115228+0000","last_fullsized":"2024-04-14T08:33:01.115228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225091+0000","last_change":"2024-04-14T08:32:58.240188+0000","last_active":"2024-04-14T08:32:59.225091+0000","last_peered":"2024-04-14T08:32:59.225091+0000","last_clean":"2024-04-14T08:32:59.225091+0000","last_became_active":"2024-04-14T08:32:58.239811+0000","last_became_peered":"2024-04-14T08:32:58.239811+0000","last_unstale":"2024-04-14T08:32:59.225091+0000","last_undegraded":"2024-04-14T08:32:59.225091+0000","last_fullsized":"2024-04-14T08:32:59.225091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:21.164569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225955+0000","last_change":"2024-04-14T08:32:54.200441+0000","last_active":"2024-04-14T08:32:59.225955+0000","last_peered":"2024-04-14T08:32:59.225955+0000","last_clean":"2024-04-14T08:32:59.225955+0000","last_became_active":"2024-04-14T08:32:54.199824+0000","last_became_peered":"2024-04-14T08:32:54.199824+0000","last_unstale":"2024-04-14T08:32:59.225955+0000","last_undegraded":"2024-04-14T08:32:59.225955+0000","last_fullsized":"2024-04-14T08:32:59.225955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404960+0000","last_change":"2024-04-14T08:32:53.202370+0000","last_active":"2024-04-14T08:32:59.404960+0000","last_peered":"2024-04-14T08:32:59.404960+0000","last_clean":"2024-04-14T08:32:59.404960+0000","last_became_active":"2024-04-14T08:32:53.202083+0000","last_became_peered":"2024-04-14T08:32:53.202083+0000","last_unstale":"2024-04-14T08:32:59.404960+0000","last_undegraded":"2024-04-14T08:32:59.404960+0000","last_fullsized":"2024-04-14T08:32:59.404960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224830+0000","last_change":"2024-04-14T08:32:58.247699+0000","last_active":"2024-04-14T08:32:59.224830+0000","last_peered":"2024-04-14T08:32:59.224830+0000","last_clean":"2024-04-14T08:32:59.224830+0000","last_became_active":"2024-04-14T08:32:58.247336+0000","last_became_peered":"2024-04-14T08:32:58.247336+0000","last_unstale":"2024-04-14T08:32:59.224830+0000","last_undegraded":"2024-04-14T08:32:59.224830+0000","last_fullsized":"2024-04-14T08:32:59.224830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:07:01.006337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220030+0000","last_change":"2024-04-14T08:32:54.198561+0000","last_active":"2024-04-14T08:33:01.220030+0000","last_peered":"2024-04-14T08:33:01.220030+0000","last_clean":"2024-04-14T08:33:01.220030+0000","last_became_active":"2024-04-14T08:32:54.198319+0000","last_became_peered":"2024-04-14T08:32:54.198319+0000","last_unstale":"2024-04-14T08:33:01.220030+0000","last_undegraded":"2024-04-14T08:33:01.220030+0000","last_fullsized":"2024-04-14T08:33:01.220030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219614+0000","last_change":"2024-04-14T08:32:53.188093+0000","last_active":"2024-04-14T08:33:01.219614+0000","last_peered":"2024-04-14T08:33:01.219614+0000","last_clean":"2024-04-14T08:33:01.219614+0000","last_became_active":"2024-04-14T08:32:53.187807+0000","last_became_peered":"2024-04-14T08:32:53.187807+0000","last_unstale":"2024-04-14T08:33:01.219614+0000","last_undegraded":"2024-04-14T08:33:01.219614+0000","last_fullsized":"2024-04-14T08:33:01.219614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:22:15.886932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405178+0000","last_change":"2024-04-14T08:32:58.234962+0000","last_active":"2024-04-14T08:32:59.405178+0000","last_peered":"2024-04-14T08:32:59.405178+0000","last_clean":"2024-04-14T08:32:59.405178+0000","last_became_active":"2024-04-14T08:32:58.234475+0000","last_became_peered":"2024-04-14T08:32:58.234475+0000","last_unstale":"2024-04-14T08:32:59.405178+0000","last_undegraded":"2024-04-14T08:32:59.405178+0000","last_fullsized":"2024-04-14T08:32:59.405178+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:07:01.006337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225877+0000","last_change":"2024-04-14T08:32:54.199377+0000","last_active":"2024-04-14T08:32:59.225877+0000","last_peered":"2024-04-14T08:32:59.225877+0000","last_clean":"2024-04-14T08:32:59.225877+0000","last_became_active":"2024-04-14T08:32:54.199105+0000","last_became_peered":"2024-04-14T08:32:54.199105+0000","last_unstale":"2024-04-14T08:32:59.225877+0000","last_undegraded":"2024-04-14T08:32:59.225877+0000","last_fullsized":"2024-04-14T08:32:59.225877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:43.600124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:03.602017+0000","last_change":"2024-04-14T08:32:53.199739+0000","last_active":"2024-04-14T08:33:03.602017+0000","last_peered":"2024-04-14T08:33:03.602017+0000","last_clean":"2024-04-14T08:33:03.602017+0000","last_became_active":"2024-04-14T08:32:53.199421+0000","last_became_peered":"2024-04-14T08:32:53.199421+0000","last_unstale":"2024-04-14T08:33:03.602017+0000","last_undegraded":"2024-04-14T08:33:03.602017+0000","last_fullsized":"2024-04-14T08:33:03.602017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610241+0000","last_change":"2024-04-14T08:32:58.243680+0000","last_active":"2024-04-14T08:33:00.610241+0000","last_peered":"2024-04-14T08:33:00.610241+0000","last_clean":"2024-04-14T08:33:00.610241+0000","last_became_active":"2024-04-14T08:32:58.243248+0000","last_became_peered":"2024-04-14T08:32:58.243248+0000","last_unstale":"2024-04-14T08:33:00.610241+0000","last_undegraded":"2024-04-14T08:33:00.610241+0000","last_fullsized":"2024-04-14T08:33:00.610241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:57.127403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225954+0000","last_change":"2024-04-14T08:32:54.199362+0000","last_active":"2024-04-14T08:32:59.225954+0000","last_peered":"2024-04-14T08:32:59.225954+0000","last_clean":"2024-04-14T08:32:59.225954+0000","last_became_active":"2024-04-14T08:32:54.199039+0000","last_became_peered":"2024-04-14T08:32:54.199039+0000","last_unstale":"2024-04-14T08:32:59.225954+0000","last_undegraded":"2024-04-14T08:32:59.225954+0000","last_fullsized":"2024-04-14T08:32:59.225954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:33.086425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610374+0000","last_change":"2024-04-14T08:32:53.198302+0000","last_active":"2024-04-14T08:33:00.610374+0000","last_peered":"2024-04-14T08:33:00.610374+0000","last_clean":"2024-04-14T08:33:00.610374+0000","last_became_active":"2024-04-14T08:32:53.198042+0000","last_became_peered":"2024-04-14T08:32:53.198042+0000","last_unstale":"2024-04-14T08:33:00.610374+0000","last_undegraded":"2024-04-14T08:33:00.610374+0000","last_fullsized":"2024-04-14T08:33:00.610374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:32.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225101+0000","last_change":"2024-04-14T08:32:58.249159+0000","last_active":"2024-04-14T08:32:59.225101+0000","last_peered":"2024-04-14T08:32:59.225101+0000","last_clean":"2024-04-14T08:32:59.225101+0000","last_became_active":"2024-04-14T08:32:58.248502+0000","last_became_peered":"2024-04-14T08:32:58.248502+0000","last_unstale":"2024-04-14T08:32:59.225101+0000","last_undegraded":"2024-04-14T08:32:59.225101+0000","last_fullsized":"2024-04-14T08:32:59.225101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:57.702107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220422+0000","last_change":"2024-04-14T08:32:54.211683+0000","last_active":"2024-04-14T08:33:01.220422+0000","last_peered":"2024-04-14T08:33:01.220422+0000","last_clean":"2024-04-14T08:33:01.220422+0000","last_became_active":"2024-04-14T08:32:54.211031+0000","last_became_peered":"2024-04-14T08:32:54.211031+0000","last_unstale":"2024-04-14T08:33:01.220422+0000","last_undegraded":"2024-04-14T08:33:01.220422+0000","last_fullsized":"2024-04-14T08:33:01.220422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405170+0000","last_change":"2024-04-14T08:32:53.198725+0000","last_active":"2024-04-14T08:32:59.405170+0000","last_peered":"2024-04-14T08:32:59.405170+0000","last_clean":"2024-04-14T08:32:59.405170+0000","last_became_active":"2024-04-14T08:32:53.198331+0000","last_became_peered":"2024-04-14T08:32:53.198331+0000","last_unstale":"2024-04-14T08:32:59.405170+0000","last_undegraded":"2024-04-14T08:32:59.405170+0000","last_fullsized":"2024-04-14T08:32:59.405170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224411+0000","last_change":"2024-04-14T08:32:58.241328+0000","last_active":"2024-04-14T08:32:59.224411+0000","last_peered":"2024-04-14T08:32:59.224411+0000","last_clean":"2024-04-14T08:32:59.224411+0000","last_became_active":"2024-04-14T08:32:58.240742+0000","last_became_peered":"2024-04-14T08:32:58.240742+0000","last_unstale":"2024-04-14T08:32:59.224411+0000","last_undegraded":"2024-04-14T08:32:59.224411+0000","last_fullsized":"2024-04-14T08:32:59.224411+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:10:58.801121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405105+0000","last_change":"2024-04-14T08:32:54.211134+0000","last_active":"2024-04-14T08:32:59.405105+0000","last_peered":"2024-04-14T08:32:59.405105+0000","last_clean":"2024-04-14T08:32:59.405105+0000","last_became_active":"2024-04-14T08:32:54.210882+0000","last_became_peered":"2024-04-14T08:32:54.210882+0000","last_unstale":"2024-04-14T08:32:59.405105+0000","last_undegraded":"2024-04-14T08:32:59.405105+0000","last_fullsized":"2024-04-14T08:32:59.405105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115518+0000","last_change":"2024-04-14T08:32:53.198595+0000","last_active":"2024-04-14T08:33:01.115518+0000","last_peered":"2024-04-14T08:33:01.115518+0000","last_clean":"2024-04-14T08:33:01.115518+0000","last_became_active":"2024-04-14T08:32:53.198284+0000","last_became_peered":"2024-04-14T08:32:53.198284+0000","last_unstale":"2024-04-14T08:33:01.115518+0000","last_undegraded":"2024-04-14T08:33:01.115518+0000","last_fullsized":"2024-04-14T08:33:01.115518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:48.807150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224443+0000","last_change":"2024-04-14T08:32:58.240139+0000","last_active":"2024-04-14T08:32:59.224443+0000","last_peered":"2024-04-14T08:32:59.224443+0000","last_clean":"2024-04-14T08:32:59.224443+0000","last_became_active":"2024-04-14T08:32:58.239751+0000","last_became_peered":"2024-04-14T08:32:58.239751+0000","last_unstale":"2024-04-14T08:32:59.224443+0000","last_undegraded":"2024-04-14T08:32:59.224443+0000","last_fullsized":"2024-04-14T08:32:59.224443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:37:10.427564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220280+0000","last_change":"2024-04-14T08:32:58.252205+0000","last_active":"2024-04-14T08:33:01.220280+0000","last_peered":"2024-04-14T08:33:01.220280+0000","last_clean":"2024-04-14T08:33:01.220280+0000","last_became_active":"2024-04-14T08:32:58.251856+0000","last_became_peered":"2024-04-14T08:32:58.251856+0000","last_unstale":"2024-04-14T08:33:01.220280+0000","last_undegraded":"2024-04-14T08:33:01.220280+0000","last_fullsized":"2024-04-14T08:33:01.220280+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:16.724895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.609964+0000","last_change":"2024-04-14T08:32:54.200507+0000","last_active":"2024-04-14T08:33:00.609964+0000","last_peered":"2024-04-14T08:33:00.609964+0000","last_clean":"2024-04-14T08:33:00.609964+0000","last_became_active":"2024-04-14T08:32:54.200256+0000","last_became_peered":"2024-04-14T08:32:54.200256+0000","last_unstale":"2024-04-14T08:33:00.609964+0000","last_undegraded":"2024-04-14T08:33:00.609964+0000","last_fullsized":"2024-04-14T08:33:00.609964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404763+0000","last_change":"2024-04-14T08:32:53.197120+0000","last_active":"2024-04-14T08:32:59.404763+0000","last_peered":"2024-04-14T08:32:59.404763+0000","last_clean":"2024-04-14T08:32:59.404763+0000","last_became_active":"2024-04-14T08:32:53.196835+0000","last_became_peered":"2024-04-14T08:32:53.196835+0000","last_unstale":"2024-04-14T08:32:59.404763+0000","last_undegraded":"2024-04-14T08:32:59.404763+0000","last_fullsized":"2024-04-14T08:32:59.404763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115520+0000","last_change":"2024-04-14T08:32:58.240951+0000","last_active":"2024-04-14T08:33:01.115520+0000","last_peered":"2024-04-14T08:33:01.115520+0000","last_clean":"2024-04-14T08:33:01.115520+0000","last_became_active":"2024-04-14T08:32:58.240542+0000","last_became_peered":"2024-04-14T08:32:58.240542+0000","last_unstale":"2024-04-14T08:33:01.115520+0000","last_undegraded":"2024-04-14T08:33:01.115520+0000","last_fullsized":"2024-04-14T08:33:01.115520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:57.127403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224403+0000","last_change":"2024-04-14T08:32:54.199647+0000","last_active":"2024-04-14T08:32:59.224403+0000","last_peered":"2024-04-14T08:32:59.224403+0000","last_clean":"2024-04-14T08:32:59.224403+0000","last_became_active":"2024-04-14T08:32:54.199365+0000","last_became_peered":"2024-04-14T08:32:54.199365+0000","last_unstale":"2024-04-14T08:32:59.224403+0000","last_undegraded":"2024-04-14T08:32:59.224403+0000","last_fullsized":"2024-04-14T08:32:59.224403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:17:06.203553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224536+0000","last_change":"2024-04-14T08:32:53.199750+0000","last_active":"2024-04-14T08:32:59.224536+0000","last_peered":"2024-04-14T08:32:59.224536+0000","last_clean":"2024-04-14T08:32:59.224536+0000","last_became_active":"2024-04-14T08:32:53.199477+0000","last_became_peered":"2024-04-14T08:32:53.199477+0000","last_unstale":"2024-04-14T08:32:59.224536+0000","last_undegraded":"2024-04-14T08:32:59.224536+0000","last_fullsized":"2024-04-14T08:32:59.224536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:03:30.482046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115345+0000","last_change":"2024-04-14T08:32:53.195193+0000","last_active":"2024-04-14T08:33:01.115345+0000","last_peered":"2024-04-14T08:33:01.115345+0000","last_clean":"2024-04-14T08:33:01.115345+0000","last_became_active":"2024-04-14T08:32:53.194887+0000","last_became_peered":"2024-04-14T08:32:53.194887+0000","last_unstale":"2024-04-14T08:33:01.115345+0000","last_undegraded":"2024-04-14T08:33:01.115345+0000","last_fullsized":"2024-04-14T08:33:01.115345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220235+0000","last_change":"2024-04-14T08:32:54.203263+0000","last_active":"2024-04-14T08:33:01.220235+0000","last_peered":"2024-04-14T08:33:01.220235+0000","last_clean":"2024-04-14T08:33:01.220235+0000","last_became_active":"2024-04-14T08:32:54.202196+0000","last_became_peered":"2024-04-14T08:32:54.202196+0000","last_unstale":"2024-04-14T08:33:01.220235+0000","last_undegraded":"2024-04-14T08:33:01.220235+0000","last_fullsized":"2024-04-14T08:33:01.220235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278810+0000","last_change":"2024-04-14T08:32:58.239935+0000","last_active":"2024-04-14T08:33:01.278810+0000","last_peered":"2024-04-14T08:33:01.278810+0000","last_clean":"2024-04-14T08:33:01.278810+0000","last_became_active":"2024-04-14T08:32:58.239542+0000","last_became_peered":"2024-04-14T08:32:58.239542+0000","last_unstale":"2024-04-14T08:33:01.278810+0000","last_undegraded":"2024-04-14T08:33:01.278810+0000","last_fullsized":"2024-04-14T08:33:01.278810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:04.288282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224729+0000","last_change":"2024-04-14T08:32:53.201101+0000","last_active":"2024-04-14T08:32:59.224729+0000","last_peered":"2024-04-14T08:32:59.224729+0000","last_clean":"2024-04-14T08:32:59.224729+0000","last_became_active":"2024-04-14T08:32:53.200781+0000","last_became_peered":"2024-04-14T08:32:53.200781+0000","last_unstale":"2024-04-14T08:32:59.224729+0000","last_undegraded":"2024-04-14T08:32:59.224729+0000","last_fullsized":"2024-04-14T08:32:59.224729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:08:09.715762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224989+0000","last_change":"2024-04-14T08:32:54.194042+0000","last_active":"2024-04-14T08:32:59.224989+0000","last_peered":"2024-04-14T08:32:59.224989+0000","last_clean":"2024-04-14T08:32:59.224989+0000","last_became_active":"2024-04-14T08:32:54.193764+0000","last_became_peered":"2024-04-14T08:32:54.193764+0000","last_unstale":"2024-04-14T08:32:59.224989+0000","last_undegraded":"2024-04-14T08:32:59.224989+0000","last_fullsized":"2024-04-14T08:32:59.224989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:50:00.196929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225098+0000","last_change":"2024-04-14T08:32:58.241996+0000","last_active":"2024-04-14T08:32:59.225098+0000","last_peered":"2024-04-14T08:32:59.225098+0000","last_clean":"2024-04-14T08:32:59.225098+0000","last_became_active":"2024-04-14T08:32:58.241641+0000","last_became_peered":"2024-04-14T08:32:58.241641+0000","last_unstale":"2024-04-14T08:32:59.225098+0000","last_undegraded":"2024-04-14T08:32:59.225098+0000","last_fullsized":"2024-04-14T08:32:59.225098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:58:22.710120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115102+0000","last_change":"2024-04-14T08:32:54.210270+0000","last_active":"2024-04-14T08:33:01.115102+0000","last_peered":"2024-04-14T08:33:01.115102+0000","last_clean":"2024-04-14T08:33:01.115102+0000","last_became_active":"2024-04-14T08:32:54.209717+0000","last_became_peered":"2024-04-14T08:32:54.209717+0000","last_unstale":"2024-04-14T08:33:01.115102+0000","last_undegraded":"2024-04-14T08:33:01.115102+0000","last_fullsized":"2024-04-14T08:33:01.115102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278775+0000","last_change":"2024-04-14T08:32:54.208720+0000","last_active":"2024-04-14T08:33:01.278775+0000","last_peered":"2024-04-14T08:33:01.278775+0000","last_clean":"2024-04-14T08:33:01.278775+0000","last_became_active":"2024-04-14T08:32:54.208426+0000","last_became_peered":"2024-04-14T08:32:54.208426+0000","last_unstale":"2024-04-14T08:33:01.278775+0000","last_undegraded":"2024-04-14T08:33:01.278775+0000","last_fullsized":"2024-04-14T08:33:01.278775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:49.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404885+0000","last_change":"2024-04-14T08:32:54.209426+0000","last_active":"2024-04-14T08:32:59.404885+0000","last_peered":"2024-04-14T08:32:59.404885+0000","last_clean":"2024-04-14T08:32:59.404885+0000","last_became_active":"2024-04-14T08:32:54.209158+0000","last_became_peered":"2024-04-14T08:32:54.209158+0000","last_unstale":"2024-04-14T08:32:59.404885+0000","last_undegraded":"2024-04-14T08:32:59.404885+0000","last_fullsized":"2024-04-14T08:32:59.404885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:12.683816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220151+0000","last_change":"2024-04-14T08:32:54.206827+0000","last_active":"2024-04-14T08:33:01.220151+0000","last_peered":"2024-04-14T08:33:01.220151+0000","last_clean":"2024-04-14T08:33:01.220151+0000","last_became_active":"2024-04-14T08:32:54.206593+0000","last_became_peered":"2024-04-14T08:32:54.206593+0000","last_unstale":"2024-04-14T08:33:01.220151+0000","last_undegraded":"2024-04-14T08:33:01.220151+0000","last_fullsized":"2024-04-14T08:33:01.220151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224854+0000","last_change":"2024-04-14T08:32:54.200872+0000","last_active":"2024-04-14T08:32:59.224854+0000","last_peered":"2024-04-14T08:32:59.224854+0000","last_clean":"2024-04-14T08:32:59.224854+0000","last_became_active":"2024-04-14T08:32:54.200616+0000","last_became_peered":"2024-04-14T08:32:54.200616+0000","last_unstale":"2024-04-14T08:32:59.224854+0000","last_undegraded":"2024-04-14T08:32:59.224854+0000","last_fullsized":"2024-04-14T08:32:59.224854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610219+0000","last_change":"2024-04-14T08:32:54.199679+0000","last_active":"2024-04-14T08:33:00.610219+0000","last_peered":"2024-04-14T08:33:00.610219+0000","last_clean":"2024-04-14T08:33:00.610219+0000","last_became_active":"2024-04-14T08:32:54.199433+0000","last_became_peered":"2024-04-14T08:32:54.199433+0000","last_unstale":"2024-04-14T08:33:00.610219+0000","last_undegraded":"2024-04-14T08:33:00.610219+0000","last_fullsized":"2024-04-14T08:33:00.610219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115645+0000","last_change":"2024-04-14T08:32:54.207939+0000","last_active":"2024-04-14T08:33:01.115645+0000","last_peered":"2024-04-14T08:33:01.115645+0000","last_clean":"2024-04-14T08:33:01.115645+0000","last_became_active":"2024-04-14T08:32:54.207460+0000","last_became_peered":"2024-04-14T08:32:54.207460+0000","last_unstale":"2024-04-14T08:33:01.115645+0000","last_undegraded":"2024-04-14T08:33:01.115645+0000","last_fullsized":"2024-04-14T08:33:01.115645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225033+0000","last_change":"2024-04-14T08:32:54.202461+0000","last_active":"2024-04-14T08:32:59.225033+0000","last_peered":"2024-04-14T08:32:59.225033+0000","last_clean":"2024-04-14T08:32:59.225033+0000","last_became_active":"2024-04-14T08:32:54.202156+0000","last_became_peered":"2024-04-14T08:32:54.202156+0000","last_unstale":"2024-04-14T08:32:59.225033+0000","last_undegraded":"2024-04-14T08:32:59.225033+0000","last_fullsized":"2024-04-14T08:32:59.225033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404730+0000","last_change":"2024-04-14T08:32:54.204603+0000","last_active":"2024-04-14T08:32:59.404730+0000","last_peered":"2024-04-14T08:32:59.404730+0000","last_clean":"2024-04-14T08:32:59.404730+0000","last_became_active":"2024-04-14T08:32:54.204323+0000","last_became_peered":"2024-04-14T08:32:54.204323+0000","last_unstale":"2024-04-14T08:32:59.404730+0000","last_undegraded":"2024-04-14T08:32:59.404730+0000","last_fullsized":"2024-04-14T08:32:59.404730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224464+0000","last_change":"2024-04-14T08:32:54.205983+0000","last_active":"2024-04-14T08:32:59.224464+0000","last_peered":"2024-04-14T08:32:59.224464+0000","last_clean":"2024-04-14T08:32:59.224464+0000","last_became_active":"2024-04-14T08:32:54.205707+0000","last_became_peered":"2024-04-14T08:32:54.205707+0000","last_unstale":"2024-04-14T08:32:59.224464+0000","last_undegraded":"2024-04-14T08:32:59.224464+0000","last_fullsized":"2024-04-14T08:32:59.224464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:05:03.511225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404857+0000","last_change":"2024-04-14T08:32:54.211979+0000","last_active":"2024-04-14T08:32:59.404857+0000","last_peered":"2024-04-14T08:32:59.404857+0000","last_clean":"2024-04-14T08:32:59.404857+0000","last_became_active":"2024-04-14T08:32:54.211481+0000","last_became_peered":"2024-04-14T08:32:54.211481+0000","last_unstale":"2024-04-14T08:32:59.404857+0000","last_undegraded":"2024-04-14T08:32:59.404857+0000","last_fullsized":"2024-04-14T08:32:59.404857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:53.086323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220046+0000","last_change":"2024-04-14T08:32:54.204727+0000","last_active":"2024-04-14T08:33:01.220046+0000","last_peered":"2024-04-14T08:33:01.220046+0000","last_clean":"2024-04-14T08:33:01.220046+0000","last_became_active":"2024-04-14T08:32:54.204472+0000","last_became_peered":"2024-04-14T08:32:54.204472+0000","last_unstale":"2024-04-14T08:33:01.220046+0000","last_undegraded":"2024-04-14T08:33:01.220046+0000","last_fullsized":"2024-04-14T08:33:01.220046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115389+0000","last_change":"2024-04-14T08:32:54.211345+0000","last_active":"2024-04-14T08:33:01.115389+0000","last_peered":"2024-04-14T08:33:01.115389+0000","last_clean":"2024-04-14T08:33:01.115389+0000","last_became_active":"2024-04-14T08:32:54.211084+0000","last_became_peered":"2024-04-14T08:32:54.211084+0000","last_unstale":"2024-04-14T08:33:01.115389+0000","last_undegraded":"2024-04-14T08:33:01.115389+0000","last_fullsized":"2024-04-14T08:33:01.115389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226252+0000","last_change":"2024-04-14T08:32:54.198283+0000","last_active":"2024-04-14T08:32:59.226252+0000","last_peered":"2024-04-14T08:32:59.226252+0000","last_clean":"2024-04-14T08:32:59.226252+0000","last_became_active":"2024-04-14T08:32:54.197647+0000","last_became_peered":"2024-04-14T08:32:54.197647+0000","last_unstale":"2024-04-14T08:32:59.226252+0000","last_undegraded":"2024-04-14T08:32:59.226252+0000","last_fullsized":"2024-04-14T08:32:59.226252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:48.960689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278917+0000","last_change":"2024-04-14T08:32:54.204246+0000","last_active":"2024-04-14T08:33:01.278917+0000","last_peered":"2024-04-14T08:33:01.278917+0000","last_clean":"2024-04-14T08:33:01.278917+0000","last_became_active":"2024-04-14T08:32:54.203956+0000","last_became_peered":"2024-04-14T08:32:54.203956+0000","last_unstale":"2024-04-14T08:33:01.278917+0000","last_undegraded":"2024-04-14T08:33:01.278917+0000","last_fullsized":"2024-04-14T08:33:01.278917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:33.086425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405290+0000","last_change":"2024-04-14T08:32:54.207841+0000","last_active":"2024-04-14T08:32:59.405290+0000","last_peered":"2024-04-14T08:32:59.405290+0000","last_clean":"2024-04-14T08:32:59.405290+0000","last_became_active":"2024-04-14T08:32:54.207412+0000","last_became_peered":"2024-04-14T08:32:54.207412+0000","last_unstale":"2024-04-14T08:32:59.405290+0000","last_undegraded":"2024-04-14T08:32:59.405290+0000","last_fullsized":"2024-04-14T08:32:59.405290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:48.043054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404630+0000","last_change":"2024-04-14T08:32:54.207321+0000","last_active":"2024-04-14T08:32:59.404630+0000","last_peered":"2024-04-14T08:32:59.404630+0000","last_clean":"2024-04-14T08:32:59.404630+0000","last_became_active":"2024-04-14T08:32:54.207058+0000","last_became_peered":"2024-04-14T08:32:54.207058+0000","last_unstale":"2024-04-14T08:32:59.404630+0000","last_undegraded":"2024-04-14T08:32:59.404630+0000","last_fullsized":"2024-04-14T08:32:59.404630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610020+0000","last_change":"2024-04-14T08:32:54.205919+0000","last_active":"2024-04-14T08:33:00.610020+0000","last_peered":"2024-04-14T08:33:00.610020+0000","last_clean":"2024-04-14T08:33:00.610020+0000","last_became_active":"2024-04-14T08:32:54.205623+0000","last_became_peered":"2024-04-14T08:32:54.205623+0000","last_unstale":"2024-04-14T08:33:00.610020+0000","last_undegraded":"2024-04-14T08:33:00.610020+0000","last_fullsized":"2024-04-14T08:33:00.610020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220007+0000","last_change":"2024-04-14T08:32:54.196317+0000","last_active":"2024-04-14T08:33:01.220007+0000","last_peered":"2024-04-14T08:33:01.220007+0000","last_clean":"2024-04-14T08:33:01.220007+0000","last_became_active":"2024-04-14T08:32:54.195481+0000","last_became_peered":"2024-04-14T08:32:54.195481+0000","last_unstale":"2024-04-14T08:33:01.220007+0000","last_undegraded":"2024-04-14T08:33:01.220007+0000","last_fullsized":"2024-04-14T08:33:01.220007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:17:06.203553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224636+0000","last_change":"2024-04-14T08:32:54.203801+0000","last_active":"2024-04-14T08:32:59.224636+0000","last_peered":"2024-04-14T08:32:59.224636+0000","last_clean":"2024-04-14T08:32:59.224636+0000","last_became_active":"2024-04-14T08:32:54.203524+0000","last_became_peered":"2024-04-14T08:32:54.203524+0000","last_unstale":"2024-04-14T08:32:59.224636+0000","last_undegraded":"2024-04-14T08:32:59.224636+0000","last_fullsized":"2024-04-14T08:32:59.224636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225873+0000","last_change":"2024-04-14T08:32:54.204997+0000","last_active":"2024-04-14T08:32:59.225873+0000","last_peered":"2024-04-14T08:32:59.225873+0000","last_clean":"2024-04-14T08:32:59.225873+0000","last_became_active":"2024-04-14T08:32:54.204757+0000","last_became_peered":"2024-04-14T08:32:54.204757+0000","last_unstale":"2024-04-14T08:32:59.225873+0000","last_undegraded":"2024-04-14T08:32:59.225873+0000","last_fullsized":"2024-04-14T08:32:59.225873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:49.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115444+0000","last_change":"2024-04-14T08:32:54.210277+0000","last_active":"2024-04-14T08:33:01.115444+0000","last_peered":"2024-04-14T08:33:01.115444+0000","last_clean":"2024-04-14T08:33:01.115444+0000","last_became_active":"2024-04-14T08:32:54.209548+0000","last_became_peered":"2024-04-14T08:32:54.209548+0000","last_unstale":"2024-04-14T08:33:01.115444+0000","last_undegraded":"2024-04-14T08:33:01.115444+0000","last_fullsized":"2024-04-14T08:33:01.115444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115625+0000","last_change":"2024-04-14T08:32:54.213828+0000","last_active":"2024-04-14T08:33:01.115625+0000","last_peered":"2024-04-14T08:33:01.115625+0000","last_clean":"2024-04-14T08:33:01.115625+0000","last_became_active":"2024-04-14T08:32:54.213551+0000","last_became_peered":"2024-04-14T08:32:54.213551+0000","last_unstale":"2024-04-14T08:33:01.115625+0000","last_undegraded":"2024-04-14T08:33:01.115625+0000","last_fullsized":"2024-04-14T08:33:01.115625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226176+0000","last_change":"2024-04-14T08:32:54.212290+0000","last_active":"2024-04-14T08:32:59.226176+0000","last_peered":"2024-04-14T08:32:59.226176+0000","last_clean":"2024-04-14T08:32:59.226176+0000","last_became_active":"2024-04-14T08:32:54.212014+0000","last_became_peered":"2024-04-14T08:32:54.212014+0000","last_unstale":"2024-04-14T08:32:59.226176+0000","last_undegraded":"2024-04-14T08:32:59.226176+0000","last_fullsized":"2024-04-14T08:32:59.226176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:50.722301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224300+0000","last_change":"2024-04-14T08:32:54.209523+0000","last_active":"2024-04-14T08:32:59.224300+0000","last_peered":"2024-04-14T08:32:59.224300+0000","last_clean":"2024-04-14T08:32:59.224300+0000","last_became_active":"2024-04-14T08:32:54.209262+0000","last_became_peered":"2024-04-14T08:32:54.209262+0000","last_unstale":"2024-04-14T08:32:59.224300+0000","last_undegraded":"2024-04-14T08:32:59.224300+0000","last_fullsized":"2024-04-14T08:32:59.224300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:25:28.934928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225401+0000","last_change":"2024-04-14T08:32:54.195208+0000","last_active":"2024-04-14T08:32:59.225401+0000","last_peered":"2024-04-14T08:32:59.225401+0000","last_clean":"2024-04-14T08:32:59.225401+0000","last_became_active":"2024-04-14T08:32:54.194916+0000","last_became_peered":"2024-04-14T08:32:54.194916+0000","last_unstale":"2024-04-14T08:32:59.225401+0000","last_undegraded":"2024-04-14T08:32:59.225401+0000","last_fullsized":"2024-04-14T08:32:59.225401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:16.536477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404780+0000","last_change":"2024-04-14T08:32:54.208737+0000","last_active":"2024-04-14T08:32:59.404780+0000","last_peered":"2024-04-14T08:32:59.404780+0000","last_clean":"2024-04-14T08:32:59.404780+0000","last_became_active":"2024-04-14T08:32:54.208439+0000","last_became_peered":"2024-04-14T08:32:54.208439+0000","last_unstale":"2024-04-14T08:32:59.404780+0000","last_undegraded":"2024-04-14T08:32:59.404780+0000","last_fullsized":"2024-04-14T08:32:59.404780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:48.960689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225788+0000","last_change":"2024-04-14T08:32:54.198295+0000","last_active":"2024-04-14T08:32:59.225788+0000","last_peered":"2024-04-14T08:32:59.225788+0000","last_clean":"2024-04-14T08:32:59.225788+0000","last_became_active":"2024-04-14T08:32:54.198052+0000","last_became_peered":"2024-04-14T08:32:54.198052+0000","last_unstale":"2024-04-14T08:32:59.225788+0000","last_undegraded":"2024-04-14T08:32:59.225788+0000","last_fullsized":"2024-04-14T08:32:59.225788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:48.043054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404955+0000","last_change":"2024-04-14T08:32:54.206537+0000","last_active":"2024-04-14T08:32:59.404955+0000","last_peered":"2024-04-14T08:32:59.404955+0000","last_clean":"2024-04-14T08:32:59.404955+0000","last_became_active":"2024-04-14T08:32:54.206246+0000","last_became_peered":"2024-04-14T08:32:54.206246+0000","last_unstale":"2024-04-14T08:32:59.404955+0000","last_undegraded":"2024-04-14T08:32:59.404955+0000","last_fullsized":"2024-04-14T08:32:59.404955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:16.536477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220192+0000","last_change":"2024-04-14T08:32:54.201364+0000","last_active":"2024-04-14T08:33:01.220192+0000","last_peered":"2024-04-14T08:33:01.220192+0000","last_clean":"2024-04-14T08:33:01.220192+0000","last_became_active":"2024-04-14T08:32:54.199521+0000","last_became_peered":"2024-04-14T08:32:54.199521+0000","last_unstale":"2024-04-14T08:33:01.220192+0000","last_undegraded":"2024-04-14T08:33:01.220192+0000","last_fullsized":"2024-04-14T08:33:01.220192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610554+0000","last_change":"2024-04-14T08:32:54.199458+0000","last_active":"2024-04-14T08:33:00.610554+0000","last_peered":"2024-04-14T08:33:00.610554+0000","last_clean":"2024-04-14T08:33:00.610554+0000","last_became_active":"2024-04-14T08:32:54.199203+0000","last_became_peered":"2024-04-14T08:32:54.199203+0000","last_unstale":"2024-04-14T08:33:00.610554+0000","last_undegraded":"2024-04-14T08:33:00.610554+0000","last_fullsized":"2024-04-14T08:33:00.610554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226040+0000","last_change":"2024-04-14T08:32:54.206343+0000","last_active":"2024-04-14T08:32:59.226040+0000","last_peered":"2024-04-14T08:32:59.226040+0000","last_clean":"2024-04-14T08:32:59.226040+0000","last_became_active":"2024-04-14T08:32:54.205925+0000","last_became_peered":"2024-04-14T08:32:54.205925+0000","last_unstale":"2024-04-14T08:32:59.226040+0000","last_undegraded":"2024-04-14T08:32:59.226040+0000","last_fullsized":"2024-04-14T08:32:59.226040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:12.683816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:33:11.247 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:33:11.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.381+0000 7fec53908640 1 Processor -- start 2024-04-14T08:33:11.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.382+0000 7fec53908640 1 -- start start 2024-04-14T08:33:11.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.382+0000 7fec53908640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fec4c0b1df0 0x7fec4c0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.382+0000 7fec53908640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fec4c0b4040 0x7fec4c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.382+0000 7fec53908640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec4c15c770 con 0x7fec4c0b21d0 2024-04-14T08:33:11.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.382+0000 7fec53908640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fec4c0b3430 con 0x7fec4c0b4040 2024-04-14T08:33:11.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.382+0000 7fec53908640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fec4c0b3700 con 0x7fec4c0b1df0 2024-04-14T08:33:11.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.383+0000 7fec51ea5640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fec4c0b4040 0x7fec4c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.383+0000 7fec51ea5640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fec4c0b4040 0x7fec4c0b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49318/0 (socket says 172.21.15.172:49318) 2024-04-14T08:33:11.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.383+0000 7fec51ea5640 1 -- 172.21.15.172:0/961041755 learned_addr learned my addr 172.21.15.172:0/961041755 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:11.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.383+0000 7fec50ea3640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fec4c0b21d0 0x7fec4c15c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35594/0 (socket says 172.21.15.172:35594) 2024-04-14T08:33:11.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.383+0000 7fec516a4640 1 --2- 172.21.15.172:0/961041755 >> v2:172.21.15.177:3300/0 conn(0x7fec4c0b1df0 0x7fec4c0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.383+0000 7fec51ea5640 1 -- 172.21.15.172:0/961041755 >> v2:172.21.15.177:3300/0 conn(0x7fec4c0b1df0 msgr2=0x7fec4c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:11.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.383+0000 7fec51ea5640 1 --2- 172.21.15.172:0/961041755 >> v2:172.21.15.177:3300/0 conn(0x7fec4c0b1df0 0x7fec4c0b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.383+0000 7fec51ea5640 1 -- 172.21.15.172:0/961041755 >> v1:172.21.15.172:6789/0 conn(0x7fec4c0b21d0 legacy=0x7fec4c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.383+0000 7fec51ea5640 1 -- 172.21.15.172:0/961041755 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c0b39e0 con 0x7fec4c0b4040 2024-04-14T08:33:11.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.383+0000 7fec43fff640 1 -- 172.21.15.172:0/961041755 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3215046145 0 0) 0x7fec4c15c770 con 0x7fec4c0b21d0 2024-04-14T08:33:11.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.384+0000 7fec51ea5640 1 --2- 172.21.15.172:0/961041755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec4c0b4040 0x7fec4c0b4460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fec4c06f4b0 tx=0x7fec4402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179a9f0af214f40b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:11.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.384+0000 7fec43fff640 1 -- 172.21.15.172:0/961041755 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4402ca80 con 0x7fec4c0b4040 2024-04-14T08:33:11.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.384+0000 7fec43fff640 1 -- 172.21.15.172:0/961041755 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec4402cc20 con 0x7fec4c0b4040 2024-04-14T08:33:11.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.384+0000 7fec43fff640 1 -- 172.21.15.172:0/961041755 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec440343f0 con 0x7fec4c0b4040 2024-04-14T08:33:11.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.384+0000 7fec53908640 1 -- 172.21.15.172:0/961041755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec4c0b4040 msgr2=0x7fec4c0b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.384+0000 7fec53908640 1 --2- 172.21.15.172:0/961041755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec4c0b4040 0x7fec4c0b4460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fec4c06f4b0 tx=0x7fec4402bee0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.384+0000 7fec53908640 1 -- 172.21.15.172:0/961041755 shutdown_connections 2024-04-14T08:33:11.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.384+0000 7fec53908640 1 --2- 172.21.15.172:0/961041755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec4c0b4040 0x7fec4c0b4460 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.384+0000 7fec53908640 1 --2- 172.21.15.172:0/961041755 >> v2:172.21.15.177:3300/0 conn(0x7fec4c0b1df0 0x7fec4c0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.384+0000 7fec53908640 1 -- 172.21.15.172:0/961041755 >> 172.21.15.172:0/961041755 conn(0x7fec4c06ec30 msgr2=0x7fec4c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:11.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.385+0000 7fec53908640 1 -- 172.21.15.172:0/961041755 shutdown_connections 2024-04-14T08:33:11.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.385+0000 7fec53908640 1 -- 172.21.15.172:0/961041755 wait complete. 2024-04-14T08:33:11.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.386+0000 7fec53908640 1 Processor -- start 2024-04-14T08:33:11.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.386+0000 7fec53908640 1 -- start start 2024-04-14T08:33:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.386+0000 7fec53908640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec4c0b1df0 0x7fec4c1348d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.386+0000 7fec53908640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec4c0b21d0 0x7fec4c0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.386+0000 7fec53908640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec4c0b4040 0x7fec4c1579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.386+0000 7fec53908640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c0b49a0 con 0x7fec4c0b21d0 2024-04-14T08:33:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.386+0000 7fec53908640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c15bb40 con 0x7fec4c0b1df0 2024-04-14T08:33:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.386+0000 7fec53908640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fec4c15b7f0 con 0x7fec4c0b4040 2024-04-14T08:33:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.386+0000 7fec50ea3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec4c0b21d0 0x7fec4c0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec516a4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec4c0b1df0 0x7fec4c1348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec50ea3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec4c0b21d0 0x7fec4c0acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49332/0 (socket says 172.21.15.172:49332) 2024-04-14T08:33:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec50ea3640 1 -- 172.21.15.172:0/2949787764 learned_addr learned my addr 172.21.15.172:0/2949787764 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec51ea5640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec4c0b4040 0x7fec4c1579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec50ea3640 1 -- 172.21.15.172:0/2949787764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec4c0b4040 msgr2=0x7fec4c1579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec50ea3640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec4c0b4040 0x7fec4c1579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec50ea3640 1 -- 172.21.15.172:0/2949787764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec4c0b1df0 msgr2=0x7fec4c1348d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec50ea3640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec4c0b1df0 0x7fec4c1348d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec50ea3640 1 -- 172.21.15.172:0/2949787764 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c157f10 con 0x7fec4c0b21d0 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec50ea3640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec4c0b21d0 0x7fec4c0acf00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00c4e0 tx=0x7fec3c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec41ffb640 1 -- 172.21.15.172:0/2949787764 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c018070 con 0x7fec4c0b21d0 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec41ffb640 1 -- 172.21.15.172:0/2949787764 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec3c007500 con 0x7fec4c0b21d0 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec53908640 1 -- 172.21.15.172:0/2949787764 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c0abe60 con 0x7fec4c0b21d0 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec41ffb640 1 -- 172.21.15.172:0/2949787764 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c004a20 con 0x7fec4c0b21d0 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.387+0000 7fec53908640 1 -- 172.21.15.172:0/2949787764 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c0ac420 con 0x7fec4c0b21d0 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.388+0000 7fec53908640 1 -- 172.21.15.172:0/2949787764 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec4c068520 con 0x7fec4c0b21d0 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.388+0000 7fec41ffb640 1 -- 172.21.15.172:0/2949787764 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec3c01b070 con 0x7fec4c0b21d0 2024-04-14T08:33:11.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.389+0000 7fec41ffb640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec3006c8f0 0x7fec3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.389+0000 7fec41ffb640 1 -- 172.21.15.172:0/2949787764 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fec3c08b090 con 0x7fec4c0b21d0 2024-04-14T08:33:11.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.389+0000 7fec516a4640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec3006c8f0 0x7fec3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.389+0000 7fec516a4640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec3006c8f0 0x7fec3006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec48009b40 tx=0x7fec48009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:11.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.393+0000 7fec41ffb640 1 -- 172.21.15.172:0/2949787764 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fec4c157f10 con 0x7fec4c0b21d0 2024-04-14T08:33:11.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.566+0000 7fec53908640 1 -- 172.21.15.172:0/2949787764 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fec4c0cfe10 con 0x7fec3006c8f0 2024-04-14T08:33:11.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.574+0000 7fec41ffb640 1 -- 172.21.15.172:0/2949787764 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fec4c0cfe10 con 0x7fec3006c8f0 2024-04-14T08:33:11.577 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:11.579 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:33:11.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.577+0000 7fec53908640 1 -- 172.21.15.172:0/2949787764 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec3006c8f0 msgr2=0x7fec3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.577+0000 7fec53908640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec3006c8f0 0x7fec3006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec48009b40 tx=0x7fec48009190 comp rx=0 tx=0).stop 2024-04-14T08:33:11.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.577+0000 7fec53908640 1 -- 172.21.15.172:0/2949787764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec4c0b21d0 msgr2=0x7fec4c0acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.577+0000 7fec53908640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec4c0b21d0 0x7fec4c0acf00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00c4e0 tx=0x7fec3c00c9a0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.577+0000 7fec53908640 1 -- 172.21.15.172:0/2949787764 shutdown_connections 2024-04-14T08:33:11.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.577+0000 7fec53908640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec3006c8f0 0x7fec3006edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.577+0000 7fec53908640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec4c0b4040 0x7fec4c1579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.577+0000 7fec53908640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec4c0b21d0 0x7fec4c0acf00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.577+0000 7fec53908640 1 --2- 172.21.15.172:0/2949787764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec4c0b1df0 0x7fec4c1348d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.577+0000 7fec53908640 1 -- 172.21.15.172:0/2949787764 >> 172.21.15.172:0/2949787764 conn(0x7fec4c06ec30 msgr2=0x7fec4c15ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:11.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.577+0000 7fec53908640 1 -- 172.21.15.172:0/2949787764 shutdown_connections 2024-04-14T08:33:11.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.577+0000 7fec53908640 1 -- 172.21.15.172:0/2949787764 wait complete. 2024-04-14T08:33:11.589 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-14T08:33:11.124560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219480,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755240,"statfs":{"total":773094113280,"available":772869365760,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.901572"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219633+0000","last_change":"2024-04-14T08:32:53.196809+0000","last_active":"2024-04-14T08:33:01.219633+0000","last_peered":"2024-04-14T08:33:01.219633+0000","last_clean":"2024-04-14T08:33:01.219633+0000","last_became_active":"2024-04-14T08:32:53.196554+0000","last_became_peered":"2024-04-14T08:32:53.196554+0000","last_unstale":"2024-04-14T08:33:01.219633+0000","last_undegraded":"2024-04-14T08:33:01.219633+0000","last_fullsized":"2024-04-14T08:33:01.219633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:43:41.734066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225244+0000","last_change":"2024-04-14T08:32:54.204652+0000","last_active":"2024-04-14T08:32:59.225244+0000","last_peered":"2024-04-14T08:32:59.225244+0000","last_clean":"2024-04-14T08:32:59.225244+0000","last_became_active":"2024-04-14T08:32:54.204361+0000","last_became_peered":"2024-04-14T08:32:54.204361+0000","last_unstale":"2024-04-14T08:32:59.225244+0000","last_undegraded":"2024-04-14T08:32:59.225244+0000","last_fullsized":"2024-04-14T08:32:59.225244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224356+0000","last_change":"2024-04-14T08:32:58.244817+0000","last_active":"2024-04-14T08:32:59.224356+0000","last_peered":"2024-04-14T08:32:59.224356+0000","last_clean":"2024-04-14T08:32:59.224356+0000","last_became_active":"2024-04-14T08:32:58.243913+0000","last_became_peered":"2024-04-14T08:32:58.243913+0000","last_unstale":"2024-04-14T08:32:59.224356+0000","last_undegraded":"2024-04-14T08:32:59.224356+0000","last_fullsized":"2024-04-14T08:32:59.224356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:53.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225340+0000","last_change":"2024-04-14T08:32:53.202277+0000","last_active":"2024-04-14T08:32:59.225340+0000","last_peered":"2024-04-14T08:32:59.225340+0000","last_clean":"2024-04-14T08:32:59.225340+0000","last_became_active":"2024-04-14T08:32:53.201980+0000","last_became_peered":"2024-04-14T08:32:53.201980+0000","last_unstale":"2024-04-14T08:32:59.225340+0000","last_undegraded":"2024-04-14T08:32:59.225340+0000","last_fullsized":"2024-04-14T08:32:59.225340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:22:15.886932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220320+0000","last_change":"2024-04-14T08:32:54.211674+0000","last_active":"2024-04-14T08:33:01.220320+0000","last_peered":"2024-04-14T08:33:01.220320+0000","last_clean":"2024-04-14T08:33:01.220320+0000","last_became_active":"2024-04-14T08:32:54.211323+0000","last_became_peered":"2024-04-14T08:32:54.211323+0000","last_unstale":"2024-04-14T08:33:01.220320+0000","last_undegraded":"2024-04-14T08:33:01.220320+0000","last_fullsized":"2024-04-14T08:33:01.220320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224678+0000","last_change":"2024-04-14T08:32:58.242594+0000","last_active":"2024-04-14T08:32:59.224678+0000","last_peered":"2024-04-14T08:32:59.224678+0000","last_clean":"2024-04-14T08:32:59.224678+0000","last_became_active":"2024-04-14T08:32:58.242238+0000","last_became_peered":"2024-04-14T08:32:58.242238+0000","last_unstale":"2024-04-14T08:32:59.224678+0000","last_undegraded":"2024-04-14T08:32:59.224678+0000","last_fullsized":"2024-04-14T08:32:59.224678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:45.716305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226152+0000","last_change":"2024-04-14T08:32:53.200133+0000","last_active":"2024-04-14T08:32:59.226152+0000","last_peered":"2024-04-14T08:32:59.226152+0000","last_clean":"2024-04-14T08:32:59.226152+0000","last_became_active":"2024-04-14T08:32:53.199380+0000","last_became_peered":"2024-04-14T08:32:53.199380+0000","last_unstale":"2024-04-14T08:32:59.226152+0000","last_undegraded":"2024-04-14T08:32:59.226152+0000","last_fullsized":"2024-04-14T08:32:59.226152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405017+0000","last_change":"2024-04-14T08:32:54.208922+0000","last_active":"2024-04-14T08:32:59.405017+0000","last_peered":"2024-04-14T08:32:59.405017+0000","last_clean":"2024-04-14T08:32:59.405017+0000","last_became_active":"2024-04-14T08:32:54.206845+0000","last_became_peered":"2024-04-14T08:32:54.206845+0000","last_unstale":"2024-04-14T08:32:59.405017+0000","last_undegraded":"2024-04-14T08:32:59.405017+0000","last_fullsized":"2024-04-14T08:32:59.405017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225258+0000","last_change":"2024-04-14T08:32:58.242683+0000","last_active":"2024-04-14T08:32:59.225258+0000","last_peered":"2024-04-14T08:32:59.225258+0000","last_clean":"2024-04-14T08:32:59.225258+0000","last_became_active":"2024-04-14T08:32:58.242189+0000","last_became_peered":"2024-04-14T08:32:58.242189+0000","last_unstale":"2024-04-14T08:32:59.225258+0000","last_undegraded":"2024-04-14T08:32:59.225258+0000","last_fullsized":"2024-04-14T08:32:59.225258+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:52.084133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225875+0000","last_change":"2024-04-14T08:32:53.200880+0000","last_active":"2024-04-14T08:32:59.225875+0000","last_peered":"2024-04-14T08:32:59.225875+0000","last_clean":"2024-04-14T08:32:59.225875+0000","last_became_active":"2024-04-14T08:32:53.200615+0000","last_became_peered":"2024-04-14T08:32:53.200615+0000","last_unstale":"2024-04-14T08:32:59.225875+0000","last_undegraded":"2024-04-14T08:32:59.225875+0000","last_fullsized":"2024-04-14T08:32:59.225875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:40:24.515179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405104+0000","last_change":"2024-04-14T08:32:54.207091+0000","last_active":"2024-04-14T08:32:59.405104+0000","last_peered":"2024-04-14T08:32:59.405104+0000","last_clean":"2024-04-14T08:32:59.405104+0000","last_became_active":"2024-04-14T08:32:54.206685+0000","last_became_peered":"2024-04-14T08:32:54.206685+0000","last_unstale":"2024-04-14T08:32:59.405104+0000","last_undegraded":"2024-04-14T08:32:59.405104+0000","last_fullsized":"2024-04-14T08:32:59.405104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:41:14.768103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224936+0000","last_change":"2024-04-14T08:32:58.237435+0000","last_active":"2024-04-14T08:32:59.224936+0000","last_peered":"2024-04-14T08:32:59.224936+0000","last_clean":"2024-04-14T08:32:59.224936+0000","last_became_active":"2024-04-14T08:32:58.237056+0000","last_became_peered":"2024-04-14T08:32:58.237056+0000","last_unstale":"2024-04-14T08:32:59.224936+0000","last_undegraded":"2024-04-14T08:32:59.224936+0000","last_fullsized":"2024-04-14T08:32:59.224936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:20.577556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115125+0000","last_change":"2024-04-14T08:32:54.202442+0000","last_active":"2024-04-14T08:33:01.115125+0000","last_peered":"2024-04-14T08:33:01.115125+0000","last_clean":"2024-04-14T08:33:01.115125+0000","last_became_active":"2024-04-14T08:32:54.201860+0000","last_became_peered":"2024-04-14T08:32:54.201860+0000","last_unstale":"2024-04-14T08:33:01.115125+0000","last_undegraded":"2024-04-14T08:33:01.115125+0000","last_fullsized":"2024-04-14T08:33:01.115125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226036+0000","last_change":"2024-04-14T08:32:53.194232+0000","last_active":"2024-04-14T08:32:59.226036+0000","last_peered":"2024-04-14T08:32:59.226036+0000","last_clean":"2024-04-14T08:32:59.226036+0000","last_became_active":"2024-04-14T08:32:53.193961+0000","last_became_peered":"2024-04-14T08:32:53.193961+0000","last_unstale":"2024-04-14T08:32:59.226036+0000","last_undegraded":"2024-04-14T08:32:59.226036+0000","last_fullsized":"2024-04-14T08:32:59.226036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:32.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226274+0000","last_change":"2024-04-14T08:32:58.239778+0000","last_active":"2024-04-14T08:32:59.226274+0000","last_peered":"2024-04-14T08:32:59.226274+0000","last_clean":"2024-04-14T08:32:59.226274+0000","last_became_active":"2024-04-14T08:32:58.239445+0000","last_became_peered":"2024-04-14T08:32:58.239445+0000","last_unstale":"2024-04-14T08:32:59.226274+0000","last_undegraded":"2024-04-14T08:32:59.226274+0000","last_fullsized":"2024-04-14T08:32:59.226274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:53.001768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224657+0000","last_change":"2024-04-14T08:32:54.202322+0000","last_active":"2024-04-14T08:32:59.224657+0000","last_peered":"2024-04-14T08:32:59.224657+0000","last_clean":"2024-04-14T08:32:59.224657+0000","last_became_active":"2024-04-14T08:32:54.202065+0000","last_became_peered":"2024-04-14T08:32:54.202065+0000","last_unstale":"2024-04-14T08:32:59.224657+0000","last_undegraded":"2024-04-14T08:32:59.224657+0000","last_fullsized":"2024-04-14T08:32:59.224657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610551+0000","last_change":"2024-04-14T08:32:53.195288+0000","last_active":"2024-04-14T08:33:00.610551+0000","last_peered":"2024-04-14T08:33:00.610551+0000","last_clean":"2024-04-14T08:33:00.610551+0000","last_became_active":"2024-04-14T08:32:53.195009+0000","last_became_peered":"2024-04-14T08:32:53.195009+0000","last_unstale":"2024-04-14T08:33:00.610551+0000","last_undegraded":"2024-04-14T08:33:00.610551+0000","last_fullsized":"2024-04-14T08:33:00.610551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225775+0000","last_change":"2024-04-14T08:32:58.238192+0000","last_active":"2024-04-14T08:32:59.225775+0000","last_peered":"2024-04-14T08:32:59.225775+0000","last_clean":"2024-04-14T08:32:59.225775+0000","last_became_active":"2024-04-14T08:32:58.237917+0000","last_became_peered":"2024-04-14T08:32:58.237917+0000","last_unstale":"2024-04-14T08:32:59.225775+0000","last_undegraded":"2024-04-14T08:32:59.225775+0000","last_fullsized":"2024-04-14T08:32:59.225775+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:41:18.809183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224457+0000","last_change":"2024-04-14T08:32:54.195485+0000","last_active":"2024-04-14T08:32:59.224457+0000","last_peered":"2024-04-14T08:32:59.224457+0000","last_clean":"2024-04-14T08:32:59.224457+0000","last_became_active":"2024-04-14T08:32:54.195192+0000","last_became_peered":"2024-04-14T08:32:54.195192+0000","last_unstale":"2024-04-14T08:32:59.224457+0000","last_undegraded":"2024-04-14T08:32:59.224457+0000","last_fullsized":"2024-04-14T08:32:59.224457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610087+0000","last_change":"2024-04-14T08:32:53.193700+0000","last_active":"2024-04-14T08:33:00.610087+0000","last_peered":"2024-04-14T08:33:00.610087+0000","last_clean":"2024-04-14T08:33:00.610087+0000","last_became_active":"2024-04-14T08:32:53.193412+0000","last_became_peered":"2024-04-14T08:32:53.193412+0000","last_unstale":"2024-04-14T08:33:00.610087+0000","last_undegraded":"2024-04-14T08:33:00.610087+0000","last_fullsized":"2024-04-14T08:33:00.610087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610793+0000","last_change":"2024-04-14T08:32:58.246138+0000","last_active":"2024-04-14T08:33:00.610793+0000","last_peered":"2024-04-14T08:33:00.610793+0000","last_clean":"2024-04-14T08:33:00.610793+0000","last_became_active":"2024-04-14T08:32:58.245731+0000","last_became_peered":"2024-04-14T08:32:58.245731+0000","last_unstale":"2024-04-14T08:33:00.610793+0000","last_undegraded":"2024-04-14T08:33:00.610793+0000","last_fullsized":"2024-04-14T08:33:00.610793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:54.763380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224951+0000","last_change":"2024-04-14T08:32:54.203712+0000","last_active":"2024-04-14T08:32:59.224951+0000","last_peered":"2024-04-14T08:32:59.224951+0000","last_clean":"2024-04-14T08:32:59.224951+0000","last_became_active":"2024-04-14T08:32:54.203433+0000","last_became_peered":"2024-04-14T08:32:54.203433+0000","last_unstale":"2024-04-14T08:32:59.224951+0000","last_undegraded":"2024-04-14T08:32:59.224951+0000","last_fullsized":"2024-04-14T08:32:59.224951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:15.534364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224734+0000","last_change":"2024-04-14T08:32:53.197850+0000","last_active":"2024-04-14T08:32:59.224734+0000","last_peered":"2024-04-14T08:32:59.224734+0000","last_clean":"2024-04-14T08:32:59.224734+0000","last_became_active":"2024-04-14T08:32:53.197575+0000","last_became_peered":"2024-04-14T08:32:53.197575+0000","last_unstale":"2024-04-14T08:32:59.224734+0000","last_undegraded":"2024-04-14T08:32:59.224734+0000","last_fullsized":"2024-04-14T08:32:59.224734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:04:36.870813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225529+0000","last_change":"2024-04-14T08:32:58.243185+0000","last_active":"2024-04-14T08:32:59.225529+0000","last_peered":"2024-04-14T08:32:59.225529+0000","last_clean":"2024-04-14T08:32:59.225529+0000","last_became_active":"2024-04-14T08:32:58.242880+0000","last_became_peered":"2024-04-14T08:32:58.242880+0000","last_unstale":"2024-04-14T08:32:59.225529+0000","last_undegraded":"2024-04-14T08:32:59.225529+0000","last_fullsized":"2024-04-14T08:32:59.225529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:47.641204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220461+0000","last_change":"2024-04-14T08:32:54.194367+0000","last_active":"2024-04-14T08:33:01.220461+0000","last_peered":"2024-04-14T08:33:01.220461+0000","last_clean":"2024-04-14T08:33:01.220461+0000","last_became_active":"2024-04-14T08:32:54.194111+0000","last_became_peered":"2024-04-14T08:32:54.194111+0000","last_unstale":"2024-04-14T08:33:01.220461+0000","last_undegraded":"2024-04-14T08:33:01.220461+0000","last_fullsized":"2024-04-14T08:33:01.220461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:05:03.511225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115632+0000","last_change":"2024-04-14T08:32:53.194969+0000","last_active":"2024-04-14T08:33:01.115632+0000","last_peered":"2024-04-14T08:33:01.115632+0000","last_clean":"2024-04-14T08:33:01.115632+0000","last_became_active":"2024-04-14T08:32:53.194665+0000","last_became_peered":"2024-04-14T08:32:53.194665+0000","last_unstale":"2024-04-14T08:33:01.115632+0000","last_undegraded":"2024-04-14T08:33:01.115632+0000","last_fullsized":"2024-04-14T08:33:01.115632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224778+0000","last_change":"2024-04-14T08:32:58.237006+0000","last_active":"2024-04-14T08:32:59.224778+0000","last_peered":"2024-04-14T08:32:59.224778+0000","last_clean":"2024-04-14T08:32:59.224778+0000","last_became_active":"2024-04-14T08:32:58.236366+0000","last_became_peered":"2024-04-14T08:32:58.236366+0000","last_unstale":"2024-04-14T08:32:59.224778+0000","last_undegraded":"2024-04-14T08:32:59.224778+0000","last_fullsized":"2024-04-14T08:32:59.224778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:49.446712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225123+0000","last_change":"2024-04-14T08:32:54.205064+0000","last_active":"2024-04-14T08:32:59.225123+0000","last_peered":"2024-04-14T08:32:59.225123+0000","last_clean":"2024-04-14T08:32:59.225123+0000","last_became_active":"2024-04-14T08:32:54.203946+0000","last_became_peered":"2024-04-14T08:32:54.203946+0000","last_unstale":"2024-04-14T08:32:59.225123+0000","last_undegraded":"2024-04-14T08:32:59.225123+0000","last_fullsized":"2024-04-14T08:32:59.225123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:16.536477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610666+0000","last_change":"2024-04-14T08:32:53.198148+0000","last_active":"2024-04-14T08:33:00.610666+0000","last_peered":"2024-04-14T08:33:00.610666+0000","last_clean":"2024-04-14T08:33:00.610666+0000","last_became_active":"2024-04-14T08:32:53.197878+0000","last_became_peered":"2024-04-14T08:32:53.197878+0000","last_unstale":"2024-04-14T08:33:00.610666+0000","last_undegraded":"2024-04-14T08:33:00.610666+0000","last_fullsized":"2024-04-14T08:33:00.610666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224235+0000","last_change":"2024-04-14T08:32:58.246772+0000","last_active":"2024-04-14T08:32:59.224235+0000","last_peered":"2024-04-14T08:32:59.224235+0000","last_clean":"2024-04-14T08:32:59.224235+0000","last_became_active":"2024-04-14T08:32:58.246388+0000","last_became_peered":"2024-04-14T08:32:58.246388+0000","last_unstale":"2024-04-14T08:32:59.224235+0000","last_undegraded":"2024-04-14T08:32:59.224235+0000","last_fullsized":"2024-04-14T08:32:59.224235+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:54.763380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610409+0000","last_change":"2024-04-14T08:32:54.194202+0000","last_active":"2024-04-14T08:33:00.610409+0000","last_peered":"2024-04-14T08:33:00.610409+0000","last_clean":"2024-04-14T08:33:00.610409+0000","last_became_active":"2024-04-14T08:32:54.193938+0000","last_became_peered":"2024-04-14T08:32:54.193938+0000","last_unstale":"2024-04-14T08:33:00.610409+0000","last_undegraded":"2024-04-14T08:33:00.610409+0000","last_fullsized":"2024-04-14T08:33:00.610409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226263+0000","last_change":"2024-04-14T08:32:53.202270+0000","last_active":"2024-04-14T08:32:59.226263+0000","last_peered":"2024-04-14T08:32:59.226263+0000","last_clean":"2024-04-14T08:32:59.226263+0000","last_became_active":"2024-04-14T08:32:53.201945+0000","last_became_peered":"2024-04-14T08:32:53.201945+0000","last_unstale":"2024-04-14T08:32:59.226263+0000","last_undegraded":"2024-04-14T08:32:59.226263+0000","last_fullsized":"2024-04-14T08:32:59.226263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219955+0000","last_change":"2024-04-14T08:32:58.248277+0000","last_active":"2024-04-14T08:33:01.219955+0000","last_peered":"2024-04-14T08:33:01.219955+0000","last_clean":"2024-04-14T08:33:01.219955+0000","last_became_active":"2024-04-14T08:32:58.247535+0000","last_became_peered":"2024-04-14T08:32:58.247535+0000","last_unstale":"2024-04-14T08:33:01.219955+0000","last_undegraded":"2024-04-14T08:33:01.219955+0000","last_fullsized":"2024-04-14T08:33:01.219955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:05.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405198+0000","last_change":"2024-04-14T08:32:54.203448+0000","last_active":"2024-04-14T08:32:59.405198+0000","last_peered":"2024-04-14T08:32:59.405198+0000","last_clean":"2024-04-14T08:32:59.405198+0000","last_became_active":"2024-04-14T08:32:54.203170+0000","last_became_peered":"2024-04-14T08:32:54.203170+0000","last_unstale":"2024-04-14T08:32:59.405198+0000","last_undegraded":"2024-04-14T08:32:59.405198+0000","last_fullsized":"2024-04-14T08:32:59.405198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:01.152061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610433+0000","last_change":"2024-04-14T08:32:53.187816+0000","last_active":"2024-04-14T08:33:00.610433+0000","last_peered":"2024-04-14T08:33:00.610433+0000","last_clean":"2024-04-14T08:33:00.610433+0000","last_became_active":"2024-04-14T08:32:53.187519+0000","last_became_peered":"2024-04-14T08:32:53.187519+0000","last_unstale":"2024-04-14T08:33:00.610433+0000","last_undegraded":"2024-04-14T08:33:00.610433+0000","last_fullsized":"2024-04-14T08:33:00.610433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610230+0000","last_change":"2024-04-14T08:32:58.246145+0000","last_active":"2024-04-14T08:33:00.610230+0000","last_peered":"2024-04-14T08:33:00.610230+0000","last_clean":"2024-04-14T08:33:00.610230+0000","last_became_active":"2024-04-14T08:32:58.245792+0000","last_became_peered":"2024-04-14T08:32:58.245792+0000","last_unstale":"2024-04-14T08:33:00.610230+0000","last_undegraded":"2024-04-14T08:33:00.610230+0000","last_fullsized":"2024-04-14T08:33:00.610230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:05.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225311+0000","last_change":"2024-04-14T08:32:54.192476+0000","last_active":"2024-04-14T08:32:59.225311+0000","last_peered":"2024-04-14T08:32:59.225311+0000","last_clean":"2024-04-14T08:32:59.225311+0000","last_became_active":"2024-04-14T08:32:54.192182+0000","last_became_peered":"2024-04-14T08:32:54.192182+0000","last_unstale":"2024-04-14T08:32:59.225311+0000","last_undegraded":"2024-04-14T08:32:59.225311+0000","last_fullsized":"2024-04-14T08:32:59.225311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:05.453493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405000+0000","last_change":"2024-04-14T08:32:53.201969+0000","last_active":"2024-04-14T08:32:59.405000+0000","last_peered":"2024-04-14T08:32:59.405000+0000","last_clean":"2024-04-14T08:32:59.405000+0000","last_became_active":"2024-04-14T08:32:53.201611+0000","last_became_peered":"2024-04-14T08:32:53.201611+0000","last_unstale":"2024-04-14T08:32:59.405000+0000","last_undegraded":"2024-04-14T08:32:59.405000+0000","last_fullsized":"2024-04-14T08:32:59.405000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610320+0000","last_change":"2024-04-14T08:32:58.249518+0000","last_active":"2024-04-14T08:33:00.610320+0000","last_peered":"2024-04-14T08:33:00.610320+0000","last_clean":"2024-04-14T08:33:00.610320+0000","last_became_active":"2024-04-14T08:32:58.249079+0000","last_became_peered":"2024-04-14T08:32:58.249079+0000","last_unstale":"2024-04-14T08:33:00.610320+0000","last_undegraded":"2024-04-14T08:33:00.610320+0000","last_fullsized":"2024-04-14T08:33:00.610320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:16.724895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610081+0000","last_change":"2024-04-14T08:32:54.194141+0000","last_active":"2024-04-14T08:33:00.610081+0000","last_peered":"2024-04-14T08:33:00.610081+0000","last_clean":"2024-04-14T08:33:00.610081+0000","last_became_active":"2024-04-14T08:32:54.193884+0000","last_became_peered":"2024-04-14T08:32:54.193884+0000","last_unstale":"2024-04-14T08:33:00.610081+0000","last_undegraded":"2024-04-14T08:33:00.610081+0000","last_fullsized":"2024-04-14T08:33:00.610081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278559+0000","last_change":"2024-04-14T08:32:53.196690+0000","last_active":"2024-04-14T08:33:01.278559+0000","last_peered":"2024-04-14T08:33:01.278559+0000","last_clean":"2024-04-14T08:33:01.278559+0000","last_became_active":"2024-04-14T08:32:53.195977+0000","last_became_peered":"2024-04-14T08:32:53.195977+0000","last_unstale":"2024-04-14T08:33:01.278559+0000","last_undegraded":"2024-04-14T08:33:01.278559+0000","last_fullsized":"2024-04-14T08:33:01.278559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610859+0000","last_change":"2024-04-14T08:32:58.249862+0000","last_active":"2024-04-14T08:33:00.610859+0000","last_peered":"2024-04-14T08:33:00.610859+0000","last_clean":"2024-04-14T08:33:00.610859+0000","last_became_active":"2024-04-14T08:32:58.248928+0000","last_became_peered":"2024-04-14T08:32:58.248928+0000","last_unstale":"2024-04-14T08:33:00.610859+0000","last_undegraded":"2024-04-14T08:33:00.610859+0000","last_fullsized":"2024-04-14T08:33:00.610859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:53.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219816+0000","last_change":"2024-04-14T08:32:54.206032+0000","last_active":"2024-04-14T08:33:01.219816+0000","last_peered":"2024-04-14T08:33:01.219816+0000","last_clean":"2024-04-14T08:33:01.219816+0000","last_became_active":"2024-04-14T08:32:54.205781+0000","last_became_peered":"2024-04-14T08:32:54.205781+0000","last_unstale":"2024-04-14T08:33:01.219816+0000","last_undegraded":"2024-04-14T08:33:01.219816+0000","last_fullsized":"2024-04-14T08:33:01.219816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:25:28.934928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224825+0000","last_change":"2024-04-14T08:32:53.195805+0000","last_active":"2024-04-14T08:32:59.224825+0000","last_peered":"2024-04-14T08:32:59.224825+0000","last_clean":"2024-04-14T08:32:59.224825+0000","last_became_active":"2024-04-14T08:32:53.195442+0000","last_became_peered":"2024-04-14T08:32:53.195442+0000","last_unstale":"2024-04-14T08:32:59.224825+0000","last_undegraded":"2024-04-14T08:32:59.224825+0000","last_fullsized":"2024-04-14T08:32:59.224825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:48.807150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220403+0000","last_change":"2024-04-14T08:32:58.247104+0000","last_active":"2024-04-14T08:33:01.220403+0000","last_peered":"2024-04-14T08:33:01.220403+0000","last_clean":"2024-04-14T08:33:01.220403+0000","last_became_active":"2024-04-14T08:32:58.246758+0000","last_became_peered":"2024-04-14T08:32:58.246758+0000","last_unstale":"2024-04-14T08:33:01.220403+0000","last_undegraded":"2024-04-14T08:33:01.220403+0000","last_fullsized":"2024-04-14T08:33:01.220403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:53.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224440+0000","last_change":"2024-04-14T08:32:54.209924+0000","last_active":"2024-04-14T08:32:59.224440+0000","last_peered":"2024-04-14T08:32:59.224440+0000","last_clean":"2024-04-14T08:32:59.224440+0000","last_became_active":"2024-04-14T08:32:54.209703+0000","last_became_peered":"2024-04-14T08:32:54.209703+0000","last_unstale":"2024-04-14T08:32:59.224440+0000","last_undegraded":"2024-04-14T08:32:59.224440+0000","last_fullsized":"2024-04-14T08:32:59.224440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404884+0000","last_change":"2024-04-14T08:32:53.197701+0000","last_active":"2024-04-14T08:32:59.404884+0000","last_peered":"2024-04-14T08:32:59.404884+0000","last_clean":"2024-04-14T08:32:59.404884+0000","last_became_active":"2024-04-14T08:32:53.197291+0000","last_became_peered":"2024-04-14T08:32:53.197291+0000","last_unstale":"2024-04-14T08:32:59.404884+0000","last_undegraded":"2024-04-14T08:32:59.404884+0000","last_fullsized":"2024-04-14T08:32:59.404884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:54:34.240052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224914+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:32:59.224914+0000","last_peered":"2024-04-14T08:32:59.224914+0000","last_clean":"2024-04-14T08:32:59.224914+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:32:59.224914+0000","last_undegraded":"2024-04-14T08:32:59.224914+0000","last_fullsized":"2024-04-14T08:32:59.224914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115413+0000","last_change":"2024-04-14T08:32:58.237249+0000","last_active":"2024-04-14T08:33:01.115413+0000","last_peered":"2024-04-14T08:33:01.115413+0000","last_clean":"2024-04-14T08:33:01.115413+0000","last_became_active":"2024-04-14T08:32:58.236861+0000","last_became_peered":"2024-04-14T08:32:58.236861+0000","last_unstale":"2024-04-14T08:33:01.115413+0000","last_undegraded":"2024-04-14T08:33:01.115413+0000","last_fullsized":"2024-04-14T08:33:01.115413+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:05.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226018+0000","last_change":"2024-04-14T08:32:54.211527+0000","last_active":"2024-04-14T08:32:59.226018+0000","last_peered":"2024-04-14T08:32:59.226018+0000","last_clean":"2024-04-14T08:32:59.226018+0000","last_became_active":"2024-04-14T08:32:54.211268+0000","last_became_peered":"2024-04-14T08:32:54.211268+0000","last_unstale":"2024-04-14T08:32:59.226018+0000","last_undegraded":"2024-04-14T08:32:59.226018+0000","last_fullsized":"2024-04-14T08:32:59.226018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:41:14.768103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278564+0000","last_change":"2024-04-14T08:32:53.198176+0000","last_active":"2024-04-14T08:33:01.278564+0000","last_peered":"2024-04-14T08:33:01.278564+0000","last_clean":"2024-04-14T08:33:01.278564+0000","last_became_active":"2024-04-14T08:32:53.197626+0000","last_became_peered":"2024-04-14T08:32:53.197626+0000","last_unstale":"2024-04-14T08:33:01.278564+0000","last_undegraded":"2024-04-14T08:33:01.278564+0000","last_fullsized":"2024-04-14T08:33:01.278564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:10:49.016747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115290+0000","last_change":"2024-04-14T08:32:58.240943+0000","last_active":"2024-04-14T08:33:01.115290+0000","last_peered":"2024-04-14T08:33:01.115290+0000","last_clean":"2024-04-14T08:33:01.115290+0000","last_became_active":"2024-04-14T08:32:58.240555+0000","last_became_peered":"2024-04-14T08:32:58.240555+0000","last_unstale":"2024-04-14T08:33:01.115290+0000","last_undegraded":"2024-04-14T08:33:01.115290+0000","last_fullsized":"2024-04-14T08:33:01.115290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:53.001768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405061+0000","last_change":"2024-04-14T08:32:54.205351+0000","last_active":"2024-04-14T08:32:59.405061+0000","last_peered":"2024-04-14T08:32:59.405061+0000","last_clean":"2024-04-14T08:32:59.405061+0000","last_became_active":"2024-04-14T08:32:54.205086+0000","last_became_peered":"2024-04-14T08:32:54.205086+0000","last_unstale":"2024-04-14T08:32:59.405061+0000","last_undegraded":"2024-04-14T08:32:59.405061+0000","last_fullsized":"2024-04-14T08:32:59.405061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:43.600124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610474+0000","last_change":"2024-04-14T08:32:53.194785+0000","last_active":"2024-04-14T08:33:00.610474+0000","last_peered":"2024-04-14T08:33:00.610474+0000","last_clean":"2024-04-14T08:33:00.610474+0000","last_became_active":"2024-04-14T08:32:53.194535+0000","last_became_peered":"2024-04-14T08:32:53.194535+0000","last_unstale":"2024-04-14T08:33:00.610474+0000","last_undegraded":"2024-04-14T08:33:00.610474+0000","last_fullsized":"2024-04-14T08:33:00.610474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:48.807150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220190+0000","last_change":"2024-04-14T08:32:58.246826+0000","last_active":"2024-04-14T08:33:01.220190+0000","last_peered":"2024-04-14T08:33:01.220190+0000","last_clean":"2024-04-14T08:33:01.220190+0000","last_became_active":"2024-04-14T08:32:58.246174+0000","last_became_peered":"2024-04-14T08:32:58.246174+0000","last_unstale":"2024-04-14T08:33:01.220190+0000","last_undegraded":"2024-04-14T08:33:01.220190+0000","last_fullsized":"2024-04-14T08:33:01.220190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:57.127403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226211+0000","last_change":"2024-04-14T08:32:54.200509+0000","last_active":"2024-04-14T08:32:59.226211+0000","last_peered":"2024-04-14T08:32:59.226211+0000","last_clean":"2024-04-14T08:32:59.226211+0000","last_became_active":"2024-04-14T08:32:54.199748+0000","last_became_peered":"2024-04-14T08:32:54.199748+0000","last_unstale":"2024-04-14T08:32:59.226211+0000","last_undegraded":"2024-04-14T08:32:59.226211+0000","last_fullsized":"2024-04-14T08:32:59.226211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:49.939225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278584+0000","last_change":"2024-04-14T08:32:53.196334+0000","last_active":"2024-04-14T08:33:01.278584+0000","last_peered":"2024-04-14T08:33:01.278584+0000","last_clean":"2024-04-14T08:33:01.278584+0000","last_became_active":"2024-04-14T08:32:53.196036+0000","last_became_peered":"2024-04-14T08:32:53.196036+0000","last_unstale":"2024-04-14T08:33:01.278584+0000","last_undegraded":"2024-04-14T08:33:01.278584+0000","last_fullsized":"2024-04-14T08:33:01.278584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404693+0000","last_change":"2024-04-14T08:32:58.239865+0000","last_active":"2024-04-14T08:32:59.404693+0000","last_peered":"2024-04-14T08:32:59.404693+0000","last_clean":"2024-04-14T08:32:59.404693+0000","last_became_active":"2024-04-14T08:32:58.239510+0000","last_became_peered":"2024-04-14T08:32:58.239510+0000","last_unstale":"2024-04-14T08:32:59.404693+0000","last_undegraded":"2024-04-14T08:32:59.404693+0000","last_fullsized":"2024-04-14T08:32:59.404693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:37:10.427564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226121+0000","last_change":"2024-04-14T08:32:54.200777+0000","last_active":"2024-04-14T08:32:59.226121+0000","last_peered":"2024-04-14T08:32:59.226121+0000","last_clean":"2024-04-14T08:32:59.226121+0000","last_became_active":"2024-04-14T08:32:54.200284+0000","last_became_peered":"2024-04-14T08:32:54.200284+0000","last_unstale":"2024-04-14T08:32:59.226121+0000","last_undegraded":"2024-04-14T08:32:59.226121+0000","last_fullsized":"2024-04-14T08:32:59.226121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:53.086323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225655+0000","last_change":"2024-04-14T08:32:53.184799+0000","last_active":"2024-04-14T08:32:59.225655+0000","last_peered":"2024-04-14T08:32:59.225655+0000","last_clean":"2024-04-14T08:32:59.225655+0000","last_became_active":"2024-04-14T08:32:53.184483+0000","last_became_peered":"2024-04-14T08:32:53.184483+0000","last_unstale":"2024-04-14T08:32:59.225655+0000","last_undegraded":"2024-04-14T08:32:59.225655+0000","last_fullsized":"2024-04-14T08:32:59.225655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:03:30.482046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224689+0000","last_change":"2024-04-14T08:32:58.243067+0000","last_active":"2024-04-14T08:32:59.224689+0000","last_peered":"2024-04-14T08:32:59.224689+0000","last_clean":"2024-04-14T08:32:59.224689+0000","last_became_active":"2024-04-14T08:32:58.242726+0000","last_became_peered":"2024-04-14T08:32:58.242726+0000","last_unstale":"2024-04-14T08:32:59.224689+0000","last_undegraded":"2024-04-14T08:32:59.224689+0000","last_fullsized":"2024-04-14T08:32:59.224689+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:09.583268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225539+0000","last_change":"2024-04-14T08:32:54.204697+0000","last_active":"2024-04-14T08:32:59.225539+0000","last_peered":"2024-04-14T08:32:59.225539+0000","last_clean":"2024-04-14T08:32:59.225539+0000","last_became_active":"2024-04-14T08:32:54.204357+0000","last_became_peered":"2024-04-14T08:32:54.204357+0000","last_unstale":"2024-04-14T08:32:59.225539+0000","last_undegraded":"2024-04-14T08:32:59.225539+0000","last_fullsized":"2024-04-14T08:32:59.225539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:01.152061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278559+0000","last_change":"2024-04-14T08:32:53.187972+0000","last_active":"2024-04-14T08:33:01.278559+0000","last_peered":"2024-04-14T08:33:01.278559+0000","last_clean":"2024-04-14T08:33:01.278559+0000","last_became_active":"2024-04-14T08:32:53.187667+0000","last_became_peered":"2024-04-14T08:32:53.187667+0000","last_unstale":"2024-04-14T08:33:01.278559+0000","last_undegraded":"2024-04-14T08:33:01.278559+0000","last_fullsized":"2024-04-14T08:33:01.278559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:54:34.240052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220316+0000","last_change":"2024-04-14T08:32:58.246500+0000","last_active":"2024-04-14T08:33:01.220316+0000","last_peered":"2024-04-14T08:33:01.220316+0000","last_clean":"2024-04-14T08:33:01.220316+0000","last_became_active":"2024-04-14T08:32:58.245970+0000","last_became_peered":"2024-04-14T08:32:58.245970+0000","last_unstale":"2024-04-14T08:33:01.220316+0000","last_undegraded":"2024-04-14T08:33:01.220316+0000","last_fullsized":"2024-04-14T08:33:01.220316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:54.763380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115215+0000","last_change":"2024-04-14T08:32:54.201957+0000","last_active":"2024-04-14T08:33:01.115215+0000","last_peered":"2024-04-14T08:33:01.115215+0000","last_clean":"2024-04-14T08:33:01.115215+0000","last_became_active":"2024-04-14T08:32:54.201618+0000","last_became_peered":"2024-04-14T08:32:54.201618+0000","last_unstale":"2024-04-14T08:33:01.115215+0000","last_undegraded":"2024-04-14T08:33:01.115215+0000","last_fullsized":"2024-04-14T08:33:01.115215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115228+0000","last_change":"2024-04-14T08:32:53.195817+0000","last_active":"2024-04-14T08:33:01.115228+0000","last_peered":"2024-04-14T08:33:01.115228+0000","last_clean":"2024-04-14T08:33:01.115228+0000","last_became_active":"2024-04-14T08:32:53.195547+0000","last_became_peered":"2024-04-14T08:32:53.195547+0000","last_unstale":"2024-04-14T08:33:01.115228+0000","last_undegraded":"2024-04-14T08:33:01.115228+0000","last_fullsized":"2024-04-14T08:33:01.115228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225091+0000","last_change":"2024-04-14T08:32:58.240188+0000","last_active":"2024-04-14T08:32:59.225091+0000","last_peered":"2024-04-14T08:32:59.225091+0000","last_clean":"2024-04-14T08:32:59.225091+0000","last_became_active":"2024-04-14T08:32:58.239811+0000","last_became_peered":"2024-04-14T08:32:58.239811+0000","last_unstale":"2024-04-14T08:32:59.225091+0000","last_undegraded":"2024-04-14T08:32:59.225091+0000","last_fullsized":"2024-04-14T08:32:59.225091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:21.164569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225955+0000","last_change":"2024-04-14T08:32:54.200441+0000","last_active":"2024-04-14T08:32:59.225955+0000","last_peered":"2024-04-14T08:32:59.225955+0000","last_clean":"2024-04-14T08:32:59.225955+0000","last_became_active":"2024-04-14T08:32:54.199824+0000","last_became_peered":"2024-04-14T08:32:54.199824+0000","last_unstale":"2024-04-14T08:32:59.225955+0000","last_undegraded":"2024-04-14T08:32:59.225955+0000","last_fullsized":"2024-04-14T08:32:59.225955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404960+0000","last_change":"2024-04-14T08:32:53.202370+0000","last_active":"2024-04-14T08:32:59.404960+0000","last_peered":"2024-04-14T08:32:59.404960+0000","last_clean":"2024-04-14T08:32:59.404960+0000","last_became_active":"2024-04-14T08:32:53.202083+0000","last_became_peered":"2024-04-14T08:32:53.202083+0000","last_unstale":"2024-04-14T08:32:59.404960+0000","last_undegraded":"2024-04-14T08:32:59.404960+0000","last_fullsized":"2024-04-14T08:32:59.404960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224830+0000","last_change":"2024-04-14T08:32:58.247699+0000","last_active":"2024-04-14T08:32:59.224830+0000","last_peered":"2024-04-14T08:32:59.224830+0000","last_clean":"2024-04-14T08:32:59.224830+0000","last_became_active":"2024-04-14T08:32:58.247336+0000","last_became_peered":"2024-04-14T08:32:58.247336+0000","last_unstale":"2024-04-14T08:32:59.224830+0000","last_undegraded":"2024-04-14T08:32:59.224830+0000","last_fullsized":"2024-04-14T08:32:59.224830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:07:01.006337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220030+0000","last_change":"2024-04-14T08:32:54.198561+0000","last_active":"2024-04-14T08:33:01.220030+0000","last_peered":"2024-04-14T08:33:01.220030+0000","last_clean":"2024-04-14T08:33:01.220030+0000","last_became_active":"2024-04-14T08:32:54.198319+0000","last_became_peered":"2024-04-14T08:32:54.198319+0000","last_unstale":"2024-04-14T08:33:01.220030+0000","last_undegraded":"2024-04-14T08:33:01.220030+0000","last_fullsized":"2024-04-14T08:33:01.220030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219614+0000","last_change":"2024-04-14T08:32:53.188093+0000","last_active":"2024-04-14T08:33:01.219614+0000","last_peered":"2024-04-14T08:33:01.219614+0000","last_clean":"2024-04-14T08:33:01.219614+0000","last_became_active":"2024-04-14T08:32:53.187807+0000","last_became_peered":"2024-04-14T08:32:53.187807+0000","last_unstale":"2024-04-14T08:33:01.219614+0000","last_undegraded":"2024-04-14T08:33:01.219614+0000","last_fullsized":"2024-04-14T08:33:01.219614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:22:15.886932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405178+0000","last_change":"2024-04-14T08:32:58.234962+0000","last_active":"2024-04-14T08:32:59.405178+0000","last_peered":"2024-04-14T08:32:59.405178+0000","last_clean":"2024-04-14T08:32:59.405178+0000","last_became_active":"2024-04-14T08:32:58.234475+0000","last_became_peered":"2024-04-14T08:32:58.234475+0000","last_unstale":"2024-04-14T08:32:59.405178+0000","last_undegraded":"2024-04-14T08:32:59.405178+0000","last_fullsized":"2024-04-14T08:32:59.405178+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:07:01.006337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225877+0000","last_change":"2024-04-14T08:32:54.199377+0000","last_active":"2024-04-14T08:32:59.225877+0000","last_peered":"2024-04-14T08:32:59.225877+0000","last_clean":"2024-04-14T08:32:59.225877+0000","last_became_active":"2024-04-14T08:32:54.199105+0000","last_became_peered":"2024-04-14T08:32:54.199105+0000","last_unstale":"2024-04-14T08:32:59.225877+0000","last_undegraded":"2024-04-14T08:32:59.225877+0000","last_fullsized":"2024-04-14T08:32:59.225877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:43.600124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:03.602017+0000","last_change":"2024-04-14T08:32:53.199739+0000","last_active":"2024-04-14T08:33:03.602017+0000","last_peered":"2024-04-14T08:33:03.602017+0000","last_clean":"2024-04-14T08:33:03.602017+0000","last_became_active":"2024-04-14T08:32:53.199421+0000","last_became_peered":"2024-04-14T08:32:53.199421+0000","last_unstale":"2024-04-14T08:33:03.602017+0000","last_undegraded":"2024-04-14T08:33:03.602017+0000","last_fullsized":"2024-04-14T08:33:03.602017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610241+0000","last_change":"2024-04-14T08:32:58.243680+0000","last_active":"2024-04-14T08:33:00.610241+0000","last_peered":"2024-04-14T08:33:00.610241+0000","last_clean":"2024-04-14T08:33:00.610241+0000","last_became_active":"2024-04-14T08:32:58.243248+0000","last_became_peered":"2024-04-14T08:32:58.243248+0000","last_unstale":"2024-04-14T08:33:00.610241+0000","last_undegraded":"2024-04-14T08:33:00.610241+0000","last_fullsized":"2024-04-14T08:33:00.610241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:57.127403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225954+0000","last_change":"2024-04-14T08:32:54.199362+0000","last_active":"2024-04-14T08:32:59.225954+0000","last_peered":"2024-04-14T08:32:59.225954+0000","last_clean":"2024-04-14T08:32:59.225954+0000","last_became_active":"2024-04-14T08:32:54.199039+0000","last_became_peered":"2024-04-14T08:32:54.199039+0000","last_unstale":"2024-04-14T08:32:59.225954+0000","last_undegraded":"2024-04-14T08:32:59.225954+0000","last_fullsized":"2024-04-14T08:32:59.225954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:33.086425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610374+0000","last_change":"2024-04-14T08:32:53.198302+0000","last_active":"2024-04-14T08:33:00.610374+0000","last_peered":"2024-04-14T08:33:00.610374+0000","last_clean":"2024-04-14T08:33:00.610374+0000","last_became_active":"2024-04-14T08:32:53.198042+0000","last_became_peered":"2024-04-14T08:32:53.198042+0000","last_unstale":"2024-04-14T08:33:00.610374+0000","last_undegraded":"2024-04-14T08:33:00.610374+0000","last_fullsized":"2024-04-14T08:33:00.610374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:32.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225101+0000","last_change":"2024-04-14T08:32:58.249159+0000","last_active":"2024-04-14T08:32:59.225101+0000","last_peered":"2024-04-14T08:32:59.225101+0000","last_clean":"2024-04-14T08:32:59.225101+0000","last_became_active":"2024-04-14T08:32:58.248502+0000","last_became_peered":"2024-04-14T08:32:58.248502+0000","last_unstale":"2024-04-14T08:32:59.225101+0000","last_undegraded":"2024-04-14T08:32:59.225101+0000","last_fullsized":"2024-04-14T08:32:59.225101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:57.702107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220422+0000","last_change":"2024-04-14T08:32:54.211683+0000","last_active":"2024-04-14T08:33:01.220422+0000","last_peered":"2024-04-14T08:33:01.220422+0000","last_clean":"2024-04-14T08:33:01.220422+0000","last_became_active":"2024-04-14T08:32:54.211031+0000","last_became_peered":"2024-04-14T08:32:54.211031+0000","last_unstale":"2024-04-14T08:33:01.220422+0000","last_undegraded":"2024-04-14T08:33:01.220422+0000","last_fullsized":"2024-04-14T08:33:01.220422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405170+0000","last_change":"2024-04-14T08:32:53.198725+0000","last_active":"2024-04-14T08:32:59.405170+0000","last_peered":"2024-04-14T08:32:59.405170+0000","last_clean":"2024-04-14T08:32:59.405170+0000","last_became_active":"2024-04-14T08:32:53.198331+0000","last_became_peered":"2024-04-14T08:32:53.198331+0000","last_unstale":"2024-04-14T08:32:59.405170+0000","last_undegraded":"2024-04-14T08:32:59.405170+0000","last_fullsized":"2024-04-14T08:32:59.405170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224411+0000","last_change":"2024-04-14T08:32:58.241328+0000","last_active":"2024-04-14T08:32:59.224411+0000","last_peered":"2024-04-14T08:32:59.224411+0000","last_clean":"2024-04-14T08:32:59.224411+0000","last_became_active":"2024-04-14T08:32:58.240742+0000","last_became_peered":"2024-04-14T08:32:58.240742+0000","last_unstale":"2024-04-14T08:32:59.224411+0000","last_undegraded":"2024-04-14T08:32:59.224411+0000","last_fullsized":"2024-04-14T08:32:59.224411+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:10:58.801121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405105+0000","last_change":"2024-04-14T08:32:54.211134+0000","last_active":"2024-04-14T08:32:59.405105+0000","last_peered":"2024-04-14T08:32:59.405105+0000","last_clean":"2024-04-14T08:32:59.405105+0000","last_became_active":"2024-04-14T08:32:54.210882+0000","last_became_peered":"2024-04-14T08:32:54.210882+0000","last_unstale":"2024-04-14T08:32:59.405105+0000","last_undegraded":"2024-04-14T08:32:59.405105+0000","last_fullsized":"2024-04-14T08:32:59.405105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115518+0000","last_change":"2024-04-14T08:32:53.198595+0000","last_active":"2024-04-14T08:33:01.115518+0000","last_peered":"2024-04-14T08:33:01.115518+0000","last_clean":"2024-04-14T08:33:01.115518+0000","last_became_active":"2024-04-14T08:32:53.198284+0000","last_became_peered":"2024-04-14T08:32:53.198284+0000","last_unstale":"2024-04-14T08:33:01.115518+0000","last_undegraded":"2024-04-14T08:33:01.115518+0000","last_fullsized":"2024-04-14T08:33:01.115518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:48.807150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224443+0000","last_change":"2024-04-14T08:32:58.240139+0000","last_active":"2024-04-14T08:32:59.224443+0000","last_peered":"2024-04-14T08:32:59.224443+0000","last_clean":"2024-04-14T08:32:59.224443+0000","last_became_active":"2024-04-14T08:32:58.239751+0000","last_became_peered":"2024-04-14T08:32:58.239751+0000","last_unstale":"2024-04-14T08:32:59.224443+0000","last_undegraded":"2024-04-14T08:32:59.224443+0000","last_fullsized":"2024-04-14T08:32:59.224443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:37:10.427564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220280+0000","last_change":"2024-04-14T08:32:58.252205+0000","last_active":"2024-04-14T08:33:01.220280+0000","last_peered":"2024-04-14T08:33:01.220280+0000","last_clean":"2024-04-14T08:33:01.220280+0000","last_became_active":"2024-04-14T08:32:58.251856+0000","last_became_peered":"2024-04-14T08:32:58.251856+0000","last_unstale":"2024-04-14T08:33:01.220280+0000","last_undegraded":"2024-04-14T08:33:01.220280+0000","last_fullsized":"2024-04-14T08:33:01.220280+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:16.724895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.609964+0000","last_change":"2024-04-14T08:32:54.200507+0000","last_active":"2024-04-14T08:33:00.609964+0000","last_peered":"2024-04-14T08:33:00.609964+0000","last_clean":"2024-04-14T08:33:00.609964+0000","last_became_active":"2024-04-14T08:32:54.200256+0000","last_became_peered":"2024-04-14T08:32:54.200256+0000","last_unstale":"2024-04-14T08:33:00.609964+0000","last_undegraded":"2024-04-14T08:33:00.609964+0000","last_fullsized":"2024-04-14T08:33:00.609964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404763+0000","last_change":"2024-04-14T08:32:53.197120+0000","last_active":"2024-04-14T08:32:59.404763+0000","last_peered":"2024-04-14T08:32:59.404763+0000","last_clean":"2024-04-14T08:32:59.404763+0000","last_became_active":"2024-04-14T08:32:53.196835+0000","last_became_peered":"2024-04-14T08:32:53.196835+0000","last_unstale":"2024-04-14T08:32:59.404763+0000","last_undegraded":"2024-04-14T08:32:59.404763+0000","last_fullsized":"2024-04-14T08:32:59.404763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115520+0000","last_change":"2024-04-14T08:32:58.240951+0000","last_active":"2024-04-14T08:33:01.115520+0000","last_peered":"2024-04-14T08:33:01.115520+0000","last_clean":"2024-04-14T08:33:01.115520+0000","last_became_active":"2024-04-14T08:32:58.240542+0000","last_became_peered":"2024-04-14T08:32:58.240542+0000","last_unstale":"2024-04-14T08:33:01.115520+0000","last_undegraded":"2024-04-14T08:33:01.115520+0000","last_fullsized":"2024-04-14T08:33:01.115520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:57.127403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224403+0000","last_change":"2024-04-14T08:32:54.199647+0000","last_active":"2024-04-14T08:32:59.224403+0000","last_peered":"2024-04-14T08:32:59.224403+0000","last_clean":"2024-04-14T08:32:59.224403+0000","last_became_active":"2024-04-14T08:32:54.199365+0000","last_became_peered":"2024-04-14T08:32:54.199365+0000","last_unstale":"2024-04-14T08:32:59.224403+0000","last_undegraded":"2024-04-14T08:32:59.224403+0000","last_fullsized":"2024-04-14T08:32:59.224403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:17:06.203553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224536+0000","last_change":"2024-04-14T08:32:53.199750+0000","last_active":"2024-04-14T08:32:59.224536+0000","last_peered":"2024-04-14T08:32:59.224536+0000","last_clean":"2024-04-14T08:32:59.224536+0000","last_became_active":"2024-04-14T08:32:53.199477+0000","last_became_peered":"2024-04-14T08:32:53.199477+0000","last_unstale":"2024-04-14T08:32:59.224536+0000","last_undegraded":"2024-04-14T08:32:59.224536+0000","last_fullsized":"2024-04-14T08:32:59.224536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:03:30.482046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115345+0000","last_change":"2024-04-14T08:32:53.195193+0000","last_active":"2024-04-14T08:33:01.115345+0000","last_peered":"2024-04-14T08:33:01.115345+0000","last_clean":"2024-04-14T08:33:01.115345+0000","last_became_active":"2024-04-14T08:32:53.194887+0000","last_became_peered":"2024-04-14T08:32:53.194887+0000","last_unstale":"2024-04-14T08:33:01.115345+0000","last_undegraded":"2024-04-14T08:33:01.115345+0000","last_fullsized":"2024-04-14T08:33:01.115345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220235+0000","last_change":"2024-04-14T08:32:54.203263+0000","last_active":"2024-04-14T08:33:01.220235+0000","last_peered":"2024-04-14T08:33:01.220235+0000","last_clean":"2024-04-14T08:33:01.220235+0000","last_became_active":"2024-04-14T08:32:54.202196+0000","last_became_peered":"2024-04-14T08:32:54.202196+0000","last_unstale":"2024-04-14T08:33:01.220235+0000","last_undegraded":"2024-04-14T08:33:01.220235+0000","last_fullsized":"2024-04-14T08:33:01.220235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278810+0000","last_change":"2024-04-14T08:32:58.239935+0000","last_active":"2024-04-14T08:33:01.278810+0000","last_peered":"2024-04-14T08:33:01.278810+0000","last_clean":"2024-04-14T08:33:01.278810+0000","last_became_active":"2024-04-14T08:32:58.239542+0000","last_became_peered":"2024-04-14T08:32:58.239542+0000","last_unstale":"2024-04-14T08:33:01.278810+0000","last_undegraded":"2024-04-14T08:33:01.278810+0000","last_fullsized":"2024-04-14T08:33:01.278810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:04.288282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224729+0000","last_change":"2024-04-14T08:32:53.201101+0000","last_active":"2024-04-14T08:32:59.224729+0000","last_peered":"2024-04-14T08:32:59.224729+0000","last_clean":"2024-04-14T08:32:59.224729+0000","last_became_active":"2024-04-14T08:32:53.200781+0000","last_became_peered":"2024-04-14T08:32:53.200781+0000","last_unstale":"2024-04-14T08:32:59.224729+0000","last_undegraded":"2024-04-14T08:32:59.224729+0000","last_fullsized":"2024-04-14T08:32:59.224729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:08:09.715762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224989+0000","last_change":"2024-04-14T08:32:54.194042+0000","last_active":"2024-04-14T08:32:59.224989+0000","last_peered":"2024-04-14T08:32:59.224989+0000","last_clean":"2024-04-14T08:32:59.224989+0000","last_became_active":"2024-04-14T08:32:54.193764+0000","last_became_peered":"2024-04-14T08:32:54.193764+0000","last_unstale":"2024-04-14T08:32:59.224989+0000","last_undegraded":"2024-04-14T08:32:59.224989+0000","last_fullsized":"2024-04-14T08:32:59.224989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:50:00.196929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225098+0000","last_change":"2024-04-14T08:32:58.241996+0000","last_active":"2024-04-14T08:32:59.225098+0000","last_peered":"2024-04-14T08:32:59.225098+0000","last_clean":"2024-04-14T08:32:59.225098+0000","last_became_active":"2024-04-14T08:32:58.241641+0000","last_became_peered":"2024-04-14T08:32:58.241641+0000","last_unstale":"2024-04-14T08:32:59.225098+0000","last_undegraded":"2024-04-14T08:32:59.225098+0000","last_fullsized":"2024-04-14T08:32:59.225098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:58:22.710120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115102+0000","last_change":"2024-04-14T08:32:54.210270+0000","last_active":"2024-04-14T08:33:01.115102+0000","last_peered":"2024-04-14T08:33:01.115102+0000","last_clean":"2024-04-14T08:33:01.115102+0000","last_became_active":"2024-04-14T08:32:54.209717+0000","last_became_peered":"2024-04-14T08:32:54.209717+0000","last_unstale":"2024-04-14T08:33:01.115102+0000","last_undegraded":"2024-04-14T08:33:01.115102+0000","last_fullsized":"2024-04-14T08:33:01.115102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278775+0000","last_change":"2024-04-14T08:32:54.208720+0000","last_active":"2024-04-14T08:33:01.278775+0000","last_peered":"2024-04-14T08:33:01.278775+0000","last_clean":"2024-04-14T08:33:01.278775+0000","last_became_active":"2024-04-14T08:32:54.208426+0000","last_became_peered":"2024-04-14T08:32:54.208426+0000","last_unstale":"2024-04-14T08:33:01.278775+0000","last_undegraded":"2024-04-14T08:33:01.278775+0000","last_fullsized":"2024-04-14T08:33:01.278775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:49.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404885+0000","last_change":"2024-04-14T08:32:54.209426+0000","last_active":"2024-04-14T08:32:59.404885+0000","last_peered":"2024-04-14T08:32:59.404885+0000","last_clean":"2024-04-14T08:32:59.404885+0000","last_became_active":"2024-04-14T08:32:54.209158+0000","last_became_peered":"2024-04-14T08:32:54.209158+0000","last_unstale":"2024-04-14T08:32:59.404885+0000","last_undegraded":"2024-04-14T08:32:59.404885+0000","last_fullsized":"2024-04-14T08:32:59.404885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:12.683816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220151+0000","last_change":"2024-04-14T08:32:54.206827+0000","last_active":"2024-04-14T08:33:01.220151+0000","last_peered":"2024-04-14T08:33:01.220151+0000","last_clean":"2024-04-14T08:33:01.220151+0000","last_became_active":"2024-04-14T08:32:54.206593+0000","last_became_peered":"2024-04-14T08:32:54.206593+0000","last_unstale":"2024-04-14T08:33:01.220151+0000","last_undegraded":"2024-04-14T08:33:01.220151+0000","last_fullsized":"2024-04-14T08:33:01.220151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224854+0000","last_change":"2024-04-14T08:32:54.200872+0000","last_active":"2024-04-14T08:32:59.224854+0000","last_peered":"2024-04-14T08:32:59.224854+0000","last_clean":"2024-04-14T08:32:59.224854+0000","last_became_active":"2024-04-14T08:32:54.200616+0000","last_became_peered":"2024-04-14T08:32:54.200616+0000","last_unstale":"2024-04-14T08:32:59.224854+0000","last_undegraded":"2024-04-14T08:32:59.224854+0000","last_fullsized":"2024-04-14T08:32:59.224854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610219+0000","last_change":"2024-04-14T08:32:54.199679+0000","last_active":"2024-04-14T08:33:00.610219+0000","last_peered":"2024-04-14T08:33:00.610219+0000","last_clean":"2024-04-14T08:33:00.610219+0000","last_became_active":"2024-04-14T08:32:54.199433+0000","last_became_peered":"2024-04-14T08:32:54.199433+0000","last_unstale":"2024-04-14T08:33:00.610219+0000","last_undegraded":"2024-04-14T08:33:00.610219+0000","last_fullsized":"2024-04-14T08:33:00.610219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115645+0000","last_change":"2024-04-14T08:32:54.207939+0000","last_active":"2024-04-14T08:33:01.115645+0000","last_peered":"2024-04-14T08:33:01.115645+0000","last_clean":"2024-04-14T08:33:01.115645+0000","last_became_active":"2024-04-14T08:32:54.207460+0000","last_became_peered":"2024-04-14T08:32:54.207460+0000","last_unstale":"2024-04-14T08:33:01.115645+0000","last_undegraded":"2024-04-14T08:33:01.115645+0000","last_fullsized":"2024-04-14T08:33:01.115645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225033+0000","last_change":"2024-04-14T08:32:54.202461+0000","last_active":"2024-04-14T08:32:59.225033+0000","last_peered":"2024-04-14T08:32:59.225033+0000","last_clean":"2024-04-14T08:32:59.225033+0000","last_became_active":"2024-04-14T08:32:54.202156+0000","last_became_peered":"2024-04-14T08:32:54.202156+0000","last_unstale":"2024-04-14T08:32:59.225033+0000","last_undegraded":"2024-04-14T08:32:59.225033+0000","last_fullsized":"2024-04-14T08:32:59.225033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404730+0000","last_change":"2024-04-14T08:32:54.204603+0000","last_active":"2024-04-14T08:32:59.404730+0000","last_peered":"2024-04-14T08:32:59.404730+0000","last_clean":"2024-04-14T08:32:59.404730+0000","last_became_active":"2024-04-14T08:32:54.204323+0000","last_became_peered":"2024-04-14T08:32:54.204323+0000","last_unstale":"2024-04-14T08:32:59.404730+0000","last_undegraded":"2024-04-14T08:32:59.404730+0000","last_fullsized":"2024-04-14T08:32:59.404730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224464+0000","last_change":"2024-04-14T08:32:54.205983+0000","last_active":"2024-04-14T08:32:59.224464+0000","last_peered":"2024-04-14T08:32:59.224464+0000","last_clean":"2024-04-14T08:32:59.224464+0000","last_became_active":"2024-04-14T08:32:54.205707+0000","last_became_peered":"2024-04-14T08:32:54.205707+0000","last_unstale":"2024-04-14T08:32:59.224464+0000","last_undegraded":"2024-04-14T08:32:59.224464+0000","last_fullsized":"2024-04-14T08:32:59.224464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:05:03.511225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404857+0000","last_change":"2024-04-14T08:32:54.211979+0000","last_active":"2024-04-14T08:32:59.404857+0000","last_peered":"2024-04-14T08:32:59.404857+0000","last_clean":"2024-04-14T08:32:59.404857+0000","last_became_active":"2024-04-14T08:32:54.211481+0000","last_became_peered":"2024-04-14T08:32:54.211481+0000","last_unstale":"2024-04-14T08:32:59.404857+0000","last_undegraded":"2024-04-14T08:32:59.404857+0000","last_fullsized":"2024-04-14T08:32:59.404857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:53.086323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220046+0000","last_change":"2024-04-14T08:32:54.204727+0000","last_active":"2024-04-14T08:33:01.220046+0000","last_peered":"2024-04-14T08:33:01.220046+0000","last_clean":"2024-04-14T08:33:01.220046+0000","last_became_active":"2024-04-14T08:32:54.204472+0000","last_became_peered":"2024-04-14T08:32:54.204472+0000","last_unstale":"2024-04-14T08:33:01.220046+0000","last_undegraded":"2024-04-14T08:33:01.220046+0000","last_fullsized":"2024-04-14T08:33:01.220046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115389+0000","last_change":"2024-04-14T08:32:54.211345+0000","last_active":"2024-04-14T08:33:01.115389+0000","last_peered":"2024-04-14T08:33:01.115389+0000","last_clean":"2024-04-14T08:33:01.115389+0000","last_became_active":"2024-04-14T08:32:54.211084+0000","last_became_peered":"2024-04-14T08:32:54.211084+0000","last_unstale":"2024-04-14T08:33:01.115389+0000","last_undegraded":"2024-04-14T08:33:01.115389+0000","last_fullsized":"2024-04-14T08:33:01.115389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226252+0000","last_change":"2024-04-14T08:32:54.198283+0000","last_active":"2024-04-14T08:32:59.226252+0000","last_peered":"2024-04-14T08:32:59.226252+0000","last_clean":"2024-04-14T08:32:59.226252+0000","last_became_active":"2024-04-14T08:32:54.197647+0000","last_became_peered":"2024-04-14T08:32:54.197647+0000","last_unstale":"2024-04-14T08:32:59.226252+0000","last_undegraded":"2024-04-14T08:32:59.226252+0000","last_fullsized":"2024-04-14T08:32:59.226252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:48.960689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278917+0000","last_change":"2024-04-14T08:32:54.204246+0000","last_active":"2024-04-14T08:33:01.278917+0000","last_peered":"2024-04-14T08:33:01.278917+0000","last_clean":"2024-04-14T08:33:01.278917+0000","last_became_active":"2024-04-14T08:32:54.203956+0000","last_became_peered":"2024-04-14T08:32:54.203956+0000","last_unstale":"2024-04-14T08:33:01.278917+0000","last_undegraded":"2024-04-14T08:33:01.278917+0000","last_fullsized":"2024-04-14T08:33:01.278917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:33.086425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405290+0000","last_change":"2024-04-14T08:32:54.207841+0000","last_active":"2024-04-14T08:32:59.405290+0000","last_peered":"2024-04-14T08:32:59.405290+0000","last_clean":"2024-04-14T08:32:59.405290+0000","last_became_active":"2024-04-14T08:32:54.207412+0000","last_became_peered":"2024-04-14T08:32:54.207412+0000","last_unstale":"2024-04-14T08:32:59.405290+0000","last_undegraded":"2024-04-14T08:32:59.405290+0000","last_fullsized":"2024-04-14T08:32:59.405290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:48.043054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404630+0000","last_change":"2024-04-14T08:32:54.207321+0000","last_active":"2024-04-14T08:32:59.404630+0000","last_peered":"2024-04-14T08:32:59.404630+0000","last_clean":"2024-04-14T08:32:59.404630+0000","last_became_active":"2024-04-14T08:32:54.207058+0000","last_became_peered":"2024-04-14T08:32:54.207058+0000","last_unstale":"2024-04-14T08:32:59.404630+0000","last_undegraded":"2024-04-14T08:32:59.404630+0000","last_fullsized":"2024-04-14T08:32:59.404630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610020+0000","last_change":"2024-04-14T08:32:54.205919+0000","last_active":"2024-04-14T08:33:00.610020+0000","last_peered":"2024-04-14T08:33:00.610020+0000","last_clean":"2024-04-14T08:33:00.610020+0000","last_became_active":"2024-04-14T08:32:54.205623+0000","last_became_peered":"2024-04-14T08:32:54.205623+0000","last_unstale":"2024-04-14T08:33:00.610020+0000","last_undegraded":"2024-04-14T08:33:00.610020+0000","last_fullsized":"2024-04-14T08:33:00.610020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220007+0000","last_change":"2024-04-14T08:32:54.196317+0000","last_active":"2024-04-14T08:33:01.220007+0000","last_peered":"2024-04-14T08:33:01.220007+0000","last_clean":"2024-04-14T08:33:01.220007+0000","last_became_active":"2024-04-14T08:32:54.195481+0000","last_became_peered":"2024-04-14T08:32:54.195481+0000","last_unstale":"2024-04-14T08:33:01.220007+0000","last_undegraded":"2024-04-14T08:33:01.220007+0000","last_fullsized":"2024-04-14T08:33:01.220007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:17:06.203553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224636+0000","last_change":"2024-04-14T08:32:54.203801+0000","last_active":"2024-04-14T08:32:59.224636+0000","last_peered":"2024-04-14T08:32:59.224636+0000","last_clean":"2024-04-14T08:32:59.224636+0000","last_became_active":"2024-04-14T08:32:54.203524+0000","last_became_peered":"2024-04-14T08:32:54.203524+0000","last_unstale":"2024-04-14T08:32:59.224636+0000","last_undegraded":"2024-04-14T08:32:59.224636+0000","last_fullsized":"2024-04-14T08:32:59.224636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225873+0000","last_change":"2024-04-14T08:32:54.204997+0000","last_active":"2024-04-14T08:32:59.225873+0000","last_peered":"2024-04-14T08:32:59.225873+0000","last_clean":"2024-04-14T08:32:59.225873+0000","last_became_active":"2024-04-14T08:32:54.204757+0000","last_became_peered":"2024-04-14T08:32:54.204757+0000","last_unstale":"2024-04-14T08:32:59.225873+0000","last_undegraded":"2024-04-14T08:32:59.225873+0000","last_fullsized":"2024-04-14T08:32:59.225873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:49.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115444+0000","last_change":"2024-04-14T08:32:54.210277+0000","last_active":"2024-04-14T08:33:01.115444+0000","last_peered":"2024-04-14T08:33:01.115444+0000","last_clean":"2024-04-14T08:33:01.115444+0000","last_became_active":"2024-04-14T08:32:54.209548+0000","last_became_peered":"2024-04-14T08:32:54.209548+0000","last_unstale":"2024-04-14T08:33:01.115444+0000","last_undegraded":"2024-04-14T08:33:01.115444+0000","last_fullsized":"2024-04-14T08:33:01.115444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115625+0000","last_change":"2024-04-14T08:32:54.213828+0000","last_active":"2024-04-14T08:33:01.115625+0000","last_peered":"2024-04-14T08:33:01.115625+0000","last_clean":"2024-04-14T08:33:01.115625+0000","last_became_active":"2024-04-14T08:32:54.213551+0000","last_became_peered":"2024-04-14T08:32:54.213551+0000","last_unstale":"2024-04-14T08:33:01.115625+0000","last_undegraded":"2024-04-14T08:33:01.115625+0000","last_fullsized":"2024-04-14T08:33:01.115625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226176+0000","last_change":"2024-04-14T08:32:54.212290+0000","last_active":"2024-04-14T08:32:59.226176+0000","last_peered":"2024-04-14T08:32:59.226176+0000","last_clean":"2024-04-14T08:32:59.226176+0000","last_became_active":"2024-04-14T08:32:54.212014+0000","last_became_peered":"2024-04-14T08:32:54.212014+0000","last_unstale":"2024-04-14T08:32:59.226176+0000","last_undegraded":"2024-04-14T08:32:59.226176+0000","last_fullsized":"2024-04-14T08:32:59.226176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:50.722301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224300+0000","last_change":"2024-04-14T08:32:54.209523+0000","last_active":"2024-04-14T08:32:59.224300+0000","last_peered":"2024-04-14T08:32:59.224300+0000","last_clean":"2024-04-14T08:32:59.224300+0000","last_became_active":"2024-04-14T08:32:54.209262+0000","last_became_peered":"2024-04-14T08:32:54.209262+0000","last_unstale":"2024-04-14T08:32:59.224300+0000","last_undegraded":"2024-04-14T08:32:59.224300+0000","last_fullsized":"2024-04-14T08:32:59.224300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:25:28.934928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225401+0000","last_change":"2024-04-14T08:32:54.195208+0000","last_active":"2024-04-14T08:32:59.225401+0000","last_peered":"2024-04-14T08:32:59.225401+0000","last_clean":"2024-04-14T08:32:59.225401+0000","last_became_active":"2024-04-14T08:32:54.194916+0000","last_became_peered":"2024-04-14T08:32:54.194916+0000","last_unstale":"2024-04-14T08:32:59.225401+0000","last_undegraded":"2024-04-14T08:32:59.225401+0000","last_fullsized":"2024-04-14T08:32:59.225401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:16.536477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404780+0000","last_change":"2024-04-14T08:32:54.208737+0000","last_active":"2024-04-14T08:32:59.404780+0000","last_peered":"2024-04-14T08:32:59.404780+0000","last_clean":"2024-04-14T08:32:59.404780+0000","last_became_active":"2024-04-14T08:32:54.208439+0000","last_became_peered":"2024-04-14T08:32:54.208439+0000","last_unstale":"2024-04-14T08:32:59.404780+0000","last_undegraded":"2024-04-14T08:32:59.404780+0000","last_fullsized":"2024-04-14T08:32:59.404780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:48.960689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225788+0000","last_change":"2024-04-14T08:32:54.198295+0000","last_active":"2024-04-14T08:32:59.225788+0000","last_peered":"2024-04-14T08:32:59.225788+0000","last_clean":"2024-04-14T08:32:59.225788+0000","last_became_active":"2024-04-14T08:32:54.198052+0000","last_became_peered":"2024-04-14T08:32:54.198052+0000","last_unstale":"2024-04-14T08:32:59.225788+0000","last_undegraded":"2024-04-14T08:32:59.225788+0000","last_fullsized":"2024-04-14T08:32:59.225788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:48.043054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404955+0000","last_change":"2024-04-14T08:32:54.206537+0000","last_active":"2024-04-14T08:32:59.404955+0000","last_peered":"2024-04-14T08:32:59.404955+0000","last_clean":"2024-04-14T08:32:59.404955+0000","last_became_active":"2024-04-14T08:32:54.206246+0000","last_became_peered":"2024-04-14T08:32:54.206246+0000","last_unstale":"2024-04-14T08:32:59.404955+0000","last_undegraded":"2024-04-14T08:32:59.404955+0000","last_fullsized":"2024-04-14T08:32:59.404955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:16.536477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220192+0000","last_change":"2024-04-14T08:32:54.201364+0000","last_active":"2024-04-14T08:33:01.220192+0000","last_peered":"2024-04-14T08:33:01.220192+0000","last_clean":"2024-04-14T08:33:01.220192+0000","last_became_active":"2024-04-14T08:32:54.199521+0000","last_became_peered":"2024-04-14T08:32:54.199521+0000","last_unstale":"2024-04-14T08:33:01.220192+0000","last_undegraded":"2024-04-14T08:33:01.220192+0000","last_fullsized":"2024-04-14T08:33:01.220192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610554+0000","last_change":"2024-04-14T08:32:54.199458+0000","last_active":"2024-04-14T08:33:00.610554+0000","last_peered":"2024-04-14T08:33:00.610554+0000","last_clean":"2024-04-14T08:33:00.610554+0000","last_became_active":"2024-04-14T08:32:54.199203+0000","last_became_peered":"2024-04-14T08:32:54.199203+0000","last_unstale":"2024-04-14T08:33:00.610554+0000","last_undegraded":"2024-04-14T08:33:00.610554+0000","last_fullsized":"2024-04-14T08:33:00.610554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226040+0000","last_change":"2024-04-14T08:32:54.206343+0000","last_active":"2024-04-14T08:32:59.226040+0000","last_peered":"2024-04-14T08:32:59.226040+0000","last_clean":"2024-04-14T08:32:59.226040+0000","last_became_active":"2024-04-14T08:32:54.205925+0000","last_became_peered":"2024-04-14T08:32:54.205925+0000","last_unstale":"2024-04-14T08:32:59.226040+0000","last_undegraded":"2024-04-14T08:32:59.226040+0000","last_fullsized":"2024-04-14T08:32:59.226040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:12.683816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:33:11.593 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-14T08:33:11.594 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-14T08:33:11.594 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-14T08:33:11.594 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-14T08:33:11.594 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-14T08:33:11.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.728+0000 7f8d7b75e640 1 Processor -- start 2024-04-14T08:33:11.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.729+0000 7f8d7b75e640 1 -- start start 2024-04-14T08:33:11.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.729+0000 7f8d7b75e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8d740d2830 0x7f8d740d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.729+0000 7f8d7b75e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7415e890 0x7f8d740d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.729+0000 7f8d7b75e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d740d3150 con 0x7f8d740cedd0 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.729+0000 7f8d7b75e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f8d740cff30 con 0x7f8d740d2830 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.729+0000 7f8d7b75e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8d740d2350 con 0x7f8d7415e890 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.729+0000 7f8d794fa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8d740d2830 0x7f8d740d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.730+0000 7f8d79cfb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f8d740cedd0 0x7f8d741621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35606/0 (socket says 172.21.15.172:35606) 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.730+0000 7f8d79cfb640 1 -- 172.21.15.172:0/3942665711 learned_addr learned my addr 172.21.15.172:0/3942665711 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.730+0000 7f8d794fa640 1 --2- 172.21.15.172:0/3942665711 >> v2:172.21.15.172:3300/0 conn(0x7f8d740d2830 0x7f8d740d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49338/0 (socket says 172.21.15.172:49338) 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.730+0000 7f8d78cf9640 1 --2- 172.21.15.172:0/3942665711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7415e890 0x7f8d740d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.730+0000 7f8d794fa640 1 -- 172.21.15.172:0/3942665711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7415e890 msgr2=0x7f8d740d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.730+0000 7f8d794fa640 1 --2- 172.21.15.172:0/3942665711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7415e890 0x7f8d740d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.730+0000 7f8d794fa640 1 -- 172.21.15.172:0/3942665711 >> v1:172.21.15.172:6789/0 conn(0x7f8d740cedd0 legacy=0x7f8d741621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.730+0000 7f8d794fa640 1 -- 172.21.15.172:0/3942665711 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d740cf320 con 0x7f8d740d2830 2024-04-14T08:33:11.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.730+0000 7f8d6bfff640 1 -- 172.21.15.172:0/3942665711 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2774735609 0 0) 0x7f8d740d3150 con 0x7f8d740cedd0 2024-04-14T08:33:11.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.731+0000 7f8d794fa640 1 --2- 172.21.15.172:0/3942665711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d740d2830 0x7f8d740d2c10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8d700097a0 tx=0x7f8d7002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54284b50f265626 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:11.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.731+0000 7f8d6bfff640 1 -- 172.21.15.172:0/3942665711 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70034070 con 0x7f8d740d2830 2024-04-14T08:33:11.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.731+0000 7f8d6bfff640 1 -- 172.21.15.172:0/3942665711 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d70030610 con 0x7f8d740d2830 2024-04-14T08:33:11.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.731+0000 7f8d6bfff640 1 -- 172.21.15.172:0/3942665711 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d700309e0 con 0x7f8d740d2830 2024-04-14T08:33:11.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.731+0000 7f8d7b75e640 1 -- 172.21.15.172:0/3942665711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d740d2830 msgr2=0x7f8d740d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.731+0000 7f8d7b75e640 1 --2- 172.21.15.172:0/3942665711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d740d2830 0x7f8d740d2c10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8d700097a0 tx=0x7f8d7002b960 comp rx=0 tx=0).stop 2024-04-14T08:33:11.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.731+0000 7f8d7b75e640 1 -- 172.21.15.172:0/3942665711 shutdown_connections 2024-04-14T08:33:11.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.731+0000 7f8d7b75e640 1 --2- 172.21.15.172:0/3942665711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7415e890 0x7f8d740d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.731+0000 7f8d7b75e640 1 --2- 172.21.15.172:0/3942665711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d740d2830 0x7f8d740d2c10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.731+0000 7f8d7b75e640 1 -- 172.21.15.172:0/3942665711 >> 172.21.15.172:0/3942665711 conn(0x7f8d7409d590 msgr2=0x7f8d740d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:11.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.731+0000 7f8d7b75e640 1 -- 172.21.15.172:0/3942665711 shutdown_connections 2024-04-14T08:33:11.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.731+0000 7f8d7b75e640 1 -- 172.21.15.172:0/3942665711 wait complete. 2024-04-14T08:33:11.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.732+0000 7f8d7b75e640 1 Processor -- start 2024-04-14T08:33:11.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.733+0000 7f8d7b75e640 1 -- start start 2024-04-14T08:33:11.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.733+0000 7f8d7b75e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d740cedd0 0x7f8d740b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.733+0000 7f8d7b75e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d740d2830 0x7f8d740cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.733+0000 7f8d7b75e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7415e890 0x7f8d740c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.733+0000 7f8d7b75e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d740cfbc0 con 0x7f8d740d2830 2024-04-14T08:33:11.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.733+0000 7f8d7b75e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8d740cf8e0 con 0x7f8d740cedd0 2024-04-14T08:33:11.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.733+0000 7f8d7b75e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8d740d1840 con 0x7f8d7415e890 2024-04-14T08:33:11.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.733+0000 7f8d78cf9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d740d2830 0x7f8d740cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.733+0000 7f8d78cf9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d740d2830 0x7f8d740cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49348/0 (socket says 172.21.15.172:49348) 2024-04-14T08:33:11.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.733+0000 7f8d78cf9640 1 -- 172.21.15.172:0/1766567690 learned_addr learned my addr 172.21.15.172:0/1766567690 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:11.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.733+0000 7f8d79cfb640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7415e890 0x7f8d740c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.733+0000 7f8d794fa640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d740cedd0 0x7f8d740b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.734+0000 7f8d78cf9640 1 -- 172.21.15.172:0/1766567690 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7415e890 msgr2=0x7f8d740c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.734+0000 7f8d78cf9640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7415e890 0x7f8d740c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.734+0000 7f8d78cf9640 1 -- 172.21.15.172:0/1766567690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d740cedd0 msgr2=0x7f8d740b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.734+0000 7f8d78cf9640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d740cedd0 0x7f8d740b6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.734+0000 7f8d78cf9640 1 -- 172.21.15.172:0/1766567690 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d740ce260 con 0x7f8d740d2830 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.734+0000 7f8d78cf9640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d740d2830 0x7f8d740cdd20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400e980 tx=0x7f8d6400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.734+0000 7f8d69ffb640 1 -- 172.21.15.172:0/1766567690 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64017070 con 0x7f8d740d2830 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.734+0000 7f8d69ffb640 1 -- 172.21.15.172:0/1766567690 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6400c9a0 con 0x7f8d740d2830 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.734+0000 7f8d69ffb640 1 -- 172.21.15.172:0/1766567690 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6400cd70 con 0x7f8d740d2830 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.734+0000 7f8d7b75e640 1 -- 172.21.15.172:0/1766567690 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d740b9960 con 0x7f8d740d2830 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.734+0000 7f8d7b75e640 1 -- 172.21.15.172:0/1766567690 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d740c8dc0 con 0x7f8d740d2830 2024-04-14T08:33:11.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.735+0000 7f8d7b75e640 1 -- 172.21.15.172:0/1766567690 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d740ce600 con 0x7f8d740d2830 2024-04-14T08:33:11.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.736+0000 7f8d69ffb640 1 -- 172.21.15.172:0/1766567690 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d6401a070 con 0x7f8d740d2830 2024-04-14T08:33:11.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.736+0000 7f8d69ffb640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8d5806c8f0 0x7f8d5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:11.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.736+0000 7f8d69ffb640 1 -- 172.21.15.172:0/1766567690 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8d6408bdd0 con 0x7f8d740d2830 2024-04-14T08:33:11.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.736+0000 7f8d794fa640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8d5806c8f0 0x7f8d5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:11.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.737+0000 7f8d794fa640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8d5806c8f0 0x7f8d5806edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d70036000 tx=0x7f8d7003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:11.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.741+0000 7f8d69ffb640 1 -- 172.21.15.172:0/1766567690 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8d740ce600 con 0x7f8d740d2830 2024-04-14T08:33:11.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.963+0000 7f8d7b75e640 1 -- 172.21.15.172:0/1766567690 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8d740d2610 con 0x7f8d740d2830 2024-04-14T08:33:11.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.966+0000 7f8d69ffb640 1 -- 172.21.15.172:0/1766567690 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174144 (secure 0 0 0) 0x7f8d64059870 con 0x7f8d740d2830 2024-04-14T08:33:11.969 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:11.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.969+0000 7f8d7b75e640 1 -- 172.21.15.172:0/1766567690 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8d5806c8f0 msgr2=0x7f8d5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.969+0000 7f8d7b75e640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8d5806c8f0 0x7f8d5806edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d70036000 tx=0x7f8d7003b040 comp rx=0 tx=0).stop 2024-04-14T08:33:11.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.969+0000 7f8d7b75e640 1 -- 172.21.15.172:0/1766567690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d740d2830 msgr2=0x7f8d740cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:11.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.969+0000 7f8d7b75e640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d740d2830 0x7f8d740cdd20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400e980 tx=0x7f8d6400ee40 comp rx=0 tx=0).stop 2024-04-14T08:33:11.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.969+0000 7f8d7b75e640 1 -- 172.21.15.172:0/1766567690 shutdown_connections 2024-04-14T08:33:11.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.969+0000 7f8d7b75e640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8d5806c8f0 0x7f8d5806edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.969+0000 7f8d7b75e640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d7415e890 0x7f8d740c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.969+0000 7f8d7b75e640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d740d2830 0x7f8d740cdd20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.969+0000 7f8d7b75e640 1 --2- 172.21.15.172:0/1766567690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d740cedd0 0x7f8d740b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:11.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.969+0000 7f8d7b75e640 1 -- 172.21.15.172:0/1766567690 >> 172.21.15.172:0/1766567690 conn(0x7f8d7409d590 msgr2=0x7f8d740b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:11.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.969+0000 7f8d7b75e640 1 -- 172.21.15.172:0/1766567690 shutdown_connections 2024-04-14T08:33:11.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:11.969+0000 7f8d7b75e640 1 -- 172.21.15.172:0/1766567690 wait complete. 2024-04-14T08:33:11.980 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1188012648},{"type":"v1","addr":"172.21.15.172:6833","nonce":1188012648}]},"active_addr":"172.21.15.172:6833/1188012648","active_change":"2024-04-14T08:32:33.110852+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":2196586107}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":3931185299}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":3483899405}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":2972939200}]}]} 2024-04-14T08:33:11.983 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-14T08:33:11.983 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-14T08:33:11.984 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:12.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.085+0000 7f04ef0de640 1 Processor -- start 2024-04-14T08:33:12.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.085+0000 7f04ef0de640 1 -- start start 2024-04-14T08:33:12.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.085+0000 7f04ef0de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 0x7f04e80ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.085+0000 7f04ef0de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f04e806c530 0x7f04e806c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.085+0000 7f04ef0de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e80b41d0 con 0x7f04e80729b0 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.086+0000 7f04ef0de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f04e80aebe0 con 0x7f04e806c530 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.086+0000 7f04ef0de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f04e80b4470 con 0x7f04e815e5b0 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.086+0000 7f04ece7a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 0x7f04e80ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.086+0000 7f04ece7a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 0x7f04e80ae390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:42988/0 (socket says 172.21.15.172:42988) 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.086+0000 7f04ece7a640 1 -- 172.21.15.172:0/3876899751 learned_addr learned my addr 172.21.15.172:0/3876899751 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.086+0000 7f04ed67b640 1 --2- 172.21.15.172:0/3876899751 >> v2:172.21.15.177:3300/0 conn(0x7f04e806c530 0x7f04e806c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.086+0000 7f04ece7a640 1 -- 172.21.15.172:0/3876899751 >> v2:172.21.15.177:3300/0 conn(0x7f04e806c530 msgr2=0x7f04e806c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.086+0000 7f04ece7a640 1 --2- 172.21.15.172:0/3876899751 >> v2:172.21.15.177:3300/0 conn(0x7f04e806c530 0x7f04e806c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.086+0000 7f04ece7a640 1 -- 172.21.15.172:0/3876899751 >> v1:172.21.15.177:6789/0 conn(0x7f04e80729b0 legacy=0x7f04e806dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.086+0000 7f04ece7a640 1 -- 172.21.15.172:0/3876899751 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e80ae8d0 con 0x7f04e815e5b0 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.086+0000 7f04ed67b640 1 --2- 172.21.15.172:0/3876899751 >> v2:172.21.15.177:3300/0 conn(0x7f04e806c530 0x7f04e806c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.087+0000 7f04ece7a640 1 --2- 172.21.15.172:0/3876899751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 0x7f04e80ae390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009e50 tx=0x7f04e002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34e005ccebba4dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.087+0000 7f04e77fe640 1 -- 172.21.15.172:0/3876899751 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e002cb00 con 0x7f04e815e5b0 2024-04-14T08:33:12.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.087+0000 7f04e77fe640 1 -- 172.21.15.172:0/3876899751 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e002cca0 con 0x7f04e815e5b0 2024-04-14T08:33:12.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.087+0000 7f04e77fe640 1 -- 172.21.15.172:0/3876899751 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0034610 con 0x7f04e815e5b0 2024-04-14T08:33:12.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.087+0000 7f04ef0de640 1 -- 172.21.15.172:0/3876899751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 msgr2=0x7f04e80ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.087+0000 7f04ef0de640 1 --2- 172.21.15.172:0/3876899751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 0x7f04e80ae390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009e50 tx=0x7f04e002bf60 comp rx=0 tx=0).stop 2024-04-14T08:33:12.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.087+0000 7f04ef0de640 1 -- 172.21.15.172:0/3876899751 shutdown_connections 2024-04-14T08:33:12.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.087+0000 7f04ef0de640 1 --2- 172.21.15.172:0/3876899751 >> v2:172.21.15.177:3300/0 conn(0x7f04e806c530 0x7f04e806c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.087+0000 7f04ef0de640 1 --2- 172.21.15.172:0/3876899751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 0x7f04e80ae390 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.087+0000 7f04ef0de640 1 -- 172.21.15.172:0/3876899751 >> 172.21.15.172:0/3876899751 conn(0x7f04e8071ba0 msgr2=0x7f04e8071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.087+0000 7f04ef0de640 1 -- 172.21.15.172:0/3876899751 shutdown_connections 2024-04-14T08:33:12.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.087+0000 7f04ef0de640 1 -- 172.21.15.172:0/3876899751 wait complete. 2024-04-14T08:33:12.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.089+0000 7f04ef0de640 1 Processor -- start 2024-04-14T08:33:12.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.089+0000 7f04ef0de640 1 -- start start 2024-04-14T08:33:12.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.089+0000 7f04ef0de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e806c530 0x7f04e814c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.089+0000 7f04ef0de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04e80729b0 0x7f04e8149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.089+0000 7f04ef0de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 0x7f04e815e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.089+0000 7f04ef0de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04e80af040 con 0x7f04e806c530 2024-04-14T08:33:12.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.089+0000 7f04ef0de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f04e80aeec0 con 0x7f04e80729b0 2024-04-14T08:33:12.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.089+0000 7f04ef0de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f04e80b4c70 con 0x7f04e815e5b0 2024-04-14T08:33:12.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ece7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e806c530 0x7f04e814c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ece7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e806c530 0x7f04e814c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49362/0 (socket says 172.21.15.172:49362) 2024-04-14T08:33:12.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ece7a640 1 -- 172.21.15.172:0/344758206 learned_addr learned my addr 172.21.15.172:0/344758206 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04e7fff640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04e80729b0 0x7f04e8149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ed67b640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 0x7f04e815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ece7a640 1 -- 172.21.15.172:0/344758206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 msgr2=0x7f04e815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ece7a640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 0x7f04e815e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ece7a640 1 -- 172.21.15.172:0/344758206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04e80729b0 msgr2=0x7f04e8149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ece7a640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04e80729b0 0x7f04e8149110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ece7a640 1 -- 172.21.15.172:0/344758206 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e814ca10 con 0x7f04e806c530 2024-04-14T08:33:12.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04e7fff640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04e80729b0 0x7f04e8149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ed67b640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 0x7f04e815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ece7a640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e806c530 0x7f04e814c4d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f04e0036040 tx=0x7f04e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04e57fa640 1 -- 172.21.15.172:0/344758206 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0044020 con 0x7f04e806c530 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04e57fa640 1 -- 172.21.15.172:0/344758206 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e002ce50 con 0x7f04e806c530 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ef0de640 1 -- 172.21.15.172:0/344758206 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e8149dd0 con 0x7f04e806c530 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04e57fa640 1 -- 172.21.15.172:0/344758206 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0009510 con 0x7f04e806c530 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.090+0000 7f04ef0de640 1 -- 172.21.15.172:0/344758206 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e814a390 con 0x7f04e806c530 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.091+0000 7f04ef0de640 1 -- 172.21.15.172:0/344758206 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e8068520 con 0x7f04e806c530 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.092+0000 7f04e57fa640 1 -- 172.21.15.172:0/344758206 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04e0047020 con 0x7f04e806c530 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.092+0000 7f04e57fa640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f04c806c8f0 0x7f04c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.092+0000 7f04e57fa640 1 -- 172.21.15.172:0/344758206 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f04e00ae5f0 con 0x7f04e806c530 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.092+0000 7f04e7fff640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f04c806c8f0 0x7f04c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.093+0000 7f04e7fff640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f04c806c8f0 0x7f04c806edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f04e8070300 tx=0x7f04d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.097+0000 7f04e57fa640 1 -- 172.21.15.172:0/344758206 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f04e004e030 con 0x7f04e806c530 2024-04-14T08:33:12.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.272+0000 7f04ef0de640 1 -- 172.21.15.172:0/344758206 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04e815db20 con 0x7f04e806c530 2024-04-14T08:33:12.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.273+0000 7f04e57fa640 1 -- 172.21.15.172:0/344758206 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7f04e814ca10 con 0x7f04e806c530 2024-04-14T08:33:12.274 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:12.274 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:12.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.275+0000 7f04ef0de640 1 -- 172.21.15.172:0/344758206 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f04c806c8f0 msgr2=0x7f04c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.275+0000 7f04ef0de640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f04c806c8f0 0x7f04c806edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f04e8070300 tx=0x7f04d0009040 comp rx=0 tx=0).stop 2024-04-14T08:33:12.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.275+0000 7f04ef0de640 1 -- 172.21.15.172:0/344758206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e806c530 msgr2=0x7f04e814c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.275+0000 7f04ef0de640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e806c530 0x7f04e814c4d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f04e0036040 tx=0x7f04e0004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:12.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.275+0000 7f04ef0de640 1 -- 172.21.15.172:0/344758206 shutdown_connections 2024-04-14T08:33:12.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.275+0000 7f04ef0de640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f04c806c8f0 0x7f04c806edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.275+0000 7f04ef0de640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04e815e5b0 0x7f04e815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.275+0000 7f04ef0de640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04e80729b0 0x7f04e8149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.275+0000 7f04ef0de640 1 --2- 172.21.15.172:0/344758206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04e806c530 0x7f04e814c4d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.275+0000 7f04ef0de640 1 -- 172.21.15.172:0/344758206 >> 172.21.15.172:0/344758206 conn(0x7f04e8071ba0 msgr2=0x7f04e80b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.275+0000 7f04ef0de640 1 -- 172.21.15.172:0/344758206 shutdown_connections 2024-04-14T08:33:12.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.275+0000 7f04ef0de640 1 -- 172.21.15.172:0/344758206 wait complete. 2024-04-14T08:33:12.287 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-14T08:33:12.288 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:12.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.387+0000 7f3b08020640 1 Processor -- start 2024-04-14T08:33:12.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.387+0000 7f3b08020640 1 -- start start 2024-04-14T08:33:12.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.388+0000 7f3b08020640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3b000b21d0 0x7f3b0015c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.388+0000 7f3b08020640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b000b3700 con 0x7f3b0015c5a0 2024-04-14T08:33:12.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.388+0000 7f3b08020640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b000b39e0 con 0x7f3b000b1df0 2024-04-14T08:33:12.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.388+0000 7f3b08020640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f3b000b3cc0 con 0x7f3b000b21d0 2024-04-14T08:33:12.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.388+0000 7f3b055bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3b000b21d0 0x7f3b0015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.388+0000 7f3b065bd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3b0015c5a0 0x7f3b000b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35620/0 (socket says 172.21.15.172:35620) 2024-04-14T08:33:12.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.388+0000 7f3b055bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3b000b21d0 0x7f3b0015c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49368/0 (socket says 172.21.15.172:49368) 2024-04-14T08:33:12.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.388+0000 7f3b065bd640 1 -- 172.21.15.172:0/3110308158 learned_addr learned my addr 172.21.15.172:0/3110308158 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:12.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.389+0000 7f3b055bb640 1 -- 172.21.15.172:0/3110308158 >> v1:172.21.15.177:6789/0 conn(0x7f3b000b1df0 legacy=0x7f3b000b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.389+0000 7f3b055bb640 1 -- 172.21.15.172:0/3110308158 >> v1:172.21.15.172:6789/0 conn(0x7f3b0015c5a0 legacy=0x7f3b000b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.389+0000 7f3b055bb640 1 -- 172.21.15.172:0/3110308158 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0015bb40 con 0x7f3b000b21d0 2024-04-14T08:33:12.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.389+0000 7f3b04dba640 1 -- 172.21.15.172:0/3110308158 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3444001791 0 0) 0x7f3b000b39e0 con 0x7f3b000b1df0 2024-04-14T08:33:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.389+0000 7f3b055bb640 1 --2- 172.21.15.172:0/3110308158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b000b21d0 0x7f3b0015c060 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3af00097a0 tx=0x7f3af002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44625516161f21b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.389+0000 7f3b04dba640 1 -- 172.21.15.172:0/3110308158 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0037070 con 0x7f3b000b21d0 2024-04-14T08:33:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.389+0000 7f3b04dba640 1 -- 172.21.15.172:0/3110308158 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af0030710 con 0x7f3b000b21d0 2024-04-14T08:33:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.390+0000 7f3b04dba640 1 -- 172.21.15.172:0/3110308158 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0030ae0 con 0x7f3b000b21d0 2024-04-14T08:33:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.390+0000 7f3b08020640 1 -- 172.21.15.172:0/3110308158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b000b21d0 msgr2=0x7f3b0015c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.390+0000 7f3b08020640 1 --2- 172.21.15.172:0/3110308158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b000b21d0 0x7f3b0015c060 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3af00097a0 tx=0x7f3af002b790 comp rx=0 tx=0).stop 2024-04-14T08:33:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.390+0000 7f3b08020640 1 -- 172.21.15.172:0/3110308158 shutdown_connections 2024-04-14T08:33:12.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.390+0000 7f3b08020640 1 --2- 172.21.15.172:0/3110308158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b000b21d0 0x7f3b0015c060 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.390+0000 7f3b08020640 1 -- 172.21.15.172:0/3110308158 >> 172.21.15.172:0/3110308158 conn(0x7f3b0006ec30 msgr2=0x7f3b0006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.390+0000 7f3b08020640 1 -- 172.21.15.172:0/3110308158 shutdown_connections 2024-04-14T08:33:12.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.390+0000 7f3b08020640 1 -- 172.21.15.172:0/3110308158 wait complete. 2024-04-14T08:33:12.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.391+0000 7f3b08020640 1 Processor -- start 2024-04-14T08:33:12.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.391+0000 7f3b08020640 1 -- start start 2024-04-14T08:33:12.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b08020640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b000b1df0 0x7f3b0013d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b08020640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b000b21d0 0x7f3b0013cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b08020640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b000ae2a0 0x7f3b0015d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b08020640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3b000b48d0 con 0x7f3b000ae2a0 2024-04-14T08:33:12.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b08020640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3b000b4750 con 0x7f3b000b1df0 2024-04-14T08:33:12.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b08020640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3b000aead0 con 0x7f3b000b21d0 2024-04-14T08:33:12.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b065bd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b000ae2a0 0x7f3b0015d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b065bd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b000ae2a0 0x7f3b0015d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49374/0 (socket says 172.21.15.172:49374) 2024-04-14T08:33:12.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b065bd640 1 -- 172.21.15.172:0/1111001699 learned_addr learned my addr 172.21.15.172:0/1111001699 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b055bb640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b000b21d0 0x7f3b0013cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b05dbc640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b000b1df0 0x7f3b0013d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b065bd640 1 -- 172.21.15.172:0/1111001699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b000b21d0 msgr2=0x7f3b0013cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b065bd640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b000b21d0 0x7f3b0013cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b065bd640 1 -- 172.21.15.172:0/1111001699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b000b1df0 msgr2=0x7f3b0013d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b065bd640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b000b1df0 0x7f3b0013d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b065bd640 1 -- 172.21.15.172:0/1111001699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b000aedb0 con 0x7f3b000ae2a0 2024-04-14T08:33:12.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.392+0000 7f3b065bd640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b000ae2a0 0x7f3b0015d980 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3af800c570 tx=0x7f3af800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.393+0000 7f3af67fc640 1 -- 172.21.15.172:0/1111001699 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8018070 con 0x7f3b000ae2a0 2024-04-14T08:33:12.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.393+0000 7f3af67fc640 1 -- 172.21.15.172:0/1111001699 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af80074e0 con 0x7f3b000ae2a0 2024-04-14T08:33:12.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.393+0000 7f3af67fc640 1 -- 172.21.15.172:0/1111001699 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8004a00 con 0x7f3b000ae2a0 2024-04-14T08:33:12.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.393+0000 7f3b08020640 1 -- 172.21.15.172:0/1111001699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b000af090 con 0x7f3b000ae2a0 2024-04-14T08:33:12.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.393+0000 7f3b08020640 1 -- 172.21.15.172:0/1111001699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b000ae750 con 0x7f3b000ae2a0 2024-04-14T08:33:12.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.394+0000 7f3b08020640 1 -- 172.21.15.172:0/1111001699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b000af260 con 0x7f3b000ae2a0 2024-04-14T08:33:12.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.394+0000 7f3af67fc640 1 -- 172.21.15.172:0/1111001699 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3af801b070 con 0x7f3b000ae2a0 2024-04-14T08:33:12.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.394+0000 7f3af67fc640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ae406c8f0 0x7f3ae406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.394+0000 7f3af67fc640 1 -- 172.21.15.172:0/1111001699 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3af808b170 con 0x7f3b000ae2a0 2024-04-14T08:33:12.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.395+0000 7f3b05dbc640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ae406c8f0 0x7f3ae406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.395+0000 7f3b05dbc640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ae406c8f0 0x7f3ae406edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0012e0 tx=0x7f3afc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.399+0000 7f3af67fc640 1 -- 172.21.15.172:0/1111001699 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3af8058c10 con 0x7f3b000ae2a0 2024-04-14T08:33:12.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.573+0000 7f3b08020640 1 -- 172.21.15.172:0/1111001699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b00068520 con 0x7f3b000ae2a0 2024-04-14T08:33:12.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.575+0000 7f3af67fc640 1 -- 172.21.15.172:0/1111001699 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7f3af805cf10 con 0x7f3b000ae2a0 2024-04-14T08:33:12.575 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:12.576 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:12.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.577+0000 7f3b08020640 1 -- 172.21.15.172:0/1111001699 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ae406c8f0 msgr2=0x7f3ae406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.577+0000 7f3b08020640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ae406c8f0 0x7f3ae406edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0012e0 tx=0x7f3afc009040 comp rx=0 tx=0).stop 2024-04-14T08:33:12.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.577+0000 7f3b08020640 1 -- 172.21.15.172:0/1111001699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b000ae2a0 msgr2=0x7f3b0015d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.577+0000 7f3b08020640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b000ae2a0 0x7f3b0015d980 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3af800c570 tx=0x7f3af800ca30 comp rx=0 tx=0).stop 2024-04-14T08:33:12.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.577+0000 7f3b08020640 1 -- 172.21.15.172:0/1111001699 shutdown_connections 2024-04-14T08:33:12.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.577+0000 7f3b08020640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ae406c8f0 0x7f3ae406edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.577+0000 7f3b08020640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b000ae2a0 0x7f3b0015d980 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.577+0000 7f3b08020640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b000b21d0 0x7f3b0013cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.577+0000 7f3b08020640 1 --2- 172.21.15.172:0/1111001699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b000b1df0 0x7f3b0013d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.577+0000 7f3b08020640 1 -- 172.21.15.172:0/1111001699 >> 172.21.15.172:0/1111001699 conn(0x7f3b0006ec30 msgr2=0x7f3b000728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.577+0000 7f3b08020640 1 -- 172.21.15.172:0/1111001699 shutdown_connections 2024-04-14T08:33:12.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.577+0000 7f3b08020640 1 -- 172.21.15.172:0/1111001699 wait complete. 2024-04-14T08:33:12.589 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-14T08:33:12.589 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-14T08:33:12.590 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-14T08:33:12.590 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-14T08:33:12.590 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-14T08:33:12.591 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-14T08:33:12.591 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-14T08:33:12.592 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-14T08:33:12.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.758+0000 7feebff1e640 1 Processor -- start 2024-04-14T08:33:12.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.759+0000 7feebff1e640 1 -- start start 2024-04-14T08:33:12.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.759+0000 7feebff1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7feeb815f910 0x7feeb80cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.760+0000 7feebff1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feeb815fcf0 con 0x7feeb80c5ac0 2024-04-14T08:33:12.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.760+0000 7feebff1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feeb80b3560 con 0x7feeb80ccef0 2024-04-14T08:33:12.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.760+0000 7feebff1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7feeb80b3280 con 0x7feeb815f910 2024-04-14T08:33:12.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.760+0000 7feebdcba640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7feeb815f910 0x7feeb80cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.760+0000 7feebdcba640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7feeb815f910 0x7feeb80cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:32904/0 (socket says 172.21.15.172:32904) 2024-04-14T08:33:12.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.760+0000 7feebdcba640 1 -- 172.21.15.172:0/2295654149 learned_addr learned my addr 172.21.15.172:0/2295654149 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.760+0000 7feebd4b9640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7feeb80ccef0 0x7feeb80cd350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42654/0 (socket says 172.21.15.172:42654) 2024-04-14T08:33:12.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.760+0000 7feebdcba640 1 -- 172.21.15.172:0/2295654149 >> v1:172.21.15.177:6789/0 conn(0x7feeb80ccef0 legacy=0x7feeb80cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.760+0000 7feebdcba640 1 -- 172.21.15.172:0/2295654149 >> v1:172.21.15.172:6789/0 conn(0x7feeb80c5ac0 legacy=0x7feeb80c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.760+0000 7feebdcba640 1 -- 172.21.15.172:0/2295654149 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb80d0560 con 0x7feeb815f910 2024-04-14T08:33:12.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.761+0000 7feebdcba640 1 --2- 172.21.15.172:0/2295654149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feeb815f910 0x7feeb80cc9b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009870 tx=0x7feeb402ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af09e0cc26e61fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.761+0000 7feebccb8640 1 -- 172.21.15.172:0/2295654149 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb402f840 con 0x7feeb815f910 2024-04-14T08:33:12.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.761+0000 7feebccb8640 1 -- 172.21.15.172:0/2295654149 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb402f9e0 con 0x7feeb815f910 2024-04-14T08:33:12.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.762+0000 7feebccb8640 1 -- 172.21.15.172:0/2295654149 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb402fdb0 con 0x7feeb815f910 2024-04-14T08:33:12.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.762+0000 7feebff1e640 1 -- 172.21.15.172:0/2295654149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feeb815f910 msgr2=0x7feeb80cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.762+0000 7feebff1e640 1 --2- 172.21.15.172:0/2295654149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feeb815f910 0x7feeb80cc9b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009870 tx=0x7feeb402ba20 comp rx=0 tx=0).stop 2024-04-14T08:33:12.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.762+0000 7feebff1e640 1 -- 172.21.15.172:0/2295654149 shutdown_connections 2024-04-14T08:33:12.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.762+0000 7feebff1e640 1 --2- 172.21.15.172:0/2295654149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feeb815f910 0x7feeb80cc9b0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009870 tx=0x7feeb402ba20 comp rx=0 tx=0).stop 2024-04-14T08:33:12.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.762+0000 7feebff1e640 1 -- 172.21.15.172:0/2295654149 >> 172.21.15.172:0/2295654149 conn(0x7feeb80d1ee0 msgr2=0x7feeb80d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.763+0000 7f523ec2d640 1 Processor -- start 2024-04-14T08:33:12.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.763+0000 7feebff1e640 1 -- 172.21.15.172:0/2295654149 shutdown_connections 2024-04-14T08:33:12.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.763+0000 7feebff1e640 1 -- 172.21.15.172:0/2295654149 wait complete. 2024-04-14T08:33:12.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.763+0000 7f0a5bd95640 1 Processor -- start 2024-04-14T08:33:12.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.763+0000 7f0a5bd95640 1 -- start start 2024-04-14T08:33:12.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.763+0000 7f523ec2d640 1 -- start start 2024-04-14T08:33:12.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.763+0000 7f4475c20640 1 Processor -- start 2024-04-14T08:33:12.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.763+0000 7f4475c20640 1 -- start start 2024-04-14T08:33:12.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.764+0000 7f0579b1e640 1 Processor -- start 2024-04-14T08:33:12.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.764+0000 7f523ec2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523000a460 0x7f5230055db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.764+0000 7f523ec2d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5230054d10 con 0x7f52300e1310 2024-04-14T08:33:12.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.764+0000 7f523ec2d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5230054ff0 con 0x7f523000ab40 2024-04-14T08:33:12.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.764+0000 7f523ec2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5230055920 con 0x7f523000a460 2024-04-14T08:33:12.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.764+0000 7f0a5bd95640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a5416f480 0x7f0a5408b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.764+0000 7f0a5bd95640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0a54072800 0x7f0a54172eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.764+0000 7f0a5bd95640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a54071760 con 0x7f0a541733f0 2024-04-14T08:33:12.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.764+0000 7f0a5bd95640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0a54071a40 con 0x7f0a54072800 2024-04-14T08:33:12.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.764+0000 7f0a5bd95640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0a54071d20 con 0x7f0a5416f480 2024-04-14T08:33:12.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.764+0000 7f0579b1e640 1 -- start start 2024-04-14T08:33:12.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.764+0000 7f8eb9636640 1 Processor -- start 2024-04-14T08:33:12.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f45d45dc640 1 Processor -- start 2024-04-14T08:33:12.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f446f7fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f44700b5790 0x7f44700d2a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42666/0 (socket says 172.21.15.172:42666) 2024-04-14T08:33:12.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7feebff1e640 1 Processor -- start 2024-04-14T08:33:12.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f5237fff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f523000ab40 0x7f5230009d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42690/0 (socket says 172.21.15.172:42690) 2024-04-14T08:33:12.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f523d1ca640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523000a460 0x7f5230055db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f523c9c9640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f52300e1310 0x7f523003a170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35660/0 (socket says 172.21.15.172:35660) 2024-04-14T08:33:12.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f5237fff640 1 -- 172.21.15.172:0/1318139865 learned_addr learned my addr 172.21.15.172:0/1318139865 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:12.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7feebff1e640 1 -- start start 2024-04-14T08:33:12.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a5a332640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f0a541733f0 0x7f0a540ad8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42680/0 (socket says 172.21.15.172:42680) 2024-04-14T08:33:12.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a59330640 1 --2- 172.21.15.172:0/2342387392 >> v2:172.21.15.172:3300/0 conn(0x7f0a54072800 0x7f0a54172eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a5a332640 1 -- 172.21.15.172:0/2342387392 learned_addr learned my addr 172.21.15.172:0/2342387392 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:12.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a59b31640 1 --2- 172.21.15.172:0/2342387392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a5416f480 0x7f0a5408b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3319372111 0 0) 0x7f0a54071760 con 0x7f0a541733f0 2024-04-14T08:33:12.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a40003520 con 0x7f0a541733f0 2024-04-14T08:33:12.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2694171091 0 0) 0x7f0a40003520 con 0x7f0a541733f0 2024-04-14T08:33:12.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a5416f480 msgr2=0x7f0a5408b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a58b2f640 1 --2- 172.21.15.172:0/2342387392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a5416f480 0x7f0a5408b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 >> v2:172.21.15.172:3300/0 conn(0x7f0a54072800 msgr2=0x7f0a54172eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f52377fe640 1 -- 172.21.15.172:0/1318139865 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1191008077 0 0) 0x7f5230054ff0 con 0x7f523000ab40 2024-04-14T08:33:12.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f52377fe640 1 -- 172.21.15.172:0/1318139865 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5228003520 con 0x7f523000ab40 2024-04-14T08:33:12.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a58b2f640 1 --2- 172.21.15.172:0/2342387392 >> v2:172.21.15.172:3300/0 conn(0x7f0a54072800 0x7f0a54172eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a54072000 con 0x7f0a541733f0 2024-04-14T08:33:12.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f0a4c0019f0 con 0x7f0a541733f0 2024-04-14T08:33:12.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f523d1ca640 1 -- 172.21.15.172:0/1318139865 >> v1:172.21.15.177:6789/0 conn(0x7f523000ab40 legacy=0x7f5230009d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 >> v1:172.21.15.177:6789/0 conn(0x7f0a541733f0 legacy=0x7f0a540ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0a58b2f640 1 --2- 172.21.15.172:0/2342387392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a400039b0 0x7f0a40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f4475c20640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700d0990 0x7f44700d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f4475c20640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44700ced40 con 0x7f44700d4c00 2024-04-14T08:33:12.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f4475c20640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44700cefe0 con 0x7f44700b5790 2024-04-14T08:33:12.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f4475c20640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f44700b5b70 con 0x7f44700d0990 2024-04-14T08:33:12.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f446f7fe640 1 -- 172.21.15.172:0/1802186793 learned_addr learned my addr 172.21.15.172:0/1802186793 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:12.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f45d45dc640 1 -- start start 2024-04-14T08:33:12.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f52377fe640 1 -- 172.21.15.172:0/1318139865 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 847567138 0 0) 0x7f5228003520 con 0x7f523000ab40 2024-04-14T08:33:12.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f0a58b2f640 1 --2- 172.21.15.172:0/2342387392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a5416f480 0x7f0a40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f523d1ca640 1 -- 172.21.15.172:0/1318139865 >> v1:172.21.15.172:6789/0 conn(0x7f52300e1310 legacy=0x7f523003a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0579b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05740b23e0 0x7f05740b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0579b1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f05740b5540 0x7f05740b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0579b1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05740712d0 con 0x7f057414c130 2024-04-14T08:33:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0579b1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f05740715b0 con 0x7f05740b5540 2024-04-14T08:33:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f0579b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0574071890 con 0x7f05740b23e0 2024-04-14T08:33:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f05737fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f057414c130 0x7f05740b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42706/0 (socket says 172.21.15.172:42706) 2024-04-14T08:33:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f05737fe640 1 -- 172.21.15.172:0/4099203692 learned_addr learned my addr 172.21.15.172:0/4099203692 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7feebff1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feeb80c5ac0 0x7feeb814d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7feebff1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feeb80ccef0 0x7feeb80ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7feebff1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feeb80cadd0 0x7feeb815de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7feebff1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feeb80b2ea0 con 0x7feeb80ccef0 2024-04-14T08:33:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7feebff1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feeb80d0b20 con 0x7feeb80c5ac0 2024-04-14T08:33:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7feebff1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7feeb80b3020 con 0x7feeb80cadd0 2024-04-14T08:33:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7feebdcba640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feeb80c5ac0 0x7feeb814d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f0a58b2f640 1 --2- 172.21.15.172:0/2342387392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a40004d40 0x7f0a400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a54071a40 con 0x7f0a40004d40 2024-04-14T08:33:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0a54071d20 con 0x7f0a400039b0 2024-04-14T08:33:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4000b9c0 con 0x7f0a5416f480 2024-04-14T08:33:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f0a59330640 1 --2- 172.21.15.172:0/2342387392 >> v2:172.21.15.172:3300/0 conn(0x7f0a54072800 0x7f0a54172eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f0a59b31640 1 --2- 172.21.15.172:0/2342387392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a400039b0 0x7f0a40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.765+0000 7f8eb9636640 1 -- start start 2024-04-14T08:33:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f8eb9636640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8eb40cff80 0x7f8eb40cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f8eb9636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eb40733b0 0x7f8eb40ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f8eb9636640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb4072f50 con 0x7f8eb415b220 2024-04-14T08:33:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f8eb9636640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb40cc110 con 0x7f8eb40cff80 2024-04-14T08:33:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f8eb9636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb40726b0 con 0x7f8eb40733b0 2024-04-14T08:33:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f8eb37fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eb40733b0 0x7f8eb40ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f8eb37fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eb40733b0 0x7f8eb40ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:43030/0 (socket says 172.21.15.172:43030) 2024-04-14T08:33:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f8eb37fe640 1 -- 172.21.15.172:0/1579792746 learned_addr learned my addr 172.21.15.172:0/1579792746 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f8eb2ffd640 1 --1- 172.21.15.172:0/1579792746 >> v1:172.21.15.177:6789/0 conn(0x7f8eb415b220 0x7f8eb40cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42722/0 (socket says 172.21.15.172:42722) 2024-04-14T08:33:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f8eb27fc640 1 --2- 172.21.15.172:0/1579792746 >> v2:172.21.15.177:3300/0 conn(0x7f8eb40cff80 0x7f8eb40cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f0572ffd640 1 --2- 172.21.15.172:0/4099203692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05740b23e0 0x7f05740b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7feebd4b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feeb80ccef0 0x7feeb80ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7feebe4bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feeb80cadd0 0x7feeb815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7feebe4bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feeb80cadd0 0x7feeb815de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:43044/0 (socket says 172.21.15.172:43044) 2024-04-14T08:33:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7feebe4bb640 1 -- 172.21.15.172:0/2989939713 learned_addr learned my addr 172.21.15.172:0/2989939713 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7feebd4b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feeb80ccef0 0x7feeb80ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49414/0 (socket says 172.21.15.172:49414) 2024-04-14T08:33:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f45d45dc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f45cc15e590 0x7f45cc15ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f45d45dc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f45cc0ceff0 0x7f45cc071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f45d45dc640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45cc0cf4c0 con 0x7f45cc143a40 2024-04-14T08:33:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f45d45dc640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f45cc071aa0 con 0x7f45cc0ceff0 2024-04-14T08:33:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f45d45dc640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f45cc071450 con 0x7f45cc15e590 2024-04-14T08:33:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3581665146 0 0) 0x7f44700ced40 con 0x7f44700d4c00 2024-04-14T08:33:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4458003520 con 0x7f44700d4c00 2024-04-14T08:33:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.766+0000 7f446ffff640 1 --2- 172.21.15.172:0/1802186793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700d0990 0x7f44700d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2957632596 0 0) 0x7f4458003520 con 0x7f44700d4c00 2024-04-14T08:33:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700d0990 msgr2=0x7f44700d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 --2- 172.21.15.172:0/1802186793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700d0990 0x7f44700d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 >> v1:172.21.15.177:6789/0 conn(0x7f44700b5790 legacy=0x7f44700d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f8eb1ffb640 1 -- 172.21.15.172:0/1579792746 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3356120593 0 0) 0x7f8eb4072f50 con 0x7f8eb415b220 2024-04-14T08:33:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f8eb1ffb640 1 -- 172.21.15.172:0/1579792746 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e9c003520 con 0x7f8eb415b220 2024-04-14T08:33:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f8eb37fe640 1 -- 172.21.15.172:0/1579792746 >> v2:172.21.15.177:3300/0 conn(0x7f8eb40cff80 msgr2=0x7f8eb40cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f8eb37fe640 1 --2- 172.21.15.172:0/1579792746 >> v2:172.21.15.177:3300/0 conn(0x7f8eb40cff80 0x7f8eb40cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f8eb37fe640 1 -- 172.21.15.172:0/1579792746 >> v1:172.21.15.177:6789/0 conn(0x7f8eb415b220 legacy=0x7f8eb40cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f8eb37fe640 1 -- 172.21.15.172:0/1579792746 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb4072990 con 0x7f8eb40733b0 2024-04-14T08:33:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44700cf2c0 con 0x7f44700d4c00 2024-04-14T08:33:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f4464001670 con 0x7f44700d4c00 2024-04-14T08:33:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 >> v1:172.21.15.172:6789/0 conn(0x7f44700d4c00 legacy=0x7f44700cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 --2- 172.21.15.172:0/1802186793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700d0990 0x7f44580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7feebe4bb640 1 -- 172.21.15.172:0/2989939713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feeb80c5ac0 msgr2=0x7feeb814d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7feebe4bb640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feeb80c5ac0 0x7feeb814d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7feebe4bb640 1 -- 172.21.15.172:0/2989939713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feeb80ccef0 msgr2=0x7feeb80ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7feebe4bb640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feeb80ccef0 0x7feeb80ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7feebe4bb640 1 -- 172.21.15.172:0/2989939713 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb80cb330 con 0x7feeb80cadd0 2024-04-14T08:33:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7feebdcba640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feeb80c5ac0 0x7feeb814d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:33:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f45d2b79640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f45cc0ceff0 0x7f45cc071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f45d2b79640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f45cc0ceff0 0x7f45cc071f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49430/0 (socket says 172.21.15.172:49430) 2024-04-14T08:33:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f45d2b79640 1 -- 172.21.15.172:0/2362697012 learned_addr learned my addr 172.21.15.172:0/2362697012 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f0a59b31640 1 -- 172.21.15.172:0/2342387392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a5416f480 msgr2=0x7f0a40004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-14T08:33:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f45d1b77640 1 --2- 172.21.15.172:0/2362697012 >> v2:172.21.15.177:3300/0 conn(0x7f45cc15e590 0x7f45cc15ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f0a59b31640 1 --2- 172.21.15.172:0/2342387392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a5416f480 0x7f0a40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f0a59b31640 1 -- 172.21.15.172:0/2342387392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a40004d40 msgr2=0x7f0a400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 --2- 172.21.15.172:0/1802186793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4458004010 0x7f4458004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 --2- 172.21.15.172:0/1802186793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4458004d40 0x7f44580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f44700b5b70 con 0x7f4458004010 2024-04-14T08:33:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f445800bb80 con 0x7f4458004d40 2024-04-14T08:33:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f445800bd80 con 0x7f44700d0990 2024-04-14T08:33:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446f7fe640 1 --2- 172.21.15.172:0/1802186793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700d0990 0x7f44580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446ffff640 1 --2- 172.21.15.172:0/1802186793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4458004d40 0x7f44580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f0a59b31640 1 --2- 172.21.15.172:0/2342387392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a40004d40 0x7f0a400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f0a59b31640 1 -- 172.21.15.172:0/2342387392 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a40003520 con 0x7f0a400039b0 2024-04-14T08:33:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1083084848 0 0) 0x7f05740712d0 con 0x7f057414c130 2024-04-14T08:33:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f055c003520 con 0x7f057414c130 2024-04-14T08:33:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1358437177 0 0) 0x7f055c003520 con 0x7f057414c130 2024-04-14T08:33:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05740b23e0 msgr2=0x7f05740b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 --2- 172.21.15.172:0/4099203692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05740b23e0 0x7f05740b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 >> v2:172.21.15.172:3300/0 conn(0x7f05740b5540 msgr2=0x7f05740b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 --2- 172.21.15.172:0/4099203692 >> v2:172.21.15.172:3300/0 conn(0x7f05740b5540 0x7f05740b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0574071b70 con 0x7f057414c130 2024-04-14T08:33:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f0568001670 con 0x7f057414c130 2024-04-14T08:33:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 >> v1:172.21.15.177:6789/0 conn(0x7f057414c130 legacy=0x7f05740b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 --2- 172.21.15.172:0/4099203692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c0039b0 0x7f055c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 --2- 172.21.15.172:0/4099203692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f055c004800 0x7f055c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 --2- 172.21.15.172:0/4099203692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c0051c0 0x7f055c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f05740715b0 con 0x7f055c004800 2024-04-14T08:33:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0574071890 con 0x7f055c0039b0 2024-04-14T08:33:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f055c01d040 con 0x7f055c0051c0 2024-04-14T08:33:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f05737fe640 1 --2- 172.21.15.172:0/4099203692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c0039b0 0x7f055c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f0573fff640 1 --2- 172.21.15.172:0/4099203692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c0051c0 0x7f055c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.768+0000 7feebe4bb640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feeb80cadd0 0x7feeb815de50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feeb000c760 tx=0x7feeb000cb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.768+0000 7f523d1ca640 1 -- 172.21.15.172:0/1318139865 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5230009b60 con 0x7f523000a460 2024-04-14T08:33:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.768+0000 7f0572ffd640 1 --2- 172.21.15.172:0/4099203692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f055c004800 0x7f055c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.768+0000 7f0573fff640 1 -- 172.21.15.172:0/4099203692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c0039b0 msgr2=0x7f055c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.768+0000 7f0573fff640 1 --2- 172.21.15.172:0/4099203692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c0039b0 0x7f055c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.768+0000 7f0573fff640 1 -- 172.21.15.172:0/4099203692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f055c004800 msgr2=0x7f055c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.768+0000 7f0573fff640 1 --2- 172.21.15.172:0/4099203692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f055c004800 0x7f055c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.768+0000 7f0573fff640 1 -- 172.21.15.172:0/4099203692 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0574071b70 con 0x7f055c0051c0 2024-04-14T08:33:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f45d1b77640 1 -- 172.21.15.172:0/2362697012 >> v2:172.21.15.172:3300/0 conn(0x7f45cc0ceff0 msgr2=0x7f45cc071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f45d1b77640 1 --2- 172.21.15.172:0/2362697012 >> v2:172.21.15.172:3300/0 conn(0x7f45cc0ceff0 0x7f45cc071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f45d1b77640 1 -- 172.21.15.172:0/2362697012 >> v1:172.21.15.177:6789/0 conn(0x7f45cc143a40 legacy=0x7f45cc15de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.768+0000 7f45d1b77640 1 -- 172.21.15.172:0/2362697012 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc071730 con 0x7f45cc15e590 2024-04-14T08:33:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.768+0000 7feeae7fc640 1 -- 172.21.15.172:0/2989939713 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0016020 con 0x7feeb80cadd0 2024-04-14T08:33:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.768+0000 7f8eb37fe640 1 --2- 172.21.15.172:0/1579792746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eb40733b0 0x7f8eb40ccae0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400bb10 tx=0x7f8ea402e170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f24445b0f76f68c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446f7fe640 1 -- 172.21.15.172:0/1802186793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4458004d40 msgr2=0x7f44580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446f7fe640 1 --2- 172.21.15.172:0/1802186793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4458004d40 0x7f44580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446f7fe640 1 -- 172.21.15.172:0/1802186793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4458004010 msgr2=0x7f4458004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446f7fe640 1 --2- 172.21.15.172:0/1802186793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4458004010 0x7f4458004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.767+0000 7f446f7fe640 1 -- 172.21.15.172:0/1802186793 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44700cf2c0 con 0x7f44700d0990 2024-04-14T08:33:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7feebff1e640 1 -- 172.21.15.172:0/2989939713 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb80cb610 con 0x7feeb80cadd0 2024-04-14T08:33:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.768+0000 7f0a59b31640 1 --2- 172.21.15.172:0/2342387392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a400039b0 0x7f0a40003dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009ac0 tx=0x7f0a40005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f64fd1ac0b76341 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7feeae7fc640 1 -- 172.21.15.172:0/2989939713 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb0006d50 con 0x7feeb80cadd0 2024-04-14T08:33:12.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7feebff1e640 1 -- 172.21.15.172:0/2989939713 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb80b4df0 con 0x7feeb80cadd0 2024-04-14T08:33:12.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f8eb1ffb640 1 -- 172.21.15.172:0/1579792746 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea403b070 con 0x7f8eb40733b0 2024-04-14T08:33:12.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f446f7fe640 1 --2- 172.21.15.172:0/1802186793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700d0990 0x7f44580039e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4460001810 tx=0x7f4458005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa7e85c00739ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f8eb1ffb640 1 -- 172.21.15.172:0/1579792746 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea4036440 con 0x7f8eb40733b0 2024-04-14T08:33:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f8eb1ffb640 1 -- 172.21.15.172:0/1579792746 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4036850 con 0x7f8eb40733b0 2024-04-14T08:33:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f45d1b77640 1 --2- 172.21.15.172:0/2362697012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45cc15e590 0x7f45cc15ba50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f45cc068dc0 tx=0x7f45c402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eee475faf66d52bc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50039070 con 0x7f0a400039b0 2024-04-14T08:33:12.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a5002c710 con 0x7f0a400039b0 2024-04-14T08:33:12.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f0a58b2f640 1 -- 172.21.15.172:0/2342387392 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50009690 con 0x7f0a400039b0 2024-04-14T08:33:12.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f0573fff640 1 --2- 172.21.15.172:0/4099203692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c0051c0 0x7f055c00b9c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0564009860 tx=0x7f055c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=863bcd76e76f9ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f523d1ca640 1 --2- 172.21.15.172:0/1318139865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523000a460 0x7f5230055db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5224009750 tx=0x7f522402b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=722a00e47e9aece6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f45d1376640 1 -- 172.21.15.172:0/2362697012 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c40315a0 con 0x7f45cc15e590 2024-04-14T08:33:12.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f45d1376640 1 -- 172.21.15.172:0/2362697012 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c4037070 con 0x7f45cc15e590 2024-04-14T08:33:12.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f45d1376640 1 -- 172.21.15.172:0/2362697012 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4031960 con 0x7f45cc15e590 2024-04-14T08:33:12.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460001460 con 0x7f44700d0990 2024-04-14T08:33:12.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44600048a0 con 0x7f44700d0990 2024-04-14T08:33:12.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.769+0000 7f446e7fc640 1 -- 172.21.15.172:0/1802186793 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460004c90 con 0x7f44700d0990 2024-04-14T08:33:12.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f52377fe640 1 -- 172.21.15.172:0/1318139865 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224037070 con 0x7f523000a460 2024-04-14T08:33:12.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f52377fe640 1 -- 172.21.15.172:0/1318139865 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5224030510 con 0x7f523000a460 2024-04-14T08:33:12.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f52377fe640 1 -- 172.21.15.172:0/1318139865 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c001670 con 0x7f523000a460 2024-04-14T08:33:12.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564037070 con 0x7f055c0051c0 2024-04-14T08:33:12.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0564032620 con 0x7f055c0051c0 2024-04-14T08:33:12.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f05727fc640 1 -- 172.21.15.172:0/4099203692 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564009460 con 0x7f055c0051c0 2024-04-14T08:33:12.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7feeae7fc640 1 -- 172.21.15.172:0/2989939713 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0003bb0 con 0x7feeb80cadd0 2024-04-14T08:33:12.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7feebff1e640 1 -- 172.21.15.172:0/2989939713 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feeb8075490 con 0x7feeb80cadd0 2024-04-14T08:33:12.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f0a5bd95640 1 -- 172.21.15.172:0/2342387392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a400039b0 msgr2=0x7f0a40003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f0a5bd95640 1 --2- 172.21.15.172:0/2342387392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a400039b0 0x7f0a40003dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009ac0 tx=0x7f0a40005830 comp rx=0 tx=0).stop 2024-04-14T08:33:12.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f0a5a332640 1 -- 172.21.15.172:0/2342387392 reap_dead start 2024-04-14T08:33:12.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f0a5bd95640 1 -- 172.21.15.172:0/2342387392 shutdown_connections 2024-04-14T08:33:12.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f0a5bd95640 1 -- 172.21.15.172:0/2342387392 >> 172.21.15.172:0/2342387392 conn(0x7f0a540b1d40 msgr2=0x7f0a540b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f0a5bd95640 1 -- 172.21.15.172:0/2342387392 shutdown_connections 2024-04-14T08:33:12.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f0a5bd95640 1 -- 172.21.15.172:0/2342387392 wait complete. 2024-04-14T08:33:12.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f8eb9636640 1 -- 172.21.15.172:0/1579792746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eb40733b0 msgr2=0x7f8eb40ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f8eb9636640 1 --2- 172.21.15.172:0/1579792746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eb40733b0 0x7f8eb40ccae0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400bb10 tx=0x7f8ea402e170 comp rx=0 tx=0).stop 2024-04-14T08:33:12.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f8eb9636640 1 -- 172.21.15.172:0/1579792746 shutdown_connections 2024-04-14T08:33:12.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f8eb9636640 1 --2- 172.21.15.172:0/1579792746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eb40733b0 0x7f8eb40ccae0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f8eb9636640 1 --2- 172.21.15.172:0/1579792746 >> v2:172.21.15.177:3300/0 conn(0x7f8eb40cff80 0x7f8eb40cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f8eb9636640 1 -- 172.21.15.172:0/1579792746 >> 172.21.15.172:0/1579792746 conn(0x7f8eb40cda00 msgr2=0x7f8eb40cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f45d45dc640 1 -- 172.21.15.172:0/2362697012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45cc15e590 msgr2=0x7f45cc15ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f45d45dc640 1 --2- 172.21.15.172:0/2362697012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45cc15e590 0x7f45cc15ba50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f45cc068dc0 tx=0x7f45c402b840 comp rx=0 tx=0).stop 2024-04-14T08:33:12.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7fe5c84c0640 1 Processor -- start 2024-04-14T08:33:12.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f4475c20640 1 -- 172.21.15.172:0/1802186793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700d0990 msgr2=0x7f44580039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f4475c20640 1 --2- 172.21.15.172:0/1802186793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700d0990 0x7f44580039e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4460001810 tx=0x7f4458005830 comp rx=0 tx=0).stop 2024-04-14T08:33:12.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.770+0000 7f446ffff640 1 -- 172.21.15.172:0/1802186793 reap_dead start 2024-04-14T08:33:12.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f8eb9636640 1 -- 172.21.15.172:0/1579792746 shutdown_connections 2024-04-14T08:33:12.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f523ec2d640 1 -- 172.21.15.172:0/1318139865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523000a460 msgr2=0x7f5230055db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f523ec2d640 1 --2- 172.21.15.172:0/1318139865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523000a460 0x7f5230055db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5224009750 tx=0x7f522402b5d0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7feeae7fc640 1 -- 172.21.15.172:0/2989939713 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feeb00050d0 con 0x7feeb80cadd0 2024-04-14T08:33:12.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f8eb9636640 1 -- 172.21.15.172:0/1579792746 wait complete. 2024-04-14T08:33:12.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7fe5c84c0640 1 -- start start 2024-04-14T08:33:12.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f523ec2d640 1 -- 172.21.15.172:0/1318139865 shutdown_connections 2024-04-14T08:33:12.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f523ec2d640 1 --2- 172.21.15.172:0/1318139865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523000a460 0x7f5230055db0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f523ec2d640 1 -- 172.21.15.172:0/1318139865 >> 172.21.15.172:0/1318139865 conn(0x7f5230056cd0 msgr2=0x7f52300570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f0579b1e640 1 -- 172.21.15.172:0/4099203692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c0051c0 msgr2=0x7f055c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f0579b1e640 1 --2- 172.21.15.172:0/4099203692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c0051c0 0x7f055c00b9c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0564009860 tx=0x7f055c005830 comp rx=0 tx=0).stop 2024-04-14T08:33:12.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f0573fff640 1 -- 172.21.15.172:0/4099203692 reap_dead start 2024-04-14T08:33:12.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f45d45dc640 1 -- 172.21.15.172:0/2362697012 shutdown_connections 2024-04-14T08:33:12.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f45d45dc640 1 --2- 172.21.15.172:0/2362697012 >> v2:172.21.15.172:3300/0 conn(0x7f45cc0ceff0 0x7f45cc071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f45d45dc640 1 --2- 172.21.15.172:0/2362697012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45cc15e590 0x7f45cc15ba50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f45d45dc640 1 -- 172.21.15.172:0/2362697012 >> 172.21.15.172:0/2362697012 conn(0x7f45cc0b1d40 msgr2=0x7f45cc0b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f0579b1e640 1 -- 172.21.15.172:0/4099203692 shutdown_connections 2024-04-14T08:33:12.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f0579b1e640 1 -- 172.21.15.172:0/4099203692 >> 172.21.15.172:0/4099203692 conn(0x7f057406b520 msgr2=0x7f057406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f4475c20640 1 -- 172.21.15.172:0/1802186793 shutdown_connections 2024-04-14T08:33:12.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f4475c20640 1 -- 172.21.15.172:0/1802186793 >> 172.21.15.172:0/1802186793 conn(0x7f447009d570 msgr2=0x7f44700d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f0579b1e640 1 -- 172.21.15.172:0/4099203692 shutdown_connections 2024-04-14T08:33:12.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f0579b1e640 1 -- 172.21.15.172:0/4099203692 wait complete. 2024-04-14T08:33:12.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f4475c20640 1 -- 172.21.15.172:0/1802186793 shutdown_connections 2024-04-14T08:33:12.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f4475c20640 1 -- 172.21.15.172:0/1802186793 wait complete. 2024-04-14T08:33:12.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7feeae7fc640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fee9c06c8f0 0x7fee9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f523ec2d640 1 -- 172.21.15.172:0/1318139865 shutdown_connections 2024-04-14T08:33:12.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f523ec2d640 1 -- 172.21.15.172:0/1318139865 wait complete. 2024-04-14T08:33:12.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7f0579b1e640 1 Processor -- start 2024-04-14T08:33:12.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.772+0000 7fe5c84c0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe5c00cd670 0x7fe5c00b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.772+0000 7fe5c84c0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c00b0910 con 0x7fe5c00c39c0 2024-04-14T08:33:12.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.772+0000 7fe5c84c0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c00b0bf0 con 0x7fe5c00ccc20 2024-04-14T08:33:12.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.772+0000 7fe5c84c0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c00b0ed0 con 0x7fe5c00cd670 2024-04-14T08:33:12.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.772+0000 7fe5c6a5d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe5c00c39c0 0x7fe5c00ca9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35666/0 (socket says 172.21.15.172:35666) 2024-04-14T08:33:12.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.772+0000 7fe5c6a5d640 1 -- 172.21.15.172:0/3391180520 learned_addr learned my addr 172.21.15.172:0/3391180520 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:12.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2958686645 0 0) 0x7fe5c00b0910 con 0x7fe5c00c39c0 2024-04-14T08:33:12.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5ac003520 con 0x7fe5c00c39c0 2024-04-14T08:33:12.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7feebdcba640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fee9c06c8f0 0x7fee9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.772+0000 7feeae7fc640 1 -- 172.21.15.172:0/2989939713 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7feeb008ba40 con 0x7feeb80cadd0 2024-04-14T08:33:12.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.772+0000 7feeae7fc640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x7fee9c076070 0x7fee9c0784b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.772+0000 7feeae7fc640 1 -- 172.21.15.172:0/2989939713 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feeb001eec0 con 0x7fee9c076070 2024-04-14T08:33:12.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.772+0000 7feeae7fc640 1 -- 172.21.15.172:0/2989939713 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feeb00902a0 con 0x7feeb80cadd0 2024-04-14T08:33:12.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.772+0000 7feebdcba640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fee9c06c8f0 0x7fee9c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feeb4036000 tx=0x7feeb4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7feebd4b9640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x7fee9c076070 0x7fee9c0784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7f8eb9636640 1 Processor -- start 2024-04-14T08:33:12.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7f8eb9636640 1 -- start start 2024-04-14T08:33:12.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f45d45dc640 1 -- 172.21.15.172:0/2362697012 shutdown_connections 2024-04-14T08:33:12.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7f0579b1e640 1 -- start start 2024-04-14T08:33:12.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7fe5c625c640 1 --2- 172.21.15.172:0/3391180520 >> v2:172.21.15.177:3300/0 conn(0x7fe5c00cd670 0x7fe5c00b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7feebd4b9640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x7fee9c076070 0x7fee9c0784b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.771+0000 7f45d45dc640 1 -- 172.21.15.172:0/2362697012 wait complete. 2024-04-14T08:33:12.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7f45d45dc640 1 Processor -- start 2024-04-14T08:33:12.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1854078019 0 0) 0x7fe5ac003520 con 0x7fe5c00c39c0 2024-04-14T08:33:12.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 >> v2:172.21.15.177:3300/0 conn(0x7fe5c00cd670 msgr2=0x7fe5c00b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7fe5c525a640 1 --2- 172.21.15.172:0/3391180520 >> v2:172.21.15.177:3300/0 conn(0x7fe5c00cd670 0x7fe5c00b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 >> v1:172.21.15.177:6789/0 conn(0x7fe5c00ccc20 legacy=0x7fe5c00c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c00b11b0 con 0x7fe5c00c39c0 2024-04-14T08:33:12.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f4475c20640 1 Processor -- start 2024-04-14T08:33:12.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7f523ec2d640 1 Processor -- start 2024-04-14T08:33:12.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7f523ec2d640 1 -- start start 2024-04-14T08:33:12.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7f45d45dc640 1 -- start start 2024-04-14T08:33:12.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fe5b0001670 con 0x7fe5c00c39c0 2024-04-14T08:33:12.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 >> v1:172.21.15.172:6789/0 conn(0x7fe5c00c39c0 legacy=0x7fe5c00ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7fe5c525a640 1 --2- 172.21.15.172:0/3391180520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ac003a00 0x7fe5ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7feeae7fc640 1 -- 172.21.15.172:0/2989939713 <== osd.4 v2:172.21.15.177:6820/4180733146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feeb001eec0 con 0x7fee9c076070 2024-04-14T08:33:12.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7f0a5bd95640 1 Processor -- start 2024-04-14T08:33:12.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.773+0000 7f0a5bd95640 1 -- start start 2024-04-14T08:33:12.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0a5bd95640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a541639a0 0x7f0a541669a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0a5bd95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a54163d80 0x7f0a54164e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0a5bd95640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a5416bd00 0x7f0a5416c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f8eb9636640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8eb40733b0 0x7f8eb40c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f8eb9636640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb40cff80 0x7f8eb415e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0579b1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c0051c0 0x7f057415d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0579b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0574143900 0x7f057413d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0a5bd95640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a541739d0 con 0x7f0a54163d80 2024-04-14T08:33:12.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0a5bd95640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0a54173850 con 0x7f0a5416bd00 2024-04-14T08:33:12.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0a5bd95640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0a54173b50 con 0x7f0a541639a0 2024-04-14T08:33:12.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0a5a332640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a5416bd00 0x7f0a5416c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0a5a332640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a5416bd00 0x7f0a5416c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33006/0 (socket says 172.21.15.172:33006) 2024-04-14T08:33:12.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0a5a332640 1 -- 172.21.15.172:0/700497502 learned_addr learned my addr 172.21.15.172:0/700497502 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7fe5c525a640 1 --2- 172.21.15.172:0/3391180520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5ac004800 0x7fe5ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f8eb9636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eb415b220 0x7f8eb414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f05737fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c0051c0 0x7f057415d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f05737fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c0051c0 0x7f057415d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33020/0 (socket says 172.21.15.172:33020) 2024-04-14T08:33:12.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f8eb9636640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4072450 con 0x7f8eb40cff80 2024-04-14T08:33:12.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f8eb9636640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb40722d0 con 0x7f8eb40733b0 2024-04-14T08:33:12.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0572ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0574143900 0x7f057413d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0572ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0574143900 0x7f057413d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:43086/0 (socket says 172.21.15.172:43086) 2024-04-14T08:33:12.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f8eb9636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb415b600 con 0x7f8eb415b220 2024-04-14T08:33:12.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f0579b1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f057413da60 0x7f0574161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7fe5c525a640 1 --2- 172.21.15.172:0/3391180520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5ac0052e0 0x7fe5ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c00b0ed0 con 0x7fe5ac003a00 2024-04-14T08:33:12.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac00c200 con 0x7fe5ac0052e0 2024-04-14T08:33:12.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe5ac00c400 con 0x7fe5ac004800 2024-04-14T08:33:12.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0a5a332640 1 -- 172.21.15.172:0/700497502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a541639a0 msgr2=0x7f0a541669a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-14T08:33:12.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0a5a332640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a541639a0 0x7f0a541669a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0a5a332640 1 -- 172.21.15.172:0/700497502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a54163d80 msgr2=0x7f0a54164e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-14T08:33:12.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f8eb27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb40cff80 0x7f8eb415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f8eb27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb40cff80 0x7f8eb415e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49452/0 (socket says 172.21.15.172:49452) 2024-04-14T08:33:12.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f8eb27fc640 1 -- 172.21.15.172:0/3287899999 learned_addr learned my addr 172.21.15.172:0/3287899999 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f8eb37fe640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eb415b220 0x7f8eb414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f523ec2d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f523000a460 0x7f52300d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f523ec2d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523000ab40 0x7f523004f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f523ec2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300e1310 0x7f52300e35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f523ec2d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52300548a0 con 0x7f523000a460 2024-04-14T08:33:12.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f523ec2d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f52300555b0 con 0x7f523000ab40 2024-04-14T08:33:12.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f523ec2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5230054a20 con 0x7f52300e1310 2024-04-14T08:33:12.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7fe5c5a5b640 1 --2- 172.21.15.172:0/3391180520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5ac004800 0x7fe5ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7fe5c6a5d640 1 --2- 172.21.15.172:0/3391180520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5ac0052e0 0x7fe5ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f4475c20640 1 -- start start 2024-04-14T08:33:12.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0a59330640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a54163d80 0x7f0a54164e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0a5a332640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a54163d80 0x7f0a54164e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0a5a332640 1 -- 172.21.15.172:0/700497502 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a541653d0 con 0x7f0a5416bd00 2024-04-14T08:33:12.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f5237fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523000ab40 0x7f523004f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f5237fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523000ab40 0x7f523004f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33056/0 (socket says 172.21.15.172:33056) 2024-04-14T08:33:12.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f5237fff640 1 -- 172.21.15.172:0/4007488927 learned_addr learned my addr 172.21.15.172:0/4007488927 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f523c9c9640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f523000a460 0x7f52300d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0a59330640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a54163d80 0x7f0a54164e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:33:12.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f8eb27fc640 1 -- 172.21.15.172:0/3287899999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eb415b220 msgr2=0x7f8eb414aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f8eb27fc640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eb415b220 0x7f8eb414aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f8eb27fc640 1 -- 172.21.15.172:0/3287899999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8eb40733b0 msgr2=0x7f8eb40c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f8eb27fc640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8eb40733b0 0x7f8eb40c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f8eb27fc640 1 -- 172.21.15.172:0/3287899999 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb40c6e30 con 0x7f8eb40cff80 2024-04-14T08:33:12.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f523d1ca640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300e1310 0x7f52300e35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0a5a332640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a5416bd00 0x7f0a5416c180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a54073760 tx=0x7f0a4c00ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f5237fff640 1 -- 172.21.15.172:0/4007488927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300e1310 msgr2=0x7f52300e35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f5237fff640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300e1310 0x7f52300e35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f5237fff640 1 -- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f523000a460 msgr2=0x7f52300d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f5237fff640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f523000a460 0x7f52300d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f5237fff640 1 -- 172.21.15.172:0/4007488927 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523004fdf0 con 0x7f523000ab40 2024-04-14T08:33:12.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7fe5c625c640 1 --2- 172.21.15.172:0/3391180520 >> v2:172.21.15.177:3300/0 conn(0x7fe5c00cd670 0x7fe5c00b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:33:12.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7fe5c6a5d640 1 -- 172.21.15.172:0/3391180520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5ac004800 msgr2=0x7fe5ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7fe5c6a5d640 1 --2- 172.21.15.172:0/3391180520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5ac004800 0x7fe5ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7fe5c6a5d640 1 -- 172.21.15.172:0/3391180520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ac003a00 msgr2=0x7fe5ac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:12.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7fe5c625c640 1 --2- 172.21.15.172:0/3391180520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ac003a00 0x7fe5ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7fe5c6a5d640 1 --2- 172.21.15.172:0/3391180520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ac003a00 0x7fe5ac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0a4a7fc640 1 -- 172.21.15.172:0/700497502 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c00a9c0 con 0x7f0a5416bd00 2024-04-14T08:33:12.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f8eb27fc640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb40cff80 0x7f8eb415e020 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8012dc0 tx=0x7f8ea800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7fe5c6a5d640 1 -- 172.21.15.172:0/3391180520 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ac003520 con 0x7fe5ac0052e0 2024-04-14T08:33:12.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f0a4a7fc640 1 -- 172.21.15.172:0/700497502 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4c00ab60 con 0x7f0a5416bd00 2024-04-14T08:33:12.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f45d45dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45cc0ceff0 0x7f45cc13e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f45d45dc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45cc143a40 0x7f45cc1f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.774+0000 7f45d45dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45cc15e590 0x7f45cc1f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f45d45dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc0b29b0 con 0x7f45cc0ceff0 2024-04-14T08:33:12.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f45d45dc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc0b2830 con 0x7f45cc143a40 2024-04-14T08:33:12.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f45d45dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f45cc0ce760 con 0x7f45cc15e590 2024-04-14T08:33:12.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f45d2378640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45cc0ceff0 0x7f45cc13e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f45d2b79640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45cc15e590 0x7f45cc1f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f45d2378640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45cc0ceff0 0x7f45cc13e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49484/0 (socket says 172.21.15.172:49484) 2024-04-14T08:33:12.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f45d2378640 1 -- 172.21.15.172:0/3628911808 learned_addr learned my addr 172.21.15.172:0/3628911808 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f5237fff640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523000ab40 0x7f523004f580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f522c0011e0 tx=0x7f522c00ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f0a5bd95640 1 -- 172.21.15.172:0/700497502 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a540ccba0 con 0x7f0a5416bd00 2024-04-14T08:33:12.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f0a5bd95640 1 -- 172.21.15.172:0/700497502 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a540cd160 con 0x7f0a5416bd00 2024-04-14T08:33:12.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f0a4a7fc640 1 -- 172.21.15.172:0/700497502 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c014420 con 0x7f0a5416bd00 2024-04-14T08:33:12.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f4475c20640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44700d0990 0x7f447015a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f4475c20640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700ccec0 0x7f44700cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f4475c20640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44700cdaa0 0x7f447015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f4475c20640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f44700d04b0 con 0x7f44700d0990 2024-04-14T08:33:12.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f4475c20640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f44700cf910 con 0x7f44700cdaa0 2024-04-14T08:33:12.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f4475c20640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f44700d0e20 con 0x7f44700ccec0 2024-04-14T08:33:12.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f52357fa640 1 -- 172.21.15.172:0/4007488927 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c005ce0 con 0x7f523000ab40 2024-04-14T08:33:12.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f523ec2d640 1 -- 172.21.15.172:0/4007488927 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52300500d0 con 0x7f523000ab40 2024-04-14T08:33:12.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f523ec2d640 1 -- 172.21.15.172:0/4007488927 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5230050690 con 0x7f523000ab40 2024-04-14T08:33:12.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0579b1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f05740b59a0 con 0x7f057413da60 2024-04-14T08:33:12.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0579b1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f057414c510 con 0x7f055c0051c0 2024-04-14T08:33:12.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0579b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f05740b5b20 con 0x7f0574143900 2024-04-14T08:33:12.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0573fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f057413da60 0x7f0574161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0573fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f057413da60 0x7f0574161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49476/0 (socket says 172.21.15.172:49476) 2024-04-14T08:33:12.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.775+0000 7f0573fff640 1 -- 172.21.15.172:0/2460656082 learned_addr learned my addr 172.21.15.172:0/2460656082 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7fe5c6a5d640 1 -- 172.21.15.172:0/3391180520 reap_dead start 2024-04-14T08:33:12.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7fe5c625c640 1 --2- 172.21.15.172:0/3391180520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ac003a00 0x7fe5ac003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:33:12.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f8e9b7fe640 1 -- 172.21.15.172:0/3287899999 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea801f070 con 0x7f8eb40cff80 2024-04-14T08:33:12.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f8e9b7fe640 1 -- 172.21.15.172:0/3287899999 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea801a630 con 0x7f8eb40cff80 2024-04-14T08:33:12.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f8e9b7fe640 1 -- 172.21.15.172:0/3287899999 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea801aa20 con 0x7f8eb40cff80 2024-04-14T08:33:12.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f45d2b79640 1 -- 172.21.15.172:0/3628911808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45cc143a40 msgr2=0x7f45cc1f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f45d2b79640 1 --2- 172.21.15.172:0/3628911808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45cc143a40 0x7f45cc1f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f45d2b79640 1 -- 172.21.15.172:0/3628911808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45cc0ceff0 msgr2=0x7f45cc13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f45d2b79640 1 --2- 172.21.15.172:0/3628911808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45cc0ceff0 0x7f45cc13e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f45d2b79640 1 -- 172.21.15.172:0/3628911808 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc1f54f0 con 0x7f45cc15e590 2024-04-14T08:33:12.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f8eb9636640 1 -- 172.21.15.172:0/3287899999 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb40c7970 con 0x7f8eb40cff80 2024-04-14T08:33:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f0572ffd640 1 -- 172.21.15.172:0/2460656082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c0051c0 msgr2=0x7f057415d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f0572ffd640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c0051c0 0x7f057415d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f0572ffd640 1 -- 172.21.15.172:0/2460656082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f057413da60 msgr2=0x7f0574161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f0572ffd640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f057413da60 0x7f0574161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f0572ffd640 1 -- 172.21.15.172:0/2460656082 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057415d710 con 0x7f0574143900 2024-04-14T08:33:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f45d2b79640 1 --2- 172.21.15.172:0/3628911808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45cc15e590 0x7f45cc1f4fb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0cff40 tx=0x7f45c800a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.776+0000 7f8eb37fe640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8eb415b220 0x7f8eb414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:33:12.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7f0572ffd640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0574143900 0x7f057413d520 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f056c009ec0 tx=0x7f056c0115a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7f8eb9636640 1 -- 172.21.15.172:0/3287899999 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb40c73b0 con 0x7f8eb40cff80 2024-04-14T08:33:12.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7f055bfff640 1 -- 172.21.15.172:0/2460656082 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c003990 con 0x7f0574143900 2024-04-14T08:33:12.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7f055bfff640 1 -- 172.21.15.172:0/2460656082 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056c01b070 con 0x7f0574143900 2024-04-14T08:33:12.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7f055bfff640 1 -- 172.21.15.172:0/2460656082 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c003b70 con 0x7f0574143900 2024-04-14T08:33:12.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7fe5c6a5d640 1 --2- 172.21.15.172:0/3391180520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5ac0052e0 0x7fe5ac00b9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00ca8c0 tx=0x7fe5ac005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6901a583b22dd677 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7f52357fa640 1 -- 172.21.15.172:0/4007488927 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522c007070 con 0x7f523000ab40 2024-04-14T08:33:12.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7f52357fa640 1 -- 172.21.15.172:0/4007488927 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c007420 con 0x7f523000ab40 2024-04-14T08:33:12.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7f0579b1e640 1 -- 172.21.15.172:0/2460656082 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0574161aa0 con 0x7f0574143900 2024-04-14T08:33:12.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7f0579b1e640 1 -- 172.21.15.172:0/2460656082 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0574162060 con 0x7f0574143900 2024-04-14T08:33:12.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.778+0000 7f523ec2d640 1 -- 172.21.15.172:0/4007488927 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52300057e0 con 0x7f523000ab40 2024-04-14T08:33:12.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.778+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0004b60 con 0x7fe5ac0052e0 2024-04-14T08:33:12.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.778+0000 7fe5c525a640 1 -- 172.21.15.172:0/3391180520 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b0005ad0 con 0x7fe5ac0052e0 2024-04-14T08:33:12.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.778+0000 7f8eb9636640 1 -- 172.21.15.172:0/3287899999 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8eb4068590 con 0x7f8eb40cff80 2024-04-14T08:33:12.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7f45c2ffd640 1 -- 172.21.15.172:0/3628911808 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8019070 con 0x7f45cc15e590 2024-04-14T08:33:12.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7f45d45dc640 1 -- 172.21.15.172:0/3628911808 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc1f57a0 con 0x7f45cc15e590 2024-04-14T08:33:12.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.777+0000 7f45d45dc640 1 -- 172.21.15.172:0/3628911808 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc1f5d60 con 0x7f45cc15e590 2024-04-14T08:33:12.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f45c0ff9640 1 -- 172.21.15.172:0/3628911808 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45cc068590 con 0x7f45cc15e590 2024-04-14T08:33:12.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.778+0000 7f446ffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44700cdaa0 0x7f447015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f446ffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44700cdaa0 0x7f447015f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33066/0 (socket says 172.21.15.172:33066) 2024-04-14T08:33:12.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f446ffff640 1 -- 172.21.15.172:0/3325519195 learned_addr learned my addr 172.21.15.172:0/3325519195 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f446effd640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700ccec0 0x7f44700cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f446f7fe640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44700d0990 0x7f447015a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f446ffff640 1 -- 172.21.15.172:0/3325519195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700ccec0 msgr2=0x7f44700cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f446ffff640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44700ccec0 0x7f44700cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f446ffff640 1 -- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44700d0990 msgr2=0x7f447015a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f446ffff640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44700d0990 0x7f447015a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f446ffff640 1 -- 172.21.15.172:0/3325519195 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44700cd480 con 0x7f44700cdaa0 2024-04-14T08:33:12.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f446ffff640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44700cdaa0 0x7f447015f9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f44680045e0 tx=0x7f4468007700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f52357fa640 1 -- 172.21.15.172:0/4007488927 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f522c0075c0 con 0x7f523000ab40 2024-04-14T08:33:12.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f52357fa640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f521c06c9f0 0x7f521c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f05737fe640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c0051c0 0x7f057415d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-14T08:33:12.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3391180520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5ac0052e0 msgr2=0x7fe5ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7fe5c84c0640 1 --2- 172.21.15.172:0/3391180520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5ac0052e0 0x7fe5ac00b9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00ca8c0 tx=0x7fe5ac005830 comp rx=0 tx=0).stop 2024-04-14T08:33:12.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f0a5bd95640 1 -- 172.21.15.172:0/700497502 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a540cc910 con 0x7f0a5416bd00 2024-04-14T08:33:12.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f0a4a7fc640 1 -- 172.21.15.172:0/700497502 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a4c011070 con 0x7f0a5416bd00 2024-04-14T08:33:12.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f45c2ffd640 1 -- 172.21.15.172:0/3628911808 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c8014410 con 0x7f45cc15e590 2024-04-14T08:33:12.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.779+0000 7f45c2ffd640 1 -- 172.21.15.172:0/3628911808 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8014820 con 0x7f45cc15e590 2024-04-14T08:33:12.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f45c2ffd640 1 -- 172.21.15.172:0/3628911808 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45c801c070 con 0x7f45cc15e590 2024-04-14T08:33:12.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f4457fff640 1 -- 172.21.15.172:0/3325519195 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44680046d0 con 0x7f44700cdaa0 2024-04-14T08:33:12.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f4457fff640 1 -- 172.21.15.172:0/3325519195 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446801a070 con 0x7f44700cdaa0 2024-04-14T08:33:12.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f4475c20640 1 -- 172.21.15.172:0/3325519195 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44700cb480 con 0x7f44700cdaa0 2024-04-14T08:33:12.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f4457fff640 1 -- 172.21.15.172:0/3325519195 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468015640 con 0x7f44700cdaa0 2024-04-14T08:33:12.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f0a4a7fc640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a3806c9f0 0x7f0a3806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f4475c20640 1 -- 172.21.15.172:0/3325519195 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44700cba40 con 0x7f44700cdaa0 2024-04-14T08:33:12.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f0a4a7fc640 1 -- 172.21.15.172:0/700497502 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0a4c08bd00 con 0x7f0a5416bd00 2024-04-14T08:33:12.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3391180520 shutdown_connections 2024-04-14T08:33:12.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7fe5c84c0640 1 --2- 172.21.15.172:0/3391180520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5ac0052e0 0x7fe5ac00b9c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3391180520 >> 172.21.15.172:0/3391180520 conn(0x7fe5c0072b40 msgr2=0x7fe5c0072f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f0a4a7fc640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x7f0a38076170 0x7f0a380785b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f8e9b7fe640 1 -- 172.21.15.172:0/3287899999 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ea8022070 con 0x7f8eb40cff80 2024-04-14T08:33:12.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f0a4a7fc640 1 -- 172.21.15.172:0/700497502 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a38078c80 con 0x7f0a38076170 2024-04-14T08:33:12.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f0a4a7fc640 1 -- 172.21.15.172:0/700497502 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0a4c08e360 con 0x7f0a5416bd00 2024-04-14T08:33:12.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f055bfff640 1 -- 172.21.15.172:0/2460656082 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f056c005d10 con 0x7f0574143900 2024-04-14T08:33:12.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f055bfff640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f056006c9c0 0x7f056006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f055bfff640 1 -- 172.21.15.172:0/2460656082 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f056c08c440 con 0x7f0574143900 2024-04-14T08:33:12.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f0559ffb640 1 -- 172.21.15.172:0/2460656082 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0574068590 con 0x7f0574143900 2024-04-14T08:33:12.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f4457fff640 1 -- 172.21.15.172:0/3325519195 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44680157e0 con 0x7f44700cdaa0 2024-04-14T08:33:12.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.780+0000 7f52357fa640 1 -- 172.21.15.172:0/4007488927 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f522c08bc80 con 0x7f523000ab40 2024-04-14T08:33:12.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f52357fa640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x7f521c076170 0x7f521c0785b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f52357fa640 1 -- 172.21.15.172:0/4007488927 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f521c078c80 con 0x7f521c076170 2024-04-14T08:33:12.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f52357fa640 1 -- 172.21.15.172:0/4007488927 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f522c08e2e0 con 0x7f523000ab40 2024-04-14T08:33:12.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f523d1ca640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x7f521c076170 0x7f521c0785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f523c9c9640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f521c06c9f0 0x7f521c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f523d1ca640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x7f521c076170 0x7f521c0785b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3391180520 shutdown_connections 2024-04-14T08:33:12.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f8e9b7fe640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8ea006c9c0 0x7f8ea006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f8e9b7fe640 1 -- 172.21.15.172:0/3287899999 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8ea808fd90 con 0x7f8eb40cff80 2024-04-14T08:33:12.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f8e9b7fe640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x7f8ea00760c0 0x7f8ea0078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f8e9b7fe640 1 -- 172.21.15.172:0/3287899999 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ea0078bd0 con 0x7f8ea00760c0 2024-04-14T08:33:12.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f8e9b7fe640 1 -- 172.21.15.172:0/3287899999 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8ea8059a90 con 0x7f8eb40cff80 2024-04-14T08:33:12.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f8eb37fe640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x7f8ea00760c0 0x7f8ea0078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f45c2ffd640 1 --2- 172.21.15.172:0/3628911808 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f45b006c9c0 0x7f45b006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f45c2ffd640 1 -- 172.21.15.172:0/3628911808 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f45c808bd10 con 0x7f45cc15e590 2024-04-14T08:33:12.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f45d2378640 1 --2- 172.21.15.172:0/3628911808 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f45b006c9c0 0x7f45b006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f45c2ffd640 1 --2- 172.21.15.172:0/3628911808 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x7f45b0076140 0x7f45b0078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f45c2ffd640 1 -- 172.21.15.172:0/3628911808 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45c8006bf0 con 0x7f45b0076140 2024-04-14T08:33:12.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f45c2ffd640 1 -- 172.21.15.172:0/3628911808 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45c808e370 con 0x7f45cc15e590 2024-04-14T08:33:12.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f4475c20640 1 -- 172.21.15.172:0/3325519195 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4450001030 con 0x7f44700cdaa0 2024-04-14T08:33:12.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f45d2378640 1 --2- 172.21.15.172:0/3628911808 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f45b006c9c0 0x7f45b006ee80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45bc005020 tx=0x7f45bc007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f45d1b77640 1 --2- 172.21.15.172:0/3628911808 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x7f45b0076140 0x7f45b0078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f055bfff640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x7f05600760b0 0x7f05600784f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f055bfff640 1 -- 172.21.15.172:0/2460656082 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f056c018e70 con 0x7f05600760b0 2024-04-14T08:33:12.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.781+0000 7f055bfff640 1 -- 172.21.15.172:0/2460656082 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f056c056140 con 0x7f0574143900 2024-04-14T08:33:12.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f05737fe640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f056006c9c0 0x7f056006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f4457fff640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f445c06c920 0x7f445c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f0a59b31640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a3806c9f0 0x7f0a3806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f4457fff640 1 -- 172.21.15.172:0/3325519195 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f446808c450 con 0x7f44700cdaa0 2024-04-14T08:33:12.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f8eb2ffd640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8ea006c9c0 0x7f8ea006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f4457fff640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x7f445c076020 0x7f445c078460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f4457fff640 1 -- 172.21.15.172:0/3325519195 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f446800fea0 con 0x7f445c076020 2024-04-14T08:33:12.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f4457fff640 1 -- 172.21.15.172:0/3325519195 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4468093050 con 0x7f44700cdaa0 2024-04-14T08:33:12.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7f0573fff640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x7f05600760b0 0x7f05600784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f8eb37fe640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x7f8ea00760c0 0x7f8ea0078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f446f7fe640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f445c06c920 0x7f445c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f0573fff640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x7f05600760b0 0x7f05600784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f0a59330640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x7f0a38076170 0x7f0a380785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.782+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3391180520 wait complete. 2024-04-14T08:33:12.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f45d1b77640 1 --2- 172.21.15.172:0/3628911808 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x7f45b0076140 0x7f45b0078580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f523c9c9640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f521c06c9f0 0x7f521c06eeb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5238069c30 tx=0x7f5238073e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f446effd640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x7f445c076020 0x7f445c078460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f52357fa640 1 -- 172.21.15.172:0/4007488927 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f521c078c80 con 0x7f521c076170 2024-04-14T08:33:12.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f05737fe640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f056006c9c0 0x7f056006ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0568000fb0 tx=0x7f0568001cb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.783+0000 7f0a59b31640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a3806c9f0 0x7f0a3806eeb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a50036000 tx=0x7f0a50030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.784+0000 7f0a59330640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x7f0a38076170 0x7f0a380785b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.784+0000 7f446f7fe640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f445c06c920 0x7f445c06ede0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4460034000 tx=0x7f4460009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.784+0000 7fe5c84c0640 1 Processor -- start 2024-04-14T08:33:12.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.784+0000 7f8eb2ffd640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8ea006c9c0 0x7f8ea006ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8eac001c70 tx=0x7f8eac007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.784+0000 7f8e9b7fe640 1 -- 172.21.15.172:0/3287899999 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ea0078bd0 con 0x7f8ea00760c0 2024-04-14T08:33:12.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.784+0000 7f446effd640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x7f445c076020 0x7f445c078460 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.784+0000 7fe5c84c0640 1 -- start start 2024-04-14T08:33:12.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.784+0000 7f055bfff640 1 -- 172.21.15.172:0/2460656082 <== osd.2 v2:172.21.15.172:6816/1609902500 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f056c018e70 con 0x7f05600760b0 2024-04-14T08:33:12.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.784+0000 7f45c2ffd640 1 -- 172.21.15.172:0/3628911808 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45c8006bf0 con 0x7f45b0076140 2024-04-14T08:33:12.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.785+0000 7f4457fff640 1 -- 172.21.15.172:0/3325519195 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f446800fea0 con 0x7f445c076020 2024-04-14T08:33:12.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.785+0000 7f0a4a7fc640 1 -- 172.21.15.172:0/700497502 <== osd.5 v2:172.21.15.177:6816/3172735715 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a38078c80 con 0x7f0a38076170 2024-04-14T08:33:12.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.785+0000 7fe5c84c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ac0052e0 0x7fe5c0158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.785+0000 7fe5c84c0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c0160c80 0x7fe5c0161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.785+0000 7fe5c84c0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c00c7d70 0x7fe5c00c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.785+0000 7fe5c625c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ac0052e0 0x7fe5c0158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.785+0000 7fe5c84c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c00cdbd0 con 0x7fe5ac0052e0 2024-04-14T08:33:12.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.785+0000 7fe5c84c0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c00cda50 con 0x7fe5c0160c80 2024-04-14T08:33:12.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.785+0000 7fe5c84c0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c00c32b0 con 0x7fe5c00c7d70 2024-04-14T08:33:12.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.785+0000 7fe5c625c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ac0052e0 0x7fe5c0158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49520/0 (socket says 172.21.15.172:49520) 2024-04-14T08:33:12.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.785+0000 7fe5c625c640 1 -- 172.21.15.172:0/3171817908 learned_addr learned my addr 172.21.15.172:0/3171817908 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.785+0000 7fe5c6a5d640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c00c7d70 0x7fe5c00c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.786+0000 7fe5c5a5b640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c0160c80 0x7fe5c0161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.786+0000 7fe5c625c640 1 -- 172.21.15.172:0/3171817908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c00c7d70 msgr2=0x7fe5c00c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.786+0000 7fe5c625c640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c00c7d70 0x7fe5c00c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.786+0000 7fe5c625c640 1 -- 172.21.15.172:0/3171817908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c0160c80 msgr2=0x7fe5c0161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.786+0000 7fe5c625c640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c0160c80 0x7fe5c0161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.786+0000 7fe5c625c640 1 -- 172.21.15.172:0/3171817908 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c00c8a90 con 0x7fe5ac0052e0 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.786+0000 7fe5c6a5d640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c00c7d70 0x7fe5c00c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.786+0000 7fe5c5a5b640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c0160c80 0x7fe5c0161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.787+0000 7fe5c625c640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ac0052e0 0x7fe5c0158640 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800de50 tx=0x7fe5b800b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.787+0000 7fe5b6ffd640 1 -- 172.21.15.172:0/3171817908 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8003990 con 0x7fe5ac0052e0 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.787+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3171817908 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c00c8d70 con 0x7fe5ac0052e0 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.790+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3171817908 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c00c9330 con 0x7fe5ac0052e0 2024-04-14T08:33:12.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.790+0000 7fe5b6ffd640 1 -- 172.21.15.172:0/3171817908 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b8003b30 con 0x7fe5ac0052e0 2024-04-14T08:33:12.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.791+0000 7fe5b6ffd640 1 -- 172.21.15.172:0/3171817908 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8004020 con 0x7fe5ac0052e0 2024-04-14T08:33:12.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.791+0000 7fe5b6ffd640 1 -- 172.21.15.172:0/3171817908 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5b8004280 con 0x7fe5ac0052e0 2024-04-14T08:33:12.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.791+0000 7fe5b6ffd640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe5a406c9f0 0x7fe5a406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.791+0000 7fe5b6ffd640 1 -- 172.21.15.172:0/3171817908 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe5b808c360 con 0x7fe5ac0052e0 2024-04-14T08:33:12.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.792+0000 7fe5c5a5b640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe5a406c9f0 0x7fe5a406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.793+0000 7fe5c5a5b640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe5a406c9f0 0x7fe5a406eeb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00c6b10 tx=0x7fe5bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.794+0000 7fe5c84c0640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x7fe5a8001650 0x7fe5a8003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:12.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.794+0000 7fe5c6a5d640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x7fe5a8001650 0x7fe5a8003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:12.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.794+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3171817908 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5a8006bb0 con 0x7fe5a8001650 2024-04-14T08:33:12.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.795+0000 7fe5c6a5d640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x7fe5a8001650 0x7fe5a8003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:12.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.796+0000 7fe5b6ffd640 1 -- 172.21.15.172:0/3171817908 <== osd.3 v2:172.21.15.172:6808/2305139122 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe5a8006bb0 con 0x7fe5a8001650 2024-04-14T08:33:12.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.803+0000 7feebff1e640 1 -- 172.21.15.172:0/2989939713 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feeb80ccc90 con 0x7fee9c076070 2024-04-14T08:33:12.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.804+0000 7feeae7fc640 1 -- 172.21.15.172:0/2989939713 <== osd.4 v2:172.21.15.177:6820/4180733146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feeb80ccc90 con 0x7fee9c076070 2024-04-14T08:33:12.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.804+0000 7fee9bfff640 1 -- 172.21.15.172:0/2989939713 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x7fee9c076070 msgr2=0x7fee9c0784b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.805+0000 7fee9bfff640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x7fee9c076070 0x7fee9c0784b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.805+0000 7fee9bfff640 1 -- 172.21.15.172:0/2989939713 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fee9c06c8f0 msgr2=0x7fee9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.805+0000 7fee9bfff640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fee9c06c8f0 0x7fee9c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feeb4036000 tx=0x7feeb4008030 comp rx=0 tx=0).stop 2024-04-14T08:33:12.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.805+0000 7fee9bfff640 1 -- 172.21.15.172:0/2989939713 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feeb80cadd0 msgr2=0x7feeb815de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.805+0000 7fee9bfff640 1 --2- 172.21.15.172:0/2989939713 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feeb80cadd0 0x7feeb815de50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feeb000c760 tx=0x7feeb000cb60 comp rx=0 tx=0).stop 2024-04-14T08:33:12.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.805+0000 7feebe4bb640 1 -- 172.21.15.172:0/2989939713 reap_dead start 2024-04-14T08:33:12.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.805+0000 7fee9bfff640 1 -- 172.21.15.172:0/2989939713 shutdown_connections 2024-04-14T08:33:12.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.805+0000 7fee9bfff640 1 -- 172.21.15.172:0/2989939713 >> 172.21.15.172:0/2989939713 conn(0x7feeb80d1ee0 msgr2=0x7feeb80c4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.805+0000 7fee9bfff640 1 -- 172.21.15.172:0/2989939713 shutdown_connections 2024-04-14T08:33:12.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.806+0000 7fee9bfff640 1 -- 172.21.15.172:0/2989939713 wait complete. 2024-04-14T08:33:12.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.817+0000 7f0559ffb640 1 -- 172.21.15.172:0/2460656082 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f057413e150 con 0x7f05600760b0 2024-04-14T08:33:12.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.817+0000 7f055bfff640 1 -- 172.21.15.172:0/2460656082 <== osd.2 v2:172.21.15.172:6816/1609902500 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f057413e150 con 0x7f05600760b0 2024-04-14T08:33:12.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.818+0000 7f0a5bd95640 1 -- 172.21.15.172:0/700497502 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a54068590 con 0x7f0a38076170 2024-04-14T08:33:12.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.818+0000 7f523ec2d640 1 -- 172.21.15.172:0/4007488927 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f52300578f0 con 0x7f521c076170 2024-04-14T08:33:12.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.818+0000 7f0579b1e640 1 -- 172.21.15.172:0/2460656082 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x7f05600760b0 msgr2=0x7f05600784f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.818+0000 7f0579b1e640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x7f05600760b0 0x7f05600784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.818+0000 7f45c0ff9640 1 -- 172.21.15.172:0/3628911808 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f45cc073590 con 0x7f45b0076140 2024-04-14T08:33:12.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.818+0000 7f0579b1e640 1 -- 172.21.15.172:0/2460656082 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f056006c9c0 msgr2=0x7f056006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.818+0000 7f0579b1e640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f056006c9c0 0x7f056006ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0568000fb0 tx=0x7f0568001cb0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.818+0000 7f0a4a7fc640 1 -- 172.21.15.172:0/700497502 <== osd.5 v2:172.21.15.177:6816/3172735715 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0a54068590 con 0x7f0a38076170 2024-04-14T08:33:12.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.818+0000 7f0579b1e640 1 -- 172.21.15.172:0/2460656082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0574143900 msgr2=0x7f057413d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.818+0000 7f0579b1e640 1 --2- 172.21.15.172:0/2460656082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0574143900 0x7f057413d520 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f056c009ec0 tx=0x7f056c0115a0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.818+0000 7f0573fff640 1 -- 172.21.15.172:0/2460656082 reap_dead start 2024-04-14T08:33:12.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0579b1e640 1 -- 172.21.15.172:0/2460656082 shutdown_connections 2024-04-14T08:33:12.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0579b1e640 1 -- 172.21.15.172:0/2460656082 >> 172.21.15.172:0/2460656082 conn(0x7f057406b520 msgr2=0x7f05740cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f52357fa640 1 -- 172.21.15.172:0/4007488927 <== osd.1 v2:172.21.15.172:6823/1219922421 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f52300578f0 con 0x7f521c076170 2024-04-14T08:33:12.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f4475c20640 1 -- 172.21.15.172:0/3325519195 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4450002dd0 con 0x7f445c076020 2024-04-14T08:33:12.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0579b1e640 1 -- 172.21.15.172:0/2460656082 shutdown_connections 2024-04-14T08:33:12.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0a5bd95640 1 -- 172.21.15.172:0/700497502 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x7f0a38076170 msgr2=0x7f0a380785b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0a5bd95640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x7f0a38076170 0x7f0a380785b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0a5bd95640 1 -- 172.21.15.172:0/700497502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a3806c9f0 msgr2=0x7f0a3806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0a5bd95640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a3806c9f0 0x7f0a3806eeb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a50036000 tx=0x7f0a50030000 comp rx=0 tx=0).stop 2024-04-14T08:33:12.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0a5bd95640 1 -- 172.21.15.172:0/700497502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a5416bd00 msgr2=0x7f0a5416c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0a5bd95640 1 --2- 172.21.15.172:0/700497502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a5416bd00 0x7f0a5416c180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a54073760 tx=0x7f0a4c00ccd0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0a5a332640 1 -- 172.21.15.172:0/700497502 reap_dead start 2024-04-14T08:33:12.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0a5bd95640 1 -- 172.21.15.172:0/700497502 shutdown_connections 2024-04-14T08:33:12.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0a5bd95640 1 -- 172.21.15.172:0/700497502 >> 172.21.15.172:0/700497502 conn(0x7f0a540b1d40 msgr2=0x7f0a540ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0579b1e640 1 -- 172.21.15.172:0/2460656082 wait complete. 2024-04-14T08:33:12.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0a5bd95640 1 -- 172.21.15.172:0/700497502 shutdown_connections 2024-04-14T08:33:12.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f523ec2d640 1 -- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x7f521c076170 msgr2=0x7f521c0785b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f523ec2d640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x7f521c076170 0x7f521c0785b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f45c2ffd640 1 -- 172.21.15.172:0/3628911808 <== osd.7 v2:172.21.15.177:6808/3609985207 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f45cc073590 con 0x7f45b0076140 2024-04-14T08:33:12.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f4457fff640 1 -- 172.21.15.172:0/3325519195 <== osd.0 v2:172.21.15.172:6800/3761711279 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4450002dd0 con 0x7f445c076020 2024-04-14T08:33:12.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.819+0000 7f0a5bd95640 1 -- 172.21.15.172:0/700497502 wait complete. 2024-04-14T08:33:12.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f45c0ff9640 1 -- 172.21.15.172:0/3628911808 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x7f45b0076140 msgr2=0x7f45b0078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f45c0ff9640 1 --2- 172.21.15.172:0/3628911808 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x7f45b0076140 0x7f45b0078580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f523ec2d640 1 -- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f521c06c9f0 msgr2=0x7f521c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f523ec2d640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f521c06c9f0 0x7f521c06eeb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5238069c30 tx=0x7f5238073e80 comp rx=0 tx=0).stop 2024-04-14T08:33:12.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f523ec2d640 1 -- 172.21.15.172:0/4007488927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523000ab40 msgr2=0x7f523004f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f523ec2d640 1 --2- 172.21.15.172:0/4007488927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f523000ab40 0x7f523004f580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f522c0011e0 tx=0x7f522c00ca10 comp rx=0 tx=0).stop 2024-04-14T08:33:12.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f45c0ff9640 1 -- 172.21.15.172:0/3628911808 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f45b006c9c0 msgr2=0x7f45b006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f45c0ff9640 1 --2- 172.21.15.172:0/3628911808 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f45b006c9c0 0x7f45b006ee80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45bc005020 tx=0x7f45bc007ef0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f45c0ff9640 1 -- 172.21.15.172:0/3628911808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45cc15e590 msgr2=0x7f45cc1f4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f45c0ff9640 1 --2- 172.21.15.172:0/3628911808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45cc15e590 0x7f45cc1f4fb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0cff40 tx=0x7f45c800a7a0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f45d2b79640 1 -- 172.21.15.172:0/3628911808 reap_dead start 2024-04-14T08:33:12.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f45c0ff9640 1 -- 172.21.15.172:0/3628911808 shutdown_connections 2024-04-14T08:33:12.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f45c0ff9640 1 -- 172.21.15.172:0/3628911808 >> 172.21.15.172:0/3628911808 conn(0x7f45cc0b1d40 msgr2=0x7f45cc06e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f523d1ca640 1 -- 172.21.15.172:0/4007488927 reap_dead start 2024-04-14T08:33:12.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f4475c20640 1 -- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x7f445c076020 msgr2=0x7f445c078460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f523ec2d640 1 -- 172.21.15.172:0/4007488927 shutdown_connections 2024-04-14T08:33:12.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f523ec2d640 1 -- 172.21.15.172:0/4007488927 >> 172.21.15.172:0/4007488927 conn(0x7f5230056cd0 msgr2=0x7f5230051630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f4475c20640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x7f445c076020 0x7f445c078460 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f45c0ff9640 1 -- 172.21.15.172:0/3628911808 shutdown_connections 2024-04-14T08:33:12.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f45c0ff9640 1 -- 172.21.15.172:0/3628911808 wait complete. 2024-04-14T08:33:12.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.820+0000 7f523ec2d640 1 -- 172.21.15.172:0/4007488927 shutdown_connections 2024-04-14T08:33:12.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.821+0000 7f4475c20640 1 -- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f445c06c920 msgr2=0x7f445c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.821+0000 7f4475c20640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f445c06c920 0x7f445c06ede0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4460034000 tx=0x7f4460009030 comp rx=0 tx=0).stop 2024-04-14T08:33:12.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.821+0000 7f4475c20640 1 -- 172.21.15.172:0/3325519195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44700cdaa0 msgr2=0x7f447015f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.821+0000 7f4475c20640 1 --2- 172.21.15.172:0/3325519195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44700cdaa0 0x7f447015f9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f44680045e0 tx=0x7f4468007700 comp rx=0 tx=0).stop 2024-04-14T08:33:12.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.821+0000 7f8eb9636640 1 -- 172.21.15.172:0/3287899999 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8eb4073120 con 0x7f8ea00760c0 2024-04-14T08:33:12.870 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:12.871 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-14T08:33:12.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.821+0000 7f523ec2d640 1 -- 172.21.15.172:0/4007488927 wait complete. 2024-04-14T08:33:12.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.821+0000 7f446ffff640 1 -- 172.21.15.172:0/3325519195 reap_dead start 2024-04-14T08:33:12.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.821+0000 7f4475c20640 1 -- 172.21.15.172:0/3325519195 shutdown_connections 2024-04-14T08:33:12.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.821+0000 7f4475c20640 1 -- 172.21.15.172:0/3325519195 >> 172.21.15.172:0/3325519195 conn(0x7f447009d570 msgr2=0x7f4470074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.821+0000 7f4475c20640 1 -- 172.21.15.172:0/3325519195 shutdown_connections 2024-04-14T08:33:12.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.821+0000 7f4475c20640 1 -- 172.21.15.172:0/3325519195 wait complete. 2024-04-14T08:33:12.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.821+0000 7f8e9b7fe640 1 -- 172.21.15.172:0/3287899999 <== osd.6 v2:172.21.15.177:6800/2961907928 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8eb4073120 con 0x7f8ea00760c0 2024-04-14T08:33:12.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.822+0000 7f8eb9636640 1 -- 172.21.15.172:0/3287899999 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x7f8ea00760c0 msgr2=0x7f8ea0078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.822+0000 7f8eb9636640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x7f8ea00760c0 0x7f8ea0078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.822+0000 7f8eb9636640 1 -- 172.21.15.172:0/3287899999 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8ea006c9c0 msgr2=0x7f8ea006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.822+0000 7f8eb9636640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8ea006c9c0 0x7f8ea006ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8eac001c70 tx=0x7f8eac007ca0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.823+0000 7f8eb9636640 1 -- 172.21.15.172:0/3287899999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb40cff80 msgr2=0x7f8eb415e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.823+0000 7f8eb9636640 1 --2- 172.21.15.172:0/3287899999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb40cff80 0x7f8eb415e020 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8012dc0 tx=0x7f8ea800c520 comp rx=0 tx=0).stop 2024-04-14T08:33:12.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.823+0000 7f8eb37fe640 1 -- 172.21.15.172:0/3287899999 reap_dead start 2024-04-14T08:33:12.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.823+0000 7f8eb9636640 1 -- 172.21.15.172:0/3287899999 shutdown_connections 2024-04-14T08:33:12.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.823+0000 7f8eb9636640 1 -- 172.21.15.172:0/3287899999 >> 172.21.15.172:0/3287899999 conn(0x7f8eb40cda00 msgr2=0x7f8eb406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.824+0000 7f8eb9636640 1 -- 172.21.15.172:0/3287899999 shutdown_connections 2024-04-14T08:33:12.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.824+0000 7f8eb9636640 1 -- 172.21.15.172:0/3287899999 wait complete. 2024-04-14T08:33:12.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.827+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3171817908 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe5a8005c40 con 0x7fe5a8001650 2024-04-14T08:33:12.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.828+0000 7fe5b6ffd640 1 -- 172.21.15.172:0/3171817908 <== osd.3 v2:172.21.15.172:6808/2305139122 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe5a8005c40 con 0x7fe5a8001650 2024-04-14T08:33:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.828+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x7fe5a8001650 msgr2=0x7fe5a8003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.828+0000 7fe5c84c0640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x7fe5a8001650 0x7fe5a8003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.829+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe5a406c9f0 msgr2=0x7fe5a406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.829+0000 7fe5c84c0640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe5a406c9f0 0x7fe5a406eeb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00c6b10 tx=0x7fe5bc009040 comp rx=0 tx=0).stop 2024-04-14T08:33:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.829+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ac0052e0 msgr2=0x7fe5c0158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.829+0000 7fe5c84c0640 1 --2- 172.21.15.172:0/3171817908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ac0052e0 0x7fe5c0158640 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800de50 tx=0x7fe5b800b7a0 comp rx=0 tx=0).stop 2024-04-14T08:33:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.829+0000 7fe5c6a5d640 1 -- 172.21.15.172:0/3171817908 reap_dead start 2024-04-14T08:33:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.829+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3171817908 shutdown_connections 2024-04-14T08:33:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.829+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3171817908 >> 172.21.15.172:0/3171817908 conn(0x7fe5c0072b40 msgr2=0x7fe5c006d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.830+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3171817908 shutdown_connections 2024-04-14T08:33:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:12.830+0000 7fe5c84c0640 1 -- 172.21.15.172:0/3171817908 wait complete. 2024-04-14T08:33:12.876 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:12.876 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:12.877 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-14T08:33:12.877 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-14T08:33:12.878 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:12.878 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-14T08:33:12.879 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:12.879 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-14T08:33:12.880 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:12.880 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-14T08:33:12.881 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:12.882 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-14T08:33:12.882 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:12.883 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-14T08:33:13.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.087+0000 7f5786bcb640 1 Processor -- start 2024-04-14T08:33:13.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.088+0000 7f063b17b640 1 Processor -- start 2024-04-14T08:33:13.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.088+0000 7f5786bcb640 1 -- start start 2024-04-14T08:33:13.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.088+0000 7f063b17b640 1 -- start start 2024-04-14T08:33:13.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.088+0000 7fa21ba36640 1 Processor -- start 2024-04-14T08:33:13.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f2a60d5e640 1 Processor -- start 2024-04-14T08:33:13.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f6d4a95e640 1 Processor -- start 2024-04-14T08:33:13.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f5786bcb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f57780e30b0 0x7f5778057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7fa21ba36640 1 -- start start 2024-04-14T08:33:13.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f1ee2ea2640 1 Processor -- start 2024-04-14T08:33:13.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7f2a60d5e640 1 -- start start 2024-04-14T08:33:13.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f6d4a95e640 1 -- start start 2024-04-14T08:33:13.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f5786bcb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f577803e830 0x7f5778053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f5786bcb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57780582c0 con 0x7f5778054120 2024-04-14T08:33:13.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f5786bcb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f5778054550 con 0x7f577803e830 2024-04-14T08:33:13.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f5786bcb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5778058560 con 0x7f57780e30b0 2024-04-14T08:33:13.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f5785bc9640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f57780e30b0 0x7f5778057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f5785bc9640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f57780e30b0 0x7f5778057d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33090/0 (socket says 172.21.15.172:33090) 2024-04-14T08:33:13.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f5785bc9640 1 -- 172.21.15.172:0/642013834 learned_addr learned my addr 172.21.15.172:0/642013834 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f57863ca640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f5778054120 0x7f57780e6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42750/0 (socket says 172.21.15.172:42750) 2024-04-14T08:33:13.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f063b17b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f062c04fc00 0x7f062c051760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f063b17b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062c008180 0x7f062c04ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f063b17b640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f062c048510 con 0x7f062c050520 2024-04-14T08:33:13.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f063b17b640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f062c0487b0 con 0x7f062c04fc00 2024-04-14T08:33:13.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f063b17b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f062c048a70 con 0x7f062c008180 2024-04-14T08:33:13.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f0633fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062c008180 0x7f062c04ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f0633fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062c008180 0x7f062c04ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:43156/0 (socket says 172.21.15.172:43156) 2024-04-14T08:33:13.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7f0633fff640 1 -- 172.21.15.172:0/3634723858 learned_addr learned my addr 172.21.15.172:0/3634723858 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f1ee2ea2640 1 -- start start 2024-04-14T08:33:13.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.089+0000 7f5f1fc1c640 1 Processor -- start 2024-04-14T08:33:13.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7f6d4a95e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6d4415e3f0 0x7f6d440aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7f1ee2ea2640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1edc0b23e0 0x7f1edc0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7fa21ba36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa21406e200 0x7fa2140ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7fa21ba36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa2141662d0 0x7fa21406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7fa21ba36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa214162990 0x7fa214071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7fa21ba36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa21406eb20 con 0x7fa21406e200 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7fa21ba36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa214166760 con 0x7fa2141662d0 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7fa21ba36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa214166a40 con 0x7fa214162990 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7fa2197d2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa21406e200 0x7fa2140ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7fa2197d2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa21406e200 0x7fa2140ad7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49546/0 (socket says 172.21.15.172:49546) 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7fa2197d2640 1 -- 172.21.15.172:0/2702443805 learned_addr learned my addr 172.21.15.172:0/2702443805 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7fa219fd3640 1 --2- 172.21.15.172:0/2702443805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa214162990 0x7fa214071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7fa218fd1640 1 --2- 172.21.15.172:0/2702443805 >> v2:172.21.15.177:3300/0 conn(0x7fa2141662d0 0x7fa21406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7fa219fd3640 1 -- 172.21.15.172:0/2702443805 >> v2:172.21.15.177:3300/0 conn(0x7fa2141662d0 msgr2=0x7fa21406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7fa219fd3640 1 --2- 172.21.15.172:0/2702443805 >> v2:172.21.15.177:3300/0 conn(0x7fa2141662d0 0x7fa21406e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7fa219fd3640 1 -- 172.21.15.172:0/2702443805 >> v2:172.21.15.172:3300/0 conn(0x7fa21406e200 msgr2=0x7fa2140ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7fa219fd3640 1 --2- 172.21.15.172:0/2702443805 >> v2:172.21.15.172:3300/0 conn(0x7fa21406e200 0x7fa2140ad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7fa219fd3640 1 -- 172.21.15.172:0/2702443805 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa214166d20 con 0x7fa214162990 2024-04-14T08:33:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7f57853c8640 1 --2- 172.21.15.172:0/642013834 >> v2:172.21.15.172:3300/0 conn(0x7f577803e830 0x7f5778053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7f5785bc9640 1 -- 172.21.15.172:0/642013834 >> v2:172.21.15.172:3300/0 conn(0x7f577803e830 msgr2=0x7f5778053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7f5784bc7640 1 -- 172.21.15.172:0/642013834 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3389735958 0 0) 0x7f57780582c0 con 0x7f5778054120 2024-04-14T08:33:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1ee2ea2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1edc0b5540 0x7f1edc0b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1edbfff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1edc0b23e0 0x7f1edc0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1edbfff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1edc0b23e0 0x7f1edc0b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33106/0 (socket says 172.21.15.172:33106) 2024-04-14T08:33:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1ee0c3e640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f1edc14c130 0x7f1edc0b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42764/0 (socket says 172.21.15.172:42764) 2024-04-14T08:33:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f6d43fff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6d4415e3f0 0x7f6d440aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7f5f1fc1c640 1 -- start start 2024-04-14T08:33:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f5f1fc1c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5f100eacf0 0x7f5f1004fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7f0633fff640 1 -- 172.21.15.172:0/3634723858 >> v2:172.21.15.172:3300/0 conn(0x7f062c04fc00 msgr2=0x7f062c051760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7f0633fff640 1 --2- 172.21.15.172:0/3634723858 >> v2:172.21.15.172:3300/0 conn(0x7f062c04fc00 0x7f062c051760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7f0633fff640 1 -- 172.21.15.172:0/3634723858 >> v1:172.21.15.172:6789/0 conn(0x7f062c050520 legacy=0x7f062c053890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f5f1fc1c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5f10006e90 0x7f5f10007300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f5f1fc1c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f10053710 con 0x7f5f1004ffc0 2024-04-14T08:33:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f5f1fc1c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f5f10007840 con 0x7f5f100eacf0 2024-04-14T08:33:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f5f1fc1c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5f10054320 con 0x7f5f10006e90 2024-04-14T08:33:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f6d43fff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6d4415e3f0 0x7f6d440aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33120/0 (socket says 172.21.15.172:33120) 2024-04-14T08:33:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f6d43fff640 1 -- 172.21.15.172:0/4016948410 learned_addr learned my addr 172.21.15.172:0/4016948410 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f6d4a95e640 1 --2- 172.21.15.172:0/4016948410 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d4406e910 0x7f6d440b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.090+0000 7f0633fff640 1 -- 172.21.15.172:0/3634723858 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062c053090 con 0x7f062c008180 2024-04-14T08:33:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1edbfff640 1 -- 172.21.15.172:0/601972341 learned_addr learned my addr 172.21.15.172:0/601972341 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f0633fff640 1 --2- 172.21.15.172:0/3634723858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062c008180 0x7f062c04ffe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f063406a340 tx=0x7f06340a6eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a29280dbf6c592 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f6d4a95e640 1 -- 172.21.15.172:0/4016948410 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d440ad990 con 0x7f6d440b3840 2024-04-14T08:33:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1ee2ea2640 1 -- 172.21.15.172:0/601972341 --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1edc0712d0 con 0x7f1edc14c130 2024-04-14T08:33:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1ee2ea2640 1 -- 172.21.15.172:0/601972341 --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f1edc0715b0 con 0x7f1edc0b5540 2024-04-14T08:33:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1ee2ea2640 1 -- 172.21.15.172:0/601972341 --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f1edc071890 con 0x7f1edc0b23e0 2024-04-14T08:33:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1ee143f640 1 --2- 172.21.15.172:0/601972341 >> v2:172.21.15.172:3300/0 conn(0x7f1edc0b5540 0x7f1edc0b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f6d4a95e640 1 -- 172.21.15.172:0/4016948410 --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f6d4415e7d0 con 0x7f6d4415e3f0 2024-04-14T08:33:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f6d4a95e640 1 -- 172.21.15.172:0/4016948410 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6d440adc70 con 0x7f6d4406e910 2024-04-14T08:33:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f5f1d9b8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5f100eacf0 0x7f5f1004fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f6d48efb640 1 --2- 172.21.15.172:0/4016948410 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d4406e910 0x7f6d440b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f06337fe640 1 -- 172.21.15.172:0/3634723858 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06340b4070 con 0x7f062c008180 2024-04-14T08:33:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f06337fe640 1 -- 172.21.15.172:0/3634723858 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06340a7d60 con 0x7f062c008180 2024-04-14T08:33:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f5f1d9b8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5f100eacf0 0x7f5f1004fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49566/0 (socket says 172.21.15.172:49566) 2024-04-14T08:33:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f5f1d9b8640 1 -- 172.21.15.172:0/154776486 learned_addr learned my addr 172.21.15.172:0/154776486 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f2a60d5e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a5c0d0cc0 0x7f2a5c0d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f2a60d5e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2a5c0d49c0 0x7f2a5c0d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f2a60d5e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a5c0d15e0 con 0x7f2a5c160f30 2024-04-14T08:33:13.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f2a60d5e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2a5c0ceec0 con 0x7f2a5c0d0cc0 2024-04-14T08:33:13.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f2a60d5e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2a5c0cfd00 con 0x7f2a5c0d49c0 2024-04-14T08:33:13.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f1128c60640 1 Processor -- start 2024-04-14T08:33:13.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f2a5a59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a5c0d0cc0 0x7f2a5c0d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1edbfff640 1 -- 172.21.15.172:0/601972341 >> v2:172.21.15.172:3300/0 conn(0x7f1edc0b5540 msgr2=0x7f1edc0b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:13.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1edbfff640 1 --2- 172.21.15.172:0/601972341 >> v2:172.21.15.172:3300/0 conn(0x7f1edc0b5540 0x7f1edc0b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1edbfff640 1 -- 172.21.15.172:0/601972341 >> v1:172.21.15.177:6789/0 conn(0x7f1edc14c130 legacy=0x7f1edc0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f1edbfff640 1 -- 172.21.15.172:0/601972341 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1edc071b70 con 0x7f1edc0b23e0 2024-04-14T08:33:13.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.091+0000 7f5f1e1b9640 1 --2- 172.21.15.172:0/154776486 >> v2:172.21.15.177:3300/0 conn(0x7f5f10006e90 0x7f5f10007300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f2a5a59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a5c0d0cc0 0x7f2a5c0d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49580/0 (socket says 172.21.15.172:49580) 2024-04-14T08:33:13.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f6d43fff640 1 -- 172.21.15.172:0/4016948410 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d4406e910 msgr2=0x7f6d440b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:13.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f6d43fff640 1 --2- 172.21.15.172:0/4016948410 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d4406e910 0x7f6d440b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f6d43fff640 1 -- 172.21.15.172:0/4016948410 >> v1:172.21.15.172:6789/0 conn(0x7f6d440b3840 legacy=0x7f6d4406e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f2a5a59c640 1 -- 172.21.15.172:0/247632968 learned_addr learned my addr 172.21.15.172:0/247632968 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f2a59d9b640 1 --2- 172.21.15.172:0/247632968 >> v2:172.21.15.177:3300/0 conn(0x7f2a5c0d49c0 0x7f2a5c0d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f6d43fff640 1 -- 172.21.15.172:0/4016948410 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d440adf50 con 0x7f6d4415e3f0 2024-04-14T08:33:13.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f5785bc9640 1 --2- 172.21.15.172:0/642013834 >> v2:172.21.15.172:3300/0 conn(0x7f577803e830 0x7f5778053be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f5785bc9640 1 -- 172.21.15.172:0/642013834 >> v1:172.21.15.177:6789/0 conn(0x7f5778054120 legacy=0x7f57780e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f5785bc9640 1 -- 172.21.15.172:0/642013834 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5778058840 con 0x7f57780e30b0 2024-04-14T08:33:13.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f5f1e1b9640 1 -- 172.21.15.172:0/154776486 >> v2:172.21.15.172:3300/0 conn(0x7f5f100eacf0 msgr2=0x7f5f1004fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:13.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f5f1e1b9640 1 --2- 172.21.15.172:0/154776486 >> v2:172.21.15.172:3300/0 conn(0x7f5f100eacf0 0x7f5f1004fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f5f1e1b9640 1 -- 172.21.15.172:0/154776486 >> v1:172.21.15.172:6789/0 conn(0x7f5f1004ffc0 legacy=0x7f5f10037bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:13.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f5f1e1b9640 1 -- 172.21.15.172:0/154776486 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f100539f0 con 0x7f5f10006e90 2024-04-14T08:33:13.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f06337fe640 1 -- 172.21.15.172:0/3634723858 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06340af660 con 0x7f062c008180 2024-04-14T08:33:13.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f2a59d9b640 1 -- 172.21.15.172:0/247632968 >> v2:172.21.15.172:3300/0 conn(0x7f2a5c0d0cc0 msgr2=0x7f2a5c0d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:13.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f2a59d9b640 1 --2- 172.21.15.172:0/247632968 >> v2:172.21.15.172:3300/0 conn(0x7f2a5c0d0cc0 0x7f2a5c0d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f2a59d9b640 1 -- 172.21.15.172:0/247632968 >> v1:172.21.15.172:6789/0 conn(0x7f2a5c160f30 legacy=0x7f2a5c161380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:13.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f2a59d9b640 1 -- 172.21.15.172:0/247632968 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c0cf0f0 con 0x7f2a5c0d49c0 2024-04-14T08:33:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f57853c8640 1 --2- 172.21.15.172:0/642013834 >> v2:172.21.15.172:3300/0 conn(0x7f577803e830 0x7f5778053be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:33:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7fa219fd3640 1 --2- 172.21.15.172:0/2702443805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa214162990 0x7fa214071470 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa21409e160 tx=0x7fa20c02bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2518e86b7822daed server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f063b17b640 1 -- 172.21.15.172:0/3634723858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062c008180 msgr2=0x7f062c04ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f063b17b640 1 --2- 172.21.15.172:0/3634723858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062c008180 0x7f062c04ffe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f063406a340 tx=0x7f06340a6eb0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f063b17b640 1 -- 172.21.15.172:0/3634723858 shutdown_connections 2024-04-14T08:33:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f063b17b640 1 --2- 172.21.15.172:0/3634723858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062c008180 0x7f062c04ffe0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7fa20bfff640 1 -- 172.21.15.172:0/2702443805 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c039070 con 0x7fa214162990 2024-04-14T08:33:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7fa20bfff640 1 -- 172.21.15.172:0/2702443805 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa20c02cb40 con 0x7fa214162990 2024-04-14T08:33:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7fa20bfff640 1 -- 172.21.15.172:0/2702443805 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c034440 con 0x7fa214162990 2024-04-14T08:33:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f1128c60640 1 -- start start 2024-04-14T08:33:13.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f063b17b640 1 --2- 172.21.15.172:0/3634723858 >> v2:172.21.15.172:3300/0 conn(0x7f062c04fc00 0x7f062c051760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.092+0000 7f063b17b640 1 -- 172.21.15.172:0/3634723858 >> 172.21.15.172:0/3634723858 conn(0x7f062c055340 msgr2=0x7f062c055750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f1edbfff640 1 --2- 172.21.15.172:0/601972341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1edc0b23e0 0x7f1edc0b5000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0b4eb0 tx=0x7f1ec802b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=876d1b2afa7331e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f063b17b640 1 -- 172.21.15.172:0/3634723858 shutdown_connections 2024-04-14T08:33:13.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f6d43fff640 1 --2- 172.21.15.172:0/4016948410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4415e3f0 0x7f6d440aea30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009870 tx=0x7f6d3402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ee61d51cf8729b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f063b17b640 1 -- 172.21.15.172:0/3634723858 wait complete. 2024-04-14T08:33:13.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f5f1e1b9640 1 --2- 172.21.15.172:0/154776486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f10006e90 0x7f5f10007300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5f18051160 tx=0x7f5f180a6e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c4211e95a44fed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f6d42ffd640 1 -- 172.21.15.172:0/4016948410 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34034070 con 0x7f6d4415e3f0 2024-04-14T08:33:13.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f6d42ffd640 1 -- 172.21.15.172:0/4016948410 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d34030840 con 0x7f6d4415e3f0 2024-04-14T08:33:13.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f6d42ffd640 1 -- 172.21.15.172:0/4016948410 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34030c70 con 0x7f6d4415e3f0 2024-04-14T08:33:13.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f2a59d9b640 1 --2- 172.21.15.172:0/247632968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a5c0d49c0 0x7f2a5c0d4e50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c06bee0 tx=0x7f2a5002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e21d4b605a0da3d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f1edb7fe640 1 -- 172.21.15.172:0/601972341 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8034070 con 0x7f1edc0b23e0 2024-04-14T08:33:13.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f1edb7fe640 1 -- 172.21.15.172:0/601972341 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec8030870 con 0x7f1edc0b23e0 2024-04-14T08:33:13.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f1edb7fe640 1 -- 172.21.15.172:0/601972341 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8030c40 con 0x7f1edc0b23e0 2024-04-14T08:33:13.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.093+0000 7f5f1c9b6640 1 -- 172.21.15.172:0/154776486 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f180b4070 con 0x7f5f10006e90 2024-04-14T08:33:13.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f5f1c9b6640 1 -- 172.21.15.172:0/154776486 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f180a7cc0 con 0x7f5f10006e90 2024-04-14T08:33:13.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f5f1c9b6640 1 -- 172.21.15.172:0/154776486 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f180af640 con 0x7f5f10006e90 2024-04-14T08:33:13.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1128c60640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f112415e670 0x7f11240b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1128c60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1124072ac0 0x7f112406b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1128c60640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11240cef70 con 0x7f11240cead0 2024-04-14T08:33:13.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1128c60640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f11240cd780 con 0x7f112415e670 2024-04-14T08:33:13.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1128c60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11240cc9c0 con 0x7f1124072ac0 2024-04-14T08:33:13.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1ee2ea2640 1 -- 172.21.15.172:0/601972341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1edc0b23e0 msgr2=0x7f1edc0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1ee2ea2640 1 --2- 172.21.15.172:0/601972341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1edc0b23e0 0x7f1edc0b5000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0b4eb0 tx=0x7f1ec802b9b0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7fa21ba36640 1 -- 172.21.15.172:0/2702443805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa214162990 msgr2=0x7fa214071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7fa21ba36640 1 --2- 172.21.15.172:0/2702443805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa214162990 0x7fa214071470 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa21409e160 tx=0x7fa20c02bd90 comp rx=0 tx=0).stop 2024-04-14T08:33:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f5785bc9640 1 --2- 172.21.15.172:0/642013834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57780e30b0 0x7f5778057d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5778056950 tx=0x7f577002ba60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59aaa995401de84a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1121d9b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f11240cead0 0x7f11240723b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35712/0 (socket says 172.21.15.172:35712) 2024-04-14T08:33:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1121d9b640 1 -- 172.21.15.172:0/2176050100 learned_addr learned my addr 172.21.15.172:0/2176050100 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f112259c640 1 --2- 172.21.15.172:0/2176050100 >> v2:172.21.15.177:3300/0 conn(0x7f112415e670 0x7f11240b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f6d4a95e640 1 -- 172.21.15.172:0/4016948410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4415e3f0 msgr2=0x7f6d440aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f6d4a95e640 1 --2- 172.21.15.172:0/4016948410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4415e3f0 0x7f6d440aea30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009870 tx=0x7f6d3402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1ee2ea2640 1 -- 172.21.15.172:0/601972341 shutdown_connections 2024-04-14T08:33:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1ee2ea2640 1 --2- 172.21.15.172:0/601972341 >> v2:172.21.15.172:3300/0 conn(0x7f1edc0b5540 0x7f1edc0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1ee2ea2640 1 --2- 172.21.15.172:0/601972341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1edc0b23e0 0x7f1edc0b5000 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1ee2ea2640 1 -- 172.21.15.172:0/601972341 >> 172.21.15.172:0/601972341 conn(0x7f1edc06b520 msgr2=0x7f1edc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7fa21ba36640 1 -- 172.21.15.172:0/2702443805 shutdown_connections 2024-04-14T08:33:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7fa21ba36640 1 --2- 172.21.15.172:0/2702443805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa214162990 0x7fa214071470 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7fa21ba36640 1 --2- 172.21.15.172:0/2702443805 >> v2:172.21.15.177:3300/0 conn(0x7fa2141662d0 0x7fa21406e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7fa21ba36640 1 --2- 172.21.15.172:0/2702443805 >> v2:172.21.15.172:3300/0 conn(0x7fa21406e200 0x7fa2140ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7fa21ba36640 1 -- 172.21.15.172:0/2702443805 >> 172.21.15.172:0/2702443805 conn(0x7fa2140b1b30 msgr2=0x7fa2140b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1ee2ea2640 1 -- 172.21.15.172:0/601972341 shutdown_connections 2024-04-14T08:33:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f1ee2ea2640 1 -- 172.21.15.172:0/601972341 wait complete. 2024-04-14T08:33:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f112159a640 1 -- 172.21.15.172:0/2176050100 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342694001 0 0) 0x7f11240cef70 con 0x7f11240cead0 2024-04-14T08:33:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7f112159a640 1 -- 172.21.15.172:0/2176050100 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f110c003520 con 0x7f11240cead0 2024-04-14T08:33:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.094+0000 7fa21ba36640 1 -- 172.21.15.172:0/2702443805 shutdown_connections 2024-04-14T08:33:13.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/154776486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f10006e90 msgr2=0x7f5f10007300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f5f1fc1c640 1 --2- 172.21.15.172:0/154776486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f10006e90 0x7f5f10007300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5f18051160 tx=0x7f5f180a6e10 comp rx=0 tx=0).stop 2024-04-14T08:33:13.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7fa21ba36640 1 -- 172.21.15.172:0/2702443805 wait complete. 2024-04-14T08:33:13.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f112259c640 1 -- 172.21.15.172:0/2176050100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1124072ac0 msgr2=0x7f112406b520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-14T08:33:13.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f112259c640 1 --2- 172.21.15.172:0/2176050100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1124072ac0 0x7f112406b520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f112259c640 1 -- 172.21.15.172:0/2176050100 >> v1:172.21.15.172:6789/0 conn(0x7f11240cead0 legacy=0x7f11240723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f112259c640 1 -- 172.21.15.172:0/2176050100 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11240ccca0 con 0x7f112415e670 2024-04-14T08:33:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/154776486 shutdown_connections 2024-04-14T08:33:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f5f1fc1c640 1 --2- 172.21.15.172:0/154776486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f10006e90 0x7f5f10007300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f5f1fc1c640 1 --2- 172.21.15.172:0/154776486 >> v2:172.21.15.172:3300/0 conn(0x7f5f100eacf0 0x7f5f1004fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/154776486 >> 172.21.15.172:0/154776486 conn(0x7f5f10009fa0 msgr2=0x7f5f1000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f2a5959a640 1 -- 172.21.15.172:0/247632968 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a500315a0 con 0x7f2a5c0d49c0 2024-04-14T08:33:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f2a5959a640 1 -- 172.21.15.172:0/247632968 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a50037070 con 0x7f2a5c0d49c0 2024-04-14T08:33:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f6d4a95e640 1 -- 172.21.15.172:0/4016948410 shutdown_connections 2024-04-14T08:33:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f6d4a95e640 1 --2- 172.21.15.172:0/4016948410 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d4406e910 0x7f6d440b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f6d4a95e640 1 --2- 172.21.15.172:0/4016948410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4415e3f0 0x7f6d440aea30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f6d4a95e640 1 -- 172.21.15.172:0/4016948410 >> 172.21.15.172:0/4016948410 conn(0x7f6d440b1b30 msgr2=0x7f6d440b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f6d4a95e640 1 -- 172.21.15.172:0/4016948410 shutdown_connections 2024-04-14T08:33:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f6d4a95e640 1 -- 172.21.15.172:0/4016948410 wait complete. 2024-04-14T08:33:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f5784bc7640 1 -- 172.21.15.172:0/642013834 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577002d820 con 0x7f57780e30b0 2024-04-14T08:33:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f5784bc7640 1 -- 172.21.15.172:0/642013834 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577002d9c0 con 0x7f57780e30b0 2024-04-14T08:33:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f5784bc7640 1 -- 172.21.15.172:0/642013834 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577002dd90 con 0x7f57780e30b0 2024-04-14T08:33:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f2a5959a640 1 -- 172.21.15.172:0/247632968 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a50031960 con 0x7f2a5c0d49c0 2024-04-14T08:33:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f063b17b640 1 Processor -- start 2024-04-14T08:33:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f063b17b640 1 -- start start 2024-04-14T08:33:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f2a60d5e640 1 -- 172.21.15.172:0/247632968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a5c0d49c0 msgr2=0x7f2a5c0d4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f2a60d5e640 1 --2- 172.21.15.172:0/247632968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a5c0d49c0 0x7f2a5c0d4e50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c06bee0 tx=0x7f2a5002b840 comp rx=0 tx=0).stop 2024-04-14T08:33:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f112259c640 1 --2- 172.21.15.172:0/2176050100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112415e670 0x7f11240b0440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f11180097a0 tx=0x7f111802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f00b39b921e899d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f2a60d5e640 1 -- 172.21.15.172:0/247632968 shutdown_connections 2024-04-14T08:33:13.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f2a60d5e640 1 --2- 172.21.15.172:0/247632968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a5c0d49c0 0x7f2a5c0d4e50 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c06bee0 tx=0x7f2a5002b840 comp rx=0 tx=0).stop 2024-04-14T08:33:13.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f2a60d5e640 1 --2- 172.21.15.172:0/247632968 >> v2:172.21.15.172:3300/0 conn(0x7f2a5c0d0cc0 0x7f2a5c0d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f2a60d5e640 1 -- 172.21.15.172:0/247632968 >> 172.21.15.172:0/247632968 conn(0x7f2a5c09d580 msgr2=0x7f2a5c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f112159a640 1 -- 172.21.15.172:0/2176050100 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180315a0 con 0x7f112415e670 2024-04-14T08:33:13.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f1ee2ea2640 1 Processor -- start 2024-04-14T08:33:13.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f2a60d5e640 1 -- 172.21.15.172:0/247632968 shutdown_connections 2024-04-14T08:33:13.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f2a60d5e640 1 -- 172.21.15.172:0/247632968 wait complete. 2024-04-14T08:33:13.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f1ee2ea2640 1 -- start start 2024-04-14T08:33:13.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f063b17b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f062c008180 0x7f062c0d86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f063b17b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062c04fc00 0x7f062c04ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f063b17b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062c050520 0x7f062c04ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f063b17b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f062c054120 con 0x7f062c008180 2024-04-14T08:33:13.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f063b17b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f062c053fa0 con 0x7f062c04fc00 2024-04-14T08:33:13.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f063b17b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f062c0542a0 con 0x7f062c050520 2024-04-14T08:33:13.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f5786bcb640 1 -- 172.21.15.172:0/642013834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57780e30b0 msgr2=0x7f5778057d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f5786bcb640 1 --2- 172.21.15.172:0/642013834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57780e30b0 0x7f5778057d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5778056950 tx=0x7f577002ba60 comp rx=0 tx=0).stop 2024-04-14T08:33:13.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7fa21ba36640 1 Processor -- start 2024-04-14T08:33:13.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7fa21ba36640 1 -- start start 2024-04-14T08:33:13.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f112159a640 1 -- 172.21.15.172:0/2176050100 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1118037070 con 0x7f112415e670 2024-04-14T08:33:13.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f0633fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062c04fc00 0x7f062c04ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.096+0000 7f0633fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062c04fc00 0x7f062c04ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33150/0 (socket says 172.21.15.172:33150) 2024-04-14T08:33:13.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f6d4a95e640 1 Processor -- start 2024-04-14T08:33:13.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f6d4a95e640 1 -- start start 2024-04-14T08:33:13.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f0633fff640 1 -- 172.21.15.172:0/2273081050 learned_addr learned my addr 172.21.15.172:0/2273081050 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f0639718640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062c050520 0x7f062c04ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f0638f17640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f062c008180 0x7f062c0d86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f0633fff640 1 -- 172.21.15.172:0/2273081050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062c050520 msgr2=0x7f062c04ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f0633fff640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062c050520 0x7f062c04ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f0633fff640 1 -- 172.21.15.172:0/2273081050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f062c008180 msgr2=0x7f062c0d86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f0633fff640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f062c008180 0x7f062c0d86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f0633fff640 1 -- 172.21.15.172:0/2273081050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062c0df260 con 0x7f062c04fc00 2024-04-14T08:33:13.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.095+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/154776486 shutdown_connections 2024-04-14T08:33:13.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f112159a640 1 -- 172.21.15.172:0/2176050100 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180315a0 con 0x7f112415e670 2024-04-14T08:33:13.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f0633fff640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062c04fc00 0x7f062c04ce20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0634068bd0 tx=0x7f06340692a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f1ee2ea2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1edc0b23e0 0x7f1edc1fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1ee2ea2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1edc0b5540 0x7f1edc145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1ee2ea2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1edc14c130 0x7f1edc15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1ee2ea2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc0b59d0 con 0x7f1edc0b23e0 2024-04-14T08:33:13.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1ee2ea2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc14c510 con 0x7f1edc14c130 2024-04-14T08:33:13.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1ee2ea2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1edc0b5b50 con 0x7f1edc0b5540 2024-04-14T08:33:13.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1128c60640 1 -- 172.21.15.172:0/2176050100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112415e670 msgr2=0x7f11240b0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f5786bcb640 1 -- 172.21.15.172:0/642013834 shutdown_connections 2024-04-14T08:33:13.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f5786bcb640 1 --2- 172.21.15.172:0/642013834 >> v2:172.21.15.172:3300/0 conn(0x7f577803e830 0x7f5778053be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f5786bcb640 1 --2- 172.21.15.172:0/642013834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57780e30b0 0x7f5778057d80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1edbfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1edc0b5540 0x7f1edc145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1edbfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1edc0b5540 0x7f1edc145150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:43206/0 (socket says 172.21.15.172:43206) 2024-04-14T08:33:13.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1edbfff640 1 -- 172.21.15.172:0/1246268634 learned_addr learned my addr 172.21.15.172:0/1246268634 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f6d4a95e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4406e910 0x7f6d4413ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1ee143f640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1edc14c130 0x7f1edc15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1ee0c3e640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1edc0b23e0 0x7f1edc1fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f2a60d5e640 1 Processor -- start 2024-04-14T08:33:13.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f6d4a95e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d440b3840 0x7f6d4409c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f6d4a95e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4415e3f0 0x7f6d44158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f06317fa640 1 -- 172.21.15.172:0/2273081050 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06340b4050 con 0x7f062c04fc00 2024-04-14T08:33:13.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f06317fa640 1 -- 172.21.15.172:0/2273081050 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06340b5040 con 0x7f062c04fc00 2024-04-14T08:33:13.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f06317fa640 1 -- 172.21.15.172:0/2273081050 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06340b9810 con 0x7f062c04fc00 2024-04-14T08:33:13.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f063b17b640 1 -- 172.21.15.172:0/2273081050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062c0df540 con 0x7f062c04fc00 2024-04-14T08:33:13.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f063b17b640 1 -- 172.21.15.172:0/2273081050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f062c0e7ca0 con 0x7f062c04fc00 2024-04-14T08:33:13.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f5786bcb640 1 -- 172.21.15.172:0/642013834 >> 172.21.15.172:0/642013834 conn(0x7f57780559d0 msgr2=0x7f5778055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f6d4a95e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d440aef70 con 0x7f6d4415e3f0 2024-04-14T08:33:13.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f6d4a95e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6d440ae510 con 0x7f6d4406e910 2024-04-14T08:33:13.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f6d4a95e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6d440b4ef0 con 0x7f6d440b3840 2024-04-14T08:33:13.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7fa21ba36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa21406e200 0x7fa214157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1128c60640 1 --2- 172.21.15.172:0/2176050100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112415e670 0x7f11240b0440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f11180097a0 tx=0x7f111802b840 comp rx=0 tx=0).stop 2024-04-14T08:33:13.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa21ba36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa214162990 0x7fa21409ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa21ba36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa21409c650 0x7fa21409d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1128c60640 1 -- 172.21.15.172:0/2176050100 shutdown_connections 2024-04-14T08:33:13.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1128c60640 1 --2- 172.21.15.172:0/2176050100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1124072ac0 0x7f112406b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1128c60640 1 --2- 172.21.15.172:0/2176050100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112415e670 0x7f11240b0440 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f11180097a0 tx=0x7f111802b840 comp rx=0 tx=0).stop 2024-04-14T08:33:13.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f1128c60640 1 -- 172.21.15.172:0/2176050100 >> 172.21.15.172:0/2176050100 conn(0x7f11240cd980 msgr2=0x7f11240cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f5786bcb640 1 -- 172.21.15.172:0/642013834 shutdown_connections 2024-04-14T08:33:13.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f1ee143f640 1 -- 172.21.15.172:0/1246268634 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1edc0b5540 msgr2=0x7f1edc145150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f1ee143f640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1edc0b5540 0x7f1edc145150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f1ee143f640 1 -- 172.21.15.172:0/1246268634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1edc0b23e0 msgr2=0x7f1edc1fc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f1ee143f640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1edc0b23e0 0x7f1edc1fc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f1ee143f640 1 -- 172.21.15.172:0/1246268634 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1edc15d630 con 0x7f1edc14c130 2024-04-14T08:33:13.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa21ba36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa214071b30 con 0x7fa214162990 2024-04-14T08:33:13.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa2197d2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa21406e200 0x7fa214157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa21ba36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2140719b0 con 0x7fa21409c650 2024-04-14T08:33:13.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa2197d2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa21406e200 0x7fa214157ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:43208/0 (socket says 172.21.15.172:43208) 2024-04-14T08:33:13.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa2197d2640 1 -- 172.21.15.172:0/3129666679 learned_addr learned my addr 172.21.15.172:0/3129666679 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa21ba36640 1 -- 172.21.15.172:0/3129666679 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa2140b3210 con 0x7fa21406e200 2024-04-14T08:33:13.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f6d48efb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4415e3f0 0x7f6d44158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f6d48efb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4415e3f0 0x7f6d44158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49600/0 (socket says 172.21.15.172:49600) 2024-04-14T08:33:13.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f6d48efb640 1 -- 172.21.15.172:0/670144756 learned_addr learned my addr 172.21.15.172:0/670144756 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f1ee143f640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1edc14c130 0x7f1edc15f0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4007d90 tx=0x7f1ed400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa219fd3640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa21409c650 0x7fa21409d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.097+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/154776486 wait complete. 2024-04-14T08:33:13.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f5f1fc1c640 1 Processor -- start 2024-04-14T08:33:13.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa2197d2640 1 -- 172.21.15.172:0/3129666679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa21409c650 msgr2=0x7fa21409d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f5f1fc1c640 1 -- start start 2024-04-14T08:33:13.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f1128c60640 1 -- 172.21.15.172:0/2176050100 shutdown_connections 2024-04-14T08:33:13.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa2197d2640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa21409c650 0x7fa21409d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f6d48efb640 1 -- 172.21.15.172:0/670144756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d440b3840 msgr2=0x7f6d4409c120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-14T08:33:13.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f6d48efb640 1 --2- 172.21.15.172:0/670144756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d440b3840 0x7f6d4409c120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f6d48efb640 1 -- 172.21.15.172:0/670144756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4406e910 msgr2=0x7f6d4413ee40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-14T08:33:13.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f6d48efb640 1 --2- 172.21.15.172:0/670144756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4406e910 0x7f6d4413ee40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f6d48efb640 1 -- 172.21.15.172:0/670144756 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4409aa70 con 0x7f6d4415e3f0 2024-04-14T08:33:13.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f1ed97fa640 1 -- 172.21.15.172:0/1246268634 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed400edd0 con 0x7f1edc14c130 2024-04-14T08:33:13.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa2197d2640 1 -- 172.21.15.172:0/3129666679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa214162990 msgr2=0x7fa21409ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f1ed97fa640 1 -- 172.21.15.172:0/1246268634 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed4003990 con 0x7f1edc14c130 2024-04-14T08:33:13.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa2197d2640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa214162990 0x7fa21409ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f1ee2ea2640 1 -- 172.21.15.172:0/1246268634 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1edc15d910 con 0x7f1edc14c130 2024-04-14T08:33:13.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7fa2197d2640 1 -- 172.21.15.172:0/3129666679 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21409b5b0 con 0x7fa21406e200 2024-04-14T08:33:13.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f6d48efb640 1 --2- 172.21.15.172:0/670144756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4415e3f0 0x7f6d44158eb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009dd0 tx=0x7f6d3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f1ee2ea2640 1 -- 172.21.15.172:0/1246268634 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1edc15ded0 con 0x7f1edc14c130 2024-04-14T08:33:13.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f063b17b640 1 -- 172.21.15.172:0/2273081050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f062c0057d0 con 0x7f062c04fc00 2024-04-14T08:33:13.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.098+0000 7f2a60d5e640 1 -- start start 2024-04-14T08:33:13.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f2a60d5e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a5c0d0cc0 0x7f2a5c0c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f2a60d5e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a5c160f30 0x7f2a5c150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f2a60d5e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a5c0c81c0 0x7f2a5c150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f2a60d5e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c0cf990 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f5f1fc1c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f10006e90 0x7f5f100c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f2a60d5e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c0cf6b0 con 0x7f2a5c160f30 2024-04-14T08:33:13.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f2a60d5e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5c0d32b0 con 0x7f2a5c0d0cc0 2024-04-14T08:33:13.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f2a5ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a5c0c81c0 0x7f2a5c150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f2a5ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a5c0c81c0 0x7f2a5c150a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49620/0 (socket says 172.21.15.172:49620) 2024-04-14T08:33:13.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f2a5ad9d640 1 -- 172.21.15.172:0/1921073403 learned_addr learned my addr 172.21.15.172:0/1921073403 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f2a5ad9d640 1 -- 172.21.15.172:0/1921073403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a5c0d0cc0 msgr2=0x7f2a5c0c7c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-14T08:33:13.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f2a5ad9d640 1 --2- 172.21.15.172:0/1921073403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a5c0d0cc0 0x7f2a5c0c7c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f2a5ad9d640 1 -- 172.21.15.172:0/1921073403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a5c160f30 msgr2=0x7f2a5c150510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-14T08:33:13.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f2a5ad9d640 1 --2- 172.21.15.172:0/1921073403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a5c160f30 0x7f2a5c150510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f2a5ad9d640 1 -- 172.21.15.172:0/1921073403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c0ca900 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f5f1fc1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f1004ffc0 0x7f5f100f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f2a5ad9d640 1 --2- 172.21.15.172:0/1921073403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a5c0c81c0 0x7f2a5c150a50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400bd60 tx=0x7f2a54009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f6d40ff9640 1 -- 172.21.15.172:0/670144756 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c018020 con 0x7f6d4415e3f0 2024-04-14T08:33:13.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f6d40ff9640 1 -- 172.21.15.172:0/670144756 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d3c014480 con 0x7f6d4415e3f0 2024-04-14T08:33:13.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.100+0000 7f6d40ff9640 1 -- 172.21.15.172:0/670144756 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c014870 con 0x7f6d4415e3f0 2024-04-14T08:33:13.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f6d4a95e640 1 -- 172.21.15.172:0/670144756 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4409ad50 con 0x7f6d4415e3f0 2024-04-14T08:33:13.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f5f1d9b8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f10006e90 0x7f5f100c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f6d4a95e640 1 -- 172.21.15.172:0/670144756 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4409b310 con 0x7f6d4415e3f0 2024-04-14T08:33:13.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f5786bcb640 1 -- 172.21.15.172:0/642013834 wait complete. 2024-04-14T08:33:13.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f5f1d9b8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f10006e90 0x7f5f100c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:43240/0 (socket says 172.21.15.172:43240) 2024-04-14T08:33:13.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f5786bcb640 1 Processor -- start 2024-04-14T08:33:13.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f5786bcb640 1 -- start start 2024-04-14T08:33:13.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7fa2197d2640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa21406e200 0x7fa214157ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa21000e8d0 tx=0x7fa21000ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.099+0000 7f1128c60640 1 -- 172.21.15.172:0/2176050100 wait complete. 2024-04-14T08:33:13.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f1128c60640 1 Processor -- start 2024-04-14T08:33:13.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f5f1d9b8640 1 -- 172.21.15.172:0/4281589632 learned_addr learned my addr 172.21.15.172:0/4281589632 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f5f1d1b7640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f1004ffc0 0x7f5f100f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f5f1fc1c640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f100eacf0 0x7f5f100cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5f100547b0 con 0x7f5f1004ffc0 2024-04-14T08:33:13.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5f10053fb0 con 0x7f5f100eacf0 2024-04-14T08:33:13.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5f10054930 con 0x7f5f10006e90 2024-04-14T08:33:13.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f1128c60640 1 -- start start 2024-04-14T08:33:13.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f5f1d1b7640 1 -- 172.21.15.172:0/4281589632 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f10006e90 msgr2=0x7f5f100c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f06317fa640 1 -- 172.21.15.172:0/2273081050 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06340c2020 con 0x7f062c04fc00 2024-04-14T08:33:13.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f06317fa640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f061806c9f0 0x7f061806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f06317fa640 1 -- 172.21.15.172:0/2273081050 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0634128c80 con 0x7f062c04fc00 2024-04-14T08:33:13.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f5f1d1b7640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f10006e90 0x7f5f100c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f5f1d1b7640 1 -- 172.21.15.172:0/4281589632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f100eacf0 msgr2=0x7f5f100cd240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-14T08:33:13.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f5f1d1b7640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f100eacf0 0x7f5f100cd240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f5f1d1b7640 1 -- 172.21.15.172:0/4281589632 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f100c84f0 con 0x7f5f1004ffc0 2024-04-14T08:33:13.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f2a4affd640 1 -- 172.21.15.172:0/1921073403 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54019070 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f5f1d1b7640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f1004ffc0 0x7f5f100f2a80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c440 tx=0x7f5f1400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7fa209ffb640 1 -- 172.21.15.172:0/3129666679 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210017070 con 0x7fa21406e200 2024-04-14T08:33:13.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7fa209ffb640 1 -- 172.21.15.172:0/3129666679 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa21000c940 con 0x7fa21406e200 2024-04-14T08:33:13.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7fa209ffb640 1 -- 172.21.15.172:0/3129666679 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21000ccf0 con 0x7fa21406e200 2024-04-14T08:33:13.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7fa21ba36640 1 -- 172.21.15.172:0/3129666679 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21409b890 con 0x7fa21406e200 2024-04-14T08:33:13.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f5786bcb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f577803e830 0x7f57780e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f5f0e7fc640 1 -- 172.21.15.172:0/4281589632 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14017070 con 0x7f5f1004ffc0 2024-04-14T08:33:13.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7fa21ba36640 1 -- 172.21.15.172:0/3129666679 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21409be50 con 0x7fa21406e200 2024-04-14T08:33:13.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f5786bcb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5778054120 0x7f5778184ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f5786bcb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57780e30b0 0x7f577800b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.101+0000 7f1ed97fa640 1 -- 172.21.15.172:0/1246268634 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4003d80 con 0x7f1edc14c130 2024-04-14T08:33:13.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f1ec6ffd640 1 -- 172.21.15.172:0/1246268634 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1edc15e200 con 0x7f1edc14c130 2024-04-14T08:33:13.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f1ed97fa640 1 -- 172.21.15.172:0/1246268634 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ed401b020 con 0x7f1edc14c130 2024-04-14T08:33:13.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f100c87d0 con 0x7f5f1004ffc0 2024-04-14T08:33:13.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f1128c60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1124072ac0 0x7f11240b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f1128c60640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240cead0 0x7f112406b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f1128c60640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112406ba60 0x7f1124151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f1128c60640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11240b00f0 con 0x7f11240cead0 2024-04-14T08:33:13.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f1128c60640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f11240cd260 con 0x7f112406ba60 2024-04-14T08:33:13.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f1128c60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f112415ea50 con 0x7f1124072ac0 2024-04-14T08:33:13.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f100f2fc0 con 0x7f5f1004ffc0 2024-04-14T08:33:13.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f5f0e7fc640 1 -- 172.21.15.172:0/4281589632 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f14005230 con 0x7f5f1004ffc0 2024-04-14T08:33:13.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f57863ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57780e30b0 0x7f577800b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f5785bc9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f577803e830 0x7f57780e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f57853c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5778054120 0x7f5778184ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f57853c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5778054120 0x7f5778184ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:43246/0 (socket says 172.21.15.172:43246) 2024-04-14T08:33:13.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f5785bc9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f577803e830 0x7f57780e40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33198/0 (socket says 172.21.15.172:33198) 2024-04-14T08:33:13.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f5786bcb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f577803bfb0 con 0x7f57780e30b0 2024-04-14T08:33:13.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f1ed97fa640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ec006c9c0 0x7f1ec006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f1121d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240cead0 0x7f112406b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f1121d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240cead0 0x7f112406b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49636/0 (socket says 172.21.15.172:49636) 2024-04-14T08:33:13.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f1121d9b640 1 -- 172.21.15.172:0/2256728622 learned_addr learned my addr 172.21.15.172:0/2256728622 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f1121d9b640 1 -- 172.21.15.172:0/2256728622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1124072ac0 msgr2=0x7f11240b6150 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-14T08:33:13.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f1121d9b640 1 --2- 172.21.15.172:0/2256728622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1124072ac0 0x7f11240b6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f1121d9b640 1 -- 172.21.15.172:0/2256728622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112406ba60 msgr2=0x7f1124151c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-14T08:33:13.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f1121d9b640 1 --2- 172.21.15.172:0/2256728622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112406ba60 0x7f1124151c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f1121d9b640 1 -- 172.21.15.172:0/2256728622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1124150be0 con 0x7f11240cead0 2024-04-14T08:33:13.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f1ed97fa640 1 -- 172.21.15.172:0/1246268634 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1ed408bfd0 con 0x7f1edc14c130 2024-04-14T08:33:13.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.104+0000 7f5f0e7fc640 1 -- 172.21.15.172:0/4281589632 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14017070 con 0x7f5f1004ffc0 2024-04-14T08:33:13.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f0638f17640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f061806c9f0 0x7f061806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f2a60d5e640 1 -- 172.21.15.172:0/1921073403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c0cabe0 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.102+0000 7f2a60d5e640 1 -- 172.21.15.172:0/1921073403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c159be0 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.104+0000 7f1ee0c3e640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ec006c9c0 0x7f1ec006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.104+0000 7f0638f17640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f061806c9f0 0x7f061806eeb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f062c04def0 tx=0x7f0628009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.104+0000 7f1121d9b640 1 --2- 172.21.15.172:0/2256728622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240cead0 0x7f112406b520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f111400c400 tx=0x7f111400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.104+0000 7f2a4affd640 1 -- 172.21.15.172:0/1921073403 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a5400cc00 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.104+0000 7f6d40ff9640 1 -- 172.21.15.172:0/670144756 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d3c005d10 con 0x7f6d4415e3f0 2024-04-14T08:33:13.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.104+0000 7f6d40ff9640 1 --2- 172.21.15.172:0/670144756 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3006c9c0 0x7f6d3006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.104+0000 7f6d40ff9640 1 -- 172.21.15.172:0/670144756 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6d3c08c9d0 con 0x7f6d4415e3f0 2024-04-14T08:33:13.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.104+0000 7f1ee0c3e640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ec006c9c0 0x7f1ec006ee80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0000f60 tx=0x7f1ed0007c80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f5786bcb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f57780063a0 con 0x7f577803e830 2024-04-14T08:33:13.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f5786bcb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f57780e3490 con 0x7f5778054120 2024-04-14T08:33:13.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.103+0000 7f57853c8640 1 -- 172.21.15.172:0/820398743 learned_addr learned my addr 172.21.15.172:0/820398743 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7f2a4affd640 1 -- 172.21.15.172:0/1921073403 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54014440 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f00005290 con 0x7f5f1004ffc0 2024-04-14T08:33:13.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7f2a60d5e640 1 -- 172.21.15.172:0/1921073403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a40005290 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7f6d43fff640 1 --2- 172.21.15.172:0/670144756 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3006c9c0 0x7f6d3006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7f57853c8640 1 -- 172.21.15.172:0/820398743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f577803e830 msgr2=0x7f57780e40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7f57853c8640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f577803e830 0x7f57780e40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7f57853c8640 1 -- 172.21.15.172:0/820398743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57780e30b0 msgr2=0x7f577800b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7f6d4a95e640 1 -- 172.21.15.172:0/670144756 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d44068520 con 0x7f6d4415e3f0 2024-04-14T08:33:13.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7fa209ffb640 1 -- 172.21.15.172:0/3129666679 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa21001a070 con 0x7fa21406e200 2024-04-14T08:33:13.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7f57853c8640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57780e30b0 0x7f577800b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7f1112ffd640 1 -- 172.21.15.172:0/2256728622 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114017070 con 0x7f11240cead0 2024-04-14T08:33:13.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7fa1f77fe640 1 -- 172.21.15.172:0/3129666679 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa214068520 con 0x7fa21406e200 2024-04-14T08:33:13.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7f57853c8640 1 -- 172.21.15.172:0/820398743 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577804ff90 con 0x7f5778054120 2024-04-14T08:33:13.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.105+0000 7f1128c60640 1 -- 172.21.15.172:0/2256728622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1124150e50 con 0x7f11240cead0 2024-04-14T08:33:13.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.106+0000 7f1128c60640 1 -- 172.21.15.172:0/2256728622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1124151410 con 0x7f11240cead0 2024-04-14T08:33:13.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.106+0000 7f2a4affd640 1 -- 172.21.15.172:0/1921073403 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a5401c070 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.106+0000 7f1112ffd640 1 -- 172.21.15.172:0/2256728622 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1114005da0 con 0x7f11240cead0 2024-04-14T08:33:13.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.106+0000 7f1112ffd640 1 -- 172.21.15.172:0/2256728622 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11140072d0 con 0x7f11240cead0 2024-04-14T08:33:13.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.106+0000 7f57863ca640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57780e30b0 0x7f577800b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:13.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.106+0000 7fa209ffb640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa1f806c9f0 0x7fa1f806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.106+0000 7fa209ffb640 1 -- 172.21.15.172:0/3129666679 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa21008c0f0 con 0x7fa21406e200 2024-04-14T08:33:13.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.106+0000 7f57853c8640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5778054120 0x7f5778184ef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f577c00c820 tx=0x7f577c00cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.106+0000 7fa218fd1640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa1f806c9f0 0x7fa1f806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.106+0000 7f2a4affd640 1 --2- 172.21.15.172:0/1921073403 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3c06c8f0 0x7f2a3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.107+0000 7f6d43fff640 1 --2- 172.21.15.172:0/670144756 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3006c9c0 0x7f6d3006ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6d34036000 tx=0x7f6d34008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.107+0000 7fa218fd1640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa1f806c9f0 0x7fa1f806eeb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa204006fd0 tx=0x7fa204008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.107+0000 7f2a5a59c640 1 --2- 172.21.15.172:0/1921073403 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3c06c8f0 0x7f2a3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.107+0000 7f1128c60640 1 -- 172.21.15.172:0/2256728622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1108005290 con 0x7f11240cead0 2024-04-14T08:33:13.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.107+0000 7f5f0e7fc640 1 -- 172.21.15.172:0/4281589632 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f1401a070 con 0x7f5f1004ffc0 2024-04-14T08:33:13.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.107+0000 7f2a4affd640 1 -- 172.21.15.172:0/1921073403 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2a5408bc60 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.107+0000 7f06317fa640 1 -- 172.21.15.172:0/2273081050 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f063412f080 con 0x7f062c04fc00 2024-04-14T08:33:13.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.107+0000 7f2a5a59c640 1 --2- 172.21.15.172:0/1921073403 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3c06c8f0 0x7f2a3c06edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c15c500 tx=0x7f2a4c009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.107+0000 7f5f0e7fc640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5efc06ccf0 0x7f5efc06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.108+0000 7f5f0e7fc640 1 -- 172.21.15.172:0/4281589632 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5f14005030 con 0x7f5f1004ffc0 2024-04-14T08:33:13.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.108+0000 7f5f1d9b8640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5efc06ccf0 0x7f5efc06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.108+0000 7f5f1d9b8640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5efc06ccf0 0x7f5efc06f1b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006f40 tx=0x7f5f08009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.110+0000 7f57767fc640 1 -- 172.21.15.172:0/820398743 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c015070 con 0x7f5778054120 2024-04-14T08:33:13.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.110+0000 7f5786bcb640 1 -- 172.21.15.172:0/820398743 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57780e4940 con 0x7f5778054120 2024-04-14T08:33:13.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.110+0000 7f5786bcb640 1 -- 172.21.15.172:0/820398743 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5778050530 con 0x7f5778054120 2024-04-14T08:33:13.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.111+0000 7fa209ffb640 1 -- 172.21.15.172:0/3129666679 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa21005a270 con 0x7fa21406e200 2024-04-14T08:33:13.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.111+0000 7f57767fc640 1 -- 172.21.15.172:0/820398743 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577c007820 con 0x7f5778054120 2024-04-14T08:33:13.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.111+0000 7f57767fc640 1 -- 172.21.15.172:0/820398743 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c015070 con 0x7f5778054120 2024-04-14T08:33:13.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.112+0000 7f5786bcb640 1 -- 172.21.15.172:0/820398743 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57780057d0 con 0x7f5778054120 2024-04-14T08:33:13.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.113+0000 7f57767fc640 1 -- 172.21.15.172:0/820398743 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f577c018070 con 0x7f5778054120 2024-04-14T08:33:13.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.112+0000 7f1ed97fa640 1 -- 172.21.15.172:0/1246268634 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1ed405a0a0 con 0x7f1edc14c130 2024-04-14T08:33:13.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.113+0000 7f57767fc640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f576406ccf0 0x7f576406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.113+0000 7f57767fc640 1 -- 172.21.15.172:0/820398743 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f577c007620 con 0x7f5778054120 2024-04-14T08:33:13.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.113+0000 7f5785bc9640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f576406ccf0 0x7f576406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.114+0000 7f5785bc9640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f576406ccf0 0x7f576406f1b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5770036000 tx=0x7f5770008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.114+0000 7f6d40ff9640 1 -- 172.21.15.172:0/670144756 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6d3c022070 con 0x7f6d4415e3f0 2024-04-14T08:33:13.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.116+0000 7f1112ffd640 1 -- 172.21.15.172:0/2256728622 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f111401a070 con 0x7f11240cead0 2024-04-14T08:33:13.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.117+0000 7f1112ffd640 1 --2- 172.21.15.172:0/2256728622 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f110406c8f0 0x7f110406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.117+0000 7f112259c640 1 --2- 172.21.15.172:0/2256728622 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f110406c8f0 0x7f110406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.117+0000 7f57767fc640 1 -- 172.21.15.172:0/820398743 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f577c059100 con 0x7f5778054120 2024-04-14T08:33:13.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.118+0000 7f112259c640 1 --2- 172.21.15.172:0/2256728622 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f110406c8f0 0x7f110406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1118036000 tx=0x7f1118008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.123+0000 7f2a4affd640 1 -- 172.21.15.172:0/1921073403 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2a54059d40 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.133+0000 7f1112ffd640 1 -- 172.21.15.172:0/2256728622 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1114057170 con 0x7f11240cead0 2024-04-14T08:33:13.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.133+0000 7f5f0e7fc640 1 -- 172.21.15.172:0/4281589632 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5f140589d0 con 0x7f5f1004ffc0 2024-04-14T08:33:13.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.143+0000 7f1112ffd640 1 -- 172.21.15.172:0/2256728622 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1114021020 con 0x7f11240cead0 2024-04-14T08:33:13.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.507+0000 7fa1f77fe640 1 -- 172.21.15.172:0/3129666679 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa2140b41f0 con 0x7fa21406e200 2024-04-14T08:33:13.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.508+0000 7fa209ffb640 1 -- 172.21.15.172:0/3129666679 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa210059b70 con 0x7fa21406e200 2024-04-14T08:33:13.508 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.511+0000 7fa21ba36640 1 -- 172.21.15.172:0/3129666679 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa1f806c9f0 msgr2=0x7fa1f806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.511+0000 7fa21ba36640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa1f806c9f0 0x7fa1f806eeb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa204006fd0 tx=0x7fa204008040 comp rx=0 tx=0).stop 2024-04-14T08:33:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.511+0000 7fa21ba36640 1 -- 172.21.15.172:0/3129666679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa21406e200 msgr2=0x7fa214157ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.511+0000 7fa21ba36640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa21406e200 0x7fa214157ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa21000e8d0 tx=0x7fa21000ed90 comp rx=0 tx=0).stop 2024-04-14T08:33:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.511+0000 7fa21ba36640 1 -- 172.21.15.172:0/3129666679 shutdown_connections 2024-04-14T08:33:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.511+0000 7fa21ba36640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa1f806c9f0 0x7fa1f806eeb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.512+0000 7fa21ba36640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa21409c650 0x7fa21409d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.512+0000 7fa21ba36640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa214162990 0x7fa21409ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.512+0000 7fa21ba36640 1 --2- 172.21.15.172:0/3129666679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa21406e200 0x7fa214157ed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.512+0000 7fa21ba36640 1 -- 172.21.15.172:0/3129666679 >> 172.21.15.172:0/3129666679 conn(0x7fa2140b1b30 msgr2=0x7fa2140b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.512+0000 7fa21ba36640 1 -- 172.21.15.172:0/3129666679 shutdown_connections 2024-04-14T08:33:13.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.516+0000 7fa21ba36640 1 -- 172.21.15.172:0/3129666679 wait complete. 2024-04-14T08:33:13.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.519+0000 7f063b17b640 1 -- 172.21.15.172:0/2273081050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f062c0e1020 con 0x7f062c04fc00 2024-04-14T08:33:13.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.520+0000 7f06317fa640 1 -- 172.21.15.172:0/2273081050 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f06340f6f00 con 0x7f062c04fc00 2024-04-14T08:33:13.520 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:13.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.523+0000 7f061effd640 1 -- 172.21.15.172:0/2273081050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f061806c9f0 msgr2=0x7f061806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.523+0000 7f061effd640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f061806c9f0 0x7f061806eeb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f062c04def0 tx=0x7f0628009110 comp rx=0 tx=0).stop 2024-04-14T08:33:13.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.523+0000 7f061effd640 1 -- 172.21.15.172:0/2273081050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062c04fc00 msgr2=0x7f062c04ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.523+0000 7f061effd640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062c04fc00 0x7f062c04ce20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0634068bd0 tx=0x7f06340692a0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.523+0000 7f061effd640 1 -- 172.21.15.172:0/2273081050 shutdown_connections 2024-04-14T08:33:13.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.523+0000 7f061effd640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f061806c9f0 0x7f061806eeb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.523+0000 7f061effd640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062c050520 0x7f062c04ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.523+0000 7f061effd640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062c04fc00 0x7f062c04ce20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.523+0000 7f061effd640 1 --2- 172.21.15.172:0/2273081050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f062c008180 0x7f062c0d86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.523+0000 7f061effd640 1 -- 172.21.15.172:0/2273081050 >> 172.21.15.172:0/2273081050 conn(0x7f062c055340 msgr2=0x7f062c005bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.523+0000 7f061effd640 1 -- 172.21.15.172:0/2273081050 shutdown_connections 2024-04-14T08:33:13.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.523+0000 7f061effd640 1 -- 172.21.15.172:0/2273081050 wait complete. 2024-04-14T08:33:13.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2024-04-14T08:33:13.534 DEBUG:teuthology.parallel:result is None 2024-04-14T08:33:13.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.536+0000 7f1ec6ffd640 1 -- 172.21.15.172:0/1246268634 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1edc068590 con 0x7f1edc14c130 2024-04-14T08:33:13.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.537+0000 7f1ed97fa640 1 -- 172.21.15.172:0/1246268634 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ed40599a0 con 0x7f1edc14c130 2024-04-14T08:33:13.538 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:13.540 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.538+0000 7f2a60d5e640 1 -- 172.21.15.172:0/1921073403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2a40005580 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.539+0000 7f2a4affd640 1 -- 172.21.15.172:0/1921073403 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2a54059640 con 0x7f2a5c0c81c0 2024-04-14T08:33:13.541 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2024-04-14T08:33:13.542 DEBUG:teuthology.parallel:result is None 2024-04-14T08:33:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.540+0000 7f1ec6ffd640 1 -- 172.21.15.172:0/1246268634 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ec006c9c0 msgr2=0x7f1ec006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.540+0000 7f1ec6ffd640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ec006c9c0 0x7f1ec006ee80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0000f60 tx=0x7f1ed0007c80 comp rx=0 tx=0).stop 2024-04-14T08:33:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.540+0000 7f1ec6ffd640 1 -- 172.21.15.172:0/1246268634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1edc14c130 msgr2=0x7f1edc15f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.540+0000 7f1ec6ffd640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1edc14c130 0x7f1edc15f0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4007d90 tx=0x7f1ed400e4b0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.540+0000 7f1ec6ffd640 1 -- 172.21.15.172:0/1246268634 shutdown_connections 2024-04-14T08:33:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.540+0000 7f1ec6ffd640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ec006c9c0 0x7f1ec006ee80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.540+0000 7f1ec6ffd640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1edc14c130 0x7f1edc15f0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.540+0000 7f1ec6ffd640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1edc0b5540 0x7f1edc145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.541+0000 7f1ec6ffd640 1 --2- 172.21.15.172:0/1246268634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1edc0b23e0 0x7f1edc1fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.541+0000 7f1ec6ffd640 1 -- 172.21.15.172:0/1246268634 >> 172.21.15.172:0/1246268634 conn(0x7f1edc06b520 msgr2=0x7f1edc0cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.541+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5f00005850 con 0x7f5f1004ffc0 2024-04-14T08:33:13.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.541+0000 7f1ec6ffd640 1 -- 172.21.15.172:0/1246268634 shutdown_connections 2024-04-14T08:33:13.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.541+0000 7f1ec6ffd640 1 -- 172.21.15.172:0/1246268634 wait complete. 2024-04-14T08:33:13.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.541+0000 7f5f0e7fc640 1 -- 172.21.15.172:0/4281589632 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5f1405ccd0 con 0x7f5f1004ffc0 2024-04-14T08:33:13.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.541+0000 7f2a60d5e640 1 -- 172.21.15.172:0/1921073403 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3c06c8f0 msgr2=0x7f2a3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.541+0000 7f2a60d5e640 1 --2- 172.21.15.172:0/1921073403 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3c06c8f0 0x7f2a3c06edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c15c500 tx=0x7f2a4c009240 comp rx=0 tx=0).stop 2024-04-14T08:33:13.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.541+0000 7f5786bcb640 1 -- 172.21.15.172:0/820398743 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f57780575d0 con 0x7f5778054120 2024-04-14T08:33:13.545 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:13.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.542+0000 7f2a60d5e640 1 -- 172.21.15.172:0/1921073403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a5c0c81c0 msgr2=0x7f2a5c150a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.542+0000 7f2a60d5e640 1 --2- 172.21.15.172:0/1921073403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a5c0c81c0 0x7f2a5c150a50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400bd60 tx=0x7f2a54009e70 comp rx=0 tx=0).stop 2024-04-14T08:33:13.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.542+0000 7f57767fc640 1 -- 172.21.15.172:0/820398743 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f577c05d400 con 0x7f5778054120 2024-04-14T08:33:13.546 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:13.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.542+0000 7f2a60d5e640 1 -- 172.21.15.172:0/1921073403 shutdown_connections 2024-04-14T08:33:13.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.542+0000 7f2a60d5e640 1 --2- 172.21.15.172:0/1921073403 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3c06c8f0 0x7f2a3c06edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.542+0000 7f2a60d5e640 1 --2- 172.21.15.172:0/1921073403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a5c0c81c0 0x7f2a5c150a50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.542+0000 7f2a60d5e640 1 --2- 172.21.15.172:0/1921073403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a5c160f30 0x7f2a5c150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.542+0000 7f2a60d5e640 1 --2- 172.21.15.172:0/1921073403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a5c0d0cc0 0x7f2a5c0c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.542+0000 7f2a60d5e640 1 -- 172.21.15.172:0/1921073403 >> 172.21.15.172:0/1921073403 conn(0x7f2a5c09d580 msgr2=0x7f2a5c0b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.542+0000 7f2a60d5e640 1 -- 172.21.15.172:0/1921073403 shutdown_connections 2024-04-14T08:33:13.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.542+0000 7f2a60d5e640 1 -- 172.21.15.172:0/1921073403 wait complete. 2024-04-14T08:33:13.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.543+0000 7f6d4a95e640 1 -- 172.21.15.172:0/670144756 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6d4409def0 con 0x7f6d4415e3f0 2024-04-14T08:33:13.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.544+0000 7f6d40ff9640 1 -- 172.21.15.172:0/670144756 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6d3c010040 con 0x7f6d4415e3f0 2024-04-14T08:33:13.548 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:13.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.544+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5efc06ccf0 msgr2=0x7f5efc06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.544+0000 7f5f1fc1c640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5efc06ccf0 0x7f5efc06f1b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006f40 tx=0x7f5f08009240 comp rx=0 tx=0).stop 2024-04-14T08:33:13.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.544+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f1004ffc0 msgr2=0x7f5f100f2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.544+0000 7f5f1fc1c640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f1004ffc0 0x7f5f100f2a80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c440 tx=0x7f5f1400c900 comp rx=0 tx=0).stop 2024-04-14T08:33:13.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 shutdown_connections 2024-04-14T08:33:13.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5f1fc1c640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5efc06ccf0 0x7f5efc06f1b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5f1fc1c640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5f100eacf0 0x7f5f100cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5f1fc1c640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f1004ffc0 0x7f5f100f2a80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5f1fc1c640 1 --2- 172.21.15.172:0/4281589632 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5f10006e90 0x7f5f100c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 >> 172.21.15.172:0/4281589632 conn(0x7f5f10009fa0 msgr2=0x7f5f10038250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 shutdown_connections 2024-04-14T08:33:13.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5f1fc1c640 1 -- 172.21.15.172:0/4281589632 wait complete. 2024-04-14T08:33:13.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5786bcb640 1 -- 172.21.15.172:0/820398743 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f576406ccf0 msgr2=0x7f576406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5786bcb640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f576406ccf0 0x7f576406f1b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5770036000 tx=0x7f5770008030 comp rx=0 tx=0).stop 2024-04-14T08:33:13.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5786bcb640 1 -- 172.21.15.172:0/820398743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5778054120 msgr2=0x7f5778184ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5786bcb640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5778054120 0x7f5778184ef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f577c00c820 tx=0x7f577c00cc20 comp rx=0 tx=0).stop 2024-04-14T08:33:13.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5786bcb640 1 -- 172.21.15.172:0/820398743 shutdown_connections 2024-04-14T08:33:13.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5786bcb640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f576406ccf0 0x7f576406f1b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5786bcb640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57780e30b0 0x7f577800b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5786bcb640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5778054120 0x7f5778184ef0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5786bcb640 1 --2- 172.21.15.172:0/820398743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f577803e830 0x7f57780e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.545+0000 7f5786bcb640 1 -- 172.21.15.172:0/820398743 >> 172.21.15.172:0/820398743 conn(0x7f57780559d0 msgr2=0x7f5778004f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.546+0000 7f5786bcb640 1 -- 172.21.15.172:0/820398743 shutdown_connections 2024-04-14T08:33:13.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.546+0000 7f5786bcb640 1 -- 172.21.15.172:0/820398743 wait complete. 2024-04-14T08:33:13.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.547+0000 7f6d4a95e640 1 -- 172.21.15.172:0/670144756 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3006c9c0 msgr2=0x7f6d3006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.547+0000 7f6d4a95e640 1 --2- 172.21.15.172:0/670144756 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3006c9c0 0x7f6d3006ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6d34036000 tx=0x7f6d34008030 comp rx=0 tx=0).stop 2024-04-14T08:33:13.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.547+0000 7f6d4a95e640 1 -- 172.21.15.172:0/670144756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4415e3f0 msgr2=0x7f6d44158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.547+0000 7f6d4a95e640 1 --2- 172.21.15.172:0/670144756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4415e3f0 0x7f6d44158eb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009dd0 tx=0x7f6d3c00c490 comp rx=0 tx=0).stop 2024-04-14T08:33:13.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.547+0000 7f6d4a95e640 1 -- 172.21.15.172:0/670144756 shutdown_connections 2024-04-14T08:33:13.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.547+0000 7f6d4a95e640 1 --2- 172.21.15.172:0/670144756 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3006c9c0 0x7f6d3006ee80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.547+0000 7f6d4a95e640 1 --2- 172.21.15.172:0/670144756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4415e3f0 0x7f6d44158eb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.547+0000 7f6d4a95e640 1 --2- 172.21.15.172:0/670144756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d440b3840 0x7f6d4409c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.547+0000 7f6d4a95e640 1 --2- 172.21.15.172:0/670144756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4406e910 0x7f6d4413ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.547+0000 7f6d4a95e640 1 -- 172.21.15.172:0/670144756 >> 172.21.15.172:0/670144756 conn(0x7f6d440b1b30 msgr2=0x7f6d440b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.548+0000 7f6d4a95e640 1 -- 172.21.15.172:0/670144756 shutdown_connections 2024-04-14T08:33:13.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.548+0000 7f6d4a95e640 1 -- 172.21.15.172:0/670144756 wait complete. 2024-04-14T08:33:13.560 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-04-14T08:33:13.560 DEBUG:teuthology.parallel:result is None 2024-04-14T08:33:13.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-04-14T08:33:13.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.560+0000 7f1128c60640 1 -- 172.21.15.172:0/2256728622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1108005850 con 0x7f11240cead0 2024-04-14T08:33:13.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.560+0000 7f1112ffd640 1 -- 172.21.15.172:0/2256728622 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f11140597d0 con 0x7f11240cead0 2024-04-14T08:33:13.561 DEBUG:teuthology.parallel:result is None 2024-04-14T08:33:13.561 INFO:teuthology.orchestra.run.smithi172.stdout:55834574857 2024-04-14T08:33:13.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.563+0000 7f1128c60640 1 -- 172.21.15.172:0/2256728622 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f110406c8f0 msgr2=0x7f110406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.563+0000 7f1128c60640 1 --2- 172.21.15.172:0/2256728622 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f110406c8f0 0x7f110406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1118036000 tx=0x7f1118008030 comp rx=0 tx=0).stop 2024-04-14T08:33:13.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.563+0000 7f1128c60640 1 -- 172.21.15.172:0/2256728622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240cead0 msgr2=0x7f112406b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.563+0000 7f1128c60640 1 --2- 172.21.15.172:0/2256728622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240cead0 0x7f112406b520 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f111400c400 tx=0x7f111400c8c0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.563+0000 7f1128c60640 1 -- 172.21.15.172:0/2256728622 shutdown_connections 2024-04-14T08:33:13.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.563+0000 7f1128c60640 1 --2- 172.21.15.172:0/2256728622 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f110406c8f0 0x7f110406edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.563+0000 7f1128c60640 1 --2- 172.21.15.172:0/2256728622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112406ba60 0x7f1124151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.563+0000 7f1128c60640 1 --2- 172.21.15.172:0/2256728622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240cead0 0x7f112406b520 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.564+0000 7f1128c60640 1 --2- 172.21.15.172:0/2256728622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1124072ac0 0x7f11240b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.564+0000 7f1128c60640 1 -- 172.21.15.172:0/2256728622 >> 172.21.15.172:0/2256728622 conn(0x7f11240cd980 msgr2=0x7f11240ae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.564+0000 7f1128c60640 1 -- 172.21.15.172:0/2256728622 shutdown_connections 2024-04-14T08:33:13.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.564+0000 7f1128c60640 1 -- 172.21.15.172:0/2256728622 wait complete. 2024-04-14T08:33:13.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2024-04-14T08:33:13.566 DEBUG:teuthology.parallel:result is None 2024-04-14T08:33:13.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2024-04-14T08:33:13.567 DEBUG:teuthology.parallel:result is None 2024-04-14T08:33:13.568 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-04-14T08:33:13.568 DEBUG:teuthology.parallel:result is None 2024-04-14T08:33:13.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2024-04-14T08:33:13.576 DEBUG:teuthology.parallel:result is None 2024-04-14T08:33:13.576 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-14T08:33:13.576 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:33:13.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.716+0000 7f8415374640 1 Processor -- start 2024-04-14T08:33:13.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.716+0000 7f8415374640 1 -- start start 2024-04-14T08:33:13.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f8415374640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f84100b1df0 0x7f84100b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f8415374640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f84100b4040 0x7f84100b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f8415374640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f841015c770 con 0x7f84100b21d0 2024-04-14T08:33:13.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f8415374640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f84100b3430 con 0x7f84100b4040 2024-04-14T08:33:13.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f8415374640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f84100b3700 con 0x7f84100b1df0 2024-04-14T08:33:13.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f840f7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f84100b4040 0x7f84100b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f840e7fc640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f84100b21d0 0x7f841015c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35722/0 (socket says 172.21.15.172:35722) 2024-04-14T08:33:13.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f840e7fc640 1 -- 172.21.15.172:0/1669538202 learned_addr learned my addr 172.21.15.172:0/1669538202 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:13.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f840effd640 1 --2- 172.21.15.172:0/1669538202 >> v2:172.21.15.177:3300/0 conn(0x7f84100b1df0 0x7f84100b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f840f7fe640 1 -- 172.21.15.172:0/1669538202 >> v2:172.21.15.177:3300/0 conn(0x7f84100b1df0 msgr2=0x7f84100b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:13.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f840f7fe640 1 --2- 172.21.15.172:0/1669538202 >> v2:172.21.15.177:3300/0 conn(0x7f84100b1df0 0x7f84100b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f840f7fe640 1 -- 172.21.15.172:0/1669538202 >> v1:172.21.15.172:6789/0 conn(0x7f84100b21d0 legacy=0x7f841015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f840dffb640 1 -- 172.21.15.172:0/1669538202 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 226424236 0 0) 0x7f841015c770 con 0x7f84100b21d0 2024-04-14T08:33:13.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.717+0000 7f840f7fe640 1 -- 172.21.15.172:0/1669538202 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84100b39e0 con 0x7f84100b4040 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.718+0000 7f840f7fe640 1 --2- 172.21.15.172:0/1669538202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100b4040 0x7f84100b4460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f841006f4b0 tx=0x7f840802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419d573cfb20b889 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.718+0000 7f840dffb640 1 -- 172.21.15.172:0/1669538202 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080315a0 con 0x7f84100b4040 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.718+0000 7f840dffb640 1 -- 172.21.15.172:0/1669538202 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8408037070 con 0x7f84100b4040 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.718+0000 7f840dffb640 1 -- 172.21.15.172:0/1669538202 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408031960 con 0x7f84100b4040 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.719+0000 7f8415374640 1 -- 172.21.15.172:0/1669538202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100b4040 msgr2=0x7f84100b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.719+0000 7f8415374640 1 --2- 172.21.15.172:0/1669538202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100b4040 0x7f84100b4460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f841006f4b0 tx=0x7f840802b840 comp rx=0 tx=0).stop 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.719+0000 7f8415374640 1 -- 172.21.15.172:0/1669538202 shutdown_connections 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.719+0000 7f8415374640 1 --2- 172.21.15.172:0/1669538202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100b4040 0x7f84100b4460 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.719+0000 7f8415374640 1 --2- 172.21.15.172:0/1669538202 >> v2:172.21.15.177:3300/0 conn(0x7f84100b1df0 0x7f84100b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.719+0000 7f8415374640 1 -- 172.21.15.172:0/1669538202 >> 172.21.15.172:0/1669538202 conn(0x7f841006ec30 msgr2=0x7f841006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.719+0000 7f8415374640 1 -- 172.21.15.172:0/1669538202 shutdown_connections 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.719+0000 7f8415374640 1 -- 172.21.15.172:0/1669538202 wait complete. 2024-04-14T08:33:13.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.720+0000 7f8415374640 1 Processor -- start 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.720+0000 7f8415374640 1 -- start start 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f8415374640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100b1df0 0x7f84101446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f8415374640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84100b21d0 0x7f84101f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f8415374640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84100b4040 0x7f84100ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f8415374640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84100b49a0 con 0x7f84100b1df0 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f8415374640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f841015bb40 con 0x7f84100b4040 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f8415374640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f841015b7f0 con 0x7f84100b21d0 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f840effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100b1df0 0x7f84101446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f840effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100b1df0 0x7f84101446a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49666/0 (socket says 172.21.15.172:49666) 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f840effd640 1 -- 172.21.15.172:0/3233024899 learned_addr learned my addr 172.21.15.172:0/3233024899 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f840f7fe640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84100b4040 0x7f84100ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f840e7fc640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84100b21d0 0x7f84101f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f840effd640 1 -- 172.21.15.172:0/3233024899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84100b21d0 msgr2=0x7f84101f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f840effd640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84100b21d0 0x7f84101f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f840effd640 1 -- 172.21.15.172:0/3233024899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84100b4040 msgr2=0x7f84100ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f840effd640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84100b4040 0x7f84100ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.721+0000 7f840effd640 1 -- 172.21.15.172:0/3233024899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84100afe10 con 0x7f84100b1df0 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.722+0000 7f840effd640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100b1df0 0x7f84101446a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f840000efc0 tx=0x7f840000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.722+0000 7f83f77fe640 1 -- 172.21.15.172:0/3233024899 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400018020 con 0x7f84100b1df0 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.722+0000 7f83f77fe640 1 -- 172.21.15.172:0/3233024899 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8400014480 con 0x7f84100b1df0 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.722+0000 7f8415374640 1 -- 172.21.15.172:0/3233024899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84100b00f0 con 0x7f84100b1df0 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.722+0000 7f83f77fe640 1 -- 172.21.15.172:0/3233024899 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400014850 con 0x7f84100b1df0 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.722+0000 7f8415374640 1 -- 172.21.15.172:0/3233024899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84100b06b0 con 0x7f84100b1df0 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.723+0000 7f8415374640 1 -- 172.21.15.172:0/3233024899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8410068520 con 0x7f84100b1df0 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.723+0000 7f83f77fe640 1 -- 172.21.15.172:0/3233024899 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8400005d10 con 0x7f84100b1df0 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.724+0000 7f83f77fe640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f83fc06c8f0 0x7f83fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.724+0000 7f83f77fe640 1 -- 172.21.15.172:0/3233024899 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f840008cb10 con 0x7f84100b1df0 2024-04-14T08:33:13.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.724+0000 7f840e7fc640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f83fc06c8f0 0x7f83fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:13.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.724+0000 7f840e7fc640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f83fc06c8f0 0x7f83fc06edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f84100af4e0 tx=0x7f8404007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:13.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.728+0000 7f83f77fe640 1 -- 172.21.15.172:0/3233024899 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8400022030 con 0x7f84100b1df0 2024-04-14T08:33:13.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.900+0000 7f8415374640 1 -- 172.21.15.172:0/3233024899 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84100afbc0 con 0x7f83fc06c8f0 2024-04-14T08:33:13.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.908+0000 7f83f77fe640 1 -- 172.21.15.172:0/3233024899 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f84100afbc0 con 0x7f83fc06c8f0 2024-04-14T08:33:13.911 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:13.913 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:33:13.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.911+0000 7f8415374640 1 -- 172.21.15.172:0/3233024899 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f83fc06c8f0 msgr2=0x7f83fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.911+0000 7f8415374640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f83fc06c8f0 0x7f83fc06edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f84100af4e0 tx=0x7f8404007d50 comp rx=0 tx=0).stop 2024-04-14T08:33:13.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.911+0000 7f8415374640 1 -- 172.21.15.172:0/3233024899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100b1df0 msgr2=0x7f84101446a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:13.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.911+0000 7f8415374640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100b1df0 0x7f84101446a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f840000efc0 tx=0x7f840000c490 comp rx=0 tx=0).stop 2024-04-14T08:33:13.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.911+0000 7f8415374640 1 -- 172.21.15.172:0/3233024899 shutdown_connections 2024-04-14T08:33:13.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.911+0000 7f8415374640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f83fc06c8f0 0x7f83fc06edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.911+0000 7f8415374640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84100b4040 0x7f84100ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.911+0000 7f8415374640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84100b21d0 0x7f84101f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.911+0000 7f8415374640 1 --2- 172.21.15.172:0/3233024899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84100b1df0 0x7f84101446a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:13.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.911+0000 7f8415374640 1 -- 172.21.15.172:0/3233024899 >> 172.21.15.172:0/3233024899 conn(0x7f841006ec30 msgr2=0x7f8410071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:13.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.911+0000 7f8415374640 1 -- 172.21.15.172:0/3233024899 shutdown_connections 2024-04-14T08:33:13.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:13.911+0000 7f8415374640 1 -- 172.21.15.172:0/3233024899 wait complete. 2024-04-14T08:33:13.926 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-14T08:33:13.125275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219480,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755240,"statfs":{"total":773094113280,"available":772869365760,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002879"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219633+0000","last_change":"2024-04-14T08:32:53.196809+0000","last_active":"2024-04-14T08:33:01.219633+0000","last_peered":"2024-04-14T08:33:01.219633+0000","last_clean":"2024-04-14T08:33:01.219633+0000","last_became_active":"2024-04-14T08:32:53.196554+0000","last_became_peered":"2024-04-14T08:32:53.196554+0000","last_unstale":"2024-04-14T08:33:01.219633+0000","last_undegraded":"2024-04-14T08:33:01.219633+0000","last_fullsized":"2024-04-14T08:33:01.219633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:43:41.734066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225244+0000","last_change":"2024-04-14T08:32:54.204652+0000","last_active":"2024-04-14T08:32:59.225244+0000","last_peered":"2024-04-14T08:32:59.225244+0000","last_clean":"2024-04-14T08:32:59.225244+0000","last_became_active":"2024-04-14T08:32:54.204361+0000","last_became_peered":"2024-04-14T08:32:54.204361+0000","last_unstale":"2024-04-14T08:32:59.225244+0000","last_undegraded":"2024-04-14T08:32:59.225244+0000","last_fullsized":"2024-04-14T08:32:59.225244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224356+0000","last_change":"2024-04-14T08:32:58.244817+0000","last_active":"2024-04-14T08:32:59.224356+0000","last_peered":"2024-04-14T08:32:59.224356+0000","last_clean":"2024-04-14T08:32:59.224356+0000","last_became_active":"2024-04-14T08:32:58.243913+0000","last_became_peered":"2024-04-14T08:32:58.243913+0000","last_unstale":"2024-04-14T08:32:59.224356+0000","last_undegraded":"2024-04-14T08:32:59.224356+0000","last_fullsized":"2024-04-14T08:32:59.224356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:53.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225340+0000","last_change":"2024-04-14T08:32:53.202277+0000","last_active":"2024-04-14T08:32:59.225340+0000","last_peered":"2024-04-14T08:32:59.225340+0000","last_clean":"2024-04-14T08:32:59.225340+0000","last_became_active":"2024-04-14T08:32:53.201980+0000","last_became_peered":"2024-04-14T08:32:53.201980+0000","last_unstale":"2024-04-14T08:32:59.225340+0000","last_undegraded":"2024-04-14T08:32:59.225340+0000","last_fullsized":"2024-04-14T08:32:59.225340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:22:15.886932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220320+0000","last_change":"2024-04-14T08:32:54.211674+0000","last_active":"2024-04-14T08:33:01.220320+0000","last_peered":"2024-04-14T08:33:01.220320+0000","last_clean":"2024-04-14T08:33:01.220320+0000","last_became_active":"2024-04-14T08:32:54.211323+0000","last_became_peered":"2024-04-14T08:32:54.211323+0000","last_unstale":"2024-04-14T08:33:01.220320+0000","last_undegraded":"2024-04-14T08:33:01.220320+0000","last_fullsized":"2024-04-14T08:33:01.220320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224678+0000","last_change":"2024-04-14T08:32:58.242594+0000","last_active":"2024-04-14T08:32:59.224678+0000","last_peered":"2024-04-14T08:32:59.224678+0000","last_clean":"2024-04-14T08:32:59.224678+0000","last_became_active":"2024-04-14T08:32:58.242238+0000","last_became_peered":"2024-04-14T08:32:58.242238+0000","last_unstale":"2024-04-14T08:32:59.224678+0000","last_undegraded":"2024-04-14T08:32:59.224678+0000","last_fullsized":"2024-04-14T08:32:59.224678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:45.716305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226152+0000","last_change":"2024-04-14T08:32:53.200133+0000","last_active":"2024-04-14T08:32:59.226152+0000","last_peered":"2024-04-14T08:32:59.226152+0000","last_clean":"2024-04-14T08:32:59.226152+0000","last_became_active":"2024-04-14T08:32:53.199380+0000","last_became_peered":"2024-04-14T08:32:53.199380+0000","last_unstale":"2024-04-14T08:32:59.226152+0000","last_undegraded":"2024-04-14T08:32:59.226152+0000","last_fullsized":"2024-04-14T08:32:59.226152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405017+0000","last_change":"2024-04-14T08:32:54.208922+0000","last_active":"2024-04-14T08:32:59.405017+0000","last_peered":"2024-04-14T08:32:59.405017+0000","last_clean":"2024-04-14T08:32:59.405017+0000","last_became_active":"2024-04-14T08:32:54.206845+0000","last_became_peered":"2024-04-14T08:32:54.206845+0000","last_unstale":"2024-04-14T08:32:59.405017+0000","last_undegraded":"2024-04-14T08:32:59.405017+0000","last_fullsized":"2024-04-14T08:32:59.405017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225258+0000","last_change":"2024-04-14T08:32:58.242683+0000","last_active":"2024-04-14T08:32:59.225258+0000","last_peered":"2024-04-14T08:32:59.225258+0000","last_clean":"2024-04-14T08:32:59.225258+0000","last_became_active":"2024-04-14T08:32:58.242189+0000","last_became_peered":"2024-04-14T08:32:58.242189+0000","last_unstale":"2024-04-14T08:32:59.225258+0000","last_undegraded":"2024-04-14T08:32:59.225258+0000","last_fullsized":"2024-04-14T08:32:59.225258+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:52.084133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225875+0000","last_change":"2024-04-14T08:32:53.200880+0000","last_active":"2024-04-14T08:32:59.225875+0000","last_peered":"2024-04-14T08:32:59.225875+0000","last_clean":"2024-04-14T08:32:59.225875+0000","last_became_active":"2024-04-14T08:32:53.200615+0000","last_became_peered":"2024-04-14T08:32:53.200615+0000","last_unstale":"2024-04-14T08:32:59.225875+0000","last_undegraded":"2024-04-14T08:32:59.225875+0000","last_fullsized":"2024-04-14T08:32:59.225875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:40:24.515179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405104+0000","last_change":"2024-04-14T08:32:54.207091+0000","last_active":"2024-04-14T08:32:59.405104+0000","last_peered":"2024-04-14T08:32:59.405104+0000","last_clean":"2024-04-14T08:32:59.405104+0000","last_became_active":"2024-04-14T08:32:54.206685+0000","last_became_peered":"2024-04-14T08:32:54.206685+0000","last_unstale":"2024-04-14T08:32:59.405104+0000","last_undegraded":"2024-04-14T08:32:59.405104+0000","last_fullsized":"2024-04-14T08:32:59.405104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:41:14.768103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224936+0000","last_change":"2024-04-14T08:32:58.237435+0000","last_active":"2024-04-14T08:32:59.224936+0000","last_peered":"2024-04-14T08:32:59.224936+0000","last_clean":"2024-04-14T08:32:59.224936+0000","last_became_active":"2024-04-14T08:32:58.237056+0000","last_became_peered":"2024-04-14T08:32:58.237056+0000","last_unstale":"2024-04-14T08:32:59.224936+0000","last_undegraded":"2024-04-14T08:32:59.224936+0000","last_fullsized":"2024-04-14T08:32:59.224936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:20.577556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115125+0000","last_change":"2024-04-14T08:32:54.202442+0000","last_active":"2024-04-14T08:33:01.115125+0000","last_peered":"2024-04-14T08:33:01.115125+0000","last_clean":"2024-04-14T08:33:01.115125+0000","last_became_active":"2024-04-14T08:32:54.201860+0000","last_became_peered":"2024-04-14T08:32:54.201860+0000","last_unstale":"2024-04-14T08:33:01.115125+0000","last_undegraded":"2024-04-14T08:33:01.115125+0000","last_fullsized":"2024-04-14T08:33:01.115125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226036+0000","last_change":"2024-04-14T08:32:53.194232+0000","last_active":"2024-04-14T08:32:59.226036+0000","last_peered":"2024-04-14T08:32:59.226036+0000","last_clean":"2024-04-14T08:32:59.226036+0000","last_became_active":"2024-04-14T08:32:53.193961+0000","last_became_peered":"2024-04-14T08:32:53.193961+0000","last_unstale":"2024-04-14T08:32:59.226036+0000","last_undegraded":"2024-04-14T08:32:59.226036+0000","last_fullsized":"2024-04-14T08:32:59.226036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:32.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226274+0000","last_change":"2024-04-14T08:32:58.239778+0000","last_active":"2024-04-14T08:32:59.226274+0000","last_peered":"2024-04-14T08:32:59.226274+0000","last_clean":"2024-04-14T08:32:59.226274+0000","last_became_active":"2024-04-14T08:32:58.239445+0000","last_became_peered":"2024-04-14T08:32:58.239445+0000","last_unstale":"2024-04-14T08:32:59.226274+0000","last_undegraded":"2024-04-14T08:32:59.226274+0000","last_fullsized":"2024-04-14T08:32:59.226274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:53.001768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224657+0000","last_change":"2024-04-14T08:32:54.202322+0000","last_active":"2024-04-14T08:32:59.224657+0000","last_peered":"2024-04-14T08:32:59.224657+0000","last_clean":"2024-04-14T08:32:59.224657+0000","last_became_active":"2024-04-14T08:32:54.202065+0000","last_became_peered":"2024-04-14T08:32:54.202065+0000","last_unstale":"2024-04-14T08:32:59.224657+0000","last_undegraded":"2024-04-14T08:32:59.224657+0000","last_fullsized":"2024-04-14T08:32:59.224657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610551+0000","last_change":"2024-04-14T08:32:53.195288+0000","last_active":"2024-04-14T08:33:00.610551+0000","last_peered":"2024-04-14T08:33:00.610551+0000","last_clean":"2024-04-14T08:33:00.610551+0000","last_became_active":"2024-04-14T08:32:53.195009+0000","last_became_peered":"2024-04-14T08:32:53.195009+0000","last_unstale":"2024-04-14T08:33:00.610551+0000","last_undegraded":"2024-04-14T08:33:00.610551+0000","last_fullsized":"2024-04-14T08:33:00.610551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225775+0000","last_change":"2024-04-14T08:32:58.238192+0000","last_active":"2024-04-14T08:32:59.225775+0000","last_peered":"2024-04-14T08:32:59.225775+0000","last_clean":"2024-04-14T08:32:59.225775+0000","last_became_active":"2024-04-14T08:32:58.237917+0000","last_became_peered":"2024-04-14T08:32:58.237917+0000","last_unstale":"2024-04-14T08:32:59.225775+0000","last_undegraded":"2024-04-14T08:32:59.225775+0000","last_fullsized":"2024-04-14T08:32:59.225775+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:41:18.809183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224457+0000","last_change":"2024-04-14T08:32:54.195485+0000","last_active":"2024-04-14T08:32:59.224457+0000","last_peered":"2024-04-14T08:32:59.224457+0000","last_clean":"2024-04-14T08:32:59.224457+0000","last_became_active":"2024-04-14T08:32:54.195192+0000","last_became_peered":"2024-04-14T08:32:54.195192+0000","last_unstale":"2024-04-14T08:32:59.224457+0000","last_undegraded":"2024-04-14T08:32:59.224457+0000","last_fullsized":"2024-04-14T08:32:59.224457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610087+0000","last_change":"2024-04-14T08:32:53.193700+0000","last_active":"2024-04-14T08:33:00.610087+0000","last_peered":"2024-04-14T08:33:00.610087+0000","last_clean":"2024-04-14T08:33:00.610087+0000","last_became_active":"2024-04-14T08:32:53.193412+0000","last_became_peered":"2024-04-14T08:32:53.193412+0000","last_unstale":"2024-04-14T08:33:00.610087+0000","last_undegraded":"2024-04-14T08:33:00.610087+0000","last_fullsized":"2024-04-14T08:33:00.610087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610793+0000","last_change":"2024-04-14T08:32:58.246138+0000","last_active":"2024-04-14T08:33:00.610793+0000","last_peered":"2024-04-14T08:33:00.610793+0000","last_clean":"2024-04-14T08:33:00.610793+0000","last_became_active":"2024-04-14T08:32:58.245731+0000","last_became_peered":"2024-04-14T08:32:58.245731+0000","last_unstale":"2024-04-14T08:33:00.610793+0000","last_undegraded":"2024-04-14T08:33:00.610793+0000","last_fullsized":"2024-04-14T08:33:00.610793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:54.763380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224951+0000","last_change":"2024-04-14T08:32:54.203712+0000","last_active":"2024-04-14T08:32:59.224951+0000","last_peered":"2024-04-14T08:32:59.224951+0000","last_clean":"2024-04-14T08:32:59.224951+0000","last_became_active":"2024-04-14T08:32:54.203433+0000","last_became_peered":"2024-04-14T08:32:54.203433+0000","last_unstale":"2024-04-14T08:32:59.224951+0000","last_undegraded":"2024-04-14T08:32:59.224951+0000","last_fullsized":"2024-04-14T08:32:59.224951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:15.534364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224734+0000","last_change":"2024-04-14T08:32:53.197850+0000","last_active":"2024-04-14T08:32:59.224734+0000","last_peered":"2024-04-14T08:32:59.224734+0000","last_clean":"2024-04-14T08:32:59.224734+0000","last_became_active":"2024-04-14T08:32:53.197575+0000","last_became_peered":"2024-04-14T08:32:53.197575+0000","last_unstale":"2024-04-14T08:32:59.224734+0000","last_undegraded":"2024-04-14T08:32:59.224734+0000","last_fullsized":"2024-04-14T08:32:59.224734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:04:36.870813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225529+0000","last_change":"2024-04-14T08:32:58.243185+0000","last_active":"2024-04-14T08:32:59.225529+0000","last_peered":"2024-04-14T08:32:59.225529+0000","last_clean":"2024-04-14T08:32:59.225529+0000","last_became_active":"2024-04-14T08:32:58.242880+0000","last_became_peered":"2024-04-14T08:32:58.242880+0000","last_unstale":"2024-04-14T08:32:59.225529+0000","last_undegraded":"2024-04-14T08:32:59.225529+0000","last_fullsized":"2024-04-14T08:32:59.225529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:47.641204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220461+0000","last_change":"2024-04-14T08:32:54.194367+0000","last_active":"2024-04-14T08:33:01.220461+0000","last_peered":"2024-04-14T08:33:01.220461+0000","last_clean":"2024-04-14T08:33:01.220461+0000","last_became_active":"2024-04-14T08:32:54.194111+0000","last_became_peered":"2024-04-14T08:32:54.194111+0000","last_unstale":"2024-04-14T08:33:01.220461+0000","last_undegraded":"2024-04-14T08:33:01.220461+0000","last_fullsized":"2024-04-14T08:33:01.220461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:05:03.511225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115632+0000","last_change":"2024-04-14T08:32:53.194969+0000","last_active":"2024-04-14T08:33:01.115632+0000","last_peered":"2024-04-14T08:33:01.115632+0000","last_clean":"2024-04-14T08:33:01.115632+0000","last_became_active":"2024-04-14T08:32:53.194665+0000","last_became_peered":"2024-04-14T08:32:53.194665+0000","last_unstale":"2024-04-14T08:33:01.115632+0000","last_undegraded":"2024-04-14T08:33:01.115632+0000","last_fullsized":"2024-04-14T08:33:01.115632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224778+0000","last_change":"2024-04-14T08:32:58.237006+0000","last_active":"2024-04-14T08:32:59.224778+0000","last_peered":"2024-04-14T08:32:59.224778+0000","last_clean":"2024-04-14T08:32:59.224778+0000","last_became_active":"2024-04-14T08:32:58.236366+0000","last_became_peered":"2024-04-14T08:32:58.236366+0000","last_unstale":"2024-04-14T08:32:59.224778+0000","last_undegraded":"2024-04-14T08:32:59.224778+0000","last_fullsized":"2024-04-14T08:32:59.224778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:49.446712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225123+0000","last_change":"2024-04-14T08:32:54.205064+0000","last_active":"2024-04-14T08:32:59.225123+0000","last_peered":"2024-04-14T08:32:59.225123+0000","last_clean":"2024-04-14T08:32:59.225123+0000","last_became_active":"2024-04-14T08:32:54.203946+0000","last_became_peered":"2024-04-14T08:32:54.203946+0000","last_unstale":"2024-04-14T08:32:59.225123+0000","last_undegraded":"2024-04-14T08:32:59.225123+0000","last_fullsized":"2024-04-14T08:32:59.225123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:16.536477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610666+0000","last_change":"2024-04-14T08:32:53.198148+0000","last_active":"2024-04-14T08:33:00.610666+0000","last_peered":"2024-04-14T08:33:00.610666+0000","last_clean":"2024-04-14T08:33:00.610666+0000","last_became_active":"2024-04-14T08:32:53.197878+0000","last_became_peered":"2024-04-14T08:32:53.197878+0000","last_unstale":"2024-04-14T08:33:00.610666+0000","last_undegraded":"2024-04-14T08:33:00.610666+0000","last_fullsized":"2024-04-14T08:33:00.610666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224235+0000","last_change":"2024-04-14T08:32:58.246772+0000","last_active":"2024-04-14T08:32:59.224235+0000","last_peered":"2024-04-14T08:32:59.224235+0000","last_clean":"2024-04-14T08:32:59.224235+0000","last_became_active":"2024-04-14T08:32:58.246388+0000","last_became_peered":"2024-04-14T08:32:58.246388+0000","last_unstale":"2024-04-14T08:32:59.224235+0000","last_undegraded":"2024-04-14T08:32:59.224235+0000","last_fullsized":"2024-04-14T08:32:59.224235+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:54.763380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610409+0000","last_change":"2024-04-14T08:32:54.194202+0000","last_active":"2024-04-14T08:33:00.610409+0000","last_peered":"2024-04-14T08:33:00.610409+0000","last_clean":"2024-04-14T08:33:00.610409+0000","last_became_active":"2024-04-14T08:32:54.193938+0000","last_became_peered":"2024-04-14T08:32:54.193938+0000","last_unstale":"2024-04-14T08:33:00.610409+0000","last_undegraded":"2024-04-14T08:33:00.610409+0000","last_fullsized":"2024-04-14T08:33:00.610409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226263+0000","last_change":"2024-04-14T08:32:53.202270+0000","last_active":"2024-04-14T08:32:59.226263+0000","last_peered":"2024-04-14T08:32:59.226263+0000","last_clean":"2024-04-14T08:32:59.226263+0000","last_became_active":"2024-04-14T08:32:53.201945+0000","last_became_peered":"2024-04-14T08:32:53.201945+0000","last_unstale":"2024-04-14T08:32:59.226263+0000","last_undegraded":"2024-04-14T08:32:59.226263+0000","last_fullsized":"2024-04-14T08:32:59.226263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219955+0000","last_change":"2024-04-14T08:32:58.248277+0000","last_active":"2024-04-14T08:33:01.219955+0000","last_peered":"2024-04-14T08:33:01.219955+0000","last_clean":"2024-04-14T08:33:01.219955+0000","last_became_active":"2024-04-14T08:32:58.247535+0000","last_became_peered":"2024-04-14T08:32:58.247535+0000","last_unstale":"2024-04-14T08:33:01.219955+0000","last_undegraded":"2024-04-14T08:33:01.219955+0000","last_fullsized":"2024-04-14T08:33:01.219955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:05.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405198+0000","last_change":"2024-04-14T08:32:54.203448+0000","last_active":"2024-04-14T08:32:59.405198+0000","last_peered":"2024-04-14T08:32:59.405198+0000","last_clean":"2024-04-14T08:32:59.405198+0000","last_became_active":"2024-04-14T08:32:54.203170+0000","last_became_peered":"2024-04-14T08:32:54.203170+0000","last_unstale":"2024-04-14T08:32:59.405198+0000","last_undegraded":"2024-04-14T08:32:59.405198+0000","last_fullsized":"2024-04-14T08:32:59.405198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:01.152061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610433+0000","last_change":"2024-04-14T08:32:53.187816+0000","last_active":"2024-04-14T08:33:00.610433+0000","last_peered":"2024-04-14T08:33:00.610433+0000","last_clean":"2024-04-14T08:33:00.610433+0000","last_became_active":"2024-04-14T08:32:53.187519+0000","last_became_peered":"2024-04-14T08:32:53.187519+0000","last_unstale":"2024-04-14T08:33:00.610433+0000","last_undegraded":"2024-04-14T08:33:00.610433+0000","last_fullsized":"2024-04-14T08:33:00.610433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610230+0000","last_change":"2024-04-14T08:32:58.246145+0000","last_active":"2024-04-14T08:33:00.610230+0000","last_peered":"2024-04-14T08:33:00.610230+0000","last_clean":"2024-04-14T08:33:00.610230+0000","last_became_active":"2024-04-14T08:32:58.245792+0000","last_became_peered":"2024-04-14T08:32:58.245792+0000","last_unstale":"2024-04-14T08:33:00.610230+0000","last_undegraded":"2024-04-14T08:33:00.610230+0000","last_fullsized":"2024-04-14T08:33:00.610230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:05.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225311+0000","last_change":"2024-04-14T08:32:54.192476+0000","last_active":"2024-04-14T08:32:59.225311+0000","last_peered":"2024-04-14T08:32:59.225311+0000","last_clean":"2024-04-14T08:32:59.225311+0000","last_became_active":"2024-04-14T08:32:54.192182+0000","last_became_peered":"2024-04-14T08:32:54.192182+0000","last_unstale":"2024-04-14T08:32:59.225311+0000","last_undegraded":"2024-04-14T08:32:59.225311+0000","last_fullsized":"2024-04-14T08:32:59.225311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:05.453493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405000+0000","last_change":"2024-04-14T08:32:53.201969+0000","last_active":"2024-04-14T08:32:59.405000+0000","last_peered":"2024-04-14T08:32:59.405000+0000","last_clean":"2024-04-14T08:32:59.405000+0000","last_became_active":"2024-04-14T08:32:53.201611+0000","last_became_peered":"2024-04-14T08:32:53.201611+0000","last_unstale":"2024-04-14T08:32:59.405000+0000","last_undegraded":"2024-04-14T08:32:59.405000+0000","last_fullsized":"2024-04-14T08:32:59.405000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610320+0000","last_change":"2024-04-14T08:32:58.249518+0000","last_active":"2024-04-14T08:33:00.610320+0000","last_peered":"2024-04-14T08:33:00.610320+0000","last_clean":"2024-04-14T08:33:00.610320+0000","last_became_active":"2024-04-14T08:32:58.249079+0000","last_became_peered":"2024-04-14T08:32:58.249079+0000","last_unstale":"2024-04-14T08:33:00.610320+0000","last_undegraded":"2024-04-14T08:33:00.610320+0000","last_fullsized":"2024-04-14T08:33:00.610320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:16.724895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610081+0000","last_change":"2024-04-14T08:32:54.194141+0000","last_active":"2024-04-14T08:33:00.610081+0000","last_peered":"2024-04-14T08:33:00.610081+0000","last_clean":"2024-04-14T08:33:00.610081+0000","last_became_active":"2024-04-14T08:32:54.193884+0000","last_became_peered":"2024-04-14T08:32:54.193884+0000","last_unstale":"2024-04-14T08:33:00.610081+0000","last_undegraded":"2024-04-14T08:33:00.610081+0000","last_fullsized":"2024-04-14T08:33:00.610081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278559+0000","last_change":"2024-04-14T08:32:53.196690+0000","last_active":"2024-04-14T08:33:01.278559+0000","last_peered":"2024-04-14T08:33:01.278559+0000","last_clean":"2024-04-14T08:33:01.278559+0000","last_became_active":"2024-04-14T08:32:53.195977+0000","last_became_peered":"2024-04-14T08:32:53.195977+0000","last_unstale":"2024-04-14T08:33:01.278559+0000","last_undegraded":"2024-04-14T08:33:01.278559+0000","last_fullsized":"2024-04-14T08:33:01.278559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610859+0000","last_change":"2024-04-14T08:32:58.249862+0000","last_active":"2024-04-14T08:33:00.610859+0000","last_peered":"2024-04-14T08:33:00.610859+0000","last_clean":"2024-04-14T08:33:00.610859+0000","last_became_active":"2024-04-14T08:32:58.248928+0000","last_became_peered":"2024-04-14T08:32:58.248928+0000","last_unstale":"2024-04-14T08:33:00.610859+0000","last_undegraded":"2024-04-14T08:33:00.610859+0000","last_fullsized":"2024-04-14T08:33:00.610859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:53.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219816+0000","last_change":"2024-04-14T08:32:54.206032+0000","last_active":"2024-04-14T08:33:01.219816+0000","last_peered":"2024-04-14T08:33:01.219816+0000","last_clean":"2024-04-14T08:33:01.219816+0000","last_became_active":"2024-04-14T08:32:54.205781+0000","last_became_peered":"2024-04-14T08:32:54.205781+0000","last_unstale":"2024-04-14T08:33:01.219816+0000","last_undegraded":"2024-04-14T08:33:01.219816+0000","last_fullsized":"2024-04-14T08:33:01.219816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:25:28.934928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224825+0000","last_change":"2024-04-14T08:32:53.195805+0000","last_active":"2024-04-14T08:32:59.224825+0000","last_peered":"2024-04-14T08:32:59.224825+0000","last_clean":"2024-04-14T08:32:59.224825+0000","last_became_active":"2024-04-14T08:32:53.195442+0000","last_became_peered":"2024-04-14T08:32:53.195442+0000","last_unstale":"2024-04-14T08:32:59.224825+0000","last_undegraded":"2024-04-14T08:32:59.224825+0000","last_fullsized":"2024-04-14T08:32:59.224825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:48.807150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220403+0000","last_change":"2024-04-14T08:32:58.247104+0000","last_active":"2024-04-14T08:33:01.220403+0000","last_peered":"2024-04-14T08:33:01.220403+0000","last_clean":"2024-04-14T08:33:01.220403+0000","last_became_active":"2024-04-14T08:32:58.246758+0000","last_became_peered":"2024-04-14T08:32:58.246758+0000","last_unstale":"2024-04-14T08:33:01.220403+0000","last_undegraded":"2024-04-14T08:33:01.220403+0000","last_fullsized":"2024-04-14T08:33:01.220403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:53.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224440+0000","last_change":"2024-04-14T08:32:54.209924+0000","last_active":"2024-04-14T08:32:59.224440+0000","last_peered":"2024-04-14T08:32:59.224440+0000","last_clean":"2024-04-14T08:32:59.224440+0000","last_became_active":"2024-04-14T08:32:54.209703+0000","last_became_peered":"2024-04-14T08:32:54.209703+0000","last_unstale":"2024-04-14T08:32:59.224440+0000","last_undegraded":"2024-04-14T08:32:59.224440+0000","last_fullsized":"2024-04-14T08:32:59.224440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404884+0000","last_change":"2024-04-14T08:32:53.197701+0000","last_active":"2024-04-14T08:32:59.404884+0000","last_peered":"2024-04-14T08:32:59.404884+0000","last_clean":"2024-04-14T08:32:59.404884+0000","last_became_active":"2024-04-14T08:32:53.197291+0000","last_became_peered":"2024-04-14T08:32:53.197291+0000","last_unstale":"2024-04-14T08:32:59.404884+0000","last_undegraded":"2024-04-14T08:32:59.404884+0000","last_fullsized":"2024-04-14T08:32:59.404884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:54:34.240052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224914+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:32:59.224914+0000","last_peered":"2024-04-14T08:32:59.224914+0000","last_clean":"2024-04-14T08:32:59.224914+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:32:59.224914+0000","last_undegraded":"2024-04-14T08:32:59.224914+0000","last_fullsized":"2024-04-14T08:32:59.224914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115413+0000","last_change":"2024-04-14T08:32:58.237249+0000","last_active":"2024-04-14T08:33:01.115413+0000","last_peered":"2024-04-14T08:33:01.115413+0000","last_clean":"2024-04-14T08:33:01.115413+0000","last_became_active":"2024-04-14T08:32:58.236861+0000","last_became_peered":"2024-04-14T08:32:58.236861+0000","last_unstale":"2024-04-14T08:33:01.115413+0000","last_undegraded":"2024-04-14T08:33:01.115413+0000","last_fullsized":"2024-04-14T08:33:01.115413+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:05.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226018+0000","last_change":"2024-04-14T08:32:54.211527+0000","last_active":"2024-04-14T08:32:59.226018+0000","last_peered":"2024-04-14T08:32:59.226018+0000","last_clean":"2024-04-14T08:32:59.226018+0000","last_became_active":"2024-04-14T08:32:54.211268+0000","last_became_peered":"2024-04-14T08:32:54.211268+0000","last_unstale":"2024-04-14T08:32:59.226018+0000","last_undegraded":"2024-04-14T08:32:59.226018+0000","last_fullsized":"2024-04-14T08:32:59.226018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:41:14.768103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278564+0000","last_change":"2024-04-14T08:32:53.198176+0000","last_active":"2024-04-14T08:33:01.278564+0000","last_peered":"2024-04-14T08:33:01.278564+0000","last_clean":"2024-04-14T08:33:01.278564+0000","last_became_active":"2024-04-14T08:32:53.197626+0000","last_became_peered":"2024-04-14T08:32:53.197626+0000","last_unstale":"2024-04-14T08:33:01.278564+0000","last_undegraded":"2024-04-14T08:33:01.278564+0000","last_fullsized":"2024-04-14T08:33:01.278564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:10:49.016747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115290+0000","last_change":"2024-04-14T08:32:58.240943+0000","last_active":"2024-04-14T08:33:01.115290+0000","last_peered":"2024-04-14T08:33:01.115290+0000","last_clean":"2024-04-14T08:33:01.115290+0000","last_became_active":"2024-04-14T08:32:58.240555+0000","last_became_peered":"2024-04-14T08:32:58.240555+0000","last_unstale":"2024-04-14T08:33:01.115290+0000","last_undegraded":"2024-04-14T08:33:01.115290+0000","last_fullsized":"2024-04-14T08:33:01.115290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:53.001768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405061+0000","last_change":"2024-04-14T08:32:54.205351+0000","last_active":"2024-04-14T08:32:59.405061+0000","last_peered":"2024-04-14T08:32:59.405061+0000","last_clean":"2024-04-14T08:32:59.405061+0000","last_became_active":"2024-04-14T08:32:54.205086+0000","last_became_peered":"2024-04-14T08:32:54.205086+0000","last_unstale":"2024-04-14T08:32:59.405061+0000","last_undegraded":"2024-04-14T08:32:59.405061+0000","last_fullsized":"2024-04-14T08:32:59.405061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:43.600124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610474+0000","last_change":"2024-04-14T08:32:53.194785+0000","last_active":"2024-04-14T08:33:00.610474+0000","last_peered":"2024-04-14T08:33:00.610474+0000","last_clean":"2024-04-14T08:33:00.610474+0000","last_became_active":"2024-04-14T08:32:53.194535+0000","last_became_peered":"2024-04-14T08:32:53.194535+0000","last_unstale":"2024-04-14T08:33:00.610474+0000","last_undegraded":"2024-04-14T08:33:00.610474+0000","last_fullsized":"2024-04-14T08:33:00.610474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:48.807150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220190+0000","last_change":"2024-04-14T08:32:58.246826+0000","last_active":"2024-04-14T08:33:01.220190+0000","last_peered":"2024-04-14T08:33:01.220190+0000","last_clean":"2024-04-14T08:33:01.220190+0000","last_became_active":"2024-04-14T08:32:58.246174+0000","last_became_peered":"2024-04-14T08:32:58.246174+0000","last_unstale":"2024-04-14T08:33:01.220190+0000","last_undegraded":"2024-04-14T08:33:01.220190+0000","last_fullsized":"2024-04-14T08:33:01.220190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:57.127403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226211+0000","last_change":"2024-04-14T08:32:54.200509+0000","last_active":"2024-04-14T08:32:59.226211+0000","last_peered":"2024-04-14T08:32:59.226211+0000","last_clean":"2024-04-14T08:32:59.226211+0000","last_became_active":"2024-04-14T08:32:54.199748+0000","last_became_peered":"2024-04-14T08:32:54.199748+0000","last_unstale":"2024-04-14T08:32:59.226211+0000","last_undegraded":"2024-04-14T08:32:59.226211+0000","last_fullsized":"2024-04-14T08:32:59.226211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:49.939225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278584+0000","last_change":"2024-04-14T08:32:53.196334+0000","last_active":"2024-04-14T08:33:01.278584+0000","last_peered":"2024-04-14T08:33:01.278584+0000","last_clean":"2024-04-14T08:33:01.278584+0000","last_became_active":"2024-04-14T08:32:53.196036+0000","last_became_peered":"2024-04-14T08:32:53.196036+0000","last_unstale":"2024-04-14T08:33:01.278584+0000","last_undegraded":"2024-04-14T08:33:01.278584+0000","last_fullsized":"2024-04-14T08:33:01.278584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404693+0000","last_change":"2024-04-14T08:32:58.239865+0000","last_active":"2024-04-14T08:32:59.404693+0000","last_peered":"2024-04-14T08:32:59.404693+0000","last_clean":"2024-04-14T08:32:59.404693+0000","last_became_active":"2024-04-14T08:32:58.239510+0000","last_became_peered":"2024-04-14T08:32:58.239510+0000","last_unstale":"2024-04-14T08:32:59.404693+0000","last_undegraded":"2024-04-14T08:32:59.404693+0000","last_fullsized":"2024-04-14T08:32:59.404693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:37:10.427564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226121+0000","last_change":"2024-04-14T08:32:54.200777+0000","last_active":"2024-04-14T08:32:59.226121+0000","last_peered":"2024-04-14T08:32:59.226121+0000","last_clean":"2024-04-14T08:32:59.226121+0000","last_became_active":"2024-04-14T08:32:54.200284+0000","last_became_peered":"2024-04-14T08:32:54.200284+0000","last_unstale":"2024-04-14T08:32:59.226121+0000","last_undegraded":"2024-04-14T08:32:59.226121+0000","last_fullsized":"2024-04-14T08:32:59.226121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:53.086323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225655+0000","last_change":"2024-04-14T08:32:53.184799+0000","last_active":"2024-04-14T08:32:59.225655+0000","last_peered":"2024-04-14T08:32:59.225655+0000","last_clean":"2024-04-14T08:32:59.225655+0000","last_became_active":"2024-04-14T08:32:53.184483+0000","last_became_peered":"2024-04-14T08:32:53.184483+0000","last_unstale":"2024-04-14T08:32:59.225655+0000","last_undegraded":"2024-04-14T08:32:59.225655+0000","last_fullsized":"2024-04-14T08:32:59.225655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:03:30.482046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224689+0000","last_change":"2024-04-14T08:32:58.243067+0000","last_active":"2024-04-14T08:32:59.224689+0000","last_peered":"2024-04-14T08:32:59.224689+0000","last_clean":"2024-04-14T08:32:59.224689+0000","last_became_active":"2024-04-14T08:32:58.242726+0000","last_became_peered":"2024-04-14T08:32:58.242726+0000","last_unstale":"2024-04-14T08:32:59.224689+0000","last_undegraded":"2024-04-14T08:32:59.224689+0000","last_fullsized":"2024-04-14T08:32:59.224689+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:09.583268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225539+0000","last_change":"2024-04-14T08:32:54.204697+0000","last_active":"2024-04-14T08:32:59.225539+0000","last_peered":"2024-04-14T08:32:59.225539+0000","last_clean":"2024-04-14T08:32:59.225539+0000","last_became_active":"2024-04-14T08:32:54.204357+0000","last_became_peered":"2024-04-14T08:32:54.204357+0000","last_unstale":"2024-04-14T08:32:59.225539+0000","last_undegraded":"2024-04-14T08:32:59.225539+0000","last_fullsized":"2024-04-14T08:32:59.225539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:01.152061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278559+0000","last_change":"2024-04-14T08:32:53.187972+0000","last_active":"2024-04-14T08:33:01.278559+0000","last_peered":"2024-04-14T08:33:01.278559+0000","last_clean":"2024-04-14T08:33:01.278559+0000","last_became_active":"2024-04-14T08:32:53.187667+0000","last_became_peered":"2024-04-14T08:32:53.187667+0000","last_unstale":"2024-04-14T08:33:01.278559+0000","last_undegraded":"2024-04-14T08:33:01.278559+0000","last_fullsized":"2024-04-14T08:33:01.278559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:54:34.240052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220316+0000","last_change":"2024-04-14T08:32:58.246500+0000","last_active":"2024-04-14T08:33:01.220316+0000","last_peered":"2024-04-14T08:33:01.220316+0000","last_clean":"2024-04-14T08:33:01.220316+0000","last_became_active":"2024-04-14T08:32:58.245970+0000","last_became_peered":"2024-04-14T08:32:58.245970+0000","last_unstale":"2024-04-14T08:33:01.220316+0000","last_undegraded":"2024-04-14T08:33:01.220316+0000","last_fullsized":"2024-04-14T08:33:01.220316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:54.763380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115215+0000","last_change":"2024-04-14T08:32:54.201957+0000","last_active":"2024-04-14T08:33:01.115215+0000","last_peered":"2024-04-14T08:33:01.115215+0000","last_clean":"2024-04-14T08:33:01.115215+0000","last_became_active":"2024-04-14T08:32:54.201618+0000","last_became_peered":"2024-04-14T08:32:54.201618+0000","last_unstale":"2024-04-14T08:33:01.115215+0000","last_undegraded":"2024-04-14T08:33:01.115215+0000","last_fullsized":"2024-04-14T08:33:01.115215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115228+0000","last_change":"2024-04-14T08:32:53.195817+0000","last_active":"2024-04-14T08:33:01.115228+0000","last_peered":"2024-04-14T08:33:01.115228+0000","last_clean":"2024-04-14T08:33:01.115228+0000","last_became_active":"2024-04-14T08:32:53.195547+0000","last_became_peered":"2024-04-14T08:32:53.195547+0000","last_unstale":"2024-04-14T08:33:01.115228+0000","last_undegraded":"2024-04-14T08:33:01.115228+0000","last_fullsized":"2024-04-14T08:33:01.115228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225091+0000","last_change":"2024-04-14T08:32:58.240188+0000","last_active":"2024-04-14T08:32:59.225091+0000","last_peered":"2024-04-14T08:32:59.225091+0000","last_clean":"2024-04-14T08:32:59.225091+0000","last_became_active":"2024-04-14T08:32:58.239811+0000","last_became_peered":"2024-04-14T08:32:58.239811+0000","last_unstale":"2024-04-14T08:32:59.225091+0000","last_undegraded":"2024-04-14T08:32:59.225091+0000","last_fullsized":"2024-04-14T08:32:59.225091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:21.164569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225955+0000","last_change":"2024-04-14T08:32:54.200441+0000","last_active":"2024-04-14T08:32:59.225955+0000","last_peered":"2024-04-14T08:32:59.225955+0000","last_clean":"2024-04-14T08:32:59.225955+0000","last_became_active":"2024-04-14T08:32:54.199824+0000","last_became_peered":"2024-04-14T08:32:54.199824+0000","last_unstale":"2024-04-14T08:32:59.225955+0000","last_undegraded":"2024-04-14T08:32:59.225955+0000","last_fullsized":"2024-04-14T08:32:59.225955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404960+0000","last_change":"2024-04-14T08:32:53.202370+0000","last_active":"2024-04-14T08:32:59.404960+0000","last_peered":"2024-04-14T08:32:59.404960+0000","last_clean":"2024-04-14T08:32:59.404960+0000","last_became_active":"2024-04-14T08:32:53.202083+0000","last_became_peered":"2024-04-14T08:32:53.202083+0000","last_unstale":"2024-04-14T08:32:59.404960+0000","last_undegraded":"2024-04-14T08:32:59.404960+0000","last_fullsized":"2024-04-14T08:32:59.404960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224830+0000","last_change":"2024-04-14T08:32:58.247699+0000","last_active":"2024-04-14T08:32:59.224830+0000","last_peered":"2024-04-14T08:32:59.224830+0000","last_clean":"2024-04-14T08:32:59.224830+0000","last_became_active":"2024-04-14T08:32:58.247336+0000","last_became_peered":"2024-04-14T08:32:58.247336+0000","last_unstale":"2024-04-14T08:32:59.224830+0000","last_undegraded":"2024-04-14T08:32:59.224830+0000","last_fullsized":"2024-04-14T08:32:59.224830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:07:01.006337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220030+0000","last_change":"2024-04-14T08:32:54.198561+0000","last_active":"2024-04-14T08:33:01.220030+0000","last_peered":"2024-04-14T08:33:01.220030+0000","last_clean":"2024-04-14T08:33:01.220030+0000","last_became_active":"2024-04-14T08:32:54.198319+0000","last_became_peered":"2024-04-14T08:32:54.198319+0000","last_unstale":"2024-04-14T08:33:01.220030+0000","last_undegraded":"2024-04-14T08:33:01.220030+0000","last_fullsized":"2024-04-14T08:33:01.220030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219614+0000","last_change":"2024-04-14T08:32:53.188093+0000","last_active":"2024-04-14T08:33:01.219614+0000","last_peered":"2024-04-14T08:33:01.219614+0000","last_clean":"2024-04-14T08:33:01.219614+0000","last_became_active":"2024-04-14T08:32:53.187807+0000","last_became_peered":"2024-04-14T08:32:53.187807+0000","last_unstale":"2024-04-14T08:33:01.219614+0000","last_undegraded":"2024-04-14T08:33:01.219614+0000","last_fullsized":"2024-04-14T08:33:01.219614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:22:15.886932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405178+0000","last_change":"2024-04-14T08:32:58.234962+0000","last_active":"2024-04-14T08:32:59.405178+0000","last_peered":"2024-04-14T08:32:59.405178+0000","last_clean":"2024-04-14T08:32:59.405178+0000","last_became_active":"2024-04-14T08:32:58.234475+0000","last_became_peered":"2024-04-14T08:32:58.234475+0000","last_unstale":"2024-04-14T08:32:59.405178+0000","last_undegraded":"2024-04-14T08:32:59.405178+0000","last_fullsized":"2024-04-14T08:32:59.405178+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:07:01.006337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225877+0000","last_change":"2024-04-14T08:32:54.199377+0000","last_active":"2024-04-14T08:32:59.225877+0000","last_peered":"2024-04-14T08:32:59.225877+0000","last_clean":"2024-04-14T08:32:59.225877+0000","last_became_active":"2024-04-14T08:32:54.199105+0000","last_became_peered":"2024-04-14T08:32:54.199105+0000","last_unstale":"2024-04-14T08:32:59.225877+0000","last_undegraded":"2024-04-14T08:32:59.225877+0000","last_fullsized":"2024-04-14T08:32:59.225877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:43.600124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:03.602017+0000","last_change":"2024-04-14T08:32:53.199739+0000","last_active":"2024-04-14T08:33:03.602017+0000","last_peered":"2024-04-14T08:33:03.602017+0000","last_clean":"2024-04-14T08:33:03.602017+0000","last_became_active":"2024-04-14T08:32:53.199421+0000","last_became_peered":"2024-04-14T08:32:53.199421+0000","last_unstale":"2024-04-14T08:33:03.602017+0000","last_undegraded":"2024-04-14T08:33:03.602017+0000","last_fullsized":"2024-04-14T08:33:03.602017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610241+0000","last_change":"2024-04-14T08:32:58.243680+0000","last_active":"2024-04-14T08:33:00.610241+0000","last_peered":"2024-04-14T08:33:00.610241+0000","last_clean":"2024-04-14T08:33:00.610241+0000","last_became_active":"2024-04-14T08:32:58.243248+0000","last_became_peered":"2024-04-14T08:32:58.243248+0000","last_unstale":"2024-04-14T08:33:00.610241+0000","last_undegraded":"2024-04-14T08:33:00.610241+0000","last_fullsized":"2024-04-14T08:33:00.610241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:57.127403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225954+0000","last_change":"2024-04-14T08:32:54.199362+0000","last_active":"2024-04-14T08:32:59.225954+0000","last_peered":"2024-04-14T08:32:59.225954+0000","last_clean":"2024-04-14T08:32:59.225954+0000","last_became_active":"2024-04-14T08:32:54.199039+0000","last_became_peered":"2024-04-14T08:32:54.199039+0000","last_unstale":"2024-04-14T08:32:59.225954+0000","last_undegraded":"2024-04-14T08:32:59.225954+0000","last_fullsized":"2024-04-14T08:32:59.225954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:33.086425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610374+0000","last_change":"2024-04-14T08:32:53.198302+0000","last_active":"2024-04-14T08:33:00.610374+0000","last_peered":"2024-04-14T08:33:00.610374+0000","last_clean":"2024-04-14T08:33:00.610374+0000","last_became_active":"2024-04-14T08:32:53.198042+0000","last_became_peered":"2024-04-14T08:32:53.198042+0000","last_unstale":"2024-04-14T08:33:00.610374+0000","last_undegraded":"2024-04-14T08:33:00.610374+0000","last_fullsized":"2024-04-14T08:33:00.610374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:32.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225101+0000","last_change":"2024-04-14T08:32:58.249159+0000","last_active":"2024-04-14T08:32:59.225101+0000","last_peered":"2024-04-14T08:32:59.225101+0000","last_clean":"2024-04-14T08:32:59.225101+0000","last_became_active":"2024-04-14T08:32:58.248502+0000","last_became_peered":"2024-04-14T08:32:58.248502+0000","last_unstale":"2024-04-14T08:32:59.225101+0000","last_undegraded":"2024-04-14T08:32:59.225101+0000","last_fullsized":"2024-04-14T08:32:59.225101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:57.702107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220422+0000","last_change":"2024-04-14T08:32:54.211683+0000","last_active":"2024-04-14T08:33:01.220422+0000","last_peered":"2024-04-14T08:33:01.220422+0000","last_clean":"2024-04-14T08:33:01.220422+0000","last_became_active":"2024-04-14T08:32:54.211031+0000","last_became_peered":"2024-04-14T08:32:54.211031+0000","last_unstale":"2024-04-14T08:33:01.220422+0000","last_undegraded":"2024-04-14T08:33:01.220422+0000","last_fullsized":"2024-04-14T08:33:01.220422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405170+0000","last_change":"2024-04-14T08:32:53.198725+0000","last_active":"2024-04-14T08:32:59.405170+0000","last_peered":"2024-04-14T08:32:59.405170+0000","last_clean":"2024-04-14T08:32:59.405170+0000","last_became_active":"2024-04-14T08:32:53.198331+0000","last_became_peered":"2024-04-14T08:32:53.198331+0000","last_unstale":"2024-04-14T08:32:59.405170+0000","last_undegraded":"2024-04-14T08:32:59.405170+0000","last_fullsized":"2024-04-14T08:32:59.405170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224411+0000","last_change":"2024-04-14T08:32:58.241328+0000","last_active":"2024-04-14T08:32:59.224411+0000","last_peered":"2024-04-14T08:32:59.224411+0000","last_clean":"2024-04-14T08:32:59.224411+0000","last_became_active":"2024-04-14T08:32:58.240742+0000","last_became_peered":"2024-04-14T08:32:58.240742+0000","last_unstale":"2024-04-14T08:32:59.224411+0000","last_undegraded":"2024-04-14T08:32:59.224411+0000","last_fullsized":"2024-04-14T08:32:59.224411+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:10:58.801121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405105+0000","last_change":"2024-04-14T08:32:54.211134+0000","last_active":"2024-04-14T08:32:59.405105+0000","last_peered":"2024-04-14T08:32:59.405105+0000","last_clean":"2024-04-14T08:32:59.405105+0000","last_became_active":"2024-04-14T08:32:54.210882+0000","last_became_peered":"2024-04-14T08:32:54.210882+0000","last_unstale":"2024-04-14T08:32:59.405105+0000","last_undegraded":"2024-04-14T08:32:59.405105+0000","last_fullsized":"2024-04-14T08:32:59.405105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115518+0000","last_change":"2024-04-14T08:32:53.198595+0000","last_active":"2024-04-14T08:33:01.115518+0000","last_peered":"2024-04-14T08:33:01.115518+0000","last_clean":"2024-04-14T08:33:01.115518+0000","last_became_active":"2024-04-14T08:32:53.198284+0000","last_became_peered":"2024-04-14T08:32:53.198284+0000","last_unstale":"2024-04-14T08:33:01.115518+0000","last_undegraded":"2024-04-14T08:33:01.115518+0000","last_fullsized":"2024-04-14T08:33:01.115518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:48.807150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224443+0000","last_change":"2024-04-14T08:32:58.240139+0000","last_active":"2024-04-14T08:32:59.224443+0000","last_peered":"2024-04-14T08:32:59.224443+0000","last_clean":"2024-04-14T08:32:59.224443+0000","last_became_active":"2024-04-14T08:32:58.239751+0000","last_became_peered":"2024-04-14T08:32:58.239751+0000","last_unstale":"2024-04-14T08:32:59.224443+0000","last_undegraded":"2024-04-14T08:32:59.224443+0000","last_fullsized":"2024-04-14T08:32:59.224443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:37:10.427564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220280+0000","last_change":"2024-04-14T08:32:58.252205+0000","last_active":"2024-04-14T08:33:01.220280+0000","last_peered":"2024-04-14T08:33:01.220280+0000","last_clean":"2024-04-14T08:33:01.220280+0000","last_became_active":"2024-04-14T08:32:58.251856+0000","last_became_peered":"2024-04-14T08:32:58.251856+0000","last_unstale":"2024-04-14T08:33:01.220280+0000","last_undegraded":"2024-04-14T08:33:01.220280+0000","last_fullsized":"2024-04-14T08:33:01.220280+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:16.724895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.609964+0000","last_change":"2024-04-14T08:32:54.200507+0000","last_active":"2024-04-14T08:33:00.609964+0000","last_peered":"2024-04-14T08:33:00.609964+0000","last_clean":"2024-04-14T08:33:00.609964+0000","last_became_active":"2024-04-14T08:32:54.200256+0000","last_became_peered":"2024-04-14T08:32:54.200256+0000","last_unstale":"2024-04-14T08:33:00.609964+0000","last_undegraded":"2024-04-14T08:33:00.609964+0000","last_fullsized":"2024-04-14T08:33:00.609964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404763+0000","last_change":"2024-04-14T08:32:53.197120+0000","last_active":"2024-04-14T08:32:59.404763+0000","last_peered":"2024-04-14T08:32:59.404763+0000","last_clean":"2024-04-14T08:32:59.404763+0000","last_became_active":"2024-04-14T08:32:53.196835+0000","last_became_peered":"2024-04-14T08:32:53.196835+0000","last_unstale":"2024-04-14T08:32:59.404763+0000","last_undegraded":"2024-04-14T08:32:59.404763+0000","last_fullsized":"2024-04-14T08:32:59.404763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115520+0000","last_change":"2024-04-14T08:32:58.240951+0000","last_active":"2024-04-14T08:33:01.115520+0000","last_peered":"2024-04-14T08:33:01.115520+0000","last_clean":"2024-04-14T08:33:01.115520+0000","last_became_active":"2024-04-14T08:32:58.240542+0000","last_became_peered":"2024-04-14T08:32:58.240542+0000","last_unstale":"2024-04-14T08:33:01.115520+0000","last_undegraded":"2024-04-14T08:33:01.115520+0000","last_fullsized":"2024-04-14T08:33:01.115520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:57.127403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224403+0000","last_change":"2024-04-14T08:32:54.199647+0000","last_active":"2024-04-14T08:32:59.224403+0000","last_peered":"2024-04-14T08:32:59.224403+0000","last_clean":"2024-04-14T08:32:59.224403+0000","last_became_active":"2024-04-14T08:32:54.199365+0000","last_became_peered":"2024-04-14T08:32:54.199365+0000","last_unstale":"2024-04-14T08:32:59.224403+0000","last_undegraded":"2024-04-14T08:32:59.224403+0000","last_fullsized":"2024-04-14T08:32:59.224403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:17:06.203553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224536+0000","last_change":"2024-04-14T08:32:53.199750+0000","last_active":"2024-04-14T08:32:59.224536+0000","last_peered":"2024-04-14T08:32:59.224536+0000","last_clean":"2024-04-14T08:32:59.224536+0000","last_became_active":"2024-04-14T08:32:53.199477+0000","last_became_peered":"2024-04-14T08:32:53.199477+0000","last_unstale":"2024-04-14T08:32:59.224536+0000","last_undegraded":"2024-04-14T08:32:59.224536+0000","last_fullsized":"2024-04-14T08:32:59.224536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:03:30.482046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115345+0000","last_change":"2024-04-14T08:32:53.195193+0000","last_active":"2024-04-14T08:33:01.115345+0000","last_peered":"2024-04-14T08:33:01.115345+0000","last_clean":"2024-04-14T08:33:01.115345+0000","last_became_active":"2024-04-14T08:32:53.194887+0000","last_became_peered":"2024-04-14T08:32:53.194887+0000","last_unstale":"2024-04-14T08:33:01.115345+0000","last_undegraded":"2024-04-14T08:33:01.115345+0000","last_fullsized":"2024-04-14T08:33:01.115345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220235+0000","last_change":"2024-04-14T08:32:54.203263+0000","last_active":"2024-04-14T08:33:01.220235+0000","last_peered":"2024-04-14T08:33:01.220235+0000","last_clean":"2024-04-14T08:33:01.220235+0000","last_became_active":"2024-04-14T08:32:54.202196+0000","last_became_peered":"2024-04-14T08:32:54.202196+0000","last_unstale":"2024-04-14T08:33:01.220235+0000","last_undegraded":"2024-04-14T08:33:01.220235+0000","last_fullsized":"2024-04-14T08:33:01.220235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278810+0000","last_change":"2024-04-14T08:32:58.239935+0000","last_active":"2024-04-14T08:33:01.278810+0000","last_peered":"2024-04-14T08:33:01.278810+0000","last_clean":"2024-04-14T08:33:01.278810+0000","last_became_active":"2024-04-14T08:32:58.239542+0000","last_became_peered":"2024-04-14T08:32:58.239542+0000","last_unstale":"2024-04-14T08:33:01.278810+0000","last_undegraded":"2024-04-14T08:33:01.278810+0000","last_fullsized":"2024-04-14T08:33:01.278810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:04.288282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224729+0000","last_change":"2024-04-14T08:32:53.201101+0000","last_active":"2024-04-14T08:32:59.224729+0000","last_peered":"2024-04-14T08:32:59.224729+0000","last_clean":"2024-04-14T08:32:59.224729+0000","last_became_active":"2024-04-14T08:32:53.200781+0000","last_became_peered":"2024-04-14T08:32:53.200781+0000","last_unstale":"2024-04-14T08:32:59.224729+0000","last_undegraded":"2024-04-14T08:32:59.224729+0000","last_fullsized":"2024-04-14T08:32:59.224729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:08:09.715762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224989+0000","last_change":"2024-04-14T08:32:54.194042+0000","last_active":"2024-04-14T08:32:59.224989+0000","last_peered":"2024-04-14T08:32:59.224989+0000","last_clean":"2024-04-14T08:32:59.224989+0000","last_became_active":"2024-04-14T08:32:54.193764+0000","last_became_peered":"2024-04-14T08:32:54.193764+0000","last_unstale":"2024-04-14T08:32:59.224989+0000","last_undegraded":"2024-04-14T08:32:59.224989+0000","last_fullsized":"2024-04-14T08:32:59.224989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:50:00.196929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225098+0000","last_change":"2024-04-14T08:32:58.241996+0000","last_active":"2024-04-14T08:32:59.225098+0000","last_peered":"2024-04-14T08:32:59.225098+0000","last_clean":"2024-04-14T08:32:59.225098+0000","last_became_active":"2024-04-14T08:32:58.241641+0000","last_became_peered":"2024-04-14T08:32:58.241641+0000","last_unstale":"2024-04-14T08:32:59.225098+0000","last_undegraded":"2024-04-14T08:32:59.225098+0000","last_fullsized":"2024-04-14T08:32:59.225098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:58:22.710120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115102+0000","last_change":"2024-04-14T08:32:54.210270+0000","last_active":"2024-04-14T08:33:01.115102+0000","last_peered":"2024-04-14T08:33:01.115102+0000","last_clean":"2024-04-14T08:33:01.115102+0000","last_became_active":"2024-04-14T08:32:54.209717+0000","last_became_peered":"2024-04-14T08:32:54.209717+0000","last_unstale":"2024-04-14T08:33:01.115102+0000","last_undegraded":"2024-04-14T08:33:01.115102+0000","last_fullsized":"2024-04-14T08:33:01.115102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278775+0000","last_change":"2024-04-14T08:32:54.208720+0000","last_active":"2024-04-14T08:33:01.278775+0000","last_peered":"2024-04-14T08:33:01.278775+0000","last_clean":"2024-04-14T08:33:01.278775+0000","last_became_active":"2024-04-14T08:32:54.208426+0000","last_became_peered":"2024-04-14T08:32:54.208426+0000","last_unstale":"2024-04-14T08:33:01.278775+0000","last_undegraded":"2024-04-14T08:33:01.278775+0000","last_fullsized":"2024-04-14T08:33:01.278775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:49.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404885+0000","last_change":"2024-04-14T08:32:54.209426+0000","last_active":"2024-04-14T08:32:59.404885+0000","last_peered":"2024-04-14T08:32:59.404885+0000","last_clean":"2024-04-14T08:32:59.404885+0000","last_became_active":"2024-04-14T08:32:54.209158+0000","last_became_peered":"2024-04-14T08:32:54.209158+0000","last_unstale":"2024-04-14T08:32:59.404885+0000","last_undegraded":"2024-04-14T08:32:59.404885+0000","last_fullsized":"2024-04-14T08:32:59.404885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:12.683816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220151+0000","last_change":"2024-04-14T08:32:54.206827+0000","last_active":"2024-04-14T08:33:01.220151+0000","last_peered":"2024-04-14T08:33:01.220151+0000","last_clean":"2024-04-14T08:33:01.220151+0000","last_became_active":"2024-04-14T08:32:54.206593+0000","last_became_peered":"2024-04-14T08:32:54.206593+0000","last_unstale":"2024-04-14T08:33:01.220151+0000","last_undegraded":"2024-04-14T08:33:01.220151+0000","last_fullsized":"2024-04-14T08:33:01.220151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224854+0000","last_change":"2024-04-14T08:32:54.200872+0000","last_active":"2024-04-14T08:32:59.224854+0000","last_peered":"2024-04-14T08:32:59.224854+0000","last_clean":"2024-04-14T08:32:59.224854+0000","last_became_active":"2024-04-14T08:32:54.200616+0000","last_became_peered":"2024-04-14T08:32:54.200616+0000","last_unstale":"2024-04-14T08:32:59.224854+0000","last_undegraded":"2024-04-14T08:32:59.224854+0000","last_fullsized":"2024-04-14T08:32:59.224854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610219+0000","last_change":"2024-04-14T08:32:54.199679+0000","last_active":"2024-04-14T08:33:00.610219+0000","last_peered":"2024-04-14T08:33:00.610219+0000","last_clean":"2024-04-14T08:33:00.610219+0000","last_became_active":"2024-04-14T08:32:54.199433+0000","last_became_peered":"2024-04-14T08:32:54.199433+0000","last_unstale":"2024-04-14T08:33:00.610219+0000","last_undegraded":"2024-04-14T08:33:00.610219+0000","last_fullsized":"2024-04-14T08:33:00.610219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115645+0000","last_change":"2024-04-14T08:32:54.207939+0000","last_active":"2024-04-14T08:33:01.115645+0000","last_peered":"2024-04-14T08:33:01.115645+0000","last_clean":"2024-04-14T08:33:01.115645+0000","last_became_active":"2024-04-14T08:32:54.207460+0000","last_became_peered":"2024-04-14T08:32:54.207460+0000","last_unstale":"2024-04-14T08:33:01.115645+0000","last_undegraded":"2024-04-14T08:33:01.115645+0000","last_fullsized":"2024-04-14T08:33:01.115645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225033+0000","last_change":"2024-04-14T08:32:54.202461+0000","last_active":"2024-04-14T08:32:59.225033+0000","last_peered":"2024-04-14T08:32:59.225033+0000","last_clean":"2024-04-14T08:32:59.225033+0000","last_became_active":"2024-04-14T08:32:54.202156+0000","last_became_peered":"2024-04-14T08:32:54.202156+0000","last_unstale":"2024-04-14T08:32:59.225033+0000","last_undegraded":"2024-04-14T08:32:59.225033+0000","last_fullsized":"2024-04-14T08:32:59.225033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404730+0000","last_change":"2024-04-14T08:32:54.204603+0000","last_active":"2024-04-14T08:32:59.404730+0000","last_peered":"2024-04-14T08:32:59.404730+0000","last_clean":"2024-04-14T08:32:59.404730+0000","last_became_active":"2024-04-14T08:32:54.204323+0000","last_became_peered":"2024-04-14T08:32:54.204323+0000","last_unstale":"2024-04-14T08:32:59.404730+0000","last_undegraded":"2024-04-14T08:32:59.404730+0000","last_fullsized":"2024-04-14T08:32:59.404730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224464+0000","last_change":"2024-04-14T08:32:54.205983+0000","last_active":"2024-04-14T08:32:59.224464+0000","last_peered":"2024-04-14T08:32:59.224464+0000","last_clean":"2024-04-14T08:32:59.224464+0000","last_became_active":"2024-04-14T08:32:54.205707+0000","last_became_peered":"2024-04-14T08:32:54.205707+0000","last_unstale":"2024-04-14T08:32:59.224464+0000","last_undegraded":"2024-04-14T08:32:59.224464+0000","last_fullsized":"2024-04-14T08:32:59.224464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:05:03.511225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404857+0000","last_change":"2024-04-14T08:32:54.211979+0000","last_active":"2024-04-14T08:32:59.404857+0000","last_peered":"2024-04-14T08:32:59.404857+0000","last_clean":"2024-04-14T08:32:59.404857+0000","last_became_active":"2024-04-14T08:32:54.211481+0000","last_became_peered":"2024-04-14T08:32:54.211481+0000","last_unstale":"2024-04-14T08:32:59.404857+0000","last_undegraded":"2024-04-14T08:32:59.404857+0000","last_fullsized":"2024-04-14T08:32:59.404857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:53.086323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220046+0000","last_change":"2024-04-14T08:32:54.204727+0000","last_active":"2024-04-14T08:33:01.220046+0000","last_peered":"2024-04-14T08:33:01.220046+0000","last_clean":"2024-04-14T08:33:01.220046+0000","last_became_active":"2024-04-14T08:32:54.204472+0000","last_became_peered":"2024-04-14T08:32:54.204472+0000","last_unstale":"2024-04-14T08:33:01.220046+0000","last_undegraded":"2024-04-14T08:33:01.220046+0000","last_fullsized":"2024-04-14T08:33:01.220046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115389+0000","last_change":"2024-04-14T08:32:54.211345+0000","last_active":"2024-04-14T08:33:01.115389+0000","last_peered":"2024-04-14T08:33:01.115389+0000","last_clean":"2024-04-14T08:33:01.115389+0000","last_became_active":"2024-04-14T08:32:54.211084+0000","last_became_peered":"2024-04-14T08:32:54.211084+0000","last_unstale":"2024-04-14T08:33:01.115389+0000","last_undegraded":"2024-04-14T08:33:01.115389+0000","last_fullsized":"2024-04-14T08:33:01.115389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226252+0000","last_change":"2024-04-14T08:32:54.198283+0000","last_active":"2024-04-14T08:32:59.226252+0000","last_peered":"2024-04-14T08:32:59.226252+0000","last_clean":"2024-04-14T08:32:59.226252+0000","last_became_active":"2024-04-14T08:32:54.197647+0000","last_became_peered":"2024-04-14T08:32:54.197647+0000","last_unstale":"2024-04-14T08:32:59.226252+0000","last_undegraded":"2024-04-14T08:32:59.226252+0000","last_fullsized":"2024-04-14T08:32:59.226252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:48.960689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278917+0000","last_change":"2024-04-14T08:32:54.204246+0000","last_active":"2024-04-14T08:33:01.278917+0000","last_peered":"2024-04-14T08:33:01.278917+0000","last_clean":"2024-04-14T08:33:01.278917+0000","last_became_active":"2024-04-14T08:32:54.203956+0000","last_became_peered":"2024-04-14T08:32:54.203956+0000","last_unstale":"2024-04-14T08:33:01.278917+0000","last_undegraded":"2024-04-14T08:33:01.278917+0000","last_fullsized":"2024-04-14T08:33:01.278917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:33.086425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405290+0000","last_change":"2024-04-14T08:32:54.207841+0000","last_active":"2024-04-14T08:32:59.405290+0000","last_peered":"2024-04-14T08:32:59.405290+0000","last_clean":"2024-04-14T08:32:59.405290+0000","last_became_active":"2024-04-14T08:32:54.207412+0000","last_became_peered":"2024-04-14T08:32:54.207412+0000","last_unstale":"2024-04-14T08:32:59.405290+0000","last_undegraded":"2024-04-14T08:32:59.405290+0000","last_fullsized":"2024-04-14T08:32:59.405290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:48.043054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404630+0000","last_change":"2024-04-14T08:32:54.207321+0000","last_active":"2024-04-14T08:32:59.404630+0000","last_peered":"2024-04-14T08:32:59.404630+0000","last_clean":"2024-04-14T08:32:59.404630+0000","last_became_active":"2024-04-14T08:32:54.207058+0000","last_became_peered":"2024-04-14T08:32:54.207058+0000","last_unstale":"2024-04-14T08:32:59.404630+0000","last_undegraded":"2024-04-14T08:32:59.404630+0000","last_fullsized":"2024-04-14T08:32:59.404630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610020+0000","last_change":"2024-04-14T08:32:54.205919+0000","last_active":"2024-04-14T08:33:00.610020+0000","last_peered":"2024-04-14T08:33:00.610020+0000","last_clean":"2024-04-14T08:33:00.610020+0000","last_became_active":"2024-04-14T08:32:54.205623+0000","last_became_peered":"2024-04-14T08:32:54.205623+0000","last_unstale":"2024-04-14T08:33:00.610020+0000","last_undegraded":"2024-04-14T08:33:00.610020+0000","last_fullsized":"2024-04-14T08:33:00.610020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220007+0000","last_change":"2024-04-14T08:32:54.196317+0000","last_active":"2024-04-14T08:33:01.220007+0000","last_peered":"2024-04-14T08:33:01.220007+0000","last_clean":"2024-04-14T08:33:01.220007+0000","last_became_active":"2024-04-14T08:32:54.195481+0000","last_became_peered":"2024-04-14T08:32:54.195481+0000","last_unstale":"2024-04-14T08:33:01.220007+0000","last_undegraded":"2024-04-14T08:33:01.220007+0000","last_fullsized":"2024-04-14T08:33:01.220007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:17:06.203553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224636+0000","last_change":"2024-04-14T08:32:54.203801+0000","last_active":"2024-04-14T08:32:59.224636+0000","last_peered":"2024-04-14T08:32:59.224636+0000","last_clean":"2024-04-14T08:32:59.224636+0000","last_became_active":"2024-04-14T08:32:54.203524+0000","last_became_peered":"2024-04-14T08:32:54.203524+0000","last_unstale":"2024-04-14T08:32:59.224636+0000","last_undegraded":"2024-04-14T08:32:59.224636+0000","last_fullsized":"2024-04-14T08:32:59.224636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225873+0000","last_change":"2024-04-14T08:32:54.204997+0000","last_active":"2024-04-14T08:32:59.225873+0000","last_peered":"2024-04-14T08:32:59.225873+0000","last_clean":"2024-04-14T08:32:59.225873+0000","last_became_active":"2024-04-14T08:32:54.204757+0000","last_became_peered":"2024-04-14T08:32:54.204757+0000","last_unstale":"2024-04-14T08:32:59.225873+0000","last_undegraded":"2024-04-14T08:32:59.225873+0000","last_fullsized":"2024-04-14T08:32:59.225873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:49.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115444+0000","last_change":"2024-04-14T08:32:54.210277+0000","last_active":"2024-04-14T08:33:01.115444+0000","last_peered":"2024-04-14T08:33:01.115444+0000","last_clean":"2024-04-14T08:33:01.115444+0000","last_became_active":"2024-04-14T08:32:54.209548+0000","last_became_peered":"2024-04-14T08:32:54.209548+0000","last_unstale":"2024-04-14T08:33:01.115444+0000","last_undegraded":"2024-04-14T08:33:01.115444+0000","last_fullsized":"2024-04-14T08:33:01.115444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115625+0000","last_change":"2024-04-14T08:32:54.213828+0000","last_active":"2024-04-14T08:33:01.115625+0000","last_peered":"2024-04-14T08:33:01.115625+0000","last_clean":"2024-04-14T08:33:01.115625+0000","last_became_active":"2024-04-14T08:32:54.213551+0000","last_became_peered":"2024-04-14T08:32:54.213551+0000","last_unstale":"2024-04-14T08:33:01.115625+0000","last_undegraded":"2024-04-14T08:33:01.115625+0000","last_fullsized":"2024-04-14T08:33:01.115625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226176+0000","last_change":"2024-04-14T08:32:54.212290+0000","last_active":"2024-04-14T08:32:59.226176+0000","last_peered":"2024-04-14T08:32:59.226176+0000","last_clean":"2024-04-14T08:32:59.226176+0000","last_became_active":"2024-04-14T08:32:54.212014+0000","last_became_peered":"2024-04-14T08:32:54.212014+0000","last_unstale":"2024-04-14T08:32:59.226176+0000","last_undegraded":"2024-04-14T08:32:59.226176+0000","last_fullsized":"2024-04-14T08:32:59.226176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:50.722301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224300+0000","last_change":"2024-04-14T08:32:54.209523+0000","last_active":"2024-04-14T08:32:59.224300+0000","last_peered":"2024-04-14T08:32:59.224300+0000","last_clean":"2024-04-14T08:32:59.224300+0000","last_became_active":"2024-04-14T08:32:54.209262+0000","last_became_peered":"2024-04-14T08:32:54.209262+0000","last_unstale":"2024-04-14T08:32:59.224300+0000","last_undegraded":"2024-04-14T08:32:59.224300+0000","last_fullsized":"2024-04-14T08:32:59.224300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:25:28.934928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225401+0000","last_change":"2024-04-14T08:32:54.195208+0000","last_active":"2024-04-14T08:32:59.225401+0000","last_peered":"2024-04-14T08:32:59.225401+0000","last_clean":"2024-04-14T08:32:59.225401+0000","last_became_active":"2024-04-14T08:32:54.194916+0000","last_became_peered":"2024-04-14T08:32:54.194916+0000","last_unstale":"2024-04-14T08:32:59.225401+0000","last_undegraded":"2024-04-14T08:32:59.225401+0000","last_fullsized":"2024-04-14T08:32:59.225401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:16.536477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404780+0000","last_change":"2024-04-14T08:32:54.208737+0000","last_active":"2024-04-14T08:32:59.404780+0000","last_peered":"2024-04-14T08:32:59.404780+0000","last_clean":"2024-04-14T08:32:59.404780+0000","last_became_active":"2024-04-14T08:32:54.208439+0000","last_became_peered":"2024-04-14T08:32:54.208439+0000","last_unstale":"2024-04-14T08:32:59.404780+0000","last_undegraded":"2024-04-14T08:32:59.404780+0000","last_fullsized":"2024-04-14T08:32:59.404780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:48.960689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225788+0000","last_change":"2024-04-14T08:32:54.198295+0000","last_active":"2024-04-14T08:32:59.225788+0000","last_peered":"2024-04-14T08:32:59.225788+0000","last_clean":"2024-04-14T08:32:59.225788+0000","last_became_active":"2024-04-14T08:32:54.198052+0000","last_became_peered":"2024-04-14T08:32:54.198052+0000","last_unstale":"2024-04-14T08:32:59.225788+0000","last_undegraded":"2024-04-14T08:32:59.225788+0000","last_fullsized":"2024-04-14T08:32:59.225788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:48.043054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404955+0000","last_change":"2024-04-14T08:32:54.206537+0000","last_active":"2024-04-14T08:32:59.404955+0000","last_peered":"2024-04-14T08:32:59.404955+0000","last_clean":"2024-04-14T08:32:59.404955+0000","last_became_active":"2024-04-14T08:32:54.206246+0000","last_became_peered":"2024-04-14T08:32:54.206246+0000","last_unstale":"2024-04-14T08:32:59.404955+0000","last_undegraded":"2024-04-14T08:32:59.404955+0000","last_fullsized":"2024-04-14T08:32:59.404955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:16.536477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220192+0000","last_change":"2024-04-14T08:32:54.201364+0000","last_active":"2024-04-14T08:33:01.220192+0000","last_peered":"2024-04-14T08:33:01.220192+0000","last_clean":"2024-04-14T08:33:01.220192+0000","last_became_active":"2024-04-14T08:32:54.199521+0000","last_became_peered":"2024-04-14T08:32:54.199521+0000","last_unstale":"2024-04-14T08:33:01.220192+0000","last_undegraded":"2024-04-14T08:33:01.220192+0000","last_fullsized":"2024-04-14T08:33:01.220192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610554+0000","last_change":"2024-04-14T08:32:54.199458+0000","last_active":"2024-04-14T08:33:00.610554+0000","last_peered":"2024-04-14T08:33:00.610554+0000","last_clean":"2024-04-14T08:33:00.610554+0000","last_became_active":"2024-04-14T08:32:54.199203+0000","last_became_peered":"2024-04-14T08:32:54.199203+0000","last_unstale":"2024-04-14T08:33:00.610554+0000","last_undegraded":"2024-04-14T08:33:00.610554+0000","last_fullsized":"2024-04-14T08:33:00.610554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226040+0000","last_change":"2024-04-14T08:32:54.206343+0000","last_active":"2024-04-14T08:32:59.226040+0000","last_peered":"2024-04-14T08:32:59.226040+0000","last_clean":"2024-04-14T08:32:59.226040+0000","last_became_active":"2024-04-14T08:32:54.205925+0000","last_became_peered":"2024-04-14T08:32:54.205925+0000","last_unstale":"2024-04-14T08:32:59.226040+0000","last_undegraded":"2024-04-14T08:32:59.226040+0000","last_fullsized":"2024-04-14T08:32:59.226040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:12.683816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:33:13.931 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:33:14.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.068+0000 7fda7d8a6640 1 Processor -- start 2024-04-14T08:33:14.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.068+0000 7fda7d8a6640 1 -- start start 2024-04-14T08:33:14.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.068+0000 7fda7d8a6640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fda7815e4d0 0x7fda780cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.068+0000 7fda7d8a6640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780d11a0 0x7fda780b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.068+0000 7fda7d8a6640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda7815e8b0 con 0x7fda780cfd90 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.068+0000 7fda7d8a6640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fda780d1f50 con 0x7fda7815e4d0 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.068+0000 7fda7d8a6640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fda780d2230 con 0x7fda780d11a0 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.069+0000 7fda767fc640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fda780cfd90 0x7fda780d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35730/0 (socket says 172.21.15.172:35730) 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.069+0000 7fda777fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780d11a0 0x7fda780b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.069+0000 7fda767fc640 1 -- 172.21.15.172:0/1829156745 learned_addr learned my addr 172.21.15.172:0/1829156745 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.069+0000 7fda76ffd640 1 --2- 172.21.15.172:0/1829156745 >> v2:172.21.15.177:3300/0 conn(0x7fda7815e4d0 0x7fda780cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.069+0000 7fda75ffb640 1 -- 172.21.15.172:0/1829156745 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3646650711 0 0) 0x7fda7815e8b0 con 0x7fda780cfd90 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.069+0000 7fda75ffb640 1 -- 172.21.15.172:0/1829156745 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda64003520 con 0x7fda780cfd90 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.069+0000 7fda777fe640 1 -- 172.21.15.172:0/1829156745 >> v2:172.21.15.177:3300/0 conn(0x7fda7815e4d0 msgr2=0x7fda780cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.069+0000 7fda777fe640 1 --2- 172.21.15.172:0/1829156745 >> v2:172.21.15.177:3300/0 conn(0x7fda7815e4d0 0x7fda780cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.069+0000 7fda777fe640 1 -- 172.21.15.172:0/1829156745 >> v1:172.21.15.172:6789/0 conn(0x7fda780cfd90 legacy=0x7fda780d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.069+0000 7fda777fe640 1 -- 172.21.15.172:0/1829156745 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda780d2510 con 0x7fda780d11a0 2024-04-14T08:33:14.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.069+0000 7fda76ffd640 1 --2- 172.21.15.172:0/1829156745 >> v2:172.21.15.177:3300/0 conn(0x7fda7815e4d0 0x7fda780cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:14.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.070+0000 7fda777fe640 1 --2- 172.21.15.172:0/1829156745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780d11a0 0x7fda780b39d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fda68009d90 tx=0x7fda6802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceaffc34ccf16ad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:14.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.070+0000 7fda75ffb640 1 -- 172.21.15.172:0/1829156745 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68039070 con 0x7fda780d11a0 2024-04-14T08:33:14.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.070+0000 7fda75ffb640 1 -- 172.21.15.172:0/1829156745 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6802cbc0 con 0x7fda780d11a0 2024-04-14T08:33:14.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.070+0000 7fda75ffb640 1 -- 172.21.15.172:0/1829156745 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68034440 con 0x7fda780d11a0 2024-04-14T08:33:14.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.070+0000 7fda7d8a6640 1 -- 172.21.15.172:0/1829156745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780d11a0 msgr2=0x7fda780b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.070+0000 7fda7d8a6640 1 --2- 172.21.15.172:0/1829156745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780d11a0 0x7fda780b39d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fda68009d90 tx=0x7fda6802be10 comp rx=0 tx=0).stop 2024-04-14T08:33:14.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.070+0000 7fda7d8a6640 1 -- 172.21.15.172:0/1829156745 shutdown_connections 2024-04-14T08:33:14.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.070+0000 7fda7d8a6640 1 --2- 172.21.15.172:0/1829156745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780d11a0 0x7fda780b39d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.070+0000 7fda7d8a6640 1 --2- 172.21.15.172:0/1829156745 >> v2:172.21.15.177:3300/0 conn(0x7fda7815e4d0 0x7fda780cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.070+0000 7fda7d8a6640 1 -- 172.21.15.172:0/1829156745 >> 172.21.15.172:0/1829156745 conn(0x7fda7809d590 msgr2=0x7fda780d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:14.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.070+0000 7fda7d8a6640 1 -- 172.21.15.172:0/1829156745 shutdown_connections 2024-04-14T08:33:14.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.070+0000 7fda7d8a6640 1 -- 172.21.15.172:0/1829156745 wait complete. 2024-04-14T08:33:14.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.071+0000 7fda7d8a6640 1 Processor -- start 2024-04-14T08:33:14.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.072+0000 7fda7d8a6640 1 -- start start 2024-04-14T08:33:14.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.072+0000 7fda7d8a6640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780cfd90 0x7fda781f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.072+0000 7fda7d8a6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda780d11a0 0x7fda781f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.072+0000 7fda7d8a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda7815e4d0 0x7fda780b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.072+0000 7fda7d8a6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda780d2ff0 con 0x7fda7815e4d0 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.072+0000 7fda7d8a6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fda780d2ad0 con 0x7fda780d11a0 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.072+0000 7fda7d8a6640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fda780b7380 con 0x7fda780cfd90 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.072+0000 7fda777fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda7815e4d0 0x7fda780b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda76ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780cfd90 0x7fda781f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda767fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda780d11a0 0x7fda781f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda777fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda7815e4d0 0x7fda780b6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49668/0 (socket says 172.21.15.172:49668) 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda777fe640 1 -- 172.21.15.172:0/2303109053 learned_addr learned my addr 172.21.15.172:0/2303109053 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda76ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780cfd90 0x7fda781f3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:43276/0 (socket says 172.21.15.172:43276) 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda777fe640 1 -- 172.21.15.172:0/2303109053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780cfd90 msgr2=0x7fda781f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda777fe640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780cfd90 0x7fda781f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda777fe640 1 -- 172.21.15.172:0/2303109053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda780d11a0 msgr2=0x7fda781f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda777fe640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda780d11a0 0x7fda781f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda777fe640 1 -- 172.21.15.172:0/2303109053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda780b75b0 con 0x7fda7815e4d0 2024-04-14T08:33:14.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda767fc640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda780d11a0 0x7fda781f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:14.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda76ffd640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780cfd90 0x7fda781f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:14.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda777fe640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda7815e4d0 0x7fda780b6e40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fda68009eb0 tx=0x7fda68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:14.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda6ffff640 1 -- 172.21.15.172:0/2303109053 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68039050 con 0x7fda7815e4d0 2024-04-14T08:33:14.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda6ffff640 1 -- 172.21.15.172:0/2303109053 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda68045070 con 0x7fda7815e4d0 2024-04-14T08:33:14.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda7d8a6640 1 -- 172.21.15.172:0/2303109053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda780b7890 con 0x7fda7815e4d0 2024-04-14T08:33:14.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda6ffff640 1 -- 172.21.15.172:0/2303109053 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda68009450 con 0x7fda7815e4d0 2024-04-14T08:33:14.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.073+0000 7fda7d8a6640 1 -- 172.21.15.172:0/2303109053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda780b7e50 con 0x7fda7815e4d0 2024-04-14T08:33:14.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.074+0000 7fda7d8a6640 1 -- 172.21.15.172:0/2303109053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda78068520 con 0x7fda7815e4d0 2024-04-14T08:33:14.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.075+0000 7fda6ffff640 1 -- 172.21.15.172:0/2303109053 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda6803e830 con 0x7fda7815e4d0 2024-04-14T08:33:14.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.075+0000 7fda6ffff640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda5806c8f0 0x7fda5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.075+0000 7fda6ffff640 1 -- 172.21.15.172:0/2303109053 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fda680ae9e0 con 0x7fda7815e4d0 2024-04-14T08:33:14.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.075+0000 7fda76ffd640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda5806c8f0 0x7fda5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.075+0000 7fda76ffd640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda5806c8f0 0x7fda5806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fda70009560 tx=0x7fda70009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:14.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.080+0000 7fda6ffff640 1 -- 172.21.15.172:0/2303109053 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fda6807c4c0 con 0x7fda7815e4d0 2024-04-14T08:33:14.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.255+0000 7fda7d8a6640 1 -- 172.21.15.172:0/2303109053 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda780d1800 con 0x7fda5806c8f0 2024-04-14T08:33:14.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.262+0000 7fda6ffff640 1 -- 172.21.15.172:0/2303109053 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fda780d1800 con 0x7fda5806c8f0 2024-04-14T08:33:14.266 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:14.268 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:33:14.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.266+0000 7fda7d8a6640 1 -- 172.21.15.172:0/2303109053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda5806c8f0 msgr2=0x7fda5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.266+0000 7fda7d8a6640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda5806c8f0 0x7fda5806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fda70009560 tx=0x7fda70009190 comp rx=0 tx=0).stop 2024-04-14T08:33:14.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.266+0000 7fda7d8a6640 1 -- 172.21.15.172:0/2303109053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda7815e4d0 msgr2=0x7fda780b6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.266+0000 7fda7d8a6640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda7815e4d0 0x7fda780b6e40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fda68009eb0 tx=0x7fda68004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:14.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.266+0000 7fda7d8a6640 1 -- 172.21.15.172:0/2303109053 shutdown_connections 2024-04-14T08:33:14.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.266+0000 7fda7d8a6640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda5806c8f0 0x7fda5806edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.266+0000 7fda7d8a6640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda7815e4d0 0x7fda780b6e40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.266+0000 7fda7d8a6640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda780d11a0 0x7fda781f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.266+0000 7fda7d8a6640 1 --2- 172.21.15.172:0/2303109053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda780cfd90 0x7fda781f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.266+0000 7fda7d8a6640 1 -- 172.21.15.172:0/2303109053 >> 172.21.15.172:0/2303109053 conn(0x7fda7809d590 msgr2=0x7fda78074a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:14.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.266+0000 7fda7d8a6640 1 -- 172.21.15.172:0/2303109053 shutdown_connections 2024-04-14T08:33:14.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.266+0000 7fda7d8a6640 1 -- 172.21.15.172:0/2303109053 wait complete. 2024-04-14T08:33:14.277 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-14T08:33:13.125275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219480,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755240,"statfs":{"total":773094113280,"available":772869365760,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002879"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219633+0000","last_change":"2024-04-14T08:32:53.196809+0000","last_active":"2024-04-14T08:33:01.219633+0000","last_peered":"2024-04-14T08:33:01.219633+0000","last_clean":"2024-04-14T08:33:01.219633+0000","last_became_active":"2024-04-14T08:32:53.196554+0000","last_became_peered":"2024-04-14T08:32:53.196554+0000","last_unstale":"2024-04-14T08:33:01.219633+0000","last_undegraded":"2024-04-14T08:33:01.219633+0000","last_fullsized":"2024-04-14T08:33:01.219633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:43:41.734066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225244+0000","last_change":"2024-04-14T08:32:54.204652+0000","last_active":"2024-04-14T08:32:59.225244+0000","last_peered":"2024-04-14T08:32:59.225244+0000","last_clean":"2024-04-14T08:32:59.225244+0000","last_became_active":"2024-04-14T08:32:54.204361+0000","last_became_peered":"2024-04-14T08:32:54.204361+0000","last_unstale":"2024-04-14T08:32:59.225244+0000","last_undegraded":"2024-04-14T08:32:59.225244+0000","last_fullsized":"2024-04-14T08:32:59.225244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224356+0000","last_change":"2024-04-14T08:32:58.244817+0000","last_active":"2024-04-14T08:32:59.224356+0000","last_peered":"2024-04-14T08:32:59.224356+0000","last_clean":"2024-04-14T08:32:59.224356+0000","last_became_active":"2024-04-14T08:32:58.243913+0000","last_became_peered":"2024-04-14T08:32:58.243913+0000","last_unstale":"2024-04-14T08:32:59.224356+0000","last_undegraded":"2024-04-14T08:32:59.224356+0000","last_fullsized":"2024-04-14T08:32:59.224356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:53.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225340+0000","last_change":"2024-04-14T08:32:53.202277+0000","last_active":"2024-04-14T08:32:59.225340+0000","last_peered":"2024-04-14T08:32:59.225340+0000","last_clean":"2024-04-14T08:32:59.225340+0000","last_became_active":"2024-04-14T08:32:53.201980+0000","last_became_peered":"2024-04-14T08:32:53.201980+0000","last_unstale":"2024-04-14T08:32:59.225340+0000","last_undegraded":"2024-04-14T08:32:59.225340+0000","last_fullsized":"2024-04-14T08:32:59.225340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:22:15.886932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220320+0000","last_change":"2024-04-14T08:32:54.211674+0000","last_active":"2024-04-14T08:33:01.220320+0000","last_peered":"2024-04-14T08:33:01.220320+0000","last_clean":"2024-04-14T08:33:01.220320+0000","last_became_active":"2024-04-14T08:32:54.211323+0000","last_became_peered":"2024-04-14T08:32:54.211323+0000","last_unstale":"2024-04-14T08:33:01.220320+0000","last_undegraded":"2024-04-14T08:33:01.220320+0000","last_fullsized":"2024-04-14T08:33:01.220320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224678+0000","last_change":"2024-04-14T08:32:58.242594+0000","last_active":"2024-04-14T08:32:59.224678+0000","last_peered":"2024-04-14T08:32:59.224678+0000","last_clean":"2024-04-14T08:32:59.224678+0000","last_became_active":"2024-04-14T08:32:58.242238+0000","last_became_peered":"2024-04-14T08:32:58.242238+0000","last_unstale":"2024-04-14T08:32:59.224678+0000","last_undegraded":"2024-04-14T08:32:59.224678+0000","last_fullsized":"2024-04-14T08:32:59.224678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:45.716305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226152+0000","last_change":"2024-04-14T08:32:53.200133+0000","last_active":"2024-04-14T08:32:59.226152+0000","last_peered":"2024-04-14T08:32:59.226152+0000","last_clean":"2024-04-14T08:32:59.226152+0000","last_became_active":"2024-04-14T08:32:53.199380+0000","last_became_peered":"2024-04-14T08:32:53.199380+0000","last_unstale":"2024-04-14T08:32:59.226152+0000","last_undegraded":"2024-04-14T08:32:59.226152+0000","last_fullsized":"2024-04-14T08:32:59.226152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405017+0000","last_change":"2024-04-14T08:32:54.208922+0000","last_active":"2024-04-14T08:32:59.405017+0000","last_peered":"2024-04-14T08:32:59.405017+0000","last_clean":"2024-04-14T08:32:59.405017+0000","last_became_active":"2024-04-14T08:32:54.206845+0000","last_became_peered":"2024-04-14T08:32:54.206845+0000","last_unstale":"2024-04-14T08:32:59.405017+0000","last_undegraded":"2024-04-14T08:32:59.405017+0000","last_fullsized":"2024-04-14T08:32:59.405017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225258+0000","last_change":"2024-04-14T08:32:58.242683+0000","last_active":"2024-04-14T08:32:59.225258+0000","last_peered":"2024-04-14T08:32:59.225258+0000","last_clean":"2024-04-14T08:32:59.225258+0000","last_became_active":"2024-04-14T08:32:58.242189+0000","last_became_peered":"2024-04-14T08:32:58.242189+0000","last_unstale":"2024-04-14T08:32:59.225258+0000","last_undegraded":"2024-04-14T08:32:59.225258+0000","last_fullsized":"2024-04-14T08:32:59.225258+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:52.084133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225875+0000","last_change":"2024-04-14T08:32:53.200880+0000","last_active":"2024-04-14T08:32:59.225875+0000","last_peered":"2024-04-14T08:32:59.225875+0000","last_clean":"2024-04-14T08:32:59.225875+0000","last_became_active":"2024-04-14T08:32:53.200615+0000","last_became_peered":"2024-04-14T08:32:53.200615+0000","last_unstale":"2024-04-14T08:32:59.225875+0000","last_undegraded":"2024-04-14T08:32:59.225875+0000","last_fullsized":"2024-04-14T08:32:59.225875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:40:24.515179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405104+0000","last_change":"2024-04-14T08:32:54.207091+0000","last_active":"2024-04-14T08:32:59.405104+0000","last_peered":"2024-04-14T08:32:59.405104+0000","last_clean":"2024-04-14T08:32:59.405104+0000","last_became_active":"2024-04-14T08:32:54.206685+0000","last_became_peered":"2024-04-14T08:32:54.206685+0000","last_unstale":"2024-04-14T08:32:59.405104+0000","last_undegraded":"2024-04-14T08:32:59.405104+0000","last_fullsized":"2024-04-14T08:32:59.405104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:41:14.768103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224936+0000","last_change":"2024-04-14T08:32:58.237435+0000","last_active":"2024-04-14T08:32:59.224936+0000","last_peered":"2024-04-14T08:32:59.224936+0000","last_clean":"2024-04-14T08:32:59.224936+0000","last_became_active":"2024-04-14T08:32:58.237056+0000","last_became_peered":"2024-04-14T08:32:58.237056+0000","last_unstale":"2024-04-14T08:32:59.224936+0000","last_undegraded":"2024-04-14T08:32:59.224936+0000","last_fullsized":"2024-04-14T08:32:59.224936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:20.577556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115125+0000","last_change":"2024-04-14T08:32:54.202442+0000","last_active":"2024-04-14T08:33:01.115125+0000","last_peered":"2024-04-14T08:33:01.115125+0000","last_clean":"2024-04-14T08:33:01.115125+0000","last_became_active":"2024-04-14T08:32:54.201860+0000","last_became_peered":"2024-04-14T08:32:54.201860+0000","last_unstale":"2024-04-14T08:33:01.115125+0000","last_undegraded":"2024-04-14T08:33:01.115125+0000","last_fullsized":"2024-04-14T08:33:01.115125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226036+0000","last_change":"2024-04-14T08:32:53.194232+0000","last_active":"2024-04-14T08:32:59.226036+0000","last_peered":"2024-04-14T08:32:59.226036+0000","last_clean":"2024-04-14T08:32:59.226036+0000","last_became_active":"2024-04-14T08:32:53.193961+0000","last_became_peered":"2024-04-14T08:32:53.193961+0000","last_unstale":"2024-04-14T08:32:59.226036+0000","last_undegraded":"2024-04-14T08:32:59.226036+0000","last_fullsized":"2024-04-14T08:32:59.226036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:32.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226274+0000","last_change":"2024-04-14T08:32:58.239778+0000","last_active":"2024-04-14T08:32:59.226274+0000","last_peered":"2024-04-14T08:32:59.226274+0000","last_clean":"2024-04-14T08:32:59.226274+0000","last_became_active":"2024-04-14T08:32:58.239445+0000","last_became_peered":"2024-04-14T08:32:58.239445+0000","last_unstale":"2024-04-14T08:32:59.226274+0000","last_undegraded":"2024-04-14T08:32:59.226274+0000","last_fullsized":"2024-04-14T08:32:59.226274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:53.001768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224657+0000","last_change":"2024-04-14T08:32:54.202322+0000","last_active":"2024-04-14T08:32:59.224657+0000","last_peered":"2024-04-14T08:32:59.224657+0000","last_clean":"2024-04-14T08:32:59.224657+0000","last_became_active":"2024-04-14T08:32:54.202065+0000","last_became_peered":"2024-04-14T08:32:54.202065+0000","last_unstale":"2024-04-14T08:32:59.224657+0000","last_undegraded":"2024-04-14T08:32:59.224657+0000","last_fullsized":"2024-04-14T08:32:59.224657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610551+0000","last_change":"2024-04-14T08:32:53.195288+0000","last_active":"2024-04-14T08:33:00.610551+0000","last_peered":"2024-04-14T08:33:00.610551+0000","last_clean":"2024-04-14T08:33:00.610551+0000","last_became_active":"2024-04-14T08:32:53.195009+0000","last_became_peered":"2024-04-14T08:32:53.195009+0000","last_unstale":"2024-04-14T08:33:00.610551+0000","last_undegraded":"2024-04-14T08:33:00.610551+0000","last_fullsized":"2024-04-14T08:33:00.610551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225775+0000","last_change":"2024-04-14T08:32:58.238192+0000","last_active":"2024-04-14T08:32:59.225775+0000","last_peered":"2024-04-14T08:32:59.225775+0000","last_clean":"2024-04-14T08:32:59.225775+0000","last_became_active":"2024-04-14T08:32:58.237917+0000","last_became_peered":"2024-04-14T08:32:58.237917+0000","last_unstale":"2024-04-14T08:32:59.225775+0000","last_undegraded":"2024-04-14T08:32:59.225775+0000","last_fullsized":"2024-04-14T08:32:59.225775+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:41:18.809183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224457+0000","last_change":"2024-04-14T08:32:54.195485+0000","last_active":"2024-04-14T08:32:59.224457+0000","last_peered":"2024-04-14T08:32:59.224457+0000","last_clean":"2024-04-14T08:32:59.224457+0000","last_became_active":"2024-04-14T08:32:54.195192+0000","last_became_peered":"2024-04-14T08:32:54.195192+0000","last_unstale":"2024-04-14T08:32:59.224457+0000","last_undegraded":"2024-04-14T08:32:59.224457+0000","last_fullsized":"2024-04-14T08:32:59.224457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610087+0000","last_change":"2024-04-14T08:32:53.193700+0000","last_active":"2024-04-14T08:33:00.610087+0000","last_peered":"2024-04-14T08:33:00.610087+0000","last_clean":"2024-04-14T08:33:00.610087+0000","last_became_active":"2024-04-14T08:32:53.193412+0000","last_became_peered":"2024-04-14T08:32:53.193412+0000","last_unstale":"2024-04-14T08:33:00.610087+0000","last_undegraded":"2024-04-14T08:33:00.610087+0000","last_fullsized":"2024-04-14T08:33:00.610087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610793+0000","last_change":"2024-04-14T08:32:58.246138+0000","last_active":"2024-04-14T08:33:00.610793+0000","last_peered":"2024-04-14T08:33:00.610793+0000","last_clean":"2024-04-14T08:33:00.610793+0000","last_became_active":"2024-04-14T08:32:58.245731+0000","last_became_peered":"2024-04-14T08:32:58.245731+0000","last_unstale":"2024-04-14T08:33:00.610793+0000","last_undegraded":"2024-04-14T08:33:00.610793+0000","last_fullsized":"2024-04-14T08:33:00.610793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:54.763380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224951+0000","last_change":"2024-04-14T08:32:54.203712+0000","last_active":"2024-04-14T08:32:59.224951+0000","last_peered":"2024-04-14T08:32:59.224951+0000","last_clean":"2024-04-14T08:32:59.224951+0000","last_became_active":"2024-04-14T08:32:54.203433+0000","last_became_peered":"2024-04-14T08:32:54.203433+0000","last_unstale":"2024-04-14T08:32:59.224951+0000","last_undegraded":"2024-04-14T08:32:59.224951+0000","last_fullsized":"2024-04-14T08:32:59.224951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:15.534364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224734+0000","last_change":"2024-04-14T08:32:53.197850+0000","last_active":"2024-04-14T08:32:59.224734+0000","last_peered":"2024-04-14T08:32:59.224734+0000","last_clean":"2024-04-14T08:32:59.224734+0000","last_became_active":"2024-04-14T08:32:53.197575+0000","last_became_peered":"2024-04-14T08:32:53.197575+0000","last_unstale":"2024-04-14T08:32:59.224734+0000","last_undegraded":"2024-04-14T08:32:59.224734+0000","last_fullsized":"2024-04-14T08:32:59.224734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:04:36.870813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225529+0000","last_change":"2024-04-14T08:32:58.243185+0000","last_active":"2024-04-14T08:32:59.225529+0000","last_peered":"2024-04-14T08:32:59.225529+0000","last_clean":"2024-04-14T08:32:59.225529+0000","last_became_active":"2024-04-14T08:32:58.242880+0000","last_became_peered":"2024-04-14T08:32:58.242880+0000","last_unstale":"2024-04-14T08:32:59.225529+0000","last_undegraded":"2024-04-14T08:32:59.225529+0000","last_fullsized":"2024-04-14T08:32:59.225529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:47.641204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220461+0000","last_change":"2024-04-14T08:32:54.194367+0000","last_active":"2024-04-14T08:33:01.220461+0000","last_peered":"2024-04-14T08:33:01.220461+0000","last_clean":"2024-04-14T08:33:01.220461+0000","last_became_active":"2024-04-14T08:32:54.194111+0000","last_became_peered":"2024-04-14T08:32:54.194111+0000","last_unstale":"2024-04-14T08:33:01.220461+0000","last_undegraded":"2024-04-14T08:33:01.220461+0000","last_fullsized":"2024-04-14T08:33:01.220461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:05:03.511225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115632+0000","last_change":"2024-04-14T08:32:53.194969+0000","last_active":"2024-04-14T08:33:01.115632+0000","last_peered":"2024-04-14T08:33:01.115632+0000","last_clean":"2024-04-14T08:33:01.115632+0000","last_became_active":"2024-04-14T08:32:53.194665+0000","last_became_peered":"2024-04-14T08:32:53.194665+0000","last_unstale":"2024-04-14T08:33:01.115632+0000","last_undegraded":"2024-04-14T08:33:01.115632+0000","last_fullsized":"2024-04-14T08:33:01.115632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224778+0000","last_change":"2024-04-14T08:32:58.237006+0000","last_active":"2024-04-14T08:32:59.224778+0000","last_peered":"2024-04-14T08:32:59.224778+0000","last_clean":"2024-04-14T08:32:59.224778+0000","last_became_active":"2024-04-14T08:32:58.236366+0000","last_became_peered":"2024-04-14T08:32:58.236366+0000","last_unstale":"2024-04-14T08:32:59.224778+0000","last_undegraded":"2024-04-14T08:32:59.224778+0000","last_fullsized":"2024-04-14T08:32:59.224778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:49.446712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225123+0000","last_change":"2024-04-14T08:32:54.205064+0000","last_active":"2024-04-14T08:32:59.225123+0000","last_peered":"2024-04-14T08:32:59.225123+0000","last_clean":"2024-04-14T08:32:59.225123+0000","last_became_active":"2024-04-14T08:32:54.203946+0000","last_became_peered":"2024-04-14T08:32:54.203946+0000","last_unstale":"2024-04-14T08:32:59.225123+0000","last_undegraded":"2024-04-14T08:32:59.225123+0000","last_fullsized":"2024-04-14T08:32:59.225123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:16.536477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610666+0000","last_change":"2024-04-14T08:32:53.198148+0000","last_active":"2024-04-14T08:33:00.610666+0000","last_peered":"2024-04-14T08:33:00.610666+0000","last_clean":"2024-04-14T08:33:00.610666+0000","last_became_active":"2024-04-14T08:32:53.197878+0000","last_became_peered":"2024-04-14T08:32:53.197878+0000","last_unstale":"2024-04-14T08:33:00.610666+0000","last_undegraded":"2024-04-14T08:33:00.610666+0000","last_fullsized":"2024-04-14T08:33:00.610666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224235+0000","last_change":"2024-04-14T08:32:58.246772+0000","last_active":"2024-04-14T08:32:59.224235+0000","last_peered":"2024-04-14T08:32:59.224235+0000","last_clean":"2024-04-14T08:32:59.224235+0000","last_became_active":"2024-04-14T08:32:58.246388+0000","last_became_peered":"2024-04-14T08:32:58.246388+0000","last_unstale":"2024-04-14T08:32:59.224235+0000","last_undegraded":"2024-04-14T08:32:59.224235+0000","last_fullsized":"2024-04-14T08:32:59.224235+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:54.763380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610409+0000","last_change":"2024-04-14T08:32:54.194202+0000","last_active":"2024-04-14T08:33:00.610409+0000","last_peered":"2024-04-14T08:33:00.610409+0000","last_clean":"2024-04-14T08:33:00.610409+0000","last_became_active":"2024-04-14T08:32:54.193938+0000","last_became_peered":"2024-04-14T08:32:54.193938+0000","last_unstale":"2024-04-14T08:33:00.610409+0000","last_undegraded":"2024-04-14T08:33:00.610409+0000","last_fullsized":"2024-04-14T08:33:00.610409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226263+0000","last_change":"2024-04-14T08:32:53.202270+0000","last_active":"2024-04-14T08:32:59.226263+0000","last_peered":"2024-04-14T08:32:59.226263+0000","last_clean":"2024-04-14T08:32:59.226263+0000","last_became_active":"2024-04-14T08:32:53.201945+0000","last_became_peered":"2024-04-14T08:32:53.201945+0000","last_unstale":"2024-04-14T08:32:59.226263+0000","last_undegraded":"2024-04-14T08:32:59.226263+0000","last_fullsized":"2024-04-14T08:32:59.226263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219955+0000","last_change":"2024-04-14T08:32:58.248277+0000","last_active":"2024-04-14T08:33:01.219955+0000","last_peered":"2024-04-14T08:33:01.219955+0000","last_clean":"2024-04-14T08:33:01.219955+0000","last_became_active":"2024-04-14T08:32:58.247535+0000","last_became_peered":"2024-04-14T08:32:58.247535+0000","last_unstale":"2024-04-14T08:33:01.219955+0000","last_undegraded":"2024-04-14T08:33:01.219955+0000","last_fullsized":"2024-04-14T08:33:01.219955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:05.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405198+0000","last_change":"2024-04-14T08:32:54.203448+0000","last_active":"2024-04-14T08:32:59.405198+0000","last_peered":"2024-04-14T08:32:59.405198+0000","last_clean":"2024-04-14T08:32:59.405198+0000","last_became_active":"2024-04-14T08:32:54.203170+0000","last_became_peered":"2024-04-14T08:32:54.203170+0000","last_unstale":"2024-04-14T08:32:59.405198+0000","last_undegraded":"2024-04-14T08:32:59.405198+0000","last_fullsized":"2024-04-14T08:32:59.405198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:01.152061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610433+0000","last_change":"2024-04-14T08:32:53.187816+0000","last_active":"2024-04-14T08:33:00.610433+0000","last_peered":"2024-04-14T08:33:00.610433+0000","last_clean":"2024-04-14T08:33:00.610433+0000","last_became_active":"2024-04-14T08:32:53.187519+0000","last_became_peered":"2024-04-14T08:32:53.187519+0000","last_unstale":"2024-04-14T08:33:00.610433+0000","last_undegraded":"2024-04-14T08:33:00.610433+0000","last_fullsized":"2024-04-14T08:33:00.610433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610230+0000","last_change":"2024-04-14T08:32:58.246145+0000","last_active":"2024-04-14T08:33:00.610230+0000","last_peered":"2024-04-14T08:33:00.610230+0000","last_clean":"2024-04-14T08:33:00.610230+0000","last_became_active":"2024-04-14T08:32:58.245792+0000","last_became_peered":"2024-04-14T08:32:58.245792+0000","last_unstale":"2024-04-14T08:33:00.610230+0000","last_undegraded":"2024-04-14T08:33:00.610230+0000","last_fullsized":"2024-04-14T08:33:00.610230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:05.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225311+0000","last_change":"2024-04-14T08:32:54.192476+0000","last_active":"2024-04-14T08:32:59.225311+0000","last_peered":"2024-04-14T08:32:59.225311+0000","last_clean":"2024-04-14T08:32:59.225311+0000","last_became_active":"2024-04-14T08:32:54.192182+0000","last_became_peered":"2024-04-14T08:32:54.192182+0000","last_unstale":"2024-04-14T08:32:59.225311+0000","last_undegraded":"2024-04-14T08:32:59.225311+0000","last_fullsized":"2024-04-14T08:32:59.225311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:05.453493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405000+0000","last_change":"2024-04-14T08:32:53.201969+0000","last_active":"2024-04-14T08:32:59.405000+0000","last_peered":"2024-04-14T08:32:59.405000+0000","last_clean":"2024-04-14T08:32:59.405000+0000","last_became_active":"2024-04-14T08:32:53.201611+0000","last_became_peered":"2024-04-14T08:32:53.201611+0000","last_unstale":"2024-04-14T08:32:59.405000+0000","last_undegraded":"2024-04-14T08:32:59.405000+0000","last_fullsized":"2024-04-14T08:32:59.405000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:54:04.446403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610320+0000","last_change":"2024-04-14T08:32:58.249518+0000","last_active":"2024-04-14T08:33:00.610320+0000","last_peered":"2024-04-14T08:33:00.610320+0000","last_clean":"2024-04-14T08:33:00.610320+0000","last_became_active":"2024-04-14T08:32:58.249079+0000","last_became_peered":"2024-04-14T08:32:58.249079+0000","last_unstale":"2024-04-14T08:33:00.610320+0000","last_undegraded":"2024-04-14T08:33:00.610320+0000","last_fullsized":"2024-04-14T08:33:00.610320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:16.724895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610081+0000","last_change":"2024-04-14T08:32:54.194141+0000","last_active":"2024-04-14T08:33:00.610081+0000","last_peered":"2024-04-14T08:33:00.610081+0000","last_clean":"2024-04-14T08:33:00.610081+0000","last_became_active":"2024-04-14T08:32:54.193884+0000","last_became_peered":"2024-04-14T08:32:54.193884+0000","last_unstale":"2024-04-14T08:33:00.610081+0000","last_undegraded":"2024-04-14T08:33:00.610081+0000","last_fullsized":"2024-04-14T08:33:00.610081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278559+0000","last_change":"2024-04-14T08:32:53.196690+0000","last_active":"2024-04-14T08:33:01.278559+0000","last_peered":"2024-04-14T08:33:01.278559+0000","last_clean":"2024-04-14T08:33:01.278559+0000","last_became_active":"2024-04-14T08:32:53.195977+0000","last_became_peered":"2024-04-14T08:32:53.195977+0000","last_unstale":"2024-04-14T08:33:01.278559+0000","last_undegraded":"2024-04-14T08:33:01.278559+0000","last_fullsized":"2024-04-14T08:33:01.278559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610859+0000","last_change":"2024-04-14T08:32:58.249862+0000","last_active":"2024-04-14T08:33:00.610859+0000","last_peered":"2024-04-14T08:33:00.610859+0000","last_clean":"2024-04-14T08:33:00.610859+0000","last_became_active":"2024-04-14T08:32:58.248928+0000","last_became_peered":"2024-04-14T08:32:58.248928+0000","last_unstale":"2024-04-14T08:33:00.610859+0000","last_undegraded":"2024-04-14T08:33:00.610859+0000","last_fullsized":"2024-04-14T08:33:00.610859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:53.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219816+0000","last_change":"2024-04-14T08:32:54.206032+0000","last_active":"2024-04-14T08:33:01.219816+0000","last_peered":"2024-04-14T08:33:01.219816+0000","last_clean":"2024-04-14T08:33:01.219816+0000","last_became_active":"2024-04-14T08:32:54.205781+0000","last_became_peered":"2024-04-14T08:32:54.205781+0000","last_unstale":"2024-04-14T08:33:01.219816+0000","last_undegraded":"2024-04-14T08:33:01.219816+0000","last_fullsized":"2024-04-14T08:33:01.219816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:25:28.934928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224825+0000","last_change":"2024-04-14T08:32:53.195805+0000","last_active":"2024-04-14T08:32:59.224825+0000","last_peered":"2024-04-14T08:32:59.224825+0000","last_clean":"2024-04-14T08:32:59.224825+0000","last_became_active":"2024-04-14T08:32:53.195442+0000","last_became_peered":"2024-04-14T08:32:53.195442+0000","last_unstale":"2024-04-14T08:32:59.224825+0000","last_undegraded":"2024-04-14T08:32:59.224825+0000","last_fullsized":"2024-04-14T08:32:59.224825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:48.807150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220403+0000","last_change":"2024-04-14T08:32:58.247104+0000","last_active":"2024-04-14T08:33:01.220403+0000","last_peered":"2024-04-14T08:33:01.220403+0000","last_clean":"2024-04-14T08:33:01.220403+0000","last_became_active":"2024-04-14T08:32:58.246758+0000","last_became_peered":"2024-04-14T08:32:58.246758+0000","last_unstale":"2024-04-14T08:33:01.220403+0000","last_undegraded":"2024-04-14T08:33:01.220403+0000","last_fullsized":"2024-04-14T08:33:01.220403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:53.980304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224440+0000","last_change":"2024-04-14T08:32:54.209924+0000","last_active":"2024-04-14T08:32:59.224440+0000","last_peered":"2024-04-14T08:32:59.224440+0000","last_clean":"2024-04-14T08:32:59.224440+0000","last_became_active":"2024-04-14T08:32:54.209703+0000","last_became_peered":"2024-04-14T08:32:54.209703+0000","last_unstale":"2024-04-14T08:32:59.224440+0000","last_undegraded":"2024-04-14T08:32:59.224440+0000","last_fullsized":"2024-04-14T08:32:59.224440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404884+0000","last_change":"2024-04-14T08:32:53.197701+0000","last_active":"2024-04-14T08:32:59.404884+0000","last_peered":"2024-04-14T08:32:59.404884+0000","last_clean":"2024-04-14T08:32:59.404884+0000","last_became_active":"2024-04-14T08:32:53.197291+0000","last_became_peered":"2024-04-14T08:32:53.197291+0000","last_unstale":"2024-04-14T08:32:59.404884+0000","last_undegraded":"2024-04-14T08:32:59.404884+0000","last_fullsized":"2024-04-14T08:32:59.404884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:54:34.240052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224914+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:32:59.224914+0000","last_peered":"2024-04-14T08:32:59.224914+0000","last_clean":"2024-04-14T08:32:59.224914+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:32:59.224914+0000","last_undegraded":"2024-04-14T08:32:59.224914+0000","last_fullsized":"2024-04-14T08:32:59.224914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115413+0000","last_change":"2024-04-14T08:32:58.237249+0000","last_active":"2024-04-14T08:33:01.115413+0000","last_peered":"2024-04-14T08:33:01.115413+0000","last_clean":"2024-04-14T08:33:01.115413+0000","last_became_active":"2024-04-14T08:32:58.236861+0000","last_became_peered":"2024-04-14T08:32:58.236861+0000","last_unstale":"2024-04-14T08:33:01.115413+0000","last_undegraded":"2024-04-14T08:33:01.115413+0000","last_fullsized":"2024-04-14T08:33:01.115413+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:05.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226018+0000","last_change":"2024-04-14T08:32:54.211527+0000","last_active":"2024-04-14T08:32:59.226018+0000","last_peered":"2024-04-14T08:32:59.226018+0000","last_clean":"2024-04-14T08:32:59.226018+0000","last_became_active":"2024-04-14T08:32:54.211268+0000","last_became_peered":"2024-04-14T08:32:54.211268+0000","last_unstale":"2024-04-14T08:32:59.226018+0000","last_undegraded":"2024-04-14T08:32:59.226018+0000","last_fullsized":"2024-04-14T08:32:59.226018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:41:14.768103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278564+0000","last_change":"2024-04-14T08:32:53.198176+0000","last_active":"2024-04-14T08:33:01.278564+0000","last_peered":"2024-04-14T08:33:01.278564+0000","last_clean":"2024-04-14T08:33:01.278564+0000","last_became_active":"2024-04-14T08:32:53.197626+0000","last_became_peered":"2024-04-14T08:32:53.197626+0000","last_unstale":"2024-04-14T08:33:01.278564+0000","last_undegraded":"2024-04-14T08:33:01.278564+0000","last_fullsized":"2024-04-14T08:33:01.278564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:10:49.016747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115290+0000","last_change":"2024-04-14T08:32:58.240943+0000","last_active":"2024-04-14T08:33:01.115290+0000","last_peered":"2024-04-14T08:33:01.115290+0000","last_clean":"2024-04-14T08:33:01.115290+0000","last_became_active":"2024-04-14T08:32:58.240555+0000","last_became_peered":"2024-04-14T08:32:58.240555+0000","last_unstale":"2024-04-14T08:33:01.115290+0000","last_undegraded":"2024-04-14T08:33:01.115290+0000","last_fullsized":"2024-04-14T08:33:01.115290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:53.001768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405061+0000","last_change":"2024-04-14T08:32:54.205351+0000","last_active":"2024-04-14T08:32:59.405061+0000","last_peered":"2024-04-14T08:32:59.405061+0000","last_clean":"2024-04-14T08:32:59.405061+0000","last_became_active":"2024-04-14T08:32:54.205086+0000","last_became_peered":"2024-04-14T08:32:54.205086+0000","last_unstale":"2024-04-14T08:32:59.405061+0000","last_undegraded":"2024-04-14T08:32:59.405061+0000","last_fullsized":"2024-04-14T08:32:59.405061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:43.600124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610474+0000","last_change":"2024-04-14T08:32:53.194785+0000","last_active":"2024-04-14T08:33:00.610474+0000","last_peered":"2024-04-14T08:33:00.610474+0000","last_clean":"2024-04-14T08:33:00.610474+0000","last_became_active":"2024-04-14T08:32:53.194535+0000","last_became_peered":"2024-04-14T08:32:53.194535+0000","last_unstale":"2024-04-14T08:33:00.610474+0000","last_undegraded":"2024-04-14T08:33:00.610474+0000","last_fullsized":"2024-04-14T08:33:00.610474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:48.807150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220190+0000","last_change":"2024-04-14T08:32:58.246826+0000","last_active":"2024-04-14T08:33:01.220190+0000","last_peered":"2024-04-14T08:33:01.220190+0000","last_clean":"2024-04-14T08:33:01.220190+0000","last_became_active":"2024-04-14T08:32:58.246174+0000","last_became_peered":"2024-04-14T08:32:58.246174+0000","last_unstale":"2024-04-14T08:33:01.220190+0000","last_undegraded":"2024-04-14T08:33:01.220190+0000","last_fullsized":"2024-04-14T08:33:01.220190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:57.127403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226211+0000","last_change":"2024-04-14T08:32:54.200509+0000","last_active":"2024-04-14T08:32:59.226211+0000","last_peered":"2024-04-14T08:32:59.226211+0000","last_clean":"2024-04-14T08:32:59.226211+0000","last_became_active":"2024-04-14T08:32:54.199748+0000","last_became_peered":"2024-04-14T08:32:54.199748+0000","last_unstale":"2024-04-14T08:32:59.226211+0000","last_undegraded":"2024-04-14T08:32:59.226211+0000","last_fullsized":"2024-04-14T08:32:59.226211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:49.939225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278584+0000","last_change":"2024-04-14T08:32:53.196334+0000","last_active":"2024-04-14T08:33:01.278584+0000","last_peered":"2024-04-14T08:33:01.278584+0000","last_clean":"2024-04-14T08:33:01.278584+0000","last_became_active":"2024-04-14T08:32:53.196036+0000","last_became_peered":"2024-04-14T08:32:53.196036+0000","last_unstale":"2024-04-14T08:33:01.278584+0000","last_undegraded":"2024-04-14T08:33:01.278584+0000","last_fullsized":"2024-04-14T08:33:01.278584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404693+0000","last_change":"2024-04-14T08:32:58.239865+0000","last_active":"2024-04-14T08:32:59.404693+0000","last_peered":"2024-04-14T08:32:59.404693+0000","last_clean":"2024-04-14T08:32:59.404693+0000","last_became_active":"2024-04-14T08:32:58.239510+0000","last_became_peered":"2024-04-14T08:32:58.239510+0000","last_unstale":"2024-04-14T08:32:59.404693+0000","last_undegraded":"2024-04-14T08:32:59.404693+0000","last_fullsized":"2024-04-14T08:32:59.404693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:37:10.427564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226121+0000","last_change":"2024-04-14T08:32:54.200777+0000","last_active":"2024-04-14T08:32:59.226121+0000","last_peered":"2024-04-14T08:32:59.226121+0000","last_clean":"2024-04-14T08:32:59.226121+0000","last_became_active":"2024-04-14T08:32:54.200284+0000","last_became_peered":"2024-04-14T08:32:54.200284+0000","last_unstale":"2024-04-14T08:32:59.226121+0000","last_undegraded":"2024-04-14T08:32:59.226121+0000","last_fullsized":"2024-04-14T08:32:59.226121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:53.086323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225655+0000","last_change":"2024-04-14T08:32:53.184799+0000","last_active":"2024-04-14T08:32:59.225655+0000","last_peered":"2024-04-14T08:32:59.225655+0000","last_clean":"2024-04-14T08:32:59.225655+0000","last_became_active":"2024-04-14T08:32:53.184483+0000","last_became_peered":"2024-04-14T08:32:53.184483+0000","last_unstale":"2024-04-14T08:32:59.225655+0000","last_undegraded":"2024-04-14T08:32:59.225655+0000","last_fullsized":"2024-04-14T08:32:59.225655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:03:30.482046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224689+0000","last_change":"2024-04-14T08:32:58.243067+0000","last_active":"2024-04-14T08:32:59.224689+0000","last_peered":"2024-04-14T08:32:59.224689+0000","last_clean":"2024-04-14T08:32:59.224689+0000","last_became_active":"2024-04-14T08:32:58.242726+0000","last_became_peered":"2024-04-14T08:32:58.242726+0000","last_unstale":"2024-04-14T08:32:59.224689+0000","last_undegraded":"2024-04-14T08:32:59.224689+0000","last_fullsized":"2024-04-14T08:32:59.224689+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:09.583268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225539+0000","last_change":"2024-04-14T08:32:54.204697+0000","last_active":"2024-04-14T08:32:59.225539+0000","last_peered":"2024-04-14T08:32:59.225539+0000","last_clean":"2024-04-14T08:32:59.225539+0000","last_became_active":"2024-04-14T08:32:54.204357+0000","last_became_peered":"2024-04-14T08:32:54.204357+0000","last_unstale":"2024-04-14T08:32:59.225539+0000","last_undegraded":"2024-04-14T08:32:59.225539+0000","last_fullsized":"2024-04-14T08:32:59.225539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:01.152061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278559+0000","last_change":"2024-04-14T08:32:53.187972+0000","last_active":"2024-04-14T08:33:01.278559+0000","last_peered":"2024-04-14T08:33:01.278559+0000","last_clean":"2024-04-14T08:33:01.278559+0000","last_became_active":"2024-04-14T08:32:53.187667+0000","last_became_peered":"2024-04-14T08:32:53.187667+0000","last_unstale":"2024-04-14T08:33:01.278559+0000","last_undegraded":"2024-04-14T08:33:01.278559+0000","last_fullsized":"2024-04-14T08:33:01.278559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:54:34.240052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220316+0000","last_change":"2024-04-14T08:32:58.246500+0000","last_active":"2024-04-14T08:33:01.220316+0000","last_peered":"2024-04-14T08:33:01.220316+0000","last_clean":"2024-04-14T08:33:01.220316+0000","last_became_active":"2024-04-14T08:32:58.245970+0000","last_became_peered":"2024-04-14T08:32:58.245970+0000","last_unstale":"2024-04-14T08:33:01.220316+0000","last_undegraded":"2024-04-14T08:33:01.220316+0000","last_fullsized":"2024-04-14T08:33:01.220316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:54.763380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115215+0000","last_change":"2024-04-14T08:32:54.201957+0000","last_active":"2024-04-14T08:33:01.115215+0000","last_peered":"2024-04-14T08:33:01.115215+0000","last_clean":"2024-04-14T08:33:01.115215+0000","last_became_active":"2024-04-14T08:32:54.201618+0000","last_became_peered":"2024-04-14T08:32:54.201618+0000","last_unstale":"2024-04-14T08:33:01.115215+0000","last_undegraded":"2024-04-14T08:33:01.115215+0000","last_fullsized":"2024-04-14T08:33:01.115215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115228+0000","last_change":"2024-04-14T08:32:53.195817+0000","last_active":"2024-04-14T08:33:01.115228+0000","last_peered":"2024-04-14T08:33:01.115228+0000","last_clean":"2024-04-14T08:33:01.115228+0000","last_became_active":"2024-04-14T08:32:53.195547+0000","last_became_peered":"2024-04-14T08:32:53.195547+0000","last_unstale":"2024-04-14T08:33:01.115228+0000","last_undegraded":"2024-04-14T08:33:01.115228+0000","last_fullsized":"2024-04-14T08:33:01.115228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225091+0000","last_change":"2024-04-14T08:32:58.240188+0000","last_active":"2024-04-14T08:32:59.225091+0000","last_peered":"2024-04-14T08:32:59.225091+0000","last_clean":"2024-04-14T08:32:59.225091+0000","last_became_active":"2024-04-14T08:32:58.239811+0000","last_became_peered":"2024-04-14T08:32:58.239811+0000","last_unstale":"2024-04-14T08:32:59.225091+0000","last_undegraded":"2024-04-14T08:32:59.225091+0000","last_fullsized":"2024-04-14T08:32:59.225091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:21.164569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225955+0000","last_change":"2024-04-14T08:32:54.200441+0000","last_active":"2024-04-14T08:32:59.225955+0000","last_peered":"2024-04-14T08:32:59.225955+0000","last_clean":"2024-04-14T08:32:59.225955+0000","last_became_active":"2024-04-14T08:32:54.199824+0000","last_became_peered":"2024-04-14T08:32:54.199824+0000","last_unstale":"2024-04-14T08:32:59.225955+0000","last_undegraded":"2024-04-14T08:32:59.225955+0000","last_fullsized":"2024-04-14T08:32:59.225955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:53.661028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404960+0000","last_change":"2024-04-14T08:32:53.202370+0000","last_active":"2024-04-14T08:32:59.404960+0000","last_peered":"2024-04-14T08:32:59.404960+0000","last_clean":"2024-04-14T08:32:59.404960+0000","last_became_active":"2024-04-14T08:32:53.202083+0000","last_became_peered":"2024-04-14T08:32:53.202083+0000","last_unstale":"2024-04-14T08:32:59.404960+0000","last_undegraded":"2024-04-14T08:32:59.404960+0000","last_fullsized":"2024-04-14T08:32:59.404960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224830+0000","last_change":"2024-04-14T08:32:58.247699+0000","last_active":"2024-04-14T08:32:59.224830+0000","last_peered":"2024-04-14T08:32:59.224830+0000","last_clean":"2024-04-14T08:32:59.224830+0000","last_became_active":"2024-04-14T08:32:58.247336+0000","last_became_peered":"2024-04-14T08:32:58.247336+0000","last_unstale":"2024-04-14T08:32:59.224830+0000","last_undegraded":"2024-04-14T08:32:59.224830+0000","last_fullsized":"2024-04-14T08:32:59.224830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:07:01.006337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220030+0000","last_change":"2024-04-14T08:32:54.198561+0000","last_active":"2024-04-14T08:33:01.220030+0000","last_peered":"2024-04-14T08:33:01.220030+0000","last_clean":"2024-04-14T08:33:01.220030+0000","last_became_active":"2024-04-14T08:32:54.198319+0000","last_became_peered":"2024-04-14T08:32:54.198319+0000","last_unstale":"2024-04-14T08:33:01.220030+0000","last_undegraded":"2024-04-14T08:33:01.220030+0000","last_fullsized":"2024-04-14T08:33:01.220030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.219614+0000","last_change":"2024-04-14T08:32:53.188093+0000","last_active":"2024-04-14T08:33:01.219614+0000","last_peered":"2024-04-14T08:33:01.219614+0000","last_clean":"2024-04-14T08:33:01.219614+0000","last_became_active":"2024-04-14T08:32:53.187807+0000","last_became_peered":"2024-04-14T08:32:53.187807+0000","last_unstale":"2024-04-14T08:33:01.219614+0000","last_undegraded":"2024-04-14T08:33:01.219614+0000","last_fullsized":"2024-04-14T08:33:01.219614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:22:15.886932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405178+0000","last_change":"2024-04-14T08:32:58.234962+0000","last_active":"2024-04-14T08:32:59.405178+0000","last_peered":"2024-04-14T08:32:59.405178+0000","last_clean":"2024-04-14T08:32:59.405178+0000","last_became_active":"2024-04-14T08:32:58.234475+0000","last_became_peered":"2024-04-14T08:32:58.234475+0000","last_unstale":"2024-04-14T08:32:59.405178+0000","last_undegraded":"2024-04-14T08:32:59.405178+0000","last_fullsized":"2024-04-14T08:32:59.405178+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:07:01.006337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225877+0000","last_change":"2024-04-14T08:32:54.199377+0000","last_active":"2024-04-14T08:32:59.225877+0000","last_peered":"2024-04-14T08:32:59.225877+0000","last_clean":"2024-04-14T08:32:59.225877+0000","last_became_active":"2024-04-14T08:32:54.199105+0000","last_became_peered":"2024-04-14T08:32:54.199105+0000","last_unstale":"2024-04-14T08:32:59.225877+0000","last_undegraded":"2024-04-14T08:32:59.225877+0000","last_fullsized":"2024-04-14T08:32:59.225877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:43.600124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:03.602017+0000","last_change":"2024-04-14T08:32:53.199739+0000","last_active":"2024-04-14T08:33:03.602017+0000","last_peered":"2024-04-14T08:33:03.602017+0000","last_clean":"2024-04-14T08:33:03.602017+0000","last_became_active":"2024-04-14T08:32:53.199421+0000","last_became_peered":"2024-04-14T08:32:53.199421+0000","last_unstale":"2024-04-14T08:33:03.602017+0000","last_undegraded":"2024-04-14T08:33:03.602017+0000","last_fullsized":"2024-04-14T08:33:03.602017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:14.527274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610241+0000","last_change":"2024-04-14T08:32:58.243680+0000","last_active":"2024-04-14T08:33:00.610241+0000","last_peered":"2024-04-14T08:33:00.610241+0000","last_clean":"2024-04-14T08:33:00.610241+0000","last_became_active":"2024-04-14T08:32:58.243248+0000","last_became_peered":"2024-04-14T08:32:58.243248+0000","last_unstale":"2024-04-14T08:33:00.610241+0000","last_undegraded":"2024-04-14T08:33:00.610241+0000","last_fullsized":"2024-04-14T08:33:00.610241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:57.127403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225954+0000","last_change":"2024-04-14T08:32:54.199362+0000","last_active":"2024-04-14T08:32:59.225954+0000","last_peered":"2024-04-14T08:32:59.225954+0000","last_clean":"2024-04-14T08:32:59.225954+0000","last_became_active":"2024-04-14T08:32:54.199039+0000","last_became_peered":"2024-04-14T08:32:54.199039+0000","last_unstale":"2024-04-14T08:32:59.225954+0000","last_undegraded":"2024-04-14T08:32:59.225954+0000","last_fullsized":"2024-04-14T08:32:59.225954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:33.086425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610374+0000","last_change":"2024-04-14T08:32:53.198302+0000","last_active":"2024-04-14T08:33:00.610374+0000","last_peered":"2024-04-14T08:33:00.610374+0000","last_clean":"2024-04-14T08:33:00.610374+0000","last_became_active":"2024-04-14T08:32:53.198042+0000","last_became_peered":"2024-04-14T08:32:53.198042+0000","last_unstale":"2024-04-14T08:33:00.610374+0000","last_undegraded":"2024-04-14T08:33:00.610374+0000","last_fullsized":"2024-04-14T08:33:00.610374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:32.079335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225101+0000","last_change":"2024-04-14T08:32:58.249159+0000","last_active":"2024-04-14T08:32:59.225101+0000","last_peered":"2024-04-14T08:32:59.225101+0000","last_clean":"2024-04-14T08:32:59.225101+0000","last_became_active":"2024-04-14T08:32:58.248502+0000","last_became_peered":"2024-04-14T08:32:58.248502+0000","last_unstale":"2024-04-14T08:32:59.225101+0000","last_undegraded":"2024-04-14T08:32:59.225101+0000","last_fullsized":"2024-04-14T08:32:59.225101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:57.702107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220422+0000","last_change":"2024-04-14T08:32:54.211683+0000","last_active":"2024-04-14T08:33:01.220422+0000","last_peered":"2024-04-14T08:33:01.220422+0000","last_clean":"2024-04-14T08:33:01.220422+0000","last_became_active":"2024-04-14T08:32:54.211031+0000","last_became_peered":"2024-04-14T08:32:54.211031+0000","last_unstale":"2024-04-14T08:33:01.220422+0000","last_undegraded":"2024-04-14T08:33:01.220422+0000","last_fullsized":"2024-04-14T08:33:01.220422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:24:43.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405170+0000","last_change":"2024-04-14T08:32:53.198725+0000","last_active":"2024-04-14T08:32:59.405170+0000","last_peered":"2024-04-14T08:32:59.405170+0000","last_clean":"2024-04-14T08:32:59.405170+0000","last_became_active":"2024-04-14T08:32:53.198331+0000","last_became_peered":"2024-04-14T08:32:53.198331+0000","last_unstale":"2024-04-14T08:32:59.405170+0000","last_undegraded":"2024-04-14T08:32:59.405170+0000","last_fullsized":"2024-04-14T08:32:59.405170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224411+0000","last_change":"2024-04-14T08:32:58.241328+0000","last_active":"2024-04-14T08:32:59.224411+0000","last_peered":"2024-04-14T08:32:59.224411+0000","last_clean":"2024-04-14T08:32:59.224411+0000","last_became_active":"2024-04-14T08:32:58.240742+0000","last_became_peered":"2024-04-14T08:32:58.240742+0000","last_unstale":"2024-04-14T08:32:59.224411+0000","last_undegraded":"2024-04-14T08:32:59.224411+0000","last_fullsized":"2024-04-14T08:32:59.224411+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:10:58.801121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405105+0000","last_change":"2024-04-14T08:32:54.211134+0000","last_active":"2024-04-14T08:32:59.405105+0000","last_peered":"2024-04-14T08:32:59.405105+0000","last_clean":"2024-04-14T08:32:59.405105+0000","last_became_active":"2024-04-14T08:32:54.210882+0000","last_became_peered":"2024-04-14T08:32:54.210882+0000","last_unstale":"2024-04-14T08:32:59.405105+0000","last_undegraded":"2024-04-14T08:32:59.405105+0000","last_fullsized":"2024-04-14T08:32:59.405105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115518+0000","last_change":"2024-04-14T08:32:53.198595+0000","last_active":"2024-04-14T08:33:01.115518+0000","last_peered":"2024-04-14T08:33:01.115518+0000","last_clean":"2024-04-14T08:33:01.115518+0000","last_became_active":"2024-04-14T08:32:53.198284+0000","last_became_peered":"2024-04-14T08:32:53.198284+0000","last_unstale":"2024-04-14T08:33:01.115518+0000","last_undegraded":"2024-04-14T08:33:01.115518+0000","last_fullsized":"2024-04-14T08:33:01.115518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:48.807150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224443+0000","last_change":"2024-04-14T08:32:58.240139+0000","last_active":"2024-04-14T08:32:59.224443+0000","last_peered":"2024-04-14T08:32:59.224443+0000","last_clean":"2024-04-14T08:32:59.224443+0000","last_became_active":"2024-04-14T08:32:58.239751+0000","last_became_peered":"2024-04-14T08:32:58.239751+0000","last_unstale":"2024-04-14T08:32:59.224443+0000","last_undegraded":"2024-04-14T08:32:59.224443+0000","last_fullsized":"2024-04-14T08:32:59.224443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:37:10.427564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220280+0000","last_change":"2024-04-14T08:32:58.252205+0000","last_active":"2024-04-14T08:33:01.220280+0000","last_peered":"2024-04-14T08:33:01.220280+0000","last_clean":"2024-04-14T08:33:01.220280+0000","last_became_active":"2024-04-14T08:32:58.251856+0000","last_became_peered":"2024-04-14T08:32:58.251856+0000","last_unstale":"2024-04-14T08:33:01.220280+0000","last_undegraded":"2024-04-14T08:33:01.220280+0000","last_fullsized":"2024-04-14T08:33:01.220280+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:16.724895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.609964+0000","last_change":"2024-04-14T08:32:54.200507+0000","last_active":"2024-04-14T08:33:00.609964+0000","last_peered":"2024-04-14T08:33:00.609964+0000","last_clean":"2024-04-14T08:33:00.609964+0000","last_became_active":"2024-04-14T08:32:54.200256+0000","last_became_peered":"2024-04-14T08:32:54.200256+0000","last_unstale":"2024-04-14T08:33:00.609964+0000","last_undegraded":"2024-04-14T08:33:00.609964+0000","last_fullsized":"2024-04-14T08:33:00.609964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404763+0000","last_change":"2024-04-14T08:32:53.197120+0000","last_active":"2024-04-14T08:32:59.404763+0000","last_peered":"2024-04-14T08:32:59.404763+0000","last_clean":"2024-04-14T08:32:59.404763+0000","last_became_active":"2024-04-14T08:32:53.196835+0000","last_became_peered":"2024-04-14T08:32:53.196835+0000","last_unstale":"2024-04-14T08:32:59.404763+0000","last_undegraded":"2024-04-14T08:32:59.404763+0000","last_fullsized":"2024-04-14T08:32:59.404763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:59.189839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115520+0000","last_change":"2024-04-14T08:32:58.240951+0000","last_active":"2024-04-14T08:33:01.115520+0000","last_peered":"2024-04-14T08:33:01.115520+0000","last_clean":"2024-04-14T08:33:01.115520+0000","last_became_active":"2024-04-14T08:32:58.240542+0000","last_became_peered":"2024-04-14T08:32:58.240542+0000","last_unstale":"2024-04-14T08:33:01.115520+0000","last_undegraded":"2024-04-14T08:33:01.115520+0000","last_fullsized":"2024-04-14T08:33:01.115520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:57.127403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224403+0000","last_change":"2024-04-14T08:32:54.199647+0000","last_active":"2024-04-14T08:32:59.224403+0000","last_peered":"2024-04-14T08:32:59.224403+0000","last_clean":"2024-04-14T08:32:59.224403+0000","last_became_active":"2024-04-14T08:32:54.199365+0000","last_became_peered":"2024-04-14T08:32:54.199365+0000","last_unstale":"2024-04-14T08:32:59.224403+0000","last_undegraded":"2024-04-14T08:32:59.224403+0000","last_fullsized":"2024-04-14T08:32:59.224403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:17:06.203553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224536+0000","last_change":"2024-04-14T08:32:53.199750+0000","last_active":"2024-04-14T08:32:59.224536+0000","last_peered":"2024-04-14T08:32:59.224536+0000","last_clean":"2024-04-14T08:32:59.224536+0000","last_became_active":"2024-04-14T08:32:53.199477+0000","last_became_peered":"2024-04-14T08:32:53.199477+0000","last_unstale":"2024-04-14T08:32:59.224536+0000","last_undegraded":"2024-04-14T08:32:59.224536+0000","last_fullsized":"2024-04-14T08:32:59.224536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:03:30.482046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115345+0000","last_change":"2024-04-14T08:32:53.195193+0000","last_active":"2024-04-14T08:33:01.115345+0000","last_peered":"2024-04-14T08:33:01.115345+0000","last_clean":"2024-04-14T08:33:01.115345+0000","last_became_active":"2024-04-14T08:32:53.194887+0000","last_became_peered":"2024-04-14T08:32:53.194887+0000","last_unstale":"2024-04-14T08:33:01.115345+0000","last_undegraded":"2024-04-14T08:33:01.115345+0000","last_fullsized":"2024-04-14T08:33:01.115345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:07.828010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220235+0000","last_change":"2024-04-14T08:32:54.203263+0000","last_active":"2024-04-14T08:33:01.220235+0000","last_peered":"2024-04-14T08:33:01.220235+0000","last_clean":"2024-04-14T08:33:01.220235+0000","last_became_active":"2024-04-14T08:32:54.202196+0000","last_became_peered":"2024-04-14T08:32:54.202196+0000","last_unstale":"2024-04-14T08:33:01.220235+0000","last_undegraded":"2024-04-14T08:33:01.220235+0000","last_fullsized":"2024-04-14T08:33:01.220235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278810+0000","last_change":"2024-04-14T08:32:58.239935+0000","last_active":"2024-04-14T08:33:01.278810+0000","last_peered":"2024-04-14T08:33:01.278810+0000","last_clean":"2024-04-14T08:33:01.278810+0000","last_became_active":"2024-04-14T08:32:58.239542+0000","last_became_peered":"2024-04-14T08:32:58.239542+0000","last_unstale":"2024-04-14T08:33:01.278810+0000","last_undegraded":"2024-04-14T08:33:01.278810+0000","last_fullsized":"2024-04-14T08:33:01.278810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:04.288282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224729+0000","last_change":"2024-04-14T08:32:53.201101+0000","last_active":"2024-04-14T08:32:59.224729+0000","last_peered":"2024-04-14T08:32:59.224729+0000","last_clean":"2024-04-14T08:32:59.224729+0000","last_became_active":"2024-04-14T08:32:53.200781+0000","last_became_peered":"2024-04-14T08:32:53.200781+0000","last_unstale":"2024-04-14T08:32:59.224729+0000","last_undegraded":"2024-04-14T08:32:59.224729+0000","last_fullsized":"2024-04-14T08:32:59.224729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:52.163826+0000","last_clean_scrub_stamp":"2024-04-14T08:32:52.163826+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:08:09.715762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224989+0000","last_change":"2024-04-14T08:32:54.194042+0000","last_active":"2024-04-14T08:32:59.224989+0000","last_peered":"2024-04-14T08:32:59.224989+0000","last_clean":"2024-04-14T08:32:59.224989+0000","last_became_active":"2024-04-14T08:32:54.193764+0000","last_became_peered":"2024-04-14T08:32:54.193764+0000","last_unstale":"2024-04-14T08:32:59.224989+0000","last_undegraded":"2024-04-14T08:32:59.224989+0000","last_fullsized":"2024-04-14T08:32:59.224989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:50:00.196929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225098+0000","last_change":"2024-04-14T08:32:58.241996+0000","last_active":"2024-04-14T08:32:59.225098+0000","last_peered":"2024-04-14T08:32:59.225098+0000","last_clean":"2024-04-14T08:32:59.225098+0000","last_became_active":"2024-04-14T08:32:58.241641+0000","last_became_peered":"2024-04-14T08:32:58.241641+0000","last_unstale":"2024-04-14T08:32:59.225098+0000","last_undegraded":"2024-04-14T08:32:59.225098+0000","last_fullsized":"2024-04-14T08:32:59.225098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:57.211995+0000","last_clean_scrub_stamp":"2024-04-14T08:32:57.211995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:58:22.710120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115102+0000","last_change":"2024-04-14T08:32:54.210270+0000","last_active":"2024-04-14T08:33:01.115102+0000","last_peered":"2024-04-14T08:33:01.115102+0000","last_clean":"2024-04-14T08:33:01.115102+0000","last_became_active":"2024-04-14T08:32:54.209717+0000","last_became_peered":"2024-04-14T08:32:54.209717+0000","last_unstale":"2024-04-14T08:33:01.115102+0000","last_undegraded":"2024-04-14T08:33:01.115102+0000","last_fullsized":"2024-04-14T08:33:01.115102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278775+0000","last_change":"2024-04-14T08:32:54.208720+0000","last_active":"2024-04-14T08:33:01.278775+0000","last_peered":"2024-04-14T08:33:01.278775+0000","last_clean":"2024-04-14T08:33:01.278775+0000","last_became_active":"2024-04-14T08:32:54.208426+0000","last_became_peered":"2024-04-14T08:32:54.208426+0000","last_unstale":"2024-04-14T08:33:01.278775+0000","last_undegraded":"2024-04-14T08:33:01.278775+0000","last_fullsized":"2024-04-14T08:33:01.278775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:49.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404885+0000","last_change":"2024-04-14T08:32:54.209426+0000","last_active":"2024-04-14T08:32:59.404885+0000","last_peered":"2024-04-14T08:32:59.404885+0000","last_clean":"2024-04-14T08:32:59.404885+0000","last_became_active":"2024-04-14T08:32:54.209158+0000","last_became_peered":"2024-04-14T08:32:54.209158+0000","last_unstale":"2024-04-14T08:32:59.404885+0000","last_undegraded":"2024-04-14T08:32:59.404885+0000","last_fullsized":"2024-04-14T08:32:59.404885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:12.683816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220151+0000","last_change":"2024-04-14T08:32:54.206827+0000","last_active":"2024-04-14T08:33:01.220151+0000","last_peered":"2024-04-14T08:33:01.220151+0000","last_clean":"2024-04-14T08:33:01.220151+0000","last_became_active":"2024-04-14T08:32:54.206593+0000","last_became_peered":"2024-04-14T08:32:54.206593+0000","last_unstale":"2024-04-14T08:33:01.220151+0000","last_undegraded":"2024-04-14T08:33:01.220151+0000","last_fullsized":"2024-04-14T08:33:01.220151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224854+0000","last_change":"2024-04-14T08:32:54.200872+0000","last_active":"2024-04-14T08:32:59.224854+0000","last_peered":"2024-04-14T08:32:59.224854+0000","last_clean":"2024-04-14T08:32:59.224854+0000","last_became_active":"2024-04-14T08:32:54.200616+0000","last_became_peered":"2024-04-14T08:32:54.200616+0000","last_unstale":"2024-04-14T08:32:59.224854+0000","last_undegraded":"2024-04-14T08:32:59.224854+0000","last_fullsized":"2024-04-14T08:32:59.224854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610219+0000","last_change":"2024-04-14T08:32:54.199679+0000","last_active":"2024-04-14T08:33:00.610219+0000","last_peered":"2024-04-14T08:33:00.610219+0000","last_clean":"2024-04-14T08:33:00.610219+0000","last_became_active":"2024-04-14T08:32:54.199433+0000","last_became_peered":"2024-04-14T08:32:54.199433+0000","last_unstale":"2024-04-14T08:33:00.610219+0000","last_undegraded":"2024-04-14T08:33:00.610219+0000","last_fullsized":"2024-04-14T08:33:00.610219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115645+0000","last_change":"2024-04-14T08:32:54.207939+0000","last_active":"2024-04-14T08:33:01.115645+0000","last_peered":"2024-04-14T08:33:01.115645+0000","last_clean":"2024-04-14T08:33:01.115645+0000","last_became_active":"2024-04-14T08:32:54.207460+0000","last_became_peered":"2024-04-14T08:32:54.207460+0000","last_unstale":"2024-04-14T08:33:01.115645+0000","last_undegraded":"2024-04-14T08:33:01.115645+0000","last_fullsized":"2024-04-14T08:33:01.115645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225033+0000","last_change":"2024-04-14T08:32:54.202461+0000","last_active":"2024-04-14T08:32:59.225033+0000","last_peered":"2024-04-14T08:32:59.225033+0000","last_clean":"2024-04-14T08:32:59.225033+0000","last_became_active":"2024-04-14T08:32:54.202156+0000","last_became_peered":"2024-04-14T08:32:54.202156+0000","last_unstale":"2024-04-14T08:32:59.225033+0000","last_undegraded":"2024-04-14T08:32:59.225033+0000","last_fullsized":"2024-04-14T08:32:59.225033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:45.405633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404730+0000","last_change":"2024-04-14T08:32:54.204603+0000","last_active":"2024-04-14T08:32:59.404730+0000","last_peered":"2024-04-14T08:32:59.404730+0000","last_clean":"2024-04-14T08:32:59.404730+0000","last_became_active":"2024-04-14T08:32:54.204323+0000","last_became_peered":"2024-04-14T08:32:54.204323+0000","last_unstale":"2024-04-14T08:32:59.404730+0000","last_undegraded":"2024-04-14T08:32:59.404730+0000","last_fullsized":"2024-04-14T08:32:59.404730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224464+0000","last_change":"2024-04-14T08:32:54.205983+0000","last_active":"2024-04-14T08:32:59.224464+0000","last_peered":"2024-04-14T08:32:59.224464+0000","last_clean":"2024-04-14T08:32:59.224464+0000","last_became_active":"2024-04-14T08:32:54.205707+0000","last_became_peered":"2024-04-14T08:32:54.205707+0000","last_unstale":"2024-04-14T08:32:59.224464+0000","last_undegraded":"2024-04-14T08:32:59.224464+0000","last_fullsized":"2024-04-14T08:32:59.224464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:05:03.511225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404857+0000","last_change":"2024-04-14T08:32:54.211979+0000","last_active":"2024-04-14T08:32:59.404857+0000","last_peered":"2024-04-14T08:32:59.404857+0000","last_clean":"2024-04-14T08:32:59.404857+0000","last_became_active":"2024-04-14T08:32:54.211481+0000","last_became_peered":"2024-04-14T08:32:54.211481+0000","last_unstale":"2024-04-14T08:32:59.404857+0000","last_undegraded":"2024-04-14T08:32:59.404857+0000","last_fullsized":"2024-04-14T08:32:59.404857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:53.086323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220046+0000","last_change":"2024-04-14T08:32:54.204727+0000","last_active":"2024-04-14T08:33:01.220046+0000","last_peered":"2024-04-14T08:33:01.220046+0000","last_clean":"2024-04-14T08:33:01.220046+0000","last_became_active":"2024-04-14T08:32:54.204472+0000","last_became_peered":"2024-04-14T08:32:54.204472+0000","last_unstale":"2024-04-14T08:33:01.220046+0000","last_undegraded":"2024-04-14T08:33:01.220046+0000","last_fullsized":"2024-04-14T08:33:01.220046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115389+0000","last_change":"2024-04-14T08:32:54.211345+0000","last_active":"2024-04-14T08:33:01.115389+0000","last_peered":"2024-04-14T08:33:01.115389+0000","last_clean":"2024-04-14T08:33:01.115389+0000","last_became_active":"2024-04-14T08:32:54.211084+0000","last_became_peered":"2024-04-14T08:32:54.211084+0000","last_unstale":"2024-04-14T08:33:01.115389+0000","last_undegraded":"2024-04-14T08:33:01.115389+0000","last_fullsized":"2024-04-14T08:33:01.115389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:16:41.675226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226252+0000","last_change":"2024-04-14T08:32:54.198283+0000","last_active":"2024-04-14T08:32:59.226252+0000","last_peered":"2024-04-14T08:32:59.226252+0000","last_clean":"2024-04-14T08:32:59.226252+0000","last_became_active":"2024-04-14T08:32:54.197647+0000","last_became_peered":"2024-04-14T08:32:54.197647+0000","last_unstale":"2024-04-14T08:32:59.226252+0000","last_undegraded":"2024-04-14T08:32:59.226252+0000","last_fullsized":"2024-04-14T08:32:59.226252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:48.960689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.278917+0000","last_change":"2024-04-14T08:32:54.204246+0000","last_active":"2024-04-14T08:33:01.278917+0000","last_peered":"2024-04-14T08:33:01.278917+0000","last_clean":"2024-04-14T08:33:01.278917+0000","last_became_active":"2024-04-14T08:32:54.203956+0000","last_became_peered":"2024-04-14T08:32:54.203956+0000","last_unstale":"2024-04-14T08:33:01.278917+0000","last_undegraded":"2024-04-14T08:33:01.278917+0000","last_fullsized":"2024-04-14T08:33:01.278917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:46:33.086425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.405290+0000","last_change":"2024-04-14T08:32:54.207841+0000","last_active":"2024-04-14T08:32:59.405290+0000","last_peered":"2024-04-14T08:32:59.405290+0000","last_clean":"2024-04-14T08:32:59.405290+0000","last_became_active":"2024-04-14T08:32:54.207412+0000","last_became_peered":"2024-04-14T08:32:54.207412+0000","last_unstale":"2024-04-14T08:32:59.405290+0000","last_undegraded":"2024-04-14T08:32:59.405290+0000","last_fullsized":"2024-04-14T08:32:59.405290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:48.043054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404630+0000","last_change":"2024-04-14T08:32:54.207321+0000","last_active":"2024-04-14T08:32:59.404630+0000","last_peered":"2024-04-14T08:32:59.404630+0000","last_clean":"2024-04-14T08:32:59.404630+0000","last_became_active":"2024-04-14T08:32:54.207058+0000","last_became_peered":"2024-04-14T08:32:54.207058+0000","last_unstale":"2024-04-14T08:32:59.404630+0000","last_undegraded":"2024-04-14T08:32:59.404630+0000","last_fullsized":"2024-04-14T08:32:59.404630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610020+0000","last_change":"2024-04-14T08:32:54.205919+0000","last_active":"2024-04-14T08:33:00.610020+0000","last_peered":"2024-04-14T08:33:00.610020+0000","last_clean":"2024-04-14T08:33:00.610020+0000","last_became_active":"2024-04-14T08:32:54.205623+0000","last_became_peered":"2024-04-14T08:32:54.205623+0000","last_unstale":"2024-04-14T08:33:00.610020+0000","last_undegraded":"2024-04-14T08:33:00.610020+0000","last_fullsized":"2024-04-14T08:33:00.610020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:55.426580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220007+0000","last_change":"2024-04-14T08:32:54.196317+0000","last_active":"2024-04-14T08:33:01.220007+0000","last_peered":"2024-04-14T08:33:01.220007+0000","last_clean":"2024-04-14T08:33:01.220007+0000","last_became_active":"2024-04-14T08:32:54.195481+0000","last_became_peered":"2024-04-14T08:32:54.195481+0000","last_unstale":"2024-04-14T08:33:01.220007+0000","last_undegraded":"2024-04-14T08:33:01.220007+0000","last_fullsized":"2024-04-14T08:33:01.220007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:17:06.203553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224636+0000","last_change":"2024-04-14T08:32:54.203801+0000","last_active":"2024-04-14T08:32:59.224636+0000","last_peered":"2024-04-14T08:32:59.224636+0000","last_clean":"2024-04-14T08:32:59.224636+0000","last_became_active":"2024-04-14T08:32:54.203524+0000","last_became_peered":"2024-04-14T08:32:54.203524+0000","last_unstale":"2024-04-14T08:32:59.224636+0000","last_undegraded":"2024-04-14T08:32:59.224636+0000","last_fullsized":"2024-04-14T08:32:59.224636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:50.003193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225873+0000","last_change":"2024-04-14T08:32:54.204997+0000","last_active":"2024-04-14T08:32:59.225873+0000","last_peered":"2024-04-14T08:32:59.225873+0000","last_clean":"2024-04-14T08:32:59.225873+0000","last_became_active":"2024-04-14T08:32:54.204757+0000","last_became_peered":"2024-04-14T08:32:54.204757+0000","last_unstale":"2024-04-14T08:32:59.225873+0000","last_undegraded":"2024-04-14T08:32:59.225873+0000","last_fullsized":"2024-04-14T08:32:59.225873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:49.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115444+0000","last_change":"2024-04-14T08:32:54.210277+0000","last_active":"2024-04-14T08:33:01.115444+0000","last_peered":"2024-04-14T08:33:01.115444+0000","last_clean":"2024-04-14T08:33:01.115444+0000","last_became_active":"2024-04-14T08:32:54.209548+0000","last_became_peered":"2024-04-14T08:32:54.209548+0000","last_unstale":"2024-04-14T08:33:01.115444+0000","last_undegraded":"2024-04-14T08:33:01.115444+0000","last_fullsized":"2024-04-14T08:33:01.115444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.115625+0000","last_change":"2024-04-14T08:32:54.213828+0000","last_active":"2024-04-14T08:33:01.115625+0000","last_peered":"2024-04-14T08:33:01.115625+0000","last_clean":"2024-04-14T08:33:01.115625+0000","last_became_active":"2024-04-14T08:32:54.213551+0000","last_became_peered":"2024-04-14T08:32:54.213551+0000","last_unstale":"2024-04-14T08:33:01.115625+0000","last_undegraded":"2024-04-14T08:33:01.115625+0000","last_fullsized":"2024-04-14T08:33:01.115625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226176+0000","last_change":"2024-04-14T08:32:54.212290+0000","last_active":"2024-04-14T08:32:59.226176+0000","last_peered":"2024-04-14T08:32:59.226176+0000","last_clean":"2024-04-14T08:32:59.226176+0000","last_became_active":"2024-04-14T08:32:54.212014+0000","last_became_peered":"2024-04-14T08:32:54.212014+0000","last_unstale":"2024-04-14T08:32:59.226176+0000","last_undegraded":"2024-04-14T08:32:59.226176+0000","last_fullsized":"2024-04-14T08:32:59.226176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:58:50.722301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.224300+0000","last_change":"2024-04-14T08:32:54.209523+0000","last_active":"2024-04-14T08:32:59.224300+0000","last_peered":"2024-04-14T08:32:59.224300+0000","last_clean":"2024-04-14T08:32:59.224300+0000","last_became_active":"2024-04-14T08:32:54.209262+0000","last_became_peered":"2024-04-14T08:32:54.209262+0000","last_unstale":"2024-04-14T08:32:59.224300+0000","last_undegraded":"2024-04-14T08:32:59.224300+0000","last_fullsized":"2024-04-14T08:32:59.224300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:25:28.934928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225401+0000","last_change":"2024-04-14T08:32:54.195208+0000","last_active":"2024-04-14T08:32:59.225401+0000","last_peered":"2024-04-14T08:32:59.225401+0000","last_clean":"2024-04-14T08:32:59.225401+0000","last_became_active":"2024-04-14T08:32:54.194916+0000","last_became_peered":"2024-04-14T08:32:54.194916+0000","last_unstale":"2024-04-14T08:32:59.225401+0000","last_undegraded":"2024-04-14T08:32:59.225401+0000","last_fullsized":"2024-04-14T08:32:59.225401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:16.536477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404780+0000","last_change":"2024-04-14T08:32:54.208737+0000","last_active":"2024-04-14T08:32:59.404780+0000","last_peered":"2024-04-14T08:32:59.404780+0000","last_clean":"2024-04-14T08:32:59.404780+0000","last_became_active":"2024-04-14T08:32:54.208439+0000","last_became_peered":"2024-04-14T08:32:54.208439+0000","last_unstale":"2024-04-14T08:32:59.404780+0000","last_undegraded":"2024-04-14T08:32:59.404780+0000","last_fullsized":"2024-04-14T08:32:59.404780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:48.960689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.225788+0000","last_change":"2024-04-14T08:32:54.198295+0000","last_active":"2024-04-14T08:32:59.225788+0000","last_peered":"2024-04-14T08:32:59.225788+0000","last_clean":"2024-04-14T08:32:59.225788+0000","last_became_active":"2024-04-14T08:32:54.198052+0000","last_became_peered":"2024-04-14T08:32:54.198052+0000","last_unstale":"2024-04-14T08:32:59.225788+0000","last_undegraded":"2024-04-14T08:32:59.225788+0000","last_fullsized":"2024-04-14T08:32:59.225788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:48.043054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.404955+0000","last_change":"2024-04-14T08:32:54.206537+0000","last_active":"2024-04-14T08:32:59.404955+0000","last_peered":"2024-04-14T08:32:59.404955+0000","last_clean":"2024-04-14T08:32:59.404955+0000","last_became_active":"2024-04-14T08:32:54.206246+0000","last_became_peered":"2024-04-14T08:32:54.206246+0000","last_unstale":"2024-04-14T08:32:59.404955+0000","last_undegraded":"2024-04-14T08:32:59.404955+0000","last_fullsized":"2024-04-14T08:32:59.404955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:26:16.536477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:01.220192+0000","last_change":"2024-04-14T08:32:54.201364+0000","last_active":"2024-04-14T08:33:01.220192+0000","last_peered":"2024-04-14T08:33:01.220192+0000","last_clean":"2024-04-14T08:33:01.220192+0000","last_became_active":"2024-04-14T08:32:54.199521+0000","last_became_peered":"2024-04-14T08:32:54.199521+0000","last_unstale":"2024-04-14T08:33:01.220192+0000","last_undegraded":"2024-04-14T08:33:01.220192+0000","last_fullsized":"2024-04-14T08:33:01.220192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:05.558440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:33:00.610554+0000","last_change":"2024-04-14T08:32:54.199458+0000","last_active":"2024-04-14T08:33:00.610554+0000","last_peered":"2024-04-14T08:33:00.610554+0000","last_clean":"2024-04-14T08:33:00.610554+0000","last_became_active":"2024-04-14T08:32:54.199203+0000","last_became_peered":"2024-04-14T08:32:54.199203+0000","last_unstale":"2024-04-14T08:33:00.610554+0000","last_undegraded":"2024-04-14T08:33:00.610554+0000","last_fullsized":"2024-04-14T08:33:00.610554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:12:05.542189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-14T08:32:59.226040+0000","last_change":"2024-04-14T08:32:54.206343+0000","last_active":"2024-04-14T08:32:59.226040+0000","last_peered":"2024-04-14T08:32:59.226040+0000","last_clean":"2024-04-14T08:32:59.226040+0000","last_became_active":"2024-04-14T08:32:54.205925+0000","last_became_peered":"2024-04-14T08:32:54.205925+0000","last_unstale":"2024-04-14T08:32:59.226040+0000","last_undegraded":"2024-04-14T08:32:59.226040+0000","last_fullsized":"2024-04-14T08:32:59.226040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:53.170916+0000","last_clean_scrub_stamp":"2024-04-14T08:32:53.170916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:49:12.683816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:33:14.282 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-14T08:33:14.282 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-14T08:33:14.282 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-14T08:33:14.282 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-14T08:33:14.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.417+0000 7fd4325e0640 1 Processor -- start 2024-04-14T08:33:14.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.417+0000 7fd4325e0640 1 -- start start 2024-04-14T08:33:14.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd4325e0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd42c15e5b0 0x7fd42c0ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd4325e0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd42c06c530 0x7fd42c06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd4325e0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd42c0b41d0 con 0x7fd42c0729b0 2024-04-14T08:33:14.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd4325e0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd42c06cf00 con 0x7fd42c15e5b0 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd4325e0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd42c0aebe0 con 0x7fd42c06c530 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd42bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd42c15e5b0 0x7fd42c0ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd42b7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd42c0729b0 0x7fd42c06dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35734/0 (socket says 172.21.15.172:35734) 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd42bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd42c15e5b0 0x7fd42c0ae390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49674/0 (socket says 172.21.15.172:49674) 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd42b7fe640 1 -- 172.21.15.172:0/353590502 learned_addr learned my addr 172.21.15.172:0/353590502 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd430b7d640 1 --2- 172.21.15.172:0/353590502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd42c06c530 0x7fd42c06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd42bfff640 1 -- 172.21.15.172:0/353590502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd42c06c530 msgr2=0x7fd42c06c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd42affd640 1 -- 172.21.15.172:0/353590502 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1417918545 0 0) 0x7fd42c0b41d0 con 0x7fd42c0729b0 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd42bfff640 1 --2- 172.21.15.172:0/353590502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd42c06c530 0x7fd42c06c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd42bfff640 1 -- 172.21.15.172:0/353590502 >> v1:172.21.15.172:6789/0 conn(0x7fd42c0729b0 legacy=0x7fd42c06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd42bfff640 1 -- 172.21.15.172:0/353590502 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42c0b4470 con 0x7fd42c15e5b0 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.418+0000 7fd430b7d640 1 --2- 172.21.15.172:0/353590502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd42c06c530 0x7fd42c06c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:14.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.419+0000 7fd42bfff640 1 --2- 172.21.15.172:0/353590502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd42c15e5b0 0x7fd42c0ae390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0097d0 tx=0x7fd41c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9672ddc644ee933 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:14.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.419+0000 7fd42affd640 1 -- 172.21.15.172:0/353590502 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c035020 con 0x7fd42c15e5b0 2024-04-14T08:33:14.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.419+0000 7fd42affd640 1 -- 172.21.15.172:0/353590502 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41c031800 con 0x7fd42c15e5b0 2024-04-14T08:33:14.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.419+0000 7fd42affd640 1 -- 172.21.15.172:0/353590502 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c031bd0 con 0x7fd42c15e5b0 2024-04-14T08:33:14.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.420+0000 7fd4325e0640 1 -- 172.21.15.172:0/353590502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd42c15e5b0 msgr2=0x7fd42c0ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.420+0000 7fd4325e0640 1 --2- 172.21.15.172:0/353590502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd42c15e5b0 0x7fd42c0ae390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0097d0 tx=0x7fd41c02b8a0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.420+0000 7fd4325e0640 1 -- 172.21.15.172:0/353590502 shutdown_connections 2024-04-14T08:33:14.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.420+0000 7fd4325e0640 1 --2- 172.21.15.172:0/353590502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd42c06c530 0x7fd42c06c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.420+0000 7fd4325e0640 1 --2- 172.21.15.172:0/353590502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd42c15e5b0 0x7fd42c0ae390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.420+0000 7fd4325e0640 1 -- 172.21.15.172:0/353590502 >> 172.21.15.172:0/353590502 conn(0x7fd42c071ba0 msgr2=0x7fd42c071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:14.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.420+0000 7fd4325e0640 1 -- 172.21.15.172:0/353590502 shutdown_connections 2024-04-14T08:33:14.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.420+0000 7fd4325e0640 1 -- 172.21.15.172:0/353590502 wait complete. 2024-04-14T08:33:14.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.421+0000 7fd4325e0640 1 Processor -- start 2024-04-14T08:33:14.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.421+0000 7fd4325e0640 1 -- start start 2024-04-14T08:33:14.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd4325e0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd42c06c530 0x7fd42c142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd4325e0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd42c0729b0 0x7fd42c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd4325e0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd42c15e5b0 0x7fd42c15eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd4325e0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c0aeec0 con 0x7fd42c0729b0 2024-04-14T08:33:14.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd4325e0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c06e330 con 0x7fd42c15e5b0 2024-04-14T08:33:14.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd4325e0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd42c0af040 con 0x7fd42c06c530 2024-04-14T08:33:14.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd42b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd42c0729b0 0x7fd42c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd42b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd42c0729b0 0x7fd42c15e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49678/0 (socket says 172.21.15.172:49678) 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd42b7fe640 1 -- 172.21.15.172:0/930307568 learned_addr learned my addr 172.21.15.172:0/930307568 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd42bfff640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd42c06c530 0x7fd42c142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd430b7d640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd42c15e5b0 0x7fd42c15eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd42b7fe640 1 -- 172.21.15.172:0/930307568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd42c06c530 msgr2=0x7fd42c142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd42b7fe640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd42c06c530 0x7fd42c142db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd42b7fe640 1 -- 172.21.15.172:0/930307568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd42c15e5b0 msgr2=0x7fd42c15eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd42b7fe640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd42c15e5b0 0x7fd42c15eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd42b7fe640 1 -- 172.21.15.172:0/930307568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42c158040 con 0x7fd42c0729b0 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd42b7fe640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd42c0729b0 0x7fd42c15e990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd42000c4d0 tx=0x7fd42000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd428ff9640 1 -- 172.21.15.172:0/930307568 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4200074e0 con 0x7fd42c0729b0 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd428ff9640 1 -- 172.21.15.172:0/930307568 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4200047e0 con 0x7fd42c0729b0 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd4325e0640 1 -- 172.21.15.172:0/930307568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42c158320 con 0x7fd42c0729b0 2024-04-14T08:33:14.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.422+0000 7fd428ff9640 1 -- 172.21.15.172:0/930307568 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420004bd0 con 0x7fd42c0729b0 2024-04-14T08:33:14.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.423+0000 7fd4325e0640 1 -- 172.21.15.172:0/930307568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42c1588e0 con 0x7fd42c0729b0 2024-04-14T08:33:14.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.423+0000 7fd4325e0640 1 -- 172.21.15.172:0/930307568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd42c068520 con 0x7fd42c0729b0 2024-04-14T08:33:14.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.424+0000 7fd428ff9640 1 -- 172.21.15.172:0/930307568 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd420011070 con 0x7fd42c0729b0 2024-04-14T08:33:14.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.424+0000 7fd428ff9640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd41806c8f0 0x7fd41806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.424+0000 7fd428ff9640 1 -- 172.21.15.172:0/930307568 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd42008bab0 con 0x7fd42c0729b0 2024-04-14T08:33:14.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.424+0000 7fd42bfff640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd41806c8f0 0x7fd41806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.425+0000 7fd42bfff640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd41806c8f0 0x7fd41806edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd41c036000 tx=0x7fd41c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:14.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.429+0000 7fd428ff9640 1 -- 172.21.15.172:0/930307568 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd420059b10 con 0x7fd42c0729b0 2024-04-14T08:33:14.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.640+0000 7fd4325e0640 1 -- 172.21.15.172:0/930307568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd42c072790 con 0x7fd42c0729b0 2024-04-14T08:33:14.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.640+0000 7fd428ff9640 1 -- 172.21.15.172:0/930307568 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd420059410 con 0x7fd42c0729b0 2024-04-14T08:33:14.641 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:14.641 INFO:teuthology.orchestra.run.smithi172.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-14T08:33:14.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.642+0000 7fd4325e0640 1 -- 172.21.15.172:0/930307568 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd41806c8f0 msgr2=0x7fd41806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.642+0000 7fd4325e0640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd41806c8f0 0x7fd41806edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd41c036000 tx=0x7fd41c008030 comp rx=0 tx=0).stop 2024-04-14T08:33:14.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.642+0000 7fd4325e0640 1 -- 172.21.15.172:0/930307568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd42c0729b0 msgr2=0x7fd42c15e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.642+0000 7fd4325e0640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd42c0729b0 0x7fd42c15e990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd42000c4d0 tx=0x7fd42000c8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.642+0000 7fd4325e0640 1 -- 172.21.15.172:0/930307568 shutdown_connections 2024-04-14T08:33:14.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.642+0000 7fd4325e0640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd41806c8f0 0x7fd41806edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.642+0000 7fd4325e0640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd42c15e5b0 0x7fd42c15eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.642+0000 7fd4325e0640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd42c0729b0 0x7fd42c15e990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.642+0000 7fd4325e0640 1 --2- 172.21.15.172:0/930307568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd42c06c530 0x7fd42c142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.642+0000 7fd4325e0640 1 -- 172.21.15.172:0/930307568 >> 172.21.15.172:0/930307568 conn(0x7fd42c071ba0 msgr2=0x7fd42c162840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:14.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.642+0000 7fd4325e0640 1 -- 172.21.15.172:0/930307568 shutdown_connections 2024-04-14T08:33:14.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.642+0000 7fd4325e0640 1 -- 172.21.15.172:0/930307568 wait complete. 2024-04-14T08:33:14.654 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-14T08:33:14.654 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:14.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.755+0000 7f56cbc76640 1 Processor -- start 2024-04-14T08:33:14.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.755+0000 7f56cbc76640 1 -- start start 2024-04-14T08:33:14.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56cbc76640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56c406b520 0x7f56c406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56cbc76640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56c406be40 con 0x7f56c40b5600 2024-04-14T08:33:14.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56cbc76640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56c415cb60 con 0x7f56c40b5b50 2024-04-14T08:33:14.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56cbc76640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f56c40ae650 con 0x7f56c406b520 2024-04-14T08:33:14.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56ca213640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f56c40b5600 0x7f56c40b4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35738/0 (socket says 172.21.15.172:35738) 2024-04-14T08:33:14.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56ca213640 1 -- 172.21.15.172:0/3330530139 learned_addr learned my addr 172.21.15.172:0/3330530139 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:14.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56c9a12640 1 --2- 172.21.15.172:0/3330530139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56c406b520 0x7f56c406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56c8a10640 1 -- 172.21.15.172:0/3330530139 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4186612202 0 0) 0x7f56c406be40 con 0x7f56c40b5600 2024-04-14T08:33:14.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56c8a10640 1 -- 172.21.15.172:0/3330530139 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56b0003520 con 0x7f56c40b5600 2024-04-14T08:33:14.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56c8a10640 1 -- 172.21.15.172:0/3330530139 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1580449597 0 0) 0x7f56c415cb60 con 0x7f56c40b5b50 2024-04-14T08:33:14.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56c8a10640 1 -- 172.21.15.172:0/3330530139 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56c406be40 con 0x7f56c40b5b50 2024-04-14T08:33:14.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56c9a12640 1 -- 172.21.15.172:0/3330530139 >> v1:172.21.15.177:6789/0 conn(0x7f56c40b5b50 legacy=0x7f56c415c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56c9a12640 1 -- 172.21.15.172:0/3330530139 >> v1:172.21.15.172:6789/0 conn(0x7f56c40b5600 legacy=0x7f56c40b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.756+0000 7f56c9a12640 1 -- 172.21.15.172:0/3330530139 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c40addb0 con 0x7f56c406b520 2024-04-14T08:33:14.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.757+0000 7f56c9a12640 1 --2- 172.21.15.172:0/3330530139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56c406b520 0x7f56c406b900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f56c00097d0 tx=0x7f56c002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a13d172b5e866602 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:14.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.757+0000 7f56c8a10640 1 -- 172.21.15.172:0/3330530139 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0035020 con 0x7f56c406b520 2024-04-14T08:33:14.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.757+0000 7f56c8a10640 1 -- 172.21.15.172:0/3330530139 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c0031890 con 0x7f56c406b520 2024-04-14T08:33:14.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.757+0000 7f56c8a10640 1 -- 172.21.15.172:0/3330530139 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0031c40 con 0x7f56c406b520 2024-04-14T08:33:14.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.758+0000 7f56cbc76640 1 -- 172.21.15.172:0/3330530139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56c406b520 msgr2=0x7f56c406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.758+0000 7f56cbc76640 1 --2- 172.21.15.172:0/3330530139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56c406b520 0x7f56c406b900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f56c00097d0 tx=0x7f56c002b930 comp rx=0 tx=0).stop 2024-04-14T08:33:14.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.758+0000 7f56cbc76640 1 -- 172.21.15.172:0/3330530139 shutdown_connections 2024-04-14T08:33:14.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.758+0000 7f56cbc76640 1 --2- 172.21.15.172:0/3330530139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56c406b520 0x7f56c406b900 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.758+0000 7f56cbc76640 1 -- 172.21.15.172:0/3330530139 >> 172.21.15.172:0/3330530139 conn(0x7f56c40d0f20 msgr2=0x7f56c40d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:14.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.758+0000 7f56cbc76640 1 -- 172.21.15.172:0/3330530139 shutdown_connections 2024-04-14T08:33:14.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.758+0000 7f56cbc76640 1 -- 172.21.15.172:0/3330530139 wait complete. 2024-04-14T08:33:14.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.759+0000 7f56cbc76640 1 Processor -- start 2024-04-14T08:33:14.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.759+0000 7f56cbc76640 1 -- start start 2024-04-14T08:33:14.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56cbc76640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c406b520 0x7f56c415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56cbc76640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56c40b5600 0x7f56c41510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56cbc76640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56c40b5b50 0x7f56c41f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56cbc76640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40b3f70 con 0x7f56c406b520 2024-04-14T08:33:14.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56cbc76640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40ae370 con 0x7f56c40b5600 2024-04-14T08:33:14.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56cbc76640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f56c409eda0 con 0x7f56c40b5b50 2024-04-14T08:33:14.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56c9a12640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c406b520 0x7f56c415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56c9a12640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c406b520 0x7f56c415b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49682/0 (socket says 172.21.15.172:49682) 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56c9a12640 1 -- 172.21.15.172:0/3205222066 learned_addr learned my addr 172.21.15.172:0/3205222066 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56ca213640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56c40b5b50 0x7f56c41f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56c9211640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56c40b5600 0x7f56c41510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56c9a12640 1 -- 172.21.15.172:0/3205222066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56c40b5b50 msgr2=0x7f56c41f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56c9a12640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56c40b5b50 0x7f56c41f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56c9a12640 1 -- 172.21.15.172:0/3205222066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56c40b5600 msgr2=0x7f56c41510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56c9a12640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56c40b5600 0x7f56c41510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56c9a12640 1 -- 172.21.15.172:0/3205222066 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c415bc70 con 0x7f56c406b520 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56c9a12640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c406b520 0x7f56c415b730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f56c0000c00 tx=0x7f56c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56ba7fc640 1 -- 172.21.15.172:0/3205222066 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0035040 con 0x7f56c406b520 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56ba7fc640 1 -- 172.21.15.172:0/3205222066 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56c0030d20 con 0x7f56c406b520 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56cbc76640 1 -- 172.21.15.172:0/3205222066 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c415a690 con 0x7f56c406b520 2024-04-14T08:33:14.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56ba7fc640 1 -- 172.21.15.172:0/3205222066 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0006e10 con 0x7f56c406b520 2024-04-14T08:33:14.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.760+0000 7f56cbc76640 1 -- 172.21.15.172:0/3205222066 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c415ac50 con 0x7f56c406b520 2024-04-14T08:33:14.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.761+0000 7f56cbc76640 1 -- 172.21.15.172:0/3205222066 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c4068520 con 0x7f56c406b520 2024-04-14T08:33:14.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.762+0000 7f56ba7fc640 1 -- 172.21.15.172:0/3205222066 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56c0038020 con 0x7f56c406b520 2024-04-14T08:33:14.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.762+0000 7f56ba7fc640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f56a806c8f0 0x7f56a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:14.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.762+0000 7f56ba7fc640 1 -- 172.21.15.172:0/3205222066 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f56c00ae3b0 con 0x7f56c406b520 2024-04-14T08:33:14.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.762+0000 7f56c9211640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f56a806c8f0 0x7f56a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:14.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.763+0000 7f56c9211640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f56a806c8f0 0x7f56a806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f56b4007fd0 tx=0x7f56b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:14.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.767+0000 7f56ba7fc640 1 -- 172.21.15.172:0/3205222066 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f56c002d040 con 0x7f56c406b520 2024-04-14T08:33:14.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.942+0000 7f56cbc76640 1 -- 172.21.15.172:0/3205222066 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56c40b0fa0 con 0x7f56c406b520 2024-04-14T08:33:14.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.944+0000 7f56ba7fc640 1 -- 172.21.15.172:0/3205222066 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7f56c007bee0 con 0x7f56c406b520 2024-04-14T08:33:14.944 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:14.944 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:14.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.946+0000 7f56cbc76640 1 -- 172.21.15.172:0/3205222066 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f56a806c8f0 msgr2=0x7f56a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.946+0000 7f56cbc76640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f56a806c8f0 0x7f56a806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f56b4007fd0 tx=0x7f56b4009040 comp rx=0 tx=0).stop 2024-04-14T08:33:14.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.946+0000 7f56cbc76640 1 -- 172.21.15.172:0/3205222066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c406b520 msgr2=0x7f56c415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:14.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.946+0000 7f56cbc76640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c406b520 0x7f56c415b730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f56c0000c00 tx=0x7f56c00046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.946+0000 7f56cbc76640 1 -- 172.21.15.172:0/3205222066 shutdown_connections 2024-04-14T08:33:14.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.946+0000 7f56cbc76640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f56a806c8f0 0x7f56a806edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.946+0000 7f56cbc76640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56c40b5b50 0x7f56c41f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.946+0000 7f56cbc76640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56c40b5600 0x7f56c41510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.946+0000 7f56cbc76640 1 --2- 172.21.15.172:0/3205222066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56c406b520 0x7f56c415b730 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:14.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.946+0000 7f56cbc76640 1 -- 172.21.15.172:0/3205222066 >> 172.21.15.172:0/3205222066 conn(0x7f56c40d0f20 msgr2=0x7f56c40b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:14.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.946+0000 7f56cbc76640 1 -- 172.21.15.172:0/3205222066 shutdown_connections 2024-04-14T08:33:14.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:14.946+0000 7f56cbc76640 1 -- 172.21.15.172:0/3205222066 wait complete. 2024-04-14T08:33:14.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:14.958 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:15.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.062+0000 7f0ad9f1e640 1 Processor -- start 2024-04-14T08:33:15.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.062+0000 7f0ad9f1e640 1 -- start start 2024-04-14T08:33:15.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad9f1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0ad406d040 0x7f0ad40b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad9f1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0ad40d0d30 0x7f0ad40cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad9f1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ad40d12e0 con 0x7f0ad415e820 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad9f1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0ad406cb90 con 0x7f0ad406d040 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad9f1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f0ad40cc660 con 0x7f0ad40d0d30 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad37fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0ad406d040 0x7f0ad40b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad37fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0ad406d040 0x7f0ad40b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49686/0 (socket says 172.21.15.172:49686) 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad37fe640 1 -- 172.21.15.172:0/3615317403 learned_addr learned my addr 172.21.15.172:0/3615317403 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad3fff640 1 --2- 172.21.15.172:0/3615317403 >> v2:172.21.15.177:3300/0 conn(0x7f0ad40d0d30 0x7f0ad40cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad37fe640 1 -- 172.21.15.172:0/3615317403 >> v2:172.21.15.177:3300/0 conn(0x7f0ad40d0d30 msgr2=0x7f0ad40cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad37fe640 1 --2- 172.21.15.172:0/3615317403 >> v2:172.21.15.177:3300/0 conn(0x7f0ad40d0d30 0x7f0ad40cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad37fe640 1 -- 172.21.15.172:0/3615317403 >> v1:172.21.15.177:6789/0 conn(0x7f0ad415e820 legacy=0x7f0ad406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.063+0000 7f0ad37fe640 1 -- 172.21.15.172:0/3615317403 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad40d1580 con 0x7f0ad406d040 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.064+0000 7f0ad37fe640 1 --2- 172.21.15.172:0/3615317403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad406d040 0x7f0ad40b4620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ad406ef90 tx=0x7f0ac402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706d89548cf1ab7e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.064+0000 7f0ad27fc640 1 -- 172.21.15.172:0/3615317403 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4034070 con 0x7f0ad406d040 2024-04-14T08:33:15.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.064+0000 7f0ad27fc640 1 -- 172.21.15.172:0/3615317403 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac4030860 con 0x7f0ad406d040 2024-04-14T08:33:15.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.064+0000 7f0ad27fc640 1 -- 172.21.15.172:0/3615317403 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4030c30 con 0x7f0ad406d040 2024-04-14T08:33:15.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.065+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3615317403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad406d040 msgr2=0x7f0ad40b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.065+0000 7f0ad9f1e640 1 --2- 172.21.15.172:0/3615317403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad406d040 0x7f0ad40b4620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ad406ef90 tx=0x7f0ac402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.065+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3615317403 shutdown_connections 2024-04-14T08:33:15.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.065+0000 7f0ad9f1e640 1 --2- 172.21.15.172:0/3615317403 >> v2:172.21.15.177:3300/0 conn(0x7f0ad40d0d30 0x7f0ad40cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.065+0000 7f0ad9f1e640 1 --2- 172.21.15.172:0/3615317403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad406d040 0x7f0ad40b4620 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.065+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3615317403 >> 172.21.15.172:0/3615317403 conn(0x7f0ad40b3b00 msgr2=0x7f0ad40b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:15.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.065+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3615317403 shutdown_connections 2024-04-14T08:33:15.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.065+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3615317403 wait complete. 2024-04-14T08:33:15.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.066+0000 7f0ad9f1e640 1 Processor -- start 2024-04-14T08:33:15.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.066+0000 7f0ad9f1e640 1 -- start start 2024-04-14T08:33:15.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad9f1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad406d040 0x7f0ad40b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad9f1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ad40d0d30 0x7f0ad41f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad9f1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ad415e820 0x7f0ad41f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad9f1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad40cd5f0 con 0x7f0ad406d040 2024-04-14T08:33:15.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad9f1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad40ccc20 con 0x7f0ad40d0d30 2024-04-14T08:33:15.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad9f1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad40cd770 con 0x7f0ad415e820 2024-04-14T08:33:15.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad406d040 0x7f0ad40b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad406d040 0x7f0ad40b7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49700/0 (socket says 172.21.15.172:49700) 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad37fe640 1 -- 172.21.15.172:0/3081805274 learned_addr learned my addr 172.21.15.172:0/3081805274 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad3fff640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ad415e820 0x7f0ad41f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad2ffd640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ad40d0d30 0x7f0ad41f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad37fe640 1 -- 172.21.15.172:0/3081805274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ad415e820 msgr2=0x7f0ad41f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad37fe640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ad415e820 0x7f0ad41f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad37fe640 1 -- 172.21.15.172:0/3081805274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ad40d0d30 msgr2=0x7f0ad41f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad37fe640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ad40d0d30 0x7f0ad41f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad37fe640 1 -- 172.21.15.172:0/3081805274 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad40cb0e0 con 0x7f0ad406d040 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad37fe640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad406d040 0x7f0ad40b7440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ac402b9e0 tx=0x7f0ac4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0abbfff640 1 -- 172.21.15.172:0/3081805274 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4048070 con 0x7f0ad406d040 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0abbfff640 1 -- 172.21.15.172:0/3081805274 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac4030660 con 0x7f0ad406d040 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3081805274 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad40cb3c0 con 0x7f0ad406d040 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.067+0000 7f0abbfff640 1 -- 172.21.15.172:0/3081805274 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4034040 con 0x7f0ad406d040 2024-04-14T08:33:15.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.068+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3081805274 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad40cb980 con 0x7f0ad406d040 2024-04-14T08:33:15.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.069+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3081805274 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad4068590 con 0x7f0ad406d040 2024-04-14T08:33:15.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.069+0000 7f0abbfff640 1 -- 172.21.15.172:0/3081805274 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ac404b070 con 0x7f0ad406d040 2024-04-14T08:33:15.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.069+0000 7f0abbfff640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0ac006c920 0x7f0ac006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.069+0000 7f0abbfff640 1 -- 172.21.15.172:0/3081805274 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0ac40add70 con 0x7f0ad406d040 2024-04-14T08:33:15.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.069+0000 7f0ad2ffd640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0ac006c920 0x7f0ac006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.070+0000 7f0ad2ffd640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0ac006c920 0x7f0ac006ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40b32c0 tx=0x7f0ac8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:15.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.074+0000 7f0abbfff640 1 -- 172.21.15.172:0/3081805274 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0ac407b800 con 0x7f0ad406d040 2024-04-14T08:33:15.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.250+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3081805274 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ad41f1b30 con 0x7f0ad406d040 2024-04-14T08:33:15.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.250+0000 7f0abbfff640 1 -- 172.21.15.172:0/3081805274 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f0ac407fb00 con 0x7f0ad406d040 2024-04-14T08:33:15.251 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:15.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.252+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3081805274 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0ac006c920 msgr2=0x7f0ac006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.252+0000 7f0ad9f1e640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0ac006c920 0x7f0ac006ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40b32c0 tx=0x7f0ac8009040 comp rx=0 tx=0).stop 2024-04-14T08:33:15.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.252+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3081805274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad406d040 msgr2=0x7f0ad40b7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.252+0000 7f0ad9f1e640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad406d040 0x7f0ad40b7440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ac402b9e0 tx=0x7f0ac4004730 comp rx=0 tx=0).stop 2024-04-14T08:33:15.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.252+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3081805274 shutdown_connections 2024-04-14T08:33:15.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.252+0000 7f0ad9f1e640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0ac006c920 0x7f0ac006ede0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.252+0000 7f0ad9f1e640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ad415e820 0x7f0ad41f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.252+0000 7f0ad9f1e640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ad40d0d30 0x7f0ad41f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.252+0000 7f0ad9f1e640 1 --2- 172.21.15.172:0/3081805274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad406d040 0x7f0ad40b7440 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.252+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3081805274 >> 172.21.15.172:0/3081805274 conn(0x7f0ad40b3b00 msgr2=0x7f0ad406f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:15.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.252+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3081805274 shutdown_connections 2024-04-14T08:33:15.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.252+0000 7f0ad9f1e640 1 -- 172.21.15.172:0/3081805274 wait complete. 2024-04-14T08:33:15.264 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:15.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.368+0000 7f73240de640 1 Processor -- start 2024-04-14T08:33:15.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.368+0000 7f73240de640 1 -- start start 2024-04-14T08:33:15.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f73240de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f731c13df30 0x7f731c0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f73240de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f731c140e80 0x7f731c0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f73240de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f731c15b620 con 0x7f731c15c380 2024-04-14T08:33:15.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f73240de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f731c0cf9b0 con 0x7f731c140e80 2024-04-14T08:33:15.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f73240de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f731c1414a0 con 0x7f731c13df30 2024-04-14T08:33:15.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f732267b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f731c140e80 0x7f731c0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f732267b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f731c140e80 0x7f731c0b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49708/0 (socket says 172.21.15.172:49708) 2024-04-14T08:33:15.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f7321679640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f731c15c380 0x7f731c140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35742/0 (socket says 172.21.15.172:35742) 2024-04-14T08:33:15.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f732267b640 1 -- 172.21.15.172:0/558749903 learned_addr learned my addr 172.21.15.172:0/558749903 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:15.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f7321e7a640 1 --2- 172.21.15.172:0/558749903 >> v2:172.21.15.177:3300/0 conn(0x7f731c13df30 0x7f731c0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f732267b640 1 -- 172.21.15.172:0/558749903 >> v2:172.21.15.177:3300/0 conn(0x7f731c13df30 msgr2=0x7f731c0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:15.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f732267b640 1 --2- 172.21.15.172:0/558749903 >> v2:172.21.15.177:3300/0 conn(0x7f731c13df30 0x7f731c0b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f732267b640 1 -- 172.21.15.172:0/558749903 >> v1:172.21.15.172:6789/0 conn(0x7f731c15c380 legacy=0x7f731c140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f732267b640 1 -- 172.21.15.172:0/558749903 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731c15b8a0 con 0x7f731c140e80 2024-04-14T08:33:15.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.369+0000 7f7320e78640 1 -- 172.21.15.172:0/558749903 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1629365876 0 0) 0x7f730c001110 con 0x7f731c15c380 2024-04-14T08:33:15.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.370+0000 7f732267b640 1 --2- 172.21.15.172:0/558749903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f731c140e80 0x7f731c0b2210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f731c06f660 tx=0x7f731402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383d7799b5f91d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:15.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.370+0000 7f7320e78640 1 -- 172.21.15.172:0/558749903 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731402cb20 con 0x7f731c140e80 2024-04-14T08:33:15.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.370+0000 7f7320e78640 1 -- 172.21.15.172:0/558749903 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731402ccc0 con 0x7f731c140e80 2024-04-14T08:33:15.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.370+0000 7f7320e78640 1 -- 172.21.15.172:0/558749903 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314034610 con 0x7f731c140e80 2024-04-14T08:33:15.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.370+0000 7f73240de640 1 -- 172.21.15.172:0/558749903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f731c140e80 msgr2=0x7f731c0b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.370+0000 7f73240de640 1 --2- 172.21.15.172:0/558749903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f731c140e80 0x7f731c0b2210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f731c06f660 tx=0x7f731402bf80 comp rx=0 tx=0).stop 2024-04-14T08:33:15.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.370+0000 7f73240de640 1 -- 172.21.15.172:0/558749903 shutdown_connections 2024-04-14T08:33:15.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.370+0000 7f73240de640 1 --2- 172.21.15.172:0/558749903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f731c140e80 0x7f731c0b2210 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.370+0000 7f73240de640 1 --2- 172.21.15.172:0/558749903 >> v2:172.21.15.177:3300/0 conn(0x7f731c13df30 0x7f731c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.370+0000 7f73240de640 1 -- 172.21.15.172:0/558749903 >> 172.21.15.172:0/558749903 conn(0x7f731c06b520 msgr2=0x7f731c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:15.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.371+0000 7f73240de640 1 -- 172.21.15.172:0/558749903 shutdown_connections 2024-04-14T08:33:15.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.371+0000 7f73240de640 1 -- 172.21.15.172:0/558749903 wait complete. 2024-04-14T08:33:15.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.372+0000 7f73240de640 1 Processor -- start 2024-04-14T08:33:15.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.372+0000 7f73240de640 1 -- start start 2024-04-14T08:33:15.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.372+0000 7f73240de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f731c13df30 0x7f731c1e3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.372+0000 7f73240de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f731c140e80 0x7f731c0b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.372+0000 7f73240de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f731c15c380 0x7f731c153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.372+0000 7f73240de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f731c13e310 con 0x7f731c13df30 2024-04-14T08:33:15.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.372+0000 7f73240de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f731c15be60 con 0x7f731c15c380 2024-04-14T08:33:15.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.372+0000 7f73240de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f731c0ce9a0 con 0x7f731c140e80 2024-04-14T08:33:15.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f7321e7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f731c13df30 0x7f731c1e3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f7321e7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f731c13df30 0x7f731c1e3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49710/0 (socket says 172.21.15.172:49710) 2024-04-14T08:33:15.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f7321e7a640 1 -- 172.21.15.172:0/2086383281 learned_addr learned my addr 172.21.15.172:0/2086383281 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:15.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f7321679640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f731c140e80 0x7f731c0b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f732267b640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f731c15c380 0x7f731c153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f7321e7a640 1 -- 172.21.15.172:0/2086383281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f731c140e80 msgr2=0x7f731c0b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f7321e7a640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f731c140e80 0x7f731c0b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f7321e7a640 1 -- 172.21.15.172:0/2086383281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f731c15c380 msgr2=0x7f731c153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f7321e7a640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f731c15c380 0x7f731c153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f7321e7a640 1 -- 172.21.15.172:0/2086383281 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731c0b1410 con 0x7f731c13df30 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f7321e7a640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f731c13df30 0x7f731c1e3ae0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f731800e9b0 tx=0x7f731800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f73127fc640 1 -- 172.21.15.172:0/2086383281 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318017070 con 0x7f731c13df30 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f73127fc640 1 -- 172.21.15.172:0/2086383281 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731800c9a0 con 0x7f731c13df30 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f73240de640 1 -- 172.21.15.172:0/2086383281 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731c0b16b0 con 0x7f731c13df30 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f73127fc640 1 -- 172.21.15.172:0/2086383281 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731800cd90 con 0x7f731c13df30 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.373+0000 7f73240de640 1 -- 172.21.15.172:0/2086383281 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f731c0b1c70 con 0x7f731c13df30 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.374+0000 7f73240de640 1 -- 172.21.15.172:0/2086383281 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f731c068590 con 0x7f731c13df30 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.375+0000 7f73127fc640 1 -- 172.21.15.172:0/2086383281 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f731801a070 con 0x7f731c13df30 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.375+0000 7f73127fc640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f730006c920 0x7f730006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.375+0000 7f73127fc640 1 -- 172.21.15.172:0/2086383281 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f731808bde0 con 0x7f731c13df30 2024-04-14T08:33:15.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.375+0000 7f7321679640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f730006c920 0x7f730006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.375+0000 7f7321679640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f730006c920 0x7f730006ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f731c13cbe0 tx=0x7f730c007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:15.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.379+0000 7f73127fc640 1 -- 172.21.15.172:0/2086383281 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7318059880 con 0x7f731c13df30 2024-04-14T08:33:15.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.556+0000 7f73240de640 1 -- 172.21.15.172:0/2086383281 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f731c0730c0 con 0x7f731c13df30 2024-04-14T08:33:15.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.557+0000 7f73127fc640 1 -- 172.21.15.172:0/2086383281 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f731805db80 con 0x7f731c13df30 2024-04-14T08:33:15.558 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:15.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.559+0000 7f73240de640 1 -- 172.21.15.172:0/2086383281 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f730006c920 msgr2=0x7f730006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.559+0000 7f73240de640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f730006c920 0x7f730006ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f731c13cbe0 tx=0x7f730c007de0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.559+0000 7f73240de640 1 -- 172.21.15.172:0/2086383281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f731c13df30 msgr2=0x7f731c1e3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.559+0000 7f73240de640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f731c13df30 0x7f731c1e3ae0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f731800e9b0 tx=0x7f731800ee70 comp rx=0 tx=0).stop 2024-04-14T08:33:15.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.559+0000 7f73240de640 1 -- 172.21.15.172:0/2086383281 shutdown_connections 2024-04-14T08:33:15.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.559+0000 7f73240de640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f730006c920 0x7f730006ede0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.559+0000 7f73240de640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f731c15c380 0x7f731c153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.559+0000 7f73240de640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f731c140e80 0x7f731c0b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.559+0000 7f73240de640 1 --2- 172.21.15.172:0/2086383281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f731c13df30 0x7f731c1e3ae0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.559+0000 7f73240de640 1 -- 172.21.15.172:0/2086383281 >> 172.21.15.172:0/2086383281 conn(0x7f731c06b520 msgr2=0x7f731c159340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:15.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.559+0000 7f73240de640 1 -- 172.21.15.172:0/2086383281 shutdown_connections 2024-04-14T08:33:15.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.559+0000 7f73240de640 1 -- 172.21.15.172:0/2086383281 wait complete. 2024-04-14T08:33:15.570 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:15.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.673+0000 7f2f7a436640 1 Processor -- start 2024-04-14T08:33:15.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.673+0000 7f2f7a436640 1 -- start start 2024-04-14T08:33:15.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.674+0000 7f2f7a436640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f74162710 0x7f2f740d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.674+0000 7f2f7a436640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f74162b70 0x7f2f740d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.674+0000 7f2f7a436640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f740cefb0 con 0x7f2f7415e430 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.674+0000 7f2f7a436640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2f740cf290 con 0x7f2f74162b70 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.674+0000 7f2f7a436640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2f740cf570 con 0x7f2f74162710 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.674+0000 7f2f789d3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f74162b70 0x7f2f740d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.674+0000 7f2f73fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2f7415e430 0x7f2f74162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35754/0 (socket says 172.21.15.172:35754) 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.674+0000 7f2f73fff640 1 -- 172.21.15.172:0/3117094815 learned_addr learned my addr 172.21.15.172:0/3117094815 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.674+0000 7f2f737fe640 1 --2- 172.21.15.172:0/3117094815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f74162710 0x7f2f740d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.675+0000 7f2f789d3640 1 -- 172.21.15.172:0/3117094815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f74162710 msgr2=0x7f2f740d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.675+0000 7f2f789d3640 1 --2- 172.21.15.172:0/3117094815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f74162710 0x7f2f740d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.675+0000 7f2f789d3640 1 -- 172.21.15.172:0/3117094815 >> v1:172.21.15.172:6789/0 conn(0x7f2f7415e430 legacy=0x7f2f74162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.675+0000 7f2f72ffd640 1 -- 172.21.15.172:0/3117094815 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 47381984 0 0) 0x7f2f740cefb0 con 0x7f2f7415e430 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.675+0000 7f2f789d3640 1 -- 172.21.15.172:0/3117094815 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f740cf850 con 0x7f2f74162b70 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.675+0000 7f2f737fe640 1 --2- 172.21.15.172:0/3117094815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f74162710 0x7f2f740d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:15.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.675+0000 7f2f789d3640 1 --2- 172.21.15.172:0/3117094815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f74162b70 0x7f2f740d2330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009d90 tx=0x7f2f6c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212d5f39ced89873 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:15.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.675+0000 7f2f72ffd640 1 -- 172.21.15.172:0/3117094815 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c039070 con 0x7f2f74162b70 2024-04-14T08:33:15.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.676+0000 7f2f72ffd640 1 -- 172.21.15.172:0/3117094815 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f6c02cc50 con 0x7f2f74162b70 2024-04-14T08:33:15.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.676+0000 7f2f72ffd640 1 -- 172.21.15.172:0/3117094815 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c034440 con 0x7f2f74162b70 2024-04-14T08:33:15.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.676+0000 7f2f7a436640 1 -- 172.21.15.172:0/3117094815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f74162b70 msgr2=0x7f2f740d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.676+0000 7f2f7a436640 1 --2- 172.21.15.172:0/3117094815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f74162b70 0x7f2f740d2330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009d90 tx=0x7f2f6c02bea0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.676+0000 7f2f7a436640 1 -- 172.21.15.172:0/3117094815 shutdown_connections 2024-04-14T08:33:15.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.676+0000 7f2f7a436640 1 --2- 172.21.15.172:0/3117094815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f74162710 0x7f2f740d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.676+0000 7f2f7a436640 1 --2- 172.21.15.172:0/3117094815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f74162b70 0x7f2f740d2330 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.676+0000 7f2f7a436640 1 -- 172.21.15.172:0/3117094815 >> 172.21.15.172:0/3117094815 conn(0x7f2f7409d560 msgr2=0x7f2f740d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:15.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.676+0000 7f2f7a436640 1 -- 172.21.15.172:0/3117094815 shutdown_connections 2024-04-14T08:33:15.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.676+0000 7f2f7a436640 1 -- 172.21.15.172:0/3117094815 wait complete. 2024-04-14T08:33:15.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.677+0000 7f2f7a436640 1 Processor -- start 2024-04-14T08:33:15.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.677+0000 7f2f7a436640 1 -- start start 2024-04-14T08:33:15.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f7a436640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f7415e430 0x7f2f740b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f7a436640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f74162710 0x7f2f741f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f7a436640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f74162b70 0x7f2f741f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f7a436640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2f740d4e20 con 0x7f2f7415e430 2024-04-14T08:33:15.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f7a436640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7415e810 con 0x7f2f74162710 2024-04-14T08:33:15.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f7a436640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2f740d0050 con 0x7f2f74162b70 2024-04-14T08:33:15.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f73fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f7415e430 0x7f2f740b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f73fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f7415e430 0x7f2f740b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49724/0 (socket says 172.21.15.172:49724) 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f73fff640 1 -- 172.21.15.172:0/3053867740 learned_addr learned my addr 172.21.15.172:0/3053867740 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f737fe640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f74162710 0x7f2f741f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f789d3640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f74162b70 0x7f2f741f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f73fff640 1 -- 172.21.15.172:0/3053867740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f74162b70 msgr2=0x7f2f741f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f73fff640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f74162b70 0x7f2f741f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f73fff640 1 -- 172.21.15.172:0/3053867740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f74162710 msgr2=0x7f2f741f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.678+0000 7f2f73fff640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f74162710 0x7f2f741f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.679+0000 7f2f73fff640 1 -- 172.21.15.172:0/3053867740 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f741f4ca0 con 0x7f2f7415e430 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.679+0000 7f2f789d3640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f74162b70 0x7f2f741f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.679+0000 7f2f73fff640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f7415e430 0x7f2f740b7d20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400c430 tx=0x7f2f6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.679+0000 7f2f70ff9640 1 -- 172.21.15.172:0/3053867740 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f640074e0 con 0x7f2f7415e430 2024-04-14T08:33:15.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.679+0000 7f2f70ff9640 1 -- 172.21.15.172:0/3053867740 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f640047e0 con 0x7f2f7415e430 2024-04-14T08:33:15.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.679+0000 7f2f7a436640 1 -- 172.21.15.172:0/3053867740 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f741f4f80 con 0x7f2f7415e430 2024-04-14T08:33:15.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.679+0000 7f2f70ff9640 1 -- 172.21.15.172:0/3053867740 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64004bd0 con 0x7f2f7415e430 2024-04-14T08:33:15.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.679+0000 7f2f7a436640 1 -- 172.21.15.172:0/3053867740 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f741f5260 con 0x7f2f7415e430 2024-04-14T08:33:15.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.680+0000 7f2f7a436640 1 -- 172.21.15.172:0/3053867740 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f74068590 con 0x7f2f7415e430 2024-04-14T08:33:15.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.680+0000 7f2f70ff9640 1 -- 172.21.15.172:0/3053867740 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f64011070 con 0x7f2f7415e430 2024-04-14T08:33:15.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.681+0000 7f2f70ff9640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2f6006c8f0 0x7f2f6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.681+0000 7f2f70ff9640 1 -- 172.21.15.172:0/3053867740 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2f6408ba80 con 0x7f2f7415e430 2024-04-14T08:33:15.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.681+0000 7f2f737fe640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2f6006c8f0 0x7f2f6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.681+0000 7f2f737fe640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2f6006c8f0 0x7f2f6006edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009560 tx=0x7f2f68009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:15.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.685+0000 7f2f70ff9640 1 -- 172.21.15.172:0/3053867740 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2f64059b70 con 0x7f2f7415e430 2024-04-14T08:33:15.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.860+0000 7f2f7a436640 1 -- 172.21.15.172:0/3053867740 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2f740d18a0 con 0x7f2f7415e430 2024-04-14T08:33:15.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.861+0000 7f2f70ff9640 1 -- 172.21.15.172:0/3053867740 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f2f64059470 con 0x7f2f7415e430 2024-04-14T08:33:15.861 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:15.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.862+0000 7f2f7a436640 1 -- 172.21.15.172:0/3053867740 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2f6006c8f0 msgr2=0x7f2f6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.862+0000 7f2f7a436640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2f6006c8f0 0x7f2f6006edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009560 tx=0x7f2f68009190 comp rx=0 tx=0).stop 2024-04-14T08:33:15.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.862+0000 7f2f7a436640 1 -- 172.21.15.172:0/3053867740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f7415e430 msgr2=0x7f2f740b7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.862+0000 7f2f7a436640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f7415e430 0x7f2f740b7d20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400c430 tx=0x7f2f6400c8f0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.862+0000 7f2f7a436640 1 -- 172.21.15.172:0/3053867740 shutdown_connections 2024-04-14T08:33:15.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.862+0000 7f2f7a436640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2f6006c8f0 0x7f2f6006edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.863+0000 7f2f7a436640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f74162b70 0x7f2f741f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.863+0000 7f2f7a436640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f74162710 0x7f2f741f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.863+0000 7f2f7a436640 1 --2- 172.21.15.172:0/3053867740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f7415e430 0x7f2f740b7d20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.863+0000 7f2f7a436640 1 -- 172.21.15.172:0/3053867740 >> 172.21.15.172:0/3053867740 conn(0x7f2f7409d560 msgr2=0x7f2f740b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:15.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.863+0000 7f2f7a436640 1 -- 172.21.15.172:0/3053867740 shutdown_connections 2024-04-14T08:33:15.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.863+0000 7f2f7a436640 1 -- 172.21.15.172:0/3053867740 wait complete. 2024-04-14T08:33:15.874 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:15.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.977+0000 7fdbcc9f0640 1 Processor -- start 2024-04-14T08:33:15.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.977+0000 7fdbcc9f0640 1 -- start start 2024-04-14T08:33:15.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.978+0000 7fdbcc9f0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc80ccca0 0x7fdbc806fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.978+0000 7fdbcc9f0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdbc8073090 0x7fdbc80b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.978+0000 7fdbcc9f0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdbc806b520 0x7fdbc806b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.978+0000 7fdbcc9f0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fdbc806be90 con 0x7fdbc806b520 2024-04-14T08:33:15.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.978+0000 7fdbcc9f0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fdbc80734f0 con 0x7fdbc8073090 2024-04-14T08:33:15.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.978+0000 7fdbcc9f0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc80ceee0 con 0x7fdbc80ccca0 2024-04-14T08:33:15.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.978+0000 7fdbc6d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdbc806b520 0x7fdbc806b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.978+0000 7fdbc6d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdbc806b520 0x7fdbc806b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49738/0 (socket says 172.21.15.172:49738) 2024-04-14T08:33:15.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.978+0000 7fdbc6d9d640 1 -- 172.21.15.172:0/4111210649 learned_addr learned my addr 172.21.15.172:0/4111210649 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:15.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.978+0000 7fdbc659c640 1 --2- 172.21.15.172:0/4111210649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc80ccca0 0x7fdbc806fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.978+0000 7fdbc5d9b640 1 --2- 172.21.15.172:0/4111210649 >> v2:172.21.15.177:3300/0 conn(0x7fdbc8073090 0x7fdbc80b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.979+0000 7fdbc659c640 1 -- 172.21.15.172:0/4111210649 >> v2:172.21.15.177:3300/0 conn(0x7fdbc8073090 msgr2=0x7fdbc80b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.979+0000 7fdbc659c640 1 --2- 172.21.15.172:0/4111210649 >> v2:172.21.15.177:3300/0 conn(0x7fdbc8073090 0x7fdbc80b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.979+0000 7fdbc659c640 1 -- 172.21.15.172:0/4111210649 >> v2:172.21.15.172:3300/0 conn(0x7fdbc806b520 msgr2=0x7fdbc806b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.979+0000 7fdbc659c640 1 --2- 172.21.15.172:0/4111210649 >> v2:172.21.15.172:3300/0 conn(0x7fdbc806b520 0x7fdbc806b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.979+0000 7fdbc659c640 1 -- 172.21.15.172:0/4111210649 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc8070150 con 0x7fdbc80ccca0 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.979+0000 7fdbc6d9d640 1 --2- 172.21.15.172:0/4111210649 >> v2:172.21.15.172:3300/0 conn(0x7fdbc806b520 0x7fdbc806b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.979+0000 7fdbc659c640 1 --2- 172.21.15.172:0/4111210649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc80ccca0 0x7fdbc806fb70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8073fd0 tx=0x7fdbb802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad771306132d64b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.979+0000 7fdbc559a640 1 -- 172.21.15.172:0/4111210649 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8039070 con 0x7fdbc80ccca0 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.979+0000 7fdbc559a640 1 -- 172.21.15.172:0/4111210649 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb80316f0 con 0x7fdbc80ccca0 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.979+0000 7fdbc559a640 1 -- 172.21.15.172:0/4111210649 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8031ac0 con 0x7fdbc80ccca0 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.980+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/4111210649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc80ccca0 msgr2=0x7fdbc806fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.980+0000 7fdbcc9f0640 1 --2- 172.21.15.172:0/4111210649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc80ccca0 0x7fdbc806fb70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8073fd0 tx=0x7fdbb802b790 comp rx=0 tx=0).stop 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.980+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/4111210649 shutdown_connections 2024-04-14T08:33:15.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.980+0000 7fdbcc9f0640 1 --2- 172.21.15.172:0/4111210649 >> v2:172.21.15.172:3300/0 conn(0x7fdbc806b520 0x7fdbc806b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.980+0000 7fdbcc9f0640 1 --2- 172.21.15.172:0/4111210649 >> v2:172.21.15.177:3300/0 conn(0x7fdbc8073090 0x7fdbc80b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.980+0000 7fdbcc9f0640 1 --2- 172.21.15.172:0/4111210649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc80ccca0 0x7fdbc806fb70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.980+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/4111210649 >> 172.21.15.172:0/4111210649 conn(0x7fdbc80cda00 msgr2=0x7fdbc80cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:15.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.980+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/4111210649 shutdown_connections 2024-04-14T08:33:15.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.980+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/4111210649 wait complete. 2024-04-14T08:33:15.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.981+0000 7fdbcc9f0640 1 Processor -- start 2024-04-14T08:33:15.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.981+0000 7fdbcc9f0640 1 -- start start 2024-04-14T08:33:15.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbcc9f0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc806b520 0x7fdbc80b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbcc9f0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc8073090 0x7fdbc8155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbcc9f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc80ccca0 0x7fdbc80c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbcc9f0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8072890 con 0x7fdbc80ccca0 2024-04-14T08:33:15.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbcc9f0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc80725b0 con 0x7fdbc8073090 2024-04-14T08:33:15.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbcc9f0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc80cd080 con 0x7fdbc806b520 2024-04-14T08:33:15.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbc6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc80ccca0 0x7fdbc80c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbc6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc80ccca0 0x7fdbc80c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49752/0 (socket says 172.21.15.172:49752) 2024-04-14T08:33:15.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbc6d9d640 1 -- 172.21.15.172:0/3731549447 learned_addr learned my addr 172.21.15.172:0/3731549447 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:15.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbc659c640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc806b520 0x7fdbc80b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbc5d9b640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc8073090 0x7fdbc8155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbc659c640 1 -- 172.21.15.172:0/3731549447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc8073090 msgr2=0x7fdbc8155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbc659c640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc8073090 0x7fdbc8155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbc659c640 1 -- 172.21.15.172:0/3731549447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc80ccca0 msgr2=0x7fdbc80c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbc659c640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc80ccca0 0x7fdbc80c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbc659c640 1 -- 172.21.15.172:0/3731549447 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc8155aa0 con 0x7fdbc806b520 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.982+0000 7fdbc6d9d640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc80ccca0 0x7fdbc80c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.983+0000 7fdbc659c640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc806b520 0x7fdbc80b0150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8033000 tx=0x7fdbb80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.983+0000 7fdbb6ffd640 1 -- 172.21.15.172:0/3731549447 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8043070 con 0x7fdbc806b520 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.983+0000 7fdbb6ffd640 1 -- 172.21.15.172:0/3731549447 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb80346a0 con 0x7fdbc806b520 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.983+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/3731549447 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc80cc300 con 0x7fdbc806b520 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.983+0000 7fdbb6ffd640 1 -- 172.21.15.172:0/3731549447 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8039040 con 0x7fdbc806b520 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.983+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/3731549447 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc80ae090 con 0x7fdbc806b520 2024-04-14T08:33:15.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.984+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/3731549447 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc8068590 con 0x7fdbc806b520 2024-04-14T08:33:15.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.984+0000 7fdbb6ffd640 1 -- 172.21.15.172:0/3731549447 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbb803c070 con 0x7fdbc806b520 2024-04-14T08:33:15.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.984+0000 7fdbb6ffd640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdba806c8f0 0x7fdba806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:15.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.985+0000 7fdbc5d9b640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdba806c8f0 0x7fdba806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:15.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.985+0000 7fdbb6ffd640 1 -- 172.21.15.172:0/3731549447 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdbb80ad870 con 0x7fdbc806b520 2024-04-14T08:33:15.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.985+0000 7fdbc5d9b640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdba806c8f0 0x7fdba806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80b4c10 tx=0x7fdbbc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:15.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:15.989+0000 7fdbb6ffd640 1 -- 172.21.15.172:0/3731549447 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdbb807b390 con 0x7fdbc806b520 2024-04-14T08:33:16.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.163+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/3731549447 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdbc806c120 con 0x7fdbc806b520 2024-04-14T08:33:16.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.164+0000 7fdbb6ffd640 1 -- 172.21.15.172:0/3731549447 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fdbb807f690 con 0x7fdbc806b520 2024-04-14T08:33:16.164 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:16.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.166+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/3731549447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdba806c8f0 msgr2=0x7fdba806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.166+0000 7fdbcc9f0640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdba806c8f0 0x7fdba806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80b4c10 tx=0x7fdbbc009190 comp rx=0 tx=0).stop 2024-04-14T08:33:16.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.166+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/3731549447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc806b520 msgr2=0x7fdbc80b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.166+0000 7fdbcc9f0640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc806b520 0x7fdbc80b0150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8033000 tx=0x7fdbb80046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.166+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/3731549447 shutdown_connections 2024-04-14T08:33:16.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.166+0000 7fdbcc9f0640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdba806c8f0 0x7fdba806edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.166+0000 7fdbcc9f0640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc80ccca0 0x7fdbc80c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.166+0000 7fdbcc9f0640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc8073090 0x7fdbc8155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.166+0000 7fdbcc9f0640 1 --2- 172.21.15.172:0/3731549447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc806b520 0x7fdbc80b0150 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.166+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/3731549447 >> 172.21.15.172:0/3731549447 conn(0x7fdbc80cda00 msgr2=0x7fdbc80b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:16.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.166+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/3731549447 shutdown_connections 2024-04-14T08:33:16.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.166+0000 7fdbcc9f0640 1 -- 172.21.15.172:0/3731549447 wait complete. 2024-04-14T08:33:16.177 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:33:16.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.277+0000 7f38c44de640 1 Processor -- start 2024-04-14T08:33:16.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.277+0000 7f38c44de640 1 -- start start 2024-04-14T08:33:16.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.278+0000 7f38c44de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f38bc0b1df0 0x7f38bc0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.278+0000 7f38c44de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38bc15c770 con 0x7f38bc0b4040 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.278+0000 7f38c44de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38bc0b3430 con 0x7f38bc0b21d0 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.278+0000 7f38c44de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f38bc0b3700 con 0x7f38bc0b1df0 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.278+0000 7f38c227a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f38bc0b1df0 0x7f38bc0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.278+0000 7f38c2a7b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f38bc0b4040 0x7f38bc0b4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35768/0 (socket says 172.21.15.172:35768) 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.278+0000 7f38c227a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f38bc0b1df0 0x7f38bc0b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49754/0 (socket says 172.21.15.172:49754) 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.278+0000 7f38c2a7b640 1 -- 172.21.15.172:0/2005675420 learned_addr learned my addr 172.21.15.172:0/2005675420 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.278+0000 7f38c1a79640 1 --1- 172.21.15.172:0/2005675420 >> v1:172.21.15.177:6789/0 conn(0x7f38bc0b21d0 0x7f38bc15c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42786/0 (socket says 172.21.15.172:42786) 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.279+0000 7f38c1278640 1 -- 172.21.15.172:0/2005675420 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1118477617 0 0) 0x7f38bc0b3430 con 0x7f38bc0b21d0 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.279+0000 7f38c1278640 1 -- 172.21.15.172:0/2005675420 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38a8003520 con 0x7f38bc0b21d0 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.279+0000 7f38c227a640 1 -- 172.21.15.172:0/2005675420 >> v1:172.21.15.177:6789/0 conn(0x7f38bc0b21d0 legacy=0x7f38bc15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.279+0000 7f38c227a640 1 -- 172.21.15.172:0/2005675420 >> v1:172.21.15.172:6789/0 conn(0x7f38bc0b4040 legacy=0x7f38bc0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.279+0000 7f38c227a640 1 -- 172.21.15.172:0/2005675420 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc0b39e0 con 0x7f38bc0b1df0 2024-04-14T08:33:16.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.279+0000 7f38c1278640 1 -- 172.21.15.172:0/2005675420 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1647246950 0 0) 0x7f38bc15c770 con 0x7f38bc0b4040 2024-04-14T08:33:16.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.279+0000 7f38c227a640 1 --2- 172.21.15.172:0/2005675420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38bc0b1df0 0x7f38bc0b2ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38b800bfa0 tx=0x7f38b802e140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed9ef9bb09cdadd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:16.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.280+0000 7f38c1278640 1 -- 172.21.15.172:0/2005675420 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b802eae0 con 0x7f38bc0b1df0 2024-04-14T08:33:16.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.280+0000 7f38c1278640 1 -- 172.21.15.172:0/2005675420 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b802ec80 con 0x7f38bc0b1df0 2024-04-14T08:33:16.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.280+0000 7f38c1278640 1 -- 172.21.15.172:0/2005675420 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8009760 con 0x7f38bc0b1df0 2024-04-14T08:33:16.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.280+0000 7f38c44de640 1 -- 172.21.15.172:0/2005675420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38bc0b1df0 msgr2=0x7f38bc0b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.280+0000 7f38c44de640 1 --2- 172.21.15.172:0/2005675420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38bc0b1df0 0x7f38bc0b2ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38b800bfa0 tx=0x7f38b802e140 comp rx=0 tx=0).stop 2024-04-14T08:33:16.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.280+0000 7f38c44de640 1 -- 172.21.15.172:0/2005675420 shutdown_connections 2024-04-14T08:33:16.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.280+0000 7f38c44de640 1 --2- 172.21.15.172:0/2005675420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38bc0b1df0 0x7f38bc0b2ef0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.280+0000 7f38c44de640 1 -- 172.21.15.172:0/2005675420 >> 172.21.15.172:0/2005675420 conn(0x7f38bc06ec30 msgr2=0x7f38bc06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:16.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.280+0000 7f38c44de640 1 -- 172.21.15.172:0/2005675420 shutdown_connections 2024-04-14T08:33:16.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.280+0000 7f38c44de640 1 -- 172.21.15.172:0/2005675420 wait complete. 2024-04-14T08:33:16.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.281+0000 7f38c44de640 1 Processor -- start 2024-04-14T08:33:16.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.281+0000 7f38c44de640 1 -- start start 2024-04-14T08:33:16.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.282+0000 7f38c44de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38bc0b1df0 0x7f38bc0aad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.282+0000 7f38c44de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38bc0b21d0 0x7f38bc130610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.282+0000 7f38c44de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38bc0b4040 0x7f38bc0a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.282+0000 7f38c44de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc15b7f0 con 0x7f38bc0b1df0 2024-04-14T08:33:16.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.282+0000 7f38c44de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc15bb40 con 0x7f38bc0b4040 2024-04-14T08:33:16.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.282+0000 7f38c44de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f38bc1552d0 con 0x7f38bc0b21d0 2024-04-14T08:33:16.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.282+0000 7f38c227a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38bc0b1df0 0x7f38bc0aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.282+0000 7f38c227a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38bc0b1df0 0x7f38bc0aad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49766/0 (socket says 172.21.15.172:49766) 2024-04-14T08:33:16.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.282+0000 7f38c227a640 1 -- 172.21.15.172:0/1996132348 learned_addr learned my addr 172.21.15.172:0/1996132348 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:16.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.282+0000 7f38c2a7b640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38bc0b4040 0x7f38bc0a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.282+0000 7f38c1a79640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38bc0b21d0 0x7f38bc130610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.283+0000 7f38c227a640 1 -- 172.21.15.172:0/1996132348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38bc0b21d0 msgr2=0x7f38bc130610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.283+0000 7f38c227a640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38bc0b21d0 0x7f38bc130610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.283+0000 7f38c227a640 1 -- 172.21.15.172:0/1996132348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38bc0b4040 msgr2=0x7f38bc0a9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.283+0000 7f38c227a640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38bc0b4040 0x7f38bc0a9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.283+0000 7f38c227a640 1 -- 172.21.15.172:0/1996132348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc130f80 con 0x7f38bc0b1df0 2024-04-14T08:33:16.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.283+0000 7f38c227a640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38bc0b1df0 0x7f38bc0aad60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f38b800bfa0 tx=0x7f38b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:16.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.283+0000 7f38b2ffd640 1 -- 172.21.15.172:0/1996132348 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8003990 con 0x7f38bc0b1df0 2024-04-14T08:33:16.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.283+0000 7f38b2ffd640 1 -- 172.21.15.172:0/1996132348 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b8003b50 con 0x7f38bc0b1df0 2024-04-14T08:33:16.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.283+0000 7f38b2ffd640 1 -- 172.21.15.172:0/1996132348 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8004030 con 0x7f38bc0b1df0 2024-04-14T08:33:16.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.283+0000 7f38c44de640 1 -- 172.21.15.172:0/1996132348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc131260 con 0x7f38bc0b1df0 2024-04-14T08:33:16.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.283+0000 7f38c44de640 1 -- 172.21.15.172:0/1996132348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc0a98e0 con 0x7f38bc0b1df0 2024-04-14T08:33:16.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.284+0000 7f38c44de640 1 -- 172.21.15.172:0/1996132348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38bc068520 con 0x7f38bc0b1df0 2024-04-14T08:33:16.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.285+0000 7f38b2ffd640 1 -- 172.21.15.172:0/1996132348 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38b80041d0 con 0x7f38bc0b1df0 2024-04-14T08:33:16.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.285+0000 7f38b2ffd640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f38a006c8f0 0x7f38a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.285+0000 7f38c1a79640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f38a006c8f0 0x7f38a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.285+0000 7f38b2ffd640 1 -- 172.21.15.172:0/1996132348 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f38b80b0c70 con 0x7f38bc0b1df0 2024-04-14T08:33:16.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.285+0000 7f38c1a79640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f38a006c8f0 0x7f38a006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f38ac007fd0 tx=0x7f38ac007da0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:16.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.289+0000 7f38b2ffd640 1 -- 172.21.15.172:0/1996132348 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f38b807e7a0 con 0x7f38bc0b1df0 2024-04-14T08:33:16.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.502+0000 7f38c44de640 1 -- 172.21.15.172:0/1996132348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38bc0b33b0 con 0x7f38bc0b1df0 2024-04-14T08:33:16.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.502+0000 7f38b2ffd640 1 -- 172.21.15.172:0/1996132348 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f38b8082aa0 con 0x7f38bc0b1df0 2024-04-14T08:33:16.503 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:16.503 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3053874257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3053874257},{"type":"v1","addr":"172.21.15.172:6835","nonce":3053874257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2790588849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2790588849},{"type":"v1","addr":"172.21.15.172:6837","nonce":2790588849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3130776573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3130776573},{"type":"v1","addr":"172.21.15.177:6834","nonce":3130776573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:32:53.535483+0000","modified":"2024-04-14T08:33:09.604218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/517963877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":517963877},{"type":"v1","addr":"172.21.15.177:6835","nonce":517963877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4309,"qdb_cluster":[4309]},"id":1}]} 2024-04-14T08:33:16.504 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 9 2024-04-14T08:33:16.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.504+0000 7f38c44de640 1 -- 172.21.15.172:0/1996132348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f38a006c8f0 msgr2=0x7f38a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.504+0000 7f38c44de640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f38a006c8f0 0x7f38a006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f38ac007fd0 tx=0x7f38ac007da0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.504+0000 7f38c44de640 1 -- 172.21.15.172:0/1996132348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38bc0b1df0 msgr2=0x7f38bc0aad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.504+0000 7f38c44de640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38bc0b1df0 0x7f38bc0aad60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f38b800bfa0 tx=0x7f38b8004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:16.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.504+0000 7f38c44de640 1 -- 172.21.15.172:0/1996132348 shutdown_connections 2024-04-14T08:33:16.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.504+0000 7f38c44de640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f38a006c8f0 0x7f38a006edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.504+0000 7f38c44de640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38bc0b4040 0x7f38bc0a9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.504+0000 7f38c44de640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38bc0b21d0 0x7f38bc130610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.504+0000 7f38c44de640 1 --2- 172.21.15.172:0/1996132348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38bc0b1df0 0x7f38bc0aad60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.504+0000 7f38c44de640 1 -- 172.21.15.172:0/1996132348 >> 172.21.15.172:0/1996132348 conn(0x7f38bc06ec30 msgr2=0x7f38bc13d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:16.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.504+0000 7f38c44de640 1 -- 172.21.15.172:0/1996132348 shutdown_connections 2024-04-14T08:33:16.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.504+0000 7f38c44de640 1 -- 172.21.15.172:0/1996132348 wait complete. 2024-04-14T08:33:16.516 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:16.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.616+0000 7f10e2b08640 1 Processor -- start 2024-04-14T08:33:16.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.616+0000 7f10e2b08640 1 -- start start 2024-04-14T08:33:16.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.617+0000 7f10e2b08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10dc0b3dd0 0x7f10dc15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.617+0000 7f10e2b08640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10dc070270 con 0x7f10dc06fdf0 2024-04-14T08:33:16.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.617+0000 7f10e2b08640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10dc0b4540 con 0x7f10dc06b520 2024-04-14T08:33:16.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.617+0000 7f10e2b08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f10dc072d80 con 0x7f10dc0b3dd0 2024-04-14T08:33:16.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.617+0000 7f10dbfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f10dc06fdf0 0x7f10dc15e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35770/0 (socket says 172.21.15.172:35770) 2024-04-14T08:33:16.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.617+0000 7f10dbfff640 1 -- 172.21.15.172:0/989608523 learned_addr learned my addr 172.21.15.172:0/989608523 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:16.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.617+0000 7f10e10a5640 1 --2- 172.21.15.172:0/989608523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10dc0b3dd0 0x7f10dc15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.617+0000 7f10db7fe640 1 -- 172.21.15.172:0/989608523 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2939168813 0 0) 0x7f10dc070270 con 0x7f10dc06fdf0 2024-04-14T08:33:16.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.617+0000 7f10db7fe640 1 -- 172.21.15.172:0/989608523 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10c4003520 con 0x7f10dc06fdf0 2024-04-14T08:33:16.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.617+0000 7f10db7fe640 1 -- 172.21.15.172:0/989608523 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 509644347 0 0) 0x7f10dc0b4540 con 0x7f10dc06b520 2024-04-14T08:33:16.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.617+0000 7f10db7fe640 1 -- 172.21.15.172:0/989608523 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10dc070270 con 0x7f10dc06b520 2024-04-14T08:33:16.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.618+0000 7f10e10a5640 1 -- 172.21.15.172:0/989608523 >> v1:172.21.15.177:6789/0 conn(0x7f10dc06b520 legacy=0x7f10dc06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.618+0000 7f10e10a5640 1 -- 172.21.15.172:0/989608523 >> v1:172.21.15.172:6789/0 conn(0x7f10dc06fdf0 legacy=0x7f10dc15e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.618+0000 7f10e10a5640 1 -- 172.21.15.172:0/989608523 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc072450 con 0x7f10dc0b3dd0 2024-04-14T08:33:16.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.618+0000 7f10e10a5640 1 --2- 172.21.15.172:0/989608523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10dc0b3dd0 0x7f10dc15efa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009dc0 tx=0x7f10d402bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6627afadd6969f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:16.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.618+0000 7f10db7fe640 1 -- 172.21.15.172:0/989608523 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4039070 con 0x7f10dc0b3dd0 2024-04-14T08:33:16.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.618+0000 7f10db7fe640 1 -- 172.21.15.172:0/989608523 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d402ccd0 con 0x7f10dc0b3dd0 2024-04-14T08:33:16.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.618+0000 7f10db7fe640 1 -- 172.21.15.172:0/989608523 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4034640 con 0x7f10dc0b3dd0 2024-04-14T08:33:16.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.619+0000 7f10e2b08640 1 -- 172.21.15.172:0/989608523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10dc0b3dd0 msgr2=0x7f10dc15efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.619+0000 7f10e2b08640 1 --2- 172.21.15.172:0/989608523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10dc0b3dd0 0x7f10dc15efa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009dc0 tx=0x7f10d402bf20 comp rx=0 tx=0).stop 2024-04-14T08:33:16.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.619+0000 7f10e2b08640 1 -- 172.21.15.172:0/989608523 shutdown_connections 2024-04-14T08:33:16.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.619+0000 7f10e2b08640 1 --2- 172.21.15.172:0/989608523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10dc0b3dd0 0x7f10dc15efa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.619+0000 7f10e2b08640 1 -- 172.21.15.172:0/989608523 >> 172.21.15.172:0/989608523 conn(0x7f10dc0d0f20 msgr2=0x7f10dc0d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:16.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.619+0000 7f10e2b08640 1 -- 172.21.15.172:0/989608523 shutdown_connections 2024-04-14T08:33:16.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.619+0000 7f10e2b08640 1 -- 172.21.15.172:0/989608523 wait complete. 2024-04-14T08:33:16.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.620+0000 7f10e2b08640 1 Processor -- start 2024-04-14T08:33:16.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.620+0000 7f10e2b08640 1 -- start start 2024-04-14T08:33:16.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e2b08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10dc06b520 0x7f10dc1459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e2b08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10dc06fdf0 0x7f10dc161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e2b08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10dc0b3dd0 0x7f10dc15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e2b08640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc0b4260 con 0x7f10dc06b520 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e2b08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc072a10 con 0x7f10dc0b3dd0 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e2b08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f10dc073210 con 0x7f10dc06fdf0 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e08a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10dc06b520 0x7f10dc1459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e08a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10dc06b520 0x7f10dc1459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49768/0 (socket says 172.21.15.172:49768) 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e08a4640 1 -- 172.21.15.172:0/97703553 learned_addr learned my addr 172.21.15.172:0/97703553 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e10a5640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10dc0b3dd0 0x7f10dc15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10dbfff640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10dc06fdf0 0x7f10dc161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e08a4640 1 -- 172.21.15.172:0/97703553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10dc06fdf0 msgr2=0x7f10dc161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e08a4640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10dc06fdf0 0x7f10dc161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e08a4640 1 -- 172.21.15.172:0/97703553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10dc0b3dd0 msgr2=0x7f10dc15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e08a4640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10dc0b3dd0 0x7f10dc15aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e08a4640 1 -- 172.21.15.172:0/97703553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc161b00 con 0x7f10dc06b520 2024-04-14T08:33:16.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.621+0000 7f10e10a5640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10dc0b3dd0 0x7f10dc15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:16.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.622+0000 7f10e08a4640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10dc06b520 0x7f10dc1459f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00c3a0 tx=0x7f10cc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:16.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.622+0000 7f10d97fa640 1 -- 172.21.15.172:0/97703553 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc017070 con 0x7f10dc06b520 2024-04-14T08:33:16.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.622+0000 7f10d97fa640 1 -- 172.21.15.172:0/97703553 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10cc0052f0 con 0x7f10dc06b520 2024-04-14T08:33:16.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.622+0000 7f10e2b08640 1 -- 172.21.15.172:0/97703553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc161de0 con 0x7f10dc06b520 2024-04-14T08:33:16.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.622+0000 7f10d97fa640 1 -- 172.21.15.172:0/97703553 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc003830 con 0x7f10dc06b520 2024-04-14T08:33:16.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.622+0000 7f10e2b08640 1 -- 172.21.15.172:0/97703553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1623a0 con 0x7f10dc06b520 2024-04-14T08:33:16.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.623+0000 7f10e2b08640 1 -- 172.21.15.172:0/97703553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc068520 con 0x7f10dc06b520 2024-04-14T08:33:16.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.623+0000 7f10d97fa640 1 -- 172.21.15.172:0/97703553 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10cc01a070 con 0x7f10dc06b520 2024-04-14T08:33:16.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.624+0000 7f10d97fa640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10c806c8f0 0x7f10c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.624+0000 7f10d97fa640 1 -- 172.21.15.172:0/97703553 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f10cc08b110 con 0x7f10dc06b520 2024-04-14T08:33:16.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.624+0000 7f10dbfff640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10c806c8f0 0x7f10c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.624+0000 7f10dbfff640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10c806c8f0 0x7f10c806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f10dc152f80 tx=0x7f10d0007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:16.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.628+0000 7f10d97fa640 1 -- 172.21.15.172:0/97703553 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f10cc058bb0 con 0x7f10dc06b520 2024-04-14T08:33:16.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.803+0000 7f10e2b08640 1 -- 172.21.15.172:0/97703553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10dc06c3b0 con 0x7f10dc06b520 2024-04-14T08:33:16.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.805+0000 7f10d97fa640 1 -- 172.21.15.172:0/97703553 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7f10cc05ceb0 con 0x7f10dc06b520 2024-04-14T08:33:16.805 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:16.806 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:16.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.807+0000 7f10e2b08640 1 -- 172.21.15.172:0/97703553 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10c806c8f0 msgr2=0x7f10c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.807+0000 7f10e2b08640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10c806c8f0 0x7f10c806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f10dc152f80 tx=0x7f10d0007d30 comp rx=0 tx=0).stop 2024-04-14T08:33:16.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.807+0000 7f10e2b08640 1 -- 172.21.15.172:0/97703553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10dc06b520 msgr2=0x7f10dc1459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.807+0000 7f10e2b08640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10dc06b520 0x7f10dc1459f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00c3a0 tx=0x7f10cc00c860 comp rx=0 tx=0).stop 2024-04-14T08:33:16.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.807+0000 7f10e2b08640 1 -- 172.21.15.172:0/97703553 shutdown_connections 2024-04-14T08:33:16.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.807+0000 7f10e2b08640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10c806c8f0 0x7f10c806edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.807+0000 7f10e2b08640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10dc0b3dd0 0x7f10dc15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.807+0000 7f10e2b08640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10dc06fdf0 0x7f10dc161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.807+0000 7f10e2b08640 1 --2- 172.21.15.172:0/97703553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10dc06b520 0x7f10dc1459f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.807+0000 7f10e2b08640 1 -- 172.21.15.172:0/97703553 >> 172.21.15.172:0/97703553 conn(0x7f10dc0d0f20 msgr2=0x7f10dc153da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:16.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.807+0000 7f10e2b08640 1 -- 172.21.15.172:0/97703553 shutdown_connections 2024-04-14T08:33:16.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.807+0000 7f10e2b08640 1 -- 172.21.15.172:0/97703553 wait complete. 2024-04-14T08:33:16.819 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:16.819 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:16.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.922+0000 7fc05ba80640 1 Processor -- start 2024-04-14T08:33:16.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.923+0000 7fc05ba80640 1 -- start start 2024-04-14T08:33:16.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.923+0000 7fc05ba80640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc0540d0cc0 0x7fc0540d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.923+0000 7fc05ba80640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc0540d49c0 0x7fc0540d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.923+0000 7fc05ba80640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0540d15e0 con 0x7fc054160f30 2024-04-14T08:33:16.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.923+0000 7fc05ba80640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc0540ceec0 con 0x7fc0540d49c0 2024-04-14T08:33:16.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.923+0000 7fc05ba80640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc0540cfd00 con 0x7fc0540d0cc0 2024-04-14T08:33:16.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.923+0000 7fc05901b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc0540d49c0 0x7fc0540d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.924+0000 7fc05981c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc0540d0cc0 0x7fc0540d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.924+0000 7fc05a01d640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fc054160f30 0x7fc054161380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42802/0 (socket says 172.21.15.172:42802) 2024-04-14T08:33:16.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.924+0000 7fc05a01d640 1 -- 172.21.15.172:0/2321042471 learned_addr learned my addr 172.21.15.172:0/2321042471 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:16.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.924+0000 7fc05981c640 1 -- 172.21.15.172:0/2321042471 >> v2:172.21.15.172:3300/0 conn(0x7fc0540d49c0 msgr2=0x7fc0540d4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:16.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.924+0000 7fc05981c640 1 --2- 172.21.15.172:0/2321042471 >> v2:172.21.15.172:3300/0 conn(0x7fc0540d49c0 0x7fc0540d4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.924+0000 7fc05981c640 1 -- 172.21.15.172:0/2321042471 >> v1:172.21.15.177:6789/0 conn(0x7fc054160f30 legacy=0x7fc054161380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.924+0000 7fc05981c640 1 -- 172.21.15.172:0/2321042471 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0540cf0f0 con 0x7fc0540d0cc0 2024-04-14T08:33:16.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.924+0000 7fc05881a640 1 -- 172.21.15.172:0/2321042471 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538821920 0 0) 0x7fc0540d15e0 con 0x7fc054160f30 2024-04-14T08:33:16.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.925+0000 7fc05981c640 1 --2- 172.21.15.172:0/2321042471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0540d0cc0 0x7fc0540d10a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc05408e7b0 tx=0x7fc05002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=981d817891ca7de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:16.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.925+0000 7fc05881a640 1 -- 172.21.15.172:0/2321042471 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050039070 con 0x7fc0540d0cc0 2024-04-14T08:33:16.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.925+0000 7fc05881a640 1 -- 172.21.15.172:0/2321042471 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc05002cc80 con 0x7fc0540d0cc0 2024-04-14T08:33:16.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.925+0000 7fc05881a640 1 -- 172.21.15.172:0/2321042471 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050034660 con 0x7fc0540d0cc0 2024-04-14T08:33:16.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.925+0000 7fc05ba80640 1 -- 172.21.15.172:0/2321042471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0540d0cc0 msgr2=0x7fc0540d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.925+0000 7fc05ba80640 1 --2- 172.21.15.172:0/2321042471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0540d0cc0 0x7fc0540d10a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc05408e7b0 tx=0x7fc05002bed0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.925+0000 7fc05ba80640 1 -- 172.21.15.172:0/2321042471 shutdown_connections 2024-04-14T08:33:16.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.925+0000 7fc05ba80640 1 --2- 172.21.15.172:0/2321042471 >> v2:172.21.15.172:3300/0 conn(0x7fc0540d49c0 0x7fc0540d4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.925+0000 7fc05ba80640 1 --2- 172.21.15.172:0/2321042471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0540d0cc0 0x7fc0540d10a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.925+0000 7fc05ba80640 1 -- 172.21.15.172:0/2321042471 >> 172.21.15.172:0/2321042471 conn(0x7fc05409d580 msgr2=0x7fc0540d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:16.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.925+0000 7fc05ba80640 1 -- 172.21.15.172:0/2321042471 shutdown_connections 2024-04-14T08:33:16.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.925+0000 7fc05ba80640 1 -- 172.21.15.172:0/2321042471 wait complete. 2024-04-14T08:33:16.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.926+0000 7fc05ba80640 1 Processor -- start 2024-04-14T08:33:16.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.927+0000 7fc05ba80640 1 -- start start 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.927+0000 7fc05ba80640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0540d0cc0 0x7fc0540c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.927+0000 7fc05ba80640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0540d49c0 0x7fc05415fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.927+0000 7fc05ba80640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc054160f30 0x7fc05409dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.927+0000 7fc05ba80640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0540cf990 con 0x7fc0540d49c0 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.927+0000 7fc05ba80640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0540cf6b0 con 0x7fc054160f30 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.927+0000 7fc05ba80640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc0540d32b0 con 0x7fc0540d0cc0 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.927+0000 7fc05901b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0540d49c0 0x7fc05415fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.927+0000 7fc05901b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0540d49c0 0x7fc05415fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49790/0 (socket says 172.21.15.172:49790) 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.927+0000 7fc05901b640 1 -- 172.21.15.172:0/2230432164 learned_addr learned my addr 172.21.15.172:0/2230432164 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.927+0000 7fc05981c640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0540d0cc0 0x7fc0540c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.927+0000 7fc05a01d640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc054160f30 0x7fc05409dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.928+0000 7fc05901b640 1 -- 172.21.15.172:0/2230432164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0540d0cc0 msgr2=0x7fc0540c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.928+0000 7fc05901b640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0540d0cc0 0x7fc0540c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.928+0000 7fc05901b640 1 -- 172.21.15.172:0/2230432164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc054160f30 msgr2=0x7fc05409dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:16.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.928+0000 7fc05901b640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc054160f30 0x7fc05409dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:16.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.928+0000 7fc05901b640 1 -- 172.21.15.172:0/2230432164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05409e6f0 con 0x7fc0540d49c0 2024-04-14T08:33:16.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.928+0000 7fc05901b640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0540d49c0 0x7fc05415fe50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc04400e9e0 tx=0x7fc04400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:16.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.928+0000 7fc04a7fc640 1 -- 172.21.15.172:0/2230432164 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc044017070 con 0x7fc0540d49c0 2024-04-14T08:33:16.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.928+0000 7fc04a7fc640 1 -- 172.21.15.172:0/2230432164 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc04400c9d0 con 0x7fc0540d49c0 2024-04-14T08:33:16.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.928+0000 7fc05ba80640 1 -- 172.21.15.172:0/2230432164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05409e9d0 con 0x7fc0540d49c0 2024-04-14T08:33:16.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.928+0000 7fc04a7fc640 1 -- 172.21.15.172:0/2230432164 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04400cdc0 con 0x7fc0540d49c0 2024-04-14T08:33:16.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.928+0000 7fc05ba80640 1 -- 172.21.15.172:0/2230432164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05409ef90 con 0x7fc0540d49c0 2024-04-14T08:33:16.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.929+0000 7fc05ba80640 1 -- 172.21.15.172:0/2230432164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc054075490 con 0x7fc0540d49c0 2024-04-14T08:33:16.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.930+0000 7fc04a7fc640 1 -- 172.21.15.172:0/2230432164 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc04401a070 con 0x7fc0540d49c0 2024-04-14T08:33:16.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.930+0000 7fc04a7fc640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc03806c8f0 0x7fc03806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:16.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.930+0000 7fc04a7fc640 1 -- 172.21.15.172:0/2230432164 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc04408c180 con 0x7fc0540d49c0 2024-04-14T08:33:16.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.930+0000 7fc05981c640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc03806c8f0 0x7fc03806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:16.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.930+0000 7fc05981c640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc03806c8f0 0x7fc03806edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc050036000 tx=0x7fc050030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:16.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:16.934+0000 7fc04a7fc640 1 -- 172.21.15.172:0/2230432164 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc044059c20 con 0x7fc0540d49c0 2024-04-14T08:33:17.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.107+0000 7fc05ba80640 1 -- 172.21.15.172:0/2230432164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc0540c9d40 con 0x7fc0540d49c0 2024-04-14T08:33:17.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.108+0000 7fc04a7fc640 1 -- 172.21.15.172:0/2230432164 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fc04405df20 con 0x7fc0540d49c0 2024-04-14T08:33:17.109 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:17.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.110+0000 7fc05ba80640 1 -- 172.21.15.172:0/2230432164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc03806c8f0 msgr2=0x7fc03806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.110+0000 7fc05ba80640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc03806c8f0 0x7fc03806edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc050036000 tx=0x7fc050030000 comp rx=0 tx=0).stop 2024-04-14T08:33:17.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.110+0000 7fc05ba80640 1 -- 172.21.15.172:0/2230432164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0540d49c0 msgr2=0x7fc05415fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.110+0000 7fc05ba80640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0540d49c0 0x7fc05415fe50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc04400e9e0 tx=0x7fc04400eea0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.110+0000 7fc05ba80640 1 -- 172.21.15.172:0/2230432164 shutdown_connections 2024-04-14T08:33:17.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.110+0000 7fc05ba80640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc03806c8f0 0x7fc03806edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.110+0000 7fc05ba80640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc054160f30 0x7fc05409dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.110+0000 7fc05ba80640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0540d49c0 0x7fc05415fe50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.110+0000 7fc05ba80640 1 --2- 172.21.15.172:0/2230432164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0540d0cc0 0x7fc0540c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.110+0000 7fc05ba80640 1 -- 172.21.15.172:0/2230432164 >> 172.21.15.172:0/2230432164 conn(0x7fc05409d580 msgr2=0x7fc0540b9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:17.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.110+0000 7fc05ba80640 1 -- 172.21.15.172:0/2230432164 shutdown_connections 2024-04-14T08:33:17.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.110+0000 7fc05ba80640 1 -- 172.21.15.172:0/2230432164 wait complete. 2024-04-14T08:33:17.121 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:17.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.226+0000 7f24eb57b640 1 Processor -- start 2024-04-14T08:33:17.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.226+0000 7f24eb57b640 1 -- start start 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.227+0000 7f24eb57b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f24e40ccca0 0x7f24e406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.227+0000 7f24eb57b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f24e406bc30 0x7f24e40b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.227+0000 7f24eb57b640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24e40ceee0 con 0x7f24e4073090 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.227+0000 7f24eb57b640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f24e4073520 con 0x7f24e406bc30 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.227+0000 7f24eb57b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f24e4070150 con 0x7f24e40ccca0 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.227+0000 7f24e9b18640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f24e406bc30 0x7f24e40b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.227+0000 7f24e8b16640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f24e4073090 0x7f24e406b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35786/0 (socket says 172.21.15.172:35786) 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.227+0000 7f24e8b16640 1 -- 172.21.15.172:0/2940153687 learned_addr learned my addr 172.21.15.172:0/2940153687 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.227+0000 7f24e9317640 1 --2- 172.21.15.172:0/2940153687 >> v2:172.21.15.177:3300/0 conn(0x7f24e40ccca0 0x7f24e406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.228+0000 7f24e9b18640 1 -- 172.21.15.172:0/2940153687 >> v2:172.21.15.177:3300/0 conn(0x7f24e40ccca0 msgr2=0x7f24e406fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.228+0000 7f24e9b18640 1 --2- 172.21.15.172:0/2940153687 >> v2:172.21.15.177:3300/0 conn(0x7f24e40ccca0 0x7f24e406fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.228+0000 7f24e9b18640 1 -- 172.21.15.172:0/2940153687 >> v1:172.21.15.172:6789/0 conn(0x7f24e4073090 legacy=0x7f24e406b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.228+0000 7f24e9b18640 1 -- 172.21.15.172:0/2940153687 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e40722d0 con 0x7f24e406bc30 2024-04-14T08:33:17.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.228+0000 7f24dbfff640 1 -- 172.21.15.172:0/2940153687 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4130382320 0 0) 0x7f24e40ceee0 con 0x7f24e4073090 2024-04-14T08:33:17.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.228+0000 7f24e9b18640 1 --2- 172.21.15.172:0/2940153687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e406bc30 0x7f24e40b0190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24e4068ce0 tx=0x7f24dc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b056e4f1f048c606 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:17.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.228+0000 7f24dbfff640 1 -- 172.21.15.172:0/2940153687 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc0315a0 con 0x7f24e406bc30 2024-04-14T08:33:17.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.228+0000 7f24dbfff640 1 -- 172.21.15.172:0/2940153687 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24dc037070 con 0x7f24e406bc30 2024-04-14T08:33:17.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.228+0000 7f24dbfff640 1 -- 172.21.15.172:0/2940153687 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc031960 con 0x7f24e406bc30 2024-04-14T08:33:17.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.229+0000 7f24eb57b640 1 -- 172.21.15.172:0/2940153687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e406bc30 msgr2=0x7f24e40b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.229+0000 7f24eb57b640 1 --2- 172.21.15.172:0/2940153687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e406bc30 0x7f24e40b0190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24e4068ce0 tx=0x7f24dc02b840 comp rx=0 tx=0).stop 2024-04-14T08:33:17.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.229+0000 7f24eb57b640 1 -- 172.21.15.172:0/2940153687 shutdown_connections 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.229+0000 7f24eb57b640 1 --2- 172.21.15.172:0/2940153687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e406bc30 0x7f24e40b0190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.229+0000 7f24eb57b640 1 --2- 172.21.15.172:0/2940153687 >> v2:172.21.15.177:3300/0 conn(0x7f24e40ccca0 0x7f24e406fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.229+0000 7f24eb57b640 1 -- 172.21.15.172:0/2940153687 >> 172.21.15.172:0/2940153687 conn(0x7f24e40cda00 msgr2=0x7f24e40cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.229+0000 7f24eb57b640 1 -- 172.21.15.172:0/2940153687 shutdown_connections 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.229+0000 7f24eb57b640 1 -- 172.21.15.172:0/2940153687 wait complete. 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.230+0000 7f24eb57b640 1 Processor -- start 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.230+0000 7f24eb57b640 1 -- start start 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24eb57b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e406bc30 0x7f24e406c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24eb57b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24e4073090 0x7f24e415ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24eb57b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24e40ccca0 0x7f24e414b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24eb57b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f24e40cd080 con 0x7f24e406bc30 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24eb57b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4072890 con 0x7f24e4073090 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24eb57b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f24e406e150 con 0x7f24e40ccca0 2024-04-14T08:33:17.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24e9317640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e406bc30 0x7f24e406c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24e9317640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e406bc30 0x7f24e406c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49808/0 (socket says 172.21.15.172:49808) 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24e9317640 1 -- 172.21.15.172:0/3654445750 learned_addr learned my addr 172.21.15.172:0/3654445750 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24e9b18640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24e40ccca0 0x7f24e414b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24e8b16640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24e4073090 0x7f24e415ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24e9317640 1 -- 172.21.15.172:0/3654445750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24e40ccca0 msgr2=0x7f24e414b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24e9317640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24e40ccca0 0x7f24e414b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24e9317640 1 -- 172.21.15.172:0/3654445750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24e4073090 msgr2=0x7f24e415ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24e9317640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24e4073090 0x7f24e415ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.231+0000 7f24e9317640 1 -- 172.21.15.172:0/3654445750 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e415b710 con 0x7f24e406bc30 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.232+0000 7f24e9317640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e406bc30 0x7f24e406c5b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f24e000efc0 tx=0x7f24e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.232+0000 7f24d9ffb640 1 -- 172.21.15.172:0/3654445750 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0018020 con 0x7f24e406bc30 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.232+0000 7f24d9ffb640 1 -- 172.21.15.172:0/3654445750 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e0014480 con 0x7f24e406bc30 2024-04-14T08:33:17.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.232+0000 7f24eb57b640 1 -- 172.21.15.172:0/3654445750 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e415b9f0 con 0x7f24e406bc30 2024-04-14T08:33:17.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.232+0000 7f24d9ffb640 1 -- 172.21.15.172:0/3654445750 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0014850 con 0x7f24e406bc30 2024-04-14T08:33:17.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.232+0000 7f24eb57b640 1 -- 172.21.15.172:0/3654445750 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e406caf0 con 0x7f24e406bc30 2024-04-14T08:33:17.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.233+0000 7f24eb57b640 1 -- 172.21.15.172:0/3654445750 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e4068590 con 0x7f24e406bc30 2024-04-14T08:33:17.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.233+0000 7f24d9ffb640 1 -- 172.21.15.172:0/3654445750 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24e0005d10 con 0x7f24e406bc30 2024-04-14T08:33:17.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.234+0000 7f24d9ffb640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f24c806c8f0 0x7f24c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.234+0000 7f24d9ffb640 1 -- 172.21.15.172:0/3654445750 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f24e008cad0 con 0x7f24e406bc30 2024-04-14T08:33:17.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.234+0000 7f24e8b16640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f24c806c8f0 0x7f24c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.234+0000 7f24e8b16640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f24c806c8f0 0x7f24c806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f24e40cc060 tx=0x7f24d4007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:17.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.238+0000 7f24d9ffb640 1 -- 172.21.15.172:0/3654445750 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f24e0022030 con 0x7f24e406bc30 2024-04-14T08:33:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.413+0000 7f24eb57b640 1 -- 172.21.15.172:0/3654445750 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f24e4074410 con 0x7f24e406bc30 2024-04-14T08:33:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.414+0000 7f24d9ffb640 1 -- 172.21.15.172:0/3654445750 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f24e0010040 con 0x7f24e406bc30 2024-04-14T08:33:17.414 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.415+0000 7f24eb57b640 1 -- 172.21.15.172:0/3654445750 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f24c806c8f0 msgr2=0x7f24c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.415+0000 7f24eb57b640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f24c806c8f0 0x7f24c806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f24e40cc060 tx=0x7f24d4007d50 comp rx=0 tx=0).stop 2024-04-14T08:33:17.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.415+0000 7f24eb57b640 1 -- 172.21.15.172:0/3654445750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e406bc30 msgr2=0x7f24e406c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.415+0000 7f24eb57b640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e406bc30 0x7f24e406c5b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f24e000efc0 tx=0x7f24e000c490 comp rx=0 tx=0).stop 2024-04-14T08:33:17.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.416+0000 7f24eb57b640 1 -- 172.21.15.172:0/3654445750 shutdown_connections 2024-04-14T08:33:17.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.416+0000 7f24eb57b640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f24c806c8f0 0x7f24c806edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.416+0000 7f24eb57b640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24e40ccca0 0x7f24e414b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.416+0000 7f24eb57b640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24e4073090 0x7f24e415ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.416+0000 7f24eb57b640 1 --2- 172.21.15.172:0/3654445750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e406bc30 0x7f24e406c5b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.416+0000 7f24eb57b640 1 -- 172.21.15.172:0/3654445750 >> 172.21.15.172:0/3654445750 conn(0x7f24e40cda00 msgr2=0x7f24e40b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:17.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.416+0000 7f24eb57b640 1 -- 172.21.15.172:0/3654445750 shutdown_connections 2024-04-14T08:33:17.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.416+0000 7f24eb57b640 1 -- 172.21.15.172:0/3654445750 wait complete. 2024-04-14T08:33:17.427 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:17.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.530+0000 7f76e91dc640 1 Processor -- start 2024-04-14T08:33:17.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.530+0000 7f76e91dc640 1 -- start start 2024-04-14T08:33:17.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e91dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76e406bc30 0x7f76e406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e91dc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f76e4073090 0x7f76e40b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e91dc640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76e40ceee0 con 0x7f76e40ccca0 2024-04-14T08:33:17.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e91dc640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f76e40734a0 con 0x7f76e4073090 2024-04-14T08:33:17.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e91dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f76e4070150 con 0x7f76e406bc30 2024-04-14T08:33:17.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e359e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f76e4073090 0x7f76e40b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e259c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76e406bc30 0x7f76e406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e359e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f76e4073090 0x7f76e40b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33398/0 (socket says 172.21.15.172:33398) 2024-04-14T08:33:17.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e2d9d640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f76e40ccca0 0x7f76e406b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42810/0 (socket says 172.21.15.172:42810) 2024-04-14T08:33:17.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e259c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76e406bc30 0x7f76e406fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:43408/0 (socket says 172.21.15.172:43408) 2024-04-14T08:33:17.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e359e640 1 -- 172.21.15.172:0/1795669536 learned_addr learned my addr 172.21.15.172:0/1795669536 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:17.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e359e640 1 -- 172.21.15.172:0/1795669536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76e406bc30 msgr2=0x7f76e406fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:17.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e359e640 1 --2- 172.21.15.172:0/1795669536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76e406bc30 0x7f76e406fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e359e640 1 -- 172.21.15.172:0/1795669536 >> v1:172.21.15.177:6789/0 conn(0x7f76e40ccca0 legacy=0x7f76e406b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e1d9b640 1 -- 172.21.15.172:0/1795669536 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 395725604 0 0) 0x7f76e40ceee0 con 0x7f76e40ccca0 2024-04-14T08:33:17.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e359e640 1 -- 172.21.15.172:0/1795669536 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e40722d0 con 0x7f76e4073090 2024-04-14T08:33:17.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.531+0000 7f76e259c640 1 --2- 172.21.15.172:0/1795669536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76e406bc30 0x7f76e406fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:17.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.532+0000 7f76e359e640 1 --2- 172.21.15.172:0/1795669536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76e4073090 0x7f76e40b0190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009e50 tx=0x7f76dc02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e46bf78f46ea4af server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:17.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.532+0000 7f76e1d9b640 1 -- 172.21.15.172:0/1795669536 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc02ca70 con 0x7f76e4073090 2024-04-14T08:33:17.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.532+0000 7f76e1d9b640 1 -- 172.21.15.172:0/1795669536 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76dc02cc10 con 0x7f76e4073090 2024-04-14T08:33:17.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.532+0000 7f76e1d9b640 1 -- 172.21.15.172:0/1795669536 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc0343f0 con 0x7f76e4073090 2024-04-14T08:33:17.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.533+0000 7f76e91dc640 1 -- 172.21.15.172:0/1795669536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76e4073090 msgr2=0x7f76e40b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.533+0000 7f76e91dc640 1 --2- 172.21.15.172:0/1795669536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76e4073090 0x7f76e40b0190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009e50 tx=0x7f76dc02bed0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.533+0000 7f76e91dc640 1 -- 172.21.15.172:0/1795669536 shutdown_connections 2024-04-14T08:33:17.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.533+0000 7f76e91dc640 1 --2- 172.21.15.172:0/1795669536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76e4073090 0x7f76e40b0190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.533+0000 7f76e91dc640 1 --2- 172.21.15.172:0/1795669536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76e406bc30 0x7f76e406fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.533+0000 7f76e91dc640 1 -- 172.21.15.172:0/1795669536 >> 172.21.15.172:0/1795669536 conn(0x7f76e40cda00 msgr2=0x7f76e40cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:17.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.533+0000 7f76e91dc640 1 -- 172.21.15.172:0/1795669536 shutdown_connections 2024-04-14T08:33:17.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.533+0000 7f76e91dc640 1 -- 172.21.15.172:0/1795669536 wait complete. 2024-04-14T08:33:17.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.534+0000 7f76e91dc640 1 Processor -- start 2024-04-14T08:33:17.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.534+0000 7f76e91dc640 1 -- start start 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.535+0000 7f76e91dc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76e406bc30 0x7f76e40c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.535+0000 7f76e91dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76e4073090 0x7f76e414d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.535+0000 7f76e91dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76e40ccca0 0x7f76e40cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.535+0000 7f76e91dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f76e40cd080 con 0x7f76e40ccca0 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.535+0000 7f76e91dc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f76e4072890 con 0x7f76e406bc30 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.535+0000 7f76e91dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f76e406e150 con 0x7f76e4073090 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.535+0000 7f76e359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76e40ccca0 0x7f76e40cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.535+0000 7f76e2d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76e406bc30 0x7f76e40c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.535+0000 7f76e359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76e40ccca0 0x7f76e40cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49824/0 (socket says 172.21.15.172:49824) 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.535+0000 7f76e359e640 1 -- 172.21.15.172:0/2229667530 learned_addr learned my addr 172.21.15.172:0/2229667530 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.535+0000 7f76e259c640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76e4073090 0x7f76e414d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.536+0000 7f76e359e640 1 -- 172.21.15.172:0/2229667530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76e4073090 msgr2=0x7f76e414d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.536+0000 7f76e359e640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76e4073090 0x7f76e414d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.536+0000 7f76e359e640 1 -- 172.21.15.172:0/2229667530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76e406bc30 msgr2=0x7f76e40c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.536+0000 7f76e359e640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76e406bc30 0x7f76e40c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.536+0000 7f76e359e640 1 -- 172.21.15.172:0/2229667530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e40c89d0 con 0x7f76e40ccca0 2024-04-14T08:33:17.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.536+0000 7f76e2d9d640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76e406bc30 0x7f76e40c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:17.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.536+0000 7f76e359e640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76e40ccca0 0x7f76e40cae80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009e50 tx=0x7f76dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:17.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.536+0000 7f76cb7fe640 1 -- 172.21.15.172:0/2229667530 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc043030 con 0x7f76e40ccca0 2024-04-14T08:33:17.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.536+0000 7f76cb7fe640 1 -- 172.21.15.172:0/2229667530 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76dc03e450 con 0x7f76e40ccca0 2024-04-14T08:33:17.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.536+0000 7f76e91dc640 1 -- 172.21.15.172:0/2229667530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e414e0f0 con 0x7f76e40ccca0 2024-04-14T08:33:17.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.536+0000 7f76cb7fe640 1 -- 172.21.15.172:0/2229667530 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc009510 con 0x7f76e40ccca0 2024-04-14T08:33:17.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.536+0000 7f76e91dc640 1 -- 172.21.15.172:0/2229667530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e414e670 con 0x7f76e40ccca0 2024-04-14T08:33:17.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.537+0000 7f76e91dc640 1 -- 172.21.15.172:0/2229667530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e4068590 con 0x7f76e40ccca0 2024-04-14T08:33:17.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.537+0000 7f76cb7fe640 1 -- 172.21.15.172:0/2229667530 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76dc046070 con 0x7f76e40ccca0 2024-04-14T08:33:17.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.538+0000 7f76cb7fe640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f76d006c8f0 0x7f76d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.538+0000 7f76cb7fe640 1 -- 172.21.15.172:0/2229667530 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f76dc0adfe0 con 0x7f76e40ccca0 2024-04-14T08:33:17.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.538+0000 7f76e2d9d640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f76d006c8f0 0x7f76d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.538+0000 7f76e2d9d640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f76d006c8f0 0x7f76d006edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f76d4007fd0 tx=0x7f76d4007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:17.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.542+0000 7f76cb7fe640 1 -- 172.21.15.172:0/2229667530 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f76dc04d070 con 0x7f76e40ccca0 2024-04-14T08:33:17.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.715+0000 7f76e91dc640 1 -- 172.21.15.172:0/2229667530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f76e406b6c0 con 0x7f76e40ccca0 2024-04-14T08:33:17.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.716+0000 7f76cb7fe640 1 -- 172.21.15.172:0/2229667530 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f76dc07bb10 con 0x7f76e40ccca0 2024-04-14T08:33:17.716 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.718+0000 7f76e91dc640 1 -- 172.21.15.172:0/2229667530 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f76d006c8f0 msgr2=0x7f76d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.718+0000 7f76e91dc640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f76d006c8f0 0x7f76d006edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f76d4007fd0 tx=0x7f76d4007de0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.718+0000 7f76e91dc640 1 -- 172.21.15.172:0/2229667530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76e40ccca0 msgr2=0x7f76e40cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.718+0000 7f76e91dc640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76e40ccca0 0x7f76e40cae80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009e50 tx=0x7f76dc004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.718+0000 7f76e91dc640 1 -- 172.21.15.172:0/2229667530 shutdown_connections 2024-04-14T08:33:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.718+0000 7f76e91dc640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f76d006c8f0 0x7f76d006edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.718+0000 7f76e91dc640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76e40ccca0 0x7f76e40cae80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.718+0000 7f76e91dc640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76e4073090 0x7f76e414d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.718+0000 7f76e91dc640 1 --2- 172.21.15.172:0/2229667530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76e406bc30 0x7f76e40c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.718+0000 7f76e91dc640 1 -- 172.21.15.172:0/2229667530 >> 172.21.15.172:0/2229667530 conn(0x7f76e40cda00 msgr2=0x7f76e406c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.718+0000 7f76e91dc640 1 -- 172.21.15.172:0/2229667530 shutdown_connections 2024-04-14T08:33:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.718+0000 7f76e91dc640 1 -- 172.21.15.172:0/2229667530 wait complete. 2024-04-14T08:33:17.730 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:17.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.833+0000 7f8b92436640 1 Processor -- start 2024-04-14T08:33:17.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.833+0000 7f8b92436640 1 -- start start 2024-04-14T08:33:17.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b92436640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8b8c159c20 0x7f8b8c0ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b92436640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8b8c15c700 0x7f8b8c1603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b92436640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b8c15cf10 con 0x7f8b8c160930 2024-04-14T08:33:17.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b92436640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f8b8c15fc80 con 0x7f8b8c159c20 2024-04-14T08:33:17.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b92436640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f8b8c15ff60 con 0x7f8b8c15c700 2024-04-14T08:33:17.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b8bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8b8c159c20 0x7f8b8c0ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b8bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8b8c159c20 0x7f8b8c0ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49826/0 (socket says 172.21.15.172:49826) 2024-04-14T08:33:17.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b909d3640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f8b8c160930 0x7f8b8c072450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35802/0 (socket says 172.21.15.172:35802) 2024-04-14T08:33:17.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b8bfff640 1 -- 172.21.15.172:0/3341259560 learned_addr learned my addr 172.21.15.172:0/3341259560 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:17.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b8b7fe640 1 --2- 172.21.15.172:0/3341259560 >> v2:172.21.15.177:3300/0 conn(0x7f8b8c15c700 0x7f8b8c1603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b8bfff640 1 -- 172.21.15.172:0/3341259560 >> v2:172.21.15.177:3300/0 conn(0x7f8b8c15c700 msgr2=0x7f8b8c1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:17.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b8bfff640 1 --2- 172.21.15.172:0/3341259560 >> v2:172.21.15.177:3300/0 conn(0x7f8b8c15c700 0x7f8b8c1603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b8bfff640 1 -- 172.21.15.172:0/3341259560 >> v1:172.21.15.172:6789/0 conn(0x7f8b8c160930 legacy=0x7f8b8c072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.834+0000 7f8b8bfff640 1 -- 172.21.15.172:0/3341259560 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c15d1f0 con 0x7f8b8c159c20 2024-04-14T08:33:17.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.835+0000 7f8b8bfff640 1 --2- 172.21.15.172:0/3341259560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b8c159c20 0x7f8b8c0ce200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c072190 tx=0x7f8b7c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9083559dfac8f973 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:17.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.835+0000 7f8b8affd640 1 -- 172.21.15.172:0/3341259560 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c034070 con 0x7f8b8c159c20 2024-04-14T08:33:17.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.835+0000 7f8b8affd640 1 -- 172.21.15.172:0/3341259560 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b7c030860 con 0x7f8b8c159c20 2024-04-14T08:33:17.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.835+0000 7f8b8affd640 1 -- 172.21.15.172:0/3341259560 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c030c30 con 0x7f8b8c159c20 2024-04-14T08:33:17.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.836+0000 7f8b92436640 1 -- 172.21.15.172:0/3341259560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b8c159c20 msgr2=0x7f8b8c0ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.836+0000 7f8b92436640 1 --2- 172.21.15.172:0/3341259560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b8c159c20 0x7f8b8c0ce200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c072190 tx=0x7f8b7c02b9a0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.836+0000 7f8b92436640 1 -- 172.21.15.172:0/3341259560 shutdown_connections 2024-04-14T08:33:17.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.836+0000 7f8b92436640 1 --2- 172.21.15.172:0/3341259560 >> v2:172.21.15.177:3300/0 conn(0x7f8b8c15c700 0x7f8b8c1603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.836+0000 7f8b92436640 1 --2- 172.21.15.172:0/3341259560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b8c159c20 0x7f8b8c0ce200 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.836+0000 7f8b92436640 1 -- 172.21.15.172:0/3341259560 >> 172.21.15.172:0/3341259560 conn(0x7f8b8c06b520 msgr2=0x7f8b8c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.836+0000 7f8b92436640 1 -- 172.21.15.172:0/3341259560 shutdown_connections 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.836+0000 7f8b92436640 1 -- 172.21.15.172:0/3341259560 wait complete. 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.837+0000 7f8b92436640 1 Processor -- start 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.837+0000 7f8b92436640 1 -- start start 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b92436640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b8c159c20 0x7f8b8c140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b92436640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b8c15c700 0x7f8b8c140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b92436640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b8c152260 0x7f8b8c1526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b92436640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c15cb80 con 0x7f8b8c15c700 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b92436640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c15a000 con 0x7f8b8c152260 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b92436640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8b8c072b60 con 0x7f8b8c159c20 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b8b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b8c15c700 0x7f8b8c140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b8b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b8c15c700 0x7f8b8c140200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49828/0 (socket says 172.21.15.172:49828) 2024-04-14T08:33:17.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b8b7fe640 1 -- 172.21.15.172:0/604194103 learned_addr learned my addr 172.21.15.172:0/604194103 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b8bfff640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b8c159c20 0x7f8b8c140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b909d3640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b8c152260 0x7f8b8c1526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b8b7fe640 1 -- 172.21.15.172:0/604194103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b8c159c20 msgr2=0x7f8b8c140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b8b7fe640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b8c159c20 0x7f8b8c140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b8b7fe640 1 -- 172.21.15.172:0/604194103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b8c152260 msgr2=0x7f8b8c1526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b8b7fe640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b8c152260 0x7f8b8c1526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b8b7fe640 1 -- 172.21.15.172:0/604194103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c155350 con 0x7f8b8c15c700 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b8b7fe640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b8c15c700 0x7f8b8c140200 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000e9b0 tx=0x7f8b8000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b88ff9640 1 -- 172.21.15.172:0/604194103 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80017070 con 0x7f8b8c15c700 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b88ff9640 1 -- 172.21.15.172:0/604194103 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b8000c9a0 con 0x7f8b8c15c700 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b92436640 1 -- 172.21.15.172:0/604194103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8c0b0fb0 con 0x7f8b8c15c700 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b88ff9640 1 -- 172.21.15.172:0/604194103 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000cd90 con 0x7f8b8c15c700 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.838+0000 7f8b92436640 1 -- 172.21.15.172:0/604194103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8c1545d0 con 0x7f8b8c15c700 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.839+0000 7f8b92436640 1 -- 172.21.15.172:0/604194103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b8c068590 con 0x7f8b8c15c700 2024-04-14T08:33:17.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.840+0000 7f8b88ff9640 1 -- 172.21.15.172:0/604194103 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b8001a070 con 0x7f8b8c15c700 2024-04-14T08:33:17.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.840+0000 7f8b88ff9640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8b7806c8f0 0x7f8b7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:17.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.840+0000 7f8b88ff9640 1 -- 172.21.15.172:0/604194103 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8b8008c170 con 0x7f8b8c15c700 2024-04-14T08:33:17.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.840+0000 7f8b8bfff640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8b7806c8f0 0x7f8b7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:17.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.841+0000 7f8b8bfff640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8b7806c8f0 0x7f8b7806edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c036000 tx=0x7f8b7c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:17.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:17.845+0000 7f8b88ff9640 1 -- 172.21.15.172:0/604194103 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8b80059c10 con 0x7f8b8c15c700 2024-04-14T08:33:18.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.019+0000 7f8b92436640 1 -- 172.21.15.172:0/604194103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8b8c15f990 con 0x7f8b8c15c700 2024-04-14T08:33:18.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.019+0000 7f8b88ff9640 1 -- 172.21.15.172:0/604194103 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f8b8005df10 con 0x7f8b8c15c700 2024-04-14T08:33:18.020 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:18.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.022+0000 7f8b92436640 1 -- 172.21.15.172:0/604194103 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8b7806c8f0 msgr2=0x7f8b7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.022+0000 7f8b92436640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8b7806c8f0 0x7f8b7806edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c036000 tx=0x7f8b7c008030 comp rx=0 tx=0).stop 2024-04-14T08:33:18.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.022+0000 7f8b92436640 1 -- 172.21.15.172:0/604194103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b8c15c700 msgr2=0x7f8b8c140200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.022+0000 7f8b92436640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b8c15c700 0x7f8b8c140200 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000e9b0 tx=0x7f8b8000ee70 comp rx=0 tx=0).stop 2024-04-14T08:33:18.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.022+0000 7f8b92436640 1 -- 172.21.15.172:0/604194103 shutdown_connections 2024-04-14T08:33:18.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.022+0000 7f8b92436640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8b7806c8f0 0x7f8b7806edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.022+0000 7f8b92436640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b8c152260 0x7f8b8c1526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.022+0000 7f8b92436640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b8c15c700 0x7f8b8c140200 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.022+0000 7f8b92436640 1 --2- 172.21.15.172:0/604194103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b8c159c20 0x7f8b8c140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.022+0000 7f8b92436640 1 -- 172.21.15.172:0/604194103 >> 172.21.15.172:0/604194103 conn(0x7f8b8c06b520 msgr2=0x7f8b8c15eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:18.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.022+0000 7f8b92436640 1 -- 172.21.15.172:0/604194103 shutdown_connections 2024-04-14T08:33:18.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.022+0000 7f8b92436640 1 -- 172.21.15.172:0/604194103 wait complete. 2024-04-14T08:33:18.033 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:33:18.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.133+0000 7fb2ceb5e640 1 Processor -- start 2024-04-14T08:33:18.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.133+0000 7fb2ceb5e640 1 -- start start 2024-04-14T08:33:18.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.134+0000 7fb2ceb5e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb2c815a690 0x7fb2c8071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.134+0000 7fb2ceb5e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c815f260 con 0x7fb2c815c240 2024-04-14T08:33:18.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.134+0000 7fb2ceb5e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c80b3360 con 0x7fb2c8072970 2024-04-14T08:33:18.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.134+0000 7fb2ceb5e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb2c80b3640 con 0x7fb2c815a690 2024-04-14T08:33:18.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.134+0000 7fb2cd0fb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb2c815c240 0x7fb2c815eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35816/0 (socket says 172.21.15.172:35816) 2024-04-14T08:33:18.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.134+0000 7fb2cd0fb640 1 -- 172.21.15.172:0/2595132068 learned_addr learned my addr 172.21.15.172:0/2595132068 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:18.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.134+0000 7fb2cc8fa640 1 --2- 172.21.15.172:0/2595132068 >> v2:172.21.15.177:3300/0 conn(0x7fb2c815a690 0x7fb2c8071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.134+0000 7fb2c77fe640 1 -- 172.21.15.172:0/2595132068 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1414757647 0 0) 0x7fb2c815f260 con 0x7fb2c815c240 2024-04-14T08:33:18.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.134+0000 7fb2c77fe640 1 -- 172.21.15.172:0/2595132068 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2b0003520 con 0x7fb2c815c240 2024-04-14T08:33:18.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.134+0000 7fb2cc8fa640 1 -- 172.21.15.172:0/2595132068 >> v1:172.21.15.177:6789/0 conn(0x7fb2c8072970 legacy=0x7fb2c815e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.134+0000 7fb2c77fe640 1 -- 172.21.15.172:0/2595132068 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4215175582 0 0) 0x7fb2c80b3360 con 0x7fb2c8072970 2024-04-14T08:33:18.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.134+0000 7fb2cc8fa640 1 -- 172.21.15.172:0/2595132068 >> v1:172.21.15.172:6789/0 conn(0x7fb2c815c240 legacy=0x7fb2c815eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.135+0000 7fb2cc8fa640 1 -- 172.21.15.172:0/2595132068 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c80b3920 con 0x7fb2c815a690 2024-04-14T08:33:18.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.135+0000 7fb2cc8fa640 1 --2- 172.21.15.172:0/2595132068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c815a690 0x7fb2c8071d60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80097a0 tx=0x7fb2b802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2a7cde0a2debc8d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.135+0000 7fb2c77fe640 1 -- 172.21.15.172:0/2595132068 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8035020 con 0x7fb2c815a690 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.136+0000 7fb2c77fe640 1 -- 172.21.15.172:0/2595132068 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b8031830 con 0x7fb2c815a690 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.136+0000 7fb2c77fe640 1 -- 172.21.15.172:0/2595132068 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8031c00 con 0x7fb2c815a690 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.136+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2595132068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c815a690 msgr2=0x7fb2c8071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.136+0000 7fb2ceb5e640 1 --2- 172.21.15.172:0/2595132068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c815a690 0x7fb2c8071d60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80097a0 tx=0x7fb2b802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.136+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2595132068 shutdown_connections 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.136+0000 7fb2ceb5e640 1 --2- 172.21.15.172:0/2595132068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c815a690 0x7fb2c8071d60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.136+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2595132068 >> 172.21.15.172:0/2595132068 conn(0x7fb2c806ec30 msgr2=0x7fb2c806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.136+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2595132068 shutdown_connections 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.136+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2595132068 wait complete. 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.137+0000 7fb2ceb5e640 1 Processor -- start 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.137+0000 7fb2ceb5e640 1 -- start start 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2ceb5e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c8072970 0x7fb2c81f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2ceb5e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2c815a690 0x7fb2c80cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2ceb5e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2c815c240 0x7fb2c81f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2ceb5e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c80b3070 con 0x7fb2c815c240 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2ceb5e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c80b2ef0 con 0x7fb2c8072970 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2ceb5e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c815aa70 con 0x7fb2c815a690 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2cd0fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2c815c240 0x7fb2c81f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2cd0fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2c815c240 0x7fb2c81f5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49838/0 (socket says 172.21.15.172:49838) 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2cd0fb640 1 -- 172.21.15.172:0/2245673149 learned_addr learned my addr 172.21.15.172:0/2245673149 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2cc8fa640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c8072970 0x7fb2c81f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2c7fff640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2c815a690 0x7fb2c80cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2cd0fb640 1 -- 172.21.15.172:0/2245673149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2c815a690 msgr2=0x7fb2c80cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2cd0fb640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2c815a690 0x7fb2c80cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2cd0fb640 1 -- 172.21.15.172:0/2245673149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c8072970 msgr2=0x7fb2c81f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2cd0fb640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c8072970 0x7fb2c81f2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2cd0fb640 1 -- 172.21.15.172:0/2245673149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c81f1f90 con 0x7fb2c815c240 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.138+0000 7fb2cc8fa640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c8072970 0x7fb2c81f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.139+0000 7fb2cd0fb640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2c815c240 0x7fb2c81f5580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000ca70 tx=0x7fb2c000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.139+0000 7fb2c57fa640 1 -- 172.21.15.172:0/2245673149 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0017070 con 0x7fb2c815c240 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.139+0000 7fb2c57fa640 1 -- 172.21.15.172:0/2245673149 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c000abd0 con 0x7fb2c815c240 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.139+0000 7fb2c57fa640 1 -- 172.21.15.172:0/2245673149 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0012400 con 0x7fb2c815c240 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.139+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2245673149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c81f6000 con 0x7fb2c815c240 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.139+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2245673149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c81f2250 con 0x7fb2c815c240 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.140+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2245673149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c8068520 con 0x7fb2c815c240 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.140+0000 7fb2c57fa640 1 -- 172.21.15.172:0/2245673149 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2c001a070 con 0x7fb2c815c240 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.140+0000 7fb2c57fa640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb2b406c8f0 0x7fb2b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.141+0000 7fb2c57fa640 1 -- 172.21.15.172:0/2245673149 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb2c008ba40 con 0x7fb2c815c240 2024-04-14T08:33:18.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.141+0000 7fb2cc8fa640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb2b406c8f0 0x7fb2b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.141+0000 7fb2cc8fa640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb2b406c8f0 0x7fb2b406edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8036000 tx=0x7fb2b803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:18.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.145+0000 7fb2c57fa640 1 -- 172.21.15.172:0/2245673149 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb2c0059570 con 0x7fb2c815c240 2024-04-14T08:33:18.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.362+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2245673149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2c80724a0 con 0x7fb2c815c240 2024-04-14T08:33:18.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.362+0000 7fb2c57fa640 1 -- 172.21.15.172:0/2245673149 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7fb2c008be20 con 0x7fb2c815c240 2024-04-14T08:33:18.363 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:18.363 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3053874257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3053874257},{"type":"v1","addr":"172.21.15.172:6835","nonce":3053874257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2790588849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2790588849},{"type":"v1","addr":"172.21.15.172:6837","nonce":2790588849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3130776573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3130776573},{"type":"v1","addr":"172.21.15.177:6834","nonce":3130776573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:32:53.535483+0000","modified":"2024-04-14T08:33:09.604218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/517963877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":517963877},{"type":"v1","addr":"172.21.15.177:6835","nonce":517963877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4309,"qdb_cluster":[4309]},"id":1}]} 2024-04-14T08:33:18.363 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 9 2024-04-14T08:33:18.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.364+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2245673149 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb2b406c8f0 msgr2=0x7fb2b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.364+0000 7fb2ceb5e640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb2b406c8f0 0x7fb2b406edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8036000 tx=0x7fb2b803d040 comp rx=0 tx=0).stop 2024-04-14T08:33:18.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.364+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2245673149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2c815c240 msgr2=0x7fb2c81f5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.364+0000 7fb2ceb5e640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2c815c240 0x7fb2c81f5580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000ca70 tx=0x7fb2c000cf30 comp rx=0 tx=0).stop 2024-04-14T08:33:18.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.364+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2245673149 shutdown_connections 2024-04-14T08:33:18.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.364+0000 7fb2ceb5e640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb2b406c8f0 0x7fb2b406edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.364+0000 7fb2ceb5e640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2c815c240 0x7fb2c81f5580 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.364+0000 7fb2ceb5e640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2c815a690 0x7fb2c80cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.364+0000 7fb2ceb5e640 1 --2- 172.21.15.172:0/2245673149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2c8072970 0x7fb2c81f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.364+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2245673149 >> 172.21.15.172:0/2245673149 conn(0x7fb2c806ec30 msgr2=0x7fb2c8153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:18.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.365+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2245673149 shutdown_connections 2024-04-14T08:33:18.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.365+0000 7fb2ceb5e640 1 -- 172.21.15.172:0/2245673149 wait complete. 2024-04-14T08:33:18.376 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:18.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.476+0000 7f5e477fe640 1 Processor -- start 2024-04-14T08:33:18.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.477+0000 7f5e477fe640 1 -- start start 2024-04-14T08:33:18.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.477+0000 7f5e477fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5e40162700 0x7f5e40169380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.477+0000 7f5e477fe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e400d0200 con 0x7f5e400d1a10 2024-04-14T08:33:18.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.477+0000 7f5e477fe640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e400cf160 con 0x7f5e4015c150 2024-04-14T08:33:18.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.477+0000 7f5e477fe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5e400cef10 con 0x7f5e40162700 2024-04-14T08:33:18.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.477+0000 7f5e44d99640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5e400d1a10 0x7f5e40161ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35828/0 (socket says 172.21.15.172:35828) 2024-04-14T08:33:18.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.478+0000 7f5e44d99640 1 -- 172.21.15.172:0/3208199531 learned_addr learned my addr 172.21.15.172:0/3208199531 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:18.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.478+0000 7f5e45d9b640 1 --2- 172.21.15.172:0/3208199531 >> v2:172.21.15.177:3300/0 conn(0x7f5e40162700 0x7f5e40169380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.478+0000 7f5e37fff640 1 -- 172.21.15.172:0/3208199531 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3645205388 0 0) 0x7f5e400d0200 con 0x7f5e400d1a10 2024-04-14T08:33:18.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.478+0000 7f5e37fff640 1 -- 172.21.15.172:0/3208199531 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e2c003520 con 0x7f5e400d1a10 2024-04-14T08:33:18.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.478+0000 7f5e37fff640 1 -- 172.21.15.172:0/3208199531 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196029957 0 0) 0x7f5e400cf160 con 0x7f5e4015c150 2024-04-14T08:33:18.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.478+0000 7f5e37fff640 1 -- 172.21.15.172:0/3208199531 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e400d0200 con 0x7f5e4015c150 2024-04-14T08:33:18.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.478+0000 7f5e45d9b640 1 -- 172.21.15.172:0/3208199531 >> v1:172.21.15.177:6789/0 conn(0x7f5e4015c150 legacy=0x7f5e400d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.478+0000 7f5e45d9b640 1 -- 172.21.15.172:0/3208199531 >> v1:172.21.15.172:6789/0 conn(0x7f5e400d1a10 legacy=0x7f5e40161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.478+0000 7f5e45d9b640 1 -- 172.21.15.172:0/3208199531 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e400cf440 con 0x7f5e40162700 2024-04-14T08:33:18.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.479+0000 7f5e45d9b640 1 --2- 172.21.15.172:0/3208199531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40162700 0x7f5e40169380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5e380097a0 tx=0x7f5e3802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6d462cd07f63efc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:18.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.479+0000 7f5e37fff640 1 -- 172.21.15.172:0/3208199531 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38034070 con 0x7f5e40162700 2024-04-14T08:33:18.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.479+0000 7f5e37fff640 1 -- 172.21.15.172:0/3208199531 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e38030610 con 0x7f5e40162700 2024-04-14T08:33:18.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.479+0000 7f5e37fff640 1 -- 172.21.15.172:0/3208199531 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e380309c0 con 0x7f5e40162700 2024-04-14T08:33:18.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.479+0000 7f5e477fe640 1 -- 172.21.15.172:0/3208199531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40162700 msgr2=0x7f5e40169380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.479+0000 7f5e477fe640 1 --2- 172.21.15.172:0/3208199531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40162700 0x7f5e40169380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5e380097a0 tx=0x7f5e3802b960 comp rx=0 tx=0).stop 2024-04-14T08:33:18.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.479+0000 7f5e477fe640 1 -- 172.21.15.172:0/3208199531 shutdown_connections 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.479+0000 7f5e477fe640 1 --2- 172.21.15.172:0/3208199531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40162700 0x7f5e40169380 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.479+0000 7f5e477fe640 1 -- 172.21.15.172:0/3208199531 >> 172.21.15.172:0/3208199531 conn(0x7f5e4009d580 msgr2=0x7f5e400d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.479+0000 7f5e477fe640 1 -- 172.21.15.172:0/3208199531 shutdown_connections 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.480+0000 7f5e477fe640 1 -- 172.21.15.172:0/3208199531 wait complete. 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e477fe640 1 Processor -- start 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e477fe640 1 -- start start 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e477fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e400d1a10 0x7f5e4015e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e477fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4015c150 0x7f5e4015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e477fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40162700 0x7f5e400b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e477fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e400cfce0 con 0x7f5e400d1a10 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e477fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e400cfa00 con 0x7f5e40162700 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e477fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e400d2e90 con 0x7f5e4015c150 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e4559a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e400d1a10 0x7f5e4015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e4559a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e400d1a10 0x7f5e4015e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49850/0 (socket says 172.21.15.172:49850) 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e4559a640 1 -- 172.21.15.172:0/2094979714 learned_addr learned my addr 172.21.15.172:0/2094979714 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e45d9b640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40162700 0x7f5e400b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.481+0000 7f5e44d99640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4015c150 0x7f5e4015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.482+0000 7f5e45d9b640 1 -- 172.21.15.172:0/2094979714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4015c150 msgr2=0x7f5e4015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.482+0000 7f5e45d9b640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4015c150 0x7f5e4015ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.482+0000 7f5e45d9b640 1 -- 172.21.15.172:0/2094979714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e400d1a10 msgr2=0x7f5e4015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.482+0000 7f5e45d9b640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e400d1a10 0x7f5e4015e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.482+0000 7f5e45d9b640 1 -- 172.21.15.172:0/2094979714 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e400b92e0 con 0x7f5e40162700 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.482+0000 7f5e4559a640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e400d1a10 0x7f5e4015e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.482+0000 7f5e45d9b640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40162700 0x7f5e400b6eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5e38033000 tx=0x7f5e380046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.482+0000 7f5e35ffb640 1 -- 172.21.15.172:0/2094979714 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38034040 con 0x7f5e40162700 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.482+0000 7f5e35ffb640 1 -- 172.21.15.172:0/2094979714 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e38030410 con 0x7f5e40162700 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.482+0000 7f5e477fe640 1 -- 172.21.15.172:0/2094979714 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e400b95c0 con 0x7f5e40162700 2024-04-14T08:33:18.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.482+0000 7f5e35ffb640 1 -- 172.21.15.172:0/2094979714 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38006e10 con 0x7f5e40162700 2024-04-14T08:33:18.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.482+0000 7f5e477fe640 1 -- 172.21.15.172:0/2094979714 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e400b9b80 con 0x7f5e40162700 2024-04-14T08:33:18.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.483+0000 7f5e477fe640 1 -- 172.21.15.172:0/2094979714 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e40075490 con 0x7f5e40162700 2024-04-14T08:33:18.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.484+0000 7f5e35ffb640 1 -- 172.21.15.172:0/2094979714 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e38037070 con 0x7f5e40162700 2024-04-14T08:33:18.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.484+0000 7f5e35ffb640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e2406c8f0 0x7f5e2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.484+0000 7f5e35ffb640 1 -- 172.21.15.172:0/2094979714 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5e380adbe0 con 0x7f5e40162700 2024-04-14T08:33:18.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.484+0000 7f5e4559a640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e2406c8f0 0x7f5e2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.484+0000 7f5e4559a640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e2406c8f0 0x7f5e2406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5e400b63b0 tx=0x7f5e3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:18.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.489+0000 7f5e35ffb640 1 -- 172.21.15.172:0/2094979714 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5e380305e0 con 0x7f5e40162700 2024-04-14T08:33:18.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.663+0000 7f5e477fe640 1 -- 172.21.15.172:0/2094979714 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e400d49c0 con 0x7f5e40162700 2024-04-14T08:33:18.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.665+0000 7f5e35ffb640 1 -- 172.21.15.172:0/2094979714 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7f5e3807b680 con 0x7f5e40162700 2024-04-14T08:33:18.665 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:18.666 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:18.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.667+0000 7f5e477fe640 1 -- 172.21.15.172:0/2094979714 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e2406c8f0 msgr2=0x7f5e2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.667+0000 7f5e477fe640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e2406c8f0 0x7f5e2406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5e400b63b0 tx=0x7f5e3c009040 comp rx=0 tx=0).stop 2024-04-14T08:33:18.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.667+0000 7f5e477fe640 1 -- 172.21.15.172:0/2094979714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40162700 msgr2=0x7f5e400b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.667+0000 7f5e477fe640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40162700 0x7f5e400b6eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5e38033000 tx=0x7f5e380046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.667+0000 7f5e477fe640 1 -- 172.21.15.172:0/2094979714 shutdown_connections 2024-04-14T08:33:18.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.667+0000 7f5e477fe640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e2406c8f0 0x7f5e2406edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.667+0000 7f5e477fe640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40162700 0x7f5e400b6eb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.667+0000 7f5e477fe640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4015c150 0x7f5e4015ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.667+0000 7f5e477fe640 1 --2- 172.21.15.172:0/2094979714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e400d1a10 0x7f5e4015e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.667+0000 7f5e477fe640 1 -- 172.21.15.172:0/2094979714 >> 172.21.15.172:0/2094979714 conn(0x7f5e4009d580 msgr2=0x7f5e401610b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:18.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.667+0000 7f5e477fe640 1 -- 172.21.15.172:0/2094979714 shutdown_connections 2024-04-14T08:33:18.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.667+0000 7f5e477fe640 1 -- 172.21.15.172:0/2094979714 wait complete. 2024-04-14T08:33:18.678 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:33:18.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.779+0000 7fbb2c860640 1 Processor -- start 2024-04-14T08:33:18.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.779+0000 7fbb2c860640 1 -- start start 2024-04-14T08:33:18.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb2c860640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbb2806fdf0 0x7fbb280b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb2c860640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbb2815e890 0x7fbb2815ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb2c860640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb28070270 con 0x7fbb2806b520 2024-04-14T08:33:18.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb2c860640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fbb280b4310 con 0x7fbb2806fdf0 2024-04-14T08:33:18.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb2c860640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fbb28072d80 con 0x7fbb2815e890 2024-04-14T08:33:18.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb1ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbb2806fdf0 0x7fbb280b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb2659c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fbb2806b520 0x7fbb2806b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35830/0 (socket says 172.21.15.172:35830) 2024-04-14T08:33:18.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb2659c640 1 -- 172.21.15.172:0/1342062770 learned_addr learned my addr 172.21.15.172:0/1342062770 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:18.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb26d9d640 1 --2- 172.21.15.172:0/1342062770 >> v2:172.21.15.177:3300/0 conn(0x7fbb2815e890 0x7fbb2815ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb1ffff640 1 -- 172.21.15.172:0/1342062770 >> v2:172.21.15.177:3300/0 conn(0x7fbb2815e890 msgr2=0x7fbb2815ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:18.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb1ffff640 1 --2- 172.21.15.172:0/1342062770 >> v2:172.21.15.177:3300/0 conn(0x7fbb2815e890 0x7fbb2815ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb1ffff640 1 -- 172.21.15.172:0/1342062770 >> v1:172.21.15.172:6789/0 conn(0x7fbb2806b520 legacy=0x7fbb2806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb1ffff640 1 -- 172.21.15.172:0/1342062770 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb280b4540 con 0x7fbb2806fdf0 2024-04-14T08:33:18.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.780+0000 7fbb25d9b640 1 -- 172.21.15.172:0/1342062770 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1709538076 0 0) 0x7fbb28070270 con 0x7fbb2806b520 2024-04-14T08:33:18.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.781+0000 7fbb1ffff640 1 --2- 172.21.15.172:0/1342062770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb2806fdf0 0x7fbb280b3dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbb2806c0e0 tx=0x7fbb1002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2531a9ab2a2e35e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:18.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.781+0000 7fbb25d9b640 1 -- 172.21.15.172:0/1342062770 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10035020 con 0x7fbb2806fdf0 2024-04-14T08:33:18.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.781+0000 7fbb25d9b640 1 -- 172.21.15.172:0/1342062770 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb10031830 con 0x7fbb2806fdf0 2024-04-14T08:33:18.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.781+0000 7fbb25d9b640 1 -- 172.21.15.172:0/1342062770 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10031c00 con 0x7fbb2806fdf0 2024-04-14T08:33:18.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.782+0000 7fbb2c860640 1 -- 172.21.15.172:0/1342062770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb2806fdf0 msgr2=0x7fbb280b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.782+0000 7fbb2c860640 1 --2- 172.21.15.172:0/1342062770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb2806fdf0 0x7fbb280b3dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbb2806c0e0 tx=0x7fbb1002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.782+0000 7fbb2c860640 1 -- 172.21.15.172:0/1342062770 shutdown_connections 2024-04-14T08:33:18.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.782+0000 7fbb2c860640 1 --2- 172.21.15.172:0/1342062770 >> v2:172.21.15.177:3300/0 conn(0x7fbb2815e890 0x7fbb2815ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.782+0000 7fbb2c860640 1 --2- 172.21.15.172:0/1342062770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb2806fdf0 0x7fbb280b3dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.782+0000 7fbb2c860640 1 -- 172.21.15.172:0/1342062770 >> 172.21.15.172:0/1342062770 conn(0x7fbb280d0f20 msgr2=0x7fbb280d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:18.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.782+0000 7fbb2c860640 1 -- 172.21.15.172:0/1342062770 shutdown_connections 2024-04-14T08:33:18.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.782+0000 7fbb2c860640 1 -- 172.21.15.172:0/1342062770 wait complete. 2024-04-14T08:33:18.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.783+0000 7fbb2c860640 1 Processor -- start 2024-04-14T08:33:18.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.783+0000 7fbb2c860640 1 -- start start 2024-04-14T08:33:18.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb2c860640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb2806b520 0x7fbb2813f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb2c860640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb2806fdf0 0x7fbb281435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb2c860640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb2815e890 0x7fbb28161600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb2c860640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbb28072a10 con 0x7fbb2806fdf0 2024-04-14T08:33:18.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb2c860640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb28072730 con 0x7fbb2806b520 2024-04-14T08:33:18.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb2c860640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbb28073210 con 0x7fbb2815e890 2024-04-14T08:33:18.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb2806fdf0 0x7fbb281435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb2806fdf0 0x7fbb281435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49874/0 (socket says 172.21.15.172:49874) 2024-04-14T08:33:18.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb1ffff640 1 -- 172.21.15.172:0/3049749405 learned_addr learned my addr 172.21.15.172:0/3049749405 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:18.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb26d9d640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb2815e890 0x7fbb28161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb2659c640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb2806b520 0x7fbb2813f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb1ffff640 1 -- 172.21.15.172:0/3049749405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb2815e890 msgr2=0x7fbb28161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb1ffff640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb2815e890 0x7fbb28161600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb1ffff640 1 -- 172.21.15.172:0/3049749405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb2806b520 msgr2=0x7fbb2813f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb1ffff640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb2806b520 0x7fbb2813f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb1ffff640 1 -- 172.21.15.172:0/3049749405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2815f920 con 0x7fbb2806fdf0 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb1ffff640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb2806fdf0 0x7fbb281435c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009890 tx=0x7fbb100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb1effd640 1 -- 172.21.15.172:0/3049749405 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1003a8b0 con 0x7fbb2806fdf0 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb1effd640 1 -- 172.21.15.172:0/3049749405 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1003aa70 con 0x7fbb2806fdf0 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb2c860640 1 -- 172.21.15.172:0/3049749405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2815fc00 con 0x7fbb2806fdf0 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb1effd640 1 -- 172.21.15.172:0/3049749405 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1003ae60 con 0x7fbb2806fdf0 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.784+0000 7fbb2c860640 1 -- 172.21.15.172:0/3049749405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb281601c0 con 0x7fbb2806fdf0 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.786+0000 7fbb2c860640 1 -- 172.21.15.172:0/3049749405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb28068520 con 0x7fbb2806fdf0 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.786+0000 7fbb1effd640 1 -- 172.21.15.172:0/3049749405 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb10038020 con 0x7fbb2806fdf0 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.786+0000 7fbb1effd640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb0806c8f0 0x7fbb0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.786+0000 7fbb1effd640 1 -- 172.21.15.172:0/3049749405 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbb100adbc0 con 0x7fbb2806fdf0 2024-04-14T08:33:18.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.786+0000 7fbb2659c640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb0806c8f0 0x7fbb0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:18.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.787+0000 7fbb2659c640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb0806c8f0 0x7fbb0806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbb18007fd0 tx=0x7fbb18007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:18.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:18.791+0000 7fbb1effd640 1 -- 172.21.15.172:0/3049749405 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbb10031800 con 0x7fbb2806fdf0 2024-04-14T08:33:19.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.002+0000 7fbb2c860640 1 -- 172.21.15.172:0/3049749405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb2806f920 con 0x7fbb2806fdf0 2024-04-14T08:33:19.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.003+0000 7fbb1effd640 1 -- 172.21.15.172:0/3049749405 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7fbb1007b5e0 con 0x7fbb2806fdf0 2024-04-14T08:33:19.004 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:19.004 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3053874257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3053874257},{"type":"v1","addr":"172.21.15.172:6835","nonce":3053874257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2790588849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2790588849},{"type":"v1","addr":"172.21.15.172:6837","nonce":2790588849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3130776573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3130776573},{"type":"v1","addr":"172.21.15.177:6834","nonce":3130776573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:32:53.535483+0000","modified":"2024-04-14T08:33:09.604218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/517963877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":517963877},{"type":"v1","addr":"172.21.15.177:6835","nonce":517963877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4309,"qdb_cluster":[4309]},"id":1}]} 2024-04-14T08:33:19.004 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 9 2024-04-14T08:33:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.005+0000 7fbb2c860640 1 -- 172.21.15.172:0/3049749405 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb0806c8f0 msgr2=0x7fbb0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.005+0000 7fbb2c860640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb0806c8f0 0x7fbb0806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbb18007fd0 tx=0x7fbb18007de0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.005+0000 7fbb2c860640 1 -- 172.21.15.172:0/3049749405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb2806fdf0 msgr2=0x7fbb281435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.005+0000 7fbb2c860640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb2806fdf0 0x7fbb281435c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009890 tx=0x7fbb100046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.005+0000 7fbb2c860640 1 -- 172.21.15.172:0/3049749405 shutdown_connections 2024-04-14T08:33:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.005+0000 7fbb2c860640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb0806c8f0 0x7fbb0806edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.005+0000 7fbb2c860640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb2815e890 0x7fbb28161600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.005+0000 7fbb2c860640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb2806fdf0 0x7fbb281435c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.005+0000 7fbb2c860640 1 --2- 172.21.15.172:0/3049749405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb2806b520 0x7fbb2813f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.005+0000 7fbb2c860640 1 -- 172.21.15.172:0/3049749405 >> 172.21.15.172:0/3049749405 conn(0x7fbb280d0f20 msgr2=0x7fbb280b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:19.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.005+0000 7fbb2c860640 1 -- 172.21.15.172:0/3049749405 shutdown_connections 2024-04-14T08:33:19.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.005+0000 7fbb2c860640 1 -- 172.21.15.172:0/3049749405 wait complete. 2024-04-14T08:33:19.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:32:53.535483+0000', 'modified': '2024-04-14T08:33:09.604218+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4309}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4309': {'gid': 4309, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6835/517963877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6833', 'nonce': 517963877}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 517963877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4309, 'qdb_cluster': [4309]} 2024-04-14T08:33:19.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:33:19.420 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-14T08:33:19.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.523+0000 7f37589ed640 1 Processor -- start 2024-04-14T08:33:19.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.523+0000 7f37589ed640 1 -- start start 2024-04-14T08:33:19.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.524+0000 7f37589ed640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f37540ce200 0x7f37541603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.524+0000 7f37589ed640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3754160930 0x7f3754072450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.524+0000 7f37589ed640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f375415cf10 con 0x7f3754159c20 2024-04-14T08:33:19.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.524+0000 7f37589ed640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f375415fc80 con 0x7f37540ce200 2024-04-14T08:33:19.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.524+0000 7f37589ed640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f375415ff60 con 0x7f3754160930 2024-04-14T08:33:19.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.524+0000 7f375259c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3754159c20 0x7f375415c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35832/0 (socket says 172.21.15.172:35832) 2024-04-14T08:33:19.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.524+0000 7f375259c640 1 -- 172.21.15.172:0/2155920562 learned_addr learned my addr 172.21.15.172:0/2155920562 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:19.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.524+0000 7f3752d9d640 1 --2- 172.21.15.172:0/2155920562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3754160930 0x7f3754072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.524+0000 7f3751d9b640 1 --2- 172.21.15.172:0/2155920562 >> v2:172.21.15.177:3300/0 conn(0x7f37540ce200 0x7f37541603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.524+0000 7f375159a640 1 -- 172.21.15.172:0/2155920562 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 463349815 0 0) 0x7f375415cf10 con 0x7f3754159c20 2024-04-14T08:33:19.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.524+0000 7f375159a640 1 -- 172.21.15.172:0/2155920562 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f373c003520 con 0x7f3754159c20 2024-04-14T08:33:19.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.525+0000 7f3751d9b640 1 -- 172.21.15.172:0/2155920562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3754160930 msgr2=0x7f3754072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:19.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.525+0000 7f3751d9b640 1 --2- 172.21.15.172:0/2155920562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3754160930 0x7f3754072450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.525+0000 7f3751d9b640 1 -- 172.21.15.172:0/2155920562 >> v1:172.21.15.172:6789/0 conn(0x7f3754159c20 legacy=0x7f375415c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:19.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.525+0000 7f3751d9b640 1 -- 172.21.15.172:0/2155920562 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375415d1f0 con 0x7f37540ce200 2024-04-14T08:33:19.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.525+0000 7f3752d9d640 1 --2- 172.21.15.172:0/2155920562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3754160930 0x7f3754072450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:19.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.525+0000 7f3751d9b640 1 --2- 172.21.15.172:0/2155920562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37540ce200 0x7f37541603f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f37480097a0 tx=0x7f374802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8e3cbb4831d24f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:19.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.525+0000 7f375159a640 1 -- 172.21.15.172:0/2155920562 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748035020 con 0x7f37540ce200 2024-04-14T08:33:19.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.525+0000 7f375159a640 1 -- 172.21.15.172:0/2155920562 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3748031830 con 0x7f37540ce200 2024-04-14T08:33:19.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.525+0000 7f375159a640 1 -- 172.21.15.172:0/2155920562 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748031c00 con 0x7f37540ce200 2024-04-14T08:33:19.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.526+0000 7f37589ed640 1 -- 172.21.15.172:0/2155920562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37540ce200 msgr2=0x7f37541603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:19.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.526+0000 7f37589ed640 1 --2- 172.21.15.172:0/2155920562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37540ce200 0x7f37541603f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f37480097a0 tx=0x7f374802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.526+0000 7f37589ed640 1 -- 172.21.15.172:0/2155920562 shutdown_connections 2024-04-14T08:33:19.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.526+0000 7f37589ed640 1 --2- 172.21.15.172:0/2155920562 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3754160930 0x7f3754072450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.526+0000 7f37589ed640 1 --2- 172.21.15.172:0/2155920562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37540ce200 0x7f37541603f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.526+0000 7f37589ed640 1 -- 172.21.15.172:0/2155920562 >> 172.21.15.172:0/2155920562 conn(0x7f375406b520 msgr2=0x7f375406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:19.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.526+0000 7f37589ed640 1 -- 172.21.15.172:0/2155920562 shutdown_connections 2024-04-14T08:33:19.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.526+0000 7f37589ed640 1 -- 172.21.15.172:0/2155920562 wait complete. 2024-04-14T08:33:19.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.527+0000 7f37589ed640 1 Processor -- start 2024-04-14T08:33:19.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.528+0000 7f37589ed640 1 -- start start 2024-04-14T08:33:19.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.528+0000 7f37589ed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37540ce200 0x7f3754144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.528+0000 7f37589ed640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3754159c20 0x7f37540b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.528+0000 7f37589ed640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37540b3680 0x7f375415ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.528+0000 7f37589ed640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3754072990 con 0x7f37540ce200 2024-04-14T08:33:19.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.528+0000 7f37589ed640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f375415a000 con 0x7f37540b3680 2024-04-14T08:33:19.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.528+0000 7f37589ed640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3754072b10 con 0x7f3754159c20 2024-04-14T08:33:19.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.528+0000 7f375259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37540ce200 0x7f3754144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.528+0000 7f375259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37540ce200 0x7f3754144700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49884/0 (socket says 172.21.15.172:49884) 2024-04-14T08:33:19.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.528+0000 7f375259c640 1 -- 172.21.15.172:0/3409495062 learned_addr learned my addr 172.21.15.172:0/3409495062 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:19.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.528+0000 7f3752d9d640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37540b3680 0x7f375415ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.528+0000 7f3751d9b640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3754159c20 0x7f37540b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.529+0000 7f375259c640 1 -- 172.21.15.172:0/3409495062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3754159c20 msgr2=0x7f37540b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:19.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.529+0000 7f375259c640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3754159c20 0x7f37540b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.529+0000 7f375259c640 1 -- 172.21.15.172:0/3409495062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37540b3680 msgr2=0x7f375415ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:19.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.529+0000 7f375259c640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37540b3680 0x7f375415ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.529+0000 7f375259c640 1 -- 172.21.15.172:0/3409495062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37540b3c00 con 0x7f37540ce200 2024-04-14T08:33:19.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.529+0000 7f375259c640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37540ce200 0x7f3754144700 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f374400cb40 tx=0x7f374400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:19.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.529+0000 7f3742ffd640 1 -- 172.21.15.172:0/3409495062 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744017070 con 0x7f37540ce200 2024-04-14T08:33:19.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.529+0000 7f3742ffd640 1 -- 172.21.15.172:0/3409495062 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f374400abd0 con 0x7f37540ce200 2024-04-14T08:33:19.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.529+0000 7f37589ed640 1 -- 172.21.15.172:0/3409495062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37540b3ee0 con 0x7f37540ce200 2024-04-14T08:33:19.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.529+0000 7f3742ffd640 1 -- 172.21.15.172:0/3409495062 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744012400 con 0x7f37540ce200 2024-04-14T08:33:19.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.529+0000 7f37589ed640 1 -- 172.21.15.172:0/3409495062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37540b44a0 con 0x7f37540ce200 2024-04-14T08:33:19.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.530+0000 7f3742ffd640 1 -- 172.21.15.172:0/3409495062 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f374401a070 con 0x7f37540ce200 2024-04-14T08:33:19.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.531+0000 7f3742ffd640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f373406c920 0x7f373406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.531+0000 7f3742ffd640 1 -- 172.21.15.172:0/3409495062 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f374408bb10 con 0x7f37540ce200 2024-04-14T08:33:19.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.531+0000 7f3751d9b640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f373406c920 0x7f373406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.531+0000 7f3751d9b640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f373406c920 0x7f373406ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3748036000 tx=0x7f3748008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:19.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.532+0000 7f375b7d4740 1 Processor -- start 2024-04-14T08:33:19.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.532+0000 7f375b7d4740 1 -- start start 2024-04-14T08:33:19.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f375b7d4740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778b60 0x55b04f74bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f375b7d4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778f40 0x55b04f77b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f375b7d4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f74c040 0x55b04f77be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f375b7d4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b04f6b5030 con 0x55b04f74c040 2024-04-14T08:33:19.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f375b7d4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b04f65d620 con 0x55b04f778f40 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f375b7d4740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55b04f6b0fe0 con 0x55b04f778b60 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3752d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f74c040 0x55b04f77be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3752d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f74c040 0x55b04f77be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49892/0 (socket says 172.21.15.172:49892) 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3752d9d640 1 -- 172.21.15.172:0/3226793351 learned_addr learned my addr 172.21.15.172:0/3226793351 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3751d9b640 1 --2- 172.21.15.172:0/3226793351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778f40 0x55b04f77b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f375259c640 1 --2- 172.21.15.172:0/3226793351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778b60 0x55b04f74bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3752d9d640 1 -- 172.21.15.172:0/3226793351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778b60 msgr2=0x55b04f74bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3752d9d640 1 --2- 172.21.15.172:0/3226793351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778b60 0x55b04f74bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3752d9d640 1 -- 172.21.15.172:0/3226793351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778f40 msgr2=0x55b04f77b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3752d9d640 1 --2- 172.21.15.172:0/3226793351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778f40 0x55b04f77b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3752d9d640 1 -- 172.21.15.172:0/3226793351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04f3c92d0 con 0x55b04f74c040 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3752d9d640 1 --2- 172.21.15.172:0/3226793351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f74c040 0x55b04f77be30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f374c009af0 tx=0x7f374c0112c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701150b94151fd52 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:19.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3733fff640 1 -- 172.21.15.172:0/3226793351 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c011d00 con 0x55b04f74c040 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3733fff640 1 -- 172.21.15.172:0/3226793351 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f374c004020 con 0x55b04f74c040 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.533+0000 7f3733fff640 1 -- 172.21.15.172:0/3226793351 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c011ea0 con 0x55b04f74c040 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.534+0000 7f375b7d4740 1 -- 172.21.15.172:0/3226793351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f74c040 msgr2=0x55b04f77be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.534+0000 7f375b7d4740 1 --2- 172.21.15.172:0/3226793351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f74c040 0x55b04f77be30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f374c009af0 tx=0x7f374c0112c0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.534+0000 7f375b7d4740 1 -- 172.21.15.172:0/3226793351 shutdown_connections 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.534+0000 7f375b7d4740 1 --2- 172.21.15.172:0/3226793351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f74c040 0x55b04f77be30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.534+0000 7f375b7d4740 1 --2- 172.21.15.172:0/3226793351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778f40 0x55b04f77b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.534+0000 7f375b7d4740 1 --2- 172.21.15.172:0/3226793351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778b60 0x55b04f74bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.534+0000 7f375b7d4740 1 -- 172.21.15.172:0/3226793351 >> 172.21.15.172:0/3226793351 conn(0x55b04f74dd70 msgr2=0x55b04f7968f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.534+0000 7f375b7d4740 1 -- 172.21.15.172:0/3226793351 shutdown_connections 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.534+0000 7f375b7d4740 1 -- 172.21.15.172:0/3226793351 wait complete. 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.535+0000 7f375b7d4740 1 -- start start 2024-04-14T08:33:19.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.535+0000 7f375b7d4740 1 Processor -- start 2024-04-14T08:33:19.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f375b7d4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778b60 0x55b04f7970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f375b7d4740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778f40 0x55b04f780220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f375b7d4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 0x55b04f77acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f375b7d4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b04f750410 con 0x55b04f77a840 2024-04-14T08:33:19.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f375b7d4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b04f6709e0 con 0x55b04f778b60 2024-04-14T08:33:19.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f375b7d4740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55b04f752c50 con 0x55b04f778f40 2024-04-14T08:33:19.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f375b7d4740 10 client.0 _ll_register_callbacks cb 0x55b04f6ad080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:33:19.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f3752d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 0x55b04f77acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f3752d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 0x55b04f77acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59392/0 (socket says 172.21.15.172:59392) 2024-04-14T08:33:19.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f3752d9d640 1 -- 172.21.15.172:0/2850757491 learned_addr learned my addr 172.21.15.172:0/2850757491 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:19.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f3751d9b640 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778f40 0x55b04f780220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f375259c640 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778b60 0x55b04f7970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f3752d9d640 1 -- 172.21.15.172:0/2850757491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778f40 msgr2=0x55b04f780220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f3752d9d640 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778f40 0x55b04f780220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f3752d9d640 1 -- 172.21.15.172:0/2850757491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778b60 msgr2=0x55b04f7970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f3752d9d640 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778b60 0x55b04f7970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f3752d9d640 1 -- 172.21.15.172:0/2850757491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04f4936b0 con 0x55b04f77a840 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f3752d9d640 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 0x55b04f77acc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f374c0043e0 tx=0x7f374c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f944e0794ccae172 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f37317fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f37317fa640 1 -- 172.21.15.172:0/2850757491 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c009710 con 0x55b04f77a840 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f37317fa640 1 -- 172.21.15.172:0/2850757491 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f374c003820 con 0x55b04f77a840 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f375b7d4740 1 -- 172.21.15.172:0/2850757491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f374c004020 con 0x55b04f77a840 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.536+0000 7f37317fa640 1 -- 172.21.15.172:0/2850757491 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f374c003a20 con 0x55b04f77a840 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f375b7d4740 10 client.4806 fetch_fsmap learned FSMap version 9 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f375b7d4740 1 -- 172.21.15.172:0/2850757491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55b04f7aa3f0 con 0x55b04f77a840 2024-04-14T08:33:19.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f37317fa640 1 -- 172.21.15.172:0/2850757491 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f374c004020 con 0x55b04f77a840 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f37317fa640 1 -- 172.21.15.172:0/2850757491 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f374c034080 con 0x55b04f77a840 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f375b7d4740 10 client.4806 fetch_fsmap finished waiting for FSMap version 9 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f375b7d4740 10 client.4806 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f375b7d4740 20 client.4806 populate_metadata read hostname 'smithi172' 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f375b7d4740 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x55b04f77e8f0 0x55b04f77ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f375b7d4740 4 client.4806 mds_command: new command op to 4309 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f375b7d4740 1 -- 172.21.15.172:0/2850757491 --> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b04f65ccc0 con 0x55b04f77e8f0 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f375259c640 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x55b04f77e8f0 0x55b04f77ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f375259c640 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x55b04f77e8f0 0x55b04f77ecd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce6d83ffb6374a96 server_cookie=95ad2e47e2c09de3 in_seq=0 out_seq=0 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.537+0000 7f37317fa640 10 client.4806 ms_handle_connect on v2:172.21.15.177:6833/517963877 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f37317fa640 1 -- 172.21.15.172:0/2850757491 <== mds.0 v2:172.21.15.177:6833/517963877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b04f65ccc0 con 0x55b04f77e8f0 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f37317fa640 10 client.4806 handle_command_reply: tid=0 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f375b7d4740 1 client.4806 shutdown 2024-04-14T08:33:19.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f375b7d4740 1 -- 172.21.15.172:0/2850757491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 msgr2=0x55b04f77acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:19.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f375b7d4740 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 0x55b04f77acc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f374c0043e0 tx=0x7f374c004730 comp rx=0 tx=0).stop 2024-04-14T08:33:19.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f375b7d4740 1 -- 172.21.15.172:0/2850757491 shutdown_connections 2024-04-14T08:33:19.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f375b7d4740 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 0x55b04f77acc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f375b7d4740 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778f40 0x55b04f780220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f375b7d4740 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x55b04f77e8f0 0x55b04f77ecd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f375b7d4740 1 --2- 172.21.15.172:0/2850757491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778b60 0x55b04f7970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f37317fa640 0 client.4806 ms_handle_reset on v2:172.21.15.177:6833/517963877 2024-04-14T08:33:19.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f375b7d4740 1 -- 172.21.15.172:0/2850757491 >> 172.21.15.172:0/2850757491 conn(0x55b04f74c040 msgr2=0x55b04f74fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:19.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f375b7d4740 1 -- 172.21.15.172:0/2850757491 shutdown_connections 2024-04-14T08:33:19.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f375b7d4740 1 -- 172.21.15.172:0/2850757491 wait complete. 2024-04-14T08:33:19.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.538+0000 7f375b7d4740 20 client.4806 trim_cache size 0 max 16384 2024-04-14T08:33:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.556+0000 7f375b7d4740 1 Processor -- start 2024-04-14T08:33:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.556+0000 7f375b7d4740 1 -- start start 2024-04-14T08:33:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f375b7d4740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778b60 0x55b04f77ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f375b7d4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f778f40 0x55b04f7b98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f375b7d4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f77a840 0x55b04f7b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f375b7d4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3754201e10 con 0x55b04f778f40 2024-04-14T08:33:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f375b7d4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3754200e30 con 0x55b04f77a840 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f375b7d4740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3754151810 con 0x55b04f778b60 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f3751d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f778f40 0x55b04f7b98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f3751d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f778f40 0x55b04f7b98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59400/0 (socket says 172.21.15.172:59400) 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f3751d9b640 1 -- 172.21.15.172:0/2523939666 learned_addr learned my addr 172.21.15.172:0/2523939666 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f375259c640 1 --2- 172.21.15.172:0/2523939666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778b60 0x55b04f77ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f3752d9d640 1 --2- 172.21.15.172:0/2523939666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f77a840 0x55b04f7b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f3751d9b640 1 -- 172.21.15.172:0/2523939666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778b60 msgr2=0x55b04f77ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f3751d9b640 1 --2- 172.21.15.172:0/2523939666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778b60 0x55b04f77ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f3751d9b640 1 -- 172.21.15.172:0/2523939666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f77a840 msgr2=0x55b04f7b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f3751d9b640 1 --2- 172.21.15.172:0/2523939666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f77a840 0x55b04f7b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.557+0000 7f3751d9b640 1 -- 172.21.15.172:0/2523939666 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04f797460 con 0x55b04f778f40 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f3751d9b640 1 --2- 172.21.15.172:0/2523939666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f778f40 0x55b04f7b98a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f37480098c0 tx=0x7f3748031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a21dbb9416f966 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f37337fe640 1 -- 172.21.15.172:0/2523939666 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748035040 con 0x55b04f778f40 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f37337fe640 1 -- 172.21.15.172:0/2523939666 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37480478f0 con 0x55b04f778f40 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f37337fe640 1 -- 172.21.15.172:0/2523939666 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748047ca0 con 0x55b04f778f40 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f375b7d4740 1 -- 172.21.15.172:0/2523939666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f778f40 msgr2=0x55b04f7b98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f375b7d4740 1 --2- 172.21.15.172:0/2523939666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f778f40 0x55b04f7b98a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f37480098c0 tx=0x7f3748031000 comp rx=0 tx=0).stop 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f375b7d4740 1 -- 172.21.15.172:0/2523939666 shutdown_connections 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f375b7d4740 1 --2- 172.21.15.172:0/2523939666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f77a840 0x55b04f7b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f375b7d4740 1 --2- 172.21.15.172:0/2523939666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f778f40 0x55b04f7b98a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f375b7d4740 1 --2- 172.21.15.172:0/2523939666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778b60 0x55b04f77ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f375b7d4740 1 -- 172.21.15.172:0/2523939666 >> 172.21.15.172:0/2523939666 conn(0x55b04f74c040 msgr2=0x55b04f780f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f375b7d4740 1 -- 172.21.15.172:0/2523939666 shutdown_connections 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.558+0000 7f375b7d4740 1 -- 172.21.15.172:0/2523939666 wait complete. 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.559+0000 7f375b7d4740 1 -- start start 2024-04-14T08:33:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.559+0000 7f375b7d4740 1 Processor -- start 2024-04-14T08:33:19.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f375b7d4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778b60 0x55b04f80ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f375b7d4740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778f40 0x55b04f779320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f375b7d4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 0x55b04f779860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f375b7d4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b04f7aaeb0 con 0x55b04f77a840 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f375b7d4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f37542007e0 con 0x55b04f778b60 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f375b7d4740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55b04f780760 con 0x55b04f778f40 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f375b7d4740 10 client.0 _ll_register_callbacks cb 0x7f374c009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f3752d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 0x55b04f779860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f3752d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 0x55b04f779860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59410/0 (socket says 172.21.15.172:59410) 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f375259c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778b60 0x55b04f80ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f3752d9d640 1 -- 172.21.15.172:0/514359475 learned_addr learned my addr 172.21.15.172:0/514359475 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f3751d9b640 1 --2- 172.21.15.172:0/514359475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778f40 0x55b04f779320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f3752d9d640 1 -- 172.21.15.172:0/514359475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778f40 msgr2=0x55b04f779320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f3752d9d640 1 --2- 172.21.15.172:0/514359475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778f40 0x55b04f779320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f3752d9d640 1 -- 172.21.15.172:0/514359475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778b60 msgr2=0x55b04f80ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f3752d9d640 1 --2- 172.21.15.172:0/514359475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778b60 0x55b04f80ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f3752d9d640 1 -- 172.21.15.172:0/514359475 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04f7ad530 con 0x55b04f77a840 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f3752d9d640 1 --2- 172.21.15.172:0/514359475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 0x55b04f779860 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f374c0043e0 tx=0x7f374c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb31256cc273ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f37317fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f37317fa640 1 -- 172.21.15.172:0/514359475 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c038070 con 0x55b04f77a840 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f37317fa640 1 -- 172.21.15.172:0/514359475 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f374c003820 con 0x55b04f77a840 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.560+0000 7f375b7d4740 1 -- 172.21.15.172:0/514359475 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37480478f0 con 0x55b04f77a840 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f37317fa640 1 -- 172.21.15.172:0/514359475 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f374c043050 con 0x55b04f77a840 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f375b7d4740 10 client.4812 fetch_fsmap learned FSMap version 9 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f375b7d4740 1 -- 172.21.15.172:0/514359475 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55b04f7b12b0 con 0x55b04f77a840 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f37317fa640 1 -- 172.21.15.172:0/514359475 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37480478f0 con 0x55b04f77a840 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f37317fa640 1 -- 172.21.15.172:0/514359475 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f374c046020 con 0x55b04f77a840 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f375b7d4740 10 client.4812 fetch_fsmap finished waiting for FSMap version 9 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f375b7d4740 10 client.4812 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f375b7d4740 20 client.4812 populate_metadata read hostname 'smithi172' 2024-04-14T08:33:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f375b7d4740 1 --2- 172.21.15.172:0/514359475 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x55b04f7a4f70 0x55b04f7a5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f375b7d4740 4 client.4812 mds_command: new command op to 4309 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f375b7d4740 1 -- 172.21.15.172:0/514359475 --> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] -- command(tid 0: {"prefix": "status"}) -- 0x55b04f782370 con 0x55b04f7a4f70 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f375259c640 1 --2- 172.21.15.172:0/514359475 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x55b04f7a4f70 0x55b04f7a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f37317fa640 10 client.4812 ms_handle_connect on v2:172.21.15.177:6833/517963877 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.561+0000 7f375259c640 1 --2- 172.21.15.172:0/514359475 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x55b04f7a4f70 0x55b04f7a5350 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4165a5806b13a0ad server_cookie=93f8226abf591741 in_seq=0 out_seq=0 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37317fa640 1 -- 172.21.15.172:0/514359475 <== mds.0 v2:172.21.15.177:6833/517963877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55b04f782370 con 0x55b04f7a4f70 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37317fa640 10 client.4812 handle_command_reply: tid=0 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f375b7d4740 1 client.4812 shutdown 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f375b7d4740 1 -- 172.21.15.172:0/514359475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 msgr2=0x55b04f779860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f375b7d4740 1 --2- 172.21.15.172:0/514359475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 0x55b04f779860 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f374c0043e0 tx=0x7f374c009190 comp rx=0 tx=0).stop 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f375b7d4740 1 -- 172.21.15.172:0/514359475 shutdown_connections 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f375b7d4740 1 --2- 172.21.15.172:0/514359475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b04f77a840 0x55b04f779860 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f375b7d4740 1 --2- 172.21.15.172:0/514359475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b04f778f40 0x55b04f779320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f375b7d4740 1 --2- 172.21.15.172:0/514359475 >> [v2:172.21.15.177:6833/517963877,v1:172.21.15.177:6835/517963877] conn(0x55b04f7a4f70 0x55b04f7a5350 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f375b7d4740 1 --2- 172.21.15.172:0/514359475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b04f778b60 0x55b04f80ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37317fa640 0 client.4812 ms_handle_reset on v2:172.21.15.177:6833/517963877 2024-04-14T08:33:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f375b7d4740 1 -- 172.21.15.172:0/514359475 >> 172.21.15.172:0/514359475 conn(0x55b04f74c040 msgr2=0x55b04f77aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f375b7d4740 1 -- 172.21.15.172:0/514359475 shutdown_connections 2024-04-14T08:33:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f375b7d4740 1 -- 172.21.15.172:0/514359475 wait complete. 2024-04-14T08:33:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f375b7d4740 20 client.4812 trim_cache size 0 max 16384 2024-04-14T08:33:19.569 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:33:19.569 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:33:19.569 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:33:19.569 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 4309, 2024-04-14T08:33:19.569 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:33:19.570 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:33:19.570 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:33:19.570 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 25.380090212999999, 2024-04-14T08:33:19.570 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 9, 2024-04-14T08:33:19.570 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 25, 2024-04-14T08:33:19.570 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 25, 2024-04-14T08:33:19.570 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 32.776718318 2024-04-14T08:33:19.570 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:33:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37589ed640 1 -- 172.21.15.172:0/3409495062 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f373406c920 msgr2=0x7f373406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37589ed640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f373406c920 0x7f373406ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3748036000 tx=0x7f3748008030 comp rx=0 tx=0).stop 2024-04-14T08:33:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37589ed640 1 -- 172.21.15.172:0/3409495062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37540ce200 msgr2=0x7f3754144700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37589ed640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37540ce200 0x7f3754144700 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f374400cb40 tx=0x7f374400cf40 comp rx=0 tx=0).stop 2024-04-14T08:33:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37589ed640 1 -- 172.21.15.172:0/3409495062 shutdown_connections 2024-04-14T08:33:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37589ed640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f373406c920 0x7f373406ede0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37589ed640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37540b3680 0x7f375415ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37589ed640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3754159c20 0x7f37540b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37589ed640 1 --2- 172.21.15.172:0/3409495062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37540ce200 0x7f3754144700 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:19.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.562+0000 7f37589ed640 1 -- 172.21.15.172:0/3409495062 >> 172.21.15.172:0/3409495062 conn(0x7f375406b520 msgr2=0x7f3754074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:19.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.563+0000 7f37589ed640 1 -- 172.21.15.172:0/3409495062 shutdown_connections 2024-04-14T08:33:19.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:19.563+0000 7f37589ed640 1 -- 172.21.15.172:0/3409495062 wait complete. 2024-04-14T08:33:19.574 INFO:teuthology.run_tasks:Running task sequential... 2024-04-14T08:33:19.584 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-14T08:33:19.584 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-14T08:33:19.585 INFO:teuthology.task.print:Enabling mgr modules 2024-04-14T08:33:19.585 INFO:teuthology.run_tasks:Running task kclient... 2024-04-14T08:33:19.598 INFO:tasks.kclient:Mounting kernel clients... 2024-04-14T08:33:19.598 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-14T08:33:19.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:19.598 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:33:19.613 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-14T08:33:19.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:19.614 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-14T08:33:19.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:19.682 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-04-14T08:33:19.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:19.788 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:33:19.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:19.803 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-04-14T08:33:19.872 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2024-04-14T08:33:19.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:33:19.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:19.874 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2024-04-14T08:33:19.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:19.889 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2024-04-14T08:33:19.957 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2024-04-14T08:33:19.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:33:19.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:19.958 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2024-04-14T08:33:20.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:20.012 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2024-04-14T08:33:20.084 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2024-04-14T08:33:20.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:33:20.086 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-14T08:33:20.086 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-14T08:33:20.086 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:33:20.086 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:33:20.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.187+0000 7f1ca6d1e640 1 Processor -- start 2024-04-14T08:33:20.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.187+0000 7f1ca6d1e640 1 -- start start 2024-04-14T08:33:20.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.188+0000 7f1ca6d1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ca00b1df0 0x7f1ca00b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:20.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.188+0000 7f1ca6d1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ca00b4040 0x7f1ca00b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:20.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.188+0000 7f1ca6d1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca00b49f0 con 0x7f1ca00b21d0 2024-04-14T08:33:20.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.188+0000 7f1ca6d1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca015c770 con 0x7f1ca00b4040 2024-04-14T08:33:20.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.188+0000 7f1ca6d1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca00b3430 con 0x7f1ca00b1df0 2024-04-14T08:33:20.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.188+0000 7f1ca52bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ca00b4040 0x7f1ca00b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:20.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.188+0000 7f1ca52bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ca00b4040 0x7f1ca00b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59424/0 (socket says 172.21.15.172:59424) 2024-04-14T08:33:20.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.188+0000 7f1ca52bb640 1 -- 172.21.15.172:0/3873084573 learned_addr learned my addr 172.21.15.172:0/3873084573 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:20.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.188+0000 7f1ca4aba640 1 --2- 172.21.15.172:0/3873084573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ca00b1df0 0x7f1ca00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:20.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.189+0000 7f1ca52bb640 1 -- 172.21.15.172:0/3873084573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ca00b1df0 msgr2=0x7f1ca00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:20.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.189+0000 7f1ca52bb640 1 --2- 172.21.15.172:0/3873084573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ca00b1df0 0x7f1ca00b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:20.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.189+0000 7f1ca52bb640 1 -- 172.21.15.172:0/3873084573 >> v1:172.21.15.177:6789/0 conn(0x7f1ca00b21d0 legacy=0x7f1ca015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:20.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.189+0000 7f1c9f7fe640 1 -- 172.21.15.172:0/3873084573 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933369952 0 0) 0x7f1ca00b49f0 con 0x7f1ca00b21d0 2024-04-14T08:33:20.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.189+0000 7f1ca52bb640 1 -- 172.21.15.172:0/3873084573 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca00b3700 con 0x7f1ca00b4040 2024-04-14T08:33:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.189+0000 7f1ca52bb640 1 --2- 172.21.15.172:0/3873084573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca00b4040 0x7f1ca00b44b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800d150 tx=0x7f1c9802e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127ee659488eb6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.190+0000 7f1c9f7fe640 1 -- 172.21.15.172:0/3873084573 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800b0d0 con 0x7f1ca00b4040 2024-04-14T08:33:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.190+0000 7f1c9f7fe640 1 -- 172.21.15.172:0/3873084573 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9800b270 con 0x7f1ca00b4040 2024-04-14T08:33:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.190+0000 7f1c9f7fe640 1 -- 172.21.15.172:0/3873084573 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98009600 con 0x7f1ca00b4040 2024-04-14T08:33:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.190+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/3873084573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca00b4040 msgr2=0x7f1ca00b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.190+0000 7f1ca6d1e640 1 --2- 172.21.15.172:0/3873084573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca00b4040 0x7f1ca00b44b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800d150 tx=0x7f1c9802e160 comp rx=0 tx=0).stop 2024-04-14T08:33:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.190+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/3873084573 shutdown_connections 2024-04-14T08:33:20.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.190+0000 7f1ca6d1e640 1 --2- 172.21.15.172:0/3873084573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca00b4040 0x7f1ca00b44b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:20.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.190+0000 7f1ca6d1e640 1 --2- 172.21.15.172:0/3873084573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ca00b1df0 0x7f1ca00b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:20.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.190+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/3873084573 >> 172.21.15.172:0/3873084573 conn(0x7f1ca006ec30 msgr2=0x7f1ca006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:20.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.190+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/3873084573 shutdown_connections 2024-04-14T08:33:20.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.190+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/3873084573 wait complete. 2024-04-14T08:33:20.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.191+0000 7f1ca6d1e640 1 Processor -- start 2024-04-14T08:33:20.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.191+0000 7f1ca6d1e640 1 -- start start 2024-04-14T08:33:20.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.192+0000 7f1ca6d1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ca00b1df0 0x7f1ca00aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:20.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.192+0000 7f1ca6d1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ca00b21d0 0x7f1ca01f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:20.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.192+0000 7f1ca6d1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca00b4040 0x7f1ca0143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:20.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.192+0000 7f1ca6d1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca015bb40 con 0x7f1ca00b4040 2024-04-14T08:33:20.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.192+0000 7f1ca6d1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca00b3cc0 con 0x7f1ca00b21d0 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.192+0000 7f1ca6d1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca015b7f0 con 0x7f1ca00b1df0 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.192+0000 7f1ca52bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca00b4040 0x7f1ca0143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.192+0000 7f1ca52bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca00b4040 0x7f1ca0143be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59436/0 (socket says 172.21.15.172:59436) 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.192+0000 7f1ca52bb640 1 -- 172.21.15.172:0/1246812974 learned_addr learned my addr 172.21.15.172:0/1246812974 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.192+0000 7f1ca4aba640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ca00b1df0 0x7f1ca00aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.192+0000 7f1c9ffff640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ca00b21d0 0x7f1ca01f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.193+0000 7f1ca52bb640 1 -- 172.21.15.172:0/1246812974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ca00b1df0 msgr2=0x7f1ca00aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.193+0000 7f1ca52bb640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ca00b1df0 0x7f1ca00aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.193+0000 7f1ca52bb640 1 -- 172.21.15.172:0/1246812974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ca00b21d0 msgr2=0x7f1ca01f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.193+0000 7f1ca52bb640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ca00b21d0 0x7f1ca01f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.193+0000 7f1ca52bb640 1 -- 172.21.15.172:0/1246812974 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca01f3d70 con 0x7f1ca00b4040 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.193+0000 7f1ca4aba640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ca00b1df0 0x7f1ca00aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.193+0000 7f1ca52bb640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca00b4040 0x7f1ca0143be0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1c98038040 tx=0x7f1c98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:20.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.193+0000 7f1c9d7fa640 1 -- 172.21.15.172:0/1246812974 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800b0d0 con 0x7f1ca00b4040 2024-04-14T08:33:20.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.193+0000 7f1c9d7fa640 1 -- 172.21.15.172:0/1246812974 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c98045070 con 0x7f1ca00b4040 2024-04-14T08:33:20.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.193+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/1246812974 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca0142d90 con 0x7f1ca00b4040 2024-04-14T08:33:20.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.193+0000 7f1c9d7fa640 1 -- 172.21.15.172:0/1246812974 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800b720 con 0x7f1ca00b4040 2024-04-14T08:33:20.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.193+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/1246812974 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca0143350 con 0x7f1ca00b4040 2024-04-14T08:33:20.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.194+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/1246812974 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca0068520 con 0x7f1ca00b4040 2024-04-14T08:33:20.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.194+0000 7f1c9d7fa640 1 -- 172.21.15.172:0/1246812974 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c9802ed10 con 0x7f1ca00b4040 2024-04-14T08:33:20.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.195+0000 7f1c9d7fa640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1c8c06c8f0 0x7f1c8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:20.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.195+0000 7f1c9d7fa640 1 -- 172.21.15.172:0/1246812974 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1c980ae8d0 con 0x7f1ca00b4040 2024-04-14T08:33:20.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.195+0000 7f1ca4aba640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1c8c06c8f0 0x7f1c8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:20.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.195+0000 7f1ca4aba640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1c8c06c8f0 0x7f1c8c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00b0970 tx=0x7f1c90009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:20.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.199+0000 7f1c9d7fa640 1 -- 172.21.15.172:0/1246812974 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1c9807c400 con 0x7f1ca00b4040 2024-04-14T08:33:20.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.414+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/1246812974 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1ca00b51e0 con 0x7f1ca00b4040 2024-04-14T08:33:20.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.414+0000 7f1c9d7fa640 1 -- 172.21.15.172:0/1246812974 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f1c98080700 con 0x7f1ca00b4040 2024-04-14T08:33:20.415 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-14T08:33:20.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.416+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/1246812974 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1c8c06c8f0 msgr2=0x7f1c8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:20.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.416+0000 7f1ca6d1e640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1c8c06c8f0 0x7f1c8c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ca00b0970 tx=0x7f1c90009190 comp rx=0 tx=0).stop 2024-04-14T08:33:20.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.416+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/1246812974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca00b4040 msgr2=0x7f1ca0143be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:20.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.416+0000 7f1ca6d1e640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca00b4040 0x7f1ca0143be0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1c98038040 tx=0x7f1c98004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:20.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.416+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/1246812974 shutdown_connections 2024-04-14T08:33:20.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.416+0000 7f1ca6d1e640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1c8c06c8f0 0x7f1c8c06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:20.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.416+0000 7f1ca6d1e640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca00b4040 0x7f1ca0143be0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:20.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.416+0000 7f1ca6d1e640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ca00b21d0 0x7f1ca01f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:20.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.416+0000 7f1ca6d1e640 1 --2- 172.21.15.172:0/1246812974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ca00b1df0 0x7f1ca00aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:20.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.416+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/1246812974 >> 172.21.15.172:0/1246812974 conn(0x7f1ca006ec30 msgr2=0x7f1ca0144760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:20.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.416+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/1246812974 shutdown_connections 2024-04-14T08:33:20.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20.416+0000 7f1ca6d1e640 1 -- 172.21.15.172:0/1246812974 wait complete. 2024-04-14T08:33:20.427 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-14T08:33:20.427 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:33:20.427 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:33:20.428 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:33:20.428 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:33:20.428 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-14T08:33:20.428 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-14T08:33:20.428 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:33:20.428 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:33:20.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:20.428 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:33:20.443 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:33:20.443 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:33:20.443 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:33:20.443 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:20.443 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:33:20.443 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:20.443 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42669sec preferred_lft 42669sec 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:33:20.444 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:20.445 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-14T08:33:20.445 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:20.445 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:20.445 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add name ceph-brx type bridge 2024-04-14T08:33:20.445 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr flush dev ceph-brx 2024-04-14T08:33:20.445 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set ceph-brx up 2024-04-14T08:33:20.445 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-14T08:33:20.445 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:20.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:20.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:20.592 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:33:20.617 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:33:20.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:20.619 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:33:20.679 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:33:20.679 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:33:20.679 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:33:20.679 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:33:20.679 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:33:20.679 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:20.679 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:20.680 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:33:20.680 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:33:20.680 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:33:20.680 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:20.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:20.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:20.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:20.816 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:33:20.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:20.832 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2024-04-14T08:33:20.886 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:20.886 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:20.886 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:33:20.887 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-14T08:33:20.887 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:20.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:21.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:21.009 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-14T08:33:21.009 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:21.009 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:21.009 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-14T08:33:21.009 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-14T08:33:21.009 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-14T08:33:21.009 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-14T08:33:21.009 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-14T08:33:21.010 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:21.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:21.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:21.136 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:21.136 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:21.136 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:33:21.136 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.0 master ceph-brx 2024-04-14T08:33:21.136 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:21.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:21.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:21.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:33:21.219 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:33:21.233 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:33:21.233 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:33:21.234 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:33:21.288 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:33:21.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:21.288 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:33:21.373 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:33:21.374 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:33:21.374 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:33:21.477 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:33:21.477 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:33:21.477 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:33:21.477 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:33:21.477 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:33:21.481 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:33:21.482 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:33:21.482 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:33:21.482 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:33:21.482 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:33:21.482 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:33:21.482 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:33:21.482 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:33:21.483 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:33:21.483 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:33:21.483 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:33:21.483 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:33:21.484 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:33:21.487 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:33:21.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:21.487 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:33:21.513 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:33:21.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:21.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:21.609 INFO:teuthology.orchestra.run.smithi172.stdout:client4604 2024-04-14T08:33:21.609 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:33:21.610 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:21.625 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:33:21.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:21.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:21.720 INFO:teuthology.orchestra.run.smithi172.stdout:client4604 2024-04-14T08:33:21.721 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:33:21.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:21.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:21.775 INFO:teuthology.orchestra.run.smithi172.stdout:client4604 2024-04-14T08:33:21.776 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8e7e3a2f-e38f-4857-b139-5df1f7a099dc.client4604/status') 2024-04-14T08:33:21.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:21.836 INFO:teuthology.orchestra.run.smithi172.stdout:instance: client.4604 (3)172.21.15.172:0/3714841043 2024-04-14T08:33:21.836 INFO:teuthology.orchestra.run.smithi172.stdout:blocklisted: false 2024-04-14T08:33:21.836 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records in 2024-04-14T08:33:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records out 2024-04-14T08:33:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:71 bytes copied, 2.4958e-05 s, 2.8 MB/s 2024-04-14T08:33:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:21.844 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-14T08:33:21.844 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-14T08:33:21.844 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-14T08:33:21.844 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:33:21.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.943+0000 7f6514e60640 1 Processor -- start 2024-04-14T08:33:21.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.943+0000 7f6514e60640 1 -- start start 2024-04-14T08:33:21.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.944+0000 7f6514e60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65100d2830 0x7f65100d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:21.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.944+0000 7f6514e60640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65100d3150 con 0x7f65100cedd0 2024-04-14T08:33:21.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.944+0000 7f6514e60640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65100cf320 con 0x7f651015e890 2024-04-14T08:33:21.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.944+0000 7f6514e60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f65100cff30 con 0x7f65100d2830 2024-04-14T08:33:21.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.944+0000 7f650ed9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f65100cedd0 0x7f65100b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48876/0 (socket says 172.21.15.172:48876) 2024-04-14T08:33:21.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.944+0000 7f650ed9d640 1 -- 172.21.15.172:0/3267081964 learned_addr learned my addr 172.21.15.172:0/3267081964 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:21.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.944+0000 7f650e59c640 1 --2- 172.21.15.172:0/3267081964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65100d2830 0x7f65100d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:21.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.945+0000 7f650d59a640 1 -- 172.21.15.172:0/3267081964 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258117076 0 0) 0x7f65100d3150 con 0x7f65100cedd0 2024-04-14T08:33:21.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.945+0000 7f650d59a640 1 -- 172.21.15.172:0/3267081964 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64f8003520 con 0x7f65100cedd0 2024-04-14T08:33:21.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.945+0000 7f650d59a640 1 -- 172.21.15.172:0/3267081964 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 917396214 0 0) 0x7f65100cf320 con 0x7f651015e890 2024-04-14T08:33:21.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.945+0000 7f650d59a640 1 -- 172.21.15.172:0/3267081964 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65100d3150 con 0x7f651015e890 2024-04-14T08:33:21.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.945+0000 7f650e59c640 1 -- 172.21.15.172:0/3267081964 >> v1:172.21.15.177:6789/0 conn(0x7f651015e890 legacy=0x7f65100d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:21.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.945+0000 7f650e59c640 1 -- 172.21.15.172:0/3267081964 >> v1:172.21.15.172:6789/0 conn(0x7f65100cedd0 legacy=0x7f65100b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:21.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.945+0000 7f650e59c640 1 -- 172.21.15.172:0/3267081964 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65100cf600 con 0x7f65100d2830 2024-04-14T08:33:21.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.946+0000 7f650e59c640 1 --2- 172.21.15.172:0/3267081964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65100d2830 0x7f65100d2c10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6500009dc0 tx=0x7f650002bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6780219a7c6b1e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:21.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.946+0000 7f650d59a640 1 -- 172.21.15.172:0/3267081964 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500039070 con 0x7f65100d2830 2024-04-14T08:33:21.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.946+0000 7f650d59a640 1 -- 172.21.15.172:0/3267081964 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f650002cd50 con 0x7f65100d2830 2024-04-14T08:33:21.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.946+0000 7f650d59a640 1 -- 172.21.15.172:0/3267081964 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500034660 con 0x7f65100d2830 2024-04-14T08:33:21.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.946+0000 7f6514e60640 1 -- 172.21.15.172:0/3267081964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65100d2830 msgr2=0x7f65100d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:21.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.946+0000 7f6514e60640 1 --2- 172.21.15.172:0/3267081964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65100d2830 0x7f65100d2c10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6500009dc0 tx=0x7f650002bfa0 comp rx=0 tx=0).stop 2024-04-14T08:33:21.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.946+0000 7f6514e60640 1 -- 172.21.15.172:0/3267081964 shutdown_connections 2024-04-14T08:33:21.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.946+0000 7f6514e60640 1 --2- 172.21.15.172:0/3267081964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65100d2830 0x7f65100d2c10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:21.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.946+0000 7f6514e60640 1 -- 172.21.15.172:0/3267081964 >> 172.21.15.172:0/3267081964 conn(0x7f651009d590 msgr2=0x7f65100d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:21.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.946+0000 7f6514e60640 1 -- 172.21.15.172:0/3267081964 shutdown_connections 2024-04-14T08:33:21.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.946+0000 7f6514e60640 1 -- 172.21.15.172:0/3267081964 wait complete. 2024-04-14T08:33:21.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.948+0000 7f6514e60640 1 Processor -- start 2024-04-14T08:33:21.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.948+0000 7f6514e60640 1 -- start start 2024-04-14T08:33:21.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.948+0000 7f6514e60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65100cedd0 0x7f651014e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:21.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.948+0000 7f6514e60640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65100d2830 0x7f65100cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.948+0000 7f6514e60640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f651015e890 0x7f65100c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.948+0000 7f6514e60640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f65100cfbc0 con 0x7f65100d2830 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.948+0000 7f6514e60640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f65100d2350 con 0x7f651015e890 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.948+0000 7f6514e60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f65100b3e20 con 0x7f65100cedd0 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f650dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65100d2830 0x7f65100cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f650dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65100d2830 0x7f65100cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59442/0 (socket says 172.21.15.172:59442) 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f650e59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65100cedd0 0x7f651014e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f650dd9b640 1 -- 172.21.15.172:0/3820343313 learned_addr learned my addr 172.21.15.172:0/3820343313 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f650ed9d640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f651015e890 0x7f65100c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f650dd9b640 1 -- 172.21.15.172:0/3820343313 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65100cedd0 msgr2=0x7f651014e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f650dd9b640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65100cedd0 0x7f651014e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f650dd9b640 1 -- 172.21.15.172:0/3820343313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f651015e890 msgr2=0x7f65100c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:21.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f650dd9b640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f651015e890 0x7f65100c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:21.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f650dd9b640 1 -- 172.21.15.172:0/3820343313 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6510160e80 con 0x7f65100d2830 2024-04-14T08:33:21.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f650dd9b640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65100d2830 0x7f65100cdb60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6504001eb0 tx=0x7f650400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:21.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f64feffd640 1 -- 172.21.15.172:0/3820343313 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504017070 con 0x7f65100d2830 2024-04-14T08:33:21.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f64feffd640 1 -- 172.21.15.172:0/3820343313 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65040052d0 con 0x7f65100d2830 2024-04-14T08:33:21.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f64feffd640 1 -- 172.21.15.172:0/3820343313 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504003810 con 0x7f65100d2830 2024-04-14T08:33:21.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f6514e60640 1 -- 172.21.15.172:0/3820343313 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6510161160 con 0x7f65100d2830 2024-04-14T08:33:21.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.949+0000 7f6514e60640 1 -- 172.21.15.172:0/3820343313 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65100cb630 con 0x7f65100d2830 2024-04-14T08:33:21.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.950+0000 7f6514e60640 1 -- 172.21.15.172:0/3820343313 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6510068520 con 0x7f65100d2830 2024-04-14T08:33:21.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.951+0000 7f64feffd640 1 -- 172.21.15.172:0/3820343313 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f650401a070 con 0x7f65100d2830 2024-04-14T08:33:21.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.951+0000 7f64feffd640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64f006c8f0 0x7f64f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:21.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.951+0000 7f64feffd640 1 -- 172.21.15.172:0/3820343313 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f650408b110 con 0x7f65100d2830 2024-04-14T08:33:21.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.951+0000 7f650e59c640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64f006c8f0 0x7f64f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:21.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.951+0000 7f650e59c640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64f006c8f0 0x7f64f006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6500036000 tx=0x7f6500030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:21.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:21.955+0000 7f64feffd640 1 -- 172.21.15.172:0/3820343313 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6504058bb0 con 0x7f65100d2830 2024-04-14T08:33:22.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.167+0000 7f6514e60640 1 -- 172.21.15.172:0/3820343313 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f65100d4960 con 0x7f65100d2830 2024-04-14T08:33:22.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.167+0000 7f64feffd640 1 -- 172.21.15.172:0/3820343313 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f650405ceb0 con 0x7f65100d2830 2024-04-14T08:33:22.168 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-14T08:33:22.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.169+0000 7f6514e60640 1 -- 172.21.15.172:0/3820343313 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64f006c8f0 msgr2=0x7f64f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:22.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.169+0000 7f6514e60640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64f006c8f0 0x7f64f006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6500036000 tx=0x7f6500030000 comp rx=0 tx=0).stop 2024-04-14T08:33:22.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.169+0000 7f6514e60640 1 -- 172.21.15.172:0/3820343313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65100d2830 msgr2=0x7f65100cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:22.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.169+0000 7f6514e60640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65100d2830 0x7f65100cdb60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6504001eb0 tx=0x7f650400c820 comp rx=0 tx=0).stop 2024-04-14T08:33:22.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.169+0000 7f6514e60640 1 -- 172.21.15.172:0/3820343313 shutdown_connections 2024-04-14T08:33:22.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.169+0000 7f6514e60640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64f006c8f0 0x7f64f006edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:22.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.169+0000 7f6514e60640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f651015e890 0x7f65100c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:22.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.169+0000 7f6514e60640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65100d2830 0x7f65100cdb60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:22.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.169+0000 7f6514e60640 1 --2- 172.21.15.172:0/3820343313 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65100cedd0 0x7f651014e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:22.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.169+0000 7f6514e60640 1 -- 172.21.15.172:0/3820343313 >> 172.21.15.172:0/3820343313 conn(0x7f651009d590 msgr2=0x7f65100d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:22.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.170+0000 7f6514e60640 1 -- 172.21.15.172:0/3820343313 shutdown_connections 2024-04-14T08:33:22.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22.170+0000 7f6514e60640 1 -- 172.21.15.172:0/3820343313 wait complete. 2024-04-14T08:33:22.181 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-14T08:33:22.181 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:33:22.181 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:33:22.181 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-14T08:33:22.181 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-14T08:33:22.181 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-14T08:33:22.181 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-14T08:33:22.181 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:33:22.181 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-14T08:33:22.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:22.182 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:33:22.197 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:33:22.197 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:33:22.197 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:33:22.197 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.197 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:33:22.197 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.197 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:22.197 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:22.197 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:33:22.198 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:22.198 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:22.198 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:33:22.198 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:22.198 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:22.198 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:33:22.198 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:22.198 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:33:22.198 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42667sec preferred_lft 42667sec 2024-04-14T08:33:22.198 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:33:22.198 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.198 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:33:22.199 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:22.199 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:33:22.199 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.199 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:33:22.199 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.199 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:33:22.199 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:33:22.199 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link tentative 2024-04-14T08:33:22.199 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:22.199 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:33:22.264 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:33:22.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:22.266 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:33:22.322 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:33:22.322 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:33:22.322 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:33:22.322 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:33:22.322 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:33:22.322 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:22.322 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:22.323 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:33:22.323 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:33:22.323 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:33:22.323 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:22.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:22.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:22.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:22.457 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:33:22.472 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:33:22.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:22.473 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2024-04-14T08:33:22.527 INFO:teuthology.orchestra.run.smithi172.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-14T08:33:22.527 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:22.527 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:22.528 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:33:22.528 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-14T08:33:22.528 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:22.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:22.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:22.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:22.649 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-14T08:33:22.675 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:33:22.675 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:33:22.675 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:33:22.676 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.676 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:33:22.676 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.676 INFO:teuthology.orchestra.run.smithi172.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:33:22.676 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 4a:a9:ea:41:bf:4d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-14T08:33:22.676 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-14T08:33:22.676 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.676 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::48a9:eaff:fe41:bf4d/64 scope link tentative 2024-04-14T08:33:22.676 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:22.677 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-14T08:33:22.744 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:33:22.744 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:33:22.744 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:33:22.744 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.744 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:33:22.744 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.744 INFO:teuthology.orchestra.run.smithi172.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:33:22.744 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 4a:a9:ea:41:bf:4d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-14T08:33:22.744 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-14T08:33:22.744 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.744 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::48a9:eaff:fe41:bf4d/64 scope link tentative 2024-04-14T08:33:22.744 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:22.746 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-14T08:33:22.746 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:22.746 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:22.746 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-14T08:33:22.746 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-14T08:33:22.746 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-14T08:33:22.746 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-14T08:33:22.746 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-14T08:33:22.746 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:22.793 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:33:22.832 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:33:22.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:22.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:22.912 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:22.912 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:33:22.912 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.1 up 2024-04-14T08:33:22.912 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.1 master ceph-brx 2024-04-14T08:33:22.912 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:33:23.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:23.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:23.035 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:33:23.035 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-14T08:33:23.049 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-14T08:33:23.050 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:33:23.050 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-14T08:33:23.104 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:33:23.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:23.105 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-14T08:33:23.190 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-14T08:33:23.190 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-14T08:33:23.190 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:33:23.198 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-14T08:33:23.198 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-14T08:33:23.198 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:33:23.198 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:33:23.198 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:33:23.203 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-14T08:33:23.203 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-14T08:33:23.203 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:33:23.203 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:33:23.203 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:33:23.203 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:33:23.203 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:33:23.203 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:33:23.204 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-14T08:33:23.204 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:33:23.204 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:33:23.204 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:33:23.204 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:33:23.205 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:33:23.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:23.206 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-14T08:33:23.233 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:33:23.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:23.330 INFO:teuthology.orchestra.run.smithi172.stdout:client4822 2024-04-14T08:33:23.330 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:33:23.334 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:23.350 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:33:23.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:23.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:23.447 INFO:teuthology.orchestra.run.smithi172.stdout:client4822 2024-04-14T08:33:23.447 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:33:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:23.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:23.504 INFO:teuthology.orchestra.run.smithi172.stdout:client4822 2024-04-14T08:33:23.504 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8e7e3a2f-e38f-4857-b139-5df1f7a099dc.client4822/status') 2024-04-14T08:33:23.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:23.566 INFO:teuthology.orchestra.run.smithi172.stdout:instance: client.4822 (3)172.21.15.172:0/3406163486 2024-04-14T08:33:23.566 INFO:teuthology.orchestra.run.smithi172.stdout:blocklisted: false 2024-04-14T08:33:23.566 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records in 2024-04-14T08:33:23.566 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records out 2024-04-14T08:33:23.566 INFO:teuthology.orchestra.run.smithi172.stderr:71 bytes copied, 2.1038e-05 s, 3.4 MB/s 2024-04-14T08:33:23.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:23.573 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-14T08:33:23.573 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-14T08:33:23.573 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-14T08:33:23.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:33:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.673+0000 7f8827895640 1 Processor -- start 2024-04-14T08:33:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.673+0000 7f8827895640 1 -- start start 2024-04-14T08:33:23.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8827895640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8820159ef0 0x7f88200cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:23.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8827895640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8820070360 0x7f882015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:23.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8827895640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88200d0550 con 0x7f882015d270 2024-04-14T08:33:23.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8827895640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f88200cf9d0 con 0x7f8820159ef0 2024-04-14T08:33:23.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8827895640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f88200cfcb0 con 0x7f8820070360 2024-04-14T08:33:23.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8825631640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8820159ef0 0x7f88200cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:23.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8824e30640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f882015d270 0x7f882006fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48878/0 (socket says 172.21.15.172:48878) 2024-04-14T08:33:23.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8824e30640 1 -- 172.21.15.172:0/3678047336 learned_addr learned my addr 172.21.15.172:0/3678047336 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:23.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8825e32640 1 --2- 172.21.15.172:0/3678047336 >> v2:172.21.15.177:3300/0 conn(0x7f8820070360 0x7f882015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:23.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8825631640 1 -- 172.21.15.172:0/3678047336 >> v2:172.21.15.177:3300/0 conn(0x7f8820070360 msgr2=0x7f882015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:23.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8825631640 1 --2- 172.21.15.172:0/3678047336 >> v2:172.21.15.177:3300/0 conn(0x7f8820070360 0x7f882015dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:23.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8825631640 1 -- 172.21.15.172:0/3678047336 >> v1:172.21.15.172:6789/0 conn(0x7f882015d270 legacy=0x7f882006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:23.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8825631640 1 -- 172.21.15.172:0/3678047336 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88200cff90 con 0x7f8820159ef0 2024-04-14T08:33:23.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.674+0000 7f8817fff640 1 -- 172.21.15.172:0/3678047336 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 284753423 0 0) 0x7f88200d0550 con 0x7f882015d270 2024-04-14T08:33:23.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.675+0000 7f8825631640 1 --2- 172.21.15.172:0/3678047336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8820159ef0 0x7f88200cf330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f88200673e0 tx=0x7f881c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efea07b11b999464 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:23.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.675+0000 7f8817fff640 1 -- 172.21.15.172:0/3678047336 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c039070 con 0x7f8820159ef0 2024-04-14T08:33:23.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.675+0000 7f8817fff640 1 -- 172.21.15.172:0/3678047336 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f881c02cbf0 con 0x7f8820159ef0 2024-04-14T08:33:23.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.675+0000 7f8817fff640 1 -- 172.21.15.172:0/3678047336 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c034440 con 0x7f8820159ef0 2024-04-14T08:33:23.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.676+0000 7f8827895640 1 -- 172.21.15.172:0/3678047336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8820159ef0 msgr2=0x7f88200cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:23.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.676+0000 7f8827895640 1 --2- 172.21.15.172:0/3678047336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8820159ef0 0x7f88200cf330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f88200673e0 tx=0x7f881c02be40 comp rx=0 tx=0).stop 2024-04-14T08:33:23.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.676+0000 7f8827895640 1 -- 172.21.15.172:0/3678047336 shutdown_connections 2024-04-14T08:33:23.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.676+0000 7f8827895640 1 --2- 172.21.15.172:0/3678047336 >> v2:172.21.15.177:3300/0 conn(0x7f8820070360 0x7f882015dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:23.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.676+0000 7f8827895640 1 --2- 172.21.15.172:0/3678047336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8820159ef0 0x7f88200cf330 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:23.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.676+0000 7f8827895640 1 -- 172.21.15.172:0/3678047336 >> 172.21.15.172:0/3678047336 conn(0x7f8820071ba0 msgr2=0x7f8820071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:23.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.676+0000 7f8827895640 1 -- 172.21.15.172:0/3678047336 shutdown_connections 2024-04-14T08:33:23.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.676+0000 7f8827895640 1 -- 172.21.15.172:0/3678047336 wait complete. 2024-04-14T08:33:23.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.677+0000 7f8827895640 1 Processor -- start 2024-04-14T08:33:23.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.677+0000 7f8827895640 1 -- start start 2024-04-14T08:33:23.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8827895640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8820070360 0x7f88201f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:23.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8827895640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8820159ef0 0x7f88201f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:23.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8827895640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f882015d270 0x7f8820145170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:23.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8827895640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f882015e220 con 0x7f8820159ef0 2024-04-14T08:33:23.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8827895640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f882015d700 con 0x7f8820070360 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8827895640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f882015e3a0 con 0x7f882015d270 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8824e30640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8820159ef0 0x7f88201f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8824e30640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8820159ef0 0x7f88201f33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59452/0 (socket says 172.21.15.172:59452) 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8824e30640 1 -- 172.21.15.172:0/2314185028 learned_addr learned my addr 172.21.15.172:0/2314185028 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8825631640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8820070360 0x7f88201f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8825e32640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f882015d270 0x7f8820145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8824e30640 1 -- 172.21.15.172:0/2314185028 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f882015d270 msgr2=0x7f8820145170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8824e30640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f882015d270 0x7f8820145170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8824e30640 1 -- 172.21.15.172:0/2314185028 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8820070360 msgr2=0x7f88201f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8824e30640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8820070360 0x7f88201f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8824e30640 1 -- 172.21.15.172:0/2314185028 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88201f31b0 con 0x7f8820159ef0 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8824e30640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8820159ef0 0x7f88201f33b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f881000c460 tx=0x7f881000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.678+0000 7f8815ffb640 1 -- 172.21.15.172:0/2314185028 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810018070 con 0x7f8820159ef0 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.679+0000 7f8815ffb640 1 -- 172.21.15.172:0/2314185028 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8810007500 con 0x7f8820159ef0 2024-04-14T08:33:23.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.679+0000 7f8815ffb640 1 -- 172.21.15.172:0/2314185028 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810004a20 con 0x7f8820159ef0 2024-04-14T08:33:23.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.679+0000 7f8827895640 1 -- 172.21.15.172:0/2314185028 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8820153dc0 con 0x7f8820159ef0 2024-04-14T08:33:23.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.679+0000 7f8827895640 1 -- 172.21.15.172:0/2314185028 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8820154380 con 0x7f8820159ef0 2024-04-14T08:33:23.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.680+0000 7f8827895640 1 -- 172.21.15.172:0/2314185028 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8820068520 con 0x7f8820159ef0 2024-04-14T08:33:23.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.680+0000 7f8815ffb640 1 -- 172.21.15.172:0/2314185028 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f881001b070 con 0x7f8820159ef0 2024-04-14T08:33:23.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.681+0000 7f8815ffb640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f880406c8f0 0x7f880406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:23.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.681+0000 7f8815ffb640 1 -- 172.21.15.172:0/2314185028 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f881008afd0 con 0x7f8820159ef0 2024-04-14T08:33:23.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.681+0000 7f8825631640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f880406c8f0 0x7f880406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:23.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.681+0000 7f8825631640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f880406c8f0 0x7f880406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f881c036000 tx=0x7f881c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:23.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.685+0000 7f8815ffb640 1 -- 172.21.15.172:0/2314185028 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f88201f31b0 con 0x7f8820159ef0 2024-04-14T08:33:23.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.897+0000 7f8827895640 1 -- 172.21.15.172:0/2314185028 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f88200727c0 con 0x7f8820159ef0 2024-04-14T08:33:23.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.898+0000 7f8815ffb640 1 -- 172.21.15.172:0/2314185028 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f8810058ae0 con 0x7f8820159ef0 2024-04-14T08:33:23.899 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-14T08:33:23.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.900+0000 7f8827895640 1 -- 172.21.15.172:0/2314185028 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f880406c8f0 msgr2=0x7f880406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:23.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.900+0000 7f8827895640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f880406c8f0 0x7f880406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f881c036000 tx=0x7f881c030000 comp rx=0 tx=0).stop 2024-04-14T08:33:23.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.900+0000 7f8827895640 1 -- 172.21.15.172:0/2314185028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8820159ef0 msgr2=0x7f88201f33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:23.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.900+0000 7f8827895640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8820159ef0 0x7f88201f33b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f881000c460 tx=0x7f881000c920 comp rx=0 tx=0).stop 2024-04-14T08:33:23.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.900+0000 7f8827895640 1 -- 172.21.15.172:0/2314185028 shutdown_connections 2024-04-14T08:33:23.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.900+0000 7f8827895640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f880406c8f0 0x7f880406edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:23.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.900+0000 7f8827895640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f882015d270 0x7f8820145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:23.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.900+0000 7f8827895640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8820159ef0 0x7f88201f33b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:23.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.900+0000 7f8827895640 1 --2- 172.21.15.172:0/2314185028 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8820070360 0x7f88201f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:23.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.900+0000 7f8827895640 1 -- 172.21.15.172:0/2314185028 >> 172.21.15.172:0/2314185028 conn(0x7f8820071ba0 msgr2=0x7f882015aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:23.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.900+0000 7f8827895640 1 -- 172.21.15.172:0/2314185028 shutdown_connections 2024-04-14T08:33:23.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:23.900+0000 7f8827895640 1 -- 172.21.15.172:0/2314185028 wait complete. 2024-04-14T08:33:23.911 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-14T08:33:23.911 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:33:23.911 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2024-04-14T08:33:23.912 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-14T08:33:23.912 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-14T08:33:23.912 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-14T08:33:23.912 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-14T08:33:23.912 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:33:23.912 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-14T08:33:23.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:23.912 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2024-04-14T08:33:23.927 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:33:23.927 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:33:23.927 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:33:23.927 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:23.927 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2024-04-14T08:33:23.927 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f0 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f1 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42654sec preferred_lft 42654sec 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::ec4:7aff:feea:b04b/64 scope link 2024-04-14T08:33:23.928 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:23.929 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-14T08:33:23.929 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:23.929 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-04-14T08:33:23.929 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add name ceph-brx type bridge 2024-04-14T08:33:23.929 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip addr flush dev ceph-brx 2024-04-14T08:33:23.929 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set ceph-brx up 2024-04-14T08:33:23.929 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-14T08:33:23.929 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-04-14T08:33:24.014 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:24.389 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:24.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:24.396 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:33:24.424 INFO:teuthology.orchestra.run.smithi177.stdout:1 2024-04-14T08:33:24.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:24.426 DEBUG:teuthology.orchestra.run.smithi177:> route 2024-04-14T08:33:24.490 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2024-04-14T08:33:24.490 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:33:24.490 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:33:24.490 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:33:24.490 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:33:24.490 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:24.490 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-04-14T08:33:24.491 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:33:24.491 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:33:24.491 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:33:24.491 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-04-14T08:33:24.578 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:24.796 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:24.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:24.804 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2024-04-14T08:33:24.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:24.819 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list-id 2024-04-14T08:33:24.875 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:24.875 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-04-14T08:33:24.875 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-14T08:33:24.875 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-14T08:33:24.875 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-04-14T08:33:24.964 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:24.996 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:25.004 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-14T08:33:25.004 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:25.005 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-04-14T08:33:25.005 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-14T08:33:25.005 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-14T08:33:25.005 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-14T08:33:25.005 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-14T08:33:25.005 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-14T08:33:25.005 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-04-14T08:33:25.059 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:25.152 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:25.161 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:25.161 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-04-14T08:33:25.161 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.0 up 2024-04-14T08:33:25.161 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set dev brx.0 master ceph-brx 2024-04-14T08:33:25.161 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-04-14T08:33:25.211 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:25.239 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:25.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:33:25.245 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-14T08:33:25.261 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-14T08:33:25.261 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:33:25.261 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-14T08:33:25.317 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:33:25.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:25.317 DEBUG:teuthology.orchestra.run.smithi177:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-14T08:33:25.402 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-14T08:33:25.402 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-14T08:33:25.402 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2024-04-14T08:33:25.550 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-14T08:33:25.550 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-14T08:33:25.550 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2024-04-14T08:33:25.550 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:33:25.550 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:33:25.558 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-14T08:33:25.558 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-14T08:33:25.558 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2024-04-14T08:33:25.558 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:33:25.558 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:33:25.558 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:33:25.559 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:33:25.559 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:33:25.559 INFO:teuthology.orchestra.run.smithi177.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-14T08:33:25.559 INFO:teuthology.orchestra.run.smithi177.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:33:25.559 INFO:teuthology.orchestra.run.smithi177.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:33:25.559 INFO:teuthology.orchestra.run.smithi177.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:33:25.559 INFO:teuthology.orchestra.run.smithi177.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:33:25.561 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:33:25.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:25.561 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-14T08:33:25.587 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:33:25.675 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:25.677 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:25.683 INFO:teuthology.orchestra.run.smithi177.stdout:client4616 2024-04-14T08:33:25.683 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:33:25.683 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:25.700 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:33:25.788 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:25.790 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:25.795 INFO:teuthology.orchestra.run.smithi177.stdout:client4616 2024-04-14T08:33:25.796 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:33:25.843 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:25.845 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:25.850 INFO:teuthology.orchestra.run.smithi177.stdout:client4616 2024-04-14T08:33:25.851 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8e7e3a2f-e38f-4857-b139-5df1f7a099dc.client4616/status') 2024-04-14T08:33:25.899 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:25.912 INFO:teuthology.orchestra.run.smithi177.stdout:instance: client.4616 (3)192.168.0.1:0/3384929923 2024-04-14T08:33:25.912 INFO:teuthology.orchestra.run.smithi177.stdout:blocklisted: false 2024-04-14T08:33:25.913 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records in 2024-04-14T08:33:25.913 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records out 2024-04-14T08:33:25.913 INFO:teuthology.orchestra.run.smithi177.stderr:69 bytes copied, 2.0755e-05 s, 3.3 MB/s 2024-04-14T08:33:25.914 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:25.920 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-14T08:33:25.920 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-14T08:33:25.920 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-14T08:33:25.920 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:33:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.032+0000 7f895b1fe640 1 Processor -- start 2024-04-14T08:33:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.032+0000 7f895b1fe640 1 -- start start 2024-04-14T08:33:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f895b1fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f89540b1df0 0x7f89540b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f895b1fe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f895415c770 con 0x7f89540b4040 2024-04-14T08:33:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f895b1fe640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89540b3430 con 0x7f89540b21d0 2024-04-14T08:33:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f895b1fe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f89540b3700 con 0x7f89540b1df0 2024-04-14T08:33:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f895979b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f89540b4040 0x7f89540b4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48880/0 (socket says 172.21.15.172:48880) 2024-04-14T08:33:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f895979b640 1 -- 172.21.15.172:0/1676189972 learned_addr learned my addr 172.21.15.172:0/1676189972 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f8958f9a640 1 --2- 172.21.15.172:0/1676189972 >> v2:172.21.15.177:3300/0 conn(0x7f89540b1df0 0x7f89540b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f89537fe640 1 -- 172.21.15.172:0/1676189972 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3004486971 0 0) 0x7f895415c770 con 0x7f89540b4040 2024-04-14T08:33:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f89537fe640 1 -- 172.21.15.172:0/1676189972 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8944005730 con 0x7f89540b4040 2024-04-14T08:33:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f89537fe640 1 -- 172.21.15.172:0/1676189972 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064189858 0 0) 0x7f89540b3430 con 0x7f89540b21d0 2024-04-14T08:33:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f89537fe640 1 -- 172.21.15.172:0/1676189972 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f895415c770 con 0x7f89540b21d0 2024-04-14T08:33:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f8958f9a640 1 -- 172.21.15.172:0/1676189972 >> v1:172.21.15.177:6789/0 conn(0x7f89540b21d0 legacy=0x7f895415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f8958f9a640 1 -- 172.21.15.172:0/1676189972 >> v1:172.21.15.172:6789/0 conn(0x7f89540b4040 legacy=0x7f89540b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.033+0000 7f8958f9a640 1 -- 172.21.15.172:0/1676189972 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89540b39e0 con 0x7f89540b1df0 2024-04-14T08:33:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.034+0000 7f8958f9a640 1 --2- 172.21.15.172:0/1676189972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89540b1df0 0x7f89540b2ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f89480097a0 tx=0x7f894802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5116d5942012f4b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.034+0000 7f89537fe640 1 -- 172.21.15.172:0/1676189972 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948034070 con 0x7f89540b1df0 2024-04-14T08:33:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.034+0000 7f89537fe640 1 -- 172.21.15.172:0/1676189972 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8948030610 con 0x7f89540b1df0 2024-04-14T08:33:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.034+0000 7f89537fe640 1 -- 172.21.15.172:0/1676189972 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89480309c0 con 0x7f89540b1df0 2024-04-14T08:33:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.035+0000 7f895b1fe640 1 -- 172.21.15.172:0/1676189972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89540b1df0 msgr2=0x7f89540b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.035+0000 7f895b1fe640 1 --2- 172.21.15.172:0/1676189972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89540b1df0 0x7f89540b2ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f89480097a0 tx=0x7f894802b960 comp rx=0 tx=0).stop 2024-04-14T08:33:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.035+0000 7f895b1fe640 1 -- 172.21.15.172:0/1676189972 shutdown_connections 2024-04-14T08:33:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.035+0000 7f895b1fe640 1 --2- 172.21.15.172:0/1676189972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89540b1df0 0x7f89540b2ef0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.035+0000 7f895b1fe640 1 -- 172.21.15.172:0/1676189972 >> 172.21.15.172:0/1676189972 conn(0x7f895406ec30 msgr2=0x7f895406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.035+0000 7f895b1fe640 1 -- 172.21.15.172:0/1676189972 shutdown_connections 2024-04-14T08:33:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.035+0000 7f895b1fe640 1 -- 172.21.15.172:0/1676189972 wait complete. 2024-04-14T08:33:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.036+0000 7f895b1fe640 1 Processor -- start 2024-04-14T08:33:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.036+0000 7f895b1fe640 1 -- start start 2024-04-14T08:33:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895b1fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89540b1df0 0x7f89540cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895b1fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89540b21d0 0x7f89540cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895b1fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89540b4040 0x7f8954139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895b1fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f895415b7f0 con 0x7f89540b4040 2024-04-14T08:33:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895b1fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f895415bb40 con 0x7f89540b1df0 2024-04-14T08:33:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895b1fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f89541552d0 con 0x7f89540b21d0 2024-04-14T08:33:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895979b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89540b4040 0x7f8954139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895979b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89540b4040 0x7f8954139920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59458/0 (socket says 172.21.15.172:59458) 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895979b640 1 -- 172.21.15.172:0/734295703 learned_addr learned my addr 172.21.15.172:0/734295703 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f8958f9a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89540b1df0 0x7f89540cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f8953fff640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89540b21d0 0x7f89540cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895979b640 1 -- 172.21.15.172:0/734295703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89540b21d0 msgr2=0x7f89540cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895979b640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89540b21d0 0x7f89540cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895979b640 1 -- 172.21.15.172:0/734295703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89540b1df0 msgr2=0x7f89540cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895979b640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89540b1df0 0x7f89540cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895979b640 1 -- 172.21.15.172:0/734295703 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89540cf6f0 con 0x7f89540b4040 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.037+0000 7f895979b640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89540b4040 0x7f8954139920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f894c00cb20 tx=0x7f894c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.038+0000 7f89517fa640 1 -- 172.21.15.172:0/734295703 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c017070 con 0x7f89540b4040 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.038+0000 7f89517fa640 1 -- 172.21.15.172:0/734295703 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f894c00aca0 con 0x7f89540b4040 2024-04-14T08:33:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.038+0000 7f89517fa640 1 -- 172.21.15.172:0/734295703 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c012600 con 0x7f89540b4040 2024-04-14T08:33:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.038+0000 7f895b1fe640 1 -- 172.21.15.172:0/734295703 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89540cf9d0 con 0x7f89540b4040 2024-04-14T08:33:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.038+0000 7f895b1fe640 1 -- 172.21.15.172:0/734295703 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89540cff90 con 0x7f89540b4040 2024-04-14T08:33:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.039+0000 7f895b1fe640 1 -- 172.21.15.172:0/734295703 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8954068520 con 0x7f89540b4040 2024-04-14T08:33:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.039+0000 7f89517fa640 1 -- 172.21.15.172:0/734295703 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f894c01a070 con 0x7f89540b4040 2024-04-14T08:33:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.040+0000 7f89517fa640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f893806c8f0 0x7f893806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.040+0000 7f89517fa640 1 -- 172.21.15.172:0/734295703 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f894c08bb60 con 0x7f89540b4040 2024-04-14T08:33:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.040+0000 7f8958f9a640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f893806c8f0 0x7f893806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.040+0000 7f8958f9a640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f893806c8f0 0x7f893806edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8948036000 tx=0x7f894803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:26.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.044+0000 7f89517fa640 1 -- 172.21.15.172:0/734295703 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f894c059600 con 0x7f89540b4040 2024-04-14T08:33:26.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.259+0000 7f895b1fe640 1 -- 172.21.15.172:0/734295703 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f895413caa0 con 0x7f89540b4040 2024-04-14T08:33:26.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.259+0000 7f89517fa640 1 -- 172.21.15.172:0/734295703 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f894c05d900 con 0x7f89540b4040 2024-04-14T08:33:26.260 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-14T08:33:26.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.261+0000 7f895b1fe640 1 -- 172.21.15.172:0/734295703 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f893806c8f0 msgr2=0x7f893806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:26.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.261+0000 7f895b1fe640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f893806c8f0 0x7f893806edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8948036000 tx=0x7f894803b040 comp rx=0 tx=0).stop 2024-04-14T08:33:26.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.261+0000 7f895b1fe640 1 -- 172.21.15.172:0/734295703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89540b4040 msgr2=0x7f8954139920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:26.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.261+0000 7f895b1fe640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89540b4040 0x7f8954139920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f894c00cb20 tx=0x7f894c00a060 comp rx=0 tx=0).stop 2024-04-14T08:33:26.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.261+0000 7f895b1fe640 1 -- 172.21.15.172:0/734295703 shutdown_connections 2024-04-14T08:33:26.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.261+0000 7f895b1fe640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f893806c8f0 0x7f893806edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:26.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.261+0000 7f895b1fe640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89540b4040 0x7f8954139920 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:26.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.261+0000 7f895b1fe640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89540b21d0 0x7f89540cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:26.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.261+0000 7f895b1fe640 1 --2- 172.21.15.172:0/734295703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89540b1df0 0x7f89540cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:26.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.261+0000 7f895b1fe640 1 -- 172.21.15.172:0/734295703 >> 172.21.15.172:0/734295703 conn(0x7f895406ec30 msgr2=0x7f89540b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:26.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.261+0000 7f895b1fe640 1 -- 172.21.15.172:0/734295703 shutdown_connections 2024-04-14T08:33:26.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:26.261+0000 7f895b1fe640 1 -- 172.21.15.172:0/734295703 wait complete. 2024-04-14T08:33:26.273 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-14T08:33:26.273 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:33:26.273 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2024-04-14T08:33:26.273 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-14T08:33:26.273 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-14T08:33:26.273 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-14T08:33:26.273 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-14T08:33:26.273 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:33:26.273 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-14T08:33:26.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:26.273 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2024-04-14T08:33:26.288 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:33:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:33:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:33:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2024-04-14T08:33:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.289 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f0 2024-04-14T08:33:26.289 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:26.289 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f1 2024-04-14T08:33:26.289 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:33:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:26.290 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:33:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:33:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42652sec preferred_lft 42652sec 2024-04-14T08:33:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::ec4:7aff:feea:b04b/64 scope link 2024-04-14T08:33:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.290 INFO:teuthology.orchestra.run.smithi177.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:33:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 86:84:ab:c4:77:9e brd ff:ff:ff:ff:ff:ff 2024-04-14T08:33:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:33:26.290 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::acc0:52ff:fee9:a0e0/64 scope link 2024-04-14T08:33:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.291 INFO:teuthology.orchestra.run.smithi177.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:33:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 86:84:ab:c4:77:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-14T08:33:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::8484:abff:fec4:779e/64 scope link tentative 2024-04-14T08:33:26.291 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:26.291 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:33:26.355 INFO:teuthology.orchestra.run.smithi177.stdout:1 2024-04-14T08:33:26.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:26.356 DEBUG:teuthology.orchestra.run.smithi177:> route 2024-04-14T08:33:26.412 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2024-04-14T08:33:26.412 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:33:26.412 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:33:26.412 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:33:26.413 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:33:26.413 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:26.413 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-04-14T08:33:26.413 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:33:26.413 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:33:26.413 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:33:26.413 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-04-14T08:33:26.500 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:26.544 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:26.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:26.551 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2024-04-14T08:33:26.566 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-14T08:33:26.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:26.566 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list-id 2024-04-14T08:33:26.620 INFO:teuthology.orchestra.run.smithi177.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-14T08:33:26.620 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:26.620 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-04-14T08:33:26.620 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-14T08:33:26.620 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-14T08:33:26.621 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-04-14T08:33:26.707 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:26.735 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:26.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:26.742 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-14T08:33:26.768 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:33:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:33:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:33:26.768 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2024-04-14T08:33:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.769 INFO:teuthology.orchestra.run.smithi177.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:33:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether ea:28:bc:9a:07:49 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-14T08:33:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-14T08:33:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::e828:bcff:fe9a:749/64 scope link 2024-04-14T08:33:26.769 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:26.770 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-14T08:33:26.840 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:33:26.840 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:33:26.841 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:33:26.841 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.841 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2024-04-14T08:33:26.841 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.841 INFO:teuthology.orchestra.run.smithi177.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:33:26.841 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether ea:28:bc:9a:07:49 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-14T08:33:26.841 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-14T08:33:26.841 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.841 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::e828:bcff:fe9a:749/64 scope link 2024-04-14T08:33:26.841 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:33:26.842 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-14T08:33:26.842 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:26.843 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-04-14T08:33:26.843 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-14T08:33:26.843 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-14T08:33:26.843 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-14T08:33:26.843 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-14T08:33:26.843 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-14T08:33:26.843 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-04-14T08:33:26.929 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:27.002 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:27.008 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:33:27.008 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2024-04-14T08:33:27.008 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.1 up 2024-04-14T08:33:27.008 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set dev brx.1 master ceph-brx 2024-04-14T08:33:27.008 DEBUG:teuthology.orchestra.run.smithi177:> ') 2024-04-14T08:33:27.056 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:27.085 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:27.093 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:33:27.093 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-14T08:33:27.108 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-14T08:33:27.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:33:27.108 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-14T08:33:27.163 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:33:27.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:27.163 DEBUG:teuthology.orchestra.run.smithi177:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-14T08:33:27.248 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-14T08:33:27.248 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-14T08:33:27.249 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2024-04-14T08:33:27.256 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-14T08:33:27.257 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-14T08:33:27.257 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2024-04-14T08:33:27.257 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:33:27.257 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:33:27.262 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-14T08:33:27.262 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-14T08:33:27.262 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2024-04-14T08:33:27.262 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:33:27.262 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:33:27.262 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:33:27.262 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:33:27.263 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:33:27.263 INFO:teuthology.orchestra.run.smithi177.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-14T08:33:27.263 INFO:teuthology.orchestra.run.smithi177.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:33:27.263 INFO:teuthology.orchestra.run.smithi177.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:33:27.263 INFO:teuthology.orchestra.run.smithi177.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:33:27.263 INFO:teuthology.orchestra.run.smithi177.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:33:27.265 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:33:27.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:27.265 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-14T08:33:27.292 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:33:27.381 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:27.382 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:27.388 INFO:teuthology.orchestra.run.smithi177.stdout:client4622 2024-04-14T08:33:27.388 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:33:27.388 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:27.403 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:33:27.490 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:27.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:27.499 INFO:teuthology.orchestra.run.smithi177.stdout:client4622 2024-04-14T08:33:27.499 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:33:27.547 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:33:27.548 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:27.554 INFO:teuthology.orchestra.run.smithi177.stdout:client4622 2024-04-14T08:33:27.554 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8e7e3a2f-e38f-4857-b139-5df1f7a099dc.client4622/status') 2024-04-14T08:33:27.601 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:33:27.614 INFO:teuthology.orchestra.run.smithi177.stdout:instance: client.4622 (3)192.168.0.2:0/3329798946 2024-04-14T08:33:27.614 INFO:teuthology.orchestra.run.smithi177.stdout:blocklisted: false 2024-04-14T08:33:27.614 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records in 2024-04-14T08:33:27.614 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records out 2024-04-14T08:33:27.614 INFO:teuthology.orchestra.run.smithi177.stderr:69 bytes copied, 3.6576e-05 s, 1.9 MB/s 2024-04-14T08:33:27.615 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-14T08:33:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:33:27.621 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-14T08:33:27.636 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:27.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.748+0000 7efcb4dbb640 1 Processor -- start 2024-04-14T08:33:27.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.748+0000 7efcb4dbb640 1 -- start start 2024-04-14T08:33:27.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.749+0000 7efcb4dbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb00d2830 0x7efcb00d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:27.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.749+0000 7efcb4dbb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efcb00cedd0 0x7efcb00b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:27.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.749+0000 7efcb4dbb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb00d3150 con 0x7efcb015e890 2024-04-14T08:33:27.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.749+0000 7efcb4dbb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7efcb00cff30 con 0x7efcb00cedd0 2024-04-14T08:33:27.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.749+0000 7efcb4dbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efcb00d2350 con 0x7efcb00d2830 2024-04-14T08:33:27.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.750+0000 7efcae59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb00d2830 0x7efcb00d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.750+0000 7efcaed9d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efcb00cedd0 0x7efcb00b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.750+0000 7efcadd9b640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7efcb015e890 0x7efcb00d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:60450/0 (socket says 172.21.15.172:60450) 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.750+0000 7efcadd9b640 1 -- 172.21.15.172:0/797252638 learned_addr learned my addr 172.21.15.172:0/797252638 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.750+0000 7efcad59a640 1 -- 172.21.15.172:0/797252638 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1877364098 0 0) 0x7efcb00d3150 con 0x7efcb015e890 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.750+0000 7efcad59a640 1 -- 172.21.15.172:0/797252638 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc9c003520 con 0x7efcb015e890 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.750+0000 7efcae59c640 1 -- 172.21.15.172:0/797252638 >> v2:172.21.15.177:3300/0 conn(0x7efcb00cedd0 msgr2=0x7efcb00b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.750+0000 7efcae59c640 1 --2- 172.21.15.172:0/797252638 >> v2:172.21.15.177:3300/0 conn(0x7efcb00cedd0 0x7efcb00b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.750+0000 7efcae59c640 1 -- 172.21.15.172:0/797252638 >> v1:172.21.15.177:6789/0 conn(0x7efcb015e890 legacy=0x7efcb00d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.750+0000 7efcae59c640 1 -- 172.21.15.172:0/797252638 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb00cf320 con 0x7efcb00d2830 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.750+0000 7efcaed9d640 1 --2- 172.21.15.172:0/797252638 >> v2:172.21.15.177:3300/0 conn(0x7efcb00cedd0 0x7efcb00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.751+0000 7efcae59c640 1 --2- 172.21.15.172:0/797252638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb00d2830 0x7efcb00d2c10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efca80097a0 tx=0x7efca802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8af8479d389d5f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.751+0000 7efcad59a640 1 -- 172.21.15.172:0/797252638 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8034070 con 0x7efcb00d2830 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.751+0000 7efcad59a640 1 -- 172.21.15.172:0/797252638 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca8030610 con 0x7efcb00d2830 2024-04-14T08:33:27.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.751+0000 7efcad59a640 1 -- 172.21.15.172:0/797252638 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca80309c0 con 0x7efcb00d2830 2024-04-14T08:33:27.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.751+0000 7efcb4dbb640 1 -- 172.21.15.172:0/797252638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb00d2830 msgr2=0x7efcb00d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:27.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.751+0000 7efcb4dbb640 1 --2- 172.21.15.172:0/797252638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb00d2830 0x7efcb00d2c10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efca80097a0 tx=0x7efca802b960 comp rx=0 tx=0).stop 2024-04-14T08:33:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.751+0000 7efcb4dbb640 1 -- 172.21.15.172:0/797252638 shutdown_connections 2024-04-14T08:33:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.751+0000 7efcb4dbb640 1 --2- 172.21.15.172:0/797252638 >> v2:172.21.15.177:3300/0 conn(0x7efcb00cedd0 0x7efcb00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.751+0000 7efcb4dbb640 1 --2- 172.21.15.172:0/797252638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb00d2830 0x7efcb00d2c10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.751+0000 7efcb4dbb640 1 -- 172.21.15.172:0/797252638 >> 172.21.15.172:0/797252638 conn(0x7efcb009d590 msgr2=0x7efcb00d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.752+0000 7efcb4dbb640 1 -- 172.21.15.172:0/797252638 shutdown_connections 2024-04-14T08:33:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.752+0000 7efcb4dbb640 1 -- 172.21.15.172:0/797252638 wait complete. 2024-04-14T08:33:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.753+0000 7efcb4dbb640 1 Processor -- start 2024-04-14T08:33:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.753+0000 7efcb4dbb640 1 -- start start 2024-04-14T08:33:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcb4dbb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb00cedd0 0x7efcb00cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcb4dbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb00d2830 0x7efcb01f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcb4dbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015e890 0x7efcb01f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcb4dbb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efcb00cfbc0 con 0x7efcb015e890 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcb4dbb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efcb00cf8e0 con 0x7efcb00cedd0 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcb4dbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efcb00b3c50 con 0x7efcb00d2830 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcaed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015e890 0x7efcb01f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcae59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb00cedd0 0x7efcb00cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcaed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015e890 0x7efcb01f4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59460/0 (socket says 172.21.15.172:59460) 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcae59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb00cedd0 0x7efcb00cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:60620/0 (socket says 172.21.15.172:60620) 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcaed9d640 1 -- 172.21.15.172:0/1398494766 learned_addr learned my addr 172.21.15.172:0/1398494766 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcadd9b640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb00d2830 0x7efcb01f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcaed9d640 1 -- 172.21.15.172:0/1398494766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb00d2830 msgr2=0x7efcb01f15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcaed9d640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb00d2830 0x7efcb01f15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcaed9d640 1 -- 172.21.15.172:0/1398494766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb00cedd0 msgr2=0x7efcb00cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcaed9d640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb00cedd0 0x7efcb00cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcaed9d640 1 -- 172.21.15.172:0/1398494766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb00cca50 con 0x7efcb015e890 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcae59c640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb00cedd0 0x7efcb00cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcaed9d640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015e890 0x7efcb01f4ac0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efca000efc0 tx=0x7efca0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:27.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efca6ffd640 1 -- 172.21.15.172:0/1398494766 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0018030 con 0x7efcb015e890 2024-04-14T08:33:27.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efca6ffd640 1 -- 172.21.15.172:0/1398494766 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca000cde0 con 0x7efcb015e890 2024-04-14T08:33:27.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcb4dbb640 1 -- 172.21.15.172:0/1398494766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb00cb9b0 con 0x7efcb015e890 2024-04-14T08:33:27.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efca6ffd640 1 -- 172.21.15.172:0/1398494766 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0003bb0 con 0x7efcb015e890 2024-04-14T08:33:27.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.754+0000 7efcb4dbb640 1 -- 172.21.15.172:0/1398494766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb00cbf70 con 0x7efcb015e890 2024-04-14T08:33:27.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.755+0000 7efcb4dbb640 1 -- 172.21.15.172:0/1398494766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb0068520 con 0x7efcb015e890 2024-04-14T08:33:27.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.756+0000 7efca6ffd640 1 -- 172.21.15.172:0/1398494766 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efca0005d10 con 0x7efcb015e890 2024-04-14T08:33:27.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.756+0000 7efca6ffd640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efc9006c8f0 0x7efc9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:27.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.756+0000 7efca6ffd640 1 -- 172.21.15.172:0/1398494766 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efca008c7b0 con 0x7efcb015e890 2024-04-14T08:33:27.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.756+0000 7efcae59c640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efc9006c8f0 0x7efc9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:27.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.757+0000 7efcae59c640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efc9006c8f0 0x7efc9006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efca8036000 tx=0x7efca803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:27.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.761+0000 7efca6ffd640 1 -- 172.21.15.172:0/1398494766 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efca0022070 con 0x7efcb015e890 2024-04-14T08:33:27.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.938+0000 7efcb4dbb640 1 -- 172.21.15.172:0/1398494766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efcb00d2610 con 0x7efcb015e890 2024-04-14T08:33:27.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.940+0000 7efca6ffd640 1 -- 172.21.15.172:0/1398494766 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7efca0010040 con 0x7efcb015e890 2024-04-14T08:33:27.940 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:27.941 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:27.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.942+0000 7efcb4dbb640 1 -- 172.21.15.172:0/1398494766 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efc9006c8f0 msgr2=0x7efc9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:27.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.942+0000 7efcb4dbb640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efc9006c8f0 0x7efc9006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efca8036000 tx=0x7efca803b040 comp rx=0 tx=0).stop 2024-04-14T08:33:27.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.942+0000 7efcb4dbb640 1 -- 172.21.15.172:0/1398494766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015e890 msgr2=0x7efcb01f4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:27.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.942+0000 7efcb4dbb640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015e890 0x7efcb01f4ac0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efca000efc0 tx=0x7efca0009f90 comp rx=0 tx=0).stop 2024-04-14T08:33:27.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.942+0000 7efcb4dbb640 1 -- 172.21.15.172:0/1398494766 shutdown_connections 2024-04-14T08:33:27.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.942+0000 7efcb4dbb640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efc9006c8f0 0x7efc9006edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:27.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.942+0000 7efcb4dbb640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcb015e890 0x7efcb01f4ac0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:27.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.942+0000 7efcb4dbb640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcb00d2830 0x7efcb01f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:27.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.942+0000 7efcb4dbb640 1 --2- 172.21.15.172:0/1398494766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcb00cedd0 0x7efcb00cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:27.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.942+0000 7efcb4dbb640 1 -- 172.21.15.172:0/1398494766 >> 172.21.15.172:0/1398494766 conn(0x7efcb009d590 msgr2=0x7efcb00d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:27.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.942+0000 7efcb4dbb640 1 -- 172.21.15.172:0/1398494766 shutdown_connections 2024-04-14T08:33:27.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:27.942+0000 7efcb4dbb640 1 -- 172.21.15.172:0/1398494766 wait complete. 2024-04-14T08:33:27.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:27.954 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:28.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.061+0000 7efd25698640 1 Processor -- start 2024-04-14T08:33:28.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.061+0000 7efd25698640 1 -- start start 2024-04-14T08:33:28.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.062+0000 7efd25698640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efd200d0cc0 0x7efd200d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.062+0000 7efd25698640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd200d49c0 0x7efd200d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.062+0000 7efd25698640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd200d15e0 con 0x7efd20160f30 2024-04-14T08:33:28.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.062+0000 7efd25698640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7efd200ceec0 con 0x7efd200d0cc0 2024-04-14T08:33:28.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.062+0000 7efd25698640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efd200cfd00 con 0x7efd200d49c0 2024-04-14T08:33:28.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.062+0000 7efd1effd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efd200d0cc0 0x7efd200d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.062+0000 7efd1effd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efd200d0cc0 0x7efd200d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59464/0 (socket says 172.21.15.172:59464) 2024-04-14T08:33:28.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.062+0000 7efd1effd640 1 -- 172.21.15.172:0/48922000 learned_addr learned my addr 172.21.15.172:0/48922000 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:28.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.062+0000 7efd1e7fc640 1 --2- 172.21.15.172:0/48922000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd200d49c0 0x7efd200d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.063+0000 7efd1dffb640 1 -- 172.21.15.172:0/48922000 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1515519459 0 0) 0x7efd200d15e0 con 0x7efd20160f30 2024-04-14T08:33:28.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.063+0000 7efd1effd640 1 -- 172.21.15.172:0/48922000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd200d49c0 msgr2=0x7efd200d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:28.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.063+0000 7efd1effd640 1 --2- 172.21.15.172:0/48922000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd200d49c0 0x7efd200d4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.063+0000 7efd1effd640 1 -- 172.21.15.172:0/48922000 >> v1:172.21.15.177:6789/0 conn(0x7efd20160f30 legacy=0x7efd201613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.063+0000 7efd1effd640 1 -- 172.21.15.172:0/48922000 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd200cf0f0 con 0x7efd200d0cc0 2024-04-14T08:33:28.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.063+0000 7efd1e7fc640 1 --2- 172.21.15.172:0/48922000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd200d49c0 0x7efd200d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:28.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.063+0000 7efd1effd640 1 --2- 172.21.15.172:0/48922000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd200d0cc0 0x7efd200d10a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd10009870 tx=0x7efd1002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b9b915dd672f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:28.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.064+0000 7efd1dffb640 1 -- 172.21.15.172:0/48922000 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10034070 con 0x7efd200d0cc0 2024-04-14T08:33:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.064+0000 7efd1dffb640 1 -- 172.21.15.172:0/48922000 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd10030840 con 0x7efd200d0cc0 2024-04-14T08:33:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.064+0000 7efd1dffb640 1 -- 172.21.15.172:0/48922000 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10030c10 con 0x7efd200d0cc0 2024-04-14T08:33:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.064+0000 7efd25698640 1 -- 172.21.15.172:0/48922000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd200d0cc0 msgr2=0x7efd200d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.064+0000 7efd25698640 1 --2- 172.21.15.172:0/48922000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd200d0cc0 0x7efd200d10a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd10009870 tx=0x7efd1002b980 comp rx=0 tx=0).stop 2024-04-14T08:33:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.064+0000 7efd25698640 1 -- 172.21.15.172:0/48922000 shutdown_connections 2024-04-14T08:33:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.064+0000 7efd25698640 1 --2- 172.21.15.172:0/48922000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd200d49c0 0x7efd200d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.064+0000 7efd25698640 1 --2- 172.21.15.172:0/48922000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd200d0cc0 0x7efd200d10a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.064+0000 7efd25698640 1 -- 172.21.15.172:0/48922000 >> 172.21.15.172:0/48922000 conn(0x7efd2009d580 msgr2=0x7efd200d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.064+0000 7efd25698640 1 -- 172.21.15.172:0/48922000 shutdown_connections 2024-04-14T08:33:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.064+0000 7efd25698640 1 -- 172.21.15.172:0/48922000 wait complete. 2024-04-14T08:33:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd25698640 1 Processor -- start 2024-04-14T08:33:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd25698640 1 -- start start 2024-04-14T08:33:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd25698640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd200d0cc0 0x7efd200b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd25698640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd200d49c0 0x7efd201f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd25698640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd20160f30 0x7efd201f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd25698640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd200cf990 con 0x7efd20160f30 2024-04-14T08:33:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd25698640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efd200cf6b0 con 0x7efd200d49c0 2024-04-14T08:33:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd25698640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efd200d32b0 con 0x7efd200d0cc0 2024-04-14T08:33:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd20160f30 0x7efd201f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd20160f30 0x7efd201f4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59476/0 (socket says 172.21.15.172:59476) 2024-04-14T08:33:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd1f7fe640 1 -- 172.21.15.172:0/1609659754 learned_addr learned my addr 172.21.15.172:0/1609659754 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd1e7fc640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd200d49c0 0x7efd201f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.066+0000 7efd1effd640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd200d0cc0 0x7efd200b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.067+0000 7efd1f7fe640 1 -- 172.21.15.172:0/1609659754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd200d0cc0 msgr2=0x7efd200b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.067+0000 7efd1f7fe640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd200d0cc0 0x7efd200b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.067+0000 7efd1f7fe640 1 -- 172.21.15.172:0/1609659754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd200d49c0 msgr2=0x7efd201f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.067+0000 7efd1f7fe640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd200d49c0 0x7efd201f5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.067+0000 7efd1f7fe640 1 -- 172.21.15.172:0/1609659754 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd201f5850 con 0x7efd20160f30 2024-04-14T08:33:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.067+0000 7efd1f7fe640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd20160f30 0x7efd201f4160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efd1800c780 tx=0x7efd1800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.067+0000 7efd177fe640 1 -- 172.21.15.172:0/1609659754 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18003b70 con 0x7efd20160f30 2024-04-14T08:33:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.067+0000 7efd177fe640 1 -- 172.21.15.172:0/1609659754 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd18003d10 con 0x7efd20160f30 2024-04-14T08:33:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.067+0000 7efd25698640 1 -- 172.21.15.172:0/1609659754 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd201f4980 con 0x7efd20160f30 2024-04-14T08:33:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.067+0000 7efd177fe640 1 -- 172.21.15.172:0/1609659754 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18005270 con 0x7efd20160f30 2024-04-14T08:33:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.067+0000 7efd25698640 1 -- 172.21.15.172:0/1609659754 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd201f4f40 con 0x7efd20160f30 2024-04-14T08:33:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.068+0000 7efd25698640 1 -- 172.21.15.172:0/1609659754 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd20075490 con 0x7efd20160f30 2024-04-14T08:33:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.069+0000 7efd177fe640 1 -- 172.21.15.172:0/1609659754 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd18005410 con 0x7efd20160f30 2024-04-14T08:33:28.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.069+0000 7efd177fe640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efd0006c8f0 0x7efd0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.070+0000 7efd177fe640 1 -- 172.21.15.172:0/1609659754 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efd1808bb90 con 0x7efd20160f30 2024-04-14T08:33:28.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.070+0000 7efd1effd640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efd0006c8f0 0x7efd0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.070+0000 7efd1effd640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efd0006c8f0 0x7efd0006edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efd10036000 tx=0x7efd10008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:28.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.074+0000 7efd177fe640 1 -- 172.21.15.172:0/1609659754 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efd180596c0 con 0x7efd20160f30 2024-04-14T08:33:28.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.256+0000 7efd25698640 1 -- 172.21.15.172:0/1609659754 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd201f5110 con 0x7efd20160f30 2024-04-14T08:33:28.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.257+0000 7efd177fe640 1 -- 172.21.15.172:0/1609659754 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7efd201f5110 con 0x7efd20160f30 2024-04-14T08:33:28.257 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:28.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.259+0000 7efd25698640 1 -- 172.21.15.172:0/1609659754 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efd0006c8f0 msgr2=0x7efd0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.259+0000 7efd25698640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efd0006c8f0 0x7efd0006edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efd10036000 tx=0x7efd10008030 comp rx=0 tx=0).stop 2024-04-14T08:33:28.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.259+0000 7efd25698640 1 -- 172.21.15.172:0/1609659754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd20160f30 msgr2=0x7efd201f4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.259+0000 7efd25698640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd20160f30 0x7efd201f4160 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efd1800c780 tx=0x7efd1800cb80 comp rx=0 tx=0).stop 2024-04-14T08:33:28.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.259+0000 7efd25698640 1 -- 172.21.15.172:0/1609659754 shutdown_connections 2024-04-14T08:33:28.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.259+0000 7efd25698640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efd0006c8f0 0x7efd0006edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.259+0000 7efd25698640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd20160f30 0x7efd201f4160 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.259+0000 7efd25698640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd200d49c0 0x7efd201f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.259+0000 7efd25698640 1 --2- 172.21.15.172:0/1609659754 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd200d0cc0 0x7efd200b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.259+0000 7efd25698640 1 -- 172.21.15.172:0/1609659754 >> 172.21.15.172:0/1609659754 conn(0x7efd2009d580 msgr2=0x7efd2006ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:28.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.259+0000 7efd25698640 1 -- 172.21.15.172:0/1609659754 shutdown_connections 2024-04-14T08:33:28.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.259+0000 7efd25698640 1 -- 172.21.15.172:0/1609659754 wait complete. 2024-04-14T08:33:28.270 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:28.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.382+0000 7f24884d1640 1 Processor -- start 2024-04-14T08:33:28.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.383+0000 7f24884d1640 1 -- start start 2024-04-14T08:33:28.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.383+0000 7f24884d1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f248013df30 0x7f24800b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.383+0000 7f24884d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2480140e80 0x7f24800b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.383+0000 7f24884d1640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f248015b620 con 0x7f248015c380 2024-04-14T08:33:28.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.383+0000 7f24884d1640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f24800cf9b0 con 0x7f248013df30 2024-04-14T08:33:28.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.383+0000 7f24884d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f24801414a0 con 0x7f2480140e80 2024-04-14T08:33:28.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.384+0000 7f248626d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f248013df30 0x7f24800b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.384+0000 7f2485a6c640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f248015c380 0x7f2480140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:60464/0 (socket says 172.21.15.172:60464) 2024-04-14T08:33:28.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.384+0000 7f2485a6c640 1 -- 172.21.15.172:0/3354800653 learned_addr learned my addr 172.21.15.172:0/3354800653 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:28.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.384+0000 7f2486a6e640 1 --2- 172.21.15.172:0/3354800653 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2480140e80 0x7f24800b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.384+0000 7f248526b640 1 -- 172.21.15.172:0/3354800653 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800692456 0 0) 0x7f247c001110 con 0x7f248015c380 2024-04-14T08:33:28.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.384+0000 7f248526b640 1 -- 172.21.15.172:0/3354800653 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f246c005730 con 0x7f248015c380 2024-04-14T08:33:28.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.384+0000 7f248626d640 1 -- 172.21.15.172:0/3354800653 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2480140e80 msgr2=0x7f24800b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:28.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.384+0000 7f248626d640 1 --2- 172.21.15.172:0/3354800653 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2480140e80 0x7f24800b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.384+0000 7f248626d640 1 -- 172.21.15.172:0/3354800653 >> v1:172.21.15.177:6789/0 conn(0x7f248015c380 legacy=0x7f2480140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.384+0000 7f248626d640 1 -- 172.21.15.172:0/3354800653 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248015b8a0 con 0x7f248013df30 2024-04-14T08:33:28.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.385+0000 7f248626d640 1 --2- 172.21.15.172:0/3354800653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248013df30 0x7f24800b1cd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f24700097a0 tx=0x7f247002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f6ffc96c4b76ce server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:28.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.385+0000 7f248526b640 1 -- 172.21.15.172:0/3354800653 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470035020 con 0x7f248013df30 2024-04-14T08:33:28.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.385+0000 7f248526b640 1 -- 172.21.15.172:0/3354800653 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2470031830 con 0x7f248013df30 2024-04-14T08:33:28.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.385+0000 7f248526b640 1 -- 172.21.15.172:0/3354800653 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470031be0 con 0x7f248013df30 2024-04-14T08:33:28.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.386+0000 7f24884d1640 1 -- 172.21.15.172:0/3354800653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248013df30 msgr2=0x7f24800b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.386+0000 7f24884d1640 1 --2- 172.21.15.172:0/3354800653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248013df30 0x7f24800b1cd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f24700097a0 tx=0x7f247002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.386+0000 7f24884d1640 1 -- 172.21.15.172:0/3354800653 shutdown_connections 2024-04-14T08:33:28.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.386+0000 7f24884d1640 1 --2- 172.21.15.172:0/3354800653 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2480140e80 0x7f24800b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.386+0000 7f24884d1640 1 --2- 172.21.15.172:0/3354800653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248013df30 0x7f24800b1cd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.386+0000 7f24884d1640 1 -- 172.21.15.172:0/3354800653 >> 172.21.15.172:0/3354800653 conn(0x7f248006b520 msgr2=0x7f248006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:28.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.386+0000 7f24884d1640 1 -- 172.21.15.172:0/3354800653 shutdown_connections 2024-04-14T08:33:28.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.386+0000 7f24884d1640 1 -- 172.21.15.172:0/3354800653 wait complete. 2024-04-14T08:33:28.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.387+0000 7f24884d1640 1 Processor -- start 2024-04-14T08:33:28.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.387+0000 7f24884d1640 1 -- start start 2024-04-14T08:33:28.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f24884d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248013df30 0x7f2480139190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f24884d1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2480140e80 0x7f24801f0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f24884d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248015c380 0x7f248015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f24884d1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f248013e310 con 0x7f248015c380 2024-04-14T08:33:28.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f24884d1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f248015be60 con 0x7f2480140e80 2024-04-14T08:33:28.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f24884d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f24800ce9a0 con 0x7f248013df30 2024-04-14T08:33:28.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f2486a6e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248015c380 0x7f248015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f248626d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248013df30 0x7f2480139190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f2485a6c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2480140e80 0x7f24801f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f248626d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248013df30 0x7f2480139190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:56122/0 (socket says 172.21.15.172:56122) 2024-04-14T08:33:28.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f248626d640 1 -- 172.21.15.172:0/521643101 learned_addr learned my addr 172.21.15.172:0/521643101 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:28.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f248626d640 1 -- 172.21.15.172:0/521643101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2480140e80 msgr2=0x7f24801f0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f248626d640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2480140e80 0x7f24801f0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f248626d640 1 -- 172.21.15.172:0/521643101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248015c380 msgr2=0x7f248015c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f248626d640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248015c380 0x7f248015c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f248626d640 1 -- 172.21.15.172:0/521643101 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2480139a90 con 0x7f248013df30 2024-04-14T08:33:28.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f2486a6e640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248015c380 0x7f248015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:28.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f248626d640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248013df30 0x7f2480139190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f247002b910 tx=0x7f24700046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:28.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f2476ffd640 1 -- 172.21.15.172:0/521643101 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470035040 con 0x7f248013df30 2024-04-14T08:33:28.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f2476ffd640 1 -- 172.21.15.172:0/521643101 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2470004450 con 0x7f248013df30 2024-04-14T08:33:28.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f2476ffd640 1 -- 172.21.15.172:0/521643101 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470003810 con 0x7f248013df30 2024-04-14T08:33:28.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f24884d1640 1 -- 172.21.15.172:0/521643101 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248015d310 con 0x7f248013df30 2024-04-14T08:33:28.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.388+0000 7f24884d1640 1 -- 172.21.15.172:0/521643101 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f24801efd90 con 0x7f248013df30 2024-04-14T08:33:28.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.390+0000 7f24884d1640 1 -- 172.21.15.172:0/521643101 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24801efb80 con 0x7f248013df30 2024-04-14T08:33:28.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.390+0000 7f2476ffd640 1 -- 172.21.15.172:0/521643101 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2470038020 con 0x7f248013df30 2024-04-14T08:33:28.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.390+0000 7f2476ffd640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f246406c8f0 0x7f246406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.390+0000 7f2476ffd640 1 -- 172.21.15.172:0/521643101 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f24700add30 con 0x7f248013df30 2024-04-14T08:33:28.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.390+0000 7f2485a6c640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f246406c8f0 0x7f246406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.391+0000 7f2485a6c640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f246406c8f0 0x7f246406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f247c005880 tx=0x7f247c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:28.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.395+0000 7f2476ffd640 1 -- 172.21.15.172:0/521643101 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f247007b860 con 0x7f248013df30 2024-04-14T08:33:28.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.577+0000 7f24884d1640 1 -- 172.21.15.172:0/521643101 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2480068590 con 0x7f248013df30 2024-04-14T08:33:28.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.577+0000 7f2476ffd640 1 -- 172.21.15.172:0/521643101 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f247007fb60 con 0x7f248013df30 2024-04-14T08:33:28.578 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:28.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.579+0000 7f24884d1640 1 -- 172.21.15.172:0/521643101 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f246406c8f0 msgr2=0x7f246406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.579+0000 7f24884d1640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f246406c8f0 0x7f246406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f247c005880 tx=0x7f247c009040 comp rx=0 tx=0).stop 2024-04-14T08:33:28.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.579+0000 7f24884d1640 1 -- 172.21.15.172:0/521643101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248013df30 msgr2=0x7f2480139190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.579+0000 7f24884d1640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248013df30 0x7f2480139190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f247002b910 tx=0x7f24700046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.579+0000 7f24884d1640 1 -- 172.21.15.172:0/521643101 shutdown_connections 2024-04-14T08:33:28.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.579+0000 7f24884d1640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f246406c8f0 0x7f246406edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.579+0000 7f24884d1640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f248015c380 0x7f248015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.579+0000 7f24884d1640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2480140e80 0x7f24801f0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.579+0000 7f24884d1640 1 --2- 172.21.15.172:0/521643101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f248013df30 0x7f2480139190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.579+0000 7f24884d1640 1 -- 172.21.15.172:0/521643101 >> 172.21.15.172:0/521643101 conn(0x7f248006b520 msgr2=0x7f248013fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:28.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.579+0000 7f24884d1640 1 -- 172.21.15.172:0/521643101 shutdown_connections 2024-04-14T08:33:28.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.579+0000 7f24884d1640 1 -- 172.21.15.172:0/521643101 wait complete. 2024-04-14T08:33:28.591 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:28.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.696+0000 7f0aa7b8c640 1 Processor -- start 2024-04-14T08:33:28.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.697+0000 7f0aa7b8c640 1 -- start start 2024-04-14T08:33:28.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.697+0000 7f0aa7b8c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0aa015c420 0x7f0aa00b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.697+0000 7f0aa7b8c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0aa006f3e0 0x7f0aa00d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.697+0000 7f0aa7b8c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0aa00ceb00 0x7f0aa00cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.697+0000 7f0aa7b8c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa00d30a0 con 0x7f0aa006f3e0 2024-04-14T08:33:28.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.697+0000 7f0aa7b8c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa006f870 con 0x7f0aa015c420 2024-04-14T08:33:28.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.697+0000 7f0aa7b8c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa00cf470 con 0x7f0aa00ceb00 2024-04-14T08:33:28.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.697+0000 7f0aa5127640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0aa006f3e0 0x7f0aa00d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.698+0000 7f0aa5127640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0aa006f3e0 0x7f0aa00d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59514/0 (socket says 172.21.15.172:59514) 2024-04-14T08:33:28.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.698+0000 7f0aa5127640 1 -- 172.21.15.172:0/2817439606 learned_addr learned my addr 172.21.15.172:0/2817439606 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:28.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.698+0000 7f0aa5928640 1 --2- 172.21.15.172:0/2817439606 >> v2:172.21.15.177:3300/0 conn(0x7f0aa015c420 0x7f0aa00b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.698+0000 7f0aa6129640 1 --2- 172.21.15.172:0/2817439606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0aa00ceb00 0x7f0aa00cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.698+0000 7f0aa5127640 1 -- 172.21.15.172:0/2817439606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0aa00ceb00 msgr2=0x7f0aa00cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:28.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.698+0000 7f0aa5127640 1 --2- 172.21.15.172:0/2817439606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0aa00ceb00 0x7f0aa00cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.698+0000 7f0aa5127640 1 -- 172.21.15.172:0/2817439606 >> v2:172.21.15.177:3300/0 conn(0x7f0aa015c420 msgr2=0x7f0aa00b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:28.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.698+0000 7f0aa5127640 1 --2- 172.21.15.172:0/2817439606 >> v2:172.21.15.177:3300/0 conn(0x7f0aa015c420 0x7f0aa00b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.698+0000 7f0aa5127640 1 -- 172.21.15.172:0/2817439606 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa00d0550 con 0x7f0aa006f3e0 2024-04-14T08:33:28.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.698+0000 7f0aa5928640 1 --2- 172.21.15.172:0/2817439606 >> v2:172.21.15.177:3300/0 conn(0x7f0aa015c420 0x7f0aa00b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:28.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa5127640 1 --2- 172.21.15.172:0/2817439606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa006f3e0 0x7f0aa00d0010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0aa008c8d0 tx=0x7f0a9c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85c1bc26cd85e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:28.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa4926640 1 -- 172.21.15.172:0/2817439606 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c035020 con 0x7f0aa006f3e0 2024-04-14T08:33:28.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa4926640 1 -- 172.21.15.172:0/2817439606 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9c031640 con 0x7f0aa006f3e0 2024-04-14T08:33:28.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa4926640 1 -- 172.21.15.172:0/2817439606 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c02d040 con 0x7f0aa006f3e0 2024-04-14T08:33:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/2817439606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa006f3e0 msgr2=0x7f0aa00d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa7b8c640 1 --2- 172.21.15.172:0/2817439606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa006f3e0 0x7f0aa00d0010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0aa008c8d0 tx=0x7f0a9c02b8f0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/2817439606 shutdown_connections 2024-04-14T08:33:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa7b8c640 1 --2- 172.21.15.172:0/2817439606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0aa00ceb00 0x7f0aa00cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa7b8c640 1 --2- 172.21.15.172:0/2817439606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa006f3e0 0x7f0aa00d0010 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa7b8c640 1 --2- 172.21.15.172:0/2817439606 >> v2:172.21.15.177:3300/0 conn(0x7f0aa015c420 0x7f0aa00b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/2817439606 >> 172.21.15.172:0/2817439606 conn(0x7f0aa008c000 msgr2=0x7f0aa00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/2817439606 shutdown_connections 2024-04-14T08:33:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.699+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/2817439606 wait complete. 2024-04-14T08:33:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa7b8c640 1 Processor -- start 2024-04-14T08:33:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa7b8c640 1 -- start start 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa7b8c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0aa00ceb00 0x7f0aa00b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa7b8c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0aa015c420 0x7f0aa00b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa7b8c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa014d4f0 0x7f0aa00ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa7b8c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa00d2dc0 con 0x7f0aa014d4f0 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa7b8c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa00ccdd0 con 0x7f0aa00ceb00 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa7b8c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa00cd070 con 0x7f0aa015c420 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa6129640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa014d4f0 0x7f0aa00ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa6129640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa014d4f0 0x7f0aa00ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59524/0 (socket says 172.21.15.172:59524) 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa6129640 1 -- 172.21.15.172:0/1504579132 learned_addr learned my addr 172.21.15.172:0/1504579132 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa5928640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0aa00ceb00 0x7f0aa00b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.701+0000 7f0aa5127640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0aa015c420 0x7f0aa00b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.702+0000 7f0aa6129640 1 -- 172.21.15.172:0/1504579132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0aa015c420 msgr2=0x7f0aa00b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.702+0000 7f0aa6129640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0aa015c420 0x7f0aa00b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.702+0000 7f0aa6129640 1 -- 172.21.15.172:0/1504579132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0aa00ceb00 msgr2=0x7f0aa00b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.702+0000 7f0aa6129640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0aa00ceb00 0x7f0aa00b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.702+0000 7f0aa6129640 1 -- 172.21.15.172:0/1504579132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa00b5310 con 0x7f0aa014d4f0 2024-04-14T08:33:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.702+0000 7f0aa6129640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa014d4f0 0x7f0aa00ca770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009dd0 tx=0x7f0a9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.702+0000 7f0a967fc640 1 -- 172.21.15.172:0/1504579132 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98018020 con 0x7f0aa014d4f0 2024-04-14T08:33:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.702+0000 7f0a967fc640 1 -- 172.21.15.172:0/1504579132 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a98014480 con 0x7f0aa014d4f0 2024-04-14T08:33:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.702+0000 7f0a967fc640 1 -- 172.21.15.172:0/1504579132 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98014870 con 0x7f0aa014d4f0 2024-04-14T08:33:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.702+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/1504579132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa00b55f0 con 0x7f0aa014d4f0 2024-04-14T08:33:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.702+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/1504579132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa00b5bb0 con 0x7f0aa014d4f0 2024-04-14T08:33:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.703+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/1504579132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa0075490 con 0x7f0aa014d4f0 2024-04-14T08:33:28.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.704+0000 7f0a967fc640 1 -- 172.21.15.172:0/1504579132 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a98005d10 con 0x7f0aa014d4f0 2024-04-14T08:33:28.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.704+0000 7f0a967fc640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a8406c8f0 0x7f0a8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:28.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.704+0000 7f0aa5928640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a8406c8f0 0x7f0a8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:28.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.704+0000 7f0a967fc640 1 -- 172.21.15.172:0/1504579132 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0a9808c920 con 0x7f0aa014d4f0 2024-04-14T08:33:28.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.705+0000 7f0aa5928640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a8406c8f0 0x7f0a8406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009560 tx=0x7f0a90009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.713+0000 7f0a967fc640 1 -- 172.21.15.172:0/1504579132 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0a98022070 con 0x7f0aa014d4f0 2024-04-14T08:33:28.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.894+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/1504579132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0aa0067670 con 0x7f0aa014d4f0 2024-04-14T08:33:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.895+0000 7f0a967fc640 1 -- 172.21.15.172:0/1504579132 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f0a98010040 con 0x7f0aa014d4f0 2024-04-14T08:33:28.895 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.897+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/1504579132 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a8406c8f0 msgr2=0x7f0a8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.897+0000 7f0aa7b8c640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a8406c8f0 0x7f0a8406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009560 tx=0x7f0a90009190 comp rx=0 tx=0).stop 2024-04-14T08:33:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.897+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/1504579132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa014d4f0 msgr2=0x7f0aa00ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.897+0000 7f0aa7b8c640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa014d4f0 0x7f0aa00ca770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009dd0 tx=0x7f0a9800c490 comp rx=0 tx=0).stop 2024-04-14T08:33:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.897+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/1504579132 shutdown_connections 2024-04-14T08:33:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.897+0000 7f0aa7b8c640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a8406c8f0 0x7f0a8406edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.897+0000 7f0aa7b8c640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa014d4f0 0x7f0aa00ca770 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.897+0000 7f0aa7b8c640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0aa015c420 0x7f0aa00b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.897+0000 7f0aa7b8c640 1 --2- 172.21.15.172:0/1504579132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0aa00ceb00 0x7f0aa00b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.897+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/1504579132 >> 172.21.15.172:0/1504579132 conn(0x7f0aa008c000 msgr2=0x7f0aa00b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.897+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/1504579132 shutdown_connections 2024-04-14T08:33:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:28.897+0000 7f0aa7b8c640 1 -- 172.21.15.172:0/1504579132 wait complete. 2024-04-14T08:33:28.908 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:29.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.013+0000 7fb71b688640 1 Processor -- start 2024-04-14T08:33:29.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.013+0000 7fb71b688640 1 -- start start 2024-04-14T08:33:29.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.014+0000 7fb71b688640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb7140cffb0 0x7fb7140cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.014+0000 7fb71b688640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7140d0860 con 0x7fb7140d1a10 2024-04-14T08:33:29.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.014+0000 7fb71b688640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7140d0b40 con 0x7fb7140ccdc0 2024-04-14T08:33:29.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.014+0000 7fb71b688640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb7140d0610 con 0x7fb7140cffb0 2024-04-14T08:33:29.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.014+0000 7fb719c25640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb7140cffb0 0x7fb7140cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.014+0000 7fb719c25640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb7140cffb0 0x7fb7140cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59532/0 (socket says 172.21.15.172:59532) 2024-04-14T08:33:29.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.014+0000 7fb719c25640 1 -- 172.21.15.172:0/2564740716 learned_addr learned my addr 172.21.15.172:0/2564740716 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:29.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.014+0000 7fb718c23640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb7140d1a10 0x7fb71415eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48890/0 (socket says 172.21.15.172:48890) 2024-04-14T08:33:29.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.014+0000 7fb70bfff640 1 -- 172.21.15.172:0/2564740716 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407654293 0 0) 0x7fb7140d0b40 con 0x7fb7140ccdc0 2024-04-14T08:33:29.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.014+0000 7fb719c25640 1 -- 172.21.15.172:0/2564740716 >> v1:172.21.15.177:6789/0 conn(0x7fb7140ccdc0 legacy=0x7fb7140cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.014+0000 7fb719c25640 1 -- 172.21.15.172:0/2564740716 >> v1:172.21.15.172:6789/0 conn(0x7fb7140d1a10 legacy=0x7fb71415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.014+0000 7fb719c25640 1 -- 172.21.15.172:0/2564740716 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71415f1f0 con 0x7fb7140cffb0 2024-04-14T08:33:29.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.015+0000 7fb719c25640 1 --2- 172.21.15.172:0/2564740716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7140cffb0 0x7fb7140cf190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009e50 tx=0x7fb70c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1385e031266c36d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:29.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.015+0000 7fb70bfff640 1 -- 172.21.15.172:0/2564740716 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c02cb20 con 0x7fb7140cffb0 2024-04-14T08:33:29.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.015+0000 7fb70bfff640 1 -- 172.21.15.172:0/2564740716 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70c02ccc0 con 0x7fb7140cffb0 2024-04-14T08:33:29.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.015+0000 7fb70bfff640 1 -- 172.21.15.172:0/2564740716 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c034610 con 0x7fb7140cffb0 2024-04-14T08:33:29.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.016+0000 7fb71b688640 1 -- 172.21.15.172:0/2564740716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7140cffb0 msgr2=0x7fb7140cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.016+0000 7fb71b688640 1 --2- 172.21.15.172:0/2564740716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7140cffb0 0x7fb7140cf190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009e50 tx=0x7fb70c02bf80 comp rx=0 tx=0).stop 2024-04-14T08:33:29.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.016+0000 7fb71b688640 1 -- 172.21.15.172:0/2564740716 shutdown_connections 2024-04-14T08:33:29.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.016+0000 7fb71b688640 1 --2- 172.21.15.172:0/2564740716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7140cffb0 0x7fb7140cf190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.016+0000 7fb71b688640 1 -- 172.21.15.172:0/2564740716 >> 172.21.15.172:0/2564740716 conn(0x7fb7140d0f20 msgr2=0x7fb7140d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:29.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.016+0000 7fb71b688640 1 -- 172.21.15.172:0/2564740716 shutdown_connections 2024-04-14T08:33:29.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.016+0000 7fb71b688640 1 -- 172.21.15.172:0/2564740716 wait complete. 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.017+0000 7fb71b688640 1 Processor -- start 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.017+0000 7fb71b688640 1 -- start start 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb71b688640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7140ccdc0 0x7fb714159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb71b688640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7140cffb0 0x7fb7140a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb71b688640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7140d1a10 0x7fb7140a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb71b688640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb7140cf6d0 con 0x7fb7140cffb0 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb71b688640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb7140ced00 con 0x7fb7140d1a10 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb71b688640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb7140cf850 con 0x7fb7140ccdc0 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb718c23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7140cffb0 0x7fb7140a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb719c25640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7140d1a10 0x7fb7140a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb718c23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7140cffb0 0x7fb7140a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59538/0 (socket says 172.21.15.172:59538) 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb718c23640 1 -- 172.21.15.172:0/896688406 learned_addr learned my addr 172.21.15.172:0/896688406 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:29.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb719424640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7140ccdc0 0x7fb714159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb718c23640 1 -- 172.21.15.172:0/896688406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7140ccdc0 msgr2=0x7fb714159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb718c23640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7140ccdc0 0x7fb714159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb718c23640 1 -- 172.21.15.172:0/896688406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7140d1a10 msgr2=0x7fb7140a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb718c23640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7140d1a10 0x7fb7140a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb718c23640 1 -- 172.21.15.172:0/896688406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7140a0030 con 0x7fb7140cffb0 2024-04-14T08:33:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb719c25640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7140d1a10 0x7fb7140a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.018+0000 7fb718c23640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7140cffb0 0x7fb7140a1a30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb71000c430 tx=0x7fb71000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.019+0000 7fb709ffb640 1 -- 172.21.15.172:0/896688406 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7100074e0 con 0x7fb7140cffb0 2024-04-14T08:33:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.019+0000 7fb709ffb640 1 -- 172.21.15.172:0/896688406 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb710007680 con 0x7fb7140cffb0 2024-04-14T08:33:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.019+0000 7fb71b688640 1 -- 172.21.15.172:0/896688406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7140a0310 con 0x7fb7140cffb0 2024-04-14T08:33:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.019+0000 7fb709ffb640 1 -- 172.21.15.172:0/896688406 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710004a00 con 0x7fb7140cffb0 2024-04-14T08:33:29.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.019+0000 7fb71b688640 1 -- 172.21.15.172:0/896688406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7140a08d0 con 0x7fb7140cffb0 2024-04-14T08:33:29.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.020+0000 7fb71b688640 1 -- 172.21.15.172:0/896688406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb714075490 con 0x7fb7140cffb0 2024-04-14T08:33:29.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.021+0000 7fb709ffb640 1 -- 172.21.15.172:0/896688406 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb71001a020 con 0x7fb7140cffb0 2024-04-14T08:33:29.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.021+0000 7fb709ffb640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb6f806c8f0 0x7fb6f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.021+0000 7fb709ffb640 1 -- 172.21.15.172:0/896688406 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb71008b560 con 0x7fb7140cffb0 2024-04-14T08:33:29.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.021+0000 7fb719424640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb6f806c8f0 0x7fb6f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.022+0000 7fb719424640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb6f806c8f0 0x7fb6f806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb704000f60 tx=0x7fb704009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:29.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.026+0000 7fb709ffb640 1 -- 172.21.15.172:0/896688406 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb710059000 con 0x7fb7140cffb0 2024-04-14T08:33:29.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.205+0000 7fb71b688640 1 -- 172.21.15.172:0/896688406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb7140c4190 con 0x7fb7140cffb0 2024-04-14T08:33:29.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.205+0000 7fb709ffb640 1 -- 172.21.15.172:0/896688406 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fb71005d300 con 0x7fb7140cffb0 2024-04-14T08:33:29.206 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:29.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.207+0000 7fb71b688640 1 -- 172.21.15.172:0/896688406 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb6f806c8f0 msgr2=0x7fb6f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.207+0000 7fb71b688640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb6f806c8f0 0x7fb6f806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb704000f60 tx=0x7fb704009040 comp rx=0 tx=0).stop 2024-04-14T08:33:29.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.207+0000 7fb71b688640 1 -- 172.21.15.172:0/896688406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7140cffb0 msgr2=0x7fb7140a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.207+0000 7fb71b688640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7140cffb0 0x7fb7140a1a30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb71000c430 tx=0x7fb71000c8f0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.207+0000 7fb71b688640 1 -- 172.21.15.172:0/896688406 shutdown_connections 2024-04-14T08:33:29.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.207+0000 7fb71b688640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb6f806c8f0 0x7fb6f806edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.207+0000 7fb71b688640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7140d1a10 0x7fb7140a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.207+0000 7fb71b688640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7140cffb0 0x7fb7140a1a30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.208+0000 7fb71b688640 1 --2- 172.21.15.172:0/896688406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7140ccdc0 0x7fb714159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.208+0000 7fb71b688640 1 -- 172.21.15.172:0/896688406 >> 172.21.15.172:0/896688406 conn(0x7fb7140d0f20 msgr2=0x7fb7140cd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:29.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.208+0000 7fb71b688640 1 -- 172.21.15.172:0/896688406 shutdown_connections 2024-04-14T08:33:29.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.208+0000 7fb71b688640 1 -- 172.21.15.172:0/896688406 wait complete. 2024-04-14T08:33:29.219 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:29.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.320+0000 7f387dd1e640 1 Processor -- start 2024-04-14T08:33:29.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.320+0000 7f387dd1e640 1 -- start start 2024-04-14T08:33:29.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.321+0000 7f387dd1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38780cf270 0x7f38780ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.321+0000 7f387dd1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38780d0160 con 0x7f387815b310 2024-04-14T08:33:29.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.321+0000 7f387dd1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38780cc1b0 con 0x7f38780cf7a0 2024-04-14T08:33:29.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.321+0000 7f387dd1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f38780cc490 con 0x7f38780cf270 2024-04-14T08:33:29.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.321+0000 7f38777fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f387815b310 0x7f387806e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48894/0 (socket says 172.21.15.172:48894) 2024-04-14T08:33:29.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.321+0000 7f38777fe640 1 -- 172.21.15.172:0/411439326 learned_addr learned my addr 172.21.15.172:0/411439326 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:29.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.321+0000 7f3877fff640 1 --2- 172.21.15.172:0/411439326 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38780cf270 0x7f38780ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.321+0000 7f3877fff640 1 -- 172.21.15.172:0/411439326 >> v1:172.21.15.177:6789/0 conn(0x7f38780cf7a0 legacy=0x7f38780ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.321+0000 7f3877fff640 1 -- 172.21.15.172:0/411439326 >> v1:172.21.15.172:6789/0 conn(0x7f387815b310 legacy=0x7f387806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.321+0000 7f3877fff640 1 -- 172.21.15.172:0/411439326 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38780cc770 con 0x7f38780cf270 2024-04-14T08:33:29.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.322+0000 7f3877fff640 1 --2- 172.21.15.172:0/411439326 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38780cf270 0x7f38780ccc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3870009dc0 tx=0x7f387002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70bacaf6abdc8aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:29.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.322+0000 7f38767fc640 1 -- 172.21.15.172:0/411439326 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870039070 con 0x7f38780cf270 2024-04-14T08:33:29.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.322+0000 7f38767fc640 1 -- 172.21.15.172:0/411439326 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f387002ccd0 con 0x7f38780cf270 2024-04-14T08:33:29.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.322+0000 7f38767fc640 1 -- 172.21.15.172:0/411439326 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870034660 con 0x7f38780cf270 2024-04-14T08:33:29.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.323+0000 7f387dd1e640 1 -- 172.21.15.172:0/411439326 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38780cf270 msgr2=0x7f38780ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.323+0000 7f387dd1e640 1 --2- 172.21.15.172:0/411439326 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38780cf270 0x7f38780ccc90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3870009dc0 tx=0x7f387002bf20 comp rx=0 tx=0).stop 2024-04-14T08:33:29.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.323+0000 7f387dd1e640 1 -- 172.21.15.172:0/411439326 shutdown_connections 2024-04-14T08:33:29.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.323+0000 7f387dd1e640 1 --2- 172.21.15.172:0/411439326 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38780cf270 0x7f38780ccc90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.323+0000 7f387dd1e640 1 -- 172.21.15.172:0/411439326 >> 172.21.15.172:0/411439326 conn(0x7f38780b2b40 msgr2=0x7f38780b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:29.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.323+0000 7f387dd1e640 1 -- 172.21.15.172:0/411439326 shutdown_connections 2024-04-14T08:33:29.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.323+0000 7f387dd1e640 1 -- 172.21.15.172:0/411439326 wait complete. 2024-04-14T08:33:29.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.324+0000 7f387dd1e640 1 Processor -- start 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.324+0000 7f387dd1e640 1 -- start start 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.325+0000 7f387dd1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38780cf270 0x7f3878153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.325+0000 7f387dd1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38780cf7a0 0x7f38780ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.325+0000 7f387dd1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f387815b310 0x7f38780caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.325+0000 7f387dd1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3878073540 con 0x7f38780cf7a0 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.325+0000 7f387dd1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f38780733c0 con 0x7f38780cf270 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.325+0000 7f387dd1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f38780cfc00 con 0x7f387815b310 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.325+0000 7f3876ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38780cf7a0 0x7f38780ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.325+0000 7f3876ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38780cf7a0 0x7f38780ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59540/0 (socket says 172.21.15.172:59540) 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.325+0000 7f3876ffd640 1 -- 172.21.15.172:0/4240693232 learned_addr learned my addr 172.21.15.172:0/4240693232 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.325+0000 7f38777fe640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38780cf270 0x7f3878153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.325+0000 7f3877fff640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f387815b310 0x7f38780caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.326+0000 7f3876ffd640 1 -- 172.21.15.172:0/4240693232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f387815b310 msgr2=0x7f38780caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.326+0000 7f3876ffd640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f387815b310 0x7f38780caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.326+0000 7f3876ffd640 1 -- 172.21.15.172:0/4240693232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38780cf270 msgr2=0x7f3878153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.326+0000 7f3876ffd640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38780cf270 0x7f3878153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.326+0000 7f3876ffd640 1 -- 172.21.15.172:0/4240693232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38780c8190 con 0x7f38780cf7a0 2024-04-14T08:33:29.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.326+0000 7f3876ffd640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38780cf7a0 0x7f38780ca510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f386c00c400 tx=0x7f386c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:29.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.326+0000 7f385ffff640 1 -- 172.21.15.172:0/4240693232 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c0052d0 con 0x7f38780cf7a0 2024-04-14T08:33:29.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.326+0000 7f385ffff640 1 -- 172.21.15.172:0/4240693232 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f386c005490 con 0x7f38780cf7a0 2024-04-14T08:33:29.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.326+0000 7f387dd1e640 1 -- 172.21.15.172:0/4240693232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38780c8430 con 0x7f38780cf7a0 2024-04-14T08:33:29.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.326+0000 7f385ffff640 1 -- 172.21.15.172:0/4240693232 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c007260 con 0x7f38780cf7a0 2024-04-14T08:33:29.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.326+0000 7f387dd1e640 1 -- 172.21.15.172:0/4240693232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38780c89f0 con 0x7f38780cf7a0 2024-04-14T08:33:29.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.327+0000 7f387dd1e640 1 -- 172.21.15.172:0/4240693232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38780cb2a0 con 0x7f38780cf7a0 2024-04-14T08:33:29.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.327+0000 7f385ffff640 1 -- 172.21.15.172:0/4240693232 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f386c007400 con 0x7f38780cf7a0 2024-04-14T08:33:29.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.328+0000 7f385ffff640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f386406c8f0 0x7f386406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.328+0000 7f385ffff640 1 -- 172.21.15.172:0/4240693232 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f386c08b2e0 con 0x7f38780cf7a0 2024-04-14T08:33:29.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.328+0000 7f38777fe640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f386406c8f0 0x7f386406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.328+0000 7f38777fe640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f386406c8f0 0x7f386406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3868005880 tx=0x7f3868009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:29.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.332+0000 7f385ffff640 1 -- 172.21.15.172:0/4240693232 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f38780cb2a0 con 0x7f38780cf7a0 2024-04-14T08:33:29.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.507+0000 7f387dd1e640 1 -- 172.21.15.172:0/4240693232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3878075490 con 0x7f38780cf7a0 2024-04-14T08:33:29.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.509+0000 7f385ffff640 1 -- 172.21.15.172:0/4240693232 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7f386c058e10 con 0x7f38780cf7a0 2024-04-14T08:33:29.509 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:29.509 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.510+0000 7f387dd1e640 1 -- 172.21.15.172:0/4240693232 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f386406c8f0 msgr2=0x7f386406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.510+0000 7f387dd1e640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f386406c8f0 0x7f386406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3868005880 tx=0x7f3868009040 comp rx=0 tx=0).stop 2024-04-14T08:33:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.510+0000 7f387dd1e640 1 -- 172.21.15.172:0/4240693232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38780cf7a0 msgr2=0x7f38780ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.510+0000 7f387dd1e640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38780cf7a0 0x7f38780ca510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f386c00c400 tx=0x7f386c00c8c0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.511+0000 7f387dd1e640 1 -- 172.21.15.172:0/4240693232 shutdown_connections 2024-04-14T08:33:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.511+0000 7f387dd1e640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f386406c8f0 0x7f386406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.511+0000 7f387dd1e640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f387815b310 0x7f38780caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.511+0000 7f387dd1e640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38780cf7a0 0x7f38780ca510 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.511+0000 7f387dd1e640 1 --2- 172.21.15.172:0/4240693232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38780cf270 0x7f3878153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.511+0000 7f387dd1e640 1 -- 172.21.15.172:0/4240693232 >> 172.21.15.172:0/4240693232 conn(0x7f38780b2b40 msgr2=0x7f38780b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.511+0000 7f387dd1e640 1 -- 172.21.15.172:0/4240693232 shutdown_connections 2024-04-14T08:33:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.511+0000 7f387dd1e640 1 -- 172.21.15.172:0/4240693232 wait complete. 2024-04-14T08:33:29.522 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:29.523 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:29.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.627+0000 7fa0f8d1e640 1 Processor -- start 2024-04-14T08:33:29.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.627+0000 7fa0f8d1e640 1 -- start start 2024-04-14T08:33:29.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.627+0000 7fa0f8d1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0f413df30 0x7fa0f40b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.627+0000 7fa0f8d1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0f415b620 con 0x7fa0f415c380 2024-04-14T08:33:29.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.627+0000 7fa0f8d1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0f41414a0 con 0x7fa0f4140e80 2024-04-14T08:33:29.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.627+0000 7fa0f8d1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa0f415b8a0 con 0x7fa0f413df30 2024-04-14T08:33:29.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.628+0000 7fa0f259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0f413df30 0x7fa0f40b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.628+0000 7fa0ebfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa0f415c380 0x7fa0f4140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46462/0 (socket says 172.21.15.172:46462) 2024-04-14T08:33:29.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.628+0000 7fa0f259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0f413df30 0x7fa0f40b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37006/0 (socket says 172.21.15.172:37006) 2024-04-14T08:33:29.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.628+0000 7fa0ebfff640 1 -- 172.21.15.172:0/1315137074 learned_addr learned my addr 172.21.15.172:0/1315137074 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:29.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.628+0000 7fa0f259c640 1 -- 172.21.15.172:0/1315137074 >> v1:172.21.15.177:6789/0 conn(0x7fa0f4140e80 legacy=0x7fa0f40ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.628+0000 7fa0f259c640 1 -- 172.21.15.172:0/1315137074 >> v1:172.21.15.172:6789/0 conn(0x7fa0f415c380 legacy=0x7fa0f4140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.628+0000 7fa0f1d9b640 1 -- 172.21.15.172:0/1315137074 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3188949700 0 0) 0x7fa0f41414a0 con 0x7fa0f4140e80 2024-04-14T08:33:29.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.628+0000 7fa0f259c640 1 -- 172.21.15.172:0/1315137074 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f415bb80 con 0x7fa0f413df30 2024-04-14T08:33:29.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.629+0000 7fa0f259c640 1 --2- 172.21.15.172:0/1315137074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0f413df30 0x7fa0f40b1cd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40097a0 tx=0x7fa0e402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa16ca3ed42026f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:29.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.629+0000 7fa0f1d9b640 1 -- 172.21.15.172:0/1315137074 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4037070 con 0x7fa0f413df30 2024-04-14T08:33:29.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.629+0000 7fa0f1d9b640 1 -- 172.21.15.172:0/1315137074 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e4030710 con 0x7fa0f413df30 2024-04-14T08:33:29.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.629+0000 7fa0f1d9b640 1 -- 172.21.15.172:0/1315137074 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4030ae0 con 0x7fa0f413df30 2024-04-14T08:33:29.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.629+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/1315137074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0f413df30 msgr2=0x7fa0f40b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.629+0000 7fa0f8d1e640 1 --2- 172.21.15.172:0/1315137074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0f413df30 0x7fa0f40b1cd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40097a0 tx=0x7fa0e402b790 comp rx=0 tx=0).stop 2024-04-14T08:33:29.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.629+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/1315137074 shutdown_connections 2024-04-14T08:33:29.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.629+0000 7fa0f8d1e640 1 --2- 172.21.15.172:0/1315137074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0f413df30 0x7fa0f40b1cd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.629+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/1315137074 >> 172.21.15.172:0/1315137074 conn(0x7fa0f406b520 msgr2=0x7fa0f406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:29.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.629+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/1315137074 shutdown_connections 2024-04-14T08:33:29.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.629+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/1315137074 wait complete. 2024-04-14T08:33:29.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.631+0000 7fa0f8d1e640 1 Processor -- start 2024-04-14T08:33:29.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.631+0000 7fa0f8d1e640 1 -- start start 2024-04-14T08:33:29.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.631+0000 7fa0f8d1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0f413df30 0x7fa0f413fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.631+0000 7fa0f8d1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0f4140e80 0x7fa0f4155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.631+0000 7fa0f8d1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0f415c380 0x7fa0f4156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.631+0000 7fa0f8d1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f40b2210 con 0x7fa0f413df30 2024-04-14T08:33:29.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.631+0000 7fa0f8d1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f413e310 con 0x7fa0f415c380 2024-04-14T08:33:29.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.631+0000 7fa0f8d1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f40b2390 con 0x7fa0f4140e80 2024-04-14T08:33:29.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.631+0000 7fa0f259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0f413df30 0x7fa0f413fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0f259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0f413df30 0x7fa0f413fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37020/0 (socket says 172.21.15.172:37020) 2024-04-14T08:33:29.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0f259c640 1 -- 172.21.15.172:0/2617234344 learned_addr learned my addr 172.21.15.172:0/2617234344 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:29.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0ebfff640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0f4140e80 0x7fa0f4155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0f2d9d640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0f415c380 0x7fa0f4156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0f259c640 1 -- 172.21.15.172:0/2617234344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0f4140e80 msgr2=0x7fa0f4155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0f259c640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0f4140e80 0x7fa0f4155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0f259c640 1 -- 172.21.15.172:0/2617234344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0f415c380 msgr2=0x7fa0f4156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0f259c640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0f415c380 0x7fa0f4156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0f259c640 1 -- 172.21.15.172:0/2617234344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f4153e00 con 0x7fa0f413df30 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0f259c640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0f413df30 0x7fa0f413fce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4035040 tx=0x7fa0e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0eaffd640 1 -- 172.21.15.172:0/2617234344 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4037040 con 0x7fa0f413df30 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0eaffd640 1 -- 172.21.15.172:0/2617234344 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e4004250 con 0x7fa0f413df30 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/2617234344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f4154080 con 0x7fa0f413df30 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0eaffd640 1 -- 172.21.15.172:0/2617234344 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40327b0 con 0x7fa0f413df30 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.632+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/2617234344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f4154640 con 0x7fa0f413df30 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.633+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/2617234344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f4068590 con 0x7fa0f413df30 2024-04-14T08:33:29.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.633+0000 7fa0eaffd640 1 -- 172.21.15.172:0/2617234344 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0e40305d0 con 0x7fa0f413df30 2024-04-14T08:33:29.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.634+0000 7fa0eaffd640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa0d406c920 0x7fa0d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.634+0000 7fa0eaffd640 1 -- 172.21.15.172:0/2617234344 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa0e40ae070 con 0x7fa0f413df30 2024-04-14T08:33:29.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.634+0000 7fa0ebfff640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa0d406c920 0x7fa0d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.634+0000 7fa0ebfff640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa0d406c920 0x7fa0d406ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4140e50 tx=0x7fa0dc007e00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:29.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.638+0000 7fa0eaffd640 1 -- 172.21.15.172:0/2617234344 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa0f4153e00 con 0x7fa0f413df30 2024-04-14T08:33:29.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.812+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/2617234344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa0f40b38b0 con 0x7fa0f413df30 2024-04-14T08:33:29.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.812+0000 7fa0eaffd640 1 -- 172.21.15.172:0/2617234344 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fa0e407bba0 con 0x7fa0f413df30 2024-04-14T08:33:29.813 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:29.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.814+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/2617234344 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa0d406c920 msgr2=0x7fa0d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.814+0000 7fa0f8d1e640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa0d406c920 0x7fa0d406ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4140e50 tx=0x7fa0dc007e00 comp rx=0 tx=0).stop 2024-04-14T08:33:29.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.814+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/2617234344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0f413df30 msgr2=0x7fa0f413fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.814+0000 7fa0f8d1e640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0f413df30 0x7fa0f413fce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4035040 tx=0x7fa0e40046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.814+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/2617234344 shutdown_connections 2024-04-14T08:33:29.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.814+0000 7fa0f8d1e640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa0d406c920 0x7fa0d406ede0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.814+0000 7fa0f8d1e640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0f415c380 0x7fa0f4156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.814+0000 7fa0f8d1e640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0f4140e80 0x7fa0f4155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.814+0000 7fa0f8d1e640 1 --2- 172.21.15.172:0/2617234344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0f413df30 0x7fa0f413fce0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.814+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/2617234344 >> 172.21.15.172:0/2617234344 conn(0x7fa0f406b520 msgr2=0x7fa0f41592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:29.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.814+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/2617234344 shutdown_connections 2024-04-14T08:33:29.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.814+0000 7fa0f8d1e640 1 -- 172.21.15.172:0/2617234344 wait complete. 2024-04-14T08:33:29.826 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:29.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.929+0000 7f006a0de640 1 Processor -- start 2024-04-14T08:33:29.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.929+0000 7f006a0de640 1 -- start start 2024-04-14T08:33:29.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.930+0000 7f006a0de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00640cc590 0x7f00640cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.930+0000 7f006a0de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00640cb780 0x7f006406e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.930+0000 7f006a0de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00640cfae0 con 0x7f006415b2c0 2024-04-14T08:33:29.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.930+0000 7f006a0de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f00640cfdc0 con 0x7f00640cb780 2024-04-14T08:33:29.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.930+0000 7f006a0de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f00640d00a0 con 0x7f00640cc590 2024-04-14T08:33:29.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.930+0000 7f0063fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00640cb780 0x7f006406e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.930+0000 7f00637fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f006415b2c0 0x7f00640cbe80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:51160/0 (socket says 172.21.15.172:51160) 2024-04-14T08:33:29.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.930+0000 7f00637fe640 1 -- 172.21.15.172:0/3541905901 learned_addr learned my addr 172.21.15.172:0/3541905901 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:29.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.930+0000 7f005bfff640 1 --2- 172.21.15.172:0/3541905901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00640cc590 0x7f00640cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.930+0000 7f0062ffd640 1 -- 172.21.15.172:0/3541905901 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 655969231 0 0) 0x7f00640cfae0 con 0x7f006415b2c0 2024-04-14T08:33:29.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.931+0000 7f0062ffd640 1 -- 172.21.15.172:0/3541905901 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0050003520 con 0x7f006415b2c0 2024-04-14T08:33:29.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.931+0000 7f0063fff640 1 -- 172.21.15.172:0/3541905901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00640cc590 msgr2=0x7f00640cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:29.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.931+0000 7f0063fff640 1 --2- 172.21.15.172:0/3541905901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00640cc590 0x7f00640cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.931+0000 7f0063fff640 1 -- 172.21.15.172:0/3541905901 >> v1:172.21.15.177:6789/0 conn(0x7f006415b2c0 legacy=0x7f00640cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.931+0000 7f0063fff640 1 -- 172.21.15.172:0/3541905901 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00640d0380 con 0x7f00640cb780 2024-04-14T08:33:29.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.931+0000 7f005bfff640 1 --2- 172.21.15.172:0/3541905901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00640cc590 0x7f00640cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:29.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.931+0000 7f0063fff640 1 --2- 172.21.15.172:0/3541905901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00640cb780 0x7f006406e8e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f005c009d90 tx=0x7f005c02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a5a75f6489a1a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:29.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.931+0000 7f0062ffd640 1 -- 172.21.15.172:0/3541905901 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c039070 con 0x7f00640cb780 2024-04-14T08:33:29.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.931+0000 7f0062ffd640 1 -- 172.21.15.172:0/3541905901 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f005c02cce0 con 0x7f00640cb780 2024-04-14T08:33:29.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.931+0000 7f0062ffd640 1 -- 172.21.15.172:0/3541905901 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c034640 con 0x7f00640cb780 2024-04-14T08:33:29.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.932+0000 7f006a0de640 1 -- 172.21.15.172:0/3541905901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00640cb780 msgr2=0x7f006406e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.932+0000 7f006a0de640 1 --2- 172.21.15.172:0/3541905901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00640cb780 0x7f006406e8e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f005c009d90 tx=0x7f005c02bf30 comp rx=0 tx=0).stop 2024-04-14T08:33:29.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.932+0000 7f006a0de640 1 -- 172.21.15.172:0/3541905901 shutdown_connections 2024-04-14T08:33:29.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.932+0000 7f006a0de640 1 --2- 172.21.15.172:0/3541905901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00640cb780 0x7f006406e8e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.932+0000 7f006a0de640 1 --2- 172.21.15.172:0/3541905901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00640cc590 0x7f00640cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.932+0000 7f006a0de640 1 -- 172.21.15.172:0/3541905901 >> 172.21.15.172:0/3541905901 conn(0x7f00640d0f20 msgr2=0x7f00640d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:29.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.932+0000 7f006a0de640 1 -- 172.21.15.172:0/3541905901 shutdown_connections 2024-04-14T08:33:29.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.932+0000 7f006a0de640 1 -- 172.21.15.172:0/3541905901 wait complete. 2024-04-14T08:33:29.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.933+0000 7f006a0de640 1 Processor -- start 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.933+0000 7f006a0de640 1 -- start start 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f006a0de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00640cb780 0x7f00640c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f006a0de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00640cc590 0x7f00640c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f006a0de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f006415b2c0 0x7f00640c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f006a0de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f006415b6a0 con 0x7f00640cc590 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f006a0de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f00640734c0 con 0x7f006415b2c0 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f006a0de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f006415b820 con 0x7f00640cb780 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f005bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00640cc590 0x7f00640c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f005bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00640cc590 0x7f00640c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37042/0 (socket says 172.21.15.172:37042) 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f005bfff640 1 -- 172.21.15.172:0/2572547801 learned_addr learned my addr 172.21.15.172:0/2572547801 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f00637fe640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00640cb780 0x7f00640c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f0063fff640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f006415b2c0 0x7f00640c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f005bfff640 1 -- 172.21.15.172:0/2572547801 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00640cb780 msgr2=0x7f00640c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f005bfff640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00640cb780 0x7f00640c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f005bfff640 1 -- 172.21.15.172:0/2572547801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f006415b2c0 msgr2=0x7f00640c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:29.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f005bfff640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f006415b2c0 0x7f00640c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:29.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f005bfff640 1 -- 172.21.15.172:0/2572547801 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00640cabf0 con 0x7f00640cc590 2024-04-14T08:33:29.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.934+0000 7f005bfff640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00640cc590 0x7f00640c7ff0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f004c00ec10 tx=0x7f004c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:29.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.935+0000 7f0060ff9640 1 -- 172.21.15.172:0/2572547801 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c017070 con 0x7f00640cc590 2024-04-14T08:33:29.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.935+0000 7f0060ff9640 1 -- 172.21.15.172:0/2572547801 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f004c00cc30 con 0x7f00640cc590 2024-04-14T08:33:29.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.935+0000 7f0060ff9640 1 -- 172.21.15.172:0/2572547801 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c013470 con 0x7f00640cc590 2024-04-14T08:33:29.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.935+0000 7f006a0de640 1 -- 172.21.15.172:0/2572547801 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00640caed0 con 0x7f00640cc590 2024-04-14T08:33:29.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.935+0000 7f006a0de640 1 -- 172.21.15.172:0/2572547801 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006409f7c0 con 0x7f00640cc590 2024-04-14T08:33:29.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.936+0000 7f006a0de640 1 -- 172.21.15.172:0/2572547801 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0064075490 con 0x7f00640cc590 2024-04-14T08:33:29.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.936+0000 7f0060ff9640 1 -- 172.21.15.172:0/2572547801 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f004c01a070 con 0x7f00640cc590 2024-04-14T08:33:29.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.936+0000 7f0060ff9640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f004406c8f0 0x7f004406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:29.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.937+0000 7f0060ff9640 1 -- 172.21.15.172:0/2572547801 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f004c08c030 con 0x7f00640cc590 2024-04-14T08:33:29.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.937+0000 7f00637fe640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f004406c8f0 0x7f004406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:29.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.937+0000 7f00637fe640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f004406c8f0 0x7f004406edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0054005880 tx=0x7f0054009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:29.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:29.941+0000 7f0060ff9640 1 -- 172.21.15.172:0/2572547801 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f004c059b60 con 0x7f00640cc590 2024-04-14T08:33:30.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.114+0000 7f006a0de640 1 -- 172.21.15.172:0/2572547801 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f00640cc140 con 0x7f00640cc590 2024-04-14T08:33:30.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.115+0000 7f0060ff9640 1 -- 172.21.15.172:0/2572547801 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f004c05de60 con 0x7f00640cc590 2024-04-14T08:33:30.115 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:30.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.117+0000 7f006a0de640 1 -- 172.21.15.172:0/2572547801 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f004406c8f0 msgr2=0x7f004406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.117+0000 7f006a0de640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f004406c8f0 0x7f004406edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0054005880 tx=0x7f0054009040 comp rx=0 tx=0).stop 2024-04-14T08:33:30.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.117+0000 7f006a0de640 1 -- 172.21.15.172:0/2572547801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00640cc590 msgr2=0x7f00640c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.117+0000 7f006a0de640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00640cc590 0x7f00640c7ff0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f004c00ec10 tx=0x7f004c00c520 comp rx=0 tx=0).stop 2024-04-14T08:33:30.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.117+0000 7f006a0de640 1 -- 172.21.15.172:0/2572547801 shutdown_connections 2024-04-14T08:33:30.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.117+0000 7f006a0de640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f004406c8f0 0x7f004406edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.117+0000 7f006a0de640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f006415b2c0 0x7f00640c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.117+0000 7f006a0de640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00640cc590 0x7f00640c7ff0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.117+0000 7f006a0de640 1 --2- 172.21.15.172:0/2572547801 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00640cb780 0x7f00640c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.117+0000 7f006a0de640 1 -- 172.21.15.172:0/2572547801 >> 172.21.15.172:0/2572547801 conn(0x7f00640d0f20 msgr2=0x7f006414f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:30.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.117+0000 7f006a0de640 1 -- 172.21.15.172:0/2572547801 shutdown_connections 2024-04-14T08:33:30.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.117+0000 7f006a0de640 1 -- 172.21.15.172:0/2572547801 wait complete. 2024-04-14T08:33:30.128 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:30.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.232+0000 7f1007e80640 1 Processor -- start 2024-04-14T08:33:30.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.233+0000 7f1007e80640 1 -- start start 2024-04-14T08:33:30.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.233+0000 7f1007e80640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f100013df30 0x7f10000b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f1007e80640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1000140e80 0x7f10000b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f1007e80640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f100015b620 con 0x7f100015c380 2024-04-14T08:33:30.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f1007e80640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f10000cf9b0 con 0x7f100013df30 2024-04-14T08:33:30.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f1007e80640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f10001414a0 con 0x7f1000140e80 2024-04-14T08:33:30.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f1005c1c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f100013df30 0x7f10000b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f1005c1c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f100013df30 0x7f10000b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37054/0 (socket says 172.21.15.172:37054) 2024-04-14T08:33:30.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f1005c1c640 1 -- 172.21.15.172:0/1787083303 learned_addr learned my addr 172.21.15.172:0/1787083303 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:30.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f100641d640 1 --2- 172.21.15.172:0/1787083303 >> v2:172.21.15.177:3300/0 conn(0x7f1000140e80 0x7f10000b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f1005c1c640 1 -- 172.21.15.172:0/1787083303 >> v2:172.21.15.177:3300/0 conn(0x7f1000140e80 msgr2=0x7f10000b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:30.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f1005c1c640 1 --2- 172.21.15.172:0/1787083303 >> v2:172.21.15.177:3300/0 conn(0x7f1000140e80 0x7f10000b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f1005c1c640 1 -- 172.21.15.172:0/1787083303 >> v1:172.21.15.177:6789/0 conn(0x7f100015c380 legacy=0x7f1000140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f1005c1c640 1 -- 172.21.15.172:0/1787083303 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100015b8a0 con 0x7f100013df30 2024-04-14T08:33:30.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.234+0000 7f100641d640 1 --2- 172.21.15.172:0/1787083303 >> v2:172.21.15.177:3300/0 conn(0x7f1000140e80 0x7f10000b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:30.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.235+0000 7f1005c1c640 1 --2- 172.21.15.172:0/1787083303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f100013df30 0x7f10000b1cd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f100015eb50 tx=0x7f0ff802e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cb24718e615fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:30.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.235+0000 7f1004c1a640 1 -- 172.21.15.172:0/1787083303 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff800b0d0 con 0x7f100013df30 2024-04-14T08:33:30.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.235+0000 7f1004c1a640 1 -- 172.21.15.172:0/1787083303 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff800b270 con 0x7f100013df30 2024-04-14T08:33:30.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.235+0000 7f1004c1a640 1 -- 172.21.15.172:0/1787083303 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8009600 con 0x7f100013df30 2024-04-14T08:33:30.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.235+0000 7f1007e80640 1 -- 172.21.15.172:0/1787083303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f100013df30 msgr2=0x7f10000b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.235+0000 7f1007e80640 1 --2- 172.21.15.172:0/1787083303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f100013df30 0x7f10000b1cd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f100015eb50 tx=0x7f0ff802e180 comp rx=0 tx=0).stop 2024-04-14T08:33:30.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.235+0000 7f1007e80640 1 -- 172.21.15.172:0/1787083303 shutdown_connections 2024-04-14T08:33:30.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.236+0000 7f1007e80640 1 --2- 172.21.15.172:0/1787083303 >> v2:172.21.15.177:3300/0 conn(0x7f1000140e80 0x7f10000b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.236+0000 7f1007e80640 1 --2- 172.21.15.172:0/1787083303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f100013df30 0x7f10000b1cd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.236+0000 7f1007e80640 1 -- 172.21.15.172:0/1787083303 >> 172.21.15.172:0/1787083303 conn(0x7f100006b520 msgr2=0x7f100006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:30.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.236+0000 7f1007e80640 1 -- 172.21.15.172:0/1787083303 shutdown_connections 2024-04-14T08:33:30.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.236+0000 7f1007e80640 1 -- 172.21.15.172:0/1787083303 wait complete. 2024-04-14T08:33:30.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.237+0000 7f1007e80640 1 Processor -- start 2024-04-14T08:33:30.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.237+0000 7f1007e80640 1 -- start start 2024-04-14T08:33:30.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f1007e80640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f100013df30 0x7f10001512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f1007e80640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1000140e80 0x7f1000137160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f1007e80640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f100015c380 0x7f10001376a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f1007e80640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f100013e310 con 0x7f100015c380 2024-04-14T08:33:30.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f1007e80640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f100015be60 con 0x7f1000140e80 2024-04-14T08:33:30.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f1007e80640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f10000ce9a0 con 0x7f100013df30 2024-04-14T08:33:30.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f100641d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f100015c380 0x7f10001376a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f100641d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f100015c380 0x7f10001376a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37066/0 (socket says 172.21.15.172:37066) 2024-04-14T08:33:30.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f1005c1c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f100013df30 0x7f10001512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f100641d640 1 -- 172.21.15.172:0/375063563 learned_addr learned my addr 172.21.15.172:0/375063563 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:30.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f100541b640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1000140e80 0x7f1000137160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f100641d640 1 -- 172.21.15.172:0/375063563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f100013df30 msgr2=0x7f10001512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f100641d640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f100013df30 0x7f10001512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f100641d640 1 -- 172.21.15.172:0/375063563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1000140e80 msgr2=0x7f1000137160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f100641d640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1000140e80 0x7f1000137160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f100641d640 1 -- 172.21.15.172:0/375063563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10000b3c00 con 0x7f100015c380 2024-04-14T08:33:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f100641d640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f100015c380 0x7f10001376a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00e930 tx=0x7f0ffc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f0ff67fc640 1 -- 172.21.15.172:0/375063563 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc017070 con 0x7f100015c380 2024-04-14T08:33:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f0ff67fc640 1 -- 172.21.15.172:0/375063563 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ffc00ca00 con 0x7f100015c380 2024-04-14T08:33:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f1007e80640 1 -- 172.21.15.172:0/375063563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10000b3ee0 con 0x7f100015c380 2024-04-14T08:33:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f0ff67fc640 1 -- 172.21.15.172:0/375063563 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc00cdd0 con 0x7f100015c380 2024-04-14T08:33:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.238+0000 7f1007e80640 1 -- 172.21.15.172:0/375063563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10000b44a0 con 0x7f100015c380 2024-04-14T08:33:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.240+0000 7f1007e80640 1 -- 172.21.15.172:0/375063563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1000068590 con 0x7f100015c380 2024-04-14T08:33:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.240+0000 7f0ff67fc640 1 -- 172.21.15.172:0/375063563 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ffc013470 con 0x7f100015c380 2024-04-14T08:33:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.241+0000 7f0ff67fc640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0fe406c8f0 0x7f0fe406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.241+0000 7f0ff67fc640 1 -- 172.21.15.172:0/375063563 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0ffc08bcb0 con 0x7f100015c380 2024-04-14T08:33:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.241+0000 7f1005c1c640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0fe406c8f0 0x7f0fe406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.241+0000 7f1005c1c640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0fe406c8f0 0x7f0fe406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8037000 tx=0x7f0ff8032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:30.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.246+0000 7f0ff67fc640 1 -- 172.21.15.172:0/375063563 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0ffc0598a0 con 0x7f100015c380 2024-04-14T08:33:30.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.422+0000 7f1007e80640 1 -- 172.21.15.172:0/375063563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10000b4670 con 0x7f100015c380 2024-04-14T08:33:30.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.422+0000 7f0ff67fc640 1 -- 172.21.15.172:0/375063563 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f0ffc0600c0 con 0x7f100015c380 2024-04-14T08:33:30.423 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.424+0000 7f1007e80640 1 -- 172.21.15.172:0/375063563 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0fe406c8f0 msgr2=0x7f0fe406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.424+0000 7f1007e80640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0fe406c8f0 0x7f0fe406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8037000 tx=0x7f0ff8032000 comp rx=0 tx=0).stop 2024-04-14T08:33:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.424+0000 7f1007e80640 1 -- 172.21.15.172:0/375063563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f100015c380 msgr2=0x7f10001376a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.424+0000 7f1007e80640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f100015c380 0x7f10001376a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00e930 tx=0x7f0ffc00edf0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.424+0000 7f1007e80640 1 -- 172.21.15.172:0/375063563 shutdown_connections 2024-04-14T08:33:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.424+0000 7f1007e80640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0fe406c8f0 0x7f0fe406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.424+0000 7f1007e80640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f100015c380 0x7f10001376a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.424+0000 7f1007e80640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1000140e80 0x7f1000137160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.424+0000 7f1007e80640 1 --2- 172.21.15.172:0/375063563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f100013df30 0x7f10001512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.424+0000 7f1007e80640 1 -- 172.21.15.172:0/375063563 >> 172.21.15.172:0/375063563 conn(0x7f100006b520 msgr2=0x7f10000729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.424+0000 7f1007e80640 1 -- 172.21.15.172:0/375063563 shutdown_connections 2024-04-14T08:33:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.424+0000 7f1007e80640 1 -- 172.21.15.172:0/375063563 wait complete. 2024-04-14T08:33:30.435 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:30.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.540+0000 7f7cd10ad640 1 Processor -- start 2024-04-14T08:33:30.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.540+0000 7f7cd10ad640 1 -- start start 2024-04-14T08:33:30.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7cd10ad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc13fa70 0x7f7ccc0af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7cd10ad640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7ccc0ce150 0x7f7ccc0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7cd10ad640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ccc071520 con 0x7f7ccc0ce5b0 2024-04-14T08:33:30.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7cd10ad640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f7ccc0712d0 con 0x7f7ccc0ce150 2024-04-14T08:33:30.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7cd10ad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7ccc071800 con 0x7f7ccc13fa70 2024-04-14T08:33:30.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7cc3fff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7ccc0ce150 0x7f7ccc0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7ccad9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc13fa70 0x7f7ccc0af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7ccb59e640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f7ccc0ce5b0 0x7f7ccc15e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:51168/0 (socket says 172.21.15.172:51168) 2024-04-14T08:33:30.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7ccb59e640 1 -- 172.21.15.172:0/288769270 learned_addr learned my addr 172.21.15.172:0/288769270 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:30.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7ccad9d640 1 -- 172.21.15.172:0/288769270 >> v2:172.21.15.177:3300/0 conn(0x7f7ccc0ce150 msgr2=0x7f7ccc0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:30.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7ccad9d640 1 --2- 172.21.15.172:0/288769270 >> v2:172.21.15.177:3300/0 conn(0x7f7ccc0ce150 0x7f7ccc0722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7ccad9d640 1 -- 172.21.15.172:0/288769270 >> v1:172.21.15.177:6789/0 conn(0x7f7ccc0ce5b0 legacy=0x7f7ccc15e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7ccad9d640 1 -- 172.21.15.172:0/288769270 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ccc071ae0 con 0x7f7ccc13fa70 2024-04-14T08:33:30.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7cca59c640 1 -- 172.21.15.172:0/288769270 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2070140818 0 0) 0x7f7ccc071520 con 0x7f7ccc0ce5b0 2024-04-14T08:33:30.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.541+0000 7f7cc3fff640 1 --2- 172.21.15.172:0/288769270 >> v2:172.21.15.177:3300/0 conn(0x7f7ccc0ce150 0x7f7ccc0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:30.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.542+0000 7f7ccad9d640 1 --2- 172.21.15.172:0/288769270 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc13fa70 0x7f7ccc0af740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0097a0 tx=0x7f7cbc02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c6d075f424bc7fb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.542+0000 7f7cca59c640 1 -- 172.21.15.172:0/288769270 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc0315a0 con 0x7f7ccc13fa70 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.542+0000 7f7cca59c640 1 -- 172.21.15.172:0/288769270 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cbc037070 con 0x7f7ccc13fa70 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.542+0000 7f7cca59c640 1 -- 172.21.15.172:0/288769270 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc031960 con 0x7f7ccc13fa70 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.543+0000 7f7cd10ad640 1 -- 172.21.15.172:0/288769270 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc13fa70 msgr2=0x7f7ccc0af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.543+0000 7f7cd10ad640 1 --2- 172.21.15.172:0/288769270 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc13fa70 0x7f7ccc0af740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0097a0 tx=0x7f7cbc02b840 comp rx=0 tx=0).stop 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.543+0000 7f7cd10ad640 1 -- 172.21.15.172:0/288769270 shutdown_connections 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.543+0000 7f7cd10ad640 1 --2- 172.21.15.172:0/288769270 >> v2:172.21.15.177:3300/0 conn(0x7f7ccc0ce150 0x7f7ccc0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.543+0000 7f7cd10ad640 1 --2- 172.21.15.172:0/288769270 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc13fa70 0x7f7ccc0af740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.543+0000 7f7cd10ad640 1 -- 172.21.15.172:0/288769270 >> 172.21.15.172:0/288769270 conn(0x7f7ccc06b520 msgr2=0x7f7ccc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.543+0000 7f7cd10ad640 1 -- 172.21.15.172:0/288769270 shutdown_connections 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.543+0000 7f7cd10ad640 1 -- 172.21.15.172:0/288769270 wait complete. 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.544+0000 7f7cd10ad640 1 Processor -- start 2024-04-14T08:33:30.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.544+0000 7f7cd10ad640 1 -- start start 2024-04-14T08:33:30.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7cd10ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ccc0ce150 0x7f7ccc134370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7cd10ad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc0ce5b0 0x7f7ccc143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7cd10ad640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc13fa70 0x7f7ccc13fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7cd10ad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc1442b0 con 0x7f7ccc0ce150 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7cd10ad640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc144130 con 0x7f7ccc13fa70 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7cd10ad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7ccc072820 con 0x7f7ccc0ce5b0 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7ccad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ccc0ce150 0x7f7ccc134370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7ccb59e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc13fa70 0x7f7ccc13fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7ccad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ccc0ce150 0x7f7ccc134370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37072/0 (socket says 172.21.15.172:37072) 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7ccad9d640 1 -- 172.21.15.172:0/2794852716 learned_addr learned my addr 172.21.15.172:0/2794852716 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7cc3fff640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc0ce5b0 0x7f7ccc143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7ccad9d640 1 -- 172.21.15.172:0/2794852716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc0ce5b0 msgr2=0x7f7ccc143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7ccad9d640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc0ce5b0 0x7f7ccc143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7ccad9d640 1 -- 172.21.15.172:0/2794852716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc13fa70 msgr2=0x7f7ccc13fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7ccad9d640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc13fa70 0x7f7ccc13fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7ccad9d640 1 -- 172.21.15.172:0/2794852716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ccc15cf90 con 0x7f7ccc0ce150 2024-04-14T08:33:30.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7cc3fff640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc0ce5b0 0x7f7ccc143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:30.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.545+0000 7f7ccb59e640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc13fa70 0x7f7ccc13fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:30.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.546+0000 7f7ccad9d640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ccc0ce150 0x7f7ccc134370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0098c0 tx=0x7f7cbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:30.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.546+0000 7f7cc37fe640 1 -- 172.21.15.172:0/2794852716 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc032630 con 0x7f7ccc0ce150 2024-04-14T08:33:30.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.546+0000 7f7cc37fe640 1 -- 172.21.15.172:0/2794852716 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cbc0327f0 con 0x7f7ccc0ce150 2024-04-14T08:33:30.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.546+0000 7f7cc37fe640 1 -- 172.21.15.172:0/2794852716 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc032be0 con 0x7f7ccc0ce150 2024-04-14T08:33:30.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.546+0000 7f7cd10ad640 1 -- 172.21.15.172:0/2794852716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ccc15d270 con 0x7f7ccc0ce150 2024-04-14T08:33:30.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.546+0000 7f7cd10ad640 1 -- 172.21.15.172:0/2794852716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ccc15d830 con 0x7f7ccc0ce150 2024-04-14T08:33:30.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.547+0000 7f7cd10ad640 1 -- 172.21.15.172:0/2794852716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ccc068590 con 0x7f7ccc0ce150 2024-04-14T08:33:30.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.547+0000 7f7cc37fe640 1 -- 172.21.15.172:0/2794852716 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cbc03b3f0 con 0x7f7ccc0ce150 2024-04-14T08:33:30.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.548+0000 7f7cc37fe640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7cac06c920 0x7f7cac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.548+0000 7f7cc37fe640 1 -- 172.21.15.172:0/2794852716 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7cbc02f030 con 0x7f7ccc0ce150 2024-04-14T08:33:30.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.548+0000 7f7cc3fff640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7cac06c920 0x7f7cac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.548+0000 7f7cc3fff640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7cac06c920 0x7f7cac06ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0cfc90 tx=0x7f7cb4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:30.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.552+0000 7f7cc37fe640 1 -- 172.21.15.172:0/2794852716 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7cbc040070 con 0x7f7ccc0ce150 2024-04-14T08:33:30.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.729+0000 7f7cd10ad640 1 -- 172.21.15.172:0/2794852716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7ccc13dca0 con 0x7f7ccc0ce150 2024-04-14T08:33:30.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.730+0000 7f7cc37fe640 1 -- 172.21.15.172:0/2794852716 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f7cbc07b590 con 0x7f7ccc0ce150 2024-04-14T08:33:30.731 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:30.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.732+0000 7f7cd10ad640 1 -- 172.21.15.172:0/2794852716 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7cac06c920 msgr2=0x7f7cac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.732+0000 7f7cd10ad640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7cac06c920 0x7f7cac06ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0cfc90 tx=0x7f7cb4009190 comp rx=0 tx=0).stop 2024-04-14T08:33:30.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.732+0000 7f7cd10ad640 1 -- 172.21.15.172:0/2794852716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ccc0ce150 msgr2=0x7f7ccc134370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.732+0000 7f7cd10ad640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ccc0ce150 0x7f7ccc134370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0098c0 tx=0x7f7cbc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.732+0000 7f7cd10ad640 1 -- 172.21.15.172:0/2794852716 shutdown_connections 2024-04-14T08:33:30.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.732+0000 7f7cd10ad640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7cac06c920 0x7f7cac06ede0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.732+0000 7f7cd10ad640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ccc13fa70 0x7f7ccc13fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.732+0000 7f7cd10ad640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ccc0ce5b0 0x7f7ccc143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.732+0000 7f7cd10ad640 1 --2- 172.21.15.172:0/2794852716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ccc0ce150 0x7f7ccc134370 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.732+0000 7f7cd10ad640 1 -- 172.21.15.172:0/2794852716 >> 172.21.15.172:0/2794852716 conn(0x7f7ccc06b520 msgr2=0x7f7ccc159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:30.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.732+0000 7f7cd10ad640 1 -- 172.21.15.172:0/2794852716 shutdown_connections 2024-04-14T08:33:30.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.732+0000 7f7cd10ad640 1 -- 172.21.15.172:0/2794852716 wait complete. 2024-04-14T08:33:30.743 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:30.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.843+0000 7fd3b1dc8640 1 Processor -- start 2024-04-14T08:33:30.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.843+0000 7fd3b1dc8640 1 -- start start 2024-04-14T08:33:30.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.844+0000 7fd3b1dc8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd3ac0b1df0 0x7fd3ac0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.844+0000 7fd3b1dc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3ac0b4040 0x7fd3ac0b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.844+0000 7fd3b1dc8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3ac0b49f0 con 0x7fd3ac0b21d0 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.844+0000 7fd3b1dc8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fd3ac15c770 con 0x7fd3ac0b1df0 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.844+0000 7fd3b1dc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd3ac0b3430 con 0x7fd3ac0b4040 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.844+0000 7fd3aaffd640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fd3ac0b21d0 0x7fd3ac15c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:51172/0 (socket says 172.21.15.172:51172) 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.844+0000 7fd3ab7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd3ac0b1df0 0x7fd3ac0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.844+0000 7fd3abfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3ac0b4040 0x7fd3ac0b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.844+0000 7fd3aaffd640 1 -- 172.21.15.172:0/4139522961 learned_addr learned my addr 172.21.15.172:0/4139522961 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.845+0000 7fd3aa7fc640 1 -- 172.21.15.172:0/4139522961 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4184200804 0 0) 0x7fd3ac0b49f0 con 0x7fd3ac0b21d0 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.845+0000 7fd3aa7fc640 1 -- 172.21.15.172:0/4139522961 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd394003520 con 0x7fd3ac0b21d0 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.845+0000 7fd3abfff640 1 -- 172.21.15.172:0/4139522961 >> v2:172.21.15.177:3300/0 conn(0x7fd3ac0b1df0 msgr2=0x7fd3ac0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.845+0000 7fd3abfff640 1 --2- 172.21.15.172:0/4139522961 >> v2:172.21.15.177:3300/0 conn(0x7fd3ac0b1df0 0x7fd3ac0b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.845+0000 7fd3abfff640 1 -- 172.21.15.172:0/4139522961 >> v1:172.21.15.177:6789/0 conn(0x7fd3ac0b21d0 legacy=0x7fd3ac15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.845+0000 7fd3abfff640 1 -- 172.21.15.172:0/4139522961 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac0b3700 con 0x7fd3ac0b4040 2024-04-14T08:33:30.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.845+0000 7fd3ab7fe640 1 --2- 172.21.15.172:0/4139522961 >> v2:172.21.15.177:3300/0 conn(0x7fd3ac0b1df0 0x7fd3ac0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:30.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.846+0000 7fd3abfff640 1 --2- 172.21.15.172:0/4139522961 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3ac0b4040 0x7fd3ac0b44b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009d90 tx=0x7fd3a402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1417b59df0caafef server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:30.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.846+0000 7fd3aa7fc640 1 -- 172.21.15.172:0/4139522961 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4039070 con 0x7fd3ac0b4040 2024-04-14T08:33:30.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.846+0000 7fd3aa7fc640 1 -- 172.21.15.172:0/4139522961 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a402cc50 con 0x7fd3ac0b4040 2024-04-14T08:33:30.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.846+0000 7fd3aa7fc640 1 -- 172.21.15.172:0/4139522961 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4034440 con 0x7fd3ac0b4040 2024-04-14T08:33:30.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.846+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/4139522961 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3ac0b4040 msgr2=0x7fd3ac0b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.846+0000 7fd3b1dc8640 1 --2- 172.21.15.172:0/4139522961 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3ac0b4040 0x7fd3ac0b44b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009d90 tx=0x7fd3a402bea0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.846+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/4139522961 shutdown_connections 2024-04-14T08:33:30.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.846+0000 7fd3b1dc8640 1 --2- 172.21.15.172:0/4139522961 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3ac0b4040 0x7fd3ac0b44b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.846+0000 7fd3b1dc8640 1 --2- 172.21.15.172:0/4139522961 >> v2:172.21.15.177:3300/0 conn(0x7fd3ac0b1df0 0x7fd3ac0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.846+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/4139522961 >> 172.21.15.172:0/4139522961 conn(0x7fd3ac06ec30 msgr2=0x7fd3ac06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:30.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.846+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/4139522961 shutdown_connections 2024-04-14T08:33:30.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.846+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/4139522961 wait complete. 2024-04-14T08:33:30.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.847+0000 7fd3b1dc8640 1 Processor -- start 2024-04-14T08:33:30.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.848+0000 7fd3b1dc8640 1 -- start start 2024-04-14T08:33:30.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.848+0000 7fd3b1dc8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac0b1df0 0x7fd3ac1f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.848+0000 7fd3b1dc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3ac0b21d0 0x7fd3ac12e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.848+0000 7fd3b1dc8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3ac0b4040 0x7fd3ac0728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.848+0000 7fd3b1dc8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac15bb40 con 0x7fd3ac0b1df0 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.848+0000 7fd3b1dc8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac0b3cc0 con 0x7fd3ac0b4040 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.848+0000 7fd3b1dc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd3ac15b7f0 con 0x7fd3ac0b21d0 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.848+0000 7fd3ab7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac0b1df0 0x7fd3ac1f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.848+0000 7fd3ab7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac0b1df0 0x7fd3ac1f12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37076/0 (socket says 172.21.15.172:37076) 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.848+0000 7fd3ab7fe640 1 -- 172.21.15.172:0/314630048 learned_addr learned my addr 172.21.15.172:0/314630048 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.848+0000 7fd3aaffd640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3ac0b21d0 0x7fd3ac12e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.848+0000 7fd3abfff640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3ac0b4040 0x7fd3ac0728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.849+0000 7fd3ab7fe640 1 -- 172.21.15.172:0/314630048 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3ac0b21d0 msgr2=0x7fd3ac12e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.849+0000 7fd3ab7fe640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3ac0b21d0 0x7fd3ac12e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.849+0000 7fd3ab7fe640 1 -- 172.21.15.172:0/314630048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3ac0b4040 msgr2=0x7fd3ac0728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.849+0000 7fd3ab7fe640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3ac0b4040 0x7fd3ac0728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.849+0000 7fd3ab7fe640 1 -- 172.21.15.172:0/314630048 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac12ed20 con 0x7fd3ac0b1df0 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.849+0000 7fd3ab7fe640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac0b1df0 0x7fd3ac1f12a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00ebe0 tx=0x7fd39c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.849+0000 7fd393fff640 1 -- 172.21.15.172:0/314630048 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c017070 con 0x7fd3ac0b1df0 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.849+0000 7fd393fff640 1 -- 172.21.15.172:0/314630048 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd39c00cbd0 con 0x7fd3ac0b1df0 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.849+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/314630048 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac12f000 con 0x7fd3ac0b1df0 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.849+0000 7fd393fff640 1 -- 172.21.15.172:0/314630048 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c013470 con 0x7fd3ac0b1df0 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.849+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/314630048 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac12f8a0 con 0x7fd3ac0b1df0 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.850+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/314630048 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ac068520 con 0x7fd3ac0b1df0 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.850+0000 7fd393fff640 1 -- 172.21.15.172:0/314630048 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd39c01a070 con 0x7fd3ac0b1df0 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.851+0000 7fd393fff640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd39806c8f0 0x7fd39806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.851+0000 7fd3aaffd640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd39806c8f0 0x7fd39806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.851+0000 7fd393fff640 1 -- 172.21.15.172:0/314630048 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd39c08c0b0 con 0x7fd3ac0b1df0 2024-04-14T08:33:30.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.851+0000 7fd3aaffd640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd39806c8f0 0x7fd39806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac12e1c0 tx=0x7fd3a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:30.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:30.855+0000 7fd393fff640 1 -- 172.21.15.172:0/314630048 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd39c059b50 con 0x7fd3ac0b1df0 2024-04-14T08:33:31.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.032+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/314630048 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3ac12f1d0 con 0x7fd3ac0b1df0 2024-04-14T08:33:31.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.033+0000 7fd393fff640 1 -- 172.21.15.172:0/314630048 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7fd39c05de50 con 0x7fd3ac0b1df0 2024-04-14T08:33:31.034 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:31.034 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:31.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.035+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/314630048 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd39806c8f0 msgr2=0x7fd39806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.035+0000 7fd3b1dc8640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd39806c8f0 0x7fd39806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac12e1c0 tx=0x7fd3a0009040 comp rx=0 tx=0).stop 2024-04-14T08:33:31.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.035+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/314630048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac0b1df0 msgr2=0x7fd3ac1f12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.035+0000 7fd3b1dc8640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac0b1df0 0x7fd3ac1f12a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00ebe0 tx=0x7fd39c00c520 comp rx=0 tx=0).stop 2024-04-14T08:33:31.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.035+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/314630048 shutdown_connections 2024-04-14T08:33:31.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.035+0000 7fd3b1dc8640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd39806c8f0 0x7fd39806edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.035+0000 7fd3b1dc8640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3ac0b4040 0x7fd3ac0728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.035+0000 7fd3b1dc8640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3ac0b21d0 0x7fd3ac12e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.035+0000 7fd3b1dc8640 1 --2- 172.21.15.172:0/314630048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3ac0b1df0 0x7fd3ac1f12a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.035+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/314630048 >> 172.21.15.172:0/314630048 conn(0x7fd3ac06ec30 msgr2=0x7fd3ac1594c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:31.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.036+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/314630048 shutdown_connections 2024-04-14T08:33:31.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.036+0000 7fd3b1dc8640 1 -- 172.21.15.172:0/314630048 wait complete. 2024-04-14T08:33:31.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:31.048 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.151+0000 7f6717e98640 1 Processor -- start 2024-04-14T08:33:31.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.151+0000 7f6717e98640 1 -- start start 2024-04-14T08:33:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.152+0000 7f6717e98640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f671013df30 0x7f67100b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.152+0000 7f6717e98640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f671015b620 con 0x7f671015c380 2024-04-14T08:33:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.152+0000 7f6717e98640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67101414a0 con 0x7f6710140e80 2024-04-14T08:33:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.152+0000 7f6717e98640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f671015b8a0 con 0x7f671013df30 2024-04-14T08:33:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.152+0000 7f6715c34640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f671013df30 0x7f67100b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.152+0000 7f6715433640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f671015c380 0x7f6710140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46470/0 (socket says 172.21.15.172:46470) 2024-04-14T08:33:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.152+0000 7f6715433640 1 -- 172.21.15.172:0/797074741 learned_addr learned my addr 172.21.15.172:0/797074741 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:31.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.153+0000 7f6715c34640 1 -- 172.21.15.172:0/797074741 >> v1:172.21.15.177:6789/0 conn(0x7f6710140e80 legacy=0x7f67100ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.153+0000 7f6715c34640 1 -- 172.21.15.172:0/797074741 >> v1:172.21.15.172:6789/0 conn(0x7f671015c380 legacy=0x7f6710140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.153+0000 7f6715c34640 1 -- 172.21.15.172:0/797074741 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671015bb80 con 0x7f671013df30 2024-04-14T08:33:31.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.153+0000 7f6715c34640 1 --2- 172.21.15.172:0/797074741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f671013df30 0x7f67100b1cd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f670c00bfa0 tx=0x7f670c02e140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e698c234d390d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:31.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.153+0000 7f6714c32640 1 -- 172.21.15.172:0/797074741 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c02eae0 con 0x7f671013df30 2024-04-14T08:33:31.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.153+0000 7f6714c32640 1 -- 172.21.15.172:0/797074741 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670c02ec80 con 0x7f671013df30 2024-04-14T08:33:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.153+0000 7f6714c32640 1 -- 172.21.15.172:0/797074741 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c009760 con 0x7f671013df30 2024-04-14T08:33:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.154+0000 7f6717e98640 1 -- 172.21.15.172:0/797074741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f671013df30 msgr2=0x7f67100b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.154+0000 7f6717e98640 1 --2- 172.21.15.172:0/797074741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f671013df30 0x7f67100b1cd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f670c00bfa0 tx=0x7f670c02e140 comp rx=0 tx=0).stop 2024-04-14T08:33:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.154+0000 7f6717e98640 1 -- 172.21.15.172:0/797074741 shutdown_connections 2024-04-14T08:33:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.154+0000 7f6717e98640 1 --2- 172.21.15.172:0/797074741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f671013df30 0x7f67100b1cd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.154+0000 7f6717e98640 1 -- 172.21.15.172:0/797074741 >> 172.21.15.172:0/797074741 conn(0x7f671006b520 msgr2=0x7f671006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.154+0000 7f6717e98640 1 -- 172.21.15.172:0/797074741 shutdown_connections 2024-04-14T08:33:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.154+0000 7f6717e98640 1 -- 172.21.15.172:0/797074741 wait complete. 2024-04-14T08:33:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.155+0000 7f6717e98640 1 Processor -- start 2024-04-14T08:33:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.155+0000 7f6717e98640 1 -- start start 2024-04-14T08:33:31.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6717e98640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f671013df30 0x7f671013fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6717e98640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6710140e80 0x7f671013f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6717e98640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f671015c380 0x7f67100b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6717e98640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f67100b2210 con 0x7f671015c380 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6717e98640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f671013e310 con 0x7f6710140e80 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6717e98640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f67100b2390 con 0x7f671013df30 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6716435640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f671015c380 0x7f67100b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6716435640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f671015c380 0x7f67100b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37092/0 (socket says 172.21.15.172:37092) 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6716435640 1 -- 172.21.15.172:0/1998822902 learned_addr learned my addr 172.21.15.172:0/1998822902 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6715c34640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f671013df30 0x7f671013fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6715433640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6710140e80 0x7f671013f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6716435640 1 -- 172.21.15.172:0/1998822902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f671013df30 msgr2=0x7f671013fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6716435640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f671013df30 0x7f671013fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6716435640 1 -- 172.21.15.172:0/1998822902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6710140e80 msgr2=0x7f671013f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6716435640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6710140e80 0x7f671013f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6716435640 1 -- 172.21.15.172:0/1998822902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671015d5d0 con 0x7f671015c380 2024-04-14T08:33:31.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.156+0000 7f6716435640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f671015c380 0x7f67100b4c30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6708001eb0 tx=0x7f670800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:31.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.157+0000 7f67067fc640 1 -- 172.21.15.172:0/1998822902 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708017070 con 0x7f671015c380 2024-04-14T08:33:31.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.157+0000 7f67067fc640 1 -- 172.21.15.172:0/1998822902 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67080052d0 con 0x7f671015c380 2024-04-14T08:33:31.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.157+0000 7f6717e98640 1 -- 172.21.15.172:0/1998822902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671015d870 con 0x7f671015c380 2024-04-14T08:33:31.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.157+0000 7f67067fc640 1 -- 172.21.15.172:0/1998822902 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708003810 con 0x7f671015c380 2024-04-14T08:33:31.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.157+0000 7f6717e98640 1 -- 172.21.15.172:0/1998822902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671015e350 con 0x7f671015c380 2024-04-14T08:33:31.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.158+0000 7f6717e98640 1 -- 172.21.15.172:0/1998822902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6710068590 con 0x7f671015c380 2024-04-14T08:33:31.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.158+0000 7f67067fc640 1 -- 172.21.15.172:0/1998822902 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f670801a070 con 0x7f671015c380 2024-04-14T08:33:31.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.159+0000 7f67067fc640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f66f406c8f0 0x7f66f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.159+0000 7f67067fc640 1 -- 172.21.15.172:0/1998822902 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f670808b100 con 0x7f671015c380 2024-04-14T08:33:31.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.159+0000 7f6715c34640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f66f406c8f0 0x7f66f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.159+0000 7f6715c34640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f66f406c8f0 0x7f66f406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f670c037000 tx=0x7f670c032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:31.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.163+0000 7f67067fc640 1 -- 172.21.15.172:0/1998822902 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6708058c30 con 0x7f671015c380 2024-04-14T08:33:31.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.336+0000 7f6717e98640 1 -- 172.21.15.172:0/1998822902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f671006bbe0 con 0x7f671015c380 2024-04-14T08:33:31.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.336+0000 7f67067fc640 1 -- 172.21.15.172:0/1998822902 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f670805cf30 con 0x7f671015c380 2024-04-14T08:33:31.337 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:31.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.338+0000 7f6717e98640 1 -- 172.21.15.172:0/1998822902 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f66f406c8f0 msgr2=0x7f66f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.338+0000 7f6717e98640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f66f406c8f0 0x7f66f406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f670c037000 tx=0x7f670c032000 comp rx=0 tx=0).stop 2024-04-14T08:33:31.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.338+0000 7f6717e98640 1 -- 172.21.15.172:0/1998822902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f671015c380 msgr2=0x7f67100b4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.338+0000 7f6717e98640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f671015c380 0x7f67100b4c30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6708001eb0 tx=0x7f670800c890 comp rx=0 tx=0).stop 2024-04-14T08:33:31.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.338+0000 7f6717e98640 1 -- 172.21.15.172:0/1998822902 shutdown_connections 2024-04-14T08:33:31.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.338+0000 7f6717e98640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f66f406c8f0 0x7f66f406edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.338+0000 7f6717e98640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f671015c380 0x7f67100b4c30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.338+0000 7f6717e98640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6710140e80 0x7f671013f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.338+0000 7f6717e98640 1 --2- 172.21.15.172:0/1998822902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f671013df30 0x7f671013fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.338+0000 7f6717e98640 1 -- 172.21.15.172:0/1998822902 >> 172.21.15.172:0/1998822902 conn(0x7f671006b520 msgr2=0x7f671015acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:31.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.338+0000 7f6717e98640 1 -- 172.21.15.172:0/1998822902 shutdown_connections 2024-04-14T08:33:31.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.338+0000 7f6717e98640 1 -- 172.21.15.172:0/1998822902 wait complete. 2024-04-14T08:33:31.350 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:31.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.454+0000 7f709a5e0640 1 Processor -- start 2024-04-14T08:33:31.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.454+0000 7f709a5e0640 1 -- start start 2024-04-14T08:33:31.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.454+0000 7f709a5e0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70940af740 0x7f70940722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.454+0000 7f709a5e0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7094072820 0x7f709413fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.454+0000 7f709a5e0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70940ce860 con 0x7f709413fa70 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.454+0000 7f709a5e0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f70940712d0 con 0x7f70940af740 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.454+0000 7f709a5e0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7094071520 con 0x7f7094072820 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.455+0000 7f70937fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70940af740 0x7f70940722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.455+0000 7f7093fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f709413fa70 0x7f70940ce150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46472/0 (socket says 172.21.15.172:46472) 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.455+0000 7f7093fff640 1 -- 172.21.15.172:0/598971603 learned_addr learned my addr 172.21.15.172:0/598971603 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.455+0000 7f7098b7d640 1 --2- 172.21.15.172:0/598971603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7094072820 0x7f709413fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.455+0000 7f70937fe640 1 -- 172.21.15.172:0/598971603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7094072820 msgr2=0x7f709413fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.455+0000 7f70937fe640 1 --2- 172.21.15.172:0/598971603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7094072820 0x7f709413fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.455+0000 7f70937fe640 1 -- 172.21.15.172:0/598971603 >> v1:172.21.15.172:6789/0 conn(0x7f709413fa70 legacy=0x7f70940ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.455+0000 7f70937fe640 1 -- 172.21.15.172:0/598971603 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7094071800 con 0x7f70940af740 2024-04-14T08:33:31.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.455+0000 7f7092ffd640 1 -- 172.21.15.172:0/598971603 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91226362 0 0) 0x7f7084001110 con 0x7f709413fa70 2024-04-14T08:33:31.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.456+0000 7f70937fe640 1 --2- 172.21.15.172:0/598971603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70940af740 0x7f70940722e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f708800a940 tx=0x7f708802ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc16711c1a61b9fc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:31.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.456+0000 7f7092ffd640 1 -- 172.21.15.172:0/598971603 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708803a070 con 0x7f70940af740 2024-04-14T08:33:31.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.456+0000 7f7092ffd640 1 -- 172.21.15.172:0/598971603 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f708802daa0 con 0x7f70940af740 2024-04-14T08:33:31.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.456+0000 7f7092ffd640 1 -- 172.21.15.172:0/598971603 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708800b040 con 0x7f70940af740 2024-04-14T08:33:31.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.456+0000 7f709a5e0640 1 -- 172.21.15.172:0/598971603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70940af740 msgr2=0x7f70940722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.456+0000 7f709a5e0640 1 --2- 172.21.15.172:0/598971603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70940af740 0x7f70940722e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f708800a940 tx=0x7f708802ccf0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.456+0000 7f709a5e0640 1 -- 172.21.15.172:0/598971603 shutdown_connections 2024-04-14T08:33:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.456+0000 7f709a5e0640 1 --2- 172.21.15.172:0/598971603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7094072820 0x7f709413fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.456+0000 7f709a5e0640 1 --2- 172.21.15.172:0/598971603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70940af740 0x7f70940722e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.456+0000 7f709a5e0640 1 -- 172.21.15.172:0/598971603 >> 172.21.15.172:0/598971603 conn(0x7f709406b520 msgr2=0x7f709406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.456+0000 7f709a5e0640 1 -- 172.21.15.172:0/598971603 shutdown_connections 2024-04-14T08:33:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.456+0000 7f709a5e0640 1 -- 172.21.15.172:0/598971603 wait complete. 2024-04-14T08:33:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.458+0000 7f709a5e0640 1 Processor -- start 2024-04-14T08:33:31.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.458+0000 7f709a5e0640 1 -- start start 2024-04-14T08:33:31.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.458+0000 7f709a5e0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7094072820 0x7f70941f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.458+0000 7f709a5e0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70940af740 0x7f70941e2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f709a5e0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f709413fa70 0x7f70941f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f709a5e0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7094072c70 con 0x7f70940af740 2024-04-14T08:33:31.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f709a5e0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7094071dc0 con 0x7f7094072820 2024-04-14T08:33:31.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f709a5e0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7094144130 con 0x7f709413fa70 2024-04-14T08:33:31.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f70937fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70940af740 0x7f70941e2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f70937fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70940af740 0x7f70941e2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37104/0 (socket says 172.21.15.172:37104) 2024-04-14T08:33:31.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f70937fe640 1 -- 172.21.15.172:0/1332184742 learned_addr learned my addr 172.21.15.172:0/1332184742 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:31.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f7093fff640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7094072820 0x7f70941f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f7098b7d640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f709413fa70 0x7f70941f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f70937fe640 1 -- 172.21.15.172:0/1332184742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f709413fa70 msgr2=0x7f70941f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f70937fe640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f709413fa70 0x7f70941f7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f70937fe640 1 -- 172.21.15.172:0/1332184742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7094072820 msgr2=0x7f70941f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f70937fe640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7094072820 0x7f70941f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f70937fe640 1 -- 172.21.15.172:0/1332184742 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70941e34f0 con 0x7f70940af740 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f70937fe640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70940af740 0x7f70941e2fb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7088000c00 tx=0x7f70880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f7090ff9640 1 -- 172.21.15.172:0/1332184742 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088045070 con 0x7f70940af740 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f7090ff9640 1 -- 172.21.15.172:0/1332184742 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7088036db0 con 0x7f70940af740 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f709a5e0640 1 -- 172.21.15.172:0/1332184742 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70941f76e0 con 0x7f70940af740 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f7090ff9640 1 -- 172.21.15.172:0/1332184742 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708803a070 con 0x7f70940af740 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.459+0000 7f709a5e0640 1 -- 172.21.15.172:0/1332184742 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70941e3d60 con 0x7f70940af740 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.460+0000 7f709a5e0640 1 -- 172.21.15.172:0/1332184742 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7094068590 con 0x7f70940af740 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.461+0000 7f7090ff9640 1 -- 172.21.15.172:0/1332184742 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7088048070 con 0x7f70940af740 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.461+0000 7f7090ff9640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f708006c8f0 0x7f708006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.461+0000 7f7090ff9640 1 -- 172.21.15.172:0/1332184742 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f70880ad4f0 con 0x7f70940af740 2024-04-14T08:33:31.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.461+0000 7f7093fff640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f708006c8f0 0x7f708006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.462+0000 7f7093fff640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f708006c8f0 0x7f708006edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7084007fd0 tx=0x7f7084007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:31.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.466+0000 7f7090ff9640 1 -- 172.21.15.172:0/1332184742 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f708807b020 con 0x7f70940af740 2024-04-14T08:33:31.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.640+0000 7f709a5e0640 1 -- 172.21.15.172:0/1332184742 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f70940b0bb0 con 0x7f70940af740 2024-04-14T08:33:31.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.641+0000 7f7090ff9640 1 -- 172.21.15.172:0/1332184742 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f708807f320 con 0x7f70940af740 2024-04-14T08:33:31.641 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:31.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.643+0000 7f709a5e0640 1 -- 172.21.15.172:0/1332184742 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f708006c8f0 msgr2=0x7f708006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.643+0000 7f709a5e0640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f708006c8f0 0x7f708006edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7084007fd0 tx=0x7f7084007de0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.643+0000 7f709a5e0640 1 -- 172.21.15.172:0/1332184742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70940af740 msgr2=0x7f70941e2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.643+0000 7f709a5e0640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70940af740 0x7f70941e2fb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7088000c00 tx=0x7f70880046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.643+0000 7f709a5e0640 1 -- 172.21.15.172:0/1332184742 shutdown_connections 2024-04-14T08:33:31.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.643+0000 7f709a5e0640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f708006c8f0 0x7f708006edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.643+0000 7f709a5e0640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f709413fa70 0x7f70941f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.643+0000 7f709a5e0640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70940af740 0x7f70941e2fb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.643+0000 7f709a5e0640 1 --2- 172.21.15.172:0/1332184742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7094072820 0x7f70941f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.643+0000 7f709a5e0640 1 -- 172.21.15.172:0/1332184742 >> 172.21.15.172:0/1332184742 conn(0x7f709406b520 msgr2=0x7f70940b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:31.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.643+0000 7f709a5e0640 1 -- 172.21.15.172:0/1332184742 shutdown_connections 2024-04-14T08:33:31.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.643+0000 7f709a5e0640 1 -- 172.21.15.172:0/1332184742 wait complete. 2024-04-14T08:33:31.654 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:31.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.757+0000 7f58d4888640 1 Processor -- start 2024-04-14T08:33:31.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.757+0000 7f58d4888640 1 -- start start 2024-04-14T08:33:31.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.758+0000 7f58d4888640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f58d00cf210 0x7f58d00b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.758+0000 7f58d4888640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58d00ccdd0 con 0x7f58d015c420 2024-04-14T08:33:31.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.758+0000 7f58d4888640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58d00d2ae0 con 0x7f58d006f3e0 2024-04-14T08:33:31.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.758+0000 7f58d4888640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f58d00d30a0 con 0x7f58d00cf210 2024-04-14T08:33:31.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.758+0000 7f58ce59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f58d015c420 0x7f58d00d0010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46476/0 (socket says 172.21.15.172:46476) 2024-04-14T08:33:31.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.758+0000 7f58ce59c640 1 -- 172.21.15.172:0/2524802249 learned_addr learned my addr 172.21.15.172:0/2524802249 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:31.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.758+0000 7f58ced9d640 1 --2- 172.21.15.172:0/2524802249 >> v2:172.21.15.177:3300/0 conn(0x7f58d00cf210 0x7f58d00b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.759+0000 7f58cdd9b640 1 -- 172.21.15.172:0/2524802249 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1554692000 0 0) 0x7f58d00ccdd0 con 0x7f58d015c420 2024-04-14T08:33:31.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.759+0000 7f58cdd9b640 1 -- 172.21.15.172:0/2524802249 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58bc003520 con 0x7f58d015c420 2024-04-14T08:33:31.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.759+0000 7f58cdd9b640 1 -- 172.21.15.172:0/2524802249 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3304059579 0 0) 0x7f58d00d2ae0 con 0x7f58d006f3e0 2024-04-14T08:33:31.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.759+0000 7f58cdd9b640 1 -- 172.21.15.172:0/2524802249 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58d00ccdd0 con 0x7f58d006f3e0 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.759+0000 7f58ced9d640 1 -- 172.21.15.172:0/2524802249 >> v1:172.21.15.177:6789/0 conn(0x7f58d006f3e0 legacy=0x7f58d00ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.759+0000 7f58ced9d640 1 -- 172.21.15.172:0/2524802249 >> v1:172.21.15.172:6789/0 conn(0x7f58d015c420 legacy=0x7f58d00d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.759+0000 7f58ced9d640 1 -- 172.21.15.172:0/2524802249 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d00d2dc0 con 0x7f58d00cf210 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.760+0000 7f58ced9d640 1 --2- 172.21.15.172:0/2524802249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d00cf210 0x7f58d00b1510 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009d90 tx=0x7f58c802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec60d76a623d88e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.760+0000 7f58cdd9b640 1 -- 172.21.15.172:0/2524802249 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8039070 con 0x7f58d00cf210 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.760+0000 7f58cdd9b640 1 -- 172.21.15.172:0/2524802249 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c802cd00 con 0x7f58d00cf210 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.760+0000 7f58cdd9b640 1 -- 172.21.15.172:0/2524802249 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8034640 con 0x7f58d00cf210 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.760+0000 7f58d4888640 1 -- 172.21.15.172:0/2524802249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d00cf210 msgr2=0x7f58d00b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.760+0000 7f58d4888640 1 --2- 172.21.15.172:0/2524802249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d00cf210 0x7f58d00b1510 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009d90 tx=0x7f58c802bf50 comp rx=0 tx=0).stop 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.760+0000 7f58d4888640 1 -- 172.21.15.172:0/2524802249 shutdown_connections 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.760+0000 7f58d4888640 1 --2- 172.21.15.172:0/2524802249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d00cf210 0x7f58d00b1510 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.760+0000 7f58d4888640 1 -- 172.21.15.172:0/2524802249 >> 172.21.15.172:0/2524802249 conn(0x7f58d008c000 msgr2=0x7f58d00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.760+0000 7f58d4888640 1 -- 172.21.15.172:0/2524802249 shutdown_connections 2024-04-14T08:33:31.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.760+0000 7f58d4888640 1 -- 172.21.15.172:0/2524802249 wait complete. 2024-04-14T08:33:31.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.762+0000 7f58d4888640 1 Processor -- start 2024-04-14T08:33:31.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.762+0000 7f58d4888640 1 -- start start 2024-04-14T08:33:31.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.762+0000 7f58d4888640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d00cf210 0x7f58d015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.762+0000 7f58d4888640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58d015c420 0x7f58d00cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.762+0000 7f58d4888640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d00b4d30 0x7f58d00b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.762+0000 7f58d4888640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f58d00b11e0 con 0x7f58d00cf210 2024-04-14T08:33:31.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.762+0000 7f58d4888640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f58d006f870 con 0x7f58d00b4d30 2024-04-14T08:33:31.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.762+0000 7f58d4888640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f58d015c800 con 0x7f58d015c420 2024-04-14T08:33:31.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58ce59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d00cf210 0x7f58d015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58ce59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d00cf210 0x7f58d015f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37118/0 (socket says 172.21.15.172:37118) 2024-04-14T08:33:31.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58ce59c640 1 -- 172.21.15.172:0/2800717381 learned_addr learned my addr 172.21.15.172:0/2800717381 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:31.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58ced9d640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d00b4d30 0x7f58d00b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58c5d9b640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58d015c420 0x7f58d00cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58ced9d640 1 -- 172.21.15.172:0/2800717381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58d015c420 msgr2=0x7f58d00cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58ced9d640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58d015c420 0x7f58d00cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58ced9d640 1 -- 172.21.15.172:0/2800717381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d00cf210 msgr2=0x7f58d015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58ced9d640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d00cf210 0x7f58d015f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58ced9d640 1 -- 172.21.15.172:0/2800717381 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d00cb680 con 0x7f58d00b4d30 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58ce59c640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d00cf210 0x7f58d015f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58ced9d640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d00b4d30 0x7f58d00b51e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f58d00ccfa0 tx=0x7f58c8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58c77fe640 1 -- 172.21.15.172:0/2800717381 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8039050 con 0x7f58d00b4d30 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58c77fe640 1 -- 172.21.15.172:0/2800717381 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c803a040 con 0x7f58d00b4d30 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58d4888640 1 -- 172.21.15.172:0/2800717381 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d00b5ab0 con 0x7f58d00b4d30 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58c77fe640 1 -- 172.21.15.172:0/2800717381 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8009450 con 0x7f58d00b4d30 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.763+0000 7f58d4888640 1 -- 172.21.15.172:0/2800717381 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d00c5040 con 0x7f58d00b4d30 2024-04-14T08:33:31.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.764+0000 7f58d4888640 1 -- 172.21.15.172:0/2800717381 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d0075490 con 0x7f58d00b4d30 2024-04-14T08:33:31.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.765+0000 7f58c77fe640 1 -- 172.21.15.172:0/2800717381 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58c8047020 con 0x7f58d00b4d30 2024-04-14T08:33:31.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.765+0000 7f58c77fe640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58b006c920 0x7f58b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:31.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.765+0000 7f58c77fe640 1 -- 172.21.15.172:0/2800717381 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f58c80add80 con 0x7f58d00b4d30 2024-04-14T08:33:31.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.765+0000 7f58ce59c640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58b006c920 0x7f58b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:31.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.766+0000 7f58ce59c640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58b006c920 0x7f58b006ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f58c0000f60 tx=0x7f58c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:31.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.770+0000 7f58c77fe640 1 -- 172.21.15.172:0/2800717381 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f58c804e020 con 0x7f58d00b4d30 2024-04-14T08:33:31.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.944+0000 7f58d4888640 1 -- 172.21.15.172:0/2800717381 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f58d00c8fa0 con 0x7f58d00b4d30 2024-04-14T08:33:31.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.945+0000 7f58c77fe640 1 -- 172.21.15.172:0/2800717381 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f58c807b8b0 con 0x7f58d00b4d30 2024-04-14T08:33:31.945 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.947+0000 7f58d4888640 1 -- 172.21.15.172:0/2800717381 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58b006c920 msgr2=0x7f58b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.947+0000 7f58d4888640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58b006c920 0x7f58b006ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f58c0000f60 tx=0x7f58c0009040 comp rx=0 tx=0).stop 2024-04-14T08:33:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.947+0000 7f58d4888640 1 -- 172.21.15.172:0/2800717381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d00b4d30 msgr2=0x7f58d00b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.947+0000 7f58d4888640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d00b4d30 0x7f58d00b51e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f58d00ccfa0 tx=0x7f58c8004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.947+0000 7f58d4888640 1 -- 172.21.15.172:0/2800717381 shutdown_connections 2024-04-14T08:33:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.947+0000 7f58d4888640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58b006c920 0x7f58b006ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.947+0000 7f58d4888640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d00b4d30 0x7f58d00b51e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.947+0000 7f58d4888640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f58d015c420 0x7f58d00cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.947+0000 7f58d4888640 1 --2- 172.21.15.172:0/2800717381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d00cf210 0x7f58d015f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.947+0000 7f58d4888640 1 -- 172.21.15.172:0/2800717381 >> 172.21.15.172:0/2800717381 conn(0x7f58d008c000 msgr2=0x7f58d00d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.947+0000 7f58d4888640 1 -- 172.21.15.172:0/2800717381 shutdown_connections 2024-04-14T08:33:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:31.947+0000 7f58d4888640 1 -- 172.21.15.172:0/2800717381 wait complete. 2024-04-14T08:33:31.958 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:32.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.062+0000 7f3274236640 1 Processor -- start 2024-04-14T08:33:32.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.062+0000 7f3274236640 1 -- start start 2024-04-14T08:33:32.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.063+0000 7f3274236640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f326c0b23e0 0x7f326c140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.063+0000 7f3274236640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f326c15c380 0x7f326c140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.063+0000 7f3274236640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f326c15b620 con 0x7f326c13df30 2024-04-14T08:33:32.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.063+0000 7f3274236640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f326c0cf9b0 con 0x7f326c15c380 2024-04-14T08:33:32.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.063+0000 7f3274236640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f326c1411f0 con 0x7f326c0b23e0 2024-04-14T08:33:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.063+0000 7f32717d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f326c0b23e0 0x7f326c140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.063+0000 7f32727d3640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f326c15c380 0x7f326c140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.063+0000 7f3271fd2640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f326c13df30 0x7f326c0b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:51204/0 (socket says 172.21.15.172:51204) 2024-04-14T08:33:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.063+0000 7f3271fd2640 1 -- 172.21.15.172:0/4247783455 learned_addr learned my addr 172.21.15.172:0/4247783455 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.064+0000 7f3270fd0640 1 -- 172.21.15.172:0/4247783455 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 597047624 0 0) 0x7f3268001490 con 0x7f326c13df30 2024-04-14T08:33:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.064+0000 7f3270fd0640 1 -- 172.21.15.172:0/4247783455 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3258003520 con 0x7f326c13df30 2024-04-14T08:33:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.064+0000 7f32727d3640 1 -- 172.21.15.172:0/4247783455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f326c0b23e0 msgr2=0x7f326c140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.064+0000 7f32727d3640 1 --2- 172.21.15.172:0/4247783455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f326c0b23e0 0x7f326c140770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.064+0000 7f32727d3640 1 -- 172.21.15.172:0/4247783455 >> v1:172.21.15.177:6789/0 conn(0x7f326c13df30 legacy=0x7f326c0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.064+0000 7f32727d3640 1 -- 172.21.15.172:0/4247783455 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c1414a0 con 0x7f326c15c380 2024-04-14T08:33:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.064+0000 7f32717d1640 1 --2- 172.21.15.172:0/4247783455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f326c0b23e0 0x7f326c140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.064+0000 7f32727d3640 1 --2- 172.21.15.172:0/4247783455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f326c15c380 0x7f326c140cb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f32640097a0 tx=0x7f326402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=880f39a49ee82e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.065+0000 7f3270fd0640 1 -- 172.21.15.172:0/4247783455 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264035020 con 0x7f326c15c380 2024-04-14T08:33:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.065+0000 7f3270fd0640 1 -- 172.21.15.172:0/4247783455 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3264031830 con 0x7f326c15c380 2024-04-14T08:33:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.065+0000 7f3270fd0640 1 -- 172.21.15.172:0/4247783455 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264031be0 con 0x7f326c15c380 2024-04-14T08:33:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.065+0000 7f3274236640 1 -- 172.21.15.172:0/4247783455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f326c15c380 msgr2=0x7f326c140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.065+0000 7f3274236640 1 --2- 172.21.15.172:0/4247783455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f326c15c380 0x7f326c140cb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f32640097a0 tx=0x7f326402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.065+0000 7f3274236640 1 -- 172.21.15.172:0/4247783455 shutdown_connections 2024-04-14T08:33:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.065+0000 7f3274236640 1 --2- 172.21.15.172:0/4247783455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f326c15c380 0x7f326c140cb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.065+0000 7f3274236640 1 --2- 172.21.15.172:0/4247783455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f326c0b23e0 0x7f326c140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.065+0000 7f3274236640 1 -- 172.21.15.172:0/4247783455 >> 172.21.15.172:0/4247783455 conn(0x7f326c06b520 msgr2=0x7f326c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.065+0000 7f3274236640 1 -- 172.21.15.172:0/4247783455 shutdown_connections 2024-04-14T08:33:32.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.065+0000 7f3274236640 1 -- 172.21.15.172:0/4247783455 wait complete. 2024-04-14T08:33:32.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.066+0000 7f3274236640 1 Processor -- start 2024-04-14T08:33:32.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.066+0000 7f3274236640 1 -- start start 2024-04-14T08:33:32.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.067+0000 7f3274236640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f326c0b23e0 0x7f326c134100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.067+0000 7f3274236640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f326c13df30 0x7f326c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.067+0000 7f3274236640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f326c15c380 0x7f326c14eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.067+0000 7f3274236640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f326c15be60 con 0x7f326c13df30 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.067+0000 7f3274236640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f326c15bb80 con 0x7f326c15c380 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.067+0000 7f3274236640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f326c13e310 con 0x7f326c0b23e0 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.067+0000 7f32717d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f326c13df30 0x7f326c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.067+0000 7f32717d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f326c13df30 0x7f326c0cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37132/0 (socket says 172.21.15.172:37132) 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.067+0000 7f32717d1640 1 -- 172.21.15.172:0/3685294046 learned_addr learned my addr 172.21.15.172:0/3685294046 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.067+0000 7f32727d3640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f326c15c380 0x7f326c14eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.067+0000 7f3271fd2640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f326c0b23e0 0x7f326c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.068+0000 7f32717d1640 1 -- 172.21.15.172:0/3685294046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f326c0b23e0 msgr2=0x7f326c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.068+0000 7f32717d1640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f326c0b23e0 0x7f326c134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.068+0000 7f32717d1640 1 -- 172.21.15.172:0/3685294046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f326c15c380 msgr2=0x7f326c14eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.068+0000 7f32717d1640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f326c15c380 0x7f326c14eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.068+0000 7f32717d1640 1 -- 172.21.15.172:0/3685294046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c0ccc10 con 0x7f326c13df30 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.068+0000 7f32727d3640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f326c15c380 0x7f326c14eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:32.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.068+0000 7f32717d1640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f326c13df30 0x7f326c0cc230 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f325c00e960 tx=0x7f325c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:32.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.068+0000 7f32627fc640 1 -- 172.21.15.172:0/3685294046 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c017070 con 0x7f326c13df30 2024-04-14T08:33:32.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.068+0000 7f32627fc640 1 -- 172.21.15.172:0/3685294046 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f325c00ca00 con 0x7f326c13df30 2024-04-14T08:33:32.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.068+0000 7f32627fc640 1 -- 172.21.15.172:0/3685294046 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c00cdd0 con 0x7f326c13df30 2024-04-14T08:33:32.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.068+0000 7f3274236640 1 -- 172.21.15.172:0/3685294046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326c1f5cb0 con 0x7f326c13df30 2024-04-14T08:33:32.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.068+0000 7f3274236640 1 -- 172.21.15.172:0/3685294046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326c14e0b0 con 0x7f326c13df30 2024-04-14T08:33:32.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.069+0000 7f3274236640 1 -- 172.21.15.172:0/3685294046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f326c155370 con 0x7f326c13df30 2024-04-14T08:33:32.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.070+0000 7f32627fc640 1 -- 172.21.15.172:0/3685294046 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f325c01a070 con 0x7f326c13df30 2024-04-14T08:33:32.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.070+0000 7f32627fc640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f325006c8f0 0x7f325006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.070+0000 7f32627fc640 1 -- 172.21.15.172:0/3685294046 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f325c08bc30 con 0x7f326c13df30 2024-04-14T08:33:32.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.070+0000 7f3271fd2640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f325006c8f0 0x7f325006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.070+0000 7f3271fd2640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f325006c8f0 0x7f325006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3268005c00 tx=0x7f3268009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:32.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.074+0000 7f32627fc640 1 -- 172.21.15.172:0/3685294046 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f326c155370 con 0x7f326c13df30 2024-04-14T08:33:32.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.248+0000 7f3274236640 1 -- 172.21.15.172:0/3685294046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f326c068590 con 0x7f326c13df30 2024-04-14T08:33:32.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.249+0000 7f32627fc640 1 -- 172.21.15.172:0/3685294046 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f325c059760 con 0x7f326c13df30 2024-04-14T08:33:32.249 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:32.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.251+0000 7f3274236640 1 -- 172.21.15.172:0/3685294046 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f325006c8f0 msgr2=0x7f325006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.251+0000 7f3274236640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f325006c8f0 0x7f325006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3268005c00 tx=0x7f3268009040 comp rx=0 tx=0).stop 2024-04-14T08:33:32.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.251+0000 7f3274236640 1 -- 172.21.15.172:0/3685294046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f326c13df30 msgr2=0x7f326c0cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.251+0000 7f3274236640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f326c13df30 0x7f326c0cc230 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f325c00e960 tx=0x7f325c00ee20 comp rx=0 tx=0).stop 2024-04-14T08:33:32.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.251+0000 7f3274236640 1 -- 172.21.15.172:0/3685294046 shutdown_connections 2024-04-14T08:33:32.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.251+0000 7f3274236640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f325006c8f0 0x7f325006edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.251+0000 7f3274236640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f326c15c380 0x7f326c14eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.251+0000 7f3274236640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f326c13df30 0x7f326c0cc230 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.251+0000 7f3274236640 1 --2- 172.21.15.172:0/3685294046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f326c0b23e0 0x7f326c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.251+0000 7f3274236640 1 -- 172.21.15.172:0/3685294046 >> 172.21.15.172:0/3685294046 conn(0x7f326c06b520 msgr2=0x7f326c0b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:32.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.251+0000 7f3274236640 1 -- 172.21.15.172:0/3685294046 shutdown_connections 2024-04-14T08:33:32.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.251+0000 7f3274236640 1 -- 172.21.15.172:0/3685294046 wait complete. 2024-04-14T08:33:32.262 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:32.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.363+0000 7f3c15ffe640 1 Processor -- start 2024-04-14T08:33:32.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.363+0000 7f3c15ffe640 1 -- start start 2024-04-14T08:33:32.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c15ffe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3c100b21d0 0x7f3c1015c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c15ffe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3c1015c5a0 0x7f3c100b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c15ffe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c100b3700 con 0x7f3c100b1df0 2024-04-14T08:33:32.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c15ffe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f3c100b39e0 con 0x7f3c1015c5a0 2024-04-14T08:33:32.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c15ffe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f3c100b3cc0 con 0x7f3c100b21d0 2024-04-14T08:33:32.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c0ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3c1015c5a0 0x7f3c100b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c0ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3c1015c5a0 0x7f3c100b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37146/0 (socket says 172.21.15.172:37146) 2024-04-14T08:33:32.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c0ffff640 1 -- 172.21.15.172:0/4135123484 learned_addr learned my addr 172.21.15.172:0/4135123484 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:32.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c0f7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3c100b1df0 0x7f3c100b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46482/0 (socket says 172.21.15.172:46482) 2024-04-14T08:33:32.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c0effd640 1 --2- 172.21.15.172:0/4135123484 >> v2:172.21.15.177:3300/0 conn(0x7f3c100b21d0 0x7f3c1015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c0ffff640 1 -- 172.21.15.172:0/4135123484 >> v2:172.21.15.177:3300/0 conn(0x7f3c100b21d0 msgr2=0x7f3c1015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:32.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/4135123484 >> v2:172.21.15.177:3300/0 conn(0x7f3c100b21d0 0x7f3c1015c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c0ffff640 1 -- 172.21.15.172:0/4135123484 >> v1:172.21.15.172:6789/0 conn(0x7f3c100b1df0 legacy=0x7f3c100b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c0e7fc640 1 -- 172.21.15.172:0/4135123484 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4027017834 0 0) 0x7f3c100b3700 con 0x7f3c100b1df0 2024-04-14T08:33:32.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.364+0000 7f3c0ffff640 1 -- 172.21.15.172:0/4135123484 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1015bb40 con 0x7f3c1015c5a0 2024-04-14T08:33:32.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.365+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/4135123484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c1015c5a0 0x7f3c100b4040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c1006c0d0 tx=0x7f3c0802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2494d42a3b977af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:32.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.365+0000 7f3c0e7fc640 1 -- 172.21.15.172:0/4135123484 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0802cb20 con 0x7f3c1015c5a0 2024-04-14T08:33:32.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.365+0000 7f3c0e7fc640 1 -- 172.21.15.172:0/4135123484 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0802ccc0 con 0x7f3c1015c5a0 2024-04-14T08:33:32.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.365+0000 7f3c0e7fc640 1 -- 172.21.15.172:0/4135123484 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08034610 con 0x7f3c1015c5a0 2024-04-14T08:33:32.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.366+0000 7f3c15ffe640 1 -- 172.21.15.172:0/4135123484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c1015c5a0 msgr2=0x7f3c100b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.366+0000 7f3c15ffe640 1 --2- 172.21.15.172:0/4135123484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c1015c5a0 0x7f3c100b4040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c1006c0d0 tx=0x7f3c0802bf80 comp rx=0 tx=0).stop 2024-04-14T08:33:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.366+0000 7f3c15ffe640 1 -- 172.21.15.172:0/4135123484 shutdown_connections 2024-04-14T08:33:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.366+0000 7f3c15ffe640 1 --2- 172.21.15.172:0/4135123484 >> v2:172.21.15.177:3300/0 conn(0x7f3c100b21d0 0x7f3c1015c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.366+0000 7f3c15ffe640 1 --2- 172.21.15.172:0/4135123484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c1015c5a0 0x7f3c100b4040 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.366+0000 7f3c15ffe640 1 -- 172.21.15.172:0/4135123484 >> 172.21.15.172:0/4135123484 conn(0x7f3c1006ec30 msgr2=0x7f3c1006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.366+0000 7f3c15ffe640 1 -- 172.21.15.172:0/4135123484 shutdown_connections 2024-04-14T08:33:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.366+0000 7f3c15ffe640 1 -- 172.21.15.172:0/4135123484 wait complete. 2024-04-14T08:33:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.367+0000 7f3c15ffe640 1 Processor -- start 2024-04-14T08:33:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.367+0000 7f3c15ffe640 1 -- start start 2024-04-14T08:33:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c15ffe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c100b1df0 0x7f3c10157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c15ffe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c100b21d0 0x7f3c101593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c15ffe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c10136150 0x7f3c10159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c15ffe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c100b4700 con 0x7f3c10136150 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c15ffe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3c100b4580 con 0x7f3c100b1df0 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c15ffe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3c100b4880 con 0x7f3c100b21d0 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c0ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c10136150 0x7f3c10159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c0ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c10136150 0x7f3c10159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37160/0 (socket says 172.21.15.172:37160) 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c0ffff640 1 -- 172.21.15.172:0/451400063 learned_addr learned my addr 172.21.15.172:0/451400063 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c0effd640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c100b21d0 0x7f3c101593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c100b1df0 0x7f3c10157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c0ffff640 1 -- 172.21.15.172:0/451400063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c100b21d0 msgr2=0x7f3c101593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c100b21d0 0x7f3c101593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c0ffff640 1 -- 172.21.15.172:0/451400063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c100b1df0 msgr2=0x7f3c10157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c100b1df0 0x7f3c10157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c0ffff640 1 -- 172.21.15.172:0/451400063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c101435a0 con 0x7f3c10136150 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c10136150 0x7f3c10159930 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3c08036040 tx=0x7f3c08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:32.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3bf7fff640 1 -- 172.21.15.172:0/451400063 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08043020 con 0x7f3c10136150 2024-04-14T08:33:32.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3bf7fff640 1 -- 172.21.15.172:0/451400063 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0803e450 con 0x7f3c10136150 2024-04-14T08:33:32.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3c15ffe640 1 -- 172.21.15.172:0/451400063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c10135670 con 0x7f3c10136150 2024-04-14T08:33:32.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.368+0000 7f3bf7fff640 1 -- 172.21.15.172:0/451400063 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08009510 con 0x7f3c10136150 2024-04-14T08:33:32.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.369+0000 7f3c15ffe640 1 -- 172.21.15.172:0/451400063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c10135c30 con 0x7f3c10136150 2024-04-14T08:33:32.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.370+0000 7f3c15ffe640 1 -- 172.21.15.172:0/451400063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c10135470 con 0x7f3c10136150 2024-04-14T08:33:32.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.370+0000 7f3bf7fff640 1 -- 172.21.15.172:0/451400063 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c08046020 con 0x7f3c10136150 2024-04-14T08:33:32.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.370+0000 7f3bf7fff640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3bfc06c8f0 0x7f3bfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.370+0000 7f3bf7fff640 1 -- 172.21.15.172:0/451400063 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3c080ae4a0 con 0x7f3c10136150 2024-04-14T08:33:32.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.371+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3bfc06c8f0 0x7f3bfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.371+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3bfc06c8f0 0x7f3bfc06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c00007fd0 tx=0x7f3c00007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:32.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.375+0000 7f3bf7fff640 1 -- 172.21.15.172:0/451400063 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3c0804d030 con 0x7f3c10136150 2024-04-14T08:33:32.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.552+0000 7f3c15ffe640 1 -- 172.21.15.172:0/451400063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c10068520 con 0x7f3c10136150 2024-04-14T08:33:32.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.553+0000 7f3bf7fff640 1 -- 172.21.15.172:0/451400063 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18574 (secure 0 0 0) 0x7f3c0807bf40 con 0x7f3c10136150 2024-04-14T08:33:32.554 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:32.554 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:32:58.586203+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:32.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.555+0000 7f3c15ffe640 1 -- 172.21.15.172:0/451400063 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3bfc06c8f0 msgr2=0x7f3bfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.555+0000 7f3c15ffe640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3bfc06c8f0 0x7f3bfc06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c00007fd0 tx=0x7f3c00007de0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.555+0000 7f3c15ffe640 1 -- 172.21.15.172:0/451400063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c10136150 msgr2=0x7f3c10159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.555+0000 7f3c15ffe640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c10136150 0x7f3c10159930 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3c08036040 tx=0x7f3c08004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:32.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.555+0000 7f3c15ffe640 1 -- 172.21.15.172:0/451400063 shutdown_connections 2024-04-14T08:33:32.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.555+0000 7f3c15ffe640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3bfc06c8f0 0x7f3bfc06edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.555+0000 7f3c15ffe640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c10136150 0x7f3c10159930 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.555+0000 7f3c15ffe640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c100b21d0 0x7f3c101593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.555+0000 7f3c15ffe640 1 --2- 172.21.15.172:0/451400063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c100b1df0 0x7f3c10157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.555+0000 7f3c15ffe640 1 -- 172.21.15.172:0/451400063 >> 172.21.15.172:0/451400063 conn(0x7f3c1006ec30 msgr2=0x7f3c100d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:32.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.555+0000 7f3c15ffe640 1 -- 172.21.15.172:0/451400063 shutdown_connections 2024-04-14T08:33:32.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.556+0000 7f3c15ffe640 1 -- 172.21.15.172:0/451400063 wait complete. 2024-04-14T08:33:32.567 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:32.568 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:32.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.671+0000 7fbb58faa640 1 Processor -- start 2024-04-14T08:33:32.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.671+0000 7fbb58faa640 1 -- start start 2024-04-14T08:33:32.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb58faa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb5415c380 0x7fbb54140e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb58faa640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb5415b620 con 0x7fbb540b23e0 2024-04-14T08:33:32.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb58faa640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb541414a0 con 0x7fbb5413df30 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb58faa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbb540cf9b0 con 0x7fbb5415c380 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb4bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fbb540b23e0 0x7fbb54140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46486/0 (socket says 172.21.15.172:46486) 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb5259c640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fbb5413df30 0x7fbb540b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:51214/0 (socket says 172.21.15.172:51214) 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb4bfff640 1 -- 172.21.15.172:0/1205750629 learned_addr learned my addr 172.21.15.172:0/1205750629 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb52d9d640 1 --2- 172.21.15.172:0/1205750629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb5415c380 0x7fbb54140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb51d9b640 1 -- 172.21.15.172:0/1205750629 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 476200337 0 0) 0x7fbb3c001110 con 0x7fbb540b23e0 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb51d9b640 1 -- 172.21.15.172:0/1205750629 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb40003520 con 0x7fbb540b23e0 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb51d9b640 1 -- 172.21.15.172:0/1205750629 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3435389586 0 0) 0x7fbb541414a0 con 0x7fbb5413df30 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb52d9d640 1 -- 172.21.15.172:0/1205750629 >> v1:172.21.15.177:6789/0 conn(0x7fbb5413df30 legacy=0x7fbb540b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb52d9d640 1 -- 172.21.15.172:0/1205750629 >> v1:172.21.15.172:6789/0 conn(0x7fbb540b23e0 legacy=0x7fbb54140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.672+0000 7fbb52d9d640 1 -- 172.21.15.172:0/1205750629 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5415b8a0 con 0x7fbb5415c380 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.673+0000 7fbb52d9d640 1 --2- 172.21.15.172:0/1205750629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb5415c380 0x7fbb54140e80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009dc0 tx=0x7fbb4c02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9135805c5a8523 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.673+0000 7fbb51d9b640 1 -- 172.21.15.172:0/1205750629 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c039070 con 0x7fbb5415c380 2024-04-14T08:33:32.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.673+0000 7fbb51d9b640 1 -- 172.21.15.172:0/1205750629 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4c02ccd0 con 0x7fbb5415c380 2024-04-14T08:33:32.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.673+0000 7fbb51d9b640 1 -- 172.21.15.172:0/1205750629 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c034640 con 0x7fbb5415c380 2024-04-14T08:33:32.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.674+0000 7fbb58faa640 1 -- 172.21.15.172:0/1205750629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb5415c380 msgr2=0x7fbb54140e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.674+0000 7fbb58faa640 1 --2- 172.21.15.172:0/1205750629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb5415c380 0x7fbb54140e80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009dc0 tx=0x7fbb4c02bf20 comp rx=0 tx=0).stop 2024-04-14T08:33:32.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.674+0000 7fbb58faa640 1 -- 172.21.15.172:0/1205750629 shutdown_connections 2024-04-14T08:33:32.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.674+0000 7fbb58faa640 1 --2- 172.21.15.172:0/1205750629 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb5415c380 0x7fbb54140e80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.674+0000 7fbb58faa640 1 -- 172.21.15.172:0/1205750629 >> 172.21.15.172:0/1205750629 conn(0x7fbb5406b520 msgr2=0x7fbb5406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:32.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.674+0000 7fbb58faa640 1 -- 172.21.15.172:0/1205750629 shutdown_connections 2024-04-14T08:33:32.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.674+0000 7fbb58faa640 1 -- 172.21.15.172:0/1205750629 wait complete. 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.675+0000 7fbb58faa640 1 Processor -- start 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.675+0000 7fbb58faa640 1 -- start start 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb58faa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb540b23e0 0x7fbb541511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb58faa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb5413df30 0x7fbb540ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb58faa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb5415c380 0x7fbb5413a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb58faa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5413e310 con 0x7fbb5413df30 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb58faa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5415be60 con 0x7fbb5415c380 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb58faa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbb54165fa0 con 0x7fbb540b23e0 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb4bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb5413df30 0x7fbb540ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb5259c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb540b23e0 0x7fbb541511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb4bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb5413df30 0x7fbb540ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37174/0 (socket says 172.21.15.172:37174) 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb4bfff640 1 -- 172.21.15.172:0/3628259068 learned_addr learned my addr 172.21.15.172:0/3628259068 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:32.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb52d9d640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb5415c380 0x7fbb5413a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb4bfff640 1 -- 172.21.15.172:0/3628259068 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb540b23e0 msgr2=0x7fbb541511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb4bfff640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb540b23e0 0x7fbb541511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb4bfff640 1 -- 172.21.15.172:0/3628259068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb5415c380 msgr2=0x7fbb5413a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb4bfff640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb5415c380 0x7fbb5413a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb4bfff640 1 -- 172.21.15.172:0/3628259068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb54151730 con 0x7fbb5413df30 2024-04-14T08:33:32.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb4bfff640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb5413df30 0x7fbb540ae0d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c430 tx=0x7fbb3c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:32.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.676+0000 7fbb4affd640 1 -- 172.21.15.172:0/3628259068 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c017070 con 0x7fbb5413df30 2024-04-14T08:33:32.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.677+0000 7fbb4affd640 1 -- 172.21.15.172:0/3628259068 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb3c005ce0 con 0x7fbb5413df30 2024-04-14T08:33:32.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.677+0000 7fbb58faa640 1 -- 172.21.15.172:0/3628259068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb540aeab0 con 0x7fbb5413df30 2024-04-14T08:33:32.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.677+0000 7fbb4affd640 1 -- 172.21.15.172:0/3628259068 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0072d0 con 0x7fbb5413df30 2024-04-14T08:33:32.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.677+0000 7fbb58faa640 1 -- 172.21.15.172:0/3628259068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb540af070 con 0x7fbb5413df30 2024-04-14T08:33:32.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.677+0000 7fbb58faa640 1 -- 172.21.15.172:0/3628259068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb54067dd0 con 0x7fbb5413df30 2024-04-14T08:33:32.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.678+0000 7fbb4affd640 1 -- 172.21.15.172:0/3628259068 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb3c01a070 con 0x7fbb5413df30 2024-04-14T08:33:32.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.678+0000 7fbb4affd640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb3406c8f0 0x7fbb3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.678+0000 7fbb4affd640 1 -- 172.21.15.172:0/3628259068 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbb3c08b1a0 con 0x7fbb5413df30 2024-04-14T08:33:32.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.678+0000 7fbb5259c640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb3406c8f0 0x7fbb3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.679+0000 7fbb5259c640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb3406c8f0 0x7fbb3406edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb44001110 tx=0x7fbb44001080 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:32.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.683+0000 7fbb4affd640 1 -- 172.21.15.172:0/3628259068 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbb54067dd0 con 0x7fbb5413df30 2024-04-14T08:33:32.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.856+0000 7fbb58faa640 1 -- 172.21.15.172:0/3628259068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb54068590 con 0x7fbb5413df30 2024-04-14T08:33:32.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.856+0000 7fbb4affd640 1 -- 172.21.15.172:0/3628259068 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fbb54151730 con 0x7fbb5413df30 2024-04-14T08:33:32.857 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:32.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.858+0000 7fbb58faa640 1 -- 172.21.15.172:0/3628259068 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb3406c8f0 msgr2=0x7fbb3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.858+0000 7fbb58faa640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb3406c8f0 0x7fbb3406edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb44001110 tx=0x7fbb44001080 comp rx=0 tx=0).stop 2024-04-14T08:33:32.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.858+0000 7fbb58faa640 1 -- 172.21.15.172:0/3628259068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb5413df30 msgr2=0x7fbb540ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.858+0000 7fbb58faa640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb5413df30 0x7fbb540ae0d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c430 tx=0x7fbb3c00c8f0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.858+0000 7fbb58faa640 1 -- 172.21.15.172:0/3628259068 shutdown_connections 2024-04-14T08:33:32.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.858+0000 7fbb58faa640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb3406c8f0 0x7fbb3406edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.858+0000 7fbb58faa640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb5415c380 0x7fbb5413a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.858+0000 7fbb58faa640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb5413df30 0x7fbb540ae0d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.858+0000 7fbb58faa640 1 --2- 172.21.15.172:0/3628259068 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb540b23e0 0x7fbb541511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.858+0000 7fbb58faa640 1 -- 172.21.15.172:0/3628259068 >> 172.21.15.172:0/3628259068 conn(0x7fbb5406b520 msgr2=0x7fbb5406d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:32.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.858+0000 7fbb58faa640 1 -- 172.21.15.172:0/3628259068 shutdown_connections 2024-04-14T08:33:32.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.858+0000 7fbb58faa640 1 -- 172.21.15.172:0/3628259068 wait complete. 2024-04-14T08:33:32.869 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:32.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.973+0000 7fb0bd02d640 1 Processor -- start 2024-04-14T08:33:32.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.973+0000 7fb0bd02d640 1 -- start start 2024-04-14T08:33:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.974+0000 7fb0bd02d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb0b815e430 0x7fb0b8162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.974+0000 7fb0bd02d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb0b8162540 0x7fb0b81629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.974+0000 7fb0bd02d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0b80cefb0 con 0x7fb0b80b3a30 2024-04-14T08:33:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.974+0000 7fb0bd02d640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb0b80cf290 con 0x7fb0b8162540 2024-04-14T08:33:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.974+0000 7fb0bd02d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb0b80cf570 con 0x7fb0b815e430 2024-04-14T08:33:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.974+0000 7fb0b659c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb0b8162540 0x7fb0b81629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.974+0000 7fb0b759e640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb0b80b3a30 0x7fb0b80b3e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46498/0 (socket says 172.21.15.172:46498) 2024-04-14T08:33:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.974+0000 7fb0b759e640 1 -- 172.21.15.172:0/3042423844 learned_addr learned my addr 172.21.15.172:0/3042423844 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.974+0000 7fb0b6d9d640 1 --2- 172.21.15.172:0/3042423844 >> v2:172.21.15.177:3300/0 conn(0x7fb0b815e430 0x7fb0b8162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.975+0000 7fb0b659c640 1 -- 172.21.15.172:0/3042423844 >> v2:172.21.15.177:3300/0 conn(0x7fb0b815e430 msgr2=0x7fb0b8162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.975+0000 7fb0b659c640 1 --2- 172.21.15.172:0/3042423844 >> v2:172.21.15.177:3300/0 conn(0x7fb0b815e430 0x7fb0b8162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.975+0000 7fb0b659c640 1 -- 172.21.15.172:0/3042423844 >> v1:172.21.15.172:6789/0 conn(0x7fb0b80b3a30 legacy=0x7fb0b80b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.975+0000 7fb0b659c640 1 -- 172.21.15.172:0/3042423844 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b80cf850 con 0x7fb0b8162540 2024-04-14T08:33:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.975+0000 7fb0b5d9b640 1 -- 172.21.15.172:0/3042423844 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1782367932 0 0) 0x7fb0b80cefb0 con 0x7fb0b80b3a30 2024-04-14T08:33:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.975+0000 7fb0b659c640 1 --2- 172.21.15.172:0/3042423844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b8162540 0x7fb0b81629d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80b6db0 tx=0x7fb0ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59044c82ee6688c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.975+0000 7fb0b5d9b640 1 -- 172.21.15.172:0/3042423844 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac035020 con 0x7fb0b8162540 2024-04-14T08:33:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.975+0000 7fb0b5d9b640 1 -- 172.21.15.172:0/3042423844 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac031830 con 0x7fb0b8162540 2024-04-14T08:33:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.975+0000 7fb0b5d9b640 1 -- 172.21.15.172:0/3042423844 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac031c00 con 0x7fb0b8162540 2024-04-14T08:33:32.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.976+0000 7fb0bd02d640 1 -- 172.21.15.172:0/3042423844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b8162540 msgr2=0x7fb0b81629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.976+0000 7fb0bd02d640 1 --2- 172.21.15.172:0/3042423844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b8162540 0x7fb0b81629d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80b6db0 tx=0x7fb0ac02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.976+0000 7fb0bd02d640 1 -- 172.21.15.172:0/3042423844 shutdown_connections 2024-04-14T08:33:32.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.976+0000 7fb0bd02d640 1 --2- 172.21.15.172:0/3042423844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b8162540 0x7fb0b81629d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.976+0000 7fb0bd02d640 1 --2- 172.21.15.172:0/3042423844 >> v2:172.21.15.177:3300/0 conn(0x7fb0b815e430 0x7fb0b8162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.976+0000 7fb0bd02d640 1 -- 172.21.15.172:0/3042423844 >> 172.21.15.172:0/3042423844 conn(0x7fb0b809d560 msgr2=0x7fb0b80d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:32.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.976+0000 7fb0bd02d640 1 -- 172.21.15.172:0/3042423844 shutdown_connections 2024-04-14T08:33:32.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.976+0000 7fb0bd02d640 1 -- 172.21.15.172:0/3042423844 wait complete. 2024-04-14T08:33:32.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.977+0000 7fb0bd02d640 1 Processor -- start 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.977+0000 7fb0bd02d640 1 -- start start 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.978+0000 7fb0bd02d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0b80b3a30 0x7fb0b8157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.978+0000 7fb0bd02d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b815e430 0x7fb0b80b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.978+0000 7fb0bd02d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b8162540 0x7fb0b80b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.978+0000 7fb0bd02d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b80d4e20 con 0x7fb0b8162540 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.978+0000 7fb0bd02d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b815e810 con 0x7fb0b815e430 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.978+0000 7fb0bd02d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b80d0050 con 0x7fb0b80b3a30 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.978+0000 7fb0b759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b8162540 0x7fb0b80b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.978+0000 7fb0b759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b8162540 0x7fb0b80b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37186/0 (socket says 172.21.15.172:37186) 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.978+0000 7fb0b759e640 1 -- 172.21.15.172:0/1310468094 learned_addr learned my addr 172.21.15.172:0/1310468094 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.978+0000 7fb0b6d9d640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0b80b3a30 0x7fb0b8157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.978+0000 7fb0b659c640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b815e430 0x7fb0b80b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.979+0000 7fb0b759e640 1 -- 172.21.15.172:0/1310468094 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0b80b3a30 msgr2=0x7fb0b8157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.979+0000 7fb0b759e640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0b80b3a30 0x7fb0b8157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.979+0000 7fb0b759e640 1 -- 172.21.15.172:0/1310468094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b815e430 msgr2=0x7fb0b80b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.979+0000 7fb0b759e640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b815e430 0x7fb0b80b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.979+0000 7fb0b759e640 1 -- 172.21.15.172:0/1310468094 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b815fa80 con 0x7fb0b8162540 2024-04-14T08:33:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.979+0000 7fb0b759e640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b8162540 0x7fb0b80b1390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000c450 tx=0x7fb0b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.979+0000 7fb09f7fe640 1 -- 172.21.15.172:0/1310468094 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0018070 con 0x7fb0b8162540 2024-04-14T08:33:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.979+0000 7fb09f7fe640 1 -- 172.21.15.172:0/1310468094 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b0007500 con 0x7fb0b8162540 2024-04-14T08:33:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.979+0000 7fb0bd02d640 1 -- 172.21.15.172:0/1310468094 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b815fd60 con 0x7fb0b8162540 2024-04-14T08:33:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.979+0000 7fb09f7fe640 1 -- 172.21.15.172:0/1310468094 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0004a20 con 0x7fb0b8162540 2024-04-14T08:33:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.979+0000 7fb0bd02d640 1 -- 172.21.15.172:0/1310468094 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b8160320 con 0x7fb0b8162540 2024-04-14T08:33:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.980+0000 7fb0bd02d640 1 -- 172.21.15.172:0/1310468094 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b8068590 con 0x7fb0b8162540 2024-04-14T08:33:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.980+0000 7fb09f7fe640 1 -- 172.21.15.172:0/1310468094 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0b001b070 con 0x7fb0b8162540 2024-04-14T08:33:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.981+0000 7fb09f7fe640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0a406c8f0 0x7fb0a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.981+0000 7fb09f7fe640 1 -- 172.21.15.172:0/1310468094 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb0b008b0b0 con 0x7fb0b8162540 2024-04-14T08:33:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.981+0000 7fb0b6d9d640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0a406c8f0 0x7fb0a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:32.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.981+0000 7fb0b6d9d640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0a406c8f0 0x7fb0a406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80113e0 tx=0x7fb0a800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:32.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:32.985+0000 7fb09f7fe640 1 -- 172.21.15.172:0/1310468094 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb0b0058b50 con 0x7fb0b8162540 2024-04-14T08:33:33.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.160+0000 7fb0bd02d640 1 -- 172.21.15.172:0/1310468094 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb0b80b68f0 con 0x7fb0b8162540 2024-04-14T08:33:33.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.161+0000 7fb09f7fe640 1 -- 172.21.15.172:0/1310468094 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fb0b005ce50 con 0x7fb0b8162540 2024-04-14T08:33:33.161 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.163+0000 7fb0bd02d640 1 -- 172.21.15.172:0/1310468094 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0a406c8f0 msgr2=0x7fb0a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.163+0000 7fb0bd02d640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0a406c8f0 0x7fb0a406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80113e0 tx=0x7fb0a800b040 comp rx=0 tx=0).stop 2024-04-14T08:33:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.163+0000 7fb0bd02d640 1 -- 172.21.15.172:0/1310468094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b8162540 msgr2=0x7fb0b80b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.163+0000 7fb0bd02d640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b8162540 0x7fb0b80b1390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000c450 tx=0x7fb0b000c910 comp rx=0 tx=0).stop 2024-04-14T08:33:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.163+0000 7fb0bd02d640 1 -- 172.21.15.172:0/1310468094 shutdown_connections 2024-04-14T08:33:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.163+0000 7fb0bd02d640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0a406c8f0 0x7fb0a406edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.163+0000 7fb0bd02d640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b8162540 0x7fb0b80b1390 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.163+0000 7fb0bd02d640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b815e430 0x7fb0b80b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.163+0000 7fb0bd02d640 1 --2- 172.21.15.172:0/1310468094 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0b80b3a30 0x7fb0b8157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.163+0000 7fb0bd02d640 1 -- 172.21.15.172:0/1310468094 >> 172.21.15.172:0/1310468094 conn(0x7fb0b809d560 msgr2=0x7fb0b806c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.163+0000 7fb0bd02d640 1 -- 172.21.15.172:0/1310468094 shutdown_connections 2024-04-14T08:33:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.163+0000 7fb0bd02d640 1 -- 172.21.15.172:0/1310468094 wait complete. 2024-04-14T08:33:33.174 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:33.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.278+0000 7f43f8088640 1 Processor -- start 2024-04-14T08:33:33.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.278+0000 7f43f8088640 1 -- start start 2024-04-14T08:33:33.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f8088640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43f013df30 0x7f43f00b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f8088640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f43f0140e80 0x7f43f00b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f8088640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43f015b620 con 0x7f43f015c380 2024-04-14T08:33:33.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f8088640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f43f00cf9b0 con 0x7f43f0140e80 2024-04-14T08:33:33.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f8088640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f43f01414a0 con 0x7f43f013df30 2024-04-14T08:33:33.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f5623640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f43f015c380 0x7f43f0140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46500/0 (socket says 172.21.15.172:46500) 2024-04-14T08:33:33.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f5623640 1 -- 172.21.15.172:0/3075506684 learned_addr learned my addr 172.21.15.172:0/3075506684 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:33.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f6625640 1 --2- 172.21.15.172:0/3075506684 >> v2:172.21.15.177:3300/0 conn(0x7f43f0140e80 0x7f43f00b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f5e24640 1 --2- 172.21.15.172:0/3075506684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43f013df30 0x7f43f00b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f4e22640 1 -- 172.21.15.172:0/3075506684 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3478575790 0 0) 0x7f43e0001110 con 0x7f43f015c380 2024-04-14T08:33:33.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f5e24640 1 -- 172.21.15.172:0/3075506684 >> v2:172.21.15.177:3300/0 conn(0x7f43f0140e80 msgr2=0x7f43f00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:33.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f5e24640 1 --2- 172.21.15.172:0/3075506684 >> v2:172.21.15.177:3300/0 conn(0x7f43f0140e80 0x7f43f00b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f5e24640 1 -- 172.21.15.172:0/3075506684 >> v1:172.21.15.172:6789/0 conn(0x7f43f015c380 legacy=0x7f43f0140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f5e24640 1 -- 172.21.15.172:0/3075506684 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f015b8a0 con 0x7f43f013df30 2024-04-14T08:33:33.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.279+0000 7f43f6625640 1 --2- 172.21.15.172:0/3075506684 >> v2:172.21.15.177:3300/0 conn(0x7f43f0140e80 0x7f43f00b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:33.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.280+0000 7f43f5e24640 1 --2- 172.21.15.172:0/3075506684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43f013df30 0x7f43f00b1cd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0097d0 tx=0x7f43ec02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8847772bf6d7cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:33.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.280+0000 7f43f4e22640 1 -- 172.21.15.172:0/3075506684 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec035020 con 0x7f43f013df30 2024-04-14T08:33:33.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.280+0000 7f43f4e22640 1 -- 172.21.15.172:0/3075506684 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43ec031800 con 0x7f43f013df30 2024-04-14T08:33:33.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.280+0000 7f43f4e22640 1 -- 172.21.15.172:0/3075506684 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec031bd0 con 0x7f43f013df30 2024-04-14T08:33:33.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.281+0000 7f43f8088640 1 -- 172.21.15.172:0/3075506684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43f013df30 msgr2=0x7f43f00b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.281+0000 7f43f8088640 1 --2- 172.21.15.172:0/3075506684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43f013df30 0x7f43f00b1cd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0097d0 tx=0x7f43ec02b8a0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.281+0000 7f43f8088640 1 -- 172.21.15.172:0/3075506684 shutdown_connections 2024-04-14T08:33:33.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.281+0000 7f43f8088640 1 --2- 172.21.15.172:0/3075506684 >> v2:172.21.15.177:3300/0 conn(0x7f43f0140e80 0x7f43f00b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.281+0000 7f43f8088640 1 --2- 172.21.15.172:0/3075506684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43f013df30 0x7f43f00b1cd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.281+0000 7f43f8088640 1 -- 172.21.15.172:0/3075506684 >> 172.21.15.172:0/3075506684 conn(0x7f43f006b520 msgr2=0x7f43f006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:33.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.281+0000 7f43f8088640 1 -- 172.21.15.172:0/3075506684 shutdown_connections 2024-04-14T08:33:33.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.281+0000 7f43f8088640 1 -- 172.21.15.172:0/3075506684 wait complete. 2024-04-14T08:33:33.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.282+0000 7f43f8088640 1 Processor -- start 2024-04-14T08:33:33.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.282+0000 7f43f8088640 1 -- start start 2024-04-14T08:33:33.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f8088640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43f013df30 0x7f43f013e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f8088640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43f0140e80 0x7f43f01524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f8088640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43f015c380 0x7f43f0152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f8088640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43f013e310 con 0x7f43f015c380 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f8088640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f43f015be60 con 0x7f43f0140e80 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f8088640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f43f00ce9a0 con 0x7f43f013df30 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f6625640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43f015c380 0x7f43f0152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f5e24640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43f013df30 0x7f43f013e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f6625640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43f015c380 0x7f43f0152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37198/0 (socket says 172.21.15.172:37198) 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f6625640 1 -- 172.21.15.172:0/2836278348 learned_addr learned my addr 172.21.15.172:0/2836278348 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f5623640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43f0140e80 0x7f43f01524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f6625640 1 -- 172.21.15.172:0/2836278348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43f013df30 msgr2=0x7f43f013e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f6625640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43f013df30 0x7f43f013e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f6625640 1 -- 172.21.15.172:0/2836278348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43f0140e80 msgr2=0x7f43f01524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f6625640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43f0140e80 0x7f43f01524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f6625640 1 -- 172.21.15.172:0/2836278348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f01fc640 con 0x7f43f015c380 2024-04-14T08:33:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f5e24640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43f013df30 0x7f43f013e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f6625640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43f015c380 0x7f43f0152a10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f43e800efc0 tx=0x7f43e8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43e67fc640 1 -- 172.21.15.172:0/2836278348 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8018030 con 0x7f43f015c380 2024-04-14T08:33:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43e67fc640 1 -- 172.21.15.172:0/2836278348 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e800cde0 con 0x7f43f015c380 2024-04-14T08:33:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43e67fc640 1 -- 172.21.15.172:0/2836278348 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8003bb0 con 0x7f43f015c380 2024-04-14T08:33:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f8088640 1 -- 172.21.15.172:0/2836278348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f0154ba0 con 0x7f43f015c380 2024-04-14T08:33:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.283+0000 7f43f8088640 1 -- 172.21.15.172:0/2836278348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f0154850 con 0x7f43f015c380 2024-04-14T08:33:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.285+0000 7f43f8088640 1 -- 172.21.15.172:0/2836278348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f0068590 con 0x7f43f015c380 2024-04-14T08:33:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.285+0000 7f43e67fc640 1 -- 172.21.15.172:0/2836278348 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43e801b070 con 0x7f43f015c380 2024-04-14T08:33:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.285+0000 7f43e67fc640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f43d406c8f0 0x7f43d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.286+0000 7f43e67fc640 1 -- 172.21.15.172:0/2836278348 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f43e808c760 con 0x7f43f015c380 2024-04-14T08:33:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.286+0000 7f43f5e24640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f43d406c8f0 0x7f43d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.286+0000 7f43f5e24640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f43d406c8f0 0x7f43d406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f43ec036000 tx=0x7f43ec03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.290+0000 7f43e67fc640 1 -- 172.21.15.172:0/2836278348 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f43e8010040 con 0x7f43f015c380 2024-04-14T08:33:33.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.470+0000 7f43f8088640 1 -- 172.21.15.172:0/2836278348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f43f006c530 con 0x7f43f015c380 2024-04-14T08:33:33.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.471+0000 7f43e67fc640 1 -- 172.21.15.172:0/2836278348 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f43e805a290 con 0x7f43f015c380 2024-04-14T08:33:33.471 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:33.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.473+0000 7f43f8088640 1 -- 172.21.15.172:0/2836278348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f43d406c8f0 msgr2=0x7f43d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.473+0000 7f43f8088640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f43d406c8f0 0x7f43d406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f43ec036000 tx=0x7f43ec03d040 comp rx=0 tx=0).stop 2024-04-14T08:33:33.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.473+0000 7f43f8088640 1 -- 172.21.15.172:0/2836278348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43f015c380 msgr2=0x7f43f0152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.473+0000 7f43f8088640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43f015c380 0x7f43f0152a10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f43e800efc0 tx=0x7f43e8009f90 comp rx=0 tx=0).stop 2024-04-14T08:33:33.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.473+0000 7f43f8088640 1 -- 172.21.15.172:0/2836278348 shutdown_connections 2024-04-14T08:33:33.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.473+0000 7f43f8088640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f43d406c8f0 0x7f43d406edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.473+0000 7f43f8088640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43f015c380 0x7f43f0152a10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.473+0000 7f43f8088640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43f0140e80 0x7f43f01524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.473+0000 7f43f8088640 1 --2- 172.21.15.172:0/2836278348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43f013df30 0x7f43f013e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.473+0000 7f43f8088640 1 -- 172.21.15.172:0/2836278348 >> 172.21.15.172:0/2836278348 conn(0x7f43f006b520 msgr2=0x7f43f015eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:33.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.473+0000 7f43f8088640 1 -- 172.21.15.172:0/2836278348 shutdown_connections 2024-04-14T08:33:33.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.473+0000 7f43f8088640 1 -- 172.21.15.172:0/2836278348 wait complete. 2024-04-14T08:33:33.484 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:33.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.588+0000 7f00fa01c640 1 Processor -- start 2024-04-14T08:33:33.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.588+0000 7f00fa01c640 1 -- start start 2024-04-14T08:33:33.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.588+0000 7f00fa01c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00f40d49c0 0x7f00f40d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.588+0000 7f00fa01c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f00f4160f30 0x7f00f41613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.588+0000 7f00fa01c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f40cf0f0 con 0x7f00f40d0cc0 2024-04-14T08:33:33.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.588+0000 7f00fa01c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f00f40ceec0 con 0x7f00f40d49c0 2024-04-14T08:33:33.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.588+0000 7f00fa01c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f00f40cfd00 con 0x7f00f4160f30 2024-04-14T08:33:33.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.589+0000 7f00f2ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00f40d49c0 0x7f00f40d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.589+0000 7f00f2ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00f40d49c0 0x7f00f40d4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37210/0 (socket says 172.21.15.172:37210) 2024-04-14T08:33:33.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.589+0000 7f00f2ffd640 1 -- 172.21.15.172:0/108833708 learned_addr learned my addr 172.21.15.172:0/108833708 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:33.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.589+0000 7f00f3fff640 1 --2- 172.21.15.172:0/108833708 >> v2:172.21.15.177:3300/0 conn(0x7f00f4160f30 0x7f00f41613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.589+0000 7f00f2ffd640 1 -- 172.21.15.172:0/108833708 >> v2:172.21.15.177:3300/0 conn(0x7f00f4160f30 msgr2=0x7f00f41613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:33.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.589+0000 7f00f2ffd640 1 --2- 172.21.15.172:0/108833708 >> v2:172.21.15.177:3300/0 conn(0x7f00f4160f30 0x7f00f41613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.589+0000 7f00f2ffd640 1 -- 172.21.15.172:0/108833708 >> v1:172.21.15.177:6789/0 conn(0x7f00f40d0cc0 legacy=0x7f00f40d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.589+0000 7f00f2ffd640 1 -- 172.21.15.172:0/108833708 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f40cf3d0 con 0x7f00f40d49c0 2024-04-14T08:33:33.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.590+0000 7f00f2ffd640 1 --2- 172.21.15.172:0/108833708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00f40d49c0 0x7f00f40d4e50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00f408d6b0 tx=0x7f00e802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a2d0f079ff4938 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:33.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.590+0000 7f00f27fc640 1 -- 172.21.15.172:0/108833708 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8034070 con 0x7f00f40d49c0 2024-04-14T08:33:33.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.590+0000 7f00f27fc640 1 -- 172.21.15.172:0/108833708 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e8030860 con 0x7f00f40d49c0 2024-04-14T08:33:33.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.590+0000 7f00f27fc640 1 -- 172.21.15.172:0/108833708 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8030c30 con 0x7f00f40d49c0 2024-04-14T08:33:33.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.590+0000 7f00fa01c640 1 -- 172.21.15.172:0/108833708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00f40d49c0 msgr2=0x7f00f40d4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.590+0000 7f00fa01c640 1 --2- 172.21.15.172:0/108833708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00f40d49c0 0x7f00f40d4e50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00f408d6b0 tx=0x7f00e802b9a0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.590+0000 7f00fa01c640 1 -- 172.21.15.172:0/108833708 shutdown_connections 2024-04-14T08:33:33.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.590+0000 7f00fa01c640 1 --2- 172.21.15.172:0/108833708 >> v2:172.21.15.177:3300/0 conn(0x7f00f4160f30 0x7f00f41613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.590+0000 7f00fa01c640 1 --2- 172.21.15.172:0/108833708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00f40d49c0 0x7f00f40d4e50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.590+0000 7f00fa01c640 1 -- 172.21.15.172:0/108833708 >> 172.21.15.172:0/108833708 conn(0x7f00f409d580 msgr2=0x7f00f40d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:33.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.590+0000 7f00fa01c640 1 -- 172.21.15.172:0/108833708 shutdown_connections 2024-04-14T08:33:33.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.590+0000 7f00fa01c640 1 -- 172.21.15.172:0/108833708 wait complete. 2024-04-14T08:33:33.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.591+0000 7f00fa01c640 1 Processor -- start 2024-04-14T08:33:33.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.592+0000 7f00fa01c640 1 -- start start 2024-04-14T08:33:33.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.592+0000 7f00fa01c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f40d0cc0 0x7f00f40c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.592+0000 7f00fa01c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00f40d49c0 0x7f00f40a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.592+0000 7f00fa01c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00f4160f30 0x7f00f41f1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.592+0000 7f00fa01c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00f40d32b0 con 0x7f00f4160f30 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.592+0000 7f00fa01c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f00f40cf990 con 0x7f00f40d0cc0 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.592+0000 7f00fa01c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f00f40d0190 con 0x7f00f40d49c0 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.592+0000 7f00f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00f4160f30 0x7f00f41f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.592+0000 7f00f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00f4160f30 0x7f00f41f1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37222/0 (socket says 172.21.15.172:37222) 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.592+0000 7f00f3fff640 1 -- 172.21.15.172:0/3195046626 learned_addr learned my addr 172.21.15.172:0/3195046626 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.592+0000 7f00f2ffd640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00f40d49c0 0x7f00f40a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.592+0000 7f00f37fe640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f40d0cc0 0x7f00f40c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.593+0000 7f00f3fff640 1 -- 172.21.15.172:0/3195046626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00f40d49c0 msgr2=0x7f00f40a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.593+0000 7f00f3fff640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00f40d49c0 0x7f00f40a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.593+0000 7f00f3fff640 1 -- 172.21.15.172:0/3195046626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f40d0cc0 msgr2=0x7f00f40c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.593+0000 7f00f3fff640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f40d0cc0 0x7f00f40c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.593+0000 7f00f3fff640 1 -- 172.21.15.172:0/3195046626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f41f1bd0 con 0x7f00f4160f30 2024-04-14T08:33:33.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.593+0000 7f00f3fff640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00f4160f30 0x7f00f41f1690 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00efc0 tx=0x7f00ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:33.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.593+0000 7f00dbfff640 1 -- 172.21.15.172:0/3195046626 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec018020 con 0x7f00f4160f30 2024-04-14T08:33:33.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.593+0000 7f00dbfff640 1 -- 172.21.15.172:0/3195046626 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ec014480 con 0x7f00f4160f30 2024-04-14T08:33:33.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.593+0000 7f00fa01c640 1 -- 172.21.15.172:0/3195046626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f40ca220 con 0x7f00f4160f30 2024-04-14T08:33:33.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.593+0000 7f00dbfff640 1 -- 172.21.15.172:0/3195046626 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec014870 con 0x7f00f4160f30 2024-04-14T08:33:33.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.593+0000 7f00fa01c640 1 -- 172.21.15.172:0/3195046626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f40ca7e0 con 0x7f00f4160f30 2024-04-14T08:33:33.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.594+0000 7f00fa01c640 1 -- 172.21.15.172:0/3195046626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f4075490 con 0x7f00f4160f30 2024-04-14T08:33:33.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.594+0000 7f00dbfff640 1 -- 172.21.15.172:0/3195046626 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00ec005d10 con 0x7f00f4160f30 2024-04-14T08:33:33.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.595+0000 7f00dbfff640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f00e006c920 0x7f00e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.595+0000 7f00dbfff640 1 -- 172.21.15.172:0/3195046626 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f00ec08c840 con 0x7f00f4160f30 2024-04-14T08:33:33.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.595+0000 7f00f37fe640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f00e006c920 0x7f00e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.595+0000 7f00f37fe640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f00e006c920 0x7f00e006ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f00e4001780 tx=0x7f00e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:33.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.599+0000 7f00dbfff640 1 -- 172.21.15.172:0/3195046626 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f00ec022070 con 0x7f00f4160f30 2024-04-14T08:33:33.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.775+0000 7f00fa01c640 1 -- 172.21.15.172:0/3195046626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f00f40d0aa0 con 0x7f00f4160f30 2024-04-14T08:33:33.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.775+0000 7f00dbfff640 1 -- 172.21.15.172:0/3195046626 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f00ec010040 con 0x7f00f4160f30 2024-04-14T08:33:33.776 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:33.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.777+0000 7f00fa01c640 1 -- 172.21.15.172:0/3195046626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f00e006c920 msgr2=0x7f00e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.777+0000 7f00fa01c640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f00e006c920 0x7f00e006ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f00e4001780 tx=0x7f00e4009040 comp rx=0 tx=0).stop 2024-04-14T08:33:33.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.777+0000 7f00fa01c640 1 -- 172.21.15.172:0/3195046626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00f4160f30 msgr2=0x7f00f41f1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.777+0000 7f00fa01c640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00f4160f30 0x7f00f41f1690 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00efc0 tx=0x7f00ec00c490 comp rx=0 tx=0).stop 2024-04-14T08:33:33.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.777+0000 7f00fa01c640 1 -- 172.21.15.172:0/3195046626 shutdown_connections 2024-04-14T08:33:33.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.777+0000 7f00fa01c640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f00e006c920 0x7f00e006ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.777+0000 7f00fa01c640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00f4160f30 0x7f00f41f1690 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.777+0000 7f00fa01c640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00f40d49c0 0x7f00f40a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.777+0000 7f00fa01c640 1 --2- 172.21.15.172:0/3195046626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f40d0cc0 0x7f00f40c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.778+0000 7f00fa01c640 1 -- 172.21.15.172:0/3195046626 >> 172.21.15.172:0/3195046626 conn(0x7f00f409d580 msgr2=0x7f00f4160770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:33.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.778+0000 7f00fa01c640 1 -- 172.21.15.172:0/3195046626 shutdown_connections 2024-04-14T08:33:33.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.778+0000 7f00fa01c640 1 -- 172.21.15.172:0/3195046626 wait complete. 2024-04-14T08:33:33.814 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-14T08:33:33.814 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:33.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.915+0000 7fb86655e640 1 Processor -- start 2024-04-14T08:33:33.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.915+0000 7fb86655e640 1 -- start start 2024-04-14T08:33:33.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.915+0000 7fb86655e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb86015c710 0x7fb8600d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.915+0000 7fb86655e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb8600cef50 0x7fb8600afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.915+0000 7fb86655e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8600aef60 con 0x7fb8600cde10 2024-04-14T08:33:33.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.915+0000 7fb86655e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb8600aed10 con 0x7fb86015c710 2024-04-14T08:33:33.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.915+0000 7fb86655e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb8600af890 con 0x7fb8600cef50 2024-04-14T08:33:33.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.916+0000 7fb85ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb86015c710 0x7fb8600d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.916+0000 7fb85ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb86015c710 0x7fb8600d0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37238/0 (socket says 172.21.15.172:37238) 2024-04-14T08:33:33.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.916+0000 7fb85ffff640 1 -- 172.21.15.172:0/2282547725 learned_addr learned my addr 172.21.15.172:0/2282547725 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:33.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.916+0000 7fb864afb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb8600cef50 0x7fb8600afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.916+0000 7fb85f7fe640 1 --1- 172.21.15.172:0/2282547725 >> v1:172.21.15.177:6789/0 conn(0x7fb8600cde10 0x7fb8600ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:51232/0 (socket says 172.21.15.172:51232) 2024-04-14T08:33:33.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.916+0000 7fb85ffff640 1 -- 172.21.15.172:0/2282547725 >> v2:172.21.15.177:3300/0 conn(0x7fb8600cef50 msgr2=0x7fb8600afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:33.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.916+0000 7fb85ffff640 1 --2- 172.21.15.172:0/2282547725 >> v2:172.21.15.177:3300/0 conn(0x7fb8600cef50 0x7fb8600afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.916+0000 7fb85ffff640 1 -- 172.21.15.172:0/2282547725 >> v1:172.21.15.177:6789/0 conn(0x7fb8600cde10 legacy=0x7fb8600ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.916+0000 7fb85ffff640 1 -- 172.21.15.172:0/2282547725 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8600af240 con 0x7fb86015c710 2024-04-14T08:33:33.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.917+0000 7fb85ffff640 1 --2- 172.21.15.172:0/2282547725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86015c710 0x7fb8600d0950 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb860069190 tx=0x7fb85402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60543c7af3765a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:33.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.917+0000 7fb85effd640 1 -- 172.21.15.172:0/2282547725 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854034070 con 0x7fb86015c710 2024-04-14T08:33:33.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.917+0000 7fb85effd640 1 -- 172.21.15.172:0/2282547725 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb854030860 con 0x7fb86015c710 2024-04-14T08:33:33.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.917+0000 7fb85effd640 1 -- 172.21.15.172:0/2282547725 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854030c30 con 0x7fb86015c710 2024-04-14T08:33:33.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.917+0000 7fb86655e640 1 -- 172.21.15.172:0/2282547725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86015c710 msgr2=0x7fb8600d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.917+0000 7fb86655e640 1 --2- 172.21.15.172:0/2282547725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86015c710 0x7fb8600d0950 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb860069190 tx=0x7fb85402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.917+0000 7fb86655e640 1 -- 172.21.15.172:0/2282547725 shutdown_connections 2024-04-14T08:33:33.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.917+0000 7fb86655e640 1 --2- 172.21.15.172:0/2282547725 >> v2:172.21.15.177:3300/0 conn(0x7fb8600cef50 0x7fb8600afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.917+0000 7fb86655e640 1 --2- 172.21.15.172:0/2282547725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86015c710 0x7fb8600d0950 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.917+0000 7fb86655e640 1 -- 172.21.15.172:0/2282547725 >> 172.21.15.172:0/2282547725 conn(0x7fb86008c000 msgr2=0x7fb8600d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:33.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.917+0000 7fb86655e640 1 -- 172.21.15.172:0/2282547725 shutdown_connections 2024-04-14T08:33:33.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.917+0000 7fb86655e640 1 -- 172.21.15.172:0/2282547725 wait complete. 2024-04-14T08:33:33.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.918+0000 7fb86655e640 1 Processor -- start 2024-04-14T08:33:33.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.919+0000 7fb86655e640 1 -- start start 2024-04-14T08:33:33.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.919+0000 7fb86655e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8600cde10 0x7fb8600caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.919+0000 7fb86655e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8600cef50 0x7fb86015cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.919+0000 7fb86655e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb86015c710 0x7fb86015d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.919+0000 7fb86655e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8600d0e90 con 0x7fb8600cde10 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.919+0000 7fb86655e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb86015caf0 con 0x7fb8600cef50 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.919+0000 7fb86655e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb8600d1010 con 0x7fb86015c710 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.919+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8600cde10 0x7fb8600caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.919+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8600cde10 0x7fb8600caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37244/0 (socket says 172.21.15.172:37244) 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.919+0000 7fb85ffff640 1 -- 172.21.15.172:0/1948517439 learned_addr learned my addr 172.21.15.172:0/1948517439 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.919+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8600cef50 0x7fb86015cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.919+0000 7fb864afb640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb86015c710 0x7fb86015d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.920+0000 7fb85ffff640 1 -- 172.21.15.172:0/1948517439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb86015c710 msgr2=0x7fb86015d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.920+0000 7fb85ffff640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb86015c710 0x7fb86015d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.920+0000 7fb85ffff640 1 -- 172.21.15.172:0/1948517439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8600cef50 msgr2=0x7fb86015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.920+0000 7fb85ffff640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8600cef50 0x7fb86015cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:33.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.920+0000 7fb85ffff640 1 -- 172.21.15.172:0/1948517439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86015d7e0 con 0x7fb8600cde10 2024-04-14T08:33:33.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.920+0000 7fb85ffff640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8600cde10 0x7fb8600caf40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb85402b9e0 tx=0x7fb8540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:33.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.920+0000 7fb85cff9640 1 -- 172.21.15.172:0/1948517439 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854034040 con 0x7fb8600cde10 2024-04-14T08:33:33.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.920+0000 7fb85cff9640 1 -- 172.21.15.172:0/1948517439 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb854039d90 con 0x7fb8600cde10 2024-04-14T08:33:33.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.920+0000 7fb85cff9640 1 -- 172.21.15.172:0/1948517439 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854043630 con 0x7fb8600cde10 2024-04-14T08:33:33.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.920+0000 7fb86655e640 1 -- 172.21.15.172:0/1948517439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8600ca460 con 0x7fb8600cde10 2024-04-14T08:33:33.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.920+0000 7fb86655e640 1 -- 172.21.15.172:0/1948517439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb860153d80 con 0x7fb8600cde10 2024-04-14T08:33:33.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.921+0000 7fb86655e640 1 -- 172.21.15.172:0/1948517439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb860075490 con 0x7fb8600cde10 2024-04-14T08:33:33.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.922+0000 7fb85cff9640 1 -- 172.21.15.172:0/1948517439 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb854037070 con 0x7fb8600cde10 2024-04-14T08:33:33.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.922+0000 7fb85cff9640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb84c06c8f0 0x7fb84c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:33.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.922+0000 7fb85cff9640 1 -- 172.21.15.172:0/1948517439 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb8540ae0c0 con 0x7fb8600cde10 2024-04-14T08:33:33.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.922+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb84c06c8f0 0x7fb84c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:33.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.923+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb84c06c8f0 0x7fb84c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb8600b8090 tx=0x7fb850007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:33.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:33.927+0000 7fb85cff9640 1 -- 172.21.15.172:0/1948517439 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb85403e070 con 0x7fb8600cde10 2024-04-14T08:33:34.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.103+0000 7fb86655e640 1 -- 172.21.15.172:0/1948517439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8600b0e50 con 0x7fb8600cde10 2024-04-14T08:33:34.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.104+0000 7fb85cff9640 1 -- 172.21.15.172:0/1948517439 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18962 (secure 0 0 0) 0x7fb85407bb40 con 0x7fb8600cde10 2024-04-14T08:33:34.105 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:34.106 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":26,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:33:33.326333+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:34.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.106+0000 7fb86655e640 1 -- 172.21.15.172:0/1948517439 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb84c06c8f0 msgr2=0x7fb84c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.106+0000 7fb86655e640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb84c06c8f0 0x7fb84c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb8600b8090 tx=0x7fb850007e80 comp rx=0 tx=0).stop 2024-04-14T08:33:34.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.106+0000 7fb86655e640 1 -- 172.21.15.172:0/1948517439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8600cde10 msgr2=0x7fb8600caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.106+0000 7fb86655e640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8600cde10 0x7fb8600caf40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb85402b9e0 tx=0x7fb8540046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.106+0000 7fb86655e640 1 -- 172.21.15.172:0/1948517439 shutdown_connections 2024-04-14T08:33:34.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.106+0000 7fb86655e640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb84c06c8f0 0x7fb84c06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.106+0000 7fb86655e640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb86015c710 0x7fb86015d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.106+0000 7fb86655e640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8600cef50 0x7fb86015cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.106+0000 7fb86655e640 1 --2- 172.21.15.172:0/1948517439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8600cde10 0x7fb8600caf40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.106+0000 7fb86655e640 1 -- 172.21.15.172:0/1948517439 >> 172.21.15.172:0/1948517439 conn(0x7fb86008c000 msgr2=0x7fb86015ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:34.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.107+0000 7fb86655e640 1 -- 172.21.15.172:0/1948517439 shutdown_connections 2024-04-14T08:33:34.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.107+0000 7fb86655e640 1 -- 172.21.15.172:0/1948517439 wait complete. 2024-04-14T08:33:34.118 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:34.119 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:34.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.224+0000 7f1e9edfa640 1 Processor -- start 2024-04-14T08:33:34.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.224+0000 7f1e9edfa640 1 -- start start 2024-04-14T08:33:34.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.224+0000 7f1e9edfa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1e980d1a20 0x7f1e9806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.225+0000 7f1e9edfa640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e980701b0 con 0x7f1e98071250 2024-04-14T08:33:34.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.225+0000 7f1e9edfa640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e98070490 con 0x7f1e9806d000 2024-04-14T08:33:34.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.225+0000 7f1e9edfa640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f1e98070dc0 con 0x7f1e980d1a20 2024-04-14T08:33:34.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.225+0000 7f1e9cb96640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1e980d1a20 0x7f1e9806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.225+0000 7f1e97fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1e98071250 0x7f1e980cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46510/0 (socket says 172.21.15.172:46510) 2024-04-14T08:33:34.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.225+0000 7f1e9cb96640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1e980d1a20 0x7f1e9806fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37248/0 (socket says 172.21.15.172:37248) 2024-04-14T08:33:34.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.225+0000 7f1e97fff640 1 -- 172.21.15.172:0/4278834743 learned_addr learned my addr 172.21.15.172:0/4278834743 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:34.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.225+0000 7f1e9d397640 1 --1- 172.21.15.172:0/4278834743 >> v1:172.21.15.177:6789/0 conn(0x7f1e9806d000 0x7f1e9815dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:51236/0 (socket says 172.21.15.172:51236) 2024-04-14T08:33:34.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.225+0000 7f1e977fe640 1 -- 172.21.15.172:0/4278834743 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4010634187 0 0) 0x7f1e98070490 con 0x7f1e9806d000 2024-04-14T08:33:34.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.225+0000 7f1e9cb96640 1 -- 172.21.15.172:0/4278834743 >> v1:172.21.15.177:6789/0 conn(0x7f1e9806d000 legacy=0x7f1e9815dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.225+0000 7f1e9cb96640 1 -- 172.21.15.172:0/4278834743 >> v1:172.21.15.172:6789/0 conn(0x7f1e98071250 legacy=0x7f1e980cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.225+0000 7f1e9cb96640 1 -- 172.21.15.172:0/4278834743 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e98070770 con 0x7f1e980d1a20 2024-04-14T08:33:34.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.226+0000 7f1e9cb96640 1 --2- 172.21.15.172:0/4278834743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e980d1a20 0x7f1e9806fb80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0097a0 tx=0x7f1e8c02b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347b9b066c0fdc83 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:34.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.226+0000 7f1e977fe640 1 -- 172.21.15.172:0/4278834743 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c037070 con 0x7f1e980d1a20 2024-04-14T08:33:34.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.226+0000 7f1e977fe640 1 -- 172.21.15.172:0/4278834743 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8c030710 con 0x7f1e980d1a20 2024-04-14T08:33:34.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.226+0000 7f1e977fe640 1 -- 172.21.15.172:0/4278834743 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c030ae0 con 0x7f1e980d1a20 2024-04-14T08:33:34.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.226+0000 7f1e9edfa640 1 -- 172.21.15.172:0/4278834743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e980d1a20 msgr2=0x7f1e9806fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.226+0000 7f1e9edfa640 1 --2- 172.21.15.172:0/4278834743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e980d1a20 0x7f1e9806fb80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0097a0 tx=0x7f1e8c02b710 comp rx=0 tx=0).stop 2024-04-14T08:33:34.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.226+0000 7f1e9edfa640 1 -- 172.21.15.172:0/4278834743 shutdown_connections 2024-04-14T08:33:34.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.226+0000 7f1e9edfa640 1 --2- 172.21.15.172:0/4278834743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e980d1a20 0x7f1e9806fb80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.226+0000 7f1e9edfa640 1 -- 172.21.15.172:0/4278834743 >> 172.21.15.172:0/4278834743 conn(0x7f1e9808c000 msgr2=0x7f1e980d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:34.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.226+0000 7f1e9edfa640 1 -- 172.21.15.172:0/4278834743 shutdown_connections 2024-04-14T08:33:34.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.226+0000 7f1e9edfa640 1 -- 172.21.15.172:0/4278834743 wait complete. 2024-04-14T08:33:34.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.228+0000 7f1e9edfa640 1 Processor -- start 2024-04-14T08:33:34.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.228+0000 7f1e9edfa640 1 -- start start 2024-04-14T08:33:34.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.228+0000 7f1e9edfa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e9806d000 0x7f1e980b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.228+0000 7f1e9edfa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e98071250 0x7f1e980c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.228+0000 7f1e9edfa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e980d1a20 0x7f1e981574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.228+0000 7f1e9edfa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1e980d1e00 con 0x7f1e9806d000 2024-04-14T08:33:34.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.228+0000 7f1e9edfa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9806d450 con 0x7f1e98071250 2024-04-14T08:33:34.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9edfa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1e980c9bb0 con 0x7f1e980d1a20 2024-04-14T08:33:34.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9cb96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e9806d000 0x7f1e980b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9cb96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e9806d000 0x7f1e980b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37258/0 (socket says 172.21.15.172:37258) 2024-04-14T08:33:34.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9cb96640 1 -- 172.21.15.172:0/1152239508 learned_addr learned my addr 172.21.15.172:0/1152239508 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:34.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e97fff640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e98071250 0x7f1e980c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9d397640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e980d1a20 0x7f1e981574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9cb96640 1 -- 172.21.15.172:0/1152239508 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e980d1a20 msgr2=0x7f1e981574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9cb96640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e980d1a20 0x7f1e981574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9cb96640 1 -- 172.21.15.172:0/1152239508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e98071250 msgr2=0x7f1e980c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9cb96640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e98071250 0x7f1e980c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9cb96640 1 -- 172.21.15.172:0/1152239508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e98157a10 con 0x7f1e9806d000 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9cb96640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e9806d000 0x7f1e980b7c00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0098c0 tx=0x7f1e8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e957fa640 1 -- 172.21.15.172:0/1152239508 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c042070 con 0x7f1e9806d000 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e957fa640 1 -- 172.21.15.172:0/1152239508 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8c032a40 con 0x7f1e9806d000 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9edfa640 1 -- 172.21.15.172:0/1152239508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e980c81c0 con 0x7f1e9806d000 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e957fa640 1 -- 172.21.15.172:0/1152239508 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c037040 con 0x7f1e9806d000 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.229+0000 7f1e9edfa640 1 -- 172.21.15.172:0/1152239508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e980c9560 con 0x7f1e9806d000 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.230+0000 7f1e9edfa640 1 -- 172.21.15.172:0/1152239508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e98075490 con 0x7f1e9806d000 2024-04-14T08:33:34.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.231+0000 7f1e957fa640 1 -- 172.21.15.172:0/1152239508 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e8c045070 con 0x7f1e9806d000 2024-04-14T08:33:34.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.231+0000 7f1e957fa640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1e7c06c8f0 0x7f1e7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.231+0000 7f1e957fa640 1 -- 172.21.15.172:0/1152239508 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1e8c0adcc0 con 0x7f1e9806d000 2024-04-14T08:33:34.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.231+0000 7f1e97fff640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1e7c06c8f0 0x7f1e7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.232+0000 7f1e97fff640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1e7c06c8f0 0x7f1e7c06edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1e981f7bf0 tx=0x7f1e84007e20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:34.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.236+0000 7f1e957fa640 1 -- 172.21.15.172:0/1152239508 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1e8c07b6a0 con 0x7f1e9806d000 2024-04-14T08:33:34.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.419+0000 7f1e9edfa640 1 -- 172.21.15.172:0/1152239508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e980d2820 con 0x7f1e9806d000 2024-04-14T08:33:34.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.420+0000 7f1e957fa640 1 -- 172.21.15.172:0/1152239508 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f1e8c07f9a0 con 0x7f1e9806d000 2024-04-14T08:33:34.420 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.421+0000 7f1e9edfa640 1 -- 172.21.15.172:0/1152239508 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1e7c06c8f0 msgr2=0x7f1e7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.421+0000 7f1e9edfa640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1e7c06c8f0 0x7f1e7c06edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1e981f7bf0 tx=0x7f1e84007e20 comp rx=0 tx=0).stop 2024-04-14T08:33:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.421+0000 7f1e9edfa640 1 -- 172.21.15.172:0/1152239508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e9806d000 msgr2=0x7f1e980b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.421+0000 7f1e9edfa640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e9806d000 0x7f1e980b7c00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0098c0 tx=0x7f1e8c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.422+0000 7f1e9edfa640 1 -- 172.21.15.172:0/1152239508 shutdown_connections 2024-04-14T08:33:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.422+0000 7f1e9edfa640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1e7c06c8f0 0x7f1e7c06edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.422+0000 7f1e9edfa640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e980d1a20 0x7f1e981574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.422+0000 7f1e9edfa640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e98071250 0x7f1e980c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.422+0000 7f1e9edfa640 1 --2- 172.21.15.172:0/1152239508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e9806d000 0x7f1e980b7c00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.422+0000 7f1e9edfa640 1 -- 172.21.15.172:0/1152239508 >> 172.21.15.172:0/1152239508 conn(0x7f1e9808c000 msgr2=0x7f1e980cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.422+0000 7f1e9edfa640 1 -- 172.21.15.172:0/1152239508 shutdown_connections 2024-04-14T08:33:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.422+0000 7f1e9edfa640 1 -- 172.21.15.172:0/1152239508 wait complete. 2024-04-14T08:33:34.433 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:34.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.537+0000 7f10c2788640 1 Processor -- start 2024-04-14T08:33:34.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.537+0000 7f10c2788640 1 -- start start 2024-04-14T08:33:34.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10c2788640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f10bc0ccc40 0x7f10bc0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10c2788640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f10bc06f520 0x7f10bc1628c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10c2788640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10bc0cfa70 con 0x7f10bc15e8a0 2024-04-14T08:33:34.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10c2788640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f10bc0cde00 con 0x7f10bc0ccc40 2024-04-14T08:33:34.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10c2788640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f10bc0cd560 con 0x7f10bc06f520 2024-04-14T08:33:34.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10bbfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f10bc0ccc40 0x7f10bc0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10bbfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f10bc0ccc40 0x7f10bc0cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37266/0 (socket says 172.21.15.172:37266) 2024-04-14T08:33:34.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10bbfff640 1 -- 172.21.15.172:0/1154448195 learned_addr learned my addr 172.21.15.172:0/1154448195 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:34.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10c0d25640 1 --2- 172.21.15.172:0/1154448195 >> v2:172.21.15.177:3300/0 conn(0x7f10bc06f520 0x7f10bc1628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10bbfff640 1 -- 172.21.15.172:0/1154448195 >> v2:172.21.15.177:3300/0 conn(0x7f10bc06f520 msgr2=0x7f10bc1628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:34.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10bbfff640 1 --2- 172.21.15.172:0/1154448195 >> v2:172.21.15.177:3300/0 conn(0x7f10bc06f520 0x7f10bc1628c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10bbfff640 1 -- 172.21.15.172:0/1154448195 >> v1:172.21.15.177:6789/0 conn(0x7f10bc15e8a0 legacy=0x7f10bc1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.538+0000 7f10bbfff640 1 -- 172.21.15.172:0/1154448195 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc0ce050 con 0x7f10bc0ccc40 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.539+0000 7f10bbfff640 1 --2- 172.21.15.172:0/1154448195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10bc0ccc40 0x7f10bc0cd020 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f10bc071610 tx=0x7f10ac02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5b0bcfb55d36d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.539+0000 7f10baffd640 1 -- 172.21.15.172:0/1154448195 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac02f850 con 0x7f10bc0ccc40 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.539+0000 7f10baffd640 1 -- 172.21.15.172:0/1154448195 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10ac02f9f0 con 0x7f10bc0ccc40 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.539+0000 7f10baffd640 1 -- 172.21.15.172:0/1154448195 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac02fdc0 con 0x7f10bc0ccc40 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.540+0000 7f10c2788640 1 -- 172.21.15.172:0/1154448195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10bc0ccc40 msgr2=0x7f10bc0cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.540+0000 7f10c2788640 1 --2- 172.21.15.172:0/1154448195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10bc0ccc40 0x7f10bc0cd020 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f10bc071610 tx=0x7f10ac02ba30 comp rx=0 tx=0).stop 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.540+0000 7f10c2788640 1 -- 172.21.15.172:0/1154448195 shutdown_connections 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.540+0000 7f10c2788640 1 --2- 172.21.15.172:0/1154448195 >> v2:172.21.15.177:3300/0 conn(0x7f10bc06f520 0x7f10bc1628c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.540+0000 7f10c2788640 1 --2- 172.21.15.172:0/1154448195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10bc0ccc40 0x7f10bc0cd020 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.540+0000 7f10c2788640 1 -- 172.21.15.172:0/1154448195 >> 172.21.15.172:0/1154448195 conn(0x7f10bc0b1d80 msgr2=0x7f10bc0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.540+0000 7f10c2788640 1 -- 172.21.15.172:0/1154448195 shutdown_connections 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.540+0000 7f10c2788640 1 -- 172.21.15.172:0/1154448195 wait complete. 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.541+0000 7f10c2788640 1 Processor -- start 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.541+0000 7f10c2788640 1 -- start start 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10c2788640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10bc06f520 0x7f10bc0991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10c2788640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10bc0ccc40 0x7f10bc0975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10c2788640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10bc15e8a0 0x7f10bc0c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10c2788640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc0ce330 con 0x7f10bc0ccc40 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10c2788640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc06cd50 con 0x7f10bc06f520 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10c2788640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f10bc0ce4b0 con 0x7f10bc15e8a0 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10bb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10bc0ccc40 0x7f10bc0975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10bb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10bc0ccc40 0x7f10bc0975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37278/0 (socket says 172.21.15.172:37278) 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10bb7fe640 1 -- 172.21.15.172:0/1684478287 learned_addr learned my addr 172.21.15.172:0/1684478287 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10bbfff640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10bc06f520 0x7f10bc0991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10c0d25640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10bc15e8a0 0x7f10bc0c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10bb7fe640 1 -- 172.21.15.172:0/1684478287 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10bc15e8a0 msgr2=0x7f10bc0c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10bb7fe640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10bc15e8a0 0x7f10bc0c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10bb7fe640 1 -- 172.21.15.172:0/1684478287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10bc06f520 msgr2=0x7f10bc0991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10bb7fe640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10bc06f520 0x7f10bc0991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10bb7fe640 1 -- 172.21.15.172:0/1684478287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc15f190 con 0x7f10bc0ccc40 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10bbfff640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10bc06f520 0x7f10bc0991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.542+0000 7f10bb7fe640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10bc0ccc40 0x7f10bc0975c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c440 tx=0x7f10b000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.543+0000 7f10b8ff9640 1 -- 172.21.15.172:0/1684478287 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b00052d0 con 0x7f10bc0ccc40 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.543+0000 7f10b8ff9640 1 -- 172.21.15.172:0/1684478287 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b0005470 con 0x7f10bc0ccc40 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.543+0000 7f10c2788640 1 -- 172.21.15.172:0/1684478287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10bc0c8eb0 con 0x7f10bc0ccc40 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.543+0000 7f10b8ff9640 1 -- 172.21.15.172:0/1684478287 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0007260 con 0x7f10bc0ccc40 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.543+0000 7f10c2788640 1 -- 172.21.15.172:0/1684478287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10bc0c9470 con 0x7f10bc0ccc40 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.544+0000 7f10c2788640 1 -- 172.21.15.172:0/1684478287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10bc0c9670 con 0x7f10bc0ccc40 2024-04-14T08:33:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.544+0000 7f10b8ff9640 1 -- 172.21.15.172:0/1684478287 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10b00074c0 con 0x7f10bc0ccc40 2024-04-14T08:33:34.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.544+0000 7f10b8ff9640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10a806c8f0 0x7f10a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.545+0000 7f10b8ff9640 1 -- 172.21.15.172:0/1684478287 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f10b008b2b0 con 0x7f10bc0ccc40 2024-04-14T08:33:34.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.545+0000 7f10bbfff640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10a806c8f0 0x7f10a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.545+0000 7f10bbfff640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10a806c8f0 0x7f10a806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10ac036000 tx=0x7f10ac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:34.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.550+0000 7f10b8ff9640 1 -- 172.21.15.172:0/1684478287 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f10bc0c9670 con 0x7f10bc0ccc40 2024-04-14T08:33:34.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.725+0000 7f10c2788640 1 -- 172.21.15.172:0/1684478287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f10bc15f530 con 0x7f10bc0ccc40 2024-04-14T08:33:34.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.725+0000 7f10b8ff9640 1 -- 172.21.15.172:0/1684478287 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f10b0058c90 con 0x7f10bc0ccc40 2024-04-14T08:33:34.726 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:34.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.727+0000 7f10c2788640 1 -- 172.21.15.172:0/1684478287 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10a806c8f0 msgr2=0x7f10a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.727+0000 7f10c2788640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10a806c8f0 0x7f10a806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10ac036000 tx=0x7f10ac008030 comp rx=0 tx=0).stop 2024-04-14T08:33:34.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.727+0000 7f10c2788640 1 -- 172.21.15.172:0/1684478287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10bc0ccc40 msgr2=0x7f10bc0975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.727+0000 7f10c2788640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10bc0ccc40 0x7f10bc0975c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c440 tx=0x7f10b000c900 comp rx=0 tx=0).stop 2024-04-14T08:33:34.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.727+0000 7f10c2788640 1 -- 172.21.15.172:0/1684478287 shutdown_connections 2024-04-14T08:33:34.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.727+0000 7f10c2788640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10a806c8f0 0x7f10a806edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.727+0000 7f10c2788640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10bc15e8a0 0x7f10bc0c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.727+0000 7f10c2788640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10bc0ccc40 0x7f10bc0975c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.727+0000 7f10c2788640 1 --2- 172.21.15.172:0/1684478287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10bc06f520 0x7f10bc0991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.727+0000 7f10c2788640 1 -- 172.21.15.172:0/1684478287 >> 172.21.15.172:0/1684478287 conn(0x7f10bc0b1d80 msgr2=0x7f10bc0ca860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:34.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.728+0000 7f10c2788640 1 -- 172.21.15.172:0/1684478287 shutdown_connections 2024-04-14T08:33:34.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.728+0000 7f10c2788640 1 -- 172.21.15.172:0/1684478287 wait complete. 2024-04-14T08:33:34.739 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:34.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.842+0000 7fdc29e36640 1 Processor -- start 2024-04-14T08:33:34.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.843+0000 7fdc29e36640 1 -- start start 2024-04-14T08:33:34.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.843+0000 7fdc29e36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdc240d1a20 0x7fdc2406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.843+0000 7fdc29e36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc24071250 0x7fdc240cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.843+0000 7fdc29e36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdc2406d000 0x7fdc2406d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.843+0000 7fdc29e36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fdc2406d9d0 con 0x7fdc2406d000 2024-04-14T08:33:34.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.843+0000 7fdc29e36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fdc24070dc0 con 0x7fdc240d1a20 2024-04-14T08:33:34.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.843+0000 7fdc29e36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdc240701b0 con 0x7fdc24071250 2024-04-14T08:33:34.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.843+0000 7fdc23fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdc2406d000 0x7fdc2406d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.844+0000 7fdc237fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdc240d1a20 0x7fdc2406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.844+0000 7fdc23fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdc2406d000 0x7fdc2406d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37288/0 (socket says 172.21.15.172:37288) 2024-04-14T08:33:34.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.844+0000 7fdc23fff640 1 -- 172.21.15.172:0/3077539462 learned_addr learned my addr 172.21.15.172:0/3077539462 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:34.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.844+0000 7fdc1affd640 1 --2- 172.21.15.172:0/3077539462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc24071250 0x7fdc240cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.844+0000 7fdc23fff640 1 -- 172.21.15.172:0/3077539462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc24071250 msgr2=0x7fdc240cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:34.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.844+0000 7fdc23fff640 1 --2- 172.21.15.172:0/3077539462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc24071250 0x7fdc240cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.844+0000 7fdc23fff640 1 -- 172.21.15.172:0/3077539462 >> v2:172.21.15.177:3300/0 conn(0x7fdc240d1a20 msgr2=0x7fdc2406fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:34.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.844+0000 7fdc23fff640 1 --2- 172.21.15.172:0/3077539462 >> v2:172.21.15.177:3300/0 conn(0x7fdc240d1a20 0x7fdc2406fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.844+0000 7fdc23fff640 1 -- 172.21.15.172:0/3077539462 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc24070490 con 0x7fdc2406d000 2024-04-14T08:33:34.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.844+0000 7fdc237fe640 1 --2- 172.21.15.172:0/3077539462 >> v2:172.21.15.177:3300/0 conn(0x7fdc240d1a20 0x7fdc2406fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:34.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.844+0000 7fdc23fff640 1 --2- 172.21.15.172:0/3077539462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc2406d000 0x7fdc2406d490 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc240740b0 tx=0x7fdc1c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce4ac455e42c279 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:34.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.845+0000 7fdc22ffd640 1 -- 172.21.15.172:0/3077539462 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c02c9e0 con 0x7fdc2406d000 2024-04-14T08:33:34.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.845+0000 7fdc22ffd640 1 -- 172.21.15.172:0/3077539462 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc1c02cb80 con 0x7fdc2406d000 2024-04-14T08:33:34.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.845+0000 7fdc22ffd640 1 -- 172.21.15.172:0/3077539462 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0343f0 con 0x7fdc2406d000 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.845+0000 7fdc29e36640 1 -- 172.21.15.172:0/3077539462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc2406d000 msgr2=0x7fdc2406d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.845+0000 7fdc29e36640 1 --2- 172.21.15.172:0/3077539462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc2406d000 0x7fdc2406d490 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdc240740b0 tx=0x7fdc1c02be40 comp rx=0 tx=0).stop 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.845+0000 7fdc29e36640 1 -- 172.21.15.172:0/3077539462 shutdown_connections 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.845+0000 7fdc29e36640 1 --2- 172.21.15.172:0/3077539462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc2406d000 0x7fdc2406d490 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.845+0000 7fdc29e36640 1 --2- 172.21.15.172:0/3077539462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc24071250 0x7fdc240cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.845+0000 7fdc29e36640 1 --2- 172.21.15.172:0/3077539462 >> v2:172.21.15.177:3300/0 conn(0x7fdc240d1a20 0x7fdc2406fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.845+0000 7fdc29e36640 1 -- 172.21.15.172:0/3077539462 >> 172.21.15.172:0/3077539462 conn(0x7fdc2408c000 msgr2=0x7fdc240d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.845+0000 7fdc29e36640 1 -- 172.21.15.172:0/3077539462 shutdown_connections 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.845+0000 7fdc29e36640 1 -- 172.21.15.172:0/3077539462 wait complete. 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.846+0000 7fdc29e36640 1 Processor -- start 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.847+0000 7fdc29e36640 1 -- start start 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.847+0000 7fdc29e36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc2406d000 0x7fdc241f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.847+0000 7fdc29e36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc24071250 0x7fdc2414fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.847+0000 7fdc29e36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc240d1a20 0x7fdc240c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.847+0000 7fdc29e36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdc240cfa70 con 0x7fdc24071250 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.847+0000 7fdc29e36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24070a50 con 0x7fdc2406d000 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.847+0000 7fdc29e36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdc240d1e00 con 0x7fdc240d1a20 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.847+0000 7fdc1affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc24071250 0x7fdc2414fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.847+0000 7fdc1affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc24071250 0x7fdc2414fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37302/0 (socket says 172.21.15.172:37302) 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.847+0000 7fdc1affd640 1 -- 172.21.15.172:0/3190351647 learned_addr learned my addr 172.21.15.172:0/3190351647 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.847+0000 7fdc237fe640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc2406d000 0x7fdc241f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.847+0000 7fdc23fff640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc240d1a20 0x7fdc240c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.848+0000 7fdc1affd640 1 -- 172.21.15.172:0/3190351647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc240d1a20 msgr2=0x7fdc240c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.848+0000 7fdc1affd640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc240d1a20 0x7fdc240c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.848+0000 7fdc1affd640 1 -- 172.21.15.172:0/3190351647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc2406d000 msgr2=0x7fdc241f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.848+0000 7fdc1affd640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc2406d000 0x7fdc241f4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.848+0000 7fdc1affd640 1 -- 172.21.15.172:0/3190351647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc241503e0 con 0x7fdc24071250 2024-04-14T08:33:34.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.848+0000 7fdc1affd640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc24071250 0x7fdc2414fea0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00e840 tx=0x7fdc0c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:34.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.848+0000 7fdc20ff9640 1 -- 172.21.15.172:0/3190351647 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c017070 con 0x7fdc24071250 2024-04-14T08:33:34.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.848+0000 7fdc20ff9640 1 -- 172.21.15.172:0/3190351647 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc0c00c940 con 0x7fdc24071250 2024-04-14T08:33:34.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.848+0000 7fdc20ff9640 1 -- 172.21.15.172:0/3190351647 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c00cd10 con 0x7fdc24071250 2024-04-14T08:33:34.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.848+0000 7fdc29e36640 1 -- 172.21.15.172:0/3190351647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc24150660 con 0x7fdc24071250 2024-04-14T08:33:34.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.848+0000 7fdc29e36640 1 -- 172.21.15.172:0/3190351647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc24150c20 con 0x7fdc24071250 2024-04-14T08:33:34.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.849+0000 7fdc29e36640 1 -- 172.21.15.172:0/3190351647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc24075490 con 0x7fdc24071250 2024-04-14T08:33:34.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.849+0000 7fdc20ff9640 1 -- 172.21.15.172:0/3190351647 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc0c01a070 con 0x7fdc24071250 2024-04-14T08:33:34.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.850+0000 7fdc20ff9640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdc0406c8f0 0x7fdc0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:34.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.850+0000 7fdc20ff9640 1 -- 172.21.15.172:0/3190351647 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdc0c08bcb0 con 0x7fdc24071250 2024-04-14T08:33:34.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.850+0000 7fdc237fe640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdc0406c8f0 0x7fdc0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:34.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.850+0000 7fdc237fe640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdc0406c8f0 0x7fdc0406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009560 tx=0x7fdc14009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:34.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:34.854+0000 7fdc20ff9640 1 -- 172.21.15.172:0/3190351647 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdc241503e0 con 0x7fdc24071250 2024-04-14T08:33:35.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.029+0000 7fdc29e36640 1 -- 172.21.15.172:0/3190351647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc240b3740 con 0x7fdc24071250 2024-04-14T08:33:35.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.029+0000 7fdc20ff9640 1 -- 172.21.15.172:0/3190351647 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fdc0c059720 con 0x7fdc24071250 2024-04-14T08:33:35.030 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:35.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.031+0000 7fdc29e36640 1 -- 172.21.15.172:0/3190351647 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdc0406c8f0 msgr2=0x7fdc0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:35.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.031+0000 7fdc29e36640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdc0406c8f0 0x7fdc0406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009560 tx=0x7fdc14009190 comp rx=0 tx=0).stop 2024-04-14T08:33:35.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.031+0000 7fdc29e36640 1 -- 172.21.15.172:0/3190351647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc24071250 msgr2=0x7fdc2414fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:35.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.031+0000 7fdc29e36640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc24071250 0x7fdc2414fea0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00e840 tx=0x7fdc0c00ed00 comp rx=0 tx=0).stop 2024-04-14T08:33:35.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.031+0000 7fdc29e36640 1 -- 172.21.15.172:0/3190351647 shutdown_connections 2024-04-14T08:33:35.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.031+0000 7fdc29e36640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdc0406c8f0 0x7fdc0406edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.031+0000 7fdc29e36640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdc240d1a20 0x7fdc240c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.031+0000 7fdc29e36640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc24071250 0x7fdc2414fea0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.031+0000 7fdc29e36640 1 --2- 172.21.15.172:0/3190351647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc2406d000 0x7fdc241f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.031+0000 7fdc29e36640 1 -- 172.21.15.172:0/3190351647 >> 172.21.15.172:0/3190351647 conn(0x7fdc2408c000 msgr2=0x7fdc240b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:35.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.032+0000 7fdc29e36640 1 -- 172.21.15.172:0/3190351647 shutdown_connections 2024-04-14T08:33:35.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.032+0000 7fdc29e36640 1 -- 172.21.15.172:0/3190351647 wait complete. 2024-04-14T08:33:35.043 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:35.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.147+0000 7fb72dfc8640 1 Processor -- start 2024-04-14T08:33:35.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.147+0000 7fb72dfc8640 1 -- start start 2024-04-14T08:33:35.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.148+0000 7fb72dfc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb72806f3e0 0x7fb7280b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.148+0000 7fb72dfc8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb7280ceb00 0x7fb7280cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.148+0000 7fb72dfc8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7280ccdd0 con 0x7fb72815c420 2024-04-14T08:33:35.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.148+0000 7fb72dfc8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb7280d30a0 con 0x7fb7280ceb00 2024-04-14T08:33:35.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.148+0000 7fb72dfc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb7280cf490 con 0x7fb72806f3e0 2024-04-14T08:33:35.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.148+0000 7fb7277fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb72815c420 0x7fb7280d0010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46520/0 (socket says 172.21.15.172:46520) 2024-04-14T08:33:35.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.148+0000 7fb7277fe640 1 -- 172.21.15.172:0/2073194730 learned_addr learned my addr 172.21.15.172:0/2073194730 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:35.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.148+0000 7fb727fff640 1 --2- 172.21.15.172:0/2073194730 >> v2:172.21.15.177:3300/0 conn(0x7fb7280ceb00 0x7fb7280cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.148+0000 7fb726ffd640 1 --2- 172.21.15.172:0/2073194730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb72806f3e0 0x7fb7280b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.149+0000 7fb7267fc640 1 -- 172.21.15.172:0/2073194730 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2381954385 0 0) 0x7fb7280ccdd0 con 0x7fb72815c420 2024-04-14T08:33:35.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.149+0000 7fb727fff640 1 -- 172.21.15.172:0/2073194730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb72806f3e0 msgr2=0x7fb7280b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:35.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.149+0000 7fb727fff640 1 --2- 172.21.15.172:0/2073194730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb72806f3e0 0x7fb7280b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.149+0000 7fb727fff640 1 -- 172.21.15.172:0/2073194730 >> v1:172.21.15.172:6789/0 conn(0x7fb72815c420 legacy=0x7fb7280d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:35.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.149+0000 7fb727fff640 1 -- 172.21.15.172:0/2073194730 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7280d2ae0 con 0x7fb7280ceb00 2024-04-14T08:33:35.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.149+0000 7fb727fff640 1 --2- 172.21.15.172:0/2073194730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7280ceb00 0x7fb7280cef50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb720009d90 tx=0x7fb72002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47411c346a42545d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:35.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.150+0000 7fb7267fc640 1 -- 172.21.15.172:0/2073194730 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720039070 con 0x7fb7280ceb00 2024-04-14T08:33:35.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.150+0000 7fb7267fc640 1 -- 172.21.15.172:0/2073194730 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb72002cc50 con 0x7fb7280ceb00 2024-04-14T08:33:35.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.150+0000 7fb7267fc640 1 -- 172.21.15.172:0/2073194730 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720034440 con 0x7fb7280ceb00 2024-04-14T08:33:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.150+0000 7fb72dfc8640 1 -- 172.21.15.172:0/2073194730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7280ceb00 msgr2=0x7fb7280cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.150+0000 7fb72dfc8640 1 --2- 172.21.15.172:0/2073194730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7280ceb00 0x7fb7280cef50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb720009d90 tx=0x7fb72002bea0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.150+0000 7fb72dfc8640 1 -- 172.21.15.172:0/2073194730 shutdown_connections 2024-04-14T08:33:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.150+0000 7fb72dfc8640 1 --2- 172.21.15.172:0/2073194730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7280ceb00 0x7fb7280cef50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.150+0000 7fb72dfc8640 1 --2- 172.21.15.172:0/2073194730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb72806f3e0 0x7fb7280b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.150+0000 7fb72dfc8640 1 -- 172.21.15.172:0/2073194730 >> 172.21.15.172:0/2073194730 conn(0x7fb72808c000 msgr2=0x7fb7280d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.150+0000 7fb72dfc8640 1 -- 172.21.15.172:0/2073194730 shutdown_connections 2024-04-14T08:33:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.150+0000 7fb72dfc8640 1 -- 172.21.15.172:0/2073194730 wait complete. 2024-04-14T08:33:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.151+0000 7fb72dfc8640 1 Processor -- start 2024-04-14T08:33:35.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.152+0000 7fb72dfc8640 1 -- start start 2024-04-14T08:33:35.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.152+0000 7fb72dfc8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7280ceb00 0x7fb72806d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.152+0000 7fb72dfc8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb72815c420 0x7fb7280b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.152+0000 7fb72dfc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7280b1a30 0x7fb7280b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.152+0000 7fb72dfc8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb7280cd070 con 0x7fb72815c420 2024-04-14T08:33:35.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.152+0000 7fb72dfc8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb72806f870 con 0x7fb7280ceb00 2024-04-14T08:33:35.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.152+0000 7fb72dfc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb7280b11e0 con 0x7fb7280b1a30 2024-04-14T08:33:35.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.152+0000 7fb726ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb72815c420 0x7fb7280b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.152+0000 7fb726ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb72815c420 0x7fb7280b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37314/0 (socket says 172.21.15.172:37314) 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.152+0000 7fb726ffd640 1 -- 172.21.15.172:0/764977508 learned_addr learned my addr 172.21.15.172:0/764977508 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.152+0000 7fb7277fe640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7280ceb00 0x7fb72806d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.152+0000 7fb727fff640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7280b1a30 0x7fb7280b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.153+0000 7fb726ffd640 1 -- 172.21.15.172:0/764977508 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7280b1a30 msgr2=0x7fb7280b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.153+0000 7fb726ffd640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7280b1a30 0x7fb7280b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.153+0000 7fb726ffd640 1 -- 172.21.15.172:0/764977508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7280ceb00 msgr2=0x7fb72806d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.153+0000 7fb726ffd640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7280ceb00 0x7fb72806d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.153+0000 7fb726ffd640 1 -- 172.21.15.172:0/764977508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72806c880 con 0x7fb72815c420 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.153+0000 7fb7277fe640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7280ceb00 0x7fb72806d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.153+0000 7fb726ffd640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb72815c420 0x7fb7280b14f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00e9b0 tx=0x7fb71c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.153+0000 7fb70ffff640 1 -- 172.21.15.172:0/764977508 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c017070 con 0x7fb72815c420 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.153+0000 7fb70ffff640 1 -- 172.21.15.172:0/764977508 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71c00c9a0 con 0x7fb72815c420 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.153+0000 7fb72dfc8640 1 -- 172.21.15.172:0/764977508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72806cb60 con 0x7fb72815c420 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.153+0000 7fb70ffff640 1 -- 172.21.15.172:0/764977508 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00cd90 con 0x7fb72815c420 2024-04-14T08:33:35.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.153+0000 7fb72dfc8640 1 -- 172.21.15.172:0/764977508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72806d120 con 0x7fb72815c420 2024-04-14T08:33:35.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.154+0000 7fb72dfc8640 1 -- 172.21.15.172:0/764977508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7280b1e40 con 0x7fb72815c420 2024-04-14T08:33:35.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.154+0000 7fb70ffff640 1 -- 172.21.15.172:0/764977508 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb71c01a070 con 0x7fb72815c420 2024-04-14T08:33:35.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.155+0000 7fb70ffff640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb71406c8f0 0x7fb71406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.155+0000 7fb7277fe640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb71406c8f0 0x7fb71406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.155+0000 7fb70ffff640 1 -- 172.21.15.172:0/764977508 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb71c08c230 con 0x7fb72815c420 2024-04-14T08:33:35.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.155+0000 7fb7277fe640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb71406c8f0 0x7fb71406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb718007fd0 tx=0x7fb718007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:35.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.159+0000 7fb70ffff640 1 -- 172.21.15.172:0/764977508 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb7280b1e40 con 0x7fb72815c420 2024-04-14T08:33:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.333+0000 7fb72dfc8640 1 -- 172.21.15.172:0/764977508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb728075490 con 0x7fb72815c420 2024-04-14T08:33:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.334+0000 7fb70ffff640 1 -- 172.21.15.172:0/764977508 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fb71c059c10 con 0x7fb72815c420 2024-04-14T08:33:35.334 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.336+0000 7fb72dfc8640 1 -- 172.21.15.172:0/764977508 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb71406c8f0 msgr2=0x7fb71406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.336+0000 7fb72dfc8640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb71406c8f0 0x7fb71406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb718007fd0 tx=0x7fb718007de0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.336+0000 7fb72dfc8640 1 -- 172.21.15.172:0/764977508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb72815c420 msgr2=0x7fb7280b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.336+0000 7fb72dfc8640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb72815c420 0x7fb7280b14f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00e9b0 tx=0x7fb71c00ee70 comp rx=0 tx=0).stop 2024-04-14T08:33:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.336+0000 7fb72dfc8640 1 -- 172.21.15.172:0/764977508 shutdown_connections 2024-04-14T08:33:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.336+0000 7fb72dfc8640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb71406c8f0 0x7fb71406edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.336+0000 7fb72dfc8640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7280b1a30 0x7fb7280b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.336+0000 7fb72dfc8640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb72815c420 0x7fb7280b14f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.336+0000 7fb72dfc8640 1 --2- 172.21.15.172:0/764977508 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7280ceb00 0x7fb72806d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.336+0000 7fb72dfc8640 1 -- 172.21.15.172:0/764977508 >> 172.21.15.172:0/764977508 conn(0x7fb72808c000 msgr2=0x7fb7280aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.336+0000 7fb72dfc8640 1 -- 172.21.15.172:0/764977508 shutdown_connections 2024-04-14T08:33:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.336+0000 7fb72dfc8640 1 -- 172.21.15.172:0/764977508 wait complete. 2024-04-14T08:33:35.348 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-14T08:33:35.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.447+0000 7f3570436640 1 Processor -- start 2024-04-14T08:33:35.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.447+0000 7f3570436640 1 -- start start 2024-04-14T08:33:35.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.448+0000 7f3570436640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f35680cc450 0x7f35680af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.448+0000 7f3570436640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f35680af340 0x7f356806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.448+0000 7f3570436640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f356806dd10 0x7f3568162170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.448+0000 7f3570436640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f356806e160 con 0x7f35680cc450 2024-04-14T08:33:35.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.448+0000 7f3570436640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f35680cdea0 con 0x7f35680af340 2024-04-14T08:33:35.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.448+0000 7f3570436640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f35680cf920 con 0x7f356806dd10 2024-04-14T08:33:35.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.449+0000 7f356e1d2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f35680cc450 0x7f35680af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.449+0000 7f356e1d2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f35680cc450 0x7f35680af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37324/0 (socket says 172.21.15.172:37324) 2024-04-14T08:33:35.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.449+0000 7f356e1d2640 1 -- 172.21.15.172:0/950083319 learned_addr learned my addr 172.21.15.172:0/950083319 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:35.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.449+0000 7f356e9d3640 1 --2- 172.21.15.172:0/950083319 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f356806dd10 0x7f3568162170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.449+0000 7f356d9d1640 1 --2- 172.21.15.172:0/950083319 >> v2:172.21.15.177:3300/0 conn(0x7f35680af340 0x7f356806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.449+0000 7f356e1d2640 1 -- 172.21.15.172:0/950083319 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f356806dd10 msgr2=0x7f3568162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:35.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.449+0000 7f356e1d2640 1 --2- 172.21.15.172:0/950083319 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f356806dd10 0x7f3568162170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.449+0000 7f356e1d2640 1 -- 172.21.15.172:0/950083319 >> v2:172.21.15.177:3300/0 conn(0x7f35680af340 msgr2=0x7f356806d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:35.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.449+0000 7f356e1d2640 1 --2- 172.21.15.172:0/950083319 >> v2:172.21.15.177:3300/0 conn(0x7f35680af340 0x7f356806d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.449+0000 7f356e1d2640 1 -- 172.21.15.172:0/950083319 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356815f9e0 con 0x7f35680cc450 2024-04-14T08:33:35.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f356e1d2640 1 --2- 172.21.15.172:0/950083319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35680cc450 0x7f35680af790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f35680b2ee0 tx=0x7f356002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d30e79916a6eee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:35.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f356d1d0640 1 -- 172.21.15.172:0/950083319 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356002ca70 con 0x7f35680cc450 2024-04-14T08:33:35.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f356d1d0640 1 -- 172.21.15.172:0/950083319 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f356002cc10 con 0x7f35680cc450 2024-04-14T08:33:35.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f356d1d0640 1 -- 172.21.15.172:0/950083319 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35600343f0 con 0x7f35680cc450 2024-04-14T08:33:35.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f3570436640 1 -- 172.21.15.172:0/950083319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35680cc450 msgr2=0x7f35680af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:35.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f3570436640 1 --2- 172.21.15.172:0/950083319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35680cc450 0x7f35680af790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f35680b2ee0 tx=0x7f356002bed0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f3570436640 1 -- 172.21.15.172:0/950083319 shutdown_connections 2024-04-14T08:33:35.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f3570436640 1 --2- 172.21.15.172:0/950083319 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f356806dd10 0x7f3568162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f3570436640 1 --2- 172.21.15.172:0/950083319 >> v2:172.21.15.177:3300/0 conn(0x7f35680af340 0x7f356806d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f3570436640 1 --2- 172.21.15.172:0/950083319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35680cc450 0x7f35680af790 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f3570436640 1 -- 172.21.15.172:0/950083319 >> 172.21.15.172:0/950083319 conn(0x7f3568072b20 msgr2=0x7f3568072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:35.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f3570436640 1 -- 172.21.15.172:0/950083319 shutdown_connections 2024-04-14T08:33:35.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.450+0000 7f3570436640 1 -- 172.21.15.172:0/950083319 wait complete. 2024-04-14T08:33:35.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f3570436640 1 Processor -- start 2024-04-14T08:33:35.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f3570436640 1 -- start start 2024-04-14T08:33:35.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f3570436640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f356806dd10 0x7f3568161710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f3570436640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35680af340 0x7f35681598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f3570436640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35680cc450 0x7f35680b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f3570436640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f356806f7a0 con 0x7f35680af340 2024-04-14T08:33:35.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f3570436640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f356806f2a0 con 0x7f35680cc450 2024-04-14T08:33:35.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f3570436640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f35680ce3d0 con 0x7f356806dd10 2024-04-14T08:33:35.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f356d9d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35680af340 0x7f35681598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f356d9d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35680af340 0x7f35681598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37326/0 (socket says 172.21.15.172:37326) 2024-04-14T08:33:35.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f356d9d1640 1 -- 172.21.15.172:0/3456409759 learned_addr learned my addr 172.21.15.172:0/3456409759 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:35.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f356e1d2640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f356806dd10 0x7f3568161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.452+0000 7f356e9d3640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35680cc450 0x7f35680b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.453+0000 7f356d9d1640 1 -- 172.21.15.172:0/3456409759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f356806dd10 msgr2=0x7f3568161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:35.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.453+0000 7f356d9d1640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f356806dd10 0x7f3568161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.453+0000 7f356d9d1640 1 -- 172.21.15.172:0/3456409759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35680cc450 msgr2=0x7f35680b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:35.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.453+0000 7f356d9d1640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35680cc450 0x7f35680b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:35.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.453+0000 7f356d9d1640 1 -- 172.21.15.172:0/3456409759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35680b5290 con 0x7f35680af340 2024-04-14T08:33:35.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.453+0000 7f356d9d1640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35680af340 0x7f35681598a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f355c00e870 tx=0x7f355c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.453+0000 7f3566ffd640 1 -- 172.21.15.172:0/3456409759 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c017070 con 0x7f35680af340 2024-04-14T08:33:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.453+0000 7f3566ffd640 1 -- 172.21.15.172:0/3456409759 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f355c00c940 con 0x7f35680af340 2024-04-14T08:33:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.453+0000 7f3570436640 1 -- 172.21.15.172:0/3456409759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35680b5550 con 0x7f35680af340 2024-04-14T08:33:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.453+0000 7f3566ffd640 1 -- 172.21.15.172:0/3456409759 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c00cd30 con 0x7f35680af340 2024-04-14T08:33:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.453+0000 7f3570436640 1 -- 172.21.15.172:0/3456409759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35680b5b10 con 0x7f35680af340 2024-04-14T08:33:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.454+0000 7f3570436640 1 -- 172.21.15.172:0/3456409759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3568068520 con 0x7f35680af340 2024-04-14T08:33:35.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.454+0000 7f3566ffd640 1 -- 172.21.15.172:0/3456409759 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f355c01a070 con 0x7f35680af340 2024-04-14T08:33:35.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.455+0000 7f3566ffd640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f355406c920 0x7f355406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:35.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.455+0000 7f3566ffd640 1 -- 172.21.15.172:0/3456409759 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f355c08bd20 con 0x7f35680af340 2024-04-14T08:33:35.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.455+0000 7f356e1d2640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f355406c920 0x7f355406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:35.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.455+0000 7f356e1d2640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f355406c920 0x7f355406ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3560035000 tx=0x7f3560030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:35.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.459+0000 7f3566ffd640 1 -- 172.21.15.172:0/3456409759 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f355c059700 con 0x7f35680af340 2024-04-14T08:33:35.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:35.703+0000 7f3570436640 1 -- 172.21.15.172:0/3456409759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f35680af120 con 0x7f35680af340 2024-04-14T08:33:36.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.345+0000 7f3566ffd640 1 -- 172.21.15.172:0/3456409759 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v63) ==== 145+0+0 (secure 0 0 0) 0x7f355c05da00 con 0x7f35680af340 2024-04-14T08:33:36.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.347+0000 7f3570436640 1 -- 172.21.15.172:0/3456409759 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f355406c920 msgr2=0x7f355406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:36.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.347+0000 7f3570436640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f355406c920 0x7f355406ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3560035000 tx=0x7f3560030000 comp rx=0 tx=0).stop 2024-04-14T08:33:36.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.347+0000 7f3570436640 1 -- 172.21.15.172:0/3456409759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35680af340 msgr2=0x7f35681598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:36.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.347+0000 7f3570436640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35680af340 0x7f35681598a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f355c00e870 tx=0x7f355c00ed30 comp rx=0 tx=0).stop 2024-04-14T08:33:36.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.347+0000 7f3570436640 1 -- 172.21.15.172:0/3456409759 shutdown_connections 2024-04-14T08:33:36.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.347+0000 7f3570436640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f355406c920 0x7f355406ede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.347+0000 7f3570436640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35680cc450 0x7f35680b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.347+0000 7f3570436640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35680af340 0x7f35681598a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.347+0000 7f3570436640 1 --2- 172.21.15.172:0/3456409759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f356806dd10 0x7f3568161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.347+0000 7f3570436640 1 -- 172.21.15.172:0/3456409759 >> 172.21.15.172:0/3456409759 conn(0x7f3568072b20 msgr2=0x7f35680cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:36.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.347+0000 7f3570436640 1 -- 172.21.15.172:0/3456409759 shutdown_connections 2024-04-14T08:33:36.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.347+0000 7f3570436640 1 -- 172.21.15.172:0/3456409759 wait complete. 2024-04-14T08:33:36.361 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-14T08:33:36.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.468+0000 7f7924bc8640 1 Processor -- start 2024-04-14T08:33:36.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.468+0000 7f7924bc8640 1 -- start start 2024-04-14T08:33:36.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f7924bc8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f792013df30 0x7f79200b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:36.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f7924bc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f792015c380 0x7f79200b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:36.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f7924bc8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f792015b620 con 0x7f7920140770 2024-04-14T08:33:36.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f7924bc8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f79200cf9b0 con 0x7f792013df30 2024-04-14T08:33:36.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f7924bc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f79201414a0 con 0x7f792015c380 2024-04-14T08:33:36.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f791e59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f792013df30 0x7f79200b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:36.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f791ed9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7920140770 0x7f7920140c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46528/0 (socket says 172.21.15.172:46528) 2024-04-14T08:33:36.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f791ed9d640 1 -- 172.21.15.172:0/923151384 learned_addr learned my addr 172.21.15.172:0/923151384 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:36.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f7915d9b640 1 --2- 172.21.15.172:0/923151384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f792015c380 0x7f79200b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:36.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f791e59c640 1 -- 172.21.15.172:0/923151384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f792015c380 msgr2=0x7f79200b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:36.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f791e59c640 1 --2- 172.21.15.172:0/923151384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f792015c380 0x7f79200b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f791e59c640 1 -- 172.21.15.172:0/923151384 >> v1:172.21.15.172:6789/0 conn(0x7f7920140770 legacy=0x7f7920140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:36.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f791dd9b640 1 -- 172.21.15.172:0/923151384 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 794317057 0 0) 0x7f7918001490 con 0x7f7920140770 2024-04-14T08:33:36.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.469+0000 7f791e59c640 1 -- 172.21.15.172:0/923151384 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792015b8a0 con 0x7f792013df30 2024-04-14T08:33:36.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.470+0000 7f791e59c640 1 --2- 172.21.15.172:0/923151384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f792013df30 0x7f79200b1cd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f79100097d0 tx=0x7f791002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141093d8c8731df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:36.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.470+0000 7f791dd9b640 1 -- 172.21.15.172:0/923151384 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910035020 con 0x7f792013df30 2024-04-14T08:33:36.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.470+0000 7f791dd9b640 1 -- 172.21.15.172:0/923151384 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7910031800 con 0x7f792013df30 2024-04-14T08:33:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.470+0000 7f791dd9b640 1 -- 172.21.15.172:0/923151384 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910031bd0 con 0x7f792013df30 2024-04-14T08:33:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.470+0000 7f7924bc8640 1 -- 172.21.15.172:0/923151384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f792013df30 msgr2=0x7f79200b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.470+0000 7f7924bc8640 1 --2- 172.21.15.172:0/923151384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f792013df30 0x7f79200b1cd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f79100097d0 tx=0x7f791002b8a0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.471+0000 7f7924bc8640 1 -- 172.21.15.172:0/923151384 shutdown_connections 2024-04-14T08:33:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.471+0000 7f7924bc8640 1 --2- 172.21.15.172:0/923151384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f792015c380 0x7f79200b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.471+0000 7f7924bc8640 1 --2- 172.21.15.172:0/923151384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f792013df30 0x7f79200b1cd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.471+0000 7f7924bc8640 1 -- 172.21.15.172:0/923151384 >> 172.21.15.172:0/923151384 conn(0x7f792006b520 msgr2=0x7f792006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.471+0000 7f7924bc8640 1 -- 172.21.15.172:0/923151384 shutdown_connections 2024-04-14T08:33:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.471+0000 7f7924bc8640 1 -- 172.21.15.172:0/923151384 wait complete. 2024-04-14T08:33:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.472+0000 7f7924bc8640 1 Processor -- start 2024-04-14T08:33:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.472+0000 7f7924bc8640 1 -- start start 2024-04-14T08:33:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7924bc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f792013df30 0x7f79201f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7924bc8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7920140770 0x7f7920153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7924bc8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f792015c380 0x7f7920154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7924bc8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f792013e310 con 0x7f7920140770 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7924bc8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f792015be60 con 0x7f792015c380 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7924bc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f79200ce9a0 con 0x7f792013df30 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7915d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7920140770 0x7f7920153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7915d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7920140770 0x7f7920153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37338/0 (socket says 172.21.15.172:37338) 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7915d9b640 1 -- 172.21.15.172:0/2499851852 learned_addr learned my addr 172.21.15.172:0/2499851852 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f791ed9d640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f792015c380 0x7f7920154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f791e59c640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f792013df30 0x7f79201f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7915d9b640 1 -- 172.21.15.172:0/2499851852 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f792013df30 msgr2=0x7f79201f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7915d9b640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f792013df30 0x7f79201f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7915d9b640 1 -- 172.21.15.172:0/2499851852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f792015c380 msgr2=0x7f7920154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:36.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7915d9b640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f792015c380 0x7f7920154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7915d9b640 1 -- 172.21.15.172:0/2499851852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79201f6d00 con 0x7f7920140770 2024-04-14T08:33:36.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f7915d9b640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7920140770 0x7f7920153e40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f790800e8a0 tx=0x7f790800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:36.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.473+0000 7f79177fe640 1 -- 172.21.15.172:0/2499851852 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908017070 con 0x7f7920140770 2024-04-14T08:33:36.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.474+0000 7f79177fe640 1 -- 172.21.15.172:0/2499851852 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790800c940 con 0x7f7920140770 2024-04-14T08:33:36.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.474+0000 7f79177fe640 1 -- 172.21.15.172:0/2499851852 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790800cd30 con 0x7f7920140770 2024-04-14T08:33:36.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.474+0000 7f7924bc8640 1 -- 172.21.15.172:0/2499851852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79201f6180 con 0x7f7920140770 2024-04-14T08:33:36.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.474+0000 7f7924bc8640 1 -- 172.21.15.172:0/2499851852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79201f6a20 con 0x7f7920140770 2024-04-14T08:33:36.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.475+0000 7f7924bc8640 1 -- 172.21.15.172:0/2499851852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7920068590 con 0x7f7920140770 2024-04-14T08:33:36.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.475+0000 7f79177fe640 1 -- 172.21.15.172:0/2499851852 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f790801a070 con 0x7f7920140770 2024-04-14T08:33:36.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.475+0000 7f79177fe640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f790006c920 0x7f790006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:36.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.476+0000 7f791e59c640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f790006c920 0x7f790006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.476+0000 7f79177fe640 1 -- 172.21.15.172:0/2499851852 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f790808bc60 con 0x7f7920140770 2024-04-14T08:33:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.476+0000 7f791e59c640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f790006c920 0x7f790006ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7910036000 tx=0x7f791003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.480+0000 7f79177fe640 1 -- 172.21.15.172:0/2499851852 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f79080596d0 con 0x7f7920140770 2024-04-14T08:33:36.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.654+0000 7f7924bc8640 1 -- 172.21.15.172:0/2499851852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f79200b38b0 con 0x7f7920140770 2024-04-14T08:33:36.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.657+0000 7f79177fe640 1 -- 172.21.15.172:0/2499851852 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f790808b020 con 0x7f7920140770 2024-04-14T08:33:36.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.659+0000 7f7924bc8640 1 -- 172.21.15.172:0/2499851852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f790006c920 msgr2=0x7f790006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:36.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.659+0000 7f7924bc8640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f790006c920 0x7f790006ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7910036000 tx=0x7f791003d040 comp rx=0 tx=0).stop 2024-04-14T08:33:36.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.659+0000 7f7924bc8640 1 -- 172.21.15.172:0/2499851852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7920140770 msgr2=0x7f7920153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:36.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.659+0000 7f7924bc8640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7920140770 0x7f7920153e40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f790800e8a0 tx=0x7f790800ed60 comp rx=0 tx=0).stop 2024-04-14T08:33:36.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.659+0000 7f7924bc8640 1 -- 172.21.15.172:0/2499851852 shutdown_connections 2024-04-14T08:33:36.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.659+0000 7f7924bc8640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f790006c920 0x7f790006ede0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.659+0000 7f7924bc8640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f792015c380 0x7f7920154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.659+0000 7f7924bc8640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7920140770 0x7f7920153e40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.659+0000 7f7924bc8640 1 --2- 172.21.15.172:0/2499851852 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f792013df30 0x7f79201f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:36.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.659+0000 7f7924bc8640 1 -- 172.21.15.172:0/2499851852 >> 172.21.15.172:0/2499851852 conn(0x7f792006b520 msgr2=0x7f79200b5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:36.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.659+0000 7f7924bc8640 1 -- 172.21.15.172:0/2499851852 shutdown_connections 2024-04-14T08:33:36.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:36.659+0000 7f7924bc8640 1 -- 172.21.15.172:0/2499851852 wait complete. 2024-04-14T08:33:36.671 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:33:36.671 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:33:36.727 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:33:36.773 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:33:36.773 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:33:36.773 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:36.790 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:33:36.790 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:36.846 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:33:36.846 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:36.901 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:37.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.041+0000 7efc08e1c640 1 Processor -- start 2024-04-14T08:33:37.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.042+0000 7efc08e1c640 1 -- start start 2024-04-14T08:33:37.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.042+0000 7efc08e1c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efc0406b520 0x7efc0406fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.042+0000 7efc08e1c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efc040b3dd0 0x7efc0406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.042+0000 7efc08e1c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc0406be40 con 0x7efc0406e130 2024-04-14T08:33:37.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.042+0000 7efc08e1c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7efc04072d80 con 0x7efc0406b520 2024-04-14T08:33:37.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.042+0000 7efc08e1c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7efc040b4540 con 0x7efc040b3dd0 2024-04-14T08:33:37.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.043+0000 7efc0259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efc0406b520 0x7efc0406fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.043+0000 7efc0259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efc0406b520 0x7efc0406fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37342/0 (socket says 172.21.15.172:37342) 2024-04-14T08:33:37.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.043+0000 7efc0259c640 1 -- 172.21.15.172:0/3020725560 learned_addr learned my addr 172.21.15.172:0/3020725560 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:37.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.043+0000 7efc01d9b640 1 --2- 172.21.15.172:0/3020725560 >> v2:172.21.15.177:3300/0 conn(0x7efc040b3dd0 0x7efc0406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.043+0000 7efc0259c640 1 -- 172.21.15.172:0/3020725560 >> v2:172.21.15.177:3300/0 conn(0x7efc040b3dd0 msgr2=0x7efc0406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:37.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.043+0000 7efc0259c640 1 --2- 172.21.15.172:0/3020725560 >> v2:172.21.15.177:3300/0 conn(0x7efc040b3dd0 0x7efc0406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.043+0000 7efc0259c640 1 -- 172.21.15.172:0/3020725560 >> v1:172.21.15.177:6789/0 conn(0x7efc0406e130 legacy=0x7efc0406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.043+0000 7efc0259c640 1 -- 172.21.15.172:0/3020725560 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc04072450 con 0x7efc0406b520 2024-04-14T08:33:37.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.044+0000 7efc0259c640 1 --2- 172.21.15.172:0/3020725560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc0406b520 0x7efc0406fdf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efc04072360 tx=0x7efbfc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbf62dabcea1d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:37.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.044+0000 7efc0159a640 1 -- 172.21.15.172:0/3020725560 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc02cb20 con 0x7efc0406b520 2024-04-14T08:33:37.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.044+0000 7efc0159a640 1 -- 172.21.15.172:0/3020725560 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbfc02ccc0 con 0x7efc0406b520 2024-04-14T08:33:37.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.044+0000 7efc0159a640 1 -- 172.21.15.172:0/3020725560 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc034610 con 0x7efc0406b520 2024-04-14T08:33:37.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.044+0000 7efc08e1c640 1 -- 172.21.15.172:0/3020725560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc0406b520 msgr2=0x7efc0406fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.044+0000 7efc08e1c640 1 --2- 172.21.15.172:0/3020725560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc0406b520 0x7efc0406fdf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efc04072360 tx=0x7efbfc02bf80 comp rx=0 tx=0).stop 2024-04-14T08:33:37.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.044+0000 7efc08e1c640 1 -- 172.21.15.172:0/3020725560 shutdown_connections 2024-04-14T08:33:37.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.044+0000 7efc08e1c640 1 --2- 172.21.15.172:0/3020725560 >> v2:172.21.15.177:3300/0 conn(0x7efc040b3dd0 0x7efc0406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.044+0000 7efc08e1c640 1 --2- 172.21.15.172:0/3020725560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc0406b520 0x7efc0406fdf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.044+0000 7efc08e1c640 1 -- 172.21.15.172:0/3020725560 >> 172.21.15.172:0/3020725560 conn(0x7efc040d0f20 msgr2=0x7efc040d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.044+0000 7efc08e1c640 1 -- 172.21.15.172:0/3020725560 shutdown_connections 2024-04-14T08:33:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.044+0000 7efc08e1c640 1 -- 172.21.15.172:0/3020725560 wait complete. 2024-04-14T08:33:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.046+0000 7efc08e1c640 1 Processor -- start 2024-04-14T08:33:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.046+0000 7efc08e1c640 1 -- start start 2024-04-14T08:33:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.046+0000 7efc08e1c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc0406b520 0x7efc04145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.046+0000 7efc08e1c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc0406e130 0x7efc041f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.046+0000 7efc08e1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc040b3dd0 0x7efc040ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.046+0000 7efc08e1c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efc040b4250 con 0x7efc040b3dd0 2024-04-14T08:33:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.046+0000 7efc08e1c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efc04072a10 con 0x7efc0406e130 2024-04-14T08:33:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.046+0000 7efc08e1c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efc04073210 con 0x7efc0406b520 2024-04-14T08:33:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc02d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc040b3dd0 0x7efc040ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc02d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc040b3dd0 0x7efc040ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37350/0 (socket says 172.21.15.172:37350) 2024-04-14T08:33:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc01d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc0406e130 0x7efc041f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc02d9d640 1 -- 172.21.15.172:0/3206064766 learned_addr learned my addr 172.21.15.172:0/3206064766 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc0259c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc0406b520 0x7efc04145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc02d9d640 1 -- 172.21.15.172:0/3206064766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc0406b520 msgr2=0x7efc04145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc02d9d640 1 --2- 172.21.15.172:0/3206064766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc0406b520 0x7efc04145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc02d9d640 1 -- 172.21.15.172:0/3206064766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc0406e130 msgr2=0x7efc041f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc02d9d640 1 --2- 172.21.15.172:0/3206064766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc0406e130 0x7efc041f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc02d9d640 1 -- 172.21.15.172:0/3206064766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc041f5450 con 0x7efc040b3dd0 2024-04-14T08:33:37.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc02d9d640 1 --2- 172.21.15.172:0/3206064766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc040b3dd0 0x7efc040ae3e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efbf400c3f0 tx=0x7efbf400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:37.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efbfaffd640 1 -- 172.21.15.172:0/3206064766 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf40052d0 con 0x7efc040b3dd0 2024-04-14T08:33:37.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efbfaffd640 1 -- 172.21.15.172:0/3206064766 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf4005490 con 0x7efc040b3dd0 2024-04-14T08:33:37.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc08e1c640 1 -- 172.21.15.172:0/3206064766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc04156120 con 0x7efc040b3dd0 2024-04-14T08:33:37.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efbfaffd640 1 -- 172.21.15.172:0/3206064766 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4007260 con 0x7efc040b3dd0 2024-04-14T08:33:37.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.047+0000 7efc08e1c640 1 -- 172.21.15.172:0/3206064766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc04155360 con 0x7efc040b3dd0 2024-04-14T08:33:37.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.048+0000 7efc08e1c640 1 -- 172.21.15.172:0/3206064766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc04068520 con 0x7efc040b3dd0 2024-04-14T08:33:37.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.049+0000 7efbfaffd640 1 -- 172.21.15.172:0/3206064766 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efbf4007400 con 0x7efc040b3dd0 2024-04-14T08:33:37.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.049+0000 7efbfaffd640 1 --2- 172.21.15.172:0/3206064766 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efbe406c920 0x7efbe406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.049+0000 7efbfaffd640 1 -- 172.21.15.172:0/3206064766 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efbf408b2d0 con 0x7efc040b3dd0 2024-04-14T08:33:37.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.049+0000 7efc0259c640 1 --2- 172.21.15.172:0/3206064766 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efbe406c920 0x7efbe406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.050+0000 7efc0259c640 1 --2- 172.21.15.172:0/3206064766 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efbe406c920 0x7efbe406ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efbfc035000 tx=0x7efbfc030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:37.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.054+0000 7efbfaffd640 1 -- 172.21.15.172:0/3206064766 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efc041f5450 con 0x7efc040b3dd0 2024-04-14T08:33:37.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.230+0000 7efc08e1c640 1 -- 172.21.15.172:0/3206064766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc0406c3b0 con 0x7efc040b3dd0 2024-04-14T08:33:37.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.231+0000 7efbfaffd640 1 -- 172.21.15.172:0/3206064766 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18962 (secure 0 0 0) 0x7efbf4058cb0 con 0x7efc040b3dd0 2024-04-14T08:33:37.232 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:37.232 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":27,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:33:34.330426+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:37.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.233+0000 7efc08e1c640 1 -- 172.21.15.172:0/3206064766 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efbe406c920 msgr2=0x7efbe406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.233+0000 7efc08e1c640 1 --2- 172.21.15.172:0/3206064766 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efbe406c920 0x7efbe406ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efbfc035000 tx=0x7efbfc030000 comp rx=0 tx=0).stop 2024-04-14T08:33:37.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.233+0000 7efc08e1c640 1 -- 172.21.15.172:0/3206064766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc040b3dd0 msgr2=0x7efc040ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.233+0000 7efc08e1c640 1 --2- 172.21.15.172:0/3206064766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc040b3dd0 0x7efc040ae3e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efbf400c3f0 tx=0x7efbf400c8b0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.233+0000 7efc08e1c640 1 -- 172.21.15.172:0/3206064766 shutdown_connections 2024-04-14T08:33:37.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.233+0000 7efc08e1c640 1 --2- 172.21.15.172:0/3206064766 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efbe406c920 0x7efbe406ede0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.233+0000 7efc08e1c640 1 --2- 172.21.15.172:0/3206064766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc040b3dd0 0x7efc040ae3e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.233+0000 7efc08e1c640 1 --2- 172.21.15.172:0/3206064766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc0406e130 0x7efc041f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.233+0000 7efc08e1c640 1 --2- 172.21.15.172:0/3206064766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc0406b520 0x7efc04145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.233+0000 7efc08e1c640 1 -- 172.21.15.172:0/3206064766 >> 172.21.15.172:0/3206064766 conn(0x7efc040d0f20 msgr2=0x7efc0406f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:37.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.233+0000 7efc08e1c640 1 -- 172.21.15.172:0/3206064766 shutdown_connections 2024-04-14T08:33:37.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.233+0000 7efc08e1c640 1 -- 172.21.15.172:0/3206064766 wait complete. 2024-04-14T08:33:37.245 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:37.246 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:37.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.350+0000 7fb99a8c8640 1 Processor -- start 2024-04-14T08:33:37.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.350+0000 7fb99a8c8640 1 -- start start 2024-04-14T08:33:37.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.350+0000 7fb99a8c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb994071250 0x7fb99406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.350+0000 7fb99a8c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb99406d000 0x7fb99406d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.350+0000 7fb99a8c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9940701b0 con 0x7fb9940d1a20 2024-04-14T08:33:37.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.350+0000 7fb99a8c8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb99406d9d0 con 0x7fb994071250 2024-04-14T08:33:37.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.350+0000 7fb99a8c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb994070dc0 con 0x7fb99406d000 2024-04-14T08:33:37.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.351+0000 7fb9937fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb994071250 0x7fb99406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.351+0000 7fb993fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb9940d1a20 0x7fb9940cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46532/0 (socket says 172.21.15.172:46532) 2024-04-14T08:33:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.351+0000 7fb993fff640 1 -- 172.21.15.172:0/976746788 learned_addr learned my addr 172.21.15.172:0/976746788 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.351+0000 7fb998e65640 1 --2- 172.21.15.172:0/976746788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb99406d000 0x7fb99406d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.351+0000 7fb9937fe640 1 -- 172.21.15.172:0/976746788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb99406d000 msgr2=0x7fb99406d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.351+0000 7fb9937fe640 1 --2- 172.21.15.172:0/976746788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb99406d000 0x7fb99406d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.351+0000 7fb9937fe640 1 -- 172.21.15.172:0/976746788 >> v1:172.21.15.172:6789/0 conn(0x7fb9940d1a20 legacy=0x7fb9940cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.351+0000 7fb992ffd640 1 -- 172.21.15.172:0/976746788 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546315885 0 0) 0x7fb9940701b0 con 0x7fb9940d1a20 2024-04-14T08:33:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.351+0000 7fb9937fe640 1 -- 172.21.15.172:0/976746788 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb994070490 con 0x7fb994071250 2024-04-14T08:33:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.351+0000 7fb998e65640 1 --2- 172.21.15.172:0/976746788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb99406d000 0x7fb99406d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.352+0000 7fb9937fe640 1 --2- 172.21.15.172:0/976746788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb994071250 0x7fb99406fb80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb9880097a0 tx=0x7fb98802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6486c29204af69b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.352+0000 7fb992ffd640 1 -- 172.21.15.172:0/976746788 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988031780 con 0x7fb994071250 2024-04-14T08:33:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.352+0000 7fb992ffd640 1 -- 172.21.15.172:0/976746788 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb988037070 con 0x7fb994071250 2024-04-14T08:33:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.352+0000 7fb992ffd640 1 -- 172.21.15.172:0/976746788 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988031b40 con 0x7fb994071250 2024-04-14T08:33:37.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.352+0000 7fb99a8c8640 1 -- 172.21.15.172:0/976746788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb994071250 msgr2=0x7fb99406fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.352+0000 7fb99a8c8640 1 --2- 172.21.15.172:0/976746788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb994071250 0x7fb99406fb80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb9880097a0 tx=0x7fb98802b820 comp rx=0 tx=0).stop 2024-04-14T08:33:37.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.352+0000 7fb99a8c8640 1 -- 172.21.15.172:0/976746788 shutdown_connections 2024-04-14T08:33:37.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.352+0000 7fb99a8c8640 1 --2- 172.21.15.172:0/976746788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb99406d000 0x7fb99406d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.352+0000 7fb99a8c8640 1 --2- 172.21.15.172:0/976746788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb994071250 0x7fb99406fb80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.352+0000 7fb99a8c8640 1 -- 172.21.15.172:0/976746788 >> 172.21.15.172:0/976746788 conn(0x7fb99408c000 msgr2=0x7fb9940d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:37.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.352+0000 7fb99a8c8640 1 -- 172.21.15.172:0/976746788 shutdown_connections 2024-04-14T08:33:37.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.353+0000 7fb99a8c8640 1 -- 172.21.15.172:0/976746788 wait complete. 2024-04-14T08:33:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.354+0000 7fb99a8c8640 1 Processor -- start 2024-04-14T08:33:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.354+0000 7fb99a8c8640 1 -- start start 2024-04-14T08:33:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.354+0000 7fb99a8c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99406d000 0x7fb9940b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.354+0000 7fb99a8c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb994071250 0x7fb99414e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb99a8c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9940d1a20 0x7fb9940cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb99a8c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb9940d1e00 con 0x7fb99406d000 2024-04-14T08:33:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb99a8c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb994070a50 con 0x7fb9940d1a20 2024-04-14T08:33:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb99a8c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb9940ca970 con 0x7fb994071250 2024-04-14T08:33:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb993fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99406d000 0x7fb9940b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb993fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99406d000 0x7fb9940b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37360/0 (socket says 172.21.15.172:37360) 2024-04-14T08:33:37.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb993fff640 1 -- 172.21.15.172:0/3645066363 learned_addr learned my addr 172.21.15.172:0/3645066363 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:37.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb998e65640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9940d1a20 0x7fb9940cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb9937fe640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb994071250 0x7fb99414e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb993fff640 1 -- 172.21.15.172:0/3645066363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb994071250 msgr2=0x7fb99414e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb993fff640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb994071250 0x7fb99414e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb993fff640 1 -- 172.21.15.172:0/3645066363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9940d1a20 msgr2=0x7fb9940cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb993fff640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9940d1a20 0x7fb9940cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb993fff640 1 -- 172.21.15.172:0/3645066363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9940cac50 con 0x7fb99406d000 2024-04-14T08:33:37.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb993fff640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99406d000 0x7fb9940b4d30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb98400c4d0 tx=0x7fb98400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:37.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb990ff9640 1 -- 172.21.15.172:0/3645066363 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9840074e0 con 0x7fb99406d000 2024-04-14T08:33:37.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb990ff9640 1 -- 172.21.15.172:0/3645066363 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9840047e0 con 0x7fb99406d000 2024-04-14T08:33:37.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb99a8c8640 1 -- 172.21.15.172:0/3645066363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9940caf30 con 0x7fb99406d000 2024-04-14T08:33:37.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb990ff9640 1 -- 172.21.15.172:0/3645066363 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984004bd0 con 0x7fb99406d000 2024-04-14T08:33:37.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.355+0000 7fb99a8c8640 1 -- 172.21.15.172:0/3645066363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9940cb4f0 con 0x7fb99406d000 2024-04-14T08:33:37.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.356+0000 7fb99a8c8640 1 -- 172.21.15.172:0/3645066363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb994075490 con 0x7fb99406d000 2024-04-14T08:33:37.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.357+0000 7fb990ff9640 1 -- 172.21.15.172:0/3645066363 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb984011070 con 0x7fb99406d000 2024-04-14T08:33:37.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.357+0000 7fb990ff9640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb98006c920 0x7fb98006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.357+0000 7fb990ff9640 1 -- 172.21.15.172:0/3645066363 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb98408bab0 con 0x7fb99406d000 2024-04-14T08:33:37.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.357+0000 7fb9937fe640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb98006c920 0x7fb98006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.358+0000 7fb9937fe640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb98006c920 0x7fb98006ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb988036000 tx=0x7fb988008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:37.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.362+0000 7fb990ff9640 1 -- 172.21.15.172:0/3645066363 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb984059ae0 con 0x7fb99406d000 2024-04-14T08:33:37.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.534+0000 7fb99a8c8640 1 -- 172.21.15.172:0/3645066363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb9940d2820 con 0x7fb99406d000 2024-04-14T08:33:37.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.535+0000 7fb990ff9640 1 -- 172.21.15.172:0/3645066363 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fb9840593e0 con 0x7fb99406d000 2024-04-14T08:33:37.535 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:37.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.537+0000 7fb99a8c8640 1 -- 172.21.15.172:0/3645066363 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb98006c920 msgr2=0x7fb98006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.537+0000 7fb99a8c8640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb98006c920 0x7fb98006ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb988036000 tx=0x7fb988008030 comp rx=0 tx=0).stop 2024-04-14T08:33:37.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.537+0000 7fb99a8c8640 1 -- 172.21.15.172:0/3645066363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99406d000 msgr2=0x7fb9940b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.537+0000 7fb99a8c8640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99406d000 0x7fb9940b4d30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb98400c4d0 tx=0x7fb98400c8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.537+0000 7fb99a8c8640 1 -- 172.21.15.172:0/3645066363 shutdown_connections 2024-04-14T08:33:37.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.537+0000 7fb99a8c8640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb98006c920 0x7fb98006ede0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.537+0000 7fb99a8c8640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9940d1a20 0x7fb9940cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.537+0000 7fb99a8c8640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb994071250 0x7fb99414e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.537+0000 7fb99a8c8640 1 --2- 172.21.15.172:0/3645066363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99406d000 0x7fb9940b4d30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.537+0000 7fb99a8c8640 1 -- 172.21.15.172:0/3645066363 >> 172.21.15.172:0/3645066363 conn(0x7fb99408c000 msgr2=0x7fb9940ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:37.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.537+0000 7fb99a8c8640 1 -- 172.21.15.172:0/3645066363 shutdown_connections 2024-04-14T08:33:37.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.537+0000 7fb99a8c8640 1 -- 172.21.15.172:0/3645066363 wait complete. 2024-04-14T08:33:37.548 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:37.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.652+0000 7fb6ab40f640 1 Processor -- start 2024-04-14T08:33:37.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.652+0000 7fb6ab40f640 1 -- start start 2024-04-14T08:33:37.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6ab40f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb6a40ccca0 0x7fb6a406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6ab40f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb6a406bc30 0x7fb6a40b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6ab40f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6a40ceee0 con 0x7fb6a4073090 2024-04-14T08:33:37.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6ab40f640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a4073520 con 0x7fb6a406bc30 2024-04-14T08:33:37.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6ab40f640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a4070150 con 0x7fb6a40ccca0 2024-04-14T08:33:37.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6a99ac640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb6a406bc30 0x7fb6a40b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6a89aa640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb6a4073090 0x7fb6a406b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46548/0 (socket says 172.21.15.172:46548) 2024-04-14T08:33:37.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6a89aa640 1 -- 172.21.15.172:0/2552077991 learned_addr learned my addr 172.21.15.172:0/2552077991 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:37.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6a91ab640 1 --2- 172.21.15.172:0/2552077991 >> v2:172.21.15.177:3300/0 conn(0x7fb6a40ccca0 0x7fb6a406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6a99ac640 1 -- 172.21.15.172:0/2552077991 >> v2:172.21.15.177:3300/0 conn(0x7fb6a40ccca0 msgr2=0x7fb6a406fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:37.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6a99ac640 1 --2- 172.21.15.172:0/2552077991 >> v2:172.21.15.177:3300/0 conn(0x7fb6a40ccca0 0x7fb6a406fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6a99ac640 1 -- 172.21.15.172:0/2552077991 >> v1:172.21.15.172:6789/0 conn(0x7fb6a4073090 legacy=0x7fb6a406b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb69bfff640 1 -- 172.21.15.172:0/2552077991 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2128044767 0 0) 0x7fb6a40ceee0 con 0x7fb6a4073090 2024-04-14T08:33:37.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6a99ac640 1 -- 172.21.15.172:0/2552077991 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a40722d0 con 0x7fb6a406bc30 2024-04-14T08:33:37.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.653+0000 7fb6a91ab640 1 --2- 172.21.15.172:0/2552077991 >> v2:172.21.15.177:3300/0 conn(0x7fb6a40ccca0 0x7fb6a406fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:37.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.654+0000 7fb6a99ac640 1 --2- 172.21.15.172:0/2552077991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6a406bc30 0x7fb6a40b0190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4068ce0 tx=0x7fb69c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7903193d10d14701 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:37.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.654+0000 7fb69bfff640 1 -- 172.21.15.172:0/2552077991 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c039070 con 0x7fb6a406bc30 2024-04-14T08:33:37.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.654+0000 7fb69bfff640 1 -- 172.21.15.172:0/2552077991 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb69c02cbf0 con 0x7fb6a406bc30 2024-04-14T08:33:37.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.654+0000 7fb69bfff640 1 -- 172.21.15.172:0/2552077991 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c034440 con 0x7fb6a406bc30 2024-04-14T08:33:37.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.655+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2552077991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6a406bc30 msgr2=0x7fb6a40b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.655+0000 7fb6ab40f640 1 --2- 172.21.15.172:0/2552077991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6a406bc30 0x7fb6a40b0190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4068ce0 tx=0x7fb69c02be40 comp rx=0 tx=0).stop 2024-04-14T08:33:37.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.655+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2552077991 shutdown_connections 2024-04-14T08:33:37.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.655+0000 7fb6ab40f640 1 --2- 172.21.15.172:0/2552077991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6a406bc30 0x7fb6a40b0190 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.655+0000 7fb6ab40f640 1 --2- 172.21.15.172:0/2552077991 >> v2:172.21.15.177:3300/0 conn(0x7fb6a40ccca0 0x7fb6a406fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.655+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2552077991 >> 172.21.15.172:0/2552077991 conn(0x7fb6a40cda00 msgr2=0x7fb6a40cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.655+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2552077991 shutdown_connections 2024-04-14T08:33:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.655+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2552077991 wait complete. 2024-04-14T08:33:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.656+0000 7fb6ab40f640 1 Processor -- start 2024-04-14T08:33:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.656+0000 7fb6ab40f640 1 -- start start 2024-04-14T08:33:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6ab40f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6a406bc30 0x7fb6a414dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6ab40f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6a4073090 0x7fb6a40cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6ab40f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6a40ccca0 0x7fb6a40cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6ab40f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a40cd080 con 0x7fb6a40ccca0 2024-04-14T08:33:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6ab40f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4072890 con 0x7fb6a406bc30 2024-04-14T08:33:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6ab40f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a406e150 con 0x7fb6a4073090 2024-04-14T08:33:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6a99ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6a40ccca0 0x7fb6a40cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6a99ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6a40ccca0 0x7fb6a40cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37382/0 (socket says 172.21.15.172:37382) 2024-04-14T08:33:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6a99ac640 1 -- 172.21.15.172:0/2984126413 learned_addr learned my addr 172.21.15.172:0/2984126413 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6a89aa640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6a4073090 0x7fb6a40cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6a91ab640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6a406bc30 0x7fb6a414dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6a99ac640 1 -- 172.21.15.172:0/2984126413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6a4073090 msgr2=0x7fb6a40cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6a99ac640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6a4073090 0x7fb6a40cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6a99ac640 1 -- 172.21.15.172:0/2984126413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6a406bc30 msgr2=0x7fb6a414dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6a99ac640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6a406bc30 0x7fb6a414dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6a99ac640 1 -- 172.21.15.172:0/2984126413 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a4154870 con 0x7fb6a40ccca0 2024-04-14T08:33:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.657+0000 7fb6a99ac640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6a40ccca0 0x7fb6a40cb5f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009e80 tx=0x7fb69c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.658+0000 7fb699ffb640 1 -- 172.21.15.172:0/2984126413 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c039050 con 0x7fb6a40ccca0 2024-04-14T08:33:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.658+0000 7fb699ffb640 1 -- 172.21.15.172:0/2984126413 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb69c045070 con 0x7fb6a40ccca0 2024-04-14T08:33:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.658+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2984126413 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a40cbc20 con 0x7fb6a40ccca0 2024-04-14T08:33:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.658+0000 7fb699ffb640 1 -- 172.21.15.172:0/2984126413 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c009450 con 0x7fb6a40ccca0 2024-04-14T08:33:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.658+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2984126413 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a414e530 con 0x7fb6a40ccca0 2024-04-14T08:33:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.659+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2984126413 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a4068590 con 0x7fb6a40ccca0 2024-04-14T08:33:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.660+0000 7fb699ffb640 1 -- 172.21.15.172:0/2984126413 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb69c03e850 con 0x7fb6a40ccca0 2024-04-14T08:33:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.660+0000 7fb699ffb640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb68806c8f0 0x7fb68806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.660+0000 7fb699ffb640 1 -- 172.21.15.172:0/2984126413 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb69c0aebd0 con 0x7fb6a40ccca0 2024-04-14T08:33:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.660+0000 7fb6a91ab640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb68806c8f0 0x7fb68806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.661+0000 7fb6a91ab640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb68806c8f0 0x7fb68806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009560 tx=0x7fb6a0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.665+0000 7fb699ffb640 1 -- 172.21.15.172:0/2984126413 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb69c07c6d0 con 0x7fb6a40ccca0 2024-04-14T08:33:37.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.840+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2984126413 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb6a40cf260 con 0x7fb6a40ccca0 2024-04-14T08:33:37.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.840+0000 7fb699ffb640 1 -- 172.21.15.172:0/2984126413 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fb69c0809d0 con 0x7fb6a40ccca0 2024-04-14T08:33:37.841 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:37.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.842+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2984126413 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb68806c8f0 msgr2=0x7fb68806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.842+0000 7fb6ab40f640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb68806c8f0 0x7fb68806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009560 tx=0x7fb6a0009190 comp rx=0 tx=0).stop 2024-04-14T08:33:37.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.842+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2984126413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6a40ccca0 msgr2=0x7fb6a40cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.842+0000 7fb6ab40f640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6a40ccca0 0x7fb6a40cb5f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009e80 tx=0x7fb69c004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:37.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.842+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2984126413 shutdown_connections 2024-04-14T08:33:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.842+0000 7fb6ab40f640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb68806c8f0 0x7fb68806edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.842+0000 7fb6ab40f640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6a40ccca0 0x7fb6a40cb5f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.842+0000 7fb6ab40f640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6a4073090 0x7fb6a40cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.842+0000 7fb6ab40f640 1 --2- 172.21.15.172:0/2984126413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6a406bc30 0x7fb6a414dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.842+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2984126413 >> 172.21.15.172:0/2984126413 conn(0x7fb6a40cda00 msgr2=0x7fb6a406c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.843+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2984126413 shutdown_connections 2024-04-14T08:33:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.843+0000 7fb6ab40f640 1 -- 172.21.15.172:0/2984126413 wait complete. 2024-04-14T08:33:37.855 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:37.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.958+0000 7f95b1d08640 1 Processor -- start 2024-04-14T08:33:37.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.958+0000 7f95b1d08640 1 -- start start 2024-04-14T08:33:37.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95b1d08640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f95ac13df30 0x7f95ac0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95b1d08640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f95ac15c380 0x7f95ac0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95b1d08640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95ac15b620 con 0x7f95ac140770 2024-04-14T08:33:37.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95b1d08640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f95ac0cf9b0 con 0x7f95ac13df30 2024-04-14T08:33:37.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95b1d08640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f95ac1414a0 con 0x7f95ac15c380 2024-04-14T08:33:37.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95ab7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f95ac13df30 0x7f95ac0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95ab7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f95ac13df30 0x7f95ac0b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37384/0 (socket says 172.21.15.172:37384) 2024-04-14T08:33:37.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95ab7fe640 1 -- 172.21.15.172:0/1546288058 learned_addr learned my addr 172.21.15.172:0/1546288058 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:37.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95aaffd640 1 --2- 172.21.15.172:0/1546288058 >> v2:172.21.15.177:3300/0 conn(0x7f95ac15c380 0x7f95ac0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95ab7fe640 1 -- 172.21.15.172:0/1546288058 >> v2:172.21.15.177:3300/0 conn(0x7f95ac15c380 msgr2=0x7f95ac0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:37.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95ab7fe640 1 --2- 172.21.15.172:0/1546288058 >> v2:172.21.15.177:3300/0 conn(0x7f95ac15c380 0x7f95ac0b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95ab7fe640 1 -- 172.21.15.172:0/1546288058 >> v1:172.21.15.177:6789/0 conn(0x7f95ac140770 legacy=0x7f95ac140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.959+0000 7f95ab7fe640 1 -- 172.21.15.172:0/1546288058 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ac15b8a0 con 0x7f95ac13df30 2024-04-14T08:33:37.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.960+0000 7f95ab7fe640 1 --2- 172.21.15.172:0/1546288058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac13df30 0x7f95ac0b1cd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f95ac06c4d0 tx=0x7f959c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f46cdfd85e6edf server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:37.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.960+0000 7f95aa7fc640 1 -- 172.21.15.172:0/1546288058 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c034070 con 0x7f95ac13df30 2024-04-14T08:33:37.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.960+0000 7f95aa7fc640 1 -- 172.21.15.172:0/1546288058 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959c030860 con 0x7f95ac13df30 2024-04-14T08:33:37.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.960+0000 7f95aa7fc640 1 -- 172.21.15.172:0/1546288058 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c030c30 con 0x7f95ac13df30 2024-04-14T08:33:37.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.961+0000 7f95b1d08640 1 -- 172.21.15.172:0/1546288058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac13df30 msgr2=0x7f95ac0b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.961+0000 7f95b1d08640 1 --2- 172.21.15.172:0/1546288058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac13df30 0x7f95ac0b1cd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f95ac06c4d0 tx=0x7f959c02b9a0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.961+0000 7f95b1d08640 1 -- 172.21.15.172:0/1546288058 shutdown_connections 2024-04-14T08:33:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.961+0000 7f95b1d08640 1 --2- 172.21.15.172:0/1546288058 >> v2:172.21.15.177:3300/0 conn(0x7f95ac15c380 0x7f95ac0b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.961+0000 7f95b1d08640 1 --2- 172.21.15.172:0/1546288058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac13df30 0x7f95ac0b1cd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.961+0000 7f95b1d08640 1 -- 172.21.15.172:0/1546288058 >> 172.21.15.172:0/1546288058 conn(0x7f95ac06b520 msgr2=0x7f95ac06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.961+0000 7f95b1d08640 1 -- 172.21.15.172:0/1546288058 shutdown_connections 2024-04-14T08:33:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.961+0000 7f95b1d08640 1 -- 172.21.15.172:0/1546288058 wait complete. 2024-04-14T08:33:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.962+0000 7f95b1d08640 1 Processor -- start 2024-04-14T08:33:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.962+0000 7f95b1d08640 1 -- start start 2024-04-14T08:33:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95b1d08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95ac13df30 0x7f95ac1f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95b1d08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95ac140770 0x7f95ac153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95b1d08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac15c380 0x7f95ac153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95b1d08640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac13e310 con 0x7f95ac15c380 2024-04-14T08:33:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95b1d08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac15be60 con 0x7f95ac13df30 2024-04-14T08:33:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95b1d08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f95ac0ce9a0 con 0x7f95ac140770 2024-04-14T08:33:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac15c380 0x7f95ac153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac15c380 0x7f95ac153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37388/0 (socket says 172.21.15.172:37388) 2024-04-14T08:33:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95aaffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95ac140770 0x7f95ac153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95abfff640 1 -- 172.21.15.172:0/963260322 learned_addr learned my addr 172.21.15.172:0/963260322 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95ab7fe640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95ac13df30 0x7f95ac1f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95abfff640 1 -- 172.21.15.172:0/963260322 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95ac140770 msgr2=0x7f95ac153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95abfff640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95ac140770 0x7f95ac153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95abfff640 1 -- 172.21.15.172:0/963260322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95ac13df30 msgr2=0x7f95ac1f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95abfff640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95ac13df30 0x7f95ac1f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95abfff640 1 -- 172.21.15.172:0/963260322 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ac0b2e40 con 0x7f95ac15c380 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.963+0000 7f95abfff640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac15c380 0x7f95ac153650 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f95a400c7d0 tx=0x7f95a400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.964+0000 7f9593fff640 1 -- 172.21.15.172:0/963260322 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4003b70 con 0x7f95ac15c380 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.964+0000 7f9593fff640 1 -- 172.21.15.172:0/963260322 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a4003d30 con 0x7f95ac15c380 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.964+0000 7f95b1d08640 1 -- 172.21.15.172:0/963260322 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95ac0b22c0 con 0x7f95ac15c380 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.964+0000 7f9593fff640 1 -- 172.21.15.172:0/963260322 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a400aa90 con 0x7f95ac15c380 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.964+0000 7f95b1d08640 1 -- 172.21.15.172:0/963260322 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95ac0b2880 con 0x7f95ac15c380 2024-04-14T08:33:37.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.965+0000 7f95b1d08640 1 -- 172.21.15.172:0/963260322 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95ac068590 con 0x7f95ac15c380 2024-04-14T08:33:37.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.965+0000 7f9593fff640 1 -- 172.21.15.172:0/963260322 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95a400ac30 con 0x7f95ac15c380 2024-04-14T08:33:37.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.965+0000 7f9593fff640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f959806c8f0 0x7f959806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:37.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.966+0000 7f9593fff640 1 -- 172.21.15.172:0/963260322 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f95a408bcc0 con 0x7f95ac15c380 2024-04-14T08:33:37.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.966+0000 7f95ab7fe640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f959806c8f0 0x7f959806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:37.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.966+0000 7f95ab7fe640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f959806c8f0 0x7f959806edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f959c036000 tx=0x7f959c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:37.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:37.970+0000 7f9593fff640 1 -- 172.21.15.172:0/963260322 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f95ac15b620 con 0x7f95ac15c380 2024-04-14T08:33:38.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.144+0000 7f95b1d08640 1 -- 172.21.15.172:0/963260322 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f95ac06bc80 con 0x7f95ac15c380 2024-04-14T08:33:38.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.144+0000 7f9593fff640 1 -- 172.21.15.172:0/963260322 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f95a4059730 con 0x7f95ac15c380 2024-04-14T08:33:38.145 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:38.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.146+0000 7f95b1d08640 1 -- 172.21.15.172:0/963260322 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f959806c8f0 msgr2=0x7f959806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.146+0000 7f95b1d08640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f959806c8f0 0x7f959806edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f959c036000 tx=0x7f959c008030 comp rx=0 tx=0).stop 2024-04-14T08:33:38.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.146+0000 7f95b1d08640 1 -- 172.21.15.172:0/963260322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac15c380 msgr2=0x7f95ac153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.146+0000 7f95b1d08640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac15c380 0x7f95ac153650 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f95a400c7d0 tx=0x7f95a400cc90 comp rx=0 tx=0).stop 2024-04-14T08:33:38.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.146+0000 7f95b1d08640 1 -- 172.21.15.172:0/963260322 shutdown_connections 2024-04-14T08:33:38.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.146+0000 7f95b1d08640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f959806c8f0 0x7f959806edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.146+0000 7f95b1d08640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95ac15c380 0x7f95ac153650 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.146+0000 7f95b1d08640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95ac140770 0x7f95ac153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.146+0000 7f95b1d08640 1 --2- 172.21.15.172:0/963260322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95ac13df30 0x7f95ac1f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.146+0000 7f95b1d08640 1 -- 172.21.15.172:0/963260322 >> 172.21.15.172:0/963260322 conn(0x7f95ac06b520 msgr2=0x7f95ac13f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:38.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.146+0000 7f95b1d08640 1 -- 172.21.15.172:0/963260322 shutdown_connections 2024-04-14T08:33:38.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.146+0000 7f95b1d08640 1 -- 172.21.15.172:0/963260322 wait complete. 2024-04-14T08:33:38.158 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:38.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.263+0000 7f573b636640 1 Processor -- start 2024-04-14T08:33:38.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.264+0000 7f573b636640 1 -- start start 2024-04-14T08:33:38.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.264+0000 7f573b636640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f573406fb70 0x7f5734073090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:38.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.264+0000 7f573b636640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57340ceee0 con 0x7f573406bc30 2024-04-14T08:33:38.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.264+0000 7f573b636640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5734070150 con 0x7f57340ccca0 2024-04-14T08:33:38.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.264+0000 7f573b636640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f57340722d0 con 0x7f573406fb70 2024-04-14T08:33:38.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.265+0000 7f5738bd1640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f573406bc30 0x7f573406e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46562/0 (socket says 172.21.15.172:46562) 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.265+0000 7f5738bd1640 1 -- 172.21.15.172:0/1901384264 learned_addr learned my addr 172.21.15.172:0/1901384264 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.265+0000 7f5739bd3640 1 --2- 172.21.15.172:0/1901384264 >> v2:172.21.15.177:3300/0 conn(0x7f573406fb70 0x7f5734073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.265+0000 7f572bfff640 1 -- 172.21.15.172:0/1901384264 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515431368 0 0) 0x7f57340ceee0 con 0x7f573406bc30 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.265+0000 7f572bfff640 1 -- 172.21.15.172:0/1901384264 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5720003520 con 0x7f573406bc30 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.265+0000 7f572bfff640 1 -- 172.21.15.172:0/1901384264 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4293477759 0 0) 0x7f5734070150 con 0x7f57340ccca0 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.265+0000 7f572bfff640 1 -- 172.21.15.172:0/1901384264 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57340ceee0 con 0x7f57340ccca0 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.265+0000 7f5739bd3640 1 -- 172.21.15.172:0/1901384264 >> v1:172.21.15.177:6789/0 conn(0x7f57340ccca0 legacy=0x7f573406b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.265+0000 7f5739bd3640 1 -- 172.21.15.172:0/1901384264 >> v1:172.21.15.172:6789/0 conn(0x7f573406bc30 legacy=0x7f573406e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.265+0000 7f5739bd3640 1 -- 172.21.15.172:0/1901384264 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57340725b0 con 0x7f573406fb70 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.266+0000 7f5739bd3640 1 --2- 172.21.15.172:0/1901384264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573406fb70 0x7f5734073090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f572c009d90 tx=0x7f572c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb0ef18e97f495c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.266+0000 7f572bfff640 1 -- 172.21.15.172:0/1901384264 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c039070 con 0x7f573406fb70 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.266+0000 7f572bfff640 1 -- 172.21.15.172:0/1901384264 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f572c02cc60 con 0x7f573406fb70 2024-04-14T08:33:38.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.266+0000 7f572bfff640 1 -- 172.21.15.172:0/1901384264 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c034660 con 0x7f573406fb70 2024-04-14T08:33:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.266+0000 7f573b636640 1 -- 172.21.15.172:0/1901384264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573406fb70 msgr2=0x7f5734073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.266+0000 7f573b636640 1 --2- 172.21.15.172:0/1901384264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573406fb70 0x7f5734073090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f572c009d90 tx=0x7f572c02beb0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.266+0000 7f573b636640 1 -- 172.21.15.172:0/1901384264 shutdown_connections 2024-04-14T08:33:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.266+0000 7f573b636640 1 --2- 172.21.15.172:0/1901384264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573406fb70 0x7f5734073090 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.266+0000 7f573b636640 1 -- 172.21.15.172:0/1901384264 >> 172.21.15.172:0/1901384264 conn(0x7f57340cda00 msgr2=0x7f57340cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.267+0000 7f573b636640 1 -- 172.21.15.172:0/1901384264 shutdown_connections 2024-04-14T08:33:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.267+0000 7f573b636640 1 -- 172.21.15.172:0/1901384264 wait complete. 2024-04-14T08:33:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f573b636640 1 Processor -- start 2024-04-14T08:33:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f573b636640 1 -- start start 2024-04-14T08:33:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f573b636640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573406bc30 0x7f57341f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f573b636640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f573406fb70 0x7f573415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f573b636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57340ccca0 0x7f573415c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f573b636640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f57340b0190 con 0x7f573406fb70 2024-04-14T08:33:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f573b636640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f57340cd080 con 0x7f573406bc30 2024-04-14T08:33:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f573b636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f57340b0310 con 0x7f57340ccca0 2024-04-14T08:33:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f5738bd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f573406fb70 0x7f573415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f5739bd3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57340ccca0 0x7f573415c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f57393d2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573406bc30 0x7f57341f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f5738bd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f573406fb70 0x7f573415a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37396/0 (socket says 172.21.15.172:37396) 2024-04-14T08:33:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.268+0000 7f5738bd1640 1 -- 172.21.15.172:0/3946670378 learned_addr learned my addr 172.21.15.172:0/3946670378 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f5738bd1640 1 -- 172.21.15.172:0/3946670378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57340ccca0 msgr2=0x7f573415c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f5738bd1640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57340ccca0 0x7f573415c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f5738bd1640 1 -- 172.21.15.172:0/3946670378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573406bc30 msgr2=0x7f57341f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f5738bd1640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573406bc30 0x7f57341f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f5738bd1640 1 -- 172.21.15.172:0/3946670378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573415b2b0 con 0x7f573406fb70 2024-04-14T08:33:38.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f57393d2640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573406bc30 0x7f57341f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:38.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f5739bd3640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57340ccca0 0x7f573415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:38.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f5738bd1640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f573406fb70 0x7f573415a860 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f572400c4d0 tx=0x7f572400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:38.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f5729ffb640 1 -- 172.21.15.172:0/3946670378 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724017070 con 0x7f573406fb70 2024-04-14T08:33:38.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f5729ffb640 1 -- 172.21.15.172:0/3946670378 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5724005d00 con 0x7f573406fb70 2024-04-14T08:33:38.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f573b636640 1 -- 172.21.15.172:0/3946670378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573415afd0 con 0x7f573406fb70 2024-04-14T08:33:38.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f5729ffb640 1 -- 172.21.15.172:0/3946670378 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57240072b0 con 0x7f573406fb70 2024-04-14T08:33:38.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.269+0000 7f573b636640 1 -- 172.21.15.172:0/3946670378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57340b3e20 con 0x7f573406fb70 2024-04-14T08:33:38.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.270+0000 7f573b636640 1 -- 172.21.15.172:0/3946670378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5734068590 con 0x7f573406fb70 2024-04-14T08:33:38.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.271+0000 7f5729ffb640 1 -- 172.21.15.172:0/3946670378 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f572401a070 con 0x7f573406fb70 2024-04-14T08:33:38.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.271+0000 7f5729ffb640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f571806c920 0x7f571806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:38.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.271+0000 7f5729ffb640 1 -- 172.21.15.172:0/3946670378 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f572408b1b0 con 0x7f573406fb70 2024-04-14T08:33:38.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.271+0000 7f57393d2640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f571806c920 0x7f571806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:38.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.272+0000 7f57393d2640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f571806c920 0x7f571806ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5730007fd0 tx=0x7f5730009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.276+0000 7f5729ffb640 1 -- 172.21.15.172:0/3946670378 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5724058ce0 con 0x7f573406fb70 2024-04-14T08:33:38.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.450+0000 7f573b636640 1 -- 172.21.15.172:0/3946670378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f573406c120 con 0x7f573406fb70 2024-04-14T08:33:38.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.451+0000 7f5729ffb640 1 -- 172.21.15.172:0/3946670378 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f572405cfe0 con 0x7f573406fb70 2024-04-14T08:33:38.451 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:38.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.453+0000 7f573b636640 1 -- 172.21.15.172:0/3946670378 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f571806c920 msgr2=0x7f571806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.453+0000 7f573b636640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f571806c920 0x7f571806ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5730007fd0 tx=0x7f5730009040 comp rx=0 tx=0).stop 2024-04-14T08:33:38.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.453+0000 7f573b636640 1 -- 172.21.15.172:0/3946670378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f573406fb70 msgr2=0x7f573415a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.453+0000 7f573b636640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f573406fb70 0x7f573415a860 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f572400c4d0 tx=0x7f572400c990 comp rx=0 tx=0).stop 2024-04-14T08:33:38.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.453+0000 7f573b636640 1 -- 172.21.15.172:0/3946670378 shutdown_connections 2024-04-14T08:33:38.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.453+0000 7f573b636640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f571806c920 0x7f571806ede0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.453+0000 7f573b636640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57340ccca0 0x7f573415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.453+0000 7f573b636640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f573406fb70 0x7f573415a860 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.453+0000 7f573b636640 1 --2- 172.21.15.172:0/3946670378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573406bc30 0x7f57341f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.453+0000 7f573b636640 1 -- 172.21.15.172:0/3946670378 >> 172.21.15.172:0/3946670378 conn(0x7f57340cda00 msgr2=0x7f5734073100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:38.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.453+0000 7f573b636640 1 -- 172.21.15.172:0/3946670378 shutdown_connections 2024-04-14T08:33:38.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.453+0000 7f573b636640 1 -- 172.21.15.172:0/3946670378 wait complete. 2024-04-14T08:33:38.464 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:33:38.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.567+0000 7f1835d5e640 1 Processor -- start 2024-04-14T08:33:38.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.567+0000 7f1835d5e640 1 -- start start 2024-04-14T08:33:38.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f1835d5e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f183015e430 0x7f1830162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:38.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f1835d5e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1830162540 0x7f18301629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:38.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f1835d5e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18300b3a30 0x7f18300b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:38.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f1835d5e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f18300cefb0 con 0x7f183015e430 2024-04-14T08:33:38.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f1835d5e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f18300cf290 con 0x7f1830162540 2024-04-14T08:33:38.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f1835d5e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f18300cf570 con 0x7f18300b3a30 2024-04-14T08:33:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f182f7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f183015e430 0x7f1830162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f182f7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f183015e430 0x7f1830162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37404/0 (socket says 172.21.15.172:37404) 2024-04-14T08:33:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f182f7fe640 1 -- 172.21.15.172:0/3004046066 learned_addr learned my addr 172.21.15.172:0/3004046066 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f1827fff640 1 --2- 172.21.15.172:0/3004046066 >> v2:172.21.15.177:3300/0 conn(0x7f1830162540 0x7f18301629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f182ffff640 1 --2- 172.21.15.172:0/3004046066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18300b3a30 0x7f18300b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f182f7fe640 1 -- 172.21.15.172:0/3004046066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18300b3a30 msgr2=0x7f18300b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f182f7fe640 1 --2- 172.21.15.172:0/3004046066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18300b3a30 0x7f18300b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f182f7fe640 1 -- 172.21.15.172:0/3004046066 >> v2:172.21.15.177:3300/0 conn(0x7f1830162540 msgr2=0x7f18301629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f182f7fe640 1 --2- 172.21.15.172:0/3004046066 >> v2:172.21.15.177:3300/0 conn(0x7f1830162540 0x7f18301629d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.568+0000 7f182f7fe640 1 -- 172.21.15.172:0/3004046066 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18300cf850 con 0x7f183015e430 2024-04-14T08:33:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.569+0000 7f182f7fe640 1 --2- 172.21.15.172:0/3004046066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183015e430 0x7f1830162000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f183008dae0 tx=0x7f182002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f3f201b75dc36f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.569+0000 7f182effd640 1 -- 172.21.15.172:0/3004046066 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820035020 con 0x7f183015e430 2024-04-14T08:33:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.569+0000 7f182effd640 1 -- 172.21.15.172:0/3004046066 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1820031870 con 0x7f183015e430 2024-04-14T08:33:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.569+0000 7f182effd640 1 -- 172.21.15.172:0/3004046066 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820031c40 con 0x7f183015e430 2024-04-14T08:33:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.569+0000 7f1835d5e640 1 -- 172.21.15.172:0/3004046066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183015e430 msgr2=0x7f1830162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.569+0000 7f1835d5e640 1 --2- 172.21.15.172:0/3004046066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183015e430 0x7f1830162000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f183008dae0 tx=0x7f182002b910 comp rx=0 tx=0).stop 2024-04-14T08:33:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.570+0000 7f1835d5e640 1 -- 172.21.15.172:0/3004046066 shutdown_connections 2024-04-14T08:33:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.570+0000 7f1835d5e640 1 --2- 172.21.15.172:0/3004046066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18300b3a30 0x7f18300b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.570+0000 7f1835d5e640 1 --2- 172.21.15.172:0/3004046066 >> v2:172.21.15.177:3300/0 conn(0x7f1830162540 0x7f18301629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.570+0000 7f1835d5e640 1 --2- 172.21.15.172:0/3004046066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f183015e430 0x7f1830162000 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.570+0000 7f1835d5e640 1 -- 172.21.15.172:0/3004046066 >> 172.21.15.172:0/3004046066 conn(0x7f183009d560 msgr2=0x7f18300d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.570+0000 7f1835d5e640 1 -- 172.21.15.172:0/3004046066 shutdown_connections 2024-04-14T08:33:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.570+0000 7f1835d5e640 1 -- 172.21.15.172:0/3004046066 wait complete. 2024-04-14T08:33:38.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.571+0000 7f1835d5e640 1 Processor -- start 2024-04-14T08:33:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.571+0000 7f1835d5e640 1 -- start start 2024-04-14T08:33:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.571+0000 7f1835d5e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18300b3a30 0x7f1830151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.571+0000 7f1835d5e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f183015e430 0x7f18300b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.571+0000 7f1835d5e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1830162540 0x7f18300b9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.571+0000 7f1835d5e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f18300d4e20 con 0x7f18300b3a30 2024-04-14T08:33:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.571+0000 7f1835d5e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f183015e810 con 0x7f1830162540 2024-04-14T08:33:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.571+0000 7f1835d5e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f18300d0050 con 0x7f183015e430 2024-04-14T08:33:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18300b3a30 0x7f1830151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18300b3a30 0x7f1830151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37410/0 (socket says 172.21.15.172:37410) 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182f7fe640 1 -- 172.21.15.172:0/2694173324 learned_addr learned my addr 172.21.15.172:0/2694173324 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f1827fff640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f183015e430 0x7f18300b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182ffff640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1830162540 0x7f18300b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182f7fe640 1 -- 172.21.15.172:0/2694173324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f183015e430 msgr2=0x7f18300b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182f7fe640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f183015e430 0x7f18300b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182f7fe640 1 -- 172.21.15.172:0/2694173324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1830162540 msgr2=0x7f18300b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182f7fe640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1830162540 0x7f18300b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182f7fe640 1 -- 172.21.15.172:0/2694173324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1830151d60 con 0x7f18300b3a30 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182f7fe640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18300b3a30 0x7f1830151820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f182002b950 tx=0x7f18200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182cff9640 1 -- 172.21.15.172:0/2694173324 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820040070 con 0x7f18300b3a30 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182cff9640 1 -- 172.21.15.172:0/2694173324 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1820031e10 con 0x7f18300b3a30 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f182cff9640 1 -- 172.21.15.172:0/2694173324 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820035040 con 0x7f18300b3a30 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f1835d5e640 1 -- 172.21.15.172:0/2694173324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18300b7750 con 0x7f18300b3a30 2024-04-14T08:33:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.572+0000 7f1835d5e640 1 -- 172.21.15.172:0/2694173324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18300b7d10 con 0x7f18300b3a30 2024-04-14T08:33:38.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.573+0000 7f1835d5e640 1 -- 172.21.15.172:0/2694173324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1830068590 con 0x7f18300b3a30 2024-04-14T08:33:38.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.574+0000 7f182cff9640 1 -- 172.21.15.172:0/2694173324 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f182004b060 con 0x7f18300b3a30 2024-04-14T08:33:38.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.574+0000 7f182cff9640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f181006c920 0x7f181006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:38.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.574+0000 7f182cff9640 1 -- 172.21.15.172:0/2694173324 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f18200ad9c0 con 0x7f18300b3a30 2024-04-14T08:33:38.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.574+0000 7f1827fff640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f181006c920 0x7f181006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:38.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.575+0000 7f1827fff640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f181006c920 0x7f181006ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f183015de10 tx=0x7f1818009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:38.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.579+0000 7f182cff9640 1 -- 172.21.15.172:0/2694173324 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f18200afe20 con 0x7f18300b3a30 2024-04-14T08:33:38.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.768+0000 7f1835d5e640 1 -- 172.21.15.172:0/2694173324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f18300d18a0 con 0x7f18300b3a30 2024-04-14T08:33:38.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.769+0000 7f182cff9640 1 -- 172.21.15.172:0/2694173324 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f182007b3a0 con 0x7f18300b3a30 2024-04-14T08:33:38.769 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:33:38.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.771+0000 7f1835d5e640 1 -- 172.21.15.172:0/2694173324 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f181006c920 msgr2=0x7f181006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.771+0000 7f1835d5e640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f181006c920 0x7f181006ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f183015de10 tx=0x7f1818009190 comp rx=0 tx=0).stop 2024-04-14T08:33:38.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.771+0000 7f1835d5e640 1 -- 172.21.15.172:0/2694173324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18300b3a30 msgr2=0x7f1830151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:38.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.771+0000 7f1835d5e640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18300b3a30 0x7f1830151820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f182002b950 tx=0x7f18200046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.771+0000 7f1835d5e640 1 -- 172.21.15.172:0/2694173324 shutdown_connections 2024-04-14T08:33:38.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.771+0000 7f1835d5e640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f181006c920 0x7f181006ede0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.771+0000 7f1835d5e640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1830162540 0x7f18300b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.771+0000 7f1835d5e640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f183015e430 0x7f18300b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.771+0000 7f1835d5e640 1 --2- 172.21.15.172:0/2694173324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18300b3a30 0x7f1830151820 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:38.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.771+0000 7f1835d5e640 1 -- 172.21.15.172:0/2694173324 >> 172.21.15.172:0/2694173324 conn(0x7f183009d560 msgr2=0x7f18300b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:38.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.771+0000 7f1835d5e640 1 -- 172.21.15.172:0/2694173324 shutdown_connections 2024-04-14T08:33:38.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:38.771+0000 7f1835d5e640 1 -- 172.21.15.172:0/2694173324 wait complete. 2024-04-14T08:33:38.783 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:33:38.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:38.783 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-14T08:33:38.812 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:33:38.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:38.812 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:33:38.867 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:33:38.868 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:38.884 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:33:38.884 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:38.940 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:33:38.940 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:38.995 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:39.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.095+0000 7f95d9ade640 1 Processor -- start 2024-04-14T08:33:39.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.096+0000 7f95d9ade640 1 -- start start 2024-04-14T08:33:39.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.096+0000 7f95d9ade640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f95d40b5b50 0x7f95d415c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.096+0000 7f95d9ade640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f95d415c990 0x7f95d40b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.096+0000 7f95d9ade640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d40addb0 con 0x7f95d406b520 2024-04-14T08:33:39.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.096+0000 7f95d9ade640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f95d40ae650 con 0x7f95d415c990 2024-04-14T08:33:39.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.096+0000 7f95d9ade640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f95d40ae090 con 0x7f95d40b5b50 2024-04-14T08:33:39.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.097+0000 7f95d3fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f95d415c990 0x7f95d40b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.097+0000 7f95d3fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f95d415c990 0x7f95d40b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37418/0 (socket says 172.21.15.172:37418) 2024-04-14T08:33:39.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.097+0000 7f95d3fff640 1 -- 172.21.15.172:0/2988226198 learned_addr learned my addr 172.21.15.172:0/2988226198 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:39.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.097+0000 7f95d2ffd640 1 --2- 172.21.15.172:0/2988226198 >> v2:172.21.15.177:3300/0 conn(0x7f95d40b5b50 0x7f95d415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.097+0000 7f95d3fff640 1 -- 172.21.15.172:0/2988226198 >> v2:172.21.15.177:3300/0 conn(0x7f95d40b5b50 msgr2=0x7f95d415c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:39.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.097+0000 7f95d3fff640 1 --2- 172.21.15.172:0/2988226198 >> v2:172.21.15.177:3300/0 conn(0x7f95d40b5b50 0x7f95d415c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.097+0000 7f95d3fff640 1 -- 172.21.15.172:0/2988226198 >> v1:172.21.15.177:6789/0 conn(0x7f95d406b520 legacy=0x7f95d406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.097+0000 7f95d3fff640 1 -- 172.21.15.172:0/2988226198 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d40ae370 con 0x7f95d415c990 2024-04-14T08:33:39.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.098+0000 7f95d3fff640 1 --2- 172.21.15.172:0/2988226198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95d415c990 0x7f95d40b4490 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f95d40b62a0 tx=0x7f95cc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256f538f49f0fe08 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:39.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.098+0000 7f95d27fc640 1 -- 172.21.15.172:0/2988226198 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc02cb20 con 0x7f95d415c990 2024-04-14T08:33:39.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.098+0000 7f95d27fc640 1 -- 172.21.15.172:0/2988226198 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95cc02ccc0 con 0x7f95d415c990 2024-04-14T08:33:39.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.098+0000 7f95d27fc640 1 -- 172.21.15.172:0/2988226198 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc034610 con 0x7f95d415c990 2024-04-14T08:33:39.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.098+0000 7f95d9ade640 1 -- 172.21.15.172:0/2988226198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95d415c990 msgr2=0x7f95d40b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.098+0000 7f95d9ade640 1 --2- 172.21.15.172:0/2988226198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95d415c990 0x7f95d40b4490 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f95d40b62a0 tx=0x7f95cc02bf80 comp rx=0 tx=0).stop 2024-04-14T08:33:39.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.098+0000 7f95d9ade640 1 -- 172.21.15.172:0/2988226198 shutdown_connections 2024-04-14T08:33:39.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.098+0000 7f95d9ade640 1 --2- 172.21.15.172:0/2988226198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95d415c990 0x7f95d40b4490 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.098+0000 7f95d9ade640 1 --2- 172.21.15.172:0/2988226198 >> v2:172.21.15.177:3300/0 conn(0x7f95d40b5b50 0x7f95d415c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.098+0000 7f95d9ade640 1 -- 172.21.15.172:0/2988226198 >> 172.21.15.172:0/2988226198 conn(0x7f95d40d0f20 msgr2=0x7f95d40d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:39.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.098+0000 7f95d9ade640 1 -- 172.21.15.172:0/2988226198 shutdown_connections 2024-04-14T08:33:39.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.098+0000 7f95d9ade640 1 -- 172.21.15.172:0/2988226198 wait complete. 2024-04-14T08:33:39.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.099+0000 7f95d9ade640 1 Processor -- start 2024-04-14T08:33:39.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.100+0000 7f95d9ade640 1 -- start start 2024-04-14T08:33:39.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.100+0000 7f95d9ade640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95d406b520 0x7f95d41f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.100+0000 7f95d9ade640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95d40b5b50 0x7f95d4153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.100+0000 7f95d9ade640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95d406e130 0x7f95d415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.100+0000 7f95d9ade640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95d409ef20 con 0x7f95d406e130 2024-04-14T08:33:39.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.100+0000 7f95d9ade640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f95d409eda0 con 0x7f95d406b520 2024-04-14T08:33:39.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.100+0000 7f95d9ade640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f95d40b5600 con 0x7f95d40b5b50 2024-04-14T08:33:39.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.100+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95d406e130 0x7f95d415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.100+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95d406e130 0x7f95d415cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37424/0 (socket says 172.21.15.172:37424) 2024-04-14T08:33:39.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.100+0000 7f95d3fff640 1 -- 172.21.15.172:0/2333426912 learned_addr learned my addr 172.21.15.172:0/2333426912 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:39.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.100+0000 7f95d2ffd640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95d40b5b50 0x7f95d4153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.100+0000 7f95d37fe640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95d406b520 0x7f95d41f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.101+0000 7f95d3fff640 1 -- 172.21.15.172:0/2333426912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95d40b5b50 msgr2=0x7f95d4153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.101+0000 7f95d3fff640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95d40b5b50 0x7f95d4153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.101+0000 7f95d3fff640 1 -- 172.21.15.172:0/2333426912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95d406b520 msgr2=0x7f95d41f7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.101+0000 7f95d3fff640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95d406b520 0x7f95d41f7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.101+0000 7f95d3fff640 1 -- 172.21.15.172:0/2333426912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d40cf6c0 con 0x7f95d406e130 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.101+0000 7f95d3fff640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95d406e130 0x7f95d415cd80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009f40 tx=0x7f95cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.101+0000 7f95bbfff640 1 -- 172.21.15.172:0/2333426912 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc043020 con 0x7f95d406e130 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.101+0000 7f95bbfff640 1 -- 172.21.15.172:0/2333426912 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95cc03e450 con 0x7f95d406e130 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.101+0000 7f95bbfff640 1 -- 172.21.15.172:0/2333426912 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc009510 con 0x7f95d406e130 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.101+0000 7f95d9ade640 1 -- 172.21.15.172:0/2333426912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d406e660 con 0x7f95d406e130 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.101+0000 7f95d9ade640 1 -- 172.21.15.172:0/2333426912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d406ec20 con 0x7f95d406e130 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.102+0000 7f95d9ade640 1 -- 172.21.15.172:0/2333426912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d4068520 con 0x7f95d406e130 2024-04-14T08:33:39.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.103+0000 7f95bbfff640 1 -- 172.21.15.172:0/2333426912 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95cc046020 con 0x7f95d406e130 2024-04-14T08:33:39.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.103+0000 7f95bbfff640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f95c006c8f0 0x7f95c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.103+0000 7f95bbfff640 1 -- 172.21.15.172:0/2333426912 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f95cc0ae590 con 0x7f95d406e130 2024-04-14T08:33:39.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.103+0000 7f95d37fe640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f95c006c8f0 0x7f95c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.103+0000 7f95d37fe640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f95c006c8f0 0x7f95c006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f95c4001780 tx=0x7f95c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:39.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.108+0000 7f95bbfff640 1 -- 172.21.15.172:0/2333426912 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f95cc04d030 con 0x7f95d406e130 2024-04-14T08:33:39.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.289+0000 7f95d9ade640 1 -- 172.21.15.172:0/2333426912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95d415c660 con 0x7f95d406e130 2024-04-14T08:33:39.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.290+0000 7f95bbfff640 1 -- 172.21.15.172:0/2333426912 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18962 (secure 0 0 0) 0x7f95cc07bf70 con 0x7f95d406e130 2024-04-14T08:33:39.291 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:39.291 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":27,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:33:34.330426+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:39.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.292+0000 7f95d9ade640 1 -- 172.21.15.172:0/2333426912 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f95c006c8f0 msgr2=0x7f95c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.292+0000 7f95d9ade640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f95c006c8f0 0x7f95c006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f95c4001780 tx=0x7f95c4009040 comp rx=0 tx=0).stop 2024-04-14T08:33:39.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.292+0000 7f95d9ade640 1 -- 172.21.15.172:0/2333426912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95d406e130 msgr2=0x7f95d415cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.292+0000 7f95d9ade640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95d406e130 0x7f95d415cd80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009f40 tx=0x7f95cc004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:39.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.292+0000 7f95d9ade640 1 -- 172.21.15.172:0/2333426912 shutdown_connections 2024-04-14T08:33:39.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.292+0000 7f95d9ade640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f95c006c8f0 0x7f95c006edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.292+0000 7f95d9ade640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95d406e130 0x7f95d415cd80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.292+0000 7f95d9ade640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95d40b5b50 0x7f95d4153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.292+0000 7f95d9ade640 1 --2- 172.21.15.172:0/2333426912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95d406b520 0x7f95d41f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.292+0000 7f95d9ade640 1 -- 172.21.15.172:0/2333426912 >> 172.21.15.172:0/2333426912 conn(0x7f95d40d0f20 msgr2=0x7f95d40b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:39.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.292+0000 7f95d9ade640 1 -- 172.21.15.172:0/2333426912 shutdown_connections 2024-04-14T08:33:39.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.293+0000 7f95d9ade640 1 -- 172.21.15.172:0/2333426912 wait complete. 2024-04-14T08:33:39.304 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:39.305 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:39.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.408+0000 7f5ad7f7b640 1 Processor -- start 2024-04-14T08:33:39.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.409+0000 7f5ad7f7b640 1 -- start start 2024-04-14T08:33:39.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.409+0000 7f5ad7f7b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5ad015c380 0x7f5ad0140e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.409+0000 7f5ad7f7b640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad015b620 con 0x7f5ad00b23e0 2024-04-14T08:33:39.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.409+0000 7f5ad7f7b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad01414a0 con 0x7f5ad013df30 2024-04-14T08:33:39.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.409+0000 7f5ad7f7b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad00cf9b0 con 0x7f5ad015c380 2024-04-14T08:33:39.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.410+0000 7f5ad5516640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5ad00b23e0 0x7f5ad0140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:46564/0 (socket says 172.21.15.172:46564) 2024-04-14T08:33:39.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.410+0000 7f5ad5516640 1 -- 172.21.15.172:0/1107991206 learned_addr learned my addr 172.21.15.172:0/1107991206 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:39.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.410+0000 7f5ad6518640 1 --2- 172.21.15.172:0/1107991206 >> v2:172.21.15.177:3300/0 conn(0x7f5ad015c380 0x7f5ad0140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.410+0000 7f5ad4d15640 1 -- 172.21.15.172:0/1107991206 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3780073381 0 0) 0x7f5ac4001110 con 0x7f5ad00b23e0 2024-04-14T08:33:39.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.410+0000 7f5ad4d15640 1 -- 172.21.15.172:0/1107991206 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ab8003520 con 0x7f5ad00b23e0 2024-04-14T08:33:39.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.410+0000 7f5ad4d15640 1 -- 172.21.15.172:0/1107991206 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 908320269 0 0) 0x7f5ad01414a0 con 0x7f5ad013df30 2024-04-14T08:33:39.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.410+0000 7f5ad4d15640 1 -- 172.21.15.172:0/1107991206 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ac4001110 con 0x7f5ad013df30 2024-04-14T08:33:39.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.410+0000 7f5ad6518640 1 -- 172.21.15.172:0/1107991206 >> v1:172.21.15.177:6789/0 conn(0x7f5ad013df30 legacy=0x7f5ad00b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.410+0000 7f5ad6518640 1 -- 172.21.15.172:0/1107991206 >> v1:172.21.15.172:6789/0 conn(0x7f5ad00b23e0 legacy=0x7f5ad0140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.410+0000 7f5ad6518640 1 -- 172.21.15.172:0/1107991206 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad015b8a0 con 0x7f5ad015c380 2024-04-14T08:33:39.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.411+0000 7f5ad6518640 1 --2- 172.21.15.172:0/1107991206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ad015c380 0x7f5ad0140e80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00097a0 tx=0x7f5ac002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=884c70e583e06a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:39.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.411+0000 7f5ad4d15640 1 -- 172.21.15.172:0/1107991206 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0034070 con 0x7f5ad015c380 2024-04-14T08:33:39.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.411+0000 7f5ad4d15640 1 -- 172.21.15.172:0/1107991206 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac0030610 con 0x7f5ad015c380 2024-04-14T08:33:39.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.411+0000 7f5ad4d15640 1 -- 172.21.15.172:0/1107991206 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00309e0 con 0x7f5ad015c380 2024-04-14T08:33:39.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.411+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/1107991206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ad015c380 msgr2=0x7f5ad0140e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.411+0000 7f5ad7f7b640 1 --2- 172.21.15.172:0/1107991206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ad015c380 0x7f5ad0140e80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00097a0 tx=0x7f5ac002b960 comp rx=0 tx=0).stop 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.412+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/1107991206 shutdown_connections 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.412+0000 7f5ad7f7b640 1 --2- 172.21.15.172:0/1107991206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ad015c380 0x7f5ad0140e80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.412+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/1107991206 >> 172.21.15.172:0/1107991206 conn(0x7f5ad006b520 msgr2=0x7f5ad006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.412+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/1107991206 shutdown_connections 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.412+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/1107991206 wait complete. 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.413+0000 7f5ad7f7b640 1 Processor -- start 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.413+0000 7f5ad7f7b640 1 -- start start 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.413+0000 7f5ad7f7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ad00b23e0 0x7f5ad015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.413+0000 7f5ad7f7b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ad013df30 0x7f5ad013c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.413+0000 7f5ad7f7b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ad015c380 0x7f5ad0137780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.413+0000 7f5ad7f7b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad013e310 con 0x7f5ad00b23e0 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.413+0000 7f5ad7f7b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad015be60 con 0x7f5ad015c380 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.413+0000 7f5ad7f7b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad0165fa0 con 0x7f5ad013df30 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad5d17640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ad00b23e0 0x7f5ad015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad5d17640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ad00b23e0 0x7f5ad015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37426/0 (socket says 172.21.15.172:37426) 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad5d17640 1 -- 172.21.15.172:0/3624271197 learned_addr learned my addr 172.21.15.172:0/3624271197 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad6518640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ad015c380 0x7f5ad0137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad5516640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ad013df30 0x7f5ad013c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad5d17640 1 -- 172.21.15.172:0/3624271197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ad013df30 msgr2=0x7f5ad013c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad5d17640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ad013df30 0x7f5ad013c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad5d17640 1 -- 172.21.15.172:0/3624271197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ad015c380 msgr2=0x7f5ad0137780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad5d17640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ad015c380 0x7f5ad0137780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad5d17640 1 -- 172.21.15.172:0/3624271197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad013c560 con 0x7f5ad00b23e0 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad6518640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ad015c380 0x7f5ad0137780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad5d17640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ad00b23e0 0x7f5ad015d5d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800c8a0 tx=0x7f5ac800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ace7fc640 1 -- 172.21.15.172:0/3624271197 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8003b70 con 0x7f5ad00b23e0 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ace7fc640 1 -- 172.21.15.172:0/3624271197 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac8003d30 con 0x7f5ad00b23e0 2024-04-14T08:33:39.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ace7fc640 1 -- 172.21.15.172:0/3624271197 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac800aa90 con 0x7f5ad00b23e0 2024-04-14T08:33:39.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/3624271197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad013c810 con 0x7f5ad00b23e0 2024-04-14T08:33:39.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.414+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/3624271197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad013cdd0 con 0x7f5ad00b23e0 2024-04-14T08:33:39.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.415+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/3624271197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad0068590 con 0x7f5ad00b23e0 2024-04-14T08:33:39.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.416+0000 7f5ace7fc640 1 -- 172.21.15.172:0/3624271197 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ac800ac30 con 0x7f5ad00b23e0 2024-04-14T08:33:39.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.416+0000 7f5ace7fc640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5abc06c8f0 0x7f5abc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.416+0000 7f5ace7fc640 1 -- 172.21.15.172:0/3624271197 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5ac808bd60 con 0x7f5ad00b23e0 2024-04-14T08:33:39.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.416+0000 7f5ad5516640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5abc06c8f0 0x7f5abc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.417+0000 7f5ad5516640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5abc06c8f0 0x7f5abc06edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5ad01f77f0 tx=0x7f5ac4007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:39.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.420+0000 7f5ace7fc640 1 -- 172.21.15.172:0/3624271197 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5ac80597d0 con 0x7f5ad00b23e0 2024-04-14T08:33:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.592+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/3624271197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ad00b1540 con 0x7f5ad00b23e0 2024-04-14T08:33:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.592+0000 7f5ace7fc640 1 -- 172.21.15.172:0/3624271197 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f5ac805dad0 con 0x7f5ad00b23e0 2024-04-14T08:33:39.593 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:39.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.594+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/3624271197 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5abc06c8f0 msgr2=0x7f5abc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.594+0000 7f5ad7f7b640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5abc06c8f0 0x7f5abc06edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5ad01f77f0 tx=0x7f5ac4007d30 comp rx=0 tx=0).stop 2024-04-14T08:33:39.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.594+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/3624271197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ad00b23e0 msgr2=0x7f5ad015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.594+0000 7f5ad7f7b640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ad00b23e0 0x7f5ad015d5d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800c8a0 tx=0x7f5ac800cd60 comp rx=0 tx=0).stop 2024-04-14T08:33:39.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.594+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/3624271197 shutdown_connections 2024-04-14T08:33:39.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.594+0000 7f5ad7f7b640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5abc06c8f0 0x7f5abc06edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.594+0000 7f5ad7f7b640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ad015c380 0x7f5ad0137780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.594+0000 7f5ad7f7b640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ad013df30 0x7f5ad013c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.594+0000 7f5ad7f7b640 1 --2- 172.21.15.172:0/3624271197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ad00b23e0 0x7f5ad015d5d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.594+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/3624271197 >> 172.21.15.172:0/3624271197 conn(0x7f5ad006b520 msgr2=0x7f5ad0159240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:39.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.594+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/3624271197 shutdown_connections 2024-04-14T08:33:39.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.594+0000 7f5ad7f7b640 1 -- 172.21.15.172:0/3624271197 wait complete. 2024-04-14T08:33:39.606 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:39.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.708+0000 7fcb744d1640 1 Processor -- start 2024-04-14T08:33:39.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.708+0000 7fcb744d1640 1 -- start start 2024-04-14T08:33:39.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.709+0000 7fcb744d1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb6c0d0cc0 0x7fcb6c0d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.709+0000 7fcb744d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6c0d4e50 0x7fcb6c161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.709+0000 7fcb744d1640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb6c0d15e0 con 0x7fcb6c0d49c0 2024-04-14T08:33:39.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.709+0000 7fcb744d1640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fcb6c0ceec0 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.709+0000 7fcb744d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcb6c0cfd00 con 0x7fcb6c0d4e50 2024-04-14T08:33:39.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.710+0000 7fcb7226d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb6c0d0cc0 0x7fcb6c0d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.710+0000 7fcb72a6e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6c0d4e50 0x7fcb6c161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.710+0000 7fcb71a6c640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fcb6c0d49c0 0x7fcb6c160f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:51290/0 (socket says 172.21.15.172:51290) 2024-04-14T08:33:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.710+0000 7fcb71a6c640 1 -- 172.21.15.172:0/93234536 learned_addr learned my addr 172.21.15.172:0/93234536 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.710+0000 7fcb7226d640 1 --2- 172.21.15.172:0/93234536 >> v2:172.21.15.172:3300/0 conn(0x7fcb6c0d0cc0 0x7fcb6c0d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59542/0 (socket says 172.21.15.172:59542) 2024-04-14T08:33:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.710+0000 7fcb72a6e640 1 -- 172.21.15.172:0/93234536 >> v2:172.21.15.172:3300/0 conn(0x7fcb6c0d0cc0 msgr2=0x7fcb6c0d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.710+0000 7fcb72a6e640 1 --2- 172.21.15.172:0/93234536 >> v2:172.21.15.172:3300/0 conn(0x7fcb6c0d0cc0 0x7fcb6c0d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.710+0000 7fcb72a6e640 1 -- 172.21.15.172:0/93234536 >> v1:172.21.15.177:6789/0 conn(0x7fcb6c0d49c0 legacy=0x7fcb6c160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.710+0000 7fcb72a6e640 1 -- 172.21.15.172:0/93234536 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6c0cf0f0 con 0x7fcb6c0d4e50 2024-04-14T08:33:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.711+0000 7fcb72a6e640 1 --2- 172.21.15.172:0/93234536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6c0d4e50 0x7fcb6c161640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009d90 tx=0x7fcb6402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c877a4892c1f26 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.711+0000 7fcb7126b640 1 -- 172.21.15.172:0/93234536 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64039070 con 0x7fcb6c0d4e50 2024-04-14T08:33:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.711+0000 7fcb7126b640 1 -- 172.21.15.172:0/93234536 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb6402cc50 con 0x7fcb6c0d4e50 2024-04-14T08:33:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.711+0000 7fcb7126b640 1 -- 172.21.15.172:0/93234536 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64034440 con 0x7fcb6c0d4e50 2024-04-14T08:33:39.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.711+0000 7fcb744d1640 1 -- 172.21.15.172:0/93234536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6c0d4e50 msgr2=0x7fcb6c161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.711+0000 7fcb744d1640 1 --2- 172.21.15.172:0/93234536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6c0d4e50 0x7fcb6c161640 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009d90 tx=0x7fcb6402bea0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.711+0000 7fcb744d1640 1 -- 172.21.15.172:0/93234536 shutdown_connections 2024-04-14T08:33:39.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.711+0000 7fcb744d1640 1 --2- 172.21.15.172:0/93234536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6c0d4e50 0x7fcb6c161640 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.711+0000 7fcb744d1640 1 --2- 172.21.15.172:0/93234536 >> v2:172.21.15.172:3300/0 conn(0x7fcb6c0d0cc0 0x7fcb6c0d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.711+0000 7fcb744d1640 1 -- 172.21.15.172:0/93234536 >> 172.21.15.172:0/93234536 conn(0x7fcb6c09d580 msgr2=0x7fcb6c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:39.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.711+0000 7fcb744d1640 1 -- 172.21.15.172:0/93234536 shutdown_connections 2024-04-14T08:33:39.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.711+0000 7fcb744d1640 1 -- 172.21.15.172:0/93234536 wait complete. 2024-04-14T08:33:39.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb744d1640 1 Processor -- start 2024-04-14T08:33:39.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb744d1640 1 -- start start 2024-04-14T08:33:39.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb744d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6c0d0cc0 0x7fcb6c1e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb744d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6c0d49c0 0x7fcb6c1e6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb744d1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb6c0d4e50 0x7fcb6c0cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb744d1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c0cf990 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb744d1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c0cf6b0 con 0x7fcb6c0d4e50 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb744d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcb6c0d32b0 con 0x7fcb6c0d49c0 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb7226d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6c0d0cc0 0x7fcb6c1e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb7226d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6c0d0cc0 0x7fcb6c1e7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59554/0 (socket says 172.21.15.172:59554) 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb7226d640 1 -- 172.21.15.172:0/1822039086 learned_addr learned my addr 172.21.15.172:0/1822039086 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb71a6c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6c0d49c0 0x7fcb6c1e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.713+0000 7fcb72a6e640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb6c0d4e50 0x7fcb6c0cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.714+0000 7fcb7226d640 1 -- 172.21.15.172:0/1822039086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6c0d49c0 msgr2=0x7fcb6c1e6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.714+0000 7fcb7226d640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6c0d49c0 0x7fcb6c1e6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.714+0000 7fcb7226d640 1 -- 172.21.15.172:0/1822039086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb6c0d4e50 msgr2=0x7fcb6c0cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.714+0000 7fcb7226d640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb6c0d4e50 0x7fcb6c0cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.714+0000 7fcb7226d640 1 -- 172.21.15.172:0/1822039086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6c1e6b40 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.714+0000 7fcb71a6c640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6c0d49c0 0x7fcb6c1e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:39.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.714+0000 7fcb7226d640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6c0d0cc0 0x7fcb6c1e7f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800e9b0 tx=0x7fcb6800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:39.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.714+0000 7fcb62ffd640 1 -- 172.21.15.172:0/1822039086 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68017070 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.714+0000 7fcb62ffd640 1 -- 172.21.15.172:0/1822039086 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb6800c9a0 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.714+0000 7fcb744d1640 1 -- 172.21.15.172:0/1822039086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6c1e6e20 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.714+0000 7fcb62ffd640 1 -- 172.21.15.172:0/1822039086 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800cd90 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.714+0000 7fcb744d1640 1 -- 172.21.15.172:0/1822039086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb6c1e73e0 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.715+0000 7fcb744d1640 1 -- 172.21.15.172:0/1822039086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb6c075490 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.715+0000 7fcb62ffd640 1 -- 172.21.15.172:0/1822039086 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb6801a070 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.716+0000 7fcb62ffd640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb5006c8f0 0x7fcb5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:39.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.716+0000 7fcb62ffd640 1 -- 172.21.15.172:0/1822039086 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcb6808c210 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.716+0000 7fcb71a6c640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb5006c8f0 0x7fcb5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:39.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.716+0000 7fcb71a6c640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb5006c8f0 0x7fcb5006edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c06e0c0 tx=0x7fcb5c007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:39.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.720+0000 7fcb62ffd640 1 -- 172.21.15.172:0/1822039086 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcb68059bf0 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.895+0000 7fcb744d1640 1 -- 172.21.15.172:0/1822039086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcb6c0d0aa0 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.895+0000 7fcb62ffd640 1 -- 172.21.15.172:0/1822039086 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fcb6805def0 con 0x7fcb6c0d0cc0 2024-04-14T08:33:39.896 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:39.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.897+0000 7fcb744d1640 1 -- 172.21.15.172:0/1822039086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb5006c8f0 msgr2=0x7fcb5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.897+0000 7fcb744d1640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb5006c8f0 0x7fcb5006edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c06e0c0 tx=0x7fcb5c007e80 comp rx=0 tx=0).stop 2024-04-14T08:33:39.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.897+0000 7fcb744d1640 1 -- 172.21.15.172:0/1822039086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6c0d0cc0 msgr2=0x7fcb6c1e7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:39.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.897+0000 7fcb744d1640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6c0d0cc0 0x7fcb6c1e7f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800e9b0 tx=0x7fcb6800ee70 comp rx=0 tx=0).stop 2024-04-14T08:33:39.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.897+0000 7fcb744d1640 1 -- 172.21.15.172:0/1822039086 shutdown_connections 2024-04-14T08:33:39.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.897+0000 7fcb744d1640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb5006c8f0 0x7fcb5006edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.897+0000 7fcb744d1640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb6c0d4e50 0x7fcb6c0cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.897+0000 7fcb744d1640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6c0d49c0 0x7fcb6c1e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.897+0000 7fcb744d1640 1 --2- 172.21.15.172:0/1822039086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6c0d0cc0 0x7fcb6c1e7f30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:39.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.897+0000 7fcb744d1640 1 -- 172.21.15.172:0/1822039086 >> 172.21.15.172:0/1822039086 conn(0x7fcb6c09d580 msgr2=0x7fcb6c1e5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:39.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.897+0000 7fcb744d1640 1 -- 172.21.15.172:0/1822039086 shutdown_connections 2024-04-14T08:33:39.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:39.897+0000 7fcb744d1640 1 -- 172.21.15.172:0/1822039086 wait complete. 2024-04-14T08:33:39.909 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:40.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.012+0000 7fc54b1c8640 1 Processor -- start 2024-04-14T08:33:40.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.012+0000 7fc54b1c8640 1 -- start start 2024-04-14T08:33:40.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.013+0000 7fc54b1c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc544162ba0 0x7fc5440d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.013+0000 7fc54b1c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5440cefb0 con 0x7fc544162710 2024-04-14T08:33:40.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.013+0000 7fc54b1c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5440cf290 con 0x7fc54415e430 2024-04-14T08:33:40.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.013+0000 7fc54b1c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc5440cf570 con 0x7fc544162ba0 2024-04-14T08:33:40.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.013+0000 7fc543fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc544162710 0x7fc5440d1df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54066/0 (socket says 172.21.15.172:54066) 2024-04-14T08:33:40.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.013+0000 7fc543fff640 1 -- 172.21.15.172:0/554263373 learned_addr learned my addr 172.21.15.172:0/554263373 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:40.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.013+0000 7fc549765640 1 --2- 172.21.15.172:0/554263373 >> v2:172.21.15.177:3300/0 conn(0x7fc544162ba0 0x7fc5440d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.014+0000 7fc5437fe640 1 -- 172.21.15.172:0/554263373 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3535363592 0 0) 0x7fc5440cefb0 con 0x7fc544162710 2024-04-14T08:33:40.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.014+0000 7fc5437fe640 1 -- 172.21.15.172:0/554263373 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc52c003520 con 0x7fc544162710 2024-04-14T08:33:40.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.014+0000 7fc5437fe640 1 -- 172.21.15.172:0/554263373 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3113883015 0 0) 0x7fc5440cf290 con 0x7fc54415e430 2024-04-14T08:33:40.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.014+0000 7fc5437fe640 1 -- 172.21.15.172:0/554263373 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5440cefb0 con 0x7fc54415e430 2024-04-14T08:33:40.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.014+0000 7fc549765640 1 -- 172.21.15.172:0/554263373 >> v1:172.21.15.177:6789/0 conn(0x7fc54415e430 legacy=0x7fc544162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.014+0000 7fc549765640 1 -- 172.21.15.172:0/554263373 >> v1:172.21.15.172:6789/0 conn(0x7fc544162710 legacy=0x7fc5440d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.014+0000 7fc549765640 1 -- 172.21.15.172:0/554263373 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5440cf850 con 0x7fc544162ba0 2024-04-14T08:33:40.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.014+0000 7fc549765640 1 --2- 172.21.15.172:0/554263373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc544162ba0 0x7fc5440d2500 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009d90 tx=0x7fc53c02bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f68d1a367937464a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:40.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.015+0000 7fc5437fe640 1 -- 172.21.15.172:0/554263373 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c039070 con 0x7fc544162ba0 2024-04-14T08:33:40.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.015+0000 7fc5437fe640 1 -- 172.21.15.172:0/554263373 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc53c02cd00 con 0x7fc544162ba0 2024-04-14T08:33:40.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.015+0000 7fc5437fe640 1 -- 172.21.15.172:0/554263373 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c034660 con 0x7fc544162ba0 2024-04-14T08:33:40.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.015+0000 7fc54b1c8640 1 -- 172.21.15.172:0/554263373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc544162ba0 msgr2=0x7fc5440d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.015+0000 7fc54b1c8640 1 --2- 172.21.15.172:0/554263373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc544162ba0 0x7fc5440d2500 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009d90 tx=0x7fc53c02bf50 comp rx=0 tx=0).stop 2024-04-14T08:33:40.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.015+0000 7fc54b1c8640 1 -- 172.21.15.172:0/554263373 shutdown_connections 2024-04-14T08:33:40.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.015+0000 7fc54b1c8640 1 --2- 172.21.15.172:0/554263373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc544162ba0 0x7fc5440d2500 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.015+0000 7fc54b1c8640 1 -- 172.21.15.172:0/554263373 >> 172.21.15.172:0/554263373 conn(0x7fc54409d560 msgr2=0x7fc5440d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:40.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.015+0000 7fc54b1c8640 1 -- 172.21.15.172:0/554263373 shutdown_connections 2024-04-14T08:33:40.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.015+0000 7fc54b1c8640 1 -- 172.21.15.172:0/554263373 wait complete. 2024-04-14T08:33:40.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.016+0000 7fc54b1c8640 1 Processor -- start 2024-04-14T08:33:40.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.016+0000 7fc54b1c8640 1 -- start start 2024-04-14T08:33:40.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.017+0000 7fc54b1c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54415e430 0x7fc5440b9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.017+0000 7fc54b1c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc544162710 0x7fc54414a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.017+0000 7fc54b1c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc544162ba0 0x7fc5440ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.017+0000 7fc54b1c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5440d4e20 con 0x7fc54415e430 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.017+0000 7fc54b1c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc54415e810 con 0x7fc544162ba0 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.017+0000 7fc54b1c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc5440d0050 con 0x7fc544162710 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.017+0000 7fc548f64640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54415e430 0x7fc5440b9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.017+0000 7fc548f64640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54415e430 0x7fc5440b9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59564/0 (socket says 172.21.15.172:59564) 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.017+0000 7fc548f64640 1 -- 172.21.15.172:0/2630479215 learned_addr learned my addr 172.21.15.172:0/2630479215 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.017+0000 7fc549765640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc544162ba0 0x7fc5440ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.017+0000 7fc543fff640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc544162710 0x7fc54414a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.018+0000 7fc548f64640 1 -- 172.21.15.172:0/2630479215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc544162710 msgr2=0x7fc54414a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.018+0000 7fc548f64640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc544162710 0x7fc54414a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.018+0000 7fc548f64640 1 -- 172.21.15.172:0/2630479215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc544162ba0 msgr2=0x7fc5440ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.018+0000 7fc548f64640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc544162ba0 0x7fc5440ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.018+0000 7fc548f64640 1 -- 172.21.15.172:0/2630479215 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54414ab60 con 0x7fc54415e430 2024-04-14T08:33:40.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.018+0000 7fc548f64640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54415e430 0x7fc5440b9740 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c430 tx=0x7fc53400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:40.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.018+0000 7fc5417fa640 1 -- 172.21.15.172:0/2630479215 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534017070 con 0x7fc54415e430 2024-04-14T08:33:40.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.018+0000 7fc5417fa640 1 -- 172.21.15.172:0/2630479215 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5340052f0 con 0x7fc54415e430 2024-04-14T08:33:40.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.018+0000 7fc54b1c8640 1 -- 172.21.15.172:0/2630479215 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5440c95b0 con 0x7fc54415e430 2024-04-14T08:33:40.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.018+0000 7fc5417fa640 1 -- 172.21.15.172:0/2630479215 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534003830 con 0x7fc54415e430 2024-04-14T08:33:40.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.018+0000 7fc54b1c8640 1 -- 172.21.15.172:0/2630479215 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5440c9b70 con 0x7fc54415e430 2024-04-14T08:33:40.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.019+0000 7fc54b1c8640 1 -- 172.21.15.172:0/2630479215 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc544068590 con 0x7fc54415e430 2024-04-14T08:33:40.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.020+0000 7fc5417fa640 1 -- 172.21.15.172:0/2630479215 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc53401a070 con 0x7fc54415e430 2024-04-14T08:33:40.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.020+0000 7fc5417fa640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc53006c8f0 0x7fc53006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.020+0000 7fc5417fa640 1 -- 172.21.15.172:0/2630479215 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc53408b250 con 0x7fc54415e430 2024-04-14T08:33:40.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.020+0000 7fc543fff640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc53006c8f0 0x7fc53006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.020+0000 7fc543fff640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc53006c8f0 0x7fc53006edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc5441f7820 tx=0x7fc538007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:40.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.024+0000 7fc5417fa640 1 -- 172.21.15.172:0/2630479215 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc54414ab60 con 0x7fc54415e430 2024-04-14T08:33:40.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.201+0000 7fc54b1c8640 1 -- 172.21.15.172:0/2630479215 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc5440d18a0 con 0x7fc54415e430 2024-04-14T08:33:40.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.202+0000 7fc5417fa640 1 -- 172.21.15.172:0/2630479215 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fc534058c40 con 0x7fc54415e430 2024-04-14T08:33:40.202 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:40.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.203+0000 7fc54b1c8640 1 -- 172.21.15.172:0/2630479215 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc53006c8f0 msgr2=0x7fc53006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.203+0000 7fc54b1c8640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc53006c8f0 0x7fc53006edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc5441f7820 tx=0x7fc538007d30 comp rx=0 tx=0).stop 2024-04-14T08:33:40.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.204+0000 7fc54b1c8640 1 -- 172.21.15.172:0/2630479215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54415e430 msgr2=0x7fc5440b9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.204+0000 7fc54b1c8640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54415e430 0x7fc5440b9740 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c430 tx=0x7fc53400c8f0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.204+0000 7fc54b1c8640 1 -- 172.21.15.172:0/2630479215 shutdown_connections 2024-04-14T08:33:40.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.204+0000 7fc54b1c8640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc53006c8f0 0x7fc53006edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.204+0000 7fc54b1c8640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc544162ba0 0x7fc5440ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.204+0000 7fc54b1c8640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc544162710 0x7fc54414a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.204+0000 7fc54b1c8640 1 --2- 172.21.15.172:0/2630479215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54415e430 0x7fc5440b9740 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.204+0000 7fc54b1c8640 1 -- 172.21.15.172:0/2630479215 >> 172.21.15.172:0/2630479215 conn(0x7fc54409d560 msgr2=0x7fc54415c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:40.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.204+0000 7fc54b1c8640 1 -- 172.21.15.172:0/2630479215 shutdown_connections 2024-04-14T08:33:40.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.204+0000 7fc54b1c8640 1 -- 172.21.15.172:0/2630479215 wait complete. 2024-04-14T08:33:40.215 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:40.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.319+0000 7fadacb1e640 1 Processor -- start 2024-04-14T08:33:40.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.319+0000 7fadacb1e640 1 -- start start 2024-04-14T08:33:40.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fadacb1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fada80ccc40 0x7fada80cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fadacb1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fada815e8a0 0x7fada81621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fadacb1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada80cfa70 con 0x7fada80b28a0 2024-04-14T08:33:40.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fadacb1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fada80cde00 con 0x7fada815e8a0 2024-04-14T08:33:40.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fadacb1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fada80cd560 con 0x7fada80ccc40 2024-04-14T08:33:40.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fada6d9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fada80b28a0 0x7fada81626f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54082/0 (socket says 172.21.15.172:54082) 2024-04-14T08:33:40.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fada659c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fada80ccc40 0x7fada80cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fada6d9d640 1 -- 172.21.15.172:0/2645625948 learned_addr learned my addr 172.21.15.172:0/2645625948 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:40.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fada5d9b640 1 --2- 172.21.15.172:0/2645625948 >> v2:172.21.15.177:3300/0 conn(0x7fada815e8a0 0x7fada81621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fada559a640 1 -- 172.21.15.172:0/2645625948 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 51635267 0 0) 0x7fada80cfa70 con 0x7fada80b28a0 2024-04-14T08:33:40.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fada659c640 1 -- 172.21.15.172:0/2645625948 >> v2:172.21.15.177:3300/0 conn(0x7fada815e8a0 msgr2=0x7fada81621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:40.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fada659c640 1 --2- 172.21.15.172:0/2645625948 >> v2:172.21.15.177:3300/0 conn(0x7fada815e8a0 0x7fada81621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fada659c640 1 -- 172.21.15.172:0/2645625948 >> v1:172.21.15.172:6789/0 conn(0x7fada80b28a0 legacy=0x7fada81626f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.320+0000 7fada659c640 1 -- 172.21.15.172:0/2645625948 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada80ce050 con 0x7fada80ccc40 2024-04-14T08:33:40.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.321+0000 7fada659c640 1 --2- 172.21.15.172:0/2645625948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fada80ccc40 0x7fada80cd020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fada80cdf80 tx=0x7fad9802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5771b2b37550e2ae server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:40.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.321+0000 7fada559a640 1 -- 172.21.15.172:0/2645625948 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98035020 con 0x7fada80ccc40 2024-04-14T08:33:40.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.321+0000 7fada559a640 1 -- 172.21.15.172:0/2645625948 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad98031890 con 0x7fada80ccc40 2024-04-14T08:33:40.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.321+0000 7fada559a640 1 -- 172.21.15.172:0/2645625948 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98031c60 con 0x7fada80ccc40 2024-04-14T08:33:40.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.322+0000 7fadacb1e640 1 -- 172.21.15.172:0/2645625948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fada80ccc40 msgr2=0x7fada80cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.322+0000 7fadacb1e640 1 --2- 172.21.15.172:0/2645625948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fada80ccc40 0x7fada80cd020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fada80cdf80 tx=0x7fad9802b930 comp rx=0 tx=0).stop 2024-04-14T08:33:40.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.322+0000 7fadacb1e640 1 -- 172.21.15.172:0/2645625948 shutdown_connections 2024-04-14T08:33:40.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.322+0000 7fadacb1e640 1 --2- 172.21.15.172:0/2645625948 >> v2:172.21.15.177:3300/0 conn(0x7fada815e8a0 0x7fada81621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.322+0000 7fadacb1e640 1 --2- 172.21.15.172:0/2645625948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fada80ccc40 0x7fada80cd020 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.322+0000 7fadacb1e640 1 -- 172.21.15.172:0/2645625948 >> 172.21.15.172:0/2645625948 conn(0x7fada80b1d80 msgr2=0x7fada80b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:40.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.322+0000 7fadacb1e640 1 -- 172.21.15.172:0/2645625948 shutdown_connections 2024-04-14T08:33:40.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.322+0000 7fadacb1e640 1 -- 172.21.15.172:0/2645625948 wait complete. 2024-04-14T08:33:40.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.323+0000 7fadacb1e640 1 Processor -- start 2024-04-14T08:33:40.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.323+0000 7fadacb1e640 1 -- start start 2024-04-14T08:33:40.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fadacb1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fada80b28a0 0x7fada81f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fadacb1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fada80ccc40 0x7fada81f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fadacb1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fada815e8a0 0x7fada8148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fadacb1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fada80ce330 con 0x7fada80b28a0 2024-04-14T08:33:40.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fadacb1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fada806cd50 con 0x7fada80ccc40 2024-04-14T08:33:40.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fadacb1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fada80ce4b0 con 0x7fada815e8a0 2024-04-14T08:33:40.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fada659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fada80b28a0 0x7fada81f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fada659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fada80b28a0 0x7fada81f3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59580/0 (socket says 172.21.15.172:59580) 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fada659c640 1 -- 172.21.15.172:0/2026386086 learned_addr learned my addr 172.21.15.172:0/2026386086 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fada5d9b640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fada80ccc40 0x7fada81f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fada6d9d640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fada815e8a0 0x7fada8148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fada659c640 1 -- 172.21.15.172:0/2026386086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fada815e8a0 msgr2=0x7fada8148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fada659c640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fada815e8a0 0x7fada8148d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fada659c640 1 -- 172.21.15.172:0/2026386086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fada80ccc40 msgr2=0x7fada81f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fada659c640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fada80ccc40 0x7fada81f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fada659c640 1 -- 172.21.15.172:0/2026386086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada815c5a0 con 0x7fada80b28a0 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fada659c640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fada80b28a0 0x7fada81f3aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fad98009890 tx=0x7fad980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fad96ffd640 1 -- 172.21.15.172:0/2026386086 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98035040 con 0x7fada80b28a0 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fad96ffd640 1 -- 172.21.15.172:0/2026386086 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad98030d20 con 0x7fada80b28a0 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.324+0000 7fadacb1e640 1 -- 172.21.15.172:0/2026386086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada815a8a0 con 0x7fada80b28a0 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.325+0000 7fad96ffd640 1 -- 172.21.15.172:0/2026386086 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad980466c0 con 0x7fada80b28a0 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.325+0000 7fadacb1e640 1 -- 172.21.15.172:0/2026386086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada815c840 con 0x7fada80b28a0 2024-04-14T08:33:40.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.326+0000 7fadacb1e640 1 -- 172.21.15.172:0/2026386086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada80c6db0 con 0x7fada80b28a0 2024-04-14T08:33:40.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.326+0000 7fad96ffd640 1 -- 172.21.15.172:0/2026386086 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad98038020 con 0x7fada80b28a0 2024-04-14T08:33:40.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.326+0000 7fad96ffd640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fad8806c8f0 0x7fad8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.327+0000 7fada5d9b640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fad8806c8f0 0x7fad8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.327+0000 7fad96ffd640 1 -- 172.21.15.172:0/2026386086 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fad980ae510 con 0x7fada80b28a0 2024-04-14T08:33:40.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.327+0000 7fada5d9b640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fad8806c8f0 0x7fad8806edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fada80b5ee0 tx=0x7fad9c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:40.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.331+0000 7fad96ffd640 1 -- 172.21.15.172:0/2026386086 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fada80c6db0 con 0x7fada80b28a0 2024-04-14T08:33:40.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.507+0000 7fadacb1e640 1 -- 172.21.15.172:0/2026386086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fada81f4520 con 0x7fada80b28a0 2024-04-14T08:33:40.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.507+0000 7fad96ffd640 1 -- 172.21.15.172:0/2026386086 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fad9802d040 con 0x7fada80b28a0 2024-04-14T08:33:40.508 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:40.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.509+0000 7fadacb1e640 1 -- 172.21.15.172:0/2026386086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fad8806c8f0 msgr2=0x7fad8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.509+0000 7fadacb1e640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fad8806c8f0 0x7fad8806edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fada80b5ee0 tx=0x7fad9c009190 comp rx=0 tx=0).stop 2024-04-14T08:33:40.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.509+0000 7fadacb1e640 1 -- 172.21.15.172:0/2026386086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fada80b28a0 msgr2=0x7fada81f3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.509+0000 7fadacb1e640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fada80b28a0 0x7fada81f3aa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fad98009890 tx=0x7fad980046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.509+0000 7fadacb1e640 1 -- 172.21.15.172:0/2026386086 shutdown_connections 2024-04-14T08:33:40.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.509+0000 7fadacb1e640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fad8806c8f0 0x7fad8806edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.509+0000 7fadacb1e640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fada815e8a0 0x7fada8148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.509+0000 7fadacb1e640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fada80ccc40 0x7fada81f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.509+0000 7fadacb1e640 1 --2- 172.21.15.172:0/2026386086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fada80b28a0 0x7fada81f3aa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.509+0000 7fadacb1e640 1 -- 172.21.15.172:0/2026386086 >> 172.21.15.172:0/2026386086 conn(0x7fada80b1d80 msgr2=0x7fada80c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:40.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.509+0000 7fadacb1e640 1 -- 172.21.15.172:0/2026386086 shutdown_connections 2024-04-14T08:33:40.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.509+0000 7fadacb1e640 1 -- 172.21.15.172:0/2026386086 wait complete. 2024-04-14T08:33:40.521 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:33:40.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.622+0000 7f93df988640 1 Processor -- start 2024-04-14T08:33:40.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.622+0000 7f93df988640 1 -- start start 2024-04-14T08:33:40.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93df988640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93d8159c20 0x7f93d80ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93df988640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93d815c700 0x7f93d81603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93df988640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d815cf10 con 0x7f93d8160930 2024-04-14T08:33:40.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93df988640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f93d815fc80 con 0x7f93d815c700 2024-04-14T08:33:40.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93df988640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f93d815ff60 con 0x7f93d8159c20 2024-04-14T08:33:40.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93dcf23640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93d815c700 0x7f93d81603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93dcf23640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93d815c700 0x7f93d81603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59582/0 (socket says 172.21.15.172:59582) 2024-04-14T08:33:40.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93dcf23640 1 -- 172.21.15.172:0/1716704930 learned_addr learned my addr 172.21.15.172:0/1716704930 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:40.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93dd724640 1 --2- 172.21.15.172:0/1716704930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93d8159c20 0x7f93d80ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93dcf23640 1 -- 172.21.15.172:0/1716704930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93d8159c20 msgr2=0x7f93d80ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:40.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93dcf23640 1 --2- 172.21.15.172:0/1716704930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93d8159c20 0x7f93d80ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93dcf23640 1 -- 172.21.15.172:0/1716704930 >> v1:172.21.15.177:6789/0 conn(0x7f93d8160930 legacy=0x7f93d8072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.623+0000 7f93dcf23640 1 -- 172.21.15.172:0/1716704930 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d815d1f0 con 0x7f93d815c700 2024-04-14T08:33:40.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.624+0000 7f93dcf23640 1 --2- 172.21.15.172:0/1716704930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d815c700 0x7f93d81603f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009870 tx=0x7f93c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98347ad1da075e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:40.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.624+0000 7f93cffff640 1 -- 172.21.15.172:0/1716704930 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8034070 con 0x7f93d815c700 2024-04-14T08:33:40.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.624+0000 7f93cffff640 1 -- 172.21.15.172:0/1716704930 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c8030840 con 0x7f93d815c700 2024-04-14T08:33:40.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.624+0000 7f93cffff640 1 -- 172.21.15.172:0/1716704930 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8030c10 con 0x7f93d815c700 2024-04-14T08:33:40.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.625+0000 7f93df988640 1 -- 172.21.15.172:0/1716704930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d815c700 msgr2=0x7f93d81603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.625+0000 7f93df988640 1 --2- 172.21.15.172:0/1716704930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d815c700 0x7f93d81603f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009870 tx=0x7f93c802b980 comp rx=0 tx=0).stop 2024-04-14T08:33:40.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.625+0000 7f93df988640 1 -- 172.21.15.172:0/1716704930 shutdown_connections 2024-04-14T08:33:40.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.625+0000 7f93df988640 1 --2- 172.21.15.172:0/1716704930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d815c700 0x7f93d81603f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.625+0000 7f93df988640 1 --2- 172.21.15.172:0/1716704930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93d8159c20 0x7f93d80ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.625+0000 7f93df988640 1 -- 172.21.15.172:0/1716704930 >> 172.21.15.172:0/1716704930 conn(0x7f93d806b520 msgr2=0x7f93d806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:40.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.625+0000 7f93df988640 1 -- 172.21.15.172:0/1716704930 shutdown_connections 2024-04-14T08:33:40.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.625+0000 7f93df988640 1 -- 172.21.15.172:0/1716704930 wait complete. 2024-04-14T08:33:40.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.626+0000 7f93df988640 1 Processor -- start 2024-04-14T08:33:40.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.626+0000 7f93df988640 1 -- start start 2024-04-14T08:33:40.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93df988640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d8159c20 0x7f93d814dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93df988640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93d815c700 0x7f93d80ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93df988640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93d80ade20 0x7f93d81e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93df988640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93d815cb80 con 0x7f93d8159c20 2024-04-14T08:33:40.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93df988640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f93d815a000 con 0x7f93d80ade20 2024-04-14T08:33:40.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93df988640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f93d8072b60 con 0x7f93d815c700 2024-04-14T08:33:40.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93dd724640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d8159c20 0x7f93d814dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93dd724640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d8159c20 0x7f93d814dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59596/0 (socket says 172.21.15.172:59596) 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93dd724640 1 -- 172.21.15.172:0/4130862558 learned_addr learned my addr 172.21.15.172:0/4130862558 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93dcf23640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93d815c700 0x7f93d80ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93ddf25640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93d80ade20 0x7f93d81e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93dd724640 1 -- 172.21.15.172:0/4130862558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93d815c700 msgr2=0x7f93d80ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93dd724640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93d815c700 0x7f93d80ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93dd724640 1 -- 172.21.15.172:0/4130862558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93d80ade20 msgr2=0x7f93d81e76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93dd724640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93d80ade20 0x7f93d81e76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93dd724640 1 -- 172.21.15.172:0/4130862558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d80aea10 con 0x7f93d8159c20 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93ddf25640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93d80ade20 0x7f93d81e76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93dd724640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d8159c20 0x7f93d814dd80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f93d400efc0 tx=0x7f93d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.627+0000 7f93cdffb640 1 -- 172.21.15.172:0/4130862558 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4018020 con 0x7f93d8159c20 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.628+0000 7f93cdffb640 1 -- 172.21.15.172:0/4130862558 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d4014480 con 0x7f93d8159c20 2024-04-14T08:33:40.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.628+0000 7f93df988640 1 -- 172.21.15.172:0/4130862558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d80af140 con 0x7f93d8159c20 2024-04-14T08:33:40.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.628+0000 7f93cdffb640 1 -- 172.21.15.172:0/4130862558 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4014850 con 0x7f93d8159c20 2024-04-14T08:33:40.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.628+0000 7f93df988640 1 -- 172.21.15.172:0/4130862558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d80ae730 con 0x7f93d8159c20 2024-04-14T08:33:40.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.629+0000 7f93df988640 1 -- 172.21.15.172:0/4130862558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d8068590 con 0x7f93d8159c20 2024-04-14T08:33:40.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.629+0000 7f93cdffb640 1 -- 172.21.15.172:0/4130862558 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93d4005d10 con 0x7f93d8159c20 2024-04-14T08:33:40.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.629+0000 7f93cdffb640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f93bc06c8f0 0x7f93bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:40.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.630+0000 7f93cdffb640 1 -- 172.21.15.172:0/4130862558 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f93d408c900 con 0x7f93d8159c20 2024-04-14T08:33:40.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.630+0000 7f93dcf23640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f93bc06c8f0 0x7f93bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:40.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.630+0000 7f93dcf23640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f93bc06c8f0 0x7f93bc06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f93c8036000 tx=0x7f93c8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:40.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.634+0000 7f93cdffb640 1 -- 172.21.15.172:0/4130862558 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f93d4022070 con 0x7f93d8159c20 2024-04-14T08:33:40.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.819+0000 7f93df988640 1 -- 172.21.15.172:0/4130862558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f93d8071e50 con 0x7f93d8159c20 2024-04-14T08:33:40.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.819+0000 7f93cdffb640 1 -- 172.21.15.172:0/4130862558 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f93d4010040 con 0x7f93d8159c20 2024-04-14T08:33:40.820 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:33:40.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.821+0000 7f93df988640 1 -- 172.21.15.172:0/4130862558 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f93bc06c8f0 msgr2=0x7f93bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.821+0000 7f93df988640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f93bc06c8f0 0x7f93bc06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f93c8036000 tx=0x7f93c8008030 comp rx=0 tx=0).stop 2024-04-14T08:33:40.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.821+0000 7f93df988640 1 -- 172.21.15.172:0/4130862558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d8159c20 msgr2=0x7f93d814dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:40.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.821+0000 7f93df988640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d8159c20 0x7f93d814dd80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f93d400efc0 tx=0x7f93d400c490 comp rx=0 tx=0).stop 2024-04-14T08:33:40.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.821+0000 7f93df988640 1 -- 172.21.15.172:0/4130862558 shutdown_connections 2024-04-14T08:33:40.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.821+0000 7f93df988640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f93bc06c8f0 0x7f93bc06edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.821+0000 7f93df988640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93d80ade20 0x7f93d81e76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.821+0000 7f93df988640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93d815c700 0x7f93d80ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.821+0000 7f93df988640 1 --2- 172.21.15.172:0/4130862558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d8159c20 0x7f93d814dd80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:40.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.821+0000 7f93df988640 1 -- 172.21.15.172:0/4130862558 >> 172.21.15.172:0/4130862558 conn(0x7f93d806b520 msgr2=0x7f93d815a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:40.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.821+0000 7f93df988640 1 -- 172.21.15.172:0/4130862558 shutdown_connections 2024-04-14T08:33:40.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:40.821+0000 7f93df988640 1 -- 172.21.15.172:0/4130862558 wait complete. 2024-04-14T08:33:40.833 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-14T08:33:40.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:40.833 DEBUG:teuthology.orchestra.run.smithi177:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-14T08:33:40.861 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:33:40.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:40.862 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-14T08:33:40.984 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:33:40.985 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:41.000 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:33:41.001 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:41.057 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:33:41.057 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:41.112 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:41.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.253+0000 7fd2cc95e640 1 Processor -- start 2024-04-14T08:33:41.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.253+0000 7fd2cc95e640 1 -- start start 2024-04-14T08:33:41.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.254+0000 7fd2cc95e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd2c80b1df0 0x7fd2c80b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.254+0000 7fd2cc95e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c80b4040 0x7fd2c80b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.254+0000 7fd2cc95e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c80b49f0 con 0x7fd2c80b21d0 2024-04-14T08:33:41.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.254+0000 7fd2cc95e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fd2c815c770 con 0x7fd2c80b1df0 2024-04-14T08:33:41.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.254+0000 7fd2cc95e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c80b3430 con 0x7fd2c80b4040 2024-04-14T08:33:41.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.254+0000 7fd2c6d9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c80b4040 0x7fd2c80b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.254+0000 7fd2c659c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd2c80b1df0 0x7fd2c80b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.254+0000 7fd2c6d9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c80b4040 0x7fd2c80b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:36446/0 (socket says 172.21.15.172:36446) 2024-04-14T08:33:41.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.254+0000 7fd2c6d9d640 1 -- 172.21.15.172:0/4270633545 learned_addr learned my addr 172.21.15.172:0/4270633545 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:41.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.255+0000 7fd2c659c640 1 -- 172.21.15.172:0/4270633545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c80b4040 msgr2=0x7fd2c80b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:41.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.255+0000 7fd2c659c640 1 --2- 172.21.15.172:0/4270633545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c80b4040 0x7fd2c80b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.255+0000 7fd2c659c640 1 -- 172.21.15.172:0/4270633545 >> v1:172.21.15.177:6789/0 conn(0x7fd2c80b21d0 legacy=0x7fd2c815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.255+0000 7fd2c659c640 1 -- 172.21.15.172:0/4270633545 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c80b3700 con 0x7fd2c80b1df0 2024-04-14T08:33:41.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.255+0000 7fd2c6d9d640 1 --2- 172.21.15.172:0/4270633545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c80b4040 0x7fd2c80b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:41.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.255+0000 7fd2c659c640 1 --2- 172.21.15.172:0/4270633545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c80b1df0 0x7fd2c80b2ef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009d90 tx=0x7fd2b802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35437f876fc5ed9a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:41.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.256+0000 7fd2c559a640 1 -- 172.21.15.172:0/4270633545 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8039070 con 0x7fd2c80b1df0 2024-04-14T08:33:41.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.256+0000 7fd2c559a640 1 -- 172.21.15.172:0/4270633545 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b802cc50 con 0x7fd2c80b1df0 2024-04-14T08:33:41.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.256+0000 7fd2c559a640 1 -- 172.21.15.172:0/4270633545 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8034440 con 0x7fd2c80b1df0 2024-04-14T08:33:41.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.256+0000 7fd2cc95e640 1 -- 172.21.15.172:0/4270633545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c80b1df0 msgr2=0x7fd2c80b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.256+0000 7fd2cc95e640 1 --2- 172.21.15.172:0/4270633545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c80b1df0 0x7fd2c80b2ef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009d90 tx=0x7fd2b802bea0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.256+0000 7fd2cc95e640 1 -- 172.21.15.172:0/4270633545 shutdown_connections 2024-04-14T08:33:41.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.256+0000 7fd2cc95e640 1 --2- 172.21.15.172:0/4270633545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c80b4040 0x7fd2c80b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.256+0000 7fd2cc95e640 1 --2- 172.21.15.172:0/4270633545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c80b1df0 0x7fd2c80b2ef0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.256+0000 7fd2cc95e640 1 -- 172.21.15.172:0/4270633545 >> 172.21.15.172:0/4270633545 conn(0x7fd2c806ec30 msgr2=0x7fd2c806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:41.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.256+0000 7fd2cc95e640 1 -- 172.21.15.172:0/4270633545 shutdown_connections 2024-04-14T08:33:41.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.256+0000 7fd2cc95e640 1 -- 172.21.15.172:0/4270633545 wait complete. 2024-04-14T08:33:41.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.257+0000 7fd2cc95e640 1 Processor -- start 2024-04-14T08:33:41.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.257+0000 7fd2cc95e640 1 -- start start 2024-04-14T08:33:41.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.258+0000 7fd2cc95e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c80b1df0 0x7fd2c80aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.258+0000 7fd2cc95e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c80b21d0 0x7fd2c8157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.258+0000 7fd2cc95e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c80b4040 0x7fd2c815a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.258+0000 7fd2cc95e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c815bb40 con 0x7fd2c80b1df0 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.258+0000 7fd2cc95e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c80b3cc0 con 0x7fd2c80b21d0 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.258+0000 7fd2cc95e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c815b7f0 con 0x7fd2c80b4040 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.258+0000 7fd2c659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c80b1df0 0x7fd2c80aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.258+0000 7fd2c659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c80b1df0 0x7fd2c80aa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59606/0 (socket says 172.21.15.172:59606) 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.258+0000 7fd2c659c640 1 -- 172.21.15.172:0/1030418421 learned_addr learned my addr 172.21.15.172:0/1030418421 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.258+0000 7fd2c5d9b640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c80b21d0 0x7fd2c8157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.258+0000 7fd2c6d9d640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c80b4040 0x7fd2c815a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.259+0000 7fd2c659c640 1 -- 172.21.15.172:0/1030418421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c80b4040 msgr2=0x7fd2c815a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.259+0000 7fd2c659c640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c80b4040 0x7fd2c815a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.259+0000 7fd2c659c640 1 -- 172.21.15.172:0/1030418421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c80b21d0 msgr2=0x7fd2c8157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.259+0000 7fd2c659c640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c80b21d0 0x7fd2c8157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.259+0000 7fd2c659c640 1 -- 172.21.15.172:0/1030418421 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c80af3e0 con 0x7fd2c80b1df0 2024-04-14T08:33:41.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.259+0000 7fd2c5d9b640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c80b21d0 0x7fd2c8157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:41.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.259+0000 7fd2c659c640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c80b1df0 0x7fd2c80aa600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009d90 tx=0x7fd2b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:41.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.259+0000 7fd2b6ffd640 1 -- 172.21.15.172:0/1030418421 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8039050 con 0x7fd2c80b1df0 2024-04-14T08:33:41.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.259+0000 7fd2b6ffd640 1 -- 172.21.15.172:0/1030418421 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b8045070 con 0x7fd2c80b1df0 2024-04-14T08:33:41.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.259+0000 7fd2cc95e640 1 -- 172.21.15.172:0/1030418421 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c80aace0 con 0x7fd2c80b1df0 2024-04-14T08:33:41.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.259+0000 7fd2b6ffd640 1 -- 172.21.15.172:0/1030418421 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b803e610 con 0x7fd2c80b1df0 2024-04-14T08:33:41.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.259+0000 7fd2cc95e640 1 -- 172.21.15.172:0/1030418421 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c80ab2a0 con 0x7fd2c80b1df0 2024-04-14T08:33:41.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.260+0000 7fd2cc95e640 1 -- 172.21.15.172:0/1030418421 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c8068520 con 0x7fd2c80b1df0 2024-04-14T08:33:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.261+0000 7fd2b6ffd640 1 -- 172.21.15.172:0/1030418421 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2b803e7b0 con 0x7fd2c80b1df0 2024-04-14T08:33:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.261+0000 7fd2b6ffd640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd2a806c8f0 0x7fd2a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.261+0000 7fd2b6ffd640 1 -- 172.21.15.172:0/1030418421 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd2b80ae980 con 0x7fd2c80b1df0 2024-04-14T08:33:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.261+0000 7fd2c5d9b640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd2a806c8f0 0x7fd2a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.261+0000 7fd2c5d9b640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd2a806c8f0 0x7fd2a806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80afef0 tx=0x7fd2bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:41.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.265+0000 7fd2b6ffd640 1 -- 172.21.15.172:0/1030418421 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd2c80af3e0 con 0x7fd2c80b1df0 2024-04-14T08:33:41.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.442+0000 7fd2cc95e640 1 -- 172.21.15.172:0/1030418421 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2c8158b00 con 0x7fd2c80b1df0 2024-04-14T08:33:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.443+0000 7fd2b6ffd640 1 -- 172.21.15.172:0/1030418421 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18962 (secure 0 0 0) 0x7fd2b807c490 con 0x7fd2c80b1df0 2024-04-14T08:33:41.445 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:41.445 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":27,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:33:34.330426+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.445+0000 7fd2cc95e640 1 -- 172.21.15.172:0/1030418421 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd2a806c8f0 msgr2=0x7fd2a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.445+0000 7fd2cc95e640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd2a806c8f0 0x7fd2a806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80afef0 tx=0x7fd2bc009040 comp rx=0 tx=0).stop 2024-04-14T08:33:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.445+0000 7fd2cc95e640 1 -- 172.21.15.172:0/1030418421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c80b1df0 msgr2=0x7fd2c80aa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.445+0000 7fd2cc95e640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c80b1df0 0x7fd2c80aa600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009d90 tx=0x7fd2b8004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.445+0000 7fd2cc95e640 1 -- 172.21.15.172:0/1030418421 shutdown_connections 2024-04-14T08:33:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.445+0000 7fd2cc95e640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd2a806c8f0 0x7fd2a806edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.445+0000 7fd2cc95e640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c80b4040 0x7fd2c815a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.445+0000 7fd2cc95e640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c80b21d0 0x7fd2c8157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.445+0000 7fd2cc95e640 1 --2- 172.21.15.172:0/1030418421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c80b1df0 0x7fd2c80aa600 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.445+0000 7fd2cc95e640 1 -- 172.21.15.172:0/1030418421 >> 172.21.15.172:0/1030418421 conn(0x7fd2c806ec30 msgr2=0x7fd2c8074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.445+0000 7fd2cc95e640 1 -- 172.21.15.172:0/1030418421 shutdown_connections 2024-04-14T08:33:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.446+0000 7fd2cc95e640 1 -- 172.21.15.172:0/1030418421 wait complete. 2024-04-14T08:33:41.457 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:41.458 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:41.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.561+0000 7eff62436640 1 Processor -- start 2024-04-14T08:33:41.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.561+0000 7eff62436640 1 -- start start 2024-04-14T08:33:41.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.562+0000 7eff62436640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7eff5c0af740 0x7eff5c0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.562+0000 7eff62436640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5c0ce860 con 0x7eff5c13fa70 2024-04-14T08:33:41.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.562+0000 7eff62436640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5c071520 con 0x7eff5c072820 2024-04-14T08:33:41.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.562+0000 7eff62436640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7eff5c0712d0 con 0x7eff5c0af740 2024-04-14T08:33:41.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.562+0000 7eff5b7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7eff5c0af740 0x7eff5c0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.562+0000 7eff5bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7eff5c13fa70 0x7eff5c0ce150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54088/0 (socket says 172.21.15.172:54088) 2024-04-14T08:33:41.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.562+0000 7eff5bfff640 1 -- 172.21.15.172:0/2071418616 learned_addr learned my addr 172.21.15.172:0/2071418616 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:41.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.562+0000 7eff5b7fe640 1 -- 172.21.15.172:0/2071418616 >> v1:172.21.15.177:6789/0 conn(0x7eff5c072820 legacy=0x7eff5c13fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.562+0000 7eff5b7fe640 1 -- 172.21.15.172:0/2071418616 >> v1:172.21.15.172:6789/0 conn(0x7eff5c13fa70 legacy=0x7eff5c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.562+0000 7eff5b7fe640 1 -- 172.21.15.172:0/2071418616 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c071800 con 0x7eff5c0af740 2024-04-14T08:33:41.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.562+0000 7eff5affd640 1 -- 172.21.15.172:0/2071418616 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3393424732 0 0) 0x7eff4c001110 con 0x7eff5c13fa70 2024-04-14T08:33:41.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.563+0000 7eff5b7fe640 1 --2- 172.21.15.172:0/2071418616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0af740 0x7eff5c0722e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7eff500097a0 tx=0x7eff5002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fd9e0db733417f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:41.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.563+0000 7eff5affd640 1 -- 172.21.15.172:0/2071418616 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50035020 con 0x7eff5c0af740 2024-04-14T08:33:41.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.563+0000 7eff5affd640 1 -- 172.21.15.172:0/2071418616 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff50031830 con 0x7eff5c0af740 2024-04-14T08:33:41.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.563+0000 7eff5affd640 1 -- 172.21.15.172:0/2071418616 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50031c00 con 0x7eff5c0af740 2024-04-14T08:33:41.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.564+0000 7eff62436640 1 -- 172.21.15.172:0/2071418616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0af740 msgr2=0x7eff5c0722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.564+0000 7eff62436640 1 --2- 172.21.15.172:0/2071418616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0af740 0x7eff5c0722e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7eff500097a0 tx=0x7eff5002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.564+0000 7eff62436640 1 -- 172.21.15.172:0/2071418616 shutdown_connections 2024-04-14T08:33:41.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.564+0000 7eff62436640 1 --2- 172.21.15.172:0/2071418616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0af740 0x7eff5c0722e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.564+0000 7eff62436640 1 -- 172.21.15.172:0/2071418616 >> 172.21.15.172:0/2071418616 conn(0x7eff5c06b520 msgr2=0x7eff5c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:41.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.564+0000 7eff62436640 1 -- 172.21.15.172:0/2071418616 shutdown_connections 2024-04-14T08:33:41.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.564+0000 7eff62436640 1 -- 172.21.15.172:0/2071418616 wait complete. 2024-04-14T08:33:41.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.565+0000 7eff62436640 1 Processor -- start 2024-04-14T08:33:41.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.565+0000 7eff62436640 1 -- start start 2024-04-14T08:33:41.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff62436640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff5c072820 0x7eff5c13c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff62436640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0af740 0x7eff5c150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff62436640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff5c13fa70 0x7eff5c1523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff62436640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c072c30 con 0x7eff5c0af740 2024-04-14T08:33:41.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff62436640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c071dc0 con 0x7eff5c072820 2024-04-14T08:33:41.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff62436640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7eff5c144130 con 0x7eff5c13fa70 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff5b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0af740 0x7eff5c150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff5b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0af740 0x7eff5c150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59628/0 (socket says 172.21.15.172:59628) 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff5b7fe640 1 -- 172.21.15.172:0/3074791282 learned_addr learned my addr 172.21.15.172:0/3074791282 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff5bfff640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff5c072820 0x7eff5c13c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff609d3640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff5c13fa70 0x7eff5c1523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff5b7fe640 1 -- 172.21.15.172:0/3074791282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff5c13fa70 msgr2=0x7eff5c1523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff5b7fe640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff5c13fa70 0x7eff5c1523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff5b7fe640 1 -- 172.21.15.172:0/3074791282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff5c072820 msgr2=0x7eff5c13c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff5b7fe640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff5c072820 0x7eff5c13c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff5b7fe640 1 -- 172.21.15.172:0/3074791282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c13cdd0 con 0x7eff5c0af740 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff5b7fe640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0af740 0x7eff5c150960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7eff500098c0 tx=0x7eff500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff58ff9640 1 -- 172.21.15.172:0/3074791282 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50041070 con 0x7eff5c0af740 2024-04-14T08:33:41.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff58ff9640 1 -- 172.21.15.172:0/3074791282 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff5003aa30 con 0x7eff5c0af740 2024-04-14T08:33:41.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.566+0000 7eff58ff9640 1 -- 172.21.15.172:0/3074791282 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50035040 con 0x7eff5c0af740 2024-04-14T08:33:41.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.567+0000 7eff62436640 1 -- 172.21.15.172:0/3074791282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5c151340 con 0x7eff5c0af740 2024-04-14T08:33:41.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.567+0000 7eff62436640 1 -- 172.21.15.172:0/3074791282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5c151900 con 0x7eff5c0af740 2024-04-14T08:33:41.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.567+0000 7eff62436640 1 -- 172.21.15.172:0/3074791282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff5c06be60 con 0x7eff5c0af740 2024-04-14T08:33:41.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.568+0000 7eff58ff9640 1 -- 172.21.15.172:0/3074791282 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff5004c060 con 0x7eff5c0af740 2024-04-14T08:33:41.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.568+0000 7eff58ff9640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7eff4806c8f0 0x7eff4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.569+0000 7eff5bfff640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7eff4806c8f0 0x7eff4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.569+0000 7eff58ff9640 1 -- 172.21.15.172:0/3074791282 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7eff500ad960 con 0x7eff5c0af740 2024-04-14T08:33:41.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.569+0000 7eff5bfff640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7eff4806c8f0 0x7eff4806edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7eff4c000f60 tx=0x7eff4c007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:41.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.573+0000 7eff58ff9640 1 -- 172.21.15.172:0/3074791282 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7eff5c06be60 con 0x7eff5c0af740 2024-04-14T08:33:41.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.746+0000 7eff62436640 1 -- 172.21.15.172:0/3074791282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff5c068590 con 0x7eff5c0af740 2024-04-14T08:33:41.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.747+0000 7eff58ff9640 1 -- 172.21.15.172:0/3074791282 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7eff5007b2b0 con 0x7eff5c0af740 2024-04-14T08:33:41.747 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:41.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.748+0000 7eff62436640 1 -- 172.21.15.172:0/3074791282 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7eff4806c8f0 msgr2=0x7eff4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.748+0000 7eff62436640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7eff4806c8f0 0x7eff4806edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7eff4c000f60 tx=0x7eff4c007b50 comp rx=0 tx=0).stop 2024-04-14T08:33:41.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.749+0000 7eff62436640 1 -- 172.21.15.172:0/3074791282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0af740 msgr2=0x7eff5c150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.749+0000 7eff62436640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0af740 0x7eff5c150960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7eff500098c0 tx=0x7eff500046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.749+0000 7eff62436640 1 -- 172.21.15.172:0/3074791282 shutdown_connections 2024-04-14T08:33:41.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.749+0000 7eff62436640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7eff4806c8f0 0x7eff4806edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.749+0000 7eff62436640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff5c13fa70 0x7eff5c1523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.749+0000 7eff62436640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0af740 0x7eff5c150960 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.749+0000 7eff62436640 1 --2- 172.21.15.172:0/3074791282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff5c072820 0x7eff5c13c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.749+0000 7eff62436640 1 -- 172.21.15.172:0/3074791282 >> 172.21.15.172:0/3074791282 conn(0x7eff5c06b520 msgr2=0x7eff5c0cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:41.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.749+0000 7eff62436640 1 -- 172.21.15.172:0/3074791282 shutdown_connections 2024-04-14T08:33:41.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.749+0000 7eff62436640 1 -- 172.21.15.172:0/3074791282 wait complete. 2024-04-14T08:33:41.760 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:41.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.864+0000 7f81bff55640 1 Processor -- start 2024-04-14T08:33:41.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.864+0000 7f81bff55640 1 -- start start 2024-04-14T08:33:41.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81bff55640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f81b815e820 0x7f81b80b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81bff55640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81b80d0d30 0x7f81b80cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81bff55640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b80d12e0 con 0x7f81b806d040 2024-04-14T08:33:41.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81bff55640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f81b806cb90 con 0x7f81b815e820 2024-04-14T08:33:41.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81bff55640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f81b80cc660 con 0x7f81b80d0d30 2024-04-14T08:33:41.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81bd4f0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f81b815e820 0x7f81b80b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81bd4f0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f81b815e820 0x7f81b80b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59644/0 (socket says 172.21.15.172:59644) 2024-04-14T08:33:41.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81be4f2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81b80d0d30 0x7f81b80cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81bdcf1640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f81b806d040 0x7f81b806d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:51340/0 (socket says 172.21.15.172:51340) 2024-04-14T08:33:41.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81bd4f0640 1 -- 172.21.15.172:0/3891600965 learned_addr learned my addr 172.21.15.172:0/3891600965 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:41.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81bccef640 1 -- 172.21.15.172:0/3891600965 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3285048180 0 0) 0x7f81b80d12e0 con 0x7f81b806d040 2024-04-14T08:33:41.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81be4f2640 1 -- 172.21.15.172:0/3891600965 >> v2:172.21.15.172:3300/0 conn(0x7f81b815e820 msgr2=0x7f81b80b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:41.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81be4f2640 1 --2- 172.21.15.172:0/3891600965 >> v2:172.21.15.172:3300/0 conn(0x7f81b815e820 0x7f81b80b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81be4f2640 1 -- 172.21.15.172:0/3891600965 >> v1:172.21.15.177:6789/0 conn(0x7f81b806d040 legacy=0x7f81b806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81be4f2640 1 -- 172.21.15.172:0/3891600965 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b80d1580 con 0x7f81b80d0d30 2024-04-14T08:33:41.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.865+0000 7f81bd4f0640 1 --2- 172.21.15.172:0/3891600965 >> v2:172.21.15.172:3300/0 conn(0x7f81b815e820 0x7f81b80b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:41.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.866+0000 7f81be4f2640 1 --2- 172.21.15.172:0/3891600965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81b80d0d30 0x7f81b80cd0b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f81b40097a0 tx=0x7f81b402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37c86a5fae5e9c45 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:41.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.866+0000 7f81bccef640 1 -- 172.21.15.172:0/3891600965 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4035020 con 0x7f81b80d0d30 2024-04-14T08:33:41.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.866+0000 7f81bccef640 1 -- 172.21.15.172:0/3891600965 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b4031810 con 0x7f81b80d0d30 2024-04-14T08:33:41.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.866+0000 7f81bccef640 1 -- 172.21.15.172:0/3891600965 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4031be0 con 0x7f81b80d0d30 2024-04-14T08:33:41.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.867+0000 7f81bff55640 1 -- 172.21.15.172:0/3891600965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81b80d0d30 msgr2=0x7f81b80cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.867+0000 7f81bff55640 1 --2- 172.21.15.172:0/3891600965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81b80d0d30 0x7f81b80cd0b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f81b40097a0 tx=0x7f81b402b8b0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.867+0000 7f81bff55640 1 -- 172.21.15.172:0/3891600965 shutdown_connections 2024-04-14T08:33:41.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.867+0000 7f81bff55640 1 --2- 172.21.15.172:0/3891600965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81b80d0d30 0x7f81b80cd0b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.867+0000 7f81bff55640 1 --2- 172.21.15.172:0/3891600965 >> v2:172.21.15.172:3300/0 conn(0x7f81b815e820 0x7f81b80b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.867+0000 7f81bff55640 1 -- 172.21.15.172:0/3891600965 >> 172.21.15.172:0/3891600965 conn(0x7f81b80b3b00 msgr2=0x7f81b80b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:41.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.867+0000 7f81bff55640 1 -- 172.21.15.172:0/3891600965 shutdown_connections 2024-04-14T08:33:41.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.867+0000 7f81bff55640 1 -- 172.21.15.172:0/3891600965 wait complete. 2024-04-14T08:33:41.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.868+0000 7f81bff55640 1 Processor -- start 2024-04-14T08:33:41.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.868+0000 7f81bff55640 1 -- start start 2024-04-14T08:33:41.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bff55640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81b806d040 0x7f81b80cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bff55640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81b80d0d30 0x7f81b81f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bff55640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b815e820 0x7f81b815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bff55640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81b80cd5f0 con 0x7f81b80d0d30 2024-04-14T08:33:41.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bff55640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f81b80ccc20 con 0x7f81b815e820 2024-04-14T08:33:41.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bff55640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f81b80cd770 con 0x7f81b806d040 2024-04-14T08:33:41.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bd4f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81b80d0d30 0x7f81b81f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bd4f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81b80d0d30 0x7f81b81f3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59660/0 (socket says 172.21.15.172:59660) 2024-04-14T08:33:41.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bd4f0640 1 -- 172.21.15.172:0/2127954296 learned_addr learned my addr 172.21.15.172:0/2127954296 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:41.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bdcf1640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81b806d040 0x7f81b80cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81be4f2640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b815e820 0x7f81b815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bd4f0640 1 -- 172.21.15.172:0/2127954296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81b806d040 msgr2=0x7f81b80cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bd4f0640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81b806d040 0x7f81b80cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bd4f0640 1 -- 172.21.15.172:0/2127954296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b815e820 msgr2=0x7f81b815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:41.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bd4f0640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b815e820 0x7f81b815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bd4f0640 1 -- 172.21.15.172:0/2127954296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b80c90e0 con 0x7f81b80d0d30 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bd4f0640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81b80d0d30 0x7f81b81f3a30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f81a800e9b0 tx=0x7f81a800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81ae7fc640 1 -- 172.21.15.172:0/2127954296 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8017070 con 0x7f81b80d0d30 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81ae7fc640 1 -- 172.21.15.172:0/2127954296 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a800c9a0 con 0x7f81b80d0d30 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81ae7fc640 1 -- 172.21.15.172:0/2127954296 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a800cd50 con 0x7f81b80d0d30 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.869+0000 7f81bff55640 1 -- 172.21.15.172:0/2127954296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b80c9380 con 0x7f81b80d0d30 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.870+0000 7f81bff55640 1 -- 172.21.15.172:0/2127954296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b8148bc0 con 0x7f81b80d0d30 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.870+0000 7f81bff55640 1 -- 172.21.15.172:0/2127954296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b8068590 con 0x7f81b80d0d30 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.871+0000 7f81ae7fc640 1 -- 172.21.15.172:0/2127954296 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81a801a070 con 0x7f81b80d0d30 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.871+0000 7f81ae7fc640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f819c06c8f0 0x7f819c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.871+0000 7f81ae7fc640 1 -- 172.21.15.172:0/2127954296 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f81a808bdb0 con 0x7f81b80d0d30 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.871+0000 7f81bdcf1640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f819c06c8f0 0x7f819c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.872+0000 7f81bdcf1640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f819c06c8f0 0x7f819c06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f81b00012e0 tx=0x7f81b0001700 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:41.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:41.876+0000 7f81ae7fc640 1 -- 172.21.15.172:0/2127954296 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f81a8059710 con 0x7f81b80d0d30 2024-04-14T08:33:42.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.049+0000 7f81bff55640 1 -- 172.21.15.172:0/2127954296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f81b806ce20 con 0x7f81b80d0d30 2024-04-14T08:33:42.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.049+0000 7f81ae7fc640 1 -- 172.21.15.172:0/2127954296 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f81a805da10 con 0x7f81b80d0d30 2024-04-14T08:33:42.050 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:42.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.051+0000 7f81bff55640 1 -- 172.21.15.172:0/2127954296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f819c06c8f0 msgr2=0x7f819c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.051+0000 7f81bff55640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f819c06c8f0 0x7f819c06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f81b00012e0 tx=0x7f81b0001700 comp rx=0 tx=0).stop 2024-04-14T08:33:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.051+0000 7f81bff55640 1 -- 172.21.15.172:0/2127954296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81b80d0d30 msgr2=0x7f81b81f3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.051+0000 7f81bff55640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81b80d0d30 0x7f81b81f3a30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f81a800e9b0 tx=0x7f81a800ee70 comp rx=0 tx=0).stop 2024-04-14T08:33:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.051+0000 7f81bff55640 1 -- 172.21.15.172:0/2127954296 shutdown_connections 2024-04-14T08:33:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.051+0000 7f81bff55640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f819c06c8f0 0x7f819c06edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.051+0000 7f81bff55640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b815e820 0x7f81b815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.051+0000 7f81bff55640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81b80d0d30 0x7f81b81f3a30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.051+0000 7f81bff55640 1 --2- 172.21.15.172:0/2127954296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81b806d040 0x7f81b80cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.051+0000 7f81bff55640 1 -- 172.21.15.172:0/2127954296 >> 172.21.15.172:0/2127954296 conn(0x7f81b80b3b00 msgr2=0x7f81b815c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.052+0000 7f81bff55640 1 -- 172.21.15.172:0/2127954296 shutdown_connections 2024-04-14T08:33:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.052+0000 7f81bff55640 1 -- 172.21.15.172:0/2127954296 wait complete. 2024-04-14T08:33:42.063 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:42.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.167+0000 7f2132436640 1 Processor -- start 2024-04-14T08:33:42.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.167+0000 7f2132436640 1 -- start start 2024-04-14T08:33:42.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f2132436640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f212c06d040 0x7f212c0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f2132436640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f212c15e820 0x7f212c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f2132436640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f212c0d12e0 con 0x7f212c0d0d30 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f2132436640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f212c06cb90 con 0x7f212c15e820 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f2132436640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f212c0cc660 con 0x7f212c06d040 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f21309d3640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f212c0d0d30 0x7f212c09aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54104/0 (socket says 172.21.15.172:54104) 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f212bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f212c06d040 0x7f212c0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f21309d3640 1 -- 172.21.15.172:0/1487282182 learned_addr learned my addr 172.21.15.172:0/1487282182 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f212b7fe640 1 --2- 172.21.15.172:0/1487282182 >> v2:172.21.15.177:3300/0 conn(0x7f212c15e820 0x7f212c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f212affd640 1 -- 172.21.15.172:0/1487282182 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2960970451 0 0) 0x7f212c0d12e0 con 0x7f212c0d0d30 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f212affd640 1 -- 172.21.15.172:0/1487282182 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2114003520 con 0x7f212c0d0d30 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f212b7fe640 1 -- 172.21.15.172:0/1487282182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f212c06d040 msgr2=0x7f212c0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f212b7fe640 1 --2- 172.21.15.172:0/1487282182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f212c06d040 0x7f212c0b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f212b7fe640 1 -- 172.21.15.172:0/1487282182 >> v1:172.21.15.172:6789/0 conn(0x7f212c0d0d30 legacy=0x7f212c09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f212b7fe640 1 -- 172.21.15.172:0/1487282182 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c06d960 con 0x7f212c15e820 2024-04-14T08:33:42.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.168+0000 7f212bfff640 1 --2- 172.21.15.172:0/1487282182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f212c06d040 0x7f212c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:42.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.169+0000 7f212b7fe640 1 --2- 172.21.15.172:0/1487282182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f212c15e820 0x7f212c06d420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f21200097a0 tx=0x7f212002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81d4a630d8852c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:42.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.169+0000 7f212affd640 1 -- 172.21.15.172:0/1487282182 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120035020 con 0x7f212c15e820 2024-04-14T08:33:42.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.169+0000 7f212affd640 1 -- 172.21.15.172:0/1487282182 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2120031830 con 0x7f212c15e820 2024-04-14T08:33:42.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.169+0000 7f212affd640 1 -- 172.21.15.172:0/1487282182 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120031be0 con 0x7f212c15e820 2024-04-14T08:33:42.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.169+0000 7f2132436640 1 -- 172.21.15.172:0/1487282182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f212c15e820 msgr2=0x7f212c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.169+0000 7f2132436640 1 --2- 172.21.15.172:0/1487282182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f212c15e820 0x7f212c06d420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f21200097a0 tx=0x7f212002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.170+0000 7f2132436640 1 -- 172.21.15.172:0/1487282182 shutdown_connections 2024-04-14T08:33:42.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.170+0000 7f2132436640 1 --2- 172.21.15.172:0/1487282182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f212c15e820 0x7f212c06d420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.170+0000 7f2132436640 1 --2- 172.21.15.172:0/1487282182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f212c06d040 0x7f212c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.170+0000 7f2132436640 1 -- 172.21.15.172:0/1487282182 >> 172.21.15.172:0/1487282182 conn(0x7f212c0b3b00 msgr2=0x7f212c0b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.170+0000 7f2132436640 1 -- 172.21.15.172:0/1487282182 shutdown_connections 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.170+0000 7f2132436640 1 -- 172.21.15.172:0/1487282182 wait complete. 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.171+0000 7f2132436640 1 Processor -- start 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.171+0000 7f2132436640 1 -- start start 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f2132436640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f212c06d040 0x7f212c14e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f2132436640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212c0d0d30 0x7f212c15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f2132436640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f212c15e820 0x7f212c15a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f2132436640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f212c0cc8b0 con 0x7f212c0d0d30 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f2132436640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f212c0ccc20 con 0x7f212c06d040 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f2132436640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f212c0ce0c0 con 0x7f212c15e820 2024-04-14T08:33:42.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f212b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212c0d0d30 0x7f212c15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f212b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212c0d0d30 0x7f212c15a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59668/0 (socket says 172.21.15.172:59668) 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f212b7fe640 1 -- 172.21.15.172:0/3401112538 learned_addr learned my addr 172.21.15.172:0/3401112538 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f212bfff640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f212c06d040 0x7f212c14e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f21309d3640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f212c15e820 0x7f212c15a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f212b7fe640 1 -- 172.21.15.172:0/3401112538 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f212c15e820 msgr2=0x7f212c15a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f212b7fe640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f212c15e820 0x7f212c15a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f212b7fe640 1 -- 172.21.15.172:0/3401112538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f212c06d040 msgr2=0x7f212c14e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f212b7fe640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f212c06d040 0x7f212c14e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f212b7fe640 1 -- 172.21.15.172:0/3401112538 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c14ea40 con 0x7f212c0d0d30 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f212b7fe640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212c0d0d30 0x7f212c15a0e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2120034040 tx=0x7f21200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f2128ff9640 1 -- 172.21.15.172:0/3401112538 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120040070 con 0x7f212c0d0d30 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f2128ff9640 1 -- 172.21.15.172:0/3401112538 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f212003e070 con 0x7f212c0d0d30 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f2132436640 1 -- 172.21.15.172:0/3401112538 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212c0b6950 con 0x7f212c0d0d30 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f2128ff9640 1 -- 172.21.15.172:0/3401112538 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120035040 con 0x7f212c0d0d30 2024-04-14T08:33:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.172+0000 7f2132436640 1 -- 172.21.15.172:0/3401112538 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f212c0b6f10 con 0x7f212c0d0d30 2024-04-14T08:33:42.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.173+0000 7f2132436640 1 -- 172.21.15.172:0/3401112538 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212c068590 con 0x7f212c0d0d30 2024-04-14T08:33:42.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.174+0000 7f2128ff9640 1 -- 172.21.15.172:0/3401112538 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2120038020 con 0x7f212c0d0d30 2024-04-14T08:33:42.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.174+0000 7f2128ff9640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f211806c8f0 0x7f211806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.174+0000 7f2128ff9640 1 -- 172.21.15.172:0/3401112538 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f21200ad610 con 0x7f212c0d0d30 2024-04-14T08:33:42.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.174+0000 7f212bfff640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f211806c8f0 0x7f211806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.175+0000 7f212bfff640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f211806c8f0 0x7f211806edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f211c009560 tx=0x7f211c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:42.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.179+0000 7f2128ff9640 1 -- 172.21.15.172:0/3401112538 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f212c14ea40 con 0x7f212c0d0d30 2024-04-14T08:33:42.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.358+0000 7f2132436640 1 -- 172.21.15.172:0/3401112538 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f212c06e270 con 0x7f212c0d0d30 2024-04-14T08:33:42.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.360+0000 7f2128ff9640 1 -- 172.21.15.172:0/3401112538 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f212007aff0 con 0x7f212c0d0d30 2024-04-14T08:33:42.360 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:42.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.362+0000 7f2132436640 1 -- 172.21.15.172:0/3401112538 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f211806c8f0 msgr2=0x7f211806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.362+0000 7f2132436640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f211806c8f0 0x7f211806edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f211c009560 tx=0x7f211c009190 comp rx=0 tx=0).stop 2024-04-14T08:33:42.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.362+0000 7f2132436640 1 -- 172.21.15.172:0/3401112538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212c0d0d30 msgr2=0x7f212c15a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.362+0000 7f2132436640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212c0d0d30 0x7f212c15a0e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2120034040 tx=0x7f21200046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.362+0000 7f2132436640 1 -- 172.21.15.172:0/3401112538 shutdown_connections 2024-04-14T08:33:42.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.362+0000 7f2132436640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f211806c8f0 0x7f211806edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.362+0000 7f2132436640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f212c15e820 0x7f212c15a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.362+0000 7f2132436640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212c0d0d30 0x7f212c15a0e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.362+0000 7f2132436640 1 --2- 172.21.15.172:0/3401112538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f212c06d040 0x7f212c14e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.362+0000 7f2132436640 1 -- 172.21.15.172:0/3401112538 >> 172.21.15.172:0/3401112538 conn(0x7f212c0b3b00 msgr2=0x7f212c15bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:42.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.362+0000 7f2132436640 1 -- 172.21.15.172:0/3401112538 shutdown_connections 2024-04-14T08:33:42.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.362+0000 7f2132436640 1 -- 172.21.15.172:0/3401112538 wait complete. 2024-04-14T08:33:42.373 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:42.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.477+0000 7fedd53b4640 1 Processor -- start 2024-04-14T08:33:42.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.477+0000 7fedd53b4640 1 -- start start 2024-04-14T08:33:42.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.478+0000 7fedd53b4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d0cc0 0x7fedd00d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.478+0000 7fedd53b4640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fedd00d4e20 0x7fedd0161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.478+0000 7fedd53b4640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd00d15e0 con 0x7fedd00d49c0 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.478+0000 7fedd53b4640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fedd00ceec0 con 0x7fedd00d4e20 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.478+0000 7fedd53b4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fedd00cfd00 con 0x7fedd00d0cc0 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.478+0000 7fedceffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d0cc0 0x7fedd00d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.478+0000 7fedcf7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fedd00d4e20 0x7fedd0161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.478+0000 7fedceffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d0cc0 0x7fedd00d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:36526/0 (socket says 172.21.15.172:36526) 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.478+0000 7fedce7fc640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fedd00d49c0 0x7fedd0160f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:51356/0 (socket says 172.21.15.172:51356) 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.478+0000 7fedceffd640 1 -- 172.21.15.172:0/2181319465 learned_addr learned my addr 172.21.15.172:0/2181319465 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.479+0000 7fedceffd640 1 -- 172.21.15.172:0/2181319465 >> v2:172.21.15.177:3300/0 conn(0x7fedd00d4e20 msgr2=0x7fedd0161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.479+0000 7fedceffd640 1 --2- 172.21.15.172:0/2181319465 >> v2:172.21.15.177:3300/0 conn(0x7fedd00d4e20 0x7fedd0161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.479+0000 7fedceffd640 1 -- 172.21.15.172:0/2181319465 >> v1:172.21.15.177:6789/0 conn(0x7fedd00d49c0 legacy=0x7fedd0160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.479+0000 7fedcdffb640 1 -- 172.21.15.172:0/2181319465 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4103263370 0 0) 0x7fedd00d15e0 con 0x7fedd00d49c0 2024-04-14T08:33:42.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.479+0000 7fedceffd640 1 -- 172.21.15.172:0/2181319465 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd00cf0f0 con 0x7fedd00d0cc0 2024-04-14T08:33:42.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.479+0000 7fedcf7fe640 1 --2- 172.21.15.172:0/2181319465 >> v2:172.21.15.177:3300/0 conn(0x7fedd00d4e20 0x7fedd0161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:42.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.480+0000 7fedceffd640 1 --2- 172.21.15.172:0/2181319465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d0cc0 0x7fedd00d10a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009870 tx=0x7fedc002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abeae339642ed8a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:42.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.480+0000 7fedcdffb640 1 -- 172.21.15.172:0/2181319465 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0034070 con 0x7fedd00d0cc0 2024-04-14T08:33:42.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.480+0000 7fedcdffb640 1 -- 172.21.15.172:0/2181319465 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc0030840 con 0x7fedd00d0cc0 2024-04-14T08:33:42.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.480+0000 7fedcdffb640 1 -- 172.21.15.172:0/2181319465 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0030c10 con 0x7fedd00d0cc0 2024-04-14T08:33:42.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.480+0000 7fedd53b4640 1 -- 172.21.15.172:0/2181319465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d0cc0 msgr2=0x7fedd00d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.480+0000 7fedd53b4640 1 --2- 172.21.15.172:0/2181319465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d0cc0 0x7fedd00d10a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009870 tx=0x7fedc002b980 comp rx=0 tx=0).stop 2024-04-14T08:33:42.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.480+0000 7fedd53b4640 1 -- 172.21.15.172:0/2181319465 shutdown_connections 2024-04-14T08:33:42.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.480+0000 7fedd53b4640 1 --2- 172.21.15.172:0/2181319465 >> v2:172.21.15.177:3300/0 conn(0x7fedd00d4e20 0x7fedd0161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.480+0000 7fedd53b4640 1 --2- 172.21.15.172:0/2181319465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d0cc0 0x7fedd00d10a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.480+0000 7fedd53b4640 1 -- 172.21.15.172:0/2181319465 >> 172.21.15.172:0/2181319465 conn(0x7fedd009d580 msgr2=0x7fedd00d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:42.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.480+0000 7fedd53b4640 1 -- 172.21.15.172:0/2181319465 shutdown_connections 2024-04-14T08:33:42.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.480+0000 7fedd53b4640 1 -- 172.21.15.172:0/2181319465 wait complete. 2024-04-14T08:33:42.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedd53b4640 1 Processor -- start 2024-04-14T08:33:42.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedd53b4640 1 -- start start 2024-04-14T08:33:42.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedd53b4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00d0cc0 0x7fedd01e1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedd53b4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd00d49c0 0x7fedd01e8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedd53b4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d4e20 0x7fedd01e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedd53b4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fedd00cf990 con 0x7fedd00d0cc0 2024-04-14T08:33:42.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedd53b4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fedd00cf6b0 con 0x7fedd00d49c0 2024-04-14T08:33:42.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedd53b4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fedd00d32b0 con 0x7fedd00d4e20 2024-04-14T08:33:42.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00d0cc0 0x7fedd01e1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedcf7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d4e20 0x7fedd01e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00d0cc0 0x7fedd01e1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59670/0 (socket says 172.21.15.172:59670) 2024-04-14T08:33:42.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedce7fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd00d49c0 0x7fedd01e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.482+0000 7fedceffd640 1 -- 172.21.15.172:0/1300450142 learned_addr learned my addr 172.21.15.172:0/1300450142 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedcf7fe640 1 -- 172.21.15.172:0/1300450142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd00d49c0 msgr2=0x7fedd01e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedcf7fe640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd00d49c0 0x7fedd01e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedcf7fe640 1 -- 172.21.15.172:0/1300450142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00d0cc0 msgr2=0x7fedd01e1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedcf7fe640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00d0cc0 0x7fedd01e1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedcf7fe640 1 -- 172.21.15.172:0/1300450142 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd00c91f0 con 0x7fedd00d4e20 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedceffd640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00d0cc0 0x7fedd01e1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedce7fc640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd00d49c0 0x7fedd01e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedcf7fe640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d4e20 0x7fedd01e47f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fedc800efc0 tx=0x7fedc800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedb77fe640 1 -- 172.21.15.172:0/1300450142 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8018030 con 0x7fedd00d4e20 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedb77fe640 1 -- 172.21.15.172:0/1300450142 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc800cd30 con 0x7fedd00d4e20 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedd53b4640 1 -- 172.21.15.172:0/1300450142 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd00c94d0 con 0x7fedd00d4e20 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedb77fe640 1 -- 172.21.15.172:0/1300450142 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8003bb0 con 0x7fedd00d4e20 2024-04-14T08:33:42.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.483+0000 7fedd53b4640 1 -- 172.21.15.172:0/1300450142 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd00c9a90 con 0x7fedd00d4e20 2024-04-14T08:33:42.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.484+0000 7fedd53b4640 1 -- 172.21.15.172:0/1300450142 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd0075490 con 0x7fedd00d4e20 2024-04-14T08:33:42.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.485+0000 7fedb77fe640 1 -- 172.21.15.172:0/1300450142 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedc801b070 con 0x7fedd00d4e20 2024-04-14T08:33:42.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.485+0000 7fedb77fe640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fedbc06c8f0 0x7fedbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.485+0000 7fedb77fe640 1 -- 172.21.15.172:0/1300450142 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fedc808c640 con 0x7fedd00d4e20 2024-04-14T08:33:42.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.485+0000 7fedceffd640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fedbc06c8f0 0x7fedbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.485+0000 7fedceffd640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fedbc06c8f0 0x7fedbc06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009870 tx=0x7fedc0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:42.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.490+0000 7fedb77fe640 1 -- 172.21.15.172:0/1300450142 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fedc8010040 con 0x7fedd00d4e20 2024-04-14T08:33:42.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.663+0000 7fedd53b4640 1 -- 172.21.15.172:0/1300450142 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fedd00cd9f0 con 0x7fedd00d4e20 2024-04-14T08:33:42.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.664+0000 7fedb77fe640 1 -- 172.21.15.172:0/1300450142 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fedc805a0b0 con 0x7fedd00d4e20 2024-04-14T08:33:42.664 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:42.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.666+0000 7fedd53b4640 1 -- 172.21.15.172:0/1300450142 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fedbc06c8f0 msgr2=0x7fedbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.666+0000 7fedd53b4640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fedbc06c8f0 0x7fedbc06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009870 tx=0x7fedc0008030 comp rx=0 tx=0).stop 2024-04-14T08:33:42.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.666+0000 7fedd53b4640 1 -- 172.21.15.172:0/1300450142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d4e20 msgr2=0x7fedd01e47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.666+0000 7fedd53b4640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d4e20 0x7fedd01e47f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fedc800efc0 tx=0x7fedc800c490 comp rx=0 tx=0).stop 2024-04-14T08:33:42.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.666+0000 7fedd53b4640 1 -- 172.21.15.172:0/1300450142 shutdown_connections 2024-04-14T08:33:42.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.666+0000 7fedd53b4640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fedbc06c8f0 0x7fedbc06edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.666+0000 7fedd53b4640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedd00d4e20 0x7fedd01e47f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.666+0000 7fedd53b4640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedd00d49c0 0x7fedd01e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.666+0000 7fedd53b4640 1 --2- 172.21.15.172:0/1300450142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedd00d0cc0 0x7fedd01e1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.666+0000 7fedd53b4640 1 -- 172.21.15.172:0/1300450142 >> 172.21.15.172:0/1300450142 conn(0x7fedd009d580 msgr2=0x7fedd00cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:42.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.666+0000 7fedd53b4640 1 -- 172.21.15.172:0/1300450142 shutdown_connections 2024-04-14T08:33:42.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.666+0000 7fedd53b4640 1 -- 172.21.15.172:0/1300450142 wait complete. 2024-04-14T08:33:42.677 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:33:42.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.780+0000 7f5e3ec1c640 1 Processor -- start 2024-04-14T08:33:42.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.780+0000 7f5e3ec1c640 1 -- start start 2024-04-14T08:33:42.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e3ec1c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5e3813fa70 0x7f5e380af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e3ec1c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e380ce150 0x7f5e380722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e3ec1c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e38071520 con 0x7f5e380ce5b0 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e3ec1c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f5e380712d0 con 0x7f5e3813fa70 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e3ec1c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e38071800 con 0x7f5e380ce150 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e3c9b8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5e3813fa70 0x7f5e380af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e3c9b8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5e3813fa70 0x7f5e380af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59676/0 (socket says 172.21.15.172:59676) 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e3c9b8640 1 -- 172.21.15.172:0/2749853803 learned_addr learned my addr 172.21.15.172:0/2749853803 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e37fff640 1 --2- 172.21.15.172:0/2749853803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e380ce150 0x7f5e380722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e3c9b8640 1 -- 172.21.15.172:0/2749853803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e380ce150 msgr2=0x7f5e380722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e3c9b8640 1 --2- 172.21.15.172:0/2749853803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e380ce150 0x7f5e380722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e3c9b8640 1 -- 172.21.15.172:0/2749853803 >> v1:172.21.15.177:6789/0 conn(0x7f5e380ce5b0 legacy=0x7f5e3815e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.781+0000 7f5e3c9b8640 1 -- 172.21.15.172:0/2749853803 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e38071ae0 con 0x7f5e3813fa70 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.782+0000 7f5e3c9b8640 1 --2- 172.21.15.172:0/2749853803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3813fa70 0x7f5e380af740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009870 tx=0x7f5e2c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451c22355fa44fce server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.782+0000 7f5e377fe640 1 -- 172.21.15.172:0/2749853803 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c035020 con 0x7f5e3813fa70 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.782+0000 7f5e377fe640 1 -- 172.21.15.172:0/2749853803 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2c031640 con 0x7f5e3813fa70 2024-04-14T08:33:42.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.782+0000 7f5e377fe640 1 -- 172.21.15.172:0/2749853803 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c02d040 con 0x7f5e3813fa70 2024-04-14T08:33:42.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.782+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/2749853803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3813fa70 msgr2=0x7f5e380af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.782+0000 7f5e3ec1c640 1 --2- 172.21.15.172:0/2749853803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3813fa70 0x7f5e380af740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009870 tx=0x7f5e2c02b8f0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.783+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/2749853803 shutdown_connections 2024-04-14T08:33:42.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.783+0000 7f5e3ec1c640 1 --2- 172.21.15.172:0/2749853803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e380ce150 0x7f5e380722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.783+0000 7f5e3ec1c640 1 --2- 172.21.15.172:0/2749853803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3813fa70 0x7f5e380af740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.783+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/2749853803 >> 172.21.15.172:0/2749853803 conn(0x7f5e3806b520 msgr2=0x7f5e3806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:42.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.783+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/2749853803 shutdown_connections 2024-04-14T08:33:42.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.783+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/2749853803 wait complete. 2024-04-14T08:33:42.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.784+0000 7f5e3ec1c640 1 Processor -- start 2024-04-14T08:33:42.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.784+0000 7f5e3ec1c640 1 -- start start 2024-04-14T08:33:42.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e3ec1c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e380ce150 0x7f5e381420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e3ec1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e380ce5b0 0x7f5e3815bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e3ec1c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e3813fa70 0x7f5e38143770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e3ec1c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381442b0 con 0x7f5e380ce5b0 2024-04-14T08:33:42.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e3ec1c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e38144130 con 0x7f5e380ce150 2024-04-14T08:33:42.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e3ec1c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e38072820 con 0x7f5e3813fa70 2024-04-14T08:33:42.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e380ce5b0 0x7f5e3815bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e380ce5b0 0x7f5e3815bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59686/0 (socket says 172.21.15.172:59686) 2024-04-14T08:33:42.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e37fff640 1 -- 172.21.15.172:0/3303337649 learned_addr learned my addr 172.21.15.172:0/3303337649 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:42.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e3c9b8640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e380ce150 0x7f5e381420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e3d1b9640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e3813fa70 0x7f5e38143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e37fff640 1 -- 172.21.15.172:0/3303337649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e3813fa70 msgr2=0x7f5e38143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e37fff640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e3813fa70 0x7f5e38143770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e37fff640 1 -- 172.21.15.172:0/3303337649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e380ce150 msgr2=0x7f5e381420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e37fff640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e380ce150 0x7f5e381420f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e37fff640 1 -- 172.21.15.172:0/3303337649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3815bff0 con 0x7f5e380ce5b0 2024-04-14T08:33:42.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e37fff640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e380ce5b0 0x7f5e3815bab0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400e980 tx=0x7f5e2400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:42.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e357fa640 1 -- 172.21.15.172:0/3303337649 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24017070 con 0x7f5e380ce5b0 2024-04-14T08:33:42.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e357fa640 1 -- 172.21.15.172:0/3303337649 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2400ca00 con 0x7f5e380ce5b0 2024-04-14T08:33:42.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/3303337649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e381426d0 con 0x7f5e380ce5b0 2024-04-14T08:33:42.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.785+0000 7f5e357fa640 1 -- 172.21.15.172:0/3303337649 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400cdd0 con 0x7f5e380ce5b0 2024-04-14T08:33:42.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.786+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/3303337649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e38142c90 con 0x7f5e380ce5b0 2024-04-14T08:33:42.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.786+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/3303337649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e38068590 con 0x7f5e380ce5b0 2024-04-14T08:33:42.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.787+0000 7f5e357fa640 1 -- 172.21.15.172:0/3303337649 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e2401a070 con 0x7f5e380ce5b0 2024-04-14T08:33:42.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.787+0000 7f5e357fa640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e1c06c920 0x7f5e1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:42.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.787+0000 7f5e3c9b8640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e1c06c920 0x7f5e1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:42.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.787+0000 7f5e357fa640 1 -- 172.21.15.172:0/3303337649 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5e2408c160 con 0x7f5e380ce5b0 2024-04-14T08:33:42.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.788+0000 7f5e3c9b8640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e1c06c920 0x7f5e1c06ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c036000 tx=0x7f5e2c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:42.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.792+0000 7f5e357fa640 1 -- 172.21.15.172:0/3303337649 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5e24059bd0 con 0x7f5e380ce5b0 2024-04-14T08:33:42.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.977+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/3303337649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5e38143cb0 con 0x7f5e380ce5b0 2024-04-14T08:33:42.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.977+0000 7f5e357fa640 1 -- 172.21.15.172:0/3303337649 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f5e2405ded0 con 0x7f5e380ce5b0 2024-04-14T08:33:42.978 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:33:42.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.979+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/3303337649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e1c06c920 msgr2=0x7f5e1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.979+0000 7f5e3ec1c640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e1c06c920 0x7f5e1c06ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c036000 tx=0x7f5e2c03c040 comp rx=0 tx=0).stop 2024-04-14T08:33:42.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.979+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/3303337649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e380ce5b0 msgr2=0x7f5e3815bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:42.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.979+0000 7f5e3ec1c640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e380ce5b0 0x7f5e3815bab0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400e980 tx=0x7f5e2400ee40 comp rx=0 tx=0).stop 2024-04-14T08:33:42.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.979+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/3303337649 shutdown_connections 2024-04-14T08:33:42.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.979+0000 7f5e3ec1c640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e1c06c920 0x7f5e1c06ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.979+0000 7f5e3ec1c640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e3813fa70 0x7f5e38143770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.979+0000 7f5e3ec1c640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e380ce5b0 0x7f5e3815bab0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.979+0000 7f5e3ec1c640 1 --2- 172.21.15.172:0/3303337649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e380ce150 0x7f5e381420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:42.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.979+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/3303337649 >> 172.21.15.172:0/3303337649 conn(0x7f5e3806b520 msgr2=0x7f5e38139910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:42.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.979+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/3303337649 shutdown_connections 2024-04-14T08:33:42.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:42.979+0000 7f5e3ec1c640 1 -- 172.21.15.172:0/3303337649 wait complete. 2024-04-14T08:33:42.991 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-14T08:33:42.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:42.991 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-14T08:33:43.019 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:33:43.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:43.020 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-14T08:33:43.075 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:33:43.075 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:43.091 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:33:43.091 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:43.146 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:33:43.146 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:33:43.201 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:43.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.301+0000 7f7c4b588640 1 Processor -- start 2024-04-14T08:33:43.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.301+0000 7f7c4b588640 1 -- start start 2024-04-14T08:33:43.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c4b588640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7c440b21d0 0x7f7c4415c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c4b588640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c4415c5a0 0x7f7c440b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c4b588640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c440b3700 con 0x7f7c440b1df0 2024-04-14T08:33:43.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c4b588640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f7c440b39e0 con 0x7f7c440b21d0 2024-04-14T08:33:43.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c4b588640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7c440b3cc0 con 0x7f7c4415c5a0 2024-04-14T08:33:43.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c48b23640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7c440b21d0 0x7f7c4415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c48b23640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7c440b21d0 0x7f7c4415c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59698/0 (socket says 172.21.15.172:59698) 2024-04-14T08:33:43.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c48b23640 1 -- 172.21.15.172:0/965235789 learned_addr learned my addr 172.21.15.172:0/965235789 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:43.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c49b25640 1 --2- 172.21.15.172:0/965235789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c4415c5a0 0x7f7c440b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c48b23640 1 -- 172.21.15.172:0/965235789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c4415c5a0 msgr2=0x7f7c440b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:43.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c48b23640 1 --2- 172.21.15.172:0/965235789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c4415c5a0 0x7f7c440b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c48b23640 1 -- 172.21.15.172:0/965235789 >> v1:172.21.15.177:6789/0 conn(0x7f7c440b1df0 legacy=0x7f7c440b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.302+0000 7f7c48b23640 1 -- 172.21.15.172:0/965235789 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4415bb40 con 0x7f7c440b21d0 2024-04-14T08:33:43.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.303+0000 7f7c48b23640 1 --2- 172.21.15.172:0/965235789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c440b21d0 0x7f7c4415c060 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009870 tx=0x7f7c3402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73b5f0a18cf0427 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:43.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.303+0000 7f7c3bfff640 1 -- 172.21.15.172:0/965235789 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34034070 con 0x7f7c440b21d0 2024-04-14T08:33:43.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.303+0000 7f7c3bfff640 1 -- 172.21.15.172:0/965235789 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c34030840 con 0x7f7c440b21d0 2024-04-14T08:33:43.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.303+0000 7f7c3bfff640 1 -- 172.21.15.172:0/965235789 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34030c10 con 0x7f7c440b21d0 2024-04-14T08:33:43.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.304+0000 7f7c4b588640 1 -- 172.21.15.172:0/965235789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c440b21d0 msgr2=0x7f7c4415c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.304+0000 7f7c4b588640 1 --2- 172.21.15.172:0/965235789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c440b21d0 0x7f7c4415c060 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009870 tx=0x7f7c3402b980 comp rx=0 tx=0).stop 2024-04-14T08:33:43.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.304+0000 7f7c4b588640 1 -- 172.21.15.172:0/965235789 shutdown_connections 2024-04-14T08:33:43.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.304+0000 7f7c4b588640 1 --2- 172.21.15.172:0/965235789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c4415c5a0 0x7f7c440b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.304+0000 7f7c4b588640 1 --2- 172.21.15.172:0/965235789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c440b21d0 0x7f7c4415c060 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.304+0000 7f7c4b588640 1 -- 172.21.15.172:0/965235789 >> 172.21.15.172:0/965235789 conn(0x7f7c4406ec30 msgr2=0x7f7c4406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:43.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.304+0000 7f7c4b588640 1 -- 172.21.15.172:0/965235789 shutdown_connections 2024-04-14T08:33:43.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.304+0000 7f7c4b588640 1 -- 172.21.15.172:0/965235789 wait complete. 2024-04-14T08:33:43.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.305+0000 7f7c4b588640 1 Processor -- start 2024-04-14T08:33:43.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.305+0000 7f7c4b588640 1 -- start start 2024-04-14T08:33:43.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.305+0000 7f7c4b588640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c440b1df0 0x7f7c44157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.305+0000 7f7c4b588640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c440b21d0 0x7f7c441f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c4b588640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c440b0cc0 0x7f7c440b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c4b588640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7c440b4700 con 0x7f7c440b0cc0 2024-04-14T08:33:43.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c4b588640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7c440b4580 con 0x7f7c440b21d0 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c4b588640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7c440b4880 con 0x7f7c440b1df0 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c49b25640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c440b0cc0 0x7f7c440b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c49b25640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c440b0cc0 0x7f7c440b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59700/0 (socket says 172.21.15.172:59700) 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c49b25640 1 -- 172.21.15.172:0/1734930955 learned_addr learned my addr 172.21.15.172:0/1734930955 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c49324640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c440b1df0 0x7f7c44157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c48b23640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c440b21d0 0x7f7c441f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c49b25640 1 -- 172.21.15.172:0/1734930955 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c440b1df0 msgr2=0x7f7c44157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c49b25640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c440b1df0 0x7f7c44157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c49b25640 1 -- 172.21.15.172:0/1734930955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c440b21d0 msgr2=0x7f7c441f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c49b25640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c440b21d0 0x7f7c441f1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c49b25640 1 -- 172.21.15.172:0/1734930955 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4413ba20 con 0x7f7c440b0cc0 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c49b25640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c440b0cc0 0x7f7c440b1170 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00efc0 tx=0x7f7c3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c39ffb640 1 -- 172.21.15.172:0/1734930955 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c018020 con 0x7f7c440b0cc0 2024-04-14T08:33:43.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c39ffb640 1 -- 172.21.15.172:0/1734930955 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c3c014480 con 0x7f7c440b0cc0 2024-04-14T08:33:43.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c39ffb640 1 -- 172.21.15.172:0/1734930955 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c014870 con 0x7f7c440b0cc0 2024-04-14T08:33:43.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c4b588640 1 -- 172.21.15.172:0/1734930955 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4413bd00 con 0x7f7c440b0cc0 2024-04-14T08:33:43.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.306+0000 7f7c4b588640 1 -- 172.21.15.172:0/1734930955 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c441f21f0 con 0x7f7c440b0cc0 2024-04-14T08:33:43.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.307+0000 7f7c4b588640 1 -- 172.21.15.172:0/1734930955 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c44068520 con 0x7f7c440b0cc0 2024-04-14T08:33:43.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.308+0000 7f7c39ffb640 1 -- 172.21.15.172:0/1734930955 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c3c005d10 con 0x7f7c440b0cc0 2024-04-14T08:33:43.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.308+0000 7f7c39ffb640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7c2806c8f0 0x7f7c2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.308+0000 7f7c39ffb640 1 -- 172.21.15.172:0/1734930955 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7c3c08cbb0 con 0x7f7c440b0cc0 2024-04-14T08:33:43.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.308+0000 7f7c49324640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7c2806c8f0 0x7f7c2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.309+0000 7f7c49324640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7c2806c8f0 0x7f7c2806edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7c40001780 tx=0x7f7c40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:43.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.313+0000 7f7c39ffb640 1 -- 172.21.15.172:0/1734930955 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7c3c022030 con 0x7f7c440b0cc0 2024-04-14T08:33:43.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.491+0000 7f7c4b588640 1 -- 172.21.15.172:0/1734930955 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c4406b8a0 con 0x7f7c440b0cc0 2024-04-14T08:33:43.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.492+0000 7f7c39ffb640 1 -- 172.21.15.172:0/1734930955 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18962 (secure 0 0 0) 0x7f7c3c010040 con 0x7f7c440b0cc0 2024-04-14T08:33:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:43.493 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":27,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:33:34.330426+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:43.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.494+0000 7f7c4b588640 1 -- 172.21.15.172:0/1734930955 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7c2806c8f0 msgr2=0x7f7c2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.494+0000 7f7c4b588640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7c2806c8f0 0x7f7c2806edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7c40001780 tx=0x7f7c40009040 comp rx=0 tx=0).stop 2024-04-14T08:33:43.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.494+0000 7f7c4b588640 1 -- 172.21.15.172:0/1734930955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c440b0cc0 msgr2=0x7f7c440b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.494+0000 7f7c4b588640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c440b0cc0 0x7f7c440b1170 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00efc0 tx=0x7f7c3c00c490 comp rx=0 tx=0).stop 2024-04-14T08:33:43.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.494+0000 7f7c4b588640 1 -- 172.21.15.172:0/1734930955 shutdown_connections 2024-04-14T08:33:43.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.494+0000 7f7c4b588640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7c2806c8f0 0x7f7c2806edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.494+0000 7f7c4b588640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c440b0cc0 0x7f7c440b1170 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.494+0000 7f7c4b588640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c440b21d0 0x7f7c441f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.495+0000 7f7c4b588640 1 --2- 172.21.15.172:0/1734930955 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c440b1df0 0x7f7c44157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.495+0000 7f7c4b588640 1 -- 172.21.15.172:0/1734930955 >> 172.21.15.172:0/1734930955 conn(0x7f7c4406ec30 msgr2=0x7f7c440728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:43.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.495+0000 7f7c4b588640 1 -- 172.21.15.172:0/1734930955 shutdown_connections 2024-04-14T08:33:43.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.495+0000 7f7c4b588640 1 -- 172.21.15.172:0/1734930955 wait complete. 2024-04-14T08:33:43.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:43.507 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:43.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.610+0000 7f4003cde640 1 Processor -- start 2024-04-14T08:33:43.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.610+0000 7f4003cde640 1 -- start start 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4003cde640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ffc159c20 0x7f3ffc0ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4003cde640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3ffc072450 0x7f3ffc0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4003cde640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ffc160b00 con 0x7f3ffc15c700 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4003cde640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f3ffc15fc80 con 0x7f3ffc072450 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4003cde640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3ffc15ff60 con 0x7f3ffc159c20 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4001279640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f3ffc15c700 0x7f3ffc1603f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:51368/0 (socket says 172.21.15.172:51368) 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4001279640 1 -- 172.21.15.172:0/2587908077 learned_addr learned my addr 172.21.15.172:0/2587908077 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f400227b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3ffc072450 0x7f3ffc0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4001a7a640 1 --2- 172.21.15.172:0/2587908077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ffc159c20 0x7f3ffc0ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4000a78640 1 -- 172.21.15.172:0/2587908077 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3215938501 0 0) 0x7f3ffc160b00 con 0x7f3ffc15c700 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4001a7a640 1 -- 172.21.15.172:0/2587908077 >> v2:172.21.15.177:3300/0 conn(0x7f3ffc072450 msgr2=0x7f3ffc0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4001a7a640 1 --2- 172.21.15.172:0/2587908077 >> v2:172.21.15.177:3300/0 conn(0x7f3ffc072450 0x7f3ffc0728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4001a7a640 1 -- 172.21.15.172:0/2587908077 >> v1:172.21.15.177:6789/0 conn(0x7f3ffc15c700 legacy=0x7f3ffc1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.611+0000 7f4001a7a640 1 -- 172.21.15.172:0/2587908077 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc15cf10 con 0x7f3ffc159c20 2024-04-14T08:33:43.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.612+0000 7f4001a7a640 1 --2- 172.21.15.172:0/2587908077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ffc159c20 0x7f3ffc0ce200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009d90 tx=0x7f3ff402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3a3f360130b04e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:43.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.612+0000 7f4000a78640 1 -- 172.21.15.172:0/2587908077 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4039070 con 0x7f3ffc159c20 2024-04-14T08:33:43.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.612+0000 7f4000a78640 1 -- 172.21.15.172:0/2587908077 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff402cc50 con 0x7f3ffc159c20 2024-04-14T08:33:43.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.612+0000 7f4000a78640 1 -- 172.21.15.172:0/2587908077 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4034440 con 0x7f3ffc159c20 2024-04-14T08:33:43.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.613+0000 7f4003cde640 1 -- 172.21.15.172:0/2587908077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ffc159c20 msgr2=0x7f3ffc0ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.613+0000 7f4003cde640 1 --2- 172.21.15.172:0/2587908077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ffc159c20 0x7f3ffc0ce200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009d90 tx=0x7f3ff402bea0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.613+0000 7f4003cde640 1 -- 172.21.15.172:0/2587908077 shutdown_connections 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.613+0000 7f4003cde640 1 --2- 172.21.15.172:0/2587908077 >> v2:172.21.15.177:3300/0 conn(0x7f3ffc072450 0x7f3ffc0728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.613+0000 7f4003cde640 1 --2- 172.21.15.172:0/2587908077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ffc159c20 0x7f3ffc0ce200 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.613+0000 7f4003cde640 1 -- 172.21.15.172:0/2587908077 >> 172.21.15.172:0/2587908077 conn(0x7f3ffc06b520 msgr2=0x7f3ffc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.613+0000 7f4003cde640 1 -- 172.21.15.172:0/2587908077 shutdown_connections 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.613+0000 7f4003cde640 1 -- 172.21.15.172:0/2587908077 wait complete. 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.614+0000 7f4003cde640 1 Processor -- start 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.614+0000 7f4003cde640 1 -- start start 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4003cde640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ffc072450 0x7f3ffc143b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4003cde640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ffc159c20 0x7f3ffc0aa850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4003cde640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ffc15c700 0x7f3ffc0b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4003cde640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc15a000 con 0x7f3ffc159c20 2024-04-14T08:33:43.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4003cde640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc15d4d0 con 0x7f3ffc072450 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4003cde640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3ffc15cb80 con 0x7f3ffc15c700 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4001279640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ffc159c20 0x7f3ffc0aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4001279640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ffc159c20 0x7f3ffc0aa850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59712/0 (socket says 172.21.15.172:59712) 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4001279640 1 -- 172.21.15.172:0/1716015377 learned_addr learned my addr 172.21.15.172:0/1716015377 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4001a7a640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ffc072450 0x7f3ffc143b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f400227b640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ffc15c700 0x7f3ffc0b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4001279640 1 -- 172.21.15.172:0/1716015377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ffc15c700 msgr2=0x7f3ffc0b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4001279640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ffc15c700 0x7f3ffc0b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4001279640 1 -- 172.21.15.172:0/1716015377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ffc072450 msgr2=0x7f3ffc143b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4001279640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ffc072450 0x7f3ffc143b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f4001279640 1 -- 172.21.15.172:0/1716015377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc0a94d0 con 0x7f3ffc159c20 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.615+0000 7f400227b640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ffc15c700 0x7f3ffc0b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:43.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.616+0000 7f4001279640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ffc159c20 0x7f3ffc0aa850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00c460 tx=0x7f3fec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:43.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.616+0000 7f3ff27fc640 1 -- 172.21.15.172:0/1716015377 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec018070 con 0x7f3ffc159c20 2024-04-14T08:33:43.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.616+0000 7f3ff27fc640 1 -- 172.21.15.172:0/1716015377 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fec007500 con 0x7f3ffc159c20 2024-04-14T08:33:43.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.616+0000 7f3ff27fc640 1 -- 172.21.15.172:0/1716015377 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec004a20 con 0x7f3ffc159c20 2024-04-14T08:33:43.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.616+0000 7f4003cde640 1 -- 172.21.15.172:0/1716015377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc0a97b0 con 0x7f3ffc159c20 2024-04-14T08:33:43.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.616+0000 7f4003cde640 1 -- 172.21.15.172:0/1716015377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc0a9d70 con 0x7f3ffc159c20 2024-04-14T08:33:43.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.617+0000 7f4003cde640 1 -- 172.21.15.172:0/1716015377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ffc068590 con 0x7f3ffc159c20 2024-04-14T08:33:43.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.617+0000 7f3ff27fc640 1 -- 172.21.15.172:0/1716015377 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3fec01b070 con 0x7f3ffc159c20 2024-04-14T08:33:43.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.617+0000 7f3ff27fc640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3fe006c8f0 0x7f3fe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.618+0000 7f3ff27fc640 1 -- 172.21.15.172:0/1716015377 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3fec08b100 con 0x7f3ffc159c20 2024-04-14T08:33:43.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.618+0000 7f4001a7a640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3fe006c8f0 0x7f3fe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.618+0000 7f4001a7a640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3fe006c8f0 0x7f3fe006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4036000 tx=0x7f3ff4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:43.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.622+0000 7f3ff27fc640 1 -- 172.21.15.172:0/1716015377 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3fec058ae0 con 0x7f3ffc159c20 2024-04-14T08:33:43.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.796+0000 7f4003cde640 1 -- 172.21.15.172:0/1716015377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ffc15b200 con 0x7f3ffc159c20 2024-04-14T08:33:43.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.797+0000 7f3ff27fc640 1 -- 172.21.15.172:0/1716015377 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f3fec05cde0 con 0x7f3ffc159c20 2024-04-14T08:33:43.797 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:43.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.799+0000 7f4003cde640 1 -- 172.21.15.172:0/1716015377 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3fe006c8f0 msgr2=0x7f3fe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.799+0000 7f4003cde640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3fe006c8f0 0x7f3fe006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4036000 tx=0x7f3ff4030000 comp rx=0 tx=0).stop 2024-04-14T08:33:43.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.799+0000 7f4003cde640 1 -- 172.21.15.172:0/1716015377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ffc159c20 msgr2=0x7f3ffc0aa850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.799+0000 7f4003cde640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ffc159c20 0x7f3ffc0aa850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00c460 tx=0x7f3fec00c920 comp rx=0 tx=0).stop 2024-04-14T08:33:43.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.799+0000 7f4003cde640 1 -- 172.21.15.172:0/1716015377 shutdown_connections 2024-04-14T08:33:43.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.799+0000 7f4003cde640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3fe006c8f0 0x7f3fe006edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.799+0000 7f4003cde640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ffc15c700 0x7f3ffc0b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.799+0000 7f4003cde640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ffc159c20 0x7f3ffc0aa850 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.799+0000 7f4003cde640 1 --2- 172.21.15.172:0/1716015377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ffc072450 0x7f3ffc143b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.799+0000 7f4003cde640 1 -- 172.21.15.172:0/1716015377 >> 172.21.15.172:0/1716015377 conn(0x7f3ffc06b520 msgr2=0x7f3ffc15ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:43.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.799+0000 7f4003cde640 1 -- 172.21.15.172:0/1716015377 shutdown_connections 2024-04-14T08:33:43.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.799+0000 7f4003cde640 1 -- 172.21.15.172:0/1716015377 wait complete. 2024-04-14T08:33:43.810 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:43.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.913+0000 7fdd60faa640 1 Processor -- start 2024-04-14T08:33:43.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.913+0000 7fdd60faa640 1 -- start start 2024-04-14T08:33:43.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.914+0000 7fdd60faa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdd5c15e430 0x7fdd5c162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.914+0000 7fdd60faa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c162540 0x7fdd5c1629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.914+0000 7fdd60faa640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd5c0cefb0 con 0x7fdd5c0b3a30 2024-04-14T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.914+0000 7fdd60faa640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fdd5c0cf290 con 0x7fdd5c15e430 2024-04-14T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.914+0000 7fdd60faa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdd5c0cf570 con 0x7fdd5c162540 2024-04-14T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.914+0000 7fdd5a59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdd5c15e430 0x7fdd5c162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.914+0000 7fdd5ad9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fdd5c0b3a30 0x7fdd5c0b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54118/0 (socket says 172.21.15.172:54118) 2024-04-14T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.914+0000 7fdd5a59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdd5c15e430 0x7fdd5c162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59718/0 (socket says 172.21.15.172:59718) 2024-04-14T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.914+0000 7fdd5ad9d640 1 -- 172.21.15.172:0/3135699848 learned_addr learned my addr 172.21.15.172:0/3135699848 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.914+0000 7fdd59d9b640 1 --2- 172.21.15.172:0/3135699848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c162540 0x7fdd5c1629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.915+0000 7fdd5a59c640 1 -- 172.21.15.172:0/3135699848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c162540 msgr2=0x7fdd5c1629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.915+0000 7fdd5a59c640 1 --2- 172.21.15.172:0/3135699848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c162540 0x7fdd5c1629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.915+0000 7fdd5a59c640 1 -- 172.21.15.172:0/3135699848 >> v1:172.21.15.172:6789/0 conn(0x7fdd5c0b3a30 legacy=0x7fdd5c0b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.915+0000 7fdd5a59c640 1 -- 172.21.15.172:0/3135699848 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5c0cf850 con 0x7fdd5c15e430 2024-04-14T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.915+0000 7fdd5959a640 1 -- 172.21.15.172:0/3135699848 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3245528495 0 0) 0x7fdd5c0cefb0 con 0x7fdd5c0b3a30 2024-04-14T08:33:43.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.915+0000 7fdd5a59c640 1 --2- 172.21.15.172:0/3135699848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd5c15e430 0x7fdd5c162000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009d90 tx=0x7fdd4c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59875d42260e674 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:43.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.916+0000 7fdd5959a640 1 -- 172.21.15.172:0/3135699848 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c039070 con 0x7fdd5c15e430 2024-04-14T08:33:43.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.916+0000 7fdd5959a640 1 -- 172.21.15.172:0/3135699848 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4c02cc50 con 0x7fdd5c15e430 2024-04-14T08:33:43.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.916+0000 7fdd5959a640 1 -- 172.21.15.172:0/3135699848 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c034440 con 0x7fdd5c15e430 2024-04-14T08:33:43.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.916+0000 7fdd60faa640 1 -- 172.21.15.172:0/3135699848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd5c15e430 msgr2=0x7fdd5c162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.916+0000 7fdd60faa640 1 --2- 172.21.15.172:0/3135699848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd5c15e430 0x7fdd5c162000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009d90 tx=0x7fdd4c02bea0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.916+0000 7fdd60faa640 1 -- 172.21.15.172:0/3135699848 shutdown_connections 2024-04-14T08:33:43.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.916+0000 7fdd60faa640 1 --2- 172.21.15.172:0/3135699848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c162540 0x7fdd5c1629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.916+0000 7fdd60faa640 1 --2- 172.21.15.172:0/3135699848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd5c15e430 0x7fdd5c162000 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.916+0000 7fdd60faa640 1 -- 172.21.15.172:0/3135699848 >> 172.21.15.172:0/3135699848 conn(0x7fdd5c09d560 msgr2=0x7fdd5c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.916+0000 7fdd60faa640 1 -- 172.21.15.172:0/3135699848 shutdown_connections 2024-04-14T08:33:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.916+0000 7fdd60faa640 1 -- 172.21.15.172:0/3135699848 wait complete. 2024-04-14T08:33:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.917+0000 7fdd60faa640 1 Processor -- start 2024-04-14T08:33:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.918+0000 7fdd60faa640 1 -- start start 2024-04-14T08:33:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.918+0000 7fdd60faa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd5c0b3a30 0x7fdd5c14bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.918+0000 7fdd60faa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd5c15e430 0x7fdd5c1f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.918+0000 7fdd60faa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c162540 0x7fdd5c1f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.918+0000 7fdd60faa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c0d4e20 con 0x7fdd5c0b3a30 2024-04-14T08:33:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.918+0000 7fdd60faa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c15e810 con 0x7fdd5c15e430 2024-04-14T08:33:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.918+0000 7fdd60faa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdd5c0d0050 con 0x7fdd5c162540 2024-04-14T08:33:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.918+0000 7fdd5a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd5c0b3a30 0x7fdd5c14bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.918+0000 7fdd5a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd5c0b3a30 0x7fdd5c14bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59734/0 (socket says 172.21.15.172:59734) 2024-04-14T08:33:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.918+0000 7fdd5a59c640 1 -- 172.21.15.172:0/1381472315 learned_addr learned my addr 172.21.15.172:0/1381472315 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.918+0000 7fdd59d9b640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd5c15e430 0x7fdd5c1f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.918+0000 7fdd5ad9d640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c162540 0x7fdd5c1f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.919+0000 7fdd5a59c640 1 -- 172.21.15.172:0/1381472315 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c162540 msgr2=0x7fdd5c1f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.919+0000 7fdd5a59c640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c162540 0x7fdd5c1f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.919+0000 7fdd5a59c640 1 -- 172.21.15.172:0/1381472315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd5c15e430 msgr2=0x7fdd5c1f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.919+0000 7fdd5a59c640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd5c15e430 0x7fdd5c1f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.919+0000 7fdd5a59c640 1 -- 172.21.15.172:0/1381472315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5c14ec40 con 0x7fdd5c0b3a30 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.919+0000 7fdd5a59c640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd5c0b3a30 0x7fdd5c14bd50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c037040 tx=0x7fdd4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.919+0000 7fdd4affd640 1 -- 172.21.15.172:0/1381472315 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c039050 con 0x7fdd5c0b3a30 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.919+0000 7fdd4affd640 1 -- 172.21.15.172:0/1381472315 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4c03a040 con 0x7fdd5c0b3a30 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.919+0000 7fdd60faa640 1 -- 172.21.15.172:0/1381472315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd5c1f4a50 con 0x7fdd5c0b3a30 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.919+0000 7fdd4affd640 1 -- 172.21.15.172:0/1381472315 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c009450 con 0x7fdd5c0b3a30 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.919+0000 7fdd60faa640 1 -- 172.21.15.172:0/1381472315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd5c14cb60 con 0x7fdd5c0b3a30 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.920+0000 7fdd60faa640 1 -- 172.21.15.172:0/1381472315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd5c068590 con 0x7fdd5c0b3a30 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.920+0000 7fdd4affd640 1 -- 172.21.15.172:0/1381472315 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd4c047020 con 0x7fdd5c0b3a30 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.921+0000 7fdd4affd640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdd3c06c8f0 0x7fdd3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.921+0000 7fdd4affd640 1 -- 172.21.15.172:0/1381472315 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdd4c0adee0 con 0x7fdd5c0b3a30 2024-04-14T08:33:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.921+0000 7fdd59d9b640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdd3c06c8f0 0x7fdd3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.921+0000 7fdd59d9b640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdd3c06c8f0 0x7fdd3c06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0a20e0 tx=0x7fdd50009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:43.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:43.925+0000 7fdd4affd640 1 -- 172.21.15.172:0/1381472315 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdd4c04e020 con 0x7fdd5c0b3a30 2024-04-14T08:33:44.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.101+0000 7fdd60faa640 1 -- 172.21.15.172:0/1381472315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdd5c0b34a0 con 0x7fdd5c0b3a30 2024-04-14T08:33:44.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.101+0000 7fdd4affd640 1 -- 172.21.15.172:0/1381472315 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fdd4c07b8c0 con 0x7fdd5c0b3a30 2024-04-14T08:33:44.102 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:44.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.103+0000 7fdd60faa640 1 -- 172.21.15.172:0/1381472315 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdd3c06c8f0 msgr2=0x7fdd3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.103+0000 7fdd60faa640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdd3c06c8f0 0x7fdd3c06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0a20e0 tx=0x7fdd50009190 comp rx=0 tx=0).stop 2024-04-14T08:33:44.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.103+0000 7fdd60faa640 1 -- 172.21.15.172:0/1381472315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd5c0b3a30 msgr2=0x7fdd5c14bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.103+0000 7fdd60faa640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd5c0b3a30 0x7fdd5c14bd50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c037040 tx=0x7fdd4c004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:44.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.103+0000 7fdd60faa640 1 -- 172.21.15.172:0/1381472315 shutdown_connections 2024-04-14T08:33:44.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.103+0000 7fdd60faa640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdd3c06c8f0 0x7fdd3c06edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.103+0000 7fdd60faa640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd5c162540 0x7fdd5c1f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.103+0000 7fdd60faa640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd5c15e430 0x7fdd5c1f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.103+0000 7fdd60faa640 1 --2- 172.21.15.172:0/1381472315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd5c0b3a30 0x7fdd5c14bd50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.103+0000 7fdd60faa640 1 -- 172.21.15.172:0/1381472315 >> 172.21.15.172:0/1381472315 conn(0x7fdd5c09d560 msgr2=0x7fdd5c0d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:44.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.103+0000 7fdd60faa640 1 -- 172.21.15.172:0/1381472315 shutdown_connections 2024-04-14T08:33:44.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.103+0000 7fdd60faa640 1 -- 172.21.15.172:0/1381472315 wait complete. 2024-04-14T08:33:44.115 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:44.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.219+0000 7fafd875e640 1 Processor -- start 2024-04-14T08:33:44.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.219+0000 7fafd875e640 1 -- start start 2024-04-14T08:33:44.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.219+0000 7fafd875e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fafd013df30 0x7fafd00b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.219+0000 7fafd875e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd0140e80 0x7fafd00b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd875e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd015b620 con 0x7fafd015c380 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd875e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fafd00cf9b0 con 0x7fafd013df30 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd875e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fafd01414a0 con 0x7fafd0140e80 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd64fa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fafd013df30 0x7fafd00b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd5cf9640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fafd015c380 0x7fafd0140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54128/0 (socket says 172.21.15.172:54128) 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd64fa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fafd013df30 0x7fafd00b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59744/0 (socket says 172.21.15.172:59744) 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd5cf9640 1 -- 172.21.15.172:0/2191277905 learned_addr learned my addr 172.21.15.172:0/2191277905 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd6cfb640 1 --2- 172.21.15.172:0/2191277905 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd0140e80 0x7fafd00b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd64fa640 1 -- 172.21.15.172:0/2191277905 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd0140e80 msgr2=0x7fafd00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd64fa640 1 --2- 172.21.15.172:0/2191277905 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd0140e80 0x7fafd00b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd64fa640 1 -- 172.21.15.172:0/2191277905 >> v1:172.21.15.172:6789/0 conn(0x7fafd015c380 legacy=0x7fafd0140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd64fa640 1 -- 172.21.15.172:0/2191277905 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd015b8a0 con 0x7fafd013df30 2024-04-14T08:33:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.220+0000 7fafd54f8640 1 -- 172.21.15.172:0/2191277905 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3464211333 0 0) 0x7fafc0001110 con 0x7fafd015c380 2024-04-14T08:33:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.221+0000 7fafd64fa640 1 --2- 172.21.15.172:0/2191277905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd013df30 0x7fafd00b1cd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009d90 tx=0x7fafcc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f080d62ef2dbbf6b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.221+0000 7fafd54f8640 1 -- 172.21.15.172:0/2191277905 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc039070 con 0x7fafd013df30 2024-04-14T08:33:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.221+0000 7fafd54f8640 1 -- 172.21.15.172:0/2191277905 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafcc02cc50 con 0x7fafd013df30 2024-04-14T08:33:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.221+0000 7fafd54f8640 1 -- 172.21.15.172:0/2191277905 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc034440 con 0x7fafd013df30 2024-04-14T08:33:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.221+0000 7fafd875e640 1 -- 172.21.15.172:0/2191277905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd013df30 msgr2=0x7fafd00b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.221+0000 7fafd875e640 1 --2- 172.21.15.172:0/2191277905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd013df30 0x7fafd00b1cd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009d90 tx=0x7fafcc02bea0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.221+0000 7fafd875e640 1 -- 172.21.15.172:0/2191277905 shutdown_connections 2024-04-14T08:33:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.221+0000 7fafd875e640 1 --2- 172.21.15.172:0/2191277905 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd0140e80 0x7fafd00b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.221+0000 7fafd875e640 1 --2- 172.21.15.172:0/2191277905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd013df30 0x7fafd00b1cd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.222+0000 7fafd875e640 1 -- 172.21.15.172:0/2191277905 >> 172.21.15.172:0/2191277905 conn(0x7fafd006b520 msgr2=0x7fafd006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.222+0000 7fafd875e640 1 -- 172.21.15.172:0/2191277905 shutdown_connections 2024-04-14T08:33:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.222+0000 7fafd875e640 1 -- 172.21.15.172:0/2191277905 wait complete. 2024-04-14T08:33:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.223+0000 7fafd875e640 1 Processor -- start 2024-04-14T08:33:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.223+0000 7fafd875e640 1 -- start start 2024-04-14T08:33:44.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.223+0000 7fafd875e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd013df30 0x7fafd0151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.223+0000 7fafd875e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd0140e80 0x7fafd0159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.223+0000 7fafd875e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafd015c380 0x7fafd0159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.223+0000 7fafd875e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fafd013e310 con 0x7fafd0140e80 2024-04-14T08:33:44.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.223+0000 7fafd875e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fafd015be60 con 0x7fafd015c380 2024-04-14T08:33:44.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.223+0000 7fafd875e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fafd00ce9a0 con 0x7fafd013df30 2024-04-14T08:33:44.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.223+0000 7fafd5cf9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd0140e80 0x7fafd0159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafd64fa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd013df30 0x7fafd0151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafd5cf9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd0140e80 0x7fafd0159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59760/0 (socket says 172.21.15.172:59760) 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafd5cf9640 1 -- 172.21.15.172:0/2393851286 learned_addr learned my addr 172.21.15.172:0/2393851286 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafd6cfb640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafd015c380 0x7fafd0159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafd5cf9640 1 -- 172.21.15.172:0/2393851286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd013df30 msgr2=0x7fafd0151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafd5cf9640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd013df30 0x7fafd0151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafd5cf9640 1 -- 172.21.15.172:0/2393851286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafd015c380 msgr2=0x7fafd0159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafd5cf9640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafd015c380 0x7fafd0159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafd5cf9640 1 -- 172.21.15.172:0/2393851286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd013be00 con 0x7fafd0140e80 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafd5cf9640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd0140e80 0x7fafd0159240 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c3f0 tx=0x7fafc000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafc6ffd640 1 -- 172.21.15.172:0/2393851286 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00074e0 con 0x7fafd0140e80 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafc6ffd640 1 -- 172.21.15.172:0/2393851286 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc00047e0 con 0x7fafd0140e80 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafd875e640 1 -- 172.21.15.172:0/2393851286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd0151690 con 0x7fafd0140e80 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafc6ffd640 1 -- 172.21.15.172:0/2393851286 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0004c10 con 0x7fafd0140e80 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.224+0000 7fafd875e640 1 -- 172.21.15.172:0/2393851286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd015a100 con 0x7fafd0140e80 2024-04-14T08:33:44.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.225+0000 7fafd875e640 1 -- 172.21.15.172:0/2393851286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd015e7b0 con 0x7fafd0140e80 2024-04-14T08:33:44.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.226+0000 7fafc6ffd640 1 -- 172.21.15.172:0/2393851286 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafc0011070 con 0x7fafd0140e80 2024-04-14T08:33:44.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.226+0000 7fafc6ffd640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fafb406c8f0 0x7fafb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.226+0000 7fafc6ffd640 1 -- 172.21.15.172:0/2393851286 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fafc008b930 con 0x7fafd0140e80 2024-04-14T08:33:44.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.226+0000 7fafd64fa640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fafb406c8f0 0x7fafb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.227+0000 7fafd64fa640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fafb406c8f0 0x7fafb406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fafcc036000 tx=0x7fafcc030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:44.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.230+0000 7fafc6ffd640 1 -- 172.21.15.172:0/2393851286 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fafd015e7b0 con 0x7fafd0140e80 2024-04-14T08:33:44.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.406+0000 7fafd875e640 1 -- 172.21.15.172:0/2393851286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fafd0068590 con 0x7fafd0140e80 2024-04-14T08:33:44.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.406+0000 7fafc6ffd640 1 -- 172.21.15.172:0/2393851286 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fafd0151690 con 0x7fafd0140e80 2024-04-14T08:33:44.407 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.408+0000 7fafd875e640 1 -- 172.21.15.172:0/2393851286 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fafb406c8f0 msgr2=0x7fafb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.408+0000 7fafd875e640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fafb406c8f0 0x7fafb406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fafcc036000 tx=0x7fafcc030000 comp rx=0 tx=0).stop 2024-04-14T08:33:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.408+0000 7fafd875e640 1 -- 172.21.15.172:0/2393851286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd0140e80 msgr2=0x7fafd0159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.408+0000 7fafd875e640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd0140e80 0x7fafd0159240 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c3f0 tx=0x7fafc000c8b0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.408+0000 7fafd875e640 1 -- 172.21.15.172:0/2393851286 shutdown_connections 2024-04-14T08:33:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.408+0000 7fafd875e640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fafb406c8f0 0x7fafb406edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.408+0000 7fafd875e640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafd015c380 0x7fafd0159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.408+0000 7fafd875e640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd0140e80 0x7fafd0159240 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.408+0000 7fafd875e640 1 --2- 172.21.15.172:0/2393851286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafd013df30 0x7fafd0151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.408+0000 7fafd875e640 1 -- 172.21.15.172:0/2393851286 >> 172.21.15.172:0/2393851286 conn(0x7fafd006b520 msgr2=0x7fafd00728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.408+0000 7fafd875e640 1 -- 172.21.15.172:0/2393851286 shutdown_connections 2024-04-14T08:33:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.408+0000 7fafd875e640 1 -- 172.21.15.172:0/2393851286 wait complete. 2024-04-14T08:33:44.420 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:44.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.523+0000 7fa1a56d1640 1 Processor -- start 2024-04-14T08:33:44.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.523+0000 7fa1a56d1640 1 -- start start 2024-04-14T08:33:44.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.523+0000 7fa1a56d1640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa1a015e650 0x7fa1a00cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.523+0000 7fa1a56d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a00d2100 0x7fa1a00d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.523+0000 7fa1a56d1640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a00b3830 con 0x7fa1a00d2d60 2024-04-14T08:33:44.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.523+0000 7fa1a56d1640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a00b4160 con 0x7fa1a015e650 2024-04-14T08:33:44.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.523+0000 7fa1a56d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a00b3b10 con 0x7fa1a00d2100 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.524+0000 7fa19f7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa1a00d2d60 0x7fa1a00d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54134/0 (socket says 172.21.15.172:54134) 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.524+0000 7fa19f7fe640 1 -- 172.21.15.172:0/2027673185 learned_addr learned my addr 172.21.15.172:0/2027673185 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.524+0000 7fa19e7fc640 1 --2- 172.21.15.172:0/2027673185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a00d2100 0x7fa1a00d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.524+0000 7fa19effd640 1 --2- 172.21.15.172:0/2027673185 >> v2:172.21.15.177:3300/0 conn(0x7fa1a015e650 0x7fa1a00cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.524+0000 7fa19dffb640 1 -- 172.21.15.172:0/2027673185 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2753188076 0 0) 0x7fa1a00b3830 con 0x7fa1a00d2d60 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.524+0000 7fa19dffb640 1 -- 172.21.15.172:0/2027673185 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa188003520 con 0x7fa1a00d2d60 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.524+0000 7fa19effd640 1 -- 172.21.15.172:0/2027673185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a00d2100 msgr2=0x7fa1a00d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.524+0000 7fa19effd640 1 --2- 172.21.15.172:0/2027673185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a00d2100 0x7fa1a00d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.524+0000 7fa19effd640 1 -- 172.21.15.172:0/2027673185 >> v1:172.21.15.172:6789/0 conn(0x7fa1a00d2d60 legacy=0x7fa1a00d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.524+0000 7fa19effd640 1 -- 172.21.15.172:0/2027673185 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a00b3df0 con 0x7fa1a015e650 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.524+0000 7fa19e7fc640 1 --2- 172.21.15.172:0/2027673185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a00d2100 0x7fa1a00d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.525+0000 7fa19effd640 1 --2- 172.21.15.172:0/2027673185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a015e650 0x7fa1a00cf860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa1900097a0 tx=0x7fa19002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51945dce00e4e1d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:44.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.525+0000 7fa19dffb640 1 -- 172.21.15.172:0/2027673185 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190034070 con 0x7fa1a015e650 2024-04-14T08:33:44.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.525+0000 7fa19dffb640 1 -- 172.21.15.172:0/2027673185 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa190030610 con 0x7fa1a015e650 2024-04-14T08:33:44.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.525+0000 7fa19dffb640 1 -- 172.21.15.172:0/2027673185 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1900309c0 con 0x7fa1a015e650 2024-04-14T08:33:44.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.525+0000 7fa1a56d1640 1 -- 172.21.15.172:0/2027673185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a015e650 msgr2=0x7fa1a00cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.525+0000 7fa1a56d1640 1 --2- 172.21.15.172:0/2027673185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a015e650 0x7fa1a00cf860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa1900097a0 tx=0x7fa19002b960 comp rx=0 tx=0).stop 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.525+0000 7fa1a56d1640 1 -- 172.21.15.172:0/2027673185 shutdown_connections 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.525+0000 7fa1a56d1640 1 --2- 172.21.15.172:0/2027673185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a00d2100 0x7fa1a00d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.525+0000 7fa1a56d1640 1 --2- 172.21.15.172:0/2027673185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a015e650 0x7fa1a00cf860 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.525+0000 7fa1a56d1640 1 -- 172.21.15.172:0/2027673185 >> 172.21.15.172:0/2027673185 conn(0x7fa1a009d580 msgr2=0x7fa1a00d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.526+0000 7fa1a56d1640 1 -- 172.21.15.172:0/2027673185 shutdown_connections 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.526+0000 7fa1a56d1640 1 -- 172.21.15.172:0/2027673185 wait complete. 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.527+0000 7fa1a56d1640 1 Processor -- start 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.527+0000 7fa1a56d1640 1 -- start start 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.527+0000 7fa1a56d1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a00d2100 0x7fa1a014f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.527+0000 7fa1a56d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a00d2d60 0x7fa1a00cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa1a56d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a015e650 0x7fa1a0161330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa1a56d1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a00cff20 con 0x7fa1a015e650 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa1a56d1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a00cfda0 con 0x7fa1a00d2100 2024-04-14T08:33:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa1a56d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a00b5600 con 0x7fa1a00d2d60 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa19f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a015e650 0x7fa1a0161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa19effd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a00d2100 0x7fa1a014f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa19f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a015e650 0x7fa1a0161330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59768/0 (socket says 172.21.15.172:59768) 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa19f7fe640 1 -- 172.21.15.172:0/4180067626 learned_addr learned my addr 172.21.15.172:0/4180067626 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa19e7fc640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a00d2d60 0x7fa1a00cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa19f7fe640 1 -- 172.21.15.172:0/4180067626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a00d2d60 msgr2=0x7fa1a00cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa19f7fe640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a00d2d60 0x7fa1a00cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa19f7fe640 1 -- 172.21.15.172:0/4180067626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a00d2100 msgr2=0x7fa1a014f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa19f7fe640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a00d2100 0x7fa1a014f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa19f7fe640 1 -- 172.21.15.172:0/4180067626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a015bde0 con 0x7fa1a015e650 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa19f7fe640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a015e650 0x7fa1a0161330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa19800cb70 tx=0x7fa19800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa1877fe640 1 -- 172.21.15.172:0/4180067626 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198017070 con 0x7fa1a015e650 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa1877fe640 1 -- 172.21.15.172:0/4180067626 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19800ad40 con 0x7fa1a015e650 2024-04-14T08:33:44.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa1a56d1640 1 -- 172.21.15.172:0/4180067626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a0160290 con 0x7fa1a015e650 2024-04-14T08:33:44.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa1877fe640 1 -- 172.21.15.172:0/4180067626 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198012620 con 0x7fa1a015e650 2024-04-14T08:33:44.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.528+0000 7fa1a56d1640 1 -- 172.21.15.172:0/4180067626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a0160850 con 0x7fa1a015e650 2024-04-14T08:33:44.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.529+0000 7fa1a56d1640 1 -- 172.21.15.172:0/4180067626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a0068590 con 0x7fa1a015e650 2024-04-14T08:33:44.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.530+0000 7fa1877fe640 1 -- 172.21.15.172:0/4180067626 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa19801a070 con 0x7fa1a015e650 2024-04-14T08:33:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.530+0000 7fa1877fe640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa18c06c920 0x7fa18c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.530+0000 7fa1877fe640 1 -- 172.21.15.172:0/4180067626 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa19808bcf0 con 0x7fa1a015e650 2024-04-14T08:33:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.530+0000 7fa19effd640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa18c06c920 0x7fa18c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.531+0000 7fa19effd640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa18c06c920 0x7fa18c06ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa190036000 tx=0x7fa19003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.534+0000 7fa1877fe640 1 -- 172.21.15.172:0/4180067626 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa198090070 con 0x7fa1a015e650 2024-04-14T08:33:44.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.710+0000 7fa1a56d1640 1 -- 172.21.15.172:0/4180067626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa1a00b34f0 con 0x7fa1a015e650 2024-04-14T08:33:44.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.710+0000 7fa1877fe640 1 -- 172.21.15.172:0/4180067626 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fa1980596d0 con 0x7fa1a015e650 2024-04-14T08:33:44.711 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.712+0000 7fa1a56d1640 1 -- 172.21.15.172:0/4180067626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa18c06c920 msgr2=0x7fa18c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.712+0000 7fa1a56d1640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa18c06c920 0x7fa18c06ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa190036000 tx=0x7fa19003b040 comp rx=0 tx=0).stop 2024-04-14T08:33:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.712+0000 7fa1a56d1640 1 -- 172.21.15.172:0/4180067626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a015e650 msgr2=0x7fa1a0161330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.712+0000 7fa1a56d1640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a015e650 0x7fa1a0161330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa19800cb70 tx=0x7fa19800a430 comp rx=0 tx=0).stop 2024-04-14T08:33:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.712+0000 7fa1a56d1640 1 -- 172.21.15.172:0/4180067626 shutdown_connections 2024-04-14T08:33:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.712+0000 7fa1a56d1640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa18c06c920 0x7fa18c06ede0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.712+0000 7fa1a56d1640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a015e650 0x7fa1a0161330 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.712+0000 7fa1a56d1640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1a00d2d60 0x7fa1a00cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.712+0000 7fa1a56d1640 1 --2- 172.21.15.172:0/4180067626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1a00d2100 0x7fa1a014f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.712+0000 7fa1a56d1640 1 -- 172.21.15.172:0/4180067626 >> 172.21.15.172:0/4180067626 conn(0x7fa1a009d580 msgr2=0x7fa1a006d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:44.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.712+0000 7fa1a56d1640 1 -- 172.21.15.172:0/4180067626 shutdown_connections 2024-04-14T08:33:44.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.712+0000 7fa1a56d1640 1 -- 172.21.15.172:0/4180067626 wait complete. 2024-04-14T08:33:44.724 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:33:44.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.825+0000 7f4d0c5bb640 1 Processor -- start 2024-04-14T08:33:44.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.825+0000 7f4d0c5bb640 1 -- start start 2024-04-14T08:33:44.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.826+0000 7f4d0c5bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d04162710 0x7f4d040d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.826+0000 7f4d0c5bb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4d04162ba0 0x7f4d040d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.826+0000 7f4d0c5bb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d040cefb0 con 0x7f4d0415e430 2024-04-14T08:33:44.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.826+0000 7f4d0c5bb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f4d040cf290 con 0x7f4d04162710 2024-04-14T08:33:44.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.826+0000 7f4d0c5bb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f4d040cf570 con 0x7f4d04162ba0 2024-04-14T08:33:44.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.826+0000 7f4d09b56640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d04162710 0x7f4d040d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.826+0000 7f4d0a357640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4d0415e430 0x7f4d04162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54150/0 (socket says 172.21.15.172:54150) 2024-04-14T08:33:44.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.826+0000 7f4d09b56640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d04162710 0x7f4d040d1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59784/0 (socket says 172.21.15.172:59784) 2024-04-14T08:33:44.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.826+0000 7f4d0a357640 1 -- 172.21.15.172:0/2134058422 learned_addr learned my addr 172.21.15.172:0/2134058422 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:44.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.826+0000 7f4d0ab58640 1 --2- 172.21.15.172:0/2134058422 >> v2:172.21.15.177:3300/0 conn(0x7f4d04162ba0 0x7f4d040d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.827+0000 7f4d09b56640 1 -- 172.21.15.172:0/2134058422 >> v2:172.21.15.177:3300/0 conn(0x7f4d04162ba0 msgr2=0x7f4d040d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:44.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.827+0000 7f4d09b56640 1 --2- 172.21.15.172:0/2134058422 >> v2:172.21.15.177:3300/0 conn(0x7f4d04162ba0 0x7f4d040d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.827+0000 7f4d09b56640 1 -- 172.21.15.172:0/2134058422 >> v1:172.21.15.172:6789/0 conn(0x7f4d0415e430 legacy=0x7f4d04162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.827+0000 7f4d09b56640 1 -- 172.21.15.172:0/2134058422 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d040cf850 con 0x7f4d04162710 2024-04-14T08:33:44.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.827+0000 7f4d09355640 1 -- 172.21.15.172:0/2134058422 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3430425622 0 0) 0x7f4d040cefb0 con 0x7f4d0415e430 2024-04-14T08:33:44.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.827+0000 7f4d09b56640 1 --2- 172.21.15.172:0/2134058422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d04162710 0x7f4d040d1df0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4d040d1840 tx=0x7f4cf402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3423fa877bd0965 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:44.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.828+0000 7f4d09355640 1 -- 172.21.15.172:0/2134058422 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4035020 con 0x7f4d04162710 2024-04-14T08:33:44.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.828+0000 7f4d09355640 1 -- 172.21.15.172:0/2134058422 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf4031830 con 0x7f4d04162710 2024-04-14T08:33:44.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.828+0000 7f4d09355640 1 -- 172.21.15.172:0/2134058422 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4031c00 con 0x7f4d04162710 2024-04-14T08:33:44.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.828+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/2134058422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d04162710 msgr2=0x7f4d040d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.828+0000 7f4d0c5bb640 1 --2- 172.21.15.172:0/2134058422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d04162710 0x7f4d040d1df0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4d040d1840 tx=0x7f4cf402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.828+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/2134058422 shutdown_connections 2024-04-14T08:33:44.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.828+0000 7f4d0c5bb640 1 --2- 172.21.15.172:0/2134058422 >> v2:172.21.15.177:3300/0 conn(0x7f4d04162ba0 0x7f4d040d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.828+0000 7f4d0c5bb640 1 --2- 172.21.15.172:0/2134058422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d04162710 0x7f4d040d1df0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.828+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/2134058422 >> 172.21.15.172:0/2134058422 conn(0x7f4d0409d560 msgr2=0x7f4d040d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.828+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/2134058422 shutdown_connections 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.828+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/2134058422 wait complete. 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.829+0000 7f4d0c5bb640 1 Processor -- start 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.829+0000 7f4d0c5bb640 1 -- start start 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0c5bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d0415e430 0x7f4d04152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0c5bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d04162710 0x7f4d040c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0c5bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d04162ba0 0x7f4d04160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0c5bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4d040d4e20 con 0x7f4d04162ba0 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0c5bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0415e810 con 0x7f4d0415e430 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0c5bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4d040d0050 con 0x7f4d04162710 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0ab58640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d04162ba0 0x7f4d04160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0ab58640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d04162ba0 0x7f4d04160f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59788/0 (socket says 172.21.15.172:59788) 2024-04-14T08:33:44.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0ab58640 1 -- 172.21.15.172:0/1087425348 learned_addr learned my addr 172.21.15.172:0/1087425348 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d09b56640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d04162710 0x7f4d040c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0a357640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d0415e430 0x7f4d04152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0ab58640 1 -- 172.21.15.172:0/1087425348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d04162710 msgr2=0x7f4d040c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0ab58640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d04162710 0x7f4d040c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0ab58640 1 -- 172.21.15.172:0/1087425348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d0415e430 msgr2=0x7f4d04152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.830+0000 7f4d0ab58640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d0415e430 0x7f4d04152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.831+0000 7f4d0ab58640 1 -- 172.21.15.172:0/1087425348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d040cc470 con 0x7f4d04162ba0 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.831+0000 7f4d0ab58640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d04162ba0 0x7f4d04160f90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00e930 tx=0x7f4cfc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.831+0000 7f4cfaffd640 1 -- 172.21.15.172:0/1087425348 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc017070 con 0x7f4d04162ba0 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.831+0000 7f4cfaffd640 1 -- 172.21.15.172:0/1087425348 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cfc00c980 con 0x7f4d04162ba0 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.831+0000 7f4cfaffd640 1 -- 172.21.15.172:0/1087425348 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00cd70 con 0x7f4d04162ba0 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.831+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/1087425348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d040cc750 con 0x7f4d04162ba0 2024-04-14T08:33:44.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.831+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/1087425348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d040ccd10 con 0x7f4d04162ba0 2024-04-14T08:33:44.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.832+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/1087425348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d04068590 con 0x7f4d04162ba0 2024-04-14T08:33:44.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.832+0000 7f4cfaffd640 1 -- 172.21.15.172:0/1087425348 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4cfc01a070 con 0x7f4d04162ba0 2024-04-14T08:33:44.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.833+0000 7f4cfaffd640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ce806c8f0 0x7f4ce806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:44.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.833+0000 7f4cfaffd640 1 -- 172.21.15.172:0/1087425348 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4cfc08c190 con 0x7f4d04162ba0 2024-04-14T08:33:44.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.833+0000 7f4d0a357640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ce806c8f0 0x7f4ce806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:44.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.833+0000 7f4d0a357640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ce806c8f0 0x7f4ce806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4d000012e0 tx=0x7f4d00001630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:44.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:44.837+0000 7f4cfaffd640 1 -- 172.21.15.172:0/1087425348 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4cfc059cc0 con 0x7f4d04162ba0 2024-04-14T08:33:45.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.022+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/1087425348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4d040d18a0 con 0x7f4d04162ba0 2024-04-14T08:33:45.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.022+0000 7f4cfaffd640 1 -- 172.21.15.172:0/1087425348 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f4cfc05dfc0 con 0x7f4d04162ba0 2024-04-14T08:33:45.023 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:33:45.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.024+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/1087425348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ce806c8f0 msgr2=0x7f4ce806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.024+0000 7f4d0c5bb640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ce806c8f0 0x7f4ce806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4d000012e0 tx=0x7f4d00001630 comp rx=0 tx=0).stop 2024-04-14T08:33:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.024+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/1087425348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d04162ba0 msgr2=0x7f4d04160f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.024+0000 7f4d0c5bb640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d04162ba0 0x7f4d04160f90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00e930 tx=0x7f4cfc00edf0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.024+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/1087425348 shutdown_connections 2024-04-14T08:33:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.024+0000 7f4d0c5bb640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ce806c8f0 0x7f4ce806edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.024+0000 7f4d0c5bb640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d04162ba0 0x7f4d04160f90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.024+0000 7f4d0c5bb640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d04162710 0x7f4d040c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.024+0000 7f4d0c5bb640 1 --2- 172.21.15.172:0/1087425348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d0415e430 0x7f4d04152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.024+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/1087425348 >> 172.21.15.172:0/1087425348 conn(0x7f4d0409d560 msgr2=0x7f4d040b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.025+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/1087425348 shutdown_connections 2024-04-14T08:33:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.025+0000 7f4d0c5bb640 1 -- 172.21.15.172:0/1087425348 wait complete. 2024-04-14T08:33:45.036 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-14T08:33:45.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:45.036 DEBUG:teuthology.orchestra.run.smithi177:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-14T08:33:45.065 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:33:45.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:33:45.066 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-14T08:33:45.121 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-14T08:33:45.121 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:33:45.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.224+0000 7f6339b1e640 1 Processor -- start 2024-04-14T08:33:45.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.225+0000 7f6339b1e640 1 -- start start 2024-04-14T08:33:45.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.225+0000 7f6339b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340d2830 0x7f63340d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.225+0000 7f6339b1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63340d3150 con 0x7f63340cedd0 2024-04-14T08:33:45.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.225+0000 7f6339b1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63340cf320 con 0x7f633415e890 2024-04-14T08:33:45.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.225+0000 7f6339b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f63340cff30 con 0x7f63340d2830 2024-04-14T08:33:45.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.225+0000 7f6333fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f63340cedd0 0x7f63340b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54166/0 (socket says 172.21.15.172:54166) 2024-04-14T08:33:45.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.225+0000 7f6333fff640 1 -- 172.21.15.172:0/2390329310 learned_addr learned my addr 172.21.15.172:0/2390329310 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:45.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.226+0000 7f63337fe640 1 --2- 172.21.15.172:0/2390329310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340d2830 0x7f63340d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.226+0000 7f6332ffd640 1 -- 172.21.15.172:0/2390329310 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3512562360 0 0) 0x7f63340d3150 con 0x7f63340cedd0 2024-04-14T08:33:45.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.226+0000 7f6332ffd640 1 -- 172.21.15.172:0/2390329310 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6320003520 con 0x7f63340cedd0 2024-04-14T08:33:45.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.226+0000 7f6332ffd640 1 -- 172.21.15.172:0/2390329310 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3024970592 0 0) 0x7f63340cf320 con 0x7f633415e890 2024-04-14T08:33:45.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.226+0000 7f6332ffd640 1 -- 172.21.15.172:0/2390329310 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63340d3150 con 0x7f633415e890 2024-04-14T08:33:45.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.226+0000 7f63337fe640 1 -- 172.21.15.172:0/2390329310 >> v1:172.21.15.177:6789/0 conn(0x7f633415e890 legacy=0x7f63340d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.226+0000 7f63337fe640 1 -- 172.21.15.172:0/2390329310 >> v1:172.21.15.172:6789/0 conn(0x7f63340cedd0 legacy=0x7f63340b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.226+0000 7f63337fe640 1 -- 172.21.15.172:0/2390329310 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63340cf600 con 0x7f63340d2830 2024-04-14T08:33:45.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.227+0000 7f63337fe640 1 --2- 172.21.15.172:0/2390329310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340d2830 0x7f63340d2c10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f63240097d0 tx=0x7f632402b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bff53ad4fa00d2a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:45.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.227+0000 7f6332ffd640 1 -- 172.21.15.172:0/2390329310 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324034070 con 0x7f63340d2830 2024-04-14T08:33:45.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.227+0000 7f6332ffd640 1 -- 172.21.15.172:0/2390329310 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6324030880 con 0x7f63340d2830 2024-04-14T08:33:45.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.227+0000 7f6332ffd640 1 -- 172.21.15.172:0/2390329310 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324030c50 con 0x7f63340d2830 2024-04-14T08:33:45.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.227+0000 7f6339b1e640 1 -- 172.21.15.172:0/2390329310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340d2830 msgr2=0x7f63340d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.227+0000 7f6339b1e640 1 --2- 172.21.15.172:0/2390329310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340d2830 0x7f63340d2c10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f63240097d0 tx=0x7f632402b9c0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.227+0000 7f6339b1e640 1 -- 172.21.15.172:0/2390329310 shutdown_connections 2024-04-14T08:33:45.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.227+0000 7f6339b1e640 1 --2- 172.21.15.172:0/2390329310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340d2830 0x7f63340d2c10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.227+0000 7f6339b1e640 1 -- 172.21.15.172:0/2390329310 >> 172.21.15.172:0/2390329310 conn(0x7f633409d590 msgr2=0x7f63340d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:45.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.227+0000 7f6339b1e640 1 -- 172.21.15.172:0/2390329310 shutdown_connections 2024-04-14T08:33:45.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.227+0000 7f6339b1e640 1 -- 172.21.15.172:0/2390329310 wait complete. 2024-04-14T08:33:45.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.229+0000 7f6339b1e640 1 Processor -- start 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.229+0000 7f6339b1e640 1 -- start start 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.229+0000 7f6339b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340cedd0 0x7f63340cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.229+0000 7f6339b1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63340d2830 0x7f63340cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.229+0000 7f6339b1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633415e890 0x7f63340cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.229+0000 7f6339b1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f63340cfbc0 con 0x7f633415e890 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f6339b1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f63340d2350 con 0x7f63340d2830 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f6339b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f63340b3e20 con 0x7f63340cedd0 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f6333fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633415e890 0x7f63340cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f63337fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340cedd0 0x7f63340cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f63337fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340cedd0 0x7f63340cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:36664/0 (socket says 172.21.15.172:36664) 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f6333fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633415e890 0x7f63340cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59792/0 (socket says 172.21.15.172:59792) 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f63337fe640 1 -- 172.21.15.172:0/216255097 learned_addr learned my addr 172.21.15.172:0/216255097 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f632affd640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63340d2830 0x7f63340cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f632affd640 1 -- 172.21.15.172:0/216255097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340cedd0 msgr2=0x7f63340cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f632affd640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340cedd0 0x7f63340cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f632affd640 1 -- 172.21.15.172:0/216255097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633415e890 msgr2=0x7f63340cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f632affd640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633415e890 0x7f63340cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f632affd640 1 -- 172.21.15.172:0/216255097 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633415fd90 con 0x7f63340d2830 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f63337fe640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340cedd0 0x7f63340cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f6333fff640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633415e890 0x7f63340cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f632affd640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63340d2830 0x7f63340cb320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f631c00c390 tx=0x7f631c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f6330ff9640 1 -- 172.21.15.172:0/216255097 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c017070 con 0x7f63340d2830 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f6330ff9640 1 -- 172.21.15.172:0/216255097 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f631c0052f0 con 0x7f63340d2830 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f6330ff9640 1 -- 172.21.15.172:0/216255097 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c003830 con 0x7f63340d2830 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f6339b1e640 1 -- 172.21.15.172:0/216255097 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6334148770 con 0x7f63340d2830 2024-04-14T08:33:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.230+0000 7f6339b1e640 1 -- 172.21.15.172:0/216255097 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63340cdee0 con 0x7f63340d2830 2024-04-14T08:33:45.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.231+0000 7f6339b1e640 1 -- 172.21.15.172:0/216255097 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6334068520 con 0x7f63340d2830 2024-04-14T08:33:45.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.232+0000 7f6330ff9640 1 -- 172.21.15.172:0/216255097 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f631c01a070 con 0x7f63340d2830 2024-04-14T08:33:45.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.232+0000 7f6330ff9640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f631406c920 0x7f631406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.232+0000 7f6330ff9640 1 -- 172.21.15.172:0/216255097 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f631c08b1b0 con 0x7f63340d2830 2024-04-14T08:33:45.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.232+0000 7f63337fe640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f631406c920 0x7f631406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.233+0000 7f63337fe640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f631406c920 0x7f631406ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f632402ba00 tx=0x7f632403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:45.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.237+0000 7f6330ff9640 1 -- 172.21.15.172:0/216255097 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f631c058b30 con 0x7f63340d2830 2024-04-14T08:33:45.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.452+0000 7f6339b1e640 1 -- 172.21.15.172:0/216255097 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63340d2610 con 0x7f63340d2830 2024-04-14T08:33:45.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.453+0000 7f6330ff9640 1 -- 172.21.15.172:0/216255097 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f631c05ce30 con 0x7f63340d2830 2024-04-14T08:33:45.453 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:45.453 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3053874257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3053874257},{"type":"v1","addr":"172.21.15.172:6835","nonce":3053874257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2790588849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2790588849},{"type":"v1","addr":"172.21.15.172:6837","nonce":2790588849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3130776573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3130776573},{"type":"v1","addr":"172.21.15.177:6834","nonce":3130776573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:32:53.535483+0000","modified":"2024-04-14T08:33:09.604218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/517963877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":517963877},{"type":"v1","addr":"172.21.15.177:6835","nonce":517963877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4309,"qdb_cluster":[4309]},"id":1}]} 2024-04-14T08:33:45.454 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 9 2024-04-14T08:33:45.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.454+0000 7f6339b1e640 1 -- 172.21.15.172:0/216255097 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f631406c920 msgr2=0x7f631406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.454+0000 7f6339b1e640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f631406c920 0x7f631406ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f632402ba00 tx=0x7f632403b040 comp rx=0 tx=0).stop 2024-04-14T08:33:45.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.454+0000 7f6339b1e640 1 -- 172.21.15.172:0/216255097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63340d2830 msgr2=0x7f63340cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.454+0000 7f6339b1e640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63340d2830 0x7f63340cb320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f631c00c390 tx=0x7f631c00c850 comp rx=0 tx=0).stop 2024-04-14T08:33:45.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.454+0000 7f6339b1e640 1 -- 172.21.15.172:0/216255097 shutdown_connections 2024-04-14T08:33:45.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.454+0000 7f6339b1e640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f631406c920 0x7f631406ede0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.454+0000 7f6339b1e640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633415e890 0x7f63340cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.454+0000 7f6339b1e640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63340d2830 0x7f63340cb320 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.454+0000 7f6339b1e640 1 --2- 172.21.15.172:0/216255097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63340cedd0 0x7f63340cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.454+0000 7f6339b1e640 1 -- 172.21.15.172:0/216255097 >> 172.21.15.172:0/216255097 conn(0x7f633409d590 msgr2=0x7f63340d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:45.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.455+0000 7f6339b1e640 1 -- 172.21.15.172:0/216255097 shutdown_connections 2024-04-14T08:33:45.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.455+0000 7f6339b1e640 1 -- 172.21.15.172:0/216255097 wait complete. 2024-04-14T08:33:45.466 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:45.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.567+0000 7fd6a559c640 1 Processor -- start 2024-04-14T08:33:45.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.567+0000 7fd6a559c640 1 -- start start 2024-04-14T08:33:45.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.568+0000 7fd6a559c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd6a00b3dd0 0x7fd6a015efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.568+0000 7fd6a559c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6a0070270 con 0x7fd6a006b520 2024-04-14T08:33:45.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.568+0000 7fd6a559c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6a00b4540 con 0x7fd6a006fdf0 2024-04-14T08:33:45.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.568+0000 7fd6a559c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fd6a0072d80 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.568+0000 7fd69effd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd6a006b520 0x7fd6a006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54182/0 (socket says 172.21.15.172:54182) 2024-04-14T08:33:45.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.568+0000 7fd69effd640 1 -- 172.21.15.172:0/2622973811 learned_addr learned my addr 172.21.15.172:0/2622973811 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:45.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.568+0000 7fd69f7fe640 1 --2- 172.21.15.172:0/2622973811 >> v2:172.21.15.177:3300/0 conn(0x7fd6a00b3dd0 0x7fd6a015efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.568+0000 7fd69dffb640 1 -- 172.21.15.172:0/2622973811 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1544858384 0 0) 0x7fd6a0070270 con 0x7fd6a006b520 2024-04-14T08:33:45.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.568+0000 7fd69dffb640 1 -- 172.21.15.172:0/2622973811 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd688003520 con 0x7fd6a006b520 2024-04-14T08:33:45.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.568+0000 7fd69dffb640 1 -- 172.21.15.172:0/2622973811 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3171325543 0 0) 0x7fd6a00b4540 con 0x7fd6a006fdf0 2024-04-14T08:33:45.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.569+0000 7fd69dffb640 1 -- 172.21.15.172:0/2622973811 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6a0070270 con 0x7fd6a006fdf0 2024-04-14T08:33:45.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.569+0000 7fd69f7fe640 1 -- 172.21.15.172:0/2622973811 >> v1:172.21.15.177:6789/0 conn(0x7fd6a006fdf0 legacy=0x7fd6a015e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.569+0000 7fd69f7fe640 1 -- 172.21.15.172:0/2622973811 >> v1:172.21.15.172:6789/0 conn(0x7fd6a006b520 legacy=0x7fd6a006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.569+0000 7fd69f7fe640 1 -- 172.21.15.172:0/2622973811 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a0072450 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.569+0000 7fd69f7fe640 1 --2- 172.21.15.172:0/2622973811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a00b3dd0 0x7fd6a015efa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd6980097a0 tx=0x7fd69802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2074ab9915b3983 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:45.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.569+0000 7fd69dffb640 1 -- 172.21.15.172:0/2622973811 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698034070 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.570+0000 7fd69dffb640 1 -- 172.21.15.172:0/2622973811 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd698030610 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.570+0000 7fd69dffb640 1 -- 172.21.15.172:0/2622973811 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6980309c0 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.570+0000 7fd6a559c640 1 -- 172.21.15.172:0/2622973811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a00b3dd0 msgr2=0x7fd6a015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.570+0000 7fd6a559c640 1 --2- 172.21.15.172:0/2622973811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a00b3dd0 0x7fd6a015efa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd6980097a0 tx=0x7fd69802b960 comp rx=0 tx=0).stop 2024-04-14T08:33:45.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.570+0000 7fd6a559c640 1 -- 172.21.15.172:0/2622973811 shutdown_connections 2024-04-14T08:33:45.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.570+0000 7fd6a559c640 1 --2- 172.21.15.172:0/2622973811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a00b3dd0 0x7fd6a015efa0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.570+0000 7fd6a559c640 1 -- 172.21.15.172:0/2622973811 >> 172.21.15.172:0/2622973811 conn(0x7fd6a00d0f20 msgr2=0x7fd6a00d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:45.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.570+0000 7fd6a559c640 1 -- 172.21.15.172:0/2622973811 shutdown_connections 2024-04-14T08:33:45.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.570+0000 7fd6a559c640 1 -- 172.21.15.172:0/2622973811 wait complete. 2024-04-14T08:33:45.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.571+0000 7fd6a559c640 1 Processor -- start 2024-04-14T08:33:45.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.571+0000 7fd6a559c640 1 -- start start 2024-04-14T08:33:45.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.572+0000 7fd6a559c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a006b520 0x7fd6a015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.572+0000 7fd6a559c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6a006fdf0 0x7fd6a0157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.572+0000 7fd6a559c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a00b3dd0 0x7fd6a0157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.572+0000 7fd6a559c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a00b4260 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.572+0000 7fd6a559c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0072a10 con 0x7fd6a006b520 2024-04-14T08:33:45.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.572+0000 7fd6a559c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a0073210 con 0x7fd6a006fdf0 2024-04-14T08:33:45.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.572+0000 7fd69f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a00b3dd0 0x7fd6a0157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.572+0000 7fd69f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a00b3dd0 0x7fd6a0157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59808/0 (socket says 172.21.15.172:59808) 2024-04-14T08:33:45.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.572+0000 7fd69f7fe640 1 -- 172.21.15.172:0/602480695 learned_addr learned my addr 172.21.15.172:0/602480695 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.572+0000 7fd69effd640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a006b520 0x7fd6a015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.572+0000 7fd69e7fc640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6a006fdf0 0x7fd6a0157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.573+0000 7fd69f7fe640 1 -- 172.21.15.172:0/602480695 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6a006fdf0 msgr2=0x7fd6a0157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.573+0000 7fd69f7fe640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6a006fdf0 0x7fd6a0157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.573+0000 7fd69f7fe640 1 -- 172.21.15.172:0/602480695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a006b520 msgr2=0x7fd6a015d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.573+0000 7fd69f7fe640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a006b520 0x7fd6a015d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.573+0000 7fd69f7fe640 1 -- 172.21.15.172:0/602480695 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a0160540 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.573+0000 7fd69f7fe640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a00b3dd0 0x7fd6a0157d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd6980097a0 tx=0x7fd6980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.573+0000 7fd6877fe640 1 -- 172.21.15.172:0/602480695 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698049070 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.573+0000 7fd6877fe640 1 -- 172.21.15.172:0/602480695 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd698030410 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.573+0000 7fd6a559c640 1 -- 172.21.15.172:0/602480695 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a0160820 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.573+0000 7fd6877fe640 1 -- 172.21.15.172:0/602480695 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698034040 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.573+0000 7fd6a559c640 1 -- 172.21.15.172:0/602480695 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a0160de0 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.574+0000 7fd6a559c640 1 -- 172.21.15.172:0/602480695 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a0068520 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.575+0000 7fd6877fe640 1 -- 172.21.15.172:0/602480695 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd69804c070 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.575+0000 7fd6877fe640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd68c06c8f0 0x7fd68c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.575+0000 7fd6877fe640 1 -- 172.21.15.172:0/602480695 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd6980adb80 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.575+0000 7fd69effd640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd68c06c8f0 0x7fd68c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.576+0000 7fd69effd640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd68c06c8f0 0x7fd68c06edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd6900012e0 tx=0x7fd690001a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:45.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.579+0000 7fd6877fe640 1 -- 172.21.15.172:0/602480695 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd69807b680 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.754+0000 7fd6a559c640 1 -- 172.21.15.172:0/602480695 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6a015e2d0 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.755+0000 7fd6877fe640 1 -- 172.21.15.172:0/602480695 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18962 (secure 0 0 0) 0x7fd69807f980 con 0x7fd6a00b3dd0 2024-04-14T08:33:45.756 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:45.756 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":27,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:33:34.330426+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.757+0000 7fd6a559c640 1 -- 172.21.15.172:0/602480695 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd68c06c8f0 msgr2=0x7fd68c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.757+0000 7fd6a559c640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd68c06c8f0 0x7fd68c06edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd6900012e0 tx=0x7fd690001a30 comp rx=0 tx=0).stop 2024-04-14T08:33:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.757+0000 7fd6a559c640 1 -- 172.21.15.172:0/602480695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a00b3dd0 msgr2=0x7fd6a0157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.757+0000 7fd6a559c640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a00b3dd0 0x7fd6a0157d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd6980097a0 tx=0x7fd6980046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.757+0000 7fd6a559c640 1 -- 172.21.15.172:0/602480695 shutdown_connections 2024-04-14T08:33:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.757+0000 7fd6a559c640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd68c06c8f0 0x7fd68c06edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.757+0000 7fd6a559c640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a00b3dd0 0x7fd6a0157d80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.757+0000 7fd6a559c640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6a006fdf0 0x7fd6a0157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.757+0000 7fd6a559c640 1 --2- 172.21.15.172:0/602480695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a006b520 0x7fd6a015d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.757+0000 7fd6a559c640 1 -- 172.21.15.172:0/602480695 >> 172.21.15.172:0/602480695 conn(0x7fd6a00d0f20 msgr2=0x7fd6a00b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:45.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.757+0000 7fd6a559c640 1 -- 172.21.15.172:0/602480695 shutdown_connections 2024-04-14T08:33:45.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.757+0000 7fd6a559c640 1 -- 172.21.15.172:0/602480695 wait complete. 2024-04-14T08:33:45.769 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:32:51.214999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:32:52.493598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-14T08:32:56.216795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-14T08:33:45.769 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:45.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.872+0000 7f1dd74b7640 1 Processor -- start 2024-04-14T08:33:45.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.873+0000 7f1dd74b7640 1 -- start start 2024-04-14T08:33:45.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.873+0000 7f1dd74b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd0162710 0x7f1dd00d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.873+0000 7f1dd74b7640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1dd0162b70 0x7f1dd00d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.873+0000 7f1dd74b7640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dd00cefb0 con 0x7f1dd015e430 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.873+0000 7f1dd74b7640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd00cf290 con 0x7f1dd0162b70 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.873+0000 7f1dd74b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd00cf570 con 0x7f1dd0162710 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.874+0000 7f1dd5a54640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1dd0162b70 0x7f1dd00d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.874+0000 7f1dd4a52640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd0162710 0x7f1dd00d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.874+0000 7f1dd5253640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1dd015e430 0x7f1dd0162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54186/0 (socket says 172.21.15.172:54186) 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.874+0000 7f1dd5253640 1 -- 172.21.15.172:0/4137268855 learned_addr learned my addr 172.21.15.172:0/4137268855 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.874+0000 7f1dd4a52640 1 -- 172.21.15.172:0/4137268855 >> v2:172.21.15.172:3300/0 conn(0x7f1dd0162b70 msgr2=0x7f1dd00d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.874+0000 7f1dd4a52640 1 --2- 172.21.15.172:0/4137268855 >> v2:172.21.15.172:3300/0 conn(0x7f1dd0162b70 0x7f1dd00d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.874+0000 7f1dd4a52640 1 -- 172.21.15.172:0/4137268855 >> v1:172.21.15.172:6789/0 conn(0x7f1dd015e430 legacy=0x7f1dd0162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.874+0000 7f1dd4a52640 1 -- 172.21.15.172:0/4137268855 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd00cf850 con 0x7f1dd0162710 2024-04-14T08:33:45.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.874+0000 7f1dc7fff640 1 -- 172.21.15.172:0/4137268855 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2076949476 0 0) 0x7f1dd00cefb0 con 0x7f1dd015e430 2024-04-14T08:33:45.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.874+0000 7f1dd5a54640 1 --2- 172.21.15.172:0/4137268855 >> v2:172.21.15.172:3300/0 conn(0x7f1dd0162b70 0x7f1dd00d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:45.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.875+0000 7f1dd4a52640 1 --2- 172.21.15.172:0/4137268855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd0162710 0x7f1dd00d1df0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00097a0 tx=0x7f1dc002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2ce009199e114a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:45.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.875+0000 7f1dc7fff640 1 -- 172.21.15.172:0/4137268855 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0035020 con 0x7f1dd0162710 2024-04-14T08:33:45.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.875+0000 7f1dc7fff640 1 -- 172.21.15.172:0/4137268855 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc0031810 con 0x7f1dd0162710 2024-04-14T08:33:45.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.875+0000 7f1dc7fff640 1 -- 172.21.15.172:0/4137268855 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0031be0 con 0x7f1dd0162710 2024-04-14T08:33:45.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.875+0000 7f1dd74b7640 1 -- 172.21.15.172:0/4137268855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd0162710 msgr2=0x7f1dd00d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.875+0000 7f1dd74b7640 1 --2- 172.21.15.172:0/4137268855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd0162710 0x7f1dd00d1df0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00097a0 tx=0x7f1dc002b8b0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.875+0000 7f1dd74b7640 1 -- 172.21.15.172:0/4137268855 shutdown_connections 2024-04-14T08:33:45.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.875+0000 7f1dd74b7640 1 --2- 172.21.15.172:0/4137268855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd0162710 0x7f1dd00d1df0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.875+0000 7f1dd74b7640 1 --2- 172.21.15.172:0/4137268855 >> v2:172.21.15.172:3300/0 conn(0x7f1dd0162b70 0x7f1dd00d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.875+0000 7f1dd74b7640 1 -- 172.21.15.172:0/4137268855 >> 172.21.15.172:0/4137268855 conn(0x7f1dd009d560 msgr2=0x7f1dd00d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:45.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.875+0000 7f1dd74b7640 1 -- 172.21.15.172:0/4137268855 shutdown_connections 2024-04-14T08:33:45.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.876+0000 7f1dd74b7640 1 -- 172.21.15.172:0/4137268855 wait complete. 2024-04-14T08:33:45.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.877+0000 7f1dd74b7640 1 Processor -- start 2024-04-14T08:33:45.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.877+0000 7f1dd74b7640 1 -- start start 2024-04-14T08:33:45.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.877+0000 7f1dd74b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dd015e430 0x7f1dd00ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.877+0000 7f1dd74b7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd0162710 0x7f1dd00ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.877+0000 7f1dd74b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd0162b70 0x7f1dd014e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.877+0000 7f1dd74b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd00d4e20 con 0x7f1dd015e430 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.877+0000 7f1dd74b7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd015e810 con 0x7f1dd0162710 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.877+0000 7f1dd74b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd00d0050 con 0x7f1dd0162b70 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.877+0000 7f1dd5253640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dd015e430 0x7f1dd00ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.877+0000 7f1dd5253640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dd015e430 0x7f1dd00ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59832/0 (socket says 172.21.15.172:59832) 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.877+0000 7f1dd5253640 1 -- 172.21.15.172:0/1981988338 learned_addr learned my addr 172.21.15.172:0/1981988338 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dd5a54640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd0162b70 0x7f1dd014e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dd4a52640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd0162710 0x7f1dd00ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dd5253640 1 -- 172.21.15.172:0/1981988338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd0162b70 msgr2=0x7f1dd014e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dd5253640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd0162b70 0x7f1dd014e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dd5253640 1 -- 172.21.15.172:0/1981988338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd0162710 msgr2=0x7f1dd00ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dd5253640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd0162710 0x7f1dd00ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:45.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dd5253640 1 -- 172.21.15.172:0/1981988338 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd00b56f0 con 0x7f1dd015e430 2024-04-14T08:33:45.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dd5253640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dd015e430 0x7f1dd00ccf50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00c4d0 tx=0x7f1dcc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:45.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dc5ffb640 1 -- 172.21.15.172:0/1981988338 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0074e0 con 0x7f1dd015e430 2024-04-14T08:33:45.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dc5ffb640 1 -- 172.21.15.172:0/1981988338 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dcc0047e0 con 0x7f1dd015e430 2024-04-14T08:33:45.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dc5ffb640 1 -- 172.21.15.172:0/1981988338 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc004bd0 con 0x7f1dd015e430 2024-04-14T08:33:45.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dd74b7640 1 -- 172.21.15.172:0/1981988338 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd00c7e80 con 0x7f1dd015e430 2024-04-14T08:33:45.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.878+0000 7f1dd74b7640 1 -- 172.21.15.172:0/1981988338 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd00b5410 con 0x7f1dd015e430 2024-04-14T08:33:45.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.879+0000 7f1dd74b7640 1 -- 172.21.15.172:0/1981988338 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd0068590 con 0x7f1dd015e430 2024-04-14T08:33:45.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.880+0000 7f1dc5ffb640 1 -- 172.21.15.172:0/1981988338 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1dcc011070 con 0x7f1dd015e430 2024-04-14T08:33:45.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.880+0000 7f1dc5ffb640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1db406c8f0 0x7f1db406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:45.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.880+0000 7f1dd4a52640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1db406c8f0 0x7f1db406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:45.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.880+0000 7f1dc5ffb640 1 -- 172.21.15.172:0/1981988338 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1dcc08b9e0 con 0x7f1dd015e430 2024-04-14T08:33:45.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.880+0000 7f1dd4a52640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1db406c8f0 0x7f1db406edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0036000 tx=0x7f1dc0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:45.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:45.885+0000 7f1dc5ffb640 1 -- 172.21.15.172:0/1981988338 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1dcc059aa0 con 0x7f1dd015e430 2024-04-14T08:33:46.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.057+0000 7f1dd74b7640 1 -- 172.21.15.172:0/1981988338 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1dd00d2b00 con 0x7f1dd015e430 2024-04-14T08:33:46.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.058+0000 7f1dc5ffb640 1 -- 172.21.15.172:0/1981988338 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f1dcc0593a0 con 0x7f1dd015e430 2024-04-14T08:33:46.059 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:46.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.060+0000 7f1dd74b7640 1 -- 172.21.15.172:0/1981988338 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1db406c8f0 msgr2=0x7f1db406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.060+0000 7f1dd74b7640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1db406c8f0 0x7f1db406edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0036000 tx=0x7f1dc0008030 comp rx=0 tx=0).stop 2024-04-14T08:33:46.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.060+0000 7f1dd74b7640 1 -- 172.21.15.172:0/1981988338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dd015e430 msgr2=0x7f1dd00ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.060+0000 7f1dd74b7640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dd015e430 0x7f1dd00ccf50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00c4d0 tx=0x7f1dcc00c8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.060+0000 7f1dd74b7640 1 -- 172.21.15.172:0/1981988338 shutdown_connections 2024-04-14T08:33:46.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.060+0000 7f1dd74b7640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1db406c8f0 0x7f1db406edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.060+0000 7f1dd74b7640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd0162b70 0x7f1dd014e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.060+0000 7f1dd74b7640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd0162710 0x7f1dd00ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.060+0000 7f1dd74b7640 1 --2- 172.21.15.172:0/1981988338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dd015e430 0x7f1dd00ccf50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.060+0000 7f1dd74b7640 1 -- 172.21.15.172:0/1981988338 >> 172.21.15.172:0/1981988338 conn(0x7f1dd009d560 msgr2=0x7f1dd00b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:46.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.060+0000 7f1dd74b7640 1 -- 172.21.15.172:0/1981988338 shutdown_connections 2024-04-14T08:33:46.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.060+0000 7f1dd74b7640 1 -- 172.21.15.172:0/1981988338 wait complete. 2024-04-14T08:33:46.071 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:33:46.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.175+0000 7f23595dc640 1 Processor -- start 2024-04-14T08:33:46.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.175+0000 7f23595dc640 1 -- start start 2024-04-14T08:33:46.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f23595dc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f235415e430 0x7f2354162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f23595dc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2354162540 0x7f23541629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f23595dc640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23540cefb0 con 0x7f23540b3a30 2024-04-14T08:33:46.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f23595dc640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f23540cf290 con 0x7f2354162540 2024-04-14T08:33:46.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f23595dc640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f23540cf570 con 0x7f235415e430 2024-04-14T08:33:46.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f23527fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2354162540 0x7f23541629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f23527fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2354162540 0x7f23541629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59844/0 (socket says 172.21.15.172:59844) 2024-04-14T08:33:46.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f23527fc640 1 -- 172.21.15.172:0/1285648077 learned_addr learned my addr 172.21.15.172:0/1285648077 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:46.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f2352ffd640 1 --2- 172.21.15.172:0/1285648077 >> v2:172.21.15.177:3300/0 conn(0x7f235415e430 0x7f2354162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f23527fc640 1 -- 172.21.15.172:0/1285648077 >> v2:172.21.15.177:3300/0 conn(0x7f235415e430 msgr2=0x7f2354162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:46.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f23527fc640 1 --2- 172.21.15.172:0/1285648077 >> v2:172.21.15.177:3300/0 conn(0x7f235415e430 0x7f2354162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f23527fc640 1 -- 172.21.15.172:0/1285648077 >> v1:172.21.15.177:6789/0 conn(0x7f23540b3a30 legacy=0x7f23540b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.176+0000 7f23527fc640 1 -- 172.21.15.172:0/1285648077 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23540cf850 con 0x7f2354162540 2024-04-14T08:33:46.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.177+0000 7f23527fc640 1 --2- 172.21.15.172:0/1285648077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162540 0x7f23541629d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f23540b6db0 tx=0x7f234802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405480d8565d0f05 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:46.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.177+0000 7f2351ffb640 1 -- 172.21.15.172:0/1285648077 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348034070 con 0x7f2354162540 2024-04-14T08:33:46.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.177+0000 7f2351ffb640 1 -- 172.21.15.172:0/1285648077 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2348030860 con 0x7f2354162540 2024-04-14T08:33:46.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.177+0000 7f2351ffb640 1 -- 172.21.15.172:0/1285648077 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348030c30 con 0x7f2354162540 2024-04-14T08:33:46.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.178+0000 7f23595dc640 1 -- 172.21.15.172:0/1285648077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162540 msgr2=0x7f23541629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.178+0000 7f23595dc640 1 --2- 172.21.15.172:0/1285648077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162540 0x7f23541629d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f23540b6db0 tx=0x7f234802b9a0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.178+0000 7f23595dc640 1 -- 172.21.15.172:0/1285648077 shutdown_connections 2024-04-14T08:33:46.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.178+0000 7f23595dc640 1 --2- 172.21.15.172:0/1285648077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162540 0x7f23541629d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.178+0000 7f23595dc640 1 --2- 172.21.15.172:0/1285648077 >> v2:172.21.15.177:3300/0 conn(0x7f235415e430 0x7f2354162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.178+0000 7f23595dc640 1 -- 172.21.15.172:0/1285648077 >> 172.21.15.172:0/1285648077 conn(0x7f235409d560 msgr2=0x7f23540d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:46.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.178+0000 7f23595dc640 1 -- 172.21.15.172:0/1285648077 shutdown_connections 2024-04-14T08:33:46.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.178+0000 7f23595dc640 1 -- 172.21.15.172:0/1285648077 wait complete. 2024-04-14T08:33:46.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.179+0000 7f23595dc640 1 Processor -- start 2024-04-14T08:33:46.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.179+0000 7f23595dc640 1 -- start start 2024-04-14T08:33:46.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23595dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23540b3a30 0x7f23540b98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23595dc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f235415e430 0x7f235414f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23595dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162540 0x7f23540c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23595dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23540d4e20 con 0x7f2354162540 2024-04-14T08:33:46.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23595dc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f235415e810 con 0x7f235415e430 2024-04-14T08:33:46.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23595dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f23540d0050 con 0x7f23540b3a30 2024-04-14T08:33:46.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23537fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162540 0x7f23540c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23537fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162540 0x7f23540c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59846/0 (socket says 172.21.15.172:59846) 2024-04-14T08:33:46.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23537fe640 1 -- 172.21.15.172:0/1873755456 learned_addr learned my addr 172.21.15.172:0/1873755456 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:46.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f2352ffd640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23540b3a30 0x7f23540b98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23527fc640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f235415e430 0x7f235414f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23537fe640 1 -- 172.21.15.172:0/1873755456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23540b3a30 msgr2=0x7f23540b98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23537fe640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23540b3a30 0x7f23540b98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23537fe640 1 -- 172.21.15.172:0/1873755456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f235415e430 msgr2=0x7f235414f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23537fe640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f235415e430 0x7f235414f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23537fe640 1 -- 172.21.15.172:0/1873755456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235414fc40 con 0x7f2354162540 2024-04-14T08:33:46.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f23537fe640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162540 0x7f23540c7d50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c7a0 tx=0x7f234c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:46.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f233b7fe640 1 -- 172.21.15.172:0/1873755456 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c003b70 con 0x7f2354162540 2024-04-14T08:33:46.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.180+0000 7f233b7fe640 1 -- 172.21.15.172:0/1873755456 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234c003d30 con 0x7f2354162540 2024-04-14T08:33:46.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.181+0000 7f23595dc640 1 -- 172.21.15.172:0/1873755456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23540cc490 con 0x7f2354162540 2024-04-14T08:33:46.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.181+0000 7f233b7fe640 1 -- 172.21.15.172:0/1873755456 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c00aa90 con 0x7f2354162540 2024-04-14T08:33:46.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.181+0000 7f23595dc640 1 -- 172.21.15.172:0/1873755456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23540cca50 con 0x7f2354162540 2024-04-14T08:33:46.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.182+0000 7f23595dc640 1 -- 172.21.15.172:0/1873755456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2354068590 con 0x7f2354162540 2024-04-14T08:33:46.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.182+0000 7f233b7fe640 1 -- 172.21.15.172:0/1873755456 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f234c00ac30 con 0x7f2354162540 2024-04-14T08:33:46.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.182+0000 7f233b7fe640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f234006c8f0 0x7f234006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.183+0000 7f2352ffd640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f234006c8f0 0x7f234006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.183+0000 7f233b7fe640 1 -- 172.21.15.172:0/1873755456 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f234c08bd60 con 0x7f2354162540 2024-04-14T08:33:46.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.183+0000 7f2352ffd640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f234006c8f0 0x7f234006edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2344009b40 tx=0x7f2344009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:46.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.187+0000 7f233b7fe640 1 -- 172.21.15.172:0/1873755456 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f235414fc40 con 0x7f2354162540 2024-04-14T08:33:46.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.364+0000 7f23595dc640 1 -- 172.21.15.172:0/1873755456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f235415d170 con 0x7f2354162540 2024-04-14T08:33:46.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.365+0000 7f233b7fe640 1 -- 172.21.15.172:0/1873755456 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f234c059740 con 0x7f2354162540 2024-04-14T08:33:46.366 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:46.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.367+0000 7f23595dc640 1 -- 172.21.15.172:0/1873755456 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f234006c8f0 msgr2=0x7f234006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.367+0000 7f23595dc640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f234006c8f0 0x7f234006edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2344009b40 tx=0x7f2344009190 comp rx=0 tx=0).stop 2024-04-14T08:33:46.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.367+0000 7f23595dc640 1 -- 172.21.15.172:0/1873755456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162540 msgr2=0x7f23540c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.367+0000 7f23595dc640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162540 0x7f23540c7d50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c7a0 tx=0x7f234c00cc60 comp rx=0 tx=0).stop 2024-04-14T08:33:46.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.367+0000 7f23595dc640 1 -- 172.21.15.172:0/1873755456 shutdown_connections 2024-04-14T08:33:46.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.367+0000 7f23595dc640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f234006c8f0 0x7f234006edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.367+0000 7f23595dc640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162540 0x7f23540c7d50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.367+0000 7f23595dc640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f235415e430 0x7f235414f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.367+0000 7f23595dc640 1 --2- 172.21.15.172:0/1873755456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23540b3a30 0x7f23540b98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.367+0000 7f23595dc640 1 -- 172.21.15.172:0/1873755456 >> 172.21.15.172:0/1873755456 conn(0x7f235409d560 msgr2=0x7f23540d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:46.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.367+0000 7f23595dc640 1 -- 172.21.15.172:0/1873755456 shutdown_connections 2024-04-14T08:33:46.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.367+0000 7f23595dc640 1 -- 172.21.15.172:0/1873755456 wait complete. 2024-04-14T08:33:46.379 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:33:46.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.483+0000 7fdde3a6d640 1 Processor -- start 2024-04-14T08:33:46.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.483+0000 7fdde3a6d640 1 -- start start 2024-04-14T08:33:46.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde3a6d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdddc0b23e0 0x7fdddc140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde3a6d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdddc15c380 0x7fdddc140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde3a6d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdddc15b620 con 0x7fdddc13df30 2024-04-14T08:33:46.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde3a6d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fdddc0cf9b0 con 0x7fdddc0b23e0 2024-04-14T08:33:46.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde3a6d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc1411f0 con 0x7fdddc15c380 2024-04-14T08:33:46.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde1809640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fdddc13df30 0x7fdddc0b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54190/0 (socket says 172.21.15.172:54190) 2024-04-14T08:33:46.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde1809640 1 -- 172.21.15.172:0/202148762 learned_addr learned my addr 172.21.15.172:0/202148762 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:46.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde1008640 1 --2- 172.21.15.172:0/202148762 >> v2:172.21.15.177:3300/0 conn(0x7fdddc0b23e0 0x7fdddc140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde200a640 1 --2- 172.21.15.172:0/202148762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdddc15c380 0x7fdddc140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde0807640 1 -- 172.21.15.172:0/202148762 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3928009774 0 0) 0x7fddd8001490 con 0x7fdddc13df30 2024-04-14T08:33:46.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde0807640 1 -- 172.21.15.172:0/202148762 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc8003520 con 0x7fdddc13df30 2024-04-14T08:33:46.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde1008640 1 -- 172.21.15.172:0/202148762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdddc15c380 msgr2=0x7fdddc140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:46.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde1008640 1 --2- 172.21.15.172:0/202148762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdddc15c380 0x7fdddc140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde1008640 1 -- 172.21.15.172:0/202148762 >> v1:172.21.15.172:6789/0 conn(0x7fdddc13df30 legacy=0x7fdddc0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde1008640 1 -- 172.21.15.172:0/202148762 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc1414a0 con 0x7fdddc0b23e0 2024-04-14T08:33:46.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.484+0000 7fdde200a640 1 --2- 172.21.15.172:0/202148762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdddc15c380 0x7fdddc140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:46.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.485+0000 7fdde1008640 1 --2- 172.21.15.172:0/202148762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdddc0b23e0 0x7fdddc140770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0097a0 tx=0x7fddcc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a922f58947c2cf48 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:46.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.485+0000 7fdde0807640 1 -- 172.21.15.172:0/202148762 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc035020 con 0x7fdddc0b23e0 2024-04-14T08:33:46.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.485+0000 7fdde0807640 1 -- 172.21.15.172:0/202148762 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddcc031830 con 0x7fdddc0b23e0 2024-04-14T08:33:46.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.485+0000 7fdde0807640 1 -- 172.21.15.172:0/202148762 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc031be0 con 0x7fdddc0b23e0 2024-04-14T08:33:46.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.486+0000 7fdde3a6d640 1 -- 172.21.15.172:0/202148762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdddc0b23e0 msgr2=0x7fdddc140770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.486+0000 7fdde3a6d640 1 --2- 172.21.15.172:0/202148762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdddc0b23e0 0x7fdddc140770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0097a0 tx=0x7fddcc02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.486+0000 7fdde3a6d640 1 -- 172.21.15.172:0/202148762 shutdown_connections 2024-04-14T08:33:46.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.486+0000 7fdde3a6d640 1 --2- 172.21.15.172:0/202148762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdddc15c380 0x7fdddc140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.486+0000 7fdde3a6d640 1 --2- 172.21.15.172:0/202148762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdddc0b23e0 0x7fdddc140770 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.486+0000 7fdde3a6d640 1 -- 172.21.15.172:0/202148762 >> 172.21.15.172:0/202148762 conn(0x7fdddc06b520 msgr2=0x7fdddc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:46.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.486+0000 7fdde3a6d640 1 -- 172.21.15.172:0/202148762 shutdown_connections 2024-04-14T08:33:46.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.486+0000 7fdde3a6d640 1 -- 172.21.15.172:0/202148762 wait complete. 2024-04-14T08:33:46.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.487+0000 7fdde3a6d640 1 Processor -- start 2024-04-14T08:33:46.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.487+0000 7fdde3a6d640 1 -- start start 2024-04-14T08:33:46.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde3a6d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdddc0b23e0 0x7fdddc1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde3a6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdddc13df30 0x7fdddc1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde3a6d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdddc15c380 0x7fdddc0b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde3a6d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc15be60 con 0x7fdddc13df30 2024-04-14T08:33:46.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde3a6d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc15bb80 con 0x7fdddc0b23e0 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde3a6d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc13e310 con 0x7fdddc15c380 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde1008640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdddc13df30 0x7fdddc1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde1008640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdddc13df30 0x7fdddc1515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59850/0 (socket says 172.21.15.172:59850) 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde1008640 1 -- 172.21.15.172:0/328779391 learned_addr learned my addr 172.21.15.172:0/328779391 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde1809640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdddc0b23e0 0x7fdddc1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde200a640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdddc15c380 0x7fdddc0b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde1008640 1 -- 172.21.15.172:0/328779391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdddc15c380 msgr2=0x7fdddc0b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde1008640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdddc15c380 0x7fdddc0b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde1008640 1 -- 172.21.15.172:0/328779391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdddc0b23e0 msgr2=0x7fdddc1566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde1008640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdddc0b23e0 0x7fdddc1566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde1008640 1 -- 172.21.15.172:0/328779391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc157a10 con 0x7fdddc13df30 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.488+0000 7fdde1008640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdddc13df30 0x7fdddc1515a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fddcc02b910 tx=0x7fddcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.489+0000 7fddd27fc640 1 -- 172.21.15.172:0/328779391 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc040070 con 0x7fdddc13df30 2024-04-14T08:33:46.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.489+0000 7fddd27fc640 1 -- 172.21.15.172:0/328779391 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddcc03a850 con 0x7fdddc13df30 2024-04-14T08:33:46.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.489+0000 7fddd27fc640 1 -- 172.21.15.172:0/328779391 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc035040 con 0x7fdddc13df30 2024-04-14T08:33:46.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.489+0000 7fdde3a6d640 1 -- 172.21.15.172:0/328779391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdddc157cf0 con 0x7fdddc13df30 2024-04-14T08:33:46.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.489+0000 7fdde3a6d640 1 -- 172.21.15.172:0/328779391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdddc1582b0 con 0x7fdddc13df30 2024-04-14T08:33:46.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.490+0000 7fdde3a6d640 1 -- 172.21.15.172:0/328779391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdddc068590 con 0x7fdddc13df30 2024-04-14T08:33:46.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.490+0000 7fddd27fc640 1 -- 172.21.15.172:0/328779391 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddcc04b060 con 0x7fdddc13df30 2024-04-14T08:33:46.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.490+0000 7fddd27fc640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fddc006c8f0 0x7fddc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.490+0000 7fddd27fc640 1 -- 172.21.15.172:0/328779391 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fddcc0ad560 con 0x7fdddc13df30 2024-04-14T08:33:46.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.490+0000 7fdde1809640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fddc006c8f0 0x7fddc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.491+0000 7fdde1809640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fddc006c8f0 0x7fddc006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005c00 tx=0x7fddd8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:46.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.495+0000 7fddd27fc640 1 -- 172.21.15.172:0/328779391 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fddcc07b590 con 0x7fdddc13df30 2024-04-14T08:33:46.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.669+0000 7fdde3a6d640 1 -- 172.21.15.172:0/328779391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdddc0cd0a0 con 0x7fdddc13df30 2024-04-14T08:33:46.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.670+0000 7fddd27fc640 1 -- 172.21.15.172:0/328779391 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fddcc07ae90 con 0x7fdddc13df30 2024-04-14T08:33:46.670 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:33:46.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.672+0000 7fdde3a6d640 1 -- 172.21.15.172:0/328779391 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fddc006c8f0 msgr2=0x7fddc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.672+0000 7fdde3a6d640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fddc006c8f0 0x7fddc006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005c00 tx=0x7fddd8009040 comp rx=0 tx=0).stop 2024-04-14T08:33:46.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.672+0000 7fdde3a6d640 1 -- 172.21.15.172:0/328779391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdddc13df30 msgr2=0x7fdddc1515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.672+0000 7fdde3a6d640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdddc13df30 0x7fdddc1515a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fddcc02b910 tx=0x7fddcc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.672+0000 7fdde3a6d640 1 -- 172.21.15.172:0/328779391 shutdown_connections 2024-04-14T08:33:46.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.672+0000 7fdde3a6d640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fddc006c8f0 0x7fddc006edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.672+0000 7fdde3a6d640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdddc15c380 0x7fdddc0b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.672+0000 7fdde3a6d640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdddc13df30 0x7fdddc1515a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.672+0000 7fdde3a6d640 1 --2- 172.21.15.172:0/328779391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdddc0b23e0 0x7fdddc1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.672+0000 7fdde3a6d640 1 -- 172.21.15.172:0/328779391 >> 172.21.15.172:0/328779391 conn(0x7fdddc06b520 msgr2=0x7fdddc0b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:46.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.672+0000 7fdde3a6d640 1 -- 172.21.15.172:0/328779391 shutdown_connections 2024-04-14T08:33:46.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.672+0000 7fdde3a6d640 1 -- 172.21.15.172:0/328779391 wait complete. 2024-04-14T08:33:46.683 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-14T08:33:46.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.788+0000 7fb9c631e640 1 Processor -- start 2024-04-14T08:33:46.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.788+0000 7fb9c631e640 1 -- start start 2024-04-14T08:33:46.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.788+0000 7fb9c631e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb9c015e430 0x7fb9c0162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.788+0000 7fb9c631e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb9c0162540 0x7fb9c01629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.788+0000 7fb9c631e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9c00cefb0 con 0x7fb9c00b3a30 2024-04-14T08:33:46.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.788+0000 7fb9c631e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb9c00cf290 con 0x7fb9c0162540 2024-04-14T08:33:46.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.788+0000 7fb9c631e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb9c00cf570 con 0x7fb9c015e430 2024-04-14T08:33:46.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.788+0000 7fb9bf7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb9c0162540 0x7fb9c01629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.789+0000 7fb9bf7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb9c0162540 0x7fb9c01629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59862/0 (socket says 172.21.15.172:59862) 2024-04-14T08:33:46.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.789+0000 7fb9bf7fe640 1 -- 172.21.15.172:0/2226754480 learned_addr learned my addr 172.21.15.172:0/2226754480 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:46.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.789+0000 7fb9bffff640 1 --2- 172.21.15.172:0/2226754480 >> v2:172.21.15.177:3300/0 conn(0x7fb9c015e430 0x7fb9c0162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.789+0000 7fb9bf7fe640 1 -- 172.21.15.172:0/2226754480 >> v2:172.21.15.177:3300/0 conn(0x7fb9c015e430 msgr2=0x7fb9c0162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.789+0000 7fb9bf7fe640 1 --2- 172.21.15.172:0/2226754480 >> v2:172.21.15.177:3300/0 conn(0x7fb9c015e430 0x7fb9c0162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.789+0000 7fb9bf7fe640 1 -- 172.21.15.172:0/2226754480 >> v1:172.21.15.177:6789/0 conn(0x7fb9c00b3a30 legacy=0x7fb9c00b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.789+0000 7fb9beffd640 1 -- 172.21.15.172:0/2226754480 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2482516576 0 0) 0x7fb9c00cefb0 con 0x7fb9c00b3a30 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.789+0000 7fb9bf7fe640 1 -- 172.21.15.172:0/2226754480 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c00cf850 con 0x7fb9c0162540 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.789+0000 7fb9bffff640 1 --2- 172.21.15.172:0/2226754480 >> v2:172.21.15.177:3300/0 conn(0x7fb9c015e430 0x7fb9c0162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.790+0000 7fb9bf7fe640 1 --2- 172.21.15.172:0/2226754480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c0162540 0x7fb9c01629d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00b6db0 tx=0x7fb9b402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932332e88b3c132d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.790+0000 7fb9beffd640 1 -- 172.21.15.172:0/2226754480 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4034070 con 0x7fb9c0162540 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.790+0000 7fb9beffd640 1 -- 172.21.15.172:0/2226754480 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b4030860 con 0x7fb9c0162540 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.790+0000 7fb9beffd640 1 -- 172.21.15.172:0/2226754480 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4030c30 con 0x7fb9c0162540 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.790+0000 7fb9c631e640 1 -- 172.21.15.172:0/2226754480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c0162540 msgr2=0x7fb9c01629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.790+0000 7fb9c631e640 1 --2- 172.21.15.172:0/2226754480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c0162540 0x7fb9c01629d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00b6db0 tx=0x7fb9b402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.790+0000 7fb9c631e640 1 -- 172.21.15.172:0/2226754480 shutdown_connections 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.790+0000 7fb9c631e640 1 --2- 172.21.15.172:0/2226754480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c0162540 0x7fb9c01629d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.790+0000 7fb9c631e640 1 --2- 172.21.15.172:0/2226754480 >> v2:172.21.15.177:3300/0 conn(0x7fb9c015e430 0x7fb9c0162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.790+0000 7fb9c631e640 1 -- 172.21.15.172:0/2226754480 >> 172.21.15.172:0/2226754480 conn(0x7fb9c009d560 msgr2=0x7fb9c00d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:46.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.790+0000 7fb9c631e640 1 -- 172.21.15.172:0/2226754480 shutdown_connections 2024-04-14T08:33:46.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.790+0000 7fb9c631e640 1 -- 172.21.15.172:0/2226754480 wait complete. 2024-04-14T08:33:46.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.791+0000 7fb9c631e640 1 Processor -- start 2024-04-14T08:33:46.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.792+0000 7fb9c631e640 1 -- start start 2024-04-14T08:33:46.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.792+0000 7fb9c631e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c00b3a30 0x7fb9c00ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.792+0000 7fb9c631e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c015e430 0x7fb9c01610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.792+0000 7fb9c631e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c0162540 0x7fb9c00cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.792+0000 7fb9c631e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c00d4e20 con 0x7fb9c00b3a30 2024-04-14T08:33:46.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.792+0000 7fb9c631e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c015e810 con 0x7fb9c015e430 2024-04-14T08:33:46.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.792+0000 7fb9c631e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c00d0050 con 0x7fb9c0162540 2024-04-14T08:33:46.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.792+0000 7fb9bffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c00b3a30 0x7fb9c00ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.792+0000 7fb9bffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c00b3a30 0x7fb9c00ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59876/0 (socket says 172.21.15.172:59876) 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.792+0000 7fb9bffff640 1 -- 172.21.15.172:0/1359829439 learned_addr learned my addr 172.21.15.172:0/1359829439 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.792+0000 7fb9bf7fe640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c015e430 0x7fb9c01610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9c48bb640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c0162540 0x7fb9c00cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9bffff640 1 -- 172.21.15.172:0/1359829439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c0162540 msgr2=0x7fb9c00cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9bffff640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c0162540 0x7fb9c00cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9bffff640 1 -- 172.21.15.172:0/1359829439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c015e430 msgr2=0x7fb9c01610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9bffff640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c015e430 0x7fb9c01610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9bffff640 1 -- 172.21.15.172:0/1359829439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c00cc5f0 con 0x7fb9c00b3a30 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9bf7fe640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c015e430 0x7fb9c01610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9bffff640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c00b3a30 0x7fb9c00ccee0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000ebe0 tx=0x7fb9b000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9bcff9640 1 -- 172.21.15.172:0/1359829439 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0017070 con 0x7fb9c00b3a30 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9bcff9640 1 -- 172.21.15.172:0/1359829439 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b000cbd0 con 0x7fb9c00b3a30 2024-04-14T08:33:46.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9bcff9640 1 -- 172.21.15.172:0/1359829439 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0013470 con 0x7fb9c00b3a30 2024-04-14T08:33:46.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9c631e640 1 -- 172.21.15.172:0/1359829439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c0161860 con 0x7fb9c00b3a30 2024-04-14T08:33:46.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.793+0000 7fb9c631e640 1 -- 172.21.15.172:0/1359829439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c00cb5d0 con 0x7fb9c00b3a30 2024-04-14T08:33:46.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.794+0000 7fb9c631e640 1 -- 172.21.15.172:0/1359829439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c0068590 con 0x7fb9c00b3a30 2024-04-14T08:33:46.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.795+0000 7fb9bcff9640 1 -- 172.21.15.172:0/1359829439 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9b001a070 con 0x7fb9c00b3a30 2024-04-14T08:33:46.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.795+0000 7fb9bcff9640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9ac06c8f0 0x7fb9ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:46.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.795+0000 7fb9bcff9640 1 -- 172.21.15.172:0/1359829439 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb9b008bdd0 con 0x7fb9c00b3a30 2024-04-14T08:33:46.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.795+0000 7fb9bf7fe640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9ac06c8f0 0x7fb9ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:46.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.796+0000 7fb9bf7fe640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9ac06c8f0 0x7fb9ac06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4036000 tx=0x7fb9b403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:46.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.799+0000 7fb9bcff9640 1 -- 172.21.15.172:0/1359829439 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb9b00597b0 con 0x7fb9c00b3a30 2024-04-14T08:33:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.973+0000 7fb9c631e640 1 -- 172.21.15.172:0/1359829439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb9c00d18a0 con 0x7fb9c00b3a30 2024-04-14T08:33:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.974+0000 7fb9bcff9640 1 -- 172.21.15.172:0/1359829439 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fb9b005dab0 con 0x7fb9c00b3a30 2024-04-14T08:33:46.974 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:33:46.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.976+0000 7fb9c631e640 1 -- 172.21.15.172:0/1359829439 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9ac06c8f0 msgr2=0x7fb9ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.976+0000 7fb9c631e640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9ac06c8f0 0x7fb9ac06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4036000 tx=0x7fb9b403c040 comp rx=0 tx=0).stop 2024-04-14T08:33:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.976+0000 7fb9c631e640 1 -- 172.21.15.172:0/1359829439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c00b3a30 msgr2=0x7fb9c00ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.976+0000 7fb9c631e640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c00b3a30 0x7fb9c00ccee0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000ebe0 tx=0x7fb9b000c520 comp rx=0 tx=0).stop 2024-04-14T08:33:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.976+0000 7fb9c631e640 1 -- 172.21.15.172:0/1359829439 shutdown_connections 2024-04-14T08:33:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.976+0000 7fb9c631e640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9ac06c8f0 0x7fb9ac06edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.976+0000 7fb9c631e640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c0162540 0x7fb9c00cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.976+0000 7fb9c631e640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c015e430 0x7fb9c01610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.976+0000 7fb9c631e640 1 --2- 172.21.15.172:0/1359829439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c00b3a30 0x7fb9c00ccee0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.976+0000 7fb9c631e640 1 -- 172.21.15.172:0/1359829439 >> 172.21.15.172:0/1359829439 conn(0x7fb9c009d560 msgr2=0x7fb9c00b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.976+0000 7fb9c631e640 1 -- 172.21.15.172:0/1359829439 shutdown_connections 2024-04-14T08:33:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:46.976+0000 7fb9c631e640 1 -- 172.21.15.172:0/1359829439 wait complete. 2024-04-14T08:33:46.987 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:33:47.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.086+0000 7f21bca60640 1 Processor -- start 2024-04-14T08:33:47.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.087+0000 7f21bca60640 1 -- start start 2024-04-14T08:33:47.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.087+0000 7f21bca60640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f21b806b520 0x7f21b806fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.087+0000 7f21bca60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21b80b3dd0 0x7f21b806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.087+0000 7f21bca60640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21b806be40 con 0x7f21b806e130 2024-04-14T08:33:47.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.087+0000 7f21bca60640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f21b8072d80 con 0x7f21b806b520 2024-04-14T08:33:47.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.087+0000 7f21bca60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f21b80b4540 con 0x7f21b80b3dd0 2024-04-14T08:33:47.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.088+0000 7f21b659c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f21b806b520 0x7f21b806fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.088+0000 7f21b6d9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f21b806e130 0x7f21b806e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54194/0 (socket says 172.21.15.172:54194) 2024-04-14T08:33:47.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.088+0000 7f21b6d9d640 1 -- 172.21.15.172:0/2745145918 learned_addr learned my addr 172.21.15.172:0/2745145918 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:47.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.088+0000 7f21b5d9b640 1 --2- 172.21.15.172:0/2745145918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21b80b3dd0 0x7f21b806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.088+0000 7f21b559a640 1 -- 172.21.15.172:0/2745145918 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3188746134 0 0) 0x7f21b806be40 con 0x7f21b806e130 2024-04-14T08:33:47.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.088+0000 7f21b559a640 1 -- 172.21.15.172:0/2745145918 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21a0003520 con 0x7f21b806e130 2024-04-14T08:33:47.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.088+0000 7f21b5d9b640 1 -- 172.21.15.172:0/2745145918 >> v2:172.21.15.172:3300/0 conn(0x7f21b806b520 msgr2=0x7f21b806fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:47.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.088+0000 7f21b5d9b640 1 --2- 172.21.15.172:0/2745145918 >> v2:172.21.15.172:3300/0 conn(0x7f21b806b520 0x7f21b806fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.088+0000 7f21b5d9b640 1 -- 172.21.15.172:0/2745145918 >> v1:172.21.15.172:6789/0 conn(0x7f21b806e130 legacy=0x7f21b806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.088+0000 7f21b5d9b640 1 -- 172.21.15.172:0/2745145918 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b8072450 con 0x7f21b80b3dd0 2024-04-14T08:33:47.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.088+0000 7f21b659c640 1 --2- 172.21.15.172:0/2745145918 >> v2:172.21.15.172:3300/0 conn(0x7f21b806b520 0x7f21b806fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:47.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.089+0000 7f21b5d9b640 1 --2- 172.21.15.172:0/2745145918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21b80b3dd0 0x7f21b806b900 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f21a80097a0 tx=0x7f21a802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d03f6cbca6fc853 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:47.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.089+0000 7f21b559a640 1 -- 172.21.15.172:0/2745145918 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8035020 con 0x7f21b80b3dd0 2024-04-14T08:33:47.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.089+0000 7f21b559a640 1 -- 172.21.15.172:0/2745145918 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a8031810 con 0x7f21b80b3dd0 2024-04-14T08:33:47.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.089+0000 7f21b559a640 1 -- 172.21.15.172:0/2745145918 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8031bc0 con 0x7f21b80b3dd0 2024-04-14T08:33:47.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.089+0000 7f21bca60640 1 -- 172.21.15.172:0/2745145918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21b80b3dd0 msgr2=0x7f21b806b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.089+0000 7f21bca60640 1 --2- 172.21.15.172:0/2745145918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21b80b3dd0 0x7f21b806b900 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f21a80097a0 tx=0x7f21a802b8b0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.089+0000 7f21bca60640 1 -- 172.21.15.172:0/2745145918 shutdown_connections 2024-04-14T08:33:47.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.089+0000 7f21bca60640 1 --2- 172.21.15.172:0/2745145918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21b80b3dd0 0x7f21b806b900 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.089+0000 7f21bca60640 1 --2- 172.21.15.172:0/2745145918 >> v2:172.21.15.172:3300/0 conn(0x7f21b806b520 0x7f21b806fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.089+0000 7f21bca60640 1 -- 172.21.15.172:0/2745145918 >> 172.21.15.172:0/2745145918 conn(0x7f21b80d0f20 msgr2=0x7f21b80d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:47.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.090+0000 7f21bca60640 1 -- 172.21.15.172:0/2745145918 shutdown_connections 2024-04-14T08:33:47.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.090+0000 7f21bca60640 1 -- 172.21.15.172:0/2745145918 wait complete. 2024-04-14T08:33:47.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.091+0000 7f21bca60640 1 Processor -- start 2024-04-14T08:33:47.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.091+0000 7f21bca60640 1 -- start start 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.091+0000 7f21bca60640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21b806b520 0x7f21b813cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.091+0000 7f21bca60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21b806e130 0x7f21b813c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.091+0000 7f21bca60640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21b80b3dd0 0x7f21b813e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.091+0000 7f21bca60640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21b80b4250 con 0x7f21b80b3dd0 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.091+0000 7f21bca60640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f21b8072a10 con 0x7f21b806b520 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.091+0000 7f21bca60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f21b8073210 con 0x7f21b806e130 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21b6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21b80b3dd0 0x7f21b813e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21b6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21b80b3dd0 0x7f21b813e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59894/0 (socket says 172.21.15.172:59894) 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21b659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21b806b520 0x7f21b813cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21b6d9d640 1 -- 172.21.15.172:0/3861020307 learned_addr learned my addr 172.21.15.172:0/3861020307 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21b5d9b640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21b806e130 0x7f21b813c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21b6d9d640 1 -- 172.21.15.172:0/3861020307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21b806e130 msgr2=0x7f21b813c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21b6d9d640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21b806e130 0x7f21b813c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21b6d9d640 1 -- 172.21.15.172:0/3861020307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21b806b520 msgr2=0x7f21b813cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21b6d9d640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21b806b520 0x7f21b813cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21b6d9d640 1 -- 172.21.15.172:0/3861020307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b814cf50 con 0x7f21b80b3dd0 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21b6d9d640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21b80b3dd0 0x7f21b813e1d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f21b000ca20 tx=0x7f21b000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21a6ffd640 1 -- 172.21.15.172:0/3861020307 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0017070 con 0x7f21b80b3dd0 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21a6ffd640 1 -- 172.21.15.172:0/3861020307 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b000abd0 con 0x7f21b80b3dd0 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21a6ffd640 1 -- 172.21.15.172:0/3861020307 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b0012400 con 0x7f21b80b3dd0 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21bca60640 1 -- 172.21.15.172:0/3861020307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b813d130 con 0x7f21b80b3dd0 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.092+0000 7f21bca60640 1 -- 172.21.15.172:0/3861020307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b813d6f0 con 0x7f21b80b3dd0 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.093+0000 7f21bca60640 1 -- 172.21.15.172:0/3861020307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b8068520 con 0x7f21b80b3dd0 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.094+0000 7f21a6ffd640 1 -- 172.21.15.172:0/3861020307 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21b001a070 con 0x7f21b80b3dd0 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.094+0000 7f21a6ffd640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f219806c8f0 0x7f219806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.094+0000 7f21a6ffd640 1 -- 172.21.15.172:0/3861020307 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f21b008bc80 con 0x7f21b80b3dd0 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.094+0000 7f21b659c640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f219806c8f0 0x7f219806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.094+0000 7f21b659c640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f219806c8f0 0x7f219806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009560 tx=0x7f21ac009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:47.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.098+0000 7f21a6ffd640 1 -- 172.21.15.172:0/3861020307 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f21b814cf50 con 0x7f21b80b3dd0 2024-04-14T08:33:47.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.311+0000 7f21bca60640 1 -- 172.21.15.172:0/3861020307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21b8074fa0 con 0x7f21b80b3dd0 2024-04-14T08:33:47.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.312+0000 7f21a6ffd640 1 -- 172.21.15.172:0/3861020307 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f21b00595e0 con 0x7f21b80b3dd0 2024-04-14T08:33:47.312 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:47.313 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3053874257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3053874257},{"type":"v1","addr":"172.21.15.172:6835","nonce":3053874257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2790588849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2790588849},{"type":"v1","addr":"172.21.15.172:6837","nonce":2790588849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3130776573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3130776573},{"type":"v1","addr":"172.21.15.177:6834","nonce":3130776573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:32:53.535483+0000","modified":"2024-04-14T08:33:09.604218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/517963877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":517963877},{"type":"v1","addr":"172.21.15.177:6835","nonce":517963877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4309,"qdb_cluster":[4309]},"id":1}]} 2024-04-14T08:33:47.313 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 9 2024-04-14T08:33:47.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.314+0000 7f21bca60640 1 -- 172.21.15.172:0/3861020307 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f219806c8f0 msgr2=0x7f219806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.314+0000 7f21bca60640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f219806c8f0 0x7f219806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009560 tx=0x7f21ac009190 comp rx=0 tx=0).stop 2024-04-14T08:33:47.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.314+0000 7f21bca60640 1 -- 172.21.15.172:0/3861020307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21b80b3dd0 msgr2=0x7f21b813e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.314+0000 7f21bca60640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21b80b3dd0 0x7f21b813e1d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f21b000ca20 tx=0x7f21b000cee0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.314+0000 7f21bca60640 1 -- 172.21.15.172:0/3861020307 shutdown_connections 2024-04-14T08:33:47.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.314+0000 7f21bca60640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f219806c8f0 0x7f219806edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.314+0000 7f21bca60640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21b80b3dd0 0x7f21b813e1d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.314+0000 7f21bca60640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21b806e130 0x7f21b813c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.314+0000 7f21bca60640 1 --2- 172.21.15.172:0/3861020307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21b806b520 0x7f21b813cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.314+0000 7f21bca60640 1 -- 172.21.15.172:0/3861020307 >> 172.21.15.172:0/3861020307 conn(0x7f21b80d0f20 msgr2=0x7f21b80b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:47.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.314+0000 7f21bca60640 1 -- 172.21.15.172:0/3861020307 shutdown_connections 2024-04-14T08:33:47.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.314+0000 7f21bca60640 1 -- 172.21.15.172:0/3861020307 wait complete. 2024-04-14T08:33:47.326 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:47.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.427+0000 7f7b99ede640 1 Processor -- start 2024-04-14T08:33:47.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.427+0000 7f7b99ede640 1 -- start start 2024-04-14T08:33:47.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b99ede640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7b940b1df0 0x7f7b940b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b99ede640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7b940b21d0 0x7f7b9415c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b99ede640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b940b3430 con 0x7f7b9415c5a0 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b99ede640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f7b940b3700 con 0x7f7b940b1df0 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b99ede640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f7b940b39e0 con 0x7f7b940b21d0 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b937fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7b940b1df0 0x7f7b940b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b937fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7b940b1df0 0x7f7b940b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59908/0 (socket says 172.21.15.172:59908) 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b937fe640 1 -- 172.21.15.172:0/383088890 learned_addr learned my addr 172.21.15.172:0/383088890 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b92ffd640 1 --2- 172.21.15.172:0/383088890 >> v2:172.21.15.177:3300/0 conn(0x7f7b940b21d0 0x7f7b9415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b937fe640 1 -- 172.21.15.172:0/383088890 >> v2:172.21.15.177:3300/0 conn(0x7f7b940b21d0 msgr2=0x7f7b9415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b937fe640 1 --2- 172.21.15.172:0/383088890 >> v2:172.21.15.177:3300/0 conn(0x7f7b940b21d0 0x7f7b9415c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b937fe640 1 -- 172.21.15.172:0/383088890 >> v1:172.21.15.177:6789/0 conn(0x7f7b9415c5a0 legacy=0x7f7b940b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.428+0000 7f7b937fe640 1 -- 172.21.15.172:0/383088890 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b940b3cc0 con 0x7f7b940b1df0 2024-04-14T08:33:47.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.429+0000 7f7b937fe640 1 --2- 172.21.15.172:0/383088890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b940b1df0 0x7f7b940b2ef0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7b9406c0d0 tx=0x7f7b8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c916c8f0c98ea11 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:47.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.429+0000 7f7b927fc640 1 -- 172.21.15.172:0/383088890 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84035020 con 0x7f7b940b1df0 2024-04-14T08:33:47.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.429+0000 7f7b927fc640 1 -- 172.21.15.172:0/383088890 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b84031870 con 0x7f7b940b1df0 2024-04-14T08:33:47.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.429+0000 7f7b927fc640 1 -- 172.21.15.172:0/383088890 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84031c40 con 0x7f7b940b1df0 2024-04-14T08:33:47.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.430+0000 7f7b99ede640 1 -- 172.21.15.172:0/383088890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b940b1df0 msgr2=0x7f7b940b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.430+0000 7f7b99ede640 1 --2- 172.21.15.172:0/383088890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b940b1df0 0x7f7b940b2ef0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7b9406c0d0 tx=0x7f7b8402b910 comp rx=0 tx=0).stop 2024-04-14T08:33:47.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.430+0000 7f7b99ede640 1 -- 172.21.15.172:0/383088890 shutdown_connections 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.430+0000 7f7b99ede640 1 --2- 172.21.15.172:0/383088890 >> v2:172.21.15.177:3300/0 conn(0x7f7b940b21d0 0x7f7b9415c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.430+0000 7f7b99ede640 1 --2- 172.21.15.172:0/383088890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b940b1df0 0x7f7b940b2ef0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.430+0000 7f7b99ede640 1 -- 172.21.15.172:0/383088890 >> 172.21.15.172:0/383088890 conn(0x7f7b9406ec30 msgr2=0x7f7b9406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.430+0000 7f7b99ede640 1 -- 172.21.15.172:0/383088890 shutdown_connections 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.430+0000 7f7b99ede640 1 -- 172.21.15.172:0/383088890 wait complete. 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.431+0000 7f7b99ede640 1 Processor -- start 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.431+0000 7f7b99ede640 1 -- start start 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b99ede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b940b1df0 0x7f7b94157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b99ede640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b940b21d0 0x7f7b941f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b99ede640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b941f36c0 0x7f7b940728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b99ede640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b940b4750 con 0x7f7b941f36c0 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b99ede640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9415b7f0 con 0x7f7b940b21d0 2024-04-14T08:33:47.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b99ede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7b940b48d0 con 0x7f7b940b1df0 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b941f36c0 0x7f7b940728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b941f36c0 0x7f7b940728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59916/0 (socket says 172.21.15.172:59916) 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b93fff640 1 -- 172.21.15.172:0/2310321197 learned_addr learned my addr 172.21.15.172:0/2310321197 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b92ffd640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b940b21d0 0x7f7b941f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b937fe640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b940b1df0 0x7f7b94157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b93fff640 1 -- 172.21.15.172:0/2310321197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b940b1df0 msgr2=0x7f7b94157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b93fff640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b940b1df0 0x7f7b94157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b93fff640 1 -- 172.21.15.172:0/2310321197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b940b21d0 msgr2=0x7f7b941f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b93fff640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b940b21d0 0x7f7b941f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b93fff640 1 -- 172.21.15.172:0/2310321197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b941f3490 con 0x7f7b941f36c0 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b93fff640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b941f36c0 0x7f7b940728c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c7d0 tx=0x7f7b8c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b7bfff640 1 -- 172.21.15.172:0/2310321197 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c003b70 con 0x7f7b941f36c0 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.432+0000 7f7b7bfff640 1 -- 172.21.15.172:0/2310321197 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8c003d30 con 0x7f7b941f36c0 2024-04-14T08:33:47.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.433+0000 7f7b99ede640 1 -- 172.21.15.172:0/2310321197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b940abae0 con 0x7f7b941f36c0 2024-04-14T08:33:47.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.433+0000 7f7b7bfff640 1 -- 172.21.15.172:0/2310321197 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00aa90 con 0x7f7b941f36c0 2024-04-14T08:33:47.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.433+0000 7f7b99ede640 1 -- 172.21.15.172:0/2310321197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b940ac080 con 0x7f7b941f36c0 2024-04-14T08:33:47.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.433+0000 7f7b99ede640 1 -- 172.21.15.172:0/2310321197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b94068520 con 0x7f7b941f36c0 2024-04-14T08:33:47.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.434+0000 7f7b7bfff640 1 -- 172.21.15.172:0/2310321197 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b8c00ac30 con 0x7f7b941f36c0 2024-04-14T08:33:47.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.434+0000 7f7b7bfff640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7b8006c920 0x7f7b8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.434+0000 7f7b7bfff640 1 -- 172.21.15.172:0/2310321197 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7b8c08bd50 con 0x7f7b941f36c0 2024-04-14T08:33:47.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.434+0000 7f7b937fe640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7b8006c920 0x7f7b8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.435+0000 7f7b937fe640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7b8006c920 0x7f7b8006ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b84036000 tx=0x7f7b8403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:47.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.439+0000 7f7b7bfff640 1 -- 172.21.15.172:0/2310321197 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7b8c08f030 con 0x7f7b941f36c0 2024-04-14T08:33:47.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.613+0000 7f7b99ede640 1 -- 172.21.15.172:0/2310321197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b9415d2d0 con 0x7f7b941f36c0 2024-04-14T08:33:47.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.614+0000 7f7b7bfff640 1 -- 172.21.15.172:0/2310321197 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18962 (secure 0 0 0) 0x7f7b8c059730 con 0x7f7b941f36c0 2024-04-14T08:33:47.615 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:47.615 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":27,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:33:34.330426+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:47.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.616+0000 7f7b99ede640 1 -- 172.21.15.172:0/2310321197 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7b8006c920 msgr2=0x7f7b8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.616+0000 7f7b99ede640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7b8006c920 0x7f7b8006ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b84036000 tx=0x7f7b8403c040 comp rx=0 tx=0).stop 2024-04-14T08:33:47.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.616+0000 7f7b99ede640 1 -- 172.21.15.172:0/2310321197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b941f36c0 msgr2=0x7f7b940728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.616+0000 7f7b99ede640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b941f36c0 0x7f7b940728c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c7d0 tx=0x7f7b8c00cc90 comp rx=0 tx=0).stop 2024-04-14T08:33:47.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.616+0000 7f7b99ede640 1 -- 172.21.15.172:0/2310321197 shutdown_connections 2024-04-14T08:33:47.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.616+0000 7f7b99ede640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7b8006c920 0x7f7b8006ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.616+0000 7f7b99ede640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b941f36c0 0x7f7b940728c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.616+0000 7f7b99ede640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b940b21d0 0x7f7b941f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.616+0000 7f7b99ede640 1 --2- 172.21.15.172:0/2310321197 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b940b1df0 0x7f7b94157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.617+0000 7f7b99ede640 1 -- 172.21.15.172:0/2310321197 >> 172.21.15.172:0/2310321197 conn(0x7f7b9406ec30 msgr2=0x7f7b94158960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:47.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.617+0000 7f7b99ede640 1 -- 172.21.15.172:0/2310321197 shutdown_connections 2024-04-14T08:33:47.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.617+0000 7f7b99ede640 1 -- 172.21.15.172:0/2310321197 wait complete. 2024-04-14T08:33:47.628 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-14T08:33:47.629 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:33:47.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.728+0000 7fc9de3a3640 1 Processor -- start 2024-04-14T08:33:47.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.729+0000 7fc9de3a3640 1 -- start start 2024-04-14T08:33:47.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.729+0000 7fc9de3a3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc9d806b520 0x7fc9d806fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.729+0000 7fc9de3a3640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc9d80b3dd0 0x7fc9d806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.729+0000 7fc9de3a3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9d806e130 0x7fc9d806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.729+0000 7fc9de3a3640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc9d8072d80 con 0x7fc9d806b520 2024-04-14T08:33:47.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.729+0000 7fc9de3a3640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc9d806be40 con 0x7fc9d80b3dd0 2024-04-14T08:33:47.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.729+0000 7fc9de3a3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d80b4540 con 0x7fc9d806e130 2024-04-14T08:33:47.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.729+0000 7fc9d7fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc9d806b520 0x7fc9d806fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.730+0000 7fc9d7fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc9d806b520 0x7fc9d806fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59918/0 (socket says 172.21.15.172:59918) 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.730+0000 7fc9d77fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc9d80b3dd0 0x7fc9d806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.730+0000 7fc9d7fff640 1 -- 172.21.15.172:0/1664503565 learned_addr learned my addr 172.21.15.172:0/1664503565 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.730+0000 7fc9dc940640 1 --2- 172.21.15.172:0/1664503565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9d806e130 0x7fc9d806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.730+0000 7fc9d7fff640 1 -- 172.21.15.172:0/1664503565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9d806e130 msgr2=0x7fc9d806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.730+0000 7fc9d7fff640 1 --2- 172.21.15.172:0/1664503565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9d806e130 0x7fc9d806e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.730+0000 7fc9d7fff640 1 -- 172.21.15.172:0/1664503565 >> v2:172.21.15.177:3300/0 conn(0x7fc9d80b3dd0 msgr2=0x7fc9d806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.730+0000 7fc9d7fff640 1 --2- 172.21.15.172:0/1664503565 >> v2:172.21.15.177:3300/0 conn(0x7fc9d80b3dd0 0x7fc9d806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.730+0000 7fc9d7fff640 1 -- 172.21.15.172:0/1664503565 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d8072450 con 0x7fc9d806b520 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.730+0000 7fc9d77fe640 1 --2- 172.21.15.172:0/1664503565 >> v2:172.21.15.177:3300/0 conn(0x7fc9d80b3dd0 0x7fc9d806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.730+0000 7fc9d7fff640 1 --2- 172.21.15.172:0/1664503565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9d806b520 0x7fc9d806fdf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8072360 tx=0x7fc9c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea4ad16a02f3bec server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.731+0000 7fc9d6ffd640 1 -- 172.21.15.172:0/1664503565 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8035020 con 0x7fc9d806b520 2024-04-14T08:33:47.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.731+0000 7fc9d6ffd640 1 -- 172.21.15.172:0/1664503565 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c8031870 con 0x7fc9d806b520 2024-04-14T08:33:47.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.731+0000 7fc9d6ffd640 1 -- 172.21.15.172:0/1664503565 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8031c40 con 0x7fc9d806b520 2024-04-14T08:33:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.731+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1664503565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9d806b520 msgr2=0x7fc9d806fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.731+0000 7fc9de3a3640 1 --2- 172.21.15.172:0/1664503565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9d806b520 0x7fc9d806fdf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8072360 tx=0x7fc9c802b910 comp rx=0 tx=0).stop 2024-04-14T08:33:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.731+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1664503565 shutdown_connections 2024-04-14T08:33:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.731+0000 7fc9de3a3640 1 --2- 172.21.15.172:0/1664503565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9d806e130 0x7fc9d806e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.731+0000 7fc9de3a3640 1 --2- 172.21.15.172:0/1664503565 >> v2:172.21.15.177:3300/0 conn(0x7fc9d80b3dd0 0x7fc9d806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.731+0000 7fc9de3a3640 1 --2- 172.21.15.172:0/1664503565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9d806b520 0x7fc9d806fdf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.731+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1664503565 >> 172.21.15.172:0/1664503565 conn(0x7fc9d80d0f20 msgr2=0x7fc9d80d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.731+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1664503565 shutdown_connections 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.731+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1664503565 wait complete. 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.732+0000 7fc9de3a3640 1 Processor -- start 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.733+0000 7fc9de3a3640 1 -- start start 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.733+0000 7fc9de3a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9d806b520 0x7fc9d8139240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.733+0000 7fc9de3a3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9d806e130 0x7fc9d8154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.733+0000 7fc9de3a3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9d80b3dd0 0x7fc9d8154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.733+0000 7fc9de3a3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d80b4250 con 0x7fc9d806b520 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.733+0000 7fc9de3a3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8072a10 con 0x7fc9d806e130 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.733+0000 7fc9de3a3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d8073210 con 0x7fc9d80b3dd0 2024-04-14T08:33:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.733+0000 7fc9d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9d806b520 0x7fc9d8139240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.733+0000 7fc9d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9d806b520 0x7fc9d8139240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59934/0 (socket says 172.21.15.172:59934) 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.733+0000 7fc9d7fff640 1 -- 172.21.15.172:0/1607562938 learned_addr learned my addr 172.21.15.172:0/1607562938 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.733+0000 7fc9dc940640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9d80b3dd0 0x7fc9d8154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.733+0000 7fc9d77fe640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9d806e130 0x7fc9d8154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.734+0000 7fc9d7fff640 1 -- 172.21.15.172:0/1607562938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9d80b3dd0 msgr2=0x7fc9d8154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.734+0000 7fc9d7fff640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9d80b3dd0 0x7fc9d8154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.734+0000 7fc9d7fff640 1 -- 172.21.15.172:0/1607562938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9d806e130 msgr2=0x7fc9d8154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.734+0000 7fc9d7fff640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9d806e130 0x7fc9d8154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.734+0000 7fc9d7fff640 1 -- 172.21.15.172:0/1607562938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d8139780 con 0x7fc9d806b520 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.734+0000 7fc9d7fff640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9d806b520 0x7fc9d8139240 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc9c802b950 tx=0x7fc9c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.734+0000 7fc9d4ff9640 1 -- 172.21.15.172:0/1607562938 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8040070 con 0x7fc9d806b520 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.734+0000 7fc9d4ff9640 1 -- 172.21.15.172:0/1607562938 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9c8031e10 con 0x7fc9d806b520 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.734+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1607562938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d81577c0 con 0x7fc9d806b520 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.734+0000 7fc9d4ff9640 1 -- 172.21.15.172:0/1607562938 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8035040 con 0x7fc9d806b520 2024-04-14T08:33:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.734+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1607562938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d8157d80 con 0x7fc9d806b520 2024-04-14T08:33:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.735+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1607562938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d8068520 con 0x7fc9d806b520 2024-04-14T08:33:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.735+0000 7fc9d4ff9640 1 -- 172.21.15.172:0/1607562938 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9c804b060 con 0x7fc9d806b520 2024-04-14T08:33:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.736+0000 7fc9d4ff9640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc9c406c8f0 0x7fc9c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.736+0000 7fc9d4ff9640 1 -- 172.21.15.172:0/1607562938 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc9c80ad9b0 con 0x7fc9d806b520 2024-04-14T08:33:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.736+0000 7fc9d77fe640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc9c406c8f0 0x7fc9c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.736+0000 7fc9d77fe640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc9c406c8f0 0x7fc9c406edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009560 tx=0x7fc9cc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.740+0000 7fc9d4ff9640 1 -- 172.21.15.172:0/1607562938 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc9c807b420 con 0x7fc9d806b520 2024-04-14T08:33:47.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.952+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1607562938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9d80ae5a0 con 0x7fc9d806b520 2024-04-14T08:33:47.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.953+0000 7fc9d4ff9640 1 -- 172.21.15.172:0/1607562938 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7fc9c807f720 con 0x7fc9d806b520 2024-04-14T08:33:47.954 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:47.954 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3053874257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3053874257},{"type":"v1","addr":"172.21.15.172:6835","nonce":3053874257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2790588849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2790588849},{"type":"v1","addr":"172.21.15.172:6837","nonce":2790588849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3130776573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3130776573},{"type":"v1","addr":"172.21.15.177:6834","nonce":3130776573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:32:53.535483+0000","modified":"2024-04-14T08:33:09.604218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/517963877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":517963877},{"type":"v1","addr":"172.21.15.177:6835","nonce":517963877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4309,"qdb_cluster":[4309]},"id":1}]} 2024-04-14T08:33:47.954 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 9 2024-04-14T08:33:47.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.955+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1607562938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc9c406c8f0 msgr2=0x7fc9c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.955+0000 7fc9de3a3640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc9c406c8f0 0x7fc9c406edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009560 tx=0x7fc9cc009190 comp rx=0 tx=0).stop 2024-04-14T08:33:47.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.955+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1607562938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9d806b520 msgr2=0x7fc9d8139240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:47.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.955+0000 7fc9de3a3640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9d806b520 0x7fc9d8139240 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc9c802b950 tx=0x7fc9c80046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.955+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1607562938 shutdown_connections 2024-04-14T08:33:47.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.955+0000 7fc9de3a3640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc9c406c8f0 0x7fc9c406edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.955+0000 7fc9de3a3640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9d80b3dd0 0x7fc9d8154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.955+0000 7fc9de3a3640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9d806e130 0x7fc9d8154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.955+0000 7fc9de3a3640 1 --2- 172.21.15.172:0/1607562938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9d806b520 0x7fc9d8139240 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:47.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.955+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1607562938 >> 172.21.15.172:0/1607562938 conn(0x7fc9d80d0f20 msgr2=0x7fc9d806e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:47.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.955+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1607562938 shutdown_connections 2024-04-14T08:33:47.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:47.955+0000 7fc9de3a3640 1 -- 172.21.15.172:0/1607562938 wait complete. 2024-04-14T08:33:47.967 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:33:48.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.067+0000 7f0744b15640 1 Processor -- start 2024-04-14T08:33:48.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.067+0000 7f0744b15640 1 -- start start 2024-04-14T08:33:48.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.068+0000 7f0744b15640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f07400d2830 0x7f07400d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.068+0000 7f0744b15640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f074015e890 0x7f07400d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.068+0000 7f0744b15640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07400d3150 con 0x7f07400cedd0 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.068+0000 7f0744b15640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f07400cff30 con 0x7f074015e890 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.068+0000 7f0744b15640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f07400d2350 con 0x7f07400d2830 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.068+0000 7f073dd9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f074015e890 0x7f07400d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.068+0000 7f073dd9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f074015e890 0x7f07400d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59950/0 (socket says 172.21.15.172:59950) 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.068+0000 7f073dd9b640 1 -- 172.21.15.172:0/1917342831 learned_addr learned my addr 172.21.15.172:0/1917342831 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.068+0000 7f073e59c640 1 --2- 172.21.15.172:0/1917342831 >> v2:172.21.15.177:3300/0 conn(0x7f07400d2830 0x7f07400d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.069+0000 7f073dd9b640 1 -- 172.21.15.172:0/1917342831 >> v2:172.21.15.177:3300/0 conn(0x7f07400d2830 msgr2=0x7f07400d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.069+0000 7f073dd9b640 1 --2- 172.21.15.172:0/1917342831 >> v2:172.21.15.177:3300/0 conn(0x7f07400d2830 0x7f07400d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.069+0000 7f073dd9b640 1 -- 172.21.15.172:0/1917342831 >> v1:172.21.15.177:6789/0 conn(0x7f07400cedd0 legacy=0x7f07401621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.069+0000 7f073dd9b640 1 -- 172.21.15.172:0/1917342831 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07400cf320 con 0x7f074015e890 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.069+0000 7f073dd9b640 1 --2- 172.21.15.172:0/1917342831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f074015e890 0x7f07400d1300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f074008c830 tx=0x7f073402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4e26adac64a42c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:48.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.070+0000 7f073d59a640 1 -- 172.21.15.172:0/1917342831 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073402cbb0 con 0x7f074015e890 2024-04-14T08:33:48.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.070+0000 7f073d59a640 1 -- 172.21.15.172:0/1917342831 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f073402cd50 con 0x7f074015e890 2024-04-14T08:33:48.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.070+0000 7f073d59a640 1 -- 172.21.15.172:0/1917342831 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734034610 con 0x7f074015e890 2024-04-14T08:33:48.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.070+0000 7f0744b15640 1 -- 172.21.15.172:0/1917342831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f074015e890 msgr2=0x7f07400d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.070+0000 7f0744b15640 1 --2- 172.21.15.172:0/1917342831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f074015e890 0x7f07400d1300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f074008c830 tx=0x7f073402c010 comp rx=0 tx=0).stop 2024-04-14T08:33:48.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.070+0000 7f0744b15640 1 -- 172.21.15.172:0/1917342831 shutdown_connections 2024-04-14T08:33:48.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.070+0000 7f0744b15640 1 --2- 172.21.15.172:0/1917342831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f074015e890 0x7f07400d1300 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.070+0000 7f0744b15640 1 --2- 172.21.15.172:0/1917342831 >> v2:172.21.15.177:3300/0 conn(0x7f07400d2830 0x7f07400d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.070+0000 7f0744b15640 1 -- 172.21.15.172:0/1917342831 >> 172.21.15.172:0/1917342831 conn(0x7f074009d590 msgr2=0x7f07400d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:48.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.070+0000 7f0744b15640 1 -- 172.21.15.172:0/1917342831 shutdown_connections 2024-04-14T08:33:48.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.070+0000 7f0744b15640 1 -- 172.21.15.172:0/1917342831 wait complete. 2024-04-14T08:33:48.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.071+0000 7f0744b15640 1 Processor -- start 2024-04-14T08:33:48.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.072+0000 7f0744b15640 1 -- start start 2024-04-14T08:33:48.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.072+0000 7f0744b15640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07400cedd0 0x7f0740160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.072+0000 7f0744b15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07400d2830 0x7f074014b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.072+0000 7f0744b15640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f074015e890 0x7f07400b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.072+0000 7f0744b15640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f07400cfbc0 con 0x7f07400d2830 2024-04-14T08:33:48.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.072+0000 7f0744b15640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f07400cf8e0 con 0x7f07400cedd0 2024-04-14T08:33:48.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.072+0000 7f0744b15640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f07400d1840 con 0x7f074015e890 2024-04-14T08:33:48.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.072+0000 7f073dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07400d2830 0x7f074014b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.072+0000 7f073dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07400d2830 0x7f074014b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59960/0 (socket says 172.21.15.172:59960) 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.072+0000 7f073dd9b640 1 -- 172.21.15.172:0/664165530 learned_addr learned my addr 172.21.15.172:0/664165530 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.072+0000 7f073e59c640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07400cedd0 0x7f0740160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.072+0000 7f073ed9d640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f074015e890 0x7f07400b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.073+0000 7f073dd9b640 1 -- 172.21.15.172:0/664165530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f074015e890 msgr2=0x7f07400b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.073+0000 7f073dd9b640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f074015e890 0x7f07400b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.073+0000 7f073dd9b640 1 -- 172.21.15.172:0/664165530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07400cedd0 msgr2=0x7f0740160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.073+0000 7f073dd9b640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07400cedd0 0x7f0740160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.073+0000 7f073dd9b640 1 -- 172.21.15.172:0/664165530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074015ff00 con 0x7f07400d2830 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.073+0000 7f073dd9b640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07400d2830 0x7f074014b780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0734000f80 tx=0x7f0734004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.073+0000 7f072effd640 1 -- 172.21.15.172:0/664165530 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734043020 con 0x7f07400d2830 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.073+0000 7f072effd640 1 -- 172.21.15.172:0/664165530 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f073403e450 con 0x7f07400d2830 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.073+0000 7f0744b15640 1 -- 172.21.15.172:0/664165530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07401601e0 con 0x7f07400d2830 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.073+0000 7f072effd640 1 -- 172.21.15.172:0/664165530 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734009510 con 0x7f07400d2830 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.073+0000 7f0744b15640 1 -- 172.21.15.172:0/664165530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07401607a0 con 0x7f07400d2830 2024-04-14T08:33:48.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.074+0000 7f0744b15640 1 -- 172.21.15.172:0/664165530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0740068520 con 0x7f07400d2830 2024-04-14T08:33:48.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.074+0000 7f072effd640 1 -- 172.21.15.172:0/664165530 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0734046020 con 0x7f07400d2830 2024-04-14T08:33:48.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.075+0000 7f072effd640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f072006c8f0 0x7f072006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.075+0000 7f072effd640 1 -- 172.21.15.172:0/664165530 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f07340ae5d0 con 0x7f07400d2830 2024-04-14T08:33:48.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.075+0000 7f073e59c640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f072006c8f0 0x7f072006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.075+0000 7f073e59c640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f072006c8f0 0x7f072006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0738009aa0 tx=0x7f0738009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:48.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.079+0000 7f072effd640 1 -- 172.21.15.172:0/664165530 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f073404d030 con 0x7f07400d2830 2024-04-14T08:33:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.290+0000 7f0744b15640 1 -- 172.21.15.172:0/664165530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07400ce4c0 con 0x7f07400d2830 2024-04-14T08:33:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.291+0000 7f072effd640 1 -- 172.21.15.172:0/664165530 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5186 (secure 0 0 0) 0x7f073407bfc0 con 0x7f07400d2830 2024-04-14T08:33:48.292 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:48.292 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3053874257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3053874257},{"type":"v1","addr":"172.21.15.172:6835","nonce":3053874257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2790588849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2790588849},{"type":"v1","addr":"172.21.15.172:6837","nonce":2790588849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3130776573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3130776573},{"type":"v1","addr":"172.21.15.177:6834","nonce":3130776573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:32:53.535483+0000","modified":"2024-04-14T08:33:09.604218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4309":{"gid":4309,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/517963877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":517963877},{"type":"v1","addr":"172.21.15.177:6835","nonce":517963877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4309,"qdb_cluster":[4309]},"id":1}]} 2024-04-14T08:33:48.292 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 9 2024-04-14T08:33:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.293+0000 7f0744b15640 1 -- 172.21.15.172:0/664165530 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f072006c8f0 msgr2=0x7f072006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.293+0000 7f0744b15640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f072006c8f0 0x7f072006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0738009aa0 tx=0x7f0738009190 comp rx=0 tx=0).stop 2024-04-14T08:33:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.293+0000 7f0744b15640 1 -- 172.21.15.172:0/664165530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07400d2830 msgr2=0x7f074014b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.293+0000 7f0744b15640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07400d2830 0x7f074014b780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0734000f80 tx=0x7f0734004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.293+0000 7f0744b15640 1 -- 172.21.15.172:0/664165530 shutdown_connections 2024-04-14T08:33:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.293+0000 7f0744b15640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f072006c8f0 0x7f072006edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.293+0000 7f0744b15640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f074015e890 0x7f07400b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.293+0000 7f0744b15640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07400d2830 0x7f074014b780 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.293+0000 7f0744b15640 1 --2- 172.21.15.172:0/664165530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07400cedd0 0x7f0740160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.293+0000 7f0744b15640 1 -- 172.21.15.172:0/664165530 >> 172.21.15.172:0/664165530 conn(0x7f074009d590 msgr2=0x7f074015c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.293+0000 7f0744b15640 1 -- 172.21.15.172:0/664165530 shutdown_connections 2024-04-14T08:33:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.293+0000 7f0744b15640 1 -- 172.21.15.172:0/664165530 wait complete. 2024-04-14T08:33:48.305 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:48.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.406+0000 7f88569c8640 1 Processor -- start 2024-04-14T08:33:48.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.406+0000 7f88569c8640 1 -- start start 2024-04-14T08:33:48.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f88569c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f88500b4040 0x7f88500b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f88569c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f885015c770 con 0x7f88500b1df0 2024-04-14T08:33:48.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f88569c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88500b3700 con 0x7f88500b21d0 2024-04-14T08:33:48.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f88569c8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f88500b39e0 con 0x7f88500b4040 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f8854f65640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f88500b4040 0x7f88500b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f884ffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f88500b1df0 0x7f88500b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54202/0 (socket says 172.21.15.172:54202) 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f884ffff640 1 -- 172.21.15.172:0/3065722252 learned_addr learned my addr 172.21.15.172:0/3065722252 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f884effd640 1 -- 172.21.15.172:0/3065722252 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 919945176 0 0) 0x7f885015c770 con 0x7f88500b1df0 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f884effd640 1 -- 172.21.15.172:0/3065722252 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8838003520 con 0x7f88500b1df0 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f884effd640 1 -- 172.21.15.172:0/3065722252 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197341076 0 0) 0x7f88500b3700 con 0x7f88500b21d0 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f8854f65640 1 -- 172.21.15.172:0/3065722252 >> v1:172.21.15.177:6789/0 conn(0x7f88500b21d0 legacy=0x7f885015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f8854f65640 1 -- 172.21.15.172:0/3065722252 >> v1:172.21.15.172:6789/0 conn(0x7f88500b1df0 legacy=0x7f88500b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.407+0000 7f8854f65640 1 -- 172.21.15.172:0/3065722252 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88500b3cc0 con 0x7f88500b4040 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.408+0000 7f8854f65640 1 --2- 172.21.15.172:0/3065722252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88500b4040 0x7f88500b4460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8848009d90 tx=0x7f884802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c9e48d1e528975 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.408+0000 7f884effd640 1 -- 172.21.15.172:0/3065722252 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848039070 con 0x7f88500b4040 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.408+0000 7f884effd640 1 -- 172.21.15.172:0/3065722252 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f884802cd00 con 0x7f88500b4040 2024-04-14T08:33:48.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.408+0000 7f884effd640 1 -- 172.21.15.172:0/3065722252 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848034640 con 0x7f88500b4040 2024-04-14T08:33:48.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.409+0000 7f88569c8640 1 -- 172.21.15.172:0/3065722252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88500b4040 msgr2=0x7f88500b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.409+0000 7f88569c8640 1 --2- 172.21.15.172:0/3065722252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88500b4040 0x7f88500b4460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8848009d90 tx=0x7f884802bf50 comp rx=0 tx=0).stop 2024-04-14T08:33:48.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.409+0000 7f88569c8640 1 -- 172.21.15.172:0/3065722252 shutdown_connections 2024-04-14T08:33:48.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.409+0000 7f88569c8640 1 --2- 172.21.15.172:0/3065722252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88500b4040 0x7f88500b4460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.409+0000 7f88569c8640 1 -- 172.21.15.172:0/3065722252 >> 172.21.15.172:0/3065722252 conn(0x7f885006ec30 msgr2=0x7f885006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:48.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.409+0000 7f88569c8640 1 -- 172.21.15.172:0/3065722252 shutdown_connections 2024-04-14T08:33:48.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.409+0000 7f88569c8640 1 -- 172.21.15.172:0/3065722252 wait complete. 2024-04-14T08:33:48.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.410+0000 7f88569c8640 1 Processor -- start 2024-04-14T08:33:48.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.410+0000 7f88569c8640 1 -- start start 2024-04-14T08:33:48.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f88569c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88500b1df0 0x7f88501efd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f88569c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88500b21d0 0x7f88500b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f88569c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88500b4040 0x7f88500b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f88569c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f885015b7f0 con 0x7f88500b4040 2024-04-14T08:33:48.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f88569c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f88500b49a0 con 0x7f88500b1df0 2024-04-14T08:33:48.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f88569c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f88501552d0 con 0x7f88500b21d0 2024-04-14T08:33:48.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f8854f65640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88500b4040 0x7f88500b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f8854f65640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88500b4040 0x7f88500b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59992/0 (socket says 172.21.15.172:59992) 2024-04-14T08:33:48.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f8854f65640 1 -- 172.21.15.172:0/3589055725 learned_addr learned my addr 172.21.15.172:0/3589055725 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:48.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f884ffff640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88500b1df0 0x7f88501efd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f884f7fe640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88500b21d0 0x7f88500b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f8854f65640 1 -- 172.21.15.172:0/3589055725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88500b21d0 msgr2=0x7f88500b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f8854f65640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88500b21d0 0x7f88500b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f8854f65640 1 -- 172.21.15.172:0/3589055725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88500b1df0 msgr2=0x7f88501efd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f8854f65640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88500b1df0 0x7f88501efd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f8854f65640 1 -- 172.21.15.172:0/3589055725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8850155a00 con 0x7f88500b4040 2024-04-14T08:33:48.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f8854f65640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88500b4040 0x7f88500b10d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8848037040 tx=0x7f8848004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:48.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f884cff9640 1 -- 172.21.15.172:0/3589055725 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848039050 con 0x7f88500b4040 2024-04-14T08:33:48.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f884cff9640 1 -- 172.21.15.172:0/3589055725 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8848045070 con 0x7f88500b4040 2024-04-14T08:33:48.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.411+0000 7f884cff9640 1 -- 172.21.15.172:0/3589055725 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848009450 con 0x7f88500b4040 2024-04-14T08:33:48.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.412+0000 7f88569c8640 1 -- 172.21.15.172:0/3589055725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8850155ce0 con 0x7f88500b4040 2024-04-14T08:33:48.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.412+0000 7f88569c8640 1 -- 172.21.15.172:0/3589055725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88501562a0 con 0x7f88500b4040 2024-04-14T08:33:48.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.412+0000 7f88569c8640 1 -- 172.21.15.172:0/3589055725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8850068520 con 0x7f88500b4040 2024-04-14T08:33:48.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.413+0000 7f884cff9640 1 -- 172.21.15.172:0/3589055725 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f884803e830 con 0x7f88500b4040 2024-04-14T08:33:48.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.413+0000 7f884cff9640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f883c06c8f0 0x7f883c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.413+0000 7f884cff9640 1 -- 172.21.15.172:0/3589055725 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f88480ae860 con 0x7f88500b4040 2024-04-14T08:33:48.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.413+0000 7f884ffff640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f883c06c8f0 0x7f883c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.414+0000 7f884ffff640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f883c06c8f0 0x7f883c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8840005880 tx=0x7f8840009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:48.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.418+0000 7f884cff9640 1 -- 172.21.15.172:0/3589055725 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f884804f020 con 0x7f88500b4040 2024-04-14T08:33:48.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.593+0000 7f88569c8640 1 -- 172.21.15.172:0/3589055725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8850072410 con 0x7f88500b4040 2024-04-14T08:33:48.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.594+0000 7f884cff9640 1 -- 172.21.15.172:0/3589055725 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18962 (secure 0 0 0) 0x7f884807c2d0 con 0x7f88500b4040 2024-04-14T08:33:48.595 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:48.595 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":27,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:33:34.330426+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:32:51.214999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-14T08:32:52.493598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-14T08:32:56.216795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:48.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.596+0000 7f88569c8640 1 -- 172.21.15.172:0/3589055725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f883c06c8f0 msgr2=0x7f883c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.596+0000 7f88569c8640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f883c06c8f0 0x7f883c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8840005880 tx=0x7f8840009040 comp rx=0 tx=0).stop 2024-04-14T08:33:48.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.596+0000 7f88569c8640 1 -- 172.21.15.172:0/3589055725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88500b4040 msgr2=0x7f88500b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.596+0000 7f88569c8640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88500b4040 0x7f88500b10d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8848037040 tx=0x7f8848004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:48.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.596+0000 7f88569c8640 1 -- 172.21.15.172:0/3589055725 shutdown_connections 2024-04-14T08:33:48.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.596+0000 7f88569c8640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f883c06c8f0 0x7f883c06edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.596+0000 7f88569c8640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88500b4040 0x7f88500b10d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.596+0000 7f88569c8640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88500b21d0 0x7f88500b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.596+0000 7f88569c8640 1 --2- 172.21.15.172:0/3589055725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88500b1df0 0x7f88501efd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.596+0000 7f88569c8640 1 -- 172.21.15.172:0/3589055725 >> 172.21.15.172:0/3589055725 conn(0x7f885006ec30 msgr2=0x7f88500b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:48.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.596+0000 7f88569c8640 1 -- 172.21.15.172:0/3589055725 shutdown_connections 2024-04-14T08:33:48.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.596+0000 7f88569c8640 1 -- 172.21.15.172:0/3589055725 wait complete. 2024-04-14T08:33:48.608 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:33:48.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.709+0000 7feba2b7b640 1 Processor -- start 2024-04-14T08:33:48.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.709+0000 7feba2b7b640 1 -- start start 2024-04-14T08:33:48.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feba2b7b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb9c13df30 0x7feb9c0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feba2b7b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9c15c380 0x7feb9c0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feba2b7b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb9c15b620 con 0x7feb9c140770 2024-04-14T08:33:48.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feba2b7b640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7feb9c0cf9b0 con 0x7feb9c13df30 2024-04-14T08:33:48.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feba2b7b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7feb9c1414a0 con 0x7feb9c15c380 2024-04-14T08:33:48.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feba0917640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb9c13df30 0x7feb9c0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feba0917640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb9c13df30 0x7feb9c0b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60004/0 (socket says 172.21.15.172:60004) 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feba0917640 1 -- 172.21.15.172:0/1077349892 learned_addr learned my addr 172.21.15.172:0/1077349892 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feb9bfff640 1 --2- 172.21.15.172:0/1077349892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9c15c380 0x7feb9c0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feb9b7fe640 1 -- 172.21.15.172:0/1077349892 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2869436521 0 0) 0x7feb94001490 con 0x7feb9c140770 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feba0917640 1 -- 172.21.15.172:0/1077349892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9c15c380 msgr2=0x7feb9c0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feba0917640 1 --2- 172.21.15.172:0/1077349892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9c15c380 0x7feb9c0b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feba0917640 1 -- 172.21.15.172:0/1077349892 >> v1:172.21.15.177:6789/0 conn(0x7feb9c140770 legacy=0x7feb9c140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.710+0000 7feba0917640 1 -- 172.21.15.172:0/1077349892 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c15b8a0 con 0x7feb9c13df30 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.711+0000 7feba0917640 1 --2- 172.21.15.172:0/1077349892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9c13df30 0x7feb9c0b1cd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb9000bb40 tx=0x7feb9002e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a081178c7476b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.711+0000 7feb9b7fe640 1 -- 172.21.15.172:0/1077349892 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9003b070 con 0x7feb9c13df30 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.711+0000 7feb9b7fe640 1 -- 172.21.15.172:0/1077349892 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb90036440 con 0x7feb9c13df30 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.711+0000 7feb9b7fe640 1 -- 172.21.15.172:0/1077349892 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90036810 con 0x7feb9c13df30 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.712+0000 7feba2b7b640 1 -- 172.21.15.172:0/1077349892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9c13df30 msgr2=0x7feb9c0b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.712+0000 7feba2b7b640 1 --2- 172.21.15.172:0/1077349892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9c13df30 0x7feb9c0b1cd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb9000bb40 tx=0x7feb9002e150 comp rx=0 tx=0).stop 2024-04-14T08:33:48.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.712+0000 7feba2b7b640 1 -- 172.21.15.172:0/1077349892 shutdown_connections 2024-04-14T08:33:48.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.712+0000 7feba2b7b640 1 --2- 172.21.15.172:0/1077349892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9c15c380 0x7feb9c0b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.712+0000 7feba2b7b640 1 --2- 172.21.15.172:0/1077349892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9c13df30 0x7feb9c0b1cd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.712+0000 7feba2b7b640 1 -- 172.21.15.172:0/1077349892 >> 172.21.15.172:0/1077349892 conn(0x7feb9c06b520 msgr2=0x7feb9c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:48.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.712+0000 7feba2b7b640 1 -- 172.21.15.172:0/1077349892 shutdown_connections 2024-04-14T08:33:48.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.712+0000 7feba2b7b640 1 -- 172.21.15.172:0/1077349892 wait complete. 2024-04-14T08:33:48.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.713+0000 7feba2b7b640 1 Processor -- start 2024-04-14T08:33:48.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.713+0000 7feba2b7b640 1 -- start start 2024-04-14T08:33:48.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba2b7b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9c13df30 0x7feb9c130560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba2b7b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb9c140770 0x7feb9c0af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba2b7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9c15c380 0x7feb9c13cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba2b7b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c13e310 con 0x7feb9c15c380 2024-04-14T08:33:48.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba2b7b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c15be60 con 0x7feb9c140770 2024-04-14T08:33:48.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba2b7b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7feb9c0ce9a0 con 0x7feb9c13df30 2024-04-14T08:33:48.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba1118640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9c15c380 0x7feb9c13cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba1118640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9c15c380 0x7feb9c13cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60012/0 (socket says 172.21.15.172:60012) 2024-04-14T08:33:48.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba1118640 1 -- 172.21.15.172:0/1611114406 learned_addr learned my addr 172.21.15.172:0/1611114406 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:48.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feb9bfff640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb9c140770 0x7feb9c0af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba0917640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9c13df30 0x7feb9c130560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba1118640 1 -- 172.21.15.172:0/1611114406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9c13df30 msgr2=0x7feb9c130560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba1118640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9c13df30 0x7feb9c130560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba1118640 1 -- 172.21.15.172:0/1611114406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb9c140770 msgr2=0x7feb9c0af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba1118640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb9c140770 0x7feb9c0af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba1118640 1 -- 172.21.15.172:0/1611114406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c0b21a0 con 0x7feb9c15c380 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba1118640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9c15c380 0x7feb9c13cb40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feb9400c7c0 tx=0x7feb9400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feb997fa640 1 -- 172.21.15.172:0/1611114406 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94003da0 con 0x7feb9c15c380 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feb997fa640 1 -- 172.21.15.172:0/1611114406 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb94005070 con 0x7feb9c15c380 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feba2b7b640 1 -- 172.21.15.172:0/1611114406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9c0b2480 con 0x7feb9c15c380 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.714+0000 7feb997fa640 1 -- 172.21.15.172:0/1611114406 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94005460 con 0x7feb9c15c380 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.715+0000 7feba2b7b640 1 -- 172.21.15.172:0/1611114406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9c0b1e50 con 0x7feb9c15c380 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.716+0000 7feba2b7b640 1 -- 172.21.15.172:0/1611114406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9c156190 con 0x7feb9c15c380 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.716+0000 7feb997fa640 1 -- 172.21.15.172:0/1611114406 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb94005600 con 0x7feb9c15c380 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.716+0000 7feb997fa640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb8006c920 0x7feb8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:48.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.716+0000 7feb997fa640 1 -- 172.21.15.172:0/1611114406 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7feb9408bc60 con 0x7feb9c15c380 2024-04-14T08:33:48.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.716+0000 7feba0917640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb8006c920 0x7feb8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:48.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.717+0000 7feba0917640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb8006c920 0x7feb8006ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb90038000 tx=0x7feb90032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:48.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.721+0000 7feb997fa640 1 -- 172.21.15.172:0/1611114406 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7feb9c15b620 con 0x7feb9c15c380 2024-04-14T08:33:48.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.932+0000 7feba2b7b640 1 -- 172.21.15.172:0/1611114406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7feb9c068590 con 0x7feb9c15c380 2024-04-14T08:33:48.933 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:33:48.933+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:33:48.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.934+0000 7feb997fa640 1 -- 172.21.15.172:0/1611114406 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7feb9408b090 con 0x7feb9c15c380 2024-04-14T08:33:48.934 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:33:48.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.936+0000 7feba2b7b640 1 -- 172.21.15.172:0/1611114406 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb8006c920 msgr2=0x7feb8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.936+0000 7feba2b7b640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb8006c920 0x7feb8006ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb90038000 tx=0x7feb90032000 comp rx=0 tx=0).stop 2024-04-14T08:33:48.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.936+0000 7feba2b7b640 1 -- 172.21.15.172:0/1611114406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9c15c380 msgr2=0x7feb9c13cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:48.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.936+0000 7feba2b7b640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9c15c380 0x7feb9c13cb40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feb9400c7c0 tx=0x7feb9400cc80 comp rx=0 tx=0).stop 2024-04-14T08:33:48.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.937+0000 7feba2b7b640 1 -- 172.21.15.172:0/1611114406 shutdown_connections 2024-04-14T08:33:48.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.937+0000 7feba2b7b640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb8006c920 0x7feb8006ede0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.937+0000 7feba2b7b640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9c15c380 0x7feb9c13cb40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.937+0000 7feba2b7b640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb9c140770 0x7feb9c0af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.937+0000 7feba2b7b640 1 --2- 172.21.15.172:0/1611114406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9c13df30 0x7feb9c130560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:48.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.937+0000 7feba2b7b640 1 -- 172.21.15.172:0/1611114406 >> 172.21.15.172:0/1611114406 conn(0x7feb9c06b520 msgr2=0x7feb9c15eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:48.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.937+0000 7feba2b7b640 1 -- 172.21.15.172:0/1611114406 shutdown_connections 2024-04-14T08:33:48.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:48.937+0000 7feba2b7b640 1 -- 172.21.15.172:0/1611114406 wait complete. 2024-04-14T08:33:48.942 INFO:tasks.ceph.mds.d.smithi177.stderr: -5238> 2024-04-14T08:32:57.786+0000 7f369c3f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:33:48.948 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-14T08:33:48.975 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-04-14T08:33:49.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.051+0000 7f20375cf640 1 Processor -- start 2024-04-14T08:33:49.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.051+0000 7f20375cf640 1 -- start start 2024-04-14T08:33:49.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.052+0000 7f20375cf640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f203015e820 0x7f20300b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:49.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.052+0000 7f20375cf640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f20300d0d30 0x7f20300cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:49.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.052+0000 7f20375cf640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20300d12e0 con 0x7f203006d040 2024-04-14T08:33:49.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.052+0000 7f20375cf640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f203006cb90 con 0x7f20300d0d30 2024-04-14T08:33:49.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.052+0000 7f20375cf640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f20300cc660 con 0x7f203015e820 2024-04-14T08:33:49.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.052+0000 7f2035b6c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f20300d0d30 0x7f20300cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:49.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.052+0000 7f2035b6c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f20300d0d30 0x7f20300cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60022/0 (socket says 172.21.15.172:60022) 2024-04-14T08:33:49.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.052+0000 7f2035b6c640 1 -- 172.21.15.172:0/985996575 learned_addr learned my addr 172.21.15.172:0/985996575 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:49.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.052+0000 7f2034b6a640 1 --2- 172.21.15.172:0/985996575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f203015e820 0x7f20300b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:49.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.053+0000 7f2035b6c640 1 -- 172.21.15.172:0/985996575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f203015e820 msgr2=0x7f20300b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:49.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.053+0000 7f2035b6c640 1 --2- 172.21.15.172:0/985996575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f203015e820 0x7f20300b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:49.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.053+0000 7f2035b6c640 1 -- 172.21.15.172:0/985996575 >> v1:172.21.15.177:6789/0 conn(0x7f203006d040 legacy=0x7f203006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:49.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.053+0000 7f2035b6c640 1 -- 172.21.15.172:0/985996575 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20300d1580 con 0x7f20300d0d30 2024-04-14T08:33:49.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.053+0000 7f2035b6c640 1 --2- 172.21.15.172:0/985996575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20300d0d30 0x7f20300cd0b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2028009e50 tx=0x7f202802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1942be6274e25d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.053+0000 7f2027fff640 1 -- 172.21.15.172:0/985996575 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202802cb00 con 0x7f20300d0d30 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.053+0000 7f2027fff640 1 -- 172.21.15.172:0/985996575 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202802cca0 con 0x7f20300d0d30 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.054+0000 7f2027fff640 1 -- 172.21.15.172:0/985996575 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028034610 con 0x7f20300d0d30 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.054+0000 7f20375cf640 1 -- 172.21.15.172:0/985996575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20300d0d30 msgr2=0x7f20300cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.054+0000 7f20375cf640 1 --2- 172.21.15.172:0/985996575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20300d0d30 0x7f20300cd0b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2028009e50 tx=0x7f202802bf60 comp rx=0 tx=0).stop 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.054+0000 7f20375cf640 1 -- 172.21.15.172:0/985996575 shutdown_connections 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.054+0000 7f20375cf640 1 --2- 172.21.15.172:0/985996575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20300d0d30 0x7f20300cd0b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.054+0000 7f20375cf640 1 --2- 172.21.15.172:0/985996575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f203015e820 0x7f20300b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.054+0000 7f20375cf640 1 -- 172.21.15.172:0/985996575 >> 172.21.15.172:0/985996575 conn(0x7f20300b3b00 msgr2=0x7f20300b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.054+0000 7f20375cf640 1 -- 172.21.15.172:0/985996575 shutdown_connections 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.054+0000 7f20375cf640 1 -- 172.21.15.172:0/985996575 wait complete. 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.055+0000 7f20375cf640 1 Processor -- start 2024-04-14T08:33:49.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.055+0000 7f20375cf640 1 -- start start 2024-04-14T08:33:49.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f20375cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203006d040 0x7f20300b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f20375cf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20300d0d30 0x7f20301f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f20375cf640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f203015e820 0x7f20300b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f20375cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f20300cd5f0 con 0x7f203006d040 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f20375cf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f20300ccc20 con 0x7f20300d0d30 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f20375cf640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f20300cd770 con 0x7f203015e820 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f203536b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203006d040 0x7f20300b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f203536b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203006d040 0x7f20300b76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60032/0 (socket says 172.21.15.172:60032) 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f203536b640 1 -- 172.21.15.172:0/482890910 learned_addr learned my addr 172.21.15.172:0/482890910 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f2034b6a640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20300d0d30 0x7f20301f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f2035b6c640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f203015e820 0x7f20300b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f203536b640 1 -- 172.21.15.172:0/482890910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f203015e820 msgr2=0x7f20300b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f203536b640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f203015e820 0x7f20300b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f203536b640 1 -- 172.21.15.172:0/482890910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20300d0d30 msgr2=0x7f20301f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f203536b640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20300d0d30 0x7f20301f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.056+0000 7f203536b640 1 -- 172.21.15.172:0/482890910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20300b1d50 con 0x7f203006d040 2024-04-14T08:33:49.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.057+0000 7f203536b640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203006d040 0x7f20300b76d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f202c00c430 tx=0x7f202c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:49.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.057+0000 7f2025ffb640 1 -- 172.21.15.172:0/482890910 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c0052d0 con 0x7f203006d040 2024-04-14T08:33:49.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.057+0000 7f2025ffb640 1 -- 172.21.15.172:0/482890910 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202c005490 con 0x7f203006d040 2024-04-14T08:33:49.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.057+0000 7f20375cf640 1 -- 172.21.15.172:0/482890910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20300b7d70 con 0x7f203006d040 2024-04-14T08:33:49.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.057+0000 7f2025ffb640 1 -- 172.21.15.172:0/482890910 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c007260 con 0x7f203006d040 2024-04-14T08:33:49.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.057+0000 7f20375cf640 1 -- 172.21.15.172:0/482890910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20300b22d0 con 0x7f203006d040 2024-04-14T08:33:49.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.058+0000 7f20375cf640 1 -- 172.21.15.172:0/482890910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2030068590 con 0x7f203006d040 2024-04-14T08:33:49.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.058+0000 7f2025ffb640 1 -- 172.21.15.172:0/482890910 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f202c007400 con 0x7f203006d040 2024-04-14T08:33:49.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.059+0000 7f2025ffb640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f201406c8f0 0x7f201406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:49.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.059+0000 7f2034b6a640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f201406c8f0 0x7f201406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:49.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.059+0000 7f2025ffb640 1 -- 172.21.15.172:0/482890910 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f202c08b460 con 0x7f203006d040 2024-04-14T08:33:49.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.059+0000 7f2034b6a640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f201406c8f0 0x7f201406edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20200079a0 tx=0x7f2020008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:49.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.064+0000 7f2025ffb640 1 -- 172.21.15.172:0/482890910 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f202c058de0 con 0x7f203006d040 2024-04-14T08:33:49.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.278+0000 7f20375cf640 1 -- 172.21.15.172:0/482890910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f20300cec00 con 0x7f203006d040 2024-04-14T08:33:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.942+0000 7f2025ffb640 1 -- 172.21.15.172:0/482890910 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f202c05d0e0 con 0x7f203006d040 2024-04-14T08:33:49.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.945+0000 7f20375cf640 1 -- 172.21.15.172:0/482890910 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f201406c8f0 msgr2=0x7f201406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.945+0000 7f20375cf640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f201406c8f0 0x7f201406edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20200079a0 tx=0x7f2020008040 comp rx=0 tx=0).stop 2024-04-14T08:33:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.945+0000 7f20375cf640 1 -- 172.21.15.172:0/482890910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203006d040 msgr2=0x7f20300b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.945+0000 7f20375cf640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203006d040 0x7f20300b76d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f202c00c430 tx=0x7f202c00c8f0 comp rx=0 tx=0).stop 2024-04-14T08:33:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.945+0000 7f20375cf640 1 -- 172.21.15.172:0/482890910 shutdown_connections 2024-04-14T08:33:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.945+0000 7f20375cf640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f201406c8f0 0x7f201406edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.945+0000 7f20375cf640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f203015e820 0x7f20300b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.945+0000 7f20375cf640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20300d0d30 0x7f20301f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.945+0000 7f20375cf640 1 --2- 172.21.15.172:0/482890910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f203006d040 0x7f20300b76d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.945+0000 7f20375cf640 1 -- 172.21.15.172:0/482890910 >> 172.21.15.172:0/482890910 conn(0x7f20300b3b00 msgr2=0x7f203006f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.945+0000 7f20375cf640 1 -- 172.21.15.172:0/482890910 shutdown_connections 2024-04-14T08:33:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:49.945+0000 7f20375cf640 1 -- 172.21.15.172:0/482890910 wait complete. 2024-04-14T08:33:49.958 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-14T08:33:50.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.064+0000 7f6a4851e640 1 Processor -- start 2024-04-14T08:33:50.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.064+0000 7f6a4851e640 1 -- start start 2024-04-14T08:33:50.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a4851e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6a4015b350 0x7f6a400cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:50.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a4851e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6a400cc170 0x7f6a400ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:50.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a4851e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a400cdb90 con 0x7f6a400cecc0 2024-04-14T08:33:50.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a4851e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f6a400cf130 con 0x7f6a4015b350 2024-04-14T08:33:50.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a4851e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f6a400cd940 con 0x7f6a400cc170 2024-04-14T08:33:50.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a462ba640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6a4015b350 0x7f6a400cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:50.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a46abb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6a400cecc0 0x7f6a4015ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34960/0 (socket says 172.21.15.172:34960) 2024-04-14T08:33:50.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a46abb640 1 -- 172.21.15.172:0/3495244144 learned_addr learned my addr 172.21.15.172:0/3495244144 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:50.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a462ba640 1 -- 172.21.15.172:0/3495244144 >> v2:172.21.15.177:3300/0 conn(0x7f6a400cc170 msgr2=0x7f6a400ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:50.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a462ba640 1 --2- 172.21.15.172:0/3495244144 >> v2:172.21.15.177:3300/0 conn(0x7f6a400cc170 0x7f6a400ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:50.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a462ba640 1 -- 172.21.15.172:0/3495244144 >> v1:172.21.15.172:6789/0 conn(0x7f6a400cecc0 legacy=0x7f6a4015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:50.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a462ba640 1 -- 172.21.15.172:0/3495244144 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a400cde70 con 0x7f6a4015b350 2024-04-14T08:33:50.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.065+0000 7f6a452b8640 1 -- 172.21.15.172:0/3495244144 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3913425088 0 0) 0x7f6a400cdb90 con 0x7f6a400cecc0 2024-04-14T08:33:50.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.066+0000 7f6a462ba640 1 --2- 172.21.15.172:0/3495244144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4015b350 0x7f6a400cbc30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6a400b25f0 tx=0x7f6a3c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2eff653046c71ba server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:50.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.066+0000 7f6a452b8640 1 -- 172.21.15.172:0/3495244144 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0315a0 con 0x7f6a4015b350 2024-04-14T08:33:50.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.066+0000 7f6a452b8640 1 -- 172.21.15.172:0/3495244144 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c037070 con 0x7f6a4015b350 2024-04-14T08:33:50.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.066+0000 7f6a452b8640 1 -- 172.21.15.172:0/3495244144 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c031960 con 0x7f6a4015b350 2024-04-14T08:33:50.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.067+0000 7f6a4851e640 1 -- 172.21.15.172:0/3495244144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4015b350 msgr2=0x7f6a400cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:50.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.067+0000 7f6a4851e640 1 --2- 172.21.15.172:0/3495244144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4015b350 0x7f6a400cbc30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6a400b25f0 tx=0x7f6a3c02b840 comp rx=0 tx=0).stop 2024-04-14T08:33:50.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.067+0000 7f6a4851e640 1 -- 172.21.15.172:0/3495244144 shutdown_connections 2024-04-14T08:33:50.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.067+0000 7f6a4851e640 1 --2- 172.21.15.172:0/3495244144 >> v2:172.21.15.177:3300/0 conn(0x7f6a400cc170 0x7f6a400ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:50.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.067+0000 7f6a4851e640 1 --2- 172.21.15.172:0/3495244144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4015b350 0x7f6a400cbc30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:50.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.067+0000 7f6a4851e640 1 -- 172.21.15.172:0/3495244144 >> 172.21.15.172:0/3495244144 conn(0x7f6a400b1a50 msgr2=0x7f6a400b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:50.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.067+0000 7f6a4851e640 1 -- 172.21.15.172:0/3495244144 shutdown_connections 2024-04-14T08:33:50.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.067+0000 7f6a4851e640 1 -- 172.21.15.172:0/3495244144 wait complete. 2024-04-14T08:33:50.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.068+0000 7f6a4851e640 1 Processor -- start 2024-04-14T08:33:50.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.068+0000 7f6a4851e640 1 -- start start 2024-04-14T08:33:50.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a4851e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a400cc170 0x7f6a400b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:50.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a4851e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a400cecc0 0x7f6a4009d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:50.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a4851e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4015b350 0x7f6a4009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a4851e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6a400cf560 con 0x7f6a4015b350 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a4851e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a400ce4c0 con 0x7f6a400cc170 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a4851e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6a400c5e80 con 0x7f6a400cecc0 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a46abb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4015b350 0x7f6a4009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a46abb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4015b350 0x7f6a4009aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57184/0 (socket says 172.21.15.172:57184) 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a46abb640 1 -- 172.21.15.172:0/1047409302 learned_addr learned my addr 172.21.15.172:0/1047409302 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a462ba640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a400cc170 0x7f6a400b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a45ab9640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a400cecc0 0x7f6a4009d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a46abb640 1 -- 172.21.15.172:0/1047409302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a400cecc0 msgr2=0x7f6a4009d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a46abb640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a400cecc0 0x7f6a4009d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a46abb640 1 -- 172.21.15.172:0/1047409302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a400cc170 msgr2=0x7f6a400b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a46abb640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a400cc170 0x7f6a400b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a46abb640 1 -- 172.21.15.172:0/1047409302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a400c8da0 con 0x7f6a4015b350 2024-04-14T08:33:50.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.069+0000 7f6a46abb640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4015b350 0x7f6a4009aa70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c7a0 tx=0x7f6a3800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:50.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.070+0000 7f6a36ffd640 1 -- 172.21.15.172:0/1047409302 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38018070 con 0x7f6a4015b350 2024-04-14T08:33:50.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.070+0000 7f6a36ffd640 1 -- 172.21.15.172:0/1047409302 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a38004d60 con 0x7f6a4015b350 2024-04-14T08:33:50.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.070+0000 7f6a4851e640 1 -- 172.21.15.172:0/1047409302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a400c9080 con 0x7f6a4015b350 2024-04-14T08:33:50.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.070+0000 7f6a36ffd640 1 -- 172.21.15.172:0/1047409302 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800a880 con 0x7f6a4015b350 2024-04-14T08:33:50.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.070+0000 7f6a4851e640 1 -- 172.21.15.172:0/1047409302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a400c9640 con 0x7f6a4015b350 2024-04-14T08:33:50.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.071+0000 7f6a4851e640 1 -- 172.21.15.172:0/1047409302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a400686a0 con 0x7f6a4015b350 2024-04-14T08:33:50.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.071+0000 7f6a36ffd640 1 -- 172.21.15.172:0/1047409302 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a3801b070 con 0x7f6a4015b350 2024-04-14T08:33:50.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.071+0000 7f6a36ffd640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6a2406c8f0 0x7f6a2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:50.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.072+0000 7f6a36ffd640 1 -- 172.21.15.172:0/1047409302 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f6a3808b3f0 con 0x7f6a4015b350 2024-04-14T08:33:50.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.072+0000 7f6a462ba640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6a2406c8f0 0x7f6a2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:50.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.072+0000 7f6a462ba640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6a2406c8f0 0x7f6a2406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c036000 tx=0x7f6a3c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:50.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.076+0000 7f6a36ffd640 1 -- 172.21.15.172:0/1047409302 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6a38058e00 con 0x7f6a4015b350 2024-04-14T08:33:50.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.256+0000 7f6a4851e640 1 -- 172.21.15.172:0/1047409302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6a4006e990 con 0x7f6a4015b350 2024-04-14T08:33:50.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.943+0000 7f6a36ffd640 1 -- 172.21.15.172:0/1047409302 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f6a3805d100 con 0x7f6a4015b350 2024-04-14T08:33:50.944 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' removed 2024-04-14T08:33:50.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.946+0000 7f6a4851e640 1 -- 172.21.15.172:0/1047409302 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6a2406c8f0 msgr2=0x7f6a2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:50.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.946+0000 7f6a4851e640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6a2406c8f0 0x7f6a2406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c036000 tx=0x7f6a3c03d040 comp rx=0 tx=0).stop 2024-04-14T08:33:50.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.946+0000 7f6a4851e640 1 -- 172.21.15.172:0/1047409302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4015b350 msgr2=0x7f6a4009aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:50.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.946+0000 7f6a4851e640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4015b350 0x7f6a4009aa70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c7a0 tx=0x7f6a3800cba0 comp rx=0 tx=0).stop 2024-04-14T08:33:50.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.946+0000 7f6a4851e640 1 -- 172.21.15.172:0/1047409302 shutdown_connections 2024-04-14T08:33:50.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.946+0000 7f6a4851e640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6a2406c8f0 0x7f6a2406edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:50.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.946+0000 7f6a4851e640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4015b350 0x7f6a4009aa70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:50.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.946+0000 7f6a4851e640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a400cecc0 0x7f6a4009d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:50.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.946+0000 7f6a4851e640 1 --2- 172.21.15.172:0/1047409302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a400cc170 0x7f6a400b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:50.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.946+0000 7f6a4851e640 1 -- 172.21.15.172:0/1047409302 >> 172.21.15.172:0/1047409302 conn(0x7f6a400b1a50 msgr2=0x7f6a400cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:50.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.946+0000 7f6a4851e640 1 -- 172.21.15.172:0/1047409302 shutdown_connections 2024-04-14T08:33:50.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:50.946+0000 7f6a4851e640 1 -- 172.21.15.172:0/1047409302 wait complete. 2024-04-14T08:33:50.960 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-14T08:33:51.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.067+0000 7f3e2ec76640 1 Processor -- start 2024-04-14T08:33:51.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.067+0000 7f3e2ec76640 1 -- start start 2024-04-14T08:33:51.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.067+0000 7f3e2ec76640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3e28161d00 0x7f3e280b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:51.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.067+0000 7f3e2ec76640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3e280b5490 0x7f3e280d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:51.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.067+0000 7f3e2ec76640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e280d2b20 0x7f3e281622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:51.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.067+0000 7f3e2ec76640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f3e280d0ba0 con 0x7f3e280b5490 2024-04-14T08:33:51.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.067+0000 7f3e2ec76640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f3e280d1720 con 0x7f3e28161d00 2024-04-14T08:33:51.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.067+0000 7f3e2ec76640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3e280d33b0 con 0x7f3e280d2b20 2024-04-14T08:33:51.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.068+0000 7f3e27fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3e280b5490 0x7f3e280d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:51.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.068+0000 7f3e27fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3e280b5490 0x7f3e280d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57196/0 (socket says 172.21.15.172:57196) 2024-04-14T08:33:51.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.068+0000 7f3e27fff640 1 -- 172.21.15.172:0/2319053381 learned_addr learned my addr 172.21.15.172:0/2319053381 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:51.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.068+0000 7f3e2d213640 1 --2- 172.21.15.172:0/2319053381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e280d2b20 0x7f3e281622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:51.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.068+0000 7f3e2ca12640 1 --2- 172.21.15.172:0/2319053381 >> v2:172.21.15.177:3300/0 conn(0x7f3e28161d00 0x7f3e280b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:51.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.068+0000 7f3e27fff640 1 -- 172.21.15.172:0/2319053381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e280d2b20 msgr2=0x7f3e281622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:51.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.068+0000 7f3e27fff640 1 --2- 172.21.15.172:0/2319053381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e280d2b20 0x7f3e281622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.068+0000 7f3e27fff640 1 -- 172.21.15.172:0/2319053381 >> v2:172.21.15.177:3300/0 conn(0x7f3e28161d00 msgr2=0x7f3e280b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:51.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.068+0000 7f3e27fff640 1 --2- 172.21.15.172:0/2319053381 >> v2:172.21.15.177:3300/0 conn(0x7f3e28161d00 0x7f3e280b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.068+0000 7f3e27fff640 1 -- 172.21.15.172:0/2319053381 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e280d1bb0 con 0x7f3e280b5490 2024-04-14T08:33:51.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e27fff640 1 --2- 172.21.15.172:0/2319053381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e280b5490 0x7f3e280d25e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3e2808c640 tx=0x7f3e1c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b484918004c9555b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:51.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e277fe640 1 -- 172.21.15.172:0/2319053381 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c02e070 con 0x7f3e280b5490 2024-04-14T08:33:51.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e277fe640 1 -- 172.21.15.172:0/2319053381 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e1c037070 con 0x7f3e280b5490 2024-04-14T08:33:51.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e277fe640 1 -- 172.21.15.172:0/2319053381 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c031a00 con 0x7f3e280b5490 2024-04-14T08:33:51.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e2ec76640 1 -- 172.21.15.172:0/2319053381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e280b5490 msgr2=0x7f3e280d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:51.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e2ec76640 1 --2- 172.21.15.172:0/2319053381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e280b5490 0x7f3e280d25e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3e2808c640 tx=0x7f3e1c02b880 comp rx=0 tx=0).stop 2024-04-14T08:33:51.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e2ec76640 1 -- 172.21.15.172:0/2319053381 shutdown_connections 2024-04-14T08:33:51.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e2ec76640 1 --2- 172.21.15.172:0/2319053381 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e280d2b20 0x7f3e281622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e2ec76640 1 --2- 172.21.15.172:0/2319053381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e280b5490 0x7f3e280d25e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e2ec76640 1 --2- 172.21.15.172:0/2319053381 >> v2:172.21.15.177:3300/0 conn(0x7f3e28161d00 0x7f3e280b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e2ec76640 1 -- 172.21.15.172:0/2319053381 >> 172.21.15.172:0/2319053381 conn(0x7f3e2809d570 msgr2=0x7f3e280d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:51.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e2ec76640 1 -- 172.21.15.172:0/2319053381 shutdown_connections 2024-04-14T08:33:51.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.069+0000 7f3e2ec76640 1 -- 172.21.15.172:0/2319053381 wait complete. 2024-04-14T08:33:51.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.071+0000 7f3e2ec76640 1 Processor -- start 2024-04-14T08:33:51.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.071+0000 7f3e2ec76640 1 -- start start 2024-04-14T08:33:51.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.071+0000 7f3e2ec76640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e280b5490 0x7f3e280b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:51.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.071+0000 7f3e2ec76640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e280d2b20 0x7f3e280b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:51.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.071+0000 7f3e2ec76640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e28161d00 0x7f3e2815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:51.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.071+0000 7f3e2ec76640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e280d10d0 con 0x7f3e28161d00 2024-04-14T08:33:51.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.071+0000 7f3e2ec76640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3e280d2f50 con 0x7f3e280d2b20 2024-04-14T08:33:51.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.071+0000 7f3e2ec76640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3e280d13b0 con 0x7f3e280b5490 2024-04-14T08:33:51.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e2d213640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e28161d00 0x7f3e2815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:51.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e2d213640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e28161d00 0x7f3e2815c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57204/0 (socket says 172.21.15.172:57204) 2024-04-14T08:33:51.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e2d213640 1 -- 172.21.15.172:0/3873859321 learned_addr learned my addr 172.21.15.172:0/3873859321 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:51.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e2ca12640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e280b5490 0x7f3e280b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:51.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e27fff640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e280d2b20 0x7f3e280b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:51.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e2d213640 1 -- 172.21.15.172:0/3873859321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e280b5490 msgr2=0x7f3e280b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:51.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e2d213640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e280b5490 0x7f3e280b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e2d213640 1 -- 172.21.15.172:0/3873859321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e280d2b20 msgr2=0x7f3e280b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:51.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e2d213640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e280d2b20 0x7f3e280b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e2d213640 1 -- 172.21.15.172:0/3873859321 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e28159a90 con 0x7f3e28161d00 2024-04-14T08:33:51.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e2d213640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e28161d00 0x7f3e2815c6f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800d830 tx=0x7f3e1800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:51.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e257fa640 1 -- 172.21.15.172:0/3873859321 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18018020 con 0x7f3e28161d00 2024-04-14T08:33:51.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e257fa640 1 -- 172.21.15.172:0/3873859321 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e180096c0 con 0x7f3e28161d00 2024-04-14T08:33:51.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e2ec76640 1 -- 172.21.15.172:0/3873859321 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e280cc570 con 0x7f3e28161d00 2024-04-14T08:33:51.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e257fa640 1 -- 172.21.15.172:0/3873859321 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18009a90 con 0x7f3e28161d00 2024-04-14T08:33:51.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.072+0000 7f3e2ec76640 1 -- 172.21.15.172:0/3873859321 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e280ccb30 con 0x7f3e28161d00 2024-04-14T08:33:51.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.073+0000 7f3e2ec76640 1 -- 172.21.15.172:0/3873859321 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e280686a0 con 0x7f3e28161d00 2024-04-14T08:33:51.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.074+0000 7f3e257fa640 1 -- 172.21.15.172:0/3873859321 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e1801b020 con 0x7f3e28161d00 2024-04-14T08:33:51.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.074+0000 7f3e257fa640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3e1406c8f0 0x7f3e1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:51.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.074+0000 7f3e257fa640 1 -- 172.21.15.172:0/3873859321 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6383+0+0 (secure 0 0 0) 0x7f3e1808c070 con 0x7f3e28161d00 2024-04-14T08:33:51.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.074+0000 7f3e2ca12640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3e1406c8f0 0x7f3e1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:51.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.074+0000 7f3e2ca12640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3e1406c8f0 0x7f3e1406edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e20003b80 tx=0x7f3e200098d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:51.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.078+0000 7f3e257fa640 1 -- 172.21.15.172:0/3873859321 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3e18022030 con 0x7f3e28161d00 2024-04-14T08:33:51.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.254+0000 7f3e2ec76640 1 -- 172.21.15.172:0/3873859321 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3e2808e320 con 0x7f3e28161d00 2024-04-14T08:33:51.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.947+0000 7f3e257fa640 1 -- 172.21.15.172:0/3873859321 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f3e18059bb0 con 0x7f3e28161d00 2024-04-14T08:33:51.949 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' removed 2024-04-14T08:33:51.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.951+0000 7f3e2ec76640 1 -- 172.21.15.172:0/3873859321 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3e1406c8f0 msgr2=0x7f3e1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:51.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.951+0000 7f3e2ec76640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3e1406c8f0 0x7f3e1406edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e20003b80 tx=0x7f3e200098d0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.951+0000 7f3e2ec76640 1 -- 172.21.15.172:0/3873859321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e28161d00 msgr2=0x7f3e2815c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:51.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.951+0000 7f3e2ec76640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e28161d00 0x7f3e2815c6f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800d830 tx=0x7f3e1800dcf0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.951+0000 7f3e2ec76640 1 -- 172.21.15.172:0/3873859321 shutdown_connections 2024-04-14T08:33:51.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.951+0000 7f3e2ec76640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3e1406c8f0 0x7f3e1406edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.951+0000 7f3e2ec76640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e28161d00 0x7f3e2815c6f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.951+0000 7f3e2ec76640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e280d2b20 0x7f3e280b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.951+0000 7f3e2ec76640 1 --2- 172.21.15.172:0/3873859321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3e280b5490 0x7f3e280b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:51.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.951+0000 7f3e2ec76640 1 -- 172.21.15.172:0/3873859321 >> 172.21.15.172:0/3873859321 conn(0x7f3e2809d570 msgr2=0x7f3e280b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:51.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.951+0000 7f3e2ec76640 1 -- 172.21.15.172:0/3873859321 shutdown_connections 2024-04-14T08:33:51.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:51.951+0000 7f3e2ec76640 1 -- 172.21.15.172:0/3873859321 wait complete. 2024-04-14T08:33:51.972 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-14T08:33:52.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.079+0000 7f8141f1e640 1 Processor -- start 2024-04-14T08:33:52.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.079+0000 7f8141f1e640 1 -- start start 2024-04-14T08:33:52.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.080+0000 7f8141f1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f813c0af340 0x7f813c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:52.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.080+0000 7f8141f1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f813c06dee0 0x7f813c0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:52.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.080+0000 7f8141f1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f813c0cf920 con 0x7f813c0cc450 2024-04-14T08:33:52.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.080+0000 7f8141f1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f813c0cdea0 con 0x7f813c0af340 2024-04-14T08:33:52.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.080+0000 7f8141f1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f813c15f9e0 con 0x7f813c06dee0 2024-04-14T08:33:52.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.080+0000 7f813affd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f813c0af340 0x7f813c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:52.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.080+0000 7f813b7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f813c0cc450 0x7f813c06d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34962/0 (socket says 172.21.15.172:34962) 2024-04-14T08:33:52.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.080+0000 7f813b7fe640 1 -- 172.21.15.172:0/3568640329 learned_addr learned my addr 172.21.15.172:0/3568640329 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:52.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.081+0000 7f813affd640 1 -- 172.21.15.172:0/3568640329 >> v2:172.21.15.177:3300/0 conn(0x7f813c06dee0 msgr2=0x7f813c0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:52.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.081+0000 7f813affd640 1 --2- 172.21.15.172:0/3568640329 >> v2:172.21.15.177:3300/0 conn(0x7f813c06dee0 0x7f813c0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:52.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.081+0000 7f813affd640 1 -- 172.21.15.172:0/3568640329 >> v1:172.21.15.172:6789/0 conn(0x7f813c0cc450 legacy=0x7f813c06d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:52.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.081+0000 7f813affd640 1 -- 172.21.15.172:0/3568640329 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813c0ce0f0 con 0x7f813c0af340 2024-04-14T08:33:52.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.081+0000 7f813a7fc640 1 -- 172.21.15.172:0/3568640329 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1845270286 0 0) 0x7f812c001110 con 0x7f813c0cc450 2024-04-14T08:33:52.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.081+0000 7f813affd640 1 --2- 172.21.15.172:0/3568640329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c0af340 0x7f813c0af790 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f813c06f190 tx=0x7f813002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c969934d3ee85a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:52.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.081+0000 7f813a7fc640 1 -- 172.21.15.172:0/3568640329 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130035020 con 0x7f813c0af340 2024-04-14T08:33:52.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.082+0000 7f813a7fc640 1 -- 172.21.15.172:0/3568640329 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8130031830 con 0x7f813c0af340 2024-04-14T08:33:52.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.082+0000 7f813a7fc640 1 -- 172.21.15.172:0/3568640329 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130031c00 con 0x7f813c0af340 2024-04-14T08:33:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.082+0000 7f8141f1e640 1 -- 172.21.15.172:0/3568640329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c0af340 msgr2=0x7f813c0af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.082+0000 7f8141f1e640 1 --2- 172.21.15.172:0/3568640329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c0af340 0x7f813c0af790 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f813c06f190 tx=0x7f813002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.082+0000 7f8141f1e640 1 -- 172.21.15.172:0/3568640329 shutdown_connections 2024-04-14T08:33:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.082+0000 7f8141f1e640 1 --2- 172.21.15.172:0/3568640329 >> v2:172.21.15.177:3300/0 conn(0x7f813c06dee0 0x7f813c0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.082+0000 7f8141f1e640 1 --2- 172.21.15.172:0/3568640329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c0af340 0x7f813c0af790 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.082+0000 7f8141f1e640 1 -- 172.21.15.172:0/3568640329 >> 172.21.15.172:0/3568640329 conn(0x7f813c072b20 msgr2=0x7f813c072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.082+0000 7f8141f1e640 1 -- 172.21.15.172:0/3568640329 shutdown_connections 2024-04-14T08:33:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.082+0000 7f8141f1e640 1 -- 172.21.15.172:0/3568640329 wait complete. 2024-04-14T08:33:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.083+0000 7f8141f1e640 1 Processor -- start 2024-04-14T08:33:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.084+0000 7f8141f1e640 1 -- start start 2024-04-14T08:33:52.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.084+0000 7f8141f1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f813c06dee0 0x7f813c0b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:52.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.084+0000 7f8141f1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c0af340 0x7f813c0b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:52.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.084+0000 7f8141f1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f813c0cc450 0x7f813c0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:52.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.084+0000 7f8141f1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f813c0ce3d0 con 0x7f813c0af340 2024-04-14T08:33:52.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.084+0000 7f8141f1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f813c06f7a0 con 0x7f813c0cc450 2024-04-14T08:33:52.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.084+0000 7f8141f1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f813c15f600 con 0x7f813c06dee0 2024-04-14T08:33:52.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.084+0000 7f813affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c0af340 0x7f813c0b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:52.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.084+0000 7f813affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c0af340 0x7f813c0b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57212/0 (socket says 172.21.15.172:57212) 2024-04-14T08:33:52.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.084+0000 7f813affd640 1 -- 172.21.15.172:0/2846314367 learned_addr learned my addr 172.21.15.172:0/2846314367 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:52.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f813b7fe640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f813c06dee0 0x7f813c0b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:52.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f813bfff640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f813c0cc450 0x7f813c0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:52.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f813affd640 1 -- 172.21.15.172:0/2846314367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f813c06dee0 msgr2=0x7f813c0b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:52.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f813affd640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f813c06dee0 0x7f813c0b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:52.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f813affd640 1 -- 172.21.15.172:0/2846314367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f813c0cc450 msgr2=0x7f813c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:52.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f813affd640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f813c0cc450 0x7f813c0d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:52.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f813affd640 1 -- 172.21.15.172:0/2846314367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813c0b5150 con 0x7f813c0af340 2024-04-14T08:33:52.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f813b7fe640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f813c06dee0 0x7f813c0b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:33:52.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f813bfff640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f813c0cc450 0x7f813c0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:33:52.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f813affd640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c0af340 0x7f813c0b4770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f813002bee0 tx=0x7f81300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:52.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f8123fff640 1 -- 172.21.15.172:0/2846314367 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130004230 con 0x7f813c0af340 2024-04-14T08:33:52.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f8123fff640 1 -- 172.21.15.172:0/2846314367 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81300043d0 con 0x7f813c0af340 2024-04-14T08:33:52.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f8141f1e640 1 -- 172.21.15.172:0/2846314367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813c0b5430 con 0x7f813c0af340 2024-04-14T08:33:52.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f8123fff640 1 -- 172.21.15.172:0/2846314367 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813003cad0 con 0x7f813c0af340 2024-04-14T08:33:52.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.085+0000 7f8141f1e640 1 -- 172.21.15.172:0/2846314367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f813c0b3e70 con 0x7f813c0af340 2024-04-14T08:33:52.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.086+0000 7f8141f1e640 1 -- 172.21.15.172:0/2846314367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f813c0686a0 con 0x7f813c0af340 2024-04-14T08:33:52.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.086+0000 7f8123fff640 1 -- 172.21.15.172:0/2846314367 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8130038020 con 0x7f813c0af340 2024-04-14T08:33:52.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.087+0000 7f8123fff640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f812806c8f0 0x7f812806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:52.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.087+0000 7f8123fff640 1 -- 172.21.15.172:0/2846314367 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f81300adf40 con 0x7f813c0af340 2024-04-14T08:33:52.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.087+0000 7f813b7fe640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f812806c8f0 0x7f812806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:52.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.087+0000 7f813b7fe640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f812806c8f0 0x7f812806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f812c000f60 tx=0x7f812c007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:52.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.091+0000 7f8123fff640 1 -- 172.21.15.172:0/2846314367 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8130041070 con 0x7f813c0af340 2024-04-14T08:33:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.276+0000 7f8141f1e640 1 -- 172.21.15.172:0/2846314367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f813c0b6250 con 0x7f813c0af340 2024-04-14T08:33:52.905 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:33:52.909 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:33:52.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.948+0000 7f8123fff640 1 -- 172.21.15.172:0/2846314367 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f813007c370 con 0x7f813c0af340 2024-04-14T08:33:52.950 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data_ec' removed 2024-04-14T08:33:52.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.952+0000 7f8141f1e640 1 -- 172.21.15.172:0/2846314367 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f812806c8f0 msgr2=0x7f812806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:52.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.952+0000 7f8141f1e640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f812806c8f0 0x7f812806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f812c000f60 tx=0x7f812c007ca0 comp rx=0 tx=0).stop 2024-04-14T08:33:52.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.952+0000 7f8141f1e640 1 -- 172.21.15.172:0/2846314367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c0af340 msgr2=0x7f813c0b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:52.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.952+0000 7f8141f1e640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c0af340 0x7f813c0b4770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f813002bee0 tx=0x7f81300046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:52.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.953+0000 7f8141f1e640 1 -- 172.21.15.172:0/2846314367 shutdown_connections 2024-04-14T08:33:52.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.953+0000 7f8141f1e640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f812806c8f0 0x7f812806edb0 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f812c000f60 tx=0x7f812c007ca0 comp rx=0 tx=0).stop 2024-04-14T08:33:52.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.953+0000 7f8141f1e640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f813c0cc450 0x7f813c0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:52.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.953+0000 7f8141f1e640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c0af340 0x7f813c0b4770 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:52.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.953+0000 7f8141f1e640 1 --2- 172.21.15.172:0/2846314367 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f813c06dee0 0x7f813c0b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:52.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.953+0000 7f8141f1e640 1 -- 172.21.15.172:0/2846314367 >> 172.21.15.172:0/2846314367 conn(0x7f813c072b20 msgr2=0x7f813c0b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:52.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.953+0000 7f8141f1e640 1 -- 172.21.15.172:0/2846314367 shutdown_connections 2024-04-14T08:33:52.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:52.953+0000 7f8141f1e640 1 -- 172.21.15.172:0/2846314367 wait complete. 2024-04-14T08:33:52.969 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-14T08:33:52.969 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-14T08:33:52.969 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-14T08:33:52.969 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:33:52.970 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-14T08:33:52.970 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-14T08:33:52.970 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-14T08:33:52.970 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:33:52.970 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-14T08:33:52.970 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-14T08:33:52.971 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-14T08:33:52.971 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:33:52.971 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-14T08:33:52.971 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-14T08:33:52.971 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-14T08:33:52.971 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:33:52.982 INFO:tasks.ceph.mds.b:Stopped 2024-04-14T08:33:52.982 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-14T08:33:52.984 INFO:tasks.ceph.mds.d:Stopped 2024-04-14T08:33:52.984 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-14T08:33:53.021 INFO:tasks.ceph.mds.a:Stopped 2024-04-14T08:33:53.022 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-14T08:33:53.023 INFO:tasks.ceph.mds.c:Stopped 2024-04-14T08:33:53.023 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-14T08:33:53.025 INFO:tasks.ceph.mds.b:Started 2024-04-14T08:33:53.025 DEBUG:teuthology.parallel:result is None 2024-04-14T08:33:53.026 INFO:tasks.ceph.mds.d:Started 2024-04-14T08:33:53.026 DEBUG:teuthology.parallel:result is None 2024-04-14T08:33:53.064 INFO:tasks.ceph.mds.a:Started 2024-04-14T08:33:53.065 DEBUG:teuthology.parallel:result is None 2024-04-14T08:33:53.065 INFO:tasks.ceph.mds.c:Started 2024-04-14T08:33:53.065 DEBUG:teuthology.parallel:result is None 2024-04-14T08:33:53.066 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-14T08:33:53.106 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-04-14T08:33:53.106 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-04-14T08:33:53.150 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:33:53.151 INFO:tasks.ceph.mds.a.smithi172.stdout:starting mds.a at 2024-04-14T08:33:53.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.178+0000 7fe40adf0640 1 Processor -- start 2024-04-14T08:33:53.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.178+0000 7fe40adf0640 1 -- start start 2024-04-14T08:33:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.179+0000 7fe40adf0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe404154890 0x7fe4040b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.179+0000 7fe40adf0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4040cf3d0 0x7fe4040cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.179+0000 7fe40adf0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe40415a8c0 con 0x7fe4040b1ec0 2024-04-14T08:33:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.179+0000 7fe40adf0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe4040cfd70 con 0x7fe4040cf3d0 2024-04-14T08:33:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.179+0000 7fe40adf0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe40415ab60 con 0x7fe404154890 2024-04-14T08:33:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.179+0000 7fe403fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4040cf3d0 0x7fe4040cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:53.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.179+0000 7fe403fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4040cf3d0 0x7fe4040cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57252/0 (socket says 172.21.15.172:57252) 2024-04-14T08:33:53.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.179+0000 7fe403fff640 1 -- 172.21.15.172:0/3627634018 learned_addr learned my addr 172.21.15.172:0/3627634018 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:53.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.179+0000 7fe408b8c640 1 --2- 172.21.15.172:0/3627634018 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe404154890 0x7fe4040b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:53.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.180+0000 7fe403fff640 1 -- 172.21.15.172:0/3627634018 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe404154890 msgr2=0x7fe4040b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:53.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.180+0000 7fe403fff640 1 --2- 172.21.15.172:0/3627634018 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe404154890 0x7fe4040b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:53.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.180+0000 7fe403fff640 1 -- 172.21.15.172:0/3627634018 >> v1:172.21.15.177:6789/0 conn(0x7fe4040b1ec0 legacy=0x7fe4040b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:53.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.180+0000 7fe403fff640 1 -- 172.21.15.172:0/3627634018 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40415ae40 con 0x7fe4040cf3d0 2024-04-14T08:33:53.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.180+0000 7fe403fff640 1 --2- 172.21.15.172:0/3627634018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4040cf3d0 0x7fe4040cf830 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0008870 tx=0x7fe3f002df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9e15dfeae6c16a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:53.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.180+0000 7fe4037fe640 1 -- 172.21.15.172:0/3627634018 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0034070 con 0x7fe4040cf3d0 2024-04-14T08:33:53.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.180+0000 7fe4037fe640 1 -- 172.21.15.172:0/3627634018 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f002ed40 con 0x7fe4040cf3d0 2024-04-14T08:33:53.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.180+0000 7fe4037fe640 1 -- 172.21.15.172:0/3627634018 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0038620 con 0x7fe4040cf3d0 2024-04-14T08:33:53.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.181+0000 7fe40adf0640 1 -- 172.21.15.172:0/3627634018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4040cf3d0 msgr2=0x7fe4040cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:53.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.181+0000 7fe40adf0640 1 --2- 172.21.15.172:0/3627634018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4040cf3d0 0x7fe4040cf830 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0008870 tx=0x7fe3f002df90 comp rx=0 tx=0).stop 2024-04-14T08:33:53.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.181+0000 7fe40adf0640 1 -- 172.21.15.172:0/3627634018 shutdown_connections 2024-04-14T08:33:53.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.181+0000 7fe40adf0640 1 --2- 172.21.15.172:0/3627634018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4040cf3d0 0x7fe4040cf830 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:53.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.181+0000 7fe40adf0640 1 --2- 172.21.15.172:0/3627634018 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe404154890 0x7fe4040b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:53.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.181+0000 7fe40adf0640 1 -- 172.21.15.172:0/3627634018 >> 172.21.15.172:0/3627634018 conn(0x7fe40406b520 msgr2=0x7fe40406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:53.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.181+0000 7fe40adf0640 1 -- 172.21.15.172:0/3627634018 shutdown_connections 2024-04-14T08:33:53.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.181+0000 7fe40adf0640 1 -- 172.21.15.172:0/3627634018 wait complete. 2024-04-14T08:33:53.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.182+0000 7fe40adf0640 1 Processor -- start 2024-04-14T08:33:53.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.182+0000 7fe40adf0640 1 -- start start 2024-04-14T08:33:53.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.183+0000 7fe40adf0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4040b1ec0 0x7fe4041f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:53.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.183+0000 7fe40adf0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4040cf3d0 0x7fe4040a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:53.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.183+0000 7fe403fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4040cf3d0 0x7fe4040a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:53.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.183+0000 7fe403fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4040cf3d0 0x7fe4040a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57258/0 (socket says 172.21.15.172:57258) 2024-04-14T08:33:53.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.183+0000 7fe40adf0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe404154890 0x7fe4041f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:53.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.183+0000 7fe403fff640 1 -- 172.21.15.172:0/4196465831 learned_addr learned my addr 172.21.15.172:0/4196465831 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:53.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.183+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe40415b6e0 con 0x7fe4040cf3d0 2024-04-14T08:33:53.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe40415b400 con 0x7fe404154890 2024-04-14T08:33:53.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe408b8c640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4040b1ec0 0x7fe4041f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:53.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe404154c70 con 0x7fe4040b1ec0 2024-04-14T08:33:53.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe40938d640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe404154890 0x7fe4041f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:53.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe403fff640 1 -- 172.21.15.172:0/4196465831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4040b1ec0 msgr2=0x7fe4041f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:53.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe403fff640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4040b1ec0 0x7fe4041f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:53.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe403fff640 1 -- 172.21.15.172:0/4196465831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe404154890 msgr2=0x7fe4041f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:53.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe403fff640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe404154890 0x7fe4041f2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:53.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe403fff640 1 -- 172.21.15.172:0/4196465831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe404156c70 con 0x7fe4040cf3d0 2024-04-14T08:33:53.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe403fff640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4040cf3d0 0x7fe4040a94d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0008990 tx=0x7fe3f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:53.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe4017fa640 1 -- 172.21.15.172:0/4196465831 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0046070 con 0x7fe4040cf3d0 2024-04-14T08:33:53.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe404156f50 con 0x7fe4040cf3d0 2024-04-14T08:33:53.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe4017fa640 1 -- 172.21.15.172:0/4196465831 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f0008e10 con 0x7fe4040cf3d0 2024-04-14T08:33:53.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe4017fa640 1 -- 172.21.15.172:0/4196465831 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0040810 con 0x7fe4040cf3d0 2024-04-14T08:33:53.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.184+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4041577f0 con 0x7fe4040cf3d0 2024-04-14T08:33:53.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.185+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe404157340 con 0x7fe4040cf3d0 2024-04-14T08:33:53.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.186+0000 7fe4017fa640 1 -- 172.21.15.172:0/4196465831 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3f0049070 con 0x7fe4040cf3d0 2024-04-14T08:33:53.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.186+0000 7fe4017fa640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe3e806c8f0 0x7fe3e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:53.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.186+0000 7fe408b8c640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe3e806c8f0 0x7fe3e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:53.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.186+0000 7fe4017fa640 1 -- 172.21.15.172:0/4196465831 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7fe3f00b1420 con 0x7fe4040cf3d0 2024-04-14T08:33:53.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.187+0000 7fe408b8c640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe3e806c8f0 0x7fe3e806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009b40 tx=0x7fe3f8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:53.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.191+0000 7fe4017fa640 1 -- 172.21.15.172:0/4196465831 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe3f007f400 con 0x7fe4040cf3d0 2024-04-14T08:33:53.225 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-14T08:33:53.225 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-14T08:33:53.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.378+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe404068590 con 0x7fe4040cf3d0 2024-04-14T08:33:53.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.973+0000 7fe4017fa640 1 -- 172.21.15.172:0/4196465831 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7fe3f0083700 con 0x7fe4040cf3d0 2024-04-14T08:33:53.975 INFO:teuthology.orchestra.run.smithi172.stderr: removed all blocklist entries 2024-04-14T08:33:53.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.976+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe3e806c8f0 msgr2=0x7fe3e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:53.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.976+0000 7fe40adf0640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe3e806c8f0 0x7fe3e806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009b40 tx=0x7fe3f8009190 comp rx=0 tx=0).stop 2024-04-14T08:33:53.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.976+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4040cf3d0 msgr2=0x7fe4040a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:53.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.976+0000 7fe40adf0640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4040cf3d0 0x7fe4040a94d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0008990 tx=0x7fe3f0004730 comp rx=0 tx=0).stop 2024-04-14T08:33:53.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.976+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 shutdown_connections 2024-04-14T08:33:53.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.976+0000 7fe40adf0640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe3e806c8f0 0x7fe3e806edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:53.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.976+0000 7fe40adf0640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe404154890 0x7fe4041f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:53.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.976+0000 7fe40adf0640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4040cf3d0 0x7fe4040a94d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:53.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.976+0000 7fe40adf0640 1 --2- 172.21.15.172:0/4196465831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4040b1ec0 0x7fe4041f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:53.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.976+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 >> 172.21.15.172:0/4196465831 conn(0x7fe40406b520 msgr2=0x7fe4040b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:53.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.976+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 shutdown_connections 2024-04-14T08:33:53.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:53.976+0000 7fe40adf0640 1 -- 172.21.15.172:0/4196465831 wait complete. 2024-04-14T08:33:53.993 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-14T08:33:54.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.096+0000 7fc1c8e36640 1 Processor -- start 2024-04-14T08:33:54.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.096+0000 7fc1c8e36640 1 -- start start 2024-04-14T08:33:54.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.097+0000 7fc1c8e36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1c4072970 0x7fc1c415b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.097+0000 7fc1c8e36640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c40b3360 con 0x7fc1c415a690 2024-04-14T08:33:54.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.097+0000 7fc1c8e36640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c40b3640 con 0x7fc1c4072470 2024-04-14T08:33:54.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.097+0000 7fc1c8e36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c40b3920 con 0x7fc1c4072970 2024-04-14T08:33:54.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.098+0000 7fc1c259c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc1c415a690 0x7fc1c4071d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34982/0 (socket says 172.21.15.172:34982) 2024-04-14T08:33:54.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.098+0000 7fc1c259c640 1 -- 172.21.15.172:0/3412012688 learned_addr learned my addr 172.21.15.172:0/3412012688 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:54.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.098+0000 7fc1c2d9d640 1 --2- 172.21.15.172:0/3412012688 >> v2:172.21.15.172:3300/0 conn(0x7fc1c4072970 0x7fc1c415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.098+0000 7fc1c159a640 1 -- 172.21.15.172:0/3412012688 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1166660096 0 0) 0x7fc1c40b3360 con 0x7fc1c415a690 2024-04-14T08:33:54.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.098+0000 7fc1c159a640 1 -- 172.21.15.172:0/3412012688 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1ac003520 con 0x7fc1c415a690 2024-04-14T08:33:54.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.098+0000 7fc1c2d9d640 1 -- 172.21.15.172:0/3412012688 >> v1:172.21.15.177:6789/0 conn(0x7fc1c4072470 legacy=0x7fc1c415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:54.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.098+0000 7fc1c2d9d640 1 -- 172.21.15.172:0/3412012688 >> v1:172.21.15.172:6789/0 conn(0x7fc1c415a690 legacy=0x7fc1c4071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.098+0000 7fc1c2d9d640 1 -- 172.21.15.172:0/3412012688 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c40b3c00 con 0x7fc1c4072970 2024-04-14T08:33:54.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.099+0000 7fc1c2d9d640 1 --2- 172.21.15.172:0/3412012688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1c4072970 0x7fc1c415b940 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00aea0 tx=0x7fc1bc02cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5961b70bd6eb6fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:54.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.099+0000 7fc1c159a640 1 -- 172.21.15.172:0/3412012688 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc02db60 con 0x7fc1c4072970 2024-04-14T08:33:54.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.099+0000 7fc1c159a640 1 -- 172.21.15.172:0/3412012688 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1bc02dd00 con 0x7fc1c4072970 2024-04-14T08:33:54.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.099+0000 7fc1c159a640 1 -- 172.21.15.172:0/3412012688 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc008640 con 0x7fc1c4072970 2024-04-14T08:33:54.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.099+0000 7fc1c8e36640 1 -- 172.21.15.172:0/3412012688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1c4072970 msgr2=0x7fc1c415b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.099+0000 7fc1c8e36640 1 --2- 172.21.15.172:0/3412012688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1c4072970 0x7fc1c415b940 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00aea0 tx=0x7fc1bc02cfc0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.099+0000 7fc1c8e36640 1 -- 172.21.15.172:0/3412012688 shutdown_connections 2024-04-14T08:33:54.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.099+0000 7fc1c8e36640 1 --2- 172.21.15.172:0/3412012688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1c4072970 0x7fc1c415b940 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.099+0000 7fc1c8e36640 1 -- 172.21.15.172:0/3412012688 >> 172.21.15.172:0/3412012688 conn(0x7fc1c406ec30 msgr2=0x7fc1c406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:54.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.099+0000 7fc1c8e36640 1 -- 172.21.15.172:0/3412012688 shutdown_connections 2024-04-14T08:33:54.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.099+0000 7fc1c8e36640 1 -- 172.21.15.172:0/3412012688 wait complete. 2024-04-14T08:33:54.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.101+0000 7fc1c8e36640 1 Processor -- start 2024-04-14T08:33:54.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.101+0000 7fc1c8e36640 1 -- start start 2024-04-14T08:33:54.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.101+0000 7fc1c8e36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1c4072470 0x7fc1c4153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.101+0000 7fc1c8e36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1c4072970 0x7fc1c40c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.101+0000 7fc1c8e36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1c415a690 0x7fc1c41f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.101+0000 7fc1c8e36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c415abf0 con 0x7fc1c4072470 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.101+0000 7fc1c8e36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c415aa70 con 0x7fc1c4072970 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.101+0000 7fc1c8e36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c415ad70 con 0x7fc1c415a690 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1c4072470 0x7fc1c4153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1c4072470 0x7fc1c4153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57264/0 (socket says 172.21.15.172:57264) 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c259c640 1 -- 172.21.15.172:0/869599474 learned_addr learned my addr 172.21.15.172:0/869599474 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c259c640 1 -- 172.21.15.172:0/869599474 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1c415a690 msgr2=0x7fc1c41f74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c1d9b640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1c4072970 0x7fc1c40c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c2d9d640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1c415a690 0x7fc1c41f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c259c640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1c415a690 0x7fc1c41f74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c259c640 1 -- 172.21.15.172:0/869599474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1c4072970 msgr2=0x7fc1c40c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c259c640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1c4072970 0x7fc1c40c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c259c640 1 -- 172.21.15.172:0/869599474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c40c9ee0 con 0x7fc1c4072470 2024-04-14T08:33:54.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c259c640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1c4072470 0x7fc1c4153ff0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800c440 tx=0x7fc1b800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:54.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1b2ffd640 1 -- 172.21.15.172:0/869599474 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8017070 con 0x7fc1c4072470 2024-04-14T08:33:54.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1b2ffd640 1 -- 172.21.15.172:0/869599474 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b8005ce0 con 0x7fc1c4072470 2024-04-14T08:33:54.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c8e36640 1 -- 172.21.15.172:0/869599474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c4153ba0 con 0x7fc1c4072470 2024-04-14T08:33:54.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1b2ffd640 1 -- 172.21.15.172:0/869599474 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80072b0 con 0x7fc1c4072470 2024-04-14T08:33:54.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.102+0000 7fc1c8e36640 1 -- 172.21.15.172:0/869599474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c41538c0 con 0x7fc1c4072470 2024-04-14T08:33:54.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.103+0000 7fc1c8e36640 1 -- 172.21.15.172:0/869599474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c4068520 con 0x7fc1c4072470 2024-04-14T08:33:54.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.104+0000 7fc1b2ffd640 1 -- 172.21.15.172:0/869599474 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1b801a070 con 0x7fc1c4072470 2024-04-14T08:33:54.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.104+0000 7fc1b2ffd640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1a406c8f0 0x7fc1a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.104+0000 7fc1b2ffd640 1 -- 172.21.15.172:0/869599474 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fc1b808ace0 con 0x7fc1c4072470 2024-04-14T08:33:54.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.104+0000 7fc1c1d9b640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1a406c8f0 0x7fc1a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.105+0000 7fc1c1d9b640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1a406c8f0 0x7fc1a406edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40cbf30 tx=0x7fc1b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:54.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.109+0000 7fc1b2ffd640 1 -- 172.21.15.172:0/869599474 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc1c40c9ee0 con 0x7fc1c4072470 2024-04-14T08:33:54.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.351+0000 7fc1c8e36640 1 -- 172.21.15.172:0/869599474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc1c415cd80 con 0x7fc1c4072470 2024-04-14T08:33:54.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.351+0000 7fc1b2ffd640 1 -- 172.21.15.172:0/869599474 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7fc1b8059390 con 0x7fc1c4072470 2024-04-14T08:33:54.353 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:54.353 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:33:54.353 INFO:teuthology.orchestra.run.smithi172.stdout: "auth_dump": [ 2024-04-14T08:33:54.353 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.353 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.a", 2024-04-14T08:33:54.353 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmnNCRDRAAz7aU3JD7iiaTQH0cNotfaw==", 2024-04-14T08:33:54.354 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.354 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:33:54.354 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:33:54.354 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:33:54.354 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.354 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.354 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.354 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.354 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.b", 2024-04-14T08:33:54.355 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtm1on4GRAAms1he+O4jWu01z0ZLLgL0A==", 2024-04-14T08:33:54.355 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.355 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:33:54.355 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:33:54.355 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:33:54.355 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.355 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.355 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.355 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.355 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.c", 2024-04-14T08:33:54.355 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmb7z0FBAAGVu3+TWshDGvd7/fLbgJ5A==", 2024-04-14T08:33:54.355 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.356 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:33:54.356 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:33:54.356 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:33:54.356 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.356 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.356 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.356 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.356 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.d", 2024-04-14T08:33:54.356 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmEEx0IRAAXiFiOK1RE/0bfyu7KI/3ng==", 2024-04-14T08:33:54.356 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.356 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:33:54.357 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:33:54.357 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:33:54.357 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.357 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.357 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.357 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.357 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.0", 2024-04-14T08:33:54.357 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB/lBtmnsWuCRAAqcuQhsrzpGVfrlgwTnFsOw==", 2024-04-14T08:33:54.357 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.357 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:33:54.358 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:33:54.358 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.358 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.358 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.358 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.358 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.1", 2024-04-14T08:33:54.358 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCClBtme9lPBxAAFB1E3g8qwZxPCdHiFogp5Q==", 2024-04-14T08:33:54.358 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.358 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:33:54.358 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:33:54.358 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.359 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.359 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.359 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.359 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.2", 2024-04-14T08:33:54.359 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCFlBtm966YBRAAWXZGq/0TuUMZTsyeR4xWww==", 2024-04-14T08:33:54.359 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.359 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:33:54.359 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:33:54.359 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.359 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.359 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.360 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.360 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.3", 2024-04-14T08:33:54.360 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCIlBtmHpktBBAAUbddNHESwyNvvp8UE/SaCw==", 2024-04-14T08:33:54.360 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.360 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:33:54.360 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:33:54.360 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.360 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.360 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.360 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.360 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.4", 2024-04-14T08:33:54.360 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCOlBtm+//7ChAA7rApbKll2vBGcDlW8q+kyg==", 2024-04-14T08:33:54.361 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.361 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:33:54.361 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:33:54.361 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.361 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.361 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.361 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.361 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.5", 2024-04-14T08:33:54.361 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCRlBtmVU4YChAAysdUosQ+TnHtjnRl1FOISA==", 2024-04-14T08:33:54.361 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.362 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:33:54.362 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:33:54.362 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.362 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.362 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.362 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.362 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.6", 2024-04-14T08:33:54.362 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCUlBtmnE7+BxAAqBn8YMwecA2NcZz63eBm7Q==", 2024-04-14T08:33:54.362 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.362 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:33:54.362 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:33:54.363 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.363 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.363 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.363 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.363 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.7", 2024-04-14T08:33:54.363 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCXlBtmN8DhBRAA6CHQ943t6qJCL77jcbOv4w==", 2024-04-14T08:33:54.363 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.363 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:33:54.363 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:33:54.363 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.363 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.363 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.364 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.364 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.0", 2024-04-14T08:33:54.364 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA==", 2024-04-14T08:33:54.364 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.364 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:33:54.364 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:33:54.364 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:33:54.364 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:33:54.364 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.364 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.364 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.364 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.1", 2024-04-14T08:33:54.365 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmp4nkLhAAhYKm8VcykVbPSohvK11I9w==", 2024-04-14T08:33:54.365 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.365 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:33:54.365 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:33:54.365 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:33:54.365 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:33:54.365 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.365 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.365 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.366 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.2", 2024-04-14T08:33:54.366 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmJoV9KxAAp6zlRX+gvhASQBHjpF9ewQ==", 2024-04-14T08:33:54.366 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.366 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:33:54.366 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:33:54.366 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:33:54.366 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:33:54.366 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.366 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.366 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.366 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.3", 2024-04-14T08:33:54.366 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmeC+pNBAASi7W4jlo6imMMzNEwLrPDA==", 2024-04-14T08:33:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:33:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:33:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:33:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:33:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.admin", 2024-04-14T08:33:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB6lBtmgnWqLBAAQ0fgarM8au3zFuMNnQIvVw==", 2024-04-14T08:33:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.368 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:33:54.368 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:33:54.368 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:33:54.368 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.368 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.368 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.368 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.368 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.bootstrap-mds", 2024-04-14T08:33:54.368 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCclBtm+DlbORAABRZuIQkrxotcwG4F35dhAg==", 2024-04-14T08:33:54.368 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.368 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile bootstrap-mds" 2024-04-14T08:33:54.369 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.369 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.369 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.369 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.bootstrap-mgr", 2024-04-14T08:33:54.369 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCclBtmskJbORAAUWuLZb5XDxbWHJB2PTKNdg==", 2024-04-14T08:33:54.369 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.369 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-14T08:33:54.369 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.369 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.369 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.369 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.bootstrap-osd", 2024-04-14T08:33:54.369 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCclBtmjUlbORAAET50wdpvyZRr2LE3EQoi/g==", 2024-04-14T08:33:54.370 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.370 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile bootstrap-osd" 2024-04-14T08:33:54.370 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.370 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.370 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.370 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.bootstrap-rbd", 2024-04-14T08:33:54.370 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCclBtmXlBbORAAO0ekqkctpOvNJeYSLcNFxg==", 2024-04-14T08:33:54.370 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.370 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-14T08:33:54.370 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.370 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.370 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.371 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-14T08:33:54.371 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCclBtmaldbORAAkrCizP12neZjJQnNu32GRA==", 2024-04-14T08:33:54.371 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.371 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-14T08:33:54.371 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.371 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.371 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.371 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.bootstrap-rgw", 2024-04-14T08:33:54.371 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCclBtmp15bORAAUBJss2nm3K1T72XGeYK7cA==", 2024-04-14T08:33:54.371 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.371 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-14T08:33:54.372 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.372 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.372 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.372 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.x", 2024-04-14T08:33:54.372 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmm5IbChAAeVcVs3Zt6+lU0wghjCcwnQ==", 2024-04-14T08:33:54.372 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.372 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:33:54.372 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:33:54.372 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.372 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.372 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:33:54.372 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:54.373 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.y", 2024-04-14T08:33:54.373 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmfddYBhAAh8zgf4W3WWmNYWNAA7WuRw==", 2024-04-14T08:33:54.373 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:33:54.373 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:33:54.373 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:33:54.373 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:33:54.373 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.373 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:54.373 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-04-14T08:33:54.373 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:33:54.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.353+0000 7fc1c8e36640 1 -- 172.21.15.172:0/869599474 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1a406c8f0 msgr2=0x7fc1a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.353+0000 7fc1c8e36640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1a406c8f0 0x7fc1a406edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40cbf30 tx=0x7fc1b4009040 comp rx=0 tx=0).stop 2024-04-14T08:33:54.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.353+0000 7fc1c8e36640 1 -- 172.21.15.172:0/869599474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1c4072470 msgr2=0x7fc1c4153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.353+0000 7fc1c8e36640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1c4072470 0x7fc1c4153ff0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800c440 tx=0x7fc1b800c900 comp rx=0 tx=0).stop 2024-04-14T08:33:54.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.353+0000 7fc1c8e36640 1 -- 172.21.15.172:0/869599474 shutdown_connections 2024-04-14T08:33:54.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.353+0000 7fc1c8e36640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1a406c8f0 0x7fc1a406edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.353+0000 7fc1c8e36640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1c415a690 0x7fc1c41f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.353+0000 7fc1c8e36640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1c4072970 0x7fc1c40c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.353+0000 7fc1c8e36640 1 --2- 172.21.15.172:0/869599474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1c4072470 0x7fc1c4153ff0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.353+0000 7fc1c8e36640 1 -- 172.21.15.172:0/869599474 >> 172.21.15.172:0/869599474 conn(0x7fc1c406ec30 msgr2=0x7fc1c40cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.353+0000 7fc1c8e36640 1 -- 172.21.15.172:0/869599474 shutdown_connections 2024-04-14T08:33:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.353+0000 7fc1c8e36640 1 -- 172.21.15.172:0/869599474 wait complete. 2024-04-14T08:33:54.376 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-14T08:33:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.477+0000 7f1deffa3640 1 Processor -- start 2024-04-14T08:33:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.477+0000 7f1deffa3640 1 -- start start 2024-04-14T08:33:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.478+0000 7f1deffa3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de80d0cc0 0x7f1de80d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.478+0000 7f1deffa3640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1de80d49c0 0x7f1de80d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.478+0000 7f1deffa3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1de8160f30 0x7f1de81613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.478+0000 7f1deffa3640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f1de80ceec0 con 0x7f1de8160f30 2024-04-14T08:33:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.478+0000 7f1deffa3640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f1de80cfd00 con 0x7f1de80d49c0 2024-04-14T08:33:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.478+0000 7f1deffa3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1de80d15e0 con 0x7f1de80d0cc0 2024-04-14T08:33:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.478+0000 7f1dee540640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1de8160f30 0x7f1de81613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.478+0000 7f1dee540640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1de8160f30 0x7f1de81613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57280/0 (socket says 172.21.15.172:57280) 2024-04-14T08:33:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.478+0000 7f1dee540640 1 -- 172.21.15.172:0/3413768405 learned_addr learned my addr 172.21.15.172:0/3413768405 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.479+0000 7f1dedd3f640 1 --2- 172.21.15.172:0/3413768405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de80d0cc0 0x7f1de80d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.479+0000 7f1ded53e640 1 --2- 172.21.15.172:0/3413768405 >> v2:172.21.15.177:3300/0 conn(0x7f1de80d49c0 0x7f1de80d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.479+0000 7f1dee540640 1 -- 172.21.15.172:0/3413768405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de80d0cc0 msgr2=0x7f1de80d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.479+0000 7f1dee540640 1 --2- 172.21.15.172:0/3413768405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de80d0cc0 0x7f1de80d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.479+0000 7f1dee540640 1 -- 172.21.15.172:0/3413768405 >> v2:172.21.15.177:3300/0 conn(0x7f1de80d49c0 msgr2=0x7f1de80d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.479+0000 7f1dee540640 1 --2- 172.21.15.172:0/3413768405 >> v2:172.21.15.177:3300/0 conn(0x7f1de80d49c0 0x7f1de80d4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.479+0000 7f1dee540640 1 -- 172.21.15.172:0/3413768405 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de80cf0f0 con 0x7f1de8160f30 2024-04-14T08:33:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1dee540640 1 --2- 172.21.15.172:0/3413768405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de8160f30 0x7f1de81613c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1de809dcb0 tx=0x7f1de002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff48eb241b9fce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1decd3d640 1 -- 172.21.15.172:0/3413768405 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0035020 con 0x7f1de8160f30 2024-04-14T08:33:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1decd3d640 1 -- 172.21.15.172:0/3413768405 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de0031870 con 0x7f1de8160f30 2024-04-14T08:33:54.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1decd3d640 1 -- 172.21.15.172:0/3413768405 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0031c40 con 0x7f1de8160f30 2024-04-14T08:33:54.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1deffa3640 1 -- 172.21.15.172:0/3413768405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de8160f30 msgr2=0x7f1de81613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1deffa3640 1 --2- 172.21.15.172:0/3413768405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de8160f30 0x7f1de81613c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1de809dcb0 tx=0x7f1de002b910 comp rx=0 tx=0).stop 2024-04-14T08:33:54.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1deffa3640 1 -- 172.21.15.172:0/3413768405 shutdown_connections 2024-04-14T08:33:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1deffa3640 1 --2- 172.21.15.172:0/3413768405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de8160f30 0x7f1de81613c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1deffa3640 1 --2- 172.21.15.172:0/3413768405 >> v2:172.21.15.177:3300/0 conn(0x7f1de80d49c0 0x7f1de80d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1deffa3640 1 --2- 172.21.15.172:0/3413768405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de80d0cc0 0x7f1de80d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1deffa3640 1 -- 172.21.15.172:0/3413768405 >> 172.21.15.172:0/3413768405 conn(0x7f1de809d580 msgr2=0x7f1de80d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1deffa3640 1 -- 172.21.15.172:0/3413768405 shutdown_connections 2024-04-14T08:33:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.480+0000 7f1deffa3640 1 -- 172.21.15.172:0/3413768405 wait complete. 2024-04-14T08:33:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.482+0000 7f1deffa3640 1 Processor -- start 2024-04-14T08:33:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.482+0000 7f1deffa3640 1 -- start start 2024-04-14T08:33:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.482+0000 7f1deffa3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de80d0cc0 0x7f1de815b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.482+0000 7f1deffa3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de80d49c0 0x7f1de809dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.482+0000 7f1deffa3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de8160f30 0x7f1de809e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.482+0000 7f1deffa3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1de80cf990 con 0x7f1de80d49c0 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.482+0000 7f1deffa3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1de80cf6b0 con 0x7f1de80d0cc0 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.482+0000 7f1deffa3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1de80d32b0 con 0x7f1de8160f30 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.482+0000 7f1ded53e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de80d49c0 0x7f1de809dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.482+0000 7f1ded53e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de80d49c0 0x7f1de809dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57282/0 (socket says 172.21.15.172:57282) 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1ded53e640 1 -- 172.21.15.172:0/2202497340 learned_addr learned my addr 172.21.15.172:0/2202497340 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1dee540640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de8160f30 0x7f1de809e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1dedd3f640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de80d0cc0 0x7f1de815b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1ded53e640 1 -- 172.21.15.172:0/2202497340 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de8160f30 msgr2=0x7f1de809e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1ded53e640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de8160f30 0x7f1de809e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1ded53e640 1 -- 172.21.15.172:0/2202497340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de80d0cc0 msgr2=0x7f1de815b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1ded53e640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de80d0cc0 0x7f1de815b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1ded53e640 1 -- 172.21.15.172:0/2202497340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de80a00c0 con 0x7f1de80d49c0 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1dedd3f640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de80d0cc0 0x7f1de815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:33:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1ded53e640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de80d49c0 0x7f1de809dff0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00e9b0 tx=0x7f1ddc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:54.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1de67fc640 1 -- 172.21.15.172:0/2202497340 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc017070 con 0x7f1de80d49c0 2024-04-14T08:33:54.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1de67fc640 1 -- 172.21.15.172:0/2202497340 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ddc00c9a0 con 0x7f1de80d49c0 2024-04-14T08:33:54.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1deffa3640 1 -- 172.21.15.172:0/2202497340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de815e2e0 con 0x7f1de80d49c0 2024-04-14T08:33:54.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1de67fc640 1 -- 172.21.15.172:0/2202497340 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc00cd90 con 0x7f1de80d49c0 2024-04-14T08:33:54.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.483+0000 7f1deffa3640 1 -- 172.21.15.172:0/2202497340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de815e8a0 con 0x7f1de80d49c0 2024-04-14T08:33:54.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.484+0000 7f1deffa3640 1 -- 172.21.15.172:0/2202497340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de80b89a0 con 0x7f1de80d49c0 2024-04-14T08:33:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.485+0000 7f1de67fc640 1 -- 172.21.15.172:0/2202497340 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ddc01a070 con 0x7f1de80d49c0 2024-04-14T08:33:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.485+0000 7f1de67fc640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1dd406c8f0 0x7f1dd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.485+0000 7f1de67fc640 1 -- 172.21.15.172:0/2202497340 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f1ddc08b980 con 0x7f1de80d49c0 2024-04-14T08:33:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.485+0000 7f1dedd3f640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1dd406c8f0 0x7f1dd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.485+0000 7f1dedd3f640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1dd406c8f0 0x7f1dd406edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80074c0 tx=0x7f1dd800d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:54.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.490+0000 7f1de67fc640 1 -- 172.21.15.172:0/2202497340 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1de80b89a0 con 0x7f1de80d49c0 2024-04-14T08:33:54.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.732+0000 7f1deffa3640 1 -- 172.21.15.172:0/2202497340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f1de8075490 con 0x7f1de80d49c0 2024-04-14T08:33:54.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.734+0000 7f1de67fc640 1 -- 172.21.15.172:0/2202497340 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f1ddc059890 con 0x7f1de80d49c0 2024-04-14T08:33:54.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.737+0000 7f1deffa3640 1 -- 172.21.15.172:0/2202497340 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1dd406c8f0 msgr2=0x7f1dd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.737+0000 7f1deffa3640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1dd406c8f0 0x7f1dd406edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80074c0 tx=0x7f1dd800d040 comp rx=0 tx=0).stop 2024-04-14T08:33:54.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.737+0000 7f1deffa3640 1 -- 172.21.15.172:0/2202497340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de80d49c0 msgr2=0x7f1de809dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.737+0000 7f1deffa3640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de80d49c0 0x7f1de809dff0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00e9b0 tx=0x7f1ddc00ee70 comp rx=0 tx=0).stop 2024-04-14T08:33:54.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.737+0000 7f1deffa3640 1 -- 172.21.15.172:0/2202497340 shutdown_connections 2024-04-14T08:33:54.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.737+0000 7f1deffa3640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1dd406c8f0 0x7f1dd406edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.737+0000 7f1deffa3640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de8160f30 0x7f1de809e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.737+0000 7f1deffa3640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de80d49c0 0x7f1de809dff0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.737+0000 7f1deffa3640 1 --2- 172.21.15.172:0/2202497340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de80d0cc0 0x7f1de815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.737+0000 7f1deffa3640 1 -- 172.21.15.172:0/2202497340 >> 172.21.15.172:0/2202497340 conn(0x7f1de809d580 msgr2=0x7f1de80cb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:54.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.737+0000 7f1deffa3640 1 -- 172.21.15.172:0/2202497340 shutdown_connections 2024-04-14T08:33:54.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.737+0000 7f1deffa3640 1 -- 172.21.15.172:0/2202497340 wait complete. 2024-04-14T08:33:54.749 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-14T08:33:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.848+0000 7f6180d1e640 1 Processor -- start 2024-04-14T08:33:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.848+0000 7f6180d1e640 1 -- start start 2024-04-14T08:33:54.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.849+0000 7f6180d1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f617c15e4b0 0x7f617c0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.849+0000 7f6180d1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f617c0cfd70 0x7f617c0d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.849+0000 7f6180d1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f617c15e890 con 0x7f617c0d0fb0 2024-04-14T08:33:54.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.849+0000 7f6180d1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f617c0b49c0 con 0x7f617c15e4b0 2024-04-14T08:33:54.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.849+0000 7f6180d1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f617c0d1f30 con 0x7f617c0cfd70 2024-04-14T08:33:54.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.849+0000 7f617ad9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f617c0d0fb0 0x7f617c161fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34992/0 (socket says 172.21.15.172:34992) 2024-04-14T08:33:54.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.849+0000 7f617ad9d640 1 -- 172.21.15.172:0/3943293638 learned_addr learned my addr 172.21.15.172:0/3943293638 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:54.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.850+0000 7f617a59c640 1 --2- 172.21.15.172:0/3943293638 >> v2:172.21.15.177:3300/0 conn(0x7f617c15e4b0 0x7f617c0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.850+0000 7f6179d9b640 1 --2- 172.21.15.172:0/3943293638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f617c0cfd70 0x7f617c0d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.850+0000 7f617959a640 1 -- 172.21.15.172:0/3943293638 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3925104815 0 0) 0x7f617c15e890 con 0x7f617c0d0fb0 2024-04-14T08:33:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.850+0000 7f617959a640 1 -- 172.21.15.172:0/3943293638 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6164003520 con 0x7f617c0d0fb0 2024-04-14T08:33:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.850+0000 7f6179d9b640 1 -- 172.21.15.172:0/3943293638 >> v2:172.21.15.177:3300/0 conn(0x7f617c15e4b0 msgr2=0x7f617c0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.850+0000 7f6179d9b640 1 --2- 172.21.15.172:0/3943293638 >> v2:172.21.15.177:3300/0 conn(0x7f617c15e4b0 0x7f617c0cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.850+0000 7f6179d9b640 1 -- 172.21.15.172:0/3943293638 >> v1:172.21.15.172:6789/0 conn(0x7f617c0d0fb0 legacy=0x7f617c161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.850+0000 7f6179d9b640 1 -- 172.21.15.172:0/3943293638 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c0d2210 con 0x7f617c0cfd70 2024-04-14T08:33:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.851+0000 7f6179d9b640 1 --2- 172.21.15.172:0/3943293638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f617c0cfd70 0x7f617c0d0a70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f616c0097a0 tx=0x7f616c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1943a796ee4f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.851+0000 7f617959a640 1 -- 172.21.15.172:0/3943293638 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c0315a0 con 0x7f617c0cfd70 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.851+0000 7f617959a640 1 -- 172.21.15.172:0/3943293638 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616c037070 con 0x7f617c0cfd70 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.851+0000 7f617959a640 1 -- 172.21.15.172:0/3943293638 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c031940 con 0x7f617c0cfd70 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.851+0000 7f6180d1e640 1 -- 172.21.15.172:0/3943293638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f617c0cfd70 msgr2=0x7f617c0d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.851+0000 7f6180d1e640 1 --2- 172.21.15.172:0/3943293638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f617c0cfd70 0x7f617c0d0a70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f616c0097a0 tx=0x7f616c02b840 comp rx=0 tx=0).stop 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.851+0000 7f6180d1e640 1 -- 172.21.15.172:0/3943293638 shutdown_connections 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.851+0000 7f6180d1e640 1 --2- 172.21.15.172:0/3943293638 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f617c0cfd70 0x7f617c0d0a70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.851+0000 7f6180d1e640 1 --2- 172.21.15.172:0/3943293638 >> v2:172.21.15.177:3300/0 conn(0x7f617c15e4b0 0x7f617c0cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.851+0000 7f6180d1e640 1 -- 172.21.15.172:0/3943293638 >> 172.21.15.172:0/3943293638 conn(0x7f617c09d570 msgr2=0x7f617c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.852+0000 7f6180d1e640 1 -- 172.21.15.172:0/3943293638 shutdown_connections 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.852+0000 7f6180d1e640 1 -- 172.21.15.172:0/3943293638 wait complete. 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.853+0000 7f6180d1e640 1 Processor -- start 2024-04-14T08:33:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.853+0000 7f6180d1e640 1 -- start start 2024-04-14T08:33:54.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f6180d1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617c0cfd70 0x7f617c0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f6180d1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f617c0d0fb0 0x7f617c156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f6180d1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f617c15e4b0 0x7f617c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f6180d1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f617c0d2ab0 con 0x7f617c0cfd70 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f6180d1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f617c0d27d0 con 0x7f617c15e4b0 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f6180d1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f617c0d2fd0 con 0x7f617c0d0fb0 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f617a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617c0cfd70 0x7f617c0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f617a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617c0cfd70 0x7f617c0cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57298/0 (socket says 172.21.15.172:57298) 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f617a59c640 1 -- 172.21.15.172:0/3304883228 learned_addr learned my addr 172.21.15.172:0/3304883228 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f6179d9b640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f617c0d0fb0 0x7f617c156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f617ad9d640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f617c15e4b0 0x7f617c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f617a59c640 1 -- 172.21.15.172:0/3304883228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f617c0d0fb0 msgr2=0x7f617c156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f617a59c640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f617c0d0fb0 0x7f617c156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f617a59c640 1 -- 172.21.15.172:0/3304883228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f617c15e4b0 msgr2=0x7f617c0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f617a59c640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f617c15e4b0 0x7f617c0c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f617a59c640 1 -- 172.21.15.172:0/3304883228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c0cb990 con 0x7f617c0cfd70 2024-04-14T08:33:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f617a59c640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617c0cfd70 0x7f617c0cb300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f617000e870 tx=0x7f617000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f616affd640 1 -- 172.21.15.172:0/3304883228 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6170017070 con 0x7f617c0cfd70 2024-04-14T08:33:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f616affd640 1 -- 172.21.15.172:0/3304883228 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f617000c940 con 0x7f617c0cfd70 2024-04-14T08:33:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f616affd640 1 -- 172.21.15.172:0/3304883228 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617000cd30 con 0x7f617c0cfd70 2024-04-14T08:33:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f6180d1e640 1 -- 172.21.15.172:0/3304883228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617c0cbc70 con 0x7f617c0cfd70 2024-04-14T08:33:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.854+0000 7f6180d1e640 1 -- 172.21.15.172:0/3304883228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617c0cc230 con 0x7f617c0cfd70 2024-04-14T08:33:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.856+0000 7f6180d1e640 1 -- 172.21.15.172:0/3304883228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f617c068590 con 0x7f617c0cfd70 2024-04-14T08:33:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.856+0000 7f616affd640 1 -- 172.21.15.172:0/3304883228 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f617001a070 con 0x7f617c0cfd70 2024-04-14T08:33:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.856+0000 7f616affd640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f615c06c8f0 0x7f615c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.856+0000 7f616affd640 1 -- 172.21.15.172:0/3304883228 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f617008bbe0 con 0x7f617c0cfd70 2024-04-14T08:33:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.856+0000 7f6179d9b640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f615c06c8f0 0x7f615c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.857+0000 7f6179d9b640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f615c06c8f0 0x7f615c06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f616c036000 tx=0x7f616c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:54.861+0000 7f616affd640 1 -- 172.21.15.172:0/3304883228 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f617005a1d0 con 0x7f617c0cfd70 2024-04-14T08:33:55.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.106+0000 7f6180d1e640 1 -- 172.21.15.172:0/3304883228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f617c0d17e0 con 0x7f617c0cfd70 2024-04-14T08:33:55.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.109+0000 7f616affd640 1 -- 172.21.15.172:0/3304883228 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f6170059ad0 con 0x7f617c0cfd70 2024-04-14T08:33:55.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.111+0000 7f6180d1e640 1 -- 172.21.15.172:0/3304883228 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f615c06c8f0 msgr2=0x7f615c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.111+0000 7f6180d1e640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f615c06c8f0 0x7f615c06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f616c036000 tx=0x7f616c03d040 comp rx=0 tx=0).stop 2024-04-14T08:33:55.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.111+0000 7f6180d1e640 1 -- 172.21.15.172:0/3304883228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617c0cfd70 msgr2=0x7f617c0cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.111+0000 7f6180d1e640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617c0cfd70 0x7f617c0cb300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f617000e870 tx=0x7f617000ed30 comp rx=0 tx=0).stop 2024-04-14T08:33:55.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.111+0000 7f6180d1e640 1 -- 172.21.15.172:0/3304883228 shutdown_connections 2024-04-14T08:33:55.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.111+0000 7f6180d1e640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f615c06c8f0 0x7f615c06edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.111+0000 7f6180d1e640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f617c15e4b0 0x7f617c0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.111+0000 7f6180d1e640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f617c0d0fb0 0x7f617c156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.111+0000 7f6180d1e640 1 --2- 172.21.15.172:0/3304883228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f617c0cfd70 0x7f617c0cb300 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.111+0000 7f6180d1e640 1 -- 172.21.15.172:0/3304883228 >> 172.21.15.172:0/3304883228 conn(0x7f617c09d570 msgr2=0x7f617c0b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:55.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.111+0000 7f6180d1e640 1 -- 172.21.15.172:0/3304883228 shutdown_connections 2024-04-14T08:33:55.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.111+0000 7f6180d1e640 1 -- 172.21.15.172:0/3304883228 wait complete. 2024-04-14T08:33:55.123 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-14T08:33:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.224+0000 7f7b14a2d640 1 Processor -- start 2024-04-14T08:33:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.224+0000 7f7b14a2d640 1 -- start start 2024-04-14T08:33:55.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.225+0000 7f7b14a2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1015e820 0x7f7b100b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.225+0000 7f7b14a2d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7b100d0d30 0x7f7b100cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.225+0000 7f7b14a2d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b100d12e0 con 0x7f7b1006d040 2024-04-14T08:33:55.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.225+0000 7f7b14a2d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f7b1006cb90 con 0x7f7b100d0d30 2024-04-14T08:33:55.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.225+0000 7f7b14a2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7b100cc660 con 0x7f7b1015e820 2024-04-14T08:33:55.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.225+0000 7f7b0e59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7b1006d040 0x7f7b1006d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35008/0 (socket says 172.21.15.172:35008) 2024-04-14T08:33:55.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.225+0000 7f7b0e59c640 1 -- 172.21.15.172:0/8291784 learned_addr learned my addr 172.21.15.172:0/8291784 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:55.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3987086146 0 0) 0x7f7b100d12e0 con 0x7f7b1006d040 2024-04-14T08:33:55.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7af8003520 con 0x7f7b1006d040 2024-04-14T08:33:55.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0ed9d640 1 --2- 172.21.15.172:0/8291784 >> v2:172.21.15.177:3300/0 conn(0x7f7b100d0d30 0x7f7b100cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0dd9b640 1 --2- 172.21.15.172:0/8291784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1015e820 0x7f7b100b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3222477693 0 0) 0x7f7af8003520 con 0x7f7b1006d040 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1015e820 msgr2=0x7f7b100b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 --2- 172.21.15.172:0/8291784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1015e820 0x7f7b100b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 >> v2:172.21.15.177:3300/0 conn(0x7f7b100d0d30 msgr2=0x7f7b100cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 --2- 172.21.15.172:0/8291784 >> v2:172.21.15.177:3300/0 conn(0x7f7b100d0d30 0x7f7b100cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b100d1580 con 0x7f7b1006d040 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f7b000019f0 con 0x7f7b1006d040 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 >> v1:172.21.15.172:6789/0 conn(0x7f7b1006d040 legacy=0x7f7b1006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 --2- 172.21.15.172:0/8291784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7af80039b0 0x7f7af8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 --2- 172.21.15.172:0/8291784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1015e820 0x7f7af8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 --2- 172.21.15.172:0/8291784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af8004d40 0x7f7af80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b100cc660 con 0x7f7af8004d40 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7af800b9c0 con 0x7f7af80039b0 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7af800bb40 con 0x7f7b1015e820 2024-04-14T08:33:55.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0ed9d640 1 --2- 172.21.15.172:0/8291784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af8004d40 0x7f7af80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0e59c640 1 --2- 172.21.15.172:0/8291784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7af80039b0 0x7f7af8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0dd9b640 1 --2- 172.21.15.172:0/8291784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1015e820 0x7f7af8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0ed9d640 1 -- 172.21.15.172:0/8291784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1015e820 msgr2=0x7f7af8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0ed9d640 1 --2- 172.21.15.172:0/8291784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1015e820 0x7f7af8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0ed9d640 1 -- 172.21.15.172:0/8291784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7af80039b0 msgr2=0x7f7af8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0ed9d640 1 --2- 172.21.15.172:0/8291784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7af80039b0 0x7f7af8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.226+0000 7f7b0ed9d640 1 -- 172.21.15.172:0/8291784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af8003520 con 0x7f7af8004d40 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.227+0000 7f7b0ed9d640 1 --2- 172.21.15.172:0/8291784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af8004d40 0x7f7af80051c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009ac0 tx=0x7f7af8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785c52b2890302cd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.227+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08039070 con 0x7f7af8004d40 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.227+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0802c710 con 0x7f7af8004d40 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.227+0000 7f7b0d59a640 1 -- 172.21.15.172:0/8291784 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08009690 con 0x7f7af8004d40 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.228+0000 7f7b14a2d640 1 -- 172.21.15.172:0/8291784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af8004d40 msgr2=0x7f7af80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.228+0000 7f7b14a2d640 1 --2- 172.21.15.172:0/8291784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af8004d40 0x7f7af80051c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009ac0 tx=0x7f7af8005830 comp rx=0 tx=0).stop 2024-04-14T08:33:55.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.228+0000 7f7b0ed9d640 1 -- 172.21.15.172:0/8291784 reap_dead start 2024-04-14T08:33:55.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.228+0000 7f7b14a2d640 1 -- 172.21.15.172:0/8291784 shutdown_connections 2024-04-14T08:33:55.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.228+0000 7f7b14a2d640 1 -- 172.21.15.172:0/8291784 >> 172.21.15.172:0/8291784 conn(0x7f7b100b3b00 msgr2=0x7f7b100b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:55.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.228+0000 7f7b14a2d640 1 -- 172.21.15.172:0/8291784 shutdown_connections 2024-04-14T08:33:55.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.228+0000 7f7b14a2d640 1 -- 172.21.15.172:0/8291784 wait complete. 2024-04-14T08:33:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.229+0000 7f7b14a2d640 1 Processor -- start 2024-04-14T08:33:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.229+0000 7f7b14a2d640 1 -- start start 2024-04-14T08:33:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b14a2d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af8004d40 0x7f7b101f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b14a2d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b1006fac0 0x7f7b1006ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b14a2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b100c9380 0x7f7b100c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b14a2d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b100b4620 con 0x7f7af8004d40 2024-04-14T08:33:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b14a2d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7b100ccc20 con 0x7f7b1006fac0 2024-04-14T08:33:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b14a2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7b100b47a0 con 0x7f7b100c9380 2024-04-14T08:33:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b0e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af8004d40 0x7f7b101f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b0e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af8004d40 0x7f7b101f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57322/0 (socket says 172.21.15.172:57322) 2024-04-14T08:33:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b0e59c640 1 -- 172.21.15.172:0/4142333491 learned_addr learned my addr 172.21.15.172:0/4142333491 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b0ed9d640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b100c9380 0x7f7b100c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b0dd9b640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b1006fac0 0x7f7b1006ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b0e59c640 1 -- 172.21.15.172:0/4142333491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b100c9380 msgr2=0x7f7b100c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b0e59c640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b100c9380 0x7f7b100c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b0e59c640 1 -- 172.21.15.172:0/4142333491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b1006fac0 msgr2=0x7f7b1006ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b0e59c640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b1006fac0 0x7f7b1006ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b0e59c640 1 -- 172.21.15.172:0/4142333491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b100b6360 con 0x7f7af8004d40 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.230+0000 7f7b0e59c640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af8004d40 0x7f7b101f6ff0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7b00007a50 tx=0x7f7b0000a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.231+0000 7f7afeffd640 1 -- 172.21.15.172:0/4142333491 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00017020 con 0x7f7af8004d40 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.231+0000 7f7afeffd640 1 -- 172.21.15.172:0/4142333491 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0000ae40 con 0x7f7af8004d40 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.231+0000 7f7afeffd640 1 -- 172.21.15.172:0/4142333491 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00013840 con 0x7f7af8004d40 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.231+0000 7f7b14a2d640 1 -- 172.21.15.172:0/4142333491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b100b6640 con 0x7f7af8004d40 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.231+0000 7f7b14a2d640 1 -- 172.21.15.172:0/4142333491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b100b6c00 con 0x7f7af8004d40 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.232+0000 7f7b14a2d640 1 -- 172.21.15.172:0/4142333491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b100b6e00 con 0x7f7af8004d40 2024-04-14T08:33:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.232+0000 7f7afeffd640 1 -- 172.21.15.172:0/4142333491 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b0001a020 con 0x7f7af8004d40 2024-04-14T08:33:55.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.233+0000 7f7afeffd640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7af006c8f0 0x7f7af006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.233+0000 7f7afeffd640 1 -- 172.21.15.172:0/4142333491 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f7b0008c6c0 con 0x7f7af8004d40 2024-04-14T08:33:55.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.233+0000 7f7b0dd9b640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7af006c8f0 0x7f7af006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.233+0000 7f7b0dd9b640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7af006c8f0 0x7f7af006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7b100cb1c0 tx=0x7f7b04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:55.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.237+0000 7f7afeffd640 1 -- 172.21.15.172:0/4142333491 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7b100b6e00 con 0x7f7af8004d40 2024-04-14T08:33:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.478+0000 7f7b14a2d640 1 -- 172.21.15.172:0/4142333491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f7b10068590 con 0x7f7af8004d40 2024-04-14T08:33:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.481+0000 7f7afeffd640 1 -- 172.21.15.172:0/4142333491 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f7b00021020 con 0x7f7af8004d40 2024-04-14T08:33:55.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.483+0000 7f7b14a2d640 1 -- 172.21.15.172:0/4142333491 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7af006c8f0 msgr2=0x7f7af006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.483+0000 7f7b14a2d640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7af006c8f0 0x7f7af006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7b100cb1c0 tx=0x7f7b04008040 comp rx=0 tx=0).stop 2024-04-14T08:33:55.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.483+0000 7f7b14a2d640 1 -- 172.21.15.172:0/4142333491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af8004d40 msgr2=0x7f7b101f6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.483+0000 7f7b14a2d640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af8004d40 0x7f7b101f6ff0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7b00007a50 tx=0x7f7b0000a790 comp rx=0 tx=0).stop 2024-04-14T08:33:55.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.483+0000 7f7b14a2d640 1 -- 172.21.15.172:0/4142333491 shutdown_connections 2024-04-14T08:33:55.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.483+0000 7f7b14a2d640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7af006c8f0 0x7f7af006edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.483+0000 7f7b14a2d640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b100c9380 0x7f7b100c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.483+0000 7f7b14a2d640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b1006fac0 0x7f7b1006ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.483+0000 7f7b14a2d640 1 --2- 172.21.15.172:0/4142333491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7af8004d40 0x7f7b101f6ff0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.483+0000 7f7b14a2d640 1 -- 172.21.15.172:0/4142333491 >> 172.21.15.172:0/4142333491 conn(0x7f7b100b3b00 msgr2=0x7f7b100cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:55.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.483+0000 7f7b14a2d640 1 -- 172.21.15.172:0/4142333491 shutdown_connections 2024-04-14T08:33:55.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.483+0000 7f7b14a2d640 1 -- 172.21.15.172:0/4142333491 wait complete. 2024-04-14T08:33:55.495 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-14T08:33:55.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.597+0000 7f33c4c76640 1 Processor -- start 2024-04-14T08:33:55.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.597+0000 7f33c4c76640 1 -- start start 2024-04-14T08:33:55.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.598+0000 7f33c4c76640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33c00ccca0 0x7f33c006fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.598+0000 7f33c4c76640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f33c0073090 0x7f33c00b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.598+0000 7f33c4c76640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f33c006b520 0x7f33c006b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.598+0000 7f33c4c76640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f33c006be90 con 0x7f33c006b520 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.598+0000 7f33c4c76640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f33c00734f0 con 0x7f33c0073090 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.598+0000 7f33c4c76640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f33c00ceee0 con 0x7f33c00ccca0 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.598+0000 7f33bed9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f33c006b520 0x7f33c006b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.598+0000 7f33bed9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f33c006b520 0x7f33c006b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57324/0 (socket says 172.21.15.172:57324) 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.598+0000 7f33bed9d640 1 -- 172.21.15.172:0/703487388 learned_addr learned my addr 172.21.15.172:0/703487388 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.599+0000 7f33bdd9b640 1 --2- 172.21.15.172:0/703487388 >> v2:172.21.15.177:3300/0 conn(0x7f33c0073090 0x7f33c00b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.598+0000 7f33be59c640 1 --2- 172.21.15.172:0/703487388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33c00ccca0 0x7f33c006fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.599+0000 7f33bed9d640 1 -- 172.21.15.172:0/703487388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33c00ccca0 msgr2=0x7f33c006fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.599+0000 7f33bed9d640 1 --2- 172.21.15.172:0/703487388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33c00ccca0 0x7f33c006fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.599+0000 7f33bed9d640 1 -- 172.21.15.172:0/703487388 >> v2:172.21.15.177:3300/0 conn(0x7f33c0073090 msgr2=0x7f33c00b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.599+0000 7f33bed9d640 1 --2- 172.21.15.172:0/703487388 >> v2:172.21.15.177:3300/0 conn(0x7f33c0073090 0x7f33c00b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.599+0000 7f33bed9d640 1 -- 172.21.15.172:0/703487388 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c0070150 con 0x7f33c006b520 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.599+0000 7f33bed9d640 1 --2- 172.21.15.172:0/703487388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c006b520 0x7f33c006b950 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f33c0073fd0 tx=0x7f33b802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42af93b61f581f9c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.600+0000 7f33bd59a640 1 -- 172.21.15.172:0/703487388 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b802c9e0 con 0x7f33c006b520 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.600+0000 7f33bd59a640 1 -- 172.21.15.172:0/703487388 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b802cb80 con 0x7f33c006b520 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.600+0000 7f33bd59a640 1 -- 172.21.15.172:0/703487388 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b80343f0 con 0x7f33c006b520 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.600+0000 7f33c4c76640 1 -- 172.21.15.172:0/703487388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c006b520 msgr2=0x7f33c006b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.600+0000 7f33c4c76640 1 --2- 172.21.15.172:0/703487388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c006b520 0x7f33c006b950 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f33c0073fd0 tx=0x7f33b802be40 comp rx=0 tx=0).stop 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.600+0000 7f33c4c76640 1 -- 172.21.15.172:0/703487388 shutdown_connections 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.600+0000 7f33c4c76640 1 --2- 172.21.15.172:0/703487388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c006b520 0x7f33c006b950 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.600+0000 7f33c4c76640 1 --2- 172.21.15.172:0/703487388 >> v2:172.21.15.177:3300/0 conn(0x7f33c0073090 0x7f33c00b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.600+0000 7f33c4c76640 1 --2- 172.21.15.172:0/703487388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33c00ccca0 0x7f33c006fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.600+0000 7f33c4c76640 1 -- 172.21.15.172:0/703487388 >> 172.21.15.172:0/703487388 conn(0x7f33c00cda00 msgr2=0x7f33c00cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:55.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.600+0000 7f33c4c76640 1 -- 172.21.15.172:0/703487388 shutdown_connections 2024-04-14T08:33:55.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.600+0000 7f33c4c76640 1 -- 172.21.15.172:0/703487388 wait complete. 2024-04-14T08:33:55.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.601+0000 7f33c4c76640 1 Processor -- start 2024-04-14T08:33:55.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.602+0000 7f33c4c76640 1 -- start start 2024-04-14T08:33:55.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.602+0000 7f33c4c76640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33c006b520 0x7f33c00cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.602+0000 7f33c4c76640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33c0073090 0x7f33c0154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.602+0000 7f33c4c76640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c00ccca0 0x7f33c00ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.602+0000 7f33c4c76640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0072890 con 0x7f33c00ccca0 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.602+0000 7f33c4c76640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f33c00725b0 con 0x7f33c0073090 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.602+0000 7f33c4c76640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f33c00cd080 con 0x7f33c006b520 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.602+0000 7f33bed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c00ccca0 0x7f33c00ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.602+0000 7f33bed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c00ccca0 0x7f33c00ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57332/0 (socket says 172.21.15.172:57332) 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.602+0000 7f33bed9d640 1 -- 172.21.15.172:0/3585083192 learned_addr learned my addr 172.21.15.172:0/3585083192 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.602+0000 7f33bdd9b640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33c0073090 0x7f33c0154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.602+0000 7f33be59c640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33c006b520 0x7f33c00cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.603+0000 7f33bed9d640 1 -- 172.21.15.172:0/3585083192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33c006b520 msgr2=0x7f33c00cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.603+0000 7f33bed9d640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33c006b520 0x7f33c00cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.603+0000 7f33bed9d640 1 -- 172.21.15.172:0/3585083192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33c0073090 msgr2=0x7f33c0154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.603+0000 7f33bed9d640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33c0073090 0x7f33c0154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.603+0000 7f33bed9d640 1 -- 172.21.15.172:0/3585083192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c0154990 con 0x7f33c00ccca0 2024-04-14T08:33:55.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.603+0000 7f33bed9d640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c00ccca0 0x7f33c00ade20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009e50 tx=0x7f33b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:55.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.603+0000 7f33aeffd640 1 -- 172.21.15.172:0/3585083192 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8043020 con 0x7f33c00ccca0 2024-04-14T08:33:55.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.603+0000 7f33aeffd640 1 -- 172.21.15.172:0/3585083192 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b803e450 con 0x7f33c00ccca0 2024-04-14T08:33:55.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.603+0000 7f33aeffd640 1 -- 172.21.15.172:0/3585083192 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8009510 con 0x7f33c00ccca0 2024-04-14T08:33:55.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.603+0000 7f33c4c76640 1 -- 172.21.15.172:0/3585083192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c00c9eb0 con 0x7f33c00ccca0 2024-04-14T08:33:55.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.603+0000 7f33c4c76640 1 -- 172.21.15.172:0/3585083192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c00ca470 con 0x7f33c00ccca0 2024-04-14T08:33:55.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.604+0000 7f33c4c76640 1 -- 172.21.15.172:0/3585083192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c0068590 con 0x7f33c00ccca0 2024-04-14T08:33:55.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.604+0000 7f33aeffd640 1 -- 172.21.15.172:0/3585083192 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33b8046020 con 0x7f33c00ccca0 2024-04-14T08:33:55.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.605+0000 7f33aeffd640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33a006c8f0 0x7f33a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.605+0000 7f33aeffd640 1 -- 172.21.15.172:0/3585083192 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f33b80ae030 con 0x7f33c00ccca0 2024-04-14T08:33:55.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.605+0000 7f33be59c640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33a006c8f0 0x7f33a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.605+0000 7f33be59c640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33a006c8f0 0x7f33a006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f33c006fed0 tx=0x7f33b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:55.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.609+0000 7f33aeffd640 1 -- 172.21.15.172:0/3585083192 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f33b804d030 con 0x7f33c00ccca0 2024-04-14T08:33:55.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.853+0000 7f33c4c76640 1 -- 172.21.15.172:0/3585083192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f33c006c120 con 0x7f33c00ccca0 2024-04-14T08:33:55.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.855+0000 7f33aeffd640 1 -- 172.21.15.172:0/3585083192 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f33c0154990 con 0x7f33c00ccca0 2024-04-14T08:33:55.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.858+0000 7f33c4c76640 1 -- 172.21.15.172:0/3585083192 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33a006c8f0 msgr2=0x7f33a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.858+0000 7f33c4c76640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33a006c8f0 0x7f33a006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f33c006fed0 tx=0x7f33b0008040 comp rx=0 tx=0).stop 2024-04-14T08:33:55.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.858+0000 7f33c4c76640 1 -- 172.21.15.172:0/3585083192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c00ccca0 msgr2=0x7f33c00ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.858+0000 7f33c4c76640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c00ccca0 0x7f33c00ade20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009e50 tx=0x7f33b8004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:55.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.858+0000 7f33c4c76640 1 -- 172.21.15.172:0/3585083192 shutdown_connections 2024-04-14T08:33:55.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.858+0000 7f33c4c76640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33a006c8f0 0x7f33a006edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.858+0000 7f33c4c76640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c00ccca0 0x7f33c00ade20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.858+0000 7f33c4c76640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33c0073090 0x7f33c0154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.858+0000 7f33c4c76640 1 --2- 172.21.15.172:0/3585083192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f33c006b520 0x7f33c00cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.858+0000 7f33c4c76640 1 -- 172.21.15.172:0/3585083192 >> 172.21.15.172:0/3585083192 conn(0x7f33c00cda00 msgr2=0x7f33c00b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:55.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.858+0000 7f33c4c76640 1 -- 172.21.15.172:0/3585083192 shutdown_connections 2024-04-14T08:33:55.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.858+0000 7f33c4c76640 1 -- 172.21.15.172:0/3585083192 wait complete. 2024-04-14T08:33:55.870 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-14T08:33:55.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.971+0000 7f4fdebed640 1 Processor -- start 2024-04-14T08:33:55.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.971+0000 7f4fdebed640 1 -- start start 2024-04-14T08:33:55.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.972+0000 7f4fdebed640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fd815e430 0x7f4fd8162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.972+0000 7f4fdebed640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4fd8162540 0x7f4fd81629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.972+0000 7f4fdebed640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd80cefb0 con 0x7f4fd80b3a30 2024-04-14T08:33:55.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.972+0000 7f4fdebed640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd80cf290 con 0x7f4fd8162540 2024-04-14T08:33:55.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.972+0000 7f4fdebed640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd80cf570 con 0x7f4fd815e430 2024-04-14T08:33:55.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.972+0000 7f4fd7fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4fd8162540 0x7f4fd81629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.972+0000 7f4fdd18a640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4fd80b3a30 0x7f4fd80b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35020/0 (socket says 172.21.15.172:35020) 2024-04-14T08:33:55.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.972+0000 7f4fdd18a640 1 -- 172.21.15.172:0/1853889348 learned_addr learned my addr 172.21.15.172:0/1853889348 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:55.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.972+0000 7f4fdc989640 1 --2- 172.21.15.172:0/1853889348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fd815e430 0x7f4fd8162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.973+0000 7f4fd7fff640 1 -- 172.21.15.172:0/1853889348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fd815e430 msgr2=0x7f4fd8162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:55.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.973+0000 7f4fd7fff640 1 --2- 172.21.15.172:0/1853889348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fd815e430 0x7f4fd8162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.973+0000 7f4fd7fff640 1 -- 172.21.15.172:0/1853889348 >> v1:172.21.15.172:6789/0 conn(0x7f4fd80b3a30 legacy=0x7f4fd80b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.973+0000 7f4fd7fff640 1 -- 172.21.15.172:0/1853889348 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd80cf850 con 0x7f4fd8162540 2024-04-14T08:33:55.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.973+0000 7f4fd7fff640 1 --2- 172.21.15.172:0/1853889348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd8162540 0x7f4fd81629a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40097a0 tx=0x7f4fc402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a2247a8d8d0b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:55.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.974+0000 7f4fd77fe640 1 -- 172.21.15.172:0/1853889348 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4035020 con 0x7f4fd8162540 2024-04-14T08:33:55.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.974+0000 7f4fd77fe640 1 -- 172.21.15.172:0/1853889348 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc4031830 con 0x7f4fd8162540 2024-04-14T08:33:55.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.974+0000 7f4fd77fe640 1 -- 172.21.15.172:0/1853889348 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4031c00 con 0x7f4fd8162540 2024-04-14T08:33:55.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.974+0000 7f4fdebed640 1 -- 172.21.15.172:0/1853889348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd8162540 msgr2=0x7f4fd81629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.974+0000 7f4fdebed640 1 --2- 172.21.15.172:0/1853889348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd8162540 0x7f4fd81629a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40097a0 tx=0x7f4fc402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.974+0000 7f4fdebed640 1 -- 172.21.15.172:0/1853889348 shutdown_connections 2024-04-14T08:33:55.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.974+0000 7f4fdebed640 1 --2- 172.21.15.172:0/1853889348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd8162540 0x7f4fd81629a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.974+0000 7f4fdebed640 1 --2- 172.21.15.172:0/1853889348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fd815e430 0x7f4fd8162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.974+0000 7f4fdebed640 1 -- 172.21.15.172:0/1853889348 >> 172.21.15.172:0/1853889348 conn(0x7f4fd809d560 msgr2=0x7f4fd80d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:55.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.974+0000 7f4fdebed640 1 -- 172.21.15.172:0/1853889348 shutdown_connections 2024-04-14T08:33:55.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.974+0000 7f4fdebed640 1 -- 172.21.15.172:0/1853889348 wait complete. 2024-04-14T08:33:55.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.975+0000 7f4fdebed640 1 Processor -- start 2024-04-14T08:33:55.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.975+0000 7f4fdebed640 1 -- start start 2024-04-14T08:33:55.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.976+0000 7f4fdebed640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fd80b3a30 0x7f4fd80b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.976+0000 7f4fdebed640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fd815e430 0x7f4fd814f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.976+0000 7f4fdebed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd8162540 0x7f4fd80cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.976+0000 7f4fdebed640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd80d4e20 con 0x7f4fd8162540 2024-04-14T08:33:55.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.976+0000 7f4fdebed640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd815e810 con 0x7f4fd815e430 2024-04-14T08:33:55.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.976+0000 7f4fdebed640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd80d0050 con 0x7f4fd80b3a30 2024-04-14T08:33:55.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.976+0000 7f4fdd18a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd8162540 0x7f4fd80cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.976+0000 7f4fdd18a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd8162540 0x7f4fd80cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57350/0 (socket says 172.21.15.172:57350) 2024-04-14T08:33:55.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.976+0000 7f4fdd18a640 1 -- 172.21.15.172:0/1665159354 learned_addr learned my addr 172.21.15.172:0/1665159354 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:55.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fdc989640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fd80b3a30 0x7f4fd80b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fd7fff640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fd815e430 0x7f4fd814f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fdd18a640 1 -- 172.21.15.172:0/1665159354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fd80b3a30 msgr2=0x7f4fd80b99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fdd18a640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fd80b3a30 0x7f4fd80b99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fdd18a640 1 -- 172.21.15.172:0/1665159354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fd815e430 msgr2=0x7f4fd814f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fdd18a640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fd815e430 0x7f4fd814f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fdd18a640 1 -- 172.21.15.172:0/1665159354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd80cccf0 con 0x7f4fd8162540 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fdc989640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fd80b3a30 0x7f4fd80b99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fd7fff640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fd815e430 0x7f4fd814f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fdd18a640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd8162540 0x7f4fd80cc580 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000cbc0 tx=0x7f4fd000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fd57fa640 1 -- 172.21.15.172:0/1665159354 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0017070 con 0x7f4fd8162540 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fd57fa640 1 -- 172.21.15.172:0/1665159354 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd000ad40 con 0x7f4fd8162540 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fdebed640 1 -- 172.21.15.172:0/1665159354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd80ccfd0 con 0x7f4fd8162540 2024-04-14T08:33:55.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fd57fa640 1 -- 172.21.15.172:0/1665159354 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0012600 con 0x7f4fd8162540 2024-04-14T08:33:55.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.977+0000 7f4fdebed640 1 -- 172.21.15.172:0/1665159354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd80cd590 con 0x7f4fd8162540 2024-04-14T08:33:55.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.978+0000 7f4fdebed640 1 -- 172.21.15.172:0/1665159354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd8068590 con 0x7f4fd8162540 2024-04-14T08:33:55.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.979+0000 7f4fd57fa640 1 -- 172.21.15.172:0/1665159354 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fd001a070 con 0x7f4fd8162540 2024-04-14T08:33:55.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.979+0000 7f4fd57fa640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4fbc06c8f0 0x7f4fbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:55.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.979+0000 7f4fd57fa640 1 -- 172.21.15.172:0/1665159354 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f4fd008b6d0 con 0x7f4fd8162540 2024-04-14T08:33:55.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.979+0000 7f4fdc989640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4fbc06c8f0 0x7f4fbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:55.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.979+0000 7f4fdc989640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4fbc06c8f0 0x7f4fbc06edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009b90 tx=0x7f4fcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:55.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:55.983+0000 7f4fd57fa640 1 -- 172.21.15.172:0/1665159354 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4fd00595e0 con 0x7f4fd8162540 2024-04-14T08:33:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.226+0000 7f4fdebed640 1 -- 172.21.15.172:0/1665159354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f4fd80d18a0 con 0x7f4fd8162540 2024-04-14T08:33:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.228+0000 7f4fd57fa640 1 -- 172.21.15.172:0/1665159354 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f4fd005d8e0 con 0x7f4fd8162540 2024-04-14T08:33:56.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.231+0000 7f4fdebed640 1 -- 172.21.15.172:0/1665159354 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4fbc06c8f0 msgr2=0x7f4fbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.231+0000 7f4fdebed640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4fbc06c8f0 0x7f4fbc06edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009b90 tx=0x7f4fcc008040 comp rx=0 tx=0).stop 2024-04-14T08:33:56.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.231+0000 7f4fdebed640 1 -- 172.21.15.172:0/1665159354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd8162540 msgr2=0x7f4fd80cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.231+0000 7f4fdebed640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd8162540 0x7f4fd80cc580 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000cbc0 tx=0x7f4fd000a430 comp rx=0 tx=0).stop 2024-04-14T08:33:56.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.231+0000 7f4fdebed640 1 -- 172.21.15.172:0/1665159354 shutdown_connections 2024-04-14T08:33:56.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.231+0000 7f4fdebed640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4fbc06c8f0 0x7f4fbc06edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.231+0000 7f4fdebed640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fd8162540 0x7f4fd80cc580 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.231+0000 7f4fdebed640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fd815e430 0x7f4fd814f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.231+0000 7f4fdebed640 1 --2- 172.21.15.172:0/1665159354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fd80b3a30 0x7f4fd80b99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.231+0000 7f4fdebed640 1 -- 172.21.15.172:0/1665159354 >> 172.21.15.172:0/1665159354 conn(0x7f4fd809d560 msgr2=0x7f4fd80d27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:56.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.231+0000 7f4fdebed640 1 -- 172.21.15.172:0/1665159354 shutdown_connections 2024-04-14T08:33:56.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.231+0000 7f4fdebed640 1 -- 172.21.15.172:0/1665159354 wait complete. 2024-04-14T08:33:56.243 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-14T08:33:56.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.344+0000 7fa0b11dc640 1 Processor -- start 2024-04-14T08:33:56.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.344+0000 7fa0b11dc640 1 -- start start 2024-04-14T08:33:56.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0b11dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ac13df30 0x7fa0ac0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:56.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0b11dc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0ac15c380 0x7fa0ac0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0b11dc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa0ac140770 0x7fa0ac140c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0b11dc640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa0ac15b620 con 0x7fa0ac15c380 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0b11dc640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa0ac1414a0 con 0x7fa0ac140770 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0b11dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ac15b8a0 con 0x7fa0ac13df30 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0aa59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0ac15c380 0x7fa0ac0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0aa59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0ac15c380 0x7fa0ac0b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57360/0 (socket says 172.21.15.172:57360) 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0aa59c640 1 -- 172.21.15.172:0/2381348880 learned_addr learned my addr 172.21.15.172:0/2381348880 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0aad9d640 1 --2- 172.21.15.172:0/2381348880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ac13df30 0x7fa0ac0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0ab59e640 1 --2- 172.21.15.172:0/2381348880 >> v2:172.21.15.177:3300/0 conn(0x7fa0ac140770 0x7fa0ac140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0aa59c640 1 -- 172.21.15.172:0/2381348880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ac13df30 msgr2=0x7fa0ac0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0aa59c640 1 --2- 172.21.15.172:0/2381348880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ac13df30 0x7fa0ac0b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0aa59c640 1 -- 172.21.15.172:0/2381348880 >> v2:172.21.15.177:3300/0 conn(0x7fa0ac140770 msgr2=0x7fa0ac140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0aa59c640 1 --2- 172.21.15.172:0/2381348880 >> v2:172.21.15.177:3300/0 conn(0x7fa0ac140770 0x7fa0ac140c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.345+0000 7fa0aa59c640 1 -- 172.21.15.172:0/2381348880 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ac15bb80 con 0x7fa0ac15c380 2024-04-14T08:33:56.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.346+0000 7fa0aa59c640 1 --2- 172.21.15.172:0/2381348880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ac15c380 0x7fa0ac0b2210 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000bb10 tx=0x7fa0a002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d452e6f4cc298c1a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:56.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.346+0000 7fa0a9d9b640 1 -- 172.21.15.172:0/2381348880 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a003b070 con 0x7fa0ac15c380 2024-04-14T08:33:56.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.346+0000 7fa0a9d9b640 1 -- 172.21.15.172:0/2381348880 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a0036440 con 0x7fa0ac15c380 2024-04-14T08:33:56.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.346+0000 7fa0a9d9b640 1 -- 172.21.15.172:0/2381348880 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0036810 con 0x7fa0ac15c380 2024-04-14T08:33:56.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.347+0000 7fa0b11dc640 1 -- 172.21.15.172:0/2381348880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ac15c380 msgr2=0x7fa0ac0b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.347+0000 7fa0b11dc640 1 --2- 172.21.15.172:0/2381348880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ac15c380 0x7fa0ac0b2210 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000bb10 tx=0x7fa0a002e0c0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.347+0000 7fa0b11dc640 1 -- 172.21.15.172:0/2381348880 shutdown_connections 2024-04-14T08:33:56.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.347+0000 7fa0b11dc640 1 --2- 172.21.15.172:0/2381348880 >> v2:172.21.15.177:3300/0 conn(0x7fa0ac140770 0x7fa0ac140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.347+0000 7fa0b11dc640 1 --2- 172.21.15.172:0/2381348880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ac15c380 0x7fa0ac0b2210 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.347+0000 7fa0b11dc640 1 --2- 172.21.15.172:0/2381348880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ac13df30 0x7fa0ac0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.347+0000 7fa0b11dc640 1 -- 172.21.15.172:0/2381348880 >> 172.21.15.172:0/2381348880 conn(0x7fa0ac06b520 msgr2=0x7fa0ac06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:56.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.347+0000 7fa0b11dc640 1 -- 172.21.15.172:0/2381348880 shutdown_connections 2024-04-14T08:33:56.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.347+0000 7fa0b11dc640 1 -- 172.21.15.172:0/2381348880 wait complete. 2024-04-14T08:33:56.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.348+0000 7fa0b11dc640 1 Processor -- start 2024-04-14T08:33:56.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.348+0000 7fa0b11dc640 1 -- start start 2024-04-14T08:33:56.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0b11dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ac13df30 0x7fa0ac153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:56.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0b11dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ac140770 0x7fa0ac1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:56.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0b11dc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ac15c380 0x7fa0ac154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:56.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0b11dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac141140 con 0x7fa0ac13df30 2024-04-14T08:33:56.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0b11dc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac13e310 con 0x7fa0ac15c380 2024-04-14T08:33:56.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0b11dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ac0ce9a0 con 0x7fa0ac140770 2024-04-14T08:33:56.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0aad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ac13df30 0x7fa0ac153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:56.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0aad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ac13df30 0x7fa0ac153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57374/0 (socket says 172.21.15.172:57374) 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0aad9d640 1 -- 172.21.15.172:0/327935804 learned_addr learned my addr 172.21.15.172:0/327935804 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0aa59c640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ac140770 0x7fa0ac1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0ab59e640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ac15c380 0x7fa0ac154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0aad9d640 1 -- 172.21.15.172:0/327935804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ac140770 msgr2=0x7fa0ac1579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0aad9d640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ac140770 0x7fa0ac1579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0aad9d640 1 -- 172.21.15.172:0/327935804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ac15c380 msgr2=0x7fa0ac154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0aad9d640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ac15c380 0x7fa0ac154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0aad9d640 1 -- 172.21.15.172:0/327935804 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ac1580c0 con 0x7fa0ac13df30 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.349+0000 7fa0aad9d640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ac13df30 0x7fa0ac153380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa09c007ef0 tx=0x7fa09c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.350+0000 7fa0937fe640 1 -- 172.21.15.172:0/327935804 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c014620 con 0x7fa0ac13df30 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.350+0000 7fa0937fe640 1 -- 172.21.15.172:0/327935804 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa09c019070 con 0x7fa0ac13df30 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.350+0000 7fa0b11dc640 1 -- 172.21.15.172:0/327935804 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ac1583a0 con 0x7fa0ac13df30 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.350+0000 7fa0937fe640 1 -- 172.21.15.172:0/327935804 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c014a00 con 0x7fa0ac13df30 2024-04-14T08:33:56.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.350+0000 7fa0b11dc640 1 -- 172.21.15.172:0/327935804 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ac0afc70 con 0x7fa0ac13df30 2024-04-14T08:33:56.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.351+0000 7fa0b11dc640 1 -- 172.21.15.172:0/327935804 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ac068590 con 0x7fa0ac13df30 2024-04-14T08:33:56.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.351+0000 7fa0937fe640 1 -- 172.21.15.172:0/327935804 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa09c014ba0 con 0x7fa0ac13df30 2024-04-14T08:33:56.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.351+0000 7fa0937fe640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa09806c8f0 0x7fa09806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:56.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.351+0000 7fa0937fe640 1 -- 172.21.15.172:0/327935804 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fa09c08c100 con 0x7fa0ac13df30 2024-04-14T08:33:56.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.351+0000 7fa0aa59c640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa09806c8f0 0x7fa09806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:56.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.352+0000 7fa0aa59c640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa09806c8f0 0x7fa09806edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0038000 tx=0x7fa0a0032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.356+0000 7fa0937fe640 1 -- 172.21.15.172:0/327935804 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa09c05a130 con 0x7fa0ac13df30 2024-04-14T08:33:56.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.598+0000 7fa0b11dc640 1 -- 172.21.15.172:0/327935804 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fa0ac0b38b0 con 0x7fa0ac13df30 2024-04-14T08:33:56.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.600+0000 7fa0937fe640 1 -- 172.21.15.172:0/327935804 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fa09c05e430 con 0x7fa0ac13df30 2024-04-14T08:33:56.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.603+0000 7fa0b11dc640 1 -- 172.21.15.172:0/327935804 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa09806c8f0 msgr2=0x7fa09806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.603+0000 7fa0b11dc640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa09806c8f0 0x7fa09806edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0038000 tx=0x7fa0a0032000 comp rx=0 tx=0).stop 2024-04-14T08:33:56.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.603+0000 7fa0b11dc640 1 -- 172.21.15.172:0/327935804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ac13df30 msgr2=0x7fa0ac153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.603+0000 7fa0b11dc640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ac13df30 0x7fa0ac153380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa09c007ef0 tx=0x7fa09c00e4b0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.603+0000 7fa0b11dc640 1 -- 172.21.15.172:0/327935804 shutdown_connections 2024-04-14T08:33:56.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.603+0000 7fa0b11dc640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa09806c8f0 0x7fa09806edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.603+0000 7fa0b11dc640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ac15c380 0x7fa0ac154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.603+0000 7fa0b11dc640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ac140770 0x7fa0ac1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.603+0000 7fa0b11dc640 1 --2- 172.21.15.172:0/327935804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ac13df30 0x7fa0ac153380 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.603+0000 7fa0b11dc640 1 -- 172.21.15.172:0/327935804 >> 172.21.15.172:0/327935804 conn(0x7fa0ac06b520 msgr2=0x7fa0ac15a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:56.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.603+0000 7fa0b11dc640 1 -- 172.21.15.172:0/327935804 shutdown_connections 2024-04-14T08:33:56.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.603+0000 7fa0b11dc640 1 -- 172.21.15.172:0/327935804 wait complete. 2024-04-14T08:33:56.615 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:33:56.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.715+0000 7f47bf78c640 1 Processor -- start 2024-04-14T08:33:56.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.715+0000 7f47bf78c640 1 -- start start 2024-04-14T08:33:56.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.716+0000 7f47bf78c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f47b80729b0 0x7f47b806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:56.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.716+0000 7f47bf78c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47b80b41d0 con 0x7f47b806e160 2024-04-14T08:33:56.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.716+0000 7f47bf78c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47b80aebe0 con 0x7f47b815e5b0 2024-04-14T08:33:56.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.716+0000 7f47bf78c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f47b80b4470 con 0x7f47b80729b0 2024-04-14T08:33:56.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.716+0000 7f47bdd29640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f47b806e160 0x7f47b806c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35030/0 (socket says 172.21.15.172:35030) 2024-04-14T08:33:56.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.716+0000 7f47bdd29640 1 -- 172.21.15.172:0/2502289231 learned_addr learned my addr 172.21.15.172:0/2502289231 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:56.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.716+0000 7f47bcd27640 1 --2- 172.21.15.172:0/2502289231 >> v2:172.21.15.177:3300/0 conn(0x7f47b80729b0 0x7f47b806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:56.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.716+0000 7f47affff640 1 -- 172.21.15.172:0/2502289231 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3970344922 0 0) 0x7f47b80b41d0 con 0x7f47b806e160 2024-04-14T08:33:56.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.716+0000 7f47affff640 1 -- 172.21.15.172:0/2502289231 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47a4003520 con 0x7f47b806e160 2024-04-14T08:33:56.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.717+0000 7f47affff640 1 -- 172.21.15.172:0/2502289231 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4074367708 0 0) 0x7f47a4003520 con 0x7f47b806e160 2024-04-14T08:33:56.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.717+0000 7f47bcd27640 1 -- 172.21.15.172:0/2502289231 >> v1:172.21.15.177:6789/0 conn(0x7f47b815e5b0 legacy=0x7f47b80ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.717+0000 7f47bcd27640 1 -- 172.21.15.172:0/2502289231 >> v1:172.21.15.172:6789/0 conn(0x7f47b806e160 legacy=0x7f47b806c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.717+0000 7f47bcd27640 1 -- 172.21.15.172:0/2502289231 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b80aeec0 con 0x7f47b80729b0 2024-04-14T08:33:56.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.717+0000 7f47bcd27640 1 --2- 172.21.15.172:0/2502289231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b80729b0 0x7f47b806dc20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f47a80097d0 tx=0x7f47a802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39314b4941c71db6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:56.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.718+0000 7f47affff640 1 -- 172.21.15.172:0/2502289231 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8038070 con 0x7f47b80729b0 2024-04-14T08:33:56.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.718+0000 7f47affff640 1 -- 172.21.15.172:0/2502289231 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47a8031500 con 0x7f47b80729b0 2024-04-14T08:33:56.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.718+0000 7f47affff640 1 -- 172.21.15.172:0/2502289231 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0001670 con 0x7f47b80729b0 2024-04-14T08:33:56.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.718+0000 7f47bf78c640 1 -- 172.21.15.172:0/2502289231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b80729b0 msgr2=0x7f47b806dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.718+0000 7f47bf78c640 1 --2- 172.21.15.172:0/2502289231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b80729b0 0x7f47b806dc20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f47a80097d0 tx=0x7f47a802b820 comp rx=0 tx=0).stop 2024-04-14T08:33:56.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.718+0000 7f47bf78c640 1 -- 172.21.15.172:0/2502289231 shutdown_connections 2024-04-14T08:33:56.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.718+0000 7f47bf78c640 1 --2- 172.21.15.172:0/2502289231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b80729b0 0x7f47b806dc20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.718+0000 7f47bf78c640 1 -- 172.21.15.172:0/2502289231 >> 172.21.15.172:0/2502289231 conn(0x7f47b8071ba0 msgr2=0x7f47b8071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:56.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.718+0000 7f47bf78c640 1 -- 172.21.15.172:0/2502289231 shutdown_connections 2024-04-14T08:33:56.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.718+0000 7f47bf78c640 1 -- 172.21.15.172:0/2502289231 wait complete. 2024-04-14T08:33:56.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.719+0000 7f47bf78c640 1 Processor -- start 2024-04-14T08:33:56.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.719+0000 7f47bf78c640 1 -- start start 2024-04-14T08:33:56.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.720+0000 7f47bf78c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47b806e160 0x7f47b8159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:56.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.720+0000 7f47bf78c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47b80729b0 0x7f47b80ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:56.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.720+0000 7f47bf78c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b815e5b0 0x7f47b80ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.720+0000 7f47bf78c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f47b80b4c70 con 0x7f47b80729b0 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.720+0000 7f47bf78c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f47b806ce90 con 0x7f47b815e5b0 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.720+0000 7f47bf78c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f47b80b4df0 con 0x7f47b806e160 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.720+0000 7f47bcd27640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47b80729b0 0x7f47b80ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.720+0000 7f47bcd27640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47b80729b0 0x7f47b80ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57384/0 (socket says 172.21.15.172:57384) 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.720+0000 7f47bcd27640 1 -- 172.21.15.172:0/2370191201 learned_addr learned my addr 172.21.15.172:0/2370191201 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.720+0000 7f47bdd29640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b815e5b0 0x7f47b80ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.720+0000 7f47bd528640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47b806e160 0x7f47b8159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.721+0000 7f47bcd27640 1 -- 172.21.15.172:0/2370191201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47b806e160 msgr2=0x7f47b8159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.721+0000 7f47bcd27640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47b806e160 0x7f47b8159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.721+0000 7f47bcd27640 1 -- 172.21.15.172:0/2370191201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b815e5b0 msgr2=0x7f47b80ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.721+0000 7f47bcd27640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b815e5b0 0x7f47b80ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.721+0000 7f47bcd27640 1 -- 172.21.15.172:0/2370191201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b815ebe0 con 0x7f47b80729b0 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.721+0000 7f47bdd29640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b815e5b0 0x7f47b80ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:33:56.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.721+0000 7f47bcd27640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47b80729b0 0x7f47b80ce250 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f47a8000c00 tx=0x7f47a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:56.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.721+0000 7f47adffb640 1 -- 172.21.15.172:0/2370191201 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8043070 con 0x7f47b80729b0 2024-04-14T08:33:56.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.721+0000 7f47adffb640 1 -- 172.21.15.172:0/2370191201 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47a803f870 con 0x7f47b80729b0 2024-04-14T08:33:56.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.721+0000 7f47bf78c640 1 -- 172.21.15.172:0/2370191201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b815eec0 con 0x7f47b80729b0 2024-04-14T08:33:56.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.721+0000 7f47adffb640 1 -- 172.21.15.172:0/2370191201 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8038040 con 0x7f47b80729b0 2024-04-14T08:33:56.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.721+0000 7f47bf78c640 1 -- 172.21.15.172:0/2370191201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b815f480 con 0x7f47b80729b0 2024-04-14T08:33:56.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.722+0000 7f47bf78c640 1 -- 172.21.15.172:0/2370191201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b8068520 con 0x7f47b80729b0 2024-04-14T08:33:56.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.722+0000 7f47adffb640 1 -- 172.21.15.172:0/2370191201 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47a8046070 con 0x7f47b80729b0 2024-04-14T08:33:56.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.723+0000 7f47adffb640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f479c06c8f0 0x7f479c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:56.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.723+0000 7f47adffb640 1 -- 172.21.15.172:0/2370191201 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f47a80ad540 con 0x7f47b80729b0 2024-04-14T08:33:56.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.723+0000 7f47bd528640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f479c06c8f0 0x7f479c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:56.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.723+0000 7f47bd528640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f479c06c8f0 0x7f479c06edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f47b40012e0 tx=0x7f47b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:56.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.727+0000 7f47adffb640 1 -- 172.21.15.172:0/2370191201 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f47a8030070 con 0x7f47b80729b0 2024-04-14T08:33:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.904+0000 7f47bf78c640 1 -- 172.21.15.172:0/2370191201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47b815bcb0 con 0x7f47b80729b0 2024-04-14T08:33:56.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.905+0000 7f47adffb640 1 -- 172.21.15.172:0/2370191201 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13224 (secure 0 0 0) 0x7f47a807bb10 con 0x7f47b80729b0 2024-04-14T08:33:56.906 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:56.906 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":35,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:33:53.985277+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:33:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.907+0000 7f47bf78c640 1 -- 172.21.15.172:0/2370191201 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f479c06c8f0 msgr2=0x7f479c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.907+0000 7f47bf78c640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f479c06c8f0 0x7f479c06edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f47b40012e0 tx=0x7f47b4009040 comp rx=0 tx=0).stop 2024-04-14T08:33:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.907+0000 7f47bf78c640 1 -- 172.21.15.172:0/2370191201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47b80729b0 msgr2=0x7f47b80ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.907+0000 7f47bf78c640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47b80729b0 0x7f47b80ce250 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f47a8000c00 tx=0x7f47a80046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.907+0000 7f47bf78c640 1 -- 172.21.15.172:0/2370191201 shutdown_connections 2024-04-14T08:33:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.907+0000 7f47bf78c640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f479c06c8f0 0x7f479c06edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.907+0000 7f47bf78c640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b815e5b0 0x7f47b80ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.907+0000 7f47bf78c640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47b80729b0 0x7f47b80ce250 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.907+0000 7f47bf78c640 1 --2- 172.21.15.172:0/2370191201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47b806e160 0x7f47b8159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.907+0000 7f47bf78c640 1 -- 172.21.15.172:0/2370191201 >> 172.21.15.172:0/2370191201 conn(0x7f47b8071ba0 msgr2=0x7f47b806fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:56.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.907+0000 7f47bf78c640 1 -- 172.21.15.172:0/2370191201 shutdown_connections 2024-04-14T08:33:56.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:56.907+0000 7f47bf78c640 1 -- 172.21.15.172:0/2370191201 wait complete. 2024-04-14T08:33:56.920 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:33:56.920 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:33:57.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.023+0000 7f49d275e640 1 Processor -- start 2024-04-14T08:33:57.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.023+0000 7f49d275e640 1 -- start start 2024-04-14T08:33:57.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.024+0000 7f49d275e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f49cc06d040 0x7f49cc0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.024+0000 7f49d275e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f49cc0d0d30 0x7f49cc0cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.024+0000 7f49d275e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49cc0d12e0 con 0x7f49cc15e820 2024-04-14T08:33:57.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.024+0000 7f49d275e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f49cc06cb90 con 0x7f49cc0d0d30 2024-04-14T08:33:57.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.024+0000 7f49d275e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f49cc0cc660 con 0x7f49cc06d040 2024-04-14T08:33:57.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.024+0000 7f49d0cfb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f49cc0d0d30 0x7f49cc0cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.024+0000 7f49cb7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f49cc15e820 0x7f49cc06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35032/0 (socket says 172.21.15.172:35032) 2024-04-14T08:33:57.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.024+0000 7f49cb7fe640 1 -- 172.21.15.172:0/1461098212 learned_addr learned my addr 172.21.15.172:0/1461098212 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:57.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.025+0000 7f49cbfff640 1 --2- 172.21.15.172:0/1461098212 >> v2:172.21.15.177:3300/0 conn(0x7f49cc06d040 0x7f49cc0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.025+0000 7f49d0cfb640 1 -- 172.21.15.172:0/1461098212 >> v2:172.21.15.177:3300/0 conn(0x7f49cc06d040 msgr2=0x7f49cc0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:57.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.025+0000 7f49d0cfb640 1 --2- 172.21.15.172:0/1461098212 >> v2:172.21.15.177:3300/0 conn(0x7f49cc06d040 0x7f49cc0b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.025+0000 7f49d0cfb640 1 -- 172.21.15.172:0/1461098212 >> v1:172.21.15.172:6789/0 conn(0x7f49cc15e820 legacy=0x7f49cc06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.025+0000 7f49d0cfb640 1 -- 172.21.15.172:0/1461098212 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc0d1580 con 0x7f49cc0d0d30 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.025+0000 7f49d0cfb640 1 --2- 172.21.15.172:0/1461098212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc0d0d30 0x7f49cc0cd0b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f49cc068210 tx=0x7f49c402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d556b9cf0c33498 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.025+0000 7f49caffd640 1 -- 172.21.15.172:0/1461098212 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4039070 con 0x7f49cc0d0d30 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.025+0000 7f49caffd640 1 -- 172.21.15.172:0/1461098212 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c402cc80 con 0x7f49cc0d0d30 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.026+0000 7f49caffd640 1 -- 172.21.15.172:0/1461098212 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4034660 con 0x7f49cc0d0d30 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.026+0000 7f49d275e640 1 -- 172.21.15.172:0/1461098212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc0d0d30 msgr2=0x7f49cc0cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.026+0000 7f49d275e640 1 --2- 172.21.15.172:0/1461098212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc0d0d30 0x7f49cc0cd0b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f49cc068210 tx=0x7f49c402bed0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.026+0000 7f49d275e640 1 -- 172.21.15.172:0/1461098212 shutdown_connections 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.026+0000 7f49d275e640 1 --2- 172.21.15.172:0/1461098212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc0d0d30 0x7f49cc0cd0b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.026+0000 7f49d275e640 1 --2- 172.21.15.172:0/1461098212 >> v2:172.21.15.177:3300/0 conn(0x7f49cc06d040 0x7f49cc0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.026+0000 7f49d275e640 1 -- 172.21.15.172:0/1461098212 >> 172.21.15.172:0/1461098212 conn(0x7f49cc0b3b00 msgr2=0x7f49cc0b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.026+0000 7f49d275e640 1 -- 172.21.15.172:0/1461098212 shutdown_connections 2024-04-14T08:33:57.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.026+0000 7f49d275e640 1 -- 172.21.15.172:0/1461098212 wait complete. 2024-04-14T08:33:57.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.027+0000 7f49d275e640 1 Processor -- start 2024-04-14T08:33:57.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.027+0000 7f49d275e640 1 -- start start 2024-04-14T08:33:57.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d275e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49cc06d040 0x7f49cc15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d275e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49cc0d0d30 0x7f49cc0b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d275e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc15e820 0x7f49cc0b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d275e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc0cd5f0 con 0x7f49cc15e820 2024-04-14T08:33:57.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d275e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc0ccc20 con 0x7f49cc0d0d30 2024-04-14T08:33:57.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d275e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f49cc0cd770 con 0x7f49cc06d040 2024-04-14T08:33:57.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d0cfb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc15e820 0x7f49cc0b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d0cfb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc15e820 0x7f49cc0b5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57412/0 (socket says 172.21.15.172:57412) 2024-04-14T08:33:57.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d0cfb640 1 -- 172.21.15.172:0/2034976359 learned_addr learned my addr 172.21.15.172:0/2034976359 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:57.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49cbfff640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49cc06d040 0x7f49cc15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49cb7fe640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49cc0d0d30 0x7f49cc0b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d0cfb640 1 -- 172.21.15.172:0/2034976359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49cc06d040 msgr2=0x7f49cc15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d0cfb640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49cc06d040 0x7f49cc15a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d0cfb640 1 -- 172.21.15.172:0/2034976359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49cc0d0d30 msgr2=0x7f49cc0b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d0cfb640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49cc0d0d30 0x7f49cc0b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.028+0000 7f49d0cfb640 1 -- 172.21.15.172:0/2034976359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc09ef10 con 0x7f49cc15e820 2024-04-14T08:33:57.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.029+0000 7f49d0cfb640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc15e820 0x7f49cc0b5220 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f49c402c4e0 tx=0x7f49c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:57.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.029+0000 7f49c8ff9640 1 -- 172.21.15.172:0/2034976359 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4039050 con 0x7f49cc15e820 2024-04-14T08:33:57.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.029+0000 7f49c8ff9640 1 -- 172.21.15.172:0/2034976359 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c4045070 con 0x7f49cc15e820 2024-04-14T08:33:57.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.029+0000 7f49c8ff9640 1 -- 172.21.15.172:0/2034976359 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c403e830 con 0x7f49cc15e820 2024-04-14T08:33:57.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.029+0000 7f49d275e640 1 -- 172.21.15.172:0/2034976359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc09f1f0 con 0x7f49cc15e820 2024-04-14T08:33:57.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.029+0000 7f49d275e640 1 -- 172.21.15.172:0/2034976359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc0b4d00 con 0x7f49cc15e820 2024-04-14T08:33:57.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.030+0000 7f49d275e640 1 -- 172.21.15.172:0/2034976359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49cc068590 con 0x7f49cc15e820 2024-04-14T08:33:57.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.030+0000 7f49c8ff9640 1 -- 172.21.15.172:0/2034976359 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49c403e9d0 con 0x7f49cc15e820 2024-04-14T08:33:57.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.031+0000 7f49c8ff9640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f49b806c8f0 0x7f49b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.031+0000 7f49c8ff9640 1 -- 172.21.15.172:0/2034976359 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f49c40ae650 con 0x7f49cc15e820 2024-04-14T08:33:57.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.031+0000 7f49cbfff640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f49b806c8f0 0x7f49b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.031+0000 7f49cbfff640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f49b806c8f0 0x7f49b806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006fd0 tx=0x7f49c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:57.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.035+0000 7f49c8ff9640 1 -- 172.21.15.172:0/2034976359 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f49c407cdf0 con 0x7f49cc15e820 2024-04-14T08:33:57.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.212+0000 7f49d275e640 1 -- 172.21.15.172:0/2034976359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f49cc06ce20 con 0x7f49cc15e820 2024-04-14T08:33:57.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.212+0000 7f49c8ff9640 1 -- 172.21.15.172:0/2034976359 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f49c407c6f0 con 0x7f49cc15e820 2024-04-14T08:33:57.214 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:33:57.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.214+0000 7f49d275e640 1 -- 172.21.15.172:0/2034976359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f49b806c8f0 msgr2=0x7f49b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.214+0000 7f49d275e640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f49b806c8f0 0x7f49b806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006fd0 tx=0x7f49c0008040 comp rx=0 tx=0).stop 2024-04-14T08:33:57.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.214+0000 7f49d275e640 1 -- 172.21.15.172:0/2034976359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc15e820 msgr2=0x7f49cc0b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.214+0000 7f49d275e640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc15e820 0x7f49cc0b5220 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f49c402c4e0 tx=0x7f49c4004a40 comp rx=0 tx=0).stop 2024-04-14T08:33:57.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.214+0000 7f49d275e640 1 -- 172.21.15.172:0/2034976359 shutdown_connections 2024-04-14T08:33:57.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.214+0000 7f49d275e640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f49b806c8f0 0x7f49b806edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.214+0000 7f49d275e640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc15e820 0x7f49cc0b5220 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.214+0000 7f49d275e640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49cc0d0d30 0x7f49cc0b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.214+0000 7f49d275e640 1 --2- 172.21.15.172:0/2034976359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49cc06d040 0x7f49cc15a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.214+0000 7f49d275e640 1 -- 172.21.15.172:0/2034976359 >> 172.21.15.172:0/2034976359 conn(0x7f49cc0b3b00 msgr2=0x7f49cc15dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:57.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.215+0000 7f49d275e640 1 -- 172.21.15.172:0/2034976359 shutdown_connections 2024-04-14T08:33:57.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.215+0000 7f49d275e640 1 -- 172.21.15.172:0/2034976359 wait complete. 2024-04-14T08:33:57.227 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-14T08:33:57.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.328+0000 7f2dbc636640 1 Processor -- start 2024-04-14T08:33:57.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.328+0000 7f2dbc636640 1 -- start start 2024-04-14T08:33:57.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.329+0000 7f2dbc636640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2db415e890 0x7f2db40d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.329+0000 7f2dbc636640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2db40cf320 con 0x7f2db40d2830 2024-04-14T08:33:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.329+0000 7f2dbc636640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2db40cf600 con 0x7f2db40cedd0 2024-04-14T08:33:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.329+0000 7f2dbc636640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2db40cff30 con 0x7f2db415e890 2024-04-14T08:33:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.329+0000 7f2db9bd1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2db415e890 0x7f2db40d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.329+0000 7f2dba3d2640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2db40d2830 0x7f2db40d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35044/0 (socket says 172.21.15.172:35044) 2024-04-14T08:33:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.329+0000 7f2db9bd1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2db415e890 0x7f2db40d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57416/0 (socket says 172.21.15.172:57416) 2024-04-14T08:33:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.329+0000 7f2db9bd1640 1 -- 172.21.15.172:0/2384127652 learned_addr learned my addr 172.21.15.172:0/2384127652 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.329+0000 7f2db9bd1640 1 -- 172.21.15.172:0/2384127652 >> v1:172.21.15.177:6789/0 conn(0x7f2db40cedd0 legacy=0x7f2db40b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.329+0000 7f2db9bd1640 1 -- 172.21.15.172:0/2384127652 >> v1:172.21.15.172:6789/0 conn(0x7f2db40d2830 legacy=0x7f2db40d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.329+0000 7f2db93d0640 1 -- 172.21.15.172:0/2384127652 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4291505606 0 0) 0x7f2db40cf320 con 0x7f2db40d2830 2024-04-14T08:33:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.329+0000 7f2db9bd1640 1 -- 172.21.15.172:0/2384127652 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db40cf8e0 con 0x7f2db415e890 2024-04-14T08:33:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.330+0000 7f2db9bd1640 1 --2- 172.21.15.172:0/2384127652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db415e890 0x7f2db40d1300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009870 tx=0x7f2da402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91084e9d2cda932 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.330+0000 7f2db93d0640 1 -- 172.21.15.172:0/2384127652 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da402f840 con 0x7f2db415e890 2024-04-14T08:33:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.330+0000 7f2db93d0640 1 -- 172.21.15.172:0/2384127652 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2da402f9e0 con 0x7f2db415e890 2024-04-14T08:33:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.330+0000 7f2db93d0640 1 -- 172.21.15.172:0/2384127652 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da402fdb0 con 0x7f2db415e890 2024-04-14T08:33:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.331+0000 7f2dbc636640 1 -- 172.21.15.172:0/2384127652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db415e890 msgr2=0x7f2db40d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.331+0000 7f2dbc636640 1 --2- 172.21.15.172:0/2384127652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db415e890 0x7f2db40d1300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009870 tx=0x7f2da402ba20 comp rx=0 tx=0).stop 2024-04-14T08:33:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.331+0000 7f2dbc636640 1 -- 172.21.15.172:0/2384127652 shutdown_connections 2024-04-14T08:33:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.331+0000 7f2dbc636640 1 --2- 172.21.15.172:0/2384127652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db415e890 0x7f2db40d1300 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.331+0000 7f2dbc636640 1 -- 172.21.15.172:0/2384127652 >> 172.21.15.172:0/2384127652 conn(0x7f2db409d590 msgr2=0x7f2db40d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.331+0000 7f2dbc636640 1 -- 172.21.15.172:0/2384127652 shutdown_connections 2024-04-14T08:33:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.331+0000 7f2dbc636640 1 -- 172.21.15.172:0/2384127652 wait complete. 2024-04-14T08:33:57.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.332+0000 7f2dbc636640 1 Processor -- start 2024-04-14T08:33:57.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.332+0000 7f2dbc636640 1 -- start start 2024-04-14T08:33:57.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbc636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db40cedd0 0x7f2db40c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbc636640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2db40d2830 0x7f2db40c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbc636640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db415e890 0x7f2db415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbc636640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2db40d1840 con 0x7f2db415e890 2024-04-14T08:33:57.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbc636640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2db40d2350 con 0x7f2db40d2830 2024-04-14T08:33:57.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbc636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2db40b3e20 con 0x7f2db40cedd0 2024-04-14T08:33:57.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbabd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db415e890 0x7f2db415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbabd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db415e890 0x7f2db415b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57424/0 (socket says 172.21.15.172:57424) 2024-04-14T08:33:57.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbabd3640 1 -- 172.21.15.172:0/2431676072 learned_addr learned my addr 172.21.15.172:0/2431676072 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:57.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dba3d2640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db40cedd0 0x7f2db40c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2db9bd1640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2db40d2830 0x7f2db40c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbabd3640 1 -- 172.21.15.172:0/2431676072 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db40cedd0 msgr2=0x7f2db40c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbabd3640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db40cedd0 0x7f2db40c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbabd3640 1 -- 172.21.15.172:0/2431676072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2db40d2830 msgr2=0x7f2db40c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbabd3640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2db40d2830 0x7f2db40c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbabd3640 1 -- 172.21.15.172:0/2431676072 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db41616c0 con 0x7f2db415e890 2024-04-14T08:33:57.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2dbabd3640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db415e890 0x7f2db415b0f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00c780 tx=0x7f2dac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:57.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.333+0000 7f2daaffd640 1 -- 172.21.15.172:0/2431676072 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac004b60 con 0x7f2db415e890 2024-04-14T08:33:57.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.334+0000 7f2daaffd640 1 -- 172.21.15.172:0/2431676072 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dac004d00 con 0x7f2db415e890 2024-04-14T08:33:57.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.334+0000 7f2dbc636640 1 -- 172.21.15.172:0/2431676072 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db4160e20 con 0x7f2db415e890 2024-04-14T08:33:57.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.334+0000 7f2daaffd640 1 -- 172.21.15.172:0/2431676072 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00aa90 con 0x7f2db415e890 2024-04-14T08:33:57.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.334+0000 7f2dbc636640 1 -- 172.21.15.172:0/2431676072 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db41613e0 con 0x7f2db415e890 2024-04-14T08:33:57.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.335+0000 7f2dbc636640 1 -- 172.21.15.172:0/2431676072 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db4068520 con 0x7f2db415e890 2024-04-14T08:33:57.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.335+0000 7f2daaffd640 1 -- 172.21.15.172:0/2431676072 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2dac00ac30 con 0x7f2db415e890 2024-04-14T08:33:57.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.335+0000 7f2daaffd640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d9806c8f0 0x7f2d9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.336+0000 7f2daaffd640 1 -- 172.21.15.172:0/2431676072 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f2dac08ba50 con 0x7f2db415e890 2024-04-14T08:33:57.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.336+0000 7f2dba3d2640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d9806c8f0 0x7f2d9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.336+0000 7f2dba3d2640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d9806c8f0 0x7f2d9806edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2db0000f60 tx=0x7f2db0007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:57.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.340+0000 7f2daaffd640 1 -- 172.21.15.172:0/2431676072 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2dac020070 con 0x7f2db415e890 2024-04-14T08:33:57.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.544+0000 7f2dbc636640 1 -- 172.21.15.172:0/2431676072 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2db4067750 con 0x7f2db415e890 2024-04-14T08:33:57.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.544+0000 7f2daaffd640 1 -- 172.21.15.172:0/2431676072 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7f2dac059a80 con 0x7f2db415e890 2024-04-14T08:33:57.546 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:33:57.546 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-04-14T08:33:57.546 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:33:57.546 INFO:teuthology.orchestra.run.smithi172.stdout: "poolnum": 1, 2024-04-14T08:33:57.546 INFO:teuthology.orchestra.run.smithi172.stdout: "poolname": ".mgr" 2024-04-14T08:33:57.546 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:33:57.547 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-04-14T08:33:57.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.546+0000 7f2dbc636640 1 -- 172.21.15.172:0/2431676072 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d9806c8f0 msgr2=0x7f2d9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.546+0000 7f2dbc636640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d9806c8f0 0x7f2d9806edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2db0000f60 tx=0x7f2db0007d50 comp rx=0 tx=0).stop 2024-04-14T08:33:57.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.546+0000 7f2dbc636640 1 -- 172.21.15.172:0/2431676072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db415e890 msgr2=0x7f2db415b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.546+0000 7f2dbc636640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db415e890 0x7f2db415b0f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00c780 tx=0x7f2dac00cc40 comp rx=0 tx=0).stop 2024-04-14T08:33:57.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.546+0000 7f2dbc636640 1 -- 172.21.15.172:0/2431676072 shutdown_connections 2024-04-14T08:33:57.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.546+0000 7f2dbc636640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d9806c8f0 0x7f2d9806edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.546+0000 7f2dbc636640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db415e890 0x7f2db415b0f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.546+0000 7f2dbc636640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2db40d2830 0x7f2db40c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.546+0000 7f2dbc636640 1 --2- 172.21.15.172:0/2431676072 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db40cedd0 0x7f2db40c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.547+0000 7f2dbc636640 1 -- 172.21.15.172:0/2431676072 >> 172.21.15.172:0/2431676072 conn(0x7f2db409d590 msgr2=0x7f2db406c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:57.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.547+0000 7f2dbc636640 1 -- 172.21.15.172:0/2431676072 shutdown_connections 2024-04-14T08:33:57.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.547+0000 7f2dbc636640 1 -- 172.21.15.172:0/2431676072 wait complete. 2024-04-14T08:33:57.559 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-14T08:33:57.559 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-14T08:33:57.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.666+0000 7f1370588640 1 Processor -- start 2024-04-14T08:33:57.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.666+0000 7f1370588640 1 -- start start 2024-04-14T08:33:57.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f1370588640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f13680b0500 0x7f136806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f1370588640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680d0a40 0x7f136815dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f1370588640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1368070250 con 0x7f136815f480 2024-04-14T08:33:57.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f1370588640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f13680cb5d0 con 0x7f13680b0500 2024-04-14T08:33:57.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f1370588640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f13680732a0 con 0x7f13680d0a40 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136db23640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f136815f480 0x7f13680d0330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35056/0 (socket says 172.21.15.172:35056) 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136db23640 1 -- 172.21.15.172:0/3199858625 learned_addr learned my addr 172.21.15.172:0/3199858625 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136eb25640 1 --2- 172.21.15.172:0/3199858625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680d0a40 0x7f136815dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136e324640 1 --2- 172.21.15.172:0/3199858625 >> v2:172.21.15.177:3300/0 conn(0x7f13680b0500 0x7f136806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 690639223 0 0) 0x7f1368070250 con 0x7f136815f480 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1354003520 con 0x7f136815f480 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1860481454 0 0) 0x7f1354003520 con 0x7f136815f480 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680d0a40 msgr2=0x7f136815dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 --2- 172.21.15.172:0/3199858625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680d0a40 0x7f136815dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 >> v2:172.21.15.177:3300/0 conn(0x7f13680b0500 msgr2=0x7f136806fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 --2- 172.21.15.172:0/3199858625 >> v2:172.21.15.177:3300/0 conn(0x7f13680b0500 0x7f136806fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1368162e60 con 0x7f136815f480 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f1358001670 con 0x7f136815f480 2024-04-14T08:33:57.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 >> v1:172.21.15.172:6789/0 conn(0x7f136815f480 legacy=0x7f13680d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 --2- 172.21.15.172:0/3199858625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13540039b0 0x7f1354003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 --2- 172.21.15.172:0/3199858625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1354004800 0x7f1354004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 --2- 172.21.15.172:0/3199858625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680d0a40 0x7f13540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f135400b9c0 con 0x7f13540039b0 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f135400bb40 con 0x7f1354004800 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f135400bcc0 con 0x7f13680d0a40 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.667+0000 7f136e324640 1 --2- 172.21.15.172:0/3199858625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13540039b0 0x7f1354003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.668+0000 7f136db23640 1 --2- 172.21.15.172:0/3199858625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1354004800 0x7f1354004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.668+0000 7f136eb25640 1 --2- 172.21.15.172:0/3199858625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680d0a40 0x7f13540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.668+0000 7f136e324640 1 -- 172.21.15.172:0/3199858625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680d0a40 msgr2=0x7f13540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.668+0000 7f136e324640 1 --2- 172.21.15.172:0/3199858625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680d0a40 0x7f13540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.668+0000 7f136e324640 1 -- 172.21.15.172:0/3199858625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1354004800 msgr2=0x7f1354004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.668+0000 7f136e324640 1 --2- 172.21.15.172:0/3199858625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1354004800 0x7f1354004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.668+0000 7f136e324640 1 -- 172.21.15.172:0/3199858625 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1354003520 con 0x7f13540039b0 2024-04-14T08:33:57.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.669+0000 7f136e324640 1 --2- 172.21.15.172:0/3199858625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13540039b0 0x7f1354003dd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1364009af0 tx=0x7f1354005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f05f26062c1cfa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:57.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.669+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364039070 con 0x7f13540039b0 2024-04-14T08:33:57.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.669+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136402c710 con 0x7f13540039b0 2024-04-14T08:33:57.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.669+0000 7f136d322640 1 -- 172.21.15.172:0/3199858625 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364009690 con 0x7f13540039b0 2024-04-14T08:33:57.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.669+0000 7f1370588640 1 -- 172.21.15.172:0/3199858625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13540039b0 msgr2=0x7f1354003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.669+0000 7f1370588640 1 --2- 172.21.15.172:0/3199858625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13540039b0 0x7f1354003dd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1364009af0 tx=0x7f1354005830 comp rx=0 tx=0).stop 2024-04-14T08:33:57.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.669+0000 7f136eb25640 1 -- 172.21.15.172:0/3199858625 reap_dead start 2024-04-14T08:33:57.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.669+0000 7f1370588640 1 -- 172.21.15.172:0/3199858625 shutdown_connections 2024-04-14T08:33:57.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.669+0000 7f1370588640 1 -- 172.21.15.172:0/3199858625 >> 172.21.15.172:0/3199858625 conn(0x7f13680cda00 msgr2=0x7f13680cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:57.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.670+0000 7f1370588640 1 -- 172.21.15.172:0/3199858625 shutdown_connections 2024-04-14T08:33:57.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.670+0000 7f1370588640 1 -- 172.21.15.172:0/3199858625 wait complete. 2024-04-14T08:33:57.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.671+0000 7f1370588640 1 Processor -- start 2024-04-14T08:33:57.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.671+0000 7f1370588640 1 -- start start 2024-04-14T08:33:57.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.671+0000 7f1370588640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13540039b0 0x7f13681549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.671+0000 7f1370588640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680cb3f0 0x7f13680cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.671+0000 7f1370588640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13680c94a0 0x7f13680c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.671+0000 7f1370588640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1368162380 con 0x7f13540039b0 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.671+0000 7f1370588640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f13681620a0 con 0x7f13680c94a0 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.671+0000 7f1370588640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1368162660 con 0x7f13680cb3f0 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f136e324640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13540039b0 0x7f13681549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f136e324640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13540039b0 0x7f13681549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57442/0 (socket says 172.21.15.172:57442) 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f136e324640 1 -- 172.21.15.172:0/486354727 learned_addr learned my addr 172.21.15.172:0/486354727 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f136eb25640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13680c94a0 0x7f13680c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f136db23640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680cb3f0 0x7f13680cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f136e324640 1 -- 172.21.15.172:0/486354727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680cb3f0 msgr2=0x7f13680cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f136e324640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680cb3f0 0x7f13680cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f136e324640 1 -- 172.21.15.172:0/486354727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13680c94a0 msgr2=0x7f13680c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f136e324640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13680c94a0 0x7f13680c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f136e324640 1 -- 172.21.15.172:0/486354727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13680ca0b0 con 0x7f13540039b0 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f136e324640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13540039b0 0x7f13681549e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1364037040 tx=0x7f1364004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:57.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f135effd640 1 -- 172.21.15.172:0/486354727 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364039050 con 0x7f13540039b0 2024-04-14T08:33:57.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f135effd640 1 -- 172.21.15.172:0/486354727 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1364045070 con 0x7f13540039b0 2024-04-14T08:33:57.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f1370588640 1 -- 172.21.15.172:0/486354727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13680ca390 con 0x7f13540039b0 2024-04-14T08:33:57.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f135effd640 1 -- 172.21.15.172:0/486354727 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364003a50 con 0x7f13540039b0 2024-04-14T08:33:57.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.672+0000 7f1370588640 1 -- 172.21.15.172:0/486354727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13680ca950 con 0x7f13540039b0 2024-04-14T08:33:57.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.673+0000 7f1370588640 1 -- 172.21.15.172:0/486354727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13680758c0 con 0x7f13540039b0 2024-04-14T08:33:57.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.674+0000 7f135effd640 1 -- 172.21.15.172:0/486354727 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1364003bf0 con 0x7f13540039b0 2024-04-14T08:33:57.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.674+0000 7f135effd640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f134c06c8f0 0x7f134c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:57.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.674+0000 7f135effd640 1 -- 172.21.15.172:0/486354727 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f13640ad870 con 0x7f13540039b0 2024-04-14T08:33:57.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.674+0000 7f136db23640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f134c06c8f0 0x7f134c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:57.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.675+0000 7f136db23640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f134c06c8f0 0x7f134c06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f13681551b0 tx=0x7f1358007970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:57.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.680+0000 7f135effd640 1 -- 172.21.15.172:0/486354727 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f136404f020 con 0x7f13540039b0 2024-04-14T08:33:57.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.864+0000 7f1370588640 1 -- 172.21.15.172:0/486354727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f13680686a0 con 0x7f13540039b0 2024-04-14T08:33:57.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.976+0000 7f135effd640 1 -- 172.21.15.172:0/486354727 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7f136407b820 con 0x7f13540039b0 2024-04-14T08:33:57.978 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' created 2024-04-14T08:33:57.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.978+0000 7f1370588640 1 -- 172.21.15.172:0/486354727 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f134c06c8f0 msgr2=0x7f134c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.978+0000 7f1370588640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f134c06c8f0 0x7f134c06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f13681551b0 tx=0x7f1358007970 comp rx=0 tx=0).stop 2024-04-14T08:33:57.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.978+0000 7f1370588640 1 -- 172.21.15.172:0/486354727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13540039b0 msgr2=0x7f13681549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:57.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.978+0000 7f1370588640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13540039b0 0x7f13681549e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1364037040 tx=0x7f1364004730 comp rx=0 tx=0).stop 2024-04-14T08:33:57.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.978+0000 7f1370588640 1 -- 172.21.15.172:0/486354727 shutdown_connections 2024-04-14T08:33:57.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.978+0000 7f1370588640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f134c06c8f0 0x7f134c06edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.978+0000 7f1370588640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13680c94a0 0x7f13680c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.978+0000 7f1370588640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13680cb3f0 0x7f13680cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.978+0000 7f1370588640 1 --2- 172.21.15.172:0/486354727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13540039b0 0x7f13681549e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:57.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.978+0000 7f1370588640 1 -- 172.21.15.172:0/486354727 >> 172.21.15.172:0/486354727 conn(0x7f13680cda00 msgr2=0x7f136815eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:57.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.979+0000 7f1370588640 1 -- 172.21.15.172:0/486354727 shutdown_connections 2024-04-14T08:33:57.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:57.979+0000 7f1370588640 1 -- 172.21.15.172:0/486354727 wait complete. 2024-04-14T08:33:57.995 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-14T08:33:58.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.103+0000 7fcfdbc1c640 1 Processor -- start 2024-04-14T08:33:58.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.104+0000 7fcfdbc1c640 1 -- start start 2024-04-14T08:33:58.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.104+0000 7fcfdbc1c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcfd415c420 0x7fcfd40b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:58.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.104+0000 7fcfdbc1c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcfd406f3e0 0x7fcfd40d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:58.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.104+0000 7fcfdbc1c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfd40d0550 con 0x7fcfd40ceb00 2024-04-14T08:33:58.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.104+0000 7fcfdbc1c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fcfd40d30a0 con 0x7fcfd415c420 2024-04-14T08:33:58.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.104+0000 7fcfdbc1c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fcfd406f870 con 0x7fcfd406f3e0 2024-04-14T08:33:58.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.105+0000 7fcfd99b8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcfd415c420 0x7fcfd40b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:58.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.105+0000 7fcfda1b9640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fcfd40ceb00 0x7fcfd40cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35058/0 (socket says 172.21.15.172:35058) 2024-04-14T08:33:58.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.105+0000 7fcfd99b8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcfd415c420 0x7fcfd40b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57446/0 (socket says 172.21.15.172:57446) 2024-04-14T08:33:58.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.105+0000 7fcfda1b9640 1 -- 172.21.15.172:0/3354390064 learned_addr learned my addr 172.21.15.172:0/3354390064 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:58.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.105+0000 7fcfd91b7640 1 --2- 172.21.15.172:0/3354390064 >> v2:172.21.15.177:3300/0 conn(0x7fcfd406f3e0 0x7fcfd40d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:58.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.105+0000 7fcfd99b8640 1 -- 172.21.15.172:0/3354390064 >> v2:172.21.15.177:3300/0 conn(0x7fcfd406f3e0 msgr2=0x7fcfd40d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:58.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.105+0000 7fcfd99b8640 1 --2- 172.21.15.172:0/3354390064 >> v2:172.21.15.177:3300/0 conn(0x7fcfd406f3e0 0x7fcfd40d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:58.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.105+0000 7fcfd99b8640 1 -- 172.21.15.172:0/3354390064 >> v1:172.21.15.172:6789/0 conn(0x7fcfd40ceb00 legacy=0x7fcfd40cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:58.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.105+0000 7fcfd99b8640 1 -- 172.21.15.172:0/3354390064 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd40ccdd0 con 0x7fcfd415c420 2024-04-14T08:33:58.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.106+0000 7fcfd99b8640 1 --2- 172.21.15.172:0/3354390064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd415c420 0x7fcfd40b1510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40740b0 tx=0x7fcfc402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4c5bfa17253a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:58.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.106+0000 7fcfd89b6640 1 -- 172.21.15.172:0/3354390064 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4035020 con 0x7fcfd415c420 2024-04-14T08:33:58.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.106+0000 7fcfd89b6640 1 -- 172.21.15.172:0/3354390064 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc4031830 con 0x7fcfd415c420 2024-04-14T08:33:58.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.106+0000 7fcfd89b6640 1 -- 172.21.15.172:0/3354390064 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4031c00 con 0x7fcfd415c420 2024-04-14T08:33:58.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.106+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/3354390064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd415c420 msgr2=0x7fcfd40b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:58.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.106+0000 7fcfdbc1c640 1 --2- 172.21.15.172:0/3354390064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd415c420 0x7fcfd40b1510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40740b0 tx=0x7fcfc402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:33:58.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.106+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/3354390064 shutdown_connections 2024-04-14T08:33:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.106+0000 7fcfdbc1c640 1 --2- 172.21.15.172:0/3354390064 >> v2:172.21.15.177:3300/0 conn(0x7fcfd406f3e0 0x7fcfd40d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.106+0000 7fcfdbc1c640 1 --2- 172.21.15.172:0/3354390064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd415c420 0x7fcfd40b1510 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.106+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/3354390064 >> 172.21.15.172:0/3354390064 conn(0x7fcfd408c000 msgr2=0x7fcfd40d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.106+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/3354390064 shutdown_connections 2024-04-14T08:33:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.107+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/3354390064 wait complete. 2024-04-14T08:33:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.108+0000 7fcfdbc1c640 1 Processor -- start 2024-04-14T08:33:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.108+0000 7fcfdbc1c640 1 -- start start 2024-04-14T08:33:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.108+0000 7fcfdbc1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd40ceb00 0x7fcfd40c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.108+0000 7fcfdbc1c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfd415c420 0x7fcfd40c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.108+0000 7fcfdbc1c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfd41f2330 0x7fcfd41f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.108+0000 7fcfdbc1c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd40cd070 con 0x7fcfd40ceb00 2024-04-14T08:33:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.108+0000 7fcfdbc1c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd40d2dc0 con 0x7fcfd41f2330 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.108+0000 7fcfdbc1c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd40b11e0 con 0x7fcfd415c420 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.108+0000 7fcfd99b8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd40ceb00 0x7fcfd40c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.108+0000 7fcfd99b8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd40ceb00 0x7fcfd40c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57454/0 (socket says 172.21.15.172:57454) 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.108+0000 7fcfd99b8640 1 -- 172.21.15.172:0/2119250626 learned_addr learned my addr 172.21.15.172:0/2119250626 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfda1b9640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfd41f2330 0x7fcfd41f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfd91b7640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfd415c420 0x7fcfd40c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfd99b8640 1 -- 172.21.15.172:0/2119250626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfd415c420 msgr2=0x7fcfd40c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfd99b8640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfd415c420 0x7fcfd40c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfd99b8640 1 -- 172.21.15.172:0/2119250626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfd41f2330 msgr2=0x7fcfd41f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfd99b8640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfd41f2330 0x7fcfd41f27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfd99b8640 1 -- 172.21.15.172:0/2119250626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd40c6ff0 con 0x7fcfd40ceb00 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfd99b8640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd40ceb00 0x7fcfd40c63a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4000c00 tx=0x7fcfc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfd27fc640 1 -- 172.21.15.172:0/2119250626 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4040070 con 0x7fcfd40ceb00 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfd27fc640 1 -- 172.21.15.172:0/2119250626 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc4039a20 con 0x7fcfd40ceb00 2024-04-14T08:33:58.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfd27fc640 1 -- 172.21.15.172:0/2119250626 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4035040 con 0x7fcfd40ceb00 2024-04-14T08:33:58.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/2119250626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd41f3590 con 0x7fcfd40ceb00 2024-04-14T08:33:58.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.109+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/2119250626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd40c72d0 con 0x7fcfd40ceb00 2024-04-14T08:33:58.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.110+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/2119250626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd4075490 con 0x7fcfd40ceb00 2024-04-14T08:33:58.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.111+0000 7fcfd27fc640 1 -- 172.21.15.172:0/2119250626 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfc404b060 con 0x7fcfd40ceb00 2024-04-14T08:33:58.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.111+0000 7fcfd27fc640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcfc006c8f0 0x7fcfc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:58.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.111+0000 7fcfd27fc640 1 -- 172.21.15.172:0/2119250626 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7fcfc40ad320 con 0x7fcfd40ceb00 2024-04-14T08:33:58.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.111+0000 7fcfd91b7640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcfc006c8f0 0x7fcfc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:58.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.111+0000 7fcfd91b7640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcfc006c8f0 0x7fcfc006edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40c8990 tx=0x7fcfc80077b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:58.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.116+0000 7fcfd27fc640 1 -- 172.21.15.172:0/2119250626 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcfc407b1e0 con 0x7fcfd40ceb00 2024-04-14T08:33:58.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.318+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/2119250626 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcfd415bc40 con 0x7fcfd40ceb00 2024-04-14T08:33:58.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.981+0000 7fcfd27fc640 1 -- 172.21.15.172:0/2119250626 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7fcfc407f4e0 con 0x7fcfd40ceb00 2024-04-14T08:33:58.983 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' created 2024-04-14T08:33:58.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.983+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/2119250626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcfc006c8f0 msgr2=0x7fcfc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:58.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.983+0000 7fcfdbc1c640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcfc006c8f0 0x7fcfc006edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40c8990 tx=0x7fcfc80077b0 comp rx=0 tx=0).stop 2024-04-14T08:33:58.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.983+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/2119250626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd40ceb00 msgr2=0x7fcfd40c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:58.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.983+0000 7fcfdbc1c640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd40ceb00 0x7fcfd40c63a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4000c00 tx=0x7fcfc40046c0 comp rx=0 tx=0).stop 2024-04-14T08:33:58.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.983+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/2119250626 shutdown_connections 2024-04-14T08:33:58.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.983+0000 7fcfdbc1c640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcfc006c8f0 0x7fcfc006edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:58.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.983+0000 7fcfdbc1c640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfd41f2330 0x7fcfd41f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:58.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.983+0000 7fcfdbc1c640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfd415c420 0x7fcfd40c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:58.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.983+0000 7fcfdbc1c640 1 --2- 172.21.15.172:0/2119250626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfd40ceb00 0x7fcfd40c63a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:58.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.983+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/2119250626 >> 172.21.15.172:0/2119250626 conn(0x7fcfd408c000 msgr2=0x7fcfd40b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:58.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.983+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/2119250626 shutdown_connections 2024-04-14T08:33:58.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:58.983+0000 7fcfdbc1c640 1 -- 172.21.15.172:0/2119250626 wait complete. 2024-04-14T08:33:59.004 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-14T08:33:59.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.110+0000 7f80b2c98640 1 Processor -- start 2024-04-14T08:33:59.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.110+0000 7f80b2c98640 1 -- start start 2024-04-14T08:33:59.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.111+0000 7f80b2c98640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f80ac140fa0 0x7f80ac162840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:59.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.111+0000 7f80b2c98640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f80ac141380 0x7f80ac0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:59.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.111+0000 7f80b2c98640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80ac0cf9b0 con 0x7f80ac0b2210 2024-04-14T08:33:59.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.111+0000 7f80b2c98640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f80ac0b2680 con 0x7f80ac140fa0 2024-04-14T08:33:59.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.111+0000 7f80b2c98640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f80ac162d80 con 0x7f80ac141380 2024-04-14T08:33:59.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.111+0000 7f80b0a34640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f80ac140fa0 0x7f80ac162840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:59.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.111+0000 7f80b1235640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f80ac0b2210 0x7f80ac0adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35062/0 (socket says 172.21.15.172:35062) 2024-04-14T08:33:59.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.111+0000 7f80b1235640 1 -- 172.21.15.172:0/1417683616 learned_addr learned my addr 172.21.15.172:0/1417683616 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:33:59.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.111+0000 7f80abfff640 1 --2- 172.21.15.172:0/1417683616 >> v2:172.21.15.177:3300/0 conn(0x7f80ac141380 0x7f80ac0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:59.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.112+0000 7f80b0a34640 1 -- 172.21.15.172:0/1417683616 >> v2:172.21.15.177:3300/0 conn(0x7f80ac141380 msgr2=0x7f80ac0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:33:59.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.112+0000 7f80b0a34640 1 --2- 172.21.15.172:0/1417683616 >> v2:172.21.15.177:3300/0 conn(0x7f80ac141380 0x7f80ac0b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:59.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.112+0000 7f80b0a34640 1 -- 172.21.15.172:0/1417683616 >> v1:172.21.15.172:6789/0 conn(0x7f80ac0b2210 legacy=0x7f80ac0adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:59.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.112+0000 7f80b0a34640 1 -- 172.21.15.172:0/1417683616 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ac15ecc0 con 0x7f80ac140fa0 2024-04-14T08:33:59.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.112+0000 7f80ab7fe640 1 -- 172.21.15.172:0/1417683616 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 636362884 0 0) 0x7f80ac0cf9b0 con 0x7f80ac0b2210 2024-04-14T08:33:59.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.112+0000 7f80abfff640 1 --2- 172.21.15.172:0/1417683616 >> v2:172.21.15.177:3300/0 conn(0x7f80ac141380 0x7f80ac0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:33:59.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.112+0000 7f80b0a34640 1 --2- 172.21.15.172:0/1417683616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80ac140fa0 0x7f80ac162840 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f80ac06f780 tx=0x7f809c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12243b3bc907783 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.112+0000 7f80ab7fe640 1 -- 172.21.15.172:0/1417683616 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c0315a0 con 0x7f80ac140fa0 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.112+0000 7f80ab7fe640 1 -- 172.21.15.172:0/1417683616 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f809c037070 con 0x7f80ac140fa0 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.112+0000 7f80ab7fe640 1 -- 172.21.15.172:0/1417683616 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c031960 con 0x7f80ac140fa0 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.113+0000 7f80b2c98640 1 -- 172.21.15.172:0/1417683616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80ac140fa0 msgr2=0x7f80ac162840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.113+0000 7f80b2c98640 1 --2- 172.21.15.172:0/1417683616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80ac140fa0 0x7f80ac162840 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f80ac06f780 tx=0x7f809c02b840 comp rx=0 tx=0).stop 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.113+0000 7f80b2c98640 1 -- 172.21.15.172:0/1417683616 shutdown_connections 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.113+0000 7f80b2c98640 1 --2- 172.21.15.172:0/1417683616 >> v2:172.21.15.177:3300/0 conn(0x7f80ac141380 0x7f80ac0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.113+0000 7f80b2c98640 1 --2- 172.21.15.172:0/1417683616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80ac140fa0 0x7f80ac162840 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.113+0000 7f80b2c98640 1 -- 172.21.15.172:0/1417683616 >> 172.21.15.172:0/1417683616 conn(0x7f80ac06b520 msgr2=0x7f80ac06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.113+0000 7f80b2c98640 1 -- 172.21.15.172:0/1417683616 shutdown_connections 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.113+0000 7f80b2c98640 1 -- 172.21.15.172:0/1417683616 wait complete. 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.114+0000 7f80b2c98640 1 Processor -- start 2024-04-14T08:33:59.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.114+0000 7f80b2c98640 1 -- start start 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80b2c98640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac0b2210 0x7f80ac1f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80b2c98640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80ac140fa0 0x7f80ac13b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80b2c98640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac141380 0x7f80ac156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80b2c98640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac15f560 con 0x7f80ac140fa0 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80b2c98640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f80ac15f280 con 0x7f80ac141380 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80b2c98640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f80ac0af3e0 con 0x7f80ac0b2210 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80ac140fa0 0x7f80ac13b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80ac140fa0 0x7f80ac13b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57476/0 (socket says 172.21.15.172:57476) 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80abfff640 1 -- 172.21.15.172:0/1846460675 learned_addr learned my addr 172.21.15.172:0/1846460675 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80b1235640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac141380 0x7f80ac156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80abfff640 1 -- 172.21.15.172:0/1846460675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac0b2210 msgr2=0x7f80ac1f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80b0a34640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac0b2210 0x7f80ac1f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80abfff640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac0b2210 0x7f80ac1f3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80abfff640 1 -- 172.21.15.172:0/1846460675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac141380 msgr2=0x7f80ac156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:33:59.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.115+0000 7f80abfff640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac141380 0x7f80ac156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:33:59.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.116+0000 7f80abfff640 1 -- 172.21.15.172:0/1846460675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ac161e70 con 0x7f80ac140fa0 2024-04-14T08:33:59.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.116+0000 7f80abfff640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80ac140fa0 0x7f80ac13b9b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c7f0 tx=0x7f80a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:59.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.116+0000 7f80a97fa640 1 -- 172.21.15.172:0/1846460675 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0015070 con 0x7f80ac140fa0 2024-04-14T08:33:59.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.116+0000 7f80a97fa640 1 -- 172.21.15.172:0/1846460675 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a0007820 con 0x7f80ac140fa0 2024-04-14T08:33:59.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.116+0000 7f80b2c98640 1 -- 172.21.15.172:0/1846460675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ac15fac0 con 0x7f80ac140fa0 2024-04-14T08:33:59.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.116+0000 7f80a97fa640 1 -- 172.21.15.172:0/1846460675 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0007c10 con 0x7f80ac140fa0 2024-04-14T08:33:59.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.116+0000 7f80b2c98640 1 -- 172.21.15.172:0/1846460675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ac13bef0 con 0x7f80ac140fa0 2024-04-14T08:33:59.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.117+0000 7f80b2c98640 1 -- 172.21.15.172:0/1846460675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80ac15f8b0 con 0x7f80ac140fa0 2024-04-14T08:33:59.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.117+0000 7f80a97fa640 1 -- 172.21.15.172:0/1846460675 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80a0018070 con 0x7f80ac140fa0 2024-04-14T08:33:59.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.118+0000 7f80a97fa640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f809806c8f0 0x7f809806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:33:59.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.118+0000 7f80a97fa640 1 -- 172.21.15.172:0/1846460675 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7f80a008bd50 con 0x7f80ac140fa0 2024-04-14T08:33:59.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.118+0000 7f80b0a34640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f809806c8f0 0x7f809806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:33:59.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.118+0000 7f80b0a34640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f809806c8f0 0x7f809806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f809c036000 tx=0x7f809c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:33:59.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.123+0000 7f80a97fa640 1 -- 172.21.15.172:0/1846460675 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f80a008f030 con 0x7f80ac140fa0 2024-04-14T08:33:59.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.338+0000 7f80b2c98640 1 -- 172.21.15.172:0/1846460675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f80ac068590 con 0x7f80ac140fa0 2024-04-14T08:33:59.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-14T08:33:59.937 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-14T08:33:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:33:59.994+0000 7f80a97fa640 1 -- 172.21.15.172:0/1846460675 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f80a005a050 con 0x7f80ac140fa0 2024-04-14T08:33:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:new fs with metadata pool 5 and data pool 6 2024-04-14T08:34:00.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.007+0000 7f80b2c98640 1 -- 172.21.15.172:0/1846460675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f809806c8f0 msgr2=0x7f809806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.007+0000 7f80b2c98640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f809806c8f0 0x7f809806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f809c036000 tx=0x7f809c008030 comp rx=0 tx=0).stop 2024-04-14T08:34:00.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.007+0000 7f80b2c98640 1 -- 172.21.15.172:0/1846460675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80ac140fa0 msgr2=0x7f80ac13b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.007+0000 7f80b2c98640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80ac140fa0 0x7f80ac13b9b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c7f0 tx=0x7f80a000ccb0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.007+0000 7f80b2c98640 1 -- 172.21.15.172:0/1846460675 shutdown_connections 2024-04-14T08:34:00.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.007+0000 7f80b2c98640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f809806c8f0 0x7f809806edb0 secure :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f809c036000 tx=0x7f809c008030 comp rx=0 tx=0).stop 2024-04-14T08:34:00.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.007+0000 7f80b2c98640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac141380 0x7f80ac156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.007+0000 7f80b2c98640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80ac140fa0 0x7f80ac13b9b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.007+0000 7f80b2c98640 1 --2- 172.21.15.172:0/1846460675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac0b2210 0x7f80ac1f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.007+0000 7f80b2c98640 1 -- 172.21.15.172:0/1846460675 >> 172.21.15.172:0/1846460675 conn(0x7f80ac06b520 msgr2=0x7f80ac072ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:00.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.009+0000 7f80b2c98640 1 -- 172.21.15.172:0/1846460675 shutdown_connections 2024-04-14T08:34:00.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.010+0000 7f80b2c98640 1 -- 172.21.15.172:0/1846460675 wait complete. 2024-04-14T08:34:00.026 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:34:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.127+0000 7f523675e640 1 Processor -- start 2024-04-14T08:34:00.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.127+0000 7f523675e640 1 -- start start 2024-04-14T08:34:00.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.128+0000 7f523675e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f52300b1df0 0x7f52300b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.128+0000 7f523675e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f523015c770 con 0x7f52300b4040 2024-04-14T08:34:00.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.128+0000 7f523675e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52300b3430 con 0x7f52300b21d0 2024-04-14T08:34:00.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.128+0000 7f523675e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f52300b3700 con 0x7f52300b1df0 2024-04-14T08:34:00.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.128+0000 7f522ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f52300b1df0 0x7f52300b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.128+0000 7f5234cfb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f52300b4040 0x7f52300b4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48292/0 (socket says 172.21.15.172:48292) 2024-04-14T08:34:00.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.128+0000 7f5234cfb640 1 -- 172.21.15.172:0/3347842869 learned_addr learned my addr 172.21.15.172:0/3347842869 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:00.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.128+0000 7f522ffff640 1 -- 172.21.15.172:0/3347842869 >> v1:172.21.15.177:6789/0 conn(0x7f52300b21d0 legacy=0x7f523015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:00.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.128+0000 7f522ffff640 1 -- 172.21.15.172:0/3347842869 >> v1:172.21.15.172:6789/0 conn(0x7f52300b4040 legacy=0x7f52300b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.128+0000 7f522effd640 1 -- 172.21.15.172:0/3347842869 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1794196116 0 0) 0x7f523015c770 con 0x7f52300b4040 2024-04-14T08:34:00.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.128+0000 7f522ffff640 1 -- 172.21.15.172:0/3347842869 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52300b39e0 con 0x7f52300b1df0 2024-04-14T08:34:00.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.129+0000 7f522ffff640 1 --2- 172.21.15.172:0/3347842869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300b1df0 0x7f52300b2ef0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f52240097a0 tx=0x7f522402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f11e3ac82b4116 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:00.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.129+0000 7f522effd640 1 -- 172.21.15.172:0/3347842869 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224035020 con 0x7f52300b1df0 2024-04-14T08:34:00.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.129+0000 7f522effd640 1 -- 172.21.15.172:0/3347842869 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5224031830 con 0x7f52300b1df0 2024-04-14T08:34:00.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.129+0000 7f522effd640 1 -- 172.21.15.172:0/3347842869 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224031c00 con 0x7f52300b1df0 2024-04-14T08:34:00.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.129+0000 7f523675e640 1 -- 172.21.15.172:0/3347842869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300b1df0 msgr2=0x7f52300b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.129+0000 7f523675e640 1 --2- 172.21.15.172:0/3347842869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300b1df0 0x7f52300b2ef0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f52240097a0 tx=0x7f522402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.130+0000 7f523675e640 1 -- 172.21.15.172:0/3347842869 shutdown_connections 2024-04-14T08:34:00.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.130+0000 7f523675e640 1 --2- 172.21.15.172:0/3347842869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300b1df0 0x7f52300b2ef0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.130+0000 7f523675e640 1 -- 172.21.15.172:0/3347842869 >> 172.21.15.172:0/3347842869 conn(0x7f523006ec30 msgr2=0x7f523006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:00.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.130+0000 7f523675e640 1 -- 172.21.15.172:0/3347842869 shutdown_connections 2024-04-14T08:34:00.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.130+0000 7f523675e640 1 -- 172.21.15.172:0/3347842869 wait complete. 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.131+0000 7f523675e640 1 Processor -- start 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.131+0000 7f523675e640 1 -- start start 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f523675e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52300b1df0 0x7f52300b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f523675e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300b21d0 0x7f5230156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f523675e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300b4040 0x7f5230153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f523675e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f523015b7f0 con 0x7f52300b21d0 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f523675e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f523015bb40 con 0x7f52300b1df0 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f523675e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f52301552d0 con 0x7f52300b4040 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300b21d0 0x7f5230156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300b21d0 0x7f5230156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48624/0 (socket says 172.21.15.172:48624) 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522f7fe640 1 -- 172.21.15.172:0/1431658908 learned_addr learned my addr 172.21.15.172:0/1431658908 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f5234cfb640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300b4040 0x7f5230153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522ffff640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52300b1df0 0x7f52300b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522f7fe640 1 -- 172.21.15.172:0/1431658908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300b4040 msgr2=0x7f5230153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522f7fe640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300b4040 0x7f5230153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522f7fe640 1 -- 172.21.15.172:0/1431658908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52300b1df0 msgr2=0x7f52300b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522f7fe640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52300b1df0 0x7f52300b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522f7fe640 1 -- 172.21.15.172:0/1431658908 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5230157180 con 0x7f52300b21d0 2024-04-14T08:34:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522f7fe640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300b21d0 0x7f5230156c40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f522000c400 tx=0x7f522000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522cff9640 1 -- 172.21.15.172:0/1431658908 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220011790 con 0x7f52300b21d0 2024-04-14T08:34:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522cff9640 1 -- 172.21.15.172:0/1431658908 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5220017070 con 0x7f52300b21d0 2024-04-14T08:34:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f523675e640 1 -- 172.21.15.172:0/1431658908 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52301f7b20 con 0x7f52300b21d0 2024-04-14T08:34:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f522cff9640 1 -- 172.21.15.172:0/1431658908 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220011bb0 con 0x7f52300b21d0 2024-04-14T08:34:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.132+0000 7f523675e640 1 -- 172.21.15.172:0/1431658908 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52301f80e0 con 0x7f52300b21d0 2024-04-14T08:34:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.133+0000 7f523675e640 1 -- 172.21.15.172:0/1431658908 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5230068520 con 0x7f52300b21d0 2024-04-14T08:34:00.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.134+0000 7f522cff9640 1 -- 172.21.15.172:0/1431658908 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f522001c460 con 0x7f52300b21d0 2024-04-14T08:34:00.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.134+0000 7f522cff9640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f521c06c8f0 0x7f521c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.134+0000 7f522cff9640 1 -- 172.21.15.172:0/1431658908 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f522008ba20 con 0x7f52300b21d0 2024-04-14T08:34:00.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.134+0000 7f522ffff640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f521c06c8f0 0x7f521c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.135+0000 7f522ffff640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f521c06c8f0 0x7f521c06edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5224036000 tx=0x7f522403e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:00.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.139+0000 7f522cff9640 1 -- 172.21.15.172:0/1431658908 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f52200595f0 con 0x7f52300b21d0 2024-04-14T08:34:00.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.319+0000 7f523675e640 1 -- 172.21.15.172:0/1431658908 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f523015b1c0 con 0x7f52300b21d0 2024-04-14T08:34:00.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.320+0000 7f522cff9640 1 -- 172.21.15.172:0/1431658908 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+17015 (secure 0 0 0) 0x7f522005d8f0 con 0x7f52300b21d0 2024-04-14T08:34:00.322 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:00.322 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":38,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:33:59.340123+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:33:57.868995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-14T08:33:58.323549+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:34:00.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.322+0000 7f523675e640 1 -- 172.21.15.172:0/1431658908 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f521c06c8f0 msgr2=0x7f521c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.322+0000 7f523675e640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f521c06c8f0 0x7f521c06edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5224036000 tx=0x7f522403e040 comp rx=0 tx=0).stop 2024-04-14T08:34:00.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.322+0000 7f523675e640 1 -- 172.21.15.172:0/1431658908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300b21d0 msgr2=0x7f5230156c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.322+0000 7f523675e640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300b21d0 0x7f5230156c40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f522000c400 tx=0x7f522000c8c0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.322+0000 7f523675e640 1 -- 172.21.15.172:0/1431658908 shutdown_connections 2024-04-14T08:34:00.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.322+0000 7f523675e640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f521c06c8f0 0x7f521c06edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.322+0000 7f523675e640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300b4040 0x7f5230153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.322+0000 7f523675e640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300b21d0 0x7f5230156c40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.322+0000 7f523675e640 1 --2- 172.21.15.172:0/1431658908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52300b1df0 0x7f52300b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.322+0000 7f523675e640 1 -- 172.21.15.172:0/1431658908 >> 172.21.15.172:0/1431658908 conn(0x7f523006ec30 msgr2=0x7f5230154920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:00.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.323+0000 7f523675e640 1 -- 172.21.15.172:0/1431658908 shutdown_connections 2024-04-14T08:34:00.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.323+0000 7f523675e640 1 -- 172.21.15.172:0/1431658908 wait complete. 2024-04-14T08:34:00.335 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:34:00.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.434+0000 7f0d6e8a2640 1 Processor -- start 2024-04-14T08:34:00.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.434+0000 7f0d6e8a2640 1 -- start start 2024-04-14T08:34:00.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d6e8a2640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0d6815d980 0x7f0d68160550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d6e8a2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d68160a90 0x7f0d680b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d6e8a2640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d6815d730 con 0x7f0d68160170 2024-04-14T08:34:00.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d6e8a2640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f0d6813f910 con 0x7f0d6815d980 2024-04-14T08:34:00.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d6e8a2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0d6813fbf0 con 0x7f0d68160a90 2024-04-14T08:34:00.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d67fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f0d68160170 0x7f0d6815d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48294/0 (socket says 172.21.15.172:48294) 2024-04-14T08:34:00.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d67fff640 1 -- 172.21.15.172:0/3684337001 learned_addr learned my addr 172.21.15.172:0/3684337001 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:00.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d6ce3f640 1 --2- 172.21.15.172:0/3684337001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d68160a90 0x7f0d680b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d677fe640 1 --2- 172.21.15.172:0/3684337001 >> v2:172.21.15.177:3300/0 conn(0x7f0d6815d980 0x7f0d68160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d66ffd640 1 -- 172.21.15.172:0/3684337001 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2470443978 0 0) 0x7f0d6815d730 con 0x7f0d68160170 2024-04-14T08:34:00.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d66ffd640 1 -- 172.21.15.172:0/3684337001 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d50003520 con 0x7f0d68160170 2024-04-14T08:34:00.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d66ffd640 1 -- 172.21.15.172:0/3684337001 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 42641364 0 0) 0x7f0d50003520 con 0x7f0d68160170 2024-04-14T08:34:00.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d6ce3f640 1 -- 172.21.15.172:0/3684337001 >> v2:172.21.15.177:3300/0 conn(0x7f0d6815d980 msgr2=0x7f0d68160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:00.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d6ce3f640 1 --2- 172.21.15.172:0/3684337001 >> v2:172.21.15.177:3300/0 conn(0x7f0d6815d980 0x7f0d68160550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d6ce3f640 1 -- 172.21.15.172:0/3684337001 >> v1:172.21.15.172:6789/0 conn(0x7f0d68160170 legacy=0x7f0d6815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.435+0000 7f0d6ce3f640 1 -- 172.21.15.172:0/3684337001 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6813fed0 con 0x7f0d68160a90 2024-04-14T08:34:00.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.436+0000 7f0d6ce3f640 1 --2- 172.21.15.172:0/3684337001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d68160a90 0x7f0d680b1b80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009d90 tx=0x7f0d6002bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65f7664692db54b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:00.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.436+0000 7f0d66ffd640 1 -- 172.21.15.172:0/3684337001 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60039070 con 0x7f0d68160a90 2024-04-14T08:34:00.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.436+0000 7f0d66ffd640 1 -- 172.21.15.172:0/3684337001 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d6002c8a0 con 0x7f0d68160a90 2024-04-14T08:34:00.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.436+0000 7f0d66ffd640 1 -- 172.21.15.172:0/3684337001 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58001690 con 0x7f0d68160a90 2024-04-14T08:34:00.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.437+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3684337001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d68160a90 msgr2=0x7f0d680b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.437+0000 7f0d6e8a2640 1 --2- 172.21.15.172:0/3684337001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d68160a90 0x7f0d680b1b80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009d90 tx=0x7f0d6002bd70 comp rx=0 tx=0).stop 2024-04-14T08:34:00.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.437+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3684337001 shutdown_connections 2024-04-14T08:34:00.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.437+0000 7f0d6e8a2640 1 --2- 172.21.15.172:0/3684337001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d68160a90 0x7f0d680b1b80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.437+0000 7f0d6e8a2640 1 --2- 172.21.15.172:0/3684337001 >> v2:172.21.15.177:3300/0 conn(0x7f0d6815d980 0x7f0d68160550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.437+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3684337001 >> 172.21.15.172:0/3684337001 conn(0x7f0d6806ec30 msgr2=0x7f0d6806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:00.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.437+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3684337001 shutdown_connections 2024-04-14T08:34:00.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.437+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3684337001 wait complete. 2024-04-14T08:34:00.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.438+0000 7f0d6e8a2640 1 Processor -- start 2024-04-14T08:34:00.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.438+0000 7f0d6e8a2640 1 -- start start 2024-04-14T08:34:00.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d6e8a2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d6815d980 0x7f0d68155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d6e8a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d68160170 0x7f0d6815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d6e8a2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d68160a90 0x7f0d6813dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d6e8a2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d680b20c0 con 0x7f0d68160170 2024-04-14T08:34:00.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d6e8a2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d68140490 con 0x7f0d6815d980 2024-04-14T08:34:00.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d6e8a2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0d680b2240 con 0x7f0d68160a90 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d68160170 0x7f0d6815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d68160170 0x7f0d6815a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48636/0 (socket says 172.21.15.172:48636) 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d677fe640 1 -- 172.21.15.172:0/3636261422 learned_addr learned my addr 172.21.15.172:0/3636261422 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d6ce3f640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d68160a90 0x7f0d6813dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d67fff640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d6815d980 0x7f0d68155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d677fe640 1 -- 172.21.15.172:0/3636261422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d68160a90 msgr2=0x7f0d6813dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d677fe640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d68160a90 0x7f0d6813dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d677fe640 1 -- 172.21.15.172:0/3636261422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d6815d980 msgr2=0x7f0d68155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d677fe640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d6815d980 0x7f0d68155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d677fe640 1 -- 172.21.15.172:0/3636261422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6815b310 con 0x7f0d68160170 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d6ce3f640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d68160a90 0x7f0d6813dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d67fff640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d6815d980 0x7f0d68155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.439+0000 7f0d677fe640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d68160170 0x7f0d6815a890 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00e9b0 tx=0x7f0d5c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:00.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.440+0000 7f0d64ff9640 1 -- 172.21.15.172:0/3636261422 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c017070 con 0x7f0d68160170 2024-04-14T08:34:00.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.440+0000 7f0d64ff9640 1 -- 172.21.15.172:0/3636261422 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5c00c9a0 con 0x7f0d68160170 2024-04-14T08:34:00.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.440+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3636261422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6815b5f0 con 0x7f0d68160170 2024-04-14T08:34:00.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.440+0000 7f0d64ff9640 1 -- 172.21.15.172:0/3636261422 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c00cd90 con 0x7f0d68160170 2024-04-14T08:34:00.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.440+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3636261422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d6815b8d0 con 0x7f0d68160170 2024-04-14T08:34:00.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.441+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3636261422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d68068520 con 0x7f0d68160170 2024-04-14T08:34:00.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.441+0000 7f0d64ff9640 1 -- 172.21.15.172:0/3636261422 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d5c01a070 con 0x7f0d68160170 2024-04-14T08:34:00.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.441+0000 7f0d64ff9640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0d5406c8f0 0x7f0d5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.441+0000 7f0d64ff9640 1 -- 172.21.15.172:0/3636261422 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0d5c08bfe0 con 0x7f0d68160170 2024-04-14T08:34:00.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.441+0000 7f0d67fff640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0d5406c8f0 0x7f0d5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.442+0000 7f0d67fff640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0d5406c8f0 0x7f0d5406edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0d58001800 tx=0x7f0d58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:00.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.446+0000 7f0d64ff9640 1 -- 172.21.15.172:0/3636261422 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0d5c05a310 con 0x7f0d68160170 2024-04-14T08:34:00.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.623+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3636261422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d6806c2c0 con 0x7f0d68160170 2024-04-14T08:34:00.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.624+0000 7f0d64ff9640 1 -- 172.21.15.172:0/3636261422 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+17015 (secure 0 0 0) 0x7f0d5c059c10 con 0x7f0d68160170 2024-04-14T08:34:00.626 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:00.626 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":38,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:33:59.340123+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:33:57.868995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-14T08:33:58.323549+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:34:00.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.626+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3636261422 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0d5406c8f0 msgr2=0x7f0d5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.626+0000 7f0d6e8a2640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0d5406c8f0 0x7f0d5406edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0d58001800 tx=0x7f0d58009040 comp rx=0 tx=0).stop 2024-04-14T08:34:00.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.626+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3636261422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d68160170 msgr2=0x7f0d6815a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.626+0000 7f0d6e8a2640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d68160170 0x7f0d6815a890 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00e9b0 tx=0x7f0d5c00ee70 comp rx=0 tx=0).stop 2024-04-14T08:34:00.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.627+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3636261422 shutdown_connections 2024-04-14T08:34:00.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.627+0000 7f0d6e8a2640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0d5406c8f0 0x7f0d5406edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.627+0000 7f0d6e8a2640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d68160a90 0x7f0d6813dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.627+0000 7f0d6e8a2640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d68160170 0x7f0d6815a890 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.627+0000 7f0d6e8a2640 1 --2- 172.21.15.172:0/3636261422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d6815d980 0x7f0d68155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.627+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3636261422 >> 172.21.15.172:0/3636261422 conn(0x7f0d6806ec30 msgr2=0x7f0d6815cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:00.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.627+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3636261422 shutdown_connections 2024-04-14T08:34:00.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.627+0000 7f0d6e8a2640 1 -- 172.21.15.172:0/3636261422 wait complete. 2024-04-14T08:34:00.639 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-14T08:34:00.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.741+0000 7f1ef34de640 1 Processor -- start 2024-04-14T08:34:00.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.741+0000 7f1ef34de640 1 -- start start 2024-04-14T08:34:00.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.742+0000 7f1ef34de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1eec0cea60 0x7f1eec1629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.742+0000 7f1ef34de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eec0b3750 con 0x7f1eec15e590 2024-04-14T08:34:00.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.742+0000 7f1ef34de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eec0cef20 con 0x7f1eec0b65a0 2024-04-14T08:34:00.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.742+0000 7f1ef34de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f1eec0cf200 con 0x7f1eec0cea60 2024-04-14T08:34:00.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.742+0000 7f1ef127a640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1eec15e590 0x7f1eec0b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48304/0 (socket says 172.21.15.172:48304) 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.742+0000 7f1ef127a640 1 -- 172.21.15.172:0/1902358532 learned_addr learned my addr 172.21.15.172:0/1902358532 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ef1a7b640 1 --2- 172.21.15.172:0/1902358532 >> v2:172.21.15.177:3300/0 conn(0x7f1eec0cea60 0x7f1eec1629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1832136944 0 0) 0x7f1eec0b3750 con 0x7f1eec15e590 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ed8003520 con 0x7f1eec15e590 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2263307843 0 0) 0x7f1eec0cef20 con 0x7f1eec0b65a0 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1eec0b3750 con 0x7f1eec0b65a0 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3062438585 0 0) 0x7f1ed8003520 con 0x7f1eec15e590 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 >> v2:172.21.15.177:3300/0 conn(0x7f1eec0cea60 msgr2=0x7f1eec1629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 --2- 172.21.15.172:0/1902358532 >> v2:172.21.15.177:3300/0 conn(0x7f1eec0cea60 0x7f1eec1629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 >> v1:172.21.15.177:6789/0 conn(0x7f1eec0b65a0 legacy=0x7f1eec0d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec0cf4e0 con 0x7f1eec15e590 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f1ee80019f0 con 0x7f1eec15e590 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 >> v1:172.21.15.172:6789/0 conn(0x7f1eec15e590 legacy=0x7f1eec0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 --2- 172.21.15.172:0/1902358532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed8003b40 0x7f1ed8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 --2- 172.21.15.172:0/1902358532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed8004800 0x7f1ed8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 --2- 172.21.15.172:0/1902358532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed80051c0 0x7f1ed800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec0cf200 con 0x7f1ed8004800 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed800c040 con 0x7f1ed8003b40 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed800c240 con 0x7f1ed80051c0 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ef0a79640 1 --2- 172.21.15.172:0/1902358532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed8004800 0x7f1ed8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ef127a640 1 --2- 172.21.15.172:0/1902358532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed8003b40 0x7f1ed8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ef1a7b640 1 --2- 172.21.15.172:0/1902358532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed80051c0 0x7f1ed800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ef0a79640 1 -- 172.21.15.172:0/1902358532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed80051c0 msgr2=0x7f1ed800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ef0a79640 1 --2- 172.21.15.172:0/1902358532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed80051c0 0x7f1ed800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ef0a79640 1 -- 172.21.15.172:0/1902358532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed8003b40 msgr2=0x7f1ed8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ef0a79640 1 --2- 172.21.15.172:0/1902358532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed8003b40 0x7f1ed8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ef0a79640 1 -- 172.21.15.172:0/1902358532 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed8003520 con 0x7f1ed8004800 2024-04-14T08:34:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.743+0000 7f1ef1a7b640 1 -- 172.21.15.172:0/1902358532 reap_dead start 2024-04-14T08:34:00.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.744+0000 7f1ef0a79640 1 --2- 172.21.15.172:0/1902358532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed8004800 0x7f1ed8004c80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0018d0 tx=0x7f1ed8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f7ebd40f5b0f25 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:00.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.744+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc037070 con 0x7f1ed8004800 2024-04-14T08:34:00.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.744+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1edc0052d0 con 0x7f1ed8004800 2024-04-14T08:34:00.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.744+0000 7f1ee3fff640 1 -- 172.21.15.172:0/1902358532 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0014f0 con 0x7f1ed8004800 2024-04-14T08:34:00.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.745+0000 7f1ef34de640 1 -- 172.21.15.172:0/1902358532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed8004800 msgr2=0x7f1ed8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.745+0000 7f1ef34de640 1 --2- 172.21.15.172:0/1902358532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed8004800 0x7f1ed8004c80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0018d0 tx=0x7f1ed8005830 comp rx=0 tx=0).stop 2024-04-14T08:34:00.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.745+0000 7f1ef34de640 1 -- 172.21.15.172:0/1902358532 shutdown_connections 2024-04-14T08:34:00.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.745+0000 7f1ef34de640 1 --2- 172.21.15.172:0/1902358532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed8004800 0x7f1ed8004c80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.745+0000 7f1ef34de640 1 -- 172.21.15.172:0/1902358532 >> 172.21.15.172:0/1902358532 conn(0x7f1eec0d0e90 msgr2=0x7f1eec0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:00.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.745+0000 7f1ef34de640 1 -- 172.21.15.172:0/1902358532 shutdown_connections 2024-04-14T08:34:00.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.745+0000 7f1ef34de640 1 -- 172.21.15.172:0/1902358532 wait complete. 2024-04-14T08:34:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.746+0000 7f1ef34de640 1 Processor -- start 2024-04-14T08:34:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.746+0000 7f1ef34de640 1 -- start start 2024-04-14T08:34:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef34de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed8004800 0x7f1eec0ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef34de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15f280 0x7f1eec15f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef34de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1eec0cd9c0 0x7f1eec0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef34de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec162b40 con 0x7f1eec15f280 2024-04-14T08:34:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef34de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec1629c0 con 0x7f1ed8004800 2024-04-14T08:34:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef34de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1eec162cc0 con 0x7f1eec0cd9c0 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef0a79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15f280 0x7f1eec15f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef0a79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15f280 0x7f1eec15f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48656/0 (socket says 172.21.15.172:48656) 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef0a79640 1 -- 172.21.15.172:0/3426471752 learned_addr learned my addr 172.21.15.172:0/3426471752 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef0a79640 1 -- 172.21.15.172:0/3426471752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1eec0cd9c0 msgr2=0x7f1eec0cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef0a79640 1 --2- 172.21.15.172:0/3426471752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1eec0cd9c0 0x7f1eec0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef0a79640 1 -- 172.21.15.172:0/3426471752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed8004800 msgr2=0x7f1eec0ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef0a79640 1 --2- 172.21.15.172:0/3426471752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed8004800 0x7f1eec0ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef0a79640 1 -- 172.21.15.172:0/3426471752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec15fdd0 con 0x7f1eec15f280 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef0a79640 1 --2- 172.21.15.172:0/3426471752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15f280 0x7f1eec15f700 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1edc035040 tx=0x7f1edc0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ee1ffb640 1 -- 172.21.15.172:0/3426471752 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc042020 con 0x7f1eec15f280 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ee1ffb640 1 -- 172.21.15.172:0/3426471752 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1edc038040 con 0x7f1eec15f280 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ee1ffb640 1 -- 172.21.15.172:0/3426471752 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc037070 con 0x7f1eec15f280 2024-04-14T08:34:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.747+0000 7f1ef34de640 1 -- 172.21.15.172:0/3426471752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eec1600b0 con 0x7f1eec15f280 2024-04-14T08:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.748+0000 7f1ef34de640 1 -- 172.21.15.172:0/3426471752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eec149a50 con 0x7f1eec15f280 2024-04-14T08:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.748+0000 7f1ef34de640 1 -- 172.21.15.172:0/3426471752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eec068520 con 0x7f1eec15f280 2024-04-14T08:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.749+0000 7f1ee1ffb640 1 -- 172.21.15.172:0/3426471752 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1edc045020 con 0x7f1eec15f280 2024-04-14T08:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.749+0000 7f1ee1ffb640 1 --2- 172.21.15.172:0/3426471752 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ed006c8f0 0x7f1ed006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.749+0000 7f1ee1ffb640 1 -- 172.21.15.172:0/3426471752 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f1edc0ad4e0 con 0x7f1eec15f280 2024-04-14T08:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.749+0000 7f1ef127a640 1 --2- 172.21.15.172:0/3426471752 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ed006c8f0 0x7f1ed006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:00.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.750+0000 7f1ef127a640 1 --2- 172.21.15.172:0/3426471752 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ed006c8f0 0x7f1ed006edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80012e0 tx=0x7f1ee8001250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:00.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.754+0000 7f1ee1ffb640 1 -- 172.21.15.172:0/3426471752 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1edc04c030 con 0x7f1eec15f280 2024-04-14T08:34:00.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.965+0000 7f1ef34de640 1 -- 172.21.15.172:0/3426471752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1eec0d1bb0 con 0x7f1eec15f280 2024-04-14T08:34:01.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:00.999+0000 7f1ee1ffb640 1 -- 172.21.15.172:0/3426471752 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f1edc07b780 con 0x7f1eec15f280 2024-04-14T08:34:01.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.001+0000 7f1ef34de640 1 -- 172.21.15.172:0/3426471752 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ed006c8f0 msgr2=0x7f1ed006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:01.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.001+0000 7f1ef34de640 1 --2- 172.21.15.172:0/3426471752 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ed006c8f0 0x7f1ed006edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80012e0 tx=0x7f1ee8001250 comp rx=0 tx=0).stop 2024-04-14T08:34:01.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.001+0000 7f1ef34de640 1 -- 172.21.15.172:0/3426471752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15f280 msgr2=0x7f1eec15f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:01.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.001+0000 7f1ef34de640 1 --2- 172.21.15.172:0/3426471752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15f280 0x7f1eec15f700 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1edc035040 tx=0x7f1edc0012d0 comp rx=0 tx=0).stop 2024-04-14T08:34:01.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.001+0000 7f1ef34de640 1 -- 172.21.15.172:0/3426471752 shutdown_connections 2024-04-14T08:34:01.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.001+0000 7f1ef34de640 1 --2- 172.21.15.172:0/3426471752 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ed006c8f0 0x7f1ed006edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:01.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.001+0000 7f1ef34de640 1 --2- 172.21.15.172:0/3426471752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1eec0cd9c0 0x7f1eec0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:01.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.001+0000 7f1ef34de640 1 --2- 172.21.15.172:0/3426471752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1eec15f280 0x7f1eec15f700 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:01.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.001+0000 7f1ef34de640 1 --2- 172.21.15.172:0/3426471752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed8004800 0x7f1eec0ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:01.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.001+0000 7f1ef34de640 1 -- 172.21.15.172:0/3426471752 >> 172.21.15.172:0/3426471752 conn(0x7f1eec0d0e90 msgr2=0x7f1eec0b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:01.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.001+0000 7f1ef34de640 1 -- 172.21.15.172:0/3426471752 shutdown_connections 2024-04-14T08:34:01.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.001+0000 7f1ef34de640 1 -- 172.21.15.172:0/3426471752 wait complete. 2024-04-14T08:34:01.015 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-14T08:34:01.015 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-14T08:34:01.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.117+0000 7f28ba71e640 1 Processor -- start 2024-04-14T08:34:01.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.117+0000 7f28ba71e640 1 -- start start 2024-04-14T08:34:01.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.118+0000 7f28ba71e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f28b4161ce0 0x7f28b40d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:01.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.118+0000 7f28ba71e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28b40d1320 0x7f28b40b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:01.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.118+0000 7f28ba71e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28b40cedc0 con 0x7f28b40d2d40 2024-04-14T08:34:01.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.118+0000 7f28ba71e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f28b40d31a0 con 0x7f28b4161ce0 2024-04-14T08:34:01.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.118+0000 7f28ba71e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f28b40d00e0 con 0x7f28b40d1320 2024-04-14T08:34:01.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.118+0000 7f28b37fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f28b40d2d40 0x7f28b40d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48306/0 (socket says 172.21.15.172:48306) 2024-04-14T08:34:01.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.118+0000 7f28b37fe640 1 -- 172.21.15.172:0/2626890703 learned_addr learned my addr 172.21.15.172:0/2626890703 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:01.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.118+0000 7f28b8cbb640 1 --2- 172.21.15.172:0/2626890703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28b40d1320 0x7f28b40b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:01.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.118+0000 7f28b3fff640 1 --2- 172.21.15.172:0/2626890703 >> v2:172.21.15.177:3300/0 conn(0x7f28b4161ce0 0x7f28b40d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:01.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.118+0000 7f28b2ffd640 1 -- 172.21.15.172:0/2626890703 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1961055291 0 0) 0x7f28a4001110 con 0x7f28b40d2d40 2024-04-14T08:34:01.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.119+0000 7f28b2ffd640 1 -- 172.21.15.172:0/2626890703 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f289c003520 con 0x7f28b40d2d40 2024-04-14T08:34:01.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.119+0000 7f28b8cbb640 1 -- 172.21.15.172:0/2626890703 >> v2:172.21.15.177:3300/0 conn(0x7f28b4161ce0 msgr2=0x7f28b40d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:01.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.119+0000 7f28b8cbb640 1 --2- 172.21.15.172:0/2626890703 >> v2:172.21.15.177:3300/0 conn(0x7f28b4161ce0 0x7f28b40d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:01.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.119+0000 7f28b8cbb640 1 -- 172.21.15.172:0/2626890703 >> v1:172.21.15.172:6789/0 conn(0x7f28b40d2d40 legacy=0x7f28b40d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:01.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.119+0000 7f28b2ffd640 1 -- 172.21.15.172:0/2626890703 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3317730273 0 0) 0x7f289c003520 con 0x7f28b40d2d40 2024-04-14T08:34:01.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.119+0000 7f28b8cbb640 1 -- 172.21.15.172:0/2626890703 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b40cf040 con 0x7f28b40d1320 2024-04-14T08:34:01.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.120+0000 7f28b8cbb640 1 --2- 172.21.15.172:0/2626890703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28b40d1320 0x7f28b40b0e50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009d90 tx=0x7f28ac02bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51aad80307ed543c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:01.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.120+0000 7f28b2ffd640 1 -- 172.21.15.172:0/2626890703 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac039070 con 0x7f28b40d1320 2024-04-14T08:34:01.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.120+0000 7f28b2ffd640 1 -- 172.21.15.172:0/2626890703 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ac02c8a0 con 0x7f28b40d1320 2024-04-14T08:34:01.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.120+0000 7f28b2ffd640 1 -- 172.21.15.172:0/2626890703 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4001840 con 0x7f28b40d1320 2024-04-14T08:34:01.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.120+0000 7f28ba71e640 1 -- 172.21.15.172:0/2626890703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28b40d1320 msgr2=0x7f28b40b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:01.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.120+0000 7f28ba71e640 1 --2- 172.21.15.172:0/2626890703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28b40d1320 0x7f28b40b0e50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009d90 tx=0x7f28ac02bd70 comp rx=0 tx=0).stop 2024-04-14T08:34:01.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.120+0000 7f28ba71e640 1 -- 172.21.15.172:0/2626890703 shutdown_connections 2024-04-14T08:34:01.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.120+0000 7f28ba71e640 1 --2- 172.21.15.172:0/2626890703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28b40d1320 0x7f28b40b0e50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:01.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.120+0000 7f28ba71e640 1 --2- 172.21.15.172:0/2626890703 >> v2:172.21.15.177:3300/0 conn(0x7f28b4161ce0 0x7f28b40d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:01.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.120+0000 7f28ba71e640 1 -- 172.21.15.172:0/2626890703 >> 172.21.15.172:0/2626890703 conn(0x7f28b409d560 msgr2=0x7f28b40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:01.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.120+0000 7f28ba71e640 1 -- 172.21.15.172:0/2626890703 shutdown_connections 2024-04-14T08:34:01.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.120+0000 7f28ba71e640 1 -- 172.21.15.172:0/2626890703 wait complete. 2024-04-14T08:34:01.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.121+0000 7f28ba71e640 1 Processor -- start 2024-04-14T08:34:01.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.122+0000 7f28ba71e640 1 -- start start 2024-04-14T08:34:01.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.122+0000 7f28ba71e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28b40d1320 0x7f28b41565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:01.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.122+0000 7f28ba71e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28b40d2d40 0x7f28b40b9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:01.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.122+0000 7f28ba71e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28b4161ce0 0x7f28b415f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.122+0000 7f28ba71e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f28b40cf8e0 con 0x7f28b4161ce0 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.122+0000 7f28ba71e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f28b40cf600 con 0x7f28b40d1320 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.122+0000 7f28ba71e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f28b40cfbc0 con 0x7f28b40d2d40 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.122+0000 7f28b8cbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28b4161ce0 0x7f28b415f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b8cbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28b4161ce0 0x7f28b415f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48662/0 (socket says 172.21.15.172:48662) 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b8cbb640 1 -- 172.21.15.172:0/2123928010 learned_addr learned my addr 172.21.15.172:0/2123928010 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b37fe640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28b40d2d40 0x7f28b40b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b3fff640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28b40d1320 0x7f28b41565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b8cbb640 1 -- 172.21.15.172:0/2123928010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28b40d2d40 msgr2=0x7f28b40b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b8cbb640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28b40d2d40 0x7f28b40b9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b8cbb640 1 -- 172.21.15.172:0/2123928010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28b40d1320 msgr2=0x7f28b41565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b8cbb640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28b40d1320 0x7f28b41565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:01.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b8cbb640 1 -- 172.21.15.172:0/2123928010 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b40b9f00 con 0x7f28b4161ce0 2024-04-14T08:34:01.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b8cbb640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28b4161ce0 0x7f28b415f2d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f28ac02c380 tx=0x7f28ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:01.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b0ff9640 1 -- 172.21.15.172:0/2123928010 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac039050 con 0x7f28b4161ce0 2024-04-14T08:34:01.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b0ff9640 1 -- 172.21.15.172:0/2123928010 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ac045070 con 0x7f28b4161ce0 2024-04-14T08:34:01.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28ba71e640 1 -- 172.21.15.172:0/2123928010 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b40c80b0 con 0x7f28b4161ce0 2024-04-14T08:34:01.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28b0ff9640 1 -- 172.21.15.172:0/2123928010 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac009450 con 0x7f28b4161ce0 2024-04-14T08:34:01.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.123+0000 7f28ba71e640 1 -- 172.21.15.172:0/2123928010 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b40c8670 con 0x7f28b4161ce0 2024-04-14T08:34:01.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.124+0000 7f28ba71e640 1 -- 172.21.15.172:0/2123928010 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b4075490 con 0x7f28b4161ce0 2024-04-14T08:34:01.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.125+0000 7f28b0ff9640 1 -- 172.21.15.172:0/2123928010 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28ac004650 con 0x7f28b4161ce0 2024-04-14T08:34:01.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.125+0000 7f28b0ff9640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f28a006c8f0 0x7f28a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:01.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.125+0000 7f28b0ff9640 1 -- 172.21.15.172:0/2123928010 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f28ac0ae390 con 0x7f28b4161ce0 2024-04-14T08:34:01.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.125+0000 7f28b3fff640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f28a006c8f0 0x7f28a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:01.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.126+0000 7f28b3fff640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f28a006c8f0 0x7f28a006edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009aa0 tx=0x7f28a8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:01.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.130+0000 7f28b0ff9640 1 -- 172.21.15.172:0/2123928010 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f28ac07c100 con 0x7f28b4161ce0 2024-04-14T08:34:01.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:01.465+0000 7f28ba71e640 1 -- 172.21.15.172:0/2123928010 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f28b40d25e0 con 0x7f28b4161ce0 2024-04-14T08:34:02.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.004+0000 7f28b0ff9640 1 -- 172.21.15.172:0/2123928010 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f28ac080400 con 0x7f28b4161ce0 2024-04-14T08:34:02.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.006+0000 7f28ba71e640 1 -- 172.21.15.172:0/2123928010 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f28a006c8f0 msgr2=0x7f28a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.006+0000 7f28ba71e640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f28a006c8f0 0x7f28a006edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009aa0 tx=0x7f28a8009190 comp rx=0 tx=0).stop 2024-04-14T08:34:02.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.006+0000 7f28ba71e640 1 -- 172.21.15.172:0/2123928010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28b4161ce0 msgr2=0x7f28b415f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.006+0000 7f28ba71e640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28b4161ce0 0x7f28b415f2d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f28ac02c380 tx=0x7f28ac004a40 comp rx=0 tx=0).stop 2024-04-14T08:34:02.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.006+0000 7f28ba71e640 1 -- 172.21.15.172:0/2123928010 shutdown_connections 2024-04-14T08:34:02.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.006+0000 7f28ba71e640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f28a006c8f0 0x7f28a006edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.006+0000 7f28ba71e640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28b4161ce0 0x7f28b415f2d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.006+0000 7f28ba71e640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28b40d2d40 0x7f28b40b9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.006+0000 7f28ba71e640 1 --2- 172.21.15.172:0/2123928010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28b40d1320 0x7f28b41565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.006+0000 7f28ba71e640 1 -- 172.21.15.172:0/2123928010 >> 172.21.15.172:0/2123928010 conn(0x7f28b409d560 msgr2=0x7f28b4157730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:02.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.006+0000 7f28ba71e640 1 -- 172.21.15.172:0/2123928010 shutdown_connections 2024-04-14T08:34:02.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.006+0000 7f28ba71e640 1 -- 172.21.15.172:0/2123928010 wait complete. 2024-04-14T08:34:02.018 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-14T08:34:02.018 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:34:02.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.119+0000 7fddfa1f0640 1 Processor -- start 2024-04-14T08:34:02.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.119+0000 7fddfa1f0640 1 -- start start 2024-04-14T08:34:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddfa1f0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fddf40d2830 0x7fddf40d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddfa1f0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fddf415e890 0x7fddf40d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddfa1f0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf40d3150 con 0x7fddf40cedd0 2024-04-14T08:34:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddfa1f0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fddf40cff30 con 0x7fddf40d2830 2024-04-14T08:34:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddfa1f0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fddf40d2350 con 0x7fddf415e890 2024-04-14T08:34:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddf37fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fddf40d2830 0x7fddf40d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddf37fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fddf40d2830 0x7fddf40d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48672/0 (socket says 172.21.15.172:48672) 2024-04-14T08:34:02.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddf37fe640 1 -- 172.21.15.172:0/3323775702 learned_addr learned my addr 172.21.15.172:0/3323775702 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:02.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddf2ffd640 1 --2- 172.21.15.172:0/3323775702 >> v2:172.21.15.177:3300/0 conn(0x7fddf415e890 0x7fddf40d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddf37fe640 1 -- 172.21.15.172:0/3323775702 >> v2:172.21.15.177:3300/0 conn(0x7fddf415e890 msgr2=0x7fddf40d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:02.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddf37fe640 1 --2- 172.21.15.172:0/3323775702 >> v2:172.21.15.177:3300/0 conn(0x7fddf415e890 0x7fddf40d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddf37fe640 1 -- 172.21.15.172:0/3323775702 >> v1:172.21.15.177:6789/0 conn(0x7fddf40cedd0 legacy=0x7fddf41621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.121+0000 7fddf37fe640 1 -- 172.21.15.172:0/3323775702 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf40cf320 con 0x7fddf40d2830 2024-04-14T08:34:02.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.122+0000 7fddf37fe640 1 --2- 172.21.15.172:0/3323775702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddf40d2830 0x7fddf40d2c10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fddf4162100 tx=0x7fdde402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63a63062d900d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:02.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.122+0000 7fddf27fc640 1 -- 172.21.15.172:0/3323775702 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde402f850 con 0x7fddf40d2830 2024-04-14T08:34:02.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.122+0000 7fddf27fc640 1 -- 172.21.15.172:0/3323775702 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde402f9f0 con 0x7fddf40d2830 2024-04-14T08:34:02.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.122+0000 7fddf27fc640 1 -- 172.21.15.172:0/3323775702 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde402fdc0 con 0x7fddf40d2830 2024-04-14T08:34:02.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.123+0000 7fddfa1f0640 1 -- 172.21.15.172:0/3323775702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddf40d2830 msgr2=0x7fddf40d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.123+0000 7fddfa1f0640 1 --2- 172.21.15.172:0/3323775702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddf40d2830 0x7fddf40d2c10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fddf4162100 tx=0x7fdde402ba30 comp rx=0 tx=0).stop 2024-04-14T08:34:02.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.123+0000 7fddfa1f0640 1 -- 172.21.15.172:0/3323775702 shutdown_connections 2024-04-14T08:34:02.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.123+0000 7fddfa1f0640 1 --2- 172.21.15.172:0/3323775702 >> v2:172.21.15.177:3300/0 conn(0x7fddf415e890 0x7fddf40d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.123+0000 7fddfa1f0640 1 --2- 172.21.15.172:0/3323775702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddf40d2830 0x7fddf40d2c10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.123+0000 7fddfa1f0640 1 -- 172.21.15.172:0/3323775702 >> 172.21.15.172:0/3323775702 conn(0x7fddf409d590 msgr2=0x7fddf40d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:02.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.123+0000 7fddfa1f0640 1 -- 172.21.15.172:0/3323775702 shutdown_connections 2024-04-14T08:34:02.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.123+0000 7fddfa1f0640 1 -- 172.21.15.172:0/3323775702 wait complete. 2024-04-14T08:34:02.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.124+0000 7fddfa1f0640 1 Processor -- start 2024-04-14T08:34:02.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.124+0000 7fddfa1f0640 1 -- start start 2024-04-14T08:34:02.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddfa1f0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddf40cedd0 0x7fddf41570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddfa1f0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddf40d2830 0x7fddf415cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddfa1f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddf415e890 0x7fddf415d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddfa1f0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fddf40cfbc0 con 0x7fddf415e890 2024-04-14T08:34:02.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddfa1f0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fddf40cf8e0 con 0x7fddf40cedd0 2024-04-14T08:34:02.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddfa1f0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fddf40d1840 con 0x7fddf40d2830 2024-04-14T08:34:02.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddf3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddf415e890 0x7fddf415d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddf3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddf415e890 0x7fddf415d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48674/0 (socket says 172.21.15.172:48674) 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddf3fff640 1 -- 172.21.15.172:0/4180971952 learned_addr learned my addr 172.21.15.172:0/4180971952 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddf37fe640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddf40cedd0 0x7fddf41570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddf2ffd640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddf40d2830 0x7fddf415cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddf3fff640 1 -- 172.21.15.172:0/4180971952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddf40d2830 msgr2=0x7fddf415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddf3fff640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddf40d2830 0x7fddf415cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddf3fff640 1 -- 172.21.15.172:0/4180971952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddf40cedd0 msgr2=0x7fddf41570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddf3fff640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddf40cedd0 0x7fddf41570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.125+0000 7fddf3fff640 1 -- 172.21.15.172:0/4180971952 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf40c9ac0 con 0x7fddf415e890 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.126+0000 7fddf3fff640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddf415e890 0x7fddf415d490 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fddec00c6e0 tx=0x7fddec00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.126+0000 7fdddbfff640 1 -- 172.21.15.172:0/4180971952 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec003b70 con 0x7fddf415e890 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.126+0000 7fdddbfff640 1 -- 172.21.15.172:0/4180971952 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddec003d30 con 0x7fddf415e890 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.126+0000 7fddfa1f0640 1 -- 172.21.15.172:0/4180971952 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf40c9da0 con 0x7fddf415e890 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.126+0000 7fdddbfff640 1 -- 172.21.15.172:0/4180971952 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec005310 con 0x7fddf415e890 2024-04-14T08:34:02.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.126+0000 7fddfa1f0640 1 -- 172.21.15.172:0/4180971952 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf40c8d50 con 0x7fddf415e890 2024-04-14T08:34:02.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.127+0000 7fddfa1f0640 1 -- 172.21.15.172:0/4180971952 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf4068520 con 0x7fddf415e890 2024-04-14T08:34:02.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.127+0000 7fdddbfff640 1 -- 172.21.15.172:0/4180971952 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddec0054b0 con 0x7fddf415e890 2024-04-14T08:34:02.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.128+0000 7fdddbfff640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdde006c8f0 0x7fdde006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.128+0000 7fdddbfff640 1 -- 172.21.15.172:0/4180971952 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fddec08b9d0 con 0x7fddf415e890 2024-04-14T08:34:02.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.128+0000 7fddf37fe640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdde006c8f0 0x7fdde006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.128+0000 7fddf37fe640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdde006c8f0 0x7fdde006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdde4036000 tx=0x7fdde4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:02.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.132+0000 7fdddbfff640 1 -- 172.21.15.172:0/4180971952 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fddec059620 con 0x7fddf415e890 2024-04-14T08:34:02.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.353+0000 7fddfa1f0640 1 -- 172.21.15.172:0/4180971952 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddf40b36d0 con 0x7fddf415e890 2024-04-14T08:34:02.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.353+0000 7fdddbfff640 1 -- 172.21.15.172:0/4180971952 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7fddec05d920 con 0x7fddf415e890 2024-04-14T08:34:02.355 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:02.355 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1038094401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1038094401},{"type":"v1","addr":"172.21.15.177:6834","nonce":1038094401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1119512744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1119512744},{"type":"v1","addr":"172.21.15.177:6835","nonce":1119512744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3054138283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3054138283},{"type":"v1","addr":"172.21.15.172:6837","nonce":3054138283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:33:59.340499+0000","modified":"2024-04-14T08:34:01.467567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5308":{"gid":5308,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6836/3420108191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3420108191},{"type":"v1","addr":"172.21.15.172:6836","nonce":3420108191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5308,"qdb_cluster":[5308]},"id":2}]} 2024-04-14T08:34:02.355 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 16 2024-04-14T08:34:02.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.355+0000 7fddfa1f0640 1 -- 172.21.15.172:0/4180971952 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdde006c8f0 msgr2=0x7fdde006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.355+0000 7fddfa1f0640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdde006c8f0 0x7fdde006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdde4036000 tx=0x7fdde4008030 comp rx=0 tx=0).stop 2024-04-14T08:34:02.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.355+0000 7fddfa1f0640 1 -- 172.21.15.172:0/4180971952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddf415e890 msgr2=0x7fddf415d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.355+0000 7fddfa1f0640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddf415e890 0x7fddf415d490 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fddec00c6e0 tx=0x7fddec00cba0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.355+0000 7fddfa1f0640 1 -- 172.21.15.172:0/4180971952 shutdown_connections 2024-04-14T08:34:02.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.355+0000 7fddfa1f0640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdde006c8f0 0x7fdde006edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.355+0000 7fddfa1f0640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddf415e890 0x7fddf415d490 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.355+0000 7fddfa1f0640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fddf40d2830 0x7fddf415cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.355+0000 7fddfa1f0640 1 --2- 172.21.15.172:0/4180971952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddf40cedd0 0x7fddf41570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.355+0000 7fddfa1f0640 1 -- 172.21.15.172:0/4180971952 >> 172.21.15.172:0/4180971952 conn(0x7fddf409d590 msgr2=0x7fddf40d4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:02.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.355+0000 7fddfa1f0640 1 -- 172.21.15.172:0/4180971952 shutdown_connections 2024-04-14T08:34:02.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.355+0000 7fddfa1f0640 1 -- 172.21.15.172:0/4180971952 wait complete. 2024-04-14T08:34:02.368 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:34:02.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.469+0000 7f2d803dc640 1 Processor -- start 2024-04-14T08:34:02.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.469+0000 7f2d803dc640 1 -- start start 2024-04-14T08:34:02.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.470+0000 7f2d803dc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2d7815c710 0x7f2d780d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.470+0000 7f2d803dc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2d780cde10 0x7f2d780ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.470+0000 7f2d803dc640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d780aef60 con 0x7f2d780ced80 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.470+0000 7f2d803dc640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2d780cf1f0 con 0x7f2d780cde10 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.470+0000 7f2d803dc640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2d780aed10 con 0x7f2d7815c710 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.470+0000 7f2d7d977640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2d780cde10 0x7f2d780ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.470+0000 7f2d7d977640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2d780cde10 0x7f2d780ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48676/0 (socket says 172.21.15.172:48676) 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.470+0000 7f2d7d977640 1 -- 172.21.15.172:0/1133596482 learned_addr learned my addr 172.21.15.172:0/1133596482 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.471+0000 7f2d7e178640 1 --2- 172.21.15.172:0/1133596482 >> v2:172.21.15.177:3300/0 conn(0x7f2d7815c710 0x7f2d780d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.471+0000 7f2d7d977640 1 -- 172.21.15.172:0/1133596482 >> v2:172.21.15.177:3300/0 conn(0x7f2d7815c710 msgr2=0x7f2d780d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.471+0000 7f2d7d977640 1 --2- 172.21.15.172:0/1133596482 >> v2:172.21.15.177:3300/0 conn(0x7f2d7815c710 0x7f2d780d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.471+0000 7f2d7d977640 1 -- 172.21.15.172:0/1133596482 >> v1:172.21.15.177:6789/0 conn(0x7f2d780ced80 legacy=0x7f2d780ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.471+0000 7f2d7d977640 1 -- 172.21.15.172:0/1133596482 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d780af240 con 0x7f2d780cde10 2024-04-14T08:34:02.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.471+0000 7f2d7d977640 1 --2- 172.21.15.172:0/1133596482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d780cde10 0x7f2d780ce840 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2d78069160 tx=0x7f2d6802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190275f07d108453 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:02.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.472+0000 7f2d7d176640 1 -- 172.21.15.172:0/1133596482 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68035020 con 0x7f2d780cde10 2024-04-14T08:34:02.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.472+0000 7f2d7d176640 1 -- 172.21.15.172:0/1133596482 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d68031870 con 0x7f2d780cde10 2024-04-14T08:34:02.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.472+0000 7f2d7d176640 1 -- 172.21.15.172:0/1133596482 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68031c40 con 0x7f2d780cde10 2024-04-14T08:34:02.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.472+0000 7f2d803dc640 1 -- 172.21.15.172:0/1133596482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d780cde10 msgr2=0x7f2d780ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.472+0000 7f2d803dc640 1 --2- 172.21.15.172:0/1133596482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d780cde10 0x7f2d780ce840 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2d78069160 tx=0x7f2d6802b910 comp rx=0 tx=0).stop 2024-04-14T08:34:02.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.472+0000 7f2d803dc640 1 -- 172.21.15.172:0/1133596482 shutdown_connections 2024-04-14T08:34:02.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.472+0000 7f2d803dc640 1 --2- 172.21.15.172:0/1133596482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d780cde10 0x7f2d780ce840 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.472+0000 7f2d803dc640 1 --2- 172.21.15.172:0/1133596482 >> v2:172.21.15.177:3300/0 conn(0x7f2d7815c710 0x7f2d780d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.472+0000 7f2d803dc640 1 -- 172.21.15.172:0/1133596482 >> 172.21.15.172:0/1133596482 conn(0x7f2d7808c000 msgr2=0x7f2d780d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:02.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.472+0000 7f2d803dc640 1 -- 172.21.15.172:0/1133596482 shutdown_connections 2024-04-14T08:34:02.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.472+0000 7f2d803dc640 1 -- 172.21.15.172:0/1133596482 wait complete. 2024-04-14T08:34:02.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.474+0000 7f2d803dc640 1 Processor -- start 2024-04-14T08:34:02.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.474+0000 7f2d803dc640 1 -- start start 2024-04-14T08:34:02.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.475+0000 7f2d803dc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d780cde10 0x7f2d781f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.475+0000 7f2d803dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d780ced80 0x7f2d780cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.475+0000 7f2d803dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815c710 0x7f2d78156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.475+0000 7f2d803dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7815caf0 con 0x7f2d780ced80 2024-04-14T08:34:02.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.475+0000 7f2d803dc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780af890 con 0x7f2d780cde10 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.475+0000 7f2d803dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2d780d0e90 con 0x7f2d7815c710 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.475+0000 7f2d7d977640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d780ced80 0x7f2d780cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.475+0000 7f2d7d977640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d780ced80 0x7f2d780cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48682/0 (socket says 172.21.15.172:48682) 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.475+0000 7f2d7d977640 1 -- 172.21.15.172:0/2716912266 learned_addr learned my addr 172.21.15.172:0/2716912266 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d7e979640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815c710 0x7f2d78156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d7e178640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d780cde10 0x7f2d781f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d7d977640 1 -- 172.21.15.172:0/2716912266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815c710 msgr2=0x7f2d78156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d7d977640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815c710 0x7f2d78156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d7d977640 1 -- 172.21.15.172:0/2716912266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d780cde10 msgr2=0x7f2d781f1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d7d977640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d780cde10 0x7f2d781f1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d7d977640 1 -- 172.21.15.172:0/2716912266 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d781f2560 con 0x7f2d780ced80 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d7d977640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d780ced80 0x7f2d780cab40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2d68000c00 tx=0x7f2d680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d6effd640 1 -- 172.21.15.172:0/2716912266 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68035040 con 0x7f2d780ced80 2024-04-14T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d6effd640 1 -- 172.21.15.172:0/2716912266 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d68030d00 con 0x7f2d780ced80 2024-04-14T08:34:02.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d803dc640 1 -- 172.21.15.172:0/2716912266 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d781f2840 con 0x7f2d780ced80 2024-04-14T08:34:02.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d6effd640 1 -- 172.21.15.172:0/2716912266 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d680456a0 con 0x7f2d780ced80 2024-04-14T08:34:02.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.476+0000 7f2d803dc640 1 -- 172.21.15.172:0/2716912266 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d78156b60 con 0x7f2d780ced80 2024-04-14T08:34:02.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.477+0000 7f2d803dc640 1 -- 172.21.15.172:0/2716912266 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d781f2d60 con 0x7f2d780ced80 2024-04-14T08:34:02.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.478+0000 7f2d6effd640 1 -- 172.21.15.172:0/2716912266 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d68038020 con 0x7f2d780ced80 2024-04-14T08:34:02.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.478+0000 7f2d6effd640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d5c06c8f0 0x7f2d5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.478+0000 7f2d6effd640 1 -- 172.21.15.172:0/2716912266 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f2d680adf90 con 0x7f2d780ced80 2024-04-14T08:34:02.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.478+0000 7f2d7e178640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d5c06c8f0 0x7f2d5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.479+0000 7f2d7e178640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d5c06c8f0 0x7f2d5c06edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d74006fd0 tx=0x7f2d74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:02.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.482+0000 7f2d6effd640 1 -- 172.21.15.172:0/2716912266 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2d781f2d60 con 0x7f2d780ced80 2024-04-14T08:34:02.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.663+0000 7f2d803dc640 1 -- 172.21.15.172:0/2716912266 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d781f53c0 con 0x7f2d780ced80 2024-04-14T08:34:02.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.664+0000 7f2d6effd640 1 -- 172.21.15.172:0/2716912266 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17002 (secure 0 0 0) 0x7f2d6807c2c0 con 0x7f2d780ced80 2024-04-14T08:34:02.666 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:02.666 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:00.984153+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:33:57.868995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-14T08:33:58.323549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:34:02.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.667+0000 7f2d803dc640 1 -- 172.21.15.172:0/2716912266 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d5c06c8f0 msgr2=0x7f2d5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.667+0000 7f2d803dc640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d5c06c8f0 0x7f2d5c06edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d74006fd0 tx=0x7f2d74008040 comp rx=0 tx=0).stop 2024-04-14T08:34:02.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.667+0000 7f2d803dc640 1 -- 172.21.15.172:0/2716912266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d780ced80 msgr2=0x7f2d780cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.667+0000 7f2d803dc640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d780ced80 0x7f2d780cab40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2d68000c00 tx=0x7f2d680046c0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.667+0000 7f2d803dc640 1 -- 172.21.15.172:0/2716912266 shutdown_connections 2024-04-14T08:34:02.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.667+0000 7f2d803dc640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d5c06c8f0 0x7f2d5c06edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.667+0000 7f2d803dc640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815c710 0x7f2d78156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.667+0000 7f2d803dc640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d780ced80 0x7f2d780cab40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.667+0000 7f2d803dc640 1 --2- 172.21.15.172:0/2716912266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d780cde10 0x7f2d781f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.667+0000 7f2d803dc640 1 -- 172.21.15.172:0/2716912266 >> 172.21.15.172:0/2716912266 conn(0x7f2d7808c000 msgr2=0x7f2d780ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:02.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.667+0000 7f2d803dc640 1 -- 172.21.15.172:0/2716912266 shutdown_connections 2024-04-14T08:34:02.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.667+0000 7f2d803dc640 1 -- 172.21.15.172:0/2716912266 wait complete. 2024-04-14T08:34:02.680 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-14T08:34:02.680 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:34:02.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.801+0000 7fc1d0c60640 1 Processor -- start 2024-04-14T08:34:02.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.801+0000 7fc1d0c60640 1 -- start start 2024-04-14T08:34:02.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.802+0000 7fc1d0c60640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1cc0d2830 0x7fc1cc0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.802+0000 7fc1d0c60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1cc0cedd0 0x7fc1cc0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.802+0000 7fc1d0c60640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1cc0d3150 con 0x7fc1cc15e890 2024-04-14T08:34:02.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.802+0000 7fc1d0c60640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc1cc0cff30 con 0x7fc1cc0d2830 2024-04-14T08:34:02.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.802+0000 7fc1d0c60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc1cc0d2350 con 0x7fc1cc0cedd0 2024-04-14T08:34:02.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.802+0000 7fc1ca59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1cc0d2830 0x7fc1cc0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.802+0000 7fc1ca59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1cc0d2830 0x7fc1cc0d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48686/0 (socket says 172.21.15.172:48686) 2024-04-14T08:34:02.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.803+0000 7fc1ca59c640 1 -- 172.21.15.172:0/1070612213 learned_addr learned my addr 172.21.15.172:0/1070612213 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:02.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.803+0000 7fc1cad9d640 1 --2- 172.21.15.172:0/1070612213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1cc0cedd0 0x7fc1cc0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.803+0000 7fc1ca59c640 1 -- 172.21.15.172:0/1070612213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1cc0cedd0 msgr2=0x7fc1cc0b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:02.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.803+0000 7fc1ca59c640 1 --2- 172.21.15.172:0/1070612213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1cc0cedd0 0x7fc1cc0b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.803+0000 7fc1ca59c640 1 -- 172.21.15.172:0/1070612213 >> v1:172.21.15.177:6789/0 conn(0x7fc1cc15e890 legacy=0x7fc1cc0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.803+0000 7fc1ca59c640 1 -- 172.21.15.172:0/1070612213 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc0cf320 con 0x7fc1cc0d2830 2024-04-14T08:34:02.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.804+0000 7fc1ca59c640 1 --2- 172.21.15.172:0/1070612213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1cc0d2830 0x7fc1cc0d2c10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009870 tx=0x7fc1c402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626272add1259bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:02.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.804+0000 7fc1c959a640 1 -- 172.21.15.172:0/1070612213 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c402f840 con 0x7fc1cc0d2830 2024-04-14T08:34:02.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.804+0000 7fc1c959a640 1 -- 172.21.15.172:0/1070612213 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c402f9e0 con 0x7fc1cc0d2830 2024-04-14T08:34:02.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.804+0000 7fc1c959a640 1 -- 172.21.15.172:0/1070612213 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c402fdb0 con 0x7fc1cc0d2830 2024-04-14T08:34:02.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.805+0000 7fc1d0c60640 1 -- 172.21.15.172:0/1070612213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1cc0d2830 msgr2=0x7fc1cc0d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.805+0000 7fc1d0c60640 1 --2- 172.21.15.172:0/1070612213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1cc0d2830 0x7fc1cc0d2c10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009870 tx=0x7fc1c402ba20 comp rx=0 tx=0).stop 2024-04-14T08:34:02.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.805+0000 7fc1d0c60640 1 -- 172.21.15.172:0/1070612213 shutdown_connections 2024-04-14T08:34:02.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.805+0000 7fc1d0c60640 1 --2- 172.21.15.172:0/1070612213 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1cc0cedd0 0x7fc1cc0b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.805+0000 7fc1d0c60640 1 --2- 172.21.15.172:0/1070612213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1cc0d2830 0x7fc1cc0d2c10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.805+0000 7fc1d0c60640 1 -- 172.21.15.172:0/1070612213 >> 172.21.15.172:0/1070612213 conn(0x7fc1cc09d590 msgr2=0x7fc1cc0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:02.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.805+0000 7fc1d0c60640 1 -- 172.21.15.172:0/1070612213 shutdown_connections 2024-04-14T08:34:02.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.805+0000 7fc1d0c60640 1 -- 172.21.15.172:0/1070612213 wait complete. 2024-04-14T08:34:02.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.806+0000 7fc1d0c60640 1 Processor -- start 2024-04-14T08:34:02.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.806+0000 7fc1d0c60640 1 -- start start 2024-04-14T08:34:02.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1d0c60640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1cc0cedd0 0x7fc1cc1f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1d0c60640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc0d2830 0x7fc1cc0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1d0c60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1cc15e890 0x7fc1cc06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1d0c60640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc0cfbc0 con 0x7fc1cc0cedd0 2024-04-14T08:34:02.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1d0c60640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc0cf8e0 con 0x7fc1cc0d2830 2024-04-14T08:34:02.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1d0c60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc1cc0b3c50 con 0x7fc1cc15e890 2024-04-14T08:34:02.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1ca59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1cc0cedd0 0x7fc1cc1f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1ca59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1cc0cedd0 0x7fc1cc1f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48696/0 (socket says 172.21.15.172:48696) 2024-04-14T08:34:02.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1ca59c640 1 -- 172.21.15.172:0/3422550608 learned_addr learned my addr 172.21.15.172:0/3422550608 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:02.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1ca59c640 1 -- 172.21.15.172:0/3422550608 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1cc15e890 msgr2=0x7fc1cc06d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1cad9d640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1cc15e890 0x7fc1cc06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1ca59c640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1cc15e890 0x7fc1cc06d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1c9d9b640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc0d2830 0x7fc1cc0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1ca59c640 1 -- 172.21.15.172:0/3422550608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc0d2830 msgr2=0x7fc1cc0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:02.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1ca59c640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc0d2830 0x7fc1cc0cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:02.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1ca59c640 1 -- 172.21.15.172:0/3422550608 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc0ce130 con 0x7fc1cc0cedd0 2024-04-14T08:34:02.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1cad9d640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1cc15e890 0x7fc1cc06d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:02.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1c9d9b640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc0d2830 0x7fc1cc0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:02.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1ca59c640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1cc0cedd0 0x7fc1cc1f72c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc1c402ba60 tx=0x7fc1c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:02.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.807+0000 7fc1baffd640 1 -- 172.21.15.172:0/3422550608 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4047070 con 0x7fc1cc0cedd0 2024-04-14T08:34:02.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.808+0000 7fc1baffd640 1 -- 172.21.15.172:0/3422550608 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c4037cb0 con 0x7fc1cc0cedd0 2024-04-14T08:34:02.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.808+0000 7fc1baffd640 1 -- 172.21.15.172:0/3422550608 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4037e60 con 0x7fc1cc0cedd0 2024-04-14T08:34:02.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.808+0000 7fc1d0c60640 1 -- 172.21.15.172:0/3422550608 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1cc1f7800 con 0x7fc1cc0cedd0 2024-04-14T08:34:02.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.808+0000 7fc1d0c60640 1 -- 172.21.15.172:0/3422550608 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1cc0cde50 con 0x7fc1cc0cedd0 2024-04-14T08:34:02.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.809+0000 7fc1d0c60640 1 -- 172.21.15.172:0/3422550608 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1cc068520 con 0x7fc1cc0cedd0 2024-04-14T08:34:02.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.809+0000 7fc1baffd640 1 -- 172.21.15.172:0/3422550608 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1c404a070 con 0x7fc1cc0cedd0 2024-04-14T08:34:02.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.810+0000 7fc1baffd640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1ac06c920 0x7fc1ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:02.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.810+0000 7fc1baffd640 1 -- 172.21.15.172:0/3422550608 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fc1c40adfc0 con 0x7fc1cc0cedd0 2024-04-14T08:34:02.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.810+0000 7fc1c9d9b640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1ac06c920 0x7fc1ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:02.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.810+0000 7fc1c9d9b640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1ac06c920 0x7fc1ac06ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0ca4d0 tx=0x7fc1c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:02.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:02.814+0000 7fc1baffd640 1 -- 172.21.15.172:0/3422550608 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc1c4033030 con 0x7fc1cc0cedd0 2024-04-14T08:34:03.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.024+0000 7fc1d0c60640 1 -- 172.21.15.172:0/3422550608 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1cc0d2610 con 0x7fc1ac06c920 2024-04-14T08:34:03.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.030+0000 7fc1baffd640 1 -- 172.21.15.172:0/3422550608 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249821 (secure 0 0 0) 0x7fc1cc0d2610 con 0x7fc1ac06c920 2024-04-14T08:34:03.034 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:03.036 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:34:03.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.033+0000 7fc1d0c60640 1 -- 172.21.15.172:0/3422550608 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1ac06c920 msgr2=0x7fc1ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.033+0000 7fc1d0c60640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1ac06c920 0x7fc1ac06ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0ca4d0 tx=0x7fc1c0009040 comp rx=0 tx=0).stop 2024-04-14T08:34:03.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.033+0000 7fc1d0c60640 1 -- 172.21.15.172:0/3422550608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1cc0cedd0 msgr2=0x7fc1cc1f72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.033+0000 7fc1d0c60640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1cc0cedd0 0x7fc1cc1f72c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc1c402ba60 tx=0x7fc1c4004730 comp rx=0 tx=0).stop 2024-04-14T08:34:03.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.033+0000 7fc1d0c60640 1 -- 172.21.15.172:0/3422550608 shutdown_connections 2024-04-14T08:34:03.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.033+0000 7fc1d0c60640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1ac06c920 0x7fc1ac06ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.033+0000 7fc1d0c60640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1cc15e890 0x7fc1cc06d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.033+0000 7fc1d0c60640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc0d2830 0x7fc1cc0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.033+0000 7fc1d0c60640 1 --2- 172.21.15.172:0/3422550608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1cc0cedd0 0x7fc1cc1f72c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.033+0000 7fc1d0c60640 1 -- 172.21.15.172:0/3422550608 >> 172.21.15.172:0/3422550608 conn(0x7fc1cc09d590 msgr2=0x7fc1cc152e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:03.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.033+0000 7fc1d0c60640 1 -- 172.21.15.172:0/3422550608 shutdown_connections 2024-04-14T08:34:03.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.033+0000 7fc1d0c60640 1 -- 172.21.15.172:0/3422550608 wait complete. 2024-04-14T08:34:03.045 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-04-14T08:34:01.134679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":191,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220676,"kb_used_data":4480,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754044,"statfs":{"total":773094113280,"available":772868141056,"internally_reserved":0,"allocated":4587520,"data_stored":2621845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000543"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.009281+0000","last_change":"2024-04-14T08:34:00.005789+0000","last_active":"2024-04-14T08:34:00.009281+0000","last_peered":"2024-04-14T08:34:00.009281+0000","last_clean":"2024-04-14T08:34:00.009281+0000","last_became_active":"2024-04-14T08:34:00.005518+0000","last_became_peered":"2024-04-14T08:34:00.005518+0000","last_unstale":"2024-04-14T08:34:00.009281+0000","last_undegraded":"2024-04-14T08:34:00.009281+0000","last_fullsized":"2024-04-14T08:34:00.009281+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:54.465535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.002033+0000","last_change":"2024-04-14T08:33:59.997834+0000","last_active":"2024-04-14T08:34:00.002033+0000","last_peered":"2024-04-14T08:34:00.002033+0000","last_clean":"2024-04-14T08:34:00.002033+0000","last_became_active":"2024-04-14T08:33:59.997418+0000","last_became_peered":"2024-04-14T08:33:59.997418+0000","last_unstale":"2024-04-14T08:34:00.002033+0000","last_undegraded":"2024-04-14T08:34:00.002033+0000","last_fullsized":"2024-04-14T08:34:00.002033+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:30:21.238910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007719+0000","last_change":"2024-04-14T08:34:00.004809+0000","last_active":"2024-04-14T08:34:00.007719+0000","last_peered":"2024-04-14T08:34:00.007719+0000","last_clean":"2024-04-14T08:34:00.007719+0000","last_became_active":"2024-04-14T08:34:00.004529+0000","last_became_peered":"2024-04-14T08:34:00.004529+0000","last_unstale":"2024-04-14T08:34:00.007719+0000","last_undegraded":"2024-04-14T08:34:00.007719+0000","last_fullsized":"2024-04-14T08:34:00.007719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:03.589235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.012809+0000","last_change":"2024-04-14T08:34:00.011255+0000","last_active":"2024-04-14T08:34:00.012809+0000","last_peered":"2024-04-14T08:34:00.012809+0000","last_clean":"2024-04-14T08:34:00.012809+0000","last_became_active":"2024-04-14T08:34:00.011001+0000","last_became_peered":"2024-04-14T08:34:00.011001+0000","last_unstale":"2024-04-14T08:34:00.012809+0000","last_undegraded":"2024-04-14T08:34:00.012809+0000","last_fullsized":"2024-04-14T08:34:00.012809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:02:58.673573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013037+0000","last_change":"2024-04-14T08:34:00.011994+0000","last_active":"2024-04-14T08:34:00.013037+0000","last_peered":"2024-04-14T08:34:00.013037+0000","last_clean":"2024-04-14T08:34:00.013037+0000","last_became_active":"2024-04-14T08:34:00.011667+0000","last_became_peered":"2024-04-14T08:34:00.011667+0000","last_unstale":"2024-04-14T08:34:00.013037+0000","last_undegraded":"2024-04-14T08:34:00.013037+0000","last_fullsized":"2024-04-14T08:34:00.013037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:00:51.825299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005146+0000","last_change":"2024-04-14T08:34:00.003096+0000","last_active":"2024-04-14T08:34:00.005146+0000","last_peered":"2024-04-14T08:34:00.005146+0000","last_clean":"2024-04-14T08:34:00.005146+0000","last_became_active":"2024-04-14T08:34:00.002694+0000","last_became_peered":"2024-04-14T08:34:00.002694+0000","last_unstale":"2024-04-14T08:34:00.005146+0000","last_undegraded":"2024-04-14T08:34:00.005146+0000","last_fullsized":"2024-04-14T08:34:00.005146+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:26:02.000109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007814+0000","last_change":"2024-04-14T08:34:00.002704+0000","last_active":"2024-04-14T08:34:00.007814+0000","last_peered":"2024-04-14T08:34:00.007814+0000","last_clean":"2024-04-14T08:34:00.007814+0000","last_became_active":"2024-04-14T08:34:00.002113+0000","last_became_peered":"2024-04-14T08:34:00.002113+0000","last_unstale":"2024-04-14T08:34:00.007814+0000","last_undegraded":"2024-04-14T08:34:00.007814+0000","last_fullsized":"2024-04-14T08:34:00.007814+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:40:27.541789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.011040+0000","last_change":"2024-04-14T08:34:00.009380+0000","last_active":"2024-04-14T08:34:00.011040+0000","last_peered":"2024-04-14T08:34:00.011040+0000","last_clean":"2024-04-14T08:34:00.011040+0000","last_became_active":"2024-04-14T08:34:00.009124+0000","last_became_peered":"2024-04-14T08:34:00.009124+0000","last_unstale":"2024-04-14T08:34:00.011040+0000","last_undegraded":"2024-04-14T08:34:00.011040+0000","last_fullsized":"2024-04-14T08:34:00.011040+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:51:42.354241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.010991+0000","last_change":"2024-04-14T08:34:00.006726+0000","last_active":"2024-04-14T08:34:00.010991+0000","last_peered":"2024-04-14T08:34:00.010991+0000","last_clean":"2024-04-14T08:34:00.010991+0000","last_became_active":"2024-04-14T08:34:00.006240+0000","last_became_peered":"2024-04-14T08:34:00.006240+0000","last_unstale":"2024-04-14T08:34:00.010991+0000","last_undegraded":"2024-04-14T08:34:00.010991+0000","last_fullsized":"2024-04-14T08:34:00.010991+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:30:21.238910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007788+0000","last_change":"2024-04-14T08:34:00.003564+0000","last_active":"2024-04-14T08:34:00.007788+0000","last_peered":"2024-04-14T08:34:00.007788+0000","last_clean":"2024-04-14T08:34:00.007788+0000","last_became_active":"2024-04-14T08:34:00.003179+0000","last_became_peered":"2024-04-14T08:34:00.003179+0000","last_unstale":"2024-04-14T08:34:00.007788+0000","last_undegraded":"2024-04-14T08:34:00.007788+0000","last_fullsized":"2024-04-14T08:34:00.007788+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:09:02.249499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.010639+0000","last_change":"2024-04-14T08:34:00.006702+0000","last_active":"2024-04-14T08:34:00.010639+0000","last_peered":"2024-04-14T08:34:00.010639+0000","last_clean":"2024-04-14T08:34:00.010639+0000","last_became_active":"2024-04-14T08:34:00.005642+0000","last_became_peered":"2024-04-14T08:34:00.005642+0000","last_unstale":"2024-04-14T08:34:00.010639+0000","last_undegraded":"2024-04-14T08:34:00.010639+0000","last_fullsized":"2024-04-14T08:34:00.010639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:58:48.218603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.004361+0000","last_change":"2024-04-14T08:33:59.993650+0000","last_active":"2024-04-14T08:34:00.004361+0000","last_peered":"2024-04-14T08:34:00.004361+0000","last_clean":"2024-04-14T08:34:00.004361+0000","last_became_active":"2024-04-14T08:33:59.993388+0000","last_became_peered":"2024-04-14T08:33:59.993388+0000","last_unstale":"2024-04-14T08:34:00.004361+0000","last_undegraded":"2024-04-14T08:34:00.004361+0000","last_fullsized":"2024-04-14T08:34:00.004361+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:16:15.779370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.010589+0000","last_change":"2024-04-14T08:34:00.005066+0000","last_active":"2024-04-14T08:34:00.010589+0000","last_peered":"2024-04-14T08:34:00.010589+0000","last_clean":"2024-04-14T08:34:00.010589+0000","last_became_active":"2024-04-14T08:34:00.004793+0000","last_became_peered":"2024-04-14T08:34:00.004793+0000","last_unstale":"2024-04-14T08:34:00.010589+0000","last_undegraded":"2024-04-14T08:34:00.010589+0000","last_fullsized":"2024-04-14T08:34:00.010589+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:51:42.354241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013283+0000","last_change":"2024-04-14T08:34:00.008628+0000","last_active":"2024-04-14T08:34:00.013283+0000","last_peered":"2024-04-14T08:34:00.013283+0000","last_clean":"2024-04-14T08:34:00.013283+0000","last_became_active":"2024-04-14T08:34:00.008064+0000","last_became_peered":"2024-04-14T08:34:00.008064+0000","last_unstale":"2024-04-14T08:34:00.013283+0000","last_undegraded":"2024-04-14T08:34:00.013283+0000","last_fullsized":"2024-04-14T08:34:00.013283+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:13.330734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007830+0000","last_change":"2024-04-14T08:34:00.002929+0000","last_active":"2024-04-14T08:34:00.007830+0000","last_peered":"2024-04-14T08:34:00.007830+0000","last_clean":"2024-04-14T08:34:00.007830+0000","last_became_active":"2024-04-14T08:34:00.001532+0000","last_became_peered":"2024-04-14T08:34:00.001532+0000","last_unstale":"2024-04-14T08:34:00.007830+0000","last_undegraded":"2024-04-14T08:34:00.007830+0000","last_fullsized":"2024-04-14T08:34:00.007830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:21.676556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:58.978291+0000","last_change":"2024-04-14T08:33:58.978291+0000","last_active":"2024-04-14T08:33:58.978291+0000","last_peered":"2024-04-14T08:33:58.978291+0000","last_clean":"2024-04-14T08:33:58.978291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:58.978291+0000","last_undegraded":"2024-04-14T08:33:58.978291+0000","last_fullsized":"2024-04-14T08:33:58.978291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:58.978291+0000","last_change":"2024-04-14T08:33:58.978291+0000","last_active":"2024-04-14T08:33:58.978291+0000","last_peered":"2024-04-14T08:33:58.978291+0000","last_clean":"2024-04-14T08:33:58.978291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:58.978291+0000","last_undegraded":"2024-04-14T08:33:58.978291+0000","last_fullsized":"2024-04-14T08:33:58.978291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.014299+0000","last_change":"2024-04-14T08:34:00.008961+0000","last_active":"2024-04-14T08:34:00.014299+0000","last_peered":"2024-04-14T08:34:00.014299+0000","last_clean":"2024-04-14T08:34:00.014299+0000","last_became_active":"2024-04-14T08:34:00.008625+0000","last_became_peered":"2024-04-14T08:34:00.008625+0000","last_unstale":"2024-04-14T08:34:00.014299+0000","last_undegraded":"2024-04-14T08:34:00.014299+0000","last_fullsized":"2024-04-14T08:34:00.014299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:26:02.000109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.006817+0000","last_change":"2024-04-14T08:33:59.999239+0000","last_active":"2024-04-14T08:34:00.006817+0000","last_peered":"2024-04-14T08:34:00.006817+0000","last_clean":"2024-04-14T08:34:00.006817+0000","last_became_active":"2024-04-14T08:33:59.998980+0000","last_became_peered":"2024-04-14T08:33:59.998980+0000","last_unstale":"2024-04-14T08:34:00.006817+0000","last_undegraded":"2024-04-14T08:34:00.006817+0000","last_fullsized":"2024-04-14T08:34:00.006817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:26:59.590582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013724+0000","last_change":"2024-04-14T08:34:00.008090+0000","last_active":"2024-04-14T08:34:00.013724+0000","last_peered":"2024-04-14T08:34:00.013724+0000","last_clean":"2024-04-14T08:34:00.013724+0000","last_became_active":"2024-04-14T08:34:00.007784+0000","last_became_peered":"2024-04-14T08:34:00.007784+0000","last_unstale":"2024-04-14T08:34:00.013724+0000","last_undegraded":"2024-04-14T08:34:00.013724+0000","last_fullsized":"2024-04-14T08:34:00.013724+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:20.676280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005352+0000","last_change":"2024-04-14T08:34:00.001292+0000","last_active":"2024-04-14T08:34:00.005352+0000","last_peered":"2024-04-14T08:34:00.005352+0000","last_clean":"2024-04-14T08:34:00.005352+0000","last_became_active":"2024-04-14T08:34:00.001000+0000","last_became_peered":"2024-04-14T08:34:00.001000+0000","last_unstale":"2024-04-14T08:34:00.005352+0000","last_undegraded":"2024-04-14T08:34:00.005352+0000","last_fullsized":"2024-04-14T08:34:00.005352+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:03.589235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013259+0000","last_change":"2024-04-14T08:34:00.007813+0000","last_active":"2024-04-14T08:34:00.013259+0000","last_peered":"2024-04-14T08:34:00.013259+0000","last_clean":"2024-04-14T08:34:00.013259+0000","last_became_active":"2024-04-14T08:34:00.005125+0000","last_became_peered":"2024-04-14T08:34:00.005125+0000","last_unstale":"2024-04-14T08:34:00.013259+0000","last_undegraded":"2024-04-14T08:34:00.013259+0000","last_fullsized":"2024-04-14T08:34:00.013259+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:26:02.000109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:58.978291+0000","last_change":"2024-04-14T08:33:58.978291+0000","last_active":"2024-04-14T08:33:58.978291+0000","last_peered":"2024-04-14T08:33:58.978291+0000","last_clean":"2024-04-14T08:33:58.978291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:58.978291+0000","last_undegraded":"2024-04-14T08:33:58.978291+0000","last_fullsized":"2024-04-14T08:33:58.978291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.002234+0000","last_change":"2024-04-14T08:33:59.995880+0000","last_active":"2024-04-14T08:34:00.002234+0000","last_peered":"2024-04-14T08:34:00.002234+0000","last_clean":"2024-04-14T08:34:00.002234+0000","last_became_active":"2024-04-14T08:33:59.995604+0000","last_became_peered":"2024-04-14T08:33:59.995604+0000","last_unstale":"2024-04-14T08:34:00.002234+0000","last_undegraded":"2024-04-14T08:34:00.002234+0000","last_fullsized":"2024-04-14T08:34:00.002234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:19:10.169283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.014261+0000","last_change":"2024-04-14T08:34:00.007911+0000","last_active":"2024-04-14T08:34:00.014261+0000","last_peered":"2024-04-14T08:34:00.014261+0000","last_clean":"2024-04-14T08:34:00.014261+0000","last_became_active":"2024-04-14T08:34:00.007635+0000","last_became_peered":"2024-04-14T08:34:00.007635+0000","last_unstale":"2024-04-14T08:34:00.014261+0000","last_undegraded":"2024-04-14T08:34:00.014261+0000","last_fullsized":"2024-04-14T08:34:00.014261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:16:15.779370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.008739+0000","last_change":"2024-04-14T08:34:00.006507+0000","last_active":"2024-04-14T08:34:00.008739+0000","last_peered":"2024-04-14T08:34:00.008739+0000","last_clean":"2024-04-14T08:34:00.008739+0000","last_became_active":"2024-04-14T08:34:00.006246+0000","last_became_peered":"2024-04-14T08:34:00.006246+0000","last_unstale":"2024-04-14T08:34:00.008739+0000","last_undegraded":"2024-04-14T08:34:00.008739+0000","last_fullsized":"2024-04-14T08:34:00.008739+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:03.589235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007423+0000","last_change":"2024-04-14T08:34:00.003623+0000","last_active":"2024-04-14T08:34:00.007423+0000","last_peered":"2024-04-14T08:34:00.007423+0000","last_clean":"2024-04-14T08:34:00.007423+0000","last_became_active":"2024-04-14T08:34:00.003330+0000","last_became_peered":"2024-04-14T08:34:00.003330+0000","last_unstale":"2024-04-14T08:34:00.007423+0000","last_undegraded":"2024-04-14T08:34:00.007423+0000","last_fullsized":"2024-04-14T08:34:00.007423+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:00:20.760892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007757+0000","last_change":"2024-04-14T08:34:00.002792+0000","last_active":"2024-04-14T08:34:00.007757+0000","last_peered":"2024-04-14T08:34:00.007757+0000","last_clean":"2024-04-14T08:34:00.007757+0000","last_became_active":"2024-04-14T08:34:00.001687+0000","last_became_peered":"2024-04-14T08:34:00.001687+0000","last_unstale":"2024-04-14T08:34:00.007757+0000","last_undegraded":"2024-04-14T08:34:00.007757+0000","last_fullsized":"2024-04-14T08:34:00.007757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:52.044408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007027+0000","last_change":"2024-04-14T08:34:00.002146+0000","last_active":"2024-04-14T08:34:00.007027+0000","last_peered":"2024-04-14T08:34:00.007027+0000","last_clean":"2024-04-14T08:34:00.007027+0000","last_became_active":"2024-04-14T08:34:00.001829+0000","last_became_peered":"2024-04-14T08:34:00.001829+0000","last_unstale":"2024-04-14T08:34:00.007027+0000","last_undegraded":"2024-04-14T08:34:00.007027+0000","last_fullsized":"2024-04-14T08:34:00.007027+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:01:07.798205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.011471+0000","last_change":"2024-04-14T08:34:00.005728+0000","last_active":"2024-04-14T08:34:00.011471+0000","last_peered":"2024-04-14T08:34:00.011471+0000","last_clean":"2024-04-14T08:34:00.011471+0000","last_became_active":"2024-04-14T08:34:00.005463+0000","last_became_peered":"2024-04-14T08:34:00.005463+0000","last_unstale":"2024-04-14T08:34:00.011471+0000","last_undegraded":"2024-04-14T08:34:00.011471+0000","last_fullsized":"2024-04-14T08:34:00.011471+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:42.487210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:58.978291+0000","last_change":"2024-04-14T08:33:58.978291+0000","last_active":"2024-04-14T08:33:58.978291+0000","last_peered":"2024-04-14T08:33:58.978291+0000","last_clean":"2024-04-14T08:33:58.978291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:58.978291+0000","last_undegraded":"2024-04-14T08:33:58.978291+0000","last_fullsized":"2024-04-14T08:33:58.978291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.002999+0000","last_change":"2024-04-14T08:34:00.000949+0000","last_active":"2024-04-14T08:34:00.002999+0000","last_peered":"2024-04-14T08:34:00.002999+0000","last_clean":"2024-04-14T08:34:00.002999+0000","last_became_active":"2024-04-14T08:34:00.000684+0000","last_became_peered":"2024-04-14T08:34:00.000684+0000","last_unstale":"2024-04-14T08:34:00.002999+0000","last_undegraded":"2024-04-14T08:34:00.002999+0000","last_fullsized":"2024-04-14T08:34:00.002999+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:20.676280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:58.978291+0000","last_change":"2024-04-14T08:33:58.978291+0000","last_active":"2024-04-14T08:33:58.978291+0000","last_peered":"2024-04-14T08:33:58.978291+0000","last_clean":"2024-04-14T08:33:58.978291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:58.978291+0000","last_undegraded":"2024-04-14T08:33:58.978291+0000","last_fullsized":"2024-04-14T08:33:58.978291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.017510+0000","last_change":"2024-04-14T08:33:59.008983+0000","last_active":"2024-04-14T08:34:00.017510+0000","last_peered":"2024-04-14T08:34:00.017510+0000","last_clean":"2024-04-14T08:34:00.017510+0000","last_became_active":"2024-04-14T08:33:59.008686+0000","last_became_peered":"2024-04-14T08:33:59.008686+0000","last_unstale":"2024-04-14T08:34:00.017510+0000","last_undegraded":"2024-04-14T08:34:00.017510+0000","last_fullsized":"2024-04-14T08:34:00.017510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:51:58.159319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.008741+0000","last_change":"2024-04-14T08:34:00.005932+0000","last_active":"2024-04-14T08:34:00.008741+0000","last_peered":"2024-04-14T08:34:00.008741+0000","last_clean":"2024-04-14T08:34:00.008741+0000","last_became_active":"2024-04-14T08:34:00.005645+0000","last_became_peered":"2024-04-14T08:34:00.005645+0000","last_unstale":"2024-04-14T08:34:00.008741+0000","last_undegraded":"2024-04-14T08:34:00.008741+0000","last_fullsized":"2024-04-14T08:34:00.008741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:46:26.546283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.015765+0000","last_change":"2024-04-14T08:33:59.005808+0000","last_active":"2024-04-14T08:34:00.015765+0000","last_peered":"2024-04-14T08:34:00.015765+0000","last_clean":"2024-04-14T08:34:00.015765+0000","last_became_active":"2024-04-14T08:33:59.005498+0000","last_became_peered":"2024-04-14T08:33:59.005498+0000","last_unstale":"2024-04-14T08:34:00.015765+0000","last_undegraded":"2024-04-14T08:34:00.015765+0000","last_fullsized":"2024-04-14T08:34:00.015765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:00:19.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.014832+0000","last_change":"2024-04-14T08:34:00.006051+0000","last_active":"2024-04-14T08:34:00.014832+0000","last_peered":"2024-04-14T08:34:00.014832+0000","last_clean":"2024-04-14T08:34:00.014832+0000","last_became_active":"2024-04-14T08:34:00.005729+0000","last_became_peered":"2024-04-14T08:34:00.005729+0000","last_unstale":"2024-04-14T08:34:00.014832+0000","last_undegraded":"2024-04-14T08:34:00.014832+0000","last_fullsized":"2024-04-14T08:34:00.014832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:20.676280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.016764+0000","last_change":"2024-04-14T08:33:59.009159+0000","last_active":"2024-04-14T08:34:00.016764+0000","last_peered":"2024-04-14T08:34:00.016764+0000","last_clean":"2024-04-14T08:34:00.016764+0000","last_became_active":"2024-04-14T08:33:59.008864+0000","last_became_peered":"2024-04-14T08:33:59.008864+0000","last_unstale":"2024-04-14T08:34:00.016764+0000","last_undegraded":"2024-04-14T08:34:00.016764+0000","last_fullsized":"2024-04-14T08:34:00.016764+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:16:14.774984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007725+0000","last_change":"2024-04-14T08:34:00.004505+0000","last_active":"2024-04-14T08:34:00.007725+0000","last_peered":"2024-04-14T08:34:00.007725+0000","last_clean":"2024-04-14T08:34:00.007725+0000","last_became_active":"2024-04-14T08:34:00.004249+0000","last_became_peered":"2024-04-14T08:34:00.004249+0000","last_unstale":"2024-04-14T08:34:00.007725+0000","last_undegraded":"2024-04-14T08:34:00.007725+0000","last_fullsized":"2024-04-14T08:34:00.007725+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:31.409872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.017223+0000","last_change":"2024-04-14T08:33:59.009688+0000","last_active":"2024-04-14T08:34:00.017223+0000","last_peered":"2024-04-14T08:34:00.017223+0000","last_clean":"2024-04-14T08:34:00.017223+0000","last_became_active":"2024-04-14T08:33:59.009321+0000","last_became_peered":"2024-04-14T08:33:59.009321+0000","last_unstale":"2024-04-14T08:34:00.017223+0000","last_undegraded":"2024-04-14T08:34:00.017223+0000","last_fullsized":"2024-04-14T08:34:00.017223+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:01:06.793819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005797+0000","last_change":"2024-04-14T08:33:59.996429+0000","last_active":"2024-04-14T08:34:00.005797+0000","last_peered":"2024-04-14T08:34:00.005797+0000","last_clean":"2024-04-14T08:34:00.005797+0000","last_became_active":"2024-04-14T08:33:59.996144+0000","last_became_peered":"2024-04-14T08:33:59.996144+0000","last_unstale":"2024-04-14T08:34:00.005797+0000","last_undegraded":"2024-04-14T08:34:00.005797+0000","last_fullsized":"2024-04-14T08:34:00.005797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:17.444588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.019207+0000","last_change":"2024-04-14T08:33:59.005231+0000","last_active":"2024-04-14T08:34:00.019207+0000","last_peered":"2024-04-14T08:34:00.019207+0000","last_clean":"2024-04-14T08:34:00.019207+0000","last_became_active":"2024-04-14T08:33:59.004517+0000","last_became_peered":"2024-04-14T08:33:59.004517+0000","last_unstale":"2024-04-14T08:34:00.019207+0000","last_undegraded":"2024-04-14T08:34:00.019207+0000","last_fullsized":"2024-04-14T08:34:00.019207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:26:00.995723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007792+0000","last_change":"2024-04-14T08:34:00.002362+0000","last_active":"2024-04-14T08:34:00.007792+0000","last_peered":"2024-04-14T08:34:00.007792+0000","last_clean":"2024-04-14T08:34:00.007792+0000","last_became_active":"2024-04-14T08:34:00.001232+0000","last_became_peered":"2024-04-14T08:34:00.001232+0000","last_unstale":"2024-04-14T08:34:00.007792+0000","last_undegraded":"2024-04-14T08:34:00.007792+0000","last_fullsized":"2024-04-14T08:34:00.007792+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:30:21.238910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:57.973905+0000","last_change":"2024-04-14T08:33:57.973905+0000","last_active":"2024-04-14T08:33:57.973905+0000","last_peered":"2024-04-14T08:33:57.973905+0000","last_clean":"2024-04-14T08:33:57.973905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:57.973905+0000","last_undegraded":"2024-04-14T08:33:57.973905+0000","last_fullsized":"2024-04-14T08:33:57.973905+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013358+0000","last_change":"2024-04-14T08:34:00.010101+0000","last_active":"2024-04-14T08:34:00.013358+0000","last_peered":"2024-04-14T08:34:00.013358+0000","last_clean":"2024-04-14T08:34:00.013358+0000","last_became_active":"2024-04-14T08:34:00.006751+0000","last_became_peered":"2024-04-14T08:34:00.006751+0000","last_unstale":"2024-04-14T08:34:00.013358+0000","last_undegraded":"2024-04-14T08:34:00.013358+0000","last_fullsized":"2024-04-14T08:34:00.013358+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:03.589235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.993637+0000","last_change":"2024-04-14T08:33:59.005088+0000","last_active":"2024-04-14T08:33:59.993637+0000","last_peered":"2024-04-14T08:33:59.993637+0000","last_clean":"2024-04-14T08:33:59.993637+0000","last_became_active":"2024-04-14T08:33:59.004802+0000","last_became_peered":"2024-04-14T08:33:59.004802+0000","last_unstale":"2024-04-14T08:33:59.993637+0000","last_undegraded":"2024-04-14T08:33:59.993637+0000","last_fullsized":"2024-04-14T08:33:59.993637+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:41.482824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.012014+0000","last_change":"2024-04-14T08:34:00.010515+0000","last_active":"2024-04-14T08:34:00.012014+0000","last_peered":"2024-04-14T08:34:00.012014+0000","last_clean":"2024-04-14T08:34:00.012014+0000","last_became_active":"2024-04-14T08:34:00.010270+0000","last_became_peered":"2024-04-14T08:34:00.010270+0000","last_unstale":"2024-04-14T08:34:00.012014+0000","last_undegraded":"2024-04-14T08:34:00.012014+0000","last_fullsized":"2024-04-14T08:34:00.012014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:17:40.287151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.996028+0000","last_change":"2024-04-14T08:33:59.009167+0000","last_active":"2024-04-14T08:33:59.996028+0000","last_peered":"2024-04-14T08:33:59.996028+0000","last_clean":"2024-04-14T08:33:59.996028+0000","last_became_active":"2024-04-14T08:33:59.008423+0000","last_became_peered":"2024-04-14T08:33:59.008423+0000","last_unstale":"2024-04-14T08:33:59.996028+0000","last_undegraded":"2024-04-14T08:33:59.996028+0000","last_fullsized":"2024-04-14T08:33:59.996028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:51:58.159319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.006940+0000","last_change":"2024-04-14T08:34:00.003551+0000","last_active":"2024-04-14T08:34:00.006940+0000","last_peered":"2024-04-14T08:34:00.006940+0000","last_clean":"2024-04-14T08:34:00.006940+0000","last_became_active":"2024-04-14T08:34:00.002714+0000","last_became_peered":"2024-04-14T08:34:00.002714+0000","last_unstale":"2024-04-14T08:34:00.006940+0000","last_undegraded":"2024-04-14T08:34:00.006940+0000","last_fullsized":"2024-04-14T08:34:00.006940+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:27:54.003457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.991943+0000","last_change":"2024-04-14T08:33:59.001289+0000","last_active":"2024-04-14T08:33:59.991943+0000","last_peered":"2024-04-14T08:33:59.991943+0000","last_clean":"2024-04-14T08:33:59.991943+0000","last_became_active":"2024-04-14T08:33:59.001011+0000","last_became_peered":"2024-04-14T08:33:59.001011+0000","last_unstale":"2024-04-14T08:33:59.991943+0000","last_undegraded":"2024-04-14T08:33:59.991943+0000","last_fullsized":"2024-04-14T08:33:59.991943+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:12.326349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:58.978291+0000","last_change":"2024-04-14T08:33:58.978291+0000","last_active":"2024-04-14T08:33:58.978291+0000","last_peered":"2024-04-14T08:33:58.978291+0000","last_clean":"2024-04-14T08:33:58.978291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:58.978291+0000","last_undegraded":"2024-04-14T08:33:58.978291+0000","last_fullsized":"2024-04-14T08:33:58.978291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.991364+0000","last_change":"2024-04-14T08:33:58.997424+0000","last_active":"2024-04-14T08:33:59.991364+0000","last_peered":"2024-04-14T08:33:59.991364+0000","last_clean":"2024-04-14T08:33:59.991364+0000","last_became_active":"2024-04-14T08:33:58.997087+0000","last_became_peered":"2024-04-14T08:33:58.997087+0000","last_unstale":"2024-04-14T08:33:59.991364+0000","last_undegraded":"2024-04-14T08:33:59.991364+0000","last_fullsized":"2024-04-14T08:33:59.991364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:19.671894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005025+0000","last_change":"2024-04-14T08:33:59.995244+0000","last_active":"2024-04-14T08:34:00.005025+0000","last_peered":"2024-04-14T08:34:00.005025+0000","last_clean":"2024-04-14T08:34:00.005025+0000","last_became_active":"2024-04-14T08:33:59.994422+0000","last_became_peered":"2024-04-14T08:33:59.994422+0000","last_unstale":"2024-04-14T08:34:00.005025+0000","last_undegraded":"2024-04-14T08:34:00.005025+0000","last_fullsized":"2024-04-14T08:34:00.005025+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:26:02.000109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.019538+0000","last_change":"2024-04-14T08:33:59.001631+0000","last_active":"2024-04-14T08:34:00.019538+0000","last_peered":"2024-04-14T08:34:00.019538+0000","last_clean":"2024-04-14T08:34:00.019538+0000","last_became_active":"2024-04-14T08:33:59.001173+0000","last_became_peered":"2024-04-14T08:33:59.001173+0000","last_unstale":"2024-04-14T08:34:00.019538+0000","last_undegraded":"2024-04-14T08:34:00.019538+0000","last_fullsized":"2024-04-14T08:34:00.019538+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:16:14.774984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013238+0000","last_change":"2024-04-14T08:34:00.004593+0000","last_active":"2024-04-14T08:34:00.013238+0000","last_peered":"2024-04-14T08:34:00.013238+0000","last_clean":"2024-04-14T08:34:00.013238+0000","last_became_active":"2024-04-14T08:34:00.004312+0000","last_became_peered":"2024-04-14T08:34:00.004312+0000","last_unstale":"2024-04-14T08:34:00.013238+0000","last_undegraded":"2024-04-14T08:34:00.013238+0000","last_fullsized":"2024-04-14T08:34:00.013238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:16:15.779370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.024437+0000","last_change":"2024-04-14T08:33:59.002849+0000","last_active":"2024-04-14T08:34:00.024437+0000","last_peered":"2024-04-14T08:34:00.024437+0000","last_clean":"2024-04-14T08:34:00.024437+0000","last_became_active":"2024-04-14T08:33:59.002544+0000","last_became_peered":"2024-04-14T08:33:59.002544+0000","last_unstale":"2024-04-14T08:34:00.024437+0000","last_undegraded":"2024-04-14T08:34:00.024437+0000","last_fullsized":"2024-04-14T08:34:00.024437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:46:25.541897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.010976+0000","last_change":"2024-04-14T08:34:00.005921+0000","last_active":"2024-04-14T08:34:00.010976+0000","last_peered":"2024-04-14T08:34:00.010976+0000","last_clean":"2024-04-14T08:34:00.010976+0000","last_became_active":"2024-04-14T08:34:00.005602+0000","last_became_peered":"2024-04-14T08:34:00.005602+0000","last_unstale":"2024-04-14T08:34:00.010976+0000","last_undegraded":"2024-04-14T08:34:00.010976+0000","last_fullsized":"2024-04-14T08:34:00.010976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:46:26.546283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.015832+0000","last_change":"2024-04-14T08:33:59.004932+0000","last_active":"2024-04-14T08:34:00.015832+0000","last_peered":"2024-04-14T08:34:00.015832+0000","last_clean":"2024-04-14T08:34:00.015832+0000","last_became_active":"2024-04-14T08:33:59.004618+0000","last_became_peered":"2024-04-14T08:33:59.004618+0000","last_unstale":"2024-04-14T08:34:00.015832+0000","last_undegraded":"2024-04-14T08:34:00.015832+0000","last_fullsized":"2024-04-14T08:34:00.015832+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:16.440202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.002086+0000","last_change":"2024-04-14T08:33:59.999556+0000","last_active":"2024-04-14T08:34:00.002086+0000","last_peered":"2024-04-14T08:34:00.002086+0000","last_clean":"2024-04-14T08:34:00.002086+0000","last_became_active":"2024-04-14T08:33:59.999303+0000","last_became_peered":"2024-04-14T08:33:59.999303+0000","last_unstale":"2024-04-14T08:34:00.002086+0000","last_undegraded":"2024-04-14T08:34:00.002086+0000","last_fullsized":"2024-04-14T08:34:00.002086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:40:27.541789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.993520+0000","last_change":"2024-04-14T08:33:59.006438+0000","last_active":"2024-04-14T08:33:59.993520+0000","last_peered":"2024-04-14T08:33:59.993520+0000","last_clean":"2024-04-14T08:33:59.993520+0000","last_became_active":"2024-04-14T08:33:59.006251+0000","last_became_peered":"2024-04-14T08:33:59.006251+0000","last_unstale":"2024-04-14T08:33:59.993520+0000","last_undegraded":"2024-04-14T08:33:59.993520+0000","last_fullsized":"2024-04-14T08:33:59.993520+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:00:19.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:58.978291+0000","last_change":"2024-04-14T08:33:58.978291+0000","last_active":"2024-04-14T08:33:58.978291+0000","last_peered":"2024-04-14T08:33:58.978291+0000","last_clean":"2024-04-14T08:33:58.978291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:58.978291+0000","last_undegraded":"2024-04-14T08:33:58.978291+0000","last_fullsized":"2024-04-14T08:33:58.978291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.994864+0000","last_change":"2024-04-14T08:33:59.009109+0000","last_active":"2024-04-14T08:33:59.994864+0000","last_peered":"2024-04-14T08:33:59.994864+0000","last_clean":"2024-04-14T08:33:59.994864+0000","last_became_active":"2024-04-14T08:33:59.008773+0000","last_became_peered":"2024-04-14T08:33:59.008773+0000","last_unstale":"2024-04-14T08:33:59.994864+0000","last_undegraded":"2024-04-14T08:33:59.994864+0000","last_fullsized":"2024-04-14T08:33:59.994864+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:16.364611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.003541+0000","last_change":"2024-04-14T08:34:00.000887+0000","last_active":"2024-04-14T08:34:00.003541+0000","last_peered":"2024-04-14T08:34:00.003541+0000","last_clean":"2024-04-14T08:34:00.003541+0000","last_became_active":"2024-04-14T08:34:00.000624+0000","last_became_peered":"2024-04-14T08:34:00.000624+0000","last_unstale":"2024-04-14T08:34:00.003541+0000","last_undegraded":"2024-04-14T08:34:00.003541+0000","last_fullsized":"2024-04-14T08:34:00.003541+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:52.044408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:57.973905+0000","last_change":"2024-04-14T08:33:57.973905+0000","last_active":"2024-04-14T08:33:57.973905+0000","last_peered":"2024-04-14T08:33:57.973905+0000","last_clean":"2024-04-14T08:33:57.973905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:57.973905+0000","last_undegraded":"2024-04-14T08:33:57.973905+0000","last_fullsized":"2024-04-14T08:33:57.973905+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.009302+0000","last_change":"2024-04-14T08:34:00.005246+0000","last_active":"2024-04-14T08:34:00.009302+0000","last_peered":"2024-04-14T08:34:00.009302+0000","last_clean":"2024-04-14T08:34:00.009302+0000","last_became_active":"2024-04-14T08:34:00.004868+0000","last_became_peered":"2024-04-14T08:34:00.004868+0000","last_unstale":"2024-04-14T08:34:00.009302+0000","last_undegraded":"2024-04-14T08:34:00.009302+0000","last_fullsized":"2024-04-14T08:34:00.009302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:09:02.249499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.019305+0000","last_change":"2024-04-14T08:33:59.010220+0000","last_active":"2024-04-14T08:34:00.019305+0000","last_peered":"2024-04-14T08:34:00.019305+0000","last_clean":"2024-04-14T08:34:00.019305+0000","last_became_active":"2024-04-14T08:33:59.009884+0000","last_became_peered":"2024-04-14T08:33:59.009884+0000","last_unstale":"2024-04-14T08:34:00.019305+0000","last_undegraded":"2024-04-14T08:34:00.019305+0000","last_fullsized":"2024-04-14T08:34:00.019305+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:12.326349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005372+0000","last_change":"2024-04-14T08:34:00.001352+0000","last_active":"2024-04-14T08:34:00.005372+0000","last_peered":"2024-04-14T08:34:00.005372+0000","last_clean":"2024-04-14T08:34:00.005372+0000","last_became_active":"2024-04-14T08:34:00.001024+0000","last_became_peered":"2024-04-14T08:34:00.001024+0000","last_unstale":"2024-04-14T08:34:00.005372+0000","last_undegraded":"2024-04-14T08:34:00.005372+0000","last_fullsized":"2024-04-14T08:34:00.005372+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:46:26.546283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.019590+0000","last_change":"2024-04-14T08:33:59.006429+0000","last_active":"2024-04-14T08:34:00.019590+0000","last_peered":"2024-04-14T08:34:00.019590+0000","last_clean":"2024-04-14T08:34:00.019590+0000","last_became_active":"2024-04-14T08:33:59.006136+0000","last_became_peered":"2024-04-14T08:33:59.006136+0000","last_unstale":"2024-04-14T08:34:00.019590+0000","last_undegraded":"2024-04-14T08:34:00.019590+0000","last_fullsized":"2024-04-14T08:34:00.019590+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:00:19.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.011329+0000","last_change":"2024-04-14T08:34:00.002231+0000","last_active":"2024-04-14T08:34:00.011329+0000","last_peered":"2024-04-14T08:34:00.011329+0000","last_clean":"2024-04-14T08:34:00.011329+0000","last_became_active":"2024-04-14T08:34:00.001970+0000","last_became_peered":"2024-04-14T08:34:00.001970+0000","last_unstale":"2024-04-14T08:34:00.011329+0000","last_undegraded":"2024-04-14T08:34:00.011329+0000","last_fullsized":"2024-04-14T08:34:00.011329+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:42.487210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.012960+0000","last_change":"2024-04-14T08:33:59.002800+0000","last_active":"2024-04-14T08:34:00.012960+0000","last_peered":"2024-04-14T08:34:00.012960+0000","last_clean":"2024-04-14T08:34:00.012960+0000","last_became_active":"2024-04-14T08:33:59.002508+0000","last_became_peered":"2024-04-14T08:33:59.002508+0000","last_unstale":"2024-04-14T08:34:00.012960+0000","last_undegraded":"2024-04-14T08:34:00.012960+0000","last_fullsized":"2024-04-14T08:34:00.012960+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:00:46.198672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.002066+0000","last_change":"2024-04-14T08:33:59.995510+0000","last_active":"2024-04-14T08:34:00.002066+0000","last_peered":"2024-04-14T08:34:00.002066+0000","last_clean":"2024-04-14T08:34:00.002066+0000","last_became_active":"2024-04-14T08:33:59.994780+0000","last_became_peered":"2024-04-14T08:33:59.994780+0000","last_unstale":"2024-04-14T08:34:00.002066+0000","last_undegraded":"2024-04-14T08:34:00.002066+0000","last_fullsized":"2024-04-14T08:34:00.002066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:48:41.632164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.993255+0000","last_change":"2024-04-14T08:33:59.009221+0000","last_active":"2024-04-14T08:33:59.993255+0000","last_peered":"2024-04-14T08:33:59.993255+0000","last_clean":"2024-04-14T08:33:59.993255+0000","last_became_active":"2024-04-14T08:33:59.008623+0000","last_became_peered":"2024-04-14T08:33:59.008623+0000","last_unstale":"2024-04-14T08:33:59.993255+0000","last_undegraded":"2024-04-14T08:33:59.993255+0000","last_fullsized":"2024-04-14T08:33:59.993255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:01:06.793819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:58.978291+0000","last_change":"2024-04-14T08:33:58.978291+0000","last_active":"2024-04-14T08:33:58.978291+0000","last_peered":"2024-04-14T08:33:58.978291+0000","last_clean":"2024-04-14T08:33:58.978291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:58.978291+0000","last_undegraded":"2024-04-14T08:33:58.978291+0000","last_fullsized":"2024-04-14T08:33:58.978291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013656+0000","last_change":"2024-04-14T08:33:59.004301+0000","last_active":"2024-04-14T08:34:00.013656+0000","last_peered":"2024-04-14T08:34:00.013656+0000","last_clean":"2024-04-14T08:34:00.013656+0000","last_became_active":"2024-04-14T08:33:59.004035+0000","last_became_peered":"2024-04-14T08:33:59.004035+0000","last_unstale":"2024-04-14T08:34:00.013656+0000","last_undegraded":"2024-04-14T08:34:00.013656+0000","last_fullsized":"2024-04-14T08:34:00.013656+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:01:43.795521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.004497+0000","last_change":"2024-04-14T08:33:59.994730+0000","last_active":"2024-04-14T08:34:00.004497+0000","last_peered":"2024-04-14T08:34:00.004497+0000","last_clean":"2024-04-14T08:34:00.004497+0000","last_became_active":"2024-04-14T08:33:59.994468+0000","last_became_peered":"2024-04-14T08:33:59.994468+0000","last_unstale":"2024-04-14T08:34:00.004497+0000","last_undegraded":"2024-04-14T08:34:00.004497+0000","last_fullsized":"2024-04-14T08:34:00.004497+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:20.676280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.986039+0000","last_change":"2024-04-14T08:33:59.008949+0000","last_active":"2024-04-14T08:33:59.986039+0000","last_peered":"2024-04-14T08:33:59.986039+0000","last_clean":"2024-04-14T08:33:59.986039+0000","last_became_active":"2024-04-14T08:33:59.008663+0000","last_became_peered":"2024-04-14T08:33:59.008663+0000","last_unstale":"2024-04-14T08:33:59.986039+0000","last_undegraded":"2024-04-14T08:33:59.986039+0000","last_fullsized":"2024-04-14T08:33:59.986039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:55:08.918254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007793+0000","last_change":"2024-04-14T08:34:00.003470+0000","last_active":"2024-04-14T08:34:00.007793+0000","last_peered":"2024-04-14T08:34:00.007793+0000","last_clean":"2024-04-14T08:34:00.007793+0000","last_became_active":"2024-04-14T08:34:00.003182+0000","last_became_peered":"2024-04-14T08:34:00.003182+0000","last_unstale":"2024-04-14T08:34:00.007793+0000","last_undegraded":"2024-04-14T08:34:00.007793+0000","last_fullsized":"2024-04-14T08:34:00.007793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:31.409872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:57.973905+0000","last_change":"2024-04-14T08:33:57.973905+0000","last_active":"2024-04-14T08:33:57.973905+0000","last_peered":"2024-04-14T08:33:57.973905+0000","last_clean":"2024-04-14T08:33:57.973905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:57.973905+0000","last_undegraded":"2024-04-14T08:33:57.973905+0000","last_fullsized":"2024-04-14T08:33:57.973905+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005304+0000","last_change":"2024-04-14T08:34:00.001744+0000","last_active":"2024-04-14T08:34:00.005304+0000","last_peered":"2024-04-14T08:34:00.005304+0000","last_clean":"2024-04-14T08:34:00.005304+0000","last_became_active":"2024-04-14T08:34:00.001348+0000","last_became_peered":"2024-04-14T08:34:00.001348+0000","last_unstale":"2024-04-14T08:34:00.005304+0000","last_undegraded":"2024-04-14T08:34:00.005304+0000","last_fullsized":"2024-04-14T08:34:00.005304+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:21.676556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.986036+0000","last_change":"2024-04-14T08:33:58.999302+0000","last_active":"2024-04-14T08:33:59.986036+0000","last_peered":"2024-04-14T08:33:59.986036+0000","last_clean":"2024-04-14T08:33:59.986036+0000","last_became_active":"2024-04-14T08:33:58.999046+0000","last_became_peered":"2024-04-14T08:33:58.999046+0000","last_unstale":"2024-04-14T08:33:59.986036+0000","last_undegraded":"2024-04-14T08:33:59.986036+0000","last_fullsized":"2024-04-14T08:33:59.986036+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:23:43.943029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.008704+0000","last_change":"2024-04-14T08:34:00.005938+0000","last_active":"2024-04-14T08:34:00.008704+0000","last_peered":"2024-04-14T08:34:00.008704+0000","last_clean":"2024-04-14T08:34:00.008704+0000","last_became_active":"2024-04-14T08:34:00.005662+0000","last_became_peered":"2024-04-14T08:34:00.005662+0000","last_unstale":"2024-04-14T08:34:00.008704+0000","last_undegraded":"2024-04-14T08:34:00.008704+0000","last_fullsized":"2024-04-14T08:34:00.008704+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:17.444588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.019390+0000","last_change":"2024-04-14T08:33:59.001989+0000","last_active":"2024-04-14T08:34:00.019390+0000","last_peered":"2024-04-14T08:34:00.019390+0000","last_clean":"2024-04-14T08:34:00.019390+0000","last_became_active":"2024-04-14T08:33:59.001644+0000","last_became_peered":"2024-04-14T08:33:59.001644+0000","last_unstale":"2024-04-14T08:34:00.019390+0000","last_undegraded":"2024-04-14T08:34:00.019390+0000","last_fullsized":"2024-04-14T08:34:00.019390+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:16.440202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:58.978291+0000","last_change":"2024-04-14T08:33:58.978291+0000","last_active":"2024-04-14T08:33:58.978291+0000","last_peered":"2024-04-14T08:33:58.978291+0000","last_clean":"2024-04-14T08:33:58.978291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:58.978291+0000","last_undegraded":"2024-04-14T08:33:58.978291+0000","last_fullsized":"2024-04-14T08:33:58.978291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.986249+0000","last_change":"2024-04-14T08:33:58.998567+0000","last_active":"2024-04-14T08:33:59.986249+0000","last_peered":"2024-04-14T08:33:59.986249+0000","last_clean":"2024-04-14T08:33:59.986249+0000","last_became_active":"2024-04-14T08:33:58.998302+0000","last_became_peered":"2024-04-14T08:33:58.998302+0000","last_unstale":"2024-04-14T08:33:59.986249+0000","last_undegraded":"2024-04-14T08:33:59.986249+0000","last_fullsized":"2024-04-14T08:33:59.986249+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:27:19.258542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.011061+0000","last_change":"2024-04-14T08:34:00.009623+0000","last_active":"2024-04-14T08:34:00.011061+0000","last_peered":"2024-04-14T08:34:00.011061+0000","last_clean":"2024-04-14T08:34:00.011061+0000","last_became_active":"2024-04-14T08:34:00.007666+0000","last_became_peered":"2024-04-14T08:34:00.007666+0000","last_unstale":"2024-04-14T08:34:00.011061+0000","last_undegraded":"2024-04-14T08:34:00.011061+0000","last_fullsized":"2024-04-14T08:34:00.011061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:58:48.218603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.989122+0000","last_change":"2024-04-14T08:33:59.001386+0000","last_active":"2024-04-14T08:33:59.989122+0000","last_peered":"2024-04-14T08:33:59.989122+0000","last_clean":"2024-04-14T08:33:59.989122+0000","last_became_active":"2024-04-14T08:33:59.001135+0000","last_became_peered":"2024-04-14T08:33:59.001135+0000","last_unstale":"2024-04-14T08:33:59.989122+0000","last_undegraded":"2024-04-14T08:33:59.989122+0000","last_fullsized":"2024-04-14T08:33:59.989122+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:01:06.793819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005776+0000","last_change":"2024-04-14T08:33:59.996287+0000","last_active":"2024-04-14T08:34:00.005776+0000","last_peered":"2024-04-14T08:34:00.005776+0000","last_clean":"2024-04-14T08:34:00.005776+0000","last_became_active":"2024-04-14T08:33:59.996006+0000","last_became_peered":"2024-04-14T08:33:59.996006+0000","last_unstale":"2024-04-14T08:34:00.005776+0000","last_undegraded":"2024-04-14T08:34:00.005776+0000","last_fullsized":"2024-04-14T08:34:00.005776+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:13.330734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.989285+0000","last_change":"2024-04-14T08:33:59.003581+0000","last_active":"2024-04-14T08:33:59.989285+0000","last_peered":"2024-04-14T08:33:59.989285+0000","last_clean":"2024-04-14T08:33:59.989285+0000","last_became_active":"2024-04-14T08:33:59.003154+0000","last_became_peered":"2024-04-14T08:33:59.003154+0000","last_unstale":"2024-04-14T08:33:59.989285+0000","last_undegraded":"2024-04-14T08:33:59.989285+0000","last_fullsized":"2024-04-14T08:33:59.989285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:41.482824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.008295+0000","last_change":"2024-04-14T08:34:00.003966+0000","last_active":"2024-04-14T08:34:00.008295+0000","last_peered":"2024-04-14T08:34:00.008295+0000","last_clean":"2024-04-14T08:34:00.008295+0000","last_became_active":"2024-04-14T08:34:00.003295+0000","last_became_peered":"2024-04-14T08:34:00.003295+0000","last_unstale":"2024-04-14T08:34:00.008295+0000","last_undegraded":"2024-04-14T08:34:00.008295+0000","last_fullsized":"2024-04-14T08:34:00.008295+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:13.330734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.997075+0000","last_change":"2024-04-14T08:33:59.003054+0000","last_active":"2024-04-14T08:33:59.997075+0000","last_peered":"2024-04-14T08:33:59.997075+0000","last_clean":"2024-04-14T08:33:59.997075+0000","last_became_active":"2024-04-14T08:33:59.002711+0000","last_became_peered":"2024-04-14T08:33:59.002711+0000","last_unstale":"2024-04-14T08:33:59.997075+0000","last_undegraded":"2024-04-14T08:33:59.997075+0000","last_fullsized":"2024-04-14T08:33:59.997075+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:42:45.693051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013301+0000","last_change":"2024-04-14T08:34:00.008844+0000","last_active":"2024-04-14T08:34:00.013301+0000","last_peered":"2024-04-14T08:34:00.013301+0000","last_clean":"2024-04-14T08:34:00.013301+0000","last_became_active":"2024-04-14T08:34:00.008352+0000","last_became_peered":"2024-04-14T08:34:00.008352+0000","last_unstale":"2024-04-14T08:34:00.013301+0000","last_undegraded":"2024-04-14T08:34:00.013301+0000","last_fullsized":"2024-04-14T08:34:00.013301+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:17.444588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.997814+0000","last_change":"2024-04-14T08:33:59.008769+0000","last_active":"2024-04-14T08:33:59.997814+0000","last_peered":"2024-04-14T08:33:59.997814+0000","last_clean":"2024-04-14T08:33:59.997814+0000","last_became_active":"2024-04-14T08:33:59.008495+0000","last_became_peered":"2024-04-14T08:33:59.008495+0000","last_unstale":"2024-04-14T08:33:59.997814+0000","last_undegraded":"2024-04-14T08:33:59.997814+0000","last_fullsized":"2024-04-14T08:33:59.997814+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:42:45.693051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:58.978291+0000","last_change":"2024-04-14T08:33:58.978291+0000","last_active":"2024-04-14T08:33:58.978291+0000","last_peered":"2024-04-14T08:33:58.978291+0000","last_clean":"2024-04-14T08:33:58.978291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:58.978291+0000","last_undegraded":"2024-04-14T08:33:58.978291+0000","last_fullsized":"2024-04-14T08:33:58.978291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.992314+0000","last_change":"2024-04-14T08:33:59.001251+0000","last_active":"2024-04-14T08:33:59.992314+0000","last_peered":"2024-04-14T08:33:59.992314+0000","last_clean":"2024-04-14T08:33:59.992314+0000","last_became_active":"2024-04-14T08:33:59.000965+0000","last_became_peered":"2024-04-14T08:33:59.000965+0000","last_unstale":"2024-04-14T08:33:59.992314+0000","last_undegraded":"2024-04-14T08:33:59.992314+0000","last_fullsized":"2024-04-14T08:33:59.992314+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:02.584849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007931+0000","last_change":"2024-04-14T08:34:00.002427+0000","last_active":"2024-04-14T08:34:00.007931+0000","last_peered":"2024-04-14T08:34:00.007931+0000","last_clean":"2024-04-14T08:34:00.007931+0000","last_became_active":"2024-04-14T08:34:00.002155+0000","last_became_peered":"2024-04-14T08:34:00.002155+0000","last_unstale":"2024-04-14T08:34:00.007931+0000","last_undegraded":"2024-04-14T08:34:00.007931+0000","last_fullsized":"2024-04-14T08:34:00.007931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:54.465535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:33:57.973905+0000","last_change":"2024-04-14T08:33:57.973905+0000","last_active":"2024-04-14T08:33:57.973905+0000","last_peered":"2024-04-14T08:33:57.973905+0000","last_clean":"2024-04-14T08:33:57.973905+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:33:57.973905+0000","last_undegraded":"2024-04-14T08:33:57.973905+0000","last_fullsized":"2024-04-14T08:33:57.973905+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":198,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-14T08:33:53.990936+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:33:53.990936+0000","last_peered":"2024-04-14T08:33:53.990936+0000","last_clean":"2024-04-14T08:33:53.990936+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:33:53.990936+0000","last_undegraded":"2024-04-14T08:33:53.990936+0000","last_fullsized":"2024-04-14T08:33:53.990936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":56,"acting":56,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":995328,"data_stored":758340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":778417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:34:03.050 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:34:03.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.188+0000 7f29ee0c0640 1 Processor -- start 2024-04-14T08:34:03.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.188+0000 7f29ee0c0640 1 -- start start 2024-04-14T08:34:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.189+0000 7f29ee0c0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29e80d2830 0x7f29e80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.189+0000 7f29ee0c0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f29e815e890 0x7f29e80d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.189+0000 7f29ee0c0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e80d3150 con 0x7f29e80cedd0 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.189+0000 7f29ee0c0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f29e80cff30 con 0x7f29e815e890 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.189+0000 7f29ee0c0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f29e80d2350 con 0x7f29e80d2830 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.189+0000 7f29e7fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f29e80cedd0 0x7f29e81621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48316/0 (socket says 172.21.15.172:48316) 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.189+0000 7f29e7fff640 1 -- 172.21.15.172:0/2711753525 learned_addr learned my addr 172.21.15.172:0/2711753525 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.189+0000 7f29e77fe640 1 --2- 172.21.15.172:0/2711753525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29e80d2830 0x7f29e80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.189+0000 7f29dffff640 1 --2- 172.21.15.172:0/2711753525 >> v2:172.21.15.177:3300/0 conn(0x7f29e815e890 0x7f29e80d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.190+0000 7f29e6ffd640 1 -- 172.21.15.172:0/2711753525 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4177066704 0 0) 0x7f29e80d3150 con 0x7f29e80cedd0 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.190+0000 7f29e6ffd640 1 -- 172.21.15.172:0/2711753525 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29d4003520 con 0x7f29e80cedd0 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.190+0000 7f29e77fe640 1 -- 172.21.15.172:0/2711753525 >> v2:172.21.15.177:3300/0 conn(0x7f29e815e890 msgr2=0x7f29e80d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.190+0000 7f29e77fe640 1 --2- 172.21.15.172:0/2711753525 >> v2:172.21.15.177:3300/0 conn(0x7f29e815e890 0x7f29e80d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.190+0000 7f29e77fe640 1 -- 172.21.15.172:0/2711753525 >> v1:172.21.15.172:6789/0 conn(0x7f29e80cedd0 legacy=0x7f29e81621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.190+0000 7f29e77fe640 1 -- 172.21.15.172:0/2711753525 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e80cf320 con 0x7f29e80d2830 2024-04-14T08:34:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.190+0000 7f29dffff640 1 --2- 172.21.15.172:0/2711753525 >> v2:172.21.15.177:3300/0 conn(0x7f29e815e890 0x7f29e80d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:34:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.191+0000 7f29e77fe640 1 --2- 172.21.15.172:0/2711753525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29e80d2830 0x7f29e80d2c10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f29d80097a0 tx=0x7f29d802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f181f9b0d25b8a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.191+0000 7f29e6ffd640 1 -- 172.21.15.172:0/2711753525 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8034070 con 0x7f29e80d2830 2024-04-14T08:34:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.191+0000 7f29e6ffd640 1 -- 172.21.15.172:0/2711753525 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d8030610 con 0x7f29e80d2830 2024-04-14T08:34:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.191+0000 7f29e6ffd640 1 -- 172.21.15.172:0/2711753525 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d80309e0 con 0x7f29e80d2830 2024-04-14T08:34:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.191+0000 7f29ee0c0640 1 -- 172.21.15.172:0/2711753525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29e80d2830 msgr2=0x7f29e80d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.191+0000 7f29ee0c0640 1 --2- 172.21.15.172:0/2711753525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29e80d2830 0x7f29e80d2c10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f29d80097a0 tx=0x7f29d802b960 comp rx=0 tx=0).stop 2024-04-14T08:34:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.191+0000 7f29ee0c0640 1 -- 172.21.15.172:0/2711753525 shutdown_connections 2024-04-14T08:34:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.191+0000 7f29ee0c0640 1 --2- 172.21.15.172:0/2711753525 >> v2:172.21.15.177:3300/0 conn(0x7f29e815e890 0x7f29e80d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.191+0000 7f29ee0c0640 1 --2- 172.21.15.172:0/2711753525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29e80d2830 0x7f29e80d2c10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.191+0000 7f29ee0c0640 1 -- 172.21.15.172:0/2711753525 >> 172.21.15.172:0/2711753525 conn(0x7f29e809d590 msgr2=0x7f29e80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:03.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.191+0000 7f29ee0c0640 1 -- 172.21.15.172:0/2711753525 shutdown_connections 2024-04-14T08:34:03.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.191+0000 7f29ee0c0640 1 -- 172.21.15.172:0/2711753525 wait complete. 2024-04-14T08:34:03.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.193+0000 7f29ee0c0640 1 Processor -- start 2024-04-14T08:34:03.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.193+0000 7f29ee0c0640 1 -- start start 2024-04-14T08:34:03.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.193+0000 7f29ee0c0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29e80cedd0 0x7f29e80cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.193+0000 7f29ee0c0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29e80d2830 0x7f29e80c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.193+0000 7f29ee0c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29e815e890 0x7f29e80ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.193+0000 7f29ee0c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f29e80cfbc0 con 0x7f29e815e890 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.193+0000 7f29ee0c0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f29e80cf8e0 con 0x7f29e80cedd0 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.193+0000 7f29ee0c0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f29e80d1840 con 0x7f29e80d2830 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.193+0000 7f29e7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29e815e890 0x7f29e80ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.193+0000 7f29e7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29e815e890 0x7f29e80ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48698/0 (socket says 172.21.15.172:48698) 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.193+0000 7f29e7fff640 1 -- 172.21.15.172:0/3660012497 learned_addr learned my addr 172.21.15.172:0/3660012497 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29e77fe640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29e80cedd0 0x7f29e80cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29dffff640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29e80d2830 0x7f29e80c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29e7fff640 1 -- 172.21.15.172:0/3660012497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29e80d2830 msgr2=0x7f29e80c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29e7fff640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29e80d2830 0x7f29e80c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29e7fff640 1 -- 172.21.15.172:0/3660012497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29e80cedd0 msgr2=0x7f29e80cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29e7fff640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29e80cedd0 0x7f29e80cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29e7fff640 1 -- 172.21.15.172:0/3660012497 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e815da50 con 0x7f29e815e890 2024-04-14T08:34:03.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29e7fff640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29e815e890 0x7f29e80ccf20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f29e000caa0 tx=0x7f29e000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:03.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29e4ff9640 1 -- 172.21.15.172:0/3660012497 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0017070 con 0x7f29e815e890 2024-04-14T08:34:03.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29e4ff9640 1 -- 172.21.15.172:0/3660012497 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e000ac00 con 0x7f29e815e890 2024-04-14T08:34:03.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29ee0c0640 1 -- 172.21.15.172:0/3660012497 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e80cb900 con 0x7f29e815e890 2024-04-14T08:34:03.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29e4ff9640 1 -- 172.21.15.172:0/3660012497 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0012400 con 0x7f29e815e890 2024-04-14T08:34:03.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.194+0000 7f29ee0c0640 1 -- 172.21.15.172:0/3660012497 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e80c9460 con 0x7f29e815e890 2024-04-14T08:34:03.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.195+0000 7f29ee0c0640 1 -- 172.21.15.172:0/3660012497 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e8068520 con 0x7f29e815e890 2024-04-14T08:34:03.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.196+0000 7f29e4ff9640 1 -- 172.21.15.172:0/3660012497 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29e001a070 con 0x7f29e815e890 2024-04-14T08:34:03.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.196+0000 7f29e4ff9640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f29c806c8f0 0x7f29c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.196+0000 7f29e4ff9640 1 -- 172.21.15.172:0/3660012497 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f29e008ba40 con 0x7f29e815e890 2024-04-14T08:34:03.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.196+0000 7f29e77fe640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f29c806c8f0 0x7f29c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.197+0000 7f29e77fe640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f29c806c8f0 0x7f29c806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f29d8036000 tx=0x7f29d803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:03.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.201+0000 7f29e4ff9640 1 -- 172.21.15.172:0/3660012497 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f29e0059690 con 0x7f29e815e890 2024-04-14T08:34:03.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.373+0000 7f29ee0c0640 1 -- 172.21.15.172:0/3660012497 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29e80d2610 con 0x7f29c806c8f0 2024-04-14T08:34:03.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.379+0000 7f29e4ff9640 1 -- 172.21.15.172:0/3660012497 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251899 (secure 0 0 0) 0x7f29e80d2610 con 0x7f29c806c8f0 2024-04-14T08:34:03.383 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:34:03.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.382+0000 7f29ee0c0640 1 -- 172.21.15.172:0/3660012497 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f29c806c8f0 msgr2=0x7f29c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.382+0000 7f29ee0c0640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f29c806c8f0 0x7f29c806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f29d8036000 tx=0x7f29d803b040 comp rx=0 tx=0).stop 2024-04-14T08:34:03.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.382+0000 7f29ee0c0640 1 -- 172.21.15.172:0/3660012497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29e815e890 msgr2=0x7f29e80ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.382+0000 7f29ee0c0640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29e815e890 0x7f29e80ccf20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f29e000caa0 tx=0x7f29e000cf60 comp rx=0 tx=0).stop 2024-04-14T08:34:03.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.382+0000 7f29ee0c0640 1 -- 172.21.15.172:0/3660012497 shutdown_connections 2024-04-14T08:34:03.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.382+0000 7f29ee0c0640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f29c806c8f0 0x7f29c806edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.382+0000 7f29ee0c0640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29e815e890 0x7f29e80ccf20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.382+0000 7f29ee0c0640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29e80d2830 0x7f29e80c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.382+0000 7f29ee0c0640 1 --2- 172.21.15.172:0/3660012497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29e80cedd0 0x7f29e80cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.382+0000 7f29ee0c0640 1 -- 172.21.15.172:0/3660012497 >> 172.21.15.172:0/3660012497 conn(0x7f29e809d590 msgr2=0x7f29e80b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:03.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.382+0000 7f29ee0c0640 1 -- 172.21.15.172:0/3660012497 shutdown_connections 2024-04-14T08:34:03.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.382+0000 7f29ee0c0640 1 -- 172.21.15.172:0/3660012497 wait complete. 2024-04-14T08:34:03.395 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-14T08:34:03.134976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220716,"kb_used_data":4532,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754004,"statfs":{"total":773094113280,"available":772868100096,"internally_reserved":0,"allocated":4640768,"data_stored":2643570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.156420"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.009281+0000","last_change":"2024-04-14T08:34:00.005789+0000","last_active":"2024-04-14T08:34:00.009281+0000","last_peered":"2024-04-14T08:34:00.009281+0000","last_clean":"2024-04-14T08:34:00.009281+0000","last_became_active":"2024-04-14T08:34:00.005518+0000","last_became_peered":"2024-04-14T08:34:00.005518+0000","last_unstale":"2024-04-14T08:34:00.009281+0000","last_undegraded":"2024-04-14T08:34:00.009281+0000","last_fullsized":"2024-04-14T08:34:00.009281+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:54.465535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.002033+0000","last_change":"2024-04-14T08:33:59.997834+0000","last_active":"2024-04-14T08:34:00.002033+0000","last_peered":"2024-04-14T08:34:00.002033+0000","last_clean":"2024-04-14T08:34:00.002033+0000","last_became_active":"2024-04-14T08:33:59.997418+0000","last_became_peered":"2024-04-14T08:33:59.997418+0000","last_unstale":"2024-04-14T08:34:00.002033+0000","last_undegraded":"2024-04-14T08:34:00.002033+0000","last_fullsized":"2024-04-14T08:34:00.002033+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:30:21.238910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007719+0000","last_change":"2024-04-14T08:34:00.004809+0000","last_active":"2024-04-14T08:34:00.007719+0000","last_peered":"2024-04-14T08:34:00.007719+0000","last_clean":"2024-04-14T08:34:00.007719+0000","last_became_active":"2024-04-14T08:34:00.004529+0000","last_became_peered":"2024-04-14T08:34:00.004529+0000","last_unstale":"2024-04-14T08:34:00.007719+0000","last_undegraded":"2024-04-14T08:34:00.007719+0000","last_fullsized":"2024-04-14T08:34:00.007719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:03.589235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.012809+0000","last_change":"2024-04-14T08:34:00.011255+0000","last_active":"2024-04-14T08:34:00.012809+0000","last_peered":"2024-04-14T08:34:00.012809+0000","last_clean":"2024-04-14T08:34:00.012809+0000","last_became_active":"2024-04-14T08:34:00.011001+0000","last_became_peered":"2024-04-14T08:34:00.011001+0000","last_unstale":"2024-04-14T08:34:00.012809+0000","last_undegraded":"2024-04-14T08:34:00.012809+0000","last_fullsized":"2024-04-14T08:34:00.012809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:02:58.673573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013037+0000","last_change":"2024-04-14T08:34:00.011994+0000","last_active":"2024-04-14T08:34:00.013037+0000","last_peered":"2024-04-14T08:34:00.013037+0000","last_clean":"2024-04-14T08:34:00.013037+0000","last_became_active":"2024-04-14T08:34:00.011667+0000","last_became_peered":"2024-04-14T08:34:00.011667+0000","last_unstale":"2024-04-14T08:34:00.013037+0000","last_undegraded":"2024-04-14T08:34:00.013037+0000","last_fullsized":"2024-04-14T08:34:00.013037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:00:51.825299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005146+0000","last_change":"2024-04-14T08:34:00.003096+0000","last_active":"2024-04-14T08:34:00.005146+0000","last_peered":"2024-04-14T08:34:00.005146+0000","last_clean":"2024-04-14T08:34:00.005146+0000","last_became_active":"2024-04-14T08:34:00.002694+0000","last_became_peered":"2024-04-14T08:34:00.002694+0000","last_unstale":"2024-04-14T08:34:00.005146+0000","last_undegraded":"2024-04-14T08:34:00.005146+0000","last_fullsized":"2024-04-14T08:34:00.005146+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:26:02.000109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007814+0000","last_change":"2024-04-14T08:34:00.002704+0000","last_active":"2024-04-14T08:34:00.007814+0000","last_peered":"2024-04-14T08:34:00.007814+0000","last_clean":"2024-04-14T08:34:00.007814+0000","last_became_active":"2024-04-14T08:34:00.002113+0000","last_became_peered":"2024-04-14T08:34:00.002113+0000","last_unstale":"2024-04-14T08:34:00.007814+0000","last_undegraded":"2024-04-14T08:34:00.007814+0000","last_fullsized":"2024-04-14T08:34:00.007814+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:40:27.541789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.011040+0000","last_change":"2024-04-14T08:34:00.009380+0000","last_active":"2024-04-14T08:34:00.011040+0000","last_peered":"2024-04-14T08:34:00.011040+0000","last_clean":"2024-04-14T08:34:00.011040+0000","last_became_active":"2024-04-14T08:34:00.009124+0000","last_became_peered":"2024-04-14T08:34:00.009124+0000","last_unstale":"2024-04-14T08:34:00.011040+0000","last_undegraded":"2024-04-14T08:34:00.011040+0000","last_fullsized":"2024-04-14T08:34:00.011040+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:51:42.354241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.010991+0000","last_change":"2024-04-14T08:34:00.006726+0000","last_active":"2024-04-14T08:34:00.010991+0000","last_peered":"2024-04-14T08:34:00.010991+0000","last_clean":"2024-04-14T08:34:00.010991+0000","last_became_active":"2024-04-14T08:34:00.006240+0000","last_became_peered":"2024-04-14T08:34:00.006240+0000","last_unstale":"2024-04-14T08:34:00.010991+0000","last_undegraded":"2024-04-14T08:34:00.010991+0000","last_fullsized":"2024-04-14T08:34:00.010991+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:30:21.238910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007788+0000","last_change":"2024-04-14T08:34:00.003564+0000","last_active":"2024-04-14T08:34:00.007788+0000","last_peered":"2024-04-14T08:34:00.007788+0000","last_clean":"2024-04-14T08:34:00.007788+0000","last_became_active":"2024-04-14T08:34:00.003179+0000","last_became_peered":"2024-04-14T08:34:00.003179+0000","last_unstale":"2024-04-14T08:34:00.007788+0000","last_undegraded":"2024-04-14T08:34:00.007788+0000","last_fullsized":"2024-04-14T08:34:00.007788+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:09:02.249499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.010639+0000","last_change":"2024-04-14T08:34:00.006702+0000","last_active":"2024-04-14T08:34:00.010639+0000","last_peered":"2024-04-14T08:34:00.010639+0000","last_clean":"2024-04-14T08:34:00.010639+0000","last_became_active":"2024-04-14T08:34:00.005642+0000","last_became_peered":"2024-04-14T08:34:00.005642+0000","last_unstale":"2024-04-14T08:34:00.010639+0000","last_undegraded":"2024-04-14T08:34:00.010639+0000","last_fullsized":"2024-04-14T08:34:00.010639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:58:48.218603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.004361+0000","last_change":"2024-04-14T08:33:59.993650+0000","last_active":"2024-04-14T08:34:00.004361+0000","last_peered":"2024-04-14T08:34:00.004361+0000","last_clean":"2024-04-14T08:34:00.004361+0000","last_became_active":"2024-04-14T08:33:59.993388+0000","last_became_peered":"2024-04-14T08:33:59.993388+0000","last_unstale":"2024-04-14T08:34:00.004361+0000","last_undegraded":"2024-04-14T08:34:00.004361+0000","last_fullsized":"2024-04-14T08:34:00.004361+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:16:15.779370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.010589+0000","last_change":"2024-04-14T08:34:00.005066+0000","last_active":"2024-04-14T08:34:00.010589+0000","last_peered":"2024-04-14T08:34:00.010589+0000","last_clean":"2024-04-14T08:34:00.010589+0000","last_became_active":"2024-04-14T08:34:00.004793+0000","last_became_peered":"2024-04-14T08:34:00.004793+0000","last_unstale":"2024-04-14T08:34:00.010589+0000","last_undegraded":"2024-04-14T08:34:00.010589+0000","last_fullsized":"2024-04-14T08:34:00.010589+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:51:42.354241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013283+0000","last_change":"2024-04-14T08:34:00.008628+0000","last_active":"2024-04-14T08:34:00.013283+0000","last_peered":"2024-04-14T08:34:00.013283+0000","last_clean":"2024-04-14T08:34:00.013283+0000","last_became_active":"2024-04-14T08:34:00.008064+0000","last_became_peered":"2024-04-14T08:34:00.008064+0000","last_unstale":"2024-04-14T08:34:00.013283+0000","last_undegraded":"2024-04-14T08:34:00.013283+0000","last_fullsized":"2024-04-14T08:34:00.013283+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:13.330734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007830+0000","last_change":"2024-04-14T08:34:00.002929+0000","last_active":"2024-04-14T08:34:00.007830+0000","last_peered":"2024-04-14T08:34:00.007830+0000","last_clean":"2024-04-14T08:34:00.007830+0000","last_became_active":"2024-04-14T08:34:00.001532+0000","last_became_peered":"2024-04-14T08:34:00.001532+0000","last_unstale":"2024-04-14T08:34:00.007830+0000","last_undegraded":"2024-04-14T08:34:00.007830+0000","last_fullsized":"2024-04-14T08:34:00.007830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:21.676556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989436+0000","last_change":"2024-04-14T08:34:00.007009+0000","last_active":"2024-04-14T08:34:00.989436+0000","last_peered":"2024-04-14T08:34:00.989436+0000","last_clean":"2024-04-14T08:34:00.989436+0000","last_became_active":"2024-04-14T08:34:00.006739+0000","last_became_peered":"2024-04-14T08:34:00.006739+0000","last_unstale":"2024-04-14T08:34:00.989436+0000","last_undegraded":"2024-04-14T08:34:00.989436+0000","last_fullsized":"2024-04-14T08:34:00.989436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:54.465535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989867+0000","last_change":"2024-04-14T08:34:00.007870+0000","last_active":"2024-04-14T08:34:00.989867+0000","last_peered":"2024-04-14T08:34:00.989867+0000","last_clean":"2024-04-14T08:34:00.989867+0000","last_became_active":"2024-04-14T08:34:00.007613+0000","last_became_peered":"2024-04-14T08:34:00.007613+0000","last_unstale":"2024-04-14T08:34:00.989867+0000","last_undegraded":"2024-04-14T08:34:00.989867+0000","last_fullsized":"2024-04-14T08:34:00.989867+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:25:36.443390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.014299+0000","last_change":"2024-04-14T08:34:00.008961+0000","last_active":"2024-04-14T08:34:00.014299+0000","last_peered":"2024-04-14T08:34:00.014299+0000","last_clean":"2024-04-14T08:34:00.014299+0000","last_became_active":"2024-04-14T08:34:00.008625+0000","last_became_peered":"2024-04-14T08:34:00.008625+0000","last_unstale":"2024-04-14T08:34:00.014299+0000","last_undegraded":"2024-04-14T08:34:00.014299+0000","last_fullsized":"2024-04-14T08:34:00.014299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:26:02.000109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.006817+0000","last_change":"2024-04-14T08:33:59.999239+0000","last_active":"2024-04-14T08:34:00.006817+0000","last_peered":"2024-04-14T08:34:00.006817+0000","last_clean":"2024-04-14T08:34:00.006817+0000","last_became_active":"2024-04-14T08:33:59.998980+0000","last_became_peered":"2024-04-14T08:33:59.998980+0000","last_unstale":"2024-04-14T08:34:00.006817+0000","last_undegraded":"2024-04-14T08:34:00.006817+0000","last_fullsized":"2024-04-14T08:34:00.006817+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:26:59.590582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013724+0000","last_change":"2024-04-14T08:34:00.008090+0000","last_active":"2024-04-14T08:34:00.013724+0000","last_peered":"2024-04-14T08:34:00.013724+0000","last_clean":"2024-04-14T08:34:00.013724+0000","last_became_active":"2024-04-14T08:34:00.007784+0000","last_became_peered":"2024-04-14T08:34:00.007784+0000","last_unstale":"2024-04-14T08:34:00.013724+0000","last_undegraded":"2024-04-14T08:34:00.013724+0000","last_fullsized":"2024-04-14T08:34:00.013724+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:20.676280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005352+0000","last_change":"2024-04-14T08:34:00.001292+0000","last_active":"2024-04-14T08:34:00.005352+0000","last_peered":"2024-04-14T08:34:00.005352+0000","last_clean":"2024-04-14T08:34:00.005352+0000","last_became_active":"2024-04-14T08:34:00.001000+0000","last_became_peered":"2024-04-14T08:34:00.001000+0000","last_unstale":"2024-04-14T08:34:00.005352+0000","last_undegraded":"2024-04-14T08:34:00.005352+0000","last_fullsized":"2024-04-14T08:34:00.005352+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:03.589235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013259+0000","last_change":"2024-04-14T08:34:00.007813+0000","last_active":"2024-04-14T08:34:00.013259+0000","last_peered":"2024-04-14T08:34:00.013259+0000","last_clean":"2024-04-14T08:34:00.013259+0000","last_became_active":"2024-04-14T08:34:00.005125+0000","last_became_peered":"2024-04-14T08:34:00.005125+0000","last_unstale":"2024-04-14T08:34:00.013259+0000","last_undegraded":"2024-04-14T08:34:00.013259+0000","last_fullsized":"2024-04-14T08:34:00.013259+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:26:02.000109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989572+0000","last_change":"2024-04-14T08:34:00.003880+0000","last_active":"2024-04-14T08:34:00.989572+0000","last_peered":"2024-04-14T08:34:00.989572+0000","last_clean":"2024-04-14T08:34:00.989572+0000","last_became_active":"2024-04-14T08:34:00.003595+0000","last_became_peered":"2024-04-14T08:34:00.003595+0000","last_unstale":"2024-04-14T08:34:00.989572+0000","last_undegraded":"2024-04-14T08:34:00.989572+0000","last_fullsized":"2024-04-14T08:34:00.989572+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:51:42.354241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.002234+0000","last_change":"2024-04-14T08:33:59.995880+0000","last_active":"2024-04-14T08:34:00.002234+0000","last_peered":"2024-04-14T08:34:00.002234+0000","last_clean":"2024-04-14T08:34:00.002234+0000","last_became_active":"2024-04-14T08:33:59.995604+0000","last_became_peered":"2024-04-14T08:33:59.995604+0000","last_unstale":"2024-04-14T08:34:00.002234+0000","last_undegraded":"2024-04-14T08:34:00.002234+0000","last_fullsized":"2024-04-14T08:34:00.002234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:19:10.169283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.014261+0000","last_change":"2024-04-14T08:34:00.007911+0000","last_active":"2024-04-14T08:34:00.014261+0000","last_peered":"2024-04-14T08:34:00.014261+0000","last_clean":"2024-04-14T08:34:00.014261+0000","last_became_active":"2024-04-14T08:34:00.007635+0000","last_became_peered":"2024-04-14T08:34:00.007635+0000","last_unstale":"2024-04-14T08:34:00.014261+0000","last_undegraded":"2024-04-14T08:34:00.014261+0000","last_fullsized":"2024-04-14T08:34:00.014261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:16:15.779370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.008739+0000","last_change":"2024-04-14T08:34:00.006507+0000","last_active":"2024-04-14T08:34:00.008739+0000","last_peered":"2024-04-14T08:34:00.008739+0000","last_clean":"2024-04-14T08:34:00.008739+0000","last_became_active":"2024-04-14T08:34:00.006246+0000","last_became_peered":"2024-04-14T08:34:00.006246+0000","last_unstale":"2024-04-14T08:34:00.008739+0000","last_undegraded":"2024-04-14T08:34:00.008739+0000","last_fullsized":"2024-04-14T08:34:00.008739+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:03.589235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007423+0000","last_change":"2024-04-14T08:34:00.003623+0000","last_active":"2024-04-14T08:34:00.007423+0000","last_peered":"2024-04-14T08:34:00.007423+0000","last_clean":"2024-04-14T08:34:00.007423+0000","last_became_active":"2024-04-14T08:34:00.003330+0000","last_became_peered":"2024-04-14T08:34:00.003330+0000","last_unstale":"2024-04-14T08:34:00.007423+0000","last_undegraded":"2024-04-14T08:34:00.007423+0000","last_fullsized":"2024-04-14T08:34:00.007423+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:00:20.760892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007757+0000","last_change":"2024-04-14T08:34:00.002792+0000","last_active":"2024-04-14T08:34:00.007757+0000","last_peered":"2024-04-14T08:34:00.007757+0000","last_clean":"2024-04-14T08:34:00.007757+0000","last_became_active":"2024-04-14T08:34:00.001687+0000","last_became_peered":"2024-04-14T08:34:00.001687+0000","last_unstale":"2024-04-14T08:34:00.007757+0000","last_undegraded":"2024-04-14T08:34:00.007757+0000","last_fullsized":"2024-04-14T08:34:00.007757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:52.044408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007027+0000","last_change":"2024-04-14T08:34:00.002146+0000","last_active":"2024-04-14T08:34:00.007027+0000","last_peered":"2024-04-14T08:34:00.007027+0000","last_clean":"2024-04-14T08:34:00.007027+0000","last_became_active":"2024-04-14T08:34:00.001829+0000","last_became_peered":"2024-04-14T08:34:00.001829+0000","last_unstale":"2024-04-14T08:34:00.007027+0000","last_undegraded":"2024-04-14T08:34:00.007027+0000","last_fullsized":"2024-04-14T08:34:00.007027+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:01:07.798205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.011471+0000","last_change":"2024-04-14T08:34:00.005728+0000","last_active":"2024-04-14T08:34:00.011471+0000","last_peered":"2024-04-14T08:34:00.011471+0000","last_clean":"2024-04-14T08:34:00.011471+0000","last_became_active":"2024-04-14T08:34:00.005463+0000","last_became_peered":"2024-04-14T08:34:00.005463+0000","last_unstale":"2024-04-14T08:34:00.011471+0000","last_undegraded":"2024-04-14T08:34:00.011471+0000","last_fullsized":"2024-04-14T08:34:00.011471+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:42.487210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989909+0000","last_change":"2024-04-14T08:34:00.004218+0000","last_active":"2024-04-14T08:34:00.989909+0000","last_peered":"2024-04-14T08:34:00.989909+0000","last_clean":"2024-04-14T08:34:00.989909+0000","last_became_active":"2024-04-14T08:34:00.003604+0000","last_became_peered":"2024-04-14T08:34:00.003604+0000","last_unstale":"2024-04-14T08:34:00.989909+0000","last_undegraded":"2024-04-14T08:34:00.989909+0000","last_fullsized":"2024-04-14T08:34:00.989909+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:30:14.829850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.002999+0000","last_change":"2024-04-14T08:34:00.000949+0000","last_active":"2024-04-14T08:34:00.002999+0000","last_peered":"2024-04-14T08:34:00.002999+0000","last_clean":"2024-04-14T08:34:00.002999+0000","last_became_active":"2024-04-14T08:34:00.000684+0000","last_became_peered":"2024-04-14T08:34:00.000684+0000","last_unstale":"2024-04-14T08:34:00.002999+0000","last_undegraded":"2024-04-14T08:34:00.002999+0000","last_fullsized":"2024-04-14T08:34:00.002999+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:20.676280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989967+0000","last_change":"2024-04-14T08:34:00.009956+0000","last_active":"2024-04-14T08:34:00.989967+0000","last_peered":"2024-04-14T08:34:00.989967+0000","last_clean":"2024-04-14T08:34:00.989967+0000","last_became_active":"2024-04-14T08:34:00.009674+0000","last_became_peered":"2024-04-14T08:34:00.009674+0000","last_unstale":"2024-04-14T08:34:00.989967+0000","last_undegraded":"2024-04-14T08:34:00.989967+0000","last_fullsized":"2024-04-14T08:34:00.989967+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:18:37.055661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.017510+0000","last_change":"2024-04-14T08:33:59.008983+0000","last_active":"2024-04-14T08:34:00.017510+0000","last_peered":"2024-04-14T08:34:00.017510+0000","last_clean":"2024-04-14T08:34:00.017510+0000","last_became_active":"2024-04-14T08:33:59.008686+0000","last_became_peered":"2024-04-14T08:33:59.008686+0000","last_unstale":"2024-04-14T08:34:00.017510+0000","last_undegraded":"2024-04-14T08:34:00.017510+0000","last_fullsized":"2024-04-14T08:34:00.017510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:51:58.159319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.008741+0000","last_change":"2024-04-14T08:34:00.005932+0000","last_active":"2024-04-14T08:34:00.008741+0000","last_peered":"2024-04-14T08:34:00.008741+0000","last_clean":"2024-04-14T08:34:00.008741+0000","last_became_active":"2024-04-14T08:34:00.005645+0000","last_became_peered":"2024-04-14T08:34:00.005645+0000","last_unstale":"2024-04-14T08:34:00.008741+0000","last_undegraded":"2024-04-14T08:34:00.008741+0000","last_fullsized":"2024-04-14T08:34:00.008741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:46:26.546283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.015765+0000","last_change":"2024-04-14T08:33:59.005808+0000","last_active":"2024-04-14T08:34:00.015765+0000","last_peered":"2024-04-14T08:34:00.015765+0000","last_clean":"2024-04-14T08:34:00.015765+0000","last_became_active":"2024-04-14T08:33:59.005498+0000","last_became_peered":"2024-04-14T08:33:59.005498+0000","last_unstale":"2024-04-14T08:34:00.015765+0000","last_undegraded":"2024-04-14T08:34:00.015765+0000","last_fullsized":"2024-04-14T08:34:00.015765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:00:19.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.014832+0000","last_change":"2024-04-14T08:34:00.006051+0000","last_active":"2024-04-14T08:34:00.014832+0000","last_peered":"2024-04-14T08:34:00.014832+0000","last_clean":"2024-04-14T08:34:00.014832+0000","last_became_active":"2024-04-14T08:34:00.005729+0000","last_became_peered":"2024-04-14T08:34:00.005729+0000","last_unstale":"2024-04-14T08:34:00.014832+0000","last_undegraded":"2024-04-14T08:34:00.014832+0000","last_fullsized":"2024-04-14T08:34:00.014832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:20.676280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.016764+0000","last_change":"2024-04-14T08:33:59.009159+0000","last_active":"2024-04-14T08:34:00.016764+0000","last_peered":"2024-04-14T08:34:00.016764+0000","last_clean":"2024-04-14T08:34:00.016764+0000","last_became_active":"2024-04-14T08:33:59.008864+0000","last_became_peered":"2024-04-14T08:33:59.008864+0000","last_unstale":"2024-04-14T08:34:00.016764+0000","last_undegraded":"2024-04-14T08:34:00.016764+0000","last_fullsized":"2024-04-14T08:34:00.016764+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:16:14.774984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007725+0000","last_change":"2024-04-14T08:34:00.004505+0000","last_active":"2024-04-14T08:34:00.007725+0000","last_peered":"2024-04-14T08:34:00.007725+0000","last_clean":"2024-04-14T08:34:00.007725+0000","last_became_active":"2024-04-14T08:34:00.004249+0000","last_became_peered":"2024-04-14T08:34:00.004249+0000","last_unstale":"2024-04-14T08:34:00.007725+0000","last_undegraded":"2024-04-14T08:34:00.007725+0000","last_fullsized":"2024-04-14T08:34:00.007725+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:31.409872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.017223+0000","last_change":"2024-04-14T08:33:59.009688+0000","last_active":"2024-04-14T08:34:00.017223+0000","last_peered":"2024-04-14T08:34:00.017223+0000","last_clean":"2024-04-14T08:34:00.017223+0000","last_became_active":"2024-04-14T08:33:59.009321+0000","last_became_peered":"2024-04-14T08:33:59.009321+0000","last_unstale":"2024-04-14T08:34:00.017223+0000","last_undegraded":"2024-04-14T08:34:00.017223+0000","last_fullsized":"2024-04-14T08:34:00.017223+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:01:06.793819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005797+0000","last_change":"2024-04-14T08:33:59.996429+0000","last_active":"2024-04-14T08:34:00.005797+0000","last_peered":"2024-04-14T08:34:00.005797+0000","last_clean":"2024-04-14T08:34:00.005797+0000","last_became_active":"2024-04-14T08:33:59.996144+0000","last_became_peered":"2024-04-14T08:33:59.996144+0000","last_unstale":"2024-04-14T08:34:00.005797+0000","last_undegraded":"2024-04-14T08:34:00.005797+0000","last_fullsized":"2024-04-14T08:34:00.005797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:17.444588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.019207+0000","last_change":"2024-04-14T08:33:59.005231+0000","last_active":"2024-04-14T08:34:00.019207+0000","last_peered":"2024-04-14T08:34:00.019207+0000","last_clean":"2024-04-14T08:34:00.019207+0000","last_became_active":"2024-04-14T08:33:59.004517+0000","last_became_peered":"2024-04-14T08:33:59.004517+0000","last_unstale":"2024-04-14T08:34:00.019207+0000","last_undegraded":"2024-04-14T08:34:00.019207+0000","last_fullsized":"2024-04-14T08:34:00.019207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:26:00.995723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007792+0000","last_change":"2024-04-14T08:34:00.002362+0000","last_active":"2024-04-14T08:34:00.007792+0000","last_peered":"2024-04-14T08:34:00.007792+0000","last_clean":"2024-04-14T08:34:00.007792+0000","last_became_active":"2024-04-14T08:34:00.001232+0000","last_became_peered":"2024-04-14T08:34:00.001232+0000","last_unstale":"2024-04-14T08:34:00.007792+0000","last_undegraded":"2024-04-14T08:34:00.007792+0000","last_fullsized":"2024-04-14T08:34:00.007792+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:30:21.238910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989551+0000","last_change":"2024-04-14T08:33:59.009208+0000","last_active":"2024-04-14T08:34:00.989551+0000","last_peered":"2024-04-14T08:34:00.989551+0000","last_clean":"2024-04-14T08:34:00.989551+0000","last_became_active":"2024-04-14T08:33:59.008945+0000","last_became_peered":"2024-04-14T08:33:59.008945+0000","last_unstale":"2024-04-14T08:34:00.989551+0000","last_undegraded":"2024-04-14T08:34:00.989551+0000","last_fullsized":"2024-04-14T08:34:00.989551+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:16:14.774984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013358+0000","last_change":"2024-04-14T08:34:00.010101+0000","last_active":"2024-04-14T08:34:00.013358+0000","last_peered":"2024-04-14T08:34:00.013358+0000","last_clean":"2024-04-14T08:34:00.013358+0000","last_became_active":"2024-04-14T08:34:00.006751+0000","last_became_peered":"2024-04-14T08:34:00.006751+0000","last_unstale":"2024-04-14T08:34:00.013358+0000","last_undegraded":"2024-04-14T08:34:00.013358+0000","last_fullsized":"2024-04-14T08:34:00.013358+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:03.589235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.993637+0000","last_change":"2024-04-14T08:33:59.005088+0000","last_active":"2024-04-14T08:33:59.993637+0000","last_peered":"2024-04-14T08:33:59.993637+0000","last_clean":"2024-04-14T08:33:59.993637+0000","last_became_active":"2024-04-14T08:33:59.004802+0000","last_became_peered":"2024-04-14T08:33:59.004802+0000","last_unstale":"2024-04-14T08:33:59.993637+0000","last_undegraded":"2024-04-14T08:33:59.993637+0000","last_fullsized":"2024-04-14T08:33:59.993637+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:41.482824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.012014+0000","last_change":"2024-04-14T08:34:00.010515+0000","last_active":"2024-04-14T08:34:00.012014+0000","last_peered":"2024-04-14T08:34:00.012014+0000","last_clean":"2024-04-14T08:34:00.012014+0000","last_became_active":"2024-04-14T08:34:00.010270+0000","last_became_peered":"2024-04-14T08:34:00.010270+0000","last_unstale":"2024-04-14T08:34:00.012014+0000","last_undegraded":"2024-04-14T08:34:00.012014+0000","last_fullsized":"2024-04-14T08:34:00.012014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:17:40.287151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.996028+0000","last_change":"2024-04-14T08:33:59.009167+0000","last_active":"2024-04-14T08:33:59.996028+0000","last_peered":"2024-04-14T08:33:59.996028+0000","last_clean":"2024-04-14T08:33:59.996028+0000","last_became_active":"2024-04-14T08:33:59.008423+0000","last_became_peered":"2024-04-14T08:33:59.008423+0000","last_unstale":"2024-04-14T08:33:59.996028+0000","last_undegraded":"2024-04-14T08:33:59.996028+0000","last_fullsized":"2024-04-14T08:33:59.996028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:51:58.159319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.006940+0000","last_change":"2024-04-14T08:34:00.003551+0000","last_active":"2024-04-14T08:34:00.006940+0000","last_peered":"2024-04-14T08:34:00.006940+0000","last_clean":"2024-04-14T08:34:00.006940+0000","last_became_active":"2024-04-14T08:34:00.002714+0000","last_became_peered":"2024-04-14T08:34:00.002714+0000","last_unstale":"2024-04-14T08:34:00.006940+0000","last_undegraded":"2024-04-14T08:34:00.006940+0000","last_fullsized":"2024-04-14T08:34:00.006940+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:27:54.003457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.991943+0000","last_change":"2024-04-14T08:33:59.001289+0000","last_active":"2024-04-14T08:33:59.991943+0000","last_peered":"2024-04-14T08:33:59.991943+0000","last_clean":"2024-04-14T08:33:59.991943+0000","last_became_active":"2024-04-14T08:33:59.001011+0000","last_became_peered":"2024-04-14T08:33:59.001011+0000","last_unstale":"2024-04-14T08:33:59.991943+0000","last_undegraded":"2024-04-14T08:33:59.991943+0000","last_fullsized":"2024-04-14T08:33:59.991943+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:12.326349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989769+0000","last_change":"2024-04-14T08:34:00.006194+0000","last_active":"2024-04-14T08:34:00.989769+0000","last_peered":"2024-04-14T08:34:00.989769+0000","last_clean":"2024-04-14T08:34:00.989769+0000","last_became_active":"2024-04-14T08:34:00.005606+0000","last_became_peered":"2024-04-14T08:34:00.005606+0000","last_unstale":"2024-04-14T08:34:00.989769+0000","last_undegraded":"2024-04-14T08:34:00.989769+0000","last_fullsized":"2024-04-14T08:34:00.989769+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:17:40.287151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.991364+0000","last_change":"2024-04-14T08:33:58.997424+0000","last_active":"2024-04-14T08:33:59.991364+0000","last_peered":"2024-04-14T08:33:59.991364+0000","last_clean":"2024-04-14T08:33:59.991364+0000","last_became_active":"2024-04-14T08:33:58.997087+0000","last_became_peered":"2024-04-14T08:33:58.997087+0000","last_unstale":"2024-04-14T08:33:59.991364+0000","last_undegraded":"2024-04-14T08:33:59.991364+0000","last_fullsized":"2024-04-14T08:33:59.991364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:19.671894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005025+0000","last_change":"2024-04-14T08:33:59.995244+0000","last_active":"2024-04-14T08:34:00.005025+0000","last_peered":"2024-04-14T08:34:00.005025+0000","last_clean":"2024-04-14T08:34:00.005025+0000","last_became_active":"2024-04-14T08:33:59.994422+0000","last_became_peered":"2024-04-14T08:33:59.994422+0000","last_unstale":"2024-04-14T08:34:00.005025+0000","last_undegraded":"2024-04-14T08:34:00.005025+0000","last_fullsized":"2024-04-14T08:34:00.005025+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:26:02.000109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.019538+0000","last_change":"2024-04-14T08:33:59.001631+0000","last_active":"2024-04-14T08:34:00.019538+0000","last_peered":"2024-04-14T08:34:00.019538+0000","last_clean":"2024-04-14T08:34:00.019538+0000","last_became_active":"2024-04-14T08:33:59.001173+0000","last_became_peered":"2024-04-14T08:33:59.001173+0000","last_unstale":"2024-04-14T08:34:00.019538+0000","last_undegraded":"2024-04-14T08:34:00.019538+0000","last_fullsized":"2024-04-14T08:34:00.019538+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:16:14.774984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013238+0000","last_change":"2024-04-14T08:34:00.004593+0000","last_active":"2024-04-14T08:34:00.013238+0000","last_peered":"2024-04-14T08:34:00.013238+0000","last_clean":"2024-04-14T08:34:00.013238+0000","last_became_active":"2024-04-14T08:34:00.004312+0000","last_became_peered":"2024-04-14T08:34:00.004312+0000","last_unstale":"2024-04-14T08:34:00.013238+0000","last_undegraded":"2024-04-14T08:34:00.013238+0000","last_fullsized":"2024-04-14T08:34:00.013238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:16:15.779370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.024437+0000","last_change":"2024-04-14T08:33:59.002849+0000","last_active":"2024-04-14T08:34:00.024437+0000","last_peered":"2024-04-14T08:34:00.024437+0000","last_clean":"2024-04-14T08:34:00.024437+0000","last_became_active":"2024-04-14T08:33:59.002544+0000","last_became_peered":"2024-04-14T08:33:59.002544+0000","last_unstale":"2024-04-14T08:34:00.024437+0000","last_undegraded":"2024-04-14T08:34:00.024437+0000","last_fullsized":"2024-04-14T08:34:00.024437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:46:25.541897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.010976+0000","last_change":"2024-04-14T08:34:00.005921+0000","last_active":"2024-04-14T08:34:00.010976+0000","last_peered":"2024-04-14T08:34:00.010976+0000","last_clean":"2024-04-14T08:34:00.010976+0000","last_became_active":"2024-04-14T08:34:00.005602+0000","last_became_peered":"2024-04-14T08:34:00.005602+0000","last_unstale":"2024-04-14T08:34:00.010976+0000","last_undegraded":"2024-04-14T08:34:00.010976+0000","last_fullsized":"2024-04-14T08:34:00.010976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:46:26.546283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.015832+0000","last_change":"2024-04-14T08:33:59.004932+0000","last_active":"2024-04-14T08:34:00.015832+0000","last_peered":"2024-04-14T08:34:00.015832+0000","last_clean":"2024-04-14T08:34:00.015832+0000","last_became_active":"2024-04-14T08:33:59.004618+0000","last_became_peered":"2024-04-14T08:33:59.004618+0000","last_unstale":"2024-04-14T08:34:00.015832+0000","last_undegraded":"2024-04-14T08:34:00.015832+0000","last_fullsized":"2024-04-14T08:34:00.015832+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:16.440202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.002086+0000","last_change":"2024-04-14T08:33:59.999556+0000","last_active":"2024-04-14T08:34:00.002086+0000","last_peered":"2024-04-14T08:34:00.002086+0000","last_clean":"2024-04-14T08:34:00.002086+0000","last_became_active":"2024-04-14T08:33:59.999303+0000","last_became_peered":"2024-04-14T08:33:59.999303+0000","last_unstale":"2024-04-14T08:34:00.002086+0000","last_undegraded":"2024-04-14T08:34:00.002086+0000","last_fullsized":"2024-04-14T08:34:00.002086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:40:27.541789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.993520+0000","last_change":"2024-04-14T08:33:59.006438+0000","last_active":"2024-04-14T08:33:59.993520+0000","last_peered":"2024-04-14T08:33:59.993520+0000","last_clean":"2024-04-14T08:33:59.993520+0000","last_became_active":"2024-04-14T08:33:59.006251+0000","last_became_peered":"2024-04-14T08:33:59.006251+0000","last_unstale":"2024-04-14T08:33:59.993520+0000","last_undegraded":"2024-04-14T08:33:59.993520+0000","last_fullsized":"2024-04-14T08:33:59.993520+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:00:19.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989742+0000","last_change":"2024-04-14T08:34:00.004016+0000","last_active":"2024-04-14T08:34:00.989742+0000","last_peered":"2024-04-14T08:34:00.989742+0000","last_clean":"2024-04-14T08:34:00.989742+0000","last_became_active":"2024-04-14T08:34:00.003660+0000","last_became_peered":"2024-04-14T08:34:00.003660+0000","last_unstale":"2024-04-14T08:34:00.989742+0000","last_undegraded":"2024-04-14T08:34:00.989742+0000","last_fullsized":"2024-04-14T08:34:00.989742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:58:48.218603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.994864+0000","last_change":"2024-04-14T08:33:59.009109+0000","last_active":"2024-04-14T08:33:59.994864+0000","last_peered":"2024-04-14T08:33:59.994864+0000","last_clean":"2024-04-14T08:33:59.994864+0000","last_became_active":"2024-04-14T08:33:59.008773+0000","last_became_peered":"2024-04-14T08:33:59.008773+0000","last_unstale":"2024-04-14T08:33:59.994864+0000","last_undegraded":"2024-04-14T08:33:59.994864+0000","last_fullsized":"2024-04-14T08:33:59.994864+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:16.364611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.003541+0000","last_change":"2024-04-14T08:34:00.000887+0000","last_active":"2024-04-14T08:34:00.003541+0000","last_peered":"2024-04-14T08:34:00.003541+0000","last_clean":"2024-04-14T08:34:00.003541+0000","last_became_active":"2024-04-14T08:34:00.000624+0000","last_became_peered":"2024-04-14T08:34:00.000624+0000","last_unstale":"2024-04-14T08:34:00.003541+0000","last_undegraded":"2024-04-14T08:34:00.003541+0000","last_fullsized":"2024-04-14T08:34:00.003541+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:52.044408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989440+0000","last_change":"2024-04-14T08:33:59.010698+0000","last_active":"2024-04-14T08:34:00.989440+0000","last_peered":"2024-04-14T08:34:00.989440+0000","last_clean":"2024-04-14T08:34:00.989440+0000","last_became_active":"2024-04-14T08:33:59.010425+0000","last_became_peered":"2024-04-14T08:33:59.010425+0000","last_unstale":"2024-04-14T08:34:00.989440+0000","last_undegraded":"2024-04-14T08:34:00.989440+0000","last_fullsized":"2024-04-14T08:34:00.989440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:41.482824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.009302+0000","last_change":"2024-04-14T08:34:00.005246+0000","last_active":"2024-04-14T08:34:00.009302+0000","last_peered":"2024-04-14T08:34:00.009302+0000","last_clean":"2024-04-14T08:34:00.009302+0000","last_became_active":"2024-04-14T08:34:00.004868+0000","last_became_peered":"2024-04-14T08:34:00.004868+0000","last_unstale":"2024-04-14T08:34:00.009302+0000","last_undegraded":"2024-04-14T08:34:00.009302+0000","last_fullsized":"2024-04-14T08:34:00.009302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:09:02.249499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.019305+0000","last_change":"2024-04-14T08:33:59.010220+0000","last_active":"2024-04-14T08:34:00.019305+0000","last_peered":"2024-04-14T08:34:00.019305+0000","last_clean":"2024-04-14T08:34:00.019305+0000","last_became_active":"2024-04-14T08:33:59.009884+0000","last_became_peered":"2024-04-14T08:33:59.009884+0000","last_unstale":"2024-04-14T08:34:00.019305+0000","last_undegraded":"2024-04-14T08:34:00.019305+0000","last_fullsized":"2024-04-14T08:34:00.019305+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:12.326349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005372+0000","last_change":"2024-04-14T08:34:00.001352+0000","last_active":"2024-04-14T08:34:00.005372+0000","last_peered":"2024-04-14T08:34:00.005372+0000","last_clean":"2024-04-14T08:34:00.005372+0000","last_became_active":"2024-04-14T08:34:00.001024+0000","last_became_peered":"2024-04-14T08:34:00.001024+0000","last_unstale":"2024-04-14T08:34:00.005372+0000","last_undegraded":"2024-04-14T08:34:00.005372+0000","last_fullsized":"2024-04-14T08:34:00.005372+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:46:26.546283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.019590+0000","last_change":"2024-04-14T08:33:59.006429+0000","last_active":"2024-04-14T08:34:00.019590+0000","last_peered":"2024-04-14T08:34:00.019590+0000","last_clean":"2024-04-14T08:34:00.019590+0000","last_became_active":"2024-04-14T08:33:59.006136+0000","last_became_peered":"2024-04-14T08:33:59.006136+0000","last_unstale":"2024-04-14T08:34:00.019590+0000","last_undegraded":"2024-04-14T08:34:00.019590+0000","last_fullsized":"2024-04-14T08:34:00.019590+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:00:19.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.011329+0000","last_change":"2024-04-14T08:34:00.002231+0000","last_active":"2024-04-14T08:34:00.011329+0000","last_peered":"2024-04-14T08:34:00.011329+0000","last_clean":"2024-04-14T08:34:00.011329+0000","last_became_active":"2024-04-14T08:34:00.001970+0000","last_became_peered":"2024-04-14T08:34:00.001970+0000","last_unstale":"2024-04-14T08:34:00.011329+0000","last_undegraded":"2024-04-14T08:34:00.011329+0000","last_fullsized":"2024-04-14T08:34:00.011329+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:42.487210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.012960+0000","last_change":"2024-04-14T08:33:59.002800+0000","last_active":"2024-04-14T08:34:00.012960+0000","last_peered":"2024-04-14T08:34:00.012960+0000","last_clean":"2024-04-14T08:34:00.012960+0000","last_became_active":"2024-04-14T08:33:59.002508+0000","last_became_peered":"2024-04-14T08:33:59.002508+0000","last_unstale":"2024-04-14T08:34:00.012960+0000","last_undegraded":"2024-04-14T08:34:00.012960+0000","last_fullsized":"2024-04-14T08:34:00.012960+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:00:46.198672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.002066+0000","last_change":"2024-04-14T08:33:59.995510+0000","last_active":"2024-04-14T08:34:00.002066+0000","last_peered":"2024-04-14T08:34:00.002066+0000","last_clean":"2024-04-14T08:34:00.002066+0000","last_became_active":"2024-04-14T08:33:59.994780+0000","last_became_peered":"2024-04-14T08:33:59.994780+0000","last_unstale":"2024-04-14T08:34:00.002066+0000","last_undegraded":"2024-04-14T08:34:00.002066+0000","last_fullsized":"2024-04-14T08:34:00.002066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:48:41.632164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.993255+0000","last_change":"2024-04-14T08:33:59.009221+0000","last_active":"2024-04-14T08:33:59.993255+0000","last_peered":"2024-04-14T08:33:59.993255+0000","last_clean":"2024-04-14T08:33:59.993255+0000","last_became_active":"2024-04-14T08:33:59.008623+0000","last_became_peered":"2024-04-14T08:33:59.008623+0000","last_unstale":"2024-04-14T08:33:59.993255+0000","last_undegraded":"2024-04-14T08:33:59.993255+0000","last_fullsized":"2024-04-14T08:33:59.993255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:01:06.793819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989891+0000","last_change":"2024-04-14T08:34:00.004624+0000","last_active":"2024-04-14T08:34:00.989891+0000","last_peered":"2024-04-14T08:34:00.989891+0000","last_clean":"2024-04-14T08:34:00.989891+0000","last_became_active":"2024-04-14T08:34:00.004365+0000","last_became_peered":"2024-04-14T08:34:00.004365+0000","last_unstale":"2024-04-14T08:34:00.989891+0000","last_undegraded":"2024-04-14T08:34:00.989891+0000","last_fullsized":"2024-04-14T08:34:00.989891+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:32:22.941044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013656+0000","last_change":"2024-04-14T08:33:59.004301+0000","last_active":"2024-04-14T08:34:00.013656+0000","last_peered":"2024-04-14T08:34:00.013656+0000","last_clean":"2024-04-14T08:34:00.013656+0000","last_became_active":"2024-04-14T08:33:59.004035+0000","last_became_peered":"2024-04-14T08:33:59.004035+0000","last_unstale":"2024-04-14T08:34:00.013656+0000","last_undegraded":"2024-04-14T08:34:00.013656+0000","last_fullsized":"2024-04-14T08:34:00.013656+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:01:43.795521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.004497+0000","last_change":"2024-04-14T08:33:59.994730+0000","last_active":"2024-04-14T08:34:00.004497+0000","last_peered":"2024-04-14T08:34:00.004497+0000","last_clean":"2024-04-14T08:34:00.004497+0000","last_became_active":"2024-04-14T08:33:59.994468+0000","last_became_peered":"2024-04-14T08:33:59.994468+0000","last_unstale":"2024-04-14T08:34:00.004497+0000","last_undegraded":"2024-04-14T08:34:00.004497+0000","last_fullsized":"2024-04-14T08:34:00.004497+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:20.676280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.986039+0000","last_change":"2024-04-14T08:33:59.008949+0000","last_active":"2024-04-14T08:33:59.986039+0000","last_peered":"2024-04-14T08:33:59.986039+0000","last_clean":"2024-04-14T08:33:59.986039+0000","last_became_active":"2024-04-14T08:33:59.008663+0000","last_became_peered":"2024-04-14T08:33:59.008663+0000","last_unstale":"2024-04-14T08:33:59.986039+0000","last_undegraded":"2024-04-14T08:33:59.986039+0000","last_fullsized":"2024-04-14T08:33:59.986039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:55:08.918254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007793+0000","last_change":"2024-04-14T08:34:00.003470+0000","last_active":"2024-04-14T08:34:00.007793+0000","last_peered":"2024-04-14T08:34:00.007793+0000","last_clean":"2024-04-14T08:34:00.007793+0000","last_became_active":"2024-04-14T08:34:00.003182+0000","last_became_peered":"2024-04-14T08:34:00.003182+0000","last_unstale":"2024-04-14T08:34:00.007793+0000","last_undegraded":"2024-04-14T08:34:00.007793+0000","last_fullsized":"2024-04-14T08:34:00.007793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:31.409872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989717+0000","last_change":"2024-04-14T08:33:59.003597+0000","last_active":"2024-04-14T08:34:00.989717+0000","last_peered":"2024-04-14T08:34:00.989717+0000","last_clean":"2024-04-14T08:34:00.989717+0000","last_became_active":"2024-04-14T08:33:59.003283+0000","last_became_peered":"2024-04-14T08:33:59.003283+0000","last_unstale":"2024-04-14T08:34:00.989717+0000","last_undegraded":"2024-04-14T08:34:00.989717+0000","last_fullsized":"2024-04-14T08:34:00.989717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:19.671894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005304+0000","last_change":"2024-04-14T08:34:00.001744+0000","last_active":"2024-04-14T08:34:00.005304+0000","last_peered":"2024-04-14T08:34:00.005304+0000","last_clean":"2024-04-14T08:34:00.005304+0000","last_became_active":"2024-04-14T08:34:00.001348+0000","last_became_peered":"2024-04-14T08:34:00.001348+0000","last_unstale":"2024-04-14T08:34:00.005304+0000","last_undegraded":"2024-04-14T08:34:00.005304+0000","last_fullsized":"2024-04-14T08:34:00.005304+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:21.676556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.986036+0000","last_change":"2024-04-14T08:33:58.999302+0000","last_active":"2024-04-14T08:33:59.986036+0000","last_peered":"2024-04-14T08:33:59.986036+0000","last_clean":"2024-04-14T08:33:59.986036+0000","last_became_active":"2024-04-14T08:33:58.999046+0000","last_became_peered":"2024-04-14T08:33:58.999046+0000","last_unstale":"2024-04-14T08:33:59.986036+0000","last_undegraded":"2024-04-14T08:33:59.986036+0000","last_fullsized":"2024-04-14T08:33:59.986036+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:23:43.943029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.008704+0000","last_change":"2024-04-14T08:34:00.005938+0000","last_active":"2024-04-14T08:34:00.008704+0000","last_peered":"2024-04-14T08:34:00.008704+0000","last_clean":"2024-04-14T08:34:00.008704+0000","last_became_active":"2024-04-14T08:34:00.005662+0000","last_became_peered":"2024-04-14T08:34:00.005662+0000","last_unstale":"2024-04-14T08:34:00.008704+0000","last_undegraded":"2024-04-14T08:34:00.008704+0000","last_fullsized":"2024-04-14T08:34:00.008704+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:17.444588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.019390+0000","last_change":"2024-04-14T08:33:59.001989+0000","last_active":"2024-04-14T08:34:00.019390+0000","last_peered":"2024-04-14T08:34:00.019390+0000","last_clean":"2024-04-14T08:34:00.019390+0000","last_became_active":"2024-04-14T08:33:59.001644+0000","last_became_peered":"2024-04-14T08:33:59.001644+0000","last_unstale":"2024-04-14T08:34:00.019390+0000","last_undegraded":"2024-04-14T08:34:00.019390+0000","last_fullsized":"2024-04-14T08:34:00.019390+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:16.440202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989995+0000","last_change":"2024-04-14T08:34:00.006416+0000","last_active":"2024-04-14T08:34:00.989995+0000","last_peered":"2024-04-14T08:34:00.989995+0000","last_clean":"2024-04-14T08:34:00.989995+0000","last_became_active":"2024-04-14T08:34:00.005944+0000","last_became_peered":"2024-04-14T08:34:00.005944+0000","last_unstale":"2024-04-14T08:34:00.989995+0000","last_undegraded":"2024-04-14T08:34:00.989995+0000","last_fullsized":"2024-04-14T08:34:00.989995+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:26:17.966190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.986249+0000","last_change":"2024-04-14T08:33:58.998567+0000","last_active":"2024-04-14T08:33:59.986249+0000","last_peered":"2024-04-14T08:33:59.986249+0000","last_clean":"2024-04-14T08:33:59.986249+0000","last_became_active":"2024-04-14T08:33:58.998302+0000","last_became_peered":"2024-04-14T08:33:58.998302+0000","last_unstale":"2024-04-14T08:33:59.986249+0000","last_undegraded":"2024-04-14T08:33:59.986249+0000","last_fullsized":"2024-04-14T08:33:59.986249+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:27:19.258542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.011061+0000","last_change":"2024-04-14T08:34:00.009623+0000","last_active":"2024-04-14T08:34:00.011061+0000","last_peered":"2024-04-14T08:34:00.011061+0000","last_clean":"2024-04-14T08:34:00.011061+0000","last_became_active":"2024-04-14T08:34:00.007666+0000","last_became_peered":"2024-04-14T08:34:00.007666+0000","last_unstale":"2024-04-14T08:34:00.011061+0000","last_undegraded":"2024-04-14T08:34:00.011061+0000","last_fullsized":"2024-04-14T08:34:00.011061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:58:48.218603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.989122+0000","last_change":"2024-04-14T08:33:59.001386+0000","last_active":"2024-04-14T08:33:59.989122+0000","last_peered":"2024-04-14T08:33:59.989122+0000","last_clean":"2024-04-14T08:33:59.989122+0000","last_became_active":"2024-04-14T08:33:59.001135+0000","last_became_peered":"2024-04-14T08:33:59.001135+0000","last_unstale":"2024-04-14T08:33:59.989122+0000","last_undegraded":"2024-04-14T08:33:59.989122+0000","last_fullsized":"2024-04-14T08:33:59.989122+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:01:06.793819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.005776+0000","last_change":"2024-04-14T08:33:59.996287+0000","last_active":"2024-04-14T08:34:00.005776+0000","last_peered":"2024-04-14T08:34:00.005776+0000","last_clean":"2024-04-14T08:34:00.005776+0000","last_became_active":"2024-04-14T08:33:59.996006+0000","last_became_peered":"2024-04-14T08:33:59.996006+0000","last_unstale":"2024-04-14T08:34:00.005776+0000","last_undegraded":"2024-04-14T08:34:00.005776+0000","last_fullsized":"2024-04-14T08:34:00.005776+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:13.330734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.989285+0000","last_change":"2024-04-14T08:33:59.003581+0000","last_active":"2024-04-14T08:33:59.989285+0000","last_peered":"2024-04-14T08:33:59.989285+0000","last_clean":"2024-04-14T08:33:59.989285+0000","last_became_active":"2024-04-14T08:33:59.003154+0000","last_became_peered":"2024-04-14T08:33:59.003154+0000","last_unstale":"2024-04-14T08:33:59.989285+0000","last_undegraded":"2024-04-14T08:33:59.989285+0000","last_fullsized":"2024-04-14T08:33:59.989285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:09:41.482824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.008295+0000","last_change":"2024-04-14T08:34:00.003966+0000","last_active":"2024-04-14T08:34:00.008295+0000","last_peered":"2024-04-14T08:34:00.008295+0000","last_clean":"2024-04-14T08:34:00.008295+0000","last_became_active":"2024-04-14T08:34:00.003295+0000","last_became_peered":"2024-04-14T08:34:00.003295+0000","last_unstale":"2024-04-14T08:34:00.008295+0000","last_undegraded":"2024-04-14T08:34:00.008295+0000","last_fullsized":"2024-04-14T08:34:00.008295+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:13.330734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.997075+0000","last_change":"2024-04-14T08:33:59.003054+0000","last_active":"2024-04-14T08:33:59.997075+0000","last_peered":"2024-04-14T08:33:59.997075+0000","last_clean":"2024-04-14T08:33:59.997075+0000","last_became_active":"2024-04-14T08:33:59.002711+0000","last_became_peered":"2024-04-14T08:33:59.002711+0000","last_unstale":"2024-04-14T08:33:59.997075+0000","last_undegraded":"2024-04-14T08:33:59.997075+0000","last_fullsized":"2024-04-14T08:33:59.997075+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:42:45.693051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.013301+0000","last_change":"2024-04-14T08:34:00.008844+0000","last_active":"2024-04-14T08:34:00.013301+0000","last_peered":"2024-04-14T08:34:00.013301+0000","last_clean":"2024-04-14T08:34:00.013301+0000","last_became_active":"2024-04-14T08:34:00.008352+0000","last_became_peered":"2024-04-14T08:34:00.008352+0000","last_unstale":"2024-04-14T08:34:00.013301+0000","last_undegraded":"2024-04-14T08:34:00.013301+0000","last_fullsized":"2024-04-14T08:34:00.013301+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:17.444588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.997814+0000","last_change":"2024-04-14T08:33:59.008769+0000","last_active":"2024-04-14T08:33:59.997814+0000","last_peered":"2024-04-14T08:33:59.997814+0000","last_clean":"2024-04-14T08:33:59.997814+0000","last_became_active":"2024-04-14T08:33:59.008495+0000","last_became_peered":"2024-04-14T08:33:59.008495+0000","last_unstale":"2024-04-14T08:33:59.997814+0000","last_undegraded":"2024-04-14T08:33:59.997814+0000","last_fullsized":"2024-04-14T08:33:59.997814+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:42:45.693051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989778+0000","last_change":"2024-04-14T08:34:00.005429+0000","last_active":"2024-04-14T08:34:00.989778+0000","last_peered":"2024-04-14T08:34:00.989778+0000","last_clean":"2024-04-14T08:34:00.989778+0000","last_became_active":"2024-04-14T08:34:00.005112+0000","last_became_peered":"2024-04-14T08:34:00.005112+0000","last_unstale":"2024-04-14T08:34:00.989778+0000","last_undegraded":"2024-04-14T08:34:00.989778+0000","last_fullsized":"2024-04-14T08:34:00.989778+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:45:03.638858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:33:59.992314+0000","last_change":"2024-04-14T08:33:59.001251+0000","last_active":"2024-04-14T08:33:59.992314+0000","last_peered":"2024-04-14T08:33:59.992314+0000","last_clean":"2024-04-14T08:33:59.992314+0000","last_became_active":"2024-04-14T08:33:59.000965+0000","last_became_peered":"2024-04-14T08:33:59.000965+0000","last_unstale":"2024-04-14T08:33:59.992314+0000","last_undegraded":"2024-04-14T08:33:59.992314+0000","last_fullsized":"2024-04-14T08:33:59.992314+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:04:02.584849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.007931+0000","last_change":"2024-04-14T08:34:00.002427+0000","last_active":"2024-04-14T08:34:00.007931+0000","last_peered":"2024-04-14T08:34:00.007931+0000","last_clean":"2024-04-14T08:34:00.007931+0000","last_became_active":"2024-04-14T08:34:00.002155+0000","last_became_peered":"2024-04-14T08:34:00.002155+0000","last_unstale":"2024-04-14T08:34:00.007931+0000","last_undegraded":"2024-04-14T08:34:00.007931+0000","last_fullsized":"2024-04-14T08:34:00.007931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:58.978291+0000","last_clean_scrub_stamp":"2024-04-14T08:33:58.978291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:54.465535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989883+0000","last_change":"2024-04-14T08:33:59.004628+0000","last_active":"2024-04-14T08:34:00.989883+0000","last_peered":"2024-04-14T08:34:00.989883+0000","last_clean":"2024-04-14T08:34:00.989883+0000","last_became_active":"2024-04-14T08:33:59.004326+0000","last_became_peered":"2024-04-14T08:33:59.004326+0000","last_unstale":"2024-04-14T08:34:00.989883+0000","last_undegraded":"2024-04-14T08:34:00.989883+0000","last_fullsized":"2024-04-14T08:34:00.989883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:33:57.973905+0000","last_clean_scrub_stamp":"2024-04-14T08:33:57.973905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:26:00.995723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":206,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-14T08:34:00.989761+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:34:00.989761+0000","last_peered":"2024-04-14T08:34:00.989761+0000","last_clean":"2024-04-14T08:34:00.989761+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:34:00.989761+0000","last_undegraded":"2024-04-14T08:34:00.989761+0000","last_fullsized":"2024-04-14T08:34:00.989761+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":780065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":778417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:34:03.399 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-14T08:34:03.399 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-14T08:34:03.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.542+0000 7f9fb898c640 1 Processor -- start 2024-04-14T08:34:03.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.543+0000 7f9fb898c640 1 -- start start 2024-04-14T08:34:03.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.543+0000 7f9fb898c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9fb40ccdc0 0x7f9fb40cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.543+0000 7f9fb898c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9fb40cffb0 0x7f9fb40cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.543+0000 7f9fb898c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fb40cd6e0 con 0x7f9fb40d1a10 2024-04-14T08:34:03.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.543+0000 7f9fb898c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f9fb40d0610 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.543+0000 7f9fb898c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f9fb40ced00 con 0x7f9fb40cffb0 2024-04-14T08:34:03.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.543+0000 7f9fb259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9fb40ccdc0 0x7f9fb40cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.543+0000 7f9fb259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9fb40ccdc0 0x7f9fb40cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48712/0 (socket says 172.21.15.172:48712) 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.543+0000 7f9fb259c640 1 -- 172.21.15.172:0/1228660044 learned_addr learned my addr 172.21.15.172:0/1228660044 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.544+0000 7f9fb2d9d640 1 --2- 172.21.15.172:0/1228660044 >> v2:172.21.15.177:3300/0 conn(0x7f9fb40cffb0 0x7f9fb40cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.544+0000 7f9fb259c640 1 -- 172.21.15.172:0/1228660044 >> v2:172.21.15.177:3300/0 conn(0x7f9fb40cffb0 msgr2=0x7f9fb40cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.544+0000 7f9fb259c640 1 --2- 172.21.15.172:0/1228660044 >> v2:172.21.15.177:3300/0 conn(0x7f9fb40cffb0 0x7f9fb40cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.544+0000 7f9fb259c640 1 -- 172.21.15.172:0/1228660044 >> v1:172.21.15.177:6789/0 conn(0x7f9fb40d1a10 legacy=0x7f9fb415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.544+0000 7f9fb259c640 1 -- 172.21.15.172:0/1228660044 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb40d0860 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.545+0000 7f9fb259c640 1 --2- 172.21.15.172:0/1228660044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb40ccdc0 0x7f9fb40cd1a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4069090 tx=0x7f9fa402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5c5ad3212f89b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.545+0000 7f9fb1d9b640 1 -- 172.21.15.172:0/1228660044 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4035020 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.545+0000 7f9fb1d9b640 1 -- 172.21.15.172:0/1228660044 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa4031870 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.545+0000 7f9fb1d9b640 1 -- 172.21.15.172:0/1228660044 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4031c40 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.545+0000 7f9fb898c640 1 -- 172.21.15.172:0/1228660044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb40ccdc0 msgr2=0x7f9fb40cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.545+0000 7f9fb898c640 1 --2- 172.21.15.172:0/1228660044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb40ccdc0 0x7f9fb40cd1a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4069090 tx=0x7f9fa402b910 comp rx=0 tx=0).stop 2024-04-14T08:34:03.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.545+0000 7f9fb898c640 1 -- 172.21.15.172:0/1228660044 shutdown_connections 2024-04-14T08:34:03.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.545+0000 7f9fb898c640 1 --2- 172.21.15.172:0/1228660044 >> v2:172.21.15.177:3300/0 conn(0x7f9fb40cffb0 0x7f9fb40cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.545+0000 7f9fb898c640 1 --2- 172.21.15.172:0/1228660044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb40ccdc0 0x7f9fb40cd1a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.545+0000 7f9fb898c640 1 -- 172.21.15.172:0/1228660044 >> 172.21.15.172:0/1228660044 conn(0x7f9fb40d0f20 msgr2=0x7f9fb40d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:03.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.545+0000 7f9fb898c640 1 -- 172.21.15.172:0/1228660044 shutdown_connections 2024-04-14T08:34:03.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.545+0000 7f9fb898c640 1 -- 172.21.15.172:0/1228660044 wait complete. 2024-04-14T08:34:03.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.547+0000 7f9fb898c640 1 Processor -- start 2024-04-14T08:34:03.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.547+0000 7f9fb898c640 1 -- start start 2024-04-14T08:34:03.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.547+0000 7f9fb898c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb40ccdc0 0x7f9fb40a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.547+0000 7f9fb898c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fb40cffb0 0x7f9fb41f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.547+0000 7f9fb898c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fb40d1a10 0x7f9fb41f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.547+0000 7f9fb898c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb40cb730 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.547+0000 7f9fb898c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb415f1f0 con 0x7f9fb40cffb0 2024-04-14T08:34:03.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.547+0000 7f9fb898c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb40cf6d0 con 0x7f9fb40d1a10 2024-04-14T08:34:03.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fb259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb40ccdc0 0x7f9fb40a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fb259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb40ccdc0 0x7f9fb40a32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48728/0 (socket says 172.21.15.172:48728) 2024-04-14T08:34:03.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fb259c640 1 -- 172.21.15.172:0/2465241945 learned_addr learned my addr 172.21.15.172:0/2465241945 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:03.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fb2d9d640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fb40d1a10 0x7f9fb41f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fa9d9b640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fb40cffb0 0x7f9fb41f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fb259c640 1 -- 172.21.15.172:0/2465241945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fb40d1a10 msgr2=0x7f9fb41f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fb259c640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fb40d1a10 0x7f9fb41f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fb259c640 1 -- 172.21.15.172:0/2465241945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fb40cffb0 msgr2=0x7f9fb41f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fb259c640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fb40cffb0 0x7f9fb41f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fb259c640 1 -- 172.21.15.172:0/2465241945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb40b5540 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fb259c640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb40ccdc0 0x7f9fb40a32a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9fa402b950 tx=0x7f9fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:03.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fab7fe640 1 -- 172.21.15.172:0/2465241945 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4040070 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fab7fe640 1 -- 172.21.15.172:0/2465241945 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa4030d00 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fb898c640 1 -- 172.21.15.172:0/2465241945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb40b5820 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fab7fe640 1 -- 172.21.15.172:0/2465241945 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4035040 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.548+0000 7f9fb898c640 1 -- 172.21.15.172:0/2465241945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb40b5de0 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.550+0000 7f9fb898c640 1 -- 172.21.15.172:0/2465241945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb4075490 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.550+0000 7f9fab7fe640 1 -- 172.21.15.172:0/2465241945 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fa4038020 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.550+0000 7f9fab7fe640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f9406c8f0 0x7f9f9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.550+0000 7f9fab7fe640 1 -- 172.21.15.172:0/2465241945 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9fa40ada30 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.550+0000 7f9fa9d9b640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f9406c8f0 0x7f9f9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.551+0000 7f9fa9d9b640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f9406c8f0 0x7f9f9406edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4075340 tx=0x7f9f9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:03.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.555+0000 7f9fab7fe640 1 -- 172.21.15.172:0/2465241945 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9fa407b690 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.807+0000 7f9fb898c640 1 -- 172.21.15.172:0/2465241945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9fb40b4770 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.809+0000 7f9fab7fe640 1 -- 172.21.15.172:0/2465241945 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f9fa407f990 con 0x7f9fb40ccdc0 2024-04-14T08:34:03.811 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2024-04-14T08:34:03.811 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA== 2024-04-14T08:34:03.811 INFO:teuthology.orchestra.run.smithi172.stdout: caps mds = "allow" 2024-04-14T08:34:03.811 INFO:teuthology.orchestra.run.smithi172.stdout: caps mon = "allow r" 2024-04-14T08:34:03.811 INFO:teuthology.orchestra.run.smithi172.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-14T08:34:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:updated caps for client.0 2024-04-14T08:34:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.811+0000 7f9fb898c640 1 -- 172.21.15.172:0/2465241945 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f9406c8f0 msgr2=0x7f9f9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.811+0000 7f9fb898c640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f9406c8f0 0x7f9f9406edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4075340 tx=0x7f9f9c009040 comp rx=0 tx=0).stop 2024-04-14T08:34:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.811+0000 7f9fb898c640 1 -- 172.21.15.172:0/2465241945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb40ccdc0 msgr2=0x7f9fb40a32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.811+0000 7f9fb898c640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb40ccdc0 0x7f9fb40a32a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9fa402b950 tx=0x7f9fa40046c0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.811+0000 7f9fb898c640 1 -- 172.21.15.172:0/2465241945 shutdown_connections 2024-04-14T08:34:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.811+0000 7f9fb898c640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f9406c8f0 0x7f9f9406edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.811+0000 7f9fb898c640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fb40d1a10 0x7f9fb41f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.811+0000 7f9fb898c640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fb40cffb0 0x7f9fb41f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.811+0000 7f9fb898c640 1 --2- 172.21.15.172:0/2465241945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fb40ccdc0 0x7f9fb40a32a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.811+0000 7f9fb898c640 1 -- 172.21.15.172:0/2465241945 >> 172.21.15.172:0/2465241945 conn(0x7f9fb40d0f20 msgr2=0x7f9fb40b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.811+0000 7f9fb898c640 1 -- 172.21.15.172:0/2465241945 shutdown_connections 2024-04-14T08:34:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.811+0000 7f9fb898c640 1 -- 172.21.15.172:0/2465241945 wait complete. 2024-04-14T08:34:03.824 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:34:03.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.924+0000 7f01afe6d640 1 Processor -- start 2024-04-14T08:34:03.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.924+0000 7f01afe6d640 1 -- start start 2024-04-14T08:34:03.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.925+0000 7f01afe6d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f01a80d2830 0x7f01a80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.925+0000 7f01afe6d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f01a80cedd0 0x7f01a80b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.925+0000 7f01afe6d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01a80d3150 con 0x7f01a815e890 2024-04-14T08:34:03.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.925+0000 7f01afe6d640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f01a80cff30 con 0x7f01a80d2830 2024-04-14T08:34:03.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.925+0000 7f01afe6d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f01a80d2350 con 0x7f01a80cedd0 2024-04-14T08:34:03.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.925+0000 7f01adc09640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f01a80d2830 0x7f01a80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.925+0000 7f01adc09640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f01a80d2830 0x7f01a80d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48736/0 (socket says 172.21.15.172:48736) 2024-04-14T08:34:03.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.925+0000 7f01adc09640 1 -- 172.21.15.172:0/1758401145 learned_addr learned my addr 172.21.15.172:0/1758401145 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:03.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.926+0000 7f01ae40a640 1 --2- 172.21.15.172:0/1758401145 >> v2:172.21.15.177:3300/0 conn(0x7f01a80cedd0 0x7f01a80b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.926+0000 7f01adc09640 1 -- 172.21.15.172:0/1758401145 >> v2:172.21.15.177:3300/0 conn(0x7f01a80cedd0 msgr2=0x7f01a80b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:03.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.926+0000 7f01adc09640 1 --2- 172.21.15.172:0/1758401145 >> v2:172.21.15.177:3300/0 conn(0x7f01a80cedd0 0x7f01a80b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.926+0000 7f01adc09640 1 -- 172.21.15.172:0/1758401145 >> v1:172.21.15.177:6789/0 conn(0x7f01a815e890 legacy=0x7f01a80d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:03.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.926+0000 7f01adc09640 1 -- 172.21.15.172:0/1758401145 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a80cf320 con 0x7f01a80d2830 2024-04-14T08:34:03.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.926+0000 7f01adc09640 1 --2- 172.21.15.172:0/1758401145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a80d2830 0x7f01a80d2c10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f01a808e4b0 tx=0x7f01a402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bb0c79b37680dc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:03.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.927+0000 7f01acc07640 1 -- 172.21.15.172:0/1758401145 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4034070 con 0x7f01a80d2830 2024-04-14T08:34:03.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.927+0000 7f01acc07640 1 -- 172.21.15.172:0/1758401145 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a4030860 con 0x7f01a80d2830 2024-04-14T08:34:03.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.927+0000 7f01acc07640 1 -- 172.21.15.172:0/1758401145 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4030c30 con 0x7f01a80d2830 2024-04-14T08:34:03.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.927+0000 7f01afe6d640 1 -- 172.21.15.172:0/1758401145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a80d2830 msgr2=0x7f01a80d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.927+0000 7f01afe6d640 1 --2- 172.21.15.172:0/1758401145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a80d2830 0x7f01a80d2c10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f01a808e4b0 tx=0x7f01a402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.927+0000 7f01afe6d640 1 -- 172.21.15.172:0/1758401145 shutdown_connections 2024-04-14T08:34:03.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.927+0000 7f01afe6d640 1 --2- 172.21.15.172:0/1758401145 >> v2:172.21.15.177:3300/0 conn(0x7f01a80cedd0 0x7f01a80b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.927+0000 7f01afe6d640 1 --2- 172.21.15.172:0/1758401145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a80d2830 0x7f01a80d2c10 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.927+0000 7f01afe6d640 1 -- 172.21.15.172:0/1758401145 >> 172.21.15.172:0/1758401145 conn(0x7f01a809d590 msgr2=0x7f01a80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:03.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.928+0000 7f01afe6d640 1 -- 172.21.15.172:0/1758401145 shutdown_connections 2024-04-14T08:34:03.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.928+0000 7f01afe6d640 1 -- 172.21.15.172:0/1758401145 wait complete. 2024-04-14T08:34:03.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.929+0000 7f01afe6d640 1 Processor -- start 2024-04-14T08:34:03.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.929+0000 7f01afe6d640 1 -- start start 2024-04-14T08:34:03.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.929+0000 7f01afe6d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01a80cedd0 0x7f01a80c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.929+0000 7f01afe6d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a80d2830 0x7f01a8152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.929+0000 7f01afe6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a815e890 0x7f01a80b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.929+0000 7f01afe6d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01a80cfbc0 con 0x7f01a815e890 2024-04-14T08:34:03.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.929+0000 7f01afe6d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f01a80cf8e0 con 0x7f01a80d2830 2024-04-14T08:34:03.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.929+0000 7f01afe6d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f01a80b3c50 con 0x7f01a80cedd0 2024-04-14T08:34:03.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.929+0000 7f01ae40a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a815e890 0x7f01a80b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.929+0000 7f01ae40a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a815e890 0x7f01a80b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48748/0 (socket says 172.21.15.172:48748) 2024-04-14T08:34:03.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01ae40a640 1 -- 172.21.15.172:0/2013939658 learned_addr learned my addr 172.21.15.172:0/2013939658 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:03.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01ad408640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a80d2830 0x7f01a8152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01adc09640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01a80cedd0 0x7f01a80c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01ae40a640 1 -- 172.21.15.172:0/2013939658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01a80cedd0 msgr2=0x7f01a80c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01ae40a640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01a80cedd0 0x7f01a80c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01ae40a640 1 -- 172.21.15.172:0/2013939658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a80d2830 msgr2=0x7f01a8152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01ae40a640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a80d2830 0x7f01a8152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01ae40a640 1 -- 172.21.15.172:0/2013939658 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a81617a0 con 0x7f01a815e890 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01adc09640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01a80cedd0 0x7f01a80c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01ad408640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a80d2830 0x7f01a8152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01ae40a640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a815e890 0x7f01a80b1d30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f01a0007e40 tx=0x7f01a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f019e7fc640 1 -- 172.21.15.172:0/2013939658 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0014620 con 0x7f01a815e890 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f019e7fc640 1 -- 172.21.15.172:0/2013939658 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a0019070 con 0x7f01a815e890 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01afe6d640 1 -- 172.21.15.172:0/2013939658 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a80c8480 con 0x7f01a815e890 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f019e7fc640 1 -- 172.21.15.172:0/2013939658 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0014a20 con 0x7f01a815e890 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.930+0000 7f01afe6d640 1 -- 172.21.15.172:0/2013939658 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a80c8a40 con 0x7f01a815e890 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.931+0000 7f01afe6d640 1 -- 172.21.15.172:0/2013939658 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a8068520 con 0x7f01a815e890 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.932+0000 7f019e7fc640 1 -- 172.21.15.172:0/2013939658 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01a0014bc0 con 0x7f01a815e890 2024-04-14T08:34:03.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.932+0000 7f019e7fc640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f018c06c920 0x7f018c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:03.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.932+0000 7f019e7fc640 1 -- 172.21.15.172:0/2013939658 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f01a008c510 con 0x7f01a815e890 2024-04-14T08:34:03.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.932+0000 7f01adc09640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f018c06c920 0x7f018c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:03.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.932+0000 7f01adc09640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f018c06c920 0x7f018c06ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f01a4036000 tx=0x7f01a403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:03.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:03.937+0000 7f019e7fc640 1 -- 172.21.15.172:0/2013939658 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f01a005a840 con 0x7f01a815e890 2024-04-14T08:34:04.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.174+0000 7f01afe6d640 1 -- 172.21.15.172:0/2013939658 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01a80d2610 con 0x7f01a815e890 2024-04-14T08:34:04.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.174+0000 7f019e7fc640 1 -- 172.21.15.172:0/2013939658 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7f01a005a140 con 0x7f01a815e890 2024-04-14T08:34:04.176 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:04.176 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1038094401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1038094401},{"type":"v1","addr":"172.21.15.177:6834","nonce":1038094401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1119512744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1119512744},{"type":"v1","addr":"172.21.15.177:6835","nonce":1119512744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3054138283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3054138283},{"type":"v1","addr":"172.21.15.172:6837","nonce":3054138283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:33:59.340499+0000","modified":"2024-04-14T08:34:01.467567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5308":{"gid":5308,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6836/3420108191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3420108191},{"type":"v1","addr":"172.21.15.172:6836","nonce":3420108191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5308,"qdb_cluster":[5308]},"id":2}]} 2024-04-14T08:34:04.176 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 16 2024-04-14T08:34:04.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.177+0000 7f01afe6d640 1 -- 172.21.15.172:0/2013939658 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f018c06c920 msgr2=0x7f018c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.177+0000 7f01afe6d640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f018c06c920 0x7f018c06ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f01a4036000 tx=0x7f01a403b040 comp rx=0 tx=0).stop 2024-04-14T08:34:04.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.177+0000 7f01afe6d640 1 -- 172.21.15.172:0/2013939658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a815e890 msgr2=0x7f01a80b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.177+0000 7f01afe6d640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a815e890 0x7f01a80b1d30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f01a0007e40 tx=0x7f01a000e4b0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.177+0000 7f01afe6d640 1 -- 172.21.15.172:0/2013939658 shutdown_connections 2024-04-14T08:34:04.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.177+0000 7f01afe6d640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f018c06c920 0x7f018c06ede0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.177+0000 7f01afe6d640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a815e890 0x7f01a80b1d30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.177+0000 7f01afe6d640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a80d2830 0x7f01a8152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.177+0000 7f01afe6d640 1 --2- 172.21.15.172:0/2013939658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01a80cedd0 0x7f01a80c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.177+0000 7f01afe6d640 1 -- 172.21.15.172:0/2013939658 >> 172.21.15.172:0/2013939658 conn(0x7f01a809d590 msgr2=0x7f01a80d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:04.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.177+0000 7f01afe6d640 1 -- 172.21.15.172:0/2013939658 shutdown_connections 2024-04-14T08:34:04.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.177+0000 7f01afe6d640 1 -- 172.21.15.172:0/2013939658 wait complete. 2024-04-14T08:34:04.592 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:33:59.340499+0000', 'modified': '2024-04-14T08:34:01.467567+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5308}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5308': {'gid': 5308, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6836/3420108191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 3420108191}, {'type': 'v1', 'addr': '172.21.15.172:6836', 'nonce': 3420108191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5308, 'qdb_cluster': [5308]} 2024-04-14T08:34:04.592 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:34:04.593 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-14T08:34:04.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.698+0000 7f0dc06de640 1 Processor -- start 2024-04-14T08:34:04.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.698+0000 7f0dc06de640 1 -- start start 2024-04-14T08:34:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.699+0000 7f0dc06de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0db8162ba0 0x7f0db80d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.699+0000 7f0dc06de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0db80cefb0 con 0x7f0db815e430 2024-04-14T08:34:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.699+0000 7f0dc06de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0db80cf290 con 0x7f0db8162710 2024-04-14T08:34:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.699+0000 7f0dc06de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0db80cf570 con 0x7f0db8162ba0 2024-04-14T08:34:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.699+0000 7f0dbec7b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0db8162ba0 0x7f0db80d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.699+0000 7f0dbe47a640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f0db815e430 0x7f0db8162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48332/0 (socket says 172.21.15.172:48332) 2024-04-14T08:34:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.699+0000 7f0dbe47a640 1 -- 172.21.15.172:0/2342939580 learned_addr learned my addr 172.21.15.172:0/2342939580 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.699+0000 7f0dbec7b640 1 -- 172.21.15.172:0/2342939580 >> v1:172.21.15.177:6789/0 conn(0x7f0db8162710 legacy=0x7f0db80d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.699+0000 7f0dbec7b640 1 -- 172.21.15.172:0/2342939580 >> v1:172.21.15.172:6789/0 conn(0x7f0db815e430 legacy=0x7f0db8162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.699+0000 7f0dbec7b640 1 -- 172.21.15.172:0/2342939580 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db80cf850 con 0x7f0db8162ba0 2024-04-14T08:34:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.699+0000 7f0dbd478640 1 -- 172.21.15.172:0/2342939580 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430448192 0 0) 0x7f0db80cefb0 con 0x7f0db815e430 2024-04-14T08:34:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.700+0000 7f0dbec7b640 1 --2- 172.21.15.172:0/2342939580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0db8162ba0 0x7f0db80d2500 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0db00097a0 tx=0x7f0db002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe47f24bd1f7d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.700+0000 7f0dbd478640 1 -- 172.21.15.172:0/2342939580 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0035020 con 0x7f0db8162ba0 2024-04-14T08:34:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.700+0000 7f0dbd478640 1 -- 172.21.15.172:0/2342939580 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db0031830 con 0x7f0db8162ba0 2024-04-14T08:34:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.700+0000 7f0dbd478640 1 -- 172.21.15.172:0/2342939580 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0031c00 con 0x7f0db8162ba0 2024-04-14T08:34:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.701+0000 7f0dc06de640 1 -- 172.21.15.172:0/2342939580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0db8162ba0 msgr2=0x7f0db80d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.701+0000 7f0dc06de640 1 --2- 172.21.15.172:0/2342939580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0db8162ba0 0x7f0db80d2500 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0db00097a0 tx=0x7f0db002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.701+0000 7f0dc06de640 1 -- 172.21.15.172:0/2342939580 shutdown_connections 2024-04-14T08:34:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.701+0000 7f0dc06de640 1 --2- 172.21.15.172:0/2342939580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0db8162ba0 0x7f0db80d2500 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.701+0000 7f0dc06de640 1 -- 172.21.15.172:0/2342939580 >> 172.21.15.172:0/2342939580 conn(0x7f0db809d560 msgr2=0x7f0db80d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.701+0000 7f0dc06de640 1 -- 172.21.15.172:0/2342939580 shutdown_connections 2024-04-14T08:34:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.701+0000 7f0dc06de640 1 -- 172.21.15.172:0/2342939580 wait complete. 2024-04-14T08:34:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.702+0000 7f0dc06de640 1 Processor -- start 2024-04-14T08:34:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.702+0000 7f0dc06de640 1 -- start start 2024-04-14T08:34:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dc06de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0db815e430 0x7f0db8150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dc06de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0db8162710 0x7f0db81e8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dc06de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0db8162ba0 0x7f0db80b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dc06de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0db80d4e20 con 0x7f0db8162ba0 2024-04-14T08:34:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dc06de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0db815e810 con 0x7f0db8162710 2024-04-14T08:34:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dc06de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0db80d0050 con 0x7f0db815e430 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dbec7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0db8162ba0 0x7f0db80b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dbec7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0db8162ba0 0x7f0db80b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48776/0 (socket says 172.21.15.172:48776) 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dbec7b640 1 -- 172.21.15.172:0/1445992245 learned_addr learned my addr 172.21.15.172:0/1445992245 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dbdc79640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0db8162710 0x7f0db81e8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0db815e430 0x7f0db8150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dbec7b640 1 -- 172.21.15.172:0/1445992245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0db815e430 msgr2=0x7f0db8150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dbec7b640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0db815e430 0x7f0db8150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dbec7b640 1 -- 172.21.15.172:0/1445992245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0db8162710 msgr2=0x7f0db81e8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dbec7b640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0db8162710 0x7f0db81e8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dbec7b640 1 -- 172.21.15.172:0/1445992245 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db815b7d0 con 0x7f0db8162ba0 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.703+0000 7f0dbec7b640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0db8162ba0 0x7f0db80b7760 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0db00098c0 tx=0x7f0db00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.704+0000 7f0db6ffd640 1 -- 172.21.15.172:0/1445992245 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0004230 con 0x7f0db8162ba0 2024-04-14T08:34:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.704+0000 7f0db6ffd640 1 -- 172.21.15.172:0/1445992245 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db00043d0 con 0x7f0db8162ba0 2024-04-14T08:34:04.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.704+0000 7f0dc06de640 1 -- 172.21.15.172:0/1445992245 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db80b9bd0 con 0x7f0db8162ba0 2024-04-14T08:34:04.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.704+0000 7f0db6ffd640 1 -- 172.21.15.172:0/1445992245 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db003c990 con 0x7f0db8162ba0 2024-04-14T08:34:04.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.704+0000 7f0dc06de640 1 -- 172.21.15.172:0/1445992245 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db81f5010 con 0x7f0db8162ba0 2024-04-14T08:34:04.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.705+0000 7f0db6ffd640 1 -- 172.21.15.172:0/1445992245 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0db0038020 con 0x7f0db8162ba0 2024-04-14T08:34:04.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.706+0000 7f0db6ffd640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0da406c8f0 0x7f0da406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.706+0000 7f0db6ffd640 1 -- 172.21.15.172:0/1445992245 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f0db00adee0 con 0x7f0db8162ba0 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.706+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0da406c8f0 0x7f0da406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.706+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0da406c8f0 0x7f0da406edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0da8007ed0 tx=0x7f0da8007e60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.707+0000 7f0dc35b7740 1 Processor -- start 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.707+0000 7f0dc35b7740 1 -- start start 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.707+0000 7f0dc35b7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.707+0000 7f0dc35b7740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4f40 0x55e7f94d78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dc35b7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94a8040 0x55e7f94d7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dc35b7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e7f9411030 con 0x55e7f94d4b60 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dc35b7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e7f93b9620 con 0x55e7f94a8040 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dc35b7740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e7f940cfe0 con 0x55e7f94d4f40 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dbe47a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dbe47a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94a7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48792/0 (socket says 172.21.15.172:48792) 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dbe47a640 1 -- 172.21.15.172:0/3057644012 learned_addr learned my addr 172.21.15.172:0/3057644012 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:04.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dbdc79640 1 --2- 172.21.15.172:0/3057644012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4f40 0x55e7f94d78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dbec7b640 1 --2- 172.21.15.172:0/3057644012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94a8040 0x55e7f94d7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dbe47a640 1 -- 172.21.15.172:0/3057644012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4f40 msgr2=0x55e7f94d78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/3057644012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4f40 0x55e7f94d78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dbe47a640 1 -- 172.21.15.172:0/3057644012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94a8040 msgr2=0x55e7f94d7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/3057644012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94a8040 0x55e7f94d7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dbe47a640 1 -- 172.21.15.172:0/3057644012 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7f91252d0 con 0x55e7f94d4b60 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/3057644012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94a7b00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0da8001d20 tx=0x7f0da8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a7090c63ea5f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0d9ffff640 1 -- 172.21.15.172:0/3057644012 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da80159c0 con 0x55e7f94d4b60 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0d9ffff640 1 -- 172.21.15.172:0/3057644012 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da8015b60 con 0x55e7f94d4b60 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.708+0000 7f0d9ffff640 1 -- 172.21.15.172:0/3057644012 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8021450 con 0x55e7f94d4b60 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.709+0000 7f0dc35b7740 1 -- 172.21.15.172:0/3057644012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 msgr2=0x55e7f94a7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.709+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/3057644012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94a7b00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0da8001d20 tx=0x7f0da8012000 comp rx=0 tx=0).stop 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.709+0000 7f0dc35b7740 1 -- 172.21.15.172:0/3057644012 shutdown_connections 2024-04-14T08:34:04.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.709+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/3057644012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94a8040 0x55e7f94d7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.709+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/3057644012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4f40 0x55e7f94d78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.709+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/3057644012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94a7b00 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.709+0000 7f0dc35b7740 1 -- 172.21.15.172:0/3057644012 >> 172.21.15.172:0/3057644012 conn(0x55e7f94a9d70 msgr2=0x55e7f94f28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:04.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.709+0000 7f0dc35b7740 1 -- 172.21.15.172:0/3057644012 shutdown_connections 2024-04-14T08:34:04.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.709+0000 7f0dc35b7740 1 -- 172.21.15.172:0/3057644012 wait complete. 2024-04-14T08:34:04.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.710+0000 7f0dc35b7740 1 -- start start 2024-04-14T08:34:04.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.710+0000 7f0dc35b7740 1 Processor -- start 2024-04-14T08:34:04.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.711+0000 7f0dc35b7740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4b60 0x55e7f94cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.711+0000 7f0dc35b7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4f40 0x55e7f94ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.711+0000 7f0dc35b7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d7820 0x55e7f94cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.711+0000 7f0dc35b7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e7f94ac410 con 0x55e7f94d4f40 2024-04-14T08:34:04.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.711+0000 7f0dc35b7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e7f93cc9e0 con 0x55e7f94d7820 2024-04-14T08:34:04.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.711+0000 7f0dc35b7740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e7f94aec50 con 0x55e7f94d4b60 2024-04-14T08:34:04.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.711+0000 7f0dc35b7740 10 client.0 _ll_register_callbacks cb 0x55e7f9409080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:34:04.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.711+0000 7f0dbdc79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4f40 0x55e7f94ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.711+0000 7f0dbdc79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4f40 0x55e7f94ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48800/0 (socket says 172.21.15.172:48800) 2024-04-14T08:34:04.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.711+0000 7f0dbdc79640 1 -- 172.21.15.172:0/1145286316 learned_addr learned my addr 172.21.15.172:0/1145286316 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.711+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4b60 0x55e7f94cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.711+0000 7f0dbec7b640 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d7820 0x55e7f94cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dbdc79640 1 -- 172.21.15.172:0/1145286316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4b60 msgr2=0x55e7f94cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dbdc79640 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4b60 0x55e7f94cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dbdc79640 1 -- 172.21.15.172:0/1145286316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d7820 msgr2=0x55e7f94cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dbdc79640 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d7820 0x55e7f94cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dbdc79640 1 -- 172.21.15.172:0/1145286316 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7f91ef6b0 con 0x55e7f94d4f40 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dbdc79640 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4f40 0x55e7f94ce3f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0db81621c0 tx=0x7f0dac00f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce8fd3274eb5d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0d9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0d9d7fa640 1 -- 172.21.15.172:0/1145286316 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac015610 con 0x55e7f94d4f40 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0d9d7fa640 1 -- 172.21.15.172:0/1145286316 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f0dac0157b0 con 0x55e7f94d4f40 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1145286316 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da8015b60 con 0x55e7f94d4f40 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0d9d7fa640 1 -- 172.21.15.172:0/1145286316 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f0dac022050 con 0x55e7f94d4f40 2024-04-14T08:34:04.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dc35b7740 10 client.5481 fetch_fsmap learned FSMap version 16 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1145286316 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55e7f94cda00 con 0x55e7f94d4f40 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0d9d7fa640 1 -- 172.21.15.172:0/1145286316 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da8015b60 con 0x55e7f94d4f40 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0d9d7fa640 1 -- 172.21.15.172:0/1145286316 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f0dac025020 con 0x55e7f94d4f40 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dc35b7740 10 client.5481 fetch_fsmap finished waiting for FSMap version 16 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dc35b7740 10 client.5481 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dc35b7740 20 client.5481 populate_metadata read hostname 'smithi172' 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.172:6835/3420108191,v1:172.21.15.172:6836/3420108191] conn(0x55e7f9506a60 0x55e7f9511620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dc35b7740 4 client.5481 mds_command: new command op to 5308 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.712+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1145286316 --> [v2:172.21.15.172:6835/3420108191,v1:172.21.15.172:6836/3420108191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7f93b8cc0 con 0x55e7f9506a60 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.713+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.172:6835/3420108191,v1:172.21.15.172:6836/3420108191] conn(0x55e7f9506a60 0x55e7f9511620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.713+0000 7f0d9d7fa640 10 client.5481 ms_handle_connect on v2:172.21.15.172:6835/3420108191 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.713+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.172:6835/3420108191,v1:172.21.15.172:6836/3420108191] conn(0x55e7f9506a60 0x55e7f9511620 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0200dc850555de7 server_cookie=10e1671bb8eb336a in_seq=0 out_seq=0 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0d9d7fa640 1 -- 172.21.15.172:0/1145286316 <== mds.0 v2:172.21.15.172:6835/3420108191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7f93b8cc0 con 0x55e7f9506a60 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0d9d7fa640 10 client.5481 handle_command_reply: tid=0 2024-04-14T08:34:04.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0dc35b7740 1 client.5481 shutdown 2024-04-14T08:34:04.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1145286316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4f40 msgr2=0x55e7f94ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4f40 0x55e7f94ce3f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0db81621c0 tx=0x7f0dac00f5b0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1145286316 shutdown_connections 2024-04-14T08:34:04.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.172:6835/3420108191,v1:172.21.15.172:6836/3420108191] conn(0x55e7f9506a60 0x55e7f9511620 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d7820 0x55e7f94cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4f40 0x55e7f94ce3f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0d9d7fa640 0 client.5481 ms_handle_reset on v2:172.21.15.172:6835/3420108191 2024-04-14T08:34:04.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/1145286316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4b60 0x55e7f94cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1145286316 >> 172.21.15.172:0/1145286316 conn(0x55e7f94a8040 msgr2=0x55e7f94aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:04.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1145286316 shutdown_connections 2024-04-14T08:34:04.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1145286316 wait complete. 2024-04-14T08:34:04.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.714+0000 7f0dc35b7740 20 client.5481 trim_cache size 0 max 16384 2024-04-14T08:34:04.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.732+0000 7f0dc35b7740 1 Processor -- start 2024-04-14T08:34:04.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.732+0000 7f0dc35b7740 1 -- start start 2024-04-14T08:34:04.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dc35b7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dc35b7740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4f40 0x55e7f94ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dc35b7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d7820 0x55e7f94cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dc35b7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0db81f6b40 con 0x55e7f94d4b60 2024-04-14T08:34:04.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dc35b7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0db80b5b50 con 0x55e7f94d7820 2024-04-14T08:34:04.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dc35b7740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0db81f64f0 con 0x55e7f94d4f40 2024-04-14T08:34:04.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dbe47a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dbe47a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48814/0 (socket says 172.21.15.172:48814) 2024-04-14T08:34:04.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dbe47a640 1 -- 172.21.15.172:0/1159483701 learned_addr learned my addr 172.21.15.172:0/1159483701 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dbe47a640 1 -- 172.21.15.172:0/1159483701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4f40 msgr2=0x55e7f94ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/1159483701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4f40 0x55e7f94ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dbe47a640 1 -- 172.21.15.172:0/1159483701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d7820 msgr2=0x55e7f94cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/1159483701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d7820 0x55e7f94cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dbe47a640 1 -- 172.21.15.172:0/1159483701 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7f94a2f30 con 0x55e7f94d4b60 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/1159483701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94ce150 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0da8001b20 tx=0x7f0da8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109325c296dbc460 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.733+0000 7f0d9f7fe640 1 -- 172.21.15.172:0/1159483701 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8034070 con 0x55e7f94d4b60 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.734+0000 7f0d9f7fe640 1 -- 172.21.15.172:0/1159483701 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da8015a40 con 0x55e7f94d4b60 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.734+0000 7f0d9f7fe640 1 -- 172.21.15.172:0/1159483701 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8004d60 con 0x55e7f94d4b60 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.734+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1159483701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 msgr2=0x55e7f94ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.734+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/1159483701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94ce150 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0da8001b20 tx=0x7f0da8013000 comp rx=0 tx=0).stop 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.734+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1159483701 shutdown_connections 2024-04-14T08:34:04.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.734+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/1159483701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d7820 0x55e7f94cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.734+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/1159483701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d4f40 0x55e7f94ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.734+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/1159483701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94ce150 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.734+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1159483701 >> 172.21.15.172:0/1159483701 conn(0x55e7f94a8040 msgr2=0x55e7f950c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:04.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.734+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1159483701 shutdown_connections 2024-04-14T08:34:04.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.734+0000 7f0dc35b7740 1 -- 172.21.15.172:0/1159483701 wait complete. 2024-04-14T08:34:04.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.735+0000 7f0dc35b7740 1 -- start start 2024-04-14T08:34:04.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.735+0000 7f0dc35b7740 1 Processor -- start 2024-04-14T08:34:04.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dc35b7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dc35b7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d4f40 0x55e7f9500fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dc35b7740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d7820 0x55e7f9501500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dc35b7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e7f94da270 con 0x55e7f94d4b60 2024-04-14T08:34:04.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dc35b7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e7f94cdf30 con 0x55e7f94d4f40 2024-04-14T08:34:04.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dc35b7740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e7f94d9e50 con 0x55e7f94d7820 2024-04-14T08:34:04.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dc35b7740 10 client.0 _ll_register_callbacks cb 0x7f0dac00d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:34:04.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dbe47a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dbe47a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48828/0 (socket says 172.21.15.172:48828) 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dbe47a640 1 -- 172.21.15.172:0/444912677 learned_addr learned my addr 172.21.15.172:0/444912677 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dbdc79640 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d4f40 0x55e7f9500fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dbec7b640 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d7820 0x55e7f9501500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dbe47a640 1 -- 172.21.15.172:0/444912677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d7820 msgr2=0x55e7f9501500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d7820 0x55e7f9501500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dbe47a640 1 -- 172.21.15.172:0/444912677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d4f40 msgr2=0x55e7f9500fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d4f40 0x55e7f9500fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dbe47a640 1 -- 172.21.15.172:0/444912677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7f9501d10 con 0x55e7f94d4b60 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dbec7b640 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d7820 0x55e7f9501500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dbe47a640 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94cb6c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0da8032040 tx=0x7f0da8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f302d85eb38eb238 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0d9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:34:04.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0d9d7fa640 1 -- 172.21.15.172:0/444912677 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8034070 con 0x55e7f94d4b60 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0d9d7fa640 1 -- 172.21.15.172:0/444912677 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f0da802f430 con 0x55e7f94d4b60 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.736+0000 7f0dc35b7740 1 -- 172.21.15.172:0/444912677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da8015a40 con 0x55e7f94d4b60 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0d9d7fa640 1 -- 172.21.15.172:0/444912677 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f0da8042050 con 0x55e7f94d4b60 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0dc35b7740 10 client.5487 fetch_fsmap learned FSMap version 16 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0dc35b7740 1 -- 172.21.15.172:0/444912677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55e7f956db50 con 0x55e7f94d4b60 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0d9d7fa640 1 -- 172.21.15.172:0/444912677 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da8015a40 con 0x55e7f94d4b60 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0d9d7fa640 1 -- 172.21.15.172:0/444912677 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f0da8045020 con 0x55e7f94d4b60 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0dc35b7740 10 client.5487 fetch_fsmap finished waiting for FSMap version 16 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0dc35b7740 10 client.5487 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0dc35b7740 20 client.5487 populate_metadata read hostname 'smithi172' 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.172:6835/3420108191,v1:172.21.15.172:6836/3420108191] conn(0x55e7f94db060 0x55e7f956c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0dc35b7740 4 client.5487 mds_command: new command op to 5308 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:34:04.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0dc35b7740 1 -- 172.21.15.172:0/444912677 --> [v2:172.21.15.172:6835/3420108191,v1:172.21.15.172:6836/3420108191] -- command(tid 0: {"prefix": "status"}) -- 0x55e7f941e570 con 0x55e7f94db060 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0dbdc79640 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.172:6835/3420108191,v1:172.21.15.172:6836/3420108191] conn(0x55e7f94db060 0x55e7f956c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0dbdc79640 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.172:6835/3420108191,v1:172.21.15.172:6836/3420108191] conn(0x55e7f94db060 0x55e7f956c970 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb69bc0b4809d1f server_cookie=c21c7d17b04286e4 in_seq=0 out_seq=0 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0d9d7fa640 10 client.5487 ms_handle_connect on v2:172.21.15.172:6835/3420108191 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0d9d7fa640 1 -- 172.21.15.172:0/444912677 <== mds.0 v2:172.21.15.172:6835/3420108191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e7f941e570 con 0x55e7f94db060 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0d9d7fa640 10 client.5487 handle_command_reply: tid=0 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.737+0000 7f0dc35b7740 1 client.5487 shutdown 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc35b7740 1 -- 172.21.15.172:0/444912677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 msgr2=0x55e7f94cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94cb6c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0da8032040 tx=0x7f0da8001250 comp rx=0 tx=0).stop 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc35b7740 1 -- 172.21.15.172:0/444912677 shutdown_connections 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.172:6835/3420108191,v1:172.21.15.172:6836/3420108191] conn(0x55e7f94db060 0x55e7f956c970 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e7f94d7820 0x55e7f9501500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7f94d4f40 0x55e7f9500fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc35b7740 1 --2- 172.21.15.172:0/444912677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7f94d4b60 0x55e7f94cb6c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc35b7740 1 -- 172.21.15.172:0/444912677 >> 172.21.15.172:0/444912677 conn(0x55e7f94a8040 msgr2=0x55e7f94d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:04.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0d9d7fa640 0 client.5487 ms_handle_reset on v2:172.21.15.172:6835/3420108191 2024-04-14T08:34:04.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc35b7740 1 -- 172.21.15.172:0/444912677 shutdown_connections 2024-04-14T08:34:04.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc35b7740 1 -- 172.21.15.172:0/444912677 wait complete. 2024-04-14T08:34:04.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc35b7740 20 client.5487 trim_cache size 0 max 16384 2024-04-14T08:34:04.746 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:34:04.746 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:34:04.746 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:34:04.746 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 5308, 2024-04-14T08:34:04.746 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:34:04.746 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:34:04.747 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:34:04.747 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 4.7415350150000002, 2024-04-14T08:34:04.747 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 16, 2024-04-14T08:34:04.747 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 39, 2024-04-14T08:34:04.747 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 39, 2024-04-14T08:34:04.747 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 11.586886782000001 2024-04-14T08:34:04.747 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:34:04.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc06de640 1 -- 172.21.15.172:0/1445992245 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0da406c8f0 msgr2=0x7f0da406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc06de640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0da406c8f0 0x7f0da406edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0da8007ed0 tx=0x7f0da8007e60 comp rx=0 tx=0).stop 2024-04-14T08:34:04.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc06de640 1 -- 172.21.15.172:0/1445992245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0db8162ba0 msgr2=0x7f0db80b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc06de640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0db8162ba0 0x7f0db80b7760 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0db00098c0 tx=0x7f0db00046c0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc06de640 1 -- 172.21.15.172:0/1445992245 shutdown_connections 2024-04-14T08:34:04.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc06de640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0da406c8f0 0x7f0da406edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc06de640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0db8162ba0 0x7f0db80b7760 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc06de640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0db8162710 0x7f0db81e8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc06de640 1 --2- 172.21.15.172:0/1445992245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0db815e430 0x7f0db8150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc06de640 1 -- 172.21.15.172:0/1445992245 >> 172.21.15.172:0/1445992245 conn(0x7f0db809d560 msgr2=0x7f0db806c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:04.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.738+0000 7f0dc06de640 1 -- 172.21.15.172:0/1445992245 shutdown_connections 2024-04-14T08:34:04.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.739+0000 7f0dc06de640 1 -- 172.21.15.172:0/1445992245 wait complete. 2024-04-14T08:34:04.751 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:34:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.855+0000 7f4791dfe640 1 Processor -- start 2024-04-14T08:34:04.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.855+0000 7f4791dfe640 1 -- start start 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f4791dfe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f478c15c150 0x7f478c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f4791dfe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f478c162700 0x7f478c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f4791dfe640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f478c0d0200 con 0x7f478c0d1840 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f4791dfe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f478c0d1cb0 con 0x7f478c15c150 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f4791dfe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f478c0cef10 con 0x7f478c162700 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f478bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f478c162700 0x7f478c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f478b7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f478c15c150 0x7f478c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f478bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f478c162700 0x7f478c0d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:53742/0 (socket says 172.21.15.172:53742) 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f478bfff640 1 -- 172.21.15.172:0/4083034120 learned_addr learned my addr 172.21.15.172:0/4083034120 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f478affd640 1 --1- 172.21.15.172:0/4083034120 >> v1:172.21.15.177:6789/0 conn(0x7f478c0d1840 0x7f478c161ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:38786/0 (socket says 172.21.15.172:38786) 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f478b7fe640 1 -- 172.21.15.172:0/4083034120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f478c162700 msgr2=0x7f478c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f478b7fe640 1 --2- 172.21.15.172:0/4083034120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f478c162700 0x7f478c0d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f478b7fe640 1 -- 172.21.15.172:0/4083034120 >> v1:172.21.15.177:6789/0 conn(0x7f478c0d1840 legacy=0x7f478c161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f478b7fe640 1 -- 172.21.15.172:0/4083034120 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478c0cf160 con 0x7f478c15c150 2024-04-14T08:34:04.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f478a7fc640 1 -- 172.21.15.172:0/4083034120 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1785363190 0 0) 0x7f478c0d0200 con 0x7f478c0d1840 2024-04-14T08:34:04.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.856+0000 7f478bfff640 1 --2- 172.21.15.172:0/4083034120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f478c162700 0x7f478c0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:34:04.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.857+0000 7f478b7fe640 1 --2- 172.21.15.172:0/4083034120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f478c15c150 0x7f478c0d1300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f477c0097a0 tx=0x7f477c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aacfce2db69f1b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:04.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.857+0000 7f478a7fc640 1 -- 172.21.15.172:0/4083034120 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c035020 con 0x7f478c15c150 2024-04-14T08:34:04.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.857+0000 7f478a7fc640 1 -- 172.21.15.172:0/4083034120 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477c031830 con 0x7f478c15c150 2024-04-14T08:34:04.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.857+0000 7f478a7fc640 1 -- 172.21.15.172:0/4083034120 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c031c00 con 0x7f478c15c150 2024-04-14T08:34:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.858+0000 7f4791dfe640 1 -- 172.21.15.172:0/4083034120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f478c15c150 msgr2=0x7f478c0d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.858+0000 7f4791dfe640 1 --2- 172.21.15.172:0/4083034120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f478c15c150 0x7f478c0d1300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f477c0097a0 tx=0x7f477c02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.858+0000 7f4791dfe640 1 -- 172.21.15.172:0/4083034120 shutdown_connections 2024-04-14T08:34:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.858+0000 7f4791dfe640 1 --2- 172.21.15.172:0/4083034120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f478c162700 0x7f478c0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.858+0000 7f4791dfe640 1 --2- 172.21.15.172:0/4083034120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f478c15c150 0x7f478c0d1300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.858+0000 7f4791dfe640 1 -- 172.21.15.172:0/4083034120 >> 172.21.15.172:0/4083034120 conn(0x7f478c09d580 msgr2=0x7f478c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.858+0000 7f4791dfe640 1 -- 172.21.15.172:0/4083034120 shutdown_connections 2024-04-14T08:34:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.858+0000 7f4791dfe640 1 -- 172.21.15.172:0/4083034120 wait complete. 2024-04-14T08:34:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.859+0000 7f4791dfe640 1 Processor -- start 2024-04-14T08:34:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.859+0000 7f4791dfe640 1 -- start start 2024-04-14T08:34:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.860+0000 7f4791dfe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f478c0d1840 0x7f478c157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.860+0000 7f4791dfe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f478c15c150 0x7f478c1495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.860+0000 7f4791dfe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f478c162700 0x7f478c15c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.860+0000 7f4791dfe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f478c0cfa00 con 0x7f478c15c150 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.860+0000 7f4791dfe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f478c0cf720 con 0x7f478c0d1840 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.860+0000 7f4791dfe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f478c0cfce0 con 0x7f478c162700 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.860+0000 7f478affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f478c15c150 0x7f478c1495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.860+0000 7f478affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f478c15c150 0x7f478c1495f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48838/0 (socket says 172.21.15.172:48838) 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.860+0000 7f478affd640 1 -- 172.21.15.172:0/1569963843 learned_addr learned my addr 172.21.15.172:0/1569963843 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.860+0000 7f478bfff640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f478c162700 0x7f478c15c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.860+0000 7f478b7fe640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f478c0d1840 0x7f478c157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.860+0000 7f478affd640 1 -- 172.21.15.172:0/1569963843 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f478c162700 msgr2=0x7f478c15c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.861+0000 7f478affd640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f478c162700 0x7f478c15c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.861+0000 7f478affd640 1 -- 172.21.15.172:0/1569963843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f478c0d1840 msgr2=0x7f478c157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.861+0000 7f478affd640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f478c0d1840 0x7f478c157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:04.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.861+0000 7f478affd640 1 -- 172.21.15.172:0/1569963843 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478c157e50 con 0x7f478c15c150 2024-04-14T08:34:04.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.861+0000 7f478affd640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f478c15c150 0x7f478c1495f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f478000c4b0 tx=0x7f478000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:04.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.861+0000 7f4773fff640 1 -- 172.21.15.172:0/1569963843 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780018070 con 0x7f478c15c150 2024-04-14T08:34:04.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.861+0000 7f4773fff640 1 -- 172.21.15.172:0/1569963843 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4780007480 con 0x7f478c15c150 2024-04-14T08:34:04.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.861+0000 7f4791dfe640 1 -- 172.21.15.172:0/1569963843 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478c158130 con 0x7f478c15c150 2024-04-14T08:34:04.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.861+0000 7f4773fff640 1 -- 172.21.15.172:0/1569963843 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780004820 con 0x7f478c15c150 2024-04-14T08:34:04.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.861+0000 7f4791dfe640 1 -- 172.21.15.172:0/1569963843 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f478c1586f0 con 0x7f478c15c150 2024-04-14T08:34:04.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.862+0000 7f4791dfe640 1 -- 172.21.15.172:0/1569963843 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f478c0d49c0 con 0x7f478c15c150 2024-04-14T08:34:04.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.862+0000 7f4773fff640 1 -- 172.21.15.172:0/1569963843 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f478001b070 con 0x7f478c15c150 2024-04-14T08:34:04.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.863+0000 7f4773fff640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f477806c920 0x7f477806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:04.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.863+0000 7f478b7fe640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f477806c920 0x7f477806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:04.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.863+0000 7f4773fff640 1 -- 172.21.15.172:0/1569963843 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f478008afe0 con 0x7f478c15c150 2024-04-14T08:34:04.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.863+0000 7f478b7fe640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f477806c920 0x7f477806ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f477c036000 tx=0x7f477c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:04.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:04.867+0000 7f4773fff640 1 -- 172.21.15.172:0/1569963843 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f47800592e0 con 0x7f478c15c150 2024-04-14T08:34:04.977 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:34:04.976+0000 7fc7b2ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:34:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.087+0000 7f4791dfe640 1 -- 172.21.15.172:0/1569963843 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f478c0d1430 con 0x7f478c15c150 2024-04-14T08:34:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.087+0000 7f4773fff640 1 -- 172.21.15.172:0/1569963843 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f4780058be0 con 0x7f478c15c150 2024-04-14T08:34:05.090 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:34:05.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.089+0000 7f4791dfe640 1 -- 172.21.15.172:0/1569963843 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f477806c920 msgr2=0x7f477806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.089+0000 7f4791dfe640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f477806c920 0x7f477806ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f477c036000 tx=0x7f477c008030 comp rx=0 tx=0).stop 2024-04-14T08:34:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.089+0000 7f4791dfe640 1 -- 172.21.15.172:0/1569963843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f478c15c150 msgr2=0x7f478c1495f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.089+0000 7f4791dfe640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f478c15c150 0x7f478c1495f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f478000c4b0 tx=0x7f478000c970 comp rx=0 tx=0).stop 2024-04-14T08:34:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.089+0000 7f4791dfe640 1 -- 172.21.15.172:0/1569963843 shutdown_connections 2024-04-14T08:34:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.089+0000 7f4791dfe640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f477806c920 0x7f477806ede0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.089+0000 7f4791dfe640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f478c162700 0x7f478c15c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:05.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.089+0000 7f4791dfe640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f478c15c150 0x7f478c1495f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:05.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.089+0000 7f4791dfe640 1 --2- 172.21.15.172:0/1569963843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f478c0d1840 0x7f478c157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:05.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.089+0000 7f4791dfe640 1 -- 172.21.15.172:0/1569963843 >> 172.21.15.172:0/1569963843 conn(0x7f478c09d580 msgr2=0x7f478c06d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:05.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.089+0000 7f4791dfe640 1 -- 172.21.15.172:0/1569963843 shutdown_connections 2024-04-14T08:34:05.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05.090+0000 7f4791dfe640 1 -- 172.21.15.172:0/1569963843 wait complete. 2024-04-14T08:34:05.102 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:34:05.102 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:34:05.102 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:34:05.102 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:34:05.102 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:34:05.102 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:34:05.102 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:34:05.102 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:34:05.102 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:34:05.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:05.103 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:34:05.118 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:34:05.118 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:34:05.118 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:34:05.118 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:05.118 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:34:05.118 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:05.118 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:34:05.118 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:34:05.118 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:34:05.118 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42624sec preferred_lft 42624sec 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:05.119 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:34:05.120 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:05.120 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:34:05.120 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:34:05.120 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:34:05.120 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:05.120 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:34:05.120 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:34:05.120 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:34:05.120 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:05.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:05.121 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:34:05.185 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:34:05.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:05.186 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:34:05.241 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:34:05.242 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:34:05.242 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:34:05.242 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:34:05.242 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:34:05.242 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:34:05.242 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:34:05.242 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:34:05.242 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:34:05.243 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:34:05.243 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:34:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:34:05.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:34:05.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:05.380 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:34:05.395 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:34:05.395 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:34:05.395 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:34:05.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:05.395 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:34:05.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:34:05.463 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:34:05.518 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:34:05.518 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:34:05.518 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:34:05.573 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:34:05.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:05.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:34:05.658 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:34:05.658 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:34:05.658 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:34:05.666 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:34:05.666 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:34:05.666 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:34:05.667 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:34:05.667 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:34:05.675 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:34:05.675 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:34:05.675 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:34:05.676 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:34:05.676 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:34:05.676 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:34:05.676 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:34:05.676 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:34:05.676 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:34:05.676 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:34:05.676 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:34:05.677 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:34:05.677 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:34:05.678 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:34:05.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:05.678 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:34:05.706 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:34:05.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:34:05.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:34:05.806 INFO:teuthology.orchestra.run.smithi172.stdout:client5493 2024-04-14T08:34:05.806 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:34:05.806 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:34:05.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:05.822 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-14T08:34:05.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:05.880 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-14T08:34:05.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:05.936 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-14T08:34:05.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:05.993 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-14T08:34:06.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:06.053 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-14T08:34:06.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:06.110 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-14T08:34:06.169 INFO:teuthology.orchestra.run.smithi172.stdout:./ 2024-04-14T08:34:06.169 INFO:teuthology.orchestra.run.smithi172.stdout:./rfile 2024-04-14T08:34:06.169 INFO:teuthology.orchestra.run.smithi172.stdout:./parentdir 2024-04-14T08:34:06.169 INFO:teuthology.orchestra.run.smithi172.stdout:./parentdir/pfile 2024-04-14T08:34:06.169 INFO:teuthology.orchestra.run.smithi172.stdout:./parentdir/childdir 2024-04-14T08:34:06.169 INFO:teuthology.orchestra.run.smithi172.stdout:./parentdir/childdir/cfile 2024-04-14T08:34:06.170 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:34:06.170 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:34:06.170 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:34:06.170 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:34:06.170 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-14T08:34:06.170 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:34:06.274 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:34:06.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:34:06.283 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:34:06.283 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:34:06.283 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:34:06.283 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:34:06.284 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-14T08:34:06.284 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:34:06.408 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627778 2024-04-14T08:34:06.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:34:06.419 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:34:06.419 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:34:06.419 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:34:06.419 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:34:06.419 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-14T08:34:06.419 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:34:06.527 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627776 2024-04-14T08:34:06.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:34:06.536 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:34:06.536 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:34:06.536 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:34:06.536 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:34:06.536 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-14T08:34:06.536 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:34:06.641 INFO:teuthology.orchestra.run.smithi172.stdout:1099511628279 2024-04-14T08:34:06.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:34:06.651 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:34:06.651 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:34:06.651 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:34:06.651 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:34:06.651 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-14T08:34:06.651 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:34:06.755 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627777 2024-04-14T08:34:06.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:34:06.765 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:34:06.765 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:34:06.765 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:34:06.765 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:34:06.765 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-14T08:34:06.765 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:34:06.870 INFO:teuthology.orchestra.run.smithi172.stdout:1099511628280 2024-04-14T08:34:06.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:34:06.879 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:34:06.879 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:34:06.935 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:34:06.936 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:34:06.991 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:34:07.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.135+0000 7fa4dc3aa640 1 Processor -- start 2024-04-14T08:34:07.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.135+0000 7fa4dc3aa640 1 -- start start 2024-04-14T08:34:07.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.135+0000 7fa4dc3aa640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa4d40b1df0 0x7fa4d40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4dc3aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4d40b4040 0x7fa4d40b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4dc3aa640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4d40b49f0 con 0x7fa4d40b21d0 2024-04-14T08:34:07.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4dc3aa640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa4d415c770 con 0x7fa4d40b1df0 2024-04-14T08:34:07.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4dc3aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d40b3430 con 0x7fa4d40b4040 2024-04-14T08:34:07.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9945640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa4d40b21d0 0x7fa4d415c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48340/0 (socket says 172.21.15.172:48340) 2024-04-14T08:34:07.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9945640 1 -- 172.21.15.172:0/2543681268 learned_addr learned my addr 172.21.15.172:0/2543681268 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:07.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3524754167 0 0) 0x7fa4d40b49f0 con 0x7fa4d40b21d0 2024-04-14T08:34:07.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4da947640 1 --2- 172.21.15.172:0/2543681268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4d40b4040 0x7fa4d40b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4c0003520 con 0x7fa4d40b21d0 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4da146640 1 --2- 172.21.15.172:0/2543681268 >> v2:172.21.15.177:3300/0 conn(0x7fa4d40b1df0 0x7fa4d40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 350069985 0 0) 0x7fa4c0003520 con 0x7fa4d40b21d0 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4d40b4040 msgr2=0x7fa4d40b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 --2- 172.21.15.172:0/2543681268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4d40b4040 0x7fa4d40b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 >> v2:172.21.15.177:3300/0 conn(0x7fa4d40b1df0 msgr2=0x7fa4d40b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 --2- 172.21.15.172:0/2543681268 >> v2:172.21.15.177:3300/0 conn(0x7fa4d40b1df0 0x7fa4d40b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d40b3700 con 0x7fa4d40b21d0 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fa4c4001670 con 0x7fa4d40b21d0 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 >> v1:172.21.15.172:6789/0 conn(0x7fa4d40b21d0 legacy=0x7fa4d415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 --2- 172.21.15.172:0/2543681268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4d40b4040 0x7fa4c00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 --2- 172.21.15.172:0/2543681268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4c0003f30 0x7fa4c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 --2- 172.21.15.172:0/2543681268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4c0004d40 0x7fa4c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d415c770 con 0x7fa4c0003f30 2024-04-14T08:34:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d40b3430 con 0x7fa4c0004d40 2024-04-14T08:34:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.136+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c000b9c0 con 0x7fa4d40b4040 2024-04-14T08:34:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.137+0000 7fa4d9945640 1 --2- 172.21.15.172:0/2543681268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4c0003f30 0x7fa4c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.137+0000 7fa4da947640 1 --2- 172.21.15.172:0/2543681268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4c0004d40 0x7fa4c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.137+0000 7fa4da146640 1 --2- 172.21.15.172:0/2543681268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4d40b4040 0x7fa4c00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.137+0000 7fa4d9945640 1 -- 172.21.15.172:0/2543681268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4d40b4040 msgr2=0x7fa4c00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.137+0000 7fa4d9945640 1 --2- 172.21.15.172:0/2543681268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4d40b4040 0x7fa4c00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.137+0000 7fa4d9945640 1 -- 172.21.15.172:0/2543681268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4c0004d40 msgr2=0x7fa4c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.137+0000 7fa4d9945640 1 --2- 172.21.15.172:0/2543681268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4c0004d40 0x7fa4c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.137+0000 7fa4d9945640 1 -- 172.21.15.172:0/2543681268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c0003520 con 0x7fa4c0003f30 2024-04-14T08:34:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.138+0000 7fa4d9945640 1 --2- 172.21.15.172:0/2543681268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4c0003f30 0x7fa4c0004800 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40b2680 tx=0x7fa4c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a00f788a2f31557 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.138+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4007d50 con 0x7fa4c0003f30 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.138+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c4030070 con 0x7fa4c0003f30 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.138+0000 7fa4d9144640 1 -- 172.21.15.172:0/2543681268 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4005d10 con 0x7fa4c0003f30 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.138+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/2543681268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4c0003f30 msgr2=0x7fa4c0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.138+0000 7fa4dc3aa640 1 --2- 172.21.15.172:0/2543681268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4c0003f30 0x7fa4c0004800 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40b2680 tx=0x7fa4c0005830 comp rx=0 tx=0).stop 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.138+0000 7fa4da947640 1 -- 172.21.15.172:0/2543681268 reap_dead start 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.139+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/2543681268 shutdown_connections 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.139+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/2543681268 >> 172.21.15.172:0/2543681268 conn(0x7fa4d406ec30 msgr2=0x7fa4d406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.139+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/2543681268 shutdown_connections 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.139+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/2543681268 wait complete. 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.140+0000 7fa4dc3aa640 1 Processor -- start 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.140+0000 7fa4dc3aa640 1 -- start start 2024-04-14T08:34:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4dc3aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4c0003f30 0x7fa4d4157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4dc3aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4d40aa520 0x7fa4d40ab410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4dc3aa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4d413e4f0 0x7fa4d413e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4dc3aa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d415bb40 con 0x7fa4d40aa520 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4dc3aa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d40b3cc0 con 0x7fa4d413e4f0 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4dc3aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d415b7f0 con 0x7fa4c0003f30 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4d9945640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4d40aa520 0x7fa4d40ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4d9945640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4d40aa520 0x7fa4d40ab410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48854/0 (socket says 172.21.15.172:48854) 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4d9945640 1 -- 172.21.15.172:0/3338459334 learned_addr learned my addr 172.21.15.172:0/3338459334 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4da146640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4c0003f30 0x7fa4d4157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4da947640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4d413e4f0 0x7fa4d413e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4d9945640 1 -- 172.21.15.172:0/3338459334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4c0003f30 msgr2=0x7fa4d4157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4d9945640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4c0003f30 0x7fa4d4157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4d9945640 1 -- 172.21.15.172:0/3338459334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4d413e4f0 msgr2=0x7fa4d413e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4d9945640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4d413e4f0 0x7fa4d413e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4d9945640 1 -- 172.21.15.172:0/3338459334 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d41582a0 con 0x7fa4d40aa520 2024-04-14T08:34:07.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4da947640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4d413e4f0 0x7fa4d413e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:07.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4d9945640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4d40aa520 0x7fa4d40ab410 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400c2d0 tx=0x7fa4d40b38d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:07.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4caffd640 1 -- 172.21.15.172:0/3338459334 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4042070 con 0x7fa4d40aa520 2024-04-14T08:34:07.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4caffd640 1 -- 172.21.15.172:0/3338459334 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c402de50 con 0x7fa4d40aa520 2024-04-14T08:34:07.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/3338459334 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d40abcb0 con 0x7fa4d40aa520 2024-04-14T08:34:07.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4caffd640 1 -- 172.21.15.172:0/3338459334 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4004e30 con 0x7fa4d40aa520 2024-04-14T08:34:07.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.141+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/3338459334 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d40ad2e0 con 0x7fa4d40aa520 2024-04-14T08:34:07.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.142+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/3338459334 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d40acdf0 con 0x7fa4d40aa520 2024-04-14T08:34:07.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.143+0000 7fa4caffd640 1 -- 172.21.15.172:0/3338459334 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4c4045070 con 0x7fa4d40aa520 2024-04-14T08:34:07.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.143+0000 7fa4caffd640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa4b806c920 0x7fa4b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.143+0000 7fa4caffd640 1 -- 172.21.15.172:0/3338459334 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa4c40af3f0 con 0x7fa4d40aa520 2024-04-14T08:34:07.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.143+0000 7fa4da146640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa4b806c920 0x7fa4b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.144+0000 7fa4da146640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa4b806c920 0x7fa4b806ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009bc0 tx=0x7fa4d0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.148+0000 7fa4caffd640 1 -- 172.21.15.172:0/3338459334 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa4c404c020 con 0x7fa4d40aa520 2024-04-14T08:34:07.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.335+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/3338459334 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4d4068520 con 0x7fa4d40aa520 2024-04-14T08:34:07.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.336+0000 7fa4caffd640 1 -- 172.21.15.172:0/3338459334 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17002 (secure 0 0 0) 0x7fa4c407d830 con 0x7fa4d40aa520 2024-04-14T08:34:07.338 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:07.341 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:00.984153+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:33:57.868995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-14T08:33:58.323549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:34:07.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.338+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/3338459334 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa4b806c920 msgr2=0x7fa4b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.338+0000 7fa4dc3aa640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa4b806c920 0x7fa4b806ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009bc0 tx=0x7fa4d0009190 comp rx=0 tx=0).stop 2024-04-14T08:34:07.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.338+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/3338459334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4d40aa520 msgr2=0x7fa4d40ab410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.338+0000 7fa4dc3aa640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4d40aa520 0x7fa4d40ab410 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400c2d0 tx=0x7fa4d40b38d0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.338+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/3338459334 shutdown_connections 2024-04-14T08:34:07.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.338+0000 7fa4dc3aa640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa4b806c920 0x7fa4b806ede0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.338+0000 7fa4dc3aa640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4d413e4f0 0x7fa4d413e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.338+0000 7fa4dc3aa640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4d40aa520 0x7fa4d40ab410 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.338+0000 7fa4dc3aa640 1 --2- 172.21.15.172:0/3338459334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4c0003f30 0x7fa4d4157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.338+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/3338459334 >> 172.21.15.172:0/3338459334 conn(0x7fa4d406ec30 msgr2=0x7fa4d40cf770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:07.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.339+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/3338459334 shutdown_connections 2024-04-14T08:34:07.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.339+0000 7fa4dc3aa640 1 -- 172.21.15.172:0/3338459334 wait complete. 2024-04-14T08:34:07.351 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:33:57.868995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:33:58.323549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:34:07.351 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:34:07.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.454+0000 7f93a920f640 1 Processor -- start 2024-04-14T08:34:07.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.454+0000 7f93a920f640 1 -- start start 2024-04-14T08:34:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.455+0000 7f93a920f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93a4162ba0 0x7f93a40d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.455+0000 7f93a920f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93a40cefb0 con 0x7f93a4162710 2024-04-14T08:34:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.455+0000 7f93a920f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93a40cf290 con 0x7f93a415e430 2024-04-14T08:34:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.455+0000 7f93a920f640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f93a40cf570 con 0x7f93a4162ba0 2024-04-14T08:34:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.455+0000 7f93a359e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93a4162ba0 0x7f93a40d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.455+0000 7f93a259c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f93a4162710 0x7f93a40d1df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48346/0 (socket says 172.21.15.172:48346) 2024-04-14T08:34:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.455+0000 7f93a259c640 1 -- 172.21.15.172:0/2389010479 learned_addr learned my addr 172.21.15.172:0/2389010479 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.456+0000 7f93a359e640 1 -- 172.21.15.172:0/2389010479 >> v1:172.21.15.177:6789/0 conn(0x7f93a415e430 legacy=0x7f93a4162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.456+0000 7f93a359e640 1 -- 172.21.15.172:0/2389010479 >> v1:172.21.15.172:6789/0 conn(0x7f93a4162710 legacy=0x7f93a40d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.456+0000 7f93a359e640 1 -- 172.21.15.172:0/2389010479 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a40cf850 con 0x7f93a4162ba0 2024-04-14T08:34:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.456+0000 7f93a1d9b640 1 -- 172.21.15.172:0/2389010479 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4114233204 0 0) 0x7f93a40cefb0 con 0x7f93a4162710 2024-04-14T08:34:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.457+0000 7f93a359e640 1 --2- 172.21.15.172:0/2389010479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a4162ba0 0x7f93a40d2500 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f939c009d90 tx=0x7f939c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c996abac5f938f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.457+0000 7f93a1d9b640 1 -- 172.21.15.172:0/2389010479 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c039070 con 0x7f93a4162ba0 2024-04-14T08:34:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.457+0000 7f93a1d9b640 1 -- 172.21.15.172:0/2389010479 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939c02cc60 con 0x7f93a4162ba0 2024-04-14T08:34:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.457+0000 7f93a1d9b640 1 -- 172.21.15.172:0/2389010479 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c034660 con 0x7f93a4162ba0 2024-04-14T08:34:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.457+0000 7f93a920f640 1 -- 172.21.15.172:0/2389010479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a4162ba0 msgr2=0x7f93a40d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.457+0000 7f93a920f640 1 --2- 172.21.15.172:0/2389010479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a4162ba0 0x7f93a40d2500 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f939c009d90 tx=0x7f939c02beb0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.457+0000 7f93a920f640 1 -- 172.21.15.172:0/2389010479 shutdown_connections 2024-04-14T08:34:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.457+0000 7f93a920f640 1 --2- 172.21.15.172:0/2389010479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a4162ba0 0x7f93a40d2500 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.457+0000 7f93a920f640 1 -- 172.21.15.172:0/2389010479 >> 172.21.15.172:0/2389010479 conn(0x7f93a409d560 msgr2=0x7f93a40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.457+0000 7f93a920f640 1 -- 172.21.15.172:0/2389010479 shutdown_connections 2024-04-14T08:34:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.457+0000 7f93a920f640 1 -- 172.21.15.172:0/2389010479 wait complete. 2024-04-14T08:34:07.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.459+0000 7f93a920f640 1 Processor -- start 2024-04-14T08:34:07.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.459+0000 7f93a920f640 1 -- start start 2024-04-14T08:34:07.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a920f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93a415e430 0x7f93a414f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a920f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a4162710 0x7f93a40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a920f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93a4162ba0 0x7f93a40b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a920f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93a40d4e20 con 0x7f93a4162710 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a920f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f93a415e810 con 0x7f93a4162ba0 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a920f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f93a40d0050 con 0x7f93a415e430 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a4162710 0x7f93a40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a4162710 0x7f93a40cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48872/0 (socket says 172.21.15.172:48872) 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a259c640 1 -- 172.21.15.172:0/431418765 learned_addr learned my addr 172.21.15.172:0/431418765 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a359e640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93a4162ba0 0x7f93a40b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a2d9d640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93a415e430 0x7f93a414f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a259c640 1 -- 172.21.15.172:0/431418765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93a415e430 msgr2=0x7f93a414f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a259c640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93a415e430 0x7f93a414f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a259c640 1 -- 172.21.15.172:0/431418765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93a4162ba0 msgr2=0x7f93a40b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a259c640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93a4162ba0 0x7f93a40b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a259c640 1 -- 172.21.15.172:0/431418765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a40cc530 con 0x7f93a4162710 2024-04-14T08:34:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a259c640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a4162710 0x7f93a40cd2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f939400c430 tx=0x7f939400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f938b7fe640 1 -- 172.21.15.172:0/431418765 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93940074e0 con 0x7f93a4162710 2024-04-14T08:34:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f938b7fe640 1 -- 172.21.15.172:0/431418765 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93940047e0 con 0x7f93a4162710 2024-04-14T08:34:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a920f640 1 -- 172.21.15.172:0/431418765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a40cc810 con 0x7f93a4162710 2024-04-14T08:34:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f938b7fe640 1 -- 172.21.15.172:0/431418765 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394004bd0 con 0x7f93a4162710 2024-04-14T08:34:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.460+0000 7f93a920f640 1 -- 172.21.15.172:0/431418765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a40ccdd0 con 0x7f93a4162710 2024-04-14T08:34:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.461+0000 7f93a920f640 1 -- 172.21.15.172:0/431418765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a4068590 con 0x7f93a4162710 2024-04-14T08:34:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.462+0000 7f938b7fe640 1 -- 172.21.15.172:0/431418765 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9394011070 con 0x7f93a4162710 2024-04-14T08:34:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.462+0000 7f938b7fe640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f939006c8f0 0x7f939006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.462+0000 7f93a2d9d640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f939006c8f0 0x7f939006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.462+0000 7f938b7fe640 1 -- 172.21.15.172:0/431418765 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f939408b930 con 0x7f93a4162710 2024-04-14T08:34:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.463+0000 7f93a2d9d640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f939006c8f0 0x7f939006edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9398006fd0 tx=0x7f9398008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:07.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.467+0000 7f938b7fe640 1 -- 172.21.15.172:0/431418765 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9394059c60 con 0x7f93a4162710 2024-04-14T08:34:07.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.691+0000 7f93a920f640 1 -- 172.21.15.172:0/431418765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f93a40d18a0 con 0x7f93a4162710 2024-04-14T08:34:07.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.692+0000 7f938b7fe640 1 -- 172.21.15.172:0/431418765 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7f9394059560 con 0x7f93a4162710 2024-04-14T08:34:07.694 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:34:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.694+0000 7f93a920f640 1 -- 172.21.15.172:0/431418765 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f939006c8f0 msgr2=0x7f939006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.694+0000 7f93a920f640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f939006c8f0 0x7f939006edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9398006fd0 tx=0x7f9398008040 comp rx=0 tx=0).stop 2024-04-14T08:34:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.694+0000 7f93a920f640 1 -- 172.21.15.172:0/431418765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a4162710 msgr2=0x7f93a40cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.694+0000 7f93a920f640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a4162710 0x7f93a40cd2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f939400c430 tx=0x7f939400c8f0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.694+0000 7f93a920f640 1 -- 172.21.15.172:0/431418765 shutdown_connections 2024-04-14T08:34:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.694+0000 7f93a920f640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f939006c8f0 0x7f939006edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.694+0000 7f93a920f640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93a4162ba0 0x7f93a40b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.694+0000 7f93a920f640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a4162710 0x7f93a40cd2f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.694+0000 7f93a920f640 1 --2- 172.21.15.172:0/431418765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93a415e430 0x7f93a414f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.694+0000 7f93a920f640 1 -- 172.21.15.172:0/431418765 >> 172.21.15.172:0/431418765 conn(0x7f93a409d560 msgr2=0x7f93a40b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.694+0000 7f93a920f640 1 -- 172.21.15.172:0/431418765 shutdown_connections 2024-04-14T08:34:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.694+0000 7f93a920f640 1 -- 172.21.15.172:0/431418765 wait complete. 2024-04-14T08:34:07.706 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:34:07.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.811+0000 7f656602d640 1 Processor -- start 2024-04-14T08:34:07.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.811+0000 7f656602d640 1 -- start start 2024-04-14T08:34:07.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f656602d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f65600d1a20 0x7f656006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f656602d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f656006d000 0x7f656006d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f656602d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65600701b0 con 0x7f6560071250 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f656602d640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f656006d9d0 con 0x7f65600d1a20 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f656602d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6560070dc0 con 0x7f656006d000 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f655f7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f65600d1a20 0x7f656006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f655effd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6560071250 0x7f65600cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48354/0 (socket says 172.21.15.172:48354) 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f655effd640 1 -- 172.21.15.172:0/733431320 learned_addr learned my addr 172.21.15.172:0/733431320 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f655ffff640 1 --2- 172.21.15.172:0/733431320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f656006d000 0x7f656006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f655f7fe640 1 -- 172.21.15.172:0/733431320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f656006d000 msgr2=0x7f656006d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f655f7fe640 1 --2- 172.21.15.172:0/733431320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f656006d000 0x7f656006d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f655f7fe640 1 -- 172.21.15.172:0/733431320 >> v1:172.21.15.172:6789/0 conn(0x7f6560071250 legacy=0x7f65600cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f655f7fe640 1 -- 172.21.15.172:0/733431320 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6560070490 con 0x7f65600d1a20 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.812+0000 7f655e7fc640 1 -- 172.21.15.172:0/733431320 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525558680 0 0) 0x7f65600701b0 con 0x7f6560071250 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.813+0000 7f655f7fe640 1 --2- 172.21.15.172:0/733431320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65600d1a20 0x7f656006fb80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f65540097a0 tx=0x7f655402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af243ac42aa3940 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:07.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.813+0000 7f655e7fc640 1 -- 172.21.15.172:0/733431320 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540315a0 con 0x7f65600d1a20 2024-04-14T08:34:07.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.813+0000 7f655e7fc640 1 -- 172.21.15.172:0/733431320 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6554037070 con 0x7f65600d1a20 2024-04-14T08:34:07.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.813+0000 7f655e7fc640 1 -- 172.21.15.172:0/733431320 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554031960 con 0x7f65600d1a20 2024-04-14T08:34:07.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.814+0000 7f656602d640 1 -- 172.21.15.172:0/733431320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65600d1a20 msgr2=0x7f656006fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.814+0000 7f656602d640 1 --2- 172.21.15.172:0/733431320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65600d1a20 0x7f656006fb80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f65540097a0 tx=0x7f655402b840 comp rx=0 tx=0).stop 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.814+0000 7f656602d640 1 -- 172.21.15.172:0/733431320 shutdown_connections 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.814+0000 7f656602d640 1 --2- 172.21.15.172:0/733431320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f656006d000 0x7f656006d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.814+0000 7f656602d640 1 --2- 172.21.15.172:0/733431320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65600d1a20 0x7f656006fb80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.814+0000 7f656602d640 1 -- 172.21.15.172:0/733431320 >> 172.21.15.172:0/733431320 conn(0x7f656008c000 msgr2=0x7f65600d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.814+0000 7f656602d640 1 -- 172.21.15.172:0/733431320 shutdown_connections 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.814+0000 7f656602d640 1 -- 172.21.15.172:0/733431320 wait complete. 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.815+0000 7f656602d640 1 Processor -- start 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.815+0000 7f656602d640 1 -- start start 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f656602d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f656006d000 0x7f6560158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f656602d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6560071250 0x7f65600cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f656602d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65600d1a20 0x7f65600cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f656602d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f65600d1e00 con 0x7f6560071250 2024-04-14T08:34:07.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f656602d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6560070a50 con 0x7f65600d1a20 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f656602d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f65600c8bf0 con 0x7f656006d000 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f655effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6560071250 0x7f65600cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f655effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6560071250 0x7f65600cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48894/0 (socket says 172.21.15.172:48894) 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f655effd640 1 -- 172.21.15.172:0/2316065209 learned_addr learned my addr 172.21.15.172:0/2316065209 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f655f7fe640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f656006d000 0x7f6560158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f655ffff640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65600d1a20 0x7f65600cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f655effd640 1 -- 172.21.15.172:0/2316065209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f656006d000 msgr2=0x7f6560158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f655effd640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f656006d000 0x7f6560158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f655effd640 1 -- 172.21.15.172:0/2316065209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65600d1a20 msgr2=0x7f65600cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f655effd640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65600d1a20 0x7f65600cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f655effd640 1 -- 172.21.15.172:0/2316065209 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65600c8000 con 0x7f6560071250 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.816+0000 7f655effd640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6560071250 0x7f65600cb300 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f655000c4e0 tx=0x7f655000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.817+0000 7f6547fff640 1 -- 172.21.15.172:0/2316065209 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550018070 con 0x7f6560071250 2024-04-14T08:34:07.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.817+0000 7f6547fff640 1 -- 172.21.15.172:0/2316065209 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6550007500 con 0x7f6560071250 2024-04-14T08:34:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.817+0000 7f6547fff640 1 -- 172.21.15.172:0/2316065209 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550004a20 con 0x7f6560071250 2024-04-14T08:34:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.817+0000 7f656602d640 1 -- 172.21.15.172:0/2316065209 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65600c82c0 con 0x7f6560071250 2024-04-14T08:34:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.817+0000 7f656602d640 1 -- 172.21.15.172:0/2316065209 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65600c8880 con 0x7f6560071250 2024-04-14T08:34:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.818+0000 7f656602d640 1 -- 172.21.15.172:0/2316065209 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6560075490 con 0x7f6560071250 2024-04-14T08:34:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.818+0000 7f6547fff640 1 -- 172.21.15.172:0/2316065209 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f655001b070 con 0x7f6560071250 2024-04-14T08:34:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.819+0000 7f6547fff640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f654c06c8f0 0x7f654c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.819+0000 7f655f7fe640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f654c06c8f0 0x7f654c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.819+0000 7f6547fff640 1 -- 172.21.15.172:0/2316065209 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f655008afa0 con 0x7f6560071250 2024-04-14T08:34:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.819+0000 7f655f7fe640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f654c06c8f0 0x7f654c06edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6554036000 tx=0x7f6554008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:07.823+0000 7f6547fff640 1 -- 172.21.15.172:0/2316065209 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6550058c20 con 0x7f6560071250 2024-04-14T08:34:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.071+0000 7f656602d640 1 -- 172.21.15.172:0/2316065209 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f65600d2820 con 0x7f6560071250 2024-04-14T08:34:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.071+0000 7f6547fff640 1 -- 172.21.15.172:0/2316065209 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) ==== 104+0+11 (secure 0 0 0) 0x7f655005cf20 con 0x7f6560071250 2024-04-14T08:34:08.073 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:34:08.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.074+0000 7f656602d640 1 -- 172.21.15.172:0/2316065209 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f654c06c8f0 msgr2=0x7f654c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.074+0000 7f656602d640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f654c06c8f0 0x7f654c06edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6554036000 tx=0x7f6554008030 comp rx=0 tx=0).stop 2024-04-14T08:34:08.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.074+0000 7f656602d640 1 -- 172.21.15.172:0/2316065209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6560071250 msgr2=0x7f65600cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.074+0000 7f656602d640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6560071250 0x7f65600cb300 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f655000c4e0 tx=0x7f655000c9a0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.074+0000 7f656602d640 1 -- 172.21.15.172:0/2316065209 shutdown_connections 2024-04-14T08:34:08.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.074+0000 7f656602d640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f654c06c8f0 0x7f654c06edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.074+0000 7f656602d640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65600d1a20 0x7f65600cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.074+0000 7f656602d640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6560071250 0x7f65600cb300 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.074+0000 7f656602d640 1 --2- 172.21.15.172:0/2316065209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f656006d000 0x7f6560158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.074+0000 7f656602d640 1 -- 172.21.15.172:0/2316065209 >> 172.21.15.172:0/2316065209 conn(0x7f656008c000 msgr2=0x7f65600ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:08.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.074+0000 7f656602d640 1 -- 172.21.15.172:0/2316065209 shutdown_connections 2024-04-14T08:34:08.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.074+0000 7f656602d640 1 -- 172.21.15.172:0/2316065209 wait complete. 2024-04-14T08:34:08.086 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:34:08.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.204+0000 7f4eaf98c640 1 Processor -- start 2024-04-14T08:34:08.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.204+0000 7f4eaf98c640 1 -- start start 2024-04-14T08:34:08.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4eaf98c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4ea815e430 0x7f4ea8162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4eaf98c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4ea8162540 0x7f4ea81629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4eaf98c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea80cefb0 con 0x7f4ea80b3a30 2024-04-14T08:34:08.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4eaf98c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea80cf290 con 0x7f4ea815e430 2024-04-14T08:34:08.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4eaf98c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea80cf570 con 0x7f4ea8162540 2024-04-14T08:34:08.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4ead728640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4ea815e430 0x7f4ea8162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4eadf29640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4ea80b3a30 0x7f4ea80b3e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48360/0 (socket says 172.21.15.172:48360) 2024-04-14T08:34:08.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4eadf29640 1 -- 172.21.15.172:0/2810788562 learned_addr learned my addr 172.21.15.172:0/2810788562 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:08.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4eacf27640 1 --2- 172.21.15.172:0/2810788562 >> v2:172.21.15.177:3300/0 conn(0x7f4ea8162540 0x7f4ea81629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4ead728640 1 -- 172.21.15.172:0/2810788562 >> v2:172.21.15.177:3300/0 conn(0x7f4ea8162540 msgr2=0x7f4ea81629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:08.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4ead728640 1 --2- 172.21.15.172:0/2810788562 >> v2:172.21.15.177:3300/0 conn(0x7f4ea8162540 0x7f4ea81629d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4ead728640 1 -- 172.21.15.172:0/2810788562 >> v1:172.21.15.172:6789/0 conn(0x7f4ea80b3a30 legacy=0x7f4ea80b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4ead728640 1 -- 172.21.15.172:0/2810788562 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea80cf850 con 0x7f4ea815e430 2024-04-14T08:34:08.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.205+0000 7f4eacf27640 1 --2- 172.21.15.172:0/2810788562 >> v2:172.21.15.177:3300/0 conn(0x7f4ea8162540 0x7f4ea81629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:08.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.206+0000 7f4ead728640 1 --2- 172.21.15.172:0/2810788562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ea815e430 0x7f4ea8162000 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4ea808dae0 tx=0x7f4ea402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6b842b91b82d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:08.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.206+0000 7f4e9ffff640 1 -- 172.21.15.172:0/2810788562 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4035020 con 0x7f4ea815e430 2024-04-14T08:34:08.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.206+0000 7f4e9ffff640 1 -- 172.21.15.172:0/2810788562 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea4031830 con 0x7f4ea815e430 2024-04-14T08:34:08.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.206+0000 7f4e9ffff640 1 -- 172.21.15.172:0/2810788562 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4031c00 con 0x7f4ea815e430 2024-04-14T08:34:08.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.207+0000 7f4eaf98c640 1 -- 172.21.15.172:0/2810788562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ea815e430 msgr2=0x7f4ea8162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.207+0000 7f4eaf98c640 1 --2- 172.21.15.172:0/2810788562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ea815e430 0x7f4ea8162000 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4ea808dae0 tx=0x7f4ea402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.207+0000 7f4eaf98c640 1 -- 172.21.15.172:0/2810788562 shutdown_connections 2024-04-14T08:34:08.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.207+0000 7f4eaf98c640 1 --2- 172.21.15.172:0/2810788562 >> v2:172.21.15.177:3300/0 conn(0x7f4ea8162540 0x7f4ea81629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.207+0000 7f4eaf98c640 1 --2- 172.21.15.172:0/2810788562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ea815e430 0x7f4ea8162000 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.207+0000 7f4eaf98c640 1 -- 172.21.15.172:0/2810788562 >> 172.21.15.172:0/2810788562 conn(0x7f4ea809d560 msgr2=0x7f4ea80d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:08.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.207+0000 7f4eaf98c640 1 -- 172.21.15.172:0/2810788562 shutdown_connections 2024-04-14T08:34:08.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.207+0000 7f4eaf98c640 1 -- 172.21.15.172:0/2810788562 wait complete. 2024-04-14T08:34:08.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.208+0000 7f4eaf98c640 1 Processor -- start 2024-04-14T08:34:08.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.208+0000 7f4eaf98c640 1 -- start start 2024-04-14T08:34:08.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eaf98c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ea80b3a30 0x7f4ea80b97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eaf98c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ea815e430 0x7f4ea8144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eaf98c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ea8162540 0x7f4ea8158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eaf98c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea80d4e20 con 0x7f4ea815e430 2024-04-14T08:34:08.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eaf98c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea815e810 con 0x7f4ea80b3a30 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eaf98c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea80d0050 con 0x7f4ea8162540 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eacf27640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ea815e430 0x7f4ea8144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eacf27640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ea815e430 0x7f4ea8144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48920/0 (socket says 172.21.15.172:48920) 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eacf27640 1 -- 172.21.15.172:0/3651722042 learned_addr learned my addr 172.21.15.172:0/3651722042 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4ead728640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ea80b3a30 0x7f4ea80b97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eadf29640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ea8162540 0x7f4ea8158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eacf27640 1 -- 172.21.15.172:0/3651722042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ea8162540 msgr2=0x7f4ea8158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eacf27640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ea8162540 0x7f4ea8158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eacf27640 1 -- 172.21.15.172:0/3651722042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ea80b3a30 msgr2=0x7f4ea80b97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eacf27640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ea80b3a30 0x7f4ea80b97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eacf27640 1 -- 172.21.15.172:0/3651722042 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea80b0e50 con 0x7f4ea815e430 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eacf27640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ea815e430 0x7f4ea8144ff0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800c7f0 tx=0x7f4e9800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:08.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4e9dffb640 1 -- 172.21.15.172:0/3651722042 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98015070 con 0x7f4ea815e430 2024-04-14T08:34:08.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4e9dffb640 1 -- 172.21.15.172:0/3651722042 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e98007820 con 0x7f4ea815e430 2024-04-14T08:34:08.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4eaf98c640 1 -- 172.21.15.172:0/3651722042 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea80b1190 con 0x7f4ea815e430 2024-04-14T08:34:08.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.209+0000 7f4e9dffb640 1 -- 172.21.15.172:0/3651722042 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98007bd0 con 0x7f4ea815e430 2024-04-14T08:34:08.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.210+0000 7f4eaf98c640 1 -- 172.21.15.172:0/3651722042 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea80b19d0 con 0x7f4ea815e430 2024-04-14T08:34:08.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.211+0000 7f4eaf98c640 1 -- 172.21.15.172:0/3651722042 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea80b9d40 con 0x7f4ea815e430 2024-04-14T08:34:08.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.211+0000 7f4e9dffb640 1 -- 172.21.15.172:0/3651722042 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e98018070 con 0x7f4ea815e430 2024-04-14T08:34:08.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.211+0000 7f4e9dffb640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4e8c06c8f0 0x7f4e8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.212+0000 7f4e9dffb640 1 -- 172.21.15.172:0/3651722042 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f4e9808be50 con 0x7f4ea815e430 2024-04-14T08:34:08.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.212+0000 7f4ead728640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4e8c06c8f0 0x7f4e8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.212+0000 7f4ead728640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4e8c06c8f0 0x7f4e8c06edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4036000 tx=0x7f4ea4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:08.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.220+0000 7f4e9dffb640 1 -- 172.21.15.172:0/3651722042 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4ea80b1190 con 0x7f4ea815e430 2024-04-14T08:34:08.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.397+0000 7f4eaf98c640 1 -- 172.21.15.172:0/3651722042 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4ea8068590 con 0x7f4ea815e430 2024-04-14T08:34:08.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.398+0000 7f4e9dffb640 1 -- 172.21.15.172:0/3651722042 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) ==== 100+0+11 (secure 0 0 0) 0x7f4e9805a0f0 con 0x7f4ea815e430 2024-04-14T08:34:08.399 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:34:08.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.400+0000 7f4eaf98c640 1 -- 172.21.15.172:0/3651722042 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4e8c06c8f0 msgr2=0x7f4e8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.400+0000 7f4eaf98c640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4e8c06c8f0 0x7f4e8c06edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4036000 tx=0x7f4ea4008030 comp rx=0 tx=0).stop 2024-04-14T08:34:08.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.400+0000 7f4eaf98c640 1 -- 172.21.15.172:0/3651722042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ea815e430 msgr2=0x7f4ea8144ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.400+0000 7f4eaf98c640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ea815e430 0x7f4ea8144ff0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800c7f0 tx=0x7f4e9800ccb0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.400+0000 7f4eaf98c640 1 -- 172.21.15.172:0/3651722042 shutdown_connections 2024-04-14T08:34:08.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.400+0000 7f4eaf98c640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4e8c06c8f0 0x7f4e8c06edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.400+0000 7f4eaf98c640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ea8162540 0x7f4ea8158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.400+0000 7f4eaf98c640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ea815e430 0x7f4ea8144ff0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.400+0000 7f4eaf98c640 1 --2- 172.21.15.172:0/3651722042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ea80b3a30 0x7f4ea80b97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.400+0000 7f4eaf98c640 1 -- 172.21.15.172:0/3651722042 >> 172.21.15.172:0/3651722042 conn(0x7f4ea809d560 msgr2=0x7f4ea815ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:08.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.400+0000 7f4eaf98c640 1 -- 172.21.15.172:0/3651722042 shutdown_connections 2024-04-14T08:34:08.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.400+0000 7f4eaf98c640 1 -- 172.21.15.172:0/3651722042 wait complete. 2024-04-14T08:34:08.414 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:34:08.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.518+0000 7f5422ea2640 1 Processor -- start 2024-04-14T08:34:08.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.518+0000 7f5422ea2640 1 -- start start 2024-04-14T08:34:08.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f5422ea2640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f541c15e430 0x7f541c162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f5422ea2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f541c162540 0x7f541c1629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f5422ea2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f541c0b3a30 0x7f541c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f5422ea2640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f541c0cefb0 con 0x7f541c162540 2024-04-14T08:34:08.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f5422ea2640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f541c0cf290 con 0x7f541c15e430 2024-04-14T08:34:08.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f5422ea2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f541c0cf570 con 0x7f541c0b3a30 2024-04-14T08:34:08.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f541bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f541c162540 0x7f541c1629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f541bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f541c162540 0x7f541c1629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48936/0 (socket says 172.21.15.172:48936) 2024-04-14T08:34:08.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f541bfff640 1 -- 172.21.15.172:0/348694180 learned_addr learned my addr 172.21.15.172:0/348694180 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:08.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f542143f640 1 --2- 172.21.15.172:0/348694180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f541c0b3a30 0x7f541c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f5420c3e640 1 --2- 172.21.15.172:0/348694180 >> v2:172.21.15.177:3300/0 conn(0x7f541c15e430 0x7f541c162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f541bfff640 1 -- 172.21.15.172:0/348694180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f541c0b3a30 msgr2=0x7f541c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:08.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f541bfff640 1 --2- 172.21.15.172:0/348694180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f541c0b3a30 0x7f541c0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f541bfff640 1 -- 172.21.15.172:0/348694180 >> v2:172.21.15.177:3300/0 conn(0x7f541c15e430 msgr2=0x7f541c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:08.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f541bfff640 1 --2- 172.21.15.172:0/348694180 >> v2:172.21.15.177:3300/0 conn(0x7f541c15e430 0x7f541c162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.519+0000 7f541bfff640 1 -- 172.21.15.172:0/348694180 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c0cf850 con 0x7f541c162540 2024-04-14T08:34:08.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.520+0000 7f541bfff640 1 --2- 172.21.15.172:0/348694180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541c162540 0x7f541c1629d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f541c0b6db0 tx=0x7f540c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab342c9f301169e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:08.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.520+0000 7f541b7fe640 1 -- 172.21.15.172:0/348694180 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c02ca70 con 0x7f541c162540 2024-04-14T08:34:08.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.520+0000 7f541b7fe640 1 -- 172.21.15.172:0/348694180 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f540c02cc10 con 0x7f541c162540 2024-04-14T08:34:08.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.520+0000 7f541b7fe640 1 -- 172.21.15.172:0/348694180 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c0343f0 con 0x7f541c162540 2024-04-14T08:34:08.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.521+0000 7f5422ea2640 1 -- 172.21.15.172:0/348694180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541c162540 msgr2=0x7f541c1629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.521+0000 7f5422ea2640 1 --2- 172.21.15.172:0/348694180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541c162540 0x7f541c1629d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f541c0b6db0 tx=0x7f540c02bed0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.521+0000 7f5422ea2640 1 -- 172.21.15.172:0/348694180 shutdown_connections 2024-04-14T08:34:08.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.521+0000 7f5422ea2640 1 --2- 172.21.15.172:0/348694180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f541c0b3a30 0x7f541c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.521+0000 7f5422ea2640 1 --2- 172.21.15.172:0/348694180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541c162540 0x7f541c1629d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.521+0000 7f5422ea2640 1 --2- 172.21.15.172:0/348694180 >> v2:172.21.15.177:3300/0 conn(0x7f541c15e430 0x7f541c162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.521+0000 7f5422ea2640 1 -- 172.21.15.172:0/348694180 >> 172.21.15.172:0/348694180 conn(0x7f541c09d560 msgr2=0x7f541c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:08.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.521+0000 7f5422ea2640 1 -- 172.21.15.172:0/348694180 shutdown_connections 2024-04-14T08:34:08.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.521+0000 7f5422ea2640 1 -- 172.21.15.172:0/348694180 wait complete. 2024-04-14T08:34:08.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.522+0000 7f5422ea2640 1 Processor -- start 2024-04-14T08:34:08.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.522+0000 7f5422ea2640 1 -- start start 2024-04-14T08:34:08.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.523+0000 7f5422ea2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541c0b3a30 0x7f541c1592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.523+0000 7f5422ea2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f541c15e430 0x7f541c158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.523+0000 7f5422ea2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f541c162540 0x7f541c0b8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.523+0000 7f5422ea2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f541c0d4e20 con 0x7f541c0b3a30 2024-04-14T08:34:08.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.523+0000 7f5422ea2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f541c15e810 con 0x7f541c162540 2024-04-14T08:34:08.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.523+0000 7f5422ea2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f541c0d0050 con 0x7f541c15e430 2024-04-14T08:34:08.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.523+0000 7f5420c3e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541c0b3a30 0x7f541c1592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.523+0000 7f5420c3e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541c0b3a30 0x7f541c1592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48944/0 (socket says 172.21.15.172:48944) 2024-04-14T08:34:08.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.523+0000 7f5420c3e640 1 -- 172.21.15.172:0/3683411772 learned_addr learned my addr 172.21.15.172:0/3683411772 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.523+0000 7f541bfff640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f541c15e430 0x7f541c158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.523+0000 7f542143f640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f541c162540 0x7f541c0b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.524+0000 7f5420c3e640 1 -- 172.21.15.172:0/3683411772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f541c15e430 msgr2=0x7f541c158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.524+0000 7f5420c3e640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f541c15e430 0x7f541c158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.524+0000 7f5420c3e640 1 -- 172.21.15.172:0/3683411772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f541c162540 msgr2=0x7f541c0b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.524+0000 7f5420c3e640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f541c162540 0x7f541c0b8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.524+0000 7f5420c3e640 1 -- 172.21.15.172:0/3683411772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c0b87a0 con 0x7f541c0b3a30 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.524+0000 7f5420c3e640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541c0b3a30 0x7f541c1592c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f540400d620 tx=0x7f540400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.524+0000 7f54197fa640 1 -- 172.21.15.172:0/3683411772 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540400b620 con 0x7f541c0b3a30 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.524+0000 7f54197fa640 1 -- 172.21.15.172:0/3683411772 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5404019070 con 0x7f541c0b3a30 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.524+0000 7f5422ea2640 1 -- 172.21.15.172:0/3683411772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c0b8a80 con 0x7f541c0b3a30 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.524+0000 7f54197fa640 1 -- 172.21.15.172:0/3683411772 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540400ba00 con 0x7f541c0b3a30 2024-04-14T08:34:08.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.524+0000 7f5422ea2640 1 -- 172.21.15.172:0/3683411772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c0b9040 con 0x7f541c0b3a30 2024-04-14T08:34:08.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.525+0000 7f5422ea2640 1 -- 172.21.15.172:0/3683411772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541c068590 con 0x7f541c0b3a30 2024-04-14T08:34:08.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.526+0000 7f54197fa640 1 -- 172.21.15.172:0/3683411772 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f540400bba0 con 0x7f541c0b3a30 2024-04-14T08:34:08.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.526+0000 7f54197fa640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f53fc06c8f0 0x7f53fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.526+0000 7f54197fa640 1 -- 172.21.15.172:0/3683411772 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f540408c010 con 0x7f541c0b3a30 2024-04-14T08:34:08.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.526+0000 7f541bfff640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f53fc06c8f0 0x7f53fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.527+0000 7f541bfff640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f53fc06c8f0 0x7f53fc06edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f540c035000 tx=0x7f540c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:08.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.531+0000 7f54197fa640 1 -- 172.21.15.172:0/3683411772 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5404059c60 con 0x7f541c0b3a30 2024-04-14T08:34:08.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.733+0000 7f5422ea2640 1 -- 172.21.15.172:0/3683411772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f541c0d18a0 con 0x7f541c0b3a30 2024-04-14T08:34:08.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.734+0000 7f54197fa640 1 -- 172.21.15.172:0/3683411772 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) ==== 80+0+0 (secure 0 0 0) 0x7f540405df60 con 0x7f541c0b3a30 2024-04-14T08:34:08.735 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:34:08.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.736+0000 7f5422ea2640 1 -- 172.21.15.172:0/3683411772 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f53fc06c8f0 msgr2=0x7f53fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.736+0000 7f5422ea2640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f53fc06c8f0 0x7f53fc06edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f540c035000 tx=0x7f540c030000 comp rx=0 tx=0).stop 2024-04-14T08:34:08.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.736+0000 7f5422ea2640 1 -- 172.21.15.172:0/3683411772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541c0b3a30 msgr2=0x7f541c1592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.736+0000 7f5422ea2640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541c0b3a30 0x7f541c1592c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f540400d620 tx=0x7f540400dae0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.736+0000 7f5422ea2640 1 -- 172.21.15.172:0/3683411772 shutdown_connections 2024-04-14T08:34:08.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.736+0000 7f5422ea2640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f53fc06c8f0 0x7f53fc06edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.736+0000 7f5422ea2640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f541c162540 0x7f541c0b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.736+0000 7f5422ea2640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f541c15e430 0x7f541c158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.736+0000 7f5422ea2640 1 --2- 172.21.15.172:0/3683411772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541c0b3a30 0x7f541c1592c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.736+0000 7f5422ea2640 1 -- 172.21.15.172:0/3683411772 >> 172.21.15.172:0/3683411772 conn(0x7f541c09d560 msgr2=0x7f541c0b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:08.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.736+0000 7f5422ea2640 1 -- 172.21.15.172:0/3683411772 shutdown_connections 2024-04-14T08:34:08.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.736+0000 7f5422ea2640 1 -- 172.21.15.172:0/3683411772 wait complete. 2024-04-14T08:34:08.749 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:34:08.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:08.749 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:34:08.798 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:34:08.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:08.798 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:34:08.824 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:34:08.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.965+0000 7f5e85f11640 1 Processor -- start 2024-04-14T08:34:08.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.965+0000 7f5e85f11640 1 -- start start 2024-04-14T08:34:08.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.966+0000 7f5e85f11640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e8008c4c0 0x7f5e8006d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.966+0000 7f5e85f11640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e800d2b10 con 0x7f5e800b7a60 2024-04-14T08:34:08.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.966+0000 7f5e85f11640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e800ce420 con 0x7f5e800cefd0 2024-04-14T08:34:08.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.966+0000 7f5e85f11640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e800d0ad0 con 0x7f5e8008c4c0 2024-04-14T08:34:08.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.966+0000 7f5e7f7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5e800b7a60 0x7f5e800ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48366/0 (socket says 172.21.15.172:48366) 2024-04-14T08:34:08.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.966+0000 7f5e7f7fe640 1 -- 172.21.15.172:0/4223117558 learned_addr learned my addr 172.21.15.172:0/4223117558 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:08.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.966+0000 7f5e7ffff640 1 --2- 172.21.15.172:0/4223117558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e8008c4c0 0x7f5e8006d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.967+0000 7f5e7effd640 1 -- 172.21.15.172:0/4223117558 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 786065211 0 0) 0x7f5e800d2b10 con 0x7f5e800b7a60 2024-04-14T08:34:08.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.967+0000 7f5e7effd640 1 -- 172.21.15.172:0/4223117558 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e6c003520 con 0x7f5e800b7a60 2024-04-14T08:34:08.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.967+0000 7f5e7effd640 1 -- 172.21.15.172:0/4223117558 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1357532374 0 0) 0x7f5e800ce420 con 0x7f5e800cefd0 2024-04-14T08:34:08.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.967+0000 7f5e7effd640 1 -- 172.21.15.172:0/4223117558 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e800d2b10 con 0x7f5e800cefd0 2024-04-14T08:34:08.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.967+0000 7f5e7ffff640 1 -- 172.21.15.172:0/4223117558 >> v1:172.21.15.177:6789/0 conn(0x7f5e800cefd0 legacy=0x7f5e8006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.967+0000 7f5e7effd640 1 -- 172.21.15.172:0/4223117558 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3869451558 0 0) 0x7f5e6c003520 con 0x7f5e800b7a60 2024-04-14T08:34:08.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.967+0000 7f5e7ffff640 1 -- 172.21.15.172:0/4223117558 >> v1:172.21.15.172:6789/0 conn(0x7f5e800b7a60 legacy=0x7f5e800ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.967+0000 7f5e7ffff640 1 -- 172.21.15.172:0/4223117558 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e800d0db0 con 0x7f5e8008c4c0 2024-04-14T08:34:08.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.968+0000 7f5e7ffff640 1 --2- 172.21.15.172:0/4223117558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e8008c4c0 0x7f5e8006d340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009dc0 tx=0x7f5e7802bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5946c43004daa80c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:08.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.968+0000 7f5e7effd640 1 -- 172.21.15.172:0/4223117558 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78039070 con 0x7f5e8008c4c0 2024-04-14T08:34:08.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.968+0000 7f5e7effd640 1 -- 172.21.15.172:0/4223117558 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7802c880 con 0x7f5e8008c4c0 2024-04-14T08:34:08.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.968+0000 7f5e7effd640 1 -- 172.21.15.172:0/4223117558 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70001670 con 0x7f5e8008c4c0 2024-04-14T08:34:08.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.968+0000 7f5e85f11640 1 -- 172.21.15.172:0/4223117558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e8008c4c0 msgr2=0x7f5e8006d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.968+0000 7f5e85f11640 1 --2- 172.21.15.172:0/4223117558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e8008c4c0 0x7f5e8006d340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009dc0 tx=0x7f5e7802bd50 comp rx=0 tx=0).stop 2024-04-14T08:34:08.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.968+0000 7f5e85f11640 1 -- 172.21.15.172:0/4223117558 shutdown_connections 2024-04-14T08:34:08.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.968+0000 7f5e85f11640 1 --2- 172.21.15.172:0/4223117558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e8008c4c0 0x7f5e8006d340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.968+0000 7f5e85f11640 1 -- 172.21.15.172:0/4223117558 >> 172.21.15.172:0/4223117558 conn(0x7f5e8008c000 msgr2=0x7f5e800d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:08.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.968+0000 7f5e85f11640 1 -- 172.21.15.172:0/4223117558 shutdown_connections 2024-04-14T08:34:08.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.968+0000 7f5e85f11640 1 -- 172.21.15.172:0/4223117558 wait complete. 2024-04-14T08:34:08.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.969+0000 7f5e85f11640 1 Processor -- start 2024-04-14T08:34:08.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.970+0000 7f5e85f11640 1 -- start start 2024-04-14T08:34:08.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.970+0000 7f5e85f11640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e8008c4c0 0x7f5e800cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.970+0000 7f5e85f11640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e800b7a60 0x7f5e801f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.970+0000 7f5e85f11640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e800cefd0 0x7f5e8009f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.970+0000 7f5e85f11640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e800b7e40 con 0x7f5e8008c4c0 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.970+0000 7f5e85f11640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e800b4e90 con 0x7f5e800cefd0 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.970+0000 7f5e85f11640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e800d2750 con 0x7f5e800b7a60 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.970+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e8008c4c0 0x7f5e800cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.970+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e8008c4c0 0x7f5e800cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48952/0 (socket says 172.21.15.172:48952) 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.970+0000 7f5e7f7fe640 1 -- 172.21.15.172:0/2242369372 learned_addr learned my addr 172.21.15.172:0/2242369372 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.970+0000 7f5e77fff640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e800b7a60 0x7f5e801f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.970+0000 7f5e7ffff640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e800cefd0 0x7f5e8009f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.971+0000 7f5e7f7fe640 1 -- 172.21.15.172:0/2242369372 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e800b7a60 msgr2=0x7f5e801f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.971+0000 7f5e7f7fe640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e800b7a60 0x7f5e801f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.971+0000 7f5e7f7fe640 1 -- 172.21.15.172:0/2242369372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e800cefd0 msgr2=0x7f5e8009f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.971+0000 7f5e7f7fe640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e800cefd0 0x7f5e8009f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.971+0000 7f5e7f7fe640 1 -- 172.21.15.172:0/2242369372 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e801f7f40 con 0x7f5e8008c4c0 2024-04-14T08:34:08.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.971+0000 7f5e7f7fe640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e8008c4c0 0x7f5e800cb520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000c7b0 tx=0x7f5e7000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.971+0000 7f5e7cff9640 1 -- 172.21.15.172:0/2242369372 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70017070 con 0x7f5e8008c4c0 2024-04-14T08:34:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.971+0000 7f5e7cff9640 1 -- 172.21.15.172:0/2242369372 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e70005af0 con 0x7f5e8008c4c0 2024-04-14T08:34:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.971+0000 7f5e85f11640 1 -- 172.21.15.172:0/2242369372 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e801f8220 con 0x7f5e8008c4c0 2024-04-14T08:34:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.971+0000 7f5e7cff9640 1 -- 172.21.15.172:0/2242369372 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70007070 con 0x7f5e8008c4c0 2024-04-14T08:34:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.971+0000 7f5e85f11640 1 -- 172.21.15.172:0/2242369372 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e801f4980 con 0x7f5e8008c4c0 2024-04-14T08:34:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.972+0000 7f5e85f11640 1 -- 172.21.15.172:0/2242369372 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e80075490 con 0x7f5e8008c4c0 2024-04-14T08:34:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.972+0000 7f5e7cff9640 1 -- 172.21.15.172:0/2242369372 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e7001a070 con 0x7f5e8008c4c0 2024-04-14T08:34:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.973+0000 7f5e7cff9640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e6006c920 0x7f5e6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.973+0000 7f5e7cff9640 1 -- 172.21.15.172:0/2242369372 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5e7008b160 con 0x7f5e8008c4c0 2024-04-14T08:34:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.973+0000 7f5e77fff640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e6006c920 0x7f5e6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.973+0000 7f5e77fff640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e6006c920 0x7f5e6006ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5e80162280 tx=0x7f5e68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:08.982+0000 7f5e7cff9640 1 -- 172.21.15.172:0/2242369372 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5e70058db0 con 0x7f5e8008c4c0 2024-04-14T08:34:09.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.385+0000 7f5e85f11640 1 -- 172.21.15.172:0/2242369372 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e80070570 con 0x7f5e8008c4c0 2024-04-14T08:34:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.385+0000 7f5e7cff9640 1 -- 172.21.15.172:0/2242369372 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7f5e7005d0b0 con 0x7f5e8008c4c0 2024-04-14T08:34:09.387 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:09.387 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1038094401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1038094401},{"type":"v1","addr":"172.21.15.177:6834","nonce":1038094401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1119512744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1119512744},{"type":"v1","addr":"172.21.15.177:6835","nonce":1119512744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3054138283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3054138283},{"type":"v1","addr":"172.21.15.172:6837","nonce":3054138283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:33:59.340499+0000","modified":"2024-04-14T08:34:01.467567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5308":{"gid":5308,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6836/3420108191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3420108191},{"type":"v1","addr":"172.21.15.172:6836","nonce":3420108191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5308,"qdb_cluster":[5308]},"id":2}]} 2024-04-14T08:34:09.387 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 16 2024-04-14T08:34:09.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.388+0000 7f5e85f11640 1 -- 172.21.15.172:0/2242369372 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e6006c920 msgr2=0x7f5e6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:09.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.388+0000 7f5e85f11640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e6006c920 0x7f5e6006ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5e80162280 tx=0x7f5e68009040 comp rx=0 tx=0).stop 2024-04-14T08:34:09.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.388+0000 7f5e85f11640 1 -- 172.21.15.172:0/2242369372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e8008c4c0 msgr2=0x7f5e800cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:09.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.388+0000 7f5e85f11640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e8008c4c0 0x7f5e800cb520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000c7b0 tx=0x7f5e7000cc70 comp rx=0 tx=0).stop 2024-04-14T08:34:09.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.388+0000 7f5e85f11640 1 -- 172.21.15.172:0/2242369372 shutdown_connections 2024-04-14T08:34:09.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.388+0000 7f5e85f11640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e6006c920 0x7f5e6006ede0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.388+0000 7f5e85f11640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e800cefd0 0x7f5e8009f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.388+0000 7f5e85f11640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e800b7a60 0x7f5e801f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.388+0000 7f5e85f11640 1 --2- 172.21.15.172:0/2242369372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e8008c4c0 0x7f5e800cb520 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.388+0000 7f5e85f11640 1 -- 172.21.15.172:0/2242369372 >> 172.21.15.172:0/2242369372 conn(0x7f5e8008c000 msgr2=0x7f5e800d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:09.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.388+0000 7f5e85f11640 1 -- 172.21.15.172:0/2242369372 shutdown_connections 2024-04-14T08:34:09.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.388+0000 7f5e85f11640 1 -- 172.21.15.172:0/2242369372 wait complete. 2024-04-14T08:34:09.401 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-14T08:34:09.538 INFO:teuthology.orchestra.run.smithi172.stdout:{"message":"","return_code":0} 2024-04-14T08:34:09.539 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-14T08:34:09.539 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:34:09.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.640+0000 7f896d4c8640 1 Processor -- start 2024-04-14T08:34:09.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.640+0000 7f896d4c8640 1 -- start start 2024-04-14T08:34:09.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f896d4c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89680cc450 0x7f89680af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:09.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f896d4c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f89680af340 0x7f896806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:09.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f896d4c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f896806dd10 0x7f8968162170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:09.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f896d4c8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f896806e180 con 0x7f89680af340 2024-04-14T08:34:09.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f896d4c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f89680cdea0 con 0x7f896806dd10 2024-04-14T08:34:09.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f896d4c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f89680cf920 con 0x7f89680cc450 2024-04-14T08:34:09.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f895e7fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f89680af340 0x7f896806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:09.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f895e7fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f89680af340 0x7f896806d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45036/0 (socket says 172.21.15.172:45036) 2024-04-14T08:34:09.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f895e7fc640 1 -- 172.21.15.172:0/436457653 learned_addr learned my addr 172.21.15.172:0/436457653 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:09.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f89677fe640 1 --2- 172.21.15.172:0/436457653 >> v2:172.21.15.177:3300/0 conn(0x7f896806dd10 0x7f8968162170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:09.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f8966ffd640 1 --2- 172.21.15.172:0/436457653 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89680cc450 0x7f89680af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:09.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f895e7fc640 1 -- 172.21.15.172:0/436457653 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89680cc450 msgr2=0x7f89680af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:09.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f895e7fc640 1 --2- 172.21.15.172:0/436457653 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89680cc450 0x7f89680af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f895e7fc640 1 -- 172.21.15.172:0/436457653 >> v2:172.21.15.177:3300/0 conn(0x7f896806dd10 msgr2=0x7f8968162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:09.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f895e7fc640 1 --2- 172.21.15.172:0/436457653 >> v2:172.21.15.177:3300/0 conn(0x7f896806dd10 0x7f8968162170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f895e7fc640 1 -- 172.21.15.172:0/436457653 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896815f9e0 con 0x7f89680af340 2024-04-14T08:34:09.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.641+0000 7f89677fe640 1 --2- 172.21.15.172:0/436457653 >> v2:172.21.15.177:3300/0 conn(0x7f896806dd10 0x7f8968162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:09.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.642+0000 7f895e7fc640 1 --2- 172.21.15.172:0/436457653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89680af340 0x7f896806d7d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8968068d30 tx=0x7f895002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91380be3284af91e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:09.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.643+0000 7f89667fc640 1 -- 172.21.15.172:0/436457653 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950035020 con 0x7f89680af340 2024-04-14T08:34:09.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.643+0000 7f89667fc640 1 -- 172.21.15.172:0/436457653 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8950031870 con 0x7f89680af340 2024-04-14T08:34:09.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.643+0000 7f89667fc640 1 -- 172.21.15.172:0/436457653 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950031c40 con 0x7f89680af340 2024-04-14T08:34:09.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.643+0000 7f896d4c8640 1 -- 172.21.15.172:0/436457653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89680af340 msgr2=0x7f896806d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:09.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.643+0000 7f896d4c8640 1 --2- 172.21.15.172:0/436457653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89680af340 0x7f896806d7d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8968068d30 tx=0x7f895002b910 comp rx=0 tx=0).stop 2024-04-14T08:34:09.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.643+0000 7f896d4c8640 1 -- 172.21.15.172:0/436457653 shutdown_connections 2024-04-14T08:34:09.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.643+0000 7f896d4c8640 1 --2- 172.21.15.172:0/436457653 >> v2:172.21.15.177:3300/0 conn(0x7f896806dd10 0x7f8968162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.643+0000 7f896d4c8640 1 --2- 172.21.15.172:0/436457653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89680af340 0x7f896806d7d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.643+0000 7f896d4c8640 1 --2- 172.21.15.172:0/436457653 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89680cc450 0x7f89680af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.643+0000 7f896d4c8640 1 -- 172.21.15.172:0/436457653 >> 172.21.15.172:0/436457653 conn(0x7f8968072b20 msgr2=0x7f8968072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:09.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.643+0000 7f896d4c8640 1 -- 172.21.15.172:0/436457653 shutdown_connections 2024-04-14T08:34:09.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.643+0000 7f896d4c8640 1 -- 172.21.15.172:0/436457653 wait complete. 2024-04-14T08:34:09.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.644+0000 7f896d4c8640 1 Processor -- start 2024-04-14T08:34:09.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.644+0000 7f896d4c8640 1 -- start start 2024-04-14T08:34:09.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.645+0000 7f896d4c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f896806dd10 0x7f89680b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:09.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.645+0000 7f896d4c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89680af340 0x7f896815a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:09.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.645+0000 7f896d4c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89680cc450 0x7f8968099890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:09.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.645+0000 7f896d4c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f896806f7a0 con 0x7f89680cc450 2024-04-14T08:34:09.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.645+0000 7f896d4c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f896806f2a0 con 0x7f896806dd10 2024-04-14T08:34:09.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.645+0000 7f896d4c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f89680ce3d0 con 0x7f89680af340 2024-04-14T08:34:09.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.645+0000 7f89677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89680cc450 0x7f8968099890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:09.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.645+0000 7f89677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89680cc450 0x7f8968099890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45046/0 (socket says 172.21.15.172:45046) 2024-04-14T08:34:09.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.645+0000 7f89677fe640 1 -- 172.21.15.172:0/3643600773 learned_addr learned my addr 172.21.15.172:0/3643600773 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:09.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.645+0000 7f8966ffd640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f896806dd10 0x7f89680b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:09.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.645+0000 7f895e7fc640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89680af340 0x7f896815a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:09.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.646+0000 7f89677fe640 1 -- 172.21.15.172:0/3643600773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89680af340 msgr2=0x7f896815a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:09.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.646+0000 7f89677fe640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89680af340 0x7f896815a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.646+0000 7f89677fe640 1 -- 172.21.15.172:0/3643600773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f896806dd10 msgr2=0x7f89680b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:09.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.646+0000 7f89677fe640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f896806dd10 0x7f89680b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.646+0000 7f89677fe640 1 -- 172.21.15.172:0/3643600773 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896815b170 con 0x7f89680cc450 2024-04-14T08:34:09.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.646+0000 7f895e7fc640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89680af340 0x7f896815a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:09.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.646+0000 7f89677fe640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89680cc450 0x7f8968099890 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8960007de0 tx=0x7f896000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:09.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.646+0000 7f895ffff640 1 -- 172.21.15.172:0/3643600773 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896001a070 con 0x7f89680cc450 2024-04-14T08:34:09.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.646+0000 7f895ffff640 1 -- 172.21.15.172:0/3643600773 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8960003990 con 0x7f89680cc450 2024-04-14T08:34:09.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.646+0000 7f895ffff640 1 -- 172.21.15.172:0/3643600773 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960003d80 con 0x7f89680cc450 2024-04-14T08:34:09.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.646+0000 7f896d4c8640 1 -- 172.21.15.172:0/3643600773 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896815b450 con 0x7f89680cc450 2024-04-14T08:34:09.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.646+0000 7f896d4c8640 1 -- 172.21.15.172:0/3643600773 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896815ba10 con 0x7f89680cc450 2024-04-14T08:34:09.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.647+0000 7f896d4c8640 1 -- 172.21.15.172:0/3643600773 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8968068520 con 0x7f89680cc450 2024-04-14T08:34:09.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.648+0000 7f895ffff640 1 -- 172.21.15.172:0/3643600773 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f896001d070 con 0x7f89680cc450 2024-04-14T08:34:09.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.648+0000 7f895ffff640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f894806c8f0 0x7f894806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:09.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.648+0000 7f8966ffd640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f894806c8f0 0x7f894806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:09.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.648+0000 7f895ffff640 1 -- 172.21.15.172:0/3643600773 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f896008bef0 con 0x7f89680cc450 2024-04-14T08:34:09.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.649+0000 7f8966ffd640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f894806c8f0 0x7f894806edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8958006fd0 tx=0x7f8958008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:09.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.653+0000 7f895ffff640 1 -- 172.21.15.172:0/3643600773 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8960010070 con 0x7f89680cc450 2024-04-14T08:34:09.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.872+0000 7f896d4c8640 1 -- 172.21.15.172:0/3643600773 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89680b6250 con 0x7f89680cc450 2024-04-14T08:34:09.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.873+0000 7f895ffff640 1 -- 172.21.15.172:0/3643600773 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7f8960059b40 con 0x7f89680cc450 2024-04-14T08:34:09.874 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:09.874 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1038094401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1038094401},{"type":"v1","addr":"172.21.15.177:6834","nonce":1038094401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1119512744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1119512744},{"type":"v1","addr":"172.21.15.177:6835","nonce":1119512744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3054138283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3054138283},{"type":"v1","addr":"172.21.15.172:6837","nonce":3054138283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:33:59.340499+0000","modified":"2024-04-14T08:34:01.467567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5308":{"gid":5308,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6836/3420108191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3420108191},{"type":"v1","addr":"172.21.15.172:6836","nonce":3420108191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5308,"qdb_cluster":[5308]},"id":2}]} 2024-04-14T08:34:09.875 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 16 2024-04-14T08:34:09.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.874+0000 7f896d4c8640 1 -- 172.21.15.172:0/3643600773 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f894806c8f0 msgr2=0x7f894806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:09.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.874+0000 7f896d4c8640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f894806c8f0 0x7f894806edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8958006fd0 tx=0x7f8958008040 comp rx=0 tx=0).stop 2024-04-14T08:34:09.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.875+0000 7f896d4c8640 1 -- 172.21.15.172:0/3643600773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89680cc450 msgr2=0x7f8968099890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:09.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.875+0000 7f896d4c8640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89680cc450 0x7f8968099890 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8960007de0 tx=0x7f896000d4a0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.875+0000 7f896d4c8640 1 -- 172.21.15.172:0/3643600773 shutdown_connections 2024-04-14T08:34:09.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.875+0000 7f896d4c8640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f894806c8f0 0x7f894806edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.875+0000 7f896d4c8640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89680cc450 0x7f8968099890 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.875+0000 7f896d4c8640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89680af340 0x7f896815a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.875+0000 7f896d4c8640 1 --2- 172.21.15.172:0/3643600773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f896806dd10 0x7f89680b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:09.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.875+0000 7f896d4c8640 1 -- 172.21.15.172:0/3643600773 >> 172.21.15.172:0/3643600773 conn(0x7f8968072b20 msgr2=0x7f896815ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:09.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.875+0000 7f896d4c8640 1 -- 172.21.15.172:0/3643600773 shutdown_connections 2024-04-14T08:34:09.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:09.875+0000 7f896d4c8640 1 -- 172.21.15.172:0/3643600773 wait complete. 2024-04-14T08:34:09.891 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-04-14T08:34:10.016 INFO:teuthology.orchestra.run.smithi172.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-14T08:34:10.017 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-14T08:34:20.018 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-14T08:34:20.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.059+0000 7f4ae5842f40 1 Processor -- start 2024-04-14T08:34:20.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.059+0000 7f4ae5842f40 1 -- start start 2024-04-14T08:34:20.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.060+0000 7f4ae5842f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aede2d80 0x55b5aeeb4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.060+0000 7f4ae5842f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5aee0f570 0x55b5aeeb5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.060+0000 7f4ae5842f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5aeeb4730 0x55b5aef4d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.060+0000 7f4ae5842f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b5aed873b0 con 0x55b5aede2d80 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.060+0000 7f4ae5842f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b5aed8b400 con 0x55b5aee0f570 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae5842f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55b5aeea7910 con 0x55b5aeeb4730 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae284b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aede2d80 0x55b5aeeb4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae284b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aede2d80 0x55b5aeeb4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48562/0 (socket says 172.21.15.172:48562) 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae284b640 1 -- 172.21.15.172:0/686943652 learned_addr learned my addr 172.21.15.172:0/686943652 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae304c640 1 --2- 172.21.15.172:0/686943652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5aeeb4730 0x55b5aef4d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4adbfff640 1 --2- 172.21.15.172:0/686943652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5aee0f570 0x55b5aeeb5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae284b640 1 -- 172.21.15.172:0/686943652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5aeeb4730 msgr2=0x55b5aef4d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae284b640 1 --2- 172.21.15.172:0/686943652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5aeeb4730 0x55b5aef4d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae284b640 1 -- 172.21.15.172:0/686943652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5aee0f570 msgr2=0x55b5aeeb5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae284b640 1 --2- 172.21.15.172:0/686943652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5aee0f570 0x55b5aeeb5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae284b640 1 -- 172.21.15.172:0/686943652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5aeea8c90 con 0x55b5aede2d80 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae284b640 1 --2- 172.21.15.172:0/686943652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aede2d80 0x55b5aeeb4f20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4001d90 tx=0x7f4ad400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b461fefac9d0ebc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae1849640 1 -- 172.21.15.172:0/686943652 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4012020 con 0x55b5aede2d80 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae1849640 1 -- 172.21.15.172:0/686943652 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad400e670 con 0x55b5aede2d80 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.061+0000 7f4ae1849640 1 -- 172.21.15.172:0/686943652 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400eac0 con 0x55b5aede2d80 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.062+0000 7f4ae5842f40 1 -- 172.21.15.172:0/686943652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aede2d80 msgr2=0x55b5aeeb4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.062+0000 7f4ae5842f40 1 --2- 172.21.15.172:0/686943652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aede2d80 0x55b5aeeb4f20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4001d90 tx=0x7f4ad400b6c0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.062+0000 7f4ae5842f40 1 -- 172.21.15.172:0/686943652 shutdown_connections 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.062+0000 7f4ae5842f40 1 --2- 172.21.15.172:0/686943652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5aeeb4730 0x55b5aef4d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.062+0000 7f4ae5842f40 1 --2- 172.21.15.172:0/686943652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5aee0f570 0x55b5aeeb5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.062+0000 7f4ae5842f40 1 --2- 172.21.15.172:0/686943652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aede2d80 0x55b5aeeb4f20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.062+0000 7f4ae5842f40 1 -- 172.21.15.172:0/686943652 >> 172.21.15.172:0/686943652 conn(0x55b5aee0ca50 msgr2=0x55b5aee0ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.062+0000 7f4ae5842f40 1 -- 172.21.15.172:0/686943652 shutdown_connections 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.062+0000 7f4ae5842f40 1 -- 172.21.15.172:0/686943652 wait complete. 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.063+0000 7f4ae5842f40 1 Processor -- start 2024-04-14T08:34:20.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.063+0000 7f4ae5842f40 1 -- start start 2024-04-14T08:34:20.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.063+0000 7f4ae5842f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5aede2d80 0x55b5aeea46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.063+0000 7f4ae5842f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5aee0f570 0x55b5aee0c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.063+0000 7f4ae5842f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aeeb4730 0x55b5aeeb4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.063+0000 7f4ae5842f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b5aeea5350 con 0x55b5aeeb4730 2024-04-14T08:34:20.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.063+0000 7f4ae5842f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b5aee0dde0 con 0x55b5aede2d80 2024-04-14T08:34:20.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.063+0000 7f4ae5842f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55b5aeeb6910 con 0x55b5aee0f570 2024-04-14T08:34:20.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae304c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aeeb4730 0x55b5aeeb4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae304c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aeeb4730 0x55b5aeeb4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48572/0 (socket says 172.21.15.172:48572) 2024-04-14T08:34:20.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4adbfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5aee0f570 0x55b5aee0c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae304c640 1 -- 172.21.15.172:0/1548907320 learned_addr learned my addr 172.21.15.172:0/1548907320 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae284b640 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5aede2d80 0x55b5aeea46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae304c640 1 -- 172.21.15.172:0/1548907320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5aee0f570 msgr2=0x55b5aee0c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae304c640 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5aee0f570 0x55b5aee0c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae304c640 1 -- 172.21.15.172:0/1548907320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5aede2d80 msgr2=0x55b5aeea46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae304c640 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5aede2d80 0x55b5aeea46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae304c640 1 -- 172.21.15.172:0/1548907320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5aeeb2390 con 0x55b5aeeb4730 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae304c640 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aeeb4730 0x55b5aeeb4d60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4adc054040 tx=0x7f4adc003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae0847640 1 -- 172.21.15.172:0/1548907320 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc005a80 con 0x55b5aeeb4730 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae0847640 1 -- 172.21.15.172:0/1548907320 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4adc05c830 con 0x55b5aeeb4730 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae0847640 1 -- 172.21.15.172:0/1548907320 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc052430 con 0x55b5aeeb4730 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae5842f40 1 -- 172.21.15.172:0/1548907320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b5aeeb1af0 con 0x55b5aeeb4730 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.064+0000 7f4ae5842f40 1 -- 172.21.15.172:0/1548907320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5aeeb20b0 con 0x55b5aeeb4730 2024-04-14T08:34:20.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.066+0000 7f4ae0847640 1 -- 172.21.15.172:0/1548907320 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4adc064020 con 0x55b5aeeb4730 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.066+0000 7f4ae0847640 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ac406b2d0 0x7f4ac406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.066+0000 7f4ae284b640 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ac406b2d0 0x7f4ac406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.066+0000 7f4ae0847640 1 -- 172.21.15.172:0/1548907320 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f4adc0cce40 con 0x55b5aeeb4730 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.067+0000 7f4ae5842f40 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55b5aeeb52a0 0x55b5aeebc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.067+0000 7f4ae5842f40 1 -- 172.21.15.172:0/1548907320 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) -- 0x55b5aeebcba0 con 0x55b5aeeb52a0 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.067+0000 7f4ae284b640 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ac406b2d0 0x7f4ac406d790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4001d90 tx=0x7f4ad401b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.067+0000 7f4adbfff640 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55b5aeeb52a0 0x55b5aeebc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.067+0000 7f4adbfff640 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55b5aeeb52a0 0x55b5aeebc660 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.068+0000 7f4adbfff640 1 -- 172.21.15.172:0/1548907320 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f4acc00e040 con 0x55b5aeeb52a0 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.068+0000 7f4ae5842f40 1 -- 172.21.15.172:0/1548907320 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55b5aeeb52a0 msgr2=0x55b5aeebc660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.068+0000 7f4ae5842f40 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55b5aeeb52a0 0x55b5aeebc660 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.068+0000 7f4ae5842f40 1 -- 172.21.15.172:0/1548907320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ac406b2d0 msgr2=0x7f4ac406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.068+0000 7f4ae5842f40 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ac406b2d0 0x7f4ac406d790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4001d90 tx=0x7f4ad401b040 comp rx=0 tx=0).stop 2024-04-14T08:34:20.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.068+0000 7f4ae5842f40 1 -- 172.21.15.172:0/1548907320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aeeb4730 msgr2=0x55b5aeeb4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.068+0000 7f4ae5842f40 1 --2- 172.21.15.172:0/1548907320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5aeeb4730 0x55b5aeeb4d60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4adc054040 tx=0x7f4adc003330 comp rx=0 tx=0).stop 2024-04-14T08:34:20.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.068+0000 7f4ae304c640 1 -- 172.21.15.172:0/1548907320 reap_dead start 2024-04-14T08:34:20.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.068+0000 7f4ae5842f40 1 -- 172.21.15.172:0/1548907320 shutdown_connections 2024-04-14T08:34:20.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.068+0000 7f4ae5842f40 1 -- 172.21.15.172:0/1548907320 >> 172.21.15.172:0/1548907320 conn(0x55b5aee0ca50 msgr2=0x55b5aeeb33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:20.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.068+0000 7f4ae5842f40 1 -- 172.21.15.172:0/1548907320 shutdown_connections 2024-04-14T08:34:20.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.068+0000 7f4ae5842f40 1 -- 172.21.15.172:0/1548907320 wait complete. 2024-04-14T08:34:20.074 INFO:teuthology.orchestra.run.smithi172.stdout:mytag 2024-04-14T08:34:20.074 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-14T08:34:20.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.157+0000 7f90d2c36f40 1 Processor -- start 2024-04-14T08:34:20.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.157+0000 7f90d2c36f40 1 -- start start 2024-04-14T08:34:20.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.157+0000 7f90d2c36f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcce30 0x560bfbdb7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.157+0000 7f90d2c36f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560bfbdcea90 0x560bfbdb6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.157+0000 7f90d2c36f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560bfbdd2640 0x560bfbdb68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.157+0000 7f90d2c36f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560bfbdd32d0 con 0x560bfbdcce30 2024-04-14T08:34:20.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.157+0000 7f90d2c36f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560bfbd303b0 con 0x560bfbdd2640 2024-04-14T08:34:20.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.157+0000 7f90d2c36f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x560bfbdceff0 con 0x560bfbdcea90 2024-04-14T08:34:20.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.157+0000 7f90cfc4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcce30 0x560bfbdb7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.157+0000 7f90cfc4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcce30 0x560bfbdb7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48596/0 (socket says 172.21.15.172:48596) 2024-04-14T08:34:20.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.157+0000 7f90cfc4b640 1 -- 172.21.15.172:0/3598881775 learned_addr learned my addr 172.21.15.172:0/3598881775 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:20.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90cf44a640 1 --2- 172.21.15.172:0/3598881775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560bfbdcea90 0x560bfbdb6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90d044c640 1 --2- 172.21.15.172:0/3598881775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560bfbdd2640 0x560bfbdb68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90cfc4b640 1 -- 172.21.15.172:0/3598881775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560bfbdcea90 msgr2=0x560bfbdb6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90cfc4b640 1 --2- 172.21.15.172:0/3598881775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560bfbdcea90 0x560bfbdb6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90cfc4b640 1 -- 172.21.15.172:0/3598881775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560bfbdd2640 msgr2=0x560bfbdb68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90cfc4b640 1 --2- 172.21.15.172:0/3598881775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560bfbdd2640 0x560bfbdb68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90cfc4b640 1 -- 172.21.15.172:0/3598881775 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfbdb8f80 con 0x560bfbdcce30 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90cf44a640 1 --2- 172.21.15.172:0/3598881775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560bfbdcea90 0x560bfbdb6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90d044c640 1 --2- 172.21.15.172:0/3598881775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560bfbdd2640 0x560bfbdb68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90cfc4b640 1 --2- 172.21.15.172:0/3598881775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcce30 0x560bfbdb7ea0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009540 tx=0x7f90c0009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e7e06bc6e19026 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90ce448640 1 -- 172.21.15.172:0/3598881775 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0017070 con 0x560bfbdcce30 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90ce448640 1 -- 172.21.15.172:0/3598881775 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c0006600 con 0x560bfbdcce30 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90ce448640 1 -- 172.21.15.172:0/3598881775 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00069d0 con 0x560bfbdcce30 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90d2c36f40 1 -- 172.21.15.172:0/3598881775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcce30 msgr2=0x560bfbdb7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90d2c36f40 1 --2- 172.21.15.172:0/3598881775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcce30 0x560bfbdb7ea0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009540 tx=0x7f90c0009a00 comp rx=0 tx=0).stop 2024-04-14T08:34:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90d2c36f40 1 -- 172.21.15.172:0/3598881775 shutdown_connections 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90d2c36f40 1 --2- 172.21.15.172:0/3598881775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560bfbdd2640 0x560bfbdb68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90d2c36f40 1 --2- 172.21.15.172:0/3598881775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560bfbdcea90 0x560bfbdb6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.158+0000 7f90d2c36f40 1 --2- 172.21.15.172:0/3598881775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcce30 0x560bfbdb7ea0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.159+0000 7f90d2c36f40 1 -- 172.21.15.172:0/3598881775 >> 172.21.15.172:0/3598881775 conn(0x560bfbdb33d0 msgr2=0x560bfbe49d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.159+0000 7f90d2c36f40 1 -- 172.21.15.172:0/3598881775 shutdown_connections 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.159+0000 7f90d2c36f40 1 -- 172.21.15.172:0/3598881775 wait complete. 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.159+0000 7f90d2c36f40 1 Processor -- start 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.159+0000 7f90d2c36f40 1 -- start start 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90d2c36f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560bfbdcce30 0x560bfbe42860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90d2c36f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcea90 0x560bfbe5f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90d2c36f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560bfbdd2640 0x560bfbdb6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90d2c36f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560bfbdb9430 con 0x560bfbdcea90 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90d2c36f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560bfbdb9260 con 0x560bfbdcce30 2024-04-14T08:34:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90d2c36f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x560bfbd87480 con 0x560bfbdd2640 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90cf44a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcea90 0x560bfbe5f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90cf44a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcea90 0x560bfbe5f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48598/0 (socket says 172.21.15.172:48598) 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90cf44a640 1 -- 172.21.15.172:0/109570904 learned_addr learned my addr 172.21.15.172:0/109570904 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90cfc4b640 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560bfbdcce30 0x560bfbe42860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90d044c640 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560bfbdd2640 0x560bfbdb6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90cf44a640 1 -- 172.21.15.172:0/109570904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560bfbdd2640 msgr2=0x560bfbdb6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90cf44a640 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560bfbdd2640 0x560bfbdb6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90cf44a640 1 -- 172.21.15.172:0/109570904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560bfbdcce30 msgr2=0x560bfbe42860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90cf44a640 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560bfbdcce30 0x560bfbe42860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90cf44a640 1 -- 172.21.15.172:0/109570904 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfbe5f910 con 0x560bfbdcea90 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90cfc4b640 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560bfbdcce30 0x560bfbe42860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90cf44a640 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcea90 0x560bfbe5f3d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f90c401cec0 tx=0x7f90c4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.160+0000 7f90cd446640 1 -- 172.21.15.172:0/109570904 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4062020 con 0x560bfbdcea90 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.161+0000 7f90cd446640 1 -- 172.21.15.172:0/109570904 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c4052b40 con 0x560bfbdcea90 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.161+0000 7f90d2c36f40 1 -- 172.21.15.172:0/109570904 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560bfbdd0940 con 0x560bfbdcea90 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.161+0000 7f90cd446640 1 -- 172.21.15.172:0/109570904 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4057040 con 0x560bfbdcea90 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.161+0000 7f90d2c36f40 1 -- 172.21.15.172:0/109570904 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfbe49910 con 0x560bfbdcea90 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.162+0000 7f90cd446640 1 -- 172.21.15.172:0/109570904 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90c4065020 con 0x560bfbdcea90 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.162+0000 7f90cd446640 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f90bc06b2d0 0x7f90bc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.162+0000 7f90cfc4b640 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f90bc06b2d0 0x7f90bc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.162+0000 7f90cd446640 1 -- 172.21.15.172:0/109570904 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f90c40cc910 con 0x560bfbdcea90 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.162+0000 7f90d2c36f40 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x560bfbe6b350 0x560bfbd88730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.162+0000 7f90d2c36f40 1 -- 172.21.15.172:0/109570904 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) -- 0x560bfbd88c70 con 0x560bfbe6b350 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.163+0000 7f90cfc4b640 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f90bc06b2d0 0x7f90bc06d790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90c0000c00 tx=0x7f90c000f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.163+0000 7f90d044c640 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x560bfbe6b350 0x560bfbd88730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.163+0000 7f90d044c640 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x560bfbe6b350 0x560bfbd88730 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.163+0000 7f90d044c640 1 -- 172.21.15.172:0/109570904 <== osd.5 v2:172.21.15.177:6816/3172735715 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f90c8003450 con 0x560bfbe6b350 2024-04-14T08:34:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.163+0000 7f90d2c36f40 1 -- 172.21.15.172:0/109570904 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x560bfbe6b350 msgr2=0x560bfbd88730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.163+0000 7f90d2c36f40 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x560bfbe6b350 0x560bfbd88730 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.164+0000 7f90d2c36f40 1 -- 172.21.15.172:0/109570904 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f90bc06b2d0 msgr2=0x7f90bc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.164+0000 7f90d2c36f40 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f90bc06b2d0 0x7f90bc06d790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90c0000c00 tx=0x7f90c000f000 comp rx=0 tx=0).stop 2024-04-14T08:34:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.164+0000 7f90d2c36f40 1 -- 172.21.15.172:0/109570904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcea90 msgr2=0x560bfbe5f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.164+0000 7f90d2c36f40 1 --2- 172.21.15.172:0/109570904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfbdcea90 0x560bfbe5f3d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f90c401cec0 tx=0x7f90c4002fb0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.164+0000 7f90d044c640 1 -- 172.21.15.172:0/109570904 reap_dead start 2024-04-14T08:34:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.164+0000 7f90d2c36f40 1 -- 172.21.15.172:0/109570904 shutdown_connections 2024-04-14T08:34:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.164+0000 7f90d2c36f40 1 -- 172.21.15.172:0/109570904 >> 172.21.15.172:0/109570904 conn(0x560bfbdb33d0 msgr2=0x560bfbe60230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.164+0000 7f90d2c36f40 1 -- 172.21.15.172:0/109570904 shutdown_connections 2024-04-14T08:34:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.164+0000 7f90d2c36f40 1 -- 172.21.15.172:0/109570904 wait complete. 2024-04-14T08:34:20.172 INFO:teuthology.orchestra.run.smithi172.stdout:mytag 2024-04-14T08:34:20.172 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-14T08:34:20.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.213+0000 7f24a020cf40 1 Processor -- start 2024-04-14T08:34:20.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.213+0000 7f24a020cf40 1 -- start start 2024-04-14T08:34:20.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f24a020cf40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b982eca500 0x55b982f833b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f24a020cf40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b982f15510 0x55b982fa43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f24a020cf40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982fa3a80 0x55b982fa48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f24a020cf40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b982e723b0 con 0x55b982fa3a80 2024-04-14T08:34:20.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f24a020cf40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b982e76400 con 0x55b982f15510 2024-04-14T08:34:20.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f24a020cf40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55b982fa33c0 con 0x55b982eca500 2024-04-14T08:34:20.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f249d64c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982fa3a80 0x55b982fa48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f249d64c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982fa3a80 0x55b982fa48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48604/0 (socket says 172.21.15.172:48604) 2024-04-14T08:34:20.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f249d64c640 1 -- 172.21.15.172:0/344098482 learned_addr learned my addr 172.21.15.172:0/344098482 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:20.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f2497fff640 1 --2- 172.21.15.172:0/344098482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b982f15510 0x55b982fa43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f249ce4b640 1 --2- 172.21.15.172:0/344098482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b982eca500 0x55b982f833b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f249d64c640 1 -- 172.21.15.172:0/344098482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b982eca500 msgr2=0x55b982f833b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:20.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f249d64c640 1 --2- 172.21.15.172:0/344098482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b982eca500 0x55b982f833b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f249d64c640 1 -- 172.21.15.172:0/344098482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b982f15510 msgr2=0x55b982fa43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:20.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f249d64c640 1 --2- 172.21.15.172:0/344098482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b982f15510 0x55b982fa43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.214+0000 7f249d64c640 1 -- 172.21.15.172:0/344098482 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b982ece140 con 0x55b982fa3a80 2024-04-14T08:34:20.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.215+0000 7f249d64c640 1 --2- 172.21.15.172:0/344098482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982fa3a80 0x55b982fa48e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f249801b8c0 tx=0x7f249804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5890b3c1a8905 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.215+0000 7f2496ffd640 1 -- 172.21.15.172:0/344098482 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498063070 con 0x55b982fa3a80 2024-04-14T08:34:20.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.215+0000 7f2496ffd640 1 -- 172.21.15.172:0/344098482 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2498052730 con 0x55b982fa3a80 2024-04-14T08:34:20.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.215+0000 7f2496ffd640 1 -- 172.21.15.172:0/344098482 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498052ae0 con 0x55b982fa3a80 2024-04-14T08:34:20.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.215+0000 7f24a020cf40 1 -- 172.21.15.172:0/344098482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982fa3a80 msgr2=0x55b982fa48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.215+0000 7f24a020cf40 1 --2- 172.21.15.172:0/344098482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982fa3a80 0x55b982fa48e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f249801b8c0 tx=0x7f249804c000 comp rx=0 tx=0).stop 2024-04-14T08:34:20.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.215+0000 7f24a020cf40 1 -- 172.21.15.172:0/344098482 shutdown_connections 2024-04-14T08:34:20.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.215+0000 7f24a020cf40 1 --2- 172.21.15.172:0/344098482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982fa3a80 0x55b982fa48e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.215+0000 7f24a020cf40 1 --2- 172.21.15.172:0/344098482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b982f15510 0x55b982fa43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.215+0000 7f24a020cf40 1 --2- 172.21.15.172:0/344098482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b982eca500 0x55b982f833b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.215+0000 7f24a020cf40 1 -- 172.21.15.172:0/344098482 >> 172.21.15.172:0/344098482 conn(0x55b982ef7a50 msgr2=0x55b982ef6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:20.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.216+0000 7f24a020cf40 1 -- 172.21.15.172:0/344098482 shutdown_connections 2024-04-14T08:34:20.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.216+0000 7f24a020cf40 1 -- 172.21.15.172:0/344098482 wait complete. 2024-04-14T08:34:20.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.216+0000 7f24a020cf40 1 Processor -- start 2024-04-14T08:34:20.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.216+0000 7f24a020cf40 1 -- start start 2024-04-14T08:34:20.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f24a020cf40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982eca500 0x55b982f7d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f24a020cf40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b982f15510 0x55b982fa4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f24a020cf40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b982fa3a80 0x55b982fa46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f24a020cf40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b982fa5d90 con 0x55b982eca500 2024-04-14T08:34:20.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f24a020cf40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b982ef6fd0 con 0x55b982fa3a80 2024-04-14T08:34:20.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f24a020cf40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55b982fa6070 con 0x55b982f15510 2024-04-14T08:34:20.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f249ce4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982eca500 0x55b982f7d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f249ce4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982eca500 0x55b982f7d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48606/0 (socket says 172.21.15.172:48606) 2024-04-14T08:34:20.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f249ce4b640 1 -- 172.21.15.172:0/3989929925 learned_addr learned my addr 172.21.15.172:0/3989929925 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:20.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f2497fff640 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b982f15510 0x55b982fa4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f249d64c640 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b982fa3a80 0x55b982fa46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f249ce4b640 1 -- 172.21.15.172:0/3989929925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b982f15510 msgr2=0x55b982fa4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f249ce4b640 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b982f15510 0x55b982fa4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f249ce4b640 1 -- 172.21.15.172:0/3989929925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b982fa3a80 msgr2=0x55b982fa46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f249ce4b640 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b982fa3a80 0x55b982fa46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f249ce4b640 1 -- 172.21.15.172:0/3989929925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b982ef9c10 con 0x55b982eca500 2024-04-14T08:34:20.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f249d64c640 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b982fa3a80 0x55b982fa46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:20.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.217+0000 7f249ce4b640 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982eca500 0x55b982f7d4f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f249000df70 tx=0x7f249000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.218+0000 7f2495ffb640 1 -- 172.21.15.172:0/3989929925 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249000be60 con 0x55b982eca500 2024-04-14T08:34:20.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.218+0000 7f2495ffb640 1 -- 172.21.15.172:0/3989929925 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2490016070 con 0x55b982eca500 2024-04-14T08:34:20.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.218+0000 7f24a020cf40 1 -- 172.21.15.172:0/3989929925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b982ef9ef0 con 0x55b982eca500 2024-04-14T08:34:20.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.218+0000 7f2495ffb640 1 -- 172.21.15.172:0/3989929925 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24900032f0 con 0x55b982eca500 2024-04-14T08:34:20.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.218+0000 7f24a020cf40 1 -- 172.21.15.172:0/3989929925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b982f877e0 con 0x55b982eca500 2024-04-14T08:34:20.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.219+0000 7f2495ffb640 1 -- 172.21.15.172:0/3989929925 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2490011480 con 0x55b982eca500 2024-04-14T08:34:20.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.219+0000 7f2495ffb640 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f248006b2d0 0x7f248006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.220+0000 7f2495ffb640 1 -- 172.21.15.172:0/3989929925 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f249008a7a0 con 0x55b982eca500 2024-04-14T08:34:20.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.220+0000 7f2497fff640 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f248006b2d0 0x7f248006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.220+0000 7f24a020cf40 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55b982f7fb50 0x55b982f886c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.220+0000 7f24a020cf40 1 -- 172.21.15.172:0/3989929925 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) -- 0x55b982fad6e0 con 0x55b982f7fb50 2024-04-14T08:34:20.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.220+0000 7f2497fff640 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f248006b2d0 0x7f248006d790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55b982ecc9b0 tx=0x7f24880012c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.220+0000 7f249d64c640 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55b982f7fb50 0x55b982f886c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f249d64c640 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55b982f7fb50 0x55b982f886c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f249d64c640 1 -- 172.21.15.172:0/3989929925 <== osd.5 v2:172.21.15.177:6816/3172735715 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f2498052be0 con 0x55b982f7fb50 2024-04-14T08:34:20.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f24a020cf40 1 -- 172.21.15.172:0/3989929925 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55b982f7fb50 msgr2=0x55b982f886c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f24a020cf40 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55b982f7fb50 0x55b982f886c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f24a020cf40 1 -- 172.21.15.172:0/3989929925 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f248006b2d0 msgr2=0x7f248006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f24a020cf40 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f248006b2d0 0x7f248006d790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55b982ecc9b0 tx=0x7f24880012c0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f24a020cf40 1 -- 172.21.15.172:0/3989929925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982eca500 msgr2=0x55b982f7d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f24a020cf40 1 --2- 172.21.15.172:0/3989929925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b982eca500 0x55b982f7d4f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f249000df70 tx=0x7f249000b5b0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f249d64c640 1 -- 172.21.15.172:0/3989929925 reap_dead start 2024-04-14T08:34:20.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f24a020cf40 1 -- 172.21.15.172:0/3989929925 shutdown_connections 2024-04-14T08:34:20.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f24a020cf40 1 -- 172.21.15.172:0/3989929925 >> 172.21.15.172:0/3989929925 conn(0x55b982ef7a50 msgr2=0x55b982ef6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:20.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f24a020cf40 1 -- 172.21.15.172:0/3989929925 shutdown_connections 2024-04-14T08:34:20.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.221+0000 7f24a020cf40 1 -- 172.21.15.172:0/3989929925 wait complete. 2024-04-14T08:34:20.232 INFO:teuthology.orchestra.run.smithi172.stdout:mytag 2024-04-14T08:34:20.232 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-14T08:34:20.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.316+0000 7fa6cb041f40 1 Processor -- start 2024-04-14T08:34:20.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.316+0000 7fa6cb041f40 1 -- start start 2024-04-14T08:34:20.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.316+0000 7fa6cb041f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6b8002460 0x55db98487e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.316+0000 7fa6cb041f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853caa0 0x55db98467f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.316+0000 7fa6cb041f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55db9853b520 0x55db9853b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.316+0000 7fa6cb041f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55db9840d3b0 con 0x55db9853caa0 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.316+0000 7fa6cb041f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55db98411400 con 0x55db9853b520 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.316+0000 7fa6cb041f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55db98539b40 con 0x7fa6b8002460 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.316+0000 7fa6c3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853caa0 0x55db98467f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.316+0000 7fa6c3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853caa0 0x55db98467f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48628/0 (socket says 172.21.15.172:48628) 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.316+0000 7fa6c3fff640 1 -- 172.21.15.172:0/3106829617 learned_addr learned my addr 172.21.15.172:0/3106829617 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6c8943640 1 --2- 172.21.15.172:0/3106829617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6b8002460 0x55db98487e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6c9144640 1 --2- 172.21.15.172:0/3106829617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55db9853b520 0x55db9853b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6c3fff640 1 -- 172.21.15.172:0/3106829617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6b8002460 msgr2=0x55db98487e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6c3fff640 1 --2- 172.21.15.172:0/3106829617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6b8002460 0x55db98487e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6c3fff640 1 -- 172.21.15.172:0/3106829617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55db9853b520 msgr2=0x55db9853b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6c3fff640 1 --2- 172.21.15.172:0/3106829617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55db9853b520 0x55db9853b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6c3fff640 1 -- 172.21.15.172:0/3106829617 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db98539db0 con 0x55db9853caa0 2024-04-14T08:34:20.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6c9144640 1 --2- 172.21.15.172:0/3106829617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55db9853b520 0x55db9853b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:20.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6c3fff640 1 --2- 172.21.15.172:0/3106829617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853caa0 0x55db98467f80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400d880 tx=0x7fa6b400d540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82dd68a120c622f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6c2ffd640 1 -- 172.21.15.172:0/3106829617 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400bb70 con 0x55db9853caa0 2024-04-14T08:34:20.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6c2ffd640 1 -- 172.21.15.172:0/3106829617 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b400bd10 con 0x55db9853caa0 2024-04-14T08:34:20.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6c2ffd640 1 -- 172.21.15.172:0/3106829617 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4016630 con 0x55db9853caa0 2024-04-14T08:34:20.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6cb041f40 1 -- 172.21.15.172:0/3106829617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853caa0 msgr2=0x55db98467f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6cb041f40 1 --2- 172.21.15.172:0/3106829617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853caa0 0x55db98467f80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400d880 tx=0x7fa6b400d540 comp rx=0 tx=0).stop 2024-04-14T08:34:20.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6cb041f40 1 -- 172.21.15.172:0/3106829617 shutdown_connections 2024-04-14T08:34:20.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6cb041f40 1 --2- 172.21.15.172:0/3106829617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55db9853b520 0x55db9853b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6cb041f40 1 --2- 172.21.15.172:0/3106829617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853caa0 0x55db98467f80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6cb041f40 1 --2- 172.21.15.172:0/3106829617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6b8002460 0x55db98487e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6cb041f40 1 -- 172.21.15.172:0/3106829617 >> 172.21.15.172:0/3106829617 conn(0x55db98492a50 msgr2=0x55db9853c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6cb041f40 1 -- 172.21.15.172:0/3106829617 shutdown_connections 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.317+0000 7fa6cb041f40 1 -- 172.21.15.172:0/3106829617 wait complete. 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.318+0000 7fa6cb041f40 1 Processor -- start 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.318+0000 7fa6cb041f40 1 -- start start 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6cb041f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853b520 0x55db98534a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6cb041f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55db9853caa0 0x55db9848fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6cb041f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6b8002460 0x55db984902f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6cb041f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55db98538f90 con 0x55db9853b520 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6cb041f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55db9853ad20 con 0x55db9853caa0 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6cb041f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55db985392a0 con 0x7fa6b8002460 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6c8943640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853b520 0x55db98534a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6c8943640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853b520 0x55db98534a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48640/0 (socket says 172.21.15.172:48640) 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6c8943640 1 -- 172.21.15.172:0/959276958 learned_addr learned my addr 172.21.15.172:0/959276958 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6c3fff640 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55db9853caa0 0x55db9848fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6c9144640 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6b8002460 0x55db984902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6c8943640 1 -- 172.21.15.172:0/959276958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6b8002460 msgr2=0x55db984902f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6c8943640 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6b8002460 0x55db984902f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6c8943640 1 -- 172.21.15.172:0/959276958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55db9853caa0 msgr2=0x55db9848fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6c8943640 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55db9853caa0 0x55db9848fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6c8943640 1 -- 172.21.15.172:0/959276958 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db98538890 con 0x55db9853b520 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6c9144640 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6b8002460 0x55db984902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.319+0000 7fa6c8943640 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853b520 0x55db98534a60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc01d1e0 tx=0x7fa6bc005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.320+0000 7fa6c1ffb640 1 -- 172.21.15.172:0/959276958 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc057040 con 0x55db9853b520 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.320+0000 7fa6c1ffb640 1 -- 172.21.15.172:0/959276958 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6bc052a30 con 0x55db9853b520 2024-04-14T08:34:20.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.320+0000 7fa6c1ffb640 1 -- 172.21.15.172:0/959276958 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc052e20 con 0x55db9853b520 2024-04-14T08:34:20.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.320+0000 7fa6cb041f40 1 -- 172.21.15.172:0/959276958 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55db98490a80 con 0x55db9853b520 2024-04-14T08:34:20.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.320+0000 7fa6cb041f40 1 -- 172.21.15.172:0/959276958 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9853c050 con 0x55db9853b520 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.321+0000 7fa6c1ffb640 1 -- 172.21.15.172:0/959276958 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6bc05a040 con 0x55db9853b520 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.322+0000 7fa6c1ffb640 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6ac06b2d0 0x7fa6ac06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.322+0000 7fa6c3fff640 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6ac06b2d0 0x7fa6ac06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.322+0000 7fa6c1ffb640 1 -- 172.21.15.172:0/959276958 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa6bc0cbee0 con 0x55db9853b520 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.322+0000 7fa6cb041f40 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55db985d19c0 0x55db985d9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.322+0000 7fa6cb041f40 1 -- 172.21.15.172:0/959276958 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) -- 0x55db985d1470 con 0x55db985d19c0 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.322+0000 7fa6c3fff640 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6ac06b2d0 0x7fa6ac06d790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55db98493540 tx=0x7fa6b4009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.322+0000 7fa6c9144640 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55db985d19c0 0x55db985d9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6c9144640 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55db985d19c0 0x55db985d9b90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6c9144640 1 -- 172.21.15.172:0/959276958 <== osd.5 v2:172.21.15.177:6816/3172735715 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fa6c40036c0 con 0x55db985d19c0 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6cb041f40 1 -- 172.21.15.172:0/959276958 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55db985d19c0 msgr2=0x55db985d9b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6cb041f40 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55db985d19c0 0x55db985d9b90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6cb041f40 1 -- 172.21.15.172:0/959276958 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6ac06b2d0 msgr2=0x7fa6ac06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6cb041f40 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6ac06b2d0 0x7fa6ac06d790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55db98493540 tx=0x7fa6b4009000 comp rx=0 tx=0).stop 2024-04-14T08:34:20.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6cb041f40 1 -- 172.21.15.172:0/959276958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853b520 msgr2=0x55db98534a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6cb041f40 1 --2- 172.21.15.172:0/959276958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55db9853b520 0x55db98534a60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc01d1e0 tx=0x7fa6bc005a00 comp rx=0 tx=0).stop 2024-04-14T08:34:20.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6c9144640 1 -- 172.21.15.172:0/959276958 reap_dead start 2024-04-14T08:34:20.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6cb041f40 1 -- 172.21.15.172:0/959276958 shutdown_connections 2024-04-14T08:34:20.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6cb041f40 1 -- 172.21.15.172:0/959276958 >> 172.21.15.172:0/959276958 conn(0x55db98492a50 msgr2=0x55db9853c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:20.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6cb041f40 1 -- 172.21.15.172:0/959276958 shutdown_connections 2024-04-14T08:34:20.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.323+0000 7fa6cb041f40 1 -- 172.21.15.172:0/959276958 wait complete. 2024-04-14T08:34:20.330 INFO:teuthology.orchestra.run.smithi172.stdout:mytag 2024-04-14T08:34:20.330 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-14T08:34:20.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.371+0000 7fa64a63bf40 1 Processor -- start 2024-04-14T08:34:20.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.371+0000 7fa64a63bf40 1 -- start start 2024-04-14T08:34:20.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa64a63bf40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56262b5dd330 0x56262b625770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa64a63bf40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262b628510 0x56262b60c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa64a63bf40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b6b6a80 0x56262b6094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa64a63bf40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56262b627870 con 0x56262b6b6a80 2024-04-14T08:34:20.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa64a63bf40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56262b5853b0 con 0x56262b628510 2024-04-14T08:34:20.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa64a63bf40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56262b627b10 con 0x56262b5dd330 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa647fa4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b6b6a80 0x56262b6094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa647fa4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b6b6a80 0x56262b6094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48668/0 (socket says 172.21.15.172:48668) 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa647fa4640 1 -- 172.21.15.172:0/161888688 learned_addr learned my addr 172.21.15.172:0/161888688 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa6477a3640 1 --2- 172.21.15.172:0/161888688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56262b5dd330 0x56262b625770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa646fa2640 1 --2- 172.21.15.172:0/161888688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262b628510 0x56262b60c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa647fa4640 1 -- 172.21.15.172:0/161888688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56262b5dd330 msgr2=0x56262b625770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa647fa4640 1 --2- 172.21.15.172:0/161888688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56262b5dd330 0x56262b625770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa647fa4640 1 -- 172.21.15.172:0/161888688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262b628510 msgr2=0x56262b60c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa647fa4640 1 --2- 172.21.15.172:0/161888688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262b628510 0x56262b60c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa647fa4640 1 -- 172.21.15.172:0/161888688 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262b5e1280 con 0x56262b6b6a80 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa647fa4640 1 --2- 172.21.15.172:0/161888688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b6b6a80 0x56262b6094d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa640002630 tx=0x7fa640002af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8422e2a40c12043 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa645fa0640 1 -- 172.21.15.172:0/161888688 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640016070 con 0x56262b6b6a80 2024-04-14T08:34:20.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa645fa0640 1 -- 172.21.15.172:0/161888688 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa64000d6b0 con 0x56262b6b6a80 2024-04-14T08:34:20.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.372+0000 7fa645fa0640 1 -- 172.21.15.172:0/161888688 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64000daa0 con 0x56262b6b6a80 2024-04-14T08:34:20.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.373+0000 7fa64a63bf40 1 -- 172.21.15.172:0/161888688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b6b6a80 msgr2=0x56262b6094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.373+0000 7fa64a63bf40 1 --2- 172.21.15.172:0/161888688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b6b6a80 0x56262b6094d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa640002630 tx=0x7fa640002af0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.373+0000 7fa64a63bf40 1 -- 172.21.15.172:0/161888688 shutdown_connections 2024-04-14T08:34:20.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.373+0000 7fa64a63bf40 1 --2- 172.21.15.172:0/161888688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b6b6a80 0x56262b6094d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.373+0000 7fa64a63bf40 1 --2- 172.21.15.172:0/161888688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262b628510 0x56262b60c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.373+0000 7fa64a63bf40 1 --2- 172.21.15.172:0/161888688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56262b5dd330 0x56262b625770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.373+0000 7fa64a63bf40 1 -- 172.21.15.172:0/161888688 >> 172.21.15.172:0/161888688 conn(0x56262b60aa50 msgr2=0x56262b698470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:20.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.373+0000 7fa64a63bf40 1 -- 172.21.15.172:0/161888688 shutdown_connections 2024-04-14T08:34:20.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.373+0000 7fa64a63bf40 1 -- 172.21.15.172:0/161888688 wait complete. 2024-04-14T08:34:20.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.373+0000 7fa64a63bf40 1 Processor -- start 2024-04-14T08:34:20.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.374+0000 7fa64a63bf40 1 -- start start 2024-04-14T08:34:20.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.374+0000 7fa64a63bf40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b5dd330 0x56262b6afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.374+0000 7fa64a63bf40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56262b628510 0x56262b624af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.374+0000 7fa64a63bf40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262b6b6a80 0x56262b74ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.374+0000 7fa64a63bf40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56262b60bd90 con 0x56262b5dd330 2024-04-14T08:34:20.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.374+0000 7fa64a63bf40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56262b60bb40 con 0x56262b6b6a80 2024-04-14T08:34:20.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.374+0000 7fa64a63bf40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56262b60c070 con 0x56262b628510 2024-04-14T08:34:20.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.374+0000 7fa6477a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b5dd330 0x56262b6afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.374+0000 7fa6477a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b5dd330 0x56262b6afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48678/0 (socket says 172.21.15.172:48678) 2024-04-14T08:34:20.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.374+0000 7fa6477a3640 1 -- 172.21.15.172:0/3222011211 learned_addr learned my addr 172.21.15.172:0/3222011211 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:20.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa646fa2640 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56262b628510 0x56262b624af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa647fa4640 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262b6b6a80 0x56262b74ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa6477a3640 1 -- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56262b628510 msgr2=0x56262b624af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa6477a3640 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56262b628510 0x56262b624af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa6477a3640 1 -- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262b6b6a80 msgr2=0x56262b74ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa6477a3640 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262b6b6a80 0x56262b74ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa6477a3640 1 -- 172.21.15.172:0/3222011211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262b624610 con 0x56262b5dd330 2024-04-14T08:34:20.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa646fa2640 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56262b628510 0x56262b624af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:20.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa647fa4640 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262b6b6a80 0x56262b74ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:20.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa6477a3640 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b5dd330 0x56262b6afab0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa63801d050 tx=0x7fa638002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa644f9e640 1 -- 172.21.15.172:0/3222011211 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638063020 con 0x56262b5dd330 2024-04-14T08:34:20.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa644f9e640 1 -- 172.21.15.172:0/3222011211 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa638058040 con 0x56262b5dd330 2024-04-14T08:34:20.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa64a63bf40 1 -- 172.21.15.172:0/3222011211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56262b6220d0 con 0x56262b5dd330 2024-04-14T08:34:20.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa644f9e640 1 -- 172.21.15.172:0/3222011211 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638057040 con 0x56262b5dd330 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.375+0000 7fa64a63bf40 1 -- 172.21.15.172:0/3222011211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56262b622670 con 0x56262b5dd330 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.377+0000 7fa644f9e640 1 -- 172.21.15.172:0/3222011211 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa638066020 con 0x56262b5dd330 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.377+0000 7fa644f9e640 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa63406b2d0 0x7fa63406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.377+0000 7fa644f9e640 1 -- 172.21.15.172:0/3222011211 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa6380cca60 con 0x56262b5dd330 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.377+0000 7fa646fa2640 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa63406b2d0 0x7fa63406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.378+0000 7fa64a63bf40 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x56262b74f180 0x56262b60d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.378+0000 7fa64a63bf40 1 -- 172.21.15.172:0/3222011211 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39) -- 0x56262b6afff0 con 0x56262b74f180 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.378+0000 7fa646fa2640 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa63406b2d0 0x7fa63406d790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x56262b698b30 tx=0x7fa63c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.378+0000 7fa647fa4640 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x56262b74f180 0x56262b60d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.378+0000 7fa647fa4640 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x56262b74f180 0x56262b60d150 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.379+0000 7fa647fa4640 1 -- 172.21.15.172:0/3222011211 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7fa640019040 con 0x56262b74f180 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-14T08:34:20.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.379+0000 7fa64a63bf40 1 -- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x56262b74f180 msgr2=0x56262b60d150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.379+0000 7fa64a63bf40 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x56262b74f180 0x56262b60d150 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.379+0000 7fa64a63bf40 1 -- 172.21.15.172:0/3222011211 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa63406b2d0 msgr2=0x7fa63406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.379+0000 7fa64a63bf40 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa63406b2d0 0x7fa63406d790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x56262b698b30 tx=0x7fa63c00b040 comp rx=0 tx=0).stop 2024-04-14T08:34:20.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.379+0000 7fa64a63bf40 1 -- 172.21.15.172:0/3222011211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b5dd330 msgr2=0x56262b6afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.379+0000 7fa64a63bf40 1 --2- 172.21.15.172:0/3222011211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56262b5dd330 0x56262b6afab0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa63801d050 tx=0x7fa638002fb0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.379+0000 7fa647fa4640 1 -- 172.21.15.172:0/3222011211 reap_dead start 2024-04-14T08:34:20.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.379+0000 7fa64a63bf40 1 -- 172.21.15.172:0/3222011211 shutdown_connections 2024-04-14T08:34:20.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.379+0000 7fa64a63bf40 1 -- 172.21.15.172:0/3222011211 >> 172.21.15.172:0/3222011211 conn(0x56262b60aa50 msgr2=0x56262b60c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:20.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.379+0000 7fa64a63bf40 1 -- 172.21.15.172:0/3222011211 shutdown_connections 2024-04-14T08:34:20.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.379+0000 7fa64a63bf40 1 -- 172.21.15.172:0/3222011211 wait complete. 2024-04-14T08:34:20.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:34:20.387 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:34:20.387 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:34:20.477 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:34:20.528 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:34:20.528 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:34:20.528 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:34:20.528 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:34:20.545 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:34:20.545 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:34:20.545 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:34:20.545 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:34:20.562 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:34:20.562 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:34:20.562 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:34:20.562 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:34:20.601 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:34:20.601 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:34:20.601 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:34:20.601 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:34:20.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:34:20.619 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:34:20.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.741+0000 7f69694d1640 1 Processor -- start 2024-04-14T08:34:20.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.741+0000 7f69694d1640 1 -- start start 2024-04-14T08:34:20.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f69694d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69640d2830 0x7f69640d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f69694d1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f696415e890 0x7f69640d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f69694d1640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69640d3150 con 0x7f69640cedd0 2024-04-14T08:34:20.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f69694d1640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f69640cff30 con 0x7f696415e890 2024-04-14T08:34:20.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f69694d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f69640d2350 con 0x7f69640d2830 2024-04-14T08:34:20.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f69627fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f696415e890 0x7f69640d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f69637fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f69640cedd0 0x7f69641621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35620/0 (socket says 172.21.15.172:35620) 2024-04-14T08:34:20.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f69637fe640 1 -- 172.21.15.172:0/2972559222 learned_addr learned my addr 172.21.15.172:0/2972559222 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:20.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f69627fc640 1 -- 172.21.15.172:0/2972559222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69640d2830 msgr2=0x7f69640d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:20.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f69627fc640 1 --2- 172.21.15.172:0/2972559222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69640d2830 0x7f69640d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f69627fc640 1 -- 172.21.15.172:0/2972559222 >> v1:172.21.15.172:6789/0 conn(0x7f69640cedd0 legacy=0x7f69641621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f6961ffb640 1 -- 172.21.15.172:0/2972559222 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4069690131 0 0) 0x7f69640d3150 con 0x7f69640cedd0 2024-04-14T08:34:20.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.742+0000 7f69627fc640 1 -- 172.21.15.172:0/2972559222 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69640cf320 con 0x7f696415e890 2024-04-14T08:34:20.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.743+0000 7f69627fc640 1 --2- 172.21.15.172:0/2972559222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f696415e890 0x7f69640d1300 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f69580097a0 tx=0x7f695802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5800b279ce9b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.743+0000 7f6961ffb640 1 -- 172.21.15.172:0/2972559222 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958034070 con 0x7f696415e890 2024-04-14T08:34:20.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.743+0000 7f6961ffb640 1 -- 172.21.15.172:0/2972559222 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6958030610 con 0x7f696415e890 2024-04-14T08:34:20.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.743+0000 7f6961ffb640 1 -- 172.21.15.172:0/2972559222 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69580309e0 con 0x7f696415e890 2024-04-14T08:34:20.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.744+0000 7f69694d1640 1 -- 172.21.15.172:0/2972559222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f696415e890 msgr2=0x7f69640d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.744+0000 7f69694d1640 1 --2- 172.21.15.172:0/2972559222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f696415e890 0x7f69640d1300 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f69580097a0 tx=0x7f695802b960 comp rx=0 tx=0).stop 2024-04-14T08:34:20.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.744+0000 7f69694d1640 1 -- 172.21.15.172:0/2972559222 shutdown_connections 2024-04-14T08:34:20.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.744+0000 7f69694d1640 1 --2- 172.21.15.172:0/2972559222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f696415e890 0x7f69640d1300 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.744+0000 7f69694d1640 1 --2- 172.21.15.172:0/2972559222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69640d2830 0x7f69640d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.744+0000 7f69694d1640 1 -- 172.21.15.172:0/2972559222 >> 172.21.15.172:0/2972559222 conn(0x7f696409d590 msgr2=0x7f69640d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:20.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.744+0000 7f69694d1640 1 -- 172.21.15.172:0/2972559222 shutdown_connections 2024-04-14T08:34:20.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.744+0000 7f69694d1640 1 -- 172.21.15.172:0/2972559222 wait complete. 2024-04-14T08:34:20.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.745+0000 7f69694d1640 1 Processor -- start 2024-04-14T08:34:20.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.745+0000 7f69694d1640 1 -- start start 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69694d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69640cedd0 0x7f69640b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69694d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69640d2830 0x7f69640cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69694d1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f696415e890 0x7f696415ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69694d1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69640cfbc0 con 0x7f69640d2830 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69694d1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f69640cf8e0 con 0x7f696415e890 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69694d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f69640d1840 con 0x7f69640cedd0 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69627fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69640d2830 0x7f69640cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69627fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69640d2830 0x7f69640cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48706/0 (socket says 172.21.15.172:48706) 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69627fc640 1 -- 172.21.15.172:0/4055667662 learned_addr learned my addr 172.21.15.172:0/4055667662 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69627fc640 1 -- 172.21.15.172:0/4055667662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69640cedd0 msgr2=0x7f69640b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f6962ffd640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69640cedd0 0x7f69640b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69627fc640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69640cedd0 0x7f69640b89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69627fc640 1 -- 172.21.15.172:0/4055667662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f696415e890 msgr2=0x7f696415ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69637fe640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f696415e890 0x7f696415ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69627fc640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f696415e890 0x7f696415ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69627fc640 1 -- 172.21.15.172:0/4055667662 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6964158230 con 0x7f69640d2830 2024-04-14T08:34:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f6962ffd640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69640cedd0 0x7f69640b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.746+0000 7f69637fe640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f696415e890 0x7f696415ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.747+0000 7f69627fc640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69640d2830 0x7f69640cb740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6958000c00 tx=0x7f69580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.747+0000 7f694b7fe640 1 -- 172.21.15.172:0/4055667662 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958034040 con 0x7f69640d2830 2024-04-14T08:34:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.747+0000 7f694b7fe640 1 -- 172.21.15.172:0/4055667662 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6958004030 con 0x7f69640d2830 2024-04-14T08:34:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.747+0000 7f69694d1640 1 -- 172.21.15.172:0/4055667662 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69640b8f00 con 0x7f69640d2830 2024-04-14T08:34:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.747+0000 7f694b7fe640 1 -- 172.21.15.172:0/4055667662 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958004420 con 0x7f69640d2830 2024-04-14T08:34:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.747+0000 7f69694d1640 1 -- 172.21.15.172:0/4055667662 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69640c9a50 con 0x7f69640d2830 2024-04-14T08:34:20.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.748+0000 7f69694d1640 1 -- 172.21.15.172:0/4055667662 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6964068520 con 0x7f69640d2830 2024-04-14T08:34:20.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.748+0000 7f694b7fe640 1 -- 172.21.15.172:0/4055667662 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6958037070 con 0x7f69640d2830 2024-04-14T08:34:20.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.749+0000 7f694b7fe640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f695006c8f0 0x7f695006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:20.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.749+0000 7f694b7fe640 1 -- 172.21.15.172:0/4055667662 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f69580ad920 con 0x7f69640d2830 2024-04-14T08:34:20.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.749+0000 7f6962ffd640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f695006c8f0 0x7f695006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:20.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.749+0000 7f6962ffd640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f695006c8f0 0x7f695006edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f695400a870 tx=0x7f6954008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:20.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.753+0000 7f694b7fe640 1 -- 172.21.15.172:0/4055667662 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f695807bd70 con 0x7f69640d2830 2024-04-14T08:34:20.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.985+0000 7f69694d1640 1 -- 172.21.15.172:0/4055667662 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69640d2610 con 0x7f69640d2830 2024-04-14T08:34:20.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.985+0000 7f694b7fe640 1 -- 172.21.15.172:0/4055667662 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7f695807b670 con 0x7f69640d2830 2024-04-14T08:34:20.987 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:20.987 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1038094401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1038094401},{"type":"v1","addr":"172.21.15.177:6834","nonce":1038094401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1119512744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1119512744},{"type":"v1","addr":"172.21.15.177:6835","nonce":1119512744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3054138283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3054138283},{"type":"v1","addr":"172.21.15.172:6837","nonce":3054138283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:33:59.340499+0000","modified":"2024-04-14T08:34:01.467567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5308":{"gid":5308,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6836/3420108191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3420108191},{"type":"v1","addr":"172.21.15.172:6836","nonce":3420108191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5308,"qdb_cluster":[5308]},"id":2}]} 2024-04-14T08:34:20.988 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 16 2024-04-14T08:34:20.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.988+0000 7f69694d1640 1 -- 172.21.15.172:0/4055667662 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f695006c8f0 msgr2=0x7f695006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.988+0000 7f69694d1640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f695006c8f0 0x7f695006edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f695400a870 tx=0x7f6954008040 comp rx=0 tx=0).stop 2024-04-14T08:34:20.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.988+0000 7f69694d1640 1 -- 172.21.15.172:0/4055667662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69640d2830 msgr2=0x7f69640cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:20.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.988+0000 7f69694d1640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69640d2830 0x7f69640cb740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6958000c00 tx=0x7f69580046c0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.988+0000 7f69694d1640 1 -- 172.21.15.172:0/4055667662 shutdown_connections 2024-04-14T08:34:20.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.988+0000 7f69694d1640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f695006c8f0 0x7f695006edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.988+0000 7f69694d1640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f696415e890 0x7f696415ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.988+0000 7f69694d1640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69640d2830 0x7f69640cb740 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.988+0000 7f69694d1640 1 --2- 172.21.15.172:0/4055667662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69640cedd0 0x7f69640b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:20.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.988+0000 7f69694d1640 1 -- 172.21.15.172:0/4055667662 >> 172.21.15.172:0/4055667662 conn(0x7f696409d590 msgr2=0x7f69640d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:20.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.988+0000 7f69694d1640 1 -- 172.21.15.172:0/4055667662 shutdown_connections 2024-04-14T08:34:20.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:20.988+0000 7f69694d1640 1 -- 172.21.15.172:0/4055667662 wait complete. 2024-04-14T08:34:21.004 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:34:21.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.104+0000 7f67ef0de640 1 Processor -- start 2024-04-14T08:34:21.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.105+0000 7f67ef0de640 1 -- start start 2024-04-14T08:34:21.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.105+0000 7f67ef0de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67e80d2830 0x7f67e80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.105+0000 7f67ef0de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f67e815e890 0x7f67e80d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.105+0000 7f67ef0de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67e80d3150 con 0x7f67e80cedd0 2024-04-14T08:34:21.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.105+0000 7f67ef0de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f67e80cff30 con 0x7f67e815e890 2024-04-14T08:34:21.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.105+0000 7f67ef0de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f67e80d2350 con 0x7f67e80d2830 2024-04-14T08:34:21.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.106+0000 7f67ece7a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67e80d2830 0x7f67e80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.106+0000 7f67dffff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f67e815e890 0x7f67e80d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.106+0000 7f67ece7a640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67e80d2830 0x7f67e80d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:54398/0 (socket says 172.21.15.172:54398) 2024-04-14T08:34:21.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.106+0000 7f67ece7a640 1 -- 172.21.15.172:0/3852453451 learned_addr learned my addr 172.21.15.172:0/3852453451 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:21.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.106+0000 7f67dffff640 1 -- 172.21.15.172:0/3852453451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67e80d2830 msgr2=0x7f67e80d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:21.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.106+0000 7f67dffff640 1 --2- 172.21.15.172:0/3852453451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67e80d2830 0x7f67e80d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.106+0000 7f67dffff640 1 -- 172.21.15.172:0/3852453451 >> v1:172.21.15.177:6789/0 conn(0x7f67e80cedd0 legacy=0x7f67e81621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.106+0000 7f67dffff640 1 -- 172.21.15.172:0/3852453451 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e80cf320 con 0x7f67e815e890 2024-04-14T08:34:21.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.106+0000 7f67df7fe640 1 -- 172.21.15.172:0/3852453451 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3465666076 0 0) 0x7f67e80d3150 con 0x7f67e80cedd0 2024-04-14T08:34:21.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.106+0000 7f67ece7a640 1 --2- 172.21.15.172:0/3852453451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67e80d2830 0x7f67e80d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:34:21.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.107+0000 7f67dffff640 1 --2- 172.21.15.172:0/3852453451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e815e890 0x7f67e80d1300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f67d80097a0 tx=0x7f67d802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa18f59e90eaedfd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:21.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.107+0000 7f67df7fe640 1 -- 172.21.15.172:0/3852453451 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8034070 con 0x7f67e815e890 2024-04-14T08:34:21.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.107+0000 7f67df7fe640 1 -- 172.21.15.172:0/3852453451 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d8030800 con 0x7f67e815e890 2024-04-14T08:34:21.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.107+0000 7f67df7fe640 1 -- 172.21.15.172:0/3852453451 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8030bd0 con 0x7f67e815e890 2024-04-14T08:34:21.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.107+0000 7f67ef0de640 1 -- 172.21.15.172:0/3852453451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e815e890 msgr2=0x7f67e80d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.107+0000 7f67ef0de640 1 --2- 172.21.15.172:0/3852453451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e815e890 0x7f67e80d1300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f67d80097a0 tx=0x7f67d802b940 comp rx=0 tx=0).stop 2024-04-14T08:34:21.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.108+0000 7f67ef0de640 1 -- 172.21.15.172:0/3852453451 shutdown_connections 2024-04-14T08:34:21.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.108+0000 7f67ef0de640 1 --2- 172.21.15.172:0/3852453451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e815e890 0x7f67e80d1300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.108+0000 7f67ef0de640 1 --2- 172.21.15.172:0/3852453451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67e80d2830 0x7f67e80d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.108+0000 7f67ef0de640 1 -- 172.21.15.172:0/3852453451 >> 172.21.15.172:0/3852453451 conn(0x7f67e809d590 msgr2=0x7f67e80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:21.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.108+0000 7f67ef0de640 1 -- 172.21.15.172:0/3852453451 shutdown_connections 2024-04-14T08:34:21.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.108+0000 7f67ef0de640 1 -- 172.21.15.172:0/3852453451 wait complete. 2024-04-14T08:34:21.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.109+0000 7f67ef0de640 1 Processor -- start 2024-04-14T08:34:21.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.109+0000 7f67ef0de640 1 -- start start 2024-04-14T08:34:21.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67ef0de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e80cedd0 0x7f67e80b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67ef0de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67e80d2830 0x7f67e80c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67ef0de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67e815e890 0x7f67e8158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67ef0de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f67e80cfbc0 con 0x7f67e80d2830 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67ef0de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67e80cf8e0 con 0x7f67e80cedd0 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67ef0de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f67e80d1840 con 0x7f67e815e890 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67dffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67e80d2830 0x7f67e80c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67dffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67e80d2830 0x7f67e80c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48716/0 (socket says 172.21.15.172:48716) 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67dffff640 1 -- 172.21.15.172:0/3854701788 learned_addr learned my addr 172.21.15.172:0/3854701788 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67ece7a640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e80cedd0 0x7f67e80b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67ed67b640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67e815e890 0x7f67e8158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67dffff640 1 -- 172.21.15.172:0/3854701788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67e815e890 msgr2=0x7f67e8158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67dffff640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67e815e890 0x7f67e8158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67dffff640 1 -- 172.21.15.172:0/3854701788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e80cedd0 msgr2=0x7f67e80b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67dffff640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e80cedd0 0x7f67e80b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67dffff640 1 -- 172.21.15.172:0/3854701788 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e81f8710 con 0x7f67e80d2830 2024-04-14T08:34:21.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.111+0000 7f67dffff640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67e80d2830 0x7f67e80c9b40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f67d8000c00 tx=0x7f67d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:21.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.112+0000 7f67dd7fa640 1 -- 172.21.15.172:0/3854701788 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8034040 con 0x7f67e80d2830 2024-04-14T08:34:21.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.112+0000 7f67dd7fa640 1 -- 172.21.15.172:0/3854701788 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d8030600 con 0x7f67e80d2830 2024-04-14T08:34:21.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.112+0000 7f67dd7fa640 1 -- 172.21.15.172:0/3854701788 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8006e10 con 0x7f67e80d2830 2024-04-14T08:34:21.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.112+0000 7f67ef0de640 1 -- 172.21.15.172:0/3854701788 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e80b7080 con 0x7f67e80d2830 2024-04-14T08:34:21.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.112+0000 7f67ef0de640 1 -- 172.21.15.172:0/3854701788 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e80b7640 con 0x7f67e80d2830 2024-04-14T08:34:21.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.113+0000 7f67ef0de640 1 -- 172.21.15.172:0/3854701788 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e8068520 con 0x7f67e80d2830 2024-04-14T08:34:21.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.113+0000 7f67dd7fa640 1 -- 172.21.15.172:0/3854701788 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67d804c060 con 0x7f67e80d2830 2024-04-14T08:34:21.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.114+0000 7f67dd7fa640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f67d406c8f0 0x7f67d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.114+0000 7f67ece7a640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f67d406c8f0 0x7f67d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.114+0000 7f67dd7fa640 1 -- 172.21.15.172:0/3854701788 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f67d80ad6f0 con 0x7f67e80d2830 2024-04-14T08:34:21.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.114+0000 7f67ece7a640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f67d406c8f0 0x7f67d406edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f67e4006ca0 tx=0x7f67e4009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:21.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.118+0000 7f67dd7fa640 1 -- 172.21.15.172:0/3854701788 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f67d80307d0 con 0x7f67e80d2830 2024-04-14T08:34:21.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.303+0000 7f67ef0de640 1 -- 172.21.15.172:0/3854701788 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67e80d2610 con 0x7f67e80d2830 2024-04-14T08:34:21.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.304+0000 7f67dd7fa640 1 -- 172.21.15.172:0/3854701788 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17002 (secure 0 0 0) 0x7f67d807b4f0 con 0x7f67e80d2830 2024-04-14T08:34:21.306 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:21.306 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:00.984153+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:33:57.868995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-14T08:33:58.323549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:34:21.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.306+0000 7f67ef0de640 1 -- 172.21.15.172:0/3854701788 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f67d406c8f0 msgr2=0x7f67d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.306+0000 7f67ef0de640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f67d406c8f0 0x7f67d406edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f67e4006ca0 tx=0x7f67e4009240 comp rx=0 tx=0).stop 2024-04-14T08:34:21.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.306+0000 7f67ef0de640 1 -- 172.21.15.172:0/3854701788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67e80d2830 msgr2=0x7f67e80c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.306+0000 7f67ef0de640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67e80d2830 0x7f67e80c9b40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f67d8000c00 tx=0x7f67d80046c0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.306+0000 7f67ef0de640 1 -- 172.21.15.172:0/3854701788 shutdown_connections 2024-04-14T08:34:21.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.306+0000 7f67ef0de640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f67d406c8f0 0x7f67d406edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.306+0000 7f67ef0de640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67e815e890 0x7f67e8158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.306+0000 7f67ef0de640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67e80d2830 0x7f67e80c9b40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.306+0000 7f67ef0de640 1 --2- 172.21.15.172:0/3854701788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67e80cedd0 0x7f67e80b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.306+0000 7f67ef0de640 1 -- 172.21.15.172:0/3854701788 >> 172.21.15.172:0/3854701788 conn(0x7f67e809d590 msgr2=0x7f67e80d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:21.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.306+0000 7f67ef0de640 1 -- 172.21.15.172:0/3854701788 shutdown_connections 2024-04-14T08:34:21.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.306+0000 7f67ef0de640 1 -- 172.21.15.172:0/3854701788 wait complete. 2024-04-14T08:34:21.322 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:33:57.868995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:33:58.323549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:34:21.322 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:34:21.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.425+0000 7f9f8691e640 1 Processor -- start 2024-04-14T08:34:21.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.425+0000 7f9f8691e640 1 -- start start 2024-04-14T08:34:21.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.426+0000 7f9f8691e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9f800d1a20 0x7f9f8006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.426+0000 7f9f8691e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8006d000 0x7f9f8006d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.426+0000 7f9f8691e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f800701b0 con 0x7f9f80071250 2024-04-14T08:34:21.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.426+0000 7f9f8691e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8006d9d0 con 0x7f9f800d1a20 2024-04-14T08:34:21.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.426+0000 7f9f8691e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9f80070dc0 con 0x7f9f8006d000 2024-04-14T08:34:21.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.426+0000 7f9f7ffff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9f800d1a20 0x7f9f8006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.426+0000 7f9f84ebb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8006d000 0x7f9f8006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.426+0000 7f9f84ebb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8006d000 0x7f9f8006d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:54410/0 (socket says 172.21.15.172:54410) 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.426+0000 7f9f7ffff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9f800d1a20 0x7f9f8006fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:35514/0 (socket says 172.21.15.172:35514) 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.426+0000 7f9f84ebb640 1 -- 172.21.15.172:0/358063769 learned_addr learned my addr 172.21.15.172:0/358063769 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.426+0000 7f9f7f7fe640 1 --1- 172.21.15.172:0/358063769 >> v1:172.21.15.177:6789/0 conn(0x7f9f80071250 0x7f9f800cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:35574/0 (socket says 172.21.15.172:35574) 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.427+0000 7f9f7ffff640 1 -- 172.21.15.172:0/358063769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8006d000 msgr2=0x7f9f8006d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.427+0000 7f9f7ffff640 1 --2- 172.21.15.172:0/358063769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8006d000 0x7f9f8006d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.427+0000 7f9f7ffff640 1 -- 172.21.15.172:0/358063769 >> v1:172.21.15.177:6789/0 conn(0x7f9f80071250 legacy=0x7f9f800cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.427+0000 7f9f7effd640 1 -- 172.21.15.172:0/358063769 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 577423527 0 0) 0x7f9f800701b0 con 0x7f9f80071250 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.427+0000 7f9f7ffff640 1 -- 172.21.15.172:0/358063769 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f80070490 con 0x7f9f800d1a20 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.427+0000 7f9f84ebb640 1 --2- 172.21.15.172:0/358063769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8006d000 0x7f9f8006d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.428+0000 7f9f7ffff640 1 --2- 172.21.15.172:0/358063769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f800d1a20 0x7f9f8006fb80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9f740097a0 tx=0x7f9f7402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b164322e8ca71017 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:21.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.428+0000 7f9f7effd640 1 -- 172.21.15.172:0/358063769 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f740315a0 con 0x7f9f800d1a20 2024-04-14T08:34:21.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.428+0000 7f9f7effd640 1 -- 172.21.15.172:0/358063769 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f74037070 con 0x7f9f800d1a20 2024-04-14T08:34:21.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.428+0000 7f9f7effd640 1 -- 172.21.15.172:0/358063769 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74031960 con 0x7f9f800d1a20 2024-04-14T08:34:21.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.428+0000 7f9f8691e640 1 -- 172.21.15.172:0/358063769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f800d1a20 msgr2=0x7f9f8006fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.428+0000 7f9f8691e640 1 --2- 172.21.15.172:0/358063769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f800d1a20 0x7f9f8006fb80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9f740097a0 tx=0x7f9f7402b840 comp rx=0 tx=0).stop 2024-04-14T08:34:21.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.428+0000 7f9f8691e640 1 -- 172.21.15.172:0/358063769 shutdown_connections 2024-04-14T08:34:21.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.428+0000 7f9f8691e640 1 --2- 172.21.15.172:0/358063769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f8006d000 0x7f9f8006d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.428+0000 7f9f8691e640 1 --2- 172.21.15.172:0/358063769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f800d1a20 0x7f9f8006fb80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.428+0000 7f9f8691e640 1 -- 172.21.15.172:0/358063769 >> 172.21.15.172:0/358063769 conn(0x7f9f8008c000 msgr2=0x7f9f800d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:21.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.428+0000 7f9f8691e640 1 -- 172.21.15.172:0/358063769 shutdown_connections 2024-04-14T08:34:21.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.428+0000 7f9f8691e640 1 -- 172.21.15.172:0/358063769 wait complete. 2024-04-14T08:34:21.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.429+0000 7f9f8691e640 1 Processor -- start 2024-04-14T08:34:21.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.430+0000 7f9f8691e640 1 -- start start 2024-04-14T08:34:21.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.430+0000 7f9f8691e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f8006d000 0x7f9f800cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.430+0000 7f9f8691e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f80071250 0x7f9f80155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.430+0000 7f9f8691e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f800d1a20 0x7f9f800c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.430+0000 7f9f8691e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9f800d1e00 con 0x7f9f8006d000 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.430+0000 7f9f8691e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80070a50 con 0x7f9f80071250 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.430+0000 7f9f8691e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9f800c89b0 con 0x7f9f800d1a20 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.430+0000 7f9f7ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f8006d000 0x7f9f800cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f7ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f8006d000 0x7f9f800cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48726/0 (socket says 172.21.15.172:48726) 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f7ffff640 1 -- 172.21.15.172:0/4213329951 learned_addr learned my addr 172.21.15.172:0/4213329951 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f7f7fe640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f80071250 0x7f9f80155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f84ebb640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f800d1a20 0x7f9f800c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f7ffff640 1 -- 172.21.15.172:0/4213329951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f800d1a20 msgr2=0x7f9f800c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f7ffff640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f800d1a20 0x7f9f800c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f7ffff640 1 -- 172.21.15.172:0/4213329951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f80071250 msgr2=0x7f9f80155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f7ffff640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f80071250 0x7f9f80155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f7ffff640 1 -- 172.21.15.172:0/4213329951 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f800c9d50 con 0x7f9f8006d000 2024-04-14T08:34:21.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f7ffff640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f8006d000 0x7f9f800cadf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9f740098c0 tx=0x7f9f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:21.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f7cff9640 1 -- 172.21.15.172:0/4213329951 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74004230 con 0x7f9f8006d000 2024-04-14T08:34:21.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f7cff9640 1 -- 172.21.15.172:0/4213329951 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f740043d0 con 0x7f9f8006d000 2024-04-14T08:34:21.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f8691e640 1 -- 172.21.15.172:0/4213329951 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f800ca030 con 0x7f9f8006d000 2024-04-14T08:34:21.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f7cff9640 1 -- 172.21.15.172:0/4213329951 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74003850 con 0x7f9f8006d000 2024-04-14T08:34:21.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.431+0000 7f9f8691e640 1 -- 172.21.15.172:0/4213329951 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f800ca5f0 con 0x7f9f8006d000 2024-04-14T08:34:21.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.432+0000 7f9f8691e640 1 -- 172.21.15.172:0/4213329951 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f80075490 con 0x7f9f8006d000 2024-04-14T08:34:21.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.433+0000 7f9f7cff9640 1 -- 172.21.15.172:0/4213329951 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f740039f0 con 0x7f9f8006d000 2024-04-14T08:34:21.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.433+0000 7f9f7cff9640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f6c06c8f0 0x7f9f6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.433+0000 7f9f7f7fe640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f6c06c8f0 0x7f9f6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.433+0000 7f9f7cff9640 1 -- 172.21.15.172:0/4213329951 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9f7402f030 con 0x7f9f8006d000 2024-04-14T08:34:21.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.434+0000 7f9f7f7fe640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f6c06c8f0 0x7f9f6c06edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b1270 tx=0x7f9f70007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:21.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.438+0000 7f9f7cff9640 1 -- 172.21.15.172:0/4213329951 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9f74041070 con 0x7f9f8006d000 2024-04-14T08:34:21.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.617+0000 7f9f8691e640 1 -- 172.21.15.172:0/4213329951 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9f800b16b0 con 0x7f9f8006d000 2024-04-14T08:34:21.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.617+0000 7f9f7cff9640 1 -- 172.21.15.172:0/4213329951 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7f9f7407b740 con 0x7f9f8006d000 2024-04-14T08:34:21.619 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:34:21.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.619+0000 7f9f8691e640 1 -- 172.21.15.172:0/4213329951 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f6c06c8f0 msgr2=0x7f9f6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.619+0000 7f9f8691e640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f6c06c8f0 0x7f9f6c06edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b1270 tx=0x7f9f70007d50 comp rx=0 tx=0).stop 2024-04-14T08:34:21.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.619+0000 7f9f8691e640 1 -- 172.21.15.172:0/4213329951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f8006d000 msgr2=0x7f9f800cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.619+0000 7f9f8691e640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f8006d000 0x7f9f800cadf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9f740098c0 tx=0x7f9f740046c0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.619+0000 7f9f8691e640 1 -- 172.21.15.172:0/4213329951 shutdown_connections 2024-04-14T08:34:21.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.619+0000 7f9f8691e640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f6c06c8f0 0x7f9f6c06edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.619+0000 7f9f8691e640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f800d1a20 0x7f9f800c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.619+0000 7f9f8691e640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f80071250 0x7f9f80155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.619+0000 7f9f8691e640 1 --2- 172.21.15.172:0/4213329951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f8006d000 0x7f9f800cadf0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.619+0000 7f9f8691e640 1 -- 172.21.15.172:0/4213329951 >> 172.21.15.172:0/4213329951 conn(0x7f9f8008c000 msgr2=0x7f9f800cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:21.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.619+0000 7f9f8691e640 1 -- 172.21.15.172:0/4213329951 shutdown_connections 2024-04-14T08:34:21.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.619+0000 7f9f8691e640 1 -- 172.21.15.172:0/4213329951 wait complete. 2024-04-14T08:34:21.632 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:34:21.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.734+0000 7f9f70c60640 1 Processor -- start 2024-04-14T08:34:21.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.734+0000 7f9f70c60640 1 -- start start 2024-04-14T08:34:21.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.735+0000 7f9f70c60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6c0d1a20 0x7f9f6c06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.735+0000 7f9f70c60640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f6c071250 0x7f9f6c0cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.735+0000 7f9f70c60640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f6c0701b0 con 0x7f9f6c06d000 2024-04-14T08:34:21.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.735+0000 7f9f70c60640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f9f6c070dc0 con 0x7f9f6c071250 2024-04-14T08:34:21.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.735+0000 7f9f70c60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6c070490 con 0x7f9f6c0d1a20 2024-04-14T08:34:21.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.735+0000 7f9f63fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f6c071250 0x7f9f6c0cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.735+0000 7f9f63fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f6c071250 0x7f9f6c0cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48730/0 (socket says 172.21.15.172:48730) 2024-04-14T08:34:21.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.735+0000 7f9f63fff640 1 -- 172.21.15.172:0/1696482875 learned_addr learned my addr 172.21.15.172:0/1696482875 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:21.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.736+0000 7f9f6a59c640 1 --2- 172.21.15.172:0/1696482875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6c0d1a20 0x7f9f6c06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.736+0000 7f9f63fff640 1 -- 172.21.15.172:0/1696482875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6c0d1a20 msgr2=0x7f9f6c06fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:21.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.736+0000 7f9f63fff640 1 --2- 172.21.15.172:0/1696482875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6c0d1a20 0x7f9f6c06fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.736+0000 7f9f63fff640 1 -- 172.21.15.172:0/1696482875 >> v1:172.21.15.177:6789/0 conn(0x7f9f6c06d000 legacy=0x7f9f6c15dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.736+0000 7f9f63fff640 1 -- 172.21.15.172:0/1696482875 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c070770 con 0x7f9f6c071250 2024-04-14T08:34:21.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.736+0000 7f9f63fff640 1 --2- 172.21.15.172:0/1696482875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f6c071250 0x7f9f6c0cf530 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009870 tx=0x7f9f5402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b223198c743b0c07 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:21.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.737+0000 7f9f69d9b640 1 -- 172.21.15.172:0/1696482875 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54035020 con 0x7f9f6c071250 2024-04-14T08:34:21.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.737+0000 7f9f69d9b640 1 -- 172.21.15.172:0/1696482875 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f54031640 con 0x7f9f6c071250 2024-04-14T08:34:21.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.737+0000 7f9f69d9b640 1 -- 172.21.15.172:0/1696482875 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5402d040 con 0x7f9f6c071250 2024-04-14T08:34:21.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.737+0000 7f9f70c60640 1 -- 172.21.15.172:0/1696482875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f6c071250 msgr2=0x7f9f6c0cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.737+0000 7f9f70c60640 1 --2- 172.21.15.172:0/1696482875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f6c071250 0x7f9f6c0cf530 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009870 tx=0x7f9f5402b8f0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.737+0000 7f9f70c60640 1 -- 172.21.15.172:0/1696482875 shutdown_connections 2024-04-14T08:34:21.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.737+0000 7f9f70c60640 1 --2- 172.21.15.172:0/1696482875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f6c071250 0x7f9f6c0cf530 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.737+0000 7f9f70c60640 1 --2- 172.21.15.172:0/1696482875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6c0d1a20 0x7f9f6c06fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.737+0000 7f9f70c60640 1 -- 172.21.15.172:0/1696482875 >> 172.21.15.172:0/1696482875 conn(0x7f9f6c08c000 msgr2=0x7f9f6c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:21.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.737+0000 7f9f70c60640 1 -- 172.21.15.172:0/1696482875 shutdown_connections 2024-04-14T08:34:21.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.737+0000 7f9f70c60640 1 -- 172.21.15.172:0/1696482875 wait complete. 2024-04-14T08:34:21.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.739+0000 7f9f70c60640 1 Processor -- start 2024-04-14T08:34:21.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.739+0000 7f9f70c60640 1 -- start start 2024-04-14T08:34:21.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.739+0000 7f9f70c60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6c06d000 0x7f9f6c0c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.739+0000 7f9f70c60640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f6c071250 0x7f9f6c1d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.739+0000 7f9f70c60640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f6c0d1a20 0x7f9f6c1d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.739+0000 7f9f70c60640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c0d1e00 con 0x7f9f6c071250 2024-04-14T08:34:21.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.739+0000 7f9f70c60640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c0cfa70 con 0x7f9f6c0d1a20 2024-04-14T08:34:21.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.739+0000 7f9f70c60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6c0afdd0 con 0x7f9f6c06d000 2024-04-14T08:34:21.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.739+0000 7f9f63fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f6c071250 0x7f9f6c1d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.739+0000 7f9f63fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f6c071250 0x7f9f6c1d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48732/0 (socket says 172.21.15.172:48732) 2024-04-14T08:34:21.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.739+0000 7f9f63fff640 1 -- 172.21.15.172:0/1780477296 learned_addr learned my addr 172.21.15.172:0/1780477296 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:21.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f6a59c640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6c06d000 0x7f9f6c0c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f6ad9d640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f6c0d1a20 0x7f9f6c1d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f63fff640 1 -- 172.21.15.172:0/1780477296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6c06d000 msgr2=0x7f9f6c0c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f63fff640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6c06d000 0x7f9f6c0c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f63fff640 1 -- 172.21.15.172:0/1780477296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f6c0d1a20 msgr2=0x7f9f6c1d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f63fff640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f6c0d1a20 0x7f9f6c1d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f63fff640 1 -- 172.21.15.172:0/1780477296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c1d5460 con 0x7f9f6c071250 2024-04-14T08:34:21.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f63fff640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f6c071250 0x7f9f6c1d4f20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9f54000c00 tx=0x7f9f540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:21.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f62ffd640 1 -- 172.21.15.172:0/1780477296 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54040070 con 0x7f9f6c071250 2024-04-14T08:34:21.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f62ffd640 1 -- 172.21.15.172:0/1780477296 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f540399f0 con 0x7f9f6c071250 2024-04-14T08:34:21.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f70c60640 1 -- 172.21.15.172:0/1780477296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6c0c97c0 con 0x7f9f6c071250 2024-04-14T08:34:21.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f62ffd640 1 -- 172.21.15.172:0/1780477296 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54035040 con 0x7f9f6c071250 2024-04-14T08:34:21.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.740+0000 7f9f70c60640 1 -- 172.21.15.172:0/1780477296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6c0b0300 con 0x7f9f6c071250 2024-04-14T08:34:21.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.741+0000 7f9f70c60640 1 -- 172.21.15.172:0/1780477296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f6c075490 con 0x7f9f6c071250 2024-04-14T08:34:21.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.742+0000 7f9f62ffd640 1 -- 172.21.15.172:0/1780477296 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f5404b060 con 0x7f9f6c071250 2024-04-14T08:34:21.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.742+0000 7f9f62ffd640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f5006c920 0x7f9f5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:21.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.742+0000 7f9f62ffd640 1 -- 172.21.15.172:0/1780477296 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9f540ad3b0 con 0x7f9f6c071250 2024-04-14T08:34:21.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.742+0000 7f9f6a59c640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f5006c920 0x7f9f5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:21.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.742+0000 7f9f6a59c640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f5006c920 0x7f9f5006ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009aa0 tx=0x7f9f5c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:21.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.746+0000 7f9f62ffd640 1 -- 172.21.15.172:0/1780477296 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9f5407b090 con 0x7f9f6c071250 2024-04-14T08:34:21.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.922+0000 7f9f70c60640 1 -- 172.21.15.172:0/1780477296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9f6c0d2820 con 0x7f9f6c071250 2024-04-14T08:34:21.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.923+0000 7f9f62ffd640 1 -- 172.21.15.172:0/1780477296 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) ==== 104+0+11 (secure 0 0 0) 0x7f9f5407f390 con 0x7f9f6c071250 2024-04-14T08:34:21.924 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:34:21.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.925+0000 7f9f70c60640 1 -- 172.21.15.172:0/1780477296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f5006c920 msgr2=0x7f9f5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.925+0000 7f9f70c60640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f5006c920 0x7f9f5006ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009aa0 tx=0x7f9f5c009190 comp rx=0 tx=0).stop 2024-04-14T08:34:21.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.925+0000 7f9f70c60640 1 -- 172.21.15.172:0/1780477296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f6c071250 msgr2=0x7f9f6c1d4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:21.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.925+0000 7f9f70c60640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f6c071250 0x7f9f6c1d4f20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9f54000c00 tx=0x7f9f540046c0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.925+0000 7f9f70c60640 1 -- 172.21.15.172:0/1780477296 shutdown_connections 2024-04-14T08:34:21.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.925+0000 7f9f70c60640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9f5006c920 0x7f9f5006ede0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.925+0000 7f9f70c60640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f6c0d1a20 0x7f9f6c1d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.925+0000 7f9f70c60640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f6c071250 0x7f9f6c1d4f20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.925+0000 7f9f70c60640 1 --2- 172.21.15.172:0/1780477296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6c06d000 0x7f9f6c0c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:21.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.925+0000 7f9f70c60640 1 -- 172.21.15.172:0/1780477296 >> 172.21.15.172:0/1780477296 conn(0x7f9f6c08c000 msgr2=0x7f9f6c0cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:21.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.925+0000 7f9f70c60640 1 -- 172.21.15.172:0/1780477296 shutdown_connections 2024-04-14T08:34:21.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:21.925+0000 7f9f70c60640 1 -- 172.21.15.172:0/1780477296 wait complete. 2024-04-14T08:34:21.938 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:34:22.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.043+0000 7f3f09755640 1 Processor -- start 2024-04-14T08:34:22.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.043+0000 7f3f09755640 1 -- start start 2024-04-14T08:34:22.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3f09755640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3f040d0cc0 0x7f3f040d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3f09755640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3f040d49c0 0x7f3f040d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3f09755640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f040d15e0 con 0x7f3f04160f30 2024-04-14T08:34:22.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3f09755640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f3f040ceec0 con 0x7f3f040d49c0 2024-04-14T08:34:22.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3f09755640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f3f040cfd00 con 0x7f3f040d0cc0 2024-04-14T08:34:22.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3efbfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3f040d49c0 0x7f3f040d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3efbfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3f040d49c0 0x7f3f040d4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48746/0 (socket says 172.21.15.172:48746) 2024-04-14T08:34:22.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3efbfff640 1 -- 172.21.15.172:0/175054158 learned_addr learned my addr 172.21.15.172:0/175054158 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:22.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3f02ffd640 1 --2- 172.21.15.172:0/175054158 >> v2:172.21.15.177:3300/0 conn(0x7f3f040d0cc0 0x7f3f040d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3efbfff640 1 -- 172.21.15.172:0/175054158 >> v2:172.21.15.177:3300/0 conn(0x7f3f040d0cc0 msgr2=0x7f3f040d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:22.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3efbfff640 1 --2- 172.21.15.172:0/175054158 >> v2:172.21.15.177:3300/0 conn(0x7f3f040d0cc0 0x7f3f040d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3efbfff640 1 -- 172.21.15.172:0/175054158 >> v1:172.21.15.177:6789/0 conn(0x7f3f04160f30 legacy=0x7f3f04161380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.044+0000 7f3efbfff640 1 -- 172.21.15.172:0/175054158 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f040cf0f0 con 0x7f3f040d49c0 2024-04-14T08:34:22.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.045+0000 7f3efbfff640 1 --2- 172.21.15.172:0/175054158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f040d49c0 0x7f3f040d4e50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3f0408e7b0 tx=0x7f3eec02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ed1d72949ee680 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:22.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.045+0000 7f3f027fc640 1 -- 172.21.15.172:0/175054158 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec034070 con 0x7f3f040d49c0 2024-04-14T08:34:22.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.045+0000 7f3f027fc640 1 -- 172.21.15.172:0/175054158 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec030860 con 0x7f3f040d49c0 2024-04-14T08:34:22.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.045+0000 7f3f027fc640 1 -- 172.21.15.172:0/175054158 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec030c30 con 0x7f3f040d49c0 2024-04-14T08:34:22.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.046+0000 7f3f09755640 1 -- 172.21.15.172:0/175054158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f040d49c0 msgr2=0x7f3f040d4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.046+0000 7f3f09755640 1 --2- 172.21.15.172:0/175054158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f040d49c0 0x7f3f040d4e50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3f0408e7b0 tx=0x7f3eec02b9a0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.046+0000 7f3f09755640 1 -- 172.21.15.172:0/175054158 shutdown_connections 2024-04-14T08:34:22.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.046+0000 7f3f09755640 1 --2- 172.21.15.172:0/175054158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f040d49c0 0x7f3f040d4e50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.046+0000 7f3f09755640 1 --2- 172.21.15.172:0/175054158 >> v2:172.21.15.177:3300/0 conn(0x7f3f040d0cc0 0x7f3f040d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.046+0000 7f3f09755640 1 -- 172.21.15.172:0/175054158 >> 172.21.15.172:0/175054158 conn(0x7f3f0409d580 msgr2=0x7f3f040d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:22.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.046+0000 7f3f09755640 1 -- 172.21.15.172:0/175054158 shutdown_connections 2024-04-14T08:34:22.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.046+0000 7f3f09755640 1 -- 172.21.15.172:0/175054158 wait complete. 2024-04-14T08:34:22.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.047+0000 7f3f09755640 1 Processor -- start 2024-04-14T08:34:22.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.047+0000 7f3f09755640 1 -- start start 2024-04-14T08:34:22.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3f09755640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f040d0cc0 0x7f3f041538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3f09755640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f040d49c0 0x7f3f040a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3f09755640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f04160f30 0x7f3f0409edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3f09755640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3f040cf990 con 0x7f3f040d49c0 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3f09755640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f040cf6b0 con 0x7f3f040d0cc0 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3f09755640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3f040d32b0 con 0x7f3f04160f30 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3efbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f040d49c0 0x7f3f040a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3efbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f040d49c0 0x7f3f040a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48750/0 (socket says 172.21.15.172:48750) 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3efbfff640 1 -- 172.21.15.172:0/4082617752 learned_addr learned my addr 172.21.15.172:0/4082617752 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3f02ffd640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f040d0cc0 0x7f3f041538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3f037fe640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f04160f30 0x7f3f0409edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3efbfff640 1 -- 172.21.15.172:0/4082617752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f04160f30 msgr2=0x7f3f0409edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3efbfff640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f04160f30 0x7f3f0409edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3efbfff640 1 -- 172.21.15.172:0/4082617752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f040d0cc0 msgr2=0x7f3f041538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3efbfff640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f040d0cc0 0x7f3f041538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3efbfff640 1 -- 172.21.15.172:0/4082617752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f040a1020 con 0x7f3f040d49c0 2024-04-14T08:34:22.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3efbfff640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f040d49c0 0x7f3f040a0ae0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3eec02b9e0 tx=0x7f3eec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:22.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.048+0000 7f3efb7fe640 1 -- 172.21.15.172:0/4082617752 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec048070 con 0x7f3f040d49c0 2024-04-14T08:34:22.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.049+0000 7f3efb7fe640 1 -- 172.21.15.172:0/4082617752 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec030660 con 0x7f3f040d49c0 2024-04-14T08:34:22.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.049+0000 7f3f09755640 1 -- 172.21.15.172:0/4082617752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0409fa40 con 0x7f3f040d49c0 2024-04-14T08:34:22.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.049+0000 7f3efb7fe640 1 -- 172.21.15.172:0/4082617752 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec034040 con 0x7f3f040d49c0 2024-04-14T08:34:22.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.049+0000 7f3f09755640 1 -- 172.21.15.172:0/4082617752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f040a0000 con 0x7f3f040d49c0 2024-04-14T08:34:22.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.050+0000 7f3f09755640 1 -- 172.21.15.172:0/4082617752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f04075490 con 0x7f3f040d49c0 2024-04-14T08:34:22.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.051+0000 7f3efb7fe640 1 -- 172.21.15.172:0/4082617752 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3eec04b070 con 0x7f3f040d49c0 2024-04-14T08:34:22.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.051+0000 7f3efb7fe640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ee406c8f0 0x7f3ee406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.051+0000 7f3f02ffd640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ee406c8f0 0x7f3ee406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.051+0000 7f3efb7fe640 1 -- 172.21.15.172:0/4082617752 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f3eec0adbe0 con 0x7f3f040d49c0 2024-04-14T08:34:22.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.052+0000 7f3f02ffd640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ee406c8f0 0x7f3ee406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009b40 tx=0x7f3ef4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:22.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.056+0000 7f3efb7fe640 1 -- 172.21.15.172:0/4082617752 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3eec07b8c0 con 0x7f3f040d49c0 2024-04-14T08:34:22.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.235+0000 7f3f09755640 1 -- 172.21.15.172:0/4082617752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3f0415fe90 con 0x7f3f040d49c0 2024-04-14T08:34:22.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.235+0000 7f3efb7fe640 1 -- 172.21.15.172:0/4082617752 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) ==== 100+0+11 (secure 0 0 0) 0x7f3eec07fbc0 con 0x7f3f040d49c0 2024-04-14T08:34:22.237 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:34:22.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.238+0000 7f3f09755640 1 -- 172.21.15.172:0/4082617752 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ee406c8f0 msgr2=0x7f3ee406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.238+0000 7f3f09755640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ee406c8f0 0x7f3ee406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009b40 tx=0x7f3ef4009190 comp rx=0 tx=0).stop 2024-04-14T08:34:22.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.238+0000 7f3f09755640 1 -- 172.21.15.172:0/4082617752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f040d49c0 msgr2=0x7f3f040a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.238+0000 7f3f09755640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f040d49c0 0x7f3f040a0ae0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3eec02b9e0 tx=0x7f3eec004730 comp rx=0 tx=0).stop 2024-04-14T08:34:22.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.238+0000 7f3f09755640 1 -- 172.21.15.172:0/4082617752 shutdown_connections 2024-04-14T08:34:22.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.238+0000 7f3f09755640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ee406c8f0 0x7f3ee406edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.238+0000 7f3f09755640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f04160f30 0x7f3f0409edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.238+0000 7f3f09755640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f040d49c0 0x7f3f040a0ae0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.238+0000 7f3f09755640 1 --2- 172.21.15.172:0/4082617752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f040d0cc0 0x7f3f041538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.238+0000 7f3f09755640 1 -- 172.21.15.172:0/4082617752 >> 172.21.15.172:0/4082617752 conn(0x7f3f0409d580 msgr2=0x7f3f04160790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:22.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.238+0000 7f3f09755640 1 -- 172.21.15.172:0/4082617752 shutdown_connections 2024-04-14T08:34:22.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.239+0000 7f3f09755640 1 -- 172.21.15.172:0/4082617752 wait complete. 2024-04-14T08:34:22.252 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:34:22.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.356+0000 7ff76c0d1640 1 Processor -- start 2024-04-14T08:34:22.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.356+0000 7ff76c0d1640 1 -- start start 2024-04-14T08:34:22.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.357+0000 7ff76c0d1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff76415e900 0x7ff764143a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.357+0000 7ff76c0d1640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff76415ee00 con 0x7ff764158a50 2024-04-14T08:34:22.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.357+0000 7ff76c0d1640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff76415f0e0 con 0x7ff764143f70 2024-04-14T08:34:22.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.357+0000 7ff76c0d1640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff764158e30 con 0x7ff76415e900 2024-04-14T08:34:22.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.357+0000 7ff76966c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff76415e900 0x7ff764143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.357+0000 7ff769e6d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff764158a50 0x7ff7640672a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35636/0 (socket says 172.21.15.172:35636) 2024-04-14T08:34:22.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.357+0000 7ff76966c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff76415e900 0x7ff764143a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48766/0 (socket says 172.21.15.172:48766) 2024-04-14T08:34:22.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.357+0000 7ff769e6d640 1 -- 172.21.15.172:0/633655638 learned_addr learned my addr 172.21.15.172:0/633655638 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:22.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.357+0000 7ff76966c640 1 -- 172.21.15.172:0/633655638 >> v1:172.21.15.177:6789/0 conn(0x7ff764143f70 legacy=0x7ff76406b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.357+0000 7ff76966c640 1 -- 172.21.15.172:0/633655638 >> v1:172.21.15.172:6789/0 conn(0x7ff764158a50 legacy=0x7ff7640672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.357+0000 7ff76966c640 1 -- 172.21.15.172:0/633655638 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76415f3c0 con 0x7ff76415e900 2024-04-14T08:34:22.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.357+0000 7ff768e6b640 1 -- 172.21.15.172:0/633655638 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2183507446 0 0) 0x7ff76415ee00 con 0x7ff764158a50 2024-04-14T08:34:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.358+0000 7ff76966c640 1 --2- 172.21.15.172:0/633655638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76415e900 0x7ff764143a30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff7580097a0 tx=0x7ff75802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf94f18298bcb86 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.358+0000 7ff768e6b640 1 -- 172.21.15.172:0/633655638 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758035020 con 0x7ff76415e900 2024-04-14T08:34:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.358+0000 7ff768e6b640 1 -- 172.21.15.172:0/633655638 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff758031830 con 0x7ff76415e900 2024-04-14T08:34:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.358+0000 7ff768e6b640 1 -- 172.21.15.172:0/633655638 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758031c00 con 0x7ff76415e900 2024-04-14T08:34:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.359+0000 7ff76c0d1640 1 -- 172.21.15.172:0/633655638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76415e900 msgr2=0x7ff764143a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.359+0000 7ff76c0d1640 1 --2- 172.21.15.172:0/633655638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76415e900 0x7ff764143a30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff7580097a0 tx=0x7ff75802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.359+0000 7ff76c0d1640 1 -- 172.21.15.172:0/633655638 shutdown_connections 2024-04-14T08:34:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.359+0000 7ff76c0d1640 1 --2- 172.21.15.172:0/633655638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76415e900 0x7ff764143a30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.359+0000 7ff76c0d1640 1 -- 172.21.15.172:0/633655638 >> 172.21.15.172:0/633655638 conn(0x7ff7640cd1b0 msgr2=0x7ff7640cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:22.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.359+0000 7ff76c0d1640 1 -- 172.21.15.172:0/633655638 shutdown_connections 2024-04-14T08:34:22.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.359+0000 7ff76c0d1640 1 -- 172.21.15.172:0/633655638 wait complete. 2024-04-14T08:34:22.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.360+0000 7ff76c0d1640 1 Processor -- start 2024-04-14T08:34:22.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.360+0000 7ff76c0d1640 1 -- start start 2024-04-14T08:34:22.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76c0d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff764143f70 0x7ff7640cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76c0d1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff764158a50 0x7ff7641f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76c0d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76415e900 0x7ff76413bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76c0d1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff76406bff0 con 0x7ff76415e900 2024-04-14T08:34:22.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76c0d1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641443a0 con 0x7ff764158a50 2024-04-14T08:34:22.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76c0d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff76406c170 con 0x7ff764143f70 2024-04-14T08:34:22.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76a66e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76415e900 0x7ff76413bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76a66e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76415e900 0x7ff76413bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48782/0 (socket says 172.21.15.172:48782) 2024-04-14T08:34:22.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76a66e640 1 -- 172.21.15.172:0/3629838663 learned_addr learned my addr 172.21.15.172:0/3629838663 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:22.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff769e6d640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff764143f70 0x7ff7640cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76966c640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff764158a50 0x7ff7641f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76a66e640 1 -- 172.21.15.172:0/3629838663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff764143f70 msgr2=0x7ff7640cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76a66e640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff764143f70 0x7ff7640cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76a66e640 1 -- 172.21.15.172:0/3629838663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff764158a50 msgr2=0x7ff7641f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76a66e640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff764158a50 0x7ff7641f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76a66e640 1 -- 172.21.15.172:0/3629838663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7640b5210 con 0x7ff76415e900 2024-04-14T08:34:22.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.361+0000 7ff76a66e640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76415e900 0x7ff76413bd70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c7a0 tx=0x7ff75c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:22.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.362+0000 7ff7567fc640 1 -- 172.21.15.172:0/3629838663 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c003b70 con 0x7ff76415e900 2024-04-14T08:34:22.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.362+0000 7ff7567fc640 1 -- 172.21.15.172:0/3629838663 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff75c003d30 con 0x7ff76415e900 2024-04-14T08:34:22.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.362+0000 7ff7567fc640 1 -- 172.21.15.172:0/3629838663 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c00aa90 con 0x7ff76415e900 2024-04-14T08:34:22.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.362+0000 7ff76c0d1640 1 -- 172.21.15.172:0/3629838663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7640b5550 con 0x7ff76415e900 2024-04-14T08:34:22.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.362+0000 7ff76c0d1640 1 -- 172.21.15.172:0/3629838663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76413e8a0 con 0x7ff76415e900 2024-04-14T08:34:22.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.363+0000 7ff76c0d1640 1 -- 172.21.15.172:0/3629838663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff764075490 con 0x7ff76415e900 2024-04-14T08:34:22.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.363+0000 7ff7567fc640 1 -- 172.21.15.172:0/3629838663 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff75c00ac30 con 0x7ff76415e900 2024-04-14T08:34:22.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.364+0000 7ff7567fc640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff74806c8f0 0x7ff74806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.364+0000 7ff769e6d640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff74806c8f0 0x7ff74806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.364+0000 7ff7567fc640 1 -- 172.21.15.172:0/3629838663 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff75c08ba60 con 0x7ff76415e900 2024-04-14T08:34:22.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.364+0000 7ff769e6d640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff74806c8f0 0x7ff74806edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff760007f70 tx=0x7ff760007f00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.368+0000 7ff7567fc640 1 -- 172.21.15.172:0/3629838663 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff7640b5550 con 0x7ff76415e900 2024-04-14T08:34:22.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.584+0000 7ff76c0d1640 1 -- 172.21.15.172:0/3629838663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff76415fcf0 con 0x7ff76415e900 2024-04-14T08:34:22.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.585+0000 7ff7567fc640 1 -- 172.21.15.172:0/3629838663 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7ff75c059740 con 0x7ff76415e900 2024-04-14T08:34:22.587 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:22.587 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1038094401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1038094401},{"type":"v1","addr":"172.21.15.177:6834","nonce":1038094401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1119512744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1119512744},{"type":"v1","addr":"172.21.15.177:6835","nonce":1119512744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3054138283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3054138283},{"type":"v1","addr":"172.21.15.172:6837","nonce":3054138283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:33:59.340499+0000","modified":"2024-04-14T08:34:01.467567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5308":{"gid":5308,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6836/3420108191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3420108191},{"type":"v1","addr":"172.21.15.172:6836","nonce":3420108191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5308,"qdb_cluster":[5308]},"id":2}]} 2024-04-14T08:34:22.587 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 16 2024-04-14T08:34:22.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.587+0000 7ff76c0d1640 1 -- 172.21.15.172:0/3629838663 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff74806c8f0 msgr2=0x7ff74806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.587+0000 7ff76c0d1640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff74806c8f0 0x7ff74806edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff760007f70 tx=0x7ff760007f00 comp rx=0 tx=0).stop 2024-04-14T08:34:22.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.587+0000 7ff76c0d1640 1 -- 172.21.15.172:0/3629838663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76415e900 msgr2=0x7ff76413bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.587+0000 7ff76c0d1640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76415e900 0x7ff76413bd70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c7a0 tx=0x7ff75c00cc60 comp rx=0 tx=0).stop 2024-04-14T08:34:22.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.587+0000 7ff76c0d1640 1 -- 172.21.15.172:0/3629838663 shutdown_connections 2024-04-14T08:34:22.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.587+0000 7ff76c0d1640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff74806c8f0 0x7ff74806edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.587+0000 7ff76c0d1640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76415e900 0x7ff76413bd70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.587+0000 7ff76c0d1640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff764158a50 0x7ff7641f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.587+0000 7ff76c0d1640 1 --2- 172.21.15.172:0/3629838663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff764143f70 0x7ff7640cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.587+0000 7ff76c0d1640 1 -- 172.21.15.172:0/3629838663 >> 172.21.15.172:0/3629838663 conn(0x7ff7640cd1b0 msgr2=0x7ff7640c5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:22.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.587+0000 7ff76c0d1640 1 -- 172.21.15.172:0/3629838663 shutdown_connections 2024-04-14T08:34:22.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.587+0000 7ff76c0d1640 1 -- 172.21.15.172:0/3629838663 wait complete. 2024-04-14T08:34:22.600 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:34:22.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.699+0000 7f6072088640 1 Processor -- start 2024-04-14T08:34:22.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.699+0000 7f6072088640 1 -- start start 2024-04-14T08:34:22.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.699+0000 7f6072088640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f606c15e890 0x7f606c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.699+0000 7f6072088640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f606c0cedd0 0x7f606c0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.699+0000 7f6072088640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606c0cf320 con 0x7f606c0d2830 2024-04-14T08:34:22.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.699+0000 7f6072088640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f606c0cff30 con 0x7f606c15e890 2024-04-14T08:34:22.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.699+0000 7f6072088640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f606c0d2350 con 0x7f606c0cedd0 2024-04-14T08:34:22.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.699+0000 7f606affd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f606c15e890 0x7f606c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.700+0000 7f606affd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f606c15e890 0x7f606c0d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48794/0 (socket says 172.21.15.172:48794) 2024-04-14T08:34:22.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.700+0000 7f606affd640 1 -- 172.21.15.172:0/4118770299 learned_addr learned my addr 172.21.15.172:0/4118770299 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:22.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.700+0000 7f606bfff640 1 --2- 172.21.15.172:0/4118770299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f606c0cedd0 0x7f606c0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.700+0000 7f606affd640 1 -- 172.21.15.172:0/4118770299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f606c0cedd0 msgr2=0x7f606c0b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:22.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.700+0000 7f606affd640 1 --2- 172.21.15.172:0/4118770299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f606c0cedd0 0x7f606c0b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.700+0000 7f606affd640 1 -- 172.21.15.172:0/4118770299 >> v1:172.21.15.177:6789/0 conn(0x7f606c0d2830 legacy=0x7f606c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.700+0000 7f606affd640 1 -- 172.21.15.172:0/4118770299 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c0cf600 con 0x7f606c15e890 2024-04-14T08:34:22.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.700+0000 7f606a7fc640 1 -- 172.21.15.172:0/4118770299 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1406973052 0 0) 0x7f606c0cf320 con 0x7f606c0d2830 2024-04-14T08:34:22.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.701+0000 7f606affd640 1 --2- 172.21.15.172:0/4118770299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f606c15e890 0x7f606c0d1300 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6060009870 tx=0x7f606002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8973264227df2d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:22.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.701+0000 7f606a7fc640 1 -- 172.21.15.172:0/4118770299 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606002f840 con 0x7f606c15e890 2024-04-14T08:34:22.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.701+0000 7f606a7fc640 1 -- 172.21.15.172:0/4118770299 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606002f9e0 con 0x7f606c15e890 2024-04-14T08:34:22.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.701+0000 7f606a7fc640 1 -- 172.21.15.172:0/4118770299 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606002fdb0 con 0x7f606c15e890 2024-04-14T08:34:22.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.701+0000 7f6072088640 1 -- 172.21.15.172:0/4118770299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f606c15e890 msgr2=0x7f606c0d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.701+0000 7f6072088640 1 --2- 172.21.15.172:0/4118770299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f606c15e890 0x7f606c0d1300 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6060009870 tx=0x7f606002ba20 comp rx=0 tx=0).stop 2024-04-14T08:34:22.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.701+0000 7f6072088640 1 -- 172.21.15.172:0/4118770299 shutdown_connections 2024-04-14T08:34:22.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.701+0000 7f6072088640 1 --2- 172.21.15.172:0/4118770299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f606c0cedd0 0x7f606c0b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.701+0000 7f6072088640 1 --2- 172.21.15.172:0/4118770299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f606c15e890 0x7f606c0d1300 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.701+0000 7f6072088640 1 -- 172.21.15.172:0/4118770299 >> 172.21.15.172:0/4118770299 conn(0x7f606c09d590 msgr2=0x7f606c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:22.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.701+0000 7f6072088640 1 -- 172.21.15.172:0/4118770299 shutdown_connections 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.701+0000 7f6072088640 1 -- 172.21.15.172:0/4118770299 wait complete. 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.702+0000 7f6072088640 1 Processor -- start 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.703+0000 7f6072088640 1 -- start start 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.703+0000 7f6072088640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f606c0cedd0 0x7f606c158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.703+0000 7f6072088640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f606c0d2830 0x7f606c0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.703+0000 7f6072088640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f606c15e890 0x7f606c0c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.703+0000 7f6072088640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f606c0d1840 con 0x7f606c0d2830 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.703+0000 7f6072088640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f606c0cfbc0 con 0x7f606c15e890 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.703+0000 7f6072088640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f606c0b3c50 con 0x7f606c0cedd0 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.703+0000 7f606affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f606c0d2830 0x7f606c0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f606affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f606c0d2830 0x7f606c0c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48810/0 (socket says 172.21.15.172:48810) 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f606affd640 1 -- 172.21.15.172:0/1146737134 learned_addr learned my addr 172.21.15.172:0/1146737134 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f606b7fe640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f606c0cedd0 0x7f606c158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f606bfff640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f606c15e890 0x7f606c0c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f606affd640 1 -- 172.21.15.172:0/1146737134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f606c0cedd0 msgr2=0x7f606c158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f606affd640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f606c0cedd0 0x7f606c158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f606affd640 1 -- 172.21.15.172:0/1146737134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f606c15e890 msgr2=0x7f606c0c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f606affd640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f606c15e890 0x7f606c0c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f606affd640 1 -- 172.21.15.172:0/1146737134 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c0c88d0 con 0x7f606c0d2830 2024-04-14T08:34:22.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f606b7fe640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f606c0cedd0 0x7f606c158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:34:22.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f606affd640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f606c0d2830 0x7f606c0c7d80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6060000c00 tx=0x7f6060004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:22.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f6053fff640 1 -- 172.21.15.172:0/1146737134 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060037bc0 con 0x7f606c0d2830 2024-04-14T08:34:22.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f6053fff640 1 -- 172.21.15.172:0/1146737134 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6060046070 con 0x7f606c0d2830 2024-04-14T08:34:22.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f6053fff640 1 -- 172.21.15.172:0/1146737134 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060041400 con 0x7f606c0d2830 2024-04-14T08:34:22.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f6072088640 1 -- 172.21.15.172:0/1146737134 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606c0c8bb0 con 0x7f606c0d2830 2024-04-14T08:34:22.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.704+0000 7f6072088640 1 -- 172.21.15.172:0/1146737134 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606c0c9170 con 0x7f606c0d2830 2024-04-14T08:34:22.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.705+0000 7f6072088640 1 -- 172.21.15.172:0/1146737134 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606c068520 con 0x7f606c0d2830 2024-04-14T08:34:22.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.706+0000 7f6053fff640 1 -- 172.21.15.172:0/1146737134 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6060040070 con 0x7f606c0d2830 2024-04-14T08:34:22.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.706+0000 7f6053fff640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f605806c8f0 0x7f605806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:22.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.706+0000 7f606b7fe640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f605806c8f0 0x7f605806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:22.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.706+0000 7f6053fff640 1 -- 172.21.15.172:0/1146737134 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f60600ae1d0 con 0x7f606c0d2830 2024-04-14T08:34:22.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.707+0000 7f606b7fe640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f605806c8f0 0x7f605806edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f605c0012e0 tx=0x7f605c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:22.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.711+0000 7f6053fff640 1 -- 172.21.15.172:0/1146737134 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6060033030 con 0x7f606c0d2830 2024-04-14T08:34:22.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.892+0000 7f6072088640 1 -- 172.21.15.172:0/1146737134 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f606c0d2610 con 0x7f606c0d2830 2024-04-14T08:34:22.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.893+0000 7f6053fff640 1 -- 172.21.15.172:0/1146737134 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17002 (secure 0 0 0) 0x7f606007bd90 con 0x7f606c0d2830 2024-04-14T08:34:22.895 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:22.895 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:00.984153+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:33:57.868995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-14T08:33:58.323549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:34:22.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.895+0000 7f6072088640 1 -- 172.21.15.172:0/1146737134 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f605806c8f0 msgr2=0x7f605806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.895+0000 7f6072088640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f605806c8f0 0x7f605806edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f605c0012e0 tx=0x7f605c009040 comp rx=0 tx=0).stop 2024-04-14T08:34:22.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.895+0000 7f6072088640 1 -- 172.21.15.172:0/1146737134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f606c0d2830 msgr2=0x7f606c0c7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:22.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.895+0000 7f6072088640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f606c0d2830 0x7f606c0c7d80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6060000c00 tx=0x7f6060004730 comp rx=0 tx=0).stop 2024-04-14T08:34:22.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.895+0000 7f6072088640 1 -- 172.21.15.172:0/1146737134 shutdown_connections 2024-04-14T08:34:22.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.895+0000 7f6072088640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f605806c8f0 0x7f605806edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.895+0000 7f6072088640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f606c15e890 0x7f606c0c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.895+0000 7f6072088640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f606c0d2830 0x7f606c0c7d80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.895+0000 7f6072088640 1 --2- 172.21.15.172:0/1146737134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f606c0cedd0 0x7f606c158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:22.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.895+0000 7f6072088640 1 -- 172.21.15.172:0/1146737134 >> 172.21.15.172:0/1146737134 conn(0x7f606c09d590 msgr2=0x7f606c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:22.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.895+0000 7f6072088640 1 -- 172.21.15.172:0/1146737134 shutdown_connections 2024-04-14T08:34:22.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:22.895+0000 7f6072088640 1 -- 172.21.15.172:0/1146737134 wait complete. 2024-04-14T08:34:22.908 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-14T08:34:22.909 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:34:23.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.010+0000 7ff7bfb8c640 1 Processor -- start 2024-04-14T08:34:23.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.010+0000 7ff7bfb8c640 1 -- start start 2024-04-14T08:34:23.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.011+0000 7ff7bfb8c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff7b80d2830 0x7ff7b80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.011+0000 7ff7bfb8c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b80cedd0 0x7ff7b80b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.011+0000 7ff7bfb8c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b80d3150 con 0x7ff7b815e890 2024-04-14T08:34:23.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.011+0000 7ff7bfb8c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b80cff30 con 0x7ff7b80d2830 2024-04-14T08:34:23.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.011+0000 7ff7bfb8c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b80d2350 con 0x7ff7b80cedd0 2024-04-14T08:34:23.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.011+0000 7ff7bd928640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff7b80d2830 0x7ff7b80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.011+0000 7ff7bd928640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff7b80d2830 0x7ff7b80d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48812/0 (socket says 172.21.15.172:48812) 2024-04-14T08:34:23.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.011+0000 7ff7bd928640 1 -- 172.21.15.172:0/1607126625 learned_addr learned my addr 172.21.15.172:0/1607126625 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:23.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.011+0000 7ff7be129640 1 --2- 172.21.15.172:0/1607126625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b80cedd0 0x7ff7b80b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.012+0000 7ff7bd928640 1 -- 172.21.15.172:0/1607126625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b80cedd0 msgr2=0x7ff7b80b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:23.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.012+0000 7ff7bd928640 1 --2- 172.21.15.172:0/1607126625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b80cedd0 0x7ff7b80b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.012+0000 7ff7bd928640 1 -- 172.21.15.172:0/1607126625 >> v1:172.21.15.177:6789/0 conn(0x7ff7b815e890 legacy=0x7ff7b80d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.012+0000 7ff7bd928640 1 -- 172.21.15.172:0/1607126625 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b80cf320 con 0x7ff7b80d2830 2024-04-14T08:34:23.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.012+0000 7ff7bd928640 1 --2- 172.21.15.172:0/1607126625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b80d2830 0x7ff7b80d2c10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009870 tx=0x7ff7b402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75dde1d29cb3bf7e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.013+0000 7ff7bc926640 1 -- 172.21.15.172:0/1607126625 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b402f840 con 0x7ff7b80d2830 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.013+0000 7ff7bc926640 1 -- 172.21.15.172:0/1607126625 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b402f9e0 con 0x7ff7b80d2830 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.013+0000 7ff7bc926640 1 -- 172.21.15.172:0/1607126625 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b402fdb0 con 0x7ff7b80d2830 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.013+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/1607126625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b80d2830 msgr2=0x7ff7b80d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.013+0000 7ff7bfb8c640 1 --2- 172.21.15.172:0/1607126625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b80d2830 0x7ff7b80d2c10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009870 tx=0x7ff7b402ba20 comp rx=0 tx=0).stop 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.013+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/1607126625 shutdown_connections 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.013+0000 7ff7bfb8c640 1 --2- 172.21.15.172:0/1607126625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b80cedd0 0x7ff7b80b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.013+0000 7ff7bfb8c640 1 --2- 172.21.15.172:0/1607126625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b80d2830 0x7ff7b80d2c10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.013+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/1607126625 >> 172.21.15.172:0/1607126625 conn(0x7ff7b809d590 msgr2=0x7ff7b80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.013+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/1607126625 shutdown_connections 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.013+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/1607126625 wait complete. 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.014+0000 7ff7bfb8c640 1 Processor -- start 2024-04-14T08:34:23.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.014+0000 7ff7bfb8c640 1 -- start start 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bfb8c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b80cedd0 0x7ff7b814f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bfb8c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b80d2830 0x7ff7b80cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bfb8c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b815e890 0x7ff7b80c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bfb8c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b80cfbc0 con 0x7ff7b80cedd0 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bfb8c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b80cf8e0 con 0x7ff7b815e890 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bfb8c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b80b3c50 con 0x7ff7b80d2830 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bd928640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b80cedd0 0x7ff7b814f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bd928640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b80cedd0 0x7ff7b814f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48816/0 (socket says 172.21.15.172:48816) 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bd928640 1 -- 172.21.15.172:0/4251336443 learned_addr learned my addr 172.21.15.172:0/4251336443 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bd127640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b80d2830 0x7ff7b80cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7be129640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b815e890 0x7ff7b80c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bd928640 1 -- 172.21.15.172:0/4251336443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b80d2830 msgr2=0x7ff7b80cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bd928640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b80d2830 0x7ff7b80cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bd928640 1 -- 172.21.15.172:0/4251336443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b815e890 msgr2=0x7ff7b80c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bd928640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b815e890 0x7ff7b80c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.015+0000 7ff7bd928640 1 -- 172.21.15.172:0/4251336443 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b8160050 con 0x7ff7b80cedd0 2024-04-14T08:34:23.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.016+0000 7ff7bd928640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b80cedd0 0x7ff7b814f7a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff7b402ba60 tx=0x7ff7b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:23.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.016+0000 7ff7ae7fc640 1 -- 172.21.15.172:0/4251336443 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4047070 con 0x7ff7b80cedd0 2024-04-14T08:34:23.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.016+0000 7ff7ae7fc640 1 -- 172.21.15.172:0/4251336443 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b4037bc0 con 0x7ff7b80cedd0 2024-04-14T08:34:23.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.016+0000 7ff7ae7fc640 1 -- 172.21.15.172:0/4251336443 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4042480 con 0x7ff7b80cedd0 2024-04-14T08:34:23.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.016+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/4251336443 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b8160330 con 0x7ff7b80cedd0 2024-04-14T08:34:23.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.016+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/4251336443 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b81608f0 con 0x7ff7b80cedd0 2024-04-14T08:34:23.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.017+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/4251336443 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b8068520 con 0x7ff7b80cedd0 2024-04-14T08:34:23.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.017+0000 7ff7ae7fc640 1 -- 172.21.15.172:0/4251336443 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7b404a070 con 0x7ff7b80cedd0 2024-04-14T08:34:23.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.018+0000 7ff7ae7fc640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff79c06c8f0 0x7ff79c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.018+0000 7ff7ae7fc640 1 -- 172.21.15.172:0/4251336443 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff7b40ade40 con 0x7ff7b80cedd0 2024-04-14T08:34:23.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.018+0000 7ff7bd127640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff79c06c8f0 0x7ff79c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.018+0000 7ff7bd127640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff79c06c8f0 0x7ff79c06edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80b65a0 tx=0x7ff7a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:23.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.022+0000 7ff7ae7fc640 1 -- 172.21.15.172:0/4251336443 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff7b4033030 con 0x7ff7b80cedd0 2024-04-14T08:34:23.095 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:34:23.098 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:34:23.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.279+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/4251336443 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7b80d4960 con 0x7ff7b80cedd0 2024-04-14T08:34:23.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.280+0000 7ff7ae7fc640 1 -- 172.21.15.172:0/4251336443 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7ff7b407bb20 con 0x7ff7b80cedd0 2024-04-14T08:34:23.282 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:23.282 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1038094401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1038094401},{"type":"v1","addr":"172.21.15.177:6834","nonce":1038094401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1119512744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1119512744},{"type":"v1","addr":"172.21.15.177:6835","nonce":1119512744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3054138283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3054138283},{"type":"v1","addr":"172.21.15.172:6837","nonce":3054138283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:33:59.340499+0000","modified":"2024-04-14T08:34:01.467567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5308":{"gid":5308,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6836/3420108191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3420108191},{"type":"v1","addr":"172.21.15.172:6836","nonce":3420108191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5308,"qdb_cluster":[5308]},"id":2}]} 2024-04-14T08:34:23.282 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 16 2024-04-14T08:34:23.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.282+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/4251336443 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff79c06c8f0 msgr2=0x7ff79c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.282+0000 7ff7bfb8c640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff79c06c8f0 0x7ff79c06edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80b65a0 tx=0x7ff7a8009040 comp rx=0 tx=0).stop 2024-04-14T08:34:23.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.282+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/4251336443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b80cedd0 msgr2=0x7ff7b814f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.282+0000 7ff7bfb8c640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b80cedd0 0x7ff7b814f7a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff7b402ba60 tx=0x7ff7b4004730 comp rx=0 tx=0).stop 2024-04-14T08:34:23.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.282+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/4251336443 shutdown_connections 2024-04-14T08:34:23.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.282+0000 7ff7bfb8c640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff79c06c8f0 0x7ff79c06edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.282+0000 7ff7bfb8c640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b815e890 0x7ff7b80c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.282+0000 7ff7bfb8c640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b80d2830 0x7ff7b80cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.282+0000 7ff7bfb8c640 1 --2- 172.21.15.172:0/4251336443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b80cedd0 0x7ff7b814f7a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.282+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/4251336443 >> 172.21.15.172:0/4251336443 conn(0x7ff7b809d590 msgr2=0x7ff7b80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:23.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.282+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/4251336443 shutdown_connections 2024-04-14T08:34:23.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.282+0000 7ff7bfb8c640 1 -- 172.21.15.172:0/4251336443 wait complete. 2024-04-14T08:34:23.295 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:34:23.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.396+0000 7ff242820640 1 Processor -- start 2024-04-14T08:34:23.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.396+0000 7ff242820640 1 -- start start 2024-04-14T08:34:23.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.396+0000 7ff242820640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff23c0ce120 0x7ff23c09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.396+0000 7ff242820640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23c0d2ba0 0x7ff23c09afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.396+0000 7ff242820640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff23c06cfd0 0x7ff23c06d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.396+0000 7ff242820640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff23c0d1610 con 0x7ff23c06cfd0 2024-04-14T08:34:23.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.396+0000 7ff242820640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff23c0d1050 con 0x7ff23c0ce120 2024-04-14T08:34:23.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.397+0000 7ff242820640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff23c0d1330 con 0x7ff23c0d2ba0 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.397+0000 7ff240dbd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff23c06cfd0 0x7ff23c06d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.397+0000 7ff240dbd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff23c06cfd0 0x7ff23c06d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48824/0 (socket says 172.21.15.172:48824) 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.397+0000 7ff240dbd640 1 -- 172.21.15.172:0/1543735169 learned_addr learned my addr 172.21.15.172:0/1543735169 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.397+0000 7ff23bfff640 1 --2- 172.21.15.172:0/1543735169 >> v2:172.21.15.177:3300/0 conn(0x7ff23c0ce120 0x7ff23c09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.397+0000 7ff23b7fe640 1 --2- 172.21.15.172:0/1543735169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23c0d2ba0 0x7ff23c09afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.397+0000 7ff240dbd640 1 -- 172.21.15.172:0/1543735169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23c0d2ba0 msgr2=0x7ff23c09afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.397+0000 7ff240dbd640 1 --2- 172.21.15.172:0/1543735169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23c0d2ba0 0x7ff23c09afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.397+0000 7ff240dbd640 1 -- 172.21.15.172:0/1543735169 >> v2:172.21.15.177:3300/0 conn(0x7ff23c0ce120 msgr2=0x7ff23c09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.397+0000 7ff240dbd640 1 --2- 172.21.15.172:0/1543735169 >> v2:172.21.15.177:3300/0 conn(0x7ff23c0ce120 0x7ff23c09aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.397+0000 7ff240dbd640 1 -- 172.21.15.172:0/1543735169 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c06c7d0 con 0x7ff23c06cfd0 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.398+0000 7ff240dbd640 1 --2- 172.21.15.172:0/1543735169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c06cfd0 0x7ff23c06d430 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff23c071ec0 tx=0x7ff23402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6c9fd78d354fda server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.398+0000 7ff23affd640 1 -- 172.21.15.172:0/1543735169 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23402ca70 con 0x7ff23c06cfd0 2024-04-14T08:34:23.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.398+0000 7ff23affd640 1 -- 172.21.15.172:0/1543735169 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff23402cc10 con 0x7ff23c06cfd0 2024-04-14T08:34:23.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.398+0000 7ff23affd640 1 -- 172.21.15.172:0/1543735169 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340343f0 con 0x7ff23c06cfd0 2024-04-14T08:34:23.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.398+0000 7ff242820640 1 -- 172.21.15.172:0/1543735169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c06cfd0 msgr2=0x7ff23c06d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.398+0000 7ff242820640 1 --2- 172.21.15.172:0/1543735169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c06cfd0 0x7ff23c06d430 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff23c071ec0 tx=0x7ff23402bed0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.398+0000 7ff242820640 1 -- 172.21.15.172:0/1543735169 shutdown_connections 2024-04-14T08:34:23.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.398+0000 7ff242820640 1 --2- 172.21.15.172:0/1543735169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c06cfd0 0x7ff23c06d430 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.398+0000 7ff242820640 1 --2- 172.21.15.172:0/1543735169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23c0d2ba0 0x7ff23c09afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.398+0000 7ff242820640 1 --2- 172.21.15.172:0/1543735169 >> v2:172.21.15.177:3300/0 conn(0x7ff23c0ce120 0x7ff23c09aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.398+0000 7ff242820640 1 -- 172.21.15.172:0/1543735169 >> 172.21.15.172:0/1543735169 conn(0x7ff23c0d2080 msgr2=0x7ff23c0d2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:23.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.398+0000 7ff242820640 1 -- 172.21.15.172:0/1543735169 shutdown_connections 2024-04-14T08:34:23.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.399+0000 7ff242820640 1 -- 172.21.15.172:0/1543735169 wait complete. 2024-04-14T08:34:23.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.400+0000 7ff242820640 1 Processor -- start 2024-04-14T08:34:23.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.400+0000 7ff242820640 1 -- start start 2024-04-14T08:34:23.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.400+0000 7ff242820640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c06cfd0 0x7ff23c09ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.400+0000 7ff242820640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23c0ce120 0x7ff23c09fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.400+0000 7ff242820640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff23c0d2ba0 0x7ff23c15fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.400+0000 7ff242820640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c0d0ce0 con 0x7ff23c06cfd0 2024-04-14T08:34:23.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.400+0000 7ff242820640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c0ce500 con 0x7ff23c0d2ba0 2024-04-14T08:34:23.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.400+0000 7ff242820640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff23c06d970 con 0x7ff23c0ce120 2024-04-14T08:34:23.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff23bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c06cfd0 0x7ff23c09ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff23bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c06cfd0 0x7ff23c09ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48836/0 (socket says 172.21.15.172:48836) 2024-04-14T08:34:23.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff23bfff640 1 -- 172.21.15.172:0/2616931519 learned_addr learned my addr 172.21.15.172:0/2616931519 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:23.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff240dbd640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff23c0d2ba0 0x7ff23c15fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff23b7fe640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23c0ce120 0x7ff23c09fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff23bfff640 1 -- 172.21.15.172:0/2616931519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23c0ce120 msgr2=0x7ff23c09fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff23bfff640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23c0ce120 0x7ff23c09fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff23bfff640 1 -- 172.21.15.172:0/2616931519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff23c0d2ba0 msgr2=0x7ff23c15fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff23bfff640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff23c0d2ba0 0x7ff23c15fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff23bfff640 1 -- 172.21.15.172:0/2616931519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c09d5d0 con 0x7ff23c06cfd0 2024-04-14T08:34:23.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff23bfff640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c06cfd0 0x7ff23c09ca30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00e9b0 tx=0x7ff22c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:23.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff238ff9640 1 -- 172.21.15.172:0/2616931519 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c017070 con 0x7ff23c06cfd0 2024-04-14T08:34:23.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff238ff9640 1 -- 172.21.15.172:0/2616931519 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff22c00c9a0 con 0x7ff23c06cfd0 2024-04-14T08:34:23.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff238ff9640 1 -- 172.21.15.172:0/2616931519 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c00cd90 con 0x7ff23c06cfd0 2024-04-14T08:34:23.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff242820640 1 -- 172.21.15.172:0/2616931519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c0c5ce0 con 0x7ff23c06cfd0 2024-04-14T08:34:23.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.401+0000 7ff242820640 1 -- 172.21.15.172:0/2616931519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c1f4160 con 0x7ff23c06cfd0 2024-04-14T08:34:23.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.402+0000 7ff242820640 1 -- 172.21.15.172:0/2616931519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23c068520 con 0x7ff23c06cfd0 2024-04-14T08:34:23.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.403+0000 7ff238ff9640 1 -- 172.21.15.172:0/2616931519 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff22c01a070 con 0x7ff23c06cfd0 2024-04-14T08:34:23.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.403+0000 7ff238ff9640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff22806c8f0 0x7ff22806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.403+0000 7ff238ff9640 1 -- 172.21.15.172:0/2616931519 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff22c08bbb0 con 0x7ff23c06cfd0 2024-04-14T08:34:23.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.403+0000 7ff23b7fe640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff22806c8f0 0x7ff22806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.404+0000 7ff23b7fe640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff22806c8f0 0x7ff22806edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff23c070d20 tx=0x7ff230009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:23.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.408+0000 7ff238ff9640 1 -- 172.21.15.172:0/2616931519 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff22c059890 con 0x7ff23c06cfd0 2024-04-14T08:34:23.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.703+0000 7ff242820640 1 -- 172.21.15.172:0/2616931519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff23c09d010 con 0x7ff23c06cfd0 2024-04-14T08:34:23.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.704+0000 7ff238ff9640 1 -- 172.21.15.172:0/2616931519 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5193 (secure 0 0 0) 0x7ff22c05db90 con 0x7ff23c06cfd0 2024-04-14T08:34:23.705 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:23.705 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1038094401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1038094401},{"type":"v1","addr":"172.21.15.177:6834","nonce":1038094401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1119512744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1119512744},{"type":"v1","addr":"172.21.15.177:6835","nonce":1119512744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3054138283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3054138283},{"type":"v1","addr":"172.21.15.172:6837","nonce":3054138283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:33:59.340499+0000","modified":"2024-04-14T08:34:01.467567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5308":{"gid":5308,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6836/3420108191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3420108191},{"type":"v1","addr":"172.21.15.172:6836","nonce":3420108191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5308,"qdb_cluster":[5308]},"id":2}]} 2024-04-14T08:34:23.706 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 16 2024-04-14T08:34:23.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.705+0000 7ff242820640 1 -- 172.21.15.172:0/2616931519 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff22806c8f0 msgr2=0x7ff22806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.705+0000 7ff242820640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff22806c8f0 0x7ff22806edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff23c070d20 tx=0x7ff230009190 comp rx=0 tx=0).stop 2024-04-14T08:34:23.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.705+0000 7ff242820640 1 -- 172.21.15.172:0/2616931519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c06cfd0 msgr2=0x7ff23c09ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.705+0000 7ff242820640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c06cfd0 0x7ff23c09ca30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00e9b0 tx=0x7ff22c00ee70 comp rx=0 tx=0).stop 2024-04-14T08:34:23.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.706+0000 7ff242820640 1 -- 172.21.15.172:0/2616931519 shutdown_connections 2024-04-14T08:34:23.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.706+0000 7ff242820640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff22806c8f0 0x7ff22806edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.706+0000 7ff242820640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff23c0d2ba0 0x7ff23c15fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.706+0000 7ff242820640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23c0ce120 0x7ff23c09fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.706+0000 7ff242820640 1 --2- 172.21.15.172:0/2616931519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c06cfd0 0x7ff23c09ca30 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.706+0000 7ff242820640 1 -- 172.21.15.172:0/2616931519 >> 172.21.15.172:0/2616931519 conn(0x7ff23c0d2080 msgr2=0x7ff23c1616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:23.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.706+0000 7ff242820640 1 -- 172.21.15.172:0/2616931519 shutdown_connections 2024-04-14T08:34:23.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.706+0000 7ff242820640 1 -- 172.21.15.172:0/2616931519 wait complete. 2024-04-14T08:34:23.718 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:34:23.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.820+0000 7f36596d1640 1 Processor -- start 2024-04-14T08:34:23.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.820+0000 7f36596d1640 1 -- start start 2024-04-14T08:34:23.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.821+0000 7f36596d1640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f365415c150 0x7f36540d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.822+0000 7f36596d1640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36540d0200 con 0x7f3654162700 2024-04-14T08:34:23.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.822+0000 7f36596d1640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36540cf160 con 0x7f36540d1840 2024-04-14T08:34:23.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.822+0000 7f36596d1640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f36540d1cb0 con 0x7f365415c150 2024-04-14T08:34:23.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.822+0000 7f36537fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3654162700 0x7f36540d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35638/0 (socket says 172.21.15.172:35638) 2024-04-14T08:34:23.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.822+0000 7f36537fe640 1 -- 172.21.15.172:0/536986921 learned_addr learned my addr 172.21.15.172:0/536986921 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:23.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.822+0000 7f3652ffd640 1 --2- 172.21.15.172:0/536986921 >> v2:172.21.15.177:3300/0 conn(0x7f365415c150 0x7f36540d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.822+0000 7f36527fc640 1 -- 172.21.15.172:0/536986921 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1151546498 0 0) 0x7f36540d0200 con 0x7f3654162700 2024-04-14T08:34:23.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.822+0000 7f36527fc640 1 -- 172.21.15.172:0/536986921 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3640003520 con 0x7f3654162700 2024-04-14T08:34:23.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.822+0000 7f3652ffd640 1 -- 172.21.15.172:0/536986921 >> v1:172.21.15.177:6789/0 conn(0x7f36540d1840 legacy=0x7f3654161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.822+0000 7f3652ffd640 1 -- 172.21.15.172:0/536986921 >> v1:172.21.15.172:6789/0 conn(0x7f3654162700 legacy=0x7f36540d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.822+0000 7f36527fc640 1 -- 172.21.15.172:0/536986921 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 870706306 0 0) 0x7f3640003520 con 0x7f3654162700 2024-04-14T08:34:23.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.822+0000 7f3652ffd640 1 -- 172.21.15.172:0/536986921 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36540cf440 con 0x7f365415c150 2024-04-14T08:34:23.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.823+0000 7f3652ffd640 1 --2- 172.21.15.172:0/536986921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f365415c150 0x7f36540d1300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f36440097a0 tx=0x7f364402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b11da5535e39d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:23.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.823+0000 7f36527fc640 1 -- 172.21.15.172:0/536986921 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644038070 con 0x7f365415c150 2024-04-14T08:34:23.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.823+0000 7f36527fc640 1 -- 172.21.15.172:0/536986921 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3644031500 con 0x7f365415c150 2024-04-14T08:34:23.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.823+0000 7f36527fc640 1 -- 172.21.15.172:0/536986921 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c001a10 con 0x7f365415c150 2024-04-14T08:34:23.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.824+0000 7f36596d1640 1 -- 172.21.15.172:0/536986921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f365415c150 msgr2=0x7f36540d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.824+0000 7f36596d1640 1 --2- 172.21.15.172:0/536986921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f365415c150 0x7f36540d1300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f36440097a0 tx=0x7f364402b820 comp rx=0 tx=0).stop 2024-04-14T08:34:23.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.824+0000 7f36596d1640 1 -- 172.21.15.172:0/536986921 shutdown_connections 2024-04-14T08:34:23.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.824+0000 7f36596d1640 1 --2- 172.21.15.172:0/536986921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f365415c150 0x7f36540d1300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.824+0000 7f36596d1640 1 -- 172.21.15.172:0/536986921 >> 172.21.15.172:0/536986921 conn(0x7f365409d580 msgr2=0x7f36540d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:23.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.824+0000 7f36596d1640 1 -- 172.21.15.172:0/536986921 shutdown_connections 2024-04-14T08:34:23.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.824+0000 7f36596d1640 1 -- 172.21.15.172:0/536986921 wait complete. 2024-04-14T08:34:23.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.826+0000 7f36596d1640 1 Processor -- start 2024-04-14T08:34:23.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.826+0000 7f36596d1640 1 -- start start 2024-04-14T08:34:23.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.827+0000 7f36596d1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36540d1840 0x7f365415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.827+0000 7f36596d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f365415c150 0x7f365415e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.827+0000 7f36596d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3654162700 0x7f365415eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.827+0000 7f36596d1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36540cfa00 con 0x7f365415c150 2024-04-14T08:34:23.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.827+0000 7f36596d1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36540cef10 con 0x7f36540d1840 2024-04-14T08:34:23.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.827+0000 7f36596d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f36540cfce0 con 0x7f3654162700 2024-04-14T08:34:23.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.827+0000 7f364bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f365415c150 0x7f365415e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.827+0000 7f364bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f365415c150 0x7f365415e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48850/0 (socket says 172.21.15.172:48850) 2024-04-14T08:34:23.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.827+0000 7f364bfff640 1 -- 172.21.15.172:0/1361192531 learned_addr learned my addr 172.21.15.172:0/1361192531 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:23.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.827+0000 7f3652ffd640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36540d1840 0x7f365415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.827+0000 7f36537fe640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3654162700 0x7f365415eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.828+0000 7f364bfff640 1 -- 172.21.15.172:0/1361192531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3654162700 msgr2=0x7f365415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.828+0000 7f364bfff640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3654162700 0x7f365415eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.828+0000 7f364bfff640 1 -- 172.21.15.172:0/1361192531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36540d1840 msgr2=0x7f365415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:23.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.828+0000 7f364bfff640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36540d1840 0x7f365415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:23.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.828+0000 7f364bfff640 1 -- 172.21.15.172:0/1361192531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36540b9e70 con 0x7f365415c150 2024-04-14T08:34:23.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.828+0000 7f364bfff640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f365415c150 0x7f365415e9b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f363c00c450 tx=0x7f363c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:23.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.828+0000 7f364b7fe640 1 -- 172.21.15.172:0/1361192531 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c0052d0 con 0x7f365415c150 2024-04-14T08:34:23.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.828+0000 7f364b7fe640 1 -- 172.21.15.172:0/1361192531 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f363c005470 con 0x7f365415c150 2024-04-14T08:34:23.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.828+0000 7f36596d1640 1 -- 172.21.15.172:0/1361192531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36540b9260 con 0x7f365415c150 2024-04-14T08:34:23.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.828+0000 7f364b7fe640 1 -- 172.21.15.172:0/1361192531 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363c012870 con 0x7f365415c150 2024-04-14T08:34:23.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.828+0000 7f36596d1640 1 -- 172.21.15.172:0/1361192531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36540b9820 con 0x7f365415c150 2024-04-14T08:34:23.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.829+0000 7f36596d1640 1 -- 172.21.15.172:0/1361192531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36540cb360 con 0x7f365415c150 2024-04-14T08:34:23.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.829+0000 7f364b7fe640 1 -- 172.21.15.172:0/1361192531 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f363c012a10 con 0x7f365415c150 2024-04-14T08:34:23.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.830+0000 7f364b7fe640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f363406c8f0 0x7f363406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:23.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.830+0000 7f364b7fe640 1 -- 172.21.15.172:0/1361192531 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f363c08b780 con 0x7f365415c150 2024-04-14T08:34:23.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.830+0000 7f3652ffd640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f363406c8f0 0x7f363406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:23.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.830+0000 7f3652ffd640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f363406c8f0 0x7f363406edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3644035000 tx=0x7f3644008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:23.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:23.834+0000 7f364b7fe640 1 -- 172.21.15.172:0/1361192531 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f36540cb360 con 0x7f365415c150 2024-04-14T08:34:24.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.019+0000 7f36596d1640 1 -- 172.21.15.172:0/1361192531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3654075490 con 0x7f365415c150 2024-04-14T08:34:24.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.020+0000 7f364b7fe640 1 -- 172.21.15.172:0/1361192531 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17002 (secure 0 0 0) 0x7f36540b9e70 con 0x7f365415c150 2024-04-14T08:34:24.022 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:24.022 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:00.984153+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:33:57.868995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-14T08:33:58.323549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:34:24.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.022+0000 7f36596d1640 1 -- 172.21.15.172:0/1361192531 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f363406c8f0 msgr2=0x7f363406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.022+0000 7f36596d1640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f363406c8f0 0x7f363406edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3644035000 tx=0x7f3644008030 comp rx=0 tx=0).stop 2024-04-14T08:34:24.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.022+0000 7f36596d1640 1 -- 172.21.15.172:0/1361192531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f365415c150 msgr2=0x7f365415e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.022+0000 7f36596d1640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f365415c150 0x7f365415e9b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f363c00c450 tx=0x7f363c00c910 comp rx=0 tx=0).stop 2024-04-14T08:34:24.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.022+0000 7f36596d1640 1 -- 172.21.15.172:0/1361192531 shutdown_connections 2024-04-14T08:34:24.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.022+0000 7f36596d1640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f363406c8f0 0x7f363406edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.022+0000 7f36596d1640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3654162700 0x7f365415eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.022+0000 7f36596d1640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f365415c150 0x7f365415e9b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.022+0000 7f36596d1640 1 --2- 172.21.15.172:0/1361192531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36540d1840 0x7f365415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.023+0000 7f36596d1640 1 -- 172.21.15.172:0/1361192531 >> 172.21.15.172:0/1361192531 conn(0x7f365409d580 msgr2=0x7f365406c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:24.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.023+0000 7f36596d1640 1 -- 172.21.15.172:0/1361192531 shutdown_connections 2024-04-14T08:34:24.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.023+0000 7f36596d1640 1 -- 172.21.15.172:0/1361192531 wait complete. 2024-04-14T08:34:24.035 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:34:24.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.136+0000 7f7007ba3640 1 Processor -- start 2024-04-14T08:34:24.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.137+0000 7f7007ba3640 1 -- start start 2024-04-14T08:34:24.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.137+0000 7f7007ba3640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f700015e430 0x7f7000162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.137+0000 7f7007ba3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70000d2e00 0x7f70001629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.137+0000 7f7007ba3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70000cefb0 con 0x7f7000162540 2024-04-14T08:34:24.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.137+0000 7f7007ba3640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f70000cf290 con 0x7f70000d2e00 2024-04-14T08:34:24.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.137+0000 7f7007ba3640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f70000cf570 con 0x7f700015e430 2024-04-14T08:34:24.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.138+0000 7f7006140640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70000d2e00 0x7f70001629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.138+0000 7f7006140640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70000d2e00 0x7f70001629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48862/0 (socket says 172.21.15.172:48862) 2024-04-14T08:34:24.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.138+0000 7f700513e640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7000162540 0x7f70000b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35652/0 (socket says 172.21.15.172:35652) 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.138+0000 7f7006140640 1 -- 172.21.15.172:0/4047052745 learned_addr learned my addr 172.21.15.172:0/4047052745 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.138+0000 7f700593f640 1 --2- 172.21.15.172:0/4047052745 >> v2:172.21.15.177:3300/0 conn(0x7f700015e430 0x7f7000162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.138+0000 7f7006140640 1 -- 172.21.15.172:0/4047052745 >> v2:172.21.15.177:3300/0 conn(0x7f700015e430 msgr2=0x7f7000162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.138+0000 7f7006140640 1 --2- 172.21.15.172:0/4047052745 >> v2:172.21.15.177:3300/0 conn(0x7f700015e430 0x7f7000162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.138+0000 7f7006140640 1 -- 172.21.15.172:0/4047052745 >> v1:172.21.15.172:6789/0 conn(0x7f7000162540 legacy=0x7f70000b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.138+0000 7f700493d640 1 -- 172.21.15.172:0/4047052745 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2941538968 0 0) 0x7f70000cefb0 con 0x7f7000162540 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.138+0000 7f7006140640 1 -- 172.21.15.172:0/4047052745 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70000cf850 con 0x7f70000d2e00 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.138+0000 7f700593f640 1 --2- 172.21.15.172:0/4047052745 >> v2:172.21.15.177:3300/0 conn(0x7f700015e430 0x7f7000162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.139+0000 7f7006140640 1 --2- 172.21.15.172:0/4047052745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70000d2e00 0x7f70001629d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f70000b6db0 tx=0x7f6ff802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b432ecd525ec3a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.139+0000 7f700493d640 1 -- 172.21.15.172:0/4047052745 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff802cb20 con 0x7f70000d2e00 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.139+0000 7f700493d640 1 -- 172.21.15.172:0/4047052745 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff802ccc0 con 0x7f70000d2e00 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.139+0000 7f700493d640 1 -- 172.21.15.172:0/4047052745 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8034610 con 0x7f70000d2e00 2024-04-14T08:34:24.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.139+0000 7f7007ba3640 1 -- 172.21.15.172:0/4047052745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70000d2e00 msgr2=0x7f70001629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.139+0000 7f7007ba3640 1 --2- 172.21.15.172:0/4047052745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70000d2e00 0x7f70001629d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f70000b6db0 tx=0x7f6ff802bf80 comp rx=0 tx=0).stop 2024-04-14T08:34:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.139+0000 7f7007ba3640 1 -- 172.21.15.172:0/4047052745 shutdown_connections 2024-04-14T08:34:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.139+0000 7f7007ba3640 1 --2- 172.21.15.172:0/4047052745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70000d2e00 0x7f70001629d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.139+0000 7f7007ba3640 1 --2- 172.21.15.172:0/4047052745 >> v2:172.21.15.177:3300/0 conn(0x7f700015e430 0x7f7000162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.139+0000 7f7007ba3640 1 -- 172.21.15.172:0/4047052745 >> 172.21.15.172:0/4047052745 conn(0x7f700009d560 msgr2=0x7f70000d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.139+0000 7f7007ba3640 1 -- 172.21.15.172:0/4047052745 shutdown_connections 2024-04-14T08:34:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.139+0000 7f7007ba3640 1 -- 172.21.15.172:0/4047052745 wait complete. 2024-04-14T08:34:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.141+0000 7f7007ba3640 1 Processor -- start 2024-04-14T08:34:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.141+0000 7f7007ba3640 1 -- start start 2024-04-14T08:34:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.141+0000 7f7007ba3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70000d2e00 0x7f70001f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.141+0000 7f7007ba3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f700015e430 0x7f70000ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.141+0000 7f7007ba3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7000162540 0x7f70000ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.141+0000 7f7007ba3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70000d4e20 con 0x7f70000d2e00 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.141+0000 7f7007ba3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f700015e810 con 0x7f7000162540 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.141+0000 7f7007ba3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f70000d0050 con 0x7f700015e430 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.141+0000 7f700593f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70000d2e00 0x7f70001f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.141+0000 7f700593f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70000d2e00 0x7f70001f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48868/0 (socket says 172.21.15.172:48868) 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.141+0000 7f700593f640 1 -- 172.21.15.172:0/651970374 learned_addr learned my addr 172.21.15.172:0/651970374 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f700513e640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f700015e430 0x7f70000ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f7006140640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7000162540 0x7f70000ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f700593f640 1 -- 172.21.15.172:0/651970374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f700015e430 msgr2=0x7f70000ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f700593f640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f700015e430 0x7f70000ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f700593f640 1 -- 172.21.15.172:0/651970374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7000162540 msgr2=0x7f70000ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f700593f640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7000162540 0x7f70000ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f700593f640 1 -- 172.21.15.172:0/651970374 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70000cd080 con 0x7f70000d2e00 2024-04-14T08:34:24.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f700593f640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70000d2e00 0x7f70001f37a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00c7f0 tx=0x7f6ffc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:24.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f6ff67fc640 1 -- 172.21.15.172:0/651970374 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc015070 con 0x7f70000d2e00 2024-04-14T08:34:24.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f6ff67fc640 1 -- 172.21.15.172:0/651970374 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ffc007a30 con 0x7f70000d2e00 2024-04-14T08:34:24.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f7007ba3640 1 -- 172.21.15.172:0/651970374 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70000cd320 con 0x7f70000d2e00 2024-04-14T08:34:24.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f6ff67fc640 1 -- 172.21.15.172:0/651970374 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc007e00 con 0x7f70000d2e00 2024-04-14T08:34:24.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.142+0000 7f7007ba3640 1 -- 172.21.15.172:0/651970374 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70000cd5c0 con 0x7f70000d2e00 2024-04-14T08:34:24.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.143+0000 7f7007ba3640 1 -- 172.21.15.172:0/651970374 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7000068590 con 0x7f70000d2e00 2024-04-14T08:34:24.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.144+0000 7f6ff67fc640 1 -- 172.21.15.172:0/651970374 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ffc018070 con 0x7f70000d2e00 2024-04-14T08:34:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.144+0000 7f6ff67fc640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6fe406c8f0 0x7f6fe406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.144+0000 7f700513e640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6fe406c8f0 0x7f6fe406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.144+0000 7f6ff67fc640 1 -- 172.21.15.172:0/651970374 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f6ffc08bca0 con 0x7f70000d2e00 2024-04-14T08:34:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.145+0000 7f700513e640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6fe406c8f0 0x7f6fe406edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f70000a2150 tx=0x7f6ff0007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.149+0000 7f6ff67fc640 1 -- 172.21.15.172:0/651970374 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6ffc059f40 con 0x7f70000d2e00 2024-04-14T08:34:24.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.367+0000 7f7007ba3640 1 -- 172.21.15.172:0/651970374 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f70000d18a0 con 0x7f70000d2e00 2024-04-14T08:34:24.369 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:34:24.367+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:34:24.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.369+0000 7f6ff67fc640 1 -- 172.21.15.172:0/651970374 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f6ffc059840 con 0x7f70000d2e00 2024-04-14T08:34:24.370 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:34:24.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.372+0000 7f7007ba3640 1 -- 172.21.15.172:0/651970374 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6fe406c8f0 msgr2=0x7f6fe406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.372+0000 7f7007ba3640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6fe406c8f0 0x7f6fe406edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f70000a2150 tx=0x7f6ff0007c00 comp rx=0 tx=0).stop 2024-04-14T08:34:24.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.372+0000 7f7007ba3640 1 -- 172.21.15.172:0/651970374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70000d2e00 msgr2=0x7f70001f37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.372+0000 7f7007ba3640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70000d2e00 0x7f70001f37a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00c7f0 tx=0x7f6ffc00cbf0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.372+0000 7f7007ba3640 1 -- 172.21.15.172:0/651970374 shutdown_connections 2024-04-14T08:34:24.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.372+0000 7f7007ba3640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6fe406c8f0 0x7f6fe406edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.372+0000 7f7007ba3640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7000162540 0x7f70000ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.372+0000 7f7007ba3640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f700015e430 0x7f70000ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.372+0000 7f7007ba3640 1 --2- 172.21.15.172:0/651970374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70000d2e00 0x7f70001f37a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.372+0000 7f7007ba3640 1 -- 172.21.15.172:0/651970374 >> 172.21.15.172:0/651970374 conn(0x7f700009d560 msgr2=0x7f700015c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:24.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.372+0000 7f7007ba3640 1 -- 172.21.15.172:0/651970374 shutdown_connections 2024-04-14T08:34:24.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.372+0000 7f7007ba3640 1 -- 172.21.15.172:0/651970374 wait complete. 2024-04-14T08:34:24.377 INFO:tasks.ceph.mds.c.smithi172.stderr: -6528> 2024-04-14T08:34:04.976+0000 7fc7b2ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:34:24.385 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-14T08:34:24.409 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:34:24.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.487+0000 7fe74219c640 1 Processor -- start 2024-04-14T08:34:24.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.487+0000 7fe74219c640 1 -- start start 2024-04-14T08:34:24.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.488+0000 7fe74219c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c0b23e0 0x7fe73c140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.488+0000 7fe74219c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe73c15b620 con 0x7fe73c15c380 2024-04-14T08:34:24.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.488+0000 7fe74219c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe73c1414a0 con 0x7fe73c13df30 2024-04-14T08:34:24.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.488+0000 7fe74219c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe73c0cf9b0 con 0x7fe73c0b23e0 2024-04-14T08:34:24.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.488+0000 7fe73bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe73c15c380 0x7fe73c140cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35658/0 (socket says 172.21.15.172:35658) 2024-04-14T08:34:24.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.488+0000 7fe73bfff640 1 -- 172.21.15.172:0/2314913232 learned_addr learned my addr 172.21.15.172:0/2314913232 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:24.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.488+0000 7fe73affd640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c0b23e0 0x7fe73c140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.488+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024113617 0 0) 0x7fe734001490 con 0x7fe73c15c380 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.488+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe724003520 con 0x7fe73c15c380 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 854776131 0 0) 0x7fe724003520 con 0x7fe73c15c380 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c0b23e0 msgr2=0x7fe73c140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c0b23e0 0x7fe73c140770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 >> v1:172.21.15.177:6789/0 conn(0x7fe73c13df30 legacy=0x7fe73c0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c15b8a0 con 0x7fe73c15c380 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fe734001bc0 con 0x7fe73c15c380 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 >> v1:172.21.15.172:6789/0 conn(0x7fe73c15c380 legacy=0x7fe73c140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7240039e0 0x7fe724003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe724004390 0x7fe724004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c0b23e0 0x7fe7240050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c0cf9b0 con 0x7fe724004390 2024-04-14T08:34:24.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe72400dbd0 con 0x7fe7240039e0 2024-04-14T08:34:24.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe72400ddd0 con 0x7fe73c0b23e0 2024-04-14T08:34:24.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73affd640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe724004390 0x7fe724004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73b7fe640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7240039e0 0x7fe724003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73bfff640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c0b23e0 0x7fe7240050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73affd640 1 -- 172.21.15.172:0/2314913232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c0b23e0 msgr2=0x7fe7240050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:24.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73affd640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c0b23e0 0x7fe7240050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73affd640 1 -- 172.21.15.172:0/2314913232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7240039e0 msgr2=0x7fe724003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:24.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73affd640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7240039e0 0x7fe724003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73affd640 1 -- 172.21.15.172:0/2314913232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe724003520 con 0x7fe724004390 2024-04-14T08:34:24.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.489+0000 7fe73bfff640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c0b23e0 0x7fe7240050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:24.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.490+0000 7fe73affd640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe724004390 0x7fe724004860 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe730009ac0 tx=0x7fe724005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6669b269b44533 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:24.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.490+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730039070 con 0x7fe724004390 2024-04-14T08:34:24.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.490+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe73002c6b0 con 0x7fe724004390 2024-04-14T08:34:24.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.490+0000 7fe73a7fc640 1 -- 172.21.15.172:0/2314913232 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730009690 con 0x7fe724004390 2024-04-14T08:34:24.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.490+0000 7fe74219c640 1 -- 172.21.15.172:0/2314913232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe724004390 msgr2=0x7fe724004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.490+0000 7fe74219c640 1 --2- 172.21.15.172:0/2314913232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe724004390 0x7fe724004860 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe730009ac0 tx=0x7fe724005830 comp rx=0 tx=0).stop 2024-04-14T08:34:24.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.490+0000 7fe73bfff640 1 -- 172.21.15.172:0/2314913232 reap_dead start 2024-04-14T08:34:24.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.491+0000 7fe74219c640 1 -- 172.21.15.172:0/2314913232 shutdown_connections 2024-04-14T08:34:24.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.491+0000 7fe74219c640 1 -- 172.21.15.172:0/2314913232 >> 172.21.15.172:0/2314913232 conn(0x7fe73c06b520 msgr2=0x7fe73c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:24.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.491+0000 7fe74219c640 1 -- 172.21.15.172:0/2314913232 shutdown_connections 2024-04-14T08:34:24.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.491+0000 7fe74219c640 1 -- 172.21.15.172:0/2314913232 wait complete. 2024-04-14T08:34:24.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.492+0000 7fe74219c640 1 Processor -- start 2024-04-14T08:34:24.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.492+0000 7fe74219c640 1 -- start start 2024-04-14T08:34:24.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe74219c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe724004390 0x7fe73c139330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe74219c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c156920 0x7fe73c156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe74219c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe73c158140 0x7fe73c1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe74219c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c13e310 con 0x7fe724004390 2024-04-14T08:34:24.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe74219c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c15be60 con 0x7fe73c158140 2024-04-14T08:34:24.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe74219c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe73c140770 con 0x7fe73c156920 2024-04-14T08:34:24.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe73b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe724004390 0x7fe73c139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe73b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe724004390 0x7fe73c139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48886/0 (socket says 172.21.15.172:48886) 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe73b7fe640 1 -- 172.21.15.172:0/2164008660 learned_addr learned my addr 172.21.15.172:0/2164008660 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe73bfff640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe73c158140 0x7fe73c1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe73affd640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c156920 0x7fe73c156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe73b7fe640 1 -- 172.21.15.172:0/2164008660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c156920 msgr2=0x7fe73c156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe73b7fe640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c156920 0x7fe73c156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe73b7fe640 1 -- 172.21.15.172:0/2164008660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe73c158140 msgr2=0x7fe73c1585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe73b7fe640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe73c158140 0x7fe73c1585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe73b7fe640 1 -- 172.21.15.172:0/2164008660 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c1573a0 con 0x7fe724004390 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe73bfff640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe73c158140 0x7fe73c1585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe73b7fe640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe724004390 0x7fe73c139330 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00f7c0 tx=0x7fe72c00fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe723fff640 1 -- 172.21.15.172:0/2164008660 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0052e0 con 0x7fe724004390 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe723fff640 1 -- 172.21.15.172:0/2164008660 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe72c018070 con 0x7fe724004390 2024-04-14T08:34:24.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe723fff640 1 -- 172.21.15.172:0/2164008660 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00d970 con 0x7fe724004390 2024-04-14T08:34:24.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe74219c640 1 -- 172.21.15.172:0/2164008660 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c157680 con 0x7fe724004390 2024-04-14T08:34:24.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.493+0000 7fe74219c640 1 -- 172.21.15.172:0/2164008660 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c157c40 con 0x7fe724004390 2024-04-14T08:34:24.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.494+0000 7fe74219c640 1 -- 172.21.15.172:0/2164008660 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73c068590 con 0x7fe724004390 2024-04-14T08:34:24.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.495+0000 7fe723fff640 1 -- 172.21.15.172:0/2164008660 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe72c00db10 con 0x7fe724004390 2024-04-14T08:34:24.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.495+0000 7fe723fff640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe72806c8f0 0x7fe72806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:24.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.495+0000 7fe723fff640 1 -- 172.21.15.172:0/2164008660 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fe72c08bd40 con 0x7fe724004390 2024-04-14T08:34:24.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.495+0000 7fe73affd640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe72806c8f0 0x7fe72806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:24.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.496+0000 7fe73affd640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe72806c8f0 0x7fe72806edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe730009ac0 tx=0x7fe730030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:24.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.500+0000 7fe723fff640 1 -- 172.21.15.172:0/2164008660 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe72c059f90 con 0x7fe724004390 2024-04-14T08:34:24.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:24.712+0000 7fe74219c640 1 -- 172.21.15.172:0/2164008660 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe73c0b38b0 con 0x7fe724004390 2024-04-14T08:34:25.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.373+0000 7fe723fff640 1 -- 172.21.15.172:0/2164008660 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7fe72c059890 con 0x7fe724004390 2024-04-14T08:34:25.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.376+0000 7fe74219c640 1 -- 172.21.15.172:0/2164008660 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe72806c8f0 msgr2=0x7fe72806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:25.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.376+0000 7fe74219c640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe72806c8f0 0x7fe72806edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe730009ac0 tx=0x7fe730030000 comp rx=0 tx=0).stop 2024-04-14T08:34:25.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.376+0000 7fe74219c640 1 -- 172.21.15.172:0/2164008660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe724004390 msgr2=0x7fe73c139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:25.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.376+0000 7fe74219c640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe724004390 0x7fe73c139330 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00f7c0 tx=0x7fe72c00fc80 comp rx=0 tx=0).stop 2024-04-14T08:34:25.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.376+0000 7fe74219c640 1 -- 172.21.15.172:0/2164008660 shutdown_connections 2024-04-14T08:34:25.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.376+0000 7fe74219c640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe72806c8f0 0x7fe72806edb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:25.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.376+0000 7fe74219c640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe73c158140 0x7fe73c1585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:25.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.376+0000 7fe74219c640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe73c156920 0x7fe73c156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:25.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.376+0000 7fe74219c640 1 --2- 172.21.15.172:0/2164008660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe724004390 0x7fe73c139330 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:25.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.376+0000 7fe74219c640 1 -- 172.21.15.172:0/2164008660 >> 172.21.15.172:0/2164008660 conn(0x7fe73c06b520 msgr2=0x7fe73c15ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:25.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.376+0000 7fe74219c640 1 -- 172.21.15.172:0/2164008660 shutdown_connections 2024-04-14T08:34:25.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.376+0000 7fe74219c640 1 -- 172.21.15.172:0/2164008660 wait complete. 2024-04-14T08:34:25.391 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-14T08:34:25.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.537+0000 7f1155fa3640 1 Processor -- start 2024-04-14T08:34:25.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.537+0000 7f1155fa3640 1 -- start start 2024-04-14T08:34:25.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f1155fa3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11500cc340 0x7f11500ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:25.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f1155fa3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11500cdb90 con 0x7f11500cecc0 2024-04-14T08:34:25.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f1155fa3640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11500cde70 con 0x7f115015b350 2024-04-14T08:34:25.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f1155fa3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11500cf150 con 0x7f11500cc340 2024-04-14T08:34:25.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f114ffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f11500cecc0 0x7f11500c4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35666/0 (socket says 172.21.15.172:35666) 2024-04-14T08:34:25.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f114ffff640 1 -- 172.21.15.172:0/3467472639 learned_addr learned my addr 172.21.15.172:0/3467472639 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:25.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f114effd640 1 --2- 172.21.15.172:0/3467472639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11500cc340 0x7f11500ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:25.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f114e7fc640 1 -- 172.21.15.172:0/3467472639 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3289119830 0 0) 0x7f11500cdb90 con 0x7f11500cecc0 2024-04-14T08:34:25.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f114e7fc640 1 -- 172.21.15.172:0/3467472639 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1138005730 con 0x7f11500cecc0 2024-04-14T08:34:25.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f114e7fc640 1 -- 172.21.15.172:0/3467472639 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2735488568 0 0) 0x7f11500cde70 con 0x7f115015b350 2024-04-14T08:34:25.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f114e7fc640 1 -- 172.21.15.172:0/3467472639 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11500cdb90 con 0x7f115015b350 2024-04-14T08:34:25.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f114e7fc640 1 -- 172.21.15.172:0/3467472639 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4069453967 0 0) 0x7f1138005730 con 0x7f11500cecc0 2024-04-14T08:34:25.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f114effd640 1 -- 172.21.15.172:0/3467472639 >> v1:172.21.15.177:6789/0 conn(0x7f115015b350 legacy=0x7f11500cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:25.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f114effd640 1 -- 172.21.15.172:0/3467472639 >> v1:172.21.15.172:6789/0 conn(0x7f11500cecc0 legacy=0x7f11500c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:25.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.538+0000 7f114effd640 1 -- 172.21.15.172:0/3467472639 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11500ce150 con 0x7f11500cc340 2024-04-14T08:34:25.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.539+0000 7f114effd640 1 --2- 172.21.15.172:0/3467472639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11500cc340 0x7f11500ce780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f11440097d0 tx=0x7f114402b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c64e8cc584c4f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:25.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.539+0000 7f114e7fc640 1 -- 172.21.15.172:0/3467472639 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144037070 con 0x7f11500cc340 2024-04-14T08:34:25.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.539+0000 7f114e7fc640 1 -- 172.21.15.172:0/3467472639 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1144030510 con 0x7f11500cc340 2024-04-14T08:34:25.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.539+0000 7f114e7fc640 1 -- 172.21.15.172:0/3467472639 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11480019f0 con 0x7f11500cc340 2024-04-14T08:34:25.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.540+0000 7f1155fa3640 1 -- 172.21.15.172:0/3467472639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11500cc340 msgr2=0x7f11500ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:25.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.540+0000 7f1155fa3640 1 --2- 172.21.15.172:0/3467472639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11500cc340 0x7f11500ce780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f11440097d0 tx=0x7f114402b770 comp rx=0 tx=0).stop 2024-04-14T08:34:25.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.540+0000 7f1155fa3640 1 -- 172.21.15.172:0/3467472639 shutdown_connections 2024-04-14T08:34:25.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.540+0000 7f1155fa3640 1 --2- 172.21.15.172:0/3467472639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11500cc340 0x7f11500ce780 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:25.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.540+0000 7f1155fa3640 1 -- 172.21.15.172:0/3467472639 >> 172.21.15.172:0/3467472639 conn(0x7f11500b1a50 msgr2=0x7f11500b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:25.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.540+0000 7f1155fa3640 1 -- 172.21.15.172:0/3467472639 shutdown_connections 2024-04-14T08:34:25.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.540+0000 7f1155fa3640 1 -- 172.21.15.172:0/3467472639 wait complete. 2024-04-14T08:34:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.541+0000 7f1155fa3640 1 Processor -- start 2024-04-14T08:34:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.541+0000 7f1155fa3640 1 -- start start 2024-04-14T08:34:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f1155fa3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11500cc340 0x7f1150158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f1155fa3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11500cecc0 0x7f11500c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f1155fa3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115015b350 0x7f115009bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f1155fa3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11500ce4c0 con 0x7f115015b350 2024-04-14T08:34:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f1155fa3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f11500cd940 con 0x7f11500cecc0 2024-04-14T08:34:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f1155fa3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11500c5580 con 0x7f11500cc340 2024-04-14T08:34:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f114ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115015b350 0x7f115009bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f114ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115015b350 0x7f115009bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48892/0 (socket says 172.21.15.172:48892) 2024-04-14T08:34:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f114ffff640 1 -- 172.21.15.172:0/3326912938 learned_addr learned my addr 172.21.15.172:0/3326912938 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f114effd640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11500cecc0 0x7f11500c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:25.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f114f7fe640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11500cc340 0x7f1150158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:25.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f114ffff640 1 -- 172.21.15.172:0/3326912938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11500cc340 msgr2=0x7f1150158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:25.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f114ffff640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11500cc340 0x7f1150158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:25.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f114ffff640 1 -- 172.21.15.172:0/3326912938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11500cecc0 msgr2=0x7f11500c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:25.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f114ffff640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11500cecc0 0x7f11500c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:25.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.542+0000 7f114ffff640 1 -- 172.21.15.172:0/3326912938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115009ab40 con 0x7f115015b350 2024-04-14T08:34:25.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.543+0000 7f114ffff640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115015b350 0x7f115009bbe0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1148001560 tx=0x7f114800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:25.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.543+0000 7f1137fff640 1 -- 172.21.15.172:0/3326912938 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148012400 con 0x7f115015b350 2024-04-14T08:34:25.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.543+0000 7f1137fff640 1 -- 172.21.15.172:0/3326912938 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11480125a0 con 0x7f115015b350 2024-04-14T08:34:25.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.543+0000 7f1155fa3640 1 -- 172.21.15.172:0/3326912938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115009ae20 con 0x7f115015b350 2024-04-14T08:34:25.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.543+0000 7f1137fff640 1 -- 172.21.15.172:0/3326912938 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11480129d0 con 0x7f115015b350 2024-04-14T08:34:25.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.543+0000 7f1155fa3640 1 -- 172.21.15.172:0/3326912938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f115009b3e0 con 0x7f115015b350 2024-04-14T08:34:25.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.544+0000 7f1155fa3640 1 -- 172.21.15.172:0/3326912938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11500686a0 con 0x7f115015b350 2024-04-14T08:34:25.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.544+0000 7f1137fff640 1 -- 172.21.15.172:0/3326912938 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1148012b70 con 0x7f115015b350 2024-04-14T08:34:25.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.545+0000 7f1137fff640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f113c06c8f0 0x7f113c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:25.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.545+0000 7f1137fff640 1 -- 172.21.15.172:0/3326912938 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7f114808c4b0 con 0x7f115015b350 2024-04-14T08:34:25.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.545+0000 7f114f7fe640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f113c06c8f0 0x7f113c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:25.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.545+0000 7f114f7fe640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f113c06c8f0 0x7f113c06edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1140007fd0 tx=0x7f1140009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:25.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.549+0000 7f1137fff640 1 -- 172.21.15.172:0/3326912938 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f114805a0b0 con 0x7f115015b350 2024-04-14T08:34:25.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:25.726+0000 7f1155fa3640 1 -- 172.21.15.172:0/3326912938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f11500b23f0 con 0x7f115015b350 2024-04-14T08:34:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.375+0000 7f1137fff640 1 -- 172.21.15.172:0/3326912938 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v41) ==== 181+0+0 (secure 0 0 0) 0x7f114805e3b0 con 0x7f115015b350 2024-04-14T08:34:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' removed 2024-04-14T08:34:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.379+0000 7f1155fa3640 1 -- 172.21.15.172:0/3326912938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f113c06c8f0 msgr2=0x7f113c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.379+0000 7f1155fa3640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f113c06c8f0 0x7f113c06edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1140007fd0 tx=0x7f1140009040 comp rx=0 tx=0).stop 2024-04-14T08:34:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.379+0000 7f1155fa3640 1 -- 172.21.15.172:0/3326912938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115015b350 msgr2=0x7f115009bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.379+0000 7f1155fa3640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115015b350 0x7f115009bbe0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1148001560 tx=0x7f114800a430 comp rx=0 tx=0).stop 2024-04-14T08:34:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.379+0000 7f1155fa3640 1 -- 172.21.15.172:0/3326912938 shutdown_connections 2024-04-14T08:34:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.379+0000 7f1155fa3640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f113c06c8f0 0x7f113c06edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.379+0000 7f1155fa3640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115015b350 0x7f115009bbe0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.379+0000 7f1155fa3640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11500cecc0 0x7f11500c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.379+0000 7f1155fa3640 1 --2- 172.21.15.172:0/3326912938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11500cc340 0x7f1150158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.379+0000 7f1155fa3640 1 -- 172.21.15.172:0/3326912938 >> 172.21.15.172:0/3326912938 conn(0x7f11500b1a50 msgr2=0x7f11500acda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.379+0000 7f1155fa3640 1 -- 172.21.15.172:0/3326912938 shutdown_connections 2024-04-14T08:34:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.379+0000 7f1155fa3640 1 -- 172.21.15.172:0/3326912938 wait complete. 2024-04-14T08:34:26.394 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-14T08:34:26.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.503+0000 7f4d95460640 1 Processor -- start 2024-04-14T08:34:26.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.503+0000 7f4d95460640 1 -- start start 2024-04-14T08:34:26.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.503+0000 7f4d95460640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d9015c2b0 0x7f4d900c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:26.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.503+0000 7f4d95460640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4d900cc060 0x7f4d9006d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:26.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.503+0000 7f4d95460640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d900ca3c0 0x7f4d900ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:26.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.503+0000 7f4d95460640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f4d900c6bf0 con 0x7f4d9015c2b0 2024-04-14T08:34:26.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.503+0000 7f4d95460640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f4d900cc580 con 0x7f4d900cc060 2024-04-14T08:34:26.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.503+0000 7f4d95460640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4d90160ae0 con 0x7f4d900ca3c0 2024-04-14T08:34:26.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.504+0000 7f4d8effd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d9015c2b0 0x7f4d900c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:26.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.504+0000 7f4d8effd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d9015c2b0 0x7f4d900c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48908/0 (socket says 172.21.15.172:48908) 2024-04-14T08:34:26.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.504+0000 7f4d8effd640 1 -- 172.21.15.172:0/817434116 learned_addr learned my addr 172.21.15.172:0/817434116 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:26.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.504+0000 7f4d8effd640 1 -- 172.21.15.172:0/817434116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d900ca3c0 msgr2=0x7f4d900ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:26.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.504+0000 7f4d8effd640 1 --2- 172.21.15.172:0/817434116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d900ca3c0 0x7f4d900ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:26.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.504+0000 7f4d8effd640 1 -- 172.21.15.172:0/817434116 >> v2:172.21.15.177:3300/0 conn(0x7f4d900cc060 msgr2=0x7f4d9006d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:26.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.504+0000 7f4d8effd640 1 --2- 172.21.15.172:0/817434116 >> v2:172.21.15.177:3300/0 conn(0x7f4d900cc060 0x7f4d9006d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:26.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.504+0000 7f4d8effd640 1 -- 172.21.15.172:0/817434116 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d900c66d0 con 0x7f4d9015c2b0 2024-04-14T08:34:26.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d8effd640 1 --2- 172.21.15.172:0/817434116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d9015c2b0 0x7f4d900c5ee0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4d90070f90 tx=0x7f4d8002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f566e61997815dba server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:26.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d8dffb640 1 -- 172.21.15.172:0/817434116 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8002ca70 con 0x7f4d9015c2b0 2024-04-14T08:34:26.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d8dffb640 1 -- 172.21.15.172:0/817434116 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d8002cc10 con 0x7f4d9015c2b0 2024-04-14T08:34:26.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d8dffb640 1 -- 172.21.15.172:0/817434116 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800343f0 con 0x7f4d9015c2b0 2024-04-14T08:34:26.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d95460640 1 -- 172.21.15.172:0/817434116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d9015c2b0 msgr2=0x7f4d900c5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:26.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d95460640 1 --2- 172.21.15.172:0/817434116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d9015c2b0 0x7f4d900c5ee0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4d90070f90 tx=0x7f4d8002bed0 comp rx=0 tx=0).stop 2024-04-14T08:34:26.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d95460640 1 -- 172.21.15.172:0/817434116 shutdown_connections 2024-04-14T08:34:26.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d95460640 1 --2- 172.21.15.172:0/817434116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d900ca3c0 0x7f4d900ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:26.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d95460640 1 --2- 172.21.15.172:0/817434116 >> v2:172.21.15.177:3300/0 conn(0x7f4d900cc060 0x7f4d9006d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:26.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d95460640 1 --2- 172.21.15.172:0/817434116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d9015c2b0 0x7f4d900c5ee0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:26.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d95460640 1 -- 172.21.15.172:0/817434116 >> 172.21.15.172:0/817434116 conn(0x7f4d900b1a50 msgr2=0x7f4d900b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:26.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d95460640 1 -- 172.21.15.172:0/817434116 shutdown_connections 2024-04-14T08:34:26.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.505+0000 7f4d95460640 1 -- 172.21.15.172:0/817434116 wait complete. 2024-04-14T08:34:26.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.507+0000 7f4d95460640 1 Processor -- start 2024-04-14T08:34:26.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.507+0000 7f4d95460640 1 -- start start 2024-04-14T08:34:26.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.507+0000 7f4d95460640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d900ca3c0 0x7f4d900c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.507+0000 7f4d95460640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d900cc060 0x7f4d9009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.507+0000 7f4d95460640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d9015c2b0 0x7f4d90156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.507+0000 7f4d95460640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4d900b2870 con 0x7f4d900cc060 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.507+0000 7f4d95460640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4d900b26f0 con 0x7f4d900ca3c0 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.507+0000 7f4d95460640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9006d820 con 0x7f4d9015c2b0 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.507+0000 7f4d8e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d900cc060 0x7f4d9009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.507+0000 7f4d8e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d900cc060 0x7f4d9009aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48918/0 (socket says 172.21.15.172:48918) 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.507+0000 7f4d8e7fc640 1 -- 172.21.15.172:0/32035433 learned_addr learned my addr 172.21.15.172:0/32035433 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d8effd640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d900ca3c0 0x7f4d900c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d8f7fe640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d9015c2b0 0x7f4d90156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d8e7fc640 1 -- 172.21.15.172:0/32035433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d9015c2b0 msgr2=0x7f4d90156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d8e7fc640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d9015c2b0 0x7f4d90156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d8e7fc640 1 -- 172.21.15.172:0/32035433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d900ca3c0 msgr2=0x7f4d900c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:26.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d8e7fc640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d900ca3c0 0x7f4d900c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:26.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d8e7fc640 1 -- 172.21.15.172:0/32035433 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d9009b370 con 0x7f4d900cc060 2024-04-14T08:34:26.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d8e7fc640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d900cc060 0x7f4d9009aa70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400b440 tx=0x7f4d8400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:26.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d777fe640 1 -- 172.21.15.172:0/32035433 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84015020 con 0x7f4d900cc060 2024-04-14T08:34:26.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d777fe640 1 -- 172.21.15.172:0/32035433 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d84011890 con 0x7f4d900cc060 2024-04-14T08:34:26.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d95460640 1 -- 172.21.15.172:0/32035433 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d9009b650 con 0x7f4d900cc060 2024-04-14T08:34:26.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d777fe640 1 -- 172.21.15.172:0/32035433 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84011c80 con 0x7f4d900cc060 2024-04-14T08:34:26.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.508+0000 7f4d95460640 1 -- 172.21.15.172:0/32035433 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d9009bc10 con 0x7f4d900cc060 2024-04-14T08:34:26.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.509+0000 7f4d95460640 1 -- 172.21.15.172:0/32035433 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d900c7970 con 0x7f4d900cc060 2024-04-14T08:34:26.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.510+0000 7f4d777fe640 1 -- 172.21.15.172:0/32035433 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d84018020 con 0x7f4d900cc060 2024-04-14T08:34:26.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.510+0000 7f4d777fe640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d7c06c8f0 0x7f4d7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:26.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.510+0000 7f4d777fe640 1 -- 172.21.15.172:0/32035433 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5749+0+0 (secure 0 0 0) 0x7f4d8408bb60 con 0x7f4d900cc060 2024-04-14T08:34:26.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.510+0000 7f4d8effd640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d7c06c8f0 0x7f4d7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:26.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.511+0000 7f4d8effd640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d7c06c8f0 0x7f4d7c06edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4d80035000 tx=0x7f4d80030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:26.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.515+0000 7f4d777fe640 1 -- 172.21.15.172:0/32035433 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4d8401f020 con 0x7f4d900cc060 2024-04-14T08:34:26.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:26.689+0000 7f4d95460640 1 -- 172.21.15.172:0/32035433 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4d900c64b0 con 0x7f4d900cc060 2024-04-14T08:34:27.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.379+0000 7f4d777fe640 1 -- 172.21.15.172:0/32035433 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v42) ==== 169+0+0 (secure 0 0 0) 0x7f4d8400d040 con 0x7f4d900cc060 2024-04-14T08:34:27.381 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' removed 2024-04-14T08:34:27.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.383+0000 7f4d95460640 1 -- 172.21.15.172:0/32035433 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d7c06c8f0 msgr2=0x7f4d7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:27.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.383+0000 7f4d95460640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d7c06c8f0 0x7f4d7c06edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4d80035000 tx=0x7f4d80030000 comp rx=0 tx=0).stop 2024-04-14T08:34:27.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.383+0000 7f4d95460640 1 -- 172.21.15.172:0/32035433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d900cc060 msgr2=0x7f4d9009aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:27.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.383+0000 7f4d95460640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d900cc060 0x7f4d9009aa70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400b440 tx=0x7f4d8400b900 comp rx=0 tx=0).stop 2024-04-14T08:34:27.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.383+0000 7f4d95460640 1 -- 172.21.15.172:0/32035433 shutdown_connections 2024-04-14T08:34:27.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.383+0000 7f4d95460640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d7c06c8f0 0x7f4d7c06edb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.383+0000 7f4d95460640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d9015c2b0 0x7f4d90156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.383+0000 7f4d95460640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d900cc060 0x7f4d9009aa70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.383+0000 7f4d95460640 1 --2- 172.21.15.172:0/32035433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d900ca3c0 0x7f4d900c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.383+0000 7f4d95460640 1 -- 172.21.15.172:0/32035433 >> 172.21.15.172:0/32035433 conn(0x7f4d900b1a50 msgr2=0x7f4d900ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:27.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.383+0000 7f4d95460640 1 -- 172.21.15.172:0/32035433 shutdown_connections 2024-04-14T08:34:27.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.383+0000 7f4d95460640 1 -- 172.21.15.172:0/32035433 wait complete. 2024-04-14T08:34:27.398 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-14T08:34:27.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.502+0000 7f6057c80640 1 Processor -- start 2024-04-14T08:34:27.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.502+0000 7f6057c80640 1 -- start start 2024-04-14T08:34:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.503+0000 7f6057c80640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f60500b3dc0 0x7f605006b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.503+0000 7f6057c80640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f605006ba60 0x7f60501683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.503+0000 7f6057c80640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f605015edd0 con 0x7f605015f460 2024-04-14T08:34:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.503+0000 7f6057c80640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f6050141960 con 0x7f60500b3dc0 2024-04-14T08:34:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.503+0000 7f6057c80640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f605006beb0 con 0x7f605006ba60 2024-04-14T08:34:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.503+0000 7f605521b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f60500b3dc0 0x7f605006b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.503+0000 7f6055a1c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f605015f460 0x7f60500b36b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35674/0 (socket says 172.21.15.172:35674) 2024-04-14T08:34:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.503+0000 7f6055a1c640 1 -- 172.21.15.172:0/55824824 learned_addr learned my addr 172.21.15.172:0/55824824 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.504+0000 7f605521b640 1 -- 172.21.15.172:0/55824824 >> v2:172.21.15.177:3300/0 conn(0x7f605006ba60 msgr2=0x7f60501683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.504+0000 7f605521b640 1 --2- 172.21.15.172:0/55824824 >> v2:172.21.15.177:3300/0 conn(0x7f605006ba60 0x7f60501683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.504+0000 7f605521b640 1 -- 172.21.15.172:0/55824824 >> v1:172.21.15.172:6789/0 conn(0x7f605015f460 legacy=0x7f60500b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.504+0000 7f605521b640 1 -- 172.21.15.172:0/55824824 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605006c100 con 0x7f60500b3dc0 2024-04-14T08:34:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.504+0000 7f6054a1a640 1 -- 172.21.15.172:0/55824824 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1357837516 0 0) 0x7f605015edd0 con 0x7f605015f460 2024-04-14T08:34:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.504+0000 7f605521b640 1 --2- 172.21.15.172:0/55824824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60500b3dc0 0x7f605006b520 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f60500b2340 tx=0x7f604002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f5874ef139e76f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.504+0000 7f6054a1a640 1 -- 172.21.15.172:0/55824824 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60400315a0 con 0x7f60500b3dc0 2024-04-14T08:34:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.504+0000 7f6054a1a640 1 -- 172.21.15.172:0/55824824 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6040037070 con 0x7f60500b3dc0 2024-04-14T08:34:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.504+0000 7f6054a1a640 1 -- 172.21.15.172:0/55824824 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040031960 con 0x7f60500b3dc0 2024-04-14T08:34:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.505+0000 7f6057c80640 1 -- 172.21.15.172:0/55824824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60500b3dc0 msgr2=0x7f605006b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.505+0000 7f6057c80640 1 --2- 172.21.15.172:0/55824824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60500b3dc0 0x7f605006b520 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f60500b2340 tx=0x7f604002b840 comp rx=0 tx=0).stop 2024-04-14T08:34:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.505+0000 7f6057c80640 1 -- 172.21.15.172:0/55824824 shutdown_connections 2024-04-14T08:34:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.505+0000 7f6057c80640 1 --2- 172.21.15.172:0/55824824 >> v2:172.21.15.177:3300/0 conn(0x7f605006ba60 0x7f60501683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.505+0000 7f6057c80640 1 --2- 172.21.15.172:0/55824824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60500b3dc0 0x7f605006b520 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.505+0000 7f6057c80640 1 -- 172.21.15.172:0/55824824 >> 172.21.15.172:0/55824824 conn(0x7f60500d0780 msgr2=0x7f60500d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.505+0000 7f6057c80640 1 -- 172.21.15.172:0/55824824 shutdown_connections 2024-04-14T08:34:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.505+0000 7f6057c80640 1 -- 172.21.15.172:0/55824824 wait complete. 2024-04-14T08:34:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.506+0000 7f6057c80640 1 Processor -- start 2024-04-14T08:34:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.506+0000 7f6057c80640 1 -- start start 2024-04-14T08:34:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6057c80640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f605006ba60 0x7f60501618d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6057c80640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60500b3dc0 0x7f6050156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6057c80640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f605015f460 0x7f60500b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6057c80640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f605006c9a0 con 0x7f605006ba60 2024-04-14T08:34:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6057c80640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f605006c6c0 con 0x7f605015f460 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6057c80640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f605015f840 con 0x7f60500b3dc0 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6055a1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f605006ba60 0x7f60501618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6055a1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f605006ba60 0x7f60501618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48938/0 (socket says 172.21.15.172:48938) 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6055a1c640 1 -- 172.21.15.172:0/696972539 learned_addr learned my addr 172.21.15.172:0/696972539 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6055a1c640 1 -- 172.21.15.172:0/696972539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60500b3dc0 msgr2=0x7f6050156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6055a1c640 1 --2- 172.21.15.172:0/696972539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60500b3dc0 0x7f6050156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6055a1c640 1 -- 172.21.15.172:0/696972539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f605015f460 msgr2=0x7f60500b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6055a1c640 1 --2- 172.21.15.172:0/696972539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f605015f460 0x7f60500b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6055a1c640 1 -- 172.21.15.172:0/696972539 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60501615b0 con 0x7f605006ba60 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.507+0000 7f6055a1c640 1 --2- 172.21.15.172:0/696972539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f605006ba60 0x7f60501618d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f604c00c8c0 tx=0x7f604c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.508+0000 7f60467fc640 1 -- 172.21.15.172:0/696972539 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c018070 con 0x7f605006ba60 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.508+0000 7f60467fc640 1 -- 172.21.15.172:0/696972539 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604c00a640 con 0x7f605006ba60 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.508+0000 7f6057c80640 1 -- 172.21.15.172:0/696972539 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60501f3f80 con 0x7f605006ba60 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.508+0000 7f60467fc640 1 -- 172.21.15.172:0/696972539 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c00aa30 con 0x7f605006ba60 2024-04-14T08:34:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.508+0000 7f6057c80640 1 -- 172.21.15.172:0/696972539 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6050158710 con 0x7f605006ba60 2024-04-14T08:34:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.509+0000 7f6057c80640 1 -- 172.21.15.172:0/696972539 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6050068590 con 0x7f605006ba60 2024-04-14T08:34:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.509+0000 7f60467fc640 1 -- 172.21.15.172:0/696972539 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f604c01b070 con 0x7f605006ba60 2024-04-14T08:34:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.510+0000 7f60467fc640 1 --2- 172.21.15.172:0/696972539 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f603406c8f0 0x7f603406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.510+0000 7f605521b640 1 --2- 172.21.15.172:0/696972539 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f603406c8f0 0x7f603406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.510+0000 7f60467fc640 1 -- 172.21.15.172:0/696972539 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5335+0+0 (secure 0 0 0) 0x7f604c08b050 con 0x7f605006ba60 2024-04-14T08:34:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.510+0000 7f605521b640 1 --2- 172.21.15.172:0/696972539 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f603406c8f0 0x7f603406edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6040036000 tx=0x7f6040030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:27.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.514+0000 7f60467fc640 1 -- 172.21.15.172:0/696972539 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f604c0595f0 con 0x7f605006ba60 2024-04-14T08:34:27.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.690+0000 7f6057c80640 1 -- 172.21.15.172:0/696972539 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f6050067670 con 0x7f605006ba60 2024-04-14T08:34:27.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.693+0000 7f60467fc640 1 -- 172.21.15.172:0/696972539 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f604c058ef0 con 0x7f605006ba60 2024-04-14T08:34:27.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.696+0000 7f6057c80640 1 -- 172.21.15.172:0/696972539 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f603406c8f0 msgr2=0x7f603406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:27.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.696+0000 7f6057c80640 1 --2- 172.21.15.172:0/696972539 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f603406c8f0 0x7f603406edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6040036000 tx=0x7f6040030000 comp rx=0 tx=0).stop 2024-04-14T08:34:27.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.696+0000 7f6057c80640 1 -- 172.21.15.172:0/696972539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f605006ba60 msgr2=0x7f60501618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:27.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.696+0000 7f6057c80640 1 --2- 172.21.15.172:0/696972539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f605006ba60 0x7f60501618d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f604c00c8c0 tx=0x7f604c00cd80 comp rx=0 tx=0).stop 2024-04-14T08:34:27.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.696+0000 7f6057c80640 1 -- 172.21.15.172:0/696972539 shutdown_connections 2024-04-14T08:34:27.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.696+0000 7f6057c80640 1 --2- 172.21.15.172:0/696972539 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f603406c8f0 0x7f603406edb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.696+0000 7f6057c80640 1 --2- 172.21.15.172:0/696972539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f605015f460 0x7f60500b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.696+0000 7f6057c80640 1 --2- 172.21.15.172:0/696972539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60500b3dc0 0x7f6050156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.696+0000 7f6057c80640 1 --2- 172.21.15.172:0/696972539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f605006ba60 0x7f60501618d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.696+0000 7f6057c80640 1 -- 172.21.15.172:0/696972539 >> 172.21.15.172:0/696972539 conn(0x7f60500d0780 msgr2=0x7f60500ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:27.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.696+0000 7f6057c80640 1 -- 172.21.15.172:0/696972539 shutdown_connections 2024-04-14T08:34:27.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.696+0000 7f6057c80640 1 -- 172.21.15.172:0/696972539 wait complete. 2024-04-14T08:34:27.708 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-14T08:34:27.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.808+0000 7f441f680640 1 Processor -- start 2024-04-14T08:34:27.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.808+0000 7f441f680640 1 -- start start 2024-04-14T08:34:27.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f441f680640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180b1df0 0x7f44180b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f441f680640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f44180b21d0 0x7f441815c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f441f680640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44180b3430 con 0x7f441815c5a0 2024-04-14T08:34:27.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f441f680640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f44180b3700 con 0x7f44180b21d0 2024-04-14T08:34:27.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f441f680640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f44180b39e0 con 0x7f44180b1df0 2024-04-14T08:34:27.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f441dc1d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f441815c5a0 0x7f44180b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35678/0 (socket says 172.21.15.172:35678) 2024-04-14T08:34:27.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f441dc1d640 1 -- 172.21.15.172:0/1758520233 learned_addr learned my addr 172.21.15.172:0/1758520233 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f441d41c640 1 --2- 172.21.15.172:0/1758520233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180b1df0 0x7f44180b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f441cc1b640 1 --2- 172.21.15.172:0/1758520233 >> v2:172.21.15.177:3300/0 conn(0x7f44180b21d0 0x7f441815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1004983313 0 0) 0x7f44180b3430 con 0x7f441815c5a0 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4404003520 con 0x7f441815c5a0 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 553794450 0 0) 0x7f4404003520 con 0x7f441815c5a0 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180b1df0 msgr2=0x7f44180b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f440ffff640 1 --2- 172.21.15.172:0/1758520233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180b1df0 0x7f44180b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 >> v2:172.21.15.177:3300/0 conn(0x7f44180b21d0 msgr2=0x7f441815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f440ffff640 1 --2- 172.21.15.172:0/1758520233 >> v2:172.21.15.177:3300/0 conn(0x7f44180b21d0 0x7f441815c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44180b3cc0 con 0x7f441815c5a0 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f44100019f0 con 0x7f441815c5a0 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.809+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 >> v1:172.21.15.172:6789/0 conn(0x7f441815c5a0 legacy=0x7f44180b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f440ffff640 1 --2- 172.21.15.172:0/1758520233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44040039b0 0x7f4404003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f440ffff640 1 --2- 172.21.15.172:0/1758520233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180b1df0 0x7f4404004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f440ffff640 1 --2- 172.21.15.172:0/1758520233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4404004d40 0x7f44040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f44180b3700 con 0x7f4404004d40 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f44180b39e0 con 0x7f44040039b0 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f440400b9c0 con 0x7f44180b1df0 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f441dc1d640 1 --2- 172.21.15.172:0/1758520233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4404004d40 0x7f44040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f441d41c640 1 --2- 172.21.15.172:0/1758520233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44040039b0 0x7f4404003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f441cc1b640 1 --2- 172.21.15.172:0/1758520233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180b1df0 0x7f4404004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f441dc1d640 1 -- 172.21.15.172:0/1758520233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180b1df0 msgr2=0x7f4404004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f441dc1d640 1 --2- 172.21.15.172:0/1758520233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180b1df0 0x7f4404004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f441dc1d640 1 -- 172.21.15.172:0/1758520233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44040039b0 msgr2=0x7f4404003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f441dc1d640 1 --2- 172.21.15.172:0/1758520233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44040039b0 0x7f4404003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.810+0000 7f441dc1d640 1 -- 172.21.15.172:0/1758520233 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4404003520 con 0x7f4404004d40 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.811+0000 7f441dc1d640 1 --2- 172.21.15.172:0/1758520233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4404004d40 0x7f44040051c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f441000a460 tx=0x7f4404005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e56e1cb3aa9ceb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:27.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.811+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441003a070 con 0x7f4404004d40 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.811+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4410035660 con 0x7f4404004d40 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.811+0000 7f440ffff640 1 -- 172.21.15.172:0/1758520233 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441000a4a0 con 0x7f4404004d40 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.811+0000 7f441f680640 1 -- 172.21.15.172:0/1758520233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4404004d40 msgr2=0x7f44040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.811+0000 7f441f680640 1 --2- 172.21.15.172:0/1758520233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4404004d40 0x7f44040051c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f441000a460 tx=0x7f4404005830 comp rx=0 tx=0).stop 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.811+0000 7f441dc1d640 1 -- 172.21.15.172:0/1758520233 reap_dead start 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.811+0000 7f441f680640 1 -- 172.21.15.172:0/1758520233 shutdown_connections 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.811+0000 7f441f680640 1 -- 172.21.15.172:0/1758520233 >> 172.21.15.172:0/1758520233 conn(0x7f441806ec30 msgr2=0x7f441806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.811+0000 7f441f680640 1 -- 172.21.15.172:0/1758520233 shutdown_connections 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.811+0000 7f441f680640 1 -- 172.21.15.172:0/1758520233 wait complete. 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.813+0000 7f441f680640 1 Processor -- start 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.813+0000 7f441f680640 1 -- start start 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.813+0000 7f441f680640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4404004d40 0x7f441813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.813+0000 7f441f680640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44180ad970 0x7f44180b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.813+0000 7f441f680640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180addf0 0x7f44180b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.813+0000 7f441f680640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f44180b4750 con 0x7f44180ad970 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.813+0000 7f441f680640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f441815b7f0 con 0x7f4404004d40 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.813+0000 7f441f680640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f44180b48d0 con 0x7f44180addf0 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.813+0000 7f441cc1b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44180ad970 0x7f44180b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.813+0000 7f441cc1b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44180ad970 0x7f44180b0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48958/0 (socket says 172.21.15.172:48958) 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.813+0000 7f441cc1b640 1 -- 172.21.15.172:0/4146342874 learned_addr learned my addr 172.21.15.172:0/4146342874 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.813+0000 7f441d41c640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4404004d40 0x7f441813d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f441dc1d640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180addf0 0x7f44180b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f441cc1b640 1 -- 172.21.15.172:0/4146342874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180addf0 msgr2=0x7f44180b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f441cc1b640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180addf0 0x7f44180b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f441cc1b640 1 -- 172.21.15.172:0/4146342874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4404004d40 msgr2=0x7f441813d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f441cc1b640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4404004d40 0x7f441813d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:27.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f441cc1b640 1 -- 172.21.15.172:0/4146342874 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441813de30 con 0x7f44180ad970 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f441d41c640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4404004d40 0x7f441813d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f441cc1b640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44180ad970 0x7f44180b0d30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4408011a70 tx=0x7f4408011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f440dffb640 1 -- 172.21.15.172:0/4146342874 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440800fb50 con 0x7f44180ad970 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f440dffb640 1 -- 172.21.15.172:0/4146342874 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440800fcf0 con 0x7f44180ad970 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f441f680640 1 -- 172.21.15.172:0/4146342874 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44180aa6f0 con 0x7f44180ad970 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f440dffb640 1 -- 172.21.15.172:0/4146342874 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440800fea0 con 0x7f44180ad970 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.814+0000 7f441f680640 1 -- 172.21.15.172:0/4146342874 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44180a9930 con 0x7f44180ad970 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.815+0000 7f441f680640 1 -- 172.21.15.172:0/4146342874 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4418068520 con 0x7f44180ad970 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.816+0000 7f440dffb640 1 -- 172.21.15.172:0/4146342874 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4408012040 con 0x7f44180ad970 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.816+0000 7f440dffb640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f43fc06c8f0 0x7f43fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.816+0000 7f440dffb640 1 -- 172.21.15.172:0/4146342874 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5335+0+0 (secure 0 0 0) 0x7f440808c070 con 0x7f44180ad970 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.816+0000 7f441d41c640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f43fc06c8f0 0x7f43fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:27.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.816+0000 7f441d41c640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f43fc06c8f0 0x7f43fc06edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4414009bc0 tx=0x7f4414008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:27.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:27.820+0000 7f440dffb640 1 -- 172.21.15.172:0/4146342874 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f441813de30 con 0x7f44180ad970 2024-04-14T08:34:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.070+0000 7f441f680640 1 -- 172.21.15.172:0/4146342874 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f441815e3c0 con 0x7f44180ad970 2024-04-14T08:34:28.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.380+0000 7f440dffb640 1 -- 172.21.15.172:0/4146342874 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v108) ==== 142+0+0 (secure 0 0 0) 0x7f4408019030 con 0x7f44180ad970 2024-04-14T08:34:28.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.383+0000 7f441f680640 1 -- 172.21.15.172:0/4146342874 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f43fc06c8f0 msgr2=0x7f43fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.384+0000 7f441f680640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f43fc06c8f0 0x7f43fc06edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4414009bc0 tx=0x7f4414008040 comp rx=0 tx=0).stop 2024-04-14T08:34:28.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.384+0000 7f441f680640 1 -- 172.21.15.172:0/4146342874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44180ad970 msgr2=0x7f44180b0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.384+0000 7f441f680640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44180ad970 0x7f44180b0d30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4408011a70 tx=0x7f4408011f30 comp rx=0 tx=0).stop 2024-04-14T08:34:28.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.384+0000 7f441f680640 1 -- 172.21.15.172:0/4146342874 shutdown_connections 2024-04-14T08:34:28.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.384+0000 7f441f680640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f43fc06c8f0 0x7f43fc06edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.384+0000 7f441f680640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f44180addf0 0x7f44180b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.384+0000 7f441f680640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44180ad970 0x7f44180b0d30 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.384+0000 7f441f680640 1 --2- 172.21.15.172:0/4146342874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4404004d40 0x7f441813d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.384+0000 7f441f680640 1 -- 172.21.15.172:0/4146342874 >> 172.21.15.172:0/4146342874 conn(0x7f441806ec30 msgr2=0x7f441815c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:28.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.384+0000 7f441f680640 1 -- 172.21.15.172:0/4146342874 shutdown_connections 2024-04-14T08:34:28.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.384+0000 7f441f680640 1 -- 172.21.15.172:0/4146342874 wait complete. 2024-04-14T08:34:28.396 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-14T08:34:28.397 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-14T08:34:28.397 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-14T08:34:28.397 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:34:28.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.497+0000 7f09043a3640 1 Processor -- start 2024-04-14T08:34:28.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.497+0000 7f09043a3640 1 -- start start 2024-04-14T08:34:28.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.498+0000 7f09043a3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f08fc15e5b0 0x7f08fc0ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.498+0000 7f09043a3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08fc0b41d0 con 0x7f08fc06c530 2024-04-14T08:34:28.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.498+0000 7f09043a3640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08fc0aebe0 con 0x7f08fc0729b0 2024-04-14T08:34:28.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.498+0000 7f09043a3640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f08fc0b4470 con 0x7f08fc15e5b0 2024-04-14T08:34:28.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.498+0000 7f090213f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f08fc15e5b0 0x7f08fc0ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:28.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.498+0000 7f0902940640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f08fc06c530 0x7f08fc0b4c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35692/0 (socket says 172.21.15.172:35692) 2024-04-14T08:34:28.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.498+0000 7f0902940640 1 -- 172.21.15.172:0/2151602428 learned_addr learned my addr 172.21.15.172:0/2151602428 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:28.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.498+0000 7f090213f640 1 -- 172.21.15.172:0/2151602428 >> v1:172.21.15.177:6789/0 conn(0x7f08fc0729b0 legacy=0x7f08fc06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:28.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.499+0000 7f090213f640 1 -- 172.21.15.172:0/2151602428 >> v1:172.21.15.172:6789/0 conn(0x7f08fc06c530 legacy=0x7f08fc0b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.499+0000 7f090213f640 1 -- 172.21.15.172:0/2151602428 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc0ae8d0 con 0x7f08fc15e5b0 2024-04-14T08:34:28.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.499+0000 7f090113d640 1 -- 172.21.15.172:0/2151602428 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1194054163 0 0) 0x7f08fc0b41d0 con 0x7f08fc06c530 2024-04-14T08:34:28.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.499+0000 7f090213f640 1 --2- 172.21.15.172:0/2151602428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc15e5b0 0x7f08fc0ae390 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009dc0 tx=0x7f08f802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3bc9bb2b86c688 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:28.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.499+0000 7f090113d640 1 -- 172.21.15.172:0/2151602428 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8039070 con 0x7f08fc15e5b0 2024-04-14T08:34:28.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.499+0000 7f090113d640 1 -- 172.21.15.172:0/2151602428 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08f802cc40 con 0x7f08fc15e5b0 2024-04-14T08:34:28.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.499+0000 7f090113d640 1 -- 172.21.15.172:0/2151602428 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8034440 con 0x7f08fc15e5b0 2024-04-14T08:34:28.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.500+0000 7f09043a3640 1 -- 172.21.15.172:0/2151602428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc15e5b0 msgr2=0x7f08fc0ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.500+0000 7f09043a3640 1 --2- 172.21.15.172:0/2151602428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc15e5b0 0x7f08fc0ae390 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009dc0 tx=0x7f08f802be90 comp rx=0 tx=0).stop 2024-04-14T08:34:28.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.500+0000 7f09043a3640 1 -- 172.21.15.172:0/2151602428 shutdown_connections 2024-04-14T08:34:28.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.500+0000 7f09043a3640 1 --2- 172.21.15.172:0/2151602428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc15e5b0 0x7f08fc0ae390 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.500+0000 7f09043a3640 1 -- 172.21.15.172:0/2151602428 >> 172.21.15.172:0/2151602428 conn(0x7f08fc071ba0 msgr2=0x7f08fc071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:28.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.500+0000 7f09043a3640 1 -- 172.21.15.172:0/2151602428 shutdown_connections 2024-04-14T08:34:28.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.500+0000 7f09043a3640 1 -- 172.21.15.172:0/2151602428 wait complete. 2024-04-14T08:34:28.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.501+0000 7f09043a3640 1 Processor -- start 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.501+0000 7f09043a3640 1 -- start start 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f09043a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc06c530 0x7f08fc0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f09043a3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08fc0729b0 0x7f08fc1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f09043a3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08fc15e5b0 0x7f08fc162000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f09043a3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc0af040 con 0x7f08fc06c530 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f09043a3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc0aeec0 con 0x7f08fc15e5b0 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f09043a3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f08fc0b5380 con 0x7f08fc0729b0 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f090213f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc06c530 0x7f08fc0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f090213f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc06c530 0x7f08fc0ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48966/0 (socket says 172.21.15.172:48966) 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f090213f640 1 -- 172.21.15.172:0/2767038182 learned_addr learned my addr 172.21.15.172:0/2767038182 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f090213f640 1 -- 172.21.15.172:0/2767038182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08fc0729b0 msgr2=0x7f08fc1538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f090213f640 1 --2- 172.21.15.172:0/2767038182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08fc0729b0 0x7f08fc1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f090213f640 1 -- 172.21.15.172:0/2767038182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08fc15e5b0 msgr2=0x7f08fc162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f090213f640 1 --2- 172.21.15.172:0/2767038182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08fc15e5b0 0x7f08fc162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f090213f640 1 -- 172.21.15.172:0/2767038182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc0cd580 con 0x7f08fc06c530 2024-04-14T08:34:28.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.502+0000 7f090213f640 1 --2- 172.21.15.172:0/2767038182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc06c530 0x7f08fc0ccfa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f08f8037040 tx=0x7f08f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:28.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.503+0000 7f08f2ffd640 1 -- 172.21.15.172:0/2767038182 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8044020 con 0x7f08fc06c530 2024-04-14T08:34:28.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.503+0000 7f08f2ffd640 1 -- 172.21.15.172:0/2767038182 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08f803f470 con 0x7f08fc06c530 2024-04-14T08:34:28.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.503+0000 7f09043a3640 1 -- 172.21.15.172:0/2767038182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc0cd860 con 0x7f08fc06c530 2024-04-14T08:34:28.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.503+0000 7f08f2ffd640 1 -- 172.21.15.172:0/2767038182 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8039040 con 0x7f08fc06c530 2024-04-14T08:34:28.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.503+0000 7f09043a3640 1 -- 172.21.15.172:0/2767038182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc15f540 con 0x7f08fc06c530 2024-04-14T08:34:28.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.504+0000 7f09043a3640 1 -- 172.21.15.172:0/2767038182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08fc068520 con 0x7f08fc06c530 2024-04-14T08:34:28.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.504+0000 7f08f2ffd640 1 -- 172.21.15.172:0/2767038182 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08f8047020 con 0x7f08fc06c530 2024-04-14T08:34:28.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.504+0000 7f08f2ffd640 1 --2- 172.21.15.172:0/2767038182 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f08e006c8f0 0x7f08e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.505+0000 7f08f2ffd640 1 -- 172.21.15.172:0/2767038182 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5335+0+0 (secure 0 0 0) 0x7f08f80ae370 con 0x7f08fc06c530 2024-04-14T08:34:28.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.505+0000 7f090193e640 1 --2- 172.21.15.172:0/2767038182 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f08e006c8f0 0x7f08e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:28.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.505+0000 7f090193e640 1 --2- 172.21.15.172:0/2767038182 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f08e006c8f0 0x7f08e006edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f08fc15f280 tx=0x7f08ec0079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:28.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.509+0000 7f08f2ffd640 1 -- 172.21.15.172:0/2767038182 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f08f804e030 con 0x7f08fc06c530 2024-04-14T08:34:28.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.684+0000 7f09043a3640 1 -- 172.21.15.172:0/2767038182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08fc0cdd80 con 0x7f08fc06c530 2024-04-14T08:34:28.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.685+0000 7f08f2ffd640 1 -- 172.21.15.172:0/2767038182 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+13355 (secure 0 0 0) 0x7f08f807ca60 con 0x7f08fc06c530 2024-04-14T08:34:28.686 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:28.686 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":42,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:27.376920+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6836/3420108191":"2024-04-15T08:34:24.369269+0000","172.21.15.172:6835/3420108191":"2024-04-15T08:34:24.369269+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:34:28.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.687+0000 7f09043a3640 1 -- 172.21.15.172:0/2767038182 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f08e006c8f0 msgr2=0x7f08e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.687+0000 7f09043a3640 1 --2- 172.21.15.172:0/2767038182 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f08e006c8f0 0x7f08e006edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f08fc15f280 tx=0x7f08ec0079b0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.687+0000 7f09043a3640 1 -- 172.21.15.172:0/2767038182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc06c530 msgr2=0x7f08fc0ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.687+0000 7f09043a3640 1 --2- 172.21.15.172:0/2767038182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc06c530 0x7f08fc0ccfa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f08f8037040 tx=0x7f08f8004a40 comp rx=0 tx=0).stop 2024-04-14T08:34:28.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.687+0000 7f09043a3640 1 -- 172.21.15.172:0/2767038182 shutdown_connections 2024-04-14T08:34:28.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.687+0000 7f09043a3640 1 --2- 172.21.15.172:0/2767038182 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f08e006c8f0 0x7f08e006edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.687+0000 7f09043a3640 1 --2- 172.21.15.172:0/2767038182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f08fc15e5b0 0x7f08fc162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.687+0000 7f09043a3640 1 --2- 172.21.15.172:0/2767038182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f08fc0729b0 0x7f08fc1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.687+0000 7f09043a3640 1 --2- 172.21.15.172:0/2767038182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc06c530 0x7f08fc0ccfa0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.687+0000 7f09043a3640 1 -- 172.21.15.172:0/2767038182 >> 172.21.15.172:0/2767038182 conn(0x7f08fc071ba0 msgr2=0x7f08fc0b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:28.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.687+0000 7f09043a3640 1 -- 172.21.15.172:0/2767038182 shutdown_connections 2024-04-14T08:34:28.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.687+0000 7f09043a3640 1 -- 172.21.15.172:0/2767038182 wait complete. 2024-04-14T08:34:28.699 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:34:28.699 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:34:28.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.801+0000 7fa6e0fc8640 1 Processor -- start 2024-04-14T08:34:28.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.801+0000 7fa6e0fc8640 1 -- start start 2024-04-14T08:34:28.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.802+0000 7fa6e0fc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6dc15c380 0x7fa6dc140e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.802+0000 7fa6e0fc8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6dc15b620 con 0x7fa6dc0b23e0 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.802+0000 7fa6e0fc8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6dc1414a0 con 0x7fa6dc13df30 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.802+0000 7fa6e0fc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc0cf9b0 con 0x7fa6dc15c380 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.802+0000 7fa6d9d9b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa6dc0b23e0 0x7fa6dc140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35700/0 (socket says 172.21.15.172:35700) 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.802+0000 7fa6d9d9b640 1 -- 172.21.15.172:0/4152295930 learned_addr learned my addr 172.21.15.172:0/4152295930 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.802+0000 7fa6dad9d640 1 --2- 172.21.15.172:0/4152295930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6dc15c380 0x7fa6dc140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2425920019 0 0) 0x7fa6d0001110 con 0x7fa6dc0b23e0 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6c4003520 con 0x7fa6dc0b23e0 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3984898547 0 0) 0x7fa6c4003520 con 0x7fa6dc0b23e0 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6dc15c380 msgr2=0x7fa6dc140e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 --2- 172.21.15.172:0/4152295930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6dc15c380 0x7fa6dc140e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 >> v1:172.21.15.177:6789/0 conn(0x7fa6dc13df30 legacy=0x7fa6dc0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc15b8a0 con 0x7fa6dc0b23e0 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fa6d0001840 con 0x7fa6dc0b23e0 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 >> v1:172.21.15.172:6789/0 conn(0x7fa6dc0b23e0 legacy=0x7fa6dc140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 --2- 172.21.15.172:0/4152295930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c40039e0 0x7fa6c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 --2- 172.21.15.172:0/4152295930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6dc15c380 0x7fa6c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 --2- 172.21.15.172:0/4152295930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6c4004d40 0x7fa6c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0cf9b0 con 0x7fa6c4004d40 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c400b9f0 con 0x7fa6c40039e0 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c400de00 con 0x7fa6dc15c380 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6dad9d640 1 --2- 172.21.15.172:0/4152295930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6c4004d40 0x7fa6c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6d9d9b640 1 --2- 172.21.15.172:0/4152295930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6dc15c380 0x7fa6c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6da59c640 1 --2- 172.21.15.172:0/4152295930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c40039e0 0x7fa6c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6dad9d640 1 -- 172.21.15.172:0/4152295930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6dc15c380 msgr2=0x7fa6c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6dad9d640 1 --2- 172.21.15.172:0/4152295930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6dc15c380 0x7fa6c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6dad9d640 1 -- 172.21.15.172:0/4152295930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c40039e0 msgr2=0x7fa6c4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:28.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6dad9d640 1 --2- 172.21.15.172:0/4152295930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c40039e0 0x7fa6c4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.803+0000 7fa6dad9d640 1 -- 172.21.15.172:0/4152295930 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c4003520 con 0x7fa6c4004d40 2024-04-14T08:34:28.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.804+0000 7fa6dad9d640 1 --2- 172.21.15.172:0/4152295930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6c4004d40 0x7fa6c40051c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009a90 tx=0x7fa6c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f795acc3a8b00cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:28.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.804+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4039070 con 0x7fa6c4004d40 2024-04-14T08:34:28.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.804+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d402c680 con 0x7fa6c4004d40 2024-04-14T08:34:28.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.804+0000 7fa6d959a640 1 -- 172.21.15.172:0/4152295930 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4009690 con 0x7fa6c4004d40 2024-04-14T08:34:28.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.805+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/4152295930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6c4004d40 msgr2=0x7fa6c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.805+0000 7fa6e0fc8640 1 --2- 172.21.15.172:0/4152295930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6c4004d40 0x7fa6c40051c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009a90 tx=0x7fa6c4005830 comp rx=0 tx=0).stop 2024-04-14T08:34:28.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.805+0000 7fa6dad9d640 1 -- 172.21.15.172:0/4152295930 reap_dead start 2024-04-14T08:34:28.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.805+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/4152295930 shutdown_connections 2024-04-14T08:34:28.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.805+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/4152295930 >> 172.21.15.172:0/4152295930 conn(0x7fa6dc06b520 msgr2=0x7fa6dc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:28.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.805+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/4152295930 shutdown_connections 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.805+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/4152295930 wait complete. 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.806+0000 7fa6e0fc8640 1 Processor -- start 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.806+0000 7fa6e0fc8640 1 -- start start 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6e0fc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c4004d40 0x7fa6dc0b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6e0fc8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6dc0adf20 0x7fa6dc139810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6e0fc8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6dc159f70 0x7fa6dc0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6e0fc8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc13e310 con 0x7fa6dc0adf20 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6e0fc8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc15be60 con 0x7fa6dc159f70 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6e0fc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc140e80 con 0x7fa6c4004d40 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6d9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6dc0adf20 0x7fa6dc139810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6d9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6dc0adf20 0x7fa6dc139810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48990/0 (socket says 172.21.15.172:48990) 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6d9d9b640 1 -- 172.21.15.172:0/3752809024 learned_addr learned my addr 172.21.15.172:0/3752809024 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6d9d9b640 1 -- 172.21.15.172:0/3752809024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c4004d40 msgr2=0x7fa6dc0b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6d9d9b640 1 --2- 172.21.15.172:0/3752809024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c4004d40 0x7fa6dc0b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6d9d9b640 1 -- 172.21.15.172:0/3752809024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6dc159f70 msgr2=0x7fa6dc0b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6d9d9b640 1 --2- 172.21.15.172:0/3752809024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6dc159f70 0x7fa6dc0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6d9d9b640 1 -- 172.21.15.172:0/3752809024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc0b4010 con 0x7fa6dc0adf20 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6d9d9b640 1 --2- 172.21.15.172:0/3752809024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6dc0adf20 0x7fa6dc139810 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0007310 tx=0x7fa6d00077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.807+0000 7fa6caffd640 1 -- 172.21.15.172:0/3752809024 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d000f8e0 con 0x7fa6dc0adf20 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.808+0000 7fa6caffd640 1 -- 172.21.15.172:0/3752809024 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d0018070 con 0x7fa6dc0adf20 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.808+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/3752809024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc15a870 con 0x7fa6dc0adf20 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.808+0000 7fa6caffd640 1 -- 172.21.15.172:0/3752809024 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d000fd00 con 0x7fa6dc0adf20 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.808+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/3752809024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc15ae30 con 0x7fa6dc0adf20 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.808+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/3752809024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6dc068590 con 0x7fa6dc0adf20 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.809+0000 7fa6caffd640 1 -- 172.21.15.172:0/3752809024 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6d001d460 con 0x7fa6dc0adf20 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.809+0000 7fa6caffd640 1 --2- 172.21.15.172:0/3752809024 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6bc06c8f0 0x7fa6bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.809+0000 7fa6caffd640 1 -- 172.21.15.172:0/3752809024 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5335+0+0 (secure 0 0 0) 0x7fa6d008ba40 con 0x7fa6dc0adf20 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.810+0000 7fa6da59c640 1 --2- 172.21.15.172:0/3752809024 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6bc06c8f0 0x7fa6bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:28.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.810+0000 7fa6da59c640 1 --2- 172.21.15.172:0/3752809024 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6bc06c8f0 0x7fa6bc06edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc002000 tx=0x7fa6cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:28.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.814+0000 7fa6caffd640 1 -- 172.21.15.172:0/3752809024 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa6dc15b620 con 0x7fa6dc0adf20 2024-04-14T08:34:28.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.990+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/3752809024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa6dc0b2800 con 0x7fa6dc0adf20 2024-04-14T08:34:28.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.990+0000 7fa6caffd640 1 -- 172.21.15.172:0/3752809024 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7fa6d005ee20 con 0x7fa6dc0adf20 2024-04-14T08:34:28.992 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:34:28.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.992+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/3752809024 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6bc06c8f0 msgr2=0x7fa6bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.992+0000 7fa6e0fc8640 1 --2- 172.21.15.172:0/3752809024 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6bc06c8f0 0x7fa6bc06edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc002000 tx=0x7fa6cc009040 comp rx=0 tx=0).stop 2024-04-14T08:34:28.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.992+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/3752809024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6dc0adf20 msgr2=0x7fa6dc139810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:28.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.992+0000 7fa6e0fc8640 1 --2- 172.21.15.172:0/3752809024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6dc0adf20 0x7fa6dc139810 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0007310 tx=0x7fa6d00077d0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.992+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/3752809024 shutdown_connections 2024-04-14T08:34:28.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.992+0000 7fa6e0fc8640 1 --2- 172.21.15.172:0/3752809024 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6bc06c8f0 0x7fa6bc06edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.992+0000 7fa6e0fc8640 1 --2- 172.21.15.172:0/3752809024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6dc159f70 0x7fa6dc0b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.992+0000 7fa6e0fc8640 1 --2- 172.21.15.172:0/3752809024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6dc0adf20 0x7fa6dc139810 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.992+0000 7fa6e0fc8640 1 --2- 172.21.15.172:0/3752809024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c4004d40 0x7fa6dc0b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:28.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.992+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/3752809024 >> 172.21.15.172:0/3752809024 conn(0x7fa6dc06b520 msgr2=0x7fa6dc071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:28.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.992+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/3752809024 shutdown_connections 2024-04-14T08:34:28.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:28.992+0000 7fa6e0fc8640 1 -- 172.21.15.172:0/3752809024 wait complete. 2024-04-14T08:34:29.005 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-14T08:34:29.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.104+0000 7f05611aa640 1 Processor -- start 2024-04-14T08:34:29.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.104+0000 7f05611aa640 1 -- start start 2024-04-14T08:34:29.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f05611aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c06b520 0x7f055c06fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f05611aa640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f055c0b4540 con 0x7f055c0b3dd0 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f05611aa640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f055c072450 con 0x7f055c15e890 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f05611aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f055c072d80 con 0x7f055c06b520 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055259c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f055c0b3dd0 0x7f055c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35710/0 (socket says 172.21.15.172:35710) 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055259c640 1 -- 172.21.15.172:0/3135437749 learned_addr learned my addr 172.21.15.172:0/3135437749 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 210972247 0 0) 0x7f055c0b4540 con 0x7f055c0b3dd0 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0548003520 con 0x7f055c0b3dd0 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055ad9d640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c06b520 0x7f055c06fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2712232309 0 0) 0x7f0548003520 con 0x7f055c0b3dd0 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c06b520 msgr2=0x7f055c06fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c06b520 0x7f055c06fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 >> v1:172.21.15.177:6789/0 conn(0x7f055c15e890 legacy=0x7f055c15ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c072730 con 0x7f055c0b3dd0 2024-04-14T08:34:29.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f0544001670 con 0x7f055c0b3dd0 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 >> v1:172.21.15.172:6789/0 conn(0x7f055c0b3dd0 legacy=0x7f055c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05480039e0 0x7f0548003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c06b520 0x7f0548004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0548004d40 0x7f05480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f055c072d80 con 0x7f05480039e0 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f054800ba70 con 0x7f0548004d40 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f054800bc70 con 0x7f055c06b520 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.105+0000 7f055ad9d640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05480039e0 0x7f0548003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.106+0000 7f055259c640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c06b520 0x7f0548004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.106+0000 7f055b59e640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0548004d40 0x7f05480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.106+0000 7f055ad9d640 1 -- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c06b520 msgr2=0x7f0548004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.106+0000 7f055ad9d640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c06b520 0x7f0548004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.106+0000 7f055ad9d640 1 -- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0548004d40 msgr2=0x7f05480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:29.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.106+0000 7f055ad9d640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0548004d40 0x7f05480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.106+0000 7f055ad9d640 1 -- 172.21.15.172:0/3135437749 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0548003520 con 0x7f05480039e0 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.106+0000 7f055259c640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f055c06b520 0x7f0548004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.106+0000 7f055b59e640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0548004d40 0x7f05480051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.106+0000 7f055ad9d640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05480039e0 0x7f0548003e00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f054c009a90 tx=0x7f0548005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef93ae6dd309361 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.107+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c039070 con 0x7f05480039e0 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.107+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f054c02c680 con 0x7f05480039e0 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.107+0000 7f055a59c640 1 -- 172.21.15.172:0/3135437749 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c009690 con 0x7f05480039e0 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.107+0000 7f05611aa640 1 -- 172.21.15.172:0/3135437749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05480039e0 msgr2=0x7f0548003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.107+0000 7f05611aa640 1 --2- 172.21.15.172:0/3135437749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05480039e0 0x7f0548003e00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f054c009a90 tx=0x7f0548005830 comp rx=0 tx=0).stop 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.107+0000 7f055b59e640 1 -- 172.21.15.172:0/3135437749 reap_dead start 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.107+0000 7f05611aa640 1 -- 172.21.15.172:0/3135437749 shutdown_connections 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.107+0000 7f05611aa640 1 -- 172.21.15.172:0/3135437749 >> 172.21.15.172:0/3135437749 conn(0x7f055c0d0f20 msgr2=0x7f055c0d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.107+0000 7f05611aa640 1 -- 172.21.15.172:0/3135437749 shutdown_connections 2024-04-14T08:34:29.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.107+0000 7f05611aa640 1 -- 172.21.15.172:0/3135437749 wait complete. 2024-04-14T08:34:29.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.108+0000 7f05611aa640 1 Processor -- start 2024-04-14T08:34:29.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.109+0000 7f05611aa640 1 -- start start 2024-04-14T08:34:29.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.109+0000 7f05611aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05480039e0 0x7f055c142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.109+0000 7f05611aa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c155d30 0x7f055c143280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.109+0000 7f05611aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f055c1437c0 0x7f055c1e9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.109+0000 7f05611aa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f055c073210 con 0x7f055c1437c0 2024-04-14T08:34:29.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.109+0000 7f05611aa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f055c0b4250 con 0x7f055c155d30 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.109+0000 7f05611aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f055c158b90 con 0x7f05480039e0 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.109+0000 7f055b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f055c1437c0 0x7f055c1e9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.109+0000 7f055b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f055c1437c0 0x7f055c1e9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49002/0 (socket says 172.21.15.172:49002) 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f055b59e640 1 -- 172.21.15.172:0/523891901 learned_addr learned my addr 172.21.15.172:0/523891901 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f055ad9d640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05480039e0 0x7f055c142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f055259c640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c155d30 0x7f055c143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f055b59e640 1 -- 172.21.15.172:0/523891901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05480039e0 msgr2=0x7f055c142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f055b59e640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05480039e0 0x7f055c142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f055b59e640 1 -- 172.21.15.172:0/523891901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c155d30 msgr2=0x7f055c143280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f055b59e640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c155d30 0x7f055c143280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f055b59e640 1 -- 172.21.15.172:0/523891901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c142e90 con 0x7f055c1437c0 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f055b59e640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f055c1437c0 0x7f055c1e9140 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0554000f80 tx=0x7f055400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f0553fff640 1 -- 172.21.15.172:0/523891901 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554006d50 con 0x7f055c1437c0 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f0553fff640 1 -- 172.21.15.172:0/523891901 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0554003970 con 0x7f055c1437c0 2024-04-14T08:34:29.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f05611aa640 1 -- 172.21.15.172:0/523891901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c1e80a0 con 0x7f055c1437c0 2024-04-14T08:34:29.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f0553fff640 1 -- 172.21.15.172:0/523891901 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554003d60 con 0x7f055c1437c0 2024-04-14T08:34:29.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.110+0000 7f05611aa640 1 -- 172.21.15.172:0/523891901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c1e8660 con 0x7f055c1437c0 2024-04-14T08:34:29.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.111+0000 7f05611aa640 1 -- 172.21.15.172:0/523891901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055c068520 con 0x7f055c1437c0 2024-04-14T08:34:29.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.112+0000 7f0553fff640 1 -- 172.21.15.172:0/523891901 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f055401b020 con 0x7f055c1437c0 2024-04-14T08:34:29.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.112+0000 7f0553fff640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f053c06c8f0 0x7f053c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.112+0000 7f055ad9d640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f053c06c8f0 0x7f053c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.112+0000 7f0553fff640 1 -- 172.21.15.172:0/523891901 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5335+0+0 (secure 0 0 0) 0x7f055408b520 con 0x7f055c1437c0 2024-04-14T08:34:29.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.113+0000 7f055ad9d640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f053c06c8f0 0x7f053c06edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f054c036000 tx=0x7f054c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:29.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.117+0000 7f0553fff640 1 -- 172.21.15.172:0/523891901 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f055c142e90 con 0x7f055c1437c0 2024-04-14T08:34:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.358+0000 7f05611aa640 1 -- 172.21.15.172:0/523891901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f055c06c3b0 con 0x7f055c1437c0 2024-04-14T08:34:29.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.385+0000 7f0553fff640 1 -- 172.21.15.172:0/523891901 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v109) ==== 152+0+0 (secure 0 0 0) 0x7f0554059ac0 con 0x7f055c1437c0 2024-04-14T08:34:29.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.387+0000 7f05611aa640 1 -- 172.21.15.172:0/523891901 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f053c06c8f0 msgr2=0x7f053c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.387+0000 7f05611aa640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f053c06c8f0 0x7f053c06edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f054c036000 tx=0x7f054c030000 comp rx=0 tx=0).stop 2024-04-14T08:34:29.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.387+0000 7f05611aa640 1 -- 172.21.15.172:0/523891901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f055c1437c0 msgr2=0x7f055c1e9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.387+0000 7f05611aa640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f055c1437c0 0x7f055c1e9140 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0554000f80 tx=0x7f055400dc00 comp rx=0 tx=0).stop 2024-04-14T08:34:29.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.387+0000 7f05611aa640 1 -- 172.21.15.172:0/523891901 shutdown_connections 2024-04-14T08:34:29.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.387+0000 7f05611aa640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f053c06c8f0 0x7f053c06edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.387+0000 7f05611aa640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f055c1437c0 0x7f055c1e9140 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.387+0000 7f05611aa640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c155d30 0x7f055c143280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.387+0000 7f05611aa640 1 --2- 172.21.15.172:0/523891901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05480039e0 0x7f055c142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.387+0000 7f05611aa640 1 -- 172.21.15.172:0/523891901 >> 172.21.15.172:0/523891901 conn(0x7f055c0d0f20 msgr2=0x7f055c0b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:29.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.387+0000 7f05611aa640 1 -- 172.21.15.172:0/523891901 shutdown_connections 2024-04-14T08:34:29.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.387+0000 7f05611aa640 1 -- 172.21.15.172:0/523891901 wait complete. 2024-04-14T08:34:29.400 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-14T08:34:29.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.502+0000 7f2cd3ab7640 1 Processor -- start 2024-04-14T08:34:29.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.502+0000 7f2cd3ab7640 1 -- start start 2024-04-14T08:34:29.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.503+0000 7f2cd3ab7640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2ccc15c420 0x7f2ccc0b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.503+0000 7f2cd3ab7640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ccc0ccdd0 con 0x7f2ccc0ceb00 2024-04-14T08:34:29.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.503+0000 7f2cd3ab7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ccc0d2ae0 con 0x7f2ccc06f3e0 2024-04-14T08:34:29.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.503+0000 7f2cd3ab7640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2ccc0d30a0 con 0x7f2ccc15c420 2024-04-14T08:34:29.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.503+0000 7f2cd1853640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2ccc15c420 0x7f2ccc0b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.503+0000 7f2cd2054640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2ccc0ceb00 0x7f2ccc0cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:35726/0 (socket says 172.21.15.172:35726) 2024-04-14T08:34:29.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.503+0000 7f2cd2054640 1 -- 172.21.15.172:0/22538506 learned_addr learned my addr 172.21.15.172:0/22538506 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:29.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.504+0000 7f2cd1853640 1 -- 172.21.15.172:0/22538506 >> v1:172.21.15.177:6789/0 conn(0x7f2ccc06f3e0 legacy=0x7f2ccc0d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:29.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.504+0000 7f2cd1853640 1 -- 172.21.15.172:0/22538506 >> v1:172.21.15.172:6789/0 conn(0x7f2ccc0ceb00 legacy=0x7f2ccc0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.504+0000 7f2cd1853640 1 -- 172.21.15.172:0/22538506 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc0d2dc0 con 0x7f2ccc15c420 2024-04-14T08:34:29.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.504+0000 7f2cd0851640 1 -- 172.21.15.172:0/22538506 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 192853405 0 0) 0x7f2ccc0ccdd0 con 0x7f2ccc0ceb00 2024-04-14T08:34:29.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.504+0000 7f2cd1853640 1 --2- 172.21.15.172:0/22538506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ccc15c420 0x7f2ccc0b1510 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009d90 tx=0x7f2cc802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a895275b0e4f774 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:29.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.505+0000 7f2cd0851640 1 -- 172.21.15.172:0/22538506 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8039070 con 0x7f2ccc15c420 2024-04-14T08:34:29.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.505+0000 7f2cd0851640 1 -- 172.21.15.172:0/22538506 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc802cc60 con 0x7f2ccc15c420 2024-04-14T08:34:29.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.505+0000 7f2cd0851640 1 -- 172.21.15.172:0/22538506 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8034660 con 0x7f2ccc15c420 2024-04-14T08:34:29.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.505+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/22538506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ccc15c420 msgr2=0x7f2ccc0b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.505+0000 7f2cd3ab7640 1 --2- 172.21.15.172:0/22538506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ccc15c420 0x7f2ccc0b1510 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009d90 tx=0x7f2cc802beb0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.505+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/22538506 shutdown_connections 2024-04-14T08:34:29.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.505+0000 7f2cd3ab7640 1 --2- 172.21.15.172:0/22538506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ccc15c420 0x7f2ccc0b1510 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.505+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/22538506 >> 172.21.15.172:0/22538506 conn(0x7f2ccc08c000 msgr2=0x7f2ccc0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:29.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.505+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/22538506 shutdown_connections 2024-04-14T08:34:29.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.505+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/22538506 wait complete. 2024-04-14T08:34:29.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.506+0000 7f2cd3ab7640 1 Processor -- start 2024-04-14T08:34:29.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.506+0000 7f2cd3ab7640 1 -- start start 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd3ab7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ccc0ceb00 0x7f2ccc0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd3ab7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ccc15c420 0x7f2ccc14fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd3ab7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ccc150580 0x7f2ccc0b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd3ab7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc0b11e0 con 0x7f2ccc150580 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd3ab7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc06f870 con 0x7f2ccc15c420 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd3ab7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2ccc15c800 con 0x7f2ccc0ceb00 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd2054640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ccc150580 0x7f2ccc0b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd2054640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ccc150580 0x7f2ccc0b6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49016/0 (socket says 172.21.15.172:49016) 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd2054640 1 -- 172.21.15.172:0/569202356 learned_addr learned my addr 172.21.15.172:0/569202356 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd1853640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ccc0ceb00 0x7f2ccc0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd2054640 1 -- 172.21.15.172:0/569202356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ccc0ceb00 msgr2=0x7f2ccc0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd2054640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ccc0ceb00 0x7f2ccc0d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd2054640 1 -- 172.21.15.172:0/569202356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ccc15c420 msgr2=0x7f2ccc14fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.507+0000 7f2cd1052640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ccc15c420 0x7f2ccc14fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.508+0000 7f2cd2054640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ccc15c420 0x7f2ccc14fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.508+0000 7f2cd2054640 1 -- 172.21.15.172:0/569202356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc0cbe00 con 0x7f2ccc150580 2024-04-14T08:34:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.508+0000 7f2cd2054640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ccc150580 0x7f2ccc0b6f00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400c450 tx=0x7f2cc400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.508+0000 7f2cc27fc640 1 -- 172.21.15.172:0/569202356 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40074e0 con 0x7f2ccc150580 2024-04-14T08:34:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.508+0000 7f2cc27fc640 1 -- 172.21.15.172:0/569202356 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc40047e0 con 0x7f2ccc150580 2024-04-14T08:34:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.508+0000 7f2cc27fc640 1 -- 172.21.15.172:0/569202356 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4004bd0 con 0x7f2ccc150580 2024-04-14T08:34:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.508+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/569202356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc0c9310 con 0x7f2ccc150580 2024-04-14T08:34:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.508+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/569202356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc0c98d0 con 0x7f2ccc150580 2024-04-14T08:34:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.509+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/569202356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ccc0cc4d0 con 0x7f2ccc150580 2024-04-14T08:34:29.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.509+0000 7f2cc27fc640 1 -- 172.21.15.172:0/569202356 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cc4011070 con 0x7f2ccc150580 2024-04-14T08:34:29.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.510+0000 7f2cc27fc640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2cb006c920 0x7f2cb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:29.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.510+0000 7f2cc27fc640 1 -- 172.21.15.172:0/569202356 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5335+0+0 (secure 0 0 0) 0x7f2cc408b520 con 0x7f2ccc150580 2024-04-14T08:34:29.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.510+0000 7f2cd1853640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2cb006c920 0x7f2cb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:29.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.510+0000 7f2cd1853640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2cb006c920 0x7f2cb006ede0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8036000 tx=0x7f2cc8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:29.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.514+0000 7f2cc27fc640 1 -- 172.21.15.172:0/569202356 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2ccc0cc4d0 con 0x7f2ccc150580 2024-04-14T08:34:29.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.687+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/569202356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2ccc0c9ad0 con 0x7f2ccc150580 2024-04-14T08:34:29.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.691+0000 7f2cc27fc640 1 -- 172.21.15.172:0/569202356 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f2ccc0c9ad0 con 0x7f2ccc150580 2024-04-14T08:34:29.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.693+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/569202356 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2cb006c920 msgr2=0x7f2cb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.693+0000 7f2cd3ab7640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2cb006c920 0x7f2cb006ede0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8036000 tx=0x7f2cc8030000 comp rx=0 tx=0).stop 2024-04-14T08:34:29.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.693+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/569202356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ccc150580 msgr2=0x7f2ccc0b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:29.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.693+0000 7f2cd3ab7640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ccc150580 0x7f2ccc0b6f00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400c450 tx=0x7f2cc400c910 comp rx=0 tx=0).stop 2024-04-14T08:34:29.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.693+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/569202356 shutdown_connections 2024-04-14T08:34:29.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.693+0000 7f2cd3ab7640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2cb006c920 0x7f2cb006ede0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.693+0000 7f2cd3ab7640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ccc150580 0x7f2ccc0b6f00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.693+0000 7f2cd3ab7640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ccc15c420 0x7f2ccc14fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.693+0000 7f2cd3ab7640 1 --2- 172.21.15.172:0/569202356 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ccc0ceb00 0x7f2ccc0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:29.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.693+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/569202356 >> 172.21.15.172:0/569202356 conn(0x7f2ccc08c000 msgr2=0x7f2ccc159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:29.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.693+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/569202356 shutdown_connections 2024-04-14T08:34:29.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:29.693+0000 7f2cd3ab7640 1 -- 172.21.15.172:0/569202356 wait complete. 2024-04-14T08:34:29.706 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:34:29.706 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:34:29.745 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:34:29.797 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:34:29.798 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:34:29.798 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:34:29.813 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:34:29.813 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:34:29.813 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:34:29.830 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:34:29.831 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:34:29.831 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:34:29.872 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:34:29.872 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:34:29.872 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:34:29.889 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:34:29.889 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-14T08:34:29.889 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:34:30.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.011+0000 7f20a3fb4640 1 Processor -- start 2024-04-14T08:34:30.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.012+0000 7f20a3fb4640 1 -- start start 2024-04-14T08:34:30.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.012+0000 7f20a3fb4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f209c0d2830 0x7f209c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.012+0000 7f20a3fb4640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f209c0cedd0 0x7f209c0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.012+0000 7f20a3fb4640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f209c0d3150 con 0x7f209c15e890 2024-04-14T08:34:30.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.012+0000 7f20a3fb4640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f209c0cff30 con 0x7f209c0cedd0 2024-04-14T08:34:30.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.012+0000 7f20a3fb4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f209c0d2350 con 0x7f209c0d2830 2024-04-14T08:34:30.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.013+0000 7f20a1d50640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f209c0d2830 0x7f209c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.013+0000 7f20a2551640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f209c0cedd0 0x7f209c0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.013+0000 7f20a1d50640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f209c0d2830 0x7f209c0d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:34642/0 (socket says 172.21.15.172:34642) 2024-04-14T08:34:30.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.013+0000 7f20a1d50640 1 -- 172.21.15.172:0/3046277189 learned_addr learned my addr 172.21.15.172:0/3046277189 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:30.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.013+0000 7f20a154f640 1 --1- 172.21.15.172:0/3046277189 >> v1:172.21.15.177:6789/0 conn(0x7f209c15e890 0x7f209c0d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:46434/0 (socket says 172.21.15.172:46434) 2024-04-14T08:34:30.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.013+0000 7f20a2551640 1 -- 172.21.15.172:0/3046277189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f209c0d2830 msgr2=0x7f209c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:30.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.013+0000 7f20a2551640 1 --2- 172.21.15.172:0/3046277189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f209c0d2830 0x7f209c0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.013+0000 7f20a2551640 1 -- 172.21.15.172:0/3046277189 >> v1:172.21.15.177:6789/0 conn(0x7f209c15e890 legacy=0x7f209c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:30.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.013+0000 7f20a0d4e640 1 -- 172.21.15.172:0/3046277189 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1379563716 0 0) 0x7f209c0d3150 con 0x7f209c15e890 2024-04-14T08:34:30.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.013+0000 7f20a2551640 1 -- 172.21.15.172:0/3046277189 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c0cf320 con 0x7f209c0cedd0 2024-04-14T08:34:30.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.013+0000 7f20a1d50640 1 --2- 172.21.15.172:0/3046277189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f209c0d2830 0x7f209c0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:34:30.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.014+0000 7f20a2551640 1 --2- 172.21.15.172:0/3046277189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f209c0cedd0 0x7f209c0b3710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f208c0097a0 tx=0x7f208c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceb3074c5c7fa620 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:30.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.014+0000 7f20a0d4e640 1 -- 172.21.15.172:0/3046277189 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c034070 con 0x7f209c0cedd0 2024-04-14T08:34:30.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.014+0000 7f20a0d4e640 1 -- 172.21.15.172:0/3046277189 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f208c030610 con 0x7f209c0cedd0 2024-04-14T08:34:30.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.014+0000 7f20a0d4e640 1 -- 172.21.15.172:0/3046277189 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c0309e0 con 0x7f209c0cedd0 2024-04-14T08:34:30.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.015+0000 7f20a3fb4640 1 -- 172.21.15.172:0/3046277189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f209c0cedd0 msgr2=0x7f209c0b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:30.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.015+0000 7f20a3fb4640 1 --2- 172.21.15.172:0/3046277189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f209c0cedd0 0x7f209c0b3710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f208c0097a0 tx=0x7f208c02b960 comp rx=0 tx=0).stop 2024-04-14T08:34:30.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.015+0000 7f20a3fb4640 1 -- 172.21.15.172:0/3046277189 shutdown_connections 2024-04-14T08:34:30.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.015+0000 7f20a3fb4640 1 --2- 172.21.15.172:0/3046277189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f209c0cedd0 0x7f209c0b3710 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.015+0000 7f20a3fb4640 1 --2- 172.21.15.172:0/3046277189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f209c0d2830 0x7f209c0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.015+0000 7f20a3fb4640 1 -- 172.21.15.172:0/3046277189 >> 172.21.15.172:0/3046277189 conn(0x7f209c09d590 msgr2=0x7f209c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:30.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.015+0000 7f20a3fb4640 1 -- 172.21.15.172:0/3046277189 shutdown_connections 2024-04-14T08:34:30.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.015+0000 7f20a3fb4640 1 -- 172.21.15.172:0/3046277189 wait complete. 2024-04-14T08:34:30.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.016+0000 7f20a3fb4640 1 Processor -- start 2024-04-14T08:34:30.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.016+0000 7f20a3fb4640 1 -- start start 2024-04-14T08:34:30.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a3fb4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f209c0cedd0 0x7f209c0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a3fb4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f209c0d2830 0x7f209c0cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a3fb4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f209c15e890 0x7f209c0b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a3fb4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f209c0cfbc0 con 0x7f209c15e890 2024-04-14T08:34:30.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a3fb4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f209c0cf8e0 con 0x7f209c0d2830 2024-04-14T08:34:30.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a3fb4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f209c0b3c50 con 0x7f209c0cedd0 2024-04-14T08:34:30.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a2551640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f209c15e890 0x7f209c0b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a2551640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f209c15e890 0x7f209c0b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47936/0 (socket says 172.21.15.172:47936) 2024-04-14T08:34:30.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a2551640 1 -- 172.21.15.172:0/2390365318 learned_addr learned my addr 172.21.15.172:0/2390365318 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:30.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a1d50640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f209c0cedd0 0x7f209c0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a154f640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f209c0d2830 0x7f209c0cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a2551640 1 -- 172.21.15.172:0/2390365318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f209c0cedd0 msgr2=0x7f209c0ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:30.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a2551640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f209c0cedd0 0x7f209c0ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a2551640 1 -- 172.21.15.172:0/2390365318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f209c0d2830 msgr2=0x7f209c0cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:30.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a2551640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f209c0d2830 0x7f209c0cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a2551640 1 -- 172.21.15.172:0/2390365318 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c0c8030 con 0x7f209c15e890 2024-04-14T08:34:30.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f20a2551640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f209c15e890 0x7f209c0b1ed0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f208c02b9a0 tx=0x7f208c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:30.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.017+0000 7f209a7fc640 1 -- 172.21.15.172:0/2390365318 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c034040 con 0x7f209c15e890 2024-04-14T08:34:30.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.018+0000 7f209a7fc640 1 -- 172.21.15.172:0/2390365318 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f208c030410 con 0x7f209c15e890 2024-04-14T08:34:30.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.018+0000 7f209a7fc640 1 -- 172.21.15.172:0/2390365318 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c006e10 con 0x7f209c15e890 2024-04-14T08:34:30.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.018+0000 7f20a3fb4640 1 -- 172.21.15.172:0/2390365318 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209c0c8310 con 0x7f209c15e890 2024-04-14T08:34:30.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.018+0000 7f20a3fb4640 1 -- 172.21.15.172:0/2390365318 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209c0c88d0 con 0x7f209c15e890 2024-04-14T08:34:30.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.019+0000 7f20a3fb4640 1 -- 172.21.15.172:0/2390365318 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f209c068520 con 0x7f209c15e890 2024-04-14T08:34:30.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.019+0000 7f209a7fc640 1 -- 172.21.15.172:0/2390365318 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f208c04c060 con 0x7f209c15e890 2024-04-14T08:34:30.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.019+0000 7f209a7fc640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f208806c8f0 0x7f208806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.020+0000 7f209a7fc640 1 -- 172.21.15.172:0/2390365318 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5335+0+0 (secure 0 0 0) 0x7f208c0ad360 con 0x7f209c15e890 2024-04-14T08:34:30.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.020+0000 7f20a1d50640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f208806c8f0 0x7f208806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.020+0000 7f20a1d50640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f208806c8f0 0x7f208806edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2094009240 tx=0x7f209400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:30.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.024+0000 7f209a7fc640 1 -- 172.21.15.172:0/2390365318 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f208c0305e0 con 0x7f209c15e890 2024-04-14T08:34:30.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.237+0000 7f20a3fb4640 1 -- 172.21.15.172:0/2390365318 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f209c0d2610 con 0x7f209c15e890 2024-04-14T08:34:30.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.237+0000 7f209a7fc640 1 -- 172.21.15.172:0/2390365318 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3878 (secure 0 0 0) 0x7f208c0abe20 con 0x7f209c15e890 2024-04-14T08:34:30.239 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:30.239 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1038094401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1038094401},{"type":"v1","addr":"172.21.15.177:6834","nonce":1038094401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1119512744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1119512744},{"type":"v1","addr":"172.21.15.177:6835","nonce":1119512744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3054138283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3054138283},{"type":"v1","addr":"172.21.15.172:6837","nonce":3054138283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/226451221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":226451221},{"type":"v1","addr":"172.21.15.172:6836","nonce":226451221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-04-14T08:34:30.239 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 18 2024-04-14T08:34:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.239+0000 7f20a3fb4640 1 -- 172.21.15.172:0/2390365318 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f208806c8f0 msgr2=0x7f208806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.239+0000 7f20a3fb4640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f208806c8f0 0x7f208806edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2094009240 tx=0x7f209400b040 comp rx=0 tx=0).stop 2024-04-14T08:34:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.239+0000 7f20a3fb4640 1 -- 172.21.15.172:0/2390365318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f209c15e890 msgr2=0x7f209c0b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.239+0000 7f20a3fb4640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f209c15e890 0x7f209c0b1ed0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f208c02b9a0 tx=0x7f208c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.239+0000 7f20a3fb4640 1 -- 172.21.15.172:0/2390365318 shutdown_connections 2024-04-14T08:34:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.239+0000 7f20a3fb4640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f208806c8f0 0x7f208806edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.239+0000 7f20a3fb4640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f209c15e890 0x7f209c0b1ed0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.239+0000 7f20a3fb4640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f209c0d2830 0x7f209c0cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.239+0000 7f20a3fb4640 1 --2- 172.21.15.172:0/2390365318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f209c0cedd0 0x7f209c0ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.239+0000 7f20a3fb4640 1 -- 172.21.15.172:0/2390365318 >> 172.21.15.172:0/2390365318 conn(0x7f209c09d590 msgr2=0x7f209c15d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.239+0000 7f20a3fb4640 1 -- 172.21.15.172:0/2390365318 shutdown_connections 2024-04-14T08:34:30.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.239+0000 7f20a3fb4640 1 -- 172.21.15.172:0/2390365318 wait complete. 2024-04-14T08:34:30.252 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-14T08:34:30.252 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-14T08:34:30.252 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-14T08:34:30.253 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:34:30.253 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-14T08:34:30.253 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-14T08:34:30.253 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-14T08:34:30.253 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:34:30.253 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-14T08:34:30.253 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-14T08:34:30.254 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-14T08:34:30.254 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:34:30.254 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-14T08:34:30.254 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-14T08:34:30.254 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-14T08:34:30.254 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:34:30.262 INFO:tasks.ceph.mds.a:Stopped 2024-04-14T08:34:30.262 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-14T08:34:30.263 INFO:tasks.ceph.mds.c:Stopped 2024-04-14T08:34:30.264 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-14T08:34:30.266 INFO:tasks.ceph.mds.b:Stopped 2024-04-14T08:34:30.266 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-14T08:34:30.269 INFO:tasks.ceph.mds.d:Stopped 2024-04-14T08:34:30.269 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-14T08:34:30.305 INFO:tasks.ceph.mds.a:Started 2024-04-14T08:34:30.306 DEBUG:teuthology.parallel:result is None 2024-04-14T08:34:30.306 INFO:tasks.ceph.mds.c:Started 2024-04-14T08:34:30.307 DEBUG:teuthology.parallel:result is None 2024-04-14T08:34:30.309 INFO:tasks.ceph.mds.b:Started 2024-04-14T08:34:30.309 DEBUG:teuthology.parallel:result is None 2024-04-14T08:34:30.309 INFO:tasks.ceph.mds.d:Started 2024-04-14T08:34:30.310 DEBUG:teuthology.parallel:result is None 2024-04-14T08:34:30.310 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-14T08:34:30.383 INFO:tasks.ceph.mds.a.smithi172.stdout:starting mds.a at 2024-04-14T08:34:30.383 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:34:30.394 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-04-14T08:34:30.395 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-04-14T08:34:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.416+0000 7f2e05dfe640 1 Processor -- start 2024-04-14T08:34:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.416+0000 7f2e05dfe640 1 -- start start 2024-04-14T08:34:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.416+0000 7f2e05dfe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2e0013df30 0x7f2e000b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2e05dfe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e0015c380 0x7f2e000b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2e05dfe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2e00140770 0x7f2e00140c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2e05dfe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2e0015b620 con 0x7f2e00140770 2024-04-14T08:34:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2e05dfe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2e001414a0 con 0x7f2e0013df30 2024-04-14T08:34:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2e05dfe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0015b8a0 con 0x7f2e0015c380 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2dfffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2e00140770 0x7f2e00140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2dfffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2e00140770 0x7f2e00140c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47974/0 (socket says 172.21.15.172:47974) 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2dfffff640 1 -- 172.21.15.172:0/1900110745 learned_addr learned my addr 172.21.15.172:0/1900110745 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2dfeffd640 1 --2- 172.21.15.172:0/1900110745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e0015c380 0x7f2e000b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2dfffff640 1 -- 172.21.15.172:0/1900110745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e0015c380 msgr2=0x7f2e000b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2dff7fe640 1 --2- 172.21.15.172:0/1900110745 >> v2:172.21.15.177:3300/0 conn(0x7f2e0013df30 0x7f2e000b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2dfffff640 1 --2- 172.21.15.172:0/1900110745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e0015c380 0x7f2e000b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2dfffff640 1 -- 172.21.15.172:0/1900110745 >> v2:172.21.15.177:3300/0 conn(0x7f2e0013df30 msgr2=0x7f2e000b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2dfffff640 1 --2- 172.21.15.172:0/1900110745 >> v2:172.21.15.177:3300/0 conn(0x7f2e0013df30 0x7f2e000b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.417+0000 7f2dfffff640 1 -- 172.21.15.172:0/1900110745 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0015bb80 con 0x7f2e00140770 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.418+0000 7f2dfffff640 1 --2- 172.21.15.172:0/1900110745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e00140770 0x7f2e00140c00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009e50 tx=0x7f2df802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838e6d70e92e8681 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.418+0000 7f2dfe7fc640 1 -- 172.21.15.172:0/1900110745 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df802ca70 con 0x7f2e00140770 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.418+0000 7f2dfe7fc640 1 -- 172.21.15.172:0/1900110745 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df802cc10 con 0x7f2e00140770 2024-04-14T08:34:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.418+0000 7f2dfe7fc640 1 -- 172.21.15.172:0/1900110745 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80343f0 con 0x7f2e00140770 2024-04-14T08:34:30.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.418+0000 7f2e05dfe640 1 -- 172.21.15.172:0/1900110745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e00140770 msgr2=0x7f2e00140c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:30.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.418+0000 7f2e05dfe640 1 --2- 172.21.15.172:0/1900110745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e00140770 0x7f2e00140c00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009e50 tx=0x7f2df802bed0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.418+0000 7f2e05dfe640 1 -- 172.21.15.172:0/1900110745 shutdown_connections 2024-04-14T08:34:30.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.418+0000 7f2e05dfe640 1 --2- 172.21.15.172:0/1900110745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e00140770 0x7f2e00140c00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.418+0000 7f2e05dfe640 1 --2- 172.21.15.172:0/1900110745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e0015c380 0x7f2e000b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.418+0000 7f2e05dfe640 1 --2- 172.21.15.172:0/1900110745 >> v2:172.21.15.177:3300/0 conn(0x7f2e0013df30 0x7f2e000b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.418+0000 7f2e05dfe640 1 -- 172.21.15.172:0/1900110745 >> 172.21.15.172:0/1900110745 conn(0x7f2e0006b520 msgr2=0x7f2e0006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:30.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.418+0000 7f2e05dfe640 1 -- 172.21.15.172:0/1900110745 shutdown_connections 2024-04-14T08:34:30.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.419+0000 7f2e05dfe640 1 -- 172.21.15.172:0/1900110745 wait complete. 2024-04-14T08:34:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.420+0000 7f2e05dfe640 1 Processor -- start 2024-04-14T08:34:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.420+0000 7f2e05dfe640 1 -- start start 2024-04-14T08:34:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.420+0000 7f2e05dfe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e0013df30 0x7f2e00151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.420+0000 7f2e05dfe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e00140770 0x7f2e00154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.420+0000 7f2e05dfe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e0015c380 0x7f2e000cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.420+0000 7f2e05dfe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00141140 con 0x7f2e0015c380 2024-04-14T08:34:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.420+0000 7f2e05dfe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0013e310 con 0x7f2e00140770 2024-04-14T08:34:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.420+0000 7f2e05dfe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2e000723e0 con 0x7f2e0013df30 2024-04-14T08:34:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2dfffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e0015c380 0x7f2e000cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2dfffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e0015c380 0x7f2e000cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47982/0 (socket says 172.21.15.172:47982) 2024-04-14T08:34:30.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2dfffff640 1 -- 172.21.15.172:0/2466036409 learned_addr learned my addr 172.21.15.172:0/2466036409 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2dff7fe640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e0013df30 0x7f2e00151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2dfeffd640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e00140770 0x7f2e00154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2dfffff640 1 -- 172.21.15.172:0/2466036409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e0013df30 msgr2=0x7f2e00151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2dfffff640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e0013df30 0x7f2e00151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2dfffff640 1 -- 172.21.15.172:0/2466036409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e00140770 msgr2=0x7f2e00154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2dfffff640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e00140770 0x7f2e00154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2dfffff640 1 -- 172.21.15.172:0/2466036409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e00152da0 con 0x7f2e0015c380 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2dfffff640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e0015c380 0x7f2e000cf170 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009e50 tx=0x7f2df8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2de7fff640 1 -- 172.21.15.172:0/2466036409 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8043020 con 0x7f2e0015c380 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2de7fff640 1 -- 172.21.15.172:0/2466036409 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2df803e450 con 0x7f2e0015c380 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2de7fff640 1 -- 172.21.15.172:0/2466036409 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8009510 con 0x7f2e0015c380 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2e05dfe640 1 -- 172.21.15.172:0/2466036409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e00153080 con 0x7f2e0015c380 2024-04-14T08:34:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.421+0000 7f2e05dfe640 1 -- 172.21.15.172:0/2466036409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e00154d90 con 0x7f2e0015c380 2024-04-14T08:34:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.422+0000 7f2e05dfe640 1 -- 172.21.15.172:0/2466036409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e00068590 con 0x7f2e0015c380 2024-04-14T08:34:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.423+0000 7f2de7fff640 1 -- 172.21.15.172:0/2466036409 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2df8046020 con 0x7f2e0015c380 2024-04-14T08:34:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.423+0000 7f2de7fff640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2dec06c8f0 0x7f2dec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.423+0000 7f2de7fff640 1 -- 172.21.15.172:0/2466036409 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f2df80ae320 con 0x7f2e0015c380 2024-04-14T08:34:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.423+0000 7f2dff7fe640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2dec06c8f0 0x7f2dec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.424+0000 7f2dff7fe640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2dec06c8f0 0x7f2dec06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2df000a970 tx=0x7f2df0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.428+0000 7f2de7fff640 1 -- 172.21.15.172:0/2466036409 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2df804d030 con 0x7f2e0015c380 2024-04-14T08:34:30.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:30.614+0000 7f2e05dfe640 1 -- 172.21.15.172:0/2466036409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2e000b38b0 con 0x7f2e0015c380 2024-04-14T08:34:31.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.392+0000 7f2de7fff640 1 -- 172.21.15.172:0/2466036409 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) ==== 97+0+0 (secure 0 0 0) 0x7f2df807c8c0 con 0x7f2e0015c380 2024-04-14T08:34:31.394 INFO:teuthology.orchestra.run.smithi172.stderr: removed all blocklist entries 2024-04-14T08:34:31.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.395+0000 7f2e05dfe640 1 -- 172.21.15.172:0/2466036409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2dec06c8f0 msgr2=0x7f2dec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:31.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.395+0000 7f2e05dfe640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2dec06c8f0 0x7f2dec06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2df000a970 tx=0x7f2df0008040 comp rx=0 tx=0).stop 2024-04-14T08:34:31.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.395+0000 7f2e05dfe640 1 -- 172.21.15.172:0/2466036409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e0015c380 msgr2=0x7f2e000cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:31.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.395+0000 7f2e05dfe640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e0015c380 0x7f2e000cf170 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009e50 tx=0x7f2df8004a40 comp rx=0 tx=0).stop 2024-04-14T08:34:31.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.395+0000 7f2e05dfe640 1 -- 172.21.15.172:0/2466036409 shutdown_connections 2024-04-14T08:34:31.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.395+0000 7f2e05dfe640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2dec06c8f0 0x7f2dec06edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.395+0000 7f2e05dfe640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e0015c380 0x7f2e000cf170 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.395+0000 7f2e05dfe640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e00140770 0x7f2e00154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.395+0000 7f2e05dfe640 1 --2- 172.21.15.172:0/2466036409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e0013df30 0x7f2e00151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.395+0000 7f2e05dfe640 1 -- 172.21.15.172:0/2466036409 >> 172.21.15.172:0/2466036409 conn(0x7f2e0006b520 msgr2=0x7f2e0013aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:31.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.395+0000 7f2e05dfe640 1 -- 172.21.15.172:0/2466036409 shutdown_connections 2024-04-14T08:34:31.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.395+0000 7f2e05dfe640 1 -- 172.21.15.172:0/2466036409 wait complete. 2024-04-14T08:34:31.411 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-14T08:34:31.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.515+0000 7f15586d5640 1 Processor -- start 2024-04-14T08:34:31.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.515+0000 7f15586d5640 1 -- start start 2024-04-14T08:34:31.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.516+0000 7f15586d5640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15500d2830 0x7f15500d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.516+0000 7f15586d5640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f155015e890 0x7f15500d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.516+0000 7f15586d5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15500cedd0 0x7f15501621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.516+0000 7f15586d5640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f15500cff30 con 0x7f15500d2830 2024-04-14T08:34:31.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.516+0000 7f15586d5640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f15500d2350 con 0x7f155015e890 2024-04-14T08:34:31.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.516+0000 7f15586d5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f15500d3150 con 0x7f15500cedd0 2024-04-14T08:34:31.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.516+0000 7f1556471640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15500d2830 0x7f15500d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.517+0000 7f1556471640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15500d2830 0x7f15500d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47992/0 (socket says 172.21.15.172:47992) 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.517+0000 7f1556471640 1 -- 172.21.15.172:0/1378195178 learned_addr learned my addr 172.21.15.172:0/1378195178 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.517+0000 7f1555c70640 1 --2- 172.21.15.172:0/1378195178 >> v2:172.21.15.177:3300/0 conn(0x7f155015e890 0x7f15500d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.517+0000 7f1556c72640 1 --2- 172.21.15.172:0/1378195178 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15500cedd0 0x7f15501621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.517+0000 7f1556471640 1 -- 172.21.15.172:0/1378195178 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15500cedd0 msgr2=0x7f15501621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.517+0000 7f1556471640 1 --2- 172.21.15.172:0/1378195178 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15500cedd0 0x7f15501621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.517+0000 7f1556471640 1 -- 172.21.15.172:0/1378195178 >> v2:172.21.15.177:3300/0 conn(0x7f155015e890 msgr2=0x7f15500d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.517+0000 7f1556471640 1 --2- 172.21.15.172:0/1378195178 >> v2:172.21.15.177:3300/0 conn(0x7f155015e890 0x7f15500d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.517+0000 7f1556471640 1 -- 172.21.15.172:0/1378195178 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15500cf320 con 0x7f15500d2830 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.517+0000 7f1555c70640 1 --2- 172.21.15.172:0/1378195178 >> v2:172.21.15.177:3300/0 conn(0x7f155015e890 0x7f15500d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f1556471640 1 --2- 172.21.15.172:0/1378195178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15500d2830 0x7f15500d2c10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1550162100 tx=0x7f154c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e1aff3da477420 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f155546f640 1 -- 172.21.15.172:0/1378195178 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c02cb00 con 0x7f15500d2830 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f155546f640 1 -- 172.21.15.172:0/1378195178 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f154c02cca0 con 0x7f15500d2830 2024-04-14T08:34:31.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f155546f640 1 -- 172.21.15.172:0/1378195178 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c034610 con 0x7f15500d2830 2024-04-14T08:34:31.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f15586d5640 1 -- 172.21.15.172:0/1378195178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15500d2830 msgr2=0x7f15500d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:31.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f15586d5640 1 --2- 172.21.15.172:0/1378195178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15500d2830 0x7f15500d2c10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1550162100 tx=0x7f154c02bf60 comp rx=0 tx=0).stop 2024-04-14T08:34:31.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f15586d5640 1 -- 172.21.15.172:0/1378195178 shutdown_connections 2024-04-14T08:34:31.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f15586d5640 1 --2- 172.21.15.172:0/1378195178 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15500cedd0 0x7f15501621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f15586d5640 1 --2- 172.21.15.172:0/1378195178 >> v2:172.21.15.177:3300/0 conn(0x7f155015e890 0x7f15500d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f15586d5640 1 --2- 172.21.15.172:0/1378195178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15500d2830 0x7f15500d2c10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f15586d5640 1 -- 172.21.15.172:0/1378195178 >> 172.21.15.172:0/1378195178 conn(0x7f155009d590 msgr2=0x7f15500d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f15586d5640 1 -- 172.21.15.172:0/1378195178 shutdown_connections 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.518+0000 7f15586d5640 1 -- 172.21.15.172:0/1378195178 wait complete. 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.519+0000 7f15586d5640 1 Processor -- start 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.520+0000 7f15586d5640 1 -- start start 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.520+0000 7f15586d5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15500cedd0 0x7f1550159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.520+0000 7f15586d5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15500d2830 0x7f15500b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.520+0000 7f15586d5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f155015e890 0x7f15500b94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.520+0000 7f15586d5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15500cfbc0 con 0x7f155015e890 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.520+0000 7f15586d5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f15500cf8e0 con 0x7f15500d2830 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.520+0000 7f15586d5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f15500d1840 con 0x7f15500cedd0 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.520+0000 7f1556c72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f155015e890 0x7f15500b94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.520+0000 7f1556c72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f155015e890 0x7f15500b94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48004/0 (socket says 172.21.15.172:48004) 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.520+0000 7f1556c72640 1 -- 172.21.15.172:0/580165146 learned_addr learned my addr 172.21.15.172:0/580165146 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.520+0000 7f1555c70640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15500d2830 0x7f15500b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.520+0000 7f1556471640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15500cedd0 0x7f1550159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.521+0000 7f1556c72640 1 -- 172.21.15.172:0/580165146 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15500cedd0 msgr2=0x7f1550159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.521+0000 7f1556c72640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15500cedd0 0x7f1550159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.521+0000 7f1556c72640 1 -- 172.21.15.172:0/580165146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15500d2830 msgr2=0x7f15500b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.521+0000 7f1556c72640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15500d2830 0x7f15500b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.521+0000 7f1556c72640 1 -- 172.21.15.172:0/580165146 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15500b7aa0 con 0x7f155015e890 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.521+0000 7f1556c72640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f155015e890 0x7f15500b94a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f154800d5d0 tx=0x7f154800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.521+0000 7f1546ffd640 1 -- 172.21.15.172:0/580165146 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15480125f0 con 0x7f155015e890 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.521+0000 7f1546ffd640 1 -- 172.21.15.172:0/580165146 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1548019070 con 0x7f155015e890 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.521+0000 7f15586d5640 1 -- 172.21.15.172:0/580165146 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15500b7d40 con 0x7f155015e890 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.521+0000 7f1546ffd640 1 -- 172.21.15.172:0/580165146 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15480129f0 con 0x7f155015e890 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.521+0000 7f15586d5640 1 -- 172.21.15.172:0/580165146 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15500b8300 con 0x7f155015e890 2024-04-14T08:34:31.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.522+0000 7f15586d5640 1 -- 172.21.15.172:0/580165146 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1550068520 con 0x7f155015e890 2024-04-14T08:34:31.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.522+0000 7f1546ffd640 1 -- 172.21.15.172:0/580165146 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1548012b90 con 0x7f155015e890 2024-04-14T08:34:31.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.523+0000 7f1546ffd640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f153406c8f0 0x7f153406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.523+0000 7f1546ffd640 1 -- 172.21.15.172:0/580165146 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5249+0+0 (secure 0 0 0) 0x7f154808bc00 con 0x7f155015e890 2024-04-14T08:34:31.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.523+0000 7f1556471640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f153406c8f0 0x7f153406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.523+0000 7f1556471640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f153406c8f0 0x7f153406edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f154c035000 tx=0x7f154c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:31.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.527+0000 7f1546ffd640 1 -- 172.21.15.172:0/580165146 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1548092080 con 0x7f155015e890 2024-04-14T08:34:31.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.768+0000 7f15586d5640 1 -- 172.21.15.172:0/580165146 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f15500c76d0 con 0x7f155015e890 2024-04-14T08:34:31.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.768+0000 7f1546ffd640 1 -- 172.21.15.172:0/580165146 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f1548059bb0 con 0x7f155015e890 2024-04-14T08:34:31.770 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:31.770 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:34:31.770 INFO:teuthology.orchestra.run.smithi172.stdout: "auth_dump": [ 2024-04-14T08:34:31.770 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.770 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.a", 2024-04-14T08:34:31.770 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmnNCRDRAAz7aU3JD7iiaTQH0cNotfaw==", 2024-04-14T08:34:31.770 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.770 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:34:31.770 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:34:31.770 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.b", 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtm1on4GRAAms1he+O4jWu01z0ZLLgL0A==", 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.771 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.c", 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmb7z0FBAAGVu3+TWshDGvd7/fLbgJ5A==", 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.d", 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmEEx0IRAAXiFiOK1RE/0bfyu7KI/3ng==", 2024-04-14T08:34:31.772 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.0", 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB/lBtmnsWuCRAAqcuQhsrzpGVfrlgwTnFsOw==", 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.1", 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCClBtme9lPBxAAFB1E3g8qwZxPCdHiFogp5Q==", 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.2", 2024-04-14T08:34:31.774 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCFlBtm966YBRAAWXZGq/0TuUMZTsyeR4xWww==", 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.3", 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCIlBtmHpktBBAAUbddNHESwyNvvp8UE/SaCw==", 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:34:31.775 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.4", 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCOlBtm+//7ChAA7rApbKll2vBGcDlW8q+kyg==", 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.5", 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCRlBtmVU4YChAAysdUosQ+TnHtjnRl1FOISA==", 2024-04-14T08:34:31.776 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.6", 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCUlBtmnE7+BxAAqBn8YMwecA2NcZz63eBm7Q==", 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.777 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.7", 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCXlBtmN8DhBRAA6CHQ943t6qJCL77jcbOv4w==", 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.0", 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA==", 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.778 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow r", 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.1", 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmp4nkLhAAhYKm8VcykVbPSohvK11I9w==", 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.779 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.2", 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmJoV9KxAAp6zlRX+gvhASQBHjpF9ewQ==", 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.3", 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmeC+pNBAASi7W4jlo6imMMzNEwLrPDA==", 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.780 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:34:31.781 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:34:31.781 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:34:31.781 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:34:31.781 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.781 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.781 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.781 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.admin", 2024-04-14T08:34:31.781 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB6lBtmgnWqLBAAQ0fgarM8au3zFuMNnQIvVw==", 2024-04-14T08:34:31.781 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.782 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:34:31.782 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.x", 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmm5IbChAAeVcVs3Zt6+lU0wghjCcwnQ==", 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:31.783 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:31.784 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.y", 2024-04-14T08:34:31.784 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmfddYBhAAh8zgf4W3WWmNYWNAA7WuRw==", 2024-04-14T08:34:31.784 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:34:31.784 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:34:31.784 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:34:31.784 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:34:31.784 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.784 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:31.784 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-04-14T08:34:31.784 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:34:31.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.770+0000 7f15586d5640 1 -- 172.21.15.172:0/580165146 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f153406c8f0 msgr2=0x7f153406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:31.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.770+0000 7f15586d5640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f153406c8f0 0x7f153406edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f154c035000 tx=0x7f154c030000 comp rx=0 tx=0).stop 2024-04-14T08:34:31.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.770+0000 7f15586d5640 1 -- 172.21.15.172:0/580165146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f155015e890 msgr2=0x7f15500b94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:31.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.770+0000 7f15586d5640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f155015e890 0x7f15500b94a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f154800d5d0 tx=0x7f154800da90 comp rx=0 tx=0).stop 2024-04-14T08:34:31.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.770+0000 7f15586d5640 1 -- 172.21.15.172:0/580165146 shutdown_connections 2024-04-14T08:34:31.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.770+0000 7f15586d5640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f153406c8f0 0x7f153406edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.770+0000 7f15586d5640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f155015e890 0x7f15500b94a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.770+0000 7f15586d5640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15500d2830 0x7f15500b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.770+0000 7f15586d5640 1 --2- 172.21.15.172:0/580165146 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15500cedd0 0x7f1550159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.770+0000 7f15586d5640 1 -- 172.21.15.172:0/580165146 >> 172.21.15.172:0/580165146 conn(0x7f155009d590 msgr2=0x7f15500d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:31.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.770+0000 7f15586d5640 1 -- 172.21.15.172:0/580165146 shutdown_connections 2024-04-14T08:34:31.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.770+0000 7f15586d5640 1 -- 172.21.15.172:0/580165146 wait complete. 2024-04-14T08:34:31.787 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:34:31.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.886+0000 7fc552db4640 1 Processor -- start 2024-04-14T08:34:31.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.886+0000 7fc552db4640 1 -- start start 2024-04-14T08:34:31.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.887+0000 7fc552db4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc54c15e5b0 0x7fc54c0ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.887+0000 7fc552db4640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc54c0729b0 0x7fc54c06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.887+0000 7fc552db4640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc54c0b41d0 con 0x7fc54c06e160 2024-04-14T08:34:31.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.887+0000 7fc552db4640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc54c0aebe0 con 0x7fc54c0729b0 2024-04-14T08:34:31.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.887+0000 7fc552db4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc54c0b4470 con 0x7fc54c15e5b0 2024-04-14T08:34:31.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.887+0000 7fc54bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc54c0729b0 0x7fc54c06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.887+0000 7fc54bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc54c0729b0 0x7fc54c06dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48010/0 (socket says 172.21.15.172:48010) 2024-04-14T08:34:31.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.887+0000 7fc54bfff640 1 -- 172.21.15.172:0/2279559167 learned_addr learned my addr 172.21.15.172:0/2279559167 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:31.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.887+0000 7fc550b50640 1 --2- 172.21.15.172:0/2279559167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc54c15e5b0 0x7fc54c0ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.888+0000 7fc54bfff640 1 -- 172.21.15.172:0/2279559167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc54c15e5b0 msgr2=0x7fc54c0ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:31.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.888+0000 7fc54bfff640 1 --2- 172.21.15.172:0/2279559167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc54c15e5b0 0x7fc54c0ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.888+0000 7fc54bfff640 1 -- 172.21.15.172:0/2279559167 >> v1:172.21.15.177:6789/0 conn(0x7fc54c06e160 legacy=0x7fc54c06c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:31.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.888+0000 7fc54bfff640 1 -- 172.21.15.172:0/2279559167 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c0ae8d0 con 0x7fc54c0729b0 2024-04-14T08:34:31.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.888+0000 7fc550b50640 1 --2- 172.21.15.172:0/2279559167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc54c15e5b0 0x7fc54c0ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:31.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.888+0000 7fc54bfff640 1 --2- 172.21.15.172:0/2279559167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c0729b0 0x7fc54c06dc20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc540009870 tx=0x7fc54002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857fe145073fc16d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:31.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.889+0000 7fc54b7fe640 1 -- 172.21.15.172:0/2279559167 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540034070 con 0x7fc54c0729b0 2024-04-14T08:34:31.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.889+0000 7fc54b7fe640 1 -- 172.21.15.172:0/2279559167 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc540030840 con 0x7fc54c0729b0 2024-04-14T08:34:31.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.889+0000 7fc54b7fe640 1 -- 172.21.15.172:0/2279559167 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540030c10 con 0x7fc54c0729b0 2024-04-14T08:34:31.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.889+0000 7fc552db4640 1 -- 172.21.15.172:0/2279559167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c0729b0 msgr2=0x7fc54c06dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:31.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.889+0000 7fc552db4640 1 --2- 172.21.15.172:0/2279559167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c0729b0 0x7fc54c06dc20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc540009870 tx=0x7fc54002b980 comp rx=0 tx=0).stop 2024-04-14T08:34:31.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.889+0000 7fc552db4640 1 -- 172.21.15.172:0/2279559167 shutdown_connections 2024-04-14T08:34:31.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.889+0000 7fc552db4640 1 --2- 172.21.15.172:0/2279559167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c0729b0 0x7fc54c06dc20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.889+0000 7fc552db4640 1 --2- 172.21.15.172:0/2279559167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc54c15e5b0 0x7fc54c0ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.889+0000 7fc552db4640 1 -- 172.21.15.172:0/2279559167 >> 172.21.15.172:0/2279559167 conn(0x7fc54c071ba0 msgr2=0x7fc54c071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.889+0000 7fc552db4640 1 -- 172.21.15.172:0/2279559167 shutdown_connections 2024-04-14T08:34:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.889+0000 7fc552db4640 1 -- 172.21.15.172:0/2279559167 wait complete. 2024-04-14T08:34:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.890+0000 7fc552db4640 1 Processor -- start 2024-04-14T08:34:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.890+0000 7fc552db4640 1 -- start start 2024-04-14T08:34:31.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.891+0000 7fc552db4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc54c06e160 0x7fc54c0cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.891+0000 7fc552db4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c0729b0 0x7fc54c1f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.891+0000 7fc552db4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c15e5b0 0x7fc54c158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.891+0000 7fc552db4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c06cdc0 con 0x7fc54c15e5b0 2024-04-14T08:34:31.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.891+0000 7fc552db4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c06cc40 con 0x7fc54c0729b0 2024-04-14T08:34:31.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.891+0000 7fc552db4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc54c0b4c70 con 0x7fc54c06e160 2024-04-14T08:34:31.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.891+0000 7fc551351640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c15e5b0 0x7fc54c158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.891+0000 7fc551351640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c15e5b0 0x7fc54c158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48014/0 (socket says 172.21.15.172:48014) 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.891+0000 7fc551351640 1 -- 172.21.15.172:0/3338132988 learned_addr learned my addr 172.21.15.172:0/3338132988 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.891+0000 7fc550b50640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc54c06e160 0x7fc54c0cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.891+0000 7fc54bfff640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c0729b0 0x7fc54c1f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.892+0000 7fc551351640 1 -- 172.21.15.172:0/3338132988 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc54c06e160 msgr2=0x7fc54c0cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.892+0000 7fc551351640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc54c06e160 0x7fc54c0cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.892+0000 7fc551351640 1 -- 172.21.15.172:0/3338132988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c0729b0 msgr2=0x7fc54c1f56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.892+0000 7fc551351640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c0729b0 0x7fc54c1f56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.892+0000 7fc551351640 1 -- 172.21.15.172:0/3338132988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c157860 con 0x7fc54c15e5b0 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.892+0000 7fc54bfff640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c0729b0 0x7fc54c1f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.892+0000 7fc551351640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c15e5b0 0x7fc54c158620 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc54400c730 tx=0x7fc54400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.892+0000 7fc5497fa640 1 -- 172.21.15.172:0/3338132988 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544016020 con 0x7fc54c15e5b0 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.892+0000 7fc5497fa640 1 -- 172.21.15.172:0/3338132988 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc544006d50 con 0x7fc54c15e5b0 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.892+0000 7fc552db4640 1 -- 172.21.15.172:0/3338132988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c157b40 con 0x7fc54c15e5b0 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.892+0000 7fc5497fa640 1 -- 172.21.15.172:0/3338132988 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544003bb0 con 0x7fc54c15e5b0 2024-04-14T08:34:31.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.892+0000 7fc552db4640 1 -- 172.21.15.172:0/3338132988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c0cd800 con 0x7fc54c15e5b0 2024-04-14T08:34:31.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.893+0000 7fc552db4640 1 -- 172.21.15.172:0/3338132988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c06dc20 con 0x7fc54c15e5b0 2024-04-14T08:34:31.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.893+0000 7fc5497fa640 1 -- 172.21.15.172:0/3338132988 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc54400a7f0 con 0x7fc54c15e5b0 2024-04-14T08:34:31.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.894+0000 7fc5497fa640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc53806c8f0 0x7fc53806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:31.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.894+0000 7fc5497fa640 1 -- 172.21.15.172:0/3338132988 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5249+0+0 (secure 0 0 0) 0x7fc54408b550 con 0x7fc54c15e5b0 2024-04-14T08:34:31.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.894+0000 7fc550b50640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc53806c8f0 0x7fc53806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:31.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.894+0000 7fc550b50640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc53806c8f0 0x7fc53806edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc53c006fd0 tx=0x7fc53c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:31.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:31.898+0000 7fc5497fa640 1 -- 172.21.15.172:0/3338132988 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc54c06dc20 con 0x7fc54c15e5b0 2024-04-14T08:34:32.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.076+0000 7fc552db4640 1 -- 172.21.15.172:0/3338132988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc54c068520 con 0x7fc54c15e5b0 2024-04-14T08:34:32.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.077+0000 7fc5497fa640 1 -- 172.21.15.172:0/3338132988 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+13224 (secure 0 0 0) 0x7fc544020020 con 0x7fc54c15e5b0 2024-04-14T08:34:32.079 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:32.079 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":46,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:31.392690+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:34:32.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.079+0000 7fc552db4640 1 -- 172.21.15.172:0/3338132988 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc53806c8f0 msgr2=0x7fc53806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.079+0000 7fc552db4640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc53806c8f0 0x7fc53806edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc53c006fd0 tx=0x7fc53c008040 comp rx=0 tx=0).stop 2024-04-14T08:34:32.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.080+0000 7fc552db4640 1 -- 172.21.15.172:0/3338132988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c15e5b0 msgr2=0x7fc54c158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.080+0000 7fc552db4640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c15e5b0 0x7fc54c158620 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc54400c730 tx=0x7fc54400cbf0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.080+0000 7fc552db4640 1 -- 172.21.15.172:0/3338132988 shutdown_connections 2024-04-14T08:34:32.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.080+0000 7fc552db4640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc53806c8f0 0x7fc53806edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.080+0000 7fc552db4640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c15e5b0 0x7fc54c158620 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.080+0000 7fc552db4640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c0729b0 0x7fc54c1f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.080+0000 7fc552db4640 1 --2- 172.21.15.172:0/3338132988 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc54c06e160 0x7fc54c0cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.080+0000 7fc552db4640 1 -- 172.21.15.172:0/3338132988 >> 172.21.15.172:0/3338132988 conn(0x7fc54c071ba0 msgr2=0x7fc54c0b5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:32.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.080+0000 7fc552db4640 1 -- 172.21.15.172:0/3338132988 shutdown_connections 2024-04-14T08:34:32.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.080+0000 7fc552db4640 1 -- 172.21.15.172:0/3338132988 wait complete. 2024-04-14T08:34:32.092 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:34:32.093 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:34:32.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.195+0000 7fa0586de640 1 Processor -- start 2024-04-14T08:34:32.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.195+0000 7fa0586de640 1 -- start start 2024-04-14T08:34:32.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.196+0000 7fa0586de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa05013fa70 0x7fa0500af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.196+0000 7fa0586de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa050071520 con 0x7fa0500ce150 2024-04-14T08:34:32.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.196+0000 7fa0586de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa050071800 con 0x7fa0500ce5e0 2024-04-14T08:34:32.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.196+0000 7fa0586de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa0500712d0 con 0x7fa05013fa70 2024-04-14T08:34:32.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.196+0000 7fa05647a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa05013fa70 0x7fa0500af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.196+0000 7fa055c79640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa0500ce150 0x7fa0500722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40054/0 (socket says 172.21.15.172:40054) 2024-04-14T08:34:32.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.196+0000 7fa05647a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa05013fa70 0x7fa0500af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48016/0 (socket says 172.21.15.172:48016) 2024-04-14T08:34:32.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.196+0000 7fa055c79640 1 -- 172.21.15.172:0/1026507453 learned_addr learned my addr 172.21.15.172:0/1026507453 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:32.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.196+0000 7fa05647a640 1 -- 172.21.15.172:0/1026507453 >> v1:172.21.15.177:6789/0 conn(0x7fa0500ce5e0 legacy=0x7fa05013fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:32.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.196+0000 7fa05647a640 1 -- 172.21.15.172:0/1026507453 >> v1:172.21.15.172:6789/0 conn(0x7fa0500ce150 legacy=0x7fa0500722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.196+0000 7fa055478640 1 -- 172.21.15.172:0/1026507453 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3392697066 0 0) 0x7fa050071520 con 0x7fa0500ce150 2024-04-14T08:34:32.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.196+0000 7fa05647a640 1 -- 172.21.15.172:0/1026507453 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa050071ae0 con 0x7fa05013fa70 2024-04-14T08:34:32.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.197+0000 7fa05647a640 1 --2- 172.21.15.172:0/1026507453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05013fa70 0x7fa0500af740 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00ba40 tx=0x7fa04c02e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ae5131b5f2248d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:32.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.197+0000 7fa055478640 1 -- 172.21.15.172:0/1026507453 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c03b070 con 0x7fa05013fa70 2024-04-14T08:34:32.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.197+0000 7fa055478640 1 -- 172.21.15.172:0/1026507453 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa04c02edc0 con 0x7fa05013fa70 2024-04-14T08:34:32.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.197+0000 7fa055478640 1 -- 172.21.15.172:0/1026507453 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c036660 con 0x7fa05013fa70 2024-04-14T08:34:32.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.198+0000 7fa0586de640 1 -- 172.21.15.172:0/1026507453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05013fa70 msgr2=0x7fa0500af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.198+0000 7fa0586de640 1 --2- 172.21.15.172:0/1026507453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05013fa70 0x7fa0500af740 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00ba40 tx=0x7fa04c02e010 comp rx=0 tx=0).stop 2024-04-14T08:34:32.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.198+0000 7fa0586de640 1 -- 172.21.15.172:0/1026507453 shutdown_connections 2024-04-14T08:34:32.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.198+0000 7fa0586de640 1 --2- 172.21.15.172:0/1026507453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05013fa70 0x7fa0500af740 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.198+0000 7fa0586de640 1 -- 172.21.15.172:0/1026507453 >> 172.21.15.172:0/1026507453 conn(0x7fa05006b520 msgr2=0x7fa05006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:32.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.198+0000 7fa0586de640 1 -- 172.21.15.172:0/1026507453 shutdown_connections 2024-04-14T08:34:32.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.198+0000 7fa0586de640 1 -- 172.21.15.172:0/1026507453 wait complete. 2024-04-14T08:34:32.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.199+0000 7fa0586de640 1 Processor -- start 2024-04-14T08:34:32.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.199+0000 7fa0586de640 1 -- start start 2024-04-14T08:34:32.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa0586de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0500ce150 0x7fa05013e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa0586de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0500ce5e0 0x7fa0500aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa0586de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05013fa70 0x7fa0501e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa0586de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0501442b0 con 0x7fa05013fa70 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa0586de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa050144130 con 0x7fa0500ce150 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa0586de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa0500729f0 con 0x7fa0500ce5e0 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa056c7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05013fa70 0x7fa0501e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa056c7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05013fa70 0x7fa0501e51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48018/0 (socket says 172.21.15.172:48018) 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa056c7b640 1 -- 172.21.15.172:0/158066447 learned_addr learned my addr 172.21.15.172:0/158066447 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa055c79640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0500ce5e0 0x7fa0500aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa05647a640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0500ce150 0x7fa05013e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa056c7b640 1 -- 172.21.15.172:0/158066447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0500ce5e0 msgr2=0x7fa0500aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa056c7b640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0500ce5e0 0x7fa0500aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa056c7b640 1 -- 172.21.15.172:0/158066447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0500ce150 msgr2=0x7fa05013e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa056c7b640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0500ce150 0x7fa05013e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa056c7b640 1 -- 172.21.15.172:0/158066447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0501e4150 con 0x7fa05013fa70 2024-04-14T08:34:32.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa056c7b640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05013fa70 0x7fa0501e51d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa04800c700 tx=0x7fa04800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:32.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa046ffd640 1 -- 172.21.15.172:0/158066447 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048016020 con 0x7fa05013fa70 2024-04-14T08:34:32.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa046ffd640 1 -- 172.21.15.172:0/158066447 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa04800aa50 con 0x7fa05013fa70 2024-04-14T08:34:32.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa0586de640 1 -- 172.21.15.172:0/158066447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0501e4410 con 0x7fa05013fa70 2024-04-14T08:34:32.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa046ffd640 1 -- 172.21.15.172:0/158066447 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04800ce10 con 0x7fa05013fa70 2024-04-14T08:34:32.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.200+0000 7fa0586de640 1 -- 172.21.15.172:0/158066447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0501e4cb0 con 0x7fa05013fa70 2024-04-14T08:34:32.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.201+0000 7fa0586de640 1 -- 172.21.15.172:0/158066447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0501e4eb0 con 0x7fa05013fa70 2024-04-14T08:34:32.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.202+0000 7fa046ffd640 1 -- 172.21.15.172:0/158066447 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa048019020 con 0x7fa05013fa70 2024-04-14T08:34:32.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.202+0000 7fa046ffd640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa03406c920 0x7fa03406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.202+0000 7fa046ffd640 1 -- 172.21.15.172:0/158066447 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5249+0+0 (secure 0 0 0) 0x7fa04808bb70 con 0x7fa05013fa70 2024-04-14T08:34:32.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.202+0000 7fa05647a640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa03406c920 0x7fa03406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.203+0000 7fa05647a640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa03406c920 0x7fa03406ede0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa04c038000 tx=0x7fa04c032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:32.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.206+0000 7fa046ffd640 1 -- 172.21.15.172:0/158066447 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa0501e4eb0 con 0x7fa05013fa70 2024-04-14T08:34:32.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.379+0000 7fa0586de640 1 -- 172.21.15.172:0/158066447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa050068590 con 0x7fa05013fa70 2024-04-14T08:34:32.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.379+0000 7fa046ffd640 1 -- 172.21.15.172:0/158066447 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7fa048020020 con 0x7fa05013fa70 2024-04-14T08:34:32.381 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:34:32.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.381+0000 7fa0586de640 1 -- 172.21.15.172:0/158066447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa03406c920 msgr2=0x7fa03406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.381+0000 7fa0586de640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa03406c920 0x7fa03406ede0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa04c038000 tx=0x7fa04c032000 comp rx=0 tx=0).stop 2024-04-14T08:34:32.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.381+0000 7fa0586de640 1 -- 172.21.15.172:0/158066447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05013fa70 msgr2=0x7fa0501e51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.381+0000 7fa0586de640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05013fa70 0x7fa0501e51d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa04800c700 tx=0x7fa04800cbc0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.381+0000 7fa0586de640 1 -- 172.21.15.172:0/158066447 shutdown_connections 2024-04-14T08:34:32.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.381+0000 7fa0586de640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa03406c920 0x7fa03406ede0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.381+0000 7fa0586de640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa05013fa70 0x7fa0501e51d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.381+0000 7fa0586de640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0500ce5e0 0x7fa0500aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.381+0000 7fa0586de640 1 --2- 172.21.15.172:0/158066447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0500ce150 0x7fa05013e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.381+0000 7fa0586de640 1 -- 172.21.15.172:0/158066447 >> 172.21.15.172:0/158066447 conn(0x7fa05006b520 msgr2=0x7fa0500b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:32.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.381+0000 7fa0586de640 1 -- 172.21.15.172:0/158066447 shutdown_connections 2024-04-14T08:34:32.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.381+0000 7fa0586de640 1 -- 172.21.15.172:0/158066447 wait complete. 2024-04-14T08:34:32.394 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-14T08:34:32.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.493+0000 7fd5d6dfe640 1 Processor -- start 2024-04-14T08:34:32.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.493+0000 7fd5d6dfe640 1 -- start start 2024-04-14T08:34:32.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5d6dfe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd5d015a690 0x7fd5d0071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5d6dfe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd5d0072970 0x7fd5d00722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5d6dfe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5d015aa70 0x7fd5d015e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5d6dfe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd5d00b3360 con 0x7fd5d0072970 2024-04-14T08:34:32.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5d6dfe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fd5d00b3640 con 0x7fd5d015a690 2024-04-14T08:34:32.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5d6dfe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d00b3920 con 0x7fd5d015aa70 2024-04-14T08:34:32.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5cffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd5d0072970 0x7fd5d00722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5cffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd5d0072970 0x7fd5d00722a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48022/0 (socket says 172.21.15.172:48022) 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5cffff640 1 -- 172.21.15.172:0/778796503 learned_addr learned my addr 172.21.15.172:0/778796503 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5d539b640 1 --2- 172.21.15.172:0/778796503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5d015aa70 0x7fd5d015e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5d4b9a640 1 --2- 172.21.15.172:0/778796503 >> v2:172.21.15.177:3300/0 conn(0x7fd5d015a690 0x7fd5d0071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5cffff640 1 -- 172.21.15.172:0/778796503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5d015aa70 msgr2=0x7fd5d015e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5cffff640 1 --2- 172.21.15.172:0/778796503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5d015aa70 0x7fd5d015e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5cffff640 1 -- 172.21.15.172:0/778796503 >> v2:172.21.15.177:3300/0 conn(0x7fd5d015a690 msgr2=0x7fd5d0071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5cffff640 1 --2- 172.21.15.172:0/778796503 >> v2:172.21.15.177:3300/0 conn(0x7fd5d015a690 0x7fd5d0071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.494+0000 7fd5cffff640 1 -- 172.21.15.172:0/778796503 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d00b3c00 con 0x7fd5d0072970 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.495+0000 7fd5cffff640 1 --2- 172.21.15.172:0/778796503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d0072970 0x7fd5d00722a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00b3850 tx=0x7fd5bc02de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6548b8137e311c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.495+0000 7fd5cf7fe640 1 -- 172.21.15.172:0/778796503 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc034070 con 0x7fd5d0072970 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.495+0000 7fd5cf7fe640 1 -- 172.21.15.172:0/778796503 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5bc02ebc0 con 0x7fd5d0072970 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.495+0000 7fd5cf7fe640 1 -- 172.21.15.172:0/778796503 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc038400 con 0x7fd5d0072970 2024-04-14T08:34:32.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.495+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/778796503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d0072970 msgr2=0x7fd5d00722a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.495+0000 7fd5d6dfe640 1 --2- 172.21.15.172:0/778796503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d0072970 0x7fd5d00722a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00b3850 tx=0x7fd5bc02de10 comp rx=0 tx=0).stop 2024-04-14T08:34:32.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.495+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/778796503 shutdown_connections 2024-04-14T08:34:32.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.495+0000 7fd5d6dfe640 1 --2- 172.21.15.172:0/778796503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5d015aa70 0x7fd5d015e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.495+0000 7fd5d6dfe640 1 --2- 172.21.15.172:0/778796503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d0072970 0x7fd5d00722a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.495+0000 7fd5d6dfe640 1 --2- 172.21.15.172:0/778796503 >> v2:172.21.15.177:3300/0 conn(0x7fd5d015a690 0x7fd5d0071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.495+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/778796503 >> 172.21.15.172:0/778796503 conn(0x7fd5d006ec30 msgr2=0x7fd5d006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:32.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.496+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/778796503 shutdown_connections 2024-04-14T08:34:32.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.496+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/778796503 wait complete. 2024-04-14T08:34:32.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.497+0000 7fd5d6dfe640 1 Processor -- start 2024-04-14T08:34:32.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.497+0000 7fd5d6dfe640 1 -- start start 2024-04-14T08:34:32.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d6dfe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d0072970 0x7fd5d015c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d6dfe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5d015a690 0x7fd5d01f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d6dfe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5d015aa70 0x7fd5d01f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d6dfe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d00b44d0 con 0x7fd5d0072970 2024-04-14T08:34:32.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d6dfe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d00b3100 con 0x7fd5d015a690 2024-04-14T08:34:32.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d6dfe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d00b4650 con 0x7fd5d015aa70 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d4b9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d0072970 0x7fd5d015c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d4b9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d0072970 0x7fd5d015c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48030/0 (socket says 172.21.15.172:48030) 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d4b9a640 1 -- 172.21.15.172:0/1612354289 learned_addr learned my addr 172.21.15.172:0/1612354289 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d539b640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5d015aa70 0x7fd5d01f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5cffff640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5d015a690 0x7fd5d01f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d4b9a640 1 -- 172.21.15.172:0/1612354289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5d015aa70 msgr2=0x7fd5d01f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d4b9a640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5d015aa70 0x7fd5d01f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d4b9a640 1 -- 172.21.15.172:0/1612354289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5d015a690 msgr2=0x7fd5d01f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d4b9a640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5d015a690 0x7fd5d01f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d4b9a640 1 -- 172.21.15.172:0/1612354289 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d01f4280 con 0x7fd5d0072970 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d4b9a640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d0072970 0x7fd5d015c0d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400d830 tx=0x7fd5c400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5cd7fa640 1 -- 172.21.15.172:0/1612354289 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4018020 con 0x7fd5d0072970 2024-04-14T08:34:32.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5cd7fa640 1 -- 172.21.15.172:0/1612354289 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c40096a0 con 0x7fd5d0072970 2024-04-14T08:34:32.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/1612354289 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d01f4560 con 0x7fd5d0072970 2024-04-14T08:34:32.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5cd7fa640 1 -- 172.21.15.172:0/1612354289 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4009a70 con 0x7fd5d0072970 2024-04-14T08:34:32.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.498+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/1612354289 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d00ceb20 con 0x7fd5d0072970 2024-04-14T08:34:32.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.499+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/1612354289 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d0068520 con 0x7fd5d0072970 2024-04-14T08:34:32.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.500+0000 7fd5cd7fa640 1 -- 172.21.15.172:0/1612354289 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5c401b020 con 0x7fd5d0072970 2024-04-14T08:34:32.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.500+0000 7fd5cd7fa640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd5b406c8f0 0x7fd5b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.500+0000 7fd5cd7fa640 1 -- 172.21.15.172:0/1612354289 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5249+0+0 (secure 0 0 0) 0x7fd5c408bc20 con 0x7fd5d0072970 2024-04-14T08:34:32.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.500+0000 7fd5cffff640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd5b406c8f0 0x7fd5b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.501+0000 7fd5cffff640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd5b406c8f0 0x7fd5b406edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc039000 tx=0x7fd5bc034000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:32.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.505+0000 7fd5cd7fa640 1 -- 172.21.15.172:0/1612354289 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd5c4022030 con 0x7fd5d0072970 2024-04-14T08:34:32.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.679+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/1612354289 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd5d00d0790 con 0x7fd5d0072970 2024-04-14T08:34:32.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.680+0000 7fd5cd7fa640 1 -- 172.21.15.172:0/1612354289 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v46) ==== 84+0+65 (secure 0 0 0) 0x7fd5c405a210 con 0x7fd5d0072970 2024-04-14T08:34:32.682 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:32.682 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-04-14T08:34:32.682 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:32.682 INFO:teuthology.orchestra.run.smithi172.stdout: "poolnum": 1, 2024-04-14T08:34:32.682 INFO:teuthology.orchestra.run.smithi172.stdout: "poolname": ".mgr" 2024-04-14T08:34:32.682 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:32.682 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-04-14T08:34:32.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.682+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/1612354289 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd5b406c8f0 msgr2=0x7fd5b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.682+0000 7fd5d6dfe640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd5b406c8f0 0x7fd5b406edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc039000 tx=0x7fd5bc034000 comp rx=0 tx=0).stop 2024-04-14T08:34:32.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.682+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/1612354289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d0072970 msgr2=0x7fd5d015c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.682+0000 7fd5d6dfe640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d0072970 0x7fd5d015c0d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400d830 tx=0x7fd5c400dcf0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.682+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/1612354289 shutdown_connections 2024-04-14T08:34:32.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.682+0000 7fd5d6dfe640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd5b406c8f0 0x7fd5b406edb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.682+0000 7fd5d6dfe640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5d015aa70 0x7fd5d01f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.682+0000 7fd5d6dfe640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5d015a690 0x7fd5d01f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.682+0000 7fd5d6dfe640 1 --2- 172.21.15.172:0/1612354289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5d0072970 0x7fd5d015c0d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.682+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/1612354289 >> 172.21.15.172:0/1612354289 conn(0x7fd5d006ec30 msgr2=0x7fd5d015d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:32.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.682+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/1612354289 shutdown_connections 2024-04-14T08:34:32.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.682+0000 7fd5d6dfe640 1 -- 172.21.15.172:0/1612354289 wait complete. 2024-04-14T08:34:32.695 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-14T08:34:32.695 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-14T08:34:32.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.798+0000 7f467257b640 1 Processor -- start 2024-04-14T08:34:32.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.798+0000 7f467257b640 1 -- start start 2024-04-14T08:34:32.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.799+0000 7f467257b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f466c09b7b0 0x7f466c06dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.799+0000 7f467257b640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f466c0d2d50 con 0x7f466c0b3610 2024-04-14T08:34:32.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.799+0000 7f467257b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f466c06ce90 con 0x7f466c09aa70 2024-04-14T08:34:32.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.799+0000 7f467257b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f466c06cc60 con 0x7f466c09b7b0 2024-04-14T08:34:32.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.799+0000 7f466b7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f466c0b3610 0x7f466c0cc690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40066/0 (socket says 172.21.15.172:40066) 2024-04-14T08:34:32.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.799+0000 7f466b7fe640 1 -- 172.21.15.172:0/3320075573 learned_addr learned my addr 172.21.15.172:0/3320075573 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:32.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f4670b18640 1 --2- 172.21.15.172:0/3320075573 >> v2:172.21.15.177:3300/0 conn(0x7f466c09b7b0 0x7f466c06dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3989283171 0 0) 0x7f466c0d2d50 con 0x7f466c0b3610 2024-04-14T08:34:32.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4654003520 con 0x7f466c0b3610 2024-04-14T08:34:32.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2138933793 0 0) 0x7f466c06ce90 con 0x7f466c09aa70 2024-04-14T08:34:32.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f466c0d2d50 con 0x7f466c09aa70 2024-04-14T08:34:32.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1025871268 0 0) 0x7f4654003520 con 0x7f466c0b3610 2024-04-14T08:34:32.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 >> v2:172.21.15.177:3300/0 conn(0x7f466c09b7b0 msgr2=0x7f466c06dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:32.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 --2- 172.21.15.172:0/3320075573 >> v2:172.21.15.177:3300/0 conn(0x7f466c09b7b0 0x7f466c06dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 >> v1:172.21.15.177:6789/0 conn(0x7f466c09aa70 legacy=0x7f466c09ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c06d170 con 0x7f466c0b3610 2024-04-14T08:34:32.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f4660001670 con 0x7f466c0b3610 2024-04-14T08:34:32.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 >> v1:172.21.15.172:6789/0 conn(0x7f466c0b3610 legacy=0x7f466c0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 --2- 172.21.15.172:0/3320075573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 0x7f4654003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 --2- 172.21.15.172:0/3320075573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4654004800 0x7f4654004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 --2- 172.21.15.172:0/3320075573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46540051c0 0x7f465400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f465400c010 con 0x7f4654003b40 2024-04-14T08:34:32.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f465400c1c0 con 0x7f46540051c0 2024-04-14T08:34:32.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f465400c3c0 con 0x7f4654004800 2024-04-14T08:34:32.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466bfff640 1 --2- 172.21.15.172:0/3320075573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 0x7f4654003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f4670b18640 1 --2- 172.21.15.172:0/3320075573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46540051c0 0x7f465400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466b7fe640 1 --2- 172.21.15.172:0/3320075573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4654004800 0x7f4654004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466bfff640 1 -- 172.21.15.172:0/3320075573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4654004800 msgr2=0x7f4654004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:32.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466bfff640 1 --2- 172.21.15.172:0/3320075573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4654004800 0x7f4654004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466bfff640 1 -- 172.21.15.172:0/3320075573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46540051c0 msgr2=0x7f465400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:32.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466bfff640 1 --2- 172.21.15.172:0/3320075573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46540051c0 0x7f465400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f4670b18640 1 -- 172.21.15.172:0/3320075573 reap_dead start 2024-04-14T08:34:32.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.800+0000 7f466bfff640 1 -- 172.21.15.172:0/3320075573 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4654003520 con 0x7f4654003b40 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.801+0000 7f466bfff640 1 --2- 172.21.15.172:0/3320075573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 0x7f4654003f40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f465c0018d0 tx=0x7f4654005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22daa45cf54fc0ef server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.801+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c037070 con 0x7f4654003b40 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.801+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f465c0052d0 con 0x7f4654003b40 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.801+0000 7f466affd640 1 -- 172.21.15.172:0/3320075573 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c0014f0 con 0x7f4654003b40 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.802+0000 7f467257b640 1 -- 172.21.15.172:0/3320075573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 msgr2=0x7f4654003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.802+0000 7f467257b640 1 --2- 172.21.15.172:0/3320075573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 0x7f4654003f40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f465c0018d0 tx=0x7f4654005830 comp rx=0 tx=0).stop 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.802+0000 7f467257b640 1 -- 172.21.15.172:0/3320075573 shutdown_connections 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.802+0000 7f467257b640 1 --2- 172.21.15.172:0/3320075573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 0x7f4654003f40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.802+0000 7f467257b640 1 -- 172.21.15.172:0/3320075573 >> 172.21.15.172:0/3320075573 conn(0x7f466c0d2040 msgr2=0x7f466c0d2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.802+0000 7f467257b640 1 -- 172.21.15.172:0/3320075573 shutdown_connections 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.802+0000 7f467257b640 1 -- 172.21.15.172:0/3320075573 wait complete. 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.803+0000 7f467257b640 1 Processor -- start 2024-04-14T08:34:32.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.803+0000 7f467257b640 1 -- start start 2024-04-14T08:34:32.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f467257b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 0x7f466c0b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f467257b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f466c0b5ee0 0x7f466c15a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f467257b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f466c15fd00 0x7f466c160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f467257b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f466c161e40 con 0x7f4654003b40 2024-04-14T08:34:32.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f467257b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f466c0cff40 con 0x7f466c0b5ee0 2024-04-14T08:34:32.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f467257b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f466c161fc0 con 0x7f466c15fd00 2024-04-14T08:34:32.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f466bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 0x7f466c0b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f466bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 0x7f466c0b79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48042/0 (socket says 172.21.15.172:48042) 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f466bfff640 1 -- 172.21.15.172:0/3729891380 learned_addr learned my addr 172.21.15.172:0/3729891380 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f4670b18640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f466c15fd00 0x7f466c160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f466b7fe640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f466c0b5ee0 0x7f466c15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f466bfff640 1 -- 172.21.15.172:0/3729891380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f466c15fd00 msgr2=0x7f466c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f466bfff640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f466c15fd00 0x7f466c160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f466bfff640 1 -- 172.21.15.172:0/3729891380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f466c0b5ee0 msgr2=0x7f466c15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f466bfff640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f466c0b5ee0 0x7f466c15a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f466bfff640 1 -- 172.21.15.172:0/3729891380 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c0b6410 con 0x7f4654003b40 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.804+0000 7f466bfff640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 0x7f466c0b79f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f465c0018d0 tx=0x7f465c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.805+0000 7f4668ff9640 1 -- 172.21.15.172:0/3729891380 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c042020 con 0x7f4654003b40 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.805+0000 7f4668ff9640 1 -- 172.21.15.172:0/3729891380 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f465c038040 con 0x7f4654003b40 2024-04-14T08:34:32.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.805+0000 7f4668ff9640 1 -- 172.21.15.172:0/3729891380 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c037070 con 0x7f4654003b40 2024-04-14T08:34:32.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.805+0000 7f467257b640 1 -- 172.21.15.172:0/3729891380 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466c0b66f0 con 0x7f4654003b40 2024-04-14T08:34:32.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.805+0000 7f467257b640 1 -- 172.21.15.172:0/3729891380 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466c0b6cb0 con 0x7f4654003b40 2024-04-14T08:34:32.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.806+0000 7f467257b640 1 -- 172.21.15.172:0/3729891380 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466c1606c0 con 0x7f4654003b40 2024-04-14T08:34:32.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.806+0000 7f4668ff9640 1 -- 172.21.15.172:0/3729891380 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f465c045020 con 0x7f4654003b40 2024-04-14T08:34:32.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.806+0000 7f4668ff9640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f465806c8f0 0x7f465806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:32.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.807+0000 7f466b7fe640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f465806c8f0 0x7f465806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:32.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.807+0000 7f4668ff9640 1 -- 172.21.15.172:0/3729891380 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5249+0+0 (secure 0 0 0) 0x7f465c0ad000 con 0x7f4654003b40 2024-04-14T08:34:32.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.807+0000 7f466b7fe640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f465806c8f0 0x7f465806edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4660000fd0 tx=0x7f4660009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:32.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.811+0000 7f4668ff9640 1 -- 172.21.15.172:0/3729891380 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f465c04c030 con 0x7f4654003b40 2024-04-14T08:34:32.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:32.986+0000 7f467257b640 1 -- 172.21.15.172:0/3729891380 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f466c0686a0 con 0x7f4654003b40 2024-04-14T08:34:33.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.397+0000 7f4668ff9640 1 -- 172.21.15.172:0/3729891380 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v47) ==== 138+0+0 (secure 0 0 0) 0x7f465c07b680 con 0x7f4654003b40 2024-04-14T08:34:33.399 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' created 2024-04-14T08:34:33.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.400+0000 7f467257b640 1 -- 172.21.15.172:0/3729891380 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f465806c8f0 msgr2=0x7f465806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:33.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.400+0000 7f467257b640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f465806c8f0 0x7f465806edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4660000fd0 tx=0x7f4660009040 comp rx=0 tx=0).stop 2024-04-14T08:34:33.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.400+0000 7f467257b640 1 -- 172.21.15.172:0/3729891380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 msgr2=0x7f466c0b79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:33.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.400+0000 7f467257b640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 0x7f466c0b79f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f465c0018d0 tx=0x7f465c0012d0 comp rx=0 tx=0).stop 2024-04-14T08:34:33.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.400+0000 7f467257b640 1 -- 172.21.15.172:0/3729891380 shutdown_connections 2024-04-14T08:34:33.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.400+0000 7f467257b640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f465806c8f0 0x7f465806edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:33.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.400+0000 7f467257b640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f466c15fd00 0x7f466c160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:33.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.400+0000 7f467257b640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f466c0b5ee0 0x7f466c15a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:33.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.400+0000 7f467257b640 1 --2- 172.21.15.172:0/3729891380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4654003b40 0x7f466c0b79f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:33.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.400+0000 7f467257b640 1 -- 172.21.15.172:0/3729891380 >> 172.21.15.172:0/3729891380 conn(0x7f466c0d2040 msgr2=0x7f466c0cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:33.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.400+0000 7f467257b640 1 -- 172.21.15.172:0/3729891380 shutdown_connections 2024-04-14T08:34:33.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.400+0000 7f467257b640 1 -- 172.21.15.172:0/3729891380 wait complete. 2024-04-14T08:34:33.416 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-14T08:34:33.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.522+0000 7fa9260de640 1 Processor -- start 2024-04-14T08:34:33.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.522+0000 7fa9260de640 1 -- start start 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.523+0000 7fa9260de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa92013df30 0x7fa9200b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.523+0000 7fa9260de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa92015c380 0x7fa9200b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.523+0000 7fa9260de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa92015b620 con 0x7fa920140770 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.523+0000 7fa9260de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa9200cf9b0 con 0x7fa92013df30 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.523+0000 7fa9260de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa9201414a0 con 0x7fa92015c380 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.523+0000 7fa91f7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa92013df30 0x7fa9200b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.523+0000 7fa91effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa92015c380 0x7fa9200b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.524+0000 7fa91ffff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fa920140770 0x7fa920140c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:46476/0 (socket says 172.21.15.172:46476) 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.524+0000 7fa91ffff640 1 -- 172.21.15.172:0/3874753685 learned_addr learned my addr 172.21.15.172:0/3874753685 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.524+0000 7fa91f7fe640 1 -- 172.21.15.172:0/3874753685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa92015c380 msgr2=0x7fa9200b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.524+0000 7fa91f7fe640 1 --2- 172.21.15.172:0/3874753685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa92015c380 0x7fa9200b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.524+0000 7fa91f7fe640 1 -- 172.21.15.172:0/3874753685 >> v1:172.21.15.177:6789/0 conn(0x7fa920140770 legacy=0x7fa920140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:33.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.524+0000 7fa91f7fe640 1 -- 172.21.15.172:0/3874753685 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92015b8a0 con 0x7fa92013df30 2024-04-14T08:34:33.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.524+0000 7fa91effd640 1 --2- 172.21.15.172:0/3874753685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa92015c380 0x7fa9200b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:34:33.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.525+0000 7fa91f7fe640 1 --2- 172.21.15.172:0/3874753685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa92013df30 0x7fa9200b1cd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097d0 tx=0x7fa91002b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eec85d500daeb031 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:33.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.525+0000 7fa91e7fc640 1 -- 172.21.15.172:0/3874753685 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910035020 con 0x7fa92013df30 2024-04-14T08:34:33.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.525+0000 7fa91e7fc640 1 -- 172.21.15.172:0/3874753685 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa910031800 con 0x7fa92013df30 2024-04-14T08:34:33.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.525+0000 7fa91e7fc640 1 -- 172.21.15.172:0/3874753685 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910031bd0 con 0x7fa92013df30 2024-04-14T08:34:33.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.525+0000 7fa9260de640 1 -- 172.21.15.172:0/3874753685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa92013df30 msgr2=0x7fa9200b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:33.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.525+0000 7fa9260de640 1 --2- 172.21.15.172:0/3874753685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa92013df30 0x7fa9200b1cd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097d0 tx=0x7fa91002b8a0 comp rx=0 tx=0).stop 2024-04-14T08:34:33.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.525+0000 7fa9260de640 1 -- 172.21.15.172:0/3874753685 shutdown_connections 2024-04-14T08:34:33.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.525+0000 7fa9260de640 1 --2- 172.21.15.172:0/3874753685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa92015c380 0x7fa9200b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:33.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.525+0000 7fa9260de640 1 --2- 172.21.15.172:0/3874753685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa92013df30 0x7fa9200b1cd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:33.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.525+0000 7fa9260de640 1 -- 172.21.15.172:0/3874753685 >> 172.21.15.172:0/3874753685 conn(0x7fa92006b520 msgr2=0x7fa92006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:33.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.525+0000 7fa9260de640 1 -- 172.21.15.172:0/3874753685 shutdown_connections 2024-04-14T08:34:33.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.525+0000 7fa9260de640 1 -- 172.21.15.172:0/3874753685 wait complete. 2024-04-14T08:34:33.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.527+0000 7fa9260de640 1 Processor -- start 2024-04-14T08:34:33.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.527+0000 7fa9260de640 1 -- start start 2024-04-14T08:34:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.527+0000 7fa9260de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa92013df30 0x7fa920159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.527+0000 7fa9260de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa920140770 0x7fa920159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.527+0000 7fa9260de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa92015c380 0x7fa920154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.527+0000 7fa9260de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa92013e310 con 0x7fa920140770 2024-04-14T08:34:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.527+0000 7fa9260de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa92015be60 con 0x7fa92015c380 2024-04-14T08:34:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.527+0000 7fa9260de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa9200ce9a0 con 0x7fa92013df30 2024-04-14T08:34:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa91effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa920140770 0x7fa920159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa91effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa920140770 0x7fa920159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48058/0 (socket says 172.21.15.172:48058) 2024-04-14T08:34:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa91effd640 1 -- 172.21.15.172:0/578014002 learned_addr learned my addr 172.21.15.172:0/578014002 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa91f7fe640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa92013df30 0x7fa920159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa91ffff640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa92015c380 0x7fa920154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa91f7fe640 1 -- 172.21.15.172:0/578014002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa92015c380 msgr2=0x7fa920154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa91f7fe640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa92015c380 0x7fa920154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa91f7fe640 1 -- 172.21.15.172:0/578014002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa920140770 msgr2=0x7fa920159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa91f7fe640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa920140770 0x7fa920159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa91f7fe640 1 -- 172.21.15.172:0/578014002 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa920137dd0 con 0x7fa92013df30 2024-04-14T08:34:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa91effd640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa920140770 0x7fa920159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:34:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa91f7fe640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa92013df30 0x7fa920159820 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa910033000 tx=0x7fa9100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa907fff640 1 -- 172.21.15.172:0/578014002 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910041070 con 0x7fa92013df30 2024-04-14T08:34:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa907fff640 1 -- 172.21.15.172:0/578014002 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa91003ad00 con 0x7fa92013df30 2024-04-14T08:34:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa9260de640 1 -- 172.21.15.172:0/578014002 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9201380b0 con 0x7fa92013df30 2024-04-14T08:34:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.528+0000 7fa907fff640 1 -- 172.21.15.172:0/578014002 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910035040 con 0x7fa92013df30 2024-04-14T08:34:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.529+0000 7fa9260de640 1 -- 172.21.15.172:0/578014002 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa920138670 con 0x7fa92013df30 2024-04-14T08:34:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.529+0000 7fa9260de640 1 -- 172.21.15.172:0/578014002 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa920068590 con 0x7fa92013df30 2024-04-14T08:34:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.530+0000 7fa907fff640 1 -- 172.21.15.172:0/578014002 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa91004c060 con 0x7fa92013df30 2024-04-14T08:34:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.530+0000 7fa907fff640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa90c06c8f0 0x7fa90c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.530+0000 7fa907fff640 1 -- 172.21.15.172:0/578014002 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(47..47 src has 1..47) ==== 5619+0+0 (secure 0 0 0) 0x7fa9100ad400 con 0x7fa92013df30 2024-04-14T08:34:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.530+0000 7fa91effd640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa90c06c8f0 0x7fa90c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.531+0000 7fa91effd640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa90c06c8f0 0x7fa90c06edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa920155ba0 tx=0x7fa914009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.535+0000 7fa907fff640 1 -- 172.21.15.172:0/578014002 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa91007b240 con 0x7fa92013df30 2024-04-14T08:34:33.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:33.710+0000 7fa9260de640 1 -- 172.21.15.172:0/578014002 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa9200b38b0 con 0x7fa92013df30 2024-04-14T08:34:34.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.404+0000 7fa907fff640 1 -- 172.21.15.172:0/578014002 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v48) ==== 170+0+0 (secure 0 0 0) 0x7fa91007f540 con 0x7fa92013df30 2024-04-14T08:34:34.406 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' created 2024-04-14T08:34:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.414+0000 7fa9260de640 1 -- 172.21.15.172:0/578014002 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa90c06c8f0 msgr2=0x7fa90c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.414+0000 7fa9260de640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa90c06c8f0 0x7fa90c06edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa920155ba0 tx=0x7fa914009190 comp rx=0 tx=0).stop 2024-04-14T08:34:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.414+0000 7fa9260de640 1 -- 172.21.15.172:0/578014002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa92013df30 msgr2=0x7fa920159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.414+0000 7fa9260de640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa92013df30 0x7fa920159820 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa910033000 tx=0x7fa9100046c0 comp rx=0 tx=0).stop 2024-04-14T08:34:34.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.414+0000 7fa9260de640 1 -- 172.21.15.172:0/578014002 shutdown_connections 2024-04-14T08:34:34.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.414+0000 7fa9260de640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa90c06c8f0 0x7fa90c06edb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:34.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.414+0000 7fa9260de640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa92015c380 0x7fa920154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:34.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.414+0000 7fa9260de640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa920140770 0x7fa920159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:34.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.414+0000 7fa9260de640 1 --2- 172.21.15.172:0/578014002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa92013df30 0x7fa920159820 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:34.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.414+0000 7fa9260de640 1 -- 172.21.15.172:0/578014002 >> 172.21.15.172:0/578014002 conn(0x7fa92006b520 msgr2=0x7fa9200b5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:34.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.415+0000 7fa9260de640 1 -- 172.21.15.172:0/578014002 shutdown_connections 2024-04-14T08:34:34.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.416+0000 7fa9260de640 1 -- 172.21.15.172:0/578014002 wait complete. 2024-04-14T08:34:34.434 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-14T08:34:34.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.538+0000 7f99741fe640 1 Processor -- start 2024-04-14T08:34:34.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.538+0000 7f99741fe640 1 -- start start 2024-04-14T08:34:34.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f99741fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f996c13df30 0x7f996c0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:34.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f99741fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f996c15c380 0x7f996c0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:34.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f99741fe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f996c15b620 con 0x7f996c140770 2024-04-14T08:34:34.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f99741fe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f996c0cf9b0 con 0x7f996c15c380 2024-04-14T08:34:34.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f99741fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f996c1414a0 con 0x7f996c13df30 2024-04-14T08:34:34.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f9971799640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f996c15c380 0x7f996c0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:34.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f997279b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f996c140770 0x7f996c140c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40080/0 (socket says 172.21.15.172:40080) 2024-04-14T08:34:34.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f997279b640 1 -- 172.21.15.172:0/1881955491 learned_addr learned my addr 172.21.15.172:0/1881955491 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:34.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f9971799640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f996c15c380 0x7f996c0b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48064/0 (socket says 172.21.15.172:48064) 2024-04-14T08:34:34.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f9971f9a640 1 --2- 172.21.15.172:0/1881955491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f996c13df30 0x7f996c0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:34.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f9971799640 1 -- 172.21.15.172:0/1881955491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f996c13df30 msgr2=0x7f996c0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:34.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f9971799640 1 --2- 172.21.15.172:0/1881955491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f996c13df30 0x7f996c0b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:34.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f9971799640 1 -- 172.21.15.172:0/1881955491 >> v1:172.21.15.172:6789/0 conn(0x7f996c140770 legacy=0x7f996c140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:34.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f9971799640 1 -- 172.21.15.172:0/1881955491 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c15b8a0 con 0x7f996c15c380 2024-04-14T08:34:34.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.539+0000 7f9970f98640 1 -- 172.21.15.172:0/1881955491 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1556878761 0 0) 0x7f9964001490 con 0x7f996c140770 2024-04-14T08:34:34.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.540+0000 7f9971799640 1 --2- 172.21.15.172:0/1881955491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f996c15c380 0x7f996c0b2210 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f995c0097d0 tx=0x7f995c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49314e20e33cca37 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:34.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.540+0000 7f9970f98640 1 -- 172.21.15.172:0/1881955491 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c035020 con 0x7f996c15c380 2024-04-14T08:34:34.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.540+0000 7f9970f98640 1 -- 172.21.15.172:0/1881955491 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c031800 con 0x7f996c15c380 2024-04-14T08:34:34.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.540+0000 7f9970f98640 1 -- 172.21.15.172:0/1881955491 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c031bd0 con 0x7f996c15c380 2024-04-14T08:34:34.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.541+0000 7f99741fe640 1 -- 172.21.15.172:0/1881955491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f996c15c380 msgr2=0x7f996c0b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:34.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.541+0000 7f99741fe640 1 --2- 172.21.15.172:0/1881955491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f996c15c380 0x7f996c0b2210 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f995c0097d0 tx=0x7f995c02b8a0 comp rx=0 tx=0).stop 2024-04-14T08:34:34.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.541+0000 7f99741fe640 1 -- 172.21.15.172:0/1881955491 shutdown_connections 2024-04-14T08:34:34.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.541+0000 7f99741fe640 1 --2- 172.21.15.172:0/1881955491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f996c15c380 0x7f996c0b2210 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:34.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.541+0000 7f99741fe640 1 --2- 172.21.15.172:0/1881955491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f996c13df30 0x7f996c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:34.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.541+0000 7f99741fe640 1 -- 172.21.15.172:0/1881955491 >> 172.21.15.172:0/1881955491 conn(0x7f996c06b520 msgr2=0x7f996c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:34.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.541+0000 7f99741fe640 1 -- 172.21.15.172:0/1881955491 shutdown_connections 2024-04-14T08:34:34.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.541+0000 7f99741fe640 1 -- 172.21.15.172:0/1881955491 wait complete. 2024-04-14T08:34:34.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.542+0000 7f99741fe640 1 Processor -- start 2024-04-14T08:34:34.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.542+0000 7f99741fe640 1 -- start start 2024-04-14T08:34:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f99741fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f996c13df30 0x7f996c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f99741fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f996c140770 0x7f996c1377f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f99741fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f996c15c380 0x7f996c13c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f99741fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f996c13e310 con 0x7f996c140770 2024-04-14T08:34:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f99741fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f996c15be60 con 0x7f996c13df30 2024-04-14T08:34:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f99741fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f996c0ce9a0 con 0x7f996c15c380 2024-04-14T08:34:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f9971799640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f996c140770 0x7f996c1377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f9971799640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f996c140770 0x7f996c1377f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48070/0 (socket says 172.21.15.172:48070) 2024-04-14T08:34:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f9971799640 1 -- 172.21.15.172:0/1813613721 learned_addr learned my addr 172.21.15.172:0/1813613721 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f997279b640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f996c15c380 0x7f996c13c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:34.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f9971f9a640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f996c13df30 0x7f996c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f9971799640 1 -- 172.21.15.172:0/1813613721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f996c15c380 msgr2=0x7f996c13c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f9971799640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f996c15c380 0x7f996c13c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f9971799640 1 -- 172.21.15.172:0/1813613721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f996c13df30 msgr2=0x7f996c15ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f9971799640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f996c13df30 0x7f996c15ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f9971799640 1 -- 172.21.15.172:0/1813613721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c156770 con 0x7f996c140770 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f9971799640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f996c140770 0x7f996c1377f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f995c034040 tx=0x7f995c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f99627fc640 1 -- 172.21.15.172:0/1813613721 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c040070 con 0x7f996c140770 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f99627fc640 1 -- 172.21.15.172:0/1813613721 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c039cf0 con 0x7f996c140770 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.543+0000 7f99627fc640 1 -- 172.21.15.172:0/1813613721 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c035040 con 0x7f996c140770 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.544+0000 7f99741fe640 1 -- 172.21.15.172:0/1813613721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996c156a10 con 0x7f996c140770 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.544+0000 7f99741fe640 1 -- 172.21.15.172:0/1813613721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996c137d30 con 0x7f996c140770 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.545+0000 7f99741fe640 1 -- 172.21.15.172:0/1813613721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f996c068590 con 0x7f996c140770 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.545+0000 7f99627fc640 1 -- 172.21.15.172:0/1813613721 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f995c038020 con 0x7f996c140770 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.545+0000 7f99627fc640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f995006c8f0 0x7f995006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.546+0000 7f9971f9a640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f995006c8f0 0x7f995006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:34.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.546+0000 7f99627fc640 1 -- 172.21.15.172:0/1813613721 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6001+0+0 (secure 0 0 0) 0x7f995c0ad7f0 con 0x7f996c140770 2024-04-14T08:34:34.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.546+0000 7f9971f9a640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f995006c8f0 0x7f995006edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9968006fd0 tx=0x7f9968008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:34.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.550+0000 7f99627fc640 1 -- 172.21.15.172:0/1813613721 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f995c07b380 con 0x7f996c140770 2024-04-14T08:34:34.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:34.762+0000 7f99741fe640 1 -- 172.21.15.172:0/1813613721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f996c0b38b0 con 0x7f996c140770 2024-04-14T08:34:35.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.421+0000 7f99627fc640 1 -- 172.21.15.172:0/1813613721 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v21) ==== 172+0+0 (secure 0 0 0) 0x7f995c07f680 con 0x7f996c140770 2024-04-14T08:34:35.424 INFO:teuthology.orchestra.run.smithi172.stderr:new fs with metadata pool 7 and data pool 8 2024-04-14T08:34:35.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.433+0000 7f99741fe640 1 -- 172.21.15.172:0/1813613721 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f995006c8f0 msgr2=0x7f995006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:35.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.433+0000 7f99741fe640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f995006c8f0 0x7f995006edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9968006fd0 tx=0x7f9968008040 comp rx=0 tx=0).stop 2024-04-14T08:34:35.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.433+0000 7f99741fe640 1 -- 172.21.15.172:0/1813613721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f996c140770 msgr2=0x7f996c1377f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:35.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.433+0000 7f99741fe640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f996c140770 0x7f996c1377f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f995c034040 tx=0x7f995c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.433+0000 7f99741fe640 1 -- 172.21.15.172:0/1813613721 shutdown_connections 2024-04-14T08:34:35.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.433+0000 7f99741fe640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f995006c8f0 0x7f995006edb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.433+0000 7f99741fe640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f996c15c380 0x7f996c13c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.433+0000 7f99741fe640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f996c140770 0x7f996c1377f0 secure :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f995c034040 tx=0x7f995c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.433+0000 7f99741fe640 1 --2- 172.21.15.172:0/1813613721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f996c13df30 0x7f996c15ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.433+0000 7f99741fe640 1 -- 172.21.15.172:0/1813613721 >> 172.21.15.172:0/1813613721 conn(0x7f996c06b520 msgr2=0x7f996c0711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:35.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.437+0000 7f99741fe640 1 -- 172.21.15.172:0/1813613721 shutdown_connections 2024-04-14T08:34:35.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.437+0000 7f99741fe640 1 -- 172.21.15.172:0/1813613721 wait complete. 2024-04-14T08:34:35.456 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:34:35.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.556+0000 7f09693c8640 1 Processor -- start 2024-04-14T08:34:35.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.556+0000 7f09693c8640 1 -- start start 2024-04-14T08:34:35.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09693c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f096415e390 0x7f09640d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09693c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f09640b4430 0x7f096406c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09693c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f096415e770 con 0x7f096415f3a0 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09693c8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f09640d0080 con 0x7f09640b4430 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09693c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f096406dba0 con 0x7f096415e390 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09637fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f09640b4430 0x7f096406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09637fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f09640b4430 0x7f096406c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48072/0 (socket says 172.21.15.172:48072) 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09637fe640 1 -- 172.21.15.172:0/2074605608 learned_addr learned my addr 172.21.15.172:0/2074605608 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09627fc640 1 --1- 172.21.15.172:0/2074605608 >> v1:172.21.15.172:6789/0 conn(0x7f096415f3a0 0x7f09640b3d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40094/0 (socket says 172.21.15.172:40094) 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f0962ffd640 1 --2- 172.21.15.172:0/2074605608 >> v2:172.21.15.177:3300/0 conn(0x7f096415e390 0x7f09640d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09637fe640 1 -- 172.21.15.172:0/2074605608 >> v2:172.21.15.177:3300/0 conn(0x7f096415e390 msgr2=0x7f09640d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09637fe640 1 --2- 172.21.15.172:0/2074605608 >> v2:172.21.15.177:3300/0 conn(0x7f096415e390 0x7f09640d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09637fe640 1 -- 172.21.15.172:0/2074605608 >> v1:172.21.15.172:6789/0 conn(0x7f096415f3a0 legacy=0x7f09640b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f0961ffb640 1 -- 172.21.15.172:0/2074605608 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2120844015 0 0) 0x7f096415e770 con 0x7f096415f3a0 2024-04-14T08:34:35.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.557+0000 7f09637fe640 1 -- 172.21.15.172:0/2074605608 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096406de80 con 0x7f09640b4430 2024-04-14T08:34:35.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.558+0000 7f09637fe640 1 --2- 172.21.15.172:0/2074605608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09640b4430 0x7f096406c630 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0964070690 tx=0x7f095c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07fc11c64c33091 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:35.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.558+0000 7f0961ffb640 1 -- 172.21.15.172:0/2074605608 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c02ca80 con 0x7f09640b4430 2024-04-14T08:34:35.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.558+0000 7f0961ffb640 1 -- 172.21.15.172:0/2074605608 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c02cc20 con 0x7f09640b4430 2024-04-14T08:34:35.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.558+0000 7f0961ffb640 1 -- 172.21.15.172:0/2074605608 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c0343f0 con 0x7f09640b4430 2024-04-14T08:34:35.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.559+0000 7f09693c8640 1 -- 172.21.15.172:0/2074605608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09640b4430 msgr2=0x7f096406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.559+0000 7f09693c8640 1 --2- 172.21.15.172:0/2074605608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09640b4430 0x7f096406c630 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0964070690 tx=0x7f095c02bee0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.559+0000 7f09693c8640 1 -- 172.21.15.172:0/2074605608 shutdown_connections 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.559+0000 7f09693c8640 1 --2- 172.21.15.172:0/2074605608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09640b4430 0x7f096406c630 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.559+0000 7f09693c8640 1 --2- 172.21.15.172:0/2074605608 >> v2:172.21.15.177:3300/0 conn(0x7f096415e390 0x7f09640d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.559+0000 7f09693c8640 1 -- 172.21.15.172:0/2074605608 >> 172.21.15.172:0/2074605608 conn(0x7f096406f820 msgr2=0x7f096406fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.559+0000 7f09693c8640 1 -- 172.21.15.172:0/2074605608 shutdown_connections 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.559+0000 7f09693c8640 1 -- 172.21.15.172:0/2074605608 wait complete. 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.560+0000 7f09693c8640 1 Processor -- start 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.560+0000 7f09693c8640 1 -- start start 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09693c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09640b4430 0x7f09640b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09693c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096415e390 0x7f09640b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09693c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096415f3a0 0x7f09640c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:35.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09693c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f096406cb70 con 0x7f096415f3a0 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09693c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f09640d0510 con 0x7f09640b4430 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09693c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f096406ccf0 con 0x7f096415e390 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09637fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096415f3a0 0x7f09640c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09637fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096415f3a0 0x7f09640c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48082/0 (socket says 172.21.15.172:48082) 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09637fe640 1 -- 172.21.15.172:0/653714603 learned_addr learned my addr 172.21.15.172:0/653714603 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09637fe640 1 -- 172.21.15.172:0/653714603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096415e390 msgr2=0x7f09640b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f0962ffd640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09640b4430 0x7f09640b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09637fe640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096415e390 0x7f09640b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09637fe640 1 -- 172.21.15.172:0/653714603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09640b4430 msgr2=0x7f09640b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09637fe640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09640b4430 0x7f09640b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09637fe640 1 -- 172.21.15.172:0/653714603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09640b6a30 con 0x7f096415f3a0 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f09637fe640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096415f3a0 0x7f09640c7be0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f095c009f40 tx=0x7f095c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.561+0000 7f094b7fe640 1 -- 172.21.15.172:0/653714603 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c043020 con 0x7f096415f3a0 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.562+0000 7f094b7fe640 1 -- 172.21.15.172:0/653714603 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c03e450 con 0x7f096415f3a0 2024-04-14T08:34:35.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.562+0000 7f094b7fe640 1 -- 172.21.15.172:0/653714603 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c009510 con 0x7f096415f3a0 2024-04-14T08:34:35.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.562+0000 7f09693c8640 1 -- 172.21.15.172:0/653714603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09640b6d10 con 0x7f096415f3a0 2024-04-14T08:34:35.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.562+0000 7f09693c8640 1 -- 172.21.15.172:0/653714603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09640b72d0 con 0x7f096415f3a0 2024-04-14T08:34:35.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.563+0000 7f09693c8640 1 -- 172.21.15.172:0/653714603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0964068520 con 0x7f096415f3a0 2024-04-14T08:34:35.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.563+0000 7f094b7fe640 1 -- 172.21.15.172:0/653714603 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f095c046020 con 0x7f096415f3a0 2024-04-14T08:34:35.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.563+0000 7f094b7fe640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f095006c8f0 0x7f095006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:35.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.564+0000 7f094b7fe640 1 -- 172.21.15.172:0/653714603 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f095c0ae580 con 0x7f096415f3a0 2024-04-14T08:34:35.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.564+0000 7f0962ffd640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f095006c8f0 0x7f095006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:35.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.564+0000 7f0962ffd640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f095006c8f0 0x7f095006edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0954006fd0 tx=0x7f0954008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:35.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.568+0000 7f094b7fe640 1 -- 172.21.15.172:0/653714603 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f095c04d030 con 0x7f096415f3a0 2024-04-14T08:34:35.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.745+0000 7f09693c8640 1 -- 172.21.15.172:0/653714603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f096406d980 con 0x7f096415f3a0 2024-04-14T08:34:35.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.746+0000 7f094b7fe640 1 -- 172.21.15.172:0/653714603 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+17015 (secure 0 0 0) 0x7f095c07c790 con 0x7f096415f3a0 2024-04-14T08:34:35.748 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:35.748 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":49,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:34.764746+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:34:32.990827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-14T08:34:33.715803+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:34:35.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.748+0000 7f09693c8640 1 -- 172.21.15.172:0/653714603 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f095006c8f0 msgr2=0x7f095006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:35.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.748+0000 7f09693c8640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f095006c8f0 0x7f095006edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0954006fd0 tx=0x7f0954008040 comp rx=0 tx=0).stop 2024-04-14T08:34:35.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.748+0000 7f09693c8640 1 -- 172.21.15.172:0/653714603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096415f3a0 msgr2=0x7f09640c7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:35.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.748+0000 7f09693c8640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096415f3a0 0x7f09640c7be0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f095c009f40 tx=0x7f095c004a40 comp rx=0 tx=0).stop 2024-04-14T08:34:35.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.748+0000 7f09693c8640 1 -- 172.21.15.172:0/653714603 shutdown_connections 2024-04-14T08:34:35.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.748+0000 7f09693c8640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f095006c8f0 0x7f095006edb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.748+0000 7f09693c8640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096415f3a0 0x7f09640c7be0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.748+0000 7f09693c8640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096415e390 0x7f09640b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.748+0000 7f09693c8640 1 --2- 172.21.15.172:0/653714603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09640b4430 0x7f09640b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.748+0000 7f09693c8640 1 -- 172.21.15.172:0/653714603 >> 172.21.15.172:0/653714603 conn(0x7f096406f820 msgr2=0x7f09640c6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:35.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.749+0000 7f09693c8640 1 -- 172.21.15.172:0/653714603 shutdown_connections 2024-04-14T08:34:35.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.749+0000 7f09693c8640 1 -- 172.21.15.172:0/653714603 wait complete. 2024-04-14T08:34:35.761 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:34:35.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.861+0000 7f50f3a36640 1 Processor -- start 2024-04-14T08:34:35.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.861+0000 7f50f3a36640 1 -- start start 2024-04-14T08:34:35.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f3a36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f50ec0ca920 0x7f50ec0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:35.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f3a36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50ec0ce3e0 0x7f50ec0d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:35.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f3a36640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50ec0ad100 con 0x7f50ec15c340 2024-04-14T08:34:35.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f3a36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f50ec0cfb90 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f3a36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f50ec0cd570 con 0x7f50ec0ca920 2024-04-14T08:34:35.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f1fd3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50ec0ce3e0 0x7f50ec0d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:35.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f1fd3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50ec0ce3e0 0x7f50ec0d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48084/0 (socket says 172.21.15.172:48084) 2024-04-14T08:34:35.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f1fd3640 1 -- 172.21.15.172:0/1776177862 learned_addr learned my addr 172.21.15.172:0/1776177862 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:35.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f17d2640 1 --1- 172.21.15.172:0/1776177862 >> v1:172.21.15.172:6789/0 conn(0x7f50ec15c340 0x7f50ec0ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40108/0 (socket says 172.21.15.172:40108) 2024-04-14T08:34:35.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f0fd1640 1 --2- 172.21.15.172:0/1776177862 >> v2:172.21.15.177:3300/0 conn(0x7f50ec0ca920 0x7f50ec0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:35.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f1fd3640 1 -- 172.21.15.172:0/1776177862 >> v2:172.21.15.177:3300/0 conn(0x7f50ec0ca920 msgr2=0x7f50ec0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:35.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f1fd3640 1 --2- 172.21.15.172:0/1776177862 >> v2:172.21.15.177:3300/0 conn(0x7f50ec0ca920 0x7f50ec0cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f1fd3640 1 -- 172.21.15.172:0/1776177862 >> v1:172.21.15.172:6789/0 conn(0x7f50ec15c340 legacy=0x7f50ec0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:35.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50e3fff640 1 -- 172.21.15.172:0/1776177862 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3910790192 0 0) 0x7f50ec0ad100 con 0x7f50ec15c340 2024-04-14T08:34:35.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.862+0000 7f50f1fd3640 1 -- 172.21.15.172:0/1776177862 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec0cd850 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.863+0000 7f50f1fd3640 1 --2- 172.21.15.172:0/1776177862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50ec0ce3e0 0x7f50ec0d0020 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f50ec072ab0 tx=0x7f50e402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fc4134784c165a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:35.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.863+0000 7f50e3fff640 1 -- 172.21.15.172:0/1776177862 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e402cb20 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.863+0000 7f50e3fff640 1 -- 172.21.15.172:0/1776177862 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e402ccc0 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.863+0000 7f50e3fff640 1 -- 172.21.15.172:0/1776177862 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4034610 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.863+0000 7f50f3a36640 1 -- 172.21.15.172:0/1776177862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50ec0ce3e0 msgr2=0x7f50ec0d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:35.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.863+0000 7f50f3a36640 1 --2- 172.21.15.172:0/1776177862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50ec0ce3e0 0x7f50ec0d0020 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f50ec072ab0 tx=0x7f50e402bf80 comp rx=0 tx=0).stop 2024-04-14T08:34:35.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.863+0000 7f50f3a36640 1 -- 172.21.15.172:0/1776177862 shutdown_connections 2024-04-14T08:34:35.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.863+0000 7f50f3a36640 1 --2- 172.21.15.172:0/1776177862 >> v2:172.21.15.177:3300/0 conn(0x7f50ec0ca920 0x7f50ec0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.863+0000 7f50f3a36640 1 --2- 172.21.15.172:0/1776177862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50ec0ce3e0 0x7f50ec0d0020 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.863+0000 7f50f3a36640 1 -- 172.21.15.172:0/1776177862 >> 172.21.15.172:0/1776177862 conn(0x7f50ec072b20 msgr2=0x7f50ec072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:35.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.864+0000 7f50f3a36640 1 -- 172.21.15.172:0/1776177862 shutdown_connections 2024-04-14T08:34:35.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.864+0000 7f50f3a36640 1 -- 172.21.15.172:0/1776177862 wait complete. 2024-04-14T08:34:35.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.865+0000 7f50f3a36640 1 Processor -- start 2024-04-14T08:34:35.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.865+0000 7f50f3a36640 1 -- start start 2024-04-14T08:34:35.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.865+0000 7f50f3a36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec0ca920 0x7f50ec06df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.865+0000 7f50f3a36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50ec0ce3e0 0x7f50ec0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.865+0000 7f50f3a36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec15c340 0x7f50ec1f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.865+0000 7f50f3a36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec0d06e0 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.865+0000 7f50f3a36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec0d0560 con 0x7f50ec0ca920 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.865+0000 7f50f3a36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f50ec0d0860 con 0x7f50ec15c340 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f0fd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50ec0ce3e0 0x7f50ec0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f0fd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50ec0ce3e0 0x7f50ec0b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48094/0 (socket says 172.21.15.172:48094) 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f0fd1640 1 -- 172.21.15.172:0/244213469 learned_addr learned my addr 172.21.15.172:0/244213469 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f17d2640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec0ca920 0x7f50ec06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f1fd3640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec15c340 0x7f50ec1f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f0fd1640 1 -- 172.21.15.172:0/244213469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec15c340 msgr2=0x7f50ec1f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f0fd1640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec15c340 0x7f50ec1f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f0fd1640 1 -- 172.21.15.172:0/244213469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec0ca920 msgr2=0x7f50ec06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f0fd1640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec0ca920 0x7f50ec06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f0fd1640 1 -- 172.21.15.172:0/244213469 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec0c94a0 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f0fd1640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50ec0ce3e0 0x7f50ec0b51e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f50e800c7f0 tx=0x7f50e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50e1ffb640 1 -- 172.21.15.172:0/244213469 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8015070 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50e1ffb640 1 -- 172.21.15.172:0/244213469 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e8004240 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f3a36640 1 -- 172.21.15.172:0/244213469 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ec0c9780 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50e1ffb640 1 -- 172.21.15.172:0/244213469 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8007620 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.866+0000 7f50f3a36640 1 -- 172.21.15.172:0/244213469 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ec0b4cc0 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.867+0000 7f50f3a36640 1 -- 172.21.15.172:0/244213469 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ec0b6250 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.868+0000 7f50e1ffb640 1 -- 172.21.15.172:0/244213469 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50e80079d0 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.868+0000 7f50e1ffb640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f50d006c8f0 0x7f50d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.868+0000 7f50f17d2640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f50d006c8f0 0x7f50d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.868+0000 7f50e1ffb640 1 -- 172.21.15.172:0/244213469 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7f50e808c030 con 0x7f50ec0ce3e0 2024-04-14T08:34:35.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.869+0000 7f50f17d2640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f50d006c8f0 0x7f50d006edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f50dc007fd0 tx=0x7f50dc007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:35.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:35.873+0000 7f50e1ffb640 1 -- 172.21.15.172:0/244213469 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f50e8059d10 con 0x7f50ec0ce3e0 2024-04-14T08:34:36.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.049+0000 7f50f3a36640 1 -- 172.21.15.172:0/244213469 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50ec0d0c20 con 0x7f50ec0ce3e0 2024-04-14T08:34:36.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.050+0000 7f50e1ffb640 1 -- 172.21.15.172:0/244213469 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+17015 (secure 0 0 0) 0x7f50e805e010 con 0x7f50ec0ce3e0 2024-04-14T08:34:36.051 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:36.051 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":49,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:34.764746+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:34:32.990827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-14T08:34:33.715803+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:34:36.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.052+0000 7f50f3a36640 1 -- 172.21.15.172:0/244213469 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f50d006c8f0 msgr2=0x7f50d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:36.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.052+0000 7f50f3a36640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f50d006c8f0 0x7f50d006edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f50dc007fd0 tx=0x7f50dc007de0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.052+0000 7f50f3a36640 1 -- 172.21.15.172:0/244213469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50ec0ce3e0 msgr2=0x7f50ec0b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:36.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.052+0000 7f50f3a36640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50ec0ce3e0 0x7f50ec0b51e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f50e800c7f0 tx=0x7f50e800ccb0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.052+0000 7f50f3a36640 1 -- 172.21.15.172:0/244213469 shutdown_connections 2024-04-14T08:34:36.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.052+0000 7f50f3a36640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f50d006c8f0 0x7f50d006edb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.052+0000 7f50f3a36640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50ec15c340 0x7f50ec1f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.052+0000 7f50f3a36640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50ec0ce3e0 0x7f50ec0b51e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.052+0000 7f50f3a36640 1 --2- 172.21.15.172:0/244213469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec0ca920 0x7f50ec06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.052+0000 7f50f3a36640 1 -- 172.21.15.172:0/244213469 >> 172.21.15.172:0/244213469 conn(0x7f50ec072b20 msgr2=0x7f50ec0c3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:36.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.052+0000 7f50f3a36640 1 -- 172.21.15.172:0/244213469 shutdown_connections 2024-04-14T08:34:36.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.052+0000 7f50f3a36640 1 -- 172.21.15.172:0/244213469 wait complete. 2024-04-14T08:34:36.065 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-14T08:34:36.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.167+0000 7fe9978de640 1 Processor -- start 2024-04-14T08:34:36.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.168+0000 7fe9978de640 1 -- start start 2024-04-14T08:34:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.168+0000 7fe9978de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9900b7a00 0x7fe9900d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.168+0000 7fe9978de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe9900cf6f0 0x7fe9900ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.168+0000 7fe9978de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe9900ceea0 0x7fe99015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.168+0000 7fe9978de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe9900d1690 con 0x7fe9900ceea0 2024-04-14T08:34:36.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.168+0000 7fe9978de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe9900cd0e0 con 0x7fe9900cf6f0 2024-04-14T08:34:36.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.168+0000 7fe9978de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe9900cc7b0 con 0x7fe9900b7a00 2024-04-14T08:34:36.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.169+0000 7fe995e7b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe9900ceea0 0x7fe99015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:36.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.169+0000 7fe995e7b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe9900ceea0 0x7fe99015dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48110/0 (socket says 172.21.15.172:48110) 2024-04-14T08:34:36.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.169+0000 7fe995e7b640 1 -- 172.21.15.172:0/923078585 learned_addr learned my addr 172.21.15.172:0/923078585 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:36.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.169+0000 7fe995e7b640 1 -- 172.21.15.172:0/923078585 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9900b7a00 msgr2=0x7fe9900d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:36.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.169+0000 7fe995e7b640 1 --2- 172.21.15.172:0/923078585 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9900b7a00 0x7fe9900d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.169+0000 7fe994e79640 1 --2- 172.21.15.172:0/923078585 >> v2:172.21.15.177:3300/0 conn(0x7fe9900cf6f0 0x7fe9900ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:36.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.169+0000 7fe995e7b640 1 -- 172.21.15.172:0/923078585 >> v2:172.21.15.177:3300/0 conn(0x7fe9900cf6f0 msgr2=0x7fe9900ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:36.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.169+0000 7fe995e7b640 1 --2- 172.21.15.172:0/923078585 >> v2:172.21.15.177:3300/0 conn(0x7fe9900cf6f0 0x7fe9900ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.169+0000 7fe995e7b640 1 -- 172.21.15.172:0/923078585 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9900cca90 con 0x7fe9900ceea0 2024-04-14T08:34:36.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe995e7b640 1 --2- 172.21.15.172:0/923078585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9900ceea0 0x7fe99015dca0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe99006ed80 tx=0x7fe98002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cee6ce95ea75753 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:36.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe98ffff640 1 -- 172.21.15.172:0/923078585 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98002e070 con 0x7fe9900ceea0 2024-04-14T08:34:36.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe98ffff640 1 -- 172.21.15.172:0/923078585 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe980037070 con 0x7fe9900ceea0 2024-04-14T08:34:36.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe98ffff640 1 -- 172.21.15.172:0/923078585 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980031a00 con 0x7fe9900ceea0 2024-04-14T08:34:36.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe9978de640 1 -- 172.21.15.172:0/923078585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9900ceea0 msgr2=0x7fe99015dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:36.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe9978de640 1 --2- 172.21.15.172:0/923078585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9900ceea0 0x7fe99015dca0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe99006ed80 tx=0x7fe98002b880 comp rx=0 tx=0).stop 2024-04-14T08:34:36.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe9978de640 1 -- 172.21.15.172:0/923078585 shutdown_connections 2024-04-14T08:34:36.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe9978de640 1 --2- 172.21.15.172:0/923078585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9900ceea0 0x7fe99015dca0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe9978de640 1 --2- 172.21.15.172:0/923078585 >> v2:172.21.15.177:3300/0 conn(0x7fe9900cf6f0 0x7fe9900ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe9978de640 1 --2- 172.21.15.172:0/923078585 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9900b7a00 0x7fe9900d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe9978de640 1 -- 172.21.15.172:0/923078585 >> 172.21.15.172:0/923078585 conn(0x7fe9900d2030 msgr2=0x7fe9900d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:36.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe9978de640 1 -- 172.21.15.172:0/923078585 shutdown_connections 2024-04-14T08:34:36.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.170+0000 7fe9978de640 1 -- 172.21.15.172:0/923078585 wait complete. 2024-04-14T08:34:36.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.171+0000 7fe9978de640 1 Processor -- start 2024-04-14T08:34:36.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.172+0000 7fe9978de640 1 -- start start 2024-04-14T08:34:36.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.172+0000 7fe9978de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9900b7a00 0x7fe99009ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.172+0000 7fe9978de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9900ceea0 0x7fe99009f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.172+0000 7fe9978de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9900cf6f0 0x7fe990150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.172+0000 7fe9978de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9900d12c0 con 0x7fe9900b7a00 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.172+0000 7fe9978de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe9900b7de0 con 0x7fe9900cf6f0 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.172+0000 7fe9978de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe9900d1440 con 0x7fe9900ceea0 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.172+0000 7fe99567a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9900b7a00 0x7fe99009ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.172+0000 7fe99567a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9900b7a00 0x7fe99009ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48126/0 (socket says 172.21.15.172:48126) 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.172+0000 7fe99567a640 1 -- 172.21.15.172:0/2828476171 learned_addr learned my addr 172.21.15.172:0/2828476171 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe994e79640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9900ceea0 0x7fe99009f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe995e7b640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9900cf6f0 0x7fe990150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe99567a640 1 -- 172.21.15.172:0/2828476171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9900ceea0 msgr2=0x7fe99009f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe99567a640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9900ceea0 0x7fe99009f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe99567a640 1 -- 172.21.15.172:0/2828476171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9900cf6f0 msgr2=0x7fe990150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe99567a640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9900cf6f0 0x7fe990150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe99567a640 1 -- 172.21.15.172:0/2828476171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe990160ab0 con 0x7fe9900b7a00 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe99567a640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9900b7a00 0x7fe99009ef00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe98800ba50 tx=0x7fe98800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe98dffb640 1 -- 172.21.15.172:0/2828476171 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988019070 con 0x7fe9900b7a00 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe98dffb640 1 -- 172.21.15.172:0/2828476171 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe98800cc00 con 0x7fe9900b7a00 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe98dffb640 1 -- 172.21.15.172:0/2828476171 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988014440 con 0x7fe9900b7a00 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe9978de640 1 -- 172.21.15.172:0/2828476171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9901e97f0 con 0x7fe9900b7a00 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.173+0000 7fe9978de640 1 -- 172.21.15.172:0/2828476171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9901e7100 con 0x7fe9900b7a00 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.174+0000 7fe9978de640 1 -- 172.21.15.172:0/2828476171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9900b6c70 con 0x7fe9900b7a00 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.174+0000 7fe98dffb640 1 -- 172.21.15.172:0/2828476171 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe98801c070 con 0x7fe9900b7a00 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.175+0000 7fe98dffb640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe97c06c8f0 0x7fe97c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.175+0000 7fe98dffb640 1 -- 172.21.15.172:0/2828476171 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6101+0+0 (secure 0 0 0) 0x7fe98808b9e0 con 0x7fe9900b7a00 2024-04-14T08:34:36.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.175+0000 7fe994e79640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe97c06c8f0 0x7fe97c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:36.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.175+0000 7fe994e79640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe97c06c8f0 0x7fe97c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe990160ca0 tx=0x7fe984008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:36.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.179+0000 7fe98dffb640 1 -- 172.21.15.172:0/2828476171 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe9900b6c70 con 0x7fe9900b7a00 2024-04-14T08:34:36.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.391+0000 7fe9978de640 1 -- 172.21.15.172:0/2828476171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe990075490 con 0x7fe9900b7a00 2024-04-14T08:34:36.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.429+0000 7fe98dffb640 1 -- 172.21.15.172:0/2828476171 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7fe988059d10 con 0x7fe9900b7a00 2024-04-14T08:34:36.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.433+0000 7fe9978de640 1 -- 172.21.15.172:0/2828476171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe97c06c8f0 msgr2=0x7fe97c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:36.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.433+0000 7fe9978de640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe97c06c8f0 0x7fe97c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe990160ca0 tx=0x7fe984008040 comp rx=0 tx=0).stop 2024-04-14T08:34:36.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.433+0000 7fe9978de640 1 -- 172.21.15.172:0/2828476171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9900b7a00 msgr2=0x7fe99009ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:36.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.433+0000 7fe9978de640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9900b7a00 0x7fe99009ef00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe98800ba50 tx=0x7fe98800bf10 comp rx=0 tx=0).stop 2024-04-14T08:34:36.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.433+0000 7fe9978de640 1 -- 172.21.15.172:0/2828476171 shutdown_connections 2024-04-14T08:34:36.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.433+0000 7fe9978de640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe97c06c8f0 0x7fe97c06edb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.433+0000 7fe9978de640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9900cf6f0 0x7fe990150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.433+0000 7fe9978de640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9900ceea0 0x7fe99009f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.433+0000 7fe9978de640 1 --2- 172.21.15.172:0/2828476171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9900b7a00 0x7fe99009ef00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.433+0000 7fe9978de640 1 -- 172.21.15.172:0/2828476171 >> 172.21.15.172:0/2828476171 conn(0x7fe9900d2030 msgr2=0x7fe9900b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:36.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.433+0000 7fe9978de640 1 -- 172.21.15.172:0/2828476171 shutdown_connections 2024-04-14T08:34:36.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.433+0000 7fe9978de640 1 -- 172.21.15.172:0/2828476171 wait complete. 2024-04-14T08:34:36.446 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-14T08:34:36.446 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-14T08:34:36.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.548+0000 7f631be36640 1 Processor -- start 2024-04-14T08:34:36.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.549+0000 7f631be36640 1 -- start start 2024-04-14T08:34:36.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.549+0000 7f631be36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f63140b65a0 0x7f63141629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.549+0000 7f631be36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63140cea60 0x7f63140d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.549+0000 7f631be36640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63140b3750 con 0x7f631415e590 2024-04-14T08:34:36.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.549+0000 7f631be36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f63140cef20 con 0x7f63140b65a0 2024-04-14T08:34:36.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.549+0000 7f631be36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f63140cf200 con 0x7f63140cea60 2024-04-14T08:34:36.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.550+0000 7f631a3d3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63140cea60 0x7f63140d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:36.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.550+0000 7f63193d1640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f63140b65a0 0x7f63141629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:36.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.550+0000 7f631a3d3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63140cea60 0x7f63140d2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:34882/0 (socket says 172.21.15.172:34882) 2024-04-14T08:34:36.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.550+0000 7f6319bd2640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f631415e590 0x7f63140b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:46480/0 (socket says 172.21.15.172:46480) 2024-04-14T08:34:36.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.550+0000 7f631a3d3640 1 -- 172.21.15.172:0/3309225774 learned_addr learned my addr 172.21.15.172:0/3309225774 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:36.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.550+0000 7f631a3d3640 1 -- 172.21.15.172:0/3309225774 >> v2:172.21.15.177:3300/0 conn(0x7f63140b65a0 msgr2=0x7f63141629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:36.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.550+0000 7f631a3d3640 1 --2- 172.21.15.172:0/3309225774 >> v2:172.21.15.177:3300/0 conn(0x7f63140b65a0 0x7f63141629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.550+0000 7f631a3d3640 1 -- 172.21.15.172:0/3309225774 >> v1:172.21.15.177:6789/0 conn(0x7f631415e590 legacy=0x7f63140b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:36.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.550+0000 7f631a3d3640 1 -- 172.21.15.172:0/3309225774 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63140cf4e0 con 0x7f63140cea60 2024-04-14T08:34:36.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.551+0000 7f631a3d3640 1 --2- 172.21.15.172:0/3309225774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63140cea60 0x7f63140d2760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f630c009e50 tx=0x7f630c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d5a14c3b4abc36c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:36.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.551+0000 7f6318bd0640 1 -- 172.21.15.172:0/3309225774 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c02cb00 con 0x7f63140cea60 2024-04-14T08:34:36.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.551+0000 7f6318bd0640 1 -- 172.21.15.172:0/3309225774 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c02cca0 con 0x7f63140cea60 2024-04-14T08:34:36.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.551+0000 7f6318bd0640 1 -- 172.21.15.172:0/3309225774 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c034610 con 0x7f63140cea60 2024-04-14T08:34:36.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.551+0000 7f631be36640 1 -- 172.21.15.172:0/3309225774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63140cea60 msgr2=0x7f63140d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:36.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.551+0000 7f631be36640 1 --2- 172.21.15.172:0/3309225774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63140cea60 0x7f63140d2760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f630c009e50 tx=0x7f630c02bf60 comp rx=0 tx=0).stop 2024-04-14T08:34:36.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.551+0000 7f631be36640 1 -- 172.21.15.172:0/3309225774 shutdown_connections 2024-04-14T08:34:36.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.551+0000 7f631be36640 1 --2- 172.21.15.172:0/3309225774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63140cea60 0x7f63140d2760 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.551+0000 7f631be36640 1 --2- 172.21.15.172:0/3309225774 >> v2:172.21.15.177:3300/0 conn(0x7f63140b65a0 0x7f63141629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.551+0000 7f631be36640 1 -- 172.21.15.172:0/3309225774 >> 172.21.15.172:0/3309225774 conn(0x7f63140d0e90 msgr2=0x7f63140d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:36.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.552+0000 7f631be36640 1 -- 172.21.15.172:0/3309225774 shutdown_connections 2024-04-14T08:34:36.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.552+0000 7f631be36640 1 -- 172.21.15.172:0/3309225774 wait complete. 2024-04-14T08:34:36.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.553+0000 7f631be36640 1 Processor -- start 2024-04-14T08:34:36.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.553+0000 7f631be36640 1 -- start start 2024-04-14T08:34:36.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.553+0000 7f631be36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63140b65a0 0x7f6314153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.553+0000 7f631be36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63140cea60 0x7f631414a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.553+0000 7f631be36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631415e590 0x7f6314155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.553+0000 7f631be36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f63140d2e20 con 0x7f631415e590 2024-04-14T08:34:36.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.553+0000 7f631be36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f63140d2ca0 con 0x7f63140b65a0 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.553+0000 7f631be36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f63140d2fa0 con 0x7f63140cea60 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.553+0000 7f631a3d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631415e590 0x7f6314155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f631a3d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631415e590 0x7f6314155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48128/0 (socket says 172.21.15.172:48128) 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f631a3d3640 1 -- 172.21.15.172:0/979378169 learned_addr learned my addr 172.21.15.172:0/979378169 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f63193d1640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63140cea60 0x7f631414a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f6319bd2640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63140b65a0 0x7f6314153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f631a3d3640 1 -- 172.21.15.172:0/979378169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63140cea60 msgr2=0x7f631414a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f631a3d3640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63140cea60 0x7f631414a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f631a3d3640 1 -- 172.21.15.172:0/979378169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63140b65a0 msgr2=0x7f6314153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f631a3d3640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63140b65a0 0x7f6314153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f631a3d3640 1 -- 172.21.15.172:0/979378169 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63141546c0 con 0x7f631415e590 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f631a3d3640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631415e590 0x7f6314155d50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f630c009e50 tx=0x7f630c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f630a7fc640 1 -- 172.21.15.172:0/979378169 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c043020 con 0x7f631415e590 2024-04-14T08:34:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f630a7fc640 1 -- 172.21.15.172:0/979378169 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c02ce50 con 0x7f631415e590 2024-04-14T08:34:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f631be36640 1 -- 172.21.15.172:0/979378169 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6314156bd0 con 0x7f631415e590 2024-04-14T08:34:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f630a7fc640 1 -- 172.21.15.172:0/979378169 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c009510 con 0x7f631415e590 2024-04-14T08:34:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.554+0000 7f631be36640 1 -- 172.21.15.172:0/979378169 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631414ab30 con 0x7f631415e590 2024-04-14T08:34:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.555+0000 7f631be36640 1 -- 172.21.15.172:0/979378169 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6314068520 con 0x7f631415e590 2024-04-14T08:34:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.556+0000 7f630a7fc640 1 -- 172.21.15.172:0/979378169 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f630c005d10 con 0x7f631415e590 2024-04-14T08:34:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.557+0000 7f630a7fc640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f62f806c920 0x7f62f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.557+0000 7f6319bd2640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f62f806c920 0x7f62f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.557+0000 7f630a7fc640 1 -- 172.21.15.172:0/979378169 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f630c0ae3e0 con 0x7f631415e590 2024-04-14T08:34:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.557+0000 7f6319bd2640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f62f806c920 0x7f62f806ede0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6310007fd0 tx=0x7f6310007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:36.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.561+0000 7f630a7fc640 1 -- 172.21.15.172:0/979378169 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f630c07c030 con 0x7f631415e590 2024-04-14T08:34:36.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:36.781+0000 7f631be36640 1 -- 172.21.15.172:0/979378169 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f63140d1bb0 con 0x7f631415e590 2024-04-14T08:34:37.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.436+0000 7f630a7fc640 1 -- 172.21.15.172:0/979378169 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) ==== 122+0+0 (secure 0 0 0) 0x7f630c080330 con 0x7f631415e590 2024-04-14T08:34:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.438+0000 7f631be36640 1 -- 172.21.15.172:0/979378169 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f62f806c920 msgr2=0x7f62f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.438+0000 7f631be36640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f62f806c920 0x7f62f806ede0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6310007fd0 tx=0x7f6310007e80 comp rx=0 tx=0).stop 2024-04-14T08:34:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.438+0000 7f631be36640 1 -- 172.21.15.172:0/979378169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631415e590 msgr2=0x7f6314155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.438+0000 7f631be36640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631415e590 0x7f6314155d50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f630c009e50 tx=0x7f630c004a40 comp rx=0 tx=0).stop 2024-04-14T08:34:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.438+0000 7f631be36640 1 -- 172.21.15.172:0/979378169 shutdown_connections 2024-04-14T08:34:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.438+0000 7f631be36640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f62f806c920 0x7f62f806ede0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.438+0000 7f631be36640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631415e590 0x7f6314155d50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.438+0000 7f631be36640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63140cea60 0x7f631414a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.438+0000 7f631be36640 1 --2- 172.21.15.172:0/979378169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63140b65a0 0x7f6314153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.438+0000 7f631be36640 1 -- 172.21.15.172:0/979378169 >> 172.21.15.172:0/979378169 conn(0x7f63140d0e90 msgr2=0x7f63140b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:37.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.438+0000 7f631be36640 1 -- 172.21.15.172:0/979378169 shutdown_connections 2024-04-14T08:34:37.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.438+0000 7f631be36640 1 -- 172.21.15.172:0/979378169 wait complete. 2024-04-14T08:34:37.454 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-14T08:34:37.454 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:34:37.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.553+0000 7f4931f88640 1 Processor -- start 2024-04-14T08:34:37.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.554+0000 7f4931f88640 1 -- start start 2024-04-14T08:34:37.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.554+0000 7f4931f88640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f492c15e640 0x7f492c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.554+0000 7f4931f88640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f492c06bc30 0x7f492c162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.554+0000 7f4931f88640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f492c0d1af0 con 0x7f492c0b0630 2024-04-14T08:34:37.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.554+0000 7f4931f88640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f492c15ea20 con 0x7f492c06bc30 2024-04-14T08:34:37.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.554+0000 7f4931f88640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f492c0c7840 con 0x7f492c15e640 2024-04-14T08:34:37.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.555+0000 7f492bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f492c06bc30 0x7f492c162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.555+0000 7f492bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f492c06bc30 0x7f492c162530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48140/0 (socket says 172.21.15.172:48140) 2024-04-14T08:34:37.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.555+0000 7f492bfff640 1 -- 172.21.15.172:0/4293481544 learned_addr learned my addr 172.21.15.172:0/4293481544 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:37.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.555+0000 7f492b7fe640 1 --2- 172.21.15.172:0/4293481544 >> v2:172.21.15.177:3300/0 conn(0x7f492c15e640 0x7f492c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.555+0000 7f492bfff640 1 -- 172.21.15.172:0/4293481544 >> v2:172.21.15.177:3300/0 conn(0x7f492c15e640 msgr2=0x7f492c0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.555+0000 7f492bfff640 1 --2- 172.21.15.172:0/4293481544 >> v2:172.21.15.177:3300/0 conn(0x7f492c15e640 0x7f492c0cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.555+0000 7f492bfff640 1 -- 172.21.15.172:0/4293481544 >> v1:172.21.15.177:6789/0 conn(0x7f492c0b0630 legacy=0x7f492c06b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.555+0000 7f492bfff640 1 -- 172.21.15.172:0/4293481544 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492c0c7b20 con 0x7f492c06bc30 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.555+0000 7f492b7fe640 1 --2- 172.21.15.172:0/4293481544 >> v2:172.21.15.177:3300/0 conn(0x7f492c15e640 0x7f492c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.556+0000 7f492bfff640 1 --2- 172.21.15.172:0/4293481544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492c06bc30 0x7f492c162530 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f492c074130 tx=0x7f492402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa9a739c5a0d805 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.556+0000 7f492affd640 1 -- 172.21.15.172:0/4293481544 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492402ca80 con 0x7f492c06bc30 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.556+0000 7f492affd640 1 -- 172.21.15.172:0/4293481544 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492402cc20 con 0x7f492c06bc30 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.556+0000 7f492affd640 1 -- 172.21.15.172:0/4293481544 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49240343f0 con 0x7f492c06bc30 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.556+0000 7f4931f88640 1 -- 172.21.15.172:0/4293481544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492c06bc30 msgr2=0x7f492c162530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.556+0000 7f4931f88640 1 --2- 172.21.15.172:0/4293481544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492c06bc30 0x7f492c162530 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f492c074130 tx=0x7f492402bee0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.556+0000 7f4931f88640 1 -- 172.21.15.172:0/4293481544 shutdown_connections 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.556+0000 7f4931f88640 1 --2- 172.21.15.172:0/4293481544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492c06bc30 0x7f492c162530 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.556+0000 7f4931f88640 1 --2- 172.21.15.172:0/4293481544 >> v2:172.21.15.177:3300/0 conn(0x7f492c15e640 0x7f492c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.556+0000 7f4931f88640 1 -- 172.21.15.172:0/4293481544 >> 172.21.15.172:0/4293481544 conn(0x7f492c0d0f30 msgr2=0x7f492c0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:37.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.556+0000 7f4931f88640 1 -- 172.21.15.172:0/4293481544 shutdown_connections 2024-04-14T08:34:37.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.556+0000 7f4931f88640 1 -- 172.21.15.172:0/4293481544 wait complete. 2024-04-14T08:34:37.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.558+0000 7f4931f88640 1 Processor -- start 2024-04-14T08:34:37.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.558+0000 7f4931f88640 1 -- start start 2024-04-14T08:34:37.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f4931f88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f492c06bc30 0x7f492c1f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f4931f88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f492c0b0630 0x7f492c0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f4931f88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492c15e640 0x7f492c0b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f4931f88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f492c06c1e0 con 0x7f492c15e640 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f4931f88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f492c0d1dd0 con 0x7f492c0b0630 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f4931f88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f492c0c5db0 con 0x7f492c06bc30 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f492bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492c15e640 0x7f492c0b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f492bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492c15e640 0x7f492c0b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48150/0 (socket says 172.21.15.172:48150) 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f492bfff640 1 -- 172.21.15.172:0/650394583 learned_addr learned my addr 172.21.15.172:0/650394583 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f492b7fe640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f492c06bc30 0x7f492c1f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f4923fff640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f492c0b0630 0x7f492c0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f492bfff640 1 -- 172.21.15.172:0/650394583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f492c06bc30 msgr2=0x7f492c1f3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f492bfff640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f492c06bc30 0x7f492c1f3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f492bfff640 1 -- 172.21.15.172:0/650394583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f492c0b0630 msgr2=0x7f492c0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f492bfff640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f492c0b0630 0x7f492c0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f492bfff640 1 -- 172.21.15.172:0/650394583 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492c1f4090 con 0x7f492c15e640 2024-04-14T08:34:37.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.559+0000 7f4923fff640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f492c0b0630 0x7f492c0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:37.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.560+0000 7f492bfff640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492c15e640 0x7f492c0b1ed0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4924009f40 tx=0x7f4924004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:37.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.560+0000 7f4928ff9640 1 -- 172.21.15.172:0/650394583 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4924043020 con 0x7f492c15e640 2024-04-14T08:34:37.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.560+0000 7f4928ff9640 1 -- 172.21.15.172:0/650394583 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492403e450 con 0x7f492c15e640 2024-04-14T08:34:37.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.560+0000 7f4928ff9640 1 -- 172.21.15.172:0/650394583 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4924009510 con 0x7f492c15e640 2024-04-14T08:34:37.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.560+0000 7f4931f88640 1 -- 172.21.15.172:0/650394583 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492c0c9950 con 0x7f492c15e640 2024-04-14T08:34:37.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.560+0000 7f4931f88640 1 -- 172.21.15.172:0/650394583 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f492c0c9f10 con 0x7f492c15e640 2024-04-14T08:34:37.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.561+0000 7f4931f88640 1 -- 172.21.15.172:0/650394583 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f492c075490 con 0x7f492c15e640 2024-04-14T08:34:37.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.561+0000 7f4928ff9640 1 -- 172.21.15.172:0/650394583 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4924046020 con 0x7f492c15e640 2024-04-14T08:34:37.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.562+0000 7f4928ff9640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f490c06c8f0 0x7f490c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.562+0000 7f492b7fe640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f490c06c8f0 0x7f490c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.562+0000 7f4928ff9640 1 -- 172.21.15.172:0/650394583 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f49240ae530 con 0x7f492c15e640 2024-04-14T08:34:37.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.562+0000 7f492b7fe640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f490c06c8f0 0x7f490c06edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f491c006fd0 tx=0x7f491c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:37.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.566+0000 7f4928ff9640 1 -- 172.21.15.172:0/650394583 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f492404d030 con 0x7f492c15e640 2024-04-14T08:34:37.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.782+0000 7f4931f88640 1 -- 172.21.15.172:0/650394583 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f492c0ae230 con 0x7f492c15e640 2024-04-14T08:34:37.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.782+0000 7f4928ff9640 1 -- 172.21.15.172:0/650394583 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5187 (secure 0 0 0) 0x7f492c1f4090 con 0x7f492c15e640 2024-04-14T08:34:37.784 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:37.784 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/296861286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":296861286},{"type":"v1","addr":"172.21.15.177:6835","nonce":296861286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/642342704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":642342704},{"type":"v1","addr":"172.21.15.177:6833","nonce":642342704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3516903987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3516903987},{"type":"v1","addr":"172.21.15.172:6836","nonce":3516903987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:34:34.765057+0000","modified":"2024-04-14T08:34:36.783999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6837/1527574817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1527574817},{"type":"v1","addr":"172.21.15.172:6837","nonce":1527574817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":3}]} 2024-04-14T08:34:37.784 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 23 2024-04-14T08:34:37.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.784+0000 7f4931f88640 1 -- 172.21.15.172:0/650394583 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f490c06c8f0 msgr2=0x7f490c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.784+0000 7f4931f88640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f490c06c8f0 0x7f490c06edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f491c006fd0 tx=0x7f491c008040 comp rx=0 tx=0).stop 2024-04-14T08:34:37.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.784+0000 7f4931f88640 1 -- 172.21.15.172:0/650394583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492c15e640 msgr2=0x7f492c0b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.784+0000 7f4931f88640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492c15e640 0x7f492c0b1ed0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4924009f40 tx=0x7f4924004a40 comp rx=0 tx=0).stop 2024-04-14T08:34:37.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.784+0000 7f4931f88640 1 -- 172.21.15.172:0/650394583 shutdown_connections 2024-04-14T08:34:37.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.784+0000 7f4931f88640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f490c06c8f0 0x7f490c06edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.784+0000 7f4931f88640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f492c15e640 0x7f492c0b1ed0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.784+0000 7f4931f88640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f492c0b0630 0x7f492c0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.784+0000 7f4931f88640 1 --2- 172.21.15.172:0/650394583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f492c06bc30 0x7f492c1f3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.784+0000 7f4931f88640 1 -- 172.21.15.172:0/650394583 >> 172.21.15.172:0/650394583 conn(0x7f492c0d0f30 msgr2=0x7f492c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:37.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.784+0000 7f4931f88640 1 -- 172.21.15.172:0/650394583 shutdown_connections 2024-04-14T08:34:37.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.784+0000 7f4931f88640 1 -- 172.21.15.172:0/650394583 wait complete. 2024-04-14T08:34:37.797 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:34:37.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.897+0000 7ff70fa2d640 1 Processor -- start 2024-04-14T08:34:37.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.898+0000 7ff70fa2d640 1 -- start start 2024-04-14T08:34:37.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff70fa2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7080cedd0 0x7ff7081694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff70fa2d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7080cf320 con 0x7ff70815e890 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff70fa2d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7080cf600 con 0x7ff7080d2830 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff70fa2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff7080cff30 con 0x7ff7080cedd0 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff70cfc8640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff70815e890 0x7ff7080d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:40110/0 (socket says 172.21.15.172:40110) 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff70cfc8640 1 -- 172.21.15.172:0/448145497 learned_addr learned my addr 172.21.15.172:0/448145497 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff70dfca640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7080cedd0 0x7ff7081694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1726357255 0 0) 0x7ff7080cf320 con 0x7ff70815e890 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6f4003520 con 0x7ff70815e890 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407109313 0 0) 0x7ff7080cf600 con 0x7ff7080d2830 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7080cf320 con 0x7ff7080d2830 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1220227546 0 0) 0x7ff6f4003520 con 0x7ff70815e890 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7080cedd0 msgr2=0x7ff7081694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff6fffff640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7080cedd0 0x7ff7081694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.899+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 >> v1:172.21.15.177:6789/0 conn(0x7ff7080d2830 legacy=0x7ff7080d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7080cf8e0 con 0x7ff70815e890 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7ff704001670 con 0x7ff70815e890 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff70dfca640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7080cedd0 0x7ff7081694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 >> v1:172.21.15.172:6789/0 conn(0x7ff70815e890 legacy=0x7ff7080d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff6fffff640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6f4003b70 0x7ff6f4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff6fffff640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7080cedd0 0x7ff6f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff6fffff640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6f4004d40 0x7ff6f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7080cff30 con 0x7ff6f4004d40 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f400ba70 con 0x7ff6f4003b70 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f400bbf0 con 0x7ff7080cedd0 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff70dfca640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6f4004d40 0x7ff6f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff70d7c9640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6f4003b70 0x7ff6f4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff70cfc8640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7080cedd0 0x7ff6f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff70dfca640 1 -- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7080cedd0 msgr2=0x7ff6f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff70dfca640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7080cedd0 0x7ff6f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff70dfca640 1 -- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6f4003b70 msgr2=0x7ff6f4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff70dfca640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6f4003b70 0x7ff6f4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff70dfca640 1 -- 172.21.15.172:0/448145497 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f4003520 con 0x7ff6f4004d40 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.900+0000 7ff70cfc8640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7080cedd0 0x7ff6f4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.901+0000 7ff70dfca640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6f4004d40 0x7ff6f40051c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff70000e380 tx=0x7ff6f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd1f05062ac6dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.901+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700039070 con 0x7ff6f4004d40 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.901+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff70002cd70 con 0x7ff6f4004d40 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.901+0000 7ff6fffff640 1 -- 172.21.15.172:0/448145497 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700009de0 con 0x7ff6f4004d40 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.902+0000 7ff70fa2d640 1 -- 172.21.15.172:0/448145497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6f4004d40 msgr2=0x7ff6f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.902+0000 7ff70fa2d640 1 --2- 172.21.15.172:0/448145497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6f4004d40 0x7ff6f40051c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff70000e380 tx=0x7ff6f4005830 comp rx=0 tx=0).stop 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.902+0000 7ff70dfca640 1 -- 172.21.15.172:0/448145497 reap_dead start 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.902+0000 7ff70fa2d640 1 -- 172.21.15.172:0/448145497 shutdown_connections 2024-04-14T08:34:37.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.902+0000 7ff70fa2d640 1 -- 172.21.15.172:0/448145497 >> 172.21.15.172:0/448145497 conn(0x7ff70809d590 msgr2=0x7ff7080d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:37.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.902+0000 7ff70fa2d640 1 -- 172.21.15.172:0/448145497 shutdown_connections 2024-04-14T08:34:37.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.902+0000 7ff70fa2d640 1 -- 172.21.15.172:0/448145497 wait complete. 2024-04-14T08:34:37.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.903+0000 7ff70fa2d640 1 Processor -- start 2024-04-14T08:34:37.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.903+0000 7ff70fa2d640 1 -- start start 2024-04-14T08:34:37.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70fa2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6f4004d40 0x7ff7080b8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70fa2d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff70815f9a0 0x7ff7080d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70fa2d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7080d0b00 0x7ff7081f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70fa2d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7080b4720 con 0x7ff7080d0b00 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70fa2d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff7080d2350 con 0x7ff70815f9a0 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70fa2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff7080b48a0 con 0x7ff6f4004d40 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70dfca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7080d0b00 0x7ff7081f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70dfca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7080d0b00 0x7ff7081f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48164/0 (socket says 172.21.15.172:48164) 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70dfca640 1 -- 172.21.15.172:0/1874563584 learned_addr learned my addr 172.21.15.172:0/1874563584 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70d7c9640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6f4004d40 0x7ff7080b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70cfc8640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff70815f9a0 0x7ff7080d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70dfca640 1 -- 172.21.15.172:0/1874563584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6f4004d40 msgr2=0x7ff7080b8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70dfca640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6f4004d40 0x7ff7080b8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70dfca640 1 -- 172.21.15.172:0/1874563584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff70815f9a0 msgr2=0x7ff7080d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70dfca640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff70815f9a0 0x7ff7080d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70dfca640 1 -- 172.21.15.172:0/1874563584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7081f5160 con 0x7ff7080d0b00 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70dfca640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7080d0b00 0x7ff7081f43f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff700037040 tx=0x7ff700009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:37.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff6fdffb640 1 -- 172.21.15.172:0/1874563584 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700043020 con 0x7ff7080d0b00 2024-04-14T08:34:37.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff6fdffb640 1 -- 172.21.15.172:0/1874563584 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff70003a040 con 0x7ff7080d0b00 2024-04-14T08:34:37.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff70fa2d640 1 -- 172.21.15.172:0/1874563584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70815eec0 con 0x7ff7080d0b00 2024-04-14T08:34:37.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.904+0000 7ff6fdffb640 1 -- 172.21.15.172:0/1874563584 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700039070 con 0x7ff7080d0b00 2024-04-14T08:34:37.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.905+0000 7ff70fa2d640 1 -- 172.21.15.172:0/1874563584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70815f480 con 0x7ff7080d0b00 2024-04-14T08:34:37.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.905+0000 7ff70fa2d640 1 -- 172.21.15.172:0/1874563584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff708068520 con 0x7ff7080d0b00 2024-04-14T08:34:37.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.906+0000 7ff6fdffb640 1 -- 172.21.15.172:0/1874563584 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff700046070 con 0x7ff7080d0b00 2024-04-14T08:34:37.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.906+0000 7ff6fdffb640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff6ec06c8f0 0x7ff6ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:37.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.906+0000 7ff6fdffb640 1 -- 172.21.15.172:0/1874563584 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7ff7000ad670 con 0x7ff7080d0b00 2024-04-14T08:34:37.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.906+0000 7ff70d7c9640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff6ec06c8f0 0x7ff6ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:37.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.907+0000 7ff70d7c9640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff6ec06c8f0 0x7ff6ec06edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff7080b32c0 tx=0x7ff6f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:37.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:37.911+0000 7ff6fdffb640 1 -- 172.21.15.172:0/1874563584 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff70007b230 con 0x7ff7080d0b00 2024-04-14T08:34:38.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.093+0000 7ff70fa2d640 1 -- 172.21.15.172:0/1874563584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff70815f650 con 0x7ff7080d0b00 2024-04-14T08:34:38.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.094+0000 7ff6fdffb640 1 -- 172.21.15.172:0/1874563584 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+17002 (secure 0 0 0) 0x7ff70007f530 con 0x7ff7080d0b00 2024-04-14T08:34:38.096 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:38.096 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":50,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:36.423005+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:34:32.990827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-14T08:34:33.715803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:34:38.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.096+0000 7ff70fa2d640 1 -- 172.21.15.172:0/1874563584 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff6ec06c8f0 msgr2=0x7ff6ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.096+0000 7ff70fa2d640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff6ec06c8f0 0x7ff6ec06edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff7080b32c0 tx=0x7ff6f8009040 comp rx=0 tx=0).stop 2024-04-14T08:34:38.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.096+0000 7ff70fa2d640 1 -- 172.21.15.172:0/1874563584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7080d0b00 msgr2=0x7ff7081f43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.096+0000 7ff70fa2d640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7080d0b00 0x7ff7081f43f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff700037040 tx=0x7ff700009750 comp rx=0 tx=0).stop 2024-04-14T08:34:38.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.096+0000 7ff70fa2d640 1 -- 172.21.15.172:0/1874563584 shutdown_connections 2024-04-14T08:34:38.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.096+0000 7ff70fa2d640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff6ec06c8f0 0x7ff6ec06edb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.096+0000 7ff70fa2d640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7080d0b00 0x7ff7081f43f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.096+0000 7ff70fa2d640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff70815f9a0 0x7ff7080d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.096+0000 7ff70fa2d640 1 --2- 172.21.15.172:0/1874563584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6f4004d40 0x7ff7080b8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.096+0000 7ff70fa2d640 1 -- 172.21.15.172:0/1874563584 >> 172.21.15.172:0/1874563584 conn(0x7ff70809d590 msgr2=0x7ff70815cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:38.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.096+0000 7ff70fa2d640 1 -- 172.21.15.172:0/1874563584 shutdown_connections 2024-04-14T08:34:38.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.096+0000 7ff70fa2d640 1 -- 172.21.15.172:0/1874563584 wait complete. 2024-04-14T08:34:38.109 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-14T08:34:38.110 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:34:38.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.210+0000 7f15461f0640 1 Processor -- start 2024-04-14T08:34:38.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.210+0000 7f15461f0640 1 -- start start 2024-04-14T08:34:38.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f15461f0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f15400729b0 0x7f154006dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f15461f0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f154006e160 0x7f154006c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f15461f0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15400b41d0 con 0x7f154015e5b0 2024-04-14T08:34:38.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f15461f0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f15400aebe0 con 0x7f154006e160 2024-04-14T08:34:38.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f15461f0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f15400b4470 con 0x7f15400729b0 2024-04-14T08:34:38.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f153ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f154006e160 0x7f154006c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f153ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f154006e160 0x7f154006c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48178/0 (socket says 172.21.15.172:48178) 2024-04-14T08:34:38.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f153ffff640 1 -- 172.21.15.172:0/4055684694 learned_addr learned my addr 172.21.15.172:0/4055684694 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:38.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f1537fff640 1 --2- 172.21.15.172:0/4055684694 >> v2:172.21.15.177:3300/0 conn(0x7f15400729b0 0x7f154006dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f153ffff640 1 -- 172.21.15.172:0/4055684694 >> v2:172.21.15.177:3300/0 conn(0x7f15400729b0 msgr2=0x7f154006dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:38.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f153ffff640 1 --2- 172.21.15.172:0/4055684694 >> v2:172.21.15.177:3300/0 conn(0x7f15400729b0 0x7f154006dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f153ffff640 1 -- 172.21.15.172:0/4055684694 >> v1:172.21.15.177:6789/0 conn(0x7f154015e5b0 legacy=0x7f15400ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.211+0000 7f153ffff640 1 -- 172.21.15.172:0/4055684694 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15400aeec0 con 0x7f154006e160 2024-04-14T08:34:38.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.212+0000 7f153ffff640 1 --2- 172.21.15.172:0/4055684694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f154006e160 0x7f154006c530 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1538009e50 tx=0x7f153802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c47706e3b2f6a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:38.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.212+0000 7f153effd640 1 -- 172.21.15.172:0/4055684694 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153802cb20 con 0x7f154006e160 2024-04-14T08:34:38.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.212+0000 7f153effd640 1 -- 172.21.15.172:0/4055684694 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153802ccc0 con 0x7f154006e160 2024-04-14T08:34:38.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.212+0000 7f153effd640 1 -- 172.21.15.172:0/4055684694 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1538034610 con 0x7f154006e160 2024-04-14T08:34:38.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.213+0000 7f15461f0640 1 -- 172.21.15.172:0/4055684694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f154006e160 msgr2=0x7f154006c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.213+0000 7f15461f0640 1 --2- 172.21.15.172:0/4055684694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f154006e160 0x7f154006c530 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1538009e50 tx=0x7f153802bf80 comp rx=0 tx=0).stop 2024-04-14T08:34:38.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.213+0000 7f15461f0640 1 -- 172.21.15.172:0/4055684694 shutdown_connections 2024-04-14T08:34:38.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.213+0000 7f15461f0640 1 --2- 172.21.15.172:0/4055684694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f154006e160 0x7f154006c530 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.213+0000 7f15461f0640 1 --2- 172.21.15.172:0/4055684694 >> v2:172.21.15.177:3300/0 conn(0x7f15400729b0 0x7f154006dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.213+0000 7f15461f0640 1 -- 172.21.15.172:0/4055684694 >> 172.21.15.172:0/4055684694 conn(0x7f1540071ba0 msgr2=0x7f1540071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:38.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.213+0000 7f15461f0640 1 -- 172.21.15.172:0/4055684694 shutdown_connections 2024-04-14T08:34:38.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.213+0000 7f15461f0640 1 -- 172.21.15.172:0/4055684694 wait complete. 2024-04-14T08:34:38.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.214+0000 7f15461f0640 1 Processor -- start 2024-04-14T08:34:38.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.214+0000 7f15461f0640 1 -- start start 2024-04-14T08:34:38.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f15461f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f154006e160 0x7f15400cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f15461f0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15400729b0 0x7f15400cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f15461f0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f154015e5b0 0x7f15400cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f15461f0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f154006ce00 con 0x7f154006e160 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f15461f0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f154006cc80 con 0x7f154015e5b0 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f15461f0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f15400b4c70 con 0x7f15400729b0 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f153f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f154006e160 0x7f15400cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f153f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f154006e160 0x7f15400cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48188/0 (socket says 172.21.15.172:48188) 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f153f7fe640 1 -- 172.21.15.172:0/3954912383 learned_addr learned my addr 172.21.15.172:0/3954912383 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f1537fff640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15400729b0 0x7f15400cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f153ffff640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f154015e5b0 0x7f15400cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f153f7fe640 1 -- 172.21.15.172:0/3954912383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15400729b0 msgr2=0x7f15400cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f153f7fe640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15400729b0 0x7f15400cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f153f7fe640 1 -- 172.21.15.172:0/3954912383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f154015e5b0 msgr2=0x7f15400cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f153f7fe640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f154015e5b0 0x7f15400cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f153f7fe640 1 -- 172.21.15.172:0/3954912383 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154015ebe0 con 0x7f154006e160 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.215+0000 7f153f7fe640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f154006e160 0x7f15400cd860 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f153000c3f0 tx=0x7f153000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:38.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.216+0000 7f153cff9640 1 -- 172.21.15.172:0/3954912383 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15300052d0 con 0x7f154006e160 2024-04-14T08:34:38.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.216+0000 7f153cff9640 1 -- 172.21.15.172:0/3954912383 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1530005490 con 0x7f154006e160 2024-04-14T08:34:38.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.216+0000 7f153cff9640 1 -- 172.21.15.172:0/3954912383 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530007260 con 0x7f154006e160 2024-04-14T08:34:38.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.216+0000 7f15461f0640 1 -- 172.21.15.172:0/3954912383 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154015eec0 con 0x7f154006e160 2024-04-14T08:34:38.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.216+0000 7f15461f0640 1 -- 172.21.15.172:0/3954912383 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f154015f480 con 0x7f154006e160 2024-04-14T08:34:38.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.217+0000 7f15461f0640 1 -- 172.21.15.172:0/3954912383 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1540068520 con 0x7f154006e160 2024-04-14T08:34:38.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.217+0000 7f153cff9640 1 -- 172.21.15.172:0/3954912383 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1530007400 con 0x7f154006e160 2024-04-14T08:34:38.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.218+0000 7f153cff9640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f152006c8f0 0x7f152006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.218+0000 7f153cff9640 1 -- 172.21.15.172:0/3954912383 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f153008b1a0 con 0x7f154006e160 2024-04-14T08:34:38.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.218+0000 7f1537fff640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f152006c8f0 0x7f152006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.218+0000 7f1537fff640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f152006c8f0 0x7f152006edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f15400cceb0 tx=0x7f1528008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:38.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.222+0000 7f153cff9640 1 -- 172.21.15.172:0/3954912383 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1530059440 con 0x7f154006e160 2024-04-14T08:34:38.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.398+0000 7f15461f0640 1 -- 172.21.15.172:0/3954912383 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f154006ed30 con 0x7f152006c8f0 2024-04-14T08:34:38.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.409+0000 7f153cff9640 1 -- 172.21.15.172:0/3954912383 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247296 (secure 0 0 0) 0x7f154006ed30 con 0x7f152006c8f0 2024-04-14T08:34:38.414 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:38.415 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:34:38.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.412+0000 7f15461f0640 1 -- 172.21.15.172:0/3954912383 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f152006c8f0 msgr2=0x7f152006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.412+0000 7f15461f0640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f152006c8f0 0x7f152006edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f15400cceb0 tx=0x7f1528008040 comp rx=0 tx=0).stop 2024-04-14T08:34:38.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.412+0000 7f15461f0640 1 -- 172.21.15.172:0/3954912383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f154006e160 msgr2=0x7f15400cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.412+0000 7f15461f0640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f154006e160 0x7f15400cd860 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f153000c3f0 tx=0x7f153000c8b0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.412+0000 7f15461f0640 1 -- 172.21.15.172:0/3954912383 shutdown_connections 2024-04-14T08:34:38.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.412+0000 7f15461f0640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f152006c8f0 0x7f152006edb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.412+0000 7f15461f0640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f154015e5b0 0x7f15400cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.412+0000 7f15461f0640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15400729b0 0x7f15400cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.412+0000 7f15461f0640 1 --2- 172.21.15.172:0/3954912383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f154006e160 0x7f15400cd860 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.412+0000 7f15461f0640 1 -- 172.21.15.172:0/3954912383 >> 172.21.15.172:0/3954912383 conn(0x7f1540071ba0 msgr2=0x7f154006fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:38.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.412+0000 7f15461f0640 1 -- 172.21.15.172:0/3954912383 shutdown_connections 2024-04-14T08:34:38.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.413+0000 7f15461f0640 1 -- 172.21.15.172:0/3954912383 wait complete. 2024-04-14T08:34:38.425 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-04-14T08:34:37.140761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221808,"kb_used_data":5616,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752912,"statfs":{"total":773094113280,"available":772866981888,"internally_reserved":0,"allocated":5750784,"data_stored":3169616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000673"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408023+0000","last_change":"2024-04-14T08:34:34.408023+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408023+0000","last_undegraded":"2024-04-14T08:34:34.408023+0000","last_fullsized":"2024-04-14T08:34:34.408023+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.441184+0000","last_change":"2024-04-14T08:34:35.434456+0000","last_active":"2024-04-14T08:34:35.441184+0000","last_peered":"2024-04-14T08:34:35.441184+0000","last_clean":"2024-04-14T08:34:35.441184+0000","last_became_active":"2024-04-14T08:34:35.434158+0000","last_became_peered":"2024-04-14T08:34:35.434158+0000","last_unstale":"2024-04-14T08:34:35.441184+0000","last_undegraded":"2024-04-14T08:34:35.441184+0000","last_fullsized":"2024-04-14T08:34:35.441184+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:40:13.833057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414650+0000","last_change":"2024-04-14T08:34:34.414650+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414650+0000","last_undegraded":"2024-04-14T08:34:34.414650+0000","last_fullsized":"2024-04-14T08:34:34.414650+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.437687+0000","last_change":"2024-04-14T08:34:35.430927+0000","last_active":"2024-04-14T08:34:35.437687+0000","last_peered":"2024-04-14T08:34:35.437687+0000","last_clean":"2024-04-14T08:34:35.437687+0000","last_became_active":"2024-04-14T08:34:35.430164+0000","last_became_peered":"2024-04-14T08:34:35.430164+0000","last_unstale":"2024-04-14T08:34:35.437687+0000","last_undegraded":"2024-04-14T08:34:35.437687+0000","last_fullsized":"2024-04-14T08:34:35.437687+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:57:15.656326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405062+0000","last_change":"2024-04-14T08:34:34.405062+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405062+0000","last_undegraded":"2024-04-14T08:34:34.405062+0000","last_fullsized":"2024-04-14T08:34:34.405062+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406804+0000","last_change":"2024-04-14T08:34:34.406804+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406804+0000","last_undegraded":"2024-04-14T08:34:34.406804+0000","last_fullsized":"2024-04-14T08:34:34.406804+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414181+0000","last_change":"2024-04-14T08:34:34.414181+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414181+0000","last_undegraded":"2024-04-14T08:34:34.414181+0000","last_fullsized":"2024-04-14T08:34:34.414181+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409156+0000","last_change":"2024-04-14T08:34:34.409156+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409156+0000","last_undegraded":"2024-04-14T08:34:34.409156+0000","last_fullsized":"2024-04-14T08:34:34.409156+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410003+0000","last_change":"2024-04-14T08:34:34.410003+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410003+0000","last_undegraded":"2024-04-14T08:34:34.410003+0000","last_fullsized":"2024-04-14T08:34:34.410003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406692+0000","last_change":"2024-04-14T08:34:34.406692+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406692+0000","last_undegraded":"2024-04-14T08:34:34.406692+0000","last_fullsized":"2024-04-14T08:34:34.406692+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405927+0000","last_change":"2024-04-14T08:34:34.405927+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405927+0000","last_undegraded":"2024-04-14T08:34:34.405927+0000","last_fullsized":"2024-04-14T08:34:34.405927+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410934+0000","last_change":"2024-04-14T08:34:34.410934+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410934+0000","last_undegraded":"2024-04-14T08:34:34.410934+0000","last_fullsized":"2024-04-14T08:34:34.410934+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406225+0000","last_change":"2024-04-14T08:34:34.406225+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406225+0000","last_undegraded":"2024-04-14T08:34:34.406225+0000","last_fullsized":"2024-04-14T08:34:34.406225+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.411333+0000","last_change":"2024-04-14T08:34:34.411333+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.411333+0000","last_undegraded":"2024-04-14T08:34:34.411333+0000","last_fullsized":"2024-04-14T08:34:34.411333+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.437809+0000","last_change":"2024-04-14T08:34:35.428986+0000","last_active":"2024-04-14T08:34:35.437809+0000","last_peered":"2024-04-14T08:34:35.437809+0000","last_clean":"2024-04-14T08:34:35.437809+0000","last_became_active":"2024-04-14T08:34:35.426840+0000","last_became_peered":"2024-04-14T08:34:35.426840+0000","last_unstale":"2024-04-14T08:34:35.437809+0000","last_undegraded":"2024-04-14T08:34:35.437809+0000","last_fullsized":"2024-04-14T08:34:35.437809+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:55:45.179250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406390+0000","last_change":"2024-04-14T08:34:34.406390+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406390+0000","last_undegraded":"2024-04-14T08:34:34.406390+0000","last_fullsized":"2024-04-14T08:34:34.406390+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414078+0000","last_change":"2024-04-14T08:34:34.414078+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414078+0000","last_undegraded":"2024-04-14T08:34:34.414078+0000","last_fullsized":"2024-04-14T08:34:34.414078+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.441163+0000","last_change":"2024-04-14T08:34:35.434292+0000","last_active":"2024-04-14T08:34:35.441163+0000","last_peered":"2024-04-14T08:34:35.441163+0000","last_clean":"2024-04-14T08:34:35.441163+0000","last_became_active":"2024-04-14T08:34:35.434018+0000","last_became_peered":"2024-04-14T08:34:35.434018+0000","last_unstale":"2024-04-14T08:34:35.441163+0000","last_undegraded":"2024-04-14T08:34:35.441163+0000","last_fullsized":"2024-04-14T08:34:35.441163+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:22:04.896638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.412491+0000","last_change":"2024-04-14T08:34:34.412491+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.412491+0000","last_undegraded":"2024-04-14T08:34:34.412491+0000","last_fullsized":"2024-04-14T08:34:34.412491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410141+0000","last_change":"2024-04-14T08:34:34.410141+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410141+0000","last_undegraded":"2024-04-14T08:34:34.410141+0000","last_fullsized":"2024-04-14T08:34:34.410141+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408219+0000","last_change":"2024-04-14T08:34:34.408219+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408219+0000","last_undegraded":"2024-04-14T08:34:34.408219+0000","last_fullsized":"2024-04-14T08:34:34.408219+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.407965+0000","last_change":"2024-04-14T08:34:34.407965+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.407965+0000","last_undegraded":"2024-04-14T08:34:34.407965+0000","last_fullsized":"2024-04-14T08:34:34.407965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405857+0000","last_change":"2024-04-14T08:34:34.405857+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405857+0000","last_undegraded":"2024-04-14T08:34:34.405857+0000","last_fullsized":"2024-04-14T08:34:34.405857+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410886+0000","last_change":"2024-04-14T08:34:34.410886+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410886+0000","last_undegraded":"2024-04-14T08:34:34.410886+0000","last_fullsized":"2024-04-14T08:34:34.410886+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413719+0000","last_change":"2024-04-14T08:34:34.413719+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413719+0000","last_undegraded":"2024-04-14T08:34:34.413719+0000","last_fullsized":"2024-04-14T08:34:34.413719+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410342+0000","last_change":"2024-04-14T08:34:34.410342+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410342+0000","last_undegraded":"2024-04-14T08:34:34.410342+0000","last_fullsized":"2024-04-14T08:34:34.410342+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406527+0000","last_change":"2024-04-14T08:34:34.406527+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406527+0000","last_undegraded":"2024-04-14T08:34:34.406527+0000","last_fullsized":"2024-04-14T08:34:34.406527+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408765+0000","last_change":"2024-04-14T08:34:34.408765+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408765+0000","last_undegraded":"2024-04-14T08:34:34.408765+0000","last_fullsized":"2024-04-14T08:34:34.408765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409655+0000","last_change":"2024-04-14T08:34:34.409655+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409655+0000","last_undegraded":"2024-04-14T08:34:34.409655+0000","last_fullsized":"2024-04-14T08:34:34.409655+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.438111+0000","last_change":"2024-04-14T08:34:35.431510+0000","last_active":"2024-04-14T08:34:35.438111+0000","last_peered":"2024-04-14T08:34:35.438111+0000","last_clean":"2024-04-14T08:34:35.438111+0000","last_became_active":"2024-04-14T08:34:35.430980+0000","last_became_peered":"2024-04-14T08:34:35.430980+0000","last_unstale":"2024-04-14T08:34:35.438111+0000","last_undegraded":"2024-04-14T08:34:35.438111+0000","last_fullsized":"2024-04-14T08:34:35.438111+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:13:28.555180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408105+0000","last_change":"2024-04-14T08:34:34.408105+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408105+0000","last_undegraded":"2024-04-14T08:34:34.408105+0000","last_fullsized":"2024-04-14T08:34:34.408105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.411421+0000","last_change":"2024-04-14T08:34:34.411421+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.411421+0000","last_undegraded":"2024-04-14T08:34:34.411421+0000","last_fullsized":"2024-04-14T08:34:34.411421+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.440794+0000","last_change":"2024-04-14T08:34:35.434489+0000","last_active":"2024-04-14T08:34:35.440794+0000","last_peered":"2024-04-14T08:34:35.440794+0000","last_clean":"2024-04-14T08:34:35.440794+0000","last_became_active":"2024-04-14T08:34:35.434142+0000","last_became_peered":"2024-04-14T08:34:35.434142+0000","last_unstale":"2024-04-14T08:34:35.440794+0000","last_undegraded":"2024-04-14T08:34:35.440794+0000","last_fullsized":"2024-04-14T08:34:35.440794+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:24:33.215747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.415922+0000","last_change":"2024-04-14T08:34:34.414663+0000","last_active":"2024-04-14T08:34:34.415922+0000","last_peered":"2024-04-14T08:34:34.415922+0000","last_clean":"2024-04-14T08:34:34.415922+0000","last_became_active":"2024-04-14T08:34:34.414390+0000","last_became_peered":"2024-04-14T08:34:34.414390+0000","last_unstale":"2024-04-14T08:34:34.415922+0000","last_undegraded":"2024-04-14T08:34:34.415922+0000","last_fullsized":"2024-04-14T08:34:34.415922+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:30:49.246797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409109+0000","last_change":"2024-04-14T08:34:34.409109+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409109+0000","last_undegraded":"2024-04-14T08:34:34.409109+0000","last_fullsized":"2024-04-14T08:34:34.409109+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.420847+0000","last_change":"2024-04-14T08:34:34.415460+0000","last_active":"2024-04-14T08:34:34.420847+0000","last_peered":"2024-04-14T08:34:34.420847+0000","last_clean":"2024-04-14T08:34:34.420847+0000","last_became_active":"2024-04-14T08:34:34.415176+0000","last_became_peered":"2024-04-14T08:34:34.415176+0000","last_unstale":"2024-04-14T08:34:34.420847+0000","last_undegraded":"2024-04-14T08:34:34.420847+0000","last_fullsized":"2024-04-14T08:34:34.420847+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:49.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410262+0000","last_change":"2024-04-14T08:34:34.410262+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410262+0000","last_undegraded":"2024-04-14T08:34:34.410262+0000","last_fullsized":"2024-04-14T08:34:34.410262+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.420592+0000","last_change":"2024-04-14T08:34:34.413412+0000","last_active":"2024-04-14T08:34:34.420592+0000","last_peered":"2024-04-14T08:34:34.420592+0000","last_clean":"2024-04-14T08:34:34.420592+0000","last_became_active":"2024-04-14T08:34:34.413050+0000","last_became_peered":"2024-04-14T08:34:34.413050+0000","last_unstale":"2024-04-14T08:34:34.420592+0000","last_undegraded":"2024-04-14T08:34:34.420592+0000","last_fullsized":"2024-04-14T08:34:34.420592+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:10.773823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410996+0000","last_change":"2024-04-14T08:34:34.410996+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410996+0000","last_undegraded":"2024-04-14T08:34:34.410996+0000","last_fullsized":"2024-04-14T08:34:34.410996+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.427275+0000","last_change":"2024-04-14T08:34:34.423624+0000","last_active":"2024-04-14T08:34:34.427275+0000","last_peered":"2024-04-14T08:34:34.427275+0000","last_clean":"2024-04-14T08:34:34.427275+0000","last_became_active":"2024-04-14T08:34:34.423373+0000","last_became_peered":"2024-04-14T08:34:34.423373+0000","last_unstale":"2024-04-14T08:34:34.427275+0000","last_undegraded":"2024-04-14T08:34:34.427275+0000","last_fullsized":"2024-04-14T08:34:34.427275+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406292+0000","last_change":"2024-04-14T08:34:34.406292+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406292+0000","last_undegraded":"2024-04-14T08:34:34.406292+0000","last_fullsized":"2024-04-14T08:34:34.406292+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424001+0000","last_change":"2024-04-14T08:34:34.420330+0000","last_active":"2024-04-14T08:34:34.424001+0000","last_peered":"2024-04-14T08:34:34.424001+0000","last_clean":"2024-04-14T08:34:34.424001+0000","last_became_active":"2024-04-14T08:34:34.419768+0000","last_became_peered":"2024-04-14T08:34:34.419768+0000","last_unstale":"2024-04-14T08:34:34.424001+0000","last_undegraded":"2024-04-14T08:34:34.424001+0000","last_fullsized":"2024-04-14T08:34:34.424001+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:23:36.394817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.439163+0000","last_change":"2024-04-14T08:34:35.431263+0000","last_active":"2024-04-14T08:34:35.439163+0000","last_peered":"2024-04-14T08:34:35.439163+0000","last_clean":"2024-04-14T08:34:35.439163+0000","last_became_active":"2024-04-14T08:34:35.431022+0000","last_became_peered":"2024-04-14T08:34:35.431022+0000","last_unstale":"2024-04-14T08:34:35.439163+0000","last_undegraded":"2024-04-14T08:34:35.439163+0000","last_fullsized":"2024-04-14T08:34:35.439163+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:56:32.524197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.416293+0000","last_change":"2024-04-14T08:34:34.413550+0000","last_active":"2024-04-14T08:34:34.416293+0000","last_peered":"2024-04-14T08:34:34.416293+0000","last_clean":"2024-04-14T08:34:34.416293+0000","last_became_active":"2024-04-14T08:34:34.413268+0000","last_became_peered":"2024-04-14T08:34:34.413268+0000","last_unstale":"2024-04-14T08:34:34.416293+0000","last_undegraded":"2024-04-14T08:34:34.416293+0000","last_fullsized":"2024-04-14T08:34:34.416293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:04.319595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408225+0000","last_change":"2024-04-14T08:34:34.408225+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408225+0000","last_undegraded":"2024-04-14T08:34:34.408225+0000","last_fullsized":"2024-04-14T08:34:34.408225+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.418469+0000","last_change":"2024-04-14T08:34:34.415171+0000","last_active":"2024-04-14T08:34:34.418469+0000","last_peered":"2024-04-14T08:34:34.418469+0000","last_clean":"2024-04-14T08:34:34.418469+0000","last_became_active":"2024-04-14T08:34:34.414890+0000","last_became_peered":"2024-04-14T08:34:34.414890+0000","last_unstale":"2024-04-14T08:34:34.418469+0000","last_undegraded":"2024-04-14T08:34:34.418469+0000","last_fullsized":"2024-04-14T08:34:34.418469+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:11.472608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.444176+0000","last_change":"2024-04-14T08:34:35.440848+0000","last_active":"2024-04-14T08:34:35.444176+0000","last_peered":"2024-04-14T08:34:35.444176+0000","last_clean":"2024-04-14T08:34:35.444176+0000","last_became_active":"2024-04-14T08:34:35.440519+0000","last_became_peered":"2024-04-14T08:34:35.440519+0000","last_unstale":"2024-04-14T08:34:35.444176+0000","last_undegraded":"2024-04-14T08:34:35.444176+0000","last_fullsized":"2024-04-14T08:34:35.444176+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:13:42.361716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424504+0000","last_change":"2024-04-14T08:34:34.419189+0000","last_active":"2024-04-14T08:34:34.424504+0000","last_peered":"2024-04-14T08:34:34.424504+0000","last_clean":"2024-04-14T08:34:34.424504+0000","last_became_active":"2024-04-14T08:34:34.418954+0000","last_became_peered":"2024-04-14T08:34:34.418954+0000","last_unstale":"2024-04-14T08:34:34.424504+0000","last_undegraded":"2024-04-14T08:34:34.424504+0000","last_fullsized":"2024-04-14T08:34:34.424504+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:54:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405634+0000","last_change":"2024-04-14T08:34:34.405634+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405634+0000","last_undegraded":"2024-04-14T08:34:34.405634+0000","last_fullsized":"2024-04-14T08:34:34.405634+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.419794+0000","last_change":"2024-04-14T08:34:34.416908+0000","last_active":"2024-04-14T08:34:35.419794+0000","last_peered":"2024-04-14T08:34:35.419794+0000","last_clean":"2024-04-14T08:34:35.419794+0000","last_became_active":"2024-04-14T08:34:34.416617+0000","last_became_peered":"2024-04-14T08:34:34.416617+0000","last_unstale":"2024-04-14T08:34:35.419794+0000","last_undegraded":"2024-04-14T08:34:35.419794+0000","last_fullsized":"2024-04-14T08:34:35.419794+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:11.743901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413953+0000","last_change":"2024-04-14T08:34:34.413953+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413953+0000","last_undegraded":"2024-04-14T08:34:34.413953+0000","last_fullsized":"2024-04-14T08:34:34.413953+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424753+0000","last_change":"2024-04-14T08:34:34.421136+0000","last_active":"2024-04-14T08:34:34.424753+0000","last_peered":"2024-04-14T08:34:34.424753+0000","last_clean":"2024-04-14T08:34:34.424753+0000","last_became_active":"2024-04-14T08:34:34.420809+0000","last_became_peered":"2024-04-14T08:34:34.420809+0000","last_unstale":"2024-04-14T08:34:34.424753+0000","last_undegraded":"2024-04-14T08:34:34.424753+0000","last_fullsized":"2024-04-14T08:34:34.424753+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:49.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.407542+0000","last_change":"2024-04-14T08:34:34.407542+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.407542+0000","last_undegraded":"2024-04-14T08:34:34.407542+0000","last_fullsized":"2024-04-14T08:34:34.407542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.448678+0000","last_change":"2024-04-14T08:34:34.422319+0000","last_active":"2024-04-14T08:34:35.448678+0000","last_peered":"2024-04-14T08:34:35.448678+0000","last_clean":"2024-04-14T08:34:35.448678+0000","last_became_active":"2024-04-14T08:34:34.421978+0000","last_became_peered":"2024-04-14T08:34:34.421978+0000","last_unstale":"2024-04-14T08:34:35.448678+0000","last_undegraded":"2024-04-14T08:34:35.448678+0000","last_fullsized":"2024-04-14T08:34:35.448678+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:11.381349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.404550+0000","last_change":"2024-04-14T08:34:34.404550+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.404550+0000","last_undegraded":"2024-04-14T08:34:34.404550+0000","last_fullsized":"2024-04-14T08:34:34.404550+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423937+0000","last_change":"2024-04-14T08:34:34.420518+0000","last_active":"2024-04-14T08:34:34.423937+0000","last_peered":"2024-04-14T08:34:34.423937+0000","last_clean":"2024-04-14T08:34:34.423937+0000","last_became_active":"2024-04-14T08:34:34.419065+0000","last_became_peered":"2024-04-14T08:34:34.419065+0000","last_unstale":"2024-04-14T08:34:34.423937+0000","last_undegraded":"2024-04-14T08:34:34.423937+0000","last_fullsized":"2024-04-14T08:34:34.423937+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:36.033184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410498+0000","last_change":"2024-04-14T08:34:34.410498+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410498+0000","last_undegraded":"2024-04-14T08:34:34.410498+0000","last_fullsized":"2024-04-14T08:34:34.410498+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423983+0000","last_change":"2024-04-14T08:34:34.421506+0000","last_active":"2024-04-14T08:34:34.423983+0000","last_peered":"2024-04-14T08:34:34.423983+0000","last_clean":"2024-04-14T08:34:34.423983+0000","last_became_active":"2024-04-14T08:34:34.420091+0000","last_became_peered":"2024-04-14T08:34:34.420091+0000","last_unstale":"2024-04-14T08:34:34.423983+0000","last_undegraded":"2024-04-14T08:34:34.423983+0000","last_fullsized":"2024-04-14T08:34:34.423983+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:30:35.547680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.412081+0000","last_change":"2024-04-14T08:34:34.412081+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.412081+0000","last_undegraded":"2024-04-14T08:34:34.412081+0000","last_fullsized":"2024-04-14T08:34:34.412081+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.429886+0000","last_change":"2024-04-14T08:34:34.424435+0000","last_active":"2024-04-14T08:34:34.429886+0000","last_peered":"2024-04-14T08:34:34.429886+0000","last_clean":"2024-04-14T08:34:34.429886+0000","last_became_active":"2024-04-14T08:34:34.424085+0000","last_became_peered":"2024-04-14T08:34:34.424085+0000","last_unstale":"2024-04-14T08:34:34.429886+0000","last_undegraded":"2024-04-14T08:34:34.429886+0000","last_fullsized":"2024-04-14T08:34:34.429886+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:04.319595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409928+0000","last_change":"2024-04-14T08:34:34.409928+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409928+0000","last_undegraded":"2024-04-14T08:34:34.409928+0000","last_fullsized":"2024-04-14T08:34:34.409928+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.419215+0000","last_change":"2024-04-14T08:34:34.414833+0000","last_active":"2024-04-14T08:34:35.419215+0000","last_peered":"2024-04-14T08:34:35.419215+0000","last_clean":"2024-04-14T08:34:35.419215+0000","last_became_active":"2024-04-14T08:34:34.414553+0000","last_became_peered":"2024-04-14T08:34:34.414553+0000","last_unstale":"2024-04-14T08:34:35.419215+0000","last_undegraded":"2024-04-14T08:34:35.419215+0000","last_fullsized":"2024-04-14T08:34:35.419215+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414459+0000","last_change":"2024-04-14T08:34:34.414459+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414459+0000","last_undegraded":"2024-04-14T08:34:34.414459+0000","last_fullsized":"2024-04-14T08:34:34.414459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.422034+0000","last_change":"2024-04-14T08:34:34.416091+0000","last_active":"2024-04-14T08:34:34.422034+0000","last_peered":"2024-04-14T08:34:34.422034+0000","last_clean":"2024-04-14T08:34:34.422034+0000","last_became_active":"2024-04-14T08:34:34.415653+0000","last_became_peered":"2024-04-14T08:34:34.415653+0000","last_unstale":"2024-04-14T08:34:34.422034+0000","last_undegraded":"2024-04-14T08:34:34.422034+0000","last_fullsized":"2024-04-14T08:34:34.422034+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.407664+0000","last_change":"2024-04-14T08:34:34.407664+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.407664+0000","last_undegraded":"2024-04-14T08:34:34.407664+0000","last_fullsized":"2024-04-14T08:34:34.407664+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.425676+0000","last_change":"2024-04-14T08:34:34.423763+0000","last_active":"2024-04-14T08:34:34.425676+0000","last_peered":"2024-04-14T08:34:34.425676+0000","last_clean":"2024-04-14T08:34:34.425676+0000","last_became_active":"2024-04-14T08:34:34.423333+0000","last_became_peered":"2024-04-14T08:34:34.423333+0000","last_unstale":"2024-04-14T08:34:34.425676+0000","last_undegraded":"2024-04-14T08:34:34.425676+0000","last_fullsized":"2024-04-14T08:34:34.425676+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:22:38.569478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405513+0000","last_change":"2024-04-14T08:34:34.405513+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405513+0000","last_undegraded":"2024-04-14T08:34:34.405513+0000","last_fullsized":"2024-04-14T08:34:34.405513+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.418045+0000","last_change":"2024-04-14T08:34:34.416569+0000","last_active":"2024-04-14T08:34:34.418045+0000","last_peered":"2024-04-14T08:34:34.418045+0000","last_clean":"2024-04-14T08:34:34.418045+0000","last_became_active":"2024-04-14T08:34:34.416306+0000","last_became_peered":"2024-04-14T08:34:34.416306+0000","last_unstale":"2024-04-14T08:34:34.418045+0000","last_undegraded":"2024-04-14T08:34:34.418045+0000","last_fullsized":"2024-04-14T08:34:34.418045+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:56.093503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405734+0000","last_change":"2024-04-14T08:34:34.405734+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405734+0000","last_undegraded":"2024-04-14T08:34:34.405734+0000","last_fullsized":"2024-04-14T08:34:34.405734+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.427399+0000","last_change":"2024-04-14T08:34:34.422659+0000","last_active":"2024-04-14T08:34:34.427399+0000","last_peered":"2024-04-14T08:34:34.427399+0000","last_clean":"2024-04-14T08:34:34.427399+0000","last_became_active":"2024-04-14T08:34:34.422434+0000","last_became_peered":"2024-04-14T08:34:34.422434+0000","last_unstale":"2024-04-14T08:34:34.427399+0000","last_undegraded":"2024-04-14T08:34:34.427399+0000","last_fullsized":"2024-04-14T08:34:34.427399+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:59:48.942079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405327+0000","last_change":"2024-04-14T08:34:34.405327+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405327+0000","last_undegraded":"2024-04-14T08:34:34.405327+0000","last_fullsized":"2024-04-14T08:34:34.405327+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.427365+0000","last_change":"2024-04-14T08:34:34.422117+0000","last_active":"2024-04-14T08:34:34.427365+0000","last_peered":"2024-04-14T08:34:34.427365+0000","last_clean":"2024-04-14T08:34:34.427365+0000","last_became_active":"2024-04-14T08:34:34.421504+0000","last_became_peered":"2024-04-14T08:34:34.421504+0000","last_unstale":"2024-04-14T08:34:34.427365+0000","last_undegraded":"2024-04-14T08:34:34.427365+0000","last_fullsized":"2024-04-14T08:34:34.427365+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:26:52.383137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409160+0000","last_change":"2024-04-14T08:34:34.409160+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409160+0000","last_undegraded":"2024-04-14T08:34:34.409160+0000","last_fullsized":"2024-04-14T08:34:34.409160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.420594+0000","last_change":"2024-04-14T08:34:34.413030+0000","last_active":"2024-04-14T08:34:34.420594+0000","last_peered":"2024-04-14T08:34:34.420594+0000","last_clean":"2024-04-14T08:34:34.420594+0000","last_became_active":"2024-04-14T08:34:34.412774+0000","last_became_peered":"2024-04-14T08:34:34.412774+0000","last_unstale":"2024-04-14T08:34:34.420594+0000","last_undegraded":"2024-04-14T08:34:34.420594+0000","last_fullsized":"2024-04-14T08:34:34.420594+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:04.319595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409571+0000","last_change":"2024-04-14T08:34:34.409571+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409571+0000","last_undegraded":"2024-04-14T08:34:34.409571+0000","last_fullsized":"2024-04-14T08:34:34.409571+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.421957+0000","last_change":"2024-04-14T08:34:34.417238+0000","last_active":"2024-04-14T08:34:34.421957+0000","last_peered":"2024-04-14T08:34:34.421957+0000","last_clean":"2024-04-14T08:34:34.421957+0000","last_became_active":"2024-04-14T08:34:34.417002+0000","last_became_peered":"2024-04-14T08:34:34.417002+0000","last_unstale":"2024-04-14T08:34:34.421957+0000","last_undegraded":"2024-04-14T08:34:34.421957+0000","last_fullsized":"2024-04-14T08:34:34.421957+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:11.743901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413056+0000","last_change":"2024-04-14T08:34:34.413056+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413056+0000","last_undegraded":"2024-04-14T08:34:34.413056+0000","last_fullsized":"2024-04-14T08:34:34.413056+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.425644+0000","last_change":"2024-04-14T08:34:34.423744+0000","last_active":"2024-04-14T08:34:34.425644+0000","last_peered":"2024-04-14T08:34:34.425644+0000","last_clean":"2024-04-14T08:34:34.425644+0000","last_became_active":"2024-04-14T08:34:34.423394+0000","last_became_peered":"2024-04-14T08:34:34.423394+0000","last_unstale":"2024-04-14T08:34:34.425644+0000","last_undegraded":"2024-04-14T08:34:34.425644+0000","last_fullsized":"2024-04-14T08:34:34.425644+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:32.608024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406093+0000","last_change":"2024-04-14T08:34:34.406093+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406093+0000","last_undegraded":"2024-04-14T08:34:34.406093+0000","last_fullsized":"2024-04-14T08:34:34.406093+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.414356+0000","last_change":"2024-04-14T08:34:34.411096+0000","last_active":"2024-04-14T08:34:34.414356+0000","last_peered":"2024-04-14T08:34:34.414356+0000","last_clean":"2024-04-14T08:34:34.414356+0000","last_became_active":"2024-04-14T08:34:34.410841+0000","last_became_peered":"2024-04-14T08:34:34.410841+0000","last_unstale":"2024-04-14T08:34:34.414356+0000","last_undegraded":"2024-04-14T08:34:34.414356+0000","last_fullsized":"2024-04-14T08:34:34.414356+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:12:26.461355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410965+0000","last_change":"2024-04-14T08:34:34.410965+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410965+0000","last_undegraded":"2024-04-14T08:34:34.410965+0000","last_fullsized":"2024-04-14T08:34:34.410965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.429860+0000","last_change":"2024-04-14T08:34:34.424338+0000","last_active":"2024-04-14T08:34:34.429860+0000","last_peered":"2024-04-14T08:34:34.429860+0000","last_clean":"2024-04-14T08:34:34.429860+0000","last_became_active":"2024-04-14T08:34:34.424057+0000","last_became_peered":"2024-04-14T08:34:34.424057+0000","last_unstale":"2024-04-14T08:34:34.429860+0000","last_undegraded":"2024-04-14T08:34:34.429860+0000","last_fullsized":"2024-04-14T08:34:34.429860+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:10.773823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406288+0000","last_change":"2024-04-14T08:34:34.406288+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406288+0000","last_undegraded":"2024-04-14T08:34:34.406288+0000","last_fullsized":"2024-04-14T08:34:34.406288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.420306+0000","last_change":"2024-04-14T08:34:34.416682+0000","last_active":"2024-04-14T08:34:35.420306+0000","last_peered":"2024-04-14T08:34:35.420306+0000","last_clean":"2024-04-14T08:34:35.420306+0000","last_became_active":"2024-04-14T08:34:34.416172+0000","last_became_peered":"2024-04-14T08:34:34.416172+0000","last_unstale":"2024-04-14T08:34:35.420306+0000","last_undegraded":"2024-04-14T08:34:35.420306+0000","last_fullsized":"2024-04-14T08:34:35.420306+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:36.033184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408713+0000","last_change":"2024-04-14T08:34:34.408713+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408713+0000","last_undegraded":"2024-04-14T08:34:34.408713+0000","last_fullsized":"2024-04-14T08:34:34.408713+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423234+0000","last_change":"2024-04-14T08:34:34.420330+0000","last_active":"2024-04-14T08:34:34.423234+0000","last_peered":"2024-04-14T08:34:34.423234+0000","last_clean":"2024-04-14T08:34:34.423234+0000","last_became_active":"2024-04-14T08:34:34.420116+0000","last_became_peered":"2024-04-14T08:34:34.420116+0000","last_unstale":"2024-04-14T08:34:34.423234+0000","last_undegraded":"2024-04-14T08:34:34.423234+0000","last_fullsized":"2024-04-14T08:34:34.423234+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:41:19.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408837+0000","last_change":"2024-04-14T08:34:34.408837+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408837+0000","last_undegraded":"2024-04-14T08:34:34.408837+0000","last_fullsized":"2024-04-14T08:34:34.408837+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.422091+0000","last_change":"2024-04-14T08:34:34.419528+0000","last_active":"2024-04-14T08:34:34.422091+0000","last_peered":"2024-04-14T08:34:34.422091+0000","last_clean":"2024-04-14T08:34:34.422091+0000","last_became_active":"2024-04-14T08:34:34.419258+0000","last_became_peered":"2024-04-14T08:34:34.419258+0000","last_unstale":"2024-04-14T08:34:34.422091+0000","last_undegraded":"2024-04-14T08:34:34.422091+0000","last_fullsized":"2024-04-14T08:34:34.422091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:11.381349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409171+0000","last_change":"2024-04-14T08:34:34.409171+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409171+0000","last_undegraded":"2024-04-14T08:34:34.409171+0000","last_fullsized":"2024-04-14T08:34:34.409171+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423956+0000","last_change":"2024-04-14T08:34:34.421662+0000","last_active":"2024-04-14T08:34:34.423956+0000","last_peered":"2024-04-14T08:34:34.423956+0000","last_clean":"2024-04-14T08:34:34.423956+0000","last_became_active":"2024-04-14T08:34:34.421382+0000","last_became_peered":"2024-04-14T08:34:34.421382+0000","last_unstale":"2024-04-14T08:34:34.423956+0000","last_undegraded":"2024-04-14T08:34:34.423956+0000","last_fullsized":"2024-04-14T08:34:34.423956+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:55:44.175481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413461+0000","last_change":"2024-04-14T08:34:34.413461+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413461+0000","last_undegraded":"2024-04-14T08:34:34.413461+0000","last_fullsized":"2024-04-14T08:34:34.413461+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424542+0000","last_change":"2024-04-14T08:34:34.419857+0000","last_active":"2024-04-14T08:34:34.424542+0000","last_peered":"2024-04-14T08:34:34.424542+0000","last_clean":"2024-04-14T08:34:34.424542+0000","last_became_active":"2024-04-14T08:34:34.419548+0000","last_became_peered":"2024-04-14T08:34:34.419548+0000","last_unstale":"2024-04-14T08:34:34.424542+0000","last_undegraded":"2024-04-14T08:34:34.424542+0000","last_fullsized":"2024-04-14T08:34:34.424542+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:10.773823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408288+0000","last_change":"2024-04-14T08:34:34.408288+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408288+0000","last_undegraded":"2024-04-14T08:34:34.408288+0000","last_fullsized":"2024-04-14T08:34:34.408288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424716+0000","last_change":"2024-04-14T08:34:34.420316+0000","last_active":"2024-04-14T08:34:34.424716+0000","last_peered":"2024-04-14T08:34:34.424716+0000","last_clean":"2024-04-14T08:34:34.424716+0000","last_became_active":"2024-04-14T08:34:34.420093+0000","last_became_peered":"2024-04-14T08:34:34.420093+0000","last_unstale":"2024-04-14T08:34:34.424716+0000","last_undegraded":"2024-04-14T08:34:34.424716+0000","last_fullsized":"2024-04-14T08:34:34.424716+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:44:18.315252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408798+0000","last_change":"2024-04-14T08:34:34.408798+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408798+0000","last_undegraded":"2024-04-14T08:34:34.408798+0000","last_fullsized":"2024-04-14T08:34:34.408798+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423952+0000","last_change":"2024-04-14T08:34:34.421158+0000","last_active":"2024-04-14T08:34:34.423952+0000","last_peered":"2024-04-14T08:34:34.423952+0000","last_clean":"2024-04-14T08:34:34.423952+0000","last_became_active":"2024-04-14T08:34:34.420780+0000","last_became_peered":"2024-04-14T08:34:34.420780+0000","last_unstale":"2024-04-14T08:34:34.423952+0000","last_undegraded":"2024-04-14T08:34:34.423952+0000","last_fullsized":"2024-04-14T08:34:34.423952+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:44:10.910055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":226,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.418795+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:34:35.418795+0000","last_peered":"2024-04-14T08:34:35.418795+0000","last_clean":"2024-04-14T08:34:35.418795+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:34:35.418795+0000","last_undegraded":"2024-04-14T08:34:35.418795+0000","last_fullsized":"2024-04-14T08:34:35.418795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1183744,"data_stored":846191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":557056,"data_stored":241865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1163264,"data_stored":838900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:34:38.429 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:34:38.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.565+0000 7fbf57f95640 1 Processor -- start 2024-04-14T08:34:38.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.566+0000 7fbf57f95640 1 -- start start 2024-04-14T08:34:38.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.566+0000 7fbf57f95640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbf5015e890 0x7fbf500d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.566+0000 7fbf57f95640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbf500cedd0 0x7fbf500b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.566+0000 7fbf57f95640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf500cf320 con 0x7fbf500d2830 2024-04-14T08:34:38.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.566+0000 7fbf57f95640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fbf500cff30 con 0x7fbf500cedd0 2024-04-14T08:34:38.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.566+0000 7fbf57f95640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fbf500d2350 con 0x7fbf5015e890 2024-04-14T08:34:38.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.567+0000 7fbf56532640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbf500cedd0 0x7fbf500b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.567+0000 7fbf56532640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbf500cedd0 0x7fbf500b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48190/0 (socket says 172.21.15.172:48190) 2024-04-14T08:34:38.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.567+0000 7fbf56532640 1 -- 172.21.15.172:0/3810469218 learned_addr learned my addr 172.21.15.172:0/3810469218 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:38.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.567+0000 7fbf55530640 1 --2- 172.21.15.172:0/3810469218 >> v2:172.21.15.177:3300/0 conn(0x7fbf5015e890 0x7fbf500d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.567+0000 7fbf56532640 1 -- 172.21.15.172:0/3810469218 >> v2:172.21.15.177:3300/0 conn(0x7fbf5015e890 msgr2=0x7fbf500d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:38.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.567+0000 7fbf56532640 1 --2- 172.21.15.172:0/3810469218 >> v2:172.21.15.177:3300/0 conn(0x7fbf5015e890 0x7fbf500d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.567+0000 7fbf56532640 1 -- 172.21.15.172:0/3810469218 >> v1:172.21.15.177:6789/0 conn(0x7fbf500d2830 legacy=0x7fbf500d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:38.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.567+0000 7fbf56532640 1 -- 172.21.15.172:0/3810469218 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf500cf600 con 0x7fbf500cedd0 2024-04-14T08:34:38.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.567+0000 7fbf55530640 1 --2- 172.21.15.172:0/3810469218 >> v2:172.21.15.177:3300/0 conn(0x7fbf5015e890 0x7fbf500d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.568+0000 7fbf56532640 1 --2- 172.21.15.172:0/3810469218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf500cedd0 0x7fbf500b3710 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbf50162100 tx=0x7fbf4c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9f4a42750f51ef server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.568+0000 7fbf54d2f640 1 -- 172.21.15.172:0/3810469218 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c02cb20 con 0x7fbf500cedd0 2024-04-14T08:34:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.568+0000 7fbf54d2f640 1 -- 172.21.15.172:0/3810469218 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf4c02ccc0 con 0x7fbf500cedd0 2024-04-14T08:34:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.568+0000 7fbf54d2f640 1 -- 172.21.15.172:0/3810469218 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c034610 con 0x7fbf500cedd0 2024-04-14T08:34:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.568+0000 7fbf57f95640 1 -- 172.21.15.172:0/3810469218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf500cedd0 msgr2=0x7fbf500b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.568+0000 7fbf57f95640 1 --2- 172.21.15.172:0/3810469218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf500cedd0 0x7fbf500b3710 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbf50162100 tx=0x7fbf4c02bf80 comp rx=0 tx=0).stop 2024-04-14T08:34:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.568+0000 7fbf57f95640 1 -- 172.21.15.172:0/3810469218 shutdown_connections 2024-04-14T08:34:38.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.568+0000 7fbf57f95640 1 --2- 172.21.15.172:0/3810469218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf500cedd0 0x7fbf500b3710 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.568+0000 7fbf57f95640 1 --2- 172.21.15.172:0/3810469218 >> v2:172.21.15.177:3300/0 conn(0x7fbf5015e890 0x7fbf500d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.568+0000 7fbf57f95640 1 -- 172.21.15.172:0/3810469218 >> 172.21.15.172:0/3810469218 conn(0x7fbf5009d590 msgr2=0x7fbf500d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.568+0000 7fbf57f95640 1 -- 172.21.15.172:0/3810469218 shutdown_connections 2024-04-14T08:34:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.568+0000 7fbf57f95640 1 -- 172.21.15.172:0/3810469218 wait complete. 2024-04-14T08:34:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.570+0000 7fbf57f95640 1 Processor -- start 2024-04-14T08:34:38.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.570+0000 7fbf57f95640 1 -- start start 2024-04-14T08:34:38.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.570+0000 7fbf57f95640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf500cedd0 0x7fbf501516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.570+0000 7fbf57f95640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf500d2830 0x7fbf50160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.570+0000 7fbf57f95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf5015e890 0x7fbf501613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.570+0000 7fbf57f95640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbf500d1840 con 0x7fbf5015e890 2024-04-14T08:34:38.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.570+0000 7fbf57f95640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbf500cfbc0 con 0x7fbf500d2830 2024-04-14T08:34:38.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.570+0000 7fbf57f95640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbf500b3c50 con 0x7fbf500cedd0 2024-04-14T08:34:38.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.570+0000 7fbf56532640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf5015e890 0x7fbf501613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.570+0000 7fbf56532640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf5015e890 0x7fbf501613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48202/0 (socket says 172.21.15.172:48202) 2024-04-14T08:34:38.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.570+0000 7fbf56532640 1 -- 172.21.15.172:0/1294227836 learned_addr learned my addr 172.21.15.172:0/1294227836 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:38.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf55530640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf500d2830 0x7fbf50160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf55d31640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf500cedd0 0x7fbf501516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf56532640 1 -- 172.21.15.172:0/1294227836 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf500cedd0 msgr2=0x7fbf501516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf56532640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf500cedd0 0x7fbf501516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf56532640 1 -- 172.21.15.172:0/1294227836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf500d2830 msgr2=0x7fbf50160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf56532640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf500d2830 0x7fbf50160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf56532640 1 -- 172.21.15.172:0/1294227836 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf50160a00 con 0x7fbf5015e890 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf55530640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf500d2830 0x7fbf50160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf56532640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf5015e890 0x7fbf501613b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c036040 tx=0x7fbf4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf467fc640 1 -- 172.21.15.172:0/1294227836 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c043020 con 0x7fbf5015e890 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf467fc640 1 -- 172.21.15.172:0/1294227836 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf4c03e450 con 0x7fbf5015e890 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf57f95640 1 -- 172.21.15.172:0/1294227836 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5015f960 con 0x7fbf5015e890 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf467fc640 1 -- 172.21.15.172:0/1294227836 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c03e860 con 0x7fbf5015e890 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.571+0000 7fbf57f95640 1 -- 172.21.15.172:0/1294227836 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5015ff20 con 0x7fbf5015e890 2024-04-14T08:34:38.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.572+0000 7fbf57f95640 1 -- 172.21.15.172:0/1294227836 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf50068520 con 0x7fbf5015e890 2024-04-14T08:34:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.573+0000 7fbf467fc640 1 -- 172.21.15.172:0/1294227836 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf4c046020 con 0x7fbf5015e890 2024-04-14T08:34:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.573+0000 7fbf467fc640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbf3406c8f0 0x7fbf3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.573+0000 7fbf467fc640 1 -- 172.21.15.172:0/1294227836 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fbf4c0ae5c0 con 0x7fbf5015e890 2024-04-14T08:34:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.573+0000 7fbf55d31640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbf3406c8f0 0x7fbf3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.574+0000 7fbf55d31640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbf3406c8f0 0x7fbf3406edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbf48003d90 tx=0x7fbf48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:38.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.581+0000 7fbf467fc640 1 -- 172.21.15.172:0/1294227836 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbf4c04d030 con 0x7fbf5015e890 2024-04-14T08:34:38.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.757+0000 7fbf57f95640 1 -- 172.21.15.172:0/1294227836 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf500d4960 con 0x7fbf3406c8f0 2024-04-14T08:34:38.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.768+0000 7fbf467fc640 1 -- 172.21.15.172:0/1294227836 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247296 (secure 0 0 0) 0x7fbf500d4960 con 0x7fbf3406c8f0 2024-04-14T08:34:38.772 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:38.773 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:34:38.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.771+0000 7fbf57f95640 1 -- 172.21.15.172:0/1294227836 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbf3406c8f0 msgr2=0x7fbf3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.771+0000 7fbf57f95640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbf3406c8f0 0x7fbf3406edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbf48003d90 tx=0x7fbf48009040 comp rx=0 tx=0).stop 2024-04-14T08:34:38.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.771+0000 7fbf57f95640 1 -- 172.21.15.172:0/1294227836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf5015e890 msgr2=0x7fbf501613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.771+0000 7fbf57f95640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf5015e890 0x7fbf501613b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c036040 tx=0x7fbf4c004a40 comp rx=0 tx=0).stop 2024-04-14T08:34:38.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.771+0000 7fbf57f95640 1 -- 172.21.15.172:0/1294227836 shutdown_connections 2024-04-14T08:34:38.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.771+0000 7fbf57f95640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbf3406c8f0 0x7fbf3406edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.771+0000 7fbf57f95640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf5015e890 0x7fbf501613b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.771+0000 7fbf57f95640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf500d2830 0x7fbf50160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.771+0000 7fbf57f95640 1 --2- 172.21.15.172:0/1294227836 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf500cedd0 0x7fbf501516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.771+0000 7fbf57f95640 1 -- 172.21.15.172:0/1294227836 >> 172.21.15.172:0/1294227836 conn(0x7fbf5009d590 msgr2=0x7fbf500b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:38.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.772+0000 7fbf57f95640 1 -- 172.21.15.172:0/1294227836 shutdown_connections 2024-04-14T08:34:38.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.772+0000 7fbf57f95640 1 -- 172.21.15.172:0/1294227836 wait complete. 2024-04-14T08:34:38.787 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-04-14T08:34:37.140761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221808,"kb_used_data":5616,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752912,"statfs":{"total":773094113280,"available":772866981888,"internally_reserved":0,"allocated":5750784,"data_stored":3169616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000673"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408023+0000","last_change":"2024-04-14T08:34:34.408023+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408023+0000","last_undegraded":"2024-04-14T08:34:34.408023+0000","last_fullsized":"2024-04-14T08:34:34.408023+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.441184+0000","last_change":"2024-04-14T08:34:35.434456+0000","last_active":"2024-04-14T08:34:35.441184+0000","last_peered":"2024-04-14T08:34:35.441184+0000","last_clean":"2024-04-14T08:34:35.441184+0000","last_became_active":"2024-04-14T08:34:35.434158+0000","last_became_peered":"2024-04-14T08:34:35.434158+0000","last_unstale":"2024-04-14T08:34:35.441184+0000","last_undegraded":"2024-04-14T08:34:35.441184+0000","last_fullsized":"2024-04-14T08:34:35.441184+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:40:13.833057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414650+0000","last_change":"2024-04-14T08:34:34.414650+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414650+0000","last_undegraded":"2024-04-14T08:34:34.414650+0000","last_fullsized":"2024-04-14T08:34:34.414650+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.437687+0000","last_change":"2024-04-14T08:34:35.430927+0000","last_active":"2024-04-14T08:34:35.437687+0000","last_peered":"2024-04-14T08:34:35.437687+0000","last_clean":"2024-04-14T08:34:35.437687+0000","last_became_active":"2024-04-14T08:34:35.430164+0000","last_became_peered":"2024-04-14T08:34:35.430164+0000","last_unstale":"2024-04-14T08:34:35.437687+0000","last_undegraded":"2024-04-14T08:34:35.437687+0000","last_fullsized":"2024-04-14T08:34:35.437687+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:57:15.656326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405062+0000","last_change":"2024-04-14T08:34:34.405062+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405062+0000","last_undegraded":"2024-04-14T08:34:34.405062+0000","last_fullsized":"2024-04-14T08:34:34.405062+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406804+0000","last_change":"2024-04-14T08:34:34.406804+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406804+0000","last_undegraded":"2024-04-14T08:34:34.406804+0000","last_fullsized":"2024-04-14T08:34:34.406804+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414181+0000","last_change":"2024-04-14T08:34:34.414181+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414181+0000","last_undegraded":"2024-04-14T08:34:34.414181+0000","last_fullsized":"2024-04-14T08:34:34.414181+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409156+0000","last_change":"2024-04-14T08:34:34.409156+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409156+0000","last_undegraded":"2024-04-14T08:34:34.409156+0000","last_fullsized":"2024-04-14T08:34:34.409156+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410003+0000","last_change":"2024-04-14T08:34:34.410003+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410003+0000","last_undegraded":"2024-04-14T08:34:34.410003+0000","last_fullsized":"2024-04-14T08:34:34.410003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406692+0000","last_change":"2024-04-14T08:34:34.406692+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406692+0000","last_undegraded":"2024-04-14T08:34:34.406692+0000","last_fullsized":"2024-04-14T08:34:34.406692+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405927+0000","last_change":"2024-04-14T08:34:34.405927+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405927+0000","last_undegraded":"2024-04-14T08:34:34.405927+0000","last_fullsized":"2024-04-14T08:34:34.405927+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410934+0000","last_change":"2024-04-14T08:34:34.410934+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410934+0000","last_undegraded":"2024-04-14T08:34:34.410934+0000","last_fullsized":"2024-04-14T08:34:34.410934+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406225+0000","last_change":"2024-04-14T08:34:34.406225+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406225+0000","last_undegraded":"2024-04-14T08:34:34.406225+0000","last_fullsized":"2024-04-14T08:34:34.406225+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.411333+0000","last_change":"2024-04-14T08:34:34.411333+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.411333+0000","last_undegraded":"2024-04-14T08:34:34.411333+0000","last_fullsized":"2024-04-14T08:34:34.411333+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.437809+0000","last_change":"2024-04-14T08:34:35.428986+0000","last_active":"2024-04-14T08:34:35.437809+0000","last_peered":"2024-04-14T08:34:35.437809+0000","last_clean":"2024-04-14T08:34:35.437809+0000","last_became_active":"2024-04-14T08:34:35.426840+0000","last_became_peered":"2024-04-14T08:34:35.426840+0000","last_unstale":"2024-04-14T08:34:35.437809+0000","last_undegraded":"2024-04-14T08:34:35.437809+0000","last_fullsized":"2024-04-14T08:34:35.437809+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:55:45.179250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406390+0000","last_change":"2024-04-14T08:34:34.406390+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406390+0000","last_undegraded":"2024-04-14T08:34:34.406390+0000","last_fullsized":"2024-04-14T08:34:34.406390+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414078+0000","last_change":"2024-04-14T08:34:34.414078+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414078+0000","last_undegraded":"2024-04-14T08:34:34.414078+0000","last_fullsized":"2024-04-14T08:34:34.414078+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.441163+0000","last_change":"2024-04-14T08:34:35.434292+0000","last_active":"2024-04-14T08:34:35.441163+0000","last_peered":"2024-04-14T08:34:35.441163+0000","last_clean":"2024-04-14T08:34:35.441163+0000","last_became_active":"2024-04-14T08:34:35.434018+0000","last_became_peered":"2024-04-14T08:34:35.434018+0000","last_unstale":"2024-04-14T08:34:35.441163+0000","last_undegraded":"2024-04-14T08:34:35.441163+0000","last_fullsized":"2024-04-14T08:34:35.441163+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:22:04.896638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.412491+0000","last_change":"2024-04-14T08:34:34.412491+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.412491+0000","last_undegraded":"2024-04-14T08:34:34.412491+0000","last_fullsized":"2024-04-14T08:34:34.412491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410141+0000","last_change":"2024-04-14T08:34:34.410141+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410141+0000","last_undegraded":"2024-04-14T08:34:34.410141+0000","last_fullsized":"2024-04-14T08:34:34.410141+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408219+0000","last_change":"2024-04-14T08:34:34.408219+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408219+0000","last_undegraded":"2024-04-14T08:34:34.408219+0000","last_fullsized":"2024-04-14T08:34:34.408219+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.407965+0000","last_change":"2024-04-14T08:34:34.407965+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.407965+0000","last_undegraded":"2024-04-14T08:34:34.407965+0000","last_fullsized":"2024-04-14T08:34:34.407965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405857+0000","last_change":"2024-04-14T08:34:34.405857+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405857+0000","last_undegraded":"2024-04-14T08:34:34.405857+0000","last_fullsized":"2024-04-14T08:34:34.405857+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410886+0000","last_change":"2024-04-14T08:34:34.410886+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410886+0000","last_undegraded":"2024-04-14T08:34:34.410886+0000","last_fullsized":"2024-04-14T08:34:34.410886+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413719+0000","last_change":"2024-04-14T08:34:34.413719+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413719+0000","last_undegraded":"2024-04-14T08:34:34.413719+0000","last_fullsized":"2024-04-14T08:34:34.413719+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410342+0000","last_change":"2024-04-14T08:34:34.410342+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410342+0000","last_undegraded":"2024-04-14T08:34:34.410342+0000","last_fullsized":"2024-04-14T08:34:34.410342+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406527+0000","last_change":"2024-04-14T08:34:34.406527+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406527+0000","last_undegraded":"2024-04-14T08:34:34.406527+0000","last_fullsized":"2024-04-14T08:34:34.406527+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408765+0000","last_change":"2024-04-14T08:34:34.408765+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408765+0000","last_undegraded":"2024-04-14T08:34:34.408765+0000","last_fullsized":"2024-04-14T08:34:34.408765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409655+0000","last_change":"2024-04-14T08:34:34.409655+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409655+0000","last_undegraded":"2024-04-14T08:34:34.409655+0000","last_fullsized":"2024-04-14T08:34:34.409655+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.438111+0000","last_change":"2024-04-14T08:34:35.431510+0000","last_active":"2024-04-14T08:34:35.438111+0000","last_peered":"2024-04-14T08:34:35.438111+0000","last_clean":"2024-04-14T08:34:35.438111+0000","last_became_active":"2024-04-14T08:34:35.430980+0000","last_became_peered":"2024-04-14T08:34:35.430980+0000","last_unstale":"2024-04-14T08:34:35.438111+0000","last_undegraded":"2024-04-14T08:34:35.438111+0000","last_fullsized":"2024-04-14T08:34:35.438111+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:13:28.555180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408105+0000","last_change":"2024-04-14T08:34:34.408105+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408105+0000","last_undegraded":"2024-04-14T08:34:34.408105+0000","last_fullsized":"2024-04-14T08:34:34.408105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.411421+0000","last_change":"2024-04-14T08:34:34.411421+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.411421+0000","last_undegraded":"2024-04-14T08:34:34.411421+0000","last_fullsized":"2024-04-14T08:34:34.411421+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.440794+0000","last_change":"2024-04-14T08:34:35.434489+0000","last_active":"2024-04-14T08:34:35.440794+0000","last_peered":"2024-04-14T08:34:35.440794+0000","last_clean":"2024-04-14T08:34:35.440794+0000","last_became_active":"2024-04-14T08:34:35.434142+0000","last_became_peered":"2024-04-14T08:34:35.434142+0000","last_unstale":"2024-04-14T08:34:35.440794+0000","last_undegraded":"2024-04-14T08:34:35.440794+0000","last_fullsized":"2024-04-14T08:34:35.440794+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:24:33.215747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.415922+0000","last_change":"2024-04-14T08:34:34.414663+0000","last_active":"2024-04-14T08:34:34.415922+0000","last_peered":"2024-04-14T08:34:34.415922+0000","last_clean":"2024-04-14T08:34:34.415922+0000","last_became_active":"2024-04-14T08:34:34.414390+0000","last_became_peered":"2024-04-14T08:34:34.414390+0000","last_unstale":"2024-04-14T08:34:34.415922+0000","last_undegraded":"2024-04-14T08:34:34.415922+0000","last_fullsized":"2024-04-14T08:34:34.415922+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:30:49.246797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409109+0000","last_change":"2024-04-14T08:34:34.409109+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409109+0000","last_undegraded":"2024-04-14T08:34:34.409109+0000","last_fullsized":"2024-04-14T08:34:34.409109+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.420847+0000","last_change":"2024-04-14T08:34:34.415460+0000","last_active":"2024-04-14T08:34:34.420847+0000","last_peered":"2024-04-14T08:34:34.420847+0000","last_clean":"2024-04-14T08:34:34.420847+0000","last_became_active":"2024-04-14T08:34:34.415176+0000","last_became_peered":"2024-04-14T08:34:34.415176+0000","last_unstale":"2024-04-14T08:34:34.420847+0000","last_undegraded":"2024-04-14T08:34:34.420847+0000","last_fullsized":"2024-04-14T08:34:34.420847+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:49.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410262+0000","last_change":"2024-04-14T08:34:34.410262+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410262+0000","last_undegraded":"2024-04-14T08:34:34.410262+0000","last_fullsized":"2024-04-14T08:34:34.410262+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.420592+0000","last_change":"2024-04-14T08:34:34.413412+0000","last_active":"2024-04-14T08:34:34.420592+0000","last_peered":"2024-04-14T08:34:34.420592+0000","last_clean":"2024-04-14T08:34:34.420592+0000","last_became_active":"2024-04-14T08:34:34.413050+0000","last_became_peered":"2024-04-14T08:34:34.413050+0000","last_unstale":"2024-04-14T08:34:34.420592+0000","last_undegraded":"2024-04-14T08:34:34.420592+0000","last_fullsized":"2024-04-14T08:34:34.420592+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:10.773823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410996+0000","last_change":"2024-04-14T08:34:34.410996+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410996+0000","last_undegraded":"2024-04-14T08:34:34.410996+0000","last_fullsized":"2024-04-14T08:34:34.410996+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.427275+0000","last_change":"2024-04-14T08:34:34.423624+0000","last_active":"2024-04-14T08:34:34.427275+0000","last_peered":"2024-04-14T08:34:34.427275+0000","last_clean":"2024-04-14T08:34:34.427275+0000","last_became_active":"2024-04-14T08:34:34.423373+0000","last_became_peered":"2024-04-14T08:34:34.423373+0000","last_unstale":"2024-04-14T08:34:34.427275+0000","last_undegraded":"2024-04-14T08:34:34.427275+0000","last_fullsized":"2024-04-14T08:34:34.427275+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406292+0000","last_change":"2024-04-14T08:34:34.406292+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406292+0000","last_undegraded":"2024-04-14T08:34:34.406292+0000","last_fullsized":"2024-04-14T08:34:34.406292+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424001+0000","last_change":"2024-04-14T08:34:34.420330+0000","last_active":"2024-04-14T08:34:34.424001+0000","last_peered":"2024-04-14T08:34:34.424001+0000","last_clean":"2024-04-14T08:34:34.424001+0000","last_became_active":"2024-04-14T08:34:34.419768+0000","last_became_peered":"2024-04-14T08:34:34.419768+0000","last_unstale":"2024-04-14T08:34:34.424001+0000","last_undegraded":"2024-04-14T08:34:34.424001+0000","last_fullsized":"2024-04-14T08:34:34.424001+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:23:36.394817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.439163+0000","last_change":"2024-04-14T08:34:35.431263+0000","last_active":"2024-04-14T08:34:35.439163+0000","last_peered":"2024-04-14T08:34:35.439163+0000","last_clean":"2024-04-14T08:34:35.439163+0000","last_became_active":"2024-04-14T08:34:35.431022+0000","last_became_peered":"2024-04-14T08:34:35.431022+0000","last_unstale":"2024-04-14T08:34:35.439163+0000","last_undegraded":"2024-04-14T08:34:35.439163+0000","last_fullsized":"2024-04-14T08:34:35.439163+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:56:32.524197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.416293+0000","last_change":"2024-04-14T08:34:34.413550+0000","last_active":"2024-04-14T08:34:34.416293+0000","last_peered":"2024-04-14T08:34:34.416293+0000","last_clean":"2024-04-14T08:34:34.416293+0000","last_became_active":"2024-04-14T08:34:34.413268+0000","last_became_peered":"2024-04-14T08:34:34.413268+0000","last_unstale":"2024-04-14T08:34:34.416293+0000","last_undegraded":"2024-04-14T08:34:34.416293+0000","last_fullsized":"2024-04-14T08:34:34.416293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:04.319595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408225+0000","last_change":"2024-04-14T08:34:34.408225+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408225+0000","last_undegraded":"2024-04-14T08:34:34.408225+0000","last_fullsized":"2024-04-14T08:34:34.408225+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.418469+0000","last_change":"2024-04-14T08:34:34.415171+0000","last_active":"2024-04-14T08:34:34.418469+0000","last_peered":"2024-04-14T08:34:34.418469+0000","last_clean":"2024-04-14T08:34:34.418469+0000","last_became_active":"2024-04-14T08:34:34.414890+0000","last_became_peered":"2024-04-14T08:34:34.414890+0000","last_unstale":"2024-04-14T08:34:34.418469+0000","last_undegraded":"2024-04-14T08:34:34.418469+0000","last_fullsized":"2024-04-14T08:34:34.418469+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:11.472608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.444176+0000","last_change":"2024-04-14T08:34:35.440848+0000","last_active":"2024-04-14T08:34:35.444176+0000","last_peered":"2024-04-14T08:34:35.444176+0000","last_clean":"2024-04-14T08:34:35.444176+0000","last_became_active":"2024-04-14T08:34:35.440519+0000","last_became_peered":"2024-04-14T08:34:35.440519+0000","last_unstale":"2024-04-14T08:34:35.444176+0000","last_undegraded":"2024-04-14T08:34:35.444176+0000","last_fullsized":"2024-04-14T08:34:35.444176+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:13:42.361716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424504+0000","last_change":"2024-04-14T08:34:34.419189+0000","last_active":"2024-04-14T08:34:34.424504+0000","last_peered":"2024-04-14T08:34:34.424504+0000","last_clean":"2024-04-14T08:34:34.424504+0000","last_became_active":"2024-04-14T08:34:34.418954+0000","last_became_peered":"2024-04-14T08:34:34.418954+0000","last_unstale":"2024-04-14T08:34:34.424504+0000","last_undegraded":"2024-04-14T08:34:34.424504+0000","last_fullsized":"2024-04-14T08:34:34.424504+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:54:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405634+0000","last_change":"2024-04-14T08:34:34.405634+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405634+0000","last_undegraded":"2024-04-14T08:34:34.405634+0000","last_fullsized":"2024-04-14T08:34:34.405634+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.419794+0000","last_change":"2024-04-14T08:34:34.416908+0000","last_active":"2024-04-14T08:34:35.419794+0000","last_peered":"2024-04-14T08:34:35.419794+0000","last_clean":"2024-04-14T08:34:35.419794+0000","last_became_active":"2024-04-14T08:34:34.416617+0000","last_became_peered":"2024-04-14T08:34:34.416617+0000","last_unstale":"2024-04-14T08:34:35.419794+0000","last_undegraded":"2024-04-14T08:34:35.419794+0000","last_fullsized":"2024-04-14T08:34:35.419794+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:11.743901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413953+0000","last_change":"2024-04-14T08:34:34.413953+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413953+0000","last_undegraded":"2024-04-14T08:34:34.413953+0000","last_fullsized":"2024-04-14T08:34:34.413953+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424753+0000","last_change":"2024-04-14T08:34:34.421136+0000","last_active":"2024-04-14T08:34:34.424753+0000","last_peered":"2024-04-14T08:34:34.424753+0000","last_clean":"2024-04-14T08:34:34.424753+0000","last_became_active":"2024-04-14T08:34:34.420809+0000","last_became_peered":"2024-04-14T08:34:34.420809+0000","last_unstale":"2024-04-14T08:34:34.424753+0000","last_undegraded":"2024-04-14T08:34:34.424753+0000","last_fullsized":"2024-04-14T08:34:34.424753+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:49.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.407542+0000","last_change":"2024-04-14T08:34:34.407542+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.407542+0000","last_undegraded":"2024-04-14T08:34:34.407542+0000","last_fullsized":"2024-04-14T08:34:34.407542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.448678+0000","last_change":"2024-04-14T08:34:34.422319+0000","last_active":"2024-04-14T08:34:35.448678+0000","last_peered":"2024-04-14T08:34:35.448678+0000","last_clean":"2024-04-14T08:34:35.448678+0000","last_became_active":"2024-04-14T08:34:34.421978+0000","last_became_peered":"2024-04-14T08:34:34.421978+0000","last_unstale":"2024-04-14T08:34:35.448678+0000","last_undegraded":"2024-04-14T08:34:35.448678+0000","last_fullsized":"2024-04-14T08:34:35.448678+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:11.381349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.404550+0000","last_change":"2024-04-14T08:34:34.404550+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.404550+0000","last_undegraded":"2024-04-14T08:34:34.404550+0000","last_fullsized":"2024-04-14T08:34:34.404550+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423937+0000","last_change":"2024-04-14T08:34:34.420518+0000","last_active":"2024-04-14T08:34:34.423937+0000","last_peered":"2024-04-14T08:34:34.423937+0000","last_clean":"2024-04-14T08:34:34.423937+0000","last_became_active":"2024-04-14T08:34:34.419065+0000","last_became_peered":"2024-04-14T08:34:34.419065+0000","last_unstale":"2024-04-14T08:34:34.423937+0000","last_undegraded":"2024-04-14T08:34:34.423937+0000","last_fullsized":"2024-04-14T08:34:34.423937+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:36.033184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410498+0000","last_change":"2024-04-14T08:34:34.410498+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410498+0000","last_undegraded":"2024-04-14T08:34:34.410498+0000","last_fullsized":"2024-04-14T08:34:34.410498+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423983+0000","last_change":"2024-04-14T08:34:34.421506+0000","last_active":"2024-04-14T08:34:34.423983+0000","last_peered":"2024-04-14T08:34:34.423983+0000","last_clean":"2024-04-14T08:34:34.423983+0000","last_became_active":"2024-04-14T08:34:34.420091+0000","last_became_peered":"2024-04-14T08:34:34.420091+0000","last_unstale":"2024-04-14T08:34:34.423983+0000","last_undegraded":"2024-04-14T08:34:34.423983+0000","last_fullsized":"2024-04-14T08:34:34.423983+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:30:35.547680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.412081+0000","last_change":"2024-04-14T08:34:34.412081+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.412081+0000","last_undegraded":"2024-04-14T08:34:34.412081+0000","last_fullsized":"2024-04-14T08:34:34.412081+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.429886+0000","last_change":"2024-04-14T08:34:34.424435+0000","last_active":"2024-04-14T08:34:34.429886+0000","last_peered":"2024-04-14T08:34:34.429886+0000","last_clean":"2024-04-14T08:34:34.429886+0000","last_became_active":"2024-04-14T08:34:34.424085+0000","last_became_peered":"2024-04-14T08:34:34.424085+0000","last_unstale":"2024-04-14T08:34:34.429886+0000","last_undegraded":"2024-04-14T08:34:34.429886+0000","last_fullsized":"2024-04-14T08:34:34.429886+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:04.319595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409928+0000","last_change":"2024-04-14T08:34:34.409928+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409928+0000","last_undegraded":"2024-04-14T08:34:34.409928+0000","last_fullsized":"2024-04-14T08:34:34.409928+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.419215+0000","last_change":"2024-04-14T08:34:34.414833+0000","last_active":"2024-04-14T08:34:35.419215+0000","last_peered":"2024-04-14T08:34:35.419215+0000","last_clean":"2024-04-14T08:34:35.419215+0000","last_became_active":"2024-04-14T08:34:34.414553+0000","last_became_peered":"2024-04-14T08:34:34.414553+0000","last_unstale":"2024-04-14T08:34:35.419215+0000","last_undegraded":"2024-04-14T08:34:35.419215+0000","last_fullsized":"2024-04-14T08:34:35.419215+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414459+0000","last_change":"2024-04-14T08:34:34.414459+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414459+0000","last_undegraded":"2024-04-14T08:34:34.414459+0000","last_fullsized":"2024-04-14T08:34:34.414459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.422034+0000","last_change":"2024-04-14T08:34:34.416091+0000","last_active":"2024-04-14T08:34:34.422034+0000","last_peered":"2024-04-14T08:34:34.422034+0000","last_clean":"2024-04-14T08:34:34.422034+0000","last_became_active":"2024-04-14T08:34:34.415653+0000","last_became_peered":"2024-04-14T08:34:34.415653+0000","last_unstale":"2024-04-14T08:34:34.422034+0000","last_undegraded":"2024-04-14T08:34:34.422034+0000","last_fullsized":"2024-04-14T08:34:34.422034+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.407664+0000","last_change":"2024-04-14T08:34:34.407664+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.407664+0000","last_undegraded":"2024-04-14T08:34:34.407664+0000","last_fullsized":"2024-04-14T08:34:34.407664+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.425676+0000","last_change":"2024-04-14T08:34:34.423763+0000","last_active":"2024-04-14T08:34:34.425676+0000","last_peered":"2024-04-14T08:34:34.425676+0000","last_clean":"2024-04-14T08:34:34.425676+0000","last_became_active":"2024-04-14T08:34:34.423333+0000","last_became_peered":"2024-04-14T08:34:34.423333+0000","last_unstale":"2024-04-14T08:34:34.425676+0000","last_undegraded":"2024-04-14T08:34:34.425676+0000","last_fullsized":"2024-04-14T08:34:34.425676+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:22:38.569478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405513+0000","last_change":"2024-04-14T08:34:34.405513+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405513+0000","last_undegraded":"2024-04-14T08:34:34.405513+0000","last_fullsized":"2024-04-14T08:34:34.405513+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.418045+0000","last_change":"2024-04-14T08:34:34.416569+0000","last_active":"2024-04-14T08:34:34.418045+0000","last_peered":"2024-04-14T08:34:34.418045+0000","last_clean":"2024-04-14T08:34:34.418045+0000","last_became_active":"2024-04-14T08:34:34.416306+0000","last_became_peered":"2024-04-14T08:34:34.416306+0000","last_unstale":"2024-04-14T08:34:34.418045+0000","last_undegraded":"2024-04-14T08:34:34.418045+0000","last_fullsized":"2024-04-14T08:34:34.418045+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:56.093503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405734+0000","last_change":"2024-04-14T08:34:34.405734+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405734+0000","last_undegraded":"2024-04-14T08:34:34.405734+0000","last_fullsized":"2024-04-14T08:34:34.405734+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.427399+0000","last_change":"2024-04-14T08:34:34.422659+0000","last_active":"2024-04-14T08:34:34.427399+0000","last_peered":"2024-04-14T08:34:34.427399+0000","last_clean":"2024-04-14T08:34:34.427399+0000","last_became_active":"2024-04-14T08:34:34.422434+0000","last_became_peered":"2024-04-14T08:34:34.422434+0000","last_unstale":"2024-04-14T08:34:34.427399+0000","last_undegraded":"2024-04-14T08:34:34.427399+0000","last_fullsized":"2024-04-14T08:34:34.427399+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:59:48.942079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405327+0000","last_change":"2024-04-14T08:34:34.405327+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405327+0000","last_undegraded":"2024-04-14T08:34:34.405327+0000","last_fullsized":"2024-04-14T08:34:34.405327+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.427365+0000","last_change":"2024-04-14T08:34:34.422117+0000","last_active":"2024-04-14T08:34:34.427365+0000","last_peered":"2024-04-14T08:34:34.427365+0000","last_clean":"2024-04-14T08:34:34.427365+0000","last_became_active":"2024-04-14T08:34:34.421504+0000","last_became_peered":"2024-04-14T08:34:34.421504+0000","last_unstale":"2024-04-14T08:34:34.427365+0000","last_undegraded":"2024-04-14T08:34:34.427365+0000","last_fullsized":"2024-04-14T08:34:34.427365+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:26:52.383137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409160+0000","last_change":"2024-04-14T08:34:34.409160+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409160+0000","last_undegraded":"2024-04-14T08:34:34.409160+0000","last_fullsized":"2024-04-14T08:34:34.409160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.420594+0000","last_change":"2024-04-14T08:34:34.413030+0000","last_active":"2024-04-14T08:34:34.420594+0000","last_peered":"2024-04-14T08:34:34.420594+0000","last_clean":"2024-04-14T08:34:34.420594+0000","last_became_active":"2024-04-14T08:34:34.412774+0000","last_became_peered":"2024-04-14T08:34:34.412774+0000","last_unstale":"2024-04-14T08:34:34.420594+0000","last_undegraded":"2024-04-14T08:34:34.420594+0000","last_fullsized":"2024-04-14T08:34:34.420594+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:04.319595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409571+0000","last_change":"2024-04-14T08:34:34.409571+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409571+0000","last_undegraded":"2024-04-14T08:34:34.409571+0000","last_fullsized":"2024-04-14T08:34:34.409571+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.421957+0000","last_change":"2024-04-14T08:34:34.417238+0000","last_active":"2024-04-14T08:34:34.421957+0000","last_peered":"2024-04-14T08:34:34.421957+0000","last_clean":"2024-04-14T08:34:34.421957+0000","last_became_active":"2024-04-14T08:34:34.417002+0000","last_became_peered":"2024-04-14T08:34:34.417002+0000","last_unstale":"2024-04-14T08:34:34.421957+0000","last_undegraded":"2024-04-14T08:34:34.421957+0000","last_fullsized":"2024-04-14T08:34:34.421957+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:11.743901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413056+0000","last_change":"2024-04-14T08:34:34.413056+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413056+0000","last_undegraded":"2024-04-14T08:34:34.413056+0000","last_fullsized":"2024-04-14T08:34:34.413056+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.425644+0000","last_change":"2024-04-14T08:34:34.423744+0000","last_active":"2024-04-14T08:34:34.425644+0000","last_peered":"2024-04-14T08:34:34.425644+0000","last_clean":"2024-04-14T08:34:34.425644+0000","last_became_active":"2024-04-14T08:34:34.423394+0000","last_became_peered":"2024-04-14T08:34:34.423394+0000","last_unstale":"2024-04-14T08:34:34.425644+0000","last_undegraded":"2024-04-14T08:34:34.425644+0000","last_fullsized":"2024-04-14T08:34:34.425644+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:32.608024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406093+0000","last_change":"2024-04-14T08:34:34.406093+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406093+0000","last_undegraded":"2024-04-14T08:34:34.406093+0000","last_fullsized":"2024-04-14T08:34:34.406093+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.414356+0000","last_change":"2024-04-14T08:34:34.411096+0000","last_active":"2024-04-14T08:34:34.414356+0000","last_peered":"2024-04-14T08:34:34.414356+0000","last_clean":"2024-04-14T08:34:34.414356+0000","last_became_active":"2024-04-14T08:34:34.410841+0000","last_became_peered":"2024-04-14T08:34:34.410841+0000","last_unstale":"2024-04-14T08:34:34.414356+0000","last_undegraded":"2024-04-14T08:34:34.414356+0000","last_fullsized":"2024-04-14T08:34:34.414356+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:12:26.461355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410965+0000","last_change":"2024-04-14T08:34:34.410965+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410965+0000","last_undegraded":"2024-04-14T08:34:34.410965+0000","last_fullsized":"2024-04-14T08:34:34.410965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.429860+0000","last_change":"2024-04-14T08:34:34.424338+0000","last_active":"2024-04-14T08:34:34.429860+0000","last_peered":"2024-04-14T08:34:34.429860+0000","last_clean":"2024-04-14T08:34:34.429860+0000","last_became_active":"2024-04-14T08:34:34.424057+0000","last_became_peered":"2024-04-14T08:34:34.424057+0000","last_unstale":"2024-04-14T08:34:34.429860+0000","last_undegraded":"2024-04-14T08:34:34.429860+0000","last_fullsized":"2024-04-14T08:34:34.429860+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:10.773823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406288+0000","last_change":"2024-04-14T08:34:34.406288+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406288+0000","last_undegraded":"2024-04-14T08:34:34.406288+0000","last_fullsized":"2024-04-14T08:34:34.406288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.420306+0000","last_change":"2024-04-14T08:34:34.416682+0000","last_active":"2024-04-14T08:34:35.420306+0000","last_peered":"2024-04-14T08:34:35.420306+0000","last_clean":"2024-04-14T08:34:35.420306+0000","last_became_active":"2024-04-14T08:34:34.416172+0000","last_became_peered":"2024-04-14T08:34:34.416172+0000","last_unstale":"2024-04-14T08:34:35.420306+0000","last_undegraded":"2024-04-14T08:34:35.420306+0000","last_fullsized":"2024-04-14T08:34:35.420306+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:36.033184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408713+0000","last_change":"2024-04-14T08:34:34.408713+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408713+0000","last_undegraded":"2024-04-14T08:34:34.408713+0000","last_fullsized":"2024-04-14T08:34:34.408713+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423234+0000","last_change":"2024-04-14T08:34:34.420330+0000","last_active":"2024-04-14T08:34:34.423234+0000","last_peered":"2024-04-14T08:34:34.423234+0000","last_clean":"2024-04-14T08:34:34.423234+0000","last_became_active":"2024-04-14T08:34:34.420116+0000","last_became_peered":"2024-04-14T08:34:34.420116+0000","last_unstale":"2024-04-14T08:34:34.423234+0000","last_undegraded":"2024-04-14T08:34:34.423234+0000","last_fullsized":"2024-04-14T08:34:34.423234+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:41:19.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408837+0000","last_change":"2024-04-14T08:34:34.408837+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408837+0000","last_undegraded":"2024-04-14T08:34:34.408837+0000","last_fullsized":"2024-04-14T08:34:34.408837+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.422091+0000","last_change":"2024-04-14T08:34:34.419528+0000","last_active":"2024-04-14T08:34:34.422091+0000","last_peered":"2024-04-14T08:34:34.422091+0000","last_clean":"2024-04-14T08:34:34.422091+0000","last_became_active":"2024-04-14T08:34:34.419258+0000","last_became_peered":"2024-04-14T08:34:34.419258+0000","last_unstale":"2024-04-14T08:34:34.422091+0000","last_undegraded":"2024-04-14T08:34:34.422091+0000","last_fullsized":"2024-04-14T08:34:34.422091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:11.381349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409171+0000","last_change":"2024-04-14T08:34:34.409171+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409171+0000","last_undegraded":"2024-04-14T08:34:34.409171+0000","last_fullsized":"2024-04-14T08:34:34.409171+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423956+0000","last_change":"2024-04-14T08:34:34.421662+0000","last_active":"2024-04-14T08:34:34.423956+0000","last_peered":"2024-04-14T08:34:34.423956+0000","last_clean":"2024-04-14T08:34:34.423956+0000","last_became_active":"2024-04-14T08:34:34.421382+0000","last_became_peered":"2024-04-14T08:34:34.421382+0000","last_unstale":"2024-04-14T08:34:34.423956+0000","last_undegraded":"2024-04-14T08:34:34.423956+0000","last_fullsized":"2024-04-14T08:34:34.423956+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:55:44.175481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413461+0000","last_change":"2024-04-14T08:34:34.413461+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413461+0000","last_undegraded":"2024-04-14T08:34:34.413461+0000","last_fullsized":"2024-04-14T08:34:34.413461+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424542+0000","last_change":"2024-04-14T08:34:34.419857+0000","last_active":"2024-04-14T08:34:34.424542+0000","last_peered":"2024-04-14T08:34:34.424542+0000","last_clean":"2024-04-14T08:34:34.424542+0000","last_became_active":"2024-04-14T08:34:34.419548+0000","last_became_peered":"2024-04-14T08:34:34.419548+0000","last_unstale":"2024-04-14T08:34:34.424542+0000","last_undegraded":"2024-04-14T08:34:34.424542+0000","last_fullsized":"2024-04-14T08:34:34.424542+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:10.773823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408288+0000","last_change":"2024-04-14T08:34:34.408288+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408288+0000","last_undegraded":"2024-04-14T08:34:34.408288+0000","last_fullsized":"2024-04-14T08:34:34.408288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424716+0000","last_change":"2024-04-14T08:34:34.420316+0000","last_active":"2024-04-14T08:34:34.424716+0000","last_peered":"2024-04-14T08:34:34.424716+0000","last_clean":"2024-04-14T08:34:34.424716+0000","last_became_active":"2024-04-14T08:34:34.420093+0000","last_became_peered":"2024-04-14T08:34:34.420093+0000","last_unstale":"2024-04-14T08:34:34.424716+0000","last_undegraded":"2024-04-14T08:34:34.424716+0000","last_fullsized":"2024-04-14T08:34:34.424716+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:44:18.315252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408798+0000","last_change":"2024-04-14T08:34:34.408798+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408798+0000","last_undegraded":"2024-04-14T08:34:34.408798+0000","last_fullsized":"2024-04-14T08:34:34.408798+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423952+0000","last_change":"2024-04-14T08:34:34.421158+0000","last_active":"2024-04-14T08:34:34.423952+0000","last_peered":"2024-04-14T08:34:34.423952+0000","last_clean":"2024-04-14T08:34:34.423952+0000","last_became_active":"2024-04-14T08:34:34.420780+0000","last_became_peered":"2024-04-14T08:34:34.420780+0000","last_unstale":"2024-04-14T08:34:34.423952+0000","last_undegraded":"2024-04-14T08:34:34.423952+0000","last_fullsized":"2024-04-14T08:34:34.423952+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:44:10.910055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":226,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.418795+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:34:35.418795+0000","last_peered":"2024-04-14T08:34:35.418795+0000","last_clean":"2024-04-14T08:34:35.418795+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:34:35.418795+0000","last_undegraded":"2024-04-14T08:34:35.418795+0000","last_fullsized":"2024-04-14T08:34:35.418795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1183744,"data_stored":846191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":557056,"data_stored":241865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1163264,"data_stored":838900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:34:38.791 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:34:38.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.927+0000 7f8505036640 1 Processor -- start 2024-04-14T08:34:38.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.927+0000 7f8505036640 1 -- start start 2024-04-14T08:34:38.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.928+0000 7f8505036640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f850015c710 0x7f85000d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.928+0000 7f8505036640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000cde10 0x7f85000ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.928+0000 7f8505036640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85000aef60 con 0x7f85000ced80 2024-04-14T08:34:38.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.928+0000 7f8505036640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f85000aed10 con 0x7f850015c710 2024-04-14T08:34:38.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.928+0000 7f8505036640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f85000af890 con 0x7f85000cde10 2024-04-14T08:34:38.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.928+0000 7f84fed9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f850015c710 0x7f85000d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.928+0000 7f84fed9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f850015c710 0x7f85000d0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48218/0 (socket says 172.21.15.172:48218) 2024-04-14T08:34:38.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.928+0000 7f84fed9d640 1 -- 172.21.15.172:0/2068890561 learned_addr learned my addr 172.21.15.172:0/2068890561 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:38.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.929+0000 7f84fe59c640 1 --2- 172.21.15.172:0/2068890561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000cde10 0x7f85000ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.929+0000 7f84fed9d640 1 -- 172.21.15.172:0/2068890561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000cde10 msgr2=0x7f85000ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:38.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.929+0000 7f84fed9d640 1 --2- 172.21.15.172:0/2068890561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000cde10 0x7f85000ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.929+0000 7f84fed9d640 1 -- 172.21.15.172:0/2068890561 >> v1:172.21.15.177:6789/0 conn(0x7f85000ced80 legacy=0x7f85000ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:38.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.929+0000 7f84fed9d640 1 -- 172.21.15.172:0/2068890561 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85000af240 con 0x7f850015c710 2024-04-14T08:34:38.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.929+0000 7f84fe59c640 1 --2- 172.21.15.172:0/2068890561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000cde10 0x7f85000ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:38.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.929+0000 7f84fed9d640 1 --2- 172.21.15.172:0/2068890561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850015c710 0x7f85000d0950 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009870 tx=0x7f84f802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3216798dfab383b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:38.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.929+0000 7f84fdd9b640 1 -- 172.21.15.172:0/2068890561 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8035020 con 0x7f850015c710 2024-04-14T08:34:38.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.930+0000 7f84fdd9b640 1 -- 172.21.15.172:0/2068890561 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f8031640 con 0x7f850015c710 2024-04-14T08:34:38.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.930+0000 7f84fdd9b640 1 -- 172.21.15.172:0/2068890561 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f802d040 con 0x7f850015c710 2024-04-14T08:34:38.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.930+0000 7f8505036640 1 -- 172.21.15.172:0/2068890561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850015c710 msgr2=0x7f85000d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.930+0000 7f8505036640 1 --2- 172.21.15.172:0/2068890561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850015c710 0x7f85000d0950 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009870 tx=0x7f84f802b8f0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.930+0000 7f8505036640 1 -- 172.21.15.172:0/2068890561 shutdown_connections 2024-04-14T08:34:38.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.930+0000 7f8505036640 1 --2- 172.21.15.172:0/2068890561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000cde10 0x7f85000ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.930+0000 7f8505036640 1 --2- 172.21.15.172:0/2068890561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850015c710 0x7f85000d0950 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.930+0000 7f8505036640 1 -- 172.21.15.172:0/2068890561 >> 172.21.15.172:0/2068890561 conn(0x7f850008c000 msgr2=0x7f85000d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:38.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.930+0000 7f8505036640 1 -- 172.21.15.172:0/2068890561 shutdown_connections 2024-04-14T08:34:38.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.930+0000 7f8505036640 1 -- 172.21.15.172:0/2068890561 wait complete. 2024-04-14T08:34:38.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.931+0000 7f8505036640 1 Processor -- start 2024-04-14T08:34:38.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.932+0000 7f8505036640 1 -- start start 2024-04-14T08:34:38.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.932+0000 7f8505036640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85000cde10 0x7f850014b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.932+0000 7f8505036640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000ced80 0x7f850015dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.932+0000 7f8505036640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850015c710 0x7f85001e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.932+0000 7f8505036640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85000d0e90 con 0x7f850015c710 2024-04-14T08:34:38.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.932+0000 7f8505036640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f850015caf0 con 0x7f85000cde10 2024-04-14T08:34:38.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.932+0000 7f8505036640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f85000d1010 con 0x7f85000ced80 2024-04-14T08:34:38.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.932+0000 7f84ff59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850015c710 0x7f85001e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.932+0000 7f84ff59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850015c710 0x7f85001e90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48232/0 (socket says 172.21.15.172:48232) 2024-04-14T08:34:38.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.932+0000 7f84ff59e640 1 -- 172.21.15.172:0/3651955729 learned_addr learned my addr 172.21.15.172:0/3651955729 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:38.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f84fed9d640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85000cde10 0x7f850014b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f84fe59c640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000ced80 0x7f850015dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f84ff59e640 1 -- 172.21.15.172:0/3651955729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000ced80 msgr2=0x7f850015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f84ff59e640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000ced80 0x7f850015dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f84ff59e640 1 -- 172.21.15.172:0/3651955729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85000cde10 msgr2=0x7f850014b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:38.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f84ff59e640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85000cde10 0x7f850014b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:38.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f84ff59e640 1 -- 172.21.15.172:0/3651955729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850015cea0 con 0x7f850015c710 2024-04-14T08:34:38.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f84ff59e640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850015c710 0x7f85001e90e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f84f000c780 tx=0x7f84f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:38.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f84f77fe640 1 -- 172.21.15.172:0/3651955729 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0004b60 con 0x7f850015c710 2024-04-14T08:34:38.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f84f77fe640 1 -- 172.21.15.172:0/3651955729 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f0004d00 con 0x7f850015c710 2024-04-14T08:34:38.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f8505036640 1 -- 172.21.15.172:0/3651955729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850015d180 con 0x7f850015c710 2024-04-14T08:34:38.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f84f77fe640 1 -- 172.21.15.172:0/3651955729 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f000aa30 con 0x7f850015c710 2024-04-14T08:34:38.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.933+0000 7f8505036640 1 -- 172.21.15.172:0/3651955729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850014b940 con 0x7f850015c710 2024-04-14T08:34:38.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.934+0000 7f8505036640 1 -- 172.21.15.172:0/3651955729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8500075490 con 0x7f850015c710 2024-04-14T08:34:38.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.934+0000 7f84f77fe640 1 -- 172.21.15.172:0/3651955729 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84f000abd0 con 0x7f850015c710 2024-04-14T08:34:38.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.935+0000 7f84f77fe640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84e006c8f0 0x7f84e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:38.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.935+0000 7f84f77fe640 1 -- 172.21.15.172:0/3651955729 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f84f008be60 con 0x7f850015c710 2024-04-14T08:34:38.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.935+0000 7f84fed9d640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84e006c8f0 0x7f84e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:38.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.935+0000 7f84fed9d640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84e006c8f0 0x7f84e006edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f84f8036000 tx=0x7f84f803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:38.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:38.939+0000 7f84f77fe640 1 -- 172.21.15.172:0/3651955729 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f84f0020070 con 0x7f850015c710 2024-04-14T08:34:39.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.148+0000 7f8505036640 1 -- 172.21.15.172:0/3651955729 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85000b0e50 con 0x7f84e006c8f0 2024-04-14T08:34:39.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.154+0000 7f84f77fe640 1 -- 172.21.15.172:0/3651955729 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247296 (secure 0 0 0) 0x7f85000b0e50 con 0x7f84e006c8f0 2024-04-14T08:34:39.158 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:39.159 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:34:39.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.157+0000 7f8505036640 1 -- 172.21.15.172:0/3651955729 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84e006c8f0 msgr2=0x7f84e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.157+0000 7f8505036640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84e006c8f0 0x7f84e006edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f84f8036000 tx=0x7f84f803c040 comp rx=0 tx=0).stop 2024-04-14T08:34:39.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.157+0000 7f8505036640 1 -- 172.21.15.172:0/3651955729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850015c710 msgr2=0x7f85001e90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.157+0000 7f8505036640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850015c710 0x7f85001e90e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f84f000c780 tx=0x7f84f000cc40 comp rx=0 tx=0).stop 2024-04-14T08:34:39.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.157+0000 7f8505036640 1 -- 172.21.15.172:0/3651955729 shutdown_connections 2024-04-14T08:34:39.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.157+0000 7f8505036640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84e006c8f0 0x7f84e006edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.157+0000 7f8505036640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850015c710 0x7f85001e90e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.157+0000 7f8505036640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000ced80 0x7f850015dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.157+0000 7f8505036640 1 --2- 172.21.15.172:0/3651955729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85000cde10 0x7f850014b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.157+0000 7f8505036640 1 -- 172.21.15.172:0/3651955729 >> 172.21.15.172:0/3651955729 conn(0x7f850008c000 msgr2=0x7f8500070bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:39.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.157+0000 7f8505036640 1 -- 172.21.15.172:0/3651955729 shutdown_connections 2024-04-14T08:34:39.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.157+0000 7f8505036640 1 -- 172.21.15.172:0/3651955729 wait complete. 2024-04-14T08:34:39.170 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-04-14T08:34:39.141054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221808,"kb_used_data":5616,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752912,"statfs":{"total":773094113280,"available":772866981888,"internally_reserved":0,"allocated":5750784,"data_stored":3169616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.741238"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408023+0000","last_change":"2024-04-14T08:34:34.408023+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408023+0000","last_undegraded":"2024-04-14T08:34:34.408023+0000","last_fullsized":"2024-04-14T08:34:34.408023+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.441184+0000","last_change":"2024-04-14T08:34:35.434456+0000","last_active":"2024-04-14T08:34:35.441184+0000","last_peered":"2024-04-14T08:34:35.441184+0000","last_clean":"2024-04-14T08:34:35.441184+0000","last_became_active":"2024-04-14T08:34:35.434158+0000","last_became_peered":"2024-04-14T08:34:35.434158+0000","last_unstale":"2024-04-14T08:34:35.441184+0000","last_undegraded":"2024-04-14T08:34:35.441184+0000","last_fullsized":"2024-04-14T08:34:35.441184+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:40:13.833057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414650+0000","last_change":"2024-04-14T08:34:34.414650+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414650+0000","last_undegraded":"2024-04-14T08:34:34.414650+0000","last_fullsized":"2024-04-14T08:34:34.414650+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.437687+0000","last_change":"2024-04-14T08:34:35.430927+0000","last_active":"2024-04-14T08:34:35.437687+0000","last_peered":"2024-04-14T08:34:35.437687+0000","last_clean":"2024-04-14T08:34:35.437687+0000","last_became_active":"2024-04-14T08:34:35.430164+0000","last_became_peered":"2024-04-14T08:34:35.430164+0000","last_unstale":"2024-04-14T08:34:35.437687+0000","last_undegraded":"2024-04-14T08:34:35.437687+0000","last_fullsized":"2024-04-14T08:34:35.437687+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:57:15.656326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405062+0000","last_change":"2024-04-14T08:34:34.405062+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405062+0000","last_undegraded":"2024-04-14T08:34:34.405062+0000","last_fullsized":"2024-04-14T08:34:34.405062+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406804+0000","last_change":"2024-04-14T08:34:34.406804+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406804+0000","last_undegraded":"2024-04-14T08:34:34.406804+0000","last_fullsized":"2024-04-14T08:34:34.406804+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414181+0000","last_change":"2024-04-14T08:34:34.414181+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414181+0000","last_undegraded":"2024-04-14T08:34:34.414181+0000","last_fullsized":"2024-04-14T08:34:34.414181+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409156+0000","last_change":"2024-04-14T08:34:34.409156+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409156+0000","last_undegraded":"2024-04-14T08:34:34.409156+0000","last_fullsized":"2024-04-14T08:34:34.409156+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410003+0000","last_change":"2024-04-14T08:34:34.410003+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410003+0000","last_undegraded":"2024-04-14T08:34:34.410003+0000","last_fullsized":"2024-04-14T08:34:34.410003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406692+0000","last_change":"2024-04-14T08:34:34.406692+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406692+0000","last_undegraded":"2024-04-14T08:34:34.406692+0000","last_fullsized":"2024-04-14T08:34:34.406692+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405927+0000","last_change":"2024-04-14T08:34:34.405927+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405927+0000","last_undegraded":"2024-04-14T08:34:34.405927+0000","last_fullsized":"2024-04-14T08:34:34.405927+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410934+0000","last_change":"2024-04-14T08:34:34.410934+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410934+0000","last_undegraded":"2024-04-14T08:34:34.410934+0000","last_fullsized":"2024-04-14T08:34:34.410934+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406225+0000","last_change":"2024-04-14T08:34:34.406225+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406225+0000","last_undegraded":"2024-04-14T08:34:34.406225+0000","last_fullsized":"2024-04-14T08:34:34.406225+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.411333+0000","last_change":"2024-04-14T08:34:34.411333+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.411333+0000","last_undegraded":"2024-04-14T08:34:34.411333+0000","last_fullsized":"2024-04-14T08:34:34.411333+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.437809+0000","last_change":"2024-04-14T08:34:35.428986+0000","last_active":"2024-04-14T08:34:35.437809+0000","last_peered":"2024-04-14T08:34:35.437809+0000","last_clean":"2024-04-14T08:34:35.437809+0000","last_became_active":"2024-04-14T08:34:35.426840+0000","last_became_peered":"2024-04-14T08:34:35.426840+0000","last_unstale":"2024-04-14T08:34:35.437809+0000","last_undegraded":"2024-04-14T08:34:35.437809+0000","last_fullsized":"2024-04-14T08:34:35.437809+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:55:45.179250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406390+0000","last_change":"2024-04-14T08:34:34.406390+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406390+0000","last_undegraded":"2024-04-14T08:34:34.406390+0000","last_fullsized":"2024-04-14T08:34:34.406390+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414078+0000","last_change":"2024-04-14T08:34:34.414078+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414078+0000","last_undegraded":"2024-04-14T08:34:34.414078+0000","last_fullsized":"2024-04-14T08:34:34.414078+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.441163+0000","last_change":"2024-04-14T08:34:35.434292+0000","last_active":"2024-04-14T08:34:35.441163+0000","last_peered":"2024-04-14T08:34:35.441163+0000","last_clean":"2024-04-14T08:34:35.441163+0000","last_became_active":"2024-04-14T08:34:35.434018+0000","last_became_peered":"2024-04-14T08:34:35.434018+0000","last_unstale":"2024-04-14T08:34:35.441163+0000","last_undegraded":"2024-04-14T08:34:35.441163+0000","last_fullsized":"2024-04-14T08:34:35.441163+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:22:04.896638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.412491+0000","last_change":"2024-04-14T08:34:34.412491+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.412491+0000","last_undegraded":"2024-04-14T08:34:34.412491+0000","last_fullsized":"2024-04-14T08:34:34.412491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410141+0000","last_change":"2024-04-14T08:34:34.410141+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410141+0000","last_undegraded":"2024-04-14T08:34:34.410141+0000","last_fullsized":"2024-04-14T08:34:34.410141+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408219+0000","last_change":"2024-04-14T08:34:34.408219+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408219+0000","last_undegraded":"2024-04-14T08:34:34.408219+0000","last_fullsized":"2024-04-14T08:34:34.408219+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.407965+0000","last_change":"2024-04-14T08:34:34.407965+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.407965+0000","last_undegraded":"2024-04-14T08:34:34.407965+0000","last_fullsized":"2024-04-14T08:34:34.407965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405857+0000","last_change":"2024-04-14T08:34:34.405857+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405857+0000","last_undegraded":"2024-04-14T08:34:34.405857+0000","last_fullsized":"2024-04-14T08:34:34.405857+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410886+0000","last_change":"2024-04-14T08:34:34.410886+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410886+0000","last_undegraded":"2024-04-14T08:34:34.410886+0000","last_fullsized":"2024-04-14T08:34:34.410886+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413719+0000","last_change":"2024-04-14T08:34:34.413719+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413719+0000","last_undegraded":"2024-04-14T08:34:34.413719+0000","last_fullsized":"2024-04-14T08:34:34.413719+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410342+0000","last_change":"2024-04-14T08:34:34.410342+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410342+0000","last_undegraded":"2024-04-14T08:34:34.410342+0000","last_fullsized":"2024-04-14T08:34:34.410342+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406527+0000","last_change":"2024-04-14T08:34:34.406527+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406527+0000","last_undegraded":"2024-04-14T08:34:34.406527+0000","last_fullsized":"2024-04-14T08:34:34.406527+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408765+0000","last_change":"2024-04-14T08:34:34.408765+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408765+0000","last_undegraded":"2024-04-14T08:34:34.408765+0000","last_fullsized":"2024-04-14T08:34:34.408765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409655+0000","last_change":"2024-04-14T08:34:34.409655+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409655+0000","last_undegraded":"2024-04-14T08:34:34.409655+0000","last_fullsized":"2024-04-14T08:34:34.409655+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.438111+0000","last_change":"2024-04-14T08:34:35.431510+0000","last_active":"2024-04-14T08:34:35.438111+0000","last_peered":"2024-04-14T08:34:35.438111+0000","last_clean":"2024-04-14T08:34:35.438111+0000","last_became_active":"2024-04-14T08:34:35.430980+0000","last_became_peered":"2024-04-14T08:34:35.430980+0000","last_unstale":"2024-04-14T08:34:35.438111+0000","last_undegraded":"2024-04-14T08:34:35.438111+0000","last_fullsized":"2024-04-14T08:34:35.438111+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:13:28.555180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408105+0000","last_change":"2024-04-14T08:34:34.408105+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408105+0000","last_undegraded":"2024-04-14T08:34:34.408105+0000","last_fullsized":"2024-04-14T08:34:34.408105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.411421+0000","last_change":"2024-04-14T08:34:34.411421+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.411421+0000","last_undegraded":"2024-04-14T08:34:34.411421+0000","last_fullsized":"2024-04-14T08:34:34.411421+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.440794+0000","last_change":"2024-04-14T08:34:35.434489+0000","last_active":"2024-04-14T08:34:35.440794+0000","last_peered":"2024-04-14T08:34:35.440794+0000","last_clean":"2024-04-14T08:34:35.440794+0000","last_became_active":"2024-04-14T08:34:35.434142+0000","last_became_peered":"2024-04-14T08:34:35.434142+0000","last_unstale":"2024-04-14T08:34:35.440794+0000","last_undegraded":"2024-04-14T08:34:35.440794+0000","last_fullsized":"2024-04-14T08:34:35.440794+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:24:33.215747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.415922+0000","last_change":"2024-04-14T08:34:34.414663+0000","last_active":"2024-04-14T08:34:34.415922+0000","last_peered":"2024-04-14T08:34:34.415922+0000","last_clean":"2024-04-14T08:34:34.415922+0000","last_became_active":"2024-04-14T08:34:34.414390+0000","last_became_peered":"2024-04-14T08:34:34.414390+0000","last_unstale":"2024-04-14T08:34:34.415922+0000","last_undegraded":"2024-04-14T08:34:34.415922+0000","last_fullsized":"2024-04-14T08:34:34.415922+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:30:49.246797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409109+0000","last_change":"2024-04-14T08:34:34.409109+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409109+0000","last_undegraded":"2024-04-14T08:34:34.409109+0000","last_fullsized":"2024-04-14T08:34:34.409109+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.420847+0000","last_change":"2024-04-14T08:34:34.415460+0000","last_active":"2024-04-14T08:34:34.420847+0000","last_peered":"2024-04-14T08:34:34.420847+0000","last_clean":"2024-04-14T08:34:34.420847+0000","last_became_active":"2024-04-14T08:34:34.415176+0000","last_became_peered":"2024-04-14T08:34:34.415176+0000","last_unstale":"2024-04-14T08:34:34.420847+0000","last_undegraded":"2024-04-14T08:34:34.420847+0000","last_fullsized":"2024-04-14T08:34:34.420847+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:49.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410262+0000","last_change":"2024-04-14T08:34:34.410262+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410262+0000","last_undegraded":"2024-04-14T08:34:34.410262+0000","last_fullsized":"2024-04-14T08:34:34.410262+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.420592+0000","last_change":"2024-04-14T08:34:34.413412+0000","last_active":"2024-04-14T08:34:34.420592+0000","last_peered":"2024-04-14T08:34:34.420592+0000","last_clean":"2024-04-14T08:34:34.420592+0000","last_became_active":"2024-04-14T08:34:34.413050+0000","last_became_peered":"2024-04-14T08:34:34.413050+0000","last_unstale":"2024-04-14T08:34:34.420592+0000","last_undegraded":"2024-04-14T08:34:34.420592+0000","last_fullsized":"2024-04-14T08:34:34.420592+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:10.773823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410996+0000","last_change":"2024-04-14T08:34:34.410996+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410996+0000","last_undegraded":"2024-04-14T08:34:34.410996+0000","last_fullsized":"2024-04-14T08:34:34.410996+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.427275+0000","last_change":"2024-04-14T08:34:34.423624+0000","last_active":"2024-04-14T08:34:34.427275+0000","last_peered":"2024-04-14T08:34:34.427275+0000","last_clean":"2024-04-14T08:34:34.427275+0000","last_became_active":"2024-04-14T08:34:34.423373+0000","last_became_peered":"2024-04-14T08:34:34.423373+0000","last_unstale":"2024-04-14T08:34:34.427275+0000","last_undegraded":"2024-04-14T08:34:34.427275+0000","last_fullsized":"2024-04-14T08:34:34.427275+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406292+0000","last_change":"2024-04-14T08:34:34.406292+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406292+0000","last_undegraded":"2024-04-14T08:34:34.406292+0000","last_fullsized":"2024-04-14T08:34:34.406292+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424001+0000","last_change":"2024-04-14T08:34:34.420330+0000","last_active":"2024-04-14T08:34:34.424001+0000","last_peered":"2024-04-14T08:34:34.424001+0000","last_clean":"2024-04-14T08:34:34.424001+0000","last_became_active":"2024-04-14T08:34:34.419768+0000","last_became_peered":"2024-04-14T08:34:34.419768+0000","last_unstale":"2024-04-14T08:34:34.424001+0000","last_undegraded":"2024-04-14T08:34:34.424001+0000","last_fullsized":"2024-04-14T08:34:34.424001+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:23:36.394817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.439163+0000","last_change":"2024-04-14T08:34:35.431263+0000","last_active":"2024-04-14T08:34:35.439163+0000","last_peered":"2024-04-14T08:34:35.439163+0000","last_clean":"2024-04-14T08:34:35.439163+0000","last_became_active":"2024-04-14T08:34:35.431022+0000","last_became_peered":"2024-04-14T08:34:35.431022+0000","last_unstale":"2024-04-14T08:34:35.439163+0000","last_undegraded":"2024-04-14T08:34:35.439163+0000","last_fullsized":"2024-04-14T08:34:35.439163+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:56:32.524197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.416293+0000","last_change":"2024-04-14T08:34:34.413550+0000","last_active":"2024-04-14T08:34:34.416293+0000","last_peered":"2024-04-14T08:34:34.416293+0000","last_clean":"2024-04-14T08:34:34.416293+0000","last_became_active":"2024-04-14T08:34:34.413268+0000","last_became_peered":"2024-04-14T08:34:34.413268+0000","last_unstale":"2024-04-14T08:34:34.416293+0000","last_undegraded":"2024-04-14T08:34:34.416293+0000","last_fullsized":"2024-04-14T08:34:34.416293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:04.319595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408225+0000","last_change":"2024-04-14T08:34:34.408225+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408225+0000","last_undegraded":"2024-04-14T08:34:34.408225+0000","last_fullsized":"2024-04-14T08:34:34.408225+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.418469+0000","last_change":"2024-04-14T08:34:34.415171+0000","last_active":"2024-04-14T08:34:34.418469+0000","last_peered":"2024-04-14T08:34:34.418469+0000","last_clean":"2024-04-14T08:34:34.418469+0000","last_became_active":"2024-04-14T08:34:34.414890+0000","last_became_peered":"2024-04-14T08:34:34.414890+0000","last_unstale":"2024-04-14T08:34:34.418469+0000","last_undegraded":"2024-04-14T08:34:34.418469+0000","last_fullsized":"2024-04-14T08:34:34.418469+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:11.472608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.444176+0000","last_change":"2024-04-14T08:34:35.440848+0000","last_active":"2024-04-14T08:34:35.444176+0000","last_peered":"2024-04-14T08:34:35.444176+0000","last_clean":"2024-04-14T08:34:35.444176+0000","last_became_active":"2024-04-14T08:34:35.440519+0000","last_became_peered":"2024-04-14T08:34:35.440519+0000","last_unstale":"2024-04-14T08:34:35.444176+0000","last_undegraded":"2024-04-14T08:34:35.444176+0000","last_fullsized":"2024-04-14T08:34:35.444176+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:13:42.361716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424504+0000","last_change":"2024-04-14T08:34:34.419189+0000","last_active":"2024-04-14T08:34:34.424504+0000","last_peered":"2024-04-14T08:34:34.424504+0000","last_clean":"2024-04-14T08:34:34.424504+0000","last_became_active":"2024-04-14T08:34:34.418954+0000","last_became_peered":"2024-04-14T08:34:34.418954+0000","last_unstale":"2024-04-14T08:34:34.424504+0000","last_undegraded":"2024-04-14T08:34:34.424504+0000","last_fullsized":"2024-04-14T08:34:34.424504+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:54:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405634+0000","last_change":"2024-04-14T08:34:34.405634+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405634+0000","last_undegraded":"2024-04-14T08:34:34.405634+0000","last_fullsized":"2024-04-14T08:34:34.405634+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.419794+0000","last_change":"2024-04-14T08:34:34.416908+0000","last_active":"2024-04-14T08:34:35.419794+0000","last_peered":"2024-04-14T08:34:35.419794+0000","last_clean":"2024-04-14T08:34:35.419794+0000","last_became_active":"2024-04-14T08:34:34.416617+0000","last_became_peered":"2024-04-14T08:34:34.416617+0000","last_unstale":"2024-04-14T08:34:35.419794+0000","last_undegraded":"2024-04-14T08:34:35.419794+0000","last_fullsized":"2024-04-14T08:34:35.419794+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:11.743901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413953+0000","last_change":"2024-04-14T08:34:34.413953+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413953+0000","last_undegraded":"2024-04-14T08:34:34.413953+0000","last_fullsized":"2024-04-14T08:34:34.413953+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424753+0000","last_change":"2024-04-14T08:34:34.421136+0000","last_active":"2024-04-14T08:34:34.424753+0000","last_peered":"2024-04-14T08:34:34.424753+0000","last_clean":"2024-04-14T08:34:34.424753+0000","last_became_active":"2024-04-14T08:34:34.420809+0000","last_became_peered":"2024-04-14T08:34:34.420809+0000","last_unstale":"2024-04-14T08:34:34.424753+0000","last_undegraded":"2024-04-14T08:34:34.424753+0000","last_fullsized":"2024-04-14T08:34:34.424753+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:49.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.407542+0000","last_change":"2024-04-14T08:34:34.407542+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.407542+0000","last_undegraded":"2024-04-14T08:34:34.407542+0000","last_fullsized":"2024-04-14T08:34:34.407542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.448678+0000","last_change":"2024-04-14T08:34:34.422319+0000","last_active":"2024-04-14T08:34:35.448678+0000","last_peered":"2024-04-14T08:34:35.448678+0000","last_clean":"2024-04-14T08:34:35.448678+0000","last_became_active":"2024-04-14T08:34:34.421978+0000","last_became_peered":"2024-04-14T08:34:34.421978+0000","last_unstale":"2024-04-14T08:34:35.448678+0000","last_undegraded":"2024-04-14T08:34:35.448678+0000","last_fullsized":"2024-04-14T08:34:35.448678+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:11.381349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.404550+0000","last_change":"2024-04-14T08:34:34.404550+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.404550+0000","last_undegraded":"2024-04-14T08:34:34.404550+0000","last_fullsized":"2024-04-14T08:34:34.404550+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423937+0000","last_change":"2024-04-14T08:34:34.420518+0000","last_active":"2024-04-14T08:34:34.423937+0000","last_peered":"2024-04-14T08:34:34.423937+0000","last_clean":"2024-04-14T08:34:34.423937+0000","last_became_active":"2024-04-14T08:34:34.419065+0000","last_became_peered":"2024-04-14T08:34:34.419065+0000","last_unstale":"2024-04-14T08:34:34.423937+0000","last_undegraded":"2024-04-14T08:34:34.423937+0000","last_fullsized":"2024-04-14T08:34:34.423937+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:36.033184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410498+0000","last_change":"2024-04-14T08:34:34.410498+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410498+0000","last_undegraded":"2024-04-14T08:34:34.410498+0000","last_fullsized":"2024-04-14T08:34:34.410498+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423983+0000","last_change":"2024-04-14T08:34:34.421506+0000","last_active":"2024-04-14T08:34:34.423983+0000","last_peered":"2024-04-14T08:34:34.423983+0000","last_clean":"2024-04-14T08:34:34.423983+0000","last_became_active":"2024-04-14T08:34:34.420091+0000","last_became_peered":"2024-04-14T08:34:34.420091+0000","last_unstale":"2024-04-14T08:34:34.423983+0000","last_undegraded":"2024-04-14T08:34:34.423983+0000","last_fullsized":"2024-04-14T08:34:34.423983+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:30:35.547680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.412081+0000","last_change":"2024-04-14T08:34:34.412081+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.412081+0000","last_undegraded":"2024-04-14T08:34:34.412081+0000","last_fullsized":"2024-04-14T08:34:34.412081+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.429886+0000","last_change":"2024-04-14T08:34:34.424435+0000","last_active":"2024-04-14T08:34:34.429886+0000","last_peered":"2024-04-14T08:34:34.429886+0000","last_clean":"2024-04-14T08:34:34.429886+0000","last_became_active":"2024-04-14T08:34:34.424085+0000","last_became_peered":"2024-04-14T08:34:34.424085+0000","last_unstale":"2024-04-14T08:34:34.429886+0000","last_undegraded":"2024-04-14T08:34:34.429886+0000","last_fullsized":"2024-04-14T08:34:34.429886+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:04.319595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409928+0000","last_change":"2024-04-14T08:34:34.409928+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409928+0000","last_undegraded":"2024-04-14T08:34:34.409928+0000","last_fullsized":"2024-04-14T08:34:34.409928+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.419215+0000","last_change":"2024-04-14T08:34:34.414833+0000","last_active":"2024-04-14T08:34:35.419215+0000","last_peered":"2024-04-14T08:34:35.419215+0000","last_clean":"2024-04-14T08:34:35.419215+0000","last_became_active":"2024-04-14T08:34:34.414553+0000","last_became_peered":"2024-04-14T08:34:34.414553+0000","last_unstale":"2024-04-14T08:34:35.419215+0000","last_undegraded":"2024-04-14T08:34:35.419215+0000","last_fullsized":"2024-04-14T08:34:35.419215+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414459+0000","last_change":"2024-04-14T08:34:34.414459+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414459+0000","last_undegraded":"2024-04-14T08:34:34.414459+0000","last_fullsized":"2024-04-14T08:34:34.414459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.422034+0000","last_change":"2024-04-14T08:34:34.416091+0000","last_active":"2024-04-14T08:34:34.422034+0000","last_peered":"2024-04-14T08:34:34.422034+0000","last_clean":"2024-04-14T08:34:34.422034+0000","last_became_active":"2024-04-14T08:34:34.415653+0000","last_became_peered":"2024-04-14T08:34:34.415653+0000","last_unstale":"2024-04-14T08:34:34.422034+0000","last_undegraded":"2024-04-14T08:34:34.422034+0000","last_fullsized":"2024-04-14T08:34:34.422034+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.407664+0000","last_change":"2024-04-14T08:34:34.407664+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.407664+0000","last_undegraded":"2024-04-14T08:34:34.407664+0000","last_fullsized":"2024-04-14T08:34:34.407664+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.425676+0000","last_change":"2024-04-14T08:34:34.423763+0000","last_active":"2024-04-14T08:34:34.425676+0000","last_peered":"2024-04-14T08:34:34.425676+0000","last_clean":"2024-04-14T08:34:34.425676+0000","last_became_active":"2024-04-14T08:34:34.423333+0000","last_became_peered":"2024-04-14T08:34:34.423333+0000","last_unstale":"2024-04-14T08:34:34.425676+0000","last_undegraded":"2024-04-14T08:34:34.425676+0000","last_fullsized":"2024-04-14T08:34:34.425676+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:22:38.569478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405513+0000","last_change":"2024-04-14T08:34:34.405513+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405513+0000","last_undegraded":"2024-04-14T08:34:34.405513+0000","last_fullsized":"2024-04-14T08:34:34.405513+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.418045+0000","last_change":"2024-04-14T08:34:34.416569+0000","last_active":"2024-04-14T08:34:34.418045+0000","last_peered":"2024-04-14T08:34:34.418045+0000","last_clean":"2024-04-14T08:34:34.418045+0000","last_became_active":"2024-04-14T08:34:34.416306+0000","last_became_peered":"2024-04-14T08:34:34.416306+0000","last_unstale":"2024-04-14T08:34:34.418045+0000","last_undegraded":"2024-04-14T08:34:34.418045+0000","last_fullsized":"2024-04-14T08:34:34.418045+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:56.093503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405734+0000","last_change":"2024-04-14T08:34:34.405734+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405734+0000","last_undegraded":"2024-04-14T08:34:34.405734+0000","last_fullsized":"2024-04-14T08:34:34.405734+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.427399+0000","last_change":"2024-04-14T08:34:34.422659+0000","last_active":"2024-04-14T08:34:34.427399+0000","last_peered":"2024-04-14T08:34:34.427399+0000","last_clean":"2024-04-14T08:34:34.427399+0000","last_became_active":"2024-04-14T08:34:34.422434+0000","last_became_peered":"2024-04-14T08:34:34.422434+0000","last_unstale":"2024-04-14T08:34:34.427399+0000","last_undegraded":"2024-04-14T08:34:34.427399+0000","last_fullsized":"2024-04-14T08:34:34.427399+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:59:48.942079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405327+0000","last_change":"2024-04-14T08:34:34.405327+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405327+0000","last_undegraded":"2024-04-14T08:34:34.405327+0000","last_fullsized":"2024-04-14T08:34:34.405327+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.427365+0000","last_change":"2024-04-14T08:34:34.422117+0000","last_active":"2024-04-14T08:34:34.427365+0000","last_peered":"2024-04-14T08:34:34.427365+0000","last_clean":"2024-04-14T08:34:34.427365+0000","last_became_active":"2024-04-14T08:34:34.421504+0000","last_became_peered":"2024-04-14T08:34:34.421504+0000","last_unstale":"2024-04-14T08:34:34.427365+0000","last_undegraded":"2024-04-14T08:34:34.427365+0000","last_fullsized":"2024-04-14T08:34:34.427365+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:26:52.383137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409160+0000","last_change":"2024-04-14T08:34:34.409160+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409160+0000","last_undegraded":"2024-04-14T08:34:34.409160+0000","last_fullsized":"2024-04-14T08:34:34.409160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.420594+0000","last_change":"2024-04-14T08:34:34.413030+0000","last_active":"2024-04-14T08:34:34.420594+0000","last_peered":"2024-04-14T08:34:34.420594+0000","last_clean":"2024-04-14T08:34:34.420594+0000","last_became_active":"2024-04-14T08:34:34.412774+0000","last_became_peered":"2024-04-14T08:34:34.412774+0000","last_unstale":"2024-04-14T08:34:34.420594+0000","last_undegraded":"2024-04-14T08:34:34.420594+0000","last_fullsized":"2024-04-14T08:34:34.420594+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:04.319595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409571+0000","last_change":"2024-04-14T08:34:34.409571+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409571+0000","last_undegraded":"2024-04-14T08:34:34.409571+0000","last_fullsized":"2024-04-14T08:34:34.409571+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.421957+0000","last_change":"2024-04-14T08:34:34.417238+0000","last_active":"2024-04-14T08:34:34.421957+0000","last_peered":"2024-04-14T08:34:34.421957+0000","last_clean":"2024-04-14T08:34:34.421957+0000","last_became_active":"2024-04-14T08:34:34.417002+0000","last_became_peered":"2024-04-14T08:34:34.417002+0000","last_unstale":"2024-04-14T08:34:34.421957+0000","last_undegraded":"2024-04-14T08:34:34.421957+0000","last_fullsized":"2024-04-14T08:34:34.421957+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:11.743901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413056+0000","last_change":"2024-04-14T08:34:34.413056+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413056+0000","last_undegraded":"2024-04-14T08:34:34.413056+0000","last_fullsized":"2024-04-14T08:34:34.413056+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.425644+0000","last_change":"2024-04-14T08:34:34.423744+0000","last_active":"2024-04-14T08:34:34.425644+0000","last_peered":"2024-04-14T08:34:34.425644+0000","last_clean":"2024-04-14T08:34:34.425644+0000","last_became_active":"2024-04-14T08:34:34.423394+0000","last_became_peered":"2024-04-14T08:34:34.423394+0000","last_unstale":"2024-04-14T08:34:34.425644+0000","last_undegraded":"2024-04-14T08:34:34.425644+0000","last_fullsized":"2024-04-14T08:34:34.425644+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:32.608024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406093+0000","last_change":"2024-04-14T08:34:34.406093+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406093+0000","last_undegraded":"2024-04-14T08:34:34.406093+0000","last_fullsized":"2024-04-14T08:34:34.406093+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.414356+0000","last_change":"2024-04-14T08:34:34.411096+0000","last_active":"2024-04-14T08:34:34.414356+0000","last_peered":"2024-04-14T08:34:34.414356+0000","last_clean":"2024-04-14T08:34:34.414356+0000","last_became_active":"2024-04-14T08:34:34.410841+0000","last_became_peered":"2024-04-14T08:34:34.410841+0000","last_unstale":"2024-04-14T08:34:34.414356+0000","last_undegraded":"2024-04-14T08:34:34.414356+0000","last_fullsized":"2024-04-14T08:34:34.414356+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:12:26.461355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410965+0000","last_change":"2024-04-14T08:34:34.410965+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410965+0000","last_undegraded":"2024-04-14T08:34:34.410965+0000","last_fullsized":"2024-04-14T08:34:34.410965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.429860+0000","last_change":"2024-04-14T08:34:34.424338+0000","last_active":"2024-04-14T08:34:34.429860+0000","last_peered":"2024-04-14T08:34:34.429860+0000","last_clean":"2024-04-14T08:34:34.429860+0000","last_became_active":"2024-04-14T08:34:34.424057+0000","last_became_peered":"2024-04-14T08:34:34.424057+0000","last_unstale":"2024-04-14T08:34:34.429860+0000","last_undegraded":"2024-04-14T08:34:34.429860+0000","last_fullsized":"2024-04-14T08:34:34.429860+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:10.773823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406288+0000","last_change":"2024-04-14T08:34:34.406288+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406288+0000","last_undegraded":"2024-04-14T08:34:34.406288+0000","last_fullsized":"2024-04-14T08:34:34.406288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.420306+0000","last_change":"2024-04-14T08:34:34.416682+0000","last_active":"2024-04-14T08:34:35.420306+0000","last_peered":"2024-04-14T08:34:35.420306+0000","last_clean":"2024-04-14T08:34:35.420306+0000","last_became_active":"2024-04-14T08:34:34.416172+0000","last_became_peered":"2024-04-14T08:34:34.416172+0000","last_unstale":"2024-04-14T08:34:35.420306+0000","last_undegraded":"2024-04-14T08:34:35.420306+0000","last_fullsized":"2024-04-14T08:34:35.420306+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:36.033184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408713+0000","last_change":"2024-04-14T08:34:34.408713+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408713+0000","last_undegraded":"2024-04-14T08:34:34.408713+0000","last_fullsized":"2024-04-14T08:34:34.408713+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423234+0000","last_change":"2024-04-14T08:34:34.420330+0000","last_active":"2024-04-14T08:34:34.423234+0000","last_peered":"2024-04-14T08:34:34.423234+0000","last_clean":"2024-04-14T08:34:34.423234+0000","last_became_active":"2024-04-14T08:34:34.420116+0000","last_became_peered":"2024-04-14T08:34:34.420116+0000","last_unstale":"2024-04-14T08:34:34.423234+0000","last_undegraded":"2024-04-14T08:34:34.423234+0000","last_fullsized":"2024-04-14T08:34:34.423234+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:41:19.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408837+0000","last_change":"2024-04-14T08:34:34.408837+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408837+0000","last_undegraded":"2024-04-14T08:34:34.408837+0000","last_fullsized":"2024-04-14T08:34:34.408837+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.422091+0000","last_change":"2024-04-14T08:34:34.419528+0000","last_active":"2024-04-14T08:34:34.422091+0000","last_peered":"2024-04-14T08:34:34.422091+0000","last_clean":"2024-04-14T08:34:34.422091+0000","last_became_active":"2024-04-14T08:34:34.419258+0000","last_became_peered":"2024-04-14T08:34:34.419258+0000","last_unstale":"2024-04-14T08:34:34.422091+0000","last_undegraded":"2024-04-14T08:34:34.422091+0000","last_fullsized":"2024-04-14T08:34:34.422091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:11.381349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409171+0000","last_change":"2024-04-14T08:34:34.409171+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409171+0000","last_undegraded":"2024-04-14T08:34:34.409171+0000","last_fullsized":"2024-04-14T08:34:34.409171+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423956+0000","last_change":"2024-04-14T08:34:34.421662+0000","last_active":"2024-04-14T08:34:34.423956+0000","last_peered":"2024-04-14T08:34:34.423956+0000","last_clean":"2024-04-14T08:34:34.423956+0000","last_became_active":"2024-04-14T08:34:34.421382+0000","last_became_peered":"2024-04-14T08:34:34.421382+0000","last_unstale":"2024-04-14T08:34:34.423956+0000","last_undegraded":"2024-04-14T08:34:34.423956+0000","last_fullsized":"2024-04-14T08:34:34.423956+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:55:44.175481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413461+0000","last_change":"2024-04-14T08:34:34.413461+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413461+0000","last_undegraded":"2024-04-14T08:34:34.413461+0000","last_fullsized":"2024-04-14T08:34:34.413461+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424542+0000","last_change":"2024-04-14T08:34:34.419857+0000","last_active":"2024-04-14T08:34:34.424542+0000","last_peered":"2024-04-14T08:34:34.424542+0000","last_clean":"2024-04-14T08:34:34.424542+0000","last_became_active":"2024-04-14T08:34:34.419548+0000","last_became_peered":"2024-04-14T08:34:34.419548+0000","last_unstale":"2024-04-14T08:34:34.424542+0000","last_undegraded":"2024-04-14T08:34:34.424542+0000","last_fullsized":"2024-04-14T08:34:34.424542+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:10.773823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408288+0000","last_change":"2024-04-14T08:34:34.408288+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408288+0000","last_undegraded":"2024-04-14T08:34:34.408288+0000","last_fullsized":"2024-04-14T08:34:34.408288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424716+0000","last_change":"2024-04-14T08:34:34.420316+0000","last_active":"2024-04-14T08:34:34.424716+0000","last_peered":"2024-04-14T08:34:34.424716+0000","last_clean":"2024-04-14T08:34:34.424716+0000","last_became_active":"2024-04-14T08:34:34.420093+0000","last_became_peered":"2024-04-14T08:34:34.420093+0000","last_unstale":"2024-04-14T08:34:34.424716+0000","last_undegraded":"2024-04-14T08:34:34.424716+0000","last_fullsized":"2024-04-14T08:34:34.424716+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:44:18.315252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408798+0000","last_change":"2024-04-14T08:34:34.408798+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408798+0000","last_undegraded":"2024-04-14T08:34:34.408798+0000","last_fullsized":"2024-04-14T08:34:34.408798+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423952+0000","last_change":"2024-04-14T08:34:34.421158+0000","last_active":"2024-04-14T08:34:34.423952+0000","last_peered":"2024-04-14T08:34:34.423952+0000","last_clean":"2024-04-14T08:34:34.423952+0000","last_became_active":"2024-04-14T08:34:34.420780+0000","last_became_peered":"2024-04-14T08:34:34.420780+0000","last_unstale":"2024-04-14T08:34:34.423952+0000","last_undegraded":"2024-04-14T08:34:34.423952+0000","last_fullsized":"2024-04-14T08:34:34.423952+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:44:10.910055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":226,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.418795+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:34:35.418795+0000","last_peered":"2024-04-14T08:34:35.418795+0000","last_clean":"2024-04-14T08:34:35.418795+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:34:35.418795+0000","last_undegraded":"2024-04-14T08:34:35.418795+0000","last_fullsized":"2024-04-14T08:34:35.418795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1183744,"data_stored":846191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":557056,"data_stored":241865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1163264,"data_stored":838900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:34:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-04-14T08:34:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.408023+0000', 'last_change': '2024-04-14T08:34:34.408023+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.408023+0000', 'last_undegraded': '2024-04-14T08:34:34.408023+0000', 'last_fullsized': '2024-04-14T08:34:34.408023+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:34:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-04-14T08:34:39.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.414650+0000', 'last_change': '2024-04-14T08:34:34.414650+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.414650+0000', 'last_undegraded': '2024-04-14T08:34:34.414650+0000', 'last_fullsized': '2024-04-14T08:34:34.414650+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2024-04-14T08:34:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.405062+0000', 'last_change': '2024-04-14T08:34:34.405062+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.405062+0000', 'last_undegraded': '2024-04-14T08:34:34.405062+0000', 'last_fullsized': '2024-04-14T08:34:34.405062+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:34:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-04-14T08:34:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.406804+0000', 'last_change': '2024-04-14T08:34:34.406804+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.406804+0000', 'last_undegraded': '2024-04-14T08:34:34.406804+0000', 'last_fullsized': '2024-04-14T08:34:34.406804+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:34:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2024-04-14T08:34:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.414181+0000', 'last_change': '2024-04-14T08:34:34.414181+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.414181+0000', 'last_undegraded': '2024-04-14T08:34:34.414181+0000', 'last_fullsized': '2024-04-14T08:34:34.414181+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-04-14T08:34:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.409156+0000', 'last_change': '2024-04-14T08:34:34.409156+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.409156+0000', 'last_undegraded': '2024-04-14T08:34:34.409156+0000', 'last_fullsized': '2024-04-14T08:34:34.409156+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:34:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-04-14T08:34:39.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.410003+0000', 'last_change': '2024-04-14T08:34:34.410003+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.410003+0000', 'last_undegraded': '2024-04-14T08:34:34.410003+0000', 'last_fullsized': '2024-04-14T08:34:34.410003+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:34:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-04-14T08:34:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.406692+0000', 'last_change': '2024-04-14T08:34:34.406692+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.406692+0000', 'last_undegraded': '2024-04-14T08:34:34.406692+0000', 'last_fullsized': '2024-04-14T08:34:34.406692+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:34:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2024-04-14T08:34:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.405927+0000', 'last_change': '2024-04-14T08:34:34.405927+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.405927+0000', 'last_undegraded': '2024-04-14T08:34:34.405927+0000', 'last_fullsized': '2024-04-14T08:34:34.405927+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:34:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-04-14T08:34:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.410934+0000', 'last_change': '2024-04-14T08:34:34.410934+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.410934+0000', 'last_undegraded': '2024-04-14T08:34:34.410934+0000', 'last_fullsized': '2024-04-14T08:34:34.410934+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:34:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-04-14T08:34:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.406225+0000', 'last_change': '2024-04-14T08:34:34.406225+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.406225+0000', 'last_undegraded': '2024-04-14T08:34:34.406225+0000', 'last_fullsized': '2024-04-14T08:34:34.406225+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:34:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2024-04-14T08:34:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.411333+0000', 'last_change': '2024-04-14T08:34:34.411333+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.411333+0000', 'last_undegraded': '2024-04-14T08:34:34.411333+0000', 'last_fullsized': '2024-04-14T08:34:34.411333+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-04-14T08:34:39.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.406390+0000', 'last_change': '2024-04-14T08:34:34.406390+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.406390+0000', 'last_undegraded': '2024-04-14T08:34:34.406390+0000', 'last_fullsized': '2024-04-14T08:34:34.406390+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:34:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2024-04-14T08:34:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.414078+0000', 'last_change': '2024-04-14T08:34:34.414078+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.414078+0000', 'last_undegraded': '2024-04-14T08:34:34.414078+0000', 'last_fullsized': '2024-04-14T08:34:34.414078+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-04-14T08:34:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.412491+0000', 'last_change': '2024-04-14T08:34:34.412491+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.412491+0000', 'last_undegraded': '2024-04-14T08:34:34.412491+0000', 'last_fullsized': '2024-04-14T08:34:34.412491+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2024-04-14T08:34:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.410141+0000', 'last_change': '2024-04-14T08:34:34.410141+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.410141+0000', 'last_undegraded': '2024-04-14T08:34:34.410141+0000', 'last_fullsized': '2024-04-14T08:34:34.410141+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:34:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2024-04-14T08:34:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.408219+0000', 'last_change': '2024-04-14T08:34:34.408219+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.408219+0000', 'last_undegraded': '2024-04-14T08:34:34.408219+0000', 'last_fullsized': '2024-04-14T08:34:34.408219+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:34:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-04-14T08:34:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.407965+0000', 'last_change': '2024-04-14T08:34:34.407965+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.407965+0000', 'last_undegraded': '2024-04-14T08:34:34.407965+0000', 'last_fullsized': '2024-04-14T08:34:34.407965+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:34:39.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-04-14T08:34:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.405857+0000', 'last_change': '2024-04-14T08:34:34.405857+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.405857+0000', 'last_undegraded': '2024-04-14T08:34:34.405857+0000', 'last_fullsized': '2024-04-14T08:34:34.405857+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:34:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-04-14T08:34:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.410886+0000', 'last_change': '2024-04-14T08:34:34.410886+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.410886+0000', 'last_undegraded': '2024-04-14T08:34:34.410886+0000', 'last_fullsized': '2024-04-14T08:34:34.410886+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2024-04-14T08:34:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.413719+0000', 'last_change': '2024-04-14T08:34:34.413719+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.413719+0000', 'last_undegraded': '2024-04-14T08:34:34.413719+0000', 'last_fullsized': '2024-04-14T08:34:34.413719+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-04-14T08:34:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.410342+0000', 'last_change': '2024-04-14T08:34:34.410342+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.410342+0000', 'last_undegraded': '2024-04-14T08:34:34.410342+0000', 'last_fullsized': '2024-04-14T08:34:34.410342+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:34:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-04-14T08:34:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.406527+0000', 'last_change': '2024-04-14T08:34:34.406527+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.406527+0000', 'last_undegraded': '2024-04-14T08:34:34.406527+0000', 'last_fullsized': '2024-04-14T08:34:34.406527+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:34:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2024-04-14T08:34:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.408765+0000', 'last_change': '2024-04-14T08:34:34.408765+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.408765+0000', 'last_undegraded': '2024-04-14T08:34:34.408765+0000', 'last_fullsized': '2024-04-14T08:34:34.408765+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:34:39.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2024-04-14T08:34:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.409655+0000', 'last_change': '2024-04-14T08:34:34.409655+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.409655+0000', 'last_undegraded': '2024-04-14T08:34:34.409655+0000', 'last_fullsized': '2024-04-14T08:34:34.409655+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-04-14T08:34:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.408105+0000', 'last_change': '2024-04-14T08:34:34.408105+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.408105+0000', 'last_undegraded': '2024-04-14T08:34:34.408105+0000', 'last_fullsized': '2024-04-14T08:34:34.408105+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2024-04-14T08:34:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.411421+0000', 'last_change': '2024-04-14T08:34:34.411421+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.411421+0000', 'last_undegraded': '2024-04-14T08:34:34.411421+0000', 'last_fullsized': '2024-04-14T08:34:34.411421+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:34:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-04-14T08:34:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.409109+0000', 'last_change': '2024-04-14T08:34:34.409109+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.409109+0000', 'last_undegraded': '2024-04-14T08:34:34.409109+0000', 'last_fullsized': '2024-04-14T08:34:34.409109+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:34:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-04-14T08:34:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.410262+0000', 'last_change': '2024-04-14T08:34:34.410262+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.410262+0000', 'last_undegraded': '2024-04-14T08:34:34.410262+0000', 'last_fullsized': '2024-04-14T08:34:34.410262+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:34:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-04-14T08:34:39.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.410996+0000', 'last_change': '2024-04-14T08:34:34.410996+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.410996+0000', 'last_undegraded': '2024-04-14T08:34:34.410996+0000', 'last_fullsized': '2024-04-14T08:34:34.410996+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:34:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-04-14T08:34:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.406292+0000', 'last_change': '2024-04-14T08:34:34.406292+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.406292+0000', 'last_undegraded': '2024-04-14T08:34:34.406292+0000', 'last_fullsized': '2024-04-14T08:34:34.406292+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:34:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-04-14T08:34:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.408225+0000', 'last_change': '2024-04-14T08:34:34.408225+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.408225+0000', 'last_undegraded': '2024-04-14T08:34:34.408225+0000', 'last_fullsized': '2024-04-14T08:34:34.408225+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:34:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-04-14T08:34:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.405634+0000', 'last_change': '2024-04-14T08:34:34.405634+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.405634+0000', 'last_undegraded': '2024-04-14T08:34:34.405634+0000', 'last_fullsized': '2024-04-14T08:34:34.405634+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:34:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-04-14T08:34:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.413953+0000', 'last_change': '2024-04-14T08:34:34.413953+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.413953+0000', 'last_undegraded': '2024-04-14T08:34:34.413953+0000', 'last_fullsized': '2024-04-14T08:34:34.413953+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:34:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-04-14T08:34:39.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.407542+0000', 'last_change': '2024-04-14T08:34:34.407542+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.407542+0000', 'last_undegraded': '2024-04-14T08:34:34.407542+0000', 'last_fullsized': '2024-04-14T08:34:34.407542+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-04-14T08:34:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.404550+0000', 'last_change': '2024-04-14T08:34:34.404550+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.404550+0000', 'last_undegraded': '2024-04-14T08:34:34.404550+0000', 'last_fullsized': '2024-04-14T08:34:34.404550+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:34:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-04-14T08:34:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.410498+0000', 'last_change': '2024-04-14T08:34:34.410498+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.410498+0000', 'last_undegraded': '2024-04-14T08:34:34.410498+0000', 'last_fullsized': '2024-04-14T08:34:34.410498+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:34:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-04-14T08:34:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.412081+0000', 'last_change': '2024-04-14T08:34:34.412081+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.412081+0000', 'last_undegraded': '2024-04-14T08:34:34.412081+0000', 'last_fullsized': '2024-04-14T08:34:34.412081+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-04-14T08:34:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.409928+0000', 'last_change': '2024-04-14T08:34:34.409928+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.409928+0000', 'last_undegraded': '2024-04-14T08:34:34.409928+0000', 'last_fullsized': '2024-04-14T08:34:34.409928+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:34:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-04-14T08:34:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.414459+0000', 'last_change': '2024-04-14T08:34:34.414459+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.414459+0000', 'last_undegraded': '2024-04-14T08:34:34.414459+0000', 'last_fullsized': '2024-04-14T08:34:34.414459+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-04-14T08:34:39.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.407664+0000', 'last_change': '2024-04-14T08:34:34.407664+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.407664+0000', 'last_undegraded': '2024-04-14T08:34:34.407664+0000', 'last_fullsized': '2024-04-14T08:34:34.407664+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:34:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-04-14T08:34:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.405513+0000', 'last_change': '2024-04-14T08:34:34.405513+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.405513+0000', 'last_undegraded': '2024-04-14T08:34:34.405513+0000', 'last_fullsized': '2024-04-14T08:34:34.405513+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:34:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-04-14T08:34:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.405734+0000', 'last_change': '2024-04-14T08:34:34.405734+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.405734+0000', 'last_undegraded': '2024-04-14T08:34:34.405734+0000', 'last_fullsized': '2024-04-14T08:34:34.405734+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:34:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-04-14T08:34:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.405327+0000', 'last_change': '2024-04-14T08:34:34.405327+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.405327+0000', 'last_undegraded': '2024-04-14T08:34:34.405327+0000', 'last_fullsized': '2024-04-14T08:34:34.405327+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:34:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-04-14T08:34:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.409160+0000', 'last_change': '2024-04-14T08:34:34.409160+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.409160+0000', 'last_undegraded': '2024-04-14T08:34:34.409160+0000', 'last_fullsized': '2024-04-14T08:34:34.409160+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:34:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-04-14T08:34:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.409571+0000', 'last_change': '2024-04-14T08:34:34.409571+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.409571+0000', 'last_undegraded': '2024-04-14T08:34:34.409571+0000', 'last_fullsized': '2024-04-14T08:34:34.409571+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:34:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-04-14T08:34:39.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.413056+0000', 'last_change': '2024-04-14T08:34:34.413056+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.413056+0000', 'last_undegraded': '2024-04-14T08:34:34.413056+0000', 'last_fullsized': '2024-04-14T08:34:34.413056+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-04-14T08:34:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.406093+0000', 'last_change': '2024-04-14T08:34:34.406093+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.406093+0000', 'last_undegraded': '2024-04-14T08:34:34.406093+0000', 'last_fullsized': '2024-04-14T08:34:34.406093+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:34:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-04-14T08:34:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.410965+0000', 'last_change': '2024-04-14T08:34:34.410965+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.410965+0000', 'last_undegraded': '2024-04-14T08:34:34.410965+0000', 'last_fullsized': '2024-04-14T08:34:34.410965+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:34:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-04-14T08:34:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.406288+0000', 'last_change': '2024-04-14T08:34:34.406288+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.406288+0000', 'last_undegraded': '2024-04-14T08:34:34.406288+0000', 'last_fullsized': '2024-04-14T08:34:34.406288+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:34:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-04-14T08:34:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.408713+0000', 'last_change': '2024-04-14T08:34:34.408713+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.408713+0000', 'last_undegraded': '2024-04-14T08:34:34.408713+0000', 'last_fullsized': '2024-04-14T08:34:34.408713+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:34:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-04-14T08:34:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.408837+0000', 'last_change': '2024-04-14T08:34:34.408837+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.408837+0000', 'last_undegraded': '2024-04-14T08:34:34.408837+0000', 'last_fullsized': '2024-04-14T08:34:34.408837+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-04-14T08:34:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.409171+0000', 'last_change': '2024-04-14T08:34:34.409171+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.409171+0000', 'last_undegraded': '2024-04-14T08:34:34.409171+0000', 'last_fullsized': '2024-04-14T08:34:34.409171+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:34:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-04-14T08:34:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.413461+0000', 'last_change': '2024-04-14T08:34:34.413461+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.413461+0000', 'last_undegraded': '2024-04-14T08:34:34.413461+0000', 'last_fullsized': '2024-04-14T08:34:34.413461+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:34:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-04-14T08:34:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.408288+0000', 'last_change': '2024-04-14T08:34:34.408288+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.408288+0000', 'last_undegraded': '2024-04-14T08:34:34.408288+0000', 'last_fullsized': '2024-04-14T08:34:34.408288+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:34:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-04-14T08:34:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:34:34.408798+0000', 'last_change': '2024-04-14T08:34:34.408798+0000', 'last_active': '2024-04-14T08:34:34.399007+0000', 'last_peered': '2024-04-14T08:34:34.399007+0000', 'last_clean': '2024-04-14T08:34:34.399007+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:34:34.408798+0000', 'last_undegraded': '2024-04-14T08:34:34.408798+0000', 'last_fullsized': '2024-04-14T08:34:34.408798+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'last_clean_scrub_stamp': '2024-04-14T08:34:34.399007+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:34:39.183 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-14T08:34:39.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.311+0000 7efd20b88640 1 Processor -- start 2024-04-14T08:34:39.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.311+0000 7efd20b88640 1 -- start start 2024-04-14T08:34:39.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd20b88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd1c0d1a70 0x7efd1c0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:39.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd20b88640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efd1c0b4260 0x7efd1c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:39.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd20b88640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd1c0d1ed0 con 0x7efd1c15e390 2024-04-14T08:34:39.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd20b88640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7efd1c0d0080 con 0x7efd1c0b4260 2024-04-14T08:34:39.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd20b88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efd1c15e770 con 0x7efd1c0d1a70 2024-04-14T08:34:39.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd19d9b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd1c0d1a70 0x7efd1c0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:39.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd1ad9d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efd1c0b4260 0x7efd1c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:39.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd19d9b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd1c0d1a70 0x7efd1c0b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:34960/0 (socket says 172.21.15.172:34960) 2024-04-14T08:34:39.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd19d9b640 1 -- 172.21.15.172:0/3460726666 learned_addr learned my addr 172.21.15.172:0/3460726666 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:39.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd1ad9d640 1 -- 172.21.15.172:0/3460726666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd1c0d1a70 msgr2=0x7efd1c0b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:39.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd1ad9d640 1 --2- 172.21.15.172:0/3460726666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd1c0d1a70 0x7efd1c0b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd1ad9d640 1 -- 172.21.15.172:0/3460726666 >> v1:172.21.15.177:6789/0 conn(0x7efd1c15e390 legacy=0x7efd1c15f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd1959a640 1 -- 172.21.15.172:0/3460726666 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2149232975 0 0) 0x7efd14001490 con 0x7efd1c15e390 2024-04-14T08:34:39.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.312+0000 7efd1ad9d640 1 -- 172.21.15.172:0/3460726666 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c06dba0 con 0x7efd1c0b4260 2024-04-14T08:34:39.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.313+0000 7efd1ad9d640 1 --2- 172.21.15.172:0/3460726666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd1c0b4260 0x7efd1c06c630 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0097a0 tx=0x7efd0c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9097bf6872d22d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:39.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.313+0000 7efd1959a640 1 -- 172.21.15.172:0/3460726666 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c0315a0 con 0x7efd1c0b4260 2024-04-14T08:34:39.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.313+0000 7efd1959a640 1 -- 172.21.15.172:0/3460726666 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd0c037070 con 0x7efd1c0b4260 2024-04-14T08:34:39.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.313+0000 7efd1959a640 1 -- 172.21.15.172:0/3460726666 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c031960 con 0x7efd1c0b4260 2024-04-14T08:34:39.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.314+0000 7efd20b88640 1 -- 172.21.15.172:0/3460726666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd1c0b4260 msgr2=0x7efd1c06c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.314+0000 7efd20b88640 1 --2- 172.21.15.172:0/3460726666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd1c0b4260 0x7efd1c06c630 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0097a0 tx=0x7efd0c02b840 comp rx=0 tx=0).stop 2024-04-14T08:34:39.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.314+0000 7efd20b88640 1 -- 172.21.15.172:0/3460726666 shutdown_connections 2024-04-14T08:34:39.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.314+0000 7efd20b88640 1 --2- 172.21.15.172:0/3460726666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd1c0b4260 0x7efd1c06c630 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.314+0000 7efd20b88640 1 --2- 172.21.15.172:0/3460726666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd1c0d1a70 0x7efd1c0b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.314+0000 7efd20b88640 1 -- 172.21.15.172:0/3460726666 >> 172.21.15.172:0/3460726666 conn(0x7efd1c06f820 msgr2=0x7efd1c06fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:39.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.314+0000 7efd20b88640 1 -- 172.21.15.172:0/3460726666 shutdown_connections 2024-04-14T08:34:39.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.314+0000 7efd20b88640 1 -- 172.21.15.172:0/3460726666 wait complete. 2024-04-14T08:34:39.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.315+0000 7efd20b88640 1 Processor -- start 2024-04-14T08:34:39.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.315+0000 7efd20b88640 1 -- start start 2024-04-14T08:34:39.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd20b88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd1c0b4260 0x7efd1c0ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:39.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd20b88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0d1a70 0x7efd1c15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:39.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd20b88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd1c15e390 0x7efd1c159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:39.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd20b88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c06cb70 con 0x7efd1c0d1a70 2024-04-14T08:34:39.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd20b88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c0d0510 con 0x7efd1c0b4260 2024-04-14T08:34:39.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd20b88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efd1c06ccf0 con 0x7efd1c15e390 2024-04-14T08:34:39.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd19d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0d1a70 0x7efd1c15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:39.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd19d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0d1a70 0x7efd1c15adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48242/0 (socket says 172.21.15.172:48242) 2024-04-14T08:34:39.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd19d9b640 1 -- 172.21.15.172:0/1471338004 learned_addr learned my addr 172.21.15.172:0/1471338004 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd1ad9d640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd1c15e390 0x7efd1c159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd1a59c640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd1c0b4260 0x7efd1c0ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd19d9b640 1 -- 172.21.15.172:0/1471338004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd1c15e390 msgr2=0x7efd1c159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd19d9b640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd1c15e390 0x7efd1c159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd19d9b640 1 -- 172.21.15.172:0/1471338004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd1c0b4260 msgr2=0x7efd1c0ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd19d9b640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd1c0b4260 0x7efd1c0ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.316+0000 7efd19d9b640 1 -- 172.21.15.172:0/1471338004 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c0b0020 con 0x7efd1c0d1a70 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.317+0000 7efd19d9b640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0d1a70 0x7efd1c15adc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efd04009fd0 tx=0x7efd0400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.317+0000 7efd12ffd640 1 -- 172.21.15.172:0/1471338004 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04017070 con 0x7efd1c0d1a70 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.317+0000 7efd12ffd640 1 -- 172.21.15.172:0/1471338004 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd0400c940 con 0x7efd1c0d1a70 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.317+0000 7efd12ffd640 1 -- 172.21.15.172:0/1471338004 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0400cd10 con 0x7efd1c0d1a70 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.317+0000 7efd20b88640 1 -- 172.21.15.172:0/1471338004 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c0b02a0 con 0x7efd1c0d1a70 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.317+0000 7efd20b88640 1 -- 172.21.15.172:0/1471338004 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c0b0860 con 0x7efd1c0d1a70 2024-04-14T08:34:39.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.318+0000 7efd20b88640 1 -- 172.21.15.172:0/1471338004 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd1c068520 con 0x7efd1c0d1a70 2024-04-14T08:34:39.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.318+0000 7efd12ffd640 1 -- 172.21.15.172:0/1471338004 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd0401a070 con 0x7efd1c0d1a70 2024-04-14T08:34:39.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.319+0000 7efd12ffd640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efcfc06c8f0 0x7efcfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:39.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.319+0000 7efd12ffd640 1 -- 172.21.15.172:0/1471338004 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7efd0408bfd0 con 0x7efd1c0d1a70 2024-04-14T08:34:39.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.319+0000 7efd1a59c640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efcfc06c8f0 0x7efcfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:39.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.319+0000 7efd1a59c640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efcfc06c8f0 0x7efcfc06edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efd14007fd0 tx=0x7efd14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:39.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.323+0000 7efd12ffd640 1 -- 172.21.15.172:0/1471338004 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efd1c0b0020 con 0x7efd1c0d1a70 2024-04-14T08:34:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.576+0000 7efd20b88640 1 -- 172.21.15.172:0/1471338004 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7efd1c06d680 con 0x7efd1c0d1a70 2024-04-14T08:34:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.577+0000 7efd12ffd640 1 -- 172.21.15.172:0/1471338004 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1516 (secure 0 0 0) 0x7efd0405a270 con 0x7efd1c0d1a70 2024-04-14T08:34:39.578 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:39.579 INFO:teuthology.orchestra.run.smithi172.stdout:{"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":130,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":8,"num_up_osds":8,"osd_up_since":1713083560,"num_in_osds":8,"osd_in_since":1713083551,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":56},{"state_name":"active+clean","count":41}],"num_pgs":97,"num_pools":3,"num_objects":4,"data_bytes":590458,"bytes_used":227131392,"bytes_avail":772866981888,"bytes_total":773094113280,"inactive_pgs_ratio":0.57731956243515015,"write_bytes_sec":178,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":23,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5745}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-04-14T08:34:25.138646+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-14T08:34:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.579+0000 7efd20b88640 1 -- 172.21.15.172:0/1471338004 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efcfc06c8f0 msgr2=0x7efcfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.579+0000 7efd20b88640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efcfc06c8f0 0x7efcfc06edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efd14007fd0 tx=0x7efd14009040 comp rx=0 tx=0).stop 2024-04-14T08:34:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.579+0000 7efd20b88640 1 -- 172.21.15.172:0/1471338004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0d1a70 msgr2=0x7efd1c15adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.579+0000 7efd20b88640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0d1a70 0x7efd1c15adc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efd04009fd0 tx=0x7efd0400ed40 comp rx=0 tx=0).stop 2024-04-14T08:34:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.579+0000 7efd20b88640 1 -- 172.21.15.172:0/1471338004 shutdown_connections 2024-04-14T08:34:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.579+0000 7efd20b88640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efcfc06c8f0 0x7efcfc06edb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.579+0000 7efd20b88640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd1c15e390 0x7efd1c159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.579+0000 7efd20b88640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0d1a70 0x7efd1c15adc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.579+0000 7efd20b88640 1 --2- 172.21.15.172:0/1471338004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd1c0b4260 0x7efd1c0ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.579+0000 7efd20b88640 1 -- 172.21.15.172:0/1471338004 >> 172.21.15.172:0/1471338004 conn(0x7efd1c06f820 msgr2=0x7efd1c0b7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:39.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.579+0000 7efd20b88640 1 -- 172.21.15.172:0/1471338004 shutdown_connections 2024-04-14T08:34:39.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.579+0000 7efd20b88640 1 -- 172.21.15.172:0/1471338004 wait complete. 2024-04-14T08:34:39.592 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-14T08:34:39.592 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:34:39.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.693+0000 7fecb3236640 1 Processor -- start 2024-04-14T08:34:39.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.694+0000 7fecb3236640 1 -- start start 2024-04-14T08:34:39.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.694+0000 7fecb3236640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fecac0b7a60 0x7fecac0ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:39.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.694+0000 7fecb3236640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecac08c4c0 0x7fecac0cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:39.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.694+0000 7fecb3236640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecac0d2b10 con 0x7fecac0cd410 2024-04-14T08:34:39.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.694+0000 7fecb3236640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fecac0d0ad0 con 0x7fecac0b7a60 2024-04-14T08:34:39.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.694+0000 7fecb3236640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fecac0d0db0 con 0x7fecac08c4c0 2024-04-14T08:34:39.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.694+0000 7fecb0fd2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fecac0b7a60 0x7fecac0ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:39.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.694+0000 7fecb0fd2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fecac0b7a60 0x7fecac0ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45710/0 (socket says 172.21.15.172:45710) 2024-04-14T08:34:39.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.695+0000 7fecb0fd2640 1 -- 172.21.15.172:0/2991875870 learned_addr learned my addr 172.21.15.172:0/2991875870 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:39.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.695+0000 7fecb0fd2640 1 -- 172.21.15.172:0/2991875870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecac08c4c0 msgr2=0x7fecac0cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:39.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.695+0000 7fecb0fd2640 1 --2- 172.21.15.172:0/2991875870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecac08c4c0 0x7fecac0cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.695+0000 7fecb0fd2640 1 -- 172.21.15.172:0/2991875870 >> v1:172.21.15.177:6789/0 conn(0x7fecac0cd410 legacy=0x7fecac0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:39.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.695+0000 7fecb0fd2640 1 -- 172.21.15.172:0/2991875870 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac0d1090 con 0x7fecac0b7a60 2024-04-14T08:34:39.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.695+0000 7fecb0fd2640 1 --2- 172.21.15.172:0/2991875870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecac0b7a60 0x7fecac0ce8c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7feca4009e50 tx=0x7feca402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a450240681c11ee server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:39.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.696+0000 7fecab7fe640 1 -- 172.21.15.172:0/2991875870 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca402c9e0 con 0x7fecac0b7a60 2024-04-14T08:34:39.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.696+0000 7fecab7fe640 1 -- 172.21.15.172:0/2991875870 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca402cb80 con 0x7fecac0b7a60 2024-04-14T08:34:39.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.696+0000 7fecab7fe640 1 -- 172.21.15.172:0/2991875870 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca40343f0 con 0x7fecac0b7a60 2024-04-14T08:34:39.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.696+0000 7fecb3236640 1 -- 172.21.15.172:0/2991875870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecac0b7a60 msgr2=0x7fecac0ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.696+0000 7fecb3236640 1 --2- 172.21.15.172:0/2991875870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecac0b7a60 0x7fecac0ce8c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7feca4009e50 tx=0x7feca402be40 comp rx=0 tx=0).stop 2024-04-14T08:34:39.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.696+0000 7fecb3236640 1 -- 172.21.15.172:0/2991875870 shutdown_connections 2024-04-14T08:34:39.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.696+0000 7fecb3236640 1 --2- 172.21.15.172:0/2991875870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecac08c4c0 0x7fecac0cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.696+0000 7fecb3236640 1 --2- 172.21.15.172:0/2991875870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecac0b7a60 0x7fecac0ce8c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.696+0000 7fecb3236640 1 -- 172.21.15.172:0/2991875870 >> 172.21.15.172:0/2991875870 conn(0x7fecac08c000 msgr2=0x7fecac0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:39.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.697+0000 7fecb3236640 1 -- 172.21.15.172:0/2991875870 shutdown_connections 2024-04-14T08:34:39.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.697+0000 7fecb3236640 1 -- 172.21.15.172:0/2991875870 wait complete. 2024-04-14T08:34:39.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.698+0000 7fecb3236640 1 Processor -- start 2024-04-14T08:34:39.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.698+0000 7fecb3236640 1 -- start start 2024-04-14T08:34:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.699+0000 7fecb3236640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecac08c4c0 0x7fecac159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.699+0000 7fecb3236640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecac0b7a60 0x7fecac15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.699+0000 7fecb3236640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecac0cd410 0x7fecac0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.699+0000 7fecb3236640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fecac0d2750 con 0x7fecac0b7a60 2024-04-14T08:34:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.699+0000 7fecb3236640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fecac0b7e40 con 0x7fecac08c4c0 2024-04-14T08:34:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.699+0000 7fecb3236640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fecac0d28d0 con 0x7fecac0cd410 2024-04-14T08:34:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.699+0000 7fecabfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecac0b7a60 0x7fecac15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.699+0000 7fecabfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecac0b7a60 0x7fecac15c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45716/0 (socket says 172.21.15.172:45716) 2024-04-14T08:34:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7fecabfff640 1 -- 172.21.15.172:0/3181043937 learned_addr learned my addr 172.21.15.172:0/3181043937 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7fecb0fd2640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecac08c4c0 0x7fecac159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7fecb17d3640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecac0cd410 0x7fecac0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7fecabfff640 1 -- 172.21.15.172:0/3181043937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecac0cd410 msgr2=0x7fecac0cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7fecabfff640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecac0cd410 0x7fecac0cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7fecabfff640 1 -- 172.21.15.172:0/3181043937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecac08c4c0 msgr2=0x7fecac159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7fecabfff640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecac08c4c0 0x7fecac159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7fecabfff640 1 -- 172.21.15.172:0/3181043937 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac15b930 con 0x7fecac0b7a60 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7fecabfff640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecac0b7a60 0x7fecac15c9d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7feca000b410 tx=0x7feca000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7feca97fa640 1 -- 172.21.15.172:0/3181043937 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0015020 con 0x7fecac0b7a60 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7feca97fa640 1 -- 172.21.15.172:0/3181043937 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca0011850 con 0x7fecac0b7a60 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7feca97fa640 1 -- 172.21.15.172:0/3181043937 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0011c40 con 0x7fecac0b7a60 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7fecb3236640 1 -- 172.21.15.172:0/3181043937 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecac15bc10 con 0x7fecac0b7a60 2024-04-14T08:34:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.700+0000 7fecb3236640 1 -- 172.21.15.172:0/3181043937 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecac15c1d0 con 0x7fecac0b7a60 2024-04-14T08:34:39.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.701+0000 7fecb3236640 1 -- 172.21.15.172:0/3181043937 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecac075490 con 0x7fecac0b7a60 2024-04-14T08:34:39.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.702+0000 7feca97fa640 1 -- 172.21.15.172:0/3181043937 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feca0018020 con 0x7fecac0b7a60 2024-04-14T08:34:39.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.702+0000 7feca97fa640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec9806c920 0x7fec9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:39.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.702+0000 7feca97fa640 1 -- 172.21.15.172:0/3181043937 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7feca008bb80 con 0x7fecac0b7a60 2024-04-14T08:34:39.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.702+0000 7fecb0fd2640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec9806c920 0x7fec9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:39.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.703+0000 7fecb0fd2640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec9806c920 0x7fec9806ede0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feca4035000 tx=0x7feca4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:39.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.707+0000 7feca97fa640 1 -- 172.21.15.172:0/3181043937 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7feca001f020 con 0x7fecac0b7a60 2024-04-14T08:34:39.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.884+0000 7fecb3236640 1 -- 172.21.15.172:0/3181043937 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fecac0748f0 con 0x7fec9806c920 2024-04-14T08:34:39.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.896+0000 7feca97fa640 1 -- 172.21.15.172:0/3181043937 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247296 (secure 0 0 0) 0x7fecac0748f0 con 0x7fec9806c920 2024-04-14T08:34:39.900 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:39.901 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:34:39.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.899+0000 7fecb3236640 1 -- 172.21.15.172:0/3181043937 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec9806c920 msgr2=0x7fec9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.899+0000 7fecb3236640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec9806c920 0x7fec9806ede0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feca4035000 tx=0x7feca4030000 comp rx=0 tx=0).stop 2024-04-14T08:34:39.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.899+0000 7fecb3236640 1 -- 172.21.15.172:0/3181043937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecac0b7a60 msgr2=0x7fecac15c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:39.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.899+0000 7fecb3236640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecac0b7a60 0x7fecac15c9d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7feca000b410 tx=0x7feca000b8d0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.899+0000 7fecb3236640 1 -- 172.21.15.172:0/3181043937 shutdown_connections 2024-04-14T08:34:39.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.899+0000 7fecb3236640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec9806c920 0x7fec9806ede0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.899+0000 7fecb3236640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fecac0cd410 0x7fecac0cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.899+0000 7fecb3236640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecac0b7a60 0x7fecac15c9d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.899+0000 7fecb3236640 1 --2- 172.21.15.172:0/3181043937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecac08c4c0 0x7fecac159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:39.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.899+0000 7fecb3236640 1 -- 172.21.15.172:0/3181043937 >> 172.21.15.172:0/3181043937 conn(0x7fecac08c000 msgr2=0x7fecac06f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:39.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.899+0000 7fecb3236640 1 -- 172.21.15.172:0/3181043937 shutdown_connections 2024-04-14T08:34:39.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:39.899+0000 7fecb3236640 1 -- 172.21.15.172:0/3181043937 wait complete. 2024-04-14T08:34:39.915 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-04-14T08:34:39.141054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221808,"kb_used_data":5616,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752912,"statfs":{"total":773094113280,"available":772866981888,"internally_reserved":0,"allocated":5750784,"data_stored":3169616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.741238"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408023+0000","last_change":"2024-04-14T08:34:34.408023+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408023+0000","last_undegraded":"2024-04-14T08:34:34.408023+0000","last_fullsized":"2024-04-14T08:34:34.408023+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.441184+0000","last_change":"2024-04-14T08:34:35.434456+0000","last_active":"2024-04-14T08:34:35.441184+0000","last_peered":"2024-04-14T08:34:35.441184+0000","last_clean":"2024-04-14T08:34:35.441184+0000","last_became_active":"2024-04-14T08:34:35.434158+0000","last_became_peered":"2024-04-14T08:34:35.434158+0000","last_unstale":"2024-04-14T08:34:35.441184+0000","last_undegraded":"2024-04-14T08:34:35.441184+0000","last_fullsized":"2024-04-14T08:34:35.441184+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:40:13.833057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414650+0000","last_change":"2024-04-14T08:34:34.414650+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414650+0000","last_undegraded":"2024-04-14T08:34:34.414650+0000","last_fullsized":"2024-04-14T08:34:34.414650+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.437687+0000","last_change":"2024-04-14T08:34:35.430927+0000","last_active":"2024-04-14T08:34:35.437687+0000","last_peered":"2024-04-14T08:34:35.437687+0000","last_clean":"2024-04-14T08:34:35.437687+0000","last_became_active":"2024-04-14T08:34:35.430164+0000","last_became_peered":"2024-04-14T08:34:35.430164+0000","last_unstale":"2024-04-14T08:34:35.437687+0000","last_undegraded":"2024-04-14T08:34:35.437687+0000","last_fullsized":"2024-04-14T08:34:35.437687+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:57:15.656326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405062+0000","last_change":"2024-04-14T08:34:34.405062+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405062+0000","last_undegraded":"2024-04-14T08:34:34.405062+0000","last_fullsized":"2024-04-14T08:34:34.405062+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406804+0000","last_change":"2024-04-14T08:34:34.406804+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406804+0000","last_undegraded":"2024-04-14T08:34:34.406804+0000","last_fullsized":"2024-04-14T08:34:34.406804+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414181+0000","last_change":"2024-04-14T08:34:34.414181+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414181+0000","last_undegraded":"2024-04-14T08:34:34.414181+0000","last_fullsized":"2024-04-14T08:34:34.414181+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409156+0000","last_change":"2024-04-14T08:34:34.409156+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409156+0000","last_undegraded":"2024-04-14T08:34:34.409156+0000","last_fullsized":"2024-04-14T08:34:34.409156+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410003+0000","last_change":"2024-04-14T08:34:34.410003+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410003+0000","last_undegraded":"2024-04-14T08:34:34.410003+0000","last_fullsized":"2024-04-14T08:34:34.410003+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406692+0000","last_change":"2024-04-14T08:34:34.406692+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406692+0000","last_undegraded":"2024-04-14T08:34:34.406692+0000","last_fullsized":"2024-04-14T08:34:34.406692+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405927+0000","last_change":"2024-04-14T08:34:34.405927+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405927+0000","last_undegraded":"2024-04-14T08:34:34.405927+0000","last_fullsized":"2024-04-14T08:34:34.405927+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410934+0000","last_change":"2024-04-14T08:34:34.410934+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410934+0000","last_undegraded":"2024-04-14T08:34:34.410934+0000","last_fullsized":"2024-04-14T08:34:34.410934+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406225+0000","last_change":"2024-04-14T08:34:34.406225+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406225+0000","last_undegraded":"2024-04-14T08:34:34.406225+0000","last_fullsized":"2024-04-14T08:34:34.406225+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.411333+0000","last_change":"2024-04-14T08:34:34.411333+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.411333+0000","last_undegraded":"2024-04-14T08:34:34.411333+0000","last_fullsized":"2024-04-14T08:34:34.411333+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.437809+0000","last_change":"2024-04-14T08:34:35.428986+0000","last_active":"2024-04-14T08:34:35.437809+0000","last_peered":"2024-04-14T08:34:35.437809+0000","last_clean":"2024-04-14T08:34:35.437809+0000","last_became_active":"2024-04-14T08:34:35.426840+0000","last_became_peered":"2024-04-14T08:34:35.426840+0000","last_unstale":"2024-04-14T08:34:35.437809+0000","last_undegraded":"2024-04-14T08:34:35.437809+0000","last_fullsized":"2024-04-14T08:34:35.437809+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:55:45.179250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406390+0000","last_change":"2024-04-14T08:34:34.406390+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406390+0000","last_undegraded":"2024-04-14T08:34:34.406390+0000","last_fullsized":"2024-04-14T08:34:34.406390+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414078+0000","last_change":"2024-04-14T08:34:34.414078+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414078+0000","last_undegraded":"2024-04-14T08:34:34.414078+0000","last_fullsized":"2024-04-14T08:34:34.414078+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.441163+0000","last_change":"2024-04-14T08:34:35.434292+0000","last_active":"2024-04-14T08:34:35.441163+0000","last_peered":"2024-04-14T08:34:35.441163+0000","last_clean":"2024-04-14T08:34:35.441163+0000","last_became_active":"2024-04-14T08:34:35.434018+0000","last_became_peered":"2024-04-14T08:34:35.434018+0000","last_unstale":"2024-04-14T08:34:35.441163+0000","last_undegraded":"2024-04-14T08:34:35.441163+0000","last_fullsized":"2024-04-14T08:34:35.441163+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:22:04.896638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.412491+0000","last_change":"2024-04-14T08:34:34.412491+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.412491+0000","last_undegraded":"2024-04-14T08:34:34.412491+0000","last_fullsized":"2024-04-14T08:34:34.412491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410141+0000","last_change":"2024-04-14T08:34:34.410141+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410141+0000","last_undegraded":"2024-04-14T08:34:34.410141+0000","last_fullsized":"2024-04-14T08:34:34.410141+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408219+0000","last_change":"2024-04-14T08:34:34.408219+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408219+0000","last_undegraded":"2024-04-14T08:34:34.408219+0000","last_fullsized":"2024-04-14T08:34:34.408219+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.407965+0000","last_change":"2024-04-14T08:34:34.407965+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.407965+0000","last_undegraded":"2024-04-14T08:34:34.407965+0000","last_fullsized":"2024-04-14T08:34:34.407965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405857+0000","last_change":"2024-04-14T08:34:34.405857+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405857+0000","last_undegraded":"2024-04-14T08:34:34.405857+0000","last_fullsized":"2024-04-14T08:34:34.405857+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410886+0000","last_change":"2024-04-14T08:34:34.410886+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410886+0000","last_undegraded":"2024-04-14T08:34:34.410886+0000","last_fullsized":"2024-04-14T08:34:34.410886+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413719+0000","last_change":"2024-04-14T08:34:34.413719+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413719+0000","last_undegraded":"2024-04-14T08:34:34.413719+0000","last_fullsized":"2024-04-14T08:34:34.413719+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410342+0000","last_change":"2024-04-14T08:34:34.410342+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410342+0000","last_undegraded":"2024-04-14T08:34:34.410342+0000","last_fullsized":"2024-04-14T08:34:34.410342+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406527+0000","last_change":"2024-04-14T08:34:34.406527+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406527+0000","last_undegraded":"2024-04-14T08:34:34.406527+0000","last_fullsized":"2024-04-14T08:34:34.406527+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408765+0000","last_change":"2024-04-14T08:34:34.408765+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408765+0000","last_undegraded":"2024-04-14T08:34:34.408765+0000","last_fullsized":"2024-04-14T08:34:34.408765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409655+0000","last_change":"2024-04-14T08:34:34.409655+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409655+0000","last_undegraded":"2024-04-14T08:34:34.409655+0000","last_fullsized":"2024-04-14T08:34:34.409655+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.438111+0000","last_change":"2024-04-14T08:34:35.431510+0000","last_active":"2024-04-14T08:34:35.438111+0000","last_peered":"2024-04-14T08:34:35.438111+0000","last_clean":"2024-04-14T08:34:35.438111+0000","last_became_active":"2024-04-14T08:34:35.430980+0000","last_became_peered":"2024-04-14T08:34:35.430980+0000","last_unstale":"2024-04-14T08:34:35.438111+0000","last_undegraded":"2024-04-14T08:34:35.438111+0000","last_fullsized":"2024-04-14T08:34:35.438111+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:13:28.555180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408105+0000","last_change":"2024-04-14T08:34:34.408105+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408105+0000","last_undegraded":"2024-04-14T08:34:34.408105+0000","last_fullsized":"2024-04-14T08:34:34.408105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.411421+0000","last_change":"2024-04-14T08:34:34.411421+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.411421+0000","last_undegraded":"2024-04-14T08:34:34.411421+0000","last_fullsized":"2024-04-14T08:34:34.411421+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.440794+0000","last_change":"2024-04-14T08:34:35.434489+0000","last_active":"2024-04-14T08:34:35.440794+0000","last_peered":"2024-04-14T08:34:35.440794+0000","last_clean":"2024-04-14T08:34:35.440794+0000","last_became_active":"2024-04-14T08:34:35.434142+0000","last_became_peered":"2024-04-14T08:34:35.434142+0000","last_unstale":"2024-04-14T08:34:35.440794+0000","last_undegraded":"2024-04-14T08:34:35.440794+0000","last_fullsized":"2024-04-14T08:34:35.440794+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:24:33.215747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.415922+0000","last_change":"2024-04-14T08:34:34.414663+0000","last_active":"2024-04-14T08:34:34.415922+0000","last_peered":"2024-04-14T08:34:34.415922+0000","last_clean":"2024-04-14T08:34:34.415922+0000","last_became_active":"2024-04-14T08:34:34.414390+0000","last_became_peered":"2024-04-14T08:34:34.414390+0000","last_unstale":"2024-04-14T08:34:34.415922+0000","last_undegraded":"2024-04-14T08:34:34.415922+0000","last_fullsized":"2024-04-14T08:34:34.415922+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:30:49.246797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409109+0000","last_change":"2024-04-14T08:34:34.409109+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409109+0000","last_undegraded":"2024-04-14T08:34:34.409109+0000","last_fullsized":"2024-04-14T08:34:34.409109+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.420847+0000","last_change":"2024-04-14T08:34:34.415460+0000","last_active":"2024-04-14T08:34:34.420847+0000","last_peered":"2024-04-14T08:34:34.420847+0000","last_clean":"2024-04-14T08:34:34.420847+0000","last_became_active":"2024-04-14T08:34:34.415176+0000","last_became_peered":"2024-04-14T08:34:34.415176+0000","last_unstale":"2024-04-14T08:34:34.420847+0000","last_undegraded":"2024-04-14T08:34:34.420847+0000","last_fullsized":"2024-04-14T08:34:34.420847+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:49.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410262+0000","last_change":"2024-04-14T08:34:34.410262+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410262+0000","last_undegraded":"2024-04-14T08:34:34.410262+0000","last_fullsized":"2024-04-14T08:34:34.410262+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.420592+0000","last_change":"2024-04-14T08:34:34.413412+0000","last_active":"2024-04-14T08:34:34.420592+0000","last_peered":"2024-04-14T08:34:34.420592+0000","last_clean":"2024-04-14T08:34:34.420592+0000","last_became_active":"2024-04-14T08:34:34.413050+0000","last_became_peered":"2024-04-14T08:34:34.413050+0000","last_unstale":"2024-04-14T08:34:34.420592+0000","last_undegraded":"2024-04-14T08:34:34.420592+0000","last_fullsized":"2024-04-14T08:34:34.420592+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:10.773823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410996+0000","last_change":"2024-04-14T08:34:34.410996+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410996+0000","last_undegraded":"2024-04-14T08:34:34.410996+0000","last_fullsized":"2024-04-14T08:34:34.410996+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.427275+0000","last_change":"2024-04-14T08:34:34.423624+0000","last_active":"2024-04-14T08:34:34.427275+0000","last_peered":"2024-04-14T08:34:34.427275+0000","last_clean":"2024-04-14T08:34:34.427275+0000","last_became_active":"2024-04-14T08:34:34.423373+0000","last_became_peered":"2024-04-14T08:34:34.423373+0000","last_unstale":"2024-04-14T08:34:34.427275+0000","last_undegraded":"2024-04-14T08:34:34.427275+0000","last_fullsized":"2024-04-14T08:34:34.427275+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406292+0000","last_change":"2024-04-14T08:34:34.406292+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406292+0000","last_undegraded":"2024-04-14T08:34:34.406292+0000","last_fullsized":"2024-04-14T08:34:34.406292+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424001+0000","last_change":"2024-04-14T08:34:34.420330+0000","last_active":"2024-04-14T08:34:34.424001+0000","last_peered":"2024-04-14T08:34:34.424001+0000","last_clean":"2024-04-14T08:34:34.424001+0000","last_became_active":"2024-04-14T08:34:34.419768+0000","last_became_peered":"2024-04-14T08:34:34.419768+0000","last_unstale":"2024-04-14T08:34:34.424001+0000","last_undegraded":"2024-04-14T08:34:34.424001+0000","last_fullsized":"2024-04-14T08:34:34.424001+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:23:36.394817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.439163+0000","last_change":"2024-04-14T08:34:35.431263+0000","last_active":"2024-04-14T08:34:35.439163+0000","last_peered":"2024-04-14T08:34:35.439163+0000","last_clean":"2024-04-14T08:34:35.439163+0000","last_became_active":"2024-04-14T08:34:35.431022+0000","last_became_peered":"2024-04-14T08:34:35.431022+0000","last_unstale":"2024-04-14T08:34:35.439163+0000","last_undegraded":"2024-04-14T08:34:35.439163+0000","last_fullsized":"2024-04-14T08:34:35.439163+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:56:32.524197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.416293+0000","last_change":"2024-04-14T08:34:34.413550+0000","last_active":"2024-04-14T08:34:34.416293+0000","last_peered":"2024-04-14T08:34:34.416293+0000","last_clean":"2024-04-14T08:34:34.416293+0000","last_became_active":"2024-04-14T08:34:34.413268+0000","last_became_peered":"2024-04-14T08:34:34.413268+0000","last_unstale":"2024-04-14T08:34:34.416293+0000","last_undegraded":"2024-04-14T08:34:34.416293+0000","last_fullsized":"2024-04-14T08:34:34.416293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:04.319595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408225+0000","last_change":"2024-04-14T08:34:34.408225+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408225+0000","last_undegraded":"2024-04-14T08:34:34.408225+0000","last_fullsized":"2024-04-14T08:34:34.408225+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.418469+0000","last_change":"2024-04-14T08:34:34.415171+0000","last_active":"2024-04-14T08:34:34.418469+0000","last_peered":"2024-04-14T08:34:34.418469+0000","last_clean":"2024-04-14T08:34:34.418469+0000","last_became_active":"2024-04-14T08:34:34.414890+0000","last_became_peered":"2024-04-14T08:34:34.414890+0000","last_unstale":"2024-04-14T08:34:34.418469+0000","last_undegraded":"2024-04-14T08:34:34.418469+0000","last_fullsized":"2024-04-14T08:34:34.418469+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:11.472608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.444176+0000","last_change":"2024-04-14T08:34:35.440848+0000","last_active":"2024-04-14T08:34:35.444176+0000","last_peered":"2024-04-14T08:34:35.444176+0000","last_clean":"2024-04-14T08:34:35.444176+0000","last_became_active":"2024-04-14T08:34:35.440519+0000","last_became_peered":"2024-04-14T08:34:35.440519+0000","last_unstale":"2024-04-14T08:34:35.444176+0000","last_undegraded":"2024-04-14T08:34:35.444176+0000","last_fullsized":"2024-04-14T08:34:35.444176+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:13:42.361716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424504+0000","last_change":"2024-04-14T08:34:34.419189+0000","last_active":"2024-04-14T08:34:34.424504+0000","last_peered":"2024-04-14T08:34:34.424504+0000","last_clean":"2024-04-14T08:34:34.424504+0000","last_became_active":"2024-04-14T08:34:34.418954+0000","last_became_peered":"2024-04-14T08:34:34.418954+0000","last_unstale":"2024-04-14T08:34:34.424504+0000","last_undegraded":"2024-04-14T08:34:34.424504+0000","last_fullsized":"2024-04-14T08:34:34.424504+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:54:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405634+0000","last_change":"2024-04-14T08:34:34.405634+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405634+0000","last_undegraded":"2024-04-14T08:34:34.405634+0000","last_fullsized":"2024-04-14T08:34:34.405634+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.419794+0000","last_change":"2024-04-14T08:34:34.416908+0000","last_active":"2024-04-14T08:34:35.419794+0000","last_peered":"2024-04-14T08:34:35.419794+0000","last_clean":"2024-04-14T08:34:35.419794+0000","last_became_active":"2024-04-14T08:34:34.416617+0000","last_became_peered":"2024-04-14T08:34:34.416617+0000","last_unstale":"2024-04-14T08:34:35.419794+0000","last_undegraded":"2024-04-14T08:34:35.419794+0000","last_fullsized":"2024-04-14T08:34:35.419794+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:11.743901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413953+0000","last_change":"2024-04-14T08:34:34.413953+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413953+0000","last_undegraded":"2024-04-14T08:34:34.413953+0000","last_fullsized":"2024-04-14T08:34:34.413953+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424753+0000","last_change":"2024-04-14T08:34:34.421136+0000","last_active":"2024-04-14T08:34:34.424753+0000","last_peered":"2024-04-14T08:34:34.424753+0000","last_clean":"2024-04-14T08:34:34.424753+0000","last_became_active":"2024-04-14T08:34:34.420809+0000","last_became_peered":"2024-04-14T08:34:34.420809+0000","last_unstale":"2024-04-14T08:34:34.424753+0000","last_undegraded":"2024-04-14T08:34:34.424753+0000","last_fullsized":"2024-04-14T08:34:34.424753+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:49.127474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.407542+0000","last_change":"2024-04-14T08:34:34.407542+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.407542+0000","last_undegraded":"2024-04-14T08:34:34.407542+0000","last_fullsized":"2024-04-14T08:34:34.407542+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.448678+0000","last_change":"2024-04-14T08:34:34.422319+0000","last_active":"2024-04-14T08:34:35.448678+0000","last_peered":"2024-04-14T08:34:35.448678+0000","last_clean":"2024-04-14T08:34:35.448678+0000","last_became_active":"2024-04-14T08:34:34.421978+0000","last_became_peered":"2024-04-14T08:34:34.421978+0000","last_unstale":"2024-04-14T08:34:35.448678+0000","last_undegraded":"2024-04-14T08:34:35.448678+0000","last_fullsized":"2024-04-14T08:34:35.448678+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:11.381349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.404550+0000","last_change":"2024-04-14T08:34:34.404550+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.404550+0000","last_undegraded":"2024-04-14T08:34:34.404550+0000","last_fullsized":"2024-04-14T08:34:34.404550+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423937+0000","last_change":"2024-04-14T08:34:34.420518+0000","last_active":"2024-04-14T08:34:34.423937+0000","last_peered":"2024-04-14T08:34:34.423937+0000","last_clean":"2024-04-14T08:34:34.423937+0000","last_became_active":"2024-04-14T08:34:34.419065+0000","last_became_peered":"2024-04-14T08:34:34.419065+0000","last_unstale":"2024-04-14T08:34:34.423937+0000","last_undegraded":"2024-04-14T08:34:34.423937+0000","last_fullsized":"2024-04-14T08:34:34.423937+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:36.033184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410498+0000","last_change":"2024-04-14T08:34:34.410498+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410498+0000","last_undegraded":"2024-04-14T08:34:34.410498+0000","last_fullsized":"2024-04-14T08:34:34.410498+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423983+0000","last_change":"2024-04-14T08:34:34.421506+0000","last_active":"2024-04-14T08:34:34.423983+0000","last_peered":"2024-04-14T08:34:34.423983+0000","last_clean":"2024-04-14T08:34:34.423983+0000","last_became_active":"2024-04-14T08:34:34.420091+0000","last_became_peered":"2024-04-14T08:34:34.420091+0000","last_unstale":"2024-04-14T08:34:34.423983+0000","last_undegraded":"2024-04-14T08:34:34.423983+0000","last_fullsized":"2024-04-14T08:34:34.423983+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:30:35.547680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.412081+0000","last_change":"2024-04-14T08:34:34.412081+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.412081+0000","last_undegraded":"2024-04-14T08:34:34.412081+0000","last_fullsized":"2024-04-14T08:34:34.412081+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.429886+0000","last_change":"2024-04-14T08:34:34.424435+0000","last_active":"2024-04-14T08:34:34.429886+0000","last_peered":"2024-04-14T08:34:34.429886+0000","last_clean":"2024-04-14T08:34:34.429886+0000","last_became_active":"2024-04-14T08:34:34.424085+0000","last_became_peered":"2024-04-14T08:34:34.424085+0000","last_unstale":"2024-04-14T08:34:34.429886+0000","last_undegraded":"2024-04-14T08:34:34.429886+0000","last_fullsized":"2024-04-14T08:34:34.429886+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:04.319595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409928+0000","last_change":"2024-04-14T08:34:34.409928+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409928+0000","last_undegraded":"2024-04-14T08:34:34.409928+0000","last_fullsized":"2024-04-14T08:34:34.409928+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.419215+0000","last_change":"2024-04-14T08:34:34.414833+0000","last_active":"2024-04-14T08:34:35.419215+0000","last_peered":"2024-04-14T08:34:35.419215+0000","last_clean":"2024-04-14T08:34:35.419215+0000","last_became_active":"2024-04-14T08:34:34.414553+0000","last_became_peered":"2024-04-14T08:34:34.414553+0000","last_unstale":"2024-04-14T08:34:35.419215+0000","last_undegraded":"2024-04-14T08:34:35.419215+0000","last_fullsized":"2024-04-14T08:34:35.419215+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.414459+0000","last_change":"2024-04-14T08:34:34.414459+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.414459+0000","last_undegraded":"2024-04-14T08:34:34.414459+0000","last_fullsized":"2024-04-14T08:34:34.414459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.422034+0000","last_change":"2024-04-14T08:34:34.416091+0000","last_active":"2024-04-14T08:34:34.422034+0000","last_peered":"2024-04-14T08:34:34.422034+0000","last_clean":"2024-04-14T08:34:34.422034+0000","last_became_active":"2024-04-14T08:34:34.415653+0000","last_became_peered":"2024-04-14T08:34:34.415653+0000","last_unstale":"2024-04-14T08:34:34.422034+0000","last_undegraded":"2024-04-14T08:34:34.422034+0000","last_fullsized":"2024-04-14T08:34:34.422034+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.407664+0000","last_change":"2024-04-14T08:34:34.407664+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.407664+0000","last_undegraded":"2024-04-14T08:34:34.407664+0000","last_fullsized":"2024-04-14T08:34:34.407664+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.425676+0000","last_change":"2024-04-14T08:34:34.423763+0000","last_active":"2024-04-14T08:34:34.425676+0000","last_peered":"2024-04-14T08:34:34.425676+0000","last_clean":"2024-04-14T08:34:34.425676+0000","last_became_active":"2024-04-14T08:34:34.423333+0000","last_became_peered":"2024-04-14T08:34:34.423333+0000","last_unstale":"2024-04-14T08:34:34.425676+0000","last_undegraded":"2024-04-14T08:34:34.425676+0000","last_fullsized":"2024-04-14T08:34:34.425676+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:22:38.569478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405513+0000","last_change":"2024-04-14T08:34:34.405513+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405513+0000","last_undegraded":"2024-04-14T08:34:34.405513+0000","last_fullsized":"2024-04-14T08:34:34.405513+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.418045+0000","last_change":"2024-04-14T08:34:34.416569+0000","last_active":"2024-04-14T08:34:34.418045+0000","last_peered":"2024-04-14T08:34:34.418045+0000","last_clean":"2024-04-14T08:34:34.418045+0000","last_became_active":"2024-04-14T08:34:34.416306+0000","last_became_peered":"2024-04-14T08:34:34.416306+0000","last_unstale":"2024-04-14T08:34:34.418045+0000","last_undegraded":"2024-04-14T08:34:34.418045+0000","last_fullsized":"2024-04-14T08:34:34.418045+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:56.093503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405734+0000","last_change":"2024-04-14T08:34:34.405734+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405734+0000","last_undegraded":"2024-04-14T08:34:34.405734+0000","last_fullsized":"2024-04-14T08:34:34.405734+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.427399+0000","last_change":"2024-04-14T08:34:34.422659+0000","last_active":"2024-04-14T08:34:34.427399+0000","last_peered":"2024-04-14T08:34:34.427399+0000","last_clean":"2024-04-14T08:34:34.427399+0000","last_became_active":"2024-04-14T08:34:34.422434+0000","last_became_peered":"2024-04-14T08:34:34.422434+0000","last_unstale":"2024-04-14T08:34:34.427399+0000","last_undegraded":"2024-04-14T08:34:34.427399+0000","last_fullsized":"2024-04-14T08:34:34.427399+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:59:48.942079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.405327+0000","last_change":"2024-04-14T08:34:34.405327+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.405327+0000","last_undegraded":"2024-04-14T08:34:34.405327+0000","last_fullsized":"2024-04-14T08:34:34.405327+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.427365+0000","last_change":"2024-04-14T08:34:34.422117+0000","last_active":"2024-04-14T08:34:34.427365+0000","last_peered":"2024-04-14T08:34:34.427365+0000","last_clean":"2024-04-14T08:34:34.427365+0000","last_became_active":"2024-04-14T08:34:34.421504+0000","last_became_peered":"2024-04-14T08:34:34.421504+0000","last_unstale":"2024-04-14T08:34:34.427365+0000","last_undegraded":"2024-04-14T08:34:34.427365+0000","last_fullsized":"2024-04-14T08:34:34.427365+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:26:52.383137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409160+0000","last_change":"2024-04-14T08:34:34.409160+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409160+0000","last_undegraded":"2024-04-14T08:34:34.409160+0000","last_fullsized":"2024-04-14T08:34:34.409160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.420594+0000","last_change":"2024-04-14T08:34:34.413030+0000","last_active":"2024-04-14T08:34:34.420594+0000","last_peered":"2024-04-14T08:34:34.420594+0000","last_clean":"2024-04-14T08:34:34.420594+0000","last_became_active":"2024-04-14T08:34:34.412774+0000","last_became_peered":"2024-04-14T08:34:34.412774+0000","last_unstale":"2024-04-14T08:34:34.420594+0000","last_undegraded":"2024-04-14T08:34:34.420594+0000","last_fullsized":"2024-04-14T08:34:34.420594+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:04.319595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409571+0000","last_change":"2024-04-14T08:34:34.409571+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409571+0000","last_undegraded":"2024-04-14T08:34:34.409571+0000","last_fullsized":"2024-04-14T08:34:34.409571+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.421957+0000","last_change":"2024-04-14T08:34:34.417238+0000","last_active":"2024-04-14T08:34:34.421957+0000","last_peered":"2024-04-14T08:34:34.421957+0000","last_clean":"2024-04-14T08:34:34.421957+0000","last_became_active":"2024-04-14T08:34:34.417002+0000","last_became_peered":"2024-04-14T08:34:34.417002+0000","last_unstale":"2024-04-14T08:34:34.421957+0000","last_undegraded":"2024-04-14T08:34:34.421957+0000","last_fullsized":"2024-04-14T08:34:34.421957+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:11.743901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413056+0000","last_change":"2024-04-14T08:34:34.413056+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413056+0000","last_undegraded":"2024-04-14T08:34:34.413056+0000","last_fullsized":"2024-04-14T08:34:34.413056+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.425644+0000","last_change":"2024-04-14T08:34:34.423744+0000","last_active":"2024-04-14T08:34:34.425644+0000","last_peered":"2024-04-14T08:34:34.425644+0000","last_clean":"2024-04-14T08:34:34.425644+0000","last_became_active":"2024-04-14T08:34:34.423394+0000","last_became_peered":"2024-04-14T08:34:34.423394+0000","last_unstale":"2024-04-14T08:34:34.425644+0000","last_undegraded":"2024-04-14T08:34:34.425644+0000","last_fullsized":"2024-04-14T08:34:34.425644+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:32.608024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406093+0000","last_change":"2024-04-14T08:34:34.406093+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406093+0000","last_undegraded":"2024-04-14T08:34:34.406093+0000","last_fullsized":"2024-04-14T08:34:34.406093+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.414356+0000","last_change":"2024-04-14T08:34:34.411096+0000","last_active":"2024-04-14T08:34:34.414356+0000","last_peered":"2024-04-14T08:34:34.414356+0000","last_clean":"2024-04-14T08:34:34.414356+0000","last_became_active":"2024-04-14T08:34:34.410841+0000","last_became_peered":"2024-04-14T08:34:34.410841+0000","last_unstale":"2024-04-14T08:34:34.414356+0000","last_undegraded":"2024-04-14T08:34:34.414356+0000","last_fullsized":"2024-04-14T08:34:34.414356+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:12:26.461355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.410965+0000","last_change":"2024-04-14T08:34:34.410965+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.410965+0000","last_undegraded":"2024-04-14T08:34:34.410965+0000","last_fullsized":"2024-04-14T08:34:34.410965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.429860+0000","last_change":"2024-04-14T08:34:34.424338+0000","last_active":"2024-04-14T08:34:34.429860+0000","last_peered":"2024-04-14T08:34:34.429860+0000","last_clean":"2024-04-14T08:34:34.429860+0000","last_became_active":"2024-04-14T08:34:34.424057+0000","last_became_peered":"2024-04-14T08:34:34.424057+0000","last_unstale":"2024-04-14T08:34:34.429860+0000","last_undegraded":"2024-04-14T08:34:34.429860+0000","last_fullsized":"2024-04-14T08:34:34.429860+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:10.773823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.406288+0000","last_change":"2024-04-14T08:34:34.406288+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.406288+0000","last_undegraded":"2024-04-14T08:34:34.406288+0000","last_fullsized":"2024-04-14T08:34:34.406288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.420306+0000","last_change":"2024-04-14T08:34:34.416682+0000","last_active":"2024-04-14T08:34:35.420306+0000","last_peered":"2024-04-14T08:34:35.420306+0000","last_clean":"2024-04-14T08:34:35.420306+0000","last_became_active":"2024-04-14T08:34:34.416172+0000","last_became_peered":"2024-04-14T08:34:34.416172+0000","last_unstale":"2024-04-14T08:34:35.420306+0000","last_undegraded":"2024-04-14T08:34:35.420306+0000","last_fullsized":"2024-04-14T08:34:35.420306+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:36.033184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408713+0000","last_change":"2024-04-14T08:34:34.408713+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408713+0000","last_undegraded":"2024-04-14T08:34:34.408713+0000","last_fullsized":"2024-04-14T08:34:34.408713+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423234+0000","last_change":"2024-04-14T08:34:34.420330+0000","last_active":"2024-04-14T08:34:34.423234+0000","last_peered":"2024-04-14T08:34:34.423234+0000","last_clean":"2024-04-14T08:34:34.423234+0000","last_became_active":"2024-04-14T08:34:34.420116+0000","last_became_peered":"2024-04-14T08:34:34.420116+0000","last_unstale":"2024-04-14T08:34:34.423234+0000","last_undegraded":"2024-04-14T08:34:34.423234+0000","last_fullsized":"2024-04-14T08:34:34.423234+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:41:19.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408837+0000","last_change":"2024-04-14T08:34:34.408837+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408837+0000","last_undegraded":"2024-04-14T08:34:34.408837+0000","last_fullsized":"2024-04-14T08:34:34.408837+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.422091+0000","last_change":"2024-04-14T08:34:34.419528+0000","last_active":"2024-04-14T08:34:34.422091+0000","last_peered":"2024-04-14T08:34:34.422091+0000","last_clean":"2024-04-14T08:34:34.422091+0000","last_became_active":"2024-04-14T08:34:34.419258+0000","last_became_peered":"2024-04-14T08:34:34.419258+0000","last_unstale":"2024-04-14T08:34:34.422091+0000","last_undegraded":"2024-04-14T08:34:34.422091+0000","last_fullsized":"2024-04-14T08:34:34.422091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:11.381349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.409171+0000","last_change":"2024-04-14T08:34:34.409171+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.409171+0000","last_undegraded":"2024-04-14T08:34:34.409171+0000","last_fullsized":"2024-04-14T08:34:34.409171+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423956+0000","last_change":"2024-04-14T08:34:34.421662+0000","last_active":"2024-04-14T08:34:34.423956+0000","last_peered":"2024-04-14T08:34:34.423956+0000","last_clean":"2024-04-14T08:34:34.423956+0000","last_became_active":"2024-04-14T08:34:34.421382+0000","last_became_peered":"2024-04-14T08:34:34.421382+0000","last_unstale":"2024-04-14T08:34:34.423956+0000","last_undegraded":"2024-04-14T08:34:34.423956+0000","last_fullsized":"2024-04-14T08:34:34.423956+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:55:44.175481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.413461+0000","last_change":"2024-04-14T08:34:34.413461+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.413461+0000","last_undegraded":"2024-04-14T08:34:34.413461+0000","last_fullsized":"2024-04-14T08:34:34.413461+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424542+0000","last_change":"2024-04-14T08:34:34.419857+0000","last_active":"2024-04-14T08:34:34.424542+0000","last_peered":"2024-04-14T08:34:34.424542+0000","last_clean":"2024-04-14T08:34:34.424542+0000","last_became_active":"2024-04-14T08:34:34.419548+0000","last_became_peered":"2024-04-14T08:34:34.419548+0000","last_unstale":"2024-04-14T08:34:34.424542+0000","last_undegraded":"2024-04-14T08:34:34.424542+0000","last_fullsized":"2024-04-14T08:34:34.424542+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:10.773823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408288+0000","last_change":"2024-04-14T08:34:34.408288+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408288+0000","last_undegraded":"2024-04-14T08:34:34.408288+0000","last_fullsized":"2024-04-14T08:34:34.408288+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.424716+0000","last_change":"2024-04-14T08:34:34.420316+0000","last_active":"2024-04-14T08:34:34.424716+0000","last_peered":"2024-04-14T08:34:34.424716+0000","last_clean":"2024-04-14T08:34:34.424716+0000","last_became_active":"2024-04-14T08:34:34.420093+0000","last_became_peered":"2024-04-14T08:34:34.420093+0000","last_unstale":"2024-04-14T08:34:34.424716+0000","last_undegraded":"2024-04-14T08:34:34.424716+0000","last_fullsized":"2024-04-14T08:34:34.424716+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:44:18.315252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-04-14T08:34:34.408798+0000","last_change":"2024-04-14T08:34:34.408798+0000","last_active":"2024-04-14T08:34:34.399007+0000","last_peered":"2024-04-14T08:34:34.399007+0000","last_clean":"2024-04-14T08:34:34.399007+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:34:34.408798+0000","last_undegraded":"2024-04-14T08:34:34.408798+0000","last_fullsized":"2024-04-14T08:34:34.408798+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-14T08:34:34.423952+0000","last_change":"2024-04-14T08:34:34.421158+0000","last_active":"2024-04-14T08:34:34.423952+0000","last_peered":"2024-04-14T08:34:34.423952+0000","last_clean":"2024-04-14T08:34:34.423952+0000","last_became_active":"2024-04-14T08:34:34.420780+0000","last_became_peered":"2024-04-14T08:34:34.420780+0000","last_unstale":"2024-04-14T08:34:34.423952+0000","last_undegraded":"2024-04-14T08:34:34.423952+0000","last_fullsized":"2024-04-14T08:34:34.423952+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:44:10.910055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":226,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-14T08:34:35.418795+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:34:35.418795+0000","last_peered":"2024-04-14T08:34:35.418795+0000","last_clean":"2024-04-14T08:34:35.418795+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:34:35.418795+0000","last_undegraded":"2024-04-14T08:34:35.418795+0000","last_fullsized":"2024-04-14T08:34:35.418795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1183744,"data_stored":846191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":557056,"data_stored":241865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1163264,"data_stored":838900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:34:40.698 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:34:40.697+0000 7f3a039f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:34:42.920 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:34:43.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.022+0000 7fe4b86a2640 1 Processor -- start 2024-04-14T08:34:43.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.022+0000 7fe4b86a2640 1 -- start start 2024-04-14T08:34:43.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b86a2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4b00d2830 0x7fe4b00d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b86a2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4b015e890 0x7fe4b00d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b86a2640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe4b00cedd0 0x7fe4b01621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b86a2640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b00cff30 con 0x7fe4b015e890 2024-04-14T08:34:43.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b86a2640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b00d2350 con 0x7fe4b00cedd0 2024-04-14T08:34:43.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b86a2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b00d3150 con 0x7fe4b00d2830 2024-04-14T08:34:43.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b5c3d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4b015e890 0x7fe4b00d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b5c3d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4b015e890 0x7fe4b00d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45728/0 (socket says 172.21.15.172:45728) 2024-04-14T08:34:43.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b5c3d640 1 -- 172.21.15.172:0/3895119787 learned_addr learned my addr 172.21.15.172:0/3895119787 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:43.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b643e640 1 --2- 172.21.15.172:0/3895119787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4b00d2830 0x7fe4b00d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b6c3f640 1 --2- 172.21.15.172:0/3895119787 >> v2:172.21.15.177:3300/0 conn(0x7fe4b00cedd0 0x7fe4b01621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b5c3d640 1 -- 172.21.15.172:0/3895119787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4b00d2830 msgr2=0x7fe4b00d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:43.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b5c3d640 1 --2- 172.21.15.172:0/3895119787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4b00d2830 0x7fe4b00d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b5c3d640 1 -- 172.21.15.172:0/3895119787 >> v2:172.21.15.177:3300/0 conn(0x7fe4b00cedd0 msgr2=0x7fe4b01621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:43.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b5c3d640 1 --2- 172.21.15.172:0/3895119787 >> v2:172.21.15.177:3300/0 conn(0x7fe4b00cedd0 0x7fe4b01621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.023+0000 7fe4b5c3d640 1 -- 172.21.15.172:0/3895119787 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b00cf320 con 0x7fe4b015e890 2024-04-14T08:34:43.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.024+0000 7fe4b5c3d640 1 --2- 172.21.15.172:0/3895119787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b015e890 0x7fe4b00d1300 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00d0530 tx=0x7fe4ac02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c306fd247391f0f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:43.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.025+0000 7fe4b543c640 1 -- 172.21.15.172:0/3895119787 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac034070 con 0x7fe4b015e890 2024-04-14T08:34:43.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.025+0000 7fe4b543c640 1 -- 172.21.15.172:0/3895119787 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac030860 con 0x7fe4b015e890 2024-04-14T08:34:43.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.025+0000 7fe4b543c640 1 -- 172.21.15.172:0/3895119787 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac030c30 con 0x7fe4b015e890 2024-04-14T08:34:43.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.025+0000 7fe4b86a2640 1 -- 172.21.15.172:0/3895119787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b015e890 msgr2=0x7fe4b00d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.025+0000 7fe4b86a2640 1 --2- 172.21.15.172:0/3895119787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b015e890 0x7fe4b00d1300 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00d0530 tx=0x7fe4ac02b9a0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.025+0000 7fe4b86a2640 1 -- 172.21.15.172:0/3895119787 shutdown_connections 2024-04-14T08:34:43.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.025+0000 7fe4b86a2640 1 --2- 172.21.15.172:0/3895119787 >> v2:172.21.15.177:3300/0 conn(0x7fe4b00cedd0 0x7fe4b01621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.025+0000 7fe4b86a2640 1 --2- 172.21.15.172:0/3895119787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b015e890 0x7fe4b00d1300 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.025+0000 7fe4b86a2640 1 --2- 172.21.15.172:0/3895119787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4b00d2830 0x7fe4b00d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.025+0000 7fe4b86a2640 1 -- 172.21.15.172:0/3895119787 >> 172.21.15.172:0/3895119787 conn(0x7fe4b009d590 msgr2=0x7fe4b00d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:43.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.025+0000 7fe4b86a2640 1 -- 172.21.15.172:0/3895119787 shutdown_connections 2024-04-14T08:34:43.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.025+0000 7fe4b86a2640 1 -- 172.21.15.172:0/3895119787 wait complete. 2024-04-14T08:34:43.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.026+0000 7fe4b86a2640 1 Processor -- start 2024-04-14T08:34:43.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.026+0000 7fe4b86a2640 1 -- start start 2024-04-14T08:34:43.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.027+0000 7fe4b86a2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4b00cedd0 0x7fe4b0159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.027+0000 7fe4b86a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b00d2830 0x7fe4b015f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.027+0000 7fe4b86a2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4b015e890 0x7fe4b00b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.027+0000 7fe4b86a2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b00cfbc0 con 0x7fe4b00d2830 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.027+0000 7fe4b86a2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b00cf8e0 con 0x7fe4b00cedd0 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.027+0000 7fe4b86a2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b00d1840 con 0x7fe4b015e890 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.027+0000 7fe4b5c3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b00d2830 0x7fe4b015f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.027+0000 7fe4b5c3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b00d2830 0x7fe4b015f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45738/0 (socket says 172.21.15.172:45738) 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.027+0000 7fe4b5c3d640 1 -- 172.21.15.172:0/2830294217 learned_addr learned my addr 172.21.15.172:0/2830294217 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.027+0000 7fe4b6c3f640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4b015e890 0x7fe4b00b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.027+0000 7fe4b643e640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4b00cedd0 0x7fe4b0159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.028+0000 7fe4b5c3d640 1 -- 172.21.15.172:0/2830294217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4b015e890 msgr2=0x7fe4b00b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.028+0000 7fe4b5c3d640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4b015e890 0x7fe4b00b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.028+0000 7fe4b5c3d640 1 -- 172.21.15.172:0/2830294217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4b00cedd0 msgr2=0x7fe4b0159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.028+0000 7fe4b5c3d640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4b00cedd0 0x7fe4b0159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.028+0000 7fe4b5c3d640 1 -- 172.21.15.172:0/2830294217 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b00c7f20 con 0x7fe4b00d2830 2024-04-14T08:34:43.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.028+0000 7fe4b6c3f640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4b015e890 0x7fe4b00b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:43.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.028+0000 7fe4b5c3d640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b00d2830 0x7fe4b015f810 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac000c00 tx=0x7fe4ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:43.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.028+0000 7fe4a6ffd640 1 -- 172.21.15.172:0/2830294217 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac048070 con 0x7fe4b00d2830 2024-04-14T08:34:43.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.028+0000 7fe4a6ffd640 1 -- 172.21.15.172:0/2830294217 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ac030660 con 0x7fe4b00d2830 2024-04-14T08:34:43.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.028+0000 7fe4b86a2640 1 -- 172.21.15.172:0/2830294217 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b0160cd0 con 0x7fe4b00d2830 2024-04-14T08:34:43.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.028+0000 7fe4a6ffd640 1 -- 172.21.15.172:0/2830294217 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac034040 con 0x7fe4b00d2830 2024-04-14T08:34:43.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.028+0000 7fe4b86a2640 1 -- 172.21.15.172:0/2830294217 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b0159d80 con 0x7fe4b00d2830 2024-04-14T08:34:43.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.029+0000 7fe4b86a2640 1 -- 172.21.15.172:0/2830294217 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b0068520 con 0x7fe4b00d2830 2024-04-14T08:34:43.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.030+0000 7fe4a6ffd640 1 -- 172.21.15.172:0/2830294217 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4ac04b070 con 0x7fe4b00d2830 2024-04-14T08:34:43.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.030+0000 7fe4a6ffd640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe49406c920 0x7fe49406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.030+0000 7fe4b643e640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe49406c920 0x7fe49406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.030+0000 7fe4a6ffd640 1 -- 172.21.15.172:0/2830294217 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fe4ac0add20 con 0x7fe4b00d2830 2024-04-14T08:34:43.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.031+0000 7fe4b643e640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe49406c920 0x7fe49406ede0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0006fd0 tx=0x7fe4a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:43.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.035+0000 7fe4a6ffd640 1 -- 172.21.15.172:0/2830294217 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe4ac07bfc0 con 0x7fe4b00d2830 2024-04-14T08:34:43.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.211+0000 7fe4b86a2640 1 -- 172.21.15.172:0/2830294217 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4b015e0b0 con 0x7fe49406c920 2024-04-14T08:34:43.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.217+0000 7fe4a6ffd640 1 -- 172.21.15.172:0/2830294217 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7fe4b015e0b0 con 0x7fe49406c920 2024-04-14T08:34:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:43.222 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:34:43.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.220+0000 7fe4b86a2640 1 -- 172.21.15.172:0/2830294217 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe49406c920 msgr2=0x7fe49406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.220+0000 7fe4b86a2640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe49406c920 0x7fe49406ede0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0006fd0 tx=0x7fe4a0008040 comp rx=0 tx=0).stop 2024-04-14T08:34:43.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.220+0000 7fe4b86a2640 1 -- 172.21.15.172:0/2830294217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b00d2830 msgr2=0x7fe4b015f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.220+0000 7fe4b86a2640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b00d2830 0x7fe4b015f810 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac000c00 tx=0x7fe4ac004730 comp rx=0 tx=0).stop 2024-04-14T08:34:43.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.220+0000 7fe4b86a2640 1 -- 172.21.15.172:0/2830294217 shutdown_connections 2024-04-14T08:34:43.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.220+0000 7fe4b86a2640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe49406c920 0x7fe49406ede0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.220+0000 7fe4b86a2640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4b015e890 0x7fe4b00b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.220+0000 7fe4b86a2640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b00d2830 0x7fe4b015f810 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.220+0000 7fe4b86a2640 1 --2- 172.21.15.172:0/2830294217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4b00cedd0 0x7fe4b0159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.220+0000 7fe4b86a2640 1 -- 172.21.15.172:0/2830294217 >> 172.21.15.172:0/2830294217 conn(0x7fe4b009d590 msgr2=0x7fe4b00d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:43.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.220+0000 7fe4b86a2640 1 -- 172.21.15.172:0/2830294217 shutdown_connections 2024-04-14T08:34:43.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.220+0000 7fe4b86a2640 1 -- 172.21.15.172:0/2830294217 wait complete. 2024-04-14T08:34:43.232 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-04-14T08:34:43.141802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222024,"kb_used_data":5864,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752696,"statfs":{"total":773094113280,"available":772866760704,"internally_reserved":0,"allocated":6004736,"data_stored":3284684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001386"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994850+0000","last_change":"2024-04-14T08:34:35.433031+0000","last_active":"2024-04-14T08:34:36.994850+0000","last_peered":"2024-04-14T08:34:36.994850+0000","last_clean":"2024-04-14T08:34:36.994850+0000","last_became_active":"2024-04-14T08:34:35.432748+0000","last_became_peered":"2024-04-14T08:34:35.432748+0000","last_unstale":"2024-04-14T08:34:36.994850+0000","last_undegraded":"2024-04-14T08:34:36.994850+0000","last_fullsized":"2024-04-14T08:34:36.994850+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:24:33.215747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.430492+0000","last_change":"2024-04-14T08:34:35.434456+0000","last_active":"2024-04-14T08:34:36.430492+0000","last_peered":"2024-04-14T08:34:36.430492+0000","last_clean":"2024-04-14T08:34:36.430492+0000","last_became_active":"2024-04-14T08:34:35.434158+0000","last_became_peered":"2024-04-14T08:34:35.434158+0000","last_unstale":"2024-04-14T08:34:36.430492+0000","last_undegraded":"2024-04-14T08:34:36.430492+0000","last_fullsized":"2024-04-14T08:34:36.430492+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:46:04.614148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436654+0000","last_change":"2024-04-14T08:34:35.430633+0000","last_active":"2024-04-14T08:34:36.436654+0000","last_peered":"2024-04-14T08:34:36.436654+0000","last_clean":"2024-04-14T08:34:36.436654+0000","last_became_active":"2024-04-14T08:34:35.429283+0000","last_became_peered":"2024-04-14T08:34:35.429283+0000","last_unstale":"2024-04-14T08:34:36.436654+0000","last_undegraded":"2024-04-14T08:34:36.436654+0000","last_fullsized":"2024-04-14T08:34:36.436654+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:32:27.118752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.430667+0000","last_change":"2024-04-14T08:34:35.430927+0000","last_active":"2024-04-14T08:34:36.430667+0000","last_peered":"2024-04-14T08:34:36.430667+0000","last_clean":"2024-04-14T08:34:36.430667+0000","last_became_active":"2024-04-14T08:34:35.430164+0000","last_became_peered":"2024-04-14T08:34:35.430164+0000","last_unstale":"2024-04-14T08:34:36.430667+0000","last_undegraded":"2024-04-14T08:34:36.430667+0000","last_fullsized":"2024-04-14T08:34:36.430667+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:10:34.162254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.429564+0000","last_change":"2024-04-14T08:34:35.427915+0000","last_active":"2024-04-14T08:34:36.429564+0000","last_peered":"2024-04-14T08:34:36.429564+0000","last_clean":"2024-04-14T08:34:36.429564+0000","last_became_active":"2024-04-14T08:34:35.426589+0000","last_became_peered":"2024-04-14T08:34:35.426589+0000","last_unstale":"2024-04-14T08:34:36.429564+0000","last_undegraded":"2024-04-14T08:34:36.429564+0000","last_fullsized":"2024-04-14T08:34:36.429564+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:12.747669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994689+0000","last_change":"2024-04-14T08:34:35.435314+0000","last_active":"2024-04-14T08:34:36.994689+0000","last_peered":"2024-04-14T08:34:36.994689+0000","last_clean":"2024-04-14T08:34:36.994689+0000","last_became_active":"2024-04-14T08:34:35.435024+0000","last_became_peered":"2024-04-14T08:34:35.435024+0000","last_unstale":"2024-04-14T08:34:36.994689+0000","last_undegraded":"2024-04-14T08:34:36.994689+0000","last_fullsized":"2024-04-14T08:34:36.994689+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:13:28.555180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436181+0000","last_change":"2024-04-14T08:34:35.429953+0000","last_active":"2024-04-14T08:34:36.436181+0000","last_peered":"2024-04-14T08:34:36.436181+0000","last_clean":"2024-04-14T08:34:36.436181+0000","last_became_active":"2024-04-14T08:34:35.429726+0000","last_became_peered":"2024-04-14T08:34:35.429726+0000","last_unstale":"2024-04-14T08:34:36.436181+0000","last_undegraded":"2024-04-14T08:34:36.436181+0000","last_fullsized":"2024-04-14T08:34:36.436181+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:20.360754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426619+0000","last_change":"2024-04-14T08:34:35.447867+0000","last_active":"2024-04-14T08:34:36.426619+0000","last_peered":"2024-04-14T08:34:36.426619+0000","last_clean":"2024-04-14T08:34:36.426619+0000","last_became_active":"2024-04-14T08:34:35.447549+0000","last_became_peered":"2024-04-14T08:34:35.447549+0000","last_unstale":"2024-04-14T08:34:36.426619+0000","last_undegraded":"2024-04-14T08:34:36.426619+0000","last_fullsized":"2024-04-14T08:34:36.426619+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:26:12.159368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.427077+0000","last_change":"2024-04-14T08:34:35.442294+0000","last_active":"2024-04-14T08:34:36.427077+0000","last_peered":"2024-04-14T08:34:36.427077+0000","last_clean":"2024-04-14T08:34:36.427077+0000","last_became_active":"2024-04-14T08:34:35.441910+0000","last_became_peered":"2024-04-14T08:34:35.441910+0000","last_unstale":"2024-04-14T08:34:36.427077+0000","last_undegraded":"2024-04-14T08:34:36.427077+0000","last_fullsized":"2024-04-14T08:34:36.427077+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:13:25.632328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994564+0000","last_change":"2024-04-14T08:34:35.434774+0000","last_active":"2024-04-14T08:34:36.994564+0000","last_peered":"2024-04-14T08:34:36.994564+0000","last_clean":"2024-04-14T08:34:36.994564+0000","last_became_active":"2024-04-14T08:34:35.434512+0000","last_became_peered":"2024-04-14T08:34:35.434512+0000","last_unstale":"2024-04-14T08:34:36.994564+0000","last_undegraded":"2024-04-14T08:34:36.994564+0000","last_fullsized":"2024-04-14T08:34:36.994564+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:56:32.524197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.995150+0000","last_change":"2024-04-14T08:34:35.433003+0000","last_active":"2024-04-14T08:34:36.995150+0000","last_peered":"2024-04-14T08:34:36.995150+0000","last_clean":"2024-04-14T08:34:36.995150+0000","last_became_active":"2024-04-14T08:34:35.432744+0000","last_became_peered":"2024-04-14T08:34:35.432744+0000","last_unstale":"2024-04-14T08:34:36.995150+0000","last_undegraded":"2024-04-14T08:34:36.995150+0000","last_fullsized":"2024-04-14T08:34:36.995150+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:45:39.059574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426955+0000","last_change":"2024-04-14T08:34:35.436318+0000","last_active":"2024-04-14T08:34:36.426955+0000","last_peered":"2024-04-14T08:34:36.426955+0000","last_clean":"2024-04-14T08:34:36.426955+0000","last_became_active":"2024-04-14T08:34:35.436035+0000","last_became_peered":"2024-04-14T08:34:35.436035+0000","last_unstale":"2024-04-14T08:34:36.426955+0000","last_undegraded":"2024-04-14T08:34:36.426955+0000","last_fullsized":"2024-04-14T08:34:36.426955+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:31:14.753068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994356+0000","last_change":"2024-04-14T08:34:35.436785+0000","last_active":"2024-04-14T08:34:36.994356+0000","last_peered":"2024-04-14T08:34:36.994356+0000","last_clean":"2024-04-14T08:34:36.994356+0000","last_became_active":"2024-04-14T08:34:35.436497+0000","last_became_peered":"2024-04-14T08:34:35.436497+0000","last_unstale":"2024-04-14T08:34:36.994356+0000","last_undegraded":"2024-04-14T08:34:36.994356+0000","last_fullsized":"2024-04-14T08:34:36.994356+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:22:04.896638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436440+0000","last_change":"2024-04-14T08:34:35.429053+0000","last_active":"2024-04-14T08:34:36.436440+0000","last_peered":"2024-04-14T08:34:36.436440+0000","last_clean":"2024-04-14T08:34:36.436440+0000","last_became_active":"2024-04-14T08:34:35.428811+0000","last_became_peered":"2024-04-14T08:34:35.428811+0000","last_unstale":"2024-04-14T08:34:36.436440+0000","last_undegraded":"2024-04-14T08:34:36.436440+0000","last_fullsized":"2024-04-14T08:34:36.436440+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:06:23.360332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.430063+0000","last_change":"2024-04-14T08:34:35.428986+0000","last_active":"2024-04-14T08:34:36.430063+0000","last_peered":"2024-04-14T08:34:36.430063+0000","last_clean":"2024-04-14T08:34:36.430063+0000","last_became_active":"2024-04-14T08:34:35.426840+0000","last_became_peered":"2024-04-14T08:34:35.426840+0000","last_unstale":"2024-04-14T08:34:36.430063+0000","last_undegraded":"2024-04-14T08:34:36.430063+0000","last_fullsized":"2024-04-14T08:34:36.430063+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:58:20.439066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994323+0000","last_change":"2024-04-14T08:34:35.435588+0000","last_active":"2024-04-14T08:34:36.994323+0000","last_peered":"2024-04-14T08:34:36.994323+0000","last_clean":"2024-04-14T08:34:36.994323+0000","last_became_active":"2024-04-14T08:34:35.435319+0000","last_became_peered":"2024-04-14T08:34:35.435319+0000","last_unstale":"2024-04-14T08:34:36.994323+0000","last_undegraded":"2024-04-14T08:34:36.994323+0000","last_fullsized":"2024-04-14T08:34:36.994323+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:12.385117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436921+0000","last_change":"2024-04-14T08:34:35.441236+0000","last_active":"2024-04-14T08:34:36.436921+0000","last_peered":"2024-04-14T08:34:36.436921+0000","last_clean":"2024-04-14T08:34:36.436921+0000","last_became_active":"2024-04-14T08:34:35.440715+0000","last_became_peered":"2024-04-14T08:34:35.440715+0000","last_unstale":"2024-04-14T08:34:36.436921+0000","last_undegraded":"2024-04-14T08:34:36.436921+0000","last_fullsized":"2024-04-14T08:34:36.436921+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:01:03.618418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.430630+0000","last_change":"2024-04-14T08:34:35.434292+0000","last_active":"2024-04-14T08:34:36.430630+0000","last_peered":"2024-04-14T08:34:36.430630+0000","last_clean":"2024-04-14T08:34:36.430630+0000","last_became_active":"2024-04-14T08:34:35.434018+0000","last_became_peered":"2024-04-14T08:34:35.434018+0000","last_unstale":"2024-04-14T08:34:36.430630+0000","last_undegraded":"2024-04-14T08:34:36.430630+0000","last_fullsized":"2024-04-14T08:34:36.430630+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:09:51.363444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436734+0000","last_change":"2024-04-14T08:34:35.430300+0000","last_active":"2024-04-14T08:34:36.436734+0000","last_peered":"2024-04-14T08:34:36.436734+0000","last_clean":"2024-04-14T08:34:36.436734+0000","last_became_active":"2024-04-14T08:34:35.430056+0000","last_became_peered":"2024-04-14T08:34:35.430056+0000","last_unstale":"2024-04-14T08:34:36.436734+0000","last_undegraded":"2024-04-14T08:34:36.436734+0000","last_fullsized":"2024-04-14T08:34:36.436734+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:13:25.632328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994921+0000","last_change":"2024-04-14T08:34:35.435556+0000","last_active":"2024-04-14T08:34:36.994921+0000","last_peered":"2024-04-14T08:34:36.994921+0000","last_clean":"2024-04-14T08:34:36.994921+0000","last_became_active":"2024-04-14T08:34:35.435291+0000","last_became_peered":"2024-04-14T08:34:35.435291+0000","last_unstale":"2024-04-14T08:34:36.994921+0000","last_undegraded":"2024-04-14T08:34:36.994921+0000","last_fullsized":"2024-04-14T08:34:36.994921+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:56:32.524197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994845+0000","last_change":"2024-04-14T08:34:35.433110+0000","last_active":"2024-04-14T08:34:36.994845+0000","last_peered":"2024-04-14T08:34:36.994845+0000","last_clean":"2024-04-14T08:34:36.994845+0000","last_became_active":"2024-04-14T08:34:35.432841+0000","last_became_peered":"2024-04-14T08:34:35.432841+0000","last_unstale":"2024-04-14T08:34:36.994845+0000","last_undegraded":"2024-04-14T08:34:36.994845+0000","last_fullsized":"2024-04-14T08:34:36.994845+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:55:45.179250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426568+0000","last_change":"2024-04-14T08:34:35.446624+0000","last_active":"2024-04-14T08:34:36.426568+0000","last_peered":"2024-04-14T08:34:36.426568+0000","last_clean":"2024-04-14T08:34:36.426568+0000","last_became_active":"2024-04-14T08:34:35.446343+0000","last_became_peered":"2024-04-14T08:34:35.446343+0000","last_unstale":"2024-04-14T08:34:36.426568+0000","last_undegraded":"2024-04-14T08:34:36.426568+0000","last_fullsized":"2024-04-14T08:34:36.426568+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:38:57.861180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994617+0000","last_change":"2024-04-14T08:34:35.432944+0000","last_active":"2024-04-14T08:34:36.994617+0000","last_peered":"2024-04-14T08:34:36.994617+0000","last_clean":"2024-04-14T08:34:36.994617+0000","last_became_active":"2024-04-14T08:34:35.432683+0000","last_became_peered":"2024-04-14T08:34:35.432683+0000","last_unstale":"2024-04-14T08:34:36.994617+0000","last_undegraded":"2024-04-14T08:34:36.994617+0000","last_fullsized":"2024-04-14T08:34:36.994617+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:18:15.707867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436147+0000","last_change":"2024-04-14T08:34:35.428757+0000","last_active":"2024-04-14T08:34:36.436147+0000","last_peered":"2024-04-14T08:34:36.436147+0000","last_clean":"2024-04-14T08:34:36.436147+0000","last_became_active":"2024-04-14T08:34:35.428486+0000","last_became_peered":"2024-04-14T08:34:35.428486+0000","last_unstale":"2024-04-14T08:34:36.436147+0000","last_undegraded":"2024-04-14T08:34:36.436147+0000","last_fullsized":"2024-04-14T08:34:36.436147+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:30:10.006927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436957+0000","last_change":"2024-04-14T08:34:35.439811+0000","last_active":"2024-04-14T08:34:36.436957+0000","last_peered":"2024-04-14T08:34:36.436957+0000","last_clean":"2024-04-14T08:34:36.436957+0000","last_became_active":"2024-04-14T08:34:35.439536+0000","last_became_peered":"2024-04-14T08:34:35.439536+0000","last_unstale":"2024-04-14T08:34:36.436957+0000","last_undegraded":"2024-04-14T08:34:36.436957+0000","last_fullsized":"2024-04-14T08:34:36.436957+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:10:57.330696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426808+0000","last_change":"2024-04-14T08:34:35.431967+0000","last_active":"2024-04-14T08:34:36.426808+0000","last_peered":"2024-04-14T08:34:36.426808+0000","last_clean":"2024-04-14T08:34:36.426808+0000","last_became_active":"2024-04-14T08:34:35.431705+0000","last_became_peered":"2024-04-14T08:34:35.431705+0000","last_unstale":"2024-04-14T08:34:36.426808+0000","last_undegraded":"2024-04-14T08:34:36.426808+0000","last_fullsized":"2024-04-14T08:34:36.426808+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:14:17.333608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994884+0000","last_change":"2024-04-14T08:34:35.426267+0000","last_active":"2024-04-14T08:34:36.994884+0000","last_peered":"2024-04-14T08:34:36.994884+0000","last_clean":"2024-04-14T08:34:36.994884+0000","last_became_active":"2024-04-14T08:34:35.426005+0000","last_became_peered":"2024-04-14T08:34:35.426005+0000","last_unstale":"2024-04-14T08:34:36.994884+0000","last_undegraded":"2024-04-14T08:34:36.994884+0000","last_fullsized":"2024-04-14T08:34:36.994884+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:13:42.361716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994938+0000","last_change":"2024-04-14T08:34:35.427243+0000","last_active":"2024-04-14T08:34:36.994938+0000","last_peered":"2024-04-14T08:34:36.994938+0000","last_clean":"2024-04-14T08:34:36.994938+0000","last_became_active":"2024-04-14T08:34:35.426966+0000","last_became_peered":"2024-04-14T08:34:35.426966+0000","last_unstale":"2024-04-14T08:34:36.994938+0000","last_undegraded":"2024-04-14T08:34:36.994938+0000","last_fullsized":"2024-04-14T08:34:36.994938+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:13:28.555180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436604+0000","last_change":"2024-04-14T08:34:35.430580+0000","last_active":"2024-04-14T08:34:36.436604+0000","last_peered":"2024-04-14T08:34:36.436604+0000","last_clean":"2024-04-14T08:34:36.436604+0000","last_became_active":"2024-04-14T08:34:35.430331+0000","last_became_peered":"2024-04-14T08:34:35.430331+0000","last_unstale":"2024-04-14T08:34:36.436604+0000","last_undegraded":"2024-04-14T08:34:36.436604+0000","last_fullsized":"2024-04-14T08:34:36.436604+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:14:17.333608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.429691+0000","last_change":"2024-04-14T08:34:35.431510+0000","last_active":"2024-04-14T08:34:36.429691+0000","last_peered":"2024-04-14T08:34:36.429691+0000","last_clean":"2024-04-14T08:34:36.429691+0000","last_became_active":"2024-04-14T08:34:35.430980+0000","last_became_peered":"2024-04-14T08:34:35.430980+0000","last_unstale":"2024-04-14T08:34:36.429691+0000","last_undegraded":"2024-04-14T08:34:36.429691+0000","last_fullsized":"2024-04-14T08:34:36.429691+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:20:49.067306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436240+0000","last_change":"2024-04-14T08:34:35.429737+0000","last_active":"2024-04-14T08:34:36.436240+0000","last_peered":"2024-04-14T08:34:36.436240+0000","last_clean":"2024-04-14T08:34:36.436240+0000","last_became_active":"2024-04-14T08:34:35.429152+0000","last_became_peered":"2024-04-14T08:34:35.429152+0000","last_unstale":"2024-04-14T08:34:36.436240+0000","last_undegraded":"2024-04-14T08:34:36.436240+0000","last_fullsized":"2024-04-14T08:34:36.436240+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:38:57.861180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.816095+0000","last_change":"2024-04-14T08:34:35.433012+0000","last_active":"2024-04-14T08:34:36.816095+0000","last_peered":"2024-04-14T08:34:36.816095+0000","last_clean":"2024-04-14T08:34:36.816095+0000","last_became_active":"2024-04-14T08:34:35.432760+0000","last_became_peered":"2024-04-14T08:34:35.432760+0000","last_unstale":"2024-04-14T08:34:36.816095+0000","last_undegraded":"2024-04-14T08:34:36.816095+0000","last_fullsized":"2024-04-14T08:34:36.816095+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:38:37.795791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.429548+0000","last_change":"2024-04-14T08:34:35.434489+0000","last_active":"2024-04-14T08:34:36.429548+0000","last_peered":"2024-04-14T08:34:36.429548+0000","last_clean":"2024-04-14T08:34:36.429548+0000","last_became_active":"2024-04-14T08:34:35.434142+0000","last_became_peered":"2024-04-14T08:34:35.434142+0000","last_unstale":"2024-04-14T08:34:36.429548+0000","last_undegraded":"2024-04-14T08:34:36.429548+0000","last_fullsized":"2024-04-14T08:34:36.429548+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:38:37.795791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994988+0000","last_change":"2024-04-14T08:34:34.414663+0000","last_active":"2024-04-14T08:34:36.994988+0000","last_peered":"2024-04-14T08:34:36.994988+0000","last_clean":"2024-04-14T08:34:36.994988+0000","last_became_active":"2024-04-14T08:34:34.414390+0000","last_became_peered":"2024-04-14T08:34:34.414390+0000","last_unstale":"2024-04-14T08:34:36.994988+0000","last_undegraded":"2024-04-14T08:34:36.994988+0000","last_fullsized":"2024-04-14T08:34:36.994988+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:44:10.910055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.816152+0000","last_change":"2024-04-14T08:34:35.438908+0000","last_active":"2024-04-14T08:34:36.816152+0000","last_peered":"2024-04-14T08:34:36.816152+0000","last_clean":"2024-04-14T08:34:36.816152+0000","last_became_active":"2024-04-14T08:34:35.438525+0000","last_became_peered":"2024-04-14T08:34:35.438525+0000","last_unstale":"2024-04-14T08:34:36.816152+0000","last_undegraded":"2024-04-14T08:34:36.816152+0000","last_fullsized":"2024-04-14T08:34:36.816152+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:58:20.439066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994669+0000","last_change":"2024-04-14T08:34:34.415460+0000","last_active":"2024-04-14T08:34:36.994669+0000","last_peered":"2024-04-14T08:34:36.994669+0000","last_clean":"2024-04-14T08:34:36.994669+0000","last_became_active":"2024-04-14T08:34:34.415176+0000","last_became_peered":"2024-04-14T08:34:34.415176+0000","last_unstale":"2024-04-14T08:34:36.994669+0000","last_undegraded":"2024-04-14T08:34:36.994669+0000","last_fullsized":"2024-04-14T08:34:36.994669+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:44:10.910055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994689+0000","last_change":"2024-04-14T08:34:35.436220+0000","last_active":"2024-04-14T08:34:36.994689+0000","last_peered":"2024-04-14T08:34:36.994689+0000","last_clean":"2024-04-14T08:34:36.994689+0000","last_became_active":"2024-04-14T08:34:35.435945+0000","last_became_peered":"2024-04-14T08:34:35.435945+0000","last_unstale":"2024-04-14T08:34:36.994689+0000","last_undegraded":"2024-04-14T08:34:36.994689+0000","last_fullsized":"2024-04-14T08:34:36.994689+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:37.036953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994929+0000","last_change":"2024-04-14T08:34:34.413412+0000","last_active":"2024-04-14T08:34:36.994929+0000","last_peered":"2024-04-14T08:34:36.994929+0000","last_clean":"2024-04-14T08:34:36.994929+0000","last_became_active":"2024-04-14T08:34:34.413050+0000","last_became_peered":"2024-04-14T08:34:34.413050+0000","last_unstale":"2024-04-14T08:34:36.994929+0000","last_undegraded":"2024-04-14T08:34:36.994929+0000","last_fullsized":"2024-04-14T08:34:36.994929+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:41:19.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426564+0000","last_change":"2024-04-14T08:34:35.432543+0000","last_active":"2024-04-14T08:34:36.426564+0000","last_peered":"2024-04-14T08:34:36.426564+0000","last_clean":"2024-04-14T08:34:36.426564+0000","last_became_active":"2024-04-14T08:34:35.432304+0000","last_became_peered":"2024-04-14T08:34:35.432304+0000","last_unstale":"2024-04-14T08:34:36.426564+0000","last_undegraded":"2024-04-14T08:34:36.426564+0000","last_fullsized":"2024-04-14T08:34:36.426564+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:20.360754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.816333+0000","last_change":"2024-04-14T08:34:34.423624+0000","last_active":"2024-04-14T08:34:36.816333+0000","last_peered":"2024-04-14T08:34:36.816333+0000","last_clean":"2024-04-14T08:34:36.816333+0000","last_became_active":"2024-04-14T08:34:34.423373+0000","last_became_peered":"2024-04-14T08:34:34.423373+0000","last_unstale":"2024-04-14T08:34:36.816333+0000","last_undegraded":"2024-04-14T08:34:36.816333+0000","last_fullsized":"2024-04-14T08:34:36.816333+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:11.381349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.430484+0000","last_change":"2024-04-14T08:34:35.424758+0000","last_active":"2024-04-14T08:34:36.430484+0000","last_peered":"2024-04-14T08:34:36.430484+0000","last_clean":"2024-04-14T08:34:36.430484+0000","last_became_active":"2024-04-14T08:34:35.424525+0000","last_became_peered":"2024-04-14T08:34:35.424525+0000","last_unstale":"2024-04-14T08:34:36.430484+0000","last_undegraded":"2024-04-14T08:34:36.430484+0000","last_fullsized":"2024-04-14T08:34:36.430484+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:37.036953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436397+0000","last_change":"2024-04-14T08:34:34.420330+0000","last_active":"2024-04-14T08:34:36.436397+0000","last_peered":"2024-04-14T08:34:36.436397+0000","last_clean":"2024-04-14T08:34:36.436397+0000","last_became_active":"2024-04-14T08:34:34.419768+0000","last_became_peered":"2024-04-14T08:34:34.419768+0000","last_unstale":"2024-04-14T08:34:36.436397+0000","last_undegraded":"2024-04-14T08:34:36.436397+0000","last_fullsized":"2024-04-14T08:34:36.436397+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:52:43.180402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.430747+0000","last_change":"2024-04-14T08:34:35.431263+0000","last_active":"2024-04-14T08:34:36.430747+0000","last_peered":"2024-04-14T08:34:36.430747+0000","last_clean":"2024-04-14T08:34:36.430747+0000","last_became_active":"2024-04-14T08:34:35.431022+0000","last_became_peered":"2024-04-14T08:34:35.431022+0000","last_unstale":"2024-04-14T08:34:36.430747+0000","last_undegraded":"2024-04-14T08:34:36.430747+0000","last_fullsized":"2024-04-14T08:34:36.430747+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:29:20.346363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.429491+0000","last_change":"2024-04-14T08:34:34.413550+0000","last_active":"2024-04-14T08:34:36.429491+0000","last_peered":"2024-04-14T08:34:36.429491+0000","last_clean":"2024-04-14T08:34:36.429491+0000","last_became_active":"2024-04-14T08:34:34.413268+0000","last_became_peered":"2024-04-14T08:34:34.413268+0000","last_unstale":"2024-04-14T08:34:36.429491+0000","last_undegraded":"2024-04-14T08:34:36.429491+0000","last_fullsized":"2024-04-14T08:34:36.429491+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:44:10.910055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.427022+0000","last_change":"2024-04-14T08:34:35.431311+0000","last_active":"2024-04-14T08:34:36.427022+0000","last_peered":"2024-04-14T08:34:36.427022+0000","last_clean":"2024-04-14T08:34:36.427022+0000","last_became_active":"2024-04-14T08:34:35.431076+0000","last_became_peered":"2024-04-14T08:34:35.431076+0000","last_unstale":"2024-04-14T08:34:36.427022+0000","last_undegraded":"2024-04-14T08:34:36.427022+0000","last_fullsized":"2024-04-14T08:34:36.427022+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:32:27.118752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994631+0000","last_change":"2024-04-14T08:34:34.415171+0000","last_active":"2024-04-14T08:34:36.994631+0000","last_peered":"2024-04-14T08:34:36.994631+0000","last_clean":"2024-04-14T08:34:36.994631+0000","last_became_active":"2024-04-14T08:34:34.414890+0000","last_became_peered":"2024-04-14T08:34:34.414890+0000","last_unstale":"2024-04-14T08:34:36.994631+0000","last_undegraded":"2024-04-14T08:34:36.994631+0000","last_fullsized":"2024-04-14T08:34:36.994631+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:41:19.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.430198+0000","last_change":"2024-04-14T08:34:35.440848+0000","last_active":"2024-04-14T08:34:36.430198+0000","last_peered":"2024-04-14T08:34:36.430198+0000","last_clean":"2024-04-14T08:34:36.430198+0000","last_became_active":"2024-04-14T08:34:35.440519+0000","last_became_peered":"2024-04-14T08:34:35.440519+0000","last_unstale":"2024-04-14T08:34:36.430198+0000","last_undegraded":"2024-04-14T08:34:36.430198+0000","last_fullsized":"2024-04-14T08:34:36.430198+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:56.783669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436313+0000","last_change":"2024-04-14T08:34:34.419189+0000","last_active":"2024-04-14T08:34:36.436313+0000","last_peered":"2024-04-14T08:34:36.436313+0000","last_clean":"2024-04-14T08:34:36.436313+0000","last_became_active":"2024-04-14T08:34:34.418954+0000","last_became_peered":"2024-04-14T08:34:34.418954+0000","last_unstale":"2024-04-14T08:34:36.436313+0000","last_undegraded":"2024-04-14T08:34:36.436313+0000","last_fullsized":"2024-04-14T08:34:36.436313+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:11:51.074321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.429352+0000","last_change":"2024-04-14T08:34:35.434585+0000","last_active":"2024-04-14T08:34:36.429352+0000","last_peered":"2024-04-14T08:34:36.429352+0000","last_clean":"2024-04-14T08:34:36.429352+0000","last_became_active":"2024-04-14T08:34:35.434287+0000","last_became_peered":"2024-04-14T08:34:35.434287+0000","last_unstale":"2024-04-14T08:34:36.429352+0000","last_undegraded":"2024-04-14T08:34:36.429352+0000","last_fullsized":"2024-04-14T08:34:36.429352+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:44:19.319020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.430314+0000","last_change":"2024-04-14T08:34:34.416908+0000","last_active":"2024-04-14T08:34:36.430314+0000","last_peered":"2024-04-14T08:34:36.430314+0000","last_clean":"2024-04-14T08:34:36.430314+0000","last_became_active":"2024-04-14T08:34:34.416617+0000","last_became_peered":"2024-04-14T08:34:34.416617+0000","last_unstale":"2024-04-14T08:34:36.430314+0000","last_undegraded":"2024-04-14T08:34:36.430314+0000","last_fullsized":"2024-04-14T08:34:36.430314+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:11.743901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.816352+0000","last_change":"2024-04-14T08:34:35.437683+0000","last_active":"2024-04-14T08:34:36.816352+0000","last_peered":"2024-04-14T08:34:36.816352+0000","last_clean":"2024-04-14T08:34:36.816352+0000","last_became_active":"2024-04-14T08:34:35.437294+0000","last_became_peered":"2024-04-14T08:34:35.437294+0000","last_unstale":"2024-04-14T08:34:36.816352+0000","last_undegraded":"2024-04-14T08:34:36.816352+0000","last_fullsized":"2024-04-14T08:34:36.816352+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:10:34.162254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.430499+0000","last_change":"2024-04-14T08:34:34.421136+0000","last_active":"2024-04-14T08:34:36.430499+0000","last_peered":"2024-04-14T08:34:36.430499+0000","last_clean":"2024-04-14T08:34:36.430499+0000","last_became_active":"2024-04-14T08:34:34.420809+0000","last_became_peered":"2024-04-14T08:34:34.420809+0000","last_unstale":"2024-04-14T08:34:36.430499+0000","last_undegraded":"2024-04-14T08:34:36.430499+0000","last_fullsized":"2024-04-14T08:34:36.430499+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:30:35.547680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436417+0000","last_change":"2024-04-14T08:34:35.427849+0000","last_active":"2024-04-14T08:34:36.436417+0000","last_peered":"2024-04-14T08:34:36.436417+0000","last_clean":"2024-04-14T08:34:36.436417+0000","last_became_active":"2024-04-14T08:34:35.427606+0000","last_became_peered":"2024-04-14T08:34:35.427606+0000","last_unstale":"2024-04-14T08:34:36.436417+0000","last_undegraded":"2024-04-14T08:34:36.436417+0000","last_fullsized":"2024-04-14T08:34:36.436417+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:26:12.159368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.430156+0000","last_change":"2024-04-14T08:34:34.422319+0000","last_active":"2024-04-14T08:34:36.430156+0000","last_peered":"2024-04-14T08:34:36.430156+0000","last_clean":"2024-04-14T08:34:36.430156+0000","last_became_active":"2024-04-14T08:34:34.421978+0000","last_became_peered":"2024-04-14T08:34:34.421978+0000","last_unstale":"2024-04-14T08:34:36.430156+0000","last_undegraded":"2024-04-14T08:34:36.430156+0000","last_fullsized":"2024-04-14T08:34:36.430156+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:11.381349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.429732+0000","last_change":"2024-04-14T08:34:35.429011+0000","last_active":"2024-04-14T08:34:36.429732+0000","last_peered":"2024-04-14T08:34:36.429732+0000","last_clean":"2024-04-14T08:34:36.429732+0000","last_became_active":"2024-04-14T08:34:35.427669+0000","last_became_peered":"2024-04-14T08:34:35.427669+0000","last_unstale":"2024-04-14T08:34:36.429732+0000","last_undegraded":"2024-04-14T08:34:36.429732+0000","last_fullsized":"2024-04-14T08:34:36.429732+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:12.385117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426963+0000","last_change":"2024-04-14T08:34:34.420518+0000","last_active":"2024-04-14T08:34:36.426963+0000","last_peered":"2024-04-14T08:34:36.426963+0000","last_clean":"2024-04-14T08:34:36.426963+0000","last_became_active":"2024-04-14T08:34:34.419065+0000","last_became_peered":"2024-04-14T08:34:34.419065+0000","last_unstale":"2024-04-14T08:34:36.426963+0000","last_undegraded":"2024-04-14T08:34:36.426963+0000","last_fullsized":"2024-04-14T08:34:36.426963+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:40:12.829289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.816143+0000","last_change":"2024-04-14T08:34:35.438529+0000","last_active":"2024-04-14T08:34:36.816143+0000","last_peered":"2024-04-14T08:34:36.816143+0000","last_clean":"2024-04-14T08:34:36.816143+0000","last_became_active":"2024-04-14T08:34:35.438215+0000","last_became_peered":"2024-04-14T08:34:35.438215+0000","last_unstale":"2024-04-14T08:34:36.816143+0000","last_undegraded":"2024-04-14T08:34:36.816143+0000","last_fullsized":"2024-04-14T08:34:36.816143+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:20:49.067306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436493+0000","last_change":"2024-04-14T08:34:34.421506+0000","last_active":"2024-04-14T08:34:36.436493+0000","last_peered":"2024-04-14T08:34:36.436493+0000","last_clean":"2024-04-14T08:34:36.436493+0000","last_became_active":"2024-04-14T08:34:34.420091+0000","last_became_peered":"2024-04-14T08:34:34.420091+0000","last_unstale":"2024-04-14T08:34:36.436493+0000","last_undegraded":"2024-04-14T08:34:36.436493+0000","last_fullsized":"2024-04-14T08:34:36.436493+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436799+0000","last_change":"2024-04-14T08:34:35.436935+0000","last_active":"2024-04-14T08:34:36.436799+0000","last_peered":"2024-04-14T08:34:36.436799+0000","last_clean":"2024-04-14T08:34:36.436799+0000","last_became_active":"2024-04-14T08:34:35.431413+0000","last_became_peered":"2024-04-14T08:34:35.431413+0000","last_unstale":"2024-04-14T08:34:36.436799+0000","last_undegraded":"2024-04-14T08:34:36.436799+0000","last_fullsized":"2024-04-14T08:34:36.436799+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:57:05.467415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994789+0000","last_change":"2024-04-14T08:34:34.424435+0000","last_active":"2024-04-14T08:34:36.994789+0000","last_peered":"2024-04-14T08:34:36.994789+0000","last_clean":"2024-04-14T08:34:36.994789+0000","last_became_active":"2024-04-14T08:34:34.424085+0000","last_became_peered":"2024-04-14T08:34:34.424085+0000","last_unstale":"2024-04-14T08:34:36.994789+0000","last_undegraded":"2024-04-14T08:34:36.994789+0000","last_fullsized":"2024-04-14T08:34:36.994789+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:54:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426830+0000","last_change":"2024-04-14T08:34:35.436216+0000","last_active":"2024-04-14T08:34:36.426830+0000","last_peered":"2024-04-14T08:34:36.426830+0000","last_clean":"2024-04-14T08:34:36.426830+0000","last_became_active":"2024-04-14T08:34:35.435974+0000","last_became_peered":"2024-04-14T08:34:35.435974+0000","last_unstale":"2024-04-14T08:34:36.426830+0000","last_undegraded":"2024-04-14T08:34:36.426830+0000","last_fullsized":"2024-04-14T08:34:36.426830+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:03:40.712334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.429945+0000","last_change":"2024-04-14T08:34:34.414833+0000","last_active":"2024-04-14T08:34:36.429945+0000","last_peered":"2024-04-14T08:34:36.429945+0000","last_clean":"2024-04-14T08:34:36.429945+0000","last_became_active":"2024-04-14T08:34:34.414553+0000","last_became_peered":"2024-04-14T08:34:34.414553+0000","last_unstale":"2024-04-14T08:34:36.429945+0000","last_undegraded":"2024-04-14T08:34:36.429945+0000","last_fullsized":"2024-04-14T08:34:36.429945+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:13.334940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436788+0000","last_change":"2024-04-14T08:34:35.441151+0000","last_active":"2024-04-14T08:34:36.436788+0000","last_peered":"2024-04-14T08:34:36.436788+0000","last_clean":"2024-04-14T08:34:36.436788+0000","last_became_active":"2024-04-14T08:34:35.440805+0000","last_became_peered":"2024-04-14T08:34:35.440805+0000","last_unstale":"2024-04-14T08:34:36.436788+0000","last_undegraded":"2024-04-14T08:34:36.436788+0000","last_fullsized":"2024-04-14T08:34:36.436788+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:04:54.692750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":22,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426718+0000","last_change":"2024-04-14T08:34:34.416091+0000","last_active":"2024-04-14T08:34:36.426718+0000","last_peered":"2024-04-14T08:34:36.426718+0000","last_clean":"2024-04-14T08:34:36.426718+0000","last_became_active":"2024-04-14T08:34:34.415653+0000","last_became_peered":"2024-04-14T08:34:34.415653+0000","last_unstale":"2024-04-14T08:34:36.426718+0000","last_undegraded":"2024-04-14T08:34:36.426718+0000","last_fullsized":"2024-04-14T08:34:36.426718+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:57:14.652557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426681+0000","last_change":"2024-04-14T08:34:35.435464+0000","last_active":"2024-04-14T08:34:36.426681+0000","last_peered":"2024-04-14T08:34:36.426681+0000","last_clean":"2024-04-14T08:34:36.426681+0000","last_became_active":"2024-04-14T08:34:35.435030+0000","last_became_peered":"2024-04-14T08:34:35.435030+0000","last_unstale":"2024-04-14T08:34:36.426681+0000","last_undegraded":"2024-04-14T08:34:36.426681+0000","last_fullsized":"2024-04-14T08:34:36.426681+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:06:23.360332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.437008+0000","last_change":"2024-04-14T08:34:34.423763+0000","last_active":"2024-04-14T08:34:36.437008+0000","last_peered":"2024-04-14T08:34:36.437008+0000","last_clean":"2024-04-14T08:34:36.437008+0000","last_became_active":"2024-04-14T08:34:34.423333+0000","last_became_peered":"2024-04-14T08:34:34.423333+0000","last_unstale":"2024-04-14T08:34:36.437008+0000","last_undegraded":"2024-04-14T08:34:36.437008+0000","last_fullsized":"2024-04-14T08:34:36.437008+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:11.381349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994384+0000","last_change":"2024-04-14T08:34:35.424743+0000","last_active":"2024-04-14T08:34:36.994384+0000","last_peered":"2024-04-14T08:34:36.994384+0000","last_clean":"2024-04-14T08:34:36.994384+0000","last_became_active":"2024-04-14T08:34:35.424474+0000","last_became_peered":"2024-04-14T08:34:35.424474+0000","last_unstale":"2024-04-14T08:34:36.994384+0000","last_undegraded":"2024-04-14T08:34:36.994384+0000","last_fullsized":"2024-04-14T08:34:36.994384+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:37.036953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.995149+0000","last_change":"2024-04-14T08:34:34.416569+0000","last_active":"2024-04-14T08:34:36.995149+0000","last_peered":"2024-04-14T08:34:36.995149+0000","last_clean":"2024-04-14T08:34:36.995149+0000","last_became_active":"2024-04-14T08:34:34.416306+0000","last_became_peered":"2024-04-14T08:34:34.416306+0000","last_unstale":"2024-04-14T08:34:36.995149+0000","last_undegraded":"2024-04-14T08:34:36.995149+0000","last_fullsized":"2024-04-14T08:34:36.995149+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:09:36.666447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.429362+0000","last_change":"2024-04-14T08:34:35.427292+0000","last_active":"2024-04-14T08:34:36.429362+0000","last_peered":"2024-04-14T08:34:36.429362+0000","last_clean":"2024-04-14T08:34:36.429362+0000","last_became_active":"2024-04-14T08:34:35.427040+0000","last_became_peered":"2024-04-14T08:34:35.427040+0000","last_unstale":"2024-04-14T08:34:36.429362+0000","last_undegraded":"2024-04-14T08:34:36.429362+0000","last_fullsized":"2024-04-14T08:34:36.429362+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:14.338708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994938+0000","last_change":"2024-04-14T08:34:34.422659+0000","last_active":"2024-04-14T08:34:36.994938+0000","last_peered":"2024-04-14T08:34:36.994938+0000","last_clean":"2024-04-14T08:34:36.994938+0000","last_became_active":"2024-04-14T08:34:34.422434+0000","last_became_peered":"2024-04-14T08:34:34.422434+0000","last_unstale":"2024-04-14T08:34:36.994938+0000","last_undegraded":"2024-04-14T08:34:36.994938+0000","last_fullsized":"2024-04-14T08:34:36.994938+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:49:32.608024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994492+0000","last_change":"2024-04-14T08:34:35.434033+0000","last_active":"2024-04-14T08:34:36.994492+0000","last_peered":"2024-04-14T08:34:36.994492+0000","last_clean":"2024-04-14T08:34:36.994492+0000","last_became_active":"2024-04-14T08:34:35.433136+0000","last_became_peered":"2024-04-14T08:34:35.433136+0000","last_unstale":"2024-04-14T08:34:36.994492+0000","last_undegraded":"2024-04-14T08:34:36.994492+0000","last_fullsized":"2024-04-14T08:34:36.994492+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:57:15.656326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.995030+0000","last_change":"2024-04-14T08:34:34.422117+0000","last_active":"2024-04-14T08:34:36.995030+0000","last_peered":"2024-04-14T08:34:36.995030+0000","last_clean":"2024-04-14T08:34:36.995030+0000","last_became_active":"2024-04-14T08:34:34.421504+0000","last_became_peered":"2024-04-14T08:34:34.421504+0000","last_unstale":"2024-04-14T08:34:36.995030+0000","last_undegraded":"2024-04-14T08:34:36.995030+0000","last_fullsized":"2024-04-14T08:34:36.995030+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:23:36.394817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.816194+0000","last_change":"2024-04-14T08:34:35.437345+0000","last_active":"2024-04-14T08:34:36.816194+0000","last_peered":"2024-04-14T08:34:36.816194+0000","last_clean":"2024-04-14T08:34:36.816194+0000","last_became_active":"2024-04-14T08:34:35.437052+0000","last_became_peered":"2024-04-14T08:34:35.437052+0000","last_unstale":"2024-04-14T08:34:36.816194+0000","last_undegraded":"2024-04-14T08:34:36.816194+0000","last_fullsized":"2024-04-14T08:34:36.816194+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:46:04.614148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994593+0000","last_change":"2024-04-14T08:34:34.413030+0000","last_active":"2024-04-14T08:34:36.994593+0000","last_peered":"2024-04-14T08:34:36.994593+0000","last_clean":"2024-04-14T08:34:36.994593+0000","last_became_active":"2024-04-14T08:34:34.412774+0000","last_became_peered":"2024-04-14T08:34:34.412774+0000","last_unstale":"2024-04-14T08:34:36.994593+0000","last_undegraded":"2024-04-14T08:34:36.994593+0000","last_fullsized":"2024-04-14T08:34:36.994593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:54:52.038741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994888+0000","last_change":"2024-04-14T08:34:35.435299+0000","last_active":"2024-04-14T08:34:36.994888+0000","last_peered":"2024-04-14T08:34:36.994888+0000","last_clean":"2024-04-14T08:34:36.994888+0000","last_became_active":"2024-04-14T08:34:35.433828+0000","last_became_peered":"2024-04-14T08:34:35.433828+0000","last_unstale":"2024-04-14T08:34:36.994888+0000","last_undegraded":"2024-04-14T08:34:36.994888+0000","last_fullsized":"2024-04-14T08:34:36.994888+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:57:15.656326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426547+0000","last_change":"2024-04-14T08:34:34.417238+0000","last_active":"2024-04-14T08:34:36.426547+0000","last_peered":"2024-04-14T08:34:36.426547+0000","last_clean":"2024-04-14T08:34:36.426547+0000","last_became_active":"2024-04-14T08:34:34.417002+0000","last_became_peered":"2024-04-14T08:34:34.417002+0000","last_unstale":"2024-04-14T08:34:36.426547+0000","last_undegraded":"2024-04-14T08:34:36.426547+0000","last_fullsized":"2024-04-14T08:34:36.426547+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:56:31.520428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436630+0000","last_change":"2024-04-14T08:34:35.428508+0000","last_active":"2024-04-14T08:34:36.436630+0000","last_peered":"2024-04-14T08:34:36.436630+0000","last_clean":"2024-04-14T08:34:36.436630+0000","last_became_active":"2024-04-14T08:34:35.428240+0000","last_became_peered":"2024-04-14T08:34:35.428240+0000","last_unstale":"2024-04-14T08:34:36.436630+0000","last_undegraded":"2024-04-14T08:34:36.436630+0000","last_fullsized":"2024-04-14T08:34:36.436630+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:31:14.753068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436284+0000","last_change":"2024-04-14T08:34:34.423744+0000","last_active":"2024-04-14T08:34:36.436284+0000","last_peered":"2024-04-14T08:34:36.436284+0000","last_clean":"2024-04-14T08:34:36.436284+0000","last_became_active":"2024-04-14T08:34:34.423394+0000","last_became_peered":"2024-04-14T08:34:34.423394+0000","last_unstale":"2024-04-14T08:34:36.436284+0000","last_undegraded":"2024-04-14T08:34:36.436284+0000","last_fullsized":"2024-04-14T08:34:36.436284+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:44:18.315252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994838+0000","last_change":"2024-04-14T08:34:35.435878+0000","last_active":"2024-04-14T08:34:36.994838+0000","last_peered":"2024-04-14T08:34:36.994838+0000","last_clean":"2024-04-14T08:34:36.994838+0000","last_became_active":"2024-04-14T08:34:35.435586+0000","last_became_peered":"2024-04-14T08:34:35.435586+0000","last_unstale":"2024-04-14T08:34:36.994838+0000","last_undegraded":"2024-04-14T08:34:36.994838+0000","last_fullsized":"2024-04-14T08:34:36.994838+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:40:13.833057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994609+0000","last_change":"2024-04-14T08:34:34.411096+0000","last_active":"2024-04-14T08:34:36.994609+0000","last_peered":"2024-04-14T08:34:36.994609+0000","last_clean":"2024-04-14T08:34:36.994609+0000","last_became_active":"2024-04-14T08:34:34.410841+0000","last_became_peered":"2024-04-14T08:34:34.410841+0000","last_unstale":"2024-04-14T08:34:36.994609+0000","last_undegraded":"2024-04-14T08:34:36.994609+0000","last_fullsized":"2024-04-14T08:34:36.994609+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:31:05.826819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.816260+0000","last_change":"2024-04-14T08:34:35.432182+0000","last_active":"2024-04-14T08:34:36.816260+0000","last_peered":"2024-04-14T08:34:36.816260+0000","last_clean":"2024-04-14T08:34:36.816260+0000","last_became_active":"2024-04-14T08:34:35.431892+0000","last_became_peered":"2024-04-14T08:34:35.431892+0000","last_unstale":"2024-04-14T08:34:36.816260+0000","last_undegraded":"2024-04-14T08:34:36.816260+0000","last_fullsized":"2024-04-14T08:34:36.816260+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:09:51.363444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994959+0000","last_change":"2024-04-14T08:34:34.424338+0000","last_active":"2024-04-14T08:34:36.994959+0000","last_peered":"2024-04-14T08:34:36.994959+0000","last_clean":"2024-04-14T08:34:36.994959+0000","last_became_active":"2024-04-14T08:34:34.424057+0000","last_became_peered":"2024-04-14T08:34:34.424057+0000","last_unstale":"2024-04-14T08:34:36.994959+0000","last_undegraded":"2024-04-14T08:34:36.994959+0000","last_fullsized":"2024-04-14T08:34:36.994959+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:30:35.547680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994457+0000","last_change":"2024-04-14T08:34:35.429962+0000","last_active":"2024-04-14T08:34:36.994457+0000","last_peered":"2024-04-14T08:34:36.994457+0000","last_clean":"2024-04-14T08:34:36.994457+0000","last_became_active":"2024-04-14T08:34:35.429687+0000","last_became_peered":"2024-04-14T08:34:35.429687+0000","last_unstale":"2024-04-14T08:34:36.994457+0000","last_undegraded":"2024-04-14T08:34:36.994457+0000","last_fullsized":"2024-04-14T08:34:36.994457+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:55:45.179250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.430267+0000","last_change":"2024-04-14T08:34:34.416682+0000","last_active":"2024-04-14T08:34:36.430267+0000","last_peered":"2024-04-14T08:34:36.430267+0000","last_clean":"2024-04-14T08:34:36.430267+0000","last_became_active":"2024-04-14T08:34:34.416172+0000","last_became_peered":"2024-04-14T08:34:34.416172+0000","last_unstale":"2024-04-14T08:34:36.430267+0000","last_undegraded":"2024-04-14T08:34:36.430267+0000","last_fullsized":"2024-04-14T08:34:36.430267+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:36.033184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426756+0000","last_change":"2024-04-14T08:34:35.437891+0000","last_active":"2024-04-14T08:34:36.426756+0000","last_peered":"2024-04-14T08:34:36.426756+0000","last_clean":"2024-04-14T08:34:36.426756+0000","last_became_active":"2024-04-14T08:34:35.437650+0000","last_became_peered":"2024-04-14T08:34:35.437650+0000","last_unstale":"2024-04-14T08:34:36.426756+0000","last_undegraded":"2024-04-14T08:34:36.426756+0000","last_fullsized":"2024-04-14T08:34:36.426756+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:57.073965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436468+0000","last_change":"2024-04-14T08:34:34.420330+0000","last_active":"2024-04-14T08:34:36.436468+0000","last_peered":"2024-04-14T08:34:36.436468+0000","last_clean":"2024-04-14T08:34:36.436468+0000","last_became_active":"2024-04-14T08:34:34.420116+0000","last_became_peered":"2024-04-14T08:34:34.420116+0000","last_unstale":"2024-04-14T08:34:36.436468+0000","last_undegraded":"2024-04-14T08:34:36.436468+0000","last_fullsized":"2024-04-14T08:34:36.436468+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:11.743901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436479+0000","last_change":"2024-04-14T08:34:35.438940+0000","last_active":"2024-04-14T08:34:36.436479+0000","last_peered":"2024-04-14T08:34:36.436479+0000","last_clean":"2024-04-14T08:34:36.436479+0000","last_became_active":"2024-04-14T08:34:35.438645+0000","last_became_peered":"2024-04-14T08:34:35.438645+0000","last_unstale":"2024-04-14T08:34:36.436479+0000","last_undegraded":"2024-04-14T08:34:36.436479+0000","last_fullsized":"2024-04-14T08:34:36.436479+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:57.073965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426694+0000","last_change":"2024-04-14T08:34:34.419528+0000","last_active":"2024-04-14T08:34:36.426694+0000","last_peered":"2024-04-14T08:34:36.426694+0000","last_clean":"2024-04-14T08:34:36.426694+0000","last_became_active":"2024-04-14T08:34:34.419258+0000","last_became_peered":"2024-04-14T08:34:34.419258+0000","last_unstale":"2024-04-14T08:34:36.426694+0000","last_undegraded":"2024-04-14T08:34:36.426694+0000","last_fullsized":"2024-04-14T08:34:36.426694+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:22:03.892869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.816170+0000","last_change":"2024-04-14T08:34:35.439976+0000","last_active":"2024-04-14T08:34:36.816170+0000","last_peered":"2024-04-14T08:34:36.816170+0000","last_clean":"2024-04-14T08:34:36.816170+0000","last_became_active":"2024-04-14T08:34:35.439473+0000","last_became_peered":"2024-04-14T08:34:35.439473+0000","last_unstale":"2024-04-14T08:34:36.816170+0000","last_undegraded":"2024-04-14T08:34:36.816170+0000","last_fullsized":"2024-04-14T08:34:36.816170+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:56.783669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.426643+0000","last_change":"2024-04-14T08:34:34.421662+0000","last_active":"2024-04-14T08:34:36.426643+0000","last_peered":"2024-04-14T08:34:36.426643+0000","last_clean":"2024-04-14T08:34:36.426643+0000","last_became_active":"2024-04-14T08:34:34.421382+0000","last_became_peered":"2024-04-14T08:34:34.421382+0000","last_unstale":"2024-04-14T08:34:36.426643+0000","last_undegraded":"2024-04-14T08:34:36.426643+0000","last_fullsized":"2024-04-14T08:34:36.426643+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:13:27.551411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436615+0000","last_change":"2024-04-14T08:34:35.427171+0000","last_active":"2024-04-14T08:34:36.436615+0000","last_peered":"2024-04-14T08:34:36.436615+0000","last_clean":"2024-04-14T08:34:36.436615+0000","last_became_active":"2024-04-14T08:34:35.426910+0000","last_became_peered":"2024-04-14T08:34:35.426910+0000","last_unstale":"2024-04-14T08:34:36.436615+0000","last_undegraded":"2024-04-14T08:34:36.436615+0000","last_fullsized":"2024-04-14T08:34:36.436615+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:03:40.712334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.429523+0000","last_change":"2024-04-14T08:34:34.419857+0000","last_active":"2024-04-14T08:34:36.429523+0000","last_peered":"2024-04-14T08:34:36.429523+0000","last_clean":"2024-04-14T08:34:36.429523+0000","last_became_active":"2024-04-14T08:34:34.419548+0000","last_became_peered":"2024-04-14T08:34:34.419548+0000","last_unstale":"2024-04-14T08:34:36.429523+0000","last_undegraded":"2024-04-14T08:34:36.429523+0000","last_fullsized":"2024-04-14T08:34:36.429523+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:41:19.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.427105+0000","last_change":"2024-04-14T08:34:35.431296+0000","last_active":"2024-04-14T08:34:36.427105+0000","last_peered":"2024-04-14T08:34:36.427105+0000","last_clean":"2024-04-14T08:34:36.427105+0000","last_became_active":"2024-04-14T08:34:35.431027+0000","last_became_peered":"2024-04-14T08:34:35.431027+0000","last_unstale":"2024-04-14T08:34:36.427105+0000","last_undegraded":"2024-04-14T08:34:36.427105+0000","last_fullsized":"2024-04-14T08:34:36.427105+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:04:54.692750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.816233+0000","last_change":"2024-04-14T08:34:34.420316+0000","last_active":"2024-04-14T08:34:36.816233+0000","last_peered":"2024-04-14T08:34:36.816233+0000","last_clean":"2024-04-14T08:34:36.816233+0000","last_became_active":"2024-04-14T08:34:34.420093+0000","last_became_peered":"2024-04-14T08:34:34.420093+0000","last_unstale":"2024-04-14T08:34:36.816233+0000","last_undegraded":"2024-04-14T08:34:36.816233+0000","last_fullsized":"2024-04-14T08:34:36.816233+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:11.743901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.994662+0000","last_change":"2024-04-14T08:34:35.436123+0000","last_active":"2024-04-14T08:34:36.994662+0000","last_peered":"2024-04-14T08:34:36.994662+0000","last_clean":"2024-04-14T08:34:36.994662+0000","last_became_active":"2024-04-14T08:34:35.435847+0000","last_became_peered":"2024-04-14T08:34:35.435847+0000","last_unstale":"2024-04-14T08:34:36.994662+0000","last_undegraded":"2024-04-14T08:34:36.994662+0000","last_fullsized":"2024-04-14T08:34:36.994662+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:34.399007+0000","last_clean_scrub_stamp":"2024-04-14T08:34:34.399007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:12.385117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"50'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.436770+0000","last_change":"2024-04-14T08:34:34.421158+0000","last_active":"2024-04-14T08:34:36.436770+0000","last_peered":"2024-04-14T08:34:36.436770+0000","last_clean":"2024-04-14T08:34:36.436770+0000","last_became_active":"2024-04-14T08:34:34.420780+0000","last_became_peered":"2024-04-14T08:34:34.420780+0000","last_unstale":"2024-04-14T08:34:36.436770+0000","last_undegraded":"2024-04-14T08:34:36.436770+0000","last_fullsized":"2024-04-14T08:34:36.436770+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:34:33.395238+0000","last_clean_scrub_stamp":"2024-04-14T08:34:33.395238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:36.033184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":228,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-14T08:34:36.429646+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:34:36.429646+0000","last_peered":"2024-04-14T08:34:36.429646+0000","last_clean":"2024-04-14T08:34:36.429646+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:34:36.429646+0000","last_undegraded":"2024-04-14T08:34:36.429646+0000","last_fullsized":"2024-04-14T08:34:36.429646+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1196032,"data_stored":852883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":602112,"data_stored":263043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":262391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":262973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":264025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":264003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":602112,"data_stored":262551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1196032,"data_stored":852815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:34:43.237 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-14T08:34:43.237 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-14T08:34:43.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.381+0000 7fd736e88640 1 Processor -- start 2024-04-14T08:34:43.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.381+0000 7fd736e88640 1 -- start start 2024-04-14T08:34:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.381+0000 7fd736e88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7300d0cc0 0x7fd7300d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd736e88640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd7300d49c0 0x7fd7300d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd736e88640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd730160f30 0x7fd7301613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd736e88640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd7300ceec0 con 0x7fd730160f30 2024-04-14T08:34:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd736e88640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fd7300cfd00 con 0x7fd7300d49c0 2024-04-14T08:34:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd736e88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd7300d15e0 con 0x7fd7300d0cc0 2024-04-14T08:34:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd735425640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd730160f30 0x7fd7301613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd735425640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd730160f30 0x7fd7301613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45750/0 (socket says 172.21.15.172:45750) 2024-04-14T08:34:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd735425640 1 -- 172.21.15.172:0/2808513379 learned_addr learned my addr 172.21.15.172:0/2808513379 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd734c24640 1 --2- 172.21.15.172:0/2808513379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7300d0cc0 0x7fd7300d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd72ffff640 1 --2- 172.21.15.172:0/2808513379 >> v2:172.21.15.177:3300/0 conn(0x7fd7300d49c0 0x7fd7300d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd735425640 1 -- 172.21.15.172:0/2808513379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7300d0cc0 msgr2=0x7fd7300d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd735425640 1 --2- 172.21.15.172:0/2808513379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7300d0cc0 0x7fd7300d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd735425640 1 -- 172.21.15.172:0/2808513379 >> v2:172.21.15.177:3300/0 conn(0x7fd7300d49c0 msgr2=0x7fd7300d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd735425640 1 --2- 172.21.15.172:0/2808513379 >> v2:172.21.15.177:3300/0 conn(0x7fd7300d49c0 0x7fd7300d4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.382+0000 7fd735425640 1 -- 172.21.15.172:0/2808513379 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7300cf0f0 con 0x7fd730160f30 2024-04-14T08:34:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.383+0000 7fd735425640 1 --2- 172.21.15.172:0/2808513379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd730160f30 0x7fd7301613c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd73009dcb0 tx=0x7fd72802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8b05d3d82e8fa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.383+0000 7fd72f7fe640 1 -- 172.21.15.172:0/2808513379 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72802ca70 con 0x7fd730160f30 2024-04-14T08:34:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.383+0000 7fd72f7fe640 1 -- 172.21.15.172:0/2808513379 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72802cc10 con 0x7fd730160f30 2024-04-14T08:34:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.383+0000 7fd72f7fe640 1 -- 172.21.15.172:0/2808513379 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7280343f0 con 0x7fd730160f30 2024-04-14T08:34:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.383+0000 7fd736e88640 1 -- 172.21.15.172:0/2808513379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd730160f30 msgr2=0x7fd7301613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.383+0000 7fd736e88640 1 --2- 172.21.15.172:0/2808513379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd730160f30 0x7fd7301613c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd73009dcb0 tx=0x7fd72802bed0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.383+0000 7fd736e88640 1 -- 172.21.15.172:0/2808513379 shutdown_connections 2024-04-14T08:34:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.383+0000 7fd736e88640 1 --2- 172.21.15.172:0/2808513379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd730160f30 0x7fd7301613c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.383+0000 7fd736e88640 1 --2- 172.21.15.172:0/2808513379 >> v2:172.21.15.177:3300/0 conn(0x7fd7300d49c0 0x7fd7300d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.383+0000 7fd736e88640 1 --2- 172.21.15.172:0/2808513379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7300d0cc0 0x7fd7300d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.383+0000 7fd736e88640 1 -- 172.21.15.172:0/2808513379 >> 172.21.15.172:0/2808513379 conn(0x7fd73009d580 msgr2=0x7fd7300d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:43.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.384+0000 7fd736e88640 1 -- 172.21.15.172:0/2808513379 shutdown_connections 2024-04-14T08:34:43.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.384+0000 7fd736e88640 1 -- 172.21.15.172:0/2808513379 wait complete. 2024-04-14T08:34:43.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.385+0000 7fd736e88640 1 Processor -- start 2024-04-14T08:34:43.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.385+0000 7fd736e88640 1 -- start start 2024-04-14T08:34:43.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd736e88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7300d0cc0 0x7fd73015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd736e88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7300d49c0 0x7fd7301f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd736e88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd730160f30 0x7fd73009fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd736e88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd7300cf990 con 0x7fd7300d49c0 2024-04-14T08:34:43.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd736e88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd7300cf6b0 con 0x7fd7300d0cc0 2024-04-14T08:34:43.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd736e88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd7300d32b0 con 0x7fd730160f30 2024-04-14T08:34:43.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd72ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7300d49c0 0x7fd7301f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd72ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7300d49c0 0x7fd7301f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45764/0 (socket says 172.21.15.172:45764) 2024-04-14T08:34:43.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd72ffff640 1 -- 172.21.15.172:0/3945516351 learned_addr learned my addr 172.21.15.172:0/3945516351 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:43.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd735425640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd730160f30 0x7fd73009fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd734c24640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7300d0cc0 0x7fd73015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd72ffff640 1 -- 172.21.15.172:0/3945516351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd730160f30 msgr2=0x7fd73009fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd72ffff640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd730160f30 0x7fd73009fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd72ffff640 1 -- 172.21.15.172:0/3945516351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7300d0cc0 msgr2=0x7fd73015ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd72ffff640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7300d0cc0 0x7fd73015ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd72ffff640 1 -- 172.21.15.172:0/3945516351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73009ebe0 con 0x7fd7300d49c0 2024-04-14T08:34:43.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd72ffff640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7300d49c0 0x7fd7301f5390 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd72400d830 tx=0x7fd72400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:43.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd72d7fa640 1 -- 172.21.15.172:0/3945516351 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72400b6c0 con 0x7fd7300d49c0 2024-04-14T08:34:43.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd72d7fa640 1 -- 172.21.15.172:0/3945516351 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd724019070 con 0x7fd7300d49c0 2024-04-14T08:34:43.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.387+0000 7fd736e88640 1 -- 172.21.15.172:0/3945516351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73009eec0 con 0x7fd7300d49c0 2024-04-14T08:34:43.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.388+0000 7fd72d7fa640 1 -- 172.21.15.172:0/3945516351 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72400ba80 con 0x7fd7300d49c0 2024-04-14T08:34:43.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.388+0000 7fd736e88640 1 -- 172.21.15.172:0/3945516351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73009f480 con 0x7fd7300d49c0 2024-04-14T08:34:43.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.389+0000 7fd736e88640 1 -- 172.21.15.172:0/3945516351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd730075490 con 0x7fd7300d49c0 2024-04-14T08:34:43.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.389+0000 7fd72d7fa640 1 -- 172.21.15.172:0/3945516351 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd72401e460 con 0x7fd7300d49c0 2024-04-14T08:34:43.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.390+0000 7fd72d7fa640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd71c06c8f0 0x7fd71c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.390+0000 7fd734c24640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd71c06c8f0 0x7fd71c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.390+0000 7fd72d7fa640 1 -- 172.21.15.172:0/3945516351 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fd72408c390 con 0x7fd7300d49c0 2024-04-14T08:34:43.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.390+0000 7fd734c24640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd71c06c8f0 0x7fd71c06edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd720006fd0 tx=0x7fd720008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:43.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.394+0000 7fd72d7fa640 1 -- 172.21.15.172:0/3945516351 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd72405a6c0 con 0x7fd7300d49c0 2024-04-14T08:34:43.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.833+0000 7fd736e88640 1 -- 172.21.15.172:0/3945516351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd7300cc5a0 con 0x7fd7300d49c0 2024-04-14T08:34:43.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.835+0000 7fd72d7fa640 1 -- 172.21.15.172:0/3945516351 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fd724059fc0 con 0x7fd7300d49c0 2024-04-14T08:34:43.837 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2024-04-14T08:34:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA== 2024-04-14T08:34:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: caps mds = "allow" 2024-04-14T08:34:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: caps mon = "allow r" 2024-04-14T08:34:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-14T08:34:43.838 INFO:teuthology.orchestra.run.smithi172.stderr:updated caps for client.0 2024-04-14T08:34:43.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.838+0000 7fd736e88640 1 -- 172.21.15.172:0/3945516351 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd71c06c8f0 msgr2=0x7fd71c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.838+0000 7fd736e88640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd71c06c8f0 0x7fd71c06edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd720006fd0 tx=0x7fd720008040 comp rx=0 tx=0).stop 2024-04-14T08:34:43.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.838+0000 7fd736e88640 1 -- 172.21.15.172:0/3945516351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7300d49c0 msgr2=0x7fd7301f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.838+0000 7fd736e88640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7300d49c0 0x7fd7301f5390 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd72400d830 tx=0x7fd72400dcf0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.838+0000 7fd736e88640 1 -- 172.21.15.172:0/3945516351 shutdown_connections 2024-04-14T08:34:43.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.838+0000 7fd736e88640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd71c06c8f0 0x7fd71c06edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.838+0000 7fd736e88640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd730160f30 0x7fd73009fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.838+0000 7fd736e88640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7300d49c0 0x7fd7301f5390 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.838+0000 7fd736e88640 1 --2- 172.21.15.172:0/3945516351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7300d0cc0 0x7fd73015ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.838+0000 7fd736e88640 1 -- 172.21.15.172:0/3945516351 >> 172.21.15.172:0/3945516351 conn(0x7fd73009d580 msgr2=0x7fd730160790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:43.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.838+0000 7fd736e88640 1 -- 172.21.15.172:0/3945516351 shutdown_connections 2024-04-14T08:34:43.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.838+0000 7fd736e88640 1 -- 172.21.15.172:0/3945516351 wait complete. 2024-04-14T08:34:43.854 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:34:43.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.955+0000 7ff641476640 1 Processor -- start 2024-04-14T08:34:43.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.955+0000 7ff641476640 1 -- start start 2024-04-14T08:34:43.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.956+0000 7ff641476640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff63c15c150 0x7ff63c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.956+0000 7ff641476640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63c0d1840 0x7ff63c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.956+0000 7ff641476640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff63c162940 con 0x7ff63c162530 2024-04-14T08:34:43.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.956+0000 7ff641476640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff63c0cef10 con 0x7ff63c15c150 2024-04-14T08:34:43.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.956+0000 7ff641476640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff63c0d1c80 con 0x7ff63c0d1840 2024-04-14T08:34:43.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.956+0000 7ff63affd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff63c15c150 0x7ff63c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.956+0000 7ff63a7fc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63c0d1840 0x7ff63c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.956+0000 7ff63affd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff63c15c150 0x7ff63c0d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:60448/0 (socket says 172.21.15.172:60448) 2024-04-14T08:34:43.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.956+0000 7ff63affd640 1 -- 172.21.15.172:0/1458529007 learned_addr learned my addr 172.21.15.172:0/1458529007 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:43.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.956+0000 7ff63b7fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7ff63c162530 0x7ff63c15ea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:40520/0 (socket says 172.21.15.172:40520) 2024-04-14T08:34:43.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.957+0000 7ff63affd640 1 -- 172.21.15.172:0/1458529007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63c0d1840 msgr2=0x7ff63c161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:43.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.957+0000 7ff63affd640 1 --2- 172.21.15.172:0/1458529007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63c0d1840 0x7ff63c161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.957+0000 7ff63affd640 1 -- 172.21.15.172:0/1458529007 >> v1:172.21.15.177:6789/0 conn(0x7ff63c162530 legacy=0x7ff63c15ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.957+0000 7ff63affd640 1 -- 172.21.15.172:0/1458529007 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c0d0200 con 0x7ff63c15c150 2024-04-14T08:34:43.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.957+0000 7ff63a7fc640 1 --2- 172.21.15.172:0/1458529007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63c0d1840 0x7ff63c161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:34:43.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.957+0000 7ff63affd640 1 --2- 172.21.15.172:0/1458529007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff63c15c150 0x7ff63c0d1300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff630009870 tx=0x7ff63002b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6fb392172de77a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.957+0000 7ff639ffb640 1 -- 172.21.15.172:0/1458529007 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630037070 con 0x7ff63c15c150 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.958+0000 7ff639ffb640 1 -- 172.21.15.172:0/1458529007 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff630004230 con 0x7ff63c15c150 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.958+0000 7ff639ffb640 1 -- 172.21.15.172:0/1458529007 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630003610 con 0x7ff63c15c150 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.958+0000 7ff641476640 1 -- 172.21.15.172:0/1458529007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff63c15c150 msgr2=0x7ff63c0d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.958+0000 7ff641476640 1 --2- 172.21.15.172:0/1458529007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff63c15c150 0x7ff63c0d1300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff630009870 tx=0x7ff63002b7e0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.958+0000 7ff641476640 1 -- 172.21.15.172:0/1458529007 shutdown_connections 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.958+0000 7ff641476640 1 --2- 172.21.15.172:0/1458529007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63c0d1840 0x7ff63c161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.958+0000 7ff641476640 1 --2- 172.21.15.172:0/1458529007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff63c15c150 0x7ff63c0d1300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.958+0000 7ff641476640 1 -- 172.21.15.172:0/1458529007 >> 172.21.15.172:0/1458529007 conn(0x7ff63c09d580 msgr2=0x7ff63c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.958+0000 7ff641476640 1 -- 172.21.15.172:0/1458529007 shutdown_connections 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.958+0000 7ff641476640 1 -- 172.21.15.172:0/1458529007 wait complete. 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.959+0000 7ff641476640 1 Processor -- start 2024-04-14T08:34:43.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.959+0000 7ff641476640 1 -- start start 2024-04-14T08:34:43.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.960+0000 7ff641476640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63c0d1840 0x7ff63c0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.960+0000 7ff641476640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63c15c150 0x7ff63c1f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.960+0000 7ff641476640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff63c162530 0x7ff63c1f8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.960+0000 7ff641476640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c0cf720 con 0x7ff63c15c150 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.960+0000 7ff641476640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c0cf440 con 0x7ff63c162530 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.960+0000 7ff641476640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff63c0cfa00 con 0x7ff63c0d1840 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.960+0000 7ff63a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63c15c150 0x7ff63c1f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.960+0000 7ff63a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63c15c150 0x7ff63c1f8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45774/0 (socket says 172.21.15.172:45774) 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.960+0000 7ff63a7fc640 1 -- 172.21.15.172:0/4194848026 learned_addr learned my addr 172.21.15.172:0/4194848026 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.960+0000 7ff63b7fe640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff63c162530 0x7ff63c1f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.960+0000 7ff63affd640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63c0d1840 0x7ff63c0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.961+0000 7ff63a7fc640 1 -- 172.21.15.172:0/4194848026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63c0d1840 msgr2=0x7ff63c0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.961+0000 7ff63a7fc640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63c0d1840 0x7ff63c0cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.961+0000 7ff63a7fc640 1 -- 172.21.15.172:0/4194848026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff63c162530 msgr2=0x7ff63c1f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.961+0000 7ff63a7fc640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff63c162530 0x7ff63c1f8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:43.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.961+0000 7ff63a7fc640 1 -- 172.21.15.172:0/4194848026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c0cc6c0 con 0x7ff63c15c150 2024-04-14T08:34:43.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.961+0000 7ff63a7fc640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63c15c150 0x7ff63c1f8340 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00ec10 tx=0x7ff62c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:43.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.961+0000 7ff6237fe640 1 -- 172.21.15.172:0/4194848026 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c017070 con 0x7ff63c15c150 2024-04-14T08:34:43.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.961+0000 7ff6237fe640 1 -- 172.21.15.172:0/4194848026 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62c00cc30 con 0x7ff63c15c150 2024-04-14T08:34:43.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.961+0000 7ff6237fe640 1 -- 172.21.15.172:0/4194848026 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c013470 con 0x7ff63c15c150 2024-04-14T08:34:43.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.961+0000 7ff641476640 1 -- 172.21.15.172:0/4194848026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63c0ccdf0 con 0x7ff63c15c150 2024-04-14T08:34:43.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.961+0000 7ff641476640 1 -- 172.21.15.172:0/4194848026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63c0cd3b0 con 0x7ff63c15c150 2024-04-14T08:34:43.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.962+0000 7ff641476640 1 -- 172.21.15.172:0/4194848026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63c075490 con 0x7ff63c15c150 2024-04-14T08:34:43.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.962+0000 7ff6237fe640 1 -- 172.21.15.172:0/4194848026 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff62c01a070 con 0x7ff63c15c150 2024-04-14T08:34:43.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.963+0000 7ff6237fe640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff62806c8f0 0x7ff62806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:43.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.963+0000 7ff63affd640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff62806c8f0 0x7ff62806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:43.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.963+0000 7ff6237fe640 1 -- 172.21.15.172:0/4194848026 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7ff62c08bc60 con 0x7ff63c15c150 2024-04-14T08:34:43.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.963+0000 7ff63affd640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff62806c8f0 0x7ff62806edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff630034000 tx=0x7ff630008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:43.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:43.967+0000 7ff6237fe640 1 -- 172.21.15.172:0/4194848026 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff63c0cc6c0 con 0x7ff63c15c150 2024-04-14T08:34:44.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.190+0000 7ff641476640 1 -- 172.21.15.172:0/4194848026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff63c0ce670 con 0x7ff63c15c150 2024-04-14T08:34:44.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.190+0000 7ff6237fe640 1 -- 172.21.15.172:0/4194848026 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5187 (secure 0 0 0) 0x7ff62c059f00 con 0x7ff63c15c150 2024-04-14T08:34:44.192 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:44.192 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/296861286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":296861286},{"type":"v1","addr":"172.21.15.177:6835","nonce":296861286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/642342704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":642342704},{"type":"v1","addr":"172.21.15.177:6833","nonce":642342704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3516903987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3516903987},{"type":"v1","addr":"172.21.15.172:6836","nonce":3516903987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:34:34.765057+0000","modified":"2024-04-14T08:34:36.783999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6837/1527574817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1527574817},{"type":"v1","addr":"172.21.15.172:6837","nonce":1527574817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":3}]} 2024-04-14T08:34:44.192 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 23 2024-04-14T08:34:44.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.192+0000 7ff641476640 1 -- 172.21.15.172:0/4194848026 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff62806c8f0 msgr2=0x7ff62806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.192+0000 7ff641476640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff62806c8f0 0x7ff62806edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff630034000 tx=0x7ff630008030 comp rx=0 tx=0).stop 2024-04-14T08:34:44.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.192+0000 7ff641476640 1 -- 172.21.15.172:0/4194848026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63c15c150 msgr2=0x7ff63c1f8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.192+0000 7ff641476640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63c15c150 0x7ff63c1f8340 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00ec10 tx=0x7ff62c00c520 comp rx=0 tx=0).stop 2024-04-14T08:34:44.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.192+0000 7ff641476640 1 -- 172.21.15.172:0/4194848026 shutdown_connections 2024-04-14T08:34:44.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.192+0000 7ff641476640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff62806c8f0 0x7ff62806edb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.192+0000 7ff641476640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff63c162530 0x7ff63c1f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.192+0000 7ff641476640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63c15c150 0x7ff63c1f8340 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.192+0000 7ff641476640 1 --2- 172.21.15.172:0/4194848026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63c0d1840 0x7ff63c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.192+0000 7ff641476640 1 -- 172.21.15.172:0/4194848026 >> 172.21.15.172:0/4194848026 conn(0x7ff63c09d580 msgr2=0x7ff63c06ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:44.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.192+0000 7ff641476640 1 -- 172.21.15.172:0/4194848026 shutdown_connections 2024-04-14T08:34:44.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.192+0000 7ff641476640 1 -- 172.21.15.172:0/4194848026 wait complete. 2024-04-14T08:34:44.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:34:34.765057+0000', 'modified': '2024-04-14T08:34:36.783999+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5745}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5745': {'gid': 5745, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6837/1527574817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 1527574817}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 1527574817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5745, 'qdb_cluster': [5745]} 2024-04-14T08:34:44.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:34:44.608 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-14T08:34:44.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.709+0000 7f8f689c8640 1 Processor -- start 2024-04-14T08:34:44.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.709+0000 7f8f689c8640 1 -- start start 2024-04-14T08:34:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f689c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8f6415e670 0x7f8f640b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f689c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8f64072ac0 0x7f8f6406e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f689c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f640cef70 con 0x7f8f640cead0 2024-04-14T08:34:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f689c8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f8f640cd780 con 0x7f8f6415e670 2024-04-14T08:34:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f689c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f8f640cc9c0 con 0x7f8f64072ac0 2024-04-14T08:34:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f6259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8f6415e670 0x7f8f640b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f6259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8f6415e670 0x7f8f640b0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45780/0 (socket says 172.21.15.172:45780) 2024-04-14T08:34:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f6259c640 1 -- 172.21.15.172:0/1083412895 learned_addr learned my addr 172.21.15.172:0/1083412895 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f62d9d640 1 --2- 172.21.15.172:0/1083412895 >> v2:172.21.15.177:3300/0 conn(0x7f8f64072ac0 0x7f8f6406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f6259c640 1 -- 172.21.15.172:0/1083412895 >> v2:172.21.15.177:3300/0 conn(0x7f8f64072ac0 msgr2=0x7f8f6406e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:44.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f6259c640 1 --2- 172.21.15.172:0/1083412895 >> v2:172.21.15.177:3300/0 conn(0x7f8f64072ac0 0x7f8f6406e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f6259c640 1 -- 172.21.15.172:0/1083412895 >> v1:172.21.15.177:6789/0 conn(0x7f8f640cead0 legacy=0x7f8f640723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:44.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.710+0000 7f8f6259c640 1 -- 172.21.15.172:0/1083412895 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f640ccca0 con 0x7f8f6415e670 2024-04-14T08:34:44.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.711+0000 7f8f6259c640 1 --2- 172.21.15.172:0/1083412895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f6415e670 0x7f8f640b0440 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8f640d0e60 tx=0x7f8f5402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9218fcc308158a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:44.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.711+0000 7f8f61d9b640 1 -- 172.21.15.172:0/1083412895 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f540317c0 con 0x7f8f6415e670 2024-04-14T08:34:44.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.711+0000 7f8f61d9b640 1 -- 172.21.15.172:0/1083412895 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f54037070 con 0x7f8f6415e670 2024-04-14T08:34:44.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.711+0000 7f8f61d9b640 1 -- 172.21.15.172:0/1083412895 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54031b80 con 0x7f8f6415e670 2024-04-14T08:34:44.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.711+0000 7f8f689c8640 1 -- 172.21.15.172:0/1083412895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f6415e670 msgr2=0x7f8f640b0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.711+0000 7f8f689c8640 1 --2- 172.21.15.172:0/1083412895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f6415e670 0x7f8f640b0440 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8f640d0e60 tx=0x7f8f5402b860 comp rx=0 tx=0).stop 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.711+0000 7f8f689c8640 1 -- 172.21.15.172:0/1083412895 shutdown_connections 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.711+0000 7f8f689c8640 1 --2- 172.21.15.172:0/1083412895 >> v2:172.21.15.177:3300/0 conn(0x7f8f64072ac0 0x7f8f6406e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.711+0000 7f8f689c8640 1 --2- 172.21.15.172:0/1083412895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f6415e670 0x7f8f640b0440 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.711+0000 7f8f689c8640 1 -- 172.21.15.172:0/1083412895 >> 172.21.15.172:0/1083412895 conn(0x7f8f640cd980 msgr2=0x7f8f640cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.712+0000 7f8f689c8640 1 -- 172.21.15.172:0/1083412895 shutdown_connections 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.712+0000 7f8f689c8640 1 -- 172.21.15.172:0/1083412895 wait complete. 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.713+0000 7f8f689c8640 1 Processor -- start 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.713+0000 7f8f689c8640 1 -- start start 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f689c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f64072ac0 0x7f8f640c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f689c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f640cead0 0x7f8f6414e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f689c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f6415e670 0x7f8f640ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f689c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8f640b00f0 con 0x7f8f640cead0 2024-04-14T08:34:44.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f689c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8f640cd260 con 0x7f8f6415e670 2024-04-14T08:34:44.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f689c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6415ea50 con 0x7f8f64072ac0 2024-04-14T08:34:44.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f59d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f640cead0 0x7f8f6414e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f59d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f640cead0 0x7f8f6414e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45796/0 (socket says 172.21.15.172:45796) 2024-04-14T08:34:44.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f59d9b640 1 -- 172.21.15.172:0/87349941 learned_addr learned my addr 172.21.15.172:0/87349941 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:44.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f62d9d640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f6415e670 0x7f8f640ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f6259c640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f64072ac0 0x7f8f640c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f59d9b640 1 -- 172.21.15.172:0/87349941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f64072ac0 msgr2=0x7f8f640c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f59d9b640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f64072ac0 0x7f8f640c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f59d9b640 1 -- 172.21.15.172:0/87349941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f6415e670 msgr2=0x7f8f640ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f59d9b640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f6415e670 0x7f8f640ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f59d9b640 1 -- 172.21.15.172:0/87349941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6414ea40 con 0x7f8f640cead0 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f62d9d640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f6415e670 0x7f8f640ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.714+0000 7f8f59d9b640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f640cead0 0x7f8f6414e500 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00b3f0 tx=0x7f8f4c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.715+0000 7f8f5b7fe640 1 -- 172.21.15.172:0/87349941 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c015020 con 0x7f8f640cead0 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.715+0000 7f8f689c8640 1 -- 172.21.15.172:0/87349941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f640c5890 con 0x7f8f640cead0 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.715+0000 7f8f5b7fe640 1 -- 172.21.15.172:0/87349941 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f4c011830 con 0x7f8f640cead0 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.715+0000 7f8f5b7fe640 1 -- 172.21.15.172:0/87349941 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c011c20 con 0x7f8f640cead0 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.715+0000 7f8f689c8640 1 -- 172.21.15.172:0/87349941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f640c5e50 con 0x7f8f640cead0 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.716+0000 7f8f5b7fe640 1 -- 172.21.15.172:0/87349941 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f4c018020 con 0x7f8f640cead0 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.717+0000 7f8f5b7fe640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8f4406c920 0x7f8f4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.717+0000 7f8f6259c640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8f4406c920 0x7f8f4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.717+0000 7f8f5b7fe640 1 -- 172.21.15.172:0/87349941 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f8f4c08b8d0 con 0x7f8f640cead0 2024-04-14T08:34:44.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.717+0000 7f8f6259c640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8f4406c920 0x7f8f4406ede0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f54036000 tx=0x7f8f5403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:44.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.718+0000 7f8f6b7b3740 1 Processor -- start 2024-04-14T08:34:44.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.718+0000 7f8f6b7b3740 1 -- start start 2024-04-14T08:34:44.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.718+0000 7f8f6b7b3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 0x55b52583cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.718+0000 7f8f6b7b3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b525869f40 0x55b52586c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.718+0000 7f8f6b7b3740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b52583d040 0x55b52586ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.718+0000 7f8f6b7b3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b5257a6030 con 0x55b525869b60 2024-04-14T08:34:44.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.718+0000 7f8f6b7b3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b52574e620 con 0x55b525869f40 2024-04-14T08:34:44.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.718+0000 7f8f6b7b3740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55b5257a1fe0 con 0x55b52583d040 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 0x55b52583cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 0x55b52583cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45808/0 (socket says 172.21.15.172:45808) 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f6259c640 1 -- 172.21.15.172:0/3757736083 learned_addr learned my addr 172.21.15.172:0/3757736083 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f59d9b640 1 --2- 172.21.15.172:0/3757736083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b525869f40 0x55b52586c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f62d9d640 1 --2- 172.21.15.172:0/3757736083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b52583d040 0x55b52586ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f6259c640 1 -- 172.21.15.172:0/3757736083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b52583d040 msgr2=0x55b52586ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f6259c640 1 --2- 172.21.15.172:0/3757736083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b52583d040 0x55b52586ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f6259c640 1 -- 172.21.15.172:0/3757736083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b525869f40 msgr2=0x55b52586c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f6259c640 1 --2- 172.21.15.172:0/3757736083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b525869f40 0x55b52586c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f6259c640 1 -- 172.21.15.172:0/3757736083 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5254ba2d0 con 0x55b525869b60 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f6259c640 1 --2- 172.21.15.172:0/3757736083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 0x55b52583cb00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f54030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eee50a8c42ae1c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f43fff640 1 -- 172.21.15.172:0/3757736083 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54032a60 con 0x55b525869b60 2024-04-14T08:34:44.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f43fff640 1 -- 172.21.15.172:0/3757736083 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f54037040 con 0x55b525869b60 2024-04-14T08:34:44.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.719+0000 7f8f43fff640 1 -- 172.21.15.172:0/3757736083 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54032c20 con 0x55b525869b60 2024-04-14T08:34:44.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.720+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/3757736083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 msgr2=0x55b52583cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.720+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/3757736083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 0x55b52583cb00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f54030000 comp rx=0 tx=0).stop 2024-04-14T08:34:44.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.720+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/3757736083 shutdown_connections 2024-04-14T08:34:44.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.720+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/3757736083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b52583d040 0x55b52586ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.720+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/3757736083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b525869f40 0x55b52586c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.720+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/3757736083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 0x55b52583cb00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.720+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/3757736083 >> 172.21.15.172:0/3757736083 conn(0x55b52583ed70 msgr2=0x55b5258878f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:44.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.720+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/3757736083 shutdown_connections 2024-04-14T08:34:44.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.720+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/3757736083 wait complete. 2024-04-14T08:34:44.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.720+0000 7f8f6b7b3740 1 -- start start 2024-04-14T08:34:44.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.721+0000 7f8f6b7b3740 1 Processor -- start 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.721+0000 7f8f6b7b3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 0x55b525856460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.721+0000 7f8f6b7b3740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b525869f40 0x55b525887d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.721+0000 7f8f6b7b3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5258569a0 0x55b525888240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.721+0000 7f8f6b7b3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b525841410 con 0x55b525869b60 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.721+0000 7f8f6b7b3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b5257619e0 con 0x55b5258569a0 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.721+0000 7f8f6b7b3740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55b525843c50 con 0x55b525869f40 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.721+0000 7f8f6b7b3740 10 client.0 _ll_register_callbacks cb 0x55b52579e080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 0x55b525856460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f6259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 0x55b525856460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45824/0 (socket says 172.21.15.172:45824) 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f6259c640 1 -- 172.21.15.172:0/2974344941 learned_addr learned my addr 172.21.15.172:0/2974344941 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f59d9b640 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b525869f40 0x55b525887d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f62d9d640 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5258569a0 0x55b525888240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f6259c640 1 -- 172.21.15.172:0/2974344941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b525869f40 msgr2=0x55b525887d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:44.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f6259c640 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b525869f40 0x55b525887d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f6259c640 1 -- 172.21.15.172:0/2974344941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5258569a0 msgr2=0x55b525888240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:44.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f6259c640 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5258569a0 0x55b525888240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f6259c640 1 -- 172.21.15.172:0/2974344941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5255846b0 con 0x55b525869b60 2024-04-14T08:34:44.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f59d9b640 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b525869f40 0x55b525887d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:44.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f62d9d640 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5258569a0 0x55b525888240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:44.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f6259c640 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 0x55b525856460 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009930 tx=0x7f8f54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762791eae71d305f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:44.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f417fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:34:44.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f417fa640 1 -- 172.21.15.172:0/2974344941 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5405d060 con 0x55b525869b60 2024-04-14T08:34:44.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f417fa640 1 -- 172.21.15.172:0/2974344941 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f8f5404b070 con 0x55b525869b60 2024-04-14T08:34:44.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2974344941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f54037040 con 0x55b525869b60 2024-04-14T08:34:44.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f417fa640 1 -- 172.21.15.172:0/2974344941 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8f54063050 con 0x55b525869b60 2024-04-14T08:34:44.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f6b7b3740 10 client.5901 fetch_fsmap learned FSMap version 23 2024-04-14T08:34:44.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.722+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2974344941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55b525859fa0 con 0x55b525869b60 2024-04-14T08:34:44.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.723+0000 7f8f417fa640 1 -- 172.21.15.172:0/2974344941 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f54037040 con 0x55b525869b60 2024-04-14T08:34:44.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.723+0000 7f8f417fa640 1 -- 172.21.15.172:0/2974344941 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f8f54066020 con 0x55b525869b60 2024-04-14T08:34:44.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.723+0000 7f8f6b7b3740 10 client.5901 fetch_fsmap finished waiting for FSMap version 23 2024-04-14T08:34:44.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.723+0000 7f8f6b7b3740 10 client.5901 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:34:44.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.723+0000 7f8f6b7b3740 20 client.5901 populate_metadata read hostname 'smithi172' 2024-04-14T08:34:44.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.723+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55b52586eec0 0x55b52586f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.723+0000 7f8f6b7b3740 4 client.5901 mds_command: new command op to 5745 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:34:44.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.723+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2974344941 --> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b52574dcc0 con 0x55b52586eec0 2024-04-14T08:34:44.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.723+0000 7f8f59d9b640 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55b52586eec0 0x55b52586f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.723+0000 7f8f417fa640 10 client.5901 ms_handle_connect on v2:172.21.15.172:6835/1527574817 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.723+0000 7f8f59d9b640 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55b52586eec0 0x55b52586f2a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc9e727fba67b9bb server_cookie=1fd804dc3eee9f8e in_seq=0 out_seq=0 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f417fa640 1 -- 172.21.15.172:0/2974344941 <== mds.0 v2:172.21.15.172:6835/1527574817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b52574dcc0 con 0x55b52586eec0 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f417fa640 10 client.5901 handle_command_reply: tid=0 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f6b7b3740 1 client.5901 shutdown 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2974344941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 msgr2=0x55b525856460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 0x55b525856460 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009930 tx=0x7f8f54004730 comp rx=0 tx=0).stop 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2974344941 shutdown_connections 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55b52586eec0 0x55b52586f2a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5258569a0 0x55b525888240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b525869f40 0x55b525887d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/2974344941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869b60 0x55b525856460 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f417fa640 0 client.5901 ms_handle_reset on v2:172.21.15.172:6835/1527574817 2024-04-14T08:34:44.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2974344941 >> 172.21.15.172:0/2974344941 conn(0x55b52583d040 msgr2=0x55b525840a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:44.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2974344941 shutdown_connections 2024-04-14T08:34:44.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2974344941 wait complete. 2024-04-14T08:34:44.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.724+0000 7f8f6b7b3740 20 client.5901 trim_cache size 0 max 16384 2024-04-14T08:34:44.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.742+0000 7f8f6b7b3740 1 Processor -- start 2024-04-14T08:34:44.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.742+0000 7f8f6b7b3740 1 -- start start 2024-04-14T08:34:44.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f6b7b3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5258569a0 0x55b52588aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f6b7b3740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b525869b60 0x55b5258681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f6b7b3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 0x55b525868720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f6b7b3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641fdee0 con 0x55b525869f40 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f6b7b3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8f640b5e70 con 0x55b5258569a0 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f6b7b3740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8f641fd890 con 0x55b525869b60 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f62d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 0x55b525868720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f62d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 0x55b525868720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45836/0 (socket says 172.21.15.172:45836) 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f62d9d640 1 -- 172.21.15.172:0/4048001498 learned_addr learned my addr 172.21.15.172:0/4048001498 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f59d9b640 1 --2- 172.21.15.172:0/4048001498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b525869b60 0x55b5258681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f6259c640 1 --2- 172.21.15.172:0/4048001498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5258569a0 0x55b52588aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f62d9d640 1 -- 172.21.15.172:0/4048001498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b525869b60 msgr2=0x55b5258681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f62d9d640 1 --2- 172.21.15.172:0/4048001498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b525869b60 0x55b5258681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f62d9d640 1 -- 172.21.15.172:0/4048001498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5258569a0 msgr2=0x55b52588aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f62d9d640 1 --2- 172.21.15.172:0/4048001498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5258569a0 0x55b52588aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f62d9d640 1 -- 172.21.15.172:0/4048001498 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5258706c0 con 0x55b525869f40 2024-04-14T08:34:44.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f6259c640 1 --2- 172.21.15.172:0/4048001498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5258569a0 0x55b52588aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.743+0000 7f8f59d9b640 1 --2- 172.21.15.172:0/4048001498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b525869b60 0x55b5258681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f62d9d640 1 --2- 172.21.15.172:0/4048001498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 0x55b525868720 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009ed0 tx=0x7f8f5c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fdc59b2f7a15f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f437fe640 1 -- 172.21.15.172:0/4048001498 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c01a070 con 0x55b525869f40 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f437fe640 1 -- 172.21.15.172:0/4048001498 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5c016850 con 0x55b525869f40 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f437fe640 1 -- 172.21.15.172:0/4048001498 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c016c40 con 0x55b525869f40 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/4048001498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 msgr2=0x55b525868720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/4048001498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 0x55b525868720 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009ed0 tx=0x7f8f5c007700 comp rx=0 tx=0).stop 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/4048001498 shutdown_connections 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/4048001498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 0x55b525868720 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/4048001498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b525869b60 0x55b5258681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/4048001498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5258569a0 0x55b52588aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/4048001498 >> 172.21.15.172:0/4048001498 conn(0x55b52583d040 msgr2=0x55b525837e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/4048001498 shutdown_connections 2024-04-14T08:34:44.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.744+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/4048001498 wait complete. 2024-04-14T08:34:44.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.745+0000 7f8f6b7b3740 1 -- start start 2024-04-14T08:34:44.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.745+0000 7f8f6b7b3740 1 Processor -- start 2024-04-14T08:34:44.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f6b7b3740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5258569a0 0x55b52586c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f6b7b3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b525869b60 0x55b525903bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f6b7b3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 0x55b525904100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f6b7b3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b525841410 con 0x55b525869f40 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f6b7b3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b5257619e0 con 0x55b525869b60 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f6b7b3740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55b5258573a0 con 0x55b5258569a0 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f6b7b3740 10 client.0 _ll_register_callbacks cb 0x7f8f54031a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f62d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 0x55b525904100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f62d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 0x55b525904100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45838/0 (socket says 172.21.15.172:45838) 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f62d9d640 1 -- 172.21.15.172:0/2308551979 learned_addr learned my addr 172.21.15.172:0/2308551979 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f6259c640 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5258569a0 0x55b52586c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f59d9b640 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b525869b60 0x55b525903bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f62d9d640 1 -- 172.21.15.172:0/2308551979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5258569a0 msgr2=0x55b52586c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f62d9d640 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5258569a0 0x55b52586c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f62d9d640 1 -- 172.21.15.172:0/2308551979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b525869b60 msgr2=0x55b525903bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f62d9d640 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b525869b60 0x55b525903bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f62d9d640 1 -- 172.21.15.172:0/2308551979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5258ac2b0 con 0x55b525869f40 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f59d9b640 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b525869b60 0x55b525903bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f62d9d640 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 0x55b525904100 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c007c00 tx=0x7f8f5c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c205be95064c527 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f417fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.746+0000 7f8f417fa640 1 -- 172.21.15.172:0/2308551979 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c01a070 con 0x55b525869f40 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2308551979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f5c016850 con 0x55b525869f40 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f417fa640 1 -- 172.21.15.172:0/2308551979 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f8f5c02ac80 con 0x55b525869f40 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f417fa640 1 -- 172.21.15.172:0/2308551979 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8f5c031050 con 0x55b525869f40 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f6b7b3740 10 client.5907 fetch_fsmap learned FSMap version 23 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2308551979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55b5258883f0 con 0x55b525869f40 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f417fa640 1 -- 172.21.15.172:0/2308551979 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5c016850 con 0x55b525869f40 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f417fa640 1 -- 172.21.15.172:0/2308551979 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f8f5c034020 con 0x55b525869f40 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f6b7b3740 10 client.5907 fetch_fsmap finished waiting for FSMap version 23 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f6b7b3740 10 client.5907 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:34:44.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f6b7b3740 20 client.5907 populate_metadata read hostname 'smithi172' 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55b5259052d0 0x55b525841b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f6b7b3740 4 client.5907 mds_command: new command op to 5745 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2308551979 --> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] -- command(tid 0: {"prefix": "status"}) -- 0x55b5257b3570 con 0x55b5259052d0 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f6259c640 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55b5259052d0 0x55b525841b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f417fa640 10 client.5907 ms_handle_connect on v2:172.21.15.172:6835/1527574817 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.747+0000 7f8f6259c640 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55b5259052d0 0x55b525841b00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=409e3c44a695c22e server_cookie=536b50cf8760e6a5 in_seq=0 out_seq=0 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f417fa640 1 -- 172.21.15.172:0/2308551979 <== mds.0 v2:172.21.15.172:6835/1527574817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b5257b3570 con 0x55b5259052d0 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f417fa640 10 client.5907 handle_command_reply: tid=0 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f6b7b3740 1 client.5907 shutdown 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2308551979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 msgr2=0x55b525904100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 0x55b525904100 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c007c00 tx=0x7f8f5c004ab0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2308551979 shutdown_connections 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55b5259052d0 0x55b525841b00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b525869f40 0x55b525904100 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b525869b60 0x55b525903bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f6b7b3740 1 --2- 172.21.15.172:0/2308551979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b5258569a0 0x55b52586c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2308551979 >> 172.21.15.172:0/2308551979 conn(0x55b52583d040 msgr2=0x55b525887ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f417fa640 0 client.5907 ms_handle_reset on v2:172.21.15.172:6835/1527574817 2024-04-14T08:34:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2308551979 shutdown_connections 2024-04-14T08:34:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f6b7b3740 1 -- 172.21.15.172:0/2308551979 wait complete. 2024-04-14T08:34:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.748+0000 7f8f6b7b3740 20 client.5907 trim_cache size 0 max 16384 2024-04-14T08:34:44.756 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:34:44.756 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:34:44.756 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:34:44.756 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 5745, 2024-04-14T08:34:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:34:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:34:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:34:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 9.3254389209999999, 2024-04-14T08:34:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 23, 2024-04-14T08:34:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 50, 2024-04-14T08:34:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 50, 2024-04-14T08:34:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 14.364706782000001 2024-04-14T08:34:44.757 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:34:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.749+0000 7f8f689c8640 1 -- 172.21.15.172:0/87349941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8f4406c920 msgr2=0x7f8f4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.749+0000 7f8f689c8640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8f4406c920 0x7f8f4406ede0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f54036000 tx=0x7f8f5403d040 comp rx=0 tx=0).stop 2024-04-14T08:34:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.749+0000 7f8f689c8640 1 -- 172.21.15.172:0/87349941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f640cead0 msgr2=0x7f8f6414e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.749+0000 7f8f689c8640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f640cead0 0x7f8f6414e500 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00b3f0 tx=0x7f8f4c00b8b0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.749+0000 7f8f689c8640 1 -- 172.21.15.172:0/87349941 shutdown_connections 2024-04-14T08:34:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.749+0000 7f8f689c8640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8f4406c920 0x7f8f4406ede0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.749+0000 7f8f689c8640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f6415e670 0x7f8f640ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.749+0000 7f8f689c8640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f640cead0 0x7f8f6414e500 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.749+0000 7f8f689c8640 1 --2- 172.21.15.172:0/87349941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f64072ac0 0x7f8f640c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.749+0000 7f8f689c8640 1 -- 172.21.15.172:0/87349941 >> 172.21.15.172:0/87349941 conn(0x7f8f640cd980 msgr2=0x7f8f6406e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:44.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.749+0000 7f8f689c8640 1 -- 172.21.15.172:0/87349941 shutdown_connections 2024-04-14T08:34:44.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.749+0000 7f8f689c8640 1 -- 172.21.15.172:0/87349941 wait complete. 2024-04-14T08:34:44.765 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:34:44.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.867+0000 7f074006d640 1 Processor -- start 2024-04-14T08:34:44.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.867+0000 7f074006d640 1 -- start start 2024-04-14T08:34:44.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.868+0000 7f074006d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07380cfdd0 0x7f0738160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.868+0000 7f074006d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f07380cd650 0x7f07380cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.868+0000 7f074006d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07380b2fb0 con 0x7f073815f8f0 2024-04-14T08:34:44.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.868+0000 7f074006d640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0738160fe0 con 0x7f07380cd650 2024-04-14T08:34:44.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.868+0000 7f074006d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f07380cd1c0 con 0x7f07380cfdd0 2024-04-14T08:34:44.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.868+0000 7f073e60a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f07380cd650 0x7f07380cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.868+0000 7f073de09640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f073815f8f0 0x7f07380b4e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:53296/0 (socket says 172.21.15.172:53296) 2024-04-14T08:34:44.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.868+0000 7f073de09640 1 -- 172.21.15.172:0/148045639 learned_addr learned my addr 172.21.15.172:0/148045639 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:44.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.868+0000 7f073d608640 1 --2- 172.21.15.172:0/148045639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07380cfdd0 0x7f0738160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.869+0000 7f073e60a640 1 -- 172.21.15.172:0/148045639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07380cfdd0 msgr2=0x7f0738160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:44.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.869+0000 7f073e60a640 1 --2- 172.21.15.172:0/148045639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07380cfdd0 0x7f0738160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.869+0000 7f073e60a640 1 -- 172.21.15.172:0/148045639 >> v1:172.21.15.172:6789/0 conn(0x7f073815f8f0 legacy=0x7f07380b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.869+0000 7f073e60a640 1 -- 172.21.15.172:0/148045639 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07380b3290 con 0x7f07380cd650 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.869+0000 7f073ce07640 1 -- 172.21.15.172:0/148045639 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2661966351 0 0) 0x7f07380b2fb0 con 0x7f073815f8f0 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.869+0000 7f073d608640 1 --2- 172.21.15.172:0/148045639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07380cfdd0 0x7f0738160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.869+0000 7f073e60a640 1 --2- 172.21.15.172:0/148045639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07380cd650 0x7f07380cdb00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0730009d90 tx=0x7f073002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be4cd1f651b2a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.870+0000 7f073ce07640 1 -- 172.21.15.172:0/148045639 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730039070 con 0x7f07380cd650 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.870+0000 7f073ce07640 1 -- 172.21.15.172:0/148045639 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f073002cc50 con 0x7f07380cd650 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.870+0000 7f073ce07640 1 -- 172.21.15.172:0/148045639 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730034440 con 0x7f07380cd650 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.870+0000 7f074006d640 1 -- 172.21.15.172:0/148045639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07380cd650 msgr2=0x7f07380cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.870+0000 7f074006d640 1 --2- 172.21.15.172:0/148045639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07380cd650 0x7f07380cdb00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0730009d90 tx=0x7f073002bea0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.870+0000 7f074006d640 1 -- 172.21.15.172:0/148045639 shutdown_connections 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.870+0000 7f074006d640 1 --2- 172.21.15.172:0/148045639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07380cfdd0 0x7f0738160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.870+0000 7f074006d640 1 --2- 172.21.15.172:0/148045639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07380cd650 0x7f07380cdb00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.870+0000 7f074006d640 1 -- 172.21.15.172:0/148045639 >> 172.21.15.172:0/148045639 conn(0x7f07380cea70 msgr2=0x7f07380cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.870+0000 7f074006d640 1 -- 172.21.15.172:0/148045639 shutdown_connections 2024-04-14T08:34:44.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.870+0000 7f074006d640 1 -- 172.21.15.172:0/148045639 wait complete. 2024-04-14T08:34:44.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.871+0000 7f074006d640 1 Processor -- start 2024-04-14T08:34:44.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.871+0000 7f074006d640 1 -- start start 2024-04-14T08:34:44.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f074006d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07380cd650 0x7f07381f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f074006d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07380cfdd0 0x7f0738150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f074006d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073815f8f0 0x7f07380b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f074006d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f07380ccb00 con 0x7f073815f8f0 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f074006d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f07380cce50 con 0x7f07380cd650 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f074006d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f07380b5550 con 0x7f07380cfdd0 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f073e60a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073815f8f0 0x7f07380b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f073e60a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073815f8f0 0x7f07380b70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45858/0 (socket says 172.21.15.172:45858) 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f073e60a640 1 -- 172.21.15.172:0/2148684639 learned_addr learned my addr 172.21.15.172:0/2148684639 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f073de09640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07380cd650 0x7f07381f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f073d608640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07380cfdd0 0x7f0738150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f073e60a640 1 -- 172.21.15.172:0/2148684639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07380cfdd0 msgr2=0x7f0738150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f073e60a640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07380cfdd0 0x7f0738150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f073e60a640 1 -- 172.21.15.172:0/2148684639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07380cd650 msgr2=0x7f07381f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f073e60a640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07380cd650 0x7f07381f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f073e60a640 1 -- 172.21.15.172:0/2148684639 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07381f4440 con 0x7f073815f8f0 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.872+0000 7f073de09640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07380cd650 0x7f07381f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:44.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.873+0000 7f073e60a640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073815f8f0 0x7f07380b70b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0730009d90 tx=0x7f0730004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:44.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.873+0000 7f072e7fc640 1 -- 172.21.15.172:0/2148684639 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730039050 con 0x7f073815f8f0 2024-04-14T08:34:44.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.873+0000 7f072e7fc640 1 -- 172.21.15.172:0/2148684639 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0730045070 con 0x7f073815f8f0 2024-04-14T08:34:44.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.873+0000 7f074006d640 1 -- 172.21.15.172:0/2148684639 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07381f4720 con 0x7f073815f8f0 2024-04-14T08:34:44.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.873+0000 7f072e7fc640 1 -- 172.21.15.172:0/2148684639 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073003e610 con 0x7f073815f8f0 2024-04-14T08:34:44.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.873+0000 7f074006d640 1 -- 172.21.15.172:0/2148684639 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07381f4ce0 con 0x7f073815f8f0 2024-04-14T08:34:44.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.874+0000 7f074006d640 1 -- 172.21.15.172:0/2148684639 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0738068520 con 0x7f073815f8f0 2024-04-14T08:34:44.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.874+0000 7f072e7fc640 1 -- 172.21.15.172:0/2148684639 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f073003e7b0 con 0x7f073815f8f0 2024-04-14T08:34:44.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.875+0000 7f072e7fc640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f071c06c8f0 0x7f071c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:44.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.875+0000 7f072e7fc640 1 -- 172.21.15.172:0/2148684639 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f07300ae740 con 0x7f073815f8f0 2024-04-14T08:34:44.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.875+0000 7f073de09640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f071c06c8f0 0x7f071c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:44.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.875+0000 7f073de09640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f071c06c8f0 0x7f071c06edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f07340010e0 tx=0x7f0734009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:44.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:44.879+0000 7f072e7fc640 1 -- 172.21.15.172:0/2148684639 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0730032e20 con 0x7f073815f8f0 2024-04-14T08:34:45.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.096+0000 7f074006d640 1 -- 172.21.15.172:0/2148684639 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f073806c310 con 0x7f073815f8f0 2024-04-14T08:34:45.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.097+0000 7f072e7fc640 1 -- 172.21.15.172:0/2148684639 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f073007ca70 con 0x7f073815f8f0 2024-04-14T08:34:45.099 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:34:45.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.099+0000 7f074006d640 1 -- 172.21.15.172:0/2148684639 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f071c06c8f0 msgr2=0x7f071c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:45.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.099+0000 7f074006d640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f071c06c8f0 0x7f071c06edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f07340010e0 tx=0x7f0734009040 comp rx=0 tx=0).stop 2024-04-14T08:34:45.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.099+0000 7f074006d640 1 -- 172.21.15.172:0/2148684639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073815f8f0 msgr2=0x7f07380b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:45.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.099+0000 7f074006d640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073815f8f0 0x7f07380b70b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0730009d90 tx=0x7f0730004a40 comp rx=0 tx=0).stop 2024-04-14T08:34:45.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.099+0000 7f074006d640 1 -- 172.21.15.172:0/2148684639 shutdown_connections 2024-04-14T08:34:45.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.099+0000 7f074006d640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f071c06c8f0 0x7f071c06edb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:45.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.099+0000 7f074006d640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073815f8f0 0x7f07380b70b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:45.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.099+0000 7f074006d640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07380cfdd0 0x7f0738150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:45.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.099+0000 7f074006d640 1 --2- 172.21.15.172:0/2148684639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07380cd650 0x7f07381f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:45.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.099+0000 7f074006d640 1 -- 172.21.15.172:0/2148684639 >> 172.21.15.172:0/2148684639 conn(0x7f07380cea70 msgr2=0x7f073806ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:45.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.099+0000 7f074006d640 1 -- 172.21.15.172:0/2148684639 shutdown_connections 2024-04-14T08:34:45.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45.099+0000 7f074006d640 1 -- 172.21.15.172:0/2148684639 wait complete. 2024-04-14T08:34:45.112 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:34:45.112 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:34:45.112 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:34:45.112 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:34:45.112 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:34:45.112 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:34:45.112 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:34:45.112 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:34:45.112 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:34:45.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:45.113 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:34:45.129 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:34:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:34:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:34:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:34:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:45.130 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:34:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:34:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:34:45.130 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:34:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:34:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:34:45.130 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:34:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42584sec preferred_lft 42584sec 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:45.131 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:34:45.132 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:34:45.132 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:34:45.132 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:45.132 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:34:45.132 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:34:45.132 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:34:45.132 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:34:45.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:45.132 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:34:45.197 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:34:45.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:45.199 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:34:45.256 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:34:45.256 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:34:45.256 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:34:45.256 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:34:45.256 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:34:45.257 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:34:45.257 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:34:45.257 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:34:45.257 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:34:45.257 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:34:45.257 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:34:45.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:34:45.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:34:45.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:45.393 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:34:45.408 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:34:45.408 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:34:45.409 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:34:45.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:45.409 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:34:45.475 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:34:45.475 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:34:45.531 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:34:45.531 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:34:45.531 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:34:45.586 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:34:45.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:45.586 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:34:45.671 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:34:45.671 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:34:45.671 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:34:45.680 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:34:45.680 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:34:45.680 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:34:45.680 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:34:45.680 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:34:45.685 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:34:45.685 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:34:45.686 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:34:45.686 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:34:45.686 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:34:45.686 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:34:45.686 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:34:45.686 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:34:45.686 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:34:45.687 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:34:45.687 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:34:45.687 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:34:45.687 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:34:45.688 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:34:45.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:45.688 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:34:45.715 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:34:45.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:34:45.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:34:45.815 INFO:teuthology.orchestra.run.smithi172.stdout:client5782 2024-04-14T08:34:45.815 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:34:45.816 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:34:45.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:45.831 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-14T08:34:45.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:45.891 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-14T08:34:45.946 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:34:45.946 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:34:45.946 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:34:45.946 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:34:45.947 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-14T08:34:45.947 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:34:46.052 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627777 2024-04-14T08:34:46.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:34:46.061 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:34:46.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.202+0000 7f739df1e640 1 Processor -- start 2024-04-14T08:34:46.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.202+0000 7f739df1e640 1 -- start start 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.203+0000 7f739df1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7398072470 0x7f739815b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.203+0000 7f739df1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7398072970 0x7f739815b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.203+0000 7f739df1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73980b3360 con 0x7f739815a690 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.203+0000 7f739df1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f73980b3640 con 0x7f7398072970 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.203+0000 7f739df1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f73980b3920 con 0x7f7398072470 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.203+0000 7f73977fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f739815a690 0x7f7398071d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:53312/0 (socket says 172.21.15.172:53312) 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.203+0000 7f73977fe640 1 -- 172.21.15.172:0/1977706912 learned_addr learned my addr 172.21.15.172:0/1977706912 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.203+0000 7f738ffff640 1 --2- 172.21.15.172:0/1977706912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7398072470 0x7f739815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.203+0000 7f7397fff640 1 --2- 172.21.15.172:0/1977706912 >> v2:172.21.15.177:3300/0 conn(0x7f7398072970 0x7f739815b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.204+0000 7f7396ffd640 1 -- 172.21.15.172:0/1977706912 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2761763465 0 0) 0x7f73980b3360 con 0x7f739815a690 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.204+0000 7f7396ffd640 1 -- 172.21.15.172:0/1977706912 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7384003520 con 0x7f739815a690 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.204+0000 7f7396ffd640 1 -- 172.21.15.172:0/1977706912 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 662210496 0 0) 0x7f7384003520 con 0x7f739815a690 2024-04-14T08:34:46.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.204+0000 7f738ffff640 1 -- 172.21.15.172:0/1977706912 >> v2:172.21.15.177:3300/0 conn(0x7f7398072970 msgr2=0x7f739815b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:46.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.204+0000 7f738ffff640 1 --2- 172.21.15.172:0/1977706912 >> v2:172.21.15.177:3300/0 conn(0x7f7398072970 0x7f739815b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.204+0000 7f738ffff640 1 -- 172.21.15.172:0/1977706912 >> v1:172.21.15.172:6789/0 conn(0x7f739815a690 legacy=0x7f7398071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.204+0000 7f738ffff640 1 -- 172.21.15.172:0/1977706912 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73980b3c00 con 0x7f7398072470 2024-04-14T08:34:46.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.205+0000 7f738ffff640 1 --2- 172.21.15.172:0/1977706912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7398072470 0x7f739815b230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f73800097a0 tx=0x7f738002b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c94adb8cfdfffcb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.205+0000 7f7396ffd640 1 -- 172.21.15.172:0/1977706912 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380037070 con 0x7f7398072470 2024-04-14T08:34:46.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.205+0000 7f7396ffd640 1 -- 172.21.15.172:0/1977706912 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7380030510 con 0x7f7398072470 2024-04-14T08:34:46.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.205+0000 7f7396ffd640 1 -- 172.21.15.172:0/1977706912 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388001670 con 0x7f7398072470 2024-04-14T08:34:46.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.205+0000 7f739df1e640 1 -- 172.21.15.172:0/1977706912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7398072470 msgr2=0x7f739815b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.205+0000 7f739df1e640 1 --2- 172.21.15.172:0/1977706912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7398072470 0x7f739815b230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f73800097a0 tx=0x7f738002b710 comp rx=0 tx=0).stop 2024-04-14T08:34:46.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.205+0000 7f739df1e640 1 -- 172.21.15.172:0/1977706912 shutdown_connections 2024-04-14T08:34:46.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.205+0000 7f739df1e640 1 --2- 172.21.15.172:0/1977706912 >> v2:172.21.15.177:3300/0 conn(0x7f7398072970 0x7f739815b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.205+0000 7f739df1e640 1 --2- 172.21.15.172:0/1977706912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7398072470 0x7f739815b230 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.205+0000 7f739df1e640 1 -- 172.21.15.172:0/1977706912 >> 172.21.15.172:0/1977706912 conn(0x7f739806ec30 msgr2=0x7f739806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:46.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.205+0000 7f739df1e640 1 -- 172.21.15.172:0/1977706912 shutdown_connections 2024-04-14T08:34:46.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.205+0000 7f739df1e640 1 -- 172.21.15.172:0/1977706912 wait complete. 2024-04-14T08:34:46.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f739df1e640 1 Processor -- start 2024-04-14T08:34:46.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f739df1e640 1 -- start start 2024-04-14T08:34:46.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f739df1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7398072470 0x7f73980c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f739df1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7398072970 0x7f739814bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f739df1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f739815a690 0x7f739814c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f739df1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f739815aa70 con 0x7f7398072470 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f739df1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f73980b3100 con 0x7f739815a690 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f739df1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f739815abf0 con 0x7f7398072970 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f73977fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7398072470 0x7f73980c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f73977fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7398072470 0x7f73980c4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45866/0 (socket says 172.21.15.172:45866) 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f73977fe640 1 -- 172.21.15.172:0/2490795365 learned_addr learned my addr 172.21.15.172:0/2490795365 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f738ffff640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7398072970 0x7f739814bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.207+0000 7f7397fff640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f739815a690 0x7f739814c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.208+0000 7f73977fe640 1 -- 172.21.15.172:0/2490795365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7398072970 msgr2=0x7f739814bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.208+0000 7f73977fe640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7398072970 0x7f739814bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.208+0000 7f73977fe640 1 -- 172.21.15.172:0/2490795365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f739815a690 msgr2=0x7f739814c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.208+0000 7f73977fe640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f739815a690 0x7f739814c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.208+0000 7f73977fe640 1 -- 172.21.15.172:0/2490795365 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739814c010 con 0x7f7398072470 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.208+0000 7f7397fff640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f739815a690 0x7f739814c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.208+0000 7f73977fe640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7398072470 0x7f73980c4e60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f73880017e0 tx=0x7f738800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.208+0000 7f7394ff9640 1 -- 172.21.15.172:0/2490795365 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388005ad0 con 0x7f7398072470 2024-04-14T08:34:46.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.208+0000 7f7394ff9640 1 -- 172.21.15.172:0/2490795365 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7388005c90 con 0x7f7398072470 2024-04-14T08:34:46.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.208+0000 7f739df1e640 1 -- 172.21.15.172:0/2490795365 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739814d280 con 0x7f7398072470 2024-04-14T08:34:46.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.208+0000 7f7394ff9640 1 -- 172.21.15.172:0/2490795365 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73880072b0 con 0x7f7398072470 2024-04-14T08:34:46.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.208+0000 7f739df1e640 1 -- 172.21.15.172:0/2490795365 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7398157980 con 0x7f7398072470 2024-04-14T08:34:46.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.209+0000 7f739df1e640 1 -- 172.21.15.172:0/2490795365 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7398068520 con 0x7f7398072470 2024-04-14T08:34:46.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.210+0000 7f7394ff9640 1 -- 172.21.15.172:0/2490795365 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7388007450 con 0x7f7398072470 2024-04-14T08:34:46.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.210+0000 7f7394ff9640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f737806c8f0 0x7f737806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.210+0000 7f738ffff640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f737806c8f0 0x7f737806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.210+0000 7f7394ff9640 1 -- 172.21.15.172:0/2490795365 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f738808b9f0 con 0x7f7398072470 2024-04-14T08:34:46.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.211+0000 7f738ffff640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f737806c8f0 0x7f737806edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7380034000 tx=0x7f7380008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.215+0000 7f7394ff9640 1 -- 172.21.15.172:0/2490795365 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f739814c010 con 0x7f7398072470 2024-04-14T08:34:46.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.428+0000 7f739df1e640 1 -- 172.21.15.172:0/2490795365 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73980cc6b0 con 0x7f7398072470 2024-04-14T08:34:46.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.429+0000 7f7394ff9640 1 -- 172.21.15.172:0/2490795365 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5187 (secure 0 0 0) 0x7f738808b070 con 0x7f7398072470 2024-04-14T08:34:46.430 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:46.431 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/296861286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":296861286},{"type":"v1","addr":"172.21.15.177:6835","nonce":296861286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/642342704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":642342704},{"type":"v1","addr":"172.21.15.177:6833","nonce":642342704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3516903987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3516903987},{"type":"v1","addr":"172.21.15.172:6836","nonce":3516903987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:34:34.765057+0000","modified":"2024-04-14T08:34:36.783999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6837/1527574817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1527574817},{"type":"v1","addr":"172.21.15.172:6837","nonce":1527574817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":3}]} 2024-04-14T08:34:46.431 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 23 2024-04-14T08:34:46.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.431+0000 7f739df1e640 1 -- 172.21.15.172:0/2490795365 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f737806c8f0 msgr2=0x7f737806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.431+0000 7f739df1e640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f737806c8f0 0x7f737806edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7380034000 tx=0x7f7380008030 comp rx=0 tx=0).stop 2024-04-14T08:34:46.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.431+0000 7f739df1e640 1 -- 172.21.15.172:0/2490795365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7398072470 msgr2=0x7f73980c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.431+0000 7f739df1e640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7398072470 0x7f73980c4e60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f73880017e0 tx=0x7f738800cbe0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.431+0000 7f739df1e640 1 -- 172.21.15.172:0/2490795365 shutdown_connections 2024-04-14T08:34:46.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.431+0000 7f739df1e640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f737806c8f0 0x7f737806edb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.431+0000 7f739df1e640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f739815a690 0x7f739814c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.431+0000 7f739df1e640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7398072970 0x7f739814bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.431+0000 7f739df1e640 1 --2- 172.21.15.172:0/2490795365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7398072470 0x7f73980c4e60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.431+0000 7f739df1e640 1 -- 172.21.15.172:0/2490795365 >> 172.21.15.172:0/2490795365 conn(0x7f739806ec30 msgr2=0x7f739806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:46.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.431+0000 7f739df1e640 1 -- 172.21.15.172:0/2490795365 shutdown_connections 2024-04-14T08:34:46.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.431+0000 7f739df1e640 1 -- 172.21.15.172:0/2490795365 wait complete. 2024-04-14T08:34:46.443 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-14T08:34:46.575 INFO:teuthology.orchestra.run.smithi172.stdout:{"message":"","return_code":0} 2024-04-14T08:34:46.575 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-14T08:34:46.575 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-14T08:34:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.616+0000 7f1766a90f40 1 Processor -- start 2024-04-14T08:34:46.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.616+0000 7f1766a90f40 1 -- start start 2024-04-14T08:34:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.616+0000 7f1766a90f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5581d7e43330 0x5581d7e463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.616+0000 7f1766a90f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 0x5581d7f1ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.616+0000 7f1766a90f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581d7f1ca80 0x5581d7ef9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.616+0000 7f1766a90f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5581d7deb3b0 con 0x5581d7e8e510 2024-04-14T08:34:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.616+0000 7f1766a90f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5581d7def400 con 0x5581d7f1ca80 2024-04-14T08:34:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.616+0000 7f1766a90f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5581d7e8d870 con 0x5581d7e43330 2024-04-14T08:34:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1763b4e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 0x5581d7f1ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1763b4e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 0x5581d7f1ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45890/0 (socket says 172.21.15.172:45890) 2024-04-14T08:34:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1763b4e640 1 -- 172.21.15.172:0/3576189969 learned_addr learned my addr 172.21.15.172:0/3576189969 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f176434f640 1 --2- 172.21.15.172:0/3576189969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5581d7e43330 0x5581d7e463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1764b50640 1 --2- 172.21.15.172:0/3576189969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581d7f1ca80 0x5581d7ef9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1763b4e640 1 -- 172.21.15.172:0/3576189969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5581d7e43330 msgr2=0x5581d7e463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1763b4e640 1 --2- 172.21.15.172:0/3576189969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5581d7e43330 0x5581d7e463e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1763b4e640 1 -- 172.21.15.172:0/3576189969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581d7f1ca80 msgr2=0x5581d7ef9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1763b4e640 1 --2- 172.21.15.172:0/3576189969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581d7f1ca80 0x5581d7ef9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1763b4e640 1 -- 172.21.15.172:0/3576189969 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d7efd5d0 con 0x5581d7e8e510 2024-04-14T08:34:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1764b50640 1 --2- 172.21.15.172:0/3576189969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581d7f1ca80 0x5581d7ef9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:34:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1763b4e640 1 --2- 172.21.15.172:0/3576189969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 0x5581d7f1ebb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1758055040 tx=0x7f175804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab314577b2e5d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1762b4c640 1 -- 172.21.15.172:0/3576189969 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758057040 con 0x5581d7e8e510 2024-04-14T08:34:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1762b4c640 1 -- 172.21.15.172:0/3576189969 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175805e680 con 0x5581d7e8e510 2024-04-14T08:34:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.617+0000 7f1762b4c640 1 -- 172.21.15.172:0/3576189969 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175805ea70 con 0x5581d7e8e510 2024-04-14T08:34:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.618+0000 7f1766a90f40 1 -- 172.21.15.172:0/3576189969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 msgr2=0x5581d7f1ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.618+0000 7f1766a90f40 1 --2- 172.21.15.172:0/3576189969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 0x5581d7f1ebb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1758055040 tx=0x7f175804c000 comp rx=0 tx=0).stop 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.618+0000 7f1766a90f40 1 -- 172.21.15.172:0/3576189969 shutdown_connections 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.618+0000 7f1766a90f40 1 --2- 172.21.15.172:0/3576189969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581d7f1ca80 0x5581d7ef9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.618+0000 7f1766a90f40 1 --2- 172.21.15.172:0/3576189969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 0x5581d7f1ebb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.618+0000 7f1766a90f40 1 --2- 172.21.15.172:0/3576189969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5581d7e43330 0x5581d7e463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.618+0000 7f1766a90f40 1 -- 172.21.15.172:0/3576189969 >> 172.21.15.172:0/3576189969 conn(0x5581d7e70a50 msgr2=0x5581d7f1db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.618+0000 7f1766a90f40 1 -- 172.21.15.172:0/3576189969 shutdown_connections 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.618+0000 7f1766a90f40 1 -- 172.21.15.172:0/3576189969 wait complete. 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.618+0000 7f1766a90f40 1 Processor -- start 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.618+0000 7f1766a90f40 1 -- start start 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.619+0000 7f1766a90f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5581d7e43330 0x5581d7f156b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.619+0000 7f1766a90f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 0x5581d7f048e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.619+0000 7f1766a90f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581d7f1ca80 0x5581d7e463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.619+0000 7f1766a90f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5581d7e45260 con 0x5581d7e8e510 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.619+0000 7f1766a90f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5581d7efe8d0 con 0x5581d7f1ca80 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.619+0000 7f1766a90f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5581d7efe5f0 con 0x5581d7e43330 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.619+0000 7f1763b4e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 0x5581d7f048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.619+0000 7f1763b4e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 0x5581d7f048e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45892/0 (socket says 172.21.15.172:45892) 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.619+0000 7f1763b4e640 1 -- 172.21.15.172:0/3103612833 learned_addr learned my addr 172.21.15.172:0/3103612833 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.619+0000 7f176434f640 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5581d7e43330 0x5581d7f156b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.619+0000 7f1764b50640 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581d7f1ca80 0x5581d7e463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.620+0000 7f1763b4e640 1 -- 172.21.15.172:0/3103612833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5581d7e43330 msgr2=0x5581d7f156b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.620+0000 7f1763b4e640 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5581d7e43330 0x5581d7f156b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.620+0000 7f1763b4e640 1 -- 172.21.15.172:0/3103612833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581d7f1ca80 msgr2=0x5581d7e463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.620+0000 7f1763b4e640 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581d7f1ca80 0x5581d7e463a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.620+0000 7f1763b4e640 1 -- 172.21.15.172:0/3103612833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d7eff1a0 con 0x5581d7e8e510 2024-04-14T08:34:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.620+0000 7f1763b4e640 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 0x5581d7f048e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f175801ce70 tx=0x7f1758002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.620+0000 7f1761b4a640 1 -- 172.21.15.172:0/3103612833 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758057070 con 0x5581d7e8e510 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.620+0000 7f1761b4a640 1 -- 172.21.15.172:0/3103612833 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175805eaf0 con 0x5581d7e8e510 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.620+0000 7f1766a90f40 1 -- 172.21.15.172:0/3103612833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5581d7eff480 con 0x5581d7e8e510 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.620+0000 7f1761b4a640 1 -- 172.21.15.172:0/3103612833 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758052440 con 0x5581d7e8e510 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.620+0000 7f1766a90f40 1 -- 172.21.15.172:0/3103612833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d7effa40 con 0x5581d7e8e510 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.621+0000 7f1761b4a640 1 -- 172.21.15.172:0/3103612833 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f175805a040 con 0x5581d7e8e510 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.622+0000 7f1761b4a640 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f175006b300 0x7f175006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.622+0000 7f1761b4a640 1 -- 172.21.15.172:0/3103612833 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f17580cd6f0 con 0x5581d7e8e510 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.622+0000 7f176434f640 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f175006b300 0x7f175006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.622+0000 7f1766a90f40 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5581d7f03a80 0x5581d7f03e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.622+0000 7f1766a90f40 1 -- 172.21.15.172:0/3103612833 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50) -- 0x5581d7f16650 con 0x5581d7f03a80 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.622+0000 7f1764b50640 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5581d7f03a80 0x5581d7f03e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.622+0000 7f176434f640 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f175006b300 0x7f175006d7c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f175400d6e0 tx=0x7f175400f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.622+0000 7f1764b50640 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5581d7f03a80 0x5581d7f03e90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.623+0000 7f1764b50640 1 -- 172.21.15.172:0/3103612833 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f175c003450 con 0x5581d7f03a80 2024-04-14T08:34:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.623+0000 7f1766a90f40 1 -- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5581d7f03a80 msgr2=0x5581d7f03e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.623+0000 7f1766a90f40 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5581d7f03a80 0x5581d7f03e90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.623+0000 7f1766a90f40 1 -- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f175006b300 msgr2=0x7f175006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.623+0000 7f1766a90f40 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f175006b300 0x7f175006d7c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f175400d6e0 tx=0x7f175400f040 comp rx=0 tx=0).stop 2024-04-14T08:34:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.623+0000 7f1766a90f40 1 -- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 msgr2=0x5581d7f048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.623+0000 7f1766a90f40 1 --2- 172.21.15.172:0/3103612833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581d7e8e510 0x5581d7f048e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f175801ce70 tx=0x7f1758002fb0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.623+0000 7f1764b50640 1 -- 172.21.15.172:0/3103612833 reap_dead start 2024-04-14T08:34:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.623+0000 7f1766a90f40 1 -- 172.21.15.172:0/3103612833 shutdown_connections 2024-04-14T08:34:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.623+0000 7f1766a90f40 1 -- 172.21.15.172:0/3103612833 >> 172.21.15.172:0/3103612833 conn(0x5581d7e70a50 msgr2=0x5581d7e73740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.623+0000 7f1766a90f40 1 -- 172.21.15.172:0/3103612833 shutdown_connections 2024-04-14T08:34:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.623+0000 7f1766a90f40 1 -- 172.21.15.172:0/3103612833 wait complete. 2024-04-14T08:34:46.631 INFO:teuthology.orchestra.run.smithi172.stdout:Yalphaparent_a 2024-04-14T08:34:46.632 DEBUG:teuthology.orchestra.run.smithi172:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-14T08:34:46.758 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:34:46.758 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 1099511627777, 2024-04-14T08:34:46.758 INFO:teuthology.orchestra.run.smithi172.stdout: "ancestors": [ 2024-04-14T08:34:46.758 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:46.758 INFO:teuthology.orchestra.run.smithi172.stdout: "dirino": 1099511627776, 2024-04-14T08:34:46.758 INFO:teuthology.orchestra.run.smithi172.stdout: "dname": "alpha", 2024-04-14T08:34:46.758 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 2 2024-04-14T08:34:46.758 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:34:46.758 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:34:46.758 INFO:teuthology.orchestra.run.smithi172.stdout: "dirino": 1, 2024-04-14T08:34:46.758 INFO:teuthology.orchestra.run.smithi172.stdout: "dname": "parent_a", 2024-04-14T08:34:46.759 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 4 2024-04-14T08:34:46.759 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:46.759 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-04-14T08:34:46.759 INFO:teuthology.orchestra.run.smithi172.stdout: "pool": 8, 2024-04-14T08:34:46.759 INFO:teuthology.orchestra.run.smithi172.stdout: "old_pools": [] 2024-04-14T08:34:46.759 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:34:46.759 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:46.764 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-14T08:34:46.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.806+0000 7f1d89e4bf40 1 Processor -- start 2024-04-14T08:34:46.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.806+0000 7f1d89e4bf40 1 -- start start 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d89e4bf40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3eaf7e430 0x55e3eb0487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d89e4bf40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eafab790 0x55e3eb04fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d89e4bf40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e3eafc3a00 0x55e3eb049090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d89e4bf40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e3eafc5c70 con 0x55e3eafab790 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d89e4bf40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e3eaf233b0 con 0x55e3eaf7e430 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d89e4bf40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e3eafc5df0 con 0x55e3eafc3a00 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d867a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eafab790 0x55e3eb04fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d867a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eafab790 0x55e3eb04fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45908/0 (socket says 172.21.15.172:45908) 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d867a2640 1 -- 172.21.15.172:0/3333650536 learned_addr learned my addr 172.21.15.172:0/3333650536 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d86fa3640 1 --2- 172.21.15.172:0/3333650536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3eaf7e430 0x55e3eb0487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d877a4640 1 --2- 172.21.15.172:0/3333650536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e3eafc3a00 0x55e3eb049090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d867a2640 1 -- 172.21.15.172:0/3333650536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e3eafc3a00 msgr2=0x55e3eb049090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d867a2640 1 --2- 172.21.15.172:0/3333650536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e3eafc3a00 0x55e3eb049090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d867a2640 1 -- 172.21.15.172:0/3333650536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3eaf7e430 msgr2=0x55e3eb0487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d867a2640 1 --2- 172.21.15.172:0/3333650536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3eaf7e430 0x55e3eb0487f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d867a2640 1 -- 172.21.15.172:0/3333650536 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3eb051de0 con 0x55e3eafab790 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d867a2640 1 --2- 172.21.15.172:0/3333650536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eafab790 0x55e3eb04fcc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00a430 tx=0x7f1d7c00a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a087f9877c00e7e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d857a0640 1 -- 172.21.15.172:0/3333650536 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00b920 con 0x55e3eafab790 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d857a0640 1 -- 172.21.15.172:0/3333650536 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d7c00f030 con 0x55e3eafab790 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.807+0000 7f1d857a0640 1 -- 172.21.15.172:0/3333650536 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00bd20 con 0x55e3eafab790 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.808+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/3333650536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eafab790 msgr2=0x55e3eb04fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.808+0000 7f1d89e4bf40 1 --2- 172.21.15.172:0/3333650536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eafab790 0x55e3eb04fcc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00a430 tx=0x7f1d7c00a8f0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.808+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/3333650536 shutdown_connections 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.808+0000 7f1d89e4bf40 1 --2- 172.21.15.172:0/3333650536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e3eafc3a00 0x55e3eb049090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.808+0000 7f1d89e4bf40 1 --2- 172.21.15.172:0/3333650536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eafab790 0x55e3eb04fcc0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.808+0000 7f1d89e4bf40 1 --2- 172.21.15.172:0/3333650536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3eaf7e430 0x55e3eb0487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.808+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/3333650536 >> 172.21.15.172:0/3333650536 conn(0x55e3eafc6ca0 msgr2=0x55e3eafac0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:46.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.808+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/3333650536 shutdown_connections 2024-04-14T08:34:46.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.808+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/3333650536 wait complete. 2024-04-14T08:34:46.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.809+0000 7f1d89e4bf40 1 Processor -- start 2024-04-14T08:34:46.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.809+0000 7f1d89e4bf40 1 -- start start 2024-04-14T08:34:46.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d89e4bf40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eaf7e430 0x55e3eafa6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d89e4bf40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3eafab790 0x55e3eafa7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d89e4bf40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e3eafc3a00 0x55e3eb047410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d89e4bf40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e3eafc47e0 con 0x55e3eaf7e430 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d89e4bf40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e3eb05b1e0 con 0x55e3eafab790 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d89e4bf40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e3eafc31d0 con 0x55e3eafc3a00 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d86fa3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eaf7e430 0x55e3eafa6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d86fa3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eaf7e430 0x55e3eafa6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45914/0 (socket says 172.21.15.172:45914) 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d86fa3640 1 -- 172.21.15.172:0/2256335212 learned_addr learned my addr 172.21.15.172:0/2256335212 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d877a4640 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e3eafc3a00 0x55e3eb047410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d867a2640 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3eafab790 0x55e3eafa7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d86fa3640 1 -- 172.21.15.172:0/2256335212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e3eafc3a00 msgr2=0x55e3eb047410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d86fa3640 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e3eafc3a00 0x55e3eb047410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d86fa3640 1 -- 172.21.15.172:0/2256335212 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3eafab790 msgr2=0x55e3eafa7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d86fa3640 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3eafab790 0x55e3eafa7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d86fa3640 1 -- 172.21.15.172:0/2256335212 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3eb04a380 con 0x55e3eaf7e430 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.810+0000 7f1d867a2640 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3eafab790 0x55e3eafa7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.811+0000 7f1d86fa3640 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eaf7e430 0x55e3eafa6ff0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800a6b0 tx=0x7f1d78002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.811+0000 7f1d6ffff640 1 -- 172.21.15.172:0/2256335212 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d78064020 con 0x55e3eaf7e430 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.811+0000 7f1d6ffff640 1 -- 172.21.15.172:0/2256335212 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d78008740 con 0x55e3eaf7e430 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.811+0000 7f1d6ffff640 1 -- 172.21.15.172:0/2256335212 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d78008b30 con 0x55e3eaf7e430 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.811+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/2256335212 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e3eb04a660 con 0x55e3eaf7e430 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.811+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/2256335212 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3eb0578c0 con 0x55e3eaf7e430 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.812+0000 7f1d6ffff640 1 -- 172.21.15.172:0/2256335212 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d78067020 con 0x55e3eaf7e430 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.812+0000 7f1d6ffff640 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1d7406b2d0 0x7f1d7406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.813+0000 7f1d6ffff640 1 -- 172.21.15.172:0/2256335212 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f1d780cc9a0 con 0x55e3eaf7e430 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.813+0000 7f1d867a2640 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1d7406b2d0 0x7f1d7406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.813+0000 7f1d89e4bf40 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55e3eafa8010 0x55e3eb047950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.813+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/2256335212 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50) -- 0x55e3eb057db0 con 0x55e3eafa8010 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.813+0000 7f1d877a4640 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55e3eafa8010 0x55e3eb047950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.813+0000 7f1d867a2640 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1d7406b2d0 0x7f1d7406d790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55e3eafc4e20 tx=0x7f1d7c016040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.813+0000 7f1d877a4640 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55e3eafa8010 0x55e3eb047950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.814+0000 7f1d877a4640 1 -- 172.21.15.172:0/2256335212 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v50'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1d8000d750 con 0x55e3eafa8010 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.814+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55e3eafa8010 msgr2=0x55e3eb047950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.814+0000 7f1d89e4bf40 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55e3eafa8010 0x55e3eb047950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.814+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1d7406b2d0 msgr2=0x7f1d7406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.814+0000 7f1d89e4bf40 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1d7406b2d0 0x7f1d7406d790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55e3eafc4e20 tx=0x7f1d7c016040 comp rx=0 tx=0).stop 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.814+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eaf7e430 msgr2=0x55e3eafa6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.814+0000 7f1d89e4bf40 1 --2- 172.21.15.172:0/2256335212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3eaf7e430 0x55e3eafa6ff0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800a6b0 tx=0x7f1d78002fb0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.815+0000 7f1d877a4640 1 -- 172.21.15.172:0/2256335212 reap_dead start 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.815+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/2256335212 shutdown_connections 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.815+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/2256335212 >> 172.21.15.172:0/2256335212 conn(0x55e3eafc6ca0 msgr2=0x55e3eafac0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.815+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/2256335212 shutdown_connections 2024-04-14T08:34:46.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.815+0000 7f1d89e4bf40 1 -- 172.21.15.172:0/2256335212 wait complete. 2024-04-14T08:34:46.822 DEBUG:teuthology.orchestra.run.smithi172:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-14T08:34:46.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:46.865 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-14T08:34:46.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.970+0000 7f834c1ed640 1 Processor -- start 2024-04-14T08:34:46.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.970+0000 7f834c1ed640 1 -- start start 2024-04-14T08:34:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f834c1ed640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f834415fa90 0x7f83440b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f834c1ed640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83440d2b30 0x7f83440d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f834c1ed640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f834415fe70 con 0x7f834409aa70 2024-04-14T08:34:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f834c1ed640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f83440b31e0 con 0x7f834415fa90 2024-04-14T08:34:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f834c1ed640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f83440ce560 con 0x7f83440d2b30 2024-04-14T08:34:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f8349f89640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f834415fa90 0x7f83440b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f8349788640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83440d2b30 0x7f83440d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f834a78a640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f834409aa70 0x7f83440ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:40556/0 (socket says 172.21.15.172:40556) 2024-04-14T08:34:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f8349f89640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f834415fa90 0x7f83440b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:60584/0 (socket says 172.21.15.172:60584) 2024-04-14T08:34:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f8349788640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83440d2b30 0x7f83440d0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:35756/0 (socket says 172.21.15.172:35756) 2024-04-14T08:34:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f834a78a640 1 -- 172.21.15.172:0/185579364 learned_addr learned my addr 172.21.15.172:0/185579364 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:34:46.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f8349f89640 1 -- 172.21.15.172:0/185579364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83440d2b30 msgr2=0x7f83440d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:46.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f8349f89640 1 --2- 172.21.15.172:0/185579364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83440d2b30 0x7f83440d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f8349f89640 1 -- 172.21.15.172:0/185579364 >> v1:172.21.15.177:6789/0 conn(0x7f834409aa70 legacy=0x7f83440ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.972+0000 7f8349f89640 1 -- 172.21.15.172:0/185579364 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83440b3480 con 0x7f834415fa90 2024-04-14T08:34:46.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.973+0000 7f8349f89640 1 --2- 172.21.15.172:0/185579364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f834415fa90 0x7f83440b0ec0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f83400097a0 tx=0x7f834002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb2f4387cb3bf11a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.973+0000 7f8348f87640 1 -- 172.21.15.172:0/185579364 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83400315a0 con 0x7f834415fa90 2024-04-14T08:34:46.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.973+0000 7f8348f87640 1 -- 172.21.15.172:0/185579364 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8340037070 con 0x7f834415fa90 2024-04-14T08:34:46.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.973+0000 7f8348f87640 1 -- 172.21.15.172:0/185579364 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8340031960 con 0x7f834415fa90 2024-04-14T08:34:46.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.974+0000 7f834c1ed640 1 -- 172.21.15.172:0/185579364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f834415fa90 msgr2=0x7f83440b0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.974+0000 7f834c1ed640 1 --2- 172.21.15.172:0/185579364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f834415fa90 0x7f83440b0ec0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f83400097a0 tx=0x7f834002b840 comp rx=0 tx=0).stop 2024-04-14T08:34:46.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.974+0000 7f834c1ed640 1 -- 172.21.15.172:0/185579364 shutdown_connections 2024-04-14T08:34:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.974+0000 7f834c1ed640 1 --2- 172.21.15.172:0/185579364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83440d2b30 0x7f83440d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.974+0000 7f834c1ed640 1 --2- 172.21.15.172:0/185579364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f834415fa90 0x7f83440b0ec0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.974+0000 7f834c1ed640 1 -- 172.21.15.172:0/185579364 >> 172.21.15.172:0/185579364 conn(0x7f83440d2040 msgr2=0x7f83440d2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.974+0000 7f834c1ed640 1 -- 172.21.15.172:0/185579364 shutdown_connections 2024-04-14T08:34:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.974+0000 7f834c1ed640 1 -- 172.21.15.172:0/185579364 wait complete. 2024-04-14T08:34:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.976+0000 7f834c1ed640 1 Processor -- start 2024-04-14T08:34:46.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.976+0000 7f834c1ed640 1 -- start start 2024-04-14T08:34:46.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.977+0000 7f834c1ed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f834409aa70 0x7f834415eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.977+0000 7f834c1ed640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83440d2b30 0x7f8344157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.977+0000 7f834c1ed640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f834415fa90 0x7f834415c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.977+0000 7f834c1ed640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83440ce840 con 0x7f834409aa70 2024-04-14T08:34:46.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.977+0000 7f834c1ed640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f834409b7c0 con 0x7f834415fa90 2024-04-14T08:34:46.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.977+0000 7f834c1ed640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f83440d10b0 con 0x7f83440d2b30 2024-04-14T08:34:46.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.977+0000 7f8349f89640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f834409aa70 0x7f834415eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.977+0000 7f8349f89640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f834409aa70 0x7f834415eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45926/0 (socket says 172.21.15.172:45926) 2024-04-14T08:34:46.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.977+0000 7f8349f89640 1 -- 172.21.15.172:0/3423813546 learned_addr learned my addr 172.21.15.172:0/3423813546 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f8349788640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83440d2b30 0x7f8344157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f834a78a640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f834415fa90 0x7f834415c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f8349f89640 1 -- 172.21.15.172:0/3423813546 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83440d2b30 msgr2=0x7f8344157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f8349f89640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83440d2b30 0x7f8344157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f8349f89640 1 -- 172.21.15.172:0/3423813546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f834415fa90 msgr2=0x7f834415c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f8349f89640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f834415fa90 0x7f834415c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f8349f89640 1 -- 172.21.15.172:0/3423813546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834409ff60 con 0x7f834409aa70 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f8349f89640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f834409aa70 0x7f834415eb80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8340034040 tx=0x7f83400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f833a7fc640 1 -- 172.21.15.172:0/3423813546 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8340032630 con 0x7f834409aa70 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f833a7fc640 1 -- 172.21.15.172:0/3423813546 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83400327f0 con 0x7f834409aa70 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f833a7fc640 1 -- 172.21.15.172:0/3423813546 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8340032be0 con 0x7f834409aa70 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f834c1ed640 1 -- 172.21.15.172:0/3423813546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834409f9a0 con 0x7f834409aa70 2024-04-14T08:34:46.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.978+0000 7f834c1ed640 1 -- 172.21.15.172:0/3423813546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8344157c90 con 0x7f834409aa70 2024-04-14T08:34:46.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.980+0000 7f833a7fc640 1 -- 172.21.15.172:0/3423813546 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f834004e430 con 0x7f834409aa70 2024-04-14T08:34:46.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.980+0000 7f833a7fc640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f832806c8f0 0x7f832806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.980+0000 7f833a7fc640 1 -- 172.21.15.172:0/3423813546 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f834002f030 con 0x7f834409aa70 2024-04-14T08:34:46.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.980+0000 7f8349788640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f832806c8f0 0x7f832806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.981+0000 7f8349788640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f832806c8f0 0x7f832806edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f83441f7c30 tx=0x7f8334008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.982+0000 7f834efd1740 1 Processor -- start 2024-04-14T08:34:46.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.982+0000 7f834efd1740 1 -- start start 2024-04-14T08:34:46.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f834efd1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d904fd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f834efd1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904cd300 0x561d904fa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f834efd1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 0x561d904faa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f834efd1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561d90437740 con 0x561d904fd2e0 2024-04-14T08:34:46.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f834efd1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561d903df930 con 0x561d904cd300 2024-04-14T08:34:46.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f834efd1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x561d90433890 con 0x561d904d0e60 2024-04-14T08:34:46.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f8349f89640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d904fd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f8349f89640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d904fd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45942/0 (socket says 172.21.15.172:45942) 2024-04-14T08:34:46.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f8349f89640 1 -- 172.21.15.172:0/2939636300 learned_addr learned my addr 172.21.15.172:0/2939636300 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:46.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f8349f89640 1 -- 172.21.15.172:0/2939636300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 msgr2=0x561d904faa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:46.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f8349f89640 1 --2- 172.21.15.172:0/2939636300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 0x561d904faa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f8349f89640 1 -- 172.21.15.172:0/2939636300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904cd300 msgr2=0x561d904fa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:46.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f8349f89640 1 --2- 172.21.15.172:0/2939636300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904cd300 0x561d904fa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f8349f89640 1 -- 172.21.15.172:0/2939636300 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d902137e0 con 0x561d904fd2e0 2024-04-14T08:34:46.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f8349f89640 1 --2- 172.21.15.172:0/2939636300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d904fd6c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f83400abe50 tx=0x7f834002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db722640d8a2128 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f83277fe640 1 -- 172.21.15.172:0/2939636300 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83400aa070 con 0x561d904fd2e0 2024-04-14T08:34:46.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f83277fe640 1 -- 172.21.15.172:0/2939636300 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8340037050 con 0x561d904fd2e0 2024-04-14T08:34:46.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.983+0000 7f83277fe640 1 -- 172.21.15.172:0/2939636300 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834007b6e0 con 0x561d904fd2e0 2024-04-14T08:34:46.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.984+0000 7f834efd1740 1 -- 172.21.15.172:0/2939636300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 msgr2=0x561d904fd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.984+0000 7f834efd1740 1 --2- 172.21.15.172:0/2939636300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d904fd6c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f83400abe50 tx=0x7f834002b840 comp rx=0 tx=0).stop 2024-04-14T08:34:46.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.984+0000 7f834efd1740 1 -- 172.21.15.172:0/2939636300 shutdown_connections 2024-04-14T08:34:46.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.984+0000 7f834efd1740 1 --2- 172.21.15.172:0/2939636300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 0x561d904faa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.984+0000 7f834efd1740 1 --2- 172.21.15.172:0/2939636300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904cd300 0x561d904fa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.984+0000 7f834efd1740 1 --2- 172.21.15.172:0/2939636300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d904fd6c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.984+0000 7f834efd1740 1 -- 172.21.15.172:0/2939636300 >> 172.21.15.172:0/2939636300 conn(0x561d904cf570 msgr2=0x561d905180f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:46.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.984+0000 7f834efd1740 1 -- 172.21.15.172:0/2939636300 shutdown_connections 2024-04-14T08:34:46.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.984+0000 7f834efd1740 1 -- 172.21.15.172:0/2939636300 wait complete. 2024-04-14T08:34:46.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.985+0000 7f834efd1740 1 -- start start 2024-04-14T08:34:46.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.985+0000 7f834efd1740 1 Processor -- start 2024-04-14T08:34:46.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f834efd1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 0x561d904ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f834efd1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d904e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f834efd1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904e7f60 0x561d904e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f834efd1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561d903df930 con 0x561d904fd2e0 2024-04-14T08:34:46.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f834efd1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561d903f2cf0 con 0x561d904e7f60 2024-04-14T08:34:46.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f834efd1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x561d90437740 con 0x561d904d0e60 2024-04-14T08:34:46.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f834efd1740 10 client.0 _ll_register_callbacks cb 0x561d903bf9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:34:46.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f8349788640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d904e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f8349788640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d904e7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45946/0 (socket says 172.21.15.172:45946) 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f8349788640 1 -- 172.21.15.172:0/3238470196 learned_addr learned my addr 172.21.15.172:0/3238470196 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f834a78a640 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904e7f60 0x561d904e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f8349f89640 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 0x561d904ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f8349788640 1 -- 172.21.15.172:0/3238470196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 msgr2=0x561d904ff2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f8349788640 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 0x561d904ff2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f8349788640 1 -- 172.21.15.172:0/3238470196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904e7f60 msgr2=0x561d904e6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f8349788640 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904e7f60 0x561d904e6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f8349788640 1 -- 172.21.15.172:0/3238470196 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d90273f70 con 0x561d904fd2e0 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f8349788640 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d904e7a20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f83340154b0 tx=0x7f8334009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec50f3d43c2fbbd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.987+0000 7f8324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f8324ff9640 1 -- 172.21.15.172:0/3238470196 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833401c050 con 0x561d904fd2e0 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f8324ff9640 1 -- 172.21.15.172:0/3238470196 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f83340145f0 con 0x561d904fd2e0 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f834efd1740 1 -- 172.21.15.172:0/3238470196 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8340037050 con 0x561d904fd2e0 2024-04-14T08:34:46.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f8324ff9640 1 -- 172.21.15.172:0/3238470196 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f833402e050 con 0x561d904fd2e0 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f834efd1740 10 client.5952 fetch_fsmap learned FSMap version 23 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f834efd1740 1 -- 172.21.15.172:0/3238470196 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x561d9010b170 con 0x561d904fd2e0 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f8324ff9640 1 -- 172.21.15.172:0/3238470196 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8340037050 con 0x561d904fd2e0 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f8324ff9640 1 -- 172.21.15.172:0/3238470196 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f8334031020 con 0x561d904fd2e0 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f834efd1740 10 client.5952 fetch_fsmap finished waiting for FSMap version 23 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f834efd1740 10 client.5952 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f834efd1740 20 client.5952 populate_metadata read hostname 'smithi172' 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f834efd1740 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x561d9052c020 0x561d9052c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f834efd1740 4 client.5952 mds_command: new command op to 5745 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f834efd1740 1 -- 172.21.15.172:0/3238470196 --> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d90444c80 con 0x561d9052c020 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.988+0000 7f8349f89640 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x561d9052c020 0x561d9052c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f8324ff9640 10 client.5952 ms_handle_connect on v2:172.21.15.172:6835/1527574817 2024-04-14T08:34:46.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f8349f89640 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x561d9052c020 0x561d9052c400 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd445323ec84c21 server_cookie=3158775688a4ba2 in_seq=0 out_seq=0 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f8324ff9640 1 -- 172.21.15.172:0/3238470196 <== mds.0 v2:172.21.15.172:6835/1527574817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d90444c80 con 0x561d9052c020 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f8324ff9640 10 client.5952 handle_command_reply: tid=0 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f834efd1740 1 client.5952 shutdown 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f834efd1740 1 -- 172.21.15.172:0/3238470196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 msgr2=0x561d904e7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f834efd1740 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d904e7a20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f83340154b0 tx=0x7f8334009190 comp rx=0 tx=0).stop 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f834efd1740 1 -- 172.21.15.172:0/3238470196 shutdown_connections 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f834efd1740 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x561d9052c020 0x561d9052c400 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f834efd1740 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904e7f60 0x561d904e6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f834efd1740 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d904e7a20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f834efd1740 1 --2- 172.21.15.172:0/3238470196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 0x561d904ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f834efd1740 1 -- 172.21.15.172:0/3238470196 >> 172.21.15.172:0/3238470196 conn(0x561d904cd300 msgr2=0x561d904d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f8324ff9640 0 client.5952 ms_handle_reset on v2:172.21.15.172:6835/1527574817 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f834efd1740 1 -- 172.21.15.172:0/3238470196 shutdown_connections 2024-04-14T08:34:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.989+0000 7f834efd1740 1 -- 172.21.15.172:0/3238470196 wait complete. 2024-04-14T08:34:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.990+0000 7f834efd1740 20 client.5952 trim_cache size 0 max 16384 2024-04-14T08:34:46.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.991+0000 7fc077ab7640 1 Processor -- start 2024-04-14T08:34:46.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.991+0000 7fc077ab7640 1 -- start start 2024-04-14T08:34:46.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc077ab7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0700b3710 0x7fc0700cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc077ab7640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc0700d0c20 0x7fc0700d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:46.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc077ab7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0700d32d0 con 0x7fc0700b3330 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc077ab7640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc0700cf410 con 0x7fc0700d0c20 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc077ab7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc0700cf6f0 con 0x7fc0700b3710 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc076054640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc0700d0c20 0x7fc0700d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc076054640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc0700d0c20 0x7fc0700d10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45948/0 (socket says 172.21.15.172:45948) 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc076054640 1 -- 172.21.15.172:0/3279801533 learned_addr learned my addr 172.21.15.172:0/3279801533 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc075052640 1 --2- 172.21.15.172:0/3279801533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0700b3710 0x7fc0700cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc076054640 1 -- 172.21.15.172:0/3279801533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0700b3710 msgr2=0x7fc0700cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc076054640 1 --2- 172.21.15.172:0/3279801533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0700b3710 0x7fc0700cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc076054640 1 -- 172.21.15.172:0/3279801533 >> v1:172.21.15.177:6789/0 conn(0x7fc0700b3330 legacy=0x7fc0700d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc076054640 1 -- 172.21.15.172:0/3279801533 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0700cf9d0 con 0x7fc0700d0c20 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.993+0000 7fc075052640 1 --2- 172.21.15.172:0/3279801533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0700b3710 0x7fc0700cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.994+0000 7fc076054640 1 --2- 172.21.15.172:0/3279801533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0700d0c20 0x7fc0700d10b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc07006bf70 tx=0x7fc06802be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7483d90d265760c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.994+0000 7fc074851640 1 -- 172.21.15.172:0/3279801533 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06802c9a0 con 0x7fc0700d0c20 2024-04-14T08:34:46.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.994+0000 7fc074851640 1 -- 172.21.15.172:0/3279801533 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc06802cb40 con 0x7fc0700d0c20 2024-04-14T08:34:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.994+0000 7fc074851640 1 -- 172.21.15.172:0/3279801533 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0680343f0 con 0x7fc0700d0c20 2024-04-14T08:34:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.995+0000 7fc077ab7640 1 -- 172.21.15.172:0/3279801533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0700d0c20 msgr2=0x7fc0700d10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.995+0000 7fc077ab7640 1 --2- 172.21.15.172:0/3279801533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0700d0c20 0x7fc0700d10b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc07006bf70 tx=0x7fc06802be00 comp rx=0 tx=0).stop 2024-04-14T08:34:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.995+0000 7fc077ab7640 1 -- 172.21.15.172:0/3279801533 shutdown_connections 2024-04-14T08:34:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.995+0000 7fc077ab7640 1 --2- 172.21.15.172:0/3279801533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0700d0c20 0x7fc0700d10b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.995+0000 7fc077ab7640 1 --2- 172.21.15.172:0/3279801533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0700b3710 0x7fc0700cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.995+0000 7fc077ab7640 1 -- 172.21.15.172:0/3279801533 >> 172.21.15.172:0/3279801533 conn(0x7fc07009d580 msgr2=0x7fc0700d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.995+0000 7fc077ab7640 1 -- 172.21.15.172:0/3279801533 shutdown_connections 2024-04-14T08:34:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.995+0000 7fc077ab7640 1 -- 172.21.15.172:0/3279801533 wait complete. 2024-04-14T08:34:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.997+0000 7fc077ab7640 1 Processor -- start 2024-04-14T08:34:47.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.997+0000 7fc077ab7640 1 -- start start 2024-04-14T08:34:47.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.998+0000 7fc077ab7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0700b3330 0x7fc0700b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.998+0000 7fc077ab7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0700b3710 0x7fc07015b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.998+0000 7fc077ab7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0700d0c20 0x7fc07015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.998+0000 7fc077ab7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0700cff90 con 0x7fc0700d0c20 2024-04-14T08:34:47.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.998+0000 7fc077ab7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0700cfcb0 con 0x7fc0700b3330 2024-04-14T08:34:47.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.998+0000 7fc077ab7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc07015de90 con 0x7fc0700b3710 2024-04-14T08:34:47.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.998+0000 7fc076054640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0700d0c20 0x7fc07015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.998+0000 7fc076054640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0700d0c20 0x7fc07015bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45952/0 (socket says 172.21.15.172:45952) 2024-04-14T08:34:47.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.998+0000 7fc076054640 1 -- 172.21.15.172:0/3365146098 learned_addr learned my addr 172.21.15.172:0/3365146098 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:47.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.998+0000 7fc075853640 1 --2- 172.21.15.172:0/3365146098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0700b3330 0x7fc0700b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.998+0000 7fc075052640 1 --2- 172.21.15.172:0/3365146098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0700b3710 0x7fc07015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.999+0000 7fc076054640 1 -- 172.21.15.172:0/3365146098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0700b3710 msgr2=0x7fc07015b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.999+0000 7fc076054640 1 --2- 172.21.15.172:0/3365146098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0700b3710 0x7fc07015b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.999+0000 7fc076054640 1 -- 172.21.15.172:0/3365146098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0700b3330 msgr2=0x7fc0700b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.999+0000 7fc076054640 1 --2- 172.21.15.172:0/3365146098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0700b3330 0x7fc0700b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.999+0000 7fc076054640 1 -- 172.21.15.172:0/3365146098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0700b85e0 con 0x7fc0700d0c20 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.999+0000 7fc075853640 1 --2- 172.21.15.172:0/3365146098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0700b3330 0x7fc0700b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.999+0000 7fc076054640 1 --2- 172.21.15.172:0/3365146098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0700d0c20 0x7fc07015bd40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc068036040 tx=0x7fc068004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.999+0000 7fc0667fc640 1 -- 172.21.15.172:0/3365146098 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068043020 con 0x7fc0700d0c20 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.999+0000 7fc0667fc640 1 -- 172.21.15.172:0/3365146098 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc06803e450 con 0x7fc0700d0c20 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.999+0000 7fc0667fc640 1 -- 172.21.15.172:0/3365146098 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06803e860 con 0x7fc0700d0c20 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.999+0000 7fc077ab7640 1 -- 172.21.15.172:0/3365146098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07015f4e0 con 0x7fc0700d0c20 2024-04-14T08:34:47.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:46.999+0000 7fc077ab7640 1 -- 172.21.15.172:0/3365146098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07015faa0 con 0x7fc0700d0c20 2024-04-14T08:34:47.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.000+0000 7fc077ab7640 1 -- 172.21.15.172:0/3365146098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc070068590 con 0x7fc0700d0c20 2024-04-14T08:34:47.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.001+0000 7fc0667fc640 1 -- 172.21.15.172:0/3365146098 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc068046020 con 0x7fc0700d0c20 2024-04-14T08:34:47.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.001+0000 7fc0667fc640 1 --2- 172.21.15.172:0/3365146098 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05406c8f0 0x7fc05406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.002+0000 7fc0667fc640 1 -- 172.21.15.172:0/3365146098 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fc0680ae9f0 con 0x7fc0700d0c20 2024-04-14T08:34:47.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.002+0000 7fc075853640 1 --2- 172.21.15.172:0/3365146098 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05406c8f0 0x7fc05406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.002+0000 7fc0667fc640 1 -- 172.21.15.172:0/3365146098 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7fc06804d030 con 0x7fc0700d0c20 2024-04-14T08:34:47.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.002+0000 7fc077ab7640 1 -- 172.21.15.172:0/3365146098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fc0700d4950 con 0x7fc0700d0c20 2024-04-14T08:34:47.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.002+0000 7fc075853640 1 --2- 172.21.15.172:0/3365146098 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05406c8f0 0x7fc05406edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc0700b2aa0 tx=0x7fc06c007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:47.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.011+0000 7f834efd1740 1 Processor -- start 2024-04-14T08:34:47.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.011+0000 7f834efd1740 1 -- start start 2024-04-14T08:34:47.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834efd1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 0x561d9051bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834efd1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904e7f60 0x561d904ffc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834efd1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d90515390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834efd1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83442011e0 con 0x561d904fd2e0 2024-04-14T08:34:47.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834efd1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8344200200 con 0x561d904e7f60 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834efd1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f834406d6f0 con 0x561d904d0e60 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834a78a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d90515390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834a78a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d90515390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45966/0 (socket says 172.21.15.172:45966) 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834a78a640 1 -- 172.21.15.172:0/1814991158 learned_addr learned my addr 172.21.15.172:0/1814991158 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834a78a640 1 -- 172.21.15.172:0/1814991158 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 msgr2=0x561d9051bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f8349788640 1 --2- 172.21.15.172:0/1814991158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904e7f60 0x561d904ffc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f8349f89640 1 --2- 172.21.15.172:0/1814991158 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 0x561d9051bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834a78a640 1 --2- 172.21.15.172:0/1814991158 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 0x561d9051bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834a78a640 1 -- 172.21.15.172:0/1814991158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904e7f60 msgr2=0x561d904ffc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834a78a640 1 --2- 172.21.15.172:0/1814991158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904e7f60 0x561d904ffc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834a78a640 1 -- 172.21.15.172:0/1814991158 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d904ecc70 con 0x561d904fd2e0 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f834a78a640 1 --2- 172.21.15.172:0/1814991158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d90515390 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f833c001e50 tx=0x7f833c011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10da8d7fb595784b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f8326ffd640 1 -- 172.21.15.172:0/1814991158 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c011e50 con 0x561d904fd2e0 2024-04-14T08:34:47.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f8326ffd640 1 -- 172.21.15.172:0/1814991158 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833c01a070 con 0x561d904fd2e0 2024-04-14T08:34:47.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.012+0000 7f8326ffd640 1 -- 172.21.15.172:0/1814991158 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c015670 con 0x561d904fd2e0 2024-04-14T08:34:47.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.013+0000 7f834efd1740 1 -- 172.21.15.172:0/1814991158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 msgr2=0x561d90515390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.013+0000 7f834efd1740 1 --2- 172.21.15.172:0/1814991158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d90515390 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f833c001e50 tx=0x7f833c011620 comp rx=0 tx=0).stop 2024-04-14T08:34:47.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.013+0000 7f834efd1740 1 -- 172.21.15.172:0/1814991158 shutdown_connections 2024-04-14T08:34:47.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.013+0000 7f834efd1740 1 --2- 172.21.15.172:0/1814991158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904fd2e0 0x561d90515390 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.013+0000 7f834efd1740 1 --2- 172.21.15.172:0/1814991158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904e7f60 0x561d904ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.013+0000 7f834efd1740 1 --2- 172.21.15.172:0/1814991158 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904d0e60 0x561d9051bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.013+0000 7f834efd1740 1 -- 172.21.15.172:0/1814991158 >> 172.21.15.172:0/1814991158 conn(0x561d904cd300 msgr2=0x561d90501db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:47.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.013+0000 7f834efd1740 1 -- 172.21.15.172:0/1814991158 shutdown_connections 2024-04-14T08:34:47.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.013+0000 7f834efd1740 1 -- 172.21.15.172:0/1814991158 wait complete. 2024-04-14T08:34:47.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.014+0000 7f834efd1740 1 -- start start 2024-04-14T08:34:47.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.014+0000 7f834efd1740 1 Processor -- start 2024-04-14T08:34:47.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f834efd1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904d0e60 0x561d90518960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f834efd1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904e7f60 0x561d904d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f834efd1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904fd2e0 0x561d904d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f834efd1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561d904cd790 con 0x561d904e7f60 2024-04-14T08:34:47.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f834efd1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f834406dd40 con 0x561d904d0e60 2024-04-14T08:34:47.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f834efd1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x561d904e7dc0 con 0x561d904fd2e0 2024-04-14T08:34:47.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f834efd1740 10 client.0 _ll_register_callbacks cb 0x7f8334009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:34:47.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8349788640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904e7f60 0x561d904d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8349788640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904e7f60 0x561d904d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45982/0 (socket says 172.21.15.172:45982) 2024-04-14T08:34:47.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8349788640 1 -- 172.21.15.172:0/2551413571 learned_addr learned my addr 172.21.15.172:0/2551413571 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8349f89640 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904d0e60 0x561d90518960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f834a78a640 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904fd2e0 0x561d904d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8349788640 1 -- 172.21.15.172:0/2551413571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904fd2e0 msgr2=0x561d904d1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8349788640 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904fd2e0 0x561d904d1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8349788640 1 -- 172.21.15.172:0/2551413571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904d0e60 msgr2=0x561d90518960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8349788640 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904d0e60 0x561d90518960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8349788640 1 -- 172.21.15.172:0/2551413571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d9057aea0 con 0x561d904e7f60 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f834a78a640 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904fd2e0 0x561d904d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8349788640 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904e7f60 0x561d904d1400 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f833400a010 tx=0x7f8334017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bd15f856e78dc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8324ff9640 1 -- 172.21.15.172:0/2551413571 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334038070 con 0x561d904e7f60 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f8324ff9640 1 -- 172.21.15.172:0/2551413571 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f8334021650 con 0x561d904e7f60 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.015+0000 7f834efd1740 1 -- 172.21.15.172:0/2551413571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f833c01a070 con 0x561d904e7f60 2024-04-14T08:34:47.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f8324ff9640 1 -- 172.21.15.172:0/2551413571 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8334043050 con 0x561d904e7f60 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f834efd1740 10 client.5964 fetch_fsmap learned FSMap version 23 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f834efd1740 1 -- 172.21.15.172:0/2551413571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x561d9052b500 con 0x561d904e7f60 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f8324ff9640 1 -- 172.21.15.172:0/2551413571 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833c01a070 con 0x561d904e7f60 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f8324ff9640 1 -- 172.21.15.172:0/2551413571 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f8334046020 con 0x561d904e7f60 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f834efd1740 10 client.5964 fetch_fsmap finished waiting for FSMap version 23 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f834efd1740 10 client.5964 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f834efd1740 20 client.5964 populate_metadata read hostname 'smithi172' 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f834efd1740 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x561d9057a990 0x561d9052a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f834efd1740 4 client.5964 mds_command: new command op to 5745 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f834efd1740 1 -- 172.21.15.172:0/2551413571 --> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x561d904cdb00 con 0x561d9057a990 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f8349f89640 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x561d9057a990 0x561d9052a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f8324ff9640 10 client.5964 ms_handle_connect on v2:172.21.15.172:6835/1527574817 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.016+0000 7f8349f89640 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x561d9057a990 0x561d9052a7e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b9faf99b32e37f server_cookie=bb95d71de6675647 in_seq=0 out_seq=0 2024-04-14T08:34:47.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f8324ff9640 1 -- 172.21.15.172:0/2551413571 <== mds.0 v2:172.21.15.172:6835/1527574817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561d904cdb00 con 0x561d9057a990 2024-04-14T08:34:47.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f8324ff9640 10 client.5964 handle_command_reply: tid=0 2024-04-14T08:34:47.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f834efd1740 1 client.5964 shutdown 2024-04-14T08:34:47.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f834efd1740 1 -- 172.21.15.172:0/2551413571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904e7f60 msgr2=0x561d904d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f834efd1740 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904e7f60 0x561d904d1400 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f833400a010 tx=0x7f8334017000 comp rx=0 tx=0).stop 2024-04-14T08:34:47.024 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f39ff9e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5782=0-4194304@1} caps={5782=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5782 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x558f4cb16680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-14T08:34:47.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f834efd1740 1 -- 172.21.15.172:0/2551413571 shutdown_connections 2024-04-14T08:34:47.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f834efd1740 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x561d9057a990 0x561d9052a7e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f834efd1740 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561d904fd2e0 0x561d904d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f834efd1740 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d904e7f60 0x561d904d1400 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f834efd1740 1 --2- 172.21.15.172:0/2551413571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561d904d0e60 0x561d90518960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f834efd1740 1 -- 172.21.15.172:0/2551413571 >> 172.21.15.172:0/2551413571 conn(0x561d904cd300 msgr2=0x561d9051bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:47.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.017+0000 7f8324ff9640 0 client.5964 ms_handle_reset on v2:172.21.15.172:6835/1527574817 2024-04-14T08:34:47.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834efd1740 1 -- 172.21.15.172:0/2551413571 shutdown_connections 2024-04-14T08:34:47.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834efd1740 1 -- 172.21.15.172:0/2551413571 wait complete. 2024-04-14T08:34:47.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834efd1740 20 client.5964 trim_cache size 0 max 16384 2024-04-14T08:34:47.026 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:34:47.026 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-04-14T08:34:47.026 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "ac73733c-1cb7-478d-a7d7-5b164c5dfc09", 2024-04-14T08:34:47.026 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-04-14T08:34:47.026 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:34:47.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834c1ed640 1 -- 172.21.15.172:0/3423813546 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f832806c8f0 msgr2=0x7f832806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834c1ed640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f832806c8f0 0x7f832806edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f83441f7c30 tx=0x7f8334008040 comp rx=0 tx=0).stop 2024-04-14T08:34:47.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834c1ed640 1 -- 172.21.15.172:0/3423813546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f834409aa70 msgr2=0x7f834415eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834c1ed640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f834409aa70 0x7f834415eb80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8340034040 tx=0x7f83400046c0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834c1ed640 1 -- 172.21.15.172:0/3423813546 shutdown_connections 2024-04-14T08:34:47.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834c1ed640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f832806c8f0 0x7f832806edb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834c1ed640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f834415fa90 0x7f834415c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834c1ed640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83440d2b30 0x7f8344157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834c1ed640 1 --2- 172.21.15.172:0/3423813546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f834409aa70 0x7f834415eb80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834c1ed640 1 -- 172.21.15.172:0/3423813546 >> 172.21.15.172:0/3423813546 conn(0x7f83440d2040 msgr2=0x7f83440c9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:47.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834c1ed640 1 -- 172.21.15.172:0/3423813546 shutdown_connections 2024-04-14T08:34:47.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.018+0000 7f834c1ed640 1 -- 172.21.15.172:0/3423813546 wait complete. 2024-04-14T08:34:47.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:34:47.031 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-14T08:34:47.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.133+0000 7fc7686c0640 1 Processor -- start 2024-04-14T08:34:47.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.133+0000 7fc7686c0640 1 -- start start 2024-04-14T08:34:47.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc7686c0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc76015c420 0x7fc7600b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc7686c0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76006f3e0 0x7fc7600d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc7686c0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7600d0550 con 0x7fc7600ceb00 2024-04-14T08:34:47.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc7686c0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc7600d30a0 con 0x7fc76015c420 2024-04-14T08:34:47.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc7686c0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc76006f870 con 0x7fc76006f3e0 2024-04-14T08:34:47.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc76645c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc76015c420 0x7fc7600b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc76645c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc76015c420 0x7fc7600b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45998/0 (socket says 172.21.15.172:45998) 2024-04-14T08:34:47.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc76645c640 1 -- 172.21.15.172:0/3411030846 learned_addr learned my addr 172.21.15.172:0/3411030846 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:47.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc765c5b640 1 --2- 172.21.15.172:0/3411030846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76006f3e0 0x7fc7600d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc76645c640 1 -- 172.21.15.172:0/3411030846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76006f3e0 msgr2=0x7fc7600d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc76645c640 1 --2- 172.21.15.172:0/3411030846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76006f3e0 0x7fc7600d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc76645c640 1 -- 172.21.15.172:0/3411030846 >> v1:172.21.15.177:6789/0 conn(0x7fc7600ceb00 legacy=0x7fc7600cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.135+0000 7fc76645c640 1 -- 172.21.15.172:0/3411030846 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7600ccdd0 con 0x7fc76015c420 2024-04-14T08:34:47.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.136+0000 7fc76645c640 1 --2- 172.21.15.172:0/3411030846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc76015c420 0x7fc7600b1510 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc758009870 tx=0x7fc75802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1674348ff4c4c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:47.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.136+0000 7fc76545a640 1 -- 172.21.15.172:0/3411030846 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758034070 con 0x7fc76015c420 2024-04-14T08:34:47.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.136+0000 7fc76545a640 1 -- 172.21.15.172:0/3411030846 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc758030840 con 0x7fc76015c420 2024-04-14T08:34:47.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.136+0000 7fc76545a640 1 -- 172.21.15.172:0/3411030846 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758030c10 con 0x7fc76015c420 2024-04-14T08:34:47.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.137+0000 7fc7686c0640 1 -- 172.21.15.172:0/3411030846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc76015c420 msgr2=0x7fc7600b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.137+0000 7fc7686c0640 1 --2- 172.21.15.172:0/3411030846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc76015c420 0x7fc7600b1510 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc758009870 tx=0x7fc75802b980 comp rx=0 tx=0).stop 2024-04-14T08:34:47.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.137+0000 7fc7686c0640 1 -- 172.21.15.172:0/3411030846 shutdown_connections 2024-04-14T08:34:47.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.137+0000 7fc7686c0640 1 --2- 172.21.15.172:0/3411030846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc76006f3e0 0x7fc7600d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.137+0000 7fc7686c0640 1 --2- 172.21.15.172:0/3411030846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc76015c420 0x7fc7600b1510 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.137+0000 7fc7686c0640 1 -- 172.21.15.172:0/3411030846 >> 172.21.15.172:0/3411030846 conn(0x7fc76008c000 msgr2=0x7fc7600d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:47.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.137+0000 7fc7686c0640 1 -- 172.21.15.172:0/3411030846 shutdown_connections 2024-04-14T08:34:47.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.137+0000 7fc7686c0640 1 -- 172.21.15.172:0/3411030846 wait complete. 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.138+0000 7fc7686c0640 1 Processor -- start 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.138+0000 7fc7686c0640 1 -- start start 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc7686c0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7600ceb00 0x7fc7600c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc7686c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc76015c420 0x7fc7601480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc7686c0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7600c8ac0 0x7fc7600b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc7686c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc7600cd070 con 0x7fc76015c420 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc7686c0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc7600d2dc0 con 0x7fc7600c8ac0 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc7686c0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc7600b11e0 con 0x7fc7600ceb00 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc765c5b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc76015c420 0x7fc7601480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc765c5b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc76015c420 0x7fc7601480c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46006/0 (socket says 172.21.15.172:46006) 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc765c5b640 1 -- 172.21.15.172:0/4027379747 learned_addr learned my addr 172.21.15.172:0/4027379747 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc76645c640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7600ceb00 0x7fc7600c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc765c5b640 1 -- 172.21.15.172:0/4027379747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7600ceb00 msgr2=0x7fc7600c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc765c5b640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7600ceb00 0x7fc7600c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc765c5b640 1 -- 172.21.15.172:0/4027379747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7600c8ac0 msgr2=0x7fc7600b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc766c5d640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7600c8ac0 0x7fc7600b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc765c5b640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7600c8ac0 0x7fc7600b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc765c5b640 1 -- 172.21.15.172:0/4027379747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7600c8f70 con 0x7fc76015c420 2024-04-14T08:34:47.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc766c5d640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7600c8ac0 0x7fc7600b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:47.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.139+0000 7fc765c5b640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc76015c420 0x7fc7601480c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc75000e9e0 tx=0x7fc75000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:47.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.140+0000 7fc756ffd640 1 -- 172.21.15.172:0/4027379747 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750017070 con 0x7fc76015c420 2024-04-14T08:34:47.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.140+0000 7fc756ffd640 1 -- 172.21.15.172:0/4027379747 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75000c9d0 con 0x7fc76015c420 2024-04-14T08:34:47.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.140+0000 7fc756ffd640 1 -- 172.21.15.172:0/4027379747 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75000cdc0 con 0x7fc76015c420 2024-04-14T08:34:47.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.140+0000 7fc7686c0640 1 -- 172.21.15.172:0/4027379747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7600c6090 con 0x7fc76015c420 2024-04-14T08:34:47.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.140+0000 7fc7686c0640 1 -- 172.21.15.172:0/4027379747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc760148980 con 0x7fc76015c420 2024-04-14T08:34:47.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.141+0000 7fc756ffd640 1 -- 172.21.15.172:0/4027379747 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc75001a070 con 0x7fc76015c420 2024-04-14T08:34:47.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.142+0000 7fc756ffd640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc74406c8f0 0x7fc74406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.142+0000 7fc756ffd640 1 -- 172.21.15.172:0/4027379747 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fc75008bc30 con 0x7fc76015c420 2024-04-14T08:34:47.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.142+0000 7fc76645c640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc74406c8f0 0x7fc74406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.142+0000 7fc76645c640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc74406c8f0 0x7fc74406edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc758036000 tx=0x7fc75803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:47.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.143+0000 7fc76b580740 1 Processor -- start 2024-04-14T08:34:47.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.143+0000 7fc76b580740 1 -- start start 2024-04-14T08:34:47.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc76b580740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542970 0x559403515910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc76b580740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542d50 0x559403545700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc76b580740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403515e50 0x559403545c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc76b580740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55940347f800 con 0x559403515e50 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc76b580740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5594034277f0 con 0x559403542d50 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc76b580740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55940347b7b0 con 0x559403542970 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc766c5d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403515e50 0x559403545c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc766c5d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403515e50 0x559403545c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46008/0 (socket says 172.21.15.172:46008) 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc766c5d640 1 -- 172.21.15.172:0/3728368605 learned_addr learned my addr 172.21.15.172:0/3728368605 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc76645c640 1 --2- 172.21.15.172:0/3728368605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542970 0x559403515910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc765c5b640 1 --2- 172.21.15.172:0/3728368605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542d50 0x559403545700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc766c5d640 1 -- 172.21.15.172:0/3728368605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542970 msgr2=0x559403515910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc766c5d640 1 --2- 172.21.15.172:0/3728368605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542970 0x559403515910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc766c5d640 1 -- 172.21.15.172:0/3728368605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542d50 msgr2=0x559403545700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc766c5d640 1 --2- 172.21.15.172:0/3728368605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542d50 0x559403545700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc766c5d640 1 -- 172.21.15.172:0/3728368605 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55940325d7d0 con 0x559403515e50 2024-04-14T08:34:47.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc766c5d640 1 --2- 172.21.15.172:0/3728368605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403515e50 0x559403545c40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc75c001e70 tx=0x7fc75c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952bfc5c20dc6b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc743fff640 1 -- 172.21.15.172:0/3728368605 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c018070 con 0x559403515e50 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc743fff640 1 -- 172.21.15.172:0/3728368605 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c003b90 con 0x559403515e50 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.144+0000 7fc743fff640 1 -- 172.21.15.172:0/3728368605 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c00de40 con 0x559403515e50 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.145+0000 7fc76b580740 1 -- 172.21.15.172:0/3728368605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403515e50 msgr2=0x559403545c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.145+0000 7fc76b580740 1 --2- 172.21.15.172:0/3728368605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403515e50 0x559403545c40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc75c001e70 tx=0x7fc75c00d520 comp rx=0 tx=0).stop 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.145+0000 7fc76b580740 1 -- 172.21.15.172:0/3728368605 shutdown_connections 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.145+0000 7fc76b580740 1 --2- 172.21.15.172:0/3728368605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403515e50 0x559403545c40 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.145+0000 7fc76b580740 1 --2- 172.21.15.172:0/3728368605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542d50 0x559403545700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.145+0000 7fc76b580740 1 --2- 172.21.15.172:0/3728368605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542970 0x559403515910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.145+0000 7fc76b580740 1 -- 172.21.15.172:0/3728368605 >> 172.21.15.172:0/3728368605 conn(0x559403517b80 msgr2=0x559403560700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.145+0000 7fc76b580740 1 -- 172.21.15.172:0/3728368605 shutdown_connections 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.145+0000 7fc76b580740 1 -- 172.21.15.172:0/3728368605 wait complete. 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.146+0000 7fc76b580740 1 -- start start 2024-04-14T08:34:47.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.146+0000 7fc76b580740 1 Processor -- start 2024-04-14T08:34:47.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc76b580740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542970 0x5594035474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc76b580740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403542d50 0x559403548df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc76b580740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55940353afc0 0x559403549330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc76b580740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55940351a120 con 0x559403542d50 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc76b580740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55940343ad00 con 0x559403542970 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc76b580740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x559403519510 con 0x55940353afc0 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc76b580740 10 client.0 _ll_register_callbacks cb 0x55940343a6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc765c5b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403542d50 0x559403548df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc765c5b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403542d50 0x559403548df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46016/0 (socket says 172.21.15.172:46016) 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc765c5b640 1 -- 172.21.15.172:0/1570268020 learned_addr learned my addr 172.21.15.172:0/1570268020 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc76645c640 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542970 0x5594035474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc766c5d640 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55940353afc0 0x559403549330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc765c5b640 1 -- 172.21.15.172:0/1570268020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55940353afc0 msgr2=0x559403549330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc765c5b640 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55940353afc0 0x559403549330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc765c5b640 1 -- 172.21.15.172:0/1570268020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542970 msgr2=0x5594035474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc765c5b640 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542970 0x5594035474c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc765c5b640 1 -- 172.21.15.172:0/1570268020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594031933f0 con 0x559403542d50 2024-04-14T08:34:47.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc765c5b640 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403542d50 0x559403548df0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc7500597f0 tx=0x7fc750059920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad7da27b70af888 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:47.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.147+0000 7fc7417fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:34:47.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc7417fa640 1 -- 172.21.15.172:0/1570268020 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750088060 con 0x559403542d50 2024-04-14T08:34:47.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc7417fa640 1 -- 172.21.15.172:0/1570268020 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fc75005ab90 con 0x559403542d50 2024-04-14T08:34:47.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc76b580740 1 -- 172.21.15.172:0/1570268020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75c003b90 con 0x559403542d50 2024-04-14T08:34:47.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc7417fa640 1 -- 172.21.15.172:0/1570268020 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc750021070 con 0x559403542d50 2024-04-14T08:34:47.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc76b580740 10 client.5976 fetch_fsmap learned FSMap version 23 2024-04-14T08:34:47.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc76b580740 1 -- 172.21.15.172:0/1570268020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5594031530f0 con 0x559403542d50 2024-04-14T08:34:47.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc7417fa640 1 -- 172.21.15.172:0/1570268020 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75007a080 con 0x559403542d50 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc7417fa640 1 -- 172.21.15.172:0/1570268020 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fc750024060 con 0x559403542d50 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc76b580740 10 client.5976 fetch_fsmap finished waiting for FSMap version 23 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc76b580740 10 client.5976 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc76b580740 20 client.5976 populate_metadata read hostname 'smithi172' 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc76b580740 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x559403544490 0x559403544890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc76b580740 4 client.5976 mds_command: new command op to 5745 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc76b580740 1 -- 172.21.15.172:0/1570268020 --> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55940348cd40 con 0x559403544490 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc76645c640 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x559403544490 0x559403544890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc7417fa640 10 client.5976 ms_handle_connect on v2:172.21.15.172:6835/1527574817 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.148+0000 7fc76645c640 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x559403544490 0x559403544890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d395f212e5e1b9 server_cookie=8599c2b14d152ebf in_seq=0 out_seq=0 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.149+0000 7fc7417fa640 1 -- 172.21.15.172:0/1570268020 <== mds.0 v2:172.21.15.172:6835/1527574817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55940348cd40 con 0x559403544490 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.149+0000 7fc7417fa640 10 client.5976 handle_command_reply: tid=0 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.149+0000 7fc76b580740 1 client.5976 shutdown 2024-04-14T08:34:47.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.149+0000 7fc76b580740 1 -- 172.21.15.172:0/1570268020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403542d50 msgr2=0x559403548df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.149+0000 7fc76b580740 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403542d50 0x559403548df0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc7500597f0 tx=0x7fc750059920 comp rx=0 tx=0).stop 2024-04-14T08:34:47.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.150+0000 7fc76b580740 1 -- 172.21.15.172:0/1570268020 shutdown_connections 2024-04-14T08:34:47.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.150+0000 7fc76b580740 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x559403544490 0x559403544890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.150+0000 7fc76b580740 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55940353afc0 0x559403549330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.150+0000 7fc76b580740 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559403542d50 0x559403548df0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.150+0000 7fc7417fa640 0 client.5976 ms_handle_reset on v2:172.21.15.172:6835/1527574817 2024-04-14T08:34:47.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.150+0000 7fc76b580740 1 --2- 172.21.15.172:0/1570268020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542970 0x5594035474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.150+0000 7fc76b580740 1 -- 172.21.15.172:0/1570268020 >> 172.21.15.172:0/1570268020 conn(0x559403515e50 msgr2=0x559403563c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:47.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.150+0000 7fc76b580740 1 -- 172.21.15.172:0/1570268020 shutdown_connections 2024-04-14T08:34:47.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.150+0000 7fc76b580740 1 -- 172.21.15.172:0/1570268020 wait complete. 2024-04-14T08:34:47.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.150+0000 7fc76b580740 20 client.5976 trim_cache size 0 max 16384 2024-04-14T08:34:47.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.167+0000 7fc76b580740 1 Processor -- start 2024-04-14T08:34:47.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.167+0000 7fc76b580740 1 -- start start 2024-04-14T08:34:47.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76b580740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 0x559403563e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76b580740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542970 0x5594035600e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76b580740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542d50 0x559403560620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76b580740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc7601ff120 con 0x55940353afc0 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76b580740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc7601fe180 con 0x559403542d50 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76b580740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc760149f90 con 0x559403542970 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76645c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 0x559403563e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76645c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 0x559403563e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46024/0 (socket says 172.21.15.172:46024) 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76645c640 1 -- 172.21.15.172:0/1446214783 learned_addr learned my addr 172.21.15.172:0/1446214783 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76645c640 1 -- 172.21.15.172:0/1446214783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542970 msgr2=0x5594035600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76645c640 1 --2- 172.21.15.172:0/1446214783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542970 0x5594035600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76645c640 1 -- 172.21.15.172:0/1446214783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542d50 msgr2=0x559403560620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc766c5d640 1 --2- 172.21.15.172:0/1446214783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542d50 0x559403560620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76645c640 1 --2- 172.21.15.172:0/1446214783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542d50 0x559403560620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76645c640 1 -- 172.21.15.172:0/1446214783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594035118a0 con 0x55940353afc0 2024-04-14T08:34:47.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc766c5d640 1 --2- 172.21.15.172:0/1446214783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542d50 0x559403560620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:47.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.168+0000 7fc76645c640 1 --2- 172.21.15.172:0/1446214783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 0x559403563e00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc7580090b0 tx=0x7fc758030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bae41e5cc193389 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:47.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.169+0000 7fc7437fe640 1 -- 172.21.15.172:0/1446214783 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758034040 con 0x55940353afc0 2024-04-14T08:34:47.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.169+0000 7fc7437fe640 1 -- 172.21.15.172:0/1446214783 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc758030cf0 con 0x55940353afc0 2024-04-14T08:34:47.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.169+0000 7fc7437fe640 1 -- 172.21.15.172:0/1446214783 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758003d00 con 0x55940353afc0 2024-04-14T08:34:47.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.169+0000 7fc76b580740 1 -- 172.21.15.172:0/1446214783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 msgr2=0x559403563e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.169+0000 7fc76b580740 1 --2- 172.21.15.172:0/1446214783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 0x559403563e00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc7580090b0 tx=0x7fc758030000 comp rx=0 tx=0).stop 2024-04-14T08:34:47.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.169+0000 7fc76b580740 1 -- 172.21.15.172:0/1446214783 shutdown_connections 2024-04-14T08:34:47.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.169+0000 7fc76b580740 1 --2- 172.21.15.172:0/1446214783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542d50 0x559403560620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.169+0000 7fc76b580740 1 --2- 172.21.15.172:0/1446214783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542970 0x5594035600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.169+0000 7fc76b580740 1 --2- 172.21.15.172:0/1446214783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 0x559403563e00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.169+0000 7fc76b580740 1 -- 172.21.15.172:0/1446214783 >> 172.21.15.172:0/1446214783 conn(0x559403515e50 msgr2=0x55940353b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:47.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.169+0000 7fc76b580740 1 -- 172.21.15.172:0/1446214783 shutdown_connections 2024-04-14T08:34:47.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.169+0000 7fc76b580740 1 -- 172.21.15.172:0/1446214783 wait complete. 2024-04-14T08:34:47.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.170+0000 7fc76b580740 1 -- start start 2024-04-14T08:34:47.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.170+0000 7fc76b580740 1 Processor -- start 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.171+0000 7fc76b580740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 0x5594035de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.171+0000 7fc76b580740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542970 0x55940351b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76b580740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542d50 0x559403544870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76b580740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594034059d0 con 0x55940353afc0 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76b580740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc7601fdbb0 con 0x559403542970 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76b580740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x559403564b20 con 0x559403542d50 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76b580740 10 client.0 _ll_register_callbacks cb 0x7fc750059f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76645c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 0x5594035de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76645c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 0x5594035de1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46038/0 (socket says 172.21.15.172:46038) 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76645c640 1 -- 172.21.15.172:0/3303480829 learned_addr learned my addr 172.21.15.172:0/3303480829 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc765c5b640 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542970 0x55940351b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc766c5d640 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542d50 0x559403544870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76645c640 1 -- 172.21.15.172:0/3303480829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542d50 msgr2=0x559403544870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76645c640 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542d50 0x559403544870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76645c640 1 -- 172.21.15.172:0/3303480829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542970 msgr2=0x55940351b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76645c640 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542970 0x55940351b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76645c640 1 -- 172.21.15.172:0/3303480829 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55940353c620 con 0x55940353afc0 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc765c5b640 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542970 0x55940351b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76645c640 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 0x5594035de1c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc758030fa0 tx=0x7fc758004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d84ea156c3c6a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc7417fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc7417fa640 1 -- 172.21.15.172:0/3303480829 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758056070 con 0x55940353afc0 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc7417fa640 1 -- 172.21.15.172:0/3303480829 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fc75805bc40 con 0x55940353afc0 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76b580740 1 -- 172.21.15.172:0/3303480829 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc758030cf0 con 0x55940353afc0 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc7417fa640 1 -- 172.21.15.172:0/3303480829 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc758034070 con 0x55940353afc0 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76b580740 10 client.5982 fetch_fsmap learned FSMap version 23 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.172+0000 7fc76b580740 1 -- 172.21.15.172:0/3303480829 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x559403584430 con 0x55940353afc0 2024-04-14T08:34:47.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc7417fa640 1 -- 172.21.15.172:0/3303480829 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc758030cf0 con 0x55940353afc0 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc7417fa640 1 -- 172.21.15.172:0/3303480829 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fc758052110 con 0x55940353afc0 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc76b580740 10 client.5982 fetch_fsmap finished waiting for FSMap version 23 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc76b580740 10 client.5982 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc76b580740 20 client.5982 populate_metadata read hostname 'smithi172' 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc76b580740 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x559403544db0 0x5594035600e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc76b580740 4 client.5982 mds_command: new command op to 5745 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc76b580740 1 -- 172.21.15.172:0/3303480829 --> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559403487090 con 0x559403544db0 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc765c5b640 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x559403544db0 0x5594035600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc7417fa640 10 client.5982 ms_handle_connect on v2:172.21.15.172:6835/1527574817 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc765c5b640 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x559403544db0 0x5594035600e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99c7a21e1bace19a server_cookie=fffdfaf7f6db4c68 in_seq=0 out_seq=0 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc7417fa640 1 -- 172.21.15.172:0/3303480829 <== mds.0 v2:172.21.15.172:6835/1527574817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x559403487090 con 0x559403544db0 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc7417fa640 10 client.5982 handle_command_reply: tid=0 2024-04-14T08:34:47.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc76b580740 1 client.5982 shutdown 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc76b580740 1 -- 172.21.15.172:0/3303480829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 msgr2=0x5594035de1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.173+0000 7fc76b580740 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 0x5594035de1c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc758030fa0 tx=0x7fc758004730 comp rx=0 tx=0).stop 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc76b580740 1 -- 172.21.15.172:0/3303480829 shutdown_connections 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc76b580740 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x559403544db0 0x5594035600e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc76b580740 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559403542d50 0x559403544870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc76b580740 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559403542970 0x55940351b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7417fa640 0 client.5982 ms_handle_reset on v2:172.21.15.172:6835/1527574817 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc76b580740 1 --2- 172.21.15.172:0/3303480829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55940353afc0 0x5594035de1c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc76b580740 1 -- 172.21.15.172:0/3303480829 >> 172.21.15.172:0/3303480829 conn(0x559403515e50 msgr2=0x559403563e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc76b580740 1 -- 172.21.15.172:0/3303480829 shutdown_connections 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc76b580740 1 -- 172.21.15.172:0/3303480829 wait complete. 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc76b580740 20 client.5982 trim_cache size 0 max 16384 2024-04-14T08:34:47.181 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:34:47.182 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-14T08:34:47.182 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-04-14T08:34:47.182 INFO:teuthology.orchestra.run.smithi172.stdout: "ac73733c-1cb7-478d-a7d7-5b164c5dfc09": { 2024-04-14T08:34:47.182 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-04-14T08:34:47.182 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "ac73733c-1cb7-478d-a7d7-5b164c5dfc09", 2024-04-14T08:34:47.182 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,repair" 2024-04-14T08:34:47.182 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:47.182 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:34:47.182 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:34:47.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7686c0640 1 -- 172.21.15.172:0/4027379747 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc74406c8f0 msgr2=0x7fc74406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7686c0640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc74406c8f0 0x7fc74406edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc758036000 tx=0x7fc75803b040 comp rx=0 tx=0).stop 2024-04-14T08:34:47.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7686c0640 1 -- 172.21.15.172:0/4027379747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc76015c420 msgr2=0x7fc7601480c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:34:47.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7686c0640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc76015c420 0x7fc7601480c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc75000e9e0 tx=0x7fc75000eea0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7686c0640 1 -- 172.21.15.172:0/4027379747 shutdown_connections 2024-04-14T08:34:47.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7686c0640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc74406c8f0 0x7fc74406edb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7686c0640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7600c8ac0 0x7fc7600b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7686c0640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc76015c420 0x7fc7601480c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7686c0640 1 --2- 172.21.15.172:0/4027379747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7600ceb00 0x7fc7600c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:34:47.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7686c0640 1 -- 172.21.15.172:0/4027379747 >> 172.21.15.172:0/4027379747 conn(0x7fc76008c000 msgr2=0x7fc7600b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:34:47.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7686c0640 1 -- 172.21.15.172:0/4027379747 shutdown_connections 2024-04-14T08:34:47.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.174+0000 7fc7686c0640 1 -- 172.21.15.172:0/4027379747 wait complete. 2024-04-14T08:34:47.187 INFO:tasks.cephfs.filesystem:scrub status for tag:ac73733c-1cb7-478d-a7d7-5b164c5dfc09 - {'path': '/', 'tag': 'ac73733c-1cb7-478d-a7d7-5b164c5dfc09', 'options': 'recursive,repair'} 2024-04-14T08:34:47.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:47.458+0000 7fc0667fc640 1 -- 172.21.15.172:0/3365146098 <== mon.0 v2:172.21.15.172:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-14T08:34:47.018687+0000) ==== 1078+0+0 (secure 0 0 0) 0x7fc0680acea0 con 0x7fc0700d0c20 2024-04-14T08:34:47.461 INFO:teuthology.orchestra.run.smithi172.stdout: cluster: 2024-04-14T08:34:47.461 INFO:teuthology.orchestra.run.smithi172.stdout: id: 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:34:47.461 INFO:teuthology.orchestra.run.smithi172.stdout: health: HEALTH_OK 2024-04-14T08:34:47.461 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:47.461 INFO:teuthology.orchestra.run.smithi172.stdout: services: 2024-04-14T08:34:47.461 INFO:teuthology.orchestra.run.smithi172.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-14T08:34:47.461 INFO:teuthology.orchestra.run.smithi172.stdout: mgr: y(active, since 2m), standbys: x 2024-04-14T08:34:47.461 INFO:teuthology.orchestra.run.smithi172.stdout: mds: 1/1 daemons up, 3 standby 2024-04-14T08:34:47.461 INFO:teuthology.orchestra.run.smithi172.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-14T08:34:47.461 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:47.461 INFO:teuthology.orchestra.run.smithi172.stdout: data: 2024-04-14T08:34:47.461 INFO:teuthology.orchestra.run.smithi172.stdout: volumes: 1/1 healthy 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout: pools: 3 pools, 97 pgs 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout: objects: 24 objects, 579 KiB 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout: usage: 217 MiB used, 720 GiB / 720 GiB avail 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout: pgs: 97 active+clean 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout: io: 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout: client: 1.4 KiB/s wr, 0 op/s rd, 4 op/s wr 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:34:47.018687+0000 mds.c [INF] scrub queued for path: / 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:34:47.018690+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:34:47.018695+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:34:47.019461+0000 mds.c [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-14T08:34:47.462 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:34:47.019473+0000 mds.c [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-14T08:34:47.463 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:34:47.019926+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-14T08:34:51.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:34:51.466+0000 7fc0667fc640 1 -- 172.21.15.172:0/3365146098 <== mon.0 v2:172.21.15.172:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-14T08:34:50.387737+0000) ==== 189+0+0 (secure 0 0 0) 0x7fc0680af000 con 0x7fc0700d0c20 2024-04-14T08:34:51.468 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:34:50.387737+0000 mds.c [INF] scrub summary: idle 2024-04-14T08:34:53.162 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:34:53.165 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:35:17.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:17.189 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-14T08:35:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.293+0000 7f551b636640 1 Processor -- start 2024-04-14T08:35:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.294+0000 7f551b636640 1 -- start start 2024-04-14T08:35:17.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.294+0000 7f551b636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551415e430 0x7f5514162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.294+0000 7f551b636640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5514162540 0x7f55141629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.294+0000 7f551b636640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f55140b3a30 0x7f55140b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.294+0000 7f551b636640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f55140cefb0 con 0x7f55140b3a30 2024-04-14T08:35:17.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.294+0000 7f551b636640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f55140cf290 con 0x7f5514162540 2024-04-14T08:35:17.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.294+0000 7f551b636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f55140cf570 con 0x7f551415e430 2024-04-14T08:35:17.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.294+0000 7f5519bd3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f55140b3a30 0x7f55140b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.295+0000 7f5519bd3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f55140b3a30 0x7f55140b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33966/0 (socket says 172.21.15.172:33966) 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.295+0000 7f5519bd3640 1 -- 172.21.15.172:0/1951384958 learned_addr learned my addr 172.21.15.172:0/1951384958 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.295+0000 7f5518bd1640 1 --2- 172.21.15.172:0/1951384958 >> v2:172.21.15.177:3300/0 conn(0x7f5514162540 0x7f55141629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.295+0000 7f55193d2640 1 --2- 172.21.15.172:0/1951384958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551415e430 0x7f5514162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.295+0000 7f5519bd3640 1 -- 172.21.15.172:0/1951384958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551415e430 msgr2=0x7f5514162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.295+0000 7f5519bd3640 1 --2- 172.21.15.172:0/1951384958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551415e430 0x7f5514162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.295+0000 7f5519bd3640 1 -- 172.21.15.172:0/1951384958 >> v2:172.21.15.177:3300/0 conn(0x7f5514162540 msgr2=0x7f55141629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.295+0000 7f5519bd3640 1 --2- 172.21.15.172:0/1951384958 >> v2:172.21.15.177:3300/0 conn(0x7f5514162540 0x7f55141629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.295+0000 7f5519bd3640 1 -- 172.21.15.172:0/1951384958 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55140cf850 con 0x7f55140b3a30 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.295+0000 7f5518bd1640 1 --2- 172.21.15.172:0/1951384958 >> v2:172.21.15.177:3300/0 conn(0x7f5514162540 0x7f55141629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.295+0000 7f55193d2640 1 --2- 172.21.15.172:0/1951384958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551415e430 0x7f5514162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f5519bd3640 1 --2- 172.21.15.172:0/1951384958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55140b3a30 0x7f55140b3e60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f551408c720 tx=0x7f550402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5e22e59f159d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f5513fff640 1 -- 172.21.15.172:0/1951384958 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550402ca70 con 0x7f55140b3a30 2024-04-14T08:35:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f5513fff640 1 -- 172.21.15.172:0/1951384958 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f550402cc10 con 0x7f55140b3a30 2024-04-14T08:35:17.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f5513fff640 1 -- 172.21.15.172:0/1951384958 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55040343f0 con 0x7f55140b3a30 2024-04-14T08:35:17.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f551b636640 1 -- 172.21.15.172:0/1951384958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55140b3a30 msgr2=0x7f55140b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f551b636640 1 --2- 172.21.15.172:0/1951384958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55140b3a30 0x7f55140b3e60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f551408c720 tx=0x7f550402bed0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f551b636640 1 -- 172.21.15.172:0/1951384958 shutdown_connections 2024-04-14T08:35:17.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f551b636640 1 --2- 172.21.15.172:0/1951384958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55140b3a30 0x7f55140b3e60 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f551b636640 1 --2- 172.21.15.172:0/1951384958 >> v2:172.21.15.177:3300/0 conn(0x7f5514162540 0x7f55141629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f551b636640 1 --2- 172.21.15.172:0/1951384958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f551415e430 0x7f5514162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f551b636640 1 -- 172.21.15.172:0/1951384958 >> 172.21.15.172:0/1951384958 conn(0x7f551409d560 msgr2=0x7f55140d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:17.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f551b636640 1 -- 172.21.15.172:0/1951384958 shutdown_connections 2024-04-14T08:35:17.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.296+0000 7f551b636640 1 -- 172.21.15.172:0/1951384958 wait complete. 2024-04-14T08:35:17.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.297+0000 7f551b636640 1 Processor -- start 2024-04-14T08:35:17.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.298+0000 7f551b636640 1 -- start start 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.298+0000 7f551b636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55140b3a30 0x7f55141f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.298+0000 7f551b636640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f551415e430 0x7f55140cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.298+0000 7f551b636640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5514162540 0x7f55140cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.298+0000 7f551b636640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f55140d4e20 con 0x7f551415e430 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.298+0000 7f551b636640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f551415e810 con 0x7f5514162540 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.298+0000 7f551b636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f55140d0050 con 0x7f55140b3a30 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.298+0000 7f5518bd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f551415e430 0x7f55140cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.298+0000 7f5518bd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f551415e430 0x7f55140cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33970/0 (socket says 172.21.15.172:33970) 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.298+0000 7f5518bd1640 1 -- 172.21.15.172:0/1394394242 learned_addr learned my addr 172.21.15.172:0/1394394242 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f5519bd3640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5514162540 0x7f55140cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f55193d2640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55140b3a30 0x7f55141f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f5518bd1640 1 -- 172.21.15.172:0/1394394242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55140b3a30 msgr2=0x7f55141f7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f5518bd1640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55140b3a30 0x7f55141f7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f5518bd1640 1 -- 172.21.15.172:0/1394394242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5514162540 msgr2=0x7f55140cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f5518bd1640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5514162540 0x7f55140cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f5518bd1640 1 -- 172.21.15.172:0/1394394242 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55141f84f0 con 0x7f551415e430 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f55193d2640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55140b3a30 0x7f55141f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f5518bd1640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f551415e430 0x7f55140cbe00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f550800e7e0 tx=0x7f550800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f5511ffb640 1 -- 172.21.15.172:0/1394394242 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550800f040 con 0x7f551415e430 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f5511ffb640 1 -- 172.21.15.172:0/1394394242 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5508004030 con 0x7f551415e430 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f5511ffb640 1 -- 172.21.15.172:0/1394394242 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508004400 con 0x7f551415e430 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f551b636640 1 -- 172.21.15.172:0/1394394242 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55140cc340 con 0x7f551415e430 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.299+0000 7f551b636640 1 -- 172.21.15.172:0/1394394242 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55141f46a0 con 0x7f551415e430 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.301+0000 7f5511ffb640 1 -- 172.21.15.172:0/1394394242 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f550800c6d0 con 0x7f551415e430 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.301+0000 7f5511ffb640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f550006c8f0 0x7f550006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.301+0000 7f5511ffb640 1 -- 172.21.15.172:0/1394394242 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f550808b9e0 con 0x7f551415e430 2024-04-14T08:35:17.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.301+0000 7f55193d2640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f550006c8f0 0x7f550006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.302+0000 7f55193d2640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f550006c8f0 0x7f550006edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f550c006fd0 tx=0x7f550c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.302+0000 7f551e439740 1 Processor -- start 2024-04-14T08:35:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.303+0000 7f551e439740 1 -- start start 2024-04-14T08:35:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f551e439740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056b970 0x55a94053e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f551e439740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 0x55a94056e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f551e439740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94053ee50 0x55a94056ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f551e439740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a9404a8800 con 0x55a94056bd50 2024-04-14T08:35:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f551e439740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a9404507f0 con 0x55a94053ee50 2024-04-14T08:35:17.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f551e439740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55a9404a47b0 con 0x55a94056b970 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f5518bd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 0x55a94056e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f5518bd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 0x55a94056e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33980/0 (socket says 172.21.15.172:33980) 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f5518bd1640 1 -- 172.21.15.172:0/1343289154 learned_addr learned my addr 172.21.15.172:0/1343289154 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f55193d2640 1 --2- 172.21.15.172:0/1343289154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056b970 0x55a94053e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f5519bd3640 1 --2- 172.21.15.172:0/1343289154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94053ee50 0x55a94056ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f5518bd1640 1 -- 172.21.15.172:0/1343289154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056b970 msgr2=0x55a94053e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f5518bd1640 1 --2- 172.21.15.172:0/1343289154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056b970 0x55a94053e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f5518bd1640 1 -- 172.21.15.172:0/1343289154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94053ee50 msgr2=0x55a94056ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f5518bd1640 1 --2- 172.21.15.172:0/1343289154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94053ee50 0x55a94056ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f5518bd1640 1 -- 172.21.15.172:0/1343289154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9402867d0 con 0x55a94056bd50 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f5518bd1640 1 --2- 172.21.15.172:0/1343289154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 0x55a94056e700 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5508089d80 tx=0x7f5508060050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e3e01eb99005b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f54faffd640 1 -- 172.21.15.172:0/1343289154 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508088070 con 0x55a94056bd50 2024-04-14T08:35:17.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f54faffd640 1 -- 172.21.15.172:0/1343289154 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5508057910 con 0x55a94056bd50 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.304+0000 7f54faffd640 1 -- 172.21.15.172:0/1343289154 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508020040 con 0x55a94056bd50 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.305+0000 7f551e439740 1 -- 172.21.15.172:0/1343289154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 msgr2=0x55a94056e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.305+0000 7f551e439740 1 --2- 172.21.15.172:0/1343289154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 0x55a94056e700 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5508089d80 tx=0x7f5508060050 comp rx=0 tx=0).stop 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.305+0000 7f551e439740 1 -- 172.21.15.172:0/1343289154 shutdown_connections 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.305+0000 7f551e439740 1 --2- 172.21.15.172:0/1343289154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94053ee50 0x55a94056ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.305+0000 7f551e439740 1 --2- 172.21.15.172:0/1343289154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 0x55a94056e700 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.305+0000 7f551e439740 1 --2- 172.21.15.172:0/1343289154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056b970 0x55a94053e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.305+0000 7f551e439740 1 -- 172.21.15.172:0/1343289154 >> 172.21.15.172:0/1343289154 conn(0x55a940540b80 msgr2=0x55a940589700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.305+0000 7f551e439740 1 -- 172.21.15.172:0/1343289154 shutdown_connections 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.305+0000 7f551e439740 1 -- 172.21.15.172:0/1343289154 wait complete. 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.306+0000 7f551e439740 1 -- start start 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.306+0000 7f551e439740 1 Processor -- start 2024-04-14T08:35:17.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f551e439740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94056b970 0x55a94055cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f551e439740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056bd50 0x55a940571400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f551e439740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 0x55a940587150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f551e439740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a940543120 con 0x55a940571940 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f551e439740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a940463d00 con 0x55a94056b970 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f551e439740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55a940542510 con 0x55a94056bd50 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f551e439740 10 client.0 _ll_register_callbacks cb 0x55a9404636f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f5519bd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 0x55a940587150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f5519bd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 0x55a940587150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33982/0 (socket says 172.21.15.172:33982) 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f5519bd3640 1 -- 172.21.15.172:0/1594394167 learned_addr learned my addr 172.21.15.172:0/1594394167 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f55193d2640 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94056b970 0x55a94055cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f5518bd1640 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056bd50 0x55a940571400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f5519bd3640 1 -- 172.21.15.172:0/1594394167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056bd50 msgr2=0x55a940571400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f5519bd3640 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056bd50 0x55a940571400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f5519bd3640 1 -- 172.21.15.172:0/1594394167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94056b970 msgr2=0x55a94055cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f5519bd3640 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94056b970 0x55a94055cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f5519bd3640 1 -- 172.21.15.172:0/1594394167 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9401bc3f0 con 0x55a940571940 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f55193d2640 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94056b970 0x55a94055cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f5519bd3640 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 0x55a940587150 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5504009f40 tx=0x7f5504004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99d40b89f962c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.307+0000 7f54effff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f54effff640 1 -- 172.21.15.172:0/1594394167 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504009510 con 0x55a940571940 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f54effff640 1 -- 172.21.15.172:0/1594394167 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f5504034920 con 0x55a940571940 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f551e439740 1 -- 172.21.15.172:0/1594394167 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5508057910 con 0x55a940571940 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f54effff640 1 -- 172.21.15.172:0/1594394167 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5504034b20 con 0x55a940571940 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f551e439740 10 client.5994 fetch_fsmap learned FSMap version 23 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f551e439740 1 -- 172.21.15.172:0/1594394167 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55a94017c0f0 con 0x55a940571940 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f54effff640 1 -- 172.21.15.172:0/1594394167 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5508057910 con 0x55a940571940 2024-04-14T08:35:17.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f54effff640 1 -- 172.21.15.172:0/1594394167 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f5504052080 con 0x55a940571940 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f551e439740 10 client.5994 fetch_fsmap finished waiting for FSMap version 23 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f551e439740 10 client.5994 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f551e439740 20 client.5994 populate_metadata read hostname 'smithi172' 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f551e439740 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55a9405898d0 0x55a940589ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f551e439740 4 client.5994 mds_command: new command op to 5745 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f551e439740 1 -- 172.21.15.172:0/1594394167 --> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9404b5d40 con 0x55a9405898d0 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f55193d2640 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55a9405898d0 0x55a940589ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.308+0000 7f54effff640 10 client.5994 ms_handle_connect on v2:172.21.15.172:6835/1527574817 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f55193d2640 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55a9405898d0 0x55a940589ce0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a101c4ad722a1359 server_cookie=5f9065ea03e5dde9 in_seq=0 out_seq=0 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f54effff640 1 -- 172.21.15.172:0/1594394167 <== mds.0 v2:172.21.15.172:6835/1527574817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9404b5d40 con 0x55a9405898d0 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f54effff640 10 client.5994 handle_command_reply: tid=0 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f551e439740 1 client.5994 shutdown 2024-04-14T08:35:17.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f551e439740 1 -- 172.21.15.172:0/1594394167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 msgr2=0x55a940587150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f551e439740 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 0x55a940587150 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5504009f40 tx=0x7f5504004ab0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f551e439740 1 -- 172.21.15.172:0/1594394167 shutdown_connections 2024-04-14T08:35:17.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f551e439740 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55a9405898d0 0x55a940589ce0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f551e439740 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 0x55a940587150 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f551e439740 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056bd50 0x55a940571400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f54effff640 0 client.5994 ms_handle_reset on v2:172.21.15.172:6835/1527574817 2024-04-14T08:35:17.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f551e439740 1 --2- 172.21.15.172:0/1594394167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94056b970 0x55a94055cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.309+0000 7f551e439740 1 -- 172.21.15.172:0/1594394167 >> 172.21.15.172:0/1594394167 conn(0x55a94053ee50 msgr2=0x55a94056e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:17.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.310+0000 7f551e439740 1 -- 172.21.15.172:0/1594394167 shutdown_connections 2024-04-14T08:35:17.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.310+0000 7f551e439740 1 -- 172.21.15.172:0/1594394167 wait complete. 2024-04-14T08:35:17.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.310+0000 7f551e439740 20 client.5994 trim_cache size 0 max 16384 2024-04-14T08:35:17.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.327+0000 7f551e439740 1 Processor -- start 2024-04-14T08:35:17.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.327+0000 7f551e439740 1 -- start start 2024-04-14T08:35:17.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f551e439740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94056b970 0x55a940589ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f551e439740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056bd50 0x55a94053da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f551e439740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 0x55a94056e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f551e439740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5514201020 con 0x55a940571940 2024-04-14T08:35:17.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f551e439740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5514160980 con 0x55a94056b970 2024-04-14T08:35:17.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f551e439740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f55140c98b0 con 0x55a94056bd50 2024-04-14T08:35:17.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f5519bd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 0x55a94056e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f5519bd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 0x55a94056e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33994/0 (socket says 172.21.15.172:33994) 2024-04-14T08:35:17.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f5519bd3640 1 -- 172.21.15.172:0/356266560 learned_addr learned my addr 172.21.15.172:0/356266560 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:17.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f5519bd3640 1 -- 172.21.15.172:0/356266560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056bd50 msgr2=0x55a94053da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f5519bd3640 1 --2- 172.21.15.172:0/356266560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056bd50 0x55a94053da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f5519bd3640 1 -- 172.21.15.172:0/356266560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94056b970 msgr2=0x55a940589ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f5519bd3640 1 --2- 172.21.15.172:0/356266560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94056b970 0x55a940589ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f5519bd3640 1 -- 172.21.15.172:0/356266560 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a94053ac20 con 0x55a940571940 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f5519bd3640 1 --2- 172.21.15.172:0/356266560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 0x55a94056e8c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5504002230 tx=0x7f5504040070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850ddc6e0dd055d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f54fa7fc640 1 -- 172.21.15.172:0/356266560 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504034e60 con 0x55a940571940 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f54fa7fc640 1 -- 172.21.15.172:0/356266560 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f550402cc10 con 0x55a940571940 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.328+0000 7f54fa7fc640 1 -- 172.21.15.172:0/356266560 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504034800 con 0x55a940571940 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.329+0000 7f551e439740 1 -- 172.21.15.172:0/356266560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 msgr2=0x55a94056e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.329+0000 7f551e439740 1 --2- 172.21.15.172:0/356266560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 0x55a94056e8c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5504002230 tx=0x7f5504040070 comp rx=0 tx=0).stop 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.329+0000 7f551e439740 1 -- 172.21.15.172:0/356266560 shutdown_connections 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.329+0000 7f551e439740 1 --2- 172.21.15.172:0/356266560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a940571940 0x55a94056e8c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.329+0000 7f551e439740 1 --2- 172.21.15.172:0/356266560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056bd50 0x55a94053da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.329+0000 7f551e439740 1 --2- 172.21.15.172:0/356266560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a94056b970 0x55a940589ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.329+0000 7f551e439740 1 -- 172.21.15.172:0/356266560 >> 172.21.15.172:0/356266560 conn(0x55a94053ee50 msgr2=0x55a940588510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:17.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.329+0000 7f551e439740 1 -- 172.21.15.172:0/356266560 shutdown_connections 2024-04-14T08:35:17.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.329+0000 7f551e439740 1 -- 172.21.15.172:0/356266560 wait complete. 2024-04-14T08:35:17.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.330+0000 7f551e439740 1 -- start start 2024-04-14T08:35:17.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.330+0000 7f551e439740 1 Processor -- start 2024-04-14T08:35:17.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.331+0000 7f551e439740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056b970 0x55a940543d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.331+0000 7f551e439740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 0x55a940598ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.331+0000 7f551e439740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a940571940 0x55a94056f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.331+0000 7f551e439740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5514160330 con 0x55a94056bd50 2024-04-14T08:35:17.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.331+0000 7f551e439740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a94056ad10 con 0x55a940571940 2024-04-14T08:35:17.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.331+0000 7f551e439740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55a94056fec0 con 0x55a94056b970 2024-04-14T08:35:17.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.331+0000 7f551e439740 10 client.0 _ll_register_callbacks cb 0x7f5504009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:35:17.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.331+0000 7f5518bd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 0x55a940598ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f5518bd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 0x55a940598ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34006/0 (socket says 172.21.15.172:34006) 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f5518bd1640 1 -- 172.21.15.172:0/622127396 learned_addr learned my addr 172.21.15.172:0/622127396 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f55193d2640 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056b970 0x55a940543d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f5519bd3640 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a940571940 0x55a94056f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f5518bd1640 1 -- 172.21.15.172:0/622127396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056b970 msgr2=0x55a940543d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f5518bd1640 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056b970 0x55a940543d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f5518bd1640 1 -- 172.21.15.172:0/622127396 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a940571940 msgr2=0x55a94056f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f5518bd1640 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a940571940 0x55a94056f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f5518bd1640 1 -- 172.21.15.172:0/622127396 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a940583eb0 con 0x55a94056bd50 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f5518bd1640 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 0x55a940598ae0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5508089e60 tx=0x7f5508089ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eafb3fcc7e6ec73 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f54effff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f54effff640 1 -- 172.21.15.172:0/622127396 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508076060 con 0x55a94056bd50 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f54effff640 1 -- 172.21.15.172:0/622127396 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f550805ed50 con 0x55a94056bd50 2024-04-14T08:35:17.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f551e439740 1 -- 172.21.15.172:0/622127396 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f550402cc10 con 0x55a94056bd50 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f54effff640 1 -- 172.21.15.172:0/622127396 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5508088070 con 0x55a94056bd50 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f551e439740 10 client.6000 fetch_fsmap learned FSMap version 23 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f551e439740 1 -- 172.21.15.172:0/622127396 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55a94056fc00 con 0x55a94056bd50 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f54effff640 1 -- 172.21.15.172:0/622127396 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f550402cc10 con 0x55a94056bd50 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.332+0000 7f54effff640 1 -- 172.21.15.172:0/622127396 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7f5508012050 con 0x55a94056bd50 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 10 client.6000 fetch_fsmap finished waiting for FSMap version 23 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 10 client.6000 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 20 client.6000 populate_metadata read hostname 'smithi172' 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55a940569f70 0x55a94053da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 4 client.6000 mds_command: new command op to 5745 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 1 -- 172.21.15.172:0/622127396 --> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a9404b0090 con 0x55a940569f70 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f55193d2640 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55a940569f70 0x55a94053da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f54effff640 10 client.6000 ms_handle_connect on v2:172.21.15.172:6835/1527574817 2024-04-14T08:35:17.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f55193d2640 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55a940569f70 0x55a94053da10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ed603806da825ed server_cookie=75e26074eacdc3b in_seq=0 out_seq=0 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f54effff640 1 -- 172.21.15.172:0/622127396 <== mds.0 v2:172.21.15.172:6835/1527574817 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a9404b0090 con 0x55a940569f70 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f54effff640 10 client.6000 handle_command_reply: tid=0 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 1 client.6000 shutdown 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 1 -- 172.21.15.172:0/622127396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 msgr2=0x55a940598ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 0x55a940598ae0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5508089e60 tx=0x7f5508089ce0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 1 -- 172.21.15.172:0/622127396 shutdown_connections 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.172:6835/1527574817,v1:172.21.15.172:6837/1527574817] conn(0x55a940569f70 0x55a94053da10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a940571940 0x55a94056f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f54effff640 0 client.6000 ms_handle_reset on v2:172.21.15.172:6835/1527574817 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a94056bd50 0x55a940598ae0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 1 --2- 172.21.15.172:0/622127396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a94056b970 0x55a940543d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 1 -- 172.21.15.172:0/622127396 >> 172.21.15.172:0/622127396 conn(0x55a94053ee50 msgr2=0x55a940588570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:17.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.333+0000 7f551e439740 1 -- 172.21.15.172:0/622127396 shutdown_connections 2024-04-14T08:35:17.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551e439740 1 -- 172.21.15.172:0/622127396 wait complete. 2024-04-14T08:35:17.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551e439740 20 client.6000 trim_cache size 0 max 16384 2024-04-14T08:35:17.340 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:35:17.340 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-04-14T08:35:17.340 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-04-14T08:35:17.340 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:35:17.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551b636640 1 -- 172.21.15.172:0/1394394242 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f550006c8f0 msgr2=0x7f550006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551b636640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f550006c8f0 0x7f550006edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f550c006fd0 tx=0x7f550c008040 comp rx=0 tx=0).stop 2024-04-14T08:35:17.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551b636640 1 -- 172.21.15.172:0/1394394242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f551415e430 msgr2=0x7f55140cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551b636640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f551415e430 0x7f55140cbe00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f550800e7e0 tx=0x7f550800eca0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551b636640 1 -- 172.21.15.172:0/1394394242 shutdown_connections 2024-04-14T08:35:17.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551b636640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f550006c8f0 0x7f550006edb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551b636640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5514162540 0x7f55140cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551b636640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f551415e430 0x7f55140cbe00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551b636640 1 --2- 172.21.15.172:0/1394394242 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55140b3a30 0x7f55141f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551b636640 1 -- 172.21.15.172:0/1394394242 >> 172.21.15.172:0/1394394242 conn(0x7f551409d560 msgr2=0x7f55140d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:17.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551b636640 1 -- 172.21.15.172:0/1394394242 shutdown_connections 2024-04-14T08:35:17.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.334+0000 7f551b636640 1 -- 172.21.15.172:0/1394394242 wait complete. 2024-04-14T08:35:17.345 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-14T08:35:17.447 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:35:17.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.548+0000 7f8992715640 1 Processor -- start 2024-04-14T08:35:17.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.548+0000 7f8992715640 1 -- start start 2024-04-14T08:35:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f8992715640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f898c0d2830 0x7f898c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f8992715640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f898c15e890 0x7f898c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f8992715640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f898c0cedd0 0x7f898c1621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f8992715640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f898c0cff30 con 0x7f898c0d2830 2024-04-14T08:35:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f8992715640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f898c0d2350 con 0x7f898c0cedd0 2024-04-14T08:35:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f8992715640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f898c0d3150 con 0x7f898c15e890 2024-04-14T08:35:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f898bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f898c0d2830 0x7f898c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f898bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f898c0d2830 0x7f898c0d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34010/0 (socket says 172.21.15.172:34010) 2024-04-14T08:35:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f898bfff640 1 -- 172.21.15.172:0/1013236496 learned_addr learned my addr 172.21.15.172:0/1013236496 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f898b7fe640 1 --2- 172.21.15.172:0/1013236496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f898c15e890 0x7f898c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f898bfff640 1 -- 172.21.15.172:0/1013236496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f898c15e890 msgr2=0x7f898c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f898bfff640 1 --2- 172.21.15.172:0/1013236496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f898c15e890 0x7f898c0d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f898bfff640 1 -- 172.21.15.172:0/1013236496 >> v2:172.21.15.177:3300/0 conn(0x7f898c0cedd0 msgr2=0x7f898c1621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f898bfff640 1 --2- 172.21.15.172:0/1013236496 >> v2:172.21.15.177:3300/0 conn(0x7f898c0cedd0 0x7f898c1621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.549+0000 7f898bfff640 1 -- 172.21.15.172:0/1013236496 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c0cf320 con 0x7f898c0d2830 2024-04-14T08:35:17.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.550+0000 7f898bfff640 1 --2- 172.21.15.172:0/1013236496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f898c0d2830 0x7f898c0d2c10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f898c0d0530 tx=0x7f898002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3148a2746a739b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.550+0000 7f898affd640 1 -- 172.21.15.172:0/1013236496 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980034070 con 0x7f898c0d2830 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.550+0000 7f898affd640 1 -- 172.21.15.172:0/1013236496 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8980030860 con 0x7f898c0d2830 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.550+0000 7f898affd640 1 -- 172.21.15.172:0/1013236496 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980030c30 con 0x7f898c0d2830 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.551+0000 7f8992715640 1 -- 172.21.15.172:0/1013236496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f898c0d2830 msgr2=0x7f898c0d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.551+0000 7f8992715640 1 --2- 172.21.15.172:0/1013236496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f898c0d2830 0x7f898c0d2c10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f898c0d0530 tx=0x7f898002b9a0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.551+0000 7f8992715640 1 -- 172.21.15.172:0/1013236496 shutdown_connections 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.551+0000 7f8992715640 1 --2- 172.21.15.172:0/1013236496 >> v2:172.21.15.177:3300/0 conn(0x7f898c0cedd0 0x7f898c1621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.551+0000 7f8992715640 1 --2- 172.21.15.172:0/1013236496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f898c15e890 0x7f898c0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.551+0000 7f8992715640 1 --2- 172.21.15.172:0/1013236496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f898c0d2830 0x7f898c0d2c10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.551+0000 7f8992715640 1 -- 172.21.15.172:0/1013236496 >> 172.21.15.172:0/1013236496 conn(0x7f898c09d590 msgr2=0x7f898c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.551+0000 7f8992715640 1 -- 172.21.15.172:0/1013236496 shutdown_connections 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.551+0000 7f8992715640 1 -- 172.21.15.172:0/1013236496 wait complete. 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.552+0000 7f8992715640 1 Processor -- start 2024-04-14T08:35:17.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.552+0000 7f8992715640 1 -- start start 2024-04-14T08:35:17.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f8992715640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f898c0cedd0 0x7f898c150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f8992715640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f898c0d2830 0x7f898c1e9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f8992715640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f898c15e890 0x7f898c0b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f8992715640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f898c0cfbc0 con 0x7f898c0cedd0 2024-04-14T08:35:17.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f8992715640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f898c0cf8e0 con 0x7f898c0d2830 2024-04-14T08:35:17.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f8992715640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f898c0d1840 con 0x7f898c15e890 2024-04-14T08:35:17.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f898bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f898c0cedd0 0x7f898c150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f898bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f898c0cedd0 0x7f898c150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34026/0 (socket says 172.21.15.172:34026) 2024-04-14T08:35:17.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f898bfff640 1 -- 172.21.15.172:0/3095322783 learned_addr learned my addr 172.21.15.172:0/3095322783 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:17.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f898b7fe640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f898c0d2830 0x7f898c1e9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f8990cb2640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f898c15e890 0x7f898c0b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f898bfff640 1 -- 172.21.15.172:0/3095322783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f898c15e890 msgr2=0x7f898c0b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f898bfff640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f898c15e890 0x7f898c0b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f898bfff640 1 -- 172.21.15.172:0/3095322783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f898c0d2830 msgr2=0x7f898c1e9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f898bfff640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f898c0d2830 0x7f898c1e9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f898bfff640 1 -- 172.21.15.172:0/3095322783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c158cc0 con 0x7f898c0cedd0 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f8990cb2640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f898c15e890 0x7f898c0b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.553+0000 7f898bfff640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f898c0cedd0 0x7f898c150620 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8980000c00 tx=0x7f89800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.554+0000 7f8988ff9640 1 -- 172.21.15.172:0/3095322783 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980034040 con 0x7f898c0cedd0 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.554+0000 7f8988ff9640 1 -- 172.21.15.172:0/3095322783 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8980039d90 con 0x7f898c0cedd0 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.554+0000 7f8988ff9640 1 -- 172.21.15.172:0/3095322783 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980043630 con 0x7f898c0cedd0 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.554+0000 7f8992715640 1 -- 172.21.15.172:0/3095322783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898c158fa0 con 0x7f898c0cedd0 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.554+0000 7f8992715640 1 -- 172.21.15.172:0/3095322783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898c159560 con 0x7f898c0cedd0 2024-04-14T08:35:17.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.555+0000 7f8992715640 1 -- 172.21.15.172:0/3095322783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898c068520 con 0x7f898c0cedd0 2024-04-14T08:35:17.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.555+0000 7f8988ff9640 1 -- 172.21.15.172:0/3095322783 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f898004a060 con 0x7f898c0cedd0 2024-04-14T08:35:17.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.555+0000 7f8988ff9640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f897806c8f0 0x7f897806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.556+0000 7f8988ff9640 1 -- 172.21.15.172:0/3095322783 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f89800adf60 con 0x7f898c0cedd0 2024-04-14T08:35:17.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.556+0000 7f898b7fe640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f897806c8f0 0x7f897806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.556+0000 7f898b7fe640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f897806c8f0 0x7f897806edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f898c154400 tx=0x7f897c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.565+0000 7f8988ff9640 1 -- 172.21.15.172:0/3095322783 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f898003e070 con 0x7f898c0cedd0 2024-04-14T08:35:17.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.782+0000 7f8992715640 1 -- 172.21.15.172:0/3095322783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f898c0d2610 con 0x7f898c0cedd0 2024-04-14T08:35:17.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.782+0000 7f8988ff9640 1 -- 172.21.15.172:0/3095322783 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5187 (secure 0 0 0) 0x7f8980037360 con 0x7f898c0cedd0 2024-04-14T08:35:17.783 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:17.783 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/296861286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":296861286},{"type":"v1","addr":"172.21.15.177:6835","nonce":296861286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/642342704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":642342704},{"type":"v1","addr":"172.21.15.177:6833","nonce":642342704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3516903987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3516903987},{"type":"v1","addr":"172.21.15.172:6836","nonce":3516903987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:34:34.765057+0000","modified":"2024-04-14T08:34:36.783999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6837/1527574817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1527574817},{"type":"v1","addr":"172.21.15.172:6837","nonce":1527574817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":3}]} 2024-04-14T08:35:17.783 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 23 2024-04-14T08:35:17.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.784+0000 7f8992715640 1 -- 172.21.15.172:0/3095322783 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f897806c8f0 msgr2=0x7f897806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.784+0000 7f8992715640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f897806c8f0 0x7f897806edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f898c154400 tx=0x7f897c008040 comp rx=0 tx=0).stop 2024-04-14T08:35:17.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.784+0000 7f8992715640 1 -- 172.21.15.172:0/3095322783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f898c0cedd0 msgr2=0x7f898c150620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.784+0000 7f8992715640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f898c0cedd0 0x7f898c150620 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8980000c00 tx=0x7f89800046c0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.784+0000 7f8992715640 1 -- 172.21.15.172:0/3095322783 shutdown_connections 2024-04-14T08:35:17.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.784+0000 7f8992715640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f897806c8f0 0x7f897806edb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.784+0000 7f8992715640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f898c15e890 0x7f898c0b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.784+0000 7f8992715640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f898c0d2830 0x7f898c1e9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.784+0000 7f8992715640 1 --2- 172.21.15.172:0/3095322783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f898c0cedd0 0x7f898c150620 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.784+0000 7f8992715640 1 -- 172.21.15.172:0/3095322783 >> 172.21.15.172:0/3095322783 conn(0x7f898c09d590 msgr2=0x7f898c15c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:17.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.785+0000 7f8992715640 1 -- 172.21.15.172:0/3095322783 shutdown_connections 2024-04-14T08:35:17.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.785+0000 7f8992715640 1 -- 172.21.15.172:0/3095322783 wait complete. 2024-04-14T08:35:17.796 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-14T08:35:17.925 INFO:teuthology.orchestra.run.smithi172.stdout:{"message":"","return_code":0} 2024-04-14T08:35:17.925 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-14T08:35:17.926 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-14T08:35:17.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.967+0000 7fcba3a3bf40 1 Processor -- start 2024-04-14T08:35:17.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.967+0000 7fcba3a3bf40 1 -- start start 2024-04-14T08:35:17.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba3a3bf40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb90003b00 0x5608625ea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba3a3bf40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608625ebfe0 0x560862726e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba3a3bf40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560862699310 0x5608627273c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba3a3bf40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5608625643b0 con 0x560862699310 2024-04-14T08:35:17.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba3a3bf40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560862568400 con 0x5608625ebfe0 2024-04-14T08:35:17.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba3a3bf40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5608626072d0 con 0x7fcb90003b00 2024-04-14T08:35:17.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba124c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560862699310 0x5608627273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba124c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560862699310 0x5608627273c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34044/0 (socket says 172.21.15.172:34044) 2024-04-14T08:35:17.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba124c640 1 -- 172.21.15.172:0/4250196149 learned_addr learned my addr 172.21.15.172:0/4250196149 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:17.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcb9bfff640 1 --2- 172.21.15.172:0/4250196149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608625ebfe0 0x560862726e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba0a4b640 1 --2- 172.21.15.172:0/4250196149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb90003b00 0x5608625ea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba124c640 1 -- 172.21.15.172:0/4250196149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb90003b00 msgr2=0x5608625ea5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba124c640 1 --2- 172.21.15.172:0/4250196149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb90003b00 0x5608625ea5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba124c640 1 -- 172.21.15.172:0/4250196149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608625ebfe0 msgr2=0x560862726e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:17.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba124c640 1 --2- 172.21.15.172:0/4250196149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608625ebfe0 0x560862726e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcba124c640 1 -- 172.21.15.172:0/4250196149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086267e610 con 0x560862699310 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.968+0000 7fcb9bfff640 1 --2- 172.21.15.172:0/4250196149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608625ebfe0 0x560862726e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcba124c640 1 --2- 172.21.15.172:0/4250196149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560862699310 0x5608627273c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00a170 tx=0x7fcb9c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c78f880e223a036 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcb9affd640 1 -- 172.21.15.172:0/4250196149 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c061070 con 0x560862699310 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcb9affd640 1 -- 172.21.15.172:0/4250196149 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb9c04f850 con 0x560862699310 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcb9affd640 1 -- 172.21.15.172:0/4250196149 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c057040 con 0x560862699310 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/4250196149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560862699310 msgr2=0x5608627273c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcba3a3bf40 1 --2- 172.21.15.172:0/4250196149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560862699310 0x5608627273c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00a170 tx=0x7fcb9c003330 comp rx=0 tx=0).stop 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/4250196149 shutdown_connections 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcba3a3bf40 1 --2- 172.21.15.172:0/4250196149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560862699310 0x5608627273c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcba3a3bf40 1 --2- 172.21.15.172:0/4250196149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5608625ebfe0 0x560862726e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcba3a3bf40 1 --2- 172.21.15.172:0/4250196149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb90003b00 0x5608625ea5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/4250196149 >> 172.21.15.172:0/4250196149 conn(0x5608625e73d0 msgr2=0x5608625ec5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:17.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/4250196149 shutdown_connections 2024-04-14T08:35:17.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.969+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/4250196149 wait complete. 2024-04-14T08:35:17.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.970+0000 7fcba3a3bf40 1 Processor -- start 2024-04-14T08:35:17.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.970+0000 7fcba3a3bf40 1 -- start start 2024-04-14T08:35:17.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.970+0000 7fcba3a3bf40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608625ebfe0 0x560862677080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba3a3bf40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560862699310 0x56086267cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba3a3bf40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb90003b00 0x56086272f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba3a3bf40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560862605710 con 0x5608625ebfe0 2024-04-14T08:35:17.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba3a3bf40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560862677f70 con 0x7fcb90003b00 2024-04-14T08:35:17.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba3a3bf40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x560862678770 con 0x560862699310 2024-04-14T08:35:17.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba0a4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608625ebfe0 0x560862677080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba0a4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608625ebfe0 0x560862677080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34052/0 (socket says 172.21.15.172:34052) 2024-04-14T08:35:17.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba0a4b640 1 -- 172.21.15.172:0/3247836823 learned_addr learned my addr 172.21.15.172:0/3247836823 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcb9bfff640 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560862699310 0x56086267cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba124c640 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb90003b00 0x56086272f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba0a4b640 1 -- 172.21.15.172:0/3247836823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560862699310 msgr2=0x56086267cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba0a4b640 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560862699310 0x56086267cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba0a4b640 1 -- 172.21.15.172:0/3247836823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb90003b00 msgr2=0x56086272f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba0a4b640 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb90003b00 0x56086272f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba0a4b640 1 -- 172.21.15.172:0/3247836823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086267c5f0 con 0x5608625ebfe0 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba0a4b640 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608625ebfe0 0x560862677080 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcb94002e70 tx=0x7fcb9400e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcb99ffb640 1 -- 172.21.15.172:0/3247836823 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400ee70 con 0x5608625ebfe0 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcb99ffb640 1 -- 172.21.15.172:0/3247836823 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb94012640 con 0x5608625ebfe0 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/3247836823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5608625bcc00 con 0x5608625ebfe0 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcb99ffb640 1 -- 172.21.15.172:0/3247836823 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb940129f0 con 0x5608625ebfe0 2024-04-14T08:35:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.971+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/3247836823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608625bcf40 con 0x5608625ebfe0 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.973+0000 7fcb99ffb640 1 -- 172.21.15.172:0/3247836823 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb94012b90 con 0x5608625ebfe0 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.973+0000 7fcb99ffb640 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb8406b2d0 0x7fcb8406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.973+0000 7fcb99ffb640 1 -- 172.21.15.172:0/3247836823 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fcb9408ae30 con 0x5608625ebfe0 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.973+0000 7fcb9bfff640 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb8406b2d0 0x7fcb8406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.973+0000 7fcba3a3bf40 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x56086267d0d0 0x56086272fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.973+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/3247836823 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50) -- 0x560862675af0 con 0x56086267d0d0 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.973+0000 7fcba124c640 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x56086267d0d0 0x56086272fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcb9bfff640 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb8406b2d0 0x7fcb8406d790 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5608625e66e0 tx=0x7fcb8c011040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba124c640 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x56086267d0d0 0x56086272fd90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba124c640 1 -- 172.21.15.172:0/3247836823 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fcb9c04fcd0 con 0x56086267d0d0 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x56086267d0d0 msgr2=0x56086272fd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba3a3bf40 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x56086267d0d0 0x56086272fd90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb8406b2d0 msgr2=0x7fcb8406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba3a3bf40 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb8406b2d0 0x7fcb8406d790 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5608625e66e0 tx=0x7fcb8c011040 comp rx=0 tx=0).stop 2024-04-14T08:35:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608625ebfe0 msgr2=0x560862677080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba3a3bf40 1 --2- 172.21.15.172:0/3247836823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5608625ebfe0 0x560862677080 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcb94002e70 tx=0x7fcb9400e6a0 comp rx=0 tx=0).stop 2024-04-14T08:35:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba124c640 1 -- 172.21.15.172:0/3247836823 reap_dead start 2024-04-14T08:35:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/3247836823 shutdown_connections 2024-04-14T08:35:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/3247836823 >> 172.21.15.172:0/3247836823 conn(0x5608625e73d0 msgr2=0x5608625ea5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/3247836823 shutdown_connections 2024-04-14T08:35:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:17.974+0000 7fcba3a3bf40 1 -- 172.21.15.172:0/3247836823 wait complete. 2024-04-14T08:35:17.981 INFO:teuthology.orchestra.run.smithi172.stdout:Yalphaparent_a 2024-04-14T08:35:17.981 DEBUG:teuthology.orchestra.run.smithi172:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-14T08:35:18.067 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:35:18.067 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 1099511627777, 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: "ancestors": [ 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: "dirino": 1099511627776, 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: "dname": "alpha", 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 2 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: "dirino": 1, 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: "dname": "parent_a", 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 4 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-04-14T08:35:18.068 INFO:teuthology.orchestra.run.smithi172.stdout: "pool": 8, 2024-04-14T08:35:18.069 INFO:teuthology.orchestra.run.smithi172.stdout: "old_pools": [] 2024-04-14T08:35:18.069 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:35:18.069 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:18.070 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:35:18.070 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:35:18.144 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:35:18.187 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:35:18.188 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:35:18.188 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:35:18.188 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:35:18.203 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:35:18.203 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:35:18.259 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:35:18.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.400+0000 7f47115fe640 1 Processor -- start 2024-04-14T08:35:18.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.400+0000 7f47115fe640 1 -- start start 2024-04-14T08:35:18.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.401+0000 7f47115fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f470c15c150 0x7f470c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.401+0000 7f47115fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470c0d1840 0x7f470c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.401+0000 7f47115fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f470c162530 0x7f470c15ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.401+0000 7f47115fe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f470c0cef10 con 0x7f470c162530 2024-04-14T08:35:18.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.401+0000 7f47115fe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f470c0d1c80 con 0x7f470c15c150 2024-04-14T08:35:18.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.401+0000 7f47115fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f470c162940 con 0x7f470c0d1840 2024-04-14T08:35:18.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.401+0000 7f470b7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f470c162530 0x7f470c15ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.401+0000 7f470b7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f470c162530 0x7f470c15ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34060/0 (socket says 172.21.15.172:34060) 2024-04-14T08:35:18.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.401+0000 7f470b7fe640 1 -- 172.21.15.172:0/2250743179 learned_addr learned my addr 172.21.15.172:0/2250743179 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:18.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.401+0000 7f470b7fe640 1 -- 172.21.15.172:0/2250743179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470c0d1840 msgr2=0x7f470c161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:18.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.401+0000 7f470b7fe640 1 --2- 172.21.15.172:0/2250743179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470c0d1840 0x7f470c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.402+0000 7f470b7fe640 1 -- 172.21.15.172:0/2250743179 >> v2:172.21.15.177:3300/0 conn(0x7f470c15c150 msgr2=0x7f470c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:18.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.402+0000 7f470affd640 1 --2- 172.21.15.172:0/2250743179 >> v2:172.21.15.177:3300/0 conn(0x7f470c15c150 0x7f470c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.402+0000 7f470b7fe640 1 --2- 172.21.15.172:0/2250743179 >> v2:172.21.15.177:3300/0 conn(0x7f470c15c150 0x7f470c0d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.402+0000 7f470b7fe640 1 -- 172.21.15.172:0/2250743179 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c0d0200 con 0x7f470c162530 2024-04-14T08:35:18.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.402+0000 7f470affd640 1 --2- 172.21.15.172:0/2250743179 >> v2:172.21.15.177:3300/0 conn(0x7f470c15c150 0x7f470c0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:18.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.402+0000 7f470b7fe640 1 --2- 172.21.15.172:0/2250743179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c162530 0x7f470c15ea20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f470c0d03d0 tx=0x7f470402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b38a4935a98714e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:18.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.402+0000 7f470a7fc640 1 -- 172.21.15.172:0/2250743179 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470402ca70 con 0x7f470c162530 2024-04-14T08:35:18.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.403+0000 7f470a7fc640 1 -- 172.21.15.172:0/2250743179 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470402cc10 con 0x7f470c162530 2024-04-14T08:35:18.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.403+0000 7f470a7fc640 1 -- 172.21.15.172:0/2250743179 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040343f0 con 0x7f470c162530 2024-04-14T08:35:18.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.403+0000 7f47115fe640 1 -- 172.21.15.172:0/2250743179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c162530 msgr2=0x7f470c15ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:18.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.403+0000 7f47115fe640 1 --2- 172.21.15.172:0/2250743179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c162530 0x7f470c15ea20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f470c0d03d0 tx=0x7f470402bed0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.403+0000 7f47115fe640 1 -- 172.21.15.172:0/2250743179 shutdown_connections 2024-04-14T08:35:18.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.403+0000 7f47115fe640 1 --2- 172.21.15.172:0/2250743179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c162530 0x7f470c15ea20 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.403+0000 7f47115fe640 1 --2- 172.21.15.172:0/2250743179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470c0d1840 0x7f470c161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.403+0000 7f47115fe640 1 --2- 172.21.15.172:0/2250743179 >> v2:172.21.15.177:3300/0 conn(0x7f470c15c150 0x7f470c0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.403+0000 7f47115fe640 1 -- 172.21.15.172:0/2250743179 >> 172.21.15.172:0/2250743179 conn(0x7f470c09d580 msgr2=0x7f470c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:18.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.403+0000 7f47115fe640 1 -- 172.21.15.172:0/2250743179 shutdown_connections 2024-04-14T08:35:18.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.403+0000 7f47115fe640 1 -- 172.21.15.172:0/2250743179 wait complete. 2024-04-14T08:35:18.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.404+0000 7f47115fe640 1 Processor -- start 2024-04-14T08:35:18.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.404+0000 7f47115fe640 1 -- start start 2024-04-14T08:35:18.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f47115fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f470c0d1840 0x7f470c15a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f47115fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c15c150 0x7f470c15c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f47115fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470c162530 0x7f470c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f47115fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f470c0cf720 con 0x7f470c15c150 2024-04-14T08:35:18.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f47115fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f470c0cf440 con 0x7f470c0d1840 2024-04-14T08:35:18.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f47115fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f470c0cfa00 con 0x7f470c162530 2024-04-14T08:35:18.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f4703fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c15c150 0x7f470c15c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f4703fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c15c150 0x7f470c15c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34068/0 (socket says 172.21.15.172:34068) 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f4703fff640 1 -- 172.21.15.172:0/838624852 learned_addr learned my addr 172.21.15.172:0/838624852 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f470affd640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f470c0d1840 0x7f470c15a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f470b7fe640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470c162530 0x7f470c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f4703fff640 1 -- 172.21.15.172:0/838624852 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470c162530 msgr2=0x7f470c15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f4703fff640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470c162530 0x7f470c15cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f4703fff640 1 -- 172.21.15.172:0/838624852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f470c0d1840 msgr2=0x7f470c15a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f4703fff640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f470c0d1840 0x7f470c15a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f4703fff640 1 -- 172.21.15.172:0/838624852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c15d180 con 0x7f470c15c150 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f470affd640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f470c0d1840 0x7f470c15a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.405+0000 7f470b7fe640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470c162530 0x7f470c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.406+0000 7f4703fff640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c15c150 0x7f470c15c5c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f46f400b4d0 tx=0x7f46f400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.406+0000 7f47037fe640 1 -- 172.21.15.172:0/838624852 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4014070 con 0x7f470c15c150 2024-04-14T08:35:18.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.406+0000 7f47037fe640 1 -- 172.21.15.172:0/838624852 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f4010870 con 0x7f470c15c150 2024-04-14T08:35:18.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.406+0000 7f47037fe640 1 -- 172.21.15.172:0/838624852 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4010c60 con 0x7f470c15c150 2024-04-14T08:35:18.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.406+0000 7f47115fe640 1 -- 172.21.15.172:0/838624852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470c15d460 con 0x7f470c15c150 2024-04-14T08:35:18.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.406+0000 7f47115fe640 1 -- 172.21.15.172:0/838624852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470c15da20 con 0x7f470c15c150 2024-04-14T08:35:18.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.407+0000 7f47115fe640 1 -- 172.21.15.172:0/838624852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470c075490 con 0x7f470c15c150 2024-04-14T08:35:18.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.407+0000 7f47037fe640 1 -- 172.21.15.172:0/838624852 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46f4017070 con 0x7f470c15c150 2024-04-14T08:35:18.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.408+0000 7f47037fe640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f46ec06c8f0 0x7f46ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.408+0000 7f47037fe640 1 -- 172.21.15.172:0/838624852 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f46f408bb80 con 0x7f470c15c150 2024-04-14T08:35:18.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.408+0000 7f470affd640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f46ec06c8f0 0x7f46ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.408+0000 7f470affd640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f46ec06c8f0 0x7f46ec06edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f46fc006fd0 tx=0x7f46fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:18.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.412+0000 7f47037fe640 1 -- 172.21.15.172:0/838624852 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f46f4059d80 con 0x7f470c15c150 2024-04-14T08:35:18.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.594+0000 7f47115fe640 1 -- 172.21.15.172:0/838624852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f470c0d49c0 con 0x7f470c15c150 2024-04-14T08:35:18.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.595+0000 7f47037fe640 1 -- 172.21.15.172:0/838624852 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+17002 (secure 0 0 0) 0x7f46f4059680 con 0x7f470c15c150 2024-04-14T08:35:18.596 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:18.596 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":50,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:36.423005+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:34:32.990827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-14T08:34:33.715803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:35:18.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.598+0000 7f47115fe640 1 -- 172.21.15.172:0/838624852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f46ec06c8f0 msgr2=0x7f46ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:18.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.598+0000 7f47115fe640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f46ec06c8f0 0x7f46ec06edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f46fc006fd0 tx=0x7f46fc008040 comp rx=0 tx=0).stop 2024-04-14T08:35:18.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.598+0000 7f47115fe640 1 -- 172.21.15.172:0/838624852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c15c150 msgr2=0x7f470c15c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:18.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.598+0000 7f47115fe640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c15c150 0x7f470c15c5c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f46f400b4d0 tx=0x7f46f400b990 comp rx=0 tx=0).stop 2024-04-14T08:35:18.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.598+0000 7f47115fe640 1 -- 172.21.15.172:0/838624852 shutdown_connections 2024-04-14T08:35:18.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.598+0000 7f47115fe640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f46ec06c8f0 0x7f46ec06edb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.598+0000 7f47115fe640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470c162530 0x7f470c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.598+0000 7f47115fe640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f470c15c150 0x7f470c15c5c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.598+0000 7f47115fe640 1 --2- 172.21.15.172:0/838624852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f470c0d1840 0x7f470c15a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.598+0000 7f47115fe640 1 -- 172.21.15.172:0/838624852 >> 172.21.15.172:0/838624852 conn(0x7f470c09d580 msgr2=0x7f470c161df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:18.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.598+0000 7f47115fe640 1 -- 172.21.15.172:0/838624852 shutdown_connections 2024-04-14T08:35:18.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.598+0000 7f47115fe640 1 -- 172.21.15.172:0/838624852 wait complete. 2024-04-14T08:35:18.610 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:34:32.990827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:34:33.715803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:35:18.610 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:35:18.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.715+0000 7fe4a6ec8640 1 Processor -- start 2024-04-14T08:35:18.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.715+0000 7fe4a6ec8640 1 -- start start 2024-04-14T08:35:18.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.715+0000 7fe4a6ec8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4a015e430 0x7fe4a0162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.715+0000 7fe4a6ec8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4a0162540 0x7fe4a01629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.715+0000 7fe4a6ec8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe4a00b3a30 0x7fe4a00b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.715+0000 7fe4a6ec8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe4a00cefb0 con 0x7fe4a015e430 2024-04-14T08:35:18.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.715+0000 7fe4a6ec8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe4a00cf290 con 0x7fe4a00b3a30 2024-04-14T08:35:18.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.715+0000 7fe4a6ec8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a00cf570 con 0x7fe4a0162540 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.715+0000 7fe4a4c64640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4a015e430 0x7fe4a0162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.716+0000 7fe4a4c64640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4a015e430 0x7fe4a0162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34084/0 (socket says 172.21.15.172:34084) 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.716+0000 7fe4a4c64640 1 -- 172.21.15.172:0/3512014796 learned_addr learned my addr 172.21.15.172:0/3512014796 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.716+0000 7fe4a5465640 1 --2- 172.21.15.172:0/3512014796 >> v2:172.21.15.177:3300/0 conn(0x7fe4a00b3a30 0x7fe4a00b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.716+0000 7fe49ffff640 1 --2- 172.21.15.172:0/3512014796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4a0162540 0x7fe4a01629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.716+0000 7fe4a4c64640 1 -- 172.21.15.172:0/3512014796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4a0162540 msgr2=0x7fe4a01629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.716+0000 7fe4a4c64640 1 --2- 172.21.15.172:0/3512014796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4a0162540 0x7fe4a01629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.716+0000 7fe4a4c64640 1 -- 172.21.15.172:0/3512014796 >> v2:172.21.15.177:3300/0 conn(0x7fe4a00b3a30 msgr2=0x7fe4a00b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.716+0000 7fe4a4c64640 1 --2- 172.21.15.172:0/3512014796 >> v2:172.21.15.177:3300/0 conn(0x7fe4a00b3a30 0x7fe4a00b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.716+0000 7fe4a4c64640 1 -- 172.21.15.172:0/3512014796 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a00cf850 con 0x7fe4a015e430 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe4a4c64640 1 --2- 172.21.15.172:0/3512014796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4a015e430 0x7fe4a0162000 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00a2180 tx=0x7fe49802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4041d46cc2bc355f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe49f7fe640 1 -- 172.21.15.172:0/3512014796 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49802ca70 con 0x7fe4a015e430 2024-04-14T08:35:18.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe49f7fe640 1 -- 172.21.15.172:0/3512014796 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49802cc10 con 0x7fe4a015e430 2024-04-14T08:35:18.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe49f7fe640 1 -- 172.21.15.172:0/3512014796 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4980343f0 con 0x7fe4a015e430 2024-04-14T08:35:18.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/3512014796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4a015e430 msgr2=0x7fe4a0162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:18.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe4a6ec8640 1 --2- 172.21.15.172:0/3512014796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4a015e430 0x7fe4a0162000 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00a2180 tx=0x7fe49802bed0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/3512014796 shutdown_connections 2024-04-14T08:35:18.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe4a6ec8640 1 --2- 172.21.15.172:0/3512014796 >> v2:172.21.15.177:3300/0 conn(0x7fe4a00b3a30 0x7fe4a00b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe4a6ec8640 1 --2- 172.21.15.172:0/3512014796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4a0162540 0x7fe4a01629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe4a6ec8640 1 --2- 172.21.15.172:0/3512014796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4a015e430 0x7fe4a0162000 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/3512014796 >> 172.21.15.172:0/3512014796 conn(0x7fe4a009d560 msgr2=0x7fe4a00d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:18.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/3512014796 shutdown_connections 2024-04-14T08:35:18.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.717+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/3512014796 wait complete. 2024-04-14T08:35:18.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.719+0000 7fe4a6ec8640 1 Processor -- start 2024-04-14T08:35:18.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.719+0000 7fe4a6ec8640 1 -- start start 2024-04-14T08:35:18.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.719+0000 7fe4a6ec8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4a00b3a30 0x7fe4a015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.719+0000 7fe4a6ec8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4a015e430 0x7fe4a01e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.719+0000 7fe4a6ec8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a0162540 0x7fe4a01e9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.719+0000 7fe4a6ec8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a00d4e20 con 0x7fe4a00b3a30 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.719+0000 7fe4a6ec8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a015e810 con 0x7fe4a0162540 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.719+0000 7fe4a6ec8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a00d0050 con 0x7fe4a015e430 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe4a4c64640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4a00b3a30 0x7fe4a015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe4a4c64640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4a00b3a30 0x7fe4a015b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34086/0 (socket says 172.21.15.172:34086) 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe4a4c64640 1 -- 172.21.15.172:0/2472234847 learned_addr learned my addr 172.21.15.172:0/2472234847 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe49ffff640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4a015e430 0x7fe4a01e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe4a5465640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a0162540 0x7fe4a01e9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe4a4c64640 1 -- 172.21.15.172:0/2472234847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4a015e430 msgr2=0x7fe4a01e90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe4a4c64640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4a015e430 0x7fe4a01e90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe4a4c64640 1 -- 172.21.15.172:0/2472234847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a0162540 msgr2=0x7fe4a01e9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe4a4c64640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a0162540 0x7fe4a01e9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe4a4c64640 1 -- 172.21.15.172:0/2472234847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a00b7580 con 0x7fe4a00b3a30 2024-04-14T08:35:18.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe4a4c64640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4a00b3a30 0x7fe4a015b940 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe498036040 tx=0x7fe498004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:18.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe49d7fa640 1 -- 172.21.15.172:0/2472234847 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498043020 con 0x7fe4a00b3a30 2024-04-14T08:35:18.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe49d7fa640 1 -- 172.21.15.172:0/2472234847 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49803e450 con 0x7fe4a00b3a30 2024-04-14T08:35:18.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/2472234847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a00b6fc0 con 0x7fe4a00b3a30 2024-04-14T08:35:18.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe49d7fa640 1 -- 172.21.15.172:0/2472234847 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498009510 con 0x7fe4a00b3a30 2024-04-14T08:35:18.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.720+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/2472234847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a01e9600 con 0x7fe4a00b3a30 2024-04-14T08:35:18.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.721+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/2472234847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a0068590 con 0x7fe4a00b3a30 2024-04-14T08:35:18.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.722+0000 7fe49d7fa640 1 -- 172.21.15.172:0/2472234847 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe498046020 con 0x7fe4a00b3a30 2024-04-14T08:35:18.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.722+0000 7fe49d7fa640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe48c06c8f0 0x7fe48c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:18.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.722+0000 7fe49d7fa640 1 -- 172.21.15.172:0/2472234847 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fe4980ae3f0 con 0x7fe4a00b3a30 2024-04-14T08:35:18.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.722+0000 7fe49ffff640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe48c06c8f0 0x7fe48c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:18.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.723+0000 7fe49ffff640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe48c06c8f0 0x7fe48c06edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00d1680 tx=0x7fe494008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:18.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.727+0000 7fe49d7fa640 1 -- 172.21.15.172:0/2472234847 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe49804d030 con 0x7fe4a00b3a30 2024-04-14T08:35:18.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.925+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/2472234847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe4a00d18a0 con 0x7fe4a00b3a30 2024-04-14T08:35:18.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.926+0000 7fe49d7fa640 1 -- 172.21.15.172:0/2472234847 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7fe4a00b7580 con 0x7fe4a00b3a30 2024-04-14T08:35:18.927 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:35:18.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.928+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/2472234847 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe48c06c8f0 msgr2=0x7fe48c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:18.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.928+0000 7fe4a6ec8640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe48c06c8f0 0x7fe48c06edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00d1680 tx=0x7fe494008040 comp rx=0 tx=0).stop 2024-04-14T08:35:18.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.928+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/2472234847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4a00b3a30 msgr2=0x7fe4a015b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:18.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.928+0000 7fe4a6ec8640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4a00b3a30 0x7fe4a015b940 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe498036040 tx=0x7fe498004a40 comp rx=0 tx=0).stop 2024-04-14T08:35:18.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.928+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/2472234847 shutdown_connections 2024-04-14T08:35:18.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.928+0000 7fe4a6ec8640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe48c06c8f0 0x7fe48c06edb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.928+0000 7fe4a6ec8640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a0162540 0x7fe4a01e9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.928+0000 7fe4a6ec8640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4a015e430 0x7fe4a01e90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.928+0000 7fe4a6ec8640 1 --2- 172.21.15.172:0/2472234847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4a00b3a30 0x7fe4a015b940 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:18.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.928+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/2472234847 >> 172.21.15.172:0/2472234847 conn(0x7fe4a009d560 msgr2=0x7fe4a00b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:18.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.928+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/2472234847 shutdown_connections 2024-04-14T08:35:18.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:18.928+0000 7fe4a6ec8640 1 -- 172.21.15.172:0/2472234847 wait complete. 2024-04-14T08:35:18.939 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:35:19.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.044+0000 7fdece78c640 1 Processor -- start 2024-04-14T08:35:19.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.044+0000 7fdece78c640 1 -- start start 2024-04-14T08:35:19.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.045+0000 7fdece78c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec815c420 0x7fdec80b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.045+0000 7fdece78c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdec80ccdd0 con 0x7fdec806f3e0 2024-04-14T08:35:19.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.045+0000 7fdece78c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdec80d2ae0 con 0x7fdec80ceb00 2024-04-14T08:35:19.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.045+0000 7fdece78c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdec80d30a0 con 0x7fdec815c420 2024-04-14T08:35:19.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.045+0000 7fdec77fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fdec806f3e0 0x7fdec80d0010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:41220/0 (socket says 172.21.15.172:41220) 2024-04-14T08:35:19.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.045+0000 7fdec77fe640 1 -- 172.21.15.172:0/786060732 learned_addr learned my addr 172.21.15.172:0/786060732 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:19.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.046+0000 7fdec7fff640 1 --2- 172.21.15.172:0/786060732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec815c420 0x7fdec80b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.046+0000 7fdec6ffd640 1 -- 172.21.15.172:0/786060732 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3716215588 0 0) 0x7fdec80ccdd0 con 0x7fdec806f3e0 2024-04-14T08:35:19.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.046+0000 7fdec6ffd640 1 -- 172.21.15.172:0/786060732 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdeb0003520 con 0x7fdec806f3e0 2024-04-14T08:35:19.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.046+0000 7fdec6ffd640 1 -- 172.21.15.172:0/786060732 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860302617 0 0) 0x7fdec80d2ae0 con 0x7fdec80ceb00 2024-04-14T08:35:19.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.046+0000 7fdec6ffd640 1 -- 172.21.15.172:0/786060732 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdec80ccdd0 con 0x7fdec80ceb00 2024-04-14T08:35:19.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.046+0000 7fdec7fff640 1 -- 172.21.15.172:0/786060732 >> v1:172.21.15.177:6789/0 conn(0x7fdec80ceb00 legacy=0x7fdec80cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.046+0000 7fdec7fff640 1 -- 172.21.15.172:0/786060732 >> v1:172.21.15.172:6789/0 conn(0x7fdec806f3e0 legacy=0x7fdec80d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.046+0000 7fdec7fff640 1 -- 172.21.15.172:0/786060732 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec80d2dc0 con 0x7fdec815c420 2024-04-14T08:35:19.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.047+0000 7fdec7fff640 1 --2- 172.21.15.172:0/786060732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec815c420 0x7fdec80b1510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80097d0 tx=0x7fdeb802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0af14c96a42854b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:19.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.047+0000 7fdec6ffd640 1 -- 172.21.15.172:0/786060732 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8035020 con 0x7fdec815c420 2024-04-14T08:35:19.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.047+0000 7fdec6ffd640 1 -- 172.21.15.172:0/786060732 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb8031890 con 0x7fdec815c420 2024-04-14T08:35:19.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.047+0000 7fdec6ffd640 1 -- 172.21.15.172:0/786060732 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8031c40 con 0x7fdec815c420 2024-04-14T08:35:19.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.048+0000 7fdece78c640 1 -- 172.21.15.172:0/786060732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec815c420 msgr2=0x7fdec80b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.048+0000 7fdece78c640 1 --2- 172.21.15.172:0/786060732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec815c420 0x7fdec80b1510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80097d0 tx=0x7fdeb802b930 comp rx=0 tx=0).stop 2024-04-14T08:35:19.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.048+0000 7fdece78c640 1 -- 172.21.15.172:0/786060732 shutdown_connections 2024-04-14T08:35:19.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.048+0000 7fdece78c640 1 --2- 172.21.15.172:0/786060732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec815c420 0x7fdec80b1510 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.048+0000 7fdece78c640 1 -- 172.21.15.172:0/786060732 >> 172.21.15.172:0/786060732 conn(0x7fdec808c000 msgr2=0x7fdec80d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:19.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.048+0000 7fdece78c640 1 -- 172.21.15.172:0/786060732 shutdown_connections 2024-04-14T08:35:19.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.048+0000 7fdece78c640 1 -- 172.21.15.172:0/786060732 wait complete. 2024-04-14T08:35:19.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.049+0000 7fdece78c640 1 Processor -- start 2024-04-14T08:35:19.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.050+0000 7fdece78c640 1 -- start start 2024-04-14T08:35:19.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdece78c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec80ceb00 0x7fdec809f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdece78c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdec815c420 0x7fdec80c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdece78c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec80c85b0 0x7fdec80b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdece78c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdec80b11e0 con 0x7fdec80c85b0 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdece78c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdec806f870 con 0x7fdec815c420 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdece78c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdec815c800 con 0x7fdec80ceb00 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdeccd29640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec80c85b0 0x7fdec80b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdeccd29640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec80c85b0 0x7fdec80b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34096/0 (socket says 172.21.15.172:34096) 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdeccd29640 1 -- 172.21.15.172:0/3008426378 learned_addr learned my addr 172.21.15.172:0/3008426378 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdec7fff640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec80ceb00 0x7fdec809f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdec77fe640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdec815c420 0x7fdec80c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdeccd29640 1 -- 172.21.15.172:0/3008426378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec80ceb00 msgr2=0x7fdec809f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdeccd29640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec80ceb00 0x7fdec809f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdeccd29640 1 -- 172.21.15.172:0/3008426378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdec815c420 msgr2=0x7fdec80c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdeccd29640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdec815c420 0x7fdec80c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdeccd29640 1 -- 172.21.15.172:0/3008426378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec80b6c40 con 0x7fdec80c85b0 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.051+0000 7fdec77fe640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdec815c420 0x7fdec80c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:19.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.052+0000 7fdeccd29640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec80c85b0 0x7fdec80b5e80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdec000c7c0 tx=0x7fdec000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:19.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.052+0000 7fdec4ff9640 1 -- 172.21.15.172:0/3008426378 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0003b70 con 0x7fdec80c85b0 2024-04-14T08:35:19.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.052+0000 7fdec4ff9640 1 -- 172.21.15.172:0/3008426378 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec0003d10 con 0x7fdec80c85b0 2024-04-14T08:35:19.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.052+0000 7fdec4ff9640 1 -- 172.21.15.172:0/3008426378 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec000aa50 con 0x7fdec80c85b0 2024-04-14T08:35:19.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.052+0000 7fdece78c640 1 -- 172.21.15.172:0/3008426378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec80b6f20 con 0x7fdec80c85b0 2024-04-14T08:35:19.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.052+0000 7fdece78c640 1 -- 172.21.15.172:0/3008426378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec80b74e0 con 0x7fdec80c85b0 2024-04-14T08:35:19.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.053+0000 7fdece78c640 1 -- 172.21.15.172:0/3008426378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec809ff10 con 0x7fdec80c85b0 2024-04-14T08:35:19.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.053+0000 7fdec4ff9640 1 -- 172.21.15.172:0/3008426378 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdec000abf0 con 0x7fdec80c85b0 2024-04-14T08:35:19.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.054+0000 7fdec4ff9640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdeb406c8f0 0x7fdeb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.054+0000 7fdec7fff640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdeb406c8f0 0x7fdeb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.054+0000 7fdec4ff9640 1 -- 172.21.15.172:0/3008426378 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fdec008c600 con 0x7fdec80c85b0 2024-04-14T08:35:19.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.054+0000 7fdec7fff640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdeb406c8f0 0x7fdeb406edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8036000 tx=0x7fdeb803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:19.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.058+0000 7fdec4ff9640 1 -- 172.21.15.172:0/3008426378 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdec005a980 con 0x7fdec80c85b0 2024-04-14T08:35:19.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.240+0000 7fdece78c640 1 -- 172.21.15.172:0/3008426378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdec8075490 con 0x7fdec80c85b0 2024-04-14T08:35:19.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.240+0000 7fdec4ff9640 1 -- 172.21.15.172:0/3008426378 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) ==== 104+0+11 (secure 0 0 0) 0x7fdec005a280 con 0x7fdec80c85b0 2024-04-14T08:35:19.241 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:35:19.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.242+0000 7fdece78c640 1 -- 172.21.15.172:0/3008426378 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdeb406c8f0 msgr2=0x7fdeb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.242+0000 7fdece78c640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdeb406c8f0 0x7fdeb406edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8036000 tx=0x7fdeb803c040 comp rx=0 tx=0).stop 2024-04-14T08:35:19.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.242+0000 7fdece78c640 1 -- 172.21.15.172:0/3008426378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec80c85b0 msgr2=0x7fdec80b5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.242+0000 7fdece78c640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec80c85b0 0x7fdec80b5e80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdec000c7c0 tx=0x7fdec000cc80 comp rx=0 tx=0).stop 2024-04-14T08:35:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.242+0000 7fdece78c640 1 -- 172.21.15.172:0/3008426378 shutdown_connections 2024-04-14T08:35:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.243+0000 7fdece78c640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdeb406c8f0 0x7fdeb406edb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.243+0000 7fdece78c640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec80c85b0 0x7fdec80b5e80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.243+0000 7fdece78c640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdec815c420 0x7fdec80c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.243+0000 7fdece78c640 1 --2- 172.21.15.172:0/3008426378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec80ceb00 0x7fdec809f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.243+0000 7fdece78c640 1 -- 172.21.15.172:0/3008426378 >> 172.21.15.172:0/3008426378 conn(0x7fdec808c000 msgr2=0x7fdec80b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.243+0000 7fdece78c640 1 -- 172.21.15.172:0/3008426378 shutdown_connections 2024-04-14T08:35:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.243+0000 7fdece78c640 1 -- 172.21.15.172:0/3008426378 wait complete. 2024-04-14T08:35:19.254 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:35:19.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.359+0000 7fb9cb5bb640 1 Processor -- start 2024-04-14T08:35:19.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.359+0000 7fb9cb5bb640 1 -- start start 2024-04-14T08:35:19.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9cb5bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c4162710 0x7fb9c40d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9cb5bb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9c40cefb0 con 0x7fb9c415e430 2024-04-14T08:35:19.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9cb5bb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9c40cf290 con 0x7fb9c4162b70 2024-04-14T08:35:19.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9cb5bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c40cf570 con 0x7fb9c4162710 2024-04-14T08:35:19.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9c9357640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb9c415e430 0x7fb9c4162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:41226/0 (socket says 172.21.15.172:41226) 2024-04-14T08:35:19.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9c9357640 1 -- 172.21.15.172:0/2704085561 learned_addr learned my addr 172.21.15.172:0/2704085561 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:19.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1301537484 0 0) 0x7fb9c40cefb0 con 0x7fb9c415e430 2024-04-14T08:35:19.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9c8b56640 1 --2- 172.21.15.172:0/2704085561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c4162710 0x7fb9c40d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9b0003520 con 0x7fb9c415e430 2024-04-14T08:35:19.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1400891943 0 0) 0x7fb9b0003520 con 0x7fb9c415e430 2024-04-14T08:35:19.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c4162710 msgr2=0x7fb9c40d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:19.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9bbfff640 1 --2- 172.21.15.172:0/2704085561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c4162710 0x7fb9c40d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 >> v1:172.21.15.177:6789/0 conn(0x7fb9c4162b70 legacy=0x7fb9c40d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c40cf850 con 0x7fb9c415e430 2024-04-14T08:35:19.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.360+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fb9c0001670 con 0x7fb9c415e430 2024-04-14T08:35:19.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 >> v1:172.21.15.172:6789/0 conn(0x7fb9c415e430 legacy=0x7fb9c4162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9bbfff640 1 --2- 172.21.15.172:0/2704085561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c4162710 0x7fb9b00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9bbfff640 1 --2- 172.21.15.172:0/2704085561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9b0003f20 0x7fb9b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9bbfff640 1 --2- 172.21.15.172:0/2704085561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9b0004d40 0x7fb9b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c40cf570 con 0x7fb9b0004d40 2024-04-14T08:35:19.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b000ba20 con 0x7fb9b0003f20 2024-04-14T08:35:19.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b000bc20 con 0x7fb9c4162710 2024-04-14T08:35:19.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9c9b58640 1 --2- 172.21.15.172:0/2704085561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9b0004d40 0x7fb9b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9c9357640 1 --2- 172.21.15.172:0/2704085561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c4162710 0x7fb9b00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9c8b56640 1 --2- 172.21.15.172:0/2704085561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9b0003f20 0x7fb9b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9c9b58640 1 -- 172.21.15.172:0/2704085561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c4162710 msgr2=0x7fb9b00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:19.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9c9b58640 1 --2- 172.21.15.172:0/2704085561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c4162710 0x7fb9b00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9c9b58640 1 -- 172.21.15.172:0/2704085561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9b0003f20 msgr2=0x7fb9b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:19.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9c9b58640 1 --2- 172.21.15.172:0/2704085561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9b0003f20 0x7fb9b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.361+0000 7fb9c9b58640 1 -- 172.21.15.172:0/2704085561 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b0003520 con 0x7fb9b0004d40 2024-04-14T08:35:19.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.362+0000 7fb9c9b58640 1 --2- 172.21.15.172:0/2704085561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9b0004d40 0x7fb9b00051c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc001f60 tx=0x7fb9b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd988a529ce3e8c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:19.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.362+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc001b30 con 0x7fb9b0004d40 2024-04-14T08:35:19.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.362+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9bc02d710 con 0x7fb9b0004d40 2024-04-14T08:35:19.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.362+0000 7fb9bbfff640 1 -- 172.21.15.172:0/2704085561 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc02db00 con 0x7fb9b0004d40 2024-04-14T08:35:19.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.362+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2704085561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9b0004d40 msgr2=0x7fb9b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.362+0000 7fb9cb5bb640 1 --2- 172.21.15.172:0/2704085561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9b0004d40 0x7fb9b00051c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc001f60 tx=0x7fb9b0005830 comp rx=0 tx=0).stop 2024-04-14T08:35:19.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.362+0000 7fb9c9b58640 1 -- 172.21.15.172:0/2704085561 reap_dead start 2024-04-14T08:35:19.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.362+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2704085561 shutdown_connections 2024-04-14T08:35:19.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.362+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2704085561 >> 172.21.15.172:0/2704085561 conn(0x7fb9c409d560 msgr2=0x7fb9c40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:19.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.363+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2704085561 shutdown_connections 2024-04-14T08:35:19.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.363+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2704085561 wait complete. 2024-04-14T08:35:19.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.364+0000 7fb9cb5bb640 1 Processor -- start 2024-04-14T08:35:19.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.364+0000 7fb9cb5bb640 1 -- start start 2024-04-14T08:35:19.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.364+0000 7fb9cb5bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c4153940 0x7fb9c40c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.364+0000 7fb9cb5bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c41e8c60 0x7fb9c40c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.364+0000 7fb9cb5bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c4158630 0x7fb9c4158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.364+0000 7fb9cb5bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c40d4e20 con 0x7fb9c4153940 2024-04-14T08:35:19.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.364+0000 7fb9cb5bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c415e810 con 0x7fb9c4158630 2024-04-14T08:35:19.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.364+0000 7fb9cb5bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c40d0050 con 0x7fb9c41e8c60 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9c9357640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c4153940 0x7fb9c40c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9c9357640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c4153940 0x7fb9c40c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34108/0 (socket says 172.21.15.172:34108) 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9c9357640 1 -- 172.21.15.172:0/2733972011 learned_addr learned my addr 172.21.15.172:0/2733972011 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9c8b56640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c41e8c60 0x7fb9c40c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9c9b58640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c4158630 0x7fb9c4158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9c9357640 1 -- 172.21.15.172:0/2733972011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c41e8c60 msgr2=0x7fb9c40c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9c9357640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c41e8c60 0x7fb9c40c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9c9357640 1 -- 172.21.15.172:0/2733972011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c4158630 msgr2=0x7fb9c4158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9c9357640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c4158630 0x7fb9c4158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9c9357640 1 -- 172.21.15.172:0/2733972011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c40c8220 con 0x7fb9c4153940 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9c9b58640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c4158630 0x7fb9c4158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9c9357640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c4153940 0x7fb9c40c92c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0001ba0 tx=0x7fb9c0007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:19.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9b9ffb640 1 -- 172.21.15.172:0/2733972011 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c000f960 con 0x7fb9c4153940 2024-04-14T08:35:19.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9b9ffb640 1 -- 172.21.15.172:0/2733972011 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9c0018070 con 0x7fb9c4153940 2024-04-14T08:35:19.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2733972011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c40c8500 con 0x7fb9c4153940 2024-04-14T08:35:19.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9b9ffb640 1 -- 172.21.15.172:0/2733972011 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c000fd80 con 0x7fb9c4153940 2024-04-14T08:35:19.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.365+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2733972011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c40c8ac0 con 0x7fb9c4153940 2024-04-14T08:35:19.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.366+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2733972011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c4068590 con 0x7fb9c4153940 2024-04-14T08:35:19.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.367+0000 7fb9b9ffb640 1 -- 172.21.15.172:0/2733972011 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9c001d460 con 0x7fb9c4153940 2024-04-14T08:35:19.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.367+0000 7fb9b9ffb640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9a806c920 0x7fb9a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.368+0000 7fb9b9ffb640 1 -- 172.21.15.172:0/2733972011 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fb9c008bdd0 con 0x7fb9c4153940 2024-04-14T08:35:19.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.368+0000 7fb9c8b56640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9a806c920 0x7fb9a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.368+0000 7fb9c8b56640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9a806c920 0x7fb9a806ede0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb9c415d740 tx=0x7fb9b4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:19.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.372+0000 7fb9b9ffb640 1 -- 172.21.15.172:0/2733972011 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb9c005a100 con 0x7fb9c4153940 2024-04-14T08:35:19.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.549+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2733972011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb9c40d18a0 con 0x7fb9c4153940 2024-04-14T08:35:19.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.550+0000 7fb9b9ffb640 1 -- 172.21.15.172:0/2733972011 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) ==== 100+0+11 (secure 0 0 0) 0x7fb9c008d0a0 con 0x7fb9c4153940 2024-04-14T08:35:19.550 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:35:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.552+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2733972011 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9a806c920 msgr2=0x7fb9a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.552+0000 7fb9cb5bb640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9a806c920 0x7fb9a806ede0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb9c415d740 tx=0x7fb9b4009190 comp rx=0 tx=0).stop 2024-04-14T08:35:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.552+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2733972011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c4153940 msgr2=0x7fb9c40c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.552+0000 7fb9cb5bb640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c4153940 0x7fb9c40c92c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0001ba0 tx=0x7fb9c0007660 comp rx=0 tx=0).stop 2024-04-14T08:35:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.552+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2733972011 shutdown_connections 2024-04-14T08:35:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.552+0000 7fb9cb5bb640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9a806c920 0x7fb9a806ede0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.552+0000 7fb9cb5bb640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9c4158630 0x7fb9c4158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.552+0000 7fb9cb5bb640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9c41e8c60 0x7fb9c40c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.552+0000 7fb9cb5bb640 1 --2- 172.21.15.172:0/2733972011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9c4153940 0x7fb9c40c92c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.552+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2733972011 >> 172.21.15.172:0/2733972011 conn(0x7fb9c409d560 msgr2=0x7fb9c40b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.552+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2733972011 shutdown_connections 2024-04-14T08:35:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.552+0000 7fb9cb5bb640 1 -- 172.21.15.172:0/2733972011 wait complete. 2024-04-14T08:35:19.563 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:35:19.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.666+0000 7fd382de0640 1 Processor -- start 2024-04-14T08:35:19.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.666+0000 7fd382de0640 1 -- start start 2024-04-14T08:35:19.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.666+0000 7fd382de0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd37c162710 0x7fd37c0d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.666+0000 7fd382de0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd37c0cefb0 con 0x7fd37c15e430 2024-04-14T08:35:19.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.666+0000 7fd382de0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd37c0cf290 con 0x7fd37c162ba0 2024-04-14T08:35:19.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.666+0000 7fd382de0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fd37c0cf570 con 0x7fd37c162710 2024-04-14T08:35:19.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd380b7c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd37c15e430 0x7fd37c162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50842/0 (socket says 172.21.15.172:50842) 2024-04-14T08:35:19.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd380b7c640 1 -- 172.21.15.172:0/1643040045 learned_addr learned my addr 172.21.15.172:0/1643040045 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:19.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37bfff640 1 --2- 172.21.15.172:0/1643040045 >> v2:172.21.15.177:3300/0 conn(0x7fd37c162710 0x7fd37c0d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2974571593 0 0) 0x7fd37c0cefb0 con 0x7fd37c15e430 2024-04-14T08:35:19.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd36c003520 con 0x7fd37c15e430 2024-04-14T08:35:19.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1715300430 0 0) 0x7fd36c003520 con 0x7fd37c15e430 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 >> v2:172.21.15.177:3300/0 conn(0x7fd37c162710 msgr2=0x7fd37c0d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/1643040045 >> v2:172.21.15.177:3300/0 conn(0x7fd37c162710 0x7fd37c0d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 >> v1:172.21.15.177:6789/0 conn(0x7fd37c162ba0 legacy=0x7fd37c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37c0cf850 con 0x7fd37c15e430 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fd370001670 con 0x7fd37c15e430 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 >> v1:172.21.15.172:6789/0 conn(0x7fd37c15e430 legacy=0x7fd37c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/1643040045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0039b0 0x7fd36c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/1643040045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd36c0043b0 0x7fd36c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/1643040045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c005080 0x7fd36c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c0cf570 con 0x7fd36c0043b0 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c00bfe0 con 0x7fd36c0039b0 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd36c00c1e0 con 0x7fd36c005080 2024-04-14T08:35:19.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd37bfff640 1 --2- 172.21.15.172:0/1643040045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd36c0043b0 0x7fd36c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.667+0000 7fd380b7c640 1 --2- 172.21.15.172:0/1643040045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0039b0 0x7fd36c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.668+0000 7fd38137d640 1 --2- 172.21.15.172:0/1643040045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c005080 0x7fd36c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.668+0000 7fd37bfff640 1 -- 172.21.15.172:0/1643040045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c005080 msgr2=0x7fd36c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.668+0000 7fd37bfff640 1 --2- 172.21.15.172:0/1643040045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c005080 0x7fd36c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.668+0000 7fd37bfff640 1 -- 172.21.15.172:0/1643040045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0039b0 msgr2=0x7fd36c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.668+0000 7fd37bfff640 1 --2- 172.21.15.172:0/1643040045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0039b0 0x7fd36c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.668+0000 7fd37bfff640 1 -- 172.21.15.172:0/1643040045 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c003520 con 0x7fd36c0043b0 2024-04-14T08:35:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.668+0000 7fd38137d640 1 --2- 172.21.15.172:0/1643040045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd36c005080 0x7fd36c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.668+0000 7fd38137d640 1 -- 172.21.15.172:0/1643040045 reap_dead start 2024-04-14T08:35:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.668+0000 7fd37bfff640 1 --2- 172.21.15.172:0/1643040045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd36c0043b0 0x7fd36c004880 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd368009ac0 tx=0x7fd36c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ba52eb964c3a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.669+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368039070 con 0x7fd36c0043b0 2024-04-14T08:35:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.669+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd36802c6b0 con 0x7fd36c0043b0 2024-04-14T08:35:19.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.669+0000 7fd37b7fe640 1 -- 172.21.15.172:0/1643040045 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368009690 con 0x7fd36c0043b0 2024-04-14T08:35:19.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.669+0000 7fd382de0640 1 -- 172.21.15.172:0/1643040045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd36c0043b0 msgr2=0x7fd36c004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.669+0000 7fd382de0640 1 --2- 172.21.15.172:0/1643040045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd36c0043b0 0x7fd36c004880 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd368009ac0 tx=0x7fd36c005830 comp rx=0 tx=0).stop 2024-04-14T08:35:19.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.669+0000 7fd382de0640 1 -- 172.21.15.172:0/1643040045 shutdown_connections 2024-04-14T08:35:19.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.669+0000 7fd382de0640 1 --2- 172.21.15.172:0/1643040045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd36c0043b0 0x7fd36c004880 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.669+0000 7fd382de0640 1 -- 172.21.15.172:0/1643040045 >> 172.21.15.172:0/1643040045 conn(0x7fd37c09d560 msgr2=0x7fd37c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:19.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.669+0000 7fd382de0640 1 -- 172.21.15.172:0/1643040045 shutdown_connections 2024-04-14T08:35:19.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.669+0000 7fd382de0640 1 -- 172.21.15.172:0/1643040045 wait complete. 2024-04-14T08:35:19.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.670+0000 7fd382de0640 1 Processor -- start 2024-04-14T08:35:19.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.670+0000 7fd382de0640 1 -- start start 2024-04-14T08:35:19.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.671+0000 7fd382de0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0043b0 0x7fd37c0b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.671+0000 7fd382de0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c141fe0 0x7fd37c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.671+0000 7fd382de0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd37c142460 0x7fd37c0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.671+0000 7fd382de0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c0d4e20 con 0x7fd37c141fe0 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.671+0000 7fd382de0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c15e810 con 0x7fd36c0043b0 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.671+0000 7fd382de0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd37c0d0050 con 0x7fd37c142460 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.671+0000 7fd37bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c141fe0 0x7fd37c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.671+0000 7fd37bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c141fe0 0x7fd37c0b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35814/0 (socket says 172.21.15.172:35814) 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.671+0000 7fd37bfff640 1 -- 172.21.15.172:0/1725589628 learned_addr learned my addr 172.21.15.172:0/1725589628 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd380b7c640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0043b0 0x7fd37c0b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd37bfff640 1 -- 172.21.15.172:0/1725589628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd37c142460 msgr2=0x7fd37c0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd38137d640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd37c142460 0x7fd37c0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd37bfff640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd37c142460 0x7fd37c0b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd37bfff640 1 -- 172.21.15.172:0/1725589628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0043b0 msgr2=0x7fd37c0b66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd37bfff640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0043b0 0x7fd37c0b66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd37bfff640 1 -- 172.21.15.172:0/1725589628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37c15fb80 con 0x7fd37c141fe0 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd380b7c640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0043b0 0x7fd37c0b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:19.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd38137d640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd37c142460 0x7fd37c0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:19.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd37bfff640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c141fe0 0x7fd37c0b3a10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd368000c00 tx=0x7fd368004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:19.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd3797fa640 1 -- 172.21.15.172:0/1725589628 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368042020 con 0x7fd37c141fe0 2024-04-14T08:35:19.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd3797fa640 1 -- 172.21.15.172:0/1725589628 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd36803a040 con 0x7fd37c141fe0 2024-04-14T08:35:19.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd3797fa640 1 -- 172.21.15.172:0/1725589628 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368039070 con 0x7fd37c141fe0 2024-04-14T08:35:19.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd382de0640 1 -- 172.21.15.172:0/1725589628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37c15fe20 con 0x7fd37c141fe0 2024-04-14T08:35:19.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.672+0000 7fd382de0640 1 -- 172.21.15.172:0/1725589628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd37c160900 con 0x7fd37c141fe0 2024-04-14T08:35:19.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.673+0000 7fd382de0640 1 -- 172.21.15.172:0/1725589628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd37c068590 con 0x7fd37c141fe0 2024-04-14T08:35:19.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.673+0000 7fd3797fa640 1 -- 172.21.15.172:0/1725589628 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd368045020 con 0x7fd37c141fe0 2024-04-14T08:35:19.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.674+0000 7fd3797fa640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd36006c920 0x7fd36006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:19.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.674+0000 7fd380b7c640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd36006c920 0x7fd36006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:19.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.674+0000 7fd3797fa640 1 -- 172.21.15.172:0/1725589628 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fd3680ad450 con 0x7fd37c141fe0 2024-04-14T08:35:19.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.674+0000 7fd380b7c640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd36006c920 0x7fd36006ede0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd370007d90 tx=0x7fd370009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:19.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.678+0000 7fd3797fa640 1 -- 172.21.15.172:0/1725589628 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd36804c030 con 0x7fd37c141fe0 2024-04-14T08:35:19.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.872+0000 7fd382de0640 1 -- 172.21.15.172:0/1725589628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd37c0d18a0 con 0x7fd37c141fe0 2024-04-14T08:35:19.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.872+0000 7fd3797fa640 1 -- 172.21.15.172:0/1725589628 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) ==== 80+0+0 (secure 0 0 0) 0x7fd36807b6f0 con 0x7fd37c141fe0 2024-04-14T08:35:19.873 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:35:19.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.874+0000 7fd382de0640 1 -- 172.21.15.172:0/1725589628 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd36006c920 msgr2=0x7fd36006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.875+0000 7fd382de0640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd36006c920 0x7fd36006ede0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd370007d90 tx=0x7fd370009040 comp rx=0 tx=0).stop 2024-04-14T08:35:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.875+0000 7fd382de0640 1 -- 172.21.15.172:0/1725589628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c141fe0 msgr2=0x7fd37c0b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.875+0000 7fd382de0640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c141fe0 0x7fd37c0b3a10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd368000c00 tx=0x7fd368004730 comp rx=0 tx=0).stop 2024-04-14T08:35:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.875+0000 7fd382de0640 1 -- 172.21.15.172:0/1725589628 shutdown_connections 2024-04-14T08:35:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.875+0000 7fd382de0640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd36006c920 0x7fd36006ede0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.875+0000 7fd382de0640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd37c142460 0x7fd37c0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.875+0000 7fd382de0640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c141fe0 0x7fd37c0b3a10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.875+0000 7fd382de0640 1 --2- 172.21.15.172:0/1725589628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c0043b0 0x7fd37c0b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.875+0000 7fd382de0640 1 -- 172.21.15.172:0/1725589628 >> 172.21.15.172:0/1725589628 conn(0x7fd37c09d560 msgr2=0x7fd37c0d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.875+0000 7fd382de0640 1 -- 172.21.15.172:0/1725589628 shutdown_connections 2024-04-14T08:35:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:19.875+0000 7fd382de0640 1 -- 172.21.15.172:0/1725589628 wait complete. 2024-04-14T08:35:19.886 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:35:19.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:19.886 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:35:19.929 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:35:19.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:19.929 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:35:19.985 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:35:19.985 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:35:19.985 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:35:20.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:35:20.003 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:35:20.003 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:35:20.003 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:35:20.040 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:35:20.040 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:35:20.040 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:35:20.041 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:35:20.059 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:35:20.059 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:35:20.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.182+0000 7fbc291bb640 1 Processor -- start 2024-04-14T08:35:20.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.182+0000 7fbc291bb640 1 -- start start 2024-04-14T08:35:20.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.183+0000 7fbc291bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbc240d2830 0x7fbc240d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.183+0000 7fbc291bb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbc240cedd0 0x7fbc240b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.183+0000 7fbc291bb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc240d3150 con 0x7fbc2415e890 2024-04-14T08:35:20.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.183+0000 7fbc291bb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fbc240cff30 con 0x7fbc240d2830 2024-04-14T08:35:20.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.183+0000 7fbc291bb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fbc240d2350 con 0x7fbc240cedd0 2024-04-14T08:35:20.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.183+0000 7fbc22d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbc240d2830 0x7fbc240d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.183+0000 7fbc22d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbc240d2830 0x7fbc240d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35818/0 (socket says 172.21.15.172:35818) 2024-04-14T08:35:20.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.183+0000 7fbc22d9d640 1 -- 172.21.15.172:0/554642081 learned_addr learned my addr 172.21.15.172:0/554642081 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:20.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.183+0000 7fbc2359e640 1 --2- 172.21.15.172:0/554642081 >> v2:172.21.15.177:3300/0 conn(0x7fbc240cedd0 0x7fbc240b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.184+0000 7fbc22d9d640 1 -- 172.21.15.172:0/554642081 >> v2:172.21.15.177:3300/0 conn(0x7fbc240cedd0 msgr2=0x7fbc240b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:20.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.184+0000 7fbc22d9d640 1 --2- 172.21.15.172:0/554642081 >> v2:172.21.15.177:3300/0 conn(0x7fbc240cedd0 0x7fbc240b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.184+0000 7fbc22d9d640 1 -- 172.21.15.172:0/554642081 >> v1:172.21.15.177:6789/0 conn(0x7fbc2415e890 legacy=0x7fbc240d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.184+0000 7fbc22d9d640 1 -- 172.21.15.172:0/554642081 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc240cf320 con 0x7fbc240d2830 2024-04-14T08:35:20.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.185+0000 7fbc22d9d640 1 --2- 172.21.15.172:0/554642081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc240d2830 0x7fbc240d2c10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbc2408e4b0 tx=0x7fbc1c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e6bc70fd23a620 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:20.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.185+0000 7fbc21d9b640 1 -- 172.21.15.172:0/554642081 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c02f850 con 0x7fbc240d2830 2024-04-14T08:35:20.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.185+0000 7fbc21d9b640 1 -- 172.21.15.172:0/554642081 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1c02f9f0 con 0x7fbc240d2830 2024-04-14T08:35:20.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.185+0000 7fbc21d9b640 1 -- 172.21.15.172:0/554642081 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c02fdc0 con 0x7fbc240d2830 2024-04-14T08:35:20.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.185+0000 7fbc291bb640 1 -- 172.21.15.172:0/554642081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc240d2830 msgr2=0x7fbc240d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.185+0000 7fbc291bb640 1 --2- 172.21.15.172:0/554642081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc240d2830 0x7fbc240d2c10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbc2408e4b0 tx=0x7fbc1c02ba30 comp rx=0 tx=0).stop 2024-04-14T08:35:20.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.185+0000 7fbc291bb640 1 -- 172.21.15.172:0/554642081 shutdown_connections 2024-04-14T08:35:20.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.185+0000 7fbc291bb640 1 --2- 172.21.15.172:0/554642081 >> v2:172.21.15.177:3300/0 conn(0x7fbc240cedd0 0x7fbc240b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.185+0000 7fbc291bb640 1 --2- 172.21.15.172:0/554642081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc240d2830 0x7fbc240d2c10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.185+0000 7fbc291bb640 1 -- 172.21.15.172:0/554642081 >> 172.21.15.172:0/554642081 conn(0x7fbc2409d590 msgr2=0x7fbc240d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:20.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.185+0000 7fbc291bb640 1 -- 172.21.15.172:0/554642081 shutdown_connections 2024-04-14T08:35:20.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.186+0000 7fbc291bb640 1 -- 172.21.15.172:0/554642081 wait complete. 2024-04-14T08:35:20.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.187+0000 7fbc291bb640 1 Processor -- start 2024-04-14T08:35:20.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.187+0000 7fbc291bb640 1 -- start start 2024-04-14T08:35:20.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc291bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc240cedd0 0x7fbc240b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc291bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc240d2830 0x7fbc2414c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc291bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc2415e890 0x7fbc2414ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc291bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbc240cfbc0 con 0x7fbc2415e890 2024-04-14T08:35:20.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc291bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbc240cf8e0 con 0x7fbc240d2830 2024-04-14T08:35:20.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc291bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbc240b3c50 con 0x7fbc240cedd0 2024-04-14T08:35:20.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc2359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc2415e890 0x7fbc2414ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc2359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc2415e890 0x7fbc2414ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35832/0 (socket says 172.21.15.172:35832) 2024-04-14T08:35:20.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc2359e640 1 -- 172.21.15.172:0/107616318 learned_addr learned my addr 172.21.15.172:0/107616318 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:20.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc22d9d640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc240cedd0 0x7fbc240b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc2259c640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc240d2830 0x7fbc2414c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc2359e640 1 -- 172.21.15.172:0/107616318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc240cedd0 msgr2=0x7fbc240b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc2359e640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc240cedd0 0x7fbc240b6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc2359e640 1 -- 172.21.15.172:0/107616318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc240d2830 msgr2=0x7fbc2414c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc2359e640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc240d2830 0x7fbc2414c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc2359e640 1 -- 172.21.15.172:0/107616318 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2415ffb0 con 0x7fbc2415e890 2024-04-14T08:35:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc2359e640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc2415e890 0x7fbc2414ce90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400e930 tx=0x7fbc1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc0b7fe640 1 -- 172.21.15.172:0/107616318 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14017070 con 0x7fbc2415e890 2024-04-14T08:35:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc0b7fe640 1 -- 172.21.15.172:0/107616318 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1400c980 con 0x7fbc2415e890 2024-04-14T08:35:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc291bb640 1 -- 172.21.15.172:0/107616318 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc24160290 con 0x7fbc2415e890 2024-04-14T08:35:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.189+0000 7fbc0b7fe640 1 -- 172.21.15.172:0/107616318 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1400cd70 con 0x7fbc2415e890 2024-04-14T08:35:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.190+0000 7fbc291bb640 1 -- 172.21.15.172:0/107616318 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc24160850 con 0x7fbc2415e890 2024-04-14T08:35:20.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.190+0000 7fbc291bb640 1 -- 172.21.15.172:0/107616318 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc24068520 con 0x7fbc2415e890 2024-04-14T08:35:20.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.191+0000 7fbc0b7fe640 1 -- 172.21.15.172:0/107616318 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc1401a070 con 0x7fbc2415e890 2024-04-14T08:35:20.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.191+0000 7fbc0b7fe640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbc1006c8f0 0x7fbc1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.191+0000 7fbc22d9d640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbc1006c8f0 0x7fbc1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.191+0000 7fbc0b7fe640 1 -- 172.21.15.172:0/107616318 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fbc1408c070 con 0x7fbc2415e890 2024-04-14T08:35:20.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.192+0000 7fbc22d9d640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbc1006c8f0 0x7fbc1006edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c036000 tx=0x7fbc1c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:20.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.200+0000 7fbc0b7fe640 1 -- 172.21.15.172:0/107616318 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbc14059d00 con 0x7fbc2415e890 2024-04-14T08:35:20.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.415+0000 7fbc291bb640 1 -- 172.21.15.172:0/107616318 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc240b21d0 con 0x7fbc2415e890 2024-04-14T08:35:20.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.416+0000 7fbc0b7fe640 1 -- 172.21.15.172:0/107616318 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5187 (secure 0 0 0) 0x7fbc1405e000 con 0x7fbc2415e890 2024-04-14T08:35:20.417 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:20.417 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/296861286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":296861286},{"type":"v1","addr":"172.21.15.177:6835","nonce":296861286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/642342704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":642342704},{"type":"v1","addr":"172.21.15.177:6833","nonce":642342704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3516903987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3516903987},{"type":"v1","addr":"172.21.15.172:6836","nonce":3516903987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:34:34.765057+0000","modified":"2024-04-14T08:34:36.783999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6837/1527574817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1527574817},{"type":"v1","addr":"172.21.15.172:6837","nonce":1527574817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":3}]} 2024-04-14T08:35:20.417 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 23 2024-04-14T08:35:20.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.418+0000 7fbc291bb640 1 -- 172.21.15.172:0/107616318 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbc1006c8f0 msgr2=0x7fbc1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.418+0000 7fbc291bb640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbc1006c8f0 0x7fbc1006edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c036000 tx=0x7fbc1c008030 comp rx=0 tx=0).stop 2024-04-14T08:35:20.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.418+0000 7fbc291bb640 1 -- 172.21.15.172:0/107616318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc2415e890 msgr2=0x7fbc2414ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.418+0000 7fbc291bb640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc2415e890 0x7fbc2414ce90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400e930 tx=0x7fbc1400edf0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.418+0000 7fbc291bb640 1 -- 172.21.15.172:0/107616318 shutdown_connections 2024-04-14T08:35:20.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.418+0000 7fbc291bb640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbc1006c8f0 0x7fbc1006edb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.418+0000 7fbc291bb640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc2415e890 0x7fbc2414ce90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.418+0000 7fbc291bb640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc240d2830 0x7fbc2414c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.418+0000 7fbc291bb640 1 --2- 172.21.15.172:0/107616318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc240cedd0 0x7fbc240b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.418+0000 7fbc291bb640 1 -- 172.21.15.172:0/107616318 >> 172.21.15.172:0/107616318 conn(0x7fbc2409d590 msgr2=0x7fbc2415c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:20.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.418+0000 7fbc291bb640 1 -- 172.21.15.172:0/107616318 shutdown_connections 2024-04-14T08:35:20.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.418+0000 7fbc291bb640 1 -- 172.21.15.172:0/107616318 wait complete. 2024-04-14T08:35:20.430 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:35:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.532+0000 7f261e715640 1 Processor -- start 2024-04-14T08:35:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.532+0000 7f261e715640 1 -- start start 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261e715640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f261815c710 0x7f26180d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261e715640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f26180cde10 0x7f26180ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261e715640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26180ced80 0x7f26180ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261e715640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f26180aed10 con 0x7f26180ced80 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261e715640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f26180af890 con 0x7f26180cde10 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261e715640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f26180aef60 con 0x7f261815c710 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261ccb2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26180ced80 0x7f26180ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261ccb2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26180ced80 0x7f26180ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35840/0 (socket says 172.21.15.172:35840) 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261ccb2640 1 -- 172.21.15.172:0/4173144110 learned_addr learned my addr 172.21.15.172:0/4173144110 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f26177fe640 1 --2- 172.21.15.172:0/4173144110 >> v2:172.21.15.177:3300/0 conn(0x7f26180cde10 0x7f26180ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f2617fff640 1 --2- 172.21.15.172:0/4173144110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f261815c710 0x7f26180d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261ccb2640 1 -- 172.21.15.172:0/4173144110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f261815c710 msgr2=0x7f26180d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261ccb2640 1 --2- 172.21.15.172:0/4173144110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f261815c710 0x7f26180d0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261ccb2640 1 -- 172.21.15.172:0/4173144110 >> v2:172.21.15.177:3300/0 conn(0x7f26180cde10 msgr2=0x7f26180ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261ccb2640 1 --2- 172.21.15.172:0/4173144110 >> v2:172.21.15.177:3300/0 conn(0x7f26180cde10 0x7f26180ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.533+0000 7f261ccb2640 1 -- 172.21.15.172:0/4173144110 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26180af240 con 0x7f26180ced80 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.534+0000 7f261ccb2640 1 --2- 172.21.15.172:0/4173144110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26180ced80 0x7f26180ccc90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f261806c500 tx=0x7f261002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3010305a2da6b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.534+0000 7f2616ffd640 1 -- 172.21.15.172:0/4173144110 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261002ca70 con 0x7f26180ced80 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.534+0000 7f2616ffd640 1 -- 172.21.15.172:0/4173144110 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f261002cc10 con 0x7f26180ced80 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.534+0000 7f2616ffd640 1 -- 172.21.15.172:0/4173144110 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100343f0 con 0x7f26180ced80 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.535+0000 7f261e715640 1 -- 172.21.15.172:0/4173144110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26180ced80 msgr2=0x7f26180ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.535+0000 7f261e715640 1 --2- 172.21.15.172:0/4173144110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26180ced80 0x7f26180ccc90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f261806c500 tx=0x7f261002bed0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.535+0000 7f261e715640 1 -- 172.21.15.172:0/4173144110 shutdown_connections 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.535+0000 7f261e715640 1 --2- 172.21.15.172:0/4173144110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26180ced80 0x7f26180ccc90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.535+0000 7f261e715640 1 --2- 172.21.15.172:0/4173144110 >> v2:172.21.15.177:3300/0 conn(0x7f26180cde10 0x7f26180ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.535+0000 7f261e715640 1 --2- 172.21.15.172:0/4173144110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f261815c710 0x7f26180d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.535+0000 7f261e715640 1 -- 172.21.15.172:0/4173144110 >> 172.21.15.172:0/4173144110 conn(0x7f261808c000 msgr2=0x7f26180d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:20.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.535+0000 7f261e715640 1 -- 172.21.15.172:0/4173144110 shutdown_connections 2024-04-14T08:35:20.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.535+0000 7f261e715640 1 -- 172.21.15.172:0/4173144110 wait complete. 2024-04-14T08:35:20.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.536+0000 7f261e715640 1 Processor -- start 2024-04-14T08:35:20.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.536+0000 7f261e715640 1 -- start start 2024-04-14T08:35:20.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f261e715640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26180cde10 0x7f26181f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f261e715640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26180ced80 0x7f261815e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f261e715640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f261815c710 0x7f26181f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f261e715640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26180d0e90 con 0x7f26180ced80 2024-04-14T08:35:20.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f261e715640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f261815caf0 con 0x7f261815c710 2024-04-14T08:35:20.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f261e715640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f26180d1010 con 0x7f26180cde10 2024-04-14T08:35:20.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f26177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26180ced80 0x7f261815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f26177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26180ced80 0x7f261815e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35848/0 (socket says 172.21.15.172:35848) 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f26177fe640 1 -- 172.21.15.172:0/3154171050 learned_addr learned my addr 172.21.15.172:0/3154171050 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f261ccb2640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f261815c710 0x7f26181f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f2617fff640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26180cde10 0x7f26181f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f26177fe640 1 -- 172.21.15.172:0/3154171050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26180cde10 msgr2=0x7f26181f2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f26177fe640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26180cde10 0x7f26181f2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f26177fe640 1 -- 172.21.15.172:0/3154171050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f261815c710 msgr2=0x7f26181f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f26177fe640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f261815c710 0x7f26181f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.537+0000 7f26177fe640 1 -- 172.21.15.172:0/3154171050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26180b4d70 con 0x7f26180ced80 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.538+0000 7f26177fe640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26180ced80 0x7f261815e630 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f260800c7f0 tx=0x7f260800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.538+0000 7f2614ff9640 1 -- 172.21.15.172:0/3154171050 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608015070 con 0x7f26180ced80 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.538+0000 7f2614ff9640 1 -- 172.21.15.172:0/3154171050 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2608004220 con 0x7f26180ced80 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.538+0000 7f261e715640 1 -- 172.21.15.172:0/3154171050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26180b5050 con 0x7f26180ced80 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.538+0000 7f2614ff9640 1 -- 172.21.15.172:0/3154171050 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608007620 con 0x7f26180ced80 2024-04-14T08:35:20.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.538+0000 7f261e715640 1 -- 172.21.15.172:0/3154171050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26180b58f0 con 0x7f26180ced80 2024-04-14T08:35:20.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.539+0000 7f261e715640 1 -- 172.21.15.172:0/3154171050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2618075490 con 0x7f26180ced80 2024-04-14T08:35:20.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.539+0000 7f2614ff9640 1 -- 172.21.15.172:0/3154171050 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26080079d0 con 0x7f26180ced80 2024-04-14T08:35:20.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.540+0000 7f2614ff9640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f260406c920 0x7f260406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.540+0000 7f2617fff640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f260406c920 0x7f260406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.540+0000 7f2614ff9640 1 -- 172.21.15.172:0/3154171050 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f260808bd10 con 0x7f26180ced80 2024-04-14T08:35:20.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.540+0000 7f2617fff640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f260406c920 0x7f260406ede0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f261815f6e0 tx=0x7f260c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:20.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.544+0000 7f2614ff9640 1 -- 172.21.15.172:0/3154171050 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f26080598e0 con 0x7f26180ced80 2024-04-14T08:35:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.771+0000 7f261e715640 1 -- 172.21.15.172:0/3154171050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26180b5ac0 con 0x7f26180ced80 2024-04-14T08:35:20.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.772+0000 7f2614ff9640 1 -- 172.21.15.172:0/3154171050 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+17002 (secure 0 0 0) 0x7f260805dbe0 con 0x7f26180ced80 2024-04-14T08:35:20.773 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:20.773 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":50,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:36.423005+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:34:32.990827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-14T08:34:33.715803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:35:20.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.774+0000 7f261e715640 1 -- 172.21.15.172:0/3154171050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f260406c920 msgr2=0x7f260406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.774+0000 7f261e715640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f260406c920 0x7f260406ede0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f261815f6e0 tx=0x7f260c008040 comp rx=0 tx=0).stop 2024-04-14T08:35:20.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.774+0000 7f261e715640 1 -- 172.21.15.172:0/3154171050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26180ced80 msgr2=0x7f261815e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.774+0000 7f261e715640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26180ced80 0x7f261815e630 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f260800c7f0 tx=0x7f260800ccb0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.774+0000 7f261e715640 1 -- 172.21.15.172:0/3154171050 shutdown_connections 2024-04-14T08:35:20.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.774+0000 7f261e715640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f260406c920 0x7f260406ede0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.774+0000 7f261e715640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f261815c710 0x7f26181f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.774+0000 7f261e715640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26180ced80 0x7f261815e630 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.774+0000 7f261e715640 1 --2- 172.21.15.172:0/3154171050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26180cde10 0x7f26181f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.774+0000 7f261e715640 1 -- 172.21.15.172:0/3154171050 >> 172.21.15.172:0/3154171050 conn(0x7f261808c000 msgr2=0x7f26180ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:20.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.775+0000 7f261e715640 1 -- 172.21.15.172:0/3154171050 shutdown_connections 2024-04-14T08:35:20.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.775+0000 7f261e715640 1 -- 172.21.15.172:0/3154171050 wait complete. 2024-04-14T08:35:20.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:34:32.990827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:34:33.715803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:35:20.787 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:35:20.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.891+0000 7f17df288640 1 Processor -- start 2024-04-14T08:35:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.891+0000 7f17df288640 1 -- start start 2024-04-14T08:35:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.892+0000 7f17df288640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f17d80d49c0 0x7f17d80d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.892+0000 7f17df288640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d8160f30 0x7f17d81613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.892+0000 7f17df288640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17d80cf0f0 con 0x7f17d80d0cc0 2024-04-14T08:35:20.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.892+0000 7f17df288640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f17d80ceec0 con 0x7f17d80d49c0 2024-04-14T08:35:20.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.892+0000 7f17df288640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f17d80cfd00 con 0x7f17d8160f30 2024-04-14T08:35:20.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.892+0000 7f17dd825640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d8160f30 0x7f17d81613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.892+0000 7f17dc823640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f17d80d49c0 0x7f17d80d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.892+0000 7f17dd825640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d8160f30 0x7f17d81613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:47106/0 (socket says 172.21.15.172:47106) 2024-04-14T08:35:20.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.892+0000 7f17dd825640 1 -- 172.21.15.172:0/2664937203 learned_addr learned my addr 172.21.15.172:0/2664937203 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:20.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.893+0000 7f17dd825640 1 -- 172.21.15.172:0/2664937203 >> v2:172.21.15.177:3300/0 conn(0x7f17d80d49c0 msgr2=0x7f17d80d4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:20.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.893+0000 7f17dd825640 1 --2- 172.21.15.172:0/2664937203 >> v2:172.21.15.177:3300/0 conn(0x7f17d80d49c0 0x7f17d80d4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.893+0000 7f17dd825640 1 -- 172.21.15.172:0/2664937203 >> v1:172.21.15.177:6789/0 conn(0x7f17d80d0cc0 legacy=0x7f17d80d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.893+0000 7f17dd825640 1 -- 172.21.15.172:0/2664937203 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d80cf3d0 con 0x7f17d8160f30 2024-04-14T08:35:20.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.893+0000 7f17dd825640 1 --2- 172.21.15.172:0/2664937203 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d8160f30 0x7f17d81613c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009870 tx=0x7f17d402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4789a7d61eae55d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:20.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.894+0000 7f17cffff640 1 -- 172.21.15.172:0/2664937203 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4034070 con 0x7f17d8160f30 2024-04-14T08:35:20.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.894+0000 7f17cffff640 1 -- 172.21.15.172:0/2664937203 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d4030840 con 0x7f17d8160f30 2024-04-14T08:35:20.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.894+0000 7f17cffff640 1 -- 172.21.15.172:0/2664937203 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4030c10 con 0x7f17d8160f30 2024-04-14T08:35:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.894+0000 7f17df288640 1 -- 172.21.15.172:0/2664937203 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d8160f30 msgr2=0x7f17d81613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.894+0000 7f17df288640 1 --2- 172.21.15.172:0/2664937203 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d8160f30 0x7f17d81613c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009870 tx=0x7f17d402b980 comp rx=0 tx=0).stop 2024-04-14T08:35:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.894+0000 7f17df288640 1 -- 172.21.15.172:0/2664937203 shutdown_connections 2024-04-14T08:35:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.894+0000 7f17df288640 1 --2- 172.21.15.172:0/2664937203 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d8160f30 0x7f17d81613c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.894+0000 7f17df288640 1 --2- 172.21.15.172:0/2664937203 >> v2:172.21.15.177:3300/0 conn(0x7f17d80d49c0 0x7f17d80d4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.894+0000 7f17df288640 1 -- 172.21.15.172:0/2664937203 >> 172.21.15.172:0/2664937203 conn(0x7f17d809d580 msgr2=0x7f17d80d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.894+0000 7f17df288640 1 -- 172.21.15.172:0/2664937203 shutdown_connections 2024-04-14T08:35:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.894+0000 7f17df288640 1 -- 172.21.15.172:0/2664937203 wait complete. 2024-04-14T08:35:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.895+0000 7f17df288640 1 Processor -- start 2024-04-14T08:35:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.896+0000 7f17df288640 1 -- start start 2024-04-14T08:35:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.896+0000 7f17df288640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d80d0cc0 0x7f17d81f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.896+0000 7f17df288640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d80d49c0 0x7f17d81f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.896+0000 7f17df288640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d8160f30 0x7f17d81f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.896+0000 7f17df288640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17d80d32b0 con 0x7f17d80d0cc0 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.896+0000 7f17df288640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f17d80cf990 con 0x7f17d8160f30 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.896+0000 7f17df288640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f17d80d0190 con 0x7f17d80d49c0 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.896+0000 7f17dd024640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d80d0cc0 0x7f17d81f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.896+0000 7f17dd024640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d80d0cc0 0x7f17d81f54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35858/0 (socket says 172.21.15.172:35858) 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.896+0000 7f17dd024640 1 -- 172.21.15.172:0/3415974600 learned_addr learned my addr 172.21.15.172:0/3415974600 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.896+0000 7f17dc823640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d80d49c0 0x7f17d81f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.896+0000 7f17dd825640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d8160f30 0x7f17d81f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.897+0000 7f17dd024640 1 -- 172.21.15.172:0/3415974600 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d80d49c0 msgr2=0x7f17d81f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.897+0000 7f17dd024640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d80d49c0 0x7f17d81f4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.897+0000 7f17dd024640 1 -- 172.21.15.172:0/3415974600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d8160f30 msgr2=0x7f17d81f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.897+0000 7f17dd024640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d8160f30 0x7f17d81f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.897+0000 7f17dd024640 1 -- 172.21.15.172:0/3415974600 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d81f52a0 con 0x7f17d80d0cc0 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.897+0000 7f17dd024640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d80d0cc0 0x7f17d81f54a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f17d000c790 tx=0x7f17d000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.897+0000 7f17cdffb640 1 -- 172.21.15.172:0/3415974600 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0016020 con 0x7f17d80d0cc0 2024-04-14T08:35:20.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.897+0000 7f17cdffb640 1 -- 172.21.15.172:0/3415974600 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d0006d50 con 0x7f17d80d0cc0 2024-04-14T08:35:20.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.897+0000 7f17cdffb640 1 -- 172.21.15.172:0/3415974600 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0003bb0 con 0x7f17d80d0cc0 2024-04-14T08:35:20.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.897+0000 7f17df288640 1 -- 172.21.15.172:0/3415974600 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d81f1ce0 con 0x7f17d80d0cc0 2024-04-14T08:35:20.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.897+0000 7f17df288640 1 -- 172.21.15.172:0/3415974600 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d81f6040 con 0x7f17d80d0cc0 2024-04-14T08:35:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.898+0000 7f17cdffb640 1 -- 172.21.15.172:0/3415974600 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17d000a850 con 0x7f17d80d0cc0 2024-04-14T08:35:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.899+0000 7f17df288640 1 -- 172.21.15.172:0/3415974600 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d8075490 con 0x7f17d80d0cc0 2024-04-14T08:35:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.899+0000 7f17cdffb640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f17bc06c8f0 0x7f17bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.899+0000 7f17cdffb640 1 -- 172.21.15.172:0/3415974600 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f17d008b820 con 0x7f17d80d0cc0 2024-04-14T08:35:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.899+0000 7f17dc823640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f17bc06c8f0 0x7f17bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.899+0000 7f17dc823640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f17bc06c8f0 0x7f17bc06edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f17d80b6910 tx=0x7f17c8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:20.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:20.903+0000 7f17cdffb640 1 -- 172.21.15.172:0/3415974600 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f17d0020020 con 0x7f17d80d0cc0 2024-04-14T08:35:21.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.082+0000 7f17df288640 1 -- 172.21.15.172:0/3415974600 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f17d80b9670 con 0x7f17d80d0cc0 2024-04-14T08:35:21.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.083+0000 7f17cdffb640 1 -- 172.21.15.172:0/3415974600 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7f17d81f52a0 con 0x7f17d80d0cc0 2024-04-14T08:35:21.083 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:35:21.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.084+0000 7f17df288640 1 -- 172.21.15.172:0/3415974600 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f17bc06c8f0 msgr2=0x7f17bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.084+0000 7f17df288640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f17bc06c8f0 0x7f17bc06edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f17d80b6910 tx=0x7f17c8009190 comp rx=0 tx=0).stop 2024-04-14T08:35:21.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.084+0000 7f17df288640 1 -- 172.21.15.172:0/3415974600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d80d0cc0 msgr2=0x7f17d81f54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.084+0000 7f17df288640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d80d0cc0 0x7f17d81f54a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f17d000c790 tx=0x7f17d000cc50 comp rx=0 tx=0).stop 2024-04-14T08:35:21.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.084+0000 7f17df288640 1 -- 172.21.15.172:0/3415974600 shutdown_connections 2024-04-14T08:35:21.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.085+0000 7f17df288640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f17bc06c8f0 0x7f17bc06edb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.085+0000 7f17df288640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d8160f30 0x7f17d81f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.085+0000 7f17df288640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17d80d49c0 0x7f17d81f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.085+0000 7f17df288640 1 --2- 172.21.15.172:0/3415974600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d80d0cc0 0x7f17d81f54a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.085+0000 7f17df288640 1 -- 172.21.15.172:0/3415974600 >> 172.21.15.172:0/3415974600 conn(0x7f17d809d580 msgr2=0x7f17d80b8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:21.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.085+0000 7f17df288640 1 -- 172.21.15.172:0/3415974600 shutdown_connections 2024-04-14T08:35:21.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.085+0000 7f17df288640 1 -- 172.21.15.172:0/3415974600 wait complete. 2024-04-14T08:35:21.096 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:35:21.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.200+0000 7f940bc76640 1 Processor -- start 2024-04-14T08:35:21.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.200+0000 7f940bc76640 1 -- start start 2024-04-14T08:35:21.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f940bc76640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f940415e430 0x7f9404162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f940bc76640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94040cefb0 con 0x7f94040d2e00 2024-04-14T08:35:21.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f940bc76640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94040cf290 con 0x7f9404162540 2024-04-14T08:35:21.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f940bc76640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f94040cf570 con 0x7f940415e430 2024-04-14T08:35:21.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f940a213640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f94040d2e00 0x7f94040b4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50844/0 (socket says 172.21.15.172:50844) 2024-04-14T08:35:21.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f940a213640 1 -- 172.21.15.172:0/2830647922 learned_addr learned my addr 172.21.15.172:0/2830647922 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:21.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f9409a12640 1 --2- 172.21.15.172:0/2830647922 >> v2:172.21.15.177:3300/0 conn(0x7f940415e430 0x7f9404162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f9408a10640 1 -- 172.21.15.172:0/2830647922 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1435457558 0 0) 0x7f94040cefb0 con 0x7f94040d2e00 2024-04-14T08:35:21.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f9408a10640 1 -- 172.21.15.172:0/2830647922 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93f0003520 con 0x7f94040d2e00 2024-04-14T08:35:21.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f9408a10640 1 -- 172.21.15.172:0/2830647922 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3973778884 0 0) 0x7f94040cf290 con 0x7f9404162540 2024-04-14T08:35:21.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f9408a10640 1 -- 172.21.15.172:0/2830647922 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94040cefb0 con 0x7f9404162540 2024-04-14T08:35:21.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f9408a10640 1 -- 172.21.15.172:0/2830647922 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1739817947 0 0) 0x7f93f0003520 con 0x7f94040d2e00 2024-04-14T08:35:21.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f9409a12640 1 -- 172.21.15.172:0/2830647922 >> v1:172.21.15.177:6789/0 conn(0x7f9404162540 legacy=0x7f94040b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f9409a12640 1 -- 172.21.15.172:0/2830647922 >> v1:172.21.15.172:6789/0 conn(0x7f94040d2e00 legacy=0x7f94040b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.201+0000 7f9409a12640 1 -- 172.21.15.172:0/2830647922 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94040cf850 con 0x7f940415e430 2024-04-14T08:35:21.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.202+0000 7f9409a12640 1 --2- 172.21.15.172:0/2830647922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f940415e430 0x7f9404162000 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94000097a0 tx=0x7f940002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f19589b15aca802 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:21.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.202+0000 7f9408a10640 1 -- 172.21.15.172:0/2830647922 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400038070 con 0x7f940415e430 2024-04-14T08:35:21.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.202+0000 7f9408a10640 1 -- 172.21.15.172:0/2830647922 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9400031500 con 0x7f940415e430 2024-04-14T08:35:21.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.202+0000 7f9408a10640 1 -- 172.21.15.172:0/2830647922 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc0019f0 con 0x7f940415e430 2024-04-14T08:35:21.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.203+0000 7f940bc76640 1 -- 172.21.15.172:0/2830647922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f940415e430 msgr2=0x7f9404162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.203+0000 7f940bc76640 1 --2- 172.21.15.172:0/2830647922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f940415e430 0x7f9404162000 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94000097a0 tx=0x7f940002b820 comp rx=0 tx=0).stop 2024-04-14T08:35:21.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.203+0000 7f940bc76640 1 -- 172.21.15.172:0/2830647922 shutdown_connections 2024-04-14T08:35:21.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.203+0000 7f940bc76640 1 --2- 172.21.15.172:0/2830647922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f940415e430 0x7f9404162000 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.203+0000 7f940bc76640 1 -- 172.21.15.172:0/2830647922 >> 172.21.15.172:0/2830647922 conn(0x7f940409d560 msgr2=0x7f94040d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:21.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.203+0000 7f940bc76640 1 -- 172.21.15.172:0/2830647922 shutdown_connections 2024-04-14T08:35:21.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.203+0000 7f940bc76640 1 -- 172.21.15.172:0/2830647922 wait complete. 2024-04-14T08:35:21.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.204+0000 7f940bc76640 1 Processor -- start 2024-04-14T08:35:21.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.204+0000 7f940bc76640 1 -- start start 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f940bc76640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040d2e00 0x7f94040b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f940bc76640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f940415e430 0x7f94040c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f940bc76640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9404162540 0x7f94041525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f940bc76640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94040d4e20 con 0x7f94040d2e00 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f940bc76640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f940415e810 con 0x7f940415e430 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f940bc76640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f94040b4850 con 0x7f9404162540 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f9409a12640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040d2e00 0x7f94040b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f9409a12640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040d2e00 0x7f94040b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35872/0 (socket says 172.21.15.172:35872) 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f9409a12640 1 -- 172.21.15.172:0/1352910094 learned_addr learned my addr 172.21.15.172:0/1352910094 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f940a213640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9404162540 0x7f94041525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f9409211640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f940415e430 0x7f94040c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f9409a12640 1 -- 172.21.15.172:0/1352910094 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9404162540 msgr2=0x7f94041525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f9409a12640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9404162540 0x7f94041525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f9409a12640 1 -- 172.21.15.172:0/1352910094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f940415e430 msgr2=0x7f94040c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f9409a12640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f940415e430 0x7f94040c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f9409a12640 1 -- 172.21.15.172:0/1352910094 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94040b7f40 con 0x7f94040d2e00 2024-04-14T08:35:21.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f9409a12640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040d2e00 0x7f94040b2710 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9400036040 tx=0x7f94000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:21.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f93fa7fc640 1 -- 172.21.15.172:0/1352910094 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400042070 con 0x7f94040d2e00 2024-04-14T08:35:21.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f93fa7fc640 1 -- 172.21.15.172:0/1352910094 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f940003d870 con 0x7f94040d2e00 2024-04-14T08:35:21.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f940bc76640 1 -- 172.21.15.172:0/1352910094 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94040c8ce0 con 0x7f94040d2e00 2024-04-14T08:35:21.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.205+0000 7f93fa7fc640 1 -- 172.21.15.172:0/1352910094 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400038040 con 0x7f94040d2e00 2024-04-14T08:35:21.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.206+0000 7f940bc76640 1 -- 172.21.15.172:0/1352910094 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94040c92a0 con 0x7f94040d2e00 2024-04-14T08:35:21.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.207+0000 7f940bc76640 1 -- 172.21.15.172:0/1352910094 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94040c8a60 con 0x7f94040d2e00 2024-04-14T08:35:21.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.207+0000 7f93fa7fc640 1 -- 172.21.15.172:0/1352910094 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9400045070 con 0x7f94040d2e00 2024-04-14T08:35:21.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.208+0000 7f93fa7fc640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f93e806c8f0 0x7f93e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.208+0000 7f9409211640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f93e806c8f0 0x7f93e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.208+0000 7f93fa7fc640 1 -- 172.21.15.172:0/1352910094 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f94000ad650 con 0x7f94040d2e00 2024-04-14T08:35:21.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.208+0000 7f9409211640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f93e806c8f0 0x7f93e806edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9404156960 tx=0x7f93f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:21.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.212+0000 7f93fa7fc640 1 -- 172.21.15.172:0/1352910094 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f94040c8a60 con 0x7f94040d2e00 2024-04-14T08:35:21.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.393+0000 7f940bc76640 1 -- 172.21.15.172:0/1352910094 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9404068590 con 0x7f94040d2e00 2024-04-14T08:35:21.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.393+0000 7f93fa7fc640 1 -- 172.21.15.172:0/1352910094 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) ==== 104+0+11 (secure 0 0 0) 0x7f9400030070 con 0x7f94040d2e00 2024-04-14T08:35:21.394 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:35:21.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.395+0000 7f940bc76640 1 -- 172.21.15.172:0/1352910094 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f93e806c8f0 msgr2=0x7f93e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.395+0000 7f940bc76640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f93e806c8f0 0x7f93e806edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9404156960 tx=0x7f93f4009040 comp rx=0 tx=0).stop 2024-04-14T08:35:21.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.395+0000 7f940bc76640 1 -- 172.21.15.172:0/1352910094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040d2e00 msgr2=0x7f94040b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.395+0000 7f940bc76640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040d2e00 0x7f94040b2710 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9400036040 tx=0x7f94000046c0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.395+0000 7f940bc76640 1 -- 172.21.15.172:0/1352910094 shutdown_connections 2024-04-14T08:35:21.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.395+0000 7f940bc76640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f93e806c8f0 0x7f93e806edb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.395+0000 7f940bc76640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9404162540 0x7f94041525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.395+0000 7f940bc76640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f940415e430 0x7f94040c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.395+0000 7f940bc76640 1 --2- 172.21.15.172:0/1352910094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94040d2e00 0x7f94040b2710 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.395+0000 7f940bc76640 1 -- 172.21.15.172:0/1352910094 >> 172.21.15.172:0/1352910094 conn(0x7f940409d560 msgr2=0x7f94040b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:21.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.395+0000 7f940bc76640 1 -- 172.21.15.172:0/1352910094 shutdown_connections 2024-04-14T08:35:21.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.395+0000 7f940bc76640 1 -- 172.21.15.172:0/1352910094 wait complete. 2024-04-14T08:35:21.406 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:35:21.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.512+0000 7f6ce735e640 1 Processor -- start 2024-04-14T08:35:21.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.512+0000 7f6ce735e640 1 -- start start 2024-04-14T08:35:21.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.513+0000 7f6ce735e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6ce0162ba0 0x7f6ce00d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.513+0000 7f6ce735e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ce00cefb0 con 0x7f6ce0162710 2024-04-14T08:35:21.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.513+0000 7f6ce735e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ce00cf290 con 0x7f6ce015e430 2024-04-14T08:35:21.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.513+0000 7f6ce735e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce00cf570 con 0x7f6ce0162ba0 2024-04-14T08:35:21.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.513+0000 7f6ce58fb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6ce0162ba0 0x7f6ce00d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.513+0000 7f6ce58fb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6ce0162ba0 0x7f6ce00d2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35888/0 (socket says 172.21.15.172:35888) 2024-04-14T08:35:21.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.513+0000 7f6ce58fb640 1 -- 172.21.15.172:0/2813532871 learned_addr learned my addr 172.21.15.172:0/2813532871 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:21.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.513+0000 7f6ce48f9640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6ce0162710 0x7f6ce00d1df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50854/0 (socket says 172.21.15.172:50854) 2024-04-14T08:35:21.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.513+0000 7f6ce58fb640 1 -- 172.21.15.172:0/2813532871 >> v1:172.21.15.177:6789/0 conn(0x7f6ce015e430 legacy=0x7f6ce0162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.513+0000 7f6ce58fb640 1 -- 172.21.15.172:0/2813532871 >> v1:172.21.15.172:6789/0 conn(0x7f6ce0162710 legacy=0x7f6ce00d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.513+0000 7f6ce58fb640 1 -- 172.21.15.172:0/2813532871 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce00cf850 con 0x7f6ce0162ba0 2024-04-14T08:35:21.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.514+0000 7f6ce58fb640 1 --2- 172.21.15.172:0/2813532871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ce0162ba0 0x7f6ce00d2500 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009e50 tx=0x7f6cd802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1c7cc7f4172ac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:21.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.515+0000 7f6cd7fff640 1 -- 172.21.15.172:0/2813532871 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd802cbb0 con 0x7f6ce0162ba0 2024-04-14T08:35:21.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.515+0000 7f6cd7fff640 1 -- 172.21.15.172:0/2813532871 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd802cd50 con 0x7f6ce0162ba0 2024-04-14T08:35:21.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.515+0000 7f6cd7fff640 1 -- 172.21.15.172:0/2813532871 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8034610 con 0x7f6ce0162ba0 2024-04-14T08:35:21.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.516+0000 7f6ce735e640 1 -- 172.21.15.172:0/2813532871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ce0162ba0 msgr2=0x7f6ce00d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.516+0000 7f6ce735e640 1 --2- 172.21.15.172:0/2813532871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ce0162ba0 0x7f6ce00d2500 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009e50 tx=0x7f6cd802c010 comp rx=0 tx=0).stop 2024-04-14T08:35:21.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.516+0000 7f6ce735e640 1 -- 172.21.15.172:0/2813532871 shutdown_connections 2024-04-14T08:35:21.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.516+0000 7f6ce735e640 1 --2- 172.21.15.172:0/2813532871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ce0162ba0 0x7f6ce00d2500 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.516+0000 7f6ce735e640 1 -- 172.21.15.172:0/2813532871 >> 172.21.15.172:0/2813532871 conn(0x7f6ce009d560 msgr2=0x7f6ce00d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:21.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.516+0000 7f6ce735e640 1 -- 172.21.15.172:0/2813532871 shutdown_connections 2024-04-14T08:35:21.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.516+0000 7f6ce735e640 1 -- 172.21.15.172:0/2813532871 wait complete. 2024-04-14T08:35:21.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.518+0000 7f6ce735e640 1 Processor -- start 2024-04-14T08:35:21.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.518+0000 7f6ce735e640 1 -- start start 2024-04-14T08:35:21.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce735e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce015e430 0x7f6ce0161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce735e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce0162710 0x7f6ce015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce735e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ce0162ba0 0x7f6ce0160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce735e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce00d4e20 con 0x7f6ce0162ba0 2024-04-14T08:35:21.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce735e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce015e810 con 0x7f6ce0162710 2024-04-14T08:35:21.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce735e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce00d0050 con 0x7f6ce015e430 2024-04-14T08:35:21.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce58fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ce0162ba0 0x7f6ce0160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce58fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ce0162ba0 0x7f6ce0160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35894/0 (socket says 172.21.15.172:35894) 2024-04-14T08:35:21.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce58fb640 1 -- 172.21.15.172:0/2500083339 learned_addr learned my addr 172.21.15.172:0/2500083339 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:21.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce48f9640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce0162710 0x7f6ce015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce50fa640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce015e430 0x7f6ce0161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce58fb640 1 -- 172.21.15.172:0/2500083339 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce015e430 msgr2=0x7f6ce0161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce58fb640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce015e430 0x7f6ce0161370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce58fb640 1 -- 172.21.15.172:0/2500083339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce0162710 msgr2=0x7f6ce015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce58fb640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce0162710 0x7f6ce015ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.519+0000 7f6ce58fb640 1 -- 172.21.15.172:0/2500083339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce00b3fa0 con 0x7f6ce0162ba0 2024-04-14T08:35:21.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.520+0000 7f6ce58fb640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ce0162ba0 0x7f6ce0160490 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009f70 tx=0x7f6cd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:21.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.520+0000 7f6cd5ffb640 1 -- 172.21.15.172:0/2500083339 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8043020 con 0x7f6ce0162ba0 2024-04-14T08:35:21.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.520+0000 7f6cd5ffb640 1 -- 172.21.15.172:0/2500083339 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd803e450 con 0x7f6ce0162ba0 2024-04-14T08:35:21.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.520+0000 7f6ce735e640 1 -- 172.21.15.172:0/2500083339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce00b4280 con 0x7f6ce0162ba0 2024-04-14T08:35:21.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.520+0000 7f6cd5ffb640 1 -- 172.21.15.172:0/2500083339 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd803e840 con 0x7f6ce0162ba0 2024-04-14T08:35:21.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.520+0000 7f6ce735e640 1 -- 172.21.15.172:0/2500083339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce00b4840 con 0x7f6ce0162ba0 2024-04-14T08:35:21.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.521+0000 7f6ce735e640 1 -- 172.21.15.172:0/2500083339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce0068590 con 0x7f6ce0162ba0 2024-04-14T08:35:21.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.522+0000 7f6cd5ffb640 1 -- 172.21.15.172:0/2500083339 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cd8046020 con 0x7f6ce0162ba0 2024-04-14T08:35:21.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.522+0000 7f6cd5ffb640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6cc406c8f0 0x7f6cc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.522+0000 7f6cd5ffb640 1 -- 172.21.15.172:0/2500083339 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f6cd80ae600 con 0x7f6ce0162ba0 2024-04-14T08:35:21.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.522+0000 7f6ce50fa640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6cc406c8f0 0x7f6cc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.522+0000 7f6ce50fa640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6cc406c8f0 0x7f6cc406edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0016e0 tx=0x7f6cdc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:21.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.530+0000 7f6cd5ffb640 1 -- 172.21.15.172:0/2500083339 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6cd804d030 con 0x7f6ce0162ba0 2024-04-14T08:35:21.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.713+0000 7f6ce735e640 1 -- 172.21.15.172:0/2500083339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6ce00d18a0 con 0x7f6ce0162ba0 2024-04-14T08:35:21.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.713+0000 7f6cd5ffb640 1 -- 172.21.15.172:0/2500083339 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) ==== 100+0+11 (secure 0 0 0) 0x7f6cd807c1c0 con 0x7f6ce0162ba0 2024-04-14T08:35:21.714 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:35:21.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.716+0000 7f6ce735e640 1 -- 172.21.15.172:0/2500083339 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6cc406c8f0 msgr2=0x7f6cc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.716+0000 7f6ce735e640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6cc406c8f0 0x7f6cc406edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0016e0 tx=0x7f6cdc009040 comp rx=0 tx=0).stop 2024-04-14T08:35:21.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.716+0000 7f6ce735e640 1 -- 172.21.15.172:0/2500083339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ce0162ba0 msgr2=0x7f6ce0160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.716+0000 7f6ce735e640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ce0162ba0 0x7f6ce0160490 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009f70 tx=0x7f6cd8004a40 comp rx=0 tx=0).stop 2024-04-14T08:35:21.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.716+0000 7f6ce735e640 1 -- 172.21.15.172:0/2500083339 shutdown_connections 2024-04-14T08:35:21.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.716+0000 7f6ce735e640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6cc406c8f0 0x7f6cc406edb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.716+0000 7f6ce735e640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ce0162ba0 0x7f6ce0160490 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.716+0000 7f6ce735e640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce0162710 0x7f6ce015ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.716+0000 7f6ce735e640 1 --2- 172.21.15.172:0/2500083339 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce015e430 0x7f6ce0161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.716+0000 7f6ce735e640 1 -- 172.21.15.172:0/2500083339 >> 172.21.15.172:0/2500083339 conn(0x7f6ce009d560 msgr2=0x7f6ce00b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:21.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.716+0000 7f6ce735e640 1 -- 172.21.15.172:0/2500083339 shutdown_connections 2024-04-14T08:35:21.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.716+0000 7f6ce735e640 1 -- 172.21.15.172:0/2500083339 wait complete. 2024-04-14T08:35:21.727 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:35:21.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.828+0000 7f967fd1e640 1 Processor -- start 2024-04-14T08:35:21.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.828+0000 7f967fd1e640 1 -- start start 2024-04-14T08:35:21.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967fd1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f967815e890 0x7f96780d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967fd1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f96780cedd0 0x7f96780b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967fd1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96780cf320 con 0x7f96780d2830 2024-04-14T08:35:21.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967fd1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f96780cff30 con 0x7f96780cedd0 2024-04-14T08:35:21.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967fd1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f96780d2350 con 0x7f967815e890 2024-04-14T08:35:21.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967e2bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f96780cedd0 0x7f96780b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967e2bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f96780cedd0 0x7f96780b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35904/0 (socket says 172.21.15.172:35904) 2024-04-14T08:35:21.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967e2bb640 1 -- 172.21.15.172:0/2933822896 learned_addr learned my addr 172.21.15.172:0/2933822896 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:21.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967d2b9640 1 --2- 172.21.15.172:0/2933822896 >> v2:172.21.15.177:3300/0 conn(0x7f967815e890 0x7f96780d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967e2bb640 1 -- 172.21.15.172:0/2933822896 >> v2:172.21.15.177:3300/0 conn(0x7f967815e890 msgr2=0x7f96780d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:21.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967e2bb640 1 --2- 172.21.15.172:0/2933822896 >> v2:172.21.15.177:3300/0 conn(0x7f967815e890 0x7f96780d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967e2bb640 1 -- 172.21.15.172:0/2933822896 >> v1:172.21.15.177:6789/0 conn(0x7f96780d2830 legacy=0x7f96780d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:21.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.829+0000 7f967e2bb640 1 -- 172.21.15.172:0/2933822896 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96780cf600 con 0x7f96780cedd0 2024-04-14T08:35:21.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.830+0000 7f967e2bb640 1 --2- 172.21.15.172:0/2933822896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96780cedd0 0x7f96780b3710 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9678162100 tx=0x7f967002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2886c5deae36baaf server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:21.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.830+0000 7f967cab8640 1 -- 172.21.15.172:0/2933822896 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967002cb20 con 0x7f96780cedd0 2024-04-14T08:35:21.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.830+0000 7f967cab8640 1 -- 172.21.15.172:0/2933822896 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f967002ccc0 con 0x7f96780cedd0 2024-04-14T08:35:21.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.830+0000 7f967cab8640 1 -- 172.21.15.172:0/2933822896 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670034610 con 0x7f96780cedd0 2024-04-14T08:35:21.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.831+0000 7f967fd1e640 1 -- 172.21.15.172:0/2933822896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96780cedd0 msgr2=0x7f96780b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.831+0000 7f967fd1e640 1 --2- 172.21.15.172:0/2933822896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96780cedd0 0x7f96780b3710 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9678162100 tx=0x7f967002bf80 comp rx=0 tx=0).stop 2024-04-14T08:35:21.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.831+0000 7f967fd1e640 1 -- 172.21.15.172:0/2933822896 shutdown_connections 2024-04-14T08:35:21.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.831+0000 7f967fd1e640 1 --2- 172.21.15.172:0/2933822896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96780cedd0 0x7f96780b3710 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.831+0000 7f967fd1e640 1 --2- 172.21.15.172:0/2933822896 >> v2:172.21.15.177:3300/0 conn(0x7f967815e890 0x7f96780d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.831+0000 7f967fd1e640 1 -- 172.21.15.172:0/2933822896 >> 172.21.15.172:0/2933822896 conn(0x7f967809d590 msgr2=0x7f96780d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:21.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.831+0000 7f967fd1e640 1 -- 172.21.15.172:0/2933822896 shutdown_connections 2024-04-14T08:35:21.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.831+0000 7f967fd1e640 1 -- 172.21.15.172:0/2933822896 wait complete. 2024-04-14T08:35:21.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.832+0000 7f967fd1e640 1 Processor -- start 2024-04-14T08:35:21.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.832+0000 7f967fd1e640 1 -- start start 2024-04-14T08:35:21.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967fd1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96780cedd0 0x7f96780b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967fd1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96780d2830 0x7f96780b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967fd1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f967815e890 0x7f96781592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967fd1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96780d1840 con 0x7f96780cedd0 2024-04-14T08:35:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967fd1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f96780cfbc0 con 0x7f967815e890 2024-04-14T08:35:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967fd1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f96780b3c50 con 0x7f96780d2830 2024-04-14T08:35:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967daba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96780cedd0 0x7f96780b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967daba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96780cedd0 0x7f96780b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35912/0 (socket says 172.21.15.172:35912) 2024-04-14T08:35:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967daba640 1 -- 172.21.15.172:0/2756481918 learned_addr learned my addr 172.21.15.172:0/2756481918 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967e2bb640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f967815e890 0x7f96781592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967d2b9640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96780d2830 0x7f96780b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967daba640 1 -- 172.21.15.172:0/2756481918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96780d2830 msgr2=0x7f96780b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967daba640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96780d2830 0x7f96780b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967daba640 1 -- 172.21.15.172:0/2756481918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f967815e890 msgr2=0x7f96781592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:21.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967daba640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f967815e890 0x7f96781592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:21.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967daba640 1 -- 172.21.15.172:0/2756481918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9678159d30 con 0x7f96780cedd0 2024-04-14T08:35:21.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f967daba640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96780cedd0 0x7f96780b6e70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f967400c410 tx=0x7f967400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:21.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.833+0000 7f966e7fc640 1 -- 172.21.15.172:0/2756481918 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674017070 con 0x7f96780cedd0 2024-04-14T08:35:21.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.834+0000 7f966e7fc640 1 -- 172.21.15.172:0/2756481918 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96740035f0 con 0x7f96780cedd0 2024-04-14T08:35:21.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.834+0000 7f967fd1e640 1 -- 172.21.15.172:0/2756481918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96780b7450 con 0x7f96780cedd0 2024-04-14T08:35:21.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.834+0000 7f966e7fc640 1 -- 172.21.15.172:0/2756481918 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674003a20 con 0x7f96780cedd0 2024-04-14T08:35:21.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.834+0000 7f967fd1e640 1 -- 172.21.15.172:0/2756481918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96780b7a10 con 0x7f96780cedd0 2024-04-14T08:35:21.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.834+0000 7f967fd1e640 1 -- 172.21.15.172:0/2756481918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9678068520 con 0x7f96780cedd0 2024-04-14T08:35:21.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.836+0000 7f966e7fc640 1 -- 172.21.15.172:0/2756481918 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f967401a070 con 0x7f96780cedd0 2024-04-14T08:35:21.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.836+0000 7f966e7fc640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f965c06c920 0x7f965c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:21.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.836+0000 7f966e7fc640 1 -- 172.21.15.172:0/2756481918 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f967408b110 con 0x7f96780cedd0 2024-04-14T08:35:21.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.836+0000 7f967d2b9640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f965c06c920 0x7f965c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:21.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.836+0000 7f967d2b9640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f965c06c920 0x7f965c06ede0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f96780c7750 tx=0x7f9668008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:21.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:21.840+0000 7f966e7fc640 1 -- 172.21.15.172:0/2756481918 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f96740010c0 con 0x7f96780cedd0 2024-04-14T08:35:22.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.068+0000 7f967fd1e640 1 -- 172.21.15.172:0/2756481918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96780d2610 con 0x7f96780cedd0 2024-04-14T08:35:22.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.068+0000 7f966e7fc640 1 -- 172.21.15.172:0/2756481918 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5187 (secure 0 0 0) 0x7f9674058d60 con 0x7f96780cedd0 2024-04-14T08:35:22.069 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:22.069 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/296861286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":296861286},{"type":"v1","addr":"172.21.15.177:6835","nonce":296861286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/642342704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":642342704},{"type":"v1","addr":"172.21.15.177:6833","nonce":642342704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3516903987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3516903987},{"type":"v1","addr":"172.21.15.172:6836","nonce":3516903987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:34:34.765057+0000","modified":"2024-04-14T08:34:36.783999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6837/1527574817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1527574817},{"type":"v1","addr":"172.21.15.172:6837","nonce":1527574817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":3}]} 2024-04-14T08:35:22.069 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 23 2024-04-14T08:35:22.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.070+0000 7f967fd1e640 1 -- 172.21.15.172:0/2756481918 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f965c06c920 msgr2=0x7f965c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.070+0000 7f967fd1e640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f965c06c920 0x7f965c06ede0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f96780c7750 tx=0x7f9668008040 comp rx=0 tx=0).stop 2024-04-14T08:35:22.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.070+0000 7f967fd1e640 1 -- 172.21.15.172:0/2756481918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96780cedd0 msgr2=0x7f96780b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.070+0000 7f967fd1e640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96780cedd0 0x7f96780b6e70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f967400c410 tx=0x7f967400c8d0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.070+0000 7f967fd1e640 1 -- 172.21.15.172:0/2756481918 shutdown_connections 2024-04-14T08:35:22.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.070+0000 7f967fd1e640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f965c06c920 0x7f965c06ede0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.070+0000 7f967fd1e640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f967815e890 0x7f96781592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.070+0000 7f967fd1e640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96780d2830 0x7f96780b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.070+0000 7f967fd1e640 1 --2- 172.21.15.172:0/2756481918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96780cedd0 0x7f96780b6e70 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.070+0000 7f967fd1e640 1 -- 172.21.15.172:0/2756481918 >> 172.21.15.172:0/2756481918 conn(0x7f967809d590 msgr2=0x7f96780d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:22.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.071+0000 7f967fd1e640 1 -- 172.21.15.172:0/2756481918 shutdown_connections 2024-04-14T08:35:22.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.071+0000 7f967fd1e640 1 -- 172.21.15.172:0/2756481918 wait complete. 2024-04-14T08:35:22.082 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:35:22.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.182+0000 7f2c0eede640 1 Processor -- start 2024-04-14T08:35:22.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.182+0000 7f2c0eede640 1 -- start start 2024-04-14T08:35:22.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.183+0000 7f2c0eede640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2c0815c150 0x7f2c080d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.183+0000 7f2c0eede640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2c080d1840 0x7f2c08161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.183+0000 7f2c0eede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c08162530 0x7f2c0815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.183+0000 7f2c0eede640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2c080d1cb0 con 0x7f2c080d1840 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.183+0000 7f2c0eede640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2c08162980 con 0x7f2c0815c150 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.183+0000 7f2c0eede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2c080cef10 con 0x7f2c08162530 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.183+0000 7f2c07fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2c080d1840 0x7f2c08161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.183+0000 7f2c07fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2c080d1840 0x7f2c08161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35922/0 (socket says 172.21.15.172:35922) 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.183+0000 7f2c07fff640 1 -- 172.21.15.172:0/2866367662 learned_addr learned my addr 172.21.15.172:0/2866367662 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.184+0000 7f2c07fff640 1 -- 172.21.15.172:0/2866367662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c08162530 msgr2=0x7f2c0815ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.184+0000 7f2c0d47b640 1 --2- 172.21.15.172:0/2866367662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c08162530 0x7f2c0815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.184+0000 7f2c0cc7a640 1 --2- 172.21.15.172:0/2866367662 >> v2:172.21.15.177:3300/0 conn(0x7f2c0815c150 0x7f2c080d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.184+0000 7f2c07fff640 1 --2- 172.21.15.172:0/2866367662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c08162530 0x7f2c0815ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.184+0000 7f2c07fff640 1 -- 172.21.15.172:0/2866367662 >> v2:172.21.15.177:3300/0 conn(0x7f2c0815c150 msgr2=0x7f2c080d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.184+0000 7f2c07fff640 1 --2- 172.21.15.172:0/2866367662 >> v2:172.21.15.177:3300/0 conn(0x7f2c0815c150 0x7f2c080d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.184+0000 7f2c07fff640 1 -- 172.21.15.172:0/2866367662 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c080d0200 con 0x7f2c080d1840 2024-04-14T08:35:22.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.184+0000 7f2c07fff640 1 --2- 172.21.15.172:0/2866367662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c080d1840 0x7f2c08161ff0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2c0806bf30 tx=0x7f2bf402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fae7dbd12c8c3d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:22.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.185+0000 7f2c077fe640 1 -- 172.21.15.172:0/2866367662 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4035020 con 0x7f2c080d1840 2024-04-14T08:35:22.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.185+0000 7f2c077fe640 1 -- 172.21.15.172:0/2866367662 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf4031640 con 0x7f2c080d1840 2024-04-14T08:35:22.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.185+0000 7f2c077fe640 1 -- 172.21.15.172:0/2866367662 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf402d040 con 0x7f2c080d1840 2024-04-14T08:35:22.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.185+0000 7f2c0eede640 1 -- 172.21.15.172:0/2866367662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c080d1840 msgr2=0x7f2c08161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.185+0000 7f2c0eede640 1 --2- 172.21.15.172:0/2866367662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c080d1840 0x7f2c08161ff0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2c0806bf30 tx=0x7f2bf402b8f0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.185+0000 7f2c0eede640 1 -- 172.21.15.172:0/2866367662 shutdown_connections 2024-04-14T08:35:22.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.185+0000 7f2c0eede640 1 --2- 172.21.15.172:0/2866367662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c08162530 0x7f2c0815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.185+0000 7f2c0eede640 1 --2- 172.21.15.172:0/2866367662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c080d1840 0x7f2c08161ff0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.185+0000 7f2c0eede640 1 --2- 172.21.15.172:0/2866367662 >> v2:172.21.15.177:3300/0 conn(0x7f2c0815c150 0x7f2c080d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.185+0000 7f2c0eede640 1 -- 172.21.15.172:0/2866367662 >> 172.21.15.172:0/2866367662 conn(0x7f2c0809d580 msgr2=0x7f2c080d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:22.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.185+0000 7f2c0eede640 1 -- 172.21.15.172:0/2866367662 shutdown_connections 2024-04-14T08:35:22.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.185+0000 7f2c0eede640 1 -- 172.21.15.172:0/2866367662 wait complete. 2024-04-14T08:35:22.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.186+0000 7f2c0eede640 1 Processor -- start 2024-04-14T08:35:22.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.186+0000 7f2c0eede640 1 -- start start 2024-04-14T08:35:22.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.187+0000 7f2c0eede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c080d1840 0x7f2c0815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.187+0000 7f2c0eede640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c0815c150 0x7f2c080b9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.187+0000 7f2c0eede640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c08162530 0x7f2c080b9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.187+0000 7f2c0eede640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2c080cf720 con 0x7f2c08162530 2024-04-14T08:35:22.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.187+0000 7f2c0eede640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2c080cf440 con 0x7f2c0815c150 2024-04-14T08:35:22.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.187+0000 7f2c0eede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2c080cfa00 con 0x7f2c080d1840 2024-04-14T08:35:22.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.187+0000 7f2c0d47b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c08162530 0x7f2c080b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.187+0000 7f2c0d47b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c08162530 0x7f2c080b9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35928/0 (socket says 172.21.15.172:35928) 2024-04-14T08:35:22.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.187+0000 7f2c0d47b640 1 -- 172.21.15.172:0/2617526763 learned_addr learned my addr 172.21.15.172:0/2617526763 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:22.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c0cc7a640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c080d1840 0x7f2c0815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c07fff640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c0815c150 0x7f2c080b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c0d47b640 1 -- 172.21.15.172:0/2617526763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c080d1840 msgr2=0x7f2c0815ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c0d47b640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c080d1840 0x7f2c0815ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c0d47b640 1 -- 172.21.15.172:0/2617526763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c0815c150 msgr2=0x7f2c080b9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c0d47b640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c0815c150 0x7f2c080b9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c0d47b640 1 -- 172.21.15.172:0/2617526763 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c080b9e80 con 0x7f2c08162530 2024-04-14T08:35:22.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c0d47b640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c08162530 0x7f2c080b9580 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000cc30 tx=0x7f2c0000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:22.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c057fa640 1 -- 172.21.15.172:0/2617526763 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0000ce40 con 0x7f2c08162530 2024-04-14T08:35:22.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c057fa640 1 -- 172.21.15.172:0/2617526763 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c00003990 con 0x7f2c08162530 2024-04-14T08:35:22.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c0eede640 1 -- 172.21.15.172:0/2617526763 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0815e250 con 0x7f2c08162530 2024-04-14T08:35:22.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c057fa640 1 -- 172.21.15.172:0/2617526763 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00003d40 con 0x7f2c08162530 2024-04-14T08:35:22.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.188+0000 7f2c0eede640 1 -- 172.21.15.172:0/2617526763 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0815e810 con 0x7f2c08162530 2024-04-14T08:35:22.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.189+0000 7f2c0eede640 1 -- 172.21.15.172:0/2617526763 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c08075490 con 0x7f2c08162530 2024-04-14T08:35:22.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.190+0000 7f2c057fa640 1 -- 172.21.15.172:0/2617526763 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c00004000 con 0x7f2c08162530 2024-04-14T08:35:22.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.190+0000 7f2c057fa640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2bec06c8f0 0x7f2bec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.190+0000 7f2c0cc7a640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2bec06c8f0 0x7f2bec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.190+0000 7f2c057fa640 1 -- 172.21.15.172:0/2617526763 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f2c0008c0f0 con 0x7f2c08162530 2024-04-14T08:35:22.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.191+0000 7f2c0cc7a640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2bec06c8f0 0x7f2bec06edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00aaf0 tx=0x7f2bfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:22.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.194+0000 7f2c057fa640 1 -- 172.21.15.172:0/2617526763 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2c00014030 con 0x7f2c08162530 2024-04-14T08:35:22.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.373+0000 7f2c0eede640 1 -- 172.21.15.172:0/2617526763 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c080d33d0 con 0x7f2c08162530 2024-04-14T08:35:22.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.374+0000 7f2c057fa640 1 -- 172.21.15.172:0/2617526763 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+17002 (secure 0 0 0) 0x7f2c00059d40 con 0x7f2c08162530 2024-04-14T08:35:22.375 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:22.375 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":50,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:36.423005+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:34:32.990827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-14T08:34:33.715803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:35:22.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.376+0000 7f2c0eede640 1 -- 172.21.15.172:0/2617526763 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2bec06c8f0 msgr2=0x7f2bec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.376+0000 7f2c0eede640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2bec06c8f0 0x7f2bec06edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00aaf0 tx=0x7f2bfc008040 comp rx=0 tx=0).stop 2024-04-14T08:35:22.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.376+0000 7f2c0eede640 1 -- 172.21.15.172:0/2617526763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c08162530 msgr2=0x7f2c080b9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.376+0000 7f2c0eede640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c08162530 0x7f2c080b9580 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000cc30 tx=0x7f2c0000a5c0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.376+0000 7f2c0eede640 1 -- 172.21.15.172:0/2617526763 shutdown_connections 2024-04-14T08:35:22.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.376+0000 7f2c0eede640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2bec06c8f0 0x7f2bec06edb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.376+0000 7f2c0eede640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c08162530 0x7f2c080b9580 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.376+0000 7f2c0eede640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c0815c150 0x7f2c080b9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.376+0000 7f2c0eede640 1 --2- 172.21.15.172:0/2617526763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c080d1840 0x7f2c0815ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.376+0000 7f2c0eede640 1 -- 172.21.15.172:0/2617526763 >> 172.21.15.172:0/2617526763 conn(0x7f2c0809d580 msgr2=0x7f2c08161df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:22.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.377+0000 7f2c0eede640 1 -- 172.21.15.172:0/2617526763 shutdown_connections 2024-04-14T08:35:22.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.377+0000 7f2c0eede640 1 -- 172.21.15.172:0/2617526763 wait complete. 2024-04-14T08:35:22.388 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-14T08:35:22.389 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:35:22.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.493+0000 7f66f64de640 1 Processor -- start 2024-04-14T08:35:22.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.493+0000 7f66f64de640 1 -- start start 2024-04-14T08:35:22.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.493+0000 7f66f64de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f66f00d2830 0x7f66f00d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.493+0000 7f66f64de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f66f00cedd0 0x7f66f00b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.493+0000 7f66f64de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66f00d3150 con 0x7f66f015e890 2024-04-14T08:35:22.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.493+0000 7f66f64de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f66f00cff30 con 0x7f66f00cedd0 2024-04-14T08:35:22.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.493+0000 7f66f64de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f66f00d2350 con 0x7f66f00d2830 2024-04-14T08:35:22.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.494+0000 7f66f4a7b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f66f00cedd0 0x7f66f00b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.494+0000 7f66ef7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f66f015e890 0x7f66f00d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50866/0 (socket says 172.21.15.172:50866) 2024-04-14T08:35:22.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.494+0000 7f66ef7fe640 1 -- 172.21.15.172:0/1721002902 learned_addr learned my addr 172.21.15.172:0/1721002902 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:22.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.494+0000 7f66f4a7b640 1 --2- 172.21.15.172:0/1721002902 >> v2:172.21.15.172:3300/0 conn(0x7f66f00cedd0 0x7f66f00b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35938/0 (socket says 172.21.15.172:35938) 2024-04-14T08:35:22.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.494+0000 7f66effff640 1 --2- 172.21.15.172:0/1721002902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f66f00d2830 0x7f66f00d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.494+0000 7f66f4a7b640 1 -- 172.21.15.172:0/1721002902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f66f00d2830 msgr2=0x7f66f00d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:22.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.494+0000 7f66f4a7b640 1 --2- 172.21.15.172:0/1721002902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f66f00d2830 0x7f66f00d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.494+0000 7f66f4a7b640 1 -- 172.21.15.172:0/1721002902 >> v1:172.21.15.172:6789/0 conn(0x7f66f015e890 legacy=0x7f66f00d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.494+0000 7f66f4a7b640 1 -- 172.21.15.172:0/1721002902 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f00cf320 con 0x7f66f00cedd0 2024-04-14T08:35:22.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.494+0000 7f66eeffd640 1 -- 172.21.15.172:0/1721002902 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 16888800 0 0) 0x7f66f00d3150 con 0x7f66f015e890 2024-04-14T08:35:22.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.495+0000 7f66f4a7b640 1 --2- 172.21.15.172:0/1721002902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f00cedd0 0x7f66f00b3710 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009d90 tx=0x7f66e802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7590c90f15818d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:22.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.495+0000 7f66eeffd640 1 -- 172.21.15.172:0/1721002902 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8039070 con 0x7f66f00cedd0 2024-04-14T08:35:22.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.495+0000 7f66eeffd640 1 -- 172.21.15.172:0/1721002902 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e802cce0 con 0x7f66f00cedd0 2024-04-14T08:35:22.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.495+0000 7f66eeffd640 1 -- 172.21.15.172:0/1721002902 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8034660 con 0x7f66f00cedd0 2024-04-14T08:35:22.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.495+0000 7f66f64de640 1 -- 172.21.15.172:0/1721002902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f00cedd0 msgr2=0x7f66f00b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.495+0000 7f66f64de640 1 --2- 172.21.15.172:0/1721002902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f00cedd0 0x7f66f00b3710 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009d90 tx=0x7f66e802bf30 comp rx=0 tx=0).stop 2024-04-14T08:35:22.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.495+0000 7f66f64de640 1 -- 172.21.15.172:0/1721002902 shutdown_connections 2024-04-14T08:35:22.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.495+0000 7f66f64de640 1 --2- 172.21.15.172:0/1721002902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f00cedd0 0x7f66f00b3710 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.495+0000 7f66f64de640 1 --2- 172.21.15.172:0/1721002902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f66f00d2830 0x7f66f00d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.495+0000 7f66f64de640 1 -- 172.21.15.172:0/1721002902 >> 172.21.15.172:0/1721002902 conn(0x7f66f009d590 msgr2=0x7f66f00d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:22.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.495+0000 7f66f64de640 1 -- 172.21.15.172:0/1721002902 shutdown_connections 2024-04-14T08:35:22.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.495+0000 7f66f64de640 1 -- 172.21.15.172:0/1721002902 wait complete. 2024-04-14T08:35:22.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.496+0000 7f66f64de640 1 Processor -- start 2024-04-14T08:35:22.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.497+0000 7f66f64de640 1 -- start start 2024-04-14T08:35:22.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.497+0000 7f66f64de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66f00cedd0 0x7f66f0154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.497+0000 7f66f64de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f00d2830 0x7f66f00b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.497+0000 7f66f64de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f66f015e890 0x7f66f00b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.497+0000 7f66f64de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66f00cfbc0 con 0x7f66f00d2830 2024-04-14T08:35:22.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.497+0000 7f66f64de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f66f00cf8e0 con 0x7f66f00cedd0 2024-04-14T08:35:22.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.497+0000 7f66f64de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f66f00b3c50 con 0x7f66f015e890 2024-04-14T08:35:22.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.497+0000 7f66ef7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f00d2830 0x7f66f00b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66ef7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f00d2830 0x7f66f00b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35940/0 (socket says 172.21.15.172:35940) 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66ef7fe640 1 -- 172.21.15.172:0/2093925283 learned_addr learned my addr 172.21.15.172:0/2093925283 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66f4a7b640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f66f015e890 0x7f66f00b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66effff640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66f00cedd0 0x7f66f0154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66ef7fe640 1 -- 172.21.15.172:0/2093925283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f66f015e890 msgr2=0x7f66f00b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66ef7fe640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f66f015e890 0x7f66f00b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66ef7fe640 1 -- 172.21.15.172:0/2093925283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66f00cedd0 msgr2=0x7f66f0154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66ef7fe640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66f00cedd0 0x7f66f0154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66ef7fe640 1 -- 172.21.15.172:0/2093925283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f00cca70 con 0x7f66f00d2830 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66ef7fe640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f00d2830 0x7f66f00b0e80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f66e400c410 tx=0x7f66e400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66ecff9640 1 -- 172.21.15.172:0/2093925283 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e40074e0 con 0x7f66f00d2830 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66ecff9640 1 -- 172.21.15.172:0/2093925283 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e40047e0 con 0x7f66f00d2830 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66ecff9640 1 -- 172.21.15.172:0/2093925283 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4004bd0 con 0x7f66f00d2830 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66f64de640 1 -- 172.21.15.172:0/2093925283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f00cc790 con 0x7f66f00d2830 2024-04-14T08:35:22.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.498+0000 7f66f64de640 1 -- 172.21.15.172:0/2093925283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f00c94b0 con 0x7f66f00d2830 2024-04-14T08:35:22.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.499+0000 7f66f64de640 1 -- 172.21.15.172:0/2093925283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f0068520 con 0x7f66f00d2830 2024-04-14T08:35:22.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.500+0000 7f66ecff9640 1 -- 172.21.15.172:0/2093925283 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66e4011070 con 0x7f66f00d2830 2024-04-14T08:35:22.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.500+0000 7f66ecff9640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f66dc06c8f0 0x7f66dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.500+0000 7f66ecff9640 1 -- 172.21.15.172:0/2093925283 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f66e408b780 con 0x7f66f00d2830 2024-04-14T08:35:22.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.500+0000 7f66effff640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f66dc06c8f0 0x7f66dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.501+0000 7f66effff640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f66dc06c8f0 0x7f66dc06edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f66e000aac0 tx=0x7f66e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:22.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.505+0000 7f66ecff9640 1 -- 172.21.15.172:0/2093925283 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f66e40593d0 con 0x7f66f00d2830 2024-04-14T08:35:22.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.750+0000 7f66f64de640 1 -- 172.21.15.172:0/2093925283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66f00b2e80 con 0x7f66f00d2830 2024-04-14T08:35:22.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.751+0000 7f66ecff9640 1 -- 172.21.15.172:0/2093925283 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5187 (secure 0 0 0) 0x7f66e405d6d0 con 0x7f66f00d2830 2024-04-14T08:35:22.751 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:22.751 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/296861286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":296861286},{"type":"v1","addr":"172.21.15.177:6835","nonce":296861286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/642342704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":642342704},{"type":"v1","addr":"172.21.15.177:6833","nonce":642342704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3516903987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3516903987},{"type":"v1","addr":"172.21.15.172:6836","nonce":3516903987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:34:34.765057+0000","modified":"2024-04-14T08:34:36.783999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6837/1527574817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1527574817},{"type":"v1","addr":"172.21.15.172:6837","nonce":1527574817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":3}]} 2024-04-14T08:35:22.752 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 23 2024-04-14T08:35:22.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.753+0000 7f66f64de640 1 -- 172.21.15.172:0/2093925283 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f66dc06c8f0 msgr2=0x7f66dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.753+0000 7f66f64de640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f66dc06c8f0 0x7f66dc06edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f66e000aac0 tx=0x7f66e0008040 comp rx=0 tx=0).stop 2024-04-14T08:35:22.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.753+0000 7f66f64de640 1 -- 172.21.15.172:0/2093925283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f00d2830 msgr2=0x7f66f00b0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.753+0000 7f66f64de640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f00d2830 0x7f66f00b0e80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f66e400c410 tx=0x7f66e400c8d0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.753+0000 7f66f64de640 1 -- 172.21.15.172:0/2093925283 shutdown_connections 2024-04-14T08:35:22.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.753+0000 7f66f64de640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f66dc06c8f0 0x7f66dc06edb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.753+0000 7f66f64de640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f66f015e890 0x7f66f00b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.753+0000 7f66f64de640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f00d2830 0x7f66f00b0e80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.753+0000 7f66f64de640 1 --2- 172.21.15.172:0/2093925283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66f00cedd0 0x7f66f0154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.754+0000 7f66f64de640 1 -- 172.21.15.172:0/2093925283 >> 172.21.15.172:0/2093925283 conn(0x7f66f009d590 msgr2=0x7f66f0074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:22.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.754+0000 7f66f64de640 1 -- 172.21.15.172:0/2093925283 shutdown_connections 2024-04-14T08:35:22.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.754+0000 7f66f64de640 1 -- 172.21.15.172:0/2093925283 wait complete. 2024-04-14T08:35:22.768 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:35:22.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.869+0000 7fb23631e640 1 Processor -- start 2024-04-14T08:35:22.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.869+0000 7fb23631e640 1 -- start start 2024-04-14T08:35:22.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.869+0000 7fb23631e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb2300d2830 0x7fb2300d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.869+0000 7fb23631e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2300cedd0 0x7fb2300b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.869+0000 7fb23631e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2300d3150 con 0x7fb23015e890 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.869+0000 7fb23631e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb2300cff30 con 0x7fb2300d2830 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.869+0000 7fb23631e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb2300d2350 con 0x7fb2300cedd0 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.869+0000 7fb22ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb2300d2830 0x7fb2300d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.869+0000 7fb22ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb2300d2830 0x7fb2300d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35944/0 (socket says 172.21.15.172:35944) 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.870+0000 7fb22ffff640 1 -- 172.21.15.172:0/1444293095 learned_addr learned my addr 172.21.15.172:0/1444293095 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.870+0000 7fb22f7fe640 1 --1- 172.21.15.172:0/1444293095 >> v1:172.21.15.172:6789/0 conn(0x7fb23015e890 0x7fb2300d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50872/0 (socket says 172.21.15.172:50872) 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.870+0000 7fb2348bb640 1 --2- 172.21.15.172:0/1444293095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2300cedd0 0x7fb2300b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.870+0000 7fb22ffff640 1 -- 172.21.15.172:0/1444293095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2300cedd0 msgr2=0x7fb2300b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.870+0000 7fb22ffff640 1 --2- 172.21.15.172:0/1444293095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2300cedd0 0x7fb2300b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.870+0000 7fb22ffff640 1 -- 172.21.15.172:0/1444293095 >> v1:172.21.15.172:6789/0 conn(0x7fb23015e890 legacy=0x7fb2300d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.870+0000 7fb22ffff640 1 -- 172.21.15.172:0/1444293095 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2300cf320 con 0x7fb2300d2830 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.870+0000 7fb22effd640 1 -- 172.21.15.172:0/1444293095 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3657097615 0 0) 0x7fb2300d3150 con 0x7fb23015e890 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.870+0000 7fb2348bb640 1 --2- 172.21.15.172:0/1444293095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2300cedd0 0x7fb2300b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.871+0000 7fb22ffff640 1 --2- 172.21.15.172:0/1444293095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2300d2830 0x7fb2300d2c10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb220009870 tx=0x7fb22002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b503aa88b3562d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.871+0000 7fb22effd640 1 -- 172.21.15.172:0/1444293095 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22002f840 con 0x7fb2300d2830 2024-04-14T08:35:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.871+0000 7fb22effd640 1 -- 172.21.15.172:0/1444293095 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22002f9e0 con 0x7fb2300d2830 2024-04-14T08:35:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.871+0000 7fb22effd640 1 -- 172.21.15.172:0/1444293095 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22002fdb0 con 0x7fb2300d2830 2024-04-14T08:35:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.871+0000 7fb23631e640 1 -- 172.21.15.172:0/1444293095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2300d2830 msgr2=0x7fb2300d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.871+0000 7fb23631e640 1 --2- 172.21.15.172:0/1444293095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2300d2830 0x7fb2300d2c10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb220009870 tx=0x7fb22002ba20 comp rx=0 tx=0).stop 2024-04-14T08:35:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.871+0000 7fb23631e640 1 -- 172.21.15.172:0/1444293095 shutdown_connections 2024-04-14T08:35:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.871+0000 7fb23631e640 1 --2- 172.21.15.172:0/1444293095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2300cedd0 0x7fb2300b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.871+0000 7fb23631e640 1 --2- 172.21.15.172:0/1444293095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2300d2830 0x7fb2300d2c10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.872+0000 7fb23631e640 1 -- 172.21.15.172:0/1444293095 >> 172.21.15.172:0/1444293095 conn(0x7fb23009d590 msgr2=0x7fb2300d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.872+0000 7fb23631e640 1 -- 172.21.15.172:0/1444293095 shutdown_connections 2024-04-14T08:35:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.872+0000 7fb23631e640 1 -- 172.21.15.172:0/1444293095 wait complete. 2024-04-14T08:35:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.873+0000 7fb23631e640 1 Processor -- start 2024-04-14T08:35:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.874+0000 7fb23631e640 1 -- start start 2024-04-14T08:35:22.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb23631e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2300cedd0 0x7fb230155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb23631e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2300d2830 0x7fb23006cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb23631e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb23015e890 0x7fb23006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb23631e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb2300cfbc0 con 0x7fb2300cedd0 2024-04-14T08:35:22.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb23631e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2300cf8e0 con 0x7fb2300d2830 2024-04-14T08:35:22.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb23631e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb2300b3c50 con 0x7fb23015e890 2024-04-14T08:35:22.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2300cedd0 0x7fb230155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2300cedd0 0x7fb230155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35950/0 (socket says 172.21.15.172:35950) 2024-04-14T08:35:22.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22ffff640 1 -- 172.21.15.172:0/2319020758 learned_addr learned my addr 172.21.15.172:0/2319020758 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:22.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb2348bb640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb23015e890 0x7fb23006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22f7fe640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2300d2830 0x7fb23006cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22ffff640 1 -- 172.21.15.172:0/2319020758 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb23015e890 msgr2=0x7fb23006d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22ffff640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb23015e890 0x7fb23006d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22ffff640 1 -- 172.21.15.172:0/2319020758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2300d2830 msgr2=0x7fb23006cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:22.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22ffff640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2300d2830 0x7fb23006cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:22.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22ffff640 1 -- 172.21.15.172:0/2319020758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2300ca660 con 0x7fb2300cedd0 2024-04-14T08:35:22.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22f7fe640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2300d2830 0x7fb23006cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:22.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22ffff640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2300cedd0 0x7fb230155f00 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb2200090b0 tx=0x7fb220004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:22.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22cff9640 1 -- 172.21.15.172:0/2319020758 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220047070 con 0x7fb2300cedd0 2024-04-14T08:35:22.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22cff9640 1 -- 172.21.15.172:0/2319020758 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb220037bc0 con 0x7fb2300cedd0 2024-04-14T08:35:22.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb23631e640 1 -- 172.21.15.172:0/2319020758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2300ca940 con 0x7fb2300cedd0 2024-04-14T08:35:22.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.875+0000 7fb22cff9640 1 -- 172.21.15.172:0/2319020758 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220004030 con 0x7fb2300cedd0 2024-04-14T08:35:22.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.876+0000 7fb23631e640 1 -- 172.21.15.172:0/2319020758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2300b6f10 con 0x7fb2300cedd0 2024-04-14T08:35:22.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.877+0000 7fb23631e640 1 -- 172.21.15.172:0/2319020758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb230068520 con 0x7fb2300cedd0 2024-04-14T08:35:22.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.877+0000 7fb22cff9640 1 -- 172.21.15.172:0/2319020758 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb22004a070 con 0x7fb2300cedd0 2024-04-14T08:35:22.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.877+0000 7fb22cff9640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb21c06c8f0 0x7fb21c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:22.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.877+0000 7fb22f7fe640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb21c06c8f0 0x7fb21c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:22.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.878+0000 7fb22cff9640 1 -- 172.21.15.172:0/2319020758 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fb2200addb0 con 0x7fb2300cedd0 2024-04-14T08:35:22.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.878+0000 7fb22f7fe640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb21c06c8f0 0x7fb21c06edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb2300b3070 tx=0x7fb224009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:22.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:22.885+0000 7fb22cff9640 1 -- 172.21.15.172:0/2319020758 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb220033030 con 0x7fb2300cedd0 2024-04-14T08:35:23.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.200+0000 7fb23631e640 1 -- 172.21.15.172:0/2319020758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2300d2610 con 0x7fb2300cedd0 2024-04-14T08:35:23.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.201+0000 7fb22cff9640 1 -- 172.21.15.172:0/2319020758 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5187 (secure 0 0 0) 0x7fb22007c050 con 0x7fb2300cedd0 2024-04-14T08:35:23.202 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:23.202 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/296861286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":296861286},{"type":"v1","addr":"172.21.15.177:6835","nonce":296861286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/642342704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":642342704},{"type":"v1","addr":"172.21.15.177:6833","nonce":642342704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3516903987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3516903987},{"type":"v1","addr":"172.21.15.172:6836","nonce":3516903987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:34:34.765057+0000","modified":"2024-04-14T08:34:36.783999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6837/1527574817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1527574817},{"type":"v1","addr":"172.21.15.172:6837","nonce":1527574817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":3}]} 2024-04-14T08:35:23.202 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 23 2024-04-14T08:35:23.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.203+0000 7fb23631e640 1 -- 172.21.15.172:0/2319020758 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb21c06c8f0 msgr2=0x7fb21c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.203+0000 7fb23631e640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb21c06c8f0 0x7fb21c06edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb2300b3070 tx=0x7fb224009040 comp rx=0 tx=0).stop 2024-04-14T08:35:23.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.203+0000 7fb23631e640 1 -- 172.21.15.172:0/2319020758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2300cedd0 msgr2=0x7fb230155f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.203+0000 7fb23631e640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2300cedd0 0x7fb230155f00 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb2200090b0 tx=0x7fb220004730 comp rx=0 tx=0).stop 2024-04-14T08:35:23.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.203+0000 7fb23631e640 1 -- 172.21.15.172:0/2319020758 shutdown_connections 2024-04-14T08:35:23.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.203+0000 7fb23631e640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb21c06c8f0 0x7fb21c06edb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.203+0000 7fb23631e640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb23015e890 0x7fb23006d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.203+0000 7fb23631e640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2300d2830 0x7fb23006cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.203+0000 7fb23631e640 1 --2- 172.21.15.172:0/2319020758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2300cedd0 0x7fb230155f00 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.203+0000 7fb23631e640 1 -- 172.21.15.172:0/2319020758 >> 172.21.15.172:0/2319020758 conn(0x7fb23009d590 msgr2=0x7fb230074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:23.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.203+0000 7fb23631e640 1 -- 172.21.15.172:0/2319020758 shutdown_connections 2024-04-14T08:35:23.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.203+0000 7fb23631e640 1 -- 172.21.15.172:0/2319020758 wait complete. 2024-04-14T08:35:23.215 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:35:23.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.316+0000 7f600ccc8640 1 Processor -- start 2024-04-14T08:35:23.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.316+0000 7f600ccc8640 1 -- start start 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f600ccc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60080cedd0 0x7f60081694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f600ccc8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60080cf320 con 0x7f60080d2830 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f600ccc8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60080cf600 con 0x7f600815e890 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f600ccc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f60080cff30 con 0x7f60080cedd0 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f600659c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f60080d2830 0x7f60080d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50874/0 (socket says 172.21.15.172:50874) 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f600659c640 1 -- 172.21.15.172:0/2587808078 learned_addr learned my addr 172.21.15.172:0/2587808078 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f6006d9d640 1 --2- 172.21.15.172:0/2587808078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60080cedd0 0x7f60081694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f600559a640 1 -- 172.21.15.172:0/2587808078 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916857920 0 0) 0x7f60080cf320 con 0x7f60080d2830 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f600559a640 1 -- 172.21.15.172:0/2587808078 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ff0003520 con 0x7f60080d2830 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f600559a640 1 -- 172.21.15.172:0/2587808078 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 295619993 0 0) 0x7f5ff0003520 con 0x7f60080d2830 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f6006d9d640 1 -- 172.21.15.172:0/2587808078 >> v1:172.21.15.177:6789/0 conn(0x7f600815e890 legacy=0x7f60080d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f6006d9d640 1 -- 172.21.15.172:0/2587808078 >> v1:172.21.15.172:6789/0 conn(0x7f60080d2830 legacy=0x7f60080d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.318+0000 7f6006d9d640 1 -- 172.21.15.172:0/2587808078 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60080cf8e0 con 0x7f60080cedd0 2024-04-14T08:35:23.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.319+0000 7f6006d9d640 1 --2- 172.21.15.172:0/2587808078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60080cedd0 0x7f60081694d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6000009dc0 tx=0x7f600002bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=178b36bb8b401e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:23.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.319+0000 7f600559a640 1 -- 172.21.15.172:0/2587808078 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000039070 con 0x7f60080cedd0 2024-04-14T08:35:23.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.319+0000 7f600559a640 1 -- 172.21.15.172:0/2587808078 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f600002c800 con 0x7f60080cedd0 2024-04-14T08:35:23.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.319+0000 7f600559a640 1 -- 172.21.15.172:0/2587808078 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc001670 con 0x7f60080cedd0 2024-04-14T08:35:23.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.320+0000 7f600ccc8640 1 -- 172.21.15.172:0/2587808078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60080cedd0 msgr2=0x7f60081694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.320+0000 7f600ccc8640 1 --2- 172.21.15.172:0/2587808078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60080cedd0 0x7f60081694d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6000009dc0 tx=0x7f600002bcd0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.320+0000 7f600ccc8640 1 -- 172.21.15.172:0/2587808078 shutdown_connections 2024-04-14T08:35:23.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.320+0000 7f600ccc8640 1 --2- 172.21.15.172:0/2587808078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60080cedd0 0x7f60081694d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.320+0000 7f600ccc8640 1 -- 172.21.15.172:0/2587808078 >> 172.21.15.172:0/2587808078 conn(0x7f600809d590 msgr2=0x7f60080d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:23.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.320+0000 7f600ccc8640 1 -- 172.21.15.172:0/2587808078 shutdown_connections 2024-04-14T08:35:23.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.320+0000 7f600ccc8640 1 -- 172.21.15.172:0/2587808078 wait complete. 2024-04-14T08:35:23.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.321+0000 7f600ccc8640 1 Processor -- start 2024-04-14T08:35:23.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.321+0000 7f600ccc8640 1 -- start start 2024-04-14T08:35:23.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f600ccc8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60080cedd0 0x7f60080ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f600ccc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60080d2830 0x7f60081f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f600ccc8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f600815e890 0x7f6008160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f600ccc8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60080b4720 con 0x7f600815e890 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f600ccc8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f60080d2350 con 0x7f60080cedd0 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f600ccc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f60080b48a0 con 0x7f60080d2830 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f6006d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f600815e890 0x7f6008160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f6006d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f600815e890 0x7f6008160ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35952/0 (socket says 172.21.15.172:35952) 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f6006d9d640 1 -- 172.21.15.172:0/573082687 learned_addr learned my addr 172.21.15.172:0/573082687 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f6005d9b640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60080d2830 0x7f60081f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f600659c640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60080cedd0 0x7f60080ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f6006d9d640 1 -- 172.21.15.172:0/573082687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60080d2830 msgr2=0x7f60081f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f6006d9d640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60080d2830 0x7f60081f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f6006d9d640 1 -- 172.21.15.172:0/573082687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60080cedd0 msgr2=0x7f60080ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f6006d9d640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60080cedd0 0x7f60080ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f6006d9d640 1 -- 172.21.15.172:0/573082687 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60080cd290 con 0x7f600815e890 2024-04-14T08:35:23.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.322+0000 7f6006d9d640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f600815e890 0x7f6008160ef0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6000009ee0 tx=0x7f6000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:23.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.323+0000 7f5ff6ffd640 1 -- 172.21.15.172:0/573082687 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000044020 con 0x7f600815e890 2024-04-14T08:35:23.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.323+0000 7f5ff6ffd640 1 -- 172.21.15.172:0/573082687 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6000034d70 con 0x7f600815e890 2024-04-14T08:35:23.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.323+0000 7f600ccc8640 1 -- 172.21.15.172:0/573082687 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60080cb320 con 0x7f600815e890 2024-04-14T08:35:23.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.323+0000 7f5ff6ffd640 1 -- 172.21.15.172:0/573082687 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000039040 con 0x7f600815e890 2024-04-14T08:35:23.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.323+0000 7f600ccc8640 1 -- 172.21.15.172:0/573082687 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60080cb8e0 con 0x7f600815e890 2024-04-14T08:35:23.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.324+0000 7f600ccc8640 1 -- 172.21.15.172:0/573082687 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6008068520 con 0x7f600815e890 2024-04-14T08:35:23.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.324+0000 7f5ff6ffd640 1 -- 172.21.15.172:0/573082687 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6000047020 con 0x7f600815e890 2024-04-14T08:35:23.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.324+0000 7f5ff6ffd640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5fe806c8f0 0x7f5fe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:23.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.325+0000 7f600659c640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5fe806c8f0 0x7f5fe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:23.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.325+0000 7f5ff6ffd640 1 -- 172.21.15.172:0/573082687 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7f60000ae4f0 con 0x7f600815e890 2024-04-14T08:35:23.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.325+0000 7f600659c640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5fe806c8f0 0x7f5fe806edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0011e0 tx=0x7f5ffc001ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:23.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.329+0000 7f5ff6ffd640 1 -- 172.21.15.172:0/573082687 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f600004e030 con 0x7f600815e890 2024-04-14T08:35:23.405 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:35:23.408 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:35:23.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.543+0000 7f600ccc8640 1 -- 172.21.15.172:0/573082687 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f600806e010 con 0x7f600815e890 2024-04-14T08:35:23.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.544+0000 7f5ff6ffd640 1 -- 172.21.15.172:0/573082687 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+17002 (secure 0 0 0) 0x7f600007c680 con 0x7f600815e890 2024-04-14T08:35:23.545 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:23.545 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":50,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:34:36.423005+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:34:32.990827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-14T08:34:33.715803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:35:23.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.546+0000 7f600ccc8640 1 -- 172.21.15.172:0/573082687 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5fe806c8f0 msgr2=0x7f5fe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.546+0000 7f600ccc8640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5fe806c8f0 0x7f5fe806edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0011e0 tx=0x7f5ffc001ba0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.546+0000 7f600ccc8640 1 -- 172.21.15.172:0/573082687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f600815e890 msgr2=0x7f6008160ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.546+0000 7f600ccc8640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f600815e890 0x7f6008160ef0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6000009ee0 tx=0x7f6000004a40 comp rx=0 tx=0).stop 2024-04-14T08:35:23.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.546+0000 7f600ccc8640 1 -- 172.21.15.172:0/573082687 shutdown_connections 2024-04-14T08:35:23.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.546+0000 7f600ccc8640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5fe806c8f0 0x7f5fe806edb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.546+0000 7f600ccc8640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f600815e890 0x7f6008160ef0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.546+0000 7f600ccc8640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60080d2830 0x7f60081f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.546+0000 7f600ccc8640 1 --2- 172.21.15.172:0/573082687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60080cedd0 0x7f60080ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.546+0000 7f600ccc8640 1 -- 172.21.15.172:0/573082687 >> 172.21.15.172:0/573082687 conn(0x7f600809d590 msgr2=0x7f6008074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:23.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.546+0000 7f600ccc8640 1 -- 172.21.15.172:0/573082687 shutdown_connections 2024-04-14T08:35:23.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.546+0000 7f600ccc8640 1 -- 172.21.15.172:0/573082687 wait complete. 2024-04-14T08:35:23.558 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:35:23.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.660+0000 7fe627cde640 1 Processor -- start 2024-04-14T08:35:23.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.661+0000 7fe627cde640 1 -- start start 2024-04-14T08:35:23.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.661+0000 7fe627cde640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe62015e430 0x7fe620162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:23.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.661+0000 7fe627cde640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe620162540 0x7fe6201629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:23.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.661+0000 7fe627cde640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6200cefb0 con 0x7fe6200b3a30 2024-04-14T08:35:23.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.661+0000 7fe627cde640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe6200cf290 con 0x7fe62015e430 2024-04-14T08:35:23.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.661+0000 7fe627cde640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe6200cf570 con 0x7fe620162540 2024-04-14T08:35:23.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.661+0000 7fe625a7a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe62015e430 0x7fe620162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:23.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.662+0000 7fe625a7a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe62015e430 0x7fe620162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35958/0 (socket says 172.21.15.172:35958) 2024-04-14T08:35:23.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.662+0000 7fe625a7a640 1 -- 172.21.15.172:0/3101641851 learned_addr learned my addr 172.21.15.172:0/3101641851 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:23.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.662+0000 7fe625279640 1 --2- 172.21.15.172:0/3101641851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe620162540 0x7fe6201629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:23.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.662+0000 7fe625a7a640 1 -- 172.21.15.172:0/3101641851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe620162540 msgr2=0x7fe6201629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:23.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.662+0000 7fe625a7a640 1 --2- 172.21.15.172:0/3101641851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe620162540 0x7fe6201629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.662+0000 7fe625a7a640 1 -- 172.21.15.172:0/3101641851 >> v1:172.21.15.177:6789/0 conn(0x7fe6200b3a30 legacy=0x7fe6200b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.662+0000 7fe625a7a640 1 -- 172.21.15.172:0/3101641851 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6200cf850 con 0x7fe62015e430 2024-04-14T08:35:23.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.663+0000 7fe625a7a640 1 --2- 172.21.15.172:0/3101641851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62015e430 0x7fe620162000 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009870 tx=0x7fe61c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f76fdd15c52a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:23.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.663+0000 7fe624a78640 1 -- 172.21.15.172:0/3101641851 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c034070 con 0x7fe62015e430 2024-04-14T08:35:23.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.663+0000 7fe624a78640 1 -- 172.21.15.172:0/3101641851 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe61c030840 con 0x7fe62015e430 2024-04-14T08:35:23.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.663+0000 7fe624a78640 1 -- 172.21.15.172:0/3101641851 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c030c10 con 0x7fe62015e430 2024-04-14T08:35:23.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.664+0000 7fe627cde640 1 -- 172.21.15.172:0/3101641851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62015e430 msgr2=0x7fe620162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.664+0000 7fe627cde640 1 --2- 172.21.15.172:0/3101641851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62015e430 0x7fe620162000 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009870 tx=0x7fe61c02b980 comp rx=0 tx=0).stop 2024-04-14T08:35:23.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.664+0000 7fe627cde640 1 -- 172.21.15.172:0/3101641851 shutdown_connections 2024-04-14T08:35:23.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.664+0000 7fe627cde640 1 --2- 172.21.15.172:0/3101641851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe620162540 0x7fe6201629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.664+0000 7fe627cde640 1 --2- 172.21.15.172:0/3101641851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe62015e430 0x7fe620162000 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.664+0000 7fe627cde640 1 -- 172.21.15.172:0/3101641851 >> 172.21.15.172:0/3101641851 conn(0x7fe62009d560 msgr2=0x7fe6200d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:23.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.664+0000 7fe627cde640 1 -- 172.21.15.172:0/3101641851 shutdown_connections 2024-04-14T08:35:23.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.664+0000 7fe627cde640 1 -- 172.21.15.172:0/3101641851 wait complete. 2024-04-14T08:35:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.665+0000 7fe627cde640 1 Processor -- start 2024-04-14T08:35:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.665+0000 7fe627cde640 1 -- start start 2024-04-14T08:35:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe627cde640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6200b3a30 0x7fe6201f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe627cde640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe62015e430 0x7fe62014e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe627cde640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe620162540 0x7fe6200b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe627cde640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200d4e20 con 0x7fe620162540 2024-04-14T08:35:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe627cde640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe62015e810 con 0x7fe6200b3a30 2024-04-14T08:35:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe627cde640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe6200d0050 con 0x7fe62015e430 2024-04-14T08:35:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe62627b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe620162540 0x7fe6200b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe62627b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe620162540 0x7fe6200b8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35968/0 (socket says 172.21.15.172:35968) 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe62627b640 1 -- 172.21.15.172:0/2343313489 learned_addr learned my addr 172.21.15.172:0/2343313489 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe625a7a640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6200b3a30 0x7fe6201f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe625279640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe62015e430 0x7fe62014e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe62627b640 1 -- 172.21.15.172:0/2343313489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe62015e430 msgr2=0x7fe62014e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe62627b640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe62015e430 0x7fe62014e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe62627b640 1 -- 172.21.15.172:0/2343313489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6200b3a30 msgr2=0x7fe6201f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe62627b640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6200b3a30 0x7fe6201f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe62627b640 1 -- 172.21.15.172:0/2343313489 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6200c94e0 con 0x7fe620162540 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe625279640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe62015e430 0x7fe62014e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe62627b640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe620162540 0x7fe6200b8b00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe61800c7a0 tx=0x7fe61800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe6167fc640 1 -- 172.21.15.172:0/2343313489 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618003b70 con 0x7fe620162540 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe6167fc640 1 -- 172.21.15.172:0/2343313489 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe618003d30 con 0x7fe620162540 2024-04-14T08:35:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe627cde640 1 -- 172.21.15.172:0/2343313489 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62014eba0 con 0x7fe620162540 2024-04-14T08:35:23.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.666+0000 7fe6167fc640 1 -- 172.21.15.172:0/2343313489 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61800aa90 con 0x7fe620162540 2024-04-14T08:35:23.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.667+0000 7fe627cde640 1 -- 172.21.15.172:0/2343313489 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6200cbb20 con 0x7fe620162540 2024-04-14T08:35:23.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.667+0000 7fe627cde640 1 -- 172.21.15.172:0/2343313489 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe620068590 con 0x7fe620162540 2024-04-14T08:35:23.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.668+0000 7fe6167fc640 1 -- 172.21.15.172:0/2343313489 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe61800ac30 con 0x7fe620162540 2024-04-14T08:35:23.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.669+0000 7fe6167fc640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe60406c8f0 0x7fe60406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:23.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.669+0000 7fe625a7a640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe60406c8f0 0x7fe60406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:23.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.669+0000 7fe6167fc640 1 -- 172.21.15.172:0/2343313489 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6101+0+0 (secure 0 0 0) 0x7fe61808bb70 con 0x7fe620162540 2024-04-14T08:35:23.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.669+0000 7fe625a7a640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe60406c8f0 0x7fe60406edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe61c036000 tx=0x7fe61c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.674+0000 7fe6167fc640 1 -- 172.21.15.172:0/2343313489 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe62014eba0 con 0x7fe620162540 2024-04-14T08:35:23.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.891+0000 7fe627cde640 1 -- 172.21.15.172:0/2343313489 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe6200c7e00 con 0x7fe620162540 2024-04-14T08:35:23.892 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:35:23.892+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:35:23.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.893+0000 7fe6167fc640 1 -- 172.21.15.172:0/2343313489 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) ==== 161+0+0 (secure 0 0 0) 0x7fe618059e10 con 0x7fe620162540 2024-04-14T08:35:23.894 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:35:23.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.896+0000 7fe627cde640 1 -- 172.21.15.172:0/2343313489 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe60406c8f0 msgr2=0x7fe60406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.896+0000 7fe627cde640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe60406c8f0 0x7fe60406edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe61c036000 tx=0x7fe61c008030 comp rx=0 tx=0).stop 2024-04-14T08:35:23.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.896+0000 7fe627cde640 1 -- 172.21.15.172:0/2343313489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe620162540 msgr2=0x7fe6200b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:23.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.896+0000 7fe627cde640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe620162540 0x7fe6200b8b00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe61800c7a0 tx=0x7fe61800cc60 comp rx=0 tx=0).stop 2024-04-14T08:35:23.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.896+0000 7fe627cde640 1 -- 172.21.15.172:0/2343313489 shutdown_connections 2024-04-14T08:35:23.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.896+0000 7fe627cde640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe60406c8f0 0x7fe60406edb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.896+0000 7fe627cde640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe620162540 0x7fe6200b8b00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.896+0000 7fe627cde640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe62015e430 0x7fe62014e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.896+0000 7fe627cde640 1 --2- 172.21.15.172:0/2343313489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6200b3a30 0x7fe6201f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:23.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.896+0000 7fe627cde640 1 -- 172.21.15.172:0/2343313489 >> 172.21.15.172:0/2343313489 conn(0x7fe62009d560 msgr2=0x7fe6200d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:23.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.896+0000 7fe627cde640 1 -- 172.21.15.172:0/2343313489 shutdown_connections 2024-04-14T08:35:23.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:23.896+0000 7fe627cde640 1 -- 172.21.15.172:0/2343313489 wait complete. 2024-04-14T08:35:23.901 INFO:tasks.ceph.mds.c.smithi172.stderr: -6012> 2024-04-14T08:34:40.697+0000 7f3a039f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:35:23.907 INFO:tasks.ceph.mds.c.smithi172.stderr: -3136> 2024-04-14T08:34:47.017+0000 7f39ff9e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5782=0-4194304@1} caps={5782=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5782 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x558f4cb16680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-14T08:35:23.909 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-14T08:35:23.934 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:35:24.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.012+0000 7f11a7408640 1 Processor -- start 2024-04-14T08:35:24.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.012+0000 7f11a7408640 1 -- start start 2024-04-14T08:35:24.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.013+0000 7f11a7408640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11a0162710 0x7f11a00d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:24.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.013+0000 7f11a7408640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f11a0162ba0 0x7f11a00d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:24.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.013+0000 7f11a7408640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a00cefb0 con 0x7f11a015e430 2024-04-14T08:35:24.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.013+0000 7f11a7408640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f11a00cf290 con 0x7f11a0162710 2024-04-14T08:35:24.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.013+0000 7f11a7408640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f11a00cf570 con 0x7f11a0162ba0 2024-04-14T08:35:24.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.013+0000 7f11a49a3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11a0162710 0x7f11a00d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:24.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.013+0000 7f11a49a3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11a0162710 0x7f11a00d1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35976/0 (socket says 172.21.15.172:35976) 2024-04-14T08:35:24.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.013+0000 7f11a49a3640 1 -- 172.21.15.172:0/1457275400 learned_addr learned my addr 172.21.15.172:0/1457275400 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:24.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.013+0000 7f11a49a3640 1 -- 172.21.15.172:0/1457275400 >> v2:172.21.15.177:3300/0 conn(0x7f11a0162ba0 msgr2=0x7f11a00d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:24.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.013+0000 7f11a49a3640 1 --2- 172.21.15.172:0/1457275400 >> v2:172.21.15.177:3300/0 conn(0x7f11a0162ba0 0x7f11a00d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:24.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.013+0000 7f11a49a3640 1 -- 172.21.15.172:0/1457275400 >> v1:172.21.15.177:6789/0 conn(0x7f11a015e430 legacy=0x7f11a0162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:24.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.013+0000 7f11a49a3640 1 -- 172.21.15.172:0/1457275400 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a00cf850 con 0x7f11a0162710 2024-04-14T08:35:24.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.014+0000 7f11a49a3640 1 --2- 172.21.15.172:0/1457275400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0162710 0x7f11a00d1df0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f11a00d1840 tx=0x7f119c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f2b467327c8c78 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:24.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.014+0000 7f1197fff640 1 -- 172.21.15.172:0/1457275400 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c035020 con 0x7f11a0162710 2024-04-14T08:35:24.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.014+0000 7f1197fff640 1 -- 172.21.15.172:0/1457275400 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119c031870 con 0x7f11a0162710 2024-04-14T08:35:24.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.014+0000 7f1197fff640 1 -- 172.21.15.172:0/1457275400 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c031c40 con 0x7f11a0162710 2024-04-14T08:35:24.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.015+0000 7f11a7408640 1 -- 172.21.15.172:0/1457275400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0162710 msgr2=0x7f11a00d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:24.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.015+0000 7f11a7408640 1 --2- 172.21.15.172:0/1457275400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0162710 0x7f11a00d1df0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f11a00d1840 tx=0x7f119c02b910 comp rx=0 tx=0).stop 2024-04-14T08:35:24.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.015+0000 7f11a7408640 1 -- 172.21.15.172:0/1457275400 shutdown_connections 2024-04-14T08:35:24.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.015+0000 7f11a7408640 1 --2- 172.21.15.172:0/1457275400 >> v2:172.21.15.177:3300/0 conn(0x7f11a0162ba0 0x7f11a00d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:24.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.015+0000 7f11a7408640 1 --2- 172.21.15.172:0/1457275400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0162710 0x7f11a00d1df0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:24.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.015+0000 7f11a7408640 1 -- 172.21.15.172:0/1457275400 >> 172.21.15.172:0/1457275400 conn(0x7f11a009d560 msgr2=0x7f11a00d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:24.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.015+0000 7f11a7408640 1 -- 172.21.15.172:0/1457275400 shutdown_connections 2024-04-14T08:35:24.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.015+0000 7f11a7408640 1 -- 172.21.15.172:0/1457275400 wait complete. 2024-04-14T08:35:24.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.016+0000 7f11a7408640 1 Processor -- start 2024-04-14T08:35:24.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.016+0000 7f11a7408640 1 -- start start 2024-04-14T08:35:24.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a7408640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11a015e430 0x7f11a01594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:24.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a7408640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0162710 0x7f11a00b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:24.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a7408640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11a0162ba0 0x7f11a00b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:24.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a7408640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11a00d4e20 con 0x7f11a0162710 2024-04-14T08:35:24.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a7408640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f11a015e810 con 0x7f11a015e430 2024-04-14T08:35:24.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a7408640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11a00d0050 con 0x7f11a0162ba0 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a49a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0162710 0x7f11a00b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a49a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0162710 0x7f11a00b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35984/0 (socket says 172.21.15.172:35984) 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a49a3640 1 -- 172.21.15.172:0/3458015590 learned_addr learned my addr 172.21.15.172:0/3458015590 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a49a3640 1 -- 172.21.15.172:0/3458015590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11a0162ba0 msgr2=0x7f11a00b7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a49a3640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11a0162ba0 0x7f11a00b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a49a3640 1 -- 172.21.15.172:0/3458015590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11a015e430 msgr2=0x7f11a01594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a51a4640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11a015e430 0x7f11a01594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a49a3640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11a015e430 0x7f11a01594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a49a3640 1 -- 172.21.15.172:0/3458015590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a015ea30 con 0x7f11a0162710 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.017+0000 7f11a49a3640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0162710 0x7f11a00b3f40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f119c000c00 tx=0x7f119c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.018+0000 7f1195ffb640 1 -- 172.21.15.172:0/3458015590 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c03ad60 con 0x7f11a0162710 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.018+0000 7f1195ffb640 1 -- 172.21.15.172:0/3458015590 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119c0443f0 con 0x7f11a0162710 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.018+0000 7f11a7408640 1 -- 172.21.15.172:0/3458015590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a00b4990 con 0x7f11a0162710 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.018+0000 7f1195ffb640 1 -- 172.21.15.172:0/3458015590 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c0447e0 con 0x7f11a0162710 2024-04-14T08:35:24.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.018+0000 7f11a7408640 1 -- 172.21.15.172:0/3458015590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a00b4f50 con 0x7f11a0162710 2024-04-14T08:35:24.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.019+0000 7f11a7408640 1 -- 172.21.15.172:0/3458015590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a0068590 con 0x7f11a0162710 2024-04-14T08:35:24.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.019+0000 7f1195ffb640 1 -- 172.21.15.172:0/3458015590 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f119c038020 con 0x7f11a0162710 2024-04-14T08:35:24.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.020+0000 7f1195ffb640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f118406c8f0 0x7f118406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:24.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.020+0000 7f11a51a4640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f118406c8f0 0x7f118406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:24.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.020+0000 7f1195ffb640 1 -- 172.21.15.172:0/3458015590 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6187+0+0 (secure 0 0 0) 0x7f119c0adf10 con 0x7f11a0162710 2024-04-14T08:35:24.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.020+0000 7f11a51a4640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f118406c8f0 0x7f118406edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f119000a730 tx=0x7f1190008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:24.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.024+0000 7f1195ffb640 1 -- 172.21.15.172:0/3458015590 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f119c03f070 con 0x7f11a0162710 2024-04-14T08:35:24.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.241+0000 7f11a7408640 1 -- 172.21.15.172:0/3458015590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f11a00d18a0 con 0x7f11a0162710 2024-04-14T08:35:24.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.895+0000 7f1195ffb640 1 -- 172.21.15.172:0/3458015590 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f119c07c1f0 con 0x7f11a0162710 2024-04-14T08:35:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.897+0000 7f11a7408640 1 -- 172.21.15.172:0/3458015590 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f118406c8f0 msgr2=0x7f118406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.897+0000 7f11a7408640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f118406c8f0 0x7f118406edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f119000a730 tx=0x7f1190008040 comp rx=0 tx=0).stop 2024-04-14T08:35:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.897+0000 7f11a7408640 1 -- 172.21.15.172:0/3458015590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0162710 msgr2=0x7f11a00b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.897+0000 7f11a7408640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0162710 0x7f11a00b3f40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f119c000c00 tx=0x7f119c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:35:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.897+0000 7f11a7408640 1 -- 172.21.15.172:0/3458015590 shutdown_connections 2024-04-14T08:35:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.897+0000 7f11a7408640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f118406c8f0 0x7f118406edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.897+0000 7f11a7408640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11a0162ba0 0x7f11a00b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.897+0000 7f11a7408640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0162710 0x7f11a00b3f40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.897+0000 7f11a7408640 1 --2- 172.21.15.172:0/3458015590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11a015e430 0x7f11a01594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:24.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.897+0000 7f11a7408640 1 -- 172.21.15.172:0/3458015590 >> 172.21.15.172:0/3458015590 conn(0x7f11a009d560 msgr2=0x7f11a00b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:24.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.897+0000 7f11a7408640 1 -- 172.21.15.172:0/3458015590 shutdown_connections 2024-04-14T08:35:24.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:24.897+0000 7f11a7408640 1 -- 172.21.15.172:0/3458015590 wait complete. 2024-04-14T08:35:24.910 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-14T08:35:25.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.056+0000 7f2a2ccde640 1 Processor -- start 2024-04-14T08:35:25.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.056+0000 7f2a2ccde640 1 -- start start 2024-04-14T08:35:25.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.057+0000 7f2a2ccde640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a280cc340 0x7f2a280ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:25.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.057+0000 7f2a2ccde640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a280cecc0 0x7f2a280c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:25.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.057+0000 7f2a2ccde640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a280cdb90 con 0x7f2a2815b350 2024-04-14T08:35:25.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.057+0000 7f2a2ccde640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2a280cf150 con 0x7f2a280cecc0 2024-04-14T08:35:25.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.057+0000 7f2a2ccde640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2a280cd940 con 0x7f2a280cc340 2024-04-14T08:35:25.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.057+0000 7f2a26d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a280cecc0 0x7f2a280c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:25.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.057+0000 7f2a26d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a280cecc0 0x7f2a280c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35988/0 (socket says 172.21.15.172:35988) 2024-04-14T08:35:25.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.057+0000 7f2a26d9d640 1 -- 172.21.15.172:0/2921133832 learned_addr learned my addr 172.21.15.172:0/2921133832 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:25.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.057+0000 7f2a26d9d640 1 -- 172.21.15.172:0/2921133832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a280cc340 msgr2=0x7f2a280ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:25.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.057+0000 7f2a26d9d640 1 --2- 172.21.15.172:0/2921133832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a280cc340 0x7f2a280ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:25.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.057+0000 7f2a26d9d640 1 -- 172.21.15.172:0/2921133832 >> v1:172.21.15.177:6789/0 conn(0x7f2a2815b350 legacy=0x7f2a280cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:25.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.057+0000 7f2a26d9d640 1 -- 172.21.15.172:0/2921133832 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a280cde70 con 0x7f2a280cecc0 2024-04-14T08:35:25.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.058+0000 7f2a26d9d640 1 --2- 172.21.15.172:0/2921133832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a280cecc0 0x7f2a280c4e70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009e50 tx=0x7f2a2002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1749e6398c60a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:25.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.058+0000 7f2a2559a640 1 -- 172.21.15.172:0/2921133832 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2002c9e0 con 0x7f2a280cecc0 2024-04-14T08:35:25.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.058+0000 7f2a2559a640 1 -- 172.21.15.172:0/2921133832 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a2002cb80 con 0x7f2a280cecc0 2024-04-14T08:35:25.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.058+0000 7f2a2559a640 1 -- 172.21.15.172:0/2921133832 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a200343f0 con 0x7f2a280cecc0 2024-04-14T08:35:25.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.059+0000 7f2a2ccde640 1 -- 172.21.15.172:0/2921133832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a280cecc0 msgr2=0x7f2a280c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:25.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.059+0000 7f2a2ccde640 1 --2- 172.21.15.172:0/2921133832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a280cecc0 0x7f2a280c4e70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009e50 tx=0x7f2a2002be40 comp rx=0 tx=0).stop 2024-04-14T08:35:25.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.059+0000 7f2a2ccde640 1 -- 172.21.15.172:0/2921133832 shutdown_connections 2024-04-14T08:35:25.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.059+0000 7f2a2ccde640 1 --2- 172.21.15.172:0/2921133832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a280cecc0 0x7f2a280c4e70 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:25.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.059+0000 7f2a2ccde640 1 --2- 172.21.15.172:0/2921133832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a280cc340 0x7f2a280ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:25.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.059+0000 7f2a2ccde640 1 -- 172.21.15.172:0/2921133832 >> 172.21.15.172:0/2921133832 conn(0x7f2a280b1a50 msgr2=0x7f2a280b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:25.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.059+0000 7f2a2ccde640 1 -- 172.21.15.172:0/2921133832 shutdown_connections 2024-04-14T08:35:25.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.059+0000 7f2a2ccde640 1 -- 172.21.15.172:0/2921133832 wait complete. 2024-04-14T08:35:25.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.060+0000 7f2a2ccde640 1 Processor -- start 2024-04-14T08:35:25.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.060+0000 7f2a2ccde640 1 -- start start 2024-04-14T08:35:25.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a2ccde640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a280cc340 0x7f2a280c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:25.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a2ccde640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a280cecc0 0x7f2a2814c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:25.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a2ccde640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a2815b350 0x7f2a280b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a2ccde640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a280cf560 con 0x7f2a280cecc0 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a2ccde640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a280ce4c0 con 0x7f2a2815b350 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a2ccde640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2a280c53b0 con 0x7f2a280cc340 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a25d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a280cecc0 0x7f2a2814c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a25d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a280cecc0 0x7f2a2814c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35992/0 (socket says 172.21.15.172:35992) 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a25d9b640 1 -- 172.21.15.172:0/29924784 learned_addr learned my addr 172.21.15.172:0/29924784 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a26d9d640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a2815b350 0x7f2a280b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a25d9b640 1 -- 172.21.15.172:0/29924784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a280cc340 msgr2=0x7f2a280c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a25d9b640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a280cc340 0x7f2a280c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a25d9b640 1 -- 172.21.15.172:0/29924784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a2815b350 msgr2=0x7f2a280b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a25d9b640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a2815b350 0x7f2a280b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a25d9b640 1 -- 172.21.15.172:0/29924784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2814cce0 con 0x7f2a280cecc0 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a26d9d640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a2815b350 0x7f2a280b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:25.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a25d9b640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a280cecc0 0x7f2a2814c490 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b480 tx=0x7f2a1000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:25.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.061+0000 7f2a1effd640 1 -- 172.21.15.172:0/29924784 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10014070 con 0x7f2a280cecc0 2024-04-14T08:35:25.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.062+0000 7f2a1effd640 1 -- 172.21.15.172:0/29924784 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a10010870 con 0x7f2a280cecc0 2024-04-14T08:35:25.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.062+0000 7f2a2ccde640 1 -- 172.21.15.172:0/29924784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a280b2b40 con 0x7f2a280cecc0 2024-04-14T08:35:25.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.062+0000 7f2a1effd640 1 -- 172.21.15.172:0/29924784 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10010c60 con 0x7f2a280cecc0 2024-04-14T08:35:25.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.062+0000 7f2a2ccde640 1 -- 172.21.15.172:0/29924784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a280b3100 con 0x7f2a280cecc0 2024-04-14T08:35:25.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.062+0000 7f2a2ccde640 1 -- 172.21.15.172:0/29924784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a280686a0 con 0x7f2a280cecc0 2024-04-14T08:35:25.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.063+0000 7f2a1effd640 1 -- 172.21.15.172:0/29924784 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a10017070 con 0x7f2a280cecc0 2024-04-14T08:35:25.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.063+0000 7f2a1effd640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a0806c8f0 0x7f2a0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:25.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.063+0000 7f2a1effd640 1 -- 172.21.15.172:0/29924784 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6187+0+0 (secure 0 0 0) 0x7f2a1008bb40 con 0x7f2a280cecc0 2024-04-14T08:35:25.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.063+0000 7f2a2659c640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a0806c8f0 0x7f2a0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:25.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.064+0000 7f2a2659c640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a0806c8f0 0x7f2a0806edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009650 tx=0x7f2a18006ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:25.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.068+0000 7f2a1effd640 1 -- 172.21.15.172:0/29924784 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2a10059e20 con 0x7f2a280cecc0 2024-04-14T08:35:25.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.244+0000 7f2a2ccde640 1 -- 172.21.15.172:0/29924784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2a280c5800 con 0x7f2a280cecc0 2024-04-14T08:35:25.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.904+0000 7f2a1effd640 1 -- 172.21.15.172:0/29924784 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) ==== 181+0+0 (secure 0 0 0) 0x7f2a10059720 con 0x7f2a280cecc0 2024-04-14T08:35:25.906 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' removed 2024-04-14T08:35:25.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.907+0000 7f2a2ccde640 1 -- 172.21.15.172:0/29924784 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a0806c8f0 msgr2=0x7f2a0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:25.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.907+0000 7f2a2ccde640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a0806c8f0 0x7f2a0806edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009650 tx=0x7f2a18006ba0 comp rx=0 tx=0).stop 2024-04-14T08:35:25.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.907+0000 7f2a2ccde640 1 -- 172.21.15.172:0/29924784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a280cecc0 msgr2=0x7f2a2814c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:25.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.907+0000 7f2a2ccde640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a280cecc0 0x7f2a2814c490 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b480 tx=0x7f2a1000b940 comp rx=0 tx=0).stop 2024-04-14T08:35:25.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.907+0000 7f2a2ccde640 1 -- 172.21.15.172:0/29924784 shutdown_connections 2024-04-14T08:35:25.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.907+0000 7f2a2ccde640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a0806c8f0 0x7f2a0806edb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:25.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.907+0000 7f2a2ccde640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a2815b350 0x7f2a280b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:25.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.907+0000 7f2a2ccde640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a280cecc0 0x7f2a2814c490 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:25.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.907+0000 7f2a2ccde640 1 --2- 172.21.15.172:0/29924784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a280cc340 0x7f2a280c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:25.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.907+0000 7f2a2ccde640 1 -- 172.21.15.172:0/29924784 >> 172.21.15.172:0/29924784 conn(0x7f2a280b1a50 msgr2=0x7f2a280ada50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:25.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.907+0000 7f2a2ccde640 1 -- 172.21.15.172:0/29924784 shutdown_connections 2024-04-14T08:35:25.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:25.907+0000 7f2a2ccde640 1 -- 172.21.15.172:0/29924784 wait complete. 2024-04-14T08:35:25.921 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-14T08:35:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.029+0000 7f11194b7640 1 Processor -- start 2024-04-14T08:35:26.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.029+0000 7f11194b7640 1 -- start start 2024-04-14T08:35:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f11194b7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f111409b180 0x7f111415d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f11194b7640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11140cfc30 0x7f11140ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f11194b7640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11140d01a0 con 0x7f111415c5a0 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f11194b7640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f111415d990 con 0x7f11140cfc30 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f11194b7640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f11140d0db0 con 0x7f111409b180 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f11137fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11140cfc30 0x7f11140ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f1112ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f111415c5a0 0x7f111409aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50882/0 (socket says 172.21.15.172:50882) 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f1112ffd640 1 -- 172.21.15.172:0/2120089009 learned_addr learned my addr 172.21.15.172:0/2120089009 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f11137fe640 1 -- 172.21.15.172:0/2120089009 >> v2:172.21.15.177:3300/0 conn(0x7f111409b180 msgr2=0x7f111415d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f11137fe640 1 --2- 172.21.15.172:0/2120089009 >> v2:172.21.15.177:3300/0 conn(0x7f111409b180 0x7f111415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f11137fe640 1 -- 172.21.15.172:0/2120089009 >> v1:172.21.15.172:6789/0 conn(0x7f111415c5a0 legacy=0x7f111409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f11127fc640 1 -- 172.21.15.172:0/2120089009 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2790849023 0 0) 0x7f11140d01a0 con 0x7f111415c5a0 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.030+0000 7f11137fe640 1 -- 172.21.15.172:0/2120089009 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11140d0480 con 0x7f11140cfc30 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.031+0000 7f11137fe640 1 --2- 172.21.15.172:0/2120089009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11140cfc30 0x7f11140ccce0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f11140d1910 tx=0x7f110c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d79bab5afd1fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.031+0000 7f11127fc640 1 -- 172.21.15.172:0/2120089009 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c039070 con 0x7f11140cfc30 2024-04-14T08:35:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.031+0000 7f11127fc640 1 -- 172.21.15.172:0/2120089009 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c02cc80 con 0x7f11140cfc30 2024-04-14T08:35:26.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.031+0000 7f11127fc640 1 -- 172.21.15.172:0/2120089009 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c034660 con 0x7f11140cfc30 2024-04-14T08:35:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.031+0000 7f11194b7640 1 -- 172.21.15.172:0/2120089009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11140cfc30 msgr2=0x7f11140ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.031+0000 7f11194b7640 1 --2- 172.21.15.172:0/2120089009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11140cfc30 0x7f11140ccce0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f11140d1910 tx=0x7f110c02bed0 comp rx=0 tx=0).stop 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.031+0000 7f11194b7640 1 -- 172.21.15.172:0/2120089009 shutdown_connections 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.031+0000 7f11194b7640 1 --2- 172.21.15.172:0/2120089009 >> v2:172.21.15.177:3300/0 conn(0x7f111409b180 0x7f111415d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.031+0000 7f11194b7640 1 --2- 172.21.15.172:0/2120089009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11140cfc30 0x7f11140ccce0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.032+0000 7f11194b7640 1 -- 172.21.15.172:0/2120089009 >> 172.21.15.172:0/2120089009 conn(0x7f11140b3b50 msgr2=0x7f11140b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.032+0000 7f11194b7640 1 -- 172.21.15.172:0/2120089009 shutdown_connections 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.032+0000 7f11194b7640 1 -- 172.21.15.172:0/2120089009 wait complete. 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.033+0000 7f11194b7640 1 Processor -- start 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.033+0000 7f11194b7640 1 -- start start 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.033+0000 7f11194b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f111409b180 0x7f11140b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.033+0000 7f11194b7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11140cfc30 0x7f11141f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.033+0000 7f11194b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f111415c5a0 0x7f11140cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.033+0000 7f11194b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11140cd220 con 0x7f111409b180 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.033+0000 7f11194b7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f11140d0a40 con 0x7f11140cfc30 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.033+0000 7f11194b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11140cd3a0 con 0x7f111415c5a0 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.033+0000 7f1112ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f111409b180 0x7f11140b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.033+0000 7f1112ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f111409b180 0x7f11140b6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35996/0 (socket says 172.21.15.172:35996) 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.033+0000 7f1112ffd640 1 -- 172.21.15.172:0/2467714678 learned_addr learned my addr 172.21.15.172:0/2467714678 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f110bfff640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11140cfc30 0x7f11141f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f11137fe640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f111415c5a0 0x7f11140cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f1112ffd640 1 -- 172.21.15.172:0/2467714678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f111415c5a0 msgr2=0x7f11140cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f1112ffd640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f111415c5a0 0x7f11140cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f1112ffd640 1 -- 172.21.15.172:0/2467714678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11140cfc30 msgr2=0x7f11141f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f1112ffd640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11140cfc30 0x7f11141f3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f1112ffd640 1 -- 172.21.15.172:0/2467714678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11141f44a0 con 0x7f111409b180 2024-04-14T08:35:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f1112ffd640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f111409b180 0x7f11140b6ce0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f110400c430 tx=0x7f110400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f110b7fe640 1 -- 172.21.15.172:0/2467714678 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11040074e0 con 0x7f111409b180 2024-04-14T08:35:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f110b7fe640 1 -- 172.21.15.172:0/2467714678 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11040047e0 con 0x7f111409b180 2024-04-14T08:35:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f11194b7640 1 -- 172.21.15.172:0/2467714678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11140b72e0 con 0x7f111409b180 2024-04-14T08:35:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f110b7fe640 1 -- 172.21.15.172:0/2467714678 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104004bd0 con 0x7f111409b180 2024-04-14T08:35:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.034+0000 7f11194b7640 1 -- 172.21.15.172:0/2467714678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11140b78a0 con 0x7f111409b180 2024-04-14T08:35:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.035+0000 7f11194b7640 1 -- 172.21.15.172:0/2467714678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111406c8b0 con 0x7f111409b180 2024-04-14T08:35:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.036+0000 7f110b7fe640 1 -- 172.21.15.172:0/2467714678 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1104011070 con 0x7f111409b180 2024-04-14T08:35:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.036+0000 7f110b7fe640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10f406c8f0 0x7f10f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.036+0000 7f110b7fe640 1 -- 172.21.15.172:0/2467714678 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5749+0+0 (secure 0 0 0) 0x7f110408b590 con 0x7f111409b180 2024-04-14T08:35:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.036+0000 7f110bfff640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10f406c8f0 0x7f10f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.036+0000 7f110bfff640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10f406c8f0 0x7f10f406edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f11140cb6a0 tx=0x7f10fc009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.040+0000 7f110b7fe640 1 -- 172.21.15.172:0/2467714678 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1104059a20 con 0x7f111409b180 2024-04-14T08:35:26.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.216+0000 7f11194b7640 1 -- 172.21.15.172:0/2467714678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f11140686a0 con 0x7f111409b180 2024-04-14T08:35:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.908+0000 7f110b7fe640 1 -- 172.21.15.172:0/2467714678 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) ==== 169+0+0 (secure 0 0 0) 0x7f1104059320 con 0x7f111409b180 2024-04-14T08:35:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' removed 2024-04-14T08:35:26.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.911+0000 7f11194b7640 1 -- 172.21.15.172:0/2467714678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10f406c8f0 msgr2=0x7f10f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:26.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.911+0000 7f11194b7640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10f406c8f0 0x7f10f406edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f11140cb6a0 tx=0x7f10fc009110 comp rx=0 tx=0).stop 2024-04-14T08:35:26.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.911+0000 7f11194b7640 1 -- 172.21.15.172:0/2467714678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f111409b180 msgr2=0x7f11140b6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:26.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.911+0000 7f11194b7640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f111409b180 0x7f11140b6ce0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f110400c430 tx=0x7f110400c830 comp rx=0 tx=0).stop 2024-04-14T08:35:26.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.912+0000 7f11194b7640 1 -- 172.21.15.172:0/2467714678 shutdown_connections 2024-04-14T08:35:26.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.912+0000 7f11194b7640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10f406c8f0 0x7f10f406edb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:26.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.912+0000 7f11194b7640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f111415c5a0 0x7f11140cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:26.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.912+0000 7f11194b7640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11140cfc30 0x7f11141f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:26.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.912+0000 7f11194b7640 1 --2- 172.21.15.172:0/2467714678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f111409b180 0x7f11140b6ce0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:26.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.912+0000 7f11194b7640 1 -- 172.21.15.172:0/2467714678 >> 172.21.15.172:0/2467714678 conn(0x7f11140b3b50 msgr2=0x7f11140b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:26.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.912+0000 7f11194b7640 1 -- 172.21.15.172:0/2467714678 shutdown_connections 2024-04-14T08:35:26.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:26.912+0000 7f11194b7640 1 -- 172.21.15.172:0/2467714678 wait complete. 2024-04-14T08:35:26.925 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-14T08:35:27.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.029+0000 7f6149915640 1 Processor -- start 2024-04-14T08:35:27.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.029+0000 7f6149915640 1 -- start start 2024-04-14T08:35:27.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.030+0000 7f6149915640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61440d0cc0 0x7f61440d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.030+0000 7f6149915640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f61440d4e20 0x7f6144161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.030+0000 7f6149915640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61440d15e0 con 0x7f61440d49c0 2024-04-14T08:35:27.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.030+0000 7f6149915640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f61440ceec0 con 0x7f61440d4e20 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.030+0000 7f6149915640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f61440cfd00 con 0x7f61440d0cc0 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.030+0000 7f61427fc640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f61440d49c0 0x7f6144160f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50884/0 (socket says 172.21.15.172:50884) 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.030+0000 7f61427fc640 1 -- 172.21.15.172:0/2427333548 learned_addr learned my addr 172.21.15.172:0/2427333548 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2558407275 0 0) 0x7f61440d15e0 con 0x7f61440d49c0 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f612c003520 con 0x7f61440d49c0 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f61437fe640 1 --2- 172.21.15.172:0/2427333548 >> v2:172.21.15.177:3300/0 conn(0x7f61440d4e20 0x7f6144161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6142ffd640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61440d0cc0 0x7f61440d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1848858047 0 0) 0x7f612c003520 con 0x7f61440d49c0 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61440d0cc0 msgr2=0x7f61440d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61440d0cc0 0x7f61440d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 >> v2:172.21.15.177:3300/0 conn(0x7f61440d4e20 msgr2=0x7f6144161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 --2- 172.21.15.172:0/2427333548 >> v2:172.21.15.177:3300/0 conn(0x7f61440d4e20 0x7f6144161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61440cf0f0 con 0x7f61440d49c0 2024-04-14T08:35:27.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f6138001670 con 0x7f61440d49c0 2024-04-14T08:35:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 >> v1:172.21.15.172:6789/0 conn(0x7f61440d49c0 legacy=0x7f6144160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f612c0039b0 0x7f612c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f612c004800 0x7f612c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612c0051c0 0x7f612c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61440cfd00 con 0x7f612c0051c0 2024-04-14T08:35:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f612c00bfb0 con 0x7f612c004800 2024-04-14T08:35:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f612c00c180 con 0x7f612c0039b0 2024-04-14T08:35:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6142ffd640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61440d0cc0 0x7f61440d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f61437fe640 1 --2- 172.21.15.172:0/2427333548 >> v2:172.21.15.177:3300/0 conn(0x7f61440d4e20 0x7f6144161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f61437fe640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612c0051c0 0x7f612c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f61427fc640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f612c004800 0x7f612c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f6142ffd640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f612c0039b0 0x7f612c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f61437fe640 1 -- 172.21.15.172:0/2427333548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f612c0039b0 msgr2=0x7f612c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:27.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f61437fe640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f612c0039b0 0x7f612c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f61437fe640 1 -- 172.21.15.172:0/2427333548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f612c004800 msgr2=0x7f612c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:27.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f61437fe640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f612c004800 0x7f612c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.031+0000 7f61437fe640 1 -- 172.21.15.172:0/2427333548 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c003520 con 0x7f612c0051c0 2024-04-14T08:35:27.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.032+0000 7f61437fe640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612c0051c0 0x7f612c00b9c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f613c0098d0 tx=0x7f612c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a467f6d939611b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:27.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.032+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c034070 con 0x7f612c0051c0 2024-04-14T08:35:27.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.032+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613c030680 con 0x7f612c0051c0 2024-04-14T08:35:27.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.032+0000 7f6141ffb640 1 -- 172.21.15.172:0/2427333548 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c00f040 con 0x7f612c0051c0 2024-04-14T08:35:27.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.033+0000 7f6149915640 1 -- 172.21.15.172:0/2427333548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612c0051c0 msgr2=0x7f612c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:27.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.033+0000 7f6149915640 1 --2- 172.21.15.172:0/2427333548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612c0051c0 0x7f612c00b9c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f613c0098d0 tx=0x7f612c005830 comp rx=0 tx=0).stop 2024-04-14T08:35:27.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.033+0000 7f61437fe640 1 -- 172.21.15.172:0/2427333548 reap_dead start 2024-04-14T08:35:27.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.033+0000 7f6149915640 1 -- 172.21.15.172:0/2427333548 shutdown_connections 2024-04-14T08:35:27.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.033+0000 7f6149915640 1 -- 172.21.15.172:0/2427333548 >> 172.21.15.172:0/2427333548 conn(0x7f614409d580 msgr2=0x7f61440d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:27.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.033+0000 7f6149915640 1 -- 172.21.15.172:0/2427333548 shutdown_connections 2024-04-14T08:35:27.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.033+0000 7f6149915640 1 -- 172.21.15.172:0/2427333548 wait complete. 2024-04-14T08:35:27.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.034+0000 7f6149915640 1 Processor -- start 2024-04-14T08:35:27.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.034+0000 7f6149915640 1 -- start start 2024-04-14T08:35:27.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6149915640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612c0051c0 0x7f61441e3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6149915640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61440c91a0 0x7f61440cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6149915640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61440cde50 0x7f61440b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6149915640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61440cf990 con 0x7f612c0051c0 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6149915640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61440cf6b0 con 0x7f61440cde50 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6149915640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f61440d32b0 con 0x7f61440c91a0 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6142ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612c0051c0 0x7f61441e3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6142ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612c0051c0 0x7f61441e3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36016/0 (socket says 172.21.15.172:36016) 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6142ffd640 1 -- 172.21.15.172:0/2362538046 learned_addr learned my addr 172.21.15.172:0/2362538046 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f61427fc640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61440c91a0 0x7f61440cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f61437fe640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61440cde50 0x7f61440b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6142ffd640 1 -- 172.21.15.172:0/2362538046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61440c91a0 msgr2=0x7f61440cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6142ffd640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61440c91a0 0x7f61440cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6142ffd640 1 -- 172.21.15.172:0/2362538046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61440cde50 msgr2=0x7f61440b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6142ffd640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61440cde50 0x7f61440b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f6142ffd640 1 -- 172.21.15.172:0/2362538046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614409ef10 con 0x7f612c0051c0 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f61437fe640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61440cde50 0x7f61440b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.035+0000 7f61427fc640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61440c91a0 0x7f61440cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.036+0000 7f6142ffd640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612c0051c0 0x7f61441e3ae0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6134004670 tx=0x7f6134007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:27.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.036+0000 7f612bfff640 1 -- 172.21.15.172:0/2362538046 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134015680 con 0x7f612c0051c0 2024-04-14T08:35:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.036+0000 7f612bfff640 1 -- 172.21.15.172:0/2362538046 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613401a070 con 0x7f612c0051c0 2024-04-14T08:35:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.036+0000 7f6149915640 1 -- 172.21.15.172:0/2362538046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614409f1f0 con 0x7f612c0051c0 2024-04-14T08:35:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.036+0000 7f612bfff640 1 -- 172.21.15.172:0/2362538046 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134015a40 con 0x7f612c0051c0 2024-04-14T08:35:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.036+0000 7f6149915640 1 -- 172.21.15.172:0/2362538046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61441e4020 con 0x7f612c0051c0 2024-04-14T08:35:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.037+0000 7f6149915640 1 -- 172.21.15.172:0/2362538046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6144075490 con 0x7f612c0051c0 2024-04-14T08:35:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.037+0000 7f612bfff640 1 -- 172.21.15.172:0/2362538046 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6134015be0 con 0x7f612c0051c0 2024-04-14T08:35:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.038+0000 7f612bfff640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f613006c8f0 0x7f613006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.038+0000 7f612bfff640 1 -- 172.21.15.172:0/2362538046 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7f613408c980 con 0x7f612c0051c0 2024-04-14T08:35:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.038+0000 7f61427fc640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f613006c8f0 0x7f613006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.038+0000 7f61427fc640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f613006c8f0 0x7f613006edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f614409eb60 tx=0x7f6138009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.042+0000 7f612bfff640 1 -- 172.21.15.172:0/2362538046 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6134024020 con 0x7f612c0051c0 2024-04-14T08:35:27.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.221+0000 7f6149915640 1 -- 172.21.15.172:0/2362538046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f61440d0aa0 con 0x7f612c0051c0 2024-04-14T08:35:27.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.224+0000 7f612bfff640 1 -- 172.21.15.172:0/2362538046 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f6134010050 con 0x7f612c0051c0 2024-04-14T08:35:27.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.226+0000 7f6149915640 1 -- 172.21.15.172:0/2362538046 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f613006c8f0 msgr2=0x7f613006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:27.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.226+0000 7f6149915640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f613006c8f0 0x7f613006edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f614409eb60 tx=0x7f6138009040 comp rx=0 tx=0).stop 2024-04-14T08:35:27.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.226+0000 7f6149915640 1 -- 172.21.15.172:0/2362538046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612c0051c0 msgr2=0x7f61441e3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:27.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.226+0000 7f6149915640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612c0051c0 0x7f61441e3ae0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6134004670 tx=0x7f6134007700 comp rx=0 tx=0).stop 2024-04-14T08:35:27.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.226+0000 7f6149915640 1 -- 172.21.15.172:0/2362538046 shutdown_connections 2024-04-14T08:35:27.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.226+0000 7f6149915640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f613006c8f0 0x7f613006edb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.226+0000 7f6149915640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61440cde50 0x7f61440b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.226+0000 7f6149915640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61440c91a0 0x7f61440cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.226+0000 7f6149915640 1 --2- 172.21.15.172:0/2362538046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612c0051c0 0x7f61441e3ae0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.226+0000 7f6149915640 1 -- 172.21.15.172:0/2362538046 >> 172.21.15.172:0/2362538046 conn(0x7f614409d580 msgr2=0x7f61440b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.226+0000 7f6149915640 1 -- 172.21.15.172:0/2362538046 shutdown_connections 2024-04-14T08:35:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.226+0000 7f6149915640 1 -- 172.21.15.172:0/2362538046 wait complete. 2024-04-14T08:35:27.237 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-14T08:35:27.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.348+0000 7ffbf9515640 1 Processor -- start 2024-04-14T08:35:27.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.348+0000 7ffbf9515640 1 -- start start 2024-04-14T08:35:27.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf9515640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf40b21d0 0x7ffbf415c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf9515640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbf40b3700 con 0x7ffbf40b1df0 2024-04-14T08:35:27.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf9515640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbf40b39e0 con 0x7ffbf415c5a0 2024-04-14T08:35:27.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf9515640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ffbf40b3cc0 con 0x7ffbf40b21d0 2024-04-14T08:35:27.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf2ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ffbf40b1df0 0x7ffbf40b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50896/0 (socket says 172.21.15.172:50896) 2024-04-14T08:35:27.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf2ffd640 1 -- 172.21.15.172:0/1369258055 learned_addr learned my addr 172.21.15.172:0/1369258055 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:27.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2940250649 0 0) 0x7ffbf40b3700 con 0x7ffbf40b1df0 2024-04-14T08:35:27.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffbdc003520 con 0x7ffbf40b1df0 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf27fc640 1 --2- 172.21.15.172:0/1369258055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf40b21d0 0x7ffbf415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2098823507 0 0) 0x7ffbdc003520 con 0x7ffbf40b1df0 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf40b21d0 msgr2=0x7ffbf415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 --2- 172.21.15.172:0/1369258055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf40b21d0 0x7ffbf415c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 >> v1:172.21.15.177:6789/0 conn(0x7ffbf415c5a0 legacy=0x7ffbf40b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf415bb40 con 0x7ffbf40b1df0 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7ffbec001670 con 0x7ffbf40b1df0 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 >> v1:172.21.15.172:6789/0 conn(0x7ffbf40b1df0 legacy=0x7ffbf40b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 --2- 172.21.15.172:0/1369258055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbdc0039e0 0x7ffbdc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 --2- 172.21.15.172:0/1369258055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbdc004800 0x7ffbdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 --2- 172.21.15.172:0/1369258055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf40b21d0 0x7ffbdc0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf40b3cc0 con 0x7ffbdc004800 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.349+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc00bb00 con 0x7ffbdc0039e0 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.350+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ffbdc00df10 con 0x7ffbf40b21d0 2024-04-14T08:35:27.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.350+0000 7ffbf27fc640 1 --2- 172.21.15.172:0/1369258055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbdc004800 0x7ffbdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.350+0000 7ffbf2ffd640 1 --2- 172.21.15.172:0/1369258055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbdc0039e0 0x7ffbdc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.350+0000 7ffbf37fe640 1 --2- 172.21.15.172:0/1369258055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf40b21d0 0x7ffbdc0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.350+0000 7ffbf27fc640 1 -- 172.21.15.172:0/1369258055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf40b21d0 msgr2=0x7ffbdc0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.350+0000 7ffbf27fc640 1 --2- 172.21.15.172:0/1369258055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf40b21d0 0x7ffbdc0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.350+0000 7ffbf27fc640 1 -- 172.21.15.172:0/1369258055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbdc0039e0 msgr2=0x7ffbdc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.350+0000 7ffbf27fc640 1 --2- 172.21.15.172:0/1369258055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbdc0039e0 0x7ffbdc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.350+0000 7ffbf27fc640 1 -- 172.21.15.172:0/1369258055 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc003520 con 0x7ffbdc004800 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.351+0000 7ffbf27fc640 1 --2- 172.21.15.172:0/1369258055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbdc004800 0x7ffbdc004c80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8007330 tx=0x7ffbdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973cb1eec69b22c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.351+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe8039070 con 0x7ffbdc004800 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.351+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbe802cca0 con 0x7ffbdc004800 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.351+0000 7ffbf1ffb640 1 -- 172.21.15.172:0/1369258055 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe8009be0 con 0x7ffbdc004800 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.351+0000 7ffbf9515640 1 -- 172.21.15.172:0/1369258055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbdc004800 msgr2=0x7ffbdc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:27.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.351+0000 7ffbf9515640 1 --2- 172.21.15.172:0/1369258055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbdc004800 0x7ffbdc004c80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8007330 tx=0x7ffbdc005830 comp rx=0 tx=0).stop 2024-04-14T08:35:27.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.351+0000 7ffbf37fe640 1 -- 172.21.15.172:0/1369258055 reap_dead start 2024-04-14T08:35:27.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.351+0000 7ffbf9515640 1 -- 172.21.15.172:0/1369258055 shutdown_connections 2024-04-14T08:35:27.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.351+0000 7ffbf9515640 1 -- 172.21.15.172:0/1369258055 >> 172.21.15.172:0/1369258055 conn(0x7ffbf406ec30 msgr2=0x7ffbf406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:27.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.351+0000 7ffbf9515640 1 -- 172.21.15.172:0/1369258055 shutdown_connections 2024-04-14T08:35:27.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.351+0000 7ffbf9515640 1 -- 172.21.15.172:0/1369258055 wait complete. 2024-04-14T08:35:27.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.353+0000 7ffbf9515640 1 Processor -- start 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.353+0000 7ffbf9515640 1 -- start start 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.353+0000 7ffbf9515640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbdc004800 0x7ffbf4157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.353+0000 7ffbf9515640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4141f90 0x7ffbf4140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.353+0000 7ffbf9515640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf4142410 0x7ffbf41408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.353+0000 7ffbf9515640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf40b48d0 con 0x7ffbf4141f90 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.353+0000 7ffbf9515640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf40b4750 con 0x7ffbdc004800 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.353+0000 7ffbf9515640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ffbf4141b00 con 0x7ffbf4142410 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.353+0000 7ffbf27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4141f90 0x7ffbf4140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.353+0000 7ffbf27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4141f90 0x7ffbf4140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36042/0 (socket says 172.21.15.172:36042) 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.353+0000 7ffbf27fc640 1 -- 172.21.15.172:0/2229931810 learned_addr learned my addr 172.21.15.172:0/2229931810 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbf2ffd640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbdc004800 0x7ffbf4157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbf37fe640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf4142410 0x7ffbf41408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbf27fc640 1 -- 172.21.15.172:0/2229931810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf4142410 msgr2=0x7ffbf41408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbf27fc640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf4142410 0x7ffbf41408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbf27fc640 1 -- 172.21.15.172:0/2229931810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbdc004800 msgr2=0x7ffbf4157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbf27fc640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbdc004800 0x7ffbf4157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbf27fc640 1 -- 172.21.15.172:0/2229931810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf413d460 con 0x7ffbf4141f90 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbf37fe640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf4142410 0x7ffbf41408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbf27fc640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4141f90 0x7ffbf4140380 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8037040 tx=0x7ffbe8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbdb7fe640 1 -- 172.21.15.172:0/2229931810 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe8042020 con 0x7ffbf4141f90 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbdb7fe640 1 -- 172.21.15.172:0/2229931810 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbe803a040 con 0x7ffbf4141f90 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbf9515640 1 -- 172.21.15.172:0/2229931810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbf4140ef0 con 0x7ffbf4141f90 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbdb7fe640 1 -- 172.21.15.172:0/2229931810 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe8039070 con 0x7ffbf4141f90 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.354+0000 7ffbf9515640 1 -- 172.21.15.172:0/2229931810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbf41414b0 con 0x7ffbf4141f90 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.355+0000 7ffbf9515640 1 -- 172.21.15.172:0/2229931810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbf4068520 con 0x7ffbf4141f90 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.355+0000 7ffbdb7fe640 1 -- 172.21.15.172:0/2229931810 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffbe8045020 con 0x7ffbf4141f90 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.356+0000 7ffbdb7fe640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffbe006c8f0 0x7ffbe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.356+0000 7ffbdb7fe640 1 -- 172.21.15.172:0/2229931810 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7ffbe80ad900 con 0x7ffbf4141f90 2024-04-14T08:35:27.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.356+0000 7ffbf2ffd640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffbe006c8f0 0x7ffbe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:27.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.356+0000 7ffbf2ffd640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffbe006c8f0 0x7ffbe006edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ffbec001510 tx=0x7ffbec0078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:27.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.360+0000 7ffbdb7fe640 1 -- 172.21.15.172:0/2229931810 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ffbe804c030 con 0x7ffbf4141f90 2024-04-14T08:35:27.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.602+0000 7ffbf9515640 1 -- 172.21.15.172:0/2229931810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7ffbf4159970 con 0x7ffbf4141f90 2024-04-14T08:35:27.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.915+0000 7ffbdb7fe640 1 -- 172.21.15.172:0/2229931810 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v151) ==== 149+0+0 (secure 0 0 0) 0x7ffbe807b8e0 con 0x7ffbf4141f90 2024-04-14T08:35:27.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.917+0000 7ffbf9515640 1 -- 172.21.15.172:0/2229931810 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffbe006c8f0 msgr2=0x7ffbe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:27.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.917+0000 7ffbf9515640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffbe006c8f0 0x7ffbe006edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ffbec001510 tx=0x7ffbec0078f0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.917+0000 7ffbf9515640 1 -- 172.21.15.172:0/2229931810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4141f90 msgr2=0x7ffbf4140380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:27.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.917+0000 7ffbf9515640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4141f90 0x7ffbf4140380 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8037040 tx=0x7ffbe8004ab0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.917+0000 7ffbf9515640 1 -- 172.21.15.172:0/2229931810 shutdown_connections 2024-04-14T08:35:27.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.917+0000 7ffbf9515640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffbe006c8f0 0x7ffbe006edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.917+0000 7ffbf9515640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbf4142410 0x7ffbf41408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.917+0000 7ffbf9515640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4141f90 0x7ffbf4140380 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.917+0000 7ffbf9515640 1 --2- 172.21.15.172:0/2229931810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbdc004800 0x7ffbf4157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:27.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.917+0000 7ffbf9515640 1 -- 172.21.15.172:0/2229931810 >> 172.21.15.172:0/2229931810 conn(0x7ffbf406ec30 msgr2=0x7ffbf4144760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:27.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.917+0000 7ffbf9515640 1 -- 172.21.15.172:0/2229931810 shutdown_connections 2024-04-14T08:35:27.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:27.917+0000 7ffbf9515640 1 -- 172.21.15.172:0/2229931810 wait complete. 2024-04-14T08:35:27.929 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-14T08:35:27.930 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-14T08:35:27.930 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-14T08:35:27.930 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:35:28.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.030+0000 7f1ed6cd1640 1 Processor -- start 2024-04-14T08:35:28.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.030+0000 7f1ed6cd1640 1 -- start start 2024-04-14T08:35:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.031+0000 7f1ed6cd1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ed0160170 0x7f1ed015d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.031+0000 7f1ed6cd1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed015d4d0 0x7f1ed00b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.031+0000 7f1ed6cd1640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed013f910 con 0x7f1ed015d020 2024-04-14T08:35:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.031+0000 7f1ed6cd1640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed013fbf0 con 0x7f1ed0160170 2024-04-14T08:35:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.031+0000 7f1ed6cd1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed013fed0 con 0x7f1ed015d4d0 2024-04-14T08:35:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.031+0000 7f1ed4a6d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ed0160170 0x7f1ed015d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.031+0000 7f1ecffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1ed015d020 0x7f1ed0160550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50912/0 (socket says 172.21.15.172:50912) 2024-04-14T08:35:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.031+0000 7f1ecffff640 1 -- 172.21.15.172:0/1385307887 learned_addr learned my addr 172.21.15.172:0/1385307887 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.031+0000 7f1ed526e640 1 --2- 172.21.15.172:0/1385307887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed015d4d0 0x7f1ed00b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.032+0000 7f1ed4a6d640 1 -- 172.21.15.172:0/1385307887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed015d4d0 msgr2=0x7f1ed00b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.032+0000 7f1ed4a6d640 1 --2- 172.21.15.172:0/1385307887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed015d4d0 0x7f1ed00b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.032+0000 7f1ed4a6d640 1 -- 172.21.15.172:0/1385307887 >> v1:172.21.15.172:6789/0 conn(0x7f1ed015d020 legacy=0x7f1ed0160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.032+0000 7f1ecf7fe640 1 -- 172.21.15.172:0/1385307887 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1877248067 0 0) 0x7f1ed013f910 con 0x7f1ed015d020 2024-04-14T08:35:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.032+0000 7f1ed4a6d640 1 -- 172.21.15.172:0/1385307887 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed01401b0 con 0x7f1ed0160170 2024-04-14T08:35:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.032+0000 7f1ed4a6d640 1 --2- 172.21.15.172:0/1385307887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed0160170 0x7f1ed015d980 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00097a0 tx=0x7f1ec002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84246ca9d6247895 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.032+0000 7f1ecf7fe640 1 -- 172.21.15.172:0/1385307887 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0035020 con 0x7f1ed0160170 2024-04-14T08:35:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.033+0000 7f1ecf7fe640 1 -- 172.21.15.172:0/1385307887 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec0031830 con 0x7f1ed0160170 2024-04-14T08:35:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.033+0000 7f1ecf7fe640 1 -- 172.21.15.172:0/1385307887 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0031c00 con 0x7f1ed0160170 2024-04-14T08:35:28.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.033+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/1385307887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed0160170 msgr2=0x7f1ed015d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.033+0000 7f1ed6cd1640 1 --2- 172.21.15.172:0/1385307887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed0160170 0x7f1ed015d980 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00097a0 tx=0x7f1ec002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.033+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/1385307887 shutdown_connections 2024-04-14T08:35:28.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.033+0000 7f1ed6cd1640 1 --2- 172.21.15.172:0/1385307887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed015d4d0 0x7f1ed00b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.033+0000 7f1ed6cd1640 1 --2- 172.21.15.172:0/1385307887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed0160170 0x7f1ed015d980 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.033+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/1385307887 >> 172.21.15.172:0/1385307887 conn(0x7f1ed006ec30 msgr2=0x7f1ed006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:28.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.033+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/1385307887 shutdown_connections 2024-04-14T08:35:28.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.033+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/1385307887 wait complete. 2024-04-14T08:35:28.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.034+0000 7f1ed6cd1640 1 Processor -- start 2024-04-14T08:35:28.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.034+0000 7f1ed6cd1640 1 -- start start 2024-04-14T08:35:28.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.035+0000 7f1ed6cd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed015d020 0x7f1ed0155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.035+0000 7f1ed6cd1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed015d4d0 0x7f1ed00b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.035+0000 7f1ed6cd1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed0160170 0x7f1ed00b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.035+0000 7f1ed6cd1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed00b20c0 con 0x7f1ed015d020 2024-04-14T08:35:28.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.035+0000 7f1ed6cd1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed0160c60 con 0x7f1ed015d4d0 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.035+0000 7f1ed6cd1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed00b2240 con 0x7f1ed0160170 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.035+0000 7f1ed4a6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed015d020 0x7f1ed0155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.035+0000 7f1ed4a6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed015d020 0x7f1ed0155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36070/0 (socket says 172.21.15.172:36070) 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.035+0000 7f1ed4a6d640 1 -- 172.21.15.172:0/3259727868 learned_addr learned my addr 172.21.15.172:0/3259727868 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ed526e640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed0160170 0x7f1ed00b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ecffff640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed015d4d0 0x7f1ed00b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ed4a6d640 1 -- 172.21.15.172:0/3259727868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed0160170 msgr2=0x7f1ed00b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ed4a6d640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed0160170 0x7f1ed00b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ed4a6d640 1 -- 172.21.15.172:0/3259727868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed015d4d0 msgr2=0x7f1ed00b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ed4a6d640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed015d4d0 0x7f1ed00b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ed4a6d640 1 -- 172.21.15.172:0/3259727868 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed00b2ff0 con 0x7f1ed015d020 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ed4a6d640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed015d020 0x7f1ed0155590 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0000c00 tx=0x7f1ec00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ecd7fa640 1 -- 172.21.15.172:0/3259727868 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0040070 con 0x7f1ed015d020 2024-04-14T08:35:28.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ecd7fa640 1 -- 172.21.15.172:0/3259727868 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec0039a00 con 0x7f1ed015d020 2024-04-14T08:35:28.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/3259727868 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed00b32d0 con 0x7f1ed015d020 2024-04-14T08:35:28.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ecd7fa640 1 -- 172.21.15.172:0/3259727868 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0035040 con 0x7f1ed015d020 2024-04-14T08:35:28.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.036+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/3259727868 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed00b3890 con 0x7f1ed015d020 2024-04-14T08:35:28.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.037+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/3259727868 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed0068520 con 0x7f1ed015d020 2024-04-14T08:35:28.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.037+0000 7f1ecd7fa640 1 -- 172.21.15.172:0/3259727868 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ec004b060 con 0x7f1ed015d020 2024-04-14T08:35:28.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.038+0000 7f1ecd7fa640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ebc06c8f0 0x7f1ebc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.038+0000 7f1ecd7fa640 1 -- 172.21.15.172:0/3259727868 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7f1ec00ad2b0 con 0x7f1ed015d020 2024-04-14T08:35:28.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.038+0000 7f1ecffff640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ebc06c8f0 0x7f1ebc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.038+0000 7f1ecffff640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ebc06c8f0 0x7f1ebc06edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00d0670 tx=0x7f1ec4007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:28.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.042+0000 7f1ecd7fa640 1 -- 172.21.15.172:0/3259727868 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1ec007b850 con 0x7f1ed015d020 2024-04-14T08:35:28.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.218+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/3259727868 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ed00cf170 con 0x7f1ed015d020 2024-04-14T08:35:28.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.219+0000 7f1ecd7fa640 1 -- 172.21.15.172:0/3259727868 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+13355 (secure 0 0 0) 0x7f1ec007b150 con 0x7f1ed015d020 2024-04-14T08:35:28.219 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:28.220 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":53,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:35:26.904835+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/1527574817":"2024-04-15T08:35:23.892531+0000","172.21.15.172:6835/1527574817":"2024-04-15T08:35:23.892531+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:35:28.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.221+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/3259727868 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ebc06c8f0 msgr2=0x7f1ebc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.221+0000 7f1ed6cd1640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ebc06c8f0 0x7f1ebc06edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00d0670 tx=0x7f1ec4007b50 comp rx=0 tx=0).stop 2024-04-14T08:35:28.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.221+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/3259727868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed015d020 msgr2=0x7f1ed0155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.221+0000 7f1ed6cd1640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed015d020 0x7f1ed0155590 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0000c00 tx=0x7f1ec00046c0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.221+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/3259727868 shutdown_connections 2024-04-14T08:35:28.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.221+0000 7f1ed6cd1640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1ebc06c8f0 0x7f1ebc06edb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.221+0000 7f1ed6cd1640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ed0160170 0x7f1ed00b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.221+0000 7f1ed6cd1640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed015d4d0 0x7f1ed00b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.221+0000 7f1ed6cd1640 1 --2- 172.21.15.172:0/3259727868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ed015d020 0x7f1ed0155590 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.221+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/3259727868 >> 172.21.15.172:0/3259727868 conn(0x7f1ed006ec30 msgr2=0x7f1ed00cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:28.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.221+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/3259727868 shutdown_connections 2024-04-14T08:35:28.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.221+0000 7f1ed6cd1640 1 -- 172.21.15.172:0/3259727868 wait complete. 2024-04-14T08:35:28.232 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:35:28.233 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:35:28.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.336+0000 7f881c2de640 1 Processor -- start 2024-04-14T08:35:28.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.336+0000 7f881c2de640 1 -- start start 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.337+0000 7f881c2de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f881415e430 0x7f8814162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.337+0000 7f881c2de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8814162540 0x7f88141629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.337+0000 7f881c2de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f88140b3a30 0x7f88140b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.337+0000 7f881c2de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f88140cefb0 con 0x7f88140b3a30 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.337+0000 7f881c2de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f88140cf290 con 0x7f881415e430 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.337+0000 7f881c2de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f88140cf570 con 0x7f8814162540 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.338+0000 7f881a87b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f88140b3a30 0x7f88140b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.338+0000 7f881a87b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f88140b3a30 0x7f88140b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36080/0 (socket says 172.21.15.172:36080) 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.338+0000 7f881a87b640 1 -- 172.21.15.172:0/3342106619 learned_addr learned my addr 172.21.15.172:0/3342106619 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.338+0000 7f881a07a640 1 --2- 172.21.15.172:0/3342106619 >> v2:172.21.15.177:3300/0 conn(0x7f881415e430 0x7f8814162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.338+0000 7f8819879640 1 --2- 172.21.15.172:0/3342106619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8814162540 0x7f88141629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.338+0000 7f881a87b640 1 -- 172.21.15.172:0/3342106619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8814162540 msgr2=0x7f88141629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:28.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.338+0000 7f881a87b640 1 --2- 172.21.15.172:0/3342106619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8814162540 0x7f88141629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.338+0000 7f881a87b640 1 -- 172.21.15.172:0/3342106619 >> v2:172.21.15.177:3300/0 conn(0x7f881415e430 msgr2=0x7f8814162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:28.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.338+0000 7f881a87b640 1 --2- 172.21.15.172:0/3342106619 >> v2:172.21.15.177:3300/0 conn(0x7f881415e430 0x7f8814162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.338+0000 7f881a87b640 1 -- 172.21.15.172:0/3342106619 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88140cf850 con 0x7f88140b3a30 2024-04-14T08:35:28.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.338+0000 7f8819879640 1 --2- 172.21.15.172:0/3342106619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8814162540 0x7f88141629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:28.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f881a87b640 1 --2- 172.21.15.172:0/3342106619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88140b3a30 0x7f88140b3e60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f88140b6db0 tx=0x7f880c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3871fc1ca4f67495 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:28.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f8819078640 1 -- 172.21.15.172:0/3342106619 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c02ca70 con 0x7f88140b3a30 2024-04-14T08:35:28.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f8819078640 1 -- 172.21.15.172:0/3342106619 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c02cc10 con 0x7f88140b3a30 2024-04-14T08:35:28.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f8819078640 1 -- 172.21.15.172:0/3342106619 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c0343f0 con 0x7f88140b3a30 2024-04-14T08:35:28.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f881c2de640 1 -- 172.21.15.172:0/3342106619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88140b3a30 msgr2=0x7f88140b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f881c2de640 1 --2- 172.21.15.172:0/3342106619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88140b3a30 0x7f88140b3e60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f88140b6db0 tx=0x7f880c02bed0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f881c2de640 1 -- 172.21.15.172:0/3342106619 shutdown_connections 2024-04-14T08:35:28.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f881c2de640 1 --2- 172.21.15.172:0/3342106619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88140b3a30 0x7f88140b3e60 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f881c2de640 1 --2- 172.21.15.172:0/3342106619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8814162540 0x7f88141629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f881c2de640 1 --2- 172.21.15.172:0/3342106619 >> v2:172.21.15.177:3300/0 conn(0x7f881415e430 0x7f8814162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f881c2de640 1 -- 172.21.15.172:0/3342106619 >> 172.21.15.172:0/3342106619 conn(0x7f881409d560 msgr2=0x7f88140d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:28.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f881c2de640 1 -- 172.21.15.172:0/3342106619 shutdown_connections 2024-04-14T08:35:28.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.339+0000 7f881c2de640 1 -- 172.21.15.172:0/3342106619 wait complete. 2024-04-14T08:35:28.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.341+0000 7f881c2de640 1 Processor -- start 2024-04-14T08:35:28.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.341+0000 7f881c2de640 1 -- start start 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.341+0000 7f881c2de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88140b3a30 0x7f88140cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.341+0000 7f881c2de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881415e430 0x7f88140b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.341+0000 7f881c2de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8814162540 0x7f88141e9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.341+0000 7f881c2de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f88140d4e20 con 0x7f881415e430 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.341+0000 7f881c2de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f881415e810 con 0x7f88140b3a30 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.341+0000 7f881c2de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f88140d0050 con 0x7f8814162540 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.341+0000 7f8819879640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881415e430 0x7f88140b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.341+0000 7f8819879640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881415e430 0x7f88140b97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36090/0 (socket says 172.21.15.172:36090) 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.341+0000 7f8819879640 1 -- 172.21.15.172:0/2888261915 learned_addr learned my addr 172.21.15.172:0/2888261915 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.342+0000 7f8819879640 1 -- 172.21.15.172:0/2888261915 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8814162540 msgr2=0x7f88141e9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.342+0000 7f8819879640 1 --2- 172.21.15.172:0/2888261915 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8814162540 0x7f88141e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.342+0000 7f8819879640 1 -- 172.21.15.172:0/2888261915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88140b3a30 msgr2=0x7f88140cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.342+0000 7f8819879640 1 --2- 172.21.15.172:0/2888261915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88140b3a30 0x7f88140cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.342+0000 7f8819879640 1 -- 172.21.15.172:0/2888261915 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88141f8bf0 con 0x7f881415e430 2024-04-14T08:35:28.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.342+0000 7f8819879640 1 --2- 172.21.15.172:0/2888261915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881415e430 0x7f88140b97f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f880400e810 tx=0x7f880400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:28.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.342+0000 7f880affd640 1 -- 172.21.15.172:0/2888261915 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880400f040 con 0x7f881415e430 2024-04-14T08:35:28.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.342+0000 7f880affd640 1 -- 172.21.15.172:0/2888261915 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8804004030 con 0x7f881415e430 2024-04-14T08:35:28.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.342+0000 7f881c2de640 1 -- 172.21.15.172:0/2888261915 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88141f7e50 con 0x7f881415e430 2024-04-14T08:35:28.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.342+0000 7f880affd640 1 -- 172.21.15.172:0/2888261915 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804004400 con 0x7f881415e430 2024-04-14T08:35:28.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.342+0000 7f881c2de640 1 -- 172.21.15.172:0/2888261915 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88141f83f0 con 0x7f881415e430 2024-04-14T08:35:28.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.343+0000 7f881c2de640 1 -- 172.21.15.172:0/2888261915 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8814068590 con 0x7f881415e430 2024-04-14T08:35:28.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.343+0000 7f880affd640 1 -- 172.21.15.172:0/2888261915 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f880400c6d0 con 0x7f881415e430 2024-04-14T08:35:28.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.344+0000 7f880affd640 1 --2- 172.21.15.172:0/2888261915 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f87f806c920 0x7f87f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.344+0000 7f880affd640 1 -- 172.21.15.172:0/2888261915 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7f880408b710 con 0x7f881415e430 2024-04-14T08:35:28.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.344+0000 7f881a07a640 1 --2- 172.21.15.172:0/2888261915 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f87f806c920 0x7f87f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.344+0000 7f881a07a640 1 --2- 172.21.15.172:0/2888261915 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f87f806c920 0x7f87f806ede0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f88100099a0 tx=0x7f8810009460 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:28.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.348+0000 7f880affd640 1 -- 172.21.15.172:0/2888261915 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8804017020 con 0x7f881415e430 2024-04-14T08:35:28.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.521+0000 7f881c2de640 1 -- 172.21.15.172:0/2888261915 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f88140d4bc0 con 0x7f881415e430 2024-04-14T08:35:28.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.521+0000 7f880affd640 1 -- 172.21.15.172:0/2888261915 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7f8804059d40 con 0x7f881415e430 2024-04-14T08:35:28.522 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:35:28.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.523+0000 7f881c2de640 1 -- 172.21.15.172:0/2888261915 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f87f806c920 msgr2=0x7f87f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.523+0000 7f881c2de640 1 --2- 172.21.15.172:0/2888261915 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f87f806c920 0x7f87f806ede0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f88100099a0 tx=0x7f8810009460 comp rx=0 tx=0).stop 2024-04-14T08:35:28.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.523+0000 7f881c2de640 1 -- 172.21.15.172:0/2888261915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881415e430 msgr2=0x7f88140b97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.523+0000 7f881c2de640 1 --2- 172.21.15.172:0/2888261915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881415e430 0x7f88140b97f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f880400e810 tx=0x7f880400ecd0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.523+0000 7f881c2de640 1 -- 172.21.15.172:0/2888261915 shutdown_connections 2024-04-14T08:35:28.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.523+0000 7f881c2de640 1 --2- 172.21.15.172:0/2888261915 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f87f806c920 0x7f87f806ede0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.523+0000 7f881c2de640 1 --2- 172.21.15.172:0/2888261915 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8814162540 0x7f88141e9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.523+0000 7f881c2de640 1 --2- 172.21.15.172:0/2888261915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881415e430 0x7f88140b97f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.523+0000 7f881c2de640 1 --2- 172.21.15.172:0/2888261915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88140b3a30 0x7f88140cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.523+0000 7f881c2de640 1 -- 172.21.15.172:0/2888261915 >> 172.21.15.172:0/2888261915 conn(0x7f881409d560 msgr2=0x7f881406d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:28.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.523+0000 7f881c2de640 1 -- 172.21.15.172:0/2888261915 shutdown_connections 2024-04-14T08:35:28.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.523+0000 7f881c2de640 1 -- 172.21.15.172:0/2888261915 wait complete. 2024-04-14T08:35:28.535 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-14T08:35:28.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.635+0000 7fb1708de640 1 Processor -- start 2024-04-14T08:35:28.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.635+0000 7fb1708de640 1 -- start start 2024-04-14T08:35:28.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.636+0000 7fb1708de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb16c0b21d0 0x7fb16c15c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.636+0000 7fb1708de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb16c15c5a0 0x7fb16c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.636+0000 7fb1708de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb16c0b3700 con 0x7fb16c0b1df0 2024-04-14T08:35:28.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.636+0000 7fb1708de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb16c0b39e0 con 0x7fb16c0b21d0 2024-04-14T08:35:28.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.636+0000 7fb1708de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb16c0b3cc0 con 0x7fb16c15c5a0 2024-04-14T08:35:28.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.636+0000 7fb16a59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb16c0b1df0 0x7fb16c0b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50918/0 (socket says 172.21.15.172:50918) 2024-04-14T08:35:28.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.636+0000 7fb16a59c640 1 -- 172.21.15.172:0/3671023292 learned_addr learned my addr 172.21.15.172:0/3671023292 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.636+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1031789139 0 0) 0x7fb16c0b3700 con 0x7fb16c0b1df0 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.636+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb154003520 con 0x7fb16c0b1df0 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.636+0000 7fb169d9b640 1 --2- 172.21.15.172:0/3671023292 >> v2:172.21.15.177:3300/0 conn(0x7fb16c0b21d0 0x7fb16c15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.636+0000 7fb16ad9d640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb16c15c5a0 0x7fb16c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 446969631 0 0) 0x7fb154003520 con 0x7fb16c0b1df0 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb16c15c5a0 msgr2=0x7fb16c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb16c15c5a0 0x7fb16c0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 >> v2:172.21.15.177:3300/0 conn(0x7fb16c0b21d0 msgr2=0x7fb16c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 --2- 172.21.15.172:0/3671023292 >> v2:172.21.15.177:3300/0 conn(0x7fb16c0b21d0 0x7fb16c15c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c15bb40 con 0x7fb16c0b1df0 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fb15c0019f0 con 0x7fb16c0b1df0 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 >> v1:172.21.15.172:6789/0 conn(0x7fb16c0b1df0 legacy=0x7fb16c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1540039b0 0x7fb154003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb154004800 0x7fb154004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1540051c0 0x7fb15400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c0b39e0 con 0x7fb154004800 2024-04-14T08:35:28.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c0b3cc0 con 0x7fb1540039b0 2024-04-14T08:35:28.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb1540045a0 con 0x7fb1540051c0 2024-04-14T08:35:28.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb169d9b640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb154004800 0x7fb154004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16a59c640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1540039b0 0x7fb154003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16ad9d640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1540051c0 0x7fb15400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb169d9b640 1 -- 172.21.15.172:0/3671023292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1540051c0 msgr2=0x7fb15400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:28.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb169d9b640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1540051c0 0x7fb15400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb169d9b640 1 -- 172.21.15.172:0/3671023292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1540039b0 msgr2=0x7fb154003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb169d9b640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1540039b0 0x7fb154003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb169d9b640 1 -- 172.21.15.172:0/3671023292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb154003520 con 0x7fb154004800 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.637+0000 7fb16ad9d640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1540051c0 0x7fb15400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.638+0000 7fb169d9b640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb154004800 0x7fb154004c80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb160009650 tx=0x7fb154005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a26e92e1893a66d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.638+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16003b070 con 0x7fb154004800 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.638+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16002e8c0 con 0x7fb154004800 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.638+0000 7fb16959a640 1 -- 172.21.15.172:0/3671023292 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160009690 con 0x7fb154004800 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.639+0000 7fb1708de640 1 -- 172.21.15.172:0/3671023292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb154004800 msgr2=0x7fb154004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.639+0000 7fb1708de640 1 --2- 172.21.15.172:0/3671023292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb154004800 0x7fb154004c80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb160009650 tx=0x7fb154005830 comp rx=0 tx=0).stop 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.639+0000 7fb16ad9d640 1 -- 172.21.15.172:0/3671023292 reap_dead start 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.639+0000 7fb1708de640 1 -- 172.21.15.172:0/3671023292 shutdown_connections 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.639+0000 7fb1708de640 1 -- 172.21.15.172:0/3671023292 >> 172.21.15.172:0/3671023292 conn(0x7fb16c06ec30 msgr2=0x7fb16c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:28.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.639+0000 7fb1708de640 1 -- 172.21.15.172:0/3671023292 shutdown_connections 2024-04-14T08:35:28.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.639+0000 7fb1708de640 1 -- 172.21.15.172:0/3671023292 wait complete. 2024-04-14T08:35:28.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.640+0000 7fb1708de640 1 Processor -- start 2024-04-14T08:35:28.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.640+0000 7fb1708de640 1 -- start start 2024-04-14T08:35:28.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb1708de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb154004800 0x7fb16c1387c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb1708de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb16c13b420 0x7fb16c13b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb1708de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb16c0cd990 0x7fb16c0cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb1708de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c0b41c0 con 0x7fb16c0cd990 2024-04-14T08:35:28.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb1708de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c0b4040 con 0x7fb154004800 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb1708de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb16c0b4340 con 0x7fb16c13b420 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb16ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb16c0cd990 0x7fb16c0cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb16ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb16c0cd990 0x7fb16c0cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36104/0 (socket says 172.21.15.172:36104) 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb16ad9d640 1 -- 172.21.15.172:0/2727020086 learned_addr learned my addr 172.21.15.172:0/2727020086 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb169d9b640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb16c13b420 0x7fb16c13b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb16a59c640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb154004800 0x7fb16c1387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb16ad9d640 1 -- 172.21.15.172:0/2727020086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb16c13b420 msgr2=0x7fb16c13b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb16ad9d640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb16c13b420 0x7fb16c13b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb16ad9d640 1 -- 172.21.15.172:0/2727020086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb154004800 msgr2=0x7fb16c1387c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb16ad9d640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb154004800 0x7fb16c1387c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb16ad9d640 1 -- 172.21.15.172:0/2727020086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c1553f0 con 0x7fb16c0cd990 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb16a59c640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb154004800 0x7fb16c1387c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:28.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb16ad9d640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb16c0cd990 0x7fb16c0cde20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0b25b0 tx=0x7fb164011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:28.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.641+0000 7fb15affd640 1 -- 172.21.15.172:0/2727020086 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164019020 con 0x7fb16c0cd990 2024-04-14T08:35:28.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.642+0000 7fb15affd640 1 -- 172.21.15.172:0/2727020086 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb164004460 con 0x7fb16c0cd990 2024-04-14T08:35:28.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.642+0000 7fb1708de640 1 -- 172.21.15.172:0/2727020086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16c0ce360 con 0x7fb16c0cd990 2024-04-14T08:35:28.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.642+0000 7fb15affd640 1 -- 172.21.15.172:0/2727020086 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164003850 con 0x7fb16c0cd990 2024-04-14T08:35:28.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.642+0000 7fb1708de640 1 -- 172.21.15.172:0/2727020086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16c155970 con 0x7fb16c0cd990 2024-04-14T08:35:28.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.643+0000 7fb1708de640 1 -- 172.21.15.172:0/2727020086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb16c068520 con 0x7fb16c0cd990 2024-04-14T08:35:28.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.643+0000 7fb15affd640 1 -- 172.21.15.172:0/2727020086 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb16401c020 con 0x7fb16c0cd990 2024-04-14T08:35:28.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.643+0000 7fb15affd640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb14c06c920 0x7fb14c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:28.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.643+0000 7fb15affd640 1 -- 172.21.15.172:0/2727020086 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7fb16408b980 con 0x7fb16c0cd990 2024-04-14T08:35:28.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.643+0000 7fb16a59c640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb14c06c920 0x7fb14c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:28.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.644+0000 7fb16a59c640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb14c06c920 0x7fb14c06ede0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb15c001350 tx=0x7fb15c006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:28.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.648+0000 7fb15affd640 1 -- 172.21.15.172:0/2727020086 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb164023030 con 0x7fb16c0cd990 2024-04-14T08:35:28.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.891+0000 7fb1708de640 1 -- 172.21.15.172:0/2727020086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7fb16c0afe30 con 0x7fb16c0cd990 2024-04-14T08:35:28.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.916+0000 7fb15affd640 1 -- 172.21.15.172:0/2727020086 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v152) ==== 172+0+0 (secure 0 0 0) 0x7fb164059fb0 con 0x7fb16c0cd990 2024-04-14T08:35:28.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.918+0000 7fb1708de640 1 -- 172.21.15.172:0/2727020086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb14c06c920 msgr2=0x7fb14c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.918+0000 7fb1708de640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb14c06c920 0x7fb14c06ede0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb15c001350 tx=0x7fb15c006040 comp rx=0 tx=0).stop 2024-04-14T08:35:28.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.918+0000 7fb1708de640 1 -- 172.21.15.172:0/2727020086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb16c0cd990 msgr2=0x7fb16c0cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:28.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.918+0000 7fb1708de640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb16c0cd990 0x7fb16c0cde20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0b25b0 tx=0x7fb164011920 comp rx=0 tx=0).stop 2024-04-14T08:35:28.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.918+0000 7fb1708de640 1 -- 172.21.15.172:0/2727020086 shutdown_connections 2024-04-14T08:35:28.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.918+0000 7fb1708de640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb14c06c920 0x7fb14c06ede0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.918+0000 7fb1708de640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb16c0cd990 0x7fb16c0cde20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.918+0000 7fb1708de640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb16c13b420 0x7fb16c13b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.918+0000 7fb1708de640 1 --2- 172.21.15.172:0/2727020086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb154004800 0x7fb16c1387c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:28.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.918+0000 7fb1708de640 1 -- 172.21.15.172:0/2727020086 >> 172.21.15.172:0/2727020086 conn(0x7fb16c06ec30 msgr2=0x7fb16c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:28.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.918+0000 7fb1708de640 1 -- 172.21.15.172:0/2727020086 shutdown_connections 2024-04-14T08:35:28.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:28.918+0000 7fb1708de640 1 -- 172.21.15.172:0/2727020086 wait complete. 2024-04-14T08:35:28.930 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-14T08:35:29.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.033+0000 7f2c9cb7b640 1 Processor -- start 2024-04-14T08:35:29.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.033+0000 7f2c9cb7b640 1 -- start start 2024-04-14T08:35:29.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.034+0000 7f2c9cb7b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2c9806d750 0x7f2c980d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.034+0000 7f2c9cb7b640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c980cc780 con 0x7f2c9815e3c0 2024-04-14T08:35:29.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.034+0000 7f2c9cb7b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c980cca60 con 0x7f2c980ce550 2024-04-14T08:35:29.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.034+0000 7f2c9cb7b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2c9806ce40 con 0x7f2c9806d750 2024-04-14T08:35:29.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.034+0000 7f2c9659c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2c9815e3c0 0x7f2c9806c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50930/0 (socket says 172.21.15.172:50930) 2024-04-14T08:35:29.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.034+0000 7f2c9659c640 1 -- 172.21.15.172:0/4232495037 learned_addr learned my addr 172.21.15.172:0/4232495037 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:29.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.034+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2096059208 0 0) 0x7f2c980cc780 con 0x7f2c9815e3c0 2024-04-14T08:35:29.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.034+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c80003520 con 0x7f2c9815e3c0 2024-04-14T08:35:29.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.034+0000 7f2c96d9d640 1 --2- 172.21.15.172:0/4232495037 >> v2:172.21.15.177:3300/0 conn(0x7f2c9806d750 0x7f2c980d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2085646758 0 0) 0x7f2c80003520 con 0x7f2c9815e3c0 2024-04-14T08:35:29.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 >> v2:172.21.15.177:3300/0 conn(0x7f2c9806d750 msgr2=0x7f2c980d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:29.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 --2- 172.21.15.172:0/4232495037 >> v2:172.21.15.177:3300/0 conn(0x7f2c9806d750 0x7f2c980d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 >> v1:172.21.15.177:6789/0 conn(0x7f2c980ce550 legacy=0x7f2c9806d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c980ccd40 con 0x7f2c9815e3c0 2024-04-14T08:35:29.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f2c88001670 con 0x7f2c9815e3c0 2024-04-14T08:35:29.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 >> v1:172.21.15.172:6789/0 conn(0x7f2c9815e3c0 legacy=0x7f2c9806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 --2- 172.21.15.172:0/4232495037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c800039b0 0x7f2c80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 --2- 172.21.15.172:0/4232495037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 0x7f2c80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 --2- 172.21.15.172:0/4232495037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c800051c0 0x7f2c8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9806ce40 con 0x7f2c80004800 2024-04-14T08:35:29.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8000c060 con 0x7f2c800039b0 2024-04-14T08:35:29.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8000c260 con 0x7f2c800051c0 2024-04-14T08:35:29.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c95d9b640 1 --2- 172.21.15.172:0/4232495037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 0x7f2c80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c9659c640 1 --2- 172.21.15.172:0/4232495037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c800039b0 0x7f2c80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c96d9d640 1 --2- 172.21.15.172:0/4232495037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c800051c0 0x7f2c8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c95d9b640 1 -- 172.21.15.172:0/4232495037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c800051c0 msgr2=0x7f2c8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:29.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c95d9b640 1 --2- 172.21.15.172:0/4232495037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c800051c0 0x7f2c8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c95d9b640 1 -- 172.21.15.172:0/4232495037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c800039b0 msgr2=0x7f2c80003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:29.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c95d9b640 1 --2- 172.21.15.172:0/4232495037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c800039b0 0x7f2c80003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c95d9b640 1 -- 172.21.15.172:0/4232495037 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c80003520 con 0x7f2c80004800 2024-04-14T08:35:29.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.035+0000 7f2c96d9d640 1 -- 172.21.15.172:0/4232495037 reap_dead start 2024-04-14T08:35:29.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.036+0000 7f2c95d9b640 1 --2- 172.21.15.172:0/4232495037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 0x7f2c80004c80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c001750 tx=0x7f2c80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabfc84e7ef2ce87 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:29.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.036+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c001370 con 0x7f2c80004800 2024-04-14T08:35:29.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.036+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c8c0052d0 con 0x7f2c80004800 2024-04-14T08:35:29.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.036+0000 7f2c9559a640 1 -- 172.21.15.172:0/4232495037 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c003830 con 0x7f2c80004800 2024-04-14T08:35:29.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.037+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/4232495037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 msgr2=0x7f2c80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.037+0000 7f2c9cb7b640 1 --2- 172.21.15.172:0/4232495037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 0x7f2c80004c80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c001750 tx=0x7f2c80005830 comp rx=0 tx=0).stop 2024-04-14T08:35:29.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.037+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/4232495037 shutdown_connections 2024-04-14T08:35:29.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.037+0000 7f2c9cb7b640 1 --2- 172.21.15.172:0/4232495037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 0x7f2c80004c80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.037+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/4232495037 >> 172.21.15.172:0/4232495037 conn(0x7f2c980b3af0 msgr2=0x7f2c980b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:29.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.037+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/4232495037 shutdown_connections 2024-04-14T08:35:29.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.037+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/4232495037 wait complete. 2024-04-14T08:35:29.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.038+0000 7f2c9cb7b640 1 Processor -- start 2024-04-14T08:35:29.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.038+0000 7f2c9cb7b640 1 -- start start 2024-04-14T08:35:29.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9cb7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 0x7f2c98160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9cb7b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c98158dd0 0x7f2c98159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9cb7b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c981492c0 0x7f2c98149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9cb7b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9806d040 con 0x7f2c80004800 2024-04-14T08:35:29.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9cb7b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9815e7a0 con 0x7f2c98158dd0 2024-04-14T08:35:29.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9cb7b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9806d1c0 con 0x7f2c981492c0 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 0x7f2c98160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 0x7f2c98160460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36126/0 (socket says 172.21.15.172:36126) 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9659c640 1 -- 172.21.15.172:0/3489457437 learned_addr learned my addr 172.21.15.172:0/3489457437 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9659c640 1 -- 172.21.15.172:0/3489457437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c981492c0 msgr2=0x7f2c98149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9659c640 1 --2- 172.21.15.172:0/3489457437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c981492c0 0x7f2c98149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9659c640 1 -- 172.21.15.172:0/3489457437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c98158dd0 msgr2=0x7f2c98159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9659c640 1 --2- 172.21.15.172:0/3489457437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c98158dd0 0x7f2c98159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9659c640 1 -- 172.21.15.172:0/3489457437 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9815f3c0 con 0x7f2c80004800 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c9659c640 1 --2- 172.21.15.172:0/3489457437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 0x7f2c98160460 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2c88004ef0 tx=0x7f2c88007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c86ffd640 1 -- 172.21.15.172:0/3489457437 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8800f730 con 0x7f2c80004800 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.039+0000 7f2c86ffd640 1 -- 172.21.15.172:0/3489457437 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c88018070 con 0x7f2c80004800 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.040+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/3489457437 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9815f6a0 con 0x7f2c80004800 2024-04-14T08:35:29.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.040+0000 7f2c86ffd640 1 -- 172.21.15.172:0/3489457437 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8800faf0 con 0x7f2c80004800 2024-04-14T08:35:29.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.040+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/3489457437 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9815fc60 con 0x7f2c80004800 2024-04-14T08:35:29.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.040+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/3489457437 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c98068590 con 0x7f2c80004800 2024-04-14T08:35:29.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.041+0000 7f2c86ffd640 1 -- 172.21.15.172:0/3489457437 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c8800fc90 con 0x7f2c80004800 2024-04-14T08:35:29.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.041+0000 7f2c86ffd640 1 --2- 172.21.15.172:0/3489457437 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c7806c8f0 0x7f2c7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.041+0000 7f2c86ffd640 1 -- 172.21.15.172:0/3489457437 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7f2c8808b880 con 0x7f2c80004800 2024-04-14T08:35:29.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.041+0000 7f2c95d9b640 1 --2- 172.21.15.172:0/3489457437 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c7806c8f0 0x7f2c7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.042+0000 7f2c95d9b640 1 --2- 172.21.15.172:0/3489457437 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c7806c8f0 0x7f2c7806edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c033000 tx=0x7f2c8c02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:29.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.046+0000 7f2c86ffd640 1 -- 172.21.15.172:0/3489457437 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2c88059eb0 con 0x7f2c80004800 2024-04-14T08:35:29.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.220+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/3489457437 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2c981615d0 con 0x7f2c80004800 2024-04-14T08:35:29.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.225+0000 7f2c86ffd640 1 -- 172.21.15.172:0/3489457437 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f2c880597b0 con 0x7f2c80004800 2024-04-14T08:35:29.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.227+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/3489457437 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c7806c8f0 msgr2=0x7f2c7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.227+0000 7f2c9cb7b640 1 --2- 172.21.15.172:0/3489457437 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c7806c8f0 0x7f2c7806edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c033000 tx=0x7f2c8c02f000 comp rx=0 tx=0).stop 2024-04-14T08:35:29.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.227+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/3489457437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 msgr2=0x7f2c98160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.227+0000 7f2c9cb7b640 1 --2- 172.21.15.172:0/3489457437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 0x7f2c98160460 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2c88004ef0 tx=0x7f2c88007430 comp rx=0 tx=0).stop 2024-04-14T08:35:29.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.227+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/3489457437 shutdown_connections 2024-04-14T08:35:29.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.228+0000 7f2c9cb7b640 1 --2- 172.21.15.172:0/3489457437 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c7806c8f0 0x7f2c7806edb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.228+0000 7f2c9cb7b640 1 --2- 172.21.15.172:0/3489457437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c981492c0 0x7f2c98149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.228+0000 7f2c9cb7b640 1 --2- 172.21.15.172:0/3489457437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c98158dd0 0x7f2c98159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.228+0000 7f2c9cb7b640 1 --2- 172.21.15.172:0/3489457437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c80004800 0x7f2c98160460 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.228+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/3489457437 >> 172.21.15.172:0/3489457437 conn(0x7f2c980b3af0 msgr2=0x7f2c98074a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:29.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.228+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/3489457437 shutdown_connections 2024-04-14T08:35:29.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.228+0000 7f2c9cb7b640 1 -- 172.21.15.172:0/3489457437 wait complete. 2024-04-14T08:35:29.239 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:35:29.239 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:35:29.288 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:35:29.337 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:35:29.338 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:35:29.338 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:35:29.353 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:35:29.353 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:35:29.353 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:35:29.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:35:29.370 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:35:29.370 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:35:29.408 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:35:29.408 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:35:29.408 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:35:29.426 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:35:29.426 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-14T08:35:29.426 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:35:29.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.548+0000 7f1d09a91640 1 Processor -- start 2024-04-14T08:35:29.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.548+0000 7f1d09a91640 1 -- start start 2024-04-14T08:35:29.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.549+0000 7f1d09a91640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d040b1df0 0x7f1d040b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.549+0000 7f1d09a91640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1d040b4040 0x7f1d040b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.549+0000 7f1d09a91640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d040b49f0 con 0x7f1d040b21d0 2024-04-14T08:35:29.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.549+0000 7f1d09a91640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f1d0415c770 con 0x7f1d040b4040 2024-04-14T08:35:29.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.549+0000 7f1d09a91640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1d040b3430 con 0x7f1d040b1df0 2024-04-14T08:35:29.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.549+0000 7f1d037fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d040b1df0 0x7f1d040b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.549+0000 7f1d03fff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1d040b4040 0x7f1d040b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.549+0000 7f1d02ffd640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f1d040b21d0 0x7f1d0415c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:44054/0 (socket says 172.21.15.172:44054) 2024-04-14T08:35:29.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.549+0000 7f1d02ffd640 1 -- 172.21.15.172:0/3358125394 learned_addr learned my addr 172.21.15.172:0/3358125394 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:29.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.550+0000 7f1d037fe640 1 -- 172.21.15.172:0/3358125394 >> v2:172.21.15.177:3300/0 conn(0x7f1d040b4040 msgr2=0x7f1d040b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:29.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.550+0000 7f1d037fe640 1 --2- 172.21.15.172:0/3358125394 >> v2:172.21.15.177:3300/0 conn(0x7f1d040b4040 0x7f1d040b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.550+0000 7f1d037fe640 1 -- 172.21.15.172:0/3358125394 >> v1:172.21.15.177:6789/0 conn(0x7f1d040b21d0 legacy=0x7f1d0415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.550+0000 7f1d037fe640 1 -- 172.21.15.172:0/3358125394 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d040b3700 con 0x7f1d040b1df0 2024-04-14T08:35:29.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.550+0000 7f1d03fff640 1 --2- 172.21.15.172:0/3358125394 >> v2:172.21.15.177:3300/0 conn(0x7f1d040b4040 0x7f1d040b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:35:29.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.550+0000 7f1d037fe640 1 --2- 172.21.15.172:0/3358125394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d040b1df0 0x7f1d040b2ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40097a0 tx=0x7f1cf402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d79b8416e7a7ff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:29.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.551+0000 7f1d027fc640 1 -- 172.21.15.172:0/3358125394 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4035020 con 0x7f1d040b1df0 2024-04-14T08:35:29.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.551+0000 7f1d027fc640 1 -- 172.21.15.172:0/3358125394 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf4031830 con 0x7f1d040b1df0 2024-04-14T08:35:29.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.551+0000 7f1d027fc640 1 -- 172.21.15.172:0/3358125394 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4031c00 con 0x7f1d040b1df0 2024-04-14T08:35:29.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.551+0000 7f1d09a91640 1 -- 172.21.15.172:0/3358125394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d040b1df0 msgr2=0x7f1d040b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.551+0000 7f1d09a91640 1 --2- 172.21.15.172:0/3358125394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d040b1df0 0x7f1d040b2ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40097a0 tx=0x7f1cf402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.551+0000 7f1d09a91640 1 -- 172.21.15.172:0/3358125394 shutdown_connections 2024-04-14T08:35:29.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.551+0000 7f1d09a91640 1 --2- 172.21.15.172:0/3358125394 >> v2:172.21.15.177:3300/0 conn(0x7f1d040b4040 0x7f1d040b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.551+0000 7f1d09a91640 1 --2- 172.21.15.172:0/3358125394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d040b1df0 0x7f1d040b2ef0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.551+0000 7f1d09a91640 1 -- 172.21.15.172:0/3358125394 >> 172.21.15.172:0/3358125394 conn(0x7f1d0406ec30 msgr2=0x7f1d0406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:29.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.551+0000 7f1d09a91640 1 -- 172.21.15.172:0/3358125394 shutdown_connections 2024-04-14T08:35:29.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.551+0000 7f1d09a91640 1 -- 172.21.15.172:0/3358125394 wait complete. 2024-04-14T08:35:29.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.552+0000 7f1d09a91640 1 Processor -- start 2024-04-14T08:35:29.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.553+0000 7f1d09a91640 1 -- start start 2024-04-14T08:35:29.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.553+0000 7f1d09a91640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d040b1df0 0x7f1d04155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.553+0000 7f1d09a91640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d040b21d0 0x7f1d0413dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.553+0000 7f1d09a91640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d040b4040 0x7f1d04156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.553+0000 7f1d09a91640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0415bb40 con 0x7f1d040b4040 2024-04-14T08:35:29.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.553+0000 7f1d09a91640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d040b3cc0 con 0x7f1d040b1df0 2024-04-14T08:35:29.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.553+0000 7f1d09a91640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1d0415b7f0 con 0x7f1d040b21d0 2024-04-14T08:35:29.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.553+0000 7f1d03fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d040b4040 0x7f1d04156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.553+0000 7f1d03fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d040b4040 0x7f1d04156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42422/0 (socket says 172.21.15.172:42422) 2024-04-14T08:35:29.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.553+0000 7f1d03fff640 1 -- 172.21.15.172:0/3803324481 learned_addr learned my addr 172.21.15.172:0/3803324481 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:29.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1d037fe640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d040b1df0 0x7f1d04155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1d02ffd640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d040b21d0 0x7f1d0413dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1d03fff640 1 -- 172.21.15.172:0/3803324481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d040b21d0 msgr2=0x7f1d0413dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1d03fff640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d040b21d0 0x7f1d0413dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1d03fff640 1 -- 172.21.15.172:0/3803324481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d040b1df0 msgr2=0x7f1d04155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1d03fff640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d040b1df0 0x7f1d04155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1d03fff640 1 -- 172.21.15.172:0/3803324481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0413a920 con 0x7f1d040b4040 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1d03fff640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d040b4040 0x7f1d04156670 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00efc0 tx=0x7f1cfc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1cebfff640 1 -- 172.21.15.172:0/3803324481 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc018030 con 0x7f1d040b4040 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1cebfff640 1 -- 172.21.15.172:0/3803324481 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cfc00cde0 con 0x7f1d040b4040 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1d09a91640 1 -- 172.21.15.172:0/3803324481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0413ac00 con 0x7f1d040b4040 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1cebfff640 1 -- 172.21.15.172:0/3803324481 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc003bb0 con 0x7f1d040b4040 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.554+0000 7f1d09a91640 1 -- 172.21.15.172:0/3803324481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0413b1c0 con 0x7f1d040b4040 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.555+0000 7f1d09a91640 1 -- 172.21.15.172:0/3803324481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d04068520 con 0x7f1d040b4040 2024-04-14T08:35:29.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.556+0000 7f1cebfff640 1 -- 172.21.15.172:0/3803324481 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cfc01b070 con 0x7f1d040b4040 2024-04-14T08:35:29.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.556+0000 7f1cebfff640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1cf006c920 0x7f1cf006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.556+0000 7f1cebfff640 1 -- 172.21.15.172:0/3803324481 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7f1cfc08c320 con 0x7f1d040b4040 2024-04-14T08:35:29.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.556+0000 7f1d037fe640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1cf006c920 0x7f1cf006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.557+0000 7f1d037fe640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1cf006c920 0x7f1cf006ede0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4036000 tx=0x7f1cf403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:29.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.561+0000 7f1cebfff640 1 -- 172.21.15.172:0/3803324481 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1cfc010040 con 0x7f1d040b4040 2024-04-14T08:35:29.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.775+0000 7f1d09a91640 1 -- 172.21.15.172:0/3803324481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d040b1060 con 0x7f1d040b4040 2024-04-14T08:35:29.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.776+0000 7f1cebfff640 1 -- 172.21.15.172:0/3803324481 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+3875 (secure 0 0 0) 0x7f1cfc05a270 con 0x7f1d040b4040 2024-04-14T08:35:29.777 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:29.777 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/296861286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":296861286},{"type":"v1","addr":"172.21.15.177:6835","nonce":296861286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/642342704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":642342704},{"type":"v1","addr":"172.21.15.177:6833","nonce":642342704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3516903987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3516903987},{"type":"v1","addr":"172.21.15.172:6836","nonce":3516903987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":6117,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3696305730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3696305730},{"type":"v1","addr":"172.21.15.172:6837","nonce":3696305730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[]} 2024-04-14T08:35:29.777 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 25 2024-04-14T08:35:29.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.778+0000 7f1d09a91640 1 -- 172.21.15.172:0/3803324481 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1cf006c920 msgr2=0x7f1cf006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.778+0000 7f1d09a91640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1cf006c920 0x7f1cf006ede0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4036000 tx=0x7f1cf403c040 comp rx=0 tx=0).stop 2024-04-14T08:35:29.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.778+0000 7f1d09a91640 1 -- 172.21.15.172:0/3803324481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d040b4040 msgr2=0x7f1d04156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.778+0000 7f1d09a91640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d040b4040 0x7f1d04156670 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00efc0 tx=0x7f1cfc009f90 comp rx=0 tx=0).stop 2024-04-14T08:35:29.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.778+0000 7f1d09a91640 1 -- 172.21.15.172:0/3803324481 shutdown_connections 2024-04-14T08:35:29.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.778+0000 7f1d09a91640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1cf006c920 0x7f1cf006ede0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.778+0000 7f1d09a91640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d040b4040 0x7f1d04156670 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.778+0000 7f1d09a91640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d040b21d0 0x7f1d0413dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.778+0000 7f1d09a91640 1 --2- 172.21.15.172:0/3803324481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d040b1df0 0x7f1d04155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.778+0000 7f1d09a91640 1 -- 172.21.15.172:0/3803324481 >> 172.21.15.172:0/3803324481 conn(0x7f1d0406ec30 msgr2=0x7f1d040b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:29.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.778+0000 7f1d09a91640 1 -- 172.21.15.172:0/3803324481 shutdown_connections 2024-04-14T08:35:29.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.778+0000 7f1d09a91640 1 -- 172.21.15.172:0/3803324481 wait complete. 2024-04-14T08:35:29.791 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-14T08:35:29.791 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-14T08:35:29.791 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-14T08:35:29.791 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:35:29.792 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-14T08:35:29.792 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-14T08:35:29.792 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-14T08:35:29.792 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:35:29.792 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-14T08:35:29.792 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-14T08:35:29.792 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-14T08:35:29.793 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:35:29.793 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-14T08:35:29.793 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-14T08:35:29.793 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-14T08:35:29.793 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:35:29.801 INFO:tasks.ceph.mds.a:Stopped 2024-04-14T08:35:29.801 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-14T08:35:29.802 INFO:tasks.ceph.mds.c:Stopped 2024-04-14T08:35:29.802 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-14T08:35:29.805 INFO:tasks.ceph.mds.b:Stopped 2024-04-14T08:35:29.805 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-14T08:35:29.806 INFO:tasks.ceph.mds.d:Stopped 2024-04-14T08:35:29.806 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-14T08:35:29.844 INFO:tasks.ceph.mds.a:Started 2024-04-14T08:35:29.844 DEBUG:teuthology.parallel:result is None 2024-04-14T08:35:29.844 INFO:tasks.ceph.mds.c:Started 2024-04-14T08:35:29.845 DEBUG:teuthology.parallel:result is None 2024-04-14T08:35:29.847 INFO:tasks.ceph.mds.b:Started 2024-04-14T08:35:29.848 DEBUG:teuthology.parallel:result is None 2024-04-14T08:35:29.848 INFO:tasks.ceph.mds.d:Started 2024-04-14T08:35:29.848 DEBUG:teuthology.parallel:result is None 2024-04-14T08:35:29.849 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-14T08:35:29.922 INFO:tasks.ceph.mds.a.smithi172.stdout:starting mds.a at 2024-04-14T08:35:29.922 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:35:29.933 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-04-14T08:35:29.934 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-04-14T08:35:29.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.954+0000 7f3ed40b7640 1 Processor -- start 2024-04-14T08:35:29.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.954+0000 7f3ed40b7640 1 -- start start 2024-04-14T08:35:29.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.955+0000 7f3ed40b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ecc15e590 0x7f3ecc1629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.955+0000 7f3ed40b7640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3ecc0b65a0 0x7f3ecc0d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.955+0000 7f3ed40b7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ecc0b3750 con 0x7f3ecc0cea60 2024-04-14T08:35:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.955+0000 7f3ed40b7640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f3ecc0cef20 con 0x7f3ecc0b65a0 2024-04-14T08:35:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.955+0000 7f3ed40b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3ecc0cf200 con 0x7f3ecc15e590 2024-04-14T08:35:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.955+0000 7f3ed2654640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3ecc0b65a0 0x7f3ecc0d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.955+0000 7f3ed2654640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3ecc0b65a0 0x7f3ecc0d2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42452/0 (socket says 172.21.15.172:42452) 2024-04-14T08:35:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.955+0000 7f3ed2654640 1 -- 172.21.15.172:0/1534724951 learned_addr learned my addr 172.21.15.172:0/1534724951 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.956+0000 7f3ed1e53640 1 --2- 172.21.15.172:0/1534724951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ecc15e590 0x7f3ecc1629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.956+0000 7f3ed2654640 1 -- 172.21.15.172:0/1534724951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ecc15e590 msgr2=0x7f3ecc1629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.956+0000 7f3ed2654640 1 --2- 172.21.15.172:0/1534724951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ecc15e590 0x7f3ecc1629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.956+0000 7f3ed2654640 1 -- 172.21.15.172:0/1534724951 >> v1:172.21.15.177:6789/0 conn(0x7f3ecc0cea60 legacy=0x7f3ecc0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.956+0000 7f3ed2654640 1 -- 172.21.15.172:0/1534724951 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc0cf4e0 con 0x7f3ecc0b65a0 2024-04-14T08:35:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.956+0000 7f3ed2654640 1 --2- 172.21.15.172:0/1534724951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc0b65a0 0x7f3ecc0d2760 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009e50 tx=0x7f3ec402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae5b9f6f492dc4b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.957+0000 7f3ed0e51640 1 -- 172.21.15.172:0/1534724951 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec402ca70 con 0x7f3ecc0b65a0 2024-04-14T08:35:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.957+0000 7f3ed0e51640 1 -- 172.21.15.172:0/1534724951 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec402cc10 con 0x7f3ecc0b65a0 2024-04-14T08:35:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.957+0000 7f3ed0e51640 1 -- 172.21.15.172:0/1534724951 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40343f0 con 0x7f3ecc0b65a0 2024-04-14T08:35:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.957+0000 7f3ed40b7640 1 -- 172.21.15.172:0/1534724951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc0b65a0 msgr2=0x7f3ecc0d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.957+0000 7f3ed40b7640 1 --2- 172.21.15.172:0/1534724951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc0b65a0 0x7f3ecc0d2760 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009e50 tx=0x7f3ec402bed0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.957+0000 7f3ed40b7640 1 -- 172.21.15.172:0/1534724951 shutdown_connections 2024-04-14T08:35:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.957+0000 7f3ed40b7640 1 --2- 172.21.15.172:0/1534724951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc0b65a0 0x7f3ecc0d2760 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.957+0000 7f3ed40b7640 1 --2- 172.21.15.172:0/1534724951 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ecc15e590 0x7f3ecc1629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.957+0000 7f3ed40b7640 1 -- 172.21.15.172:0/1534724951 >> 172.21.15.172:0/1534724951 conn(0x7f3ecc0d0e90 msgr2=0x7f3ecc0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.957+0000 7f3ed40b7640 1 -- 172.21.15.172:0/1534724951 shutdown_connections 2024-04-14T08:35:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.957+0000 7f3ed40b7640 1 -- 172.21.15.172:0/1534724951 wait complete. 2024-04-14T08:35:29.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.958+0000 7f3ed40b7640 1 Processor -- start 2024-04-14T08:35:29.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.958+0000 7f3ed40b7640 1 -- start start 2024-04-14T08:35:29.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.959+0000 7f3ed40b7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ecc0b65a0 0x7f3ecc161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.959+0000 7f3ed40b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ecc0cea60 0x7f3ecc0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.959+0000 7f3ed40b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc15e590 0x7f3ecc0d4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.959+0000 7f3ed40b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc0d2e20 con 0x7f3ecc15e590 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.959+0000 7f3ed40b7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc0d2ca0 con 0x7f3ecc0b65a0 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.959+0000 7f3ed40b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3ecc0d2fa0 con 0x7f3ecc0cea60 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.959+0000 7f3ed2654640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc15e590 0x7f3ecc0d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.959+0000 7f3ed2654640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc15e590 0x7f3ecc0d4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42468/0 (socket says 172.21.15.172:42468) 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.959+0000 7f3ed2654640 1 -- 172.21.15.172:0/3371336606 learned_addr learned my addr 172.21.15.172:0/3371336606 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.959+0000 7f3ed1e53640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ecc0b65a0 0x7f3ecc161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.959+0000 7f3ed1652640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ecc0cea60 0x7f3ecc0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.960+0000 7f3ed2654640 1 -- 172.21.15.172:0/3371336606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ecc0cea60 msgr2=0x7f3ecc0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.960+0000 7f3ed2654640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ecc0cea60 0x7f3ecc0c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.960+0000 7f3ed2654640 1 -- 172.21.15.172:0/3371336606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ecc0b65a0 msgr2=0x7f3ecc161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.960+0000 7f3ed2654640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ecc0b65a0 0x7f3ecc161050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:29.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.960+0000 7f3ed2654640 1 -- 172.21.15.172:0/3371336606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc0c9380 con 0x7f3ecc15e590 2024-04-14T08:35:29.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.960+0000 7f3ed2654640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc15e590 0x7f3ecc0d4410 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009e50 tx=0x7f3ec4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:29.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.960+0000 7f3ec27fc640 1 -- 172.21.15.172:0/3371336606 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4043020 con 0x7f3ecc15e590 2024-04-14T08:35:29.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.960+0000 7f3ec27fc640 1 -- 172.21.15.172:0/3371336606 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec403e450 con 0x7f3ecc15e590 2024-04-14T08:35:29.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.960+0000 7f3ed40b7640 1 -- 172.21.15.172:0/3371336606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc0c9660 con 0x7f3ecc15e590 2024-04-14T08:35:29.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.960+0000 7f3ec27fc640 1 -- 172.21.15.172:0/3371336606 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4009510 con 0x7f3ecc15e590 2024-04-14T08:35:29.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.960+0000 7f3ed40b7640 1 -- 172.21.15.172:0/3371336606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc0c9c20 con 0x7f3ecc15e590 2024-04-14T08:35:29.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.961+0000 7f3ed40b7640 1 -- 172.21.15.172:0/3371336606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ecc068590 con 0x7f3ecc15e590 2024-04-14T08:35:29.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.961+0000 7f3ec27fc640 1 -- 172.21.15.172:0/3371336606 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ec4046020 con 0x7f3ecc15e590 2024-04-14T08:35:29.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.962+0000 7f3ec27fc640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3eb006c8f0 0x7f3eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:29.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.962+0000 7f3ec27fc640 1 -- 172.21.15.172:0/3371336606 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5335+0+0 (secure 0 0 0) 0x7f3ec40ae260 con 0x7f3ecc15e590 2024-04-14T08:35:29.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.962+0000 7f3ed1e53640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3eb006c8f0 0x7f3eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:29.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.962+0000 7f3ed1e53640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3eb006c8f0 0x7f3eb006edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00aa20 tx=0x7f3ebc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:29.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:29.966+0000 7f3ec27fc640 1 -- 172.21.15.172:0/3371336606 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3ec404d030 con 0x7f3ecc15e590 2024-04-14T08:35:30.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.155+0000 7f3ed40b7640 1 -- 172.21.15.172:0/3371336606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3ecc0b9830 con 0x7f3ecc15e590 2024-04-14T08:35:30.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.934+0000 7f3ec27fc640 1 -- 172.21.15.172:0/3371336606 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v55) ==== 97+0+0 (secure 0 0 0) 0x7f3ec407c1f0 con 0x7f3ecc15e590 2024-04-14T08:35:30.935 INFO:teuthology.orchestra.run.smithi172.stderr: removed all blocklist entries 2024-04-14T08:35:30.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.937+0000 7f3ed40b7640 1 -- 172.21.15.172:0/3371336606 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3eb006c8f0 msgr2=0x7f3eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:30.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.937+0000 7f3ed40b7640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3eb006c8f0 0x7f3eb006edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00aa20 tx=0x7f3ebc008040 comp rx=0 tx=0).stop 2024-04-14T08:35:30.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.937+0000 7f3ed40b7640 1 -- 172.21.15.172:0/3371336606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc15e590 msgr2=0x7f3ecc0d4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:30.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.937+0000 7f3ed40b7640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc15e590 0x7f3ecc0d4410 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009e50 tx=0x7f3ec4004a40 comp rx=0 tx=0).stop 2024-04-14T08:35:30.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.937+0000 7f3ed40b7640 1 -- 172.21.15.172:0/3371336606 shutdown_connections 2024-04-14T08:35:30.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.937+0000 7f3ed40b7640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3eb006c8f0 0x7f3eb006edb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:30.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.937+0000 7f3ed40b7640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ecc15e590 0x7f3ecc0d4410 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:30.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.937+0000 7f3ed40b7640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ecc0cea60 0x7f3ecc0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:30.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.937+0000 7f3ed40b7640 1 --2- 172.21.15.172:0/3371336606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ecc0b65a0 0x7f3ecc161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:30.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.937+0000 7f3ed40b7640 1 -- 172.21.15.172:0/3371336606 >> 172.21.15.172:0/3371336606 conn(0x7f3ecc0d0e90 msgr2=0x7f3ecc15d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:30.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.937+0000 7f3ed40b7640 1 -- 172.21.15.172:0/3371336606 shutdown_connections 2024-04-14T08:35:30.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:30.937+0000 7f3ed40b7640 1 -- 172.21.15.172:0/3371336606 wait complete. 2024-04-14T08:35:30.950 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-14T08:35:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.053+0000 7f01f7408640 1 Processor -- start 2024-04-14T08:35:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.053+0000 7f01f7408640 1 -- start start 2024-04-14T08:35:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.053+0000 7f01f7408640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f01f0072470 0x7f01f015b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.053+0000 7f01f7408640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f00b3360 con 0x7f01f015a690 2024-04-14T08:35:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.054+0000 7f01f7408640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f00b3640 con 0x7f01f0072970 2024-04-14T08:35:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.054+0000 7f01f7408640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f01f00b3920 con 0x7f01f0072470 2024-04-14T08:35:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.054+0000 7f01f49a3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f01f0072470 0x7f01f015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.054+0000 7f01f51a4640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f01f015a690 0x7f01f0071d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59824/0 (socket says 172.21.15.172:59824) 2024-04-14T08:35:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.054+0000 7f01f51a4640 1 -- 172.21.15.172:0/2698030175 learned_addr learned my addr 172.21.15.172:0/2698030175 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.054+0000 7f01f49a3640 1 -- 172.21.15.172:0/2698030175 >> v1:172.21.15.177:6789/0 conn(0x7f01f0072970 legacy=0x7f01f015b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.054+0000 7f01f49a3640 1 -- 172.21.15.172:0/2698030175 >> v1:172.21.15.172:6789/0 conn(0x7f01f015a690 legacy=0x7f01f0071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.054+0000 7f01effff640 1 -- 172.21.15.172:0/2698030175 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3244439246 0 0) 0x7f01f00b3360 con 0x7f01f015a690 2024-04-14T08:35:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.054+0000 7f01f49a3640 1 -- 172.21.15.172:0/2698030175 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f00b3c00 con 0x7f01f0072470 2024-04-14T08:35:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.055+0000 7f01f49a3640 1 --2- 172.21.15.172:0/2698030175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01f0072470 0x7f01f015b230 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f01e40097a0 tx=0x7f01e402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b152cb0d9fd95d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.055+0000 7f01effff640 1 -- 172.21.15.172:0/2698030175 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e40315a0 con 0x7f01f0072470 2024-04-14T08:35:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.055+0000 7f01effff640 1 -- 172.21.15.172:0/2698030175 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e4037070 con 0x7f01f0072470 2024-04-14T08:35:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.055+0000 7f01effff640 1 -- 172.21.15.172:0/2698030175 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4031960 con 0x7f01f0072470 2024-04-14T08:35:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.055+0000 7f01f7408640 1 -- 172.21.15.172:0/2698030175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01f0072470 msgr2=0x7f01f015b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.055+0000 7f01f7408640 1 --2- 172.21.15.172:0/2698030175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01f0072470 0x7f01f015b230 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f01e40097a0 tx=0x7f01e402b840 comp rx=0 tx=0).stop 2024-04-14T08:35:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.055+0000 7f01f7408640 1 -- 172.21.15.172:0/2698030175 shutdown_connections 2024-04-14T08:35:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.055+0000 7f01f7408640 1 --2- 172.21.15.172:0/2698030175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01f0072470 0x7f01f015b230 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.055+0000 7f01f7408640 1 -- 172.21.15.172:0/2698030175 >> 172.21.15.172:0/2698030175 conn(0x7f01f006ec30 msgr2=0x7f01f006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.055+0000 7f01f7408640 1 -- 172.21.15.172:0/2698030175 shutdown_connections 2024-04-14T08:35:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.055+0000 7f01f7408640 1 -- 172.21.15.172:0/2698030175 wait complete. 2024-04-14T08:35:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.057+0000 7f01f7408640 1 Processor -- start 2024-04-14T08:35:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.057+0000 7f01f7408640 1 -- start start 2024-04-14T08:35:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.057+0000 7f01f7408640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f0072470 0x7f01f01445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.057+0000 7f01f7408640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01f0072970 0x7f01f01410d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.057+0000 7f01f7408640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01f015a690 0x7f01f0150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.057+0000 7f01f7408640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01f015abf0 con 0x7f01f015a690 2024-04-14T08:35:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.057+0000 7f01f7408640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f01f015aa70 con 0x7f01f0072470 2024-04-14T08:35:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.057+0000 7f01f7408640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f01f015ad70 con 0x7f01f0072970 2024-04-14T08:35:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.057+0000 7f01f59a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01f015a690 0x7f01f0150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.057+0000 7f01f59a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01f015a690 0x7f01f0150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42488/0 (socket says 172.21.15.172:42488) 2024-04-14T08:35:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.057+0000 7f01f59a5640 1 -- 172.21.15.172:0/775538677 learned_addr learned my addr 172.21.15.172:0/775538677 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01f51a4640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f0072470 0x7f01f01445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01f49a3640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01f0072970 0x7f01f01410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01f59a5640 1 -- 172.21.15.172:0/775538677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01f0072970 msgr2=0x7f01f01410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01f59a5640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01f0072970 0x7f01f01410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01f59a5640 1 -- 172.21.15.172:0/775538677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f0072470 msgr2=0x7f01f01445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01f59a5640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f0072470 0x7f01f01445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01f59a5640 1 -- 172.21.15.172:0/775538677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f0152600 con 0x7f01f015a690 2024-04-14T08:35:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01f59a5640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01f015a690 0x7f01f0150880 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f01e800c410 tx=0x7f01e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01edffb640 1 -- 172.21.15.172:0/775538677 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8017070 con 0x7f01f015a690 2024-04-14T08:35:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01edffb640 1 -- 172.21.15.172:0/775538677 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e8007230 con 0x7f01f015a690 2024-04-14T08:35:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01f7408640 1 -- 172.21.15.172:0/775538677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f014fda0 con 0x7f01f015a690 2024-04-14T08:35:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01edffb640 1 -- 172.21.15.172:0/775538677 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8007660 con 0x7f01f015a690 2024-04-14T08:35:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.058+0000 7f01f7408640 1 -- 172.21.15.172:0/775538677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f0141610 con 0x7f01f015a690 2024-04-14T08:35:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.059+0000 7f01f7408640 1 -- 172.21.15.172:0/775538677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f0068520 con 0x7f01f015a690 2024-04-14T08:35:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.060+0000 7f01edffb640 1 -- 172.21.15.172:0/775538677 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01e801a070 con 0x7f01f015a690 2024-04-14T08:35:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.060+0000 7f01edffb640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01dc06c920 0x7f01dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.060+0000 7f01edffb640 1 -- 172.21.15.172:0/775538677 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7f01e808acb0 con 0x7f01f015a690 2024-04-14T08:35:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.060+0000 7f01f51a4640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01dc06c920 0x7f01dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.060+0000 7f01f51a4640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01dc06c920 0x7f01dc06ede0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f01e00012e0 tx=0x7f01e0005eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:31.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.064+0000 7f01edffb640 1 -- 172.21.15.172:0/775538677 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f01e80592a0 con 0x7f01f015a690 2024-04-14T08:35:31.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.308+0000 7f01f7408640 1 -- 172.21.15.172:0/775538677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f01f00cc6b0 con 0x7f01f015a690 2024-04-14T08:35:31.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.308+0000 7f01edffb640 1 -- 172.21.15.172:0/775538677 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f01e8058ba0 con 0x7f01f015a690 2024-04-14T08:35:31.309 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:31.309 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:35:31.309 INFO:teuthology.orchestra.run.smithi172.stdout: "auth_dump": [ 2024-04-14T08:35:31.309 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.309 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.a", 2024-04-14T08:35:31.309 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmnNCRDRAAz7aU3JD7iiaTQH0cNotfaw==", 2024-04-14T08:35:31.309 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.b", 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtm1on4GRAAms1he+O4jWu01z0ZLLgL0A==", 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:35:31.310 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.c", 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmb7z0FBAAGVu3+TWshDGvd7/fLbgJ5A==", 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.311 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.d", 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmEEx0IRAAXiFiOK1RE/0bfyu7KI/3ng==", 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.0", 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB/lBtmnsWuCRAAqcuQhsrzpGVfrlgwTnFsOw==", 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:35:31.312 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.1", 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCClBtme9lPBxAAFB1E3g8qwZxPCdHiFogp5Q==", 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.2", 2024-04-14T08:35:31.313 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCFlBtm966YBRAAWXZGq/0TuUMZTsyeR4xWww==", 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.3", 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCIlBtmHpktBBAAUbddNHESwyNvvp8UE/SaCw==", 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.314 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.4", 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCOlBtm+//7ChAA7rApbKll2vBGcDlW8q+kyg==", 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.5", 2024-04-14T08:35:31.315 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCRlBtmVU4YChAAysdUosQ+TnHtjnRl1FOISA==", 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.6", 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCUlBtmnE7+BxAAqBn8YMwecA2NcZz63eBm7Q==", 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:35:31.316 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.7", 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCXlBtmN8DhBRAA6CHQ943t6qJCL77jcbOv4w==", 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.0", 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA==", 2024-04-14T08:35:31.317 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow r", 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.1", 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmp4nkLhAAhYKm8VcykVbPSohvK11I9w==", 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:35:31.318 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.2", 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmJoV9KxAAp6zlRX+gvhASQBHjpF9ewQ==", 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.3", 2024-04-14T08:35:31.319 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmeC+pNBAASi7W4jlo6imMMzNEwLrPDA==", 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.admin", 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB6lBtmgnWqLBAAQ0fgarM8au3zFuMNnQIvVw==", 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:35:31.320 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.x", 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmm5IbChAAeVcVs3Zt6+lU0wghjCcwnQ==", 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.y", 2024-04-14T08:35:31.321 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmfddYBhAAh8zgf4W3WWmNYWNAA7WuRw==", 2024-04-14T08:35:31.322 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:35:31.322 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:35:31.322 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:35:31.322 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:35:31.322 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.322 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:31.322 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-04-14T08:35:31.322 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:35:31.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.310+0000 7f01f7408640 1 -- 172.21.15.172:0/775538677 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01dc06c920 msgr2=0x7f01dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.310+0000 7f01f7408640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01dc06c920 0x7f01dc06ede0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f01e00012e0 tx=0x7f01e0005eb0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.310+0000 7f01f7408640 1 -- 172.21.15.172:0/775538677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01f015a690 msgr2=0x7f01f0150880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.310+0000 7f01f7408640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01f015a690 0x7f01f0150880 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f01e800c410 tx=0x7f01e800c8d0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.310+0000 7f01f7408640 1 -- 172.21.15.172:0/775538677 shutdown_connections 2024-04-14T08:35:31.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.310+0000 7f01f7408640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01dc06c920 0x7f01dc06ede0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.310+0000 7f01f7408640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01f015a690 0x7f01f0150880 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.310+0000 7f01f7408640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01f0072970 0x7f01f01410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.310+0000 7f01f7408640 1 --2- 172.21.15.172:0/775538677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f0072470 0x7f01f01445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.310+0000 7f01f7408640 1 -- 172.21.15.172:0/775538677 >> 172.21.15.172:0/775538677 conn(0x7f01f006ec30 msgr2=0x7f01f015d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:31.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.311+0000 7f01f7408640 1 -- 172.21.15.172:0/775538677 shutdown_connections 2024-04-14T08:35:31.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.311+0000 7f01f7408640 1 -- 172.21.15.172:0/775538677 wait complete. 2024-04-14T08:35:31.325 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:35:31.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.432+0000 7f0940260640 1 Processor -- start 2024-04-14T08:35:31.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.433+0000 7f0940260640 1 -- start start 2024-04-14T08:35:31.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.433+0000 7f0940260640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f09380d2830 0x7f09380d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.433+0000 7f0940260640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09380d3150 con 0x7f093815e890 2024-04-14T08:35:31.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.433+0000 7f0940260640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09380cf320 con 0x7f09380cedd0 2024-04-14T08:35:31.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.433+0000 7f0940260640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f09380cff30 con 0x7f09380d2830 2024-04-14T08:35:31.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.433+0000 7f093d7fb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f093815e890 0x7f09380d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59832/0 (socket says 172.21.15.172:59832) 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.433+0000 7f093d7fb640 1 -- 172.21.15.172:0/3026693212 learned_addr learned my addr 172.21.15.172:0/3026693212 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093dffc640 1 --2- 172.21.15.172:0/3026693212 >> v2:172.21.15.177:3300/0 conn(0x7f09380d2830 0x7f09380d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971326449 0 0) 0x7f09380d3150 con 0x7f093815e890 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0924003520 con 0x7f093815e890 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3132047257 0 0) 0x7f0924003520 con 0x7f093815e890 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 >> v2:172.21.15.177:3300/0 conn(0x7f09380d2830 msgr2=0x7f09380d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 --2- 172.21.15.172:0/3026693212 >> v2:172.21.15.177:3300/0 conn(0x7f09380d2830 0x7f09380d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 >> v1:172.21.15.177:6789/0 conn(0x7f09380cedd0 legacy=0x7f09380b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09380cf600 con 0x7f093815e890 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f0928001670 con 0x7f093815e890 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093dffc640 1 --2- 172.21.15.172:0/3026693212 >> v2:172.21.15.177:3300/0 conn(0x7f09380d2830 0x7f09380d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 >> v1:172.21.15.172:6789/0 conn(0x7f093815e890 legacy=0x7f09380d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 --2- 172.21.15.172:0/3026693212 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09240039b0 0x7f0924003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 --2- 172.21.15.172:0/3026693212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0924004800 0x7f0924004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 --2- 172.21.15.172:0/3026693212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09240051c0 0x7f092400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f09380cff30 con 0x7f0924004800 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f092400c070 con 0x7f09240039b0 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f092400c270 con 0x7f09240051c0 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093d7fb640 1 --2- 172.21.15.172:0/3026693212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0924004800 0x7f0924004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093dffc640 1 --2- 172.21.15.172:0/3026693212 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09240039b0 0x7f0924003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093e7fd640 1 --2- 172.21.15.172:0/3026693212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09240051c0 0x7f092400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093d7fb640 1 -- 172.21.15.172:0/3026693212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09240051c0 msgr2=0x7f092400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093d7fb640 1 --2- 172.21.15.172:0/3026693212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09240051c0 0x7f092400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093d7fb640 1 -- 172.21.15.172:0/3026693212 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09240039b0 msgr2=0x7f0924003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093d7fb640 1 --2- 172.21.15.172:0/3026693212 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09240039b0 0x7f0924003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093d7fb640 1 -- 172.21.15.172:0/3026693212 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0924003520 con 0x7f0924004800 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093e7fd640 1 --2- 172.21.15.172:0/3026693212 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09240051c0 0x7f092400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:31.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.434+0000 7f093e7fd640 1 -- 172.21.15.172:0/3026693212 reap_dead start 2024-04-14T08:35:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.435+0000 7f093d7fb640 1 --2- 172.21.15.172:0/3026693212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0924004800 0x7f0924004c80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f092800c210 tx=0x7f0924005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49643e6a02eb9a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.435+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928007d30 con 0x7f0924004800 2024-04-14T08:35:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.435+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0928030070 con 0x7f0924004800 2024-04-14T08:35:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.435+0000 7f093cffa640 1 -- 172.21.15.172:0/3026693212 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092802cda0 con 0x7f0924004800 2024-04-14T08:35:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.436+0000 7f0940260640 1 -- 172.21.15.172:0/3026693212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0924004800 msgr2=0x7f0924004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.436+0000 7f0940260640 1 --2- 172.21.15.172:0/3026693212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0924004800 0x7f0924004c80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f092800c210 tx=0x7f0924005830 comp rx=0 tx=0).stop 2024-04-14T08:35:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.436+0000 7f0940260640 1 -- 172.21.15.172:0/3026693212 shutdown_connections 2024-04-14T08:35:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.436+0000 7f0940260640 1 --2- 172.21.15.172:0/3026693212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0924004800 0x7f0924004c80 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.436+0000 7f0940260640 1 -- 172.21.15.172:0/3026693212 >> 172.21.15.172:0/3026693212 conn(0x7f093809d590 msgr2=0x7f09380d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.436+0000 7f0940260640 1 -- 172.21.15.172:0/3026693212 shutdown_connections 2024-04-14T08:35:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.436+0000 7f0940260640 1 -- 172.21.15.172:0/3026693212 wait complete. 2024-04-14T08:35:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.437+0000 7f0940260640 1 Processor -- start 2024-04-14T08:35:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.437+0000 7f0940260640 1 -- start start 2024-04-14T08:35:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f0940260640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0924004800 0x7f0938158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f0940260640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09380c7d80 0x7f09380b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f0940260640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09380b8060 0x7f093815fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f0940260640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f09380cfbc0 con 0x7f09380b8060 2024-04-14T08:35:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f0940260640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f09380d2350 con 0x7f09380c7d80 2024-04-14T08:35:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f0940260640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f09380d2c10 con 0x7f0924004800 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f093e7fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09380b8060 0x7f093815fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f093e7fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09380b8060 0x7f093815fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42504/0 (socket says 172.21.15.172:42504) 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f093e7fd640 1 -- 172.21.15.172:0/2946270359 learned_addr learned my addr 172.21.15.172:0/2946270359 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f093dffc640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0924004800 0x7f0938158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f093d7fb640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09380c7d80 0x7f09380b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f093e7fd640 1 -- 172.21.15.172:0/2946270359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0924004800 msgr2=0x7f0938158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f093e7fd640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0924004800 0x7f0938158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f093e7fd640 1 -- 172.21.15.172:0/2946270359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09380c7d80 msgr2=0x7f09380b7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f093e7fd640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09380c7d80 0x7f09380b7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.438+0000 7f093e7fd640 1 -- 172.21.15.172:0/2946270359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0938159750 con 0x7f09380b8060 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.439+0000 7f093e7fd640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09380b8060 0x7f093815fcf0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f093808d6e0 tx=0x7f093000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.439+0000 7f092e7fc640 1 -- 172.21.15.172:0/2946270359 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930003970 con 0x7f09380b8060 2024-04-14T08:35:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.439+0000 7f092e7fc640 1 -- 172.21.15.172:0/2946270359 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0930018070 con 0x7f09380b8060 2024-04-14T08:35:31.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.439+0000 7f0940260640 1 -- 172.21.15.172:0/2946270359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09380b8670 con 0x7f09380b8060 2024-04-14T08:35:31.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.439+0000 7f092e7fc640 1 -- 172.21.15.172:0/2946270359 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930003d50 con 0x7f09380b8060 2024-04-14T08:35:31.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.439+0000 7f0940260640 1 -- 172.21.15.172:0/2946270359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09380b4ec0 con 0x7f09380b8060 2024-04-14T08:35:31.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.440+0000 7f0940260640 1 -- 172.21.15.172:0/2946270359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0938068520 con 0x7f09380b8060 2024-04-14T08:35:31.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.440+0000 7f092e7fc640 1 -- 172.21.15.172:0/2946270359 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09300050d0 con 0x7f09380b8060 2024-04-14T08:35:31.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.441+0000 7f092e7fc640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f091c06c8f0 0x7f091c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.441+0000 7f093dffc640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f091c06c8f0 0x7f091c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.441+0000 7f092e7fc640 1 -- 172.21.15.172:0/2946270359 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7f093008b200 con 0x7f09380b8060 2024-04-14T08:35:31.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.441+0000 7f093dffc640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f091c06c8f0 0x7f091c06edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0934009dc0 tx=0x7f0934009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:31.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.445+0000 7f092e7fc640 1 -- 172.21.15.172:0/2946270359 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0930022020 con 0x7f09380b8060 2024-04-14T08:35:31.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.623+0000 7f0940260640 1 -- 172.21.15.172:0/2946270359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09380d2610 con 0x7f09380b8060 2024-04-14T08:35:31.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.623+0000 7f092e7fc640 1 -- 172.21.15.172:0/2946270359 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+13224 (secure 0 0 0) 0x7f093008b020 con 0x7f09380b8060 2024-04-14T08:35:31.624 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:31.624 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":57,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:35:30.940764+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:35:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.626+0000 7f0940260640 1 -- 172.21.15.172:0/2946270359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f091c06c8f0 msgr2=0x7f091c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.626+0000 7f0940260640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f091c06c8f0 0x7f091c06edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0934009dc0 tx=0x7f0934009190 comp rx=0 tx=0).stop 2024-04-14T08:35:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.626+0000 7f0940260640 1 -- 172.21.15.172:0/2946270359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09380b8060 msgr2=0x7f093815fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.626+0000 7f0940260640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09380b8060 0x7f093815fcf0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f093808d6e0 tx=0x7f093000eb90 comp rx=0 tx=0).stop 2024-04-14T08:35:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.626+0000 7f0940260640 1 -- 172.21.15.172:0/2946270359 shutdown_connections 2024-04-14T08:35:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.626+0000 7f0940260640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f091c06c8f0 0x7f091c06edb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.626+0000 7f0940260640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09380b8060 0x7f093815fcf0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.626+0000 7f0940260640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09380c7d80 0x7f09380b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.626+0000 7f0940260640 1 --2- 172.21.15.172:0/2946270359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0924004800 0x7f0938158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.626+0000 7f0940260640 1 -- 172.21.15.172:0/2946270359 >> 172.21.15.172:0/2946270359 conn(0x7f093809d590 msgr2=0x7f09380b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:31.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.626+0000 7f0940260640 1 -- 172.21.15.172:0/2946270359 shutdown_connections 2024-04-14T08:35:31.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.626+0000 7f0940260640 1 -- 172.21.15.172:0/2946270359 wait complete. 2024-04-14T08:35:31.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:35:31.637 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:35:31.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.740+0000 7f378d00f640 1 Processor -- start 2024-04-14T08:35:31.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.740+0000 7f378d00f640 1 -- start start 2024-04-14T08:35:31.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.741+0000 7f378d00f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378813df30 0x7f37880b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.741+0000 7f378d00f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3788140e80 0x7f37880b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.741+0000 7f378d00f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f378815b620 con 0x7f378815c380 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.741+0000 7f378d00f640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f37880cf9b0 con 0x7f3788140e80 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.741+0000 7f378d00f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f37881414a0 con 0x7f378813df30 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.741+0000 7f378659c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f378815c380 0x7f3788140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59840/0 (socket says 172.21.15.172:59840) 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.741+0000 7f378659c640 1 -- 172.21.15.172:0/871415530 learned_addr learned my addr 172.21.15.172:0/871415530 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f378759e640 1 --2- 172.21.15.172:0/871415530 >> v2:172.21.15.177:3300/0 conn(0x7f3788140e80 0x7f37880b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4021965056 0 0) 0x7f3778001110 con 0x7f378815c380 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3786d9d640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378813df30 0x7f37880b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3774003520 con 0x7f378815c380 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3451707591 0 0) 0x7f3774003520 con 0x7f378815c380 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378813df30 msgr2=0x7f37880b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378813df30 0x7f37880b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 >> v2:172.21.15.177:3300/0 conn(0x7f3788140e80 msgr2=0x7f37880b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 --2- 172.21.15.172:0/871415530 >> v2:172.21.15.177:3300/0 conn(0x7f3788140e80 0x7f37880b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378815b8a0 con 0x7f378815c380 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f3778001840 con 0x7f378815c380 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 >> v1:172.21.15.172:6789/0 conn(0x7f378815c380 legacy=0x7f3788140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37740039b0 0x7f3774003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774004800 0x7f3774004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378813df30 0x7f37740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37880cf9b0 con 0x7f3774004800 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f37881414a0 con 0x7f37740039b0 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f377400b9c0 con 0x7f378813df30 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f378659c640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774004800 0x7f3774004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3786d9d640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37740039b0 0x7f3774003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f378759e640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378813df30 0x7f37740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f378659c640 1 -- 172.21.15.172:0/871415530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378813df30 msgr2=0x7f37740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:31.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f378659c640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378813df30 0x7f37740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f378659c640 1 -- 172.21.15.172:0/871415530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37740039b0 msgr2=0x7f3774003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f378659c640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37740039b0 0x7f3774003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f378659c640 1 -- 172.21.15.172:0/871415530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3774003520 con 0x7f3774004800 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.742+0000 7f3786d9d640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37740039b0 0x7f3774003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.743+0000 7f378659c640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774004800 0x7f3774004c80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f377800c4c0 tx=0x7f3774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7376bdcd2754ad server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.743+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377800c0e0 con 0x7f3774004800 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.743+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3778030070 con 0x7f3774004800 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.743+0000 7f3785d9b640 1 -- 172.21.15.172:0/871415530 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780058d0 con 0x7f3774004800 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.744+0000 7f378d00f640 1 -- 172.21.15.172:0/871415530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774004800 msgr2=0x7f3774004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.744+0000 7f378d00f640 1 --2- 172.21.15.172:0/871415530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774004800 0x7f3774004c80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f377800c4c0 tx=0x7f3774005830 comp rx=0 tx=0).stop 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.744+0000 7f378759e640 1 -- 172.21.15.172:0/871415530 reap_dead start 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.744+0000 7f378d00f640 1 -- 172.21.15.172:0/871415530 shutdown_connections 2024-04-14T08:35:31.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.744+0000 7f378d00f640 1 -- 172.21.15.172:0/871415530 >> 172.21.15.172:0/871415530 conn(0x7f378806b520 msgr2=0x7f378806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:31.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.744+0000 7f378d00f640 1 -- 172.21.15.172:0/871415530 shutdown_connections 2024-04-14T08:35:31.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.744+0000 7f378d00f640 1 -- 172.21.15.172:0/871415530 wait complete. 2024-04-14T08:35:31.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.745+0000 7f378d00f640 1 Processor -- start 2024-04-14T08:35:31.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.745+0000 7f378d00f640 1 -- start start 2024-04-14T08:35:31.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f378d00f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774004800 0x7f37881396e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f378d00f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3788155f70 0x7f37881563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f378d00f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37880b2090 0x7f37880b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f378d00f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f378813e310 con 0x7f3774004800 2024-04-14T08:35:31.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f378d00f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f378815be60 con 0x7f3788155f70 2024-04-14T08:35:31.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f378d00f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f37880b1cd0 con 0x7f37880b2090 2024-04-14T08:35:31.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f3786d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774004800 0x7f37881396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f3786d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774004800 0x7f37881396e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42534/0 (socket says 172.21.15.172:42534) 2024-04-14T08:35:31.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f3786d9d640 1 -- 172.21.15.172:0/3782268759 learned_addr learned my addr 172.21.15.172:0/3782268759 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:31.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f378759e640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37880b2090 0x7f37880b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f3786d9d640 1 -- 172.21.15.172:0/3782268759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37880b2090 msgr2=0x7f37880b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f378659c640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3788155f70 0x7f37881563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f3786d9d640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37880b2090 0x7f37880b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f3786d9d640 1 -- 172.21.15.172:0/3782268759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3788155f70 msgr2=0x7f37881563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f3786d9d640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3788155f70 0x7f37881563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f3786d9d640 1 -- 172.21.15.172:0/3782268759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37881f46a0 con 0x7f3774004800 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f378759e640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37880b2090 0x7f37880b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.746+0000 7f378659c640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3788155f70 0x7f37881563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.747+0000 7f3786d9d640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774004800 0x7f37881396e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f37700115a0 tx=0x7f3770011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.747+0000 7f377f7fe640 1 -- 172.21.15.172:0/3782268759 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770015870 con 0x7f3774004800 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.747+0000 7f377f7fe640 1 -- 172.21.15.172:0/3782268759 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3770015a10 con 0x7f3774004800 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.747+0000 7f378d00f640 1 -- 172.21.15.172:0/3782268759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37881f4980 con 0x7f3774004800 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.747+0000 7f377f7fe640 1 -- 172.21.15.172:0/3782268759 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770015dc0 con 0x7f3774004800 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.747+0000 7f378d00f640 1 -- 172.21.15.172:0/3782268759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37881f4f40 con 0x7f3774004800 2024-04-14T08:35:31.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.748+0000 7f378d00f640 1 -- 172.21.15.172:0/3782268759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3788068590 con 0x7f3774004800 2024-04-14T08:35:31.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.748+0000 7f377f7fe640 1 -- 172.21.15.172:0/3782268759 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f377000f480 con 0x7f3774004800 2024-04-14T08:35:31.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.749+0000 7f377f7fe640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f376806c8f0 0x7f376806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:31.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.749+0000 7f377f7fe640 1 -- 172.21.15.172:0/3782268759 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7f377008c170 con 0x7f3774004800 2024-04-14T08:35:31.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.749+0000 7f378659c640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f376806c8f0 0x7f376806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:31.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.749+0000 7f378659c640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f376806c8f0 0x7f376806edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3778034000 tx=0x7f3778009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:31.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.753+0000 7f377f7fe640 1 -- 172.21.15.172:0/3782268759 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3770019030 con 0x7f3774004800 2024-04-14T08:35:31.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.927+0000 7f378d00f640 1 -- 172.21.15.172:0/3782268759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f37880b38b0 con 0x7f3774004800 2024-04-14T08:35:31.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.927+0000 7f377f7fe640 1 -- 172.21.15.172:0/3782268759 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) ==== 93+0+10 (secure 0 0 0) 0x7f377005a760 con 0x7f3774004800 2024-04-14T08:35:31.928 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:35:31.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.929+0000 7f378d00f640 1 -- 172.21.15.172:0/3782268759 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f376806c8f0 msgr2=0x7f376806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.929+0000 7f378d00f640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f376806c8f0 0x7f376806edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3778034000 tx=0x7f3778009030 comp rx=0 tx=0).stop 2024-04-14T08:35:31.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.929+0000 7f378d00f640 1 -- 172.21.15.172:0/3782268759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774004800 msgr2=0x7f37881396e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:31.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.929+0000 7f378d00f640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774004800 0x7f37881396e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f37700115a0 tx=0x7f3770011a60 comp rx=0 tx=0).stop 2024-04-14T08:35:31.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.929+0000 7f378d00f640 1 -- 172.21.15.172:0/3782268759 shutdown_connections 2024-04-14T08:35:31.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.929+0000 7f378d00f640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f376806c8f0 0x7f376806edb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.929+0000 7f378d00f640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37880b2090 0x7f37880b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.929+0000 7f378d00f640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3788155f70 0x7f37881563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.929+0000 7f378d00f640 1 --2- 172.21.15.172:0/3782268759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3774004800 0x7f37881396e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:31.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.929+0000 7f378d00f640 1 -- 172.21.15.172:0/3782268759 >> 172.21.15.172:0/3782268759 conn(0x7f378806b520 msgr2=0x7f37880711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:31.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.929+0000 7f378d00f640 1 -- 172.21.15.172:0/3782268759 shutdown_connections 2024-04-14T08:35:31.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:31.929+0000 7f378d00f640 1 -- 172.21.15.172:0/3782268759 wait complete. 2024-04-14T08:35:31.942 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-14T08:35:32.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.043+0000 7f451560f640 1 Processor -- start 2024-04-14T08:35:32.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.043+0000 7f451560f640 1 -- start start 2024-04-14T08:35:32.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.043+0000 7f451560f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f45100d2830 0x7f45100d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.043+0000 7f451560f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f451015e890 0x7f45100d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.043+0000 7f451560f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45100d3150 con 0x7f45100cedd0 2024-04-14T08:35:32.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.043+0000 7f451560f640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f45100cff30 con 0x7f45100d2830 2024-04-14T08:35:32.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.043+0000 7f451560f640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f45100d2350 con 0x7f451015e890 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.044+0000 7f450effd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f45100d2830 0x7f45100d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.044+0000 7f450f7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f45100cedd0 0x7f45101621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59842/0 (socket says 172.21.15.172:59842) 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.044+0000 7f450f7fe640 1 -- 172.21.15.172:0/4188024323 learned_addr learned my addr 172.21.15.172:0/4188024323 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.044+0000 7f450effd640 1 -- 172.21.15.172:0/4188024323 >> v2:172.21.15.177:3300/0 conn(0x7f451015e890 msgr2=0x7f45100d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.044+0000 7f450e7fc640 1 --2- 172.21.15.172:0/4188024323 >> v2:172.21.15.177:3300/0 conn(0x7f451015e890 0x7f45100d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.044+0000 7f450effd640 1 --2- 172.21.15.172:0/4188024323 >> v2:172.21.15.177:3300/0 conn(0x7f451015e890 0x7f45100d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.044+0000 7f450dffb640 1 -- 172.21.15.172:0/4188024323 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 646875696 0 0) 0x7f45100d3150 con 0x7f45100cedd0 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.044+0000 7f450effd640 1 -- 172.21.15.172:0/4188024323 >> v1:172.21.15.172:6789/0 conn(0x7f45100cedd0 legacy=0x7f45101621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.044+0000 7f450effd640 1 -- 172.21.15.172:0/4188024323 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45100cf320 con 0x7f45100d2830 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.044+0000 7f450e7fc640 1 --2- 172.21.15.172:0/4188024323 >> v2:172.21.15.177:3300/0 conn(0x7f451015e890 0x7f45100d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.045+0000 7f450effd640 1 --2- 172.21.15.172:0/4188024323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45100d2830 0x7f45100d2c10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4510162100 tx=0x7f450002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75dcfea1a8300053 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.045+0000 7f450dffb640 1 -- 172.21.15.172:0/4188024323 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500039070 con 0x7f45100d2830 2024-04-14T08:35:32.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.045+0000 7f450dffb640 1 -- 172.21.15.172:0/4188024323 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450002cd00 con 0x7f45100d2830 2024-04-14T08:35:32.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.045+0000 7f450dffb640 1 -- 172.21.15.172:0/4188024323 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500034660 con 0x7f45100d2830 2024-04-14T08:35:32.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.045+0000 7f451560f640 1 -- 172.21.15.172:0/4188024323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45100d2830 msgr2=0x7f45100d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:32.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.045+0000 7f451560f640 1 --2- 172.21.15.172:0/4188024323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45100d2830 0x7f45100d2c10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4510162100 tx=0x7f450002bf50 comp rx=0 tx=0).stop 2024-04-14T08:35:32.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.045+0000 7f451560f640 1 -- 172.21.15.172:0/4188024323 shutdown_connections 2024-04-14T08:35:32.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.045+0000 7f451560f640 1 --2- 172.21.15.172:0/4188024323 >> v2:172.21.15.177:3300/0 conn(0x7f451015e890 0x7f45100d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.045+0000 7f451560f640 1 --2- 172.21.15.172:0/4188024323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45100d2830 0x7f45100d2c10 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.045+0000 7f451560f640 1 -- 172.21.15.172:0/4188024323 >> 172.21.15.172:0/4188024323 conn(0x7f451009d590 msgr2=0x7f45100d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:32.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.045+0000 7f451560f640 1 -- 172.21.15.172:0/4188024323 shutdown_connections 2024-04-14T08:35:32.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.045+0000 7f451560f640 1 -- 172.21.15.172:0/4188024323 wait complete. 2024-04-14T08:35:32.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.047+0000 7f451560f640 1 Processor -- start 2024-04-14T08:35:32.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.047+0000 7f451560f640 1 -- start start 2024-04-14T08:35:32.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.047+0000 7f451560f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45100cedd0 0x7f451014e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.047+0000 7f451560f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45100d2830 0x7f45100b9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.047+0000 7f451560f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f451015e890 0x7f451014ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.047+0000 7f451560f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f45100cfbc0 con 0x7f45100d2830 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.047+0000 7f451560f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f45100cf8e0 con 0x7f451015e890 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.047+0000 7f451560f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f45100d1840 con 0x7f45100cedd0 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.047+0000 7f450e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45100d2830 0x7f45100b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f450e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45100d2830 0x7f45100b9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42556/0 (socket says 172.21.15.172:42556) 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f450e7fc640 1 -- 172.21.15.172:0/1351609818 learned_addr learned my addr 172.21.15.172:0/1351609818 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f450f7fe640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f451015e890 0x7f451014ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f450effd640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45100cedd0 0x7f451014e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f450e7fc640 1 -- 172.21.15.172:0/1351609818 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45100cedd0 msgr2=0x7f451014e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f450e7fc640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45100cedd0 0x7f451014e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f450e7fc640 1 -- 172.21.15.172:0/1351609818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f451015e890 msgr2=0x7f451014ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f450e7fc640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f451015e890 0x7f451014ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f450e7fc640 1 -- 172.21.15.172:0/1351609818 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45100b9cd0 con 0x7f45100d2830 2024-04-14T08:35:32.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f450f7fe640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f451015e890 0x7f451014ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:32.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f450e7fc640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45100d2830 0x7f45100b9530 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f450400c7f0 tx=0x7f450400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:32.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f44f77fe640 1 -- 172.21.15.172:0/1351609818 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504015070 con 0x7f45100d2830 2024-04-14T08:35:32.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f44f77fe640 1 -- 172.21.15.172:0/1351609818 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4504007a30 con 0x7f45100d2830 2024-04-14T08:35:32.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f44f77fe640 1 -- 172.21.15.172:0/1351609818 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504007e00 con 0x7f45100d2830 2024-04-14T08:35:32.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f451560f640 1 -- 172.21.15.172:0/1351609818 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45100c9650 con 0x7f45100d2830 2024-04-14T08:35:32.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.048+0000 7f451560f640 1 -- 172.21.15.172:0/1351609818 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45100c9c10 con 0x7f45100d2830 2024-04-14T08:35:32.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.049+0000 7f451560f640 1 -- 172.21.15.172:0/1351609818 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4510068520 con 0x7f45100d2830 2024-04-14T08:35:32.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.050+0000 7f44f77fe640 1 -- 172.21.15.172:0/1351609818 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4504018070 con 0x7f45100d2830 2024-04-14T08:35:32.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.050+0000 7f44f77fe640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f44fc06c8f0 0x7f44fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.050+0000 7f44f77fe640 1 -- 172.21.15.172:0/1351609818 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7f450408b9a0 con 0x7f45100d2830 2024-04-14T08:35:32.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.050+0000 7f450effd640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f44fc06c8f0 0x7f44fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.050+0000 7f450effd640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f44fc06c8f0 0x7f44fc06edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4500036000 tx=0x7f4500030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:32.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.054+0000 7f44f77fe640 1 -- 172.21.15.172:0/1351609818 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4504059f90 con 0x7f45100d2830 2024-04-14T08:35:32.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.230+0000 7f451560f640 1 -- 172.21.15.172:0/1351609818 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f45100d2610 con 0x7f45100d2830 2024-04-14T08:35:32.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.230+0000 7f44f77fe640 1 -- 172.21.15.172:0/1351609818 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) ==== 84+0+65 (secure 0 0 0) 0x7f4504059890 con 0x7f45100d2830 2024-04-14T08:35:32.231 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:32.231 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-04-14T08:35:32.231 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:32.231 INFO:teuthology.orchestra.run.smithi172.stdout: "poolnum": 1, 2024-04-14T08:35:32.231 INFO:teuthology.orchestra.run.smithi172.stdout: "poolname": ".mgr" 2024-04-14T08:35:32.231 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:32.231 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-04-14T08:35:32.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.232+0000 7f451560f640 1 -- 172.21.15.172:0/1351609818 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f44fc06c8f0 msgr2=0x7f44fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:32.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.232+0000 7f451560f640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f44fc06c8f0 0x7f44fc06edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4500036000 tx=0x7f4500030000 comp rx=0 tx=0).stop 2024-04-14T08:35:32.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.232+0000 7f451560f640 1 -- 172.21.15.172:0/1351609818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45100d2830 msgr2=0x7f45100b9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:32.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.232+0000 7f451560f640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45100d2830 0x7f45100b9530 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f450400c7f0 tx=0x7f450400cbf0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.232+0000 7f451560f640 1 -- 172.21.15.172:0/1351609818 shutdown_connections 2024-04-14T08:35:32.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.232+0000 7f451560f640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f44fc06c8f0 0x7f44fc06edb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.232+0000 7f451560f640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f451015e890 0x7f451014ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.232+0000 7f451560f640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45100d2830 0x7f45100b9530 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.232+0000 7f451560f640 1 --2- 172.21.15.172:0/1351609818 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45100cedd0 0x7f451014e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.232+0000 7f451560f640 1 -- 172.21.15.172:0/1351609818 >> 172.21.15.172:0/1351609818 conn(0x7f451009d590 msgr2=0x7f45100d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:32.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.232+0000 7f451560f640 1 -- 172.21.15.172:0/1351609818 shutdown_connections 2024-04-14T08:35:32.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.232+0000 7f451560f640 1 -- 172.21.15.172:0/1351609818 wait complete. 2024-04-14T08:35:32.244 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-14T08:35:32.244 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-14T08:35:32.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.348+0000 7f3ac4d55640 1 Processor -- start 2024-04-14T08:35:32.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.348+0000 7f3ac4d55640 1 -- start start 2024-04-14T08:35:32.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.348+0000 7f3ac4d55640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015b350 0x7f3ac00cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.348+0000 7f3ac4d55640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3ac00cee90 0x7f3ac00c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.348+0000 7f3ac4d55640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ac00cdb90 con 0x7f3ac00cc170 2024-04-14T08:35:32.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.348+0000 7f3ac4d55640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f3ac00cd940 con 0x7f3ac00cee90 2024-04-14T08:35:32.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.348+0000 7f3ac4d55640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac00ce4c0 con 0x7f3ac015b350 2024-04-14T08:35:32.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3ab7fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3ac00cc170 0x7f3ac00ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59846/0 (socket says 172.21.15.172:59846) 2024-04-14T08:35:32.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3ab7fff640 1 -- 172.21.15.172:0/821153648 learned_addr learned my addr 172.21.15.172:0/821153648 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:32.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abe59c640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015b350 0x7f3ac00cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abed9d640 1 --2- 172.21.15.172:0/821153648 >> v2:172.21.15.177:3300/0 conn(0x7f3ac00cee90 0x7f3ac00c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3883065533 0 0) 0x7f3ac00cdb90 con 0x7f3ac00cc170 2024-04-14T08:35:32.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3aac005730 con 0x7f3ac00cc170 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3758877427 0 0) 0x7f3aac005730 con 0x7f3ac00cc170 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015b350 msgr2=0x7f3ac00cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015b350 0x7f3ac00cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 >> v2:172.21.15.177:3300/0 conn(0x7f3ac00cee90 msgr2=0x7f3ac00c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 --2- 172.21.15.172:0/821153648 >> v2:172.21.15.177:3300/0 conn(0x7f3ac00cee90 0x7f3ac00c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac00cde70 con 0x7f3ac00cc170 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f3aa8001670 con 0x7f3ac00cc170 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 >> v1:172.21.15.172:6789/0 conn(0x7f3ac00cc170 legacy=0x7f3ac00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3aac005bc0 0x7f3aac005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aac006a10 0x7f3aac006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015b350 0x7f3aac0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac00cd940 con 0x7f3aac005bc0 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac00ce4c0 con 0x7f3aac006a10 2024-04-14T08:35:32.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3aac00dc20 con 0x7f3ac015b350 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abe59c640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3aac005bc0 0x7f3aac005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3ab7fff640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aac006a10 0x7f3aac006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.349+0000 7f3abed9d640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015b350 0x7f3aac0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.350+0000 7f3abe59c640 1 -- 172.21.15.172:0/821153648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015b350 msgr2=0x7f3aac0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.350+0000 7f3abe59c640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015b350 0x7f3aac0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.350+0000 7f3abe59c640 1 -- 172.21.15.172:0/821153648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aac006a10 msgr2=0x7f3aac006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.350+0000 7f3abe59c640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aac006a10 0x7f3aac006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.350+0000 7f3abe59c640 1 -- 172.21.15.172:0/821153648 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aac005730 con 0x7f3aac005bc0 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.350+0000 7f3ab7fff640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aac006a10 0x7f3aac006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.350+0000 7f3abe59c640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3aac005bc0 0x7f3aac005fe0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009af0 tx=0x7f3aac007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e3bc32eb25e680 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.350+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0039070 con 0x7f3aac005bc0 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.350+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab002c6b0 con 0x7f3aac005bc0 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.350+0000 7f3abdd9b640 1 -- 172.21.15.172:0/821153648 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0009690 con 0x7f3aac005bc0 2024-04-14T08:35:32.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.351+0000 7f3ac4d55640 1 -- 172.21.15.172:0/821153648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3aac005bc0 msgr2=0x7f3aac005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:32.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.351+0000 7f3ac4d55640 1 --2- 172.21.15.172:0/821153648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3aac005bc0 0x7f3aac005fe0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009af0 tx=0x7f3aac007a40 comp rx=0 tx=0).stop 2024-04-14T08:35:32.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.351+0000 7f3abed9d640 1 -- 172.21.15.172:0/821153648 reap_dead start 2024-04-14T08:35:32.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.351+0000 7f3ac4d55640 1 -- 172.21.15.172:0/821153648 shutdown_connections 2024-04-14T08:35:32.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.351+0000 7f3ac4d55640 1 -- 172.21.15.172:0/821153648 >> 172.21.15.172:0/821153648 conn(0x7f3ac00b1a50 msgr2=0x7f3ac00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:32.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.351+0000 7f3ac4d55640 1 -- 172.21.15.172:0/821153648 shutdown_connections 2024-04-14T08:35:32.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.351+0000 7f3ac4d55640 1 -- 172.21.15.172:0/821153648 wait complete. 2024-04-14T08:35:32.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.352+0000 7f3ac4d55640 1 Processor -- start 2024-04-14T08:35:32.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.352+0000 7f3ac4d55640 1 -- start start 2024-04-14T08:35:32.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3ac4d55640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aac005bc0 0x7f3ac014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3ac4d55640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015df90 0x7f3ac00b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3ac4d55640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ac015e410 0x7f3ac00b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3ac4d55640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac00cbc30 con 0x7f3ac015e410 2024-04-14T08:35:32.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3ac4d55640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac00cf560 con 0x7f3aac005bc0 2024-04-14T08:35:32.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3ac4d55640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3ac00cbdb0 con 0x7f3ac015df90 2024-04-14T08:35:32.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3abed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ac015e410 0x7f3ac00b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3abed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ac015e410 0x7f3ac00b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42574/0 (socket says 172.21.15.172:42574) 2024-04-14T08:35:32.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3abed9d640 1 -- 172.21.15.172:0/2652217402 learned_addr learned my addr 172.21.15.172:0/2652217402 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3abe59c640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aac005bc0 0x7f3ac014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3ab7fff640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015df90 0x7f3ac00b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3abed9d640 1 -- 172.21.15.172:0/2652217402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015df90 msgr2=0x7f3ac00b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3abed9d640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015df90 0x7f3ac00b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3abed9d640 1 -- 172.21.15.172:0/2652217402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aac005bc0 msgr2=0x7f3ac014e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3abed9d640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aac005bc0 0x7f3ac014e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3abed9d640 1 -- 172.21.15.172:0/2652217402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac009c1e0 con 0x7f3ac015e410 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.353+0000 7f3ab7fff640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015df90 0x7f3ac00b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.354+0000 7f3abed9d640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ac015e410 0x7f3ac00b1270 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0070f90 tx=0x7f3ab80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.354+0000 7f3ab6ffd640 1 -- 172.21.15.172:0/2652217402 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8003990 con 0x7f3ac015e410 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.354+0000 7f3ab6ffd640 1 -- 172.21.15.172:0/2652217402 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab801b070 con 0x7f3ac015e410 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.354+0000 7f3ac4d55640 1 -- 172.21.15.172:0/2652217402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac009c4c0 con 0x7f3ac015e410 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.354+0000 7f3ab6ffd640 1 -- 172.21.15.172:0/2652217402 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8003b70 con 0x7f3ac015e410 2024-04-14T08:35:32.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.354+0000 7f3ac4d55640 1 -- 172.21.15.172:0/2652217402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac009ca80 con 0x7f3ac015e410 2024-04-14T08:35:32.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.355+0000 7f3ac4d55640 1 -- 172.21.15.172:0/2652217402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac00c76e0 con 0x7f3ac015e410 2024-04-14T08:35:32.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.355+0000 7f3ab6ffd640 1 -- 172.21.15.172:0/2652217402 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ab8003d10 con 0x7f3ac015e410 2024-04-14T08:35:32.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.355+0000 7f3ab6ffd640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3aa006c8f0 0x7f3aa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:32.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.356+0000 7f3ab6ffd640 1 -- 172.21.15.172:0/2652217402 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7f3ab808c110 con 0x7f3ac015e410 2024-04-14T08:35:32.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.356+0000 7f3abe59c640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3aa006c8f0 0x7f3aa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:32.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.356+0000 7f3abe59c640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3aa006c8f0 0x7f3aa006edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0036000 tx=0x7f3ab0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:32.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.360+0000 7f3ab6ffd640 1 -- 172.21.15.172:0/2652217402 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3ac00c76e0 con 0x7f3ac015e410 2024-04-14T08:35:32.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.534+0000 7f3ac4d55640 1 -- 172.21.15.172:0/2652217402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3ac015a010 con 0x7f3ac015e410 2024-04-14T08:35:32.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.945+0000 7f3ab6ffd640 1 -- 172.21.15.172:0/2652217402 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) ==== 138+0+0 (secure 0 0 0) 0x7f3ab805a700 con 0x7f3ac015e410 2024-04-14T08:35:32.946 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' created 2024-04-14T08:35:32.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.948+0000 7f3ac4d55640 1 -- 172.21.15.172:0/2652217402 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3aa006c8f0 msgr2=0x7f3aa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:32.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.948+0000 7f3ac4d55640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3aa006c8f0 0x7f3aa006edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0036000 tx=0x7f3ab0030000 comp rx=0 tx=0).stop 2024-04-14T08:35:32.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.948+0000 7f3ac4d55640 1 -- 172.21.15.172:0/2652217402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ac015e410 msgr2=0x7f3ac00b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:32.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.948+0000 7f3ac4d55640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ac015e410 0x7f3ac00b1270 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0070f90 tx=0x7f3ab80115a0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.948+0000 7f3ac4d55640 1 -- 172.21.15.172:0/2652217402 shutdown_connections 2024-04-14T08:35:32.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.948+0000 7f3ac4d55640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3aa006c8f0 0x7f3aa006edb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.948+0000 7f3ac4d55640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ac015e410 0x7f3ac00b1270 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.948+0000 7f3ac4d55640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ac015df90 0x7f3ac00b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.948+0000 7f3ac4d55640 1 --2- 172.21.15.172:0/2652217402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3aac005bc0 0x7f3ac014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:32.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.948+0000 7f3ac4d55640 1 -- 172.21.15.172:0/2652217402 >> 172.21.15.172:0/2652217402 conn(0x7f3ac00b1a50 msgr2=0x7f3ac0159560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:32.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.948+0000 7f3ac4d55640 1 -- 172.21.15.172:0/2652217402 shutdown_connections 2024-04-14T08:35:32.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:32.948+0000 7f3ac4d55640 1 -- 172.21.15.172:0/2652217402 wait complete. 2024-04-14T08:35:32.963 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-14T08:35:33.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.070+0000 7fc9e8676640 1 Processor -- start 2024-04-14T08:35:33.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.070+0000 7fc9e8676640 1 -- start start 2024-04-14T08:35:33.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.071+0000 7fc9e8676640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc9e0162ba0 0x7fc9e00d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:33.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.071+0000 7fc9e8676640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e00cefb0 con 0x7fc9e015e430 2024-04-14T08:35:33.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.071+0000 7fc9e8676640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e00cf290 con 0x7fc9e0162710 2024-04-14T08:35:33.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.071+0000 7fc9e8676640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc9e00cf570 con 0x7fc9e0162ba0 2024-04-14T08:35:33.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.071+0000 7fc9e6412640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc9e015e430 0x7fc9e0162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59850/0 (socket says 172.21.15.172:59850) 2024-04-14T08:35:33.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.071+0000 7fc9e6412640 1 -- 172.21.15.172:0/3142722287 learned_addr learned my addr 172.21.15.172:0/3142722287 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:33.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.071+0000 7fc9e6c13640 1 --2- 172.21.15.172:0/3142722287 >> v2:172.21.15.172:3300/0 conn(0x7fc9e0162ba0 0x7fc9e00d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:33.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.072+0000 7fc9e5410640 1 -- 172.21.15.172:0/3142722287 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2657816844 0 0) 0x7fc9e00cefb0 con 0x7fc9e015e430 2024-04-14T08:35:33.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.072+0000 7fc9e5410640 1 -- 172.21.15.172:0/3142722287 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9cc003520 con 0x7fc9e015e430 2024-04-14T08:35:33.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.072+0000 7fc9e6c13640 1 -- 172.21.15.172:0/3142722287 >> v1:172.21.15.177:6789/0 conn(0x7fc9e0162710 legacy=0x7fc9e00d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:33.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.072+0000 7fc9e6c13640 1 -- 172.21.15.172:0/3142722287 >> v1:172.21.15.172:6789/0 conn(0x7fc9e015e430 legacy=0x7fc9e0162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:33.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.072+0000 7fc9e6c13640 1 -- 172.21.15.172:0/3142722287 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e00cf850 con 0x7fc9e0162ba0 2024-04-14T08:35:33.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.072+0000 7fc9e6c13640 1 --2- 172.21.15.172:0/3142722287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9e0162ba0 0x7fc9e00d2500 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009d90 tx=0x7fc9d802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50de2c2110cefe16 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:33.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.073+0000 7fc9e5410640 1 -- 172.21.15.172:0/3142722287 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8039070 con 0x7fc9e0162ba0 2024-04-14T08:35:33.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.073+0000 7fc9e5410640 1 -- 172.21.15.172:0/3142722287 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d802cc60 con 0x7fc9e0162ba0 2024-04-14T08:35:33.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.073+0000 7fc9e5410640 1 -- 172.21.15.172:0/3142722287 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8034640 con 0x7fc9e0162ba0 2024-04-14T08:35:33.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.073+0000 7fc9e8676640 1 -- 172.21.15.172:0/3142722287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9e0162ba0 msgr2=0x7fc9e00d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:33.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.073+0000 7fc9e8676640 1 --2- 172.21.15.172:0/3142722287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9e0162ba0 0x7fc9e00d2500 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009d90 tx=0x7fc9d802beb0 comp rx=0 tx=0).stop 2024-04-14T08:35:33.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.073+0000 7fc9e8676640 1 -- 172.21.15.172:0/3142722287 shutdown_connections 2024-04-14T08:35:33.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.073+0000 7fc9e8676640 1 --2- 172.21.15.172:0/3142722287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9e0162ba0 0x7fc9e00d2500 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:33.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.073+0000 7fc9e8676640 1 -- 172.21.15.172:0/3142722287 >> 172.21.15.172:0/3142722287 conn(0x7fc9e009d560 msgr2=0x7fc9e00d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:33.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.073+0000 7fc9e8676640 1 -- 172.21.15.172:0/3142722287 shutdown_connections 2024-04-14T08:35:33.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.073+0000 7fc9e8676640 1 -- 172.21.15.172:0/3142722287 wait complete. 2024-04-14T08:35:33.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.074+0000 7fc9e8676640 1 Processor -- start 2024-04-14T08:35:33.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.074+0000 7fc9e8676640 1 -- start start 2024-04-14T08:35:33.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e8676640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9e015e430 0x7fc9e014b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:33.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e8676640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9e0162710 0x7fc9e00c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:33.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e8676640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9e0162ba0 0x7fc9e00b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:33.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e8676640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e00d4e20 con 0x7fc9e015e430 2024-04-14T08:35:33.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e8676640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e015e810 con 0x7fc9e0162ba0 2024-04-14T08:35:33.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e8676640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e00d0050 con 0x7fc9e0162710 2024-04-14T08:35:33.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e6412640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9e015e430 0x7fc9e014b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:33.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e6412640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9e015e430 0x7fc9e014b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42596/0 (socket says 172.21.15.172:42596) 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e6412640 1 -- 172.21.15.172:0/415117409 learned_addr learned my addr 172.21.15.172:0/415117409 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e6c13640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9e0162ba0 0x7fc9e00b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e5c11640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9e0162710 0x7fc9e00c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e6412640 1 -- 172.21.15.172:0/415117409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9e0162710 msgr2=0x7fc9e00c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e6412640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9e0162710 0x7fc9e00c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e6412640 1 -- 172.21.15.172:0/415117409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9e0162ba0 msgr2=0x7fc9e00b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e6412640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9e0162ba0 0x7fc9e00b6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.075+0000 7fc9e6412640 1 -- 172.21.15.172:0/415117409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e00c8ec0 con 0x7fc9e015e430 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.076+0000 7fc9e6c13640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9e0162ba0 0x7fc9e00b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.076+0000 7fc9e6412640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9e015e430 0x7fc9e014b730 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00c390 tx=0x7fc9dc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.076+0000 7fc9d6ffd640 1 -- 172.21.15.172:0/415117409 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc017070 con 0x7fc9e015e430 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.076+0000 7fc9d6ffd640 1 -- 172.21.15.172:0/415117409 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9dc005ce0 con 0x7fc9e015e430 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.076+0000 7fc9e8676640 1 -- 172.21.15.172:0/415117409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e00c9700 con 0x7fc9e015e430 2024-04-14T08:35:33.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.076+0000 7fc9d6ffd640 1 -- 172.21.15.172:0/415117409 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0072f0 con 0x7fc9e015e430 2024-04-14T08:35:33.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.076+0000 7fc9e8676640 1 -- 172.21.15.172:0/415117409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e00c9420 con 0x7fc9e015e430 2024-04-14T08:35:33.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.077+0000 7fc9e8676640 1 -- 172.21.15.172:0/415117409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e0068590 con 0x7fc9e015e430 2024-04-14T08:35:33.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.077+0000 7fc9d6ffd640 1 -- 172.21.15.172:0/415117409 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9dc01a070 con 0x7fc9e015e430 2024-04-14T08:35:33.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.078+0000 7fc9d6ffd640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc9c406c8f0 0x7fc9c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:33.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.078+0000 7fc9d6ffd640 1 -- 172.21.15.172:0/415117409 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5619+0+0 (secure 0 0 0) 0x7fc9dc08ae90 con 0x7fc9e015e430 2024-04-14T08:35:33.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.078+0000 7fc9e5c11640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc9c406c8f0 0x7fc9c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:33.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.078+0000 7fc9e5c11640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc9c406c8f0 0x7fc9c406edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0006fd0 tx=0x7fc9d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:33.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.082+0000 7fc9d6ffd640 1 -- 172.21.15.172:0/415117409 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc9e00c8ec0 con 0x7fc9e015e430 2024-04-14T08:35:33.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.259+0000 7fc9e8676640 1 -- 172.21.15.172:0/415117409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc9e00d18a0 con 0x7fc9e015e430 2024-04-14T08:35:33.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.952+0000 7fc9d6ffd640 1 -- 172.21.15.172:0/415117409 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) ==== 170+0+0 (secure 0 0 0) 0x7fc9dc0593a0 con 0x7fc9e015e430 2024-04-14T08:35:33.953 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' created 2024-04-14T08:35:33.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.955+0000 7fc9e8676640 1 -- 172.21.15.172:0/415117409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc9c406c8f0 msgr2=0x7fc9c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:33.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.955+0000 7fc9e8676640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc9c406c8f0 0x7fc9c406edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0006fd0 tx=0x7fc9d0008040 comp rx=0 tx=0).stop 2024-04-14T08:35:33.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.955+0000 7fc9e8676640 1 -- 172.21.15.172:0/415117409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9e015e430 msgr2=0x7fc9e014b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:33.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.955+0000 7fc9e8676640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9e015e430 0x7fc9e014b730 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00c390 tx=0x7fc9dc00c850 comp rx=0 tx=0).stop 2024-04-14T08:35:33.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.955+0000 7fc9e8676640 1 -- 172.21.15.172:0/415117409 shutdown_connections 2024-04-14T08:35:33.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.955+0000 7fc9e8676640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc9c406c8f0 0x7fc9c406edb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:33.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.955+0000 7fc9e8676640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9e0162ba0 0x7fc9e00b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:33.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.955+0000 7fc9e8676640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9e0162710 0x7fc9e00c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:33.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.955+0000 7fc9e8676640 1 --2- 172.21.15.172:0/415117409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9e015e430 0x7fc9e014b730 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:33.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.955+0000 7fc9e8676640 1 -- 172.21.15.172:0/415117409 >> 172.21.15.172:0/415117409 conn(0x7fc9e009d560 msgr2=0x7fc9e00b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:33.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.955+0000 7fc9e8676640 1 -- 172.21.15.172:0/415117409 shutdown_connections 2024-04-14T08:35:33.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:33.955+0000 7fc9e8676640 1 -- 172.21.15.172:0/415117409 wait complete. 2024-04-14T08:35:33.981 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-14T08:35:34.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.087+0000 7f77187bb640 1 Processor -- start 2024-04-14T08:35:34.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.087+0000 7f77187bb640 1 -- start start 2024-04-14T08:35:34.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.088+0000 7f77187bb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f771015e430 0x7f7710162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:34.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.088+0000 7f77187bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f77100d2e00 0x7f77101629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:34.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.088+0000 7f77187bb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77100cefb0 con 0x7f7710162540 2024-04-14T08:35:34.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.088+0000 7f77187bb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f77100cf290 con 0x7f77100d2e00 2024-04-14T08:35:34.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.088+0000 7f77187bb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f77100cf570 con 0x7f771015e430 2024-04-14T08:35:34.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.089+0000 7f7716d58640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f77100d2e00 0x7f77101629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:34.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.089+0000 7f7716d58640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f77100d2e00 0x7f77101629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42612/0 (socket says 172.21.15.172:42612) 2024-04-14T08:35:34.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.089+0000 7f7716d58640 1 -- 172.21.15.172:0/356739311 learned_addr learned my addr 172.21.15.172:0/356739311 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:34.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.089+0000 7f7716557640 1 --2- 172.21.15.172:0/356739311 >> v2:172.21.15.177:3300/0 conn(0x7f771015e430 0x7f7710162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:34.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.089+0000 7f7716d58640 1 -- 172.21.15.172:0/356739311 >> v2:172.21.15.177:3300/0 conn(0x7f771015e430 msgr2=0x7f7710162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:34.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.089+0000 7f7716d58640 1 --2- 172.21.15.172:0/356739311 >> v2:172.21.15.177:3300/0 conn(0x7f771015e430 0x7f7710162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:34.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.089+0000 7f7716d58640 1 -- 172.21.15.172:0/356739311 >> v1:172.21.15.177:6789/0 conn(0x7f7710162540 legacy=0x7f77100b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:34.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.089+0000 7f7716d58640 1 -- 172.21.15.172:0/356739311 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77100cf850 con 0x7f77100d2e00 2024-04-14T08:35:34.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.090+0000 7f7716d58640 1 --2- 172.21.15.172:0/356739311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77100d2e00 0x7f77101629d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f77100b6db0 tx=0x7f770002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33feffae048caaee server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:34.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.090+0000 7f7715555640 1 -- 172.21.15.172:0/356739311 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700034070 con 0x7f77100d2e00 2024-04-14T08:35:34.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.090+0000 7f7715555640 1 -- 172.21.15.172:0/356739311 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7700030860 con 0x7f77100d2e00 2024-04-14T08:35:34.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.090+0000 7f7715555640 1 -- 172.21.15.172:0/356739311 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700030c30 con 0x7f77100d2e00 2024-04-14T08:35:34.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.090+0000 7f77187bb640 1 -- 172.21.15.172:0/356739311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77100d2e00 msgr2=0x7f77101629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:34.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.090+0000 7f77187bb640 1 --2- 172.21.15.172:0/356739311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77100d2e00 0x7f77101629d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f77100b6db0 tx=0x7f770002b9a0 comp rx=0 tx=0).stop 2024-04-14T08:35:34.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.090+0000 7f77187bb640 1 -- 172.21.15.172:0/356739311 shutdown_connections 2024-04-14T08:35:34.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.090+0000 7f77187bb640 1 --2- 172.21.15.172:0/356739311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77100d2e00 0x7f77101629d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:34.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.090+0000 7f77187bb640 1 --2- 172.21.15.172:0/356739311 >> v2:172.21.15.177:3300/0 conn(0x7f771015e430 0x7f7710162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:34.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.090+0000 7f77187bb640 1 -- 172.21.15.172:0/356739311 >> 172.21.15.172:0/356739311 conn(0x7f771009d560 msgr2=0x7f77100d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:34.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.090+0000 7f77187bb640 1 -- 172.21.15.172:0/356739311 shutdown_connections 2024-04-14T08:35:34.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.090+0000 7f77187bb640 1 -- 172.21.15.172:0/356739311 wait complete. 2024-04-14T08:35:34.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.092+0000 7f77187bb640 1 Processor -- start 2024-04-14T08:35:34.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.092+0000 7f77187bb640 1 -- start start 2024-04-14T08:35:34.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.092+0000 7f77187bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77100d2e00 0x7f7710155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:34.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.092+0000 7f77187bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f771015e430 0x7f77100b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:34.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.092+0000 7f77187bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7710162540 0x7f77100b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:34.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.092+0000 7f77187bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f77100d4e20 con 0x7f771015e430 2024-04-14T08:35:34.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.092+0000 7f77187bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f771015e810 con 0x7f77100d2e00 2024-04-14T08:35:34.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.092+0000 7f77187bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f77100d0050 con 0x7f7710162540 2024-04-14T08:35:34.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.092+0000 7f7715d56640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f771015e430 0x7f77100b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:34.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.092+0000 7f7715d56640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f771015e430 0x7f77100b4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42624/0 (socket says 172.21.15.172:42624) 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.092+0000 7f7715d56640 1 -- 172.21.15.172:0/865190099 learned_addr learned my addr 172.21.15.172:0/865190099 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f7716557640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77100d2e00 0x7f7710155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f7716d58640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7710162540 0x7f77100b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f7715d56640 1 -- 172.21.15.172:0/865190099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7710162540 msgr2=0x7f77100b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f7715d56640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7710162540 0x7f77100b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f7715d56640 1 -- 172.21.15.172:0/865190099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77100d2e00 msgr2=0x7f7710155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f7715d56640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77100d2e00 0x7f7710155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f7715d56640 1 -- 172.21.15.172:0/865190099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77100b6d80 con 0x7f771015e430 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f7716557640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77100d2e00 0x7f7710155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f7715d56640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f771015e430 0x7f77100b4300 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f770400c430 tx=0x7f770400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f770effd640 1 -- 172.21.15.172:0/865190099 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704017070 con 0x7f771015e430 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f770effd640 1 -- 172.21.15.172:0/865190099 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77040052d0 con 0x7f771015e430 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f77187bb640 1 -- 172.21.15.172:0/865190099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77100b7060 con 0x7f771015e430 2024-04-14T08:35:34.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f770effd640 1 -- 172.21.15.172:0/865190099 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704003830 con 0x7f771015e430 2024-04-14T08:35:34.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.093+0000 7f77187bb640 1 -- 172.21.15.172:0/865190099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77100b7900 con 0x7f771015e430 2024-04-14T08:35:34.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.094+0000 7f77187bb640 1 -- 172.21.15.172:0/865190099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7710068590 con 0x7f771015e430 2024-04-14T08:35:34.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.095+0000 7f770effd640 1 -- 172.21.15.172:0/865190099 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f770401a070 con 0x7f771015e430 2024-04-14T08:35:34.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.095+0000 7f770effd640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f76fc06c8f0 0x7f76fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:34.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.095+0000 7f770effd640 1 -- 172.21.15.172:0/865190099 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6001+0+0 (secure 0 0 0) 0x7f770408afd0 con 0x7f771015e430 2024-04-14T08:35:34.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.095+0000 7f7716557640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f76fc06c8f0 0x7f76fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:34.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.095+0000 7f7716557640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f76fc06c8f0 0x7f76fc06edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7708007520 tx=0x7f770800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:34.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.100+0000 7f770effd640 1 -- 172.21.15.172:0/865190099 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f77040592d0 con 0x7f771015e430 2024-04-14T08:35:34.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.315+0000 7f77187bb640 1 -- 172.21.15.172:0/865190099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f77100b0ff0 con 0x7f771015e430 2024-04-14T08:35:34.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.954+0000 7f770effd640 1 -- 172.21.15.172:0/865190099 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v28) ==== 173+0+0 (secure 0 0 0) 0x7f7704058bd0 con 0x7f771015e430 2024-04-14T08:35:34.955 INFO:teuthology.orchestra.run.smithi172.stderr:new fs with metadata pool 9 and data pool 10 2024-04-14T08:35:34.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.957+0000 7f77187bb640 1 -- 172.21.15.172:0/865190099 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f76fc06c8f0 msgr2=0x7f76fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:34.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.957+0000 7f77187bb640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f76fc06c8f0 0x7f76fc06edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7708007520 tx=0x7f770800b040 comp rx=0 tx=0).stop 2024-04-14T08:35:34.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.957+0000 7f77187bb640 1 -- 172.21.15.172:0/865190099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f771015e430 msgr2=0x7f77100b4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:34.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.957+0000 7f77187bb640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f771015e430 0x7f77100b4300 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f770400c430 tx=0x7f770400c8f0 comp rx=0 tx=0).stop 2024-04-14T08:35:34.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.958+0000 7f77187bb640 1 -- 172.21.15.172:0/865190099 shutdown_connections 2024-04-14T08:35:34.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.958+0000 7f77187bb640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f76fc06c8f0 0x7f76fc06edb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:34.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.958+0000 7f77187bb640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7710162540 0x7f77100b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:34.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.958+0000 7f77187bb640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f771015e430 0x7f77100b4300 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:34.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.958+0000 7f77187bb640 1 --2- 172.21.15.172:0/865190099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77100d2e00 0x7f7710155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:34.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.958+0000 7f77187bb640 1 -- 172.21.15.172:0/865190099 >> 172.21.15.172:0/865190099 conn(0x7f771009d560 msgr2=0x7f77100b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:34.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.958+0000 7f77187bb640 1 -- 172.21.15.172:0/865190099 shutdown_connections 2024-04-14T08:35:34.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:34.959+0000 7f77187bb640 1 -- 172.21.15.172:0/865190099 wait complete. 2024-04-14T08:35:34.986 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:35:35.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.089+0000 7f0d203c8640 1 Processor -- start 2024-04-14T08:35:35.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.089+0000 7f0d203c8640 1 -- start start 2024-04-14T08:35:35.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.090+0000 7f0d203c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0d180b1df0 0x7f0d180b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.090+0000 7f0d203c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d1815c770 con 0x7f0d180b21d0 2024-04-14T08:35:35.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.090+0000 7f0d203c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d180b3430 con 0x7f0d180b4040 2024-04-14T08:35:35.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.090+0000 7f0d203c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f0d180b3700 con 0x7f0d180b1df0 2024-04-14T08:35:35.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.090+0000 7f0d1d963640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f0d180b21d0 0x7f0d1815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59866/0 (socket says 172.21.15.172:59866) 2024-04-14T08:35:35.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.090+0000 7f0d1d963640 1 -- 172.21.15.172:0/3446988929 learned_addr learned my addr 172.21.15.172:0/3446988929 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:35.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.090+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1981261010 0 0) 0x7f0d1815c770 con 0x7f0d180b21d0 2024-04-14T08:35:35.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.090+0000 7f0d1e164640 1 --2- 172.21.15.172:0/3446988929 >> v2:172.21.15.177:3300/0 conn(0x7f0d180b1df0 0x7f0d180b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.090+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d04003520 con 0x7f0d180b21d0 2024-04-14T08:35:35.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 990856347 0 0) 0x7f0d04003520 con 0x7f0d180b21d0 2024-04-14T08:35:35.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 >> v2:172.21.15.177:3300/0 conn(0x7f0d180b1df0 msgr2=0x7f0d180b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:35.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 --2- 172.21.15.172:0/3446988929 >> v2:172.21.15.177:3300/0 conn(0x7f0d180b1df0 0x7f0d180b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 >> v1:172.21.15.177:6789/0 conn(0x7f0d180b4040 legacy=0x7f0d180b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d180b39e0 con 0x7f0d180b21d0 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f0d08001670 con 0x7f0d180b21d0 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 >> v1:172.21.15.172:6789/0 conn(0x7f0d180b21d0 legacy=0x7f0d1815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 --2- 172.21.15.172:0/3446988929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d040039b0 0x7f0d04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 --2- 172.21.15.172:0/3446988929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d04004800 0x7f0d04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 --2- 172.21.15.172:0/3446988929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d040051c0 0x7f0d0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d180b3700 con 0x7f0d040051c0 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0400c060 con 0x7f0d040039b0 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0d0400c260 con 0x7f0d04004800 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1e965640 1 --2- 172.21.15.172:0/3446988929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d040051c0 0x7f0d0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1e164640 1 --2- 172.21.15.172:0/3446988929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d040039b0 0x7f0d04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1d963640 1 --2- 172.21.15.172:0/3446988929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d04004800 0x7f0d04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1e965640 1 -- 172.21.15.172:0/3446988929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d04004800 msgr2=0x7f0d04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:35.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1e965640 1 --2- 172.21.15.172:0/3446988929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d04004800 0x7f0d04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1e965640 1 -- 172.21.15.172:0/3446988929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d040039b0 msgr2=0x7f0d04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:35.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1e965640 1 --2- 172.21.15.172:0/3446988929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d040039b0 0x7f0d04003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.091+0000 7f0d1e965640 1 -- 172.21.15.172:0/3446988929 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d04003520 con 0x7f0d040051c0 2024-04-14T08:35:35.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.092+0000 7f0d1e965640 1 -- 172.21.15.172:0/3446988929 reap_dead start 2024-04-14T08:35:35.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.092+0000 7f0d1e965640 1 --2- 172.21.15.172:0/3446988929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d040051c0 0x7f0d0400b9c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0d10001750 tx=0x7f0d04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62c4fc9f5a74d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:35.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.092+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10001370 con 0x7f0d040051c0 2024-04-14T08:35:35.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.092+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d100052f0 con 0x7f0d040051c0 2024-04-14T08:35:35.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.092+0000 7f0d1d162640 1 -- 172.21.15.172:0/3446988929 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10003830 con 0x7f0d040051c0 2024-04-14T08:35:35.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.093+0000 7f0d203c8640 1 -- 172.21.15.172:0/3446988929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d040051c0 msgr2=0x7f0d0400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.093+0000 7f0d203c8640 1 --2- 172.21.15.172:0/3446988929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d040051c0 0x7f0d0400b9c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0d10001750 tx=0x7f0d04005830 comp rx=0 tx=0).stop 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.093+0000 7f0d203c8640 1 -- 172.21.15.172:0/3446988929 shutdown_connections 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.093+0000 7f0d203c8640 1 --2- 172.21.15.172:0/3446988929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d040051c0 0x7f0d0400b9c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.093+0000 7f0d203c8640 1 -- 172.21.15.172:0/3446988929 >> 172.21.15.172:0/3446988929 conn(0x7f0d1806ec30 msgr2=0x7f0d1806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.093+0000 7f0d203c8640 1 -- 172.21.15.172:0/3446988929 shutdown_connections 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.093+0000 7f0d203c8640 1 -- 172.21.15.172:0/3446988929 wait complete. 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.094+0000 7f0d203c8640 1 Processor -- start 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.094+0000 7f0d203c8640 1 -- start start 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d203c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d040051c0 0x7f0d1813b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d203c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d1813dbf0 0x7f0d180d0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d203c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d181592a0 0x7f0d18159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d203c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1815b7f0 con 0x7f0d1813dbf0 2024-04-14T08:35:35.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d203c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1815bb40 con 0x7f0d181592a0 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d203c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0d180b2ef0 con 0x7f0d040051c0 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d1d963640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d1813dbf0 0x7f0d180d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d1d963640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d1813dbf0 0x7f0d180d0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42646/0 (socket says 172.21.15.172:42646) 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d1d963640 1 -- 172.21.15.172:0/2646759961 learned_addr learned my addr 172.21.15.172:0/2646759961 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d1e164640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d040051c0 0x7f0d1813b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d1e965640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d181592a0 0x7f0d18159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d1d963640 1 -- 172.21.15.172:0/2646759961 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d040051c0 msgr2=0x7f0d1813b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d1d963640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d040051c0 0x7f0d1813b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d1d963640 1 -- 172.21.15.172:0/2646759961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d181592a0 msgr2=0x7f0d18159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d1d963640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d181592a0 0x7f0d18159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d1d963640 1 -- 172.21.15.172:0/2646759961 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d180cf680 con 0x7f0d1813dbf0 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d1e965640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d181592a0 0x7f0d18159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:35.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d1d963640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d1813dbf0 0x7f0d180d0740 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0d080072f0 tx=0x7f0d080077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d0effd640 1 -- 172.21.15.172:0/2646759961 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0800f960 con 0x7f0d1813dbf0 2024-04-14T08:35:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d0effd640 1 -- 172.21.15.172:0/2646759961 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d08018070 con 0x7f0d1813dbf0 2024-04-14T08:35:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d203c8640 1 -- 172.21.15.172:0/2646759961 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1813e070 con 0x7f0d1813dbf0 2024-04-14T08:35:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.095+0000 7f0d0effd640 1 -- 172.21.15.172:0/2646759961 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0800fd40 con 0x7f0d1813dbf0 2024-04-14T08:35:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.096+0000 7f0d203c8640 1 -- 172.21.15.172:0/2646759961 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d180cfbe0 con 0x7f0d1813dbf0 2024-04-14T08:35:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.097+0000 7f0d203c8640 1 -- 172.21.15.172:0/2646759961 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d18068520 con 0x7f0d1813dbf0 2024-04-14T08:35:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.097+0000 7f0d0effd640 1 -- 172.21.15.172:0/2646759961 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d0801d460 con 0x7f0d1813dbf0 2024-04-14T08:35:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.097+0000 7f0d0effd640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0cfc06c8f0 0x7f0cfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.097+0000 7f0d0effd640 1 -- 172.21.15.172:0/2646759961 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f0d0808bd60 con 0x7f0d1813dbf0 2024-04-14T08:35:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.097+0000 7f0d1e164640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0cfc06c8f0 0x7f0cfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.098+0000 7f0d1e164640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0cfc06c8f0 0x7f0cfc06edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009fd0 tx=0x7f0d1400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:35.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.102+0000 7f0d0effd640 1 -- 172.21.15.172:0/2646759961 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0d180cf680 con 0x7f0d1813dbf0 2024-04-14T08:35:35.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.281+0000 7f0d203c8640 1 -- 172.21.15.172:0/2646759961 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d1813d3e0 con 0x7f0d1813dbf0 2024-04-14T08:35:35.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.282+0000 7f0d0effd640 1 -- 172.21.15.172:0/2646759961 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16957 (secure 0 0 0) 0x7f0d0805a000 con 0x7f0d1813dbf0 2024-04-14T08:35:35.283 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:35.283 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":60,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:35:34.316237+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:35:32.539067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-14T08:35:33.263123+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:35:35.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.284+0000 7f0d203c8640 1 -- 172.21.15.172:0/2646759961 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0cfc06c8f0 msgr2=0x7f0cfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.284+0000 7f0d203c8640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0cfc06c8f0 0x7f0cfc06edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009fd0 tx=0x7f0d1400b040 comp rx=0 tx=0).stop 2024-04-14T08:35:35.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.284+0000 7f0d203c8640 1 -- 172.21.15.172:0/2646759961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d1813dbf0 msgr2=0x7f0d180d0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.284+0000 7f0d203c8640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d1813dbf0 0x7f0d180d0740 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0d080072f0 tx=0x7f0d080077b0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.284+0000 7f0d203c8640 1 -- 172.21.15.172:0/2646759961 shutdown_connections 2024-04-14T08:35:35.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.284+0000 7f0d203c8640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0cfc06c8f0 0x7f0cfc06edb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.284+0000 7f0d203c8640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d181592a0 0x7f0d18159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.284+0000 7f0d203c8640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d1813dbf0 0x7f0d180d0740 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.284+0000 7f0d203c8640 1 --2- 172.21.15.172:0/2646759961 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d040051c0 0x7f0d1813b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.284+0000 7f0d203c8640 1 -- 172.21.15.172:0/2646759961 >> 172.21.15.172:0/2646759961 conn(0x7f0d1806ec30 msgr2=0x7f0d1815a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:35.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.284+0000 7f0d203c8640 1 -- 172.21.15.172:0/2646759961 shutdown_connections 2024-04-14T08:35:35.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.284+0000 7f0d203c8640 1 -- 172.21.15.172:0/2646759961 wait complete. 2024-04-14T08:35:35.296 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:35:35.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.397+0000 7fb979988640 1 Processor -- start 2024-04-14T08:35:35.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.397+0000 7fb979988640 1 -- start start 2024-04-14T08:35:35.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.398+0000 7fb979988640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb97415e890 0x7fb9740d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.398+0000 7fb979988640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9740cf320 con 0x7fb9740d2830 2024-04-14T08:35:35.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.398+0000 7fb979988640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9740cf600 con 0x7fb9740cedd0 2024-04-14T08:35:35.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.398+0000 7fb979988640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb9740cff30 con 0x7fb97415e890 2024-04-14T08:35:35.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.398+0000 7fb9727fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb97415e890 0x7fb9740d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.398+0000 7fb972ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb9740d2830 0x7fb9740d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59878/0 (socket says 172.21.15.172:59878) 2024-04-14T08:35:35.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.398+0000 7fb972ffd640 1 -- 172.21.15.172:0/3835836453 learned_addr learned my addr 172.21.15.172:0/3835836453 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:35.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.399+0000 7fb971ffb640 1 -- 172.21.15.172:0/3835836453 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3686548431 0 0) 0x7fb9740cf320 con 0x7fb9740d2830 2024-04-14T08:35:35.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.399+0000 7fb971ffb640 1 -- 172.21.15.172:0/3835836453 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb95c003520 con 0x7fb9740d2830 2024-04-14T08:35:35.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.399+0000 7fb9727fc640 1 -- 172.21.15.172:0/3835836453 >> v1:172.21.15.177:6789/0 conn(0x7fb9740cedd0 legacy=0x7fb9740b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.399+0000 7fb9727fc640 1 -- 172.21.15.172:0/3835836453 >> v1:172.21.15.172:6789/0 conn(0x7fb9740d2830 legacy=0x7fb9740d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.399+0000 7fb9727fc640 1 -- 172.21.15.172:0/3835836453 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9740cf8e0 con 0x7fb97415e890 2024-04-14T08:35:35.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.399+0000 7fb9727fc640 1 --2- 172.21.15.172:0/3835836453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb97415e890 0x7fb9740d1300 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb9680097a0 tx=0x7fb96802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3cab2d7f253713 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:35.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.399+0000 7fb971ffb640 1 -- 172.21.15.172:0/3835836453 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96802f810 con 0x7fb97415e890 2024-04-14T08:35:35.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.400+0000 7fb971ffb640 1 -- 172.21.15.172:0/3835836453 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96802f9b0 con 0x7fb97415e890 2024-04-14T08:35:35.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.400+0000 7fb971ffb640 1 -- 172.21.15.172:0/3835836453 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96802fd80 con 0x7fb97415e890 2024-04-14T08:35:35.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.400+0000 7fb979988640 1 -- 172.21.15.172:0/3835836453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb97415e890 msgr2=0x7fb9740d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.400+0000 7fb979988640 1 --2- 172.21.15.172:0/3835836453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb97415e890 0x7fb9740d1300 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb9680097a0 tx=0x7fb96802b9f0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.400+0000 7fb979988640 1 -- 172.21.15.172:0/3835836453 shutdown_connections 2024-04-14T08:35:35.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.400+0000 7fb979988640 1 --2- 172.21.15.172:0/3835836453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb97415e890 0x7fb9740d1300 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.400+0000 7fb979988640 1 -- 172.21.15.172:0/3835836453 >> 172.21.15.172:0/3835836453 conn(0x7fb97409d590 msgr2=0x7fb9740d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.400+0000 7fb979988640 1 -- 172.21.15.172:0/3835836453 shutdown_connections 2024-04-14T08:35:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.400+0000 7fb979988640 1 -- 172.21.15.172:0/3835836453 wait complete. 2024-04-14T08:35:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.401+0000 7fb979988640 1 Processor -- start 2024-04-14T08:35:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.401+0000 7fb979988640 1 -- start start 2024-04-14T08:35:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb979988640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9740cedd0 0x7fb9740cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb979988640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9740d2830 0x7fb9740ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb979988640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb97415e890 0x7fb974152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb979988640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb9740d1840 con 0x7fb97415e890 2024-04-14T08:35:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb979988640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb9740d2350 con 0x7fb9740cedd0 2024-04-14T08:35:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb979988640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb9740b3e20 con 0x7fb9740d2830 2024-04-14T08:35:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb9737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb97415e890 0x7fb974152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb9737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb97415e890 0x7fb974152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42678/0 (socket says 172.21.15.172:42678) 2024-04-14T08:35:35.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb9737fe640 1 -- 172.21.15.172:0/1432975413 learned_addr learned my addr 172.21.15.172:0/1432975413 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:35.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb9727fc640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9740d2830 0x7fb9740ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb972ffd640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9740cedd0 0x7fb9740cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb9737fe640 1 -- 172.21.15.172:0/1432975413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9740d2830 msgr2=0x7fb9740ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.402+0000 7fb9737fe640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9740d2830 0x7fb9740ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.403+0000 7fb9737fe640 1 -- 172.21.15.172:0/1432975413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9740cedd0 msgr2=0x7fb9740cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.403+0000 7fb9737fe640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9740cedd0 0x7fb9740cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.403+0000 7fb9737fe640 1 -- 172.21.15.172:0/1432975413 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9740cabe0 con 0x7fb97415e890 2024-04-14T08:35:35.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.403+0000 7fb9727fc640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9740d2830 0x7fb9740ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:35.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.403+0000 7fb972ffd640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9740cedd0 0x7fb9740cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:35.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.403+0000 7fb9737fe640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb97415e890 0x7fb974152050 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb96c001eb0 tx=0x7fb96c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:35.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.403+0000 7fb95bfff640 1 -- 172.21.15.172:0/1432975413 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c017070 con 0x7fb97415e890 2024-04-14T08:35:35.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.403+0000 7fb95bfff640 1 -- 172.21.15.172:0/1432975413 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96c0052d0 con 0x7fb97415e890 2024-04-14T08:35:35.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.403+0000 7fb979988640 1 -- 172.21.15.172:0/1432975413 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97415f720 con 0x7fb97415e890 2024-04-14T08:35:35.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.403+0000 7fb95bfff640 1 -- 172.21.15.172:0/1432975413 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c003810 con 0x7fb97415e890 2024-04-14T08:35:35.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.403+0000 7fb979988640 1 -- 172.21.15.172:0/1432975413 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97415fce0 con 0x7fb97415e890 2024-04-14T08:35:35.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.404+0000 7fb979988640 1 -- 172.21.15.172:0/1432975413 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb974068520 con 0x7fb97415e890 2024-04-14T08:35:35.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.404+0000 7fb95bfff640 1 -- 172.21.15.172:0/1432975413 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb96c01a070 con 0x7fb97415e890 2024-04-14T08:35:35.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.405+0000 7fb95bfff640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb96006c8f0 0x7fb96006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.405+0000 7fb972ffd640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb96006c8f0 0x7fb96006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.405+0000 7fb95bfff640 1 -- 172.21.15.172:0/1432975413 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fb96c08b040 con 0x7fb97415e890 2024-04-14T08:35:35.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.405+0000 7fb972ffd640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb96006c8f0 0x7fb96006edb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb964001560 tx=0x7fb964009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:35.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.409+0000 7fb95bfff640 1 -- 172.21.15.172:0/1432975413 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb96c0592e0 con 0x7fb97415e890 2024-04-14T08:35:35.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.587+0000 7fb979988640 1 -- 172.21.15.172:0/1432975413 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9740d2610 con 0x7fb97415e890 2024-04-14T08:35:35.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.588+0000 7fb95bfff640 1 -- 172.21.15.172:0/1432975413 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16957 (secure 0 0 0) 0x7fb96c058be0 con 0x7fb97415e890 2024-04-14T08:35:35.589 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:35.589 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":60,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:35:34.316237+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:35:32.539067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-14T08:35:33.263123+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:35:35.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.590+0000 7fb979988640 1 -- 172.21.15.172:0/1432975413 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb96006c8f0 msgr2=0x7fb96006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.590+0000 7fb979988640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb96006c8f0 0x7fb96006edb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb964001560 tx=0x7fb964009040 comp rx=0 tx=0).stop 2024-04-14T08:35:35.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.590+0000 7fb979988640 1 -- 172.21.15.172:0/1432975413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb97415e890 msgr2=0x7fb974152050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.590+0000 7fb979988640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb97415e890 0x7fb974152050 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb96c001eb0 tx=0x7fb96c00c890 comp rx=0 tx=0).stop 2024-04-14T08:35:35.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.590+0000 7fb979988640 1 -- 172.21.15.172:0/1432975413 shutdown_connections 2024-04-14T08:35:35.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.590+0000 7fb979988640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb96006c8f0 0x7fb96006edb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.590+0000 7fb979988640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb97415e890 0x7fb974152050 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.590+0000 7fb979988640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9740d2830 0x7fb9740ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.590+0000 7fb979988640 1 --2- 172.21.15.172:0/1432975413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9740cedd0 0x7fb9740cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.590+0000 7fb979988640 1 -- 172.21.15.172:0/1432975413 >> 172.21.15.172:0/1432975413 conn(0x7fb97409d590 msgr2=0x7fb9740b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:35.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.591+0000 7fb979988640 1 -- 172.21.15.172:0/1432975413 shutdown_connections 2024-04-14T08:35:35.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.591+0000 7fb979988640 1 -- 172.21.15.172:0/1432975413 wait complete. 2024-04-14T08:35:35.602 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-14T08:35:35.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.706+0000 7f84dd55e640 1 Processor -- start 2024-04-14T08:35:35.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.706+0000 7f84dd55e640 1 -- start start 2024-04-14T08:35:35.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.707+0000 7f84dd55e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f84d8161b70 0x7f84d80d25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.707+0000 7f84dd55e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d80d10d0 0x7f84d80cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.707+0000 7f84dd55e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d80b3bb0 con 0x7f84d80d0c40 2024-04-14T08:35:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.707+0000 7f84dd55e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f84d80d2bd0 con 0x7f84d8161b70 2024-04-14T08:35:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.707+0000 7f84dd55e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f84d80d2eb0 con 0x7f84d80d10d0 2024-04-14T08:35:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.708+0000 7f84d77fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d80d10d0 0x7f84d80cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.708+0000 7f84d6ffd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f84d8161b70 0x7f84d80d25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.708+0000 7f84d77fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d80d10d0 0x7f84d80cfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:42182/0 (socket says 172.21.15.172:42182) 2024-04-14T08:35:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.708+0000 7f84d77fe640 1 -- 172.21.15.172:0/1831905075 learned_addr learned my addr 172.21.15.172:0/1831905075 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.708+0000 7f84cffff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f84d80d0c40 0x7f84d80b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:44160/0 (socket says 172.21.15.172:44160) 2024-04-14T08:35:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.708+0000 7f84d77fe640 1 -- 172.21.15.172:0/1831905075 >> v2:172.21.15.177:3300/0 conn(0x7f84d8161b70 msgr2=0x7f84d80d25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.708+0000 7f84d77fe640 1 --2- 172.21.15.172:0/1831905075 >> v2:172.21.15.177:3300/0 conn(0x7f84d8161b70 0x7f84d80d25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.708+0000 7f84d77fe640 1 -- 172.21.15.172:0/1831905075 >> v1:172.21.15.177:6789/0 conn(0x7f84d80d0c40 legacy=0x7f84d80b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.708+0000 7f84d77fe640 1 -- 172.21.15.172:0/1831905075 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d80d3190 con 0x7f84d80d10d0 2024-04-14T08:35:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.708+0000 7f84d6ffd640 1 --2- 172.21.15.172:0/1831905075 >> v2:172.21.15.177:3300/0 conn(0x7f84d8161b70 0x7f84d80d25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:35:35.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.709+0000 7f84d77fe640 1 --2- 172.21.15.172:0/1831905075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d80d10d0 0x7f84d80cfe50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009e50 tx=0x7f84d002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc71123ed509584a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:35.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.709+0000 7f84d67fc640 1 -- 172.21.15.172:0/1831905075 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d002cb00 con 0x7f84d80d10d0 2024-04-14T08:35:35.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.709+0000 7f84d67fc640 1 -- 172.21.15.172:0/1831905075 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d002cca0 con 0x7f84d80d10d0 2024-04-14T08:35:35.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.709+0000 7f84d67fc640 1 -- 172.21.15.172:0/1831905075 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0034610 con 0x7f84d80d10d0 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.709+0000 7f84dd55e640 1 -- 172.21.15.172:0/1831905075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d80d10d0 msgr2=0x7f84d80cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.709+0000 7f84dd55e640 1 --2- 172.21.15.172:0/1831905075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d80d10d0 0x7f84d80cfe50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009e50 tx=0x7f84d002bf60 comp rx=0 tx=0).stop 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.709+0000 7f84dd55e640 1 -- 172.21.15.172:0/1831905075 shutdown_connections 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.709+0000 7f84dd55e640 1 --2- 172.21.15.172:0/1831905075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d80d10d0 0x7f84d80cfe50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.709+0000 7f84dd55e640 1 --2- 172.21.15.172:0/1831905075 >> v2:172.21.15.177:3300/0 conn(0x7f84d8161b70 0x7f84d80d25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.709+0000 7f84dd55e640 1 -- 172.21.15.172:0/1831905075 >> 172.21.15.172:0/1831905075 conn(0x7f84d809d580 msgr2=0x7f84d80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.709+0000 7f84dd55e640 1 -- 172.21.15.172:0/1831905075 shutdown_connections 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.709+0000 7f84dd55e640 1 -- 172.21.15.172:0/1831905075 wait complete. 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.711+0000 7f84dd55e640 1 Processor -- start 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.711+0000 7f84dd55e640 1 -- start start 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.711+0000 7f84dd55e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84d80d0c40 0x7f84d80b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.711+0000 7f84dd55e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d80d10d0 0x7f84d806c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.711+0000 7f84dd55e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d8161b70 0x7f84d814f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.711+0000 7f84dd55e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8161f50 con 0x7f84d80d0c40 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.711+0000 7f84dd55e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f84d80d14d0 con 0x7f84d80d10d0 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.711+0000 7f84dd55e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f84d80b6e10 con 0x7f84d8161b70 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.711+0000 7f84d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84d80d0c40 0x7f84d80b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.711+0000 7f84d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84d80d0c40 0x7f84d80b7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42688/0 (socket says 172.21.15.172:42688) 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.711+0000 7f84d6ffd640 1 -- 172.21.15.172:0/1820243656 learned_addr learned my addr 172.21.15.172:0/1820243656 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84d77fe640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d8161b70 0x7f84d814f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84cffff640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d80d10d0 0x7f84d806c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84d6ffd640 1 -- 172.21.15.172:0/1820243656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d8161b70 msgr2=0x7f84d814f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84d6ffd640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d8161b70 0x7f84d814f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84d6ffd640 1 -- 172.21.15.172:0/1820243656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d80d10d0 msgr2=0x7f84d806c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84d6ffd640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d80d10d0 0x7f84d806c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84d6ffd640 1 -- 172.21.15.172:0/1820243656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d80b7060 con 0x7f84d80d0c40 2024-04-14T08:35:35.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84cffff640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d80d10d0 0x7f84d806c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:35:35.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84d6ffd640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84d80d0c40 0x7f84d80b7e20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f84c800e960 tx=0x7f84c800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:35.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84cf7fe640 1 -- 172.21.15.172:0/1820243656 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8017070 con 0x7f84d80d0c40 2024-04-14T08:35:35.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84cf7fe640 1 -- 172.21.15.172:0/1820243656 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c800ca00 con 0x7f84d80d0c40 2024-04-14T08:35:35.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84dd55e640 1 -- 172.21.15.172:0/1820243656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d80b7340 con 0x7f84d80d0c40 2024-04-14T08:35:35.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84cf7fe640 1 -- 172.21.15.172:0/1820243656 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c800cdd0 con 0x7f84d80d0c40 2024-04-14T08:35:35.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.712+0000 7f84dd55e640 1 -- 172.21.15.172:0/1820243656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d80b7900 con 0x7f84d80d0c40 2024-04-14T08:35:35.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.713+0000 7f84dd55e640 1 -- 172.21.15.172:0/1820243656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d8068520 con 0x7f84d80d0c40 2024-04-14T08:35:35.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.714+0000 7f84cf7fe640 1 -- 172.21.15.172:0/1820243656 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84c801a070 con 0x7f84d80d0c40 2024-04-14T08:35:35.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.714+0000 7f84cf7fe640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84b806c8f0 0x7f84b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:35.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.714+0000 7f84cf7fe640 1 -- 172.21.15.172:0/1820243656 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7f84c808bec0 con 0x7f84d80d0c40 2024-04-14T08:35:35.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.714+0000 7f84cffff640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84b806c8f0 0x7f84b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:35.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.714+0000 7f84cffff640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84b806c8f0 0x7f84b806edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f84d80d0bb0 tx=0x7f84c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:35.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.718+0000 7f84cf7fe640 1 -- 172.21.15.172:0/1820243656 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f84c805a1f0 con 0x7f84d80d0c40 2024-04-14T08:35:35.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.931+0000 7f84dd55e640 1 -- 172.21.15.172:0/1820243656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f84d80b8420 con 0x7f84d80d0c40 2024-04-14T08:35:35.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.970+0000 7f84cf7fe640 1 -- 172.21.15.172:0/1820243656 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f84c8059af0 con 0x7f84d80d0c40 2024-04-14T08:35:35.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.973+0000 7f84dd55e640 1 -- 172.21.15.172:0/1820243656 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84b806c8f0 msgr2=0x7f84b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.973+0000 7f84dd55e640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84b806c8f0 0x7f84b806edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f84d80d0bb0 tx=0x7f84c0009040 comp rx=0 tx=0).stop 2024-04-14T08:35:35.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.973+0000 7f84dd55e640 1 -- 172.21.15.172:0/1820243656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84d80d0c40 msgr2=0x7f84d80b7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:35.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.973+0000 7f84dd55e640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84d80d0c40 0x7f84d80b7e20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f84c800e960 tx=0x7f84c800ee20 comp rx=0 tx=0).stop 2024-04-14T08:35:35.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.974+0000 7f84dd55e640 1 -- 172.21.15.172:0/1820243656 shutdown_connections 2024-04-14T08:35:35.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.974+0000 7f84dd55e640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84b806c8f0 0x7f84b806edb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.974+0000 7f84dd55e640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d8161b70 0x7f84d814f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.974+0000 7f84dd55e640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d80d10d0 0x7f84d806c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.974+0000 7f84dd55e640 1 --2- 172.21.15.172:0/1820243656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84d80d0c40 0x7f84d80b7e20 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:35.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.974+0000 7f84dd55e640 1 -- 172.21.15.172:0/1820243656 >> 172.21.15.172:0/1820243656 conn(0x7f84d809d580 msgr2=0x7f84d815e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:35.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.974+0000 7f84dd55e640 1 -- 172.21.15.172:0/1820243656 shutdown_connections 2024-04-14T08:35:35.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:35.974+0000 7f84dd55e640 1 -- 172.21.15.172:0/1820243656 wait complete. 2024-04-14T08:35:35.985 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-14T08:35:35.986 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-14T08:35:36.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.089+0000 7ff6239bb640 1 Processor -- start 2024-04-14T08:35:36.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.089+0000 7ff6239bb640 1 -- start start 2024-04-14T08:35:36.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.090+0000 7ff6239bb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff61c15c220 0x7ff61c0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:36.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.090+0000 7ff6239bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff61c0d0770 0x7ff61c06b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:36.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.090+0000 7ff6239bb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff61c15c600 con 0x7ff61c06ba60 2024-04-14T08:35:36.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.090+0000 7ff6239bb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff61c160cc0 con 0x7ff61c0d0770 2024-04-14T08:35:36.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.090+0000 7ff6239bb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff61c1600b0 con 0x7ff61c15c220 2024-04-14T08:35:36.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.090+0000 7ff620f56640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff61c0d0770 0x7ff61c06b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:36.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.090+0000 7ff621f58640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff61c06ba60 0x7ff61c072670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59886/0 (socket says 172.21.15.172:59886) 2024-04-14T08:35:36.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.090+0000 7ff621f58640 1 -- 172.21.15.172:0/3316022118 learned_addr learned my addr 172.21.15.172:0/3316022118 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:36.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.090+0000 7ff621757640 1 --2- 172.21.15.172:0/3316022118 >> v2:172.21.15.177:3300/0 conn(0x7ff61c15c220 0x7ff61c0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:36.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.090+0000 7ff620f56640 1 -- 172.21.15.172:0/3316022118 >> v2:172.21.15.177:3300/0 conn(0x7ff61c15c220 msgr2=0x7ff61c0addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:36.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.090+0000 7ff620f56640 1 --2- 172.21.15.172:0/3316022118 >> v2:172.21.15.177:3300/0 conn(0x7ff61c15c220 0x7ff61c0addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:36.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.090+0000 7ff620f56640 1 -- 172.21.15.172:0/3316022118 >> v1:172.21.15.172:6789/0 conn(0x7ff61c06ba60 legacy=0x7ff61c072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:36.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.091+0000 7ff613fff640 1 -- 172.21.15.172:0/3316022118 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3440079602 0 0) 0x7ff61c15c600 con 0x7ff61c06ba60 2024-04-14T08:35:36.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.091+0000 7ff620f56640 1 -- 172.21.15.172:0/3316022118 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c160390 con 0x7ff61c0d0770 2024-04-14T08:35:36.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.091+0000 7ff620f56640 1 --2- 172.21.15.172:0/3316022118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff61c0d0770 0x7ff61c06b520 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff61c06d840 tx=0x7ff60c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e529292ecdfc7023 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.091+0000 7ff613fff640 1 -- 172.21.15.172:0/3316022118 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c035020 con 0x7ff61c0d0770 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.091+0000 7ff613fff640 1 -- 172.21.15.172:0/3316022118 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60c031830 con 0x7ff61c0d0770 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.091+0000 7ff613fff640 1 -- 172.21.15.172:0/3316022118 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c031c00 con 0x7ff61c0d0770 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.092+0000 7ff6239bb640 1 -- 172.21.15.172:0/3316022118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff61c0d0770 msgr2=0x7ff61c06b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.092+0000 7ff6239bb640 1 --2- 172.21.15.172:0/3316022118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff61c0d0770 0x7ff61c06b520 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff61c06d840 tx=0x7ff60c02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.092+0000 7ff6239bb640 1 -- 172.21.15.172:0/3316022118 shutdown_connections 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.092+0000 7ff6239bb640 1 --2- 172.21.15.172:0/3316022118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff61c0d0770 0x7ff61c06b520 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.092+0000 7ff6239bb640 1 --2- 172.21.15.172:0/3316022118 >> v2:172.21.15.177:3300/0 conn(0x7ff61c15c220 0x7ff61c0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.092+0000 7ff6239bb640 1 -- 172.21.15.172:0/3316022118 >> 172.21.15.172:0/3316022118 conn(0x7ff61c0cd980 msgr2=0x7ff61c0cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.092+0000 7ff6239bb640 1 -- 172.21.15.172:0/3316022118 shutdown_connections 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.092+0000 7ff6239bb640 1 -- 172.21.15.172:0/3316022118 wait complete. 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.093+0000 7ff6239bb640 1 Processor -- start 2024-04-14T08:35:36.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.093+0000 7ff6239bb640 1 -- start start 2024-04-14T08:35:36.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff6239bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff61c06ba60 0x7ff61c06f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:36.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff6239bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff61c0d0770 0x7ff61c14c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:36.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff6239bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff61c15c220 0x7ff61c0b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:36.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff6239bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c15fda0 con 0x7ff61c0d0770 2024-04-14T08:35:36.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff6239bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c160950 con 0x7ff61c06ba60 2024-04-14T08:35:36.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff6239bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff61c0cb780 con 0x7ff61c15c220 2024-04-14T08:35:36.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff620f56640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff61c0d0770 0x7ff61c14c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:36.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff620f56640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff61c0d0770 0x7ff61c14c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42702/0 (socket says 172.21.15.172:42702) 2024-04-14T08:35:36.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff620f56640 1 -- 172.21.15.172:0/1326693349 learned_addr learned my addr 172.21.15.172:0/1326693349 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff621f58640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff61c15c220 0x7ff61c0b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff621757640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff61c06ba60 0x7ff61c06f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff620f56640 1 -- 172.21.15.172:0/1326693349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff61c15c220 msgr2=0x7ff61c0b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff620f56640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff61c15c220 0x7ff61c0b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff620f56640 1 -- 172.21.15.172:0/1326693349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff61c06ba60 msgr2=0x7ff61c06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff620f56640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff61c06ba60 0x7ff61c06f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff620f56640 1 -- 172.21.15.172:0/1326693349 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c06e680 con 0x7ff61c0d0770 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.094+0000 7ff620f56640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff61c0d0770 0x7ff61c14c450 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009890 tx=0x7ff60c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.095+0000 7ff611ffb640 1 -- 172.21.15.172:0/1326693349 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c041070 con 0x7ff61c0d0770 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.095+0000 7ff611ffb640 1 -- 172.21.15.172:0/1326693349 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60c03b920 con 0x7ff61c0d0770 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.095+0000 7ff6239bb640 1 -- 172.21.15.172:0/1326693349 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61c06e960 con 0x7ff61c0d0770 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.095+0000 7ff611ffb640 1 -- 172.21.15.172:0/1326693349 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c035040 con 0x7ff61c0d0770 2024-04-14T08:35:36.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.095+0000 7ff6239bb640 1 -- 172.21.15.172:0/1326693349 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61c06ef20 con 0x7ff61c0d0770 2024-04-14T08:35:36.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.096+0000 7ff6239bb640 1 -- 172.21.15.172:0/1326693349 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61c068520 con 0x7ff61c0d0770 2024-04-14T08:35:36.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.096+0000 7ff611ffb640 1 -- 172.21.15.172:0/1326693349 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff60c038020 con 0x7ff61c0d0770 2024-04-14T08:35:36.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.097+0000 7ff611ffb640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff60006c8f0 0x7ff60006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:36.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.097+0000 7ff611ffb640 1 -- 172.21.15.172:0/1326693349 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7ff60c0ad5a0 con 0x7ff61c0d0770 2024-04-14T08:35:36.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.097+0000 7ff621757640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff60006c8f0 0x7ff60006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:36.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.097+0000 7ff621757640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff60006c8f0 0x7ff60006edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff618009b40 tx=0x7ff618009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:36.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.101+0000 7ff611ffb640 1 -- 172.21.15.172:0/1326693349 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff60c031800 con 0x7ff61c0d0770 2024-04-14T08:35:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.314+0000 7ff6239bb640 1 -- 172.21.15.172:0/1326693349 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff61c0c75f0 con 0x7ff61c0d0770 2024-04-14T08:35:36.943 INFO:tasks.ceph.mds.d.smithi177.stderr:2024-04-14T08:35:36.941+0000 7f9d66ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:35:36.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.974+0000 7ff611ffb640 1 -- 172.21.15.172:0/1326693349 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7ff60c07b1f0 con 0x7ff61c0d0770 2024-04-14T08:35:36.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.977+0000 7ff6239bb640 1 -- 172.21.15.172:0/1326693349 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff60006c8f0 msgr2=0x7ff60006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:36.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.977+0000 7ff6239bb640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff60006c8f0 0x7ff60006edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff618009b40 tx=0x7ff618009190 comp rx=0 tx=0).stop 2024-04-14T08:35:36.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.977+0000 7ff6239bb640 1 -- 172.21.15.172:0/1326693349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff61c0d0770 msgr2=0x7ff61c14c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:36.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.977+0000 7ff6239bb640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff61c0d0770 0x7ff61c14c450 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009890 tx=0x7ff60c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:35:36.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.977+0000 7ff6239bb640 1 -- 172.21.15.172:0/1326693349 shutdown_connections 2024-04-14T08:35:36.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.977+0000 7ff6239bb640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff60006c8f0 0x7ff60006edb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:36.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.977+0000 7ff6239bb640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff61c15c220 0x7ff61c0b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:36.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.977+0000 7ff6239bb640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff61c0d0770 0x7ff61c14c450 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:36.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.977+0000 7ff6239bb640 1 --2- 172.21.15.172:0/1326693349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff61c06ba60 0x7ff61c06f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:36.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.977+0000 7ff6239bb640 1 -- 172.21.15.172:0/1326693349 >> 172.21.15.172:0/1326693349 conn(0x7ff61c0cd980 msgr2=0x7ff61c0addb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:36.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.977+0000 7ff6239bb640 1 -- 172.21.15.172:0/1326693349 shutdown_connections 2024-04-14T08:35:36.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:36.977+0000 7ff6239bb640 1 -- 172.21.15.172:0/1326693349 wait complete. 2024-04-14T08:35:36.988 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-14T08:35:36.988 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:35:37.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.088+0000 7f957759c640 1 Processor -- start 2024-04-14T08:35:37.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.089+0000 7f957759c640 1 -- start start 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.089+0000 7f957759c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f957015e890 0x7f95700d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.089+0000 7f957759c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f95700cedd0 0x7f95700b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.089+0000 7f957759c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95700cf320 con 0x7f95700d2830 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.089+0000 7f957759c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f95700cff30 con 0x7f95700cedd0 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.089+0000 7f957759c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f95700d2350 con 0x7f957015e890 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.090+0000 7f9575338640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f95700d2830 0x7f95700d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59900/0 (socket says 172.21.15.172:59900) 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.090+0000 7f9575338640 1 -- 172.21.15.172:0/339096930 learned_addr learned my addr 172.21.15.172:0/339096930 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.090+0000 7f9574b37640 1 --2- 172.21.15.172:0/339096930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f957015e890 0x7f95700d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.090+0000 7f9575b39640 1 --2- 172.21.15.172:0/339096930 >> v2:172.21.15.177:3300/0 conn(0x7f95700cedd0 0x7f95700b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.090+0000 7f9567fff640 1 -- 172.21.15.172:0/339096930 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370971788 0 0) 0x7f95700cf320 con 0x7f95700d2830 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.090+0000 7f9567fff640 1 -- 172.21.15.172:0/339096930 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f955c003520 con 0x7f95700d2830 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.090+0000 7f9575b39640 1 -- 172.21.15.172:0/339096930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f957015e890 msgr2=0x7f95700d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:37.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.090+0000 7f9575b39640 1 --2- 172.21.15.172:0/339096930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f957015e890 0x7f95700d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.090+0000 7f9567fff640 1 -- 172.21.15.172:0/339096930 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2777227338 0 0) 0x7f955c003520 con 0x7f95700d2830 2024-04-14T08:35:37.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.090+0000 7f9575b39640 1 -- 172.21.15.172:0/339096930 >> v1:172.21.15.172:6789/0 conn(0x7f95700d2830 legacy=0x7f95700d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.090+0000 7f9575b39640 1 -- 172.21.15.172:0/339096930 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95700cf600 con 0x7f95700cedd0 2024-04-14T08:35:37.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.090+0000 7f9574b37640 1 --2- 172.21.15.172:0/339096930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f957015e890 0x7f95700d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:35:37.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.091+0000 7f9575b39640 1 --2- 172.21.15.172:0/339096930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95700cedd0 0x7f95700b3710 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f95680097a0 tx=0x7f956802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d556fb632dd0dff server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:37.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.091+0000 7f9567fff640 1 -- 172.21.15.172:0/339096930 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568038070 con 0x7f95700cedd0 2024-04-14T08:35:37.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.091+0000 7f9567fff640 1 -- 172.21.15.172:0/339096930 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9568031500 con 0x7f95700cedd0 2024-04-14T08:35:37.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.091+0000 7f9567fff640 1 -- 172.21.15.172:0/339096930 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c0019f0 con 0x7f95700cedd0 2024-04-14T08:35:37.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.091+0000 7f957759c640 1 -- 172.21.15.172:0/339096930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95700cedd0 msgr2=0x7f95700b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.091+0000 7f957759c640 1 --2- 172.21.15.172:0/339096930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95700cedd0 0x7f95700b3710 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f95680097a0 tx=0x7f956802b820 comp rx=0 tx=0).stop 2024-04-14T08:35:37.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.092+0000 7f957759c640 1 -- 172.21.15.172:0/339096930 shutdown_connections 2024-04-14T08:35:37.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.092+0000 7f957759c640 1 --2- 172.21.15.172:0/339096930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95700cedd0 0x7f95700b3710 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.092+0000 7f957759c640 1 --2- 172.21.15.172:0/339096930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f957015e890 0x7f95700d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.092+0000 7f957759c640 1 -- 172.21.15.172:0/339096930 >> 172.21.15.172:0/339096930 conn(0x7f957009d590 msgr2=0x7f95700d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:37.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.092+0000 7f957759c640 1 -- 172.21.15.172:0/339096930 shutdown_connections 2024-04-14T08:35:37.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.092+0000 7f957759c640 1 -- 172.21.15.172:0/339096930 wait complete. 2024-04-14T08:35:37.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.093+0000 7f957759c640 1 Processor -- start 2024-04-14T08:35:37.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.093+0000 7f957759c640 1 -- start start 2024-04-14T08:35:37.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.093+0000 7f957759c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95700cedd0 0x7f95700b9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.093+0000 7f957759c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95700d2830 0x7f95700c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f957759c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f957015e890 0x7f95700c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f957759c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95700d1840 con 0x7f95700cedd0 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f957759c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f95700cfbc0 con 0x7f95700d2830 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f957759c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f95700b3c50 con 0x7f957015e890 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9575338640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95700cedd0 0x7f95700b9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9575338640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95700cedd0 0x7f95700b9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42712/0 (socket says 172.21.15.172:42712) 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9575338640 1 -- 172.21.15.172:0/3252073422 learned_addr learned my addr 172.21.15.172:0/3252073422 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9574b37640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95700d2830 0x7f95700c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9575b39640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f957015e890 0x7f95700c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9575338640 1 -- 172.21.15.172:0/3252073422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f957015e890 msgr2=0x7f95700c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9575338640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f957015e890 0x7f95700c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9575338640 1 -- 172.21.15.172:0/3252073422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95700d2830 msgr2=0x7f95700c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9575338640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95700d2830 0x7f95700c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9575338640 1 -- 172.21.15.172:0/3252073422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95700cac30 con 0x7f95700cedd0 2024-04-14T08:35:37.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9575338640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95700cedd0 0x7f95700b9660 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f956c001910 tx=0x7f956c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:37.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9565ffb640 1 -- 172.21.15.172:0/3252073422 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c012400 con 0x7f95700cedd0 2024-04-14T08:35:37.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9565ffb640 1 -- 172.21.15.172:0/3252073422 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c0125c0 con 0x7f95700cedd0 2024-04-14T08:35:37.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f957759c640 1 -- 172.21.15.172:0/3252073422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95700cbc70 con 0x7f95700cedd0 2024-04-14T08:35:37.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.094+0000 7f9565ffb640 1 -- 172.21.15.172:0/3252073422 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c0129b0 con 0x7f95700cedd0 2024-04-14T08:35:37.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.095+0000 7f957759c640 1 -- 172.21.15.172:0/3252073422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95700cc1f0 con 0x7f95700cedd0 2024-04-14T08:35:37.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.095+0000 7f957759c640 1 -- 172.21.15.172:0/3252073422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95700d4d90 con 0x7f95700cedd0 2024-04-14T08:35:37.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.096+0000 7f9565ffb640 1 -- 172.21.15.172:0/3252073422 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f956c012b50 con 0x7f95700cedd0 2024-04-14T08:35:37.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.096+0000 7f9565ffb640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f955406c8f0 0x7f955406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.096+0000 7f9565ffb640 1 -- 172.21.15.172:0/3252073422 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f956c08c4a0 con 0x7f95700cedd0 2024-04-14T08:35:37.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.096+0000 7f9574b37640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f955406c8f0 0x7f955406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.097+0000 7f9574b37640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f955406c8f0 0x7f955406edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9560009560 tx=0x7f9560009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:37.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.101+0000 7f9565ffb640 1 -- 172.21.15.172:0/3252073422 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f95700d4d90 con 0x7f95700cedd0 2024-04-14T08:35:37.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.315+0000 7f957759c640 1 -- 172.21.15.172:0/3252073422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9570068520 con 0x7f95700cedd0 2024-04-14T08:35:37.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.316+0000 7f9565ffb640 1 -- 172.21.15.172:0/3252073422 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5194 (secure 0 0 0) 0x7f956c05a100 con 0x7f95700cedd0 2024-04-14T08:35:37.316 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:37.317 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3621780015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3621780015},{"type":"v1","addr":"172.21.15.172:6836","nonce":3621780015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3894683748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3894683748},{"type":"v1","addr":"172.21.15.172:6837","nonce":3894683748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3625745587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3625745587},{"type":"v1","addr":"172.21.15.177:6834","nonce":3625745587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:35:34.316542+0000","modified":"2024-04-14T08:35:36.314930+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/4176013980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":4176013980},{"type":"v1","addr":"172.21.15.177:6835","nonce":4176013980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-14T08:35:37.317 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 30 2024-04-14T08:35:37.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.318+0000 7f957759c640 1 -- 172.21.15.172:0/3252073422 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f955406c8f0 msgr2=0x7f955406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.318+0000 7f957759c640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f955406c8f0 0x7f955406edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9560009560 tx=0x7f9560009190 comp rx=0 tx=0).stop 2024-04-14T08:35:37.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.318+0000 7f957759c640 1 -- 172.21.15.172:0/3252073422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95700cedd0 msgr2=0x7f95700b9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.318+0000 7f957759c640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95700cedd0 0x7f95700b9660 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f956c001910 tx=0x7f956c00a430 comp rx=0 tx=0).stop 2024-04-14T08:35:37.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.318+0000 7f957759c640 1 -- 172.21.15.172:0/3252073422 shutdown_connections 2024-04-14T08:35:37.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.318+0000 7f957759c640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f955406c8f0 0x7f955406edb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.318+0000 7f957759c640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f957015e890 0x7f95700c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.318+0000 7f957759c640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95700d2830 0x7f95700c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.318+0000 7f957759c640 1 --2- 172.21.15.172:0/3252073422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95700cedd0 0x7f95700b9660 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.318+0000 7f957759c640 1 -- 172.21.15.172:0/3252073422 >> 172.21.15.172:0/3252073422 conn(0x7f957009d590 msgr2=0x7f957006c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:37.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.318+0000 7f957759c640 1 -- 172.21.15.172:0/3252073422 shutdown_connections 2024-04-14T08:35:37.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.318+0000 7f957759c640 1 -- 172.21.15.172:0/3252073422 wait complete. 2024-04-14T08:35:37.329 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:35:37.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.430+0000 7febbdc6d640 1 Processor -- start 2024-04-14T08:35:37.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.430+0000 7febbdc6d640 1 -- start start 2024-04-14T08:35:37.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febbdc6d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febb80b21d0 0x7febb815c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febbdc6d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb80b3700 con 0x7febb815c5a0 2024-04-14T08:35:37.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febbdc6d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb80b39e0 con 0x7febb80b1df0 2024-04-14T08:35:37.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febbdc6d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7febb80b3cc0 con 0x7febb80b21d0 2024-04-14T08:35:37.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febb7fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7febb815c5a0 0x7febb80b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59904/0 (socket says 172.21.15.172:59904) 2024-04-14T08:35:37.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febb7fff640 1 -- 172.21.15.172:0/473587179 learned_addr learned my addr 172.21.15.172:0/473587179 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:37.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febb6ffd640 1 --2- 172.21.15.172:0/473587179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febb80b21d0 0x7febb815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febb67fc640 1 -- 172.21.15.172:0/473587179 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2458236807 0 0) 0x7febb80b3700 con 0x7febb815c5a0 2024-04-14T08:35:37.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febb67fc640 1 -- 172.21.15.172:0/473587179 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feba0003520 con 0x7febb815c5a0 2024-04-14T08:35:37.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febb67fc640 1 -- 172.21.15.172:0/473587179 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 483021744 0 0) 0x7febb80b39e0 con 0x7febb80b1df0 2024-04-14T08:35:37.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febb6ffd640 1 -- 172.21.15.172:0/473587179 >> v1:172.21.15.177:6789/0 conn(0x7febb80b1df0 legacy=0x7febb80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febb6ffd640 1 -- 172.21.15.172:0/473587179 >> v1:172.21.15.172:6789/0 conn(0x7febb815c5a0 legacy=0x7febb80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.431+0000 7febb6ffd640 1 -- 172.21.15.172:0/473587179 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb815bb40 con 0x7febb80b21d0 2024-04-14T08:35:37.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.432+0000 7febb6ffd640 1 --2- 172.21.15.172:0/473587179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febb80b21d0 0x7febb815c060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7febac0097d0 tx=0x7febac02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f61676bca144968c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:37.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.432+0000 7febb67fc640 1 -- 172.21.15.172:0/473587179 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac035020 con 0x7febb80b21d0 2024-04-14T08:35:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.432+0000 7febb67fc640 1 -- 172.21.15.172:0/473587179 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febac031890 con 0x7febb80b21d0 2024-04-14T08:35:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.432+0000 7febb67fc640 1 -- 172.21.15.172:0/473587179 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac031c40 con 0x7febb80b21d0 2024-04-14T08:35:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.433+0000 7febbdc6d640 1 -- 172.21.15.172:0/473587179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febb80b21d0 msgr2=0x7febb815c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.433+0000 7febbdc6d640 1 --2- 172.21.15.172:0/473587179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febb80b21d0 0x7febb815c060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7febac0097d0 tx=0x7febac02b930 comp rx=0 tx=0).stop 2024-04-14T08:35:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.433+0000 7febbdc6d640 1 -- 172.21.15.172:0/473587179 shutdown_connections 2024-04-14T08:35:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.433+0000 7febbdc6d640 1 --2- 172.21.15.172:0/473587179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febb80b21d0 0x7febb815c060 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.433+0000 7febbdc6d640 1 -- 172.21.15.172:0/473587179 >> 172.21.15.172:0/473587179 conn(0x7febb806ec30 msgr2=0x7febb806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.433+0000 7febbdc6d640 1 -- 172.21.15.172:0/473587179 shutdown_connections 2024-04-14T08:35:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.433+0000 7febbdc6d640 1 -- 172.21.15.172:0/473587179 wait complete. 2024-04-14T08:35:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.434+0000 7febbdc6d640 1 Processor -- start 2024-04-14T08:35:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.434+0000 7febbdc6d640 1 -- start start 2024-04-14T08:35:37.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febbdc6d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febb80b1df0 0x7febb81f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febbdc6d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febb80b21d0 0x7febb80ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febbdc6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb80aea90 0x7febb80cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febbdc6d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7febb80b48d0 con 0x7febb80aea90 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febbdc6d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7febb80b4750 con 0x7febb80b1df0 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febbdc6d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7febb80ad7d0 con 0x7febb80b21d0 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febb7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb80aea90 0x7febb80cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febb7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb80aea90 0x7febb80cfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42716/0 (socket says 172.21.15.172:42716) 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febb7fff640 1 -- 172.21.15.172:0/3145035398 learned_addr learned my addr 172.21.15.172:0/3145035398 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febb77fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febb80b1df0 0x7febb81f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febb6ffd640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febb80b21d0 0x7febb80ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febb7fff640 1 -- 172.21.15.172:0/3145035398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febb80b21d0 msgr2=0x7febb80ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febb7fff640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febb80b21d0 0x7febb80ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febb7fff640 1 -- 172.21.15.172:0/3145035398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febb80b1df0 msgr2=0x7febb81f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febb7fff640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febb80b1df0 0x7febb81f4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febb7fff640 1 -- 172.21.15.172:0/3145035398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb80acbc0 con 0x7febb80aea90 2024-04-14T08:35:37.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.435+0000 7febb7fff640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb80aea90 0x7febb80cfe80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7febb0002230 tx=0x7febb000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:37.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.436+0000 7feb9ffff640 1 -- 172.21.15.172:0/3145035398 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0017070 con 0x7febb80aea90 2024-04-14T08:35:37.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.436+0000 7feb9ffff640 1 -- 172.21.15.172:0/3145035398 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb000ac30 con 0x7febb80aea90 2024-04-14T08:35:37.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.436+0000 7febbdc6d640 1 -- 172.21.15.172:0/3145035398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb80acea0 con 0x7febb80aea90 2024-04-14T08:35:37.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.436+0000 7feb9ffff640 1 -- 172.21.15.172:0/3145035398 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0012600 con 0x7febb80aea90 2024-04-14T08:35:37.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.436+0000 7febbdc6d640 1 -- 172.21.15.172:0/3145035398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb80ad460 con 0x7febb80aea90 2024-04-14T08:35:37.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.437+0000 7febbdc6d640 1 -- 172.21.15.172:0/3145035398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb8068520 con 0x7febb80aea90 2024-04-14T08:35:37.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.437+0000 7feb9ffff640 1 -- 172.21.15.172:0/3145035398 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febb001a070 con 0x7febb80aea90 2024-04-14T08:35:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.437+0000 7feb9ffff640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feba406c8f0 0x7feba406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.438+0000 7feb9ffff640 1 -- 172.21.15.172:0/3145035398 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7febb008b9b0 con 0x7febb80aea90 2024-04-14T08:35:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.438+0000 7febb77fe640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feba406c8f0 0x7feba406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.438+0000 7febb77fe640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feba406c8f0 0x7feba406edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feba8000f60 tx=0x7feba8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:37.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.442+0000 7feb9ffff640 1 -- 172.21.15.172:0/3145035398 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7febb000ae20 con 0x7febb80aea90 2024-04-14T08:35:37.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.619+0000 7febbdc6d640 1 -- 172.21.15.172:0/3145035398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febb80cf840 con 0x7febb80aea90 2024-04-14T08:35:37.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.620+0000 7feb9ffff640 1 -- 172.21.15.172:0/3145035398 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16944 (secure 0 0 0) 0x7febb0059600 con 0x7febb80aea90 2024-04-14T08:35:37.621 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:37.621 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":61,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:35:35.966329+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:35:32.539067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-14T08:35:33.263123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:35:37.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.622+0000 7febbdc6d640 1 -- 172.21.15.172:0/3145035398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feba406c8f0 msgr2=0x7feba406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.622+0000 7febbdc6d640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feba406c8f0 0x7feba406edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feba8000f60 tx=0x7feba8009040 comp rx=0 tx=0).stop 2024-04-14T08:35:37.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.622+0000 7febbdc6d640 1 -- 172.21.15.172:0/3145035398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb80aea90 msgr2=0x7febb80cfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.622+0000 7febbdc6d640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb80aea90 0x7febb80cfe80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7febb0002230 tx=0x7febb000cf70 comp rx=0 tx=0).stop 2024-04-14T08:35:37.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.622+0000 7febbdc6d640 1 -- 172.21.15.172:0/3145035398 shutdown_connections 2024-04-14T08:35:37.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.622+0000 7febbdc6d640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feba406c8f0 0x7feba406edb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.622+0000 7febbdc6d640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febb80aea90 0x7febb80cfe80 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.622+0000 7febbdc6d640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7febb80b21d0 0x7febb80ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.622+0000 7febbdc6d640 1 --2- 172.21.15.172:0/3145035398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febb80b1df0 0x7febb81f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.622+0000 7febbdc6d640 1 -- 172.21.15.172:0/3145035398 >> 172.21.15.172:0/3145035398 conn(0x7febb806ec30 msgr2=0x7febb80b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:37.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.622+0000 7febbdc6d640 1 -- 172.21.15.172:0/3145035398 shutdown_connections 2024-04-14T08:35:37.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.623+0000 7febbdc6d640 1 -- 172.21.15.172:0/3145035398 wait complete. 2024-04-14T08:35:37.634 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-14T08:35:37.634 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:35:37.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.734+0000 7effd64c8640 1 Processor -- start 2024-04-14T08:35:37.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.734+0000 7effd64c8640 1 -- start start 2024-04-14T08:35:37.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effd64c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7effd00b1df0 0x7effd00b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effd64c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effd00b21d0 0x7effd015c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effd64c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effd00b3430 con 0x7effd015c5a0 2024-04-14T08:35:37.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effd64c8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7effd00b3700 con 0x7effd00b1df0 2024-04-14T08:35:37.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effd64c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7effd00b39e0 con 0x7effd00b21d0 2024-04-14T08:35:37.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effcffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7effd00b1df0 0x7effd00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effcffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7effd00b1df0 0x7effd00b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42730/0 (socket says 172.21.15.172:42730) 2024-04-14T08:35:37.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effcffff640 1 -- 172.21.15.172:0/1216771917 learned_addr learned my addr 172.21.15.172:0/1216771917 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:37.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effcf7fe640 1 --2- 172.21.15.172:0/1216771917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effd00b21d0 0x7effd015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effcffff640 1 -- 172.21.15.172:0/1216771917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effd00b21d0 msgr2=0x7effd015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:37.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effcffff640 1 --2- 172.21.15.172:0/1216771917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effd00b21d0 0x7effd015c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effcffff640 1 -- 172.21.15.172:0/1216771917 >> v1:172.21.15.177:6789/0 conn(0x7effd015c5a0 legacy=0x7effd00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.735+0000 7effcffff640 1 -- 172.21.15.172:0/1216771917 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd00b3cc0 con 0x7effd00b1df0 2024-04-14T08:35:37.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.736+0000 7effcffff640 1 --2- 172.21.15.172:0/1216771917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd00b1df0 0x7effd00b2ef0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7effc000bb10 tx=0x7effc002e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502121e61dedd746 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:37.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.736+0000 7effceffd640 1 -- 172.21.15.172:0/1216771917 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc003b070 con 0x7effd00b1df0 2024-04-14T08:35:37.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.736+0000 7effceffd640 1 -- 172.21.15.172:0/1216771917 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc0036440 con 0x7effd00b1df0 2024-04-14T08:35:37.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.736+0000 7effceffd640 1 -- 172.21.15.172:0/1216771917 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0036810 con 0x7effd00b1df0 2024-04-14T08:35:37.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.737+0000 7effd64c8640 1 -- 172.21.15.172:0/1216771917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd00b1df0 msgr2=0x7effd00b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.737+0000 7effd64c8640 1 --2- 172.21.15.172:0/1216771917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd00b1df0 0x7effd00b2ef0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7effc000bb10 tx=0x7effc002e150 comp rx=0 tx=0).stop 2024-04-14T08:35:37.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.737+0000 7effd64c8640 1 -- 172.21.15.172:0/1216771917 shutdown_connections 2024-04-14T08:35:37.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.737+0000 7effd64c8640 1 --2- 172.21.15.172:0/1216771917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effd00b21d0 0x7effd015c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.737+0000 7effd64c8640 1 --2- 172.21.15.172:0/1216771917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd00b1df0 0x7effd00b2ef0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.737+0000 7effd64c8640 1 -- 172.21.15.172:0/1216771917 >> 172.21.15.172:0/1216771917 conn(0x7effd006ec30 msgr2=0x7effd006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:37.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.737+0000 7effd64c8640 1 -- 172.21.15.172:0/1216771917 shutdown_connections 2024-04-14T08:35:37.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.737+0000 7effd64c8640 1 -- 172.21.15.172:0/1216771917 wait complete. 2024-04-14T08:35:37.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.738+0000 7effd64c8640 1 Processor -- start 2024-04-14T08:35:37.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.738+0000 7effd64c8640 1 -- start start 2024-04-14T08:35:37.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd64c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effd00b1df0 0x7effd01e20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd64c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effd00b21d0 0x7effd01e3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd64c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd0129f60 0x7effd012c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd64c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7effd00b4750 con 0x7effd0129f60 2024-04-14T08:35:37.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd64c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7effd015b7f0 con 0x7effd00b1df0 2024-04-14T08:35:37.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd64c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7effd00b48d0 con 0x7effd00b21d0 2024-04-14T08:35:37.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd4a65640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd0129f60 0x7effd012c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effcffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effd00b1df0 0x7effd01e20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effcf7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effd00b21d0 0x7effd01e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd4a65640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd0129f60 0x7effd012c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42732/0 (socket says 172.21.15.172:42732) 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd4a65640 1 -- 172.21.15.172:0/3874444303 learned_addr learned my addr 172.21.15.172:0/3874444303 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd4a65640 1 -- 172.21.15.172:0/3874444303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effd00b21d0 msgr2=0x7effd01e3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd4a65640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effd00b21d0 0x7effd01e3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd4a65640 1 -- 172.21.15.172:0/3874444303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effd00b1df0 msgr2=0x7effd01e20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd4a65640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effd00b1df0 0x7effd01e20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effd4a65640 1 -- 172.21.15.172:0/3874444303 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd01e3d40 con 0x7effd0129f60 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.739+0000 7effcf7fe640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effd00b21d0 0x7effd01e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.740+0000 7effd4a65640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd0129f60 0x7effd012c830 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7effc800c730 tx=0x7effc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.740+0000 7effccff9640 1 -- 172.21.15.172:0/3874444303 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8016020 con 0x7effd0129f60 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.740+0000 7effccff9640 1 -- 172.21.15.172:0/3874444303 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc8006d50 con 0x7effd0129f60 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.740+0000 7effd64c8640 1 -- 172.21.15.172:0/3874444303 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd01e3fe0 con 0x7effd0129f60 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.740+0000 7effccff9640 1 -- 172.21.15.172:0/3874444303 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8003b90 con 0x7effd0129f60 2024-04-14T08:35:37.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.740+0000 7effd64c8640 1 -- 172.21.15.172:0/3874444303 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd01e45a0 con 0x7effd0129f60 2024-04-14T08:35:37.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.741+0000 7effd64c8640 1 -- 172.21.15.172:0/3874444303 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd0068520 con 0x7effd0129f60 2024-04-14T08:35:37.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.741+0000 7effccff9640 1 -- 172.21.15.172:0/3874444303 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effc80050d0 con 0x7effd0129f60 2024-04-14T08:35:37.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.741+0000 7effccff9640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effbc06c8f0 0x7effbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:37.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.742+0000 7effccff9640 1 -- 172.21.15.172:0/3874444303 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7effc808b740 con 0x7effd0129f60 2024-04-14T08:35:37.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.742+0000 7effcffff640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effbc06c8f0 0x7effbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:37.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.742+0000 7effcffff640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effbc06c8f0 0x7effbc06edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7effc0038000 tx=0x7effc0032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:37.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.746+0000 7effccff9640 1 -- 172.21.15.172:0/3874444303 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7effc8020020 con 0x7effd0129f60 2024-04-14T08:35:37.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.917+0000 7effd64c8640 1 -- 172.21.15.172:0/3874444303 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7effd006c530 con 0x7effbc06c8f0 2024-04-14T08:35:37.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.923+0000 7effccff9640 1 -- 172.21.15.172:0/3874444303 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7effd006c530 con 0x7effbc06c8f0 2024-04-14T08:35:37.927 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:37.928 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:35:37.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.926+0000 7effd64c8640 1 -- 172.21.15.172:0/3874444303 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effbc06c8f0 msgr2=0x7effbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.926+0000 7effd64c8640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effbc06c8f0 0x7effbc06edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7effc0038000 tx=0x7effc0032000 comp rx=0 tx=0).stop 2024-04-14T08:35:37.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.926+0000 7effd64c8640 1 -- 172.21.15.172:0/3874444303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd0129f60 msgr2=0x7effd012c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:37.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.926+0000 7effd64c8640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd0129f60 0x7effd012c830 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7effc800c730 tx=0x7effc800cbf0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.926+0000 7effd64c8640 1 -- 172.21.15.172:0/3874444303 shutdown_connections 2024-04-14T08:35:37.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.926+0000 7effd64c8640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effbc06c8f0 0x7effbc06edb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.926+0000 7effd64c8640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd0129f60 0x7effd012c830 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.926+0000 7effd64c8640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effd00b21d0 0x7effd01e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.926+0000 7effd64c8640 1 --2- 172.21.15.172:0/3874444303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effd00b1df0 0x7effd01e20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:37.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.926+0000 7effd64c8640 1 -- 172.21.15.172:0/3874444303 >> 172.21.15.172:0/3874444303 conn(0x7effd006ec30 msgr2=0x7effd0144760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:37.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.927+0000 7effd64c8640 1 -- 172.21.15.172:0/3874444303 shutdown_connections 2024-04-14T08:35:37.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:37.927+0000 7effd64c8640 1 -- 172.21.15.172:0/3874444303 wait complete. 2024-04-14T08:35:37.938 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-04-14T08:35:37.151434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222980,"kb_used_data":6780,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751740,"statfs":{"total":773094113280,"available":772865781760,"internally_reserved":0,"allocated":6942720,"data_stored":3725746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.202597"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984677+0000","last_change":"2024-04-14T08:35:34.980641+0000","last_active":"2024-04-14T08:35:34.984677+0000","last_peered":"2024-04-14T08:35:34.984677+0000","last_clean":"2024-04-14T08:35:34.984677+0000","last_became_active":"2024-04-14T08:35:34.980371+0000","last_became_peered":"2024-04-14T08:35:34.980371+0000","last_unstale":"2024-04-14T08:35:34.984677+0000","last_undegraded":"2024-04-14T08:35:34.984677+0000","last_fullsized":"2024-04-14T08:35:34.984677+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:58:09.951538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984648+0000","last_change":"2024-04-14T08:35:34.981257+0000","last_active":"2024-04-14T08:35:34.984648+0000","last_peered":"2024-04-14T08:35:34.984648+0000","last_clean":"2024-04-14T08:35:34.984648+0000","last_became_active":"2024-04-14T08:35:34.980970+0000","last_became_peered":"2024-04-14T08:35:34.980970+0000","last_unstale":"2024-04-14T08:35:34.984648+0000","last_undegraded":"2024-04-14T08:35:34.984648+0000","last_fullsized":"2024-04-14T08:35:34.984648+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:12:14.921095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.976003+0000","last_change":"2024-04-14T08:35:34.975665+0000","last_active":"2024-04-14T08:35:35.976003+0000","last_peered":"2024-04-14T08:35:35.976003+0000","last_clean":"2024-04-14T08:35:35.976003+0000","last_became_active":"2024-04-14T08:35:34.974503+0000","last_became_peered":"2024-04-14T08:35:34.974503+0000","last_unstale":"2024-04-14T08:35:35.976003+0000","last_undegraded":"2024-04-14T08:35:35.976003+0000","last_fullsized":"2024-04-14T08:35:35.976003+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:21:27.827950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.986149+0000","last_change":"2024-04-14T08:35:34.979055+0000","last_active":"2024-04-14T08:35:34.986149+0000","last_peered":"2024-04-14T08:35:34.986149+0000","last_clean":"2024-04-14T08:35:34.986149+0000","last_became_active":"2024-04-14T08:35:34.978754+0000","last_became_peered":"2024-04-14T08:35:34.978754+0000","last_unstale":"2024-04-14T08:35:34.986149+0000","last_undegraded":"2024-04-14T08:35:34.986149+0000","last_fullsized":"2024-04-14T08:35:34.986149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:54.240477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.988030+0000","last_change":"2024-04-14T08:35:34.986822+0000","last_active":"2024-04-14T08:35:34.988030+0000","last_peered":"2024-04-14T08:35:34.988030+0000","last_clean":"2024-04-14T08:35:34.988030+0000","last_became_active":"2024-04-14T08:35:34.986521+0000","last_became_peered":"2024-04-14T08:35:34.986521+0000","last_unstale":"2024-04-14T08:35:34.988030+0000","last_undegraded":"2024-04-14T08:35:34.988030+0000","last_fullsized":"2024-04-14T08:35:34.988030+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:24:44.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.978802+0000","last_change":"2024-04-14T08:35:34.972317+0000","last_active":"2024-04-14T08:35:34.978802+0000","last_peered":"2024-04-14T08:35:34.978802+0000","last_clean":"2024-04-14T08:35:34.978802+0000","last_became_active":"2024-04-14T08:35:34.968042+0000","last_became_peered":"2024-04-14T08:35:34.968042+0000","last_unstale":"2024-04-14T08:35:34.978802+0000","last_undegraded":"2024-04-14T08:35:34.978802+0000","last_fullsized":"2024-04-14T08:35:34.978802+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:05:01.671432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.979008+0000","last_change":"2024-04-14T08:35:34.975125+0000","last_active":"2024-04-14T08:35:34.979008+0000","last_peered":"2024-04-14T08:35:34.979008+0000","last_clean":"2024-04-14T08:35:34.979008+0000","last_became_active":"2024-04-14T08:35:34.974408+0000","last_became_peered":"2024-04-14T08:35:34.974408+0000","last_unstale":"2024-04-14T08:35:34.979008+0000","last_undegraded":"2024-04-14T08:35:34.979008+0000","last_fullsized":"2024-04-14T08:35:34.979008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:00:12.295396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.982994+0000","last_change":"2024-04-14T08:35:34.978660+0000","last_active":"2024-04-14T08:35:34.982994+0000","last_peered":"2024-04-14T08:35:34.982994+0000","last_clean":"2024-04-14T08:35:34.982994+0000","last_became_active":"2024-04-14T08:35:34.977718+0000","last_became_peered":"2024-04-14T08:35:34.977718+0000","last_unstale":"2024-04-14T08:35:34.982994+0000","last_undegraded":"2024-04-14T08:35:34.982994+0000","last_fullsized":"2024-04-14T08:35:34.982994+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:43:37.834408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.976149+0000","last_change":"2024-04-14T08:35:34.976790+0000","last_active":"2024-04-14T08:35:35.976149+0000","last_peered":"2024-04-14T08:35:35.976149+0000","last_clean":"2024-04-14T08:35:35.976149+0000","last_became_active":"2024-04-14T08:35:34.976512+0000","last_became_peered":"2024-04-14T08:35:34.976512+0000","last_unstale":"2024-04-14T08:35:35.976149+0000","last_undegraded":"2024-04-14T08:35:35.976149+0000","last_fullsized":"2024-04-14T08:35:35.976149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:54:12.148162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.974967+0000","last_change":"2024-04-14T08:35:34.977317+0000","last_active":"2024-04-14T08:35:35.974967+0000","last_peered":"2024-04-14T08:35:35.974967+0000","last_clean":"2024-04-14T08:35:35.974967+0000","last_became_active":"2024-04-14T08:35:34.977032+0000","last_became_peered":"2024-04-14T08:35:34.977032+0000","last_unstale":"2024-04-14T08:35:35.974967+0000","last_undegraded":"2024-04-14T08:35:35.974967+0000","last_fullsized":"2024-04-14T08:35:35.974967+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:38:32.452020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985060+0000","last_change":"2024-04-14T08:35:34.980617+0000","last_active":"2024-04-14T08:35:34.985060+0000","last_peered":"2024-04-14T08:35:34.985060+0000","last_clean":"2024-04-14T08:35:34.985060+0000","last_became_active":"2024-04-14T08:35:34.980349+0000","last_became_peered":"2024-04-14T08:35:34.980349+0000","last_unstale":"2024-04-14T08:35:34.985060+0000","last_undegraded":"2024-04-14T08:35:34.985060+0000","last_fullsized":"2024-04-14T08:35:34.985060+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:11:56.878423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.986427+0000","last_change":"2024-04-14T08:35:34.984084+0000","last_active":"2024-04-14T08:35:34.986427+0000","last_peered":"2024-04-14T08:35:34.986427+0000","last_clean":"2024-04-14T08:35:34.986427+0000","last_became_active":"2024-04-14T08:35:34.983802+0000","last_became_peered":"2024-04-14T08:35:34.983802+0000","last_unstale":"2024-04-14T08:35:34.986427+0000","last_undegraded":"2024-04-14T08:35:34.986427+0000","last_fullsized":"2024-04-14T08:35:34.986427+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:15:59.982298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.988562+0000","last_change":"2024-04-14T08:35:34.987874+0000","last_active":"2024-04-14T08:35:34.988562+0000","last_peered":"2024-04-14T08:35:34.988562+0000","last_clean":"2024-04-14T08:35:34.988562+0000","last_became_active":"2024-04-14T08:35:34.987562+0000","last_became_peered":"2024-04-14T08:35:34.987562+0000","last_unstale":"2024-04-14T08:35:34.988562+0000","last_undegraded":"2024-04-14T08:35:34.988562+0000","last_fullsized":"2024-04-14T08:35:34.988562+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:58:45.944045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975631+0000","last_change":"2024-04-14T08:35:34.973945+0000","last_active":"2024-04-14T08:35:35.975631+0000","last_peered":"2024-04-14T08:35:35.975631+0000","last_clean":"2024-04-14T08:35:35.975631+0000","last_became_active":"2024-04-14T08:35:34.973641+0000","last_became_peered":"2024-04-14T08:35:34.973641+0000","last_unstale":"2024-04-14T08:35:35.975631+0000","last_undegraded":"2024-04-14T08:35:35.975631+0000","last_fullsized":"2024-04-14T08:35:35.975631+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:37:45.979299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975118+0000","last_change":"2024-04-14T08:35:34.982413+0000","last_active":"2024-04-14T08:35:35.975118+0000","last_peered":"2024-04-14T08:35:35.975118+0000","last_clean":"2024-04-14T08:35:35.975118+0000","last_became_active":"2024-04-14T08:35:34.982131+0000","last_became_peered":"2024-04-14T08:35:34.982131+0000","last_unstale":"2024-04-14T08:35:35.975118+0000","last_undegraded":"2024-04-14T08:35:35.975118+0000","last_fullsized":"2024-04-14T08:35:35.975118+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:31:26.383953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983058+0000","last_change":"2024-04-14T08:35:34.980705+0000","last_active":"2024-04-14T08:35:34.983058+0000","last_peered":"2024-04-14T08:35:34.983058+0000","last_clean":"2024-04-14T08:35:34.983058+0000","last_became_active":"2024-04-14T08:35:34.980387+0000","last_became_peered":"2024-04-14T08:35:34.980387+0000","last_unstale":"2024-04-14T08:35:34.983058+0000","last_undegraded":"2024-04-14T08:35:34.983058+0000","last_fullsized":"2024-04-14T08:35:34.983058+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:23:11.932844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984986+0000","last_change":"2024-04-14T08:35:34.982825+0000","last_active":"2024-04-14T08:35:34.984986+0000","last_peered":"2024-04-14T08:35:34.984986+0000","last_clean":"2024-04-14T08:35:34.984986+0000","last_became_active":"2024-04-14T08:35:34.982497+0000","last_became_peered":"2024-04-14T08:35:34.982497+0000","last_unstale":"2024-04-14T08:35:34.984986+0000","last_undegraded":"2024-04-14T08:35:34.984986+0000","last_fullsized":"2024-04-14T08:35:34.984986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:39:34.437624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984943+0000","last_change":"2024-04-14T08:35:34.979449+0000","last_active":"2024-04-14T08:35:34.984943+0000","last_peered":"2024-04-14T08:35:34.984943+0000","last_clean":"2024-04-14T08:35:34.984943+0000","last_became_active":"2024-04-14T08:35:34.978857+0000","last_became_peered":"2024-04-14T08:35:34.978857+0000","last_unstale":"2024-04-14T08:35:34.984943+0000","last_undegraded":"2024-04-14T08:35:34.984943+0000","last_fullsized":"2024-04-14T08:35:34.984943+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:58:05.015142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.981695+0000","last_change":"2024-04-14T08:35:34.974253+0000","last_active":"2024-04-14T08:35:34.981695+0000","last_peered":"2024-04-14T08:35:34.981695+0000","last_clean":"2024-04-14T08:35:34.981695+0000","last_became_active":"2024-04-14T08:35:34.974001+0000","last_became_peered":"2024-04-14T08:35:34.974001+0000","last_unstale":"2024-04-14T08:35:34.981695+0000","last_undegraded":"2024-04-14T08:35:34.981695+0000","last_fullsized":"2024-04-14T08:35:34.981695+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:15:16.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.978555+0000","last_change":"2024-04-14T08:35:34.975278+0000","last_active":"2024-04-14T08:35:34.978555+0000","last_peered":"2024-04-14T08:35:34.978555+0000","last_clean":"2024-04-14T08:35:34.978555+0000","last_became_active":"2024-04-14T08:35:34.974982+0000","last_became_peered":"2024-04-14T08:35:34.974982+0000","last_unstale":"2024-04-14T08:35:34.978555+0000","last_undegraded":"2024-04-14T08:35:34.978555+0000","last_fullsized":"2024-04-14T08:35:34.978555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:05:01.671432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.986261+0000","last_change":"2024-04-14T08:35:34.985102+0000","last_active":"2024-04-14T08:35:34.986261+0000","last_peered":"2024-04-14T08:35:34.986261+0000","last_clean":"2024-04-14T08:35:34.986261+0000","last_became_active":"2024-04-14T08:35:34.984845+0000","last_became_peered":"2024-04-14T08:35:34.984845+0000","last_unstale":"2024-04-14T08:35:34.986261+0000","last_undegraded":"2024-04-14T08:35:34.986261+0000","last_fullsized":"2024-04-14T08:35:34.986261+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:40:39.599055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.967842+0000","last_change":"2024-04-14T08:35:34.965393+0000","last_active":"2024-04-14T08:35:34.967842+0000","last_peered":"2024-04-14T08:35:34.967842+0000","last_clean":"2024-04-14T08:35:34.967842+0000","last_became_active":"2024-04-14T08:35:34.964879+0000","last_became_peered":"2024-04-14T08:35:34.964879+0000","last_unstale":"2024-04-14T08:35:34.967842+0000","last_undegraded":"2024-04-14T08:35:34.967842+0000","last_fullsized":"2024-04-14T08:35:34.967842+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:31:26.383953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.967781+0000","last_change":"2024-04-14T08:35:34.964422+0000","last_active":"2024-04-14T08:35:34.967781+0000","last_peered":"2024-04-14T08:35:34.967781+0000","last_clean":"2024-04-14T08:35:34.967781+0000","last_became_active":"2024-04-14T08:35:34.964139+0000","last_became_peered":"2024-04-14T08:35:34.964139+0000","last_unstale":"2024-04-14T08:35:34.967781+0000","last_undegraded":"2024-04-14T08:35:34.967781+0000","last_fullsized":"2024-04-14T08:35:34.967781+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:38:32.452020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.974528+0000","last_change":"2024-04-14T08:35:34.975927+0000","last_active":"2024-04-14T08:35:35.974528+0000","last_peered":"2024-04-14T08:35:35.974528+0000","last_clean":"2024-04-14T08:35:35.974528+0000","last_became_active":"2024-04-14T08:35:34.974807+0000","last_became_peered":"2024-04-14T08:35:34.974807+0000","last_unstale":"2024-04-14T08:35:35.974528+0000","last_undegraded":"2024-04-14T08:35:35.974528+0000","last_fullsized":"2024-04-14T08:35:35.974528+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:06.090229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983938+0000","last_change":"2024-04-14T08:35:34.981889+0000","last_active":"2024-04-14T08:35:34.983938+0000","last_peered":"2024-04-14T08:35:34.983938+0000","last_clean":"2024-04-14T08:35:34.983938+0000","last_became_active":"2024-04-14T08:35:34.979724+0000","last_became_peered":"2024-04-14T08:35:34.979724+0000","last_unstale":"2024-04-14T08:35:34.983938+0000","last_undegraded":"2024-04-14T08:35:34.983938+0000","last_fullsized":"2024-04-14T08:35:34.983938+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:50:25.888282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983145+0000","last_change":"2024-04-14T08:35:34.980471+0000","last_active":"2024-04-14T08:35:34.983145+0000","last_peered":"2024-04-14T08:35:34.983145+0000","last_clean":"2024-04-14T08:35:34.983145+0000","last_became_active":"2024-04-14T08:35:34.980103+0000","last_became_peered":"2024-04-14T08:35:34.980103+0000","last_unstale":"2024-04-14T08:35:34.983145+0000","last_undegraded":"2024-04-14T08:35:34.983145+0000","last_fullsized":"2024-04-14T08:35:34.983145+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:34:23.436578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.977632+0000","last_change":"2024-04-14T08:35:34.970704+0000","last_active":"2024-04-14T08:35:34.977632+0000","last_peered":"2024-04-14T08:35:34.977632+0000","last_clean":"2024-04-14T08:35:34.977632+0000","last_became_active":"2024-04-14T08:35:34.970429+0000","last_became_peered":"2024-04-14T08:35:34.970429+0000","last_unstale":"2024-04-14T08:35:34.977632+0000","last_undegraded":"2024-04-14T08:35:34.977632+0000","last_fullsized":"2024-04-14T08:35:34.977632+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:33:26.666479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984521+0000","last_change":"2024-04-14T08:35:34.978255+0000","last_active":"2024-04-14T08:35:34.984521+0000","last_peered":"2024-04-14T08:35:34.984521+0000","last_clean":"2024-04-14T08:35:34.984521+0000","last_became_active":"2024-04-14T08:35:34.977956+0000","last_became_peered":"2024-04-14T08:35:34.977956+0000","last_unstale":"2024-04-14T08:35:34.984521+0000","last_undegraded":"2024-04-14T08:35:34.984521+0000","last_fullsized":"2024-04-14T08:35:34.984521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:55.003593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975859+0000","last_change":"2024-04-14T08:35:34.979185+0000","last_active":"2024-04-14T08:35:35.975859+0000","last_peered":"2024-04-14T08:35:35.975859+0000","last_clean":"2024-04-14T08:35:35.975859+0000","last_became_active":"2024-04-14T08:35:34.978928+0000","last_became_peered":"2024-04-14T08:35:34.978928+0000","last_unstale":"2024-04-14T08:35:35.975859+0000","last_undegraded":"2024-04-14T08:35:35.975859+0000","last_fullsized":"2024-04-14T08:35:35.975859+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:07:49.315642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987629+0000","last_change":"2024-04-14T08:35:34.985793+0000","last_active":"2024-04-14T08:35:34.987629+0000","last_peered":"2024-04-14T08:35:34.987629+0000","last_clean":"2024-04-14T08:35:34.987629+0000","last_became_active":"2024-04-14T08:35:34.985496+0000","last_became_peered":"2024-04-14T08:35:34.985496+0000","last_unstale":"2024-04-14T08:35:34.987629+0000","last_undegraded":"2024-04-14T08:35:34.987629+0000","last_fullsized":"2024-04-14T08:35:34.987629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:46:01.496201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.977416+0000","last_change":"2024-04-14T08:35:34.970027+0000","last_active":"2024-04-14T08:35:34.977416+0000","last_peered":"2024-04-14T08:35:34.977416+0000","last_clean":"2024-04-14T08:35:34.977416+0000","last_became_active":"2024-04-14T08:35:34.969772+0000","last_became_peered":"2024-04-14T08:35:34.969772+0000","last_unstale":"2024-04-14T08:35:34.977416+0000","last_undegraded":"2024-04-14T08:35:34.977416+0000","last_fullsized":"2024-04-14T08:35:34.977416+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:32:14.300795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.979212+0000","last_change":"2024-04-14T08:35:34.972505+0000","last_active":"2024-04-14T08:35:34.979212+0000","last_peered":"2024-04-14T08:35:34.979212+0000","last_clean":"2024-04-14T08:35:34.979212+0000","last_became_active":"2024-04-14T08:35:34.968653+0000","last_became_peered":"2024-04-14T08:35:34.968653+0000","last_unstale":"2024-04-14T08:35:34.979212+0000","last_undegraded":"2024-04-14T08:35:34.979212+0000","last_fullsized":"2024-04-14T08:35:34.979212+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:07:49.315642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987463+0000","last_change":"2024-04-14T08:35:34.986135+0000","last_active":"2024-04-14T08:35:34.987463+0000","last_peered":"2024-04-14T08:35:34.987463+0000","last_clean":"2024-04-14T08:35:34.987463+0000","last_became_active":"2024-04-14T08:35:34.985868+0000","last_became_peered":"2024-04-14T08:35:34.985868+0000","last_unstale":"2024-04-14T08:35:34.987463+0000","last_undegraded":"2024-04-14T08:35:34.987463+0000","last_fullsized":"2024-04-14T08:35:34.987463+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:11:56.878423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"60'4","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987609+0000","last_change":"2024-04-14T08:35:33.975983+0000","last_active":"2024-04-14T08:35:34.987609+0000","last_peered":"2024-04-14T08:35:34.987609+0000","last_clean":"2024-04-14T08:35:34.987609+0000","last_became_active":"2024-04-14T08:35:33.975715+0000","last_became_peered":"2024-04-14T08:35:33.975715+0000","last_unstale":"2024-04-14T08:35:34.987609+0000","last_undegraded":"2024-04-14T08:35:34.987609+0000","last_fullsized":"2024-04-14T08:35:34.987609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:58:04.011401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984833+0000","last_change":"2024-04-14T08:35:34.977214+0000","last_active":"2024-04-14T08:35:34.984833+0000","last_peered":"2024-04-14T08:35:34.984833+0000","last_clean":"2024-04-14T08:35:34.984833+0000","last_became_active":"2024-04-14T08:35:34.976925+0000","last_became_peered":"2024-04-14T08:35:34.976925+0000","last_unstale":"2024-04-14T08:35:34.984833+0000","last_undegraded":"2024-04-14T08:35:34.984833+0000","last_fullsized":"2024-04-14T08:35:34.984833+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:14:25.180055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984268+0000","last_change":"2024-04-14T08:35:33.968042+0000","last_active":"2024-04-14T08:35:34.984268+0000","last_peered":"2024-04-14T08:35:34.984268+0000","last_clean":"2024-04-14T08:35:34.984268+0000","last_became_active":"2024-04-14T08:35:33.967770+0000","last_became_peered":"2024-04-14T08:35:33.967770+0000","last_unstale":"2024-04-14T08:35:34.984268+0000","last_undegraded":"2024-04-14T08:35:34.984268+0000","last_fullsized":"2024-04-14T08:35:34.984268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:15:13.907170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.978774+0000","last_change":"2024-04-14T08:35:34.966969+0000","last_active":"2024-04-14T08:35:34.978774+0000","last_peered":"2024-04-14T08:35:34.978774+0000","last_clean":"2024-04-14T08:35:34.978774+0000","last_became_active":"2024-04-14T08:35:34.966513+0000","last_became_peered":"2024-04-14T08:35:34.966513+0000","last_unstale":"2024-04-14T08:35:34.978774+0000","last_undegraded":"2024-04-14T08:35:34.978774+0000","last_fullsized":"2024-04-14T08:35:34.978774+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:31:26.383953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983714+0000","last_change":"2024-04-14T08:35:33.973904+0000","last_active":"2024-04-14T08:35:34.983714+0000","last_peered":"2024-04-14T08:35:34.983714+0000","last_clean":"2024-04-14T08:35:34.983714+0000","last_became_active":"2024-04-14T08:35:33.973351+0000","last_became_peered":"2024-04-14T08:35:33.973351+0000","last_unstale":"2024-04-14T08:35:34.983714+0000","last_undegraded":"2024-04-14T08:35:34.983714+0000","last_fullsized":"2024-04-14T08:35:34.983714+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:40.214958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975705+0000","last_change":"2024-04-14T08:35:34.981126+0000","last_active":"2024-04-14T08:35:35.975705+0000","last_peered":"2024-04-14T08:35:35.975705+0000","last_clean":"2024-04-14T08:35:35.975705+0000","last_became_active":"2024-04-14T08:35:34.980816+0000","last_became_peered":"2024-04-14T08:35:34.980816+0000","last_unstale":"2024-04-14T08:35:35.975705+0000","last_undegraded":"2024-04-14T08:35:35.975705+0000","last_fullsized":"2024-04-14T08:35:35.975705+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:05:01.671432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985469+0000","last_change":"2024-04-14T08:35:33.966087+0000","last_active":"2024-04-14T08:35:34.985469+0000","last_peered":"2024-04-14T08:35:34.985469+0000","last_clean":"2024-04-14T08:35:34.985469+0000","last_became_active":"2024-04-14T08:35:33.965760+0000","last_became_peered":"2024-04-14T08:35:33.965760+0000","last_unstale":"2024-04-14T08:35:34.985469+0000","last_undegraded":"2024-04-14T08:35:34.985469+0000","last_fullsized":"2024-04-14T08:35:34.985469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:38:31.448280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.978803+0000","last_change":"2024-04-14T08:35:34.974129+0000","last_active":"2024-04-14T08:35:34.978803+0000","last_peered":"2024-04-14T08:35:34.978803+0000","last_clean":"2024-04-14T08:35:34.978803+0000","last_became_active":"2024-04-14T08:35:34.973836+0000","last_became_peered":"2024-04-14T08:35:34.973836+0000","last_unstale":"2024-04-14T08:35:34.978803+0000","last_undegraded":"2024-04-14T08:35:34.978803+0000","last_fullsized":"2024-04-14T08:35:34.978803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:13.886435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.982701+0000","last_change":"2024-04-14T08:35:33.973956+0000","last_active":"2024-04-14T08:35:34.982701+0000","last_peered":"2024-04-14T08:35:34.982701+0000","last_clean":"2024-04-14T08:35:34.982701+0000","last_became_active":"2024-04-14T08:35:33.973591+0000","last_became_peered":"2024-04-14T08:35:33.973591+0000","last_unstale":"2024-04-14T08:35:34.982701+0000","last_undegraded":"2024-04-14T08:35:34.982701+0000","last_fullsized":"2024-04-14T08:35:34.982701+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:05.086489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984186+0000","last_change":"2024-04-14T08:35:34.971220+0000","last_active":"2024-04-14T08:35:34.984186+0000","last_peered":"2024-04-14T08:35:34.984186+0000","last_clean":"2024-04-14T08:35:34.984186+0000","last_became_active":"2024-04-14T08:35:34.970952+0000","last_became_peered":"2024-04-14T08:35:34.970952+0000","last_unstale":"2024-04-14T08:35:34.984186+0000","last_undegraded":"2024-04-14T08:35:34.984186+0000","last_fullsized":"2024-04-14T08:35:34.984186+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:11:56.878423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.978535+0000","last_change":"2024-04-14T08:35:33.968291+0000","last_active":"2024-04-14T08:35:34.978535+0000","last_peered":"2024-04-14T08:35:34.978535+0000","last_clean":"2024-04-14T08:35:34.978535+0000","last_became_active":"2024-04-14T08:35:33.968006+0000","last_became_peered":"2024-04-14T08:35:33.968006+0000","last_unstale":"2024-04-14T08:35:34.978535+0000","last_undegraded":"2024-04-14T08:35:34.978535+0000","last_fullsized":"2024-04-14T08:35:34.978535+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:04:24.732447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987055+0000","last_change":"2024-04-14T08:35:34.982650+0000","last_active":"2024-04-14T08:35:34.987055+0000","last_peered":"2024-04-14T08:35:34.987055+0000","last_clean":"2024-04-14T08:35:34.987055+0000","last_became_active":"2024-04-14T08:35:34.982341+0000","last_became_peered":"2024-04-14T08:35:34.982341+0000","last_unstale":"2024-04-14T08:35:34.987055+0000","last_undegraded":"2024-04-14T08:35:34.987055+0000","last_fullsized":"2024-04-14T08:35:34.987055+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:54.240477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.957620+0000","last_change":"2024-04-14T08:35:33.970786+0000","last_active":"2024-04-14T08:35:34.957620+0000","last_peered":"2024-04-14T08:35:34.957620+0000","last_clean":"2024-04-14T08:35:34.957620+0000","last_became_active":"2024-04-14T08:35:33.970517+0000","last_became_peered":"2024-04-14T08:35:33.970517+0000","last_unstale":"2024-04-14T08:35:34.957620+0000","last_undegraded":"2024-04-14T08:35:34.957620+0000","last_fullsized":"2024-04-14T08:35:34.957620+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:30:33.901631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.973260+0000","last_change":"2024-04-14T08:35:34.970678+0000","last_active":"2024-04-14T08:35:34.973260+0000","last_peered":"2024-04-14T08:35:34.973260+0000","last_clean":"2024-04-14T08:35:34.973260+0000","last_became_active":"2024-04-14T08:35:34.970392+0000","last_became_peered":"2024-04-14T08:35:34.970392+0000","last_unstale":"2024-04-14T08:35:34.973260+0000","last_undegraded":"2024-04-14T08:35:34.973260+0000","last_fullsized":"2024-04-14T08:35:34.973260+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:37:45.979299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.965867+0000","last_change":"2024-04-14T08:35:33.971081+0000","last_active":"2024-04-14T08:35:34.965867+0000","last_peered":"2024-04-14T08:35:34.965867+0000","last_clean":"2024-04-14T08:35:34.965867+0000","last_became_active":"2024-04-14T08:35:33.970230+0000","last_became_peered":"2024-04-14T08:35:33.970230+0000","last_unstale":"2024-04-14T08:35:34.965867+0000","last_undegraded":"2024-04-14T08:35:34.965867+0000","last_fullsized":"2024-04-14T08:35:34.965867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:07:21.904319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984376+0000","last_change":"2024-04-14T08:35:34.977569+0000","last_active":"2024-04-14T08:35:34.984376+0000","last_peered":"2024-04-14T08:35:34.984376+0000","last_clean":"2024-04-14T08:35:34.984376+0000","last_became_active":"2024-04-14T08:35:34.977059+0000","last_became_peered":"2024-04-14T08:35:34.977059+0000","last_unstale":"2024-04-14T08:35:34.984376+0000","last_undegraded":"2024-04-14T08:35:34.984376+0000","last_fullsized":"2024-04-14T08:35:34.984376+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:24:44.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.976094+0000","last_change":"2024-04-14T08:35:33.973678+0000","last_active":"2024-04-14T08:35:35.976094+0000","last_peered":"2024-04-14T08:35:35.976094+0000","last_clean":"2024-04-14T08:35:35.976094+0000","last_became_active":"2024-04-14T08:35:33.973406+0000","last_became_peered":"2024-04-14T08:35:33.973406+0000","last_unstale":"2024-04-14T08:35:35.976094+0000","last_undegraded":"2024-04-14T08:35:35.976094+0000","last_fullsized":"2024-04-14T08:35:35.976094+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:50:24.884541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.982384+0000","last_change":"2024-04-14T08:35:34.977003+0000","last_active":"2024-04-14T08:35:34.982384+0000","last_peered":"2024-04-14T08:35:34.982384+0000","last_clean":"2024-04-14T08:35:34.982384+0000","last_became_active":"2024-04-14T08:35:34.976733+0000","last_became_peered":"2024-04-14T08:35:34.976733+0000","last_unstale":"2024-04-14T08:35:34.982384+0000","last_undegraded":"2024-04-14T08:35:34.982384+0000","last_fullsized":"2024-04-14T08:35:34.982384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:07:49.315642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.962234+0000","last_change":"2024-04-14T08:35:33.973363+0000","last_active":"2024-04-14T08:35:34.962234+0000","last_peered":"2024-04-14T08:35:34.962234+0000","last_clean":"2024-04-14T08:35:34.962234+0000","last_became_active":"2024-04-14T08:35:33.973080+0000","last_became_peered":"2024-04-14T08:35:33.973080+0000","last_unstale":"2024-04-14T08:35:34.962234+0000","last_undegraded":"2024-04-14T08:35:34.962234+0000","last_fullsized":"2024-04-14T08:35:34.962234+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:07:21.904319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987092+0000","last_change":"2024-04-14T08:35:34.982715+0000","last_active":"2024-04-14T08:35:34.987092+0000","last_peered":"2024-04-14T08:35:34.987092+0000","last_clean":"2024-04-14T08:35:34.987092+0000","last_became_active":"2024-04-14T08:35:34.982431+0000","last_became_peered":"2024-04-14T08:35:34.982431+0000","last_unstale":"2024-04-14T08:35:34.987092+0000","last_undegraded":"2024-04-14T08:35:34.987092+0000","last_fullsized":"2024-04-14T08:35:34.987092+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:58:05.015142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.976021+0000","last_change":"2024-04-14T08:35:33.967119+0000","last_active":"2024-04-14T08:35:35.976021+0000","last_peered":"2024-04-14T08:35:35.976021+0000","last_clean":"2024-04-14T08:35:35.976021+0000","last_became_active":"2024-04-14T08:35:33.966808+0000","last_became_peered":"2024-04-14T08:35:33.966808+0000","last_unstale":"2024-04-14T08:35:35.976021+0000","last_undegraded":"2024-04-14T08:35:35.976021+0000","last_fullsized":"2024-04-14T08:35:35.976021+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:39:59.215310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.977714+0000","last_change":"2024-04-14T08:35:34.969279+0000","last_active":"2024-04-14T08:35:34.977714+0000","last_peered":"2024-04-14T08:35:34.977714+0000","last_clean":"2024-04-14T08:35:34.977714+0000","last_became_active":"2024-04-14T08:35:34.968998+0000","last_became_peered":"2024-04-14T08:35:34.968998+0000","last_unstale":"2024-04-14T08:35:34.977714+0000","last_undegraded":"2024-04-14T08:35:34.977714+0000","last_fullsized":"2024-04-14T08:35:34.977714+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:04:40.260061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'6","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.992231+0000","last_change":"2024-04-14T08:35:33.981182+0000","last_active":"2024-04-14T08:35:34.992231+0000","last_peered":"2024-04-14T08:35:34.992231+0000","last_clean":"2024-04-14T08:35:34.992231+0000","last_became_active":"2024-04-14T08:35:33.980946+0000","last_became_peered":"2024-04-14T08:35:33.980946+0000","last_unstale":"2024-04-14T08:35:34.992231+0000","last_undegraded":"2024-04-14T08:35:34.992231+0000","last_fullsized":"2024-04-14T08:35:34.992231+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:14:24.176314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.977545+0000","last_change":"2024-04-14T08:35:34.966216+0000","last_active":"2024-04-14T08:35:34.977545+0000","last_peered":"2024-04-14T08:35:34.977545+0000","last_clean":"2024-04-14T08:35:34.977545+0000","last_became_active":"2024-04-14T08:35:34.965935+0000","last_became_peered":"2024-04-14T08:35:34.965935+0000","last_unstale":"2024-04-14T08:35:34.977545+0000","last_undegraded":"2024-04-14T08:35:34.977545+0000","last_fullsized":"2024-04-14T08:35:34.977545+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:15:16.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975972+0000","last_change":"2024-04-14T08:35:33.970680+0000","last_active":"2024-04-14T08:35:35.975972+0000","last_peered":"2024-04-14T08:35:35.975972+0000","last_clean":"2024-04-14T08:35:35.975972+0000","last_became_active":"2024-04-14T08:35:33.970400+0000","last_became_peered":"2024-04-14T08:35:33.970400+0000","last_unstale":"2024-04-14T08:35:35.975972+0000","last_undegraded":"2024-04-14T08:35:35.975972+0000","last_fullsized":"2024-04-14T08:35:35.975972+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:34:22.432838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985677+0000","last_change":"2024-04-14T08:35:34.980477+0000","last_active":"2024-04-14T08:35:34.985677+0000","last_peered":"2024-04-14T08:35:34.985677+0000","last_clean":"2024-04-14T08:35:34.985677+0000","last_became_active":"2024-04-14T08:35:34.980166+0000","last_became_peered":"2024-04-14T08:35:34.980166+0000","last_unstale":"2024-04-14T08:35:34.985677+0000","last_undegraded":"2024-04-14T08:35:34.985677+0000","last_fullsized":"2024-04-14T08:35:34.985677+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:40:00.219050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.966699+0000","last_change":"2024-04-14T08:35:33.973263+0000","last_active":"2024-04-14T08:35:34.966699+0000","last_peered":"2024-04-14T08:35:34.966699+0000","last_clean":"2024-04-14T08:35:34.966699+0000","last_became_active":"2024-04-14T08:35:33.972938+0000","last_became_peered":"2024-04-14T08:35:33.972938+0000","last_unstale":"2024-04-14T08:35:34.966699+0000","last_undegraded":"2024-04-14T08:35:34.966699+0000","last_fullsized":"2024-04-14T08:35:34.966699+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:12:13.917355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.981756+0000","last_change":"2024-04-14T08:35:34.975125+0000","last_active":"2024-04-14T08:35:34.981756+0000","last_peered":"2024-04-14T08:35:34.981756+0000","last_clean":"2024-04-14T08:35:34.981756+0000","last_became_active":"2024-04-14T08:35:34.974828+0000","last_became_peered":"2024-04-14T08:35:34.974828+0000","last_unstale":"2024-04-14T08:35:34.981756+0000","last_undegraded":"2024-04-14T08:35:34.981756+0000","last_fullsized":"2024-04-14T08:35:34.981756+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:04:40.260061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975296+0000","last_change":"2024-04-14T08:35:33.967260+0000","last_active":"2024-04-14T08:35:35.975296+0000","last_peered":"2024-04-14T08:35:35.975296+0000","last_clean":"2024-04-14T08:35:35.975296+0000","last_became_active":"2024-04-14T08:35:33.966990+0000","last_became_peered":"2024-04-14T08:35:33.966990+0000","last_unstale":"2024-04-14T08:35:35.975296+0000","last_undegraded":"2024-04-14T08:35:35.975296+0000","last_fullsized":"2024-04-14T08:35:35.975296+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:15:58.978558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984091+0000","last_change":"2024-04-14T08:35:34.971125+0000","last_active":"2024-04-14T08:35:34.984091+0000","last_peered":"2024-04-14T08:35:34.984091+0000","last_clean":"2024-04-14T08:35:34.984091+0000","last_became_active":"2024-04-14T08:35:34.970856+0000","last_became_peered":"2024-04-14T08:35:34.970856+0000","last_unstale":"2024-04-14T08:35:34.984091+0000","last_undegraded":"2024-04-14T08:35:34.984091+0000","last_fullsized":"2024-04-14T08:35:34.984091+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:03.166145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"60'8","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.980889+0000","last_change":"2024-04-14T08:35:33.967647+0000","last_active":"2024-04-14T08:35:34.980889+0000","last_peered":"2024-04-14T08:35:34.980889+0000","last_clean":"2024-04-14T08:35:34.980889+0000","last_became_active":"2024-04-14T08:35:33.966518+0000","last_became_peered":"2024-04-14T08:35:33.966518+0000","last_unstale":"2024-04-14T08:35:34.980889+0000","last_undegraded":"2024-04-14T08:35:34.980889+0000","last_fullsized":"2024-04-14T08:35:34.980889+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:27:10.703355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975282+0000","last_change":"2024-04-14T08:35:34.973150+0000","last_active":"2024-04-14T08:35:35.975282+0000","last_peered":"2024-04-14T08:35:35.975282+0000","last_clean":"2024-04-14T08:35:35.975282+0000","last_became_active":"2024-04-14T08:35:34.972894+0000","last_became_peered":"2024-04-14T08:35:34.972894+0000","last_unstale":"2024-04-14T08:35:35.975282+0000","last_undegraded":"2024-04-14T08:35:35.975282+0000","last_fullsized":"2024-04-14T08:35:35.975282+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:15:14.910911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985808+0000","last_change":"2024-04-14T08:35:33.974085+0000","last_active":"2024-04-14T08:35:34.985808+0000","last_peered":"2024-04-14T08:35:34.985808+0000","last_clean":"2024-04-14T08:35:34.985808+0000","last_became_active":"2024-04-14T08:35:33.973843+0000","last_became_peered":"2024-04-14T08:35:33.973843+0000","last_unstale":"2024-04-14T08:35:34.985808+0000","last_undegraded":"2024-04-14T08:35:34.985808+0000","last_fullsized":"2024-04-14T08:35:34.985808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:07.764538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985105+0000","last_change":"2024-04-14T08:35:34.980917+0000","last_active":"2024-04-14T08:35:34.985105+0000","last_peered":"2024-04-14T08:35:34.985105+0000","last_clean":"2024-04-14T08:35:34.985105+0000","last_became_active":"2024-04-14T08:35:34.980648+0000","last_became_peered":"2024-04-14T08:35:34.980648+0000","last_unstale":"2024-04-14T08:35:34.985105+0000","last_undegraded":"2024-04-14T08:35:34.985105+0000","last_fullsized":"2024-04-14T08:35:34.985105+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:14:25.180055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975476+0000","last_change":"2024-04-14T08:35:33.965648+0000","last_active":"2024-04-14T08:35:35.975476+0000","last_peered":"2024-04-14T08:35:35.975476+0000","last_clean":"2024-04-14T08:35:35.975476+0000","last_became_active":"2024-04-14T08:35:33.965358+0000","last_became_peered":"2024-04-14T08:35:33.965358+0000","last_unstale":"2024-04-14T08:35:35.975476+0000","last_undegraded":"2024-04-14T08:35:35.975476+0000","last_fullsized":"2024-04-14T08:35:35.975476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:43:36.830667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984966+0000","last_change":"2024-04-14T08:35:34.979662+0000","last_active":"2024-04-14T08:35:34.984966+0000","last_peered":"2024-04-14T08:35:34.984966+0000","last_clean":"2024-04-14T08:35:34.984966+0000","last_became_active":"2024-04-14T08:35:34.979189+0000","last_became_peered":"2024-04-14T08:35:34.979189+0000","last_unstale":"2024-04-14T08:35:34.984966+0000","last_undegraded":"2024-04-14T08:35:34.984966+0000","last_fullsized":"2024-04-14T08:35:34.984966+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:03.166145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.976109+0000","last_change":"2024-04-14T08:35:33.966547+0000","last_active":"2024-04-14T08:35:35.976109+0000","last_peered":"2024-04-14T08:35:35.976109+0000","last_clean":"2024-04-14T08:35:35.976109+0000","last_became_active":"2024-04-14T08:35:33.966268+0000","last_became_peered":"2024-04-14T08:35:33.966268+0000","last_unstale":"2024-04-14T08:35:35.976109+0000","last_undegraded":"2024-04-14T08:35:35.976109+0000","last_fullsized":"2024-04-14T08:35:35.976109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:45:52.647959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985536+0000","last_change":"2024-04-14T08:35:34.975973+0000","last_active":"2024-04-14T08:35:34.985536+0000","last_peered":"2024-04-14T08:35:34.985536+0000","last_clean":"2024-04-14T08:35:34.985536+0000","last_became_active":"2024-04-14T08:35:34.975627+0000","last_became_peered":"2024-04-14T08:35:34.975627+0000","last_unstale":"2024-04-14T08:35:34.985536+0000","last_undegraded":"2024-04-14T08:35:34.985536+0000","last_fullsized":"2024-04-14T08:35:34.985536+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:32:14.300795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.958060+0000","last_change":"2024-04-14T08:35:33.968718+0000","last_active":"2024-04-14T08:35:34.958060+0000","last_peered":"2024-04-14T08:35:34.958060+0000","last_clean":"2024-04-14T08:35:34.958060+0000","last_became_active":"2024-04-14T08:35:33.968417+0000","last_became_peered":"2024-04-14T08:35:33.968417+0000","last_unstale":"2024-04-14T08:35:34.958060+0000","last_undegraded":"2024-04-14T08:35:34.958060+0000","last_fullsized":"2024-04-14T08:35:34.958060+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:40.214958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.979188+0000","last_change":"2024-04-14T08:35:34.967773+0000","last_active":"2024-04-14T08:35:34.979188+0000","last_peered":"2024-04-14T08:35:34.979188+0000","last_clean":"2024-04-14T08:35:34.979188+0000","last_became_active":"2024-04-14T08:35:34.966365+0000","last_became_peered":"2024-04-14T08:35:34.966365+0000","last_unstale":"2024-04-14T08:35:34.979188+0000","last_undegraded":"2024-04-14T08:35:34.979188+0000","last_fullsized":"2024-04-14T08:35:34.979188+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:37:45.979299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985603+0000","last_change":"2024-04-14T08:35:33.970189+0000","last_active":"2024-04-14T08:35:34.985603+0000","last_peered":"2024-04-14T08:35:34.985603+0000","last_clean":"2024-04-14T08:35:34.985603+0000","last_became_active":"2024-04-14T08:35:33.969936+0000","last_became_peered":"2024-04-14T08:35:33.969936+0000","last_unstale":"2024-04-14T08:35:34.985603+0000","last_undegraded":"2024-04-14T08:35:34.985603+0000","last_fullsized":"2024-04-14T08:35:34.985603+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:17:21.187424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983660+0000","last_change":"2024-04-14T08:35:34.981851+0000","last_active":"2024-04-14T08:35:34.983660+0000","last_peered":"2024-04-14T08:35:34.983660+0000","last_clean":"2024-04-14T08:35:34.983660+0000","last_became_active":"2024-04-14T08:35:34.981299+0000","last_became_peered":"2024-04-14T08:35:34.981299+0000","last_unstale":"2024-04-14T08:35:34.983660+0000","last_undegraded":"2024-04-14T08:35:34.983660+0000","last_fullsized":"2024-04-14T08:35:34.983660+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:45:53.651699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.966630+0000","last_change":"2024-04-14T08:35:33.973659+0000","last_active":"2024-04-14T08:35:34.966630+0000","last_peered":"2024-04-14T08:35:34.966630+0000","last_clean":"2024-04-14T08:35:34.966630+0000","last_became_active":"2024-04-14T08:35:33.972363+0000","last_became_peered":"2024-04-14T08:35:33.972363+0000","last_unstale":"2024-04-14T08:35:34.966630+0000","last_undegraded":"2024-04-14T08:35:34.966630+0000","last_fullsized":"2024-04-14T08:35:34.966630+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:58:08.947797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985073+0000","last_change":"2024-04-14T08:35:34.980777+0000","last_active":"2024-04-14T08:35:34.985073+0000","last_peered":"2024-04-14T08:35:34.985073+0000","last_clean":"2024-04-14T08:35:34.985073+0000","last_became_active":"2024-04-14T08:35:34.980501+0000","last_became_peered":"2024-04-14T08:35:34.980501+0000","last_unstale":"2024-04-14T08:35:34.985073+0000","last_undegraded":"2024-04-14T08:35:34.985073+0000","last_fullsized":"2024-04-14T08:35:34.985073+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:33:26.666479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.961098+0000","last_change":"2024-04-14T08:35:33.967333+0000","last_active":"2024-04-14T08:35:34.961098+0000","last_peered":"2024-04-14T08:35:34.961098+0000","last_clean":"2024-04-14T08:35:34.961098+0000","last_became_active":"2024-04-14T08:35:33.967077+0000","last_became_peered":"2024-04-14T08:35:33.967077+0000","last_unstale":"2024-04-14T08:35:34.961098+0000","last_undegraded":"2024-04-14T08:35:34.961098+0000","last_fullsized":"2024-04-14T08:35:34.961098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:04:24.732447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983568+0000","last_change":"2024-04-14T08:35:34.981326+0000","last_active":"2024-04-14T08:35:34.983568+0000","last_peered":"2024-04-14T08:35:34.983568+0000","last_clean":"2024-04-14T08:35:34.983568+0000","last_became_active":"2024-04-14T08:35:34.980780+0000","last_became_peered":"2024-04-14T08:35:34.980780+0000","last_unstale":"2024-04-14T08:35:34.983568+0000","last_undegraded":"2024-04-14T08:35:34.983568+0000","last_fullsized":"2024-04-14T08:35:34.983568+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:21:27.827950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985425+0000","last_change":"2024-04-14T08:35:33.975602+0000","last_active":"2024-04-14T08:35:34.985425+0000","last_peered":"2024-04-14T08:35:34.985425+0000","last_clean":"2024-04-14T08:35:34.985425+0000","last_became_active":"2024-04-14T08:35:33.975316+0000","last_became_peered":"2024-04-14T08:35:33.975316+0000","last_unstale":"2024-04-14T08:35:34.985425+0000","last_undegraded":"2024-04-14T08:35:34.985425+0000","last_fullsized":"2024-04-14T08:35:34.985425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:51:21.880044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.978919+0000","last_change":"2024-04-14T08:35:34.974091+0000","last_active":"2024-04-14T08:35:34.978919+0000","last_peered":"2024-04-14T08:35:34.978919+0000","last_clean":"2024-04-14T08:35:34.978919+0000","last_became_active":"2024-04-14T08:35:34.973788+0000","last_became_peered":"2024-04-14T08:35:34.973788+0000","last_unstale":"2024-04-14T08:35:34.978919+0000","last_undegraded":"2024-04-14T08:35:34.978919+0000","last_fullsized":"2024-04-14T08:35:34.978919+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:45:18.866747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.966964+0000","last_change":"2024-04-14T08:35:33.974207+0000","last_active":"2024-04-14T08:35:34.966964+0000","last_peered":"2024-04-14T08:35:34.966964+0000","last_clean":"2024-04-14T08:35:34.966964+0000","last_became_active":"2024-04-14T08:35:33.973920+0000","last_became_peered":"2024-04-14T08:35:33.973920+0000","last_unstale":"2024-04-14T08:35:34.966964+0000","last_undegraded":"2024-04-14T08:35:34.966964+0000","last_fullsized":"2024-04-14T08:35:34.966964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:39:33.433883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983167+0000","last_change":"2024-04-14T08:35:34.981037+0000","last_active":"2024-04-14T08:35:34.983167+0000","last_peered":"2024-04-14T08:35:34.983167+0000","last_clean":"2024-04-14T08:35:34.983167+0000","last_became_active":"2024-04-14T08:35:34.980776+0000","last_became_peered":"2024-04-14T08:35:34.980776+0000","last_unstale":"2024-04-14T08:35:34.983167+0000","last_undegraded":"2024-04-14T08:35:34.983167+0000","last_fullsized":"2024-04-14T08:35:34.983167+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:15:08.923865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.961521+0000","last_change":"2024-04-14T08:35:33.973611+0000","last_active":"2024-04-14T08:35:34.961521+0000","last_peered":"2024-04-14T08:35:34.961521+0000","last_clean":"2024-04-14T08:35:34.961521+0000","last_became_active":"2024-04-14T08:35:33.972704+0000","last_became_peered":"2024-04-14T08:35:33.972704+0000","last_unstale":"2024-04-14T08:35:34.961521+0000","last_undegraded":"2024-04-14T08:35:34.961521+0000","last_fullsized":"2024-04-14T08:35:34.961521+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:05.086489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.981667+0000","last_change":"2024-04-14T08:35:34.973414+0000","last_active":"2024-04-14T08:35:34.981667+0000","last_peered":"2024-04-14T08:35:34.981667+0000","last_clean":"2024-04-14T08:35:34.981667+0000","last_became_active":"2024-04-14T08:35:34.973124+0000","last_became_peered":"2024-04-14T08:35:34.973124+0000","last_unstale":"2024-04-14T08:35:34.981667+0000","last_undegraded":"2024-04-14T08:35:34.981667+0000","last_fullsized":"2024-04-14T08:35:34.981667+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:54:56.621692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.964628+0000","last_change":"2024-04-14T08:35:33.976356+0000","last_active":"2024-04-14T08:35:34.964628+0000","last_peered":"2024-04-14T08:35:34.964628+0000","last_clean":"2024-04-14T08:35:34.964628+0000","last_became_active":"2024-04-14T08:35:33.976063+0000","last_became_peered":"2024-04-14T08:35:33.976063+0000","last_unstale":"2024-04-14T08:35:34.964628+0000","last_undegraded":"2024-04-14T08:35:34.964628+0000","last_fullsized":"2024-04-14T08:35:34.964628+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:53.236736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987321+0000","last_change":"2024-04-14T08:35:34.985322+0000","last_active":"2024-04-14T08:35:34.987321+0000","last_peered":"2024-04-14T08:35:34.987321+0000","last_clean":"2024-04-14T08:35:34.987321+0000","last_became_active":"2024-04-14T08:35:34.985066+0000","last_became_peered":"2024-04-14T08:35:34.985066+0000","last_unstale":"2024-04-14T08:35:34.987321+0000","last_undegraded":"2024-04-14T08:35:34.987321+0000","last_fullsized":"2024-04-14T08:35:34.987321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:03.166145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.976069+0000","last_change":"2024-04-14T08:35:33.970482+0000","last_active":"2024-04-14T08:35:35.976069+0000","last_peered":"2024-04-14T08:35:35.976069+0000","last_clean":"2024-04-14T08:35:35.976069+0000","last_became_active":"2024-04-14T08:35:33.970197+0000","last_became_peered":"2024-04-14T08:35:33.970197+0000","last_unstale":"2024-04-14T08:35:35.976069+0000","last_undegraded":"2024-04-14T08:35:35.976069+0000","last_fullsized":"2024-04-14T08:35:35.976069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:15:07.920124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.971736+0000","last_change":"2024-04-14T08:35:34.966023+0000","last_active":"2024-04-14T08:35:34.971736+0000","last_peered":"2024-04-14T08:35:34.971736+0000","last_clean":"2024-04-14T08:35:34.971736+0000","last_became_active":"2024-04-14T08:35:34.965650+0000","last_became_peered":"2024-04-14T08:35:34.965650+0000","last_unstale":"2024-04-14T08:35:34.971736+0000","last_undegraded":"2024-04-14T08:35:34.971736+0000","last_fullsized":"2024-04-14T08:35:34.971736+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:54:56.621692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.958245+0000","last_change":"2024-04-14T08:35:33.972393+0000","last_active":"2024-04-14T08:35:34.958245+0000","last_peered":"2024-04-14T08:35:34.958245+0000","last_clean":"2024-04-14T08:35:34.958245+0000","last_became_active":"2024-04-14T08:35:33.972129+0000","last_became_peered":"2024-04-14T08:35:33.972129+0000","last_unstale":"2024-04-14T08:35:34.958245+0000","last_undegraded":"2024-04-14T08:35:34.958245+0000","last_fullsized":"2024-04-14T08:35:34.958245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:15:13.907170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984159+0000","last_change":"2024-04-14T08:35:34.972009+0000","last_active":"2024-04-14T08:35:34.984159+0000","last_peered":"2024-04-14T08:35:34.984159+0000","last_clean":"2024-04-14T08:35:34.984159+0000","last_became_active":"2024-04-14T08:35:34.971468+0000","last_became_peered":"2024-04-14T08:35:34.971468+0000","last_unstale":"2024-04-14T08:35:34.984159+0000","last_undegraded":"2024-04-14T08:35:34.984159+0000","last_fullsized":"2024-04-14T08:35:34.984159+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:19:15.795385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.957802+0000","last_change":"2024-04-14T08:35:33.970907+0000","last_active":"2024-04-14T08:35:34.957802+0000","last_peered":"2024-04-14T08:35:34.957802+0000","last_clean":"2024-04-14T08:35:34.957802+0000","last_became_active":"2024-04-14T08:35:33.969919+0000","last_became_peered":"2024-04-14T08:35:33.969919+0000","last_unstale":"2024-04-14T08:35:34.957802+0000","last_undegraded":"2024-04-14T08:35:34.957802+0000","last_fullsized":"2024-04-14T08:35:34.957802+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:17:21.187424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983028+0000","last_change":"2024-04-14T08:35:34.980934+0000","last_active":"2024-04-14T08:35:34.983028+0000","last_peered":"2024-04-14T08:35:34.983028+0000","last_clean":"2024-04-14T08:35:34.983028+0000","last_became_active":"2024-04-14T08:35:34.980683+0000","last_became_peered":"2024-04-14T08:35:34.980683+0000","last_unstale":"2024-04-14T08:35:34.983028+0000","last_undegraded":"2024-04-14T08:35:34.983028+0000","last_fullsized":"2024-04-14T08:35:34.983028+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:41:36.584680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.961159+0000","last_change":"2024-04-14T08:35:33.978148+0000","last_active":"2024-04-14T08:35:34.961159+0000","last_peered":"2024-04-14T08:35:34.961159+0000","last_clean":"2024-04-14T08:35:34.961159+0000","last_became_active":"2024-04-14T08:35:33.977887+0000","last_became_peered":"2024-04-14T08:35:33.977887+0000","last_unstale":"2024-04-14T08:35:34.961159+0000","last_undegraded":"2024-04-14T08:35:34.961159+0000","last_fullsized":"2024-04-14T08:35:34.961159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:33:25.662739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987478+0000","last_change":"2024-04-14T08:35:34.986565+0000","last_active":"2024-04-14T08:35:34.987478+0000","last_peered":"2024-04-14T08:35:34.987478+0000","last_clean":"2024-04-14T08:35:34.987478+0000","last_became_active":"2024-04-14T08:35:34.986150+0000","last_became_peered":"2024-04-14T08:35:34.986150+0000","last_unstale":"2024-04-14T08:35:34.987478+0000","last_undegraded":"2024-04-14T08:35:34.987478+0000","last_fullsized":"2024-04-14T08:35:34.987478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:19:15.795385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.957720+0000","last_change":"2024-04-14T08:35:33.973263+0000","last_active":"2024-04-14T08:35:34.957720+0000","last_peered":"2024-04-14T08:35:34.957720+0000","last_clean":"2024-04-14T08:35:34.957720+0000","last_became_active":"2024-04-14T08:35:33.972625+0000","last_became_peered":"2024-04-14T08:35:33.972625+0000","last_unstale":"2024-04-14T08:35:34.957720+0000","last_undegraded":"2024-04-14T08:35:34.957720+0000","last_fullsized":"2024-04-14T08:35:34.957720+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:37:38.252823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":251,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975109+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:35:35.975109+0000","last_peered":"2024-04-14T08:35:35.975109+0000","last_clean":"2024-04-14T08:35:35.975109+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:35:35.975109+0000","last_undegraded":"2024-04-14T08:35:35.975109+0000","last_fullsized":"2024-04-14T08:35:35.975109+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28332,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343508,"statfs":{"total":96636764160,"available":96607752192,"internally_reserved":0,"allocated":1335296,"data_stored":914347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":316036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":316036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":737280,"data_stored":324815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":316036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":316036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":316036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28300,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343540,"statfs":{"total":96636764160,"available":96607784960,"internally_reserved":0,"allocated":1306624,"data_stored":906404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:35:37.942 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:35:38.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.078+0000 7f340a31e640 1 Processor -- start 2024-04-14T08:35:38.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.078+0000 7f340a31e640 1 -- start start 2024-04-14T08:35:38.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.078+0000 7f340a31e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f340415e5b0 0x7f34040ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.079+0000 7f340a31e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34040b41d0 con 0x7f34040729b0 2024-04-14T08:35:38.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.079+0000 7f340a31e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34040aebe0 con 0x7f340406c530 2024-04-14T08:35:38.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.079+0000 7f340a31e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f34040b4470 con 0x7f340415e5b0 2024-04-14T08:35:38.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.079+0000 7f3403fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f340415e5b0 0x7f34040ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.079+0000 7f34037fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f34040729b0 0x7f340406dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59910/0 (socket says 172.21.15.172:59910) 2024-04-14T08:35:38.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.079+0000 7f34037fe640 1 -- 172.21.15.172:0/2592713619 learned_addr learned my addr 172.21.15.172:0/2592713619 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:38.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.079+0000 7f3403fff640 1 -- 172.21.15.172:0/2592713619 >> v1:172.21.15.177:6789/0 conn(0x7f340406c530 legacy=0x7f34040b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.079+0000 7f3403fff640 1 -- 172.21.15.172:0/2592713619 >> v1:172.21.15.172:6789/0 conn(0x7f34040729b0 legacy=0x7f340406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.079+0000 7f3403fff640 1 -- 172.21.15.172:0/2592713619 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34040ae8d0 con 0x7f340415e5b0 2024-04-14T08:35:38.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.079+0000 7f3402ffd640 1 -- 172.21.15.172:0/2592713619 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1154871935 0 0) 0x7f34040b41d0 con 0x7f34040729b0 2024-04-14T08:35:38.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.080+0000 7f3403fff640 1 --2- 172.21.15.172:0/2592713619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f340415e5b0 0x7f34040ae390 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f33f40097d0 tx=0x7f33f402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234fa8bd786066cc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:38.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.080+0000 7f3402ffd640 1 -- 172.21.15.172:0/2592713619 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4034070 con 0x7f340415e5b0 2024-04-14T08:35:38.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.080+0000 7f3402ffd640 1 -- 172.21.15.172:0/2592713619 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f4030610 con 0x7f340415e5b0 2024-04-14T08:35:38.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.080+0000 7f3402ffd640 1 -- 172.21.15.172:0/2592713619 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f40309e0 con 0x7f340415e5b0 2024-04-14T08:35:38.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.080+0000 7f340a31e640 1 -- 172.21.15.172:0/2592713619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f340415e5b0 msgr2=0x7f34040ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.080+0000 7f340a31e640 1 --2- 172.21.15.172:0/2592713619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f340415e5b0 0x7f34040ae390 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f33f40097d0 tx=0x7f33f402b960 comp rx=0 tx=0).stop 2024-04-14T08:35:38.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.080+0000 7f340a31e640 1 -- 172.21.15.172:0/2592713619 shutdown_connections 2024-04-14T08:35:38.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.080+0000 7f340a31e640 1 --2- 172.21.15.172:0/2592713619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f340415e5b0 0x7f34040ae390 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.080+0000 7f340a31e640 1 -- 172.21.15.172:0/2592713619 >> 172.21.15.172:0/2592713619 conn(0x7f3404071ba0 msgr2=0x7f3404071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:38.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.080+0000 7f340a31e640 1 -- 172.21.15.172:0/2592713619 shutdown_connections 2024-04-14T08:35:38.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.081+0000 7f340a31e640 1 -- 172.21.15.172:0/2592713619 wait complete. 2024-04-14T08:35:38.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.082+0000 7f340a31e640 1 Processor -- start 2024-04-14T08:35:38.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.082+0000 7f340a31e640 1 -- start start 2024-04-14T08:35:38.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.082+0000 7f340a31e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f340406c530 0x7f3404142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.082+0000 7f340a31e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34040729b0 0x7f34040cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.082+0000 7f340a31e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f340415e5b0 0x7f34041613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.082+0000 7f340a31e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34040af040 con 0x7f34040729b0 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.082+0000 7f340a31e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f34040aeec0 con 0x7f340406c530 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.082+0000 7f340a31e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f34040b5380 con 0x7f340415e5b0 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f34037fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34040729b0 0x7f34040cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f34088bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f340415e5b0 0x7f34041613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f34037fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34040729b0 0x7f34040cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42744/0 (socket says 172.21.15.172:42744) 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f34088bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f340415e5b0 0x7f34041613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:42240/0 (socket says 172.21.15.172:42240) 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f34037fe640 1 -- 172.21.15.172:0/1954883772 learned_addr learned my addr 172.21.15.172:0/1954883772 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f3403fff640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f340406c530 0x7f3404142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f34037fe640 1 -- 172.21.15.172:0/1954883772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f340415e5b0 msgr2=0x7f34041613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f34037fe640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f340415e5b0 0x7f34041613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f34037fe640 1 -- 172.21.15.172:0/1954883772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f340406c530 msgr2=0x7f3404142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f34037fe640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f340406c530 0x7f3404142b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f34037fe640 1 -- 172.21.15.172:0/1954883772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34041618f0 con 0x7f34040729b0 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f34037fe640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34040729b0 0x7f34040cd6b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f33f800c4d0 tx=0x7f33f800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:38.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f3400ff9640 1 -- 172.21.15.172:0/1954883772 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8018070 con 0x7f34040729b0 2024-04-14T08:35:38.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f3400ff9640 1 -- 172.21.15.172:0/1954883772 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f80074e0 con 0x7f34040729b0 2024-04-14T08:35:38.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f340a31e640 1 -- 172.21.15.172:0/1954883772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3404157520 con 0x7f34040729b0 2024-04-14T08:35:38.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f3400ff9640 1 -- 172.21.15.172:0/1954883772 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8004a20 con 0x7f34040729b0 2024-04-14T08:35:38.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.083+0000 7f340a31e640 1 -- 172.21.15.172:0/1954883772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3404157ae0 con 0x7f34040729b0 2024-04-14T08:35:38.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.084+0000 7f340a31e640 1 -- 172.21.15.172:0/1954883772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3404068520 con 0x7f34040729b0 2024-04-14T08:35:38.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.085+0000 7f3400ff9640 1 -- 172.21.15.172:0/1954883772 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33f801b070 con 0x7f34040729b0 2024-04-14T08:35:38.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.085+0000 7f3400ff9640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33f006c8f0 0x7f33f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.085+0000 7f3400ff9640 1 -- 172.21.15.172:0/1954883772 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f33f808aea0 con 0x7f34040729b0 2024-04-14T08:35:38.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.085+0000 7f3403fff640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33f006c8f0 0x7f33f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.086+0000 7f3403fff640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33f006c8f0 0x7f33f006edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f33f4036000 tx=0x7f33f403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:38.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.090+0000 7f3400ff9640 1 -- 172.21.15.172:0/1954883772 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f33f8058b80 con 0x7f34040729b0 2024-04-14T08:35:38.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.262+0000 7f340a31e640 1 -- 172.21.15.172:0/1954883772 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3404072790 con 0x7f33f006c8f0 2024-04-14T08:35:38.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.268+0000 7f3400ff9640 1 -- 172.21.15.172:0/1954883772 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f3404072790 con 0x7f33f006c8f0 2024-04-14T08:35:38.271 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:35:38.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.271+0000 7f340a31e640 1 -- 172.21.15.172:0/1954883772 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33f006c8f0 msgr2=0x7f33f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.271+0000 7f340a31e640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33f006c8f0 0x7f33f006edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f33f4036000 tx=0x7f33f403b040 comp rx=0 tx=0).stop 2024-04-14T08:35:38.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.271+0000 7f340a31e640 1 -- 172.21.15.172:0/1954883772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34040729b0 msgr2=0x7f34040cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.271+0000 7f340a31e640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34040729b0 0x7f34040cd6b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f33f800c4d0 tx=0x7f33f800c990 comp rx=0 tx=0).stop 2024-04-14T08:35:38.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.271+0000 7f340a31e640 1 -- 172.21.15.172:0/1954883772 shutdown_connections 2024-04-14T08:35:38.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.271+0000 7f340a31e640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f33f006c8f0 0x7f33f006edb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.271+0000 7f340a31e640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f340415e5b0 0x7f34041613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.271+0000 7f340a31e640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34040729b0 0x7f34040cd6b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.271+0000 7f340a31e640 1 --2- 172.21.15.172:0/1954883772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f340406c530 0x7f3404142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.271+0000 7f340a31e640 1 -- 172.21.15.172:0/1954883772 >> 172.21.15.172:0/1954883772 conn(0x7f3404071ba0 msgr2=0x7f34040b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:38.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.271+0000 7f340a31e640 1 -- 172.21.15.172:0/1954883772 shutdown_connections 2024-04-14T08:35:38.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.271+0000 7f340a31e640 1 -- 172.21.15.172:0/1954883772 wait complete. 2024-04-14T08:35:38.283 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-04-14T08:35:37.151434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222980,"kb_used_data":6780,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751740,"statfs":{"total":773094113280,"available":772865781760,"internally_reserved":0,"allocated":6942720,"data_stored":3725746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.202597"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984677+0000","last_change":"2024-04-14T08:35:34.980641+0000","last_active":"2024-04-14T08:35:34.984677+0000","last_peered":"2024-04-14T08:35:34.984677+0000","last_clean":"2024-04-14T08:35:34.984677+0000","last_became_active":"2024-04-14T08:35:34.980371+0000","last_became_peered":"2024-04-14T08:35:34.980371+0000","last_unstale":"2024-04-14T08:35:34.984677+0000","last_undegraded":"2024-04-14T08:35:34.984677+0000","last_fullsized":"2024-04-14T08:35:34.984677+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:58:09.951538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984648+0000","last_change":"2024-04-14T08:35:34.981257+0000","last_active":"2024-04-14T08:35:34.984648+0000","last_peered":"2024-04-14T08:35:34.984648+0000","last_clean":"2024-04-14T08:35:34.984648+0000","last_became_active":"2024-04-14T08:35:34.980970+0000","last_became_peered":"2024-04-14T08:35:34.980970+0000","last_unstale":"2024-04-14T08:35:34.984648+0000","last_undegraded":"2024-04-14T08:35:34.984648+0000","last_fullsized":"2024-04-14T08:35:34.984648+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:12:14.921095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.976003+0000","last_change":"2024-04-14T08:35:34.975665+0000","last_active":"2024-04-14T08:35:35.976003+0000","last_peered":"2024-04-14T08:35:35.976003+0000","last_clean":"2024-04-14T08:35:35.976003+0000","last_became_active":"2024-04-14T08:35:34.974503+0000","last_became_peered":"2024-04-14T08:35:34.974503+0000","last_unstale":"2024-04-14T08:35:35.976003+0000","last_undegraded":"2024-04-14T08:35:35.976003+0000","last_fullsized":"2024-04-14T08:35:35.976003+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:21:27.827950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.986149+0000","last_change":"2024-04-14T08:35:34.979055+0000","last_active":"2024-04-14T08:35:34.986149+0000","last_peered":"2024-04-14T08:35:34.986149+0000","last_clean":"2024-04-14T08:35:34.986149+0000","last_became_active":"2024-04-14T08:35:34.978754+0000","last_became_peered":"2024-04-14T08:35:34.978754+0000","last_unstale":"2024-04-14T08:35:34.986149+0000","last_undegraded":"2024-04-14T08:35:34.986149+0000","last_fullsized":"2024-04-14T08:35:34.986149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:54.240477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.988030+0000","last_change":"2024-04-14T08:35:34.986822+0000","last_active":"2024-04-14T08:35:34.988030+0000","last_peered":"2024-04-14T08:35:34.988030+0000","last_clean":"2024-04-14T08:35:34.988030+0000","last_became_active":"2024-04-14T08:35:34.986521+0000","last_became_peered":"2024-04-14T08:35:34.986521+0000","last_unstale":"2024-04-14T08:35:34.988030+0000","last_undegraded":"2024-04-14T08:35:34.988030+0000","last_fullsized":"2024-04-14T08:35:34.988030+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:24:44.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.978802+0000","last_change":"2024-04-14T08:35:34.972317+0000","last_active":"2024-04-14T08:35:34.978802+0000","last_peered":"2024-04-14T08:35:34.978802+0000","last_clean":"2024-04-14T08:35:34.978802+0000","last_became_active":"2024-04-14T08:35:34.968042+0000","last_became_peered":"2024-04-14T08:35:34.968042+0000","last_unstale":"2024-04-14T08:35:34.978802+0000","last_undegraded":"2024-04-14T08:35:34.978802+0000","last_fullsized":"2024-04-14T08:35:34.978802+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:05:01.671432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.979008+0000","last_change":"2024-04-14T08:35:34.975125+0000","last_active":"2024-04-14T08:35:34.979008+0000","last_peered":"2024-04-14T08:35:34.979008+0000","last_clean":"2024-04-14T08:35:34.979008+0000","last_became_active":"2024-04-14T08:35:34.974408+0000","last_became_peered":"2024-04-14T08:35:34.974408+0000","last_unstale":"2024-04-14T08:35:34.979008+0000","last_undegraded":"2024-04-14T08:35:34.979008+0000","last_fullsized":"2024-04-14T08:35:34.979008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:00:12.295396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.982994+0000","last_change":"2024-04-14T08:35:34.978660+0000","last_active":"2024-04-14T08:35:34.982994+0000","last_peered":"2024-04-14T08:35:34.982994+0000","last_clean":"2024-04-14T08:35:34.982994+0000","last_became_active":"2024-04-14T08:35:34.977718+0000","last_became_peered":"2024-04-14T08:35:34.977718+0000","last_unstale":"2024-04-14T08:35:34.982994+0000","last_undegraded":"2024-04-14T08:35:34.982994+0000","last_fullsized":"2024-04-14T08:35:34.982994+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:43:37.834408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.976149+0000","last_change":"2024-04-14T08:35:34.976790+0000","last_active":"2024-04-14T08:35:35.976149+0000","last_peered":"2024-04-14T08:35:35.976149+0000","last_clean":"2024-04-14T08:35:35.976149+0000","last_became_active":"2024-04-14T08:35:34.976512+0000","last_became_peered":"2024-04-14T08:35:34.976512+0000","last_unstale":"2024-04-14T08:35:35.976149+0000","last_undegraded":"2024-04-14T08:35:35.976149+0000","last_fullsized":"2024-04-14T08:35:35.976149+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:54:12.148162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.974967+0000","last_change":"2024-04-14T08:35:34.977317+0000","last_active":"2024-04-14T08:35:35.974967+0000","last_peered":"2024-04-14T08:35:35.974967+0000","last_clean":"2024-04-14T08:35:35.974967+0000","last_became_active":"2024-04-14T08:35:34.977032+0000","last_became_peered":"2024-04-14T08:35:34.977032+0000","last_unstale":"2024-04-14T08:35:35.974967+0000","last_undegraded":"2024-04-14T08:35:35.974967+0000","last_fullsized":"2024-04-14T08:35:35.974967+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:38:32.452020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985060+0000","last_change":"2024-04-14T08:35:34.980617+0000","last_active":"2024-04-14T08:35:34.985060+0000","last_peered":"2024-04-14T08:35:34.985060+0000","last_clean":"2024-04-14T08:35:34.985060+0000","last_became_active":"2024-04-14T08:35:34.980349+0000","last_became_peered":"2024-04-14T08:35:34.980349+0000","last_unstale":"2024-04-14T08:35:34.985060+0000","last_undegraded":"2024-04-14T08:35:34.985060+0000","last_fullsized":"2024-04-14T08:35:34.985060+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:11:56.878423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.986427+0000","last_change":"2024-04-14T08:35:34.984084+0000","last_active":"2024-04-14T08:35:34.986427+0000","last_peered":"2024-04-14T08:35:34.986427+0000","last_clean":"2024-04-14T08:35:34.986427+0000","last_became_active":"2024-04-14T08:35:34.983802+0000","last_became_peered":"2024-04-14T08:35:34.983802+0000","last_unstale":"2024-04-14T08:35:34.986427+0000","last_undegraded":"2024-04-14T08:35:34.986427+0000","last_fullsized":"2024-04-14T08:35:34.986427+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:15:59.982298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.988562+0000","last_change":"2024-04-14T08:35:34.987874+0000","last_active":"2024-04-14T08:35:34.988562+0000","last_peered":"2024-04-14T08:35:34.988562+0000","last_clean":"2024-04-14T08:35:34.988562+0000","last_became_active":"2024-04-14T08:35:34.987562+0000","last_became_peered":"2024-04-14T08:35:34.987562+0000","last_unstale":"2024-04-14T08:35:34.988562+0000","last_undegraded":"2024-04-14T08:35:34.988562+0000","last_fullsized":"2024-04-14T08:35:34.988562+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:58:45.944045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975631+0000","last_change":"2024-04-14T08:35:34.973945+0000","last_active":"2024-04-14T08:35:35.975631+0000","last_peered":"2024-04-14T08:35:35.975631+0000","last_clean":"2024-04-14T08:35:35.975631+0000","last_became_active":"2024-04-14T08:35:34.973641+0000","last_became_peered":"2024-04-14T08:35:34.973641+0000","last_unstale":"2024-04-14T08:35:35.975631+0000","last_undegraded":"2024-04-14T08:35:35.975631+0000","last_fullsized":"2024-04-14T08:35:35.975631+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:37:45.979299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975118+0000","last_change":"2024-04-14T08:35:34.982413+0000","last_active":"2024-04-14T08:35:35.975118+0000","last_peered":"2024-04-14T08:35:35.975118+0000","last_clean":"2024-04-14T08:35:35.975118+0000","last_became_active":"2024-04-14T08:35:34.982131+0000","last_became_peered":"2024-04-14T08:35:34.982131+0000","last_unstale":"2024-04-14T08:35:35.975118+0000","last_undegraded":"2024-04-14T08:35:35.975118+0000","last_fullsized":"2024-04-14T08:35:35.975118+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:31:26.383953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983058+0000","last_change":"2024-04-14T08:35:34.980705+0000","last_active":"2024-04-14T08:35:34.983058+0000","last_peered":"2024-04-14T08:35:34.983058+0000","last_clean":"2024-04-14T08:35:34.983058+0000","last_became_active":"2024-04-14T08:35:34.980387+0000","last_became_peered":"2024-04-14T08:35:34.980387+0000","last_unstale":"2024-04-14T08:35:34.983058+0000","last_undegraded":"2024-04-14T08:35:34.983058+0000","last_fullsized":"2024-04-14T08:35:34.983058+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:23:11.932844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984986+0000","last_change":"2024-04-14T08:35:34.982825+0000","last_active":"2024-04-14T08:35:34.984986+0000","last_peered":"2024-04-14T08:35:34.984986+0000","last_clean":"2024-04-14T08:35:34.984986+0000","last_became_active":"2024-04-14T08:35:34.982497+0000","last_became_peered":"2024-04-14T08:35:34.982497+0000","last_unstale":"2024-04-14T08:35:34.984986+0000","last_undegraded":"2024-04-14T08:35:34.984986+0000","last_fullsized":"2024-04-14T08:35:34.984986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:39:34.437624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984943+0000","last_change":"2024-04-14T08:35:34.979449+0000","last_active":"2024-04-14T08:35:34.984943+0000","last_peered":"2024-04-14T08:35:34.984943+0000","last_clean":"2024-04-14T08:35:34.984943+0000","last_became_active":"2024-04-14T08:35:34.978857+0000","last_became_peered":"2024-04-14T08:35:34.978857+0000","last_unstale":"2024-04-14T08:35:34.984943+0000","last_undegraded":"2024-04-14T08:35:34.984943+0000","last_fullsized":"2024-04-14T08:35:34.984943+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:58:05.015142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.981695+0000","last_change":"2024-04-14T08:35:34.974253+0000","last_active":"2024-04-14T08:35:34.981695+0000","last_peered":"2024-04-14T08:35:34.981695+0000","last_clean":"2024-04-14T08:35:34.981695+0000","last_became_active":"2024-04-14T08:35:34.974001+0000","last_became_peered":"2024-04-14T08:35:34.974001+0000","last_unstale":"2024-04-14T08:35:34.981695+0000","last_undegraded":"2024-04-14T08:35:34.981695+0000","last_fullsized":"2024-04-14T08:35:34.981695+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:15:16.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.978555+0000","last_change":"2024-04-14T08:35:34.975278+0000","last_active":"2024-04-14T08:35:34.978555+0000","last_peered":"2024-04-14T08:35:34.978555+0000","last_clean":"2024-04-14T08:35:34.978555+0000","last_became_active":"2024-04-14T08:35:34.974982+0000","last_became_peered":"2024-04-14T08:35:34.974982+0000","last_unstale":"2024-04-14T08:35:34.978555+0000","last_undegraded":"2024-04-14T08:35:34.978555+0000","last_fullsized":"2024-04-14T08:35:34.978555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:05:01.671432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.986261+0000","last_change":"2024-04-14T08:35:34.985102+0000","last_active":"2024-04-14T08:35:34.986261+0000","last_peered":"2024-04-14T08:35:34.986261+0000","last_clean":"2024-04-14T08:35:34.986261+0000","last_became_active":"2024-04-14T08:35:34.984845+0000","last_became_peered":"2024-04-14T08:35:34.984845+0000","last_unstale":"2024-04-14T08:35:34.986261+0000","last_undegraded":"2024-04-14T08:35:34.986261+0000","last_fullsized":"2024-04-14T08:35:34.986261+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:40:39.599055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.967842+0000","last_change":"2024-04-14T08:35:34.965393+0000","last_active":"2024-04-14T08:35:34.967842+0000","last_peered":"2024-04-14T08:35:34.967842+0000","last_clean":"2024-04-14T08:35:34.967842+0000","last_became_active":"2024-04-14T08:35:34.964879+0000","last_became_peered":"2024-04-14T08:35:34.964879+0000","last_unstale":"2024-04-14T08:35:34.967842+0000","last_undegraded":"2024-04-14T08:35:34.967842+0000","last_fullsized":"2024-04-14T08:35:34.967842+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:31:26.383953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.967781+0000","last_change":"2024-04-14T08:35:34.964422+0000","last_active":"2024-04-14T08:35:34.967781+0000","last_peered":"2024-04-14T08:35:34.967781+0000","last_clean":"2024-04-14T08:35:34.967781+0000","last_became_active":"2024-04-14T08:35:34.964139+0000","last_became_peered":"2024-04-14T08:35:34.964139+0000","last_unstale":"2024-04-14T08:35:34.967781+0000","last_undegraded":"2024-04-14T08:35:34.967781+0000","last_fullsized":"2024-04-14T08:35:34.967781+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:38:32.452020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.974528+0000","last_change":"2024-04-14T08:35:34.975927+0000","last_active":"2024-04-14T08:35:35.974528+0000","last_peered":"2024-04-14T08:35:35.974528+0000","last_clean":"2024-04-14T08:35:35.974528+0000","last_became_active":"2024-04-14T08:35:34.974807+0000","last_became_peered":"2024-04-14T08:35:34.974807+0000","last_unstale":"2024-04-14T08:35:35.974528+0000","last_undegraded":"2024-04-14T08:35:35.974528+0000","last_fullsized":"2024-04-14T08:35:35.974528+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:06.090229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983938+0000","last_change":"2024-04-14T08:35:34.981889+0000","last_active":"2024-04-14T08:35:34.983938+0000","last_peered":"2024-04-14T08:35:34.983938+0000","last_clean":"2024-04-14T08:35:34.983938+0000","last_became_active":"2024-04-14T08:35:34.979724+0000","last_became_peered":"2024-04-14T08:35:34.979724+0000","last_unstale":"2024-04-14T08:35:34.983938+0000","last_undegraded":"2024-04-14T08:35:34.983938+0000","last_fullsized":"2024-04-14T08:35:34.983938+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:50:25.888282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983145+0000","last_change":"2024-04-14T08:35:34.980471+0000","last_active":"2024-04-14T08:35:34.983145+0000","last_peered":"2024-04-14T08:35:34.983145+0000","last_clean":"2024-04-14T08:35:34.983145+0000","last_became_active":"2024-04-14T08:35:34.980103+0000","last_became_peered":"2024-04-14T08:35:34.980103+0000","last_unstale":"2024-04-14T08:35:34.983145+0000","last_undegraded":"2024-04-14T08:35:34.983145+0000","last_fullsized":"2024-04-14T08:35:34.983145+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:34:23.436578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.977632+0000","last_change":"2024-04-14T08:35:34.970704+0000","last_active":"2024-04-14T08:35:34.977632+0000","last_peered":"2024-04-14T08:35:34.977632+0000","last_clean":"2024-04-14T08:35:34.977632+0000","last_became_active":"2024-04-14T08:35:34.970429+0000","last_became_peered":"2024-04-14T08:35:34.970429+0000","last_unstale":"2024-04-14T08:35:34.977632+0000","last_undegraded":"2024-04-14T08:35:34.977632+0000","last_fullsized":"2024-04-14T08:35:34.977632+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:33:26.666479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984521+0000","last_change":"2024-04-14T08:35:34.978255+0000","last_active":"2024-04-14T08:35:34.984521+0000","last_peered":"2024-04-14T08:35:34.984521+0000","last_clean":"2024-04-14T08:35:34.984521+0000","last_became_active":"2024-04-14T08:35:34.977956+0000","last_became_peered":"2024-04-14T08:35:34.977956+0000","last_unstale":"2024-04-14T08:35:34.984521+0000","last_undegraded":"2024-04-14T08:35:34.984521+0000","last_fullsized":"2024-04-14T08:35:34.984521+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:55.003593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975859+0000","last_change":"2024-04-14T08:35:34.979185+0000","last_active":"2024-04-14T08:35:35.975859+0000","last_peered":"2024-04-14T08:35:35.975859+0000","last_clean":"2024-04-14T08:35:35.975859+0000","last_became_active":"2024-04-14T08:35:34.978928+0000","last_became_peered":"2024-04-14T08:35:34.978928+0000","last_unstale":"2024-04-14T08:35:35.975859+0000","last_undegraded":"2024-04-14T08:35:35.975859+0000","last_fullsized":"2024-04-14T08:35:35.975859+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:07:49.315642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987629+0000","last_change":"2024-04-14T08:35:34.985793+0000","last_active":"2024-04-14T08:35:34.987629+0000","last_peered":"2024-04-14T08:35:34.987629+0000","last_clean":"2024-04-14T08:35:34.987629+0000","last_became_active":"2024-04-14T08:35:34.985496+0000","last_became_peered":"2024-04-14T08:35:34.985496+0000","last_unstale":"2024-04-14T08:35:34.987629+0000","last_undegraded":"2024-04-14T08:35:34.987629+0000","last_fullsized":"2024-04-14T08:35:34.987629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:46:01.496201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.977416+0000","last_change":"2024-04-14T08:35:34.970027+0000","last_active":"2024-04-14T08:35:34.977416+0000","last_peered":"2024-04-14T08:35:34.977416+0000","last_clean":"2024-04-14T08:35:34.977416+0000","last_became_active":"2024-04-14T08:35:34.969772+0000","last_became_peered":"2024-04-14T08:35:34.969772+0000","last_unstale":"2024-04-14T08:35:34.977416+0000","last_undegraded":"2024-04-14T08:35:34.977416+0000","last_fullsized":"2024-04-14T08:35:34.977416+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:32:14.300795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.979212+0000","last_change":"2024-04-14T08:35:34.972505+0000","last_active":"2024-04-14T08:35:34.979212+0000","last_peered":"2024-04-14T08:35:34.979212+0000","last_clean":"2024-04-14T08:35:34.979212+0000","last_became_active":"2024-04-14T08:35:34.968653+0000","last_became_peered":"2024-04-14T08:35:34.968653+0000","last_unstale":"2024-04-14T08:35:34.979212+0000","last_undegraded":"2024-04-14T08:35:34.979212+0000","last_fullsized":"2024-04-14T08:35:34.979212+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:07:49.315642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987463+0000","last_change":"2024-04-14T08:35:34.986135+0000","last_active":"2024-04-14T08:35:34.987463+0000","last_peered":"2024-04-14T08:35:34.987463+0000","last_clean":"2024-04-14T08:35:34.987463+0000","last_became_active":"2024-04-14T08:35:34.985868+0000","last_became_peered":"2024-04-14T08:35:34.985868+0000","last_unstale":"2024-04-14T08:35:34.987463+0000","last_undegraded":"2024-04-14T08:35:34.987463+0000","last_fullsized":"2024-04-14T08:35:34.987463+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:11:56.878423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"60'4","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987609+0000","last_change":"2024-04-14T08:35:33.975983+0000","last_active":"2024-04-14T08:35:34.987609+0000","last_peered":"2024-04-14T08:35:34.987609+0000","last_clean":"2024-04-14T08:35:34.987609+0000","last_became_active":"2024-04-14T08:35:33.975715+0000","last_became_peered":"2024-04-14T08:35:33.975715+0000","last_unstale":"2024-04-14T08:35:34.987609+0000","last_undegraded":"2024-04-14T08:35:34.987609+0000","last_fullsized":"2024-04-14T08:35:34.987609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:58:04.011401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984833+0000","last_change":"2024-04-14T08:35:34.977214+0000","last_active":"2024-04-14T08:35:34.984833+0000","last_peered":"2024-04-14T08:35:34.984833+0000","last_clean":"2024-04-14T08:35:34.984833+0000","last_became_active":"2024-04-14T08:35:34.976925+0000","last_became_peered":"2024-04-14T08:35:34.976925+0000","last_unstale":"2024-04-14T08:35:34.984833+0000","last_undegraded":"2024-04-14T08:35:34.984833+0000","last_fullsized":"2024-04-14T08:35:34.984833+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:14:25.180055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984268+0000","last_change":"2024-04-14T08:35:33.968042+0000","last_active":"2024-04-14T08:35:34.984268+0000","last_peered":"2024-04-14T08:35:34.984268+0000","last_clean":"2024-04-14T08:35:34.984268+0000","last_became_active":"2024-04-14T08:35:33.967770+0000","last_became_peered":"2024-04-14T08:35:33.967770+0000","last_unstale":"2024-04-14T08:35:34.984268+0000","last_undegraded":"2024-04-14T08:35:34.984268+0000","last_fullsized":"2024-04-14T08:35:34.984268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:15:13.907170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.978774+0000","last_change":"2024-04-14T08:35:34.966969+0000","last_active":"2024-04-14T08:35:34.978774+0000","last_peered":"2024-04-14T08:35:34.978774+0000","last_clean":"2024-04-14T08:35:34.978774+0000","last_became_active":"2024-04-14T08:35:34.966513+0000","last_became_peered":"2024-04-14T08:35:34.966513+0000","last_unstale":"2024-04-14T08:35:34.978774+0000","last_undegraded":"2024-04-14T08:35:34.978774+0000","last_fullsized":"2024-04-14T08:35:34.978774+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:31:26.383953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983714+0000","last_change":"2024-04-14T08:35:33.973904+0000","last_active":"2024-04-14T08:35:34.983714+0000","last_peered":"2024-04-14T08:35:34.983714+0000","last_clean":"2024-04-14T08:35:34.983714+0000","last_became_active":"2024-04-14T08:35:33.973351+0000","last_became_peered":"2024-04-14T08:35:33.973351+0000","last_unstale":"2024-04-14T08:35:34.983714+0000","last_undegraded":"2024-04-14T08:35:34.983714+0000","last_fullsized":"2024-04-14T08:35:34.983714+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:40.214958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975705+0000","last_change":"2024-04-14T08:35:34.981126+0000","last_active":"2024-04-14T08:35:35.975705+0000","last_peered":"2024-04-14T08:35:35.975705+0000","last_clean":"2024-04-14T08:35:35.975705+0000","last_became_active":"2024-04-14T08:35:34.980816+0000","last_became_peered":"2024-04-14T08:35:34.980816+0000","last_unstale":"2024-04-14T08:35:35.975705+0000","last_undegraded":"2024-04-14T08:35:35.975705+0000","last_fullsized":"2024-04-14T08:35:35.975705+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:05:01.671432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985469+0000","last_change":"2024-04-14T08:35:33.966087+0000","last_active":"2024-04-14T08:35:34.985469+0000","last_peered":"2024-04-14T08:35:34.985469+0000","last_clean":"2024-04-14T08:35:34.985469+0000","last_became_active":"2024-04-14T08:35:33.965760+0000","last_became_peered":"2024-04-14T08:35:33.965760+0000","last_unstale":"2024-04-14T08:35:34.985469+0000","last_undegraded":"2024-04-14T08:35:34.985469+0000","last_fullsized":"2024-04-14T08:35:34.985469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:38:31.448280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.978803+0000","last_change":"2024-04-14T08:35:34.974129+0000","last_active":"2024-04-14T08:35:34.978803+0000","last_peered":"2024-04-14T08:35:34.978803+0000","last_clean":"2024-04-14T08:35:34.978803+0000","last_became_active":"2024-04-14T08:35:34.973836+0000","last_became_peered":"2024-04-14T08:35:34.973836+0000","last_unstale":"2024-04-14T08:35:34.978803+0000","last_undegraded":"2024-04-14T08:35:34.978803+0000","last_fullsized":"2024-04-14T08:35:34.978803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:13.886435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.982701+0000","last_change":"2024-04-14T08:35:33.973956+0000","last_active":"2024-04-14T08:35:34.982701+0000","last_peered":"2024-04-14T08:35:34.982701+0000","last_clean":"2024-04-14T08:35:34.982701+0000","last_became_active":"2024-04-14T08:35:33.973591+0000","last_became_peered":"2024-04-14T08:35:33.973591+0000","last_unstale":"2024-04-14T08:35:34.982701+0000","last_undegraded":"2024-04-14T08:35:34.982701+0000","last_fullsized":"2024-04-14T08:35:34.982701+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:05.086489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984186+0000","last_change":"2024-04-14T08:35:34.971220+0000","last_active":"2024-04-14T08:35:34.984186+0000","last_peered":"2024-04-14T08:35:34.984186+0000","last_clean":"2024-04-14T08:35:34.984186+0000","last_became_active":"2024-04-14T08:35:34.970952+0000","last_became_peered":"2024-04-14T08:35:34.970952+0000","last_unstale":"2024-04-14T08:35:34.984186+0000","last_undegraded":"2024-04-14T08:35:34.984186+0000","last_fullsized":"2024-04-14T08:35:34.984186+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:11:56.878423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.978535+0000","last_change":"2024-04-14T08:35:33.968291+0000","last_active":"2024-04-14T08:35:34.978535+0000","last_peered":"2024-04-14T08:35:34.978535+0000","last_clean":"2024-04-14T08:35:34.978535+0000","last_became_active":"2024-04-14T08:35:33.968006+0000","last_became_peered":"2024-04-14T08:35:33.968006+0000","last_unstale":"2024-04-14T08:35:34.978535+0000","last_undegraded":"2024-04-14T08:35:34.978535+0000","last_fullsized":"2024-04-14T08:35:34.978535+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:04:24.732447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987055+0000","last_change":"2024-04-14T08:35:34.982650+0000","last_active":"2024-04-14T08:35:34.987055+0000","last_peered":"2024-04-14T08:35:34.987055+0000","last_clean":"2024-04-14T08:35:34.987055+0000","last_became_active":"2024-04-14T08:35:34.982341+0000","last_became_peered":"2024-04-14T08:35:34.982341+0000","last_unstale":"2024-04-14T08:35:34.987055+0000","last_undegraded":"2024-04-14T08:35:34.987055+0000","last_fullsized":"2024-04-14T08:35:34.987055+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:54.240477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.957620+0000","last_change":"2024-04-14T08:35:33.970786+0000","last_active":"2024-04-14T08:35:34.957620+0000","last_peered":"2024-04-14T08:35:34.957620+0000","last_clean":"2024-04-14T08:35:34.957620+0000","last_became_active":"2024-04-14T08:35:33.970517+0000","last_became_peered":"2024-04-14T08:35:33.970517+0000","last_unstale":"2024-04-14T08:35:34.957620+0000","last_undegraded":"2024-04-14T08:35:34.957620+0000","last_fullsized":"2024-04-14T08:35:34.957620+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:30:33.901631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.973260+0000","last_change":"2024-04-14T08:35:34.970678+0000","last_active":"2024-04-14T08:35:34.973260+0000","last_peered":"2024-04-14T08:35:34.973260+0000","last_clean":"2024-04-14T08:35:34.973260+0000","last_became_active":"2024-04-14T08:35:34.970392+0000","last_became_peered":"2024-04-14T08:35:34.970392+0000","last_unstale":"2024-04-14T08:35:34.973260+0000","last_undegraded":"2024-04-14T08:35:34.973260+0000","last_fullsized":"2024-04-14T08:35:34.973260+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:37:45.979299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.965867+0000","last_change":"2024-04-14T08:35:33.971081+0000","last_active":"2024-04-14T08:35:34.965867+0000","last_peered":"2024-04-14T08:35:34.965867+0000","last_clean":"2024-04-14T08:35:34.965867+0000","last_became_active":"2024-04-14T08:35:33.970230+0000","last_became_peered":"2024-04-14T08:35:33.970230+0000","last_unstale":"2024-04-14T08:35:34.965867+0000","last_undegraded":"2024-04-14T08:35:34.965867+0000","last_fullsized":"2024-04-14T08:35:34.965867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:07:21.904319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984376+0000","last_change":"2024-04-14T08:35:34.977569+0000","last_active":"2024-04-14T08:35:34.984376+0000","last_peered":"2024-04-14T08:35:34.984376+0000","last_clean":"2024-04-14T08:35:34.984376+0000","last_became_active":"2024-04-14T08:35:34.977059+0000","last_became_peered":"2024-04-14T08:35:34.977059+0000","last_unstale":"2024-04-14T08:35:34.984376+0000","last_undegraded":"2024-04-14T08:35:34.984376+0000","last_fullsized":"2024-04-14T08:35:34.984376+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:24:44.423464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.976094+0000","last_change":"2024-04-14T08:35:33.973678+0000","last_active":"2024-04-14T08:35:35.976094+0000","last_peered":"2024-04-14T08:35:35.976094+0000","last_clean":"2024-04-14T08:35:35.976094+0000","last_became_active":"2024-04-14T08:35:33.973406+0000","last_became_peered":"2024-04-14T08:35:33.973406+0000","last_unstale":"2024-04-14T08:35:35.976094+0000","last_undegraded":"2024-04-14T08:35:35.976094+0000","last_fullsized":"2024-04-14T08:35:35.976094+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:50:24.884541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.982384+0000","last_change":"2024-04-14T08:35:34.977003+0000","last_active":"2024-04-14T08:35:34.982384+0000","last_peered":"2024-04-14T08:35:34.982384+0000","last_clean":"2024-04-14T08:35:34.982384+0000","last_became_active":"2024-04-14T08:35:34.976733+0000","last_became_peered":"2024-04-14T08:35:34.976733+0000","last_unstale":"2024-04-14T08:35:34.982384+0000","last_undegraded":"2024-04-14T08:35:34.982384+0000","last_fullsized":"2024-04-14T08:35:34.982384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:07:49.315642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.962234+0000","last_change":"2024-04-14T08:35:33.973363+0000","last_active":"2024-04-14T08:35:34.962234+0000","last_peered":"2024-04-14T08:35:34.962234+0000","last_clean":"2024-04-14T08:35:34.962234+0000","last_became_active":"2024-04-14T08:35:33.973080+0000","last_became_peered":"2024-04-14T08:35:33.973080+0000","last_unstale":"2024-04-14T08:35:34.962234+0000","last_undegraded":"2024-04-14T08:35:34.962234+0000","last_fullsized":"2024-04-14T08:35:34.962234+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:07:21.904319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987092+0000","last_change":"2024-04-14T08:35:34.982715+0000","last_active":"2024-04-14T08:35:34.987092+0000","last_peered":"2024-04-14T08:35:34.987092+0000","last_clean":"2024-04-14T08:35:34.987092+0000","last_became_active":"2024-04-14T08:35:34.982431+0000","last_became_peered":"2024-04-14T08:35:34.982431+0000","last_unstale":"2024-04-14T08:35:34.987092+0000","last_undegraded":"2024-04-14T08:35:34.987092+0000","last_fullsized":"2024-04-14T08:35:34.987092+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:58:05.015142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.976021+0000","last_change":"2024-04-14T08:35:33.967119+0000","last_active":"2024-04-14T08:35:35.976021+0000","last_peered":"2024-04-14T08:35:35.976021+0000","last_clean":"2024-04-14T08:35:35.976021+0000","last_became_active":"2024-04-14T08:35:33.966808+0000","last_became_peered":"2024-04-14T08:35:33.966808+0000","last_unstale":"2024-04-14T08:35:35.976021+0000","last_undegraded":"2024-04-14T08:35:35.976021+0000","last_fullsized":"2024-04-14T08:35:35.976021+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:39:59.215310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.977714+0000","last_change":"2024-04-14T08:35:34.969279+0000","last_active":"2024-04-14T08:35:34.977714+0000","last_peered":"2024-04-14T08:35:34.977714+0000","last_clean":"2024-04-14T08:35:34.977714+0000","last_became_active":"2024-04-14T08:35:34.968998+0000","last_became_peered":"2024-04-14T08:35:34.968998+0000","last_unstale":"2024-04-14T08:35:34.977714+0000","last_undegraded":"2024-04-14T08:35:34.977714+0000","last_fullsized":"2024-04-14T08:35:34.977714+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:04:40.260061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'6","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.992231+0000","last_change":"2024-04-14T08:35:33.981182+0000","last_active":"2024-04-14T08:35:34.992231+0000","last_peered":"2024-04-14T08:35:34.992231+0000","last_clean":"2024-04-14T08:35:34.992231+0000","last_became_active":"2024-04-14T08:35:33.980946+0000","last_became_peered":"2024-04-14T08:35:33.980946+0000","last_unstale":"2024-04-14T08:35:34.992231+0000","last_undegraded":"2024-04-14T08:35:34.992231+0000","last_fullsized":"2024-04-14T08:35:34.992231+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:14:24.176314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.977545+0000","last_change":"2024-04-14T08:35:34.966216+0000","last_active":"2024-04-14T08:35:34.977545+0000","last_peered":"2024-04-14T08:35:34.977545+0000","last_clean":"2024-04-14T08:35:34.977545+0000","last_became_active":"2024-04-14T08:35:34.965935+0000","last_became_peered":"2024-04-14T08:35:34.965935+0000","last_unstale":"2024-04-14T08:35:34.977545+0000","last_undegraded":"2024-04-14T08:35:34.977545+0000","last_fullsized":"2024-04-14T08:35:34.977545+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:15:16.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975972+0000","last_change":"2024-04-14T08:35:33.970680+0000","last_active":"2024-04-14T08:35:35.975972+0000","last_peered":"2024-04-14T08:35:35.975972+0000","last_clean":"2024-04-14T08:35:35.975972+0000","last_became_active":"2024-04-14T08:35:33.970400+0000","last_became_peered":"2024-04-14T08:35:33.970400+0000","last_unstale":"2024-04-14T08:35:35.975972+0000","last_undegraded":"2024-04-14T08:35:35.975972+0000","last_fullsized":"2024-04-14T08:35:35.975972+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:34:22.432838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985677+0000","last_change":"2024-04-14T08:35:34.980477+0000","last_active":"2024-04-14T08:35:34.985677+0000","last_peered":"2024-04-14T08:35:34.985677+0000","last_clean":"2024-04-14T08:35:34.985677+0000","last_became_active":"2024-04-14T08:35:34.980166+0000","last_became_peered":"2024-04-14T08:35:34.980166+0000","last_unstale":"2024-04-14T08:35:34.985677+0000","last_undegraded":"2024-04-14T08:35:34.985677+0000","last_fullsized":"2024-04-14T08:35:34.985677+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:40:00.219050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.966699+0000","last_change":"2024-04-14T08:35:33.973263+0000","last_active":"2024-04-14T08:35:34.966699+0000","last_peered":"2024-04-14T08:35:34.966699+0000","last_clean":"2024-04-14T08:35:34.966699+0000","last_became_active":"2024-04-14T08:35:33.972938+0000","last_became_peered":"2024-04-14T08:35:33.972938+0000","last_unstale":"2024-04-14T08:35:34.966699+0000","last_undegraded":"2024-04-14T08:35:34.966699+0000","last_fullsized":"2024-04-14T08:35:34.966699+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:12:13.917355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.981756+0000","last_change":"2024-04-14T08:35:34.975125+0000","last_active":"2024-04-14T08:35:34.981756+0000","last_peered":"2024-04-14T08:35:34.981756+0000","last_clean":"2024-04-14T08:35:34.981756+0000","last_became_active":"2024-04-14T08:35:34.974828+0000","last_became_peered":"2024-04-14T08:35:34.974828+0000","last_unstale":"2024-04-14T08:35:34.981756+0000","last_undegraded":"2024-04-14T08:35:34.981756+0000","last_fullsized":"2024-04-14T08:35:34.981756+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:04:40.260061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975296+0000","last_change":"2024-04-14T08:35:33.967260+0000","last_active":"2024-04-14T08:35:35.975296+0000","last_peered":"2024-04-14T08:35:35.975296+0000","last_clean":"2024-04-14T08:35:35.975296+0000","last_became_active":"2024-04-14T08:35:33.966990+0000","last_became_peered":"2024-04-14T08:35:33.966990+0000","last_unstale":"2024-04-14T08:35:35.975296+0000","last_undegraded":"2024-04-14T08:35:35.975296+0000","last_fullsized":"2024-04-14T08:35:35.975296+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:15:58.978558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984091+0000","last_change":"2024-04-14T08:35:34.971125+0000","last_active":"2024-04-14T08:35:34.984091+0000","last_peered":"2024-04-14T08:35:34.984091+0000","last_clean":"2024-04-14T08:35:34.984091+0000","last_became_active":"2024-04-14T08:35:34.970856+0000","last_became_peered":"2024-04-14T08:35:34.970856+0000","last_unstale":"2024-04-14T08:35:34.984091+0000","last_undegraded":"2024-04-14T08:35:34.984091+0000","last_fullsized":"2024-04-14T08:35:34.984091+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:03.166145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"60'8","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.980889+0000","last_change":"2024-04-14T08:35:33.967647+0000","last_active":"2024-04-14T08:35:34.980889+0000","last_peered":"2024-04-14T08:35:34.980889+0000","last_clean":"2024-04-14T08:35:34.980889+0000","last_became_active":"2024-04-14T08:35:33.966518+0000","last_became_peered":"2024-04-14T08:35:33.966518+0000","last_unstale":"2024-04-14T08:35:34.980889+0000","last_undegraded":"2024-04-14T08:35:34.980889+0000","last_fullsized":"2024-04-14T08:35:34.980889+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:27:10.703355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975282+0000","last_change":"2024-04-14T08:35:34.973150+0000","last_active":"2024-04-14T08:35:35.975282+0000","last_peered":"2024-04-14T08:35:35.975282+0000","last_clean":"2024-04-14T08:35:35.975282+0000","last_became_active":"2024-04-14T08:35:34.972894+0000","last_became_peered":"2024-04-14T08:35:34.972894+0000","last_unstale":"2024-04-14T08:35:35.975282+0000","last_undegraded":"2024-04-14T08:35:35.975282+0000","last_fullsized":"2024-04-14T08:35:35.975282+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:15:14.910911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985808+0000","last_change":"2024-04-14T08:35:33.974085+0000","last_active":"2024-04-14T08:35:34.985808+0000","last_peered":"2024-04-14T08:35:34.985808+0000","last_clean":"2024-04-14T08:35:34.985808+0000","last_became_active":"2024-04-14T08:35:33.973843+0000","last_became_peered":"2024-04-14T08:35:33.973843+0000","last_unstale":"2024-04-14T08:35:34.985808+0000","last_undegraded":"2024-04-14T08:35:34.985808+0000","last_fullsized":"2024-04-14T08:35:34.985808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:07.764538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985105+0000","last_change":"2024-04-14T08:35:34.980917+0000","last_active":"2024-04-14T08:35:34.985105+0000","last_peered":"2024-04-14T08:35:34.985105+0000","last_clean":"2024-04-14T08:35:34.985105+0000","last_became_active":"2024-04-14T08:35:34.980648+0000","last_became_peered":"2024-04-14T08:35:34.980648+0000","last_unstale":"2024-04-14T08:35:34.985105+0000","last_undegraded":"2024-04-14T08:35:34.985105+0000","last_fullsized":"2024-04-14T08:35:34.985105+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:14:25.180055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975476+0000","last_change":"2024-04-14T08:35:33.965648+0000","last_active":"2024-04-14T08:35:35.975476+0000","last_peered":"2024-04-14T08:35:35.975476+0000","last_clean":"2024-04-14T08:35:35.975476+0000","last_became_active":"2024-04-14T08:35:33.965358+0000","last_became_peered":"2024-04-14T08:35:33.965358+0000","last_unstale":"2024-04-14T08:35:35.975476+0000","last_undegraded":"2024-04-14T08:35:35.975476+0000","last_fullsized":"2024-04-14T08:35:35.975476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:43:36.830667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984966+0000","last_change":"2024-04-14T08:35:34.979662+0000","last_active":"2024-04-14T08:35:34.984966+0000","last_peered":"2024-04-14T08:35:34.984966+0000","last_clean":"2024-04-14T08:35:34.984966+0000","last_became_active":"2024-04-14T08:35:34.979189+0000","last_became_peered":"2024-04-14T08:35:34.979189+0000","last_unstale":"2024-04-14T08:35:34.984966+0000","last_undegraded":"2024-04-14T08:35:34.984966+0000","last_fullsized":"2024-04-14T08:35:34.984966+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:03.166145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"60'1","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.976109+0000","last_change":"2024-04-14T08:35:33.966547+0000","last_active":"2024-04-14T08:35:35.976109+0000","last_peered":"2024-04-14T08:35:35.976109+0000","last_clean":"2024-04-14T08:35:35.976109+0000","last_became_active":"2024-04-14T08:35:33.966268+0000","last_became_peered":"2024-04-14T08:35:33.966268+0000","last_unstale":"2024-04-14T08:35:35.976109+0000","last_undegraded":"2024-04-14T08:35:35.976109+0000","last_fullsized":"2024-04-14T08:35:35.976109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:45:52.647959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985536+0000","last_change":"2024-04-14T08:35:34.975973+0000","last_active":"2024-04-14T08:35:34.985536+0000","last_peered":"2024-04-14T08:35:34.985536+0000","last_clean":"2024-04-14T08:35:34.985536+0000","last_became_active":"2024-04-14T08:35:34.975627+0000","last_became_peered":"2024-04-14T08:35:34.975627+0000","last_unstale":"2024-04-14T08:35:34.985536+0000","last_undegraded":"2024-04-14T08:35:34.985536+0000","last_fullsized":"2024-04-14T08:35:34.985536+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:32:14.300795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.958060+0000","last_change":"2024-04-14T08:35:33.968718+0000","last_active":"2024-04-14T08:35:34.958060+0000","last_peered":"2024-04-14T08:35:34.958060+0000","last_clean":"2024-04-14T08:35:34.958060+0000","last_became_active":"2024-04-14T08:35:33.968417+0000","last_became_peered":"2024-04-14T08:35:33.968417+0000","last_unstale":"2024-04-14T08:35:34.958060+0000","last_undegraded":"2024-04-14T08:35:34.958060+0000","last_fullsized":"2024-04-14T08:35:34.958060+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:59:40.214958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.979188+0000","last_change":"2024-04-14T08:35:34.967773+0000","last_active":"2024-04-14T08:35:34.979188+0000","last_peered":"2024-04-14T08:35:34.979188+0000","last_clean":"2024-04-14T08:35:34.979188+0000","last_became_active":"2024-04-14T08:35:34.966365+0000","last_became_peered":"2024-04-14T08:35:34.966365+0000","last_unstale":"2024-04-14T08:35:34.979188+0000","last_undegraded":"2024-04-14T08:35:34.979188+0000","last_fullsized":"2024-04-14T08:35:34.979188+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:37:45.979299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985603+0000","last_change":"2024-04-14T08:35:33.970189+0000","last_active":"2024-04-14T08:35:34.985603+0000","last_peered":"2024-04-14T08:35:34.985603+0000","last_clean":"2024-04-14T08:35:34.985603+0000","last_became_active":"2024-04-14T08:35:33.969936+0000","last_became_peered":"2024-04-14T08:35:33.969936+0000","last_unstale":"2024-04-14T08:35:34.985603+0000","last_undegraded":"2024-04-14T08:35:34.985603+0000","last_fullsized":"2024-04-14T08:35:34.985603+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:17:21.187424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983660+0000","last_change":"2024-04-14T08:35:34.981851+0000","last_active":"2024-04-14T08:35:34.983660+0000","last_peered":"2024-04-14T08:35:34.983660+0000","last_clean":"2024-04-14T08:35:34.983660+0000","last_became_active":"2024-04-14T08:35:34.981299+0000","last_became_peered":"2024-04-14T08:35:34.981299+0000","last_unstale":"2024-04-14T08:35:34.983660+0000","last_undegraded":"2024-04-14T08:35:34.983660+0000","last_fullsized":"2024-04-14T08:35:34.983660+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:45:53.651699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.966630+0000","last_change":"2024-04-14T08:35:33.973659+0000","last_active":"2024-04-14T08:35:34.966630+0000","last_peered":"2024-04-14T08:35:34.966630+0000","last_clean":"2024-04-14T08:35:34.966630+0000","last_became_active":"2024-04-14T08:35:33.972363+0000","last_became_peered":"2024-04-14T08:35:33.972363+0000","last_unstale":"2024-04-14T08:35:34.966630+0000","last_undegraded":"2024-04-14T08:35:34.966630+0000","last_fullsized":"2024-04-14T08:35:34.966630+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:58:08.947797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985073+0000","last_change":"2024-04-14T08:35:34.980777+0000","last_active":"2024-04-14T08:35:34.985073+0000","last_peered":"2024-04-14T08:35:34.985073+0000","last_clean":"2024-04-14T08:35:34.985073+0000","last_became_active":"2024-04-14T08:35:34.980501+0000","last_became_peered":"2024-04-14T08:35:34.980501+0000","last_unstale":"2024-04-14T08:35:34.985073+0000","last_undegraded":"2024-04-14T08:35:34.985073+0000","last_fullsized":"2024-04-14T08:35:34.985073+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:33:26.666479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.961098+0000","last_change":"2024-04-14T08:35:33.967333+0000","last_active":"2024-04-14T08:35:34.961098+0000","last_peered":"2024-04-14T08:35:34.961098+0000","last_clean":"2024-04-14T08:35:34.961098+0000","last_became_active":"2024-04-14T08:35:33.967077+0000","last_became_peered":"2024-04-14T08:35:33.967077+0000","last_unstale":"2024-04-14T08:35:34.961098+0000","last_undegraded":"2024-04-14T08:35:34.961098+0000","last_fullsized":"2024-04-14T08:35:34.961098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:04:24.732447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983568+0000","last_change":"2024-04-14T08:35:34.981326+0000","last_active":"2024-04-14T08:35:34.983568+0000","last_peered":"2024-04-14T08:35:34.983568+0000","last_clean":"2024-04-14T08:35:34.983568+0000","last_became_active":"2024-04-14T08:35:34.980780+0000","last_became_peered":"2024-04-14T08:35:34.980780+0000","last_unstale":"2024-04-14T08:35:34.983568+0000","last_undegraded":"2024-04-14T08:35:34.983568+0000","last_fullsized":"2024-04-14T08:35:34.983568+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:21:27.827950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.985425+0000","last_change":"2024-04-14T08:35:33.975602+0000","last_active":"2024-04-14T08:35:34.985425+0000","last_peered":"2024-04-14T08:35:34.985425+0000","last_clean":"2024-04-14T08:35:34.985425+0000","last_became_active":"2024-04-14T08:35:33.975316+0000","last_became_peered":"2024-04-14T08:35:33.975316+0000","last_unstale":"2024-04-14T08:35:34.985425+0000","last_undegraded":"2024-04-14T08:35:34.985425+0000","last_fullsized":"2024-04-14T08:35:34.985425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:51:21.880044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.978919+0000","last_change":"2024-04-14T08:35:34.974091+0000","last_active":"2024-04-14T08:35:34.978919+0000","last_peered":"2024-04-14T08:35:34.978919+0000","last_clean":"2024-04-14T08:35:34.978919+0000","last_became_active":"2024-04-14T08:35:34.973788+0000","last_became_peered":"2024-04-14T08:35:34.973788+0000","last_unstale":"2024-04-14T08:35:34.978919+0000","last_undegraded":"2024-04-14T08:35:34.978919+0000","last_fullsized":"2024-04-14T08:35:34.978919+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:45:18.866747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.966964+0000","last_change":"2024-04-14T08:35:33.974207+0000","last_active":"2024-04-14T08:35:34.966964+0000","last_peered":"2024-04-14T08:35:34.966964+0000","last_clean":"2024-04-14T08:35:34.966964+0000","last_became_active":"2024-04-14T08:35:33.973920+0000","last_became_peered":"2024-04-14T08:35:33.973920+0000","last_unstale":"2024-04-14T08:35:34.966964+0000","last_undegraded":"2024-04-14T08:35:34.966964+0000","last_fullsized":"2024-04-14T08:35:34.966964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:39:33.433883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983167+0000","last_change":"2024-04-14T08:35:34.981037+0000","last_active":"2024-04-14T08:35:34.983167+0000","last_peered":"2024-04-14T08:35:34.983167+0000","last_clean":"2024-04-14T08:35:34.983167+0000","last_became_active":"2024-04-14T08:35:34.980776+0000","last_became_peered":"2024-04-14T08:35:34.980776+0000","last_unstale":"2024-04-14T08:35:34.983167+0000","last_undegraded":"2024-04-14T08:35:34.983167+0000","last_fullsized":"2024-04-14T08:35:34.983167+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:15:08.923865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.961521+0000","last_change":"2024-04-14T08:35:33.973611+0000","last_active":"2024-04-14T08:35:34.961521+0000","last_peered":"2024-04-14T08:35:34.961521+0000","last_clean":"2024-04-14T08:35:34.961521+0000","last_became_active":"2024-04-14T08:35:33.972704+0000","last_became_peered":"2024-04-14T08:35:33.972704+0000","last_unstale":"2024-04-14T08:35:34.961521+0000","last_undegraded":"2024-04-14T08:35:34.961521+0000","last_fullsized":"2024-04-14T08:35:34.961521+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:05.086489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.981667+0000","last_change":"2024-04-14T08:35:34.973414+0000","last_active":"2024-04-14T08:35:34.981667+0000","last_peered":"2024-04-14T08:35:34.981667+0000","last_clean":"2024-04-14T08:35:34.981667+0000","last_became_active":"2024-04-14T08:35:34.973124+0000","last_became_peered":"2024-04-14T08:35:34.973124+0000","last_unstale":"2024-04-14T08:35:34.981667+0000","last_undegraded":"2024-04-14T08:35:34.981667+0000","last_fullsized":"2024-04-14T08:35:34.981667+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:54:56.621692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.964628+0000","last_change":"2024-04-14T08:35:33.976356+0000","last_active":"2024-04-14T08:35:34.964628+0000","last_peered":"2024-04-14T08:35:34.964628+0000","last_clean":"2024-04-14T08:35:34.964628+0000","last_became_active":"2024-04-14T08:35:33.976063+0000","last_became_peered":"2024-04-14T08:35:33.976063+0000","last_unstale":"2024-04-14T08:35:34.964628+0000","last_undegraded":"2024-04-14T08:35:34.964628+0000","last_fullsized":"2024-04-14T08:35:34.964628+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:53.236736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987321+0000","last_change":"2024-04-14T08:35:34.985322+0000","last_active":"2024-04-14T08:35:34.987321+0000","last_peered":"2024-04-14T08:35:34.987321+0000","last_clean":"2024-04-14T08:35:34.987321+0000","last_became_active":"2024-04-14T08:35:34.985066+0000","last_became_peered":"2024-04-14T08:35:34.985066+0000","last_unstale":"2024-04-14T08:35:34.987321+0000","last_undegraded":"2024-04-14T08:35:34.987321+0000","last_fullsized":"2024-04-14T08:35:34.987321+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:03.166145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.976069+0000","last_change":"2024-04-14T08:35:33.970482+0000","last_active":"2024-04-14T08:35:35.976069+0000","last_peered":"2024-04-14T08:35:35.976069+0000","last_clean":"2024-04-14T08:35:35.976069+0000","last_became_active":"2024-04-14T08:35:33.970197+0000","last_became_peered":"2024-04-14T08:35:33.970197+0000","last_unstale":"2024-04-14T08:35:35.976069+0000","last_undegraded":"2024-04-14T08:35:35.976069+0000","last_fullsized":"2024-04-14T08:35:35.976069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:15:07.920124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.971736+0000","last_change":"2024-04-14T08:35:34.966023+0000","last_active":"2024-04-14T08:35:34.971736+0000","last_peered":"2024-04-14T08:35:34.971736+0000","last_clean":"2024-04-14T08:35:34.971736+0000","last_became_active":"2024-04-14T08:35:34.965650+0000","last_became_peered":"2024-04-14T08:35:34.965650+0000","last_unstale":"2024-04-14T08:35:34.971736+0000","last_undegraded":"2024-04-14T08:35:34.971736+0000","last_fullsized":"2024-04-14T08:35:34.971736+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:54:56.621692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.958245+0000","last_change":"2024-04-14T08:35:33.972393+0000","last_active":"2024-04-14T08:35:34.958245+0000","last_peered":"2024-04-14T08:35:34.958245+0000","last_clean":"2024-04-14T08:35:34.958245+0000","last_became_active":"2024-04-14T08:35:33.972129+0000","last_became_peered":"2024-04-14T08:35:33.972129+0000","last_unstale":"2024-04-14T08:35:34.958245+0000","last_undegraded":"2024-04-14T08:35:34.958245+0000","last_fullsized":"2024-04-14T08:35:34.958245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:15:13.907170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.984159+0000","last_change":"2024-04-14T08:35:34.972009+0000","last_active":"2024-04-14T08:35:34.984159+0000","last_peered":"2024-04-14T08:35:34.984159+0000","last_clean":"2024-04-14T08:35:34.984159+0000","last_became_active":"2024-04-14T08:35:34.971468+0000","last_became_peered":"2024-04-14T08:35:34.971468+0000","last_unstale":"2024-04-14T08:35:34.984159+0000","last_undegraded":"2024-04-14T08:35:34.984159+0000","last_fullsized":"2024-04-14T08:35:34.984159+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:19:15.795385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.957802+0000","last_change":"2024-04-14T08:35:33.970907+0000","last_active":"2024-04-14T08:35:34.957802+0000","last_peered":"2024-04-14T08:35:34.957802+0000","last_clean":"2024-04-14T08:35:34.957802+0000","last_became_active":"2024-04-14T08:35:33.969919+0000","last_became_peered":"2024-04-14T08:35:33.969919+0000","last_unstale":"2024-04-14T08:35:34.957802+0000","last_undegraded":"2024-04-14T08:35:34.957802+0000","last_fullsized":"2024-04-14T08:35:34.957802+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:17:21.187424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.983028+0000","last_change":"2024-04-14T08:35:34.980934+0000","last_active":"2024-04-14T08:35:34.983028+0000","last_peered":"2024-04-14T08:35:34.983028+0000","last_clean":"2024-04-14T08:35:34.983028+0000","last_became_active":"2024-04-14T08:35:34.980683+0000","last_became_peered":"2024-04-14T08:35:34.980683+0000","last_unstale":"2024-04-14T08:35:34.983028+0000","last_undegraded":"2024-04-14T08:35:34.983028+0000","last_fullsized":"2024-04-14T08:35:34.983028+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:41:36.584680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.961159+0000","last_change":"2024-04-14T08:35:33.978148+0000","last_active":"2024-04-14T08:35:34.961159+0000","last_peered":"2024-04-14T08:35:34.961159+0000","last_clean":"2024-04-14T08:35:34.961159+0000","last_became_active":"2024-04-14T08:35:33.977887+0000","last_became_peered":"2024-04-14T08:35:33.977887+0000","last_unstale":"2024-04-14T08:35:34.961159+0000","last_undegraded":"2024-04-14T08:35:34.961159+0000","last_fullsized":"2024-04-14T08:35:34.961159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:33:25.662739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.987478+0000","last_change":"2024-04-14T08:35:34.986565+0000","last_active":"2024-04-14T08:35:34.987478+0000","last_peered":"2024-04-14T08:35:34.987478+0000","last_clean":"2024-04-14T08:35:34.987478+0000","last_became_active":"2024-04-14T08:35:34.986150+0000","last_became_peered":"2024-04-14T08:35:34.986150+0000","last_unstale":"2024-04-14T08:35:34.987478+0000","last_undegraded":"2024-04-14T08:35:34.987478+0000","last_fullsized":"2024-04-14T08:35:34.987478+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:33.946733+0000","last_clean_scrub_stamp":"2024-04-14T08:35:33.946733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:19:15.795385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-14T08:35:34.957720+0000","last_change":"2024-04-14T08:35:33.973263+0000","last_active":"2024-04-14T08:35:34.957720+0000","last_peered":"2024-04-14T08:35:34.957720+0000","last_clean":"2024-04-14T08:35:34.957720+0000","last_became_active":"2024-04-14T08:35:33.972625+0000","last_became_peered":"2024-04-14T08:35:33.972625+0000","last_unstale":"2024-04-14T08:35:34.957720+0000","last_undegraded":"2024-04-14T08:35:34.957720+0000","last_fullsized":"2024-04-14T08:35:34.957720+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:35:32.942993+0000","last_clean_scrub_stamp":"2024-04-14T08:35:32.942993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:37:38.252823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":251,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-14T08:35:35.975109+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:35:35.975109+0000","last_peered":"2024-04-14T08:35:35.975109+0000","last_clean":"2024-04-14T08:35:35.975109+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:35:35.975109+0000","last_undegraded":"2024-04-14T08:35:35.975109+0000","last_fullsized":"2024-04-14T08:35:35.975109+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28332,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343508,"statfs":{"total":96636764160,"available":96607752192,"internally_reserved":0,"allocated":1335296,"data_stored":914347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":316036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":316036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":737280,"data_stored":324815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":316036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":316036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":316036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28300,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343540,"statfs":{"total":96636764160,"available":96607784960,"internally_reserved":0,"allocated":1306624,"data_stored":906404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:35:38.287 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-14T08:35:38.287 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-14T08:35:38.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.431+0000 7f5fb3ffe640 1 Processor -- start 2024-04-14T08:35:38.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.432+0000 7f5fb3ffe640 1 -- start start 2024-04-14T08:35:38.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.432+0000 7f5fb3ffe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5fac15c950 0x7f5fac0cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.432+0000 7f5fb3ffe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fac0b47c0 0x7f5fac165f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.432+0000 7f5fb3ffe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fac0cce80 con 0x7f5fac0cff30 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.432+0000 7f5fb3ffe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f5fac0cd160 con 0x7f5fac15c950 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.432+0000 7f5fb3ffe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac0cd440 con 0x7f5fac0b47c0 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.433+0000 7f5fb1d9a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5fac15c950 0x7f5fac0cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.433+0000 7f5fb1599640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5fac0cff30 0x7f5fac15cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59918/0 (socket says 172.21.15.172:59918) 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.433+0000 7f5fb1599640 1 -- 172.21.15.172:0/335669752 learned_addr learned my addr 172.21.15.172:0/335669752 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.433+0000 7f5fb259b640 1 --2- 172.21.15.172:0/335669752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fac0b47c0 0x7f5fac165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.433+0000 7f5fb1d9a640 1 -- 172.21.15.172:0/335669752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fac0b47c0 msgr2=0x7f5fac165f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.433+0000 7f5fb1d9a640 1 --2- 172.21.15.172:0/335669752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fac0b47c0 0x7f5fac165f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.433+0000 7f5fb1d9a640 1 -- 172.21.15.172:0/335669752 >> v1:172.21.15.172:6789/0 conn(0x7f5fac0cff30 legacy=0x7f5fac15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.433+0000 7f5fb1d9a640 1 -- 172.21.15.172:0/335669752 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac0cd720 con 0x7f5fac15c950 2024-04-14T08:35:38.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.433+0000 7f5fb0d98640 1 -- 172.21.15.172:0/335669752 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2771628918 0 0) 0x7f5fac0cce80 con 0x7f5fac0cff30 2024-04-14T08:35:38.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.434+0000 7f5fb1d9a640 1 --2- 172.21.15.172:0/335669752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fac15c950 0x7f5fac0cc8a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0097a0 tx=0x7f5f9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da693ff07e27fd8b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:38.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.434+0000 7f5fb0d98640 1 -- 172.21.15.172:0/335669752 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c035020 con 0x7f5fac15c950 2024-04-14T08:35:38.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.434+0000 7f5fb0d98640 1 -- 172.21.15.172:0/335669752 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f9c031830 con 0x7f5fac15c950 2024-04-14T08:35:38.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.434+0000 7f5fb0d98640 1 -- 172.21.15.172:0/335669752 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c031c00 con 0x7f5fac15c950 2024-04-14T08:35:38.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.434+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/335669752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fac15c950 msgr2=0x7f5fac0cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.434+0000 7f5fb3ffe640 1 --2- 172.21.15.172:0/335669752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fac15c950 0x7f5fac0cc8a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0097a0 tx=0x7f5f9c02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.434+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/335669752 shutdown_connections 2024-04-14T08:35:38.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.434+0000 7f5fb3ffe640 1 --2- 172.21.15.172:0/335669752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fac0b47c0 0x7f5fac165f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.434+0000 7f5fb3ffe640 1 --2- 172.21.15.172:0/335669752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fac15c950 0x7f5fac0cc8a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.434+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/335669752 >> 172.21.15.172:0/335669752 conn(0x7f5fac0d0780 msgr2=0x7f5fac0d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:38.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.434+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/335669752 shutdown_connections 2024-04-14T08:35:38.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.434+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/335669752 wait complete. 2024-04-14T08:35:38.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb3ffe640 1 Processor -- start 2024-04-14T08:35:38.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb3ffe640 1 -- start start 2024-04-14T08:35:38.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb3ffe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac0b47c0 0x7f5fac1f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb3ffe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fac0cff30 0x7f5fac1ef140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb3ffe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fac15c950 0x7f5fac143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb3ffe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac0d0560 con 0x7f5fac15c950 2024-04-14T08:35:38.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb3ffe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac0d03e0 con 0x7f5fac0b47c0 2024-04-14T08:35:38.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb3ffe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac15d440 con 0x7f5fac0cff30 2024-04-14T08:35:38.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb259b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fac15c950 0x7f5fac143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb259b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fac15c950 0x7f5fac143ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42776/0 (socket says 172.21.15.172:42776) 2024-04-14T08:35:38.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb259b640 1 -- 172.21.15.172:0/3339409398 learned_addr learned my addr 172.21.15.172:0/3339409398 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:38.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb1599640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fac0cff30 0x7f5fac1ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.436+0000 7f5fb1d9a640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac0b47c0 0x7f5fac1f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.437+0000 7f5fb259b640 1 -- 172.21.15.172:0/3339409398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fac0cff30 msgr2=0x7f5fac1ef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.437+0000 7f5fb259b640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fac0cff30 0x7f5fac1ef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.437+0000 7f5fb259b640 1 -- 172.21.15.172:0/3339409398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac0b47c0 msgr2=0x7f5fac1f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.437+0000 7f5fb259b640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac0b47c0 0x7f5fac1f26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.437+0000 7f5fb259b640 1 -- 172.21.15.172:0/3339409398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac0a9800 con 0x7f5fac15c950 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.437+0000 7f5fb259b640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fac15c950 0x7f5fac143ed0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400efc0 tx=0x7f5fa400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.437+0000 7f5fa27fc640 1 -- 172.21.15.172:0/3339409398 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4018020 con 0x7f5fac15c950 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.437+0000 7f5fa27fc640 1 -- 172.21.15.172:0/3339409398 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa4014480 con 0x7f5fac15c950 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.437+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/3339409398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac1f5820 con 0x7f5fac15c950 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.437+0000 7f5fa27fc640 1 -- 172.21.15.172:0/3339409398 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4014870 con 0x7f5fac15c950 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.437+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/3339409398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac1f5de0 con 0x7f5fac15c950 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.438+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/3339409398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac068590 con 0x7f5fac15c950 2024-04-14T08:35:38.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.438+0000 7f5fa27fc640 1 -- 172.21.15.172:0/3339409398 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fa4005d10 con 0x7f5fac15c950 2024-04-14T08:35:38.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.439+0000 7f5fa27fc640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5f9006c8f0 0x7f5f9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.439+0000 7f5fb1d9a640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5f9006c8f0 0x7f5f9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.439+0000 7f5fa27fc640 1 -- 172.21.15.172:0/3339409398 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f5fa408c690 con 0x7f5fac15c950 2024-04-14T08:35:38.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.439+0000 7f5fb1d9a640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5f9006c8f0 0x7f5f9006edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c036000 tx=0x7f5f9c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:38.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.443+0000 7f5fa27fc640 1 -- 172.21.15.172:0/3339409398 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5fa4022070 con 0x7f5fac15c950 2024-04-14T08:35:38.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.687+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/3339409398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5fac15ce60 con 0x7f5fac15c950 2024-04-14T08:35:38.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.689+0000 7f5fa27fc640 1 -- 172.21.15.172:0/3339409398 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f5fac0a9800 con 0x7f5fac15c950 2024-04-14T08:35:38.689 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2024-04-14T08:35:38.689 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA== 2024-04-14T08:35:38.690 INFO:teuthology.orchestra.run.smithi172.stdout: caps mds = "allow" 2024-04-14T08:35:38.690 INFO:teuthology.orchestra.run.smithi172.stdout: caps mon = "allow r" 2024-04-14T08:35:38.690 INFO:teuthology.orchestra.run.smithi172.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-14T08:35:38.690 INFO:teuthology.orchestra.run.smithi172.stderr:updated caps for client.0 2024-04-14T08:35:38.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.692+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/3339409398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5f9006c8f0 msgr2=0x7f5f9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.692+0000 7f5fb3ffe640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5f9006c8f0 0x7f5f9006edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c036000 tx=0x7f5f9c03c040 comp rx=0 tx=0).stop 2024-04-14T08:35:38.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.692+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/3339409398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fac15c950 msgr2=0x7f5fac143ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.692+0000 7f5fb3ffe640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fac15c950 0x7f5fac143ed0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400efc0 tx=0x7f5fa400c490 comp rx=0 tx=0).stop 2024-04-14T08:35:38.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.692+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/3339409398 shutdown_connections 2024-04-14T08:35:38.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.692+0000 7f5fb3ffe640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5f9006c8f0 0x7f5f9006edb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.692+0000 7f5fb3ffe640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fac15c950 0x7f5fac143ed0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.692+0000 7f5fb3ffe640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fac0cff30 0x7f5fac1ef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.692+0000 7f5fb3ffe640 1 --2- 172.21.15.172:0/3339409398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac0b47c0 0x7f5fac1f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.692+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/3339409398 >> 172.21.15.172:0/3339409398 conn(0x7f5fac0d0780 msgr2=0x7f5fac156320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:38.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.692+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/3339409398 shutdown_connections 2024-04-14T08:35:38.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.692+0000 7f5fb3ffe640 1 -- 172.21.15.172:0/3339409398 wait complete. 2024-04-14T08:35:38.703 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:35:38.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.804+0000 7f65e75fe640 1 Processor -- start 2024-04-14T08:35:38.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.804+0000 7f65e75fe640 1 -- start start 2024-04-14T08:35:38.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65e75fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f65e006c530 0x7f65e006c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65e75fe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65e00b41d0 con 0x7f65e00729b0 2024-04-14T08:35:38.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65e75fe640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65e00aebe0 con 0x7f65e015e5b0 2024-04-14T08:35:38.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65e75fe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f65e006cee0 con 0x7f65e006c530 2024-04-14T08:35:38.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65e4b99640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f65e00729b0 0x7f65e006dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59924/0 (socket says 172.21.15.172:59924) 2024-04-14T08:35:38.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65e4b99640 1 -- 172.21.15.172:0/1265461059 learned_addr learned my addr 172.21.15.172:0/1265461059 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:38.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65e5b9b640 1 --2- 172.21.15.172:0/1265461059 >> v2:172.21.15.177:3300/0 conn(0x7f65e006c530 0x7f65e006c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65d7fff640 1 -- 172.21.15.172:0/1265461059 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2942287381 0 0) 0x7f65e00b41d0 con 0x7f65e00729b0 2024-04-14T08:35:38.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65d7fff640 1 -- 172.21.15.172:0/1265461059 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65cc003520 con 0x7f65e00729b0 2024-04-14T08:35:38.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65d7fff640 1 -- 172.21.15.172:0/1265461059 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3563652121 0 0) 0x7f65e00aebe0 con 0x7f65e015e5b0 2024-04-14T08:35:38.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65d7fff640 1 -- 172.21.15.172:0/1265461059 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65e00b41d0 con 0x7f65e015e5b0 2024-04-14T08:35:38.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65e5b9b640 1 -- 172.21.15.172:0/1265461059 >> v1:172.21.15.177:6789/0 conn(0x7f65e015e5b0 legacy=0x7f65e00ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65e5b9b640 1 -- 172.21.15.172:0/1265461059 >> v1:172.21.15.172:6789/0 conn(0x7f65e00729b0 legacy=0x7f65e006dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.805+0000 7f65e5b9b640 1 -- 172.21.15.172:0/1265461059 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e00b4470 con 0x7f65e006c530 2024-04-14T08:35:38.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.806+0000 7f65e5b9b640 1 --2- 172.21.15.172:0/1265461059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e006c530 0x7f65e006c9a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009dc0 tx=0x7f65d802bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2feafe2019f889b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:38.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.806+0000 7f65d7fff640 1 -- 172.21.15.172:0/1265461059 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8039070 con 0x7f65e006c530 2024-04-14T08:35:38.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.806+0000 7f65d7fff640 1 -- 172.21.15.172:0/1265461059 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d802ccd0 con 0x7f65e006c530 2024-04-14T08:35:38.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.806+0000 7f65d7fff640 1 -- 172.21.15.172:0/1265461059 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8034660 con 0x7f65e006c530 2024-04-14T08:35:38.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.807+0000 7f65e75fe640 1 -- 172.21.15.172:0/1265461059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e006c530 msgr2=0x7f65e006c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.807+0000 7f65e75fe640 1 --2- 172.21.15.172:0/1265461059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e006c530 0x7f65e006c9a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009dc0 tx=0x7f65d802bf20 comp rx=0 tx=0).stop 2024-04-14T08:35:38.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.807+0000 7f65e75fe640 1 -- 172.21.15.172:0/1265461059 shutdown_connections 2024-04-14T08:35:38.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.807+0000 7f65e75fe640 1 --2- 172.21.15.172:0/1265461059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e006c530 0x7f65e006c9a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.807+0000 7f65e75fe640 1 -- 172.21.15.172:0/1265461059 >> 172.21.15.172:0/1265461059 conn(0x7f65e0071ba0 msgr2=0x7f65e0071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:38.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.807+0000 7f65e75fe640 1 -- 172.21.15.172:0/1265461059 shutdown_connections 2024-04-14T08:35:38.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.807+0000 7f65e75fe640 1 -- 172.21.15.172:0/1265461059 wait complete. 2024-04-14T08:35:38.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.808+0000 7f65e75fe640 1 Processor -- start 2024-04-14T08:35:38.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.808+0000 7f65e75fe640 1 -- start start 2024-04-14T08:35:38.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e75fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e006c530 0x7f65e0142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e75fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e00729b0 0x7f65e01612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e75fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65e015e5b0 0x7f65e00cf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e75fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f65e00af040 con 0x7f65e006c530 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e75fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f65e00aeec0 con 0x7f65e00729b0 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e75fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f65e00b4c70 con 0x7f65e015e5b0 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e539a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e006c530 0x7f65e0142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e539a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e006c530 0x7f65e0142820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42790/0 (socket says 172.21.15.172:42790) 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e539a640 1 -- 172.21.15.172:0/1931241520 learned_addr learned my addr 172.21.15.172:0/1931241520 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e4b99640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e00729b0 0x7f65e01612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e5b9b640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65e015e5b0 0x7f65e00cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e539a640 1 -- 172.21.15.172:0/1931241520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65e015e5b0 msgr2=0x7f65e00cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e539a640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65e015e5b0 0x7f65e00cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e539a640 1 -- 172.21.15.172:0/1931241520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e00729b0 msgr2=0x7f65e01612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e539a640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e00729b0 0x7f65e01612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:38.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e539a640 1 -- 172.21.15.172:0/1931241520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e0142d60 con 0x7f65e006c530 2024-04-14T08:35:38.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.809+0000 7f65e5b9b640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65e015e5b0 0x7f65e00cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:35:38.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.810+0000 7f65e539a640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e006c530 0x7f65e0142820 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00c430 tx=0x7f65dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:38.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.810+0000 7f65d5ffb640 1 -- 172.21.15.172:0/1931241520 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc017070 con 0x7f65e006c530 2024-04-14T08:35:38.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.810+0000 7f65d5ffb640 1 -- 172.21.15.172:0/1931241520 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65dc0052f0 con 0x7f65e006c530 2024-04-14T08:35:38.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.810+0000 7f65d5ffb640 1 -- 172.21.15.172:0/1931241520 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc003830 con 0x7f65e006c530 2024-04-14T08:35:38.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.810+0000 7f65e75fe640 1 -- 172.21.15.172:0/1931241520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e0146000 con 0x7f65e006c530 2024-04-14T08:35:38.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.810+0000 7f65e75fe640 1 -- 172.21.15.172:0/1931241520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e0160210 con 0x7f65e006c530 2024-04-14T08:35:38.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.811+0000 7f65e75fe640 1 -- 172.21.15.172:0/1931241520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e0068520 con 0x7f65e006c530 2024-04-14T08:35:38.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.811+0000 7f65d5ffb640 1 -- 172.21.15.172:0/1931241520 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65dc01a070 con 0x7f65e006c530 2024-04-14T08:35:38.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.812+0000 7f65d5ffb640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f65c406c8f0 0x7f65c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:38.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.812+0000 7f65e4b99640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f65c406c8f0 0x7f65c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:38.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.812+0000 7f65d5ffb640 1 -- 172.21.15.172:0/1931241520 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f65dc08af80 con 0x7f65e006c530 2024-04-14T08:35:38.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.812+0000 7f65e4b99640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f65c406c8f0 0x7f65c406edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f65e00b0d80 tx=0x7f65d0007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:38.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:38.816+0000 7f65d5ffb640 1 -- 172.21.15.172:0/1931241520 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f65e0142d60 con 0x7f65e006c530 2024-04-14T08:35:39.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.027+0000 7f65e75fe640 1 -- 172.21.15.172:0/1931241520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65e00b3f10 con 0x7f65e006c530 2024-04-14T08:35:39.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.028+0000 7f65d5ffb640 1 -- 172.21.15.172:0/1931241520 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5194 (secure 0 0 0) 0x7f65dc058c60 con 0x7f65e006c530 2024-04-14T08:35:39.028 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:39.029 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3621780015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3621780015},{"type":"v1","addr":"172.21.15.172:6836","nonce":3621780015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3894683748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3894683748},{"type":"v1","addr":"172.21.15.172:6837","nonce":3894683748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3625745587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3625745587},{"type":"v1","addr":"172.21.15.177:6834","nonce":3625745587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:35:34.316542+0000","modified":"2024-04-14T08:35:36.314930+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/4176013980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":4176013980},{"type":"v1","addr":"172.21.15.177:6835","nonce":4176013980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-14T08:35:39.029 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 30 2024-04-14T08:35:39.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.030+0000 7f65e75fe640 1 -- 172.21.15.172:0/1931241520 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f65c406c8f0 msgr2=0x7f65c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.030+0000 7f65e75fe640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f65c406c8f0 0x7f65c406edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f65e00b0d80 tx=0x7f65d0007d30 comp rx=0 tx=0).stop 2024-04-14T08:35:39.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.030+0000 7f65e75fe640 1 -- 172.21.15.172:0/1931241520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e006c530 msgr2=0x7f65e0142820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.030+0000 7f65e75fe640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e006c530 0x7f65e0142820 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00c430 tx=0x7f65dc00c8f0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.030+0000 7f65e75fe640 1 -- 172.21.15.172:0/1931241520 shutdown_connections 2024-04-14T08:35:39.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.030+0000 7f65e75fe640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f65c406c8f0 0x7f65c406edb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.030+0000 7f65e75fe640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65e015e5b0 0x7f65e00cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.030+0000 7f65e75fe640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65e00729b0 0x7f65e01612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.030+0000 7f65e75fe640 1 --2- 172.21.15.172:0/1931241520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e006c530 0x7f65e0142820 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.030+0000 7f65e75fe640 1 -- 172.21.15.172:0/1931241520 >> 172.21.15.172:0/1931241520 conn(0x7f65e0071ba0 msgr2=0x7f65e015cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:39.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.030+0000 7f65e75fe640 1 -- 172.21.15.172:0/1931241520 shutdown_connections 2024-04-14T08:35:39.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.030+0000 7f65e75fe640 1 -- 172.21.15.172:0/1931241520 wait complete. 2024-04-14T08:35:39.445 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:35:34.316542+0000', 'modified': '2024-04-14T08:35:36.314930+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6204}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6204': {'gid': 6204, 'name': 'd', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6835/4176013980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6833', 'nonce': 4176013980}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 4176013980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6204, 'qdb_cluster': [6204]} 2024-04-14T08:35:39.445 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:35:39.445 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-14T08:35:39.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.548+0000 7ff236fb4640 1 Processor -- start 2024-04-14T08:35:39.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.548+0000 7ff236fb4640 1 -- start start 2024-04-14T08:35:39.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff236fb4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23015c950 0x7ff2300cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff236fb4640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2300cce80 con 0x7ff2300cff30 2024-04-14T08:35:39.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff236fb4640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2300cd160 con 0x7ff2300b47c0 2024-04-14T08:35:39.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff236fb4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff2300cd440 con 0x7ff23015c950 2024-04-14T08:35:39.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff22ffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff2300cff30 0x7ff23015cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55276/0 (socket says 172.21.15.172:55276) 2024-04-14T08:35:39.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff22ffff640 1 -- 172.21.15.172:0/250444935 learned_addr learned my addr 172.21.15.172:0/250444935 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:39.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff234d50640 1 --2- 172.21.15.172:0/250444935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23015c950 0x7ff2300cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff22f7fe640 1 -- 172.21.15.172:0/250444935 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 346270454 0 0) 0x7ff2300cce80 con 0x7ff2300cff30 2024-04-14T08:35:39.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff22f7fe640 1 -- 172.21.15.172:0/250444935 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff220003520 con 0x7ff2300cff30 2024-04-14T08:35:39.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff22f7fe640 1 -- 172.21.15.172:0/250444935 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1837737178 0 0) 0x7ff2300cd160 con 0x7ff2300b47c0 2024-04-14T08:35:39.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff22f7fe640 1 -- 172.21.15.172:0/250444935 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2300cce80 con 0x7ff2300b47c0 2024-04-14T08:35:39.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff234d50640 1 -- 172.21.15.172:0/250444935 >> v1:172.21.15.177:6789/0 conn(0x7ff2300b47c0 legacy=0x7ff230165f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff234d50640 1 -- 172.21.15.172:0/250444935 >> v1:172.21.15.172:6789/0 conn(0x7ff2300cff30 legacy=0x7ff23015cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.549+0000 7ff234d50640 1 -- 172.21.15.172:0/250444935 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2300cd720 con 0x7ff23015c950 2024-04-14T08:35:39.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.550+0000 7ff234d50640 1 --2- 172.21.15.172:0/250444935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23015c950 0x7ff2300cc8a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff2240097d0 tx=0x7ff22402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bbec55f6425acb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:39.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.550+0000 7ff22f7fe640 1 -- 172.21.15.172:0/250444935 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224035020 con 0x7ff23015c950 2024-04-14T08:35:39.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.550+0000 7ff22f7fe640 1 -- 172.21.15.172:0/250444935 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff224031890 con 0x7ff23015c950 2024-04-14T08:35:39.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.550+0000 7ff22f7fe640 1 -- 172.21.15.172:0/250444935 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224031c60 con 0x7ff23015c950 2024-04-14T08:35:39.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.551+0000 7ff236fb4640 1 -- 172.21.15.172:0/250444935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23015c950 msgr2=0x7ff2300cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.551+0000 7ff236fb4640 1 --2- 172.21.15.172:0/250444935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23015c950 0x7ff2300cc8a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff2240097d0 tx=0x7ff22402b930 comp rx=0 tx=0).stop 2024-04-14T08:35:39.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.551+0000 7ff236fb4640 1 -- 172.21.15.172:0/250444935 shutdown_connections 2024-04-14T08:35:39.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.551+0000 7ff236fb4640 1 --2- 172.21.15.172:0/250444935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff23015c950 0x7ff2300cc8a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.551+0000 7ff236fb4640 1 -- 172.21.15.172:0/250444935 >> 172.21.15.172:0/250444935 conn(0x7ff2300d0780 msgr2=0x7ff2300d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:39.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.551+0000 7ff236fb4640 1 -- 172.21.15.172:0/250444935 shutdown_connections 2024-04-14T08:35:39.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.551+0000 7ff236fb4640 1 -- 172.21.15.172:0/250444935 wait complete. 2024-04-14T08:35:39.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.552+0000 7ff236fb4640 1 Processor -- start 2024-04-14T08:35:39.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.552+0000 7ff236fb4640 1 -- start start 2024-04-14T08:35:39.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff236fb4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2300b47c0 0x7ff2300ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff236fb4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2300cff30 0x7ff23006dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff236fb4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff23015c950 0x7ff2301473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff236fb4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff2300d0530 con 0x7ff2300cff30 2024-04-14T08:35:39.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff236fb4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2300d03b0 con 0x7ff23015c950 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff236fb4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff23015d440 con 0x7ff2300b47c0 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff22ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2300cff30 0x7ff23006dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff22ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2300cff30 0x7ff23006dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38100/0 (socket says 172.21.15.172:38100) 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff22ffff640 1 -- 172.21.15.172:0/529861394 learned_addr learned my addr 172.21.15.172:0/529861394 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff234d50640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2300b47c0 0x7ff2300ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff235551640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff23015c950 0x7ff2301473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff22ffff640 1 -- 172.21.15.172:0/529861394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2300b47c0 msgr2=0x7ff2300ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff22ffff640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2300b47c0 0x7ff2300ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff22ffff640 1 -- 172.21.15.172:0/529861394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff23015c950 msgr2=0x7ff2301473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff22ffff640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff23015c950 0x7ff2301473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff22ffff640 1 -- 172.21.15.172:0/529861394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2300adec0 con 0x7ff2300cff30 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff22ffff640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2300cff30 0x7ff23006dcd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00c530 tx=0x7ff21c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:39.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.553+0000 7ff22d7fa640 1 -- 172.21.15.172:0/529861394 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c017070 con 0x7ff2300cff30 2024-04-14T08:35:39.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.554+0000 7ff22d7fa640 1 -- 172.21.15.172:0/529861394 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff21c005ce0 con 0x7ff2300cff30 2024-04-14T08:35:39.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.554+0000 7ff236fb4640 1 -- 172.21.15.172:0/529861394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2300ae1a0 con 0x7ff2300cff30 2024-04-14T08:35:39.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.554+0000 7ff22d7fa640 1 -- 172.21.15.172:0/529861394 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c007270 con 0x7ff2300cff30 2024-04-14T08:35:39.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.554+0000 7ff236fb4640 1 -- 172.21.15.172:0/529861394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2300ae760 con 0x7ff2300cff30 2024-04-14T08:35:39.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.555+0000 7ff22d7fa640 1 -- 172.21.15.172:0/529861394 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff21c01a070 con 0x7ff2300cff30 2024-04-14T08:35:39.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.555+0000 7ff22d7fa640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff21406c8f0 0x7ff21406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.555+0000 7ff22d7fa640 1 -- 172.21.15.172:0/529861394 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7ff21c08af20 con 0x7ff2300cff30 2024-04-14T08:35:39.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.555+0000 7ff234d50640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff21406c8f0 0x7ff21406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.556+0000 7ff234d50640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff21406c8f0 0x7ff21406edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff224036000 tx=0x7ff22403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:39.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.556+0000 7ff239d96740 1 Processor -- start 2024-04-14T08:35:39.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.556+0000 7ff239d96740 1 -- start start 2024-04-14T08:35:39.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff239d96740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce0874b60 0x55fce0847b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff239d96740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874f40 0x55fce08778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff239d96740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0848040 0x55fce0877e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff239d96740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fce07b1030 con 0x55fce0874f40 2024-04-14T08:35:39.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff239d96740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fce0759620 con 0x55fce0874b60 2024-04-14T08:35:39.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff239d96740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55fce07acfe0 con 0x55fce0848040 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff22ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874f40 0x55fce08778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff22ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874f40 0x55fce08778f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38106/0 (socket says 172.21.15.172:38106) 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff22ffff640 1 -- 172.21.15.172:0/3544505694 learned_addr learned my addr 172.21.15.172:0/3544505694 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff235551640 1 --2- 172.21.15.172:0/3544505694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0848040 0x55fce0877e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff234d50640 1 --2- 172.21.15.172:0/3544505694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce0874b60 0x55fce0847b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff22ffff640 1 -- 172.21.15.172:0/3544505694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0848040 msgr2=0x55fce0877e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff22ffff640 1 --2- 172.21.15.172:0/3544505694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0848040 0x55fce0877e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff22ffff640 1 -- 172.21.15.172:0/3544505694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce0874b60 msgr2=0x55fce0847b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff22ffff640 1 --2- 172.21.15.172:0/3544505694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce0874b60 0x55fce0847b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.557+0000 7ff22ffff640 1 -- 172.21.15.172:0/3544505694 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce04c52d0 con 0x55fce0874f40 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff22ffff640 1 --2- 172.21.15.172:0/3544505694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874f40 0x55fce08778f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0612c0 tx=0x7ff21c058e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0320e6f8bdf033f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff21a7fc640 1 -- 172.21.15.172:0/3544505694 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c086020 con 0x55fce0874f40 2024-04-14T08:35:39.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff21a7fc640 1 -- 172.21.15.172:0/3544505694 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff21c056fc0 con 0x55fce0874f40 2024-04-14T08:35:39.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff21a7fc640 1 -- 172.21.15.172:0/3544505694 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c061360 con 0x55fce0874f40 2024-04-14T08:35:39.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff239d96740 1 -- 172.21.15.172:0/3544505694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874f40 msgr2=0x55fce08778f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff239d96740 1 --2- 172.21.15.172:0/3544505694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874f40 0x55fce08778f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0612c0 tx=0x7ff21c058e00 comp rx=0 tx=0).stop 2024-04-14T08:35:39.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff239d96740 1 -- 172.21.15.172:0/3544505694 shutdown_connections 2024-04-14T08:35:39.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff239d96740 1 --2- 172.21.15.172:0/3544505694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0848040 0x55fce0877e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff239d96740 1 --2- 172.21.15.172:0/3544505694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874f40 0x55fce08778f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff239d96740 1 --2- 172.21.15.172:0/3544505694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce0874b60 0x55fce0847b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff239d96740 1 -- 172.21.15.172:0/3544505694 >> 172.21.15.172:0/3544505694 conn(0x55fce0849d70 msgr2=0x55fce08928f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:39.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff239d96740 1 -- 172.21.15.172:0/3544505694 shutdown_connections 2024-04-14T08:35:39.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.558+0000 7ff239d96740 1 -- 172.21.15.172:0/3544505694 wait complete. 2024-04-14T08:35:39.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.559+0000 7ff239d96740 1 -- start start 2024-04-14T08:35:39.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.559+0000 7ff239d96740 1 Processor -- start 2024-04-14T08:35:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff239d96740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 0x55fce0867eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff239d96740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874f40 0x55fce08914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff239d96740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce087b0e0 0x55fce087b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff239d96740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fce07b1030 con 0x55fce0874b60 2024-04-14T08:35:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff239d96740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fce076c9e0 con 0x55fce087b0e0 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff239d96740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55fce084c410 con 0x55fce0874f40 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff239d96740 10 client.0 _ll_register_callbacks cb 0x55fce07a9080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff234d50640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 0x55fce0867eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff234d50640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 0x55fce0867eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38122/0 (socket says 172.21.15.172:38122) 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff234d50640 1 -- 172.21.15.172:0/2917410958 learned_addr learned my addr 172.21.15.172:0/2917410958 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff22ffff640 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874f40 0x55fce08914a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff235551640 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce087b0e0 0x55fce087b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff234d50640 1 -- 172.21.15.172:0/2917410958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874f40 msgr2=0x55fce08914a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff234d50640 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874f40 0x55fce08914a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff234d50640 1 -- 172.21.15.172:0/2917410958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce087b0e0 msgr2=0x55fce087b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff234d50640 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce087b0e0 0x55fce087b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff234d50640 1 -- 172.21.15.172:0/2917410958 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce058f6b0 con 0x55fce0874b60 2024-04-14T08:35:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.560+0000 7ff234d50640 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 0x55fce0867eb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff2240098c0 tx=0x7ff224004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbc9de7dd541fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff20f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff20f7fe640 1 -- 172.21.15.172:0/2917410958 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224035040 con 0x55fce0874b60 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff20f7fe640 1 -- 172.21.15.172:0/2917410958 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7ff2240448d0 con 0x55fce0874b60 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff239d96740 1 -- 172.21.15.172:0/2917410958 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff21c056fc0 con 0x55fce0874b60 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff20f7fe640 1 -- 172.21.15.172:0/2917410958 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff22405b050 con 0x55fce0874b60 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff239d96740 10 client.6342 fetch_fsmap learned FSMap version 30 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff239d96740 1 -- 172.21.15.172:0/2917410958 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x55fce087a950 con 0x55fce0874b60 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff20f7fe640 1 -- 172.21.15.172:0/2917410958 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff21c056fc0 con 0x55fce0874b60 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff20f7fe640 1 -- 172.21.15.172:0/2917410958 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 3494+0+0 (secure 0 0 0) 0x7ff22405e020 con 0x55fce0874b60 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff239d96740 10 client.6342 fetch_fsmap finished waiting for FSMap version 30 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff239d96740 10 client.6342 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff239d96740 20 client.6342 populate_metadata read hostname 'smithi172' 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff239d96740 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55fce087a050 0x55fce08b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff239d96740 4 client.6342 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff239d96740 1 -- 172.21.15.172:0/2917410958 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fce0758cc0 con 0x55fce087a050 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.561+0000 7ff22ffff640 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55fce087a050 0x55fce08b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.562+0000 7ff20f7fe640 10 client.6342 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.562+0000 7ff22ffff640 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55fce087a050 0x55fce08b16d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63712a002735d23f server_cookie=e9bbe244b51c3730 in_seq=0 out_seq=0 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.562+0000 7ff20f7fe640 1 -- 172.21.15.172:0/2917410958 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fce0758cc0 con 0x55fce087a050 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.562+0000 7ff20f7fe640 10 client.6342 handle_command_reply: tid=0 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.562+0000 7ff239d96740 1 client.6342 shutdown 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.563+0000 7ff239d96740 1 -- 172.21.15.172:0/2917410958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 msgr2=0x55fce0867eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.563+0000 7ff239d96740 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 0x55fce0867eb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff2240098c0 tx=0x7ff224004730 comp rx=0 tx=0).stop 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.563+0000 7ff239d96740 1 -- 172.21.15.172:0/2917410958 shutdown_connections 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.563+0000 7ff239d96740 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55fce087a050 0x55fce08b16d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.563+0000 7ff239d96740 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce087b0e0 0x55fce087b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.563+0000 7ff239d96740 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874f40 0x55fce08914a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.563+0000 7ff20f7fe640 0 client.6342 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:35:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.563+0000 7ff239d96740 1 --2- 172.21.15.172:0/2917410958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 0x55fce0867eb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.563+0000 7ff239d96740 1 -- 172.21.15.172:0/2917410958 >> 172.21.15.172:0/2917410958 conn(0x55fce0848040 msgr2=0x55fce084bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.563+0000 7ff239d96740 1 -- 172.21.15.172:0/2917410958 shutdown_connections 2024-04-14T08:35:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.563+0000 7ff239d96740 1 -- 172.21.15.172:0/2917410958 wait complete. 2024-04-14T08:35:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.563+0000 7ff239d96740 20 client.6342 trim_cache size 0 max 16384 2024-04-14T08:35:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.581+0000 7ff239d96740 1 Processor -- start 2024-04-14T08:35:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.581+0000 7ff239d96740 1 -- start start 2024-04-14T08:35:39.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.581+0000 7ff239d96740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874b60 0x55fce08a6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.581+0000 7ff239d96740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce0874f40 0x55fce087b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.581+0000 7ff239d96740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce087b0e0 0x55fce0895ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.581+0000 7ff239d96740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff230200d30 con 0x55fce087b0e0 2024-04-14T08:35:39.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff239d96740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2301ffde0 con 0x55fce0874f40 2024-04-14T08:35:39.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff239d96740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff230148b10 con 0x55fce0874b60 2024-04-14T08:35:39.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff235551640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce087b0e0 0x55fce0895ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff235551640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce087b0e0 0x55fce0895ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38136/0 (socket says 172.21.15.172:38136) 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff22ffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce0874f40 0x55fce087b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff235551640 1 -- 172.21.15.172:0/1348822858 learned_addr learned my addr 172.21.15.172:0/1348822858 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff234d50640 1 --2- 172.21.15.172:0/1348822858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874b60 0x55fce08a6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff235551640 1 -- 172.21.15.172:0/1348822858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874b60 msgr2=0x55fce08a6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff235551640 1 --2- 172.21.15.172:0/1348822858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874b60 0x55fce08a6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff235551640 1 -- 172.21.15.172:0/1348822858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce0874f40 msgr2=0x55fce087b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff235551640 1 --2- 172.21.15.172:0/1348822858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce0874f40 0x55fce087b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff235551640 1 -- 172.21.15.172:0/1348822858 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce08a2330 con 0x55fce087b0e0 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff235551640 1 --2- 172.21.15.172:0/1348822858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce087b0e0 0x55fce0895ff0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff228001fd0 tx=0x7ff2280016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88ce18050206914 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff219ffb640 1 -- 172.21.15.172:0/1348822858 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228006b50 con 0x55fce087b0e0 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff219ffb640 1 -- 172.21.15.172:0/1348822858 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff228006cf0 con 0x55fce087b0e0 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.582+0000 7ff219ffb640 1 -- 172.21.15.172:0/1348822858 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228003b90 con 0x55fce087b0e0 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.583+0000 7ff239d96740 1 -- 172.21.15.172:0/1348822858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce087b0e0 msgr2=0x55fce0895ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.583+0000 7ff239d96740 1 --2- 172.21.15.172:0/1348822858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce087b0e0 0x55fce0895ff0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff228001fd0 tx=0x7ff2280016d0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.583+0000 7ff239d96740 1 -- 172.21.15.172:0/1348822858 shutdown_connections 2024-04-14T08:35:39.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.583+0000 7ff239d96740 1 --2- 172.21.15.172:0/1348822858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce087b0e0 0x55fce0895ff0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.583+0000 7ff239d96740 1 --2- 172.21.15.172:0/1348822858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce0874f40 0x55fce087b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.583+0000 7ff239d96740 1 --2- 172.21.15.172:0/1348822858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874b60 0x55fce08a6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.583+0000 7ff239d96740 1 -- 172.21.15.172:0/1348822858 >> 172.21.15.172:0/1348822858 conn(0x55fce0848040 msgr2=0x55fce0867270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:39.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.583+0000 7ff239d96740 1 -- 172.21.15.172:0/1348822858 shutdown_connections 2024-04-14T08:35:39.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.583+0000 7ff239d96740 1 -- 172.21.15.172:0/1348822858 wait complete. 2024-04-14T08:35:39.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.583+0000 7ff239d96740 1 -- start start 2024-04-14T08:35:39.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff239d96740 1 Processor -- start 2024-04-14T08:35:39.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff239d96740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 0x55fce08f0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff239d96740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874f40 0x55fce08b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff239d96740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce087b0e0 0x55fce08956f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff239d96740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fce08a2c30 con 0x55fce0874b60 2024-04-14T08:35:39.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff239d96740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff230149160 con 0x55fce087b0e0 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff239d96740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55fce08793b0 con 0x55fce0874f40 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff239d96740 10 client.0 _ll_register_callbacks cb 0x7ff224009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff234d50640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 0x55fce08f0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff234d50640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 0x55fce08f0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38138/0 (socket says 172.21.15.172:38138) 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff234d50640 1 -- 172.21.15.172:0/1914389351 learned_addr learned my addr 172.21.15.172:0/1914389351 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff22ffff640 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874f40 0x55fce08b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.584+0000 7ff235551640 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce087b0e0 0x55fce08956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff234d50640 1 -- 172.21.15.172:0/1914389351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874f40 msgr2=0x55fce08b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff234d50640 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874f40 0x55fce08b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff234d50640 1 -- 172.21.15.172:0/1914389351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce087b0e0 msgr2=0x55fce08956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff234d50640 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce087b0e0 0x55fce08956f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff234d50640 1 -- 172.21.15.172:0/1914389351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce08b5fd0 con 0x55fce0874b60 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff234d50640 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 0x55fce08f0f00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff2240098f0 tx=0x7ff224009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9132d4297b44f0fa server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:39.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff20f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff20f7fe640 1 -- 172.21.15.172:0/1914389351 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224031c00 con 0x55fce0874b60 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff20f7fe640 1 -- 172.21.15.172:0/1914389351 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7ff22404e810 con 0x55fce0874b60 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff239d96740 1 -- 172.21.15.172:0/1914389351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff228006cf0 con 0x55fce0874b60 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff20f7fe640 1 -- 172.21.15.172:0/1914389351 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff22406d050 con 0x55fce0874b60 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff239d96740 10 client.6348 fetch_fsmap learned FSMap version 30 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff239d96740 1 -- 172.21.15.172:0/1914389351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x55fce08951d0 con 0x55fce0874b60 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff20f7fe640 1 -- 172.21.15.172:0/1914389351 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff228006cf0 con 0x55fce0874b60 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff20f7fe640 1 -- 172.21.15.172:0/1914389351 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 3494+0+0 (secure 0 0 0) 0x7ff224070020 con 0x55fce0874b60 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff239d96740 10 client.6348 fetch_fsmap finished waiting for FSMap version 30 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff239d96740 10 client.6348 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff239d96740 20 client.6348 populate_metadata read hostname 'smithi172' 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff239d96740 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55fce08b6e80 0x55fce08468d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff239d96740 4 client.6348 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:35:39.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.585+0000 7ff239d96740 1 -- 172.21.15.172:0/1914389351 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "status"}) -- 0x55fce07be570 con 0x55fce08b6e80 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff22ffff640 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55fce08b6e80 0x55fce08468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff22ffff640 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55fce08b6e80 0x55fce08468d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4840e9ec9a78f493 server_cookie=fced8b06aad4663 in_seq=0 out_seq=0 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff20f7fe640 10 client.6348 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff20f7fe640 1 -- 172.21.15.172:0/1914389351 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55fce07be570 con 0x55fce08b6e80 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff20f7fe640 10 client.6348 handle_command_reply: tid=0 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff239d96740 1 client.6348 shutdown 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff239d96740 1 -- 172.21.15.172:0/1914389351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 msgr2=0x55fce08f0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff239d96740 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 0x55fce08f0f00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff2240098f0 tx=0x7ff224009190 comp rx=0 tx=0).stop 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff239d96740 1 -- 172.21.15.172:0/1914389351 shutdown_connections 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff239d96740 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55fce08b6e80 0x55fce08468d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff239d96740 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fce087b0e0 0x55fce08956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff239d96740 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fce0874f40 0x55fce08b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff239d96740 1 --2- 172.21.15.172:0/1914389351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fce0874b60 0x55fce08f0f00 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff20f7fe640 0 client.6348 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:35:39.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff239d96740 1 -- 172.21.15.172:0/1914389351 >> 172.21.15.172:0/1914389351 conn(0x55fce0848040 msgr2=0x55fce0895ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:39.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff239d96740 1 -- 172.21.15.172:0/1914389351 shutdown_connections 2024-04-14T08:35:39.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.586+0000 7ff239d96740 1 -- 172.21.15.172:0/1914389351 wait complete. 2024-04-14T08:35:39.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff239d96740 20 client.6348 trim_cache size 0 max 16384 2024-04-14T08:35:39.594 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:35:39.594 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:35:39.594 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:35:39.594 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 6204, 2024-04-14T08:35:39.595 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:35:39.595 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:35:39.595 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:35:39.595 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 4.6313320190000002, 2024-04-14T08:35:39.595 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 30, 2024-04-14T08:35:39.595 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 61, 2024-04-14T08:35:39.595 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 61, 2024-04-14T08:35:39.595 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 9.6516910780000007 2024-04-14T08:35:39.595 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:35:39.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff236fb4640 1 -- 172.21.15.172:0/529861394 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff21406c8f0 msgr2=0x7ff21406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff236fb4640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff21406c8f0 0x7ff21406edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff224036000 tx=0x7ff22403c040 comp rx=0 tx=0).stop 2024-04-14T08:35:39.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff236fb4640 1 -- 172.21.15.172:0/529861394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2300cff30 msgr2=0x7ff23006dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff236fb4640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2300cff30 0x7ff23006dcd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00c530 tx=0x7ff21c00c9f0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff236fb4640 1 -- 172.21.15.172:0/529861394 shutdown_connections 2024-04-14T08:35:39.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff236fb4640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff21406c8f0 0x7ff21406edb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff236fb4640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff23015c950 0x7ff2301473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff236fb4640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2300cff30 0x7ff23006dcd0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff236fb4640 1 --2- 172.21.15.172:0/529861394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2300b47c0 0x7ff2300ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff236fb4640 1 -- 172.21.15.172:0/529861394 >> 172.21.15.172:0/529861394 conn(0x7ff2300d0780 msgr2=0x7ff23006c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:39.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff236fb4640 1 -- 172.21.15.172:0/529861394 shutdown_connections 2024-04-14T08:35:39.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.587+0000 7ff236fb4640 1 -- 172.21.15.172:0/529861394 wait complete. 2024-04-14T08:35:39.598 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:35:39.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.699+0000 7fc9824d1640 1 Processor -- start 2024-04-14T08:35:39.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.699+0000 7fc9824d1640 1 -- start start 2024-04-14T08:35:39.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.700+0000 7fc9824d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc97c0b5b50 0x7fc97c15c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.700+0000 7fc9824d1640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97c0addb0 con 0x7fc97c06b520 2024-04-14T08:35:39.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.700+0000 7fc9824d1640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97c0ae090 con 0x7fc97c15c990 2024-04-14T08:35:39.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.700+0000 7fc9824d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc97c0ae650 con 0x7fc97c0b5b50 2024-04-14T08:35:39.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.700+0000 7fc97bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc97c06b520 0x7fc97c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55288/0 (socket says 172.21.15.172:55288) 2024-04-14T08:35:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.700+0000 7fc97bfff640 1 -- 172.21.15.172:0/1112089720 learned_addr learned my addr 172.21.15.172:0/1112089720 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.700+0000 7fc97b7fe640 1 --2- 172.21.15.172:0/1112089720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc97c0b5b50 0x7fc97c15c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.701+0000 7fc97affd640 1 -- 172.21.15.172:0/1112089720 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2169424296 0 0) 0x7fc97c0addb0 con 0x7fc97c06b520 2024-04-14T08:35:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.701+0000 7fc97affd640 1 -- 172.21.15.172:0/1112089720 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc964003520 con 0x7fc97c06b520 2024-04-14T08:35:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.701+0000 7fc97affd640 1 -- 172.21.15.172:0/1112089720 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2617037444 0 0) 0x7fc97c0ae090 con 0x7fc97c15c990 2024-04-14T08:35:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.701+0000 7fc97affd640 1 -- 172.21.15.172:0/1112089720 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc97c0addb0 con 0x7fc97c15c990 2024-04-14T08:35:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.701+0000 7fc97b7fe640 1 -- 172.21.15.172:0/1112089720 >> v1:172.21.15.177:6789/0 conn(0x7fc97c15c990 legacy=0x7fc97c0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.701+0000 7fc97b7fe640 1 -- 172.21.15.172:0/1112089720 >> v1:172.21.15.172:6789/0 conn(0x7fc97c06b520 legacy=0x7fc97c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.701+0000 7fc97b7fe640 1 -- 172.21.15.172:0/1112089720 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c0ae370 con 0x7fc97c0b5b50 2024-04-14T08:35:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.701+0000 7fc97b7fe640 1 --2- 172.21.15.172:0/1112089720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc97c0b5b50 0x7fc97c15c450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc9700097d0 tx=0x7fc97002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d11da6689ecfb31 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.702+0000 7fc97affd640 1 -- 172.21.15.172:0/1112089720 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970035020 con 0x7fc97c0b5b50 2024-04-14T08:35:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.702+0000 7fc97affd640 1 -- 172.21.15.172:0/1112089720 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc970031890 con 0x7fc97c0b5b50 2024-04-14T08:35:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.702+0000 7fc97affd640 1 -- 172.21.15.172:0/1112089720 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970031c60 con 0x7fc97c0b5b50 2024-04-14T08:35:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.702+0000 7fc9824d1640 1 -- 172.21.15.172:0/1112089720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc97c0b5b50 msgr2=0x7fc97c15c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.702+0000 7fc9824d1640 1 --2- 172.21.15.172:0/1112089720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc97c0b5b50 0x7fc97c15c450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc9700097d0 tx=0x7fc97002b930 comp rx=0 tx=0).stop 2024-04-14T08:35:39.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.702+0000 7fc9824d1640 1 -- 172.21.15.172:0/1112089720 shutdown_connections 2024-04-14T08:35:39.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.702+0000 7fc9824d1640 1 --2- 172.21.15.172:0/1112089720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc97c0b5b50 0x7fc97c15c450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.702+0000 7fc9824d1640 1 -- 172.21.15.172:0/1112089720 >> 172.21.15.172:0/1112089720 conn(0x7fc97c0d0f20 msgr2=0x7fc97c0d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:39.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.702+0000 7fc9824d1640 1 -- 172.21.15.172:0/1112089720 shutdown_connections 2024-04-14T08:35:39.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.702+0000 7fc9824d1640 1 -- 172.21.15.172:0/1112089720 wait complete. 2024-04-14T08:35:39.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.703+0000 7fc9824d1640 1 Processor -- start 2024-04-14T08:35:39.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.704+0000 7fc9824d1640 1 -- start start 2024-04-14T08:35:39.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.704+0000 7fc9824d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc97c06b520 0x7fc97c159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.704+0000 7fc9824d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc97c0b5b50 0x7fc97c151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.704+0000 7fc9824d1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc97c1f1d10 0x7fc97c1f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.704+0000 7fc9824d1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c09ef20 con 0x7fc97c06b520 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.704+0000 7fc9824d1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c09eda0 con 0x7fc97c1f1d10 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.704+0000 7fc9824d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc97c0b5600 con 0x7fc97c0b5b50 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.704+0000 7fc97bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc97c06b520 0x7fc97c159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.704+0000 7fc97bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc97c06b520 0x7fc97c159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38150/0 (socket says 172.21.15.172:38150) 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.704+0000 7fc97bfff640 1 -- 172.21.15.172:0/1838899823 learned_addr learned my addr 172.21.15.172:0/1838899823 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.704+0000 7fc97b7fe640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc97c0b5b50 0x7fc97c151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.704+0000 7fc980a6e640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc97c1f1d10 0x7fc97c1f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.705+0000 7fc97bfff640 1 -- 172.21.15.172:0/1838899823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc97c0b5b50 msgr2=0x7fc97c151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.705+0000 7fc97bfff640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc97c0b5b50 0x7fc97c151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.705+0000 7fc97bfff640 1 -- 172.21.15.172:0/1838899823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc97c1f1d10 msgr2=0x7fc97c1f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.705+0000 7fc97bfff640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc97c1f1d10 0x7fc97c1f21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.705+0000 7fc97bfff640 1 -- 172.21.15.172:0/1838899823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c152610 con 0x7fc97c06b520 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.705+0000 7fc97bfff640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc97c06b520 0x7fc97c159e80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00c530 tx=0x7fc96c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.705+0000 7fc978ff9640 1 -- 172.21.15.172:0/1838899823 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c017070 con 0x7fc97c06b520 2024-04-14T08:35:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.705+0000 7fc978ff9640 1 -- 172.21.15.172:0/1838899823 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc96c005ce0 con 0x7fc97c06b520 2024-04-14T08:35:39.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.705+0000 7fc9824d1640 1 -- 172.21.15.172:0/1838899823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97c1528f0 con 0x7fc97c06b520 2024-04-14T08:35:39.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.705+0000 7fc978ff9640 1 -- 172.21.15.172:0/1838899823 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c007270 con 0x7fc97c06b520 2024-04-14T08:35:39.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.705+0000 7fc9824d1640 1 -- 172.21.15.172:0/1838899823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97c0b0da0 con 0x7fc97c06b520 2024-04-14T08:35:39.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.706+0000 7fc9824d1640 1 -- 172.21.15.172:0/1838899823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97c068520 con 0x7fc97c06b520 2024-04-14T08:35:39.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.706+0000 7fc978ff9640 1 -- 172.21.15.172:0/1838899823 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc96c01a070 con 0x7fc97c06b520 2024-04-14T08:35:39.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.707+0000 7fc978ff9640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc96806c8f0 0x7fc96806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:39.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.707+0000 7fc978ff9640 1 -- 172.21.15.172:0/1838899823 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fc96c08b050 con 0x7fc97c06b520 2024-04-14T08:35:39.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.707+0000 7fc97b7fe640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc96806c8f0 0x7fc96806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:39.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.707+0000 7fc97b7fe640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc96806c8f0 0x7fc96806edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc970036000 tx=0x7fc970008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:39.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.711+0000 7fc978ff9640 1 -- 172.21.15.172:0/1838899823 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc96c058c20 con 0x7fc97c06b520 2024-04-14T08:35:39.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.922+0000 7fc9824d1640 1 -- 172.21.15.172:0/1838899823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc97c0b3c50 con 0x7fc97c06b520 2024-04-14T08:35:39.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.923+0000 7fc978ff9640 1 -- 172.21.15.172:0/1838899823 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) ==== 53+0+73 (secure 0 0 0) 0x7fc96c05cf20 con 0x7fc97c06b520 2024-04-14T08:35:39.923 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:35:39.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.925+0000 7fc9824d1640 1 -- 172.21.15.172:0/1838899823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc96806c8f0 msgr2=0x7fc96806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.925+0000 7fc9824d1640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc96806c8f0 0x7fc96806edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc970036000 tx=0x7fc970008030 comp rx=0 tx=0).stop 2024-04-14T08:35:39.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.925+0000 7fc9824d1640 1 -- 172.21.15.172:0/1838899823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc97c06b520 msgr2=0x7fc97c159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:39.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.925+0000 7fc9824d1640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc97c06b520 0x7fc97c159e80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00c530 tx=0x7fc96c00c9f0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.925+0000 7fc9824d1640 1 -- 172.21.15.172:0/1838899823 shutdown_connections 2024-04-14T08:35:39.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.925+0000 7fc9824d1640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc96806c8f0 0x7fc96806edb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.925+0000 7fc9824d1640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc97c1f1d10 0x7fc97c1f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.925+0000 7fc9824d1640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc97c0b5b50 0x7fc97c151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.925+0000 7fc9824d1640 1 --2- 172.21.15.172:0/1838899823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc97c06b520 0x7fc97c159e80 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:39.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.925+0000 7fc9824d1640 1 -- 172.21.15.172:0/1838899823 >> 172.21.15.172:0/1838899823 conn(0x7fc97c0d0f20 msgr2=0x7fc97c0afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:39.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.925+0000 7fc9824d1640 1 -- 172.21.15.172:0/1838899823 shutdown_connections 2024-04-14T08:35:39.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:39.925+0000 7fc9824d1640 1 -- 172.21.15.172:0/1838899823 wait complete. 2024-04-14T08:35:39.936 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:35:39.936 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:35:39.936 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:35:39.936 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:35:39.936 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:35:39.937 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:35:39.937 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:35:39.937 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:35:39.937 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:35:39.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:39.937 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:35:39.952 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:35:39.952 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:35:39.952 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:35:39.952 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:35:39.952 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:35:39.953 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42529sec preferred_lft 42529sec 2024-04-14T08:35:39.954 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:35:39.954 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:35:39.954 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:35:39.954 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:35:39.954 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:35:39.954 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:35:39.954 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:35:39.954 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:35:39.954 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:35:39.954 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:35:39.954 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:35:39.954 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:35:39.955 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:35:39.955 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:35:39.955 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:35:39.955 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:35:39.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:39.955 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:35:40.018 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:35:40.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:40.020 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:35:40.076 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:35:40.076 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:35:40.076 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:35:40.076 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:35:40.076 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:35:40.077 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:35:40.077 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:35:40.077 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:35:40.077 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:35:40.077 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:35:40.077 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:35:40.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:35:40.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:35:40.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:40.213 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:35:40.227 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:35:40.227 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:35:40.228 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:35:40.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:40.228 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:35:40.294 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:35:40.294 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:35:40.350 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:35:40.350 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:35:40.351 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:35:40.405 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:35:40.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:40.405 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:35:40.490 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:35:40.490 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:35:40.490 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:35:40.498 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:35:40.498 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:35:40.498 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:35:40.499 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:35:40.499 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:35:40.504 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:35:40.504 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:35:40.504 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:35:40.504 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:35:40.504 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:35:40.504 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:35:40.504 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:35:40.504 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:35:40.505 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:35:40.505 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:35:40.505 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:35:40.505 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:35:40.505 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:35:40.506 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:35:40.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:40.506 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:35:40.533 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:35:40.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:35:40.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:35:40.634 INFO:teuthology.orchestra.run.smithi172.stdout:client6115 2024-04-14T08:35:40.634 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:35:40.634 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:35:40.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:40.650 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-14T08:35:40.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:40.709 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-14T08:35:40.765 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:35:40.766 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:35:40.766 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:35:40.766 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:35:40.766 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-14T08:35:40.766 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:35:40.871 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627777 2024-04-14T08:35:40.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:35:40.879 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:35:41.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.021+0000 7fa738ad1640 1 Processor -- start 2024-04-14T08:35:41.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.021+0000 7fa738ad1640 1 -- start start 2024-04-14T08:35:41.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.022+0000 7fa738ad1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa734072970 0x7fa73415b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.022+0000 7fa738ad1640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7340b3360 con 0x7fa73415a690 2024-04-14T08:35:41.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.022+0000 7fa738ad1640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7340b3640 con 0x7fa734072470 2024-04-14T08:35:41.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.022+0000 7fa738ad1640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa7340b3920 con 0x7fa734072970 2024-04-14T08:35:41.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.022+0000 7fa732d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa734072970 0x7fa73415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.022+0000 7fa73259c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa73415a690 0x7fa734071d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55292/0 (socket says 172.21.15.172:55292) 2024-04-14T08:35:41.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.022+0000 7fa73259c640 1 -- 172.21.15.172:0/2426123054 learned_addr learned my addr 172.21.15.172:0/2426123054 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:41.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.022+0000 7fa732d9d640 1 -- 172.21.15.172:0/2426123054 >> v1:172.21.15.177:6789/0 conn(0x7fa734072470 legacy=0x7fa73415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.022+0000 7fa732d9d640 1 -- 172.21.15.172:0/2426123054 >> v1:172.21.15.172:6789/0 conn(0x7fa73415a690 legacy=0x7fa734071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.022+0000 7fa731d9b640 1 -- 172.21.15.172:0/2426123054 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 999809306 0 0) 0x7fa7340b3360 con 0x7fa73415a690 2024-04-14T08:35:41.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.022+0000 7fa732d9d640 1 -- 172.21.15.172:0/2426123054 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7340b3c00 con 0x7fa734072970 2024-04-14T08:35:41.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.023+0000 7fa732d9d640 1 --2- 172.21.15.172:0/2426123054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa734072970 0x7fa73415b940 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00bfa0 tx=0x7fa72c02e0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf1c037a93f4b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.023+0000 7fa731d9b640 1 -- 172.21.15.172:0/2426123054 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c034070 con 0x7fa734072970 2024-04-14T08:35:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.023+0000 7fa731d9b640 1 -- 172.21.15.172:0/2426123054 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72c038400 con 0x7fa734072970 2024-04-14T08:35:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.023+0000 7fa731d9b640 1 -- 172.21.15.172:0/2426123054 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c02ee60 con 0x7fa734072970 2024-04-14T08:35:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.024+0000 7fa738ad1640 1 -- 172.21.15.172:0/2426123054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa734072970 msgr2=0x7fa73415b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.024+0000 7fa738ad1640 1 --2- 172.21.15.172:0/2426123054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa734072970 0x7fa73415b940 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00bfa0 tx=0x7fa72c02e0b0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.024+0000 7fa738ad1640 1 -- 172.21.15.172:0/2426123054 shutdown_connections 2024-04-14T08:35:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.024+0000 7fa738ad1640 1 --2- 172.21.15.172:0/2426123054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa734072970 0x7fa73415b940 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.024+0000 7fa738ad1640 1 -- 172.21.15.172:0/2426123054 >> 172.21.15.172:0/2426123054 conn(0x7fa73406ec30 msgr2=0x7fa73406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.024+0000 7fa738ad1640 1 -- 172.21.15.172:0/2426123054 shutdown_connections 2024-04-14T08:35:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.024+0000 7fa738ad1640 1 -- 172.21.15.172:0/2426123054 wait complete. 2024-04-14T08:35:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.025+0000 7fa738ad1640 1 Processor -- start 2024-04-14T08:35:41.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.025+0000 7fa738ad1640 1 -- start start 2024-04-14T08:35:41.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa738ad1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa734072470 0x7fa734142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa738ad1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa734072970 0x7fa7341518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa738ad1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa73415a690 0x7fa734151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa738ad1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa73415abf0 con 0x7fa734072470 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa738ad1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa73415aa70 con 0x7fa73415a690 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa738ad1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa73415ad70 con 0x7fa734072970 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa73259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa734072470 0x7fa734142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa73259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa734072470 0x7fa734142960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38164/0 (socket says 172.21.15.172:38164) 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa73259c640 1 -- 172.21.15.172:0/2843254371 learned_addr learned my addr 172.21.15.172:0/2843254371 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa72bfff640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa734072970 0x7fa7341518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa732d9d640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa73415a690 0x7fa734151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa73259c640 1 -- 172.21.15.172:0/2843254371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa734072970 msgr2=0x7fa7341518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa73259c640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa734072970 0x7fa7341518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa73259c640 1 -- 172.21.15.172:0/2843254371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa73415a690 msgr2=0x7fa734151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa73259c640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa73415a690 0x7fa734151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa73259c640 1 -- 172.21.15.172:0/2843254371 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7340cdd40 con 0x7fa734072470 2024-04-14T08:35:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa73259c640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa734072470 0x7fa734142960 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa72400c3f0 tx=0x7fa72400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa72affd640 1 -- 172.21.15.172:0/2843254371 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7240074e0 con 0x7fa734072470 2024-04-14T08:35:41.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa72affd640 1 -- 172.21.15.172:0/2843254371 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7240047e0 con 0x7fa734072470 2024-04-14T08:35:41.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa72affd640 1 -- 172.21.15.172:0/2843254371 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724004c10 con 0x7fa734072470 2024-04-14T08:35:41.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa738ad1640 1 -- 172.21.15.172:0/2843254371 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7340ce020 con 0x7fa734072470 2024-04-14T08:35:41.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.026+0000 7fa738ad1640 1 -- 172.21.15.172:0/2843254371 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7340ce5e0 con 0x7fa734072470 2024-04-14T08:35:41.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.028+0000 7fa738ad1640 1 -- 172.21.15.172:0/2843254371 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa734068520 con 0x7fa734072470 2024-04-14T08:35:41.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.028+0000 7fa72affd640 1 -- 172.21.15.172:0/2843254371 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa724011070 con 0x7fa734072470 2024-04-14T08:35:41.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.028+0000 7fa72affd640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa71406c8f0 0x7fa71406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.028+0000 7fa72affd640 1 -- 172.21.15.172:0/2843254371 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fa72408b7d0 con 0x7fa734072470 2024-04-14T08:35:41.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.028+0000 7fa72bfff640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa71406c8f0 0x7fa71406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.029+0000 7fa72bfff640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa71406c8f0 0x7fa71406edb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa734150290 tx=0x7fa71c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.033+0000 7fa72affd640 1 -- 172.21.15.172:0/2843254371 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa724059420 con 0x7fa734072470 2024-04-14T08:35:41.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.245+0000 7fa738ad1640 1 -- 172.21.15.172:0/2843254371 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7340b4bc0 con 0x7fa734072470 2024-04-14T08:35:41.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.246+0000 7fa72affd640 1 -- 172.21.15.172:0/2843254371 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5194 (secure 0 0 0) 0x7fa72405d720 con 0x7fa734072470 2024-04-14T08:35:41.247 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:41.247 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3621780015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3621780015},{"type":"v1","addr":"172.21.15.172:6836","nonce":3621780015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3894683748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3894683748},{"type":"v1","addr":"172.21.15.172:6837","nonce":3894683748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3625745587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3625745587},{"type":"v1","addr":"172.21.15.177:6834","nonce":3625745587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:35:34.316542+0000","modified":"2024-04-14T08:35:36.314930+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/4176013980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":4176013980},{"type":"v1","addr":"172.21.15.177:6835","nonce":4176013980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-14T08:35:41.247 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 30 2024-04-14T08:35:41.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.248+0000 7fa738ad1640 1 -- 172.21.15.172:0/2843254371 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa71406c8f0 msgr2=0x7fa71406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.248+0000 7fa738ad1640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa71406c8f0 0x7fa71406edb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa734150290 tx=0x7fa71c009040 comp rx=0 tx=0).stop 2024-04-14T08:35:41.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.248+0000 7fa738ad1640 1 -- 172.21.15.172:0/2843254371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa734072470 msgr2=0x7fa734142960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.248+0000 7fa738ad1640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa734072470 0x7fa734142960 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa72400c3f0 tx=0x7fa72400c8b0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.248+0000 7fa738ad1640 1 -- 172.21.15.172:0/2843254371 shutdown_connections 2024-04-14T08:35:41.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.248+0000 7fa738ad1640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa71406c8f0 0x7fa71406edb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.248+0000 7fa738ad1640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa73415a690 0x7fa734151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.248+0000 7fa738ad1640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa734072970 0x7fa7341518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.248+0000 7fa738ad1640 1 --2- 172.21.15.172:0/2843254371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa734072470 0x7fa734142960 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.248+0000 7fa738ad1640 1 -- 172.21.15.172:0/2843254371 >> 172.21.15.172:0/2843254371 conn(0x7fa73406ec30 msgr2=0x7fa7340cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.248+0000 7fa738ad1640 1 -- 172.21.15.172:0/2843254371 shutdown_connections 2024-04-14T08:35:41.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.248+0000 7fa738ad1640 1 -- 172.21.15.172:0/2843254371 wait complete. 2024-04-14T08:35:41.259 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-14T08:35:41.391 INFO:teuthology.orchestra.run.smithi177.stdout:{"message":"","return_code":0} 2024-04-14T08:35:41.392 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-14T08:35:41.392 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-14T08:35:41.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.435+0000 7fca16041f40 1 Processor -- start 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.435+0000 7fca16041f40 1 -- start start 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca16041f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e724bdf50 0x563e7252c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca16041f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563e725361b0 0x563e7254bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca16041f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e7254e0b0 0x563e72534000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca16041f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563e7254e890 con 0x563e7254e0b0 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca16041f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563e7241e3b0 con 0x563e724bdf50 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca16041f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563e7254eb30 con 0x563e725361b0 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca14144640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e7254e0b0 0x563e72534000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca14144640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e7254e0b0 0x563e72534000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38174/0 (socket says 172.21.15.172:38174) 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca14144640 1 -- 172.21.15.172:0/1082322992 learned_addr learned my addr 172.21.15.172:0/1082322992 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca13142640 1 --2- 172.21.15.172:0/1082322992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563e725361b0 0x563e7254bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca13943640 1 --2- 172.21.15.172:0/1082322992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e724bdf50 0x563e7252c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca14144640 1 -- 172.21.15.172:0/1082322992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563e725361b0 msgr2=0x563e7254bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca14144640 1 --2- 172.21.15.172:0/1082322992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563e725361b0 0x563e7254bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca14144640 1 -- 172.21.15.172:0/1082322992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e724bdf50 msgr2=0x563e7252c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca14144640 1 --2- 172.21.15.172:0/1082322992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e724bdf50 0x563e7252c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca14144640 1 -- 172.21.15.172:0/1082322992 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e72535700 con 0x563e7254e0b0 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca14144640 1 --2- 172.21.15.172:0/1082322992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e7254e0b0 0x563e72534000 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fca0c001d00 tx=0x7fca0c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05128ccff5a7ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca12140640 1 -- 172.21.15.172:0/1082322992 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00bd80 con 0x563e7254e0b0 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.436+0000 7fca12140640 1 -- 172.21.15.172:0/1082322992 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0c001e50 con 0x563e7254e0b0 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.437+0000 7fca12140640 1 -- 172.21.15.172:0/1082322992 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00f640 con 0x563e7254e0b0 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.437+0000 7fca16041f40 1 -- 172.21.15.172:0/1082322992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e7254e0b0 msgr2=0x563e72534000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.437+0000 7fca16041f40 1 --2- 172.21.15.172:0/1082322992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e7254e0b0 0x563e72534000 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fca0c001d00 tx=0x7fca0c00b5b0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.437+0000 7fca16041f40 1 -- 172.21.15.172:0/1082322992 shutdown_connections 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.437+0000 7fca16041f40 1 --2- 172.21.15.172:0/1082322992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e7254e0b0 0x563e72534000 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.437+0000 7fca16041f40 1 --2- 172.21.15.172:0/1082322992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563e725361b0 0x563e7254bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.437+0000 7fca16041f40 1 --2- 172.21.15.172:0/1082322992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e724bdf50 0x563e7252c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.437+0000 7fca16041f40 1 -- 172.21.15.172:0/1082322992 >> 172.21.15.172:0/1082322992 conn(0x563e724a13d0 msgr2=0x563e72531a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.437+0000 7fca16041f40 1 -- 172.21.15.172:0/1082322992 shutdown_connections 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.437+0000 7fca16041f40 1 -- 172.21.15.172:0/1082322992 wait complete. 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca16041f40 1 Processor -- start 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca16041f40 1 -- start start 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca16041f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563e724bdf50 0x563e725e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca16041f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e725361b0 0x563e72530490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca16041f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e7254e0b0 0x563e72553960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca16041f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563e7254c440 con 0x563e725361b0 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca16041f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563e7252cab0 con 0x563e7254e0b0 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca16041f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563e72535460 con 0x563e724bdf50 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca13142640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e725361b0 0x563e72530490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca13142640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e725361b0 0x563e72530490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38190/0 (socket says 172.21.15.172:38190) 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca13142640 1 -- 172.21.15.172:0/1766168734 learned_addr learned my addr 172.21.15.172:0/1766168734 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca13943640 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563e724bdf50 0x563e725e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.438+0000 7fca14144640 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e7254e0b0 0x563e72553960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.439+0000 7fca13142640 1 -- 172.21.15.172:0/1766168734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563e724bdf50 msgr2=0x563e725e79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.439+0000 7fca13142640 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563e724bdf50 0x563e725e79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.439+0000 7fca13142640 1 -- 172.21.15.172:0/1766168734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e7254e0b0 msgr2=0x563e72553960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.439+0000 7fca13142640 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e7254e0b0 0x563e72553960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.439+0000 7fca13142640 1 -- 172.21.15.172:0/1766168734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e725e7f20 con 0x563e725361b0 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.439+0000 7fca13142640 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e725361b0 0x563e72530490 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fca0801ced0 tx=0x7fca08002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.439+0000 7fca1113e640 1 -- 172.21.15.172:0/1766168734 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08063020 con 0x563e725361b0 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.439+0000 7fca1113e640 1 -- 172.21.15.172:0/1766168734 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca08058040 con 0x563e725361b0 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.439+0000 7fca16041f40 1 -- 172.21.15.172:0/1766168734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563e72531340 con 0x563e725361b0 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.439+0000 7fca1113e640 1 -- 172.21.15.172:0/1766168734 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08057040 con 0x563e725361b0 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.439+0000 7fca16041f40 1 -- 172.21.15.172:0/1766168734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e725315e0 con 0x563e725361b0 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.440+0000 7fca1113e640 1 -- 172.21.15.172:0/1766168734 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca08066020 con 0x563e725361b0 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.441+0000 7fca1113e640 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fca0006b2d0 0x7fca0006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.441+0000 7fca1113e640 1 -- 172.21.15.172:0/1766168734 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fca080cca70 con 0x563e725361b0 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.441+0000 7fca13943640 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fca0006b2d0 0x7fca0006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.441+0000 7fca16041f40 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x563e7254b040 0x563e724c0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.441+0000 7fca16041f40 1 -- 172.21.15.172:0/1766168734 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61) -- 0x563e724c0980 con 0x563e7254b040 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.441+0000 7fca14144640 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x563e7254b040 0x563e724c0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.441+0000 7fca13943640 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fca0006b2d0 0x7fca0006d790 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x563e72531250 tx=0x7fca04007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca14144640 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x563e7254b040 0x563e724c0440 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca14144640 1 -- 172.21.15.172:0/1766168734 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7fca0c01d450 con 0x563e7254b040 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca16041f40 1 -- 172.21.15.172:0/1766168734 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x563e7254b040 msgr2=0x563e724c0440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca16041f40 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x563e7254b040 0x563e724c0440 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca16041f40 1 -- 172.21.15.172:0/1766168734 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fca0006b2d0 msgr2=0x7fca0006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca16041f40 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fca0006b2d0 0x7fca0006d790 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x563e72531250 tx=0x7fca04007040 comp rx=0 tx=0).stop 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca16041f40 1 -- 172.21.15.172:0/1766168734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e725361b0 msgr2=0x563e72530490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca16041f40 1 --2- 172.21.15.172:0/1766168734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e725361b0 0x563e72530490 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fca0801ced0 tx=0x7fca08002fb0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca14144640 1 -- 172.21.15.172:0/1766168734 reap_dead start 2024-04-14T08:35:41.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca16041f40 1 -- 172.21.15.172:0/1766168734 shutdown_connections 2024-04-14T08:35:41.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca16041f40 1 -- 172.21.15.172:0/1766168734 >> 172.21.15.172:0/1766168734 conn(0x563e724a13d0 msgr2=0x563e7254bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca16041f40 1 -- 172.21.15.172:0/1766168734 shutdown_connections 2024-04-14T08:35:41.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.442+0000 7fca16041f40 1 -- 172.21.15.172:0/1766168734 wait complete. 2024-04-14T08:35:41.449 INFO:teuthology.orchestra.run.smithi172.stdout:Xfiledir_test 2024-04-14T08:35:41.450 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:41.450 DEBUG:teuthology.orchestra.run.smithi172:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-14T08:35:41.536 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:35:41.536 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 1099511627777, 2024-04-14T08:35:41.536 INFO:teuthology.orchestra.run.smithi172.stdout: "ancestors": [ 2024-04-14T08:35:41.536 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:41.536 INFO:teuthology.orchestra.run.smithi172.stdout: "dirino": 1099511627776, 2024-04-14T08:35:41.536 INFO:teuthology.orchestra.run.smithi172.stdout: "dname": "file", 2024-04-14T08:35:41.536 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 2 2024-04-14T08:35:41.536 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:35:41.536 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:35:41.537 INFO:teuthology.orchestra.run.smithi172.stdout: "dirino": 1, 2024-04-14T08:35:41.537 INFO:teuthology.orchestra.run.smithi172.stdout: "dname": "dir_test", 2024-04-14T08:35:41.537 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 4 2024-04-14T08:35:41.537 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:41.537 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-04-14T08:35:41.537 INFO:teuthology.orchestra.run.smithi172.stdout: "pool": 10, 2024-04-14T08:35:41.537 INFO:teuthology.orchestra.run.smithi172.stdout: "old_pools": [] 2024-04-14T08:35:41.537 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:35:41.537 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:35:41.538 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-14T08:35:41.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.621+0000 7f89c4cd9f40 1 Processor -- start 2024-04-14T08:35:41.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.621+0000 7f89c4cd9f40 1 -- start start 2024-04-14T08:35:41.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.621+0000 7f89c4cd9f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a6a1c8040 0x560a6a2097d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.621+0000 7f89c4cd9f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560a6a1dc0d0 0x560a6a207600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.621+0000 7f89c4cd9f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a297f80 0x560a6a346360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.621+0000 7f89c4cd9f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560a6a1813b0 con 0x560a6a297f80 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.621+0000 7f89c4cd9f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560a6a185400 con 0x560a6a1c8040 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.621+0000 7f89c4cd9f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x560a6a1d8e90 con 0x560a6a1dc0d0 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.621+0000 7f89c2da4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a297f80 0x560a6a346360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.621+0000 7f89c2da4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a297f80 0x560a6a346360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38222/0 (socket says 172.21.15.172:38222) 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c1da2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560a6a1dc0d0 0x560a6a207600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c2da4640 1 -- 172.21.15.172:0/3336369163 learned_addr learned my addr 172.21.15.172:0/3336369163 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c25a3640 1 --2- 172.21.15.172:0/3336369163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a6a1c8040 0x560a6a2097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c2da4640 1 -- 172.21.15.172:0/3336369163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560a6a1dc0d0 msgr2=0x560a6a207600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c2da4640 1 --2- 172.21.15.172:0/3336369163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560a6a1dc0d0 0x560a6a207600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c2da4640 1 -- 172.21.15.172:0/3336369163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a6a1c8040 msgr2=0x560a6a2097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c2da4640 1 --2- 172.21.15.172:0/3336369163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a6a1c8040 0x560a6a2097d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c2da4640 1 -- 172.21.15.172:0/3336369163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6a204030 con 0x560a6a297f80 2024-04-14T08:35:41.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c2da4640 1 --2- 172.21.15.172:0/3336369163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a297f80 0x560a6a346360 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f89bc001d50 tx=0x7f89bc00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c5de0e2dd3aef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c0da0640 1 -- 172.21.15.172:0/3336369163 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00bd80 con 0x560a6a297f80 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c0da0640 1 -- 172.21.15.172:0/3336369163 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89bc00f400 con 0x560a6a297f80 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c0da0640 1 -- 172.21.15.172:0/3336369163 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00f7f0 con 0x560a6a297f80 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/3336369163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a297f80 msgr2=0x560a6a346360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.622+0000 7f89c4cd9f40 1 --2- 172.21.15.172:0/3336369163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a297f80 0x560a6a346360 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f89bc001d50 tx=0x7f89bc00b5b0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.623+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/3336369163 shutdown_connections 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.623+0000 7f89c4cd9f40 1 --2- 172.21.15.172:0/3336369163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a297f80 0x560a6a346360 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.623+0000 7f89c4cd9f40 1 --2- 172.21.15.172:0/3336369163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560a6a1dc0d0 0x560a6a207600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.623+0000 7f89c4cd9f40 1 --2- 172.21.15.172:0/3336369163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a6a1c8040 0x560a6a2097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.623+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/3336369163 >> 172.21.15.172:0/3336369163 conn(0x560a6a2043f0 msgr2=0x560a6a2220f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.623+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/3336369163 shutdown_connections 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.623+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/3336369163 wait complete. 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.623+0000 7f89c4cd9f40 1 Processor -- start 2024-04-14T08:35:41.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.623+0000 7f89c4cd9f40 1 -- start start 2024-04-14T08:35:41.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c4cd9f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a1c8040 0x560a6a2ab660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c4cd9f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a6a1dc0d0 0x560a6a28ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c4cd9f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560a6a297f80 0x560a6a1dc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c4cd9f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560a6a2b9fe0 con 0x560a6a1c8040 2024-04-14T08:35:41.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c4cd9f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560a6a2965e0 con 0x560a6a1dc0d0 2024-04-14T08:35:41.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c4cd9f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x560a6a208fd0 con 0x560a6a297f80 2024-04-14T08:35:41.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c25a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a1c8040 0x560a6a2ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c25a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a1c8040 0x560a6a2ab660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38238/0 (socket says 172.21.15.172:38238) 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c25a3640 1 -- 172.21.15.172:0/221990475 learned_addr learned my addr 172.21.15.172:0/221990475 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c2da4640 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560a6a297f80 0x560a6a1dc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c1da2640 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a6a1dc0d0 0x560a6a28ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c25a3640 1 -- 172.21.15.172:0/221990475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560a6a297f80 msgr2=0x560a6a1dc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c25a3640 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560a6a297f80 0x560a6a1dc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c25a3640 1 -- 172.21.15.172:0/221990475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a6a1dc0d0 msgr2=0x560a6a28ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c25a3640 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a6a1dc0d0 0x560a6a28ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c25a3640 1 -- 172.21.15.172:0/221990475 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6a2afe40 con 0x560a6a1c8040 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c2da4640 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560a6a297f80 0x560a6a1dc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.624+0000 7f89c25a3640 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a1c8040 0x560a6a2ab660 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f89b4003f00 tx=0x7f89b4001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.625+0000 7f89b37fe640 1 -- 172.21.15.172:0/221990475 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b40123f0 con 0x560a6a1c8040 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.625+0000 7f89b37fe640 1 -- 172.21.15.172:0/221990475 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b4017070 con 0x560a6a1c8040 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.625+0000 7f89b37fe640 1 -- 172.21.15.172:0/221990475 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b40127f0 con 0x560a6a1c8040 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.625+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/221990475 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560a6a2b0120 con 0x560a6a1c8040 2024-04-14T08:35:41.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.625+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/221990475 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6a2b06e0 con 0x560a6a1c8040 2024-04-14T08:35:41.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.626+0000 7f89b37fe640 1 -- 172.21.15.172:0/221990475 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89b4012990 con 0x560a6a1c8040 2024-04-14T08:35:41.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.626+0000 7f89b37fe640 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f89a406b2d0 0x7f89a406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.626+0000 7f89b37fe640 1 -- 172.21.15.172:0/221990475 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f89b408a710 con 0x560a6a1c8040 2024-04-14T08:35:41.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.626+0000 7f89c1da2640 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f89a406b2d0 0x7f89a406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.627+0000 7f89c4cd9f40 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x560a6a2b0ee0 0x560a6a2b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.627+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/221990475 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61) -- 0x560a6a290500 con 0x560a6a2b0ee0 2024-04-14T08:35:41.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.627+0000 7f89c2da4640 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x560a6a2b0ee0 0x560a6a2b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.627+0000 7f89c1da2640 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f89a406b2d0 0x7f89a406d790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f89b8054000 tx=0x7f89b804e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.627+0000 7f89c2da4640 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x560a6a2b0ee0 0x560a6a2b1300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.628+0000 7f89c2da4640 1 -- 172.21.15.172:0/221990475 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v61'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f89bc00f960 con 0x560a6a2b0ee0 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.628+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/221990475 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x560a6a2b0ee0 msgr2=0x560a6a2b1300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.628+0000 7f89c4cd9f40 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x560a6a2b0ee0 0x560a6a2b1300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.628+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/221990475 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f89a406b2d0 msgr2=0x7f89a406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.628+0000 7f89c4cd9f40 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f89a406b2d0 0x7f89a406d790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f89b8054000 tx=0x7f89b804e000 comp rx=0 tx=0).stop 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.628+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/221990475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a1c8040 msgr2=0x560a6a2ab660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.628+0000 7f89c4cd9f40 1 --2- 172.21.15.172:0/221990475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a6a1c8040 0x560a6a2ab660 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f89b4003f00 tx=0x7f89b4001830 comp rx=0 tx=0).stop 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.628+0000 7f89c2da4640 1 -- 172.21.15.172:0/221990475 reap_dead start 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.628+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/221990475 shutdown_connections 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.628+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/221990475 >> 172.21.15.172:0/221990475 conn(0x560a6a2043f0 msgr2=0x560a6a290e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.628+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/221990475 shutdown_connections 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.628+0000 7f89c4cd9f40 1 -- 172.21.15.172:0/221990475 wait complete. 2024-04-14T08:35:41.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:41.633 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-14T08:35:41.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.777+0000 7fa8fd4ad640 1 Processor -- start 2024-04-14T08:35:41.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.777+0000 7fa8fd4ad640 1 -- start start 2024-04-14T08:35:41.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8fd4ad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f80b30a0 0x7fa8f80d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8fd4ad640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8f80d1bb0 con 0x7fa8f8161d00 2024-04-14T08:35:41.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8fd4ad640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8f80d0df0 con 0x7fa8f80d2b20 2024-04-14T08:35:41.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8fd4ad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f80d0ba0 con 0x7fa8f80b30a0 2024-04-14T08:35:41.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8f6ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa8f8161d00 0x7fa8f80b5490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55306/0 (socket says 172.21.15.172:55306) 2024-04-14T08:35:41.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8f6ffd640 1 -- 172.21.15.172:0/4091076055 learned_addr learned my addr 172.21.15.172:0/4091076055 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:41.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8f67fc640 1 --2- 172.21.15.172:0/4091076055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f80b30a0 0x7fa8f80d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8f5ffb640 1 -- 172.21.15.172:0/4091076055 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4190449468 0 0) 0x7fa8f80d1bb0 con 0x7fa8f8161d00 2024-04-14T08:35:41.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8f5ffb640 1 -- 172.21.15.172:0/4091076055 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8e0003520 con 0x7fa8f8161d00 2024-04-14T08:35:41.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8f67fc640 1 -- 172.21.15.172:0/4091076055 >> v1:172.21.15.177:6789/0 conn(0x7fa8f80d2b20 legacy=0x7fa8f80cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8f5ffb640 1 -- 172.21.15.172:0/4091076055 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3734095808 0 0) 0x7fa8f80d0df0 con 0x7fa8f80d2b20 2024-04-14T08:35:41.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8f67fc640 1 -- 172.21.15.172:0/4091076055 >> v1:172.21.15.172:6789/0 conn(0x7fa8f8161d00 legacy=0x7fa8f80b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.778+0000 7fa8f67fc640 1 -- 172.21.15.172:0/4091076055 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f80d10d0 con 0x7fa8f80b30a0 2024-04-14T08:35:41.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.779+0000 7fa8f67fc640 1 --2- 172.21.15.172:0/4091076055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f80b30a0 0x7fa8f80d25e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0097d0 tx=0x7fa8ec02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49fdb9ce396d720a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.779+0000 7fa8f5ffb640 1 -- 172.21.15.172:0/4091076055 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec035020 con 0x7fa8f80b30a0 2024-04-14T08:35:41.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.779+0000 7fa8f5ffb640 1 -- 172.21.15.172:0/4091076055 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8ec031800 con 0x7fa8f80b30a0 2024-04-14T08:35:41.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.779+0000 7fa8f5ffb640 1 -- 172.21.15.172:0/4091076055 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec031bb0 con 0x7fa8f80b30a0 2024-04-14T08:35:41.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.780+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/4091076055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f80b30a0 msgr2=0x7fa8f80d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.780+0000 7fa8fd4ad640 1 --2- 172.21.15.172:0/4091076055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f80b30a0 0x7fa8f80d25e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0097d0 tx=0x7fa8ec02b8a0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.780+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/4091076055 shutdown_connections 2024-04-14T08:35:41.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.780+0000 7fa8fd4ad640 1 --2- 172.21.15.172:0/4091076055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f80b30a0 0x7fa8f80d25e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.780+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/4091076055 >> 172.21.15.172:0/4091076055 conn(0x7fa8f809d570 msgr2=0x7fa8f80d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.780+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/4091076055 shutdown_connections 2024-04-14T08:35:41.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.780+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/4091076055 wait complete. 2024-04-14T08:35:41.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.781+0000 7fa8fd4ad640 1 Processor -- start 2024-04-14T08:35:41.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.781+0000 7fa8fd4ad640 1 -- start start 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8fd4ad640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f80b30a0 0x7fa8f80b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8fd4ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f80d2b20 0x7fa8f80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8fd4ad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f8161d00 0x7fa8f80b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8fd4ad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f80d33b0 con 0x7fa8f80d2b20 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8fd4ad640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f80d1720 con 0x7fa8f80b30a0 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8fd4ad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f80cf610 con 0x7fa8f8161d00 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8f67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f80d2b20 0x7fa8f80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8f67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f80d2b20 0x7fa8f80cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38244/0 (socket says 172.21.15.172:38244) 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8f67fc640 1 -- 172.21.15.172:0/3474237483 learned_addr learned my addr 172.21.15.172:0/3474237483 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f80b30a0 0x7fa8f80b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8f77fe640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f8161d00 0x7fa8f80b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8f67fc640 1 -- 172.21.15.172:0/3474237483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f8161d00 msgr2=0x7fa8f80b86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8f67fc640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f8161d00 0x7fa8f80b86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8f67fc640 1 -- 172.21.15.172:0/3474237483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f80b30a0 msgr2=0x7fa8f80b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8f67fc640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f80b30a0 0x7fa8f80b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8f67fc640 1 -- 172.21.15.172:0/3474237483 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f81f4e00 con 0x7fa8f80d2b20 2024-04-14T08:35:41.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8f67fc640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f80d2b20 0x7fa8f80cd700 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec000c00 tx=0x7fa8ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8df7fe640 1 -- 172.21.15.172:0/3474237483 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec041070 con 0x7fa8f80d2b20 2024-04-14T08:35:41.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.782+0000 7fa8df7fe640 1 -- 172.21.15.172:0/3474237483 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8ec03ad00 con 0x7fa8f80d2b20 2024-04-14T08:35:41.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.783+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/3474237483 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f80b7000 con 0x7fa8f80d2b20 2024-04-14T08:35:41.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.783+0000 7fa8df7fe640 1 -- 172.21.15.172:0/3474237483 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec035040 con 0x7fa8f80d2b20 2024-04-14T08:35:41.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.783+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/3474237483 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f80b75c0 con 0x7fa8f80d2b20 2024-04-14T08:35:41.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.784+0000 7fa8df7fe640 1 -- 172.21.15.172:0/3474237483 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8ec04c060 con 0x7fa8f80d2b20 2024-04-14T08:35:41.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.784+0000 7fa8df7fe640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8e406c8f0 0x7fa8e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.785+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8e406c8f0 0x7fa8e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.785+0000 7fa8df7fe640 1 -- 172.21.15.172:0/3474237483 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fa8ec0ad6c0 con 0x7fa8f80d2b20 2024-04-14T08:35:41.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.785+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8e406c8f0 0x7fa8e406edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8005c00 tx=0x7fa8e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.785+0000 7fa90036f740 1 Processor -- start 2024-04-14T08:35:41.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.786+0000 7fa90036f740 1 -- start start 2024-04-14T08:35:41.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.786+0000 7fa90036f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d7ff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.786+0000 7fa90036f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c7d0 0x55e61d82f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.786+0000 7fa90036f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d7ff8d0 0x55e61d82f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.786+0000 7fa90036f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e61d768400 con 0x55e61d7ff8d0 2024-04-14T08:35:41.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.786+0000 7fa90036f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e61d706a10 con 0x55e61d82c7d0 2024-04-14T08:35:41.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.786+0000 7fa90036f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e61d7643b0 con 0x55e61d82c3f0 2024-04-14T08:35:41.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.786+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d7ff8d0 0x55e61d82f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.786+0000 7fa8f6ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d7ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8f67fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c7d0 0x55e61d82f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8f67fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c7d0 0x55e61d82f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:51372/0 (socket says 172.21.15.172:51372) 2024-04-14T08:35:41.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8f6ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d7ff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:57380/0 (socket says 172.21.15.172:57380) 2024-04-14T08:35:41.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8f67fc640 1 -- 172.21.15.172:0/1879117879 learned_addr learned my addr 172.21.15.172:0/1879117879 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d7ff8d0 0x55e61d82f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38250/0 (socket says 172.21.15.172:38250) 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8f6ffd640 1 -- 172.21.15.172:0/1879117879 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c7d0 msgr2=0x55e61d82f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/1879117879 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c7d0 0x55e61d82f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8f6ffd640 1 -- 172.21.15.172:0/1879117879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d7ff8d0 msgr2=0x55e61d82f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/1879117879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d7ff8d0 0x55e61d82f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8f6ffd640 1 -- 172.21.15.172:0/1879117879 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61d730930 con 0x55e61d82c3f0 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8f67fc640 1 --2- 172.21.15.172:0/1879117879 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c7d0 0x55e61d82f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/1879117879 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d7ff390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80155c0 tx=0x7fa8e8001250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2900ae11b345f2ea server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8dcff9640 1 -- 172.21.15.172:0/1879117879 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8025070 con 0x55e61d82c3f0 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8dcff9640 1 -- 172.21.15.172:0/1879117879 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e8013680 con 0x55e61d82c3f0 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.787+0000 7fa8dcff9640 1 -- 172.21.15.172:0/1879117879 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8013a50 con 0x55e61d82c3f0 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.788+0000 7fa90036f740 1 -- 172.21.15.172:0/1879117879 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 msgr2=0x55e61d7ff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.788+0000 7fa90036f740 1 --2- 172.21.15.172:0/1879117879 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d7ff390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80155c0 tx=0x7fa8e8001250 comp rx=0 tx=0).stop 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.788+0000 7fa90036f740 1 -- 172.21.15.172:0/1879117879 shutdown_connections 2024-04-14T08:35:41.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.788+0000 7fa90036f740 1 --2- 172.21.15.172:0/1879117879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d7ff8d0 0x55e61d82f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.788+0000 7fa90036f740 1 --2- 172.21.15.172:0/1879117879 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c7d0 0x55e61d82f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.788+0000 7fa90036f740 1 --2- 172.21.15.172:0/1879117879 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d7ff390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.788+0000 7fa90036f740 1 -- 172.21.15.172:0/1879117879 >> 172.21.15.172:0/1879117879 conn(0x55e61d801600 msgr2=0x55e61d84a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.788+0000 7fa90036f740 1 -- 172.21.15.172:0/1879117879 shutdown_connections 2024-04-14T08:35:41.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.788+0000 7fa90036f740 1 -- 172.21.15.172:0/1879117879 wait complete. 2024-04-14T08:35:41.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.788+0000 7fa90036f740 1 -- start start 2024-04-14T08:35:41.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa90036f740 1 Processor -- start 2024-04-14T08:35:41.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa90036f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d824bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa90036f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d82c7d0 0x55e61d827700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa90036f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d85b1c0 0x55e61d827c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa90036f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e61d803ca0 con 0x55e61d82c7d0 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa90036f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e61d7046e0 con 0x55e61d85b1c0 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa90036f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e61d8064e0 con 0x55e61d82c3f0 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa90036f740 10 client.0 _ll_register_callbacks cb 0x55e61d47bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa8f67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d82c7d0 0x55e61d827700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa8f67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d82c7d0 0x55e61d827700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38256/0 (socket says 172.21.15.172:38256) 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa8f67fc640 1 -- 172.21.15.172:0/4065254883 learned_addr learned my addr 172.21.15.172:0/4065254883 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa8f77fe640 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d85b1c0 0x55e61d827c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.789+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d824bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa8f67fc640 1 -- 172.21.15.172:0/4065254883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 msgr2=0x55e61d824bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa8f67fc640 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d824bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa8f67fc640 1 -- 172.21.15.172:0/4065254883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d85b1c0 msgr2=0x55e61d827c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa8f67fc640 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d85b1c0 0x55e61d827c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa8f67fc640 1 -- 172.21.15.172:0/4065254883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61d5487b0 con 0x55e61d82c7d0 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa8f67fc640 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d82c7d0 0x55e61d827700 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec02b8e0 tx=0x7fa8ec031800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c641d4977ed45625 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa8d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa8d5ffb640 1 -- 172.21.15.172:0/4065254883 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec07f350 con 0x55e61d82c7d0 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa8d5ffb640 1 -- 172.21.15.172:0/4065254883 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fa8ec07a750 con 0x55e61d82c7d0 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa90036f740 1 -- 172.21.15.172:0/4065254883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e8013680 con 0x55e61d82c7d0 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa8d5ffb640 1 -- 172.21.15.172:0/4065254883 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8ec083e30 con 0x55e61d82c7d0 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa90036f740 10 client.6396 fetch_fsmap learned FSMap version 30 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa90036f740 1 -- 172.21.15.172:0/4065254883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x55e61d5a8f40 con 0x55e61d82c7d0 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa8d5ffb640 1 -- 172.21.15.172:0/4065254883 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e8013680 con 0x55e61d82c7d0 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa8d5ffb640 1 -- 172.21.15.172:0/4065254883 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 3494+0+0 (secure 0 0 0) 0x7fa8ec077050 con 0x55e61d82c7d0 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa90036f740 10 client.6396 fetch_fsmap finished waiting for FSMap version 30 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa90036f740 10 client.6396 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa90036f740 20 client.6396 populate_metadata read hostname 'smithi172' 2024-04-14T08:35:41.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa90036f740 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55e61d85e200 0x55e61d85e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa90036f740 4 client.6396 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.790+0000 7fa90036f740 1 -- 172.21.15.172:0/4065254883 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e61d775940 con 0x55e61d85e200 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.791+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55e61d85e200 0x55e61d85e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.791+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55e61d85e200 0x55e61d85e5e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802d17a12022868f server_cookie=c6e336f742d1a4e5 in_seq=0 out_seq=0 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.791+0000 7fa8d5ffb640 10 client.6396 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.791+0000 7fa8d5ffb640 1 -- 172.21.15.172:0/4065254883 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e61d775940 con 0x55e61d85e200 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.791+0000 7fa8d5ffb640 10 client.6396 handle_command_reply: tid=0 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.791+0000 7fa90036f740 1 client.6396 shutdown 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.792+0000 7fa90036f740 1 -- 172.21.15.172:0/4065254883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d82c7d0 msgr2=0x55e61d827700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.792+0000 7fa90036f740 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d82c7d0 0x55e61d827700 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec02b8e0 tx=0x7fa8ec031800 comp rx=0 tx=0).stop 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.792+0000 7fa90036f740 1 -- 172.21.15.172:0/4065254883 shutdown_connections 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.792+0000 7fa90036f740 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55e61d85e200 0x55e61d85e5e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.792+0000 7fa90036f740 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d85b1c0 0x55e61d827c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.792+0000 7fa8d5ffb640 0 client.6396 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.792+0000 7fa90036f740 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d82c7d0 0x55e61d827700 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.792+0000 7fa90036f740 1 --2- 172.21.15.172:0/4065254883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d824bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.792+0000 7fa90036f740 1 -- 172.21.15.172:0/4065254883 >> 172.21.15.172:0/4065254883 conn(0x55e61d7ff8d0 msgr2=0x55e61d804290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.792+0000 7fa90036f740 1 -- 172.21.15.172:0/4065254883 shutdown_connections 2024-04-14T08:35:41.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.792+0000 7fa90036f740 1 -- 172.21.15.172:0/4065254883 wait complete. 2024-04-14T08:35:41.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.792+0000 7fa90036f740 20 client.6396 trim_cache size 0 max 16384 2024-04-14T08:35:41.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.809+0000 7fa90036f740 1 Processor -- start 2024-04-14T08:35:41.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.809+0000 7fa90036f740 1 -- start start 2024-04-14T08:35:41.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa90036f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d84d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa90036f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c7d0 0x55e61d82f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa90036f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 0x55e61d859d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa90036f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f8200c80 con 0x55e61d85b1c0 2024-04-14T08:35:41.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa90036f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f81ffcd0 con 0x55e61d82c7d0 2024-04-14T08:35:41.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa90036f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f81f8440 con 0x55e61d82c3f0 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 0x55e61d859d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 0x55e61d859d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38272/0 (socket says 172.21.15.172:38272) 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa8f77fe640 1 -- 172.21.15.172:0/2925921919 learned_addr learned my addr 172.21.15.172:0/2925921919 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/2925921919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d84d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa8f67fc640 1 --2- 172.21.15.172:0/2925921919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c7d0 0x55e61d82f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa8f77fe640 1 -- 172.21.15.172:0/2925921919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 msgr2=0x55e61d84d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa8f77fe640 1 --2- 172.21.15.172:0/2925921919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d84d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa8f77fe640 1 -- 172.21.15.172:0/2925921919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c7d0 msgr2=0x55e61d82f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa8f77fe640 1 --2- 172.21.15.172:0/2925921919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c7d0 0x55e61d82f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.810+0000 7fa8f77fe640 1 -- 172.21.15.172:0/2925921919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61d43c0a0 con 0x55e61d85b1c0 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa8f77fe640 1 --2- 172.21.15.172:0/2925921919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 0x55e61d859d00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0001850 tx=0x7fa8f0001d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a879532ac8aa705 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa8d7fff640 1 -- 172.21.15.172:0/2925921919 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0005320 con 0x55e61d85b1c0 2024-04-14T08:35:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa8d7fff640 1 -- 172.21.15.172:0/2925921919 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f0015950 con 0x55e61d85b1c0 2024-04-14T08:35:41.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa8d7fff640 1 -- 172.21.15.172:0/2925921919 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0015d00 con 0x55e61d85b1c0 2024-04-14T08:35:41.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa90036f740 1 -- 172.21.15.172:0/2925921919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 msgr2=0x55e61d859d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa90036f740 1 --2- 172.21.15.172:0/2925921919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 0x55e61d859d00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0001850 tx=0x7fa8f0001d10 comp rx=0 tx=0).stop 2024-04-14T08:35:41.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa90036f740 1 -- 172.21.15.172:0/2925921919 shutdown_connections 2024-04-14T08:35:41.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa90036f740 1 --2- 172.21.15.172:0/2925921919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 0x55e61d859d00 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa90036f740 1 --2- 172.21.15.172:0/2925921919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c7d0 0x55e61d82f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa90036f740 1 --2- 172.21.15.172:0/2925921919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c3f0 0x55e61d84d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa90036f740 1 -- 172.21.15.172:0/2925921919 >> 172.21.15.172:0/2925921919 conn(0x55e61d7ff8d0 msgr2=0x55e61d826940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa90036f740 1 -- 172.21.15.172:0/2925921919 shutdown_connections 2024-04-14T08:35:41.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.811+0000 7fa90036f740 1 -- 172.21.15.172:0/2925921919 wait complete. 2024-04-14T08:35:41.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.812+0000 7fa90036f740 1 -- start start 2024-04-14T08:35:41.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.812+0000 7fa90036f740 1 Processor -- start 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa90036f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c3f0 0x55e61d8be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa90036f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c7d0 0x55e61d833090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa90036f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 0x55e61d823910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa90036f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f80ccf30 con 0x55e61d85b1c0 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa90036f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e61d800df0 con 0x55e61d82c3f0 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa90036f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e61d82f3e0 con 0x55e61d82c7d0 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa90036f740 10 client.0 _ll_register_callbacks cb 0x7fa8ec07acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 0x55e61d823910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 0x55e61d823910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38282/0 (socket says 172.21.15.172:38282) 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8f6ffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c3f0 0x55e61d8be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8f67fc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c7d0 0x55e61d833090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8f77fe640 1 -- 172.21.15.172:0/313139642 learned_addr learned my addr 172.21.15.172:0/313139642 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8f6ffd640 1 -- 172.21.15.172:0/313139642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c7d0 msgr2=0x55e61d833090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/313139642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c7d0 0x55e61d833090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8f6ffd640 1 -- 172.21.15.172:0/313139642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 msgr2=0x55e61d823910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/313139642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 0x55e61d823910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8f6ffd640 1 -- 172.21.15.172:0/313139642 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61d82f1c0 con 0x55e61d82c3f0 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8f77fe640 1 --2- 172.21.15.172:0/313139642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 0x55e61d823910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8f6ffd640 1 --2- 172.21.15.172:0/313139642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c3f0 0x55e61d8be8f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8022000 tx=0x7fa8e8001420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=569a9944a51fbabb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8d5ffb640 1 -- 172.21.15.172:0/313139642 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8025040 con 0x55e61d82c3f0 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa8d5ffb640 1 -- 172.21.15.172:0/313139642 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fa8e8001ec0 con 0x55e61d82c3f0 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.813+0000 7fa90036f740 1 -- 172.21.15.172:0/313139642 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f0015950 con 0x55e61d82c3f0 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa8d5ffb640 1 -- 172.21.15.172:0/313139642 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8e803e050 con 0x55e61d82c3f0 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa90036f740 10 client.6157 fetch_fsmap learned FSMap version 30 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa90036f740 1 -- 172.21.15.172:0/313139642 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x55e61d82e7c0 con 0x55e61d82c3f0 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa8d5ffb640 1 -- 172.21.15.172:0/313139642 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f0015950 con 0x55e61d82c3f0 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa8d5ffb640 1 -- 172.21.15.172:0/313139642 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 30) ==== 3494+0+0 (secure 0 0 0) 0x7fa8e8013ad0 con 0x55e61d82c3f0 2024-04-14T08:35:41.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa90036f740 10 client.6157 fetch_fsmap finished waiting for FSMap version 30 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa90036f740 10 client.6157 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa90036f740 20 client.6157 populate_metadata read hostname 'smithi172' 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa90036f740 1 --2- 172.21.15.172:0/313139642 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55e61d823e50 0x55e61d8c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa90036f740 4 client.6157 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa90036f740 1 -- 172.21.15.172:0/313139642 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55e61d76fc90 con 0x55e61d823e50 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa8f67fc640 1 --2- 172.21.15.172:0/313139642 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55e61d823e50 0x55e61d8c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa8d5ffb640 10 client.6157 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.814+0000 7fa8f67fc640 1 --2- 172.21.15.172:0/313139642 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55e61d823e50 0x55e61d8c4780 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=637dc57a1e6a3190 server_cookie=23f6ba4426a34bd8 in_seq=0 out_seq=0 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa8d5ffb640 1 -- 172.21.15.172:0/313139642 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e61d76fc90 con 0x55e61d823e50 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa8d5ffb640 10 client.6157 handle_command_reply: tid=0 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa90036f740 1 client.6157 shutdown 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa90036f740 1 -- 172.21.15.172:0/313139642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c3f0 msgr2=0x55e61d8be8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa90036f740 1 --2- 172.21.15.172:0/313139642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c3f0 0x55e61d8be8f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8022000 tx=0x7fa8e8001420 comp rx=0 tx=0).stop 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa90036f740 1 -- 172.21.15.172:0/313139642 shutdown_connections 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa90036f740 1 --2- 172.21.15.172:0/313139642 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55e61d823e50 0x55e61d8c4780 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa90036f740 1 --2- 172.21.15.172:0/313139642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e61d85b1c0 0x55e61d823910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa90036f740 1 --2- 172.21.15.172:0/313139642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e61d82c7d0 0x55e61d833090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa8d5ffb640 0 client.6157 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:35:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa90036f740 1 --2- 172.21.15.172:0/313139642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e61d82c3f0 0x55e61d8be8f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa90036f740 1 -- 172.21.15.172:0/313139642 >> 172.21.15.172:0/313139642 conn(0x55e61d7ff8d0 msgr2=0x55e61d849b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.815+0000 7fa90036f740 1 -- 172.21.15.172:0/313139642 shutdown_connections 2024-04-14T08:35:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa90036f740 1 -- 172.21.15.172:0/313139642 wait complete. 2024-04-14T08:35:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa90036f740 20 client.6157 trim_cache size 0 max 16384 2024-04-14T08:35:41.821 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:35:41.821 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-04-14T08:35:41.821 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "88346069-7461-4514-9361-a012d7be5193", 2024-04-14T08:35:41.821 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-04-14T08:35:41.821 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:35:41.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/3474237483 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8e406c8f0 msgr2=0x7fa8e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa8fd4ad640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8e406c8f0 0x7fa8e406edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8005c00 tx=0x7fa8e8009040 comp rx=0 tx=0).stop 2024-04-14T08:35:41.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/3474237483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f80d2b20 msgr2=0x7fa8f80cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa8fd4ad640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f80d2b20 0x7fa8f80cd700 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec000c00 tx=0x7fa8ec0046c0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/3474237483 shutdown_connections 2024-04-14T08:35:41.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa8fd4ad640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8e406c8f0 0x7fa8e406edb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa8fd4ad640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f8161d00 0x7fa8f80b86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa8fd4ad640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f80d2b20 0x7fa8f80cd700 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa8fd4ad640 1 --2- 172.21.15.172:0/3474237483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f80b30a0 0x7fa8f80b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/3474237483 >> 172.21.15.172:0/3474237483 conn(0x7fa8f809d570 msgr2=0x7fa8f8074a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/3474237483 shutdown_connections 2024-04-14T08:35:41.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.816+0000 7fa8fd4ad640 1 -- 172.21.15.172:0/3474237483 wait complete. 2024-04-14T08:35:41.823 INFO:tasks.ceph.mds.d.smithi177.stderr:2024-04-14T08:35:41.816+0000 7f9d62fe9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6115=0-4194304@1} caps={6115=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6115 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x555d1cc8a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-14T08:35:41.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:35:41.828 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-14T08:35:41.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.930+0000 7f43a611e640 1 Processor -- start 2024-04-14T08:35:41.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.930+0000 7f43a611e640 1 -- start start 2024-04-14T08:35:41.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f43a611e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a00729f0 0x7f43a0165490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f43a611e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a00ce860 con 0x7f43a00af740 2024-04-14T08:35:41.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f43a611e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a0071520 con 0x7f43a013fa70 2024-04-14T08:35:41.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f43a611e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f43a00712d0 con 0x7f43a00729f0 2024-04-14T08:35:41.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f439effd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f43a00af740 0x7f43a00722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55310/0 (socket says 172.21.15.172:55310) 2024-04-14T08:35:41.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f439effd640 1 -- 172.21.15.172:0/2400488860 learned_addr learned my addr 172.21.15.172:0/2400488860 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:35:41.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f439ffff640 1 --2- 172.21.15.172:0/2400488860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a00729f0 0x7f43a0165490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f439e7fc640 1 -- 172.21.15.172:0/2400488860 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588952123 0 0) 0x7f4394001110 con 0x7f43a00af740 2024-04-14T08:35:41.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f439e7fc640 1 -- 172.21.15.172:0/2400488860 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4388003520 con 0x7f43a00af740 2024-04-14T08:35:41.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f439e7fc640 1 -- 172.21.15.172:0/2400488860 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 718947516 0 0) 0x7f43a0071520 con 0x7f43a013fa70 2024-04-14T08:35:41.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f439ffff640 1 -- 172.21.15.172:0/2400488860 >> v1:172.21.15.177:6789/0 conn(0x7f43a013fa70 legacy=0x7f43a00ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f439ffff640 1 -- 172.21.15.172:0/2400488860 >> v1:172.21.15.172:6789/0 conn(0x7f43a00af740 legacy=0x7f43a00722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.931+0000 7f439ffff640 1 -- 172.21.15.172:0/2400488860 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a0071800 con 0x7f43a00729f0 2024-04-14T08:35:41.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.932+0000 7f439ffff640 1 --2- 172.21.15.172:0/2400488860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a00729f0 0x7f43a0165490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f439800aed0 tx=0x7f439802cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f6aff32159e406 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.932+0000 7f439e7fc640 1 -- 172.21.15.172:0/2400488860 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439802db30 con 0x7f43a00729f0 2024-04-14T08:35:41.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.932+0000 7f439e7fc640 1 -- 172.21.15.172:0/2400488860 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439802dcd0 con 0x7f43a00729f0 2024-04-14T08:35:41.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.932+0000 7f439e7fc640 1 -- 172.21.15.172:0/2400488860 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398008640 con 0x7f43a00729f0 2024-04-14T08:35:41.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.933+0000 7f43a611e640 1 -- 172.21.15.172:0/2400488860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a00729f0 msgr2=0x7f43a0165490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.933+0000 7f43a611e640 1 --2- 172.21.15.172:0/2400488860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a00729f0 0x7f43a0165490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f439800aed0 tx=0x7f439802cf90 comp rx=0 tx=0).stop 2024-04-14T08:35:41.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.933+0000 7f43a611e640 1 -- 172.21.15.172:0/2400488860 shutdown_connections 2024-04-14T08:35:41.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.933+0000 7f43a611e640 1 --2- 172.21.15.172:0/2400488860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a00729f0 0x7f43a0165490 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.933+0000 7f43a611e640 1 -- 172.21.15.172:0/2400488860 >> 172.21.15.172:0/2400488860 conn(0x7f43a006b520 msgr2=0x7f43a006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.933+0000 7f43a611e640 1 -- 172.21.15.172:0/2400488860 shutdown_connections 2024-04-14T08:35:41.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.933+0000 7f43a611e640 1 -- 172.21.15.172:0/2400488860 wait complete. 2024-04-14T08:35:41.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.934+0000 7f43a611e640 1 Processor -- start 2024-04-14T08:35:41.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.934+0000 7f43a611e640 1 -- start start 2024-04-14T08:35:41.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f43a611e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43a00729f0 0x7f43a0158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f43a611e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43a00af740 0x7f43a013d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f43a611e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a013fa70 0x7f43a013d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f43a611e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0144130 con 0x7f43a00af740 2024-04-14T08:35:41.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f43a611e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0071dc0 con 0x7f43a00729f0 2024-04-14T08:35:41.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f43a611e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f43a01442b0 con 0x7f43a013fa70 2024-04-14T08:35:41.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f439effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43a00af740 0x7f43a013d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f439ffff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a013fa70 0x7f43a013d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f439effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43a00af740 0x7f43a013d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38288/0 (socket says 172.21.15.172:38288) 2024-04-14T08:35:41.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f439effd640 1 -- 172.21.15.172:0/2521638588 learned_addr learned my addr 172.21.15.172:0/2521638588 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f439f7fe640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43a00729f0 0x7f43a0158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f439ffff640 1 -- 172.21.15.172:0/2521638588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43a00729f0 msgr2=0x7f43a0158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f439ffff640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43a00729f0 0x7f43a0158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f439ffff640 1 -- 172.21.15.172:0/2521638588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43a00af740 msgr2=0x7f43a013d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f439ffff640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43a00af740 0x7f43a013d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f439ffff640 1 -- 172.21.15.172:0/2521638588 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a00b3960 con 0x7f43a013fa70 2024-04-14T08:35:41.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.935+0000 7f439ffff640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a013fa70 0x7f43a013d940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4398033000 tx=0x7f4398004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.936+0000 7f4387fff640 1 -- 172.21.15.172:0/2521638588 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398044020 con 0x7f43a013fa70 2024-04-14T08:35:41.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.936+0000 7f4387fff640 1 -- 172.21.15.172:0/2521638588 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4398040690 con 0x7f43a013fa70 2024-04-14T08:35:41.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.936+0000 7f4387fff640 1 -- 172.21.15.172:0/2521638588 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439800a560 con 0x7f43a013fa70 2024-04-14T08:35:41.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.936+0000 7f43a611e640 1 -- 172.21.15.172:0/2521638588 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a01433a0 con 0x7f43a013fa70 2024-04-14T08:35:41.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.936+0000 7f43a611e640 1 -- 172.21.15.172:0/2521638588 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a01594b0 con 0x7f43a013fa70 2024-04-14T08:35:41.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.937+0000 7f4387fff640 1 -- 172.21.15.172:0/2521638588 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4398047020 con 0x7f43a013fa70 2024-04-14T08:35:41.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.937+0000 7f4387fff640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f438c06c8f0 0x7f438c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.938+0000 7f4387fff640 1 -- 172.21.15.172:0/2521638588 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f43980ae860 con 0x7f43a013fa70 2024-04-14T08:35:41.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.938+0000 7f439f7fe640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f438c06c8f0 0x7f438c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.938+0000 7f439f7fe640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f438c06c8f0 0x7f438c06edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4390000f60 tx=0x7f4390009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.938+0000 7f43a9002740 1 Processor -- start 2024-04-14T08:35:41.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.938+0000 7f43a9002740 1 -- start start 2024-04-14T08:35:41.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.939+0000 7f43a9002740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 0x563094009910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.939+0000 7f43a9002740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 0x563094039700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.939+0000 7f43a9002740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094009e50 0x563094039c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.939+0000 7f43a9002740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563093f73800 con 0x563094036970 2024-04-14T08:35:41.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.939+0000 7f43a9002740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563093f1b7f0 con 0x563094009e50 2024-04-14T08:35:41.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.939+0000 7f43a9002740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563093f6f7b0 con 0x563094036d50 2024-04-14T08:35:41.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.939+0000 7f439f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 0x563094009910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.939+0000 7f439f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 0x563094009910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38302/0 (socket says 172.21.15.172:38302) 2024-04-14T08:35:41.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.939+0000 7f439f7fe640 1 -- 172.21.15.172:0/869169503 learned_addr learned my addr 172.21.15.172:0/869169503 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.939+0000 7f439effd640 1 --2- 172.21.15.172:0/869169503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 0x563094039700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.939+0000 7f439ffff640 1 --2- 172.21.15.172:0/869169503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094009e50 0x563094039c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.940+0000 7f439f7fe640 1 -- 172.21.15.172:0/869169503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 msgr2=0x563094039700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.940+0000 7f439f7fe640 1 --2- 172.21.15.172:0/869169503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 0x563094039700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.940+0000 7f439f7fe640 1 -- 172.21.15.172:0/869169503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094009e50 msgr2=0x563094039c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.940+0000 7f439f7fe640 1 --2- 172.21.15.172:0/869169503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094009e50 0x563094039c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.940+0000 7f439f7fe640 1 -- 172.21.15.172:0/869169503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563093d517d0 con 0x563094036970 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.940+0000 7f439f7fe640 1 --2- 172.21.15.172:0/869169503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 0x563094009910 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f439000fbd0 tx=0x7f4390012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc247b324dc122a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.940+0000 7f43857fa640 1 -- 172.21.15.172:0/869169503 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390025070 con 0x563094036970 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.940+0000 7f43857fa640 1 -- 172.21.15.172:0/869169503 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439001b050 con 0x563094036970 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.940+0000 7f43857fa640 1 -- 172.21.15.172:0/869169503 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390014de0 con 0x563094036970 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.941+0000 7f43a9002740 1 -- 172.21.15.172:0/869169503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 msgr2=0x563094009910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.941+0000 7f43a9002740 1 --2- 172.21.15.172:0/869169503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 0x563094009910 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f439000fbd0 tx=0x7f4390012000 comp rx=0 tx=0).stop 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.941+0000 7f43a9002740 1 -- 172.21.15.172:0/869169503 shutdown_connections 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.941+0000 7f43a9002740 1 --2- 172.21.15.172:0/869169503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094009e50 0x563094039c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.941+0000 7f43a9002740 1 --2- 172.21.15.172:0/869169503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 0x563094039700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.941+0000 7f43a9002740 1 --2- 172.21.15.172:0/869169503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 0x563094009910 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.941+0000 7f43a9002740 1 -- 172.21.15.172:0/869169503 >> 172.21.15.172:0/869169503 conn(0x56309400bb80 msgr2=0x563094054700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.941+0000 7f43a9002740 1 -- 172.21.15.172:0/869169503 shutdown_connections 2024-04-14T08:35:41.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.941+0000 7f43a9002740 1 -- 172.21.15.172:0/869169503 wait complete. 2024-04-14T08:35:41.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.941+0000 7f43a9002740 1 -- start start 2024-04-14T08:35:41.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.941+0000 7f43a9002740 1 Processor -- start 2024-04-14T08:35:41.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f43a9002740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094036970 0x563094035710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f43a9002740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 0x563094030170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f43a9002740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094023f70 0x56309403c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f43a9002740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56309400e120 con 0x563094023f70 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f43a9002740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563093f2ed00 con 0x563094036970 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f43a9002740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56309400d510 con 0x563094036d50 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f43a9002740 10 client.0 _ll_register_callbacks cb 0x563093f2e6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f439ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094023f70 0x56309403c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f439f7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094036970 0x563094035710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f439ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094023f70 0x56309403c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38304/0 (socket says 172.21.15.172:38304) 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f439effd640 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 0x563094030170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f439ffff640 1 -- 172.21.15.172:0/3497275378 learned_addr learned my addr 172.21.15.172:0/3497275378 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f439ffff640 1 -- 172.21.15.172:0/3497275378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 msgr2=0x563094030170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f439ffff640 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 0x563094030170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f439ffff640 1 -- 172.21.15.172:0/3497275378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094036970 msgr2=0x563094035710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f439ffff640 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094036970 0x563094035710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f439ffff640 1 -- 172.21.15.172:0/3497275378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563093c873f0 con 0x563094023f70 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.942+0000 7f439f7fe640 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094036970 0x563094035710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:35:41.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f439ffff640 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094023f70 0x56309403c980 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f439802d5a0 tx=0x7f439807bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5514910421986f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f437e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f437e7fc640 1 -- 172.21.15.172:0/3497275378 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398084ff0 con 0x563094023f70 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f437e7fc640 1 -- 172.21.15.172:0/3497275378 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f439807c720 con 0x563094023f70 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f43a9002740 1 -- 172.21.15.172:0/3497275378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f439001b050 con 0x563094023f70 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f437e7fc640 1 -- 172.21.15.172:0/3497275378 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f439807c920 con 0x563094023f70 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f43a9002740 10 client.6414 fetch_fsmap learned FSMap version 30 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f43a9002740 1 -- 172.21.15.172:0/3497275378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x563093c470f0 con 0x563094023f70 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f437e7fc640 1 -- 172.21.15.172:0/3497275378 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439001b050 con 0x563094023f70 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f437e7fc640 1 -- 172.21.15.172:0/3497275378 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 3494+0+0 (secure 0 0 0) 0x7f4398084b60 con 0x563094023f70 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f43a9002740 10 client.6414 fetch_fsmap finished waiting for FSMap version 30 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f43a9002740 10 client.6414 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f43a9002740 20 client.6414 populate_metadata read hostname 'smithi172' 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f43a9002740 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563094032e10 0x5630940647b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f43a9002740 4 client.6414 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:35:41.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f43a9002740 1 -- 172.21.15.172:0/3497275378 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563093f80d40 con 0x563094032e10 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.943+0000 7f439f7fe640 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563094032e10 0x5630940647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f439f7fe640 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563094032e10 0x5630940647b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492faabf8c28ab86 server_cookie=23c9e77ea9d34e13 in_seq=0 out_seq=0 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f437e7fc640 10 client.6414 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f437e7fc640 1 -- 172.21.15.172:0/3497275378 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563093f80d40 con 0x563094032e10 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f437e7fc640 10 client.6414 handle_command_reply: tid=0 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f43a9002740 1 client.6414 shutdown 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f43a9002740 1 -- 172.21.15.172:0/3497275378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094023f70 msgr2=0x56309403c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f43a9002740 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094023f70 0x56309403c980 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f439802d5a0 tx=0x7f439807bee0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f43a9002740 1 -- 172.21.15.172:0/3497275378 shutdown_connections 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f43a9002740 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563094032e10 0x5630940647b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f43a9002740 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094023f70 0x56309403c980 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f43a9002740 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 0x563094030170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f43a9002740 1 --2- 172.21.15.172:0/3497275378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094036970 0x563094035710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f437e7fc640 0 client.6414 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:35:41.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.944+0000 7f43a9002740 1 -- 172.21.15.172:0/3497275378 >> 172.21.15.172:0/3497275378 conn(0x563094009e50 msgr2=0x563094057c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.945+0000 7f43a9002740 1 -- 172.21.15.172:0/3497275378 shutdown_connections 2024-04-14T08:35:41.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.945+0000 7f43a9002740 1 -- 172.21.15.172:0/3497275378 wait complete. 2024-04-14T08:35:41.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.945+0000 7f43a9002740 20 client.6414 trim_cache size 0 max 16384 2024-04-14T08:35:41.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.962+0000 7f43a9002740 1 Processor -- start 2024-04-14T08:35:41.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.962+0000 7f43a9002740 1 -- start start 2024-04-14T08:35:41.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f43a9002740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094023f70 0x56309402ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f43a9002740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036970 0x563094032150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f43a9002740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036d50 0x5630940332e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f43a9002740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0200f70 con 0x563094036d50 2024-04-14T08:35:41.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f43a9002740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f43a01fffc0 con 0x563094023f70 2024-04-14T08:35:41.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f43a9002740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f43a01ff970 con 0x563094036970 2024-04-14T08:35:41.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f439ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036d50 0x5630940332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f439ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036d50 0x5630940332e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38320/0 (socket says 172.21.15.172:38320) 2024-04-14T08:35:41.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f439ffff640 1 -- 172.21.15.172:0/3813985357 learned_addr learned my addr 172.21.15.172:0/3813985357 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f439f7fe640 1 --2- 172.21.15.172:0/3813985357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094023f70 0x56309402ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f439effd640 1 --2- 172.21.15.172:0/3813985357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036970 0x563094032150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f439ffff640 1 -- 172.21.15.172:0/3813985357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036970 msgr2=0x563094032150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f439ffff640 1 --2- 172.21.15.172:0/3813985357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036970 0x563094032150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f439ffff640 1 -- 172.21.15.172:0/3813985357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094023f70 msgr2=0x56309402ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f439ffff640 1 --2- 172.21.15.172:0/3813985357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094023f70 0x56309402ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f439ffff640 1 -- 172.21.15.172:0/3813985357 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56309400cee0 con 0x563094036d50 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f439ffff640 1 --2- 172.21.15.172:0/3813985357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036d50 0x5630940332e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f439802cb70 tx=0x7f439804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc92e8202f43901c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f4384ff9640 1 -- 172.21.15.172:0/3813985357 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398084ff0 con 0x563094036d50 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f4384ff9640 1 -- 172.21.15.172:0/3813985357 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439802db40 con 0x563094036d50 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.963+0000 7f4384ff9640 1 -- 172.21.15.172:0/3813985357 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439807b910 con 0x563094036d50 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.964+0000 7f43a9002740 1 -- 172.21.15.172:0/3813985357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036d50 msgr2=0x5630940332e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.964+0000 7f43a9002740 1 --2- 172.21.15.172:0/3813985357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036d50 0x5630940332e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f439802cb70 tx=0x7f439804b000 comp rx=0 tx=0).stop 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.964+0000 7f43a9002740 1 -- 172.21.15.172:0/3813985357 shutdown_connections 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.964+0000 7f43a9002740 1 --2- 172.21.15.172:0/3813985357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036d50 0x5630940332e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.964+0000 7f43a9002740 1 --2- 172.21.15.172:0/3813985357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036970 0x563094032150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.964+0000 7f43a9002740 1 --2- 172.21.15.172:0/3813985357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094023f70 0x56309402ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.964+0000 7f43a9002740 1 -- 172.21.15.172:0/3813985357 >> 172.21.15.172:0/3813985357 conn(0x563094009e50 msgr2=0x56309402ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.964+0000 7f43a9002740 1 -- 172.21.15.172:0/3813985357 shutdown_connections 2024-04-14T08:35:41.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.964+0000 7f43a9002740 1 -- 172.21.15.172:0/3813985357 wait complete. 2024-04-14T08:35:41.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.965+0000 7f43a9002740 1 -- start start 2024-04-14T08:35:41.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.965+0000 7f43a9002740 1 Processor -- start 2024-04-14T08:35:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.965+0000 7f43a9002740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094023f70 0x563094030b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.965+0000 7f43a9002740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 0x56309402e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.965+0000 7f43a9002740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 0x56309402eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.965+0000 7f43a9002740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5630940243b0 con 0x563094036970 2024-04-14T08:35:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.965+0000 7f43a9002740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563094009800 con 0x563094023f70 2024-04-14T08:35:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.965+0000 7f43a9002740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563094023410 con 0x563094036d50 2024-04-14T08:35:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.965+0000 7f43a9002740 10 client.0 _ll_register_callbacks cb 0x7f439807c320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:35:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f439effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 0x56309402e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f439effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 0x56309402e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38330/0 (socket says 172.21.15.172:38330) 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f439effd640 1 -- 172.21.15.172:0/550416948 learned_addr learned my addr 172.21.15.172:0/550416948 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f439f7fe640 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094023f70 0x563094030b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f439ffff640 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 0x56309402eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f439effd640 1 -- 172.21.15.172:0/550416948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 msgr2=0x56309402eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f439effd640 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 0x56309402eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f439effd640 1 -- 172.21.15.172:0/550416948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094023f70 msgr2=0x563094030b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f439effd640 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094023f70 0x563094030b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f439effd640 1 -- 172.21.15.172:0/550416948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563094055550 con 0x563094036970 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f439effd640 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 0x56309402e5f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4394005880 tx=0x7f4394001870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f38aeb367cab8d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f437e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f437e7fc640 1 -- 172.21.15.172:0/550416948 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43940166f0 con 0x563094036970 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f437e7fc640 1 -- 172.21.15.172:0/550416948 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f4394010040 con 0x563094036970 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f43a9002740 1 -- 172.21.15.172:0/550416948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f439802db40 con 0x563094036970 2024-04-14T08:35:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f437e7fc640 1 -- 172.21.15.172:0/550416948 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4394016890 con 0x563094036970 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f43a9002740 10 client.6420 fetch_fsmap learned FSMap version 30 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.966+0000 7f43a9002740 1 -- 172.21.15.172:0/550416948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x5630940d6700 con 0x563094036970 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f437e7fc640 1 -- 172.21.15.172:0/550416948 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439802db40 con 0x563094036970 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f437e7fc640 1 -- 172.21.15.172:0/550416948 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 3494+0+0 (secure 0 0 0) 0x7f4394016a30 con 0x563094036970 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f43a9002740 10 client.6420 fetch_fsmap finished waiting for FSMap version 30 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f43a9002740 10 client.6420 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f43a9002740 20 client.6420 populate_metadata read hostname 'smithi172' 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f43a9002740 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563094064030 0x5630940d7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f43a9002740 4 client.6420 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f43a9002740 1 -- 172.21.15.172:0/550416948 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563093f7b090 con 0x563094064030 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f439f7fe640 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563094064030 0x5630940d7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f437e7fc640 10 client.6420 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f439f7fe640 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563094064030 0x5630940d7570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e271ae9fa38b2d server_cookie=e4de2da9599a6abf in_seq=0 out_seq=0 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f437e7fc640 1 -- 172.21.15.172:0/550416948 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x563093f7b090 con 0x563094064030 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f437e7fc640 10 client.6420 handle_command_reply: tid=0 2024-04-14T08:35:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f43a9002740 1 client.6420 shutdown 2024-04-14T08:35:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f43a9002740 1 -- 172.21.15.172:0/550416948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 msgr2=0x56309402e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f43a9002740 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 0x56309402e5f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4394005880 tx=0x7f4394001870 comp rx=0 tx=0).stop 2024-04-14T08:35:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f43a9002740 1 -- 172.21.15.172:0/550416948 shutdown_connections 2024-04-14T08:35:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.967+0000 7f43a9002740 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563094064030 0x5630940d7570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a9002740 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563094036d50 0x56309402eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a9002740 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563094036970 0x56309402e5f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f437e7fc640 0 client.6420 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:35:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a9002740 1 --2- 172.21.15.172:0/550416948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563094023f70 0x563094030b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a9002740 1 -- 172.21.15.172:0/550416948 >> 172.21.15.172:0/550416948 conn(0x563094009e50 msgr2=0x56309400fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a9002740 1 -- 172.21.15.172:0/550416948 shutdown_connections 2024-04-14T08:35:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a9002740 1 -- 172.21.15.172:0/550416948 wait complete. 2024-04-14T08:35:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a9002740 20 client.6420 trim_cache size 0 max 16384 2024-04-14T08:35:41.974 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:35:41.974 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-14T08:35:41.974 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-04-14T08:35:41.974 INFO:teuthology.orchestra.run.smithi172.stdout: "88346069-7461-4514-9361-a012d7be5193": { 2024-04-14T08:35:41.974 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-04-14T08:35:41.974 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "88346069-7461-4514-9361-a012d7be5193", 2024-04-14T08:35:41.974 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive" 2024-04-14T08:35:41.974 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:41.974 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:35:41.974 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:35:41.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a611e640 1 -- 172.21.15.172:0/2521638588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f438c06c8f0 msgr2=0x7f438c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a611e640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f438c06c8f0 0x7f438c06edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4390000f60 tx=0x7f4390009040 comp rx=0 tx=0).stop 2024-04-14T08:35:41.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a611e640 1 -- 172.21.15.172:0/2521638588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a013fa70 msgr2=0x7f43a013d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:35:41.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a611e640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a013fa70 0x7f43a013d940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4398033000 tx=0x7f4398004a40 comp rx=0 tx=0).stop 2024-04-14T08:35:41.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a611e640 1 -- 172.21.15.172:0/2521638588 shutdown_connections 2024-04-14T08:35:41.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a611e640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f438c06c8f0 0x7f438c06edb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a611e640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43a013fa70 0x7f43a013d940 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a611e640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43a00af740 0x7f43a013d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a611e640 1 --2- 172.21.15.172:0/2521638588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43a00729f0 0x7f43a0158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:35:41.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a611e640 1 -- 172.21.15.172:0/2521638588 >> 172.21.15.172:0/2521638588 conn(0x7f43a006b520 msgr2=0x7f43a0159fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:35:41.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a611e640 1 -- 172.21.15.172:0/2521638588 shutdown_connections 2024-04-14T08:35:41.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:35:41.968+0000 7f43a611e640 1 -- 172.21.15.172:0/2521638588 wait complete. 2024-04-14T08:35:41.980 INFO:tasks.cephfs.filesystem:scrub status for tag:88346069-7461-4514-9361-a012d7be5193 - {'path': '/', 'tag': '88346069-7461-4514-9361-a012d7be5193', 'options': 'recursive'} 2024-04-14T08:35:46.995 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:35:46.995+0000 7f0f2508f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-14T08:35:53.471 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:35:53.474 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:36:11.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:36:11.981 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-14T08:36:12.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.095+0000 7f7390880640 1 Processor -- start 2024-04-14T08:36:12.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.095+0000 7f7390880640 1 -- start start 2024-04-14T08:36:12.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.096+0000 7f7390880640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f738c15e430 0x7f738c162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.096+0000 7f7390880640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 0x7f738c1629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.096+0000 7f7390880640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f738c0cefb0 con 0x7f738c162540 2024-04-14T08:36:12.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.096+0000 7f7390880640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f738c0cf290 con 0x7f738c15e430 2024-04-14T08:36:12.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.096+0000 7f7390880640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f738c0cf570 con 0x7f738c0d2e00 2024-04-14T08:36:12.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.097+0000 7f738ad9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 0x7f738c1629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.097+0000 7f738a59c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f738c15e430 0x7f738c162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.097+0000 7f7389d9b640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f738c162540 0x7f738c0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:40664/0 (socket says 172.21.15.172:40664) 2024-04-14T08:36:12.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.097+0000 7f7389d9b640 1 -- 172.21.15.172:0/3348293191 learned_addr learned my addr 172.21.15.172:0/3348293191 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:12.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.097+0000 7f738a59c640 1 -- 172.21.15.172:0/3348293191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 msgr2=0x7f738c1629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.097+0000 7f738a59c640 1 --2- 172.21.15.172:0/3348293191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 0x7f738c1629d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.097+0000 7f738a59c640 1 -- 172.21.15.172:0/3348293191 >> v1:172.21.15.177:6789/0 conn(0x7f738c162540 legacy=0x7f738c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.097+0000 7f738a59c640 1 -- 172.21.15.172:0/3348293191 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c0cf850 con 0x7f738c15e430 2024-04-14T08:36:12.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.097+0000 7f738959a640 1 -- 172.21.15.172:0/3348293191 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 707008344 0 0) 0x7f738c0cefb0 con 0x7f738c162540 2024-04-14T08:36:12.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.097+0000 7f738ad9d640 1 --2- 172.21.15.172:0/3348293191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 0x7f738c1629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:12.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.098+0000 7f738a59c640 1 --2- 172.21.15.172:0/3348293191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f738c15e430 0x7f738c162000 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f737c009d90 tx=0x7f737c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1397d401db60a454 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.098+0000 7f738959a640 1 -- 172.21.15.172:0/3348293191 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c039070 con 0x7f738c15e430 2024-04-14T08:36:12.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.098+0000 7f738959a640 1 -- 172.21.15.172:0/3348293191 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f737c02cc50 con 0x7f738c15e430 2024-04-14T08:36:12.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.098+0000 7f738959a640 1 -- 172.21.15.172:0/3348293191 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c034440 con 0x7f738c15e430 2024-04-14T08:36:12.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.098+0000 7f7390880640 1 -- 172.21.15.172:0/3348293191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f738c15e430 msgr2=0x7f738c162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.098+0000 7f7390880640 1 --2- 172.21.15.172:0/3348293191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f738c15e430 0x7f738c162000 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f737c009d90 tx=0x7f737c02bea0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.098+0000 7f7390880640 1 -- 172.21.15.172:0/3348293191 shutdown_connections 2024-04-14T08:36:12.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.098+0000 7f7390880640 1 --2- 172.21.15.172:0/3348293191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 0x7f738c1629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.098+0000 7f7390880640 1 --2- 172.21.15.172:0/3348293191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f738c15e430 0x7f738c162000 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.098+0000 7f7390880640 1 -- 172.21.15.172:0/3348293191 >> 172.21.15.172:0/3348293191 conn(0x7f738c09d560 msgr2=0x7f738c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.098+0000 7f7390880640 1 -- 172.21.15.172:0/3348293191 shutdown_connections 2024-04-14T08:36:12.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.098+0000 7f7390880640 1 -- 172.21.15.172:0/3348293191 wait complete. 2024-04-14T08:36:12.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.100+0000 7f7390880640 1 Processor -- start 2024-04-14T08:36:12.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.100+0000 7f7390880640 1 -- start start 2024-04-14T08:36:12.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.100+0000 7f7390880640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 0x7f738c161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7390880640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f738c15e430 0x7f738c0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7390880640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c162540 0x7f738c0cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7390880640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f738c0d4e20 con 0x7f738c162540 2024-04-14T08:36:12.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7390880640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f738c15e810 con 0x7f738c15e430 2024-04-14T08:36:12.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7390880640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f738c0d0050 con 0x7f738c0d2e00 2024-04-14T08:36:12.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f738a59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 0x7f738c161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f738ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c162540 0x7f738c0cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f738a59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 0x7f738c161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:36028/0 (socket says 172.21.15.172:36028) 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7389d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f738c15e430 0x7f738c0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f738ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c162540 0x7f738c0cc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46170/0 (socket says 172.21.15.172:46170) 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f738a59c640 1 -- 172.21.15.172:0/1788720106 learned_addr learned my addr 172.21.15.172:0/1788720106 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7389d9b640 1 -- 172.21.15.172:0/1788720106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 msgr2=0x7f738c161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7389d9b640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 0x7f738c161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7389d9b640 1 -- 172.21.15.172:0/1788720106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c162540 msgr2=0x7f738c0cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7389d9b640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c162540 0x7f738c0cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7389d9b640 1 -- 172.21.15.172:0/1788720106 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c15ebe0 con 0x7f738c15e430 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f738ad9d640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c162540 0x7f738c0cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f738a59c640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 0x7f738c161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7389d9b640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f738c15e430 0x7f738c0cdaa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f738c0d1870 tx=0x7f738000c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f737affd640 1 -- 172.21.15.172:0/1788720106 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380017070 con 0x7f738c15e430 2024-04-14T08:36:12.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f737affd640 1 -- 172.21.15.172:0/1788720106 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7380005ce0 con 0x7f738c15e430 2024-04-14T08:36:12.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7390880640 1 -- 172.21.15.172:0/1788720106 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738c15eec0 con 0x7f738c15e430 2024-04-14T08:36:12.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f737affd640 1 -- 172.21.15.172:0/1788720106 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73800072f0 con 0x7f738c15e430 2024-04-14T08:36:12.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.101+0000 7f7390880640 1 -- 172.21.15.172:0/1788720106 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f738c15f480 con 0x7f738c15e430 2024-04-14T08:36:12.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.103+0000 7f737affd640 1 -- 172.21.15.172:0/1788720106 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f738001a070 con 0x7f738c15e430 2024-04-14T08:36:12.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.103+0000 7f737affd640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f736c06c8f0 0x7f736c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.103+0000 7f737affd640 1 -- 172.21.15.172:0/1788720106 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f738008af00 con 0x7f738c15e430 2024-04-14T08:36:12.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.103+0000 7f738a59c640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f736c06c8f0 0x7f736c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.104+0000 7f738a59c640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f736c06c8f0 0x7f736c06edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f737c000f80 tx=0x7f737c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.104+0000 7f7393665740 1 Processor -- start 2024-04-14T08:36:12.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.105+0000 7f7393665740 1 -- start start 2024-04-14T08:36:12.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.105+0000 7f7393665740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 0x563200f55910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.105+0000 7f7393665740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82d50 0x563200f85700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.105+0000 7f7393665740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f55e50 0x563200f85c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.105+0000 7f7393665740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563200ebf800 con 0x563200f82970 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.105+0000 7f7393665740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563200e677f0 con 0x563200f82d50 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.105+0000 7f7393665740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563200ebb7b0 con 0x563200f55e50 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.105+0000 7f738a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 0x563200f55910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.105+0000 7f738a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 0x563200f55910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46178/0 (socket says 172.21.15.172:46178) 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.105+0000 7f738a59c640 1 -- 172.21.15.172:0/3488919783 learned_addr learned my addr 172.21.15.172:0/3488919783 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.105+0000 7f7389d9b640 1 --2- 172.21.15.172:0/3488919783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82d50 0x563200f85700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.105+0000 7f738ad9d640 1 --2- 172.21.15.172:0/3488919783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f55e50 0x563200f85c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.106+0000 7f738a59c640 1 -- 172.21.15.172:0/3488919783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f55e50 msgr2=0x563200f85c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.106+0000 7f738a59c640 1 --2- 172.21.15.172:0/3488919783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f55e50 0x563200f85c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.106+0000 7f738a59c640 1 -- 172.21.15.172:0/3488919783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82d50 msgr2=0x563200f85700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.106+0000 7f738a59c640 1 --2- 172.21.15.172:0/3488919783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82d50 0x563200f85700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.106+0000 7f738a59c640 1 -- 172.21.15.172:0/3488919783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563200c9d7d0 con 0x563200f82970 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.106+0000 7f738a59c640 1 --2- 172.21.15.172:0/3488919783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 0x563200f55910 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f737c009e80 tx=0x7f737c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce0f73dd78f9491 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.106+0000 7f736bfff640 1 -- 172.21.15.172:0/3488919783 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c04d070 con 0x563200f82970 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.106+0000 7f736bfff640 1 -- 172.21.15.172:0/3488919783 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f737c03a040 con 0x563200f82970 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.106+0000 7f736bfff640 1 -- 172.21.15.172:0/3488919783 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c043040 con 0x563200f82970 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.107+0000 7f7393665740 1 -- 172.21.15.172:0/3488919783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 msgr2=0x563200f55910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.107+0000 7f7393665740 1 --2- 172.21.15.172:0/3488919783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 0x563200f55910 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f737c009e80 tx=0x7f737c02e000 comp rx=0 tx=0).stop 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.107+0000 7f7393665740 1 -- 172.21.15.172:0/3488919783 shutdown_connections 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.107+0000 7f7393665740 1 --2- 172.21.15.172:0/3488919783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f55e50 0x563200f85c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.107+0000 7f7393665740 1 --2- 172.21.15.172:0/3488919783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82d50 0x563200f85700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.107+0000 7f7393665740 1 --2- 172.21.15.172:0/3488919783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 0x563200f55910 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.107+0000 7f7393665740 1 -- 172.21.15.172:0/3488919783 >> 172.21.15.172:0/3488919783 conn(0x563200f57b80 msgr2=0x563200fa0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.107+0000 7f7393665740 1 -- 172.21.15.172:0/3488919783 shutdown_connections 2024-04-14T08:36:12.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.107+0000 7f7393665740 1 -- 172.21.15.172:0/3488919783 wait complete. 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.107+0000 7f7393665740 1 -- start start 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.107+0000 7f7393665740 1 Processor -- start 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.108+0000 7f7393665740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82970 0x563200f99d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.108+0000 7f7393665740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 0x563200f9ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.108+0000 7f7393665740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f9b270 0x563200f744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.108+0000 7f7393665740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563200f5a120 con 0x563200f82d50 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.108+0000 7f7393665740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563200e7ad00 con 0x563200f82970 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.108+0000 7f7393665740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563200f59510 con 0x563200f9b270 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.108+0000 7f7393665740 10 client.0 _ll_register_callbacks cb 0x563200e7a6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.108+0000 7f7389d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 0x563200f9ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.108+0000 7f7389d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 0x563200f9ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46184/0 (socket says 172.21.15.172:46184) 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.108+0000 7f7389d9b640 1 -- 172.21.15.172:0/2599541783 learned_addr learned my addr 172.21.15.172:0/2599541783 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.108+0000 7f738a59c640 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82970 0x563200f99d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.108+0000 7f738ad9d640 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f9b270 0x563200f744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7389d9b640 1 -- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f9b270 msgr2=0x563200f744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7389d9b640 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f9b270 0x563200f744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7389d9b640 1 -- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82970 msgr2=0x563200f99d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7389d9b640 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82970 0x563200f99d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7389d9b640 1 -- 172.21.15.172:0/2599541783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563200bd33f0 con 0x563200f82d50 2024-04-14T08:36:12.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f738a59c640 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82970 0x563200f99d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:12.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7389d9b640 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 0x563200f9ad30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f738000cd20 tx=0x7f7380058d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233bedb4d9f6bb1d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f73697fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:36:12.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f73697fa640 1 -- 172.21.15.172:0/2599541783 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380071020 con 0x563200f82d50 2024-04-14T08:36:12.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f73697fa640 1 -- 172.21.15.172:0/2599541783 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f7380058030 con 0x563200f82d50 2024-04-14T08:36:12.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7393665740 1 -- 172.21.15.172:0/2599541783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f737c03a040 con 0x563200f82d50 2024-04-14T08:36:12.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f73697fa640 1 -- 172.21.15.172:0/2599541783 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7380059e60 con 0x563200f82d50 2024-04-14T08:36:12.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7393665740 10 client.6429 fetch_fsmap learned FSMap version 31 2024-04-14T08:36:12.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7393665740 1 -- 172.21.15.172:0/2599541783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x563200b930f0 con 0x563200f82d50 2024-04-14T08:36:12.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f73697fa640 1 -- 172.21.15.172:0/2599541783 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f737c03a040 con 0x563200f82d50 2024-04-14T08:36:12.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f73697fa640 1 -- 172.21.15.172:0/2599541783 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7f738005f0f0 con 0x563200f82d50 2024-04-14T08:36:12.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7393665740 10 client.6429 fetch_fsmap finished waiting for FSMap version 31 2024-04-14T08:36:12.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7393665740 10 client.6429 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-14T08:36:12.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7393665740 20 client.6429 populate_metadata read hostname 'smithi172' 2024-04-14T08:36:12.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7393665740 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563200f88780 0x563200f88ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7393665740 4 client.6429 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:36:12.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.109+0000 7f7393665740 1 -- 172.21.15.172:0/2599541783 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563200eccd40 con 0x563200f88780 2024-04-14T08:36:12.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.110+0000 7f738a59c640 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563200f88780 0x563200f88ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.110+0000 7f73697fa640 10 client.6429 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:12.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.110+0000 7f738a59c640 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563200f88780 0x563200f88ba0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eebe692272dad280 server_cookie=bb283e23b1712fa in_seq=0 out_seq=0 2024-04-14T08:36:12.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f73697fa640 1 -- 172.21.15.172:0/2599541783 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563200eccd40 con 0x563200f88780 2024-04-14T08:36:12.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f73697fa640 10 client.6429 handle_command_reply: tid=0 2024-04-14T08:36:12.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f7393665740 1 client.6429 shutdown 2024-04-14T08:36:12.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f7393665740 1 -- 172.21.15.172:0/2599541783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 msgr2=0x563200f9ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f7393665740 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 0x563200f9ad30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f738000cd20 tx=0x7f7380058d20 comp rx=0 tx=0).stop 2024-04-14T08:36:12.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f7393665740 1 -- 172.21.15.172:0/2599541783 shutdown_connections 2024-04-14T08:36:12.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f7393665740 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563200f88780 0x563200f88ba0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f7393665740 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f9b270 0x563200f744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f7393665740 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 0x563200f9ad30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f7393665740 1 --2- 172.21.15.172:0/2599541783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82970 0x563200f99d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f73697fa640 0 client.6429 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:12.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f7393665740 1 -- 172.21.15.172:0/2599541783 >> 172.21.15.172:0/2599541783 conn(0x563200f55e50 msgr2=0x563200f856f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f7393665740 1 -- 172.21.15.172:0/2599541783 shutdown_connections 2024-04-14T08:36:12.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f7393665740 1 -- 172.21.15.172:0/2599541783 wait complete. 2024-04-14T08:36:12.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.111+0000 7f7393665740 20 client.6429 trim_cache size 0 max 16384 2024-04-14T08:36:12.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.128+0000 7f7393665740 1 Processor -- start 2024-04-14T08:36:12.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.128+0000 7f7393665740 1 -- start start 2024-04-14T08:36:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7393665740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82970 0x563200f86fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7393665740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 0x563200f59510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7393665740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f9b270 0x563200f59a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7393665740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f738c200e00 con 0x563200f82d50 2024-04-14T08:36:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7393665740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f738c1ffed0 con 0x563200f82970 2024-04-14T08:36:12.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7393665740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f738c0c9850 con 0x563200f9b270 2024-04-14T08:36:12.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7389d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 0x563200f59510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7389d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 0x563200f59510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46198/0 (socket says 172.21.15.172:46198) 2024-04-14T08:36:12.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7389d9b640 1 -- 172.21.15.172:0/4181393185 learned_addr learned my addr 172.21.15.172:0/4181393185 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f738ad9d640 1 --2- 172.21.15.172:0/4181393185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f9b270 0x563200f59a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f738a59c640 1 --2- 172.21.15.172:0/4181393185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82970 0x563200f86fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7389d9b640 1 -- 172.21.15.172:0/4181393185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f9b270 msgr2=0x563200f59a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7389d9b640 1 --2- 172.21.15.172:0/4181393185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f9b270 0x563200f59a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7389d9b640 1 -- 172.21.15.172:0/4181393185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82970 msgr2=0x563200f86fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7389d9b640 1 --2- 172.21.15.172:0/4181393185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82970 0x563200f86fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f7389d9b640 1 -- 172.21.15.172:0/4181393185 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563200f51c20 con 0x563200f82d50 2024-04-14T08:36:12.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f738a59c640 1 --2- 172.21.15.172:0/4181393185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82970 0x563200f86fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:36:12.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.129+0000 7f738ad9d640 1 --2- 172.21.15.172:0/4181393185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f9b270 0x563200f59a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:36:12.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f7389d9b640 1 --2- 172.21.15.172:0/4181393185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 0x563200f59510 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f738005d080 tx=0x7f738005c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33643d3b2651a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f736b7fe640 1 -- 172.21.15.172:0/4181393185 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380071040 con 0x563200f82d50 2024-04-14T08:36:12.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f736b7fe640 1 -- 172.21.15.172:0/4181393185 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f738000ce50 con 0x563200f82d50 2024-04-14T08:36:12.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f736b7fe640 1 -- 172.21.15.172:0/4181393185 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380061710 con 0x563200f82d50 2024-04-14T08:36:12.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f7393665740 1 -- 172.21.15.172:0/4181393185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 msgr2=0x563200f59510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f7393665740 1 --2- 172.21.15.172:0/4181393185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 0x563200f59510 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f738005d080 tx=0x7f738005c000 comp rx=0 tx=0).stop 2024-04-14T08:36:12.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f7393665740 1 -- 172.21.15.172:0/4181393185 shutdown_connections 2024-04-14T08:36:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f7393665740 1 --2- 172.21.15.172:0/4181393185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f9b270 0x563200f59a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f7393665740 1 --2- 172.21.15.172:0/4181393185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82d50 0x563200f59510 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f7393665740 1 --2- 172.21.15.172:0/4181393185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f82970 0x563200f86fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f7393665740 1 -- 172.21.15.172:0/4181393185 >> 172.21.15.172:0/4181393185 conn(0x563200f55e50 msgr2=0x563200f742c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f7393665740 1 -- 172.21.15.172:0/4181393185 shutdown_connections 2024-04-14T08:36:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.130+0000 7f7393665740 1 -- 172.21.15.172:0/4181393185 wait complete. 2024-04-14T08:36:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.131+0000 7f7393665740 1 -- start start 2024-04-14T08:36:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.131+0000 7f7393665740 1 Processor -- start 2024-04-14T08:36:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f7393665740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 0x563200f59510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f7393665740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f82d50 0x563200f59a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f7393665740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f9b270 0x563200f5a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f7393665740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563200f73bb0 con 0x563200f82970 2024-04-14T08:36:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f7393665740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f738c1ff8b0 con 0x563200f9b270 2024-04-14T08:36:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f7393665740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563200f572e0 con 0x563200f82d50 2024-04-14T08:36:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f7393665740 10 client.0 _ll_register_callbacks cb 0x7f7380058a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:36:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f738a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 0x563200f59510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f738a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 0x563200f59510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46204/0 (socket says 172.21.15.172:46204) 2024-04-14T08:36:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f738a59c640 1 -- 172.21.15.172:0/3682962065 learned_addr learned my addr 172.21.15.172:0/3682962065 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f7389d9b640 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f82d50 0x563200f59a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f738ad9d640 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f9b270 0x563200f5a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f738a59c640 1 -- 172.21.15.172:0/3682962065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f82d50 msgr2=0x563200f59a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f738a59c640 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f82d50 0x563200f59a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f738a59c640 1 -- 172.21.15.172:0/3682962065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f9b270 msgr2=0x563200f5a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f738a59c640 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f9b270 0x563200f5a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f738a59c640 1 -- 172.21.15.172:0/3682962065 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563200f86440 con 0x563200f82970 2024-04-14T08:36:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f738a59c640 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 0x563200f59510 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f737c041040 tx=0x7f737c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b584a739d9edcc26 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f73697fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:36:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.132+0000 7f73697fa640 1 -- 172.21.15.172:0/3682962065 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737c061060 con 0x563200f82970 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f73697fa640 1 -- 172.21.15.172:0/3682962065 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f737c034930 con 0x563200f82970 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f7393665740 1 -- 172.21.15.172:0/3682962065 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f738000ce50 con 0x563200f82970 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f73697fa640 1 -- 172.21.15.172:0/3682962065 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f737c067050 con 0x563200f82970 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f7393665740 10 client.6435 fetch_fsmap learned FSMap version 31 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f7393665740 1 -- 172.21.15.172:0/3682962065 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x563200f99a00 con 0x563200f82970 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f73697fa640 1 -- 172.21.15.172:0/3682962065 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f738000ce50 con 0x563200f82970 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f73697fa640 1 -- 172.21.15.172:0/3682962065 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7f737c06a020 con 0x563200f82970 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f7393665740 10 client.6435 fetch_fsmap finished waiting for FSMap version 31 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f7393665740 10 client.6435 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f7393665740 20 client.6435 populate_metadata read hostname 'smithi172' 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f7393665740 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563200fb5a70 0x563200f5a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f7393665740 4 client.6435 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f7393665740 1 -- 172.21.15.172:0/3682962065 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563200ec7090 con 0x563200fb5a70 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f7389d9b640 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563200fb5a70 0x563200f5a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f73697fa640 10 client.6435 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.133+0000 7f7389d9b640 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563200fb5a70 0x563200f5a870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf0bf53a57f4a391 server_cookie=9aae24e7d33bcd45 in_seq=0 out_seq=0 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f73697fa640 1 -- 172.21.15.172:0/3682962065 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563200ec7090 con 0x563200fb5a70 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f73697fa640 10 client.6435 handle_command_reply: tid=0 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7393665740 1 client.6435 shutdown 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7393665740 1 -- 172.21.15.172:0/3682962065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 msgr2=0x563200f59510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7393665740 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 0x563200f59510 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f737c041040 tx=0x7f737c02e070 comp rx=0 tx=0).stop 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7393665740 1 -- 172.21.15.172:0/3682962065 shutdown_connections 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7393665740 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x563200fb5a70 0x563200f5a870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7393665740 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563200f9b270 0x563200f5a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7393665740 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563200f82d50 0x563200f59a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7393665740 1 --2- 172.21.15.172:0/3682962065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563200f82970 0x563200f59510 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f73697fa640 0 client.6435 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7393665740 1 -- 172.21.15.172:0/3682962065 >> 172.21.15.172:0/3682962065 conn(0x563200f55e50 msgr2=0x563200f97890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7393665740 1 -- 172.21.15.172:0/3682962065 shutdown_connections 2024-04-14T08:36:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7393665740 1 -- 172.21.15.172:0/3682962065 wait complete. 2024-04-14T08:36:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7393665740 20 client.6435 trim_cache size 0 max 16384 2024-04-14T08:36:12.143 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:36:12.143 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-04-14T08:36:12.143 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-04-14T08:36:12.143 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:36:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7390880640 1 -- 172.21.15.172:0/1788720106 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f736c06c8f0 msgr2=0x7f736c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7390880640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f736c06c8f0 0x7f736c06edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f737c000f80 tx=0x7f737c030000 comp rx=0 tx=0).stop 2024-04-14T08:36:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7390880640 1 -- 172.21.15.172:0/1788720106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f738c15e430 msgr2=0x7f738c0cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7390880640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f738c15e430 0x7f738c0cdaa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f738c0d1870 tx=0x7f738000c820 comp rx=0 tx=0).stop 2024-04-14T08:36:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7390880640 1 -- 172.21.15.172:0/1788720106 shutdown_connections 2024-04-14T08:36:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.134+0000 7f7390880640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f736c06c8f0 0x7f736c06edb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.135+0000 7f7390880640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f738c162540 0x7f738c0cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.135+0000 7f7390880640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f738c15e430 0x7f738c0cdaa0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.135+0000 7f7390880640 1 --2- 172.21.15.172:0/1788720106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f738c0d2e00 0x7f738c161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.135+0000 7f7390880640 1 -- 172.21.15.172:0/1788720106 >> 172.21.15.172:0/1788720106 conn(0x7f738c09d560 msgr2=0x7f738c0d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.135+0000 7f7390880640 1 -- 172.21.15.172:0/1788720106 shutdown_connections 2024-04-14T08:36:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.135+0000 7f7390880640 1 -- 172.21.15.172:0/1788720106 wait complete. 2024-04-14T08:36:12.146 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-14T08:36:12.146 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:36:12.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.248+0000 7fb8aaa2d640 1 Processor -- start 2024-04-14T08:36:12.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.248+0000 7fb8aaa2d640 1 -- start start 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8aaa2d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8a415a690 0x7fb8a4071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8aaa2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8a415c240 0x7fb8a40722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8aaa2d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8a40b3360 con 0x7fb8a4072970 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8aaa2d640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a40b3640 con 0x7fb8a415a690 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8aaa2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a40b3920 con 0x7fb8a415c240 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8a3fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8a415a690 0x7fb8a4071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8a3fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8a415a690 0x7fb8a4071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46218/0 (socket says 172.21.15.172:46218) 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8a3fff640 1 -- 172.21.15.172:0/2474176295 learned_addr learned my addr 172.21.15.172:0/2474176295 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8a8fca640 1 --2- 172.21.15.172:0/2474176295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8a415c240 0x7fb8a40722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8a3fff640 1 -- 172.21.15.172:0/2474176295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8a415c240 msgr2=0x7fb8a40722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8a3fff640 1 --2- 172.21.15.172:0/2474176295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8a415c240 0x7fb8a40722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8a3fff640 1 -- 172.21.15.172:0/2474176295 >> v1:172.21.15.177:6789/0 conn(0x7fb8a4072970 legacy=0x7fb8a415e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.249+0000 7fb8a3fff640 1 -- 172.21.15.172:0/2474176295 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a40b3c00 con 0x7fb8a415a690 2024-04-14T08:36:12.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.250+0000 7fb8a3fff640 1 --2- 172.21.15.172:0/2474176295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a415a690 0x7fb8a4071d60 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb89400bb10 tx=0x7fb89402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b096226fe0c94f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.250+0000 7fb8a2ffd640 1 -- 172.21.15.172:0/2474176295 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89403b070 con 0x7fb8a415a690 2024-04-14T08:36:12.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.250+0000 7fb8a2ffd640 1 -- 172.21.15.172:0/2474176295 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb894036440 con 0x7fb8a415a690 2024-04-14T08:36:12.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.250+0000 7fb8a2ffd640 1 -- 172.21.15.172:0/2474176295 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894036810 con 0x7fb8a415a690 2024-04-14T08:36:12.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.251+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2474176295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a415a690 msgr2=0x7fb8a4071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.251+0000 7fb8aaa2d640 1 --2- 172.21.15.172:0/2474176295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a415a690 0x7fb8a4071d60 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb89400bb10 tx=0x7fb89402e0c0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.251+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2474176295 shutdown_connections 2024-04-14T08:36:12.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.251+0000 7fb8aaa2d640 1 --2- 172.21.15.172:0/2474176295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8a415c240 0x7fb8a40722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.251+0000 7fb8aaa2d640 1 --2- 172.21.15.172:0/2474176295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a415a690 0x7fb8a4071d60 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.251+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2474176295 >> 172.21.15.172:0/2474176295 conn(0x7fb8a406ec30 msgr2=0x7fb8a406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.251+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2474176295 shutdown_connections 2024-04-14T08:36:12.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.251+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2474176295 wait complete. 2024-04-14T08:36:12.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.252+0000 7fb8aaa2d640 1 Processor -- start 2024-04-14T08:36:12.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.252+0000 7fb8aaa2d640 1 -- start start 2024-04-14T08:36:12.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8aaa2d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a4072970 0x7fb8a40c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8aaa2d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8a415a690 0x7fb8a40ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8aaa2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8a415c240 0x7fb8a40c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8aaa2d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a415aa70 con 0x7fb8a4072970 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8aaa2d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a40b3100 con 0x7fb8a415a690 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8aaa2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a415abf0 con 0x7fb8a415c240 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a4072970 0x7fb8a40c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a4072970 0x7fb8a40c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46220/0 (socket says 172.21.15.172:46220) 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a3fff640 1 -- 172.21.15.172:0/2088028569 learned_addr learned my addr 172.21.15.172:0/2088028569 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a37fe640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8a415a690 0x7fb8a40ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a8fca640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8a415c240 0x7fb8a40c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a3fff640 1 -- 172.21.15.172:0/2088028569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8a415c240 msgr2=0x7fb8a40c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a3fff640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8a415c240 0x7fb8a40c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a3fff640 1 -- 172.21.15.172:0/2088028569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8a415a690 msgr2=0x7fb8a40ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a3fff640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8a415a690 0x7fb8a40ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a3fff640 1 -- 172.21.15.172:0/2088028569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a40c6970 con 0x7fb8a4072970 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a37fe640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8a415a690 0x7fb8a40ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a3fff640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a4072970 0x7fb8a40c8820 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb89402e100 tx=0x7fb8940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.253+0000 7fb8a0ff9640 1 -- 172.21.15.172:0/2088028569 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894045030 con 0x7fb8a4072970 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.254+0000 7fb8a0ff9640 1 -- 172.21.15.172:0/2088028569 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb894034440 con 0x7fb8a4072970 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.254+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2088028569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a40c6c50 con 0x7fb8a4072970 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.254+0000 7fb8a0ff9640 1 -- 172.21.15.172:0/2088028569 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89403b040 con 0x7fb8a4072970 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.254+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2088028569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a40c9c30 con 0x7fb8a4072970 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.255+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2088028569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a40c8dc0 con 0x7fb8a4072970 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.255+0000 7fb8a0ff9640 1 -- 172.21.15.172:0/2088028569 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb894048070 con 0x7fb8a4072970 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.255+0000 7fb8a0ff9640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb89006c8f0 0x7fb89006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.255+0000 7fb8a0ff9640 1 -- 172.21.15.172:0/2088028569 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fb8940b0470 con 0x7fb8a4072970 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.255+0000 7fb8a37fe640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb89006c8f0 0x7fb89006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.256+0000 7fb8a37fe640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb89006c8f0 0x7fb89006edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40cbf30 tx=0x7fb898009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.260+0000 7fb8a0ff9640 1 -- 172.21.15.172:0/2088028569 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb8a40c8dc0 con 0x7fb8a4072970 2024-04-14T08:36:12.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.470+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2088028569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8a4068520 con 0x7fb8a4072970 2024-04-14T08:36:12.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.471+0000 7fb8a0ff9640 1 -- 172.21.15.172:0/2088028569 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5194 (secure 0 0 0) 0x7fb89407e270 con 0x7fb8a4072970 2024-04-14T08:36:12.471 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:12.472 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3621780015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3621780015},{"type":"v1","addr":"172.21.15.172:6836","nonce":3621780015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3894683748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3894683748},{"type":"v1","addr":"172.21.15.172:6837","nonce":3894683748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3625745587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3625745587},{"type":"v1","addr":"172.21.15.177:6834","nonce":3625745587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:35:34.316542+0000","modified":"2024-04-14T08:35:46.992276+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":6,"addr":"172.21.15.177:6835/4176013980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":4176013980},{"type":"v1","addr":"172.21.15.177:6835","nonce":4176013980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-14T08:36:12.472 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 31 2024-04-14T08:36:12.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.473+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2088028569 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb89006c8f0 msgr2=0x7fb89006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.473+0000 7fb8aaa2d640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb89006c8f0 0x7fb89006edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40cbf30 tx=0x7fb898009040 comp rx=0 tx=0).stop 2024-04-14T08:36:12.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.473+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2088028569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a4072970 msgr2=0x7fb8a40c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.473+0000 7fb8aaa2d640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a4072970 0x7fb8a40c8820 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb89402e100 tx=0x7fb8940046c0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.473+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2088028569 shutdown_connections 2024-04-14T08:36:12.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.473+0000 7fb8aaa2d640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb89006c8f0 0x7fb89006edb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.473+0000 7fb8aaa2d640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8a415c240 0x7fb8a40c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.473+0000 7fb8aaa2d640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8a415a690 0x7fb8a40ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.473+0000 7fb8aaa2d640 1 --2- 172.21.15.172:0/2088028569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a4072970 0x7fb8a40c8820 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.473+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2088028569 >> 172.21.15.172:0/2088028569 conn(0x7fb8a406ec30 msgr2=0x7fb8a4158720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.473+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2088028569 shutdown_connections 2024-04-14T08:36:12.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.473+0000 7fb8aaa2d640 1 -- 172.21.15.172:0/2088028569 wait complete. 2024-04-14T08:36:12.484 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-04-14T08:36:12.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.586+0000 7ff1b4691640 1 Processor -- start 2024-04-14T08:36:12.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.586+0000 7ff1b4691640 1 -- start start 2024-04-14T08:36:12.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.587+0000 7ff1b4691640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1ac0d09d0 0x7ff1ac0cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.587+0000 7ff1b4691640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff1ac0cd650 0x7ff1ac0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.587+0000 7ff1b4691640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1ac0d2df0 con 0x7ff1ac15c4e0 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.587+0000 7ff1b4691640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff1ac0cdb50 con 0x7ff1ac0cd650 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.587+0000 7ff1b4691640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ac0cde30 con 0x7ff1ac0d09d0 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.587+0000 7ff1b242d640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7ff1ac15c4e0 0x7ff1ac0d02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:40672/0 (socket says 172.21.15.172:40672) 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.587+0000 7ff1b242d640 1 -- 172.21.15.172:0/2439158757 learned_addr learned my addr 172.21.15.172:0/2439158757 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.587+0000 7ff1b2c2e640 1 --2- 172.21.15.172:0/2439158757 >> v2:172.21.15.177:3300/0 conn(0x7ff1ac0cd650 0x7ff1ac0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.587+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/2439158757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1ac0d09d0 0x7ff1ac0cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.588+0000 7ff1b142b640 1 -- 172.21.15.172:0/2439158757 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1466656818 0 0) 0x7ff1ac0d2df0 con 0x7ff1ac15c4e0 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.588+0000 7ff1b142b640 1 -- 172.21.15.172:0/2439158757 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff198003520 con 0x7ff1ac15c4e0 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.588+0000 7ff1b2c2e640 1 -- 172.21.15.172:0/2439158757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1ac0d09d0 msgr2=0x7ff1ac0cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.588+0000 7ff1b2c2e640 1 --2- 172.21.15.172:0/2439158757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1ac0d09d0 0x7ff1ac0cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.588+0000 7ff1b2c2e640 1 -- 172.21.15.172:0/2439158757 >> v1:172.21.15.177:6789/0 conn(0x7ff1ac15c4e0 legacy=0x7ff1ac0d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.588+0000 7ff1b2c2e640 1 -- 172.21.15.172:0/2439158757 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac0b1990 con 0x7ff1ac0cd650 2024-04-14T08:36:12.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.588+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/2439158757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1ac0d09d0 0x7ff1ac0cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:12.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.589+0000 7ff1b2c2e640 1 --2- 172.21.15.172:0/2439158757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1ac0cd650 0x7ff1ac0b2470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40097a0 tx=0x7ff1a402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9938dc01e307074a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.589+0000 7ff1b142b640 1 -- 172.21.15.172:0/2439158757 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4035020 con 0x7ff1ac0cd650 2024-04-14T08:36:12.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.589+0000 7ff1b142b640 1 -- 172.21.15.172:0/2439158757 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a4031830 con 0x7ff1ac0cd650 2024-04-14T08:36:12.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.589+0000 7ff1b142b640 1 -- 172.21.15.172:0/2439158757 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4031be0 con 0x7ff1ac0cd650 2024-04-14T08:36:12.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.589+0000 7ff1b4691640 1 -- 172.21.15.172:0/2439158757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1ac0cd650 msgr2=0x7ff1ac0b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.589+0000 7ff1b4691640 1 --2- 172.21.15.172:0/2439158757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1ac0cd650 0x7ff1ac0b2470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40097a0 tx=0x7ff1a402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.589+0000 7ff1b4691640 1 -- 172.21.15.172:0/2439158757 shutdown_connections 2024-04-14T08:36:12.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.589+0000 7ff1b4691640 1 --2- 172.21.15.172:0/2439158757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1ac0cd650 0x7ff1ac0b2470 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.589+0000 7ff1b4691640 1 --2- 172.21.15.172:0/2439158757 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1ac0d09d0 0x7ff1ac0cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.589+0000 7ff1b4691640 1 -- 172.21.15.172:0/2439158757 >> 172.21.15.172:0/2439158757 conn(0x7ff1ac0d2030 msgr2=0x7ff1ac0d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.589+0000 7ff1b4691640 1 -- 172.21.15.172:0/2439158757 shutdown_connections 2024-04-14T08:36:12.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.589+0000 7ff1b4691640 1 -- 172.21.15.172:0/2439158757 wait complete. 2024-04-14T08:36:12.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.590+0000 7ff1b4691640 1 Processor -- start 2024-04-14T08:36:12.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.590+0000 7ff1b4691640 1 -- start start 2024-04-14T08:36:12.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.591+0000 7ff1b4691640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1ac0cd650 0x7ff1ac15f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.591+0000 7ff1b4691640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1ac0d09d0 0x7ff1ac15d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.591+0000 7ff1b4691640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1ac15c4e0 0x7ff1ac0b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.591+0000 7ff1b4691640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac0b2b30 con 0x7ff1ac0cd650 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.591+0000 7ff1b4691640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac0b29b0 con 0x7ff1ac0d09d0 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.591+0000 7ff1b4691640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ac0b2cb0 con 0x7ff1ac15c4e0 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.591+0000 7ff1b242d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1ac0cd650 0x7ff1ac15f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.591+0000 7ff1b242d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1ac0cd650 0x7ff1ac15f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46222/0 (socket says 172.21.15.172:46222) 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.591+0000 7ff1b242d640 1 -- 172.21.15.172:0/2482841606 learned_addr learned my addr 172.21.15.172:0/2482841606 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.591+0000 7ff1b2c2e640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1ac15c4e0 0x7ff1ac0b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.591+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1ac0d09d0 0x7ff1ac15d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.592+0000 7ff1b242d640 1 -- 172.21.15.172:0/2482841606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1ac15c4e0 msgr2=0x7ff1ac0b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.592+0000 7ff1b242d640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1ac15c4e0 0x7ff1ac0b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.592+0000 7ff1b242d640 1 -- 172.21.15.172:0/2482841606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1ac0d09d0 msgr2=0x7ff1ac15d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.592+0000 7ff1b242d640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1ac0d09d0 0x7ff1ac15d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.592+0000 7ff1b242d640 1 -- 172.21.15.172:0/2482841606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac0c7660 con 0x7ff1ac0cd650 2024-04-14T08:36:12.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.592+0000 7ff1b242d640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1ac0cd650 0x7ff1ac15f1a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800caa0 tx=0x7ff1a800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.592+0000 7ff1a2ffd640 1 -- 172.21.15.172:0/2482841606 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8017070 con 0x7ff1ac0cd650 2024-04-14T08:36:12.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.592+0000 7ff1a2ffd640 1 -- 172.21.15.172:0/2482841606 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a800ac00 con 0x7ff1ac0cd650 2024-04-14T08:36:12.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.592+0000 7ff1a2ffd640 1 -- 172.21.15.172:0/2482841606 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8012400 con 0x7ff1ac0cd650 2024-04-14T08:36:12.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.592+0000 7ff1b4691640 1 -- 172.21.15.172:0/2482841606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ac0c7940 con 0x7ff1ac0cd650 2024-04-14T08:36:12.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.592+0000 7ff1b4691640 1 -- 172.21.15.172:0/2482841606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ac0c7f00 con 0x7ff1ac0cd650 2024-04-14T08:36:12.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.593+0000 7ff1a2ffd640 1 -- 172.21.15.172:0/2482841606 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1a801a070 con 0x7ff1ac0cd650 2024-04-14T08:36:12.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.594+0000 7ff1a2ffd640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff19006c8f0 0x7ff19006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.594+0000 7ff1a2ffd640 1 -- 172.21.15.172:0/2482841606 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7ff1a808b9f0 con 0x7ff1ac0cd650 2024-04-14T08:36:12.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.594+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff19006c8f0 0x7ff19006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.594+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff19006c8f0 0x7ff19006edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009560 tx=0x7ff19c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.595+0000 7ff1b7471740 1 Processor -- start 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.595+0000 7ff1b7471740 1 -- start start 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b7471740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055be9bd90 0x56055be9c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b7471740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bea1bb0 0x56055be9fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b7471740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055bee9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b7471740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56055be635a0 con 0x56055bea1bb0 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b7471740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56055be05680 con 0x56055be9bd90 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b7471740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56055bda3860 con 0x56055bee94a0 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b1c2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bea1bb0 0x56055be9fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b1c2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bea1bb0 0x56055be9fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46226/0 (socket says 172.21.15.172:46226) 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b1c2c640 1 -- 172.21.15.172:0/2666920898 learned_addr learned my addr 172.21.15.172:0/2666920898 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b2c2e640 1 --2- 172.21.15.172:0/2666920898 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055bee9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b242d640 1 --2- 172.21.15.172:0/2666920898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055be9bd90 0x56055be9c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b242d640 1 -- 172.21.15.172:0/2666920898 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 msgr2=0x56055bee9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b242d640 1 --2- 172.21.15.172:0/2666920898 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055bee9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b242d640 1 -- 172.21.15.172:0/2666920898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bea1bb0 msgr2=0x56055be9fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b242d640 1 --2- 172.21.15.172:0/2666920898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bea1bb0 0x56055be9fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b242d640 1 -- 172.21.15.172:0/2666920898 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055bdd21a0 con 0x56055be9bd90 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/2666920898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bea1bb0 0x56055be9fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff1b242d640 1 --2- 172.21.15.172:0/2666920898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055be9bd90 0x56055be9c170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8089de0 tx=0x7ff1a805da00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2401e2aa5f892b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.596+0000 7ff18ffff640 1 -- 172.21.15.172:0/2666920898 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8055cb0 con 0x56055be9bd90 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.597+0000 7ff18ffff640 1 -- 172.21.15.172:0/2666920898 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a8061ce0 con 0x56055be9bd90 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.597+0000 7ff18ffff640 1 -- 172.21.15.172:0/2666920898 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8055e50 con 0x56055be9bd90 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.597+0000 7ff1b7471740 1 -- 172.21.15.172:0/2666920898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055be9bd90 msgr2=0x56055be9c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.597+0000 7ff1b7471740 1 --2- 172.21.15.172:0/2666920898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055be9bd90 0x56055be9c170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8089de0 tx=0x7ff1a805da00 comp rx=0 tx=0).stop 2024-04-14T08:36:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.597+0000 7ff1b7471740 1 -- 172.21.15.172:0/2666920898 shutdown_connections 2024-04-14T08:36:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.597+0000 7ff1b7471740 1 --2- 172.21.15.172:0/2666920898 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055bee9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.597+0000 7ff1b7471740 1 --2- 172.21.15.172:0/2666920898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bea1bb0 0x56055be9fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.597+0000 7ff1b7471740 1 --2- 172.21.15.172:0/2666920898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055be9bd90 0x56055be9c170 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.597+0000 7ff1b7471740 1 -- 172.21.15.172:0/2666920898 >> 172.21.15.172:0/2666920898 conn(0x56055be9de10 msgr2=0x56055be9f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.597+0000 7ff1b7471740 1 -- 172.21.15.172:0/2666920898 shutdown_connections 2024-04-14T08:36:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.597+0000 7ff1b7471740 1 -- 172.21.15.172:0/2666920898 wait complete. 2024-04-14T08:36:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.598+0000 7ff1b7471740 1 -- start start 2024-04-14T08:36:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.598+0000 7ff1b7471740 1 Processor -- start 2024-04-14T08:36:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b7471740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bea1bb0 0x56055becf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b7471740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055becb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b7471740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055becbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b7471740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56055bda11d0 con 0x56055bec0f30 2024-04-14T08:36:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b7471740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56055be01630 con 0x56055bea1bb0 2024-04-14T08:36:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b7471740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56055bec8500 con 0x56055bee94a0 2024-04-14T08:36:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b7471740 10 client.0 _ll_register_callbacks cb 0x56055bdc4010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:36:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b2c2e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055becbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b2c2e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055becbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46240/0 (socket says 172.21.15.172:46240) 2024-04-14T08:36:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b2c2e640 1 -- 172.21.15.172:0/4060347623 learned_addr learned my addr 172.21.15.172:0/4060347623 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055becb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b242d640 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bea1bb0 0x56055becf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b1c2c640 1 -- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bea1bb0 msgr2=0x56055becf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bea1bb0 0x56055becf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b1c2c640 1 -- 172.21.15.172:0/4060347623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 msgr2=0x56055becbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055becbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b1c2c640 1 -- 172.21.15.172:0/4060347623 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x56055bbe57c0 con 0x56055bee94a0 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b2c2e640 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055becbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b242d640 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bea1bb0 0x56055becf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055becb9d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff19c003640 tx=0x7ff19c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68e118f900da56cd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff18d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.177:3301/0 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.599+0000 7ff18d7fa640 1 -- 172.21.15.172:0/4060347623 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c027070 con 0x56055bee94a0 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff18d7fa640 1 -- 172.21.15.172:0/4060347623 <== mon.2 v2:172.21.15.177:3301/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7ff19c014650 con 0x56055bee94a0 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff1b7471740 1 -- 172.21.15.172:0/4060347623 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a8061ce0 con 0x56055bee94a0 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff18d7fa640 1 -- 172.21.15.172:0/4060347623 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff19c032050 con 0x56055bee94a0 2024-04-14T08:36:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff1b7471740 10 client.5702 fetch_fsmap learned FSMap version 31 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff1b7471740 1 -- 172.21.15.172:0/4060347623 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x56055bc45f50 con 0x56055bee94a0 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff18d7fa640 1 -- 172.21.15.172:0/4060347623 <== mon.2 v2:172.21.15.177:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a8061ce0 con 0x56055bee94a0 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff18d7fa640 1 -- 172.21.15.172:0/4060347623 <== mon.2 v2:172.21.15.177:3301/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7ff19c035020 con 0x56055bee94a0 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff1b7471740 10 client.5702 fetch_fsmap finished waiting for FSMap version 31 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff1b7471740 10 client.5702 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff1b7471740 20 client.5702 populate_metadata read hostname 'smithi172' 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff1b7471740 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x56055bec2a10 0x56055bf06060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff1b7471740 4 client.5702 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff1b7471740 1 -- 172.21.15.172:0/4060347623 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56055be18eb0 con 0x56055bec2a10 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff1b242d640 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x56055bec2a10 0x56055bf06060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff1b242d640 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x56055bec2a10 0x56055bf06060 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8630acf1b295717 server_cookie=3a4f301f6084c6b7 in_seq=0 out_seq=0 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.600+0000 7ff18d7fa640 10 client.5702 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff18d7fa640 1 -- 172.21.15.172:0/4060347623 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56055be18eb0 con 0x56055bec2a10 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff18d7fa640 10 client.5702 handle_command_reply: tid=0 2024-04-14T08:36:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff1b7471740 1 client.5702 shutdown 2024-04-14T08:36:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff1b7471740 1 -- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 msgr2=0x56055becb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff1b7471740 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055becb9d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff19c003640 tx=0x7ff19c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff1b7471740 1 -- 172.21.15.172:0/4060347623 shutdown_connections 2024-04-14T08:36:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff1b7471740 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x56055bec2a10 0x56055bf06060 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff1b7471740 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055becbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff18d7fa640 0 client.5702 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff1b7471740 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055becb9d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff1b7471740 1 --2- 172.21.15.172:0/4060347623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bea1bb0 0x56055becf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff1b7471740 1 -- 172.21.15.172:0/4060347623 >> 172.21.15.172:0/4060347623 conn(0x56055be9bd90 msgr2=0x56055bec7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff1b7471740 1 -- 172.21.15.172:0/4060347623 shutdown_connections 2024-04-14T08:36:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff1b7471740 1 -- 172.21.15.172:0/4060347623 wait complete. 2024-04-14T08:36:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.601+0000 7ff1b7471740 20 client.5702 trim_cache size 0 max 16384 2024-04-14T08:36:12.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b7471740 1 Processor -- start 2024-04-14T08:36:12.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b7471740 1 -- start start 2024-04-14T08:36:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b7471740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bea1bb0 0x56055bec7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b7471740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055be9af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b7471740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bee94a0 0x56055be9b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b7471740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1fed00 con 0x56055bec0f30 2024-04-14T08:36:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b7471740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1fddd0 con 0x56055bee94a0 2024-04-14T08:36:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b7471740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ac0b6860 con 0x56055bea1bb0 2024-04-14T08:36:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b1c2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055be9af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b1c2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055be9af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46252/0 (socket says 172.21.15.172:46252) 2024-04-14T08:36:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b1c2c640 1 -- 172.21.15.172:0/2984018011 learned_addr learned my addr 172.21.15.172:0/2984018011 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b2c2e640 1 --2- 172.21.15.172:0/2984018011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bee94a0 0x56055be9b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.615+0000 7ff1b242d640 1 --2- 172.21.15.172:0/2984018011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bea1bb0 0x56055bec7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.616+0000 7ff1b1c2c640 1 -- 172.21.15.172:0/2984018011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bea1bb0 msgr2=0x56055bec7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.616+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/2984018011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bea1bb0 0x56055bec7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.616+0000 7ff1b1c2c640 1 -- 172.21.15.172:0/2984018011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bee94a0 msgr2=0x56055be9b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.616+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/2984018011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bee94a0 0x56055be9b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.616+0000 7ff1b1c2c640 1 -- 172.21.15.172:0/2984018011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055bad90b0 con 0x56055bec0f30 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.616+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/2984018011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055be9af30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff19c011a30 tx=0x7ff19c024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166408b829614774 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.616+0000 7ff18f7fe640 1 -- 172.21.15.172:0/2984018011 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c03a070 con 0x56055bec0f30 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.616+0000 7ff18f7fe640 1 -- 172.21.15.172:0/2984018011 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff19c014650 con 0x56055bec0f30 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.616+0000 7ff18f7fe640 1 -- 172.21.15.172:0/2984018011 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c027040 con 0x56055bec0f30 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.616+0000 7ff1b7471740 1 -- 172.21.15.172:0/2984018011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 msgr2=0x56055be9af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.616+0000 7ff1b7471740 1 --2- 172.21.15.172:0/2984018011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055be9af30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff19c011a30 tx=0x7ff19c024000 comp rx=0 tx=0).stop 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.617+0000 7ff1b7471740 1 -- 172.21.15.172:0/2984018011 shutdown_connections 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.617+0000 7ff1b7471740 1 --2- 172.21.15.172:0/2984018011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bee94a0 0x56055be9b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.617+0000 7ff1b7471740 1 --2- 172.21.15.172:0/2984018011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055be9af30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.617+0000 7ff1b7471740 1 --2- 172.21.15.172:0/2984018011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bea1bb0 0x56055bec7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.617+0000 7ff1b7471740 1 -- 172.21.15.172:0/2984018011 >> 172.21.15.172:0/2984018011 conn(0x56055be9bd90 msgr2=0x56055becdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.617+0000 7ff1b7471740 1 -- 172.21.15.172:0/2984018011 shutdown_connections 2024-04-14T08:36:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.617+0000 7ff1b7471740 1 -- 172.21.15.172:0/2984018011 wait complete. 2024-04-14T08:36:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.617+0000 7ff1b7471740 1 -- start start 2024-04-14T08:36:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.617+0000 7ff1b7471740 1 Processor -- start 2024-04-14T08:36:12.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b7471740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bea1bb0 0x56055bf5d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b7471740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055bf5ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b7471740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055be9af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b7471740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56055becfcd0 con 0x56055bec0f30 2024-04-14T08:36:12.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b7471740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1fd820 con 0x56055bea1bb0 2024-04-14T08:36:12.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b7471740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56055bec1bd0 con 0x56055bee94a0 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b7471740 10 client.0 _ll_register_callbacks cb 0x7ff19c011b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b1c2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055bf5ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b1c2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055bf5ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46258/0 (socket says 172.21.15.172:46258) 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b1c2c640 1 -- 172.21.15.172:0/651454996 learned_addr learned my addr 172.21.15.172:0/651454996 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b242d640 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bea1bb0 0x56055bf5d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b2c2e640 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055be9af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b1c2c640 1 -- 172.21.15.172:0/651454996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 msgr2=0x56055be9af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055be9af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.618+0000 7ff1b1c2c640 1 -- 172.21.15.172:0/651454996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bea1bb0 msgr2=0x56055bf5d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bea1bb0 0x56055bf5d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b1c2c640 1 -- 172.21.15.172:0/651454996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055bf604a0 con 0x56055bec0f30 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b1c2c640 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055bf5ff60 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff19c039040 tx=0x7ff19c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3f6f1bf434426e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff18d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:36:12.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff18d7fa640 1 -- 172.21.15.172:0/651454996 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c027040 con 0x56055bec0f30 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff18d7fa640 1 -- 172.21.15.172:0/651454996 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7ff19c02c440 con 0x56055bec0f30 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b7471740 1 -- 172.21.15.172:0/651454996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff19c014650 con 0x56055bec0f30 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff18d7fa640 1 -- 172.21.15.172:0/651454996 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff19c03a040 con 0x56055bec0f30 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b7471740 10 client.6456 fetch_fsmap learned FSMap version 31 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b7471740 1 -- 172.21.15.172:0/651454996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x56055bee0c80 con 0x56055bec0f30 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff18d7fa640 1 -- 172.21.15.172:0/651454996 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff19c014650 con 0x56055bec0f30 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff18d7fa640 1 -- 172.21.15.172:0/651454996 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7ff19c035050 con 0x56055bec0f30 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b7471740 10 client.6456 fetch_fsmap finished waiting for FSMap version 31 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b7471740 10 client.6456 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b7471740 20 client.6456 populate_metadata read hostname 'smithi172' 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b7471740 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x56055bebdf50 0x56055bec68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b7471740 4 client.6456 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-14T08:36:12.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b7471740 1 -- 172.21.15.172:0/651454996 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x56055bda64a0 con 0x56055bebdf50 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.619+0000 7ff1b242d640 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x56055bebdf50 0x56055bec68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff18d7fa640 10 client.6456 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b242d640 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x56055bebdf50 0x56055bec68d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ebaddd5c1c0465 server_cookie=a4c03193f8efc6f7 in_seq=0 out_seq=0 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff18d7fa640 1 -- 172.21.15.172:0/651454996 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+140 (crc 0 0 0) 0x56055bda64a0 con 0x56055bebdf50 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff18d7fa640 10 client.6456 handle_command_reply: tid=0 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b7471740 1 client.6456 shutdown 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b7471740 1 -- 172.21.15.172:0/651454996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 msgr2=0x56055bf5ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b7471740 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055bf5ff60 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff19c039040 tx=0x7ff19c019000 comp rx=0 tx=0).stop 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b7471740 1 -- 172.21.15.172:0/651454996 shutdown_connections 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b7471740 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x56055bebdf50 0x56055bec68d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b7471740 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56055bee94a0 0x56055be9af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b7471740 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055bec0f30 0x56055bf5ff60 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b7471740 1 --2- 172.21.15.172:0/651454996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56055bea1bb0 0x56055bf5d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff18d7fa640 0 client.6456 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:12.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b7471740 1 -- 172.21.15.172:0/651454996 >> 172.21.15.172:0/651454996 conn(0x56055be9bd90 msgr2=0x56055bea2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b7471740 1 -- 172.21.15.172:0/651454996 shutdown_connections 2024-04-14T08:36:12.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b7471740 1 -- 172.21.15.172:0/651454996 wait complete. 2024-04-14T08:36:12.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.620+0000 7ff1b7471740 20 client.6456 trim_cache size 0 max 16384 2024-04-14T08:36:12.628 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:12.628 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-04-14T08:36:12.628 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:12.628 INFO:teuthology.orchestra.run.smithi172.stdout: "damage_type": "backtrace", 2024-04-14T08:36:12.628 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 855729797, 2024-04-14T08:36:12.628 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 1099511627777, 2024-04-14T08:36:12.629 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/dir_test/file" 2024-04-14T08:36:12.629 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:12.629 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-04-14T08:36:12.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.621+0000 7ff1b4691640 1 -- 172.21.15.172:0/2482841606 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff19006c8f0 msgr2=0x7ff19006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.621+0000 7ff1b4691640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff19006c8f0 0x7ff19006edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009560 tx=0x7ff19c009190 comp rx=0 tx=0).stop 2024-04-14T08:36:12.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.621+0000 7ff1b4691640 1 -- 172.21.15.172:0/2482841606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1ac0cd650 msgr2=0x7ff1ac15f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.621+0000 7ff1b4691640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1ac0cd650 0x7ff1ac15f1a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800caa0 tx=0x7ff1a800cf60 comp rx=0 tx=0).stop 2024-04-14T08:36:12.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.621+0000 7ff1b4691640 1 -- 172.21.15.172:0/2482841606 shutdown_connections 2024-04-14T08:36:12.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.621+0000 7ff1b4691640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff19006c8f0 0x7ff19006edb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.621+0000 7ff1b4691640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1ac15c4e0 0x7ff1ac0b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.621+0000 7ff1b4691640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1ac0d09d0 0x7ff1ac15d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.621+0000 7ff1b4691640 1 --2- 172.21.15.172:0/2482841606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1ac0cd650 0x7ff1ac15f1a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.621+0000 7ff1b4691640 1 -- 172.21.15.172:0/2482841606 >> 172.21.15.172:0/2482841606 conn(0x7ff1ac0d2030 msgr2=0x7ff1ac0cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.621+0000 7ff1b4691640 1 -- 172.21.15.172:0/2482841606 shutdown_connections 2024-04-14T08:36:12.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.621+0000 7ff1b4691640 1 -- 172.21.15.172:0/2482841606 wait complete. 2024-04-14T08:36:12.632 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-14T08:36:12.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.731+0000 7faa9b3c8640 1 Processor -- start 2024-04-14T08:36:12.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.731+0000 7faa9b3c8640 1 -- start start 2024-04-14T08:36:12.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa9b3c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faa9415f400 0x7faa9415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa9b3c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7faa940b1b80 0x7faa940b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa9b3c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9413f600 con 0x7faa9415d9d0 2024-04-14T08:36:12.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa9b3c8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7faa940b2520 con 0x7faa9415f400 2024-04-14T08:36:12.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa9b3c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7faa94140690 con 0x7faa940b1b80 2024-04-14T08:36:12.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa99164640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faa9415f400 0x7faa9415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa98963640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7faa9415d9d0 0x7faa9415d190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48054/0 (socket says 172.21.15.172:48054) 2024-04-14T08:36:12.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa98963640 1 -- 172.21.15.172:0/2112833202 learned_addr learned my addr 172.21.15.172:0/2112833202 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:12.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa99965640 1 --2- 172.21.15.172:0/2112833202 >> v2:172.21.15.177:3300/0 conn(0x7faa940b1b80 0x7faa940b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa99164640 1 -- 172.21.15.172:0/2112833202 >> v2:172.21.15.177:3300/0 conn(0x7faa940b1b80 msgr2=0x7faa940b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:12.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa99164640 1 --2- 172.21.15.172:0/2112833202 >> v2:172.21.15.177:3300/0 conn(0x7faa940b1b80 0x7faa940b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa99164640 1 -- 172.21.15.172:0/2112833202 >> v1:172.21.15.172:6789/0 conn(0x7faa9415d9d0 legacy=0x7faa9415d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa8bfff640 1 -- 172.21.15.172:0/2112833202 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1177915244 0 0) 0x7faa84001110 con 0x7faa9415d9d0 2024-04-14T08:36:12.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.732+0000 7faa99164640 1 -- 172.21.15.172:0/2112833202 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9413fa80 con 0x7faa9415f400 2024-04-14T08:36:12.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.733+0000 7faa99164640 1 --2- 172.21.15.172:0/2112833202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa9415f400 0x7faa9415f7e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7faa9406f330 tx=0x7faa9002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8ee41d360b848c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.733+0000 7faa8bfff640 1 -- 172.21.15.172:0/2112833202 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90035020 con 0x7faa9415f400 2024-04-14T08:36:12.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.733+0000 7faa8bfff640 1 -- 172.21.15.172:0/2112833202 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa90031830 con 0x7faa9415f400 2024-04-14T08:36:12.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.733+0000 7faa8bfff640 1 -- 172.21.15.172:0/2112833202 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90031c00 con 0x7faa9415f400 2024-04-14T08:36:12.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.734+0000 7faa9b3c8640 1 -- 172.21.15.172:0/2112833202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa9415f400 msgr2=0x7faa9415f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.734+0000 7faa9b3c8640 1 --2- 172.21.15.172:0/2112833202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa9415f400 0x7faa9415f7e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7faa9406f330 tx=0x7faa9002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.734+0000 7faa9b3c8640 1 -- 172.21.15.172:0/2112833202 shutdown_connections 2024-04-14T08:36:12.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.734+0000 7faa9b3c8640 1 --2- 172.21.15.172:0/2112833202 >> v2:172.21.15.177:3300/0 conn(0x7faa940b1b80 0x7faa940b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.734+0000 7faa9b3c8640 1 --2- 172.21.15.172:0/2112833202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa9415f400 0x7faa9415f7e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.734+0000 7faa9b3c8640 1 -- 172.21.15.172:0/2112833202 >> 172.21.15.172:0/2112833202 conn(0x7faa9406ec30 msgr2=0x7faa9406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.734+0000 7faa9b3c8640 1 -- 172.21.15.172:0/2112833202 shutdown_connections 2024-04-14T08:36:12.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.734+0000 7faa9b3c8640 1 -- 172.21.15.172:0/2112833202 wait complete. 2024-04-14T08:36:12.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.735+0000 7faa9b3c8640 1 Processor -- start 2024-04-14T08:36:12.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.735+0000 7faa9b3c8640 1 -- start start 2024-04-14T08:36:12.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa9b3c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa940b1b80 0x7faa941365e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa9b3c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa9415d9d0 0x7faa94153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa9b3c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa9415f400 0x7faa941567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa9b3c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faa94140320 con 0x7faa940b1b80 2024-04-14T08:36:12.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa9b3c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faa94140040 con 0x7faa9415d9d0 2024-04-14T08:36:12.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa9b3c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7faa9415de60 con 0x7faa9415f400 2024-04-14T08:36:12.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa99164640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa940b1b80 0x7faa941365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa99164640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa940b1b80 0x7faa941365e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46282/0 (socket says 172.21.15.172:46282) 2024-04-14T08:36:12.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa99164640 1 -- 172.21.15.172:0/1735041789 learned_addr learned my addr 172.21.15.172:0/1735041789 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:12.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa99965640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa9415f400 0x7faa941567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa98963640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa9415d9d0 0x7faa94153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa99164640 1 -- 172.21.15.172:0/1735041789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa9415f400 msgr2=0x7faa941567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa99164640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa9415f400 0x7faa941567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa99164640 1 -- 172.21.15.172:0/1735041789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa9415d9d0 msgr2=0x7faa94153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa99164640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa9415d9d0 0x7faa94153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa99164640 1 -- 172.21.15.172:0/1735041789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9413cf20 con 0x7faa940b1b80 2024-04-14T08:36:12.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa99164640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa940b1b80 0x7faa941365e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faa90000c00 tx=0x7faa900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa89ffb640 1 -- 172.21.15.172:0/1735041789 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90035040 con 0x7faa940b1b80 2024-04-14T08:36:12.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa89ffb640 1 -- 172.21.15.172:0/1735041789 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa90004230 con 0x7faa940b1b80 2024-04-14T08:36:12.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa9b3c8640 1 -- 172.21.15.172:0/1735041789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9413c160 con 0x7faa940b1b80 2024-04-14T08:36:12.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.736+0000 7faa89ffb640 1 -- 172.21.15.172:0/1735041789 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9003a640 con 0x7faa940b1b80 2024-04-14T08:36:12.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.737+0000 7faa9b3c8640 1 -- 172.21.15.172:0/1735041789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9413c720 con 0x7faa940b1b80 2024-04-14T08:36:12.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.737+0000 7faa9b3c8640 1 -- 172.21.15.172:0/1735041789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa94068520 con 0x7faa940b1b80 2024-04-14T08:36:12.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.738+0000 7faa89ffb640 1 -- 172.21.15.172:0/1735041789 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa90038020 con 0x7faa940b1b80 2024-04-14T08:36:12.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.738+0000 7faa89ffb640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faa7806c8f0 0x7faa7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:12.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.738+0000 7faa89ffb640 1 -- 172.21.15.172:0/1735041789 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7faa900ad7e0 con 0x7faa940b1b80 2024-04-14T08:36:12.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.738+0000 7faa98963640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faa7806c8f0 0x7faa7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:12.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.739+0000 7faa98963640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faa7806c8f0 0x7faa7806edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faa94149f60 tx=0x7faa84007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:12.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.743+0000 7faa89ffb640 1 -- 172.21.15.172:0/1735041789 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7faa9007b4c0 con 0x7faa940b1b80 2024-04-14T08:36:12.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.958+0000 7faa9b3c8640 1 -- 172.21.15.172:0/1735041789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7faa9406c2c0 con 0x7faa940b1b80 2024-04-14T08:36:12.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.958+0000 7faa89ffb640 1 -- 172.21.15.172:0/1735041789 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7faa9007f7c0 con 0x7faa940b1b80 2024-04-14T08:36:12.959 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:12.959 INFO:teuthology.orchestra.run.smithi172.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-14T08:36:12.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.960+0000 7faa9b3c8640 1 -- 172.21.15.172:0/1735041789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faa7806c8f0 msgr2=0x7faa7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.960+0000 7faa9b3c8640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faa7806c8f0 0x7faa7806edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faa94149f60 tx=0x7faa84007b50 comp rx=0 tx=0).stop 2024-04-14T08:36:12.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.960+0000 7faa9b3c8640 1 -- 172.21.15.172:0/1735041789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa940b1b80 msgr2=0x7faa941365e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:12.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.960+0000 7faa9b3c8640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa940b1b80 0x7faa941365e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faa90000c00 tx=0x7faa900046c0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.960+0000 7faa9b3c8640 1 -- 172.21.15.172:0/1735041789 shutdown_connections 2024-04-14T08:36:12.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.960+0000 7faa9b3c8640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faa7806c8f0 0x7faa7806edb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.960+0000 7faa9b3c8640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa9415f400 0x7faa941567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.960+0000 7faa9b3c8640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa9415d9d0 0x7faa94153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.960+0000 7faa9b3c8640 1 --2- 172.21.15.172:0/1735041789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa940b1b80 0x7faa941365e0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:12.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.960+0000 7faa9b3c8640 1 -- 172.21.15.172:0/1735041789 >> 172.21.15.172:0/1735041789 conn(0x7faa9406ec30 msgr2=0x7faa940b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:12.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.960+0000 7faa9b3c8640 1 -- 172.21.15.172:0/1735041789 shutdown_connections 2024-04-14T08:36:12.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:12.960+0000 7faa9b3c8640 1 -- 172.21.15.172:0/1735041789 wait complete. 2024-04-14T08:36:12.971 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-14T08:36:12.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:36:12.972 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-14T08:36:13.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.077+0000 7fd17595e640 1 Processor -- start 2024-04-14T08:36:13.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.077+0000 7fd17595e640 1 -- start start 2024-04-14T08:36:13.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.077+0000 7fd17595e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd17009aa70 0x7fd17009ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.077+0000 7fd17595e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd1700b1100 0x7fd170162120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.077+0000 7fd17595e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd17009b390 con 0x7fd17006f4d0 2024-04-14T08:36:13.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.077+0000 7fd17595e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd1700b4670 con 0x7fd1700b1100 2024-04-14T08:36:13.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.077+0000 7fd17595e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd1700b4950 con 0x7fd17009aa70 2024-04-14T08:36:13.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.078+0000 7fd16effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd17009aa70 0x7fd17009ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.078+0000 7fd16e7fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd1700b1100 0x7fd170162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.078+0000 7fd16f7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd17006f4d0 0x7fd170162660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:48060/0 (socket says 172.21.15.172:48060) 2024-04-14T08:36:13.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.078+0000 7fd16f7fe640 1 -- 172.21.15.172:0/2195516090 learned_addr learned my addr 172.21.15.172:0/2195516090 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:13.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.078+0000 7fd16e7fc640 1 -- 172.21.15.172:0/2195516090 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd17009aa70 msgr2=0x7fd17009ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.078+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/2195516090 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd17009aa70 0x7fd17009ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.078+0000 7fd16e7fc640 1 -- 172.21.15.172:0/2195516090 >> v1:172.21.15.172:6789/0 conn(0x7fd17006f4d0 legacy=0x7fd170162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.078+0000 7fd16e7fc640 1 -- 172.21.15.172:0/2195516090 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17015eee0 con 0x7fd1700b1100 2024-04-14T08:36:13.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.078+0000 7fd16dffb640 1 -- 172.21.15.172:0/2195516090 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4157054880 0 0) 0x7fd17009b390 con 0x7fd17006f4d0 2024-04-14T08:36:13.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.078+0000 7fd16effd640 1 --2- 172.21.15.172:0/2195516090 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd17009aa70 0x7fd17009ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:13.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.079+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/2195516090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1700b1100 0x7fd170162120 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd1640097a0 tx=0x7fd16402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ddd8c90e10e2c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.079+0000 7fd16dffb640 1 -- 172.21.15.172:0/2195516090 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164031780 con 0x7fd1700b1100 2024-04-14T08:36:13.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.079+0000 7fd16dffb640 1 -- 172.21.15.172:0/2195516090 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd164037070 con 0x7fd1700b1100 2024-04-14T08:36:13.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.079+0000 7fd16dffb640 1 -- 172.21.15.172:0/2195516090 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164031b40 con 0x7fd1700b1100 2024-04-14T08:36:13.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.079+0000 7fd17595e640 1 -- 172.21.15.172:0/2195516090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1700b1100 msgr2=0x7fd170162120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.079+0000 7fd17595e640 1 --2- 172.21.15.172:0/2195516090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1700b1100 0x7fd170162120 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd1640097a0 tx=0x7fd16402b820 comp rx=0 tx=0).stop 2024-04-14T08:36:13.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.079+0000 7fd17595e640 1 -- 172.21.15.172:0/2195516090 shutdown_connections 2024-04-14T08:36:13.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.079+0000 7fd17595e640 1 --2- 172.21.15.172:0/2195516090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1700b1100 0x7fd170162120 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.079+0000 7fd17595e640 1 --2- 172.21.15.172:0/2195516090 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd17009aa70 0x7fd17009ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.079+0000 7fd17595e640 1 -- 172.21.15.172:0/2195516090 >> 172.21.15.172:0/2195516090 conn(0x7fd1700b3b50 msgr2=0x7fd1700b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:13.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.079+0000 7fd17595e640 1 -- 172.21.15.172:0/2195516090 shutdown_connections 2024-04-14T08:36:13.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.079+0000 7fd17595e640 1 -- 172.21.15.172:0/2195516090 wait complete. 2024-04-14T08:36:13.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.080+0000 7fd17595e640 1 Processor -- start 2024-04-14T08:36:13.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.081+0000 7fd17595e640 1 -- start start 2024-04-14T08:36:13.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.081+0000 7fd17595e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17006f4d0 0x7fd1700b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.081+0000 7fd17595e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd17009aa70 0x7fd1700c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.081+0000 7fd17595e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1700b1100 0x7fd1700c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.081+0000 7fd17595e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd17015eb10 con 0x7fd17006f4d0 2024-04-14T08:36:13.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.081+0000 7fd17595e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1700d11e0 con 0x7fd17009aa70 2024-04-14T08:36:13.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.081+0000 7fd17595e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd17015ec90 con 0x7fd1700b1100 2024-04-14T08:36:13.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.081+0000 7fd16effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17006f4d0 0x7fd1700b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.081+0000 7fd16effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17006f4d0 0x7fd1700b4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46304/0 (socket says 172.21.15.172:46304) 2024-04-14T08:36:13.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.081+0000 7fd16effd640 1 -- 172.21.15.172:0/2796052929 learned_addr learned my addr 172.21.15.172:0/2796052929 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:13.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.081+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd17009aa70 0x7fd1700c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.081+0000 7fd16f7fe640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1700b1100 0x7fd1700c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.082+0000 7fd16effd640 1 -- 172.21.15.172:0/2796052929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1700b1100 msgr2=0x7fd1700c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.082+0000 7fd16effd640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1700b1100 0x7fd1700c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.082+0000 7fd16effd640 1 -- 172.21.15.172:0/2796052929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd17009aa70 msgr2=0x7fd1700c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.082+0000 7fd16effd640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd17009aa70 0x7fd1700c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.082+0000 7fd16effd640 1 -- 172.21.15.172:0/2796052929 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1700c8620 con 0x7fd17006f4d0 2024-04-14T08:36:13.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.082+0000 7fd16f7fe640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1700b1100 0x7fd1700c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:36:13.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.082+0000 7fd16effd640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17006f4d0 0x7fd1700b4c50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd16000e960 tx=0x7fd16000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.082+0000 7fd157fff640 1 -- 172.21.15.172:0/2796052929 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160017070 con 0x7fd17006f4d0 2024-04-14T08:36:13.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.082+0000 7fd157fff640 1 -- 172.21.15.172:0/2796052929 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd16000ca00 con 0x7fd17006f4d0 2024-04-14T08:36:13.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.082+0000 7fd157fff640 1 -- 172.21.15.172:0/2796052929 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16000cdd0 con 0x7fd17006f4d0 2024-04-14T08:36:13.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.082+0000 7fd17595e640 1 -- 172.21.15.172:0/2796052929 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1700c6de0 con 0x7fd17006f4d0 2024-04-14T08:36:13.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.082+0000 7fd17595e640 1 -- 172.21.15.172:0/2796052929 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1700c73a0 con 0x7fd17006f4d0 2024-04-14T08:36:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.084+0000 7fd157fff640 1 -- 172.21.15.172:0/2796052929 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd16001a070 con 0x7fd17006f4d0 2024-04-14T08:36:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.084+0000 7fd157fff640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd15c06c8f0 0x7fd15c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.084+0000 7fd157fff640 1 -- 172.21.15.172:0/2796052929 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fd16008bea0 con 0x7fd17006f4d0 2024-04-14T08:36:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.084+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd15c06c8f0 0x7fd15c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.084+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd15c06c8f0 0x7fd15c06edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd164036000 tx=0x7fd164008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.085+0000 7fd178832740 1 Processor -- start 2024-04-14T08:36:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.085+0000 7fd178832740 1 -- start start 2024-04-14T08:36:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd178832740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03130 0x55f3b9bd60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd178832740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 0x55f3b9c05ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd178832740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9bd6610 0x55f3b9c06400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd178832740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f3b9b3f540 con 0x55f3b9c03510 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd178832740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f3b9ae78a0 con 0x55f3b9c03130 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd178832740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f3b9b3b4f0 con 0x55f3b9bd6610 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd16e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 0x55f3b9c05ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd16e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 0x55f3b9c05ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46316/0 (socket says 172.21.15.172:46316) 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd16e7fc640 1 -- 172.21.15.172:0/4071853573 learned_addr learned my addr 172.21.15.172:0/4071853573 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd16f7fe640 1 --2- 172.21.15.172:0/4071853573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9bd6610 0x55f3b9c06400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd16effd640 1 --2- 172.21.15.172:0/4071853573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03130 0x55f3b9bd60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd16e7fc640 1 -- 172.21.15.172:0/4071853573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9bd6610 msgr2=0x55f3b9c06400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/4071853573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9bd6610 0x55f3b9c06400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd16e7fc640 1 -- 172.21.15.172:0/4071853573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03130 msgr2=0x55f3b9bd60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/4071853573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03130 0x55f3b9bd60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd16e7fc640 1 -- 172.21.15.172:0/4071853573 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3b991f940 con 0x55f3b9c03510 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd16effd640 1 --2- 172.21.15.172:0/4071853573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03130 0x55f3b9bd60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:36:13.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/4071853573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 0x55f3b9c05ec0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd164009890 tx=0x7fd164030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5344c4404278e4e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd1557fa640 1 -- 172.21.15.172:0/4071853573 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164031580 con 0x55f3b9c03510 2024-04-14T08:36:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd1557fa640 1 -- 172.21.15.172:0/4071853573 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd164037040 con 0x55f3b9c03510 2024-04-14T08:36:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.086+0000 7fd1557fa640 1 -- 172.21.15.172:0/4071853573 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164031940 con 0x55f3b9c03510 2024-04-14T08:36:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.087+0000 7fd178832740 1 -- 172.21.15.172:0/4071853573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 msgr2=0x55f3b9c05ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.087+0000 7fd178832740 1 --2- 172.21.15.172:0/4071853573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 0x55f3b9c05ec0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd164009890 tx=0x7fd164030000 comp rx=0 tx=0).stop 2024-04-14T08:36:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.087+0000 7fd178832740 1 -- 172.21.15.172:0/4071853573 shutdown_connections 2024-04-14T08:36:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.087+0000 7fd178832740 1 --2- 172.21.15.172:0/4071853573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9bd6610 0x55f3b9c06400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.087+0000 7fd178832740 1 --2- 172.21.15.172:0/4071853573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 0x55f3b9c05ec0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.087+0000 7fd178832740 1 --2- 172.21.15.172:0/4071853573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03130 0x55f3b9bd60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.087+0000 7fd178832740 1 -- 172.21.15.172:0/4071853573 >> 172.21.15.172:0/4071853573 conn(0x55f3b9bd8340 msgr2=0x55f3b9c20ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.087+0000 7fd178832740 1 -- 172.21.15.172:0/4071853573 shutdown_connections 2024-04-14T08:36:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.087+0000 7fd178832740 1 -- 172.21.15.172:0/4071853573 wait complete. 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.088+0000 7fd178832740 1 -- start start 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.088+0000 7fd178832740 1 Processor -- start 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd178832740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9c03130 0x55f3b9c09540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd178832740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03510 0x55f3b9bf0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd178832740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 0x55f3b9c07f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd178832740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f3b9bda8e0 con 0x55f3b9bf0f80 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd178832740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f3b9afac60 con 0x55f3b9c03510 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd178832740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f3b9bd9cd0 con 0x55f3b9c03130 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd178832740 10 client.0 _ll_register_callbacks cb 0x55f3b9852d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 0x55f3b9c07f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16e7fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03510 0x55f3b9bf0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9c03130 0x55f3b9c09540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 0x55f3b9c07f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46328/0 (socket says 172.21.15.172:46328) 2024-04-14T08:36:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16f7fe640 1 -- 172.21.15.172:0/4155898318 learned_addr learned my addr 172.21.15.172:0/4155898318 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16e7fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03510 0x55f3b9bf0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:51772/0 (socket says 172.21.15.172:51772) 2024-04-14T08:36:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16e7fc640 1 -- 172.21.15.172:0/4155898318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9c03130 msgr2=0x55f3b9c09540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9c03130 0x55f3b9c09540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16e7fc640 1 -- 172.21.15.172:0/4155898318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 msgr2=0x55f3b9c07f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 0x55f3b9c07f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16e7fc640 1 -- 172.21.15.172:0/4155898318 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3b99800d0 con 0x55f3b9c03510 2024-04-14T08:36:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16effd640 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9c03130 0x55f3b9c09540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16f7fe640 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 0x55f3b9c07f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03510 0x55f3b9bf0a40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd16404a000 tx=0x7fd164004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d20f7224a008431 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd14e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-04-14T08:36:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd14e7fc640 1 -- 172.21.15.172:0/4155898318 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164059070 con 0x55f3b9c03510 2024-04-14T08:36:13.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd14e7fc640 1 -- 172.21.15.172:0/4155898318 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fd164031da0 con 0x55f3b9c03510 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.089+0000 7fd178832740 1 -- 172.21.15.172:0/4155898318 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd164037040 con 0x55f3b9c03510 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd14e7fc640 1 -- 172.21.15.172:0/4155898318 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd164005990 con 0x55f3b9c03510 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd178832740 10 client.6238 fetch_fsmap learned FSMap version 31 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd178832740 1 -- 172.21.15.172:0/4155898318 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x55f3b9813230 con 0x55f3b9c03510 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd14e7fc640 1 -- 172.21.15.172:0/4155898318 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd164037040 con 0x55f3b9c03510 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd14e7fc640 1 -- 172.21.15.172:0/4155898318 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7fd164005b30 con 0x55f3b9c03510 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd178832740 10 client.6238 fetch_fsmap finished waiting for FSMap version 31 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd178832740 10 client.6238 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd178832740 20 client.6238 populate_metadata read hostname 'smithi172' 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd178832740 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55f3b9c08910 0x55f3b9bf2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd178832740 4 client.6238 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd178832740 1 -- 172.21.15.172:0/4155898318 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3b9b4ca80 con 0x55f3b9c08910 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd16effd640 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55f3b9c08910 0x55f3b9bf2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd14e7fc640 10 client.6238 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.090+0000 7fd16effd640 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55f3b9c08910 0x55f3b9bf2640 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f83c4dc09d525d0 server_cookie=f860b843ca43e091 in_seq=0 out_seq=0 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd14e7fc640 1 -- 172.21.15.172:0/4155898318 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3b9b4ca80 con 0x55f3b9c08910 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd14e7fc640 10 client.6238 handle_command_reply: tid=0 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd178832740 1 client.6238 shutdown 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd178832740 1 -- 172.21.15.172:0/4155898318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03510 msgr2=0x55f3b9bf0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd178832740 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03510 0x55f3b9bf0a40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd16404a000 tx=0x7fd164004730 comp rx=0 tx=0).stop 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd178832740 1 -- 172.21.15.172:0/4155898318 shutdown_connections 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd178832740 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55f3b9c08910 0x55f3b9bf2640 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd178832740 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 0x55f3b9c07f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd178832740 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03510 0x55f3b9bf0a40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd178832740 1 --2- 172.21.15.172:0/4155898318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9c03130 0x55f3b9c09540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd14e7fc640 0 client.6238 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd178832740 1 -- 172.21.15.172:0/4155898318 >> 172.21.15.172:0/4155898318 conn(0x55f3b9bd6610 msgr2=0x55f3b9bd57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:13.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd178832740 1 -- 172.21.15.172:0/4155898318 shutdown_connections 2024-04-14T08:36:13.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd178832740 1 -- 172.21.15.172:0/4155898318 wait complete. 2024-04-14T08:36:13.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.091+0000 7fd178832740 20 client.6238 trim_cache size 0 max 16384 2024-04-14T08:36:13.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.109+0000 7fd178832740 1 Processor -- start 2024-04-14T08:36:13.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.109+0000 7fd178832740 1 -- start start 2024-04-14T08:36:13.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.109+0000 7fd178832740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9bf0f80 0x55f3b9c1ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.109+0000 7fd178832740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03130 0x55f3b9c05940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.109+0000 7fd178832740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 0x55f3b9c05e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.109+0000 7fd178832740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd170200f70 con 0x55f3b9c03510 2024-04-14T08:36:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.109+0000 7fd178832740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701fff50 con 0x55f3b9c03130 2024-04-14T08:36:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.109+0000 7fd178832740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd1701ff2b0 con 0x55f3b9bf0f80 2024-04-14T08:36:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 0x55f3b9c05e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd16e7fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03130 0x55f3b9c05940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd16f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 0x55f3b9c05e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46336/0 (socket says 172.21.15.172:46336) 2024-04-14T08:36:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd16f7fe640 1 -- 172.21.15.172:0/2824737050 learned_addr learned my addr 172.21.15.172:0/2824737050 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd16effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9bf0f80 0x55f3b9c1ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd16f7fe640 1 -- 172.21.15.172:0/2824737050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9bf0f80 msgr2=0x55f3b9c1ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd16f7fe640 1 --2- 172.21.15.172:0/2824737050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9bf0f80 0x55f3b9c1ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd16f7fe640 1 -- 172.21.15.172:0/2824737050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03130 msgr2=0x55f3b9c05940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd16f7fe640 1 --2- 172.21.15.172:0/2824737050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03130 0x55f3b9c05940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd16f7fe640 1 -- 172.21.15.172:0/2824737050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3b9c08cf0 con 0x55f3b9c03510 2024-04-14T08:36:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/2824737050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03130 0x55f3b9c05940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd16f7fe640 1 --2- 172.21.15.172:0/2824737050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 0x55f3b9c05e80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd168007980 tx=0x7fd168014710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297d7cc4b51f4c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd154ff9640 1 -- 172.21.15.172:0/2824737050 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168014e60 con 0x55f3b9c03510 2024-04-14T08:36:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd154ff9640 1 -- 172.21.15.172:0/2824737050 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd168016630 con 0x55f3b9c03510 2024-04-14T08:36:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.110+0000 7fd154ff9640 1 -- 172.21.15.172:0/2824737050 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1680169e0 con 0x55f3b9c03510 2024-04-14T08:36:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.111+0000 7fd178832740 1 -- 172.21.15.172:0/2824737050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 msgr2=0x55f3b9c05e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.111+0000 7fd178832740 1 --2- 172.21.15.172:0/2824737050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 0x55f3b9c05e80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd168007980 tx=0x7fd168014710 comp rx=0 tx=0).stop 2024-04-14T08:36:13.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.111+0000 7fd178832740 1 -- 172.21.15.172:0/2824737050 shutdown_connections 2024-04-14T08:36:13.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.111+0000 7fd178832740 1 --2- 172.21.15.172:0/2824737050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9c03510 0x55f3b9c05e80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.111+0000 7fd178832740 1 --2- 172.21.15.172:0/2824737050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03130 0x55f3b9c05940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.111+0000 7fd178832740 1 --2- 172.21.15.172:0/2824737050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9bf0f80 0x55f3b9c1ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.111+0000 7fd178832740 1 -- 172.21.15.172:0/2824737050 >> 172.21.15.172:0/2824737050 conn(0x55f3b9bd6610 msgr2=0x55f3b9c1a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:13.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.111+0000 7fd178832740 1 -- 172.21.15.172:0/2824737050 shutdown_connections 2024-04-14T08:36:13.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.111+0000 7fd178832740 1 -- 172.21.15.172:0/2824737050 wait complete. 2024-04-14T08:36:13.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.111+0000 7fd178832740 1 -- start start 2024-04-14T08:36:13.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.111+0000 7fd178832740 1 Processor -- start 2024-04-14T08:36:13.116 INFO:tasks.ceph.mds.d.smithi177.stderr:2024-04-14T08:36:13.114+0000 7f9d62fe9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) caps={6115=pAsLsXsFs/-@2},l=6115(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x555d1cc8a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-14T08:36:13.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.112+0000 7fd178832740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 0x55f3b9c16850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.112+0000 7fd178832740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9c03130 0x55f3b9c301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.112+0000 7fd178832740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03510 0x55f3b9c1ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.112+0000 7fd178832740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701ff900 con 0x55f3b9bf0f80 2024-04-14T08:36:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.112+0000 7fd178832740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f3b9bda6e0 con 0x55f3b9c03510 2024-04-14T08:36:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.112+0000 7fd178832740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f3b9c172d0 con 0x55f3b9c03130 2024-04-14T08:36:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.112+0000 7fd178832740 10 client.0 _ll_register_callbacks cb 0x7fd168005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:36:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.112+0000 7fd16effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 0x55f3b9c16850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.112+0000 7fd16effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 0x55f3b9c16850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46344/0 (socket says 172.21.15.172:46344) 2024-04-14T08:36:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.112+0000 7fd16effd640 1 -- 172.21.15.172:0/2755126741 learned_addr learned my addr 172.21.15.172:0/2755126741 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.112+0000 7fd16f7fe640 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03510 0x55f3b9c1ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.112+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9c03130 0x55f3b9c301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd16effd640 1 -- 172.21.15.172:0/2755126741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9c03130 msgr2=0x55f3b9c301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd16effd640 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9c03130 0x55f3b9c301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd16effd640 1 -- 172.21.15.172:0/2755126741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03510 msgr2=0x55f3b9c1ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd16effd640 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03510 0x55f3b9c1ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd16effd640 1 -- 172.21.15.172:0/2755126741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3b9c33b80 con 0x55f3b9bf0f80 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd16effd640 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 0x55f3b9c16850 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd16000c520 tx=0x7fd160060040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6df6d1f31ae47 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd14e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd14e7fc640 1 -- 172.21.15.172:0/2755126741 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160021040 con 0x55f3b9bf0f80 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd14e7fc640 1 -- 172.21.15.172:0/2755126741 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fd1600600b0 con 0x55f3b9bf0f80 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd178832740 1 -- 172.21.15.172:0/2755126741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd168016630 con 0x55f3b9bf0f80 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd14e7fc640 1 -- 172.21.15.172:0/2755126741 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd1600602b0 con 0x55f3b9bf0f80 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd178832740 10 client.6486 fetch_fsmap learned FSMap version 31 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd178832740 1 -- 172.21.15.172:0/2755126741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x55f3b9c17010 con 0x55f3b9bf0f80 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd14e7fc640 1 -- 172.21.15.172:0/2755126741 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd168016630 con 0x55f3b9bf0f80 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd14e7fc640 1 -- 172.21.15.172:0/2755126741 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7fd160083ba0 con 0x55f3b9bf0f80 2024-04-14T08:36:13.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd178832740 10 client.6486 fetch_fsmap finished waiting for FSMap version 31 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd178832740 10 client.6486 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd178832740 20 client.6486 populate_metadata read hostname 'smithi172' 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd178832740 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55f3b9c83bf0 0x55f3b9c1b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd178832740 4 client.6486 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd178832740 1 -- 172.21.15.172:0/2755126741 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x55f3b9b46dd0 con 0x55f3b9c83bf0 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.113+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55f3b9c83bf0 0x55f3b9c1b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd14e7fc640 10 client.6486 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd16e7fc640 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55f3b9c83bf0 0x55f3b9c1b0b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74cf55965cbc8111 server_cookie=e050c3422b47310 in_seq=0 out_seq=0 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd14e7fc640 1 -- 172.21.15.172:0/2755126741 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f3b9b46dd0 con 0x55f3b9c83bf0 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd14e7fc640 10 client.6486 handle_command_reply: tid=0 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd178832740 1 client.6486 shutdown 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd178832740 1 -- 172.21.15.172:0/2755126741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 msgr2=0x55f3b9c16850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd178832740 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 0x55f3b9c16850 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd16000c520 tx=0x7fd160060040 comp rx=0 tx=0).stop 2024-04-14T08:36:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd178832740 1 -- 172.21.15.172:0/2755126741 shutdown_connections 2024-04-14T08:36:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd178832740 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55f3b9c83bf0 0x55f3b9c1b0b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd178832740 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f3b9c03510 0x55f3b9c1ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd14e7fc640 0 client.6486 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd178832740 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f3b9c03130 0x55f3b9c301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd178832740 1 --2- 172.21.15.172:0/2755126741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3b9bf0f80 0x55f3b9c16850 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd178832740 1 -- 172.21.15.172:0/2755126741 >> 172.21.15.172:0/2755126741 conn(0x55f3b9bd6610 msgr2=0x55f3b9c214a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd178832740 1 -- 172.21.15.172:0/2755126741 shutdown_connections 2024-04-14T08:36:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.114+0000 7fd178832740 1 -- 172.21.15.172:0/2755126741 wait complete. 2024-04-14T08:36:13.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd178832740 20 client.6486 trim_cache size 0 max 16384 2024-04-14T08:36:13.123 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:36:13.124 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-04-14T08:36:13.124 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "f12e16de-33de-4557-afe3-dca0826a9814", 2024-04-14T08:36:13.124 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-04-14T08:36:13.124 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:36:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd17595e640 1 -- 172.21.15.172:0/2796052929 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd15c06c8f0 msgr2=0x7fd15c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd17595e640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd15c06c8f0 0x7fd15c06edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd164036000 tx=0x7fd164008030 comp rx=0 tx=0).stop 2024-04-14T08:36:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd17595e640 1 -- 172.21.15.172:0/2796052929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17006f4d0 msgr2=0x7fd1700b4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd17595e640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17006f4d0 0x7fd1700b4c50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd16000e960 tx=0x7fd16000ee20 comp rx=0 tx=0).stop 2024-04-14T08:36:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd17595e640 1 -- 172.21.15.172:0/2796052929 shutdown_connections 2024-04-14T08:36:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd17595e640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd15c06c8f0 0x7fd15c06edb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd17595e640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1700b1100 0x7fd1700c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd17595e640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd17009aa70 0x7fd1700c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd17595e640 1 --2- 172.21.15.172:0/2796052929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17006f4d0 0x7fd1700b4c50 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd17595e640 1 -- 172.21.15.172:0/2796052929 >> 172.21.15.172:0/2796052929 conn(0x7fd1700b3b50 msgr2=0x7fd1701567f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd17595e640 1 -- 172.21.15.172:0/2796052929 shutdown_connections 2024-04-14T08:36:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.115+0000 7fd17595e640 1 -- 172.21.15.172:0/2796052929 wait complete. 2024-04-14T08:36:13.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:36:13.127 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-14T08:36:13.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.230+0000 7f2547060640 1 Processor -- start 2024-04-14T08:36:13.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.230+0000 7f2547060640 1 -- start start 2024-04-14T08:36:13.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2547060640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f25400d0cc0 0x7f25400d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2547060640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f25400d49c0 0x7f25400d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2547060640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2540160f30 0x7f25401613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2547060640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f25400ceec0 con 0x7f25400d49c0 2024-04-14T08:36:13.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2547060640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f25400cfd00 con 0x7f25400d0cc0 2024-04-14T08:36:13.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2547060640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f25400d15e0 con 0x7f2540160f30 2024-04-14T08:36:13.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2537fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f25400d49c0 0x7f25400d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2537fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f25400d49c0 0x7f25400d4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46350/0 (socket says 172.21.15.172:46350) 2024-04-14T08:36:13.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2537fff640 1 -- 172.21.15.172:0/1461518621 learned_addr learned my addr 172.21.15.172:0/1461518621 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:13.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f25455fd640 1 --2- 172.21.15.172:0/1461518621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2540160f30 0x7f25401613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2544dfc640 1 --2- 172.21.15.172:0/1461518621 >> v2:172.21.15.177:3300/0 conn(0x7f25400d0cc0 0x7f25400d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2537fff640 1 -- 172.21.15.172:0/1461518621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2540160f30 msgr2=0x7f25401613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2537fff640 1 --2- 172.21.15.172:0/1461518621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2540160f30 0x7f25401613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2537fff640 1 -- 172.21.15.172:0/1461518621 >> v2:172.21.15.177:3300/0 conn(0x7f25400d0cc0 msgr2=0x7f25400d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2537fff640 1 --2- 172.21.15.172:0/1461518621 >> v2:172.21.15.177:3300/0 conn(0x7f25400d0cc0 0x7f25400d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.231+0000 7f2537fff640 1 -- 172.21.15.172:0/1461518621 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25400cf0f0 con 0x7f25400d49c0 2024-04-14T08:36:13.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.232+0000 7f2537fff640 1 --2- 172.21.15.172:0/1461518621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25400d49c0 0x7f25400d4e50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f254008e7b0 tx=0x7f253002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874c66cfb4b43cce server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.232+0000 7f25377fe640 1 -- 172.21.15.172:0/1461518621 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530035020 con 0x7f25400d49c0 2024-04-14T08:36:13.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.232+0000 7f25377fe640 1 -- 172.21.15.172:0/1461518621 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2530031870 con 0x7f25400d49c0 2024-04-14T08:36:13.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.232+0000 7f25377fe640 1 -- 172.21.15.172:0/1461518621 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530031c40 con 0x7f25400d49c0 2024-04-14T08:36:13.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.233+0000 7f2547060640 1 -- 172.21.15.172:0/1461518621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25400d49c0 msgr2=0x7f25400d4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.233+0000 7f2547060640 1 --2- 172.21.15.172:0/1461518621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25400d49c0 0x7f25400d4e50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f254008e7b0 tx=0x7f253002b910 comp rx=0 tx=0).stop 2024-04-14T08:36:13.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.233+0000 7f2547060640 1 -- 172.21.15.172:0/1461518621 shutdown_connections 2024-04-14T08:36:13.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.233+0000 7f2547060640 1 --2- 172.21.15.172:0/1461518621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2540160f30 0x7f25401613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.233+0000 7f2547060640 1 --2- 172.21.15.172:0/1461518621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25400d49c0 0x7f25400d4e50 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.233+0000 7f2547060640 1 --2- 172.21.15.172:0/1461518621 >> v2:172.21.15.177:3300/0 conn(0x7f25400d0cc0 0x7f25400d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.233+0000 7f2547060640 1 -- 172.21.15.172:0/1461518621 >> 172.21.15.172:0/1461518621 conn(0x7f254009d580 msgr2=0x7f25400d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:13.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.233+0000 7f2547060640 1 -- 172.21.15.172:0/1461518621 shutdown_connections 2024-04-14T08:36:13.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.233+0000 7f2547060640 1 -- 172.21.15.172:0/1461518621 wait complete. 2024-04-14T08:36:13.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.234+0000 7f2547060640 1 Processor -- start 2024-04-14T08:36:13.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.234+0000 7f2547060640 1 -- start start 2024-04-14T08:36:13.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f2547060640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25400d0cc0 0x7f25400a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f2547060640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25400d49c0 0x7f2540153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f2547060640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2540160f30 0x7f25400b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f2547060640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f25400cf990 con 0x7f2540160f30 2024-04-14T08:36:13.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f2547060640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f25400cf6b0 con 0x7f25400d0cc0 2024-04-14T08:36:13.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f2547060640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f25400d32b0 con 0x7f25400d49c0 2024-04-14T08:36:13.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f25455fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2540160f30 0x7f25400b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f25455fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2540160f30 0x7f25400b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46356/0 (socket says 172.21.15.172:46356) 2024-04-14T08:36:13.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f25455fd640 1 -- 172.21.15.172:0/4206259974 learned_addr learned my addr 172.21.15.172:0/4206259974 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:13.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f2544dfc640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25400d0cc0 0x7f25400a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f2537fff640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25400d49c0 0x7f2540153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f25455fd640 1 -- 172.21.15.172:0/4206259974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25400d49c0 msgr2=0x7f2540153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f25455fd640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25400d49c0 0x7f2540153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f25455fd640 1 -- 172.21.15.172:0/4206259974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25400d0cc0 msgr2=0x7f25400a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f25455fd640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25400d0cc0 0x7f25400a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.235+0000 7f25455fd640 1 -- 172.21.15.172:0/4206259974 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25400a0b90 con 0x7f2540160f30 2024-04-14T08:36:13.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.236+0000 7f25455fd640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2540160f30 0x7f25400b18d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f253800efc0 tx=0x7f253800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.236+0000 7f25357fa640 1 -- 172.21.15.172:0/4206259974 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538018020 con 0x7f2540160f30 2024-04-14T08:36:13.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.236+0000 7f25357fa640 1 -- 172.21.15.172:0/4206259974 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2538014480 con 0x7f2540160f30 2024-04-14T08:36:13.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.236+0000 7f25357fa640 1 -- 172.21.15.172:0/4206259974 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538014870 con 0x7f2540160f30 2024-04-14T08:36:13.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.236+0000 7f2547060640 1 -- 172.21.15.172:0/4206259974 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25400a0e70 con 0x7f2540160f30 2024-04-14T08:36:13.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.236+0000 7f2547060640 1 -- 172.21.15.172:0/4206259974 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25400a1430 con 0x7f2540160f30 2024-04-14T08:36:13.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.237+0000 7f25357fa640 1 -- 172.21.15.172:0/4206259974 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2538005d10 con 0x7f2540160f30 2024-04-14T08:36:13.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.238+0000 7f25357fa640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f252c06c8f0 0x7f252c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.238+0000 7f25357fa640 1 -- 172.21.15.172:0/4206259974 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f253808c5d0 con 0x7f2540160f30 2024-04-14T08:36:13.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.238+0000 7f2544dfc640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f252c06c8f0 0x7f252c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.238+0000 7f2544dfc640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f252c06c8f0 0x7f252c06edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f253c009ac0 tx=0x7f253c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.238+0000 7f2549e4a740 1 Processor -- start 2024-04-14T08:36:13.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.239+0000 7f2549e4a740 1 -- start start 2024-04-14T08:36:13.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.239+0000 7f2549e4a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2aa7970 0x55acd2a7a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.239+0000 7f2549e4a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7d50 0x55acd2aaa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.239+0000 7f2549e4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2a7ae50 0x55acd2aaac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.239+0000 7f2549e4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55acd29e4800 con 0x55acd2a7ae50 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.239+0000 7f2549e4a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55acd298c7f0 con 0x55acd2aa7d50 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.239+0000 7f2549e4a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55acd29e07b0 con 0x55acd2aa7970 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.239+0000 7f25455fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2a7ae50 0x55acd2aaac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.239+0000 7f25455fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2a7ae50 0x55acd2aaac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46372/0 (socket says 172.21.15.172:46372) 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.239+0000 7f25455fd640 1 -- 172.21.15.172:0/2057050837 learned_addr learned my addr 172.21.15.172:0/2057050837 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.239+0000 7f2544dfc640 1 --2- 172.21.15.172:0/2057050837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2aa7970 0x55acd2a7a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.239+0000 7f2537fff640 1 --2- 172.21.15.172:0/2057050837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7d50 0x55acd2aaa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f25455fd640 1 -- 172.21.15.172:0/2057050837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2aa7970 msgr2=0x55acd2a7a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f25455fd640 1 --2- 172.21.15.172:0/2057050837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2aa7970 0x55acd2a7a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f25455fd640 1 -- 172.21.15.172:0/2057050837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7d50 msgr2=0x55acd2aaa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f25455fd640 1 --2- 172.21.15.172:0/2057050837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7d50 0x55acd2aaa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f25455fd640 1 -- 172.21.15.172:0/2057050837 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acd27c27d0 con 0x55acd2a7ae50 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f25455fd640 1 --2- 172.21.15.172:0/2057050837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2a7ae50 0x55acd2aaac40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f25380628b0 tx=0x7f253805e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f8cf579406748e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f25267fc640 1 -- 172.21.15.172:0/2057050837 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538072020 con 0x55acd2a7ae50 2024-04-14T08:36:13.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f25267fc640 1 -- 172.21.15.172:0/2057050837 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25380585b0 con 0x55acd2a7ae50 2024-04-14T08:36:13.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f25267fc640 1 -- 172.21.15.172:0/2057050837 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253805e170 con 0x55acd2a7ae50 2024-04-14T08:36:13.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f2549e4a740 1 -- 172.21.15.172:0/2057050837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2a7ae50 msgr2=0x55acd2aaac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f2549e4a740 1 --2- 172.21.15.172:0/2057050837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2a7ae50 0x55acd2aaac40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f25380628b0 tx=0x7f253805e5b0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f2549e4a740 1 -- 172.21.15.172:0/2057050837 shutdown_connections 2024-04-14T08:36:13.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f2549e4a740 1 --2- 172.21.15.172:0/2057050837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2a7ae50 0x55acd2aaac40 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f2549e4a740 1 --2- 172.21.15.172:0/2057050837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7d50 0x55acd2aaa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.240+0000 7f2549e4a740 1 --2- 172.21.15.172:0/2057050837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2aa7970 0x55acd2a7a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.241+0000 7f2549e4a740 1 -- 172.21.15.172:0/2057050837 >> 172.21.15.172:0/2057050837 conn(0x55acd2a7cb80 msgr2=0x55acd2ac5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:13.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.241+0000 7f2549e4a740 1 -- 172.21.15.172:0/2057050837 shutdown_connections 2024-04-14T08:36:13.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.241+0000 7f2549e4a740 1 -- 172.21.15.172:0/2057050837 wait complete. 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.241+0000 7f2549e4a740 1 -- start start 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.241+0000 7f2549e4a740 1 Processor -- start 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f2549e4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 0x55acd2a95280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f2549e4a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7d50 0x55acd2aad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f2549e4a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2a957c0 0x55acd2aad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f2549e4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55acd2a7f120 con 0x55acd2aa7970 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f2549e4a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55acd299fd00 con 0x55acd2aa7d50 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f2549e4a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55acd2a7e510 con 0x55acd2a957c0 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f2549e4a740 10 client.0 _ll_register_callbacks cb 0x55acd299f6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f2544dfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 0x55acd2a95280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f2544dfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 0x55acd2a95280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46376/0 (socket says 172.21.15.172:46376) 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f2544dfc640 1 -- 172.21.15.172:0/4191323591 learned_addr learned my addr 172.21.15.172:0/4191323591 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:13.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f25455fd640 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2a957c0 0x55acd2aad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f2537fff640 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7d50 0x55acd2aad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f25455fd640 1 -- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7d50 msgr2=0x55acd2aad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f25455fd640 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7d50 0x55acd2aad180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f25455fd640 1 -- 172.21.15.172:0/4191323591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 msgr2=0x55acd2a95280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f25455fd640 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 0x55acd2a95280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f25455fd640 1 -- 172.21.15.172:0/4191323591 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55acd26f83f0 con 0x55acd2a957c0 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.242+0000 7f2544dfc640 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 0x55acd2a95280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f25455fd640 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2a957c0 0x55acd2aad6c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2538056710 tx=0x7f253801f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b7e23b17ede6a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f251f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.177:3301/0 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f251f7fe640 1 -- 172.21.15.172:0/4191323591 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538072040 con 0x55acd2a957c0 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f251f7fe640 1 -- 172.21.15.172:0/4191323591 <== mon.2 v2:172.21.15.177:3301/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f2538059900 con 0x55acd2a957c0 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f2549e4a740 1 -- 172.21.15.172:0/4191323591 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25380585b0 con 0x55acd2a957c0 2024-04-14T08:36:13.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f251f7fe640 1 -- 172.21.15.172:0/4191323591 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2538080050 con 0x55acd2a957c0 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f2549e4a740 10 client.5741 fetch_fsmap learned FSMap version 31 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f2549e4a740 1 -- 172.21.15.172:0/4191323591 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x55acd26b80f0 con 0x55acd2a957c0 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f251f7fe640 1 -- 172.21.15.172:0/4191323591 <== mon.2 v2:172.21.15.177:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25380585b0 con 0x55acd2a957c0 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f251f7fe640 1 -- 172.21.15.172:0/4191323591 <== mon.2 v2:172.21.15.177:3301/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7f2538083020 con 0x55acd2a957c0 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f2549e4a740 10 client.5741 fetch_fsmap finished waiting for FSMap version 31 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f2549e4a740 10 client.5741 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f2549e4a740 20 client.5741 populate_metadata read hostname 'smithi172' 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f2549e4a740 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55acd2ac05d0 0x55acd2a97e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f2549e4a740 4 client.5741 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f2549e4a740 1 -- 172.21.15.172:0/4191323591 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acd29f1d40 con 0x55acd2ac05d0 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.243+0000 7f2544dfc640 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55acd2ac05d0 0x55acd2a97e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.244+0000 7f2544dfc640 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55acd2ac05d0 0x55acd2a97e60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2c1e17ce270e9af server_cookie=7a9a05ca417e120a in_seq=0 out_seq=0 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.244+0000 7f251f7fe640 10 client.5741 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:13.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.244+0000 7f251f7fe640 1 -- 172.21.15.172:0/4191323591 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55acd29f1d40 con 0x55acd2ac05d0 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.244+0000 7f251f7fe640 10 client.5741 handle_command_reply: tid=0 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.244+0000 7f2549e4a740 1 client.5741 shutdown 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.245+0000 7f2549e4a740 1 -- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2a957c0 msgr2=0x55acd2aad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.245+0000 7f2549e4a740 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2a957c0 0x55acd2aad6c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2538056710 tx=0x7f253801f040 comp rx=0 tx=0).stop 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.245+0000 7f2549e4a740 1 -- 172.21.15.172:0/4191323591 shutdown_connections 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.245+0000 7f2549e4a740 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55acd2ac05d0 0x55acd2a97e60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.245+0000 7f2549e4a740 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2a957c0 0x55acd2aad6c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.245+0000 7f2549e4a740 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7d50 0x55acd2aad180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.245+0000 7f2549e4a740 1 --2- 172.21.15.172:0/4191323591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 0x55acd2a95280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.245+0000 7f251f7fe640 0 client.5741 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.245+0000 7f2549e4a740 1 -- 172.21.15.172:0/4191323591 >> 172.21.15.172:0/4191323591 conn(0x55acd2a7ae50 msgr2=0x55acd2ac8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.245+0000 7f2549e4a740 1 -- 172.21.15.172:0/4191323591 shutdown_connections 2024-04-14T08:36:13.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.245+0000 7f2549e4a740 1 -- 172.21.15.172:0/4191323591 wait complete. 2024-04-14T08:36:13.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.245+0000 7f2549e4a740 20 client.5741 trim_cache size 0 max 16384 2024-04-14T08:36:13.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.262+0000 7f2549e4a740 1 Processor -- start 2024-04-14T08:36:13.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.262+0000 7f2549e4a740 1 -- start start 2024-04-14T08:36:13.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2549e4a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2a957c0 0x55acd2abf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2549e4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 0x55acd2ac8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2549e4a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2aa7d50 0x55acd2ac9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2549e4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f25402010d0 con 0x55acd2aa7970 2024-04-14T08:36:13.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2549e4a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f25402000f0 con 0x55acd2a957c0 2024-04-14T08:36:13.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2549e4a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f254009ecb0 con 0x55acd2aa7d50 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2537fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 0x55acd2ac8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2537fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 0x55acd2ac8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46382/0 (socket says 172.21.15.172:46382) 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2537fff640 1 -- 172.21.15.172:0/2291620262 learned_addr learned my addr 172.21.15.172:0/2291620262 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2544dfc640 1 --2- 172.21.15.172:0/2291620262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2a957c0 0x55acd2abf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f25455fd640 1 --2- 172.21.15.172:0/2291620262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2aa7d50 0x55acd2ac9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2537fff640 1 -- 172.21.15.172:0/2291620262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2aa7d50 msgr2=0x55acd2ac9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2537fff640 1 --2- 172.21.15.172:0/2291620262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2aa7d50 0x55acd2ac9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2537fff640 1 -- 172.21.15.172:0/2291620262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2a957c0 msgr2=0x55acd2abf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2537fff640 1 --2- 172.21.15.172:0/2291620262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2a957c0 0x55acd2abf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2537fff640 1 -- 172.21.15.172:0/2291620262 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acd2a76c20 con 0x55acd2aa7970 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f25455fd640 1 --2- 172.21.15.172:0/2291620262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2aa7d50 0x55acd2ac9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2537fff640 1 --2- 172.21.15.172:0/2291620262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 0x55acd2ac8c10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2530009960 tx=0x7f2530030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c86b1026b86828 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2525ffb640 1 -- 172.21.15.172:0/2291620262 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530035040 con 0x55acd2aa7970 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2525ffb640 1 -- 172.21.15.172:0/2291620262 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25300318f0 con 0x55acd2aa7970 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.263+0000 7f2525ffb640 1 -- 172.21.15.172:0/2291620262 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530031ca0 con 0x55acd2aa7970 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.264+0000 7f2549e4a740 1 -- 172.21.15.172:0/2291620262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 msgr2=0x55acd2ac8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.264+0000 7f2549e4a740 1 --2- 172.21.15.172:0/2291620262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 0x55acd2ac8c10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2530009960 tx=0x7f2530030000 comp rx=0 tx=0).stop 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.264+0000 7f2549e4a740 1 -- 172.21.15.172:0/2291620262 shutdown_connections 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.264+0000 7f2549e4a740 1 --2- 172.21.15.172:0/2291620262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2aa7d50 0x55acd2ac9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.264+0000 7f2549e4a740 1 --2- 172.21.15.172:0/2291620262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7970 0x55acd2ac8c10 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.264+0000 7f2549e4a740 1 --2- 172.21.15.172:0/2291620262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2a957c0 0x55acd2abf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.264+0000 7f2549e4a740 1 -- 172.21.15.172:0/2291620262 >> 172.21.15.172:0/2291620262 conn(0x55acd2a7ae50 msgr2=0x55acd2a76110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.264+0000 7f2549e4a740 1 -- 172.21.15.172:0/2291620262 shutdown_connections 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.264+0000 7f2549e4a740 1 -- 172.21.15.172:0/2291620262 wait complete. 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.264+0000 7f2549e4a740 1 -- start start 2024-04-14T08:36:13.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.265+0000 7f2549e4a740 1 Processor -- start 2024-04-14T08:36:13.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.265+0000 7f2549e4a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2a957c0 0x55acd2ac8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.265+0000 7f2549e4a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7970 0x55acd2ac9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.265+0000 7f2549e4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7d50 0x55acd2a99b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.265+0000 7f2549e4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2540154e10 con 0x55acd2aa7d50 2024-04-14T08:36:13.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.265+0000 7f2549e4a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55acd2abfa80 con 0x55acd2aa7970 2024-04-14T08:36:13.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.265+0000 7f2549e4a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55acd2a9aa50 con 0x55acd2a957c0 2024-04-14T08:36:13.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.265+0000 7f2549e4a740 10 client.0 _ll_register_callbacks cb 0x7f2538059ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:36:13.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f25455fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7d50 0x55acd2a99b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f2537fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7970 0x55acd2ac9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f2544dfc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2a957c0 0x55acd2ac8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f25455fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7d50 0x55acd2a99b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46394/0 (socket says 172.21.15.172:46394) 2024-04-14T08:36:13.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f25455fd640 1 -- 172.21.15.172:0/1801098730 learned_addr learned my addr 172.21.15.172:0/1801098730 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:13.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f25455fd640 1 -- 172.21.15.172:0/1801098730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2a957c0 msgr2=0x55acd2ac8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f25455fd640 1 --2- 172.21.15.172:0/1801098730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2a957c0 0x55acd2ac8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f25455fd640 1 -- 172.21.15.172:0/1801098730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7970 msgr2=0x55acd2ac9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:13.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f25455fd640 1 --2- 172.21.15.172:0/1801098730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7970 0x55acd2ac9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f25455fd640 1 -- 172.21.15.172:0/1801098730 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acd2ad3fc0 con 0x55acd2aa7d50 2024-04-14T08:36:13.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f2537fff640 1 --2- 172.21.15.172:0/1801098730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7970 0x55acd2ac9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:13.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f25455fd640 1 --2- 172.21.15.172:0/1801098730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7d50 0x55acd2a99b30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2538056c30 tx=0x7f253805e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45ed59b8abb073e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:13.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f251f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:36:13.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f251f7fe640 1 -- 172.21.15.172:0/1801098730 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538072050 con 0x55acd2aa7d50 2024-04-14T08:36:13.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f251f7fe640 1 -- 172.21.15.172:0/1801098730 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f2538056000 con 0x55acd2aa7d50 2024-04-14T08:36:13.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f2549e4a740 1 -- 172.21.15.172:0/1801098730 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25300318f0 con 0x55acd2aa7d50 2024-04-14T08:36:13.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f251f7fe640 1 -- 172.21.15.172:0/1801098730 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f25380585b0 con 0x55acd2aa7d50 2024-04-14T08:36:13.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f2549e4a740 10 client.6504 fetch_fsmap learned FSMap version 31 2024-04-14T08:36:13.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f2549e4a740 1 -- 172.21.15.172:0/1801098730 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) -- 0x55acd2b3e530 con 0x55acd2aa7d50 2024-04-14T08:36:13.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f251f7fe640 1 -- 172.21.15.172:0/1801098730 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25300318f0 con 0x55acd2aa7d50 2024-04-14T08:36:13.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f251f7fe640 1 -- 172.21.15.172:0/1801098730 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 3494+0+0 (secure 0 0 0) 0x7f2538083070 con 0x55acd2aa7d50 2024-04-14T08:36:13.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f2549e4a740 10 client.6504 fetch_fsmap finished waiting for FSMap version 31 2024-04-14T08:36:13.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f2549e4a740 10 client.6504 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-14T08:36:13.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f2549e4a740 20 client.6504 populate_metadata read hostname 'smithi172' 2024-04-14T08:36:13.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.266+0000 7f2549e4a740 1 --2- 172.21.15.172:0/1801098730 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55acd2b42a50 0x55acd2a9a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:13.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 4 client.6504 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:36:13.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 1 -- 172.21.15.172:0/1801098730 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55acd29ec090 con 0x55acd2b42a50 2024-04-14T08:36:13.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2544dfc640 1 --2- 172.21.15.172:0/1801098730 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55acd2b42a50 0x55acd2a9a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:13.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f251f7fe640 10 client.6504 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:13.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2544dfc640 1 --2- 172.21.15.172:0/1801098730 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55acd2b42a50 0x55acd2a9a070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a643546a861b06 server_cookie=aa318e208e123e4 in_seq=0 out_seq=0 2024-04-14T08:36:13.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f251f7fe640 1 -- 172.21.15.172:0/1801098730 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55acd29ec090 con 0x55acd2b42a50 2024-04-14T08:36:13.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f251f7fe640 10 client.6504 handle_command_reply: tid=0 2024-04-14T08:36:13.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 1 client.6504 shutdown 2024-04-14T08:36:13.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 1 -- 172.21.15.172:0/1801098730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7d50 msgr2=0x55acd2a99b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 1 --2- 172.21.15.172:0/1801098730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7d50 0x55acd2a99b30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2538056c30 tx=0x7f253805e000 comp rx=0 tx=0).stop 2024-04-14T08:36:13.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 1 -- 172.21.15.172:0/1801098730 shutdown_connections 2024-04-14T08:36:13.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 1 --2- 172.21.15.172:0/1801098730 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x55acd2b42a50 0x55acd2a9a070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 1 --2- 172.21.15.172:0/1801098730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acd2aa7d50 0x55acd2a99b30 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 1 --2- 172.21.15.172:0/1801098730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55acd2aa7970 0x55acd2ac9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f251f7fe640 0 client.6504 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:13.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 1 --2- 172.21.15.172:0/1801098730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55acd2a957c0 0x55acd2ac8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 1 -- 172.21.15.172:0/1801098730 >> 172.21.15.172:0/1801098730 conn(0x55acd2a7ae50 msgr2=0x55acd2aad070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:13.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 1 -- 172.21.15.172:0/1801098730 shutdown_connections 2024-04-14T08:36:13.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.267+0000 7f2549e4a740 1 -- 172.21.15.172:0/1801098730 wait complete. 2024-04-14T08:36:13.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2549e4a740 20 client.6504 trim_cache size 0 max 16384 2024-04-14T08:36:13.276 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:36:13.276 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-14T08:36:13.276 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-04-14T08:36:13.276 INFO:teuthology.orchestra.run.smithi172.stdout: "f12e16de-33de-4557-afe3-dca0826a9814": { 2024-04-14T08:36:13.276 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-04-14T08:36:13.276 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "f12e16de-33de-4557-afe3-dca0826a9814", 2024-04-14T08:36:13.276 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,repair,force" 2024-04-14T08:36:13.276 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:13.276 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:13.276 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:36:13.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2547060640 1 -- 172.21.15.172:0/4206259974 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f252c06c8f0 msgr2=0x7f252c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2547060640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f252c06c8f0 0x7f252c06edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f253c009ac0 tx=0x7f253c009190 comp rx=0 tx=0).stop 2024-04-14T08:36:13.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2547060640 1 -- 172.21.15.172:0/4206259974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2540160f30 msgr2=0x7f25400b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:13.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2547060640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2540160f30 0x7f25400b18d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f253800efc0 tx=0x7f253800c490 comp rx=0 tx=0).stop 2024-04-14T08:36:13.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2547060640 1 -- 172.21.15.172:0/4206259974 shutdown_connections 2024-04-14T08:36:13.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2547060640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f252c06c8f0 0x7f252c06edb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2547060640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2540160f30 0x7f25400b18d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2547060640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25400d49c0 0x7f2540153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2547060640 1 --2- 172.21.15.172:0/4206259974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25400d0cc0 0x7f25400a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:13.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2547060640 1 -- 172.21.15.172:0/4206259974 >> 172.21.15.172:0/4206259974 conn(0x7f254009d580 msgr2=0x7f25400cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:13.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2547060640 1 -- 172.21.15.172:0/4206259974 shutdown_connections 2024-04-14T08:36:13.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:13.268+0000 7f2547060640 1 -- 172.21.15.172:0/4206259974 wait complete. 2024-04-14T08:36:13.279 INFO:tasks.cephfs.filesystem:scrub status for tag:f12e16de-33de-4557-afe3-dca0826a9814 - {'path': '/', 'tag': 'f12e16de-33de-4557-afe3-dca0826a9814', 'options': 'recursive,repair,force'} 2024-04-14T08:36:23.538 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:36:23.541 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:36:43.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:36:43.293 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-14T08:36:43.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.407+0000 7f0e86008640 1 Processor -- start 2024-04-14T08:36:43.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.407+0000 7f0e86008640 1 -- start start 2024-04-14T08:36:43.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.407+0000 7f0e86008640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e80162710 0x7f0e800d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.407+0000 7f0e86008640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0e80162b70 0x7f0e800d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.407+0000 7f0e86008640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e800cefb0 con 0x7f0e8015e430 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.407+0000 7f0e86008640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f0e800cf290 con 0x7f0e80162b70 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.407+0000 7f0e86008640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0e800cf570 con 0x7f0e80162710 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.408+0000 7f0e7ffff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0e80162b70 0x7f0e800d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.408+0000 7f0e7effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e80162710 0x7f0e800d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.408+0000 7f0e7f7fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f0e8015e430 0x7f0e80162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:49482/0 (socket says 172.21.15.172:49482) 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.408+0000 7f0e7f7fe640 1 -- 172.21.15.172:0/592213631 learned_addr learned my addr 172.21.15.172:0/592213631 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.408+0000 7f0e7e7fc640 1 -- 172.21.15.172:0/592213631 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3953105181 0 0) 0x7f0e800cefb0 con 0x7f0e8015e430 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.408+0000 7f0e7e7fc640 1 -- 172.21.15.172:0/592213631 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e6c003520 con 0x7f0e8015e430 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.408+0000 7f0e7ffff640 1 -- 172.21.15.172:0/592213631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e80162710 msgr2=0x7f0e800d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.408+0000 7f0e7ffff640 1 --2- 172.21.15.172:0/592213631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e80162710 0x7f0e800d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.408+0000 7f0e7ffff640 1 -- 172.21.15.172:0/592213631 >> v1:172.21.15.177:6789/0 conn(0x7f0e8015e430 legacy=0x7f0e80162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.408+0000 7f0e7ffff640 1 -- 172.21.15.172:0/592213631 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e800cf850 con 0x7f0e80162b70 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.408+0000 7f0e7effd640 1 --2- 172.21.15.172:0/592213631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e80162710 0x7f0e800d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:43.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.409+0000 7f0e7ffff640 1 --2- 172.21.15.172:0/592213631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e80162b70 0x7f0e800d2330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009d90 tx=0x7f0e7802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c778a185e3899fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.409+0000 7f0e7e7fc640 1 -- 172.21.15.172:0/592213631 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78039070 con 0x7f0e80162b70 2024-04-14T08:36:43.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.409+0000 7f0e7e7fc640 1 -- 172.21.15.172:0/592213631 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e7802cc50 con 0x7f0e80162b70 2024-04-14T08:36:43.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.409+0000 7f0e7e7fc640 1 -- 172.21.15.172:0/592213631 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78034440 con 0x7f0e80162b70 2024-04-14T08:36:43.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.409+0000 7f0e86008640 1 -- 172.21.15.172:0/592213631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e80162b70 msgr2=0x7f0e800d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.409+0000 7f0e86008640 1 --2- 172.21.15.172:0/592213631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e80162b70 0x7f0e800d2330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009d90 tx=0x7f0e7802bea0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.410+0000 7f0e86008640 1 -- 172.21.15.172:0/592213631 shutdown_connections 2024-04-14T08:36:43.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.410+0000 7f0e86008640 1 --2- 172.21.15.172:0/592213631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e80162b70 0x7f0e800d2330 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.410+0000 7f0e86008640 1 --2- 172.21.15.172:0/592213631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e80162710 0x7f0e800d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.410+0000 7f0e86008640 1 -- 172.21.15.172:0/592213631 >> 172.21.15.172:0/592213631 conn(0x7f0e8009d560 msgr2=0x7f0e800d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.410+0000 7f0e86008640 1 -- 172.21.15.172:0/592213631 shutdown_connections 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.410+0000 7f0e86008640 1 -- 172.21.15.172:0/592213631 wait complete. 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.411+0000 7f0e86008640 1 Processor -- start 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.411+0000 7f0e86008640 1 -- start start 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e86008640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e8015e430 0x7f0e800b97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e86008640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e80162710 0x7f0e800b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e86008640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e80162b70 0x7f0e8015f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e86008640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e800d4e20 con 0x7f0e80162b70 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e86008640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8015e810 con 0x7f0e80162710 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e86008640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0e800d0050 con 0x7f0e8015e430 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e7ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e80162b70 0x7f0e8015f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e7f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e8015e430 0x7f0e800b97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e7effd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e80162710 0x7f0e800b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e7f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e8015e430 0x7f0e800b97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:39238/0 (socket says 172.21.15.172:39238) 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e7effd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e80162710 0x7f0e800b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:47234/0 (socket says 172.21.15.172:47234) 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e7f7fe640 1 -- 172.21.15.172:0/3515809410 learned_addr learned my addr 172.21.15.172:0/3515809410 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e7ffff640 1 -- 172.21.15.172:0/3515809410 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e8015e430 msgr2=0x7f0e800b97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e7ffff640 1 --2- 172.21.15.172:0/3515809410 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e8015e430 0x7f0e800b97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e7ffff640 1 -- 172.21.15.172:0/3515809410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e80162710 msgr2=0x7f0e800b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e7ffff640 1 --2- 172.21.15.172:0/3515809410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e80162710 0x7f0e800b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e7ffff640 1 -- 172.21.15.172:0/3515809410 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e800b8420 con 0x7f0e80162b70 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.412+0000 7f0e7ffff640 1 --2- 172.21.15.172:0/3515809410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e80162b70 0x7f0e8015f7b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0e78033000 tx=0x7f0e78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.413+0000 7f0e77fff640 1 -- 172.21.15.172:0/3515809410 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78039050 con 0x7f0e80162b70 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.413+0000 7f0e77fff640 1 -- 172.21.15.172:0/3515809410 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e78045070 con 0x7f0e80162b70 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.413+0000 7f0e77fff640 1 -- 172.21.15.172:0/3515809410 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78009450 con 0x7f0e80162b70 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.413+0000 7f0e86008640 1 -- 172.21.15.172:0/3515809410 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e800b8700 con 0x7f0e80162b70 2024-04-14T08:36:43.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.413+0000 7f0e86008640 1 -- 172.21.15.172:0/3515809410 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e800b8cc0 con 0x7f0e80162b70 2024-04-14T08:36:43.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.414+0000 7f0e77fff640 1 -- 172.21.15.172:0/3515809410 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e7803e610 con 0x7f0e80162b70 2024-04-14T08:36:43.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.415+0000 7f0e77fff640 1 --2- 172.21.15.172:0/3515809410 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0e6006c8f0 0x7f0e6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.415+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/3515809410 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0e6006c8f0 0x7f0e6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.415+0000 7f0e77fff640 1 -- 172.21.15.172:0/3515809410 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f0e780ae4b0 con 0x7f0e80162b70 2024-04-14T08:36:43.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.415+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/3515809410 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0e6006c8f0 0x7f0e6006edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0e68005880 tx=0x7f0e68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.416+0000 7f0e88deb740 1 Processor -- start 2024-04-14T08:36:43.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.416+0000 7f0e88deb740 1 -- start start 2024-04-14T08:36:43.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e88deb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748b970 0x562ee745e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e88deb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 0x562ee748e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e88deb740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee745ee50 0x562ee748ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e88deb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562ee73c8800 con 0x562ee748bd50 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e88deb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562ee73707f0 con 0x562ee748b970 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e88deb740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x562ee73c47b0 con 0x562ee745ee50 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e7effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 0x562ee748e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e7effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 0x562ee748e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43334/0 (socket says 172.21.15.172:43334) 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e7effd640 1 -- 172.21.15.172:0/1089285875 learned_addr learned my addr 172.21.15.172:0/1089285875 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/1089285875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748b970 0x562ee745e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e7ffff640 1 --2- 172.21.15.172:0/1089285875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee745ee50 0x562ee748ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e7effd640 1 -- 172.21.15.172:0/1089285875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee745ee50 msgr2=0x562ee748ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e7effd640 1 --2- 172.21.15.172:0/1089285875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee745ee50 0x562ee748ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e7effd640 1 -- 172.21.15.172:0/1089285875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748b970 msgr2=0x562ee745e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e7effd640 1 --2- 172.21.15.172:0/1089285875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748b970 0x562ee745e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e7effd640 1 -- 172.21.15.172:0/1089285875 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee71a67d0 con 0x562ee748bd50 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/1089285875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748b970 0x562ee745e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e7effd640 1 --2- 172.21.15.172:0/1089285875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 0x562ee748e700 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009b50 tx=0x7f0e70011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fa6612929e4633 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e757fa640 1 -- 172.21.15.172:0/1089285875 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70019070 con 0x562ee748bd50 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e757fa640 1 -- 172.21.15.172:0/1089285875 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e70009e50 con 0x562ee748bd50 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.417+0000 7f0e757fa640 1 -- 172.21.15.172:0/1089285875 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70004270 con 0x562ee748bd50 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.418+0000 7f0e88deb740 1 -- 172.21.15.172:0/1089285875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 msgr2=0x562ee748e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.418+0000 7f0e88deb740 1 --2- 172.21.15.172:0/1089285875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 0x562ee748e700 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009b50 tx=0x7f0e70011690 comp rx=0 tx=0).stop 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.418+0000 7f0e88deb740 1 -- 172.21.15.172:0/1089285875 shutdown_connections 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.418+0000 7f0e88deb740 1 --2- 172.21.15.172:0/1089285875 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee745ee50 0x562ee748ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.418+0000 7f0e88deb740 1 --2- 172.21.15.172:0/1089285875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 0x562ee748e700 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.418+0000 7f0e88deb740 1 --2- 172.21.15.172:0/1089285875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748b970 0x562ee745e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.418+0000 7f0e88deb740 1 -- 172.21.15.172:0/1089285875 >> 172.21.15.172:0/1089285875 conn(0x562ee7460b80 msgr2=0x562ee74a9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.418+0000 7f0e88deb740 1 -- 172.21.15.172:0/1089285875 shutdown_connections 2024-04-14T08:36:43.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.418+0000 7f0e88deb740 1 -- 172.21.15.172:0/1089285875 wait complete. 2024-04-14T08:36:43.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.419+0000 7f0e88deb740 1 -- start start 2024-04-14T08:36:43.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.419+0000 7f0e88deb740 1 Processor -- start 2024-04-14T08:36:43.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.419+0000 7f0e88deb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748b970 0x562ee74a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e88deb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 0x562ee747c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e88deb740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee74a6ff0 0x562ee74a7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e88deb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562ee7463120 con 0x562ee748bd50 2024-04-14T08:36:43.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e88deb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562ee7383d00 con 0x562ee748b970 2024-04-14T08:36:43.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e88deb740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x562ee7462510 con 0x562ee74a6ff0 2024-04-14T08:36:43.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e88deb740 10 client.0 _ll_register_callbacks cb 0x562ee73836f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:36:43.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e7effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 0x562ee747c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e7effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 0x562ee747c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43348/0 (socket says 172.21.15.172:43348) 2024-04-14T08:36:43.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e7effd640 1 -- 172.21.15.172:0/446233924 learned_addr learned my addr 172.21.15.172:0/446233924 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748b970 0x562ee74a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e7ffff640 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee74a6ff0 0x562ee74a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e7effd640 1 -- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee74a6ff0 msgr2=0x562ee74a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e7effd640 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee74a6ff0 0x562ee74a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e7effd640 1 -- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748b970 msgr2=0x562ee74a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e7effd640 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748b970 0x562ee74a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e7effd640 1 -- 172.21.15.172:0/446233924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ee70dc3f0 con 0x562ee748bd50 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748b970 0x562ee74a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e7effd640 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 0x562ee747c320 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0e70018000 tx=0x7f0e70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d022acf4e88ac3ce server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e5e7fc640 1 -- 172.21.15.172:0/446233924 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70019040 con 0x562ee748bd50 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e5e7fc640 1 -- 172.21.15.172:0/446233924 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f0e70003820 con 0x562ee748bd50 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.420+0000 7f0e88deb740 1 -- 172.21.15.172:0/446233924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e70009e50 con 0x562ee748bd50 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e5e7fc640 1 -- 172.21.15.172:0/446233924 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0e70031050 con 0x562ee748bd50 2024-04-14T08:36:43.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e88deb740 10 client.6513 fetch_fsmap learned FSMap version 32 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e88deb740 1 -- 172.21.15.172:0/446233924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x562ee709c0f0 con 0x562ee748bd50 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e5e7fc640 1 -- 172.21.15.172:0/446233924 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e70009e50 con 0x562ee748bd50 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e5e7fc640 1 -- 172.21.15.172:0/446233924 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7f0e70034020 con 0x562ee748bd50 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e88deb740 10 client.6513 fetch_fsmap finished waiting for FSMap version 32 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e88deb740 10 client.6513 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e88deb740 20 client.6513 populate_metadata read hostname 'smithi172' 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e88deb740 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x562ee7491590 0x562ee7491a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e88deb740 4 client.6513 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e88deb740 1 -- 172.21.15.172:0/446233924 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ee73d5d40 con 0x562ee7491590 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x562ee7491590 0x562ee7491a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x562ee7491590 0x562ee7491a00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47e17ba0a9722e39 server_cookie=daeb5c2c3836db69 in_seq=0 out_seq=0 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.421+0000 7f0e5e7fc640 10 client.6513 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e5e7fc640 1 -- 172.21.15.172:0/446233924 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ee73d5d40 con 0x562ee7491590 2024-04-14T08:36:43.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e5e7fc640 10 client.6513 handle_command_reply: tid=0 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e88deb740 1 client.6513 shutdown 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e88deb740 1 -- 172.21.15.172:0/446233924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 msgr2=0x562ee747c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e88deb740 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 0x562ee747c320 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0e70018000 tx=0x7f0e70004730 comp rx=0 tx=0).stop 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e88deb740 1 -- 172.21.15.172:0/446233924 shutdown_connections 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e88deb740 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x562ee7491590 0x562ee7491a00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e88deb740 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee74a6ff0 0x562ee74a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e88deb740 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748bd50 0x562ee747c320 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e5e7fc640 0 client.6513 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e88deb740 1 --2- 172.21.15.172:0/446233924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748b970 0x562ee74a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e88deb740 1 -- 172.21.15.172:0/446233924 >> 172.21.15.172:0/446233924 conn(0x562ee745ee50 msgr2=0x562ee74acc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e88deb740 1 -- 172.21.15.172:0/446233924 shutdown_connections 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.422+0000 7f0e88deb740 1 -- 172.21.15.172:0/446233924 wait complete. 2024-04-14T08:36:43.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.423+0000 7f0e88deb740 20 client.6513 trim_cache size 0 max 16384 2024-04-14T08:36:43.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.440+0000 7f0e88deb740 1 Processor -- start 2024-04-14T08:36:43.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.440+0000 7f0e88deb740 1 -- start start 2024-04-14T08:36:43.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e88deb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748b970 0x562ee748ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e88deb740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee748bd50 0x562ee74ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e88deb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee74a6ff0 0x562ee748e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e88deb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e80201160 con 0x562ee748b970 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e88deb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0e80200140 con 0x562ee74a6ff0 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e88deb740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0e800c86b0 con 0x562ee748bd50 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e7f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748b970 0x562ee748ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e7f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748b970 0x562ee748ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43350/0 (socket says 172.21.15.172:43350) 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e7f7fe640 1 -- 172.21.15.172:0/1254762120 learned_addr learned my addr 172.21.15.172:0/1254762120 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e7effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee748bd50 0x562ee74ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e7ffff640 1 --2- 172.21.15.172:0/1254762120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee74a6ff0 0x562ee748e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e7f7fe640 1 -- 172.21.15.172:0/1254762120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee748bd50 msgr2=0x562ee74ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/1254762120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee748bd50 0x562ee74ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e7f7fe640 1 -- 172.21.15.172:0/1254762120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee74a6ff0 msgr2=0x562ee748e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/1254762120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee74a6ff0 0x562ee748e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e7f7fe640 1 -- 172.21.15.172:0/1254762120 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee7465f20 con 0x562ee748b970 2024-04-14T08:36:43.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.441+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/1254762120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748b970 0x562ee748ad50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0e801f7ae0 tx=0x7f0e6800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fd2466ced0b833 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.442+0000 7f0e74ff9640 1 -- 172.21.15.172:0/1254762120 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6802f070 con 0x562ee748b970 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.442+0000 7f0e74ff9640 1 -- 172.21.15.172:0/1254762120 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6801b040 con 0x562ee748b970 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.442+0000 7f0e74ff9640 1 -- 172.21.15.172:0/1254762120 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6802ad50 con 0x562ee748b970 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.442+0000 7f0e88deb740 1 -- 172.21.15.172:0/1254762120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748b970 msgr2=0x562ee748ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.442+0000 7f0e88deb740 1 --2- 172.21.15.172:0/1254762120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748b970 0x562ee748ad50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0e801f7ae0 tx=0x7f0e6800a000 comp rx=0 tx=0).stop 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.442+0000 7f0e88deb740 1 -- 172.21.15.172:0/1254762120 shutdown_connections 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.442+0000 7f0e88deb740 1 --2- 172.21.15.172:0/1254762120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee74a6ff0 0x562ee748e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.442+0000 7f0e88deb740 1 --2- 172.21.15.172:0/1254762120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee748bd50 0x562ee74ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.442+0000 7f0e88deb740 1 --2- 172.21.15.172:0/1254762120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee748b970 0x562ee748ad50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.442+0000 7f0e88deb740 1 -- 172.21.15.172:0/1254762120 >> 172.21.15.172:0/1254762120 conn(0x562ee745ee50 msgr2=0x562ee74919a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.442+0000 7f0e88deb740 1 -- 172.21.15.172:0/1254762120 shutdown_connections 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.442+0000 7f0e88deb740 1 -- 172.21.15.172:0/1254762120 wait complete. 2024-04-14T08:36:43.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.443+0000 7f0e88deb740 1 -- start start 2024-04-14T08:36:43.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.443+0000 7f0e88deb740 1 Processor -- start 2024-04-14T08:36:43.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e88deb740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee748b970 0x562ee74ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e88deb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748bd50 0x562ee75220a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e88deb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee74a6ff0 0x562ee747c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e88deb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e800b7ac0 con 0x562ee74a6ff0 2024-04-14T08:36:43.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e88deb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562ee73d6470 con 0x562ee748bd50 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e88deb740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x562ee748f940 con 0x562ee748b970 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e88deb740 10 client.0 _ll_register_callbacks cb 0x7f0e700099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e7ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee74a6ff0 0x562ee747c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e7ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee74a6ff0 0x562ee747c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43360/0 (socket says 172.21.15.172:43360) 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e7f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee748b970 0x562ee74ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e7ffff640 1 -- 172.21.15.172:0/3493189491 learned_addr learned my addr 172.21.15.172:0/3493189491 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e7effd640 1 --2- 172.21.15.172:0/3493189491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748bd50 0x562ee75220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e7ffff640 1 -- 172.21.15.172:0/3493189491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee748b970 msgr2=0x562ee74ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e7ffff640 1 --2- 172.21.15.172:0/3493189491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee748b970 0x562ee74ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e7ffff640 1 -- 172.21.15.172:0/3493189491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748bd50 msgr2=0x562ee75220a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e7ffff640 1 --2- 172.21.15.172:0/3493189491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748bd50 0x562ee75220a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e7ffff640 1 -- 172.21.15.172:0/3493189491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ee748f4b0 con 0x562ee74a6ff0 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e7ffff640 1 --2- 172.21.15.172:0/3493189491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee74a6ff0 0x562ee747c200 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0e7807bcd0 tx=0x7f0e78080390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6642019d5610bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:36:43.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e5e7fc640 1 -- 172.21.15.172:0/3493189491 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7807a390 con 0x562ee74a6ff0 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e5e7fc640 1 -- 172.21.15.172:0/3493189491 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f0e78084670 con 0x562ee74a6ff0 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e88deb740 1 -- 172.21.15.172:0/3493189491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e6801b040 con 0x562ee74a6ff0 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.444+0000 7f0e5e7fc640 1 -- 172.21.15.172:0/3493189491 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0e78084870 con 0x562ee74a6ff0 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e88deb740 10 client.6519 fetch_fsmap learned FSMap version 32 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e88deb740 1 -- 172.21.15.172:0/3493189491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x562ee748f6c0 con 0x562ee74a6ff0 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e5e7fc640 1 -- 172.21.15.172:0/3493189491 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6801b040 con 0x562ee74a6ff0 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e5e7fc640 1 -- 172.21.15.172:0/3493189491 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7f0e780acb40 con 0x562ee74a6ff0 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e88deb740 10 client.6519 fetch_fsmap finished waiting for FSMap version 32 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e88deb740 10 client.6519 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e88deb740 20 client.6519 populate_metadata read hostname 'smithi172' 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e88deb740 1 --2- 172.21.15.172:0/3493189491 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x562ee748e290 0x562ee748e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e88deb740 4 client.6519 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e88deb740 1 -- 172.21.15.172:0/3493189491 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562ee73d0090 con 0x562ee748e290 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/3493189491 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x562ee748e290 0x562ee748e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e5e7fc640 10 client.6519 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e7f7fe640 1 --2- 172.21.15.172:0/3493189491 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x562ee748e290 0x562ee748e6a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e42b00a1f864f2 server_cookie=d03f9132b9b03632 in_seq=0 out_seq=0 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e5e7fc640 1 -- 172.21.15.172:0/3493189491 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562ee73d0090 con 0x562ee748e290 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e5e7fc640 10 client.6519 handle_command_reply: tid=0 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.445+0000 7f0e88deb740 1 client.6519 shutdown 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e88deb740 1 -- 172.21.15.172:0/3493189491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee74a6ff0 msgr2=0x562ee747c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e88deb740 1 --2- 172.21.15.172:0/3493189491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee74a6ff0 0x562ee747c200 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0e7807bcd0 tx=0x7f0e78080390 comp rx=0 tx=0).stop 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e88deb740 1 -- 172.21.15.172:0/3493189491 shutdown_connections 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e88deb740 1 --2- 172.21.15.172:0/3493189491 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x562ee748e290 0x562ee748e6a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e88deb740 1 --2- 172.21.15.172:0/3493189491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ee74a6ff0 0x562ee747c200 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e88deb740 1 --2- 172.21.15.172:0/3493189491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562ee748bd50 0x562ee75220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e5e7fc640 0 client.6519 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e88deb740 1 --2- 172.21.15.172:0/3493189491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562ee748b970 0x562ee74ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e88deb740 1 -- 172.21.15.172:0/3493189491 >> 172.21.15.172:0/3493189491 conn(0x562ee745ee50 msgr2=0x562ee748acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e88deb740 1 -- 172.21.15.172:0/3493189491 shutdown_connections 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e88deb740 1 -- 172.21.15.172:0/3493189491 wait complete. 2024-04-14T08:36:43.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e88deb740 20 client.6519 trim_cache size 0 max 16384 2024-04-14T08:36:43.454 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:36:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-04-14T08:36:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-04-14T08:36:43.454 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:36:43.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e86008640 1 -- 172.21.15.172:0/3515809410 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0e6006c8f0 msgr2=0x7f0e6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e86008640 1 --2- 172.21.15.172:0/3515809410 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0e6006c8f0 0x7f0e6006edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0e68005880 tx=0x7f0e68009040 comp rx=0 tx=0).stop 2024-04-14T08:36:43.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e86008640 1 -- 172.21.15.172:0/3515809410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e80162b70 msgr2=0x7f0e8015f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e86008640 1 --2- 172.21.15.172:0/3515809410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e80162b70 0x7f0e8015f7b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0e78033000 tx=0x7f0e78004a40 comp rx=0 tx=0).stop 2024-04-14T08:36:43.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e86008640 1 -- 172.21.15.172:0/3515809410 shutdown_connections 2024-04-14T08:36:43.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e86008640 1 --2- 172.21.15.172:0/3515809410 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0e6006c8f0 0x7f0e6006edb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e86008640 1 --2- 172.21.15.172:0/3515809410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e80162b70 0x7f0e8015f7b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e86008640 1 --2- 172.21.15.172:0/3515809410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e80162710 0x7f0e800b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e86008640 1 --2- 172.21.15.172:0/3515809410 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e8015e430 0x7f0e800b97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e86008640 1 -- 172.21.15.172:0/3515809410 >> 172.21.15.172:0/3515809410 conn(0x7f0e8009d560 msgr2=0x7f0e8015c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e86008640 1 -- 172.21.15.172:0/3515809410 shutdown_connections 2024-04-14T08:36:43.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.446+0000 7f0e86008640 1 -- 172.21.15.172:0/3515809410 wait complete. 2024-04-14T08:36:43.459 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-14T08:36:43.459 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:36:43.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.558+0000 7f3ee1c36640 1 Processor -- start 2024-04-14T08:36:43.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.558+0000 7f3ee1c36640 1 -- start start 2024-04-14T08:36:43.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.559+0000 7f3ee1c36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3edc0b1df0 0x7f3edc0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.559+0000 7f3ee1c36640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3edc15c770 con 0x7f3edc0b21d0 2024-04-14T08:36:43.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.559+0000 7f3ee1c36640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3edc0b3430 con 0x7f3edc0b4040 2024-04-14T08:36:43.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.559+0000 7f3ee1c36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f3edc0b3700 con 0x7f3edc0b1df0 2024-04-14T08:36:43.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.559+0000 7f3edaffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3edc0b21d0 0x7f3edc15c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47010/0 (socket says 172.21.15.172:47010) 2024-04-14T08:36:43.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.559+0000 7f3edaffd640 1 -- 172.21.15.172:0/2576444593 learned_addr learned my addr 172.21.15.172:0/2576444593 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:43.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.560+0000 7f3edb7fe640 1 --2- 172.21.15.172:0/2576444593 >> v2:172.21.15.177:3300/0 conn(0x7f3edc0b1df0 0x7f3edc0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.560+0000 7f3eda7fc640 1 -- 172.21.15.172:0/2576444593 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3616123476 0 0) 0x7f3edc15c770 con 0x7f3edc0b21d0 2024-04-14T08:36:43.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.560+0000 7f3eda7fc640 1 -- 172.21.15.172:0/2576444593 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ec4003520 con 0x7f3edc0b21d0 2024-04-14T08:36:43.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.560+0000 7f3edb7fe640 1 -- 172.21.15.172:0/2576444593 >> v1:172.21.15.177:6789/0 conn(0x7f3edc0b4040 legacy=0x7f3edc0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.560+0000 7f3eda7fc640 1 -- 172.21.15.172:0/2576444593 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 317786821 0 0) 0x7f3edc0b3430 con 0x7f3edc0b4040 2024-04-14T08:36:43.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.560+0000 7f3edb7fe640 1 -- 172.21.15.172:0/2576444593 >> v1:172.21.15.172:6789/0 conn(0x7f3edc0b21d0 legacy=0x7f3edc15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.560+0000 7f3edb7fe640 1 -- 172.21.15.172:0/2576444593 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc0b39e0 con 0x7f3edc0b1df0 2024-04-14T08:36:43.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.561+0000 7f3edb7fe640 1 --2- 172.21.15.172:0/2576444593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3edc0b1df0 0x7f3edc0b2ef0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097a0 tx=0x7f3ecc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=985a19b625363a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.561+0000 7f3eda7fc640 1 -- 172.21.15.172:0/2576444593 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc034070 con 0x7f3edc0b1df0 2024-04-14T08:36:43.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.561+0000 7f3eda7fc640 1 -- 172.21.15.172:0/2576444593 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ecc030610 con 0x7f3edc0b1df0 2024-04-14T08:36:43.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.561+0000 7f3eda7fc640 1 -- 172.21.15.172:0/2576444593 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc0309c0 con 0x7f3edc0b1df0 2024-04-14T08:36:43.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.561+0000 7f3ee1c36640 1 -- 172.21.15.172:0/2576444593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3edc0b1df0 msgr2=0x7f3edc0b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.561+0000 7f3ee1c36640 1 --2- 172.21.15.172:0/2576444593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3edc0b1df0 0x7f3edc0b2ef0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097a0 tx=0x7f3ecc02b960 comp rx=0 tx=0).stop 2024-04-14T08:36:43.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.561+0000 7f3ee1c36640 1 -- 172.21.15.172:0/2576444593 shutdown_connections 2024-04-14T08:36:43.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.561+0000 7f3ee1c36640 1 --2- 172.21.15.172:0/2576444593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3edc0b1df0 0x7f3edc0b2ef0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.561+0000 7f3ee1c36640 1 -- 172.21.15.172:0/2576444593 >> 172.21.15.172:0/2576444593 conn(0x7f3edc06ec30 msgr2=0x7f3edc06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.561+0000 7f3ee1c36640 1 -- 172.21.15.172:0/2576444593 shutdown_connections 2024-04-14T08:36:43.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.561+0000 7f3ee1c36640 1 -- 172.21.15.172:0/2576444593 wait complete. 2024-04-14T08:36:43.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.562+0000 7f3ee1c36640 1 Processor -- start 2024-04-14T08:36:43.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.563+0000 7f3ee1c36640 1 -- start start 2024-04-14T08:36:43.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.563+0000 7f3ee1c36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3edc0b1df0 0x7f3edc0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.563+0000 7f3ee1c36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3edc0b21d0 0x7f3edc0cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.563+0000 7f3ee1c36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3edc0b4040 0x7f3edc1f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.563+0000 7f3ee1c36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc15b7f0 con 0x7f3edc0b1df0 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.563+0000 7f3ee1c36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc15bb40 con 0x7f3edc0b4040 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.563+0000 7f3ee1c36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3edc1552d0 con 0x7f3edc0b21d0 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3edb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3edc0b1df0 0x7f3edc0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3edaffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3edc0b21d0 0x7f3edc0cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3edb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3edc0b1df0 0x7f3edc0ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43368/0 (socket says 172.21.15.172:43368) 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3edb7fe640 1 -- 172.21.15.172:0/1121370857 learned_addr learned my addr 172.21.15.172:0/1121370857 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3edbfff640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3edc0b4040 0x7f3edc1f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3edb7fe640 1 -- 172.21.15.172:0/1121370857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3edc0b21d0 msgr2=0x7f3edc0cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3edb7fe640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3edc0b21d0 0x7f3edc0cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3edb7fe640 1 -- 172.21.15.172:0/1121370857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3edc0b4040 msgr2=0x7f3edc1f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3edb7fe640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3edc0b4040 0x7f3edc1f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3edb7fe640 1 -- 172.21.15.172:0/1121370857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc0ce680 con 0x7f3edc0b1df0 2024-04-14T08:36:43.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3edb7fe640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3edc0b1df0 0x7f3edc0ce140 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc02b9a0 tx=0x7f3ecc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3ec3fff640 1 -- 172.21.15.172:0/1121370857 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc034040 con 0x7f3edc0b1df0 2024-04-14T08:36:43.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3ec3fff640 1 -- 172.21.15.172:0/1121370857 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ecc030410 con 0x7f3edc0b1df0 2024-04-14T08:36:43.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3ec3fff640 1 -- 172.21.15.172:0/1121370857 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc039c70 con 0x7f3edc0b1df0 2024-04-14T08:36:43.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3ee1c36640 1 -- 172.21.15.172:0/1121370857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3edc0ce950 con 0x7f3edc0b1df0 2024-04-14T08:36:43.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.564+0000 7f3ee1c36640 1 -- 172.21.15.172:0/1121370857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3edc0cef10 con 0x7f3edc0b1df0 2024-04-14T08:36:43.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.565+0000 7f3ee1c36640 1 -- 172.21.15.172:0/1121370857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3edc068520 con 0x7f3edc0b1df0 2024-04-14T08:36:43.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.566+0000 7f3ec3fff640 1 -- 172.21.15.172:0/1121370857 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ecc037070 con 0x7f3edc0b1df0 2024-04-14T08:36:43.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.566+0000 7f3ec3fff640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ec806c8f0 0x7f3ec806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.566+0000 7f3ec3fff640 1 -- 172.21.15.172:0/1121370857 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f3ecc0adc10 con 0x7f3edc0b1df0 2024-04-14T08:36:43.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.566+0000 7f3edaffd640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ec806c8f0 0x7f3ec806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.566+0000 7f3edaffd640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ec806c8f0 0x7f3ec806edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0aebe0 tx=0x7f3ed0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.571+0000 7f3ec3fff640 1 -- 172.21.15.172:0/1121370857 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3ecc07b7d0 con 0x7f3edc0b1df0 2024-04-14T08:36:43.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.783+0000 7f3ee1c36640 1 -- 172.21.15.172:0/1121370857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3edc0b1380 con 0x7f3edc0b1df0 2024-04-14T08:36:43.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.783+0000 7f3ec3fff640 1 -- 172.21.15.172:0/1121370857 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5195 (secure 0 0 0) 0x7f3ecc07fad0 con 0x7f3edc0b1df0 2024-04-14T08:36:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:43.785 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3621780015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3621780015},{"type":"v1","addr":"172.21.15.172:6836","nonce":3621780015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3894683748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3894683748},{"type":"v1","addr":"172.21.15.172:6837","nonce":3894683748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3625745587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3625745587},{"type":"v1","addr":"172.21.15.177:6834","nonce":3625745587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:35:34.316542+0000","modified":"2024-04-14T08:36:14.993490+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6835/4176013980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":4176013980},{"type":"v1","addr":"172.21.15.177:6835","nonce":4176013980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-14T08:36:43.786 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 32 2024-04-14T08:36:43.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.786+0000 7f3ee1c36640 1 -- 172.21.15.172:0/1121370857 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ec806c8f0 msgr2=0x7f3ec806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.786+0000 7f3ee1c36640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ec806c8f0 0x7f3ec806edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0aebe0 tx=0x7f3ed0009040 comp rx=0 tx=0).stop 2024-04-14T08:36:43.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.786+0000 7f3ee1c36640 1 -- 172.21.15.172:0/1121370857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3edc0b1df0 msgr2=0x7f3edc0ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.786+0000 7f3ee1c36640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3edc0b1df0 0x7f3edc0ce140 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc02b9a0 tx=0x7f3ecc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.786+0000 7f3ee1c36640 1 -- 172.21.15.172:0/1121370857 shutdown_connections 2024-04-14T08:36:43.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.786+0000 7f3ee1c36640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ec806c8f0 0x7f3ec806edb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.786+0000 7f3ee1c36640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3edc0b4040 0x7f3edc1f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.786+0000 7f3ee1c36640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3edc0b21d0 0x7f3edc0cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.786+0000 7f3ee1c36640 1 --2- 172.21.15.172:0/1121370857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3edc0b1df0 0x7f3edc0ce140 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.786+0000 7f3ee1c36640 1 -- 172.21.15.172:0/1121370857 >> 172.21.15.172:0/1121370857 conn(0x7f3edc06ec30 msgr2=0x7f3edc0728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.786+0000 7f3ee1c36640 1 -- 172.21.15.172:0/1121370857 shutdown_connections 2024-04-14T08:36:43.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.786+0000 7f3ee1c36640 1 -- 172.21.15.172:0/1121370857 wait complete. 2024-04-14T08:36:43.799 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-04-14T08:36:43.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.900+0000 7f26dc420640 1 Processor -- start 2024-04-14T08:36:43.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.900+0000 7f26dc420640 1 -- start start 2024-04-14T08:36:43.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.901+0000 7f26dc420640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26d415b770 0x7f26d40cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.901+0000 7f26dc420640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26d40cf430 0x7f26d40731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.901+0000 7f26dc420640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d40ce390 con 0x7f26d415bb50 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.901+0000 7f26dc420640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f26d40ce670 con 0x7f26d40cf430 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.901+0000 7f26dc420640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f26d40ce950 con 0x7f26d415b770 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.902+0000 7f26d99bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26d40cf430 0x7f26d40731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.902+0000 7f26da9bd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f26d415bb50 0x7f26d415bfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47018/0 (socket says 172.21.15.172:47018) 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.902+0000 7f26da9bd640 1 -- 172.21.15.172:0/479396201 learned_addr learned my addr 172.21.15.172:0/479396201 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.902+0000 7f26da1bc640 1 --2- 172.21.15.172:0/479396201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26d415b770 0x7f26d40cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.902+0000 7f26da1bc640 1 -- 172.21.15.172:0/479396201 >> v2:172.21.15.172:3300/0 conn(0x7f26d40cf430 msgr2=0x7f26d40731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.902+0000 7f26da1bc640 1 --2- 172.21.15.172:0/479396201 >> v2:172.21.15.172:3300/0 conn(0x7f26d40cf430 0x7f26d40731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.902+0000 7f26da1bc640 1 -- 172.21.15.172:0/479396201 >> v1:172.21.15.172:6789/0 conn(0x7f26d415bb50 legacy=0x7f26d415bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.902+0000 7f26da1bc640 1 -- 172.21.15.172:0/479396201 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d40cec30 con 0x7f26d415b770 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.902+0000 7f26d91ba640 1 -- 172.21.15.172:0/479396201 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1074635449 0 0) 0x7f26d40ce390 con 0x7f26d415bb50 2024-04-14T08:36:43.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.902+0000 7f26d99bb640 1 --2- 172.21.15.172:0/479396201 >> v2:172.21.15.172:3300/0 conn(0x7f26d40cf430 0x7f26d40731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:43.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.903+0000 7f26da1bc640 1 --2- 172.21.15.172:0/479396201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26d415b770 0x7f26d40cddb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f26d00097a0 tx=0x7f26d002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=640f111e221efb8d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.903+0000 7f26d91ba640 1 -- 172.21.15.172:0/479396201 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0039070 con 0x7f26d415b770 2024-04-14T08:36:43.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.903+0000 7f26d91ba640 1 -- 172.21.15.172:0/479396201 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d00316f0 con 0x7f26d415b770 2024-04-14T08:36:43.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.903+0000 7f26d91ba640 1 -- 172.21.15.172:0/479396201 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0031ac0 con 0x7f26d415b770 2024-04-14T08:36:43.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.903+0000 7f26dc420640 1 -- 172.21.15.172:0/479396201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26d415b770 msgr2=0x7f26d40cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.903+0000 7f26dc420640 1 --2- 172.21.15.172:0/479396201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26d415b770 0x7f26d40cddb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f26d00097a0 tx=0x7f26d002b790 comp rx=0 tx=0).stop 2024-04-14T08:36:43.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.903+0000 7f26dc420640 1 -- 172.21.15.172:0/479396201 shutdown_connections 2024-04-14T08:36:43.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.903+0000 7f26dc420640 1 --2- 172.21.15.172:0/479396201 >> v2:172.21.15.172:3300/0 conn(0x7f26d40cf430 0x7f26d40731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.903+0000 7f26dc420640 1 --2- 172.21.15.172:0/479396201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26d415b770 0x7f26d40cddb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.903+0000 7f26dc420640 1 -- 172.21.15.172:0/479396201 >> 172.21.15.172:0/479396201 conn(0x7f26d40b1c60 msgr2=0x7f26d40b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.904+0000 7f26dc420640 1 -- 172.21.15.172:0/479396201 shutdown_connections 2024-04-14T08:36:43.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.904+0000 7f26dc420640 1 -- 172.21.15.172:0/479396201 wait complete. 2024-04-14T08:36:43.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.905+0000 7f26dc420640 1 Processor -- start 2024-04-14T08:36:43.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.905+0000 7f26dc420640 1 -- start start 2024-04-14T08:36:43.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.905+0000 7f26dc420640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d40cf430 0x7f26d41411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.905+0000 7f26dc420640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d415b770 0x7f26d406de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.905+0000 7f26dc420640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26d415bb50 0x7f26d406e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.905+0000 7f26dc420640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26d415c8c0 con 0x7f26d415b770 2024-04-14T08:36:43.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.905+0000 7f26dc420640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f26d40cf890 con 0x7f26d40cf430 2024-04-14T08:36:43.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.905+0000 7f26dc420640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f26d415ca40 con 0x7f26d415bb50 2024-04-14T08:36:43.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.905+0000 7f26d99bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d415b770 0x7f26d406de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.905+0000 7f26d99bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d415b770 0x7f26d406de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43376/0 (socket says 172.21.15.172:43376) 2024-04-14T08:36:43.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.905+0000 7f26d99bb640 1 -- 172.21.15.172:0/4228014789 learned_addr learned my addr 172.21.15.172:0/4228014789 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:43.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26da1bc640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d40cf430 0x7f26d41411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26da9bd640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26d415bb50 0x7f26d406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26d99bb640 1 -- 172.21.15.172:0/4228014789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26d415bb50 msgr2=0x7f26d406e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26d99bb640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26d415bb50 0x7f26d406e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26d99bb640 1 -- 172.21.15.172:0/4228014789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d40cf430 msgr2=0x7f26d41411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26d99bb640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d40cf430 0x7f26d41411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26d99bb640 1 -- 172.21.15.172:0/4228014789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d40caf80 con 0x7f26d415b770 2024-04-14T08:36:43.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26d99bb640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d415b770 0x7f26d406de40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f26c400e990 tx=0x7f26c400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26caffd640 1 -- 172.21.15.172:0/4228014789 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4017070 con 0x7f26d415b770 2024-04-14T08:36:43.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26caffd640 1 -- 172.21.15.172:0/4228014789 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c400c9e0 con 0x7f26d415b770 2024-04-14T08:36:43.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26dc420640 1 -- 172.21.15.172:0/4228014789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d40cb260 con 0x7f26d415b770 2024-04-14T08:36:43.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26caffd640 1 -- 172.21.15.172:0/4228014789 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c400cdb0 con 0x7f26d415b770 2024-04-14T08:36:43.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.906+0000 7f26dc420640 1 -- 172.21.15.172:0/4228014789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d40cb820 con 0x7f26d415b770 2024-04-14T08:36:43.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.908+0000 7f26caffd640 1 -- 172.21.15.172:0/4228014789 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26c401a070 con 0x7f26d415b770 2024-04-14T08:36:43.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.908+0000 7f26caffd640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f26b806c8f0 0x7f26b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.908+0000 7f26da1bc640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f26b806c8f0 0x7f26b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.908+0000 7f26caffd640 1 -- 172.21.15.172:0/4228014789 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f26c408bea0 con 0x7f26d415b770 2024-04-14T08:36:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.909+0000 7f26da1bc640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f26b806c8f0 0x7f26b806edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f26d0036000 tx=0x7f26d0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.909+0000 7f26df207740 1 Processor -- start 2024-04-14T08:36:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.910+0000 7f26df207740 1 -- start start 2024-04-14T08:36:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.910+0000 7f26df207740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf7369e0 0x564fcf7071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.910+0000 7f26df207740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf733530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.910+0000 7f26df207740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf733ef0 0x564fcf702ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.910+0000 7f26df207740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564fcf6ca5a0 con 0x564fcf736dc0 2024-04-14T08:36:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.910+0000 7f26df207740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564fcf66c680 con 0x564fcf733ef0 2024-04-14T08:36:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.910+0000 7f26df207740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x564fcf60a860 con 0x564fcf7369e0 2024-04-14T08:36:43.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.910+0000 7f26d99bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf733530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.910+0000 7f26da1bc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf7369e0 0x564fcf7071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.911+0000 7f26d99bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf733530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43380/0 (socket says 172.21.15.172:43380) 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.911+0000 7f26d99bb640 1 -- 172.21.15.172:0/193289492 learned_addr learned my addr 172.21.15.172:0/193289492 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.911+0000 7f26da9bd640 1 --2- 172.21.15.172:0/193289492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf733ef0 0x564fcf702ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.911+0000 7f26d99bb640 1 -- 172.21.15.172:0/193289492 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf7369e0 msgr2=0x564fcf7071a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.911+0000 7f26d99bb640 1 --2- 172.21.15.172:0/193289492 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf7369e0 0x564fcf7071a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.911+0000 7f26d99bb640 1 -- 172.21.15.172:0/193289492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf733ef0 msgr2=0x564fcf702ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.911+0000 7f26d99bb640 1 --2- 172.21.15.172:0/193289492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf733ef0 0x564fcf702ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.911+0000 7f26d99bb640 1 -- 172.21.15.172:0/193289492 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fcf6391a0 con 0x564fcf736dc0 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.911+0000 7f26d99bb640 1 --2- 172.21.15.172:0/193289492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf733530 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f26c408a180 tx=0x7f26c4059550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54da650bc8d7b71b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.911+0000 7f26b7fff640 1 -- 172.21.15.172:0/193289492 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4021070 con 0x564fcf736dc0 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.911+0000 7f26b7fff640 1 -- 172.21.15.172:0/193289492 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c4057b90 con 0x564fcf736dc0 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.911+0000 7f26b7fff640 1 -- 172.21.15.172:0/193289492 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4056160 con 0x564fcf736dc0 2024-04-14T08:36:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.912+0000 7f26df207740 1 -- 172.21.15.172:0/193289492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 msgr2=0x564fcf733530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.912+0000 7f26df207740 1 --2- 172.21.15.172:0/193289492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf733530 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f26c408a180 tx=0x7f26c4059550 comp rx=0 tx=0).stop 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.912+0000 7f26df207740 1 -- 172.21.15.172:0/193289492 shutdown_connections 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.912+0000 7f26df207740 1 --2- 172.21.15.172:0/193289492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf733ef0 0x564fcf702ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.912+0000 7f26df207740 1 --2- 172.21.15.172:0/193289492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf733530 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.912+0000 7f26df207740 1 --2- 172.21.15.172:0/193289492 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf7369e0 0x564fcf7071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.912+0000 7f26df207740 1 -- 172.21.15.172:0/193289492 >> 172.21.15.172:0/193289492 conn(0x564fcf704e10 msgr2=0x564fcf7066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.912+0000 7f26df207740 1 -- 172.21.15.172:0/193289492 shutdown_connections 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.912+0000 7f26df207740 1 -- 172.21.15.172:0/193289492 wait complete. 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.912+0000 7f26df207740 1 -- start start 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.912+0000 7f26df207740 1 Processor -- start 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.913+0000 7f26df207740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf7369e0 0x564fcf74a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.913+0000 7f26df207740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf7574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.913+0000 7f26df207740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf762120 0x564fcf762580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.913+0000 7f26df207740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564fcf6081d0 con 0x564fcf736dc0 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.913+0000 7f26df207740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564fcf668630 con 0x564fcf762120 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.913+0000 7f26df207740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x564fcf74fe00 con 0x564fcf7369e0 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.913+0000 7f26df207740 10 client.0 _ll_register_callbacks cb 0x564fcf62b010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.913+0000 7f26d99bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf7574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.913+0000 7f26d99bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf7574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43388/0 (socket says 172.21.15.172:43388) 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.913+0000 7f26d99bb640 1 -- 172.21.15.172:0/3024989599 learned_addr learned my addr 172.21.15.172:0/3024989599 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.913+0000 7f26da1bc640 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf7369e0 0x564fcf74a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26da9bd640 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf762120 0x564fcf762580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26d99bb640 1 -- 172.21.15.172:0/3024989599 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf7369e0 msgr2=0x564fcf74a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26d99bb640 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf7369e0 0x564fcf74a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26d99bb640 1 -- 172.21.15.172:0/3024989599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf762120 msgr2=0x564fcf762580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26d99bb640 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf762120 0x564fcf762580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26d99bb640 1 -- 172.21.15.172:0/3024989599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fcf44c7c0 con 0x564fcf736dc0 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26d99bb640 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf7574e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f26c400e960 tx=0x7f26c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd953048a4d9265a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26b57fa640 1 -- 172.21.15.172:0/3024989599 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40602c0 con 0x564fcf736dc0 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26b57fa640 1 -- 172.21.15.172:0/3024989599 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f26c4083ba0 con 0x564fcf736dc0 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26df207740 1 -- 172.21.15.172:0/3024989599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c4057b90 con 0x564fcf736dc0 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26b57fa640 1 -- 172.21.15.172:0/3024989599 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f26c4083da0 con 0x564fcf736dc0 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26df207740 10 client.6540 fetch_fsmap learned FSMap version 32 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26df207740 1 -- 172.21.15.172:0/3024989599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x564fcf4acf50 con 0x564fcf736dc0 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26b57fa640 1 -- 172.21.15.172:0/3024989599 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c4057b90 con 0x564fcf736dc0 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26b57fa640 1 -- 172.21.15.172:0/3024989599 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7f26c4058f40 con 0x564fcf736dc0 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26df207740 10 client.6540 fetch_fsmap finished waiting for FSMap version 32 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26df207740 10 client.6540 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26df207740 20 client.6540 populate_metadata read hostname 'smithi172' 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26df207740 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x564fcf734f40 0x564fcf76fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26df207740 4 client.6540 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:36:43.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.914+0000 7f26df207740 1 -- 172.21.15.172:0/3024989599 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fcf67feb0 con 0x564fcf734f40 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.915+0000 7f26da1bc640 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x564fcf734f40 0x564fcf76fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.915+0000 7f26da1bc640 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x564fcf734f40 0x564fcf76fe50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=381f7d51563baaed server_cookie=f77a6306aa110503 in_seq=0 out_seq=0 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.915+0000 7f26b57fa640 10 client.6540 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.915+0000 7f26b57fa640 1 -- 172.21.15.172:0/3024989599 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fcf67feb0 con 0x564fcf734f40 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.915+0000 7f26b57fa640 10 client.6540 handle_command_reply: tid=0 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26df207740 1 client.6540 shutdown 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26df207740 1 -- 172.21.15.172:0/3024989599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 msgr2=0x564fcf7574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26df207740 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf7574e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f26c400e960 tx=0x7f26c4000bc0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26df207740 1 -- 172.21.15.172:0/3024989599 shutdown_connections 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26df207740 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x564fcf734f40 0x564fcf76fe50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26df207740 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf762120 0x564fcf762580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26df207740 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf7574e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26df207740 1 --2- 172.21.15.172:0/3024989599 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf7369e0 0x564fcf74a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26b57fa640 0 client.6540 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:43.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26df207740 1 -- 172.21.15.172:0/3024989599 >> 172.21.15.172:0/3024989599 conn(0x564fcf733ef0 msgr2=0x564fcf74f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26df207740 1 -- 172.21.15.172:0/3024989599 shutdown_connections 2024-04-14T08:36:43.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26df207740 1 -- 172.21.15.172:0/3024989599 wait complete. 2024-04-14T08:36:43.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.916+0000 7f26df207740 20 client.6540 trim_cache size 0 max 16384 2024-04-14T08:36:43.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.929+0000 7f26df207740 1 Processor -- start 2024-04-14T08:36:43.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.929+0000 7f26df207740 1 -- start start 2024-04-14T08:36:43.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26df207740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf7369e0 0x564fcf72c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26df207740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf7755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26df207740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf762120 0x564fcf7590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26df207740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26d40c9b10 con 0x564fcf736dc0 2024-04-14T08:36:43.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26df207740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f26d40c8e10 con 0x564fcf7369e0 2024-04-14T08:36:43.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26df207740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f26d40c9460 con 0x564fcf762120 2024-04-14T08:36:43.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26d99bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf7755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26d99bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf7755c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43398/0 (socket says 172.21.15.172:43398) 2024-04-14T08:36:43.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26d99bb640 1 -- 172.21.15.172:0/720480283 learned_addr learned my addr 172.21.15.172:0/720480283 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:43.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26da9bd640 1 --2- 172.21.15.172:0/720480283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf762120 0x564fcf7590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26da1bc640 1 --2- 172.21.15.172:0/720480283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf7369e0 0x564fcf72c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26d99bb640 1 -- 172.21.15.172:0/720480283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf762120 msgr2=0x564fcf7590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26d99bb640 1 --2- 172.21.15.172:0/720480283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf762120 0x564fcf7590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26d99bb640 1 -- 172.21.15.172:0/720480283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf7369e0 msgr2=0x564fcf72c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26d99bb640 1 --2- 172.21.15.172:0/720480283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf7369e0 0x564fcf72c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26d99bb640 1 -- 172.21.15.172:0/720480283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fcf7628f0 con 0x564fcf736dc0 2024-04-14T08:36:43.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26da1bc640 1 --2- 172.21.15.172:0/720480283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf7369e0 0x564fcf72c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:36:43.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26d99bb640 1 --2- 172.21.15.172:0/720480283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf7755c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f26c4072190 tx=0x7f26c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170dcd513a0e675c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26b77fe640 1 -- 172.21.15.172:0/720480283 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4021070 con 0x564fcf736dc0 2024-04-14T08:36:43.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26b77fe640 1 -- 172.21.15.172:0/720480283 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c4058f40 con 0x564fcf736dc0 2024-04-14T08:36:43.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.930+0000 7f26b77fe640 1 -- 172.21.15.172:0/720480283 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40602b0 con 0x564fcf736dc0 2024-04-14T08:36:43.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.931+0000 7f26df207740 1 -- 172.21.15.172:0/720480283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 msgr2=0x564fcf7755c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.931+0000 7f26df207740 1 --2- 172.21.15.172:0/720480283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf7755c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f26c4072190 tx=0x7f26c4014000 comp rx=0 tx=0).stop 2024-04-14T08:36:43.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.931+0000 7f26df207740 1 -- 172.21.15.172:0/720480283 shutdown_connections 2024-04-14T08:36:43.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.931+0000 7f26df207740 1 --2- 172.21.15.172:0/720480283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf762120 0x564fcf7590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.931+0000 7f26df207740 1 --2- 172.21.15.172:0/720480283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf736dc0 0x564fcf7755c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.931+0000 7f26df207740 1 --2- 172.21.15.172:0/720480283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf7369e0 0x564fcf72c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.931+0000 7f26df207740 1 -- 172.21.15.172:0/720480283 >> 172.21.15.172:0/720480283 conn(0x564fcf733ef0 msgr2=0x564fcf709580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.931+0000 7f26df207740 1 -- 172.21.15.172:0/720480283 shutdown_connections 2024-04-14T08:36:43.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.931+0000 7f26df207740 1 -- 172.21.15.172:0/720480283 wait complete. 2024-04-14T08:36:43.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.932+0000 7f26df207740 1 -- start start 2024-04-14T08:36:43.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.932+0000 7f26df207740 1 Processor -- start 2024-04-14T08:36:43.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.932+0000 7f26df207740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf7369e0 0x564fcf7cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.932+0000 7f26df207740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf736dc0 0x564fcf7095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.932+0000 7f26df207740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf762120 0x564fcf773e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.932+0000 7f26df207740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564fcf6081d0 con 0x564fcf7369e0 2024-04-14T08:36:43.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26df207740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564fcf668630 con 0x564fcf736dc0 2024-04-14T08:36:43.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26df207740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x564fcf735610 con 0x564fcf762120 2024-04-14T08:36:43.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26df207740 10 client.0 _ll_register_callbacks cb 0x7f26c4071070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:36:43.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26da1bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf7369e0 0x564fcf7cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26da1bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf7369e0 0x564fcf7cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43406/0 (socket says 172.21.15.172:43406) 2024-04-14T08:36:43.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26da1bc640 1 -- 172.21.15.172:0/3850243221 learned_addr learned my addr 172.21.15.172:0/3850243221 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:43.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26da9bd640 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf762120 0x564fcf773e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26d99bb640 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf736dc0 0x564fcf7095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26da1bc640 1 -- 172.21.15.172:0/3850243221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf762120 msgr2=0x564fcf773e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26da1bc640 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf762120 0x564fcf773e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26da1bc640 1 -- 172.21.15.172:0/3850243221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf736dc0 msgr2=0x564fcf7095f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26da1bc640 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf736dc0 0x564fcf7095f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26da1bc640 1 -- 172.21.15.172:0/3850243221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fcf3400b0 con 0x564fcf7369e0 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26da1bc640 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf7369e0 0x564fcf7cbc70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f26d0000c00 tx=0x7f26d002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8b0e443e7f9b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26b57fa640 1 -- 172.21.15.172:0/3850243221 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0058060 con 0x564fcf7369e0 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26b57fa640 1 -- 172.21.15.172:0/3850243221 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f26d0031d50 con 0x564fcf7369e0 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26df207740 1 -- 172.21.15.172:0/3850243221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c4058f40 con 0x564fcf7369e0 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26b57fa640 1 -- 172.21.15.172:0/3850243221 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f26d0042040 con 0x564fcf7369e0 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26df207740 10 client.6546 fetch_fsmap learned FSMap version 32 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26df207740 1 -- 172.21.15.172:0/3850243221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x564fcf708450 con 0x564fcf7369e0 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.933+0000 7f26b57fa640 1 -- 172.21.15.172:0/3850243221 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c4058f40 con 0x564fcf7369e0 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26b57fa640 1 -- 172.21.15.172:0/3850243221 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 32) ==== 3494+0+0 (secure 0 0 0) 0x7f26d0053b50 con 0x564fcf7369e0 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 10 client.6546 fetch_fsmap finished waiting for FSMap version 32 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 10 client.6546 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:36:43.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 20 client.6546 populate_metadata read hostname 'smithi172' 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x564fcf75ebc0 0x564fcf74d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 4 client.6546 mds_command: new command op to 6204 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 1 -- 172.21.15.172:0/3850243221 --> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x564fcf7490c0 con 0x564fcf75ebc0 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26d99bb640 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x564fcf75ebc0 0x564fcf74d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26b57fa640 10 client.6546 ms_handle_connect on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26d99bb640 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x564fcf75ebc0 0x564fcf74d350 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21ea1fcc1bf33b0e server_cookie=fa38c505613e96e4 in_seq=0 out_seq=0 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26b57fa640 1 -- 172.21.15.172:0/3850243221 <== mds.0 v2:172.21.15.177:6833/4176013980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564fcf7490c0 con 0x564fcf75ebc0 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26b57fa640 10 client.6546 handle_command_reply: tid=0 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 1 client.6546 shutdown 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 1 -- 172.21.15.172:0/3850243221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf7369e0 msgr2=0x564fcf7cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf7369e0 0x564fcf7cbc70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f26d0000c00 tx=0x7f26d002e000 comp rx=0 tx=0).stop 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 1 -- 172.21.15.172:0/3850243221 shutdown_connections 2024-04-14T08:36:43.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.177:6833/4176013980,v1:172.21.15.177:6835/4176013980] conn(0x564fcf75ebc0 0x564fcf74d350 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fcf762120 0x564fcf773e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fcf736dc0 0x564fcf7095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26b57fa640 0 client.6546 ms_handle_reset on v2:172.21.15.177:6833/4176013980 2024-04-14T08:36:43.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 1 --2- 172.21.15.172:0/3850243221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fcf7369e0 0x564fcf7cbc70 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.934+0000 7f26df207740 1 -- 172.21.15.172:0/3850243221 >> 172.21.15.172:0/3850243221 conn(0x564fcf733ef0 msgr2=0x564fcf74abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26df207740 1 -- 172.21.15.172:0/3850243221 shutdown_connections 2024-04-14T08:36:43.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26df207740 1 -- 172.21.15.172:0/3850243221 wait complete. 2024-04-14T08:36:43.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26df207740 20 client.6546 trim_cache size 0 max 16384 2024-04-14T08:36:43.943 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:43.944 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-04-14T08:36:43.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26dc420640 1 -- 172.21.15.172:0/4228014789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f26b806c8f0 msgr2=0x7f26b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26dc420640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f26b806c8f0 0x7f26b806edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f26d0036000 tx=0x7f26d0008030 comp rx=0 tx=0).stop 2024-04-14T08:36:43.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26dc420640 1 -- 172.21.15.172:0/4228014789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d415b770 msgr2=0x7f26d406de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:43.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26dc420640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d415b770 0x7f26d406de40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f26c400e990 tx=0x7f26c400ee50 comp rx=0 tx=0).stop 2024-04-14T08:36:43.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26dc420640 1 -- 172.21.15.172:0/4228014789 shutdown_connections 2024-04-14T08:36:43.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26dc420640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f26b806c8f0 0x7f26b806edb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26dc420640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26d415bb50 0x7f26d406e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26dc420640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26d415b770 0x7f26d406de40 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26dc420640 1 --2- 172.21.15.172:0/4228014789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d40cf430 0x7f26d41411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:43.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26dc420640 1 -- 172.21.15.172:0/4228014789 >> 172.21.15.172:0/4228014789 conn(0x7f26d40b1c60 msgr2=0x7f26d415ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:43.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26dc420640 1 -- 172.21.15.172:0/4228014789 shutdown_connections 2024-04-14T08:36:43.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:43.935+0000 7f26dc420640 1 -- 172.21.15.172:0/4228014789 wait complete. 2024-04-14T08:36:43.947 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-14T08:36:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.046+0000 7ff09d1c8640 1 Processor -- start 2024-04-14T08:36:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.046+0000 7ff09d1c8640 1 -- start start 2024-04-14T08:36:44.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.046+0000 7ff09d1c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff0980b1df0 0x7ff0980b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.047+0000 7ff09d1c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09815c770 con 0x7ff0980b4040 2024-04-14T08:36:44.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.047+0000 7ff09d1c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0980b3430 con 0x7ff0980b21d0 2024-04-14T08:36:44.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.047+0000 7ff09d1c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff0980b3700 con 0x7ff0980b1df0 2024-04-14T08:36:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.047+0000 7ff09759e640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff0980b4040 0x7ff0980b4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47026/0 (socket says 172.21.15.172:47026) 2024-04-14T08:36:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.047+0000 7ff09759e640 1 -- 172.21.15.172:0/1746080765 learned_addr learned my addr 172.21.15.172:0/1746080765 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.047+0000 7ff096d9d640 1 --2- 172.21.15.172:0/1746080765 >> v2:172.21.15.177:3300/0 conn(0x7ff0980b1df0 0x7ff0980b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.047+0000 7ff095d9b640 1 -- 172.21.15.172:0/1746080765 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595417620 0 0) 0x7ff09815c770 con 0x7ff0980b4040 2024-04-14T08:36:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.047+0000 7ff095d9b640 1 -- 172.21.15.172:0/1746080765 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff080003520 con 0x7ff0980b4040 2024-04-14T08:36:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.047+0000 7ff096d9d640 1 -- 172.21.15.172:0/1746080765 >> v1:172.21.15.177:6789/0 conn(0x7ff0980b21d0 legacy=0x7ff09815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.047+0000 7ff096d9d640 1 -- 172.21.15.172:0/1746080765 >> v1:172.21.15.172:6789/0 conn(0x7ff0980b4040 legacy=0x7ff0980b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.047+0000 7ff096d9d640 1 -- 172.21.15.172:0/1746080765 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0980b39e0 con 0x7ff0980b1df0 2024-04-14T08:36:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.048+0000 7ff096d9d640 1 --2- 172.21.15.172:0/1746080765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0980b1df0 0x7ff0980b2ef0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff0880097a0 tx=0x7ff08802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebe40da762ec6d05 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:44.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.048+0000 7ff095d9b640 1 -- 172.21.15.172:0/1746080765 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088034070 con 0x7ff0980b1df0 2024-04-14T08:36:44.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.048+0000 7ff095d9b640 1 -- 172.21.15.172:0/1746080765 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff088030610 con 0x7ff0980b1df0 2024-04-14T08:36:44.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.048+0000 7ff095d9b640 1 -- 172.21.15.172:0/1746080765 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0880309c0 con 0x7ff0980b1df0 2024-04-14T08:36:44.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.048+0000 7ff09d1c8640 1 -- 172.21.15.172:0/1746080765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0980b1df0 msgr2=0x7ff0980b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.048+0000 7ff09d1c8640 1 --2- 172.21.15.172:0/1746080765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0980b1df0 0x7ff0980b2ef0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff0880097a0 tx=0x7ff08802b960 comp rx=0 tx=0).stop 2024-04-14T08:36:44.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.049+0000 7ff09d1c8640 1 -- 172.21.15.172:0/1746080765 shutdown_connections 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.049+0000 7ff09d1c8640 1 --2- 172.21.15.172:0/1746080765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0980b1df0 0x7ff0980b2ef0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.049+0000 7ff09d1c8640 1 -- 172.21.15.172:0/1746080765 >> 172.21.15.172:0/1746080765 conn(0x7ff09806ec30 msgr2=0x7ff09806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.049+0000 7ff09d1c8640 1 -- 172.21.15.172:0/1746080765 shutdown_connections 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.049+0000 7ff09d1c8640 1 -- 172.21.15.172:0/1746080765 wait complete. 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.050+0000 7ff09d1c8640 1 Processor -- start 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.050+0000 7ff09d1c8640 1 -- start start 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09d1c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0980b1df0 0x7ff098158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09d1c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0980b21d0 0x7ff09813ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09d1c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0980b4040 0x7ff0980a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09d1c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff09815b7f0 con 0x7ff0980b21d0 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09d1c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff09815bb40 con 0x7ff0980b1df0 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09d1c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff0981552d0 con 0x7ff0980b4040 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0980b21d0 0x7ff09813ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0980b21d0 0x7ff09813ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43414/0 (socket says 172.21.15.172:43414) 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09659c640 1 -- 172.21.15.172:0/3612701570 learned_addr learned my addr 172.21.15.172:0/3612701570 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff096d9d640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0980b1df0 0x7ff098158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09759e640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0980b4040 0x7ff0980a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09659c640 1 -- 172.21.15.172:0/3612701570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0980b4040 msgr2=0x7ff0980a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09659c640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0980b4040 0x7ff0980a9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09659c640 1 -- 172.21.15.172:0/3612701570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0980b1df0 msgr2=0x7ff098158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09659c640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0980b1df0 0x7ff098158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09659c640 1 -- 172.21.15.172:0/3612701570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff098157170 con 0x7ff0980b21d0 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09659c640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0980b21d0 0x7ff09813ad80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c400 tx=0x7ff08c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff07f7fe640 1 -- 172.21.15.172:0/3612701570 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c017070 con 0x7ff0980b21d0 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff07f7fe640 1 -- 172.21.15.172:0/3612701570 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08c0052d0 con 0x7ff0980b21d0 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.051+0000 7ff09d1c8640 1 -- 172.21.15.172:0/3612701570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0981573f0 con 0x7ff0980b21d0 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.052+0000 7ff07f7fe640 1 -- 172.21.15.172:0/3612701570 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c003830 con 0x7ff0980b21d0 2024-04-14T08:36:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.052+0000 7ff09d1c8640 1 -- 172.21.15.172:0/3612701570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff098157c90 con 0x7ff0980b21d0 2024-04-14T08:36:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.053+0000 7ff09d1c8640 1 -- 172.21.15.172:0/3612701570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff098068520 con 0x7ff0980b21d0 2024-04-14T08:36:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.053+0000 7ff07f7fe640 1 -- 172.21.15.172:0/3612701570 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff08c01a070 con 0x7ff0980b21d0 2024-04-14T08:36:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.053+0000 7ff07f7fe640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff08406c8f0 0x7ff08406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.053+0000 7ff07f7fe640 1 -- 172.21.15.172:0/3612701570 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7ff08c08afa0 con 0x7ff0980b21d0 2024-04-14T08:36:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.053+0000 7ff096d9d640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff08406c8f0 0x7ff08406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.054+0000 7ff096d9d640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff08406c8f0 0x7ff08406edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff088036000 tx=0x7ff088008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.058+0000 7ff07f7fe640 1 -- 172.21.15.172:0/3612701570 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff098157170 con 0x7ff0980b21d0 2024-04-14T08:36:44.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.271+0000 7ff09d1c8640 1 -- 172.21.15.172:0/3612701570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff098071f40 con 0x7ff0980b21d0 2024-04-14T08:36:44.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.272+0000 7ff07f7fe640 1 -- 172.21.15.172:0/3612701570 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff08c058bf0 con 0x7ff0980b21d0 2024-04-14T08:36:44.273 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:44.273 INFO:teuthology.orchestra.run.smithi172.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-14T08:36:44.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.274+0000 7ff09d1c8640 1 -- 172.21.15.172:0/3612701570 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff08406c8f0 msgr2=0x7ff08406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.274+0000 7ff09d1c8640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff08406c8f0 0x7ff08406edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff088036000 tx=0x7ff088008030 comp rx=0 tx=0).stop 2024-04-14T08:36:44.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.274+0000 7ff09d1c8640 1 -- 172.21.15.172:0/3612701570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0980b21d0 msgr2=0x7ff09813ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.274+0000 7ff09d1c8640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0980b21d0 0x7ff09813ad80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c400 tx=0x7ff08c00c8c0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.274+0000 7ff09d1c8640 1 -- 172.21.15.172:0/3612701570 shutdown_connections 2024-04-14T08:36:44.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.274+0000 7ff09d1c8640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff08406c8f0 0x7ff08406edb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.274+0000 7ff09d1c8640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff0980b4040 0x7ff0980a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.274+0000 7ff09d1c8640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0980b21d0 0x7ff09813ad80 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.274+0000 7ff09d1c8640 1 --2- 172.21.15.172:0/3612701570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0980b1df0 0x7ff098158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.274+0000 7ff09d1c8640 1 -- 172.21.15.172:0/3612701570 >> 172.21.15.172:0/3612701570 conn(0x7ff09806ec30 msgr2=0x7ff09813c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:44.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.274+0000 7ff09d1c8640 1 -- 172.21.15.172:0/3612701570 shutdown_connections 2024-04-14T08:36:44.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.274+0000 7ff09d1c8640 1 -- 172.21.15.172:0/3612701570 wait complete. 2024-04-14T08:36:44.286 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-14T08:36:44.286 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:36:44.287 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:36:44.327 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:36:44.373 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:36:44.374 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:36:44.374 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:36:44.374 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:36:44.391 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:36:44.391 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:36:44.447 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:36:44.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.586+0000 7fbdaa820640 1 Processor -- start 2024-04-14T08:36:44.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.586+0000 7fbdaa820640 1 -- start start 2024-04-14T08:36:44.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbdaa820640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbda4160170 0x7fbda415d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbdaa820640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbda415d020 0x7fbda4160550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbdaa820640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbda413f910 con 0x7fbda4160a90 2024-04-14T08:36:44.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbdaa820640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fbda413fbf0 con 0x7fbda4160170 2024-04-14T08:36:44.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbdaa820640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbda413fed0 con 0x7fbda415d020 2024-04-14T08:36:44.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbda3fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbda4160170 0x7fbda415d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbda8dbd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fbda4160a90 0x7fbda40b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47032/0 (socket says 172.21.15.172:47032) 2024-04-14T08:36:44.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbda8dbd640 1 -- 172.21.15.172:0/1510204501 learned_addr learned my addr 172.21.15.172:0/1510204501 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:44.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbda37fe640 1 --2- 172.21.15.172:0/1510204501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbda415d020 0x7fbda4160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbda3fff640 1 -- 172.21.15.172:0/1510204501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbda415d020 msgr2=0x7fbda4160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:44.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbda3fff640 1 --2- 172.21.15.172:0/1510204501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbda415d020 0x7fbda4160550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbda3fff640 1 -- 172.21.15.172:0/1510204501 >> v1:172.21.15.172:6789/0 conn(0x7fbda4160a90 legacy=0x7fbda40b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbda2ffd640 1 -- 172.21.15.172:0/1510204501 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 185708815 0 0) 0x7fbda413f910 con 0x7fbda4160a90 2024-04-14T08:36:44.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.587+0000 7fbda3fff640 1 -- 172.21.15.172:0/1510204501 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda41401b0 con 0x7fbda4160170 2024-04-14T08:36:44.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.588+0000 7fbda3fff640 1 --2- 172.21.15.172:0/1510204501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda4160170 0x7fbda415d980 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbd940097a0 tx=0x7fbd9402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40173fd46c9059c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:44.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.588+0000 7fbda2ffd640 1 -- 172.21.15.172:0/1510204501 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94035020 con 0x7fbda4160170 2024-04-14T08:36:44.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.588+0000 7fbda2ffd640 1 -- 172.21.15.172:0/1510204501 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd94031830 con 0x7fbda4160170 2024-04-14T08:36:44.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.588+0000 7fbda2ffd640 1 -- 172.21.15.172:0/1510204501 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94031c00 con 0x7fbda4160170 2024-04-14T08:36:44.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.589+0000 7fbdaa820640 1 -- 172.21.15.172:0/1510204501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda4160170 msgr2=0x7fbda415d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.589+0000 7fbdaa820640 1 --2- 172.21.15.172:0/1510204501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda4160170 0x7fbda415d980 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbd940097a0 tx=0x7fbd9402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.589+0000 7fbdaa820640 1 -- 172.21.15.172:0/1510204501 shutdown_connections 2024-04-14T08:36:44.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.589+0000 7fbdaa820640 1 --2- 172.21.15.172:0/1510204501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbda415d020 0x7fbda4160550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.589+0000 7fbdaa820640 1 --2- 172.21.15.172:0/1510204501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda4160170 0x7fbda415d980 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.589+0000 7fbdaa820640 1 -- 172.21.15.172:0/1510204501 >> 172.21.15.172:0/1510204501 conn(0x7fbda406ec30 msgr2=0x7fbda406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:44.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.589+0000 7fbdaa820640 1 -- 172.21.15.172:0/1510204501 shutdown_connections 2024-04-14T08:36:44.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.589+0000 7fbdaa820640 1 -- 172.21.15.172:0/1510204501 wait complete. 2024-04-14T08:36:44.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.590+0000 7fbdaa820640 1 Processor -- start 2024-04-14T08:36:44.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.590+0000 7fbdaa820640 1 -- start start 2024-04-14T08:36:44.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbdaa820640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbda415d020 0x7fbda41e66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbdaa820640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda4160170 0x7fbda4154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbdaa820640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbda4160a90 0x7fbda40b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbdaa820640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbda40b2410 con 0x7fbda4160170 2024-04-14T08:36:44.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbdaa820640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbda40b2290 con 0x7fbda4160a90 2024-04-14T08:36:44.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbdaa820640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbda415d4a0 con 0x7fbda415d020 2024-04-14T08:36:44.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda4160170 0x7fbda4154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda4160170 0x7fbda4154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43434/0 (socket says 172.21.15.172:43434) 2024-04-14T08:36:44.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbda37fe640 1 -- 172.21.15.172:0/1283166966 learned_addr learned my addr 172.21.15.172:0/1283166966 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:44.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbda3fff640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbda415d020 0x7fbda41e66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbda8dbd640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbda4160a90 0x7fbda40b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbda37fe640 1 -- 172.21.15.172:0/1283166966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbda415d020 msgr2=0x7fbda41e66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbda37fe640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbda415d020 0x7fbda41e66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbda37fe640 1 -- 172.21.15.172:0/1283166966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbda4160a90 msgr2=0x7fbda40b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbda37fe640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbda4160a90 0x7fbda40b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbda37fe640 1 -- 172.21.15.172:0/1283166966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda40b3cd0 con 0x7fbda4160170 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbda37fe640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda4160170 0x7fbda4154e50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800e9b0 tx=0x7fbd9800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.591+0000 7fbda0ff9640 1 -- 172.21.15.172:0/1283166966 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98017070 con 0x7fbda4160170 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.592+0000 7fbda0ff9640 1 -- 172.21.15.172:0/1283166966 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9800c9a0 con 0x7fbda4160170 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.592+0000 7fbdaa820640 1 -- 172.21.15.172:0/1283166966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda40b3fb0 con 0x7fbda4160170 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.592+0000 7fbda0ff9640 1 -- 172.21.15.172:0/1283166966 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800cd90 con 0x7fbda4160170 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.592+0000 7fbdaa820640 1 -- 172.21.15.172:0/1283166966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda41e61a0 con 0x7fbda4160170 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.592+0000 7fbdaa820640 1 -- 172.21.15.172:0/1283166966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda4068520 con 0x7fbda4160170 2024-04-14T08:36:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.593+0000 7fbda0ff9640 1 -- 172.21.15.172:0/1283166966 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd9801a070 con 0x7fbda4160170 2024-04-14T08:36:44.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.593+0000 7fbda0ff9640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbd9006c920 0x7fbd9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.593+0000 7fbda0ff9640 1 -- 172.21.15.172:0/1283166966 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fbd9808c050 con 0x7fbda4160170 2024-04-14T08:36:44.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.593+0000 7fbda3fff640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbd9006c920 0x7fbd9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.594+0000 7fbda3fff640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbd9006c920 0x7fbd9006ede0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbd94036000 tx=0x7fbd9403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:44.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.598+0000 7fbda0ff9640 1 -- 172.21.15.172:0/1283166966 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbd98059c10 con 0x7fbda4160170 2024-04-14T08:36:44.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.772+0000 7fbdaa820640 1 -- 172.21.15.172:0/1283166966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbda406c2c0 con 0x7fbda4160170 2024-04-14T08:36:44.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.773+0000 7fbda0ff9640 1 -- 172.21.15.172:0/1283166966 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17184 (secure 0 0 0) 0x7fbd9805df10 con 0x7fbda4160170 2024-04-14T08:36:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:44.775 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":63,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:36:35.260134+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:35:32.539067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-14T08:35:33.263123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.17","mappings":[{"from":4,"to":5}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:36:44.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.775+0000 7fbdaa820640 1 -- 172.21.15.172:0/1283166966 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbd9006c920 msgr2=0x7fbd9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.775+0000 7fbdaa820640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbd9006c920 0x7fbd9006ede0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbd94036000 tx=0x7fbd9403d040 comp rx=0 tx=0).stop 2024-04-14T08:36:44.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.775+0000 7fbdaa820640 1 -- 172.21.15.172:0/1283166966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda4160170 msgr2=0x7fbda4154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.775+0000 7fbdaa820640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda4160170 0x7fbda4154e50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800e9b0 tx=0x7fbd9800ee70 comp rx=0 tx=0).stop 2024-04-14T08:36:44.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.775+0000 7fbdaa820640 1 -- 172.21.15.172:0/1283166966 shutdown_connections 2024-04-14T08:36:44.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.775+0000 7fbdaa820640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbd9006c920 0x7fbd9006ede0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.775+0000 7fbdaa820640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbda4160a90 0x7fbda40b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.775+0000 7fbdaa820640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda4160170 0x7fbda4154e50 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.775+0000 7fbdaa820640 1 --2- 172.21.15.172:0/1283166966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbda415d020 0x7fbda41e66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.775+0000 7fbdaa820640 1 -- 172.21.15.172:0/1283166966 >> 172.21.15.172:0/1283166966 conn(0x7fbda406ec30 msgr2=0x7fbda415f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:44.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.775+0000 7fbdaa820640 1 -- 172.21.15.172:0/1283166966 shutdown_connections 2024-04-14T08:36:44.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.775+0000 7fbdaa820640 1 -- 172.21.15.172:0/1283166966 wait complete. 2024-04-14T08:36:44.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:35:32.539067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:35:33.263123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:36:44.788 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:36:44.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.890+0000 7ff1c3115640 1 Processor -- start 2024-04-14T08:36:44.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.890+0000 7ff1c3115640 1 -- start start 2024-04-14T08:36:44.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1c3115640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff1bc159c20 0x7ff1bc0ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1c3115640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1bc072450 0x7ff1bc0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1c3115640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1bc160b00 con 0x7ff1bc15c700 2024-04-14T08:36:44.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1c3115640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff1bc15fc80 con 0x7ff1bc159c20 2024-04-14T08:36:44.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1c3115640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff1bc15ff60 con 0x7ff1bc072450 2024-04-14T08:36:44.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1c0eb1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff1bc159c20 0x7ff1bc0ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1bbfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff1bc15c700 0x7ff1bc1603f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47046/0 (socket says 172.21.15.172:47046) 2024-04-14T08:36:44.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1bbfff640 1 -- 172.21.15.172:0/833463704 learned_addr learned my addr 172.21.15.172:0/833463704 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:44.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1c16b2640 1 --2- 172.21.15.172:0/833463704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1bc072450 0x7ff1bc0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1c16b2640 1 -- 172.21.15.172:0/833463704 >> v2:172.21.15.172:3300/0 conn(0x7ff1bc159c20 msgr2=0x7ff1bc0ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:44.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1c16b2640 1 --2- 172.21.15.172:0/833463704 >> v2:172.21.15.172:3300/0 conn(0x7ff1bc159c20 0x7ff1bc0ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1c16b2640 1 -- 172.21.15.172:0/833463704 >> v1:172.21.15.172:6789/0 conn(0x7ff1bc15c700 legacy=0x7ff1bc1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1c16b2640 1 -- 172.21.15.172:0/833463704 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc15cf10 con 0x7ff1bc072450 2024-04-14T08:36:44.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.891+0000 7ff1c0eb1640 1 --2- 172.21.15.172:0/833463704 >> v2:172.21.15.172:3300/0 conn(0x7ff1bc159c20 0x7ff1bc0ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:44.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.892+0000 7ff1c16b2640 1 --2- 172.21.15.172:0/833463704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1bc072450 0x7ff1bc0728e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009d90 tx=0x7ff1b402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af60630704a61e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.892+0000 7ff1bb7fe640 1 -- 172.21.15.172:0/833463704 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4039070 con 0x7ff1bc072450 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.892+0000 7ff1bb7fe640 1 -- 172.21.15.172:0/833463704 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b402cc50 con 0x7ff1bc072450 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.892+0000 7ff1bb7fe640 1 -- 172.21.15.172:0/833463704 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4034440 con 0x7ff1bc072450 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.893+0000 7ff1c3115640 1 -- 172.21.15.172:0/833463704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1bc072450 msgr2=0x7ff1bc0728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.893+0000 7ff1c3115640 1 --2- 172.21.15.172:0/833463704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1bc072450 0x7ff1bc0728e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009d90 tx=0x7ff1b402bea0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.893+0000 7ff1c3115640 1 -- 172.21.15.172:0/833463704 shutdown_connections 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.893+0000 7ff1c3115640 1 --2- 172.21.15.172:0/833463704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1bc072450 0x7ff1bc0728e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.893+0000 7ff1c3115640 1 --2- 172.21.15.172:0/833463704 >> v2:172.21.15.172:3300/0 conn(0x7ff1bc159c20 0x7ff1bc0ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.893+0000 7ff1c3115640 1 -- 172.21.15.172:0/833463704 >> 172.21.15.172:0/833463704 conn(0x7ff1bc06b520 msgr2=0x7ff1bc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.893+0000 7ff1c3115640 1 -- 172.21.15.172:0/833463704 shutdown_connections 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.893+0000 7ff1c3115640 1 -- 172.21.15.172:0/833463704 wait complete. 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.894+0000 7ff1c3115640 1 Processor -- start 2024-04-14T08:36:44.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.894+0000 7ff1c3115640 1 -- start start 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c3115640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1bc072450 0x7ff1bc1553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c3115640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1bc159c20 0x7ff1bc13e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c3115640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1bc15c700 0x7ff1bc13e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c3115640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc15a000 con 0x7ff1bc15c700 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c3115640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc15d4d0 con 0x7ff1bc159c20 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c3115640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff1bc15cb80 con 0x7ff1bc072450 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c16b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1bc15c700 0x7ff1bc13e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c16b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1bc15c700 0x7ff1bc13e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43452/0 (socket says 172.21.15.172:43452) 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c16b2640 1 -- 172.21.15.172:0/3861901256 learned_addr learned my addr 172.21.15.172:0/3861901256 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c0eb1640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1bc072450 0x7ff1bc1553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1bbfff640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1bc159c20 0x7ff1bc13e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c16b2640 1 -- 172.21.15.172:0/3861901256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1bc072450 msgr2=0x7ff1bc1553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c16b2640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1bc072450 0x7ff1bc1553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c16b2640 1 -- 172.21.15.172:0/3861901256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1bc159c20 msgr2=0x7ff1bc13e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:44.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c16b2640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1bc159c20 0x7ff1bc13e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:44.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c16b2640 1 -- 172.21.15.172:0/3861901256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc154610 con 0x7ff1bc15c700 2024-04-14T08:36:44.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c16b2640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1bc15c700 0x7ff1bc13e9c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009d90 tx=0x7ff1b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:44.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1b97fa640 1 -- 172.21.15.172:0/3861901256 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4039050 con 0x7ff1bc15c700 2024-04-14T08:36:44.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1b97fa640 1 -- 172.21.15.172:0/3861901256 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b4045070 con 0x7ff1bc15c700 2024-04-14T08:36:44.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.895+0000 7ff1c3115640 1 -- 172.21.15.172:0/3861901256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1bc1548f0 con 0x7ff1bc15c700 2024-04-14T08:36:44.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.896+0000 7ff1b97fa640 1 -- 172.21.15.172:0/3861901256 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4009450 con 0x7ff1bc15c700 2024-04-14T08:36:44.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.896+0000 7ff1c3115640 1 -- 172.21.15.172:0/3861901256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1bc154eb0 con 0x7ff1bc15c700 2024-04-14T08:36:44.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.897+0000 7ff1c3115640 1 -- 172.21.15.172:0/3861901256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1bc068590 con 0x7ff1bc15c700 2024-04-14T08:36:44.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.897+0000 7ff1b97fa640 1 -- 172.21.15.172:0/3861901256 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1b403e610 con 0x7ff1bc15c700 2024-04-14T08:36:44.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.897+0000 7ff1b97fa640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff1a006c8f0 0x7ff1a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:44.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.897+0000 7ff1b97fa640 1 -- 172.21.15.172:0/3861901256 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7ff1b40ae670 con 0x7ff1bc15c700 2024-04-14T08:36:44.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.897+0000 7ff1c0eb1640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff1a006c8f0 0x7ff1a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:44.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.898+0000 7ff1c0eb1640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff1a006c8f0 0x7ff1a006edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009560 tx=0x7ff1b0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:44.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:44.902+0000 7ff1b97fa640 1 -- 172.21.15.172:0/3861901256 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff1b407c230 con 0x7ff1bc15c700 2024-04-14T08:36:45.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.074+0000 7ff1c3115640 1 -- 172.21.15.172:0/3861901256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff1bc071e50 con 0x7ff1bc15c700 2024-04-14T08:36:45.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.075+0000 7ff1b97fa640 1 -- 172.21.15.172:0/3861901256 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7ff1b4080530 con 0x7ff1bc15c700 2024-04-14T08:36:45.076 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:36:45.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.077+0000 7ff1c3115640 1 -- 172.21.15.172:0/3861901256 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff1a006c8f0 msgr2=0x7ff1a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.077+0000 7ff1c3115640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff1a006c8f0 0x7ff1a006edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009560 tx=0x7ff1b0009190 comp rx=0 tx=0).stop 2024-04-14T08:36:45.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.077+0000 7ff1c3115640 1 -- 172.21.15.172:0/3861901256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1bc15c700 msgr2=0x7ff1bc13e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.077+0000 7ff1c3115640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1bc15c700 0x7ff1bc13e9c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009d90 tx=0x7ff1b4004a40 comp rx=0 tx=0).stop 2024-04-14T08:36:45.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.077+0000 7ff1c3115640 1 -- 172.21.15.172:0/3861901256 shutdown_connections 2024-04-14T08:36:45.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.077+0000 7ff1c3115640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff1a006c8f0 0x7ff1a006edb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.077+0000 7ff1c3115640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1bc15c700 0x7ff1bc13e9c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.077+0000 7ff1c3115640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1bc159c20 0x7ff1bc13e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.077+0000 7ff1c3115640 1 --2- 172.21.15.172:0/3861901256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1bc072450 0x7ff1bc1553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.077+0000 7ff1c3115640 1 -- 172.21.15.172:0/3861901256 >> 172.21.15.172:0/3861901256 conn(0x7ff1bc06b520 msgr2=0x7ff1bc15bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:45.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.077+0000 7ff1c3115640 1 -- 172.21.15.172:0/3861901256 shutdown_connections 2024-04-14T08:36:45.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.077+0000 7ff1c3115640 1 -- 172.21.15.172:0/3861901256 wait complete. 2024-04-14T08:36:45.089 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:36:45.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.193+0000 7f42f2ede640 1 Processor -- start 2024-04-14T08:36:45.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.193+0000 7f42f2ede640 1 -- start start 2024-04-14T08:36:45.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.193+0000 7f42f2ede640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42ec15e430 0x7f42ec162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.193+0000 7f42f2ede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42ec0d2e00 0x7f42ec1629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.193+0000 7f42f2ede640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42ec0cefb0 con 0x7f42ec162540 2024-04-14T08:36:45.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.193+0000 7f42f2ede640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f42ec0cf290 con 0x7f42ec15e430 2024-04-14T08:36:45.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.193+0000 7f42f2ede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f42ec0cf570 con 0x7f42ec0d2e00 2024-04-14T08:36:45.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.194+0000 7f42f0c7a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42ec15e430 0x7f42ec162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.194+0000 7f42ebfff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f42ec162540 0x7f42ec0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:49508/0 (socket says 172.21.15.172:49508) 2024-04-14T08:36:45.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.194+0000 7f42f0c7a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42ec15e430 0x7f42ec162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43468/0 (socket says 172.21.15.172:43468) 2024-04-14T08:36:45.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.194+0000 7f42ebfff640 1 -- 172.21.15.172:0/840608126 learned_addr learned my addr 172.21.15.172:0/840608126 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:45.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.194+0000 7f42f147b640 1 --2- 172.21.15.172:0/840608126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42ec0d2e00 0x7f42ec1629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.194+0000 7f42f0c7a640 1 -- 172.21.15.172:0/840608126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42ec0d2e00 msgr2=0x7f42ec1629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:45.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.194+0000 7f42f0c7a640 1 --2- 172.21.15.172:0/840608126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42ec0d2e00 0x7f42ec1629d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.194+0000 7f42f0c7a640 1 -- 172.21.15.172:0/840608126 >> v1:172.21.15.177:6789/0 conn(0x7f42ec162540 legacy=0x7f42ec0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.194+0000 7f42f0c7a640 1 -- 172.21.15.172:0/840608126 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ec0cf850 con 0x7f42ec15e430 2024-04-14T08:36:45.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.195+0000 7f42f0c7a640 1 --2- 172.21.15.172:0/840608126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42ec15e430 0x7f42ec162000 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f42e00097a0 tx=0x7f42e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7931ec5d953845 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:45.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.195+0000 7f42eb7fe640 1 -- 172.21.15.172:0/840608126 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0035020 con 0x7f42ec15e430 2024-04-14T08:36:45.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.195+0000 7f42eb7fe640 1 -- 172.21.15.172:0/840608126 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e0031830 con 0x7f42ec15e430 2024-04-14T08:36:45.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.195+0000 7f42eb7fe640 1 -- 172.21.15.172:0/840608126 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0031c00 con 0x7f42ec15e430 2024-04-14T08:36:45.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.195+0000 7f42f2ede640 1 -- 172.21.15.172:0/840608126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42ec15e430 msgr2=0x7f42ec162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.195+0000 7f42f2ede640 1 --2- 172.21.15.172:0/840608126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42ec15e430 0x7f42ec162000 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f42e00097a0 tx=0x7f42e002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.195+0000 7f42f2ede640 1 -- 172.21.15.172:0/840608126 shutdown_connections 2024-04-14T08:36:45.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.195+0000 7f42f2ede640 1 --2- 172.21.15.172:0/840608126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42ec0d2e00 0x7f42ec1629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.195+0000 7f42f2ede640 1 --2- 172.21.15.172:0/840608126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42ec15e430 0x7f42ec162000 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.195+0000 7f42f2ede640 1 -- 172.21.15.172:0/840608126 >> 172.21.15.172:0/840608126 conn(0x7f42ec09d560 msgr2=0x7f42ec0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:45.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.195+0000 7f42f2ede640 1 -- 172.21.15.172:0/840608126 shutdown_connections 2024-04-14T08:36:45.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.195+0000 7f42f2ede640 1 -- 172.21.15.172:0/840608126 wait complete. 2024-04-14T08:36:45.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.197+0000 7f42f2ede640 1 Processor -- start 2024-04-14T08:36:45.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.197+0000 7f42f2ede640 1 -- start start 2024-04-14T08:36:45.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.197+0000 7f42f2ede640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42ec0d2e00 0x7f42ec15fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.197+0000 7f42f2ede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42ec15e430 0x7f42ec0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.197+0000 7f42f2ede640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42ec162540 0x7f42ec0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.197+0000 7f42f2ede640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec0d4e20 con 0x7f42ec162540 2024-04-14T08:36:45.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.197+0000 7f42f2ede640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec15e810 con 0x7f42ec0d2e00 2024-04-14T08:36:45.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.197+0000 7f42f2ede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f42ec0d0050 con 0x7f42ec15e430 2024-04-14T08:36:45.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f147b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42ec162540 0x7f42ec0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f147b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42ec162540 0x7f42ec0ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43474/0 (socket says 172.21.15.172:43474) 2024-04-14T08:36:45.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f147b640 1 -- 172.21.15.172:0/2427489256 learned_addr learned my addr 172.21.15.172:0/2427489256 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:45.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f0c7a640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42ec0d2e00 0x7f42ec15fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42ebfff640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42ec15e430 0x7f42ec0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f147b640 1 -- 172.21.15.172:0/2427489256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42ec15e430 msgr2=0x7f42ec0b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f147b640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42ec15e430 0x7f42ec0b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f147b640 1 -- 172.21.15.172:0/2427489256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42ec0d2e00 msgr2=0x7f42ec15fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f147b640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42ec0d2e00 0x7f42ec15fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f147b640 1 -- 172.21.15.172:0/2427489256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ec15e990 con 0x7f42ec162540 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f0c7a640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42ec0d2e00 0x7f42ec15fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f147b640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42ec162540 0x7f42ec0ccf90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f42e400efc0 tx=0x7f42e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42e97fa640 1 -- 172.21.15.172:0/2427489256 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4018020 con 0x7f42ec162540 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42e97fa640 1 -- 172.21.15.172:0/2427489256 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e4014480 con 0x7f42ec162540 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42e97fa640 1 -- 172.21.15.172:0/2427489256 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4014870 con 0x7f42ec162540 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f2ede640 1 -- 172.21.15.172:0/2427489256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ec15ec50 con 0x7f42ec162540 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.198+0000 7f42f2ede640 1 -- 172.21.15.172:0/2427489256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ec15f210 con 0x7f42ec162540 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.199+0000 7f42f2ede640 1 -- 172.21.15.172:0/2427489256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ec068590 con 0x7f42ec162540 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.200+0000 7f42e97fa640 1 -- 172.21.15.172:0/2427489256 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42e4005d10 con 0x7f42ec162540 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.200+0000 7f42e97fa640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f42d006c8f0 0x7f42d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.200+0000 7f42e97fa640 1 -- 172.21.15.172:0/2427489256 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f42e408c720 con 0x7f42ec162540 2024-04-14T08:36:45.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.200+0000 7f42f0c7a640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f42d006c8f0 0x7f42d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.201+0000 7f42f0c7a640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f42d006c8f0 0x7f42d006edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f42e0036000 tx=0x7f42e0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:45.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.205+0000 7f42e97fa640 1 -- 172.21.15.172:0/2427489256 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f42e4022070 con 0x7f42ec162540 2024-04-14T08:36:45.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.379+0000 7f42f2ede640 1 -- 172.21.15.172:0/2427489256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f42ec0d18a0 con 0x7f42ec162540 2024-04-14T08:36:45.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.379+0000 7f42e97fa640 1 -- 172.21.15.172:0/2427489256 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) ==== 104+0+11 (secure 0 0 0) 0x7f42e4010040 con 0x7f42ec162540 2024-04-14T08:36:45.381 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:36:45.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.381+0000 7f42f2ede640 1 -- 172.21.15.172:0/2427489256 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f42d006c8f0 msgr2=0x7f42d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.381+0000 7f42f2ede640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f42d006c8f0 0x7f42d006edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f42e0036000 tx=0x7f42e0008030 comp rx=0 tx=0).stop 2024-04-14T08:36:45.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.381+0000 7f42f2ede640 1 -- 172.21.15.172:0/2427489256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42ec162540 msgr2=0x7f42ec0ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.381+0000 7f42f2ede640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42ec162540 0x7f42ec0ccf90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f42e400efc0 tx=0x7f42e400c490 comp rx=0 tx=0).stop 2024-04-14T08:36:45.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.382+0000 7f42f2ede640 1 -- 172.21.15.172:0/2427489256 shutdown_connections 2024-04-14T08:36:45.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.382+0000 7f42f2ede640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f42d006c8f0 0x7f42d006edb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.382+0000 7f42f2ede640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42ec162540 0x7f42ec0ccf90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.382+0000 7f42f2ede640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42ec15e430 0x7f42ec0b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.382+0000 7f42f2ede640 1 --2- 172.21.15.172:0/2427489256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42ec0d2e00 0x7f42ec15fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.382+0000 7f42f2ede640 1 -- 172.21.15.172:0/2427489256 >> 172.21.15.172:0/2427489256 conn(0x7f42ec09d560 msgr2=0x7f42ec0d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:45.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.382+0000 7f42f2ede640 1 -- 172.21.15.172:0/2427489256 shutdown_connections 2024-04-14T08:36:45.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.382+0000 7f42f2ede640 1 -- 172.21.15.172:0/2427489256 wait complete. 2024-04-14T08:36:45.394 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:36:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.496+0000 7f89abf1e640 1 Processor -- start 2024-04-14T08:36:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.496+0000 7f89abf1e640 1 -- start start 2024-04-14T08:36:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.496+0000 7f89abf1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f89a40ccca0 0x7f89a406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.496+0000 7f89abf1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f89a4073090 0x7f89a40b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.496+0000 7f89abf1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a40ceee0 con 0x7f89a406b520 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.496+0000 7f89abf1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f89a4073520 con 0x7f89a4073090 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.496+0000 7f89abf1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f89a4070150 con 0x7f89a40ccca0 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.497+0000 7f89aa4bb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f89a406b520 0x7f89a406b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47060/0 (socket says 172.21.15.172:47060) 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.497+0000 7f89a94b9640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f89a4073090 0x7f89a40b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.497+0000 7f89aa4bb640 1 -- 172.21.15.172:0/9237556 learned_addr learned my addr 172.21.15.172:0/9237556 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.497+0000 7f89a9cba640 1 --2- 172.21.15.172:0/9237556 >> v2:172.21.15.177:3300/0 conn(0x7f89a40ccca0 0x7f89a406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.497+0000 7f89a94b9640 1 -- 172.21.15.172:0/9237556 >> v2:172.21.15.177:3300/0 conn(0x7f89a40ccca0 msgr2=0x7f89a406fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.497+0000 7f89a94b9640 1 --2- 172.21.15.172:0/9237556 >> v2:172.21.15.177:3300/0 conn(0x7f89a40ccca0 0x7f89a406fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.497+0000 7f89a94b9640 1 -- 172.21.15.172:0/9237556 >> v1:172.21.15.172:6789/0 conn(0x7f89a406b520 legacy=0x7f89a406b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.497+0000 7f89a94b9640 1 -- 172.21.15.172:0/9237556 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a40722d0 con 0x7f89a4073090 2024-04-14T08:36:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.497+0000 7f89a8cb8640 1 -- 172.21.15.172:0/9237556 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4047568858 0 0) 0x7f89a40ceee0 con 0x7f89a406b520 2024-04-14T08:36:45.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.498+0000 7f89a94b9640 1 --2- 172.21.15.172:0/9237556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89a4073090 0x7f89a40b0190 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f89a4068ce0 tx=0x7f899802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893a85801d3d6f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.498+0000 7f89a8cb8640 1 -- 172.21.15.172:0/9237556 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89980315a0 con 0x7f89a4073090 2024-04-14T08:36:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.498+0000 7f89a8cb8640 1 -- 172.21.15.172:0/9237556 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8998037070 con 0x7f89a4073090 2024-04-14T08:36:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.498+0000 7f89a8cb8640 1 -- 172.21.15.172:0/9237556 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998031960 con 0x7f89a4073090 2024-04-14T08:36:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.498+0000 7f89abf1e640 1 -- 172.21.15.172:0/9237556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89a4073090 msgr2=0x7f89a40b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.498+0000 7f89abf1e640 1 --2- 172.21.15.172:0/9237556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89a4073090 0x7f89a40b0190 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f89a4068ce0 tx=0x7f899802b840 comp rx=0 tx=0).stop 2024-04-14T08:36:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.498+0000 7f89abf1e640 1 -- 172.21.15.172:0/9237556 shutdown_connections 2024-04-14T08:36:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.498+0000 7f89abf1e640 1 --2- 172.21.15.172:0/9237556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89a4073090 0x7f89a40b0190 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.498+0000 7f89abf1e640 1 --2- 172.21.15.172:0/9237556 >> v2:172.21.15.177:3300/0 conn(0x7f89a40ccca0 0x7f89a406fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.498+0000 7f89abf1e640 1 -- 172.21.15.172:0/9237556 >> 172.21.15.172:0/9237556 conn(0x7f89a40cda00 msgr2=0x7f89a40cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.498+0000 7f89abf1e640 1 -- 172.21.15.172:0/9237556 shutdown_connections 2024-04-14T08:36:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.499+0000 7f89abf1e640 1 -- 172.21.15.172:0/9237556 wait complete. 2024-04-14T08:36:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89abf1e640 1 Processor -- start 2024-04-14T08:36:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89abf1e640 1 -- start start 2024-04-14T08:36:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89abf1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89a406b520 0x7f89a41533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89abf1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89a4073090 0x7f89a40cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89abf1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89a40ccca0 0x7f89a40aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89abf1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f89a40cd080 con 0x7f89a4073090 2024-04-14T08:36:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89abf1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4072890 con 0x7f89a406b520 2024-04-14T08:36:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89abf1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f89a406e150 con 0x7f89a40ccca0 2024-04-14T08:36:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89a94b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89a4073090 0x7f89a40cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89a94b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89a4073090 0x7f89a40cb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43498/0 (socket says 172.21.15.172:43498) 2024-04-14T08:36:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89a94b9640 1 -- 172.21.15.172:0/4174799977 learned_addr learned my addr 172.21.15.172:0/4174799977 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89a9cba640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89a406b520 0x7f89a41533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.500+0000 7f89aa4bb640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89a40ccca0 0x7f89a40aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.501+0000 7f89a94b9640 1 -- 172.21.15.172:0/4174799977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89a40ccca0 msgr2=0x7f89a40aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.501+0000 7f89a94b9640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89a40ccca0 0x7f89a40aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.501+0000 7f89a94b9640 1 -- 172.21.15.172:0/4174799977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89a406b520 msgr2=0x7f89a41533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.501+0000 7f89a94b9640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89a406b520 0x7f89a41533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.501+0000 7f89a94b9640 1 -- 172.21.15.172:0/4174799977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a40c9210 con 0x7f89a4073090 2024-04-14T08:36:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.501+0000 7f89a9cba640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89a406b520 0x7f89a41533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.501+0000 7f89a94b9640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89a4073090 0x7f89a40cb0e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f899802be50 tx=0x7f89980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.501+0000 7f89a27fc640 1 -- 172.21.15.172:0/4174799977 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998032630 con 0x7f89a4073090 2024-04-14T08:36:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.501+0000 7f89a27fc640 1 -- 172.21.15.172:0/4174799977 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89980327d0 con 0x7f89a4073090 2024-04-14T08:36:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.501+0000 7f89abf1e640 1 -- 172.21.15.172:0/4174799977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a40c94f0 con 0x7f89a4073090 2024-04-14T08:36:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.501+0000 7f89a27fc640 1 -- 172.21.15.172:0/4174799977 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998032bc0 con 0x7f89a4073090 2024-04-14T08:36:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.501+0000 7f89abf1e640 1 -- 172.21.15.172:0/4174799977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a40c9ab0 con 0x7f89a4073090 2024-04-14T08:36:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.502+0000 7f89abf1e640 1 -- 172.21.15.172:0/4174799977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a4068590 con 0x7f89a4073090 2024-04-14T08:36:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.502+0000 7f89a27fc640 1 -- 172.21.15.172:0/4174799977 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f899804c020 con 0x7f89a4073090 2024-04-14T08:36:45.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.503+0000 7f89a27fc640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f899006c8f0 0x7f899006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.503+0000 7f89a27fc640 1 -- 172.21.15.172:0/4174799977 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f899802f020 con 0x7f89a4073090 2024-04-14T08:36:45.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.503+0000 7f89a9cba640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f899006c8f0 0x7f899006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.503+0000 7f89a9cba640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f899006c8f0 0x7f899006edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f89a40c4a30 tx=0x7f899c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:45.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.507+0000 7f89a27fc640 1 -- 172.21.15.172:0/4174799977 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8998040070 con 0x7f89a4073090 2024-04-14T08:36:45.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.681+0000 7f89abf1e640 1 -- 172.21.15.172:0/4174799977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f89a40cf260 con 0x7f89a4073090 2024-04-14T08:36:45.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.682+0000 7f89a27fc640 1 -- 172.21.15.172:0/4174799977 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) ==== 100+0+11 (secure 0 0 0) 0x7f899807b360 con 0x7f89a4073090 2024-04-14T08:36:45.683 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:36:45.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.684+0000 7f89abf1e640 1 -- 172.21.15.172:0/4174799977 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f899006c8f0 msgr2=0x7f899006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.684+0000 7f89abf1e640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f899006c8f0 0x7f899006edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f89a40c4a30 tx=0x7f899c009190 comp rx=0 tx=0).stop 2024-04-14T08:36:45.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.684+0000 7f89abf1e640 1 -- 172.21.15.172:0/4174799977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89a4073090 msgr2=0x7f89a40cb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.684+0000 7f89abf1e640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89a4073090 0x7f89a40cb0e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f899802be50 tx=0x7f89980046c0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.684+0000 7f89abf1e640 1 -- 172.21.15.172:0/4174799977 shutdown_connections 2024-04-14T08:36:45.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.684+0000 7f89abf1e640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f899006c8f0 0x7f899006edb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.684+0000 7f89abf1e640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89a40ccca0 0x7f89a40aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.684+0000 7f89abf1e640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89a4073090 0x7f89a40cb0e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.684+0000 7f89abf1e640 1 --2- 172.21.15.172:0/4174799977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89a406b520 0x7f89a41533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.684+0000 7f89abf1e640 1 -- 172.21.15.172:0/4174799977 >> 172.21.15.172:0/4174799977 conn(0x7f89a40cda00 msgr2=0x7f89a4074a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:45.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.684+0000 7f89abf1e640 1 -- 172.21.15.172:0/4174799977 shutdown_connections 2024-04-14T08:36:45.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.684+0000 7f89abf1e640 1 -- 172.21.15.172:0/4174799977 wait complete. 2024-04-14T08:36:45.696 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:36:45.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.796+0000 7f0029908640 1 Processor -- start 2024-04-14T08:36:45.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.796+0000 7f0029908640 1 -- start start 2024-04-14T08:36:45.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.797+0000 7f0029908640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00240d0eb0 0x7f00240b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.797+0000 7f0029908640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f002415e890 con 0x7f00240d0a70 2024-04-14T08:36:45.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.797+0000 7f0029908640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00240d1f30 con 0x7f002415e4b0 2024-04-14T08:36:45.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.797+0000 7f0029908640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f00240d2210 con 0x7f00240d0eb0 2024-04-14T08:36:45.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.797+0000 7f001bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f00240d0a70 0x7f00240b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47072/0 (socket says 172.21.15.172:47072) 2024-04-14T08:36:45.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.797+0000 7f001bfff640 1 -- 172.21.15.172:0/66402243 learned_addr learned my addr 172.21.15.172:0/66402243 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:45.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.797+0000 7f00237fe640 1 --2- 172.21.15.172:0/66402243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00240d0eb0 0x7f00240b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.797+0000 7f00227fc640 1 -- 172.21.15.172:0/66402243 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2444334342 0 0) 0x7f002415e890 con 0x7f00240d0a70 2024-04-14T08:36:45.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.797+0000 7f00237fe640 1 -- 172.21.15.172:0/66402243 >> v1:172.21.15.177:6789/0 conn(0x7f002415e4b0 legacy=0x7f00240cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.797+0000 7f00237fe640 1 -- 172.21.15.172:0/66402243 >> v1:172.21.15.172:6789/0 conn(0x7f00240d0a70 legacy=0x7f00240b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.797+0000 7f00237fe640 1 -- 172.21.15.172:0/66402243 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00240d24f0 con 0x7f00240d0eb0 2024-04-14T08:36:45.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.798+0000 7f00237fe640 1 --2- 172.21.15.172:0/66402243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00240d0eb0 0x7f00240b40c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f001c009dc0 tx=0x7f001c02bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57ecaa4c058f7ec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:45.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.798+0000 7f00227fc640 1 -- 172.21.15.172:0/66402243 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c039070 con 0x7f00240d0eb0 2024-04-14T08:36:45.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.798+0000 7f00227fc640 1 -- 172.21.15.172:0/66402243 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f001c02ca10 con 0x7f00240d0eb0 2024-04-14T08:36:45.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.798+0000 7f00227fc640 1 -- 172.21.15.172:0/66402243 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c02cde0 con 0x7f00240d0eb0 2024-04-14T08:36:45.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.799+0000 7f0029908640 1 -- 172.21.15.172:0/66402243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00240d0eb0 msgr2=0x7f00240b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.799+0000 7f0029908640 1 --2- 172.21.15.172:0/66402243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00240d0eb0 0x7f00240b40c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f001c009dc0 tx=0x7f001c02bce0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.799+0000 7f0029908640 1 -- 172.21.15.172:0/66402243 shutdown_connections 2024-04-14T08:36:45.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.799+0000 7f0029908640 1 --2- 172.21.15.172:0/66402243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00240d0eb0 0x7f00240b40c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.799+0000 7f0029908640 1 -- 172.21.15.172:0/66402243 >> 172.21.15.172:0/66402243 conn(0x7f002409d570 msgr2=0x7f00240d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:45.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.799+0000 7f0029908640 1 -- 172.21.15.172:0/66402243 shutdown_connections 2024-04-14T08:36:45.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.799+0000 7f0029908640 1 -- 172.21.15.172:0/66402243 wait complete. 2024-04-14T08:36:45.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.800+0000 7f0029908640 1 Processor -- start 2024-04-14T08:36:45.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.800+0000 7f0029908640 1 -- start start 2024-04-14T08:36:45.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f0029908640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00240d0a70 0x7f00241e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f0029908640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00240d0eb0 0x7f00241e8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f0029908640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002415e4b0 0x7f00240b9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f0029908640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00240d1340 con 0x7f002415e4b0 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f0029908640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f00240d2ab0 con 0x7f00240d0eb0 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f0029908640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f00240d2fd0 con 0x7f00240d0a70 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f00237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002415e4b0 0x7f00240b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f00237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002415e4b0 0x7f00240b9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43504/0 (socket says 172.21.15.172:43504) 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f00237fe640 1 -- 172.21.15.172:0/3662055005 learned_addr learned my addr 172.21.15.172:0/3662055005 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f0022ffd640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00240d0a70 0x7f00241e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f001bfff640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00240d0eb0 0x7f00241e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f00237fe640 1 -- 172.21.15.172:0/3662055005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00240d0a70 msgr2=0x7f00241e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f00237fe640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00240d0a70 0x7f00241e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f00237fe640 1 -- 172.21.15.172:0/3662055005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00240d0eb0 msgr2=0x7f00241e8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f00237fe640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00240d0eb0 0x7f00241e8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f00237fe640 1 -- 172.21.15.172:0/3662055005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00240b9c70 con 0x7f002415e4b0 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f00237fe640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002415e4b0 0x7f00240b9730 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f001c037040 tx=0x7f001c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:45.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.801+0000 7f0020ff9640 1 -- 172.21.15.172:0/3662055005 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c044020 con 0x7f002415e4b0 2024-04-14T08:36:45.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.802+0000 7f0020ff9640 1 -- 172.21.15.172:0/3662055005 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f001c034cf0 con 0x7f002415e4b0 2024-04-14T08:36:45.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.802+0000 7f0020ff9640 1 -- 172.21.15.172:0/3662055005 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c039040 con 0x7f002415e4b0 2024-04-14T08:36:45.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.802+0000 7f0029908640 1 -- 172.21.15.172:0/3662055005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00240c8290 con 0x7f002415e4b0 2024-04-14T08:36:45.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.802+0000 7f0029908640 1 -- 172.21.15.172:0/3662055005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00240c8800 con 0x7f002415e4b0 2024-04-14T08:36:45.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.803+0000 7f0029908640 1 -- 172.21.15.172:0/3662055005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0024068590 con 0x7f002415e4b0 2024-04-14T08:36:45.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.803+0000 7f0020ff9640 1 -- 172.21.15.172:0/3662055005 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f001c047020 con 0x7f002415e4b0 2024-04-14T08:36:45.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.803+0000 7f0020ff9640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f000406c8f0 0x7f000406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:45.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.803+0000 7f0022ffd640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f000406c8f0 0x7f000406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:45.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.803+0000 7f0020ff9640 1 -- 172.21.15.172:0/3662055005 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f001c0ae340 con 0x7f002415e4b0 2024-04-14T08:36:45.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.804+0000 7f0022ffd640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f000406c8f0 0x7f000406edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0014000f60 tx=0x7f0014009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:45.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.808+0000 7f0020ff9640 1 -- 172.21.15.172:0/3662055005 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f001c04e030 con 0x7f002415e4b0 2024-04-14T08:36:45.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.995+0000 7f0029908640 1 -- 172.21.15.172:0/3662055005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f00240caa60 con 0x7f002415e4b0 2024-04-14T08:36:45.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.996+0000 7f0020ff9640 1 -- 172.21.15.172:0/3662055005 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) ==== 80+0+0 (secure 0 0 0) 0x7f00240b9c70 con 0x7f002415e4b0 2024-04-14T08:36:45.997 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:36:45.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.998+0000 7f0029908640 1 -- 172.21.15.172:0/3662055005 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f000406c8f0 msgr2=0x7f000406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.998+0000 7f0029908640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f000406c8f0 0x7f000406edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0014000f60 tx=0x7f0014009040 comp rx=0 tx=0).stop 2024-04-14T08:36:46.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.998+0000 7f0029908640 1 -- 172.21.15.172:0/3662055005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002415e4b0 msgr2=0x7f00240b9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.998+0000 7f0029908640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002415e4b0 0x7f00240b9730 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f001c037040 tx=0x7f001c004a40 comp rx=0 tx=0).stop 2024-04-14T08:36:46.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.998+0000 7f0029908640 1 -- 172.21.15.172:0/3662055005 shutdown_connections 2024-04-14T08:36:46.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.998+0000 7f0029908640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f000406c8f0 0x7f000406edb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.998+0000 7f0029908640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f002415e4b0 0x7f00240b9730 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.998+0000 7f0029908640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00240d0eb0 0x7f00241e8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.998+0000 7f0029908640 1 --2- 172.21.15.172:0/3662055005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f00240d0a70 0x7f00241e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.998+0000 7f0029908640 1 -- 172.21.15.172:0/3662055005 >> 172.21.15.172:0/3662055005 conn(0x7f002409d570 msgr2=0x7f00240cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:46.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.998+0000 7f0029908640 1 -- 172.21.15.172:0/3662055005 shutdown_connections 2024-04-14T08:36:46.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:45.998+0000 7f0029908640 1 -- 172.21.15.172:0/3662055005 wait complete. 2024-04-14T08:36:46.010 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:36:46.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:36:46.010 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:36:46.061 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:36:46.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:36:46.061 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:36:46.083 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:36:46.084 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:36:46.084 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:36:46.099 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:36:46.099 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:36:46.099 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:36:46.099 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:36:46.139 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:36:46.139 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:36:46.139 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:36:46.139 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:36:46.154 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:36:46.155 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:36:46.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.278+0000 7ff7b8b7b640 1 Processor -- start 2024-04-14T08:36:46.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.278+0000 7ff7b8b7b640 1 -- start start 2024-04-14T08:36:46.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.279+0000 7ff7b8b7b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff7b415e890 0x7ff7b40d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.279+0000 7ff7b8b7b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b40cedd0 0x7ff7b40b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.279+0000 7ff7b8b7b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b40cf320 con 0x7ff7b40d2830 2024-04-14T08:36:46.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.279+0000 7ff7b8b7b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b40cff30 con 0x7ff7b415e890 2024-04-14T08:36:46.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.279+0000 7ff7b8b7b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b40d2350 con 0x7ff7b40cedd0 2024-04-14T08:36:46.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.280+0000 7ff7b1d9b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff7b415e890 0x7ff7b40d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.280+0000 7ff7b2d9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b40cedd0 0x7ff7b40b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.280+0000 7ff7b259c640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7ff7b40d2830 0x7ff7b40d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:49522/0 (socket says 172.21.15.172:49522) 2024-04-14T08:36:46.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.280+0000 7ff7b259c640 1 -- 172.21.15.172:0/2106116593 learned_addr learned my addr 172.21.15.172:0/2106116593 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:46.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.280+0000 7ff7b1d9b640 1 -- 172.21.15.172:0/2106116593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b40cedd0 msgr2=0x7ff7b40b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:46.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.280+0000 7ff7b1d9b640 1 --2- 172.21.15.172:0/2106116593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b40cedd0 0x7ff7b40b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.280+0000 7ff7b1d9b640 1 -- 172.21.15.172:0/2106116593 >> v1:172.21.15.177:6789/0 conn(0x7ff7b40d2830 legacy=0x7ff7b40d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.280+0000 7ff7b1d9b640 1 -- 172.21.15.172:0/2106116593 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b40cf600 con 0x7ff7b415e890 2024-04-14T08:36:46.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.280+0000 7ff7b2d9d640 1 --2- 172.21.15.172:0/2106116593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b40cedd0 0x7ff7b40b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:46.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.281+0000 7ff7b1d9b640 1 --2- 172.21.15.172:0/2106116593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b415e890 0x7ff7b40d1300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40097a0 tx=0x7ff7a402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a748fd82b4d5c801 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:46.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.281+0000 7ff7b159a640 1 -- 172.21.15.172:0/2106116593 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4034070 con 0x7ff7b415e890 2024-04-14T08:36:46.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.281+0000 7ff7b159a640 1 -- 172.21.15.172:0/2106116593 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a4030610 con 0x7ff7b415e890 2024-04-14T08:36:46.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.281+0000 7ff7b159a640 1 -- 172.21.15.172:0/2106116593 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40309e0 con 0x7ff7b415e890 2024-04-14T08:36:46.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.281+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/2106116593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b415e890 msgr2=0x7ff7b40d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.281+0000 7ff7b8b7b640 1 --2- 172.21.15.172:0/2106116593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b415e890 0x7ff7b40d1300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40097a0 tx=0x7ff7a402b960 comp rx=0 tx=0).stop 2024-04-14T08:36:46.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.281+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/2106116593 shutdown_connections 2024-04-14T08:36:46.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.281+0000 7ff7b8b7b640 1 --2- 172.21.15.172:0/2106116593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b40cedd0 0x7ff7b40b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.281+0000 7ff7b8b7b640 1 --2- 172.21.15.172:0/2106116593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b415e890 0x7ff7b40d1300 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.281+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/2106116593 >> 172.21.15.172:0/2106116593 conn(0x7ff7b409d590 msgr2=0x7ff7b40d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:46.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.281+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/2106116593 shutdown_connections 2024-04-14T08:36:46.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.282+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/2106116593 wait complete. 2024-04-14T08:36:46.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.283+0000 7ff7b8b7b640 1 Processor -- start 2024-04-14T08:36:46.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.283+0000 7ff7b8b7b640 1 -- start start 2024-04-14T08:36:46.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.283+0000 7ff7b8b7b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b40cedd0 0x7ff7b4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.283+0000 7ff7b8b7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b40d2830 0x7ff7b40c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.283+0000 7ff7b8b7b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b415e890 0x7ff7b40ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.283+0000 7ff7b8b7b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b40d1840 con 0x7ff7b40d2830 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.283+0000 7ff7b8b7b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b40cfbc0 con 0x7ff7b415e890 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.283+0000 7ff7b8b7b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b40b3c50 con 0x7ff7b40cedd0 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b40d2830 0x7ff7b40c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b40d2830 0x7ff7b40c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43516/0 (socket says 172.21.15.172:43516) 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b2d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b415e890 0x7ff7b40ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b1d9b640 1 -- 172.21.15.172:0/548888569 learned_addr learned my addr 172.21.15.172:0/548888569 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b259c640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b40cedd0 0x7ff7b4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b1d9b640 1 -- 172.21.15.172:0/548888569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b40cedd0 msgr2=0x7ff7b4157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b1d9b640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b40cedd0 0x7ff7b4157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b1d9b640 1 -- 172.21.15.172:0/548888569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b415e890 msgr2=0x7ff7b40ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b1d9b640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b415e890 0x7ff7b40ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b1d9b640 1 -- 172.21.15.172:0/548888569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b40c9310 con 0x7ff7b40d2830 2024-04-14T08:36:46.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b259c640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b40cedd0 0x7ff7b4157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:46.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b1d9b640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b40d2830 0x7ff7b40c8a10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40098c0 tx=0x7ff7a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:46.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7a2ffd640 1 -- 172.21.15.172:0/548888569 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4034040 con 0x7ff7b40d2830 2024-04-14T08:36:46.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7a2ffd640 1 -- 172.21.15.172:0/548888569 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a403aaa0 con 0x7ff7b40d2830 2024-04-14T08:36:46.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/548888569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b40c95f0 con 0x7ff7b40d2830 2024-04-14T08:36:46.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7a2ffd640 1 -- 172.21.15.172:0/548888569 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4006e10 con 0x7ff7b40d2830 2024-04-14T08:36:46.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.284+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/548888569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b415f6a0 con 0x7ff7b40d2830 2024-04-14T08:36:46.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.285+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/548888569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b40c9810 con 0x7ff7b40d2830 2024-04-14T08:36:46.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.286+0000 7ff7a2ffd640 1 -- 172.21.15.172:0/548888569 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7a404b060 con 0x7ff7b40d2830 2024-04-14T08:36:46.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.286+0000 7ff7a2ffd640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff79406c920 0x7ff79406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.286+0000 7ff7a2ffd640 1 -- 172.21.15.172:0/548888569 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7ff7a40ad9b0 con 0x7ff7b40d2830 2024-04-14T08:36:46.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.286+0000 7ff7b259c640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff79406c920 0x7ff79406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.287+0000 7ff7b259c640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff79406c920 0x7ff79406ede0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40a2110 tx=0x7ff7a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:46.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.291+0000 7ff7a2ffd640 1 -- 172.21.15.172:0/548888569 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff7a403f070 con 0x7ff7b40d2830 2024-04-14T08:36:46.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.503+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/548888569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7b415f1e0 con 0x7ff7b40d2830 2024-04-14T08:36:46.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.504+0000 7ff7a2ffd640 1 -- 172.21.15.172:0/548888569 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5195 (secure 0 0 0) 0x7ff7b415f1e0 con 0x7ff7b40d2830 2024-04-14T08:36:46.506 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:46.506 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3621780015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3621780015},{"type":"v1","addr":"172.21.15.172:6836","nonce":3621780015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3894683748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3894683748},{"type":"v1","addr":"172.21.15.172:6837","nonce":3894683748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3625745587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3625745587},{"type":"v1","addr":"172.21.15.177:6834","nonce":3625745587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:35:34.316542+0000","modified":"2024-04-14T08:36:14.993490+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6835/4176013980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":4176013980},{"type":"v1","addr":"172.21.15.177:6835","nonce":4176013980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-14T08:36:46.506 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 32 2024-04-14T08:36:46.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.506+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/548888569 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff79406c920 msgr2=0x7ff79406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.506+0000 7ff7b8b7b640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff79406c920 0x7ff79406ede0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40a2110 tx=0x7ff7a8009040 comp rx=0 tx=0).stop 2024-04-14T08:36:46.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.506+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/548888569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b40d2830 msgr2=0x7ff7b40c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.506+0000 7ff7b8b7b640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b40d2830 0x7ff7b40c8a10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40098c0 tx=0x7ff7a40046c0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.506+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/548888569 shutdown_connections 2024-04-14T08:36:46.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.506+0000 7ff7b8b7b640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff79406c920 0x7ff79406ede0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.506+0000 7ff7b8b7b640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7b415e890 0x7ff7b40ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.506+0000 7ff7b8b7b640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7b40d2830 0x7ff7b40c8a10 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.506+0000 7ff7b8b7b640 1 --2- 172.21.15.172:0/548888569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7b40cedd0 0x7ff7b4157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.506+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/548888569 >> 172.21.15.172:0/548888569 conn(0x7ff7b409d590 msgr2=0x7ff7b40d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:46.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.506+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/548888569 shutdown_connections 2024-04-14T08:36:46.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.506+0000 7ff7b8b7b640 1 -- 172.21.15.172:0/548888569 wait complete. 2024-04-14T08:36:46.518 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:36:46.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.617+0000 7fb69ac77640 1 Processor -- start 2024-04-14T08:36:46.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.617+0000 7fb69ac77640 1 -- start start 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb69ac77640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6940729b0 0x7fb69406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb69ac77640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb69406e160 0x7fb69406c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb69ac77640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6940b41d0 con 0x7fb69415e5b0 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb69ac77640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb6940aebe0 con 0x7fb69406e160 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb69ac77640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb6940b4470 con 0x7fb6940729b0 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb699214640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb69406e160 0x7fb69406c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb699214640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb69406e160 0x7fb69406c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43522/0 (socket says 172.21.15.172:43522) 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb699214640 1 -- 172.21.15.172:0/120087054 learned_addr learned my addr 172.21.15.172:0/120087054 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb693fff640 1 --2- 172.21.15.172:0/120087054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6940729b0 0x7fb69406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb699214640 1 -- 172.21.15.172:0/120087054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6940729b0 msgr2=0x7fb69406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb699214640 1 --2- 172.21.15.172:0/120087054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6940729b0 0x7fb69406dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb699214640 1 -- 172.21.15.172:0/120087054 >> v1:172.21.15.177:6789/0 conn(0x7fb69415e5b0 legacy=0x7fb6940ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb699214640 1 -- 172.21.15.172:0/120087054 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6940aeec0 con 0x7fb69406e160 2024-04-14T08:36:46.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.618+0000 7fb6937fe640 1 -- 172.21.15.172:0/120087054 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1836659540 0 0) 0x7fb6940b41d0 con 0x7fb69415e5b0 2024-04-14T08:36:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.619+0000 7fb699214640 1 --2- 172.21.15.172:0/120087054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69406e160 0x7fb69406c530 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009e50 tx=0x7fb68c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3fb3d74374b534 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.619+0000 7fb6937fe640 1 -- 172.21.15.172:0/120087054 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c02cb00 con 0x7fb69406e160 2024-04-14T08:36:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.619+0000 7fb6937fe640 1 -- 172.21.15.172:0/120087054 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb68c02cca0 con 0x7fb69406e160 2024-04-14T08:36:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.619+0000 7fb6937fe640 1 -- 172.21.15.172:0/120087054 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c034610 con 0x7fb69406e160 2024-04-14T08:36:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.620+0000 7fb69ac77640 1 -- 172.21.15.172:0/120087054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69406e160 msgr2=0x7fb69406c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.620+0000 7fb69ac77640 1 --2- 172.21.15.172:0/120087054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69406e160 0x7fb69406c530 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009e50 tx=0x7fb68c02bf60 comp rx=0 tx=0).stop 2024-04-14T08:36:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.620+0000 7fb69ac77640 1 -- 172.21.15.172:0/120087054 shutdown_connections 2024-04-14T08:36:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.620+0000 7fb69ac77640 1 --2- 172.21.15.172:0/120087054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69406e160 0x7fb69406c530 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.620+0000 7fb69ac77640 1 --2- 172.21.15.172:0/120087054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6940729b0 0x7fb69406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.620+0000 7fb69ac77640 1 -- 172.21.15.172:0/120087054 >> 172.21.15.172:0/120087054 conn(0x7fb694071ba0 msgr2=0x7fb694071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.620+0000 7fb69ac77640 1 -- 172.21.15.172:0/120087054 shutdown_connections 2024-04-14T08:36:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.620+0000 7fb69ac77640 1 -- 172.21.15.172:0/120087054 wait complete. 2024-04-14T08:36:46.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.621+0000 7fb69ac77640 1 Processor -- start 2024-04-14T08:36:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.621+0000 7fb69ac77640 1 -- start start 2024-04-14T08:36:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb69ac77640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb69406e160 0x7fb6941591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb69ac77640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6940729b0 0x7fb6941607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb69ac77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69415e5b0 0x7fb694160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb69ac77640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb69406ce00 con 0x7fb69415e5b0 2024-04-14T08:36:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb69ac77640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb69406cc80 con 0x7fb6940729b0 2024-04-14T08:36:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb69ac77640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb6940b4c70 con 0x7fb69406e160 2024-04-14T08:36:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb699214640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69415e5b0 0x7fb694160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb699214640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69415e5b0 0x7fb694160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43538/0 (socket says 172.21.15.172:43538) 2024-04-14T08:36:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb699214640 1 -- 172.21.15.172:0/802121732 learned_addr learned my addr 172.21.15.172:0/802121732 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:46.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb693fff640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6940729b0 0x7fb6941607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb698a13640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb69406e160 0x7fb6941591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb699214640 1 -- 172.21.15.172:0/802121732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb69406e160 msgr2=0x7fb6941591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb699214640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb69406e160 0x7fb6941591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb699214640 1 -- 172.21.15.172:0/802121732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6940729b0 msgr2=0x7fb6941607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb699214640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6940729b0 0x7fb6941607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb699214640 1 -- 172.21.15.172:0/802121732 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6940ce090 con 0x7fb69415e5b0 2024-04-14T08:36:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb699214640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69415e5b0 0x7fb694160ce0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb68c036040 tx=0x7fb68c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb6917fa640 1 -- 172.21.15.172:0/802121732 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c043030 con 0x7fb69415e5b0 2024-04-14T08:36:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb6917fa640 1 -- 172.21.15.172:0/802121732 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb68c02ce50 con 0x7fb69415e5b0 2024-04-14T08:36:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb69ac77640 1 -- 172.21.15.172:0/802121732 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6940ce370 con 0x7fb69415e5b0 2024-04-14T08:36:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.622+0000 7fb6917fa640 1 -- 172.21.15.172:0/802121732 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c009510 con 0x7fb69415e5b0 2024-04-14T08:36:46.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.623+0000 7fb69ac77640 1 -- 172.21.15.172:0/802121732 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6940ce930 con 0x7fb69415e5b0 2024-04-14T08:36:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.624+0000 7fb69ac77640 1 -- 172.21.15.172:0/802121732 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb694068520 con 0x7fb69415e5b0 2024-04-14T08:36:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.624+0000 7fb6917fa640 1 -- 172.21.15.172:0/802121732 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb68c046070 con 0x7fb69415e5b0 2024-04-14T08:36:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.624+0000 7fb6917fa640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb67806c8f0 0x7fb67806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.624+0000 7fb6917fa640 1 -- 172.21.15.172:0/802121732 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fb68c0add60 con 0x7fb69415e5b0 2024-04-14T08:36:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.625+0000 7fb698a13640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb67806c8f0 0x7fb67806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.625+0000 7fb698a13640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb67806c8f0 0x7fb67806edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb688000f60 tx=0x7fb688009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.629+0000 7fb6917fa640 1 -- 172.21.15.172:0/802121732 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb68c04d070 con 0x7fb69415e5b0 2024-04-14T08:36:46.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.804+0000 7fb69ac77640 1 -- 172.21.15.172:0/802121732 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb69415d810 con 0x7fb69415e5b0 2024-04-14T08:36:46.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.805+0000 7fb6917fa640 1 -- 172.21.15.172:0/802121732 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17184 (secure 0 0 0) 0x7fb68c07b920 con 0x7fb69415e5b0 2024-04-14T08:36:46.807 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:46.807 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":63,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:36:35.260134+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:35:32.539067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-14T08:35:33.263123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.17","mappings":[{"from":4,"to":5}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:36:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.807+0000 7fb69ac77640 1 -- 172.21.15.172:0/802121732 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb67806c8f0 msgr2=0x7fb67806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.807+0000 7fb69ac77640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb67806c8f0 0x7fb67806edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb688000f60 tx=0x7fb688009040 comp rx=0 tx=0).stop 2024-04-14T08:36:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.807+0000 7fb69ac77640 1 -- 172.21.15.172:0/802121732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69415e5b0 msgr2=0x7fb694160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.807+0000 7fb69ac77640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69415e5b0 0x7fb694160ce0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb68c036040 tx=0x7fb68c004a40 comp rx=0 tx=0).stop 2024-04-14T08:36:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.807+0000 7fb69ac77640 1 -- 172.21.15.172:0/802121732 shutdown_connections 2024-04-14T08:36:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.807+0000 7fb69ac77640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb67806c8f0 0x7fb67806edb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.807+0000 7fb69ac77640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69415e5b0 0x7fb694160ce0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.807+0000 7fb69ac77640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6940729b0 0x7fb6941607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.807+0000 7fb69ac77640 1 --2- 172.21.15.172:0/802121732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb69406e160 0x7fb6941591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.807+0000 7fb69ac77640 1 -- 172.21.15.172:0/802121732 >> 172.21.15.172:0/802121732 conn(0x7fb694071ba0 msgr2=0x7fb6940b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:46.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.808+0000 7fb69ac77640 1 -- 172.21.15.172:0/802121732 shutdown_connections 2024-04-14T08:36:46.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.808+0000 7fb69ac77640 1 -- 172.21.15.172:0/802121732 wait complete. 2024-04-14T08:36:46.820 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:35:32.539067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:35:33.263123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:36:46.820 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:36:46.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.923+0000 7f4d5d5a3640 1 Processor -- start 2024-04-14T08:36:46.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.923+0000 7f4d5d5a3640 1 -- start start 2024-04-14T08:36:46.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d5d5a3640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4d5815e5b0 0x7f4d580d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d5d5a3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d580cff30 0x7f4d580d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d5d5a3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d580cf170 con 0x7f4d580d27f0 2024-04-14T08:36:46.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d5d5a3640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f4d580cf450 con 0x7f4d580cff30 2024-04-14T08:36:46.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d5d5a3640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f4d580cf730 con 0x7f4d5815e5b0 2024-04-14T08:36:46.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d4ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d580cff30 0x7f4d580d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d577fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4d580d27f0 0x7f4d58162160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47082/0 (socket says 172.21.15.172:47082) 2024-04-14T08:36:46.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d577fe640 1 -- 172.21.15.172:0/2493804569 learned_addr learned my addr 172.21.15.172:0/2493804569 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:46.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d56ffd640 1 --2- 172.21.15.172:0/2493804569 >> v2:172.21.15.177:3300/0 conn(0x7f4d5815e5b0 0x7f4d580d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d4ffff640 1 -- 172.21.15.172:0/2493804569 >> v2:172.21.15.177:3300/0 conn(0x7f4d5815e5b0 msgr2=0x7f4d580d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:46.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d4ffff640 1 --2- 172.21.15.172:0/2493804569 >> v2:172.21.15.177:3300/0 conn(0x7f4d5815e5b0 0x7f4d580d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d4ffff640 1 -- 172.21.15.172:0/2493804569 >> v1:172.21.15.172:6789/0 conn(0x7f4d580d27f0 legacy=0x7f4d58162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d4ffff640 1 -- 172.21.15.172:0/2493804569 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d580cfa10 con 0x7f4d580cff30 2024-04-14T08:36:46.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.924+0000 7f4d567fc640 1 -- 172.21.15.172:0/2493804569 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1859779262 0 0) 0x7f4d580cf170 con 0x7f4d580d27f0 2024-04-14T08:36:46.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.925+0000 7f4d4ffff640 1 --2- 172.21.15.172:0/2493804569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d580cff30 0x7f4d580d4b30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4d580d4170 tx=0x7f4d4002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb111821f24dc09d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:46.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.925+0000 7f4d567fc640 1 -- 172.21.15.172:0/2493804569 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d400315a0 con 0x7f4d580cff30 2024-04-14T08:36:46.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.925+0000 7f4d567fc640 1 -- 172.21.15.172:0/2493804569 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d40037070 con 0x7f4d580cff30 2024-04-14T08:36:46.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.925+0000 7f4d567fc640 1 -- 172.21.15.172:0/2493804569 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40031960 con 0x7f4d580cff30 2024-04-14T08:36:46.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.926+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/2493804569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d580cff30 msgr2=0x7f4d580d4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.926+0000 7f4d5d5a3640 1 --2- 172.21.15.172:0/2493804569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d580cff30 0x7f4d580d4b30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4d580d4170 tx=0x7f4d4002b840 comp rx=0 tx=0).stop 2024-04-14T08:36:46.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.926+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/2493804569 shutdown_connections 2024-04-14T08:36:46.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.926+0000 7f4d5d5a3640 1 --2- 172.21.15.172:0/2493804569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d580cff30 0x7f4d580d4b30 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.926+0000 7f4d5d5a3640 1 --2- 172.21.15.172:0/2493804569 >> v2:172.21.15.177:3300/0 conn(0x7f4d5815e5b0 0x7f4d580d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.926+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/2493804569 >> 172.21.15.172:0/2493804569 conn(0x7f4d5809d550 msgr2=0x7f4d580d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:46.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.926+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/2493804569 shutdown_connections 2024-04-14T08:36:46.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.926+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/2493804569 wait complete. 2024-04-14T08:36:46.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.927+0000 7f4d5d5a3640 1 Processor -- start 2024-04-14T08:36:46.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.927+0000 7f4d5d5a3640 1 -- start start 2024-04-14T08:36:46.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d5d5a3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d580cff30 0x7f4d5814d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d5d5a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d580d27f0 0x7f4d580b9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d5d5a3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d5815e5b0 0x7f4d580c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d5d5a3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4d580d2c40 con 0x7f4d580d27f0 2024-04-14T08:36:46.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d5d5a3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5815e990 con 0x7f4d580cff30 2024-04-14T08:36:46.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d5d5a3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4d580d2dc0 con 0x7f4d5815e5b0 2024-04-14T08:36:46.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d4ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d580d27f0 0x7f4d580b9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d4ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d580d27f0 0x7f4d580b9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43552/0 (socket says 172.21.15.172:43552) 2024-04-14T08:36:46.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d4ffff640 1 -- 172.21.15.172:0/3401345675 learned_addr learned my addr 172.21.15.172:0/3401345675 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:46.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d56ffd640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d580cff30 0x7f4d5814d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d577fe640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d5815e5b0 0x7f4d580c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d4ffff640 1 -- 172.21.15.172:0/3401345675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d5815e5b0 msgr2=0x7f4d580c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d4ffff640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d5815e5b0 0x7f4d580c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d4ffff640 1 -- 172.21.15.172:0/3401345675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d580cff30 msgr2=0x7f4d5814d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d4ffff640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d580cff30 0x7f4d5814d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d4ffff640 1 -- 172.21.15.172:0/3401345675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5814e590 con 0x7f4d580d27f0 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d56ffd640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d580cff30 0x7f4d5814d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d4ffff640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d580d27f0 0x7f4d580b9020 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009890 tx=0x7f4d400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d4f7fe640 1 -- 172.21.15.172:0/3401345675 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40032630 con 0x7f4d580d27f0 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d4f7fe640 1 -- 172.21.15.172:0/3401345675 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d400327f0 con 0x7f4d580d27f0 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/3401345675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5814e870 con 0x7f4d580d27f0 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d4f7fe640 1 -- 172.21.15.172:0/3401345675 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40032be0 con 0x7f4d580d27f0 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.928+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/3401345675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5814ee30 con 0x7f4d580d27f0 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.929+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/3401345675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d58068590 con 0x7f4d580d27f0 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.930+0000 7f4d4f7fe640 1 -- 172.21.15.172:0/3401345675 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d4004c020 con 0x7f4d580d27f0 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.930+0000 7f4d4f7fe640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d3806c920 0x7f4d3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:46.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.930+0000 7f4d4f7fe640 1 -- 172.21.15.172:0/3401345675 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f4d4002f020 con 0x7f4d580d27f0 2024-04-14T08:36:46.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.930+0000 7f4d56ffd640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d3806c920 0x7f4d3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:46.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.931+0000 7f4d56ffd640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d3806c920 0x7f4d3806ede0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009e90 tx=0x7f4d48009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:46.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:46.935+0000 7f4d4f7fe640 1 -- 172.21.15.172:0/3401345675 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4d40040070 con 0x7f4d580d27f0 2024-04-14T08:36:47.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.111+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/3401345675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d580d25d0 con 0x7f4d580d27f0 2024-04-14T08:36:47.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.111+0000 7f4d4f7fe640 1 -- 172.21.15.172:0/3401345675 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7f4d4007b440 con 0x7f4d580d27f0 2024-04-14T08:36:47.113 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:36:47.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.113+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/3401345675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d3806c920 msgr2=0x7f4d3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.113+0000 7f4d5d5a3640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d3806c920 0x7f4d3806ede0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009e90 tx=0x7f4d48009190 comp rx=0 tx=0).stop 2024-04-14T08:36:47.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.113+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/3401345675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d580d27f0 msgr2=0x7f4d580b9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.113+0000 7f4d5d5a3640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d580d27f0 0x7f4d580b9020 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009890 tx=0x7f4d400046c0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.113+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/3401345675 shutdown_connections 2024-04-14T08:36:47.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.113+0000 7f4d5d5a3640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d3806c920 0x7f4d3806ede0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.113+0000 7f4d5d5a3640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d5815e5b0 0x7f4d580c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.113+0000 7f4d5d5a3640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d580d27f0 0x7f4d580b9020 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.113+0000 7f4d5d5a3640 1 --2- 172.21.15.172:0/3401345675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d580cff30 0x7f4d5814d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.113+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/3401345675 >> 172.21.15.172:0/3401345675 conn(0x7f4d5809d550 msgr2=0x7f4d580cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:47.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.113+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/3401345675 shutdown_connections 2024-04-14T08:36:47.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.113+0000 7f4d5d5a3640 1 -- 172.21.15.172:0/3401345675 wait complete. 2024-04-14T08:36:47.125 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:36:47.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.229+0000 7f31ebbbb640 1 Processor -- start 2024-04-14T08:36:47.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.229+0000 7f31ebbbb640 1 -- start start 2024-04-14T08:36:47.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.230+0000 7f31ebbbb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f31e4162710 0x7f31e40d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.230+0000 7f31ebbbb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e40cefb0 con 0x7f31e415e430 2024-04-14T08:36:47.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.230+0000 7f31ebbbb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e40cf290 con 0x7f31e4162ba0 2024-04-14T08:36:47.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.230+0000 7f31ebbbb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f31e40cf570 con 0x7f31e4162710 2024-04-14T08:36:47.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.230+0000 7f31e9156640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f31e4162710 0x7f31e40d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.230+0000 7f31ea158640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f31e4162ba0 0x7f31e40d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:49542/0 (socket says 172.21.15.172:49542) 2024-04-14T08:36:47.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.230+0000 7f31e9957640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f31e415e430 0x7f31e4162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47094/0 (socket says 172.21.15.172:47094) 2024-04-14T08:36:47.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.230+0000 7f31e9957640 1 -- 172.21.15.172:0/4019636890 learned_addr learned my addr 172.21.15.172:0/4019636890 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:47.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.230+0000 7f31e9156640 1 -- 172.21.15.172:0/4019636890 >> v1:172.21.15.177:6789/0 conn(0x7f31e4162ba0 legacy=0x7f31e40d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.230+0000 7f31e9156640 1 -- 172.21.15.172:0/4019636890 >> v1:172.21.15.172:6789/0 conn(0x7f31e415e430 legacy=0x7f31e4162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.230+0000 7f31e8955640 1 -- 172.21.15.172:0/4019636890 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3963385682 0 0) 0x7f31e40cf290 con 0x7f31e4162ba0 2024-04-14T08:36:47.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.230+0000 7f31e9156640 1 -- 172.21.15.172:0/4019636890 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e40cf850 con 0x7f31e4162710 2024-04-14T08:36:47.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.231+0000 7f31e9156640 1 --2- 172.21.15.172:0/4019636890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e4162710 0x7f31e40d1df0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f31d40097a0 tx=0x7f31d402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4377fb95666d43df server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:47.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.231+0000 7f31e8955640 1 -- 172.21.15.172:0/4019636890 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4037070 con 0x7f31e4162710 2024-04-14T08:36:47.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.231+0000 7f31e8955640 1 -- 172.21.15.172:0/4019636890 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d4030710 con 0x7f31e4162710 2024-04-14T08:36:47.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.231+0000 7f31e8955640 1 -- 172.21.15.172:0/4019636890 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4030ae0 con 0x7f31e4162710 2024-04-14T08:36:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.232+0000 7f31ebbbb640 1 -- 172.21.15.172:0/4019636890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e4162710 msgr2=0x7f31e40d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.232+0000 7f31ebbbb640 1 --2- 172.21.15.172:0/4019636890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e4162710 0x7f31e40d1df0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f31d40097a0 tx=0x7f31d402b790 comp rx=0 tx=0).stop 2024-04-14T08:36:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.232+0000 7f31ebbbb640 1 -- 172.21.15.172:0/4019636890 shutdown_connections 2024-04-14T08:36:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.232+0000 7f31ebbbb640 1 --2- 172.21.15.172:0/4019636890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e4162710 0x7f31e40d1df0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.232+0000 7f31ebbbb640 1 -- 172.21.15.172:0/4019636890 >> 172.21.15.172:0/4019636890 conn(0x7f31e409d560 msgr2=0x7f31e40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.232+0000 7f31ebbbb640 1 -- 172.21.15.172:0/4019636890 shutdown_connections 2024-04-14T08:36:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.232+0000 7f31ebbbb640 1 -- 172.21.15.172:0/4019636890 wait complete. 2024-04-14T08:36:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.233+0000 7f31ebbbb640 1 Processor -- start 2024-04-14T08:36:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.233+0000 7f31ebbbb640 1 -- start start 2024-04-14T08:36:47.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.233+0000 7f31ebbbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31e415e430 0x7f31e40b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ebbbb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31e4162710 0x7f31e41f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ebbbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e4162ba0 0x7f31e40b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ebbbb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f31e40d4e20 con 0x7f31e4162ba0 2024-04-14T08:36:47.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ebbbb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f31e415e810 con 0x7f31e4162710 2024-04-14T08:36:47.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ebbbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f31e40d0050 con 0x7f31e415e430 2024-04-14T08:36:47.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ea158640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e4162ba0 0x7f31e40b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ea158640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e4162ba0 0x7f31e40b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43572/0 (socket says 172.21.15.172:43572) 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ea158640 1 -- 172.21.15.172:0/661913454 learned_addr learned my addr 172.21.15.172:0/661913454 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31e9156640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31e4162710 0x7f31e41f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31e9957640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31e415e430 0x7f31e40b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ea158640 1 -- 172.21.15.172:0/661913454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31e415e430 msgr2=0x7f31e40b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ea158640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31e415e430 0x7f31e40b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ea158640 1 -- 172.21.15.172:0/661913454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31e4162710 msgr2=0x7f31e41f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ea158640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31e4162710 0x7f31e41f3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ea158640 1 -- 172.21.15.172:0/661913454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e40b3f50 con 0x7f31e4162ba0 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ea158640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e4162ba0 0x7f31e40b7e30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c7a0 tx=0x7f31dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31da7fc640 1 -- 172.21.15.172:0/661913454 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc018070 con 0x7f31e4162ba0 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31da7fc640 1 -- 172.21.15.172:0/661913454 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31dc00a640 con 0x7f31e4162ba0 2024-04-14T08:36:47.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ebbbb640 1 -- 172.21.15.172:0/661913454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e40b41f0 con 0x7f31e4162ba0 2024-04-14T08:36:47.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31da7fc640 1 -- 172.21.15.172:0/661913454 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc00aab0 con 0x7f31e4162ba0 2024-04-14T08:36:47.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.234+0000 7f31ebbbb640 1 -- 172.21.15.172:0/661913454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e40b47b0 con 0x7f31e4162ba0 2024-04-14T08:36:47.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.235+0000 7f31ebbbb640 1 -- 172.21.15.172:0/661913454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e4068590 con 0x7f31e4162ba0 2024-04-14T08:36:47.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.236+0000 7f31da7fc640 1 -- 172.21.15.172:0/661913454 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31dc01b070 con 0x7f31e4162ba0 2024-04-14T08:36:47.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.236+0000 7f31da7fc640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f31c806c8f0 0x7f31c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.236+0000 7f31e9957640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f31c806c8f0 0x7f31c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.236+0000 7f31da7fc640 1 -- 172.21.15.172:0/661913454 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f31dc08b220 con 0x7f31e4162ba0 2024-04-14T08:36:47.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.237+0000 7f31e9957640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f31c806c8f0 0x7f31c806edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f31e0000f60 tx=0x7f31e0007e20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:47.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.241+0000 7f31da7fc640 1 -- 172.21.15.172:0/661913454 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f31dc058e70 con 0x7f31e4162ba0 2024-04-14T08:36:47.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.415+0000 7f31ebbbb640 1 -- 172.21.15.172:0/661913454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f31e40b1f90 con 0x7f31e4162ba0 2024-04-14T08:36:47.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.415+0000 7f31da7fc640 1 -- 172.21.15.172:0/661913454 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) ==== 104+0+11 (secure 0 0 0) 0x7f31dc05d170 con 0x7f31e4162ba0 2024-04-14T08:36:47.417 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:36:47.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.417+0000 7f31ebbbb640 1 -- 172.21.15.172:0/661913454 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f31c806c8f0 msgr2=0x7f31c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.417+0000 7f31ebbbb640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f31c806c8f0 0x7f31c806edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f31e0000f60 tx=0x7f31e0007e20 comp rx=0 tx=0).stop 2024-04-14T08:36:47.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.417+0000 7f31ebbbb640 1 -- 172.21.15.172:0/661913454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e4162ba0 msgr2=0x7f31e40b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.417+0000 7f31ebbbb640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e4162ba0 0x7f31e40b7e30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c7a0 tx=0x7f31dc00cc60 comp rx=0 tx=0).stop 2024-04-14T08:36:47.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.417+0000 7f31ebbbb640 1 -- 172.21.15.172:0/661913454 shutdown_connections 2024-04-14T08:36:47.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.417+0000 7f31ebbbb640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f31c806c8f0 0x7f31c806edb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.417+0000 7f31ebbbb640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e4162ba0 0x7f31e40b7e30 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.417+0000 7f31ebbbb640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31e4162710 0x7f31e41f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.417+0000 7f31ebbbb640 1 --2- 172.21.15.172:0/661913454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31e415e430 0x7f31e40b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.417+0000 7f31ebbbb640 1 -- 172.21.15.172:0/661913454 >> 172.21.15.172:0/661913454 conn(0x7f31e409d560 msgr2=0x7f31e415c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:47.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.417+0000 7f31ebbbb640 1 -- 172.21.15.172:0/661913454 shutdown_connections 2024-04-14T08:36:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.417+0000 7f31ebbbb640 1 -- 172.21.15.172:0/661913454 wait complete. 2024-04-14T08:36:47.431 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:36:47.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.533+0000 7ff7363fe640 1 Processor -- start 2024-04-14T08:36:47.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.533+0000 7ff7363fe640 1 -- start start 2024-04-14T08:36:47.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.533+0000 7ff7363fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff730071250 0x7ff73006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.533+0000 7ff7363fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff73006d000 0x7ff73006d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.533+0000 7ff7363fe640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7300701b0 con 0x7ff7300d1a20 2024-04-14T08:36:47.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.533+0000 7ff7363fe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff73006d9d0 con 0x7ff73006d000 2024-04-14T08:36:47.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.533+0000 7ff7363fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff730070dc0 con 0x7ff730071250 2024-04-14T08:36:47.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.534+0000 7ff73499b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff73006d000 0x7ff73006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.534+0000 7ff73499b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff73006d000 0x7ff73006d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43586/0 (socket says 172.21.15.172:43586) 2024-04-14T08:36:47.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.534+0000 7ff72f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff730071250 0x7ff73006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.534+0000 7ff73499b640 1 -- 172.21.15.172:0/645135943 learned_addr learned my addr 172.21.15.172:0/645135943 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:47.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.534+0000 7ff72ffff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7ff7300d1a20 0x7ff7300cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:49558/0 (socket says 172.21.15.172:49558) 2024-04-14T08:36:47.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.534+0000 7ff73499b640 1 -- 172.21.15.172:0/645135943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff730071250 msgr2=0x7ff73006fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:47.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.534+0000 7ff73499b640 1 --2- 172.21.15.172:0/645135943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff730071250 0x7ff73006fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.534+0000 7ff73499b640 1 -- 172.21.15.172:0/645135943 >> v1:172.21.15.177:6789/0 conn(0x7ff7300d1a20 legacy=0x7ff7300cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.534+0000 7ff72effd640 1 -- 172.21.15.172:0/645135943 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 252173342 0 0) 0x7ff7300701b0 con 0x7ff7300d1a20 2024-04-14T08:36:47.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.534+0000 7ff73499b640 1 -- 172.21.15.172:0/645135943 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff730070490 con 0x7ff73006d000 2024-04-14T08:36:47.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.535+0000 7ff73499b640 1 --2- 172.21.15.172:0/645135943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff73006d000 0x7ff73006d490 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff728009e50 tx=0x7ff72802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8caa3a6fc5d775 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:47.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.535+0000 7ff72effd640 1 -- 172.21.15.172:0/645135943 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72802ca70 con 0x7ff73006d000 2024-04-14T08:36:47.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.535+0000 7ff72effd640 1 -- 172.21.15.172:0/645135943 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff72802cc10 con 0x7ff73006d000 2024-04-14T08:36:47.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.535+0000 7ff72effd640 1 -- 172.21.15.172:0/645135943 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7280343f0 con 0x7ff73006d000 2024-04-14T08:36:47.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.535+0000 7ff7363fe640 1 -- 172.21.15.172:0/645135943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff73006d000 msgr2=0x7ff73006d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.535+0000 7ff7363fe640 1 --2- 172.21.15.172:0/645135943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff73006d000 0x7ff73006d490 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff728009e50 tx=0x7ff72802bed0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.535+0000 7ff7363fe640 1 -- 172.21.15.172:0/645135943 shutdown_connections 2024-04-14T08:36:47.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.535+0000 7ff7363fe640 1 --2- 172.21.15.172:0/645135943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff73006d000 0x7ff73006d490 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.535+0000 7ff7363fe640 1 --2- 172.21.15.172:0/645135943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff730071250 0x7ff73006fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.535+0000 7ff7363fe640 1 -- 172.21.15.172:0/645135943 >> 172.21.15.172:0/645135943 conn(0x7ff73008c000 msgr2=0x7ff7300d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:47.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.535+0000 7ff7363fe640 1 -- 172.21.15.172:0/645135943 shutdown_connections 2024-04-14T08:36:47.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.535+0000 7ff7363fe640 1 -- 172.21.15.172:0/645135943 wait complete. 2024-04-14T08:36:47.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.537+0000 7ff7363fe640 1 Processor -- start 2024-04-14T08:36:47.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.537+0000 7ff7363fe640 1 -- start start 2024-04-14T08:36:47.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.537+0000 7ff7363fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff73006d000 0x7ff7301f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.537+0000 7ff7363fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff730071250 0x7ff7300b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.537+0000 7ff7363fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7300d1a20 0x7ff7301f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.537+0000 7ff7363fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7300d1e00 con 0x7ff730071250 2024-04-14T08:36:47.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.537+0000 7ff7363fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff730070a50 con 0x7ff7300d1a20 2024-04-14T08:36:47.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.537+0000 7ff7363fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff7300c9f90 con 0x7ff73006d000 2024-04-14T08:36:47.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff730071250 0x7ff7300b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff730071250 0x7ff7300b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43590/0 (socket says 172.21.15.172:43590) 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72f7fe640 1 -- 172.21.15.172:0/671087647 learned_addr learned my addr 172.21.15.172:0/671087647 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72ffff640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff73006d000 0x7ff7301f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff73499b640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7300d1a20 0x7ff7301f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72f7fe640 1 -- 172.21.15.172:0/671087647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff73006d000 msgr2=0x7ff7301f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72f7fe640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff73006d000 0x7ff7301f1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72f7fe640 1 -- 172.21.15.172:0/671087647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7300d1a20 msgr2=0x7ff7301f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72f7fe640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7300d1a20 0x7ff7301f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72f7fe640 1 -- 172.21.15.172:0/671087647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7300c9380 con 0x7ff730071250 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72f7fe640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff730071250 0x7ff7300b5740 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff72400e980 tx=0x7ff72400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72cff9640 1 -- 172.21.15.172:0/671087647 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724017070 con 0x7ff730071250 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72cff9640 1 -- 172.21.15.172:0/671087647 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff72400ca00 con 0x7ff730071250 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff7363fe640 1 -- 172.21.15.172:0/671087647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7300c9660 con 0x7ff730071250 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff72cff9640 1 -- 172.21.15.172:0/671087647 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72400cdd0 con 0x7ff730071250 2024-04-14T08:36:47.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.538+0000 7ff7363fe640 1 -- 172.21.15.172:0/671087647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7300c9c20 con 0x7ff730071250 2024-04-14T08:36:47.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.539+0000 7ff7363fe640 1 -- 172.21.15.172:0/671087647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff730075490 con 0x7ff730071250 2024-04-14T08:36:47.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.540+0000 7ff72cff9640 1 -- 172.21.15.172:0/671087647 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff72401a070 con 0x7ff730071250 2024-04-14T08:36:47.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.540+0000 7ff72cff9640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff71c06c8f0 0x7ff71c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.540+0000 7ff72ffff640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff71c06c8f0 0x7ff71c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.540+0000 7ff72cff9640 1 -- 172.21.15.172:0/671087647 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7ff72408bfb0 con 0x7ff730071250 2024-04-14T08:36:47.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.541+0000 7ff72ffff640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff71c06c8f0 0x7ff71c06edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff720000f60 tx=0x7ff720007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:47.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.545+0000 7ff72cff9640 1 -- 172.21.15.172:0/671087647 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff724059c00 con 0x7ff730071250 2024-04-14T08:36:47.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.721+0000 7ff7363fe640 1 -- 172.21.15.172:0/671087647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff7300d2820 con 0x7ff730071250 2024-04-14T08:36:47.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.721+0000 7ff72cff9640 1 -- 172.21.15.172:0/671087647 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) ==== 100+0+11 (secure 0 0 0) 0x7ff72405df00 con 0x7ff730071250 2024-04-14T08:36:47.723 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:36:47.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.723+0000 7ff7363fe640 1 -- 172.21.15.172:0/671087647 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff71c06c8f0 msgr2=0x7ff71c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.723+0000 7ff7363fe640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff71c06c8f0 0x7ff71c06edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff720000f60 tx=0x7ff720007d50 comp rx=0 tx=0).stop 2024-04-14T08:36:47.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.723+0000 7ff7363fe640 1 -- 172.21.15.172:0/671087647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff730071250 msgr2=0x7ff7300b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.723+0000 7ff7363fe640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff730071250 0x7ff7300b5740 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff72400e980 tx=0x7ff72400ee40 comp rx=0 tx=0).stop 2024-04-14T08:36:47.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.723+0000 7ff7363fe640 1 -- 172.21.15.172:0/671087647 shutdown_connections 2024-04-14T08:36:47.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.723+0000 7ff7363fe640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff71c06c8f0 0x7ff71c06edb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.723+0000 7ff7363fe640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7300d1a20 0x7ff7301f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.723+0000 7ff7363fe640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff730071250 0x7ff7300b5740 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.723+0000 7ff7363fe640 1 --2- 172.21.15.172:0/671087647 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff73006d000 0x7ff7301f1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.723+0000 7ff7363fe640 1 -- 172.21.15.172:0/671087647 >> 172.21.15.172:0/671087647 conn(0x7ff73008c000 msgr2=0x7ff7300b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:47.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.723+0000 7ff7363fe640 1 -- 172.21.15.172:0/671087647 shutdown_connections 2024-04-14T08:36:47.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.723+0000 7ff7363fe640 1 -- 172.21.15.172:0/671087647 wait complete. 2024-04-14T08:36:47.735 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:36:47.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.834+0000 7f4b4891e640 1 Processor -- start 2024-04-14T08:36:47.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.834+0000 7f4b4891e640 1 -- start start 2024-04-14T08:36:47.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.835+0000 7f4b4891e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4b4415c710 0x7f4b440d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.835+0000 7f4b4891e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b440aef60 con 0x7f4b440cef50 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.835+0000 7f4b4891e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b440af240 con 0x7f4b440cde10 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.835+0000 7f4b4891e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f4b440aed10 con 0x7f4b4415c710 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.835+0000 7f4b42d9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4b440cef50 0x7f4b440afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47102/0 (socket says 172.21.15.172:47102) 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.835+0000 7f4b42d9d640 1 -- 172.21.15.172:0/1994299580 learned_addr learned my addr 172.21.15.172:0/1994299580 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.835+0000 7f4b4259c640 1 --2- 172.21.15.172:0/1994299580 >> v2:172.21.15.177:3300/0 conn(0x7f4b4415c710 0x7f4b440d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.836+0000 7f4b4159a640 1 -- 172.21.15.172:0/1994299580 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 772858069 0 0) 0x7f4b440aef60 con 0x7f4b440cef50 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.836+0000 7f4b4159a640 1 -- 172.21.15.172:0/1994299580 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b2c003520 con 0x7f4b440cef50 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.836+0000 7f4b4159a640 1 -- 172.21.15.172:0/1994299580 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819468497 0 0) 0x7f4b440af240 con 0x7f4b440cde10 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.836+0000 7f4b4159a640 1 -- 172.21.15.172:0/1994299580 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b440aef60 con 0x7f4b440cde10 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.836+0000 7f4b4259c640 1 -- 172.21.15.172:0/1994299580 >> v1:172.21.15.177:6789/0 conn(0x7f4b440cde10 legacy=0x7f4b440ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.836+0000 7f4b4259c640 1 -- 172.21.15.172:0/1994299580 >> v1:172.21.15.172:6789/0 conn(0x7f4b440cef50 legacy=0x7f4b440afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.836+0000 7f4b4259c640 1 -- 172.21.15.172:0/1994299580 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b440af520 con 0x7f4b4415c710 2024-04-14T08:36:47.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.836+0000 7f4b4259c640 1 --2- 172.21.15.172:0/1994299580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b4415c710 0x7f4b440d0950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009d90 tx=0x7f4b3402bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49e46e9ec0a467b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:47.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.837+0000 7f4b4159a640 1 -- 172.21.15.172:0/1994299580 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34039070 con 0x7f4b4415c710 2024-04-14T08:36:47.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.837+0000 7f4b4159a640 1 -- 172.21.15.172:0/1994299580 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3402cd00 con 0x7f4b4415c710 2024-04-14T08:36:47.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.837+0000 7f4b4159a640 1 -- 172.21.15.172:0/1994299580 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34034660 con 0x7f4b4415c710 2024-04-14T08:36:47.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.837+0000 7f4b4891e640 1 -- 172.21.15.172:0/1994299580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b4415c710 msgr2=0x7f4b440d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.837+0000 7f4b4891e640 1 --2- 172.21.15.172:0/1994299580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b4415c710 0x7f4b440d0950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009d90 tx=0x7f4b3402bf50 comp rx=0 tx=0).stop 2024-04-14T08:36:47.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.837+0000 7f4b4891e640 1 -- 172.21.15.172:0/1994299580 shutdown_connections 2024-04-14T08:36:47.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.837+0000 7f4b4891e640 1 --2- 172.21.15.172:0/1994299580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b4415c710 0x7f4b440d0950 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.837+0000 7f4b4891e640 1 -- 172.21.15.172:0/1994299580 >> 172.21.15.172:0/1994299580 conn(0x7f4b4408c000 msgr2=0x7f4b440d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:47.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.837+0000 7f4b4891e640 1 -- 172.21.15.172:0/1994299580 shutdown_connections 2024-04-14T08:36:47.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.837+0000 7f4b4891e640 1 -- 172.21.15.172:0/1994299580 wait complete. 2024-04-14T08:36:47.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b4891e640 1 Processor -- start 2024-04-14T08:36:47.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b4891e640 1 -- start start 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b4891e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b440cde10 0x7f4b44158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b4891e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b440cef50 0x7f4b440c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b4891e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b4415c710 0x7f4b440c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b4891e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4b440d0e90 con 0x7f4b4415c710 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b4891e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4b440af890 con 0x7f4b440cde10 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b4891e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4b440d1010 con 0x7f4b440cef50 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b42d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b4415c710 0x7f4b440c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b4259c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b440cde10 0x7f4b44158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b4259c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b440cde10 0x7f4b44158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:47470/0 (socket says 172.21.15.172:47470) 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b4259c640 1 -- 172.21.15.172:0/2551444222 learned_addr learned my addr 172.21.15.172:0/2551444222 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b42d9d640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b4415c710 0x7f4b440c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43600/0 (socket says 172.21.15.172:43600) 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.839+0000 7f4b41d9b640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b440cef50 0x7f4b440c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.840+0000 7f4b4259c640 1 -- 172.21.15.172:0/2551444222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b440cef50 msgr2=0x7f4b440c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.840+0000 7f4b4259c640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b440cef50 0x7f4b440c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.840+0000 7f4b4259c640 1 -- 172.21.15.172:0/2551444222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b4415c710 msgr2=0x7f4b440c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.840+0000 7f4b4259c640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b4415c710 0x7f4b440c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.840+0000 7f4b4259c640 1 -- 172.21.15.172:0/2551444222 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b44159be0 con 0x7f4b440cde10 2024-04-14T08:36:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.840+0000 7f4b4259c640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b440cde10 0x7f4b44158750 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009d90 tx=0x7f4b34004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.840+0000 7f4b32ffd640 1 -- 172.21.15.172:0/2551444222 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34039050 con 0x7f4b440cde10 2024-04-14T08:36:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.840+0000 7f4b32ffd640 1 -- 172.21.15.172:0/2551444222 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b34045070 con 0x7f4b440cde10 2024-04-14T08:36:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.840+0000 7f4b4891e640 1 -- 172.21.15.172:0/2551444222 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b44159ec0 con 0x7f4b440cde10 2024-04-14T08:36:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.840+0000 7f4b32ffd640 1 -- 172.21.15.172:0/2551444222 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34009450 con 0x7f4b440cde10 2024-04-14T08:36:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.840+0000 7f4b4891e640 1 -- 172.21.15.172:0/2551444222 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4415a480 con 0x7f4b440cde10 2024-04-14T08:36:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.841+0000 7f4b4891e640 1 -- 172.21.15.172:0/2551444222 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b44075490 con 0x7f4b440cde10 2024-04-14T08:36:47.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.841+0000 7f4b32ffd640 1 -- 172.21.15.172:0/2551444222 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b3403e830 con 0x7f4b440cde10 2024-04-14T08:36:47.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.842+0000 7f4b32ffd640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4b2406c8f0 0x7f4b2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:47.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.842+0000 7f4b32ffd640 1 -- 172.21.15.172:0/2551444222 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f4b340ae830 con 0x7f4b440cde10 2024-04-14T08:36:47.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.842+0000 7f4b41d9b640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4b2406c8f0 0x7f4b2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:47.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.842+0000 7f4b41d9b640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4b2406c8f0 0x7f4b2406edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4b38007fd0 tx=0x7f4b38009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:47.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:47.847+0000 7f4b32ffd640 1 -- 172.21.15.172:0/2551444222 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4b3407c510 con 0x7f4b440cde10 2024-04-14T08:36:48.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.059+0000 7f4b4891e640 1 -- 172.21.15.172:0/2551444222 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b440741a0 con 0x7f4b440cde10 2024-04-14T08:36:48.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.059+0000 7f4b32ffd640 1 -- 172.21.15.172:0/2551444222 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5195 (secure 0 0 0) 0x7f4b34080810 con 0x7f4b440cde10 2024-04-14T08:36:48.062 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:48.062 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3621780015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3621780015},{"type":"v1","addr":"172.21.15.172:6836","nonce":3621780015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3894683748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3894683748},{"type":"v1","addr":"172.21.15.172:6837","nonce":3894683748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3625745587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3625745587},{"type":"v1","addr":"172.21.15.177:6834","nonce":3625745587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:35:34.316542+0000","modified":"2024-04-14T08:36:14.993490+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6835/4176013980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":4176013980},{"type":"v1","addr":"172.21.15.177:6835","nonce":4176013980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-14T08:36:48.063 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 32 2024-04-14T08:36:48.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.062+0000 7f4b4891e640 1 -- 172.21.15.172:0/2551444222 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4b2406c8f0 msgr2=0x7f4b2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.062+0000 7f4b4891e640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4b2406c8f0 0x7f4b2406edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4b38007fd0 tx=0x7f4b38009040 comp rx=0 tx=0).stop 2024-04-14T08:36:48.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.062+0000 7f4b4891e640 1 -- 172.21.15.172:0/2551444222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b440cde10 msgr2=0x7f4b44158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.062+0000 7f4b4891e640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b440cde10 0x7f4b44158750 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009d90 tx=0x7f4b34004a40 comp rx=0 tx=0).stop 2024-04-14T08:36:48.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.062+0000 7f4b4891e640 1 -- 172.21.15.172:0/2551444222 shutdown_connections 2024-04-14T08:36:48.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.062+0000 7f4b4891e640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4b2406c8f0 0x7f4b2406edb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.062+0000 7f4b4891e640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b4415c710 0x7f4b440c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.062+0000 7f4b4891e640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b440cef50 0x7f4b440c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.062+0000 7f4b4891e640 1 --2- 172.21.15.172:0/2551444222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b440cde10 0x7f4b44158750 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.062+0000 7f4b4891e640 1 -- 172.21.15.172:0/2551444222 >> 172.21.15.172:0/2551444222 conn(0x7f4b4408c000 msgr2=0x7f4b440708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:48.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.062+0000 7f4b4891e640 1 -- 172.21.15.172:0/2551444222 shutdown_connections 2024-04-14T08:36:48.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.062+0000 7f4b4891e640 1 -- 172.21.15.172:0/2551444222 wait complete. 2024-04-14T08:36:48.074 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:36:48.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.172+0000 7f2b813bb640 1 Processor -- start 2024-04-14T08:36:48.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.172+0000 7f2b813bb640 1 -- start start 2024-04-14T08:36:48.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.173+0000 7f2b813bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2b7c0cef50 0x7f2b7c1672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.173+0000 7f2b813bb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b7c0aef60 con 0x7f2b7c0cde10 2024-04-14T08:36:48.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.173+0000 7f2b813bb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b7c0af240 con 0x7f2b7c15c710 2024-04-14T08:36:48.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.173+0000 7f2b813bb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2b7c0aed10 con 0x7f2b7c0cef50 2024-04-14T08:36:48.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.173+0000 7f2b7b7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2b7c0cef50 0x7f2b7c1672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.173+0000 7f2b7a7fc640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2b7c0cde10 0x7f2b7c0ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47106/0 (socket says 172.21.15.172:47106) 2024-04-14T08:36:48.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.173+0000 7f2b7a7fc640 1 -- 172.21.15.172:0/1462918922 learned_addr learned my addr 172.21.15.172:0/1462918922 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:48.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.174+0000 7f2b7b7fe640 1 -- 172.21.15.172:0/1462918922 >> v1:172.21.15.177:6789/0 conn(0x7f2b7c15c710 legacy=0x7f2b7c0d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.174+0000 7f2b7b7fe640 1 -- 172.21.15.172:0/1462918922 >> v1:172.21.15.172:6789/0 conn(0x7f2b7c0cde10 legacy=0x7f2b7c0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.174+0000 7f2b79ffb640 1 -- 172.21.15.172:0/1462918922 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2549851163 0 0) 0x7f2b7c0af240 con 0x7f2b7c15c710 2024-04-14T08:36:48.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.174+0000 7f2b7b7fe640 1 -- 172.21.15.172:0/1462918922 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c0af520 con 0x7f2b7c0cef50 2024-04-14T08:36:48.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.174+0000 7f2b7b7fe640 1 --2- 172.21.15.172:0/1462918922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b7c0cef50 0x7f2b7c1672c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097a0 tx=0x7f2b6c02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9b8c1827aa7c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:48.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.175+0000 7f2b79ffb640 1 -- 172.21.15.172:0/1462918922 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c037070 con 0x7f2b7c0cef50 2024-04-14T08:36:48.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.175+0000 7f2b79ffb640 1 -- 172.21.15.172:0/1462918922 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c030710 con 0x7f2b7c0cef50 2024-04-14T08:36:48.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.175+0000 7f2b79ffb640 1 -- 172.21.15.172:0/1462918922 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c030ae0 con 0x7f2b7c0cef50 2024-04-14T08:36:48.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.175+0000 7f2b813bb640 1 -- 172.21.15.172:0/1462918922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b7c0cef50 msgr2=0x7f2b7c1672c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.175+0000 7f2b813bb640 1 --2- 172.21.15.172:0/1462918922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b7c0cef50 0x7f2b7c1672c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097a0 tx=0x7f2b6c02b790 comp rx=0 tx=0).stop 2024-04-14T08:36:48.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.175+0000 7f2b813bb640 1 -- 172.21.15.172:0/1462918922 shutdown_connections 2024-04-14T08:36:48.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.175+0000 7f2b813bb640 1 --2- 172.21.15.172:0/1462918922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b7c0cef50 0x7f2b7c1672c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.175+0000 7f2b813bb640 1 -- 172.21.15.172:0/1462918922 >> 172.21.15.172:0/1462918922 conn(0x7f2b7c08c000 msgr2=0x7f2b7c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:48.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.175+0000 7f2b813bb640 1 -- 172.21.15.172:0/1462918922 shutdown_connections 2024-04-14T08:36:48.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.175+0000 7f2b813bb640 1 -- 172.21.15.172:0/1462918922 wait complete. 2024-04-14T08:36:48.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.176+0000 7f2b813bb640 1 Processor -- start 2024-04-14T08:36:48.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.177+0000 7f2b813bb640 1 -- start start 2024-04-14T08:36:48.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.177+0000 7f2b813bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b7c0cde10 0x7f2b7c14a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.177+0000 7f2b813bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b7c0cef50 0x7f2b7c149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.177+0000 7f2b813bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b7c15c710 0x7f2b7c0b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.177+0000 7f2b813bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c0b0d30 con 0x7f2b7c0cde10 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.177+0000 7f2b813bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c0af890 con 0x7f2b7c0cef50 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.177+0000 7f2b813bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7c0b0eb0 con 0x7f2b7c15c710 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.177+0000 7f2b7affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b7c0cde10 0x7f2b7c14a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.177+0000 7f2b7affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b7c0cde10 0x7f2b7c14a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43622/0 (socket says 172.21.15.172:43622) 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.177+0000 7f2b7affd640 1 -- 172.21.15.172:0/3393349501 learned_addr learned my addr 172.21.15.172:0/3393349501 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.177+0000 7f2b7a7fc640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b7c0cef50 0x7f2b7c149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.177+0000 7f2b7b7fe640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b7c15c710 0x7f2b7c0b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.178+0000 7f2b7affd640 1 -- 172.21.15.172:0/3393349501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b7c15c710 msgr2=0x7f2b7c0b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.178+0000 7f2b7affd640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b7c15c710 0x7f2b7c0b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.178+0000 7f2b7affd640 1 -- 172.21.15.172:0/3393349501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b7c0cef50 msgr2=0x7f2b7c149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.178+0000 7f2b7affd640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b7c0cef50 0x7f2b7c149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.178+0000 7f2b7affd640 1 -- 172.21.15.172:0/3393349501 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c0cbf50 con 0x7f2b7c0cde10 2024-04-14T08:36:48.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.178+0000 7f2b7affd640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b7c0cde10 0x7f2b7c14a630 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400c7f0 tx=0x7f2b7400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:48.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.178+0000 7f2b737fe640 1 -- 172.21.15.172:0/3393349501 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74017070 con 0x7f2b7c0cde10 2024-04-14T08:36:48.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.178+0000 7f2b737fe640 1 -- 172.21.15.172:0/3393349501 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b74003da0 con 0x7f2b7c0cde10 2024-04-14T08:36:48.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.178+0000 7f2b813bb640 1 -- 172.21.15.172:0/3393349501 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c15e6f0 con 0x7f2b7c0cde10 2024-04-14T08:36:48.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.178+0000 7f2b737fe640 1 -- 172.21.15.172:0/3393349501 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74005290 con 0x7f2b7c0cde10 2024-04-14T08:36:48.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.178+0000 7f2b813bb640 1 -- 172.21.15.172:0/3393349501 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c0cb990 con 0x7f2b7c0cde10 2024-04-14T08:36:48.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.179+0000 7f2b813bb640 1 -- 172.21.15.172:0/3393349501 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7c075490 con 0x7f2b7c0cde10 2024-04-14T08:36:48.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.179+0000 7f2b737fe640 1 -- 172.21.15.172:0/3393349501 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b7401a070 con 0x7f2b7c0cde10 2024-04-14T08:36:48.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.180+0000 7f2b737fe640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2b5c06c920 0x7f2b5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.180+0000 7f2b737fe640 1 -- 172.21.15.172:0/3393349501 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f2b7408b2c0 con 0x7f2b7c0cde10 2024-04-14T08:36:48.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.180+0000 7f2b7a7fc640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2b5c06c920 0x7f2b5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.180+0000 7f2b7a7fc640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2b5c06c920 0x7f2b5c06ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c06f640 tx=0x7f2b64007e20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:48.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.184+0000 7f2b737fe640 1 -- 172.21.15.172:0/3393349501 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2b74058f10 con 0x7f2b7c0cde10 2024-04-14T08:36:48.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.360+0000 7f2b813bb640 1 -- 172.21.15.172:0/3393349501 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b7c15bd10 con 0x7f2b7c0cde10 2024-04-14T08:36:48.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.361+0000 7f2b737fe640 1 -- 172.21.15.172:0/3393349501 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17184 (secure 0 0 0) 0x7f2b7405d210 con 0x7f2b7c0cde10 2024-04-14T08:36:48.364 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:48.365 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":63,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:36:35.260134+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:35:32.539067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-14T08:35:33.263123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.17","mappings":[{"from":4,"to":5}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:36:48.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.363+0000 7f2b813bb640 1 -- 172.21.15.172:0/3393349501 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2b5c06c920 msgr2=0x7f2b5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.363+0000 7f2b813bb640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2b5c06c920 0x7f2b5c06ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c06f640 tx=0x7f2b64007e20 comp rx=0 tx=0).stop 2024-04-14T08:36:48.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.363+0000 7f2b813bb640 1 -- 172.21.15.172:0/3393349501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b7c0cde10 msgr2=0x7f2b7c14a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.363+0000 7f2b813bb640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b7c0cde10 0x7f2b7c14a630 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400c7f0 tx=0x7f2b7400ccb0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.363+0000 7f2b813bb640 1 -- 172.21.15.172:0/3393349501 shutdown_connections 2024-04-14T08:36:48.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.363+0000 7f2b813bb640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2b5c06c920 0x7f2b5c06ede0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.363+0000 7f2b813bb640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b7c15c710 0x7f2b7c0b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.363+0000 7f2b813bb640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b7c0cef50 0x7f2b7c149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.363+0000 7f2b813bb640 1 --2- 172.21.15.172:0/3393349501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b7c0cde10 0x7f2b7c14a630 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.363+0000 7f2b813bb640 1 -- 172.21.15.172:0/3393349501 >> 172.21.15.172:0/3393349501 conn(0x7f2b7c08c000 msgr2=0x7f2b7c0ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:48.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.364+0000 7f2b813bb640 1 -- 172.21.15.172:0/3393349501 shutdown_connections 2024-04-14T08:36:48.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.364+0000 7f2b813bb640 1 -- 172.21.15.172:0/3393349501 wait complete. 2024-04-14T08:36:48.377 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-14T08:36:48.377 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:36:48.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.483+0000 7efcd1020640 1 Processor -- start 2024-04-14T08:36:48.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.483+0000 7efcd1020640 1 -- start start 2024-04-14T08:36:48.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efcd1020640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efccc15c340 0x7efccc0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efcd1020640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efccc0ce5b0 0x7efccc0d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efcd1020640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efccc0ad100 con 0x7efccc0ca920 2024-04-14T08:36:48.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efcd1020640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7efccc0cfb90 con 0x7efccc0ce5b0 2024-04-14T08:36:48.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efcd1020640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efccc0cd570 con 0x7efccc15c340 2024-04-14T08:36:48.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efccad9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efccc15c340 0x7efccc0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efccb59e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efccc0ce5b0 0x7efccc0d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efccad9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efccc15c340 0x7efccc0ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:39532/0 (socket says 172.21.15.172:39532) 2024-04-14T08:36:48.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efccad9d640 1 -- 172.21.15.172:0/3439248909 learned_addr learned my addr 172.21.15.172:0/3439248909 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:48.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efcc3fff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7efccc0ca920 0x7efccc0cdea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:49586/0 (socket says 172.21.15.172:49586) 2024-04-14T08:36:48.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efccb59e640 1 -- 172.21.15.172:0/3439248909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efccc15c340 msgr2=0x7efccc0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:48.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efccb59e640 1 --2- 172.21.15.172:0/3439248909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efccc15c340 0x7efccc0ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efccb59e640 1 -- 172.21.15.172:0/3439248909 >> v1:172.21.15.177:6789/0 conn(0x7efccc0ca920 legacy=0x7efccc0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efccb59e640 1 -- 172.21.15.172:0/3439248909 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efccc0cd850 con 0x7efccc0ce5b0 2024-04-14T08:36:48.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.484+0000 7efcca59c640 1 -- 172.21.15.172:0/3439248909 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3567158021 0 0) 0x7efccc0ad100 con 0x7efccc0ca920 2024-04-14T08:36:48.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.485+0000 7efccb59e640 1 --2- 172.21.15.172:0/3439248909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efccc0ce5b0 0x7efccc0d0020 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009d90 tx=0x7efcc402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=578d306d8e99688e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:48.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.485+0000 7efcca59c640 1 -- 172.21.15.172:0/3439248909 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4039070 con 0x7efccc0ce5b0 2024-04-14T08:36:48.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.485+0000 7efcca59c640 1 -- 172.21.15.172:0/3439248909 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc402cc50 con 0x7efccc0ce5b0 2024-04-14T08:36:48.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.485+0000 7efcca59c640 1 -- 172.21.15.172:0/3439248909 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4034440 con 0x7efccc0ce5b0 2024-04-14T08:36:48.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.486+0000 7efcd1020640 1 -- 172.21.15.172:0/3439248909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efccc0ce5b0 msgr2=0x7efccc0d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.486+0000 7efcd1020640 1 --2- 172.21.15.172:0/3439248909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efccc0ce5b0 0x7efccc0d0020 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009d90 tx=0x7efcc402bea0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.486+0000 7efcd1020640 1 -- 172.21.15.172:0/3439248909 shutdown_connections 2024-04-14T08:36:48.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.486+0000 7efcd1020640 1 --2- 172.21.15.172:0/3439248909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efccc0ce5b0 0x7efccc0d0020 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.486+0000 7efcd1020640 1 --2- 172.21.15.172:0/3439248909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efccc15c340 0x7efccc0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.486+0000 7efcd1020640 1 -- 172.21.15.172:0/3439248909 >> 172.21.15.172:0/3439248909 conn(0x7efccc072b20 msgr2=0x7efccc072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:48.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.486+0000 7efcd1020640 1 -- 172.21.15.172:0/3439248909 shutdown_connections 2024-04-14T08:36:48.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.486+0000 7efcd1020640 1 -- 172.21.15.172:0/3439248909 wait complete. 2024-04-14T08:36:48.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.487+0000 7efcd1020640 1 Processor -- start 2024-04-14T08:36:48.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.487+0000 7efcd1020640 1 -- start start 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efcd1020640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efccc0ca920 0x7efccc157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efcd1020640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efccc0ce5b0 0x7efccc1e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efcd1020640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efccc15c340 0x7efccc1e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efcd1020640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efccc0d0560 con 0x7efccc0ce5b0 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efcd1020640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efccc0cd260 con 0x7efccc0ca920 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efcd1020640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efccc0d06e0 con 0x7efccc15c340 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efcc3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efccc0ce5b0 0x7efccc1e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efcc3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efccc0ce5b0 0x7efccc1e7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43638/0 (socket says 172.21.15.172:43638) 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efcc3fff640 1 -- 172.21.15.172:0/3054677949 learned_addr learned my addr 172.21.15.172:0/3054677949 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efccb59e640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efccc15c340 0x7efccc1e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efccad9d640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efccc0ca920 0x7efccc157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efccb59e640 1 -- 172.21.15.172:0/3054677949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efccc0ca920 msgr2=0x7efccc157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efccb59e640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efccc0ca920 0x7efccc157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efccb59e640 1 -- 172.21.15.172:0/3054677949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efccc0ce5b0 msgr2=0x7efccc1e7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efccb59e640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efccc0ce5b0 0x7efccc1e7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efccb59e640 1 -- 172.21.15.172:0/3054677949 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efccc154160 con 0x7efccc15c340 2024-04-14T08:36:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efcc3fff640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efccc0ce5b0 0x7efccc1e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efccad9d640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efccc0ca920 0x7efccc157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efccb59e640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efccc15c340 0x7efccc1e82d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efcc4033000 tx=0x7efcc4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.488+0000 7efcc37fe640 1 -- 172.21.15.172:0/3054677949 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4044020 con 0x7efccc15c340 2024-04-14T08:36:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.489+0000 7efcc37fe640 1 -- 172.21.15.172:0/3054677949 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc403a040 con 0x7efccc15c340 2024-04-14T08:36:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.489+0000 7efcd1020640 1 -- 172.21.15.172:0/3054677949 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efccc1587a0 con 0x7efccc15c340 2024-04-14T08:36:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.489+0000 7efcc37fe640 1 -- 172.21.15.172:0/3054677949 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4039050 con 0x7efccc15c340 2024-04-14T08:36:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.489+0000 7efcd1020640 1 -- 172.21.15.172:0/3054677949 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efccc155140 con 0x7efccc15c340 2024-04-14T08:36:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.490+0000 7efcd1020640 1 -- 172.21.15.172:0/3054677949 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efccc068520 con 0x7efccc15c340 2024-04-14T08:36:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.490+0000 7efcc37fe640 1 -- 172.21.15.172:0/3054677949 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcc4003990 con 0x7efccc15c340 2024-04-14T08:36:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.490+0000 7efcc37fe640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efcac06c8f0 0x7efcac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.490+0000 7efcc37fe640 1 -- 172.21.15.172:0/3054677949 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7efcc40adff0 con 0x7efccc15c340 2024-04-14T08:36:48.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.490+0000 7efccad9d640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efcac06c8f0 0x7efcac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.491+0000 7efccad9d640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efcac06c8f0 0x7efcac06edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009f20 tx=0x7efcbc009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:48.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.495+0000 7efcc37fe640 1 -- 172.21.15.172:0/3054677949 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efcc404e020 con 0x7efccc15c340 2024-04-14T08:36:48.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.707+0000 7efcd1020640 1 -- 172.21.15.172:0/3054677949 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efccc0b6250 con 0x7efccc15c340 2024-04-14T08:36:48.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.708+0000 7efcc37fe640 1 -- 172.21.15.172:0/3054677949 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5195 (secure 0 0 0) 0x7efcc407bc40 con 0x7efccc15c340 2024-04-14T08:36:48.710 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:48.710 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3621780015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3621780015},{"type":"v1","addr":"172.21.15.172:6836","nonce":3621780015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3894683748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3894683748},{"type":"v1","addr":"172.21.15.172:6837","nonce":3894683748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3625745587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3625745587},{"type":"v1","addr":"172.21.15.177:6834","nonce":3625745587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:35:34.316542+0000","modified":"2024-04-14T08:36:14.993490+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6835/4176013980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":4176013980},{"type":"v1","addr":"172.21.15.177:6835","nonce":4176013980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-14T08:36:48.710 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 32 2024-04-14T08:36:48.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.710+0000 7efcd1020640 1 -- 172.21.15.172:0/3054677949 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efcac06c8f0 msgr2=0x7efcac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.710+0000 7efcd1020640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efcac06c8f0 0x7efcac06edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009f20 tx=0x7efcbc009240 comp rx=0 tx=0).stop 2024-04-14T08:36:48.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.710+0000 7efcd1020640 1 -- 172.21.15.172:0/3054677949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efccc15c340 msgr2=0x7efccc1e82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.710+0000 7efcd1020640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efccc15c340 0x7efccc1e82d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efcc4033000 tx=0x7efcc4004a40 comp rx=0 tx=0).stop 2024-04-14T08:36:48.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.710+0000 7efcd1020640 1 -- 172.21.15.172:0/3054677949 shutdown_connections 2024-04-14T08:36:48.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.710+0000 7efcd1020640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efcac06c8f0 0x7efcac06edb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.710+0000 7efcd1020640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efccc15c340 0x7efccc1e82d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.710+0000 7efcd1020640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efccc0ce5b0 0x7efccc1e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.710+0000 7efcd1020640 1 --2- 172.21.15.172:0/3054677949 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efccc0ca920 0x7efccc157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.710+0000 7efcd1020640 1 -- 172.21.15.172:0/3054677949 >> 172.21.15.172:0/3054677949 conn(0x7efccc072b20 msgr2=0x7efccc0c5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:48.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.710+0000 7efcd1020640 1 -- 172.21.15.172:0/3054677949 shutdown_connections 2024-04-14T08:36:48.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.710+0000 7efcd1020640 1 -- 172.21.15.172:0/3054677949 wait complete. 2024-04-14T08:36:48.722 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:36:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.822+0000 7fcbbfd1e640 1 Processor -- start 2024-04-14T08:36:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.822+0000 7fcbbfd1e640 1 -- start start 2024-04-14T08:36:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.823+0000 7fcbbfd1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbb809ace0 0x7fcbb806f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.823+0000 7fcbbfd1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcbb80b16a0 0x7fcbb806f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.823+0000 7fcbbfd1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbb80d0ad0 con 0x7fcbb809b0c0 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.823+0000 7fcbbfd1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fcbb8070c80 con 0x7fcbb80b16a0 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.823+0000 7fcbbfd1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb80d0d70 con 0x7fcbb809ace0 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.823+0000 7fcbbe2bb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fcbb809b0c0 0x7fcbb81605d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47110/0 (socket says 172.21.15.172:47110) 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.823+0000 7fcbbe2bb640 1 -- 172.21.15.172:0/78037746 learned_addr learned my addr 172.21.15.172:0/78037746 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.823+0000 7fcbbd2b9640 1 --2- 172.21.15.172:0/78037746 >> v2:172.21.15.177:3300/0 conn(0x7fcbb80b16a0 0x7fcbb806f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.823+0000 7fcbbdaba640 1 --2- 172.21.15.172:0/78037746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbb809ace0 0x7fcbb806f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.824+0000 7fcbbcab8640 1 -- 172.21.15.172:0/78037746 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 915852265 0 0) 0x7fcbb80d0ad0 con 0x7fcbb809b0c0 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.824+0000 7fcbbcab8640 1 -- 172.21.15.172:0/78037746 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcba4003520 con 0x7fcbb809b0c0 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.824+0000 7fcbbdaba640 1 -- 172.21.15.172:0/78037746 >> v2:172.21.15.177:3300/0 conn(0x7fcbb80b16a0 msgr2=0x7fcbb806f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.824+0000 7fcbbdaba640 1 --2- 172.21.15.172:0/78037746 >> v2:172.21.15.177:3300/0 conn(0x7fcbb80b16a0 0x7fcbb806f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.824+0000 7fcbbdaba640 1 -- 172.21.15.172:0/78037746 >> v1:172.21.15.172:6789/0 conn(0x7fcbb809b0c0 legacy=0x7fcbb81605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.824+0000 7fcbbdaba640 1 -- 172.21.15.172:0/78037746 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb80ccde0 con 0x7fcbb809ace0 2024-04-14T08:36:48.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.824+0000 7fcbbd2b9640 1 --2- 172.21.15.172:0/78037746 >> v2:172.21.15.177:3300/0 conn(0x7fcbb80b16a0 0x7fcbb806f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.824+0000 7fcbbdaba640 1 --2- 172.21.15.172:0/78037746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbb809ace0 0x7fcbb806f390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009d90 tx=0x7fcbb402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5627762719144ee server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.825+0000 7fcbbcab8640 1 -- 172.21.15.172:0/78037746 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4039070 con 0x7fcbb809ace0 2024-04-14T08:36:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.825+0000 7fcbbcab8640 1 -- 172.21.15.172:0/78037746 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb402cbc0 con 0x7fcbb809ace0 2024-04-14T08:36:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.825+0000 7fcbbcab8640 1 -- 172.21.15.172:0/78037746 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4034440 con 0x7fcbb809ace0 2024-04-14T08:36:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.825+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/78037746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbb809ace0 msgr2=0x7fcbb806f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.825+0000 7fcbbfd1e640 1 --2- 172.21.15.172:0/78037746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbb809ace0 0x7fcbb806f390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009d90 tx=0x7fcbb402be10 comp rx=0 tx=0).stop 2024-04-14T08:36:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.825+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/78037746 shutdown_connections 2024-04-14T08:36:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.825+0000 7fcbbfd1e640 1 --2- 172.21.15.172:0/78037746 >> v2:172.21.15.177:3300/0 conn(0x7fcbb80b16a0 0x7fcbb806f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.825+0000 7fcbbfd1e640 1 --2- 172.21.15.172:0/78037746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbb809ace0 0x7fcbb806f390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.825+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/78037746 >> 172.21.15.172:0/78037746 conn(0x7fcbb80d2080 msgr2=0x7fcbb80d2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.825+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/78037746 shutdown_connections 2024-04-14T08:36:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.825+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/78037746 wait complete. 2024-04-14T08:36:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.826+0000 7fcbbfd1e640 1 Processor -- start 2024-04-14T08:36:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.826+0000 7fcbbfd1e640 1 -- start start 2024-04-14T08:36:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.827+0000 7fcbbfd1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbb809ace0 0x7fcbb8156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.827+0000 7fcbbfd1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbb809b0c0 0x7fcbb8158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.827+0000 7fcbbfd1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbb80b16a0 0x7fcbb8158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.827+0000 7fcbbfd1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb80711d0 con 0x7fcbb809b0c0 2024-04-14T08:36:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.827+0000 7fcbbfd1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb80cca60 con 0x7fcbb809ace0 2024-04-14T08:36:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.827+0000 7fcbbfd1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb8071350 con 0x7fcbb80b16a0 2024-04-14T08:36:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.827+0000 7fcbbd2b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbb809b0c0 0x7fcbb8158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.827+0000 7fcbbd2b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbb809b0c0 0x7fcbb8158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43640/0 (socket says 172.21.15.172:43640) 2024-04-14T08:36:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.827+0000 7fcbbdaba640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbb809ace0 0x7fcbb8156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.827+0000 7fcbbd2b9640 1 -- 172.21.15.172:0/3599337722 learned_addr learned my addr 172.21.15.172:0/3599337722 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.827+0000 7fcbbe2bb640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbb80b16a0 0x7fcbb8158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.828+0000 7fcbbd2b9640 1 -- 172.21.15.172:0/3599337722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbb80b16a0 msgr2=0x7fcbb8158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.828+0000 7fcbbd2b9640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbb80b16a0 0x7fcbb8158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.828+0000 7fcbbd2b9640 1 -- 172.21.15.172:0/3599337722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbb809ace0 msgr2=0x7fcbb8156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.828+0000 7fcbbd2b9640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbb809ace0 0x7fcbb8156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.828+0000 7fcbbd2b9640 1 -- 172.21.15.172:0/3599337722 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb8157040 con 0x7fcbb809b0c0 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.828+0000 7fcbbdaba640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbb809ace0 0x7fcbb8156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.828+0000 7fcbbd2b9640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbb809b0c0 0x7fcbb8158700 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcba800e960 tx=0x7fcba800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.828+0000 7fcbae7fc640 1 -- 172.21.15.172:0/3599337722 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8017070 con 0x7fcbb809b0c0 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.828+0000 7fcbae7fc640 1 -- 172.21.15.172:0/3599337722 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba800c9a0 con 0x7fcbb809b0c0 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.828+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/3599337722 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb80b5c40 con 0x7fcbb809b0c0 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.828+0000 7fcbae7fc640 1 -- 172.21.15.172:0/3599337722 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba800cd70 con 0x7fcbb809b0c0 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.828+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/3599337722 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb80b6200 con 0x7fcbb809b0c0 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.829+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/3599337722 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb8067dd0 con 0x7fcbb809b0c0 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.829+0000 7fcbae7fc640 1 -- 172.21.15.172:0/3599337722 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcba801a070 con 0x7fcbb809b0c0 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.830+0000 7fcbae7fc640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb9c06c8f0 0x7fcb9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.830+0000 7fcbae7fc640 1 -- 172.21.15.172:0/3599337722 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fcba808be50 con 0x7fcbb809b0c0 2024-04-14T08:36:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.830+0000 7fcbbdaba640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb9c06c8f0 0x7fcb9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:48.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.830+0000 7fcbbdaba640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb9c06c8f0 0x7fcb9c06edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4036000 tx=0x7fcbb4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:48.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:48.835+0000 7fcbae7fc640 1 -- 172.21.15.172:0/3599337722 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcbb8067dd0 con 0x7fcbb809b0c0 2024-04-14T08:36:49.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.048+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/3599337722 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbb8068520 con 0x7fcbb809b0c0 2024-04-14T08:36:49.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.048+0000 7fcbae7fc640 1 -- 172.21.15.172:0/3599337722 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5195 (secure 0 0 0) 0x7fcba8059aa0 con 0x7fcbb809b0c0 2024-04-14T08:36:49.050 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:49.050 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3621780015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3621780015},{"type":"v1","addr":"172.21.15.172:6836","nonce":3621780015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3894683748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3894683748},{"type":"v1","addr":"172.21.15.172:6837","nonce":3894683748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3625745587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3625745587},{"type":"v1","addr":"172.21.15.177:6834","nonce":3625745587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:35:34.316542+0000","modified":"2024-04-14T08:36:14.993490+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6204":{"gid":6204,"name":"d","rank":0,"incarnation":28,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6835/4176013980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":4176013980},{"type":"v1","addr":"172.21.15.177:6835","nonce":4176013980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6204,"qdb_cluster":[6204]},"id":4}]} 2024-04-14T08:36:49.050 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 32 2024-04-14T08:36:49.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.050+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/3599337722 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb9c06c8f0 msgr2=0x7fcb9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.050+0000 7fcbbfd1e640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb9c06c8f0 0x7fcb9c06edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4036000 tx=0x7fcbb4030000 comp rx=0 tx=0).stop 2024-04-14T08:36:49.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.050+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/3599337722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbb809b0c0 msgr2=0x7fcbb8158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.050+0000 7fcbbfd1e640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbb809b0c0 0x7fcbb8158700 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcba800e960 tx=0x7fcba800ee20 comp rx=0 tx=0).stop 2024-04-14T08:36:49.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.050+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/3599337722 shutdown_connections 2024-04-14T08:36:49.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.050+0000 7fcbbfd1e640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb9c06c8f0 0x7fcb9c06edb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.050+0000 7fcbbfd1e640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbb80b16a0 0x7fcbb8158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.050+0000 7fcbbfd1e640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbb809b0c0 0x7fcbb8158700 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.050+0000 7fcbbfd1e640 1 --2- 172.21.15.172:0/3599337722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbb809ace0 0x7fcbb8156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.050+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/3599337722 >> 172.21.15.172:0/3599337722 conn(0x7fcbb80d2080 msgr2=0x7fcbb80cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:49.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.050+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/3599337722 shutdown_connections 2024-04-14T08:36:49.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.050+0000 7fcbbfd1e640 1 -- 172.21.15.172:0/3599337722 wait complete. 2024-04-14T08:36:49.063 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:36:49.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.162+0000 7fbad16de640 1 Processor -- start 2024-04-14T08:36:49.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.162+0000 7fbad16de640 1 -- start start 2024-04-14T08:36:49.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.163+0000 7fbad16de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbacc15c710 0x7fbacc0d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.163+0000 7fbad16de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbacc0cde10 0x7fbacc0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.163+0000 7fbad16de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbacc0aef60 con 0x7fbacc0ced80 2024-04-14T08:36:49.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.163+0000 7fbad16de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fbacc0cf1f0 con 0x7fbacc0cde10 2024-04-14T08:36:49.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.163+0000 7fbad16de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fbacc0aed10 con 0x7fbacc15c710 2024-04-14T08:36:49.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.163+0000 7fbaca7fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbacc0cde10 0x7fbacc0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.163+0000 7fbaca7fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbacc0cde10 0x7fbacc0ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43650/0 (socket says 172.21.15.172:43650) 2024-04-14T08:36:49.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.163+0000 7fbaca7fc640 1 -- 172.21.15.172:0/2478382682 learned_addr learned my addr 172.21.15.172:0/2478382682 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:49.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.163+0000 7fbacaffd640 1 --2- 172.21.15.172:0/2478382682 >> v2:172.21.15.177:3300/0 conn(0x7fbacc15c710 0x7fbacc0d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.164+0000 7fbac9ffb640 1 -- 172.21.15.172:0/2478382682 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3825431144 0 0) 0x7fbacc0aef60 con 0x7fbacc0ced80 2024-04-14T08:36:49.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.164+0000 7fbaca7fc640 1 -- 172.21.15.172:0/2478382682 >> v2:172.21.15.177:3300/0 conn(0x7fbacc15c710 msgr2=0x7fbacc0d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:49.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.164+0000 7fbaca7fc640 1 --2- 172.21.15.172:0/2478382682 >> v2:172.21.15.177:3300/0 conn(0x7fbacc15c710 0x7fbacc0d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.164+0000 7fbaca7fc640 1 -- 172.21.15.172:0/2478382682 >> v1:172.21.15.177:6789/0 conn(0x7fbacc0ced80 legacy=0x7fbacc0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.164+0000 7fbaca7fc640 1 -- 172.21.15.172:0/2478382682 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc0af240 con 0x7fbacc0cde10 2024-04-14T08:36:49.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.164+0000 7fbacaffd640 1 --2- 172.21.15.172:0/2478382682 >> v2:172.21.15.177:3300/0 conn(0x7fbacc15c710 0x7fbacc0d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:49.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.164+0000 7fbaca7fc640 1 --2- 172.21.15.172:0/2478382682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc0cde10 0x7fbacc0ce840 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbacc069160 tx=0x7fbab402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8071a4da4837844f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:49.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.165+0000 7fbac9ffb640 1 -- 172.21.15.172:0/2478382682 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4034070 con 0x7fbacc0cde10 2024-04-14T08:36:49.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.165+0000 7fbac9ffb640 1 -- 172.21.15.172:0/2478382682 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab4030860 con 0x7fbacc0cde10 2024-04-14T08:36:49.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.165+0000 7fbac9ffb640 1 -- 172.21.15.172:0/2478382682 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4030c30 con 0x7fbacc0cde10 2024-04-14T08:36:49.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.165+0000 7fbad16de640 1 -- 172.21.15.172:0/2478382682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc0cde10 msgr2=0x7fbacc0ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.165+0000 7fbad16de640 1 --2- 172.21.15.172:0/2478382682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc0cde10 0x7fbacc0ce840 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbacc069160 tx=0x7fbab402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.165+0000 7fbad16de640 1 -- 172.21.15.172:0/2478382682 shutdown_connections 2024-04-14T08:36:49.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.165+0000 7fbad16de640 1 --2- 172.21.15.172:0/2478382682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc0cde10 0x7fbacc0ce840 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.165+0000 7fbad16de640 1 --2- 172.21.15.172:0/2478382682 >> v2:172.21.15.177:3300/0 conn(0x7fbacc15c710 0x7fbacc0d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.165+0000 7fbad16de640 1 -- 172.21.15.172:0/2478382682 >> 172.21.15.172:0/2478382682 conn(0x7fbacc08c000 msgr2=0x7fbacc0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:49.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.165+0000 7fbad16de640 1 -- 172.21.15.172:0/2478382682 shutdown_connections 2024-04-14T08:36:49.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.165+0000 7fbad16de640 1 -- 172.21.15.172:0/2478382682 wait complete. 2024-04-14T08:36:49.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.166+0000 7fbad16de640 1 Processor -- start 2024-04-14T08:36:49.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.167+0000 7fbad16de640 1 -- start start 2024-04-14T08:36:49.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.167+0000 7fbad16de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc0cde10 0x7fbacc0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.167+0000 7fbad16de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbacc0ced80 0x7fbacc0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.167+0000 7fbad16de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbacc15c710 0x7fbacc0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.167+0000 7fbad16de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc15caf0 con 0x7fbacc0cde10 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.167+0000 7fbad16de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc0af890 con 0x7fbacc15c710 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.167+0000 7fbad16de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbacc0d0e90 con 0x7fbacc0ced80 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.167+0000 7fbacaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc0cde10 0x7fbacc0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.167+0000 7fbacaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc0cde10 0x7fbacc0c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43662/0 (socket says 172.21.15.172:43662) 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.167+0000 7fbacaffd640 1 -- 172.21.15.172:0/1887344104 learned_addr learned my addr 172.21.15.172:0/1887344104 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.167+0000 7fbacb7fe640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbacc15c710 0x7fbacc0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.167+0000 7fbaca7fc640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbacc0ced80 0x7fbacc0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.168+0000 7fbacaffd640 1 -- 172.21.15.172:0/1887344104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbacc0ced80 msgr2=0x7fbacc0cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.168+0000 7fbacaffd640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbacc0ced80 0x7fbacc0cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.168+0000 7fbacaffd640 1 -- 172.21.15.172:0/1887344104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbacc15c710 msgr2=0x7fbacc0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.168+0000 7fbacaffd640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbacc15c710 0x7fbacc0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.168+0000 7fbacaffd640 1 -- 172.21.15.172:0/1887344104 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc0cc390 con 0x7fbacc0cde10 2024-04-14T08:36:49.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.168+0000 7fbacaffd640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc0cde10 0x7fbacc0c9110 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00efc0 tx=0x7fbabc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:49.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.168+0000 7fbac37fe640 1 -- 172.21.15.172:0/1887344104 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc018020 con 0x7fbacc0cde10 2024-04-14T08:36:49.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.168+0000 7fbac37fe640 1 -- 172.21.15.172:0/1887344104 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbabc014680 con 0x7fbacc0cde10 2024-04-14T08:36:49.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.168+0000 7fbad16de640 1 -- 172.21.15.172:0/1887344104 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbacc0cbaf0 con 0x7fbacc0cde10 2024-04-14T08:36:49.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.168+0000 7fbac37fe640 1 -- 172.21.15.172:0/1887344104 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc014a50 con 0x7fbacc0cde10 2024-04-14T08:36:49.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.168+0000 7fbad16de640 1 -- 172.21.15.172:0/1887344104 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbacc0cc0b0 con 0x7fbacc0cde10 2024-04-14T08:36:49.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.169+0000 7fbad16de640 1 -- 172.21.15.172:0/1887344104 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbacc075490 con 0x7fbacc0cde10 2024-04-14T08:36:49.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.169+0000 7fbac37fe640 1 -- 172.21.15.172:0/1887344104 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbabc005d10 con 0x7fbacc0cde10 2024-04-14T08:36:49.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.170+0000 7fbac37fe640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbaac06c8f0 0x7fbaac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.170+0000 7fbac37fe640 1 -- 172.21.15.172:0/1887344104 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fbabc08c950 con 0x7fbacc0cde10 2024-04-14T08:36:49.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.170+0000 7fbaca7fc640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbaac06c8f0 0x7fbaac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.170+0000 7fbaca7fc640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbaac06c8f0 0x7fbaac06edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbab4036000 tx=0x7fbab4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:49.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.174+0000 7fbac37fe640 1 -- 172.21.15.172:0/1887344104 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbabc022030 con 0x7fbacc0cde10 2024-04-14T08:36:49.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.351+0000 7fbad16de640 1 -- 172.21.15.172:0/1887344104 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbacc0cd920 con 0x7fbacc0cde10 2024-04-14T08:36:49.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.352+0000 7fbac37fe640 1 -- 172.21.15.172:0/1887344104 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17184 (secure 0 0 0) 0x7fbabc010040 con 0x7fbacc0cde10 2024-04-14T08:36:49.354 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:49.354 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":63,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:36:35.260134+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:35:32.539067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-14T08:35:33.263123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":7,"to":2}]},{"pgid":"10.17","mappings":[{"from":4,"to":5}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]},{"pgid":"10.37","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:36:49.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.354+0000 7fbad16de640 1 -- 172.21.15.172:0/1887344104 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbaac06c8f0 msgr2=0x7fbaac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.354+0000 7fbad16de640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbaac06c8f0 0x7fbaac06edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbab4036000 tx=0x7fbab4008030 comp rx=0 tx=0).stop 2024-04-14T08:36:49.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.354+0000 7fbad16de640 1 -- 172.21.15.172:0/1887344104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc0cde10 msgr2=0x7fbacc0c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.354+0000 7fbad16de640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc0cde10 0x7fbacc0c9110 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00efc0 tx=0x7fbabc00c490 comp rx=0 tx=0).stop 2024-04-14T08:36:49.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.354+0000 7fbad16de640 1 -- 172.21.15.172:0/1887344104 shutdown_connections 2024-04-14T08:36:49.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.354+0000 7fbad16de640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbaac06c8f0 0x7fbaac06edb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.354+0000 7fbad16de640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbacc15c710 0x7fbacc0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.354+0000 7fbad16de640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbacc0ced80 0x7fbacc0cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.354+0000 7fbad16de640 1 --2- 172.21.15.172:0/1887344104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbacc0cde10 0x7fbacc0c9110 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.354+0000 7fbad16de640 1 -- 172.21.15.172:0/1887344104 >> 172.21.15.172:0/1887344104 conn(0x7fbacc08c000 msgr2=0x7fbacc15cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:49.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.354+0000 7fbad16de640 1 -- 172.21.15.172:0/1887344104 shutdown_connections 2024-04-14T08:36:49.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.354+0000 7fbad16de640 1 -- 172.21.15.172:0/1887344104 wait complete. 2024-04-14T08:36:49.367 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:36:49.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.468+0000 7f6125d1e640 1 Processor -- start 2024-04-14T08:36:49.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.468+0000 7f6125d1e640 1 -- start start 2024-04-14T08:36:49.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f6125d1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f612013fa70 0x7f61200af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f6125d1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f61200ce150 0x7f61200722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f6125d1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6120071520 con 0x7f61200ce5e0 2024-04-14T08:36:49.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f6125d1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f61200712d0 con 0x7f612013fa70 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f6125d1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f6120071800 con 0x7f61200ce150 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f611f7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f612013fa70 0x7f61200af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f611f7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f612013fa70 0x7f61200af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43676/0 (socket says 172.21.15.172:43676) 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f611f7fe640 1 -- 172.21.15.172:0/577882838 learned_addr learned my addr 172.21.15.172:0/577882838 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f611effd640 1 --2- 172.21.15.172:0/577882838 >> v2:172.21.15.177:3300/0 conn(0x7f61200ce150 0x7f61200722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f611f7fe640 1 -- 172.21.15.172:0/577882838 >> v2:172.21.15.177:3300/0 conn(0x7f61200ce150 msgr2=0x7f61200722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f611e7fc640 1 -- 172.21.15.172:0/577882838 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3573511684 0 0) 0x7f6120071520 con 0x7f61200ce5e0 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f611f7fe640 1 --2- 172.21.15.172:0/577882838 >> v2:172.21.15.177:3300/0 conn(0x7f61200ce150 0x7f61200722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f611f7fe640 1 -- 172.21.15.172:0/577882838 >> v1:172.21.15.177:6789/0 conn(0x7f61200ce5e0 legacy=0x7f612015e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f611f7fe640 1 -- 172.21.15.172:0/577882838 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6120071ae0 con 0x7f612013fa70 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.469+0000 7f611effd640 1 --2- 172.21.15.172:0/577882838 >> v2:172.21.15.177:3300/0 conn(0x7f61200ce150 0x7f61200722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.470+0000 7f611f7fe640 1 --2- 172.21.15.172:0/577882838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612013fa70 0x7f61200af740 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f61200afc80 tx=0x7f611002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9a440d616c5a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:49.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.470+0000 7f611e7fc640 1 -- 172.21.15.172:0/577882838 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110035020 con 0x7f612013fa70 2024-04-14T08:36:49.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.470+0000 7f611e7fc640 1 -- 172.21.15.172:0/577882838 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6110031870 con 0x7f612013fa70 2024-04-14T08:36:49.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.470+0000 7f611e7fc640 1 -- 172.21.15.172:0/577882838 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110031c40 con 0x7f612013fa70 2024-04-14T08:36:49.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.470+0000 7f6125d1e640 1 -- 172.21.15.172:0/577882838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612013fa70 msgr2=0x7f61200af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.470+0000 7f6125d1e640 1 --2- 172.21.15.172:0/577882838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612013fa70 0x7f61200af740 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f61200afc80 tx=0x7f611002b910 comp rx=0 tx=0).stop 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.471+0000 7f6125d1e640 1 -- 172.21.15.172:0/577882838 shutdown_connections 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.471+0000 7f6125d1e640 1 --2- 172.21.15.172:0/577882838 >> v2:172.21.15.177:3300/0 conn(0x7f61200ce150 0x7f61200722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.471+0000 7f6125d1e640 1 --2- 172.21.15.172:0/577882838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612013fa70 0x7f61200af740 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.471+0000 7f6125d1e640 1 -- 172.21.15.172:0/577882838 >> 172.21.15.172:0/577882838 conn(0x7f612006b520 msgr2=0x7f612006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.471+0000 7f6125d1e640 1 -- 172.21.15.172:0/577882838 shutdown_connections 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.471+0000 7f6125d1e640 1 -- 172.21.15.172:0/577882838 wait complete. 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.472+0000 7f6125d1e640 1 Processor -- start 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.472+0000 7f6125d1e640 1 -- start start 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f6125d1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61200ce150 0x7f61201410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f6125d1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61200ce5e0 0x7f6120139810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f6125d1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612013fa70 0x7f612014e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f6125d1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61201442b0 con 0x7f612013fa70 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f6125d1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6120144130 con 0x7f61200ce5e0 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f6125d1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6120072820 con 0x7f61200ce150 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f611ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612013fa70 0x7f612014e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f611ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612013fa70 0x7f612014e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43690/0 (socket says 172.21.15.172:43690) 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f611ffff640 1 -- 172.21.15.172:0/3192791821 learned_addr learned my addr 172.21.15.172:0/3192791821 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f611f7fe640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61200ce150 0x7f61201410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f611effd640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61200ce5e0 0x7f6120139810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f611ffff640 1 -- 172.21.15.172:0/3192791821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61200ce150 msgr2=0x7f61201410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f611ffff640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61200ce150 0x7f61201410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f611ffff640 1 -- 172.21.15.172:0/3192791821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61200ce5e0 msgr2=0x7f6120139810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f611ffff640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61200ce5e0 0x7f6120139810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f611ffff640 1 -- 172.21.15.172:0/3192791821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612015a730 con 0x7f612013fa70 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f611ffff640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612013fa70 0x7f612014e870 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f611800e9d0 tx=0x7f611800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:49.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f6107fff640 1 -- 172.21.15.172:0/3192791821 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118015070 con 0x7f612013fa70 2024-04-14T08:36:49.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f6107fff640 1 -- 172.21.15.172:0/3192791821 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6118003da0 con 0x7f612013fa70 2024-04-14T08:36:49.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f6107fff640 1 -- 172.21.15.172:0/3192791821 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611800cbb0 con 0x7f612013fa70 2024-04-14T08:36:49.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f6125d1e640 1 -- 172.21.15.172:0/3192791821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612015bec0 con 0x7f612013fa70 2024-04-14T08:36:49.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.473+0000 7f6125d1e640 1 -- 172.21.15.172:0/3192791821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612015c760 con 0x7f612013fa70 2024-04-14T08:36:49.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.474+0000 7f6125d1e640 1 -- 172.21.15.172:0/3192791821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6120068590 con 0x7f612013fa70 2024-04-14T08:36:49.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.475+0000 7f6107fff640 1 -- 172.21.15.172:0/3192791821 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6118018070 con 0x7f612013fa70 2024-04-14T08:36:49.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.475+0000 7f6107fff640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f610c06c8f0 0x7f610c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.475+0000 7f6107fff640 1 -- 172.21.15.172:0/3192791821 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f611808e100 con 0x7f612013fa70 2024-04-14T08:36:49.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.475+0000 7f611f7fe640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f610c06c8f0 0x7f610c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.476+0000 7f611f7fe640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f610c06c8f0 0x7f610c06edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6110036000 tx=0x7f611003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:49.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.480+0000 7f6107fff640 1 -- 172.21.15.172:0/3192791821 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f611805bd50 con 0x7f612013fa70 2024-04-14T08:36:49.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.693+0000 7f6125d1e640 1 -- 172.21.15.172:0/3192791821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f61200cf8b0 con 0x7f612013fa70 2024-04-14T08:36:49.695 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:36:49.694+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:36:49.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.695+0000 7f6107fff640 1 -- 172.21.15.172:0/3192791821 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) ==== 161+0+0 (secure 0 0 0) 0x7f6118060050 con 0x7f612013fa70 2024-04-14T08:36:49.697 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:36:49.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.697+0000 7f6125d1e640 1 -- 172.21.15.172:0/3192791821 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f610c06c8f0 msgr2=0x7f610c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.697+0000 7f6125d1e640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f610c06c8f0 0x7f610c06edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6110036000 tx=0x7f611003c040 comp rx=0 tx=0).stop 2024-04-14T08:36:49.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.697+0000 7f6125d1e640 1 -- 172.21.15.172:0/3192791821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612013fa70 msgr2=0x7f612014e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.697+0000 7f6125d1e640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612013fa70 0x7f612014e870 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f611800e9d0 tx=0x7f611800ee90 comp rx=0 tx=0).stop 2024-04-14T08:36:49.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.697+0000 7f6125d1e640 1 -- 172.21.15.172:0/3192791821 shutdown_connections 2024-04-14T08:36:49.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.697+0000 7f6125d1e640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f610c06c8f0 0x7f610c06edb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.697+0000 7f6125d1e640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f612013fa70 0x7f612014e870 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.697+0000 7f6125d1e640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61200ce5e0 0x7f6120139810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.697+0000 7f6125d1e640 1 --2- 172.21.15.172:0/3192791821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61200ce150 0x7f61201410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.697+0000 7f6125d1e640 1 -- 172.21.15.172:0/3192791821 >> 172.21.15.172:0/3192791821 conn(0x7f612006b520 msgr2=0x7f61200b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:49.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.698+0000 7f6125d1e640 1 -- 172.21.15.172:0/3192791821 shutdown_connections 2024-04-14T08:36:49.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.698+0000 7f6125d1e640 1 -- 172.21.15.172:0/3192791821 wait complete. 2024-04-14T08:36:49.703 INFO:tasks.ceph.mds.d.smithi177.stderr: -7687> 2024-04-14T08:35:36.941+0000 7f9d66ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:36:49.708 INFO:tasks.ceph.mds.d.smithi177.stderr: -4813> 2024-04-14T08:35:41.816+0000 7f9d62fe9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6115=0-4194304@1} caps={6115=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6115 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x555d1cc8a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-14T08:36:49.709 INFO:tasks.ceph.mds.d.smithi177.stderr: -2849> 2024-04-14T08:36:13.114+0000 7f9d62fe9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) caps={6115=pAsLsXsFs/-@2},l=6115(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x555d1cc8a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-14T08:36:49.712 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-14T08:36:49.735 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-04-14T08:36:49.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.813+0000 7f4a22c69640 1 Processor -- start 2024-04-14T08:36:49.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.813+0000 7f4a22c69640 1 -- start start 2024-04-14T08:36:49.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a22c69640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c13df30 0x7f4a1c0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a22c69640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4a1c15c380 0x7f4a1c0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a22c69640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a1c15b620 con 0x7f4a1c140770 2024-04-14T08:36:49.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a22c69640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f4a1c0cf9b0 con 0x7f4a1c15c380 2024-04-14T08:36:49.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a22c69640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1c1414a0 con 0x7f4a1c13df30 2024-04-14T08:36:49.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a1bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4a1c15c380 0x7f4a1c0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a1bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4a1c15c380 0x7f4a1c0b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39578/0 (socket says 172.21.15.172:39578) 2024-04-14T08:36:49.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a1bfff640 1 -- 172.21.15.172:0/3118923481 learned_addr learned my addr 172.21.15.172:0/3118923481 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:49.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a21206640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4a1c140770 0x7f4a1c140c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:52276/0 (socket says 172.21.15.172:52276) 2024-04-14T08:36:49.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a20a05640 1 --2- 172.21.15.172:0/3118923481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c13df30 0x7f4a1c0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a1bfff640 1 -- 172.21.15.172:0/3118923481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c13df30 msgr2=0x7f4a1c0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:49.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a1bfff640 1 --2- 172.21.15.172:0/3118923481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c13df30 0x7f4a1c0b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a1bfff640 1 -- 172.21.15.172:0/3118923481 >> v1:172.21.15.172:6789/0 conn(0x7f4a1c140770 legacy=0x7f4a1c140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a1bfff640 1 -- 172.21.15.172:0/3118923481 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c15b8a0 con 0x7f4a1c15c380 2024-04-14T08:36:49.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a1b7fe640 1 -- 172.21.15.172:0/3118923481 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 277729962 0 0) 0x7f4a14001490 con 0x7f4a1c140770 2024-04-14T08:36:49.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.814+0000 7f4a20a05640 1 --2- 172.21.15.172:0/3118923481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c13df30 0x7f4a1c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:36:49.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.815+0000 7f4a1bfff640 1 --2- 172.21.15.172:0/3118923481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a1c15c380 0x7f4a1c0b2210 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4a080098a0 tx=0x7f4a0802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9265598e2e57875 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.815+0000 7f4a1b7fe640 1 -- 172.21.15.172:0/3118923481 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08034070 con 0x7f4a1c15c380 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.815+0000 7f4a1b7fe640 1 -- 172.21.15.172:0/3118923481 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a08030840 con 0x7f4a1c15c380 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.815+0000 7f4a1b7fe640 1 -- 172.21.15.172:0/3118923481 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08030c10 con 0x7f4a1c15c380 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.816+0000 7f4a22c69640 1 -- 172.21.15.172:0/3118923481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a1c15c380 msgr2=0x7f4a1c0b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.816+0000 7f4a22c69640 1 --2- 172.21.15.172:0/3118923481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a1c15c380 0x7f4a1c0b2210 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4a080098a0 tx=0x7f4a0802b980 comp rx=0 tx=0).stop 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.816+0000 7f4a22c69640 1 -- 172.21.15.172:0/3118923481 shutdown_connections 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.816+0000 7f4a22c69640 1 --2- 172.21.15.172:0/3118923481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a1c15c380 0x7f4a1c0b2210 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.816+0000 7f4a22c69640 1 --2- 172.21.15.172:0/3118923481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c13df30 0x7f4a1c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.816+0000 7f4a22c69640 1 -- 172.21.15.172:0/3118923481 >> 172.21.15.172:0/3118923481 conn(0x7f4a1c06b520 msgr2=0x7f4a1c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.816+0000 7f4a22c69640 1 -- 172.21.15.172:0/3118923481 shutdown_connections 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.816+0000 7f4a22c69640 1 -- 172.21.15.172:0/3118923481 wait complete. 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.817+0000 7f4a22c69640 1 Processor -- start 2024-04-14T08:36:49.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.817+0000 7f4a22c69640 1 -- start start 2024-04-14T08:36:49.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a22c69640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a1c13df30 0x7f4a1c159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a22c69640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c140770 0x7f4a1c12d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a22c69640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a1c15c380 0x7f4a1c13c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a22c69640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c13e310 con 0x7f4a1c13df30 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a22c69640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c15be60 con 0x7f4a1c15c380 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a22c69640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1c0ce9a0 con 0x7f4a1c140770 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a20a05640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a1c13df30 0x7f4a1c159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a20a05640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a1c13df30 0x7f4a1c159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39580/0 (socket says 172.21.15.172:39580) 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a20a05640 1 -- 172.21.15.172:0/968517394 learned_addr learned my addr 172.21.15.172:0/968517394 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a20a05640 1 -- 172.21.15.172:0/968517394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c140770 msgr2=0x7f4a1c12d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a1bfff640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c140770 0x7f4a1c12d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a21206640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a1c15c380 0x7f4a1c13c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a20a05640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c140770 0x7f4a1c12d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a20a05640 1 -- 172.21.15.172:0/968517394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a1c15c380 msgr2=0x7f4a1c13c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:49.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a20a05640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a1c15c380 0x7f4a1c13c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:49.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a20a05640 1 -- 172.21.15.172:0/968517394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c12db90 con 0x7f4a1c13df30 2024-04-14T08:36:49.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a20a05640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a1c13df30 0x7f4a1c159c50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000c7f0 tx=0x7f4a1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:49.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a197fa640 1 -- 172.21.15.172:0/968517394 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10015070 con 0x7f4a1c13df30 2024-04-14T08:36:49.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a197fa640 1 -- 172.21.15.172:0/968517394 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a10007a30 con 0x7f4a1c13df30 2024-04-14T08:36:49.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.818+0000 7f4a22c69640 1 -- 172.21.15.172:0/968517394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1c15a350 con 0x7f4a1c13df30 2024-04-14T08:36:49.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.819+0000 7f4a197fa640 1 -- 172.21.15.172:0/968517394 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10007e00 con 0x7f4a1c13df30 2024-04-14T08:36:49.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.819+0000 7f4a22c69640 1 -- 172.21.15.172:0/968517394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1c15a910 con 0x7f4a1c13df30 2024-04-14T08:36:49.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.819+0000 7f4a22c69640 1 -- 172.21.15.172:0/968517394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1c068590 con 0x7f4a1c13df30 2024-04-14T08:36:49.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.820+0000 7f4a197fa640 1 -- 172.21.15.172:0/968517394 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a10018070 con 0x7f4a1c13df30 2024-04-14T08:36:49.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.820+0000 7f4a197fa640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4a0006c8f0 0x7f4a0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:49.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.821+0000 7f4a1bfff640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4a0006c8f0 0x7f4a0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:49.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.821+0000 7f4a197fa640 1 -- 172.21.15.172:0/968517394 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6332+0+0 (secure 0 0 0) 0x7f4a1008bd80 con 0x7f4a1c13df30 2024-04-14T08:36:49.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.821+0000 7f4a1bfff640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4a0006c8f0 0x7f4a0006edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4a08036000 tx=0x7f4a08008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:49.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:49.825+0000 7f4a197fa640 1 -- 172.21.15.172:0/968517394 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4a10059f40 con 0x7f4a1c13df30 2024-04-14T08:36:50.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.038+0000 7f4a22c69640 1 -- 172.21.15.172:0/968517394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4a1c13f850 con 0x7f4a1c13df30 2024-04-14T08:36:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.697+0000 7f4a197fa640 1 -- 172.21.15.172:0/968517394 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) ==== 104+0+0 (secure 0 0 0) 0x7f4a10059840 con 0x7f4a1c13df30 2024-04-14T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.699+0000 7f4a22c69640 1 -- 172.21.15.172:0/968517394 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4a0006c8f0 msgr2=0x7f4a0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.699+0000 7f4a22c69640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4a0006c8f0 0x7f4a0006edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4a08036000 tx=0x7f4a08008030 comp rx=0 tx=0).stop 2024-04-14T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.700+0000 7f4a22c69640 1 -- 172.21.15.172:0/968517394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a1c13df30 msgr2=0x7f4a1c159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.700+0000 7f4a22c69640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a1c13df30 0x7f4a1c159c50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000c7f0 tx=0x7f4a1000cbf0 comp rx=0 tx=0).stop 2024-04-14T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.700+0000 7f4a22c69640 1 -- 172.21.15.172:0/968517394 shutdown_connections 2024-04-14T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.700+0000 7f4a22c69640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4a0006c8f0 0x7f4a0006edb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.700+0000 7f4a22c69640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a1c15c380 0x7f4a1c13c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.700+0000 7f4a22c69640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a1c140770 0x7f4a1c12d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.700+0000 7f4a22c69640 1 --2- 172.21.15.172:0/968517394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a1c13df30 0x7f4a1c159c50 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.700+0000 7f4a22c69640 1 -- 172.21.15.172:0/968517394 >> 172.21.15.172:0/968517394 conn(0x7f4a1c06b520 msgr2=0x7f4a1c072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.700+0000 7f4a22c69640 1 -- 172.21.15.172:0/968517394 shutdown_connections 2024-04-14T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.700+0000 7f4a22c69640 1 -- 172.21.15.172:0/968517394 wait complete. 2024-04-14T08:36:50.715 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-14T08:36:50.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.823+0000 7f2f0bab7640 1 Processor -- start 2024-04-14T08:36:50.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.824+0000 7f2f0bab7640 1 -- start start 2024-04-14T08:36:50.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.824+0000 7f2f0bab7640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f04161d00 0x7f2f040b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:50.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.824+0000 7f2f0bab7640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f040d1bb0 con 0x7f2f040b5490 2024-04-14T08:36:50.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.824+0000 7f2f0bab7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f040d0df0 con 0x7f2f040d2cf0 2024-04-14T08:36:50.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.824+0000 7f2f0bab7640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2f040d0ba0 con 0x7f2f04161d00 2024-04-14T08:36:50.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.825+0000 7f2f09052640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2f040b5490 0x7f2f040d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:52288/0 (socket says 172.21.15.172:52288) 2024-04-14T08:36:50.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.825+0000 7f2f09052640 1 -- 172.21.15.172:0/1121297743 learned_addr learned my addr 172.21.15.172:0/1121297743 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:50.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.825+0000 7f2f09853640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f04161d00 0x7f2f040b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:50.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.825+0000 7f2f09853640 1 -- 172.21.15.172:0/1121297743 >> v1:172.21.15.177:6789/0 conn(0x7f2f040d2cf0 legacy=0x7f2f040cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:50.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.825+0000 7f2f09853640 1 -- 172.21.15.172:0/1121297743 >> v1:172.21.15.172:6789/0 conn(0x7f2f040b5490 legacy=0x7f2f040d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:50.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.825+0000 7f2f08851640 1 -- 172.21.15.172:0/1121297743 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2112181882 0 0) 0x7f2f040d1bb0 con 0x7f2f040b5490 2024-04-14T08:36:50.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.825+0000 7f2f09853640 1 -- 172.21.15.172:0/1121297743 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f040d10d0 con 0x7f2f04161d00 2024-04-14T08:36:50.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.826+0000 7f2f09853640 1 --2- 172.21.15.172:0/1121297743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f04161d00 0x7f2f040b30a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2f000097a0 tx=0x7f2f0002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1554e1aa1a0390ac server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:50.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.826+0000 7f2f08851640 1 -- 172.21.15.172:0/1121297743 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f000315a0 con 0x7f2f04161d00 2024-04-14T08:36:50.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.826+0000 7f2f08851640 1 -- 172.21.15.172:0/1121297743 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f00037070 con 0x7f2f04161d00 2024-04-14T08:36:50.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.826+0000 7f2f08851640 1 -- 172.21.15.172:0/1121297743 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00031960 con 0x7f2f04161d00 2024-04-14T08:36:50.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.826+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1121297743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f04161d00 msgr2=0x7f2f040b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:50.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.826+0000 7f2f0bab7640 1 --2- 172.21.15.172:0/1121297743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f04161d00 0x7f2f040b30a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2f000097a0 tx=0x7f2f0002b840 comp rx=0 tx=0).stop 2024-04-14T08:36:50.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.826+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1121297743 shutdown_connections 2024-04-14T08:36:50.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.826+0000 7f2f0bab7640 1 --2- 172.21.15.172:0/1121297743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f04161d00 0x7f2f040b30a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:50.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.826+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1121297743 >> 172.21.15.172:0/1121297743 conn(0x7f2f0409d570 msgr2=0x7f2f040d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:50.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.827+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1121297743 shutdown_connections 2024-04-14T08:36:50.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.827+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1121297743 wait complete. 2024-04-14T08:36:50.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.828+0000 7f2f0bab7640 1 Processor -- start 2024-04-14T08:36:50.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.828+0000 7f2f0bab7640 1 -- start start 2024-04-14T08:36:50.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.828+0000 7f2f0bab7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f040b5490 0x7f2f04161190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:50.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.828+0000 7f2f0bab7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f040d2cf0 0x7f2f040c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:50.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.828+0000 7f2f0bab7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f04161d00 0x7f2f0415de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:50.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.828+0000 7f2f0bab7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2f040d33b0 con 0x7f2f040b5490 2024-04-14T08:36:50.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.828+0000 7f2f0bab7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2f040d1720 con 0x7f2f04161d00 2024-04-14T08:36:50.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.828+0000 7f2f0bab7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2f040b5940 con 0x7f2f040d2cf0 2024-04-14T08:36:50.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.828+0000 7f2f09853640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f040b5490 0x7f2f04161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:50.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.828+0000 7f2f09853640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f040b5490 0x7f2f04161190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39598/0 (socket says 172.21.15.172:39598) 2024-04-14T08:36:50.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.828+0000 7f2f09853640 1 -- 172.21.15.172:0/1857289719 learned_addr learned my addr 172.21.15.172:0/1857289719 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:50.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2f09052640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f040d2cf0 0x7f2f040c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:50.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2f0a054640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f04161d00 0x7f2f0415de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2f09853640 1 -- 172.21.15.172:0/1857289719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f040d2cf0 msgr2=0x7f2f040c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2f09853640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f040d2cf0 0x7f2f040c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2f09853640 1 -- 172.21.15.172:0/1857289719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f04161d00 msgr2=0x7f2f0415de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2f09853640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f04161d00 0x7f2f0415de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2f09853640 1 -- 172.21.15.172:0/1857289719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f040c88a0 con 0x7f2f040b5490 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2f09853640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f040b5490 0x7f2f04161190 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2f000097a0 tx=0x7f2f000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2efa7fc640 1 -- 172.21.15.172:0/1857289719 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00032840 con 0x7f2f040b5490 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2efa7fc640 1 -- 172.21.15.172:0/1857289719 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f000329e0 con 0x7f2f040b5490 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2efa7fc640 1 -- 172.21.15.172:0/1857289719 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00032dd0 con 0x7f2f040b5490 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1857289719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f040c8b80 con 0x7f2f040b5490 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.829+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1857289719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f040c9140 con 0x7f2f040b5490 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.830+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1857289719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f040686a0 con 0x7f2f040b5490 2024-04-14T08:36:50.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.831+0000 7f2efa7fc640 1 -- 172.21.15.172:0/1857289719 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f0003b3f0 con 0x7f2f040b5490 2024-04-14T08:36:50.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.831+0000 7f2efa7fc640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ee806c920 0x7f2ee806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:50.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.831+0000 7f2efa7fc640 1 -- 172.21.15.172:0/1857289719 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6332+0+0 (secure 0 0 0) 0x7f2f0002f030 con 0x7f2f040b5490 2024-04-14T08:36:50.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.831+0000 7f2f09052640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ee806c920 0x7f2ee806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:50.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.831+0000 7f2f09052640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ee806c920 0x7f2ee806ede0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2f041616d0 tx=0x7f2ef4007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:50.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:50.836+0000 7f2efa7fc640 1 -- 172.21.15.172:0/1857289719 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2f00040070 con 0x7f2f040b5490 2024-04-14T08:36:51.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.009+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1857289719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2f0408e320 con 0x7f2f040b5490 2024-04-14T08:36:51.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.703+0000 7f2efa7fc640 1 -- 172.21.15.172:0/1857289719 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) ==== 181+0+0 (secure 0 0 0) 0x7f2f0007b710 con 0x7f2f040b5490 2024-04-14T08:36:51.705 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' removed 2024-04-14T08:36:51.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.706+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1857289719 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ee806c920 msgr2=0x7f2ee806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:51.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.706+0000 7f2f0bab7640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ee806c920 0x7f2ee806ede0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2f041616d0 tx=0x7f2ef4007de0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.706+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1857289719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f040b5490 msgr2=0x7f2f04161190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:51.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.706+0000 7f2f0bab7640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f040b5490 0x7f2f04161190 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2f000097a0 tx=0x7f2f000046c0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.707+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1857289719 shutdown_connections 2024-04-14T08:36:51.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.707+0000 7f2f0bab7640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ee806c920 0x7f2ee806ede0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.707+0000 7f2f0bab7640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f04161d00 0x7f2f0415de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.707+0000 7f2f0bab7640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2f040d2cf0 0x7f2f040c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.707+0000 7f2f0bab7640 1 --2- 172.21.15.172:0/1857289719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f040b5490 0x7f2f04161190 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.707+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1857289719 >> 172.21.15.172:0/1857289719 conn(0x7f2f0409d570 msgr2=0x7f2f0406d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:51.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.707+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1857289719 shutdown_connections 2024-04-14T08:36:51.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.707+0000 7f2f0bab7640 1 -- 172.21.15.172:0/1857289719 wait complete. 2024-04-14T08:36:51.721 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-14T08:36:51.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.825+0000 7f47d3c2d640 1 Processor -- start 2024-04-14T08:36:51.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.825+0000 7f47d3c2d640 1 -- start start 2024-04-14T08:36:51.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d3c2d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f47cc159ef0 0x7f47cc0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:51.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d3c2d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f47cc15d270 0x7f47cc06fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:51.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d3c2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47cc070190 0x7f47cc15fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:51.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d3c2d640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f47cc0705a0 con 0x7f47cc15d270 2024-04-14T08:36:51.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d3c2d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f47cc0d0550 con 0x7f47cc159ef0 2024-04-14T08:36:51.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d3c2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f47cc0cf9d0 con 0x7f47cc070190 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d11c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f47cc15d270 0x7f47cc06fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d11c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f47cc15d270 0x7f47cc06fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39614/0 (socket says 172.21.15.172:39614) 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d11c8640 1 -- 172.21.15.172:0/1777882495 learned_addr learned my addr 172.21.15.172:0/1777882495 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d21ca640 1 --2- 172.21.15.172:0/1777882495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47cc070190 0x7f47cc15fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d19c9640 1 --2- 172.21.15.172:0/1777882495 >> v2:172.21.15.177:3300/0 conn(0x7f47cc159ef0 0x7f47cc0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d11c8640 1 -- 172.21.15.172:0/1777882495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47cc070190 msgr2=0x7f47cc15fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d11c8640 1 --2- 172.21.15.172:0/1777882495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47cc070190 0x7f47cc15fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d11c8640 1 -- 172.21.15.172:0/1777882495 >> v2:172.21.15.177:3300/0 conn(0x7f47cc159ef0 msgr2=0x7f47cc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d11c8640 1 --2- 172.21.15.172:0/1777882495 >> v2:172.21.15.177:3300/0 conn(0x7f47cc159ef0 0x7f47cc0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d11c8640 1 -- 172.21.15.172:0/1777882495 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc0cfcb0 con 0x7f47cc15d270 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d21ca640 1 --2- 172.21.15.172:0/1777882495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47cc070190 0x7f47cc15fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.826+0000 7f47d19c9640 1 --2- 172.21.15.172:0/1777882495 >> v2:172.21.15.177:3300/0 conn(0x7f47cc159ef0 0x7f47cc0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:36:51.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.827+0000 7f47d11c8640 1 --2- 172.21.15.172:0/1777882495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47cc15d270 0x7f47cc06fc50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f47cc06d6e0 tx=0x7f47bc02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55cbe5402375b37 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:51.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.827+0000 7f47d09c7640 1 -- 172.21.15.172:0/1777882495 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0317d0 con 0x7f47cc15d270 2024-04-14T08:36:51.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.827+0000 7f47d09c7640 1 -- 172.21.15.172:0/1777882495 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47bc037070 con 0x7f47cc15d270 2024-04-14T08:36:51.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.827+0000 7f47d09c7640 1 -- 172.21.15.172:0/1777882495 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc031b90 con 0x7f47cc15d270 2024-04-14T08:36:51.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.828+0000 7f47d3c2d640 1 -- 172.21.15.172:0/1777882495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47cc15d270 msgr2=0x7f47cc06fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:51.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.828+0000 7f47d3c2d640 1 --2- 172.21.15.172:0/1777882495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47cc15d270 0x7f47cc06fc50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f47cc06d6e0 tx=0x7f47bc02b870 comp rx=0 tx=0).stop 2024-04-14T08:36:51.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.828+0000 7f47d3c2d640 1 -- 172.21.15.172:0/1777882495 shutdown_connections 2024-04-14T08:36:51.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.828+0000 7f47d3c2d640 1 --2- 172.21.15.172:0/1777882495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47cc070190 0x7f47cc15fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.828+0000 7f47d3c2d640 1 --2- 172.21.15.172:0/1777882495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47cc15d270 0x7f47cc06fc50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.828+0000 7f47d3c2d640 1 --2- 172.21.15.172:0/1777882495 >> v2:172.21.15.177:3300/0 conn(0x7f47cc159ef0 0x7f47cc0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.828+0000 7f47d3c2d640 1 -- 172.21.15.172:0/1777882495 >> 172.21.15.172:0/1777882495 conn(0x7f47cc071ba0 msgr2=0x7f47cc071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:51.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.828+0000 7f47d3c2d640 1 -- 172.21.15.172:0/1777882495 shutdown_connections 2024-04-14T08:36:51.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.828+0000 7f47d3c2d640 1 -- 172.21.15.172:0/1777882495 wait complete. 2024-04-14T08:36:51.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.829+0000 7f47d3c2d640 1 Processor -- start 2024-04-14T08:36:51.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.829+0000 7f47d3c2d640 1 -- start start 2024-04-14T08:36:51.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d3c2d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47cc070190 0x7f47cc0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:51.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d3c2d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47cc159ef0 0x7f47cc14f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:51.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d3c2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47cc15d270 0x7f47cc0ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:51.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d3c2d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc15d700 con 0x7f47cc159ef0 2024-04-14T08:36:51.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d3c2d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc0d0270 con 0x7f47cc070190 2024-04-14T08:36:51.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d3c2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f47cc160320 con 0x7f47cc15d270 2024-04-14T08:36:51.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d11c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47cc159ef0 0x7f47cc14f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:51.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d11c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47cc159ef0 0x7f47cc14f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39618/0 (socket says 172.21.15.172:39618) 2024-04-14T08:36:51.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d11c8640 1 -- 172.21.15.172:0/4135522270 learned_addr learned my addr 172.21.15.172:0/4135522270 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:51.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d19c9640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47cc070190 0x7f47cc0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:51.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d21ca640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47cc15d270 0x7f47cc0ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:51.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d11c8640 1 -- 172.21.15.172:0/4135522270 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47cc15d270 msgr2=0x7f47cc0ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:51.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d11c8640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47cc15d270 0x7f47cc0ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d11c8640 1 -- 172.21.15.172:0/4135522270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47cc070190 msgr2=0x7f47cc0c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d11c8640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47cc070190 0x7f47cc0c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d11c8640 1 -- 172.21.15.172:0/4135522270 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc0aec60 con 0x7f47cc159ef0 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d19c9640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47cc070190 0x7f47cc0c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d21ca640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47cc15d270 0x7f47cc0ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d11c8640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47cc159ef0 0x7f47cc14f840 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f47bc000c00 tx=0x7f47bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47c27fc640 1 -- 172.21.15.172:0/4135522270 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc032870 con 0x7f47cc159ef0 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47c27fc640 1 -- 172.21.15.172:0/4135522270 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47bc032a10 con 0x7f47cc159ef0 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47c27fc640 1 -- 172.21.15.172:0/4135522270 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc032e00 con 0x7f47cc159ef0 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.830+0000 7f47d3c2d640 1 -- 172.21.15.172:0/4135522270 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc0aef40 con 0x7f47cc159ef0 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.831+0000 7f47d3c2d640 1 -- 172.21.15.172:0/4135522270 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc06e4b0 con 0x7f47cc159ef0 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.832+0000 7f47d3c2d640 1 -- 172.21.15.172:0/4135522270 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47cc0686a0 con 0x7f47cc159ef0 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.832+0000 7f47c27fc640 1 -- 172.21.15.172:0/4135522270 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47bc0453f0 con 0x7f47cc159ef0 2024-04-14T08:36:51.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.832+0000 7f47c27fc640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f47b006c8f0 0x7f47b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:51.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.833+0000 7f47d19c9640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f47b006c8f0 0x7f47b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:51.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.833+0000 7f47c27fc640 1 -- 172.21.15.172:0/4135522270 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5865+0+0 (secure 0 0 0) 0x7f47bc0add60 con 0x7f47cc159ef0 2024-04-14T08:36:51.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.833+0000 7f47d19c9640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f47b006c8f0 0x7f47b006edb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f47c800aa30 tx=0x7f47c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:51.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:51.837+0000 7f47c27fc640 1 -- 172.21.15.172:0/4135522270 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f47bc040070 con 0x7f47cc159ef0 2024-04-14T08:36:52.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.009+0000 7f47d3c2d640 1 -- 172.21.15.172:0/4135522270 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f47cc0cddf0 con 0x7f47cc159ef0 2024-04-14T08:36:52.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.709+0000 7f47c27fc640 1 -- 172.21.15.172:0/4135522270 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) ==== 169+0+0 (secure 0 0 0) 0x7f47bc07c180 con 0x7f47cc159ef0 2024-04-14T08:36:52.711 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' removed 2024-04-14T08:36:52.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.713+0000 7f47d3c2d640 1 -- 172.21.15.172:0/4135522270 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f47b006c8f0 msgr2=0x7f47b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:52.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.714+0000 7f47d3c2d640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f47b006c8f0 0x7f47b006edb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f47c800aa30 tx=0x7f47c8008040 comp rx=0 tx=0).stop 2024-04-14T08:36:52.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.714+0000 7f47d3c2d640 1 -- 172.21.15.172:0/4135522270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47cc159ef0 msgr2=0x7f47cc14f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:52.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.714+0000 7f47d3c2d640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47cc159ef0 0x7f47cc14f840 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f47bc000c00 tx=0x7f47bc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:36:52.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.714+0000 7f47d3c2d640 1 -- 172.21.15.172:0/4135522270 shutdown_connections 2024-04-14T08:36:52.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.714+0000 7f47d3c2d640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f47b006c8f0 0x7f47b006edb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:52.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.714+0000 7f47d3c2d640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47cc15d270 0x7f47cc0ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:52.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.714+0000 7f47d3c2d640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47cc159ef0 0x7f47cc14f840 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:52.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.714+0000 7f47d3c2d640 1 --2- 172.21.15.172:0/4135522270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47cc070190 0x7f47cc0c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:52.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.714+0000 7f47d3c2d640 1 -- 172.21.15.172:0/4135522270 >> 172.21.15.172:0/4135522270 conn(0x7f47cc071ba0 msgr2=0x7f47cc0ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:52.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.714+0000 7f47d3c2d640 1 -- 172.21.15.172:0/4135522270 shutdown_connections 2024-04-14T08:36:52.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.714+0000 7f47d3c2d640 1 -- 172.21.15.172:0/4135522270 wait complete. 2024-04-14T08:36:52.730 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-14T08:36:52.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.835+0000 7f194b5dc640 1 Processor -- start 2024-04-14T08:36:52.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.835+0000 7f194b5dc640 1 -- start start 2024-04-14T08:36:52.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.836+0000 7f194b5dc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f194406f3e0 0x7f19440b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:52.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.836+0000 7f194b5dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19440ceb00 0x7f19440cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:52.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.836+0000 7f194b5dc640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19440ccdd0 con 0x7f194415c420 2024-04-14T08:36:52.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.836+0000 7f194b5dc640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f19440d30a0 con 0x7f194406f3e0 2024-04-14T08:36:52.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.836+0000 7f194b5dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f194406f870 con 0x7f19440ceb00 2024-04-14T08:36:52.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.836+0000 7f1949378640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f194415c420 0x7f19440d0010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:52290/0 (socket says 172.21.15.172:52290) 2024-04-14T08:36:52.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.836+0000 7f1949378640 1 -- 172.21.15.172:0/1525129038 learned_addr learned my addr 172.21.15.172:0/1525129038 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:52.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1185116686 0 0) 0x7f19440ccdd0 con 0x7f194415c420 2024-04-14T08:36:52.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1930003520 con 0x7f194415c420 2024-04-14T08:36:52.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f1948b77640 1 --2- 172.21.15.172:0/1525129038 >> v2:172.21.15.177:3300/0 conn(0x7f194406f3e0 0x7f19440b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:52.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f1949b79640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19440ceb00 0x7f19440cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1069964349 0 0) 0x7f1930003520 con 0x7f194415c420 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19440ceb00 msgr2=0x7f19440cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19440ceb00 0x7f19440cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 >> v2:172.21.15.177:3300/0 conn(0x7f194406f3e0 msgr2=0x7f19440b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 --2- 172.21.15.172:0/1525129038 >> v2:172.21.15.177:3300/0 conn(0x7f194406f3e0 0x7f19440b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19440d2ae0 con 0x7f194415c420 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f19400019f0 con 0x7f194415c420 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f1949b79640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19440ceb00 0x7f19440cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 >> v1:172.21.15.172:6789/0 conn(0x7f194415c420 legacy=0x7f19440d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19300039b0 0x7f1930003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19440ceb00 0x7f1930004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1930004d40 0x7f19300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f19440d30a0 con 0x7f1930004d40 2024-04-14T08:36:52.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f194406f870 con 0x7f19300039b0 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f193000b9c0 con 0x7f19440ceb00 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f1949b79640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1930004d40 0x7f19300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f1949378640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19300039b0 0x7f1930003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f1948b77640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19440ceb00 0x7f1930004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f1949b79640 1 -- 172.21.15.172:0/1525129038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19440ceb00 msgr2=0x7f1930004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f1949b79640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19440ceb00 0x7f1930004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f1949b79640 1 -- 172.21.15.172:0/1525129038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19300039b0 msgr2=0x7f1930003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f1949b79640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19300039b0 0x7f1930003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.837+0000 7f1949b79640 1 -- 172.21.15.172:0/1525129038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1930003520 con 0x7f1930004d40 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.838+0000 7f1949b79640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1930004d40 0x7f19300051c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f193c0098a0 tx=0x7f1930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a6180f3e5454ec server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.838+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c00f040 con 0x7f1930004d40 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.838+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c004230 con 0x7f1930004d40 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.838+0000 7f193bfff640 1 -- 172.21.15.172:0/1525129038 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c003610 con 0x7f1930004d40 2024-04-14T08:36:52.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.839+0000 7f194b5dc640 1 -- 172.21.15.172:0/1525129038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1930004d40 msgr2=0x7f19300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:52.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.839+0000 7f194b5dc640 1 --2- 172.21.15.172:0/1525129038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1930004d40 0x7f19300051c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f193c0098a0 tx=0x7f1930005830 comp rx=0 tx=0).stop 2024-04-14T08:36:52.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.839+0000 7f1949b79640 1 -- 172.21.15.172:0/1525129038 reap_dead start 2024-04-14T08:36:52.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.839+0000 7f194b5dc640 1 -- 172.21.15.172:0/1525129038 shutdown_connections 2024-04-14T08:36:52.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.839+0000 7f194b5dc640 1 -- 172.21.15.172:0/1525129038 >> 172.21.15.172:0/1525129038 conn(0x7f194408c000 msgr2=0x7f19440d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:52.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.839+0000 7f194b5dc640 1 -- 172.21.15.172:0/1525129038 shutdown_connections 2024-04-14T08:36:52.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.839+0000 7f194b5dc640 1 -- 172.21.15.172:0/1525129038 wait complete. 2024-04-14T08:36:52.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.840+0000 7f194b5dc640 1 Processor -- start 2024-04-14T08:36:52.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.840+0000 7f194b5dc640 1 -- start start 2024-04-14T08:36:52.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f194b5dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19441f1af0 0x7f1944149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:52.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f194b5dc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19441f1ed0 0x7f194409bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:52.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f194b5dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f194414b380 0x7f194414bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:52.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f194b5dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f19440b11e0 con 0x7f194414b380 2024-04-14T08:36:52.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f194b5dc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f19440cd070 con 0x7f19441f1ed0 2024-04-14T08:36:52.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f194b5dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f194415c800 con 0x7f19441f1af0 2024-04-14T08:36:52.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f1949b79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f194414b380 0x7f194414bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:52.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f1949b79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f194414b380 0x7f194414bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39628/0 (socket says 172.21.15.172:39628) 2024-04-14T08:36:52.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f1949b79640 1 -- 172.21.15.172:0/1269029388 learned_addr learned my addr 172.21.15.172:0/1269029388 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:52.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f1948b77640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19441f1ed0 0x7f194409bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f1949378640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19441f1af0 0x7f1944149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f1949b79640 1 -- 172.21.15.172:0/1269029388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19441f1af0 msgr2=0x7f1944149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f1949b79640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19441f1af0 0x7f1944149c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f1949b79640 1 -- 172.21.15.172:0/1269029388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19441f1ed0 msgr2=0x7f194409bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f1949b79640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19441f1ed0 0x7f194409bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f1949b79640 1 -- 172.21.15.172:0/1269029388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19440c8440 con 0x7f194414b380 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f1948b77640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19441f1ed0 0x7f194409bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.841+0000 7f1949b79640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f194414b380 0x7f194414bcb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f193c0099c0 tx=0x7f193c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.842+0000 7f1939ffb640 1 -- 172.21.15.172:0/1269029388 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c048070 con 0x7f194414b380 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.842+0000 7f1939ffb640 1 -- 172.21.15.172:0/1269029388 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c00f000 con 0x7f194414b380 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.842+0000 7f194b5dc640 1 -- 172.21.15.172:0/1269029388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f194414a2e0 con 0x7f194414b380 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.842+0000 7f1939ffb640 1 -- 172.21.15.172:0/1269029388 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c006e10 con 0x7f194414b380 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.842+0000 7f194b5dc640 1 -- 172.21.15.172:0/1269029388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f194414a8a0 con 0x7f194414b380 2024-04-14T08:36:52.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.843+0000 7f194b5dc640 1 -- 172.21.15.172:0/1269029388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1944075490 con 0x7f194414b380 2024-04-14T08:36:52.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.843+0000 7f1939ffb640 1 -- 172.21.15.172:0/1269029388 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f193c030670 con 0x7f194414b380 2024-04-14T08:36:52.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.844+0000 7f1939ffb640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f192806c920 0x7f192806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:52.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.844+0000 7f1939ffb640 1 -- 172.21.15.172:0/1269029388 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f193c0ace30 con 0x7f194414b380 2024-04-14T08:36:52.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.844+0000 7f1949378640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f192806c920 0x7f192806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:52.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.844+0000 7f1949378640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f192806c920 0x7f192806ede0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1940004bc0 tx=0x7f1940006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:52.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:52.848+0000 7f1939ffb640 1 -- 172.21.15.172:0/1269029388 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f193c034020 con 0x7f194414b380 2024-04-14T08:36:53.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.028+0000 7f194b5dc640 1 -- 172.21.15.172:0/1269029388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f19440d2820 con 0x7f194414b380 2024-04-14T08:36:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.031+0000 7f1939ffb640 1 -- 172.21.15.172:0/1269029388 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f193c07b460 con 0x7f194414b380 2024-04-14T08:36:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.033+0000 7f194b5dc640 1 -- 172.21.15.172:0/1269029388 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f192806c920 msgr2=0x7f192806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.033+0000 7f194b5dc640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f192806c920 0x7f192806ede0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1940004bc0 tx=0x7f1940006040 comp rx=0 tx=0).stop 2024-04-14T08:36:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.033+0000 7f194b5dc640 1 -- 172.21.15.172:0/1269029388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f194414b380 msgr2=0x7f194414bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.033+0000 7f194b5dc640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f194414b380 0x7f194414bcb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f193c0099c0 tx=0x7f193c004730 comp rx=0 tx=0).stop 2024-04-14T08:36:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.033+0000 7f194b5dc640 1 -- 172.21.15.172:0/1269029388 shutdown_connections 2024-04-14T08:36:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.033+0000 7f194b5dc640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f192806c920 0x7f192806ede0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.033+0000 7f194b5dc640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f194414b380 0x7f194414bcb0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.033+0000 7f194b5dc640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19441f1ed0 0x7f194409bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.033+0000 7f194b5dc640 1 --2- 172.21.15.172:0/1269029388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19441f1af0 0x7f1944149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.033+0000 7f194b5dc640 1 -- 172.21.15.172:0/1269029388 >> 172.21.15.172:0/1269029388 conn(0x7f194408c000 msgr2=0x7f19440c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.033+0000 7f194b5dc640 1 -- 172.21.15.172:0/1269029388 shutdown_connections 2024-04-14T08:36:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.033+0000 7f194b5dc640 1 -- 172.21.15.172:0/1269029388 wait complete. 2024-04-14T08:36:53.046 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-14T08:36:53.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.145+0000 7f7bcee36640 1 Processor -- start 2024-04-14T08:36:53.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.145+0000 7f7bcee36640 1 -- start start 2024-04-14T08:36:53.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.146+0000 7f7bcee36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7bc80b21d0 0x7f7bc815c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:53.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.146+0000 7f7bcee36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7bc815c5a0 0x7f7bc80b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:53.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.146+0000 7f7bcee36640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc80b3700 con 0x7f7bc80b1df0 2024-04-14T08:36:53.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.146+0000 7f7bcee36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc80b39e0 con 0x7f7bc80b21d0 2024-04-14T08:36:53.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.146+0000 7f7bcee36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc80b3cc0 con 0x7f7bc815c5a0 2024-04-14T08:36:53.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.146+0000 7f7bc7fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7bc80b21d0 0x7f7bc815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:53.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.146+0000 7f7bccbd2640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7bc80b1df0 0x7f7bc80b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:52296/0 (socket says 172.21.15.172:52296) 2024-04-14T08:36:53.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.146+0000 7f7bccbd2640 1 -- 172.21.15.172:0/2404211183 learned_addr learned my addr 172.21.15.172:0/2404211183 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:53.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.147+0000 7f7bc7fff640 1 -- 172.21.15.172:0/2404211183 >> v2:172.21.15.177:3300/0 conn(0x7f7bc815c5a0 msgr2=0x7f7bc80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:53.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.147+0000 7f7bcd3d3640 1 --2- 172.21.15.172:0/2404211183 >> v2:172.21.15.177:3300/0 conn(0x7f7bc815c5a0 0x7f7bc80b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:53.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.147+0000 7f7bc7fff640 1 --2- 172.21.15.172:0/2404211183 >> v2:172.21.15.177:3300/0 conn(0x7f7bc815c5a0 0x7f7bc80b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.147+0000 7f7bc7fff640 1 -- 172.21.15.172:0/2404211183 >> v1:172.21.15.172:6789/0 conn(0x7f7bc80b1df0 legacy=0x7f7bc80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:53.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.147+0000 7f7bc7fff640 1 -- 172.21.15.172:0/2404211183 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc815bb40 con 0x7f7bc80b21d0 2024-04-14T08:36:53.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.147+0000 7f7bc77fe640 1 -- 172.21.15.172:0/2404211183 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 6865307 0 0) 0x7f7bc80b3700 con 0x7f7bc80b1df0 2024-04-14T08:36:53.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.147+0000 7f7bcd3d3640 1 --2- 172.21.15.172:0/2404211183 >> v2:172.21.15.177:3300/0 conn(0x7f7bc815c5a0 0x7f7bc80b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:36:53.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.147+0000 7f7bc7fff640 1 --2- 172.21.15.172:0/2404211183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bc80b21d0 0x7f7bc815c060 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80b3fa0 tx=0x7f7bbc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e6cbe48d0fa867 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:53.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.147+0000 7f7bc77fe640 1 -- 172.21.15.172:0/2404211183 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc035020 con 0x7f7bc80b21d0 2024-04-14T08:36:53.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.147+0000 7f7bc77fe640 1 -- 172.21.15.172:0/2404211183 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bbc031830 con 0x7f7bc80b21d0 2024-04-14T08:36:53.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.147+0000 7f7bc77fe640 1 -- 172.21.15.172:0/2404211183 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc031c00 con 0x7f7bc80b21d0 2024-04-14T08:36:53.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.148+0000 7f7bcee36640 1 -- 172.21.15.172:0/2404211183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bc80b21d0 msgr2=0x7f7bc815c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:53.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.148+0000 7f7bcee36640 1 --2- 172.21.15.172:0/2404211183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bc80b21d0 0x7f7bc815c060 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80b3fa0 tx=0x7f7bbc02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.148+0000 7f7bcee36640 1 -- 172.21.15.172:0/2404211183 shutdown_connections 2024-04-14T08:36:53.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.148+0000 7f7bcee36640 1 --2- 172.21.15.172:0/2404211183 >> v2:172.21.15.177:3300/0 conn(0x7f7bc815c5a0 0x7f7bc80b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.148+0000 7f7bcee36640 1 --2- 172.21.15.172:0/2404211183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bc80b21d0 0x7f7bc815c060 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.148+0000 7f7bcee36640 1 -- 172.21.15.172:0/2404211183 >> 172.21.15.172:0/2404211183 conn(0x7f7bc806ec30 msgr2=0x7f7bc806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:53.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.148+0000 7f7bcee36640 1 -- 172.21.15.172:0/2404211183 shutdown_connections 2024-04-14T08:36:53.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.148+0000 7f7bcee36640 1 -- 172.21.15.172:0/2404211183 wait complete. 2024-04-14T08:36:53.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.149+0000 7f7bcee36640 1 Processor -- start 2024-04-14T08:36:53.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.149+0000 7f7bcee36640 1 -- start start 2024-04-14T08:36:53.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.150+0000 7f7bcee36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc80b1df0 0x7f7bc80ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:53.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.150+0000 7f7bcee36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bc80b21d0 0x7f7bc8143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.150+0000 7f7bcee36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bc81f5190 0x7f7bc81f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.150+0000 7f7bcee36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc80b4700 con 0x7f7bc80b21d0 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.150+0000 7f7bcee36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc80b4580 con 0x7f7bc81f5190 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.150+0000 7f7bcee36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc80b4880 con 0x7f7bc80b1df0 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.150+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bc80b21d0 0x7f7bc8143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.150+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bc80b21d0 0x7f7bc8143d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39640/0 (socket says 172.21.15.172:39640) 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.150+0000 7f7bc7fff640 1 -- 172.21.15.172:0/1145029059 learned_addr learned my addr 172.21.15.172:0/1145029059 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.150+0000 7f7bcd3d3640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bc81f5190 0x7f7bc81f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.150+0000 7f7bccbd2640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc80b1df0 0x7f7bc80ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.150+0000 7f7bc7fff640 1 -- 172.21.15.172:0/1145029059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc80b1df0 msgr2=0x7f7bc80ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.151+0000 7f7bc7fff640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc80b1df0 0x7f7bc80ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.151+0000 7f7bc7fff640 1 -- 172.21.15.172:0/1145029059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bc81f5190 msgr2=0x7f7bc81f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:53.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.151+0000 7f7bc7fff640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bc81f5190 0x7f7bc81f5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.151+0000 7f7bc7fff640 1 -- 172.21.15.172:0/1145029059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc81f1ea0 con 0x7f7bc80b21d0 2024-04-14T08:36:53.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.151+0000 7f7bc7fff640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bc80b21d0 0x7f7bc8143d90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009890 tx=0x7f7bbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:53.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.151+0000 7f7bc57fa640 1 -- 172.21.15.172:0/1145029059 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc040070 con 0x7f7bc80b21d0 2024-04-14T08:36:53.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.151+0000 7f7bc57fa640 1 -- 172.21.15.172:0/1145029059 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bbc039a20 con 0x7f7bc80b21d0 2024-04-14T08:36:53.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.151+0000 7f7bcee36640 1 -- 172.21.15.172:0/1145029059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc81f5d00 con 0x7f7bc80b21d0 2024-04-14T08:36:53.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.151+0000 7f7bc57fa640 1 -- 172.21.15.172:0/1145029059 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc035040 con 0x7f7bc80b21d0 2024-04-14T08:36:53.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.151+0000 7f7bcee36640 1 -- 172.21.15.172:0/1145029059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc80cd800 con 0x7f7bc80b21d0 2024-04-14T08:36:53.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.152+0000 7f7bcee36640 1 -- 172.21.15.172:0/1145029059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc8068520 con 0x7f7bc80b21d0 2024-04-14T08:36:53.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.152+0000 7f7bc57fa640 1 -- 172.21.15.172:0/1145029059 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bbc038020 con 0x7f7bc80b21d0 2024-04-14T08:36:53.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.153+0000 7f7bc57fa640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7bb406c8f0 0x7f7bb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:53.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.153+0000 7f7bc57fa640 1 -- 172.21.15.172:0/1145029059 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f7bbc0ad410 con 0x7f7bc80b21d0 2024-04-14T08:36:53.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.153+0000 7f7bccbd2640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7bb406c8f0 0x7f7bb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:53.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.153+0000 7f7bccbd2640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7bb406c8f0 0x7f7bb406edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8000f60 tx=0x7f7bb8007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:53.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.157+0000 7f7bc57fa640 1 -- 172.21.15.172:0/1145029059 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7bbc07b9b0 con 0x7f7bc80b21d0 2024-04-14T08:36:53.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.401+0000 7f7bcee36640 1 -- 172.21.15.172:0/1145029059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f7bc81442d0 con 0x7f7bc80b21d0 2024-04-14T08:36:53.600 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:36:53.603 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:36:53.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.711+0000 7f7bc57fa640 1 -- 172.21.15.172:0/1145029059 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v212) ==== 169+0+0 (secure 0 0 0) 0x7f7bbc07b2b0 con 0x7f7bc80b21d0 2024-04-14T08:36:53.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.713+0000 7f7bcee36640 1 -- 172.21.15.172:0/1145029059 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7bb406c8f0 msgr2=0x7f7bb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:53.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.713+0000 7f7bcee36640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7bb406c8f0 0x7f7bb406edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8000f60 tx=0x7f7bb8007c00 comp rx=0 tx=0).stop 2024-04-14T08:36:53.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.713+0000 7f7bcee36640 1 -- 172.21.15.172:0/1145029059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bc80b21d0 msgr2=0x7f7bc8143d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:53.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.713+0000 7f7bcee36640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bc80b21d0 0x7f7bc8143d90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009890 tx=0x7f7bbc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.713+0000 7f7bcee36640 1 -- 172.21.15.172:0/1145029059 shutdown_connections 2024-04-14T08:36:53.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.713+0000 7f7bcee36640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7bb406c8f0 0x7f7bb406edb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.713+0000 7f7bcee36640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bc81f5190 0x7f7bc81f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.713+0000 7f7bcee36640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bc80b21d0 0x7f7bc8143d90 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.713+0000 7f7bcee36640 1 --2- 172.21.15.172:0/1145029059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bc80b1df0 0x7f7bc80ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.713+0000 7f7bcee36640 1 -- 172.21.15.172:0/1145029059 >> 172.21.15.172:0/1145029059 conn(0x7f7bc806ec30 msgr2=0x7f7bc815ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:53.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.713+0000 7f7bcee36640 1 -- 172.21.15.172:0/1145029059 shutdown_connections 2024-04-14T08:36:53.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.713+0000 7f7bcee36640 1 -- 172.21.15.172:0/1145029059 wait complete. 2024-04-14T08:36:53.727 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-14T08:36:53.727 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-14T08:36:53.727 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-14T08:36:53.727 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:36:53.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.868+0000 7facbbbfe640 1 Processor -- start 2024-04-14T08:36:53.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.868+0000 7facbbbfe640 1 -- start start 2024-04-14T08:36:53.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facbbbfe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7facb415f400 0x7facb415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:53.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facbbbfe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7facb40b1b80 0x7facb40b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:53.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facbbbfe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facb413f600 con 0x7facb415d9d0 2024-04-14T08:36:53.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facbbbfe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7facb40b2520 con 0x7facb415f400 2024-04-14T08:36:53.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facbbbfe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7facb4140690 con 0x7facb40b1b80 2024-04-14T08:36:53.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facb999a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7facb415f400 0x7facb415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:53.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facb9199640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7facb415d9d0 0x7facb415d190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:52306/0 (socket says 172.21.15.172:52306) 2024-04-14T08:36:53.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facb9199640 1 -- 172.21.15.172:0/3763651036 learned_addr learned my addr 172.21.15.172:0/3763651036 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:53.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facb999a640 1 -- 172.21.15.172:0/3763651036 >> v2:172.21.15.177:3300/0 conn(0x7facb40b1b80 msgr2=0x7facb40b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:53.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facba19b640 1 --2- 172.21.15.172:0/3763651036 >> v2:172.21.15.177:3300/0 conn(0x7facb40b1b80 0x7facb40b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:53.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facb999a640 1 --2- 172.21.15.172:0/3763651036 >> v2:172.21.15.177:3300/0 conn(0x7facb40b1b80 0x7facb40b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facb8998640 1 -- 172.21.15.172:0/3763651036 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 32046798 0 0) 0x7faca4001110 con 0x7facb415d9d0 2024-04-14T08:36:53.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facb999a640 1 -- 172.21.15.172:0/3763651036 >> v1:172.21.15.172:6789/0 conn(0x7facb415d9d0 legacy=0x7facb415d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:53.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.869+0000 7facb999a640 1 -- 172.21.15.172:0/3763651036 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb413fa80 con 0x7facb415f400 2024-04-14T08:36:53.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.870+0000 7facb999a640 1 --2- 172.21.15.172:0/3763651036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facb415f400 0x7facb415f7e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7facb406f330 tx=0x7facb002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3208ebd70770035 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:53.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.870+0000 7facb8998640 1 -- 172.21.15.172:0/3763651036 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0035020 con 0x7facb415f400 2024-04-14T08:36:53.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.870+0000 7facb8998640 1 -- 172.21.15.172:0/3763651036 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb0031830 con 0x7facb415f400 2024-04-14T08:36:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.870+0000 7facb8998640 1 -- 172.21.15.172:0/3763651036 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0031c00 con 0x7facb415f400 2024-04-14T08:36:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.871+0000 7facbbbfe640 1 -- 172.21.15.172:0/3763651036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facb415f400 msgr2=0x7facb415f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.871+0000 7facbbbfe640 1 --2- 172.21.15.172:0/3763651036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facb415f400 0x7facb415f7e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7facb406f330 tx=0x7facb002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.871+0000 7facbbbfe640 1 -- 172.21.15.172:0/3763651036 shutdown_connections 2024-04-14T08:36:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.871+0000 7facbbbfe640 1 --2- 172.21.15.172:0/3763651036 >> v2:172.21.15.177:3300/0 conn(0x7facb40b1b80 0x7facb40b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.871+0000 7facbbbfe640 1 --2- 172.21.15.172:0/3763651036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facb415f400 0x7facb415f7e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.871+0000 7facbbbfe640 1 -- 172.21.15.172:0/3763651036 >> 172.21.15.172:0/3763651036 conn(0x7facb406ec30 msgr2=0x7facb406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.871+0000 7facbbbfe640 1 -- 172.21.15.172:0/3763651036 shutdown_connections 2024-04-14T08:36:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.871+0000 7facbbbfe640 1 -- 172.21.15.172:0/3763651036 wait complete. 2024-04-14T08:36:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.872+0000 7facbbbfe640 1 Processor -- start 2024-04-14T08:36:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.872+0000 7facbbbfe640 1 -- start start 2024-04-14T08:36:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.872+0000 7facbbbfe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facb40b1b80 0x7facb4156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.872+0000 7facbbbfe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facb415d9d0 0x7facb4153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facbbbfe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facb415f400 0x7facb41f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facbbbfe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7facb4140320 con 0x7facb40b1b80 2024-04-14T08:36:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facbbbfe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7facb4140040 con 0x7facb415d9d0 2024-04-14T08:36:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facbbbfe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7facb415de60 con 0x7facb415f400 2024-04-14T08:36:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facb999a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facb40b1b80 0x7facb4156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facb999a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facb40b1b80 0x7facb4156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39644/0 (socket says 172.21.15.172:39644) 2024-04-14T08:36:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facb999a640 1 -- 172.21.15.172:0/1377842741 learned_addr learned my addr 172.21.15.172:0/1377842741 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facb999a640 1 -- 172.21.15.172:0/1377842741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facb415f400 msgr2=0x7facb41f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facb999a640 1 --2- 172.21.15.172:0/1377842741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facb415f400 0x7facb41f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facb999a640 1 -- 172.21.15.172:0/1377842741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facb415d9d0 msgr2=0x7facb4153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facb999a640 1 --2- 172.21.15.172:0/1377842741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facb415d9d0 0x7facb4153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facb999a640 1 -- 172.21.15.172:0/1377842741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb4154210 con 0x7facb40b1b80 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facb999a640 1 --2- 172.21.15.172:0/1377842741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facb40b1b80 0x7facb4156190 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7facb0000c00 tx=0x7facb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facaa7fc640 1 -- 172.21.15.172:0/1377842741 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0040070 con 0x7facb40b1b80 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facaa7fc640 1 -- 172.21.15.172:0/1377842741 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb0039a20 con 0x7facb40b1b80 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facbbbfe640 1 -- 172.21.15.172:0/1377842741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb4156850 con 0x7facb40b1b80 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facaa7fc640 1 -- 172.21.15.172:0/1377842741 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0035040 con 0x7facb40b1b80 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.873+0000 7facbbbfe640 1 -- 172.21.15.172:0/1377842741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb413b8e0 con 0x7facb40b1b80 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.874+0000 7facbbbfe640 1 -- 172.21.15.172:0/1377842741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb4068520 con 0x7facb40b1b80 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.875+0000 7facaa7fc640 1 -- 172.21.15.172:0/1377842741 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facb004b060 con 0x7facb40b1b80 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.875+0000 7facaa7fc640 1 --2- 172.21.15.172:0/1377842741 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fac9806c8f0 0x7fac9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:53.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.875+0000 7facaa7fc640 1 -- 172.21.15.172:0/1377842741 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7facb00ad430 con 0x7facb40b1b80 2024-04-14T08:36:53.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.875+0000 7facb9199640 1 --2- 172.21.15.172:0/1377842741 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fac9806c8f0 0x7fac9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:53.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.875+0000 7facb9199640 1 --2- 172.21.15.172:0/1377842741 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fac9806c8f0 0x7fac9806edb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7facb41f6540 tx=0x7faca4001310 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:53.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:53.879+0000 7facaa7fc640 1 -- 172.21.15.172:0/1377842741 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7facb007b9d0 con 0x7facb40b1b80 2024-04-14T08:36:54.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.058+0000 7facbbbfe640 1 -- 172.21.15.172:0/1377842741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facb406c2c0 con 0x7facb40b1b80 2024-04-14T08:36:54.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.059+0000 7facaa7fc640 1 -- 172.21.15.172:0/1377842741 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+13356 (secure 0 0 0) 0x7facb007b2d0 con 0x7facb40b1b80 2024-04-14T08:36:54.060 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:54.060 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":66,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:36:52.705635+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/4176013980":"2024-04-15T08:36:49.695889+0000","172.21.15.177:6833/4176013980":"2024-04-15T08:36:49.695889+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:36:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.061+0000 7facbbbfe640 1 -- 172.21.15.172:0/1377842741 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fac9806c8f0 msgr2=0x7fac9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.061+0000 7facbbbfe640 1 --2- 172.21.15.172:0/1377842741 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fac9806c8f0 0x7fac9806edb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7facb41f6540 tx=0x7faca4001310 comp rx=0 tx=0).stop 2024-04-14T08:36:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.061+0000 7facbbbfe640 1 -- 172.21.15.172:0/1377842741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facb40b1b80 msgr2=0x7facb4156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.061+0000 7facbbbfe640 1 --2- 172.21.15.172:0/1377842741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facb40b1b80 0x7facb4156190 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7facb0000c00 tx=0x7facb00046c0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.061+0000 7facbbbfe640 1 -- 172.21.15.172:0/1377842741 shutdown_connections 2024-04-14T08:36:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.061+0000 7facbbbfe640 1 --2- 172.21.15.172:0/1377842741 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fac9806c8f0 0x7fac9806edb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.061+0000 7facbbbfe640 1 --2- 172.21.15.172:0/1377842741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7facb415f400 0x7facb41f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.061+0000 7facbbbfe640 1 --2- 172.21.15.172:0/1377842741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facb415d9d0 0x7facb4153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.061+0000 7facbbbfe640 1 --2- 172.21.15.172:0/1377842741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facb40b1b80 0x7facb4156190 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.061+0000 7facbbbfe640 1 -- 172.21.15.172:0/1377842741 >> 172.21.15.172:0/1377842741 conn(0x7facb406ec30 msgr2=0x7facb4072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.061+0000 7facbbbfe640 1 -- 172.21.15.172:0/1377842741 shutdown_connections 2024-04-14T08:36:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.061+0000 7facbbbfe640 1 -- 172.21.15.172:0/1377842741 wait complete. 2024-04-14T08:36:54.073 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:36:54.073 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:36:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.175+0000 7fc1ec636640 1 Processor -- start 2024-04-14T08:36:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.175+0000 7fc1ec636640 1 -- start start 2024-04-14T08:36:54.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.176+0000 7fc1ec636640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc1e415e430 0x7fc1e4162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:54.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.176+0000 7fc1ec636640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1e40d2e00 0x7fc1e41629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:54.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.176+0000 7fc1ec636640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1e40cefb0 con 0x7fc1e4162540 2024-04-14T08:36:54.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.176+0000 7fc1ec636640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc1e40cf290 con 0x7fc1e40d2e00 2024-04-14T08:36:54.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.176+0000 7fc1ec636640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc1e40cf570 con 0x7fc1e415e430 2024-04-14T08:36:54.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.176+0000 7fc1eabd3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1e40d2e00 0x7fc1e41629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:54.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.177+0000 7fc1eabd3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1e40d2e00 0x7fc1e41629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39654/0 (socket says 172.21.15.172:39654) 2024-04-14T08:36:54.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.177+0000 7fc1eabd3640 1 -- 172.21.15.172:0/4106123780 learned_addr learned my addr 172.21.15.172:0/4106123780 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:54.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.177+0000 7fc1ea3d2640 1 --2- 172.21.15.172:0/4106123780 >> v2:172.21.15.177:3300/0 conn(0x7fc1e415e430 0x7fc1e4162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:54.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.177+0000 7fc1eabd3640 1 -- 172.21.15.172:0/4106123780 >> v2:172.21.15.177:3300/0 conn(0x7fc1e415e430 msgr2=0x7fc1e4162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:54.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.177+0000 7fc1eabd3640 1 --2- 172.21.15.172:0/4106123780 >> v2:172.21.15.177:3300/0 conn(0x7fc1e415e430 0x7fc1e4162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.177+0000 7fc1eabd3640 1 -- 172.21.15.172:0/4106123780 >> v1:172.21.15.177:6789/0 conn(0x7fc1e4162540 legacy=0x7fc1e40b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:54.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.177+0000 7fc1eabd3640 1 -- 172.21.15.172:0/4106123780 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e40cf850 con 0x7fc1e40d2e00 2024-04-14T08:36:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.178+0000 7fc1eabd3640 1 --2- 172.21.15.172:0/4106123780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e40d2e00 0x7fc1e41629d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40b6db0 tx=0x7fc1dc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e341ad7c50339d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.178+0000 7fc1e93d0640 1 -- 172.21.15.172:0/4106123780 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc02cb20 con 0x7fc1e40d2e00 2024-04-14T08:36:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.178+0000 7fc1e93d0640 1 -- 172.21.15.172:0/4106123780 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1dc02ccc0 con 0x7fc1e40d2e00 2024-04-14T08:36:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.178+0000 7fc1e93d0640 1 -- 172.21.15.172:0/4106123780 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc034610 con 0x7fc1e40d2e00 2024-04-14T08:36:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.178+0000 7fc1ec636640 1 -- 172.21.15.172:0/4106123780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e40d2e00 msgr2=0x7fc1e41629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.178+0000 7fc1ec636640 1 --2- 172.21.15.172:0/4106123780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e40d2e00 0x7fc1e41629d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40b6db0 tx=0x7fc1dc02bf80 comp rx=0 tx=0).stop 2024-04-14T08:36:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.178+0000 7fc1ec636640 1 -- 172.21.15.172:0/4106123780 shutdown_connections 2024-04-14T08:36:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.178+0000 7fc1ec636640 1 --2- 172.21.15.172:0/4106123780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e40d2e00 0x7fc1e41629d0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.178+0000 7fc1ec636640 1 --2- 172.21.15.172:0/4106123780 >> v2:172.21.15.177:3300/0 conn(0x7fc1e415e430 0x7fc1e4162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.178+0000 7fc1ec636640 1 -- 172.21.15.172:0/4106123780 >> 172.21.15.172:0/4106123780 conn(0x7fc1e409d560 msgr2=0x7fc1e40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.178+0000 7fc1ec636640 1 -- 172.21.15.172:0/4106123780 shutdown_connections 2024-04-14T08:36:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.178+0000 7fc1ec636640 1 -- 172.21.15.172:0/4106123780 wait complete. 2024-04-14T08:36:54.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.180+0000 7fc1ec636640 1 Processor -- start 2024-04-14T08:36:54.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.180+0000 7fc1ec636640 1 -- start start 2024-04-14T08:36:54.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.180+0000 7fc1ec636640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e40d2e00 0x7fc1e4155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:54.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.180+0000 7fc1ec636640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e415e430 0x7fc1e40b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:54.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.180+0000 7fc1ec636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e4162540 0x7fc1e40b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:54.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.180+0000 7fc1ec636640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e40d4e20 con 0x7fc1e40d2e00 2024-04-14T08:36:54.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.180+0000 7fc1ec636640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e415e810 con 0x7fc1e415e430 2024-04-14T08:36:54.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.180+0000 7fc1ec636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e40d0050 con 0x7fc1e4162540 2024-04-14T08:36:54.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.180+0000 7fc1ea3d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e40d2e00 0x7fc1e4155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.180+0000 7fc1ea3d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e40d2e00 0x7fc1e4155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39662/0 (socket says 172.21.15.172:39662) 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.180+0000 7fc1ea3d2640 1 -- 172.21.15.172:0/3180156576 learned_addr learned my addr 172.21.15.172:0/3180156576 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1eabd3640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e4162540 0x7fc1e40b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1e9bd1640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e415e430 0x7fc1e40b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1ea3d2640 1 -- 172.21.15.172:0/3180156576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e4162540 msgr2=0x7fc1e40b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1ea3d2640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e4162540 0x7fc1e40b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1ea3d2640 1 -- 172.21.15.172:0/3180156576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e415e430 msgr2=0x7fc1e40b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1ea3d2640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e415e430 0x7fc1e40b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1ea3d2640 1 -- 172.21.15.172:0/3180156576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e40b6d80 con 0x7fc1e40d2e00 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1ea3d2640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e40d2e00 0x7fc1e4155ad0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000e9e0 tx=0x7fc1e000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1daffd640 1 -- 172.21.15.172:0/3180156576 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0017070 con 0x7fc1e40d2e00 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1daffd640 1 -- 172.21.15.172:0/3180156576 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e000c9d0 con 0x7fc1e40d2e00 2024-04-14T08:36:54.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1ec636640 1 -- 172.21.15.172:0/3180156576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e40b7060 con 0x7fc1e40d2e00 2024-04-14T08:36:54.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1daffd640 1 -- 172.21.15.172:0/3180156576 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000cdc0 con 0x7fc1e40d2e00 2024-04-14T08:36:54.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.181+0000 7fc1ec636640 1 -- 172.21.15.172:0/3180156576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e40b7900 con 0x7fc1e40d2e00 2024-04-14T08:36:54.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.182+0000 7fc1ec636640 1 -- 172.21.15.172:0/3180156576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e4068590 con 0x7fc1e40d2e00 2024-04-14T08:36:54.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.182+0000 7fc1daffd640 1 -- 172.21.15.172:0/3180156576 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1e001a070 con 0x7fc1e40d2e00 2024-04-14T08:36:54.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.183+0000 7fc1daffd640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1c806c8f0 0x7fc1c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:54.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.183+0000 7fc1daffd640 1 -- 172.21.15.172:0/3180156576 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7fc1e008bd30 con 0x7fc1e40d2e00 2024-04-14T08:36:54.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.183+0000 7fc1e9bd1640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1c806c8f0 0x7fc1c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:54.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.183+0000 7fc1e9bd1640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1c806c8f0 0x7fc1c806edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4161f10 tx=0x7fc1d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:54.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.187+0000 7fc1daffd640 1 -- 172.21.15.172:0/3180156576 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc1e005a2d0 con 0x7fc1e40d2e00 2024-04-14T08:36:54.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.358+0000 7fc1ec636640 1 -- 172.21.15.172:0/3180156576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc1e40d18a0 con 0x7fc1e40d2e00 2024-04-14T08:36:54.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.359+0000 7fc1daffd640 1 -- 172.21.15.172:0/3180156576 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7fc1e0059bd0 con 0x7fc1e40d2e00 2024-04-14T08:36:54.360 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:36:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.360+0000 7fc1ec636640 1 -- 172.21.15.172:0/3180156576 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1c806c8f0 msgr2=0x7fc1c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.361+0000 7fc1ec636640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1c806c8f0 0x7fc1c806edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4161f10 tx=0x7fc1d4009040 comp rx=0 tx=0).stop 2024-04-14T08:36:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.361+0000 7fc1ec636640 1 -- 172.21.15.172:0/3180156576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e40d2e00 msgr2=0x7fc1e4155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.361+0000 7fc1ec636640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e40d2e00 0x7fc1e4155ad0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000e9e0 tx=0x7fc1e000eea0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.361+0000 7fc1ec636640 1 -- 172.21.15.172:0/3180156576 shutdown_connections 2024-04-14T08:36:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.361+0000 7fc1ec636640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1c806c8f0 0x7fc1c806edb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.361+0000 7fc1ec636640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e4162540 0x7fc1e40b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.361+0000 7fc1ec636640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e415e430 0x7fc1e40b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.361+0000 7fc1ec636640 1 --2- 172.21.15.172:0/3180156576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1e40d2e00 0x7fc1e4155ad0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.361+0000 7fc1ec636640 1 -- 172.21.15.172:0/3180156576 >> 172.21.15.172:0/3180156576 conn(0x7fc1e409d560 msgr2=0x7fc1e40b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.361+0000 7fc1ec636640 1 -- 172.21.15.172:0/3180156576 shutdown_connections 2024-04-14T08:36:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.361+0000 7fc1ec636640 1 -- 172.21.15.172:0/3180156576 wait complete. 2024-04-14T08:36:54.373 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-14T08:36:54.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.473+0000 7f85048d5640 1 Processor -- start 2024-04-14T08:36:54.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.473+0000 7f85048d5640 1 -- start start 2024-04-14T08:36:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f85048d5640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f85000b21d0 0x7f850015c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f85048d5640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f850015c5a0 0x7f85000b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f85048d5640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85000b3700 con 0x7f85000b1df0 2024-04-14T08:36:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f85048d5640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f85000b39e0 con 0x7f85000b21d0 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f85048d5640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f85000b3cc0 con 0x7f850015c5a0 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f84fdd9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f85000b21d0 0x7f850015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f84fdd9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f85000b21d0 0x7f850015c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39670/0 (socket says 172.21.15.172:39670) 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f84fdd9b640 1 -- 172.21.15.172:0/1926025703 learned_addr learned my addr 172.21.15.172:0/1926025703 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f84fdd9b640 1 -- 172.21.15.172:0/1926025703 >> v2:172.21.15.177:3300/0 conn(0x7f850015c5a0 msgr2=0x7f85000b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f84fed9d640 1 --2- 172.21.15.172:0/1926025703 >> v2:172.21.15.177:3300/0 conn(0x7f850015c5a0 0x7f85000b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f84fdd9b640 1 --2- 172.21.15.172:0/1926025703 >> v2:172.21.15.177:3300/0 conn(0x7f850015c5a0 0x7f85000b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f84fdd9b640 1 -- 172.21.15.172:0/1926025703 >> v1:172.21.15.177:6789/0 conn(0x7f85000b1df0 legacy=0x7f85000b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.474+0000 7f84fdd9b640 1 -- 172.21.15.172:0/1926025703 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850015bb40 con 0x7f85000b21d0 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.475+0000 7f84fdd9b640 1 --2- 172.21.15.172:0/1926025703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85000b21d0 0x7f850015c060 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f85000b3fa0 tx=0x7f84f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefb6636d9fd8895 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.475+0000 7f84fd59a640 1 -- 172.21.15.172:0/1926025703 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4035020 con 0x7f85000b21d0 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.475+0000 7f84fd59a640 1 -- 172.21.15.172:0/1926025703 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f4031870 con 0x7f85000b21d0 2024-04-14T08:36:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.475+0000 7f84fd59a640 1 -- 172.21.15.172:0/1926025703 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4031c40 con 0x7f85000b21d0 2024-04-14T08:36:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.475+0000 7f85048d5640 1 -- 172.21.15.172:0/1926025703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85000b21d0 msgr2=0x7f850015c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.475+0000 7f85048d5640 1 --2- 172.21.15.172:0/1926025703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85000b21d0 0x7f850015c060 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f85000b3fa0 tx=0x7f84f402b910 comp rx=0 tx=0).stop 2024-04-14T08:36:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.476+0000 7f85048d5640 1 -- 172.21.15.172:0/1926025703 shutdown_connections 2024-04-14T08:36:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.476+0000 7f85048d5640 1 --2- 172.21.15.172:0/1926025703 >> v2:172.21.15.177:3300/0 conn(0x7f850015c5a0 0x7f85000b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.476+0000 7f85048d5640 1 --2- 172.21.15.172:0/1926025703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85000b21d0 0x7f850015c060 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.476+0000 7f85048d5640 1 -- 172.21.15.172:0/1926025703 >> 172.21.15.172:0/1926025703 conn(0x7f850006ec30 msgr2=0x7f850006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.476+0000 7f85048d5640 1 -- 172.21.15.172:0/1926025703 shutdown_connections 2024-04-14T08:36:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.476+0000 7f85048d5640 1 -- 172.21.15.172:0/1926025703 wait complete. 2024-04-14T08:36:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.477+0000 7f85048d5640 1 Processor -- start 2024-04-14T08:36:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.477+0000 7f85048d5640 1 -- start start 2024-04-14T08:36:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f85048d5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85000b1df0 0x7f85000cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f85048d5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000b21d0 0x7f850015a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f85048d5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8500159680 0x7f85000cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f85048d5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85000b4700 con 0x7f85000b1df0 2024-04-14T08:36:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f85048d5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f85000b4580 con 0x7f8500159680 2024-04-14T08:36:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f85048d5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f85000b4880 con 0x7f85000b21d0 2024-04-14T08:36:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84fe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85000b1df0 0x7f85000cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84fe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85000b1df0 0x7f85000cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39678/0 (socket says 172.21.15.172:39678) 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84fe59c640 1 -- 172.21.15.172:0/3301005661 learned_addr learned my addr 172.21.15.172:0/3301005661 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84fed9d640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8500159680 0x7f85000cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84fdd9b640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000b21d0 0x7f850015a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84fe59c640 1 -- 172.21.15.172:0/3301005661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000b21d0 msgr2=0x7f850015a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84fe59c640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000b21d0 0x7f850015a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84fe59c640 1 -- 172.21.15.172:0/3301005661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8500159680 msgr2=0x7f85000cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84fe59c640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8500159680 0x7f85000cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84fe59c640 1 -- 172.21.15.172:0/3301005661 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85000ae470 con 0x7f85000b1df0 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84fed9d640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8500159680 0x7f85000cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84fe59c640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85000b1df0 0x7f85000cec60 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f84f000b3f0 tx=0x7f84f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84eeffd640 1 -- 172.21.15.172:0/3301005661 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0015020 con 0x7f85000b1df0 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84eeffd640 1 -- 172.21.15.172:0/3301005661 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f0011830 con 0x7f85000b1df0 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f84eeffd640 1 -- 172.21.15.172:0/3301005661 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0011c20 con 0x7f85000b1df0 2024-04-14T08:36:54.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f85048d5640 1 -- 172.21.15.172:0/3301005661 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85000ae750 con 0x7f85000b1df0 2024-04-14T08:36:54.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.478+0000 7f85048d5640 1 -- 172.21.15.172:0/3301005661 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85000aed10 con 0x7f85000b1df0 2024-04-14T08:36:54.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.479+0000 7f85048d5640 1 -- 172.21.15.172:0/3301005661 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8500068520 con 0x7f85000b1df0 2024-04-14T08:36:54.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.480+0000 7f84eeffd640 1 -- 172.21.15.172:0/3301005661 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84f0018020 con 0x7f85000b1df0 2024-04-14T08:36:54.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.480+0000 7f84eeffd640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84e006c8f0 0x7f84e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.480+0000 7f84eeffd640 1 -- 172.21.15.172:0/3301005661 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f84f008b600 con 0x7f85000b1df0 2024-04-14T08:36:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.480+0000 7f84fdd9b640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84e006c8f0 0x7f84e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.481+0000 7f84fdd9b640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84e006c8f0 0x7f84e006edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84f4036000 tx=0x7f84f403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.485+0000 7f84eeffd640 1 -- 172.21.15.172:0/3301005661 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f84f001f030 con 0x7f85000b1df0 2024-04-14T08:36:54.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:54.726+0000 7f85048d5640 1 -- 172.21.15.172:0/3301005661 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f85000b14f0 con 0x7f85000b1df0 2024-04-14T08:36:55.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.719+0000 7f84eeffd640 1 -- 172.21.15.172:0/3301005661 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v214) ==== 169+0+0 (secure 0 0 0) 0x7f84f00172c0 con 0x7f85000b1df0 2024-04-14T08:36:55.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.721+0000 7f85048d5640 1 -- 172.21.15.172:0/3301005661 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84e006c8f0 msgr2=0x7f84e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:55.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.721+0000 7f85048d5640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84e006c8f0 0x7f84e006edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84f4036000 tx=0x7f84f403c040 comp rx=0 tx=0).stop 2024-04-14T08:36:55.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.721+0000 7f85048d5640 1 -- 172.21.15.172:0/3301005661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85000b1df0 msgr2=0x7f85000cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:55.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.721+0000 7f85048d5640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85000b1df0 0x7f85000cec60 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f84f000b3f0 tx=0x7f84f000b8b0 comp rx=0 tx=0).stop 2024-04-14T08:36:55.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.721+0000 7f85048d5640 1 -- 172.21.15.172:0/3301005661 shutdown_connections 2024-04-14T08:36:55.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.721+0000 7f85048d5640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f84e006c8f0 0x7f84e006edb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:55.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.721+0000 7f85048d5640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8500159680 0x7f85000cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:55.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.721+0000 7f85048d5640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f85000b21d0 0x7f850015a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:55.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.721+0000 7f85048d5640 1 --2- 172.21.15.172:0/3301005661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85000b1df0 0x7f85000cec60 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:55.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.721+0000 7f85048d5640 1 -- 172.21.15.172:0/3301005661 >> 172.21.15.172:0/3301005661 conn(0x7f850006ec30 msgr2=0x7f850015cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:55.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.722+0000 7f85048d5640 1 -- 172.21.15.172:0/3301005661 shutdown_connections 2024-04-14T08:36:55.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.722+0000 7f85048d5640 1 -- 172.21.15.172:0/3301005661 wait complete. 2024-04-14T08:36:55.736 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-14T08:36:55.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.844+0000 7f5ded155640 1 Processor -- start 2024-04-14T08:36:55.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.844+0000 7f5ded155640 1 -- start start 2024-04-14T08:36:55.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.845+0000 7f5ded155640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5de813df30 0x7f5de80b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:55.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.845+0000 7f5ded155640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5de815c380 0x7f5de80b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:55.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.845+0000 7f5ded155640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de815b620 con 0x7f5de8140770 2024-04-14T08:36:55.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.845+0000 7f5ded155640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5de80cf9b0 con 0x7f5de813df30 2024-04-14T08:36:55.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.845+0000 7f5ded155640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5de81414a0 con 0x7f5de815c380 2024-04-14T08:36:55.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.845+0000 7f5de659c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5de815c380 0x7f5de80b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:55.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.845+0000 7f5de659c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5de815c380 0x7f5de80b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:34272/0 (socket says 172.21.15.172:34272) 2024-04-14T08:36:55.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.845+0000 7f5de659c640 1 -- 172.21.15.172:0/881101972 learned_addr learned my addr 172.21.15.172:0/881101972 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:55.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.845+0000 7f5de6d9d640 1 --2- 172.21.15.172:0/881101972 >> v2:172.21.15.177:3300/0 conn(0x7f5de813df30 0x7f5de80b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:55.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.846+0000 7f5de5d9b640 1 -- 172.21.15.172:0/881101972 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3050550592 0 0) 0x7f5de0001490 con 0x7f5de8140770 2024-04-14T08:36:55.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.846+0000 7f5de5d9b640 1 -- 172.21.15.172:0/881101972 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5dd0003520 con 0x7f5de8140770 2024-04-14T08:36:55.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.846+0000 7f5de659c640 1 -- 172.21.15.172:0/881101972 >> v2:172.21.15.177:3300/0 conn(0x7f5de813df30 msgr2=0x7f5de80b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:55.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.846+0000 7f5de659c640 1 --2- 172.21.15.172:0/881101972 >> v2:172.21.15.177:3300/0 conn(0x7f5de813df30 0x7f5de80b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:55.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.846+0000 7f5de659c640 1 -- 172.21.15.172:0/881101972 >> v1:172.21.15.177:6789/0 conn(0x7f5de8140770 legacy=0x7f5de8140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:55.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.846+0000 7f5de659c640 1 -- 172.21.15.172:0/881101972 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de815b8a0 con 0x7f5de815c380 2024-04-14T08:36:55.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.846+0000 7f5de659c640 1 --2- 172.21.15.172:0/881101972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5de815c380 0x7f5de80b2210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0098a0 tx=0x7f5ddc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f8180ff60726a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:55.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.847+0000 7f5de5d9b640 1 -- 172.21.15.172:0/881101972 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc034070 con 0x7f5de815c380 2024-04-14T08:36:55.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.847+0000 7f5de5d9b640 1 -- 172.21.15.172:0/881101972 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ddc030840 con 0x7f5de815c380 2024-04-14T08:36:55.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.847+0000 7f5de5d9b640 1 -- 172.21.15.172:0/881101972 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc030bf0 con 0x7f5de815c380 2024-04-14T08:36:55.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.847+0000 7f5ded155640 1 -- 172.21.15.172:0/881101972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5de815c380 msgr2=0x7f5de80b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:55.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.847+0000 7f5ded155640 1 --2- 172.21.15.172:0/881101972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5de815c380 0x7f5de80b2210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0098a0 tx=0x7f5ddc02b980 comp rx=0 tx=0).stop 2024-04-14T08:36:55.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.847+0000 7f5ded155640 1 -- 172.21.15.172:0/881101972 shutdown_connections 2024-04-14T08:36:55.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.847+0000 7f5ded155640 1 --2- 172.21.15.172:0/881101972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5de815c380 0x7f5de80b2210 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:55.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.847+0000 7f5ded155640 1 --2- 172.21.15.172:0/881101972 >> v2:172.21.15.177:3300/0 conn(0x7f5de813df30 0x7f5de80b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:55.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.847+0000 7f5ded155640 1 -- 172.21.15.172:0/881101972 >> 172.21.15.172:0/881101972 conn(0x7f5de806b520 msgr2=0x7f5de806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:55.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.847+0000 7f5ded155640 1 -- 172.21.15.172:0/881101972 shutdown_connections 2024-04-14T08:36:55.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.847+0000 7f5ded155640 1 -- 172.21.15.172:0/881101972 wait complete. 2024-04-14T08:36:55.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.849+0000 7f5ded155640 1 Processor -- start 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.849+0000 7f5ded155640 1 -- start start 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.849+0000 7f5ded155640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de813df30 0x7f5de8151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.849+0000 7f5ded155640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de8140770 0x7f5de8153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.849+0000 7f5ded155640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5de815c380 0x7f5de80aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.849+0000 7f5ded155640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5de813e310 con 0x7f5de813df30 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.849+0000 7f5ded155640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5de815be60 con 0x7f5de8140770 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.849+0000 7f5ded155640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5de80ce9a0 con 0x7f5de815c380 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5de6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de813df30 0x7f5de8151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5de659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de8140770 0x7f5de8153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5de659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de8140770 0x7f5de8153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:54854/0 (socket says 172.21.15.172:54854) 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5de659c640 1 -- 172.21.15.172:0/4141483429 learned_addr learned my addr 172.21.15.172:0/4141483429 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5de759e640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5de815c380 0x7f5de80aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:55.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5de659c640 1 -- 172.21.15.172:0/4141483429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5de815c380 msgr2=0x7f5de80aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:55.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5de659c640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5de815c380 0x7f5de80aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:55.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5de659c640 1 -- 172.21.15.172:0/4141483429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de813df30 msgr2=0x7f5de8151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:55.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5de659c640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de813df30 0x7f5de8151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:55.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5de659c640 1 -- 172.21.15.172:0/4141483429 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de8154280 con 0x7f5de8140770 2024-04-14T08:36:55.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5de6d9d640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de813df30 0x7f5de8151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:36:55.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5de659c640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de8140770 0x7f5de8153d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0090b0 tx=0x7f5ddc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:55.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5dcf7fe640 1 -- 172.21.15.172:0/4141483429 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc034050 con 0x7f5de8140770 2024-04-14T08:36:55.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5dcf7fe640 1 -- 172.21.15.172:0/4141483429 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ddc003820 con 0x7f5de8140770 2024-04-14T08:36:55.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5ded155640 1 -- 172.21.15.172:0/4141483429 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de8154650 con 0x7f5de8140770 2024-04-14T08:36:55.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5dcf7fe640 1 -- 172.21.15.172:0/4141483429 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc003bd0 con 0x7f5de8140770 2024-04-14T08:36:55.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.850+0000 7f5ded155640 1 -- 172.21.15.172:0/4141483429 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de8138400 con 0x7f5de8140770 2024-04-14T08:36:55.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.851+0000 7f5ded155640 1 -- 172.21.15.172:0/4141483429 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de8068590 con 0x7f5de8140770 2024-04-14T08:36:55.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.852+0000 7f5dcf7fe640 1 -- 172.21.15.172:0/4141483429 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ddc037070 con 0x7f5de8140770 2024-04-14T08:36:55.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.852+0000 7f5dcf7fe640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5dd406c8f0 0x7f5dd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:55.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.852+0000 7f5dcf7fe640 1 -- 172.21.15.172:0/4141483429 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f5ddc0ad920 con 0x7f5de8140770 2024-04-14T08:36:55.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.852+0000 7f5de6d9d640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5dd406c8f0 0x7f5dd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:55.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.853+0000 7f5de6d9d640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5dd406c8f0 0x7f5dd406edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009f30 tx=0x7f5dd8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:55.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:55.857+0000 7f5dcf7fe640 1 -- 172.21.15.172:0/4141483429 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5de8154280 con 0x7f5de8140770 2024-04-14T08:36:56.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.034+0000 7f5ded155640 1 -- 172.21.15.172:0/4141483429 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5de80b38b0 con 0x7f5de8140770 2024-04-14T08:36:56.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.042+0000 7f5dcf7fe640 1 -- 172.21.15.172:0/4141483429 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f5ddc07b880 con 0x7f5de8140770 2024-04-14T08:36:56.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.044+0000 7f5ded155640 1 -- 172.21.15.172:0/4141483429 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5dd406c8f0 msgr2=0x7f5dd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.044+0000 7f5ded155640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5dd406c8f0 0x7f5dd406edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009f30 tx=0x7f5dd8009190 comp rx=0 tx=0).stop 2024-04-14T08:36:56.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.044+0000 7f5ded155640 1 -- 172.21.15.172:0/4141483429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de8140770 msgr2=0x7f5de8153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.044+0000 7f5ded155640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de8140770 0x7f5de8153d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0090b0 tx=0x7f5ddc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.044+0000 7f5ded155640 1 -- 172.21.15.172:0/4141483429 shutdown_connections 2024-04-14T08:36:56.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.044+0000 7f5ded155640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5dd406c8f0 0x7f5dd406edb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.044+0000 7f5ded155640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5de815c380 0x7f5de80aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.044+0000 7f5ded155640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de8140770 0x7f5de8153d40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.044+0000 7f5ded155640 1 --2- 172.21.15.172:0/4141483429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de813df30 0x7f5de8151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.044+0000 7f5ded155640 1 -- 172.21.15.172:0/4141483429 >> 172.21.15.172:0/4141483429 conn(0x7f5de806b520 msgr2=0x7f5de80b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:56.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.044+0000 7f5ded155640 1 -- 172.21.15.172:0/4141483429 shutdown_connections 2024-04-14T08:36:56.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.044+0000 7f5ded155640 1 -- 172.21.15.172:0/4141483429 wait complete. 2024-04-14T08:36:56.056 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:36:56.056 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:36:56.095 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:36:56.137 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:36:56.137 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:36:56.137 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:36:56.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:36:56.153 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:36:56.153 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:36:56.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:36:56.195 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:36:56.195 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:36:56.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:36:56.211 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:36:56.211 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:36:56.252 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:36:56.252 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-14T08:36:56.253 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:36:56.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.351+0000 7f567eeb7640 1 Processor -- start 2024-04-14T08:36:56.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.351+0000 7f567eeb7640 1 -- start start 2024-04-14T08:36:56.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.351+0000 7f567eeb7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f56780b1df0 0x7f56780b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.351+0000 7f567eeb7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780b4040 0x7f56780b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.351+0000 7f567eeb7640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56780b49f0 con 0x7f56780b21d0 2024-04-14T08:36:56.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.351+0000 7f567eeb7640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f567815c770 con 0x7f56780b1df0 2024-04-14T08:36:56.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.351+0000 7f567eeb7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f56780b3430 con 0x7f56780b4040 2024-04-14T08:36:56.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.351+0000 7f5677fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f56780b21d0 0x7f567815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:52318/0 (socket says 172.21.15.172:52318) 2024-04-14T08:36:56.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f5677fff640 1 -- 172.21.15.172:0/1650395477 learned_addr learned my addr 172.21.15.172:0/1650395477 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:56.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4011864771 0 0) 0x7f56780b49f0 con 0x7f56780b21d0 2024-04-14T08:36:56.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5668003520 con 0x7f56780b21d0 2024-04-14T08:36:56.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f567cc53640 1 --2- 172.21.15.172:0/1650395477 >> v2:172.21.15.177:3300/0 conn(0x7f56780b1df0 0x7f56780b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f567d454640 1 --2- 172.21.15.172:0/1650395477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780b4040 0x7f56780b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3036617246 0 0) 0x7f5668003520 con 0x7f56780b21d0 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780b4040 msgr2=0x7f56780b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 --2- 172.21.15.172:0/1650395477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780b4040 0x7f56780b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 >> v2:172.21.15.177:3300/0 conn(0x7f56780b1df0 msgr2=0x7f56780b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 --2- 172.21.15.172:0/1650395477 >> v2:172.21.15.177:3300/0 conn(0x7f56780b1df0 0x7f56780b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56780b3700 con 0x7f56780b21d0 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f5664001670 con 0x7f56780b21d0 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 >> v1:172.21.15.172:6789/0 conn(0x7f56780b21d0 legacy=0x7f567815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 --2- 172.21.15.172:0/1650395477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780b4040 0x7f56680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 --2- 172.21.15.172:0/1650395477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5668003f30 0x7f5668004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 --2- 172.21.15.172:0/1650395477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5668004d40 0x7f56680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f567815c770 con 0x7f5668004d40 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f56780b3430 con 0x7f5668003f30 2024-04-14T08:36:56.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f566800ba10 con 0x7f56780b4040 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f567d454640 1 --2- 172.21.15.172:0/1650395477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5668004d40 0x7f56680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f567cc53640 1 --2- 172.21.15.172:0/1650395477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780b4040 0x7f56680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.352+0000 7f5677fff640 1 --2- 172.21.15.172:0/1650395477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5668003f30 0x7f5668004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.353+0000 7f567d454640 1 -- 172.21.15.172:0/1650395477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780b4040 msgr2=0x7f56680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.353+0000 7f567d454640 1 --2- 172.21.15.172:0/1650395477 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780b4040 0x7f56680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.353+0000 7f567d454640 1 -- 172.21.15.172:0/1650395477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5668003f30 msgr2=0x7f5668004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.353+0000 7f567d454640 1 --2- 172.21.15.172:0/1650395477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5668003f30 0x7f5668004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.353+0000 7f567d454640 1 -- 172.21.15.172:0/1650395477 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5668003520 con 0x7f5668004d40 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.353+0000 7f567d454640 1 --2- 172.21.15.172:0/1650395477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5668004d40 0x7f56680051c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5670007360 tx=0x7f5668005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f15acc8f3d48ef server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.353+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670039070 con 0x7f5668004d40 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.353+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567002ccd0 con 0x7f5668004d40 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.353+0000 7f56777fe640 1 -- 172.21.15.172:0/1650395477 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670009be0 con 0x7f5668004d40 2024-04-14T08:36:56.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.354+0000 7f567eeb7640 1 -- 172.21.15.172:0/1650395477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5668004d40 msgr2=0x7f56680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.354+0000 7f567eeb7640 1 --2- 172.21.15.172:0/1650395477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5668004d40 0x7f56680051c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5670007360 tx=0x7f5668005830 comp rx=0 tx=0).stop 2024-04-14T08:36:56.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.354+0000 7f567d454640 1 -- 172.21.15.172:0/1650395477 reap_dead start 2024-04-14T08:36:56.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.354+0000 7f567eeb7640 1 -- 172.21.15.172:0/1650395477 shutdown_connections 2024-04-14T08:36:56.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.354+0000 7f567eeb7640 1 -- 172.21.15.172:0/1650395477 >> 172.21.15.172:0/1650395477 conn(0x7f567806ec30 msgr2=0x7f567806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:56.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.354+0000 7f567eeb7640 1 -- 172.21.15.172:0/1650395477 shutdown_connections 2024-04-14T08:36:56.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.354+0000 7f567eeb7640 1 -- 172.21.15.172:0/1650395477 wait complete. 2024-04-14T08:36:56.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.355+0000 7f567eeb7640 1 Processor -- start 2024-04-14T08:36:56.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.355+0000 7f567eeb7640 1 -- start start 2024-04-14T08:36:56.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f567eeb7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780ad8e0 0x7f5678142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f567eeb7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5678159d90 0x7f5678141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f567eeb7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56780adcc0 0x7f56780ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f567eeb7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f567815bb40 con 0x7f5678159d90 2024-04-14T08:36:56.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f567eeb7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f56780b3cc0 con 0x7f56780adcc0 2024-04-14T08:36:56.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f567eeb7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f567815b7f0 con 0x7f56780ad8e0 2024-04-14T08:36:56.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f5677fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5678159d90 0x7f5678141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f5677fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5678159d90 0x7f5678141550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39686/0 (socket says 172.21.15.172:39686) 2024-04-14T08:36:56.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f5677fff640 1 -- 172.21.15.172:0/3894957059 learned_addr learned my addr 172.21.15.172:0/3894957059 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:56.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f567d454640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56780adcc0 0x7f56780ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f567cc53640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780ad8e0 0x7f5678142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f5677fff640 1 -- 172.21.15.172:0/3894957059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780ad8e0 msgr2=0x7f5678142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f5677fff640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780ad8e0 0x7f5678142d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f5677fff640 1 -- 172.21.15.172:0/3894957059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56780adcc0 msgr2=0x7f56780ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f5677fff640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56780adcc0 0x7f56780ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.356+0000 7f5677fff640 1 -- 172.21.15.172:0/3894957059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5678141cc0 con 0x7f5678159d90 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.357+0000 7f5677fff640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5678159d90 0x7f5678141550 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f56640072c0 tx=0x7f5664007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.357+0000 7f56757fa640 1 -- 172.21.15.172:0/3894957059 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566400f9c0 con 0x7f5678159d90 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.357+0000 7f56757fa640 1 -- 172.21.15.172:0/3894957059 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5664018070 con 0x7f5678159d90 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.357+0000 7f567eeb7640 1 -- 172.21.15.172:0/3894957059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5678141fa0 con 0x7f5678159d90 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.357+0000 7f56757fa640 1 -- 172.21.15.172:0/3894957059 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566400fda0 con 0x7f5678159d90 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.357+0000 7f567eeb7640 1 -- 172.21.15.172:0/3894957059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5678142560 con 0x7f5678159d90 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.358+0000 7f567eeb7640 1 -- 172.21.15.172:0/3894957059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56780cbea0 con 0x7f5678159d90 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.358+0000 7f56757fa640 1 -- 172.21.15.172:0/3894957059 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f566401d460 con 0x7f5678159d90 2024-04-14T08:36:56.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.358+0000 7f56757fa640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f565c06c8f0 0x7f565c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.359+0000 7f56757fa640 1 -- 172.21.15.172:0/3894957059 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f566408bbf0 con 0x7f5678159d90 2024-04-14T08:36:56.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.359+0000 7f567cc53640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f565c06c8f0 0x7f565c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.359+0000 7f567cc53640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f565c06c8f0 0x7f565c06edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f566c009b80 tx=0x7f566c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:56.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.363+0000 7f56757fa640 1 -- 172.21.15.172:0/3894957059 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f56780cbea0 con 0x7f5678159d90 2024-04-14T08:36:56.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.573+0000 7f567eeb7640 1 -- 172.21.15.172:0/3894957059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5678068520 con 0x7f5678159d90 2024-04-14T08:36:56.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.573+0000 7f56757fa640 1 -- 172.21.15.172:0/3894957059 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+3881 (secure 0 0 0) 0x7f566405a190 con 0x7f5678159d90 2024-04-14T08:36:56.575 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:56.575 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2882141266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2882141266},{"type":"v1","addr":"172.21.15.177:6835","nonce":2882141266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3621780015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3621780015},{"type":"v1","addr":"172.21.15.172:6836","nonce":3621780015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":6189,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/3894683748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3894683748},{"type":"v1","addr":"172.21.15.172:6837","nonce":3894683748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3625745587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3625745587},{"type":"v1","addr":"172.21.15.177:6834","nonce":3625745587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[]} 2024-04-14T08:36:56.576 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 34 2024-04-14T08:36:56.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.575+0000 7f567eeb7640 1 -- 172.21.15.172:0/3894957059 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f565c06c8f0 msgr2=0x7f565c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.575+0000 7f567eeb7640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f565c06c8f0 0x7f565c06edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f566c009b80 tx=0x7f566c008040 comp rx=0 tx=0).stop 2024-04-14T08:36:56.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.575+0000 7f567eeb7640 1 -- 172.21.15.172:0/3894957059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5678159d90 msgr2=0x7f5678141550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.575+0000 7f567eeb7640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5678159d90 0x7f5678141550 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f56640072c0 tx=0x7f5664007780 comp rx=0 tx=0).stop 2024-04-14T08:36:56.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.575+0000 7f567eeb7640 1 -- 172.21.15.172:0/3894957059 shutdown_connections 2024-04-14T08:36:56.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.575+0000 7f567eeb7640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f565c06c8f0 0x7f565c06edb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.575+0000 7f567eeb7640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56780adcc0 0x7f56780ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.575+0000 7f567eeb7640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5678159d90 0x7f5678141550 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.575+0000 7f567eeb7640 1 --2- 172.21.15.172:0/3894957059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56780ad8e0 0x7f5678142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.575+0000 7f567eeb7640 1 -- 172.21.15.172:0/3894957059 >> 172.21.15.172:0/3894957059 conn(0x7f567806ec30 msgr2=0x7f56780cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:56.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.576+0000 7f567eeb7640 1 -- 172.21.15.172:0/3894957059 shutdown_connections 2024-04-14T08:36:56.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.576+0000 7f567eeb7640 1 -- 172.21.15.172:0/3894957059 wait complete. 2024-04-14T08:36:56.588 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-14T08:36:56.588 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-14T08:36:56.589 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-14T08:36:56.589 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:36:56.589 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-14T08:36:56.589 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-14T08:36:56.589 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-14T08:36:56.589 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:36:56.590 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-14T08:36:56.590 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-14T08:36:56.590 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-14T08:36:56.590 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:36:56.590 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-14T08:36:56.590 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-14T08:36:56.591 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-14T08:36:56.591 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:36:56.601 INFO:tasks.ceph.mds.a:Stopped 2024-04-14T08:36:56.601 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-14T08:36:56.602 INFO:tasks.ceph.mds.c:Stopped 2024-04-14T08:36:56.602 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-14T08:36:56.603 INFO:tasks.ceph.mds.b:Stopped 2024-04-14T08:36:56.603 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-14T08:36:56.604 INFO:tasks.ceph.mds.d:Stopped 2024-04-14T08:36:56.604 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-14T08:36:56.644 INFO:tasks.ceph.mds.a:Started 2024-04-14T08:36:56.644 DEBUG:teuthology.parallel:result is None 2024-04-14T08:36:56.645 INFO:tasks.ceph.mds.c:Started 2024-04-14T08:36:56.645 DEBUG:teuthology.parallel:result is None 2024-04-14T08:36:56.646 INFO:tasks.ceph.mds.b:Started 2024-04-14T08:36:56.647 DEBUG:teuthology.parallel:result is None 2024-04-14T08:36:56.647 INFO:tasks.ceph.mds.d:Started 2024-04-14T08:36:56.647 DEBUG:teuthology.parallel:result is None 2024-04-14T08:36:56.648 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-14T08:36:56.721 INFO:tasks.ceph.mds.a.smithi172.stdout:starting mds.a at 2024-04-14T08:36:56.721 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:36:56.734 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-04-14T08:36:56.735 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-04-14T08:36:56.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.752+0000 7f115fe0f640 1 Processor -- start 2024-04-14T08:36:56.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.752+0000 7f115fe0f640 1 -- start start 2024-04-14T08:36:56.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.753+0000 7f115fe0f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f115815e430 0x7f1158162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.753+0000 7f115fe0f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11580cefb0 con 0x7f1158162540 2024-04-14T08:36:56.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.753+0000 7f115fe0f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11580cf290 con 0x7f11580d2e00 2024-04-14T08:36:56.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.753+0000 7f115fe0f640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f11580cf570 con 0x7f115815e430 2024-04-14T08:36:56.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.753+0000 7f115dbab640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f115815e430 0x7f1158162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.753+0000 7f115d3aa640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1158162540 0x7f11580b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:52342/0 (socket says 172.21.15.172:52342) 2024-04-14T08:36:56.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.753+0000 7f115d3aa640 1 -- 172.21.15.172:0/492646866 learned_addr learned my addr 172.21.15.172:0/492646866 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:56.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.754+0000 7f115dbab640 1 -- 172.21.15.172:0/492646866 >> v1:172.21.15.177:6789/0 conn(0x7f11580d2e00 legacy=0x7f11580b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.754+0000 7f115dbab640 1 -- 172.21.15.172:0/492646866 >> v1:172.21.15.172:6789/0 conn(0x7f1158162540 legacy=0x7f11580b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.754+0000 7f115cba9640 1 -- 172.21.15.172:0/492646866 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3953558929 0 0) 0x7f11580cefb0 con 0x7f1158162540 2024-04-14T08:36:56.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.754+0000 7f115dbab640 1 -- 172.21.15.172:0/492646866 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11580cf850 con 0x7f115815e430 2024-04-14T08:36:56.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.754+0000 7f115dbab640 1 --2- 172.21.15.172:0/492646866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115815e430 0x7f1158162000 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f114c009720 tx=0x7f114c02b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdf5fc60497b902 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:56.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.755+0000 7f115cba9640 1 -- 172.21.15.172:0/492646866 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c037070 con 0x7f115815e430 2024-04-14T08:36:56.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.755+0000 7f115cba9640 1 -- 172.21.15.172:0/492646866 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f114c030710 con 0x7f115815e430 2024-04-14T08:36:56.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.755+0000 7f115cba9640 1 -- 172.21.15.172:0/492646866 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c030ae0 con 0x7f115815e430 2024-04-14T08:36:56.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.755+0000 7f115fe0f640 1 -- 172.21.15.172:0/492646866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115815e430 msgr2=0x7f1158162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.755+0000 7f115fe0f640 1 --2- 172.21.15.172:0/492646866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115815e430 0x7f1158162000 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f114c009720 tx=0x7f114c02b730 comp rx=0 tx=0).stop 2024-04-14T08:36:56.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.755+0000 7f115fe0f640 1 -- 172.21.15.172:0/492646866 shutdown_connections 2024-04-14T08:36:56.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.755+0000 7f115fe0f640 1 --2- 172.21.15.172:0/492646866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f115815e430 0x7f1158162000 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.755+0000 7f115fe0f640 1 -- 172.21.15.172:0/492646866 >> 172.21.15.172:0/492646866 conn(0x7f115809d560 msgr2=0x7f11580d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:56.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.755+0000 7f115fe0f640 1 -- 172.21.15.172:0/492646866 shutdown_connections 2024-04-14T08:36:56.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.755+0000 7f115fe0f640 1 -- 172.21.15.172:0/492646866 wait complete. 2024-04-14T08:36:56.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.756+0000 7f115fe0f640 1 Processor -- start 2024-04-14T08:36:56.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.756+0000 7f115fe0f640 1 -- start start 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.757+0000 7f115fe0f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11580d2e00 0x7f1158158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.757+0000 7f115fe0f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f115815e430 0x7f1158160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.757+0000 7f115fe0f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158162540 0x7f11581614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.757+0000 7f115fe0f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11580d4e20 con 0x7f1158162540 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.757+0000 7f115fe0f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f115815e810 con 0x7f11580d2e00 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.757+0000 7f115fe0f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11580b4850 con 0x7f115815e430 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.757+0000 7f115e3ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158162540 0x7f11581614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.757+0000 7f115e3ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158162540 0x7f11581614d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39738/0 (socket says 172.21.15.172:39738) 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.757+0000 7f115e3ac640 1 -- 172.21.15.172:0/3008823706 learned_addr learned my addr 172.21.15.172:0/3008823706 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.757+0000 7f115dbab640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11580d2e00 0x7f1158158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f115d3aa640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f115815e430 0x7f1158160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f115e3ac640 1 -- 172.21.15.172:0/3008823706 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f115815e430 msgr2=0x7f1158160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f115e3ac640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f115815e430 0x7f1158160f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f115e3ac640 1 -- 172.21.15.172:0/3008823706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11580d2e00 msgr2=0x7f1158158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:56.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f115e3ac640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11580d2e00 0x7f1158158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:56.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f115e3ac640 1 -- 172.21.15.172:0/3008823706 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11580b6d80 con 0x7f1158162540 2024-04-14T08:36:56.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f115dbab640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11580d2e00 0x7f1158158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:36:56.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f115e3ac640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158162540 0x7f11581614d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f115000c8c0 tx=0x7f115000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:56.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f114a7fc640 1 -- 172.21.15.172:0/3008823706 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150017070 con 0x7f1158162540 2024-04-14T08:36:56.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f114a7fc640 1 -- 172.21.15.172:0/3008823706 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1150003da0 con 0x7f1158162540 2024-04-14T08:36:56.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f114a7fc640 1 -- 172.21.15.172:0/3008823706 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150005290 con 0x7f1158162540 2024-04-14T08:36:56.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f115fe0f640 1 -- 172.21.15.172:0/3008823706 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11580b7060 con 0x7f1158162540 2024-04-14T08:36:56.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.758+0000 7f115fe0f640 1 -- 172.21.15.172:0/3008823706 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11580b7f10 con 0x7f1158162540 2024-04-14T08:36:56.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.759+0000 7f115fe0f640 1 -- 172.21.15.172:0/3008823706 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11580757d0 con 0x7f1158162540 2024-04-14T08:36:56.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.759+0000 7f114a7fc640 1 -- 172.21.15.172:0/3008823706 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f115001a070 con 0x7f1158162540 2024-04-14T08:36:56.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.760+0000 7f114a7fc640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f113c06c8f0 0x7f113c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:56.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.760+0000 7f114a7fc640 1 -- 172.21.15.172:0/3008823706 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f115008b120 con 0x7f1158162540 2024-04-14T08:36:56.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.760+0000 7f115dbab640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f113c06c8f0 0x7f113c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:56.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.760+0000 7f115dbab640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f113c06c8f0 0x7f113c06edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f114c034000 tx=0x7f114c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:56.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.764+0000 7f114a7fc640 1 -- 172.21.15.172:0/3008823706 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f11580757d0 con 0x7f1158162540 2024-04-14T08:36:56.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:56.950+0000 7f115fe0f640 1 -- 172.21.15.172:0/3008823706 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1158068590 con 0x7f1158162540 2024-04-14T08:36:57.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.736+0000 7f114a7fc640 1 -- 172.21.15.172:0/3008823706 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) ==== 97+0+0 (secure 0 0 0) 0x7f1150059750 con 0x7f1158162540 2024-04-14T08:36:57.738 INFO:teuthology.orchestra.run.smithi172.stderr: removed all blocklist entries 2024-04-14T08:36:57.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.739+0000 7f115fe0f640 1 -- 172.21.15.172:0/3008823706 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f113c06c8f0 msgr2=0x7f113c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:57.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.739+0000 7f115fe0f640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f113c06c8f0 0x7f113c06edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f114c034000 tx=0x7f114c008030 comp rx=0 tx=0).stop 2024-04-14T08:36:57.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.739+0000 7f115fe0f640 1 -- 172.21.15.172:0/3008823706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158162540 msgr2=0x7f11581614d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:57.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.739+0000 7f115fe0f640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158162540 0x7f11581614d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f115000c8c0 tx=0x7f115000cd80 comp rx=0 tx=0).stop 2024-04-14T08:36:57.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.739+0000 7f115fe0f640 1 -- 172.21.15.172:0/3008823706 shutdown_connections 2024-04-14T08:36:57.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.739+0000 7f115fe0f640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f113c06c8f0 0x7f113c06edb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:57.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.739+0000 7f115fe0f640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158162540 0x7f11581614d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:57.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.739+0000 7f115fe0f640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f115815e430 0x7f1158160f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:57.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.739+0000 7f115fe0f640 1 --2- 172.21.15.172:0/3008823706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11580d2e00 0x7f1158158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:57.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.739+0000 7f115fe0f640 1 -- 172.21.15.172:0/3008823706 >> 172.21.15.172:0/3008823706 conn(0x7f115809d560 msgr2=0x7f11580d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:57.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.739+0000 7f115fe0f640 1 -- 172.21.15.172:0/3008823706 shutdown_connections 2024-04-14T08:36:57.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.739+0000 7f115fe0f640 1 -- 172.21.15.172:0/3008823706 wait complete. 2024-04-14T08:36:57.753 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-14T08:36:57.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.856+0000 7fe3496d1640 1 Processor -- start 2024-04-14T08:36:57.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.856+0000 7fe3496d1640 1 -- start start 2024-04-14T08:36:57.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.857+0000 7fe3496d1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe3440d1a70 0x7fe3440b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:57.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.857+0000 7fe3496d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3440b4260 0x7fe34406c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:57.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.857+0000 7fe3496d1640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3440d1ed0 con 0x7fe34415e390 2024-04-14T08:36:57.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.857+0000 7fe3496d1640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe3440b46d0 con 0x7fe3440d1a70 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.857+0000 7fe3496d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe3440d0080 con 0x7fe3440b4260 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.857+0000 7fe33bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe3440d1a70 0x7fe3440b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.857+0000 7fe342ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe34415e390 0x7fe34415f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:52358/0 (socket says 172.21.15.172:52358) 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.857+0000 7fe342ffd640 1 -- 172.21.15.172:0/626956569 learned_addr learned my addr 172.21.15.172:0/626956569 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.857+0000 7fe33bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe3440d1a70 0x7fe3440b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39742/0 (socket says 172.21.15.172:39742) 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.858+0000 7fe3437fe640 1 --2- 172.21.15.172:0/626956569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3440b4260 0x7fe34406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.858+0000 7fe33bfff640 1 -- 172.21.15.172:0/626956569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3440b4260 msgr2=0x7fe34406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.858+0000 7fe33bfff640 1 --2- 172.21.15.172:0/626956569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3440b4260 0x7fe34406c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.858+0000 7fe33bfff640 1 -- 172.21.15.172:0/626956569 >> v1:172.21.15.172:6789/0 conn(0x7fe34415e390 legacy=0x7fe34415f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.858+0000 7fe3427fc640 1 -- 172.21.15.172:0/626956569 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1222291391 0 0) 0x7fe334001110 con 0x7fe34415e390 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.858+0000 7fe33bfff640 1 -- 172.21.15.172:0/626956569 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34415e770 con 0x7fe3440d1a70 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.858+0000 7fe3437fe640 1 --2- 172.21.15.172:0/626956569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3440b4260 0x7fe34406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.858+0000 7fe33bfff640 1 --2- 172.21.15.172:0/626956569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3440d1a70 0x7fe3440b3d20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe32c0097a0 tx=0x7fe32c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade926ad9432b63b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:57.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.859+0000 7fe3427fc640 1 -- 172.21.15.172:0/626956569 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0315a0 con 0x7fe3440d1a70 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.859+0000 7fe3427fc640 1 -- 172.21.15.172:0/626956569 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32c037070 con 0x7fe3440d1a70 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.859+0000 7fe3427fc640 1 -- 172.21.15.172:0/626956569 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c031960 con 0x7fe3440d1a70 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.859+0000 7fe3496d1640 1 -- 172.21.15.172:0/626956569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3440d1a70 msgr2=0x7fe3440b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.859+0000 7fe3496d1640 1 --2- 172.21.15.172:0/626956569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3440d1a70 0x7fe3440b3d20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe32c0097a0 tx=0x7fe32c02b840 comp rx=0 tx=0).stop 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.859+0000 7fe3496d1640 1 -- 172.21.15.172:0/626956569 shutdown_connections 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.859+0000 7fe3496d1640 1 --2- 172.21.15.172:0/626956569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3440b4260 0x7fe34406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.859+0000 7fe3496d1640 1 --2- 172.21.15.172:0/626956569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3440d1a70 0x7fe3440b3d20 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.859+0000 7fe3496d1640 1 -- 172.21.15.172:0/626956569 >> 172.21.15.172:0/626956569 conn(0x7fe34406f820 msgr2=0x7fe34406fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.859+0000 7fe3496d1640 1 -- 172.21.15.172:0/626956569 shutdown_connections 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.859+0000 7fe3496d1640 1 -- 172.21.15.172:0/626956569 wait complete. 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.860+0000 7fe3496d1640 1 Processor -- start 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.860+0000 7fe3496d1640 1 -- start start 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3496d1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3440b4260 0x7fe3441f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3496d1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3440d1a70 0x7fe3440b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:57.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3496d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe34415e390 0x7fe3440b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3496d1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3440d0510 con 0x7fe34415e390 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3496d1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe34406de80 con 0x7fe3440d1a70 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3496d1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe34406cb70 con 0x7fe3440b4260 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3437fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe34415e390 0x7fe3440b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3437fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe34415e390 0x7fe3440b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39754/0 (socket says 172.21.15.172:39754) 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3437fe640 1 -- 172.21.15.172:0/1692014496 learned_addr learned my addr 172.21.15.172:0/1692014496 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe342ffd640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3440b4260 0x7fe3441f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe33bfff640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3440d1a70 0x7fe3440b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3437fe640 1 -- 172.21.15.172:0/1692014496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3440b4260 msgr2=0x7fe3441f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3437fe640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3440b4260 0x7fe3441f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3437fe640 1 -- 172.21.15.172:0/1692014496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3440d1a70 msgr2=0x7fe3440b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3437fe640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3440d1a70 0x7fe3440b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:57.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.861+0000 7fe3437fe640 1 -- 172.21.15.172:0/1692014496 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3441f78e0 con 0x7fe34415e390 2024-04-14T08:36:57.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.862+0000 7fe3437fe640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe34415e390 0x7fe3440b3220 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00cef0 tx=0x7fe33c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:57.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.862+0000 7fe33b7fe640 1 -- 172.21.15.172:0/1692014496 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c01a070 con 0x7fe34415e390 2024-04-14T08:36:57.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.862+0000 7fe33b7fe640 1 -- 172.21.15.172:0/1692014496 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe33c00d860 con 0x7fe34415e390 2024-04-14T08:36:57.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.862+0000 7fe3496d1640 1 -- 172.21.15.172:0/1692014496 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe344156670 con 0x7fe34415e390 2024-04-14T08:36:57.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.862+0000 7fe33b7fe640 1 -- 172.21.15.172:0/1692014496 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c00dc50 con 0x7fe34415e390 2024-04-14T08:36:57.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.862+0000 7fe3496d1640 1 -- 172.21.15.172:0/1692014496 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe344156c30 con 0x7fe34415e390 2024-04-14T08:36:57.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.863+0000 7fe3496d1640 1 -- 172.21.15.172:0/1692014496 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe344068520 con 0x7fe34415e390 2024-04-14T08:36:57.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.863+0000 7fe33b7fe640 1 -- 172.21.15.172:0/1692014496 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe33c01d070 con 0x7fe34415e390 2024-04-14T08:36:57.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.864+0000 7fe33b7fe640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe32406c8f0 0x7fe32406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:57.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.864+0000 7fe33b7fe640 1 -- 172.21.15.172:0/1692014496 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7fe33c08c260 con 0x7fe34415e390 2024-04-14T08:36:57.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.864+0000 7fe342ffd640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe32406c8f0 0x7fe32406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:57.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.864+0000 7fe342ffd640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe32406c8f0 0x7fe32406edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe334000f60 tx=0x7fe334007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:57.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:57.868+0000 7fe33b7fe640 1 -- 172.21.15.172:0/1692014496 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe33c05a910 con 0x7fe34415e390 2024-04-14T08:36:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.110+0000 7fe3496d1640 1 -- 172.21.15.172:0/1692014496 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe344159900 con 0x7fe34415e390 2024-04-14T08:36:58.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.110+0000 7fe33b7fe640 1 -- 172.21.15.172:0/1692014496 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7fe33c05a210 con 0x7fe34415e390 2024-04-14T08:36:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:58.112 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:36:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: "auth_dump": [ 2024-04-14T08:36:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.a", 2024-04-14T08:36:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmnNCRDRAAz7aU3JD7iiaTQH0cNotfaw==", 2024-04-14T08:36:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:36:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:36:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:36:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.b", 2024-04-14T08:36:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtm1on4GRAAms1he+O4jWu01z0ZLLgL0A==", 2024-04-14T08:36:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.c", 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmb7z0FBAAGVu3+TWshDGvd7/fLbgJ5A==", 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:36:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.d", 2024-04-14T08:36:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmEEx0IRAAXiFiOK1RE/0bfyu7KI/3ng==", 2024-04-14T08:36:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:36:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:36:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:36:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.0", 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB/lBtmnsWuCRAAqcuQhsrzpGVfrlgwTnFsOw==", 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.1", 2024-04-14T08:36:58.116 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCClBtme9lPBxAAFB1E3g8qwZxPCdHiFogp5Q==", 2024-04-14T08:36:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:36:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:36:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.2", 2024-04-14T08:36:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCFlBtm966YBRAAWXZGq/0TuUMZTsyeR4xWww==", 2024-04-14T08:36:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:36:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:36:58.118 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.118 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.118 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.118 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.118 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.3", 2024-04-14T08:36:58.118 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCIlBtmHpktBBAAUbddNHESwyNvvp8UE/SaCw==", 2024-04-14T08:36:58.118 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.118 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:36:58.118 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:36:58.118 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.118 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.4", 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCOlBtm+//7ChAA7rApbKll2vBGcDlW8q+kyg==", 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.5", 2024-04-14T08:36:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCRlBtmVU4YChAAysdUosQ+TnHtjnRl1FOISA==", 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.6", 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCUlBtmnE7+BxAAqBn8YMwecA2NcZz63eBm7Q==", 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.120 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.7", 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCXlBtmN8DhBRAA6CHQ943t6qJCL77jcbOv4w==", 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.0", 2024-04-14T08:36:58.121 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA==", 2024-04-14T08:36:58.122 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.122 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:36:58.122 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow r", 2024-04-14T08:36:58.122 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-14T08:36:58.122 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.122 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.122 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.122 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.1", 2024-04-14T08:36:58.122 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmp4nkLhAAhYKm8VcykVbPSohvK11I9w==", 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.2", 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmJoV9KxAAp6zlRX+gvhASQBHjpF9ewQ==", 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:36:58.123 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.3", 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmeC+pNBAASi7W4jlo6imMMzNEwLrPDA==", 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.124 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.admin", 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB6lBtmgnWqLBAAQ0fgarM8au3zFuMNnQIvVw==", 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.x", 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmm5IbChAAeVcVs3Zt6+lU0wghjCcwnQ==", 2024-04-14T08:36:58.125 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.y", 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmfddYBhAAh8zgf4W3WWmNYWNAA7WuRw==", 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:36:58.126 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.127 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:58.127 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-04-14T08:36:58.127 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:36:58.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.112+0000 7fe3496d1640 1 -- 172.21.15.172:0/1692014496 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe32406c8f0 msgr2=0x7fe32406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.112+0000 7fe3496d1640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe32406c8f0 0x7fe32406edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe334000f60 tx=0x7fe334007c00 comp rx=0 tx=0).stop 2024-04-14T08:36:58.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.112+0000 7fe3496d1640 1 -- 172.21.15.172:0/1692014496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe34415e390 msgr2=0x7fe3440b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.112+0000 7fe3496d1640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe34415e390 0x7fe3440b3220 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00cef0 tx=0x7fe33c00a710 comp rx=0 tx=0).stop 2024-04-14T08:36:58.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.112+0000 7fe3496d1640 1 -- 172.21.15.172:0/1692014496 shutdown_connections 2024-04-14T08:36:58.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.112+0000 7fe3496d1640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe32406c8f0 0x7fe32406edb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.112+0000 7fe3496d1640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe34415e390 0x7fe3440b3220 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.112+0000 7fe3496d1640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3440d1a70 0x7fe3440b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.112+0000 7fe3496d1640 1 --2- 172.21.15.172:0/1692014496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe3440b4260 0x7fe3441f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.112+0000 7fe3496d1640 1 -- 172.21.15.172:0/1692014496 >> 172.21.15.172:0/1692014496 conn(0x7fe34406f820 msgr2=0x7fe344153a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:58.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.112+0000 7fe3496d1640 1 -- 172.21.15.172:0/1692014496 shutdown_connections 2024-04-14T08:36:58.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.112+0000 7fe3496d1640 1 -- 172.21.15.172:0/1692014496 wait complete. 2024-04-14T08:36:58.130 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:36:58.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.229+0000 7f1babc1c640 1 Processor -- start 2024-04-14T08:36:58.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.230+0000 7f1babc1c640 1 -- start start 2024-04-14T08:36:58.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.230+0000 7f1babc1c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ba415c150 0x7f1ba40d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.230+0000 7f1babc1c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1ba40d1840 0x7f1ba4161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.230+0000 7f1babc1c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba40d0200 con 0x7f1ba4162530 2024-04-14T08:36:58.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.230+0000 7f1babc1c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba40d1cb0 con 0x7f1ba415c150 2024-04-14T08:36:58.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.230+0000 7f1babc1c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba4162980 con 0x7f1ba40d1840 2024-04-14T08:36:58.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.231+0000 7f1ba99b8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ba415c150 0x7f1ba40d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.231+0000 7f1ba99b8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ba415c150 0x7f1ba40d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39760/0 (socket says 172.21.15.172:39760) 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.231+0000 7f1ba99b8640 1 -- 172.21.15.172:0/279604154 learned_addr learned my addr 172.21.15.172:0/279604154 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.231+0000 7f1ba91b7640 1 --2- 172.21.15.172:0/279604154 >> v2:172.21.15.177:3300/0 conn(0x7f1ba40d1840 0x7f1ba4161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.231+0000 7f1ba99b8640 1 -- 172.21.15.172:0/279604154 >> v2:172.21.15.177:3300/0 conn(0x7f1ba40d1840 msgr2=0x7f1ba4161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.231+0000 7f1ba99b8640 1 --2- 172.21.15.172:0/279604154 >> v2:172.21.15.177:3300/0 conn(0x7f1ba40d1840 0x7f1ba4161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.231+0000 7f1ba99b8640 1 -- 172.21.15.172:0/279604154 >> v1:172.21.15.177:6789/0 conn(0x7f1ba4162530 legacy=0x7f1ba415ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.231+0000 7f1ba99b8640 1 -- 172.21.15.172:0/279604154 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba40cf160 con 0x7f1ba415c150 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.232+0000 7f1ba99b8640 1 --2- 172.21.15.172:0/279604154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ba415c150 0x7f1ba40d1300 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40ce890 tx=0x7f1b9402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66554e19bba15115 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.232+0000 7f1ba89b6640 1 -- 172.21.15.172:0/279604154 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94034070 con 0x7f1ba415c150 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.232+0000 7f1ba89b6640 1 -- 172.21.15.172:0/279604154 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b94030860 con 0x7f1ba415c150 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.232+0000 7f1ba89b6640 1 -- 172.21.15.172:0/279604154 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94030c30 con 0x7f1ba415c150 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.232+0000 7f1babc1c640 1 -- 172.21.15.172:0/279604154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ba415c150 msgr2=0x7f1ba40d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.232+0000 7f1babc1c640 1 --2- 172.21.15.172:0/279604154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ba415c150 0x7f1ba40d1300 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40ce890 tx=0x7f1b9402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.232+0000 7f1babc1c640 1 -- 172.21.15.172:0/279604154 shutdown_connections 2024-04-14T08:36:58.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.232+0000 7f1babc1c640 1 --2- 172.21.15.172:0/279604154 >> v2:172.21.15.177:3300/0 conn(0x7f1ba40d1840 0x7f1ba4161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.232+0000 7f1babc1c640 1 --2- 172.21.15.172:0/279604154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ba415c150 0x7f1ba40d1300 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.232+0000 7f1babc1c640 1 -- 172.21.15.172:0/279604154 >> 172.21.15.172:0/279604154 conn(0x7f1ba409d580 msgr2=0x7f1ba40d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:58.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.232+0000 7f1babc1c640 1 -- 172.21.15.172:0/279604154 shutdown_connections 2024-04-14T08:36:58.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.232+0000 7f1babc1c640 1 -- 172.21.15.172:0/279604154 wait complete. 2024-04-14T08:36:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.234+0000 7f1babc1c640 1 Processor -- start 2024-04-14T08:36:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.234+0000 7f1babc1c640 1 -- start start 2024-04-14T08:36:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.234+0000 7f1babc1c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ba40d1840 0x7f1ba415efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.234+0000 7f1babc1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ba415c150 0x7f1ba41f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.234+0000 7f1babc1c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ba4162530 0x7f1ba41f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.234+0000 7f1babc1c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba40cf720 con 0x7f1ba415c150 2024-04-14T08:36:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.234+0000 7f1babc1c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba40cef10 con 0x7f1ba4162530 2024-04-14T08:36:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.234+0000 7f1babc1c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba40cfa00 con 0x7f1ba40d1840 2024-04-14T08:36:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.234+0000 7f1ba91b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ba415c150 0x7f1ba41f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.234+0000 7f1ba91b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ba415c150 0x7f1ba41f3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39776/0 (socket says 172.21.15.172:39776) 2024-04-14T08:36:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.234+0000 7f1ba91b7640 1 -- 172.21.15.172:0/2822193953 learned_addr learned my addr 172.21.15.172:0/2822193953 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1ba99b8640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ba40d1840 0x7f1ba415efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1baa1b9640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ba4162530 0x7f1ba41f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1ba91b7640 1 -- 172.21.15.172:0/2822193953 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ba40d1840 msgr2=0x7f1ba415efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1ba91b7640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ba40d1840 0x7f1ba415efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1ba91b7640 1 -- 172.21.15.172:0/2822193953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ba4162530 msgr2=0x7f1ba41f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1ba91b7640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ba4162530 0x7f1ba41f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1ba91b7640 1 -- 172.21.15.172:0/2822193953 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba41f83e0 con 0x7f1ba415c150 2024-04-14T08:36:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1ba91b7640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ba415c150 0x7f1ba41f3700 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000e9e0 tx=0x7f1ba000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1b9a7fc640 1 -- 172.21.15.172:0/2822193953 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0017070 con 0x7f1ba415c150 2024-04-14T08:36:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1b9a7fc640 1 -- 172.21.15.172:0/2822193953 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba000c970 con 0x7f1ba415c150 2024-04-14T08:36:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1babc1c640 1 -- 172.21.15.172:0/2822193953 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba41f8100 con 0x7f1ba415c150 2024-04-14T08:36:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1b9a7fc640 1 -- 172.21.15.172:0/2822193953 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000cd60 con 0x7f1ba415c150 2024-04-14T08:36:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.235+0000 7f1babc1c640 1 -- 172.21.15.172:0/2822193953 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba415e7d0 con 0x7f1ba415c150 2024-04-14T08:36:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.236+0000 7f1babc1c640 1 -- 172.21.15.172:0/2822193953 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba4075490 con 0x7f1ba415c150 2024-04-14T08:36:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.236+0000 7f1b9a7fc640 1 -- 172.21.15.172:0/2822193953 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ba001a070 con 0x7f1ba415c150 2024-04-14T08:36:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.237+0000 7f1b9a7fc640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1b8806c8f0 0x7f1b8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.237+0000 7f1b9a7fc640 1 -- 172.21.15.172:0/2822193953 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7f1ba008b870 con 0x7f1ba415c150 2024-04-14T08:36:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.237+0000 7f1ba99b8640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1b8806c8f0 0x7f1b8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.237+0000 7f1ba99b8640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1b8806c8f0 0x7f1b8806edb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1b94036000 tx=0x7f1b9403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:58.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.241+0000 7f1b9a7fc640 1 -- 172.21.15.172:0/2822193953 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1ba0059ef0 con 0x7f1ba415c150 2024-04-14T08:36:58.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.419+0000 7f1babc1c640 1 -- 172.21.15.172:0/2822193953 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ba40b9e30 con 0x7f1ba415c150 2024-04-14T08:36:58.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.420+0000 7f1b9a7fc640 1 -- 172.21.15.172:0/2822193953 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+13225 (secure 0 0 0) 0x7f1ba00600c0 con 0x7f1ba415c150 2024-04-14T08:36:58.421 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:58.421 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":70,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:36:57.750997+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:36:58.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.422+0000 7f1babc1c640 1 -- 172.21.15.172:0/2822193953 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1b8806c8f0 msgr2=0x7f1b8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.422+0000 7f1babc1c640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1b8806c8f0 0x7f1b8806edb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1b94036000 tx=0x7f1b9403b040 comp rx=0 tx=0).stop 2024-04-14T08:36:58.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.422+0000 7f1babc1c640 1 -- 172.21.15.172:0/2822193953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ba415c150 msgr2=0x7f1ba41f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.422+0000 7f1babc1c640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ba415c150 0x7f1ba41f3700 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000e9e0 tx=0x7f1ba000eea0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.422+0000 7f1babc1c640 1 -- 172.21.15.172:0/2822193953 shutdown_connections 2024-04-14T08:36:58.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.422+0000 7f1babc1c640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1b8806c8f0 0x7f1b8806edb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.422+0000 7f1babc1c640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ba4162530 0x7f1ba41f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.422+0000 7f1babc1c640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ba415c150 0x7f1ba41f3700 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.422+0000 7f1babc1c640 1 --2- 172.21.15.172:0/2822193953 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ba40d1840 0x7f1ba415efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.422+0000 7f1babc1c640 1 -- 172.21.15.172:0/2822193953 >> 172.21.15.172:0/2822193953 conn(0x7f1ba409d580 msgr2=0x7f1ba40d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:58.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.422+0000 7f1babc1c640 1 -- 172.21.15.172:0/2822193953 shutdown_connections 2024-04-14T08:36:58.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.422+0000 7f1babc1c640 1 -- 172.21.15.172:0/2822193953 wait complete. 2024-04-14T08:36:58.435 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:36:58.435 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:36:58.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.537+0000 7f92bd9c8640 1 Processor -- start 2024-04-14T08:36:58.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.537+0000 7f92bd9c8640 1 -- start start 2024-04-14T08:36:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92bd9c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b80d0890 0x7f92b80b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92bd9c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f92b80cfc70 0x7f92b80af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92bd9c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b80aee60 con 0x7f92b815c550 2024-04-14T08:36:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92bd9c8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f92b80cd960 con 0x7f92b80cfc70 2024-04-14T08:36:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92bd9c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f92b80cdc40 con 0x7f92b80d0890 2024-04-14T08:36:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92b77fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f92b80cfc70 0x7f92b80af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92b6ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f92b815c550 0x7f92b80d0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:52364/0 (socket says 172.21.15.172:52364) 2024-04-14T08:36:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92b6ffd640 1 -- 172.21.15.172:0/1987107176 learned_addr learned my addr 172.21.15.172:0/1987107176 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92b67fc640 1 --2- 172.21.15.172:0/1987107176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b80d0890 0x7f92b80b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92b77fe640 1 -- 172.21.15.172:0/1987107176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b80d0890 msgr2=0x7f92b80b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92b77fe640 1 --2- 172.21.15.172:0/1987107176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b80d0890 0x7f92b80b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92b77fe640 1 -- 172.21.15.172:0/1987107176 >> v1:172.21.15.172:6789/0 conn(0x7f92b815c550 legacy=0x7f92b80d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92b77fe640 1 -- 172.21.15.172:0/1987107176 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b80cdf20 con 0x7f92b80cfc70 2024-04-14T08:36:58.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.538+0000 7f92b5ffb640 1 -- 172.21.15.172:0/1987107176 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3450347411 0 0) 0x7f92b80aee60 con 0x7f92b815c550 2024-04-14T08:36:58.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.539+0000 7f92b77fe640 1 --2- 172.21.15.172:0/1987107176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b80cfc70 0x7f92b80af620 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009d90 tx=0x7f92b002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e347ad2b634971 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:58.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.539+0000 7f92b5ffb640 1 -- 172.21.15.172:0/1987107176 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0039070 con 0x7f92b80cfc70 2024-04-14T08:36:58.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.539+0000 7f92b5ffb640 1 -- 172.21.15.172:0/1987107176 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b002cc50 con 0x7f92b80cfc70 2024-04-14T08:36:58.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.539+0000 7f92b5ffb640 1 -- 172.21.15.172:0/1987107176 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0034440 con 0x7f92b80cfc70 2024-04-14T08:36:58.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.540+0000 7f92bd9c8640 1 -- 172.21.15.172:0/1987107176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b80cfc70 msgr2=0x7f92b80af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.540+0000 7f92bd9c8640 1 --2- 172.21.15.172:0/1987107176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b80cfc70 0x7f92b80af620 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009d90 tx=0x7f92b002bea0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.540+0000 7f92bd9c8640 1 -- 172.21.15.172:0/1987107176 shutdown_connections 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.540+0000 7f92bd9c8640 1 --2- 172.21.15.172:0/1987107176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b80d0890 0x7f92b80b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.540+0000 7f92bd9c8640 1 --2- 172.21.15.172:0/1987107176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b80cfc70 0x7f92b80af620 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.540+0000 7f92bd9c8640 1 -- 172.21.15.172:0/1987107176 >> 172.21.15.172:0/1987107176 conn(0x7f92b80b3b00 msgr2=0x7f92b80b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.540+0000 7f92bd9c8640 1 -- 172.21.15.172:0/1987107176 shutdown_connections 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.540+0000 7f92bd9c8640 1 -- 172.21.15.172:0/1987107176 wait complete. 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.541+0000 7f92bd9c8640 1 Processor -- start 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.541+0000 7f92bd9c8640 1 -- start start 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92bd9c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b80cfc70 0x7f92b81f8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92bd9c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b80d0890 0x7f92b80c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92bd9c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92b815c550 0x7f92b806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92bd9c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80afce0 con 0x7f92b80d0890 2024-04-14T08:36:58.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92bd9c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80afb60 con 0x7f92b815c550 2024-04-14T08:36:58.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92bd9c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f92b80afe60 con 0x7f92b80cfc70 2024-04-14T08:36:58.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92b67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b80d0890 0x7f92b80c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92b67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b80d0890 0x7f92b80c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39802/0 (socket says 172.21.15.172:39802) 2024-04-14T08:36:58.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92b67fc640 1 -- 172.21.15.172:0/3582658444 learned_addr learned my addr 172.21.15.172:0/3582658444 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:58.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92b77fe640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92b815c550 0x7f92b806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92b6ffd640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b80cfc70 0x7f92b81f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92b67fc640 1 -- 172.21.15.172:0/3582658444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b80cfc70 msgr2=0x7f92b81f8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92b67fc640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b80cfc70 0x7f92b81f8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92b67fc640 1 -- 172.21.15.172:0/3582658444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92b815c550 msgr2=0x7f92b806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92b67fc640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92b815c550 0x7f92b806c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92b67fc640 1 -- 172.21.15.172:0/3582658444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b806d010 con 0x7f92b80d0890 2024-04-14T08:36:58.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92b77fe640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92b815c550 0x7f92b806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:36:58.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.542+0000 7f92b67fc640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b80d0890 0x7f92b80c7d60 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00c820 tx=0x7f92ac00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:58.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.543+0000 7f929ffff640 1 -- 172.21.15.172:0/3582658444 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac015070 con 0x7f92b80d0890 2024-04-14T08:36:58.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.543+0000 7f929ffff640 1 -- 172.21.15.172:0/3582658444 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ac004240 con 0x7f92b80d0890 2024-04-14T08:36:58.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.543+0000 7f929ffff640 1 -- 172.21.15.172:0/3582658444 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00ce10 con 0x7f92b80d0890 2024-04-14T08:36:58.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.543+0000 7f92bd9c8640 1 -- 172.21.15.172:0/3582658444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b806d2f0 con 0x7f92b80d0890 2024-04-14T08:36:58.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.543+0000 7f92bd9c8640 1 -- 172.21.15.172:0/3582658444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b81f7ff0 con 0x7f92b80d0890 2024-04-14T08:36:58.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.544+0000 7f92bd9c8640 1 -- 172.21.15.172:0/3582658444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b80af620 con 0x7f92b80d0890 2024-04-14T08:36:58.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.544+0000 7f929ffff640 1 -- 172.21.15.172:0/3582658444 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92ac007830 con 0x7f92b80d0890 2024-04-14T08:36:58.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.544+0000 7f929ffff640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f92a406c8f0 0x7f92a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.544+0000 7f929ffff640 1 -- 172.21.15.172:0/3582658444 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7f92ac08b880 con 0x7f92b80d0890 2024-04-14T08:36:58.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.544+0000 7f92b6ffd640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f92a406c8f0 0x7f92a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.545+0000 7f92b6ffd640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f92a406c8f0 0x7f92a406edb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f92a8000f60 tx=0x7f92a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:58.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.549+0000 7f929ffff640 1 -- 172.21.15.172:0/3582658444 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f92b80af620 con 0x7f92b80d0890 2024-04-14T08:36:58.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.723+0000 7f92bd9c8640 1 -- 172.21.15.172:0/3582658444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92b8071360 con 0x7f92b80d0890 2024-04-14T08:36:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.725+0000 7f929ffff640 1 -- 172.21.15.172:0/3582658444 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f92b8071360 con 0x7f92b80d0890 2024-04-14T08:36:58.727 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:36:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.728+0000 7f92bd9c8640 1 -- 172.21.15.172:0/3582658444 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f92a406c8f0 msgr2=0x7f92a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.728+0000 7f92bd9c8640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f92a406c8f0 0x7f92a406edb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f92a8000f60 tx=0x7f92a8009040 comp rx=0 tx=0).stop 2024-04-14T08:36:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.728+0000 7f92bd9c8640 1 -- 172.21.15.172:0/3582658444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b80d0890 msgr2=0x7f92b80c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.728+0000 7f92bd9c8640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b80d0890 0x7f92b80c7d60 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00c820 tx=0x7f92ac00cce0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.728+0000 7f92bd9c8640 1 -- 172.21.15.172:0/3582658444 shutdown_connections 2024-04-14T08:36:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.728+0000 7f92bd9c8640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f92a406c8f0 0x7f92a406edb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.728+0000 7f92bd9c8640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92b815c550 0x7f92b806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.728+0000 7f92bd9c8640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92b80d0890 0x7f92b80c7d60 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.728+0000 7f92bd9c8640 1 --2- 172.21.15.172:0/3582658444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b80cfc70 0x7f92b81f8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.728+0000 7f92bd9c8640 1 -- 172.21.15.172:0/3582658444 >> 172.21.15.172:0/3582658444 conn(0x7f92b80b3b00 msgr2=0x7f92b80b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.728+0000 7f92bd9c8640 1 -- 172.21.15.172:0/3582658444 shutdown_connections 2024-04-14T08:36:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.728+0000 7f92bd9c8640 1 -- 172.21.15.172:0/3582658444 wait complete. 2024-04-14T08:36:58.741 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-14T08:36:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.841+0000 7ff5f2c0f640 1 Processor -- start 2024-04-14T08:36:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.841+0000 7ff5f2c0f640 1 -- start start 2024-04-14T08:36:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.842+0000 7ff5f2c0f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5ec144a80 0x7ff5ec0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.842+0000 7ff5f2c0f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5ec144e60 con 0x7ff5ec15ece0 2024-04-14T08:36:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.842+0000 7ff5f2c0f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5ec15f730 con 0x7ff5ec0b2290 2024-04-14T08:36:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.842+0000 7ff5f2c0f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec15fa10 con 0x7ff5ec144a80 2024-04-14T08:36:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.842+0000 7ff5ebfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff5ec15ece0 0x7ff5ec0b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:52376/0 (socket says 172.21.15.172:52376) 2024-04-14T08:36:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.842+0000 7ff5ebfff640 1 -- 172.21.15.172:0/440077664 learned_addr learned my addr 172.21.15.172:0/440077664 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5f09ab640 1 --2- 172.21.15.172:0/440077664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5ec144a80 0x7ff5ec0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1187511257 0 0) 0x7ff5ec144e60 con 0x7ff5ec15ece0 2024-04-14T08:36:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5dc003520 con 0x7ff5ec15ece0 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3082611306 0 0) 0x7ff5dc003520 con 0x7ff5ec15ece0 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5ec144a80 msgr2=0x7ff5ec0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 --2- 172.21.15.172:0/440077664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5ec144a80 0x7ff5ec0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 >> v1:172.21.15.177:6789/0 conn(0x7ff5ec0b2290 legacy=0x7ff5ec0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec0b3100 con 0x7ff5ec15ece0 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7ff5d8001670 con 0x7ff5ec15ece0 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 >> v1:172.21.15.172:6789/0 conn(0x7ff5ec15ece0 legacy=0x7ff5ec0b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 --2- 172.21.15.172:0/440077664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5dc0039e0 0x7ff5dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 --2- 172.21.15.172:0/440077664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5dc004800 0x7ff5dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 --2- 172.21.15.172:0/440077664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5ec144a80 0x7ff5dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec15fa10 con 0x7ff5dc004800 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc00ba70 con 0x7ff5dc0039e0 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff5dc00de80 con 0x7ff5ec144a80 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5ebfff640 1 --2- 172.21.15.172:0/440077664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5dc004800 0x7ff5dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5f09ab640 1 --2- 172.21.15.172:0/440077664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5dc0039e0 0x7ff5dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5f11ac640 1 --2- 172.21.15.172:0/440077664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5ec144a80 0x7ff5dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5ebfff640 1 -- 172.21.15.172:0/440077664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5ec144a80 msgr2=0x7ff5dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5ebfff640 1 --2- 172.21.15.172:0/440077664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5ec144a80 0x7ff5dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5ebfff640 1 -- 172.21.15.172:0/440077664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5dc0039e0 msgr2=0x7ff5dc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5ebfff640 1 --2- 172.21.15.172:0/440077664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5dc0039e0 0x7ff5dc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.843+0000 7ff5ebfff640 1 -- 172.21.15.172:0/440077664 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5dc003520 con 0x7ff5dc004800 2024-04-14T08:36:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.844+0000 7ff5ebfff640 1 --2- 172.21.15.172:0/440077664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5dc004800 0x7ff5dc004c80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c210 tx=0x7ff5dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9551c7f6066f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.844+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8007cf0 con 0x7ff5dc004800 2024-04-14T08:36:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.844+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d8030070 con 0x7ff5dc004800 2024-04-14T08:36:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.844+0000 7ff5eb7fe640 1 -- 172.21.15.172:0/440077664 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8005d30 con 0x7ff5dc004800 2024-04-14T08:36:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.845+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/440077664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5dc004800 msgr2=0x7ff5dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.845+0000 7ff5f2c0f640 1 --2- 172.21.15.172:0/440077664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5dc004800 0x7ff5dc004c80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c210 tx=0x7ff5dc005830 comp rx=0 tx=0).stop 2024-04-14T08:36:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.845+0000 7ff5f11ac640 1 -- 172.21.15.172:0/440077664 reap_dead start 2024-04-14T08:36:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.845+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/440077664 shutdown_connections 2024-04-14T08:36:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.845+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/440077664 >> 172.21.15.172:0/440077664 conn(0x7ff5ec06ec30 msgr2=0x7ff5ec06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.845+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/440077664 shutdown_connections 2024-04-14T08:36:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.845+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/440077664 wait complete. 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.846+0000 7ff5f2c0f640 1 Processor -- start 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.846+0000 7ff5f2c0f640 1 -- start start 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f2c0f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5dc004800 0x7ff5ec142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f2c0f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5ec141cc0 0x7ff5ec1efd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f2c0f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5ec1f3ac0 0x7ff5ec1f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f2c0f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec0b4a30 con 0x7ff5dc004800 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f2c0f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec0b48b0 con 0x7ff5ec141cc0 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f2c0f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec15f140 con 0x7ff5ec1f3ac0 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f09ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5dc004800 0x7ff5ec142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f09ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5dc004800 0x7ff5ec142370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39824/0 (socket says 172.21.15.172:39824) 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f09ab640 1 -- 172.21.15.172:0/638038171 learned_addr learned my addr 172.21.15.172:0/638038171 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f09ab640 1 -- 172.21.15.172:0/638038171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5ec1f3ac0 msgr2=0x7ff5ec1f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f09ab640 1 --2- 172.21.15.172:0/638038171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5ec1f3ac0 0x7ff5ec1f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f09ab640 1 -- 172.21.15.172:0/638038171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5ec141cc0 msgr2=0x7ff5ec1efd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f09ab640 1 --2- 172.21.15.172:0/638038171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5ec141cc0 0x7ff5ec1efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f09ab640 1 -- 172.21.15.172:0/638038171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec142140 con 0x7ff5dc004800 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.847+0000 7ff5f09ab640 1 --2- 172.21.15.172:0/638038171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5dc004800 0x7ff5ec142370 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0011920 tx=0x7ff5e0011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.848+0000 7ff5e97fa640 1 -- 172.21.15.172:0/638038171 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000faf0 con 0x7ff5dc004800 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.848+0000 7ff5e97fa640 1 -- 172.21.15.172:0/638038171 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e000fc90 con 0x7ff5dc004800 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.848+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/638038171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec1f05f0 con 0x7ff5dc004800 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.848+0000 7ff5e97fa640 1 -- 172.21.15.172:0/638038171 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0014440 con 0x7ff5dc004800 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.848+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/638038171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec1f0bb0 con 0x7ff5dc004800 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.848+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/638038171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ec1f3390 con 0x7ff5dc004800 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.849+0000 7ff5e97fa640 1 -- 172.21.15.172:0/638038171 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5e0012040 con 0x7ff5dc004800 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.849+0000 7ff5e97fa640 1 --2- 172.21.15.172:0/638038171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff5d006c920 0x7ff5d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:58.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.849+0000 7ff5e97fa640 1 -- 172.21.15.172:0/638038171 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7ff5e008bfd0 con 0x7ff5dc004800 2024-04-14T08:36:58.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.849+0000 7ff5ebfff640 1 --2- 172.21.15.172:0/638038171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff5d006c920 0x7ff5d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:58.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.850+0000 7ff5ebfff640 1 --2- 172.21.15.172:0/638038171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff5d006c920 0x7ff5d006ede0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8034000 tx=0x7ff5d8039040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:58.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:58.854+0000 7ff5e97fa640 1 -- 172.21.15.172:0/638038171 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff5e0019030 con 0x7ff5dc004800 2024-04-14T08:36:59.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.030+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/638038171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff5ec068520 con 0x7ff5dc004800 2024-04-14T08:36:59.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.030+0000 7ff5e97fa640 1 -- 172.21.15.172:0/638038171 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7ff5e005a5d0 con 0x7ff5dc004800 2024-04-14T08:36:59.032 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:36:59.032 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-04-14T08:36:59.032 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:36:59.032 INFO:teuthology.orchestra.run.smithi172.stdout: "poolnum": 1, 2024-04-14T08:36:59.033 INFO:teuthology.orchestra.run.smithi172.stdout: "poolname": ".mgr" 2024-04-14T08:36:59.033 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:36:59.033 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-04-14T08:36:59.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.032+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/638038171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff5d006c920 msgr2=0x7ff5d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:59.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.032+0000 7ff5f2c0f640 1 --2- 172.21.15.172:0/638038171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff5d006c920 0x7ff5d006ede0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8034000 tx=0x7ff5d8039040 comp rx=0 tx=0).stop 2024-04-14T08:36:59.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.032+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/638038171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5dc004800 msgr2=0x7ff5ec142370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:59.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.032+0000 7ff5f2c0f640 1 --2- 172.21.15.172:0/638038171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5dc004800 0x7ff5ec142370 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0011920 tx=0x7ff5e0011de0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.033+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/638038171 shutdown_connections 2024-04-14T08:36:59.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.033+0000 7ff5f2c0f640 1 --2- 172.21.15.172:0/638038171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff5d006c920 0x7ff5d006ede0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.033+0000 7ff5f2c0f640 1 --2- 172.21.15.172:0/638038171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff5ec1f3ac0 0x7ff5ec1f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.033+0000 7ff5f2c0f640 1 --2- 172.21.15.172:0/638038171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5ec141cc0 0x7ff5ec1efd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.033+0000 7ff5f2c0f640 1 --2- 172.21.15.172:0/638038171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5dc004800 0x7ff5ec142370 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.033+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/638038171 >> 172.21.15.172:0/638038171 conn(0x7ff5ec06ec30 msgr2=0x7ff5ec0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:59.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.033+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/638038171 shutdown_connections 2024-04-14T08:36:59.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.033+0000 7ff5f2c0f640 1 -- 172.21.15.172:0/638038171 wait complete. 2024-04-14T08:36:59.045 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-14T08:36:59.045 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-14T08:36:59.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.148+0000 7fa93800f640 1 Processor -- start 2024-04-14T08:36:59.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.148+0000 7fa93800f640 1 -- start start 2024-04-14T08:36:59.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.149+0000 7fa93800f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa930161d00 0x7fa9300b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:59.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.149+0000 7fa93800f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9300d2cf0 0x7fa9300cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.149+0000 7fa93800f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9300d1bb0 con 0x7fa9300b5490 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.149+0000 7fa93800f640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa9300d0ba0 con 0x7fa930161d00 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.149+0000 7fa93800f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa9300d1720 con 0x7fa9300d2cf0 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.149+0000 7fa935dab640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa930161d00 0x7fa9300b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.149+0000 7fa935dab640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa930161d00 0x7fa9300b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39834/0 (socket says 172.21.15.172:39834) 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.149+0000 7fa9355aa640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa9300b5490 0x7fa9300d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:52380/0 (socket says 172.21.15.172:52380) 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.149+0000 7fa935dab640 1 -- 172.21.15.172:0/235716111 learned_addr learned my addr 172.21.15.172:0/235716111 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.150+0000 7fa9365ac640 1 --2- 172.21.15.172:0/235716111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9300d2cf0 0x7fa9300cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.150+0000 7fa935dab640 1 -- 172.21.15.172:0/235716111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9300d2cf0 msgr2=0x7fa9300cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.150+0000 7fa935dab640 1 --2- 172.21.15.172:0/235716111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9300d2cf0 0x7fa9300cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.150+0000 7fa935dab640 1 -- 172.21.15.172:0/235716111 >> v1:172.21.15.172:6789/0 conn(0x7fa9300b5490 legacy=0x7fa9300d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.150+0000 7fa935dab640 1 -- 172.21.15.172:0/235716111 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9300d0df0 con 0x7fa930161d00 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.150+0000 7fa934da9640 1 -- 172.21.15.172:0/235716111 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3053272316 0 0) 0x7fa9300d1bb0 con 0x7fa9300b5490 2024-04-14T08:36:59.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.150+0000 7fa9365ac640 1 --2- 172.21.15.172:0/235716111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9300d2cf0 0x7fa9300cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:36:59.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.150+0000 7fa935dab640 1 --2- 172.21.15.172:0/235716111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa930161d00 0x7fa9300b30a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa920009e50 tx=0x7fa92002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3df491a811d50a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:59.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.151+0000 7fa934da9640 1 -- 172.21.15.172:0/235716111 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92002ca70 con 0x7fa930161d00 2024-04-14T08:36:59.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.151+0000 7fa934da9640 1 -- 172.21.15.172:0/235716111 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa92002cc10 con 0x7fa930161d00 2024-04-14T08:36:59.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.151+0000 7fa934da9640 1 -- 172.21.15.172:0/235716111 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9200343f0 con 0x7fa930161d00 2024-04-14T08:36:59.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.151+0000 7fa93800f640 1 -- 172.21.15.172:0/235716111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa930161d00 msgr2=0x7fa9300b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:59.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.151+0000 7fa93800f640 1 --2- 172.21.15.172:0/235716111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa930161d00 0x7fa9300b30a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa920009e50 tx=0x7fa92002bed0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.151+0000 7fa93800f640 1 -- 172.21.15.172:0/235716111 shutdown_connections 2024-04-14T08:36:59.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.151+0000 7fa93800f640 1 --2- 172.21.15.172:0/235716111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9300d2cf0 0x7fa9300cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.151+0000 7fa93800f640 1 --2- 172.21.15.172:0/235716111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa930161d00 0x7fa9300b30a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.151+0000 7fa93800f640 1 -- 172.21.15.172:0/235716111 >> 172.21.15.172:0/235716111 conn(0x7fa93009d570 msgr2=0x7fa9300d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:59.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.151+0000 7fa93800f640 1 -- 172.21.15.172:0/235716111 shutdown_connections 2024-04-14T08:36:59.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.151+0000 7fa93800f640 1 -- 172.21.15.172:0/235716111 wait complete. 2024-04-14T08:36:59.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.152+0000 7fa93800f640 1 Processor -- start 2024-04-14T08:36:59.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.152+0000 7fa93800f640 1 -- start start 2024-04-14T08:36:59.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.153+0000 7fa93800f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9300b5490 0x7fa9300c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:59.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.153+0000 7fa93800f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9300d2cf0 0x7fa930159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:59.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.153+0000 7fa93800f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa930161d00 0x7fa9300b91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:59.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.153+0000 7fa93800f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa9300d13b0 con 0x7fa9300d2cf0 2024-04-14T08:36:59.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.153+0000 7fa93800f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa9300d33b0 con 0x7fa930161d00 2024-04-14T08:36:59.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.153+0000 7fa93800f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa9300b5940 con 0x7fa9300b5490 2024-04-14T08:36:59.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.153+0000 7fa9355aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9300d2cf0 0x7fa930159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:59.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.153+0000 7fa9355aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9300d2cf0 0x7fa930159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39836/0 (socket says 172.21.15.172:39836) 2024-04-14T08:36:59.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.153+0000 7fa9355aa640 1 -- 172.21.15.172:0/3146242399 learned_addr learned my addr 172.21.15.172:0/3146242399 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:59.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.153+0000 7fa935dab640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9300b5490 0x7fa9300c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:59.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.153+0000 7fa9365ac640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa930161d00 0x7fa9300b91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.154+0000 7fa9355aa640 1 -- 172.21.15.172:0/3146242399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9300b5490 msgr2=0x7fa9300c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.154+0000 7fa9355aa640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9300b5490 0x7fa9300c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.154+0000 7fa9355aa640 1 -- 172.21.15.172:0/3146242399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa930161d00 msgr2=0x7fa9300b91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.154+0000 7fa9355aa640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa930161d00 0x7fa9300b91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.154+0000 7fa9355aa640 1 -- 172.21.15.172:0/3146242399 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9300c8db0 con 0x7fa9300d2cf0 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.154+0000 7fa9365ac640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa930161d00 0x7fa9300b91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.154+0000 7fa9355aa640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9300d2cf0 0x7fa930159600 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa92400c420 tx=0x7fa92400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.154+0000 7fa92e7fc640 1 -- 172.21.15.172:0/3146242399 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9240074e0 con 0x7fa9300d2cf0 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.154+0000 7fa92e7fc640 1 -- 172.21.15.172:0/3146242399 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9240047e0 con 0x7fa9300d2cf0 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.154+0000 7fa92e7fc640 1 -- 172.21.15.172:0/3146242399 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924004c10 con 0x7fa9300d2cf0 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.154+0000 7fa93800f640 1 -- 172.21.15.172:0/3146242399 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9300c9090 con 0x7fa9300d2cf0 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.154+0000 7fa93800f640 1 -- 172.21.15.172:0/3146242399 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9300c9650 con 0x7fa9300d2cf0 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.155+0000 7fa93800f640 1 -- 172.21.15.172:0/3146242399 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9300686a0 con 0x7fa9300d2cf0 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.155+0000 7fa92e7fc640 1 -- 172.21.15.172:0/3146242399 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa924011070 con 0x7fa9300d2cf0 2024-04-14T08:36:59.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.156+0000 7fa92e7fc640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa91c06c8f0 0x7fa91c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:59.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.156+0000 7fa935dab640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa91c06c8f0 0x7fa91c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:59.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.156+0000 7fa92e7fc640 1 -- 172.21.15.172:0/3146242399 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5249+0+0 (secure 0 0 0) 0x7fa92408b520 con 0x7fa9300d2cf0 2024-04-14T08:36:59.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.156+0000 7fa935dab640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa91c06c8f0 0x7fa91c06edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa920035000 tx=0x7fa920030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:59.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.160+0000 7fa92e7fc640 1 -- 172.21.15.172:0/3146242399 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa924059b10 con 0x7fa9300d2cf0 2024-04-14T08:36:59.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.336+0000 7fa93800f640 1 -- 172.21.15.172:0/3146242399 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa9300b9700 con 0x7fa9300d2cf0 2024-04-14T08:36:59.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.741+0000 7fa92e7fc640 1 -- 172.21.15.172:0/3146242399 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7fa924059410 con 0x7fa9300d2cf0 2024-04-14T08:36:59.742 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' created 2024-04-14T08:36:59.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.744+0000 7fa93800f640 1 -- 172.21.15.172:0/3146242399 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa91c06c8f0 msgr2=0x7fa91c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:59.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.744+0000 7fa93800f640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa91c06c8f0 0x7fa91c06edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa920035000 tx=0x7fa920030000 comp rx=0 tx=0).stop 2024-04-14T08:36:59.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.744+0000 7fa93800f640 1 -- 172.21.15.172:0/3146242399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9300d2cf0 msgr2=0x7fa930159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:59.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.744+0000 7fa93800f640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9300d2cf0 0x7fa930159600 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa92400c420 tx=0x7fa92400c8e0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.744+0000 7fa93800f640 1 -- 172.21.15.172:0/3146242399 shutdown_connections 2024-04-14T08:36:59.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.744+0000 7fa93800f640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa91c06c8f0 0x7fa91c06edb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.744+0000 7fa93800f640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa930161d00 0x7fa9300b91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.744+0000 7fa93800f640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9300d2cf0 0x7fa930159600 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.744+0000 7fa93800f640 1 --2- 172.21.15.172:0/3146242399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa9300b5490 0x7fa9300c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.744+0000 7fa93800f640 1 -- 172.21.15.172:0/3146242399 >> 172.21.15.172:0/3146242399 conn(0x7fa93009d570 msgr2=0x7fa9300b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:59.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.744+0000 7fa93800f640 1 -- 172.21.15.172:0/3146242399 shutdown_connections 2024-04-14T08:36:59.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.745+0000 7fa93800f640 1 -- 172.21.15.172:0/3146242399 wait complete. 2024-04-14T08:36:59.759 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-14T08:36:59.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.866+0000 7fa0f0ede640 1 Processor -- start 2024-04-14T08:36:59.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.866+0000 7fa0f0ede640 1 -- start start 2024-04-14T08:36:59.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.867+0000 7fa0f0ede640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0ec15c950 0x7fa0ec0cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:59.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.867+0000 7fa0f0ede640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa0ec0cff30 0x7fa0ec15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:59.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.867+0000 7fa0f0ede640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0ec0cce80 con 0x7fa0ec15d270 2024-04-14T08:36:59.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.867+0000 7fa0f0ede640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa0ec0cd160 con 0x7fa0ec15c950 2024-04-14T08:36:59.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.867+0000 7fa0f0ede640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa0ec0cd440 con 0x7fa0ec0cff30 2024-04-14T08:36:59.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.867+0000 7fa0ea59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0ec15c950 0x7fa0ec0cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:59.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.867+0000 7fa0ead9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa0ec15d270 0x7fa0ec0b37b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58502/0 (socket says 172.21.15.172:58502) 2024-04-14T08:36:59.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.867+0000 7fa0ead9d640 1 -- 172.21.15.172:0/2989028003 learned_addr learned my addr 172.21.15.172:0/2989028003 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:36:59.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.867+0000 7fa0e9d9b640 1 --2- 172.21.15.172:0/2989028003 >> v2:172.21.15.177:3300/0 conn(0x7fa0ec0cff30 0x7fa0ec15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:59.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.868+0000 7fa0ea59c640 1 -- 172.21.15.172:0/2989028003 >> v2:172.21.15.177:3300/0 conn(0x7fa0ec0cff30 msgr2=0x7fa0ec15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:36:59.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.868+0000 7fa0ea59c640 1 --2- 172.21.15.172:0/2989028003 >> v2:172.21.15.177:3300/0 conn(0x7fa0ec0cff30 0x7fa0ec15cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.868+0000 7fa0ea59c640 1 -- 172.21.15.172:0/2989028003 >> v1:172.21.15.172:6789/0 conn(0x7fa0ec15d270 legacy=0x7fa0ec0b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:59.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.868+0000 7fa0ea59c640 1 -- 172.21.15.172:0/2989028003 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec0cd720 con 0x7fa0ec15c950 2024-04-14T08:36:59.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.868+0000 7fa0e959a640 1 -- 172.21.15.172:0/2989028003 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1967337686 0 0) 0x7fa0ec0cce80 con 0x7fa0ec15d270 2024-04-14T08:36:59.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.868+0000 7fa0ea59c640 1 --2- 172.21.15.172:0/2989028003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ec15c950 0x7fa0ec0cc8a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0b02f0 tx=0x7fa0e402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b932aa2075bf93c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:59.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.868+0000 7fa0e959a640 1 -- 172.21.15.172:0/2989028003 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4039070 con 0x7fa0ec15c950 2024-04-14T08:36:59.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.868+0000 7fa0e959a640 1 -- 172.21.15.172:0/2989028003 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e402cc80 con 0x7fa0ec15c950 2024-04-14T08:36:59.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.868+0000 7fa0e959a640 1 -- 172.21.15.172:0/2989028003 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4034660 con 0x7fa0ec15c950 2024-04-14T08:36:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.869+0000 7fa0f0ede640 1 -- 172.21.15.172:0/2989028003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ec15c950 msgr2=0x7fa0ec0cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.869+0000 7fa0f0ede640 1 --2- 172.21.15.172:0/2989028003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ec15c950 0x7fa0ec0cc8a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0b02f0 tx=0x7fa0e402bed0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.869+0000 7fa0f0ede640 1 -- 172.21.15.172:0/2989028003 shutdown_connections 2024-04-14T08:36:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.869+0000 7fa0f0ede640 1 --2- 172.21.15.172:0/2989028003 >> v2:172.21.15.177:3300/0 conn(0x7fa0ec0cff30 0x7fa0ec15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.869+0000 7fa0f0ede640 1 --2- 172.21.15.172:0/2989028003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ec15c950 0x7fa0ec0cc8a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.869+0000 7fa0f0ede640 1 -- 172.21.15.172:0/2989028003 >> 172.21.15.172:0/2989028003 conn(0x7fa0ec0d0780 msgr2=0x7fa0ec0d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:36:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.869+0000 7fa0f0ede640 1 -- 172.21.15.172:0/2989028003 shutdown_connections 2024-04-14T08:36:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.869+0000 7fa0f0ede640 1 -- 172.21.15.172:0/2989028003 wait complete. 2024-04-14T08:36:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.870+0000 7fa0f0ede640 1 Processor -- start 2024-04-14T08:36:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.871+0000 7fa0f0ede640 1 -- start start 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.871+0000 7fa0f0ede640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ec0cff30 0x7fa0ec1f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.871+0000 7fa0f0ede640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ec15c950 0x7fa0ec14be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.871+0000 7fa0f0ede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ec15d270 0x7fa0ec14e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.871+0000 7fa0f0ede640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec0d03e0 con 0x7fa0ec15c950 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.871+0000 7fa0f0ede640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec15d6a0 con 0x7fa0ec0cff30 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.871+0000 7fa0f0ede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ec0d0560 con 0x7fa0ec15d270 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.871+0000 7fa0e9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ec15c950 0x7fa0ec14be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.871+0000 7fa0e9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ec15c950 0x7fa0ec14be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55152/0 (socket says 172.21.15.172:55152) 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.871+0000 7fa0e9d9b640 1 -- 172.21.15.172:0/4273440454 learned_addr learned my addr 172.21.15.172:0/4273440454 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.872+0000 7fa0e9d9b640 1 -- 172.21.15.172:0/4273440454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ec15d270 msgr2=0x7fa0ec14e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.872+0000 7fa0e9d9b640 1 --2- 172.21.15.172:0/4273440454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ec15d270 0x7fa0ec14e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.872+0000 7fa0e9d9b640 1 -- 172.21.15.172:0/4273440454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ec0cff30 msgr2=0x7fa0ec1f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.872+0000 7fa0e9d9b640 1 --2- 172.21.15.172:0/4273440454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ec0cff30 0x7fa0ec1f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:36:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.872+0000 7fa0e9d9b640 1 -- 172.21.15.172:0/4273440454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec1f4d80 con 0x7fa0ec15c950 2024-04-14T08:36:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.872+0000 7fa0e9d9b640 1 --2- 172.21.15.172:0/4273440454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ec15c950 0x7fa0ec14be10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000d800 tx=0x7fa0e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.872+0000 7fa0daffd640 1 -- 172.21.15.172:0/4273440454 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e000b6c0 con 0x7fa0ec15c950 2024-04-14T08:36:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.872+0000 7fa0daffd640 1 -- 172.21.15.172:0/4273440454 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e0019070 con 0x7fa0ec15c950 2024-04-14T08:36:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.872+0000 7fa0daffd640 1 -- 172.21.15.172:0/4273440454 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e000ba80 con 0x7fa0ec15c950 2024-04-14T08:36:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.872+0000 7fa0f0ede640 1 -- 172.21.15.172:0/4273440454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec1f4260 con 0x7fa0ec15c950 2024-04-14T08:36:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.872+0000 7fa0f0ede640 1 -- 172.21.15.172:0/4273440454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec1f4aa0 con 0x7fa0ec15c950 2024-04-14T08:36:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.873+0000 7fa0f0ede640 1 -- 172.21.15.172:0/4273440454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ec068590 con 0x7fa0ec15c950 2024-04-14T08:36:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.873+0000 7fa0daffd640 1 -- 172.21.15.172:0/4273440454 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0e000bc20 con 0x7fa0ec15c950 2024-04-14T08:36:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.874+0000 7fa0daffd640 1 --2- 172.21.15.172:0/4273440454 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa0cc06c8f0 0x7fa0cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:36:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.874+0000 7fa0ea59c640 1 --2- 172.21.15.172:0/4273440454 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa0cc06c8f0 0x7fa0cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:36:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.874+0000 7fa0daffd640 1 -- 172.21.15.172:0/4273440454 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5619+0+0 (secure 0 0 0) 0x7fa0e008bf60 con 0x7fa0ec15c950 2024-04-14T08:36:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.874+0000 7fa0ea59c640 1 --2- 172.21.15.172:0/4273440454 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa0cc06c8f0 0x7fa0cc06edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4036000 tx=0x7fa0e402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:36:59.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:36:59.879+0000 7fa0daffd640 1 -- 172.21.15.172:0/4273440454 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa0ec1f4260 con 0x7fa0ec15c950 2024-04-14T08:37:00.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.054+0000 7fa0f0ede640 1 -- 172.21.15.172:0/4273440454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa0ec156680 con 0x7fa0ec15c950 2024-04-14T08:37:00.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.747+0000 7fa0daffd640 1 -- 172.21.15.172:0/4273440454 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) ==== 170+0+0 (secure 0 0 0) 0x7fa0e001c360 con 0x7fa0ec15c950 2024-04-14T08:37:00.749 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' created 2024-04-14T08:37:00.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.750+0000 7fa0f0ede640 1 -- 172.21.15.172:0/4273440454 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa0cc06c8f0 msgr2=0x7fa0cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:00.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.750+0000 7fa0f0ede640 1 --2- 172.21.15.172:0/4273440454 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa0cc06c8f0 0x7fa0cc06edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4036000 tx=0x7fa0e402e000 comp rx=0 tx=0).stop 2024-04-14T08:37:00.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.750+0000 7fa0f0ede640 1 -- 172.21.15.172:0/4273440454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ec15c950 msgr2=0x7fa0ec14be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:00.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.750+0000 7fa0f0ede640 1 --2- 172.21.15.172:0/4273440454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ec15c950 0x7fa0ec14be10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000d800 tx=0x7fa0e000dcc0 comp rx=0 tx=0).stop 2024-04-14T08:37:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.750+0000 7fa0f0ede640 1 -- 172.21.15.172:0/4273440454 shutdown_connections 2024-04-14T08:37:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.750+0000 7fa0f0ede640 1 --2- 172.21.15.172:0/4273440454 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa0cc06c8f0 0x7fa0cc06edb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.750+0000 7fa0f0ede640 1 --2- 172.21.15.172:0/4273440454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0ec15d270 0x7fa0ec14e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.750+0000 7fa0f0ede640 1 --2- 172.21.15.172:0/4273440454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0ec15c950 0x7fa0ec14be10 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.750+0000 7fa0f0ede640 1 --2- 172.21.15.172:0/4273440454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0ec0cff30 0x7fa0ec1f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:00.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.750+0000 7fa0f0ede640 1 -- 172.21.15.172:0/4273440454 >> 172.21.15.172:0/4273440454 conn(0x7fa0ec0d0780 msgr2=0x7fa0ec0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:00.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.751+0000 7fa0f0ede640 1 -- 172.21.15.172:0/4273440454 shutdown_connections 2024-04-14T08:37:00.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.757+0000 7fa0f0ede640 1 -- 172.21.15.172:0/4273440454 wait complete. 2024-04-14T08:37:00.781 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-14T08:37:00.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.887+0000 7f192418c640 1 Processor -- start 2024-04-14T08:37:00.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.888+0000 7f192418c640 1 -- start start 2024-04-14T08:37:00.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.888+0000 7f192418c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f191c13df30 0x7f191c0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:00.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.888+0000 7f192418c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f191c140e80 0x7f191c0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:00.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.888+0000 7f192418c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f191c15b620 con 0x7f191c15c380 2024-04-14T08:37:00.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.888+0000 7f192418c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f191c0cf9b0 con 0x7f191c140e80 2024-04-14T08:37:00.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.888+0000 7f192418c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f191c1414a0 con 0x7f191c13df30 2024-04-14T08:37:00.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.889+0000 7f1922729640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f191c140e80 0x7f191c0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:00.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.889+0000 7f1922729640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f191c140e80 0x7f191c0b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55158/0 (socket says 172.21.15.172:55158) 2024-04-14T08:37:00.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.889+0000 7f1922729640 1 -- 172.21.15.172:0/4271713190 learned_addr learned my addr 172.21.15.172:0/4271713190 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:00.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.889+0000 7f1921f28640 1 --2- 172.21.15.172:0/4271713190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f191c13df30 0x7f191c0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:00.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.889+0000 7f1922729640 1 -- 172.21.15.172:0/4271713190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f191c13df30 msgr2=0x7f191c0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:00.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.889+0000 7f1922729640 1 --2- 172.21.15.172:0/4271713190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f191c13df30 0x7f191c0b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:00.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.889+0000 7f1922729640 1 -- 172.21.15.172:0/4271713190 >> v1:172.21.15.177:6789/0 conn(0x7f191c15c380 legacy=0x7f191c140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:00.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.889+0000 7f1922729640 1 -- 172.21.15.172:0/4271713190 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c15b8a0 con 0x7f191c140e80 2024-04-14T08:37:00.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.890+0000 7f1922729640 1 --2- 172.21.15.172:0/4271713190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f191c140e80 0x7f191c0b2210 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1918009e80 tx=0x7f191802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d81b3d9c6abc7f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:00.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.890+0000 7f1920f26640 1 -- 172.21.15.172:0/4271713190 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191802ca50 con 0x7f191c140e80 2024-04-14T08:37:00.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.890+0000 7f1920f26640 1 -- 172.21.15.172:0/4271713190 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f191802cbf0 con 0x7f191c140e80 2024-04-14T08:37:00.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.890+0000 7f1920f26640 1 -- 172.21.15.172:0/4271713190 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19180343f0 con 0x7f191c140e80 2024-04-14T08:37:00.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.890+0000 7f192418c640 1 -- 172.21.15.172:0/4271713190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f191c140e80 msgr2=0x7f191c0b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:00.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.890+0000 7f192418c640 1 --2- 172.21.15.172:0/4271713190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f191c140e80 0x7f191c0b2210 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1918009e80 tx=0x7f191802beb0 comp rx=0 tx=0).stop 2024-04-14T08:37:00.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.890+0000 7f192418c640 1 -- 172.21.15.172:0/4271713190 shutdown_connections 2024-04-14T08:37:00.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.890+0000 7f192418c640 1 --2- 172.21.15.172:0/4271713190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f191c140e80 0x7f191c0b2210 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:00.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.890+0000 7f192418c640 1 --2- 172.21.15.172:0/4271713190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f191c13df30 0x7f191c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:00.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.890+0000 7f192418c640 1 -- 172.21.15.172:0/4271713190 >> 172.21.15.172:0/4271713190 conn(0x7f191c06b520 msgr2=0x7f191c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:00.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.890+0000 7f192418c640 1 -- 172.21.15.172:0/4271713190 shutdown_connections 2024-04-14T08:37:00.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.890+0000 7f192418c640 1 -- 172.21.15.172:0/4271713190 wait complete. 2024-04-14T08:37:00.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.892+0000 7f192418c640 1 Processor -- start 2024-04-14T08:37:00.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.892+0000 7f192418c640 1 -- start start 2024-04-14T08:37:00.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.892+0000 7f192418c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f191c13df30 0x7f191c13efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:00.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.892+0000 7f192418c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f191c140e80 0x7f191c1f0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:00.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.892+0000 7f192418c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f191c15c380 0x7f191c1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:00.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.892+0000 7f192418c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f191c13e310 con 0x7f191c13df30 2024-04-14T08:37:00.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.892+0000 7f192418c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f191c15be60 con 0x7f191c140e80 2024-04-14T08:37:00.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.892+0000 7f192418c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f191c0ce9a0 con 0x7f191c15c380 2024-04-14T08:37:00.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.892+0000 7f1921f28640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f191c13df30 0x7f191c13efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:00.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.892+0000 7f1921f28640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f191c13df30 0x7f191c13efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55162/0 (socket says 172.21.15.172:55162) 2024-04-14T08:37:00.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.892+0000 7f1921f28640 1 -- 172.21.15.172:0/498864538 learned_addr learned my addr 172.21.15.172:0/498864538 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:00.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.892+0000 7f1922729640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f191c15c380 0x7f191c1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:00.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.893+0000 7f1921727640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f191c140e80 0x7f191c1f0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:00.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.893+0000 7f1921f28640 1 -- 172.21.15.172:0/498864538 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f191c15c380 msgr2=0x7f191c1530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:00.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.893+0000 7f1921f28640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f191c15c380 0x7f191c1530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:00.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.893+0000 7f1921f28640 1 -- 172.21.15.172:0/498864538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f191c140e80 msgr2=0x7f191c1f0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:00.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.893+0000 7f1921f28640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f191c140e80 0x7f191c1f0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:00.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.893+0000 7f1921f28640 1 -- 172.21.15.172:0/498864538 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c13f910 con 0x7f191c13df30 2024-04-14T08:37:00.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.893+0000 7f1921f28640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f191c13df30 0x7f191c13efd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f191400c7f0 tx=0x7f191400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:00.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.893+0000 7f19127fc640 1 -- 172.21.15.172:0/498864538 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914015070 con 0x7f191c13df30 2024-04-14T08:37:00.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.893+0000 7f19127fc640 1 -- 172.21.15.172:0/498864538 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1914004240 con 0x7f191c13df30 2024-04-14T08:37:00.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.893+0000 7f192418c640 1 -- 172.21.15.172:0/498864538 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191c13fbf0 con 0x7f191c13df30 2024-04-14T08:37:00.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.893+0000 7f19127fc640 1 -- 172.21.15.172:0/498864538 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914007620 con 0x7f191c13df30 2024-04-14T08:37:00.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.893+0000 7f192418c640 1 -- 172.21.15.172:0/498864538 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191c140490 con 0x7f191c13df30 2024-04-14T08:37:00.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.894+0000 7f192418c640 1 -- 172.21.15.172:0/498864538 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f191c13fdc0 con 0x7f191c13df30 2024-04-14T08:37:00.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.894+0000 7f19127fc640 1 -- 172.21.15.172:0/498864538 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19140079d0 con 0x7f191c13df30 2024-04-14T08:37:00.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.895+0000 7f19127fc640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f190006c8f0 0x7f190006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:00.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.895+0000 7f1921727640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f190006c8f0 0x7f190006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:00.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.895+0000 7f19127fc640 1 -- 172.21.15.172:0/498864538 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6001+0+0 (secure 0 0 0) 0x7f191408c0f0 con 0x7f191c13df30 2024-04-14T08:37:00.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.895+0000 7f1921727640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f190006c8f0 0x7f190006edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f191c13d260 tx=0x7f190c009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:00.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:00.900+0000 7f19127fc640 1 -- 172.21.15.172:0/498864538 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1914059d10 con 0x7f191c13df30 2024-04-14T08:37:01.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.112+0000 7f192418c640 1 -- 172.21.15.172:0/498864538 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f191c0cfce0 con 0x7f191c13df30 2024-04-14T08:37:01.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.756+0000 7f19127fc640 1 -- 172.21.15.172:0/498864538 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v37) ==== 174+0+0 (secure 0 0 0) 0x7f191c0cfce0 con 0x7f191c13df30 2024-04-14T08:37:01.758 INFO:teuthology.orchestra.run.smithi172.stderr:new fs with metadata pool 11 and data pool 12 2024-04-14T08:37:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.769+0000 7f192418c640 1 -- 172.21.15.172:0/498864538 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f190006c8f0 msgr2=0x7f190006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.769+0000 7f192418c640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f190006c8f0 0x7f190006edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f191c13d260 tx=0x7f190c009060 comp rx=0 tx=0).stop 2024-04-14T08:37:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.769+0000 7f192418c640 1 -- 172.21.15.172:0/498864538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f191c13df30 msgr2=0x7f191c13efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.769+0000 7f192418c640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f191c13df30 0x7f191c13efd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f191400c7f0 tx=0x7f191400ccb0 comp rx=0 tx=0).stop 2024-04-14T08:37:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.770+0000 7f192418c640 1 -- 172.21.15.172:0/498864538 shutdown_connections 2024-04-14T08:37:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.770+0000 7f192418c640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f190006c8f0 0x7f190006edb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.770+0000 7f192418c640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f191c15c380 0x7f191c1530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.770+0000 7f192418c640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f191c140e80 0x7f191c1f0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.770+0000 7f192418c640 1 --2- 172.21.15.172:0/498864538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f191c13df30 0x7f191c13efd0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.770+0000 7f192418c640 1 -- 172.21.15.172:0/498864538 >> 172.21.15.172:0/498864538 conn(0x7f191c06b520 msgr2=0x7f191c15eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.771+0000 7f192418c640 1 -- 172.21.15.172:0/498864538 shutdown_connections 2024-04-14T08:37:01.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.771+0000 7f192418c640 1 -- 172.21.15.172:0/498864538 wait complete. 2024-04-14T08:37:01.790 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:37:01.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.889+0000 7f35b0d5e640 1 Processor -- start 2024-04-14T08:37:01.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.889+0000 7f35b0d5e640 1 -- start start 2024-04-14T08:37:01.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.890+0000 7f35b0d5e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac0b1df0 0x7f35ac0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:01.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.890+0000 7f35b0d5e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ac15c770 con 0x7f35ac0b21d0 2024-04-14T08:37:01.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.890+0000 7f35b0d5e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ac0b3430 con 0x7f35ac0b4040 2024-04-14T08:37:01.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.890+0000 7f35b0d5e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f35ac0b3700 con 0x7f35ac0b1df0 2024-04-14T08:37:01.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.890+0000 7f35a9d9b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f35ac0b21d0 0x7f35ac15c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58504/0 (socket says 172.21.15.172:58504) 2024-04-14T08:37:01.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.890+0000 7f35a9d9b640 1 -- 172.21.15.172:0/3471674350 learned_addr learned my addr 172.21.15.172:0/3471674350 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:01.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.891+0000 7f35aa59c640 1 --2- 172.21.15.172:0/3471674350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac0b1df0 0x7f35ac0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:01.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.891+0000 7f35a959a640 1 -- 172.21.15.172:0/3471674350 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1121317903 0 0) 0x7f35ac15c770 con 0x7f35ac0b21d0 2024-04-14T08:37:01.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.891+0000 7f35a959a640 1 -- 172.21.15.172:0/3471674350 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3594005730 con 0x7f35ac0b21d0 2024-04-14T08:37:01.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.891+0000 7f35a959a640 1 -- 172.21.15.172:0/3471674350 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 218234358 0 0) 0x7f35ac0b3430 con 0x7f35ac0b4040 2024-04-14T08:37:01.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.891+0000 7f35a959a640 1 -- 172.21.15.172:0/3471674350 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35ac15c770 con 0x7f35ac0b4040 2024-04-14T08:37:01.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.891+0000 7f35a959a640 1 -- 172.21.15.172:0/3471674350 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 89814685 0 0) 0x7f3594005730 con 0x7f35ac0b21d0 2024-04-14T08:37:01.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.891+0000 7f35aa59c640 1 -- 172.21.15.172:0/3471674350 >> v1:172.21.15.177:6789/0 conn(0x7f35ac0b4040 legacy=0x7f35ac0b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:01.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.891+0000 7f35aa59c640 1 -- 172.21.15.172:0/3471674350 >> v1:172.21.15.172:6789/0 conn(0x7f35ac0b21d0 legacy=0x7f35ac15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:01.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.891+0000 7f35aa59c640 1 -- 172.21.15.172:0/3471674350 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac0b39e0 con 0x7f35ac0b1df0 2024-04-14T08:37:01.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.892+0000 7f35aa59c640 1 --2- 172.21.15.172:0/3471674350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac0b1df0 0x7f35ac0b2ef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f35a40097d0 tx=0x7f35a402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be3e86d6fd1f8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:01.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.892+0000 7f35a959a640 1 -- 172.21.15.172:0/3471674350 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4037070 con 0x7f35ac0b1df0 2024-04-14T08:37:01.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.892+0000 7f35a959a640 1 -- 172.21.15.172:0/3471674350 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35a4030510 con 0x7f35ac0b1df0 2024-04-14T08:37:01.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.892+0000 7f35a959a640 1 -- 172.21.15.172:0/3471674350 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0001670 con 0x7f35ac0b1df0 2024-04-14T08:37:01.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.893+0000 7f35b0d5e640 1 -- 172.21.15.172:0/3471674350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac0b1df0 msgr2=0x7f35ac0b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:01.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.893+0000 7f35b0d5e640 1 --2- 172.21.15.172:0/3471674350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac0b1df0 0x7f35ac0b2ef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f35a40097d0 tx=0x7f35a402b800 comp rx=0 tx=0).stop 2024-04-14T08:37:01.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.893+0000 7f35b0d5e640 1 -- 172.21.15.172:0/3471674350 shutdown_connections 2024-04-14T08:37:01.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.893+0000 7f35b0d5e640 1 --2- 172.21.15.172:0/3471674350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac0b1df0 0x7f35ac0b2ef0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:01.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.893+0000 7f35b0d5e640 1 -- 172.21.15.172:0/3471674350 >> 172.21.15.172:0/3471674350 conn(0x7f35ac06ec30 msgr2=0x7f35ac06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:01.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.893+0000 7f35b0d5e640 1 -- 172.21.15.172:0/3471674350 shutdown_connections 2024-04-14T08:37:01.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.893+0000 7f35b0d5e640 1 -- 172.21.15.172:0/3471674350 wait complete. 2024-04-14T08:37:01.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.894+0000 7f35b0d5e640 1 Processor -- start 2024-04-14T08:37:01.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.894+0000 7f35b0d5e640 1 -- start start 2024-04-14T08:37:01.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35b0d5e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35ac0b1df0 0x7f35ac1f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:01.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35b0d5e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac0b21d0 0x7f35ac158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:01.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35b0d5e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac0b4040 0x7f35ac0728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:01.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35b0d5e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac15b7f0 con 0x7f35ac0b4040 2024-04-14T08:37:01.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35b0d5e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac15bb40 con 0x7f35ac0b1df0 2024-04-14T08:37:01.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35b0d5e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f35ac1552d0 con 0x7f35ac0b21d0 2024-04-14T08:37:01.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35aad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac0b4040 0x7f35ac0728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:01.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35aad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac0b4040 0x7f35ac0728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55170/0 (socket says 172.21.15.172:55170) 2024-04-14T08:37:01.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35aad9d640 1 -- 172.21.15.172:0/2597398743 learned_addr learned my addr 172.21.15.172:0/2597398743 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:01.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35a9d9b640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac0b21d0 0x7f35ac158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:01.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35aa59c640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35ac0b1df0 0x7f35ac1f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:01.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35aad9d640 1 -- 172.21.15.172:0/2597398743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac0b21d0 msgr2=0x7f35ac158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:01.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35aad9d640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac0b21d0 0x7f35ac158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:01.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35aad9d640 1 -- 172.21.15.172:0/2597398743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35ac0b1df0 msgr2=0x7f35ac1f3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:01.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35aad9d640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35ac0b1df0 0x7f35ac1f3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:01.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35aad9d640 1 -- 172.21.15.172:0/2597398743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac0ceba0 con 0x7f35ac0b4040 2024-04-14T08:37:01.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35a9d9b640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac0b21d0 0x7f35ac158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:01.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35aa59c640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35ac0b1df0 0x7f35ac1f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:01.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35aad9d640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac0b4040 0x7f35ac0728c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f359c00c710 tx=0x7f359c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:01.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f359affd640 1 -- 172.21.15.172:0/2597398743 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c003b70 con 0x7f35ac0b4040 2024-04-14T08:37:01.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f359affd640 1 -- 172.21.15.172:0/2597398743 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c003d10 con 0x7f35ac0b4040 2024-04-14T08:37:01.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f35b0d5e640 1 -- 172.21.15.172:0/2597398743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac156030 con 0x7f35ac0b4040 2024-04-14T08:37:01.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.895+0000 7f359affd640 1 -- 172.21.15.172:0/2597398743 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c00a9f0 con 0x7f35ac0b4040 2024-04-14T08:37:01.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.896+0000 7f35b0d5e640 1 -- 172.21.15.172:0/2597398743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac1565f0 con 0x7f35ac0b4040 2024-04-14T08:37:01.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.897+0000 7f35b0d5e640 1 -- 172.21.15.172:0/2597398743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac068520 con 0x7f35ac0b4040 2024-04-14T08:37:01.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.897+0000 7f359affd640 1 -- 172.21.15.172:0/2597398743 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f359c00ab90 con 0x7f35ac0b4040 2024-04-14T08:37:01.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.897+0000 7f359affd640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f358c06c8f0 0x7f358c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:01.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.898+0000 7f35aa59c640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f358c06c8f0 0x7f358c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:01.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.898+0000 7f359affd640 1 -- 172.21.15.172:0/2597398743 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6101+0+0 (secure 0 0 0) 0x7f359c08bb20 con 0x7f35ac0b4040 2024-04-14T08:37:01.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.898+0000 7f35aa59c640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f358c06c8f0 0x7f358c06edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f35a4034000 tx=0x7f35a4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:01.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:01.902+0000 7f359affd640 1 -- 172.21.15.172:0/2597398743 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f359c00ce20 con 0x7f35ac0b4040 2024-04-14T08:37:02.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.078+0000 7f35b0d5e640 1 -- 172.21.15.172:0/2597398743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35ac0cc650 con 0x7f35ac0b4040 2024-04-14T08:37:02.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.079+0000 7f359affd640 1 -- 172.21.15.172:0/2597398743 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+17018 (secure 0 0 0) 0x7f35ac0cc650 con 0x7f35ac0b4040 2024-04-14T08:37:02.081 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:02.081 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":73,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:37:01.114727+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:36:59.342045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-14T08:37:00.059338+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:37:02.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.082+0000 7f35b0d5e640 1 -- 172.21.15.172:0/2597398743 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f358c06c8f0 msgr2=0x7f358c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.082+0000 7f35b0d5e640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f358c06c8f0 0x7f358c06edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f35a4034000 tx=0x7f35a4008030 comp rx=0 tx=0).stop 2024-04-14T08:37:02.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.082+0000 7f35b0d5e640 1 -- 172.21.15.172:0/2597398743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac0b4040 msgr2=0x7f35ac0728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.082+0000 7f35b0d5e640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac0b4040 0x7f35ac0728c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f359c00c710 tx=0x7f359c00cbd0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.082+0000 7f35b0d5e640 1 -- 172.21.15.172:0/2597398743 shutdown_connections 2024-04-14T08:37:02.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.082+0000 7f35b0d5e640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f358c06c8f0 0x7f358c06edb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.082+0000 7f35b0d5e640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35ac0b4040 0x7f35ac0728c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.082+0000 7f35b0d5e640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35ac0b21d0 0x7f35ac158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.082+0000 7f35b0d5e640 1 --2- 172.21.15.172:0/2597398743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35ac0b1df0 0x7f35ac1f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.082+0000 7f35b0d5e640 1 -- 172.21.15.172:0/2597398743 >> 172.21.15.172:0/2597398743 conn(0x7f35ac06ec30 msgr2=0x7f35ac15cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:02.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.082+0000 7f35b0d5e640 1 -- 172.21.15.172:0/2597398743 shutdown_connections 2024-04-14T08:37:02.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.082+0000 7f35b0d5e640 1 -- 172.21.15.172:0/2597398743 wait complete. 2024-04-14T08:37:02.095 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:37:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.194+0000 7fd60491e640 1 Processor -- start 2024-04-14T08:37:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.194+0000 7fd60491e640 1 -- start start 2024-04-14T08:37:02.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.195+0000 7fd60491e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd600163670 0x7fd600163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.195+0000 7fd60491e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd6000b1b80 0x7fd6000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.195+0000 7fd60491e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd600071620 0x7fd600071a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.195+0000 7fd60491e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd600167390 con 0x7fd600071620 2024-04-14T08:37:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.195+0000 7fd60491e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fd6001665d0 con 0x7fd6000b1b80 2024-04-14T08:37:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.195+0000 7fd60491e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd6001668b0 con 0x7fd600163670 2024-04-14T08:37:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.196+0000 7fd5fed9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd600071620 0x7fd600071a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.196+0000 7fd5fed9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd600071620 0x7fd600071a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55178/0 (socket says 172.21.15.172:55178) 2024-04-14T08:37:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.196+0000 7fd5fed9d640 1 -- 172.21.15.172:0/2598279503 learned_addr learned my addr 172.21.15.172:0/2598279503 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.196+0000 7fd5fe59c640 1 --2- 172.21.15.172:0/2598279503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd600163670 0x7fd600163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.196+0000 7fd5fdd9b640 1 --2- 172.21.15.172:0/2598279503 >> v2:172.21.15.177:3300/0 conn(0x7fd6000b1b80 0x7fd6000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.196+0000 7fd5fed9d640 1 -- 172.21.15.172:0/2598279503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd600163670 msgr2=0x7fd600163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.196+0000 7fd5fed9d640 1 --2- 172.21.15.172:0/2598279503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd600163670 0x7fd600163a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.196+0000 7fd5fed9d640 1 -- 172.21.15.172:0/2598279503 >> v2:172.21.15.177:3300/0 conn(0x7fd6000b1b80 msgr2=0x7fd6000b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.196+0000 7fd5fed9d640 1 --2- 172.21.15.172:0/2598279503 >> v2:172.21.15.177:3300/0 conn(0x7fd6000b1b80 0x7fd6000b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.196+0000 7fd5fed9d640 1 -- 172.21.15.172:0/2598279503 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd600166b90 con 0x7fd600071620 2024-04-14T08:37:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.196+0000 7fd5fdd9b640 1 --2- 172.21.15.172:0/2598279503 >> v2:172.21.15.177:3300/0 conn(0x7fd6000b1b80 0x7fd6000b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd5fed9d640 1 --2- 172.21.15.172:0/2598279503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd600071620 0x7fd600071a90 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd60006f880 tx=0x7fd5f002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68d62f37e84f383 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd5fd59a640 1 -- 172.21.15.172:0/2598279503 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f002ca70 con 0x7fd600071620 2024-04-14T08:37:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd5fd59a640 1 -- 172.21.15.172:0/2598279503 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f002cc10 con 0x7fd600071620 2024-04-14T08:37:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd5fd59a640 1 -- 172.21.15.172:0/2598279503 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f00343f0 con 0x7fd600071620 2024-04-14T08:37:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd60491e640 1 -- 172.21.15.172:0/2598279503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd600071620 msgr2=0x7fd600071a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd60491e640 1 --2- 172.21.15.172:0/2598279503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd600071620 0x7fd600071a90 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd60006f880 tx=0x7fd5f002bed0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd60491e640 1 -- 172.21.15.172:0/2598279503 shutdown_connections 2024-04-14T08:37:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd60491e640 1 --2- 172.21.15.172:0/2598279503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd600071620 0x7fd600071a90 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd60491e640 1 --2- 172.21.15.172:0/2598279503 >> v2:172.21.15.177:3300/0 conn(0x7fd6000b1b80 0x7fd6000b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd60491e640 1 --2- 172.21.15.172:0/2598279503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd600163670 0x7fd600163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd60491e640 1 -- 172.21.15.172:0/2598279503 >> 172.21.15.172:0/2598279503 conn(0x7fd60006ec30 msgr2=0x7fd60006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd60491e640 1 -- 172.21.15.172:0/2598279503 shutdown_connections 2024-04-14T08:37:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.197+0000 7fd60491e640 1 -- 172.21.15.172:0/2598279503 wait complete. 2024-04-14T08:37:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.199+0000 7fd60491e640 1 Processor -- start 2024-04-14T08:37:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.199+0000 7fd60491e640 1 -- start start 2024-04-14T08:37:02.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.199+0000 7fd60491e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd600071620 0x7fd600151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.199+0000 7fd60491e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6000b1b80 0x7fd6000cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.199+0000 7fd60491e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd600163670 0x7fd6000cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.199+0000 7fd60491e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd6000722c0 con 0x7fd600071620 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.199+0000 7fd60491e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd600071fd0 con 0x7fd600163670 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd60491e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd6000cfe10 con 0x7fd6000b1b80 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5fe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd600071620 0x7fd600151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5fe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd600071620 0x7fd600151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55180/0 (socket says 172.21.15.172:55180) 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5fe59c640 1 -- 172.21.15.172:0/941867633 learned_addr learned my addr 172.21.15.172:0/941867633 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5fed9d640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd600163670 0x7fd6000cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5fdd9b640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6000b1b80 0x7fd6000cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5fe59c640 1 -- 172.21.15.172:0/941867633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6000b1b80 msgr2=0x7fd6000cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5fe59c640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6000b1b80 0x7fd6000cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5fe59c640 1 -- 172.21.15.172:0/941867633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd600163670 msgr2=0x7fd6000cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5fe59c640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd600163670 0x7fd6000cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5fe59c640 1 -- 172.21.15.172:0/941867633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd600151660 con 0x7fd600071620 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5fe59c640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd600071620 0x7fd600151120 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800d620 tx=0x7fd5f800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5f6ffd640 1 -- 172.21.15.172:0/941867633 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800b620 con 0x7fd600071620 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5f6ffd640 1 -- 172.21.15.172:0/941867633 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f8019070 con 0x7fd600071620 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd60491e640 1 -- 172.21.15.172:0/941867633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6000c5a50 con 0x7fd600071620 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd5f6ffd640 1 -- 172.21.15.172:0/941867633 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800ba00 con 0x7fd600071620 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.200+0000 7fd60491e640 1 -- 172.21.15.172:0/941867633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6000c6010 con 0x7fd600071620 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.201+0000 7fd60491e640 1 -- 172.21.15.172:0/941867633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd600068520 con 0x7fd600071620 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.202+0000 7fd5f6ffd640 1 -- 172.21.15.172:0/941867633 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5f800bba0 con 0x7fd600071620 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.202+0000 7fd5f6ffd640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd5e006c8f0 0x7fd5e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.202+0000 7fd5fdd9b640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd5e006c8f0 0x7fd5e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.202+0000 7fd5f6ffd640 1 -- 172.21.15.172:0/941867633 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6101+0+0 (secure 0 0 0) 0x7fd5f808c110 con 0x7fd600071620 2024-04-14T08:37:02.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.203+0000 7fd5fdd9b640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd5e006c8f0 0x7fd5e006edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8006fd0 tx=0x7fd5e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:02.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.207+0000 7fd5f6ffd640 1 -- 172.21.15.172:0/941867633 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd5f805a3b0 con 0x7fd600071620 2024-04-14T08:37:02.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.383+0000 7fd60491e640 1 -- 172.21.15.172:0/941867633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd60006c2c0 con 0x7fd600071620 2024-04-14T08:37:02.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.384+0000 7fd5f6ffd640 1 -- 172.21.15.172:0/941867633 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+17018 (secure 0 0 0) 0x7fd5f8059cb0 con 0x7fd600071620 2024-04-14T08:37:02.385 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:02.386 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":73,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:37:01.114727+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:36:59.342045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-14T08:37:00.059338+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:37:02.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.386+0000 7fd60491e640 1 -- 172.21.15.172:0/941867633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd5e006c8f0 msgr2=0x7fd5e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.386+0000 7fd60491e640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd5e006c8f0 0x7fd5e006edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8006fd0 tx=0x7fd5e8008040 comp rx=0 tx=0).stop 2024-04-14T08:37:02.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.386+0000 7fd60491e640 1 -- 172.21.15.172:0/941867633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd600071620 msgr2=0x7fd600151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.386+0000 7fd60491e640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd600071620 0x7fd600151120 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800d620 tx=0x7fd5f800dae0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.386+0000 7fd60491e640 1 -- 172.21.15.172:0/941867633 shutdown_connections 2024-04-14T08:37:02.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.386+0000 7fd60491e640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd5e006c8f0 0x7fd5e006edb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.386+0000 7fd60491e640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd600163670 0x7fd6000cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.386+0000 7fd60491e640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6000b1b80 0x7fd6000cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.386+0000 7fd60491e640 1 --2- 172.21.15.172:0/941867633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd600071620 0x7fd600151120 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.386+0000 7fd60491e640 1 -- 172.21.15.172:0/941867633 >> 172.21.15.172:0/941867633 conn(0x7fd60006ec30 msgr2=0x7fd60006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:02.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.386+0000 7fd60491e640 1 -- 172.21.15.172:0/941867633 shutdown_connections 2024-04-14T08:37:02.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.386+0000 7fd60491e640 1 -- 172.21.15.172:0/941867633 wait complete. 2024-04-14T08:37:02.398 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-14T08:37:02.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.501+0000 7f8bb678c640 1 Processor -- start 2024-04-14T08:37:02.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.501+0000 7f8bb678c640 1 -- start start 2024-04-14T08:37:02.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8bb678c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8bb0144ec0 0x7f8bb0145640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8bb678c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bb01452a0 con 0x7f8bb0071a20 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8bb678c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bb00cecc0 con 0x7f8bb00ce7c0 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8bb678c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f8bb00cefa0 con 0x7f8bb0144ec0 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8bb4d29640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f8bb0071a20 0x7f8bb006d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58506/0 (socket says 172.21.15.172:58506) 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8bb4d29640 1 -- 172.21.15.172:0/2665253187 learned_addr learned my addr 172.21.15.172:0/2665253187 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8baffff640 1 --2- 172.21.15.172:0/2665253187 >> v2:172.21.15.177:3300/0 conn(0x7f8bb0144ec0 0x7f8bb0145640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131413045 0 0) 0x7f8bb01452a0 con 0x7f8bb0071a20 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b98003520 con 0x7f8bb0071a20 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 182272381 0 0) 0x7f8b98003520 con 0x7f8bb0071a20 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 >> v2:172.21.15.177:3300/0 conn(0x7f8bb0144ec0 msgr2=0x7f8bb0145640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8baeffd640 1 --2- 172.21.15.172:0/2665253187 >> v2:172.21.15.177:3300/0 conn(0x7f8bb0144ec0 0x7f8bb0145640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 >> v1:172.21.15.177:6789/0 conn(0x7f8bb00ce7c0 legacy=0x7f8bb0071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb00cf280 con 0x7f8bb0071a20 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f8ba80019f0 con 0x7f8bb0071a20 2024-04-14T08:37:02.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.502+0000 7f8baffff640 1 --2- 172.21.15.172:0/2665253187 >> v2:172.21.15.177:3300/0 conn(0x7f8bb0144ec0 0x7f8bb0145640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 >> v1:172.21.15.172:6789/0 conn(0x7f8bb0071a20 legacy=0x7f8bb006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8baeffd640 1 --2- 172.21.15.172:0/2665253187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b980039b0 0x7f8b98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8baeffd640 1 --2- 172.21.15.172:0/2665253187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b98004800 0x7f8b98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8baeffd640 1 --2- 172.21.15.172:0/2665253187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b980051c0 0x7f8b9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb00cefa0 con 0x7f8b980051c0 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9800c060 con 0x7f8b980039b0 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8b9800c260 con 0x7f8b98004800 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8bb4d29640 1 --2- 172.21.15.172:0/2665253187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b980051c0 0x7f8b9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8baffff640 1 --2- 172.21.15.172:0/2665253187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b980039b0 0x7f8b98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8baf7fe640 1 --2- 172.21.15.172:0/2665253187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b98004800 0x7f8b98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8bb4d29640 1 -- 172.21.15.172:0/2665253187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b98004800 msgr2=0x7f8b98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8bb4d29640 1 --2- 172.21.15.172:0/2665253187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b98004800 0x7f8b98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8bb4d29640 1 -- 172.21.15.172:0/2665253187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b980039b0 msgr2=0x7f8b98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:02.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8bb4d29640 1 --2- 172.21.15.172:0/2665253187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b980039b0 0x7f8b98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.503+0000 7f8bb4d29640 1 -- 172.21.15.172:0/2665253187 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b98003520 con 0x7f8b980051c0 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.504+0000 7f8bb4d29640 1 -- 172.21.15.172:0/2665253187 reap_dead start 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.504+0000 7f8bb4d29640 1 --2- 172.21.15.172:0/2665253187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b980051c0 0x7f8b9800b9c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800a840 tx=0x7f8b98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67bf3d28d9cf0e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.504+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba800a460 con 0x7f8b980051c0 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.504+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba8034650 con 0x7f8b980051c0 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.504+0000 7f8baeffd640 1 -- 172.21.15.172:0/2665253187 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8034a40 con 0x7f8b980051c0 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.504+0000 7f8bb678c640 1 -- 172.21.15.172:0/2665253187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b980051c0 msgr2=0x7f8b9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.504+0000 7f8bb678c640 1 --2- 172.21.15.172:0/2665253187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b980051c0 0x7f8b9800b9c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800a840 tx=0x7f8b98005830 comp rx=0 tx=0).stop 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.504+0000 7f8bb678c640 1 -- 172.21.15.172:0/2665253187 shutdown_connections 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.504+0000 7f8bb678c640 1 --2- 172.21.15.172:0/2665253187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b980051c0 0x7f8b9800b9c0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.504+0000 7f8bb678c640 1 -- 172.21.15.172:0/2665253187 >> 172.21.15.172:0/2665253187 conn(0x7f8bb00b1c60 msgr2=0x7f8bb00b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.504+0000 7f8bb678c640 1 -- 172.21.15.172:0/2665253187 shutdown_connections 2024-04-14T08:37:02.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.504+0000 7f8bb678c640 1 -- 172.21.15.172:0/2665253187 wait complete. 2024-04-14T08:37:02.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.506+0000 7f8bb678c640 1 Processor -- start 2024-04-14T08:37:02.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.506+0000 7f8bb678c640 1 -- start start 2024-04-14T08:37:02.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.506+0000 7f8bb678c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b980051c0 0x7f8bb01f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.506+0000 7f8bb678c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb015ea80 0x7f8bb015ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.506+0000 7f8bb678c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb015c7b0 0x7f8bb015cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.506+0000 7f8bb678c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0071490 con 0x7f8bb015c7b0 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.506+0000 7f8bb678c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0071310 con 0x7f8bb015ea80 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.506+0000 7f8bb678c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb0071610 con 0x7f8b980051c0 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bb4d29640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb015c7b0 0x7f8bb015cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bb4d29640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb015c7b0 0x7f8bb015cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55200/0 (socket says 172.21.15.172:55200) 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bb4d29640 1 -- 172.21.15.172:0/3828567359 learned_addr learned my addr 172.21.15.172:0/3828567359 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8baffff640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b980051c0 0x7f8bb01f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8baf7fe640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb015ea80 0x7f8bb015ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bb4d29640 1 -- 172.21.15.172:0/3828567359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b980051c0 msgr2=0x7f8bb01f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bb4d29640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b980051c0 0x7f8bb01f2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bb4d29640 1 -- 172.21.15.172:0/3828567359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb015ea80 msgr2=0x7f8bb015ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bb4d29640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb015ea80 0x7f8bb015ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bb4d29640 1 -- 172.21.15.172:0/3828567359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb015f640 con 0x7f8bb015c7b0 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8baf7fe640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb015ea80 0x7f8bb015ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:02.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bb4d29640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb015c7b0 0x7f8bb015cc40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800a930 tx=0x7f8bb00cf450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:02.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bacff9640 1 -- 172.21.15.172:0/3828567359 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8043070 con 0x7f8bb015c7b0 2024-04-14T08:37:02.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bacff9640 1 -- 172.21.15.172:0/3828567359 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba8001250 con 0x7f8bb015c7b0 2024-04-14T08:37:02.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bb678c640 1 -- 172.21.15.172:0/3828567359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb015f920 con 0x7f8bb015c7b0 2024-04-14T08:37:02.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bacff9640 1 -- 172.21.15.172:0/3828567359 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8039040 con 0x7f8bb015c7b0 2024-04-14T08:37:02.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.507+0000 7f8bb678c640 1 -- 172.21.15.172:0/3828567359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb015fee0 con 0x7f8bb015c7b0 2024-04-14T08:37:02.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.508+0000 7f8bb678c640 1 -- 172.21.15.172:0/3828567359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb0068520 con 0x7f8bb015c7b0 2024-04-14T08:37:02.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.509+0000 7f8bacff9640 1 -- 172.21.15.172:0/3828567359 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ba8046070 con 0x7f8bb015c7b0 2024-04-14T08:37:02.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.509+0000 7f8bacff9640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8b9c06c920 0x7f8b9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.509+0000 7f8bacff9640 1 -- 172.21.15.172:0/3828567359 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6101+0+0 (secure 0 0 0) 0x7f8ba80acf70 con 0x7f8bb015c7b0 2024-04-14T08:37:02.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.509+0000 7f8baffff640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8b9c06c920 0x7f8b9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.510+0000 7f8baffff640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8b9c06c920 0x7f8b9c06ede0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8bb01e8a50 tx=0x7f8ba0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:02.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.514+0000 7f8bacff9640 1 -- 172.21.15.172:0/3828567359 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8ba807abc0 con 0x7f8bb015c7b0 2024-04-14T08:37:02.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.724+0000 7f8bb678c640 1 -- 172.21.15.172:0/3828567359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8bb01437b0 con 0x7f8bb015c7b0 2024-04-14T08:37:02.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.759+0000 7f8bacff9640 1 -- 172.21.15.172:0/3828567359 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) ==== 118+0+0 (secure 0 0 0) 0x7f8ba807eec0 con 0x7f8bb015c7b0 2024-04-14T08:37:02.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.762+0000 7f8bb678c640 1 -- 172.21.15.172:0/3828567359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8b9c06c920 msgr2=0x7f8b9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.762+0000 7f8bb678c640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8b9c06c920 0x7f8b9c06ede0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8bb01e8a50 tx=0x7f8ba0008040 comp rx=0 tx=0).stop 2024-04-14T08:37:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.762+0000 7f8bb678c640 1 -- 172.21.15.172:0/3828567359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb015c7b0 msgr2=0x7f8bb015cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.762+0000 7f8bb678c640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb015c7b0 0x7f8bb015cc40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800a930 tx=0x7f8bb00cf450 comp rx=0 tx=0).stop 2024-04-14T08:37:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.762+0000 7f8bb678c640 1 -- 172.21.15.172:0/3828567359 shutdown_connections 2024-04-14T08:37:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.762+0000 7f8bb678c640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8b9c06c920 0x7f8b9c06ede0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.762+0000 7f8bb678c640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb015c7b0 0x7f8bb015cc40 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.762+0000 7f8bb678c640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb015ea80 0x7f8bb015ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.762+0000 7f8bb678c640 1 --2- 172.21.15.172:0/3828567359 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b980051c0 0x7f8bb01f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.762+0000 7f8bb678c640 1 -- 172.21.15.172:0/3828567359 >> 172.21.15.172:0/3828567359 conn(0x7f8bb00b1c60 msgr2=0x7f8bb00b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.762+0000 7f8bb678c640 1 -- 172.21.15.172:0/3828567359 shutdown_connections 2024-04-14T08:37:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.762+0000 7f8bb678c640 1 -- 172.21.15.172:0/3828567359 wait complete. 2024-04-14T08:37:02.778 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-14T08:37:02.778 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-14T08:37:02.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.885+0000 7fc2d2f08640 1 Processor -- start 2024-04-14T08:37:02.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.885+0000 7fc2d2f08640 1 -- start start 2024-04-14T08:37:02.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.885+0000 7fc2d2f08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2cc0d2f10 0x7fc2cc0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.885+0000 7fc2d2f08640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc2cc0d1150 0x7fc2cc0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2d2f08640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2cc0cedc0 con 0x7fc2cc161ce0 2024-04-14T08:37:02.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2d2f08640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc2cc0d15e0 con 0x7fc2cc0d1150 2024-04-14T08:37:02.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2d2f08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc2cc0d00e0 con 0x7fc2cc0d2f10 2024-04-14T08:37:02.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2d14a5640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc2cc0d1150 0x7fc2cc0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2cbfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2cc0d2f10 0x7fc2cc0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2d14a5640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc2cc0d1150 0x7fc2cc0b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:32824/0 (socket says 172.21.15.172:32824) 2024-04-14T08:37:02.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2cbfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2cc0d2f10 0x7fc2cc0d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:37714/0 (socket says 172.21.15.172:37714) 2024-04-14T08:37:02.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2d14a5640 1 -- 172.21.15.172:0/2362086050 learned_addr learned my addr 172.21.15.172:0/2362086050 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:02.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2d0ca4640 1 --1- 172.21.15.172:0/2362086050 >> v1:172.21.15.177:6789/0 conn(0x7fc2cc161ce0 0x7fc2cc0d2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:49446/0 (socket says 172.21.15.172:49446) 2024-04-14T08:37:02.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2cbfff640 1 -- 172.21.15.172:0/2362086050 >> v2:172.21.15.177:3300/0 conn(0x7fc2cc0d1150 msgr2=0x7fc2cc0b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:02.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2cbfff640 1 --2- 172.21.15.172:0/2362086050 >> v2:172.21.15.177:3300/0 conn(0x7fc2cc0d1150 0x7fc2cc0b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2cbfff640 1 -- 172.21.15.172:0/2362086050 >> v1:172.21.15.177:6789/0 conn(0x7fc2cc161ce0 legacy=0x7fc2cc0d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2cbfff640 1 -- 172.21.15.172:0/2362086050 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc0cf040 con 0x7fc2cc0d2f10 2024-04-14T08:37:02.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.886+0000 7fc2d14a5640 1 --2- 172.21.15.172:0/2362086050 >> v2:172.21.15.177:3300/0 conn(0x7fc2cc0d1150 0x7fc2cc0b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:37:02.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.887+0000 7fc2cbfff640 1 --2- 172.21.15.172:0/2362086050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2cc0d2f10 0x7fc2cc0d0c10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80097a0 tx=0x7fc2b802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2063c5806b33a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:02.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.887+0000 7fc2cb7fe640 1 -- 172.21.15.172:0/2362086050 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8035020 con 0x7fc2cc0d2f10 2024-04-14T08:37:02.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.887+0000 7fc2cb7fe640 1 -- 172.21.15.172:0/2362086050 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b8031830 con 0x7fc2cc0d2f10 2024-04-14T08:37:02.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.887+0000 7fc2cb7fe640 1 -- 172.21.15.172:0/2362086050 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8031c00 con 0x7fc2cc0d2f10 2024-04-14T08:37:02.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.888+0000 7fc2d2f08640 1 -- 172.21.15.172:0/2362086050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2cc0d2f10 msgr2=0x7fc2cc0d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.888+0000 7fc2d2f08640 1 --2- 172.21.15.172:0/2362086050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2cc0d2f10 0x7fc2cc0d0c10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80097a0 tx=0x7fc2b802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.888+0000 7fc2d2f08640 1 -- 172.21.15.172:0/2362086050 shutdown_connections 2024-04-14T08:37:02.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.888+0000 7fc2d2f08640 1 --2- 172.21.15.172:0/2362086050 >> v2:172.21.15.177:3300/0 conn(0x7fc2cc0d1150 0x7fc2cc0b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.888+0000 7fc2d2f08640 1 --2- 172.21.15.172:0/2362086050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2cc0d2f10 0x7fc2cc0d0c10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.888+0000 7fc2d2f08640 1 -- 172.21.15.172:0/2362086050 >> 172.21.15.172:0/2362086050 conn(0x7fc2cc09d560 msgr2=0x7fc2cc0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:02.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.888+0000 7fc2d2f08640 1 -- 172.21.15.172:0/2362086050 shutdown_connections 2024-04-14T08:37:02.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.888+0000 7fc2d2f08640 1 -- 172.21.15.172:0/2362086050 wait complete. 2024-04-14T08:37:02.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.890+0000 7fc2d2f08640 1 Processor -- start 2024-04-14T08:37:02.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.890+0000 7fc2d2f08640 1 -- start start 2024-04-14T08:37:02.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.891+0000 7fc2d2f08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2cc0d1150 0x7fc2cc1f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.891+0000 7fc2d2f08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2cc0d2f10 0x7fc2cc0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.891+0000 7fc2d2f08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2cc161ce0 0x7fc2cc150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.891+0000 7fc2d2f08640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc0cf8e0 con 0x7fc2cc0d1150 2024-04-14T08:37:02.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.891+0000 7fc2d2f08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc0cf600 con 0x7fc2cc0d2f10 2024-04-14T08:37:02.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.891+0000 7fc2d2f08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc2cc0cfbc0 con 0x7fc2cc161ce0 2024-04-14T08:37:02.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.891+0000 7fc2d0ca4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2cc0d1150 0x7fc2cc1f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.891+0000 7fc2d0ca4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2cc0d1150 0x7fc2cc1f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55212/0 (socket says 172.21.15.172:55212) 2024-04-14T08:37:02.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.891+0000 7fc2d0ca4640 1 -- 172.21.15.172:0/1996261485 learned_addr learned my addr 172.21.15.172:0/1996261485 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:02.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.891+0000 7fc2cbfff640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2cc0d2f10 0x7fc2cc0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.892+0000 7fc2d14a5640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2cc161ce0 0x7fc2cc150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.892+0000 7fc2d0ca4640 1 -- 172.21.15.172:0/1996261485 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2cc161ce0 msgr2=0x7fc2cc150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.892+0000 7fc2d0ca4640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2cc161ce0 0x7fc2cc150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.892+0000 7fc2d0ca4640 1 -- 172.21.15.172:0/1996261485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2cc0d2f10 msgr2=0x7fc2cc0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:02.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.892+0000 7fc2d0ca4640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2cc0d2f10 0x7fc2cc0cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:02.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.892+0000 7fc2d0ca4640 1 -- 172.21.15.172:0/1996261485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc0ce140 con 0x7fc2cc0d1150 2024-04-14T08:37:02.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.892+0000 7fc2d0ca4640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2cc0d1150 0x7fc2cc1f4480 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000c3c0 tx=0x7fc2c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:02.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.892+0000 7fc2c97fa640 1 -- 172.21.15.172:0/1996261485 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00074e0 con 0x7fc2cc0d1150 2024-04-14T08:37:02.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.892+0000 7fc2c97fa640 1 -- 172.21.15.172:0/1996261485 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c0007680 con 0x7fc2cc0d1150 2024-04-14T08:37:02.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.892+0000 7fc2d2f08640 1 -- 172.21.15.172:0/1996261485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc0ce420 con 0x7fc2cc0d1150 2024-04-14T08:37:02.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.892+0000 7fc2c97fa640 1 -- 172.21.15.172:0/1996261485 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0004a00 con 0x7fc2cc0d1150 2024-04-14T08:37:02.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.892+0000 7fc2d2f08640 1 -- 172.21.15.172:0/1996261485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc0c9380 con 0x7fc2cc0d1150 2024-04-14T08:37:02.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.893+0000 7fc2d2f08640 1 -- 172.21.15.172:0/1996261485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2cc075490 con 0x7fc2cc0d1150 2024-04-14T08:37:02.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.894+0000 7fc2c97fa640 1 -- 172.21.15.172:0/1996261485 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2c001a020 con 0x7fc2cc0d1150 2024-04-14T08:37:02.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.894+0000 7fc2c97fa640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc2b006c920 0x7fc2b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:02.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.894+0000 7fc2c97fa640 1 -- 172.21.15.172:0/1996261485 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fc2c008b340 con 0x7fc2cc0d1150 2024-04-14T08:37:02.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.894+0000 7fc2cbfff640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc2b006c920 0x7fc2b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:02.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.895+0000 7fc2cbfff640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc2b006c920 0x7fc2b006ede0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8036000 tx=0x7fc2b803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:02.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:02.899+0000 7fc2c97fa640 1 -- 172.21.15.172:0/1996261485 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc2cc0cedc0 con 0x7fc2cc0d1150 2024-04-14T08:37:03.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.118+0000 7fc2d2f08640 1 -- 172.21.15.172:0/1996261485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc2cc0d25e0 con 0x7fc2cc0d1150 2024-04-14T08:37:03.739 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:37:03.737+0000 7f13d59f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:37:03.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.766+0000 7fc2c97fa640 1 -- 172.21.15.172:0/1996261485 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v39) ==== 122+0+0 (secure 0 0 0) 0x7fc2c0058fa0 con 0x7fc2cc0d1150 2024-04-14T08:37:03.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.768+0000 7fc2d2f08640 1 -- 172.21.15.172:0/1996261485 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc2b006c920 msgr2=0x7fc2b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:03.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.768+0000 7fc2d2f08640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc2b006c920 0x7fc2b006ede0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8036000 tx=0x7fc2b803c040 comp rx=0 tx=0).stop 2024-04-14T08:37:03.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.768+0000 7fc2d2f08640 1 -- 172.21.15.172:0/1996261485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2cc0d1150 msgr2=0x7fc2cc1f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:03.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.768+0000 7fc2d2f08640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2cc0d1150 0x7fc2cc1f4480 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000c3c0 tx=0x7fc2c000c880 comp rx=0 tx=0).stop 2024-04-14T08:37:03.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.768+0000 7fc2d2f08640 1 -- 172.21.15.172:0/1996261485 shutdown_connections 2024-04-14T08:37:03.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.768+0000 7fc2d2f08640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc2b006c920 0x7fc2b006ede0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:03.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.768+0000 7fc2d2f08640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2cc161ce0 0x7fc2cc150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:03.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.768+0000 7fc2d2f08640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2cc0d2f10 0x7fc2cc0cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:03.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.768+0000 7fc2d2f08640 1 --2- 172.21.15.172:0/1996261485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2cc0d1150 0x7fc2cc1f4480 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:03.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.768+0000 7fc2d2f08640 1 -- 172.21.15.172:0/1996261485 >> 172.21.15.172:0/1996261485 conn(0x7fc2cc09d560 msgr2=0x7fc2cc0b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:03.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.768+0000 7fc2d2f08640 1 -- 172.21.15.172:0/1996261485 shutdown_connections 2024-04-14T08:37:03.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.768+0000 7fc2d2f08640 1 -- 172.21.15.172:0/1996261485 wait complete. 2024-04-14T08:37:03.781 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-14T08:37:03.781 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:37:03.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.882+0000 7fcc68ede640 1 Processor -- start 2024-04-14T08:37:03.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.882+0000 7fcc68ede640 1 -- start start 2024-04-14T08:37:03.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.883+0000 7fcc68ede640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcc640cefd0 0x7fcc6406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:03.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.883+0000 7fcc68ede640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc640d2b10 con 0x7fcc640b7a60 2024-04-14T08:37:03.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.883+0000 7fcc68ede640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc640ce420 con 0x7fcc6408c4c0 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.883+0000 7fcc68ede640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fcc640d0ad0 con 0x7fcc640cefd0 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.883+0000 7fcc6259c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fcc640b7a60 0x7fcc640ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58516/0 (socket says 172.21.15.172:58516) 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.883+0000 7fcc6259c640 1 -- 172.21.15.172:0/1377003322 learned_addr learned my addr 172.21.15.172:0/1377003322 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.883+0000 7fcc61d9b640 1 --2- 172.21.15.172:0/1377003322 >> v2:172.21.15.177:3300/0 conn(0x7fcc640cefd0 0x7fcc6406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.883+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1403462857 0 0) 0x7fcc640d2b10 con 0x7fcc640b7a60 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.883+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc4c003520 con 0x7fcc640b7a60 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.883+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 120305362 0 0) 0x7fcc4c003520 con 0x7fcc640b7a60 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.883+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 >> v2:172.21.15.177:3300/0 conn(0x7fcc640cefd0 msgr2=0x7fcc6406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.883+0000 7fcc6159a640 1 --2- 172.21.15.172:0/1377003322 >> v2:172.21.15.177:3300/0 conn(0x7fcc640cefd0 0x7fcc6406cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 >> v1:172.21.15.177:6789/0 conn(0x7fcc6408c4c0 legacy=0x7fcc6406d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc640d0db0 con 0x7fcc640b7a60 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fcc5c001670 con 0x7fcc640b7a60 2024-04-14T08:37:03.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 >> v1:172.21.15.172:6789/0 conn(0x7fcc640b7a60 legacy=0x7fcc640ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc6159a640 1 --2- 172.21.15.172:0/1377003322 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc4c0039b0 0x7fcc4c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc6159a640 1 --2- 172.21.15.172:0/1377003322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc4c0043b0 0x7fcc4c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc6159a640 1 --2- 172.21.15.172:0/1377003322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c005120 0x7fcc4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc640d0ad0 con 0x7fcc4c005120 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c00c060 con 0x7fcc4c0043b0 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4c00c260 con 0x7fcc4c0039b0 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc62d9d640 1 --2- 172.21.15.172:0/1377003322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c005120 0x7fcc4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc61d9b640 1 --2- 172.21.15.172:0/1377003322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc4c0043b0 0x7fcc4c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc6259c640 1 --2- 172.21.15.172:0/1377003322 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc4c0039b0 0x7fcc4c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc62d9d640 1 -- 172.21.15.172:0/1377003322 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc4c0039b0 msgr2=0x7fcc4c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc62d9d640 1 --2- 172.21.15.172:0/1377003322 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc4c0039b0 0x7fcc4c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc62d9d640 1 -- 172.21.15.172:0/1377003322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc4c0043b0 msgr2=0x7fcc4c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc62d9d640 1 --2- 172.21.15.172:0/1377003322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc4c0043b0 0x7fcc4c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:03.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.884+0000 7fcc62d9d640 1 -- 172.21.15.172:0/1377003322 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c003520 con 0x7fcc4c005120 2024-04-14T08:37:03.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.885+0000 7fcc62d9d640 1 -- 172.21.15.172:0/1377003322 reap_dead start 2024-04-14T08:37:03.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.885+0000 7fcc62d9d640 1 --2- 172.21.15.172:0/1377003322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c005120 0x7fcc4c00b9c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fcc6408c8d0 tx=0x7fcc4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a387aeb7b745bb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:03.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.885+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54001440 con 0x7fcc4c005120 2024-04-14T08:37:03.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.885+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc540052f0 con 0x7fcc4c005120 2024-04-14T08:37:03.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.885+0000 7fcc6159a640 1 -- 172.21.15.172:0/1377003322 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54003830 con 0x7fcc4c005120 2024-04-14T08:37:03.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.885+0000 7fcc68ede640 1 -- 172.21.15.172:0/1377003322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c005120 msgr2=0x7fcc4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:03.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.885+0000 7fcc68ede640 1 --2- 172.21.15.172:0/1377003322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c005120 0x7fcc4c00b9c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fcc6408c8d0 tx=0x7fcc4c005830 comp rx=0 tx=0).stop 2024-04-14T08:37:03.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.886+0000 7fcc68ede640 1 -- 172.21.15.172:0/1377003322 shutdown_connections 2024-04-14T08:37:03.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.886+0000 7fcc68ede640 1 --2- 172.21.15.172:0/1377003322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c005120 0x7fcc4c00b9c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:03.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.886+0000 7fcc68ede640 1 -- 172.21.15.172:0/1377003322 >> 172.21.15.172:0/1377003322 conn(0x7fcc6408c000 msgr2=0x7fcc640d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:03.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.886+0000 7fcc68ede640 1 -- 172.21.15.172:0/1377003322 shutdown_connections 2024-04-14T08:37:03.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.886+0000 7fcc68ede640 1 -- 172.21.15.172:0/1377003322 wait complete. 2024-04-14T08:37:03.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.887+0000 7fcc68ede640 1 Processor -- start 2024-04-14T08:37:03.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.887+0000 7fcc68ede640 1 -- start start 2024-04-14T08:37:03.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc68ede640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc4c005120 0x7fcc640c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:03.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc68ede640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6414d050 0x7fcc6414c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc68ede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc6415ed70 0x7fcc6415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc68ede640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc640b7e40 con 0x7fcc6414d050 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc68ede640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcc640b4e90 con 0x7fcc4c005120 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc68ede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcc640d2750 con 0x7fcc6415ed70 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc61d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6414d050 0x7fcc6414c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc61d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6414d050 0x7fcc6414c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55226/0 (socket says 172.21.15.172:55226) 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc61d9b640 1 -- 172.21.15.172:0/11627506 learned_addr learned my addr 172.21.15.172:0/11627506 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc6259c640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc4c005120 0x7fcc640c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc62d9d640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc6415ed70 0x7fcc6415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc61d9b640 1 -- 172.21.15.172:0/11627506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc6415ed70 msgr2=0x7fcc6415f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc61d9b640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc6415ed70 0x7fcc6415f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc61d9b640 1 -- 172.21.15.172:0/11627506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc4c005120 msgr2=0x7fcc640c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc61d9b640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc4c005120 0x7fcc640c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc61d9b640 1 -- 172.21.15.172:0/11627506 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc641f7850 con 0x7fcc6414d050 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc61d9b640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6414d050 0x7fcc6414c7a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009fc0 tx=0x7fcc5800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc52ffd640 1 -- 172.21.15.172:0/11627506 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58015430 con 0x7fcc6414d050 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc52ffd640 1 -- 172.21.15.172:0/11627506 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc580155d0 con 0x7fcc6414d050 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc52ffd640 1 -- 172.21.15.172:0/11627506 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc580159c0 con 0x7fcc6414d050 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc68ede640 1 -- 172.21.15.172:0/11627506 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc641f7b30 con 0x7fcc6414d050 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.888+0000 7fcc68ede640 1 -- 172.21.15.172:0/11627506 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc641f80f0 con 0x7fcc6414d050 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.890+0000 7fcc68ede640 1 -- 172.21.15.172:0/11627506 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc6406e320 con 0x7fcc6414d050 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.890+0000 7fcc52ffd640 1 -- 172.21.15.172:0/11627506 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc58015b60 con 0x7fcc6414d050 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.890+0000 7fcc52ffd640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcc4406c8f0 0x7fcc4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.890+0000 7fcc52ffd640 1 -- 172.21.15.172:0/11627506 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fcc5808cdf0 con 0x7fcc6414d050 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.891+0000 7fcc6259c640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcc4406c8f0 0x7fcc4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:03.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.891+0000 7fcc6259c640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcc4406c8f0 0x7fcc4406edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c001730 tx=0x7fcc5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:03.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:03.895+0000 7fcc52ffd640 1 -- 172.21.15.172:0/11627506 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcc6406e320 con 0x7fcc6414d050 2024-04-14T08:37:04.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.121+0000 7fcc68ede640 1 -- 172.21.15.172:0/11627506 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc64075490 con 0x7fcc6414d050 2024-04-14T08:37:04.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.122+0000 7fcc52ffd640 1 -- 172.21.15.172:0/11627506 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5192 (secure 0 0 0) 0x7fcc58019030 con 0x7fcc6414d050 2024-04-14T08:37:04.123 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:04.124 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:37:01.115031+0000","modified":"2024-04-14T08:37:03.119917+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6753":{"gid":6753,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6837/2134832221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2134832221},{"type":"v1","addr":"172.21.15.172:6837","nonce":2134832221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6753,"qdb_cluster":[6753]},"id":5}]} 2024-04-14T08:37:04.124 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 39 2024-04-14T08:37:04.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.124+0000 7fcc68ede640 1 -- 172.21.15.172:0/11627506 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcc4406c8f0 msgr2=0x7fcc4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.124+0000 7fcc68ede640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcc4406c8f0 0x7fcc4406edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c001730 tx=0x7fcc5c009040 comp rx=0 tx=0).stop 2024-04-14T08:37:04.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.124+0000 7fcc68ede640 1 -- 172.21.15.172:0/11627506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6414d050 msgr2=0x7fcc6414c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.124+0000 7fcc68ede640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6414d050 0x7fcc6414c7a0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009fc0 tx=0x7fcc5800f580 comp rx=0 tx=0).stop 2024-04-14T08:37:04.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.124+0000 7fcc68ede640 1 -- 172.21.15.172:0/11627506 shutdown_connections 2024-04-14T08:37:04.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.124+0000 7fcc68ede640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcc4406c8f0 0x7fcc4406edb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.124+0000 7fcc68ede640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc6415ed70 0x7fcc6415f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.124+0000 7fcc68ede640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc6414d050 0x7fcc6414c7a0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.124+0000 7fcc68ede640 1 --2- 172.21.15.172:0/11627506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc4c005120 0x7fcc640c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.124+0000 7fcc68ede640 1 -- 172.21.15.172:0/11627506 >> 172.21.15.172:0/11627506 conn(0x7fcc6408c000 msgr2=0x7fcc6406f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:04.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.124+0000 7fcc68ede640 1 -- 172.21.15.172:0/11627506 shutdown_connections 2024-04-14T08:37:04.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.124+0000 7fcc68ede640 1 -- 172.21.15.172:0/11627506 wait complete. 2024-04-14T08:37:04.136 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:37:04.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.237+0000 7f6d677dc640 1 Processor -- start 2024-04-14T08:37:04.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.237+0000 7f6d677dc640 1 -- start start 2024-04-14T08:37:04.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.238+0000 7f6d677dc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6d600ce120 0x7f6d6009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.238+0000 7f6d677dc640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d600d1050 con 0x7f6d600703a0 2024-04-14T08:37:04.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.238+0000 7f6d677dc640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d600d1330 con 0x7f6d600d2ba0 2024-04-14T08:37:04.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.238+0000 7f6d677dc640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f6d600d1610 con 0x7f6d600ce120 2024-04-14T08:37:04.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.238+0000 7f6d65d79640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6d600703a0 0x7f6d6006d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58518/0 (socket says 172.21.15.172:58518) 2024-04-14T08:37:04.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.238+0000 7f6d65d79640 1 -- 172.21.15.172:0/1063298481 learned_addr learned my addr 172.21.15.172:0/1063298481 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:04.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d65578640 1 --2- 172.21.15.172:0/1063298481 >> v2:172.21.15.177:3300/0 conn(0x7f6d600ce120 0x7f6d6009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1066593034 0 0) 0x7f6d600d1050 con 0x7f6d600703a0 2024-04-14T08:37:04.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d4c003520 con 0x7f6d600703a0 2024-04-14T08:37:04.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2337129553 0 0) 0x7f6d4c003520 con 0x7f6d600703a0 2024-04-14T08:37:04.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 >> v2:172.21.15.177:3300/0 conn(0x7f6d600ce120 msgr2=0x7f6d6009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 --2- 172.21.15.172:0/1063298481 >> v2:172.21.15.177:3300/0 conn(0x7f6d600ce120 0x7f6d6009aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 >> v1:172.21.15.177:6789/0 conn(0x7f6d600d2ba0 legacy=0x7f6d6006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6006c7d0 con 0x7f6d600703a0 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f6d580019f0 con 0x7f6d600703a0 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 >> v1:172.21.15.172:6789/0 conn(0x7f6d600703a0 legacy=0x7f6d6006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 --2- 172.21.15.172:0/1063298481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d4c0039b0 0x7f6d4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 --2- 172.21.15.172:0/1063298481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4c004800 0x7f6d4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 --2- 172.21.15.172:0/1063298481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4c0051c0 0x7f6d4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c00c030 con 0x7f6d4c004800 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c00c1e0 con 0x7f6d4c0051c0 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6d4c00c3e0 con 0x7f6d4c0039b0 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d64d77640 1 --2- 172.21.15.172:0/1063298481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4c004800 0x7f6d4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d65578640 1 --2- 172.21.15.172:0/1063298481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d4c0039b0 0x7f6d4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d65d79640 1 --2- 172.21.15.172:0/1063298481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4c0051c0 0x7f6d4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d64d77640 1 -- 172.21.15.172:0/1063298481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d4c0039b0 msgr2=0x7f6d4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:04.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d64d77640 1 --2- 172.21.15.172:0/1063298481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d4c0039b0 0x7f6d4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d64d77640 1 -- 172.21.15.172:0/1063298481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4c0051c0 msgr2=0x7f6d4c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:04.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d64d77640 1 --2- 172.21.15.172:0/1063298481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4c0051c0 0x7f6d4c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d65d79640 1 -- 172.21.15.172:0/1063298481 reap_dead start 2024-04-14T08:37:04.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.239+0000 7f6d64d77640 1 -- 172.21.15.172:0/1063298481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c003520 con 0x7f6d4c004800 2024-04-14T08:37:04.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.241+0000 7f6d64d77640 1 --2- 172.21.15.172:0/1063298481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4c004800 0x7f6d4c004c80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6d50001750 tx=0x7f6d4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef802761cf365022 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:04.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.241+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50001370 con 0x7f6d4c004800 2024-04-14T08:37:04.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.241+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d500052d0 con 0x7f6d4c004800 2024-04-14T08:37:04.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.241+0000 7f6d57fff640 1 -- 172.21.15.172:0/1063298481 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50003830 con 0x7f6d4c004800 2024-04-14T08:37:04.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.241+0000 7f6d677dc640 1 -- 172.21.15.172:0/1063298481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4c004800 msgr2=0x7f6d4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.241+0000 7f6d677dc640 1 --2- 172.21.15.172:0/1063298481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4c004800 0x7f6d4c004c80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6d50001750 tx=0x7f6d4c005830 comp rx=0 tx=0).stop 2024-04-14T08:37:04.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.241+0000 7f6d677dc640 1 -- 172.21.15.172:0/1063298481 shutdown_connections 2024-04-14T08:37:04.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.241+0000 7f6d677dc640 1 --2- 172.21.15.172:0/1063298481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4c004800 0x7f6d4c004c80 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.241+0000 7f6d677dc640 1 -- 172.21.15.172:0/1063298481 >> 172.21.15.172:0/1063298481 conn(0x7f6d600d2080 msgr2=0x7f6d600d2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:04.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.241+0000 7f6d677dc640 1 -- 172.21.15.172:0/1063298481 shutdown_connections 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.241+0000 7f6d677dc640 1 -- 172.21.15.172:0/1063298481 wait complete. 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.242+0000 7f6d677dc640 1 Processor -- start 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.243+0000 7f6d677dc640 1 -- start start 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.243+0000 7f6d677dc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4c004800 0x7f6d6015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.243+0000 7f6d677dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d6015f310 0x7f6d6009c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.243+0000 7f6d677dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d6009ddc0 0x7f6d6009ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.243+0000 7f6d677dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d600d0ce0 con 0x7f6d6009ddc0 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.243+0000 7f6d677dc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6d600ce500 con 0x7f6d4c004800 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.243+0000 7f6d677dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6d6009aa70 con 0x7f6d6015f310 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.243+0000 7f6d65d79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d6009ddc0 0x7f6d6009ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.243+0000 7f6d65d79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d6009ddc0 0x7f6d6009ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55242/0 (socket says 172.21.15.172:55242) 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.243+0000 7f6d65d79640 1 -- 172.21.15.172:0/1700511748 learned_addr learned my addr 172.21.15.172:0/1700511748 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.243+0000 7f6d65578640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4c004800 0x7f6d6015eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.243+0000 7f6d64d77640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d6015f310 0x7f6d6009c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.244+0000 7f6d65d79640 1 -- 172.21.15.172:0/1700511748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d6015f310 msgr2=0x7f6d6009c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.244+0000 7f6d65d79640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d6015f310 0x7f6d6009c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.244+0000 7f6d65d79640 1 -- 172.21.15.172:0/1700511748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4c004800 msgr2=0x7f6d6015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.244+0000 7f6d65d79640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4c004800 0x7f6d6015eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.244+0000 7f6d65d79640 1 -- 172.21.15.172:0/1700511748 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6009cd20 con 0x7f6d6009ddc0 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.244+0000 7f6d65d79640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d6009ddc0 0x7f6d6009ec60 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6d58002230 tx=0x7f6d58007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.244+0000 7f6d55ffb640 1 -- 172.21.15.172:0/1700511748 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5800abc0 con 0x7f6d6009ddc0 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.244+0000 7f6d55ffb640 1 -- 172.21.15.172:0/1700511748 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d58018070 con 0x7f6d6009ddc0 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.244+0000 7f6d55ffb640 1 -- 172.21.15.172:0/1700511748 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d580133f0 con 0x7f6d6009ddc0 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.244+0000 7f6d677dc640 1 -- 172.21.15.172:0/1700511748 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6009d000 con 0x7f6d6009ddc0 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.244+0000 7f6d677dc640 1 -- 172.21.15.172:0/1700511748 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6009d5c0 con 0x7f6d6009ddc0 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.245+0000 7f6d677dc640 1 -- 172.21.15.172:0/1700511748 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d60068520 con 0x7f6d6009ddc0 2024-04-14T08:37:04.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.245+0000 7f6d55ffb640 1 -- 172.21.15.172:0/1700511748 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d58013590 con 0x7f6d6009ddc0 2024-04-14T08:37:04.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.246+0000 7f6d55ffb640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d4406c8f0 0x7f6d4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.246+0000 7f6d55ffb640 1 -- 172.21.15.172:0/1700511748 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f6d5808c280 con 0x7f6d6009ddc0 2024-04-14T08:37:04.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.246+0000 7f6d65578640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d4406c8f0 0x7f6d4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.246+0000 7f6d65578640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d4406c8f0 0x7f6d4406edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009b00 tx=0x7f6d5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:04.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.250+0000 7f6d55ffb640 1 -- 172.21.15.172:0/1700511748 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6d5805a520 con 0x7f6d6009ddc0 2024-04-14T08:37:04.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.433+0000 7f6d677dc640 1 -- 172.21.15.172:0/1700511748 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d600b79c0 con 0x7f6d6009ddc0 2024-04-14T08:37:04.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.434+0000 7f6d55ffb640 1 -- 172.21.15.172:0/1700511748 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+17005 (secure 0 0 0) 0x7f6d58059e20 con 0x7f6d6009ddc0 2024-04-14T08:37:04.436 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:04.436 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":74,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:37:02.756327+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:36:59.342045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-14T08:37:00.059338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:37:04.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.437+0000 7f6d677dc640 1 -- 172.21.15.172:0/1700511748 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d4406c8f0 msgr2=0x7f6d4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.437+0000 7f6d677dc640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d4406c8f0 0x7f6d4406edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009b00 tx=0x7f6d5c008040 comp rx=0 tx=0).stop 2024-04-14T08:37:04.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.437+0000 7f6d677dc640 1 -- 172.21.15.172:0/1700511748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d6009ddc0 msgr2=0x7f6d6009ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.437+0000 7f6d677dc640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d6009ddc0 0x7f6d6009ec60 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6d58002230 tx=0x7f6d58007b00 comp rx=0 tx=0).stop 2024-04-14T08:37:04.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.437+0000 7f6d677dc640 1 -- 172.21.15.172:0/1700511748 shutdown_connections 2024-04-14T08:37:04.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.437+0000 7f6d677dc640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d4406c8f0 0x7f6d4406edb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.437+0000 7f6d677dc640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d6009ddc0 0x7f6d6009ec60 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.437+0000 7f6d677dc640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d6015f310 0x7f6d6009c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.437+0000 7f6d677dc640 1 --2- 172.21.15.172:0/1700511748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d4c004800 0x7f6d6015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.437+0000 7f6d677dc640 1 -- 172.21.15.172:0/1700511748 >> 172.21.15.172:0/1700511748 conn(0x7f6d600d2080 msgr2=0x7f6d60161590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:04.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.437+0000 7f6d677dc640 1 -- 172.21.15.172:0/1700511748 shutdown_connections 2024-04-14T08:37:04.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.437+0000 7f6d677dc640 1 -- 172.21.15.172:0/1700511748 wait complete. 2024-04-14T08:37:04.450 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-14T08:37:04.450 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:37:04.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.554+0000 7f6778d55640 1 Processor -- start 2024-04-14T08:37:04.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.554+0000 7f6778d55640 1 -- start start 2024-04-14T08:37:04.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f6778d55640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f677415e390 0x7f67740d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f6778d55640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f677415f3a0 0x7f67740b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f6778d55640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f677415e770 con 0x7f67740b4260 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f6778d55640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f67740d0080 con 0x7f677415f3a0 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f6778d55640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f677406d640 con 0x7f677415e390 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f676bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f677415f3a0 0x7f67740b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f676bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f677415f3a0 0x7f67740b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55258/0 (socket says 172.21.15.172:55258) 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f676bfff640 1 -- 172.21.15.172:0/3767545842 learned_addr learned my addr 172.21.15.172:0/3767545842 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f677259c640 1 --2- 172.21.15.172:0/3767545842 >> v2:172.21.15.177:3300/0 conn(0x7f677415e390 0x7f67740d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f676bfff640 1 -- 172.21.15.172:0/3767545842 >> v2:172.21.15.177:3300/0 conn(0x7f677415e390 msgr2=0x7f67740d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f676bfff640 1 --2- 172.21.15.172:0/3767545842 >> v2:172.21.15.177:3300/0 conn(0x7f677415e390 0x7f67740d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f676bfff640 1 -- 172.21.15.172:0/3767545842 >> v1:172.21.15.177:6789/0 conn(0x7f67740b4260 legacy=0x7f6774161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.555+0000 7f676bfff640 1 -- 172.21.15.172:0/3767545842 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677406dba0 con 0x7f677415f3a0 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.556+0000 7f676bfff640 1 --2- 172.21.15.172:0/3767545842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677415f3a0 0x7f67740b3d20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6774070690 tx=0x7f675c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1821e01f4f7b4203 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:04.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.556+0000 7f6771d9b640 1 -- 172.21.15.172:0/3767545842 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c035020 con 0x7f677415f3a0 2024-04-14T08:37:04.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.556+0000 7f6771d9b640 1 -- 172.21.15.172:0/3767545842 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f675c031870 con 0x7f677415f3a0 2024-04-14T08:37:04.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.556+0000 7f6771d9b640 1 -- 172.21.15.172:0/3767545842 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c031c40 con 0x7f677415f3a0 2024-04-14T08:37:04.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.557+0000 7f6778d55640 1 -- 172.21.15.172:0/3767545842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677415f3a0 msgr2=0x7f67740b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.557+0000 7f6778d55640 1 --2- 172.21.15.172:0/3767545842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677415f3a0 0x7f67740b3d20 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6774070690 tx=0x7f675c02b910 comp rx=0 tx=0).stop 2024-04-14T08:37:04.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.557+0000 7f6778d55640 1 -- 172.21.15.172:0/3767545842 shutdown_connections 2024-04-14T08:37:04.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.557+0000 7f6778d55640 1 --2- 172.21.15.172:0/3767545842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677415f3a0 0x7f67740b3d20 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.557+0000 7f6778d55640 1 --2- 172.21.15.172:0/3767545842 >> v2:172.21.15.177:3300/0 conn(0x7f677415e390 0x7f67740d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.557+0000 7f6778d55640 1 -- 172.21.15.172:0/3767545842 >> 172.21.15.172:0/3767545842 conn(0x7f677406f820 msgr2=0x7f677406fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:04.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.557+0000 7f6778d55640 1 -- 172.21.15.172:0/3767545842 shutdown_connections 2024-04-14T08:37:04.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.557+0000 7f6778d55640 1 -- 172.21.15.172:0/3767545842 wait complete. 2024-04-14T08:37:04.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.558+0000 7f6778d55640 1 Processor -- start 2024-04-14T08:37:04.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.558+0000 7f6778d55640 1 -- start start 2024-04-14T08:37:04.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f6778d55640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67740b4260 0x7f6774149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f6778d55640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f677415e390 0x7f67740c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f6778d55640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f677415f3a0 0x7f67740c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f6778d55640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f677415f800 con 0x7f67740b4260 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f6778d55640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67740b4690 con 0x7f677415e390 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f6778d55640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f67740d0510 con 0x7f677415f3a0 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f677259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67740b4260 0x7f6774149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f677259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67740b4260 0x7f6774149dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55260/0 (socket says 172.21.15.172:55260) 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f677259c640 1 -- 172.21.15.172:0/1683895994 learned_addr learned my addr 172.21.15.172:0/1683895994 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f676bfff640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f677415e390 0x7f67740c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f6772d9d640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f677415f3a0 0x7f67740c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f677259c640 1 -- 172.21.15.172:0/1683895994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f677415f3a0 msgr2=0x7f67740c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f677259c640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f677415f3a0 0x7f67740c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f677259c640 1 -- 172.21.15.172:0/1683895994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f677415e390 msgr2=0x7f67740c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f677259c640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f677415e390 0x7f67740c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f677259c640 1 -- 172.21.15.172:0/1683895994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67740afbc0 con 0x7f67740b4260 2024-04-14T08:37:04.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f6772d9d640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f677415f3a0 0x7f67740c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f676bfff640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f677415e390 0x7f67740c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.559+0000 7f677259c640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67740b4260 0x7f6774149dd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f676400d620 tx=0x7f676400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.560+0000 7f676affd640 1 -- 172.21.15.172:0/1683895994 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676400b620 con 0x7f67740b4260 2024-04-14T08:37:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.560+0000 7f676affd640 1 -- 172.21.15.172:0/1683895994 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6764019070 con 0x7f67740b4260 2024-04-14T08:37:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.560+0000 7f6778d55640 1 -- 172.21.15.172:0/1683895994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677414d4c0 con 0x7f67740b4260 2024-04-14T08:37:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.560+0000 7f676affd640 1 -- 172.21.15.172:0/1683895994 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676400ba00 con 0x7f67740b4260 2024-04-14T08:37:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.560+0000 7f6778d55640 1 -- 172.21.15.172:0/1683895994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67740b0160 con 0x7f67740b4260 2024-04-14T08:37:04.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.561+0000 7f6778d55640 1 -- 172.21.15.172:0/1683895994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6774068520 con 0x7f67740b4260 2024-04-14T08:37:04.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.561+0000 7f676affd640 1 -- 172.21.15.172:0/1683895994 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f676400bba0 con 0x7f67740b4260 2024-04-14T08:37:04.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.562+0000 7f676affd640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f675406c8f0 0x7f675406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.562+0000 7f676affd640 1 -- 172.21.15.172:0/1683895994 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f676408c160 con 0x7f67740b4260 2024-04-14T08:37:04.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.562+0000 7f676bfff640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f675406c8f0 0x7f675406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.562+0000 7f676bfff640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f675406c8f0 0x7f675406edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f675c036000 tx=0x7f675c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:04.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.566+0000 7f676affd640 1 -- 172.21.15.172:0/1683895994 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f676405a370 con 0x7f67740b4260 2024-04-14T08:37:04.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.745+0000 7f6778d55640 1 -- 172.21.15.172:0/1683895994 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6774160170 con 0x7f675406c8f0 2024-04-14T08:37:04.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.751+0000 7f676affd640 1 -- 172.21.15.172:0/1683895994 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241152 (secure 0 0 0) 0x7f6774160170 con 0x7f675406c8f0 2024-04-14T08:37:04.755 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:04.756 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:37:04.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.753+0000 7f6778d55640 1 -- 172.21.15.172:0/1683895994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f675406c8f0 msgr2=0x7f675406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.753+0000 7f6778d55640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f675406c8f0 0x7f675406edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f675c036000 tx=0x7f675c008030 comp rx=0 tx=0).stop 2024-04-14T08:37:04.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.753+0000 7f6778d55640 1 -- 172.21.15.172:0/1683895994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67740b4260 msgr2=0x7f6774149dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.753+0000 7f6778d55640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67740b4260 0x7f6774149dd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f676400d620 tx=0x7f676400dae0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.753+0000 7f6778d55640 1 -- 172.21.15.172:0/1683895994 shutdown_connections 2024-04-14T08:37:04.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.753+0000 7f6778d55640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f675406c8f0 0x7f675406edb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.753+0000 7f6778d55640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f677415f3a0 0x7f67740c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.753+0000 7f6778d55640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f677415e390 0x7f67740c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.753+0000 7f6778d55640 1 --2- 172.21.15.172:0/1683895994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67740b4260 0x7f6774149dd0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.753+0000 7f6778d55640 1 -- 172.21.15.172:0/1683895994 >> 172.21.15.172:0/1683895994 conn(0x7f677406f820 msgr2=0x7f67740b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:04.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.754+0000 7f6778d55640 1 -- 172.21.15.172:0/1683895994 shutdown_connections 2024-04-14T08:37:04.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.754+0000 7f6778d55640 1 -- 172.21.15.172:0/1683895994 wait complete. 2024-04-14T08:37:04.766 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-04-14T08:37:03.171539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232408,"kb_used_data":7896,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754742312,"statfs":{"total":773094113280,"available":772856127488,"internally_reserved":0,"allocated":8085504,"data_stored":4278662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000652"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754006+0000","last_change":"2024-04-14T08:37:00.754006+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754006+0000","last_undegraded":"2024-04-14T08:37:00.754006+0000","last_fullsized":"2024-04-14T08:37:00.754006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754635+0000","last_change":"2024-04-14T08:37:00.754635+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754635+0000","last_undegraded":"2024-04-14T08:37:00.754635+0000","last_fullsized":"2024-04-14T08:37:00.754635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754995+0000","last_change":"2024-04-14T08:37:00.754995+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754995+0000","last_undegraded":"2024-04-14T08:37:00.754995+0000","last_fullsized":"2024-04-14T08:37:00.754995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754088+0000","last_change":"2024-04-14T08:37:00.754088+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754088+0000","last_undegraded":"2024-04-14T08:37:00.754088+0000","last_fullsized":"2024-04-14T08:37:00.754088+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.753387+0000","last_change":"2024-04-14T08:37:00.753387+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.753387+0000","last_undegraded":"2024-04-14T08:37:00.753387+0000","last_fullsized":"2024-04-14T08:37:00.753387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.746972+0000","last_change":"2024-04-14T08:36:59.746972+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.746972+0000","last_undegraded":"2024-04-14T08:36:59.746972+0000","last_fullsized":"2024-04-14T08:36:59.746972+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.767745+0000","last_change":"2024-04-14T08:37:00.766237+0000","last_active":"2024-04-14T08:37:00.767745+0000","last_peered":"2024-04-14T08:37:00.767745+0000","last_clean":"2024-04-14T08:37:00.767745+0000","last_became_active":"2024-04-14T08:37:00.765985+0000","last_became_peered":"2024-04-14T08:37:00.765985+0000","last_unstale":"2024-04-14T08:37:00.767745+0000","last_undegraded":"2024-04-14T08:37:00.767745+0000","last_fullsized":"2024-04-14T08:37:00.767745+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:19.304261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.769204+0000","last_change":"2024-04-14T08:37:00.766957+0000","last_active":"2024-04-14T08:37:00.769204+0000","last_peered":"2024-04-14T08:37:00.769204+0000","last_clean":"2024-04-14T08:37:00.769204+0000","last_became_active":"2024-04-14T08:37:00.766644+0000","last_became_peered":"2024-04-14T08:37:00.766644+0000","last_unstale":"2024-04-14T08:37:00.769204+0000","last_undegraded":"2024-04-14T08:37:00.769204+0000","last_fullsized":"2024-04-14T08:37:00.769204+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:44:20.142398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.750574+0000","last_change":"2024-04-14T08:36:59.750574+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.750574+0000","last_undegraded":"2024-04-14T08:36:59.750574+0000","last_fullsized":"2024-04-14T08:36:59.750574+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.745285+0000","last_change":"2024-04-14T08:36:59.745285+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.745285+0000","last_undegraded":"2024-04-14T08:36:59.745285+0000","last_fullsized":"2024-04-14T08:36:59.745285+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.748309+0000","last_change":"2024-04-14T08:36:59.748309+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.748309+0000","last_undegraded":"2024-04-14T08:36:59.748309+0000","last_fullsized":"2024-04-14T08:36:59.748309+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.748996+0000","last_change":"2024-04-14T08:36:59.748996+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.748996+0000","last_undegraded":"2024-04-14T08:36:59.748996+0000","last_fullsized":"2024-04-14T08:36:59.748996+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.747408+0000","last_change":"2024-04-14T08:36:59.747408+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.747408+0000","last_undegraded":"2024-04-14T08:36:59.747408+0000","last_fullsized":"2024-04-14T08:36:59.747408+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.769827+0000","last_change":"2024-04-14T08:37:00.768750+0000","last_active":"2024-04-14T08:37:00.769827+0000","last_peered":"2024-04-14T08:37:00.769827+0000","last_clean":"2024-04-14T08:37:00.769827+0000","last_became_active":"2024-04-14T08:37:00.768482+0000","last_became_peered":"2024-04-14T08:37:00.768482+0000","last_unstale":"2024-04-14T08:37:00.769827+0000","last_undegraded":"2024-04-14T08:37:00.769827+0000","last_fullsized":"2024-04-14T08:37:00.769827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:37:57.505670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.748311+0000","last_change":"2024-04-14T08:36:59.748311+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.748311+0000","last_undegraded":"2024-04-14T08:36:59.748311+0000","last_fullsized":"2024-04-14T08:36:59.748311+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.746817+0000","last_change":"2024-04-14T08:36:59.746817+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.746817+0000","last_undegraded":"2024-04-14T08:36:59.746817+0000","last_fullsized":"2024-04-14T08:36:59.746817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.749952+0000","last_change":"2024-04-14T08:36:59.749952+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.749952+0000","last_undegraded":"2024-04-14T08:36:59.749952+0000","last_fullsized":"2024-04-14T08:36:59.749952+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.748184+0000","last_change":"2024-04-14T08:36:59.748184+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.748184+0000","last_undegraded":"2024-04-14T08:36:59.748184+0000","last_fullsized":"2024-04-14T08:36:59.748184+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.769156+0000","last_change":"2024-04-14T08:37:00.767584+0000","last_active":"2024-04-14T08:37:00.769156+0000","last_peered":"2024-04-14T08:37:00.769156+0000","last_clean":"2024-04-14T08:37:00.769156+0000","last_became_active":"2024-04-14T08:37:00.767283+0000","last_became_peered":"2024-04-14T08:37:00.767283+0000","last_unstale":"2024-04-14T08:37:00.769156+0000","last_undegraded":"2024-04-14T08:37:00.769156+0000","last_fullsized":"2024-04-14T08:37:00.769156+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:04:42.747296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.747119+0000","last_change":"2024-04-14T08:36:59.747119+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.747119+0000","last_undegraded":"2024-04-14T08:36:59.747119+0000","last_fullsized":"2024-04-14T08:36:59.747119+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.749000+0000","last_change":"2024-04-14T08:36:59.749000+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.749000+0000","last_undegraded":"2024-04-14T08:36:59.749000+0000","last_fullsized":"2024-04-14T08:36:59.749000+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.765738+0000","last_change":"2024-04-14T08:37:00.761998+0000","last_active":"2024-04-14T08:37:00.765738+0000","last_peered":"2024-04-14T08:37:00.765738+0000","last_clean":"2024-04-14T08:37:00.765738+0000","last_became_active":"2024-04-14T08:37:00.761717+0000","last_became_peered":"2024-04-14T08:37:00.761717+0000","last_unstale":"2024-04-14T08:37:00.765738+0000","last_undegraded":"2024-04-14T08:37:00.765738+0000","last_fullsized":"2024-04-14T08:37:00.765738+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:18:48.866099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.745510+0000","last_change":"2024-04-14T08:36:59.745510+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.745510+0000","last_undegraded":"2024-04-14T08:36:59.745510+0000","last_fullsized":"2024-04-14T08:36:59.745510+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.747226+0000","last_change":"2024-04-14T08:36:59.747226+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.747226+0000","last_undegraded":"2024-04-14T08:36:59.747226+0000","last_fullsized":"2024-04-14T08:36:59.747226+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.745063+0000","last_change":"2024-04-14T08:36:59.745063+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.745063+0000","last_undegraded":"2024-04-14T08:36:59.745063+0000","last_fullsized":"2024-04-14T08:36:59.745063+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.746834+0000","last_change":"2024-04-14T08:36:59.746834+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.746834+0000","last_undegraded":"2024-04-14T08:36:59.746834+0000","last_fullsized":"2024-04-14T08:36:59.746834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.747671+0000","last_change":"2024-04-14T08:36:59.747671+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.747671+0000","last_undegraded":"2024-04-14T08:36:59.747671+0000","last_fullsized":"2024-04-14T08:36:59.747671+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.746929+0000","last_change":"2024-04-14T08:36:59.746929+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.746929+0000","last_undegraded":"2024-04-14T08:36:59.746929+0000","last_fullsized":"2024-04-14T08:36:59.746929+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.748525+0000","last_change":"2024-04-14T08:36:59.748525+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.748525+0000","last_undegraded":"2024-04-14T08:36:59.748525+0000","last_fullsized":"2024-04-14T08:36:59.748525+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.749074+0000","last_change":"2024-04-14T08:36:59.749074+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.749074+0000","last_undegraded":"2024-04-14T08:36:59.749074+0000","last_fullsized":"2024-04-14T08:36:59.749074+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.764073+0000","last_change":"2024-04-14T08:37:00.761456+0000","last_active":"2024-04-14T08:37:00.764073+0000","last_peered":"2024-04-14T08:37:00.764073+0000","last_clean":"2024-04-14T08:37:00.764073+0000","last_became_active":"2024-04-14T08:37:00.761137+0000","last_became_peered":"2024-04-14T08:37:00.761137+0000","last_unstale":"2024-04-14T08:37:00.764073+0000","last_undegraded":"2024-04-14T08:37:00.764073+0000","last_fullsized":"2024-04-14T08:37:00.764073+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:12:04.773469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.764175+0000","last_change":"2024-04-14T08:37:00.759937+0000","last_active":"2024-04-14T08:37:00.764175+0000","last_peered":"2024-04-14T08:37:00.764175+0000","last_clean":"2024-04-14T08:37:00.764175+0000","last_became_active":"2024-04-14T08:37:00.759674+0000","last_became_peered":"2024-04-14T08:37:00.759674+0000","last_unstale":"2024-04-14T08:37:00.764175+0000","last_undegraded":"2024-04-14T08:37:00.764175+0000","last_fullsized":"2024-04-14T08:37:00.764175+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:51.579563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.749818+0000","last_change":"2024-04-14T08:36:59.749818+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.749818+0000","last_undegraded":"2024-04-14T08:36:59.749818+0000","last_fullsized":"2024-04-14T08:36:59.749818+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.746961+0000","last_change":"2024-04-14T08:36:59.746961+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.746961+0000","last_undegraded":"2024-04-14T08:36:59.746961+0000","last_fullsized":"2024-04-14T08:36:59.746961+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.749739+0000","last_change":"2024-04-14T08:36:59.749739+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.749739+0000","last_undegraded":"2024-04-14T08:36:59.749739+0000","last_fullsized":"2024-04-14T08:36:59.749739+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.745063+0000","last_change":"2024-04-14T08:36:59.745063+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.745063+0000","last_undegraded":"2024-04-14T08:36:59.745063+0000","last_fullsized":"2024-04-14T08:36:59.745063+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.753179+0000","last_change":"2024-04-14T08:37:00.753179+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.753179+0000","last_undegraded":"2024-04-14T08:37:00.753179+0000","last_fullsized":"2024-04-14T08:37:00.753179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.747295+0000","last_change":"2024-04-14T08:36:59.747295+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.747295+0000","last_undegraded":"2024-04-14T08:36:59.747295+0000","last_fullsized":"2024-04-14T08:36:59.747295+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":274,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.754905+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:37:00.754905+0000","last_peered":"2024-04-14T08:37:00.754905+0000","last_clean":"2024-04-14T08:37:00.754905+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:37:00.754905+0000","last_undegraded":"2024-04-14T08:37:00.754905+0000","last_fullsized":"2024-04-14T08:37:00.754905+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1474560,"data_stored":987323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":384079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":384079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":856064,"data_stored":384079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":868352,"data_stored":390288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":384079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":390288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28436,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343404,"statfs":{"total":96636764160,"available":96607645696,"internally_reserved":0,"allocated":1449984,"data_stored":974447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:37:04.770 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:37:04.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.906+0000 7f8ac2660640 1 Processor -- start 2024-04-14T08:37:04.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.906+0000 7f8ac2660640 1 -- start start 2024-04-14T08:37:04.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.907+0000 7f8ac2660640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8abc162700 0x7f8abc169380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.907+0000 7f8ac2660640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8abc0d0200 con 0x7f8abc0d1a10 2024-04-14T08:37:04.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.907+0000 7f8ac2660640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8abc0cf160 con 0x7f8abc15c150 2024-04-14T08:37:04.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.907+0000 7f8ac2660640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f8abc0cef10 con 0x7f8abc162700 2024-04-14T08:37:04.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.907+0000 7f8ac0bfd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8abc162700 0x7f8abc169380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.907+0000 7f8ac0bfd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8abc162700 0x7f8abc169380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55276/0 (socket says 172.21.15.172:55276) 2024-04-14T08:37:04.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.907+0000 7f8ac0bfd640 1 -- 172.21.15.172:0/819693134 learned_addr learned my addr 172.21.15.172:0/819693134 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:04.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.907+0000 7f8abb7fe640 1 --1- 172.21.15.172:0/819693134 >> v1:172.21.15.172:6789/0 conn(0x7f8abc0d1a10 0x7f8abc161ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58520/0 (socket says 172.21.15.172:58520) 2024-04-14T08:37:04.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.907+0000 7f8ac0bfd640 1 -- 172.21.15.172:0/819693134 >> v1:172.21.15.177:6789/0 conn(0x7f8abc15c150 legacy=0x7f8abc0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.907+0000 7f8ac0bfd640 1 -- 172.21.15.172:0/819693134 >> v1:172.21.15.172:6789/0 conn(0x7f8abc0d1a10 legacy=0x7f8abc161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.907+0000 7f8ac0bfd640 1 -- 172.21.15.172:0/819693134 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc0cf440 con 0x7f8abc162700 2024-04-14T08:37:04.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.907+0000 7f8abaffd640 1 -- 172.21.15.172:0/819693134 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2806348740 0 0) 0x7f8abc0d0200 con 0x7f8abc0d1a10 2024-04-14T08:37:04.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.908+0000 7f8ac0bfd640 1 --2- 172.21.15.172:0/819693134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8abc162700 0x7f8abc169380 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009e50 tx=0x7f8ab402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8044618811af186d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:04.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.908+0000 7f8abaffd640 1 -- 172.21.15.172:0/819693134 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab402cbb0 con 0x7f8abc162700 2024-04-14T08:37:04.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.908+0000 7f8abaffd640 1 -- 172.21.15.172:0/819693134 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab402cd50 con 0x7f8abc162700 2024-04-14T08:37:04.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.908+0000 7f8abaffd640 1 -- 172.21.15.172:0/819693134 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4034610 con 0x7f8abc162700 2024-04-14T08:37:04.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.909+0000 7f8ac2660640 1 -- 172.21.15.172:0/819693134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8abc162700 msgr2=0x7f8abc169380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.909+0000 7f8ac2660640 1 --2- 172.21.15.172:0/819693134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8abc162700 0x7f8abc169380 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009e50 tx=0x7f8ab402c010 comp rx=0 tx=0).stop 2024-04-14T08:37:04.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.909+0000 7f8ac2660640 1 -- 172.21.15.172:0/819693134 shutdown_connections 2024-04-14T08:37:04.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.909+0000 7f8ac2660640 1 --2- 172.21.15.172:0/819693134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8abc162700 0x7f8abc169380 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.909+0000 7f8ac2660640 1 -- 172.21.15.172:0/819693134 >> 172.21.15.172:0/819693134 conn(0x7f8abc09d580 msgr2=0x7f8abc0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:04.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.909+0000 7f8ac2660640 1 -- 172.21.15.172:0/819693134 shutdown_connections 2024-04-14T08:37:04.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.909+0000 7f8ac2660640 1 -- 172.21.15.172:0/819693134 wait complete. 2024-04-14T08:37:04.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.910+0000 7f8ac2660640 1 Processor -- start 2024-04-14T08:37:04.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.910+0000 7f8ac2660640 1 -- start start 2024-04-14T08:37:04.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8ac2660640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0d1a10 0x7f8abc0b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8ac2660640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8abc15c150 0x7f8abc15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8ac2660640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8abc162700 0x7f8abc15dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8ac2660640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc0cfce0 con 0x7f8abc162700 2024-04-14T08:37:04.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8ac2660640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc0cfa00 con 0x7f8abc15c150 2024-04-14T08:37:04.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8ac2660640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8abc0d2e90 con 0x7f8abc0d1a10 2024-04-14T08:37:04.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8ac0bfd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8abc162700 0x7f8abc15dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8ac0bfd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8abc162700 0x7f8abc15dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55282/0 (socket says 172.21.15.172:55282) 2024-04-14T08:37:04.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8ac0bfd640 1 -- 172.21.15.172:0/83004621 learned_addr learned my addr 172.21.15.172:0/83004621 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:04.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8ac0bfd640 1 -- 172.21.15.172:0/83004621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0d1a10 msgr2=0x7f8abc0b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8ac0bfd640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0d1a10 0x7f8abc0b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8ac0bfd640 1 -- 172.21.15.172:0/83004621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8abc15c150 msgr2=0x7f8abc15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:04.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.911+0000 7f8abb7fe640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8abc15c150 0x7f8abc15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.912+0000 7f8ac0bfd640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8abc15c150 0x7f8abc15d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:04.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.912+0000 7f8ac0bfd640 1 -- 172.21.15.172:0/83004621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc15eb70 con 0x7f8abc162700 2024-04-14T08:37:04.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.912+0000 7f8ac0bfd640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8abc162700 0x7f8abc15dd30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009e50 tx=0x7f8ab4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:04.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.912+0000 7f8ab8ff9640 1 -- 172.21.15.172:0/83004621 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4043020 con 0x7f8abc162700 2024-04-14T08:37:04.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.912+0000 7f8ab8ff9640 1 -- 172.21.15.172:0/83004621 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab403e450 con 0x7f8abc162700 2024-04-14T08:37:04.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.912+0000 7f8ac2660640 1 -- 172.21.15.172:0/83004621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc15ee50 con 0x7f8abc162700 2024-04-14T08:37:04.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.912+0000 7f8ab8ff9640 1 -- 172.21.15.172:0/83004621 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4009510 con 0x7f8abc162700 2024-04-14T08:37:04.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.912+0000 7f8ac2660640 1 -- 172.21.15.172:0/83004621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc0b77e0 con 0x7f8abc162700 2024-04-14T08:37:04.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.913+0000 7f8ac2660640 1 -- 172.21.15.172:0/83004621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8abc075490 con 0x7f8abc162700 2024-04-14T08:37:04.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.913+0000 7f8ab8ff9640 1 -- 172.21.15.172:0/83004621 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ab4046020 con 0x7f8abc162700 2024-04-14T08:37:04.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.914+0000 7f8ab8ff9640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8aa806c8f0 0x7f8aa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:04.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.914+0000 7f8ab8ff9640 1 -- 172.21.15.172:0/83004621 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f8ab40ae5c0 con 0x7f8abc162700 2024-04-14T08:37:04.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.914+0000 7f8abbfff640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8aa806c8f0 0x7f8aa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:04.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.914+0000 7f8abbfff640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8aa806c8f0 0x7f8aa806edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0b7110 tx=0x7f8ab0007950 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:04.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:04.922+0000 7f8ab8ff9640 1 -- 172.21.15.172:0/83004621 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8ab404d030 con 0x7f8abc162700 2024-04-14T08:37:05.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.100+0000 7f8ac2660640 1 -- 172.21.15.172:0/83004621 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8abc0d49c0 con 0x7f8aa806c8f0 2024-04-14T08:37:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.106+0000 7f8ab8ff9640 1 -- 172.21.15.172:0/83004621 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241152 (secure 0 0 0) 0x7f8abc0d49c0 con 0x7f8aa806c8f0 2024-04-14T08:37:05.110 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:05.111 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:37:05.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.109+0000 7f8ac2660640 1 -- 172.21.15.172:0/83004621 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8aa806c8f0 msgr2=0x7f8aa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.109+0000 7f8ac2660640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8aa806c8f0 0x7f8aa806edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0b7110 tx=0x7f8ab0007950 comp rx=0 tx=0).stop 2024-04-14T08:37:05.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.109+0000 7f8ac2660640 1 -- 172.21.15.172:0/83004621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8abc162700 msgr2=0x7f8abc15dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.109+0000 7f8ac2660640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8abc162700 0x7f8abc15dd30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009e50 tx=0x7f8ab4004a40 comp rx=0 tx=0).stop 2024-04-14T08:37:05.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.109+0000 7f8ac2660640 1 -- 172.21.15.172:0/83004621 shutdown_connections 2024-04-14T08:37:05.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.109+0000 7f8ac2660640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8aa806c8f0 0x7f8aa806edb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.109+0000 7f8ac2660640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8abc162700 0x7f8abc15dd30 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.109+0000 7f8ac2660640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8abc15c150 0x7f8abc15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.109+0000 7f8ac2660640 1 --2- 172.21.15.172:0/83004621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8abc0d1a10 0x7f8abc0b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.109+0000 7f8ac2660640 1 -- 172.21.15.172:0/83004621 >> 172.21.15.172:0/83004621 conn(0x7f8abc09d580 msgr2=0x7f8abc0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:05.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.109+0000 7f8ac2660640 1 -- 172.21.15.172:0/83004621 shutdown_connections 2024-04-14T08:37:05.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.109+0000 7f8ac2660640 1 -- 172.21.15.172:0/83004621 wait complete. 2024-04-14T08:37:05.123 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-04-14T08:37:03.171539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":232408,"kb_used_data":7896,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754742312,"statfs":{"total":773094113280,"available":772856127488,"internally_reserved":0,"allocated":8085504,"data_stored":4278662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000652"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754006+0000","last_change":"2024-04-14T08:37:00.754006+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754006+0000","last_undegraded":"2024-04-14T08:37:00.754006+0000","last_fullsized":"2024-04-14T08:37:00.754006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754635+0000","last_change":"2024-04-14T08:37:00.754635+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754635+0000","last_undegraded":"2024-04-14T08:37:00.754635+0000","last_fullsized":"2024-04-14T08:37:00.754635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754995+0000","last_change":"2024-04-14T08:37:00.754995+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754995+0000","last_undegraded":"2024-04-14T08:37:00.754995+0000","last_fullsized":"2024-04-14T08:37:00.754995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754088+0000","last_change":"2024-04-14T08:37:00.754088+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754088+0000","last_undegraded":"2024-04-14T08:37:00.754088+0000","last_fullsized":"2024-04-14T08:37:00.754088+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.753387+0000","last_change":"2024-04-14T08:37:00.753387+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.753387+0000","last_undegraded":"2024-04-14T08:37:00.753387+0000","last_fullsized":"2024-04-14T08:37:00.753387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.746972+0000","last_change":"2024-04-14T08:36:59.746972+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.746972+0000","last_undegraded":"2024-04-14T08:36:59.746972+0000","last_fullsized":"2024-04-14T08:36:59.746972+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.767745+0000","last_change":"2024-04-14T08:37:00.766237+0000","last_active":"2024-04-14T08:37:00.767745+0000","last_peered":"2024-04-14T08:37:00.767745+0000","last_clean":"2024-04-14T08:37:00.767745+0000","last_became_active":"2024-04-14T08:37:00.765985+0000","last_became_peered":"2024-04-14T08:37:00.765985+0000","last_unstale":"2024-04-14T08:37:00.767745+0000","last_undegraded":"2024-04-14T08:37:00.767745+0000","last_fullsized":"2024-04-14T08:37:00.767745+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:19.304261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.769204+0000","last_change":"2024-04-14T08:37:00.766957+0000","last_active":"2024-04-14T08:37:00.769204+0000","last_peered":"2024-04-14T08:37:00.769204+0000","last_clean":"2024-04-14T08:37:00.769204+0000","last_became_active":"2024-04-14T08:37:00.766644+0000","last_became_peered":"2024-04-14T08:37:00.766644+0000","last_unstale":"2024-04-14T08:37:00.769204+0000","last_undegraded":"2024-04-14T08:37:00.769204+0000","last_fullsized":"2024-04-14T08:37:00.769204+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:44:20.142398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.750574+0000","last_change":"2024-04-14T08:36:59.750574+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.750574+0000","last_undegraded":"2024-04-14T08:36:59.750574+0000","last_fullsized":"2024-04-14T08:36:59.750574+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.745285+0000","last_change":"2024-04-14T08:36:59.745285+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.745285+0000","last_undegraded":"2024-04-14T08:36:59.745285+0000","last_fullsized":"2024-04-14T08:36:59.745285+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.748309+0000","last_change":"2024-04-14T08:36:59.748309+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.748309+0000","last_undegraded":"2024-04-14T08:36:59.748309+0000","last_fullsized":"2024-04-14T08:36:59.748309+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.748996+0000","last_change":"2024-04-14T08:36:59.748996+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.748996+0000","last_undegraded":"2024-04-14T08:36:59.748996+0000","last_fullsized":"2024-04-14T08:36:59.748996+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.747408+0000","last_change":"2024-04-14T08:36:59.747408+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.747408+0000","last_undegraded":"2024-04-14T08:36:59.747408+0000","last_fullsized":"2024-04-14T08:36:59.747408+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.769827+0000","last_change":"2024-04-14T08:37:00.768750+0000","last_active":"2024-04-14T08:37:00.769827+0000","last_peered":"2024-04-14T08:37:00.769827+0000","last_clean":"2024-04-14T08:37:00.769827+0000","last_became_active":"2024-04-14T08:37:00.768482+0000","last_became_peered":"2024-04-14T08:37:00.768482+0000","last_unstale":"2024-04-14T08:37:00.769827+0000","last_undegraded":"2024-04-14T08:37:00.769827+0000","last_fullsized":"2024-04-14T08:37:00.769827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:37:57.505670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.748311+0000","last_change":"2024-04-14T08:36:59.748311+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.748311+0000","last_undegraded":"2024-04-14T08:36:59.748311+0000","last_fullsized":"2024-04-14T08:36:59.748311+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.746817+0000","last_change":"2024-04-14T08:36:59.746817+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.746817+0000","last_undegraded":"2024-04-14T08:36:59.746817+0000","last_fullsized":"2024-04-14T08:36:59.746817+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.749952+0000","last_change":"2024-04-14T08:36:59.749952+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.749952+0000","last_undegraded":"2024-04-14T08:36:59.749952+0000","last_fullsized":"2024-04-14T08:36:59.749952+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.748184+0000","last_change":"2024-04-14T08:36:59.748184+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.748184+0000","last_undegraded":"2024-04-14T08:36:59.748184+0000","last_fullsized":"2024-04-14T08:36:59.748184+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.769156+0000","last_change":"2024-04-14T08:37:00.767584+0000","last_active":"2024-04-14T08:37:00.769156+0000","last_peered":"2024-04-14T08:37:00.769156+0000","last_clean":"2024-04-14T08:37:00.769156+0000","last_became_active":"2024-04-14T08:37:00.767283+0000","last_became_peered":"2024-04-14T08:37:00.767283+0000","last_unstale":"2024-04-14T08:37:00.769156+0000","last_undegraded":"2024-04-14T08:37:00.769156+0000","last_fullsized":"2024-04-14T08:37:00.769156+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:04:42.747296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.747119+0000","last_change":"2024-04-14T08:36:59.747119+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.747119+0000","last_undegraded":"2024-04-14T08:36:59.747119+0000","last_fullsized":"2024-04-14T08:36:59.747119+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.749000+0000","last_change":"2024-04-14T08:36:59.749000+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.749000+0000","last_undegraded":"2024-04-14T08:36:59.749000+0000","last_fullsized":"2024-04-14T08:36:59.749000+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.765738+0000","last_change":"2024-04-14T08:37:00.761998+0000","last_active":"2024-04-14T08:37:00.765738+0000","last_peered":"2024-04-14T08:37:00.765738+0000","last_clean":"2024-04-14T08:37:00.765738+0000","last_became_active":"2024-04-14T08:37:00.761717+0000","last_became_peered":"2024-04-14T08:37:00.761717+0000","last_unstale":"2024-04-14T08:37:00.765738+0000","last_undegraded":"2024-04-14T08:37:00.765738+0000","last_fullsized":"2024-04-14T08:37:00.765738+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:18:48.866099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.745510+0000","last_change":"2024-04-14T08:36:59.745510+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.745510+0000","last_undegraded":"2024-04-14T08:36:59.745510+0000","last_fullsized":"2024-04-14T08:36:59.745510+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.747226+0000","last_change":"2024-04-14T08:36:59.747226+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.747226+0000","last_undegraded":"2024-04-14T08:36:59.747226+0000","last_fullsized":"2024-04-14T08:36:59.747226+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.745063+0000","last_change":"2024-04-14T08:36:59.745063+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.745063+0000","last_undegraded":"2024-04-14T08:36:59.745063+0000","last_fullsized":"2024-04-14T08:36:59.745063+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.746834+0000","last_change":"2024-04-14T08:36:59.746834+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.746834+0000","last_undegraded":"2024-04-14T08:36:59.746834+0000","last_fullsized":"2024-04-14T08:36:59.746834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.747671+0000","last_change":"2024-04-14T08:36:59.747671+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.747671+0000","last_undegraded":"2024-04-14T08:36:59.747671+0000","last_fullsized":"2024-04-14T08:36:59.747671+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.746929+0000","last_change":"2024-04-14T08:36:59.746929+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.746929+0000","last_undegraded":"2024-04-14T08:36:59.746929+0000","last_fullsized":"2024-04-14T08:36:59.746929+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.748525+0000","last_change":"2024-04-14T08:36:59.748525+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.748525+0000","last_undegraded":"2024-04-14T08:36:59.748525+0000","last_fullsized":"2024-04-14T08:36:59.748525+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.749074+0000","last_change":"2024-04-14T08:36:59.749074+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.749074+0000","last_undegraded":"2024-04-14T08:36:59.749074+0000","last_fullsized":"2024-04-14T08:36:59.749074+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.764073+0000","last_change":"2024-04-14T08:37:00.761456+0000","last_active":"2024-04-14T08:37:00.764073+0000","last_peered":"2024-04-14T08:37:00.764073+0000","last_clean":"2024-04-14T08:37:00.764073+0000","last_became_active":"2024-04-14T08:37:00.761137+0000","last_became_peered":"2024-04-14T08:37:00.761137+0000","last_unstale":"2024-04-14T08:37:00.764073+0000","last_undegraded":"2024-04-14T08:37:00.764073+0000","last_fullsized":"2024-04-14T08:37:00.764073+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:12:04.773469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.764175+0000","last_change":"2024-04-14T08:37:00.759937+0000","last_active":"2024-04-14T08:37:00.764175+0000","last_peered":"2024-04-14T08:37:00.764175+0000","last_clean":"2024-04-14T08:37:00.764175+0000","last_became_active":"2024-04-14T08:37:00.759674+0000","last_became_peered":"2024-04-14T08:37:00.759674+0000","last_unstale":"2024-04-14T08:37:00.764175+0000","last_undegraded":"2024-04-14T08:37:00.764175+0000","last_fullsized":"2024-04-14T08:37:00.764175+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:51.579563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.749818+0000","last_change":"2024-04-14T08:36:59.749818+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.749818+0000","last_undegraded":"2024-04-14T08:36:59.749818+0000","last_fullsized":"2024-04-14T08:36:59.749818+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.746961+0000","last_change":"2024-04-14T08:36:59.746961+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.746961+0000","last_undegraded":"2024-04-14T08:36:59.746961+0000","last_fullsized":"2024-04-14T08:36:59.746961+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.749739+0000","last_change":"2024-04-14T08:36:59.749739+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.749739+0000","last_undegraded":"2024-04-14T08:36:59.749739+0000","last_fullsized":"2024-04-14T08:36:59.749739+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-14T08:37:00.742875+0000","last_change":"2024-04-14T08:37:00.742875+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.742875+0000","last_undegraded":"2024-04-14T08:37:00.742875+0000","last_fullsized":"2024-04-14T08:37:00.742875+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.745063+0000","last_change":"2024-04-14T08:36:59.745063+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.745063+0000","last_undegraded":"2024-04-14T08:36:59.745063+0000","last_fullsized":"2024-04-14T08:36:59.745063+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.753179+0000","last_change":"2024-04-14T08:37:00.753179+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.753179+0000","last_undegraded":"2024-04-14T08:37:00.753179+0000","last_fullsized":"2024-04-14T08:37:00.753179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-04-14T08:36:59.747295+0000","last_change":"2024-04-14T08:36:59.747295+0000","last_active":"2024-04-14T08:36:59.734147+0000","last_peered":"2024-04-14T08:36:59.734147+0000","last_clean":"2024-04-14T08:36:59.734147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:36:59.747295+0000","last_undegraded":"2024-04-14T08:36:59.747295+0000","last_fullsized":"2024-04-14T08:36:59.747295+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":274,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.754905+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:37:00.754905+0000","last_peered":"2024-04-14T08:37:00.754905+0000","last_clean":"2024-04-14T08:37:00.754905+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:37:00.754905+0000","last_undegraded":"2024-04-14T08:37:00.754905+0000","last_fullsized":"2024-04-14T08:37:00.754905+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1474560,"data_stored":987323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":384079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":384079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":856064,"data_stored":384079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":868352,"data_stored":390288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":384079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":390288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28436,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343404,"statfs":{"total":96636764160,"available":96607645696,"internally_reserved":0,"allocated":1449984,"data_stored":974447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:37:05.127 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:37:05.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.264+0000 7fcb8951e640 1 Processor -- start 2024-04-14T08:37:05.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.264+0000 7fcb8951e640 1 -- start start 2024-04-14T08:37:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.265+0000 7fcb8951e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb8415c150 0x7fcb840d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.265+0000 7fcb8951e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcb84162700 0x7fcb840d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.265+0000 7fcb8951e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb840d0200 con 0x7fcb840d1840 2024-04-14T08:37:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.265+0000 7fcb8951e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fcb840d1cb0 con 0x7fcb8415c150 2024-04-14T08:37:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.265+0000 7fcb8951e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fcb840cef10 con 0x7fcb84162700 2024-04-14T08:37:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.265+0000 7fcb82ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb8415c150 0x7fcb840d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.265+0000 7fcb82ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb8415c150 0x7fcb840d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55288/0 (socket says 172.21.15.172:55288) 2024-04-14T08:37:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.265+0000 7fcb82ffd640 1 -- 172.21.15.172:0/1075380733 learned_addr learned my addr 172.21.15.172:0/1075380733 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.266+0000 7fcb837fe640 1 --2- 172.21.15.172:0/1075380733 >> v2:172.21.15.177:3300/0 conn(0x7fcb84162700 0x7fcb840d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.266+0000 7fcb82ffd640 1 -- 172.21.15.172:0/1075380733 >> v2:172.21.15.177:3300/0 conn(0x7fcb84162700 msgr2=0x7fcb840d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.266+0000 7fcb82ffd640 1 --2- 172.21.15.172:0/1075380733 >> v2:172.21.15.177:3300/0 conn(0x7fcb84162700 0x7fcb840d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.266+0000 7fcb82ffd640 1 -- 172.21.15.172:0/1075380733 >> v1:172.21.15.172:6789/0 conn(0x7fcb840d1840 legacy=0x7fcb84161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.266+0000 7fcb82ffd640 1 -- 172.21.15.172:0/1075380733 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb840cf160 con 0x7fcb8415c150 2024-04-14T08:37:05.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.267+0000 7fcb82ffd640 1 --2- 172.21.15.172:0/1075380733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb8415c150 0x7fcb840d1300 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fcb840ba330 tx=0x7fcb7402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efedf1e09d9194a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:05.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.267+0000 7fcb827fc640 1 -- 172.21.15.172:0/1075380733 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74034070 con 0x7fcb8415c150 2024-04-14T08:37:05.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.267+0000 7fcb827fc640 1 -- 172.21.15.172:0/1075380733 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb74030860 con 0x7fcb8415c150 2024-04-14T08:37:05.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.267+0000 7fcb827fc640 1 -- 172.21.15.172:0/1075380733 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74030c30 con 0x7fcb8415c150 2024-04-14T08:37:05.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.268+0000 7fcb8951e640 1 -- 172.21.15.172:0/1075380733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb8415c150 msgr2=0x7fcb840d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.268+0000 7fcb8951e640 1 --2- 172.21.15.172:0/1075380733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb8415c150 0x7fcb840d1300 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fcb840ba330 tx=0x7fcb7402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.268+0000 7fcb8951e640 1 -- 172.21.15.172:0/1075380733 shutdown_connections 2024-04-14T08:37:05.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.268+0000 7fcb8951e640 1 --2- 172.21.15.172:0/1075380733 >> v2:172.21.15.177:3300/0 conn(0x7fcb84162700 0x7fcb840d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.268+0000 7fcb8951e640 1 --2- 172.21.15.172:0/1075380733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb8415c150 0x7fcb840d1300 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.268+0000 7fcb8951e640 1 -- 172.21.15.172:0/1075380733 >> 172.21.15.172:0/1075380733 conn(0x7fcb8409d580 msgr2=0x7fcb840d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:05.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.268+0000 7fcb8951e640 1 -- 172.21.15.172:0/1075380733 shutdown_connections 2024-04-14T08:37:05.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.268+0000 7fcb8951e640 1 -- 172.21.15.172:0/1075380733 wait complete. 2024-04-14T08:37:05.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.270+0000 7fcb8951e640 1 Processor -- start 2024-04-14T08:37:05.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.270+0000 7fcb8951e640 1 -- start start 2024-04-14T08:37:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb8951e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb840d1840 0x7fcb8415b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb8951e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb8415c150 0x7fcb8415ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb8951e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb84162700 0x7fcb84160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb8951e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcb840cfa00 con 0x7fcb840d1840 2024-04-14T08:37:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb8951e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcb840cf720 con 0x7fcb8415c150 2024-04-14T08:37:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb8951e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcb840cfce0 con 0x7fcb84162700 2024-04-14T08:37:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb82ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb840d1840 0x7fcb8415b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb82ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb840d1840 0x7fcb8415b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55300/0 (socket says 172.21.15.172:55300) 2024-04-14T08:37:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb82ffd640 1 -- 172.21.15.172:0/1470473934 learned_addr learned my addr 172.21.15.172:0/1470473934 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb7bfff640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb8415c150 0x7fcb8415ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb837fe640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb84162700 0x7fcb84160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb82ffd640 1 -- 172.21.15.172:0/1470473934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb84162700 msgr2=0x7fcb84160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb82ffd640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb84162700 0x7fcb84160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb82ffd640 1 -- 172.21.15.172:0/1470473934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb8415c150 msgr2=0x7fcb8415ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb82ffd640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb8415c150 0x7fcb8415ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb82ffd640 1 -- 172.21.15.172:0/1470473934 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8415dde0 con 0x7fcb840d1840 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.271+0000 7fcb7bfff640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb8415c150 0x7fcb8415ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.272+0000 7fcb82ffd640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb840d1840 0x7fcb8415b370 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcb7402b9e0 tx=0x7fcb74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.272+0000 7fcb7b7fe640 1 -- 172.21.15.172:0/1470473934 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74048070 con 0x7fcb840d1840 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.272+0000 7fcb7b7fe640 1 -- 172.21.15.172:0/1470473934 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb74030660 con 0x7fcb840d1840 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.272+0000 7fcb8951e640 1 -- 172.21.15.172:0/1470473934 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb840cb4b0 con 0x7fcb840d1840 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.272+0000 7fcb7b7fe640 1 -- 172.21.15.172:0/1470473934 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb74034040 con 0x7fcb840d1840 2024-04-14T08:37:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.272+0000 7fcb8951e640 1 -- 172.21.15.172:0/1470473934 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8415d820 con 0x7fcb840d1840 2024-04-14T08:37:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.273+0000 7fcb8951e640 1 -- 172.21.15.172:0/1470473934 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb840cb680 con 0x7fcb840d1840 2024-04-14T08:37:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.274+0000 7fcb7b7fe640 1 -- 172.21.15.172:0/1470473934 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb7404b070 con 0x7fcb840d1840 2024-04-14T08:37:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.274+0000 7fcb7b7fe640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb6406c920 0x7fcb6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.274+0000 7fcb7bfff640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb6406c920 0x7fcb6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:05.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.274+0000 7fcb7b7fe640 1 -- 172.21.15.172:0/1470473934 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fcb740adbf0 con 0x7fcb840d1840 2024-04-14T08:37:05.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.275+0000 7fcb7bfff640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb6406c920 0x7fcb6406ede0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c007fd0 tx=0x7fcb6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:05.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.279+0000 7fcb7b7fe640 1 -- 172.21.15.172:0/1470473934 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcb7407bf20 con 0x7fcb840d1840 2024-04-14T08:37:05.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.453+0000 7fcb8951e640 1 -- 172.21.15.172:0/1470473934 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb840d49c0 con 0x7fcb6406c920 2024-04-14T08:37:05.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.459+0000 7fcb7b7fe640 1 -- 172.21.15.172:0/1470473934 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251570 (secure 0 0 0) 0x7fcb840d49c0 con 0x7fcb6406c920 2024-04-14T08:37:05.463 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:05.464 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:37:05.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.462+0000 7fcb8951e640 1 -- 172.21.15.172:0/1470473934 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb6406c920 msgr2=0x7fcb6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.462+0000 7fcb8951e640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb6406c920 0x7fcb6406ede0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c007fd0 tx=0x7fcb6c009040 comp rx=0 tx=0).stop 2024-04-14T08:37:05.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.462+0000 7fcb8951e640 1 -- 172.21.15.172:0/1470473934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb840d1840 msgr2=0x7fcb8415b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.462+0000 7fcb8951e640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb840d1840 0x7fcb8415b370 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcb7402b9e0 tx=0x7fcb74004730 comp rx=0 tx=0).stop 2024-04-14T08:37:05.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.462+0000 7fcb8951e640 1 -- 172.21.15.172:0/1470473934 shutdown_connections 2024-04-14T08:37:05.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.462+0000 7fcb8951e640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb6406c920 0x7fcb6406ede0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.462+0000 7fcb8951e640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb84162700 0x7fcb84160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.462+0000 7fcb8951e640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb8415c150 0x7fcb8415ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.462+0000 7fcb8951e640 1 --2- 172.21.15.172:0/1470473934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb840d1840 0x7fcb8415b370 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.462+0000 7fcb8951e640 1 -- 172.21.15.172:0/1470473934 >> 172.21.15.172:0/1470473934 conn(0x7fcb8409d580 msgr2=0x7fcb840d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:05.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.462+0000 7fcb8951e640 1 -- 172.21.15.172:0/1470473934 shutdown_connections 2024-04-14T08:37:05.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.462+0000 7fcb8951e640 1 -- 172.21.15.172:0/1470473934 wait complete. 2024-04-14T08:37:05.475 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-04-14T08:37:05.171818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236924,"kb_used_data":8268,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754737796,"statfs":{"total":773094113280,"available":772851503104,"internally_reserved":0,"allocated":8466432,"data_stored":4458187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.428822"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754006+0000","last_change":"2024-04-14T08:37:00.754006+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754006+0000","last_undegraded":"2024-04-14T08:37:00.754006+0000","last_fullsized":"2024-04-14T08:37:00.754006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763468+0000","last_change":"2024-04-14T08:37:01.775666+0000","last_active":"2024-04-14T08:37:02.763468+0000","last_peered":"2024-04-14T08:37:02.763468+0000","last_clean":"2024-04-14T08:37:02.763468+0000","last_became_active":"2024-04-14T08:37:01.775390+0000","last_became_peered":"2024-04-14T08:37:01.775390+0000","last_unstale":"2024-04-14T08:37:02.763468+0000","last_undegraded":"2024-04-14T08:37:02.763468+0000","last_fullsized":"2024-04-14T08:37:02.763468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:28:31.420209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208954+0000","last_change":"2024-04-14T08:37:01.763121+0000","last_active":"2024-04-14T08:37:03.208954+0000","last_peered":"2024-04-14T08:37:03.208954+0000","last_clean":"2024-04-14T08:37:03.208954+0000","last_became_active":"2024-04-14T08:37:01.762855+0000","last_became_peered":"2024-04-14T08:37:01.762855+0000","last_unstale":"2024-04-14T08:37:03.208954+0000","last_undegraded":"2024-04-14T08:37:03.208954+0000","last_fullsized":"2024-04-14T08:37:03.208954+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:52.588292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208816+0000","last_change":"2024-04-14T08:37:01.771514+0000","last_active":"2024-04-14T08:37:03.208816+0000","last_peered":"2024-04-14T08:37:03.208816+0000","last_clean":"2024-04-14T08:37:03.208816+0000","last_became_active":"2024-04-14T08:37:01.770791+0000","last_became_peered":"2024-04-14T08:37:01.770791+0000","last_unstale":"2024-04-14T08:37:03.208816+0000","last_undegraded":"2024-04-14T08:37:03.208816+0000","last_fullsized":"2024-04-14T08:37:03.208816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:19:22.374600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209464+0000","last_change":"2024-04-14T08:37:01.771310+0000","last_active":"2024-04-14T08:37:03.209464+0000","last_peered":"2024-04-14T08:37:03.209464+0000","last_clean":"2024-04-14T08:37:03.209464+0000","last_became_active":"2024-04-14T08:37:01.770803+0000","last_became_peered":"2024-04-14T08:37:01.770803+0000","last_unstale":"2024-04-14T08:37:03.209464+0000","last_undegraded":"2024-04-14T08:37:03.209464+0000","last_fullsized":"2024-04-14T08:37:03.209464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:01:08.014841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208926+0000","last_change":"2024-04-14T08:37:01.769338+0000","last_active":"2024-04-14T08:37:03.208926+0000","last_peered":"2024-04-14T08:37:03.208926+0000","last_clean":"2024-04-14T08:37:03.208926+0000","last_became_active":"2024-04-14T08:37:01.768525+0000","last_became_peered":"2024-04-14T08:37:01.768525+0000","last_unstale":"2024-04-14T08:37:03.208926+0000","last_undegraded":"2024-04-14T08:37:03.208926+0000","last_fullsized":"2024-04-14T08:37:03.208926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:37:40.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144460+0000","last_change":"2024-04-14T08:37:01.777169+0000","last_active":"2024-04-14T08:37:03.144460+0000","last_peered":"2024-04-14T08:37:03.144460+0000","last_clean":"2024-04-14T08:37:03.144460+0000","last_became_active":"2024-04-14T08:37:01.776878+0000","last_became_peered":"2024-04-14T08:37:01.776878+0000","last_unstale":"2024-04-14T08:37:03.144460+0000","last_undegraded":"2024-04-14T08:37:03.144460+0000","last_fullsized":"2024-04-14T08:37:03.144460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:18:49.874828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209058+0000","last_change":"2024-04-14T08:37:01.766727+0000","last_active":"2024-04-14T08:37:03.209058+0000","last_peered":"2024-04-14T08:37:03.209058+0000","last_clean":"2024-04-14T08:37:03.209058+0000","last_became_active":"2024-04-14T08:37:01.766433+0000","last_became_peered":"2024-04-14T08:37:01.766433+0000","last_unstale":"2024-04-14T08:37:03.209058+0000","last_undegraded":"2024-04-14T08:37:03.209058+0000","last_fullsized":"2024-04-14T08:37:03.209058+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:05:52.532330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768847+0000","last_change":"2024-04-14T08:37:01.769926+0000","last_active":"2024-04-14T08:37:02.768847+0000","last_peered":"2024-04-14T08:37:02.768847+0000","last_clean":"2024-04-14T08:37:02.768847+0000","last_became_active":"2024-04-14T08:37:01.769289+0000","last_became_peered":"2024-04-14T08:37:01.769289+0000","last_unstale":"2024-04-14T08:37:02.768847+0000","last_undegraded":"2024-04-14T08:37:02.768847+0000","last_fullsized":"2024-04-14T08:37:02.768847+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:36:56.878085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208763+0000","last_change":"2024-04-14T08:37:01.771927+0000","last_active":"2024-04-14T08:37:03.208763+0000","last_peered":"2024-04-14T08:37:03.208763+0000","last_clean":"2024-04-14T08:37:03.208763+0000","last_became_active":"2024-04-14T08:37:01.771592+0000","last_became_peered":"2024-04-14T08:37:01.771592+0000","last_unstale":"2024-04-14T08:37:03.208763+0000","last_undegraded":"2024-04-14T08:37:03.208763+0000","last_fullsized":"2024-04-14T08:37:03.208763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:32:15.297929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763325+0000","last_change":"2024-04-14T08:37:01.776864+0000","last_active":"2024-04-14T08:37:02.763325+0000","last_peered":"2024-04-14T08:37:02.763325+0000","last_clean":"2024-04-14T08:37:02.763325+0000","last_became_active":"2024-04-14T08:37:01.776407+0000","last_became_peered":"2024-04-14T08:37:01.776407+0000","last_unstale":"2024-04-14T08:37:02.763325+0000","last_undegraded":"2024-04-14T08:37:02.763325+0000","last_fullsized":"2024-04-14T08:37:02.763325+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:37:27.733921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208874+0000","last_change":"2024-04-14T08:37:01.773170+0000","last_active":"2024-04-14T08:37:03.208874+0000","last_peered":"2024-04-14T08:37:03.208874+0000","last_clean":"2024-04-14T08:37:03.208874+0000","last_became_active":"2024-04-14T08:37:01.772844+0000","last_became_peered":"2024-04-14T08:37:01.772844+0000","last_unstale":"2024-04-14T08:37:03.208874+0000","last_undegraded":"2024-04-14T08:37:03.208874+0000","last_fullsized":"2024-04-14T08:37:03.208874+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:24:26.449359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209485+0000","last_change":"2024-04-14T08:37:01.771520+0000","last_active":"2024-04-14T08:37:03.209485+0000","last_peered":"2024-04-14T08:37:03.209485+0000","last_clean":"2024-04-14T08:37:03.209485+0000","last_became_active":"2024-04-14T08:37:01.770987+0000","last_became_peered":"2024-04-14T08:37:01.770987+0000","last_unstale":"2024-04-14T08:37:03.209485+0000","last_undegraded":"2024-04-14T08:37:03.209485+0000","last_fullsized":"2024-04-14T08:37:03.209485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:02:32.886371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209059+0000","last_change":"2024-04-14T08:37:01.769523+0000","last_active":"2024-04-14T08:37:03.209059+0000","last_peered":"2024-04-14T08:37:03.209059+0000","last_clean":"2024-04-14T08:37:03.209059+0000","last_became_active":"2024-04-14T08:37:01.768799+0000","last_became_peered":"2024-04-14T08:37:01.768799+0000","last_unstale":"2024-04-14T08:37:03.209059+0000","last_undegraded":"2024-04-14T08:37:03.209059+0000","last_fullsized":"2024-04-14T08:37:03.209059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:34.673210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763192+0000","last_change":"2024-04-14T08:37:01.767566+0000","last_active":"2024-04-14T08:37:02.763192+0000","last_peered":"2024-04-14T08:37:02.763192+0000","last_clean":"2024-04-14T08:37:02.763192+0000","last_became_active":"2024-04-14T08:37:01.767302+0000","last_became_peered":"2024-04-14T08:37:01.767302+0000","last_unstale":"2024-04-14T08:37:02.763192+0000","last_undegraded":"2024-04-14T08:37:02.763192+0000","last_fullsized":"2024-04-14T08:37:02.763192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:23:26.664547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754635+0000","last_change":"2024-04-14T08:37:00.754635+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754635+0000","last_undegraded":"2024-04-14T08:37:00.754635+0000","last_fullsized":"2024-04-14T08:37:00.754635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209553+0000","last_change":"2024-04-14T08:37:01.773766+0000","last_active":"2024-04-14T08:37:03.209553+0000","last_peered":"2024-04-14T08:37:03.209553+0000","last_clean":"2024-04-14T08:37:03.209553+0000","last_became_active":"2024-04-14T08:37:01.772696+0000","last_became_peered":"2024-04-14T08:37:01.772696+0000","last_unstale":"2024-04-14T08:37:03.209553+0000","last_undegraded":"2024-04-14T08:37:03.209553+0000","last_fullsized":"2024-04-14T08:37:03.209553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:16:41.707052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208679+0000","last_change":"2024-04-14T08:37:01.766048+0000","last_active":"2024-04-14T08:37:03.208679+0000","last_peered":"2024-04-14T08:37:03.208679+0000","last_clean":"2024-04-14T08:37:03.208679+0000","last_became_active":"2024-04-14T08:37:01.765770+0000","last_became_peered":"2024-04-14T08:37:01.765770+0000","last_unstale":"2024-04-14T08:37:03.208679+0000","last_undegraded":"2024-04-14T08:37:03.208679+0000","last_fullsized":"2024-04-14T08:37:03.208679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:46:54.083005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208793+0000","last_change":"2024-04-14T08:37:01.776271+0000","last_active":"2024-04-14T08:37:03.208793+0000","last_peered":"2024-04-14T08:37:03.208793+0000","last_clean":"2024-04-14T08:37:03.208793+0000","last_became_active":"2024-04-14T08:37:01.776041+0000","last_became_peered":"2024-04-14T08:37:01.776041+0000","last_unstale":"2024-04-14T08:37:03.208793+0000","last_undegraded":"2024-04-14T08:37:03.208793+0000","last_fullsized":"2024-04-14T08:37:03.208793+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:46.505374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209553+0000","last_change":"2024-04-14T08:37:01.772630+0000","last_active":"2024-04-14T08:37:03.209553+0000","last_peered":"2024-04-14T08:37:03.209553+0000","last_clean":"2024-04-14T08:37:03.209553+0000","last_became_active":"2024-04-14T08:37:01.772322+0000","last_became_peered":"2024-04-14T08:37:01.772322+0000","last_unstale":"2024-04-14T08:37:03.209553+0000","last_undegraded":"2024-04-14T08:37:03.209553+0000","last_fullsized":"2024-04-14T08:37:03.209553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:39:59.248162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768463+0000","last_change":"2024-04-14T08:37:01.767943+0000","last_active":"2024-04-14T08:37:02.768463+0000","last_peered":"2024-04-14T08:37:02.768463+0000","last_clean":"2024-04-14T08:37:02.768463+0000","last_became_active":"2024-04-14T08:37:01.767670+0000","last_became_peered":"2024-04-14T08:37:01.767670+0000","last_unstale":"2024-04-14T08:37:02.768463+0000","last_undegraded":"2024-04-14T08:37:02.768463+0000","last_fullsized":"2024-04-14T08:37:02.768463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:21:49.412049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209008+0000","last_change":"2024-04-14T08:37:01.774274+0000","last_active":"2024-04-14T08:37:03.209008+0000","last_peered":"2024-04-14T08:37:03.209008+0000","last_clean":"2024-04-14T08:37:03.209008+0000","last_became_active":"2024-04-14T08:37:01.773936+0000","last_became_peered":"2024-04-14T08:37:01.773936+0000","last_unstale":"2024-04-14T08:37:03.209008+0000","last_undegraded":"2024-04-14T08:37:03.209008+0000","last_fullsized":"2024-04-14T08:37:03.209008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:32:01.701513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.767951+0000","last_change":"2024-04-14T08:37:01.769566+0000","last_active":"2024-04-14T08:37:02.767951+0000","last_peered":"2024-04-14T08:37:02.767951+0000","last_clean":"2024-04-14T08:37:02.767951+0000","last_became_active":"2024-04-14T08:37:01.768995+0000","last_became_peered":"2024-04-14T08:37:01.768995+0000","last_unstale":"2024-04-14T08:37:02.767951+0000","last_undegraded":"2024-04-14T08:37:02.767951+0000","last_fullsized":"2024-04-14T08:37:02.767951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:27:45.852443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763062+0000","last_change":"2024-04-14T08:37:01.769784+0000","last_active":"2024-04-14T08:37:02.763062+0000","last_peered":"2024-04-14T08:37:02.763062+0000","last_clean":"2024-04-14T08:37:02.763062+0000","last_became_active":"2024-04-14T08:37:01.769518+0000","last_became_peered":"2024-04-14T08:37:01.769518+0000","last_unstale":"2024-04-14T08:37:02.763062+0000","last_undegraded":"2024-04-14T08:37:02.763062+0000","last_fullsized":"2024-04-14T08:37:02.763062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:34.673210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763297+0000","last_change":"2024-04-14T08:37:01.770106+0000","last_active":"2024-04-14T08:37:02.763297+0000","last_peered":"2024-04-14T08:37:02.763297+0000","last_clean":"2024-04-14T08:37:02.763297+0000","last_became_active":"2024-04-14T08:37:01.769875+0000","last_became_peered":"2024-04-14T08:37:01.769875+0000","last_unstale":"2024-04-14T08:37:02.763297+0000","last_undegraded":"2024-04-14T08:37:02.763297+0000","last_fullsized":"2024-04-14T08:37:02.763297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:33:20.004698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208800+0000","last_change":"2024-04-14T08:37:01.773240+0000","last_active":"2024-04-14T08:37:03.208800+0000","last_peered":"2024-04-14T08:37:03.208800+0000","last_clean":"2024-04-14T08:37:03.208800+0000","last_became_active":"2024-04-14T08:37:01.772799+0000","last_became_peered":"2024-04-14T08:37:01.772799+0000","last_unstale":"2024-04-14T08:37:03.208800+0000","last_undegraded":"2024-04-14T08:37:03.208800+0000","last_fullsized":"2024-04-14T08:37:03.208800+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:35:08.892944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754995+0000","last_change":"2024-04-14T08:37:00.754995+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754995+0000","last_undegraded":"2024-04-14T08:37:00.754995+0000","last_fullsized":"2024-04-14T08:37:00.754995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209656+0000","last_change":"2024-04-14T08:37:01.770780+0000","last_active":"2024-04-14T08:37:03.209656+0000","last_peered":"2024-04-14T08:37:03.209656+0000","last_clean":"2024-04-14T08:37:03.209656+0000","last_became_active":"2024-04-14T08:37:01.770341+0000","last_became_peered":"2024-04-14T08:37:01.770341+0000","last_unstale":"2024-04-14T08:37:03.209656+0000","last_undegraded":"2024-04-14T08:37:03.209656+0000","last_fullsized":"2024-04-14T08:37:03.209656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:32:53.180094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763329+0000","last_change":"2024-04-14T08:37:01.772687+0000","last_active":"2024-04-14T08:37:02.763329+0000","last_peered":"2024-04-14T08:37:02.763329+0000","last_clean":"2024-04-14T08:37:02.763329+0000","last_became_active":"2024-04-14T08:37:01.768377+0000","last_became_peered":"2024-04-14T08:37:01.768377+0000","last_unstale":"2024-04-14T08:37:02.763329+0000","last_undegraded":"2024-04-14T08:37:02.763329+0000","last_fullsized":"2024-04-14T08:37:02.763329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:54:27.630750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754088+0000","last_change":"2024-04-14T08:37:00.754088+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754088+0000","last_undegraded":"2024-04-14T08:37:00.754088+0000","last_fullsized":"2024-04-14T08:37:00.754088+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208452+0000","last_change":"2024-04-14T08:37:01.769355+0000","last_active":"2024-04-14T08:37:03.208452+0000","last_peered":"2024-04-14T08:37:03.208452+0000","last_clean":"2024-04-14T08:37:03.208452+0000","last_became_active":"2024-04-14T08:37:01.769040+0000","last_became_peered":"2024-04-14T08:37:01.769040+0000","last_unstale":"2024-04-14T08:37:03.208452+0000","last_undegraded":"2024-04-14T08:37:03.208452+0000","last_fullsized":"2024-04-14T08:37:03.208452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:36:56.878085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.753387+0000","last_change":"2024-04-14T08:37:00.753387+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.753387+0000","last_undegraded":"2024-04-14T08:37:00.753387+0000","last_fullsized":"2024-04-14T08:37:00.753387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208275+0000","last_change":"2024-04-14T08:37:01.765375+0000","last_active":"2024-04-14T08:37:03.208275+0000","last_peered":"2024-04-14T08:37:03.208275+0000","last_clean":"2024-04-14T08:37:03.208275+0000","last_became_active":"2024-04-14T08:37:01.765108+0000","last_became_peered":"2024-04-14T08:37:01.765108+0000","last_unstale":"2024-04-14T08:37:03.208275+0000","last_undegraded":"2024-04-14T08:37:03.208275+0000","last_fullsized":"2024-04-14T08:37:03.208275+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:50:41.201503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209527+0000","last_change":"2024-04-14T08:37:00.766950+0000","last_active":"2024-04-14T08:37:03.209527+0000","last_peered":"2024-04-14T08:37:03.209527+0000","last_clean":"2024-04-14T08:37:03.209527+0000","last_became_active":"2024-04-14T08:37:00.766663+0000","last_became_peered":"2024-04-14T08:37:00.766663+0000","last_unstale":"2024-04-14T08:37:03.209527+0000","last_undegraded":"2024-04-14T08:37:03.209527+0000","last_fullsized":"2024-04-14T08:37:03.209527+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:45:03.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763294+0000","last_change":"2024-04-14T08:37:01.765910+0000","last_active":"2024-04-14T08:37:02.763294+0000","last_peered":"2024-04-14T08:37:02.763294+0000","last_clean":"2024-04-14T08:37:02.763294+0000","last_became_active":"2024-04-14T08:37:01.765648+0000","last_became_peered":"2024-04-14T08:37:01.765648+0000","last_unstale":"2024-04-14T08:37:02.763294+0000","last_undegraded":"2024-04-14T08:37:02.763294+0000","last_fullsized":"2024-04-14T08:37:02.763294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:14:50.531363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.767745+0000","last_change":"2024-04-14T08:37:00.766237+0000","last_active":"2024-04-14T08:37:00.767745+0000","last_peered":"2024-04-14T08:37:00.767745+0000","last_clean":"2024-04-14T08:37:00.767745+0000","last_became_active":"2024-04-14T08:37:00.765985+0000","last_became_peered":"2024-04-14T08:37:00.765985+0000","last_unstale":"2024-04-14T08:37:00.767745+0000","last_undegraded":"2024-04-14T08:37:00.767745+0000","last_fullsized":"2024-04-14T08:37:00.767745+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:19.304261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768551+0000","last_change":"2024-04-14T08:37:01.772297+0000","last_active":"2024-04-14T08:37:02.768551+0000","last_peered":"2024-04-14T08:37:02.768551+0000","last_clean":"2024-04-14T08:37:02.768551+0000","last_became_active":"2024-04-14T08:37:01.771577+0000","last_became_peered":"2024-04-14T08:37:01.771577+0000","last_unstale":"2024-04-14T08:37:02.768551+0000","last_undegraded":"2024-04-14T08:37:02.768551+0000","last_fullsized":"2024-04-14T08:37:02.768551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:22:46.811060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.769204+0000","last_change":"2024-04-14T08:37:00.766957+0000","last_active":"2024-04-14T08:37:00.769204+0000","last_peered":"2024-04-14T08:37:00.769204+0000","last_clean":"2024-04-14T08:37:00.769204+0000","last_became_active":"2024-04-14T08:37:00.766644+0000","last_became_peered":"2024-04-14T08:37:00.766644+0000","last_unstale":"2024-04-14T08:37:00.769204+0000","last_undegraded":"2024-04-14T08:37:00.769204+0000","last_fullsized":"2024-04-14T08:37:00.769204+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:44:20.142398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768478+0000","last_change":"2024-04-14T08:37:01.767364+0000","last_active":"2024-04-14T08:37:02.768478+0000","last_peered":"2024-04-14T08:37:02.768478+0000","last_clean":"2024-04-14T08:37:02.768478+0000","last_became_active":"2024-04-14T08:37:01.767103+0000","last_became_peered":"2024-04-14T08:37:01.767103+0000","last_unstale":"2024-04-14T08:37:02.768478+0000","last_undegraded":"2024-04-14T08:37:02.768478+0000","last_fullsized":"2024-04-14T08:37:02.768478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:55:08.633674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144522+0000","last_change":"2024-04-14T08:37:00.762961+0000","last_active":"2024-04-14T08:37:03.144522+0000","last_peered":"2024-04-14T08:37:03.144522+0000","last_clean":"2024-04-14T08:37:03.144522+0000","last_became_active":"2024-04-14T08:37:00.762514+0000","last_became_peered":"2024-04-14T08:37:00.762514+0000","last_unstale":"2024-04-14T08:37:03.144522+0000","last_undegraded":"2024-04-14T08:37:03.144522+0000","last_fullsized":"2024-04-14T08:37:03.144522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:04:32.846382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763395+0000","last_change":"2024-04-14T08:37:01.776280+0000","last_active":"2024-04-14T08:37:02.763395+0000","last_peered":"2024-04-14T08:37:02.763395+0000","last_clean":"2024-04-14T08:37:02.763395+0000","last_became_active":"2024-04-14T08:37:01.775999+0000","last_became_peered":"2024-04-14T08:37:01.775999+0000","last_unstale":"2024-04-14T08:37:02.763395+0000","last_undegraded":"2024-04-14T08:37:02.763395+0000","last_fullsized":"2024-04-14T08:37:02.763395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:46:11.850094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768525+0000","last_change":"2024-04-14T08:37:00.759304+0000","last_active":"2024-04-14T08:37:02.768525+0000","last_peered":"2024-04-14T08:37:02.768525+0000","last_clean":"2024-04-14T08:37:02.768525+0000","last_became_active":"2024-04-14T08:37:00.758994+0000","last_became_peered":"2024-04-14T08:37:00.758994+0000","last_unstale":"2024-04-14T08:37:02.768525+0000","last_undegraded":"2024-04-14T08:37:02.768525+0000","last_fullsized":"2024-04-14T08:37:02.768525+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:58:04.441713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763466+0000","last_change":"2024-04-14T08:37:01.776652+0000","last_active":"2024-04-14T08:37:02.763466+0000","last_peered":"2024-04-14T08:37:02.763466+0000","last_clean":"2024-04-14T08:37:02.763466+0000","last_became_active":"2024-04-14T08:37:01.776078+0000","last_became_peered":"2024-04-14T08:37:01.776078+0000","last_unstale":"2024-04-14T08:37:02.763466+0000","last_undegraded":"2024-04-14T08:37:02.763466+0000","last_fullsized":"2024-04-14T08:37:02.763466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:36:16.762703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208453+0000","last_change":"2024-04-14T08:37:00.766216+0000","last_active":"2024-04-14T08:37:03.208453+0000","last_peered":"2024-04-14T08:37:03.208453+0000","last_clean":"2024-04-14T08:37:03.208453+0000","last_became_active":"2024-04-14T08:37:00.765950+0000","last_became_peered":"2024-04-14T08:37:00.765950+0000","last_unstale":"2024-04-14T08:37:03.208453+0000","last_undegraded":"2024-04-14T08:37:03.208453+0000","last_fullsized":"2024-04-14T08:37:03.208453+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:18:48.866099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768872+0000","last_change":"2024-04-14T08:37:01.768600+0000","last_active":"2024-04-14T08:37:02.768872+0000","last_peered":"2024-04-14T08:37:02.768872+0000","last_clean":"2024-04-14T08:37:02.768872+0000","last_became_active":"2024-04-14T08:37:01.768160+0000","last_became_peered":"2024-04-14T08:37:01.768160+0000","last_unstale":"2024-04-14T08:37:02.768872+0000","last_undegraded":"2024-04-14T08:37:02.768872+0000","last_fullsized":"2024-04-14T08:37:02.768872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:46:54.083005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763237+0000","last_change":"2024-04-14T08:37:00.766724+0000","last_active":"2024-04-14T08:37:02.763237+0000","last_peered":"2024-04-14T08:37:02.763237+0000","last_clean":"2024-04-14T08:37:02.763237+0000","last_became_active":"2024-04-14T08:37:00.766452+0000","last_became_peered":"2024-04-14T08:37:00.766452+0000","last_unstale":"2024-04-14T08:37:02.763237+0000","last_undegraded":"2024-04-14T08:37:02.763237+0000","last_fullsized":"2024-04-14T08:37:02.763237+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:13:12.633337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144878+0000","last_change":"2024-04-14T08:37:01.777172+0000","last_active":"2024-04-14T08:37:03.144878+0000","last_peered":"2024-04-14T08:37:03.144878+0000","last_clean":"2024-04-14T08:37:03.144878+0000","last_became_active":"2024-04-14T08:37:01.776849+0000","last_became_peered":"2024-04-14T08:37:01.776849+0000","last_unstale":"2024-04-14T08:37:03.144878+0000","last_undegraded":"2024-04-14T08:37:03.144878+0000","last_fullsized":"2024-04-14T08:37:03.144878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:13:13.642066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209108+0000","last_change":"2024-04-14T08:37:00.765486+0000","last_active":"2024-04-14T08:37:03.209108+0000","last_peered":"2024-04-14T08:37:03.209108+0000","last_clean":"2024-04-14T08:37:03.209108+0000","last_became_active":"2024-04-14T08:37:00.765223+0000","last_became_peered":"2024-04-14T08:37:00.765223+0000","last_unstale":"2024-04-14T08:37:03.209108+0000","last_undegraded":"2024-04-14T08:37:03.209108+0000","last_fullsized":"2024-04-14T08:37:03.209108+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:13:40.708508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763070+0000","last_change":"2024-04-14T08:37:01.762529+0000","last_active":"2024-04-14T08:37:02.763070+0000","last_peered":"2024-04-14T08:37:02.763070+0000","last_clean":"2024-04-14T08:37:02.763070+0000","last_became_active":"2024-04-14T08:37:01.762258+0000","last_became_peered":"2024-04-14T08:37:01.762258+0000","last_unstale":"2024-04-14T08:37:02.763070+0000","last_undegraded":"2024-04-14T08:37:02.763070+0000","last_fullsized":"2024-04-14T08:37:02.763070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:54.396153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.769827+0000","last_change":"2024-04-14T08:37:00.768750+0000","last_active":"2024-04-14T08:37:00.769827+0000","last_peered":"2024-04-14T08:37:00.769827+0000","last_clean":"2024-04-14T08:37:00.769827+0000","last_became_active":"2024-04-14T08:37:00.768482+0000","last_became_peered":"2024-04-14T08:37:00.768482+0000","last_unstale":"2024-04-14T08:37:00.769827+0000","last_undegraded":"2024-04-14T08:37:00.769827+0000","last_fullsized":"2024-04-14T08:37:00.769827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:37:57.505670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208778+0000","last_change":"2024-04-14T08:37:01.764722+0000","last_active":"2024-04-14T08:37:03.208778+0000","last_peered":"2024-04-14T08:37:03.208778+0000","last_clean":"2024-04-14T08:37:03.208778+0000","last_became_active":"2024-04-14T08:37:01.764416+0000","last_became_peered":"2024-04-14T08:37:01.764416+0000","last_unstale":"2024-04-14T08:37:03.208778+0000","last_undegraded":"2024-04-14T08:37:03.208778+0000","last_fullsized":"2024-04-14T08:37:03.208778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:54:12.547002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208818+0000","last_change":"2024-04-14T08:37:00.762713+0000","last_active":"2024-04-14T08:37:03.208818+0000","last_peered":"2024-04-14T08:37:03.208818+0000","last_clean":"2024-04-14T08:37:03.208818+0000","last_became_active":"2024-04-14T08:37:00.762429+0000","last_became_peered":"2024-04-14T08:37:00.762429+0000","last_unstale":"2024-04-14T08:37:03.208818+0000","last_undegraded":"2024-04-14T08:37:03.208818+0000","last_fullsized":"2024-04-14T08:37:03.208818+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:04:42.747296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209258+0000","last_change":"2024-04-14T08:37:01.769140+0000","last_active":"2024-04-14T08:37:03.209258+0000","last_peered":"2024-04-14T08:37:03.209258+0000","last_clean":"2024-04-14T08:37:03.209258+0000","last_became_active":"2024-04-14T08:37:01.768251+0000","last_became_peered":"2024-04-14T08:37:01.768251+0000","last_unstale":"2024-04-14T08:37:03.209258+0000","last_undegraded":"2024-04-14T08:37:03.209258+0000","last_fullsized":"2024-04-14T08:37:03.209258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:54.396153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209671+0000","last_change":"2024-04-14T08:37:00.765119+0000","last_active":"2024-04-14T08:37:03.209671+0000","last_peered":"2024-04-14T08:37:03.209671+0000","last_clean":"2024-04-14T08:37:03.209671+0000","last_became_active":"2024-04-14T08:37:00.764836+0000","last_became_peered":"2024-04-14T08:37:00.764836+0000","last_unstale":"2024-04-14T08:37:03.209671+0000","last_undegraded":"2024-04-14T08:37:03.209671+0000","last_fullsized":"2024-04-14T08:37:03.209671+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:35:49.223991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208803+0000","last_change":"2024-04-14T08:37:01.764255+0000","last_active":"2024-04-14T08:37:03.208803+0000","last_peered":"2024-04-14T08:37:03.208803+0000","last_clean":"2024-04-14T08:37:03.208803+0000","last_became_active":"2024-04-14T08:37:01.763949+0000","last_became_peered":"2024-04-14T08:37:01.763949+0000","last_unstale":"2024-04-14T08:37:03.208803+0000","last_undegraded":"2024-04-14T08:37:03.208803+0000","last_fullsized":"2024-04-14T08:37:03.208803+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:16:13.344979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144550+0000","last_change":"2024-04-14T08:37:00.764900+0000","last_active":"2024-04-14T08:37:03.144550+0000","last_peered":"2024-04-14T08:37:03.144550+0000","last_clean":"2024-04-14T08:37:03.144550+0000","last_became_active":"2024-04-14T08:37:00.764635+0000","last_became_peered":"2024-04-14T08:37:00.764635+0000","last_unstale":"2024-04-14T08:37:03.144550+0000","last_undegraded":"2024-04-14T08:37:03.144550+0000","last_fullsized":"2024-04-14T08:37:03.144550+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:53:02.315116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208925+0000","last_change":"2024-04-14T08:37:01.768718+0000","last_active":"2024-04-14T08:37:03.208925+0000","last_peered":"2024-04-14T08:37:03.208925+0000","last_clean":"2024-04-14T08:37:03.208925+0000","last_became_active":"2024-04-14T08:37:01.768388+0000","last_became_peered":"2024-04-14T08:37:01.768388+0000","last_unstale":"2024-04-14T08:37:03.208925+0000","last_undegraded":"2024-04-14T08:37:03.208925+0000","last_fullsized":"2024-04-14T08:37:03.208925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:40:43.924405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208482+0000","last_change":"2024-04-14T08:37:00.767201+0000","last_active":"2024-04-14T08:37:03.208482+0000","last_peered":"2024-04-14T08:37:03.208482+0000","last_clean":"2024-04-14T08:37:03.208482+0000","last_became_active":"2024-04-14T08:37:00.766926+0000","last_became_peered":"2024-04-14T08:37:00.766926+0000","last_unstale":"2024-04-14T08:37:03.208482+0000","last_undegraded":"2024-04-14T08:37:03.208482+0000","last_fullsized":"2024-04-14T08:37:03.208482+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:44:20.142398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144667+0000","last_change":"2024-04-14T08:37:01.766150+0000","last_active":"2024-04-14T08:37:03.144667+0000","last_peered":"2024-04-14T08:37:03.144667+0000","last_clean":"2024-04-14T08:37:03.144667+0000","last_became_active":"2024-04-14T08:37:01.765865+0000","last_became_peered":"2024-04-14T08:37:01.765865+0000","last_unstale":"2024-04-14T08:37:03.144667+0000","last_undegraded":"2024-04-14T08:37:03.144667+0000","last_fullsized":"2024-04-14T08:37:03.144667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:04:43.756024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.769156+0000","last_change":"2024-04-14T08:37:00.767584+0000","last_active":"2024-04-14T08:37:00.769156+0000","last_peered":"2024-04-14T08:37:00.769156+0000","last_clean":"2024-04-14T08:37:00.769156+0000","last_became_active":"2024-04-14T08:37:00.767283+0000","last_became_peered":"2024-04-14T08:37:00.767283+0000","last_unstale":"2024-04-14T08:37:00.769156+0000","last_undegraded":"2024-04-14T08:37:00.769156+0000","last_fullsized":"2024-04-14T08:37:00.769156+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:04:42.747296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144726+0000","last_change":"2024-04-14T08:37:01.765838+0000","last_active":"2024-04-14T08:37:03.144726+0000","last_peered":"2024-04-14T08:37:03.144726+0000","last_clean":"2024-04-14T08:37:03.144726+0000","last_became_active":"2024-04-14T08:37:01.765086+0000","last_became_peered":"2024-04-14T08:37:01.765086+0000","last_unstale":"2024-04-14T08:37:03.144726+0000","last_undegraded":"2024-04-14T08:37:03.144726+0000","last_fullsized":"2024-04-14T08:37:03.144726+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:57.576896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209258+0000","last_change":"2024-04-14T08:37:00.766886+0000","last_active":"2024-04-14T08:37:03.209258+0000","last_peered":"2024-04-14T08:37:03.209258+0000","last_clean":"2024-04-14T08:37:03.209258+0000","last_became_active":"2024-04-14T08:37:00.766612+0000","last_became_peered":"2024-04-14T08:37:00.766612+0000","last_unstale":"2024-04-14T08:37:03.209258+0000","last_undegraded":"2024-04-14T08:37:03.209258+0000","last_fullsized":"2024-04-14T08:37:03.209258+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:41:00.225037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144883+0000","last_change":"2024-04-14T08:37:01.767730+0000","last_active":"2024-04-14T08:37:03.144883+0000","last_peered":"2024-04-14T08:37:03.144883+0000","last_clean":"2024-04-14T08:37:03.144883+0000","last_became_active":"2024-04-14T08:37:01.767480+0000","last_became_peered":"2024-04-14T08:37:01.767480+0000","last_unstale":"2024-04-14T08:37:03.144883+0000","last_undegraded":"2024-04-14T08:37:03.144883+0000","last_fullsized":"2024-04-14T08:37:03.144883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:17.690169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763123+0000","last_change":"2024-04-14T08:37:00.770164+0000","last_active":"2024-04-14T08:37:02.763123+0000","last_peered":"2024-04-14T08:37:02.763123+0000","last_clean":"2024-04-14T08:37:02.763123+0000","last_became_active":"2024-04-14T08:37:00.769881+0000","last_became_peered":"2024-04-14T08:37:00.769881+0000","last_unstale":"2024-04-14T08:37:02.763123+0000","last_undegraded":"2024-04-14T08:37:02.763123+0000","last_fullsized":"2024-04-14T08:37:02.763123+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:56.568167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208321+0000","last_change":"2024-04-14T08:37:01.775317+0000","last_active":"2024-04-14T08:37:03.208321+0000","last_peered":"2024-04-14T08:37:03.208321+0000","last_clean":"2024-04-14T08:37:03.208321+0000","last_became_active":"2024-04-14T08:37:01.775078+0000","last_became_peered":"2024-04-14T08:37:01.775078+0000","last_unstale":"2024-04-14T08:37:03.208321+0000","last_undegraded":"2024-04-14T08:37:03.208321+0000","last_fullsized":"2024-04-14T08:37:03.208321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:46:55.486835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.765738+0000","last_change":"2024-04-14T08:37:00.761998+0000","last_active":"2024-04-14T08:37:00.765738+0000","last_peered":"2024-04-14T08:37:00.765738+0000","last_clean":"2024-04-14T08:37:00.765738+0000","last_became_active":"2024-04-14T08:37:00.761717+0000","last_became_peered":"2024-04-14T08:37:00.761717+0000","last_unstale":"2024-04-14T08:37:00.765738+0000","last_undegraded":"2024-04-14T08:37:00.765738+0000","last_fullsized":"2024-04-14T08:37:00.765738+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:18:48.866099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209214+0000","last_change":"2024-04-14T08:37:01.773566+0000","last_active":"2024-04-14T08:37:03.209214+0000","last_peered":"2024-04-14T08:37:03.209214+0000","last_clean":"2024-04-14T08:37:03.209214+0000","last_became_active":"2024-04-14T08:37:01.773256+0000","last_became_peered":"2024-04-14T08:37:01.773256+0000","last_unstale":"2024-04-14T08:37:03.209214+0000","last_undegraded":"2024-04-14T08:37:03.209214+0000","last_fullsized":"2024-04-14T08:37:03.209214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:18:48.987307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768932+0000","last_change":"2024-04-14T08:37:00.763865+0000","last_active":"2024-04-14T08:37:02.768932+0000","last_peered":"2024-04-14T08:37:02.768932+0000","last_clean":"2024-04-14T08:37:02.768932+0000","last_became_active":"2024-04-14T08:37:00.763574+0000","last_became_peered":"2024-04-14T08:37:00.763574+0000","last_unstale":"2024-04-14T08:37:02.768932+0000","last_undegraded":"2024-04-14T08:37:02.768932+0000","last_fullsized":"2024-04-14T08:37:02.768932+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:07.846968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208793+0000","last_change":"2024-04-14T08:37:01.775524+0000","last_active":"2024-04-14T08:37:03.208793+0000","last_peered":"2024-04-14T08:37:03.208793+0000","last_clean":"2024-04-14T08:37:03.208793+0000","last_became_active":"2024-04-14T08:37:01.774838+0000","last_became_peered":"2024-04-14T08:37:01.774838+0000","last_unstale":"2024-04-14T08:37:03.208793+0000","last_undegraded":"2024-04-14T08:37:03.208793+0000","last_fullsized":"2024-04-14T08:37:03.208793+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:39:06.052705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209602+0000","last_change":"2024-04-14T08:37:00.767309+0000","last_active":"2024-04-14T08:37:03.209602+0000","last_peered":"2024-04-14T08:37:03.209602+0000","last_clean":"2024-04-14T08:37:03.209602+0000","last_became_active":"2024-04-14T08:37:00.767051+0000","last_became_peered":"2024-04-14T08:37:00.767051+0000","last_unstale":"2024-04-14T08:37:03.209602+0000","last_undegraded":"2024-04-14T08:37:03.209602+0000","last_fullsized":"2024-04-14T08:37:03.209602+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:52:48.671198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208603+0000","last_change":"2024-04-14T08:37:01.775690+0000","last_active":"2024-04-14T08:37:03.208603+0000","last_peered":"2024-04-14T08:37:03.208603+0000","last_clean":"2024-04-14T08:37:03.208603+0000","last_became_active":"2024-04-14T08:37:01.775321+0000","last_became_peered":"2024-04-14T08:37:01.775321+0000","last_unstale":"2024-04-14T08:37:03.208603+0000","last_undegraded":"2024-04-14T08:37:03.208603+0000","last_fullsized":"2024-04-14T08:37:03.208603+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:59:46.004037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208926+0000","last_change":"2024-04-14T08:37:00.767992+0000","last_active":"2024-04-14T08:37:03.208926+0000","last_peered":"2024-04-14T08:37:03.208926+0000","last_clean":"2024-04-14T08:37:03.208926+0000","last_became_active":"2024-04-14T08:37:00.767722+0000","last_became_peered":"2024-04-14T08:37:00.767722+0000","last_unstale":"2024-04-14T08:37:03.208926+0000","last_undegraded":"2024-04-14T08:37:03.208926+0000","last_fullsized":"2024-04-14T08:37:03.208926+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:34.765628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763294+0000","last_change":"2024-04-14T08:37:01.768521+0000","last_active":"2024-04-14T08:37:02.763294+0000","last_peered":"2024-04-14T08:37:02.763294+0000","last_clean":"2024-04-14T08:37:02.763294+0000","last_became_active":"2024-04-14T08:37:01.768215+0000","last_became_peered":"2024-04-14T08:37:01.768215+0000","last_unstale":"2024-04-14T08:37:02.763294+0000","last_undegraded":"2024-04-14T08:37:02.763294+0000","last_fullsized":"2024-04-14T08:37:02.763294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:58:15.131818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208650+0000","last_change":"2024-04-14T08:37:00.762968+0000","last_active":"2024-04-14T08:37:03.208650+0000","last_peered":"2024-04-14T08:37:03.208650+0000","last_clean":"2024-04-14T08:37:03.208650+0000","last_became_active":"2024-04-14T08:37:00.762672+0000","last_became_peered":"2024-04-14T08:37:00.762672+0000","last_unstale":"2024-04-14T08:37:03.208650+0000","last_undegraded":"2024-04-14T08:37:03.208650+0000","last_fullsized":"2024-04-14T08:37:03.208650+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:33.484846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763477+0000","last_change":"2024-04-14T08:37:01.765020+0000","last_active":"2024-04-14T08:37:02.763477+0000","last_peered":"2024-04-14T08:37:02.763477+0000","last_clean":"2024-04-14T08:37:02.763477+0000","last_became_active":"2024-04-14T08:37:01.764736+0000","last_became_peered":"2024-04-14T08:37:01.764736+0000","last_unstale":"2024-04-14T08:37:02.763477+0000","last_undegraded":"2024-04-14T08:37:02.763477+0000","last_fullsized":"2024-04-14T08:37:02.763477+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:19:53.181475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208499+0000","last_change":"2024-04-14T08:37:00.766579+0000","last_active":"2024-04-14T08:37:03.208499+0000","last_peered":"2024-04-14T08:37:03.208499+0000","last_clean":"2024-04-14T08:37:03.208499+0000","last_became_active":"2024-04-14T08:37:00.766265+0000","last_became_peered":"2024-04-14T08:37:00.766265+0000","last_unstale":"2024-04-14T08:37:03.208499+0000","last_undegraded":"2024-04-14T08:37:03.208499+0000","last_fullsized":"2024-04-14T08:37:03.208499+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:17.340883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144974+0000","last_change":"2024-04-14T08:37:01.766292+0000","last_active":"2024-04-14T08:37:03.144974+0000","last_peered":"2024-04-14T08:37:03.144974+0000","last_clean":"2024-04-14T08:37:03.144974+0000","last_became_active":"2024-04-14T08:37:01.766029+0000","last_became_peered":"2024-04-14T08:37:01.766029+0000","last_unstale":"2024-04-14T08:37:03.144974+0000","last_undegraded":"2024-04-14T08:37:03.144974+0000","last_fullsized":"2024-04-14T08:37:03.144974+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:34.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208449+0000","last_change":"2024-04-14T08:37:00.761704+0000","last_active":"2024-04-14T08:37:03.208449+0000","last_peered":"2024-04-14T08:37:03.208449+0000","last_clean":"2024-04-14T08:37:03.208449+0000","last_became_active":"2024-04-14T08:37:00.761307+0000","last_became_peered":"2024-04-14T08:37:00.761307+0000","last_unstale":"2024-04-14T08:37:03.208449+0000","last_undegraded":"2024-04-14T08:37:03.208449+0000","last_fullsized":"2024-04-14T08:37:03.208449+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:16.681440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.767946+0000","last_change":"2024-04-14T08:37:01.771199+0000","last_active":"2024-04-14T08:37:02.767946+0000","last_peered":"2024-04-14T08:37:02.767946+0000","last_clean":"2024-04-14T08:37:02.767946+0000","last_became_active":"2024-04-14T08:37:01.770913+0000","last_became_peered":"2024-04-14T08:37:01.770913+0000","last_unstale":"2024-04-14T08:37:02.767946+0000","last_undegraded":"2024-04-14T08:37:02.767946+0000","last_fullsized":"2024-04-14T08:37:02.767946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:53:03.323845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208688+0000","last_change":"2024-04-14T08:37:00.763100+0000","last_active":"2024-04-14T08:37:03.208688+0000","last_peered":"2024-04-14T08:37:03.208688+0000","last_clean":"2024-04-14T08:37:03.208688+0000","last_became_active":"2024-04-14T08:37:00.762812+0000","last_became_peered":"2024-04-14T08:37:00.762812+0000","last_unstale":"2024-04-14T08:37:03.208688+0000","last_undegraded":"2024-04-14T08:37:03.208688+0000","last_fullsized":"2024-04-14T08:37:03.208688+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:19.304261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144693+0000","last_change":"2024-04-14T08:37:01.769227+0000","last_active":"2024-04-14T08:37:03.144693+0000","last_peered":"2024-04-14T08:37:03.144693+0000","last_clean":"2024-04-14T08:37:03.144693+0000","last_became_active":"2024-04-14T08:37:01.768970+0000","last_became_peered":"2024-04-14T08:37:01.768970+0000","last_unstale":"2024-04-14T08:37:03.144693+0000","last_undegraded":"2024-04-14T08:37:03.144693+0000","last_fullsized":"2024-04-14T08:37:03.144693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:37:58.514399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763543+0000","last_change":"2024-04-14T08:37:00.771984+0000","last_active":"2024-04-14T08:37:02.763543+0000","last_peered":"2024-04-14T08:37:02.763543+0000","last_clean":"2024-04-14T08:37:02.763543+0000","last_became_active":"2024-04-14T08:37:00.771642+0000","last_became_peered":"2024-04-14T08:37:00.771642+0000","last_unstale":"2024-04-14T08:37:02.763543+0000","last_undegraded":"2024-04-14T08:37:02.763543+0000","last_fullsized":"2024-04-14T08:37:02.763543+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:16.681440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768644+0000","last_change":"2024-04-14T08:37:01.772533+0000","last_active":"2024-04-14T08:37:02.768644+0000","last_peered":"2024-04-14T08:37:02.768644+0000","last_clean":"2024-04-14T08:37:02.768644+0000","last_became_active":"2024-04-14T08:37:01.771911+0000","last_became_peered":"2024-04-14T08:37:01.771911+0000","last_unstale":"2024-04-14T08:37:02.768644+0000","last_undegraded":"2024-04-14T08:37:02.768644+0000","last_fullsized":"2024-04-14T08:37:02.768644+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:50:41.201503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.764073+0000","last_change":"2024-04-14T08:37:00.761456+0000","last_active":"2024-04-14T08:37:00.764073+0000","last_peered":"2024-04-14T08:37:00.764073+0000","last_clean":"2024-04-14T08:37:00.764073+0000","last_became_active":"2024-04-14T08:37:00.761137+0000","last_became_peered":"2024-04-14T08:37:00.761137+0000","last_unstale":"2024-04-14T08:37:00.764073+0000","last_undegraded":"2024-04-14T08:37:00.764073+0000","last_fullsized":"2024-04-14T08:37:00.764073+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:12:04.773469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763403+0000","last_change":"2024-04-14T08:37:01.762799+0000","last_active":"2024-04-14T08:37:02.763403+0000","last_peered":"2024-04-14T08:37:02.763403+0000","last_clean":"2024-04-14T08:37:02.763403+0000","last_became_active":"2024-04-14T08:37:01.762522+0000","last_became_peered":"2024-04-14T08:37:01.762522+0000","last_unstale":"2024-04-14T08:37:02.763403+0000","last_undegraded":"2024-04-14T08:37:02.763403+0000","last_fullsized":"2024-04-14T08:37:02.763403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:32.773224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.764175+0000","last_change":"2024-04-14T08:37:00.759937+0000","last_active":"2024-04-14T08:37:00.764175+0000","last_peered":"2024-04-14T08:37:00.764175+0000","last_clean":"2024-04-14T08:37:00.764175+0000","last_became_active":"2024-04-14T08:37:00.759674+0000","last_became_peered":"2024-04-14T08:37:00.759674+0000","last_unstale":"2024-04-14T08:37:00.764175+0000","last_undegraded":"2024-04-14T08:37:00.764175+0000","last_fullsized":"2024-04-14T08:37:00.764175+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:51.579563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144485+0000","last_change":"2024-04-14T08:37:01.766885+0000","last_active":"2024-04-14T08:37:03.144485+0000","last_peered":"2024-04-14T08:37:03.144485+0000","last_clean":"2024-04-14T08:37:03.144485+0000","last_became_active":"2024-04-14T08:37:01.766617+0000","last_became_peered":"2024-04-14T08:37:01.766617+0000","last_unstale":"2024-04-14T08:37:03.144485+0000","last_undegraded":"2024-04-14T08:37:03.144485+0000","last_fullsized":"2024-04-14T08:37:03.144485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:20.312990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144312+0000","last_change":"2024-04-14T08:37:00.760515+0000","last_active":"2024-04-14T08:37:03.144312+0000","last_peered":"2024-04-14T08:37:03.144312+0000","last_clean":"2024-04-14T08:37:03.144312+0000","last_became_active":"2024-04-14T08:37:00.760240+0000","last_became_peered":"2024-04-14T08:37:00.760240+0000","last_unstale":"2024-04-14T08:37:03.144312+0000","last_undegraded":"2024-04-14T08:37:03.144312+0000","last_fullsized":"2024-04-14T08:37:03.144312+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:12:19.288825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144650+0000","last_change":"2024-04-14T08:37:01.773147+0000","last_active":"2024-04-14T08:37:03.144650+0000","last_peered":"2024-04-14T08:37:03.144650+0000","last_clean":"2024-04-14T08:37:03.144650+0000","last_became_active":"2024-04-14T08:37:01.772776+0000","last_became_peered":"2024-04-14T08:37:01.772776+0000","last_unstale":"2024-04-14T08:37:03.144650+0000","last_undegraded":"2024-04-14T08:37:03.144650+0000","last_fullsized":"2024-04-14T08:37:03.144650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:44:21.151127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209058+0000","last_change":"2024-04-14T08:37:00.766360+0000","last_active":"2024-04-14T08:37:03.209058+0000","last_peered":"2024-04-14T08:37:03.209058+0000","last_clean":"2024-04-14T08:37:03.209058+0000","last_became_active":"2024-04-14T08:37:00.766057+0000","last_became_peered":"2024-04-14T08:37:00.766057+0000","last_unstale":"2024-04-14T08:37:03.209058+0000","last_undegraded":"2024-04-14T08:37:03.209058+0000","last_fullsized":"2024-04-14T08:37:03.209058+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:34.555691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208839+0000","last_change":"2024-04-14T08:37:01.763582+0000","last_active":"2024-04-14T08:37:03.208839+0000","last_peered":"2024-04-14T08:37:03.208839+0000","last_clean":"2024-04-14T08:37:03.208839+0000","last_became_active":"2024-04-14T08:37:01.763312+0000","last_became_peered":"2024-04-14T08:37:01.763312+0000","last_unstale":"2024-04-14T08:37:03.208839+0000","last_undegraded":"2024-04-14T08:37:03.208839+0000","last_fullsized":"2024-04-14T08:37:03.208839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:16:37.842242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144522+0000","last_change":"2024-04-14T08:37:00.766273+0000","last_active":"2024-04-14T08:37:03.144522+0000","last_peered":"2024-04-14T08:37:03.144522+0000","last_clean":"2024-04-14T08:37:03.144522+0000","last_became_active":"2024-04-14T08:37:00.766000+0000","last_became_peered":"2024-04-14T08:37:00.766000+0000","last_unstale":"2024-04-14T08:37:03.144522+0000","last_undegraded":"2024-04-14T08:37:03.144522+0000","last_fullsized":"2024-04-14T08:37:03.144522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:27:44.843714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209131+0000","last_change":"2024-04-14T08:37:01.757979+0000","last_active":"2024-04-14T08:37:03.209131+0000","last_peered":"2024-04-14T08:37:03.209131+0000","last_clean":"2024-04-14T08:37:03.209131+0000","last_became_active":"2024-04-14T08:37:01.757697+0000","last_became_peered":"2024-04-14T08:37:01.757697+0000","last_unstale":"2024-04-14T08:37:03.209131+0000","last_undegraded":"2024-04-14T08:37:03.209131+0000","last_fullsized":"2024-04-14T08:37:03.209131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:12:05.782198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208328+0000","last_change":"2024-04-14T08:37:00.760752+0000","last_active":"2024-04-14T08:37:03.208328+0000","last_peered":"2024-04-14T08:37:03.208328+0000","last_clean":"2024-04-14T08:37:03.208328+0000","last_became_active":"2024-04-14T08:37:00.760459+0000","last_became_peered":"2024-04-14T08:37:00.760459+0000","last_unstale":"2024-04-14T08:37:03.208328+0000","last_undegraded":"2024-04-14T08:37:03.208328+0000","last_fullsized":"2024-04-14T08:37:03.208328+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:07:13.636523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.753179+0000","last_change":"2024-04-14T08:37:00.753179+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.753179+0000","last_undegraded":"2024-04-14T08:37:00.753179+0000","last_fullsized":"2024-04-14T08:37:00.753179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209382+0000","last_change":"2024-04-14T08:37:00.768182+0000","last_active":"2024-04-14T08:37:03.209382+0000","last_peered":"2024-04-14T08:37:03.209382+0000","last_clean":"2024-04-14T08:37:03.209382+0000","last_became_active":"2024-04-14T08:37:00.767282+0000","last_became_peered":"2024-04-14T08:37:00.767282+0000","last_unstale":"2024-04-14T08:37:03.209382+0000","last_undegraded":"2024-04-14T08:37:03.209382+0000","last_fullsized":"2024-04-14T08:37:03.209382+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:24.794262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":274,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.754905+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:37:00.754905+0000","last_peered":"2024-04-14T08:37:00.754905+0000","last_clean":"2024-04-14T08:37:00.754905+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:37:00.754905+0000","last_undegraded":"2024-04-14T08:37:00.754905+0000","last_fullsized":"2024-04-14T08:37:00.754905+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1474560,"data_stored":987323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":412426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":909312,"data_stored":410848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":925696,"data_stored":412530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":411396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":411556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":913408,"data_stored":410926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":1001182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:37:05.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2024-04-14T08:37:05.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:37:00.754006+0000', 'last_change': '2024-04-14T08:37:00.754006+0000', 'last_active': '2024-04-14T08:37:00.742875+0000', 'last_peered': '2024-04-14T08:37:00.742875+0000', 'last_clean': '2024-04-14T08:37:00.742875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:37:00.754006+0000', 'last_undegraded': '2024-04-14T08:37:00.754006+0000', 'last_fullsized': '2024-04-14T08:37:00.754006+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'last_clean_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-14T08:37:05.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2024-04-14T08:37:05.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:37:00.754635+0000', 'last_change': '2024-04-14T08:37:00.754635+0000', 'last_active': '2024-04-14T08:37:00.742875+0000', 'last_peered': '2024-04-14T08:37:00.742875+0000', 'last_clean': '2024-04-14T08:37:00.742875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:37:00.754635+0000', 'last_undegraded': '2024-04-14T08:37:00.754635+0000', 'last_fullsized': '2024-04-14T08:37:00.754635+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'last_clean_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-14T08:37:05.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2024-04-14T08:37:05.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:37:00.754995+0000', 'last_change': '2024-04-14T08:37:00.754995+0000', 'last_active': '2024-04-14T08:37:00.742875+0000', 'last_peered': '2024-04-14T08:37:00.742875+0000', 'last_clean': '2024-04-14T08:37:00.742875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:37:00.754995+0000', 'last_undegraded': '2024-04-14T08:37:00.754995+0000', 'last_fullsized': '2024-04-14T08:37:00.754995+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'last_clean_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-14T08:37:05.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2024-04-14T08:37:05.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:37:00.754088+0000', 'last_change': '2024-04-14T08:37:00.754088+0000', 'last_active': '2024-04-14T08:37:00.742875+0000', 'last_peered': '2024-04-14T08:37:00.742875+0000', 'last_clean': '2024-04-14T08:37:00.742875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:37:00.754088+0000', 'last_undegraded': '2024-04-14T08:37:00.754088+0000', 'last_fullsized': '2024-04-14T08:37:00.754088+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'last_clean_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-14T08:37:05.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2024-04-14T08:37:05.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:37:00.753387+0000', 'last_change': '2024-04-14T08:37:00.753387+0000', 'last_active': '2024-04-14T08:37:00.742875+0000', 'last_peered': '2024-04-14T08:37:00.742875+0000', 'last_clean': '2024-04-14T08:37:00.742875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:37:00.753387+0000', 'last_undegraded': '2024-04-14T08:37:00.753387+0000', 'last_fullsized': '2024-04-14T08:37:00.753387+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'last_clean_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-14T08:37:05.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2024-04-14T08:37:05.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 72, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:37:00.753179+0000', 'last_change': '2024-04-14T08:37:00.753179+0000', 'last_active': '2024-04-14T08:37:00.742875+0000', 'last_peered': '2024-04-14T08:37:00.742875+0000', 'last_clean': '2024-04-14T08:37:00.742875+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:37:00.753179+0000', 'last_undegraded': '2024-04-14T08:37:00.753179+0000', 'last_fullsized': '2024-04-14T08:37:00.753179+0000', 'mapping_epoch': 72, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 72, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'last_clean_scrub_stamp': '2024-04-14T08:37:00.742875+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-14T08:37:05.483 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-14T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.624+0000 7f8232da3640 1 Processor -- start 2024-04-14T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.624+0000 7f8232da3640 1 -- start start 2024-04-14T08:37:05.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.625+0000 7f8232da3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f822c0d2830 0x7f822c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:05.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.625+0000 7f8232da3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f822c0cedd0 0x7f822c0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:05.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.625+0000 7f8232da3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f822c0d3150 con 0x7f822c15e890 2024-04-14T08:37:05.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.625+0000 7f8232da3640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f822c0cff30 con 0x7f822c0d2830 2024-04-14T08:37:05.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.625+0000 7f8232da3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f822c0d2350 con 0x7f822c0cedd0 2024-04-14T08:37:05.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.625+0000 7f8230b3f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f822c0d2830 0x7f822c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:05.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.625+0000 7f8223fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f822c15e890 0x7f822c0d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58542/0 (socket says 172.21.15.172:58542) 2024-04-14T08:37:05.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.625+0000 7f8230b3f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f822c0d2830 0x7f822c0d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55314/0 (socket says 172.21.15.172:55314) 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.625+0000 7f8223fff640 1 -- 172.21.15.172:0/2616058651 learned_addr learned my addr 172.21.15.172:0/2616058651 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.626+0000 7f8231340640 1 --2- 172.21.15.172:0/2616058651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f822c0cedd0 0x7f822c0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.626+0000 7f8230b3f640 1 -- 172.21.15.172:0/2616058651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f822c0cedd0 msgr2=0x7f822c0b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.626+0000 7f8230b3f640 1 --2- 172.21.15.172:0/2616058651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f822c0cedd0 0x7f822c0b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.626+0000 7f8230b3f640 1 -- 172.21.15.172:0/2616058651 >> v1:172.21.15.172:6789/0 conn(0x7f822c15e890 legacy=0x7f822c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.626+0000 7f82237fe640 1 -- 172.21.15.172:0/2616058651 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2492452743 0 0) 0x7f822c0d3150 con 0x7f822c15e890 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.626+0000 7f8230b3f640 1 -- 172.21.15.172:0/2616058651 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c0cf320 con 0x7f822c0d2830 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.626+0000 7f8230b3f640 1 --2- 172.21.15.172:0/2616058651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f822c0d2830 0x7f822c0d2c10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f82280097a0 tx=0x7f822802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f934ecce1e7ffe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.626+0000 7f82237fe640 1 -- 172.21.15.172:0/2616058651 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228034070 con 0x7f822c0d2830 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.627+0000 7f82237fe640 1 -- 172.21.15.172:0/2616058651 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8228030610 con 0x7f822c0d2830 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.627+0000 7f82237fe640 1 -- 172.21.15.172:0/2616058651 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82280309e0 con 0x7f822c0d2830 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.627+0000 7f8232da3640 1 -- 172.21.15.172:0/2616058651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f822c0d2830 msgr2=0x7f822c0d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.627+0000 7f8232da3640 1 --2- 172.21.15.172:0/2616058651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f822c0d2830 0x7f822c0d2c10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f82280097a0 tx=0x7f822802b960 comp rx=0 tx=0).stop 2024-04-14T08:37:05.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.627+0000 7f8232da3640 1 -- 172.21.15.172:0/2616058651 shutdown_connections 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.627+0000 7f8232da3640 1 --2- 172.21.15.172:0/2616058651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f822c0cedd0 0x7f822c0b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.627+0000 7f8232da3640 1 --2- 172.21.15.172:0/2616058651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f822c0d2830 0x7f822c0d2c10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.627+0000 7f8232da3640 1 -- 172.21.15.172:0/2616058651 >> 172.21.15.172:0/2616058651 conn(0x7f822c09d590 msgr2=0x7f822c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.627+0000 7f8232da3640 1 -- 172.21.15.172:0/2616058651 shutdown_connections 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.627+0000 7f8232da3640 1 -- 172.21.15.172:0/2616058651 wait complete. 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.629+0000 7f8232da3640 1 Processor -- start 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.629+0000 7f8232da3640 1 -- start start 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.630+0000 7f8232da3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f822c0cedd0 0x7f822c0c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.630+0000 7f8232da3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f822c0d2830 0x7f822c0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.630+0000 7f8232da3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f822c15e890 0x7f822c0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.630+0000 7f8232da3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0cfbc0 con 0x7f822c0cedd0 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.630+0000 7f8232da3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0cf8e0 con 0x7f822c15e890 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.630+0000 7f8232da3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f822c0b3c50 con 0x7f822c0d2830 2024-04-14T08:37:05.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.630+0000 7f8230b3f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f822c0cedd0 0x7f822c0c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:05.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.630+0000 7f8230b3f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f822c0cedd0 0x7f822c0c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55322/0 (socket says 172.21.15.172:55322) 2024-04-14T08:37:05.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.630+0000 7f8230b3f640 1 -- 172.21.15.172:0/674107491 learned_addr learned my addr 172.21.15.172:0/674107491 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:05.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f8223fff640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f822c0d2830 0x7f822c0c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:05.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f8231340640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f822c15e890 0x7f822c0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:05.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f8230b3f640 1 -- 172.21.15.172:0/674107491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f822c0d2830 msgr2=0x7f822c0c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f8230b3f640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f822c0d2830 0x7f822c0c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f8230b3f640 1 -- 172.21.15.172:0/674107491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f822c15e890 msgr2=0x7f822c0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f8230b3f640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f822c15e890 0x7f822c0ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f8230b3f640 1 -- 172.21.15.172:0/674107491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c0cb320 con 0x7f822c0cedd0 2024-04-14T08:37:05.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f8231340640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f822c15e890 0x7f822c0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:05.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f8230b3f640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f822c0cedd0 0x7f822c0c8220 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f822802bf70 tx=0x7f82280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:05.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f82217fa640 1 -- 172.21.15.172:0/674107491 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228034040 con 0x7f822c0cedd0 2024-04-14T08:37:05.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f82217fa640 1 -- 172.21.15.172:0/674107491 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8228039a10 con 0x7f822c0cedd0 2024-04-14T08:37:05.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f82217fa640 1 -- 172.21.15.172:0/674107491 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228039e20 con 0x7f822c0cedd0 2024-04-14T08:37:05.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f8232da3640 1 -- 172.21.15.172:0/674107491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822c0ca880 con 0x7f822c0cedd0 2024-04-14T08:37:05.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.631+0000 7f8232da3640 1 -- 172.21.15.172:0/674107491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822c0cb5e0 con 0x7f822c0cedd0 2024-04-14T08:37:05.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.632+0000 7f8232da3640 1 -- 172.21.15.172:0/674107491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f822c068520 con 0x7f822c0cedd0 2024-04-14T08:37:05.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.633+0000 7f82217fa640 1 -- 172.21.15.172:0/674107491 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8228037070 con 0x7f822c0cedd0 2024-04-14T08:37:05.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.633+0000 7f82217fa640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f821806c8f0 0x7f821806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:05.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.633+0000 7f82217fa640 1 -- 172.21.15.172:0/674107491 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f82280adc00 con 0x7f822c0cedd0 2024-04-14T08:37:05.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.633+0000 7f8223fff640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f821806c8f0 0x7f821806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:05.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.633+0000 7f8223fff640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f821806c8f0 0x7f821806edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f822c159e40 tx=0x7f821c007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:05.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.637+0000 7f82217fa640 1 -- 172.21.15.172:0/674107491 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f82280b4080 con 0x7f822c0cedd0 2024-04-14T08:37:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.881+0000 7f8232da3640 1 -- 172.21.15.172:0/674107491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f822c0d2610 con 0x7f822c0cedd0 2024-04-14T08:37:05.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.881+0000 7f82217fa640 1 -- 172.21.15.172:0/674107491 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1100 (secure 0 0 0) 0x7f822807bf30 con 0x7f822c0cedd0 2024-04-14T08:37:05.883 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:05.883 INFO:teuthology.orchestra.run.smithi172.stdout:{"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":276,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":74,"num_osds":8,"num_up_osds":8,"osd_up_since":1713083560,"num_in_osds":8,"osd_in_since":1713083551,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":91},{"state_name":"creating+peering","count":6}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592756,"bytes_used":242610176,"bytes_avail":772851503104,"bytes_total":773094113280,"inactive_pgs_ratio":0.061855670064687729,"write_bytes_sec":2452,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":39,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6753}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-14T08:36:47.168716+0000","services":{}},"progress_events":{}} 2024-04-14T08:37:05.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.883+0000 7f8232da3640 1 -- 172.21.15.172:0/674107491 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f821806c8f0 msgr2=0x7f821806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.883+0000 7f8232da3640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f821806c8f0 0x7f821806edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f822c159e40 tx=0x7f821c007de0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.883+0000 7f8232da3640 1 -- 172.21.15.172:0/674107491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f822c0cedd0 msgr2=0x7f822c0c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:05.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.883+0000 7f8232da3640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f822c0cedd0 0x7f822c0c8220 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f822802bf70 tx=0x7f82280046c0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.883+0000 7f8232da3640 1 -- 172.21.15.172:0/674107491 shutdown_connections 2024-04-14T08:37:05.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.883+0000 7f8232da3640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f821806c8f0 0x7f821806edb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.883+0000 7f8232da3640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f822c15e890 0x7f822c0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.883+0000 7f8232da3640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f822c0d2830 0x7f822c0c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.883+0000 7f8232da3640 1 --2- 172.21.15.172:0/674107491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f822c0cedd0 0x7f822c0c8220 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:05.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.883+0000 7f8232da3640 1 -- 172.21.15.172:0/674107491 >> 172.21.15.172:0/674107491 conn(0x7f822c09d590 msgr2=0x7f822c0d4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:05.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.883+0000 7f8232da3640 1 -- 172.21.15.172:0/674107491 shutdown_connections 2024-04-14T08:37:05.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.883+0000 7f8232da3640 1 -- 172.21.15.172:0/674107491 wait complete. 2024-04-14T08:37:05.896 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-14T08:37:05.896 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:37:05.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.996+0000 7f492115e640 1 Processor -- start 2024-04-14T08:37:05.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.996+0000 7f492115e640 1 -- start start 2024-04-14T08:37:06.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f492115e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f491c0d2830 0x7f491c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:06.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f492115e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f491c15e890 0x7f491c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:06.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f492115e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f491c0cedd0 0x7f491c1621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:06.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f492115e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f491c0cff30 con 0x7f491c0d2830 2024-04-14T08:37:06.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f492115e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f491c0d2350 con 0x7f491c0cedd0 2024-04-14T08:37:06.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f492115e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f491c0d3150 con 0x7f491c15e890 2024-04-14T08:37:06.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f491ad9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f491c0d2830 0x7f491c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:06.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f491ad9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f491c0d2830 0x7f491c0d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55334/0 (socket says 172.21.15.172:55334) 2024-04-14T08:37:06.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f491ad9d640 1 -- 172.21.15.172:0/962304036 learned_addr learned my addr 172.21.15.172:0/962304036 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f491b59e640 1 --2- 172.21.15.172:0/962304036 >> v2:172.21.15.177:3300/0 conn(0x7f491c0cedd0 0x7f491c1621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f4913fff640 1 --2- 172.21.15.172:0/962304036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f491c15e890 0x7f491c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f491ad9d640 1 -- 172.21.15.172:0/962304036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f491c15e890 msgr2=0x7f491c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f491ad9d640 1 --2- 172.21.15.172:0/962304036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f491c15e890 0x7f491c0d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f491ad9d640 1 -- 172.21.15.172:0/962304036 >> v2:172.21.15.177:3300/0 conn(0x7f491c0cedd0 msgr2=0x7f491c1621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f491ad9d640 1 --2- 172.21.15.172:0/962304036 >> v2:172.21.15.177:3300/0 conn(0x7f491c0cedd0 0x7f491c1621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.997+0000 7f491ad9d640 1 -- 172.21.15.172:0/962304036 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491c0cf320 con 0x7f491c0d2830 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.998+0000 7f491ad9d640 1 --2- 172.21.15.172:0/962304036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c0d2830 0x7f491c0d2c10 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f491c0d0530 tx=0x7f490c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f67fb6a29b84948 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.998+0000 7f491a59c640 1 -- 172.21.15.172:0/962304036 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c034070 con 0x7f491c0d2830 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.998+0000 7f491a59c640 1 -- 172.21.15.172:0/962304036 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490c030860 con 0x7f491c0d2830 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.998+0000 7f491a59c640 1 -- 172.21.15.172:0/962304036 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c030c30 con 0x7f491c0d2830 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.999+0000 7f492115e640 1 -- 172.21.15.172:0/962304036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c0d2830 msgr2=0x7f491c0d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:06.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.999+0000 7f492115e640 1 --2- 172.21.15.172:0/962304036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c0d2830 0x7f491c0d2c10 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f491c0d0530 tx=0x7f490c02b9a0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.999+0000 7f492115e640 1 -- 172.21.15.172:0/962304036 shutdown_connections 2024-04-14T08:37:06.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.999+0000 7f492115e640 1 --2- 172.21.15.172:0/962304036 >> v2:172.21.15.177:3300/0 conn(0x7f491c0cedd0 0x7f491c1621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.999+0000 7f492115e640 1 --2- 172.21.15.172:0/962304036 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f491c15e890 0x7f491c0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.999+0000 7f492115e640 1 --2- 172.21.15.172:0/962304036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c0d2830 0x7f491c0d2c10 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.999+0000 7f492115e640 1 -- 172.21.15.172:0/962304036 >> 172.21.15.172:0/962304036 conn(0x7f491c09d590 msgr2=0x7f491c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:06.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.999+0000 7f492115e640 1 -- 172.21.15.172:0/962304036 shutdown_connections 2024-04-14T08:37:06.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:05.999+0000 7f492115e640 1 -- 172.21.15.172:0/962304036 wait complete. 2024-04-14T08:37:06.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.000+0000 7f492115e640 1 Processor -- start 2024-04-14T08:37:06.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.001+0000 7f492115e640 1 -- start start 2024-04-14T08:37:06.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.001+0000 7f492115e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f491c0cedd0 0x7f491c155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:06.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.001+0000 7f492115e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f491c0d2830 0x7f491c0b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:06.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.001+0000 7f492115e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c15e890 0x7f491c155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:06.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.001+0000 7f492115e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f491c0cfbc0 con 0x7f491c15e890 2024-04-14T08:37:06.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.001+0000 7f492115e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f491c0cf8e0 con 0x7f491c0d2830 2024-04-14T08:37:06.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.001+0000 7f492115e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f491c0d1840 con 0x7f491c0cedd0 2024-04-14T08:37:06.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.001+0000 7f491b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c15e890 0x7f491c155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:06.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.001+0000 7f491b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c15e890 0x7f491c155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55336/0 (socket says 172.21.15.172:55336) 2024-04-14T08:37:06.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.001+0000 7f491b59e640 1 -- 172.21.15.172:0/3516996081 learned_addr learned my addr 172.21.15.172:0/3516996081 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:06.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f491ad9d640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f491c0cedd0 0x7f491c155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:06.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f4913fff640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f491c0d2830 0x7f491c0b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:06.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f491b59e640 1 -- 172.21.15.172:0/3516996081 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f491c0cedd0 msgr2=0x7f491c155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:06.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f491b59e640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f491c0cedd0 0x7f491c155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f491b59e640 1 -- 172.21.15.172:0/3516996081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f491c0d2830 msgr2=0x7f491c0b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:06.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f491b59e640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f491c0d2830 0x7f491c0b7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f491b59e640 1 -- 172.21.15.172:0/3516996081 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491c0b9410 con 0x7f491c15e890 2024-04-14T08:37:06.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f491ad9d640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f491c0cedd0 0x7f491c155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:06.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f4913fff640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f491c0d2830 0x7f491c0b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:06.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f491b59e640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c15e890 0x7f491c155eb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4914007e60 tx=0x7f491400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:06.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f49137fe640 1 -- 172.21.15.172:0/3516996081 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914014420 con 0x7f491c15e890 2024-04-14T08:37:06.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f49137fe640 1 -- 172.21.15.172:0/3516996081 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4914019070 con 0x7f491c15e890 2024-04-14T08:37:06.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f492115e640 1 -- 172.21.15.172:0/3516996081 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491c0b8890 con 0x7f491c15e890 2024-04-14T08:37:06.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f49137fe640 1 -- 172.21.15.172:0/3516996081 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914014820 con 0x7f491c15e890 2024-04-14T08:37:06.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.002+0000 7f492115e640 1 -- 172.21.15.172:0/3516996081 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491c0b8e50 con 0x7f491c15e890 2024-04-14T08:37:06.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.003+0000 7f492115e640 1 -- 172.21.15.172:0/3516996081 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f491c068520 con 0x7f491c15e890 2024-04-14T08:37:06.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.004+0000 7f49137fe640 1 -- 172.21.15.172:0/3516996081 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49140149c0 con 0x7f491c15e890 2024-04-14T08:37:06.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.004+0000 7f49137fe640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f48fc06c920 0x7f48fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:06.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.004+0000 7f491ad9d640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f48fc06c920 0x7f48fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:06.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.004+0000 7f49137fe640 1 -- 172.21.15.172:0/3516996081 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f491408c480 con 0x7f491c15e890 2024-04-14T08:37:06.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.004+0000 7f491ad9d640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f48fc06c920 0x7f48fc06ede0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f490c036000 tx=0x7f490c03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:06.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.008+0000 7f49137fe640 1 -- 172.21.15.172:0/3516996081 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f491405a7b0 con 0x7f491c15e890 2024-04-14T08:37:06.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.228+0000 7f492115e640 1 -- 172.21.15.172:0/3516996081 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f491c0d2610 con 0x7f48fc06c920 2024-04-14T08:37:06.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.235+0000 7f49137fe640 1 -- 172.21.15.172:0/3516996081 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251570 (secure 0 0 0) 0x7f491c0d2610 con 0x7f48fc06c920 2024-04-14T08:37:06.239 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:06.240 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:37:06.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.238+0000 7f492115e640 1 -- 172.21.15.172:0/3516996081 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f48fc06c920 msgr2=0x7f48fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:06.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.238+0000 7f492115e640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f48fc06c920 0x7f48fc06ede0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f490c036000 tx=0x7f490c03b040 comp rx=0 tx=0).stop 2024-04-14T08:37:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.238+0000 7f492115e640 1 -- 172.21.15.172:0/3516996081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c15e890 msgr2=0x7f491c155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.238+0000 7f492115e640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c15e890 0x7f491c155eb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4914007e60 tx=0x7f491400e4b0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.238+0000 7f492115e640 1 -- 172.21.15.172:0/3516996081 shutdown_connections 2024-04-14T08:37:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.238+0000 7f492115e640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f48fc06c920 0x7f48fc06ede0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.238+0000 7f492115e640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c15e890 0x7f491c155eb0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.238+0000 7f492115e640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f491c0d2830 0x7f491c0b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.238+0000 7f492115e640 1 --2- 172.21.15.172:0/3516996081 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f491c0cedd0 0x7f491c155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.238+0000 7f492115e640 1 -- 172.21.15.172:0/3516996081 >> 172.21.15.172:0/3516996081 conn(0x7f491c09d590 msgr2=0x7f491c0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.238+0000 7f492115e640 1 -- 172.21.15.172:0/3516996081 shutdown_connections 2024-04-14T08:37:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:06.238+0000 7f492115e640 1 -- 172.21.15.172:0/3516996081 wait complete. 2024-04-14T08:37:06.250 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-04-14T08:37:05.171818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":200,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236924,"kb_used_data":8268,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754737796,"statfs":{"total":773094113280,"available":772851503104,"internally_reserved":0,"allocated":8466432,"data_stored":4458187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.428822"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754006+0000","last_change":"2024-04-14T08:37:00.754006+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754006+0000","last_undegraded":"2024-04-14T08:37:00.754006+0000","last_fullsized":"2024-04-14T08:37:00.754006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763468+0000","last_change":"2024-04-14T08:37:01.775666+0000","last_active":"2024-04-14T08:37:02.763468+0000","last_peered":"2024-04-14T08:37:02.763468+0000","last_clean":"2024-04-14T08:37:02.763468+0000","last_became_active":"2024-04-14T08:37:01.775390+0000","last_became_peered":"2024-04-14T08:37:01.775390+0000","last_unstale":"2024-04-14T08:37:02.763468+0000","last_undegraded":"2024-04-14T08:37:02.763468+0000","last_fullsized":"2024-04-14T08:37:02.763468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:28:31.420209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208954+0000","last_change":"2024-04-14T08:37:01.763121+0000","last_active":"2024-04-14T08:37:03.208954+0000","last_peered":"2024-04-14T08:37:03.208954+0000","last_clean":"2024-04-14T08:37:03.208954+0000","last_became_active":"2024-04-14T08:37:01.762855+0000","last_became_peered":"2024-04-14T08:37:01.762855+0000","last_unstale":"2024-04-14T08:37:03.208954+0000","last_undegraded":"2024-04-14T08:37:03.208954+0000","last_fullsized":"2024-04-14T08:37:03.208954+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:52.588292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208816+0000","last_change":"2024-04-14T08:37:01.771514+0000","last_active":"2024-04-14T08:37:03.208816+0000","last_peered":"2024-04-14T08:37:03.208816+0000","last_clean":"2024-04-14T08:37:03.208816+0000","last_became_active":"2024-04-14T08:37:01.770791+0000","last_became_peered":"2024-04-14T08:37:01.770791+0000","last_unstale":"2024-04-14T08:37:03.208816+0000","last_undegraded":"2024-04-14T08:37:03.208816+0000","last_fullsized":"2024-04-14T08:37:03.208816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:19:22.374600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209464+0000","last_change":"2024-04-14T08:37:01.771310+0000","last_active":"2024-04-14T08:37:03.209464+0000","last_peered":"2024-04-14T08:37:03.209464+0000","last_clean":"2024-04-14T08:37:03.209464+0000","last_became_active":"2024-04-14T08:37:01.770803+0000","last_became_peered":"2024-04-14T08:37:01.770803+0000","last_unstale":"2024-04-14T08:37:03.209464+0000","last_undegraded":"2024-04-14T08:37:03.209464+0000","last_fullsized":"2024-04-14T08:37:03.209464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:01:08.014841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208926+0000","last_change":"2024-04-14T08:37:01.769338+0000","last_active":"2024-04-14T08:37:03.208926+0000","last_peered":"2024-04-14T08:37:03.208926+0000","last_clean":"2024-04-14T08:37:03.208926+0000","last_became_active":"2024-04-14T08:37:01.768525+0000","last_became_peered":"2024-04-14T08:37:01.768525+0000","last_unstale":"2024-04-14T08:37:03.208926+0000","last_undegraded":"2024-04-14T08:37:03.208926+0000","last_fullsized":"2024-04-14T08:37:03.208926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:37:40.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144460+0000","last_change":"2024-04-14T08:37:01.777169+0000","last_active":"2024-04-14T08:37:03.144460+0000","last_peered":"2024-04-14T08:37:03.144460+0000","last_clean":"2024-04-14T08:37:03.144460+0000","last_became_active":"2024-04-14T08:37:01.776878+0000","last_became_peered":"2024-04-14T08:37:01.776878+0000","last_unstale":"2024-04-14T08:37:03.144460+0000","last_undegraded":"2024-04-14T08:37:03.144460+0000","last_fullsized":"2024-04-14T08:37:03.144460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:18:49.874828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209058+0000","last_change":"2024-04-14T08:37:01.766727+0000","last_active":"2024-04-14T08:37:03.209058+0000","last_peered":"2024-04-14T08:37:03.209058+0000","last_clean":"2024-04-14T08:37:03.209058+0000","last_became_active":"2024-04-14T08:37:01.766433+0000","last_became_peered":"2024-04-14T08:37:01.766433+0000","last_unstale":"2024-04-14T08:37:03.209058+0000","last_undegraded":"2024-04-14T08:37:03.209058+0000","last_fullsized":"2024-04-14T08:37:03.209058+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:05:52.532330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768847+0000","last_change":"2024-04-14T08:37:01.769926+0000","last_active":"2024-04-14T08:37:02.768847+0000","last_peered":"2024-04-14T08:37:02.768847+0000","last_clean":"2024-04-14T08:37:02.768847+0000","last_became_active":"2024-04-14T08:37:01.769289+0000","last_became_peered":"2024-04-14T08:37:01.769289+0000","last_unstale":"2024-04-14T08:37:02.768847+0000","last_undegraded":"2024-04-14T08:37:02.768847+0000","last_fullsized":"2024-04-14T08:37:02.768847+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:36:56.878085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208763+0000","last_change":"2024-04-14T08:37:01.771927+0000","last_active":"2024-04-14T08:37:03.208763+0000","last_peered":"2024-04-14T08:37:03.208763+0000","last_clean":"2024-04-14T08:37:03.208763+0000","last_became_active":"2024-04-14T08:37:01.771592+0000","last_became_peered":"2024-04-14T08:37:01.771592+0000","last_unstale":"2024-04-14T08:37:03.208763+0000","last_undegraded":"2024-04-14T08:37:03.208763+0000","last_fullsized":"2024-04-14T08:37:03.208763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:32:15.297929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763325+0000","last_change":"2024-04-14T08:37:01.776864+0000","last_active":"2024-04-14T08:37:02.763325+0000","last_peered":"2024-04-14T08:37:02.763325+0000","last_clean":"2024-04-14T08:37:02.763325+0000","last_became_active":"2024-04-14T08:37:01.776407+0000","last_became_peered":"2024-04-14T08:37:01.776407+0000","last_unstale":"2024-04-14T08:37:02.763325+0000","last_undegraded":"2024-04-14T08:37:02.763325+0000","last_fullsized":"2024-04-14T08:37:02.763325+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:37:27.733921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208874+0000","last_change":"2024-04-14T08:37:01.773170+0000","last_active":"2024-04-14T08:37:03.208874+0000","last_peered":"2024-04-14T08:37:03.208874+0000","last_clean":"2024-04-14T08:37:03.208874+0000","last_became_active":"2024-04-14T08:37:01.772844+0000","last_became_peered":"2024-04-14T08:37:01.772844+0000","last_unstale":"2024-04-14T08:37:03.208874+0000","last_undegraded":"2024-04-14T08:37:03.208874+0000","last_fullsized":"2024-04-14T08:37:03.208874+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:24:26.449359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209485+0000","last_change":"2024-04-14T08:37:01.771520+0000","last_active":"2024-04-14T08:37:03.209485+0000","last_peered":"2024-04-14T08:37:03.209485+0000","last_clean":"2024-04-14T08:37:03.209485+0000","last_became_active":"2024-04-14T08:37:01.770987+0000","last_became_peered":"2024-04-14T08:37:01.770987+0000","last_unstale":"2024-04-14T08:37:03.209485+0000","last_undegraded":"2024-04-14T08:37:03.209485+0000","last_fullsized":"2024-04-14T08:37:03.209485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:02:32.886371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209059+0000","last_change":"2024-04-14T08:37:01.769523+0000","last_active":"2024-04-14T08:37:03.209059+0000","last_peered":"2024-04-14T08:37:03.209059+0000","last_clean":"2024-04-14T08:37:03.209059+0000","last_became_active":"2024-04-14T08:37:01.768799+0000","last_became_peered":"2024-04-14T08:37:01.768799+0000","last_unstale":"2024-04-14T08:37:03.209059+0000","last_undegraded":"2024-04-14T08:37:03.209059+0000","last_fullsized":"2024-04-14T08:37:03.209059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:34.673210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763192+0000","last_change":"2024-04-14T08:37:01.767566+0000","last_active":"2024-04-14T08:37:02.763192+0000","last_peered":"2024-04-14T08:37:02.763192+0000","last_clean":"2024-04-14T08:37:02.763192+0000","last_became_active":"2024-04-14T08:37:01.767302+0000","last_became_peered":"2024-04-14T08:37:01.767302+0000","last_unstale":"2024-04-14T08:37:02.763192+0000","last_undegraded":"2024-04-14T08:37:02.763192+0000","last_fullsized":"2024-04-14T08:37:02.763192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:23:26.664547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754635+0000","last_change":"2024-04-14T08:37:00.754635+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754635+0000","last_undegraded":"2024-04-14T08:37:00.754635+0000","last_fullsized":"2024-04-14T08:37:00.754635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209553+0000","last_change":"2024-04-14T08:37:01.773766+0000","last_active":"2024-04-14T08:37:03.209553+0000","last_peered":"2024-04-14T08:37:03.209553+0000","last_clean":"2024-04-14T08:37:03.209553+0000","last_became_active":"2024-04-14T08:37:01.772696+0000","last_became_peered":"2024-04-14T08:37:01.772696+0000","last_unstale":"2024-04-14T08:37:03.209553+0000","last_undegraded":"2024-04-14T08:37:03.209553+0000","last_fullsized":"2024-04-14T08:37:03.209553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:16:41.707052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208679+0000","last_change":"2024-04-14T08:37:01.766048+0000","last_active":"2024-04-14T08:37:03.208679+0000","last_peered":"2024-04-14T08:37:03.208679+0000","last_clean":"2024-04-14T08:37:03.208679+0000","last_became_active":"2024-04-14T08:37:01.765770+0000","last_became_peered":"2024-04-14T08:37:01.765770+0000","last_unstale":"2024-04-14T08:37:03.208679+0000","last_undegraded":"2024-04-14T08:37:03.208679+0000","last_fullsized":"2024-04-14T08:37:03.208679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:46:54.083005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208793+0000","last_change":"2024-04-14T08:37:01.776271+0000","last_active":"2024-04-14T08:37:03.208793+0000","last_peered":"2024-04-14T08:37:03.208793+0000","last_clean":"2024-04-14T08:37:03.208793+0000","last_became_active":"2024-04-14T08:37:01.776041+0000","last_became_peered":"2024-04-14T08:37:01.776041+0000","last_unstale":"2024-04-14T08:37:03.208793+0000","last_undegraded":"2024-04-14T08:37:03.208793+0000","last_fullsized":"2024-04-14T08:37:03.208793+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:46.505374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209553+0000","last_change":"2024-04-14T08:37:01.772630+0000","last_active":"2024-04-14T08:37:03.209553+0000","last_peered":"2024-04-14T08:37:03.209553+0000","last_clean":"2024-04-14T08:37:03.209553+0000","last_became_active":"2024-04-14T08:37:01.772322+0000","last_became_peered":"2024-04-14T08:37:01.772322+0000","last_unstale":"2024-04-14T08:37:03.209553+0000","last_undegraded":"2024-04-14T08:37:03.209553+0000","last_fullsized":"2024-04-14T08:37:03.209553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:39:59.248162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768463+0000","last_change":"2024-04-14T08:37:01.767943+0000","last_active":"2024-04-14T08:37:02.768463+0000","last_peered":"2024-04-14T08:37:02.768463+0000","last_clean":"2024-04-14T08:37:02.768463+0000","last_became_active":"2024-04-14T08:37:01.767670+0000","last_became_peered":"2024-04-14T08:37:01.767670+0000","last_unstale":"2024-04-14T08:37:02.768463+0000","last_undegraded":"2024-04-14T08:37:02.768463+0000","last_fullsized":"2024-04-14T08:37:02.768463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:21:49.412049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209008+0000","last_change":"2024-04-14T08:37:01.774274+0000","last_active":"2024-04-14T08:37:03.209008+0000","last_peered":"2024-04-14T08:37:03.209008+0000","last_clean":"2024-04-14T08:37:03.209008+0000","last_became_active":"2024-04-14T08:37:01.773936+0000","last_became_peered":"2024-04-14T08:37:01.773936+0000","last_unstale":"2024-04-14T08:37:03.209008+0000","last_undegraded":"2024-04-14T08:37:03.209008+0000","last_fullsized":"2024-04-14T08:37:03.209008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:32:01.701513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.767951+0000","last_change":"2024-04-14T08:37:01.769566+0000","last_active":"2024-04-14T08:37:02.767951+0000","last_peered":"2024-04-14T08:37:02.767951+0000","last_clean":"2024-04-14T08:37:02.767951+0000","last_became_active":"2024-04-14T08:37:01.768995+0000","last_became_peered":"2024-04-14T08:37:01.768995+0000","last_unstale":"2024-04-14T08:37:02.767951+0000","last_undegraded":"2024-04-14T08:37:02.767951+0000","last_fullsized":"2024-04-14T08:37:02.767951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:27:45.852443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763062+0000","last_change":"2024-04-14T08:37:01.769784+0000","last_active":"2024-04-14T08:37:02.763062+0000","last_peered":"2024-04-14T08:37:02.763062+0000","last_clean":"2024-04-14T08:37:02.763062+0000","last_became_active":"2024-04-14T08:37:01.769518+0000","last_became_peered":"2024-04-14T08:37:01.769518+0000","last_unstale":"2024-04-14T08:37:02.763062+0000","last_undegraded":"2024-04-14T08:37:02.763062+0000","last_fullsized":"2024-04-14T08:37:02.763062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:34.673210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763297+0000","last_change":"2024-04-14T08:37:01.770106+0000","last_active":"2024-04-14T08:37:02.763297+0000","last_peered":"2024-04-14T08:37:02.763297+0000","last_clean":"2024-04-14T08:37:02.763297+0000","last_became_active":"2024-04-14T08:37:01.769875+0000","last_became_peered":"2024-04-14T08:37:01.769875+0000","last_unstale":"2024-04-14T08:37:02.763297+0000","last_undegraded":"2024-04-14T08:37:02.763297+0000","last_fullsized":"2024-04-14T08:37:02.763297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:33:20.004698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208800+0000","last_change":"2024-04-14T08:37:01.773240+0000","last_active":"2024-04-14T08:37:03.208800+0000","last_peered":"2024-04-14T08:37:03.208800+0000","last_clean":"2024-04-14T08:37:03.208800+0000","last_became_active":"2024-04-14T08:37:01.772799+0000","last_became_peered":"2024-04-14T08:37:01.772799+0000","last_unstale":"2024-04-14T08:37:03.208800+0000","last_undegraded":"2024-04-14T08:37:03.208800+0000","last_fullsized":"2024-04-14T08:37:03.208800+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:35:08.892944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754995+0000","last_change":"2024-04-14T08:37:00.754995+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754995+0000","last_undegraded":"2024-04-14T08:37:00.754995+0000","last_fullsized":"2024-04-14T08:37:00.754995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209656+0000","last_change":"2024-04-14T08:37:01.770780+0000","last_active":"2024-04-14T08:37:03.209656+0000","last_peered":"2024-04-14T08:37:03.209656+0000","last_clean":"2024-04-14T08:37:03.209656+0000","last_became_active":"2024-04-14T08:37:01.770341+0000","last_became_peered":"2024-04-14T08:37:01.770341+0000","last_unstale":"2024-04-14T08:37:03.209656+0000","last_undegraded":"2024-04-14T08:37:03.209656+0000","last_fullsized":"2024-04-14T08:37:03.209656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:32:53.180094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763329+0000","last_change":"2024-04-14T08:37:01.772687+0000","last_active":"2024-04-14T08:37:02.763329+0000","last_peered":"2024-04-14T08:37:02.763329+0000","last_clean":"2024-04-14T08:37:02.763329+0000","last_became_active":"2024-04-14T08:37:01.768377+0000","last_became_peered":"2024-04-14T08:37:01.768377+0000","last_unstale":"2024-04-14T08:37:02.763329+0000","last_undegraded":"2024-04-14T08:37:02.763329+0000","last_fullsized":"2024-04-14T08:37:02.763329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:54:27.630750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.754088+0000","last_change":"2024-04-14T08:37:00.754088+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.754088+0000","last_undegraded":"2024-04-14T08:37:00.754088+0000","last_fullsized":"2024-04-14T08:37:00.754088+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208452+0000","last_change":"2024-04-14T08:37:01.769355+0000","last_active":"2024-04-14T08:37:03.208452+0000","last_peered":"2024-04-14T08:37:03.208452+0000","last_clean":"2024-04-14T08:37:03.208452+0000","last_became_active":"2024-04-14T08:37:01.769040+0000","last_became_peered":"2024-04-14T08:37:01.769040+0000","last_unstale":"2024-04-14T08:37:03.208452+0000","last_undegraded":"2024-04-14T08:37:03.208452+0000","last_fullsized":"2024-04-14T08:37:03.208452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:36:56.878085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.753387+0000","last_change":"2024-04-14T08:37:00.753387+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.753387+0000","last_undegraded":"2024-04-14T08:37:00.753387+0000","last_fullsized":"2024-04-14T08:37:00.753387+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208275+0000","last_change":"2024-04-14T08:37:01.765375+0000","last_active":"2024-04-14T08:37:03.208275+0000","last_peered":"2024-04-14T08:37:03.208275+0000","last_clean":"2024-04-14T08:37:03.208275+0000","last_became_active":"2024-04-14T08:37:01.765108+0000","last_became_peered":"2024-04-14T08:37:01.765108+0000","last_unstale":"2024-04-14T08:37:03.208275+0000","last_undegraded":"2024-04-14T08:37:03.208275+0000","last_fullsized":"2024-04-14T08:37:03.208275+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:50:41.201503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209527+0000","last_change":"2024-04-14T08:37:00.766950+0000","last_active":"2024-04-14T08:37:03.209527+0000","last_peered":"2024-04-14T08:37:03.209527+0000","last_clean":"2024-04-14T08:37:03.209527+0000","last_became_active":"2024-04-14T08:37:00.766663+0000","last_became_peered":"2024-04-14T08:37:00.766663+0000","last_unstale":"2024-04-14T08:37:03.209527+0000","last_undegraded":"2024-04-14T08:37:03.209527+0000","last_fullsized":"2024-04-14T08:37:03.209527+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:45:03.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763294+0000","last_change":"2024-04-14T08:37:01.765910+0000","last_active":"2024-04-14T08:37:02.763294+0000","last_peered":"2024-04-14T08:37:02.763294+0000","last_clean":"2024-04-14T08:37:02.763294+0000","last_became_active":"2024-04-14T08:37:01.765648+0000","last_became_peered":"2024-04-14T08:37:01.765648+0000","last_unstale":"2024-04-14T08:37:02.763294+0000","last_undegraded":"2024-04-14T08:37:02.763294+0000","last_fullsized":"2024-04-14T08:37:02.763294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:14:50.531363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.767745+0000","last_change":"2024-04-14T08:37:00.766237+0000","last_active":"2024-04-14T08:37:00.767745+0000","last_peered":"2024-04-14T08:37:00.767745+0000","last_clean":"2024-04-14T08:37:00.767745+0000","last_became_active":"2024-04-14T08:37:00.765985+0000","last_became_peered":"2024-04-14T08:37:00.765985+0000","last_unstale":"2024-04-14T08:37:00.767745+0000","last_undegraded":"2024-04-14T08:37:00.767745+0000","last_fullsized":"2024-04-14T08:37:00.767745+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:19.304261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768551+0000","last_change":"2024-04-14T08:37:01.772297+0000","last_active":"2024-04-14T08:37:02.768551+0000","last_peered":"2024-04-14T08:37:02.768551+0000","last_clean":"2024-04-14T08:37:02.768551+0000","last_became_active":"2024-04-14T08:37:01.771577+0000","last_became_peered":"2024-04-14T08:37:01.771577+0000","last_unstale":"2024-04-14T08:37:02.768551+0000","last_undegraded":"2024-04-14T08:37:02.768551+0000","last_fullsized":"2024-04-14T08:37:02.768551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:22:46.811060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.769204+0000","last_change":"2024-04-14T08:37:00.766957+0000","last_active":"2024-04-14T08:37:00.769204+0000","last_peered":"2024-04-14T08:37:00.769204+0000","last_clean":"2024-04-14T08:37:00.769204+0000","last_became_active":"2024-04-14T08:37:00.766644+0000","last_became_peered":"2024-04-14T08:37:00.766644+0000","last_unstale":"2024-04-14T08:37:00.769204+0000","last_undegraded":"2024-04-14T08:37:00.769204+0000","last_fullsized":"2024-04-14T08:37:00.769204+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:44:20.142398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768478+0000","last_change":"2024-04-14T08:37:01.767364+0000","last_active":"2024-04-14T08:37:02.768478+0000","last_peered":"2024-04-14T08:37:02.768478+0000","last_clean":"2024-04-14T08:37:02.768478+0000","last_became_active":"2024-04-14T08:37:01.767103+0000","last_became_peered":"2024-04-14T08:37:01.767103+0000","last_unstale":"2024-04-14T08:37:02.768478+0000","last_undegraded":"2024-04-14T08:37:02.768478+0000","last_fullsized":"2024-04-14T08:37:02.768478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:55:08.633674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144522+0000","last_change":"2024-04-14T08:37:00.762961+0000","last_active":"2024-04-14T08:37:03.144522+0000","last_peered":"2024-04-14T08:37:03.144522+0000","last_clean":"2024-04-14T08:37:03.144522+0000","last_became_active":"2024-04-14T08:37:00.762514+0000","last_became_peered":"2024-04-14T08:37:00.762514+0000","last_unstale":"2024-04-14T08:37:03.144522+0000","last_undegraded":"2024-04-14T08:37:03.144522+0000","last_fullsized":"2024-04-14T08:37:03.144522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:04:32.846382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763395+0000","last_change":"2024-04-14T08:37:01.776280+0000","last_active":"2024-04-14T08:37:02.763395+0000","last_peered":"2024-04-14T08:37:02.763395+0000","last_clean":"2024-04-14T08:37:02.763395+0000","last_became_active":"2024-04-14T08:37:01.775999+0000","last_became_peered":"2024-04-14T08:37:01.775999+0000","last_unstale":"2024-04-14T08:37:02.763395+0000","last_undegraded":"2024-04-14T08:37:02.763395+0000","last_fullsized":"2024-04-14T08:37:02.763395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:46:11.850094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768525+0000","last_change":"2024-04-14T08:37:00.759304+0000","last_active":"2024-04-14T08:37:02.768525+0000","last_peered":"2024-04-14T08:37:02.768525+0000","last_clean":"2024-04-14T08:37:02.768525+0000","last_became_active":"2024-04-14T08:37:00.758994+0000","last_became_peered":"2024-04-14T08:37:00.758994+0000","last_unstale":"2024-04-14T08:37:02.768525+0000","last_undegraded":"2024-04-14T08:37:02.768525+0000","last_fullsized":"2024-04-14T08:37:02.768525+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:58:04.441713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763466+0000","last_change":"2024-04-14T08:37:01.776652+0000","last_active":"2024-04-14T08:37:02.763466+0000","last_peered":"2024-04-14T08:37:02.763466+0000","last_clean":"2024-04-14T08:37:02.763466+0000","last_became_active":"2024-04-14T08:37:01.776078+0000","last_became_peered":"2024-04-14T08:37:01.776078+0000","last_unstale":"2024-04-14T08:37:02.763466+0000","last_undegraded":"2024-04-14T08:37:02.763466+0000","last_fullsized":"2024-04-14T08:37:02.763466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:36:16.762703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208453+0000","last_change":"2024-04-14T08:37:00.766216+0000","last_active":"2024-04-14T08:37:03.208453+0000","last_peered":"2024-04-14T08:37:03.208453+0000","last_clean":"2024-04-14T08:37:03.208453+0000","last_became_active":"2024-04-14T08:37:00.765950+0000","last_became_peered":"2024-04-14T08:37:00.765950+0000","last_unstale":"2024-04-14T08:37:03.208453+0000","last_undegraded":"2024-04-14T08:37:03.208453+0000","last_fullsized":"2024-04-14T08:37:03.208453+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:18:48.866099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768872+0000","last_change":"2024-04-14T08:37:01.768600+0000","last_active":"2024-04-14T08:37:02.768872+0000","last_peered":"2024-04-14T08:37:02.768872+0000","last_clean":"2024-04-14T08:37:02.768872+0000","last_became_active":"2024-04-14T08:37:01.768160+0000","last_became_peered":"2024-04-14T08:37:01.768160+0000","last_unstale":"2024-04-14T08:37:02.768872+0000","last_undegraded":"2024-04-14T08:37:02.768872+0000","last_fullsized":"2024-04-14T08:37:02.768872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:46:54.083005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763237+0000","last_change":"2024-04-14T08:37:00.766724+0000","last_active":"2024-04-14T08:37:02.763237+0000","last_peered":"2024-04-14T08:37:02.763237+0000","last_clean":"2024-04-14T08:37:02.763237+0000","last_became_active":"2024-04-14T08:37:00.766452+0000","last_became_peered":"2024-04-14T08:37:00.766452+0000","last_unstale":"2024-04-14T08:37:02.763237+0000","last_undegraded":"2024-04-14T08:37:02.763237+0000","last_fullsized":"2024-04-14T08:37:02.763237+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:13:12.633337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144878+0000","last_change":"2024-04-14T08:37:01.777172+0000","last_active":"2024-04-14T08:37:03.144878+0000","last_peered":"2024-04-14T08:37:03.144878+0000","last_clean":"2024-04-14T08:37:03.144878+0000","last_became_active":"2024-04-14T08:37:01.776849+0000","last_became_peered":"2024-04-14T08:37:01.776849+0000","last_unstale":"2024-04-14T08:37:03.144878+0000","last_undegraded":"2024-04-14T08:37:03.144878+0000","last_fullsized":"2024-04-14T08:37:03.144878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:13:13.642066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209108+0000","last_change":"2024-04-14T08:37:00.765486+0000","last_active":"2024-04-14T08:37:03.209108+0000","last_peered":"2024-04-14T08:37:03.209108+0000","last_clean":"2024-04-14T08:37:03.209108+0000","last_became_active":"2024-04-14T08:37:00.765223+0000","last_became_peered":"2024-04-14T08:37:00.765223+0000","last_unstale":"2024-04-14T08:37:03.209108+0000","last_undegraded":"2024-04-14T08:37:03.209108+0000","last_fullsized":"2024-04-14T08:37:03.209108+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:13:40.708508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763070+0000","last_change":"2024-04-14T08:37:01.762529+0000","last_active":"2024-04-14T08:37:02.763070+0000","last_peered":"2024-04-14T08:37:02.763070+0000","last_clean":"2024-04-14T08:37:02.763070+0000","last_became_active":"2024-04-14T08:37:01.762258+0000","last_became_peered":"2024-04-14T08:37:01.762258+0000","last_unstale":"2024-04-14T08:37:02.763070+0000","last_undegraded":"2024-04-14T08:37:02.763070+0000","last_fullsized":"2024-04-14T08:37:02.763070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:54.396153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.769827+0000","last_change":"2024-04-14T08:37:00.768750+0000","last_active":"2024-04-14T08:37:00.769827+0000","last_peered":"2024-04-14T08:37:00.769827+0000","last_clean":"2024-04-14T08:37:00.769827+0000","last_became_active":"2024-04-14T08:37:00.768482+0000","last_became_peered":"2024-04-14T08:37:00.768482+0000","last_unstale":"2024-04-14T08:37:00.769827+0000","last_undegraded":"2024-04-14T08:37:00.769827+0000","last_fullsized":"2024-04-14T08:37:00.769827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:37:57.505670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208778+0000","last_change":"2024-04-14T08:37:01.764722+0000","last_active":"2024-04-14T08:37:03.208778+0000","last_peered":"2024-04-14T08:37:03.208778+0000","last_clean":"2024-04-14T08:37:03.208778+0000","last_became_active":"2024-04-14T08:37:01.764416+0000","last_became_peered":"2024-04-14T08:37:01.764416+0000","last_unstale":"2024-04-14T08:37:03.208778+0000","last_undegraded":"2024-04-14T08:37:03.208778+0000","last_fullsized":"2024-04-14T08:37:03.208778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:54:12.547002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208818+0000","last_change":"2024-04-14T08:37:00.762713+0000","last_active":"2024-04-14T08:37:03.208818+0000","last_peered":"2024-04-14T08:37:03.208818+0000","last_clean":"2024-04-14T08:37:03.208818+0000","last_became_active":"2024-04-14T08:37:00.762429+0000","last_became_peered":"2024-04-14T08:37:00.762429+0000","last_unstale":"2024-04-14T08:37:03.208818+0000","last_undegraded":"2024-04-14T08:37:03.208818+0000","last_fullsized":"2024-04-14T08:37:03.208818+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:04:42.747296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209258+0000","last_change":"2024-04-14T08:37:01.769140+0000","last_active":"2024-04-14T08:37:03.209258+0000","last_peered":"2024-04-14T08:37:03.209258+0000","last_clean":"2024-04-14T08:37:03.209258+0000","last_became_active":"2024-04-14T08:37:01.768251+0000","last_became_peered":"2024-04-14T08:37:01.768251+0000","last_unstale":"2024-04-14T08:37:03.209258+0000","last_undegraded":"2024-04-14T08:37:03.209258+0000","last_fullsized":"2024-04-14T08:37:03.209258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:54.396153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209671+0000","last_change":"2024-04-14T08:37:00.765119+0000","last_active":"2024-04-14T08:37:03.209671+0000","last_peered":"2024-04-14T08:37:03.209671+0000","last_clean":"2024-04-14T08:37:03.209671+0000","last_became_active":"2024-04-14T08:37:00.764836+0000","last_became_peered":"2024-04-14T08:37:00.764836+0000","last_unstale":"2024-04-14T08:37:03.209671+0000","last_undegraded":"2024-04-14T08:37:03.209671+0000","last_fullsized":"2024-04-14T08:37:03.209671+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:35:49.223991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208803+0000","last_change":"2024-04-14T08:37:01.764255+0000","last_active":"2024-04-14T08:37:03.208803+0000","last_peered":"2024-04-14T08:37:03.208803+0000","last_clean":"2024-04-14T08:37:03.208803+0000","last_became_active":"2024-04-14T08:37:01.763949+0000","last_became_peered":"2024-04-14T08:37:01.763949+0000","last_unstale":"2024-04-14T08:37:03.208803+0000","last_undegraded":"2024-04-14T08:37:03.208803+0000","last_fullsized":"2024-04-14T08:37:03.208803+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:16:13.344979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144550+0000","last_change":"2024-04-14T08:37:00.764900+0000","last_active":"2024-04-14T08:37:03.144550+0000","last_peered":"2024-04-14T08:37:03.144550+0000","last_clean":"2024-04-14T08:37:03.144550+0000","last_became_active":"2024-04-14T08:37:00.764635+0000","last_became_peered":"2024-04-14T08:37:00.764635+0000","last_unstale":"2024-04-14T08:37:03.144550+0000","last_undegraded":"2024-04-14T08:37:03.144550+0000","last_fullsized":"2024-04-14T08:37:03.144550+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:53:02.315116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208925+0000","last_change":"2024-04-14T08:37:01.768718+0000","last_active":"2024-04-14T08:37:03.208925+0000","last_peered":"2024-04-14T08:37:03.208925+0000","last_clean":"2024-04-14T08:37:03.208925+0000","last_became_active":"2024-04-14T08:37:01.768388+0000","last_became_peered":"2024-04-14T08:37:01.768388+0000","last_unstale":"2024-04-14T08:37:03.208925+0000","last_undegraded":"2024-04-14T08:37:03.208925+0000","last_fullsized":"2024-04-14T08:37:03.208925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:40:43.924405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208482+0000","last_change":"2024-04-14T08:37:00.767201+0000","last_active":"2024-04-14T08:37:03.208482+0000","last_peered":"2024-04-14T08:37:03.208482+0000","last_clean":"2024-04-14T08:37:03.208482+0000","last_became_active":"2024-04-14T08:37:00.766926+0000","last_became_peered":"2024-04-14T08:37:00.766926+0000","last_unstale":"2024-04-14T08:37:03.208482+0000","last_undegraded":"2024-04-14T08:37:03.208482+0000","last_fullsized":"2024-04-14T08:37:03.208482+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:44:20.142398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144667+0000","last_change":"2024-04-14T08:37:01.766150+0000","last_active":"2024-04-14T08:37:03.144667+0000","last_peered":"2024-04-14T08:37:03.144667+0000","last_clean":"2024-04-14T08:37:03.144667+0000","last_became_active":"2024-04-14T08:37:01.765865+0000","last_became_peered":"2024-04-14T08:37:01.765865+0000","last_unstale":"2024-04-14T08:37:03.144667+0000","last_undegraded":"2024-04-14T08:37:03.144667+0000","last_fullsized":"2024-04-14T08:37:03.144667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:04:43.756024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.769156+0000","last_change":"2024-04-14T08:37:00.767584+0000","last_active":"2024-04-14T08:37:00.769156+0000","last_peered":"2024-04-14T08:37:00.769156+0000","last_clean":"2024-04-14T08:37:00.769156+0000","last_became_active":"2024-04-14T08:37:00.767283+0000","last_became_peered":"2024-04-14T08:37:00.767283+0000","last_unstale":"2024-04-14T08:37:00.769156+0000","last_undegraded":"2024-04-14T08:37:00.769156+0000","last_fullsized":"2024-04-14T08:37:00.769156+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:04:42.747296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144726+0000","last_change":"2024-04-14T08:37:01.765838+0000","last_active":"2024-04-14T08:37:03.144726+0000","last_peered":"2024-04-14T08:37:03.144726+0000","last_clean":"2024-04-14T08:37:03.144726+0000","last_became_active":"2024-04-14T08:37:01.765086+0000","last_became_peered":"2024-04-14T08:37:01.765086+0000","last_unstale":"2024-04-14T08:37:03.144726+0000","last_undegraded":"2024-04-14T08:37:03.144726+0000","last_fullsized":"2024-04-14T08:37:03.144726+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:57.576896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209258+0000","last_change":"2024-04-14T08:37:00.766886+0000","last_active":"2024-04-14T08:37:03.209258+0000","last_peered":"2024-04-14T08:37:03.209258+0000","last_clean":"2024-04-14T08:37:03.209258+0000","last_became_active":"2024-04-14T08:37:00.766612+0000","last_became_peered":"2024-04-14T08:37:00.766612+0000","last_unstale":"2024-04-14T08:37:03.209258+0000","last_undegraded":"2024-04-14T08:37:03.209258+0000","last_fullsized":"2024-04-14T08:37:03.209258+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:41:00.225037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144883+0000","last_change":"2024-04-14T08:37:01.767730+0000","last_active":"2024-04-14T08:37:03.144883+0000","last_peered":"2024-04-14T08:37:03.144883+0000","last_clean":"2024-04-14T08:37:03.144883+0000","last_became_active":"2024-04-14T08:37:01.767480+0000","last_became_peered":"2024-04-14T08:37:01.767480+0000","last_unstale":"2024-04-14T08:37:03.144883+0000","last_undegraded":"2024-04-14T08:37:03.144883+0000","last_fullsized":"2024-04-14T08:37:03.144883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:17.690169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763123+0000","last_change":"2024-04-14T08:37:00.770164+0000","last_active":"2024-04-14T08:37:02.763123+0000","last_peered":"2024-04-14T08:37:02.763123+0000","last_clean":"2024-04-14T08:37:02.763123+0000","last_became_active":"2024-04-14T08:37:00.769881+0000","last_became_peered":"2024-04-14T08:37:00.769881+0000","last_unstale":"2024-04-14T08:37:02.763123+0000","last_undegraded":"2024-04-14T08:37:02.763123+0000","last_fullsized":"2024-04-14T08:37:02.763123+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:56.568167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208321+0000","last_change":"2024-04-14T08:37:01.775317+0000","last_active":"2024-04-14T08:37:03.208321+0000","last_peered":"2024-04-14T08:37:03.208321+0000","last_clean":"2024-04-14T08:37:03.208321+0000","last_became_active":"2024-04-14T08:37:01.775078+0000","last_became_peered":"2024-04-14T08:37:01.775078+0000","last_unstale":"2024-04-14T08:37:03.208321+0000","last_undegraded":"2024-04-14T08:37:03.208321+0000","last_fullsized":"2024-04-14T08:37:03.208321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:46:55.486835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.765738+0000","last_change":"2024-04-14T08:37:00.761998+0000","last_active":"2024-04-14T08:37:00.765738+0000","last_peered":"2024-04-14T08:37:00.765738+0000","last_clean":"2024-04-14T08:37:00.765738+0000","last_became_active":"2024-04-14T08:37:00.761717+0000","last_became_peered":"2024-04-14T08:37:00.761717+0000","last_unstale":"2024-04-14T08:37:00.765738+0000","last_undegraded":"2024-04-14T08:37:00.765738+0000","last_fullsized":"2024-04-14T08:37:00.765738+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:18:48.866099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209214+0000","last_change":"2024-04-14T08:37:01.773566+0000","last_active":"2024-04-14T08:37:03.209214+0000","last_peered":"2024-04-14T08:37:03.209214+0000","last_clean":"2024-04-14T08:37:03.209214+0000","last_became_active":"2024-04-14T08:37:01.773256+0000","last_became_peered":"2024-04-14T08:37:01.773256+0000","last_unstale":"2024-04-14T08:37:03.209214+0000","last_undegraded":"2024-04-14T08:37:03.209214+0000","last_fullsized":"2024-04-14T08:37:03.209214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:18:48.987307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768932+0000","last_change":"2024-04-14T08:37:00.763865+0000","last_active":"2024-04-14T08:37:02.768932+0000","last_peered":"2024-04-14T08:37:02.768932+0000","last_clean":"2024-04-14T08:37:02.768932+0000","last_became_active":"2024-04-14T08:37:00.763574+0000","last_became_peered":"2024-04-14T08:37:00.763574+0000","last_unstale":"2024-04-14T08:37:02.768932+0000","last_undegraded":"2024-04-14T08:37:02.768932+0000","last_fullsized":"2024-04-14T08:37:02.768932+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:07.846968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208793+0000","last_change":"2024-04-14T08:37:01.775524+0000","last_active":"2024-04-14T08:37:03.208793+0000","last_peered":"2024-04-14T08:37:03.208793+0000","last_clean":"2024-04-14T08:37:03.208793+0000","last_became_active":"2024-04-14T08:37:01.774838+0000","last_became_peered":"2024-04-14T08:37:01.774838+0000","last_unstale":"2024-04-14T08:37:03.208793+0000","last_undegraded":"2024-04-14T08:37:03.208793+0000","last_fullsized":"2024-04-14T08:37:03.208793+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:39:06.052705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209602+0000","last_change":"2024-04-14T08:37:00.767309+0000","last_active":"2024-04-14T08:37:03.209602+0000","last_peered":"2024-04-14T08:37:03.209602+0000","last_clean":"2024-04-14T08:37:03.209602+0000","last_became_active":"2024-04-14T08:37:00.767051+0000","last_became_peered":"2024-04-14T08:37:00.767051+0000","last_unstale":"2024-04-14T08:37:03.209602+0000","last_undegraded":"2024-04-14T08:37:03.209602+0000","last_fullsized":"2024-04-14T08:37:03.209602+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:52:48.671198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208603+0000","last_change":"2024-04-14T08:37:01.775690+0000","last_active":"2024-04-14T08:37:03.208603+0000","last_peered":"2024-04-14T08:37:03.208603+0000","last_clean":"2024-04-14T08:37:03.208603+0000","last_became_active":"2024-04-14T08:37:01.775321+0000","last_became_peered":"2024-04-14T08:37:01.775321+0000","last_unstale":"2024-04-14T08:37:03.208603+0000","last_undegraded":"2024-04-14T08:37:03.208603+0000","last_fullsized":"2024-04-14T08:37:03.208603+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:59:46.004037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208926+0000","last_change":"2024-04-14T08:37:00.767992+0000","last_active":"2024-04-14T08:37:03.208926+0000","last_peered":"2024-04-14T08:37:03.208926+0000","last_clean":"2024-04-14T08:37:03.208926+0000","last_became_active":"2024-04-14T08:37:00.767722+0000","last_became_peered":"2024-04-14T08:37:00.767722+0000","last_unstale":"2024-04-14T08:37:03.208926+0000","last_undegraded":"2024-04-14T08:37:03.208926+0000","last_fullsized":"2024-04-14T08:37:03.208926+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:34.765628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763294+0000","last_change":"2024-04-14T08:37:01.768521+0000","last_active":"2024-04-14T08:37:02.763294+0000","last_peered":"2024-04-14T08:37:02.763294+0000","last_clean":"2024-04-14T08:37:02.763294+0000","last_became_active":"2024-04-14T08:37:01.768215+0000","last_became_peered":"2024-04-14T08:37:01.768215+0000","last_unstale":"2024-04-14T08:37:02.763294+0000","last_undegraded":"2024-04-14T08:37:02.763294+0000","last_fullsized":"2024-04-14T08:37:02.763294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:58:15.131818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208650+0000","last_change":"2024-04-14T08:37:00.762968+0000","last_active":"2024-04-14T08:37:03.208650+0000","last_peered":"2024-04-14T08:37:03.208650+0000","last_clean":"2024-04-14T08:37:03.208650+0000","last_became_active":"2024-04-14T08:37:00.762672+0000","last_became_peered":"2024-04-14T08:37:00.762672+0000","last_unstale":"2024-04-14T08:37:03.208650+0000","last_undegraded":"2024-04-14T08:37:03.208650+0000","last_fullsized":"2024-04-14T08:37:03.208650+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:33.484846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763477+0000","last_change":"2024-04-14T08:37:01.765020+0000","last_active":"2024-04-14T08:37:02.763477+0000","last_peered":"2024-04-14T08:37:02.763477+0000","last_clean":"2024-04-14T08:37:02.763477+0000","last_became_active":"2024-04-14T08:37:01.764736+0000","last_became_peered":"2024-04-14T08:37:01.764736+0000","last_unstale":"2024-04-14T08:37:02.763477+0000","last_undegraded":"2024-04-14T08:37:02.763477+0000","last_fullsized":"2024-04-14T08:37:02.763477+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:19:53.181475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208499+0000","last_change":"2024-04-14T08:37:00.766579+0000","last_active":"2024-04-14T08:37:03.208499+0000","last_peered":"2024-04-14T08:37:03.208499+0000","last_clean":"2024-04-14T08:37:03.208499+0000","last_became_active":"2024-04-14T08:37:00.766265+0000","last_became_peered":"2024-04-14T08:37:00.766265+0000","last_unstale":"2024-04-14T08:37:03.208499+0000","last_undegraded":"2024-04-14T08:37:03.208499+0000","last_fullsized":"2024-04-14T08:37:03.208499+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:17.340883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144974+0000","last_change":"2024-04-14T08:37:01.766292+0000","last_active":"2024-04-14T08:37:03.144974+0000","last_peered":"2024-04-14T08:37:03.144974+0000","last_clean":"2024-04-14T08:37:03.144974+0000","last_became_active":"2024-04-14T08:37:01.766029+0000","last_became_peered":"2024-04-14T08:37:01.766029+0000","last_unstale":"2024-04-14T08:37:03.144974+0000","last_undegraded":"2024-04-14T08:37:03.144974+0000","last_fullsized":"2024-04-14T08:37:03.144974+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:34.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208449+0000","last_change":"2024-04-14T08:37:00.761704+0000","last_active":"2024-04-14T08:37:03.208449+0000","last_peered":"2024-04-14T08:37:03.208449+0000","last_clean":"2024-04-14T08:37:03.208449+0000","last_became_active":"2024-04-14T08:37:00.761307+0000","last_became_peered":"2024-04-14T08:37:00.761307+0000","last_unstale":"2024-04-14T08:37:03.208449+0000","last_undegraded":"2024-04-14T08:37:03.208449+0000","last_fullsized":"2024-04-14T08:37:03.208449+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:16.681440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.767946+0000","last_change":"2024-04-14T08:37:01.771199+0000","last_active":"2024-04-14T08:37:02.767946+0000","last_peered":"2024-04-14T08:37:02.767946+0000","last_clean":"2024-04-14T08:37:02.767946+0000","last_became_active":"2024-04-14T08:37:01.770913+0000","last_became_peered":"2024-04-14T08:37:01.770913+0000","last_unstale":"2024-04-14T08:37:02.767946+0000","last_undegraded":"2024-04-14T08:37:02.767946+0000","last_fullsized":"2024-04-14T08:37:02.767946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:53:03.323845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208688+0000","last_change":"2024-04-14T08:37:00.763100+0000","last_active":"2024-04-14T08:37:03.208688+0000","last_peered":"2024-04-14T08:37:03.208688+0000","last_clean":"2024-04-14T08:37:03.208688+0000","last_became_active":"2024-04-14T08:37:00.762812+0000","last_became_peered":"2024-04-14T08:37:00.762812+0000","last_unstale":"2024-04-14T08:37:03.208688+0000","last_undegraded":"2024-04-14T08:37:03.208688+0000","last_fullsized":"2024-04-14T08:37:03.208688+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:19.304261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144693+0000","last_change":"2024-04-14T08:37:01.769227+0000","last_active":"2024-04-14T08:37:03.144693+0000","last_peered":"2024-04-14T08:37:03.144693+0000","last_clean":"2024-04-14T08:37:03.144693+0000","last_became_active":"2024-04-14T08:37:01.768970+0000","last_became_peered":"2024-04-14T08:37:01.768970+0000","last_unstale":"2024-04-14T08:37:03.144693+0000","last_undegraded":"2024-04-14T08:37:03.144693+0000","last_fullsized":"2024-04-14T08:37:03.144693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:37:58.514399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763543+0000","last_change":"2024-04-14T08:37:00.771984+0000","last_active":"2024-04-14T08:37:02.763543+0000","last_peered":"2024-04-14T08:37:02.763543+0000","last_clean":"2024-04-14T08:37:02.763543+0000","last_became_active":"2024-04-14T08:37:00.771642+0000","last_became_peered":"2024-04-14T08:37:00.771642+0000","last_unstale":"2024-04-14T08:37:02.763543+0000","last_undegraded":"2024-04-14T08:37:02.763543+0000","last_fullsized":"2024-04-14T08:37:02.763543+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:16.681440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768644+0000","last_change":"2024-04-14T08:37:01.772533+0000","last_active":"2024-04-14T08:37:02.768644+0000","last_peered":"2024-04-14T08:37:02.768644+0000","last_clean":"2024-04-14T08:37:02.768644+0000","last_became_active":"2024-04-14T08:37:01.771911+0000","last_became_peered":"2024-04-14T08:37:01.771911+0000","last_unstale":"2024-04-14T08:37:02.768644+0000","last_undegraded":"2024-04-14T08:37:02.768644+0000","last_fullsized":"2024-04-14T08:37:02.768644+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:50:41.201503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.764073+0000","last_change":"2024-04-14T08:37:00.761456+0000","last_active":"2024-04-14T08:37:00.764073+0000","last_peered":"2024-04-14T08:37:00.764073+0000","last_clean":"2024-04-14T08:37:00.764073+0000","last_became_active":"2024-04-14T08:37:00.761137+0000","last_became_peered":"2024-04-14T08:37:00.761137+0000","last_unstale":"2024-04-14T08:37:00.764073+0000","last_undegraded":"2024-04-14T08:37:00.764073+0000","last_fullsized":"2024-04-14T08:37:00.764073+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:12:04.773469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763403+0000","last_change":"2024-04-14T08:37:01.762799+0000","last_active":"2024-04-14T08:37:02.763403+0000","last_peered":"2024-04-14T08:37:02.763403+0000","last_clean":"2024-04-14T08:37:02.763403+0000","last_became_active":"2024-04-14T08:37:01.762522+0000","last_became_peered":"2024-04-14T08:37:01.762522+0000","last_unstale":"2024-04-14T08:37:02.763403+0000","last_undegraded":"2024-04-14T08:37:02.763403+0000","last_fullsized":"2024-04-14T08:37:02.763403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:32.773224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.764175+0000","last_change":"2024-04-14T08:37:00.759937+0000","last_active":"2024-04-14T08:37:00.764175+0000","last_peered":"2024-04-14T08:37:00.764175+0000","last_clean":"2024-04-14T08:37:00.764175+0000","last_became_active":"2024-04-14T08:37:00.759674+0000","last_became_peered":"2024-04-14T08:37:00.759674+0000","last_unstale":"2024-04-14T08:37:00.764175+0000","last_undegraded":"2024-04-14T08:37:00.764175+0000","last_fullsized":"2024-04-14T08:37:00.764175+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:51.579563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144485+0000","last_change":"2024-04-14T08:37:01.766885+0000","last_active":"2024-04-14T08:37:03.144485+0000","last_peered":"2024-04-14T08:37:03.144485+0000","last_clean":"2024-04-14T08:37:03.144485+0000","last_became_active":"2024-04-14T08:37:01.766617+0000","last_became_peered":"2024-04-14T08:37:01.766617+0000","last_unstale":"2024-04-14T08:37:03.144485+0000","last_undegraded":"2024-04-14T08:37:03.144485+0000","last_fullsized":"2024-04-14T08:37:03.144485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:20.312990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144312+0000","last_change":"2024-04-14T08:37:00.760515+0000","last_active":"2024-04-14T08:37:03.144312+0000","last_peered":"2024-04-14T08:37:03.144312+0000","last_clean":"2024-04-14T08:37:03.144312+0000","last_became_active":"2024-04-14T08:37:00.760240+0000","last_became_peered":"2024-04-14T08:37:00.760240+0000","last_unstale":"2024-04-14T08:37:03.144312+0000","last_undegraded":"2024-04-14T08:37:03.144312+0000","last_fullsized":"2024-04-14T08:37:03.144312+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:12:19.288825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144650+0000","last_change":"2024-04-14T08:37:01.773147+0000","last_active":"2024-04-14T08:37:03.144650+0000","last_peered":"2024-04-14T08:37:03.144650+0000","last_clean":"2024-04-14T08:37:03.144650+0000","last_became_active":"2024-04-14T08:37:01.772776+0000","last_became_peered":"2024-04-14T08:37:01.772776+0000","last_unstale":"2024-04-14T08:37:03.144650+0000","last_undegraded":"2024-04-14T08:37:03.144650+0000","last_fullsized":"2024-04-14T08:37:03.144650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:44:21.151127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209058+0000","last_change":"2024-04-14T08:37:00.766360+0000","last_active":"2024-04-14T08:37:03.209058+0000","last_peered":"2024-04-14T08:37:03.209058+0000","last_clean":"2024-04-14T08:37:03.209058+0000","last_became_active":"2024-04-14T08:37:00.766057+0000","last_became_peered":"2024-04-14T08:37:00.766057+0000","last_unstale":"2024-04-14T08:37:03.209058+0000","last_undegraded":"2024-04-14T08:37:03.209058+0000","last_fullsized":"2024-04-14T08:37:03.209058+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:34.555691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208839+0000","last_change":"2024-04-14T08:37:01.763582+0000","last_active":"2024-04-14T08:37:03.208839+0000","last_peered":"2024-04-14T08:37:03.208839+0000","last_clean":"2024-04-14T08:37:03.208839+0000","last_became_active":"2024-04-14T08:37:01.763312+0000","last_became_peered":"2024-04-14T08:37:01.763312+0000","last_unstale":"2024-04-14T08:37:03.208839+0000","last_undegraded":"2024-04-14T08:37:03.208839+0000","last_fullsized":"2024-04-14T08:37:03.208839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:16:37.842242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144522+0000","last_change":"2024-04-14T08:37:00.766273+0000","last_active":"2024-04-14T08:37:03.144522+0000","last_peered":"2024-04-14T08:37:03.144522+0000","last_clean":"2024-04-14T08:37:03.144522+0000","last_became_active":"2024-04-14T08:37:00.766000+0000","last_became_peered":"2024-04-14T08:37:00.766000+0000","last_unstale":"2024-04-14T08:37:03.144522+0000","last_undegraded":"2024-04-14T08:37:03.144522+0000","last_fullsized":"2024-04-14T08:37:03.144522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:27:44.843714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209131+0000","last_change":"2024-04-14T08:37:01.757979+0000","last_active":"2024-04-14T08:37:03.209131+0000","last_peered":"2024-04-14T08:37:03.209131+0000","last_clean":"2024-04-14T08:37:03.209131+0000","last_became_active":"2024-04-14T08:37:01.757697+0000","last_became_peered":"2024-04-14T08:37:01.757697+0000","last_unstale":"2024-04-14T08:37:03.209131+0000","last_undegraded":"2024-04-14T08:37:03.209131+0000","last_fullsized":"2024-04-14T08:37:03.209131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:12:05.782198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208328+0000","last_change":"2024-04-14T08:37:00.760752+0000","last_active":"2024-04-14T08:37:03.208328+0000","last_peered":"2024-04-14T08:37:03.208328+0000","last_clean":"2024-04-14T08:37:03.208328+0000","last_became_active":"2024-04-14T08:37:00.760459+0000","last_became_peered":"2024-04-14T08:37:00.760459+0000","last_unstale":"2024-04-14T08:37:03.208328+0000","last_undegraded":"2024-04-14T08:37:03.208328+0000","last_fullsized":"2024-04-14T08:37:03.208328+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:07:13.636523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":72,"state":"creating+peering","last_fresh":"2024-04-14T08:37:00.753179+0000","last_change":"2024-04-14T08:37:00.753179+0000","last_active":"2024-04-14T08:37:00.742875+0000","last_peered":"2024-04-14T08:37:00.742875+0000","last_clean":"2024-04-14T08:37:00.742875+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:37:00.753179+0000","last_undegraded":"2024-04-14T08:37:00.753179+0000","last_fullsized":"2024-04-14T08:37:00.753179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209382+0000","last_change":"2024-04-14T08:37:00.768182+0000","last_active":"2024-04-14T08:37:03.209382+0000","last_peered":"2024-04-14T08:37:03.209382+0000","last_clean":"2024-04-14T08:37:03.209382+0000","last_became_active":"2024-04-14T08:37:00.767282+0000","last_became_peered":"2024-04-14T08:37:00.767282+0000","last_unstale":"2024-04-14T08:37:03.209382+0000","last_undegraded":"2024-04-14T08:37:03.209382+0000","last_fullsized":"2024-04-14T08:37:03.209382+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:24.794262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":274,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-04-14T08:37:00.754905+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:37:00.754905+0000","last_peered":"2024-04-14T08:37:00.754905+0000","last_clean":"2024-04-14T08:37:00.754905+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:37:00.754905+0000","last_undegraded":"2024-04-14T08:37:00.754905+0000","last_fullsized":"2024-04-14T08:37:00.754905+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1474560,"data_stored":987323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":412426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":909312,"data_stored":410848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":925696,"data_stored":412530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":411396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":411556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":913408,"data_stored":410926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":1001182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:37:09.255 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:37:09.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.360+0000 7f9836cad640 1 Processor -- start 2024-04-14T08:37:09.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.360+0000 7f9836cad640 1 -- start start 2024-04-14T08:37:09.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f9836cad640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f983015e890 0x7f98300d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f9836cad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300cedd0 0x7f98300b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f9836cad640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98300cf320 con 0x7f98300d2830 2024-04-14T08:37:09.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f9836cad640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f98300cff30 con 0x7f983015e890 2024-04-14T08:37:09.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f9836cad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f98300d2350 con 0x7f98300cedd0 2024-04-14T08:37:09.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f9834a49640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f98300d2830 0x7f98300d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58558/0 (socket says 172.21.15.172:58558) 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f9834a49640 1 -- 172.21.15.172:0/2317191311 learned_addr learned my addr 172.21.15.172:0/2317191311 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f983524a640 1 --2- 172.21.15.172:0/2317191311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300cedd0 0x7f98300b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f982ffff640 1 --2- 172.21.15.172:0/2317191311 >> v2:172.21.15.177:3300/0 conn(0x7f983015e890 0x7f98300d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2281185712 0 0) 0x7f98300cf320 con 0x7f98300d2830 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9820003520 con 0x7f98300d2830 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4118508798 0 0) 0x7f9820003520 con 0x7f98300d2830 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300cedd0 msgr2=0x7f98300b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f982f7fe640 1 --2- 172.21.15.172:0/2317191311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300cedd0 0x7f98300b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 >> v2:172.21.15.177:3300/0 conn(0x7f983015e890 msgr2=0x7f98300d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f982f7fe640 1 --2- 172.21.15.172:0/2317191311 >> v2:172.21.15.177:3300/0 conn(0x7f983015e890 0x7f98300d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.362+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98300cf600 con 0x7f98300d2830 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f9824001670 con 0x7f98300d2830 2024-04-14T08:37:09.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 >> v1:172.21.15.172:6789/0 conn(0x7f98300d2830 legacy=0x7f98300d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982f7fe640 1 --2- 172.21.15.172:0/2317191311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98200039b0 0x7f9820003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982f7fe640 1 --2- 172.21.15.172:0/2317191311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9820004800 0x7f9820004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982f7fe640 1 --2- 172.21.15.172:0/2317191311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300cedd0 0x7f98200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98300cff30 con 0x7f9820004800 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98300d2350 con 0x7f98200039b0 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f982000b9c0 con 0x7f98300cedd0 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982ffff640 1 --2- 172.21.15.172:0/2317191311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9820004800 0x7f9820004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f9834a49640 1 --2- 172.21.15.172:0/2317191311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98200039b0 0x7f9820003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f983524a640 1 --2- 172.21.15.172:0/2317191311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300cedd0 0x7f98200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982ffff640 1 -- 172.21.15.172:0/2317191311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300cedd0 msgr2=0x7f98200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982ffff640 1 --2- 172.21.15.172:0/2317191311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300cedd0 0x7f98200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982ffff640 1 -- 172.21.15.172:0/2317191311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98200039b0 msgr2=0x7f9820003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:09.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982ffff640 1 --2- 172.21.15.172:0/2317191311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98200039b0 0x7f9820003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.363+0000 7f982ffff640 1 -- 172.21.15.172:0/2317191311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9820003520 con 0x7f9820004800 2024-04-14T08:37:09.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.364+0000 7f982ffff640 1 --2- 172.21.15.172:0/2317191311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9820004800 0x7f9820004c80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f981c009ac0 tx=0x7f9820005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90689606c30352b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:09.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.364+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c039070 con 0x7f9820004800 2024-04-14T08:37:09.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.364+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981c02c710 con 0x7f9820004800 2024-04-14T08:37:09.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.364+0000 7f982f7fe640 1 -- 172.21.15.172:0/2317191311 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c009690 con 0x7f9820004800 2024-04-14T08:37:09.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.364+0000 7f9836cad640 1 -- 172.21.15.172:0/2317191311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9820004800 msgr2=0x7f9820004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:09.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.364+0000 7f9836cad640 1 --2- 172.21.15.172:0/2317191311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9820004800 0x7f9820004c80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f981c009ac0 tx=0x7f9820005830 comp rx=0 tx=0).stop 2024-04-14T08:37:09.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.364+0000 7f983524a640 1 -- 172.21.15.172:0/2317191311 reap_dead start 2024-04-14T08:37:09.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.364+0000 7f9836cad640 1 -- 172.21.15.172:0/2317191311 shutdown_connections 2024-04-14T08:37:09.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.364+0000 7f9836cad640 1 -- 172.21.15.172:0/2317191311 >> 172.21.15.172:0/2317191311 conn(0x7f983009d590 msgr2=0x7f98300d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:09.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.365+0000 7f9836cad640 1 -- 172.21.15.172:0/2317191311 shutdown_connections 2024-04-14T08:37:09.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.365+0000 7f9836cad640 1 -- 172.21.15.172:0/2317191311 wait complete. 2024-04-14T08:37:09.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.366+0000 7f9836cad640 1 Processor -- start 2024-04-14T08:37:09.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.366+0000 7f9836cad640 1 -- start start 2024-04-14T08:37:09.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.366+0000 7f9836cad640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9820004800 0x7f98301517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.366+0000 7f9836cad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98300b95d0 0x7f98300c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.366+0000 7f9836cad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300b9a50 0x7f98300c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f9836cad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98300d1840 con 0x7f98300b95d0 2024-04-14T08:37:09.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f9836cad640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98300cfbc0 con 0x7f9820004800 2024-04-14T08:37:09.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f9836cad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f98300b3710 con 0x7f98300b9a50 2024-04-14T08:37:09.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f982ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98300b95d0 0x7f98300c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f982ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98300b95d0 0x7f98300c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55356/0 (socket says 172.21.15.172:55356) 2024-04-14T08:37:09.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f982ffff640 1 -- 172.21.15.172:0/1244752867 learned_addr learned my addr 172.21.15.172:0/1244752867 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:09.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f9834a49640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9820004800 0x7f98301517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f983524a640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300b9a50 0x7f98300c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f982ffff640 1 -- 172.21.15.172:0/1244752867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300b9a50 msgr2=0x7f98300c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:09.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f982ffff640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300b9a50 0x7f98300c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f982ffff640 1 -- 172.21.15.172:0/1244752867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9820004800 msgr2=0x7f98301517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f982ffff640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9820004800 0x7f98301517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f982ffff640 1 -- 172.21.15.172:0/1244752867 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98300ca470 con 0x7f98300b95d0 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f9834a49640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9820004800 0x7f98301517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f982ffff640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98300b95d0 0x7f98300c9460 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f981c000c00 tx=0x7f981c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f982d7fa640 1 -- 172.21.15.172:0/1244752867 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c039050 con 0x7f98300b95d0 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f982d7fa640 1 -- 172.21.15.172:0/1244752867 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981c03a040 con 0x7f98300b95d0 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f9836cad640 1 -- 172.21.15.172:0/1244752867 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98300ca750 con 0x7f98300b95d0 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f982d7fa640 1 -- 172.21.15.172:0/1244752867 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c0044e0 con 0x7f98300b95d0 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.367+0000 7f9836cad640 1 -- 172.21.15.172:0/1244752867 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98300c9ee0 con 0x7f98300b95d0 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.368+0000 7f9836cad640 1 -- 172.21.15.172:0/1244752867 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98300cac70 con 0x7f98300b95d0 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.369+0000 7f982d7fa640 1 -- 172.21.15.172:0/1244752867 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f981c047020 con 0x7f98300b95d0 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.369+0000 7f982d7fa640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f981406c920 0x7f981406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.369+0000 7f9834a49640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f981406c920 0x7f981406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.369+0000 7f982d7fa640 1 -- 172.21.15.172:0/1244752867 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f981c0ad560 con 0x7f98300b95d0 2024-04-14T08:37:09.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.370+0000 7f9834a49640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f981406c920 0x7f981406ede0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9824007fd0 tx=0x7f9824009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:09.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.374+0000 7f982d7fa640 1 -- 172.21.15.172:0/1244752867 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f981c07b240 con 0x7f98300b95d0 2024-04-14T08:37:09.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.552+0000 7f9836cad640 1 -- 172.21.15.172:0/1244752867 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9830068520 con 0x7f981406c920 2024-04-14T08:37:09.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.558+0000 7f982d7fa640 1 -- 172.21.15.172:0/1244752867 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7f9830068520 con 0x7f981406c920 2024-04-14T08:37:09.562 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:09.563 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:37:09.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.561+0000 7f9836cad640 1 -- 172.21.15.172:0/1244752867 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f981406c920 msgr2=0x7f981406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:09.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.561+0000 7f9836cad640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f981406c920 0x7f981406ede0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9824007fd0 tx=0x7f9824009040 comp rx=0 tx=0).stop 2024-04-14T08:37:09.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.561+0000 7f9836cad640 1 -- 172.21.15.172:0/1244752867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98300b95d0 msgr2=0x7f98300c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:09.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.561+0000 7f9836cad640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98300b95d0 0x7f98300c9460 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f981c000c00 tx=0x7f981c004730 comp rx=0 tx=0).stop 2024-04-14T08:37:09.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.562+0000 7f9836cad640 1 -- 172.21.15.172:0/1244752867 shutdown_connections 2024-04-14T08:37:09.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.562+0000 7f9836cad640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f981406c920 0x7f981406ede0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.562+0000 7f9836cad640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98300b9a50 0x7f98300c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.562+0000 7f9836cad640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98300b95d0 0x7f98300c9460 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.562+0000 7f9836cad640 1 --2- 172.21.15.172:0/1244752867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9820004800 0x7f98301517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.562+0000 7f9836cad640 1 -- 172.21.15.172:0/1244752867 >> 172.21.15.172:0/1244752867 conn(0x7f983009d590 msgr2=0x7f98300d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:09.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.562+0000 7f9836cad640 1 -- 172.21.15.172:0/1244752867 shutdown_connections 2024-04-14T08:37:09.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.562+0000 7f9836cad640 1 -- 172.21.15.172:0/1244752867 wait complete. 2024-04-14T08:37:09.574 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-04-14T08:37:09.172560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236960,"kb_used_data":8296,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754737760,"statfs":{"total":773094113280,"available":772851466240,"internally_reserved":0,"allocated":8495104,"data_stored":4472068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001377"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768205+0000","last_change":"2024-04-14T08:37:01.763679+0000","last_active":"2024-04-14T08:37:02.768205+0000","last_peered":"2024-04-14T08:37:02.768205+0000","last_clean":"2024-04-14T08:37:02.768205+0000","last_became_active":"2024-04-14T08:37:01.763375+0000","last_became_peered":"2024-04-14T08:37:01.763375+0000","last_unstale":"2024-04-14T08:37:02.768205+0000","last_undegraded":"2024-04-14T08:37:02.768205+0000","last_fullsized":"2024-04-14T08:37:02.768205+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:37:40.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763468+0000","last_change":"2024-04-14T08:37:01.775666+0000","last_active":"2024-04-14T08:37:02.763468+0000","last_peered":"2024-04-14T08:37:02.763468+0000","last_clean":"2024-04-14T08:37:02.763468+0000","last_became_active":"2024-04-14T08:37:01.775390+0000","last_became_peered":"2024-04-14T08:37:01.775390+0000","last_unstale":"2024-04-14T08:37:02.763468+0000","last_undegraded":"2024-04-14T08:37:02.763468+0000","last_fullsized":"2024-04-14T08:37:02.763468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:28:31.420209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208954+0000","last_change":"2024-04-14T08:37:01.763121+0000","last_active":"2024-04-14T08:37:03.208954+0000","last_peered":"2024-04-14T08:37:03.208954+0000","last_clean":"2024-04-14T08:37:03.208954+0000","last_became_active":"2024-04-14T08:37:01.762855+0000","last_became_peered":"2024-04-14T08:37:01.762855+0000","last_unstale":"2024-04-14T08:37:03.208954+0000","last_undegraded":"2024-04-14T08:37:03.208954+0000","last_fullsized":"2024-04-14T08:37:03.208954+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:52.588292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208816+0000","last_change":"2024-04-14T08:37:01.771514+0000","last_active":"2024-04-14T08:37:03.208816+0000","last_peered":"2024-04-14T08:37:03.208816+0000","last_clean":"2024-04-14T08:37:03.208816+0000","last_became_active":"2024-04-14T08:37:01.770791+0000","last_became_peered":"2024-04-14T08:37:01.770791+0000","last_unstale":"2024-04-14T08:37:03.208816+0000","last_undegraded":"2024-04-14T08:37:03.208816+0000","last_fullsized":"2024-04-14T08:37:03.208816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:19:22.374600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209464+0000","last_change":"2024-04-14T08:37:01.771310+0000","last_active":"2024-04-14T08:37:03.209464+0000","last_peered":"2024-04-14T08:37:03.209464+0000","last_clean":"2024-04-14T08:37:03.209464+0000","last_became_active":"2024-04-14T08:37:01.770803+0000","last_became_peered":"2024-04-14T08:37:01.770803+0000","last_unstale":"2024-04-14T08:37:03.209464+0000","last_undegraded":"2024-04-14T08:37:03.209464+0000","last_fullsized":"2024-04-14T08:37:03.209464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:01:08.014841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208926+0000","last_change":"2024-04-14T08:37:01.769338+0000","last_active":"2024-04-14T08:37:03.208926+0000","last_peered":"2024-04-14T08:37:03.208926+0000","last_clean":"2024-04-14T08:37:03.208926+0000","last_became_active":"2024-04-14T08:37:01.768525+0000","last_became_peered":"2024-04-14T08:37:01.768525+0000","last_unstale":"2024-04-14T08:37:03.208926+0000","last_undegraded":"2024-04-14T08:37:03.208926+0000","last_fullsized":"2024-04-14T08:37:03.208926+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:37:40.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144460+0000","last_change":"2024-04-14T08:37:01.777169+0000","last_active":"2024-04-14T08:37:03.144460+0000","last_peered":"2024-04-14T08:37:03.144460+0000","last_clean":"2024-04-14T08:37:03.144460+0000","last_became_active":"2024-04-14T08:37:01.776878+0000","last_became_peered":"2024-04-14T08:37:01.776878+0000","last_unstale":"2024-04-14T08:37:03.144460+0000","last_undegraded":"2024-04-14T08:37:03.144460+0000","last_fullsized":"2024-04-14T08:37:03.144460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:18:49.874828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209058+0000","last_change":"2024-04-14T08:37:01.766727+0000","last_active":"2024-04-14T08:37:03.209058+0000","last_peered":"2024-04-14T08:37:03.209058+0000","last_clean":"2024-04-14T08:37:03.209058+0000","last_became_active":"2024-04-14T08:37:01.766433+0000","last_became_peered":"2024-04-14T08:37:01.766433+0000","last_unstale":"2024-04-14T08:37:03.209058+0000","last_undegraded":"2024-04-14T08:37:03.209058+0000","last_fullsized":"2024-04-14T08:37:03.209058+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:05:52.532330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768847+0000","last_change":"2024-04-14T08:37:01.769926+0000","last_active":"2024-04-14T08:37:02.768847+0000","last_peered":"2024-04-14T08:37:02.768847+0000","last_clean":"2024-04-14T08:37:02.768847+0000","last_became_active":"2024-04-14T08:37:01.769289+0000","last_became_peered":"2024-04-14T08:37:01.769289+0000","last_unstale":"2024-04-14T08:37:02.768847+0000","last_undegraded":"2024-04-14T08:37:02.768847+0000","last_fullsized":"2024-04-14T08:37:02.768847+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:36:56.878085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208763+0000","last_change":"2024-04-14T08:37:01.771927+0000","last_active":"2024-04-14T08:37:03.208763+0000","last_peered":"2024-04-14T08:37:03.208763+0000","last_clean":"2024-04-14T08:37:03.208763+0000","last_became_active":"2024-04-14T08:37:01.771592+0000","last_became_peered":"2024-04-14T08:37:01.771592+0000","last_unstale":"2024-04-14T08:37:03.208763+0000","last_undegraded":"2024-04-14T08:37:03.208763+0000","last_fullsized":"2024-04-14T08:37:03.208763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:32:15.297929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763325+0000","last_change":"2024-04-14T08:37:01.776864+0000","last_active":"2024-04-14T08:37:02.763325+0000","last_peered":"2024-04-14T08:37:02.763325+0000","last_clean":"2024-04-14T08:37:02.763325+0000","last_became_active":"2024-04-14T08:37:01.776407+0000","last_became_peered":"2024-04-14T08:37:01.776407+0000","last_unstale":"2024-04-14T08:37:02.763325+0000","last_undegraded":"2024-04-14T08:37:02.763325+0000","last_fullsized":"2024-04-14T08:37:02.763325+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:37:27.733921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208874+0000","last_change":"2024-04-14T08:37:01.773170+0000","last_active":"2024-04-14T08:37:03.208874+0000","last_peered":"2024-04-14T08:37:03.208874+0000","last_clean":"2024-04-14T08:37:03.208874+0000","last_became_active":"2024-04-14T08:37:01.772844+0000","last_became_peered":"2024-04-14T08:37:01.772844+0000","last_unstale":"2024-04-14T08:37:03.208874+0000","last_undegraded":"2024-04-14T08:37:03.208874+0000","last_fullsized":"2024-04-14T08:37:03.208874+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:24:26.449359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209485+0000","last_change":"2024-04-14T08:37:01.771520+0000","last_active":"2024-04-14T08:37:03.209485+0000","last_peered":"2024-04-14T08:37:03.209485+0000","last_clean":"2024-04-14T08:37:03.209485+0000","last_became_active":"2024-04-14T08:37:01.770987+0000","last_became_peered":"2024-04-14T08:37:01.770987+0000","last_unstale":"2024-04-14T08:37:03.209485+0000","last_undegraded":"2024-04-14T08:37:03.209485+0000","last_fullsized":"2024-04-14T08:37:03.209485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:02:32.886371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209059+0000","last_change":"2024-04-14T08:37:01.769523+0000","last_active":"2024-04-14T08:37:03.209059+0000","last_peered":"2024-04-14T08:37:03.209059+0000","last_clean":"2024-04-14T08:37:03.209059+0000","last_became_active":"2024-04-14T08:37:01.768799+0000","last_became_peered":"2024-04-14T08:37:01.768799+0000","last_unstale":"2024-04-14T08:37:03.209059+0000","last_undegraded":"2024-04-14T08:37:03.209059+0000","last_fullsized":"2024-04-14T08:37:03.209059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:34.673210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763192+0000","last_change":"2024-04-14T08:37:01.767566+0000","last_active":"2024-04-14T08:37:02.763192+0000","last_peered":"2024-04-14T08:37:02.763192+0000","last_clean":"2024-04-14T08:37:02.763192+0000","last_became_active":"2024-04-14T08:37:01.767302+0000","last_became_peered":"2024-04-14T08:37:01.767302+0000","last_unstale":"2024-04-14T08:37:02.763192+0000","last_undegraded":"2024-04-14T08:37:02.763192+0000","last_fullsized":"2024-04-14T08:37:02.763192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:23:26.664547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768759+0000","last_change":"2024-04-14T08:37:01.768724+0000","last_active":"2024-04-14T08:37:02.768759+0000","last_peered":"2024-04-14T08:37:02.768759+0000","last_clean":"2024-04-14T08:37:02.768759+0000","last_became_active":"2024-04-14T08:37:01.768475+0000","last_became_peered":"2024-04-14T08:37:01.768475+0000","last_unstale":"2024-04-14T08:37:02.768759+0000","last_undegraded":"2024-04-14T08:37:02.768759+0000","last_fullsized":"2024-04-14T08:37:02.768759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:54.396153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209553+0000","last_change":"2024-04-14T08:37:01.773766+0000","last_active":"2024-04-14T08:37:03.209553+0000","last_peered":"2024-04-14T08:37:03.209553+0000","last_clean":"2024-04-14T08:37:03.209553+0000","last_became_active":"2024-04-14T08:37:01.772696+0000","last_became_peered":"2024-04-14T08:37:01.772696+0000","last_unstale":"2024-04-14T08:37:03.209553+0000","last_undegraded":"2024-04-14T08:37:03.209553+0000","last_fullsized":"2024-04-14T08:37:03.209553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:16:41.707052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208679+0000","last_change":"2024-04-14T08:37:01.766048+0000","last_active":"2024-04-14T08:37:03.208679+0000","last_peered":"2024-04-14T08:37:03.208679+0000","last_clean":"2024-04-14T08:37:03.208679+0000","last_became_active":"2024-04-14T08:37:01.765770+0000","last_became_peered":"2024-04-14T08:37:01.765770+0000","last_unstale":"2024-04-14T08:37:03.208679+0000","last_undegraded":"2024-04-14T08:37:03.208679+0000","last_fullsized":"2024-04-14T08:37:03.208679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:46:54.083005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208793+0000","last_change":"2024-04-14T08:37:01.776271+0000","last_active":"2024-04-14T08:37:03.208793+0000","last_peered":"2024-04-14T08:37:03.208793+0000","last_clean":"2024-04-14T08:37:03.208793+0000","last_became_active":"2024-04-14T08:37:01.776041+0000","last_became_peered":"2024-04-14T08:37:01.776041+0000","last_unstale":"2024-04-14T08:37:03.208793+0000","last_undegraded":"2024-04-14T08:37:03.208793+0000","last_fullsized":"2024-04-14T08:37:03.208793+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:46.505374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209553+0000","last_change":"2024-04-14T08:37:01.772630+0000","last_active":"2024-04-14T08:37:03.209553+0000","last_peered":"2024-04-14T08:37:03.209553+0000","last_clean":"2024-04-14T08:37:03.209553+0000","last_became_active":"2024-04-14T08:37:01.772322+0000","last_became_peered":"2024-04-14T08:37:01.772322+0000","last_unstale":"2024-04-14T08:37:03.209553+0000","last_undegraded":"2024-04-14T08:37:03.209553+0000","last_fullsized":"2024-04-14T08:37:03.209553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:39:59.248162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768463+0000","last_change":"2024-04-14T08:37:01.767943+0000","last_active":"2024-04-14T08:37:02.768463+0000","last_peered":"2024-04-14T08:37:02.768463+0000","last_clean":"2024-04-14T08:37:02.768463+0000","last_became_active":"2024-04-14T08:37:01.767670+0000","last_became_peered":"2024-04-14T08:37:01.767670+0000","last_unstale":"2024-04-14T08:37:02.768463+0000","last_undegraded":"2024-04-14T08:37:02.768463+0000","last_fullsized":"2024-04-14T08:37:02.768463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:21:49.412049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209008+0000","last_change":"2024-04-14T08:37:01.774274+0000","last_active":"2024-04-14T08:37:03.209008+0000","last_peered":"2024-04-14T08:37:03.209008+0000","last_clean":"2024-04-14T08:37:03.209008+0000","last_became_active":"2024-04-14T08:37:01.773936+0000","last_became_peered":"2024-04-14T08:37:01.773936+0000","last_unstale":"2024-04-14T08:37:03.209008+0000","last_undegraded":"2024-04-14T08:37:03.209008+0000","last_fullsized":"2024-04-14T08:37:03.209008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:32:01.701513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.767951+0000","last_change":"2024-04-14T08:37:01.769566+0000","last_active":"2024-04-14T08:37:02.767951+0000","last_peered":"2024-04-14T08:37:02.767951+0000","last_clean":"2024-04-14T08:37:02.767951+0000","last_became_active":"2024-04-14T08:37:01.768995+0000","last_became_peered":"2024-04-14T08:37:01.768995+0000","last_unstale":"2024-04-14T08:37:02.767951+0000","last_undegraded":"2024-04-14T08:37:02.767951+0000","last_fullsized":"2024-04-14T08:37:02.767951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:27:45.852443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763062+0000","last_change":"2024-04-14T08:37:01.769784+0000","last_active":"2024-04-14T08:37:02.763062+0000","last_peered":"2024-04-14T08:37:02.763062+0000","last_clean":"2024-04-14T08:37:02.763062+0000","last_became_active":"2024-04-14T08:37:01.769518+0000","last_became_peered":"2024-04-14T08:37:01.769518+0000","last_unstale":"2024-04-14T08:37:02.763062+0000","last_undegraded":"2024-04-14T08:37:02.763062+0000","last_fullsized":"2024-04-14T08:37:02.763062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:34.673210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763297+0000","last_change":"2024-04-14T08:37:01.770106+0000","last_active":"2024-04-14T08:37:02.763297+0000","last_peered":"2024-04-14T08:37:02.763297+0000","last_clean":"2024-04-14T08:37:02.763297+0000","last_became_active":"2024-04-14T08:37:01.769875+0000","last_became_peered":"2024-04-14T08:37:01.769875+0000","last_unstale":"2024-04-14T08:37:02.763297+0000","last_undegraded":"2024-04-14T08:37:02.763297+0000","last_fullsized":"2024-04-14T08:37:02.763297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:33:20.004698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208800+0000","last_change":"2024-04-14T08:37:01.773240+0000","last_active":"2024-04-14T08:37:03.208800+0000","last_peered":"2024-04-14T08:37:03.208800+0000","last_clean":"2024-04-14T08:37:03.208800+0000","last_became_active":"2024-04-14T08:37:01.772799+0000","last_became_peered":"2024-04-14T08:37:01.772799+0000","last_unstale":"2024-04-14T08:37:03.208800+0000","last_undegraded":"2024-04-14T08:37:03.208800+0000","last_fullsized":"2024-04-14T08:37:03.208800+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:35:08.892944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768120+0000","last_change":"2024-04-14T08:37:01.765507+0000","last_active":"2024-04-14T08:37:02.768120+0000","last_peered":"2024-04-14T08:37:02.768120+0000","last_clean":"2024-04-14T08:37:02.768120+0000","last_became_active":"2024-04-14T08:37:01.765103+0000","last_became_peered":"2024-04-14T08:37:01.765103+0000","last_unstale":"2024-04-14T08:37:02.768120+0000","last_undegraded":"2024-04-14T08:37:02.768120+0000","last_fullsized":"2024-04-14T08:37:02.768120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:56:46.505374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209656+0000","last_change":"2024-04-14T08:37:01.770780+0000","last_active":"2024-04-14T08:37:03.209656+0000","last_peered":"2024-04-14T08:37:03.209656+0000","last_clean":"2024-04-14T08:37:03.209656+0000","last_became_active":"2024-04-14T08:37:01.770341+0000","last_became_peered":"2024-04-14T08:37:01.770341+0000","last_unstale":"2024-04-14T08:37:03.209656+0000","last_undegraded":"2024-04-14T08:37:03.209656+0000","last_fullsized":"2024-04-14T08:37:03.209656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:32:53.180094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763329+0000","last_change":"2024-04-14T08:37:01.772687+0000","last_active":"2024-04-14T08:37:02.763329+0000","last_peered":"2024-04-14T08:37:02.763329+0000","last_clean":"2024-04-14T08:37:02.763329+0000","last_became_active":"2024-04-14T08:37:01.768377+0000","last_became_peered":"2024-04-14T08:37:01.768377+0000","last_unstale":"2024-04-14T08:37:02.763329+0000","last_undegraded":"2024-04-14T08:37:02.763329+0000","last_fullsized":"2024-04-14T08:37:02.763329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:54:27.630750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.769082+0000","last_change":"2024-04-14T08:37:01.763021+0000","last_active":"2024-04-14T08:37:02.769082+0000","last_peered":"2024-04-14T08:37:02.769082+0000","last_clean":"2024-04-14T08:37:02.769082+0000","last_became_active":"2024-04-14T08:37:01.762705+0000","last_became_peered":"2024-04-14T08:37:01.762705+0000","last_unstale":"2024-04-14T08:37:02.769082+0000","last_undegraded":"2024-04-14T08:37:02.769082+0000","last_fullsized":"2024-04-14T08:37:02.769082+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:58:15.131818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208452+0000","last_change":"2024-04-14T08:37:01.769355+0000","last_active":"2024-04-14T08:37:03.208452+0000","last_peered":"2024-04-14T08:37:03.208452+0000","last_clean":"2024-04-14T08:37:03.208452+0000","last_became_active":"2024-04-14T08:37:01.769040+0000","last_became_peered":"2024-04-14T08:37:01.769040+0000","last_unstale":"2024-04-14T08:37:03.208452+0000","last_undegraded":"2024-04-14T08:37:03.208452+0000","last_fullsized":"2024-04-14T08:37:03.208452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:36:56.878085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768915+0000","last_change":"2024-04-14T08:37:01.765748+0000","last_active":"2024-04-14T08:37:02.768915+0000","last_peered":"2024-04-14T08:37:02.768915+0000","last_clean":"2024-04-14T08:37:02.768915+0000","last_became_active":"2024-04-14T08:37:01.765259+0000","last_became_peered":"2024-04-14T08:37:01.765259+0000","last_unstale":"2024-04-14T08:37:02.768915+0000","last_undegraded":"2024-04-14T08:37:02.768915+0000","last_fullsized":"2024-04-14T08:37:02.768915+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:23:26.664547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208275+0000","last_change":"2024-04-14T08:37:01.765375+0000","last_active":"2024-04-14T08:37:03.208275+0000","last_peered":"2024-04-14T08:37:03.208275+0000","last_clean":"2024-04-14T08:37:03.208275+0000","last_became_active":"2024-04-14T08:37:01.765108+0000","last_became_peered":"2024-04-14T08:37:01.765108+0000","last_unstale":"2024-04-14T08:37:03.208275+0000","last_undegraded":"2024-04-14T08:37:03.208275+0000","last_fullsized":"2024-04-14T08:37:03.208275+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:50:41.201503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209527+0000","last_change":"2024-04-14T08:37:00.766950+0000","last_active":"2024-04-14T08:37:03.209527+0000","last_peered":"2024-04-14T08:37:03.209527+0000","last_clean":"2024-04-14T08:37:03.209527+0000","last_became_active":"2024-04-14T08:37:00.766663+0000","last_became_peered":"2024-04-14T08:37:00.766663+0000","last_unstale":"2024-04-14T08:37:03.209527+0000","last_undegraded":"2024-04-14T08:37:03.209527+0000","last_fullsized":"2024-04-14T08:37:03.209527+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:45:03.621821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763294+0000","last_change":"2024-04-14T08:37:01.765910+0000","last_active":"2024-04-14T08:37:02.763294+0000","last_peered":"2024-04-14T08:37:02.763294+0000","last_clean":"2024-04-14T08:37:02.763294+0000","last_became_active":"2024-04-14T08:37:01.765648+0000","last_became_peered":"2024-04-14T08:37:01.765648+0000","last_unstale":"2024-04-14T08:37:02.763294+0000","last_undegraded":"2024-04-14T08:37:02.763294+0000","last_fullsized":"2024-04-14T08:37:02.763294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:14:50.531363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768009+0000","last_change":"2024-04-14T08:37:00.766237+0000","last_active":"2024-04-14T08:37:02.768009+0000","last_peered":"2024-04-14T08:37:02.768009+0000","last_clean":"2024-04-14T08:37:02.768009+0000","last_became_active":"2024-04-14T08:37:00.765985+0000","last_became_peered":"2024-04-14T08:37:00.765985+0000","last_unstale":"2024-04-14T08:37:02.768009+0000","last_undegraded":"2024-04-14T08:37:02.768009+0000","last_fullsized":"2024-04-14T08:37:02.768009+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:24.794262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768551+0000","last_change":"2024-04-14T08:37:01.772297+0000","last_active":"2024-04-14T08:37:02.768551+0000","last_peered":"2024-04-14T08:37:02.768551+0000","last_clean":"2024-04-14T08:37:02.768551+0000","last_became_active":"2024-04-14T08:37:01.771577+0000","last_became_peered":"2024-04-14T08:37:01.771577+0000","last_unstale":"2024-04-14T08:37:02.768551+0000","last_undegraded":"2024-04-14T08:37:02.768551+0000","last_fullsized":"2024-04-14T08:37:02.768551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:22:46.811060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768091+0000","last_change":"2024-04-14T08:37:00.766957+0000","last_active":"2024-04-14T08:37:02.768091+0000","last_peered":"2024-04-14T08:37:02.768091+0000","last_clean":"2024-04-14T08:37:02.768091+0000","last_became_active":"2024-04-14T08:37:00.766644+0000","last_became_peered":"2024-04-14T08:37:00.766644+0000","last_unstale":"2024-04-14T08:37:02.768091+0000","last_undegraded":"2024-04-14T08:37:02.768091+0000","last_fullsized":"2024-04-14T08:37:02.768091+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:16.681440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768478+0000","last_change":"2024-04-14T08:37:01.767364+0000","last_active":"2024-04-14T08:37:02.768478+0000","last_peered":"2024-04-14T08:37:02.768478+0000","last_clean":"2024-04-14T08:37:02.768478+0000","last_became_active":"2024-04-14T08:37:01.767103+0000","last_became_peered":"2024-04-14T08:37:01.767103+0000","last_unstale":"2024-04-14T08:37:02.768478+0000","last_undegraded":"2024-04-14T08:37:02.768478+0000","last_fullsized":"2024-04-14T08:37:02.768478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:55:08.633674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144522+0000","last_change":"2024-04-14T08:37:00.762961+0000","last_active":"2024-04-14T08:37:03.144522+0000","last_peered":"2024-04-14T08:37:03.144522+0000","last_clean":"2024-04-14T08:37:03.144522+0000","last_became_active":"2024-04-14T08:37:00.762514+0000","last_became_peered":"2024-04-14T08:37:00.762514+0000","last_unstale":"2024-04-14T08:37:03.144522+0000","last_undegraded":"2024-04-14T08:37:03.144522+0000","last_fullsized":"2024-04-14T08:37:03.144522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:04:32.846382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763395+0000","last_change":"2024-04-14T08:37:01.776280+0000","last_active":"2024-04-14T08:37:02.763395+0000","last_peered":"2024-04-14T08:37:02.763395+0000","last_clean":"2024-04-14T08:37:02.763395+0000","last_became_active":"2024-04-14T08:37:01.775999+0000","last_became_peered":"2024-04-14T08:37:01.775999+0000","last_unstale":"2024-04-14T08:37:02.763395+0000","last_undegraded":"2024-04-14T08:37:02.763395+0000","last_fullsized":"2024-04-14T08:37:02.763395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:46:11.850094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768525+0000","last_change":"2024-04-14T08:37:00.759304+0000","last_active":"2024-04-14T08:37:02.768525+0000","last_peered":"2024-04-14T08:37:02.768525+0000","last_clean":"2024-04-14T08:37:02.768525+0000","last_became_active":"2024-04-14T08:37:00.758994+0000","last_became_peered":"2024-04-14T08:37:00.758994+0000","last_unstale":"2024-04-14T08:37:02.768525+0000","last_undegraded":"2024-04-14T08:37:02.768525+0000","last_fullsized":"2024-04-14T08:37:02.768525+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:58:04.441713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763466+0000","last_change":"2024-04-14T08:37:01.776652+0000","last_active":"2024-04-14T08:37:02.763466+0000","last_peered":"2024-04-14T08:37:02.763466+0000","last_clean":"2024-04-14T08:37:02.763466+0000","last_became_active":"2024-04-14T08:37:01.776078+0000","last_became_peered":"2024-04-14T08:37:01.776078+0000","last_unstale":"2024-04-14T08:37:02.763466+0000","last_undegraded":"2024-04-14T08:37:02.763466+0000","last_fullsized":"2024-04-14T08:37:02.763466+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:36:16.762703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208453+0000","last_change":"2024-04-14T08:37:00.766216+0000","last_active":"2024-04-14T08:37:03.208453+0000","last_peered":"2024-04-14T08:37:03.208453+0000","last_clean":"2024-04-14T08:37:03.208453+0000","last_became_active":"2024-04-14T08:37:00.765950+0000","last_became_peered":"2024-04-14T08:37:00.765950+0000","last_unstale":"2024-04-14T08:37:03.208453+0000","last_undegraded":"2024-04-14T08:37:03.208453+0000","last_fullsized":"2024-04-14T08:37:03.208453+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:18:48.866099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768872+0000","last_change":"2024-04-14T08:37:01.768600+0000","last_active":"2024-04-14T08:37:02.768872+0000","last_peered":"2024-04-14T08:37:02.768872+0000","last_clean":"2024-04-14T08:37:02.768872+0000","last_became_active":"2024-04-14T08:37:01.768160+0000","last_became_peered":"2024-04-14T08:37:01.768160+0000","last_unstale":"2024-04-14T08:37:02.768872+0000","last_undegraded":"2024-04-14T08:37:02.768872+0000","last_fullsized":"2024-04-14T08:37:02.768872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:46:54.083005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763237+0000","last_change":"2024-04-14T08:37:00.766724+0000","last_active":"2024-04-14T08:37:02.763237+0000","last_peered":"2024-04-14T08:37:02.763237+0000","last_clean":"2024-04-14T08:37:02.763237+0000","last_became_active":"2024-04-14T08:37:00.766452+0000","last_became_peered":"2024-04-14T08:37:00.766452+0000","last_unstale":"2024-04-14T08:37:02.763237+0000","last_undegraded":"2024-04-14T08:37:02.763237+0000","last_fullsized":"2024-04-14T08:37:02.763237+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:13:12.633337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144878+0000","last_change":"2024-04-14T08:37:01.777172+0000","last_active":"2024-04-14T08:37:03.144878+0000","last_peered":"2024-04-14T08:37:03.144878+0000","last_clean":"2024-04-14T08:37:03.144878+0000","last_became_active":"2024-04-14T08:37:01.776849+0000","last_became_peered":"2024-04-14T08:37:01.776849+0000","last_unstale":"2024-04-14T08:37:03.144878+0000","last_undegraded":"2024-04-14T08:37:03.144878+0000","last_fullsized":"2024-04-14T08:37:03.144878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:13:13.642066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209108+0000","last_change":"2024-04-14T08:37:00.765486+0000","last_active":"2024-04-14T08:37:03.209108+0000","last_peered":"2024-04-14T08:37:03.209108+0000","last_clean":"2024-04-14T08:37:03.209108+0000","last_became_active":"2024-04-14T08:37:00.765223+0000","last_became_peered":"2024-04-14T08:37:00.765223+0000","last_unstale":"2024-04-14T08:37:03.209108+0000","last_undegraded":"2024-04-14T08:37:03.209108+0000","last_fullsized":"2024-04-14T08:37:03.209108+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:13:40.708508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763070+0000","last_change":"2024-04-14T08:37:01.762529+0000","last_active":"2024-04-14T08:37:02.763070+0000","last_peered":"2024-04-14T08:37:02.763070+0000","last_clean":"2024-04-14T08:37:02.763070+0000","last_became_active":"2024-04-14T08:37:01.762258+0000","last_became_peered":"2024-04-14T08:37:01.762258+0000","last_unstale":"2024-04-14T08:37:02.763070+0000","last_undegraded":"2024-04-14T08:37:02.763070+0000","last_fullsized":"2024-04-14T08:37:02.763070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:54.396153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768314+0000","last_change":"2024-04-14T08:37:00.768750+0000","last_active":"2024-04-14T08:37:02.768314+0000","last_peered":"2024-04-14T08:37:02.768314+0000","last_clean":"2024-04-14T08:37:02.768314+0000","last_became_active":"2024-04-14T08:37:00.768482+0000","last_became_peered":"2024-04-14T08:37:00.768482+0000","last_unstale":"2024-04-14T08:37:02.768314+0000","last_undegraded":"2024-04-14T08:37:02.768314+0000","last_fullsized":"2024-04-14T08:37:02.768314+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:32:14.289200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208778+0000","last_change":"2024-04-14T08:37:01.764722+0000","last_active":"2024-04-14T08:37:03.208778+0000","last_peered":"2024-04-14T08:37:03.208778+0000","last_clean":"2024-04-14T08:37:03.208778+0000","last_became_active":"2024-04-14T08:37:01.764416+0000","last_became_peered":"2024-04-14T08:37:01.764416+0000","last_unstale":"2024-04-14T08:37:03.208778+0000","last_undegraded":"2024-04-14T08:37:03.208778+0000","last_fullsized":"2024-04-14T08:37:03.208778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:54:12.547002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208818+0000","last_change":"2024-04-14T08:37:00.762713+0000","last_active":"2024-04-14T08:37:03.208818+0000","last_peered":"2024-04-14T08:37:03.208818+0000","last_clean":"2024-04-14T08:37:03.208818+0000","last_became_active":"2024-04-14T08:37:00.762429+0000","last_became_peered":"2024-04-14T08:37:00.762429+0000","last_unstale":"2024-04-14T08:37:03.208818+0000","last_undegraded":"2024-04-14T08:37:03.208818+0000","last_fullsized":"2024-04-14T08:37:03.208818+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:04:42.747296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209258+0000","last_change":"2024-04-14T08:37:01.769140+0000","last_active":"2024-04-14T08:37:03.209258+0000","last_peered":"2024-04-14T08:37:03.209258+0000","last_clean":"2024-04-14T08:37:03.209258+0000","last_became_active":"2024-04-14T08:37:01.768251+0000","last_became_peered":"2024-04-14T08:37:01.768251+0000","last_unstale":"2024-04-14T08:37:03.209258+0000","last_undegraded":"2024-04-14T08:37:03.209258+0000","last_fullsized":"2024-04-14T08:37:03.209258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:54.396153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209671+0000","last_change":"2024-04-14T08:37:00.765119+0000","last_active":"2024-04-14T08:37:03.209671+0000","last_peered":"2024-04-14T08:37:03.209671+0000","last_clean":"2024-04-14T08:37:03.209671+0000","last_became_active":"2024-04-14T08:37:00.764836+0000","last_became_peered":"2024-04-14T08:37:00.764836+0000","last_unstale":"2024-04-14T08:37:03.209671+0000","last_undegraded":"2024-04-14T08:37:03.209671+0000","last_fullsized":"2024-04-14T08:37:03.209671+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:35:49.223991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208803+0000","last_change":"2024-04-14T08:37:01.764255+0000","last_active":"2024-04-14T08:37:03.208803+0000","last_peered":"2024-04-14T08:37:03.208803+0000","last_clean":"2024-04-14T08:37:03.208803+0000","last_became_active":"2024-04-14T08:37:01.763949+0000","last_became_peered":"2024-04-14T08:37:01.763949+0000","last_unstale":"2024-04-14T08:37:03.208803+0000","last_undegraded":"2024-04-14T08:37:03.208803+0000","last_fullsized":"2024-04-14T08:37:03.208803+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:16:13.344979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144550+0000","last_change":"2024-04-14T08:37:00.764900+0000","last_active":"2024-04-14T08:37:03.144550+0000","last_peered":"2024-04-14T08:37:03.144550+0000","last_clean":"2024-04-14T08:37:03.144550+0000","last_became_active":"2024-04-14T08:37:00.764635+0000","last_became_peered":"2024-04-14T08:37:00.764635+0000","last_unstale":"2024-04-14T08:37:03.144550+0000","last_undegraded":"2024-04-14T08:37:03.144550+0000","last_fullsized":"2024-04-14T08:37:03.144550+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:53:02.315116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208925+0000","last_change":"2024-04-14T08:37:01.768718+0000","last_active":"2024-04-14T08:37:03.208925+0000","last_peered":"2024-04-14T08:37:03.208925+0000","last_clean":"2024-04-14T08:37:03.208925+0000","last_became_active":"2024-04-14T08:37:01.768388+0000","last_became_peered":"2024-04-14T08:37:01.768388+0000","last_unstale":"2024-04-14T08:37:03.208925+0000","last_undegraded":"2024-04-14T08:37:03.208925+0000","last_fullsized":"2024-04-14T08:37:03.208925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:40:43.924405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208482+0000","last_change":"2024-04-14T08:37:00.767201+0000","last_active":"2024-04-14T08:37:03.208482+0000","last_peered":"2024-04-14T08:37:03.208482+0000","last_clean":"2024-04-14T08:37:03.208482+0000","last_became_active":"2024-04-14T08:37:00.766926+0000","last_became_peered":"2024-04-14T08:37:00.766926+0000","last_unstale":"2024-04-14T08:37:03.208482+0000","last_undegraded":"2024-04-14T08:37:03.208482+0000","last_fullsized":"2024-04-14T08:37:03.208482+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:44:20.142398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144667+0000","last_change":"2024-04-14T08:37:01.766150+0000","last_active":"2024-04-14T08:37:03.144667+0000","last_peered":"2024-04-14T08:37:03.144667+0000","last_clean":"2024-04-14T08:37:03.144667+0000","last_became_active":"2024-04-14T08:37:01.765865+0000","last_became_peered":"2024-04-14T08:37:01.765865+0000","last_unstale":"2024-04-14T08:37:03.144667+0000","last_undegraded":"2024-04-14T08:37:03.144667+0000","last_fullsized":"2024-04-14T08:37:03.144667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:04:43.756024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.769041+0000","last_change":"2024-04-14T08:37:00.767584+0000","last_active":"2024-04-14T08:37:02.769041+0000","last_peered":"2024-04-14T08:37:02.769041+0000","last_clean":"2024-04-14T08:37:02.769041+0000","last_became_active":"2024-04-14T08:37:00.767283+0000","last_became_peered":"2024-04-14T08:37:00.767283+0000","last_unstale":"2024-04-14T08:37:02.769041+0000","last_undegraded":"2024-04-14T08:37:02.769041+0000","last_fullsized":"2024-04-14T08:37:02.769041+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:13:12.633337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144726+0000","last_change":"2024-04-14T08:37:01.765838+0000","last_active":"2024-04-14T08:37:03.144726+0000","last_peered":"2024-04-14T08:37:03.144726+0000","last_clean":"2024-04-14T08:37:03.144726+0000","last_became_active":"2024-04-14T08:37:01.765086+0000","last_became_peered":"2024-04-14T08:37:01.765086+0000","last_unstale":"2024-04-14T08:37:03.144726+0000","last_undegraded":"2024-04-14T08:37:03.144726+0000","last_fullsized":"2024-04-14T08:37:03.144726+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:57.576896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209258+0000","last_change":"2024-04-14T08:37:00.766886+0000","last_active":"2024-04-14T08:37:03.209258+0000","last_peered":"2024-04-14T08:37:03.209258+0000","last_clean":"2024-04-14T08:37:03.209258+0000","last_became_active":"2024-04-14T08:37:00.766612+0000","last_became_peered":"2024-04-14T08:37:00.766612+0000","last_unstale":"2024-04-14T08:37:03.209258+0000","last_undegraded":"2024-04-14T08:37:03.209258+0000","last_fullsized":"2024-04-14T08:37:03.209258+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:41:00.225037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144883+0000","last_change":"2024-04-14T08:37:01.767730+0000","last_active":"2024-04-14T08:37:03.144883+0000","last_peered":"2024-04-14T08:37:03.144883+0000","last_clean":"2024-04-14T08:37:03.144883+0000","last_became_active":"2024-04-14T08:37:01.767480+0000","last_became_peered":"2024-04-14T08:37:01.767480+0000","last_unstale":"2024-04-14T08:37:03.144883+0000","last_undegraded":"2024-04-14T08:37:03.144883+0000","last_fullsized":"2024-04-14T08:37:03.144883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:17.690169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763123+0000","last_change":"2024-04-14T08:37:00.770164+0000","last_active":"2024-04-14T08:37:02.763123+0000","last_peered":"2024-04-14T08:37:02.763123+0000","last_clean":"2024-04-14T08:37:02.763123+0000","last_became_active":"2024-04-14T08:37:00.769881+0000","last_became_peered":"2024-04-14T08:37:00.769881+0000","last_unstale":"2024-04-14T08:37:02.763123+0000","last_undegraded":"2024-04-14T08:37:02.763123+0000","last_fullsized":"2024-04-14T08:37:02.763123+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:14:56.568167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208321+0000","last_change":"2024-04-14T08:37:01.775317+0000","last_active":"2024-04-14T08:37:03.208321+0000","last_peered":"2024-04-14T08:37:03.208321+0000","last_clean":"2024-04-14T08:37:03.208321+0000","last_became_active":"2024-04-14T08:37:01.775078+0000","last_became_peered":"2024-04-14T08:37:01.775078+0000","last_unstale":"2024-04-14T08:37:03.208321+0000","last_undegraded":"2024-04-14T08:37:03.208321+0000","last_fullsized":"2024-04-14T08:37:03.208321+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:46:55.486835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768431+0000","last_change":"2024-04-14T08:37:00.761998+0000","last_active":"2024-04-14T08:37:02.768431+0000","last_peered":"2024-04-14T08:37:02.768431+0000","last_clean":"2024-04-14T08:37:02.768431+0000","last_became_active":"2024-04-14T08:37:00.761717+0000","last_became_peered":"2024-04-14T08:37:00.761717+0000","last_unstale":"2024-04-14T08:37:02.768431+0000","last_undegraded":"2024-04-14T08:37:02.768431+0000","last_fullsized":"2024-04-14T08:37:02.768431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:54:11.538274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209214+0000","last_change":"2024-04-14T08:37:01.773566+0000","last_active":"2024-04-14T08:37:03.209214+0000","last_peered":"2024-04-14T08:37:03.209214+0000","last_clean":"2024-04-14T08:37:03.209214+0000","last_became_active":"2024-04-14T08:37:01.773256+0000","last_became_peered":"2024-04-14T08:37:01.773256+0000","last_unstale":"2024-04-14T08:37:03.209214+0000","last_undegraded":"2024-04-14T08:37:03.209214+0000","last_fullsized":"2024-04-14T08:37:03.209214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:18:48.987307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768932+0000","last_change":"2024-04-14T08:37:00.763865+0000","last_active":"2024-04-14T08:37:02.768932+0000","last_peered":"2024-04-14T08:37:02.768932+0000","last_clean":"2024-04-14T08:37:02.768932+0000","last_became_active":"2024-04-14T08:37:00.763574+0000","last_became_peered":"2024-04-14T08:37:00.763574+0000","last_unstale":"2024-04-14T08:37:02.768932+0000","last_undegraded":"2024-04-14T08:37:02.768932+0000","last_fullsized":"2024-04-14T08:37:02.768932+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:48:07.846968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208793+0000","last_change":"2024-04-14T08:37:01.775524+0000","last_active":"2024-04-14T08:37:03.208793+0000","last_peered":"2024-04-14T08:37:03.208793+0000","last_clean":"2024-04-14T08:37:03.208793+0000","last_became_active":"2024-04-14T08:37:01.774838+0000","last_became_peered":"2024-04-14T08:37:01.774838+0000","last_unstale":"2024-04-14T08:37:03.208793+0000","last_undegraded":"2024-04-14T08:37:03.208793+0000","last_fullsized":"2024-04-14T08:37:03.208793+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:39:06.052705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209602+0000","last_change":"2024-04-14T08:37:00.767309+0000","last_active":"2024-04-14T08:37:03.209602+0000","last_peered":"2024-04-14T08:37:03.209602+0000","last_clean":"2024-04-14T08:37:03.209602+0000","last_became_active":"2024-04-14T08:37:00.767051+0000","last_became_peered":"2024-04-14T08:37:00.767051+0000","last_unstale":"2024-04-14T08:37:03.209602+0000","last_undegraded":"2024-04-14T08:37:03.209602+0000","last_fullsized":"2024-04-14T08:37:03.209602+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:52:48.671198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208603+0000","last_change":"2024-04-14T08:37:01.775690+0000","last_active":"2024-04-14T08:37:03.208603+0000","last_peered":"2024-04-14T08:37:03.208603+0000","last_clean":"2024-04-14T08:37:03.208603+0000","last_became_active":"2024-04-14T08:37:01.775321+0000","last_became_peered":"2024-04-14T08:37:01.775321+0000","last_unstale":"2024-04-14T08:37:03.208603+0000","last_undegraded":"2024-04-14T08:37:03.208603+0000","last_fullsized":"2024-04-14T08:37:03.208603+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:59:46.004037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208926+0000","last_change":"2024-04-14T08:37:00.767992+0000","last_active":"2024-04-14T08:37:03.208926+0000","last_peered":"2024-04-14T08:37:03.208926+0000","last_clean":"2024-04-14T08:37:03.208926+0000","last_became_active":"2024-04-14T08:37:00.767722+0000","last_became_peered":"2024-04-14T08:37:00.767722+0000","last_unstale":"2024-04-14T08:37:03.208926+0000","last_undegraded":"2024-04-14T08:37:03.208926+0000","last_fullsized":"2024-04-14T08:37:03.208926+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:42:34.765628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763294+0000","last_change":"2024-04-14T08:37:01.768521+0000","last_active":"2024-04-14T08:37:02.763294+0000","last_peered":"2024-04-14T08:37:02.763294+0000","last_clean":"2024-04-14T08:37:02.763294+0000","last_became_active":"2024-04-14T08:37:01.768215+0000","last_became_peered":"2024-04-14T08:37:01.768215+0000","last_unstale":"2024-04-14T08:37:02.763294+0000","last_undegraded":"2024-04-14T08:37:02.763294+0000","last_fullsized":"2024-04-14T08:37:02.763294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:58:15.131818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208650+0000","last_change":"2024-04-14T08:37:00.762968+0000","last_active":"2024-04-14T08:37:03.208650+0000","last_peered":"2024-04-14T08:37:03.208650+0000","last_clean":"2024-04-14T08:37:03.208650+0000","last_became_active":"2024-04-14T08:37:00.762672+0000","last_became_peered":"2024-04-14T08:37:00.762672+0000","last_unstale":"2024-04-14T08:37:03.208650+0000","last_undegraded":"2024-04-14T08:37:03.208650+0000","last_fullsized":"2024-04-14T08:37:03.208650+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:33.484846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763477+0000","last_change":"2024-04-14T08:37:01.765020+0000","last_active":"2024-04-14T08:37:02.763477+0000","last_peered":"2024-04-14T08:37:02.763477+0000","last_clean":"2024-04-14T08:37:02.763477+0000","last_became_active":"2024-04-14T08:37:01.764736+0000","last_became_peered":"2024-04-14T08:37:01.764736+0000","last_unstale":"2024-04-14T08:37:02.763477+0000","last_undegraded":"2024-04-14T08:37:02.763477+0000","last_fullsized":"2024-04-14T08:37:02.763477+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:19:53.181475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208499+0000","last_change":"2024-04-14T08:37:00.766579+0000","last_active":"2024-04-14T08:37:03.208499+0000","last_peered":"2024-04-14T08:37:03.208499+0000","last_clean":"2024-04-14T08:37:03.208499+0000","last_became_active":"2024-04-14T08:37:00.766265+0000","last_became_peered":"2024-04-14T08:37:00.766265+0000","last_unstale":"2024-04-14T08:37:03.208499+0000","last_undegraded":"2024-04-14T08:37:03.208499+0000","last_fullsized":"2024-04-14T08:37:03.208499+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:17.340883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144974+0000","last_change":"2024-04-14T08:37:01.766292+0000","last_active":"2024-04-14T08:37:03.144974+0000","last_peered":"2024-04-14T08:37:03.144974+0000","last_clean":"2024-04-14T08:37:03.144974+0000","last_became_active":"2024-04-14T08:37:01.766029+0000","last_became_peered":"2024-04-14T08:37:01.766029+0000","last_unstale":"2024-04-14T08:37:03.144974+0000","last_undegraded":"2024-04-14T08:37:03.144974+0000","last_fullsized":"2024-04-14T08:37:03.144974+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:34.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208449+0000","last_change":"2024-04-14T08:37:00.761704+0000","last_active":"2024-04-14T08:37:03.208449+0000","last_peered":"2024-04-14T08:37:03.208449+0000","last_clean":"2024-04-14T08:37:03.208449+0000","last_became_active":"2024-04-14T08:37:00.761307+0000","last_became_peered":"2024-04-14T08:37:00.761307+0000","last_unstale":"2024-04-14T08:37:03.208449+0000","last_undegraded":"2024-04-14T08:37:03.208449+0000","last_fullsized":"2024-04-14T08:37:03.208449+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:16.681440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.767946+0000","last_change":"2024-04-14T08:37:01.771199+0000","last_active":"2024-04-14T08:37:02.767946+0000","last_peered":"2024-04-14T08:37:02.767946+0000","last_clean":"2024-04-14T08:37:02.767946+0000","last_became_active":"2024-04-14T08:37:01.770913+0000","last_became_peered":"2024-04-14T08:37:01.770913+0000","last_unstale":"2024-04-14T08:37:02.767946+0000","last_undegraded":"2024-04-14T08:37:02.767946+0000","last_fullsized":"2024-04-14T08:37:02.767946+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:53:03.323845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208688+0000","last_change":"2024-04-14T08:37:00.763100+0000","last_active":"2024-04-14T08:37:03.208688+0000","last_peered":"2024-04-14T08:37:03.208688+0000","last_clean":"2024-04-14T08:37:03.208688+0000","last_became_active":"2024-04-14T08:37:00.762812+0000","last_became_peered":"2024-04-14T08:37:00.762812+0000","last_unstale":"2024-04-14T08:37:03.208688+0000","last_undegraded":"2024-04-14T08:37:03.208688+0000","last_fullsized":"2024-04-14T08:37:03.208688+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:19.304261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144693+0000","last_change":"2024-04-14T08:37:01.769227+0000","last_active":"2024-04-14T08:37:03.144693+0000","last_peered":"2024-04-14T08:37:03.144693+0000","last_clean":"2024-04-14T08:37:03.144693+0000","last_became_active":"2024-04-14T08:37:01.768970+0000","last_became_peered":"2024-04-14T08:37:01.768970+0000","last_unstale":"2024-04-14T08:37:03.144693+0000","last_undegraded":"2024-04-14T08:37:03.144693+0000","last_fullsized":"2024-04-14T08:37:03.144693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:37:58.514399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763543+0000","last_change":"2024-04-14T08:37:00.771984+0000","last_active":"2024-04-14T08:37:02.763543+0000","last_peered":"2024-04-14T08:37:02.763543+0000","last_clean":"2024-04-14T08:37:02.763543+0000","last_became_active":"2024-04-14T08:37:00.771642+0000","last_became_peered":"2024-04-14T08:37:00.771642+0000","last_unstale":"2024-04-14T08:37:02.763543+0000","last_undegraded":"2024-04-14T08:37:02.763543+0000","last_fullsized":"2024-04-14T08:37:02.763543+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:16.681440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768644+0000","last_change":"2024-04-14T08:37:01.772533+0000","last_active":"2024-04-14T08:37:02.768644+0000","last_peered":"2024-04-14T08:37:02.768644+0000","last_clean":"2024-04-14T08:37:02.768644+0000","last_became_active":"2024-04-14T08:37:01.771911+0000","last_became_peered":"2024-04-14T08:37:01.771911+0000","last_unstale":"2024-04-14T08:37:02.768644+0000","last_undegraded":"2024-04-14T08:37:02.768644+0000","last_fullsized":"2024-04-14T08:37:02.768644+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:50:41.201503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768237+0000","last_change":"2024-04-14T08:37:00.761456+0000","last_active":"2024-04-14T08:37:02.768237+0000","last_peered":"2024-04-14T08:37:02.768237+0000","last_clean":"2024-04-14T08:37:02.768237+0000","last_became_active":"2024-04-14T08:37:00.761137+0000","last_became_peered":"2024-04-14T08:37:00.761137+0000","last_unstale":"2024-04-14T08:37:02.768237+0000","last_undegraded":"2024-04-14T08:37:02.768237+0000","last_fullsized":"2024-04-14T08:37:02.768237+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:17.340883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.763403+0000","last_change":"2024-04-14T08:37:01.762799+0000","last_active":"2024-04-14T08:37:02.763403+0000","last_peered":"2024-04-14T08:37:02.763403+0000","last_clean":"2024-04-14T08:37:02.763403+0000","last_became_active":"2024-04-14T08:37:01.762522+0000","last_became_peered":"2024-04-14T08:37:01.762522+0000","last_unstale":"2024-04-14T08:37:02.763403+0000","last_undegraded":"2024-04-14T08:37:02.763403+0000","last_fullsized":"2024-04-14T08:37:02.763403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:32.773224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.767901+0000","last_change":"2024-04-14T08:37:00.759937+0000","last_active":"2024-04-14T08:37:02.767901+0000","last_peered":"2024-04-14T08:37:02.767901+0000","last_clean":"2024-04-14T08:37:02.767901+0000","last_became_active":"2024-04-14T08:37:00.759674+0000","last_became_peered":"2024-04-14T08:37:00.759674+0000","last_unstale":"2024-04-14T08:37:02.767901+0000","last_undegraded":"2024-04-14T08:37:02.767901+0000","last_fullsized":"2024-04-14T08:37:02.767901+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:33.484846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144485+0000","last_change":"2024-04-14T08:37:01.766885+0000","last_active":"2024-04-14T08:37:03.144485+0000","last_peered":"2024-04-14T08:37:03.144485+0000","last_clean":"2024-04-14T08:37:03.144485+0000","last_became_active":"2024-04-14T08:37:01.766617+0000","last_became_peered":"2024-04-14T08:37:01.766617+0000","last_unstale":"2024-04-14T08:37:03.144485+0000","last_undegraded":"2024-04-14T08:37:03.144485+0000","last_fullsized":"2024-04-14T08:37:03.144485+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:19:20.312990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144312+0000","last_change":"2024-04-14T08:37:00.760515+0000","last_active":"2024-04-14T08:37:03.144312+0000","last_peered":"2024-04-14T08:37:03.144312+0000","last_clean":"2024-04-14T08:37:03.144312+0000","last_became_active":"2024-04-14T08:37:00.760240+0000","last_became_peered":"2024-04-14T08:37:00.760240+0000","last_unstale":"2024-04-14T08:37:03.144312+0000","last_undegraded":"2024-04-14T08:37:03.144312+0000","last_fullsized":"2024-04-14T08:37:03.144312+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:12:19.288825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144650+0000","last_change":"2024-04-14T08:37:01.773147+0000","last_active":"2024-04-14T08:37:03.144650+0000","last_peered":"2024-04-14T08:37:03.144650+0000","last_clean":"2024-04-14T08:37:03.144650+0000","last_became_active":"2024-04-14T08:37:01.772776+0000","last_became_peered":"2024-04-14T08:37:01.772776+0000","last_unstale":"2024-04-14T08:37:03.144650+0000","last_undegraded":"2024-04-14T08:37:03.144650+0000","last_fullsized":"2024-04-14T08:37:03.144650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:44:21.151127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209058+0000","last_change":"2024-04-14T08:37:00.766360+0000","last_active":"2024-04-14T08:37:03.209058+0000","last_peered":"2024-04-14T08:37:03.209058+0000","last_clean":"2024-04-14T08:37:03.209058+0000","last_became_active":"2024-04-14T08:37:00.766057+0000","last_became_peered":"2024-04-14T08:37:00.766057+0000","last_unstale":"2024-04-14T08:37:03.209058+0000","last_undegraded":"2024-04-14T08:37:03.209058+0000","last_fullsized":"2024-04-14T08:37:03.209058+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:49:34.555691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208839+0000","last_change":"2024-04-14T08:37:01.763582+0000","last_active":"2024-04-14T08:37:03.208839+0000","last_peered":"2024-04-14T08:37:03.208839+0000","last_clean":"2024-04-14T08:37:03.208839+0000","last_became_active":"2024-04-14T08:37:01.763312+0000","last_became_peered":"2024-04-14T08:37:01.763312+0000","last_unstale":"2024-04-14T08:37:03.208839+0000","last_undegraded":"2024-04-14T08:37:03.208839+0000","last_fullsized":"2024-04-14T08:37:03.208839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:16:37.842242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.144522+0000","last_change":"2024-04-14T08:37:00.766273+0000","last_active":"2024-04-14T08:37:03.144522+0000","last_peered":"2024-04-14T08:37:03.144522+0000","last_clean":"2024-04-14T08:37:03.144522+0000","last_became_active":"2024-04-14T08:37:00.766000+0000","last_became_peered":"2024-04-14T08:37:00.766000+0000","last_unstale":"2024-04-14T08:37:03.144522+0000","last_undegraded":"2024-04-14T08:37:03.144522+0000","last_fullsized":"2024-04-14T08:37:03.144522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:27:44.843714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209131+0000","last_change":"2024-04-14T08:37:01.757979+0000","last_active":"2024-04-14T08:37:03.209131+0000","last_peered":"2024-04-14T08:37:03.209131+0000","last_clean":"2024-04-14T08:37:03.209131+0000","last_became_active":"2024-04-14T08:37:01.757697+0000","last_became_peered":"2024-04-14T08:37:01.757697+0000","last_unstale":"2024-04-14T08:37:03.209131+0000","last_undegraded":"2024-04-14T08:37:03.209131+0000","last_fullsized":"2024-04-14T08:37:03.209131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:12:05.782198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.208328+0000","last_change":"2024-04-14T08:37:00.760752+0000","last_active":"2024-04-14T08:37:03.208328+0000","last_peered":"2024-04-14T08:37:03.208328+0000","last_clean":"2024-04-14T08:37:03.208328+0000","last_became_active":"2024-04-14T08:37:00.760459+0000","last_became_peered":"2024-04-14T08:37:00.760459+0000","last_unstale":"2024-04-14T08:37:03.208328+0000","last_undegraded":"2024-04-14T08:37:03.208328+0000","last_fullsized":"2024-04-14T08:37:03.208328+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:07:13.636523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768294+0000","last_change":"2024-04-14T08:37:01.765646+0000","last_active":"2024-04-14T08:37:02.768294+0000","last_peered":"2024-04-14T08:37:02.768294+0000","last_clean":"2024-04-14T08:37:02.768294+0000","last_became_active":"2024-04-14T08:37:01.765325+0000","last_became_peered":"2024-04-14T08:37:01.765325+0000","last_unstale":"2024-04-14T08:37:02.768294+0000","last_undegraded":"2024-04-14T08:37:02.768294+0000","last_fullsized":"2024-04-14T08:37:02.768294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:37:00.742875+0000","last_clean_scrub_stamp":"2024-04-14T08:37:00.742875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:34.673210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:03.209382+0000","last_change":"2024-04-14T08:37:00.768182+0000","last_active":"2024-04-14T08:37:03.209382+0000","last_peered":"2024-04-14T08:37:03.209382+0000","last_clean":"2024-04-14T08:37:03.209382+0000","last_became_active":"2024-04-14T08:37:00.767282+0000","last_became_peered":"2024-04-14T08:37:00.767282+0000","last_unstale":"2024-04-14T08:37:03.209382+0000","last_undegraded":"2024-04-14T08:37:03.209382+0000","last_fullsized":"2024-04-14T08:37:03.209382+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:36:59.734147+0000","last_clean_scrub_stamp":"2024-04-14T08:36:59.734147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:24.794262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":278,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-14T08:37:02.768951+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:37:02.768951+0000","last_peered":"2024-04-14T08:37:02.768951+0000","last_clean":"2024-04-14T08:37:02.768951+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:37:02.768951+0000","last_undegraded":"2024-04-14T08:37:02.768951+0000","last_fullsized":"2024-04-14T08:37:02.768951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1503232,"data_stored":1001204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":412426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":909312,"data_stored":410848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":925696,"data_stored":412530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":411396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":411556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":913408,"data_stored":410926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1499136,"data_stored":1001182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:37:09.579 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-14T08:37:09.579 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-14T08:37:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.722+0000 7fb5ea6de640 1 Processor -- start 2024-04-14T08:37:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.722+0000 7fb5ea6de640 1 -- start start 2024-04-14T08:37:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.723+0000 7fb5ea6de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e416e570 0x7fb5e40b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.723+0000 7fb5ea6de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb5e40cd7f0 0x7fb5e40b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.723+0000 7fb5ea6de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e40b4030 con 0x7fb5e408b300 2024-04-14T08:37:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.723+0000 7fb5ea6de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e4072cf0 con 0x7fb5e40cd7f0 2024-04-14T08:37:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.723+0000 7fb5ea6de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e416e950 con 0x7fb5e416e570 2024-04-14T08:37:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.723+0000 7fb5e8c7b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb5e408b300 0x7fb5e416eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59184/0 (socket says 172.21.15.172:59184) 2024-04-14T08:37:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.723+0000 7fb5e8c7b640 1 -- 172.21.15.172:0/3683057772 learned_addr learned my addr 172.21.15.172:0/3683057772 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.723+0000 7fb5e3fff640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e416e570 0x7fb5e40b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.723+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3974996872 0 0) 0x7fb5e40b4030 con 0x7fb5e408b300 2024-04-14T08:37:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.723+0000 7fb5e37fe640 1 --2- 172.21.15.172:0/3683057772 >> v2:172.21.15.177:3300/0 conn(0x7fb5e40cd7f0 0x7fb5e40b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5cc003520 con 0x7fb5e408b300 2024-04-14T08:37:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 248225476 0 0) 0x7fb5cc003520 con 0x7fb5e408b300 2024-04-14T08:37:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e416e570 msgr2=0x7fb5e40b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e416e570 0x7fb5e40b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 >> v2:172.21.15.177:3300/0 conn(0x7fb5e40cd7f0 msgr2=0x7fb5e40b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 --2- 172.21.15.172:0/3683057772 >> v2:172.21.15.177:3300/0 conn(0x7fb5e40cd7f0 0x7fb5e40b5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e40b4310 con 0x7fb5e408b300 2024-04-14T08:37:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fb5dc0019f0 con 0x7fb5e408b300 2024-04-14T08:37:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 >> v1:172.21.15.172:6789/0 conn(0x7fb5e408b300 legacy=0x7fb5e416eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e416e570 0x7fb5cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5cc003f30 0x7fb5cc0089a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5cc008ee0 0x7fb5cc00b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4072cf0 con 0x7fb5cc008ee0 2024-04-14T08:37:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc00ba50 con 0x7fb5cc003f30 2024-04-14T08:37:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb5cc00bc50 con 0x7fb5e416e570 2024-04-14T08:37:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e37fe640 1 --2- 172.21.15.172:0/3683057772 >> v2:172.21.15.177:3300/0 conn(0x7fb5e40cd7f0 0x7fb5e40b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.742+0000 7fb5e8c7b640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5cc008ee0 0x7fb5cc00b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.743+0000 7fb5e3fff640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e416e570 0x7fb5cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.743+0000 7fb5e37fe640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5cc003f30 0x7fb5cc0089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.743+0000 7fb5e8c7b640 1 -- 172.21.15.172:0/3683057772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e416e570 msgr2=0x7fb5cc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.743+0000 7fb5e8c7b640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e416e570 0x7fb5cc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.743+0000 7fb5e8c7b640 1 -- 172.21.15.172:0/3683057772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5cc003f30 msgr2=0x7fb5cc0089a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.743+0000 7fb5e8c7b640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5cc003f30 0x7fb5cc0089a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.743+0000 7fb5e8c7b640 1 -- 172.21.15.172:0/3683057772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc003520 con 0x7fb5cc008ee0 2024-04-14T08:37:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.743+0000 7fb5e3fff640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e416e570 0x7fb5cc0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.743+0000 7fb5e8c7b640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5cc008ee0 0x7fb5cc00b2d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00a8b0 tx=0x7fb5cc004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ffc9575dfbcb57 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:09.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.744+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00a4d0 con 0x7fb5cc008ee0 2024-04-14T08:37:09.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.744+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5dc0072d0 con 0x7fb5cc008ee0 2024-04-14T08:37:09.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.744+0000 7fb5e2ffd640 1 -- 172.21.15.172:0/3683057772 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0076c0 con 0x7fb5cc008ee0 2024-04-14T08:37:09.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.744+0000 7fb5ea6de640 1 -- 172.21.15.172:0/3683057772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5cc008ee0 msgr2=0x7fb5cc00b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:09.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.744+0000 7fb5ea6de640 1 --2- 172.21.15.172:0/3683057772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5cc008ee0 0x7fb5cc00b2d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00a8b0 tx=0x7fb5cc004820 comp rx=0 tx=0).stop 2024-04-14T08:37:09.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.744+0000 7fb5e8c7b640 1 -- 172.21.15.172:0/3683057772 reap_dead start 2024-04-14T08:37:09.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.744+0000 7fb5ea6de640 1 -- 172.21.15.172:0/3683057772 shutdown_connections 2024-04-14T08:37:09.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.744+0000 7fb5ea6de640 1 -- 172.21.15.172:0/3683057772 >> 172.21.15.172:0/3683057772 conn(0x7fb5e40b1d40 msgr2=0x7fb5e40b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:09.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.744+0000 7fb5ea6de640 1 -- 172.21.15.172:0/3683057772 shutdown_connections 2024-04-14T08:37:09.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.744+0000 7fb5ea6de640 1 -- 172.21.15.172:0/3683057772 wait complete. 2024-04-14T08:37:09.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.745+0000 7fb5ea6de640 1 Processor -- start 2024-04-14T08:37:09.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.746+0000 7fb5ea6de640 1 -- start start 2024-04-14T08:37:09.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.746+0000 7fb5ea6de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5cc008ee0 0x7fb5e40c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.746+0000 7fb5ea6de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5e40c67a0 0x7fb5e415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.746+0000 7fb5ea6de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e4163610 0x7fb5e40cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.746+0000 7fb5ea6de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e40b5b40 con 0x7fb5e40c67a0 2024-04-14T08:37:09.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.746+0000 7fb5ea6de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e40717c0 con 0x7fb5cc008ee0 2024-04-14T08:37:09.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.746+0000 7fb5ea6de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e40b3a50 con 0x7fb5e4163610 2024-04-14T08:37:09.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.746+0000 7fb5e37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5e40c67a0 0x7fb5e415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.746+0000 7fb5e37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5e40c67a0 0x7fb5e415a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46630/0 (socket says 172.21.15.172:46630) 2024-04-14T08:37:09.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.746+0000 7fb5e37fe640 1 -- 172.21.15.172:0/2200423774 learned_addr learned my addr 172.21.15.172:0/2200423774 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:09.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5e3fff640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5cc008ee0 0x7fb5e40c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5e8c7b640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e4163610 0x7fb5e40cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5e37fe640 1 -- 172.21.15.172:0/2200423774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e4163610 msgr2=0x7fb5e40cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:09.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5e37fe640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e4163610 0x7fb5e40cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5e37fe640 1 -- 172.21.15.172:0/2200423774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5cc008ee0 msgr2=0x7fb5e40c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:09.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5e37fe640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5cc008ee0 0x7fb5e40c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:09.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5e37fe640 1 -- 172.21.15.172:0/2200423774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e415abb0 con 0x7fb5e40c67a0 2024-04-14T08:37:09.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5e37fe640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5e40c67a0 0x7fb5e415a670 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400fd10 tx=0x7fb5d4009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:09.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5e0ff9640 1 -- 172.21.15.172:0/2200423774 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4017070 con 0x7fb5e40c67a0 2024-04-14T08:37:09.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5e0ff9640 1 -- 172.21.15.172:0/2200423774 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d400a040 con 0x7fb5e40c67a0 2024-04-14T08:37:09.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5ea6de640 1 -- 172.21.15.172:0/2200423774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e40c5700 con 0x7fb5e40c67a0 2024-04-14T08:37:09.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5e0ff9640 1 -- 172.21.15.172:0/2200423774 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4004460 con 0x7fb5e40c67a0 2024-04-14T08:37:09.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.747+0000 7fb5ea6de640 1 -- 172.21.15.172:0/2200423774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e40c5cc0 con 0x7fb5e40c67a0 2024-04-14T08:37:09.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.748+0000 7fb5ea6de640 1 -- 172.21.15.172:0/2200423774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e4068590 con 0x7fb5e40c67a0 2024-04-14T08:37:09.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.749+0000 7fb5e0ff9640 1 -- 172.21.15.172:0/2200423774 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5d4003610 con 0x7fb5e40c67a0 2024-04-14T08:37:09.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.749+0000 7fb5e0ff9640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb5d006c8f0 0x7fb5d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:09.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.749+0000 7fb5e0ff9640 1 -- 172.21.15.172:0/2200423774 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fb5d408c940 con 0x7fb5e40c67a0 2024-04-14T08:37:09.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.749+0000 7fb5e3fff640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb5d006c8f0 0x7fb5d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:09.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.750+0000 7fb5e3fff640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb5d006c8f0 0x7fb5d006edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800db70 tx=0x7fb5d800d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:09.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:09.754+0000 7fb5e0ff9640 1 -- 172.21.15.172:0/2200423774 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb5d405ab30 con 0x7fb5e40c67a0 2024-04-14T08:37:10.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.010+0000 7fb5ea6de640 1 -- 172.21.15.172:0/2200423774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb5e4072e70 con 0x7fb5e40c67a0 2024-04-14T08:37:10.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.012+0000 7fb5e0ff9640 1 -- 172.21.15.172:0/2200423774 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fb5d405a430 con 0x7fb5e40c67a0 2024-04-14T08:37:10.013 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2024-04-14T08:37:10.013 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA== 2024-04-14T08:37:10.014 INFO:teuthology.orchestra.run.smithi172.stdout: caps mds = "allow" 2024-04-14T08:37:10.014 INFO:teuthology.orchestra.run.smithi172.stdout: caps mon = "allow r" 2024-04-14T08:37:10.014 INFO:teuthology.orchestra.run.smithi172.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-14T08:37:10.014 INFO:teuthology.orchestra.run.smithi172.stderr:updated caps for client.0 2024-04-14T08:37:10.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.014+0000 7fb5ea6de640 1 -- 172.21.15.172:0/2200423774 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb5d006c8f0 msgr2=0x7fb5d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.014+0000 7fb5ea6de640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb5d006c8f0 0x7fb5d006edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800db70 tx=0x7fb5d800d040 comp rx=0 tx=0).stop 2024-04-14T08:37:10.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.014+0000 7fb5ea6de640 1 -- 172.21.15.172:0/2200423774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5e40c67a0 msgr2=0x7fb5e415a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.014+0000 7fb5ea6de640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5e40c67a0 0x7fb5e415a670 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400fd10 tx=0x7fb5d4009670 comp rx=0 tx=0).stop 2024-04-14T08:37:10.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.014+0000 7fb5ea6de640 1 -- 172.21.15.172:0/2200423774 shutdown_connections 2024-04-14T08:37:10.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.014+0000 7fb5ea6de640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb5d006c8f0 0x7fb5d006edb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.014+0000 7fb5ea6de640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e4163610 0x7fb5e40cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.014+0000 7fb5ea6de640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5e40c67a0 0x7fb5e415a670 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.014+0000 7fb5ea6de640 1 --2- 172.21.15.172:0/2200423774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5cc008ee0 0x7fb5e40c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.014+0000 7fb5ea6de640 1 -- 172.21.15.172:0/2200423774 >> 172.21.15.172:0/2200423774 conn(0x7fb5e40b1d40 msgr2=0x7fb5e416a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:10.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.014+0000 7fb5ea6de640 1 -- 172.21.15.172:0/2200423774 shutdown_connections 2024-04-14T08:37:10.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.014+0000 7fb5ea6de640 1 -- 172.21.15.172:0/2200423774 wait complete. 2024-04-14T08:37:10.027 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:37:10.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.127+0000 7f2dbf7dc640 1 Processor -- start 2024-04-14T08:37:10.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.127+0000 7f2dbf7dc640 1 -- start start 2024-04-14T08:37:10.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.128+0000 7f2dbf7dc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2db80d2830 0x7f2db80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.128+0000 7f2dbf7dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db80cedd0 0x7f2db80b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.128+0000 7f2dbf7dc640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2db80d3150 con 0x7f2db815e890 2024-04-14T08:37:10.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.128+0000 7f2dbf7dc640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2db80cff30 con 0x7f2db80d2830 2024-04-14T08:37:10.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.128+0000 7f2dbf7dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2db80d2350 con 0x7f2db80cedd0 2024-04-14T08:37:10.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.128+0000 7f2dbd578640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2db80d2830 0x7f2db80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.128+0000 7f2dbdd79640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db80cedd0 0x7f2db80b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.128+0000 7f2dbdd79640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db80cedd0 0x7f2db80b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:40142/0 (socket says 172.21.15.172:40142) 2024-04-14T08:37:10.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.128+0000 7f2dbdd79640 1 -- 172.21.15.172:0/172314 learned_addr learned my addr 172.21.15.172:0/172314 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:10.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.128+0000 7f2dbd578640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2db80d2830 0x7f2db80d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:37652/0 (socket says 172.21.15.172:37652) 2024-04-14T08:37:10.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.129+0000 7f2dbdd79640 1 -- 172.21.15.172:0/172314 >> v2:172.21.15.177:3300/0 conn(0x7f2db80d2830 msgr2=0x7f2db80d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:10.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.129+0000 7f2dbdd79640 1 --2- 172.21.15.172:0/172314 >> v2:172.21.15.177:3300/0 conn(0x7f2db80d2830 0x7f2db80d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.129+0000 7f2dbdd79640 1 -- 172.21.15.172:0/172314 >> v1:172.21.15.177:6789/0 conn(0x7f2db815e890 legacy=0x7f2db80d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.129+0000 7f2dbdd79640 1 -- 172.21.15.172:0/172314 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db80cf320 con 0x7f2db80cedd0 2024-04-14T08:37:10.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.129+0000 7f2dbd578640 1 --2- 172.21.15.172:0/172314 >> v2:172.21.15.177:3300/0 conn(0x7f2db80d2830 0x7f2db80d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:37:10.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.130+0000 7f2dbdd79640 1 --2- 172.21.15.172:0/172314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db80cedd0 0x7f2db80b3710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2db0009e50 tx=0x7f2db002bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a05dd5df5f93a125 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:10.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.130+0000 7f2daffff640 1 -- 172.21.15.172:0/172314 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db002cb90 con 0x7f2db80cedd0 2024-04-14T08:37:10.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.130+0000 7f2daffff640 1 -- 172.21.15.172:0/172314 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db002cd30 con 0x7f2db80cedd0 2024-04-14T08:37:10.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.130+0000 7f2daffff640 1 -- 172.21.15.172:0/172314 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0034610 con 0x7f2db80cedd0 2024-04-14T08:37:10.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.130+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/172314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db80cedd0 msgr2=0x7f2db80b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.130+0000 7f2dbf7dc640 1 --2- 172.21.15.172:0/172314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db80cedd0 0x7f2db80b3710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2db0009e50 tx=0x7f2db002bff0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.130+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/172314 shutdown_connections 2024-04-14T08:37:10.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.130+0000 7f2dbf7dc640 1 --2- 172.21.15.172:0/172314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db80cedd0 0x7f2db80b3710 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.130+0000 7f2dbf7dc640 1 --2- 172.21.15.172:0/172314 >> v2:172.21.15.177:3300/0 conn(0x7f2db80d2830 0x7f2db80d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.130+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/172314 >> 172.21.15.172:0/172314 conn(0x7f2db809d590 msgr2=0x7f2db80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:10.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.130+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/172314 shutdown_connections 2024-04-14T08:37:10.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.131+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/172314 wait complete. 2024-04-14T08:37:10.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.132+0000 7f2dbf7dc640 1 Processor -- start 2024-04-14T08:37:10.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.132+0000 7f2dbf7dc640 1 -- start start 2024-04-14T08:37:10.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.132+0000 7f2dbf7dc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2db80cedd0 0x7f2db8159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.132+0000 7f2dbf7dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db80d2830 0x7f2db80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.132+0000 7f2dbf7dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db815e890 0x7f2db80b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.132+0000 7f2dbf7dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2db80cfbc0 con 0x7f2db815e890 2024-04-14T08:37:10.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.132+0000 7f2dbf7dc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2db80cf8e0 con 0x7f2db80cedd0 2024-04-14T08:37:10.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.132+0000 7f2dbf7dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2db80b3c50 con 0x7f2db80d2830 2024-04-14T08:37:10.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.132+0000 7f2dbdd79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db815e890 0x7f2db80b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dbdd79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db815e890 0x7f2db80b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46638/0 (socket says 172.21.15.172:46638) 2024-04-14T08:37:10.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dbdd79640 1 -- 172.21.15.172:0/3414308433 learned_addr learned my addr 172.21.15.172:0/3414308433 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:10.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dbcd77640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db80d2830 0x7f2db80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dbd578640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2db80cedd0 0x7f2db8159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dbdd79640 1 -- 172.21.15.172:0/3414308433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db80d2830 msgr2=0x7f2db80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dbdd79640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db80d2830 0x7f2db80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dbdd79640 1 -- 172.21.15.172:0/3414308433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2db80cedd0 msgr2=0x7f2db8159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dbdd79640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2db80cedd0 0x7f2db8159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dbdd79640 1 -- 172.21.15.172:0/3414308433 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db815ed40 con 0x7f2db815e890 2024-04-14T08:37:10.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dbdd79640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db815e890 0x7f2db80b5590 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2db0036040 tx=0x7f2db0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:10.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dadffb640 1 -- 172.21.15.172:0/3414308433 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0043020 con 0x7f2db815e890 2024-04-14T08:37:10.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dadffb640 1 -- 172.21.15.172:0/3414308433 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db003e450 con 0x7f2db815e890 2024-04-14T08:37:10.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dadffb640 1 -- 172.21.15.172:0/3414308433 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0009510 con 0x7f2db815e890 2024-04-14T08:37:10.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/3414308433 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db815fa90 con 0x7f2db815e890 2024-04-14T08:37:10.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.133+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/3414308433 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db815f290 con 0x7f2db815e890 2024-04-14T08:37:10.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.134+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/3414308433 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db80750a0 con 0x7f2db815e890 2024-04-14T08:37:10.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.135+0000 7f2dadffb640 1 -- 172.21.15.172:0/3414308433 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2db0046020 con 0x7f2db815e890 2024-04-14T08:37:10.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.136+0000 7f2dadffb640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d9c06c920 0x7f2d9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.136+0000 7f2dadffb640 1 -- 172.21.15.172:0/3414308433 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f2db00ae190 con 0x7f2db815e890 2024-04-14T08:37:10.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.136+0000 7f2dbd578640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d9c06c920 0x7f2d9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.136+0000 7f2dbd578640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d9c06c920 0x7f2d9c06ede0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2db80b30b0 tx=0x7f2db4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:10.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.140+0000 7f2dadffb640 1 -- 172.21.15.172:0/3414308433 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2db80750a0 con 0x7f2db815e890 2024-04-14T08:37:10.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.359+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/3414308433 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2db80d0b70 con 0x7f2db815e890 2024-04-14T08:37:10.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.360+0000 7f2dadffb640 1 -- 172.21.15.172:0/3414308433 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5192 (secure 0 0 0) 0x7f2db004d030 con 0x7f2db815e890 2024-04-14T08:37:10.361 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:10.361 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:37:01.115031+0000","modified":"2024-04-14T08:37:03.119917+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6753":{"gid":6753,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6837/2134832221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2134832221},{"type":"v1","addr":"172.21.15.172:6837","nonce":2134832221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6753,"qdb_cluster":[6753]},"id":5}]} 2024-04-14T08:37:10.362 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 39 2024-04-14T08:37:10.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.362+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/3414308433 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d9c06c920 msgr2=0x7f2d9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.362+0000 7f2dbf7dc640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d9c06c920 0x7f2d9c06ede0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2db80b30b0 tx=0x7f2db4009190 comp rx=0 tx=0).stop 2024-04-14T08:37:10.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.362+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/3414308433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db815e890 msgr2=0x7f2db80b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.362+0000 7f2dbf7dc640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db815e890 0x7f2db80b5590 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2db0036040 tx=0x7f2db0004a40 comp rx=0 tx=0).stop 2024-04-14T08:37:10.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.363+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/3414308433 shutdown_connections 2024-04-14T08:37:10.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.363+0000 7f2dbf7dc640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d9c06c920 0x7f2d9c06ede0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.363+0000 7f2dbf7dc640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2db815e890 0x7f2db80b5590 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.363+0000 7f2dbf7dc640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2db80d2830 0x7f2db80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.363+0000 7f2dbf7dc640 1 --2- 172.21.15.172:0/3414308433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2db80cedd0 0x7f2db8159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.363+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/3414308433 >> 172.21.15.172:0/3414308433 conn(0x7f2db809d590 msgr2=0x7f2db806c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:10.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.363+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/3414308433 shutdown_connections 2024-04-14T08:37:10.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.363+0000 7f2dbf7dc640 1 -- 172.21.15.172:0/3414308433 wait complete. 2024-04-14T08:37:10.778 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:37:01.115031+0000', 'modified': '2024-04-14T08:37:03.119917+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6753}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6753': {'gid': 6753, 'name': 'c', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6837/2134832221', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 2134832221}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 2134832221}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6753, 'qdb_cluster': [6753]} 2024-04-14T08:37:10.778 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:37:10.778 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-14T08:37:10.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.880+0000 7f48a495e640 1 Processor -- start 2024-04-14T08:37:10.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.880+0000 7f48a495e640 1 -- start start 2024-04-14T08:37:10.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f48a495e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f48a00d1a20 0x7f48a006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f48a495e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48a0071250 0x7f48a00cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f48a495e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48a00701b0 con 0x7f48a006d000 2024-04-14T08:37:10.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f48a495e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f48a0070dc0 con 0x7f48a00d1a20 2024-04-14T08:37:10.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f48a495e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f48a0070490 con 0x7f48a0071250 2024-04-14T08:37:10.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f489e59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f48a00d1a20 0x7f48a006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f489ed9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f48a006d000 0x7f48a015dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59192/0 (socket says 172.21.15.172:59192) 2024-04-14T08:37:10.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f489ed9d640 1 -- 172.21.15.172:0/2836086400 learned_addr learned my addr 172.21.15.172:0/2836086400 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:10.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f489dd9b640 1 --2- 172.21.15.172:0/2836086400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48a0071250 0x7f48a00cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f489e59c640 1 -- 172.21.15.172:0/2836086400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48a0071250 msgr2=0x7f48a00cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:10.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f489e59c640 1 --2- 172.21.15.172:0/2836086400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48a0071250 0x7f48a00cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f489e59c640 1 -- 172.21.15.172:0/2836086400 >> v1:172.21.15.172:6789/0 conn(0x7f48a006d000 legacy=0x7f48a015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f489e59c640 1 -- 172.21.15.172:0/2836086400 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a0070770 con 0x7f48a00d1a20 2024-04-14T08:37:10.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.881+0000 7f489d59a640 1 -- 172.21.15.172:0/2836086400 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 972081347 0 0) 0x7f48a00701b0 con 0x7f48a006d000 2024-04-14T08:37:10.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.882+0000 7f489e59c640 1 --2- 172.21.15.172:0/2836086400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a00d1a20 0x7f48a006fb80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f48900097a0 tx=0x7f489002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc86232ac2c84c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:10.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.882+0000 7f489d59a640 1 -- 172.21.15.172:0/2836086400 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48900315a0 con 0x7f48a00d1a20 2024-04-14T08:37:10.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.882+0000 7f489d59a640 1 -- 172.21.15.172:0/2836086400 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4890037070 con 0x7f48a00d1a20 2024-04-14T08:37:10.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.882+0000 7f489d59a640 1 -- 172.21.15.172:0/2836086400 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4890031960 con 0x7f48a00d1a20 2024-04-14T08:37:10.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.883+0000 7f48a495e640 1 -- 172.21.15.172:0/2836086400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a00d1a20 msgr2=0x7f48a006fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.883+0000 7f48a495e640 1 --2- 172.21.15.172:0/2836086400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a00d1a20 0x7f48a006fb80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f48900097a0 tx=0x7f489002b840 comp rx=0 tx=0).stop 2024-04-14T08:37:10.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.883+0000 7f48a495e640 1 -- 172.21.15.172:0/2836086400 shutdown_connections 2024-04-14T08:37:10.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.883+0000 7f48a495e640 1 --2- 172.21.15.172:0/2836086400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48a0071250 0x7f48a00cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.883+0000 7f48a495e640 1 --2- 172.21.15.172:0/2836086400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a00d1a20 0x7f48a006fb80 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.883+0000 7f48a495e640 1 -- 172.21.15.172:0/2836086400 >> 172.21.15.172:0/2836086400 conn(0x7f48a008c000 msgr2=0x7f48a00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:10.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.883+0000 7f48a495e640 1 -- 172.21.15.172:0/2836086400 shutdown_connections 2024-04-14T08:37:10.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.883+0000 7f48a495e640 1 -- 172.21.15.172:0/2836086400 wait complete. 2024-04-14T08:37:10.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.884+0000 7f48a495e640 1 Processor -- start 2024-04-14T08:37:10.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.884+0000 7f48a495e640 1 -- start start 2024-04-14T08:37:10.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f48a495e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48a006d000 0x7f48a01602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f48a495e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48a0071250 0x7f48a0156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f48a495e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a00d1a20 0x7f48a01e9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f48a495e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f48a00d1e00 con 0x7f48a00d1a20 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f48a495e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f48a00cfa70 con 0x7f48a0071250 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f48a495e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f48a00af150 con 0x7f48a006d000 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f489ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a00d1a20 0x7f48a01e9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f489ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a00d1a20 0x7f48a01e9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46660/0 (socket says 172.21.15.172:46660) 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f489ed9d640 1 -- 172.21.15.172:0/3197385531 learned_addr learned my addr 172.21.15.172:0/3197385531 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f489e59c640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48a006d000 0x7f48a01602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f489dd9b640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48a0071250 0x7f48a0156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f489ed9d640 1 -- 172.21.15.172:0/3197385531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48a006d000 msgr2=0x7f48a01602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f489ed9d640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48a006d000 0x7f48a01602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f489ed9d640 1 -- 172.21.15.172:0/3197385531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48a0071250 msgr2=0x7f48a0156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f489ed9d640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48a0071250 0x7f48a0156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f489ed9d640 1 -- 172.21.15.172:0/3197385531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a00aee20 con 0x7f48a00d1a20 2024-04-14T08:37:10.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f489dd9b640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48a0071250 0x7f48a0156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:10.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.885+0000 7f489ed9d640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a00d1a20 0x7f48a01e9f70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f489800c7a0 tx=0x7f489800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:10.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.886+0000 7f488effd640 1 -- 172.21.15.172:0/3197385531 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898018070 con 0x7f48a00d1a20 2024-04-14T08:37:10.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.886+0000 7f488effd640 1 -- 172.21.15.172:0/3197385531 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489800a640 con 0x7f48a00d1a20 2024-04-14T08:37:10.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.886+0000 7f488effd640 1 -- 172.21.15.172:0/3197385531 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489800aab0 con 0x7f48a00d1a20 2024-04-14T08:37:10.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.886+0000 7f48a495e640 1 -- 172.21.15.172:0/3197385531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a00b7210 con 0x7f48a00d1a20 2024-04-14T08:37:10.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.886+0000 7f48a495e640 1 -- 172.21.15.172:0/3197385531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a00b77d0 con 0x7f48a00d1a20 2024-04-14T08:37:10.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.888+0000 7f488effd640 1 -- 172.21.15.172:0/3197385531 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f489801b070 con 0x7f48a00d1a20 2024-04-14T08:37:10.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.888+0000 7f488effd640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f488006c8f0 0x7f488006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.888+0000 7f488effd640 1 -- 172.21.15.172:0/3197385531 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f489808b2d0 con 0x7f48a00d1a20 2024-04-14T08:37:10.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.888+0000 7f489e59c640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f488006c8f0 0x7f488006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.888+0000 7f48a7827740 1 Processor -- start 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.889+0000 7f48a7827740 1 -- start start 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.889+0000 7f489e59c640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f488006c8f0 0x7f488006edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4890036000 tx=0x7f4890008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.889+0000 7f48a7827740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231fb60 0x55ecd22f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.889+0000 7f48a7827740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231ff40 0x55ecd23228f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.889+0000 7f48a7827740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd22f3040 0x55ecd2322e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.889+0000 7f48a7827740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ecd225c030 con 0x55ecd22f3040 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.889+0000 7f48a7827740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ecd2204620 con 0x55ecd231ff40 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.889+0000 7f48a7827740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ecd2257fe0 con 0x55ecd231fb60 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f489ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd22f3040 0x55ecd2322e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f489ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd22f3040 0x55ecd2322e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46672/0 (socket says 172.21.15.172:46672) 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f489ed9d640 1 -- 172.21.15.172:0/3630433039 learned_addr learned my addr 172.21.15.172:0/3630433039 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f489e59c640 1 --2- 172.21.15.172:0/3630433039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231fb60 0x55ecd22f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f489dd9b640 1 --2- 172.21.15.172:0/3630433039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231ff40 0x55ecd23228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f489ed9d640 1 -- 172.21.15.172:0/3630433039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231fb60 msgr2=0x55ecd22f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f489ed9d640 1 --2- 172.21.15.172:0/3630433039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231fb60 0x55ecd22f2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f489ed9d640 1 -- 172.21.15.172:0/3630433039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231ff40 msgr2=0x55ecd23228f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f489ed9d640 1 --2- 172.21.15.172:0/3630433039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231ff40 0x55ecd23228f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f489ed9d640 1 -- 172.21.15.172:0/3630433039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecd1f702d0 con 0x55ecd22f3040 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f489ed9d640 1 --2- 172.21.15.172:0/3630433039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd22f3040 0x55ecd2322e30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4898089610 tx=0x7f4898059630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b872b87e0a281 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f487ffff640 1 -- 172.21.15.172:0/3630433039 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898061a50 con 0x55ecd22f3040 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f487ffff640 1 -- 172.21.15.172:0/3630433039 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4898057170 con 0x55ecd22f3040 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.890+0000 7f487ffff640 1 -- 172.21.15.172:0/3630433039 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489800ac50 con 0x55ecd22f3040 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.891+0000 7f48a7827740 1 -- 172.21.15.172:0/3630433039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd22f3040 msgr2=0x55ecd2322e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.891+0000 7f48a7827740 1 --2- 172.21.15.172:0/3630433039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd22f3040 0x55ecd2322e30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4898089610 tx=0x7f4898059630 comp rx=0 tx=0).stop 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.891+0000 7f48a7827740 1 -- 172.21.15.172:0/3630433039 shutdown_connections 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.891+0000 7f48a7827740 1 --2- 172.21.15.172:0/3630433039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd22f3040 0x55ecd2322e30 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.891+0000 7f48a7827740 1 --2- 172.21.15.172:0/3630433039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231ff40 0x55ecd23228f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.891+0000 7f48a7827740 1 --2- 172.21.15.172:0/3630433039 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231fb60 0x55ecd22f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.891+0000 7f48a7827740 1 -- 172.21.15.172:0/3630433039 >> 172.21.15.172:0/3630433039 conn(0x55ecd22f4d70 msgr2=0x55ecd233d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.891+0000 7f48a7827740 1 -- 172.21.15.172:0/3630433039 shutdown_connections 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.891+0000 7f48a7827740 1 -- 172.21.15.172:0/3630433039 wait complete. 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.891+0000 7f48a7827740 1 -- start start 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.892+0000 7f48a7827740 1 Processor -- start 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.892+0000 7f48a7827740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231fb60 0x55ecd233dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.892+0000 7f48a7827740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231ff40 0x55ecd2318f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.892+0000 7f48a7827740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231afe0 0x55ecd231b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.892+0000 7f48a7827740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ecd22f7410 con 0x55ecd231fb60 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.892+0000 7f48a7827740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ecd22179e0 con 0x55ecd231afe0 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.892+0000 7f48a7827740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ecd22f9c50 con 0x55ecd231ff40 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f48a7827740 10 client.0 _ll_register_callbacks cb 0x55ecd2254080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f489e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231fb60 0x55ecd233dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f489e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231fb60 0x55ecd233dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46688/0 (socket says 172.21.15.172:46688) 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f489e59c640 1 -- 172.21.15.172:0/4024969910 learned_addr learned my addr 172.21.15.172:0/4024969910 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f489ed9d640 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231afe0 0x55ecd231b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f489dd9b640 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231ff40 0x55ecd2318f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f489e59c640 1 -- 172.21.15.172:0/4024969910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231ff40 msgr2=0x55ecd2318f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f489e59c640 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231ff40 0x55ecd2318f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f489e59c640 1 -- 172.21.15.172:0/4024969910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231afe0 msgr2=0x55ecd231b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f489e59c640 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231afe0 0x55ecd231b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f489e59c640 1 -- 172.21.15.172:0/4024969910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecd203a6b0 con 0x55ecd231fb60 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f489e59c640 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231fb60 0x55ecd233dcd0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f48900098c0 tx=0x7f489002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d961fba3fa3b68a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f487d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f487d7fa640 1 -- 172.21.15.172:0/4024969910 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4890031d00 con 0x55ecd231fb60 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f487d7fa640 1 -- 172.21.15.172:0/4024969910 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f4890054c70 con 0x55ecd231fb60 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f48a7827740 1 -- 172.21.15.172:0/4024969910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4898057170 con 0x55ecd231fb60 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f487d7fa640 1 -- 172.21.15.172:0/4024969910 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4890054e70 con 0x55ecd231fb60 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f48a7827740 10 client.6927 fetch_fsmap learned FSMap version 39 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.893+0000 7f48a7827740 1 -- 172.21.15.172:0/4024969910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55ecd231b9a0 con 0x55ecd231fb60 2024-04-14T08:37:10.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.894+0000 7f487d7fa640 1 -- 172.21.15.172:0/4024969910 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4898057170 con 0x55ecd231fb60 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.894+0000 7f487d7fa640 1 -- 172.21.15.172:0/4024969910 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 3494+0+0 (secure 0 0 0) 0x7f489005b050 con 0x55ecd231fb60 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.894+0000 7f48a7827740 10 client.6927 fetch_fsmap finished waiting for FSMap version 39 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.894+0000 7f48a7827740 10 client.6927 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.894+0000 7f48a7827740 20 client.6927 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.894+0000 7f48a7827740 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.172:6835/2134832221,v1:172.21.15.172:6837/2134832221] conn(0x55ecd2351fe0 0x55ecd235ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.894+0000 7f48a7827740 4 client.6927 mds_command: new command op to 6753 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.894+0000 7f48a7827740 1 -- 172.21.15.172:0/4024969910 --> [v2:172.21.15.172:6835/2134832221,v1:172.21.15.172:6837/2134832221] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecd2203cc0 con 0x55ecd2351fe0 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.894+0000 7f489dd9b640 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.172:6835/2134832221,v1:172.21.15.172:6837/2134832221] conn(0x55ecd2351fe0 0x55ecd235ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f487d7fa640 10 client.6927 ms_handle_connect on v2:172.21.15.172:6835/2134832221 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f489dd9b640 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.172:6835/2134832221,v1:172.21.15.172:6837/2134832221] conn(0x55ecd2351fe0 0x55ecd235ccd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f0b86487f205d0 server_cookie=eb6ce3e4461b4f87 in_seq=0 out_seq=0 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f487d7fa640 1 -- 172.21.15.172:0/4024969910 <== mds.0 v2:172.21.15.172:6835/2134832221 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecd2203cc0 con 0x55ecd2351fe0 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f487d7fa640 10 client.6927 handle_command_reply: tid=0 2024-04-14T08:37:10.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f48a7827740 1 client.6927 shutdown 2024-04-14T08:37:10.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f48a7827740 1 -- 172.21.15.172:0/4024969910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231fb60 msgr2=0x55ecd233dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f48a7827740 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231fb60 0x55ecd233dcd0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f48900098c0 tx=0x7f489002e000 comp rx=0 tx=0).stop 2024-04-14T08:37:10.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f48a7827740 1 -- 172.21.15.172:0/4024969910 shutdown_connections 2024-04-14T08:37:10.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f48a7827740 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.172:6835/2134832221,v1:172.21.15.172:6837/2134832221] conn(0x55ecd2351fe0 0x55ecd235ccd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f48a7827740 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231afe0 0x55ecd231b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f48a7827740 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231ff40 0x55ecd2318f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f487d7fa640 0 client.6927 ms_handle_reset on v2:172.21.15.172:6835/2134832221 2024-04-14T08:37:10.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f48a7827740 1 --2- 172.21.15.172:0/4024969910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231fb60 0x55ecd233dcd0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f48a7827740 1 -- 172.21.15.172:0/4024969910 >> 172.21.15.172:0/4024969910 conn(0x55ecd22f3040 msgr2=0x55ecd2340e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:10.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f48a7827740 1 -- 172.21.15.172:0/4024969910 shutdown_connections 2024-04-14T08:37:10.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.895+0000 7f48a7827740 1 -- 172.21.15.172:0/4024969910 wait complete. 2024-04-14T08:37:10.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.896+0000 7f48a7827740 20 client.6927 trim_cache size 0 max 16384 2024-04-14T08:37:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.914+0000 7f48a7827740 1 Processor -- start 2024-04-14T08:37:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.914+0000 7f48a7827740 1 -- start start 2024-04-14T08:37:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.914+0000 7f48a7827740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231afe0 0x55ecd2340490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.914+0000 7f48a7827740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231fb60 0x55ecd233dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.914+0000 7f48a7827740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231ff40 0x55ecd233e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.914+0000 7f48a7827740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f48a01fff30 con 0x55ecd231ff40 2024-04-14T08:37:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.914+0000 7f48a7827740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f48a00afed0 con 0x55ecd231fb60 2024-04-14T08:37:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.914+0000 7f48a7827740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f48a00b0520 con 0x55ecd231afe0 2024-04-14T08:37:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.914+0000 7f489ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231ff40 0x55ecd233e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f489ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231ff40 0x55ecd233e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46690/0 (socket says 172.21.15.172:46690) 2024-04-14T08:37:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f489ed9d640 1 -- 172.21.15.172:0/4162018525 learned_addr learned my addr 172.21.15.172:0/4162018525 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f489e59c640 1 --2- 172.21.15.172:0/4162018525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231afe0 0x55ecd2340490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f489dd9b640 1 --2- 172.21.15.172:0/4162018525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231fb60 0x55ecd233dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f489ed9d640 1 -- 172.21.15.172:0/4162018525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231afe0 msgr2=0x55ecd2340490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f489ed9d640 1 --2- 172.21.15.172:0/4162018525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231afe0 0x55ecd2340490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f489ed9d640 1 -- 172.21.15.172:0/4162018525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231fb60 msgr2=0x55ecd233dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f489ed9d640 1 --2- 172.21.15.172:0/4162018525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231fb60 0x55ecd233dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f489ed9d640 1 -- 172.21.15.172:0/4162018525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecd2326b10 con 0x55ecd231ff40 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f489ed9d640 1 --2- 172.21.15.172:0/4162018525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231ff40 0x55ecd233e020 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4898059720 tx=0x7f4898009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9395a965bef48c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f487f7fe640 1 -- 172.21.15.172:0/4162018525 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898071070 con 0x55ecd231ff40 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f487f7fe640 1 -- 172.21.15.172:0/4162018525 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489805d220 con 0x55ecd231ff40 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.915+0000 7f487f7fe640 1 -- 172.21.15.172:0/4162018525 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898057170 con 0x55ecd231ff40 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.916+0000 7f48a7827740 1 -- 172.21.15.172:0/4162018525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231ff40 msgr2=0x55ecd233e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.916+0000 7f48a7827740 1 --2- 172.21.15.172:0/4162018525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231ff40 0x55ecd233e020 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4898059720 tx=0x7f4898009070 comp rx=0 tx=0).stop 2024-04-14T08:37:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.916+0000 7f48a7827740 1 -- 172.21.15.172:0/4162018525 shutdown_connections 2024-04-14T08:37:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.916+0000 7f48a7827740 1 --2- 172.21.15.172:0/4162018525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231ff40 0x55ecd233e020 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.916+0000 7f48a7827740 1 --2- 172.21.15.172:0/4162018525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231fb60 0x55ecd233dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.916+0000 7f48a7827740 1 --2- 172.21.15.172:0/4162018525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231afe0 0x55ecd2340490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.916+0000 7f48a7827740 1 -- 172.21.15.172:0/4162018525 >> 172.21.15.172:0/4162018525 conn(0x55ecd22f3040 msgr2=0x55ecd2357730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.916+0000 7f48a7827740 1 -- 172.21.15.172:0/4162018525 shutdown_connections 2024-04-14T08:37:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.916+0000 7f48a7827740 1 -- 172.21.15.172:0/4162018525 wait complete. 2024-04-14T08:37:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.916+0000 7f48a7827740 1 -- start start 2024-04-14T08:37:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.916+0000 7f48a7827740 1 Processor -- start 2024-04-14T08:37:10.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.917+0000 7f48a7827740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231afe0 0x55ecd233ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.917+0000 7f48a7827740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231fb60 0x55ecd233ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.917+0000 7f48a7827740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231ff40 0x55ecd2361140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.917+0000 7f48a7827740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ecd2324ba0 con 0x55ecd231afe0 2024-04-14T08:37:10.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.917+0000 7f48a7827740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ecd2324ee0 con 0x55ecd231ff40 2024-04-14T08:37:10.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.917+0000 7f48a7827740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ecd23228e0 con 0x55ecd231fb60 2024-04-14T08:37:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.917+0000 7f48a7827740 10 client.0 _ll_register_callbacks cb 0x7f4890003aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.917+0000 7f489e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231afe0 0x55ecd233ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.917+0000 7f489e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231afe0 0x55ecd233ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46706/0 (socket says 172.21.15.172:46706) 2024-04-14T08:37:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.917+0000 7f489e59c640 1 -- 172.21.15.172:0/638889249 learned_addr learned my addr 172.21.15.172:0/638889249 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f489e59c640 1 -- 172.21.15.172:0/638889249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231fb60 msgr2=0x55ecd233ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f489e59c640 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231fb60 0x55ecd233ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f489ed9d640 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231ff40 0x55ecd2361140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f489e59c640 1 -- 172.21.15.172:0/638889249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231ff40 msgr2=0x55ecd2361140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f489e59c640 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231ff40 0x55ecd2361140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f489e59c640 1 -- 172.21.15.172:0/638889249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecd2362b00 con 0x55ecd231afe0 2024-04-14T08:37:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f489ed9d640 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231ff40 0x55ecd2361140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f489e59c640 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231afe0 0x55ecd233ea20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f489002b880 tx=0x7f4890004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc19701763ecc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f487d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f487d7fa640 1 -- 172.21.15.172:0/638889249 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4890005990 con 0x55ecd231afe0 2024-04-14T08:37:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f487d7fa640 1 -- 172.21.15.172:0/638889249 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f489002f040 con 0x55ecd231afe0 2024-04-14T08:37:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f48a7827740 1 -- 172.21.15.172:0/638889249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f489805d220 con 0x55ecd231afe0 2024-04-14T08:37:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f487d7fa640 1 -- 172.21.15.172:0/638889249 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4890069050 con 0x55ecd231afe0 2024-04-14T08:37:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f48a7827740 10 client.6933 fetch_fsmap learned FSMap version 39 2024-04-14T08:37:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f48a7827740 1 -- 172.21.15.172:0/638889249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55ecd233f680 con 0x55ecd231afe0 2024-04-14T08:37:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f487d7fa640 1 -- 172.21.15.172:0/638889249 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489805d220 con 0x55ecd231afe0 2024-04-14T08:37:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f487d7fa640 1 -- 172.21.15.172:0/638889249 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 3494+0+0 (secure 0 0 0) 0x7f489006c020 con 0x55ecd231afe0 2024-04-14T08:37:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f48a7827740 10 client.6933 fetch_fsmap finished waiting for FSMap version 39 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f48a7827740 10 client.6933 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f48a7827740 20 client.6933 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f48a7827740 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.172:6835/2134832221,v1:172.21.15.172:6837/2134832221] conn(0x55ecd2325150 0x55ecd2325530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f48a7827740 4 client.6933 mds_command: new command op to 6753 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.918+0000 7f48a7827740 1 -- 172.21.15.172:0/638889249 --> [v2:172.21.15.172:6835/2134832221,v1:172.21.15.172:6837/2134832221] -- command(tid 0: {"prefix": "status"}) -- 0x55ecd2269570 con 0x55ecd2325150 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f489dd9b640 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.172:6835/2134832221,v1:172.21.15.172:6837/2134832221] conn(0x55ecd2325150 0x55ecd2325530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f487d7fa640 10 client.6933 ms_handle_connect on v2:172.21.15.172:6835/2134832221 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f489dd9b640 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.172:6835/2134832221,v1:172.21.15.172:6837/2134832221] conn(0x55ecd2325150 0x55ecd2325530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adcb45b741718b5b server_cookie=263c11744a4c120c in_seq=0 out_seq=0 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f487d7fa640 1 -- 172.21.15.172:0/638889249 <== mds.0 v2:172.21.15.172:6835/2134832221 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55ecd2269570 con 0x55ecd2325150 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f487d7fa640 10 client.6933 handle_command_reply: tid=0 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f48a7827740 1 client.6933 shutdown 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f48a7827740 1 -- 172.21.15.172:0/638889249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231afe0 msgr2=0x55ecd233ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f48a7827740 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231afe0 0x55ecd233ea20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f489002b880 tx=0x7f4890004730 comp rx=0 tx=0).stop 2024-04-14T08:37:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f48a7827740 1 -- 172.21.15.172:0/638889249 shutdown_connections 2024-04-14T08:37:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f48a7827740 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.172:6835/2134832221,v1:172.21.15.172:6837/2134832221] conn(0x55ecd2325150 0x55ecd2325530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f48a7827740 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ecd231ff40 0x55ecd2361140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f48a7827740 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ecd231fb60 0x55ecd233ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f487d7fa640 0 client.6933 ms_handle_reset on v2:172.21.15.172:6835/2134832221 2024-04-14T08:37:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f48a7827740 1 --2- 172.21.15.172:0/638889249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ecd231afe0 0x55ecd233ea20 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f48a7827740 1 -- 172.21.15.172:0/638889249 >> 172.21.15.172:0/638889249 conn(0x55ecd22f3040 msgr2=0x55ecd2326250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f48a7827740 1 -- 172.21.15.172:0/638889249 shutdown_connections 2024-04-14T08:37:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f48a7827740 1 -- 172.21.15.172:0/638889249 wait complete. 2024-04-14T08:37:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.919+0000 7f48a7827740 20 client.6933 trim_cache size 0 max 16384 2024-04-14T08:37:10.929 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:37:10.929 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:37:10.929 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:37:10.929 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 6753, 2024-04-14T08:37:10.929 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:37:10.929 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:37:10.929 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:37:10.929 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 9.1642335100000007, 2024-04-14T08:37:10.929 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 39, 2024-04-14T08:37:10.929 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 74, 2024-04-14T08:37:10.929 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 74, 2024-04-14T08:37:10.930 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 14.197735216 2024-04-14T08:37:10.930 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:37:10.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.920+0000 7f48a495e640 1 -- 172.21.15.172:0/3197385531 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f488006c8f0 msgr2=0x7f488006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.920+0000 7f48a495e640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f488006c8f0 0x7f488006edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4890036000 tx=0x7f4890008030 comp rx=0 tx=0).stop 2024-04-14T08:37:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.920+0000 7f48a495e640 1 -- 172.21.15.172:0/3197385531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a00d1a20 msgr2=0x7f48a01e9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.920+0000 7f48a495e640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a00d1a20 0x7f48a01e9f70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f489800c7a0 tx=0x7f489800cc60 comp rx=0 tx=0).stop 2024-04-14T08:37:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.920+0000 7f48a495e640 1 -- 172.21.15.172:0/3197385531 shutdown_connections 2024-04-14T08:37:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.920+0000 7f48a495e640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f488006c8f0 0x7f488006edb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.920+0000 7f48a495e640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a00d1a20 0x7f48a01e9f70 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.920+0000 7f48a495e640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48a0071250 0x7f48a0156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.920+0000 7f48a495e640 1 --2- 172.21.15.172:0/3197385531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48a006d000 0x7f48a01602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.920+0000 7f48a495e640 1 -- 172.21.15.172:0/3197385531 >> 172.21.15.172:0/3197385531 conn(0x7f48a008c000 msgr2=0x7f48a00cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.920+0000 7f48a495e640 1 -- 172.21.15.172:0/3197385531 shutdown_connections 2024-04-14T08:37:10.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:10.920+0000 7f48a495e640 1 -- 172.21.15.172:0/3197385531 wait complete. 2024-04-14T08:37:10.933 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:37:11.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.034+0000 7fc413836640 1 Processor -- start 2024-04-14T08:37:11.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.034+0000 7fc413836640 1 -- start start 2024-04-14T08:37:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc413836640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc40c0d2830 0x7fc40c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc413836640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc40c15e890 0x7fc40c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc413836640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc40c0d3150 con 0x7fc40c0cedd0 2024-04-14T08:37:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc413836640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc40c0cff30 con 0x7fc40c0d2830 2024-04-14T08:37:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc413836640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc40c0d2350 con 0x7fc40c15e890 2024-04-14T08:37:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc4115d2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc40c0d2830 0x7fc40c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc411dd3640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc40c0cedd0 0x7fc40c1621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59206/0 (socket says 172.21.15.172:59206) 2024-04-14T08:37:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc411dd3640 1 -- 172.21.15.172:0/1357175251 learned_addr learned my addr 172.21.15.172:0/1357175251 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:11.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc410dd1640 1 --2- 172.21.15.172:0/1357175251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc40c15e890 0x7fc40c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:11.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc4115d2640 1 -- 172.21.15.172:0/1357175251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc40c15e890 msgr2=0x7fc40c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:11.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc4115d2640 1 --2- 172.21.15.172:0/1357175251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc40c15e890 0x7fc40c0d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:11.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc4115d2640 1 -- 172.21.15.172:0/1357175251 >> v1:172.21.15.172:6789/0 conn(0x7fc40c0cedd0 legacy=0x7fc40c1621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:11.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc4115d2640 1 -- 172.21.15.172:0/1357175251 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c0cf320 con 0x7fc40c0d2830 2024-04-14T08:37:11.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.035+0000 7fc3fffff640 1 -- 172.21.15.172:0/1357175251 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3135665720 0 0) 0x7fc40c0d3150 con 0x7fc40c0cedd0 2024-04-14T08:37:11.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.036+0000 7fc4115d2640 1 --2- 172.21.15.172:0/1357175251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc40c0d2830 0x7fc40c0d2c10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc4080097a0 tx=0x7fc40802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace777826d1311c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:11.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.037+0000 7fc3fffff640 1 -- 172.21.15.172:0/1357175251 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408034070 con 0x7fc40c0d2830 2024-04-14T08:37:11.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.037+0000 7fc3fffff640 1 -- 172.21.15.172:0/1357175251 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc408030610 con 0x7fc40c0d2830 2024-04-14T08:37:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.037+0000 7fc3fffff640 1 -- 172.21.15.172:0/1357175251 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4080309e0 con 0x7fc40c0d2830 2024-04-14T08:37:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.037+0000 7fc413836640 1 -- 172.21.15.172:0/1357175251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc40c0d2830 msgr2=0x7fc40c0d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.037+0000 7fc413836640 1 --2- 172.21.15.172:0/1357175251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc40c0d2830 0x7fc40c0d2c10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc4080097a0 tx=0x7fc40802b960 comp rx=0 tx=0).stop 2024-04-14T08:37:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.037+0000 7fc413836640 1 -- 172.21.15.172:0/1357175251 shutdown_connections 2024-04-14T08:37:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.037+0000 7fc413836640 1 --2- 172.21.15.172:0/1357175251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc40c15e890 0x7fc40c0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.037+0000 7fc413836640 1 --2- 172.21.15.172:0/1357175251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc40c0d2830 0x7fc40c0d2c10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.037+0000 7fc413836640 1 -- 172.21.15.172:0/1357175251 >> 172.21.15.172:0/1357175251 conn(0x7fc40c09d590 msgr2=0x7fc40c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.037+0000 7fc413836640 1 -- 172.21.15.172:0/1357175251 shutdown_connections 2024-04-14T08:37:11.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.037+0000 7fc413836640 1 -- 172.21.15.172:0/1357175251 wait complete. 2024-04-14T08:37:11.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.038+0000 7fc413836640 1 Processor -- start 2024-04-14T08:37:11.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.038+0000 7fc413836640 1 -- start start 2024-04-14T08:37:11.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.039+0000 7fc413836640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc40c0cedd0 0x7fc40c0b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:11.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.039+0000 7fc413836640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc40c0d2830 0x7fc40c0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:11.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.039+0000 7fc413836640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc40c15e890 0x7fc40c0c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:11.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.039+0000 7fc413836640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c0cfbc0 con 0x7fc40c15e890 2024-04-14T08:37:11.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.039+0000 7fc413836640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c0cf8e0 con 0x7fc40c0d2830 2024-04-14T08:37:11.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.039+0000 7fc413836640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc40c0d1840 con 0x7fc40c0cedd0 2024-04-14T08:37:11.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.039+0000 7fc411dd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc40c15e890 0x7fc40c0c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:11.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.039+0000 7fc411dd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc40c15e890 0x7fc40c0c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46720/0 (socket says 172.21.15.172:46720) 2024-04-14T08:37:11.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.039+0000 7fc411dd3640 1 -- 172.21.15.172:0/62177131 learned_addr learned my addr 172.21.15.172:0/62177131 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.039+0000 7fc410dd1640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc40c0d2830 0x7fc40c0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.039+0000 7fc4115d2640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc40c0cedd0 0x7fc40c0b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.040+0000 7fc411dd3640 1 -- 172.21.15.172:0/62177131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc40c0cedd0 msgr2=0x7fc40c0b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.040+0000 7fc411dd3640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc40c0cedd0 0x7fc40c0b6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.040+0000 7fc411dd3640 1 -- 172.21.15.172:0/62177131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc40c0d2830 msgr2=0x7fc40c0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.040+0000 7fc411dd3640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc40c0d2830 0x7fc40c0cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.040+0000 7fc411dd3640 1 -- 172.21.15.172:0/62177131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c0ce260 con 0x7fc40c15e890 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.040+0000 7fc411dd3640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc40c15e890 0x7fc40c0c95c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc40400c700 tx=0x7fc40400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.040+0000 7fc3fdffb640 1 -- 172.21.15.172:0/62177131 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404018070 con 0x7fc40c15e890 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.040+0000 7fc3fdffb640 1 -- 172.21.15.172:0/62177131 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40400ce10 con 0x7fc40c15e890 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.040+0000 7fc413836640 1 -- 172.21.15.172:0/62177131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40c0b9960 con 0x7fc40c15e890 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.040+0000 7fc3fdffb640 1 -- 172.21.15.172:0/62177131 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40400a800 con 0x7fc40c15e890 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.040+0000 7fc413836640 1 -- 172.21.15.172:0/62177131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40c0c8dc0 con 0x7fc40c15e890 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.041+0000 7fc413836640 1 -- 172.21.15.172:0/62177131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40c0ce600 con 0x7fc40c15e890 2024-04-14T08:37:11.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.042+0000 7fc3fdffb640 1 -- 172.21.15.172:0/62177131 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc40401b070 con 0x7fc40c15e890 2024-04-14T08:37:11.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.042+0000 7fc3fdffb640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc3f006c8f0 0x7fc3f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:11.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.042+0000 7fc3fdffb640 1 -- 172.21.15.172:0/62177131 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fc40408b1f0 con 0x7fc40c15e890 2024-04-14T08:37:11.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.042+0000 7fc4115d2640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc3f006c8f0 0x7fc3f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:11.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.042+0000 7fc4115d2640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc3f006c8f0 0x7fc3f006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc408036000 tx=0x7fc40803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:11.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.046+0000 7fc3fdffb640 1 -- 172.21.15.172:0/62177131 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc40c0ce600 con 0x7fc40c15e890 2024-04-14T08:37:11.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.266+0000 7fc413836640 1 -- 172.21.15.172:0/62177131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc40c0d2610 con 0x7fc40c15e890 2024-04-14T08:37:11.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.267+0000 7fc3fdffb640 1 -- 172.21.15.172:0/62177131 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) ==== 53+0+73 (secure 0 0 0) 0x7fc404059490 con 0x7fc40c15e890 2024-04-14T08:37:11.268 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:37:11.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.268+0000 7fc413836640 1 -- 172.21.15.172:0/62177131 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc3f006c8f0 msgr2=0x7fc3f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:11.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.269+0000 7fc413836640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc3f006c8f0 0x7fc3f006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc408036000 tx=0x7fc40803b040 comp rx=0 tx=0).stop 2024-04-14T08:37:11.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.269+0000 7fc413836640 1 -- 172.21.15.172:0/62177131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc40c15e890 msgr2=0x7fc40c0c95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:11.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.269+0000 7fc413836640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc40c15e890 0x7fc40c0c95c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc40400c700 tx=0x7fc40400cbc0 comp rx=0 tx=0).stop 2024-04-14T08:37:11.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.269+0000 7fc413836640 1 -- 172.21.15.172:0/62177131 shutdown_connections 2024-04-14T08:37:11.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.269+0000 7fc413836640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc3f006c8f0 0x7fc3f006edb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:11.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.269+0000 7fc413836640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc40c15e890 0x7fc40c0c95c0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:11.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.269+0000 7fc413836640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc40c0d2830 0x7fc40c0cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:11.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.269+0000 7fc413836640 1 --2- 172.21.15.172:0/62177131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc40c0cedd0 0x7fc40c0b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:11.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.269+0000 7fc413836640 1 -- 172.21.15.172:0/62177131 >> 172.21.15.172:0/62177131 conn(0x7fc40c09d590 msgr2=0x7fc40c0b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:11.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.269+0000 7fc413836640 1 -- 172.21.15.172:0/62177131 shutdown_connections 2024-04-14T08:37:11.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11.269+0000 7fc413836640 1 -- 172.21.15.172:0/62177131 wait complete. 2024-04-14T08:37:11.281 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:37:11.281 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:37:11.282 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:37:11.282 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:37:11.282 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:37:11.282 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:37:11.282 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:37:11.282 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:37:11.282 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:37:11.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:11.283 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:37:11.298 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:37:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:37:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:37:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:37:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:11.298 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:37:11.299 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:37:11.299 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:37:11.299 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:37:11.299 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:37:11.299 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:37:11.302 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:37:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:37:11.302 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:37:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:37:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:37:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42438sec preferred_lft 42438sec 2024-04-14T08:37:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:37:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:11.302 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:37:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:37:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:37:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:37:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:11.303 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:37:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:37:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:37:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:11.303 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:37:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:37:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:37:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:11.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:11.304 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:37:11.364 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:37:11.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:11.366 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:37:11.421 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:37:11.421 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:37:11.421 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:37:11.422 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:37:11.422 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:37:11.422 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:37:11.422 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:37:11.422 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:37:11.422 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:37:11.422 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:37:11.422 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:37:11.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:37:11.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:37:11.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:11.557 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:37:11.573 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:37:11.573 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:37:11.574 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:37:11.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:11.574 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:37:11.641 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:37:11.642 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:37:11.697 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:37:11.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:37:11.697 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:37:11.752 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:37:11.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:11.752 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:37:11.837 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:37:11.837 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:37:11.837 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:37:11.846 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:37:11.846 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:37:11.846 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:37:11.846 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:37:11.846 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:37:11.852 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:37:11.852 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:37:11.852 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:37:11.852 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:37:11.852 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:37:11.852 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:37:11.852 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:37:11.852 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:37:11.853 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:37:11.853 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:37:11.853 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:37:11.853 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:37:11.853 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:37:11.854 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:37:11.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:11.854 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:37:11.882 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:37:11.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:37:11.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:37:11.982 INFO:teuthology.orchestra.run.smithi172.stdout:client6008 2024-04-14T08:37:11.982 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:37:11.982 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:37:11.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:11.997 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-14T08:37:12.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:12.057 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-14T08:37:12.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:12.113 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-14T08:37:12.169 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:37:12.170 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:37:12.170 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:37:12.170 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:37:12.170 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-14T08:37:12.170 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:37:12.279 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627776 2024-04-14T08:37:12.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:37:12.288 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:37:12.288 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:37:12.344 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:37:12.344 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:37:12.399 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:37:12.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.542+0000 7fdb50498640 1 Processor -- start 2024-04-14T08:37:12.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.542+0000 7fdb50498640 1 -- start start 2024-04-14T08:37:12.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.543+0000 7fdb50498640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdb480d2830 0x7fdb480d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.543+0000 7fdb50498640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb480d3150 con 0x7fdb4815e890 2024-04-14T08:37:12.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.543+0000 7fdb50498640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb480cf320 con 0x7fdb480cedd0 2024-04-14T08:37:12.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.543+0000 7fdb50498640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fdb480cff30 con 0x7fdb480d2830 2024-04-14T08:37:12.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.543+0000 7fdb4da33640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fdb4815e890 0x7fdb480d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59220/0 (socket says 172.21.15.172:59220) 2024-04-14T08:37:12.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.543+0000 7fdb4da33640 1 -- 172.21.15.172:0/1553331148 learned_addr learned my addr 172.21.15.172:0/1553331148 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:12.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.543+0000 7fdb4e234640 1 --2- 172.21.15.172:0/1553331148 >> v2:172.21.15.177:3300/0 conn(0x7fdb480d2830 0x7fdb480d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.543+0000 7fdb4d232640 1 -- 172.21.15.172:0/1553331148 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3245426745 0 0) 0x7fdb480d3150 con 0x7fdb4815e890 2024-04-14T08:37:12.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.544+0000 7fdb4d232640 1 -- 172.21.15.172:0/1553331148 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb34003520 con 0x7fdb4815e890 2024-04-14T08:37:12.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.544+0000 7fdb4d232640 1 -- 172.21.15.172:0/1553331148 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 102943401 0 0) 0x7fdb34003520 con 0x7fdb4815e890 2024-04-14T08:37:12.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.544+0000 7fdb4e234640 1 -- 172.21.15.172:0/1553331148 >> v1:172.21.15.177:6789/0 conn(0x7fdb480cedd0 legacy=0x7fdb480b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:12.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.544+0000 7fdb4e234640 1 -- 172.21.15.172:0/1553331148 >> v1:172.21.15.172:6789/0 conn(0x7fdb4815e890 legacy=0x7fdb480d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:12.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.544+0000 7fdb4e234640 1 -- 172.21.15.172:0/1553331148 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb480cf600 con 0x7fdb480d2830 2024-04-14T08:37:12.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.545+0000 7fdb4e234640 1 --2- 172.21.15.172:0/1553331148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb480d2830 0x7fdb480d2c10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdb400097a0 tx=0x7fdb4002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ad0d8225f37f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:12.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.545+0000 7fdb4d232640 1 -- 172.21.15.172:0/1553331148 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38001670 con 0x7fdb480d2830 2024-04-14T08:37:12.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.545+0000 7fdb4d232640 1 -- 172.21.15.172:0/1553331148 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb40037070 con 0x7fdb480d2830 2024-04-14T08:37:12.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.545+0000 7fdb4d232640 1 -- 172.21.15.172:0/1553331148 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40031790 con 0x7fdb480d2830 2024-04-14T08:37:12.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.545+0000 7fdb50498640 1 -- 172.21.15.172:0/1553331148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb480d2830 msgr2=0x7fdb480d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:12.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.545+0000 7fdb50498640 1 --2- 172.21.15.172:0/1553331148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb480d2830 0x7fdb480d2c10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdb400097a0 tx=0x7fdb4002b8b0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.545+0000 7fdb50498640 1 -- 172.21.15.172:0/1553331148 shutdown_connections 2024-04-14T08:37:12.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.545+0000 7fdb50498640 1 --2- 172.21.15.172:0/1553331148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb480d2830 0x7fdb480d2c10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.545+0000 7fdb50498640 1 -- 172.21.15.172:0/1553331148 >> 172.21.15.172:0/1553331148 conn(0x7fdb4809d590 msgr2=0x7fdb480d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:12.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.545+0000 7fdb50498640 1 -- 172.21.15.172:0/1553331148 shutdown_connections 2024-04-14T08:37:12.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.545+0000 7fdb50498640 1 -- 172.21.15.172:0/1553331148 wait complete. 2024-04-14T08:37:12.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.547+0000 7fdb50498640 1 Processor -- start 2024-04-14T08:37:12.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.547+0000 7fdb50498640 1 -- start start 2024-04-14T08:37:12.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.547+0000 7fdb50498640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb480cedd0 0x7fdb481582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.547+0000 7fdb50498640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb480d2830 0x7fdb480b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.547+0000 7fdb50498640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb4815e890 0x7fdb48161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.547+0000 7fdb50498640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdb480cfbc0 con 0x7fdb480d2830 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.547+0000 7fdb50498640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdb480d2350 con 0x7fdb4815e890 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.547+0000 7fdb50498640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdb480b3e20 con 0x7fdb480cedd0 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.547+0000 7fdb4da33640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb480d2830 0x7fdb480b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.547+0000 7fdb4da33640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb480d2830 0x7fdb480b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46724/0 (socket says 172.21.15.172:46724) 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.547+0000 7fdb4da33640 1 -- 172.21.15.172:0/2632903033 learned_addr learned my addr 172.21.15.172:0/2632903033 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.547+0000 7fdb4e234640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb480cedd0 0x7fdb481582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb4ea35640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb4815e890 0x7fdb48161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb4da33640 1 -- 172.21.15.172:0/2632903033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb480cedd0 msgr2=0x7fdb481582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb4da33640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb480cedd0 0x7fdb481582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb4da33640 1 -- 172.21.15.172:0/2632903033 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb4815e890 msgr2=0x7fdb48161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb4da33640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb4815e890 0x7fdb48161570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb4da33640 1 -- 172.21.15.172:0/2632903033 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb480b7500 con 0x7fdb480d2830 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb4ea35640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb4815e890 0x7fdb48161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:12.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb4e234640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb480cedd0 0x7fdb481582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:12.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb4da33640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb480d2830 0x7fdb480b6e70 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c520 tx=0x7fdb3800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:12.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb3effd640 1 -- 172.21.15.172:0/2632903033 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38017070 con 0x7fdb480d2830 2024-04-14T08:37:12.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb3effd640 1 -- 172.21.15.172:0/2632903033 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb38005d00 con 0x7fdb480d2830 2024-04-14T08:37:12.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb3effd640 1 -- 172.21.15.172:0/2632903033 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb380072b0 con 0x7fdb480d2830 2024-04-14T08:37:12.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb50498640 1 -- 172.21.15.172:0/2632903033 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb480b77e0 con 0x7fdb480d2830 2024-04-14T08:37:12.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.548+0000 7fdb50498640 1 -- 172.21.15.172:0/2632903033 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb480b7da0 con 0x7fdb480d2830 2024-04-14T08:37:12.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.549+0000 7fdb50498640 1 -- 172.21.15.172:0/2632903033 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb48068520 con 0x7fdb480d2830 2024-04-14T08:37:12.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.550+0000 7fdb3effd640 1 -- 172.21.15.172:0/2632903033 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb3801a070 con 0x7fdb480d2830 2024-04-14T08:37:12.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.550+0000 7fdb3effd640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdb2c06c8f0 0x7fdb2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.550+0000 7fdb4e234640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdb2c06c8f0 0x7fdb2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.550+0000 7fdb3effd640 1 -- 172.21.15.172:0/2632903033 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fdb3808b120 con 0x7fdb480d2830 2024-04-14T08:37:12.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.551+0000 7fdb4e234640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdb2c06c8f0 0x7fdb2c06edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdb40036000 tx=0x7fdb4003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:12.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.554+0000 7fdb3effd640 1 -- 172.21.15.172:0/2632903033 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdb38059450 con 0x7fdb480d2830 2024-04-14T08:37:12.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.731+0000 7fdb50498640 1 -- 172.21.15.172:0/2632903033 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb480b8b50 con 0x7fdb480d2830 2024-04-14T08:37:12.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.732+0000 7fdb3effd640 1 -- 172.21.15.172:0/2632903033 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+17005 (secure 0 0 0) 0x7fdb38058d50 con 0x7fdb480d2830 2024-04-14T08:37:12.734 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:12.734 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":74,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:37:02.756327+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:36:59.342045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-14T08:37:00.059338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:37:12.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.735+0000 7fdb50498640 1 -- 172.21.15.172:0/2632903033 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdb2c06c8f0 msgr2=0x7fdb2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:12.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.735+0000 7fdb50498640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdb2c06c8f0 0x7fdb2c06edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdb40036000 tx=0x7fdb4003d040 comp rx=0 tx=0).stop 2024-04-14T08:37:12.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.735+0000 7fdb50498640 1 -- 172.21.15.172:0/2632903033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb480d2830 msgr2=0x7fdb480b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:12.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.735+0000 7fdb50498640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb480d2830 0x7fdb480b6e70 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c520 tx=0x7fdb3800c9e0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.735+0000 7fdb50498640 1 -- 172.21.15.172:0/2632903033 shutdown_connections 2024-04-14T08:37:12.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.735+0000 7fdb50498640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdb2c06c8f0 0x7fdb2c06edb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.735+0000 7fdb50498640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb4815e890 0x7fdb48161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.735+0000 7fdb50498640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb480d2830 0x7fdb480b6e70 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.735+0000 7fdb50498640 1 --2- 172.21.15.172:0/2632903033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb480cedd0 0x7fdb481582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.735+0000 7fdb50498640 1 -- 172.21.15.172:0/2632903033 >> 172.21.15.172:0/2632903033 conn(0x7fdb4809d590 msgr2=0x7fdb480b58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:12.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.735+0000 7fdb50498640 1 -- 172.21.15.172:0/2632903033 shutdown_connections 2024-04-14T08:37:12.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.735+0000 7fdb50498640 1 -- 172.21.15.172:0/2632903033 wait complete. 2024-04-14T08:37:12.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:36:59.342045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:37:00.059338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:37:12.748 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:37:12.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.869+0000 7f9666636640 1 Processor -- start 2024-04-14T08:37:12.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.869+0000 7f9666636640 1 -- start start 2024-04-14T08:37:12.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.870+0000 7f9666636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96600d0cc0 0x7f96600d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.870+0000 7f9666636640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f96600d49c0 0x7f96600d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.870+0000 7f9666636640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96600d15e0 con 0x7f9660160f30 2024-04-14T08:37:12.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.870+0000 7f9666636640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f96600ceec0 con 0x7f96600d49c0 2024-04-14T08:37:12.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.870+0000 7f9666636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f96600cfd00 con 0x7f96600d0cc0 2024-04-14T08:37:12.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.870+0000 7f9664bd3640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9660160f30 0x7f96601613c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59222/0 (socket says 172.21.15.172:59222) 2024-04-14T08:37:12.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.870+0000 7f9664bd3640 1 -- 172.21.15.172:0/750047766 learned_addr learned my addr 172.21.15.172:0/750047766 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:12.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.870+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2179418199 0 0) 0x7f96600d15e0 con 0x7f9660160f30 2024-04-14T08:37:12.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.870+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9648003520 con 0x7f9660160f30 2024-04-14T08:37:12.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.870+0000 7f965f7fe640 1 --2- 172.21.15.172:0/750047766 >> v2:172.21.15.177:3300/0 conn(0x7f96600d49c0 0x7f96600d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.870+0000 7f965ffff640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96600d0cc0 0x7f96600d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3564617658 0 0) 0x7f9648003520 con 0x7f9660160f30 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96600d0cc0 msgr2=0x7f96600d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96600d0cc0 0x7f96600d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 >> v2:172.21.15.177:3300/0 conn(0x7f96600d49c0 msgr2=0x7f96600d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 --2- 172.21.15.172:0/750047766 >> v2:172.21.15.177:3300/0 conn(0x7f96600d49c0 0x7f96600d4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96600cf0f0 con 0x7f9660160f30 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f96580019f0 con 0x7f9660160f30 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 >> v1:172.21.15.172:6789/0 conn(0x7f9660160f30 legacy=0x7f96601613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96480039b0 0x7f9648003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9648004800 0x7f9648004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96480051c0 0x7f964800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96600cfd00 con 0x7f9648004800 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f964800bfb0 con 0x7f96480039b0 2024-04-14T08:37:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f964800c180 con 0x7f96480051c0 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965ffff640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96600d0cc0 0x7f96600d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965f7fe640 1 --2- 172.21.15.172:0/750047766 >> v2:172.21.15.177:3300/0 conn(0x7f96600d49c0 0x7f96600d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965f7fe640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9648004800 0x7f9648004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f9664bd3640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96480051c0 0x7f964800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965ffff640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96480039b0 0x7f9648003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965f7fe640 1 -- 172.21.15.172:0/750047766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96480051c0 msgr2=0x7f964800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965f7fe640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96480051c0 0x7f964800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965f7fe640 1 -- 172.21.15.172:0/750047766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96480039b0 msgr2=0x7f9648003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965f7fe640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96480039b0 0x7f9648003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.871+0000 7f965f7fe640 1 -- 172.21.15.172:0/750047766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9648003520 con 0x7f9648004800 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.872+0000 7f965f7fe640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9648004800 0x7f9648004c80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f96500098d0 tx=0x7f9648005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7f5d3015834c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.872+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650034070 con 0x7f9648004800 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.872+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9650030680 con 0x7f9648004800 2024-04-14T08:37:12.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.872+0000 7f965effd640 1 -- 172.21.15.172:0/750047766 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965000f040 con 0x7f9648004800 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.873+0000 7f9666636640 1 -- 172.21.15.172:0/750047766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9648004800 msgr2=0x7f9648004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.873+0000 7f9666636640 1 --2- 172.21.15.172:0/750047766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9648004800 0x7f9648004c80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f96500098d0 tx=0x7f9648005830 comp rx=0 tx=0).stop 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.873+0000 7f9664bd3640 1 -- 172.21.15.172:0/750047766 reap_dead start 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.873+0000 7f9666636640 1 -- 172.21.15.172:0/750047766 shutdown_connections 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.873+0000 7f9666636640 1 -- 172.21.15.172:0/750047766 >> 172.21.15.172:0/750047766 conn(0x7f966009d580 msgr2=0x7f96600d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.873+0000 7f9666636640 1 -- 172.21.15.172:0/750047766 shutdown_connections 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.873+0000 7f9666636640 1 -- 172.21.15.172:0/750047766 wait complete. 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.874+0000 7f9666636640 1 Processor -- start 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.874+0000 7f9666636640 1 -- start start 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f9666636640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9648004800 0x7f96600ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f9666636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f966015b140 0x7f96601f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f9666636640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f966009dff0 0x7f966009faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f9666636640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96600cf990 con 0x7f9648004800 2024-04-14T08:37:12.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f9666636640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f96600cf6b0 con 0x7f966009dff0 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f9666636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f96600d32b0 con 0x7f966015b140 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f965ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9648004800 0x7f96600ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f965ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9648004800 0x7f96600ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46742/0 (socket says 172.21.15.172:46742) 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f965ffff640 1 -- 172.21.15.172:0/2339999091 learned_addr learned my addr 172.21.15.172:0/2339999091 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f9664bd3640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f966009dff0 0x7f966009faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f965f7fe640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f966015b140 0x7f96601f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f965ffff640 1 -- 172.21.15.172:0/2339999091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f966015b140 msgr2=0x7f96601f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f965ffff640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f966015b140 0x7f96601f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f965ffff640 1 -- 172.21.15.172:0/2339999091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f966009dff0 msgr2=0x7f966009faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f965ffff640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f966009dff0 0x7f966009faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.875+0000 7f965ffff640 1 -- 172.21.15.172:0/2339999091 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96601f4ee0 con 0x7f9648004800 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.876+0000 7f965ffff640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9648004800 0x7f96600ca4f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9654011540 tx=0x7f9654011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.876+0000 7f965cff9640 1 -- 172.21.15.172:0/2339999091 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654019020 con 0x7f9648004800 2024-04-14T08:37:12.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.876+0000 7f965cff9640 1 -- 172.21.15.172:0/2339999091 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9654004440 con 0x7f9648004800 2024-04-14T08:37:12.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.876+0000 7f965cff9640 1 -- 172.21.15.172:0/2339999091 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965400c070 con 0x7f9648004800 2024-04-14T08:37:12.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.876+0000 7f9666636640 1 -- 172.21.15.172:0/2339999091 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96600caad0 con 0x7f9648004800 2024-04-14T08:37:12.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.876+0000 7f9666636640 1 -- 172.21.15.172:0/2339999091 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96600cb090 con 0x7f9648004800 2024-04-14T08:37:12.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.877+0000 7f9666636640 1 -- 172.21.15.172:0/2339999091 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9660075490 con 0x7f9648004800 2024-04-14T08:37:12.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.877+0000 7f965cff9640 1 -- 172.21.15.172:0/2339999091 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f965401c020 con 0x7f9648004800 2024-04-14T08:37:12.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.878+0000 7f965cff9640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f964c06c8f0 0x7f964c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:12.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.878+0000 7f965cff9640 1 -- 172.21.15.172:0/2339999091 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f965408bae0 con 0x7f9648004800 2024-04-14T08:37:12.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.878+0000 7f965f7fe640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f964c06c8f0 0x7f964c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:12.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.878+0000 7f965f7fe640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f964c06c8f0 0x7f964c06edb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9650036000 tx=0x7f9650031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:12.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:12.882+0000 7f965cff9640 1 -- 172.21.15.172:0/2339999091 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9654023030 con 0x7f9648004800 2024-04-14T08:37:13.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.062+0000 7f9666636640 1 -- 172.21.15.172:0/2339999091 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f96600d11a0 con 0x7f9648004800 2024-04-14T08:37:13.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.062+0000 7f965cff9640 1 -- 172.21.15.172:0/2339999091 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f9654059e10 con 0x7f9648004800 2024-04-14T08:37:13.064 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:37:13.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.064+0000 7f9666636640 1 -- 172.21.15.172:0/2339999091 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f964c06c8f0 msgr2=0x7f964c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.064+0000 7f9666636640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f964c06c8f0 0x7f964c06edb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9650036000 tx=0x7f9650031000 comp rx=0 tx=0).stop 2024-04-14T08:37:13.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.064+0000 7f9666636640 1 -- 172.21.15.172:0/2339999091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9648004800 msgr2=0x7f96600ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.064+0000 7f9666636640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9648004800 0x7f96600ca4f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9654011540 tx=0x7f9654011a00 comp rx=0 tx=0).stop 2024-04-14T08:37:13.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.064+0000 7f9666636640 1 -- 172.21.15.172:0/2339999091 shutdown_connections 2024-04-14T08:37:13.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.064+0000 7f9666636640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f964c06c8f0 0x7f964c06edb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.064+0000 7f9666636640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f966009dff0 0x7f966009faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.064+0000 7f9666636640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f966015b140 0x7f96601f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.064+0000 7f9666636640 1 --2- 172.21.15.172:0/2339999091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9648004800 0x7f96600ca4f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.064+0000 7f9666636640 1 -- 172.21.15.172:0/2339999091 >> 172.21.15.172:0/2339999091 conn(0x7f966009d580 msgr2=0x7f96600cb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:13.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.064+0000 7f9666636640 1 -- 172.21.15.172:0/2339999091 shutdown_connections 2024-04-14T08:37:13.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.064+0000 7f9666636640 1 -- 172.21.15.172:0/2339999091 wait complete. 2024-04-14T08:37:13.077 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:37:13.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.181+0000 7f8821436640 1 Processor -- start 2024-04-14T08:37:13.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.181+0000 7f8821436640 1 -- start start 2024-04-14T08:37:13.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.181+0000 7f8821436640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f881c0d1a20 0x7f881c06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.181+0000 7f8821436640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f881c071250 0x7f881c0cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f8821436640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881c0701b0 con 0x7f881c06d000 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f8821436640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f881c070dc0 con 0x7f881c071250 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f8821436640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f881c070490 con 0x7f881c0d1a20 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f881a7fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f881c071250 0x7f881c0cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f881a7fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f881c071250 0x7f881c0cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46758/0 (socket says 172.21.15.172:46758) 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f881a7fc640 1 -- 172.21.15.172:0/409976631 learned_addr learned my addr 172.21.15.172:0/409976631 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f881b7fe640 1 --1- 172.21.15.172:0/409976631 >> v1:172.21.15.172:6789/0 conn(0x7f881c06d000 0x7f881c15dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59238/0 (socket says 172.21.15.172:59238) 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f881affd640 1 --2- 172.21.15.172:0/409976631 >> v2:172.21.15.177:3300/0 conn(0x7f881c0d1a20 0x7f881c06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f881a7fc640 1 -- 172.21.15.172:0/409976631 >> v2:172.21.15.177:3300/0 conn(0x7f881c0d1a20 msgr2=0x7f881c06fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f881a7fc640 1 --2- 172.21.15.172:0/409976631 >> v2:172.21.15.177:3300/0 conn(0x7f881c0d1a20 0x7f881c06fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f881a7fc640 1 -- 172.21.15.172:0/409976631 >> v1:172.21.15.172:6789/0 conn(0x7f881c06d000 legacy=0x7f881c15dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f881a7fc640 1 -- 172.21.15.172:0/409976631 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c070770 con 0x7f881c071250 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.182+0000 7f8819ffb640 1 -- 172.21.15.172:0/409976631 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3622857200 0 0) 0x7f881c0701b0 con 0x7f881c06d000 2024-04-14T08:37:13.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.183+0000 7f881a7fc640 1 --2- 172.21.15.172:0/409976631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881c071250 0x7f881c0cf530 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f881c0740b0 tx=0x7f881002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb23f6a9ffa1184a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.183+0000 7f8819ffb640 1 -- 172.21.15.172:0/409976631 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810035020 con 0x7f881c071250 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.183+0000 7f8819ffb640 1 -- 172.21.15.172:0/409976631 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8810031870 con 0x7f881c071250 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.183+0000 7f8819ffb640 1 -- 172.21.15.172:0/409976631 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810031c40 con 0x7f881c071250 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.183+0000 7f8821436640 1 -- 172.21.15.172:0/409976631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881c071250 msgr2=0x7f881c0cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.183+0000 7f8821436640 1 --2- 172.21.15.172:0/409976631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881c071250 0x7f881c0cf530 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f881c0740b0 tx=0x7f881002b910 comp rx=0 tx=0).stop 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.183+0000 7f8821436640 1 -- 172.21.15.172:0/409976631 shutdown_connections 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.183+0000 7f8821436640 1 --2- 172.21.15.172:0/409976631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881c071250 0x7f881c0cf530 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.183+0000 7f8821436640 1 --2- 172.21.15.172:0/409976631 >> v2:172.21.15.177:3300/0 conn(0x7f881c0d1a20 0x7f881c06fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.183+0000 7f8821436640 1 -- 172.21.15.172:0/409976631 >> 172.21.15.172:0/409976631 conn(0x7f881c08c000 msgr2=0x7f881c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.184+0000 7f8821436640 1 -- 172.21.15.172:0/409976631 shutdown_connections 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.184+0000 7f8821436640 1 -- 172.21.15.172:0/409976631 wait complete. 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.185+0000 7f8821436640 1 Processor -- start 2024-04-14T08:37:13.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.186+0000 7f8821436640 1 -- start start 2024-04-14T08:37:13.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f8821436640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881c06d000 0x7f881c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f8821436640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f881c071250 0x7f881c15eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f8821436640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f881c0d1a20 0x7f881c158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f8821436640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f881c0d1e00 con 0x7f881c06d000 2024-04-14T08:37:13.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f8821436640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f881c0cfa70 con 0x7f881c0d1a20 2024-04-14T08:37:13.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f8821436640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f881c15f0c0 con 0x7f881c071250 2024-04-14T08:37:13.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f881affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881c06d000 0x7f881c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f881affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881c06d000 0x7f881c0cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46760/0 (socket says 172.21.15.172:46760) 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f881affd640 1 -- 172.21.15.172:0/1380138297 learned_addr learned my addr 172.21.15.172:0/1380138297 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f881b7fe640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f881c0d1a20 0x7f881c158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f881a7fc640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f881c071250 0x7f881c15eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f881affd640 1 -- 172.21.15.172:0/1380138297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f881c071250 msgr2=0x7f881c15eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f881affd640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f881c071250 0x7f881c15eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f881affd640 1 -- 172.21.15.172:0/1380138297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f881c0d1a20 msgr2=0x7f881c158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f881affd640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f881c0d1a20 0x7f881c158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f881affd640 1 -- 172.21.15.172:0/1380138297 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c15f320 con 0x7f881c06d000 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f881affd640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881c06d000 0x7f881c0cbdf0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f880c00e8a0 tx=0x7f880c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f88037fe640 1 -- 172.21.15.172:0/1380138297 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c017070 con 0x7f881c06d000 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f88037fe640 1 -- 172.21.15.172:0/1380138297 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c00c940 con 0x7f881c06d000 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f8821436640 1 -- 172.21.15.172:0/1380138297 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c15f600 con 0x7f881c06d000 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.187+0000 7f88037fe640 1 -- 172.21.15.172:0/1380138297 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c00cd30 con 0x7f881c06d000 2024-04-14T08:37:13.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.188+0000 7f8821436640 1 -- 172.21.15.172:0/1380138297 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c0c9480 con 0x7f881c06d000 2024-04-14T08:37:13.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.188+0000 7f8821436640 1 -- 172.21.15.172:0/1380138297 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881c15fc80 con 0x7f881c06d000 2024-04-14T08:37:13.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.189+0000 7f88037fe640 1 -- 172.21.15.172:0/1380138297 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f880c01a070 con 0x7f881c06d000 2024-04-14T08:37:13.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.189+0000 7f88037fe640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f880806c8f0 0x7f880806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.189+0000 7f88037fe640 1 -- 172.21.15.172:0/1380138297 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f880c08bea0 con 0x7f881c06d000 2024-04-14T08:37:13.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.189+0000 7f881a7fc640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f880806c8f0 0x7f880806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.190+0000 7f881a7fc640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f880806c8f0 0x7f880806edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8810036000 tx=0x7f881003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:13.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.194+0000 7f88037fe640 1 -- 172.21.15.172:0/1380138297 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f881c15fc80 con 0x7f881c06d000 2024-04-14T08:37:13.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.376+0000 7f8821436640 1 -- 172.21.15.172:0/1380138297 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f881c075490 con 0x7f881c06d000 2024-04-14T08:37:13.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.376+0000 7f88037fe640 1 -- 172.21.15.172:0/1380138297 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) ==== 104+0+11 (secure 0 0 0) 0x7f880c05a1d0 con 0x7f881c06d000 2024-04-14T08:37:13.378 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:37:13.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.379+0000 7f8821436640 1 -- 172.21.15.172:0/1380138297 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f880806c8f0 msgr2=0x7f880806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.379+0000 7f8821436640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f880806c8f0 0x7f880806edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8810036000 tx=0x7f881003c040 comp rx=0 tx=0).stop 2024-04-14T08:37:13.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.379+0000 7f8821436640 1 -- 172.21.15.172:0/1380138297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881c06d000 msgr2=0x7f881c0cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.379+0000 7f8821436640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881c06d000 0x7f881c0cbdf0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f880c00e8a0 tx=0x7f880c00ed60 comp rx=0 tx=0).stop 2024-04-14T08:37:13.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.379+0000 7f8821436640 1 -- 172.21.15.172:0/1380138297 shutdown_connections 2024-04-14T08:37:13.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.379+0000 7f8821436640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f880806c8f0 0x7f880806edb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.379+0000 7f8821436640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f881c0d1a20 0x7f881c158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.379+0000 7f8821436640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f881c071250 0x7f881c15eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.379+0000 7f8821436640 1 --2- 172.21.15.172:0/1380138297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f881c06d000 0x7f881c0cbdf0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.379+0000 7f8821436640 1 -- 172.21.15.172:0/1380138297 >> 172.21.15.172:0/1380138297 conn(0x7f881c08c000 msgr2=0x7f881c0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:13.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.379+0000 7f8821436640 1 -- 172.21.15.172:0/1380138297 shutdown_connections 2024-04-14T08:37:13.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.379+0000 7f8821436640 1 -- 172.21.15.172:0/1380138297 wait complete. 2024-04-14T08:37:13.391 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:37:13.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.494+0000 7fdbcb3fe640 1 Processor -- start 2024-04-14T08:37:13.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.494+0000 7fdbcb3fe640 1 -- start start 2024-04-14T08:37:13.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbcb3fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdbc415e430 0x7fdbc4162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbcb3fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdbc4162540 0x7fdbc41629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbcb3fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc40b3a30 0x7fdbc40b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbcb3fe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fdbc40cefb0 con 0x7fdbc4162540 2024-04-14T08:37:13.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbcb3fe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fdbc40cf290 con 0x7fdbc415e430 2024-04-14T08:37:13.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbcb3fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc40cf570 con 0x7fdbc40b3a30 2024-04-14T08:37:13.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbc8999640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdbc4162540 0x7fdbc41629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbc8999640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdbc4162540 0x7fdbc41629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46768/0 (socket says 172.21.15.172:46768) 2024-04-14T08:37:13.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbc8999640 1 -- 172.21.15.172:0/2162949345 learned_addr learned my addr 172.21.15.172:0/2162949345 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:13.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbc919a640 1 --2- 172.21.15.172:0/2162949345 >> v2:172.21.15.177:3300/0 conn(0x7fdbc415e430 0x7fdbc4162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbc999b640 1 --2- 172.21.15.172:0/2162949345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc40b3a30 0x7fdbc40b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbc8999640 1 -- 172.21.15.172:0/2162949345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc40b3a30 msgr2=0x7fdbc40b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:13.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbc8999640 1 --2- 172.21.15.172:0/2162949345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc40b3a30 0x7fdbc40b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbc8999640 1 -- 172.21.15.172:0/2162949345 >> v2:172.21.15.177:3300/0 conn(0x7fdbc415e430 msgr2=0x7fdbc4162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:13.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbc8999640 1 --2- 172.21.15.172:0/2162949345 >> v2:172.21.15.177:3300/0 conn(0x7fdbc415e430 0x7fdbc4162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.495+0000 7fdbc8999640 1 -- 172.21.15.172:0/2162949345 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc40cf850 con 0x7fdbc4162540 2024-04-14T08:37:13.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbc8999640 1 --2- 172.21.15.172:0/2162949345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc4162540 0x7fdbc41629d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40b6db0 tx=0x7fdbb802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd45e608bb1baf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:13.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbc3fff640 1 -- 172.21.15.172:0/2162949345 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8035020 con 0x7fdbc4162540 2024-04-14T08:37:13.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbc3fff640 1 -- 172.21.15.172:0/2162949345 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb8031870 con 0x7fdbc4162540 2024-04-14T08:37:13.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbc3fff640 1 -- 172.21.15.172:0/2162949345 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8031c40 con 0x7fdbc4162540 2024-04-14T08:37:13.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/2162949345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc4162540 msgr2=0x7fdbc41629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbcb3fe640 1 --2- 172.21.15.172:0/2162949345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc4162540 0x7fdbc41629d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40b6db0 tx=0x7fdbb802b910 comp rx=0 tx=0).stop 2024-04-14T08:37:13.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/2162949345 shutdown_connections 2024-04-14T08:37:13.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbcb3fe640 1 --2- 172.21.15.172:0/2162949345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc40b3a30 0x7fdbc40b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbcb3fe640 1 --2- 172.21.15.172:0/2162949345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc4162540 0x7fdbc41629d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbcb3fe640 1 --2- 172.21.15.172:0/2162949345 >> v2:172.21.15.177:3300/0 conn(0x7fdbc415e430 0x7fdbc4162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/2162949345 >> 172.21.15.172:0/2162949345 conn(0x7fdbc409d560 msgr2=0x7fdbc40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:13.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/2162949345 shutdown_connections 2024-04-14T08:37:13.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.497+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/2162949345 wait complete. 2024-04-14T08:37:13.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.499+0000 7fdbcb3fe640 1 Processor -- start 2024-04-14T08:37:13.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.499+0000 7fdbcb3fe640 1 -- start start 2024-04-14T08:37:13.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbcb3fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc40b3a30 0x7fdbc41592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbcb3fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc415e430 0x7fdbc4158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbcb3fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc4162540 0x7fdbc40b8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbcb3fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc40d4e20 con 0x7fdbc4162540 2024-04-14T08:37:13.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbcb3fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc415e810 con 0x7fdbc40b3a30 2024-04-14T08:37:13.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbcb3fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc40d0050 con 0x7fdbc415e430 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbc999b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc4162540 0x7fdbc40b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbc999b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc4162540 0x7fdbc40b8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46776/0 (socket says 172.21.15.172:46776) 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbc999b640 1 -- 172.21.15.172:0/400783057 learned_addr learned my addr 172.21.15.172:0/400783057 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbc919a640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc40b3a30 0x7fdbc41592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbc8999640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc415e430 0x7fdbc4158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbc999b640 1 -- 172.21.15.172:0/400783057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc415e430 msgr2=0x7fdbc4158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbc999b640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc415e430 0x7fdbc4158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbc999b640 1 -- 172.21.15.172:0/400783057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc40b3a30 msgr2=0x7fdbc41592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbc999b640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc40b3a30 0x7fdbc41592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbc999b640 1 -- 172.21.15.172:0/400783057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc40b87a0 con 0x7fdbc4162540 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbc999b640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc4162540 0x7fdbc40b8110 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00ef00 tx=0x7fdbbc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.500+0000 7fdbc1ffb640 1 -- 172.21.15.172:0/400783057 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc018020 con 0x7fdbc4162540 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.501+0000 7fdbc1ffb640 1 -- 172.21.15.172:0/400783057 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbbc00bd20 con 0x7fdbc4162540 2024-04-14T08:37:13.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.501+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/400783057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc40b8a80 con 0x7fdbc4162540 2024-04-14T08:37:13.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.501+0000 7fdbc1ffb640 1 -- 172.21.15.172:0/400783057 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc014640 con 0x7fdbc4162540 2024-04-14T08:37:13.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.501+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/400783057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc40b9040 con 0x7fdbc4162540 2024-04-14T08:37:13.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.502+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/400783057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc4068590 con 0x7fdbc4162540 2024-04-14T08:37:13.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.502+0000 7fdbc1ffb640 1 -- 172.21.15.172:0/400783057 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbbc01b020 con 0x7fdbc4162540 2024-04-14T08:37:13.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.503+0000 7fdbc1ffb640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdbb006c8f0 0x7fdbb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.503+0000 7fdbc1ffb640 1 -- 172.21.15.172:0/400783057 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fdbbc08c3a0 con 0x7fdbc4162540 2024-04-14T08:37:13.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.503+0000 7fdbc919a640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdbb006c8f0 0x7fdbb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.503+0000 7fdbc919a640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdbb006c8f0 0x7fdbb006edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40b63d0 tx=0x7fdbb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:13.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.507+0000 7fdbc1ffb640 1 -- 172.21.15.172:0/400783057 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdbbc022030 con 0x7fdbc4162540 2024-04-14T08:37:13.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.825+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/400783057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdbc40b1bd0 con 0x7fdbc4162540 2024-04-14T08:37:13.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.825+0000 7fdbc1ffb640 1 -- 172.21.15.172:0/400783057 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) ==== 100+0+11 (secure 0 0 0) 0x7fdbbc05a640 con 0x7fdbc4162540 2024-04-14T08:37:13.827 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:37:13.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.827+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/400783057 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdbb006c8f0 msgr2=0x7fdbb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.827+0000 7fdbcb3fe640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdbb006c8f0 0x7fdbb006edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40b63d0 tx=0x7fdbb4008040 comp rx=0 tx=0).stop 2024-04-14T08:37:13.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.827+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/400783057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc4162540 msgr2=0x7fdbc40b8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.827+0000 7fdbcb3fe640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc4162540 0x7fdbc40b8110 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00ef00 tx=0x7fdbbc00b480 comp rx=0 tx=0).stop 2024-04-14T08:37:13.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.827+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/400783057 shutdown_connections 2024-04-14T08:37:13.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.827+0000 7fdbcb3fe640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdbb006c8f0 0x7fdbb006edb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.827+0000 7fdbcb3fe640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbc4162540 0x7fdbc40b8110 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.827+0000 7fdbcb3fe640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdbc415e430 0x7fdbc4158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.827+0000 7fdbcb3fe640 1 --2- 172.21.15.172:0/400783057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbc40b3a30 0x7fdbc41592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.827+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/400783057 >> 172.21.15.172:0/400783057 conn(0x7fdbc409d560 msgr2=0x7fdbc40b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:13.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.828+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/400783057 shutdown_connections 2024-04-14T08:37:13.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.828+0000 7fdbcb3fe640 1 -- 172.21.15.172:0/400783057 wait complete. 2024-04-14T08:37:13.840 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:37:13.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.977+0000 7fd1f7915640 1 Processor -- start 2024-04-14T08:37:13.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.977+0000 7fd1f7915640 1 -- start start 2024-04-14T08:37:13.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.978+0000 7fd1f7915640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1f00b23e0 0x7fd1f0140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.978+0000 7fd1f7915640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1f015b620 con 0x7fd1f013df30 2024-04-14T08:37:13.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.978+0000 7fd1f7915640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1f01414a0 con 0x7fd1f015c380 2024-04-14T08:37:13.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.978+0000 7fd1f7915640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f00cf9b0 con 0x7fd1f00b23e0 2024-04-14T08:37:13.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.978+0000 7fd1f56b1640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd1f013df30 0x7fd1f00b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59250/0 (socket says 172.21.15.172:59250) 2024-04-14T08:37:13.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.978+0000 7fd1f56b1640 1 -- 172.21.15.172:0/129126929 learned_addr learned my addr 172.21.15.172:0/129126929 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:13.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.978+0000 7fd1f4eb0640 1 --2- 172.21.15.172:0/129126929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1f00b23e0 0x7fd1f0140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.978+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4231432455 0 0) 0x7fd1ec001110 con 0x7fd1f013df30 2024-04-14T08:37:13.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.978+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1dc003520 con 0x7fd1f013df30 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.978+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3006539567 0 0) 0x7fd1dc003520 con 0x7fd1f013df30 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.978+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1f00b23e0 msgr2=0x7fd1f0140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.978+0000 7fd1e7fff640 1 --2- 172.21.15.172:0/129126929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1f00b23e0 0x7fd1f0140770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 >> v1:172.21.15.177:6789/0 conn(0x7fd1f015c380 legacy=0x7fd1f0140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f015b8a0 con 0x7fd1f013df30 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fd1ec001840 con 0x7fd1f013df30 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 >> v1:172.21.15.172:6789/0 conn(0x7fd1f013df30 legacy=0x7fd1f00b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1e7fff640 1 --2- 172.21.15.172:0/129126929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1f00b23e0 0x7fd1dc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1e7fff640 1 --2- 172.21.15.172:0/129126929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1dc003f60 0x7fd1dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1e7fff640 1 --2- 172.21.15.172:0/129126929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc004d40 0x7fd1dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f00cf9b0 con 0x7fd1dc004d40 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc00baa0 con 0x7fd1dc003f60 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd1dc00deb0 con 0x7fd1f00b23e0 2024-04-14T08:37:13.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1f5eb2640 1 --2- 172.21.15.172:0/129126929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc004d40 0x7fd1dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1f56b1640 1 --2- 172.21.15.172:0/129126929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1f00b23e0 0x7fd1dc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1f4eb0640 1 --2- 172.21.15.172:0/129126929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1dc003f60 0x7fd1dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1f5eb2640 1 -- 172.21.15.172:0/129126929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1f00b23e0 msgr2=0x7fd1dc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1f5eb2640 1 --2- 172.21.15.172:0/129126929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1f00b23e0 0x7fd1dc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1f5eb2640 1 -- 172.21.15.172:0/129126929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1dc003f60 msgr2=0x7fd1dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1f5eb2640 1 --2- 172.21.15.172:0/129126929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1dc003f60 0x7fd1dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.979+0000 7fd1f5eb2640 1 -- 172.21.15.172:0/129126929 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1dc003520 con 0x7fd1dc004d40 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.980+0000 7fd1f5eb2640 1 --2- 172.21.15.172:0/129126929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc004d40 0x7fd1dc0051c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8001e20 tx=0x7fd1dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821a87e02f3537bf server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.980+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8001a70 con 0x7fd1dc004d40 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.980+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e8003b70 con 0x7fd1dc004d40 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.980+0000 7fd1e7fff640 1 -- 172.21.15.172:0/129126929 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80050d0 con 0x7fd1dc004d40 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.980+0000 7fd1f7915640 1 -- 172.21.15.172:0/129126929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc004d40 msgr2=0x7fd1dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.980+0000 7fd1f7915640 1 --2- 172.21.15.172:0/129126929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc004d40 0x7fd1dc0051c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8001e20 tx=0x7fd1dc005830 comp rx=0 tx=0).stop 2024-04-14T08:37:13.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.980+0000 7fd1f5eb2640 1 -- 172.21.15.172:0/129126929 reap_dead start 2024-04-14T08:37:13.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.980+0000 7fd1f7915640 1 -- 172.21.15.172:0/129126929 shutdown_connections 2024-04-14T08:37:13.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.980+0000 7fd1f7915640 1 -- 172.21.15.172:0/129126929 >> 172.21.15.172:0/129126929 conn(0x7fd1f006b520 msgr2=0x7fd1f006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:13.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.981+0000 7fd1f7915640 1 -- 172.21.15.172:0/129126929 shutdown_connections 2024-04-14T08:37:13.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.981+0000 7fd1f7915640 1 -- 172.21.15.172:0/129126929 wait complete. 2024-04-14T08:37:13.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.982+0000 7fd1f7915640 1 Processor -- start 2024-04-14T08:37:13.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.982+0000 7fd1f7915640 1 -- start start 2024-04-14T08:37:13.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.982+0000 7fd1f7915640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1dc004d40 0x7fd1f00b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.982+0000 7fd1f7915640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f013cb90 0x7fd1f01ee240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.982+0000 7fd1f7915640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1f00b1390 0x7fd1f00b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.982+0000 7fd1f7915640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f013e310 con 0x7fd1f013cb90 2024-04-14T08:37:13.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.982+0000 7fd1f7915640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f015be60 con 0x7fd1f00b1390 2024-04-14T08:37:13.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.982+0000 7fd1f7915640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f0140770 con 0x7fd1dc004d40 2024-04-14T08:37:13.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f4eb0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f013cb90 0x7fd1f01ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f4eb0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f013cb90 0x7fd1f01ee240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46794/0 (socket says 172.21.15.172:46794) 2024-04-14T08:37:13.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f4eb0640 1 -- 172.21.15.172:0/2127409577 learned_addr learned my addr 172.21.15.172:0/2127409577 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:13.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f5eb2640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1f00b1390 0x7fd1f00b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f56b1640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1dc004d40 0x7fd1f00b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f4eb0640 1 -- 172.21.15.172:0/2127409577 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1dc004d40 msgr2=0x7fd1f00b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f4eb0640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1dc004d40 0x7fd1f00b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f4eb0640 1 -- 172.21.15.172:0/2127409577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1f00b1390 msgr2=0x7fd1f00b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f4eb0640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1f00b1390 0x7fd1f00b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f4eb0640 1 -- 172.21.15.172:0/2127409577 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f013d200 con 0x7fd1f013cb90 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f56b1640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1dc004d40 0x7fd1f00b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f4eb0640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f013cb90 0x7fd1f01ee240 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00119a0 tx=0x7fd1e0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1e5ffb640 1 -- 172.21.15.172:0/2127409577 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e000fa70 con 0x7fd1f013cb90 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1e5ffb640 1 -- 172.21.15.172:0/2127409577 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e000fc10 con 0x7fd1f013cb90 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f7915640 1 -- 172.21.15.172:0/2127409577 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f01f4590 con 0x7fd1f013cb90 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1e5ffb640 1 -- 172.21.15.172:0/2127409577 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0014440 con 0x7fd1f013cb90 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.983+0000 7fd1f7915640 1 -- 172.21.15.172:0/2127409577 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f01f4b50 con 0x7fd1f013cb90 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.984+0000 7fd1f7915640 1 -- 172.21.15.172:0/2127409577 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f00755b0 con 0x7fd1f013cb90 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.985+0000 7fd1e5ffb640 1 -- 172.21.15.172:0/2127409577 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1e0012040 con 0x7fd1f013cb90 2024-04-14T08:37:13.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.985+0000 7fd1e5ffb640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd1d406c920 0x7fd1d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:13.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.986+0000 7fd1f56b1640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd1d406c920 0x7fd1d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:13.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.986+0000 7fd1e5ffb640 1 -- 172.21.15.172:0/2127409577 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fd1e008c2a0 con 0x7fd1f013cb90 2024-04-14T08:37:13.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.986+0000 7fd1f56b1640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd1d406c920 0x7fd1d406ede0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0012b0 tx=0x7fd1ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:13.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:13.990+0000 7fd1e5ffb640 1 -- 172.21.15.172:0/2127409577 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd1e0019030 con 0x7fd1f013cb90 2024-04-14T08:37:14.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.183+0000 7fd1f7915640 1 -- 172.21.15.172:0/2127409577 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd1f0068590 con 0x7fd1f013cb90 2024-04-14T08:37:14.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.183+0000 7fd1e5ffb640 1 -- 172.21.15.172:0/2127409577 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) ==== 80+0+0 (secure 0 0 0) 0x7fd1e005a540 con 0x7fd1f013cb90 2024-04-14T08:37:14.308 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:37:14.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.185+0000 7fd1f7915640 1 -- 172.21.15.172:0/2127409577 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd1d406c920 msgr2=0x7fd1d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:14.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.185+0000 7fd1f7915640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd1d406c920 0x7fd1d406ede0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0012b0 tx=0x7fd1ec009040 comp rx=0 tx=0).stop 2024-04-14T08:37:14.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.185+0000 7fd1f7915640 1 -- 172.21.15.172:0/2127409577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f013cb90 msgr2=0x7fd1f01ee240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:14.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.185+0000 7fd1f7915640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f013cb90 0x7fd1f01ee240 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00119a0 tx=0x7fd1e0011e60 comp rx=0 tx=0).stop 2024-04-14T08:37:14.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.186+0000 7fd1f7915640 1 -- 172.21.15.172:0/2127409577 shutdown_connections 2024-04-14T08:37:14.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.186+0000 7fd1f7915640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd1d406c920 0x7fd1d406ede0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.186+0000 7fd1f7915640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1f00b1390 0x7fd1f00b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.186+0000 7fd1f7915640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f013cb90 0x7fd1f01ee240 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.186+0000 7fd1f7915640 1 --2- 172.21.15.172:0/2127409577 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1dc004d40 0x7fd1f00b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.186+0000 7fd1f7915640 1 -- 172.21.15.172:0/2127409577 >> 172.21.15.172:0/2127409577 conn(0x7fd1f006b520 msgr2=0x7fd1f00b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:14.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.186+0000 7fd1f7915640 1 -- 172.21.15.172:0/2127409577 shutdown_connections 2024-04-14T08:37:14.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.186+0000 7fd1f7915640 1 -- 172.21.15.172:0/2127409577 wait complete. 2024-04-14T08:37:14.309 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:37:14.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:14.309 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:37:14.365 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:37:14.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:14.365 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:37:14.391 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:37:14.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.532+0000 7f6a56a2d640 1 Processor -- start 2024-04-14T08:37:14.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.532+0000 7f6a56a2d640 1 -- start start 2024-04-14T08:37:14.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.533+0000 7f6a56a2d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6a500b7a60 0x7f6a500ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:14.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.533+0000 7f6a56a2d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a500d2b10 con 0x7f6a5008c4c0 2024-04-14T08:37:14.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.533+0000 7f6a56a2d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a500d0ad0 con 0x7f6a5006dc40 2024-04-14T08:37:14.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.533+0000 7f6a56a2d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f6a500d0db0 con 0x7f6a500b7a60 2024-04-14T08:37:14.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.533+0000 7f6a4f7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6a5008c4c0 0x7f6a500cd410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59260/0 (socket says 172.21.15.172:59260) 2024-04-14T08:37:14.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.533+0000 7f6a4f7fe640 1 -- 172.21.15.172:0/2460057353 learned_addr learned my addr 172.21.15.172:0/2460057353 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:14.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4ffff640 1 --2- 172.21.15.172:0/2460057353 >> v2:172.21.15.177:3300/0 conn(0x7f6a500b7a60 0x7f6a500ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:14.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3345759727 0 0) 0x7f6a500d2b10 con 0x7f6a5008c4c0 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a38003520 con 0x7f6a5008c4c0 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2421447390 0 0) 0x7f6a38003520 con 0x7f6a5008c4c0 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 >> v2:172.21.15.177:3300/0 conn(0x7f6a500b7a60 msgr2=0x7f6a500ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 --2- 172.21.15.172:0/2460057353 >> v2:172.21.15.177:3300/0 conn(0x7f6a500b7a60 0x7f6a500ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 >> v1:172.21.15.177:6789/0 conn(0x7f6a5006dc40 legacy=0x7f6a500cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a500d1090 con 0x7f6a5008c4c0 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f6a44001670 con 0x7f6a5008c4c0 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 >> v1:172.21.15.172:6789/0 conn(0x7f6a5008c4c0 legacy=0x7f6a500cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 --2- 172.21.15.172:0/2460057353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 0x7f6a38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 --2- 172.21.15.172:0/2460057353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a38004800 0x7f6a38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 --2- 172.21.15.172:0/2460057353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a380051c0 0x7f6a3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6a500d0db0 con 0x7f6a380039b0 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3800c160 con 0x7f6a380051c0 2024-04-14T08:37:14.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3800c360 con 0x7f6a38004800 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4ffff640 1 --2- 172.21.15.172:0/2460057353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 0x7f6a38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a54fca640 1 --2- 172.21.15.172:0/2460057353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a380051c0 0x7f6a3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4f7fe640 1 --2- 172.21.15.172:0/2460057353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a38004800 0x7f6a38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4ffff640 1 -- 172.21.15.172:0/2460057353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a38004800 msgr2=0x7f6a38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4ffff640 1 --2- 172.21.15.172:0/2460057353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a38004800 0x7f6a38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4ffff640 1 -- 172.21.15.172:0/2460057353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a380051c0 msgr2=0x7f6a3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4ffff640 1 --2- 172.21.15.172:0/2460057353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a380051c0 0x7f6a3800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a4ffff640 1 -- 172.21.15.172:0/2460057353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a38003520 con 0x7f6a380039b0 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a54fca640 1 --2- 172.21.15.172:0/2460057353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a380051c0 0x7f6a3800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.534+0000 7f6a54fca640 1 -- 172.21.15.172:0/2460057353 reap_dead start 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.535+0000 7f6a4ffff640 1 --2- 172.21.15.172:0/2460057353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 0x7f6a38003dd0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009ac0 tx=0x7f6a38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf55fff670da85eb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.535+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40039070 con 0x7f6a380039b0 2024-04-14T08:37:14.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.535+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a4002c6b0 con 0x7f6a380039b0 2024-04-14T08:37:14.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.535+0000 7f6a4effd640 1 -- 172.21.15.172:0/2460057353 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40009690 con 0x7f6a380039b0 2024-04-14T08:37:14.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.536+0000 7f6a56a2d640 1 -- 172.21.15.172:0/2460057353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 msgr2=0x7f6a38003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:14.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.536+0000 7f6a56a2d640 1 --2- 172.21.15.172:0/2460057353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 0x7f6a38003dd0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009ac0 tx=0x7f6a38005830 comp rx=0 tx=0).stop 2024-04-14T08:37:14.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.536+0000 7f6a56a2d640 1 -- 172.21.15.172:0/2460057353 shutdown_connections 2024-04-14T08:37:14.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.536+0000 7f6a56a2d640 1 --2- 172.21.15.172:0/2460057353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 0x7f6a38003dd0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.536+0000 7f6a56a2d640 1 -- 172.21.15.172:0/2460057353 >> 172.21.15.172:0/2460057353 conn(0x7f6a5008c000 msgr2=0x7f6a500d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:14.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.536+0000 7f6a56a2d640 1 -- 172.21.15.172:0/2460057353 shutdown_connections 2024-04-14T08:37:14.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.536+0000 7f6a56a2d640 1 -- 172.21.15.172:0/2460057353 wait complete. 2024-04-14T08:37:14.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.537+0000 7f6a56a2d640 1 Processor -- start 2024-04-14T08:37:14.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.537+0000 7f6a56a2d640 1 -- start start 2024-04-14T08:37:14.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.538+0000 7f6a56a2d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 0x7f6a50154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.538+0000 7f6a56a2d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a501f4c90 0x7f6a5009c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.538+0000 7f6a56a2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a501f5340 0x7f6a50161700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.538+0000 7f6a56a2d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6a500d2750 con 0x7f6a380039b0 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.538+0000 7f6a56a2d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a500b7e40 con 0x7f6a501f4c90 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.538+0000 7f6a56a2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6a500d28d0 con 0x7f6a501f5340 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.538+0000 7f6a4ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 0x7f6a50154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.538+0000 7f6a4ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 0x7f6a50154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46802/0 (socket says 172.21.15.172:46802) 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.538+0000 7f6a4ffff640 1 -- 172.21.15.172:0/3264533876 learned_addr learned my addr 172.21.15.172:0/3264533876 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.538+0000 7f6a4f7fe640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a501f4c90 0x7f6a5009c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.538+0000 7f6a54fca640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a501f5340 0x7f6a50161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.539+0000 7f6a4ffff640 1 -- 172.21.15.172:0/3264533876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a501f5340 msgr2=0x7f6a50161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.539+0000 7f6a4ffff640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a501f5340 0x7f6a50161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.539+0000 7f6a4ffff640 1 -- 172.21.15.172:0/3264533876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a501f4c90 msgr2=0x7f6a5009c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:14.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.539+0000 7f6a4ffff640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a501f4c90 0x7f6a5009c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.539+0000 7f6a4ffff640 1 -- 172.21.15.172:0/3264533876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5009cd10 con 0x7f6a380039b0 2024-04-14T08:37:14.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.539+0000 7f6a4ffff640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 0x7f6a50154600 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009be0 tx=0x7f6a40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:14.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.539+0000 7f6a4cff9640 1 -- 172.21.15.172:0/3264533876 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40042020 con 0x7f6a380039b0 2024-04-14T08:37:14.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.539+0000 7f6a4cff9640 1 -- 172.21.15.172:0/3264533876 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a4003a040 con 0x7f6a380039b0 2024-04-14T08:37:14.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.539+0000 7f6a56a2d640 1 -- 172.21.15.172:0/3264533876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a501f50f0 con 0x7f6a380039b0 2024-04-14T08:37:14.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.539+0000 7f6a4cff9640 1 -- 172.21.15.172:0/3264533876 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40039070 con 0x7f6a380039b0 2024-04-14T08:37:14.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.539+0000 7f6a56a2d640 1 -- 172.21.15.172:0/3264533876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5009cfa0 con 0x7f6a380039b0 2024-04-14T08:37:14.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.540+0000 7f6a56a2d640 1 -- 172.21.15.172:0/3264533876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a50075490 con 0x7f6a380039b0 2024-04-14T08:37:14.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.540+0000 7f6a4cff9640 1 -- 172.21.15.172:0/3264533876 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a40045020 con 0x7f6a380039b0 2024-04-14T08:37:14.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.541+0000 7f6a4cff9640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6a3c06c8f0 0x7f6a3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:14.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.541+0000 7f6a4f7fe640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6a3c06c8f0 0x7f6a3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:14.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.541+0000 7f6a4cff9640 1 -- 172.21.15.172:0/3264533876 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f6a400ad000 con 0x7f6a380039b0 2024-04-14T08:37:14.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.541+0000 7f6a4f7fe640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6a3c06c8f0 0x7f6a3c06edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6a44001cc0 tx=0x7f6a44007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:14.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.545+0000 7f6a4cff9640 1 -- 172.21.15.172:0/3264533876 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6a4004c070 con 0x7f6a380039b0 2024-04-14T08:37:14.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.846+0000 7f6a56a2d640 1 -- 172.21.15.172:0/3264533876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a500b71c0 con 0x7f6a380039b0 2024-04-14T08:37:14.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.846+0000 7f6a4cff9640 1 -- 172.21.15.172:0/3264533876 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5192 (secure 0 0 0) 0x7f6a4007ac50 con 0x7f6a380039b0 2024-04-14T08:37:14.848 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:14.848 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:37:01.115031+0000","modified":"2024-04-14T08:37:03.119917+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6753},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6753":{"gid":6753,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6837/2134832221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2134832221},{"type":"v1","addr":"172.21.15.172:6837","nonce":2134832221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6753,"qdb_cluster":[6753]},"id":5}]} 2024-04-14T08:37:14.848 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 39 2024-04-14T08:37:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.848+0000 7f6a56a2d640 1 -- 172.21.15.172:0/3264533876 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6a3c06c8f0 msgr2=0x7f6a3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.848+0000 7f6a56a2d640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6a3c06c8f0 0x7f6a3c06edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6a44001cc0 tx=0x7f6a44007040 comp rx=0 tx=0).stop 2024-04-14T08:37:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.848+0000 7f6a56a2d640 1 -- 172.21.15.172:0/3264533876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 msgr2=0x7f6a50154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.848+0000 7f6a56a2d640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 0x7f6a50154600 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009be0 tx=0x7f6a40004730 comp rx=0 tx=0).stop 2024-04-14T08:37:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.848+0000 7f6a56a2d640 1 -- 172.21.15.172:0/3264533876 shutdown_connections 2024-04-14T08:37:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.848+0000 7f6a56a2d640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6a3c06c8f0 0x7f6a3c06edb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.848+0000 7f6a56a2d640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a501f5340 0x7f6a50161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.848+0000 7f6a56a2d640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a501f4c90 0x7f6a5009c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.848+0000 7f6a56a2d640 1 --2- 172.21.15.172:0/3264533876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a380039b0 0x7f6a50154600 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.848+0000 7f6a56a2d640 1 -- 172.21.15.172:0/3264533876 >> 172.21.15.172:0/3264533876 conn(0x7f6a5008c000 msgr2=0x7f6a500cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.848+0000 7f6a56a2d640 1 -- 172.21.15.172:0/3264533876 shutdown_connections 2024-04-14T08:37:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:14.848+0000 7f6a56a2d640 1 -- 172.21.15.172:0/3264533876 wait complete. 2024-04-14T08:37:14.864 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-14T08:37:14.998 INFO:teuthology.orchestra.run.smithi172.stdout:{"message":"","return_code":0} 2024-04-14T08:37:14.999 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-14T08:37:14.999 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:37:15.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.101+0000 7f2c3c2de640 1 Processor -- start 2024-04-14T08:37:15.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.101+0000 7f2c3c2de640 1 -- start start 2024-04-14T08:37:15.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.102+0000 7f2c3c2de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2c34162710 0x7f2c340d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.102+0000 7f2c3c2de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2c34162ba0 0x7f2c340d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.102+0000 7f2c3c2de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c340cefb0 con 0x7f2c3415e430 2024-04-14T08:37:15.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.102+0000 7f2c3c2de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2c340cf290 con 0x7f2c34162ba0 2024-04-14T08:37:15.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.102+0000 7f2c3c2de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2c340cf570 con 0x7f2c34162710 2024-04-14T08:37:15.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.102+0000 7f2c3a87b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2c34162ba0 0x7f2c340d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.102+0000 7f2c3a07a640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2c3415e430 0x7f2c34162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59262/0 (socket says 172.21.15.172:59262) 2024-04-14T08:37:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.102+0000 7f2c3a07a640 1 -- 172.21.15.172:0/2476605576 learned_addr learned my addr 172.21.15.172:0/2476605576 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.103+0000 7f2c3a87b640 1 -- 172.21.15.172:0/2476605576 >> v2:172.21.15.177:3300/0 conn(0x7f2c34162710 msgr2=0x7f2c340d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.103+0000 7f2c3a87b640 1 --2- 172.21.15.172:0/2476605576 >> v2:172.21.15.177:3300/0 conn(0x7f2c34162710 0x7f2c340d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.103+0000 7f2c3a87b640 1 -- 172.21.15.172:0/2476605576 >> v1:172.21.15.172:6789/0 conn(0x7f2c3415e430 legacy=0x7f2c34162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.103+0000 7f2c3a87b640 1 -- 172.21.15.172:0/2476605576 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c340cf850 con 0x7f2c34162ba0 2024-04-14T08:37:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.103+0000 7f2c39078640 1 -- 172.21.15.172:0/2476605576 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3280764509 0 0) 0x7f2c340cefb0 con 0x7f2c3415e430 2024-04-14T08:37:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.103+0000 7f2c3a87b640 1 --2- 172.21.15.172:0/2476605576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c34162ba0 0x7f2c340d2330 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f2c3408dae0 tx=0x7f2c2c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642751ba12317cbe server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.104+0000 7f2c39078640 1 -- 172.21.15.172:0/2476605576 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c039070 con 0x7f2c34162ba0 2024-04-14T08:37:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.104+0000 7f2c39078640 1 -- 172.21.15.172:0/2476605576 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c02cc80 con 0x7f2c34162ba0 2024-04-14T08:37:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.104+0000 7f2c39078640 1 -- 172.21.15.172:0/2476605576 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c034660 con 0x7f2c34162ba0 2024-04-14T08:37:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.104+0000 7f2c3c2de640 1 -- 172.21.15.172:0/2476605576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c34162ba0 msgr2=0x7f2c340d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.104+0000 7f2c3c2de640 1 --2- 172.21.15.172:0/2476605576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c34162ba0 0x7f2c340d2330 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f2c3408dae0 tx=0x7f2c2c02bed0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.104+0000 7f2c3c2de640 1 -- 172.21.15.172:0/2476605576 shutdown_connections 2024-04-14T08:37:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.104+0000 7f2c3c2de640 1 --2- 172.21.15.172:0/2476605576 >> v2:172.21.15.177:3300/0 conn(0x7f2c34162710 0x7f2c340d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.104+0000 7f2c3c2de640 1 --2- 172.21.15.172:0/2476605576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c34162ba0 0x7f2c340d2330 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.104+0000 7f2c3c2de640 1 -- 172.21.15.172:0/2476605576 >> 172.21.15.172:0/2476605576 conn(0x7f2c3409d560 msgr2=0x7f2c340d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.105+0000 7f2c3c2de640 1 -- 172.21.15.172:0/2476605576 shutdown_connections 2024-04-14T08:37:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.105+0000 7f2c3c2de640 1 -- 172.21.15.172:0/2476605576 wait complete. 2024-04-14T08:37:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.106+0000 7f2c3c2de640 1 Processor -- start 2024-04-14T08:37:15.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.106+0000 7f2c3c2de640 1 -- start start 2024-04-14T08:37:15.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c3c2de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c3415e430 0x7f2c341527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c3c2de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c34162710 0x7f2c340b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c3c2de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c34162ba0 0x7f2c340b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c3c2de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2c340d4e20 con 0x7f2c34162710 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c3c2de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3415e810 con 0x7f2c3415e430 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c3c2de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2c340d0050 con 0x7f2c34162ba0 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c39879640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c34162710 0x7f2c340b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c39879640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c34162710 0x7f2c340b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46818/0 (socket says 172.21.15.172:46818) 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c39879640 1 -- 172.21.15.172:0/615881968 learned_addr learned my addr 172.21.15.172:0/615881968 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c3a07a640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c3415e430 0x7f2c341527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c3a87b640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c34162ba0 0x7f2c340b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c39879640 1 -- 172.21.15.172:0/615881968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c34162ba0 msgr2=0x7f2c340b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c39879640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c34162ba0 0x7f2c340b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c39879640 1 -- 172.21.15.172:0/615881968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c3415e430 msgr2=0x7f2c341527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c39879640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c3415e430 0x7f2c341527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c39879640 1 -- 172.21.15.172:0/615881968 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c34152ce0 con 0x7f2c34162710 2024-04-14T08:37:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c39879640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c34162710 0x7f2c340b4bf0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400b410 tx=0x7f2c2400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c2affd640 1 -- 172.21.15.172:0/615881968 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24015020 con 0x7f2c34162710 2024-04-14T08:37:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c2affd640 1 -- 172.21.15.172:0/615881968 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c24011850 con 0x7f2c34162710 2024-04-14T08:37:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c3c2de640 1 -- 172.21.15.172:0/615881968 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c340b9c70 con 0x7f2c34162710 2024-04-14T08:37:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c2affd640 1 -- 172.21.15.172:0/615881968 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24011c40 con 0x7f2c34162710 2024-04-14T08:37:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.108+0000 7f2c3c2de640 1 -- 172.21.15.172:0/615881968 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c340b5a20 con 0x7f2c34162710 2024-04-14T08:37:15.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.109+0000 7f2c3c2de640 1 -- 172.21.15.172:0/615881968 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c34068590 con 0x7f2c34162710 2024-04-14T08:37:15.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.110+0000 7f2c2affd640 1 -- 172.21.15.172:0/615881968 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c24018020 con 0x7f2c34162710 2024-04-14T08:37:15.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.111+0000 7f2c2affd640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c1806c8f0 0x7f2c1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.111+0000 7f2c2affd640 1 -- 172.21.15.172:0/615881968 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f2c2408bbe0 con 0x7f2c34162710 2024-04-14T08:37:15.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.111+0000 7f2c3a07a640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c1806c8f0 0x7f2c1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.111+0000 7f2c3a07a640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c1806c8f0 0x7f2c1806edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2c30000f60 tx=0x7f2c30007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:15.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.115+0000 7f2c2affd640 1 -- 172.21.15.172:0/615881968 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2c2401f020 con 0x7f2c34162710 2024-04-14T08:37:15.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.354+0000 7f2c3c2de640 1 -- 172.21.15.172:0/615881968 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2c340d18a0 con 0x7f2c34162710 2024-04-14T08:37:15.356 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:37:15.355+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:37:15.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.361+0000 7f2c2affd640 1 -- 172.21.15.172:0/615881968 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) ==== 161+0+0 (secure 0 0 0) 0x7f2c34152ce0 con 0x7f2c34162710 2024-04-14T08:37:15.362 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:37:15.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.363+0000 7f2c3c2de640 1 -- 172.21.15.172:0/615881968 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c1806c8f0 msgr2=0x7f2c1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.363+0000 7f2c3c2de640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c1806c8f0 0x7f2c1806edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2c30000f60 tx=0x7f2c30007c00 comp rx=0 tx=0).stop 2024-04-14T08:37:15.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.363+0000 7f2c3c2de640 1 -- 172.21.15.172:0/615881968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c34162710 msgr2=0x7f2c340b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.363+0000 7f2c3c2de640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c34162710 0x7f2c340b4bf0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400b410 tx=0x7f2c2400b8d0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.363+0000 7f2c3c2de640 1 -- 172.21.15.172:0/615881968 shutdown_connections 2024-04-14T08:37:15.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.363+0000 7f2c3c2de640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c1806c8f0 0x7f2c1806edb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.363+0000 7f2c3c2de640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c34162ba0 0x7f2c340b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.363+0000 7f2c3c2de640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c34162710 0x7f2c340b4bf0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.363+0000 7f2c3c2de640 1 --2- 172.21.15.172:0/615881968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c3415e430 0x7f2c341527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.363+0000 7f2c3c2de640 1 -- 172.21.15.172:0/615881968 >> 172.21.15.172:0/615881968 conn(0x7f2c3409d560 msgr2=0x7f2c3406c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:15.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.363+0000 7f2c3c2de640 1 -- 172.21.15.172:0/615881968 shutdown_connections 2024-04-14T08:37:15.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.363+0000 7f2c3c2de640 1 -- 172.21.15.172:0/615881968 wait complete. 2024-04-14T08:37:15.367 INFO:tasks.ceph.mds.c.smithi172.stderr: -4784> 2024-04-14T08:37:03.737+0000 7f13d59f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:37:15.379 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-14T08:37:15.401 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:37:15.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.421+0000 7f8afc072f40 1 Processor -- start 2024-04-14T08:37:15.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.421+0000 7f8afc072f40 1 -- start start 2024-04-14T08:37:15.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8afc072f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c2158b8180 0x55c21589bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8afc072f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2159436d0 0x55c21586db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8afc072f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c215940e20 0x55c215944ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8afc072f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c2158153b0 con 0x55c2159436d0 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8afc072f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c21586cb60 con 0x55c215940e20 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8afc072f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55c215898480 con 0x55c2158b8180 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8af914e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2159436d0 0x55c21586db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8af914e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2159436d0 0x55c21586db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46836/0 (socket says 172.21.15.172:46836) 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8af914e640 1 -- 172.21.15.172:0/3486713973 learned_addr learned my addr 172.21.15.172:0/3486713973 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8af994f640 1 --2- 172.21.15.172:0/3486713973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c2158b8180 0x55c21589bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8afa150640 1 --2- 172.21.15.172:0/3486713973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c215940e20 0x55c215944ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8af914e640 1 -- 172.21.15.172:0/3486713973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c2158b8180 msgr2=0x55c21589bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8af914e640 1 --2- 172.21.15.172:0/3486713973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c2158b8180 0x55c21589bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8af914e640 1 -- 172.21.15.172:0/3486713973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c215940e20 msgr2=0x55c215944ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.422+0000 7f8af914e640 1 --2- 172.21.15.172:0/3486713973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c215940e20 0x55c215944ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8af914e640 1 -- 172.21.15.172:0/3486713973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c215942580 con 0x55c2159436d0 2024-04-14T08:37:15.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8afa150640 1 --2- 172.21.15.172:0/3486713973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c215940e20 0x55c215944ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:15.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8af914e640 1 --2- 172.21.15.172:0/3486713973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2159436d0 0x55c21586db00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400a3f0 tx=0x7f8ae400a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de42b6e463034e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:15.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8af3fff640 1 -- 172.21.15.172:0/3486713973 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4010070 con 0x55c2159436d0 2024-04-14T08:37:15.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8af3fff640 1 -- 172.21.15.172:0/3486713973 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae400c8f0 con 0x55c2159436d0 2024-04-14T08:37:15.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8af3fff640 1 -- 172.21.15.172:0/3486713973 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400cce0 con 0x55c2159436d0 2024-04-14T08:37:15.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8afc072f40 1 -- 172.21.15.172:0/3486713973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2159436d0 msgr2=0x55c21586db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8afc072f40 1 --2- 172.21.15.172:0/3486713973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2159436d0 0x55c21586db00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400a3f0 tx=0x7f8ae400a8b0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8afc072f40 1 -- 172.21.15.172:0/3486713973 shutdown_connections 2024-04-14T08:37:15.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8afc072f40 1 --2- 172.21.15.172:0/3486713973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c215940e20 0x55c215944ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8afc072f40 1 --2- 172.21.15.172:0/3486713973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2159436d0 0x55c21586db00 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8afc072f40 1 --2- 172.21.15.172:0/3486713973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c2158b8180 0x55c21589bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8afc072f40 1 -- 172.21.15.172:0/3486713973 >> 172.21.15.172:0/3486713973 conn(0x55c215898c70 msgr2=0x55c2158b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:15.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8afc072f40 1 -- 172.21.15.172:0/3486713973 shutdown_connections 2024-04-14T08:37:15.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.423+0000 7f8afc072f40 1 -- 172.21.15.172:0/3486713973 wait complete. 2024-04-14T08:37:15.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.424+0000 7f8afc072f40 1 Processor -- start 2024-04-14T08:37:15.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.424+0000 7f8afc072f40 1 -- start start 2024-04-14T08:37:15.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8afc072f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c2158b8180 0x55c2159decd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8afc072f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c215940e20 0x55c21593d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8afc072f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2159436d0 0x55c215944c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8afc072f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c215946bb0 con 0x55c215940e20 2024-04-14T08:37:15.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8afc072f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c2158986d0 con 0x55c2159436d0 2024-04-14T08:37:15.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8afc072f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55c2159468a0 con 0x55c2158b8180 2024-04-14T08:37:15.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af914e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c215940e20 0x55c21593d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af914e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c215940e20 0x55c21593d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46850/0 (socket says 172.21.15.172:46850) 2024-04-14T08:37:15.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af914e640 1 -- 172.21.15.172:0/952016159 learned_addr learned my addr 172.21.15.172:0/952016159 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:15.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8afa150640 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2159436d0 0x55c215944c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af994f640 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c2158b8180 0x55c2159decd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af914e640 1 -- 172.21.15.172:0/952016159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c2158b8180 msgr2=0x55c2159decd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af914e640 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c2158b8180 0x55c2159decd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af914e640 1 -- 172.21.15.172:0/952016159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2159436d0 msgr2=0x55c215944c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af914e640 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2159436d0 0x55c215944c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af914e640 1 -- 172.21.15.172:0/952016159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2158ae2b0 con 0x55c215940e20 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af914e640 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c215940e20 0x55c21593d3f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400a8f0 tx=0x7f8ae4002f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af2ffd640 1 -- 172.21.15.172:0/952016159 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4025070 con 0x55c215940e20 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af2ffd640 1 -- 172.21.15.172:0/952016159 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae4015960 con 0x55c215940e20 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8afc072f40 1 -- 172.21.15.172:0/952016159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c2158b69e0 con 0x55c215940e20 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8af2ffd640 1 -- 172.21.15.172:0/952016159 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4010050 con 0x55c215940e20 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.425+0000 7f8afc072f40 1 -- 172.21.15.172:0/952016159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21593e2f0 con 0x55c215940e20 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.427+0000 7f8af2ffd640 1 -- 172.21.15.172:0/952016159 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ae4028070 con 0x55c215940e20 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.427+0000 7f8af2ffd640 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8adc06b2d0 0x7f8adc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.427+0000 7f8af2ffd640 1 -- 172.21.15.172:0/952016159 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f8ae408bf00 con 0x55c215940e20 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.427+0000 7f8af994f640 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8adc06b2d0 0x7f8adc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.427+0000 7f8afc072f40 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55c21593dbd0 0x55c2158b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.427+0000 7f8afc072f40 1 -- 172.21.15.172:0/952016159 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75) -- 0x55c2158b7720 con 0x55c21593dbd0 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.428+0000 7f8afa150640 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55c21593dbd0 0x55c2158b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.428+0000 7f8af994f640 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8adc06b2d0 0x7f8adc06d790 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8aec052000 tx=0x7f8aec04d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.428+0000 7f8afa150640 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55c21593dbd0 0x55c2158b71e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.431+0000 7f8afa150640 1 -- 172.21.15.172:0/952016159 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v75'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8af4007700 con 0x55c21593dbd0 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.431+0000 7f8afc072f40 1 -- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55c21593dbd0 msgr2=0x55c2158b71e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.431+0000 7f8afc072f40 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55c21593dbd0 0x55c2158b71e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.431+0000 7f8afc072f40 1 -- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8adc06b2d0 msgr2=0x7f8adc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.431+0000 7f8afc072f40 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8adc06b2d0 0x7f8adc06d790 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8aec052000 tx=0x7f8aec04d000 comp rx=0 tx=0).stop 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.431+0000 7f8afc072f40 1 -- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c215940e20 msgr2=0x55c21593d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.431+0000 7f8afc072f40 1 --2- 172.21.15.172:0/952016159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c215940e20 0x55c21593d3f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400a8f0 tx=0x7f8ae4002f30 comp rx=0 tx=0).stop 2024-04-14T08:37:15.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.431+0000 7f8afa150640 1 -- 172.21.15.172:0/952016159 reap_dead start 2024-04-14T08:37:15.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.431+0000 7f8afc072f40 1 -- 172.21.15.172:0/952016159 shutdown_connections 2024-04-14T08:37:15.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.431+0000 7f8afc072f40 1 -- 172.21.15.172:0/952016159 >> 172.21.15.172:0/952016159 conn(0x55c215898c70 msgr2=0x55c21594de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:15.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.431+0000 7f8afc072f40 1 -- 172.21.15.172:0/952016159 shutdown_connections 2024-04-14T08:37:15.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.431+0000 7f8afc072f40 1 -- 172.21.15.172:0/952016159 wait complete. 2024-04-14T08:37:15.438 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-14T08:37:15.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.581+0000 7fc6af0de640 1 Processor -- start 2024-04-14T08:37:15.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.581+0000 7fc6af0de640 1 -- start start 2024-04-14T08:37:15.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6af0de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6a8162a80 0x7fc6a80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6af0de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc6a8161f10 0x7fc6a80b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6af0de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6a8162e60 con 0x7fc6a806f810 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6af0de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc6a815f6d0 con 0x7fc6a8162a80 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6af0de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc6a815eac0 con 0x7fc6a8161f10 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6ace7a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6a8162a80 0x7fc6a80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6ace7a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6a8162a80 0x7fc6a80b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46860/0 (socket says 172.21.15.172:46860) 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6ace7a640 1 -- 172.21.15.172:0/1402988623 learned_addr learned my addr 172.21.15.172:0/1402988623 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6a7fff640 1 --2- 172.21.15.172:0/1402988623 >> v2:172.21.15.177:3300/0 conn(0x7fc6a8161f10 0x7fc6a80b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6ace7a640 1 -- 172.21.15.172:0/1402988623 >> v2:172.21.15.177:3300/0 conn(0x7fc6a8161f10 msgr2=0x7fc6a80b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6ace7a640 1 --2- 172.21.15.172:0/1402988623 >> v2:172.21.15.177:3300/0 conn(0x7fc6a8161f10 0x7fc6a80b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6ace7a640 1 -- 172.21.15.172:0/1402988623 >> v1:172.21.15.177:6789/0 conn(0x7fc6a806f810 legacy=0x7fc6a806e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.582+0000 7fc6ace7a640 1 -- 172.21.15.172:0/1402988623 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a815eda0 con 0x7fc6a8162a80 2024-04-14T08:37:15.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.583+0000 7fc6ace7a640 1 --2- 172.21.15.172:0/1402988623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a8162a80 0x7fc6a80b3940 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80b57e0 tx=0x7fc69c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb35a2f8900275c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:15.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.583+0000 7fc6a77fe640 1 -- 172.21.15.172:0/1402988623 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c034070 con 0x7fc6a8162a80 2024-04-14T08:37:15.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.583+0000 7fc6a77fe640 1 -- 172.21.15.172:0/1402988623 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc69c030860 con 0x7fc6a8162a80 2024-04-14T08:37:15.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.583+0000 7fc6a77fe640 1 -- 172.21.15.172:0/1402988623 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c030c30 con 0x7fc6a8162a80 2024-04-14T08:37:15.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.584+0000 7fc6af0de640 1 -- 172.21.15.172:0/1402988623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a8162a80 msgr2=0x7fc6a80b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.584+0000 7fc6af0de640 1 --2- 172.21.15.172:0/1402988623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a8162a80 0x7fc6a80b3940 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80b57e0 tx=0x7fc69c02b9a0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.584+0000 7fc6af0de640 1 -- 172.21.15.172:0/1402988623 shutdown_connections 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.584+0000 7fc6af0de640 1 --2- 172.21.15.172:0/1402988623 >> v2:172.21.15.177:3300/0 conn(0x7fc6a8161f10 0x7fc6a80b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.584+0000 7fc6af0de640 1 --2- 172.21.15.172:0/1402988623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a8162a80 0x7fc6a80b3940 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.584+0000 7fc6af0de640 1 -- 172.21.15.172:0/1402988623 >> 172.21.15.172:0/1402988623 conn(0x7fc6a80b29b0 msgr2=0x7fc6a80d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.584+0000 7fc6af0de640 1 -- 172.21.15.172:0/1402988623 shutdown_connections 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.584+0000 7fc6af0de640 1 -- 172.21.15.172:0/1402988623 wait complete. 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.585+0000 7fc6af0de640 1 Processor -- start 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.585+0000 7fc6af0de640 1 -- start start 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6af0de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a806f810 0x7fc6a80ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6af0de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6a8161f10 0x7fc6a80aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6af0de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6a8162a80 0x7fc6a81e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6af0de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a8067310 con 0x7fc6a806f810 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6af0de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a815f360 con 0x7fc6a8161f10 2024-04-14T08:37:15.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6af0de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc6a806c630 con 0x7fc6a8162a80 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6ace7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a806f810 0x7fc6a80ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6ace7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a806f810 0x7fc6a80ae930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46866/0 (socket says 172.21.15.172:46866) 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6ace7a640 1 -- 172.21.15.172:0/1728519941 learned_addr learned my addr 172.21.15.172:0/1728519941 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6ad67b640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6a8162a80 0x7fc6a81e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6a7fff640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6a8161f10 0x7fc6a80aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6ace7a640 1 -- 172.21.15.172:0/1728519941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6a8162a80 msgr2=0x7fc6a81e7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6ace7a640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6a8162a80 0x7fc6a81e7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6ace7a640 1 -- 172.21.15.172:0/1728519941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6a8161f10 msgr2=0x7fc6a80aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6ace7a640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6a8161f10 0x7fc6a80aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6ace7a640 1 -- 172.21.15.172:0/1728519941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a8142350 con 0x7fc6a806f810 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6ace7a640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a806f810 0x7fc6a80ae930 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc69c02b9e0 tx=0x7fc69c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6a57fa640 1 -- 172.21.15.172:0/1728519941 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c034040 con 0x7fc6a806f810 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6a57fa640 1 -- 172.21.15.172:0/1728519941 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc69c039d90 con 0x7fc6a806f810 2024-04-14T08:37:15.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6af0de640 1 -- 172.21.15.172:0/1728519941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a8142630 con 0x7fc6a806f810 2024-04-14T08:37:15.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.586+0000 7fc6a57fa640 1 -- 172.21.15.172:0/1728519941 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c043610 con 0x7fc6a806f810 2024-04-14T08:37:15.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.587+0000 7fc6af0de640 1 -- 172.21.15.172:0/1728519941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a8155c60 con 0x7fc6a806f810 2024-04-14T08:37:15.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.587+0000 7fc6af0de640 1 -- 172.21.15.172:0/1728519941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a80b3940 con 0x7fc6a806f810 2024-04-14T08:37:15.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.588+0000 7fc6a57fa640 1 -- 172.21.15.172:0/1728519941 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc69c04a060 con 0x7fc6a806f810 2024-04-14T08:37:15.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.588+0000 7fc6a57fa640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc68c06c8f0 0x7fc68c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:15.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.589+0000 7fc6a7fff640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc68c06c8f0 0x7fc68c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:15.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.589+0000 7fc6a57fa640 1 -- 172.21.15.172:0/1728519941 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fc69c0add60 con 0x7fc6a806f810 2024-04-14T08:37:15.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.589+0000 7fc6a7fff640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc68c06c8f0 0x7fc68c06edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8145f30 tx=0x7fc694008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:15.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.593+0000 7fc6a57fa640 1 -- 172.21.15.172:0/1728519941 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc6a80b3940 con 0x7fc6a806f810 2024-04-14T08:37:15.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:15.807+0000 7fc6af0de640 1 -- 172.21.15.172:0/1728519941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc6a80d0c60 con 0x7fc6a806f810 2024-04-14T08:37:16.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.363+0000 7fc6a57fa640 1 -- 172.21.15.172:0/1728519941 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v41) ==== 162+0+0 (secure 0 0 0) 0x7fc6a80d0c60 con 0x7fc6a806f810 2024-04-14T08:37:16.364 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-14T08:37:16.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.365+0000 7fc6af0de640 1 -- 172.21.15.172:0/1728519941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc68c06c8f0 msgr2=0x7fc68c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:16.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.365+0000 7fc6af0de640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc68c06c8f0 0x7fc68c06edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8145f30 tx=0x7fc694008040 comp rx=0 tx=0).stop 2024-04-14T08:37:16.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.365+0000 7fc6af0de640 1 -- 172.21.15.172:0/1728519941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a806f810 msgr2=0x7fc6a80ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:16.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.365+0000 7fc6af0de640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a806f810 0x7fc6a80ae930 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc69c02b9e0 tx=0x7fc69c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.365+0000 7fc6af0de640 1 -- 172.21.15.172:0/1728519941 shutdown_connections 2024-04-14T08:37:16.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.365+0000 7fc6af0de640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc68c06c8f0 0x7fc68c06edb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.365+0000 7fc6af0de640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6a8162a80 0x7fc6a81e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.365+0000 7fc6af0de640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6a8161f10 0x7fc6a80aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.365+0000 7fc6af0de640 1 --2- 172.21.15.172:0/1728519941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6a806f810 0x7fc6a80ae930 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.365+0000 7fc6af0de640 1 -- 172.21.15.172:0/1728519941 >> 172.21.15.172:0/1728519941 conn(0x7fc6a80b29b0 msgr2=0x7fc6a80b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:16.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.365+0000 7fc6af0de640 1 -- 172.21.15.172:0/1728519941 shutdown_connections 2024-04-14T08:37:16.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.365+0000 7fc6af0de640 1 -- 172.21.15.172:0/1728519941 wait complete. 2024-04-14T08:37:16.381 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:37:16.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.483+0000 7f3f178de640 1 Processor -- start 2024-04-14T08:37:16.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.483+0000 7f3f178de640 1 -- start start 2024-04-14T08:37:16.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.484+0000 7f3f178de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f100b1df0 0x7f3f100b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:16.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.484+0000 7f3f178de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1015c770 con 0x7f3f100b4040 2024-04-14T08:37:16.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.484+0000 7f3f178de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f100b3430 con 0x7f3f100b21d0 2024-04-14T08:37:16.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.484+0000 7f3f178de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3f100b3700 con 0x7f3f100b1df0 2024-04-14T08:37:16.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.484+0000 7f3f15e7b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3f100b4040 0x7f3f100b44b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59274/0 (socket says 172.21.15.172:59274) 2024-04-14T08:37:16.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.484+0000 7f3f15e7b640 1 -- 172.21.15.172:0/64751179 learned_addr learned my addr 172.21.15.172:0/64751179 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:16.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.484+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2205324450 0 0) 0x7f3f1015c770 con 0x7f3f100b4040 2024-04-14T08:37:16.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.484+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3efc003520 con 0x7f3f100b4040 2024-04-14T08:37:16.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.484+0000 7f3f1567a640 1 --2- 172.21.15.172:0/64751179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f100b1df0 0x7f3f100b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2072873591 0 0) 0x7f3efc003520 con 0x7f3f100b4040 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f100b1df0 msgr2=0x7f3f100b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 --2- 172.21.15.172:0/64751179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f100b1df0 0x7f3f100b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 >> v1:172.21.15.177:6789/0 conn(0x7f3f100b21d0 legacy=0x7f3f1015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f100b39e0 con 0x7f3f100b4040 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f3f08001670 con 0x7f3f100b4040 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 >> v1:172.21.15.172:6789/0 conn(0x7f3f100b4040 legacy=0x7f3f100b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 --2- 172.21.15.172:0/64751179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3efc0039e0 0x7f3efc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 --2- 172.21.15.172:0/64751179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3efc004800 0x7f3efc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 --2- 172.21.15.172:0/64751179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f100b1df0 0x7f3efc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3f100b3700 con 0x7f3efc004800 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc00baa0 con 0x7f3efc0039e0 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3efc00bca0 con 0x7f3f100b1df0 2024-04-14T08:37:16.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f14e79640 1 --2- 172.21.15.172:0/64751179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3efc004800 0x7f3efc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f1567a640 1 --2- 172.21.15.172:0/64751179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3efc0039e0 0x7f3efc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f15e7b640 1 --2- 172.21.15.172:0/64751179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f100b1df0 0x7f3efc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f14e79640 1 -- 172.21.15.172:0/64751179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f100b1df0 msgr2=0x7f3efc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f14e79640 1 --2- 172.21.15.172:0/64751179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f100b1df0 0x7f3efc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f14e79640 1 -- 172.21.15.172:0/64751179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3efc0039e0 msgr2=0x7f3efc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f14e79640 1 --2- 172.21.15.172:0/64751179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3efc0039e0 0x7f3efc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.485+0000 7f3f14e79640 1 -- 172.21.15.172:0/64751179 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc003520 con 0x7f3efc004800 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.486+0000 7f3f14e79640 1 --2- 172.21.15.172:0/64751179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3efc004800 0x7f3efc004c80 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3f00001720 tx=0x7f3efc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbe7e87a7d08d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.486+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00001370 con 0x7f3efc004800 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.486+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f000052d0 con 0x7f3efc004800 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.486+0000 7f3f07fff640 1 -- 172.21.15.172:0/64751179 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00003830 con 0x7f3efc004800 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.486+0000 7f3f178de640 1 -- 172.21.15.172:0/64751179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3efc004800 msgr2=0x7f3efc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.486+0000 7f3f178de640 1 --2- 172.21.15.172:0/64751179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3efc004800 0x7f3efc004c80 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3f00001720 tx=0x7f3efc005830 comp rx=0 tx=0).stop 2024-04-14T08:37:16.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.486+0000 7f3f15e7b640 1 -- 172.21.15.172:0/64751179 reap_dead start 2024-04-14T08:37:16.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.486+0000 7f3f178de640 1 -- 172.21.15.172:0/64751179 shutdown_connections 2024-04-14T08:37:16.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.486+0000 7f3f178de640 1 -- 172.21.15.172:0/64751179 >> 172.21.15.172:0/64751179 conn(0x7f3f1006ec30 msgr2=0x7f3f1006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:16.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.487+0000 7f3f178de640 1 -- 172.21.15.172:0/64751179 shutdown_connections 2024-04-14T08:37:16.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.487+0000 7f3f178de640 1 -- 172.21.15.172:0/64751179 wait complete. 2024-04-14T08:37:16.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.488+0000 7f3f178de640 1 Processor -- start 2024-04-14T08:37:16.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.488+0000 7f3f178de640 1 -- start start 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.488+0000 7f3f178de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3efc004800 0x7f3f101f0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.488+0000 7f3f178de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f1013e4f0 0x7f3f1013c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.488+0000 7f3f178de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f1013e970 0x7f3f1013c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.488+0000 7f3f178de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1015b7f0 con 0x7f3f1013e970 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.488+0000 7f3f178de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1015bb40 con 0x7f3f1013e4f0 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.488+0000 7f3f178de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3f100b2ef0 con 0x7f3efc004800 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f15e7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f1013e970 0x7f3f1013c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f15e7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f1013e970 0x7f3f1013c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46886/0 (socket says 172.21.15.172:46886) 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f15e7b640 1 -- 172.21.15.172:0/2012560129 learned_addr learned my addr 172.21.15.172:0/2012560129 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f14e79640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f1013e4f0 0x7f3f1013c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f1567a640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3efc004800 0x7f3f101f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f15e7b640 1 -- 172.21.15.172:0/2012560129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3efc004800 msgr2=0x7f3f101f0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f15e7b640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3efc004800 0x7f3f101f0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f15e7b640 1 -- 172.21.15.172:0/2012560129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f1013e4f0 msgr2=0x7f3f1013c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f15e7b640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f1013e4f0 0x7f3f1013c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f15e7b640 1 -- 172.21.15.172:0/2012560129 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f100ce6c0 con 0x7f3f1013e970 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f1567a640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3efc004800 0x7f3f101f0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f15e7b640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f1013e970 0x7f3f1013c9c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3f100b3fa0 tx=0x7f3f08007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f05ffb640 1 -- 172.21.15.172:0/2012560129 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0801a070 con 0x7f3f1013e970 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f05ffb640 1 -- 172.21.15.172:0/2012560129 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f0800f680 con 0x7f3f1013e970 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f178de640 1 -- 172.21.15.172:0/2012560129 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f100cee00 con 0x7f3f1013e970 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f05ffb640 1 -- 172.21.15.172:0/2012560129 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800fa50 con 0x7f3f1013e970 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.489+0000 7f3f178de640 1 -- 172.21.15.172:0/2012560129 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f100cf3c0 con 0x7f3f1013e970 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.490+0000 7f3f178de640 1 -- 172.21.15.172:0/2012560129 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f10068520 con 0x7f3f1013e970 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.491+0000 7f3f05ffb640 1 -- 172.21.15.172:0/2012560129 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f0801d070 con 0x7f3f1013e970 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.491+0000 7f3f05ffb640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ef406c8f0 0x7f3ef406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.491+0000 7f3f05ffb640 1 -- 172.21.15.172:0/2012560129 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f3f0808b260 con 0x7f3f1013e970 2024-04-14T08:37:16.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.491+0000 7f3f1567a640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ef406c8f0 0x7f3ef406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:16.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.491+0000 7f3f1567a640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ef406c8f0 0x7f3ef406edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3f1013b3e0 tx=0x7f3f0c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:16.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.496+0000 7f3f05ffb640 1 -- 172.21.15.172:0/2012560129 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3f080594b0 con 0x7f3f1013e970 2024-04-14T08:37:16.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.708+0000 7f3f178de640 1 -- 172.21.15.172:0/2012560129 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f101f0d80 con 0x7f3f1013e970 2024-04-14T08:37:16.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.709+0000 7f3f05ffb640 1 -- 172.21.15.172:0/2012560129 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5186 (secure 0 0 0) 0x7f3f08058db0 con 0x7f3f1013e970 2024-04-14T08:37:16.711 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:16.711 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:37:01.115031+0000","modified":"2024-04-14T08:37:16.366432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":42,"state":"up:replay","state_seq":1,"addr":"172.21.15.172:6837/2920402198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2920402198},{"type":"v1","addr":"172.21.15.172:6837","nonce":2920402198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-14T08:37:16.711 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 42 2024-04-14T08:37:16.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.711+0000 7f3f178de640 1 -- 172.21.15.172:0/2012560129 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ef406c8f0 msgr2=0x7f3ef406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:16.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.711+0000 7f3f178de640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ef406c8f0 0x7f3ef406edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3f1013b3e0 tx=0x7f3f0c00b040 comp rx=0 tx=0).stop 2024-04-14T08:37:16.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.711+0000 7f3f178de640 1 -- 172.21.15.172:0/2012560129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f1013e970 msgr2=0x7f3f1013c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:16.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.711+0000 7f3f178de640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f1013e970 0x7f3f1013c9c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3f100b3fa0 tx=0x7f3f08007430 comp rx=0 tx=0).stop 2024-04-14T08:37:16.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.711+0000 7f3f178de640 1 -- 172.21.15.172:0/2012560129 shutdown_connections 2024-04-14T08:37:16.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.711+0000 7f3f178de640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3ef406c8f0 0x7f3ef406edb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.711+0000 7f3f178de640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f1013e970 0x7f3f1013c9c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.711+0000 7f3f178de640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f1013e4f0 0x7f3f1013c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.711+0000 7f3f178de640 1 --2- 172.21.15.172:0/2012560129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3efc004800 0x7f3f101f0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:16.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.711+0000 7f3f178de640 1 -- 172.21.15.172:0/2012560129 >> 172.21.15.172:0/2012560129 conn(0x7f3f1006ec30 msgr2=0x7f3f100b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:16.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.711+0000 7f3f178de640 1 -- 172.21.15.172:0/2012560129 shutdown_connections 2024-04-14T08:37:16.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:16.711+0000 7f3f178de640 1 -- 172.21.15.172:0/2012560129 wait complete. 2024-04-14T08:37:17.126 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:37:01.115031+0000', 'modified': '2024-04-14T08:37:16.366432+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 75, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6993}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6993': {'gid': 6993, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.172:6837/2920402198', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 2920402198}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 2920402198}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-14T08:37:17.126 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6993:up:replay 2024-04-14T08:37:18.127 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:37:18.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.238+0000 7f7654bfe640 1 Processor -- start 2024-04-14T08:37:18.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.238+0000 7f7654bfe640 1 -- start start 2024-04-14T08:37:18.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f7654bfe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f765015a690 0x7f7650071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:18.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f7654bfe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7650072970 0x7f76500722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:18.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f7654bfe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f765015aa70 0x7f765015e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:18.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f7654bfe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f76500b3360 con 0x7f765015a690 2024-04-14T08:37:18.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f7654bfe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f76500b3640 con 0x7f7650072970 2024-04-14T08:37:18.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f7654bfe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f76500b3920 con 0x7f765015aa70 2024-04-14T08:37:18.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f764e59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f765015a690 0x7f7650071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:18.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f764e59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f765015a690 0x7f7650071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46890/0 (socket says 172.21.15.172:46890) 2024-04-14T08:37:18.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f764e59c640 1 -- 172.21.15.172:0/3919079948 learned_addr learned my addr 172.21.15.172:0/3919079948 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:18.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f7645d9b640 1 --2- 172.21.15.172:0/3919079948 >> v2:172.21.15.177:3300/0 conn(0x7f7650072970 0x7f76500722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:18.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f764ed9d640 1 --2- 172.21.15.172:0/3919079948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f765015aa70 0x7f765015e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:18.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f764e59c640 1 -- 172.21.15.172:0/3919079948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f765015aa70 msgr2=0x7f765015e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:18.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f764e59c640 1 --2- 172.21.15.172:0/3919079948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f765015aa70 0x7f765015e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:18.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f764e59c640 1 -- 172.21.15.172:0/3919079948 >> v2:172.21.15.177:3300/0 conn(0x7f7650072970 msgr2=0x7f76500722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:18.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f764e59c640 1 --2- 172.21.15.172:0/3919079948 >> v2:172.21.15.177:3300/0 conn(0x7f7650072970 0x7f76500722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:18.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f764e59c640 1 -- 172.21.15.172:0/3919079948 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76500b3c00 con 0x7f765015a690 2024-04-14T08:37:18.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.239+0000 7f7645d9b640 1 --2- 172.21.15.172:0/3919079948 >> v2:172.21.15.177:3300/0 conn(0x7f7650072970 0x7f76500722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:18.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.240+0000 7f764e59c640 1 --2- 172.21.15.172:0/3919079948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f765015a690 0x7f7650071d60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f76500b3850 tx=0x7f764002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc3084f07a18613 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:18.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.240+0000 7f764dd9b640 1 -- 172.21.15.172:0/3919079948 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76400317c0 con 0x7f765015a690 2024-04-14T08:37:18.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.240+0000 7f764dd9b640 1 -- 172.21.15.172:0/3919079948 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7640037070 con 0x7f765015a690 2024-04-14T08:37:18.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.240+0000 7f764dd9b640 1 -- 172.21.15.172:0/3919079948 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640031b80 con 0x7f765015a690 2024-04-14T08:37:18.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.241+0000 7f7654bfe640 1 -- 172.21.15.172:0/3919079948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f765015a690 msgr2=0x7f7650071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:18.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.241+0000 7f7654bfe640 1 --2- 172.21.15.172:0/3919079948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f765015a690 0x7f7650071d60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f76500b3850 tx=0x7f764002b860 comp rx=0 tx=0).stop 2024-04-14T08:37:18.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.241+0000 7f7654bfe640 1 -- 172.21.15.172:0/3919079948 shutdown_connections 2024-04-14T08:37:18.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.241+0000 7f7654bfe640 1 --2- 172.21.15.172:0/3919079948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f765015aa70 0x7f765015e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:18.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.241+0000 7f7654bfe640 1 --2- 172.21.15.172:0/3919079948 >> v2:172.21.15.177:3300/0 conn(0x7f7650072970 0x7f76500722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.241+0000 7f7654bfe640 1 --2- 172.21.15.172:0/3919079948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f765015a690 0x7f7650071d60 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.241+0000 7f7654bfe640 1 -- 172.21.15.172:0/3919079948 >> 172.21.15.172:0/3919079948 conn(0x7f765006ec30 msgr2=0x7f765006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.241+0000 7f7654bfe640 1 -- 172.21.15.172:0/3919079948 shutdown_connections 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.241+0000 7f7654bfe640 1 -- 172.21.15.172:0/3919079948 wait complete. 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.242+0000 7f7654bfe640 1 Processor -- start 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.242+0000 7f7654bfe640 1 -- start start 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.243+0000 7f7654bfe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7650072970 0x7f765014b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.243+0000 7f7654bfe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f765015a690 0x7f765015b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.243+0000 7f7654bfe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f765015aa70 0x7f765015b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.243+0000 7f7654bfe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f76500b44d0 con 0x7f765015a690 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.243+0000 7f7654bfe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f76500b3100 con 0x7f7650072970 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.243+0000 7f7654bfe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f76500b4650 con 0x7f765015aa70 2024-04-14T08:37:18.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.243+0000 7f7645d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f765015a690 0x7f765015b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.243+0000 7f7645d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f765015a690 0x7f765015b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46904/0 (socket says 172.21.15.172:46904) 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.243+0000 7f7645d9b640 1 -- 172.21.15.172:0/1132576032 learned_addr learned my addr 172.21.15.172:0/1132576032 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.243+0000 7f764ed9d640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f765015aa70 0x7f765015b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.243+0000 7f764e59c640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7650072970 0x7f765014b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.243+0000 7f7645d9b640 1 -- 172.21.15.172:0/1132576032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f765015aa70 msgr2=0x7f765015b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.244+0000 7f7645d9b640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f765015aa70 0x7f765015b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.244+0000 7f7645d9b640 1 -- 172.21.15.172:0/1132576032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7650072970 msgr2=0x7f765014b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.244+0000 7f7645d9b640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7650072970 0x7f765014b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.244+0000 7f7645d9b640 1 -- 172.21.15.172:0/1132576032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765014f100 con 0x7f765015a690 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.244+0000 7f7645d9b640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f765015a690 0x7f765015b210 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f763800d7d0 tx=0x7f763800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.244+0000 7f76477fe640 1 -- 172.21.15.172:0/1132576032 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638004030 con 0x7f765015a690 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.244+0000 7f76477fe640 1 -- 172.21.15.172:0/1132576032 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76380041d0 con 0x7f765015a690 2024-04-14T08:37:18.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.244+0000 7f7654bfe640 1 -- 172.21.15.172:0/1132576032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76500ad980 con 0x7f765015a690 2024-04-14T08:37:18.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.244+0000 7f76477fe640 1 -- 172.21.15.172:0/1132576032 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76380096a0 con 0x7f765015a690 2024-04-14T08:37:18.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.244+0000 7f7654bfe640 1 -- 172.21.15.172:0/1132576032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76500adf40 con 0x7f765015a690 2024-04-14T08:37:18.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.245+0000 7f7654bfe640 1 -- 172.21.15.172:0/1132576032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7650068520 con 0x7f765015a690 2024-04-14T08:37:18.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.245+0000 7f76477fe640 1 -- 172.21.15.172:0/1132576032 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7638009840 con 0x7f765015a690 2024-04-14T08:37:18.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.246+0000 7f76477fe640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f763006c8f0 0x7f763006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:18.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.246+0000 7f764e59c640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f763006c8f0 0x7f763006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:18.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.246+0000 7f76477fe640 1 -- 172.21.15.172:0/1132576032 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f763808c430 con 0x7f765015a690 2024-04-14T08:37:18.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.246+0000 7f764e59c640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f763006c8f0 0x7f763006edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7640036000 tx=0x7f764003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:18.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.251+0000 7f76477fe640 1 -- 172.21.15.172:0/1132576032 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f763805a0f0 con 0x7f765015a690 2024-04-14T08:37:18.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.464+0000 7f7654bfe640 1 -- 172.21.15.172:0/1132576032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76500d04a0 con 0x7f765015a690 2024-04-14T08:37:18.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.464+0000 7f76477fe640 1 -- 172.21.15.172:0/1132576032 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5186 (secure 0 0 0) 0x7f763805e3f0 con 0x7f765015a690 2024-04-14T08:37:18.466 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:18.466 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:37:01.115031+0000","modified":"2024-04-14T08:37:17.372278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":42,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.172:6837/2920402198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2920402198},{"type":"v1","addr":"172.21.15.172:6837","nonce":2920402198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-14T08:37:18.466 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 44 2024-04-14T08:37:18.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.466+0000 7f7654bfe640 1 -- 172.21.15.172:0/1132576032 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f763006c8f0 msgr2=0x7f763006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:18.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.466+0000 7f7654bfe640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f763006c8f0 0x7f763006edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7640036000 tx=0x7f764003d040 comp rx=0 tx=0).stop 2024-04-14T08:37:18.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.466+0000 7f7654bfe640 1 -- 172.21.15.172:0/1132576032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f765015a690 msgr2=0x7f765015b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:18.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.466+0000 7f7654bfe640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f765015a690 0x7f765015b210 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f763800d7d0 tx=0x7f763800dc90 comp rx=0 tx=0).stop 2024-04-14T08:37:18.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.466+0000 7f7654bfe640 1 -- 172.21.15.172:0/1132576032 shutdown_connections 2024-04-14T08:37:18.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.466+0000 7f7654bfe640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f763006c8f0 0x7f763006edb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:18.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.466+0000 7f7654bfe640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f765015aa70 0x7f765015b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:18.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.466+0000 7f7654bfe640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f765015a690 0x7f765015b210 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:18.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.466+0000 7f7654bfe640 1 --2- 172.21.15.172:0/1132576032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7650072970 0x7f765014b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:18.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.466+0000 7f7654bfe640 1 -- 172.21.15.172:0/1132576032 >> 172.21.15.172:0/1132576032 conn(0x7f765006ec30 msgr2=0x7f765006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:18.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.466+0000 7f7654bfe640 1 -- 172.21.15.172:0/1132576032 shutdown_connections 2024-04-14T08:37:18.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:18.466+0000 7f7654bfe640 1 -- 172.21.15.172:0/1132576032 wait complete. 2024-04-14T08:37:18.881 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:37:01.115031+0000', 'modified': '2024-04-14T08:37:17.372278+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 75, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6993}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6993': {'gid': 6993, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.172:6837/2920402198', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 2920402198}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 2920402198}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-14T08:37:18.882 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6993:up:rejoin 2024-04-14T08:37:19.883 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:37:19.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.988+0000 7fbddd1fe640 1 Processor -- start 2024-04-14T08:37:19.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.988+0000 7fbddd1fe640 1 -- start start 2024-04-14T08:37:19.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbddd1fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbdd80d2830 0x7fbdd80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:19.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbddd1fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdd80cedd0 0x7fbdd80b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:19.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbddd1fe640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdd80d3150 con 0x7fbdd815e890 2024-04-14T08:37:19.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbddd1fe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fbdd80cff30 con 0x7fbdd80d2830 2024-04-14T08:37:19.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbddd1fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd80d2350 con 0x7fbdd80cedd0 2024-04-14T08:37:19.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbdd6d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbdd80d2830 0x7fbdd80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:19.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbdd6d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbdd80d2830 0x7fbdd80d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47972/0 (socket says 172.21.15.172:47972) 2024-04-14T08:37:19.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbdd6d9d640 1 -- 172.21.15.172:0/3069950324 learned_addr learned my addr 172.21.15.172:0/3069950324 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:19.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbdd759e640 1 --2- 172.21.15.172:0/3069950324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdd80cedd0 0x7fbdd80b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:19.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbdd6d9d640 1 -- 172.21.15.172:0/3069950324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdd80cedd0 msgr2=0x7fbdd80b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:19.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbdd6d9d640 1 --2- 172.21.15.172:0/3069950324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdd80cedd0 0x7fbdd80b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:19.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbdd6d9d640 1 -- 172.21.15.172:0/3069950324 >> v1:172.21.15.177:6789/0 conn(0x7fbdd815e890 legacy=0x7fbdd80d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:19.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.989+0000 7fbdd6d9d640 1 -- 172.21.15.172:0/3069950324 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd80cf320 con 0x7fbdd80d2830 2024-04-14T08:37:19.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.990+0000 7fbdd6d9d640 1 --2- 172.21.15.172:0/3069950324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdd80d2830 0x7fbdd80d2c10 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009870 tx=0x7fbdc802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac963e56c5913cf server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:19.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.990+0000 7fbdd5d9b640 1 -- 172.21.15.172:0/3069950324 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8034070 con 0x7fbdd80d2830 2024-04-14T08:37:19.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.990+0000 7fbdd5d9b640 1 -- 172.21.15.172:0/3069950324 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc8030840 con 0x7fbdd80d2830 2024-04-14T08:37:19.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.990+0000 7fbdd5d9b640 1 -- 172.21.15.172:0/3069950324 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8030c10 con 0x7fbdd80d2830 2024-04-14T08:37:19.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.991+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3069950324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdd80d2830 msgr2=0x7fbdd80d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:19.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.991+0000 7fbddd1fe640 1 --2- 172.21.15.172:0/3069950324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdd80d2830 0x7fbdd80d2c10 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009870 tx=0x7fbdc802b980 comp rx=0 tx=0).stop 2024-04-14T08:37:19.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.991+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3069950324 shutdown_connections 2024-04-14T08:37:19.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.991+0000 7fbddd1fe640 1 --2- 172.21.15.172:0/3069950324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdd80cedd0 0x7fbdd80b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:19.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.991+0000 7fbddd1fe640 1 --2- 172.21.15.172:0/3069950324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdd80d2830 0x7fbdd80d2c10 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:19.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.991+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3069950324 >> 172.21.15.172:0/3069950324 conn(0x7fbdd809d590 msgr2=0x7fbdd80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:19.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.991+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3069950324 shutdown_connections 2024-04-14T08:37:19.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.991+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3069950324 wait complete. 2024-04-14T08:37:19.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.992+0000 7fbddd1fe640 1 Processor -- start 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.992+0000 7fbddd1fe640 1 -- start start 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbddd1fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdd80cedd0 0x7fbdd80c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbddd1fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdd80d2830 0x7fbdd80b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbddd1fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd815e890 0x7fbdd814b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbddd1fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd80cfbc0 con 0x7fbdd80d2830 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbddd1fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd80cf8e0 con 0x7fbdd815e890 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbddd1fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd80b3c50 con 0x7fbdd80cedd0 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbdd659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdd80d2830 0x7fbdd80b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbdd659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdd80d2830 0x7fbdd80b7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47986/0 (socket says 172.21.15.172:47986) 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbdd659c640 1 -- 172.21.15.172:0/3434001357 learned_addr learned my addr 172.21.15.172:0/3434001357 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbdd659c640 1 -- 172.21.15.172:0/3434001357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdd80cedd0 msgr2=0x7fbdd80c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbdd759e640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd815e890 0x7fbdd814b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:19.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbdd6d9d640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdd80cedd0 0x7fbdd80c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:19.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbdd659c640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdd80cedd0 0x7fbdd80c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:19.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbdd659c640 1 -- 172.21.15.172:0/3434001357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd815e890 msgr2=0x7fbdd814b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:19.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbdd659c640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd815e890 0x7fbdd814b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:19.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbdd659c640 1 -- 172.21.15.172:0/3434001357 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd80c83d0 con 0x7fbdd80d2830 2024-04-14T08:37:19.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.993+0000 7fbdd759e640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd815e890 0x7fbdd814b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:19.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.994+0000 7fbdd659c640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdd80d2830 0x7fbdd80b7070 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00b3f0 tx=0x7fbdcc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:19.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.994+0000 7fbdbf7fe640 1 -- 172.21.15.172:0/3434001357 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc015020 con 0x7fbdd80d2830 2024-04-14T08:37:19.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.994+0000 7fbdbf7fe640 1 -- 172.21.15.172:0/3434001357 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdcc011830 con 0x7fbdd80d2830 2024-04-14T08:37:19.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.994+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3434001357 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd80c86b0 con 0x7fbdd80d2830 2024-04-14T08:37:19.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.994+0000 7fbdbf7fe640 1 -- 172.21.15.172:0/3434001357 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc011c20 con 0x7fbdd80d2830 2024-04-14T08:37:19.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.994+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3434001357 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd80c8f50 con 0x7fbdd80d2830 2024-04-14T08:37:19.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.995+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3434001357 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd80c9150 con 0x7fbdd80d2830 2024-04-14T08:37:19.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.995+0000 7fbdbf7fe640 1 -- 172.21.15.172:0/3434001357 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbdcc018020 con 0x7fbdd80d2830 2024-04-14T08:37:19.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.996+0000 7fbdbf7fe640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbdc406c8f0 0x7fbdc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:19.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.996+0000 7fbdd6d9d640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbdc406c8f0 0x7fbdc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:19.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.996+0000 7fbdbf7fe640 1 -- 172.21.15.172:0/3434001357 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fbdcc08b9d0 con 0x7fbdd80d2830 2024-04-14T08:37:19.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:19.996+0000 7fbdd6d9d640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbdc406c8f0 0x7fbdc406edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8036000 tx=0x7fbdc803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:20.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.000+0000 7fbdbf7fe640 1 -- 172.21.15.172:0/3434001357 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbdd80c9150 con 0x7fbdd80d2830 2024-04-14T08:37:20.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.213+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3434001357 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdd80c8880 con 0x7fbdd80d2830 2024-04-14T08:37:20.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.214+0000 7fbdbf7fe640 1 -- 172.21.15.172:0/3434001357 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5193 (secure 0 0 0) 0x7fbdcc01f030 con 0x7fbdd80d2830 2024-04-14T08:37:20.215 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:20.215 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:37:01.115031+0000","modified":"2024-04-14T08:37:19.376016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.172:6837/2920402198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2920402198},{"type":"v1","addr":"172.21.15.172:6837","nonce":2920402198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":5}]} 2024-04-14T08:37:20.215 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 45 2024-04-14T08:37:20.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.216+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3434001357 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbdc406c8f0 msgr2=0x7fbdc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.216+0000 7fbddd1fe640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbdc406c8f0 0x7fbdc406edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8036000 tx=0x7fbdc803b040 comp rx=0 tx=0).stop 2024-04-14T08:37:20.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.216+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3434001357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdd80d2830 msgr2=0x7fbdd80b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.216+0000 7fbddd1fe640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdd80d2830 0x7fbdd80b7070 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00b3f0 tx=0x7fbdcc00b8b0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.216+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3434001357 shutdown_connections 2024-04-14T08:37:20.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.216+0000 7fbddd1fe640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbdc406c8f0 0x7fbdc406edb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.216+0000 7fbddd1fe640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd815e890 0x7fbdd814b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.216+0000 7fbddd1fe640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdd80d2830 0x7fbdd80b7070 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.216+0000 7fbddd1fe640 1 --2- 172.21.15.172:0/3434001357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdd80cedd0 0x7fbdd80c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.216+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3434001357 >> 172.21.15.172:0/3434001357 conn(0x7fbdd809d590 msgr2=0x7fbdd80d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:20.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.216+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3434001357 shutdown_connections 2024-04-14T08:37:20.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.216+0000 7fbddd1fe640 1 -- 172.21.15.172:0/3434001357 wait complete. 2024-04-14T08:37:20.631 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:37:01.115031+0000', 'modified': '2024-04-14T08:37:19.376016+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 75, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6993}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6993': {'gid': 6993, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.172:6837/2920402198', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 2920402198}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 2920402198}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6993, 'qdb_cluster': [6993]} 2024-04-14T08:37:20.632 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:37:20.632 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-14T08:37:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.750+0000 7f3c81008640 1 Processor -- start 2024-04-14T08:37:20.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.750+0000 7f3c81008640 1 -- start start 2024-04-14T08:37:20.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.751+0000 7f3c81008640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3c7c13fa70 0x7f3c7c0af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.751+0000 7f3c81008640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3c7c0ce150 0x7f3c7c0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.751+0000 7f3c81008640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c7c071520 con 0x7f3c7c0ce5e0 2024-04-14T08:37:20.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.751+0000 7f3c81008640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f3c7c0712d0 con 0x7f3c7c0ce150 2024-04-14T08:37:20.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.751+0000 7f3c81008640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f3c7c071800 con 0x7f3c7c13fa70 2024-04-14T08:37:20.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.751+0000 7f3c7a59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3c7c0ce150 0x7f3c7c0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.751+0000 7f3c7b59e640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3c7c0ce5e0 0x7f3c7c15e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:42914/0 (socket says 172.21.15.172:42914) 2024-04-14T08:37:20.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.751+0000 7f3c7b59e640 1 -- 172.21.15.172:0/168318933 learned_addr learned my addr 172.21.15.172:0/168318933 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:20.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.752+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/168318933 >> v2:172.21.15.177:3300/0 conn(0x7f3c7c13fa70 0x7f3c7c0af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.752+0000 7f3c7a59c640 1 -- 172.21.15.172:0/168318933 >> v2:172.21.15.177:3300/0 conn(0x7f3c7c13fa70 msgr2=0x7f3c7c0af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:20.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.752+0000 7f3c7a59c640 1 --2- 172.21.15.172:0/168318933 >> v2:172.21.15.177:3300/0 conn(0x7f3c7c13fa70 0x7f3c7c0af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.752+0000 7f3c7a59c640 1 -- 172.21.15.172:0/168318933 >> v1:172.21.15.172:6789/0 conn(0x7f3c7c0ce5e0 legacy=0x7f3c7c15e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.752+0000 7f3c79d9b640 1 -- 172.21.15.172:0/168318933 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1524611090 0 0) 0x7f3c7c071520 con 0x7f3c7c0ce5e0 2024-04-14T08:37:20.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.752+0000 7f3c7a59c640 1 -- 172.21.15.172:0/168318933 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7c071ae0 con 0x7f3c7c0ce150 2024-04-14T08:37:20.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.752+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/168318933 >> v2:172.21.15.177:3300/0 conn(0x7f3c7c13fa70 0x7f3c7c0af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:20.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.752+0000 7f3c7a59c640 1 --2- 172.21.15.172:0/168318933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c7c0ce150 0x7f3c7c0722e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0b2cf0 tx=0x7f3c6c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5227ec6dad87bb05 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:20.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.753+0000 7f3c79d9b640 1 -- 172.21.15.172:0/168318933 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0315a0 con 0x7f3c7c0ce150 2024-04-14T08:37:20.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.753+0000 7f3c79d9b640 1 -- 172.21.15.172:0/168318933 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c037070 con 0x7f3c7c0ce150 2024-04-14T08:37:20.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.753+0000 7f3c79d9b640 1 -- 172.21.15.172:0/168318933 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c031960 con 0x7f3c7c0ce150 2024-04-14T08:37:20.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.753+0000 7f3c81008640 1 -- 172.21.15.172:0/168318933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c7c0ce150 msgr2=0x7f3c7c0722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.753+0000 7f3c81008640 1 --2- 172.21.15.172:0/168318933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c7c0ce150 0x7f3c7c0722e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0b2cf0 tx=0x7f3c6c02b840 comp rx=0 tx=0).stop 2024-04-14T08:37:20.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.753+0000 7f3c81008640 1 -- 172.21.15.172:0/168318933 shutdown_connections 2024-04-14T08:37:20.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.753+0000 7f3c81008640 1 --2- 172.21.15.172:0/168318933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c7c0ce150 0x7f3c7c0722e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.753+0000 7f3c81008640 1 --2- 172.21.15.172:0/168318933 >> v2:172.21.15.177:3300/0 conn(0x7f3c7c13fa70 0x7f3c7c0af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.753+0000 7f3c81008640 1 -- 172.21.15.172:0/168318933 >> 172.21.15.172:0/168318933 conn(0x7f3c7c06b520 msgr2=0x7f3c7c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:20.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.753+0000 7f3c81008640 1 -- 172.21.15.172:0/168318933 shutdown_connections 2024-04-14T08:37:20.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.753+0000 7f3c81008640 1 -- 172.21.15.172:0/168318933 wait complete. 2024-04-14T08:37:20.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.755+0000 7f3c81008640 1 Processor -- start 2024-04-14T08:37:20.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.755+0000 7f3c81008640 1 -- start start 2024-04-14T08:37:20.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c81008640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c7c0ce150 0x7f3c7c143550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c81008640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c7c0ce5e0 0x7f3c7c15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c81008640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c7c13fa70 0x7f3c7c15ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c81008640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1442b0 con 0x7f3c7c0ce5e0 2024-04-14T08:37:20.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c81008640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c144130 con 0x7f3c7c0ce150 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c81008640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7c072820 con 0x7f3c7c13fa70 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c7a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c7c0ce5e0 0x7f3c7c15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c7a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c7c0ce5e0 0x7f3c7c15c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47994/0 (socket says 172.21.15.172:47994) 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c7a59c640 1 -- 172.21.15.172:0/1815448078 learned_addr learned my addr 172.21.15.172:0/1815448078 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c7c0ce150 0x7f3c7c143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c7b59e640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c7c13fa70 0x7f3c7c15ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c7a59c640 1 -- 172.21.15.172:0/1815448078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c7c13fa70 msgr2=0x7f3c7c15ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c7a59c640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c7c13fa70 0x7f3c7c15ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c7a59c640 1 -- 172.21.15.172:0/1815448078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c7c0ce150 msgr2=0x7f3c7c143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c7a59c640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c7c0ce150 0x7f3c7c143550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c7a59c640 1 -- 172.21.15.172:0/1815448078 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7c15b0b0 con 0x7f3c7c0ce5e0 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c7a59c640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c7c0ce5e0 0x7f3c7c15c3f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009890 tx=0x7f3c6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c737fe640 1 -- 172.21.15.172:0/1815448078 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c032630 con 0x7f3c7c0ce5e0 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c737fe640 1 -- 172.21.15.172:0/1815448078 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c0327d0 con 0x7f3c7c0ce5e0 2024-04-14T08:37:20.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c737fe640 1 -- 172.21.15.172:0/1815448078 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c032bc0 con 0x7f3c7c0ce5e0 2024-04-14T08:37:20.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c81008640 1 -- 172.21.15.172:0/1815448078 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7c15b350 con 0x7f3c7c0ce5e0 2024-04-14T08:37:20.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.757+0000 7f3c81008640 1 -- 172.21.15.172:0/1815448078 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7c15b910 con 0x7f3c7c0ce5e0 2024-04-14T08:37:20.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.759+0000 7f3c737fe640 1 -- 172.21.15.172:0/1815448078 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c6c04c020 con 0x7f3c7c0ce5e0 2024-04-14T08:37:20.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.760+0000 7f3c737fe640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3c5c06c8f0 0x7f3c5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.760+0000 7f3c737fe640 1 -- 172.21.15.172:0/1815448078 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f3c6c02f020 con 0x7f3c7c0ce5e0 2024-04-14T08:37:20.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.760+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3c5c06c8f0 0x7f3c5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.760+0000 7f3c83dee740 1 Processor -- start 2024-04-14T08:37:20.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.760+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3c5c06c8f0 0x7f3c5c06edb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3c64006fd0 tx=0x7f3c64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:20.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.760+0000 7f3c83dee740 1 -- start start 2024-04-14T08:37:20.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.761+0000 7f3c83dee740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b635e0 0x55eca9b612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.761+0000 7f3c83dee740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b639c0 0x55eca9b1ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.761+0000 7f3c83dee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b42f20 0x55eca9b1b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.761+0000 7f3c83dee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55eca9a7f030 con 0x55eca9b42f20 2024-04-14T08:37:20.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.761+0000 7f3c83dee740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55eca9a27620 con 0x55eca9b635e0 2024-04-14T08:37:20.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.761+0000 7f3c83dee740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55eca9a7afe0 con 0x55eca9b639c0 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.761+0000 7f3c7b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b42f20 0x55eca9b1b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.761+0000 7f3c7b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b42f20 0x55eca9b1b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47996/0 (socket says 172.21.15.172:47996) 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.761+0000 7f3c7b59e640 1 -- 172.21.15.172:0/647663324 learned_addr learned my addr 172.21.15.172:0/647663324 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.761+0000 7f3c7a59c640 1 --2- 172.21.15.172:0/647663324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b639c0 0x55eca9b1ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.761+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/647663324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b635e0 0x55eca9b612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.762+0000 7f3c7b59e640 1 -- 172.21.15.172:0/647663324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b639c0 msgr2=0x55eca9b1ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.762+0000 7f3c7b59e640 1 --2- 172.21.15.172:0/647663324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b639c0 0x55eca9b1ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.762+0000 7f3c7b59e640 1 -- 172.21.15.172:0/647663324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b635e0 msgr2=0x55eca9b612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.762+0000 7f3c7b59e640 1 --2- 172.21.15.172:0/647663324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b635e0 0x55eca9b612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.762+0000 7f3c7b59e640 1 -- 172.21.15.172:0/647663324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eca97932d0 con 0x55eca9b42f20 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.762+0000 7f3c7b59e640 1 --2- 172.21.15.172:0/647663324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b42f20 0x55eca9b1b150 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3c740072f0 tx=0x7f3c740077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496f9cf4d795ce19 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.762+0000 7f3c70ff9640 1 -- 172.21.15.172:0/647663324 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74019070 con 0x55eca9b42f20 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.762+0000 7f3c70ff9640 1 -- 172.21.15.172:0/647663324 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c740079a0 con 0x55eca9b42f20 2024-04-14T08:37:20.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.762+0000 7f3c70ff9640 1 -- 172.21.15.172:0/647663324 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74001fe0 con 0x55eca9b42f20 2024-04-14T08:37:20.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.763+0000 7f3c83dee740 1 -- 172.21.15.172:0/647663324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b42f20 msgr2=0x55eca9b1b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.763+0000 7f3c83dee740 1 --2- 172.21.15.172:0/647663324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b42f20 0x55eca9b1b150 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3c740072f0 tx=0x7f3c740077b0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.763+0000 7f3c83dee740 1 -- 172.21.15.172:0/647663324 shutdown_connections 2024-04-14T08:37:20.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.763+0000 7f3c83dee740 1 --2- 172.21.15.172:0/647663324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b42f20 0x55eca9b1b150 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.763+0000 7f3c83dee740 1 --2- 172.21.15.172:0/647663324 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b639c0 0x55eca9b1ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.763+0000 7f3c83dee740 1 --2- 172.21.15.172:0/647663324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b635e0 0x55eca9b612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.763+0000 7f3c83dee740 1 -- 172.21.15.172:0/647663324 >> 172.21.15.172:0/647663324 conn(0x55eca9b17d70 msgr2=0x55eca9b608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:20.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.763+0000 7f3c83dee740 1 -- 172.21.15.172:0/647663324 shutdown_connections 2024-04-14T08:37:20.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.763+0000 7f3c83dee740 1 -- 172.21.15.172:0/647663324 wait complete. 2024-04-14T08:37:20.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.764+0000 7f3c83dee740 1 -- start start 2024-04-14T08:37:20.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.764+0000 7f3c83dee740 1 Processor -- start 2024-04-14T08:37:20.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c83dee740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b635e0 0x55eca9b5aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c83dee740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b639c0 0x55eca9b3ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c83dee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 0x55eca9b43400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c83dee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55eca9b163a0 con 0x55eca9b5c570 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c83dee740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55eca9a3a9e0 con 0x55eca9b639c0 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c83dee740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55eca9b1a1a0 con 0x55eca9b635e0 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c83dee740 10 client.0 _ll_register_callbacks cb 0x55eca9a77080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c7b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 0x55eca9b43400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c7b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 0x55eca9b43400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48010/0 (socket says 172.21.15.172:48010) 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c7b59e640 1 -- 172.21.15.172:0/104953923 learned_addr learned my addr 172.21.15.172:0/104953923 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c7a59c640 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b639c0 0x55eca9b3ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b635e0 0x55eca9b5aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c7b59e640 1 -- 172.21.15.172:0/104953923 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b635e0 msgr2=0x55eca9b5aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c7b59e640 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b635e0 0x55eca9b5aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c7b59e640 1 -- 172.21.15.172:0/104953923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b639c0 msgr2=0x55eca9b3ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c7b59e640 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b639c0 0x55eca9b3ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c7b59e640 1 -- 172.21.15.172:0/104953923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eca985d6b0 con 0x55eca9b5c570 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.765+0000 7f3c7a59c640 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b639c0 0x55eca9b3ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c7b59e640 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 0x55eca9b43400 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3c74017040 tx=0x7f3c74007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb7e2a97a24a2a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c59ffb640 1 -- 172.21.15.172:0/104953923 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74023030 con 0x55eca9b5c570 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c83dee740 1 -- 172.21.15.172:0/104953923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c740079a0 con 0x55eca9b5c570 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c59ffb640 1 -- 172.21.15.172:0/104953923 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f3c7400f070 con 0x55eca9b5c570 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c59ffb640 1 -- 172.21.15.172:0/104953923 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3c74019030 con 0x55eca9b5c570 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c83dee740 10 client.7041 fetch_fsmap learned FSMap version 45 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c83dee740 1 -- 172.21.15.172:0/104953923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55eca9b5ba90 con 0x55eca9b5c570 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c59ffb640 1 -- 172.21.15.172:0/104953923 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c740079a0 con 0x55eca9b5c570 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c59ffb640 1 -- 172.21.15.172:0/104953923 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7f3c74004b60 con 0x55eca9b5c570 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c83dee740 10 client.7041 fetch_fsmap finished waiting for FSMap version 45 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c83dee740 10 client.7041 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:37:20.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c83dee740 20 client.7041 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c83dee740 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x55eca9b2a780 0x55eca9b2c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c83dee740 4 client.7041 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c83dee740 1 -- 172.21.15.172:0/104953923 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eca9a26cc0 con 0x55eca9b2a780 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.766+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x55eca9b2a780 0x55eca9b2c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.767+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x55eca9b2a780 0x55eca9b2c680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22eafa3cd5f9f21a server_cookie=a1581712b747d589 in_seq=0 out_seq=0 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.767+0000 7f3c59ffb640 10 client.7041 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.767+0000 7f3c59ffb640 1 -- 172.21.15.172:0/104953923 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eca9a26cc0 con 0x55eca9b2a780 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.767+0000 7f3c59ffb640 10 client.7041 handle_command_reply: tid=0 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.767+0000 7f3c83dee740 1 client.7041 shutdown 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.767+0000 7f3c83dee740 1 -- 172.21.15.172:0/104953923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 msgr2=0x55eca9b43400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.767+0000 7f3c83dee740 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 0x55eca9b43400 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3c74017040 tx=0x7f3c74007d90 comp rx=0 tx=0).stop 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.767+0000 7f3c83dee740 1 -- 172.21.15.172:0/104953923 shutdown_connections 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.767+0000 7f3c83dee740 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x55eca9b2a780 0x55eca9b2c680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.768+0000 7f3c83dee740 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 0x55eca9b43400 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.768+0000 7f3c83dee740 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b639c0 0x55eca9b3ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.768+0000 7f3c59ffb640 0 client.7041 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:20.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.768+0000 7f3c83dee740 1 --2- 172.21.15.172:0/104953923 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b635e0 0x55eca9b5aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.768+0000 7f3c83dee740 1 -- 172.21.15.172:0/104953923 >> 172.21.15.172:0/104953923 conn(0x55eca9b42f20 msgr2=0x55eca9b63e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:20.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.768+0000 7f3c83dee740 1 -- 172.21.15.172:0/104953923 shutdown_connections 2024-04-14T08:37:20.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.768+0000 7f3c83dee740 1 -- 172.21.15.172:0/104953923 wait complete. 2024-04-14T08:37:20.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.768+0000 7f3c83dee740 20 client.7041 trim_cache size 0 max 16384 2024-04-14T08:37:20.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.786+0000 7f3c83dee740 1 Processor -- start 2024-04-14T08:37:20.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.786+0000 7f3c83dee740 1 -- start start 2024-04-14T08:37:20.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.786+0000 7f3c83dee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 0x55eca9b41a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.786+0000 7f3c83dee740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b635e0 0x55eca9b1a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.786+0000 7f3c83dee740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b639c0 0x55eca9b495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.786+0000 7f3c83dee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c0b4a70 con 0x55eca9b5c570 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.786+0000 7f3c83dee740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c141a00 con 0x55eca9b635e0 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.786+0000 7f3c83dee740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7c0b4420 con 0x55eca9b639c0 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.786+0000 7f3c7ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 0x55eca9b41a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.786+0000 7f3c7ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 0x55eca9b41a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48016/0 (socket says 172.21.15.172:48016) 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.786+0000 7f3c7ad9d640 1 -- 172.21.15.172:0/2680047762 learned_addr learned my addr 172.21.15.172:0/2680047762 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c7ad9d640 1 -- 172.21.15.172:0/2680047762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b639c0 msgr2=0x55eca9b495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/2680047762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b639c0 0x55eca9b495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c7ad9d640 1 -- 172.21.15.172:0/2680047762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b635e0 msgr2=0x55eca9b1a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/2680047762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b635e0 0x55eca9b1a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c7ad9d640 1 -- 172.21.15.172:0/2680047762 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eca9b43770 con 0x55eca9b5c570 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/2680047762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 0x55eca9b41a60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400d840 tx=0x7f3c64015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc2247a46c9ff9c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c5bfff640 1 -- 172.21.15.172:0/2680047762 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6400b040 con 0x55eca9b5c570 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c5bfff640 1 -- 172.21.15.172:0/2680047762 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6402e070 con 0x55eca9b5c570 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c5bfff640 1 -- 172.21.15.172:0/2680047762 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64029340 con 0x55eca9b5c570 2024-04-14T08:37:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c83dee740 1 -- 172.21.15.172:0/2680047762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 msgr2=0x55eca9b41a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c83dee740 1 --2- 172.21.15.172:0/2680047762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 0x55eca9b41a60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400d840 tx=0x7f3c64015000 comp rx=0 tx=0).stop 2024-04-14T08:37:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c83dee740 1 -- 172.21.15.172:0/2680047762 shutdown_connections 2024-04-14T08:37:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c83dee740 1 --2- 172.21.15.172:0/2680047762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b639c0 0x55eca9b495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c83dee740 1 --2- 172.21.15.172:0/2680047762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b635e0 0x55eca9b1a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c83dee740 1 --2- 172.21.15.172:0/2680047762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b5c570 0x55eca9b41a60 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c83dee740 1 -- 172.21.15.172:0/2680047762 >> 172.21.15.172:0/2680047762 conn(0x55eca9b42f20 msgr2=0x55eca9b10e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c83dee740 1 -- 172.21.15.172:0/2680047762 shutdown_connections 2024-04-14T08:37:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.787+0000 7f3c83dee740 1 -- 172.21.15.172:0/2680047762 wait complete. 2024-04-14T08:37:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.788+0000 7f3c83dee740 1 -- start start 2024-04-14T08:37:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.788+0000 7f3c83dee740 1 Processor -- start 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c83dee740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b5c570 0x55eca9b73480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c83dee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b635e0 0x55eca9bdbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c83dee740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b639c0 0x55eca9b77470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c83dee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55eca9b3db50 con 0x55eca9b635e0 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c83dee740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55eca9b5bd70 con 0x55eca9b639c0 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c83dee740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55eca9b2a530 con 0x55eca9b5c570 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c83dee740 10 client.0 _ll_register_callbacks cb 0x7f3c74017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c7a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b635e0 0x55eca9bdbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c7a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b635e0 0x55eca9bdbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48028/0 (socket says 172.21.15.172:48028) 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c7a59c640 1 -- 172.21.15.172:0/842735257 learned_addr learned my addr 172.21.15.172:0/842735257 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c7a59c640 1 -- 172.21.15.172:0/842735257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b5c570 msgr2=0x55eca9b73480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c7a59c640 1 --2- 172.21.15.172:0/842735257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b5c570 0x55eca9b73480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c7a59c640 1 -- 172.21.15.172:0/842735257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b639c0 msgr2=0x55eca9b77470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c7a59c640 1 --2- 172.21.15.172:0/842735257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b639c0 0x55eca9b77470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c7a59c640 1 -- 172.21.15.172:0/842735257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eca9b44280 con 0x55eca9b635e0 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c7a59c640 1 --2- 172.21.15.172:0/842735257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b635e0 0x55eca9bdbce0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009860 tx=0x7f3c6c07b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1259f654d70f080c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c59ffb640 1 -- 172.21.15.172:0/842735257 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c092070 con 0x55eca9b635e0 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c59ffb640 1 -- 172.21.15.172:0/842735257 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f3c6c0abaf0 con 0x55eca9b635e0 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.789+0000 7f3c83dee740 1 -- 172.21.15.172:0/842735257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c6402e070 con 0x55eca9b635e0 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c59ffb640 1 -- 172.21.15.172:0/842735257 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3c6c089050 con 0x55eca9b635e0 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c83dee740 10 client.7047 fetch_fsmap learned FSMap version 45 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c83dee740 1 -- 172.21.15.172:0/842735257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55eca9b44450 con 0x55eca9b635e0 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c59ffb640 1 -- 172.21.15.172:0/842735257 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6402e070 con 0x55eca9b635e0 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c59ffb640 1 -- 172.21.15.172:0/842735257 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7f3c6c08c020 con 0x55eca9b635e0 2024-04-14T08:37:20.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c83dee740 10 client.7047 fetch_fsmap finished waiting for FSMap version 45 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c83dee740 10 client.7047 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c83dee740 20 client.7047 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c83dee740 1 --2- 172.21.15.172:0/842735257 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x55eca9bdf250 0x55eca9b2ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c83dee740 4 client.7047 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c83dee740 1 -- 172.21.15.172:0/842735257 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "status"}) -- 0x55eca9a8c570 con 0x55eca9bdf250 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/842735257 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x55eca9bdf250 0x55eca9b2ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c59ffb640 10 client.7047 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.790+0000 7f3c7ad9d640 1 --2- 172.21.15.172:0/842735257 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x55eca9bdf250 0x55eca9b2ad20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10897fb52418dbef server_cookie=9e2dc730262bd351 in_seq=0 out_seq=0 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c59ffb640 1 -- 172.21.15.172:0/842735257 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55eca9a8c570 con 0x55eca9bdf250 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c59ffb640 10 client.7047 handle_command_reply: tid=0 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c83dee740 1 client.7047 shutdown 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c83dee740 1 -- 172.21.15.172:0/842735257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b635e0 msgr2=0x55eca9bdbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c83dee740 1 --2- 172.21.15.172:0/842735257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b635e0 0x55eca9bdbce0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009860 tx=0x7f3c6c07b120 comp rx=0 tx=0).stop 2024-04-14T08:37:20.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c83dee740 1 -- 172.21.15.172:0/842735257 shutdown_connections 2024-04-14T08:37:20.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c83dee740 1 --2- 172.21.15.172:0/842735257 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x55eca9bdf250 0x55eca9b2ad20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c83dee740 1 --2- 172.21.15.172:0/842735257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eca9b639c0 0x55eca9b77470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c83dee740 1 --2- 172.21.15.172:0/842735257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eca9b635e0 0x55eca9bdbce0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c59ffb640 0 client.7047 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:20.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c83dee740 1 --2- 172.21.15.172:0/842735257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55eca9b5c570 0x55eca9b73480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c83dee740 1 -- 172.21.15.172:0/842735257 >> 172.21.15.172:0/842735257 conn(0x55eca9b42f20 msgr2=0x55eca9b1a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:20.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c83dee740 1 -- 172.21.15.172:0/842735257 shutdown_connections 2024-04-14T08:37:20.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c83dee740 1 -- 172.21.15.172:0/842735257 wait complete. 2024-04-14T08:37:20.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c83dee740 20 client.7047 trim_cache size 0 max 16384 2024-04-14T08:37:20.797 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:37:20.797 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:37:20.797 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:37:20.798 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 6993, 2024-04-14T08:37:20.798 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:37:20.798 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:37:20.798 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:37:20.798 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 4.4200459700000003, 2024-04-14T08:37:20.798 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 45, 2024-04-14T08:37:20.798 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 75, 2024-04-14T08:37:20.798 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 75, 2024-04-14T08:37:20.798 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 5.3884197150000004 2024-04-14T08:37:20.798 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:37:20.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c81008640 1 -- 172.21.15.172:0/1815448078 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3c5c06c8f0 msgr2=0x7f3c5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c81008640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3c5c06c8f0 0x7f3c5c06edb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3c64006fd0 tx=0x7f3c64008040 comp rx=0 tx=0).stop 2024-04-14T08:37:20.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c81008640 1 -- 172.21.15.172:0/1815448078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c7c0ce5e0 msgr2=0x7f3c7c15c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c81008640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c7c0ce5e0 0x7f3c7c15c3f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009890 tx=0x7f3c6c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c81008640 1 -- 172.21.15.172:0/1815448078 shutdown_connections 2024-04-14T08:37:20.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c81008640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3c5c06c8f0 0x7f3c5c06edb0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c81008640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c7c13fa70 0x7f3c7c15ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c81008640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c7c0ce5e0 0x7f3c7c15c3f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c81008640 1 --2- 172.21.15.172:0/1815448078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c7c0ce150 0x7f3c7c143550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c81008640 1 -- 172.21.15.172:0/1815448078 >> 172.21.15.172:0/1815448078 conn(0x7f3c7c06b520 msgr2=0x7f3c7c13e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:20.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c81008640 1 -- 172.21.15.172:0/1815448078 shutdown_connections 2024-04-14T08:37:20.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.791+0000 7f3c81008640 1 -- 172.21.15.172:0/1815448078 wait complete. 2024-04-14T08:37:20.806 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:37:20.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.940+0000 7f1596a08640 1 Processor -- start 2024-04-14T08:37:20.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.940+0000 7f1596a08640 1 -- start start 2024-04-14T08:37:20.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.941+0000 7f1596a08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f159015e4d0 0x7f15900cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.941+0000 7f1596a08640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f15900cfd90 0x7f15900d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.941+0000 7f1596a08640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15900d0fd0 0x7f1590161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.941+0000 7f1596a08640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f159015e8b0 con 0x7f15900d0fd0 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.941+0000 7f1596a08640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f15900d1f50 con 0x7f15900cfd90 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.941+0000 7f1596a08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f15900d2230 con 0x7f159015e4d0 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.941+0000 7f1594fa5640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15900d0fd0 0x7f1590161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.942+0000 7f1594fa5640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15900d0fd0 0x7f1590161fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48030/0 (socket says 172.21.15.172:48030) 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.942+0000 7f1594fa5640 1 -- 172.21.15.172:0/2593276811 learned_addr learned my addr 172.21.15.172:0/2593276811 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.942+0000 7f158f7fe640 1 --2- 172.21.15.172:0/2593276811 >> v2:172.21.15.177:3300/0 conn(0x7f15900cfd90 0x7f15900d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.942+0000 7f158ffff640 1 --2- 172.21.15.172:0/2593276811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f159015e4d0 0x7f15900cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.942+0000 7f1594fa5640 1 -- 172.21.15.172:0/2593276811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f159015e4d0 msgr2=0x7f15900cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.942+0000 7f1594fa5640 1 --2- 172.21.15.172:0/2593276811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f159015e4d0 0x7f15900cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.942+0000 7f1594fa5640 1 -- 172.21.15.172:0/2593276811 >> v2:172.21.15.177:3300/0 conn(0x7f15900cfd90 msgr2=0x7f15900d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:20.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.942+0000 7f1594fa5640 1 --2- 172.21.15.172:0/2593276811 >> v2:172.21.15.177:3300/0 conn(0x7f15900cfd90 0x7f15900d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.942+0000 7f1594fa5640 1 -- 172.21.15.172:0/2593276811 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15900d2510 con 0x7f15900d0fd0 2024-04-14T08:37:20.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.943+0000 7f1594fa5640 1 --2- 172.21.15.172:0/2593276811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15900d0fd0 0x7f1590161fc0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f159009dcc0 tx=0x7f158802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b744a106183f66e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:20.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.943+0000 7f158effd640 1 -- 172.21.15.172:0/2593276811 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158802c9e0 con 0x7f15900d0fd0 2024-04-14T08:37:20.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.943+0000 7f158effd640 1 -- 172.21.15.172:0/2593276811 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158802cb80 con 0x7f15900d0fd0 2024-04-14T08:37:20.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.943+0000 7f158effd640 1 -- 172.21.15.172:0/2593276811 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15880343f0 con 0x7f15900d0fd0 2024-04-14T08:37:20.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.943+0000 7f1596a08640 1 -- 172.21.15.172:0/2593276811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15900d0fd0 msgr2=0x7f1590161fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.943+0000 7f1596a08640 1 --2- 172.21.15.172:0/2593276811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15900d0fd0 0x7f1590161fc0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f159009dcc0 tx=0x7f158802be40 comp rx=0 tx=0).stop 2024-04-14T08:37:20.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.943+0000 7f1596a08640 1 -- 172.21.15.172:0/2593276811 shutdown_connections 2024-04-14T08:37:20.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.943+0000 7f1596a08640 1 --2- 172.21.15.172:0/2593276811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15900d0fd0 0x7f1590161fc0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.943+0000 7f1596a08640 1 --2- 172.21.15.172:0/2593276811 >> v2:172.21.15.177:3300/0 conn(0x7f15900cfd90 0x7f15900d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.943+0000 7f1596a08640 1 --2- 172.21.15.172:0/2593276811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f159015e4d0 0x7f15900cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.943+0000 7f1596a08640 1 -- 172.21.15.172:0/2593276811 >> 172.21.15.172:0/2593276811 conn(0x7f159009d590 msgr2=0x7f15900d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:20.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.943+0000 7f1596a08640 1 -- 172.21.15.172:0/2593276811 shutdown_connections 2024-04-14T08:37:20.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.944+0000 7f1596a08640 1 -- 172.21.15.172:0/2593276811 wait complete. 2024-04-14T08:37:20.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.945+0000 7f1596a08640 1 Processor -- start 2024-04-14T08:37:20.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.945+0000 7f1596a08640 1 -- start start 2024-04-14T08:37:20.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.945+0000 7f1596a08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15900cfd90 0x7f15900b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1596a08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15900d0fd0 0x7f159014ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1596a08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f159015e4d0 0x7f15900b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1596a08640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15900d2ff0 con 0x7f159015e4d0 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1596a08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f15900d2ad0 con 0x7f15900d0fd0 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1596a08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f15900b9150 con 0x7f15900cfd90 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1594fa5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f159015e4d0 0x7f15900b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1594fa5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f159015e4d0 0x7f15900b7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48036/0 (socket says 172.21.15.172:48036) 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1594fa5640 1 -- 172.21.15.172:0/3780666588 learned_addr learned my addr 172.21.15.172:0/3780666588 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f158f7fe640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15900d0fd0 0x7f159014ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f158ffff640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15900cfd90 0x7f15900b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1594fa5640 1 -- 172.21.15.172:0/3780666588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15900cfd90 msgr2=0x7f15900b8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1594fa5640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15900cfd90 0x7f15900b8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1594fa5640 1 -- 172.21.15.172:0/3780666588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15900d0fd0 msgr2=0x7f159014ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1594fa5640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15900d0fd0 0x7f159014ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:20.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1594fa5640 1 -- 172.21.15.172:0/3780666588 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15900b9430 con 0x7f159015e4d0 2024-04-14T08:37:20.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f158f7fe640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15900d0fd0 0x7f159014ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:20.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1594fa5640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f159015e4d0 0x7f15900b7000 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1588036040 tx=0x7f1588004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:20.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f158cff9640 1 -- 172.21.15.172:0/3780666588 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588043020 con 0x7f159015e4d0 2024-04-14T08:37:20.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f158cff9640 1 -- 172.21.15.172:0/3780666588 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158803e450 con 0x7f159015e4d0 2024-04-14T08:37:20.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1596a08640 1 -- 172.21.15.172:0/3780666588 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15900b9710 con 0x7f159015e4d0 2024-04-14T08:37:20.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f158cff9640 1 -- 172.21.15.172:0/3780666588 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588009510 con 0x7f159015e4d0 2024-04-14T08:37:20.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.946+0000 7f1596a08640 1 -- 172.21.15.172:0/3780666588 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159014b2b0 con 0x7f159015e4d0 2024-04-14T08:37:20.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.947+0000 7f1596a08640 1 -- 172.21.15.172:0/3780666588 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1590068520 con 0x7f159015e4d0 2024-04-14T08:37:20.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.948+0000 7f158cff9640 1 -- 172.21.15.172:0/3780666588 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1588005d10 con 0x7f159015e4d0 2024-04-14T08:37:20.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.948+0000 7f158cff9640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f157c06c920 0x7f157c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:20.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.948+0000 7f158cff9640 1 -- 172.21.15.172:0/3780666588 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f15880ae240 con 0x7f159015e4d0 2024-04-14T08:37:20.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.948+0000 7f158ffff640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f157c06c920 0x7f157c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:20.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.949+0000 7f158ffff640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f157c06c920 0x7f157c06ede0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1584006fd0 tx=0x7f1584008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:20.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:20.953+0000 7f158cff9640 1 -- 172.21.15.172:0/3780666588 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f158804d020 con 0x7f159015e4d0 2024-04-14T08:37:21.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.174+0000 7f1596a08640 1 -- 172.21.15.172:0/3780666588 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f15900d1800 con 0x7f159015e4d0 2024-04-14T08:37:21.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.174+0000 7f158cff9640 1 -- 172.21.15.172:0/3780666588 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7f158807c400 con 0x7f159015e4d0 2024-04-14T08:37:21.175 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:37:21.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.176+0000 7f1596a08640 1 -- 172.21.15.172:0/3780666588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f157c06c920 msgr2=0x7f157c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:21.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.176+0000 7f1596a08640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f157c06c920 0x7f157c06ede0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1584006fd0 tx=0x7f1584008040 comp rx=0 tx=0).stop 2024-04-14T08:37:21.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.176+0000 7f1596a08640 1 -- 172.21.15.172:0/3780666588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f159015e4d0 msgr2=0x7f15900b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:21.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.176+0000 7f1596a08640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f159015e4d0 0x7f15900b7000 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1588036040 tx=0x7f1588004a40 comp rx=0 tx=0).stop 2024-04-14T08:37:21.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.176+0000 7f1596a08640 1 -- 172.21.15.172:0/3780666588 shutdown_connections 2024-04-14T08:37:21.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.176+0000 7f1596a08640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f157c06c920 0x7f157c06ede0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:21.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.176+0000 7f1596a08640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f159015e4d0 0x7f15900b7000 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:21.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.176+0000 7f1596a08640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15900d0fd0 0x7f159014ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.176+0000 7f1596a08640 1 --2- 172.21.15.172:0/3780666588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15900cfd90 0x7f15900b8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.176+0000 7f1596a08640 1 -- 172.21.15.172:0/3780666588 >> 172.21.15.172:0/3780666588 conn(0x7f159009d590 msgr2=0x7f15900b5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.176+0000 7f1596a08640 1 -- 172.21.15.172:0/3780666588 shutdown_connections 2024-04-14T08:37:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21.176+0000 7f1596a08640 1 -- 172.21.15.172:0/3780666588 wait complete. 2024-04-14T08:37:21.188 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:37:21.188 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:37:21.188 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:37:21.188 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:37:21.188 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:37:21.188 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:37:21.188 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:37:21.188 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:37:21.188 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:37:21.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:21.188 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:37:21.204 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:37:21.204 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:37:21.204 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:37:21.205 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:37:21.206 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:37:21.206 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42428sec preferred_lft 42428sec 2024-04-14T08:37:21.206 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:37:21.206 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:21.206 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:37:21.206 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:37:21.206 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:37:21.206 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:21.207 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:37:21.207 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:21.207 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:37:21.207 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:37:21.207 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:37:21.207 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:21.207 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:37:21.207 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:37:21.207 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:37:21.207 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:37:21.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:21.208 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:37:21.270 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:37:21.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:21.272 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:37:21.327 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:37:21.327 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:37:21.327 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:37:21.328 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:37:21.328 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:37:21.328 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:37:21.328 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:37:21.328 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:37:21.328 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:37:21.328 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:37:21.328 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:37:21.372 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:37:21.372+0000 7f81e5be5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:37:21.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:37:21.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:37:21.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:21.463 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:37:21.518 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:37:21.518 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:37:21.519 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:37:21.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:21.519 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:37:21.586 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:37:21.586 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:37:21.641 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:37:21.641 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:37:21.642 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:37:21.695 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:37:21.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:21.696 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:37:21.780 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:37:21.781 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:37:21.781 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:37:21.789 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:37:21.789 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:37:21.789 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:37:21.789 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:37:21.789 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:37:21.794 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:37:21.794 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:37:21.794 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:37:21.794 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:37:21.794 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:37:21.794 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:37:21.795 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:37:21.795 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:37:21.795 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:37:21.795 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:37:21.795 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:37:21.795 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:37:21.795 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:37:21.796 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:37:21.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:21.796 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:37:21.823 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:37:21.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:37:21.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:37:21.919 INFO:teuthology.orchestra.run.smithi172.stdout:client6700 2024-04-14T08:37:21.920 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:37:21.920 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:37:21.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:21.937 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-14T08:37:21.993 INFO:teuthology.orchestra.run.smithi172.stdout:file_undamaged 2024-04-14T08:37:21.995 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:37:21.995 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:37:21.995 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:37:21.995 DEBUG:teuthology.orchestra.run.smithi172:> import json 2024-04-14T08:37:21.995 DEBUG:teuthology.orchestra.run.smithi172:> import sys 2024-04-14T08:37:21.995 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:37:21.995 DEBUG:teuthology.orchestra.run.smithi172:> try: 2024-04-14T08:37:21.995 DEBUG:teuthology.orchestra.run.smithi172:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-14T08:37:21.996 DEBUG:teuthology.orchestra.run.smithi172:> except OSError as e: 2024-04-14T08:37:21.996 DEBUG:teuthology.orchestra.run.smithi172:> sys.exit(e.errno) 2024-04-14T08:37:21.996 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:37:21.996 DEBUG:teuthology.orchestra.run.smithi172:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-14T08:37:21.996 DEBUG:teuthology.orchestra.run.smithi172:> print(json.dumps( 2024-04-14T08:37:21.996 DEBUG:teuthology.orchestra.run.smithi172:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-14T08:37:21.996 DEBUG:teuthology.orchestra.run.smithi172:> indent=2)) 2024-04-14T08:37:21.996 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:37:22.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-14T08:37:22.111 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-14T08:37:22.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:22.112 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-14T08:37:22.167 INFO:teuthology.orchestra.run.smithi172.stdout:2 2024-04-14T08:37:22.167 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:37:22.167 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:37:22.223 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:37:22.223 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:37:22.279 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:37:22.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.419+0000 7f8409ead640 1 Processor -- start 2024-04-14T08:37:22.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.420+0000 7f8409ead640 1 -- start start 2024-04-14T08:37:22.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.420+0000 7f8409ead640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84040cedd0 0x7f84041694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.420+0000 7f8409ead640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84040cf320 con 0x7f84040d2830 2024-04-14T08:37:22.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.420+0000 7f8409ead640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84040cf600 con 0x7f840415e890 2024-04-14T08:37:22.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.420+0000 7f8409ead640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f84040cff30 con 0x7f84040cedd0 2024-04-14T08:37:22.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.420+0000 7f84037fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f84040d2830 0x7f84040d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:42922/0 (socket says 172.21.15.172:42922) 2024-04-14T08:37:22.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.420+0000 7f84037fe640 1 -- 172.21.15.172:0/2855544329 learned_addr learned my addr 172.21.15.172:0/2855544329 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:22.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8403fff640 1 --2- 172.21.15.172:0/2855544329 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84040cedd0 0x7f84041694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1973546995 0 0) 0x7f84040cf320 con 0x7f84040d2830 2024-04-14T08:37:22.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83f0003520 con 0x7f84040d2830 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1820732092 0 0) 0x7f83f0003520 con 0x7f84040d2830 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84040cedd0 msgr2=0x7f84041694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 --2- 172.21.15.172:0/2855544329 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84040cedd0 0x7f84041694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 >> v1:172.21.15.177:6789/0 conn(0x7f840415e890 legacy=0x7f84040d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84040cf8e0 con 0x7f84040d2830 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f83f4001670 con 0x7f84040d2830 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 >> v1:172.21.15.172:6789/0 conn(0x7f84040d2830 legacy=0x7f84040d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 --2- 172.21.15.172:0/2855544329 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84040cedd0 0x7f83f00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 --2- 172.21.15.172:0/2855544329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f0003f20 0x7f83f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 --2- 172.21.15.172:0/2855544329 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83f0004d40 0x7f83f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84040cff30 con 0x7f83f0003f20 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f83f000ba20 con 0x7f83f0004d40 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f83f000bc20 con 0x7f84040cedd0 2024-04-14T08:37:22.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f83faffd640 1 --2- 172.21.15.172:0/2855544329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f0003f20 0x7f83f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f84037fe640 1 --2- 172.21.15.172:0/2855544329 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84040cedd0 0x7f83f00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f8403fff640 1 --2- 172.21.15.172:0/2855544329 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83f0004d40 0x7f83f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f83faffd640 1 -- 172.21.15.172:0/2855544329 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84040cedd0 msgr2=0x7f83f00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f83faffd640 1 --2- 172.21.15.172:0/2855544329 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84040cedd0 0x7f83f00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f83faffd640 1 -- 172.21.15.172:0/2855544329 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83f0004d40 msgr2=0x7f83f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f83faffd640 1 --2- 172.21.15.172:0/2855544329 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83f0004d40 0x7f83f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.421+0000 7f83faffd640 1 -- 172.21.15.172:0/2855544329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f0003520 con 0x7f83f0003f20 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.422+0000 7f83faffd640 1 --2- 172.21.15.172:0/2855544329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f0003f20 0x7f83f0004800 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f83ec001860 tx=0x7f83f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230b11f44b63b3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.422+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec001430 con 0x7f83f0003f20 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.422+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ec032620 con 0x7f83f0003f20 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.422+0000 7f8402ffd640 1 -- 172.21.15.172:0/2855544329 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec032a10 con 0x7f83f0003f20 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.423+0000 7f8409ead640 1 -- 172.21.15.172:0/2855544329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f0003f20 msgr2=0x7f83f0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:22.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.423+0000 7f8409ead640 1 --2- 172.21.15.172:0/2855544329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83f0003f20 0x7f83f0004800 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f83ec001860 tx=0x7f83f0005830 comp rx=0 tx=0).stop 2024-04-14T08:37:22.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.423+0000 7f8403fff640 1 -- 172.21.15.172:0/2855544329 reap_dead start 2024-04-14T08:37:22.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.423+0000 7f8409ead640 1 -- 172.21.15.172:0/2855544329 shutdown_connections 2024-04-14T08:37:22.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.423+0000 7f8409ead640 1 -- 172.21.15.172:0/2855544329 >> 172.21.15.172:0/2855544329 conn(0x7f840409d590 msgr2=0x7f84040d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:22.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.423+0000 7f8409ead640 1 -- 172.21.15.172:0/2855544329 shutdown_connections 2024-04-14T08:37:22.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.423+0000 7f8409ead640 1 -- 172.21.15.172:0/2855544329 wait complete. 2024-04-14T08:37:22.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.424+0000 7f8409ead640 1 Processor -- start 2024-04-14T08:37:22.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.425+0000 7f8409ead640 1 -- start start 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.425+0000 7f8409ead640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83f0003f20 0x7f84040c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.425+0000 7f8409ead640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84040ca4b0 0x7f840415c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.425+0000 7f8409ead640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8404152690 0x7f840415c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.425+0000 7f8409ead640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84040b4720 con 0x7f84040ca4b0 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.425+0000 7f8409ead640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f84040d2350 con 0x7f8404152690 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.425+0000 7f8409ead640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f84040b48a0 con 0x7f83f0003f20 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.425+0000 7f83faffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84040ca4b0 0x7f840415c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.425+0000 7f83faffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84040ca4b0 0x7f840415c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48064/0 (socket says 172.21.15.172:48064) 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.425+0000 7f83faffd640 1 -- 172.21.15.172:0/3627059298 learned_addr learned my addr 172.21.15.172:0/3627059298 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f8403fff640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8404152690 0x7f840415c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f84037fe640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83f0003f20 0x7f84040c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f83faffd640 1 -- 172.21.15.172:0/3627059298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83f0003f20 msgr2=0x7f84040c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:22.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f83faffd640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83f0003f20 0x7f84040c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f83faffd640 1 -- 172.21.15.172:0/3627059298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8404152690 msgr2=0x7f840415c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f83faffd640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8404152690 0x7f840415c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f83faffd640 1 -- 172.21.15.172:0/3627059298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84040ca9e0 con 0x7f84040ca4b0 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f8403fff640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8404152690 0x7f840415c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f84037fe640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83f0003f20 0x7f84040c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f83faffd640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84040ca4b0 0x7f840415c3b0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0018c0 tx=0x7f83ec001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f8400ff9640 1 -- 172.21.15.172:0/3627059298 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec040070 con 0x7f84040ca4b0 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f8400ff9640 1 -- 172.21.15.172:0/3627059298 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ec007050 con 0x7f84040ca4b0 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f8409ead640 1 -- 172.21.15.172:0/3627059298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840415f720 con 0x7f84040ca4b0 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f8400ff9640 1 -- 172.21.15.172:0/3627059298 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec007400 con 0x7f84040ca4b0 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.426+0000 7f8409ead640 1 -- 172.21.15.172:0/3627059298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840415fce0 con 0x7f84040ca4b0 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.427+0000 7f8409ead640 1 -- 172.21.15.172:0/3627059298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f840406deb0 con 0x7f84040ca4b0 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.427+0000 7f8400ff9640 1 -- 172.21.15.172:0/3627059298 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83ec043070 con 0x7f84040ca4b0 2024-04-14T08:37:22.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.428+0000 7f8400ff9640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f83e406c920 0x7f83e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.428+0000 7f8400ff9640 1 -- 172.21.15.172:0/3627059298 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f83ec0ad9f0 con 0x7f84040ca4b0 2024-04-14T08:37:22.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.428+0000 7f84037fe640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f83e406c920 0x7f83e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.429+0000 7f84037fe640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f83e406c920 0x7f83e406ede0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f84041f7a00 tx=0x7f83f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:22.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.433+0000 7f8400ff9640 1 -- 172.21.15.172:0/3627059298 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f840406deb0 con 0x7f84040ca4b0 2024-04-14T08:37:22.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.610+0000 7f8409ead640 1 -- 172.21.15.172:0/3627059298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8404068520 con 0x7f84040ca4b0 2024-04-14T08:37:22.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.611+0000 7f8400ff9640 1 -- 172.21.15.172:0/3627059298 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+17136 (secure 0 0 0) 0x7f83ec07bc40 con 0x7f84040ca4b0 2024-04-14T08:37:22.612 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:22.612 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":75,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:37:15.356496+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:36:59.342045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-14T08:37:00.059338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/2134832221":"2024-04-15T08:37:15.356472+0000","172.21.15.172:6835/2134832221":"2024-04-15T08:37:15.356472+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:37:22.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.613+0000 7f8409ead640 1 -- 172.21.15.172:0/3627059298 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f83e406c920 msgr2=0x7f83e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:22.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.613+0000 7f8409ead640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f83e406c920 0x7f83e406ede0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f84041f7a00 tx=0x7f83f4009040 comp rx=0 tx=0).stop 2024-04-14T08:37:22.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.613+0000 7f8409ead640 1 -- 172.21.15.172:0/3627059298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84040ca4b0 msgr2=0x7f840415c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:22.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.613+0000 7f8409ead640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84040ca4b0 0x7f840415c3b0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0018c0 tx=0x7f83ec001230 comp rx=0 tx=0).stop 2024-04-14T08:37:22.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.613+0000 7f8409ead640 1 -- 172.21.15.172:0/3627059298 shutdown_connections 2024-04-14T08:37:22.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.613+0000 7f8409ead640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f83e406c920 0x7f83e406ede0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.613+0000 7f8409ead640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8404152690 0x7f840415c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.613+0000 7f8409ead640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84040ca4b0 0x7f840415c3b0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.613+0000 7f8409ead640 1 --2- 172.21.15.172:0/3627059298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f83f0003f20 0x7f84040c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.613+0000 7f8409ead640 1 -- 172.21.15.172:0/3627059298 >> 172.21.15.172:0/3627059298 conn(0x7f840409d590 msgr2=0x7f84040d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:22.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.613+0000 7f8409ead640 1 -- 172.21.15.172:0/3627059298 shutdown_connections 2024-04-14T08:37:22.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.613+0000 7f8409ead640 1 -- 172.21.15.172:0/3627059298 wait complete. 2024-04-14T08:37:22.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:36:59.342045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:37:00.059338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:37:22.626 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:37:22.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.730+0000 7f8dbe17b640 1 Processor -- start 2024-04-14T08:37:22.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.730+0000 7f8dbe17b640 1 -- start start 2024-04-14T08:37:22.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.730+0000 7f8dbe17b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8db815e430 0x7f8db8162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.730+0000 7f8dbe17b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8db8162540 0x7f8db81629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.730+0000 7f8dbe17b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8db80b3a30 0x7f8db80b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.730+0000 7f8dbe17b640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f8db80cefb0 con 0x7f8db815e430 2024-04-14T08:37:22.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.730+0000 7f8dbe17b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f8db80cf290 con 0x7f8db80b3a30 2024-04-14T08:37:22.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.730+0000 7f8dbe17b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8db80cf570 con 0x7f8db8162540 2024-04-14T08:37:22.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.730+0000 7f8db77fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8db815e430 0x7f8db8162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.731+0000 7f8db7fff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8db80b3a30 0x7f8db80b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.731+0000 7f8db77fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8db815e430 0x7f8db8162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48072/0 (socket says 172.21.15.172:48072) 2024-04-14T08:37:22.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.731+0000 7f8db77fe640 1 -- 172.21.15.172:0/1290283083 learned_addr learned my addr 172.21.15.172:0/1290283083 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:22.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.731+0000 7f8db6ffd640 1 --2- 172.21.15.172:0/1290283083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8db8162540 0x7f8db81629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.731+0000 7f8db7fff640 1 -- 172.21.15.172:0/1290283083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8db8162540 msgr2=0x7f8db81629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:22.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.731+0000 7f8db7fff640 1 --2- 172.21.15.172:0/1290283083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8db8162540 0x7f8db81629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.731+0000 7f8db7fff640 1 -- 172.21.15.172:0/1290283083 >> v2:172.21.15.172:3300/0 conn(0x7f8db815e430 msgr2=0x7f8db8162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:22.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.731+0000 7f8db7fff640 1 --2- 172.21.15.172:0/1290283083 >> v2:172.21.15.172:3300/0 conn(0x7f8db815e430 0x7f8db8162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.731+0000 7f8db7fff640 1 -- 172.21.15.172:0/1290283083 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db80cf850 con 0x7f8db80b3a30 2024-04-14T08:37:22.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.731+0000 7f8db77fe640 1 --2- 172.21.15.172:0/1290283083 >> v2:172.21.15.172:3300/0 conn(0x7f8db815e430 0x7f8db8162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:37:22.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.731+0000 7f8db6ffd640 1 --2- 172.21.15.172:0/1290283083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8db8162540 0x7f8db81629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:37:22.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8db7fff640 1 --2- 172.21.15.172:0/1290283083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8db80b3a30 0x7f8db80b3e60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8db80a2180 tx=0x7f8db002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4836661c35fb28c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:22.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8db67fc640 1 -- 172.21.15.172:0/1290283083 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db00315a0 con 0x7f8db80b3a30 2024-04-14T08:37:22.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8db67fc640 1 -- 172.21.15.172:0/1290283083 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db0037070 con 0x7f8db80b3a30 2024-04-14T08:37:22.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8db67fc640 1 -- 172.21.15.172:0/1290283083 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0031960 con 0x7f8db80b3a30 2024-04-14T08:37:22.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8dbe17b640 1 -- 172.21.15.172:0/1290283083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8db80b3a30 msgr2=0x7f8db80b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:22.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8dbe17b640 1 --2- 172.21.15.172:0/1290283083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8db80b3a30 0x7f8db80b3e60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8db80a2180 tx=0x7f8db002b840 comp rx=0 tx=0).stop 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8dbe17b640 1 -- 172.21.15.172:0/1290283083 shutdown_connections 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8dbe17b640 1 --2- 172.21.15.172:0/1290283083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8db80b3a30 0x7f8db80b3e60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8dbe17b640 1 --2- 172.21.15.172:0/1290283083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8db8162540 0x7f8db81629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8dbe17b640 1 --2- 172.21.15.172:0/1290283083 >> v2:172.21.15.172:3300/0 conn(0x7f8db815e430 0x7f8db8162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8dbe17b640 1 -- 172.21.15.172:0/1290283083 >> 172.21.15.172:0/1290283083 conn(0x7f8db809d560 msgr2=0x7f8db80d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8dbe17b640 1 -- 172.21.15.172:0/1290283083 shutdown_connections 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.732+0000 7f8dbe17b640 1 -- 172.21.15.172:0/1290283083 wait complete. 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.734+0000 7f8dbe17b640 1 Processor -- start 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.734+0000 7f8dbe17b640 1 -- start start 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.734+0000 7f8dbe17b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8db80b3a30 0x7f8db80ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.734+0000 7f8dbe17b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8db815e430 0x7f8db80cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.734+0000 7f8dbe17b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db8162540 0x7f8db80c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.734+0000 7f8dbe17b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8db80d4e20 con 0x7f8db8162540 2024-04-14T08:37:22.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.734+0000 7f8dbe17b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8db815e810 con 0x7f8db80b3a30 2024-04-14T08:37:22.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.734+0000 7f8dbe17b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8db80d0050 con 0x7f8db815e430 2024-04-14T08:37:22.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8db7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db8162540 0x7f8db80c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8db7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db8162540 0x7f8db80c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48076/0 (socket says 172.21.15.172:48076) 2024-04-14T08:37:22.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8db7fff640 1 -- 172.21.15.172:0/2928692124 learned_addr learned my addr 172.21.15.172:0/2928692124 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:22.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8db77fe640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8db80b3a30 0x7f8db80ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8db6ffd640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8db815e430 0x7f8db80cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8db7fff640 1 -- 172.21.15.172:0/2928692124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8db815e430 msgr2=0x7f8db80cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:22.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8db7fff640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8db815e430 0x7f8db80cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8db7fff640 1 -- 172.21.15.172:0/2928692124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8db80b3a30 msgr2=0x7f8db80ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:22.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8db7fff640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8db80b3a30 0x7f8db80ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8db7fff640 1 -- 172.21.15.172:0/2928692124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db80cb830 con 0x7f8db8162540 2024-04-14T08:37:22.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8db7fff640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db8162540 0x7f8db80c7fc0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8db0034040 tx=0x7f8db00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:22.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8daffff640 1 -- 172.21.15.172:0/2928692124 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0032630 con 0x7f8db8162540 2024-04-14T08:37:22.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8daffff640 1 -- 172.21.15.172:0/2928692124 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db00327f0 con 0x7f8db8162540 2024-04-14T08:37:22.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8dbe17b640 1 -- 172.21.15.172:0/2928692124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db80cbab0 con 0x7f8db8162540 2024-04-14T08:37:22.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8daffff640 1 -- 172.21.15.172:0/2928692124 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0032be0 con 0x7f8db8162540 2024-04-14T08:37:22.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.735+0000 7f8dbe17b640 1 -- 172.21.15.172:0/2928692124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db80cc070 con 0x7f8db8162540 2024-04-14T08:37:22.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.736+0000 7f8dbe17b640 1 -- 172.21.15.172:0/2928692124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db8068590 con 0x7f8db8162540 2024-04-14T08:37:22.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.737+0000 7f8daffff640 1 -- 172.21.15.172:0/2928692124 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8db004c020 con 0x7f8db8162540 2024-04-14T08:37:22.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.737+0000 7f8daffff640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8d9806c920 0x7f8d9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:22.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.737+0000 7f8daffff640 1 -- 172.21.15.172:0/2928692124 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f8db002f020 con 0x7f8db8162540 2024-04-14T08:37:22.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.737+0000 7f8db77fe640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8d9806c920 0x7f8d9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:22.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.738+0000 7f8db77fe640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8d9806c920 0x7f8d9806ede0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8da80072c0 tx=0x7f8da800d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:22.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.742+0000 7f8daffff640 1 -- 172.21.15.172:0/2928692124 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8db80cb830 con 0x7f8db8162540 2024-04-14T08:37:22.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.922+0000 7f8dbe17b640 1 -- 172.21.15.172:0/2928692124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8db80d18a0 con 0x7f8db8162540 2024-04-14T08:37:22.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.922+0000 7f8daffff640 1 -- 172.21.15.172:0/2928692124 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f8db0040070 con 0x7f8db8162540 2024-04-14T08:37:22.923 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:37:22.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.924+0000 7f8dbe17b640 1 -- 172.21.15.172:0/2928692124 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8d9806c920 msgr2=0x7f8d9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:22.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.924+0000 7f8dbe17b640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8d9806c920 0x7f8d9806ede0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8da80072c0 tx=0x7f8da800d040 comp rx=0 tx=0).stop 2024-04-14T08:37:22.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.924+0000 7f8dbe17b640 1 -- 172.21.15.172:0/2928692124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db8162540 msgr2=0x7f8db80c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:22.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.924+0000 7f8dbe17b640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db8162540 0x7f8db80c7fc0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8db0034040 tx=0x7f8db00046c0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.924+0000 7f8dbe17b640 1 -- 172.21.15.172:0/2928692124 shutdown_connections 2024-04-14T08:37:22.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.924+0000 7f8dbe17b640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8d9806c920 0x7f8d9806ede0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.924+0000 7f8dbe17b640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db8162540 0x7f8db80c7fc0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.924+0000 7f8dbe17b640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8db815e430 0x7f8db80cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.924+0000 7f8dbe17b640 1 --2- 172.21.15.172:0/2928692124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8db80b3a30 0x7f8db80ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:22.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.924+0000 7f8dbe17b640 1 -- 172.21.15.172:0/2928692124 >> 172.21.15.172:0/2928692124 conn(0x7f8db809d560 msgr2=0x7f8db80b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:22.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.924+0000 7f8dbe17b640 1 -- 172.21.15.172:0/2928692124 shutdown_connections 2024-04-14T08:37:22.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:22.924+0000 7f8dbe17b640 1 -- 172.21.15.172:0/2928692124 wait complete. 2024-04-14T08:37:22.936 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:37:23.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.040+0000 7f871d2ad640 1 Processor -- start 2024-04-14T08:37:23.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.040+0000 7f871d2ad640 1 -- start start 2024-04-14T08:37:23.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.041+0000 7f871d2ad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871815c420 0x7f87180b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.041+0000 7f871d2ad640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f87180ceb00 0x7f87180cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.041+0000 7f871d2ad640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87180ccdd0 con 0x7f871806f3e0 2024-04-14T08:37:23.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.041+0000 7f871d2ad640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f87180d30a0 con 0x7f87180ceb00 2024-04-14T08:37:23.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.041+0000 7f871d2ad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f87180cf490 con 0x7f871815c420 2024-04-14T08:37:23.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.041+0000 7f87177fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f87180ceb00 0x7f87180cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.041+0000 7f87177fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f87180ceb00 0x7f87180cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48078/0 (socket says 172.21.15.172:48078) 2024-04-14T08:37:23.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.041+0000 7f87177fe640 1 -- 172.21.15.172:0/2789468013 learned_addr learned my addr 172.21.15.172:0/2789468013 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:23.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.041+0000 7f8716ffd640 1 --2- 172.21.15.172:0/2789468013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871815c420 0x7f87180b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.042+0000 7f87177fe640 1 -- 172.21.15.172:0/2789468013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871815c420 msgr2=0x7f87180b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:23.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.042+0000 7f87177fe640 1 --2- 172.21.15.172:0/2789468013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871815c420 0x7f87180b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.042+0000 7f87177fe640 1 -- 172.21.15.172:0/2789468013 >> v1:172.21.15.177:6789/0 conn(0x7f871806f3e0 legacy=0x7f87180d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.042+0000 7f87177fe640 1 -- 172.21.15.172:0/2789468013 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87180d2ae0 con 0x7f87180ceb00 2024-04-14T08:37:23.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.042+0000 7f87177fe640 1 --2- 172.21.15.172:0/2789468013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87180ceb00 0x7f87180cef50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8708009870 tx=0x7f870802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a705d3bd430ba425 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.042+0000 7f8715ffb640 1 -- 172.21.15.172:0/2789468013 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708034070 con 0x7f87180ceb00 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.042+0000 7f8715ffb640 1 -- 172.21.15.172:0/2789468013 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8708030840 con 0x7f87180ceb00 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.043+0000 7f8715ffb640 1 -- 172.21.15.172:0/2789468013 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708030c10 con 0x7f87180ceb00 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.043+0000 7f871d2ad640 1 -- 172.21.15.172:0/2789468013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87180ceb00 msgr2=0x7f87180cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.043+0000 7f871d2ad640 1 --2- 172.21.15.172:0/2789468013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87180ceb00 0x7f87180cef50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8708009870 tx=0x7f870802b980 comp rx=0 tx=0).stop 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.043+0000 7f871d2ad640 1 -- 172.21.15.172:0/2789468013 shutdown_connections 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.043+0000 7f871d2ad640 1 --2- 172.21.15.172:0/2789468013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87180ceb00 0x7f87180cef50 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.043+0000 7f871d2ad640 1 --2- 172.21.15.172:0/2789468013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871815c420 0x7f87180b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.043+0000 7f871d2ad640 1 -- 172.21.15.172:0/2789468013 >> 172.21.15.172:0/2789468013 conn(0x7f871808c000 msgr2=0x7f87180d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.043+0000 7f871d2ad640 1 -- 172.21.15.172:0/2789468013 shutdown_connections 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.043+0000 7f871d2ad640 1 -- 172.21.15.172:0/2789468013 wait complete. 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.044+0000 7f871d2ad640 1 Processor -- start 2024-04-14T08:37:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.044+0000 7f871d2ad640 1 -- start start 2024-04-14T08:37:23.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f871d2ad640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87180ceb00 0x7f8718156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f871d2ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f871815c420 0x7f87180c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f871d2ad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871814d460 0x7f871814dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f871d2ad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f87180cd070 con 0x7f871815c420 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f871d2ad640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f871806f870 con 0x7f87180ceb00 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f871d2ad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f87180b11e0 con 0x7f871814d460 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f87167fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f871815c420 0x7f87180c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f87167fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f871815c420 0x7f87180c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48088/0 (socket says 172.21.15.172:48088) 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f87167fc640 1 -- 172.21.15.172:0/3111248914 learned_addr learned my addr 172.21.15.172:0/3111248914 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f8716ffd640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87180ceb00 0x7f8718156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f87177fe640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871814d460 0x7f871814dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f87167fc640 1 -- 172.21.15.172:0/3111248914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871814d460 msgr2=0x7f871814dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f87167fc640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871814d460 0x7f871814dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.045+0000 7f87167fc640 1 -- 172.21.15.172:0/3111248914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87180ceb00 msgr2=0x7f8718156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.046+0000 7f87167fc640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87180ceb00 0x7f8718156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.046+0000 7f87167fc640 1 -- 172.21.15.172:0/3111248914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87180c6100 con 0x7f871815c420 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.046+0000 7f8716ffd640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87180ceb00 0x7f8718156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.046+0000 7f87177fe640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871814d460 0x7f871814dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.046+0000 7f87167fc640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f871815c420 0x7f87180c5990 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f870000c400 tx=0x7f870000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.046+0000 7f870f7fe640 1 -- 172.21.15.172:0/3111248914 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87000052d0 con 0x7f871815c420 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.046+0000 7f870f7fe640 1 -- 172.21.15.172:0/3111248914 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8700005490 con 0x7f871815c420 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.046+0000 7f871d2ad640 1 -- 172.21.15.172:0/3111248914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87180c63e0 con 0x7f871815c420 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.046+0000 7f870f7fe640 1 -- 172.21.15.172:0/3111248914 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700007260 con 0x7f871815c420 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.046+0000 7f871d2ad640 1 -- 172.21.15.172:0/3111248914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87180c69a0 con 0x7f871815c420 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.047+0000 7f871d2ad640 1 -- 172.21.15.172:0/3111248914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8718075490 con 0x7f871815c420 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.047+0000 7f870f7fe640 1 -- 172.21.15.172:0/3111248914 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8700007400 con 0x7f871815c420 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.048+0000 7f870f7fe640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f86f806c920 0x7f86f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.048+0000 7f870f7fe640 1 -- 172.21.15.172:0/3111248914 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f870008b240 con 0x7f871815c420 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.048+0000 7f8716ffd640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f86f806c920 0x7f86f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.048+0000 7f8716ffd640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f86f806c920 0x7f86f806ede0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8710009fd0 tx=0x7f871000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:23.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.052+0000 7f870f7fe640 1 -- 172.21.15.172:0/3111248914 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8700059490 con 0x7f871815c420 2024-04-14T08:37:23.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.234+0000 7f871d2ad640 1 -- 172.21.15.172:0/3111248914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f87180d2820 con 0x7f871815c420 2024-04-14T08:37:23.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.234+0000 7f870f7fe640 1 -- 172.21.15.172:0/3111248914 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) ==== 104+0+11 (secure 0 0 0) 0x7f8700058d90 con 0x7f871815c420 2024-04-14T08:37:23.235 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:37:23.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.236+0000 7f871d2ad640 1 -- 172.21.15.172:0/3111248914 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f86f806c920 msgr2=0x7f86f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.236+0000 7f871d2ad640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f86f806c920 0x7f86f806ede0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8710009fd0 tx=0x7f871000b040 comp rx=0 tx=0).stop 2024-04-14T08:37:23.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.236+0000 7f871d2ad640 1 -- 172.21.15.172:0/3111248914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f871815c420 msgr2=0x7f87180c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.236+0000 7f871d2ad640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f871815c420 0x7f87180c5990 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f870000c400 tx=0x7f870000c8c0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.236+0000 7f871d2ad640 1 -- 172.21.15.172:0/3111248914 shutdown_connections 2024-04-14T08:37:23.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.236+0000 7f871d2ad640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f86f806c920 0x7f86f806ede0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.236+0000 7f871d2ad640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f871814d460 0x7f871814dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.236+0000 7f871d2ad640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f871815c420 0x7f87180c5990 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.236+0000 7f871d2ad640 1 --2- 172.21.15.172:0/3111248914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87180ceb00 0x7f8718156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.237+0000 7f871d2ad640 1 -- 172.21.15.172:0/3111248914 >> 172.21.15.172:0/3111248914 conn(0x7f871808c000 msgr2=0x7f8718158730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:23.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.237+0000 7f871d2ad640 1 -- 172.21.15.172:0/3111248914 shutdown_connections 2024-04-14T08:37:23.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.237+0000 7f871d2ad640 1 -- 172.21.15.172:0/3111248914 wait complete. 2024-04-14T08:37:23.248 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:37:23.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.351+0000 7f2ee3236640 1 Processor -- start 2024-04-14T08:37:23.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.352+0000 7f2ee3236640 1 -- start start 2024-04-14T08:37:23.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.352+0000 7f2ee3236640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2edc162ba0 0x7f2edc0d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.352+0000 7f2ee3236640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2edc0cefb0 con 0x7f2edc15e430 2024-04-14T08:37:23.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.352+0000 7f2ee3236640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2edc0cf290 con 0x7f2edc162710 2024-04-14T08:37:23.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.352+0000 7f2ee3236640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2edc0cf570 con 0x7f2edc162ba0 2024-04-14T08:37:23.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.352+0000 7f2ee0fd2640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2edc15e430 0x7f2edc162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:42936/0 (socket says 172.21.15.172:42936) 2024-04-14T08:37:23.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.352+0000 7f2ee0fd2640 1 -- 172.21.15.172:0/4220472552 learned_addr learned my addr 172.21.15.172:0/4220472552 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:23.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2ee17d3640 1 --2- 172.21.15.172:0/4220472552 >> v2:172.21.15.177:3300/0 conn(0x7f2edc162ba0 0x7f2edc0d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193926138 0 0) 0x7f2edc0cefb0 con 0x7f2edc15e430 2024-04-14T08:37:23.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ecc003520 con 0x7f2edc15e430 2024-04-14T08:37:23.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1153113659 0 0) 0x7f2ecc003520 con 0x7f2edc15e430 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 >> v2:172.21.15.177:3300/0 conn(0x7f2edc162ba0 msgr2=0x7f2edc0d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 --2- 172.21.15.172:0/4220472552 >> v2:172.21.15.177:3300/0 conn(0x7f2edc162ba0 0x7f2edc0d2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 >> v1:172.21.15.177:6789/0 conn(0x7f2edc162710 legacy=0x7f2edc0d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc0cf850 con 0x7f2edc15e430 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f2ed0001670 con 0x7f2edc15e430 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 >> v1:172.21.15.172:6789/0 conn(0x7f2edc15e430 legacy=0x7f2edc162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ecc0039b0 0x7f2ecc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 0x7f2ecc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ecc0051c0 0x7f2ecc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc0cf570 con 0x7f2ecc004800 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc00c060 con 0x7f2ecc0051c0 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2ecc00c260 con 0x7f2ecc0039b0 2024-04-14T08:37:23.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edbfff640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 0x7f2ecc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2ee0fd2640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ecc0039b0 0x7f2ecc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2ee17d3640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ecc0051c0 0x7f2ecc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edbfff640 1 -- 172.21.15.172:0/4220472552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ecc0039b0 msgr2=0x7f2ecc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edbfff640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ecc0039b0 0x7f2ecc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edbfff640 1 -- 172.21.15.172:0/4220472552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ecc0051c0 msgr2=0x7f2ecc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edbfff640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ecc0051c0 0x7f2ecc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2edbfff640 1 -- 172.21.15.172:0/4220472552 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ecc003520 con 0x7f2ecc004800 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2ee0fd2640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ecc0039b0 0x7f2ecc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2ee17d3640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ecc0051c0 0x7f2ecc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.353+0000 7f2ee17d3640 1 -- 172.21.15.172:0/4220472552 reap_dead start 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.354+0000 7f2edbfff640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 0x7f2ecc004c80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8001750 tx=0x7f2ecc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ebcfdf4d8a7fba server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.354+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8001370 con 0x7f2ecc004800 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.354+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec80052d0 con 0x7f2ecc004800 2024-04-14T08:37:23.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.354+0000 7f2edb7fe640 1 -- 172.21.15.172:0/4220472552 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8003830 con 0x7f2ecc004800 2024-04-14T08:37:23.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.355+0000 7f2ee3236640 1 -- 172.21.15.172:0/4220472552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 msgr2=0x7f2ecc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.355+0000 7f2ee3236640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 0x7f2ecc004c80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8001750 tx=0x7f2ecc005830 comp rx=0 tx=0).stop 2024-04-14T08:37:23.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.355+0000 7f2ee3236640 1 -- 172.21.15.172:0/4220472552 shutdown_connections 2024-04-14T08:37:23.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.355+0000 7f2ee3236640 1 --2- 172.21.15.172:0/4220472552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 0x7f2ecc004c80 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.355+0000 7f2ee3236640 1 -- 172.21.15.172:0/4220472552 >> 172.21.15.172:0/4220472552 conn(0x7f2edc09d560 msgr2=0x7f2edc0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:23.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.355+0000 7f2ee3236640 1 -- 172.21.15.172:0/4220472552 shutdown_connections 2024-04-14T08:37:23.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.355+0000 7f2ee3236640 1 -- 172.21.15.172:0/4220472552 wait complete. 2024-04-14T08:37:23.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.356+0000 7f2ee3236640 1 Processor -- start 2024-04-14T08:37:23.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.356+0000 7f2ee3236640 1 -- start start 2024-04-14T08:37:23.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee3236640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 0x7f2edc0b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee3236640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2edc1f7100 0x7f2edc0b8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee3236640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2edc0b4ae0 0x7f2edc0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee3236640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc0d4e20 con 0x7f2ecc004800 2024-04-14T08:37:23.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee3236640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc15e810 con 0x7f2edc1f7100 2024-04-14T08:37:23.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee3236640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2edc0d0050 con 0x7f2edc0b4ae0 2024-04-14T08:37:23.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee0fd2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 0x7f2edc0b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee0fd2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 0x7f2edc0b7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48112/0 (socket says 172.21.15.172:48112) 2024-04-14T08:37:23.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee0fd2640 1 -- 172.21.15.172:0/290284597 learned_addr learned my addr 172.21.15.172:0/290284597 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:23.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee0fd2640 1 -- 172.21.15.172:0/290284597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2edc0b4ae0 msgr2=0x7f2edc0b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee0fd2640 1 --2- 172.21.15.172:0/290284597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2edc0b4ae0 0x7f2edc0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee0fd2640 1 -- 172.21.15.172:0/290284597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2edc1f7100 msgr2=0x7f2edc0b8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee0fd2640 1 --2- 172.21.15.172:0/290284597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2edc1f7100 0x7f2edc0b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee0fd2640 1 -- 172.21.15.172:0/290284597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc0b54a0 con 0x7f2ecc004800 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.357+0000 7f2ee0fd2640 1 --2- 172.21.15.172:0/290284597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 0x7f2edc0b7e00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000e6f0 tx=0x7f2ed000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.358+0000 7f2ed97fa640 1 -- 172.21.15.172:0/290284597 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000c640 con 0x7f2ecc004800 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.358+0000 7f2ed97fa640 1 -- 172.21.15.172:0/290284597 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed0018070 con 0x7f2ecc004800 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.358+0000 7f2ee3236640 1 -- 172.21.15.172:0/290284597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc160c90 con 0x7f2ecc004800 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.358+0000 7f2ed97fa640 1 -- 172.21.15.172:0/290284597 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000c9e0 con 0x7f2ecc004800 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.358+0000 7f2ee3236640 1 -- 172.21.15.172:0/290284597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc161250 con 0x7f2ecc004800 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.359+0000 7f2ee3236640 1 -- 172.21.15.172:0/290284597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2edc068590 con 0x7f2ecc004800 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.360+0000 7f2ed97fa640 1 -- 172.21.15.172:0/290284597 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ed000cb80 con 0x7f2ecc004800 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.360+0000 7f2ed97fa640 1 --2- 172.21.15.172:0/290284597 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ec006c8f0 0x7f2ec006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.360+0000 7f2edbfff640 1 --2- 172.21.15.172:0/290284597 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ec006c8f0 0x7f2ec006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.360+0000 7f2ed97fa640 1 -- 172.21.15.172:0/290284597 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f2ed008ba00 con 0x7f2ecc004800 2024-04-14T08:37:23.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.361+0000 7f2edbfff640 1 --2- 172.21.15.172:0/290284597 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ec006c8f0 0x7f2ec006edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8033000 tx=0x7f2ec802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:23.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.365+0000 7f2ed97fa640 1 -- 172.21.15.172:0/290284597 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2ed0059ce0 con 0x7f2ecc004800 2024-04-14T08:37:23.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.546+0000 7f2ee3236640 1 -- 172.21.15.172:0/290284597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2edc0d18a0 con 0x7f2ecc004800 2024-04-14T08:37:23.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.546+0000 7f2ed97fa640 1 -- 172.21.15.172:0/290284597 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) ==== 100+0+11 (secure 0 0 0) 0x7f2ed00595e0 con 0x7f2ecc004800 2024-04-14T08:37:23.547 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:37:23.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.548+0000 7f2ee3236640 1 -- 172.21.15.172:0/290284597 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ec006c8f0 msgr2=0x7f2ec006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.548+0000 7f2ee3236640 1 --2- 172.21.15.172:0/290284597 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ec006c8f0 0x7f2ec006edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8033000 tx=0x7f2ec802f000 comp rx=0 tx=0).stop 2024-04-14T08:37:23.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.548+0000 7f2ee3236640 1 -- 172.21.15.172:0/290284597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 msgr2=0x7f2edc0b7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.548+0000 7f2ee3236640 1 --2- 172.21.15.172:0/290284597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 0x7f2edc0b7e00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000e6f0 tx=0x7f2ed000ebb0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.548+0000 7f2ee3236640 1 -- 172.21.15.172:0/290284597 shutdown_connections 2024-04-14T08:37:23.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.548+0000 7f2ee3236640 1 --2- 172.21.15.172:0/290284597 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ec006c8f0 0x7f2ec006edb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.548+0000 7f2ee3236640 1 --2- 172.21.15.172:0/290284597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2edc0b4ae0 0x7f2edc0b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.548+0000 7f2ee3236640 1 --2- 172.21.15.172:0/290284597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2edc1f7100 0x7f2edc0b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.548+0000 7f2ee3236640 1 --2- 172.21.15.172:0/290284597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ecc004800 0x7f2edc0b7e00 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.548+0000 7f2ee3236640 1 -- 172.21.15.172:0/290284597 >> 172.21.15.172:0/290284597 conn(0x7f2edc09d560 msgr2=0x7f2edc0d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:23.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.548+0000 7f2ee3236640 1 -- 172.21.15.172:0/290284597 shutdown_connections 2024-04-14T08:37:23.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.548+0000 7f2ee3236640 1 -- 172.21.15.172:0/290284597 wait complete. 2024-04-14T08:37:23.560 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:37:23.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.665+0000 7f0b50c08640 1 Processor -- start 2024-04-14T08:37:23.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.665+0000 7f0b50c08640 1 -- start start 2024-04-14T08:37:23.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.665+0000 7f0b50c08640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0b4c15e430 0x7f0b4c162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.665+0000 7f0b50c08640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0b4c162540 0x7f0b4c1629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.665+0000 7f0b50c08640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b4c0cefb0 con 0x7f0b4c0b3a30 2024-04-14T08:37:23.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.665+0000 7f0b50c08640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0b4c0cf290 con 0x7f0b4c162540 2024-04-14T08:37:23.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.665+0000 7f0b50c08640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f0b4c0cf570 con 0x7f0b4c15e430 2024-04-14T08:37:23.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.666+0000 7f0b43fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0b4c162540 0x7f0b4c1629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.666+0000 7f0b43fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0b4c162540 0x7f0b4c1629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48126/0 (socket says 172.21.15.172:48126) 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.666+0000 7f0b43fff640 1 -- 172.21.15.172:0/624883870 learned_addr learned my addr 172.21.15.172:0/624883870 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.666+0000 7f0b43fff640 1 -- 172.21.15.172:0/624883870 >> v2:172.21.15.177:3300/0 conn(0x7f0b4c15e430 msgr2=0x7f0b4c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.666+0000 7f0b4a59c640 1 --2- 172.21.15.172:0/624883870 >> v2:172.21.15.177:3300/0 conn(0x7f0b4c15e430 0x7f0b4c162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.666+0000 7f0b43fff640 1 --2- 172.21.15.172:0/624883870 >> v2:172.21.15.177:3300/0 conn(0x7f0b4c15e430 0x7f0b4c162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.666+0000 7f0b43fff640 1 -- 172.21.15.172:0/624883870 >> v1:172.21.15.177:6789/0 conn(0x7f0b4c0b3a30 legacy=0x7f0b4c0b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.666+0000 7f0b43fff640 1 -- 172.21.15.172:0/624883870 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4c0cf850 con 0x7f0b4c162540 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.666+0000 7f0b4a59c640 1 --2- 172.21.15.172:0/624883870 >> v2:172.21.15.177:3300/0 conn(0x7f0b4c15e430 0x7f0b4c162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.667+0000 7f0b43fff640 1 --2- 172.21.15.172:0/624883870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c162540 0x7f0b4c1629d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0b6db0 tx=0x7f0b3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276d008d2ccafb7f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.667+0000 7f0b49d9b640 1 -- 172.21.15.172:0/624883870 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34035020 con 0x7f0b4c162540 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.667+0000 7f0b49d9b640 1 -- 172.21.15.172:0/624883870 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b34031870 con 0x7f0b4c162540 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.667+0000 7f0b49d9b640 1 -- 172.21.15.172:0/624883870 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34031c40 con 0x7f0b4c162540 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.667+0000 7f0b50c08640 1 -- 172.21.15.172:0/624883870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c162540 msgr2=0x7f0b4c1629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.667+0000 7f0b50c08640 1 --2- 172.21.15.172:0/624883870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c162540 0x7f0b4c1629d0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0b6db0 tx=0x7f0b3402b910 comp rx=0 tx=0).stop 2024-04-14T08:37:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.667+0000 7f0b50c08640 1 -- 172.21.15.172:0/624883870 shutdown_connections 2024-04-14T08:37:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.667+0000 7f0b50c08640 1 --2- 172.21.15.172:0/624883870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c162540 0x7f0b4c1629d0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.668+0000 7f0b50c08640 1 --2- 172.21.15.172:0/624883870 >> v2:172.21.15.177:3300/0 conn(0x7f0b4c15e430 0x7f0b4c162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.668+0000 7f0b50c08640 1 -- 172.21.15.172:0/624883870 >> 172.21.15.172:0/624883870 conn(0x7f0b4c09d560 msgr2=0x7f0b4c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.668+0000 7f0b50c08640 1 -- 172.21.15.172:0/624883870 shutdown_connections 2024-04-14T08:37:23.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.668+0000 7f0b50c08640 1 -- 172.21.15.172:0/624883870 wait complete. 2024-04-14T08:37:23.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.669+0000 7f0b50c08640 1 Processor -- start 2024-04-14T08:37:23.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.669+0000 7f0b50c08640 1 -- start start 2024-04-14T08:37:23.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b50c08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b4c0b3a30 0x7f0b4c0ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b50c08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c15e430 0x7f0b4c14d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b50c08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c162540 0x7f0b4c14d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b50c08640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c0d4e20 con 0x7f0b4c15e430 2024-04-14T08:37:23.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b50c08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c15e810 con 0x7f0b4c162540 2024-04-14T08:37:23.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b50c08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4c0d0050 con 0x7f0b4c0b3a30 2024-04-14T08:37:23.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b43fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c15e430 0x7f0b4c14d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b43fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c15e430 0x7f0b4c14d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48128/0 (socket says 172.21.15.172:48128) 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b43fff640 1 -- 172.21.15.172:0/2893422301 learned_addr learned my addr 172.21.15.172:0/2893422301 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b4a59c640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b4c0b3a30 0x7f0b4c0ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b4ad9d640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c162540 0x7f0b4c14d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b43fff640 1 -- 172.21.15.172:0/2893422301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b4c0b3a30 msgr2=0x7f0b4c0ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b43fff640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b4c0b3a30 0x7f0b4c0ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b43fff640 1 -- 172.21.15.172:0/2893422301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c162540 msgr2=0x7f0b4c14d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b43fff640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c162540 0x7f0b4c14d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b43fff640 1 -- 172.21.15.172:0/2893422301 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4c0b6e20 con 0x7f0b4c15e430 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b43fff640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c15e430 0x7f0b4c14d120 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0b34000c00 tx=0x7f0b340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b42ffd640 1 -- 172.21.15.172:0/2893422301 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34040070 con 0x7f0b4c15e430 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b42ffd640 1 -- 172.21.15.172:0/2893422301 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3403a420 con 0x7f0b4c15e430 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b42ffd640 1 -- 172.21.15.172:0/2893422301 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34035040 con 0x7f0b4c15e430 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b50c08640 1 -- 172.21.15.172:0/2893422301 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4c0b70c0 con 0x7f0b4c15e430 2024-04-14T08:37:23.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.670+0000 7f0b50c08640 1 -- 172.21.15.172:0/2893422301 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b4c0b7680 con 0x7f0b4c15e430 2024-04-14T08:37:23.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.671+0000 7f0b50c08640 1 -- 172.21.15.172:0/2893422301 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b4c068590 con 0x7f0b4c15e430 2024-04-14T08:37:23.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.672+0000 7f0b42ffd640 1 -- 172.21.15.172:0/2893422301 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b3404b060 con 0x7f0b4c15e430 2024-04-14T08:37:23.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.672+0000 7f0b42ffd640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0b2c06c8f0 0x7f0b2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:23.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.673+0000 7f0b4a59c640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0b2c06c8f0 0x7f0b2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:23.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.673+0000 7f0b42ffd640 1 -- 172.21.15.172:0/2893422301 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f0b340adab0 con 0x7f0b4c15e430 2024-04-14T08:37:23.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.673+0000 7f0b4a59c640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0b2c06c8f0 0x7f0b2c06edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00ab60 tx=0x7f0b3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:23.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.677+0000 7f0b42ffd640 1 -- 172.21.15.172:0/2893422301 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0b3407b740 con 0x7f0b4c15e430 2024-04-14T08:37:23.699 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:37:23.701 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:37:23.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.868+0000 7f0b50c08640 1 -- 172.21.15.172:0/2893422301 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0b4c0d18a0 con 0x7f0b4c15e430 2024-04-14T08:37:23.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.868+0000 7f0b42ffd640 1 -- 172.21.15.172:0/2893422301 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v75) ==== 80+0+124 (secure 0 0 0) 0x7f0b3407fa40 con 0x7f0b4c15e430 2024-04-14T08:37:23.869 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6837/2134832221 2024-04-15T08:37:15.356472+0000 2024-04-14T08:37:23.869 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6835/2134832221 2024-04-15T08:37:15.356472+0000 2024-04-14T08:37:23.869 INFO:teuthology.orchestra.run.smithi172.stderr:listed 2 entries 2024-04-14T08:37:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.870+0000 7f0b50c08640 1 -- 172.21.15.172:0/2893422301 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0b2c06c8f0 msgr2=0x7f0b2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.870+0000 7f0b50c08640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0b2c06c8f0 0x7f0b2c06edb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00ab60 tx=0x7f0b3c008040 comp rx=0 tx=0).stop 2024-04-14T08:37:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.870+0000 7f0b50c08640 1 -- 172.21.15.172:0/2893422301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c15e430 msgr2=0x7f0b4c14d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.870+0000 7f0b50c08640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c15e430 0x7f0b4c14d120 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0b34000c00 tx=0x7f0b340046c0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.870+0000 7f0b50c08640 1 -- 172.21.15.172:0/2893422301 shutdown_connections 2024-04-14T08:37:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.870+0000 7f0b50c08640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0b2c06c8f0 0x7f0b2c06edb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.870+0000 7f0b50c08640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c162540 0x7f0b4c14d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.870+0000 7f0b50c08640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c15e430 0x7f0b4c14d120 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.870+0000 7f0b50c08640 1 --2- 172.21.15.172:0/2893422301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b4c0b3a30 0x7f0b4c0ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.870+0000 7f0b50c08640 1 -- 172.21.15.172:0/2893422301 >> 172.21.15.172:0/2893422301 conn(0x7f0b4c09d560 msgr2=0x7f0b4c0b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.870+0000 7f0b50c08640 1 -- 172.21.15.172:0/2893422301 shutdown_connections 2024-04-14T08:37:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:23.870+0000 7f0b50c08640 1 -- 172.21.15.172:0/2893422301 wait complete. 2024-04-14T08:37:23.881 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:37:23.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:23.882 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:37:23.912 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:37:23.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:23.912 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:37:23.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:23.967 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-14T08:37:24.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.114+0000 7f6e88698640 1 Processor -- start 2024-04-14T08:37:24.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.114+0000 7f6e88698640 1 -- start start 2024-04-14T08:37:24.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e88698640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6e80161d00 0x7f6e800b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e88698640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e800b5490 0x7f6e800d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e88698640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6e800d2b20 0x7f6e801622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e88698640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f6e800d0ba0 con 0x7f6e800d2b20 2024-04-14T08:37:24.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e88698640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f6e800d1720 con 0x7f6e80161d00 2024-04-14T08:37:24.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e88698640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6e800d33b0 con 0x7f6e800b5490 2024-04-14T08:37:24.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e86c35640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6e800d2b20 0x7f6e801622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e86c35640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6e800d2b20 0x7f6e801622c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48130/0 (socket says 172.21.15.172:48130) 2024-04-14T08:37:24.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e86c35640 1 -- 172.21.15.172:0/2615547973 learned_addr learned my addr 172.21.15.172:0/2615547973 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e85c33640 1 --2- 172.21.15.172:0/2615547973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e800b5490 0x7f6e800d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e86434640 1 --2- 172.21.15.172:0/2615547973 >> v2:172.21.15.177:3300/0 conn(0x7f6e80161d00 0x7f6e800b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e86c35640 1 -- 172.21.15.172:0/2615547973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e800b5490 msgr2=0x7f6e800d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e86c35640 1 --2- 172.21.15.172:0/2615547973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e800b5490 0x7f6e800d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e86c35640 1 -- 172.21.15.172:0/2615547973 >> v2:172.21.15.177:3300/0 conn(0x7f6e80161d00 msgr2=0x7f6e800b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e86c35640 1 --2- 172.21.15.172:0/2615547973 >> v2:172.21.15.177:3300/0 conn(0x7f6e80161d00 0x7f6e800b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.115+0000 7f6e86c35640 1 -- 172.21.15.172:0/2615547973 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e800d1bb0 con 0x7f6e800d2b20 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.116+0000 7f6e86c35640 1 --2- 172.21.15.172:0/2615547973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e800d2b20 0x7f6e801622c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6e8008c640 tx=0x7f6e7802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58258144ac30cdd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.116+0000 7f6e85432640 1 -- 172.21.15.172:0/2615547973 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7802c9e0 con 0x7f6e800d2b20 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.116+0000 7f6e85432640 1 -- 172.21.15.172:0/2615547973 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e7802cb80 con 0x7f6e800d2b20 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.116+0000 7f6e85432640 1 -- 172.21.15.172:0/2615547973 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e780343f0 con 0x7f6e800d2b20 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.117+0000 7f6e88698640 1 -- 172.21.15.172:0/2615547973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e800d2b20 msgr2=0x7f6e801622c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.117+0000 7f6e88698640 1 --2- 172.21.15.172:0/2615547973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e800d2b20 0x7f6e801622c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6e8008c640 tx=0x7f6e7802be40 comp rx=0 tx=0).stop 2024-04-14T08:37:24.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.117+0000 7f6e88698640 1 -- 172.21.15.172:0/2615547973 shutdown_connections 2024-04-14T08:37:24.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.117+0000 7f6e88698640 1 --2- 172.21.15.172:0/2615547973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e800d2b20 0x7f6e801622c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.117+0000 7f6e88698640 1 --2- 172.21.15.172:0/2615547973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e800b5490 0x7f6e800d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.117+0000 7f6e88698640 1 --2- 172.21.15.172:0/2615547973 >> v2:172.21.15.177:3300/0 conn(0x7f6e80161d00 0x7f6e800b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.117+0000 7f6e88698640 1 -- 172.21.15.172:0/2615547973 >> 172.21.15.172:0/2615547973 conn(0x7f6e8009d570 msgr2=0x7f6e800d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.117+0000 7f6e88698640 1 -- 172.21.15.172:0/2615547973 shutdown_connections 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.117+0000 7f6e88698640 1 -- 172.21.15.172:0/2615547973 wait complete. 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.118+0000 7f6e88698640 1 Processor -- start 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.118+0000 7f6e88698640 1 -- start start 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e88698640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e800b5490 0x7f6e801f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e88698640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e800d2b20 0x7f6e800b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e88698640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e80161d00 0x7f6e800b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e88698640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e800d13b0 con 0x7f6e800d2b20 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e88698640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6e800d10d0 con 0x7f6e80161d00 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e88698640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6e800b5940 con 0x7f6e800b5490 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e85c33640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e800d2b20 0x7f6e800b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e85c33640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e800d2b20 0x7f6e800b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48140/0 (socket says 172.21.15.172:48140) 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e85c33640 1 -- 172.21.15.172:0/1167313109 learned_addr learned my addr 172.21.15.172:0/1167313109 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:24.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e86c35640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e80161d00 0x7f6e800b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e86434640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e800b5490 0x7f6e801f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e85c33640 1 -- 172.21.15.172:0/1167313109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e800b5490 msgr2=0x7f6e801f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e85c33640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e800b5490 0x7f6e801f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e85c33640 1 -- 172.21.15.172:0/1167313109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e80161d00 msgr2=0x7f6e800b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e85c33640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e80161d00 0x7f6e800b8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e85c33640 1 -- 172.21.15.172:0/1167313109 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e800c7f10 con 0x7f6e800d2b20 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e85c33640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e800d2b20 0x7f6e800b77c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00d830 tx=0x7f6e7c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e76ffd640 1 -- 172.21.15.172:0/1167313109 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c018030 con 0x7f6e800d2b20 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e76ffd640 1 -- 172.21.15.172:0/1167313109 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e7c00be10 con 0x7f6e800d2b20 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e88698640 1 -- 172.21.15.172:0/1167313109 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e800c81f0 con 0x7f6e800d2b20 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.119+0000 7f6e76ffd640 1 -- 172.21.15.172:0/1167313109 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c009850 con 0x7f6e800d2b20 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.120+0000 7f6e88698640 1 -- 172.21.15.172:0/1167313109 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e800cac40 con 0x7f6e800d2b20 2024-04-14T08:37:24.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.121+0000 7f6e76ffd640 1 -- 172.21.15.172:0/1167313109 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e7c01b070 con 0x7f6e800d2b20 2024-04-14T08:37:24.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.122+0000 7f6e76ffd640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6e6406c8f0 0x7f6e6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.122+0000 7f6e86434640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6e6406c8f0 0x7f6e6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.122+0000 7f6e76ffd640 1 -- 172.21.15.172:0/1167313109 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f6e7c08be90 con 0x7f6e800d2b20 2024-04-14T08:37:24.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.122+0000 7f6e86434640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6e6406c8f0 0x7f6e6406edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6e70006fd0 tx=0x7f6e70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.122+0000 7f6e8b47d740 1 Processor -- start 2024-04-14T08:37:24.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.122+0000 7f6e8b47d740 1 -- start start 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e8b47d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52c79dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e8b47d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 0x560d52ca9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e8b47d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52c7a300 0x560d52caa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e8b47d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560d52be3ac0 con 0x560d52ca6e20 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e8b47d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560d52b81a80 con 0x560d52ca7200 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e8b47d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x560d52bdfa70 con 0x560d52c7a300 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e86434640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52c79dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e86434640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52c79dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48142/0 (socket says 172.21.15.172:48142) 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e86434640 1 -- 172.21.15.172:0/2574690225 learned_addr learned my addr 172.21.15.172:0/2574690225 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e86434640 1 -- 172.21.15.172:0/2574690225 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52c7a300 msgr2=0x560d52caa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e86434640 1 --2- 172.21.15.172:0/2574690225 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52c7a300 0x560d52caa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e86434640 1 -- 172.21.15.172:0/2574690225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 msgr2=0x560d52ca9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.123+0000 7f6e86434640 1 --2- 172.21.15.172:0/2574690225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 0x560d52ca9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.124+0000 7f6e86434640 1 -- 172.21.15.172:0/2574690225 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d529c3a20 con 0x560d52ca6e20 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.124+0000 7f6e86434640 1 --2- 172.21.15.172:0/2574690225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52c79dc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000fa50 tx=0x7f6e70012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba1c0a9825b3b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.124+0000 7f6e63fff640 1 -- 172.21.15.172:0/2574690225 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7001b050 con 0x560d52ca6e20 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.124+0000 7f6e63fff640 1 -- 172.21.15.172:0/2574690225 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e70025070 con 0x560d52ca6e20 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.124+0000 7f6e63fff640 1 -- 172.21.15.172:0/2574690225 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70014de0 con 0x560d52ca6e20 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.124+0000 7f6e8b47d740 1 -- 172.21.15.172:0/2574690225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 msgr2=0x560d52c79dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.124+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/2574690225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52c79dc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000fa50 tx=0x7f6e70012000 comp rx=0 tx=0).stop 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.125+0000 7f6e8b47d740 1 -- 172.21.15.172:0/2574690225 shutdown_connections 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.125+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/2574690225 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52c7a300 0x560d52caa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.125+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/2574690225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 0x560d52ca9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.125+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/2574690225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52c79dc0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.125+0000 7f6e8b47d740 1 -- 172.21.15.172:0/2574690225 >> 172.21.15.172:0/2574690225 conn(0x560d52c7c030 msgr2=0x560d52cc4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.125+0000 7f6e8b47d740 1 -- 172.21.15.172:0/2574690225 shutdown_connections 2024-04-14T08:37:24.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.125+0000 7f6e8b47d740 1 -- 172.21.15.172:0/2574690225 wait complete. 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.125+0000 7f6e8b47d740 1 -- start start 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.125+0000 7f6e8b47d740 1 Processor -- start 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.126+0000 7f6e8b47d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52ca6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.126+0000 7f6e8b47d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 0x560d52cbfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.126+0000 7f6e8b47d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52cbc120 0x560d52cbcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.126+0000 7f6e8b47d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560d52c7e5d0 con 0x560d52ca6e20 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.126+0000 7f6e8b47d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560d52b7f410 con 0x560d52ca7200 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.126+0000 7f6e8b47d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x560d52c7d9c0 con 0x560d52cbc120 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.126+0000 7f6e8b47d740 10 client.0 _ll_register_callbacks cb 0x560d52b9a9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.126+0000 7f6e86434640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52ca6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.126+0000 7f6e86434640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52ca6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48152/0 (socket says 172.21.15.172:48152) 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.126+0000 7f6e86434640 1 -- 172.21.15.172:0/1459102965 learned_addr learned my addr 172.21.15.172:0/1459102965 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:24.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e86c35640 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52cbc120 0x560d52cbcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e85c33640 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 0x560d52cbfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e86434640 1 -- 172.21.15.172:0/1459102965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52cbc120 msgr2=0x560d52cbcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e86434640 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52cbc120 0x560d52cbcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e86434640 1 -- 172.21.15.172:0/1459102965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 msgr2=0x560d52cbfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e86434640 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 0x560d52cbfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e86434640 1 -- 172.21.15.172:0/1459102965 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d528b7310 con 0x560d52ca6e20 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e85c33640 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 0x560d52cbfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e86434640 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52ca6380 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f6e70014830 tx=0x7f6e70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b98b4fa5ec14057 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e617fa640 1 -- 172.21.15.172:0/1459102965 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70020cb0 con 0x560d52ca6e20 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e617fa640 1 -- 172.21.15.172:0/1459102965 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f6e70035cd0 con 0x560d52ca6e20 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e8b47d740 1 -- 172.21.15.172:0/1459102965 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e70025070 con 0x560d52ca6e20 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e617fa640 1 -- 172.21.15.172:0/1459102965 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6e7001b020 con 0x560d52ca6e20 2024-04-14T08:37:24.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e8b47d740 10 client.7095 fetch_fsmap learned FSMap version 45 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e8b47d740 1 -- 172.21.15.172:0/1459102965 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x560d52a241b0 con 0x560d52ca6e20 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e617fa640 1 -- 172.21.15.172:0/1459102965 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e70025070 con 0x560d52ca6e20 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e617fa640 1 -- 172.21.15.172:0/1459102965 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7f6e70032070 con 0x560d52ca6e20 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e8b47d740 10 client.7095 fetch_fsmap finished waiting for FSMap version 45 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e8b47d740 10 client.7095 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e8b47d740 20 client.7095 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.127+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x560d52cbc500 0x560d52cac840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.128+0000 7f6e8b47d740 4 client.7095 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.128+0000 7f6e8b47d740 1 -- 172.21.15.172:0/1459102965 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d52bf1000 con 0x560d52cbc500 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.128+0000 7f6e85c33640 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x560d52cbc500 0x560d52cac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.128+0000 7f6e85c33640 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x560d52cbc500 0x560d52cac840 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95cf865bffc04fce server_cookie=b4561b82b0fac30a in_seq=0 out_seq=0 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.128+0000 7f6e617fa640 10 client.7095 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:24.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.128+0000 7f6e617fa640 1 -- 172.21.15.172:0/1459102965 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d52bf1000 con 0x560d52cbc500 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.128+0000 7f6e617fa640 10 client.7095 handle_command_reply: tid=0 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e8b47d740 1 client.7095 shutdown 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e8b47d740 1 -- 172.21.15.172:0/1459102965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 msgr2=0x560d52ca6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52ca6380 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f6e70014830 tx=0x7f6e70004730 comp rx=0 tx=0).stop 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e8b47d740 1 -- 172.21.15.172:0/1459102965 shutdown_connections 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x560d52cbc500 0x560d52cac840 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e617fa640 0 client.7095 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52cbc120 0x560d52cbcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 0x560d52cbfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/1459102965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52ca6380 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e8b47d740 1 -- 172.21.15.172:0/1459102965 >> 172.21.15.172:0/1459102965 conn(0x560d52c7a300 msgr2=0x560d52cc80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e8b47d740 1 -- 172.21.15.172:0/1459102965 shutdown_connections 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e8b47d740 1 -- 172.21.15.172:0/1459102965 wait complete. 2024-04-14T08:37:24.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.129+0000 7f6e8b47d740 20 client.7095 trim_cache size 0 max 16384 2024-04-14T08:37:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.147+0000 7f6e8b47d740 1 Processor -- start 2024-04-14T08:37:24.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.147+0000 7f6e8b47d740 1 -- start start 2024-04-14T08:37:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.147+0000 7f6e8b47d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52cc83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.147+0000 7f6e8b47d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52ca7200 0x560d52cc8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.147+0000 7f6e8b47d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52cbc120 0x560d52cc4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.147+0000 7f6e8b47d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e80201250 con 0x560d52ca6e20 2024-04-14T08:37:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.147+0000 7f6e8b47d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6e802002d0 con 0x560d52cbc120 2024-04-14T08:37:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.147+0000 7f6e8b47d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6e801f4ec0 con 0x560d52ca7200 2024-04-14T08:37:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.147+0000 7f6e86434640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52cc83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.147+0000 7f6e86434640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52cc83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48154/0 (socket says 172.21.15.172:48154) 2024-04-14T08:37:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.147+0000 7f6e86434640 1 -- 172.21.15.172:0/920196891 learned_addr learned my addr 172.21.15.172:0/920196891 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.148+0000 7f6e86434640 1 -- 172.21.15.172:0/920196891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52ca7200 msgr2=0x560d52cc8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.148+0000 7f6e86434640 1 --2- 172.21.15.172:0/920196891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52ca7200 0x560d52cc8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.148+0000 7f6e86434640 1 -- 172.21.15.172:0/920196891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52cbc120 msgr2=0x560d52cc4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.148+0000 7f6e86434640 1 --2- 172.21.15.172:0/920196891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52cbc120 0x560d52cc4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.148+0000 7f6e86434640 1 -- 172.21.15.172:0/920196891 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d52890490 con 0x560d52ca6e20 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.148+0000 7f6e86434640 1 --2- 172.21.15.172:0/920196891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52cc83e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f6e700148e0 tx=0x7f6e7002d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6114b38d27deb995 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.148+0000 7f6e637fe640 1 -- 172.21.15.172:0/920196891 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70020ab0 con 0x560d52ca6e20 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.148+0000 7f6e637fe640 1 -- 172.21.15.172:0/920196891 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e70025040 con 0x560d52ca6e20 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.148+0000 7f6e637fe640 1 -- 172.21.15.172:0/920196891 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70034440 con 0x560d52ca6e20 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.149+0000 7f6e8b47d740 1 -- 172.21.15.172:0/920196891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 msgr2=0x560d52cc83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.149+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/920196891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52cc83e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f6e700148e0 tx=0x7f6e7002d040 comp rx=0 tx=0).stop 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.149+0000 7f6e8b47d740 1 -- 172.21.15.172:0/920196891 shutdown_connections 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.149+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/920196891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52cbc120 0x560d52cc4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.149+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/920196891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52ca7200 0x560d52cc8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.149+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/920196891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52ca6e20 0x560d52cc83e0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.149+0000 7f6e8b47d740 1 -- 172.21.15.172:0/920196891 >> 172.21.15.172:0/920196891 conn(0x560d52c7a300 msgr2=0x560d52c75330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:24.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.149+0000 7f6e8b47d740 1 -- 172.21.15.172:0/920196891 shutdown_connections 2024-04-14T08:37:24.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.149+0000 7f6e8b47d740 1 -- 172.21.15.172:0/920196891 wait complete. 2024-04-14T08:37:24.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.149+0000 7f6e8b47d740 1 -- start start 2024-04-14T08:37:24.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.150+0000 7f6e8b47d740 1 Processor -- start 2024-04-14T08:37:24.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e8b47d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52ca6e20 0x560d52cc83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e8b47d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 0x560d52cc8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e8b47d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52cbc120 0x560d52cd73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e8b47d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e801ffcb0 con 0x560d52cbc120 2024-04-14T08:37:24.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e8b47d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560d52caad10 con 0x560d52ca7200 2024-04-14T08:37:24.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e8b47d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x560d52ce9210 con 0x560d52ca6e20 2024-04-14T08:37:24.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e8b47d740 10 client.0 _ll_register_callbacks cb 0x7f6e7000f8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:24.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e86c35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52cbc120 0x560d52cd73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e86c35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52cbc120 0x560d52cd73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48170/0 (socket says 172.21.15.172:48170) 2024-04-14T08:37:24.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e86c35640 1 -- 172.21.15.172:0/918698934 learned_addr learned my addr 172.21.15.172:0/918698934 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:24.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e85c33640 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 0x560d52cc8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e86434640 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52ca6e20 0x560d52cc83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e86c35640 1 -- 172.21.15.172:0/918698934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52ca6e20 msgr2=0x560d52cc83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e86c35640 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52ca6e20 0x560d52cc83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e86c35640 1 -- 172.21.15.172:0/918698934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 msgr2=0x560d52cc8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e86c35640 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 0x560d52cc8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e86c35640 1 -- 172.21.15.172:0/918698934 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d52ce8ff0 con 0x560d52cbc120 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.151+0000 7f6e86434640 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52ca6e20 0x560d52cc83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e86c35640 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52cbc120 0x560d52cd73d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6e7802c450 tx=0x7f6e7803d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2f771c45ac71b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e617fa640 1 -- 172.21.15.172:0/918698934 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e78009510 con 0x560d52cbc120 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e617fa640 1 -- 172.21.15.172:0/918698934 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f6e78048c90 con 0x560d52cbc120 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e8b47d740 1 -- 172.21.15.172:0/918698934 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e70025040 con 0x560d52cbc120 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e617fa640 1 -- 172.21.15.172:0/918698934 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6e7803dba0 con 0x560d52cbc120 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e8b47d740 10 client.7101 fetch_fsmap learned FSMap version 45 2024-04-14T08:37:24.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e8b47d740 1 -- 172.21.15.172:0/918698934 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x560d52cc4d40 con 0x560d52cbc120 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e617fa640 1 -- 172.21.15.172:0/918698934 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e70025040 con 0x560d52cbc120 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e617fa640 1 -- 172.21.15.172:0/918698934 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7f6e7804f050 con 0x560d52cbc120 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e8b47d740 10 client.7101 fetch_fsmap finished waiting for FSMap version 45 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e8b47d740 10 client.7101 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e8b47d740 20 client.7101 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x560d52cd3e30 0x560d52cd7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e8b47d740 4 client.7101 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e8b47d740 1 -- 172.21.15.172:0/918698934 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x560d52beb350 con 0x560d52cd3e30 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.152+0000 7f6e86434640 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x560d52cd3e30 0x560d52cd7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.153+0000 7f6e617fa640 10 client.7101 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.153+0000 7f6e86434640 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x560d52cd3e30 0x560d52cd7910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ef7de472a8b1d server_cookie=4d4064b3c8264c39 in_seq=0 out_seq=0 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.153+0000 7f6e617fa640 1 -- 172.21.15.172:0/918698934 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560d52beb350 con 0x560d52cd3e30 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.153+0000 7f6e617fa640 10 client.7101 handle_command_reply: tid=0 2024-04-14T08:37:24.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.153+0000 7f6e8b47d740 1 client.7101 shutdown 2024-04-14T08:37:24.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.153+0000 7f6e8b47d740 1 -- 172.21.15.172:0/918698934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52cbc120 msgr2=0x560d52cd73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.153+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52cbc120 0x560d52cd73d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6e7802c450 tx=0x7f6e7803d620 comp rx=0 tx=0).stop 2024-04-14T08:37:24.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e8b47d740 1 -- 172.21.15.172:0/918698934 shutdown_connections 2024-04-14T08:37:24.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x560d52cd3e30 0x560d52cd7910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560d52cbc120 0x560d52cd73d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d52ca7200 0x560d52cc8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e617fa640 0 client.7101 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:24.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e8b47d740 1 --2- 172.21.15.172:0/918698934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560d52ca6e20 0x560d52cc83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e8b47d740 1 -- 172.21.15.172:0/918698934 >> 172.21.15.172:0/918698934 conn(0x560d52c7a300 msgr2=0x560d52cd43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:24.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e8b47d740 1 -- 172.21.15.172:0/918698934 shutdown_connections 2024-04-14T08:37:24.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e8b47d740 1 -- 172.21.15.172:0/918698934 wait complete. 2024-04-14T08:37:24.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e8b47d740 20 client.7101 trim_cache size 0 max 16384 2024-04-14T08:37:24.160 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:37:24.160 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-04-14T08:37:24.160 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "009aa82f-9365-4819-868d-a09ae8c58a4d", 2024-04-14T08:37:24.160 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-04-14T08:37:24.160 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:37:24.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e88698640 1 -- 172.21.15.172:0/1167313109 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6e6406c8f0 msgr2=0x7f6e6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e88698640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6e6406c8f0 0x7f6e6406edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6e70006fd0 tx=0x7f6e70008040 comp rx=0 tx=0).stop 2024-04-14T08:37:24.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e88698640 1 -- 172.21.15.172:0/1167313109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e800d2b20 msgr2=0x7f6e800b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e88698640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e800d2b20 0x7f6e800b77c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00d830 tx=0x7f6e7c00dcf0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e88698640 1 -- 172.21.15.172:0/1167313109 shutdown_connections 2024-04-14T08:37:24.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e88698640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6e6406c8f0 0x7f6e6406edb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e88698640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e80161d00 0x7f6e800b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e88698640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e800d2b20 0x7f6e800b77c0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e88698640 1 --2- 172.21.15.172:0/1167313109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e800b5490 0x7f6e801f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e88698640 1 -- 172.21.15.172:0/1167313109 >> 172.21.15.172:0/1167313109 conn(0x7f6e8009d570 msgr2=0x7f6e8006d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:24.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e88698640 1 -- 172.21.15.172:0/1167313109 shutdown_connections 2024-04-14T08:37:24.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.154+0000 7f6e88698640 1 -- 172.21.15.172:0/1167313109 wait complete. 2024-04-14T08:37:24.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:24.166 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-14T08:37:24.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.268+0000 7fa318636640 1 Processor -- start 2024-04-14T08:37:24.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.269+0000 7fa318636640 1 -- start start 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.269+0000 7fa318636640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa31015c420 0x7fa3100b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.269+0000 7fa318636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3100ceb00 0x7fa3100cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.269+0000 7fa318636640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3100ccdd0 con 0x7fa31006f3e0 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.270+0000 7fa318636640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa3100d30a0 con 0x7fa31015c420 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.270+0000 7fa318636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa31006f870 con 0x7fa3100ceb00 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.270+0000 7fa3163d2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa31015c420 0x7fa3100b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.270+0000 7fa3163d2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa31015c420 0x7fa3100b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48184/0 (socket says 172.21.15.172:48184) 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.270+0000 7fa3163d2640 1 -- 172.21.15.172:0/1952777707 learned_addr learned my addr 172.21.15.172:0/1952777707 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.270+0000 7fa316bd3640 1 --2- 172.21.15.172:0/1952777707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3100ceb00 0x7fa3100cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.270+0000 7fa3163d2640 1 -- 172.21.15.172:0/1952777707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3100ceb00 msgr2=0x7fa3100cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.270+0000 7fa3163d2640 1 --2- 172.21.15.172:0/1952777707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3100ceb00 0x7fa3100cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.270+0000 7fa3163d2640 1 -- 172.21.15.172:0/1952777707 >> v1:172.21.15.177:6789/0 conn(0x7fa31006f3e0 legacy=0x7fa3100d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.270+0000 7fa3163d2640 1 -- 172.21.15.172:0/1952777707 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3100d2ae0 con 0x7fa31015c420 2024-04-14T08:37:24.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.271+0000 7fa3163d2640 1 --2- 172.21.15.172:0/1952777707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa31015c420 0x7fa3100b1510 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009870 tx=0x7fa30c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bca36820259ac54 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.271+0000 7fa3153d0640 1 -- 172.21.15.172:0/1952777707 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c034070 con 0x7fa31015c420 2024-04-14T08:37:24.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.271+0000 7fa3153d0640 1 -- 172.21.15.172:0/1952777707 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30c030840 con 0x7fa31015c420 2024-04-14T08:37:24.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.271+0000 7fa3153d0640 1 -- 172.21.15.172:0/1952777707 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c030c10 con 0x7fa31015c420 2024-04-14T08:37:24.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.271+0000 7fa318636640 1 -- 172.21.15.172:0/1952777707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa31015c420 msgr2=0x7fa3100b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.271+0000 7fa318636640 1 --2- 172.21.15.172:0/1952777707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa31015c420 0x7fa3100b1510 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009870 tx=0x7fa30c02b980 comp rx=0 tx=0).stop 2024-04-14T08:37:24.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.272+0000 7fa318636640 1 -- 172.21.15.172:0/1952777707 shutdown_connections 2024-04-14T08:37:24.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.272+0000 7fa318636640 1 --2- 172.21.15.172:0/1952777707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3100ceb00 0x7fa3100cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.272+0000 7fa318636640 1 --2- 172.21.15.172:0/1952777707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa31015c420 0x7fa3100b1510 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.272+0000 7fa318636640 1 -- 172.21.15.172:0/1952777707 >> 172.21.15.172:0/1952777707 conn(0x7fa31008c000 msgr2=0x7fa3100d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:24.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.272+0000 7fa318636640 1 -- 172.21.15.172:0/1952777707 shutdown_connections 2024-04-14T08:37:24.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.272+0000 7fa318636640 1 -- 172.21.15.172:0/1952777707 wait complete. 2024-04-14T08:37:24.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.273+0000 7fa318636640 1 Processor -- start 2024-04-14T08:37:24.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.273+0000 7fa318636640 1 -- start start 2024-04-14T08:37:24.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.273+0000 7fa318636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3100ceb00 0x7fa3100b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.273+0000 7fa318636640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa31015c420 0x7fa3101572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.273+0000 7fa318636640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100c7e80 0x7fa3101e5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.273+0000 7fa318636640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3100cd070 con 0x7fa3100c7e80 2024-04-14T08:37:24.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.273+0000 7fa318636640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa3100cf470 con 0x7fa31015c420 2024-04-14T08:37:24.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.273+0000 7fa318636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa3100b11e0 con 0x7fa3100ceb00 2024-04-14T08:37:24.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa316bd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100c7e80 0x7fa3101e5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa316bd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100c7e80 0x7fa3101e5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48190/0 (socket says 172.21.15.172:48190) 2024-04-14T08:37:24.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa316bd3640 1 -- 172.21.15.172:0/2065455257 learned_addr learned my addr 172.21.15.172:0/2065455257 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:24.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa315bd1640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa31015c420 0x7fa3101572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa3163d2640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3100ceb00 0x7fa3100b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa316bd3640 1 -- 172.21.15.172:0/2065455257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3100ceb00 msgr2=0x7fa3100b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa316bd3640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3100ceb00 0x7fa3100b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa316bd3640 1 -- 172.21.15.172:0/2065455257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa31015c420 msgr2=0x7fa3101572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa316bd3640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa31015c420 0x7fa3101572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa316bd3640 1 -- 172.21.15.172:0/2065455257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa310157810 con 0x7fa3100c7e80 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa3163d2640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3100ceb00 0x7fa3100b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa316bd3640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100c7e80 0x7fa3101e5df0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa308007e70 tx=0x7fa30800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa306ffd640 1 -- 172.21.15.172:0/2065455257 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308014420 con 0x7fa3100c7e80 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa306ffd640 1 -- 172.21.15.172:0/2065455257 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa308019070 con 0x7fa3100c7e80 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa318636640 1 -- 172.21.15.172:0/2065455257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3100b18b0 con 0x7fa3100c7e80 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa306ffd640 1 -- 172.21.15.172:0/2065455257 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308014820 con 0x7fa3100c7e80 2024-04-14T08:37:24.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.274+0000 7fa318636640 1 -- 172.21.15.172:0/2065455257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3100b1e70 con 0x7fa3100c7e80 2024-04-14T08:37:24.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.276+0000 7fa306ffd640 1 -- 172.21.15.172:0/2065455257 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3080149c0 con 0x7fa3100c7e80 2024-04-14T08:37:24.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.276+0000 7fa306ffd640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa2f406c920 0x7fa2f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.277+0000 7fa3163d2640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa2f406c920 0x7fa2f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.277+0000 7fa306ffd640 1 -- 172.21.15.172:0/2065455257 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fa30808c440 con 0x7fa3100c7e80 2024-04-14T08:37:24.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.277+0000 7fa3163d2640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa2f406c920 0x7fa2f406ede0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa30c036000 tx=0x7fa30c03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.277+0000 7fa31b43f740 1 Processor -- start 2024-04-14T08:37:24.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.277+0000 7fa31b43f740 1 -- start start 2024-04-14T08:37:24.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa31b43f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c6f8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa31b43f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c725d50 0x56082c728700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa31b43f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c6f8e50 0x56082c728c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa31b43f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56082c662800 con 0x56082c725d50 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa31b43f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56082c60a7f0 con 0x56082c725970 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa31b43f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56082c65e7b0 con 0x56082c6f8e50 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa315bd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c725d50 0x56082c728700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa315bd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c725d50 0x56082c728700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48202/0 (socket says 172.21.15.172:48202) 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa315bd1640 1 -- 172.21.15.172:0/2431597288 learned_addr learned my addr 172.21.15.172:0/2431597288 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa3163d2640 1 --2- 172.21.15.172:0/2431597288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c6f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa316bd3640 1 --2- 172.21.15.172:0/2431597288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c6f8e50 0x56082c728c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa315bd1640 1 -- 172.21.15.172:0/2431597288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c6f8e50 msgr2=0x56082c728c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa315bd1640 1 --2- 172.21.15.172:0/2431597288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c6f8e50 0x56082c728c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa315bd1640 1 -- 172.21.15.172:0/2431597288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 msgr2=0x56082c6f8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.278+0000 7fa315bd1640 1 --2- 172.21.15.172:0/2431597288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c6f8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.279+0000 7fa315bd1640 1 -- 172.21.15.172:0/2431597288 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082c4407d0 con 0x56082c725d50 2024-04-14T08:37:24.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.279+0000 7fa3163d2640 1 --2- 172.21.15.172:0/2431597288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c6f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:24.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.279+0000 7fa316bd3640 1 --2- 172.21.15.172:0/2431597288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c6f8e50 0x56082c728c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:24.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.279+0000 7fa315bd1640 1 --2- 172.21.15.172:0/2431597288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c725d50 0x56082c728700 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa300001820 tx=0x7fa300007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6525e24d1fc28141 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.279+0000 7fa2f3fff640 1 -- 172.21.15.172:0/2431597288 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3000052e0 con 0x56082c725d50 2024-04-14T08:37:24.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.279+0000 7fa2f3fff640 1 -- 172.21.15.172:0/2431597288 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa300018070 con 0x56082c725d50 2024-04-14T08:37:24.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.279+0000 7fa2f3fff640 1 -- 172.21.15.172:0/2431597288 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30000f9b0 con 0x56082c725d50 2024-04-14T08:37:24.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.281+0000 7fa31b43f740 1 -- 172.21.15.172:0/2431597288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c725d50 msgr2=0x56082c728700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.281+0000 7fa31b43f740 1 --2- 172.21.15.172:0/2431597288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c725d50 0x56082c728700 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa300001820 tx=0x7fa300007e50 comp rx=0 tx=0).stop 2024-04-14T08:37:24.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.281+0000 7fa31b43f740 1 -- 172.21.15.172:0/2431597288 shutdown_connections 2024-04-14T08:37:24.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.281+0000 7fa31b43f740 1 --2- 172.21.15.172:0/2431597288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c6f8e50 0x56082c728c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.281+0000 7fa31b43f740 1 --2- 172.21.15.172:0/2431597288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c725d50 0x56082c728700 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.281+0000 7fa31b43f740 1 --2- 172.21.15.172:0/2431597288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c6f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.281+0000 7fa31b43f740 1 -- 172.21.15.172:0/2431597288 >> 172.21.15.172:0/2431597288 conn(0x56082c6fab80 msgr2=0x56082c743700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:24.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.281+0000 7fa31b43f740 1 -- 172.21.15.172:0/2431597288 shutdown_connections 2024-04-14T08:37:24.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.281+0000 7fa31b43f740 1 -- 172.21.15.172:0/2431597288 wait complete. 2024-04-14T08:37:24.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.282+0000 7fa31b43f740 1 -- start start 2024-04-14T08:37:24.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.282+0000 7fa31b43f740 1 Processor -- start 2024-04-14T08:37:24.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.283+0000 7fa31b43f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c72a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.283+0000 7fa31b43f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725d50 0x56082c7537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.283+0000 7fa31b43f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c753d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.283+0000 7fa31b43f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56082c6fd120 con 0x56082c756630 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.283+0000 7fa31b43f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56082c61dd00 con 0x56082c725970 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.283+0000 7fa31b43f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56082c6fc510 con 0x56082c725d50 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.283+0000 7fa31b43f740 10 client.0 _ll_register_callbacks cb 0x56082c61d6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.283+0000 7fa316bd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c753d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.283+0000 7fa316bd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c753d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48204/0 (socket says 172.21.15.172:48204) 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.283+0000 7fa316bd3640 1 -- 172.21.15.172:0/3584868525 learned_addr learned my addr 172.21.15.172:0/3584868525 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa3163d2640 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c72a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa315bd1640 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725d50 0x56082c7537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa316bd3640 1 -- 172.21.15.172:0/3584868525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725d50 msgr2=0x56082c7537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa316bd3640 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725d50 0x56082c7537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa316bd3640 1 -- 172.21.15.172:0/3584868525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 msgr2=0x56082c72a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa316bd3640 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c72a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa316bd3640 1 -- 172.21.15.172:0/3584868525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082c3763f0 con 0x56082c756630 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa3163d2640 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c72a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa316bd3640 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c753d30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa30800e9b0 tx=0x7fa30805a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7802ab600e796d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa2f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa2f17fa640 1 -- 172.21.15.172:0/3584868525 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308062b70 con 0x56082c756630 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa2f17fa640 1 -- 172.21.15.172:0/3584868525 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fa30808c070 con 0x56082c756630 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa31b43f740 1 -- 172.21.15.172:0/3584868525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa300018070 con 0x56082c756630 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa2f17fa640 1 -- 172.21.15.172:0/3584868525 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa30808c270 con 0x56082c756630 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa31b43f740 10 client.7113 fetch_fsmap learned FSMap version 45 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa31b43f740 1 -- 172.21.15.172:0/3584868525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x56082c3360f0 con 0x56082c756630 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa2f17fa640 1 -- 172.21.15.172:0/3584868525 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa300018070 con 0x56082c756630 2024-04-14T08:37:24.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa2f17fa640 1 -- 172.21.15.172:0/3584868525 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7fa3080626e0 con 0x56082c756630 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa31b43f740 10 client.7113 fetch_fsmap finished waiting for FSMap version 45 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa31b43f740 10 client.7113 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa31b43f740 20 client.7113 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.284+0000 7fa31b43f740 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56082c73ccf0 0x56082c760d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.285+0000 7fa31b43f740 4 client.7113 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.285+0000 7fa31b43f740 1 -- 172.21.15.172:0/3584868525 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56082c66fd40 con 0x56082c73ccf0 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.285+0000 7fa3163d2640 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56082c73ccf0 0x56082c760d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.285+0000 7fa2f17fa640 10 client.7113 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.285+0000 7fa3163d2640 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56082c73ccf0 0x56082c760d10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c045d1ab5016c0 server_cookie=875cf5bf32a21c9a in_seq=0 out_seq=0 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.285+0000 7fa2f17fa640 1 -- 172.21.15.172:0/3584868525 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56082c66fd40 con 0x56082c73ccf0 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.285+0000 7fa2f17fa640 10 client.7113 handle_command_reply: tid=0 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa31b43f740 1 client.7113 shutdown 2024-04-14T08:37:24.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa31b43f740 1 -- 172.21.15.172:0/3584868525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 msgr2=0x56082c753d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa31b43f740 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c753d30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa30800e9b0 tx=0x7fa30805a050 comp rx=0 tx=0).stop 2024-04-14T08:37:24.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa31b43f740 1 -- 172.21.15.172:0/3584868525 shutdown_connections 2024-04-14T08:37:24.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa31b43f740 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56082c73ccf0 0x56082c760d10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa31b43f740 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c753d30 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa31b43f740 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725d50 0x56082c7537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa2f17fa640 0 client.7113 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:24.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa31b43f740 1 --2- 172.21.15.172:0/3584868525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c72a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa31b43f740 1 -- 172.21.15.172:0/3584868525 >> 172.21.15.172:0/3584868525 conn(0x56082c6f8e50 msgr2=0x56082c7286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:24.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa31b43f740 1 -- 172.21.15.172:0/3584868525 shutdown_connections 2024-04-14T08:37:24.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa31b43f740 1 -- 172.21.15.172:0/3584868525 wait complete. 2024-04-14T08:37:24.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.286+0000 7fa31b43f740 20 client.7113 trim_cache size 0 max 16384 2024-04-14T08:37:24.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.303+0000 7fa31b43f740 1 Processor -- start 2024-04-14T08:37:24.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.303+0000 7fa31b43f740 1 -- start start 2024-04-14T08:37:24.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa31b43f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c72a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa31b43f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725d50 0x56082c746660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa31b43f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c746ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa31b43f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101ff090 con 0x56082c756630 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa31b43f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101fe190 con 0x56082c725970 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa31b43f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa3101f33d0 con 0x56082c725d50 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa316bd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c746ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa316bd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c746ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48208/0 (socket says 172.21.15.172:48208) 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa316bd3640 1 -- 172.21.15.172:0/1272690210 learned_addr learned my addr 172.21.15.172:0/1272690210 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa316bd3640 1 -- 172.21.15.172:0/1272690210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725d50 msgr2=0x56082c746660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa3163d2640 1 --2- 172.21.15.172:0/1272690210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c72a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa316bd3640 1 --2- 172.21.15.172:0/1272690210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725d50 0x56082c746660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa316bd3640 1 -- 172.21.15.172:0/1272690210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 msgr2=0x56082c72a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa316bd3640 1 --2- 172.21.15.172:0/1272690210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c72a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa316bd3640 1 -- 172.21.15.172:0/1272690210 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082c6fa2e0 con 0x56082c756630 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.304+0000 7fa3163d2640 1 --2- 172.21.15.172:0/1272690210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c72a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa316bd3640 1 --2- 172.21.15.172:0/1272690210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c746ba0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa30800eb30 tx=0x7fa30808c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9dc8aea34f0599 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa2f37fe640 1 -- 172.21.15.172:0/1272690210 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308062b70 con 0x56082c756630 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa2f37fe640 1 -- 172.21.15.172:0/1272690210 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30808c070 con 0x56082c756630 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa2f37fe640 1 -- 172.21.15.172:0/1272690210 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3080626e0 con 0x56082c756630 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa31b43f740 1 -- 172.21.15.172:0/1272690210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 msgr2=0x56082c746ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa31b43f740 1 --2- 172.21.15.172:0/1272690210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c746ba0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa30800eb30 tx=0x7fa30808c000 comp rx=0 tx=0).stop 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa31b43f740 1 -- 172.21.15.172:0/1272690210 shutdown_connections 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa31b43f740 1 --2- 172.21.15.172:0/1272690210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c746ba0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa31b43f740 1 --2- 172.21.15.172:0/1272690210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725d50 0x56082c746660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa31b43f740 1 --2- 172.21.15.172:0/1272690210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725970 0x56082c72a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa31b43f740 1 -- 172.21.15.172:0/1272690210 >> 172.21.15.172:0/1272690210 conn(0x56082c6f8e50 msgr2=0x56082c72be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa31b43f740 1 -- 172.21.15.172:0/1272690210 shutdown_connections 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.305+0000 7fa31b43f740 1 -- 172.21.15.172:0/1272690210 wait complete. 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.306+0000 7fa31b43f740 1 -- start start 2024-04-14T08:37:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.306+0000 7fa31b43f740 1 Processor -- start 2024-04-14T08:37:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa31b43f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725970 0x56082c7667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa31b43f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725d50 0x56082c754a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa31b43f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c754f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa31b43f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3100c7800 con 0x56082c756630 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa31b43f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56082c7559d0 con 0x56082c725d50 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa31b43f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56082c7c7230 con 0x56082c725970 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa31b43f740 10 client.0 _ll_register_callbacks cb 0x7fa3080562b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa316bd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c754f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa316bd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c754f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48214/0 (socket says 172.21.15.172:48214) 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa316bd3640 1 -- 172.21.15.172:0/1439466567 learned_addr learned my addr 172.21.15.172:0/1439466567 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa315bd1640 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725d50 0x56082c754a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa3163d2640 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725970 0x56082c7667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa316bd3640 1 -- 172.21.15.172:0/1439466567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725970 msgr2=0x56082c7667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa316bd3640 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725970 0x56082c7667b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa316bd3640 1 -- 172.21.15.172:0/1439466567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725d50 msgr2=0x56082c754a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa316bd3640 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725d50 0x56082c754a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa316bd3640 1 -- 172.21.15.172:0/1439466567 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082c753cb0 con 0x56082c756630 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa316bd3640 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c754f50 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fa30808a7e0 tx=0x7fa308016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a019d93581a1db61 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:24.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.307+0000 7fa2f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa2f17fa640 1 -- 172.21.15.172:0/1439466567 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308062b70 con 0x56082c756630 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa2f17fa640 1 -- 172.21.15.172:0/1439466567 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fa308058030 con 0x56082c756630 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa31b43f740 1 -- 172.21.15.172:0/1439466567 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa30808c070 con 0x56082c756630 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa2f17fa640 1 -- 172.21.15.172:0/1439466567 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa30808a410 con 0x56082c756630 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa31b43f740 10 client.7119 fetch_fsmap learned FSMap version 45 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa31b43f740 1 -- 172.21.15.172:0/1439466567 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x56082c72bdd0 con 0x56082c756630 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa2f17fa640 1 -- 172.21.15.172:0/1439466567 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30808c070 con 0x56082c756630 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa2f17fa640 1 -- 172.21.15.172:0/1439466567 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7fa308080070 con 0x56082c756630 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa31b43f740 10 client.7119 fetch_fsmap finished waiting for FSMap version 45 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa31b43f740 10 client.7119 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa31b43f740 20 client.7119 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa31b43f740 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56082c751d00 0x56082c752110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa31b43f740 4 client.7119 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:37:24.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa31b43f740 1 -- 172.21.15.172:0/1439466567 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56082c73f370 con 0x56082c751d00 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa3163d2640 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56082c751d00 0x56082c752110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.308+0000 7fa2f17fa640 10 client.7119 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa3163d2640 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56082c751d00 0x56082c752110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3982565c520c6e5 server_cookie=f73c75db67cd33b0 in_seq=0 out_seq=0 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa2f17fa640 1 -- 172.21.15.172:0/1439466567 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56082c73f370 con 0x56082c751d00 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa2f17fa640 10 client.7119 handle_command_reply: tid=0 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa31b43f740 1 client.7119 shutdown 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa31b43f740 1 -- 172.21.15.172:0/1439466567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 msgr2=0x56082c754f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa31b43f740 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c754f50 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fa30808a7e0 tx=0x7fa308016040 comp rx=0 tx=0).stop 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa31b43f740 1 -- 172.21.15.172:0/1439466567 shutdown_connections 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa31b43f740 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56082c751d00 0x56082c752110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa31b43f740 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56082c756630 0x56082c754f50 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa31b43f740 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56082c725d50 0x56082c754a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa31b43f740 1 --2- 172.21.15.172:0/1439466567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56082c725970 0x56082c7667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa2f17fa640 0 client.7119 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:24.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa31b43f740 1 -- 172.21.15.172:0/1439466567 >> 172.21.15.172:0/1439466567 conn(0x56082c6f8e50 msgr2=0x56082c7463b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:24.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa31b43f740 1 -- 172.21.15.172:0/1439466567 shutdown_connections 2024-04-14T08:37:24.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa31b43f740 1 -- 172.21.15.172:0/1439466567 wait complete. 2024-04-14T08:37:24.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.309+0000 7fa31b43f740 20 client.7119 trim_cache size 0 max 16384 2024-04-14T08:37:24.316 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:37:24.316 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-14T08:37:24.316 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-04-14T08:37:24.316 INFO:teuthology.orchestra.run.smithi172.stdout: "009aa82f-9365-4819-868d-a09ae8c58a4d": { 2024-04-14T08:37:24.316 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/subdir", 2024-04-14T08:37:24.316 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "009aa82f-9365-4819-868d-a09ae8c58a4d", 2024-04-14T08:37:24.316 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive" 2024-04-14T08:37:24.317 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:37:24.317 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:37:24.317 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:37:24.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.310+0000 7fa318636640 1 -- 172.21.15.172:0/2065455257 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa2f406c920 msgr2=0x7fa2f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.310+0000 7fa318636640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa2f406c920 0x7fa2f406ede0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa30c036000 tx=0x7fa30c03b040 comp rx=0 tx=0).stop 2024-04-14T08:37:24.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.310+0000 7fa318636640 1 -- 172.21.15.172:0/2065455257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100c7e80 msgr2=0x7fa3101e5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:24.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.310+0000 7fa318636640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100c7e80 0x7fa3101e5df0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa308007e70 tx=0x7fa30800e4b0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.310+0000 7fa318636640 1 -- 172.21.15.172:0/2065455257 shutdown_connections 2024-04-14T08:37:24.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.310+0000 7fa318636640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa2f406c920 0x7fa2f406ede0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.310+0000 7fa318636640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100c7e80 0x7fa3101e5df0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.310+0000 7fa318636640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa31015c420 0x7fa3101572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.310+0000 7fa318636640 1 --2- 172.21.15.172:0/2065455257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3100ceb00 0x7fa3100b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:24.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.310+0000 7fa318636640 1 -- 172.21.15.172:0/2065455257 >> 172.21.15.172:0/2065455257 conn(0x7fa31008c000 msgr2=0x7fa3100d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:24.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.310+0000 7fa318636640 1 -- 172.21.15.172:0/2065455257 shutdown_connections 2024-04-14T08:37:24.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:24.310+0000 7fa318636640 1 -- 172.21.15.172:0/2065455257 wait complete. 2024-04-14T08:37:24.321 INFO:tasks.cephfs.filesystem:scrub status for tag:009aa82f-9365-4819-868d-a09ae8c58a4d - {'path': '/subdir', 'tag': '009aa82f-9365-4819-868d-a09ae8c58a4d', 'options': 'recursive'} 2024-04-14T08:37:26.394 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:37:26.393+0000 7f0f2508f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-14T08:37:53.782 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:37:53.785 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:37:54.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:54.323 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-14T08:37:54.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.431+0000 7f5e8a80f640 1 Processor -- start 2024-04-14T08:37:54.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.431+0000 7f5e8a80f640 1 -- start start 2024-04-14T08:37:54.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.432+0000 7f5e8a80f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5e840d0ab0 0x7f5e840ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.432+0000 7f5e8a80f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e8406fee0 con 0x7f5e8406f460 2024-04-14T08:37:54.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.432+0000 7f5e8a80f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e840ce850 con 0x7f5e8406edc0 2024-04-14T08:37:54.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.432+0000 7f5e8a80f640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5e840affe0 con 0x7f5e840d0ab0 2024-04-14T08:37:54.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.432+0000 7f5e88dac640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5e8406f460 0x7f5e8415dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:42102/0 (socket says 172.21.15.172:42102) 2024-04-14T08:37:54.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.432+0000 7f5e88dac640 1 -- 172.21.15.172:0/3285787486 learned_addr learned my addr 172.21.15.172:0/3285787486 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:54.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e837fe640 1 --2- 172.21.15.172:0/3285787486 >> v2:172.21.15.177:3300/0 conn(0x7f5e840d0ab0 0x7f5e840ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4225499031 0 0) 0x7f5e8406fee0 con 0x7f5e8406f460 2024-04-14T08:37:54.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e6c003520 con 0x7f5e8406f460 2024-04-14T08:37:54.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2189335828 0 0) 0x7f5e6c003520 con 0x7f5e8406f460 2024-04-14T08:37:54.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 >> v2:172.21.15.177:3300/0 conn(0x7f5e840d0ab0 msgr2=0x7f5e840ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 --2- 172.21.15.172:0/3285787486 >> v2:172.21.15.177:3300/0 conn(0x7f5e840d0ab0 0x7f5e840ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 >> v1:172.21.15.177:6789/0 conn(0x7f5e8406edc0 legacy=0x7f5e840d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e840b02c0 con 0x7f5e8406f460 2024-04-14T08:37:54.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f5e7c0019f0 con 0x7f5e8406f460 2024-04-14T08:37:54.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 >> v1:172.21.15.172:6789/0 conn(0x7f5e8406f460 legacy=0x7f5e8415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 --2- 172.21.15.172:0/3285787486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 0x7f5e6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 --2- 172.21.15.172:0/3285787486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e6c004800 0x7f5e6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 --2- 172.21.15.172:0/3285787486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6c0051c0 0x7f5e6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e840affe0 con 0x7f5e6c0039b0 2024-04-14T08:37:54.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c00c060 con 0x7f5e6c004800 2024-04-14T08:37:54.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e6c00c260 con 0x7f5e6c0051c0 2024-04-14T08:37:54.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e83fff640 1 --2- 172.21.15.172:0/3285787486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 0x7f5e6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e837fe640 1 --2- 172.21.15.172:0/3285787486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e6c004800 0x7f5e6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e88dac640 1 --2- 172.21.15.172:0/3285787486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6c0051c0 0x7f5e6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e83fff640 1 -- 172.21.15.172:0/3285787486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6c0051c0 msgr2=0x7f5e6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e83fff640 1 --2- 172.21.15.172:0/3285787486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6c0051c0 0x7f5e6c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e83fff640 1 -- 172.21.15.172:0/3285787486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e6c004800 msgr2=0x7f5e6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e83fff640 1 --2- 172.21.15.172:0/3285787486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e6c004800 0x7f5e6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e83fff640 1 -- 172.21.15.172:0/3285787486 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6c003520 con 0x7f5e6c0039b0 2024-04-14T08:37:54.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.433+0000 7f5e88dac640 1 --2- 172.21.15.172:0/3285787486 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6c0051c0 0x7f5e6c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:54.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.434+0000 7f5e88dac640 1 -- 172.21.15.172:0/3285787486 reap_dead start 2024-04-14T08:37:54.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.434+0000 7f5e83fff640 1 --2- 172.21.15.172:0/3285787486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 0x7f5e6c003dd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5e74001750 tx=0x7f5e6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab260c51f17843a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.434+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74001370 con 0x7f5e6c0039b0 2024-04-14T08:37:54.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.434+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e740052d0 con 0x7f5e6c0039b0 2024-04-14T08:37:54.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.434+0000 7f5e82ffd640 1 -- 172.21.15.172:0/3285787486 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74003830 con 0x7f5e6c0039b0 2024-04-14T08:37:54.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.435+0000 7f5e8a80f640 1 -- 172.21.15.172:0/3285787486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 msgr2=0x7f5e6c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.435+0000 7f5e8a80f640 1 --2- 172.21.15.172:0/3285787486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 0x7f5e6c003dd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5e74001750 tx=0x7f5e6c005830 comp rx=0 tx=0).stop 2024-04-14T08:37:54.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.435+0000 7f5e8a80f640 1 -- 172.21.15.172:0/3285787486 shutdown_connections 2024-04-14T08:37:54.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.435+0000 7f5e8a80f640 1 --2- 172.21.15.172:0/3285787486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 0x7f5e6c003dd0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.435+0000 7f5e8a80f640 1 -- 172.21.15.172:0/3285787486 >> 172.21.15.172:0/3285787486 conn(0x7f5e840cda00 msgr2=0x7f5e840cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.435+0000 7f5e8a80f640 1 -- 172.21.15.172:0/3285787486 shutdown_connections 2024-04-14T08:37:54.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.435+0000 7f5e8a80f640 1 -- 172.21.15.172:0/3285787486 wait complete. 2024-04-14T08:37:54.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.436+0000 7f5e8a80f640 1 Processor -- start 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.436+0000 7f5e8a80f640 1 -- start start 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e8a80f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 0x7f5e841dca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e8a80f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e840b0ea0 0x7f5e840b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e8a80f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e840c6170 0x7f5e840c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e8a80f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8406e130 con 0x7f5e6c0039b0 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e8a80f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8406f1a0 con 0x7f5e840c6170 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e8a80f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e840c5c70 con 0x7f5e840b0ea0 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 0x7f5e841dca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 0x7f5e841dca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59686/0 (socket says 172.21.15.172:59686) 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e83fff640 1 -- 172.21.15.172:0/1624833950 learned_addr learned my addr 172.21.15.172:0/1624833950 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e88dac640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e840c6170 0x7f5e840c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e837fe640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e840b0ea0 0x7f5e840b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e83fff640 1 -- 172.21.15.172:0/1624833950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e840b0ea0 msgr2=0x7f5e840b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e83fff640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e840b0ea0 0x7f5e840b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e83fff640 1 -- 172.21.15.172:0/1624833950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e840c6170 msgr2=0x7f5e840c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e83fff640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e840c6170 0x7f5e840c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.437+0000 7f5e83fff640 1 -- 172.21.15.172:0/1624833950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e840c4bd0 con 0x7f5e6c0039b0 2024-04-14T08:37:54.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.438+0000 7f5e88dac640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e840c6170 0x7f5e840c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:54.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.438+0000 7f5e83fff640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 0x7f5e841dca00 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5e74005040 tx=0x7f5e74001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.438+0000 7f5e80ff9640 1 -- 172.21.15.172:0/1624833950 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74041070 con 0x7f5e6c0039b0 2024-04-14T08:37:54.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.438+0000 7f5e80ff9640 1 -- 172.21.15.172:0/1624833950 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e74005480 con 0x7f5e6c0039b0 2024-04-14T08:37:54.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.438+0000 7f5e80ff9640 1 -- 172.21.15.172:0/1624833950 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74007a30 con 0x7f5e6c0039b0 2024-04-14T08:37:54.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.438+0000 7f5e8a80f640 1 -- 172.21.15.172:0/1624833950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e840c4eb0 con 0x7f5e6c0039b0 2024-04-14T08:37:54.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.438+0000 7f5e8a80f640 1 -- 172.21.15.172:0/1624833950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e840c5470 con 0x7f5e6c0039b0 2024-04-14T08:37:54.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.440+0000 7f5e80ff9640 1 -- 172.21.15.172:0/1624833950 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e74044070 con 0x7f5e6c0039b0 2024-04-14T08:37:54.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.440+0000 7f5e80ff9640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e7006c920 0x7f5e7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.440+0000 7f5e80ff9640 1 -- 172.21.15.172:0/1624833950 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f5e740ad6d0 con 0x7f5e6c0039b0 2024-04-14T08:37:54.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.440+0000 7f5e837fe640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e7006c920 0x7f5e7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.441+0000 7f5e837fe640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e7006c920 0x7f5e7006ede0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5e841e95e0 tx=0x7f5e78009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.441+0000 7f5e8d5f3740 1 Processor -- start 2024-04-14T08:37:54.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.441+0000 7f5e8d5f3740 1 -- start start 2024-04-14T08:37:54.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e8d5f3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cf63910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e8d5f3740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 0x56173cf93700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e8d5f3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf63e50 0x56173cf93c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e8d5f3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56173cecd800 con 0x56173cf90970 2024-04-14T08:37:54.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e8d5f3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56173ce757f0 con 0x56173cf63e50 2024-04-14T08:37:54.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e8d5f3740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56173cec97b0 con 0x56173cf90d50 2024-04-14T08:37:54.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cf63910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cf63910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59690/0 (socket says 172.21.15.172:59690) 2024-04-14T08:37:54.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e83fff640 1 -- 172.21.15.172:0/1465343040 learned_addr learned my addr 172.21.15.172:0/1465343040 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:54.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e837fe640 1 --2- 172.21.15.172:0/1465343040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 0x56173cf93700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e88dac640 1 --2- 172.21.15.172:0/1465343040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf63e50 0x56173cf93c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e83fff640 1 -- 172.21.15.172:0/1465343040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 msgr2=0x56173cf93700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e83fff640 1 --2- 172.21.15.172:0/1465343040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 0x56173cf93700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e83fff640 1 -- 172.21.15.172:0/1465343040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf63e50 msgr2=0x56173cf93c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e83fff640 1 --2- 172.21.15.172:0/1465343040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf63e50 0x56173cf93c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.442+0000 7f5e83fff640 1 -- 172.21.15.172:0/1465343040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56173ccab7d0 con 0x56173cf90970 2024-04-14T08:37:54.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.443+0000 7f5e83fff640 1 --2- 172.21.15.172:0/1465343040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cf63910 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5e740aba50 tx=0x7f5e7407b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1d3a85d368e1ac server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.443+0000 7f5e69ffb640 1 -- 172.21.15.172:0/1465343040 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74083fb0 con 0x56173cf90970 2024-04-14T08:37:54.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.443+0000 7f5e69ffb640 1 -- 172.21.15.172:0/1465343040 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e74006e10 con 0x56173cf90970 2024-04-14T08:37:54.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.443+0000 7f5e69ffb640 1 -- 172.21.15.172:0/1465343040 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74001370 con 0x56173cf90970 2024-04-14T08:37:54.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.444+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/1465343040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 msgr2=0x56173cf63910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.444+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/1465343040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cf63910 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5e740aba50 tx=0x7f5e7407b490 comp rx=0 tx=0).stop 2024-04-14T08:37:54.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.444+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/1465343040 shutdown_connections 2024-04-14T08:37:54.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.444+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/1465343040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf63e50 0x56173cf93c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.444+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/1465343040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 0x56173cf93700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.444+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/1465343040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cf63910 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.444+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/1465343040 >> 172.21.15.172:0/1465343040 conn(0x56173cf65b80 msgr2=0x56173cfae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.444+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/1465343040 shutdown_connections 2024-04-14T08:37:54.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.444+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/1465343040 wait complete. 2024-04-14T08:37:54.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.444+0000 7f5e8d5f3740 1 -- start start 2024-04-14T08:37:54.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e8d5f3740 1 Processor -- start 2024-04-14T08:37:54.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e8d5f3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cf88c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e8d5f3740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 0x56173cf77bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e8d5f3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf8bc60 0x56173cf8c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e8d5f3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56173ce757f0 con 0x56173cf90970 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e8d5f3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56173ce88d00 con 0x56173cf8bc60 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e8d5f3740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56173cecd800 con 0x56173cf90d50 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e8d5f3740 10 client.0 _ll_register_callbacks cb 0x56173ce886f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cf88c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cf88c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59696/0 (socket says 172.21.15.172:59696) 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e83fff640 1 -- 172.21.15.172:0/760722077 learned_addr learned my addr 172.21.15.172:0/760722077 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e88dac640 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf8bc60 0x56173cf8c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.445+0000 7f5e837fe640 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 0x56173cf77bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e83fff640 1 -- 172.21.15.172:0/760722077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 msgr2=0x56173cf77bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e83fff640 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 0x56173cf77bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e83fff640 1 -- 172.21.15.172:0/760722077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf8bc60 msgr2=0x56173cf8c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e83fff640 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf8bc60 0x56173cf8c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e83fff640 1 -- 172.21.15.172:0/760722077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56173cbe13f0 con 0x56173cf90970 2024-04-14T08:37:54.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e83fff640 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cf88c60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5e740abb00 tx=0x7f5e740ad050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34b96c182015d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:54.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e62ffd640 1 -- 172.21.15.172:0/760722077 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74083fb0 con 0x56173cf90970 2024-04-14T08:37:54.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e62ffd640 1 -- 172.21.15.172:0/760722077 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f5e74083b20 con 0x56173cf90970 2024-04-14T08:37:54.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/760722077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e74006e10 con 0x56173cf90970 2024-04-14T08:37:54.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e62ffd640 1 -- 172.21.15.172:0/760722077 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5e74083d20 con 0x56173cf90970 2024-04-14T08:37:54.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e8d5f3740 10 client.7131 fetch_fsmap learned FSMap version 46 2024-04-14T08:37:54.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/760722077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x56173cba10f0 con 0x56173cf90970 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e62ffd640 1 -- 172.21.15.172:0/760722077 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e74006e10 con 0x56173cf90970 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e62ffd640 1 -- 172.21.15.172:0/760722077 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7f5e74087030 con 0x56173cf90970 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e8d5f3740 10 client.7131 fetch_fsmap finished waiting for FSMap version 46 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e8d5f3740 10 client.7131 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e8d5f3740 20 client.7131 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56173cfc1fa0 0x56173cf96a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e8d5f3740 4 client.7131 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.446+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/760722077 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56173cedad40 con 0x56173cfc1fa0 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.447+0000 7f5e837fe640 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56173cfc1fa0 0x56173cf96a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.447+0000 7f5e62ffd640 10 client.7131 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.447+0000 7f5e837fe640 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56173cfc1fa0 0x56173cf96a90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b58500758b0ce2 server_cookie=d01a3669916f1cf2 in_seq=0 out_seq=0 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.447+0000 7f5e62ffd640 1 -- 172.21.15.172:0/760722077 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56173cedad40 con 0x56173cfc1fa0 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.447+0000 7f5e62ffd640 10 client.7131 handle_command_reply: tid=0 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.447+0000 7f5e8d5f3740 1 client.7131 shutdown 2024-04-14T08:37:54.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.448+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/760722077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 msgr2=0x56173cf88c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.448+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cf88c60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5e740abb00 tx=0x7f5e740ad050 comp rx=0 tx=0).stop 2024-04-14T08:37:54.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.448+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/760722077 shutdown_connections 2024-04-14T08:37:54.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.448+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56173cfc1fa0 0x56173cf96a90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.448+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf8bc60 0x56173cf8c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.448+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 0x56173cf77bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.448+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/760722077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cf88c60 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.448+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/760722077 >> 172.21.15.172:0/760722077 conn(0x56173cf63e50 msgr2=0x56173cfb1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.448+0000 7f5e62ffd640 0 client.7131 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:54.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.448+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/760722077 shutdown_connections 2024-04-14T08:37:54.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.448+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/760722077 wait complete. 2024-04-14T08:37:54.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.448+0000 7f5e8d5f3740 20 client.7131 trim_cache size 0 max 16384 2024-04-14T08:37:54.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.465+0000 7f5e8d5f3740 1 Processor -- start 2024-04-14T08:37:54.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.465+0000 7f5e8d5f3740 1 -- start start 2024-04-14T08:37:54.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e8d5f3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf8bc60 0x56173cfb1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e8d5f3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf90970 0x56173cfb2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e8d5f3740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 0x56173cfd1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e8d5f3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e84201970 con 0x56173cf8bc60 2024-04-14T08:37:54.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e8d5f3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e842009e0 con 0x56173cf90970 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e8d5f3740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e841e7660 con 0x56173cf90d50 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf8bc60 0x56173cfb1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf8bc60 0x56173cfb1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59712/0 (socket says 172.21.15.172:59712) 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e83fff640 1 -- 172.21.15.172:0/627090948 learned_addr learned my addr 172.21.15.172:0/627090948 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e837fe640 1 --2- 172.21.15.172:0/627090948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf90970 0x56173cfb2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e88dac640 1 --2- 172.21.15.172:0/627090948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 0x56173cfd1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e83fff640 1 -- 172.21.15.172:0/627090948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 msgr2=0x56173cfd1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e83fff640 1 --2- 172.21.15.172:0/627090948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 0x56173cfd1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e83fff640 1 -- 172.21.15.172:0/627090948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf90970 msgr2=0x56173cfb2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e83fff640 1 --2- 172.21.15.172:0/627090948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf90970 0x56173cfb2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e83fff640 1 -- 172.21.15.172:0/627090948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56173cf66fc0 con 0x56173cf8bc60 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.466+0000 7f5e83fff640 1 --2- 172.21.15.172:0/627090948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf8bc60 0x56173cfb1c10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5e740771c0 tx=0x7f5e74091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b551c45b412be65 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.467+0000 7f5e697fa640 1 -- 172.21.15.172:0/627090948 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74083fb0 con 0x56173cf8bc60 2024-04-14T08:37:54.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.467+0000 7f5e697fa640 1 -- 172.21.15.172:0/627090948 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e74079470 con 0x56173cf8bc60 2024-04-14T08:37:54.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.467+0000 7f5e697fa640 1 -- 172.21.15.172:0/627090948 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74077240 con 0x56173cf8bc60 2024-04-14T08:37:54.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.467+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/627090948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf8bc60 msgr2=0x56173cfb1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.467+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/627090948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf8bc60 0x56173cfb1c10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5e740771c0 tx=0x7f5e74091040 comp rx=0 tx=0).stop 2024-04-14T08:37:54.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.467+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/627090948 shutdown_connections 2024-04-14T08:37:54.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.467+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/627090948 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf90d50 0x56173cfd1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.467+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/627090948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf90970 0x56173cfb2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.467+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/627090948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf8bc60 0x56173cfb1c10 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.467+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/627090948 >> 172.21.15.172:0/627090948 conn(0x56173cf63e50 msgr2=0x56173cf5f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.467+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/627090948 shutdown_connections 2024-04-14T08:37:54.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.467+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/627090948 wait complete. 2024-04-14T08:37:54.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.468+0000 7f5e8d5f3740 1 -- start start 2024-04-14T08:37:54.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.468+0000 7f5e8d5f3740 1 Processor -- start 2024-04-14T08:37:54.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e8d5f3740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf8bc60 0x56173cfb1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e8d5f3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cfb2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e8d5f3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf90d50 0x56173cf68610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e8d5f3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e841e7cb0 con 0x56173cf90970 2024-04-14T08:37:54.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e8d5f3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56173cf92620 con 0x56173cf90d50 2024-04-14T08:37:54.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e8d5f3740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56173cfc0950 con 0x56173cf8bc60 2024-04-14T08:37:54.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e8d5f3740 10 client.0 _ll_register_callbacks cb 0x7f5e7407bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:54.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cfb2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cfb2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59724/0 (socket says 172.21.15.172:59724) 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e837fe640 1 -- 172.21.15.172:0/2214275159 learned_addr learned my addr 172.21.15.172:0/2214275159 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e83fff640 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf8bc60 0x56173cfb1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e88dac640 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf90d50 0x56173cf68610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e837fe640 1 -- 172.21.15.172:0/2214275159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf8bc60 msgr2=0x56173cfb1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e837fe640 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf8bc60 0x56173cfb1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e837fe640 1 -- 172.21.15.172:0/2214275159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf90d50 msgr2=0x56173cf68610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e837fe640 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf90d50 0x56173cf68610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e837fe640 1 -- 172.21.15.172:0/2214275159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56173cfd2de0 con 0x56173cf90970 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e837fe640 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cfb2150 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x56173cf916b0 tx=0x7f5e78017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8ed459adf4e4f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e62ffd640 1 -- 172.21.15.172:0/2214275159 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78032070 con 0x56173cf90970 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e62ffd640 1 -- 172.21.15.172:0/2214275159 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f5e7802e3b0 con 0x56173cf90970 2024-04-14T08:37:54.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.469+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/2214275159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e74079470 con 0x56173cf90970 2024-04-14T08:37:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e62ffd640 1 -- 172.21.15.172:0/2214275159 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5e7803d050 con 0x56173cf90970 2024-04-14T08:37:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e8d5f3740 10 client.7137 fetch_fsmap learned FSMap version 46 2024-04-14T08:37:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/2214275159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x56173cfc0680 con 0x56173cf90970 2024-04-14T08:37:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e62ffd640 1 -- 172.21.15.172:0/2214275159 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e74079470 con 0x56173cf90970 2024-04-14T08:37:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e62ffd640 1 -- 172.21.15.172:0/2214275159 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7f5e78040020 con 0x56173cf90970 2024-04-14T08:37:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e8d5f3740 10 client.7137 fetch_fsmap finished waiting for FSMap version 46 2024-04-14T08:37:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e8d5f3740 10 client.7137 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-14T08:37:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e8d5f3740 20 client.7137 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56173d027cd0 0x56173cf68b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e8d5f3740 4 client.7137 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:37:54.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/2214275159 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56173ced5090 con 0x56173d027cd0 2024-04-14T08:37:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e83fff640 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56173d027cd0 0x56173cf68b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e62ffd640 10 client.7137 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e83fff640 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56173d027cd0 0x56173cf68b50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fbc4afb26cedf1 server_cookie=b90628f6a2c10761 in_seq=0 out_seq=0 2024-04-14T08:37:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e62ffd640 1 -- 172.21.15.172:0/2214275159 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56173ced5090 con 0x56173d027cd0 2024-04-14T08:37:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.470+0000 7f5e62ffd640 10 client.7137 handle_command_reply: tid=0 2024-04-14T08:37:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8d5f3740 1 client.7137 shutdown 2024-04-14T08:37:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/2214275159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 msgr2=0x56173cfb2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cfb2150 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x56173cf916b0 tx=0x7f5e78017070 comp rx=0 tx=0).stop 2024-04-14T08:37:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/2214275159 shutdown_connections 2024-04-14T08:37:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56173d027cd0 0x56173cf68b50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56173cf90d50 0x56173cf68610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e62ffd640 0 client.7137 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56173cf90970 0x56173cfb2150 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8d5f3740 1 --2- 172.21.15.172:0/2214275159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56173cf8bc60 0x56173cfb1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/2214275159 >> 172.21.15.172:0/2214275159 conn(0x56173cf63e50 msgr2=0x56173cf8b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/2214275159 shutdown_connections 2024-04-14T08:37:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8d5f3740 1 -- 172.21.15.172:0/2214275159 wait complete. 2024-04-14T08:37:54.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8d5f3740 20 client.7137 trim_cache size 0 max 16384 2024-04-14T08:37:54.479 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:37:54.479 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-04-14T08:37:54.480 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-04-14T08:37:54.480 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:37:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8a80f640 1 -- 172.21.15.172:0/1624833950 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e7006c920 msgr2=0x7f5e7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8a80f640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e7006c920 0x7f5e7006ede0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5e841e95e0 tx=0x7f5e78009190 comp rx=0 tx=0).stop 2024-04-14T08:37:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8a80f640 1 -- 172.21.15.172:0/1624833950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 msgr2=0x7f5e841dca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8a80f640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 0x7f5e841dca00 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5e74005040 tx=0x7f5e74001120 comp rx=0 tx=0).stop 2024-04-14T08:37:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8a80f640 1 -- 172.21.15.172:0/1624833950 shutdown_connections 2024-04-14T08:37:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8a80f640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e7006c920 0x7f5e7006ede0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8a80f640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e840c6170 0x7f5e840c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8a80f640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e840b0ea0 0x7f5e840b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8a80f640 1 --2- 172.21.15.172:0/1624833950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e6c0039b0 0x7f5e841dca00 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.471+0000 7f5e8a80f640 1 -- 172.21.15.172:0/1624833950 >> 172.21.15.172:0/1624833950 conn(0x7f5e840cda00 msgr2=0x7f5e840addb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.472+0000 7f5e8a80f640 1 -- 172.21.15.172:0/1624833950 shutdown_connections 2024-04-14T08:37:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.472+0000 7f5e8a80f640 1 -- 172.21.15.172:0/1624833950 wait complete. 2024-04-14T08:37:54.483 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-14T08:37:54.483 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:37:54.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.585+0000 7fda15915640 1 Processor -- start 2024-04-14T08:37:54.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.585+0000 7fda15915640 1 -- start start 2024-04-14T08:37:54.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda15915640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fda100b1df0 0x7fda100b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda15915640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda100b21d0 0x7fda1015c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda15915640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda100b3430 con 0x7fda1015c5a0 2024-04-14T08:37:54.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda15915640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fda100b3700 con 0x7fda100b1df0 2024-04-14T08:37:54.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda15915640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fda100b39e0 con 0x7fda100b21d0 2024-04-14T08:37:54.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda0effd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fda100b1df0 0x7fda100b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda07fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda100b21d0 0x7fda1015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda0effd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fda100b1df0 0x7fda100b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:55188/0 (socket says 172.21.15.172:55188) 2024-04-14T08:37:54.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda0f7fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fda1015c5a0 0x7fda100b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:38162/0 (socket says 172.21.15.172:38162) 2024-04-14T08:37:54.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda07fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda100b21d0 0x7fda1015c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:55318/0 (socket says 172.21.15.172:55318) 2024-04-14T08:37:54.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda0effd640 1 -- 172.21.15.172:0/395743514 learned_addr learned my addr 172.21.15.172:0/395743514 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda07fff640 1 -- 172.21.15.172:0/395743514 >> v2:172.21.15.177:3300/0 conn(0x7fda100b1df0 msgr2=0x7fda100b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda07fff640 1 --2- 172.21.15.172:0/395743514 >> v2:172.21.15.177:3300/0 conn(0x7fda100b1df0 0x7fda100b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda07fff640 1 -- 172.21.15.172:0/395743514 >> v1:172.21.15.177:6789/0 conn(0x7fda1015c5a0 legacy=0x7fda100b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.586+0000 7fda07fff640 1 -- 172.21.15.172:0/395743514 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda100b3cc0 con 0x7fda100b21d0 2024-04-14T08:37:54.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.587+0000 7fda07fff640 1 --2- 172.21.15.172:0/395743514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda100b21d0 0x7fda1015c060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80097a0 tx=0x7fd9f802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cefb5ca075124cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.587+0000 7fda0e7fc640 1 -- 172.21.15.172:0/395743514 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8035020 con 0x7fda100b21d0 2024-04-14T08:37:54.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.587+0000 7fda0e7fc640 1 -- 172.21.15.172:0/395743514 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f8031830 con 0x7fda100b21d0 2024-04-14T08:37:54.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.587+0000 7fda0e7fc640 1 -- 172.21.15.172:0/395743514 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8031c00 con 0x7fda100b21d0 2024-04-14T08:37:54.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.588+0000 7fda15915640 1 -- 172.21.15.172:0/395743514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda100b21d0 msgr2=0x7fda1015c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.588+0000 7fda15915640 1 --2- 172.21.15.172:0/395743514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda100b21d0 0x7fda1015c060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80097a0 tx=0x7fd9f802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.588+0000 7fda15915640 1 -- 172.21.15.172:0/395743514 shutdown_connections 2024-04-14T08:37:54.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.588+0000 7fda15915640 1 --2- 172.21.15.172:0/395743514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda100b21d0 0x7fda1015c060 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.588+0000 7fda15915640 1 --2- 172.21.15.172:0/395743514 >> v2:172.21.15.177:3300/0 conn(0x7fda100b1df0 0x7fda100b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.588+0000 7fda15915640 1 -- 172.21.15.172:0/395743514 >> 172.21.15.172:0/395743514 conn(0x7fda1006ec30 msgr2=0x7fda1006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.588+0000 7fda15915640 1 -- 172.21.15.172:0/395743514 shutdown_connections 2024-04-14T08:37:54.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.588+0000 7fda15915640 1 -- 172.21.15.172:0/395743514 wait complete. 2024-04-14T08:37:54.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.589+0000 7fda15915640 1 Processor -- start 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.589+0000 7fda15915640 1 -- start start 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda15915640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda100b1df0 0x7fda100aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda15915640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda100b21d0 0x7fda10143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda15915640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda10141870 0x7fda10141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda15915640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda100b4750 con 0x7fda100b21d0 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda15915640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fda1015b7f0 con 0x7fda100b1df0 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda15915640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fda100b48d0 con 0x7fda10141870 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda07fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda100b21d0 0x7fda10143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda07fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda100b21d0 0x7fda10143c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59730/0 (socket says 172.21.15.172:59730) 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda07fff640 1 -- 172.21.15.172:0/4013502447 learned_addr learned my addr 172.21.15.172:0/4013502447 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda0f7fe640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda10141870 0x7fda10141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda0effd640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda100b1df0 0x7fda100aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda07fff640 1 -- 172.21.15.172:0/4013502447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda10141870 msgr2=0x7fda10141cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda07fff640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda10141870 0x7fda10141cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda07fff640 1 -- 172.21.15.172:0/4013502447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda100b1df0 msgr2=0x7fda100aa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda07fff640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda100b1df0 0x7fda100aa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda07fff640 1 -- 172.21.15.172:0/4013502447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda101407d0 con 0x7fda100b21d0 2024-04-14T08:37:54.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda07fff640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda100b21d0 0x7fda10143c10 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd9f802bee0 tx=0x7fd9f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.590+0000 7fda0cff9640 1 -- 172.21.15.172:0/4013502447 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8041070 con 0x7fda100b21d0 2024-04-14T08:37:54.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.591+0000 7fda0cff9640 1 -- 172.21.15.172:0/4013502447 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f803f070 con 0x7fda100b21d0 2024-04-14T08:37:54.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.591+0000 7fda15915640 1 -- 172.21.15.172:0/4013502447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda10140ab0 con 0x7fda100b21d0 2024-04-14T08:37:54.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.591+0000 7fda0cff9640 1 -- 172.21.15.172:0/4013502447 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8035040 con 0x7fda100b21d0 2024-04-14T08:37:54.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.591+0000 7fda15915640 1 -- 172.21.15.172:0/4013502447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda10141070 con 0x7fda100b21d0 2024-04-14T08:37:54.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.592+0000 7fda15915640 1 -- 172.21.15.172:0/4013502447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda10068520 con 0x7fda100b21d0 2024-04-14T08:37:54.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.592+0000 7fda0cff9640 1 -- 172.21.15.172:0/4013502447 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9f8038020 con 0x7fda100b21d0 2024-04-14T08:37:54.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.593+0000 7fda0cff9640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd9f006c8f0 0x7fd9f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.593+0000 7fda0effd640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd9f006c8f0 0x7fd9f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.593+0000 7fda0cff9640 1 -- 172.21.15.172:0/4013502447 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fd9f80ad500 con 0x7fda100b21d0 2024-04-14T08:37:54.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.593+0000 7fda0effd640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd9f006c8f0 0x7fd9f006edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fda00009bd0 tx=0x7fda00012040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.597+0000 7fda0cff9640 1 -- 172.21.15.172:0/4013502447 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd9f8031800 con 0x7fda100b21d0 2024-04-14T08:37:54.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.829+0000 7fda15915640 1 -- 172.21.15.172:0/4013502447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda101413e0 con 0x7fda100b21d0 2024-04-14T08:37:54.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.829+0000 7fda0cff9640 1 -- 172.21.15.172:0/4013502447 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5193 (secure 0 0 0) 0x7fd9f807b190 con 0x7fda100b21d0 2024-04-14T08:37:54.830 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:54.830 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:37:01.115031+0000","modified":"2024-04-14T08:37:26.376162+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":6,"addr":"172.21.15.172:6837/2920402198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2920402198},{"type":"v1","addr":"172.21.15.172:6837","nonce":2920402198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":5}]} 2024-04-14T08:37:54.831 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 46 2024-04-14T08:37:54.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.832+0000 7fda15915640 1 -- 172.21.15.172:0/4013502447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd9f006c8f0 msgr2=0x7fd9f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.832+0000 7fda15915640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd9f006c8f0 0x7fd9f006edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fda00009bd0 tx=0x7fda00012040 comp rx=0 tx=0).stop 2024-04-14T08:37:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.832+0000 7fda15915640 1 -- 172.21.15.172:0/4013502447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda100b21d0 msgr2=0x7fda10143c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.832+0000 7fda15915640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda100b21d0 0x7fda10143c10 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd9f802bee0 tx=0x7fd9f80046c0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.832+0000 7fda15915640 1 -- 172.21.15.172:0/4013502447 shutdown_connections 2024-04-14T08:37:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.832+0000 7fda15915640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd9f006c8f0 0x7fd9f006edb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.832+0000 7fda15915640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda10141870 0x7fda10141cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.832+0000 7fda15915640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda100b21d0 0x7fda10143c10 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.832+0000 7fda15915640 1 --2- 172.21.15.172:0/4013502447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda100b1df0 0x7fda100aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.832+0000 7fda15915640 1 -- 172.21.15.172:0/4013502447 >> 172.21.15.172:0/4013502447 conn(0x7fda1006ec30 msgr2=0x7fda10144760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.832+0000 7fda15915640 1 -- 172.21.15.172:0/4013502447 shutdown_connections 2024-04-14T08:37:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.832+0000 7fda15915640 1 -- 172.21.15.172:0/4013502447 wait complete. 2024-04-14T08:37:54.844 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-14T08:37:54.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.946+0000 7ff3108de640 1 Processor -- start 2024-04-14T08:37:54.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.946+0000 7ff3108de640 1 -- start start 2024-04-14T08:37:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.947+0000 7ff3108de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff30c0b65a0 0x7ff30c1629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.947+0000 7ff3108de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff30c0cea60 0x7ff30c0d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.947+0000 7ff3108de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff30c0b3750 con 0x7ff30c15e590 2024-04-14T08:37:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.947+0000 7ff3108de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff30c0cef20 con 0x7ff30c0cea60 2024-04-14T08:37:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.947+0000 7ff3108de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff30c0cf200 con 0x7ff30c0b65a0 2024-04-14T08:37:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.948+0000 7ff30ad9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff30c0cea60 0x7ff30c0d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.948+0000 7ff30a59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff30c15e590 0x7ff30c0b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:42104/0 (socket says 172.21.15.172:42104) 2024-04-14T08:37:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.948+0000 7ff30a59c640 1 -- 172.21.15.172:0/438292412 learned_addr learned my addr 172.21.15.172:0/438292412 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.948+0000 7ff309d9b640 1 --2- 172.21.15.172:0/438292412 >> v2:172.21.15.177:3300/0 conn(0x7ff30c0b65a0 0x7ff30c1629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.948+0000 7ff30ad9d640 1 -- 172.21.15.172:0/438292412 >> v2:172.21.15.177:3300/0 conn(0x7ff30c0b65a0 msgr2=0x7ff30c1629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.948+0000 7ff30ad9d640 1 --2- 172.21.15.172:0/438292412 >> v2:172.21.15.177:3300/0 conn(0x7ff30c0b65a0 0x7ff30c1629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.948+0000 7ff30ad9d640 1 -- 172.21.15.172:0/438292412 >> v1:172.21.15.172:6789/0 conn(0x7ff30c15e590 legacy=0x7ff30c0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.948+0000 7ff30ad9d640 1 -- 172.21.15.172:0/438292412 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c0cf4e0 con 0x7ff30c0cea60 2024-04-14T08:37:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.948+0000 7ff30959a640 1 -- 172.21.15.172:0/438292412 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1820485126 0 0) 0x7ff30c0b3750 con 0x7ff30c15e590 2024-04-14T08:37:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.948+0000 7ff309d9b640 1 --2- 172.21.15.172:0/438292412 >> v2:172.21.15.177:3300/0 conn(0x7ff30c0b65a0 0x7ff30c1629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:54.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.949+0000 7ff30ad9d640 1 --2- 172.21.15.172:0/438292412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff30c0cea60 0x7ff30c0d2760 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0d5510 tx=0x7ff2fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76138c17a25846a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.949+0000 7ff30959a640 1 -- 172.21.15.172:0/438292412 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc035020 con 0x7ff30c0cea60 2024-04-14T08:37:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.949+0000 7ff30959a640 1 -- 172.21.15.172:0/438292412 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2fc031830 con 0x7ff30c0cea60 2024-04-14T08:37:54.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.949+0000 7ff30959a640 1 -- 172.21.15.172:0/438292412 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc031c00 con 0x7ff30c0cea60 2024-04-14T08:37:54.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.949+0000 7ff3108de640 1 -- 172.21.15.172:0/438292412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff30c0cea60 msgr2=0x7ff30c0d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.949+0000 7ff3108de640 1 --2- 172.21.15.172:0/438292412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff30c0cea60 0x7ff30c0d2760 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0d5510 tx=0x7ff2fc02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.950+0000 7ff3108de640 1 -- 172.21.15.172:0/438292412 shutdown_connections 2024-04-14T08:37:54.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.950+0000 7ff3108de640 1 --2- 172.21.15.172:0/438292412 >> v2:172.21.15.177:3300/0 conn(0x7ff30c0b65a0 0x7ff30c1629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.950+0000 7ff3108de640 1 --2- 172.21.15.172:0/438292412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff30c0cea60 0x7ff30c0d2760 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.950+0000 7ff3108de640 1 -- 172.21.15.172:0/438292412 >> 172.21.15.172:0/438292412 conn(0x7ff30c0d0e90 msgr2=0x7ff30c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.950+0000 7ff3108de640 1 -- 172.21.15.172:0/438292412 shutdown_connections 2024-04-14T08:37:54.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.950+0000 7ff3108de640 1 -- 172.21.15.172:0/438292412 wait complete. 2024-04-14T08:37:54.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.952+0000 7ff3108de640 1 Processor -- start 2024-04-14T08:37:54.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.952+0000 7ff3108de640 1 -- start start 2024-04-14T08:37:54.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff3108de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff30c0b65a0 0x7ff30c161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff3108de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff30c0cea60 0x7ff30c15e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff3108de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30c15e590 0x7ff30c14fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff3108de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c0d2e20 con 0x7ff30c0b65a0 2024-04-14T08:37:54.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff3108de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c0d2ca0 con 0x7ff30c15e590 2024-04-14T08:37:54.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff3108de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff30c0d2fa0 con 0x7ff30c0cea60 2024-04-14T08:37:54.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff30a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff30c0b65a0 0x7ff30c161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff30a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff30c0b65a0 0x7ff30c161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59746/0 (socket says 172.21.15.172:59746) 2024-04-14T08:37:54.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff30a59c640 1 -- 172.21.15.172:0/1350434419 learned_addr learned my addr 172.21.15.172:0/1350434419 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff30a59c640 1 -- 172.21.15.172:0/1350434419 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff30c0cea60 msgr2=0x7ff30c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff30a59c640 1 --2- 172.21.15.172:0/1350434419 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff30c0cea60 0x7ff30c15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff30a59c640 1 -- 172.21.15.172:0/1350434419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30c15e590 msgr2=0x7ff30c14fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff30a59c640 1 --2- 172.21.15.172:0/1350434419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30c15e590 0x7ff30c14fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff30a59c640 1 -- 172.21.15.172:0/1350434419 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c15f0e0 con 0x7ff30c0b65a0 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff30a59c640 1 --2- 172.21.15.172:0/1350434419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff30c0b65a0 0x7ff30c161490 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff30400c430 tx=0x7ff30400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff2faffd640 1 -- 172.21.15.172:0/1350434419 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3040074e0 con 0x7ff30c0b65a0 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff2faffd640 1 -- 172.21.15.172:0/1350434419 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3040047e0 con 0x7ff30c0b65a0 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff2faffd640 1 -- 172.21.15.172:0/1350434419 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304004bd0 con 0x7ff30c0b65a0 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.953+0000 7ff3108de640 1 -- 172.21.15.172:0/1350434419 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c15f3c0 con 0x7ff30c0b65a0 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.954+0000 7ff3108de640 1 -- 172.21.15.172:0/1350434419 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c15f980 con 0x7ff30c0b65a0 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.955+0000 7ff2faffd640 1 -- 172.21.15.172:0/1350434419 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff304011070 con 0x7ff30c0b65a0 2024-04-14T08:37:54.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.955+0000 7ff2faffd640 1 --2- 172.21.15.172:0/1350434419 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff2ec06c8f0 0x7ff2ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.955+0000 7ff2faffd640 1 -- 172.21.15.172:0/1350434419 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7ff30408b9f0 con 0x7ff30c0b65a0 2024-04-14T08:37:54.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.955+0000 7ff309d9b640 1 --2- 172.21.15.172:0/1350434419 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff2ec06c8f0 0x7ff2ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.956+0000 7ff309d9b640 1 --2- 172.21.15.172:0/1350434419 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff2ec06c8f0 0x7ff2ec06edb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff3000094c0 tx=0x7ff300009430 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.957+0000 7ff3137ba740 1 Processor -- start 2024-04-14T08:37:54.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.958+0000 7ff3137ba740 1 -- start start 2024-04-14T08:37:54.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff3137ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3160d90 0x5629c3161170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff3137ba740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c3166bb0 0x5629c3164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff3137ba740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c31ae4a0 0x5629c31ae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff3137ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5629c31285a0 con 0x5629c3160d90 2024-04-14T08:37:54.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff3137ba740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5629c30ca680 con 0x5629c31ae4a0 2024-04-14T08:37:54.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff3137ba740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5629c3068860 con 0x5629c3166bb0 2024-04-14T08:37:54.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff30a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3160d90 0x5629c3161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff30a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3160d90 0x5629c3161170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59758/0 (socket says 172.21.15.172:59758) 2024-04-14T08:37:54.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff30a59c640 1 -- 172.21.15.172:0/2854335000 learned_addr learned my addr 172.21.15.172:0/2854335000 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:54.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff309d9b640 1 --2- 172.21.15.172:0/2854335000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c3166bb0 0x5629c3164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff30ad9d640 1 --2- 172.21.15.172:0/2854335000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c31ae4a0 0x5629c31ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff30a59c640 1 -- 172.21.15.172:0/2854335000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c3166bb0 msgr2=0x5629c3164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff30a59c640 1 --2- 172.21.15.172:0/2854335000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c3166bb0 0x5629c3164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff30a59c640 1 -- 172.21.15.172:0/2854335000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c31ae4a0 msgr2=0x5629c31ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff30a59c640 1 --2- 172.21.15.172:0/2854335000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c31ae4a0 0x5629c31ae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff30a59c640 1 -- 172.21.15.172:0/2854335000 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c30971a0 con 0x5629c3160d90 2024-04-14T08:37:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff30a59c640 1 --2- 172.21.15.172:0/2854335000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3160d90 0x5629c3161170 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff304089d40 tx=0x7ff304059d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0fc0fc8a50a83e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.959+0000 7ff2ebfff640 1 -- 172.21.15.172:0/2854335000 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304062120 con 0x5629c3160d90 2024-04-14T08:37:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.960+0000 7ff2ebfff640 1 -- 172.21.15.172:0/2854335000 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff304057840 con 0x5629c3160d90 2024-04-14T08:37:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.960+0000 7ff2ebfff640 1 -- 172.21.15.172:0/2854335000 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30401ad50 con 0x5629c3160d90 2024-04-14T08:37:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.960+0000 7ff3137ba740 1 -- 172.21.15.172:0/2854335000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3160d90 msgr2=0x5629c3161170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.960+0000 7ff3137ba740 1 --2- 172.21.15.172:0/2854335000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3160d90 0x5629c3161170 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff304089d40 tx=0x7ff304059d80 comp rx=0 tx=0).stop 2024-04-14T08:37:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.960+0000 7ff3137ba740 1 -- 172.21.15.172:0/2854335000 shutdown_connections 2024-04-14T08:37:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.960+0000 7ff3137ba740 1 --2- 172.21.15.172:0/2854335000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c31ae4a0 0x5629c31ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.960+0000 7ff3137ba740 1 --2- 172.21.15.172:0/2854335000 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c3166bb0 0x5629c3164de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.960+0000 7ff3137ba740 1 --2- 172.21.15.172:0/2854335000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3160d90 0x5629c3161170 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.960+0000 7ff3137ba740 1 -- 172.21.15.172:0/2854335000 >> 172.21.15.172:0/2854335000 conn(0x5629c3162e10 msgr2=0x5629c31646d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.960+0000 7ff3137ba740 1 -- 172.21.15.172:0/2854335000 shutdown_connections 2024-04-14T08:37:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.960+0000 7ff3137ba740 1 -- 172.21.15.172:0/2854335000 wait complete. 2024-04-14T08:37:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.961+0000 7ff3137ba740 1 -- start start 2024-04-14T08:37:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.961+0000 7ff3137ba740 1 Processor -- start 2024-04-14T08:37:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff3137ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 0x5629c3185c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff3137ba740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c31bc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff3137ba740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3193ae0 0x5629c31a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff3137ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5629c30661d0 con 0x5629c3166bb0 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff3137ba740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5629c30c6630 con 0x5629c3193ae0 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff3137ba740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5629c318d500 con 0x5629c31ae4a0 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff3137ba740 10 client.0 _ll_register_callbacks cb 0x5629c3089010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff30a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 0x5629c3185c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff30a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 0x5629c3185c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59760/0 (socket says 172.21.15.172:59760) 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff30a59c640 1 -- 172.21.15.172:0/34148480 learned_addr learned my addr 172.21.15.172:0/34148480 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff309d9b640 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c31bc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff30ad9d640 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3193ae0 0x5629c31a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff30a59c640 1 -- 172.21.15.172:0/34148480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 msgr2=0x5629c31bc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff30a59c640 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c31bc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff30a59c640 1 -- 172.21.15.172:0/34148480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3193ae0 msgr2=0x5629c31a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff30a59c640 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3193ae0 0x5629c31a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff30a59c640 1 -- 172.21.15.172:0/34148480 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c2eaa7c0 con 0x5629c3166bb0 2024-04-14T08:37:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff30ad9d640 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3193ae0 0x5629c31a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff30a59c640 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 0x5629c3185c70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff304059240 tx=0x7ff304000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7276a135ce733c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.962+0000 7ff2e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff2e97fa640 1 -- 172.21.15.172:0/34148480 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304076060 con 0x5629c3166bb0 2024-04-14T08:37:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff2e97fa640 1 -- 172.21.15.172:0/34148480 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7ff304055bb0 con 0x5629c3166bb0 2024-04-14T08:37:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff3137ba740 1 -- 172.21.15.172:0/34148480 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff304057840 con 0x5629c3166bb0 2024-04-14T08:37:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff2e97fa640 1 -- 172.21.15.172:0/34148480 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff304078e60 con 0x5629c3166bb0 2024-04-14T08:37:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff3137ba740 10 client.7155 fetch_fsmap learned FSMap version 46 2024-04-14T08:37:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff3137ba740 1 -- 172.21.15.172:0/34148480 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x5629c2f0af50 con 0x5629c3166bb0 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff2e97fa640 1 -- 172.21.15.172:0/34148480 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff304057840 con 0x5629c3166bb0 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff2e97fa640 1 -- 172.21.15.172:0/34148480 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7ff304058990 con 0x5629c3166bb0 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff3137ba740 10 client.7155 fetch_fsmap finished waiting for FSMap version 46 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff3137ba740 10 client.7155 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff3137ba740 20 client.7155 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff3137ba740 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5629c3191d10 0x5629c31920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff3137ba740 4 client.7155 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff3137ba740 1 -- 172.21.15.172:0/34148480 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629c30ddeb0 con 0x5629c3191d10 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.963+0000 7ff309d9b640 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5629c3191d10 0x5629c31920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff2e97fa640 10 client.7155 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff309d9b640 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5629c3191d10 0x5629c31920f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a552ed6f3be83f server_cookie=b0fc80235e91f77f in_seq=0 out_seq=0 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff2e97fa640 1 -- 172.21.15.172:0/34148480 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629c30ddeb0 con 0x5629c3191d10 2024-04-14T08:37:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff2e97fa640 10 client.7155 handle_command_reply: tid=0 2024-04-14T08:37:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff3137ba740 1 client.7155 shutdown 2024-04-14T08:37:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff3137ba740 1 -- 172.21.15.172:0/34148480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 msgr2=0x5629c3185c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff3137ba740 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 0x5629c3185c70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff304059240 tx=0x7ff304000ed0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff3137ba740 1 -- 172.21.15.172:0/34148480 shutdown_connections 2024-04-14T08:37:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff3137ba740 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5629c3191d10 0x5629c31920f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff3137ba740 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3193ae0 0x5629c31a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff3137ba740 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c31bc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff3137ba740 1 --2- 172.21.15.172:0/34148480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 0x5629c3185c70 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff2e97fa640 0 client.7155 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff3137ba740 1 -- 172.21.15.172:0/34148480 >> 172.21.15.172:0/34148480 conn(0x5629c3160d90 msgr2=0x5629c31602c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff3137ba740 1 -- 172.21.15.172:0/34148480 shutdown_connections 2024-04-14T08:37:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.964+0000 7ff3137ba740 1 -- 172.21.15.172:0/34148480 wait complete. 2024-04-14T08:37:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.965+0000 7ff3137ba740 20 client.7155 trim_cache size 0 max 16384 2024-04-14T08:37:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.978+0000 7ff3137ba740 1 Processor -- start 2024-04-14T08:37:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.978+0000 7ff3137ba740 1 -- start start 2024-04-14T08:37:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff3137ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 0x5629c31ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff3137ba740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3193ae0 0x5629c318fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff3137ba740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c3190050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff3137ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c200040 con 0x5629c3166bb0 2024-04-14T08:37:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff3137ba740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c0cd220 con 0x5629c3193ae0 2024-04-14T08:37:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff3137ba740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff30c1ff9f0 con 0x5629c31ae4a0 2024-04-14T08:37:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff30a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 0x5629c31ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff30a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 0x5629c31ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59776/0 (socket says 172.21.15.172:59776) 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff30a59c640 1 -- 172.21.15.172:0/3649087364 learned_addr learned my addr 172.21.15.172:0/3649087364 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff309d9b640 1 --2- 172.21.15.172:0/3649087364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3193ae0 0x5629c318fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff30ad9d640 1 --2- 172.21.15.172:0/3649087364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c3190050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff30a59c640 1 -- 172.21.15.172:0/3649087364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 msgr2=0x5629c3190050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff30a59c640 1 --2- 172.21.15.172:0/3649087364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c3190050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff30a59c640 1 -- 172.21.15.172:0/3649087364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3193ae0 msgr2=0x5629c318fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff30a59c640 1 --2- 172.21.15.172:0/3649087364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3193ae0 0x5629c318fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff30a59c640 1 -- 172.21.15.172:0/3649087364 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c2d9e0b0 con 0x5629c3166bb0 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff309d9b640 1 --2- 172.21.15.172:0/3649087364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3193ae0 0x5629c318fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff30ad9d640 1 --2- 172.21.15.172:0/3649087364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c3190050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff30a59c640 1 --2- 172.21.15.172:0/3649087364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 0x5629c31ac2f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff3040577f0 tx=0x7ff304085070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5b3c3d02345806 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff2eb7fe640 1 -- 172.21.15.172:0/3649087364 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304059080 con 0x5629c3166bb0 2024-04-14T08:37:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff2eb7fe640 1 -- 172.21.15.172:0/3649087364 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff304058990 con 0x5629c3166bb0 2024-04-14T08:37:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.979+0000 7ff2eb7fe640 1 -- 172.21.15.172:0/3649087364 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304055290 con 0x5629c3166bb0 2024-04-14T08:37:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.980+0000 7ff3137ba740 1 -- 172.21.15.172:0/3649087364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 msgr2=0x5629c31ac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.980+0000 7ff3137ba740 1 --2- 172.21.15.172:0/3649087364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 0x5629c31ac2f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff3040577f0 tx=0x7ff304085070 comp rx=0 tx=0).stop 2024-04-14T08:37:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.980+0000 7ff3137ba740 1 -- 172.21.15.172:0/3649087364 shutdown_connections 2024-04-14T08:37:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.980+0000 7ff3137ba740 1 --2- 172.21.15.172:0/3649087364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c3190050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.980+0000 7ff3137ba740 1 --2- 172.21.15.172:0/3649087364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3193ae0 0x5629c318fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.980+0000 7ff3137ba740 1 --2- 172.21.15.172:0/3649087364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3166bb0 0x5629c31ac2f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.980+0000 7ff3137ba740 1 -- 172.21.15.172:0/3649087364 >> 172.21.15.172:0/3649087364 conn(0x5629c3160d90 msgr2=0x5629c3185a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.980+0000 7ff3137ba740 1 -- 172.21.15.172:0/3649087364 shutdown_connections 2024-04-14T08:37:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.980+0000 7ff3137ba740 1 -- 172.21.15.172:0/3649087364 wait complete. 2024-04-14T08:37:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.981+0000 7ff3137ba740 1 -- start start 2024-04-14T08:37:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.981+0000 7ff3137ba740 1 Processor -- start 2024-04-14T08:37:54.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff3137ba740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3166bb0 0x5629c3165b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff3137ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3193ae0 0x5629c315fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff3137ba740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c31ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff3137ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5629c30661d0 con 0x5629c3193ae0 2024-04-14T08:37:54.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff3137ba740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5629c30c6630 con 0x5629c3166bb0 2024-04-14T08:37:54.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff3137ba740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5629c31a44a0 con 0x5629c31ae4a0 2024-04-14T08:37:54.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff3137ba740 10 client.0 _ll_register_callbacks cb 0x7ff30406df50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:54.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff309d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3193ae0 0x5629c315fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff309d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3193ae0 0x5629c315fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59790/0 (socket says 172.21.15.172:59790) 2024-04-14T08:37:54.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff309d9b640 1 -- 172.21.15.172:0/2322258656 learned_addr learned my addr 172.21.15.172:0/2322258656 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:54.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff30a59c640 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3166bb0 0x5629c3165b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff30ad9d640 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c31ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff309d9b640 1 -- 172.21.15.172:0/2322258656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 msgr2=0x5629c31ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff309d9b640 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c31ac2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff309d9b640 1 -- 172.21.15.172:0/2322258656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3166bb0 msgr2=0x5629c3165b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:54.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff309d9b640 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3166bb0 0x5629c3165b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff309d9b640 1 -- 172.21.15.172:0/2322258656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c3190400 con 0x5629c3193ae0 2024-04-14T08:37:54.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff30a59c640 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3166bb0 0x5629c3165b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:54.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff309d9b640 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3193ae0 0x5629c315fa30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff300007fd0 tx=0x7ff300010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a44f5a241cc065 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:54.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff2e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:54.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff2e97fa640 1 -- 172.21.15.172:0/2322258656 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300026250 con 0x5629c3193ae0 2024-04-14T08:37:54.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff2e97fa640 1 -- 172.21.15.172:0/2322258656 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7ff3000263f0 con 0x5629c3193ae0 2024-04-14T08:37:54.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff3137ba740 1 -- 172.21.15.172:0/2322258656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff304058990 con 0x5629c3193ae0 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff2e97fa640 1 -- 172.21.15.172:0/2322258656 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff300033050 con 0x5629c3193ae0 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff3137ba740 10 client.7161 fetch_fsmap learned FSMap version 46 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.982+0000 7ff3137ba740 1 -- 172.21.15.172:0/2322258656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x5629c3160490 con 0x5629c3193ae0 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff2e97fa640 1 -- 172.21.15.172:0/2322258656 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff304058990 con 0x5629c3193ae0 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff2e97fa640 1 -- 172.21.15.172:0/2322258656 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7ff300036020 con 0x5629c3193ae0 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff3137ba740 10 client.7161 fetch_fsmap finished waiting for FSMap version 46 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff3137ba740 10 client.7161 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff3137ba740 20 client.7161 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff3137ba740 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5629c31ce910 0x5629c31cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff3137ba740 4 client.7161 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff3137ba740 1 -- 172.21.15.172:0/2322258656 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5629c31a57f0 con 0x5629c31ce910 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff30a59c640 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5629c31ce910 0x5629c31cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff2e97fa640 10 client.7161 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff30a59c640 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5629c31ce910 0x5629c31cecf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=591525c3927156fc server_cookie=1e1b039806a63278 in_seq=0 out_seq=0 2024-04-14T08:37:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff2e97fa640 1 -- 172.21.15.172:0/2322258656 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x5629c31a57f0 con 0x5629c31ce910 2024-04-14T08:37:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff2e97fa640 10 client.7161 handle_command_reply: tid=0 2024-04-14T08:37:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff3137ba740 1 client.7161 shutdown 2024-04-14T08:37:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff3137ba740 1 -- 172.21.15.172:0/2322258656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3193ae0 msgr2=0x5629c315fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.983+0000 7ff3137ba740 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3193ae0 0x5629c315fa30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff300007fd0 tx=0x7ff300010000 comp rx=0 tx=0).stop 2024-04-14T08:37:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3137ba740 1 -- 172.21.15.172:0/2322258656 shutdown_connections 2024-04-14T08:37:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3137ba740 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5629c31ce910 0x5629c31cecf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3137ba740 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5629c31ae4a0 0x5629c31ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3137ba740 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c3193ae0 0x5629c315fa30 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff2e97fa640 0 client.7161 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3137ba740 1 --2- 172.21.15.172:0/2322258656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5629c3166bb0 0x5629c3165b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3137ba740 1 -- 172.21.15.172:0/2322258656 >> 172.21.15.172:0/2322258656 conn(0x5629c3160d90 msgr2=0x5629c3192410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3137ba740 1 -- 172.21.15.172:0/2322258656 shutdown_connections 2024-04-14T08:37:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3137ba740 1 -- 172.21.15.172:0/2322258656 wait complete. 2024-04-14T08:37:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3137ba740 20 client.7161 trim_cache size 0 max 16384 2024-04-14T08:37:54.993 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:54.993 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-04-14T08:37:54.994 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:37:54.994 INFO:teuthology.orchestra.run.smithi172.stdout: "damage_type": "dentry", 2024-04-14T08:37:54.994 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 3804048562, 2024-04-14T08:37:54.994 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 1099511627776, 2024-04-14T08:37:54.994 INFO:teuthology.orchestra.run.smithi172.stdout: "frag": "*", 2024-04-14T08:37:54.994 INFO:teuthology.orchestra.run.smithi172.stdout: "dname": "file_to_be_damaged", 2024-04-14T08:37:54.994 INFO:teuthology.orchestra.run.smithi172.stdout: "snap_id": "head", 2024-04-14T08:37:54.994 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-14T08:37:54.994 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:37:54.994 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-04-14T08:37:54.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3108de640 1 -- 172.21.15.172:0/1350434419 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff2ec06c8f0 msgr2=0x7ff2ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3108de640 1 --2- 172.21.15.172:0/1350434419 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff2ec06c8f0 0x7ff2ec06edb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff3000094c0 tx=0x7ff300009430 comp rx=0 tx=0).stop 2024-04-14T08:37:54.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3108de640 1 -- 172.21.15.172:0/1350434419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff30c0b65a0 msgr2=0x7ff30c161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:54.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3108de640 1 --2- 172.21.15.172:0/1350434419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff30c0b65a0 0x7ff30c161490 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff30400c430 tx=0x7ff30400c8f0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3108de640 1 -- 172.21.15.172:0/1350434419 shutdown_connections 2024-04-14T08:37:54.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3108de640 1 --2- 172.21.15.172:0/1350434419 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff2ec06c8f0 0x7ff2ec06edb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3108de640 1 --2- 172.21.15.172:0/1350434419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30c15e590 0x7ff30c14fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3108de640 1 --2- 172.21.15.172:0/1350434419 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff30c0cea60 0x7ff30c15e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3108de640 1 --2- 172.21.15.172:0/1350434419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff30c0b65a0 0x7ff30c161490 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:54.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3108de640 1 -- 172.21.15.172:0/1350434419 >> 172.21.15.172:0/1350434419 conn(0x7ff30c0d0e90 msgr2=0x7ff30c0b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:54.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3108de640 1 -- 172.21.15.172:0/1350434419 shutdown_connections 2024-04-14T08:37:54.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:54.984+0000 7ff3108de640 1 -- 172.21.15.172:0/1350434419 wait complete. 2024-04-14T08:37:54.999 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-14T08:37:55.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.098+0000 7f68a122d640 1 Processor -- start 2024-04-14T08:37:55.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.098+0000 7f68a122d640 1 -- start start 2024-04-14T08:37:55.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.099+0000 7f68a122d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f689c15e640 0x7f689c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.099+0000 7f68a122d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f689c0d1af0 con 0x7f689c06bc30 2024-04-14T08:37:55.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.099+0000 7f68a122d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f689c0c7840 con 0x7f689c0b0630 2024-04-14T08:37:55.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.099+0000 7f68a122d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f689c15ea20 con 0x7f689c15e640 2024-04-14T08:37:55.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.099+0000 7f689b59e640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f689c06bc30 0x7f689c162530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:42108/0 (socket says 172.21.15.172:42108) 2024-04-14T08:37:55.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.099+0000 7f689b59e640 1 -- 172.21.15.172:0/494328902 learned_addr learned my addr 172.21.15.172:0/494328902 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:55.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.099+0000 7f689ad9d640 1 --2- 172.21.15.172:0/494328902 >> v2:172.21.15.177:3300/0 conn(0x7f689c15e640 0x7f689c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.099+0000 7f689a59c640 1 -- 172.21.15.172:0/494328902 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 929687436 0 0) 0x7f689c0d1af0 con 0x7f689c06bc30 2024-04-14T08:37:55.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.099+0000 7f689a59c640 1 -- 172.21.15.172:0/494328902 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6888003520 con 0x7f689c06bc30 2024-04-14T08:37:55.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.099+0000 7f689ad9d640 1 -- 172.21.15.172:0/494328902 >> v1:172.21.15.177:6789/0 conn(0x7f689c0b0630 legacy=0x7f689c06b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.099+0000 7f689ad9d640 1 -- 172.21.15.172:0/494328902 >> v1:172.21.15.172:6789/0 conn(0x7f689c06bc30 legacy=0x7f689c162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.099+0000 7f689ad9d640 1 -- 172.21.15.172:0/494328902 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c0c7b20 con 0x7f689c15e640 2024-04-14T08:37:55.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.100+0000 7f689ad9d640 1 --2- 172.21.15.172:0/494328902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f689c15e640 0x7f689c0cee30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f688c0097a0 tx=0x7f688c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb965d308b2a854f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.100+0000 7f689a59c640 1 -- 172.21.15.172:0/494328902 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c035020 con 0x7f689c15e640 2024-04-14T08:37:55.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.100+0000 7f689a59c640 1 -- 172.21.15.172:0/494328902 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f688c031830 con 0x7f689c15e640 2024-04-14T08:37:55.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.100+0000 7f689a59c640 1 -- 172.21.15.172:0/494328902 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c031be0 con 0x7f689c15e640 2024-04-14T08:37:55.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.101+0000 7f68a122d640 1 -- 172.21.15.172:0/494328902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f689c15e640 msgr2=0x7f689c0cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.101+0000 7f68a122d640 1 --2- 172.21.15.172:0/494328902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f689c15e640 0x7f689c0cee30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f688c0097a0 tx=0x7f688c02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.101+0000 7f68a122d640 1 -- 172.21.15.172:0/494328902 shutdown_connections 2024-04-14T08:37:55.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.101+0000 7f68a122d640 1 --2- 172.21.15.172:0/494328902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f689c15e640 0x7f689c0cee30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.101+0000 7f68a122d640 1 -- 172.21.15.172:0/494328902 >> 172.21.15.172:0/494328902 conn(0x7f689c0d0f30 msgr2=0x7f689c0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.101+0000 7f68a122d640 1 -- 172.21.15.172:0/494328902 shutdown_connections 2024-04-14T08:37:55.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.101+0000 7f68a122d640 1 -- 172.21.15.172:0/494328902 wait complete. 2024-04-14T08:37:55.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.103+0000 7f68a122d640 1 Processor -- start 2024-04-14T08:37:55.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.103+0000 7f68a122d640 1 -- start start 2024-04-14T08:37:55.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.104+0000 7f68a122d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f689c06bc30 0x7f689c0cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.104+0000 7f68a122d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689c0b0630 0x7f689c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.104+0000 7f68a122d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f689c15e640 0x7f689c1f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.104+0000 7f68a122d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f689c06c1e0 con 0x7f689c0b0630 2024-04-14T08:37:55.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.104+0000 7f68a122d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f689c0d1dd0 con 0x7f689c06bc30 2024-04-14T08:37:55.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.104+0000 7f68a122d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f689c0c5db0 con 0x7f689c15e640 2024-04-14T08:37:55.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.104+0000 7f6893fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689c0b0630 0x7f689c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.104+0000 7f6893fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689c0b0630 0x7f689c0b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59800/0 (socket says 172.21.15.172:59800) 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.104+0000 7f6893fff640 1 -- 172.21.15.172:0/1918195154 learned_addr learned my addr 172.21.15.172:0/1918195154 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.104+0000 7f689ad9d640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f689c06bc30 0x7f689c0cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.104+0000 7f689b59e640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f689c15e640 0x7f689c1f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.105+0000 7f6893fff640 1 -- 172.21.15.172:0/1918195154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f689c15e640 msgr2=0x7f689c1f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.105+0000 7f6893fff640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f689c15e640 0x7f689c1f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.105+0000 7f6893fff640 1 -- 172.21.15.172:0/1918195154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f689c06bc30 msgr2=0x7f689c0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.105+0000 7f6893fff640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f689c06bc30 0x7f689c0cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.105+0000 7f6893fff640 1 -- 172.21.15.172:0/1918195154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c0b62b0 con 0x7f689c0b0630 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.105+0000 7f6893fff640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689c0b0630 0x7f689c0b5970 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f688400c400 tx=0x7f688400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.105+0000 7f68937fe640 1 -- 172.21.15.172:0/1918195154 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840052d0 con 0x7f689c0b0630 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.105+0000 7f68937fe640 1 -- 172.21.15.172:0/1918195154 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6884005490 con 0x7f689c0b0630 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.105+0000 7f68a122d640 1 -- 172.21.15.172:0/1918195154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c0ce1d0 con 0x7f689c0b0630 2024-04-14T08:37:55.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.105+0000 7f68937fe640 1 -- 172.21.15.172:0/1918195154 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884007260 con 0x7f689c0b0630 2024-04-14T08:37:55.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.105+0000 7f68a122d640 1 -- 172.21.15.172:0/1918195154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c0b3e80 con 0x7f689c0b0630 2024-04-14T08:37:55.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.106+0000 7f68a122d640 1 -- 172.21.15.172:0/1918195154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689c075490 con 0x7f689c0b0630 2024-04-14T08:37:55.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.107+0000 7f68937fe640 1 -- 172.21.15.172:0/1918195154 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6884007400 con 0x7f689c0b0630 2024-04-14T08:37:55.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.107+0000 7f68937fe640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f687c06c8f0 0x7f687c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.107+0000 7f689ad9d640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f687c06c8f0 0x7f687c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.107+0000 7f68937fe640 1 -- 172.21.15.172:0/1918195154 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f688408b210 con 0x7f689c0b0630 2024-04-14T08:37:55.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.108+0000 7f689ad9d640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f687c06c8f0 0x7f687c06edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f688c036000 tx=0x7f688c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.112+0000 7f68937fe640 1 -- 172.21.15.172:0/1918195154 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6884058e10 con 0x7f689c0b0630 2024-04-14T08:37:55.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.333+0000 7f68a122d640 1 -- 172.21.15.172:0/1918195154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f689c0b2350 con 0x7f689c0b0630 2024-04-14T08:37:55.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.333+0000 7f68937fe640 1 -- 172.21.15.172:0/1918195154 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f688405d110 con 0x7f689c0b0630 2024-04-14T08:37:55.334 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:37:55.334 INFO:teuthology.orchestra.run.smithi172.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-14T08:37:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.335+0000 7f68a122d640 1 -- 172.21.15.172:0/1918195154 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f687c06c8f0 msgr2=0x7f687c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.335+0000 7f68a122d640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f687c06c8f0 0x7f687c06edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f688c036000 tx=0x7f688c008030 comp rx=0 tx=0).stop 2024-04-14T08:37:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.335+0000 7f68a122d640 1 -- 172.21.15.172:0/1918195154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689c0b0630 msgr2=0x7f689c0b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.335+0000 7f68a122d640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689c0b0630 0x7f689c0b5970 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f688400c400 tx=0x7f688400c8c0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.335+0000 7f68a122d640 1 -- 172.21.15.172:0/1918195154 shutdown_connections 2024-04-14T08:37:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.335+0000 7f68a122d640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f687c06c8f0 0x7f687c06edb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.336+0000 7f68a122d640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f689c15e640 0x7f689c1f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.336+0000 7f68a122d640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689c0b0630 0x7f689c0b5970 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.336+0000 7f68a122d640 1 --2- 172.21.15.172:0/1918195154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f689c06bc30 0x7f689c0cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.336+0000 7f68a122d640 1 -- 172.21.15.172:0/1918195154 >> 172.21.15.172:0/1918195154 conn(0x7f689c0d0f30 msgr2=0x7f689c0d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.336+0000 7f68a122d640 1 -- 172.21.15.172:0/1918195154 shutdown_connections 2024-04-14T08:37:55.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.336+0000 7f68a122d640 1 -- 172.21.15.172:0/1918195154 wait complete. 2024-04-14T08:37:55.347 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-14T08:37:55.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:55.348 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-14T08:37:55.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.456+0000 7f3265a0f640 1 Processor -- start 2024-04-14T08:37:55.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.456+0000 7f3265a0f640 1 -- start start 2024-04-14T08:37:55.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.456+0000 7f3265a0f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3260161c00 0x7f32600d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.456+0000 7f3265a0f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32600d1c20 con 0x7f32600d2cf0 2024-04-14T08:37:55.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.456+0000 7f3265a0f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32600d10e0 con 0x7f32600b3ba0 2024-04-14T08:37:55.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.456+0000 7f3265a0f640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f32600cf1a0 con 0x7f3260161c00 2024-04-14T08:37:55.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.456+0000 7f325f7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f32600d2cf0 0x7f32600cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:42110/0 (socket says 172.21.15.172:42110) 2024-04-14T08:37:55.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.456+0000 7f325f7fe640 1 -- 172.21.15.172:0/125357072 learned_addr learned my addr 172.21.15.172:0/125357072 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:37:55.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.457+0000 7f325effd640 1 --2- 172.21.15.172:0/125357072 >> v2:172.21.15.177:3300/0 conn(0x7f3260161c00 0x7f32600d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.457+0000 7f325dffb640 1 -- 172.21.15.172:0/125357072 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2987237697 0 0) 0x7f32600d1c20 con 0x7f32600d2cf0 2024-04-14T08:37:55.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.457+0000 7f325dffb640 1 -- 172.21.15.172:0/125357072 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3248003520 con 0x7f32600d2cf0 2024-04-14T08:37:55.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.457+0000 7f325effd640 1 -- 172.21.15.172:0/125357072 >> v1:172.21.15.177:6789/0 conn(0x7f32600b3ba0 legacy=0x7f32600d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.457+0000 7f325effd640 1 -- 172.21.15.172:0/125357072 >> v1:172.21.15.172:6789/0 conn(0x7f32600d2cf0 legacy=0x7f32600cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.457+0000 7f325effd640 1 -- 172.21.15.172:0/125357072 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32600d1380 con 0x7f3260161c00 2024-04-14T08:37:55.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.458+0000 7f325effd640 1 --2- 172.21.15.172:0/125357072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3260161c00 0x7f32600d0ba0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f32580097a0 tx=0x7f325802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f5409ceef511ace server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.458+0000 7f325dffb640 1 -- 172.21.15.172:0/125357072 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258034070 con 0x7f3260161c00 2024-04-14T08:37:55.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.458+0000 7f325dffb640 1 -- 172.21.15.172:0/125357072 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3258030610 con 0x7f3260161c00 2024-04-14T08:37:55.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.458+0000 7f325dffb640 1 -- 172.21.15.172:0/125357072 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32580309e0 con 0x7f3260161c00 2024-04-14T08:37:55.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.459+0000 7f3265a0f640 1 -- 172.21.15.172:0/125357072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3260161c00 msgr2=0x7f32600d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.459+0000 7f3265a0f640 1 --2- 172.21.15.172:0/125357072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3260161c00 0x7f32600d0ba0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f32580097a0 tx=0x7f325802b960 comp rx=0 tx=0).stop 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.459+0000 7f3265a0f640 1 -- 172.21.15.172:0/125357072 shutdown_connections 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.459+0000 7f3265a0f640 1 --2- 172.21.15.172:0/125357072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3260161c00 0x7f32600d0ba0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.459+0000 7f3265a0f640 1 -- 172.21.15.172:0/125357072 >> 172.21.15.172:0/125357072 conn(0x7f326009d570 msgr2=0x7f32600d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.459+0000 7f3265a0f640 1 -- 172.21.15.172:0/125357072 shutdown_connections 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.459+0000 7f3265a0f640 1 -- 172.21.15.172:0/125357072 wait complete. 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.460+0000 7f3265a0f640 1 Processor -- start 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.460+0000 7f3265a0f640 1 -- start start 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f3265a0f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32600b3ba0 0x7f32600c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f3265a0f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32600d2cf0 0x7f32600b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f3265a0f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3260161c00 0x7f32600c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f3265a0f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3260161fe0 con 0x7f3260161c00 2024-04-14T08:37:55.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f3265a0f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f32600d1940 con 0x7f32600b3ba0 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f3265a0f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f32600b3110 con 0x7f32600d2cf0 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f325f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3260161c00 0x7f32600c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f325f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3260161c00 0x7f32600c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59816/0 (socket says 172.21.15.172:59816) 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f325f7fe640 1 -- 172.21.15.172:0/476596970 learned_addr learned my addr 172.21.15.172:0/476596970 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f325effd640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32600b3ba0 0x7f32600c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f325e7fc640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32600d2cf0 0x7f32600b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f325f7fe640 1 -- 172.21.15.172:0/476596970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32600d2cf0 msgr2=0x7f32600b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f325f7fe640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32600d2cf0 0x7f32600b6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f325f7fe640 1 -- 172.21.15.172:0/476596970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32600b3ba0 msgr2=0x7f32600c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f325f7fe640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32600b3ba0 0x7f32600c7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f325f7fe640 1 -- 172.21.15.172:0/476596970 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32600c8400 con 0x7f3260161c00 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f325e7fc640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32600d2cf0 0x7f32600b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f325f7fe640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3260161c00 0x7f32600c8920 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f325000cb10 tx=0x7f325000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f3247fff640 1 -- 172.21.15.172:0/476596970 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250017070 con 0x7f3260161c00 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f3247fff640 1 -- 172.21.15.172:0/476596970 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f325000ac70 con 0x7f3260161c00 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f3265a0f640 1 -- 172.21.15.172:0/476596970 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32600cbf30 con 0x7f3260161c00 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.461+0000 7f3247fff640 1 -- 172.21.15.172:0/476596970 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325000ae60 con 0x7f3260161c00 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.462+0000 7f3265a0f640 1 -- 172.21.15.172:0/476596970 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32600cc4f0 con 0x7f3260161c00 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.464+0000 7f3247fff640 1 -- 172.21.15.172:0/476596970 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f325001a070 con 0x7f3260161c00 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.464+0000 7f3247fff640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f324c06c8f0 0x7f324c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.464+0000 7f3247fff640 1 -- 172.21.15.172:0/476596970 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f325008ba40 con 0x7f3260161c00 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.464+0000 7f325effd640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f324c06c8f0 0x7f324c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.464+0000 7f32687fa740 1 Processor -- start 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.464+0000 7f32687fa740 1 -- start start 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.464+0000 7f325effd640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f324c06c8f0 0x7f324c06edb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3258036000 tx=0x7f325803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f32687fa740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 0x5580b96e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f32687fa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e420 0x5580b9710dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f32687fa740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b96e1520 0x5580b9711310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f32687fa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5580b95f1d10 con 0x5580b970e420 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f32687fa740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5580b964a650 con 0x5580b96e1520 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f32687fa740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5580b95f29c0 con 0x5580b970e040 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f325e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e420 0x5580b9710dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f325e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e420 0x5580b9710dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59820/0 (socket says 172.21.15.172:59820) 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f325e7fc640 1 -- 172.21.15.172:0/3117372102 learned_addr learned my addr 172.21.15.172:0/3117372102 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f325e7fc640 1 -- 172.21.15.172:0/3117372102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 msgr2=0x5580b96e0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f325e7fc640 1 --2- 172.21.15.172:0/3117372102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 0x5580b96e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f325e7fc640 1 -- 172.21.15.172:0/3117372102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b96e1520 msgr2=0x5580b9711310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f325f7fe640 1 --2- 172.21.15.172:0/3117372102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b96e1520 0x5580b9711310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f325e7fc640 1 --2- 172.21.15.172:0/3117372102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b96e1520 0x5580b9711310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f325e7fc640 1 -- 172.21.15.172:0/3117372102 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b9426a50 con 0x5580b970e420 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f325f7fe640 1 --2- 172.21.15.172:0/3117372102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b96e1520 0x5580b9711310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:55.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f325e7fc640 1 --2- 172.21.15.172:0/3117372102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e420 0x5580b9710dd0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3254001610 tx=0x7f3254001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77640864b337dee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f32457fa640 1 -- 172.21.15.172:0/3117372102 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254019070 con 0x5580b970e420 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f32457fa640 1 -- 172.21.15.172:0/3117372102 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32540050d0 con 0x5580b970e420 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.466+0000 7f32457fa640 1 -- 172.21.15.172:0/3117372102 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254005480 con 0x5580b970e420 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.467+0000 7f32687fa740 1 -- 172.21.15.172:0/3117372102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e420 msgr2=0x5580b9710dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.467+0000 7f32687fa740 1 --2- 172.21.15.172:0/3117372102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e420 0x5580b9710dd0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3254001610 tx=0x7f3254001ad0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.467+0000 7f32687fa740 1 -- 172.21.15.172:0/3117372102 shutdown_connections 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.467+0000 7f32687fa740 1 --2- 172.21.15.172:0/3117372102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b96e1520 0x5580b9711310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.467+0000 7f32687fa740 1 --2- 172.21.15.172:0/3117372102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e420 0x5580b9710dd0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.467+0000 7f32687fa740 1 --2- 172.21.15.172:0/3117372102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 0x5580b96e0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.467+0000 7f32687fa740 1 -- 172.21.15.172:0/3117372102 >> 172.21.15.172:0/3117372102 conn(0x5580b96e3250 msgr2=0x5580b972bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.467+0000 7f32687fa740 1 -- 172.21.15.172:0/3117372102 shutdown_connections 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.467+0000 7f32687fa740 1 -- 172.21.15.172:0/3117372102 wait complete. 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.468+0000 7f32687fa740 1 -- start start 2024-04-14T08:37:55.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.468+0000 7f32687fa740 1 Processor -- start 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f32687fa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e040 0x5580b970bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f32687fa740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b973ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f32687fa740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b9714580 0x5580b97149c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f32687fa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5580b9605d50 con 0x5580b970e040 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f32687fa740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5580b9646600 con 0x5580b970e420 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f32687fa740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5580b96e58f0 con 0x5580b9714580 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f32687fa740 10 client.0 _ll_register_callbacks cb 0x5580b960a540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f325effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e040 0x5580b970bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f325effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e040 0x5580b970bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59832/0 (socket says 172.21.15.172:59832) 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f325effd640 1 -- 172.21.15.172:0/4198423907 learned_addr learned my addr 172.21.15.172:0/4198423907 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f325e7fc640 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b973ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f325f7fe640 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b9714580 0x5580b97149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f325effd640 1 -- 172.21.15.172:0/4198423907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b9714580 msgr2=0x5580b97149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f325effd640 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b9714580 0x5580b97149c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f325effd640 1 -- 172.21.15.172:0/4198423907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 msgr2=0x5580b973ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f325effd640 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b973ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f325effd640 1 -- 172.21.15.172:0/4198423907 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b931e3e0 con 0x5580b970e040 2024-04-14T08:37:55.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f325effd640 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e040 0x5580b970bba0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3258039e70 tx=0x7f32580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae121e1b96a78631 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f323e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:55.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f323e7fc640 1 -- 172.21.15.172:0/4198423907 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258055060 con 0x5580b970e040 2024-04-14T08:37:55.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f323e7fc640 1 -- 172.21.15.172:0/4198423907 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f3258045d30 con 0x5580b970e040 2024-04-14T08:37:55.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.469+0000 7f32687fa740 1 -- 172.21.15.172:0/4198423907 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32540050d0 con 0x5580b970e040 2024-04-14T08:37:55.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f323e7fc640 1 -- 172.21.15.172:0/4198423907 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f325805b050 con 0x5580b970e040 2024-04-14T08:37:55.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f32687fa740 10 client.7179 fetch_fsmap learned FSMap version 46 2024-04-14T08:37:55.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f32687fa740 1 -- 172.21.15.172:0/4198423907 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x5580b94871e0 con 0x5580b970e040 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f323e7fc640 1 -- 172.21.15.172:0/4198423907 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32540050d0 con 0x5580b970e040 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f323e7fc640 1 -- 172.21.15.172:0/4198423907 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7f325805e020 con 0x5580b970e040 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f32687fa740 10 client.7179 fetch_fsmap finished waiting for FSMap version 46 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f32687fa740 10 client.7179 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f32687fa740 20 client.7179 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f32687fa740 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5580b9712ca0 0x5580b97130b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f32687fa740 4 client.7179 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f32687fa740 1 -- 172.21.15.172:0/4198423907 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580b9657b90 con 0x5580b9712ca0 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f325e7fc640 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5580b9712ca0 0x5580b97130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f323e7fc640 10 client.7179 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.470+0000 7f325e7fc640 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5580b9712ca0 0x5580b97130b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=542e521d896a31f9 server_cookie=413783d24283e77 in_seq=0 out_seq=0 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f323e7fc640 1 -- 172.21.15.172:0/4198423907 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580b9657b90 con 0x5580b9712ca0 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f323e7fc640 10 client.7179 handle_command_reply: tid=0 2024-04-14T08:37:55.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f32687fa740 1 client.7179 shutdown 2024-04-14T08:37:55.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f32687fa740 1 -- 172.21.15.172:0/4198423907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e040 msgr2=0x5580b970bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f32687fa740 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e040 0x5580b970bba0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3258039e70 tx=0x7f32580046c0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f32687fa740 1 -- 172.21.15.172:0/4198423907 shutdown_connections 2024-04-14T08:37:55.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f32687fa740 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5580b9712ca0 0x5580b97130b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f323e7fc640 0 client.7179 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:55.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f32687fa740 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b9714580 0x5580b97149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f32687fa740 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b973ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f32687fa740 1 --2- 172.21.15.172:0/4198423907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b970e040 0x5580b970bba0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f32687fa740 1 -- 172.21.15.172:0/4198423907 >> 172.21.15.172:0/4198423907 conn(0x5580b96e1520 msgr2=0x5580b96e4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f32687fa740 1 -- 172.21.15.172:0/4198423907 shutdown_connections 2024-04-14T08:37:55.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f32687fa740 1 -- 172.21.15.172:0/4198423907 wait complete. 2024-04-14T08:37:55.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.471+0000 7f32687fa740 20 client.7179 trim_cache size 0 max 16384 2024-04-14T08:37:55.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.489+0000 7f32687fa740 1 Processor -- start 2024-04-14T08:37:55.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.489+0000 7f32687fa740 1 -- start start 2024-04-14T08:37:55.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.489+0000 7f32687fa740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 0x5580b9718950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.489+0000 7f32687fa740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b970c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.489+0000 7f32687fa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 0x5580b970ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.489+0000 7f32687fa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3260201050 con 0x5580b9712ca0 2024-04-14T08:37:55.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.489+0000 7f32687fa740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f32602000b0 con 0x5580b970e420 2024-04-14T08:37:55.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f32687fa740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f32601ff440 con 0x5580b970e040 2024-04-14T08:37:55.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f325f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 0x5580b970ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f325f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 0x5580b970ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59842/0 (socket says 172.21.15.172:59842) 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f325f7fe640 1 -- 172.21.15.172:0/4016610614 learned_addr learned my addr 172.21.15.172:0/4016610614 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f325e7fc640 1 --2- 172.21.15.172:0/4016610614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b970c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f325effd640 1 --2- 172.21.15.172:0/4016610614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 0x5580b9718950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f325f7fe640 1 -- 172.21.15.172:0/4016610614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 msgr2=0x5580b9718950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f325f7fe640 1 --2- 172.21.15.172:0/4016610614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 0x5580b9718950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f325f7fe640 1 -- 172.21.15.172:0/4016610614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 msgr2=0x5580b970c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f325f7fe640 1 --2- 172.21.15.172:0/4016610614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b970c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f325f7fe640 1 -- 172.21.15.172:0/4016610614 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b9714d30 con 0x5580b9712ca0 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f325e7fc640 1 --2- 172.21.15.172:0/4016610614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b970c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f325f7fe640 1 --2- 172.21.15.172:0/4016610614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 0x5580b970ca10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3250059340 tx=0x7f32500014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4451ebcfab96654 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f3244ff9640 1 -- 172.21.15.172:0/4016610614 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250088070 con 0x5580b9712ca0 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f3244ff9640 1 -- 172.21.15.172:0/4016610614 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3250061ce0 con 0x5580b9712ca0 2024-04-14T08:37:55.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.490+0000 7f3244ff9640 1 -- 172.21.15.172:0/4016610614 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250071070 con 0x5580b9712ca0 2024-04-14T08:37:55.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.491+0000 7f32687fa740 1 -- 172.21.15.172:0/4016610614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 msgr2=0x5580b970ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.491+0000 7f32687fa740 1 --2- 172.21.15.172:0/4016610614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 0x5580b970ca10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3250059340 tx=0x7f32500014d0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.491+0000 7f32687fa740 1 -- 172.21.15.172:0/4016610614 shutdown_connections 2024-04-14T08:37:55.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.491+0000 7f32687fa740 1 --2- 172.21.15.172:0/4016610614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 0x5580b970ca10 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.491+0000 7f32687fa740 1 --2- 172.21.15.172:0/4016610614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b970c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.491+0000 7f32687fa740 1 --2- 172.21.15.172:0/4016610614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 0x5580b9718950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.491+0000 7f32687fa740 1 -- 172.21.15.172:0/4016610614 >> 172.21.15.172:0/4016610614 conn(0x5580b96e1520 msgr2=0x5580b96dc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.491+0000 7f32687fa740 1 -- 172.21.15.172:0/4016610614 shutdown_connections 2024-04-14T08:37:55.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.491+0000 7f32687fa740 1 -- 172.21.15.172:0/4016610614 wait complete. 2024-04-14T08:37:55.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.492+0000 7f32687fa740 1 -- start start 2024-04-14T08:37:55.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.492+0000 7f32687fa740 1 Processor -- start 2024-04-14T08:37:55.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f32687fa740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 0x5580b973dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f32687fa740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b97a85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f32687fa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 0x5580b97a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f32687fa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32601ffa90 con 0x5580b9712ca0 2024-04-14T08:37:55.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f32687fa740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5580b972d8d0 con 0x5580b970e420 2024-04-14T08:37:55.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f32687fa740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5580b973afd0 con 0x5580b970e040 2024-04-14T08:37:55.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f32687fa740 10 client.0 _ll_register_callbacks cb 0x7f3258009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:55.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f325f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 0x5580b97a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f325f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 0x5580b97a8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59844/0 (socket says 172.21.15.172:59844) 2024-04-14T08:37:55.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f325f7fe640 1 -- 172.21.15.172:0/2220015088 learned_addr learned my addr 172.21.15.172:0/2220015088 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:55.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f325effd640 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 0x5580b973dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f325e7fc640 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b97a85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.493+0000 7f325f7fe640 1 -- 172.21.15.172:0/2220015088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 msgr2=0x5580b973dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f325f7fe640 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 0x5580b973dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f325f7fe640 1 -- 172.21.15.172:0/2220015088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 msgr2=0x5580b97a85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f325f7fe640 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b97a85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f325f7fe640 1 -- 172.21.15.172:0/2220015088 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b970bf10 con 0x5580b9712ca0 2024-04-14T08:37:55.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f325effd640 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 0x5580b973dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:55.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f325e7fc640 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b97a85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:55.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f325f7fe640 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 0x5580b97a8ae0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3250059580 tx=0x7f3250001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263cdc6eb264edf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f323e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:55.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f323e7fc640 1 -- 172.21.15.172:0/2220015088 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250076060 con 0x5580b9712ca0 2024-04-14T08:37:55.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f323e7fc640 1 -- 172.21.15.172:0/2220015088 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f3250058a90 con 0x5580b9712ca0 2024-04-14T08:37:55.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f32687fa740 1 -- 172.21.15.172:0/2220015088 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3250061ce0 con 0x5580b9712ca0 2024-04-14T08:37:55.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f323e7fc640 1 -- 172.21.15.172:0/2220015088 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3250088070 con 0x5580b9712ca0 2024-04-14T08:37:55.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f32687fa740 10 client.7185 fetch_fsmap learned FSMap version 46 2024-04-14T08:37:55.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f32687fa740 1 -- 172.21.15.172:0/2220015088 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x5580b92f74a0 con 0x5580b9712ca0 2024-04-14T08:37:55.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f323e7fc640 1 -- 172.21.15.172:0/2220015088 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3250061ce0 con 0x5580b9712ca0 2024-04-14T08:37:55.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f323e7fc640 1 -- 172.21.15.172:0/2220015088 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7f3250055390 con 0x5580b9712ca0 2024-04-14T08:37:55.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f32687fa740 10 client.7185 fetch_fsmap finished waiting for FSMap version 46 2024-04-14T08:37:55.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f32687fa740 10 client.7185 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-14T08:37:55.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f32687fa740 20 client.7185 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:55.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f32687fa740 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5580b9714580 0x5580b970a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f32687fa740 4 client.7185 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-14T08:37:55.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f32687fa740 1 -- 172.21.15.172:0/2220015088 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x5580b9651ee0 con 0x5580b9714580 2024-04-14T08:37:55.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.494+0000 7f325effd640 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5580b9714580 0x5580b970a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f323e7fc640 10 client.7185 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:55.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f325effd640 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5580b9714580 0x5580b970a440 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7808cf8134f153a server_cookie=905531a1d1089e6c in_seq=0 out_seq=0 2024-04-14T08:37:55.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f323e7fc640 1 -- 172.21.15.172:0/2220015088 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5580b9651ee0 con 0x5580b9714580 2024-04-14T08:37:55.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f323e7fc640 10 client.7185 handle_command_reply: tid=0 2024-04-14T08:37:55.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f32687fa740 1 client.7185 shutdown 2024-04-14T08:37:55.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f32687fa740 1 -- 172.21.15.172:0/2220015088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 msgr2=0x5580b97a8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f32687fa740 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 0x5580b97a8ae0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3250059580 tx=0x7f3250001250 comp rx=0 tx=0).stop 2024-04-14T08:37:55.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f32687fa740 1 -- 172.21.15.172:0/2220015088 shutdown_connections 2024-04-14T08:37:55.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f32687fa740 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5580b9714580 0x5580b970a440 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f32687fa740 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580b9712ca0 0x5580b97a8ae0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f32687fa740 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5580b970e420 0x5580b97a85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f32687fa740 1 --2- 172.21.15.172:0/2220015088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5580b970e040 0x5580b973dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f32687fa740 1 -- 172.21.15.172:0/2220015088 >> 172.21.15.172:0/2220015088 conn(0x5580b96e1520 msgr2=0x5580b96e45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.495+0000 7f323e7fc640 0 client.7185 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:55.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f32687fa740 1 -- 172.21.15.172:0/2220015088 shutdown_connections 2024-04-14T08:37:55.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f32687fa740 1 -- 172.21.15.172:0/2220015088 wait complete. 2024-04-14T08:37:55.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f32687fa740 20 client.7185 trim_cache size 0 max 16384 2024-04-14T08:37:55.503 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:37:55.504 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-04-14T08:37:55.504 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "52d99e10-b424-493a-9dfb-310838b39bc9", 2024-04-14T08:37:55.504 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-04-14T08:37:55.504 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:37:55.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f3265a0f640 1 -- 172.21.15.172:0/476596970 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f324c06c8f0 msgr2=0x7f324c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f3265a0f640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f324c06c8f0 0x7f324c06edb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3258036000 tx=0x7f325803b040 comp rx=0 tx=0).stop 2024-04-14T08:37:55.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f3265a0f640 1 -- 172.21.15.172:0/476596970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3260161c00 msgr2=0x7f32600c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f3265a0f640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3260161c00 0x7f32600c8920 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f325000cb10 tx=0x7f325000a060 comp rx=0 tx=0).stop 2024-04-14T08:37:55.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f3265a0f640 1 -- 172.21.15.172:0/476596970 shutdown_connections 2024-04-14T08:37:55.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f3265a0f640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f324c06c8f0 0x7f324c06edb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f3265a0f640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3260161c00 0x7f32600c8920 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f3265a0f640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32600d2cf0 0x7f32600b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f3265a0f640 1 --2- 172.21.15.172:0/476596970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32600b3ba0 0x7f32600c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f3265a0f640 1 -- 172.21.15.172:0/476596970 >> 172.21.15.172:0/476596970 conn(0x7f326009d570 msgr2=0x7f32600b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f3265a0f640 1 -- 172.21.15.172:0/476596970 shutdown_connections 2024-04-14T08:37:55.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.496+0000 7f3265a0f640 1 -- 172.21.15.172:0/476596970 wait complete. 2024-04-14T08:37:55.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:37:55.508 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-14T08:37:55.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.611+0000 7ff974420640 1 Processor -- start 2024-04-14T08:37:55.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.611+0000 7ff974420640 1 -- start start 2024-04-14T08:37:55.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff974420640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c15e430 0x7ff96c162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff974420640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff96c0d2e00 0x7ff96c1629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff974420640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff96c0cefb0 con 0x7ff96c162540 2024-04-14T08:37:55.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff974420640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff96c0cf290 con 0x7ff96c0d2e00 2024-04-14T08:37:55.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff974420640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c0cf570 con 0x7ff96c15e430 2024-04-14T08:37:55.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff9729bd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff96c0d2e00 0x7ff96c1629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff9729bd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff96c0d2e00 0x7ff96c1629a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59850/0 (socket says 172.21.15.172:59850) 2024-04-14T08:37:55.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff9719bb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff96c162540 0x7ff96c0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:42120/0 (socket says 172.21.15.172:42120) 2024-04-14T08:37:55.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff9729bd640 1 -- 172.21.15.172:0/57561947 learned_addr learned my addr 172.21.15.172:0/57561947 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:55.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff9721bc640 1 --2- 172.21.15.172:0/57561947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c15e430 0x7ff96c162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff9729bd640 1 -- 172.21.15.172:0/57561947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c15e430 msgr2=0x7ff96c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff9729bd640 1 --2- 172.21.15.172:0/57561947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c15e430 0x7ff96c162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff9729bd640 1 -- 172.21.15.172:0/57561947 >> v1:172.21.15.172:6789/0 conn(0x7ff96c162540 legacy=0x7ff96c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff9729bd640 1 -- 172.21.15.172:0/57561947 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c0cf850 con 0x7ff96c0d2e00 2024-04-14T08:37:55.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.612+0000 7ff9711ba640 1 -- 172.21.15.172:0/57561947 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2413536902 0 0) 0x7ff96c0cefb0 con 0x7ff96c162540 2024-04-14T08:37:55.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.613+0000 7ff9729bd640 1 --2- 172.21.15.172:0/57561947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff96c0d2e00 0x7ff96c1629a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff964009e50 tx=0x7ff96402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddb8ed5bb909534 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.613+0000 7ff9711ba640 1 -- 172.21.15.172:0/57561947 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96402cb00 con 0x7ff96c0d2e00 2024-04-14T08:37:55.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.613+0000 7ff9711ba640 1 -- 172.21.15.172:0/57561947 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96402cca0 con 0x7ff96c0d2e00 2024-04-14T08:37:55.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.613+0000 7ff9711ba640 1 -- 172.21.15.172:0/57561947 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964034610 con 0x7ff96c0d2e00 2024-04-14T08:37:55.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.614+0000 7ff974420640 1 -- 172.21.15.172:0/57561947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff96c0d2e00 msgr2=0x7ff96c1629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.614+0000 7ff974420640 1 --2- 172.21.15.172:0/57561947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff96c0d2e00 0x7ff96c1629a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff964009e50 tx=0x7ff96402bf60 comp rx=0 tx=0).stop 2024-04-14T08:37:55.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.614+0000 7ff974420640 1 -- 172.21.15.172:0/57561947 shutdown_connections 2024-04-14T08:37:55.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.614+0000 7ff974420640 1 --2- 172.21.15.172:0/57561947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff96c0d2e00 0x7ff96c1629a0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.614+0000 7ff974420640 1 --2- 172.21.15.172:0/57561947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c15e430 0x7ff96c162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.614+0000 7ff974420640 1 -- 172.21.15.172:0/57561947 >> 172.21.15.172:0/57561947 conn(0x7ff96c09d560 msgr2=0x7ff96c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.614+0000 7ff974420640 1 -- 172.21.15.172:0/57561947 shutdown_connections 2024-04-14T08:37:55.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.614+0000 7ff974420640 1 -- 172.21.15.172:0/57561947 wait complete. 2024-04-14T08:37:55.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.615+0000 7ff974420640 1 Processor -- start 2024-04-14T08:37:55.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.615+0000 7ff974420640 1 -- start start 2024-04-14T08:37:55.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff974420640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c0d2e00 0x7ff96c0b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff974420640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff96c15e430 0x7ff96c0cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff974420640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff96c162540 0x7ff96c0cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff974420640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c0d4e20 con 0x7ff96c162540 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff974420640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c15e810 con 0x7ff96c15e430 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff974420640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c0d0050 con 0x7ff96c0d2e00 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff9729bd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff96c162540 0x7ff96c0cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff9729bd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff96c162540 0x7ff96c0cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59858/0 (socket says 172.21.15.172:59858) 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff9729bd640 1 -- 172.21.15.172:0/3717469898 learned_addr learned my addr 172.21.15.172:0/3717469898 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff9721bc640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c0d2e00 0x7ff96c0b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff9719bb640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff96c15e430 0x7ff96c0cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff9729bd640 1 -- 172.21.15.172:0/3717469898 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c0d2e00 msgr2=0x7ff96c0b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff9729bd640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c0d2e00 0x7ff96c0b8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff9729bd640 1 -- 172.21.15.172:0/3717469898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff96c15e430 msgr2=0x7ff96c0cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff9729bd640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff96c15e430 0x7ff96c0cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff9729bd640 1 -- 172.21.15.172:0/3717469898 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c0cc960 con 0x7ff96c162540 2024-04-14T08:37:55.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff9721bc640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c0d2e00 0x7ff96c0b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:55.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.616+0000 7ff9719bb640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff96c15e430 0x7ff96c0cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:55.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.617+0000 7ff9729bd640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff96c162540 0x7ff96c0cc420 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7ff964009e50 tx=0x7ff964004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.617+0000 7ff962ffd640 1 -- 172.21.15.172:0/3717469898 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964043030 con 0x7ff96c162540 2024-04-14T08:37:55.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.617+0000 7ff962ffd640 1 -- 172.21.15.172:0/3717469898 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96402ce50 con 0x7ff96c162540 2024-04-14T08:37:55.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.617+0000 7ff974420640 1 -- 172.21.15.172:0/3717469898 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c0ccbe0 con 0x7ff96c162540 2024-04-14T08:37:55.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.617+0000 7ff962ffd640 1 -- 172.21.15.172:0/3717469898 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964009510 con 0x7ff96c162540 2024-04-14T08:37:55.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.617+0000 7ff974420640 1 -- 172.21.15.172:0/3717469898 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c0cd1a0 con 0x7ff96c162540 2024-04-14T08:37:55.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.618+0000 7ff962ffd640 1 -- 172.21.15.172:0/3717469898 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff964046070 con 0x7ff96c162540 2024-04-14T08:37:55.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.619+0000 7ff962ffd640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff95006c920 0x7ff95006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.619+0000 7ff962ffd640 1 -- 172.21.15.172:0/3717469898 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7ff9640adf70 con 0x7ff96c162540 2024-04-14T08:37:55.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.619+0000 7ff9721bc640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff95006c920 0x7ff95006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.619+0000 7ff9721bc640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff95006c920 0x7ff95006ede0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff95c006fd0 tx=0x7ff95c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.621+0000 7ff97720a740 1 Processor -- start 2024-04-14T08:37:55.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.621+0000 7ff97720a740 1 -- start start 2024-04-14T08:37:55.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.622+0000 7ff97720a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c5e0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.622+0000 7ff97720a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c60dd50 0x56248c610700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.622+0000 7ff97720a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c5e0e50 0x56248c610c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.622+0000 7ff97720a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56248c54a800 con 0x56248c60d970 2024-04-14T08:37:55.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.622+0000 7ff97720a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56248c4f27f0 con 0x56248c60dd50 2024-04-14T08:37:55.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.622+0000 7ff97720a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56248c5467b0 con 0x56248c5e0e50 2024-04-14T08:37:55.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.622+0000 7ff9721bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c5e0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff9721bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c5e0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59860/0 (socket says 172.21.15.172:59860) 2024-04-14T08:37:55.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff9721bc640 1 -- 172.21.15.172:0/2194857044 learned_addr learned my addr 172.21.15.172:0/2194857044 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff9729bd640 1 --2- 172.21.15.172:0/2194857044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c5e0e50 0x56248c610c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff9719bb640 1 --2- 172.21.15.172:0/2194857044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c60dd50 0x56248c610700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff9721bc640 1 -- 172.21.15.172:0/2194857044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c5e0e50 msgr2=0x56248c610c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff9721bc640 1 --2- 172.21.15.172:0/2194857044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c5e0e50 0x56248c610c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff9721bc640 1 -- 172.21.15.172:0/2194857044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c60dd50 msgr2=0x56248c610700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff9721bc640 1 --2- 172.21.15.172:0/2194857044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c60dd50 0x56248c610700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff9721bc640 1 -- 172.21.15.172:0/2194857044 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56248c3287d0 con 0x56248c60d970 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff9721bc640 1 --2- 172.21.15.172:0/2194857044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c5e0910 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e8f0 tx=0x7ff95c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4ba13c6b5f5329 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff94ffff640 1 -- 172.21.15.172:0/2194857044 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c025070 con 0x56248c60d970 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff94ffff640 1 -- 172.21.15.172:0/2194857044 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c015640 con 0x56248c60d970 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.623+0000 7ff94ffff640 1 -- 172.21.15.172:0/2194857044 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00f040 con 0x56248c60d970 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.624+0000 7ff97720a740 1 -- 172.21.15.172:0/2194857044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 msgr2=0x56248c5e0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.624+0000 7ff97720a740 1 --2- 172.21.15.172:0/2194857044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c5e0910 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e8f0 tx=0x7ff95c011000 comp rx=0 tx=0).stop 2024-04-14T08:37:55.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.624+0000 7ff97720a740 1 -- 172.21.15.172:0/2194857044 shutdown_connections 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.624+0000 7ff97720a740 1 --2- 172.21.15.172:0/2194857044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c5e0e50 0x56248c610c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.624+0000 7ff97720a740 1 --2- 172.21.15.172:0/2194857044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c60dd50 0x56248c610700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.624+0000 7ff97720a740 1 --2- 172.21.15.172:0/2194857044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c5e0910 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.624+0000 7ff97720a740 1 -- 172.21.15.172:0/2194857044 >> 172.21.15.172:0/2194857044 conn(0x56248c5e2b80 msgr2=0x56248c62b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.624+0000 7ff97720a740 1 -- 172.21.15.172:0/2194857044 shutdown_connections 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.624+0000 7ff97720a740 1 -- 172.21.15.172:0/2194857044 wait complete. 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.625+0000 7ff97720a740 1 -- start start 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.625+0000 7ff97720a740 1 Processor -- start 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff97720a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c625550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff97720a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60dd50 0x56248c612740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff97720a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c601f40 0x56248c612c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff97720a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56248c5e5120 con 0x56248c60d970 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff97720a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56248c505d00 con 0x56248c601f40 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff97720a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56248c5e4510 con 0x56248c60dd50 2024-04-14T08:37:55.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff97720a740 10 client.0 _ll_register_callbacks cb 0x56248c5056f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:55.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff9721bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c625550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff9721bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c625550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59876/0 (socket says 172.21.15.172:59876) 2024-04-14T08:37:55.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff9721bc640 1 -- 172.21.15.172:0/2845810353 learned_addr learned my addr 172.21.15.172:0/2845810353 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:55.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff9719bb640 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60dd50 0x56248c612740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff9729bd640 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c601f40 0x56248c612c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff9721bc640 1 -- 172.21.15.172:0/2845810353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60dd50 msgr2=0x56248c612740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff9721bc640 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60dd50 0x56248c612740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff9721bc640 1 -- 172.21.15.172:0/2845810353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c601f40 msgr2=0x56248c612c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff9721bc640 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c601f40 0x56248c612c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff9721bc640 1 -- 172.21.15.172:0/2845810353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56248c25e3f0 con 0x56248c60d970 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff9721bc640 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c625550 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009160 tx=0x7ff95c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19412aa0a5a71119 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff94d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff94d7fa640 1 -- 172.21.15.172:0/2845810353 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c025070 con 0x56248c60d970 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff94d7fa640 1 -- 172.21.15.172:0/2845810353 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7ff95c015a00 con 0x56248c60d970 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.626+0000 7ff97720a740 1 -- 172.21.15.172:0/2845810353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff95c015640 con 0x56248c60d970 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff94d7fa640 1 -- 172.21.15.172:0/2845810353 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff95c015c00 con 0x56248c60d970 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff97720a740 10 client.7200 fetch_fsmap learned FSMap version 46 2024-04-14T08:37:55.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff97720a740 1 -- 172.21.15.172:0/2845810353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x56248c21e0f0 con 0x56248c60d970 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff94d7fa640 1 -- 172.21.15.172:0/2845810353 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c015640 con 0x56248c60d970 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff94d7fa640 1 -- 172.21.15.172:0/2845810353 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7ff95c03a080 con 0x56248c60d970 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff97720a740 10 client.7200 fetch_fsmap finished waiting for FSMap version 46 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff97720a740 10 client.7200 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff97720a740 20 client.7200 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff97720a740 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56248c624160 0x56248c6245a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff97720a740 4 client.7200 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff97720a740 1 -- 172.21.15.172:0/2845810353 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56248c557d40 con 0x56248c624160 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff9719bb640 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56248c624160 0x56248c6245a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff9719bb640 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56248c624160 0x56248c6245a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b180dcd7c04521b4 server_cookie=4fefc937cf492ac8 in_seq=0 out_seq=0 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.627+0000 7ff94d7fa640 10 client.7200 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.628+0000 7ff94d7fa640 1 -- 172.21.15.172:0/2845810353 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56248c557d40 con 0x56248c624160 2024-04-14T08:37:55.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.628+0000 7ff94d7fa640 10 client.7200 handle_command_reply: tid=0 2024-04-14T08:37:55.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.628+0000 7ff97720a740 1 client.7200 shutdown 2024-04-14T08:37:55.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.628+0000 7ff97720a740 1 -- 172.21.15.172:0/2845810353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 msgr2=0x56248c625550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.628+0000 7ff97720a740 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c625550 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009160 tx=0x7ff95c015000 comp rx=0 tx=0).stop 2024-04-14T08:37:55.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.628+0000 7ff97720a740 1 -- 172.21.15.172:0/2845810353 shutdown_connections 2024-04-14T08:37:55.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.628+0000 7ff97720a740 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56248c624160 0x56248c6245a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.628+0000 7ff97720a740 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c601f40 0x56248c612c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.628+0000 7ff97720a740 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60dd50 0x56248c612740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.628+0000 7ff97720a740 1 --2- 172.21.15.172:0/2845810353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c625550 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.628+0000 7ff94d7fa640 0 client.7200 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:55.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.628+0000 7ff97720a740 1 -- 172.21.15.172:0/2845810353 >> 172.21.15.172:0/2845810353 conn(0x56248c5e0e50 msgr2=0x56248c6106f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.629+0000 7ff97720a740 1 -- 172.21.15.172:0/2845810353 shutdown_connections 2024-04-14T08:37:55.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.629+0000 7ff97720a740 1 -- 172.21.15.172:0/2845810353 wait complete. 2024-04-14T08:37:55.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.629+0000 7ff97720a740 20 client.7200 trim_cache size 0 max 16384 2024-04-14T08:37:55.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.646+0000 7ff97720a740 1 Processor -- start 2024-04-14T08:37:55.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.646+0000 7ff97720a740 1 -- start start 2024-04-14T08:37:55.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff97720a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c601f40 0x56248c626700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff97720a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60d970 0x56248c63acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff97720a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c60dd50 0x56248c64e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff97720a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c201190 con 0x56248c601f40 2024-04-14T08:37:55.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff97720a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c200190 con 0x56248c60dd50 2024-04-14T08:37:55.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff97720a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c0b4470 con 0x56248c60d970 2024-04-14T08:37:55.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9721bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c601f40 0x56248c626700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9721bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c601f40 0x56248c626700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59888/0 (socket says 172.21.15.172:59888) 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9721bc640 1 -- 172.21.15.172:0/848876643 learned_addr learned my addr 172.21.15.172:0/848876643 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9719bb640 1 --2- 172.21.15.172:0/848876643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60d970 0x56248c63acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9721bc640 1 -- 172.21.15.172:0/848876643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60d970 msgr2=0x56248c63acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9729bd640 1 --2- 172.21.15.172:0/848876643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c60dd50 0x56248c64e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9721bc640 1 --2- 172.21.15.172:0/848876643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60d970 0x56248c63acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9721bc640 1 -- 172.21.15.172:0/848876643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c60dd50 msgr2=0x56248c64e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9721bc640 1 --2- 172.21.15.172:0/848876643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c60dd50 0x56248c64e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9721bc640 1 -- 172.21.15.172:0/848876643 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56248c5dcc20 con 0x56248c601f40 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9719bb640 1 --2- 172.21.15.172:0/848876643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60d970 0x56248c63acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9729bd640 1 --2- 172.21.15.172:0/848876643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c60dd50 0x56248c64e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff9721bc640 1 --2- 172.21.15.172:0/848876643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c601f40 0x56248c626700 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e840 tx=0x7ff95c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288770b685d4a520 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff94f7fe640 1 -- 172.21.15.172:0/848876643 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c025040 con 0x56248c601f40 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff94f7fe640 1 -- 172.21.15.172:0/848876643 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c02f070 con 0x56248c601f40 2024-04-14T08:37:55.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.647+0000 7ff94f7fe640 1 -- 172.21.15.172:0/848876643 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c015640 con 0x56248c601f40 2024-04-14T08:37:55.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.648+0000 7ff97720a740 1 -- 172.21.15.172:0/848876643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c601f40 msgr2=0x56248c626700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.648+0000 7ff97720a740 1 --2- 172.21.15.172:0/848876643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c601f40 0x56248c626700 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e840 tx=0x7ff95c01e000 comp rx=0 tx=0).stop 2024-04-14T08:37:55.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.648+0000 7ff97720a740 1 -- 172.21.15.172:0/848876643 shutdown_connections 2024-04-14T08:37:55.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.648+0000 7ff97720a740 1 --2- 172.21.15.172:0/848876643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c60dd50 0x56248c64e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.648+0000 7ff97720a740 1 --2- 172.21.15.172:0/848876643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60d970 0x56248c63acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.648+0000 7ff97720a740 1 --2- 172.21.15.172:0/848876643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c601f40 0x56248c626700 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.648+0000 7ff97720a740 1 -- 172.21.15.172:0/848876643 >> 172.21.15.172:0/848876643 conn(0x56248c5e0e50 msgr2=0x56248c5dc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.648+0000 7ff97720a740 1 -- 172.21.15.172:0/848876643 shutdown_connections 2024-04-14T08:37:55.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.648+0000 7ff97720a740 1 -- 172.21.15.172:0/848876643 wait complete. 2024-04-14T08:37:55.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.648+0000 7ff97720a740 1 -- start start 2024-04-14T08:37:55.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.648+0000 7ff97720a740 1 Processor -- start 2024-04-14T08:37:55.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.649+0000 7ff97720a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c601f40 0x56248c5fedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.649+0000 7ff97720a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c6a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.649+0000 7ff97720a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60dd50 0x56248c6127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.649+0000 7ff97720a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1ffb40 con 0x56248c60d970 2024-04-14T08:37:55.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.649+0000 7ff97720a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56248c62e340 con 0x56248c601f40 2024-04-14T08:37:55.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.649+0000 7ff97720a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56248c6a4980 con 0x56248c60dd50 2024-04-14T08:37:55.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.649+0000 7ff97720a740 10 client.0 _ll_register_callbacks cb 0x7ff95c004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:37:55.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.649+0000 7ff9719bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c6a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.649+0000 7ff9719bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c6a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59894/0 (socket says 172.21.15.172:59894) 2024-04-14T08:37:55.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.649+0000 7ff9719bb640 1 -- 172.21.15.172:0/3874535671 learned_addr learned my addr 172.21.15.172:0/3874535671 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:37:55.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.649+0000 7ff9729bd640 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60dd50 0x56248c6127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.649+0000 7ff9721bc640 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c601f40 0x56248c5fedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff9719bb640 1 -- 172.21.15.172:0/3874535671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60dd50 msgr2=0x56248c6127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff9719bb640 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60dd50 0x56248c6127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff9719bb640 1 -- 172.21.15.172:0/3874535671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c601f40 msgr2=0x56248c5fedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:37:55.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff9719bb640 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c601f40 0x56248c5fedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff9719bb640 1 -- 172.21.15.172:0/3874535671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56248c6a2340 con 0x56248c60d970 2024-04-14T08:37:55.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff9719bb640 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c6a1e00 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff96c1f76b0 tx=0x7ff968005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc5959d4cadebba server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:37:55.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff94d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:37:55.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff94d7fa640 1 -- 172.21.15.172:0/3874535671 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968015940 con 0x56248c60d970 2024-04-14T08:37:55.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff94d7fa640 1 -- 172.21.15.172:0/3874535671 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7ff968015ae0 con 0x56248c60d970 2024-04-14T08:37:55.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff97720a740 1 -- 172.21.15.172:0/3874535671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff95c02f070 con 0x56248c60d970 2024-04-14T08:37:55.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff94d7fa640 1 -- 172.21.15.172:0/3874535671 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff96802a050 con 0x56248c60d970 2024-04-14T08:37:55.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff97720a740 10 client.7206 fetch_fsmap learned FSMap version 46 2024-04-14T08:37:55.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff97720a740 1 -- 172.21.15.172:0/3874535671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x56248c6a4b20 con 0x56248c60d970 2024-04-14T08:37:55.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff94d7fa640 1 -- 172.21.15.172:0/3874535671 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c02f070 con 0x56248c60d970 2024-04-14T08:37:55.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff94d7fa640 1 -- 172.21.15.172:0/3874535671 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7ff96802d020 con 0x56248c60d970 2024-04-14T08:37:55.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff97720a740 10 client.7206 fetch_fsmap finished waiting for FSMap version 46 2024-04-14T08:37:55.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff97720a740 10 client.7206 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-14T08:37:55.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff97720a740 20 client.7206 populate_metadata read hostname 'smithi172' 2024-04-14T08:37:55.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.650+0000 7ff97720a740 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56248c612cf0 0x56248c6106f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:37:55.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 4 client.7206 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:37:55.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 1 -- 172.21.15.172:0/3874535671 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56248c552090 con 0x56248c612cf0 2024-04-14T08:37:55.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff9721bc640 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56248c612cf0 0x56248c6106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:37:55.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff94d7fa640 10 client.7206 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:55.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff9721bc640 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56248c612cf0 0x56248c6106f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef91e9faeb084bd server_cookie=d74d14b92de65775 in_seq=0 out_seq=0 2024-04-14T08:37:55.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff94d7fa640 1 -- 172.21.15.172:0/3874535671 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x56248c552090 con 0x56248c612cf0 2024-04-14T08:37:55.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff94d7fa640 10 client.7206 handle_command_reply: tid=0 2024-04-14T08:37:55.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 1 client.7206 shutdown 2024-04-14T08:37:55.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 1 -- 172.21.15.172:0/3874535671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 msgr2=0x56248c6a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c6a1e00 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff96c1f76b0 tx=0x7ff968005c40 comp rx=0 tx=0).stop 2024-04-14T08:37:55.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 1 -- 172.21.15.172:0/3874535671 shutdown_connections 2024-04-14T08:37:55.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x56248c612cf0 0x56248c6106f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56248c60dd50 0x56248c6127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56248c60d970 0x56248c6a1e00 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 1 --2- 172.21.15.172:0/3874535671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56248c601f40 0x56248c5fedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff94d7fa640 0 client.7206 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:37:55.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 1 -- 172.21.15.172:0/3874535671 >> 172.21.15.172:0/3874535671 conn(0x56248c5e0e50 msgr2=0x56248c62db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 1 -- 172.21.15.172:0/3874535671 shutdown_connections 2024-04-14T08:37:55.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.651+0000 7ff97720a740 1 -- 172.21.15.172:0/3874535671 wait complete. 2024-04-14T08:37:55.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff97720a740 20 client.7206 trim_cache size 0 max 16384 2024-04-14T08:37:55.671 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:37:55.671 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-14T08:37:55.671 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-04-14T08:37:55.671 INFO:teuthology.orchestra.run.smithi172.stdout: "52d99e10-b424-493a-9dfb-310838b39bc9": { 2024-04-14T08:37:55.671 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/subdir", 2024-04-14T08:37:55.671 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "52d99e10-b424-493a-9dfb-310838b39bc9", 2024-04-14T08:37:55.671 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,repair" 2024-04-14T08:37:55.671 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:37:55.671 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:37:55.671 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:37:55.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff974420640 1 -- 172.21.15.172:0/3717469898 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff95006c920 msgr2=0x7ff95006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff974420640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff95006c920 0x7ff95006ede0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff95c006fd0 tx=0x7ff95c008040 comp rx=0 tx=0).stop 2024-04-14T08:37:55.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff974420640 1 -- 172.21.15.172:0/3717469898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff96c162540 msgr2=0x7ff96c0cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:37:55.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff974420640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff96c162540 0x7ff96c0cc420 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7ff964009e50 tx=0x7ff964004a40 comp rx=0 tx=0).stop 2024-04-14T08:37:55.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff974420640 1 -- 172.21.15.172:0/3717469898 shutdown_connections 2024-04-14T08:37:55.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff974420640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff95006c920 0x7ff95006ede0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff974420640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff96c162540 0x7ff96c0cc420 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff974420640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff96c15e430 0x7ff96c0cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff974420640 1 --2- 172.21.15.172:0/3717469898 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c0d2e00 0x7ff96c0b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:37:55.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff974420640 1 -- 172.21.15.172:0/3717469898 >> 172.21.15.172:0/3717469898 conn(0x7ff96c09d560 msgr2=0x7ff96c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:37:55.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff974420640 1 -- 172.21.15.172:0/3717469898 shutdown_connections 2024-04-14T08:37:55.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:37:55.652+0000 7ff974420640 1 -- 172.21.15.172:0/3717469898 wait complete. 2024-04-14T08:37:55.675 INFO:tasks.cephfs.filesystem:scrub status for tag:52d99e10-b424-493a-9dfb-310838b39bc9 - {'path': '/subdir', 'tag': '52d99e10-b424-493a-9dfb-310838b39bc9', 'options': 'recursive,repair'} 2024-04-14T08:38:24.078 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:38:24.082 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:38:25.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:25.676 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-14T08:38:25.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.780+0000 7f96e17bb640 1 Processor -- start 2024-04-14T08:38:25.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.780+0000 7f96e17bb640 1 -- start start 2024-04-14T08:38:25.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.781+0000 7f96e17bb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f96dc15e430 0x7f96dc162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.781+0000 7f96e17bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96dc162540 0x7f96dc1629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.781+0000 7f96e17bb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96dc0cefb0 con 0x7f96dc0b3a30 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.781+0000 7f96e17bb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f96dc0cf290 con 0x7f96dc15e430 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.781+0000 7f96e17bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f96dc0cf570 con 0x7f96dc162540 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.781+0000 7f96db7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f96dc0b3a30 0x7f96dc0b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43766/0 (socket says 172.21.15.172:43766) 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.781+0000 7f96db7fe640 1 -- 172.21.15.172:0/1351766281 learned_addr learned my addr 172.21.15.172:0/1351766281 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.781+0000 7f96daffd640 1 --2- 172.21.15.172:0/1351766281 >> v2:172.21.15.177:3300/0 conn(0x7f96dc15e430 0x7f96dc162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.781+0000 7f96da7fc640 1 --2- 172.21.15.172:0/1351766281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96dc162540 0x7f96dc1629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.781+0000 7f96d9ffb640 1 -- 172.21.15.172:0/1351766281 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 402569970 0 0) 0x7f96dc0cefb0 con 0x7f96dc0b3a30 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.781+0000 7f96d9ffb640 1 -- 172.21.15.172:0/1351766281 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96c4003520 con 0x7f96dc0b3a30 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.782+0000 7f96daffd640 1 -- 172.21.15.172:0/1351766281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96dc162540 msgr2=0x7f96dc1629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.782+0000 7f96daffd640 1 --2- 172.21.15.172:0/1351766281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96dc162540 0x7f96dc1629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.782+0000 7f96daffd640 1 -- 172.21.15.172:0/1351766281 >> v1:172.21.15.172:6789/0 conn(0x7f96dc0b3a30 legacy=0x7f96dc0b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.782+0000 7f96d9ffb640 1 -- 172.21.15.172:0/1351766281 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1579252163 0 0) 0x7f96c4003520 con 0x7f96dc0b3a30 2024-04-14T08:38:25.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.782+0000 7f96daffd640 1 -- 172.21.15.172:0/1351766281 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc0cf850 con 0x7f96dc15e430 2024-04-14T08:38:25.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.782+0000 7f96da7fc640 1 --2- 172.21.15.172:0/1351766281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96dc162540 0x7f96dc1629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:25.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.782+0000 7f96daffd640 1 --2- 172.21.15.172:0/1351766281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96dc15e430 0x7f96dc162000 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f96d00097a0 tx=0x7f96d002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aad64ebd0c6085d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:25.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.783+0000 7f96d9ffb640 1 -- 172.21.15.172:0/1351766281 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0037070 con 0x7f96dc15e430 2024-04-14T08:38:25.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.783+0000 7f96d9ffb640 1 -- 172.21.15.172:0/1351766281 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d0030510 con 0x7f96dc15e430 2024-04-14T08:38:25.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.783+0000 7f96d9ffb640 1 -- 172.21.15.172:0/1351766281 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d40019f0 con 0x7f96dc15e430 2024-04-14T08:38:25.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.783+0000 7f96e17bb640 1 -- 172.21.15.172:0/1351766281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96dc15e430 msgr2=0x7f96dc162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.783+0000 7f96e17bb640 1 --2- 172.21.15.172:0/1351766281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96dc15e430 0x7f96dc162000 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f96d00097a0 tx=0x7f96d002b790 comp rx=0 tx=0).stop 2024-04-14T08:38:25.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.783+0000 7f96e17bb640 1 -- 172.21.15.172:0/1351766281 shutdown_connections 2024-04-14T08:38:25.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.783+0000 7f96e17bb640 1 --2- 172.21.15.172:0/1351766281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96dc162540 0x7f96dc1629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.783+0000 7f96e17bb640 1 --2- 172.21.15.172:0/1351766281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96dc15e430 0x7f96dc162000 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.783+0000 7f96e17bb640 1 -- 172.21.15.172:0/1351766281 >> 172.21.15.172:0/1351766281 conn(0x7f96dc09d560 msgr2=0x7f96dc0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:25.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.783+0000 7f96e17bb640 1 -- 172.21.15.172:0/1351766281 shutdown_connections 2024-04-14T08:38:25.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.784+0000 7f96e17bb640 1 -- 172.21.15.172:0/1351766281 wait complete. 2024-04-14T08:38:25.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.785+0000 7f96e17bb640 1 Processor -- start 2024-04-14T08:38:25.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.785+0000 7f96e17bb640 1 -- start start 2024-04-14T08:38:25.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96e17bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96dc0b3a30 0x7f96dc15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96e17bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96dc15e430 0x7f96dc0b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96e17bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96dc162540 0x7f96dc0cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96e17bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0d4e20 con 0x7f96dc15e430 2024-04-14T08:38:25.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96e17bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc15e810 con 0x7f96dc162540 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96e17bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f96dc0d0050 con 0x7f96dc0b3a30 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96da7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96dc15e430 0x7f96dc0b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96da7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96dc15e430 0x7f96dc0b6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41208/0 (socket says 172.21.15.172:41208) 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96da7fc640 1 -- 172.21.15.172:0/3145027676 learned_addr learned my addr 172.21.15.172:0/3145027676 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96daffd640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96dc0b3a30 0x7f96dc15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96db7fe640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96dc162540 0x7f96dc0cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96da7fc640 1 -- 172.21.15.172:0/3145027676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96dc0b3a30 msgr2=0x7f96dc15a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96da7fc640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96dc0b3a30 0x7f96dc15a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96da7fc640 1 -- 172.21.15.172:0/3145027676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96dc162540 msgr2=0x7f96dc0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96da7fc640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96dc162540 0x7f96dc0cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.786+0000 7f96da7fc640 1 -- 172.21.15.172:0/3145027676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc0ccc60 con 0x7f96dc15e430 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.787+0000 7f96da7fc640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96dc15e430 0x7f96dc0b6d80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00ec10 tx=0x7f96cc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:25.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.787+0000 7f96c37fe640 1 -- 172.21.15.172:0/3145027676 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc017070 con 0x7f96dc15e430 2024-04-14T08:38:25.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.787+0000 7f96c37fe640 1 -- 172.21.15.172:0/3145027676 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96cc00cc30 con 0x7f96dc15e430 2024-04-14T08:38:25.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.787+0000 7f96e17bb640 1 -- 172.21.15.172:0/3145027676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc160030 con 0x7f96dc15e430 2024-04-14T08:38:25.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.787+0000 7f96c37fe640 1 -- 172.21.15.172:0/3145027676 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc013470 con 0x7f96dc15e430 2024-04-14T08:38:25.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.787+0000 7f96e17bb640 1 -- 172.21.15.172:0/3145027676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc1605f0 con 0x7f96dc15e430 2024-04-14T08:38:25.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.788+0000 7f96c37fe640 1 -- 172.21.15.172:0/3145027676 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96cc01a070 con 0x7f96dc15e430 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.789+0000 7f96c37fe640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f96c806c8f0 0x7f96c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.789+0000 7f96daffd640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f96c806c8f0 0x7f96c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.789+0000 7f96c37fe640 1 -- 172.21.15.172:0/3145027676 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f96cc08c040 con 0x7f96dc15e430 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.789+0000 7f96daffd640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f96c806c8f0 0x7f96c806edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f96d0034000 tx=0x7f96d002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.790+0000 7f96e459f740 1 Processor -- start 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.790+0000 7f96e459f740 1 -- start start 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96e459f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cb1970 0x556252c84910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96e459f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cb4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96e459f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252c84e50 0x556252cb4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96e459f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556252bee800 con 0x556252cb1d50 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96e459f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556252b967f0 con 0x556252cb1970 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96e459f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x556252bea7b0 con 0x556252c84e50 2024-04-14T08:38:25.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96da7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cb4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96da7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cb4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41220/0 (socket says 172.21.15.172:41220) 2024-04-14T08:38:25.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96da7fc640 1 -- 172.21.15.172:0/1961121074 learned_addr learned my addr 172.21.15.172:0/1961121074 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:25.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96db7fe640 1 --2- 172.21.15.172:0/1961121074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252c84e50 0x556252cb4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96daffd640 1 --2- 172.21.15.172:0/1961121074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cb1970 0x556252c84910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96da7fc640 1 -- 172.21.15.172:0/1961121074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252c84e50 msgr2=0x556252cb4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:25.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96da7fc640 1 --2- 172.21.15.172:0/1961121074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252c84e50 0x556252cb4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96da7fc640 1 -- 172.21.15.172:0/1961121074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cb1970 msgr2=0x556252c84910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:25.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96da7fc640 1 --2- 172.21.15.172:0/1961121074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cb1970 0x556252c84910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96da7fc640 1 -- 172.21.15.172:0/1961121074 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562529cc7d0 con 0x556252cb1d50 2024-04-14T08:38:25.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96da7fc640 1 --2- 172.21.15.172:0/1961121074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cb4700 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f96cc062240 tx=0x7f96cc055f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b01221062c3ae37 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:25.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96c0ff9640 1 -- 172.21.15.172:0/1961121074 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc021040 con 0x556252cb1d50 2024-04-14T08:38:25.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96c0ff9640 1 -- 172.21.15.172:0/1961121074 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96cc05df40 con 0x556252cb1d50 2024-04-14T08:38:25.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.791+0000 7f96c0ff9640 1 -- 172.21.15.172:0/1961121074 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc062770 con 0x556252cb1d50 2024-04-14T08:38:25.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.792+0000 7f96e459f740 1 -- 172.21.15.172:0/1961121074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 msgr2=0x556252cb4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.792+0000 7f96e459f740 1 --2- 172.21.15.172:0/1961121074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cb4700 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f96cc062240 tx=0x7f96cc055f60 comp rx=0 tx=0).stop 2024-04-14T08:38:25.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.792+0000 7f96e459f740 1 -- 172.21.15.172:0/1961121074 shutdown_connections 2024-04-14T08:38:25.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.792+0000 7f96e459f740 1 --2- 172.21.15.172:0/1961121074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252c84e50 0x556252cb4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.792+0000 7f96e459f740 1 --2- 172.21.15.172:0/1961121074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cb4700 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.792+0000 7f96e459f740 1 --2- 172.21.15.172:0/1961121074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cb1970 0x556252c84910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.792+0000 7f96e459f740 1 -- 172.21.15.172:0/1961121074 >> 172.21.15.172:0/1961121074 conn(0x556252c86b80 msgr2=0x556252ccf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:25.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.792+0000 7f96e459f740 1 -- 172.21.15.172:0/1961121074 shutdown_connections 2024-04-14T08:38:25.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.792+0000 7f96e459f740 1 -- 172.21.15.172:0/1961121074 wait complete. 2024-04-14T08:38:25.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.793+0000 7f96e459f740 1 -- start start 2024-04-14T08:38:25.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.793+0000 7f96e459f740 1 Processor -- start 2024-04-14T08:38:25.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.793+0000 7f96e459f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252cb1970 0x556252cb6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.793+0000 7f96e459f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cc9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96e459f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cca460 0x556252c9f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96e459f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556252c89120 con 0x556252cb1d50 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96e459f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556252ba9d00 con 0x556252cca460 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96e459f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x556252c88510 con 0x556252cb1970 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96e459f740 10 client.0 _ll_register_callbacks cb 0x556252ba96f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96da7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cc9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96da7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cc9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41236/0 (socket says 172.21.15.172:41236) 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96da7fc640 1 -- 172.21.15.172:0/2588175192 learned_addr learned my addr 172.21.15.172:0/2588175192 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96daffd640 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252cb1970 0x556252cb6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96da7fc640 1 -- 172.21.15.172:0/2588175192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252cb1970 msgr2=0x556252cb6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96da7fc640 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252cb1970 0x556252cb6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96da7fc640 1 -- 172.21.15.172:0/2588175192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cca460 msgr2=0x556252c9f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96db7fe640 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cca460 0x556252c9f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96da7fc640 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cca460 0x556252c9f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96da7fc640 1 -- 172.21.15.172:0/2588175192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562529023f0 con 0x556252cb1d50 2024-04-14T08:38:25.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96da7fc640 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cc9f20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f96cc070040 tx=0x7f96cc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6427e7c35f50da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:25.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:38:25.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96b9ffb640 1 -- 172.21.15.172:0/2588175192 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc021040 con 0x556252cb1d50 2024-04-14T08:38:25.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96b9ffb640 1 -- 172.21.15.172:0/2588175192 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f96cc074ca0 con 0x556252cb1d50 2024-04-14T08:38:25.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96e459f740 1 -- 172.21.15.172:0/2588175192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96cc05df40 con 0x556252cb1d50 2024-04-14T08:38:25.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96b9ffb640 1 -- 172.21.15.172:0/2588175192 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f96cc074ea0 con 0x556252cb1d50 2024-04-14T08:38:25.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96e459f740 10 client.7218 fetch_fsmap learned FSMap version 47 2024-04-14T08:38:25.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.794+0000 7f96e459f740 1 -- 172.21.15.172:0/2588175192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x5562528c20f0 con 0x556252cb1d50 2024-04-14T08:38:25.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.795+0000 7f96b9ffb640 1 -- 172.21.15.172:0/2588175192 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96cc05df40 con 0x556252cb1d50 2024-04-14T08:38:25.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.795+0000 7f96b9ffb640 1 -- 172.21.15.172:0/2588175192 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 47) ==== 3494+0+0 (secure 0 0 0) 0x7f96cc058fe0 con 0x556252cb1d50 2024-04-14T08:38:25.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.795+0000 7f96e459f740 10 client.7218 fetch_fsmap finished waiting for FSMap version 47 2024-04-14T08:38:25.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.795+0000 7f96e459f740 10 client.7218 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-14T08:38:25.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.795+0000 7f96e459f740 20 client.7218 populate_metadata read hostname 'smithi172' 2024-04-14T08:38:25.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.795+0000 7f96e459f740 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x556252ca53e0 0x556252ca5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.795+0000 7f96e459f740 4 client.7218 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:38:25.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.795+0000 7f96e459f740 1 -- 172.21.15.172:0/2588175192 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556252bfbd40 con 0x556252ca53e0 2024-04-14T08:38:25.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.795+0000 7f96daffd640 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x556252ca53e0 0x556252ca5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.795+0000 7f96b9ffb640 10 client.7218 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:38:25.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.795+0000 7f96daffd640 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x556252ca53e0 0x556252ca5850 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d51529636f233a server_cookie=3122d5f987913c52 in_seq=0 out_seq=0 2024-04-14T08:38:25.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96b9ffb640 1 -- 172.21.15.172:0/2588175192 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556252bfbd40 con 0x556252ca53e0 2024-04-14T08:38:25.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96b9ffb640 10 client.7218 handle_command_reply: tid=0 2024-04-14T08:38:25.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96e459f740 1 client.7218 shutdown 2024-04-14T08:38:25.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96e459f740 1 -- 172.21.15.172:0/2588175192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 msgr2=0x556252cc9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96e459f740 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cc9f20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f96cc070040 tx=0x7f96cc000bc0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96e459f740 1 -- 172.21.15.172:0/2588175192 shutdown_connections 2024-04-14T08:38:25.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96e459f740 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x556252ca53e0 0x556252ca5850 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96e459f740 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cca460 0x556252c9f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96e459f740 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cc9f20 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96b9ffb640 0 client.7218 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:38:25.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96e459f740 1 --2- 172.21.15.172:0/2588175192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252cb1970 0x556252cb6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96e459f740 1 -- 172.21.15.172:0/2588175192 >> 172.21.15.172:0/2588175192 conn(0x556252c84e50 msgr2=0x556252cd2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:25.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96e459f740 1 -- 172.21.15.172:0/2588175192 shutdown_connections 2024-04-14T08:38:25.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96e459f740 1 -- 172.21.15.172:0/2588175192 wait complete. 2024-04-14T08:38:25.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.796+0000 7f96e459f740 20 client.7218 trim_cache size 0 max 16384 2024-04-14T08:38:25.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.813+0000 7f96e459f740 1 Processor -- start 2024-04-14T08:38:25.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.814+0000 7f96e459f740 1 -- start start 2024-04-14T08:38:25.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.814+0000 7f96e459f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252ca53e0 0x556252cb4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.814+0000 7f96e459f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252cb1970 0x556252cc90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.814+0000 7f96e459f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cc95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.814+0000 7f96e459f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc201010 con 0x556252cb1d50 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.814+0000 7f96e459f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc200090 con 0x556252ca53e0 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.814+0000 7f96e459f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f96dc1f80e0 con 0x556252cb1970 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.814+0000 7f96db7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cc95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.814+0000 7f96db7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cc95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41252/0 (socket says 172.21.15.172:41252) 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.814+0000 7f96db7fe640 1 -- 172.21.15.172:0/3348173386 learned_addr learned my addr 172.21.15.172:0/3348173386 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96db7fe640 1 -- 172.21.15.172:0/3348173386 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252cb1970 msgr2=0x556252cc90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96db7fe640 1 --2- 172.21.15.172:0/3348173386 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252cb1970 0x556252cc90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96db7fe640 1 -- 172.21.15.172:0/3348173386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252ca53e0 msgr2=0x556252cb4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96db7fe640 1 --2- 172.21.15.172:0/3348173386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252ca53e0 0x556252cb4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96db7fe640 1 -- 172.21.15.172:0/3348173386 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556252cb7340 con 0x556252cb1d50 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96db7fe640 1 --2- 172.21.15.172:0/3348173386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cc95f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f96d4001a00 tx=0x7f96d4007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d4ef82a6205eb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96bbfff640 1 -- 172.21.15.172:0/3348173386 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4010040 con 0x556252cb1d50 2024-04-14T08:38:25.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96bbfff640 1 -- 172.21.15.172:0/3348173386 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d401a070 con 0x556252cb1d50 2024-04-14T08:38:25.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96bbfff640 1 -- 172.21.15.172:0/3348173386 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4015850 con 0x556252cb1d50 2024-04-14T08:38:25.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96e459f740 1 -- 172.21.15.172:0/3348173386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 msgr2=0x556252cc95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96e459f740 1 --2- 172.21.15.172:0/3348173386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cc95f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f96d4001a00 tx=0x7f96d4007420 comp rx=0 tx=0).stop 2024-04-14T08:38:25.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96e459f740 1 -- 172.21.15.172:0/3348173386 shutdown_connections 2024-04-14T08:38:25.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96e459f740 1 --2- 172.21.15.172:0/3348173386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cc95f0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96e459f740 1 --2- 172.21.15.172:0/3348173386 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252cb1970 0x556252cc90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96e459f740 1 --2- 172.21.15.172:0/3348173386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252ca53e0 0x556252cb4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96e459f740 1 -- 172.21.15.172:0/3348173386 >> 172.21.15.172:0/3348173386 conn(0x556252c84e50 msgr2=0x556252c800a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:25.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96e459f740 1 -- 172.21.15.172:0/3348173386 shutdown_connections 2024-04-14T08:38:25.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.815+0000 7f96e459f740 1 -- 172.21.15.172:0/3348173386 wait complete. 2024-04-14T08:38:25.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.816+0000 7f96e459f740 1 -- start start 2024-04-14T08:38:25.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.816+0000 7f96e459f740 1 Processor -- start 2024-04-14T08:38:25.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96e459f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252ca53e0 0x556252ca43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96e459f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cb1970 0x556252ca4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96e459f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cd0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96e459f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1ffaa0 con 0x556252cb1d50 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96e459f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556252c895e0 con 0x556252cb1970 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96e459f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x556252d49d20 con 0x556252ca53e0 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96e459f740 10 client.0 _ll_register_callbacks cb 0x7f96cc072180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96db7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cd0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96db7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cd0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41256/0 (socket says 172.21.15.172:41256) 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96db7fe640 1 -- 172.21.15.172:0/1867961345 learned_addr learned my addr 172.21.15.172:0/1867961345 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96daffd640 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252ca53e0 0x556252ca43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96da7fc640 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cb1970 0x556252ca4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96db7fe640 1 -- 172.21.15.172:0/1867961345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252ca53e0 msgr2=0x556252ca43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96db7fe640 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252ca53e0 0x556252ca43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96db7fe640 1 -- 172.21.15.172:0/1867961345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cb1970 msgr2=0x556252ca4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96db7fe640 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cb1970 0x556252ca4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96db7fe640 1 -- 172.21.15.172:0/1867961345 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556252d49110 con 0x556252cb1d50 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96db7fe640 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cd0990 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f96d4017040 tx=0x7f96d4001de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cd2b13be5abe62 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.817+0000 7f96b9ffb640 1 -- 172.21.15.172:0/1867961345 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4023070 con 0x556252cb1d50 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96b9ffb640 1 -- 172.21.15.172:0/1867961345 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f96d4028ca0 con 0x556252cb1d50 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96e459f740 1 -- 172.21.15.172:0/1867961345 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d401a070 con 0x556252cb1d50 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96b9ffb640 1 -- 172.21.15.172:0/1867961345 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f96d401f400 con 0x556252cb1d50 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96e459f740 10 client.7224 fetch_fsmap learned FSMap version 47 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96e459f740 1 -- 172.21.15.172:0/1867961345 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x556252c89320 con 0x556252cb1d50 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96b9ffb640 1 -- 172.21.15.172:0/1867961345 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d401a070 con 0x556252cb1d50 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96b9ffb640 1 -- 172.21.15.172:0/1867961345 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 47) ==== 3494+0+0 (secure 0 0 0) 0x7f96d401f5a0 con 0x556252cb1d50 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96e459f740 10 client.7224 fetch_fsmap finished waiting for FSMap version 47 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96e459f740 10 client.7224 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-14T08:38:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96e459f740 20 client.7224 populate_metadata read hostname 'smithi172' 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96e459f740 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x556252cca460 0x556252cd0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96e459f740 4 client.7224 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96e459f740 1 -- 172.21.15.172:0/1867961345 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556252bf6090 con 0x556252cca460 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96daffd640 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x556252cca460 0x556252cd0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96b9ffb640 10 client.7224 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.818+0000 7f96daffd640 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x556252cca460 0x556252cd0ed0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228630fc25dbc38e server_cookie=73c6d226350431a2 in_seq=0 out_seq=0 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96b9ffb640 1 -- 172.21.15.172:0/1867961345 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556252bf6090 con 0x556252cca460 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96b9ffb640 10 client.7224 handle_command_reply: tid=0 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e459f740 1 client.7224 shutdown 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e459f740 1 -- 172.21.15.172:0/1867961345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 msgr2=0x556252cd0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e459f740 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cd0990 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f96d4017040 tx=0x7f96d4001de0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e459f740 1 -- 172.21.15.172:0/1867961345 shutdown_connections 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e459f740 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x556252cca460 0x556252cd0ed0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e459f740 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556252cb1d50 0x556252cd0990 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e459f740 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556252cb1970 0x556252ca4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96b9ffb640 0 client.7224 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:38:25.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e459f740 1 --2- 172.21.15.172:0/1867961345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556252ca53e0 0x556252ca43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e459f740 1 -- 172.21.15.172:0/1867961345 >> 172.21.15.172:0/1867961345 conn(0x556252c84e50 msgr2=0x556252cb4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:25.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e459f740 1 -- 172.21.15.172:0/1867961345 shutdown_connections 2024-04-14T08:38:25.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e459f740 1 -- 172.21.15.172:0/1867961345 wait complete. 2024-04-14T08:38:25.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e459f740 20 client.7224 trim_cache size 0 max 16384 2024-04-14T08:38:25.826 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:38:25.826 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-04-14T08:38:25.827 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-04-14T08:38:25.827 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:38:25.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e17bb640 1 -- 172.21.15.172:0/3145027676 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f96c806c8f0 msgr2=0x7f96c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e17bb640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f96c806c8f0 0x7f96c806edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f96d0034000 tx=0x7f96d002f000 comp rx=0 tx=0).stop 2024-04-14T08:38:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e17bb640 1 -- 172.21.15.172:0/3145027676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96dc15e430 msgr2=0x7f96dc0b6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e17bb640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96dc15e430 0x7f96dc0b6d80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00ec10 tx=0x7f96cc00c520 comp rx=0 tx=0).stop 2024-04-14T08:38:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e17bb640 1 -- 172.21.15.172:0/3145027676 shutdown_connections 2024-04-14T08:38:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e17bb640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f96c806c8f0 0x7f96c806edb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e17bb640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96dc162540 0x7f96dc0cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e17bb640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96dc15e430 0x7f96dc0b6d80 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e17bb640 1 --2- 172.21.15.172:0/3145027676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96dc0b3a30 0x7f96dc15a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.819+0000 7f96e17bb640 1 -- 172.21.15.172:0/3145027676 >> 172.21.15.172:0/3145027676 conn(0x7f96dc09d560 msgr2=0x7f96dc0b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.820+0000 7f96e17bb640 1 -- 172.21.15.172:0/3145027676 shutdown_connections 2024-04-14T08:38:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.820+0000 7f96e17bb640 1 -- 172.21.15.172:0/3145027676 wait complete. 2024-04-14T08:38:25.831 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-14T08:38:25.832 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:38:25.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.948+0000 7f7f31ede640 1 Processor -- start 2024-04-14T08:38:25.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.948+0000 7f7f31ede640 1 -- start start 2024-04-14T08:38:25.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f31ede640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7f2c0d2830 0x7f7f2c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f31ede640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7f2c0cedd0 0x7f7f2c0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f31ede640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f2c0d3150 con 0x7f7f2c15e890 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f31ede640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f7f2c0cff30 con 0x7f7f2c0cedd0 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f31ede640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f7f2c0d2350 con 0x7f7f2c0d2830 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f2affd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7f2c15e890 0x7f7f2c0d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43778/0 (socket says 172.21.15.172:43778) 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f2affd640 1 -- 172.21.15.172:0/3025486632 learned_addr learned my addr 172.21.15.172:0/3025486632 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f2bfff640 1 --2- 172.21.15.172:0/3025486632 >> v2:172.21.15.172:3300/0 conn(0x7f7f2c0cedd0 0x7f7f2c0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f2b7fe640 1 --2- 172.21.15.172:0/3025486632 >> v2:172.21.15.177:3300/0 conn(0x7f7f2c0d2830 0x7f7f2c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f2bfff640 1 -- 172.21.15.172:0/3025486632 >> v2:172.21.15.177:3300/0 conn(0x7f7f2c0d2830 msgr2=0x7f7f2c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f2bfff640 1 --2- 172.21.15.172:0/3025486632 >> v2:172.21.15.177:3300/0 conn(0x7f7f2c0d2830 0x7f7f2c0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f2bfff640 1 -- 172.21.15.172:0/3025486632 >> v1:172.21.15.172:6789/0 conn(0x7f7f2c15e890 legacy=0x7f7f2c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f2a7fc640 1 -- 172.21.15.172:0/3025486632 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3149866401 0 0) 0x7f7f2c0d3150 con 0x7f7f2c15e890 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.949+0000 7f7f2bfff640 1 -- 172.21.15.172:0/3025486632 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c0cf320 con 0x7f7f2c0cedd0 2024-04-14T08:38:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.950+0000 7f7f2bfff640 1 --2- 172.21.15.172:0/3025486632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f2c0cedd0 0x7f7f2c0b3710 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c08c830 tx=0x7f7f2402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88da5e1ccd4a21a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.950+0000 7f7f2a7fc640 1 -- 172.21.15.172:0/3025486632 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24039070 con 0x7f7f2c0cedd0 2024-04-14T08:38:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.950+0000 7f7f2a7fc640 1 -- 172.21.15.172:0/3025486632 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f2402cd00 con 0x7f7f2c0cedd0 2024-04-14T08:38:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.950+0000 7f7f2a7fc640 1 -- 172.21.15.172:0/3025486632 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24034660 con 0x7f7f2c0cedd0 2024-04-14T08:38:25.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.951+0000 7f7f31ede640 1 -- 172.21.15.172:0/3025486632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f2c0cedd0 msgr2=0x7f7f2c0b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.951+0000 7f7f31ede640 1 --2- 172.21.15.172:0/3025486632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f2c0cedd0 0x7f7f2c0b3710 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c08c830 tx=0x7f7f2402bf50 comp rx=0 tx=0).stop 2024-04-14T08:38:25.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.951+0000 7f7f31ede640 1 -- 172.21.15.172:0/3025486632 shutdown_connections 2024-04-14T08:38:25.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.951+0000 7f7f31ede640 1 --2- 172.21.15.172:0/3025486632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f2c0cedd0 0x7f7f2c0b3710 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.951+0000 7f7f31ede640 1 --2- 172.21.15.172:0/3025486632 >> v2:172.21.15.177:3300/0 conn(0x7f7f2c0d2830 0x7f7f2c0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.951+0000 7f7f31ede640 1 -- 172.21.15.172:0/3025486632 >> 172.21.15.172:0/3025486632 conn(0x7f7f2c09d590 msgr2=0x7f7f2c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:25.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.951+0000 7f7f31ede640 1 -- 172.21.15.172:0/3025486632 shutdown_connections 2024-04-14T08:38:25.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.951+0000 7f7f31ede640 1 -- 172.21.15.172:0/3025486632 wait complete. 2024-04-14T08:38:25.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.952+0000 7f7f31ede640 1 Processor -- start 2024-04-14T08:38:25.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.952+0000 7f7f31ede640 1 -- start start 2024-04-14T08:38:25.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f31ede640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c0cedd0 0x7f7f2c0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f31ede640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f2c0d2830 0x7f7f2c159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f31ede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f2c15e890 0x7f7f2c0b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f31ede640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c0cfbc0 con 0x7f7f2c0d2830 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f31ede640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c0cf8e0 con 0x7f7f2c0cedd0 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f31ede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2c0b3c50 con 0x7f7f2c15e890 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f2affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f2c0d2830 0x7f7f2c159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f2affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f2c0d2830 0x7f7f2c159840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41274/0 (socket says 172.21.15.172:41274) 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f2affd640 1 -- 172.21.15.172:0/219708969 learned_addr learned my addr 172.21.15.172:0/219708969 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f2b7fe640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c0cedd0 0x7f7f2c0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f2bfff640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f2c15e890 0x7f7f2c0b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f2affd640 1 -- 172.21.15.172:0/219708969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f2c15e890 msgr2=0x7f7f2c0b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f2affd640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f2c15e890 0x7f7f2c0b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f2affd640 1 -- 172.21.15.172:0/219708969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c0cedd0 msgr2=0x7f7f2c0c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f2affd640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c0cedd0 0x7f7f2c0c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:25.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f2affd640 1 -- 172.21.15.172:0/219708969 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c1f8400 con 0x7f7f2c0d2830 2024-04-14T08:38:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.953+0000 7f7f2affd640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f2c0d2830 0x7f7f2c159840 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c430 tx=0x7f7f2000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.954+0000 7f7f13fff640 1 -- 172.21.15.172:0/219708969 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f200074e0 con 0x7f7f2c0d2830 2024-04-14T08:38:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.954+0000 7f7f13fff640 1 -- 172.21.15.172:0/219708969 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f200047e0 con 0x7f7f2c0d2830 2024-04-14T08:38:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.954+0000 7f7f13fff640 1 -- 172.21.15.172:0/219708969 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20004bd0 con 0x7f7f2c0d2830 2024-04-14T08:38:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.954+0000 7f7f31ede640 1 -- 172.21.15.172:0/219708969 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2c1602e0 con 0x7f7f2c0d2830 2024-04-14T08:38:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.954+0000 7f7f31ede640 1 -- 172.21.15.172:0/219708969 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2c0b13c0 con 0x7f7f2c0d2830 2024-04-14T08:38:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.955+0000 7f7f31ede640 1 -- 172.21.15.172:0/219708969 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f2c068520 con 0x7f7f2c0d2830 2024-04-14T08:38:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.955+0000 7f7f13fff640 1 -- 172.21.15.172:0/219708969 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f20011070 con 0x7f7f2c0d2830 2024-04-14T08:38:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.956+0000 7f7f13fff640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7f1806c8f0 0x7f7f1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.956+0000 7f7f13fff640 1 -- 172.21.15.172:0/219708969 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f7f2008b980 con 0x7f7f2c0d2830 2024-04-14T08:38:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.956+0000 7f7f2b7fe640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7f1806c8f0 0x7f7f1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.956+0000 7f7f2b7fe640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7f1806c8f0 0x7f7f1806edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006fd0 tx=0x7f7f1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:25.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:25.965+0000 7f7f13fff640 1 -- 172.21.15.172:0/219708969 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7f20059c60 con 0x7f7f2c0d2830 2024-04-14T08:38:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.187+0000 7f7f31ede640 1 -- 172.21.15.172:0/219708969 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f2c0d2610 con 0x7f7f2c0d2830 2024-04-14T08:38:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.187+0000 7f7f13fff640 1 -- 172.21.15.172:0/219708969 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5194 (secure 0 0 0) 0x7f7f20059560 con 0x7f7f2c0d2830 2024-04-14T08:38:26.188 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:26.188 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:37:01.115031+0000","modified":"2024-04-14T08:38:02.376656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":15,"addr":"172.21.15.172:6837/2920402198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2920402198},{"type":"v1","addr":"172.21.15.172:6837","nonce":2920402198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":5}]} 2024-04-14T08:38:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 47 2024-04-14T08:38:26.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.189+0000 7f7f31ede640 1 -- 172.21.15.172:0/219708969 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7f1806c8f0 msgr2=0x7f7f1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.189+0000 7f7f31ede640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7f1806c8f0 0x7f7f1806edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006fd0 tx=0x7f7f1c008040 comp rx=0 tx=0).stop 2024-04-14T08:38:26.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.189+0000 7f7f31ede640 1 -- 172.21.15.172:0/219708969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f2c0d2830 msgr2=0x7f7f2c159840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.189+0000 7f7f31ede640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f2c0d2830 0x7f7f2c159840 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000c430 tx=0x7f7f2000c8f0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.189+0000 7f7f31ede640 1 -- 172.21.15.172:0/219708969 shutdown_connections 2024-04-14T08:38:26.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.189+0000 7f7f31ede640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7f1806c8f0 0x7f7f1806edb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.189+0000 7f7f31ede640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f2c15e890 0x7f7f2c0b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.189+0000 7f7f31ede640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f2c0d2830 0x7f7f2c159840 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.189+0000 7f7f31ede640 1 --2- 172.21.15.172:0/219708969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f2c0cedd0 0x7f7f2c0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.189+0000 7f7f31ede640 1 -- 172.21.15.172:0/219708969 >> 172.21.15.172:0/219708969 conn(0x7f7f2c09d590 msgr2=0x7f7f2c15db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:26.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.190+0000 7f7f31ede640 1 -- 172.21.15.172:0/219708969 shutdown_connections 2024-04-14T08:38:26.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.190+0000 7f7f31ede640 1 -- 172.21.15.172:0/219708969 wait complete. 2024-04-14T08:38:26.201 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-14T08:38:26.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.341+0000 7f625122d640 1 Processor -- start 2024-04-14T08:38:26.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.341+0000 7f625122d640 1 -- start start 2024-04-14T08:38:26.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f625122d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f624c0b65a0 0x7f624c1629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f625122d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f624c0cea60 0x7f624c0d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f625122d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f624c0b3750 con 0x7f624c15e590 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f625122d640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f624c0cef20 con 0x7f624c0cea60 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f625122d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f624c0cf200 con 0x7f624c0b65a0 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f624b59e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f624c0cea60 0x7f624c0d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f624ad9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f624c15e590 0x7f624c0b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43794/0 (socket says 172.21.15.172:43794) 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f624ad9d640 1 -- 172.21.15.172:0/2683204970 learned_addr learned my addr 172.21.15.172:0/2683204970 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f624a59c640 1 --2- 172.21.15.172:0/2683204970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f624c0b65a0 0x7f624c1629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f624b59e640 1 -- 172.21.15.172:0/2683204970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f624c0b65a0 msgr2=0x7f624c1629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f624b59e640 1 --2- 172.21.15.172:0/2683204970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f624c0b65a0 0x7f624c1629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f624b59e640 1 -- 172.21.15.172:0/2683204970 >> v1:172.21.15.172:6789/0 conn(0x7f624c15e590 legacy=0x7f624c0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f6249d9b640 1 -- 172.21.15.172:0/2683204970 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677874906 0 0) 0x7f624c0b3750 con 0x7f624c15e590 2024-04-14T08:38:26.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.342+0000 7f624b59e640 1 -- 172.21.15.172:0/2683204970 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624c0cf4e0 con 0x7f624c0cea60 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.343+0000 7f624b59e640 1 --2- 172.21.15.172:0/2683204970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f624c0cea60 0x7f624c0d2760 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6244009d90 tx=0x7f624402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6881fe1abc45e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.343+0000 7f6249d9b640 1 -- 172.21.15.172:0/2683204970 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244039070 con 0x7f624c0cea60 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.343+0000 7f6249d9b640 1 -- 172.21.15.172:0/2683204970 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624402cc50 con 0x7f624c0cea60 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.343+0000 7f6249d9b640 1 -- 172.21.15.172:0/2683204970 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244034440 con 0x7f624c0cea60 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.344+0000 7f625122d640 1 -- 172.21.15.172:0/2683204970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f624c0cea60 msgr2=0x7f624c0d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.344+0000 7f625122d640 1 --2- 172.21.15.172:0/2683204970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f624c0cea60 0x7f624c0d2760 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6244009d90 tx=0x7f624402bea0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.344+0000 7f625122d640 1 -- 172.21.15.172:0/2683204970 shutdown_connections 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.344+0000 7f625122d640 1 --2- 172.21.15.172:0/2683204970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f624c0b65a0 0x7f624c1629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.344+0000 7f625122d640 1 --2- 172.21.15.172:0/2683204970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f624c0cea60 0x7f624c0d2760 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.344+0000 7f625122d640 1 -- 172.21.15.172:0/2683204970 >> 172.21.15.172:0/2683204970 conn(0x7f624c0d0e90 msgr2=0x7f624c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.344+0000 7f625122d640 1 -- 172.21.15.172:0/2683204970 shutdown_connections 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.344+0000 7f625122d640 1 -- 172.21.15.172:0/2683204970 wait complete. 2024-04-14T08:38:26.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.345+0000 7f625122d640 1 Processor -- start 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.345+0000 7f625122d640 1 -- start start 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f625122d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f624c0b65a0 0x7f624c1e8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f625122d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f624c0cea60 0x7f624c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f625122d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f624c15e590 0x7f624c0b8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f625122d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f624c0d2e20 con 0x7f624c15e590 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f625122d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f624c0d2ca0 con 0x7f624c0b65a0 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f625122d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f624c0d2fa0 con 0x7f624c0cea60 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f624b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f624c15e590 0x7f624c0b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f624b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f624c15e590 0x7f624c0b8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41296/0 (socket says 172.21.15.172:41296) 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f624b59e640 1 -- 172.21.15.172:0/3043119154 learned_addr learned my addr 172.21.15.172:0/3043119154 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f624b59e640 1 -- 172.21.15.172:0/3043119154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f624c0cea60 msgr2=0x7f624c15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f624b59e640 1 --2- 172.21.15.172:0/3043119154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f624c0cea60 0x7f624c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f624b59e640 1 -- 172.21.15.172:0/3043119154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f624c0b65a0 msgr2=0x7f624c1e8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f624b59e640 1 --2- 172.21.15.172:0/3043119154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f624c0b65a0 0x7f624c1e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f624b59e640 1 -- 172.21.15.172:0/3043119154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624c1e8880 con 0x7f624c15e590 2024-04-14T08:38:26.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.346+0000 7f624b59e640 1 --2- 172.21.15.172:0/3043119154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f624c15e590 0x7f624c0b8720 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6244037040 tx=0x7f6244004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:26.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.347+0000 7f62337fe640 1 -- 172.21.15.172:0/3043119154 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244044020 con 0x7f624c15e590 2024-04-14T08:38:26.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.347+0000 7f62337fe640 1 -- 172.21.15.172:0/3043119154 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624403a040 con 0x7f624c15e590 2024-04-14T08:38:26.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.347+0000 7f625122d640 1 -- 172.21.15.172:0/3043119154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624c1e8b60 con 0x7f624c15e590 2024-04-14T08:38:26.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.347+0000 7f62337fe640 1 -- 172.21.15.172:0/3043119154 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244039050 con 0x7f624c15e590 2024-04-14T08:38:26.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.347+0000 7f625122d640 1 -- 172.21.15.172:0/3043119154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624c1f4f00 con 0x7f624c15e590 2024-04-14T08:38:26.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.349+0000 7f62337fe640 1 -- 172.21.15.172:0/3043119154 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6244003990 con 0x7f624c15e590 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.349+0000 7f62337fe640 1 --2- 172.21.15.172:0/3043119154 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f623806c920 0x7f623806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.349+0000 7f62337fe640 1 -- 172.21.15.172:0/3043119154 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f62440ae0b0 con 0x7f624c15e590 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.349+0000 7f624ad9d640 1 --2- 172.21.15.172:0/3043119154 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f623806c920 0x7f623806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.349+0000 7f624ad9d640 1 --2- 172.21.15.172:0/3043119154 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f623806c920 0x7f623806ede0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f623c007fd0 tx=0x7f623c007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.350+0000 7f625400b740 1 Processor -- start 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.350+0000 7f625400b740 1 -- start start 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f625400b740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b503fd90 0x5626b5040170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f625400b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b5045bb0 0x5626b5043de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f625400b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 0x5626b508d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f625400b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5626b50075a0 con 0x5626b508d4a0 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f625400b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5626b4fa9680 con 0x5626b5045bb0 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f625400b740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5626b4f47860 con 0x5626b503fd90 2024-04-14T08:38:26.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 0x5626b508d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 0x5626b508d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41300/0 (socket says 172.21.15.172:41300) 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624b59e640 1 -- 172.21.15.172:0/3578609813 learned_addr learned my addr 172.21.15.172:0/3578609813 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624a59c640 1 --2- 172.21.15.172:0/3578609813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b5045bb0 0x5626b5043de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624ad9d640 1 --2- 172.21.15.172:0/3578609813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b503fd90 0x5626b5040170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624b59e640 1 -- 172.21.15.172:0/3578609813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b503fd90 msgr2=0x5626b5040170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624b59e640 1 --2- 172.21.15.172:0/3578609813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b503fd90 0x5626b5040170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624b59e640 1 -- 172.21.15.172:0/3578609813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b5045bb0 msgr2=0x5626b5043de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624b59e640 1 --2- 172.21.15.172:0/3578609813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b5045bb0 0x5626b5043de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624b59e640 1 -- 172.21.15.172:0/3578609813 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626b4f761a0 con 0x5626b508d4a0 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624ad9d640 1 --2- 172.21.15.172:0/3578609813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b503fd90 0x5626b5040170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624a59c640 1 --2- 172.21.15.172:0/3578609813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b5045bb0 0x5626b5043de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f624b59e640 1 --2- 172.21.15.172:0/3578609813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 0x5626b508d920 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6244084340 tx=0x7f624402cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b863bab25560196d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:26.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f6230ff9640 1 -- 172.21.15.172:0/3578609813 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244080040 con 0x5626b508d4a0 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f6230ff9640 1 -- 172.21.15.172:0/3578609813 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6244084870 con 0x5626b508d4a0 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.351+0000 7f6230ff9640 1 -- 172.21.15.172:0/3578609813 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244079f90 con 0x5626b508d4a0 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.352+0000 7f625400b740 1 -- 172.21.15.172:0/3578609813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 msgr2=0x5626b508d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.352+0000 7f625400b740 1 --2- 172.21.15.172:0/3578609813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 0x5626b508d920 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6244084340 tx=0x7f624402cc50 comp rx=0 tx=0).stop 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.352+0000 7f625400b740 1 -- 172.21.15.172:0/3578609813 shutdown_connections 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.352+0000 7f625400b740 1 --2- 172.21.15.172:0/3578609813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 0x5626b508d920 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.352+0000 7f625400b740 1 --2- 172.21.15.172:0/3578609813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b5045bb0 0x5626b5043de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.352+0000 7f625400b740 1 --2- 172.21.15.172:0/3578609813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b503fd90 0x5626b5040170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.352+0000 7f625400b740 1 -- 172.21.15.172:0/3578609813 >> 172.21.15.172:0/3578609813 conn(0x5626b5041e10 msgr2=0x5626b50436d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.352+0000 7f625400b740 1 -- 172.21.15.172:0/3578609813 shutdown_connections 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.352+0000 7f625400b740 1 -- 172.21.15.172:0/3578609813 wait complete. 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.353+0000 7f625400b740 1 -- start start 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.353+0000 7f625400b740 1 Processor -- start 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.353+0000 7f625400b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5045bb0 0x5626b508a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.353+0000 7f625400b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b508d4a0 0x5626b5085150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.353+0000 7f625400b740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5085690 0x5626b5085fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.353+0000 7f625400b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5626b4f451d0 con 0x5626b5045bb0 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.353+0000 7f625400b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5626b4fa5630 con 0x5626b508d4a0 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.353+0000 7f625400b740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5626b50075a0 con 0x5626b5085690 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.353+0000 7f625400b740 10 client.0 _ll_register_callbacks cb 0x5626b4f68010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f624ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5045bb0 0x5626b508a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f624ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5045bb0 0x5626b508a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41304/0 (socket says 172.21.15.172:41304) 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f624ad9d640 1 -- 172.21.15.172:0/1063770298 learned_addr learned my addr 172.21.15.172:0/1063770298 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f624a59c640 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b508d4a0 0x5626b5085150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f624b59e640 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5085690 0x5626b5085fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f624ad9d640 1 -- 172.21.15.172:0/1063770298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5085690 msgr2=0x5626b5085fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f624ad9d640 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5085690 0x5626b5085fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f624ad9d640 1 -- 172.21.15.172:0/1063770298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b508d4a0 msgr2=0x5626b5085150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f624ad9d640 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b508d4a0 0x5626b5085150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f624ad9d640 1 -- 172.21.15.172:0/1063770298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626b4d897c0 con 0x5626b5045bb0 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f624b59e640 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5085690 0x5626b5085fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f624ad9d640 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5045bb0 0x5626b508a570 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f623c005bc0 tx=0x7f623c001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e56cf72f1741ee server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f6229ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f6229ffb640 1 -- 172.21.15.172:0/1063770298 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c0169d0 con 0x5626b5045bb0 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f6229ffb640 1 -- 172.21.15.172:0/1063770298 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f623c023ca0 con 0x5626b5045bb0 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f625400b740 1 -- 172.21.15.172:0/1063770298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6244084870 con 0x5626b5045bb0 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f6229ffb640 1 -- 172.21.15.172:0/1063770298 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f623c016b70 con 0x5626b5045bb0 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f625400b740 10 client.7248 fetch_fsmap learned FSMap version 47 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f625400b740 1 -- 172.21.15.172:0/1063770298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x5626b4de9f50 con 0x5626b5045bb0 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.354+0000 7f6229ffb640 1 -- 172.21.15.172:0/1063770298 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6244084870 con 0x5626b5045bb0 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.355+0000 7f6229ffb640 1 -- 172.21.15.172:0/1063770298 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 47) ==== 3494+0+0 (secure 0 0 0) 0x7f623c014430 con 0x5626b5045bb0 2024-04-14T08:38:26.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.355+0000 7f625400b740 10 client.7248 fetch_fsmap finished waiting for FSMap version 47 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.355+0000 7f625400b740 10 client.7248 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.355+0000 7f625400b740 20 client.7248 populate_metadata read hostname 'smithi172' 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.355+0000 7f625400b740 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5626b509b9d0 0x5626b50a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.355+0000 7f625400b740 4 client.7248 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.355+0000 7f625400b740 1 -- 172.21.15.172:0/1063770298 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626b4fbceb0 con 0x5626b509b9d0 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.355+0000 7f624a59c640 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5626b509b9d0 0x5626b50a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.355+0000 7f624a59c640 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5626b509b9d0 0x5626b50a8fa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d8956e9d1570a5 server_cookie=6b85f0f51f06bc5b in_seq=0 out_seq=0 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.355+0000 7f6229ffb640 10 client.7248 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f6229ffb640 1 -- 172.21.15.172:0/1063770298 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626b4fbceb0 con 0x5626b509b9d0 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f6229ffb640 10 client.7248 handle_command_reply: tid=0 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f625400b740 1 client.7248 shutdown 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f625400b740 1 -- 172.21.15.172:0/1063770298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5045bb0 msgr2=0x5626b508a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f625400b740 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5045bb0 0x5626b508a570 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f623c005bc0 tx=0x7f623c001080 comp rx=0 tx=0).stop 2024-04-14T08:38:26.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f625400b740 1 -- 172.21.15.172:0/1063770298 shutdown_connections 2024-04-14T08:38:26.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f625400b740 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5626b509b9d0 0x5626b50a8fa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f625400b740 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5085690 0x5626b5085fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f6229ffb640 0 client.7248 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:38:26.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f625400b740 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b508d4a0 0x5626b5085150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f625400b740 1 --2- 172.21.15.172:0/1063770298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5045bb0 0x5626b508a570 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f625400b740 1 -- 172.21.15.172:0/1063770298 >> 172.21.15.172:0/1063770298 conn(0x5626b503fd90 msgr2=0x5626b508b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:26.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f625400b740 1 -- 172.21.15.172:0/1063770298 shutdown_connections 2024-04-14T08:38:26.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f625400b740 1 -- 172.21.15.172:0/1063770298 wait complete. 2024-04-14T08:38:26.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.356+0000 7f625400b740 20 client.7248 trim_cache size 0 max 16384 2024-04-14T08:38:26.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.369+0000 7f625400b740 1 Processor -- start 2024-04-14T08:38:26.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.369+0000 7f625400b740 1 -- start start 2024-04-14T08:38:26.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f625400b740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5045bb0 0x5626b50703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f625400b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b5085690 0x5626b508a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f625400b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 0x5626b50846c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f625400b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f624c200030 con 0x5626b508d4a0 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f625400b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f624c1606d0 con 0x5626b5085690 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f625400b740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f624c1ff9e0 con 0x5626b5045bb0 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f624b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 0x5626b50846c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f624b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 0x5626b50846c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41312/0 (socket says 172.21.15.172:41312) 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f624b59e640 1 -- 172.21.15.172:0/2541164658 learned_addr learned my addr 172.21.15.172:0/2541164658 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f624a59c640 1 --2- 172.21.15.172:0/2541164658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b5085690 0x5626b508a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f624ad9d640 1 --2- 172.21.15.172:0/2541164658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5045bb0 0x5626b50703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f624b59e640 1 -- 172.21.15.172:0/2541164658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5045bb0 msgr2=0x5626b50703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f624b59e640 1 --2- 172.21.15.172:0/2541164658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5045bb0 0x5626b50703b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f624b59e640 1 -- 172.21.15.172:0/2541164658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b5085690 msgr2=0x5626b508a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:26.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f624b59e640 1 --2- 172.21.15.172:0/2541164658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b5085690 0x5626b508a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f624b59e640 1 -- 172.21.15.172:0/2541164658 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626b50854c0 con 0x5626b508d4a0 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f624ad9d640 1 --2- 172.21.15.172:0/2541164658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5045bb0 0x5626b50703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f624b59e640 1 --2- 172.21.15.172:0/2541164658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 0x5626b50846c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f624407b990 tx=0x7f624407e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398ff8414fa805c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.370+0000 7f622bfff640 1 -- 172.21.15.172:0/2541164658 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244084870 con 0x5626b508d4a0 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.371+0000 7f622bfff640 1 -- 172.21.15.172:0/2541164658 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6244080040 con 0x5626b508d4a0 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.371+0000 7f622bfff640 1 -- 172.21.15.172:0/2541164658 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624407bc90 con 0x5626b508d4a0 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.371+0000 7f625400b740 1 -- 172.21.15.172:0/2541164658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 msgr2=0x5626b50846c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.371+0000 7f625400b740 1 --2- 172.21.15.172:0/2541164658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 0x5626b50846c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f624407b990 tx=0x7f624407e000 comp rx=0 tx=0).stop 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.371+0000 7f625400b740 1 -- 172.21.15.172:0/2541164658 shutdown_connections 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.371+0000 7f625400b740 1 --2- 172.21.15.172:0/2541164658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b508d4a0 0x5626b50846c0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.371+0000 7f625400b740 1 --2- 172.21.15.172:0/2541164658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b5085690 0x5626b508a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.371+0000 7f625400b740 1 --2- 172.21.15.172:0/2541164658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5045bb0 0x5626b50703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.371+0000 7f625400b740 1 -- 172.21.15.172:0/2541164658 >> 172.21.15.172:0/2541164658 conn(0x5626b503fd90 msgr2=0x5626b5085c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:26.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.371+0000 7f625400b740 1 -- 172.21.15.172:0/2541164658 shutdown_connections 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.371+0000 7f625400b740 1 -- 172.21.15.172:0/2541164658 wait complete. 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.372+0000 7f625400b740 1 -- start start 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.372+0000 7f625400b740 1 Processor -- start 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f625400b740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5045bb0 0x5626b506b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f625400b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5085690 0x5626b5099b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f625400b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b508d4a0 0x5626b509a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f625400b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5626b509b850 con 0x5626b5085690 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f625400b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5626b509b3b0 con 0x5626b508d4a0 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f625400b740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5626b509be40 con 0x5626b5045bb0 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f625400b740 10 client.0 _ll_register_callbacks cb 0x7f623c001240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f624a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5085690 0x5626b5099b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f624a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5085690 0x5626b5099b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41328/0 (socket says 172.21.15.172:41328) 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f624a59c640 1 -- 172.21.15.172:0/1770904333 learned_addr learned my addr 172.21.15.172:0/1770904333 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:26.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f624ad9d640 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5045bb0 0x5626b506b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f624b59e640 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b508d4a0 0x5626b509a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f624a59c640 1 -- 172.21.15.172:0/1770904333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5045bb0 msgr2=0x5626b506b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f624a59c640 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5045bb0 0x5626b506b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f624a59c640 1 -- 172.21.15.172:0/1770904333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b508d4a0 msgr2=0x5626b509a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f624a59c640 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b508d4a0 0x5626b509a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f624a59c640 1 -- 172.21.15.172:0/1770904333 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626b4c7d0b0 con 0x5626b5085690 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f624b59e640 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b508d4a0 0x5626b509a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f624a59c640 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5085690 0x5626b5099b00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f624c15c080 tx=0x7f624001cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a974de97c9d5bb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.373+0000 7f6229ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f6229ffb640 1 -- 172.21.15.172:0/1770904333 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240004240 con 0x5626b5085690 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f6229ffb640 1 -- 172.21.15.172:0/1770904333 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f62400043e0 con 0x5626b5085690 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f625400b740 1 -- 172.21.15.172:0/1770904333 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6244080040 con 0x5626b5085690 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f6229ffb640 1 -- 172.21.15.172:0/1770904333 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f624002c050 con 0x5626b5085690 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f625400b740 10 client.7254 fetch_fsmap learned FSMap version 47 2024-04-14T08:38:26.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f625400b740 1 -- 172.21.15.172:0/1770904333 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x5626b5105a20 con 0x5626b5085690 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f6229ffb640 1 -- 172.21.15.172:0/1770904333 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6244080040 con 0x5626b5085690 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f6229ffb640 1 -- 172.21.15.172:0/1770904333 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 47) ==== 3494+0+0 (secure 0 0 0) 0x7f624002f020 con 0x5626b5085690 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f625400b740 10 client.7254 fetch_fsmap finished waiting for FSMap version 47 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f625400b740 10 client.7254 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f625400b740 20 client.7254 populate_metadata read hostname 'smithi172' 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f625400b740 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5626b506f190 0x5626b506f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f625400b740 4 client.7254 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f625400b740 1 -- 172.21.15.172:0/1770904333 --> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5626b5042570 con 0x5626b506f190 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f624ad9d640 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5626b506f190 0x5626b506f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f624ad9d640 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5626b506f190 0x5626b506f570 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95e19b4645ae5b2a server_cookie=49acb360abba5fc in_seq=0 out_seq=0 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.374+0000 7f6229ffb640 10 client.7254 ms_handle_connect on v2:172.21.15.172:6835/2920402198 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f6229ffb640 1 -- 172.21.15.172:0/1770904333 <== mds.0 v2:172.21.15.172:6835/2920402198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626b5042570 con 0x5626b506f190 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f6229ffb640 10 client.7254 handle_command_reply: tid=0 2024-04-14T08:38:26.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f625400b740 1 client.7254 shutdown 2024-04-14T08:38:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f625400b740 1 -- 172.21.15.172:0/1770904333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5085690 msgr2=0x5626b5099b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f625400b740 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5085690 0x5626b5099b00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f624c15c080 tx=0x7f624001cf20 comp rx=0 tx=0).stop 2024-04-14T08:38:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f625400b740 1 -- 172.21.15.172:0/1770904333 shutdown_connections 2024-04-14T08:38:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f625400b740 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.172:6835/2920402198,v1:172.21.15.172:6837/2920402198] conn(0x5626b506f190 0x5626b506f570 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f625400b740 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626b508d4a0 0x5626b509a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f625400b740 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626b5085690 0x5626b5099b00 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f6229ffb640 0 client.7254 ms_handle_reset on v2:172.21.15.172:6835/2920402198 2024-04-14T08:38:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f625400b740 1 --2- 172.21.15.172:0/1770904333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626b5045bb0 0x5626b506b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f625400b740 1 -- 172.21.15.172:0/1770904333 >> 172.21.15.172:0/1770904333 conn(0x5626b503fd90 msgr2=0x5626b508bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f625400b740 1 -- 172.21.15.172:0/1770904333 shutdown_connections 2024-04-14T08:38:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f625400b740 1 -- 172.21.15.172:0/1770904333 wait complete. 2024-04-14T08:38:26.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.375+0000 7f625400b740 20 client.7254 trim_cache size 0 max 16384 2024-04-14T08:38:26.383 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:26.383 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-04-14T08:38:26.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.376+0000 7f625122d640 1 -- 172.21.15.172:0/3043119154 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f623806c920 msgr2=0x7f623806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.376+0000 7f625122d640 1 --2- 172.21.15.172:0/3043119154 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f623806c920 0x7f623806ede0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f623c007fd0 tx=0x7f623c007960 comp rx=0 tx=0).stop 2024-04-14T08:38:26.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.376+0000 7f625122d640 1 -- 172.21.15.172:0/3043119154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f624c15e590 msgr2=0x7f624c0b8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.376+0000 7f625122d640 1 --2- 172.21.15.172:0/3043119154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f624c15e590 0x7f624c0b8720 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6244037040 tx=0x7f6244004a40 comp rx=0 tx=0).stop 2024-04-14T08:38:26.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.376+0000 7f625122d640 1 -- 172.21.15.172:0/3043119154 shutdown_connections 2024-04-14T08:38:26.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.376+0000 7f625122d640 1 --2- 172.21.15.172:0/3043119154 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f623806c920 0x7f623806ede0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.376+0000 7f625122d640 1 --2- 172.21.15.172:0/3043119154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f624c15e590 0x7f624c0b8720 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.376+0000 7f625122d640 1 --2- 172.21.15.172:0/3043119154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f624c0cea60 0x7f624c15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.376+0000 7f625122d640 1 --2- 172.21.15.172:0/3043119154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f624c0b65a0 0x7f624c1e8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.376+0000 7f625122d640 1 -- 172.21.15.172:0/3043119154 >> 172.21.15.172:0/3043119154 conn(0x7f624c0d0e90 msgr2=0x7f624c0b7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:26.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.376+0000 7f625122d640 1 -- 172.21.15.172:0/3043119154 shutdown_connections 2024-04-14T08:38:26.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.376+0000 7f625122d640 1 -- 172.21.15.172:0/3043119154 wait complete. 2024-04-14T08:38:26.387 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-14T08:38:26.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.486+0000 7fd41adc8640 1 Processor -- start 2024-04-14T08:38:26.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.487+0000 7fd41adc8640 1 -- start start 2024-04-14T08:38:26.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.487+0000 7fd41adc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd41415e890 0x7fd4140d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.487+0000 7fd41adc8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd4140cedd0 0x7fd4140b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.487+0000 7fd41adc8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4140cf320 con 0x7fd4140d2830 2024-04-14T08:38:26.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.487+0000 7fd41adc8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd4140cff30 con 0x7fd4140cedd0 2024-04-14T08:38:26.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.487+0000 7fd41adc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd4140d2350 con 0x7fd41415e890 2024-04-14T08:38:26.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.487+0000 7fd419365640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd4140cedd0 0x7fd4140b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.487+0000 7fd419365640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd4140cedd0 0x7fd4140b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41340/0 (socket says 172.21.15.172:41340) 2024-04-14T08:38:26.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.487+0000 7fd419365640 1 -- 172.21.15.172:0/593776665 learned_addr learned my addr 172.21.15.172:0/593776665 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:26.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.488+0000 7fd413fff640 1 --2- 172.21.15.172:0/593776665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd41415e890 0x7fd4140d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.488+0000 7fd419365640 1 -- 172.21.15.172:0/593776665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd41415e890 msgr2=0x7fd4140d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:26.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.488+0000 7fd419365640 1 --2- 172.21.15.172:0/593776665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd41415e890 0x7fd4140d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.488+0000 7fd419365640 1 -- 172.21.15.172:0/593776665 >> v1:172.21.15.177:6789/0 conn(0x7fd4140d2830 legacy=0x7fd4140d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.488+0000 7fd419365640 1 -- 172.21.15.172:0/593776665 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4140cf600 con 0x7fd4140cedd0 2024-04-14T08:38:26.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.488+0000 7fd413fff640 1 --2- 172.21.15.172:0/593776665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd41415e890 0x7fd4140d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:26.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.488+0000 7fd419365640 1 --2- 172.21.15.172:0/593776665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4140cedd0 0x7fd4140b3710 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009e50 tx=0x7fd40c02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75169a4ce97fc19b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:26.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.489+0000 7fd4137fe640 1 -- 172.21.15.172:0/593776665 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c02cb90 con 0x7fd4140cedd0 2024-04-14T08:38:26.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.489+0000 7fd4137fe640 1 -- 172.21.15.172:0/593776665 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd40c02cd30 con 0x7fd4140cedd0 2024-04-14T08:38:26.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.489+0000 7fd4137fe640 1 -- 172.21.15.172:0/593776665 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c034610 con 0x7fd4140cedd0 2024-04-14T08:38:26.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.489+0000 7fd41adc8640 1 -- 172.21.15.172:0/593776665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4140cedd0 msgr2=0x7fd4140b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.489+0000 7fd41adc8640 1 --2- 172.21.15.172:0/593776665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4140cedd0 0x7fd4140b3710 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009e50 tx=0x7fd40c02bff0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.489+0000 7fd41adc8640 1 -- 172.21.15.172:0/593776665 shutdown_connections 2024-04-14T08:38:26.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.489+0000 7fd41adc8640 1 --2- 172.21.15.172:0/593776665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4140cedd0 0x7fd4140b3710 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.489+0000 7fd41adc8640 1 --2- 172.21.15.172:0/593776665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd41415e890 0x7fd4140d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.489+0000 7fd41adc8640 1 -- 172.21.15.172:0/593776665 >> 172.21.15.172:0/593776665 conn(0x7fd41409d590 msgr2=0x7fd4140d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:26.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.489+0000 7fd41adc8640 1 -- 172.21.15.172:0/593776665 shutdown_connections 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.489+0000 7fd41adc8640 1 -- 172.21.15.172:0/593776665 wait complete. 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.491+0000 7fd41adc8640 1 Processor -- start 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.491+0000 7fd41adc8640 1 -- start start 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.491+0000 7fd41adc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4140cedd0 0x7fd4140cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.491+0000 7fd41adc8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4140d2830 0x7fd414154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.491+0000 7fd41adc8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd41415e890 0x7fd4141606b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.491+0000 7fd41adc8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd4140d1840 con 0x7fd41415e890 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.491+0000 7fd41adc8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd4140cfbc0 con 0x7fd4140d2830 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.491+0000 7fd41adc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd4140b3c50 con 0x7fd4140cedd0 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd419365640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd41415e890 0x7fd4141606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd419365640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd41415e890 0x7fd4141606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41354/0 (socket says 172.21.15.172:41354) 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd419365640 1 -- 172.21.15.172:0/3106008940 learned_addr learned my addr 172.21.15.172:0/3106008940 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:26.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd413fff640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4140d2830 0x7fd414154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd418b64640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4140cedd0 0x7fd4140cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd419365640 1 -- 172.21.15.172:0/3106008940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4140cedd0 msgr2=0x7fd4140cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd419365640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4140cedd0 0x7fd4140cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd419365640 1 -- 172.21.15.172:0/3106008940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4140d2830 msgr2=0x7fd414154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd419365640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4140d2830 0x7fd414154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd419365640 1 -- 172.21.15.172:0/3106008940 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4140c8160 con 0x7fd41415e890 2024-04-14T08:38:26.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd419365640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd41415e890 0x7fd4141606b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd40c036040 tx=0x7fd40c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:26.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd4117fa640 1 -- 172.21.15.172:0/3106008940 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c043020 con 0x7fd41415e890 2024-04-14T08:38:26.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd4117fa640 1 -- 172.21.15.172:0/3106008940 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd40c03e450 con 0x7fd41415e890 2024-04-14T08:38:26.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd4117fa640 1 -- 172.21.15.172:0/3106008940 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c009510 con 0x7fd41415e890 2024-04-14T08:38:26.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd41adc8640 1 -- 172.21.15.172:0/3106008940 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41415fbd0 con 0x7fd41415e890 2024-04-14T08:38:26.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.492+0000 7fd41adc8640 1 -- 172.21.15.172:0/3106008940 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4140c8440 con 0x7fd41415e890 2024-04-14T08:38:26.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.493+0000 7fd41adc8640 1 -- 172.21.15.172:0/3106008940 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4140d41d0 con 0x7fd41415e890 2024-04-14T08:38:26.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.494+0000 7fd4117fa640 1 -- 172.21.15.172:0/3106008940 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd40c046020 con 0x7fd41415e890 2024-04-14T08:38:26.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.494+0000 7fd4117fa640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd3f806c8f0 0x7fd3f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.495+0000 7fd418b64640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd3f806c8f0 0x7fd3f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.495+0000 7fd4117fa640 1 -- 172.21.15.172:0/3106008940 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fd40c0ae370 con 0x7fd41415e890 2024-04-14T08:38:26.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.495+0000 7fd418b64640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd3f806c8f0 0x7fd3f806edb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd408001780 tx=0x7fd408009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:26.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.499+0000 7fd4117fa640 1 -- 172.21.15.172:0/3106008940 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd40c04d030 con 0x7fd41415e890 2024-04-14T08:38:26.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.718+0000 7fd41adc8640 1 -- 172.21.15.172:0/3106008940 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd414068520 con 0x7fd41415e890 2024-04-14T08:38:26.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.718+0000 7fd4117fa640 1 -- 172.21.15.172:0/3106008940 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd40c07bef0 con 0x7fd41415e890 2024-04-14T08:38:26.719 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:26.719 INFO:teuthology.orchestra.run.smithi172.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-14T08:38:26.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.720+0000 7fd41adc8640 1 -- 172.21.15.172:0/3106008940 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd3f806c8f0 msgr2=0x7fd3f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.720+0000 7fd41adc8640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd3f806c8f0 0x7fd3f806edb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd408001780 tx=0x7fd408009040 comp rx=0 tx=0).stop 2024-04-14T08:38:26.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.720+0000 7fd41adc8640 1 -- 172.21.15.172:0/3106008940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd41415e890 msgr2=0x7fd4141606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.720+0000 7fd41adc8640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd41415e890 0x7fd4141606b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd40c036040 tx=0x7fd40c004a40 comp rx=0 tx=0).stop 2024-04-14T08:38:26.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.721+0000 7fd41adc8640 1 -- 172.21.15.172:0/3106008940 shutdown_connections 2024-04-14T08:38:26.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.721+0000 7fd41adc8640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd3f806c8f0 0x7fd3f806edb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.721+0000 7fd41adc8640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd41415e890 0x7fd4141606b0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.721+0000 7fd41adc8640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4140d2830 0x7fd414154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.721+0000 7fd41adc8640 1 --2- 172.21.15.172:0/3106008940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4140cedd0 0x7fd4140cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.721+0000 7fd41adc8640 1 -- 172.21.15.172:0/3106008940 >> 172.21.15.172:0/3106008940 conn(0x7fd41409d590 msgr2=0x7fd4140d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:26.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.721+0000 7fd41adc8640 1 -- 172.21.15.172:0/3106008940 shutdown_connections 2024-04-14T08:38:26.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.721+0000 7fd41adc8640 1 -- 172.21.15.172:0/3106008940 wait complete. 2024-04-14T08:38:26.734 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-14T08:38:26.734 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:38:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.840+0000 7f823eab7640 1 Processor -- start 2024-04-14T08:38:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.840+0000 7f823eab7640 1 -- start start 2024-04-14T08:38:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.841+0000 7f823eab7640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f82380afbb0 0x7f82380b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.841+0000 7f823eab7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806b520 0x7f823806b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.841+0000 7f823eab7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82380b01b0 con 0x7f823815c480 2024-04-14T08:38:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.841+0000 7f823eab7640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f82380726b0 con 0x7f82380afbb0 2024-04-14T08:38:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.841+0000 7f823eab7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8238073250 con 0x7f823806b520 2024-04-14T08:38:26.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.841+0000 7f8237fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f82380afbb0 0x7f82380b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.841+0000 7f8237fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f82380afbb0 0x7f82380b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41364/0 (socket says 172.21.15.172:41364) 2024-04-14T08:38:26.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.841+0000 7f8237fff640 1 -- 172.21.15.172:0/2449857586 learned_addr learned my addr 172.21.15.172:0/2449857586 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:26.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.841+0000 7f823d054640 1 --2- 172.21.15.172:0/2449857586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806b520 0x7f823806b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.841+0000 7f8237fff640 1 -- 172.21.15.172:0/2449857586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806b520 msgr2=0x7f823806b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:26.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.842+0000 7f8237fff640 1 --2- 172.21.15.172:0/2449857586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806b520 0x7f823806b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.842+0000 7f8237fff640 1 -- 172.21.15.172:0/2449857586 >> v1:172.21.15.177:6789/0 conn(0x7f823815c480 legacy=0x7f8238072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:26.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.842+0000 7f8237fff640 1 -- 172.21.15.172:0/2449857586 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82380b0490 con 0x7f82380afbb0 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.842+0000 7f8237fff640 1 --2- 172.21.15.172:0/2449857586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82380afbb0 0x7f82380b5670 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f822c009870 tx=0x7f822c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eda5cf0a8af988f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.842+0000 7f82377fe640 1 -- 172.21.15.172:0/2449857586 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c034070 con 0x7f82380afbb0 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.842+0000 7f82377fe640 1 -- 172.21.15.172:0/2449857586 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c030840 con 0x7f82380afbb0 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.842+0000 7f82377fe640 1 -- 172.21.15.172:0/2449857586 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c030c10 con 0x7f82380afbb0 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.843+0000 7f823eab7640 1 -- 172.21.15.172:0/2449857586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82380afbb0 msgr2=0x7f82380b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.843+0000 7f823eab7640 1 --2- 172.21.15.172:0/2449857586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82380afbb0 0x7f82380b5670 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f822c009870 tx=0x7f822c02b980 comp rx=0 tx=0).stop 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.843+0000 7f823eab7640 1 -- 172.21.15.172:0/2449857586 shutdown_connections 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.843+0000 7f823eab7640 1 --2- 172.21.15.172:0/2449857586 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806b520 0x7f823806b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.843+0000 7f823eab7640 1 --2- 172.21.15.172:0/2449857586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82380afbb0 0x7f82380b5670 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.843+0000 7f823eab7640 1 -- 172.21.15.172:0/2449857586 >> 172.21.15.172:0/2449857586 conn(0x7f82380b2b40 msgr2=0x7f82380b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.843+0000 7f823eab7640 1 -- 172.21.15.172:0/2449857586 shutdown_connections 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.843+0000 7f823eab7640 1 -- 172.21.15.172:0/2449857586 wait complete. 2024-04-14T08:38:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.844+0000 7f823eab7640 1 Processor -- start 2024-04-14T08:38:26.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.844+0000 7f823eab7640 1 -- start start 2024-04-14T08:38:26.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.845+0000 7f823eab7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806b520 0x7f823809e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.845+0000 7f823eab7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82380afbb0 0x7f82380d0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.845+0000 7f823eab7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823815c480 0x7f823814d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:26.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.845+0000 7f823eab7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f82380b5ef0 con 0x7f82380afbb0 2024-04-14T08:38:26.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.845+0000 7f823eab7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8238072340 con 0x7f823815c480 2024-04-14T08:38:26.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.845+0000 7f823eab7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f823815fe80 con 0x7f823806b520 2024-04-14T08:38:26.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.845+0000 7f8237fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82380afbb0 0x7f82380d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.845+0000 7f8237fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82380afbb0 0x7f82380d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41376/0 (socket says 172.21.15.172:41376) 2024-04-14T08:38:26.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.845+0000 7f8237fff640 1 -- 172.21.15.172:0/614236512 learned_addr learned my addr 172.21.15.172:0/614236512 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:26.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f823c853640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806b520 0x7f823809e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f823d054640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823815c480 0x7f823814d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:26.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f8237fff640 1 -- 172.21.15.172:0/614236512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806b520 msgr2=0x7f823809e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f8237fff640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806b520 0x7f823809e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f8237fff640 1 -- 172.21.15.172:0/614236512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823815c480 msgr2=0x7f823814d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:26.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f8237fff640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823815c480 0x7f823814d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:26.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f8237fff640 1 -- 172.21.15.172:0/614236512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823809bf80 con 0x7f82380afbb0 2024-04-14T08:38:26.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f823c853640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806b520 0x7f823809e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:26.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f823d054640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823815c480 0x7f823814d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:26.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f8237fff640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82380afbb0 0x7f82380d0440 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f822c000c00 tx=0x7f822c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:26.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f82357fa640 1 -- 172.21.15.172:0/614236512 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c048070 con 0x7f82380afbb0 2024-04-14T08:38:26.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f82357fa640 1 -- 172.21.15.172:0/614236512 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c030640 con 0x7f82380afbb0 2024-04-14T08:38:26.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f823eab7640 1 -- 172.21.15.172:0/614236512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823815dac0 con 0x7f82380afbb0 2024-04-14T08:38:26.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f82357fa640 1 -- 172.21.15.172:0/614236512 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c034040 con 0x7f82380afbb0 2024-04-14T08:38:26.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.846+0000 7f823eab7640 1 -- 172.21.15.172:0/614236512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82381f2050 con 0x7f82380afbb0 2024-04-14T08:38:26.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.847+0000 7f823eab7640 1 -- 172.21.15.172:0/614236512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f823815d8b0 con 0x7f82380afbb0 2024-04-14T08:38:26.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.848+0000 7f82357fa640 1 -- 172.21.15.172:0/614236512 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f822c04b070 con 0x7f82380afbb0 2024-04-14T08:38:26.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.848+0000 7f82357fa640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f822406c8f0 0x7f822406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:27.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.848+0000 7f823c853640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f822406c8f0 0x7f822406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:27.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.848+0000 7f82357fa640 1 -- 172.21.15.172:0/614236512 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f822c0adca0 con 0x7f82380afbb0 2024-04-14T08:38:27.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.849+0000 7f823c853640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f822406c8f0 0x7f822406edb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f82381f55a0 tx=0x7f8228009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:27.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:26.853+0000 7f82357fa640 1 -- 172.21.15.172:0/614236512 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f822c07bef0 con 0x7f82380afbb0 2024-04-14T08:38:27.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.073+0000 7f823eab7640 1 -- 172.21.15.172:0/614236512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8238075490 con 0x7f82380afbb0 2024-04-14T08:38:27.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.073+0000 7f82357fa640 1 -- 172.21.15.172:0/614236512 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7f822c07b7f0 con 0x7f82380afbb0 2024-04-14T08:38:27.499 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:38:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.075+0000 7f823eab7640 1 -- 172.21.15.172:0/614236512 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f822406c8f0 msgr2=0x7f822406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.075+0000 7f823eab7640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f822406c8f0 0x7f822406edb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f82381f55a0 tx=0x7f8228009040 comp rx=0 tx=0).stop 2024-04-14T08:38:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.075+0000 7f823eab7640 1 -- 172.21.15.172:0/614236512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82380afbb0 msgr2=0x7f82380d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.075+0000 7f823eab7640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82380afbb0 0x7f82380d0440 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f822c000c00 tx=0x7f822c004730 comp rx=0 tx=0).stop 2024-04-14T08:38:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.075+0000 7f823eab7640 1 -- 172.21.15.172:0/614236512 shutdown_connections 2024-04-14T08:38:27.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.075+0000 7f823eab7640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f822406c8f0 0x7f822406edb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:27.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.075+0000 7f823eab7640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f823815c480 0x7f823814d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:27.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.075+0000 7f823eab7640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82380afbb0 0x7f82380d0440 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:27.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.075+0000 7f823eab7640 1 --2- 172.21.15.172:0/614236512 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f823806b520 0x7f823809e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:27.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.075+0000 7f823eab7640 1 -- 172.21.15.172:0/614236512 >> 172.21.15.172:0/614236512 conn(0x7f82380b2b40 msgr2=0x7f82380b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:27.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.075+0000 7f823eab7640 1 -- 172.21.15.172:0/614236512 shutdown_connections 2024-04-14T08:38:27.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27.075+0000 7f823eab7640 1 -- 172.21.15.172:0/614236512 wait complete. 2024-04-14T08:38:27.505 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:38:27.505 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:38:27.505 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:38:27.505 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:38:27.505 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:38:27.505 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:38:27.506 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:38:27.506 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:38:27.506 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:38:27.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:27.507 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:38:27.524 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:38:27.524 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:38:27.524 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:38:27.524 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:27.524 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:38:27.524 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:27.524 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:38:27.524 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:38:27.525 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:38:27.525 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:38:27.525 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:38:27.525 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:38:27.525 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:38:27.525 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:38:27.525 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:38:27.525 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:38:27.525 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42362sec preferred_lft 42362sec 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:27.526 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:38:27.527 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:38:27.527 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:38:27.527 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:27.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:27.527 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:38:27.590 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:38:27.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:27.592 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:38:27.648 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:38:27.648 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:38:27.648 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:38:27.648 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:38:27.648 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:38:27.649 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:38:27.649 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:38:27.649 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:38:27.649 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:38:27.649 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:38:27.649 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:38:27.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:38:27.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:38:27.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:27.789 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:38:27.804 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:38:27.804 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:38:27.804 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:38:27.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:27.804 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:38:27.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:38:27.875 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:38:27.931 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:38:27.931 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:38:27.931 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:38:27.986 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:38:27.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:27.986 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:38:28.070 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:38:28.070 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:38:28.070 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:38:28.079 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:38:28.079 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:38:28.079 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:38:28.079 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:38:28.079 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:38:28.084 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:38:28.085 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:38:28.085 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:38:28.085 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:38:28.085 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:38:28.085 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:38:28.085 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:38:28.085 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:38:28.085 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:38:28.086 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:38:28.086 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:38:28.086 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:38:28.086 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:38:28.087 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:38:28.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:28.087 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:38:28.118 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:38:28.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:38:28.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:38:28.214 INFO:teuthology.orchestra.run.smithi172.stdout:client6856 2024-04-14T08:38:28.214 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:38:28.214 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:28.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:28.230 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-14T08:38:28.285 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:38:28.286 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-14T08:38:28.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.370+0000 7faf7e3b7f40 1 Processor -- start 2024-04-14T08:38:28.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.370+0000 7faf7e3b7f40 1 -- start start 2024-04-14T08:38:28.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7e3b7f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 0x55706485a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7e3b7f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55706485a850 0x5570647cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7e3b7f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570647ce450 0x5570646f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7e3b7f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55706469e3b0 con 0x7faf640030f0 2024-04-14T08:38:28.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7e3b7f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5570646a2400 con 0x5570647ce450 2024-04-14T08:38:28.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7e3b7f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5570647c4900 con 0x55706485a850 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7bc4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 0x55706485a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7bc4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 0x55706485a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41390/0 (socket says 172.21.15.172:41390) 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7bc4b640 1 -- 172.21.15.172:0/1637402278 learned_addr learned my addr 172.21.15.172:0/1637402278 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7bc4b640 1 -- 172.21.15.172:0/1637402278 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55706485a850 msgr2=0x5570647cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7bc4b640 1 --2- 172.21.15.172:0/1637402278 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55706485a850 0x5570647cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7bc4b640 1 -- 172.21.15.172:0/1637402278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570647ce450 msgr2=0x5570646f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7bc4b640 1 --2- 172.21.15.172:0/1637402278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570647ce450 0x5570646f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7bc4b640 1 -- 172.21.15.172:0/1637402278 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570647c50c0 con 0x7faf640030f0 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.371+0000 7faf7bc4b640 1 --2- 172.21.15.172:0/1637402278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 0x55706485a310 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7faf70002e90 tx=0x7faf70009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1933aaadb946bbcd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.372+0000 7faf7a448640 1 -- 172.21.15.172:0/1637402278 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70013470 con 0x7faf640030f0 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.372+0000 7faf7a448640 1 -- 172.21.15.172:0/1637402278 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf70013610 con 0x7faf640030f0 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.372+0000 7faf7a448640 1 -- 172.21.15.172:0/1637402278 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf700139c0 con 0x7faf640030f0 2024-04-14T08:38:28.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.372+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/1637402278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 msgr2=0x55706485a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:28.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.372+0000 7faf7e3b7f40 1 --2- 172.21.15.172:0/1637402278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 0x55706485a310 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7faf70002e90 tx=0x7faf70009720 comp rx=0 tx=0).stop 2024-04-14T08:38:28.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.372+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/1637402278 shutdown_connections 2024-04-14T08:38:28.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.372+0000 7faf7e3b7f40 1 --2- 172.21.15.172:0/1637402278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5570647ce450 0x5570646f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.372+0000 7faf7e3b7f40 1 --2- 172.21.15.172:0/1637402278 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55706485a850 0x5570647cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.372+0000 7faf7e3b7f40 1 --2- 172.21.15.172:0/1637402278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 0x55706485a310 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.372+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/1637402278 >> 172.21.15.172:0/1637402278 conn(0x557064741ca0 msgr2=0x5570647cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:28.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.372+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/1637402278 shutdown_connections 2024-04-14T08:38:28.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.372+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/1637402278 wait complete. 2024-04-14T08:38:28.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.373+0000 7faf7e3b7f40 1 Processor -- start 2024-04-14T08:38:28.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.373+0000 7faf7e3b7f40 1 -- start start 2024-04-14T08:38:28.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.373+0000 7faf7e3b7f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5570647ce450 0x5570647ba760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.373+0000 7faf7e3b7f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55706485a850 0x5570646f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.373+0000 7faf7e3b7f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 0x5570647bda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.373+0000 7faf7e3b7f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5570647cd270 con 0x7faf640030f0 2024-04-14T08:38:28.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.373+0000 7faf7e3b7f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5570646f97e0 con 0x55706485a850 2024-04-14T08:38:28.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.373+0000 7faf7e3b7f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5570646f9500 con 0x5570647ce450 2024-04-14T08:38:28.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.373+0000 7faf7c44c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 0x5570647bda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:28.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf7c44c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 0x5570647bda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41400/0 (socket says 172.21.15.172:41400) 2024-04-14T08:38:28.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf7bc4b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5570647ce450 0x5570647ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:28.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf7c44c640 1 -- 172.21.15.172:0/2834379320 learned_addr learned my addr 172.21.15.172:0/2834379320 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:28.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf7b44a640 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55706485a850 0x5570646f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:28.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf7c44c640 1 -- 172.21.15.172:0/2834379320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5570647ce450 msgr2=0x5570647ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf7c44c640 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5570647ce450 0x5570647ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf7c44c640 1 -- 172.21.15.172:0/2834379320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55706485a850 msgr2=0x5570646f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf7c44c640 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55706485a850 0x5570646f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf7c44c640 1 -- 172.21.15.172:0/2834379320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570647baca0 con 0x7faf640030f0 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf7c44c640 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 0x5570647bda20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x557064742d20 tx=0x7faf7400c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf79446640 1 -- 172.21.15.172:0/2834379320 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74014410 con 0x7faf640030f0 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf79446640 1 -- 172.21.15.172:0/2834379320 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf74019070 con 0x7faf640030f0 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf79446640 1 -- 172.21.15.172:0/2834379320 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf740147f0 con 0x7faf640030f0 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/2834379320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5570647c5b30 con 0x7faf640030f0 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.374+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/2834379320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706485c2e0 con 0x7faf640030f0 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.376+0000 7faf79446640 1 -- 172.21.15.172:0/2834379320 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf74011070 con 0x7faf640030f0 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.376+0000 7faf79446640 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faf6806b2d0 0x7faf6806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.376+0000 7faf7bc4b640 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faf6806b2d0 0x7faf6806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:28.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.376+0000 7faf79446640 1 -- 172.21.15.172:0/2834379320 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7faf7408a5c0 con 0x7faf640030f0 2024-04-14T08:38:28.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.376+0000 7faf7e3b7f40 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5570648665e0 0x557064866a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.376+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/2834379320 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75) -- 0x557064866f80 con 0x5570648665e0 2024-04-14T08:38:28.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.377+0000 7faf7b44a640 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5570648665e0 0x557064866a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:28.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.377+0000 7faf7bc4b640 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faf6806b2d0 0x7faf6806d790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7faf70001030 tx=0x7faf7000d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:28.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.377+0000 7faf7b44a640 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5570648665e0 0x557064866a40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:28.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.378+0000 7faf7b44a640 1 -- 172.21.15.172:0/2834379320 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v75'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faf6c05e040 con 0x5570648665e0 2024-04-14T08:38:28.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.378+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5570648665e0 msgr2=0x557064866a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:28.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.378+0000 7faf7e3b7f40 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5570648665e0 0x557064866a40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.379+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faf6806b2d0 msgr2=0x7faf6806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:28.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.379+0000 7faf7e3b7f40 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faf6806b2d0 0x7faf6806d790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7faf70001030 tx=0x7faf7000d000 comp rx=0 tx=0).stop 2024-04-14T08:38:28.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.379+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 msgr2=0x5570647bda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:28.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.379+0000 7faf7e3b7f40 1 --2- 172.21.15.172:0/2834379320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf640030f0 0x5570647bda20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x557064742d20 tx=0x7faf7400c530 comp rx=0 tx=0).stop 2024-04-14T08:38:28.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.379+0000 7faf7c44c640 1 -- 172.21.15.172:0/2834379320 reap_dead start 2024-04-14T08:38:28.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.379+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/2834379320 shutdown_connections 2024-04-14T08:38:28.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.379+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/2834379320 >> 172.21.15.172:0/2834379320 conn(0x557064741ca0 msgr2=0x557064727350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:28.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.379+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/2834379320 shutdown_connections 2024-04-14T08:38:28.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.379+0000 7faf7e3b7f40 1 -- 172.21.15.172:0/2834379320 wait complete. 2024-04-14T08:38:28.383 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:38:28.383 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:38:28.436 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:38:28.491 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:38:28.491 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:38:28.492 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:38:28.492 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:28.507 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:38:28.507 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:28.563 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:38:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.703+0000 7f078ab55640 1 Processor -- start 2024-04-14T08:38:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.703+0000 7f078ab55640 1 -- start start 2024-04-14T08:38:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.704+0000 7f078ab55640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07840d2830 0x7f07840d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.704+0000 7f078ab55640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f078415e890 0x7f07840d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.704+0000 7f078ab55640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f07840cedd0 0x7f07841621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.704+0000 7f078ab55640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f07840cff30 con 0x7f07840cedd0 2024-04-14T08:38:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.704+0000 7f078ab55640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f07840d2350 con 0x7f078415e890 2024-04-14T08:38:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.704+0000 7f078ab55640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f07840d3150 con 0x7f07840d2830 2024-04-14T08:38:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.704+0000 7f07890f2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f07840cedd0 0x7f07841621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.704+0000 7f07890f2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f07840cedd0 0x7f07841621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41412/0 (socket says 172.21.15.172:41412) 2024-04-14T08:38:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.705+0000 7f07890f2640 1 -- 172.21.15.172:0/3389339247 learned_addr learned my addr 172.21.15.172:0/3389339247 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:28.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.705+0000 7f07888f1640 1 --2- 172.21.15.172:0/3389339247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07840d2830 0x7f07840d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:28.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.705+0000 7f0783fff640 1 --2- 172.21.15.172:0/3389339247 >> v2:172.21.15.177:3300/0 conn(0x7f078415e890 0x7f07840d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:28.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.705+0000 7f07890f2640 1 -- 172.21.15.172:0/3389339247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07840d2830 msgr2=0x7f07840d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:28.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.705+0000 7f07890f2640 1 --2- 172.21.15.172:0/3389339247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07840d2830 0x7f07840d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.705+0000 7f07890f2640 1 -- 172.21.15.172:0/3389339247 >> v2:172.21.15.177:3300/0 conn(0x7f078415e890 msgr2=0x7f07840d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:28.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.705+0000 7f07890f2640 1 --2- 172.21.15.172:0/3389339247 >> v2:172.21.15.177:3300/0 conn(0x7f078415e890 0x7f07840d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.705+0000 7f07890f2640 1 -- 172.21.15.172:0/3389339247 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07840cf320 con 0x7f07840cedd0 2024-04-14T08:38:28.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.706+0000 7f07890f2640 1 --2- 172.21.15.172:0/3389339247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07840cedd0 0x7f07841621a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f078409dcc0 tx=0x7f077c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1494ba360e48916 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:28.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.706+0000 7f07837fe640 1 -- 172.21.15.172:0/3389339247 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c02cb00 con 0x7f07840cedd0 2024-04-14T08:38:28.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.706+0000 7f07837fe640 1 -- 172.21.15.172:0/3389339247 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077c02cca0 con 0x7f07840cedd0 2024-04-14T08:38:28.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.706+0000 7f07837fe640 1 -- 172.21.15.172:0/3389339247 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c034610 con 0x7f07840cedd0 2024-04-14T08:38:28.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.707+0000 7f078ab55640 1 -- 172.21.15.172:0/3389339247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07840cedd0 msgr2=0x7f07841621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:28.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.707+0000 7f078ab55640 1 --2- 172.21.15.172:0/3389339247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07840cedd0 0x7f07841621a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f078409dcc0 tx=0x7f077c02bf60 comp rx=0 tx=0).stop 2024-04-14T08:38:28.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.707+0000 7f078ab55640 1 -- 172.21.15.172:0/3389339247 shutdown_connections 2024-04-14T08:38:28.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.707+0000 7f078ab55640 1 --2- 172.21.15.172:0/3389339247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07840cedd0 0x7f07841621a0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.707+0000 7f078ab55640 1 --2- 172.21.15.172:0/3389339247 >> v2:172.21.15.177:3300/0 conn(0x7f078415e890 0x7f07840d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.707+0000 7f078ab55640 1 --2- 172.21.15.172:0/3389339247 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07840d2830 0x7f07840d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.707+0000 7f078ab55640 1 -- 172.21.15.172:0/3389339247 >> 172.21.15.172:0/3389339247 conn(0x7f078409d590 msgr2=0x7f07840d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:28.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.707+0000 7f078ab55640 1 -- 172.21.15.172:0/3389339247 shutdown_connections 2024-04-14T08:38:28.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.707+0000 7f078ab55640 1 -- 172.21.15.172:0/3389339247 wait complete. 2024-04-14T08:38:28.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.708+0000 7f078ab55640 1 Processor -- start 2024-04-14T08:38:28.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.708+0000 7f078ab55640 1 -- start start 2024-04-14T08:38:28.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f078ab55640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07840cedd0 0x7f07840cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f078ab55640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07840d2830 0x7f07840ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f078ab55640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f078415e890 0x7f07840cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f078ab55640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f07840cfbc0 con 0x7f07840cedd0 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f078ab55640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f07840cf8e0 con 0x7f078415e890 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f078ab55640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f07840d1840 con 0x7f07840d2830 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f07888f1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07840cedd0 0x7f07840cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f07888f1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07840cedd0 0x7f07840cb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41428/0 (socket says 172.21.15.172:41428) 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f07888f1640 1 -- 172.21.15.172:0/3678346860 learned_addr learned my addr 172.21.15.172:0/3678346860 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f07888f1640 1 -- 172.21.15.172:0/3678346860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07840d2830 msgr2=0x7f07840ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f07888f1640 1 --2- 172.21.15.172:0/3678346860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07840d2830 0x7f07840ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f07888f1640 1 -- 172.21.15.172:0/3678346860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f078415e890 msgr2=0x7f07840cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f07888f1640 1 --2- 172.21.15.172:0/3678346860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f078415e890 0x7f07840cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f07888f1640 1 -- 172.21.15.172:0/3678346860 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07840cac60 con 0x7f07840cedd0 2024-04-14T08:38:28.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.709+0000 7f07888f1640 1 --2- 172.21.15.172:0/3678346860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07840cedd0 0x7f07840cb7e0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f077800e870 tx=0x7f077800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:28.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.710+0000 7f07817fa640 1 -- 172.21.15.172:0/3678346860 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778017070 con 0x7f07840cedd0 2024-04-14T08:38:28.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.710+0000 7f07817fa640 1 -- 172.21.15.172:0/3678346860 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077800c940 con 0x7f07840cedd0 2024-04-14T08:38:28.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.710+0000 7f078ab55640 1 -- 172.21.15.172:0/3678346860 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07840b71e0 con 0x7f07840cedd0 2024-04-14T08:38:28.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.710+0000 7f07817fa640 1 -- 172.21.15.172:0/3678346860 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077800cd30 con 0x7f07840cedd0 2024-04-14T08:38:28.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.710+0000 7f078ab55640 1 -- 172.21.15.172:0/3678346860 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07840b77a0 con 0x7f07840cedd0 2024-04-14T08:38:28.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.711+0000 7f078ab55640 1 -- 172.21.15.172:0/3678346860 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0784068520 con 0x7f07840cedd0 2024-04-14T08:38:28.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.711+0000 7f07817fa640 1 -- 172.21.15.172:0/3678346860 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f077801a070 con 0x7f07840cedd0 2024-04-14T08:38:28.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.711+0000 7f07817fa640 1 --2- 172.21.15.172:0/3678346860 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f076806c8f0 0x7f076806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:28.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.712+0000 7f07817fa640 1 -- 172.21.15.172:0/3678346860 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f077808bc00 con 0x7f07840cedd0 2024-04-14T08:38:28.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.712+0000 7f0783fff640 1 --2- 172.21.15.172:0/3678346860 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f076806c8f0 0x7f076806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:28.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.712+0000 7f0783fff640 1 --2- 172.21.15.172:0/3678346860 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f076806c8f0 0x7f076806edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f07840d3770 tx=0x7f0770009500 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:28.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.716+0000 7f07817fa640 1 -- 172.21.15.172:0/3678346860 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0778092080 con 0x7f07840cedd0 2024-04-14T08:38:28.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.901+0000 7f078ab55640 1 -- 172.21.15.172:0/3678346860 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07840d2610 con 0x7f07840cedd0 2024-04-14T08:38:28.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.903+0000 7f07817fa640 1 -- 172.21.15.172:0/3678346860 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+17136 (secure 0 0 0) 0x7f0778059e50 con 0x7f07840cedd0 2024-04-14T08:38:28.904 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:28.904 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":75,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:37:15.356496+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:36:59.342045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-14T08:37:00.059338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/2134832221":"2024-04-15T08:37:15.356472+0000","172.21.15.172:6835/2134832221":"2024-04-15T08:37:15.356472+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:38:28.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.905+0000 7f078ab55640 1 -- 172.21.15.172:0/3678346860 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f076806c8f0 msgr2=0x7f076806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:28.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.905+0000 7f078ab55640 1 --2- 172.21.15.172:0/3678346860 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f076806c8f0 0x7f076806edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f07840d3770 tx=0x7f0770009500 comp rx=0 tx=0).stop 2024-04-14T08:38:28.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.905+0000 7f078ab55640 1 -- 172.21.15.172:0/3678346860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07840cedd0 msgr2=0x7f07840cb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:28.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.905+0000 7f078ab55640 1 --2- 172.21.15.172:0/3678346860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07840cedd0 0x7f07840cb7e0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f077800e870 tx=0x7f077800ed30 comp rx=0 tx=0).stop 2024-04-14T08:38:28.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.905+0000 7f078ab55640 1 -- 172.21.15.172:0/3678346860 shutdown_connections 2024-04-14T08:38:28.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.905+0000 7f078ab55640 1 --2- 172.21.15.172:0/3678346860 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f076806c8f0 0x7f076806edb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.905+0000 7f078ab55640 1 --2- 172.21.15.172:0/3678346860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f078415e890 0x7f07840cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.905+0000 7f078ab55640 1 --2- 172.21.15.172:0/3678346860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f07840d2830 0x7f07840ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.905+0000 7f078ab55640 1 --2- 172.21.15.172:0/3678346860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07840cedd0 0x7f07840cb7e0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:28.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.905+0000 7f078ab55640 1 -- 172.21.15.172:0/3678346860 >> 172.21.15.172:0/3678346860 conn(0x7f078409d590 msgr2=0x7f078406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:28.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.905+0000 7f078ab55640 1 -- 172.21.15.172:0/3678346860 shutdown_connections 2024-04-14T08:38:28.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:28.905+0000 7f078ab55640 1 -- 172.21.15.172:0/3678346860 wait complete. 2024-04-14T08:38:28.916 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:36:59.342045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:37:00.059338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:38:28.917 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:38:29.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.024+0000 7f30ed9e0640 1 Processor -- start 2024-04-14T08:38:29.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.024+0000 7f30ed9e0640 1 -- start start 2024-04-14T08:38:29.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30ed9e0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e80ce550 0x7f30e806d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30ed9e0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f30e806d580 0x7f30e80d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30ed9e0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30e80cc780 con 0x7f30e815e3c0 2024-04-14T08:38:29.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30ed9e0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f30e806ce40 con 0x7f30e806d580 2024-04-14T08:38:29.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30ed9e0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f30e80cca60 con 0x7f30e80ce550 2024-04-14T08:38:29.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30e77fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f30e806d580 0x7f30e80d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30e6ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f30e815e3c0 0x7f30e806c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43826/0 (socket says 172.21.15.172:43826) 2024-04-14T08:38:29.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30e6ffd640 1 -- 172.21.15.172:0/238729884 learned_addr learned my addr 172.21.15.172:0/238729884 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:29.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30e67fc640 1 --2- 172.21.15.172:0/238729884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e80ce550 0x7f30e806d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30e77fe640 1 -- 172.21.15.172:0/238729884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e80ce550 msgr2=0x7f30e806d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:29.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30e77fe640 1 --2- 172.21.15.172:0/238729884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e80ce550 0x7f30e806d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30e77fe640 1 -- 172.21.15.172:0/238729884 >> v1:172.21.15.172:6789/0 conn(0x7f30e815e3c0 legacy=0x7f30e806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30e77fe640 1 -- 172.21.15.172:0/238729884 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e80ccd40 con 0x7f30e806d580 2024-04-14T08:38:29.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30e5ffb640 1 -- 172.21.15.172:0/238729884 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 297773095 0 0) 0x7f30e80cc780 con 0x7f30e815e3c0 2024-04-14T08:38:29.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.025+0000 7f30e67fc640 1 --2- 172.21.15.172:0/238729884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e80ce550 0x7f30e806d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:29.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.026+0000 7f30e77fe640 1 --2- 172.21.15.172:0/238729884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e806d580 0x7f30e80d0240 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f30d80097a0 tx=0x7f30d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414578a4ead25699 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:29.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.026+0000 7f30e5ffb640 1 -- 172.21.15.172:0/238729884 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8035020 con 0x7f30e806d580 2024-04-14T08:38:29.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.026+0000 7f30e5ffb640 1 -- 172.21.15.172:0/238729884 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d8031830 con 0x7f30e806d580 2024-04-14T08:38:29.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.026+0000 7f30e5ffb640 1 -- 172.21.15.172:0/238729884 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8031c00 con 0x7f30e806d580 2024-04-14T08:38:29.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.027+0000 7f30ed9e0640 1 -- 172.21.15.172:0/238729884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e806d580 msgr2=0x7f30e80d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.027+0000 7f30ed9e0640 1 --2- 172.21.15.172:0/238729884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e806d580 0x7f30e80d0240 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f30d80097a0 tx=0x7f30d802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.027+0000 7f30ed9e0640 1 -- 172.21.15.172:0/238729884 shutdown_connections 2024-04-14T08:38:29.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.027+0000 7f30ed9e0640 1 --2- 172.21.15.172:0/238729884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e80ce550 0x7f30e806d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.027+0000 7f30ed9e0640 1 --2- 172.21.15.172:0/238729884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e806d580 0x7f30e80d0240 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.027+0000 7f30ed9e0640 1 -- 172.21.15.172:0/238729884 >> 172.21.15.172:0/238729884 conn(0x7f30e80b3af0 msgr2=0x7f30e80b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:29.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.027+0000 7f30ed9e0640 1 -- 172.21.15.172:0/238729884 shutdown_connections 2024-04-14T08:38:29.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.027+0000 7f30ed9e0640 1 -- 172.21.15.172:0/238729884 wait complete. 2024-04-14T08:38:29.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.028+0000 7f30ed9e0640 1 Processor -- start 2024-04-14T08:38:29.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.028+0000 7f30ed9e0640 1 -- start start 2024-04-14T08:38:29.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30ed9e0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e806d580 0x7f30e81f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30ed9e0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e80ce550 0x7f30e80c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30ed9e0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e815e3c0 0x7f30e81f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30ed9e0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30e80d0780 con 0x7f30e815e3c0 2024-04-14T08:38:29.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30ed9e0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f30e815e7a0 con 0x7f30e80ce550 2024-04-14T08:38:29.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30ed9e0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f30e80d0900 con 0x7f30e806d580 2024-04-14T08:38:29.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e815e3c0 0x7f30e81f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e815e3c0 0x7f30e81f7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41442/0 (socket says 172.21.15.172:41442) 2024-04-14T08:38:29.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30e77fe640 1 -- 172.21.15.172:0/2923476204 learned_addr learned my addr 172.21.15.172:0/2923476204 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:29.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30e67fc640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e80ce550 0x7f30e80c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30e6ffd640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e806d580 0x7f30e81f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30e77fe640 1 -- 172.21.15.172:0/2923476204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e806d580 msgr2=0x7f30e81f0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30e77fe640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e806d580 0x7f30e81f0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30e77fe640 1 -- 172.21.15.172:0/2923476204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e80ce550 msgr2=0x7f30e80c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30e77fe640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e80ce550 0x7f30e80c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30e77fe640 1 -- 172.21.15.172:0/2923476204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e80cc450 con 0x7f30e815e3c0 2024-04-14T08:38:29.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.029+0000 7f30e67fc640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e80ce550 0x7f30e80c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:29.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.030+0000 7f30e77fe640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e815e3c0 0x7f30e81f7060 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f30d80097a0 tx=0x7f30d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:29.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.030+0000 7f30dffff640 1 -- 172.21.15.172:0/2923476204 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8035070 con 0x7f30e815e3c0 2024-04-14T08:38:29.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.030+0000 7f30dffff640 1 -- 172.21.15.172:0/2923476204 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d8004230 con 0x7f30e815e3c0 2024-04-14T08:38:29.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.030+0000 7f30dffff640 1 -- 172.21.15.172:0/2923476204 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d803a6f0 con 0x7f30e815e3c0 2024-04-14T08:38:29.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.030+0000 7f30ed9e0640 1 -- 172.21.15.172:0/2923476204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e80b5740 con 0x7f30e815e3c0 2024-04-14T08:38:29.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.030+0000 7f30ed9e0640 1 -- 172.21.15.172:0/2923476204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e80cbc50 con 0x7f30e815e3c0 2024-04-14T08:38:29.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.031+0000 7f30ed9e0640 1 -- 172.21.15.172:0/2923476204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e8068590 con 0x7f30e815e3c0 2024-04-14T08:38:29.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.031+0000 7f30dffff640 1 -- 172.21.15.172:0/2923476204 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30d8038020 con 0x7f30e815e3c0 2024-04-14T08:38:29.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.032+0000 7f30dffff640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30c806c8f0 0x7f30c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.032+0000 7f30dffff640 1 -- 172.21.15.172:0/2923476204 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f30d80ad9a0 con 0x7f30e815e3c0 2024-04-14T08:38:29.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.032+0000 7f30e6ffd640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30c806c8f0 0x7f30c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.032+0000 7f30e6ffd640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30c806c8f0 0x7f30c806edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f30e0007ed0 tx=0x7f30e0007e60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:29.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.037+0000 7f30dffff640 1 -- 172.21.15.172:0/2923476204 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f30d8040070 con 0x7f30e815e3c0 2024-04-14T08:38:29.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.232+0000 7f30ed9e0640 1 -- 172.21.15.172:0/2923476204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f30e80b09c0 con 0x7f30e815e3c0 2024-04-14T08:38:29.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.233+0000 7f30dffff640 1 -- 172.21.15.172:0/2923476204 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f30d807bc80 con 0x7f30e815e3c0 2024-04-14T08:38:29.233 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:38:29.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.235+0000 7f30ed9e0640 1 -- 172.21.15.172:0/2923476204 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30c806c8f0 msgr2=0x7f30c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.235+0000 7f30ed9e0640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30c806c8f0 0x7f30c806edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f30e0007ed0 tx=0x7f30e0007e60 comp rx=0 tx=0).stop 2024-04-14T08:38:29.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.235+0000 7f30ed9e0640 1 -- 172.21.15.172:0/2923476204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e815e3c0 msgr2=0x7f30e81f7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.235+0000 7f30ed9e0640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e815e3c0 0x7f30e81f7060 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f30d80097a0 tx=0x7f30d80046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.235+0000 7f30ed9e0640 1 -- 172.21.15.172:0/2923476204 shutdown_connections 2024-04-14T08:38:29.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.235+0000 7f30ed9e0640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30c806c8f0 0x7f30c806edb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.235+0000 7f30ed9e0640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e815e3c0 0x7f30e81f7060 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.235+0000 7f30ed9e0640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e80ce550 0x7f30e80c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.235+0000 7f30ed9e0640 1 --2- 172.21.15.172:0/2923476204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e806d580 0x7f30e81f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.235+0000 7f30ed9e0640 1 -- 172.21.15.172:0/2923476204 >> 172.21.15.172:0/2923476204 conn(0x7f30e80b3af0 msgr2=0x7f30e8074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:29.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.235+0000 7f30ed9e0640 1 -- 172.21.15.172:0/2923476204 shutdown_connections 2024-04-14T08:38:29.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.235+0000 7f30ed9e0640 1 -- 172.21.15.172:0/2923476204 wait complete. 2024-04-14T08:38:29.246 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:38:29.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.350+0000 7f5187fed640 1 Processor -- start 2024-04-14T08:38:29.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.350+0000 7f5187fed640 1 -- start start 2024-04-14T08:38:29.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.351+0000 7f5187fed640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f518015c420 0x7f51800b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.351+0000 7f5187fed640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f518006f3e0 0x7f51800d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.351+0000 7f5187fed640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51800d0550 con 0x7f51800ceb00 2024-04-14T08:38:29.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.351+0000 7f5187fed640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f51800d30a0 con 0x7f518015c420 2024-04-14T08:38:29.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.351+0000 7f5187fed640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f518006f870 con 0x7f518006f3e0 2024-04-14T08:38:29.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.351+0000 7f5185d89640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f518015c420 0x7f51800b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.351+0000 7f518658a640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f51800ceb00 0x7f51800cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43842/0 (socket says 172.21.15.172:43842) 2024-04-14T08:38:29.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.351+0000 7f518658a640 1 -- 172.21.15.172:0/951104468 learned_addr learned my addr 172.21.15.172:0/951104468 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:29.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.352+0000 7f5185588640 1 --2- 172.21.15.172:0/951104468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f518006f3e0 0x7f51800d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.352+0000 7f5185d89640 1 -- 172.21.15.172:0/951104468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f518006f3e0 msgr2=0x7f51800d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:29.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.352+0000 7f5185d89640 1 --2- 172.21.15.172:0/951104468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f518006f3e0 0x7f51800d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.352+0000 7f5185d89640 1 -- 172.21.15.172:0/951104468 >> v1:172.21.15.172:6789/0 conn(0x7f51800ceb00 legacy=0x7f51800cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.352+0000 7f5184d87640 1 -- 172.21.15.172:0/951104468 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4235340844 0 0) 0x7f51800d0550 con 0x7f51800ceb00 2024-04-14T08:38:29.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.352+0000 7f5185d89640 1 -- 172.21.15.172:0/951104468 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51800ccdd0 con 0x7f518015c420 2024-04-14T08:38:29.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.352+0000 7f5185d89640 1 --2- 172.21.15.172:0/951104468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015c420 0x7f51800b1510 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f517c0097a0 tx=0x7f517c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339105431d9d868f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:29.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.353+0000 7f5184d87640 1 -- 172.21.15.172:0/951104468 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c035020 con 0x7f518015c420 2024-04-14T08:38:29.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.353+0000 7f5184d87640 1 -- 172.21.15.172:0/951104468 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f517c031830 con 0x7f518015c420 2024-04-14T08:38:29.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.353+0000 7f5184d87640 1 -- 172.21.15.172:0/951104468 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c031c00 con 0x7f518015c420 2024-04-14T08:38:29.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.353+0000 7f5187fed640 1 -- 172.21.15.172:0/951104468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015c420 msgr2=0x7f51800b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.353+0000 7f5187fed640 1 --2- 172.21.15.172:0/951104468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015c420 0x7f51800b1510 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f517c0097a0 tx=0x7f517c02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.353+0000 7f5187fed640 1 -- 172.21.15.172:0/951104468 shutdown_connections 2024-04-14T08:38:29.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.353+0000 7f5187fed640 1 --2- 172.21.15.172:0/951104468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f518006f3e0 0x7f51800d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.353+0000 7f5187fed640 1 --2- 172.21.15.172:0/951104468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015c420 0x7f51800b1510 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.353+0000 7f5187fed640 1 -- 172.21.15.172:0/951104468 >> 172.21.15.172:0/951104468 conn(0x7f518008c000 msgr2=0x7f51800d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:29.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.353+0000 7f5187fed640 1 -- 172.21.15.172:0/951104468 shutdown_connections 2024-04-14T08:38:29.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.353+0000 7f5187fed640 1 -- 172.21.15.172:0/951104468 wait complete. 2024-04-14T08:38:29.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.354+0000 7f5187fed640 1 Processor -- start 2024-04-14T08:38:29.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.355+0000 7f5187fed640 1 -- start start 2024-04-14T08:38:29.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.355+0000 7f5187fed640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51800ceb00 0x7f5180159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.355+0000 7f5187fed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015c420 0x7f51800c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.355+0000 7f5187fed640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51800c91e0 0x7f51800d0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.355+0000 7f5187fed640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f51800cd070 con 0x7f518015c420 2024-04-14T08:38:29.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.355+0000 7f5187fed640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f51800d2dc0 con 0x7f51800ceb00 2024-04-14T08:38:29.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.355+0000 7f5187fed640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f51800b11e0 con 0x7f51800c91e0 2024-04-14T08:38:29.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.355+0000 7f5185588640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015c420 0x7f51800c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.355+0000 7f5185588640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015c420 0x7f51800c6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41454/0 (socket says 172.21.15.172:41454) 2024-04-14T08:38:29.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.355+0000 7f5185588640 1 -- 172.21.15.172:0/967750069 learned_addr learned my addr 172.21.15.172:0/967750069 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:29.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f5185d89640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51800ceb00 0x7f5180159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f518658a640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51800c91e0 0x7f51800d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f5185588640 1 -- 172.21.15.172:0/967750069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51800c91e0 msgr2=0x7f51800d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f5185588640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51800c91e0 0x7f51800d0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f5185588640 1 -- 172.21.15.172:0/967750069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51800ceb00 msgr2=0x7f5180159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f5185588640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51800ceb00 0x7f5180159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f5185588640 1 -- 172.21.15.172:0/967750069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51800c6670 con 0x7f518015c420 2024-04-14T08:38:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f5185d89640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51800ceb00 0x7f5180159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f5185588640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015c420 0x7f51800c6130 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f517000d860 tx=0x7f517000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f51767fc640 1 -- 172.21.15.172:0/967750069 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517000de50 con 0x7f518015c420 2024-04-14T08:38:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f51767fc640 1 -- 172.21.15.172:0/967750069 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5170019070 con 0x7f518015c420 2024-04-14T08:38:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f5187fed640 1 -- 172.21.15.172:0/967750069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51800c9730 con 0x7f518015c420 2024-04-14T08:38:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f51767fc640 1 -- 172.21.15.172:0/967750069 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517000baf0 con 0x7f518015c420 2024-04-14T08:38:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.356+0000 7f5187fed640 1 -- 172.21.15.172:0/967750069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51800c9cf0 con 0x7f518015c420 2024-04-14T08:38:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.357+0000 7f5187fed640 1 -- 172.21.15.172:0/967750069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51800ca4c0 con 0x7f518015c420 2024-04-14T08:38:29.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.358+0000 7f51767fc640 1 -- 172.21.15.172:0/967750069 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f517000bc90 con 0x7f518015c420 2024-04-14T08:38:29.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.358+0000 7f51767fc640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f516406c920 0x7f516406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.358+0000 7f5185d89640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f516406c920 0x7f516406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.358+0000 7f51767fc640 1 -- 172.21.15.172:0/967750069 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f517008c200 con 0x7f518015c420 2024-04-14T08:38:29.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.359+0000 7f5185d89640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f516406c920 0x7f516406ede0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f517c036000 tx=0x7f517c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:29.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.363+0000 7f51767fc640 1 -- 172.21.15.172:0/967750069 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f517005a3f0 con 0x7f518015c420 2024-04-14T08:38:29.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.544+0000 7f5187fed640 1 -- 172.21.15.172:0/967750069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5180075490 con 0x7f518015c420 2024-04-14T08:38:29.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.545+0000 7f51767fc640 1 -- 172.21.15.172:0/967750069 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) ==== 104+0+11 (secure 0 0 0) 0x7f5170059cf0 con 0x7f518015c420 2024-04-14T08:38:29.545 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:38:29.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.547+0000 7f5187fed640 1 -- 172.21.15.172:0/967750069 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f516406c920 msgr2=0x7f516406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.547+0000 7f5187fed640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f516406c920 0x7f516406ede0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f517c036000 tx=0x7f517c03c040 comp rx=0 tx=0).stop 2024-04-14T08:38:29.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.547+0000 7f5187fed640 1 -- 172.21.15.172:0/967750069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015c420 msgr2=0x7f51800c6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.547+0000 7f5187fed640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015c420 0x7f51800c6130 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f517000d860 tx=0x7f517000dd20 comp rx=0 tx=0).stop 2024-04-14T08:38:29.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.547+0000 7f5187fed640 1 -- 172.21.15.172:0/967750069 shutdown_connections 2024-04-14T08:38:29.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.547+0000 7f5187fed640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f516406c920 0x7f516406ede0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.547+0000 7f5187fed640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51800c91e0 0x7f51800d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.547+0000 7f5187fed640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518015c420 0x7f51800c6130 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.547+0000 7f5187fed640 1 --2- 172.21.15.172:0/967750069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51800ceb00 0x7f5180159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.547+0000 7f5187fed640 1 -- 172.21.15.172:0/967750069 >> 172.21.15.172:0/967750069 conn(0x7f518008c000 msgr2=0x7f51800b4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:29.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.547+0000 7f5187fed640 1 -- 172.21.15.172:0/967750069 shutdown_connections 2024-04-14T08:38:29.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.547+0000 7f5187fed640 1 -- 172.21.15.172:0/967750069 wait complete. 2024-04-14T08:38:29.558 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:38:29.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.662+0000 7f7fc82c8640 1 Processor -- start 2024-04-14T08:38:29.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.662+0000 7f7fc82c8640 1 -- start start 2024-04-14T08:38:29.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.663+0000 7f7fc82c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7fc0162ba0 0x7f7fc00d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.663+0000 7f7fc82c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fc00cefb0 con 0x7f7fc015e430 2024-04-14T08:38:29.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.663+0000 7f7fc82c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fc00cf290 con 0x7f7fc0162710 2024-04-14T08:38:29.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.663+0000 7f7fc82c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f7fc00cf570 con 0x7f7fc0162ba0 2024-04-14T08:38:29.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.663+0000 7f7fc6064640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7fc015e430 0x7f7fc0162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54566/0 (socket says 172.21.15.172:54566) 2024-04-14T08:38:29.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.663+0000 7f7fc6064640 1 -- 172.21.15.172:0/3998183164 learned_addr learned my addr 172.21.15.172:0/3998183164 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:29.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.663+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678437507 0 0) 0x7f7fc00cefb0 con 0x7f7fc015e430 2024-04-14T08:38:29.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.663+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fac003520 con 0x7f7fc015e430 2024-04-14T08:38:29.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.663+0000 7f7fc6865640 1 --2- 172.21.15.172:0/3998183164 >> v2:172.21.15.177:3300/0 conn(0x7f7fc0162ba0 0x7f7fc00d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.663+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1298266736 0 0) 0x7f7fac003520 con 0x7f7fc015e430 2024-04-14T08:38:29.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 >> v2:172.21.15.177:3300/0 conn(0x7f7fc0162ba0 msgr2=0x7f7fc00d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:29.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5062640 1 --2- 172.21.15.172:0/3998183164 >> v2:172.21.15.177:3300/0 conn(0x7f7fc0162ba0 0x7f7fc00d2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 >> v1:172.21.15.177:6789/0 conn(0x7f7fc0162710 legacy=0x7f7fc00d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc00cf850 con 0x7f7fc015e430 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f7fb0001670 con 0x7f7fc015e430 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 >> v1:172.21.15.172:6789/0 conn(0x7f7fc015e430 legacy=0x7f7fc0162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5062640 1 --2- 172.21.15.172:0/3998183164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fac0039b0 0x7f7fac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5062640 1 --2- 172.21.15.172:0/3998183164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fac004800 0x7f7fac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5062640 1 --2- 172.21.15.172:0/3998183164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fac0051c0 0x7f7fac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc00cf570 con 0x7f7fac004800 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac00c060 con 0x7f7fac0051c0 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7fac00c260 con 0x7f7fac0039b0 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5863640 1 --2- 172.21.15.172:0/3998183164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fac004800 0x7f7fac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc6064640 1 --2- 172.21.15.172:0/3998183164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fac0039b0 0x7f7fac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc6865640 1 --2- 172.21.15.172:0/3998183164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fac0051c0 0x7f7fac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5863640 1 -- 172.21.15.172:0/3998183164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fac0039b0 msgr2=0x7f7fac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:29.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5863640 1 --2- 172.21.15.172:0/3998183164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fac0039b0 0x7f7fac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5863640 1 -- 172.21.15.172:0/3998183164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fac0051c0 msgr2=0x7f7fac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5863640 1 --2- 172.21.15.172:0/3998183164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fac0051c0 0x7f7fac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc5863640 1 -- 172.21.15.172:0/3998183164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac003520 con 0x7f7fac004800 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc6865640 1 --2- 172.21.15.172:0/3998183164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fac0051c0 0x7f7fac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.664+0000 7f7fc6865640 1 -- 172.21.15.172:0/3998183164 reap_dead start 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.665+0000 7f7fc5863640 1 --2- 172.21.15.172:0/3998183164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fac004800 0x7f7fac004c80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc001750 tx=0x7f7fac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e253658c1915b8c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.665+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc001370 con 0x7f7fac004800 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.665+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fbc0052d0 con 0x7f7fac004800 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.665+0000 7f7fc5062640 1 -- 172.21.15.172:0/3998183164 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc003830 con 0x7f7fac004800 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.665+0000 7f7fc82c8640 1 -- 172.21.15.172:0/3998183164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fac004800 msgr2=0x7f7fac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.665+0000 7f7fc82c8640 1 --2- 172.21.15.172:0/3998183164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fac004800 0x7f7fac004c80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc001750 tx=0x7f7fac005830 comp rx=0 tx=0).stop 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.666+0000 7f7fc82c8640 1 -- 172.21.15.172:0/3998183164 shutdown_connections 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.666+0000 7f7fc82c8640 1 --2- 172.21.15.172:0/3998183164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fac004800 0x7f7fac004c80 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.666+0000 7f7fc82c8640 1 -- 172.21.15.172:0/3998183164 >> 172.21.15.172:0/3998183164 conn(0x7f7fc009d560 msgr2=0x7f7fc00d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:29.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.666+0000 7f7fc82c8640 1 -- 172.21.15.172:0/3998183164 shutdown_connections 2024-04-14T08:38:29.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.666+0000 7f7fc82c8640 1 -- 172.21.15.172:0/3998183164 wait complete. 2024-04-14T08:38:29.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.667+0000 7f7fc82c8640 1 Processor -- start 2024-04-14T08:38:29.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.667+0000 7f7fc82c8640 1 -- start start 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.667+0000 7f7fc82c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fac004800 0x7f7fc00b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.667+0000 7f7fc82c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc015fb90 0x7f7fc015fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.667+0000 7f7fc82c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fc0159270 0x7f7fc01596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.667+0000 7f7fc82c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc00d4e20 con 0x7f7fc0159270 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.667+0000 7f7fc82c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc015e810 con 0x7f7fac004800 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.667+0000 7f7fc82c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc00d0050 con 0x7f7fc015fb90 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc6865640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fc0159270 0x7f7fc01596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc6865640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fc0159270 0x7f7fc01596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35490/0 (socket says 172.21.15.172:35490) 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc6865640 1 -- 172.21.15.172:0/4210393694 learned_addr learned my addr 172.21.15.172:0/4210393694 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc6064640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fac004800 0x7f7fc00b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc5863640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc015fb90 0x7f7fc015fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc6865640 1 -- 172.21.15.172:0/4210393694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc015fb90 msgr2=0x7f7fc015fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc6865640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc015fb90 0x7f7fc015fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc6865640 1 -- 172.21.15.172:0/4210393694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fac004800 msgr2=0x7f7fc00b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc6865640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fac004800 0x7f7fc00b8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc6865640 1 -- 172.21.15.172:0/4210393694 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc00b5e50 con 0x7f7fc0159270 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc6865640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fc0159270 0x7f7fc01596f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00a2180 tx=0x7f7fb8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fb6ffd640 1 -- 172.21.15.172:0/4210393694 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb801a030 con 0x7f7fc0159270 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fb6ffd640 1 -- 172.21.15.172:0/4210393694 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb8010da0 con 0x7f7fc0159270 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fb6ffd640 1 -- 172.21.15.172:0/4210393694 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8003bd0 con 0x7f7fc0159270 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc82c8640 1 -- 172.21.15.172:0/4210393694 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc00b6130 con 0x7f7fc0159270 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.668+0000 7f7fc82c8640 1 -- 172.21.15.172:0/4210393694 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc00b66f0 con 0x7f7fc0159270 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.669+0000 7f7fc82c8640 1 -- 172.21.15.172:0/4210393694 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc00c7e10 con 0x7f7fc0159270 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.670+0000 7f7fb6ffd640 1 -- 172.21.15.172:0/4210393694 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fb801d070 con 0x7f7fc0159270 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.671+0000 7f7fb6ffd640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7fa406c8f0 0x7f7fa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.671+0000 7f7fb6ffd640 1 -- 172.21.15.172:0/4210393694 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f7fb808c1c0 con 0x7f7fc0159270 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.671+0000 7f7fc6064640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7fa406c8f0 0x7f7fa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.671+0000 7f7fc6064640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7fa406c8f0 0x7f7fa406edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0001050 tx=0x7f7fb0000f80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:29.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.675+0000 7f7fb6ffd640 1 -- 172.21.15.172:0/4210393694 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7fc00c7e10 con 0x7f7fc0159270 2024-04-14T08:38:29.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.858+0000 7f7fc82c8640 1 -- 172.21.15.172:0/4210393694 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7fc0068590 con 0x7f7fc0159270 2024-04-14T08:38:29.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.858+0000 7f7fb6ffd640 1 -- 172.21.15.172:0/4210393694 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) ==== 100+0+11 (secure 0 0 0) 0x7f7fb8059e50 con 0x7f7fc0159270 2024-04-14T08:38:29.859 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:38:29.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.860+0000 7f7fc82c8640 1 -- 172.21.15.172:0/4210393694 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7fa406c8f0 msgr2=0x7f7fa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.860+0000 7f7fc82c8640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7fa406c8f0 0x7f7fa406edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0001050 tx=0x7f7fb0000f80 comp rx=0 tx=0).stop 2024-04-14T08:38:29.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.860+0000 7f7fc82c8640 1 -- 172.21.15.172:0/4210393694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fc0159270 msgr2=0x7f7fc01596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.860+0000 7f7fc82c8640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fc0159270 0x7f7fc01596f0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00a2180 tx=0x7f7fb8010590 comp rx=0 tx=0).stop 2024-04-14T08:38:29.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.860+0000 7f7fc82c8640 1 -- 172.21.15.172:0/4210393694 shutdown_connections 2024-04-14T08:38:29.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.860+0000 7f7fc82c8640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7fa406c8f0 0x7f7fa406edb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.860+0000 7f7fc82c8640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fc0159270 0x7f7fc01596f0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.860+0000 7f7fc82c8640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc015fb90 0x7f7fc015fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.860+0000 7f7fc82c8640 1 --2- 172.21.15.172:0/4210393694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fac004800 0x7f7fc00b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.860+0000 7f7fc82c8640 1 -- 172.21.15.172:0/4210393694 >> 172.21.15.172:0/4210393694 conn(0x7f7fc009d560 msgr2=0x7f7fc00b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:29.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.861+0000 7f7fc82c8640 1 -- 172.21.15.172:0/4210393694 shutdown_connections 2024-04-14T08:38:29.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.861+0000 7f7fc82c8640 1 -- 172.21.15.172:0/4210393694 wait complete. 2024-04-14T08:38:29.875 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:38:29.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.978+0000 7f4099d1e640 1 Processor -- start 2024-04-14T08:38:29.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.978+0000 7f4099d1e640 1 -- start start 2024-04-14T08:38:29.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.978+0000 7f4099d1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f409415e430 0x7f4094162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.978+0000 7f4099d1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40940cefb0 con 0x7f4094162540 2024-04-14T08:38:29.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.978+0000 7f4099d1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40940cf290 con 0x7f40940d2e00 2024-04-14T08:38:29.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.978+0000 7f4099d1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f40940cf570 con 0x7f409415e430 2024-04-14T08:38:29.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.979+0000 7f40937fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f409415e430 0x7f4094162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.979+0000 7f40937fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f409415e430 0x7f4094162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35498/0 (socket says 172.21.15.172:35498) 2024-04-14T08:38:29.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.979+0000 7f40937fe640 1 -- 172.21.15.172:0/3293886736 learned_addr learned my addr 172.21.15.172:0/3293886736 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:29.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.979+0000 7f4092ffd640 1 --1- 172.21.15.172:0/3293886736 >> v1:172.21.15.172:6789/0 conn(0x7f4094162540 0x7f40940b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54580/0 (socket says 172.21.15.172:54580) 2024-04-14T08:38:29.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.979+0000 7f40937fe640 1 -- 172.21.15.172:0/3293886736 >> v1:172.21.15.177:6789/0 conn(0x7f40940d2e00 legacy=0x7f40940b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:29.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.979+0000 7f40937fe640 1 -- 172.21.15.172:0/3293886736 >> v1:172.21.15.172:6789/0 conn(0x7f4094162540 legacy=0x7f40940b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.979+0000 7f40937fe640 1 -- 172.21.15.172:0/3293886736 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40940cf850 con 0x7f409415e430 2024-04-14T08:38:29.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.980+0000 7f40937fe640 1 --2- 172.21.15.172:0/3293886736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f409415e430 0x7f4094162000 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f408c009870 tx=0x7f408c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc87e58ea9f83af server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:29.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.980+0000 7f40927fc640 1 -- 172.21.15.172:0/3293886736 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c037070 con 0x7f409415e430 2024-04-14T08:38:29.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.980+0000 7f40927fc640 1 -- 172.21.15.172:0/3293886736 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f408c004230 con 0x7f409415e430 2024-04-14T08:38:29.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.980+0000 7f40927fc640 1 -- 172.21.15.172:0/3293886736 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c003610 con 0x7f409415e430 2024-04-14T08:38:29.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.981+0000 7f4099d1e640 1 -- 172.21.15.172:0/3293886736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f409415e430 msgr2=0x7f4094162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.981+0000 7f4099d1e640 1 --2- 172.21.15.172:0/3293886736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f409415e430 0x7f4094162000 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f408c009870 tx=0x7f408c02b7e0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.981+0000 7f4099d1e640 1 -- 172.21.15.172:0/3293886736 shutdown_connections 2024-04-14T08:38:29.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.981+0000 7f4099d1e640 1 --2- 172.21.15.172:0/3293886736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f409415e430 0x7f4094162000 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.981+0000 7f4099d1e640 1 -- 172.21.15.172:0/3293886736 >> 172.21.15.172:0/3293886736 conn(0x7f409409d560 msgr2=0x7f40940d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:29.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.981+0000 7f4099d1e640 1 -- 172.21.15.172:0/3293886736 shutdown_connections 2024-04-14T08:38:29.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.981+0000 7f4099d1e640 1 -- 172.21.15.172:0/3293886736 wait complete. 2024-04-14T08:38:29.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.982+0000 7f4099d1e640 1 Processor -- start 2024-04-14T08:38:29.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.982+0000 7f4099d1e640 1 -- start start 2024-04-14T08:38:29.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4099d1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40940d2e00 0x7f409415e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4099d1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f409415e430 0x7f40940b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4099d1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094162540 0x7f4094158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4099d1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f40940d4e20 con 0x7f409415e430 2024-04-14T08:38:29.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4099d1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f409415e810 con 0x7f40940d2e00 2024-04-14T08:38:29.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4099d1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f40940b4850 con 0x7f4094162540 2024-04-14T08:38:29.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4092ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f409415e430 0x7f40940b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4092ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f409415e430 0x7f40940b8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35514/0 (socket says 172.21.15.172:35514) 2024-04-14T08:38:29.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4092ffd640 1 -- 172.21.15.172:0/3021090878 learned_addr learned my addr 172.21.15.172:0/3021090878 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:29.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f40937fe640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40940d2e00 0x7f409415e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4092ffd640 1 -- 172.21.15.172:0/3021090878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094162540 msgr2=0x7f4094158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4093fff640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094162540 0x7f4094158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4092ffd640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094162540 0x7f4094158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4092ffd640 1 -- 172.21.15.172:0/3021090878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40940d2e00 msgr2=0x7f409415e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:29.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4092ffd640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40940d2e00 0x7f409415e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:29.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4092ffd640 1 -- 172.21.15.172:0/3021090878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40940b9090 con 0x7f409415e430 2024-04-14T08:38:29.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f40937fe640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40940d2e00 0x7f409415e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:29.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4093fff640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094162540 0x7f4094158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:29.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4092ffd640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f409415e430 0x7f40940b8b50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f407c00c780 tx=0x7f407c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:29.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f408bfff640 1 -- 172.21.15.172:0/3021090878 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c018070 con 0x7f409415e430 2024-04-14T08:38:29.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f408bfff640 1 -- 172.21.15.172:0/3021090878 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f407c0072d0 con 0x7f409415e430 2024-04-14T08:38:29.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f4099d1e640 1 -- 172.21.15.172:0/3021090878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4094159390 con 0x7f409415e430 2024-04-14T08:38:29.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.983+0000 7f408bfff640 1 -- 172.21.15.172:0/3021090878 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c0047e0 con 0x7f409415e430 2024-04-14T08:38:29.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.984+0000 7f4099d1e640 1 -- 172.21.15.172:0/3021090878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40940b8070 con 0x7f409415e430 2024-04-14T08:38:29.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.985+0000 7f4099d1e640 1 -- 172.21.15.172:0/3021090878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4094068590 con 0x7f409415e430 2024-04-14T08:38:29.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.985+0000 7f408bfff640 1 -- 172.21.15.172:0/3021090878 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f407c004980 con 0x7f409415e430 2024-04-14T08:38:29.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.986+0000 7f408bfff640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f407406c8f0 0x7f407406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:29.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.986+0000 7f408bfff640 1 -- 172.21.15.172:0/3021090878 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f407c08b220 con 0x7f409415e430 2024-04-14T08:38:29.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.986+0000 7f40937fe640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f407406c8f0 0x7f407406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:29.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.986+0000 7f40937fe640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f407406c8f0 0x7f407406edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f408c034000 tx=0x7f408c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:29.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:29.994+0000 7f408bfff640 1 -- 172.21.15.172:0/3021090878 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4094159390 con 0x7f409415e430 2024-04-14T08:38:30.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.184+0000 7f4099d1e640 1 -- 172.21.15.172:0/3021090878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f40940d18a0 con 0x7f409415e430 2024-04-14T08:38:30.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.184+0000 7f408bfff640 1 -- 172.21.15.172:0/3021090878 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v75) ==== 80+0+124 (secure 0 0 0) 0x7f40940b9090 con 0x7f409415e430 2024-04-14T08:38:30.185 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6837/2134832221 2024-04-15T08:37:15.356472+0000 2024-04-14T08:38:30.185 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6835/2134832221 2024-04-15T08:37:15.356472+0000 2024-04-14T08:38:30.185 INFO:teuthology.orchestra.run.smithi172.stderr:listed 2 entries 2024-04-14T08:38:30.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.186+0000 7f4099d1e640 1 -- 172.21.15.172:0/3021090878 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f407406c8f0 msgr2=0x7f407406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:30.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.186+0000 7f4099d1e640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f407406c8f0 0x7f407406edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f408c034000 tx=0x7f408c008030 comp rx=0 tx=0).stop 2024-04-14T08:38:30.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.186+0000 7f4099d1e640 1 -- 172.21.15.172:0/3021090878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f409415e430 msgr2=0x7f40940b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:30.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.186+0000 7f4099d1e640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f409415e430 0x7f40940b8b50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f407c00c780 tx=0x7f407c00cc40 comp rx=0 tx=0).stop 2024-04-14T08:38:30.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.186+0000 7f4099d1e640 1 -- 172.21.15.172:0/3021090878 shutdown_connections 2024-04-14T08:38:30.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.186+0000 7f4099d1e640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f407406c8f0 0x7f407406edb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.186+0000 7f4099d1e640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094162540 0x7f4094158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.186+0000 7f4099d1e640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f409415e430 0x7f40940b8b50 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.186+0000 7f4099d1e640 1 --2- 172.21.15.172:0/3021090878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40940d2e00 0x7f409415e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.186+0000 7f4099d1e640 1 -- 172.21.15.172:0/3021090878 >> 172.21.15.172:0/3021090878 conn(0x7f409409d560 msgr2=0x7f409415c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:30.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.186+0000 7f4099d1e640 1 -- 172.21.15.172:0/3021090878 shutdown_connections 2024-04-14T08:38:30.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.186+0000 7f4099d1e640 1 -- 172.21.15.172:0/3021090878 wait complete. 2024-04-14T08:38:30.198 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:38:30.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:30.198 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:38:30.228 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:38:30.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:30.228 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:38:30.284 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:38:30.284 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:38:30.284 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:30.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:38:30.304 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:38:30.304 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:38:30.304 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:30.339 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:38:30.339 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:38:30.339 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:38:30.339 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:30.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:38:30.362 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:38:30.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.479+0000 7fa19f908640 1 Processor -- start 2024-04-14T08:38:30.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.480+0000 7fa19f908640 1 -- start start 2024-04-14T08:38:30.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.480+0000 7fa19f908640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa1980d2830 0x7fa1980d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.480+0000 7fa19f908640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1980cedd0 0x7fa1980b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.480+0000 7fa19f908640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1980d3150 con 0x7fa19815e890 2024-04-14T08:38:30.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.480+0000 7fa19f908640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa1980cff30 con 0x7fa1980d2830 2024-04-14T08:38:30.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.480+0000 7fa19f908640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1980d2350 con 0x7fa1980cedd0 2024-04-14T08:38:30.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.480+0000 7fa19d6a4640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa1980d2830 0x7fa1980d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.481+0000 7fa19d6a4640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa1980d2830 0x7fa1980d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35524/0 (socket says 172.21.15.172:35524) 2024-04-14T08:38:30.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.481+0000 7fa19d6a4640 1 -- 172.21.15.172:0/65030669 learned_addr learned my addr 172.21.15.172:0/65030669 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:30.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.481+0000 7fa19dea5640 1 --2- 172.21.15.172:0/65030669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1980cedd0 0x7fa1980b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.481+0000 7fa19d6a4640 1 -- 172.21.15.172:0/65030669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1980cedd0 msgr2=0x7fa1980b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:30.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.481+0000 7fa19d6a4640 1 --2- 172.21.15.172:0/65030669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1980cedd0 0x7fa1980b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.481+0000 7fa19d6a4640 1 -- 172.21.15.172:0/65030669 >> v1:172.21.15.177:6789/0 conn(0x7fa19815e890 legacy=0x7fa1980d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:30.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.481+0000 7fa19d6a4640 1 -- 172.21.15.172:0/65030669 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1980cf320 con 0x7fa1980d2830 2024-04-14T08:38:30.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.481+0000 7fa19d6a4640 1 --2- 172.21.15.172:0/65030669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1980d2830 0x7fa1980d2c10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa188009870 tx=0x7fa18802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18301e5d1d72b858 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:30.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.482+0000 7fa18ffff640 1 -- 172.21.15.172:0/65030669 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188034070 con 0x7fa1980d2830 2024-04-14T08:38:30.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.482+0000 7fa18ffff640 1 -- 172.21.15.172:0/65030669 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa188030840 con 0x7fa1980d2830 2024-04-14T08:38:30.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.482+0000 7fa18ffff640 1 -- 172.21.15.172:0/65030669 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188030c10 con 0x7fa1980d2830 2024-04-14T08:38:30.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.482+0000 7fa19f908640 1 -- 172.21.15.172:0/65030669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1980d2830 msgr2=0x7fa1980d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:30.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.482+0000 7fa19f908640 1 --2- 172.21.15.172:0/65030669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1980d2830 0x7fa1980d2c10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa188009870 tx=0x7fa18802b980 comp rx=0 tx=0).stop 2024-04-14T08:38:30.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.482+0000 7fa19f908640 1 -- 172.21.15.172:0/65030669 shutdown_connections 2024-04-14T08:38:30.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.482+0000 7fa19f908640 1 --2- 172.21.15.172:0/65030669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1980cedd0 0x7fa1980b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.482+0000 7fa19f908640 1 --2- 172.21.15.172:0/65030669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1980d2830 0x7fa1980d2c10 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.482+0000 7fa19f908640 1 -- 172.21.15.172:0/65030669 >> 172.21.15.172:0/65030669 conn(0x7fa19809d590 msgr2=0x7fa1980d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:30.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.482+0000 7fa19f908640 1 -- 172.21.15.172:0/65030669 shutdown_connections 2024-04-14T08:38:30.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.482+0000 7fa19f908640 1 -- 172.21.15.172:0/65030669 wait complete. 2024-04-14T08:38:30.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.484+0000 7fa19f908640 1 Processor -- start 2024-04-14T08:38:30.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.484+0000 7fa19f908640 1 -- start start 2024-04-14T08:38:30.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.485+0000 7fa19f908640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1980cedd0 0x7fa1980c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.485+0000 7fa19f908640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1980d2830 0x7fa1980b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.485+0000 7fa19f908640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa19815e890 0x7fa19814b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.485+0000 7fa19f908640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1980cfbc0 con 0x7fa1980cedd0 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.485+0000 7fa19f908640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1980cf8e0 con 0x7fa1980d2830 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.485+0000 7fa19f908640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1980b3c50 con 0x7fa19815e890 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.485+0000 7fa19d6a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1980cedd0 0x7fa1980c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.485+0000 7fa19d6a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1980cedd0 0x7fa1980c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35530/0 (socket says 172.21.15.172:35530) 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.485+0000 7fa19d6a4640 1 -- 172.21.15.172:0/1770121273 learned_addr learned my addr 172.21.15.172:0/1770121273 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa19cea3640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1980d2830 0x7fa1980b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa19dea5640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa19815e890 0x7fa19814b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa19d6a4640 1 -- 172.21.15.172:0/1770121273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa19815e890 msgr2=0x7fa19814b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa19d6a4640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa19815e890 0x7fa19814b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa19d6a4640 1 -- 172.21.15.172:0/1770121273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1980d2830 msgr2=0x7fa1980b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa19d6a4640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1980d2830 0x7fa1980b7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa19d6a4640 1 -- 172.21.15.172:0/1770121273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1980c83d0 con 0x7fa1980cedd0 2024-04-14T08:38:30.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa19cea3640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1980d2830 0x7fa1980b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:30.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa19d6a4640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1980cedd0 0x7fa1980c7d80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa188009870 tx=0x7fa188004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:30.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa18dffb640 1 -- 172.21.15.172:0/1770121273 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188048070 con 0x7fa1980cedd0 2024-04-14T08:38:30.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa18dffb640 1 -- 172.21.15.172:0/1770121273 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa188030640 con 0x7fa1980cedd0 2024-04-14T08:38:30.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa19f908640 1 -- 172.21.15.172:0/1770121273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1980c86b0 con 0x7fa1980cedd0 2024-04-14T08:38:30.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa18dffb640 1 -- 172.21.15.172:0/1770121273 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188034040 con 0x7fa1980cedd0 2024-04-14T08:38:30.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.486+0000 7fa19f908640 1 -- 172.21.15.172:0/1770121273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1980c8f50 con 0x7fa1980cedd0 2024-04-14T08:38:30.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.487+0000 7fa19f908640 1 -- 172.21.15.172:0/1770121273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1980c8880 con 0x7fa1980cedd0 2024-04-14T08:38:30.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.488+0000 7fa18dffb640 1 -- 172.21.15.172:0/1770121273 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa18804b070 con 0x7fa1980cedd0 2024-04-14T08:38:30.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.488+0000 7fa18dffb640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa17c06c8f0 0x7fa17c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.488+0000 7fa18dffb640 1 -- 172.21.15.172:0/1770121273 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fa1880adcb0 con 0x7fa1980cedd0 2024-04-14T08:38:30.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.488+0000 7fa19cea3640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa17c06c8f0 0x7fa17c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.489+0000 7fa19cea3640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa17c06c8f0 0x7fa17c06edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa19815ee40 tx=0x7fa194008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.493+0000 7fa18dffb640 1 -- 172.21.15.172:0/1770121273 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa18807bf00 con 0x7fa1980cedd0 2024-04-14T08:38:30.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.712+0000 7fa19f908640 1 -- 172.21.15.172:0/1770121273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa198068520 con 0x7fa1980cedd0 2024-04-14T08:38:30.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.713+0000 7fa18dffb640 1 -- 172.21.15.172:0/1770121273 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5194 (secure 0 0 0) 0x7fa18807b800 con 0x7fa1980cedd0 2024-04-14T08:38:30.713 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:30.714 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:37:01.115031+0000","modified":"2024-04-14T08:38:02.376656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":15,"addr":"172.21.15.172:6837/2920402198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2920402198},{"type":"v1","addr":"172.21.15.172:6837","nonce":2920402198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":5}]} 2024-04-14T08:38:30.714 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 47 2024-04-14T08:38:30.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.715+0000 7fa19f908640 1 -- 172.21.15.172:0/1770121273 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa17c06c8f0 msgr2=0x7fa17c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:30.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.715+0000 7fa19f908640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa17c06c8f0 0x7fa17c06edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa19815ee40 tx=0x7fa194008040 comp rx=0 tx=0).stop 2024-04-14T08:38:30.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.715+0000 7fa19f908640 1 -- 172.21.15.172:0/1770121273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1980cedd0 msgr2=0x7fa1980c7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:30.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.715+0000 7fa19f908640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1980cedd0 0x7fa1980c7d80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa188009870 tx=0x7fa188004730 comp rx=0 tx=0).stop 2024-04-14T08:38:30.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.715+0000 7fa19f908640 1 -- 172.21.15.172:0/1770121273 shutdown_connections 2024-04-14T08:38:30.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.715+0000 7fa19f908640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa17c06c8f0 0x7fa17c06edb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.715+0000 7fa19f908640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa19815e890 0x7fa19814b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.715+0000 7fa19f908640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1980d2830 0x7fa1980b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.715+0000 7fa19f908640 1 --2- 172.21.15.172:0/1770121273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1980cedd0 0x7fa1980c7d80 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.715+0000 7fa19f908640 1 -- 172.21.15.172:0/1770121273 >> 172.21.15.172:0/1770121273 conn(0x7fa19809d590 msgr2=0x7fa1980d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:30.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.715+0000 7fa19f908640 1 -- 172.21.15.172:0/1770121273 shutdown_connections 2024-04-14T08:38:30.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.715+0000 7fa19f908640 1 -- 172.21.15.172:0/1770121273 wait complete. 2024-04-14T08:38:30.726 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:38:30.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.828+0000 7f5782c88640 1 Processor -- start 2024-04-14T08:38:30.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.828+0000 7f5782c88640 1 -- start start 2024-04-14T08:38:30.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f5782c88640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f577c15c710 0x7f577c0d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f5782c88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0cde10 0x7f577c0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f5782c88640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f577c0aef60 con 0x7f577c0ced80 2024-04-14T08:38:30.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f5782c88640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f577c0aed10 con 0x7f577c15c710 2024-04-14T08:38:30.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f5782c88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f577c0af890 con 0x7f577c0cde10 2024-04-14T08:38:30.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f5781225640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f577c0ced80 0x7f577c0ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54590/0 (socket says 172.21.15.172:54590) 2024-04-14T08:38:30.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f5781225640 1 -- 172.21.15.172:0/425334058 learned_addr learned my addr 172.21.15.172:0/425334058 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:30.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f5780a24640 1 --2- 172.21.15.172:0/425334058 >> v2:172.21.15.177:3300/0 conn(0x7f577c15c710 0x7f577c0d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f577bfff640 1 --2- 172.21.15.172:0/425334058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0cde10 0x7f577c0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 347523849 0 0) 0x7f577c0aef60 con 0x7f577c0ced80 2024-04-14T08:38:30.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5764003520 con 0x7f577c0ced80 2024-04-14T08:38:30.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2716405415 0 0) 0x7f5764003520 con 0x7f577c0ced80 2024-04-14T08:38:30.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0cde10 msgr2=0x7f577c0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:30.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f577b7fe640 1 --2- 172.21.15.172:0/425334058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0cde10 0x7f577c0ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 >> v2:172.21.15.177:3300/0 conn(0x7f577c15c710 msgr2=0x7f577c0d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:30.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f577b7fe640 1 --2- 172.21.15.172:0/425334058 >> v2:172.21.15.177:3300/0 conn(0x7f577c15c710 0x7f577c0d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.829+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c0af240 con 0x7f577c0ced80 2024-04-14T08:38:30.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f57740019f0 con 0x7f577c0ced80 2024-04-14T08:38:30.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 >> v1:172.21.15.172:6789/0 conn(0x7f577c0ced80 legacy=0x7f577c0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f577b7fe640 1 --2- 172.21.15.172:0/425334058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57640039b0 0x7f5764003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f577b7fe640 1 --2- 172.21.15.172:0/425334058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0cde10 0x7f5764004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f577b7fe640 1 --2- 172.21.15.172:0/425334058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5764004d40 0x7f57640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f577c0aed10 con 0x7f57640039b0 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f577c0af890 con 0x7f5764004d40 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f576400b9c0 con 0x7f577c0cde10 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f5780a24640 1 --2- 172.21.15.172:0/425334058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57640039b0 0x7f5764003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f577bfff640 1 --2- 172.21.15.172:0/425334058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0cde10 0x7f5764004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f5781225640 1 --2- 172.21.15.172:0/425334058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5764004d40 0x7f57640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f5780a24640 1 -- 172.21.15.172:0/425334058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0cde10 msgr2=0x7f5764004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f5780a24640 1 --2- 172.21.15.172:0/425334058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0cde10 0x7f5764004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f5780a24640 1 -- 172.21.15.172:0/425334058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5764004d40 msgr2=0x7f57640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f5780a24640 1 --2- 172.21.15.172:0/425334058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5764004d40 0x7f57640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.830+0000 7f5780a24640 1 -- 172.21.15.172:0/425334058 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5764003520 con 0x7f57640039b0 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.831+0000 7f5780a24640 1 --2- 172.21.15.172:0/425334058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57640039b0 0x7f5764003dd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f576c009ac0 tx=0x7f5764005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81954064d51a5b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.831+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c039070 con 0x7f57640039b0 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.831+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f576c02c710 con 0x7f57640039b0 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.831+0000 7f577b7fe640 1 -- 172.21.15.172:0/425334058 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c009690 con 0x7f57640039b0 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.831+0000 7f5782c88640 1 -- 172.21.15.172:0/425334058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57640039b0 msgr2=0x7f5764003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.831+0000 7f5782c88640 1 --2- 172.21.15.172:0/425334058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57640039b0 0x7f5764003dd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f576c009ac0 tx=0x7f5764005830 comp rx=0 tx=0).stop 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.831+0000 7f5781225640 1 -- 172.21.15.172:0/425334058 reap_dead start 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.831+0000 7f5782c88640 1 -- 172.21.15.172:0/425334058 shutdown_connections 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.831+0000 7f5782c88640 1 -- 172.21.15.172:0/425334058 >> 172.21.15.172:0/425334058 conn(0x7f577c08c000 msgr2=0x7f577c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.831+0000 7f5782c88640 1 -- 172.21.15.172:0/425334058 shutdown_connections 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.831+0000 7f5782c88640 1 -- 172.21.15.172:0/425334058 wait complete. 2024-04-14T08:38:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.833+0000 7f5782c88640 1 Processor -- start 2024-04-14T08:38:30.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.833+0000 7f5782c88640 1 -- start start 2024-04-14T08:38:30.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.833+0000 7f5782c88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57640039b0 0x7f577c0b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.833+0000 7f5782c88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0ca160 0x7f577c157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.833+0000 7f5782c88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f577c0b3ac0 0x7f577c0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.833+0000 7f5782c88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f577c0d0e90 con 0x7f577c0b3ac0 2024-04-14T08:38:30.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.833+0000 7f5782c88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f577c15caf0 con 0x7f57640039b0 2024-04-14T08:38:30.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.833+0000 7f5782c88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f577c0d1010 con 0x7f577c0ca160 2024-04-14T08:38:30.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f5781225640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f577c0b3ac0 0x7f577c0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f5781225640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f577c0b3ac0 0x7f577c0b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35550/0 (socket says 172.21.15.172:35550) 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f5781225640 1 -- 172.21.15.172:0/863243380 learned_addr learned my addr 172.21.15.172:0/863243380 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f5780a24640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57640039b0 0x7f577c0b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f577bfff640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0ca160 0x7f577c157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f5781225640 1 -- 172.21.15.172:0/863243380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0ca160 msgr2=0x7f577c157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f5781225640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0ca160 0x7f577c157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f5781225640 1 -- 172.21.15.172:0/863243380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57640039b0 msgr2=0x7f577c0b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f5781225640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57640039b0 0x7f577c0b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f5781225640 1 -- 172.21.15.172:0/863243380 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c0ca9b0 con 0x7f577c0b3ac0 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f5781225640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f577c0b3ac0 0x7f577c0b3f40 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f5774002070 tx=0x7f577400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f57797fa640 1 -- 172.21.15.172:0/863243380 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577400ec10 con 0x7f577c0b3ac0 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f57797fa640 1 -- 172.21.15.172:0/863243380 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5774018070 con 0x7f577c0b3ac0 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f57797fa640 1 -- 172.21.15.172:0/863243380 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774013420 con 0x7f577c0b3ac0 2024-04-14T08:38:30.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f5782c88640 1 -- 172.21.15.172:0/863243380 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577c0cac90 con 0x7f577c0b3ac0 2024-04-14T08:38:30.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.834+0000 7f5782c88640 1 -- 172.21.15.172:0/863243380 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577c06c7d0 con 0x7f577c0b3ac0 2024-04-14T08:38:30.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.836+0000 7f57797fa640 1 -- 172.21.15.172:0/863243380 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57740135c0 con 0x7f577c0b3ac0 2024-04-14T08:38:30.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.836+0000 7f5782c88640 1 -- 172.21.15.172:0/863243380 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f577c0cb1b0 con 0x7f577c0b3ac0 2024-04-14T08:38:30.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.836+0000 7f57797fa640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f576806c920 0x7f576806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:30.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.836+0000 7f5780a24640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f576806c920 0x7f576806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:30.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.836+0000 7f57797fa640 1 -- 172.21.15.172:0/863243380 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f577408bec0 con 0x7f577c0b3ac0 2024-04-14T08:38:30.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.837+0000 7f5780a24640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f576806c920 0x7f576806ede0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f576c036000 tx=0x7f576c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:30.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:30.841+0000 7f57797fa640 1 -- 172.21.15.172:0/863243380 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f577405a1a0 con 0x7f577c0b3ac0 2024-04-14T08:38:31.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.024+0000 7f5782c88640 1 -- 172.21.15.172:0/863243380 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f577c075490 con 0x7f577c0b3ac0 2024-04-14T08:38:31.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.026+0000 7f57797fa640 1 -- 172.21.15.172:0/863243380 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+17136 (secure 0 0 0) 0x7f5774059aa0 con 0x7f577c0b3ac0 2024-04-14T08:38:31.027 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:31.027 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":75,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:37:15.356496+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:36:59.342045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-14T08:37:00.059338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/2134832221":"2024-04-15T08:37:15.356472+0000","172.21.15.172:6835/2134832221":"2024-04-15T08:37:15.356472+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:38:31.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.028+0000 7f5782c88640 1 -- 172.21.15.172:0/863243380 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f576806c920 msgr2=0x7f576806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.028+0000 7f5782c88640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f576806c920 0x7f576806ede0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f576c036000 tx=0x7f576c030000 comp rx=0 tx=0).stop 2024-04-14T08:38:31.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.028+0000 7f5782c88640 1 -- 172.21.15.172:0/863243380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f577c0b3ac0 msgr2=0x7f577c0b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.028+0000 7f5782c88640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f577c0b3ac0 0x7f577c0b3f40 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f5774002070 tx=0x7f577400e530 comp rx=0 tx=0).stop 2024-04-14T08:38:31.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.028+0000 7f5782c88640 1 -- 172.21.15.172:0/863243380 shutdown_connections 2024-04-14T08:38:31.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.028+0000 7f5782c88640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f576806c920 0x7f576806ede0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.028+0000 7f5782c88640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f577c0b3ac0 0x7f577c0b3f40 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.028+0000 7f5782c88640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f577c0ca160 0x7f577c157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.028+0000 7f5782c88640 1 --2- 172.21.15.172:0/863243380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57640039b0 0x7f577c0b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.028+0000 7f5782c88640 1 -- 172.21.15.172:0/863243380 >> 172.21.15.172:0/863243380 conn(0x7f577c08c000 msgr2=0x7f577c0b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:31.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.028+0000 7f5782c88640 1 -- 172.21.15.172:0/863243380 shutdown_connections 2024-04-14T08:38:31.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.028+0000 7f5782c88640 1 -- 172.21.15.172:0/863243380 wait complete. 2024-04-14T08:38:31.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:36:59.342045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:37:00.059338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:38:31.042 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:38:31.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.147+0000 7f230d82d640 1 Processor -- start 2024-04-14T08:38:31.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.147+0000 7f230d82d640 1 -- start start 2024-04-14T08:38:31.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f230d82d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2308071250 0x7f230806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f230d82d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f230806d000 0x7f230806d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f230d82d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23080701b0 con 0x7f23080d1a20 2024-04-14T08:38:31.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f230d82d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f230806d9d0 con 0x7f2308071250 2024-04-14T08:38:31.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f230d82d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2308070dc0 con 0x7f230806d000 2024-04-14T08:38:31.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f2306ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f23080d1a20 0x7f23080cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54604/0 (socket says 172.21.15.172:54604) 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f2306ffd640 1 -- 172.21.15.172:0/1632043595 learned_addr learned my addr 172.21.15.172:0/1632043595 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f23077fe640 1 --2- 172.21.15.172:0/1632043595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f230806d000 0x7f230806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f23067fc640 1 --2- 172.21.15.172:0/1632043595 >> v2:172.21.15.177:3300/0 conn(0x7f2308071250 0x7f230806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f2305ffb640 1 -- 172.21.15.172:0/1632043595 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614633048 0 0) 0x7f23080701b0 con 0x7f23080d1a20 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f2305ffb640 1 -- 172.21.15.172:0/1632043595 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22f0003520 con 0x7f23080d1a20 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f23077fe640 1 -- 172.21.15.172:0/1632043595 >> v2:172.21.15.177:3300/0 conn(0x7f2308071250 msgr2=0x7f230806fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f23077fe640 1 --2- 172.21.15.172:0/1632043595 >> v2:172.21.15.177:3300/0 conn(0x7f2308071250 0x7f230806fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f23077fe640 1 -- 172.21.15.172:0/1632043595 >> v1:172.21.15.172:6789/0 conn(0x7f23080d1a20 legacy=0x7f23080cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.148+0000 7f23077fe640 1 -- 172.21.15.172:0/1632043595 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2308070490 con 0x7f230806d000 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.149+0000 7f23077fe640 1 --2- 172.21.15.172:0/1632043595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f230806d000 0x7f230806d490 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2300009d90 tx=0x7f230002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbcc5586f8d82f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.149+0000 7f2305ffb640 1 -- 172.21.15.172:0/1632043595 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300039070 con 0x7f230806d000 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.149+0000 7f2305ffb640 1 -- 172.21.15.172:0/1632043595 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230002cbc0 con 0x7f230806d000 2024-04-14T08:38:31.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.149+0000 7f2305ffb640 1 -- 172.21.15.172:0/1632043595 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300034440 con 0x7f230806d000 2024-04-14T08:38:31.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.150+0000 7f230d82d640 1 -- 172.21.15.172:0/1632043595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f230806d000 msgr2=0x7f230806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.150+0000 7f230d82d640 1 --2- 172.21.15.172:0/1632043595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f230806d000 0x7f230806d490 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2300009d90 tx=0x7f230002be10 comp rx=0 tx=0).stop 2024-04-14T08:38:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.150+0000 7f230d82d640 1 -- 172.21.15.172:0/1632043595 shutdown_connections 2024-04-14T08:38:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.150+0000 7f230d82d640 1 --2- 172.21.15.172:0/1632043595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f230806d000 0x7f230806d490 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.150+0000 7f230d82d640 1 --2- 172.21.15.172:0/1632043595 >> v2:172.21.15.177:3300/0 conn(0x7f2308071250 0x7f230806fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.150+0000 7f230d82d640 1 -- 172.21.15.172:0/1632043595 >> 172.21.15.172:0/1632043595 conn(0x7f230808c000 msgr2=0x7f23080d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.150+0000 7f230d82d640 1 -- 172.21.15.172:0/1632043595 shutdown_connections 2024-04-14T08:38:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.150+0000 7f230d82d640 1 -- 172.21.15.172:0/1632043595 wait complete. 2024-04-14T08:38:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.152+0000 7f230d82d640 1 Processor -- start 2024-04-14T08:38:31.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.152+0000 7f230d82d640 1 -- start start 2024-04-14T08:38:31.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.153+0000 7f230d82d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230806d000 0x7f23080b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.153+0000 7f230d82d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2308071250 0x7f23080b7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.153+0000 7f230d82d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23080d1a20 0x7f23081535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f230d82d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23080d1e00 con 0x7f230806d000 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f230d82d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2308070a50 con 0x7f23080d1a20 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f230d82d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2308160e30 con 0x7f2308071250 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f2306ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230806d000 0x7f23080b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f2306ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230806d000 0x7f23080b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35554/0 (socket says 172.21.15.172:35554) 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f2306ffd640 1 -- 172.21.15.172:0/1986647164 learned_addr learned my addr 172.21.15.172:0/1986647164 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f23067fc640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2308071250 0x7f23080b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f23077fe640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23080d1a20 0x7f23081535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f2306ffd640 1 -- 172.21.15.172:0/1986647164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2308071250 msgr2=0x7f23080b7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f2306ffd640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2308071250 0x7f23080b7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f2306ffd640 1 -- 172.21.15.172:0/1986647164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23080d1a20 msgr2=0x7f23081535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f2306ffd640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23080d1a20 0x7f23081535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f2306ffd640 1 -- 172.21.15.172:0/1986647164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2308153ae0 con 0x7f230806d000 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f2306ffd640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230806d000 0x7f23080b0b40 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f22f800c390 tx=0x7f22f800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f22effff640 1 -- 172.21.15.172:0/1986647164 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8017070 con 0x7f230806d000 2024-04-14T08:38:31.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f22effff640 1 -- 172.21.15.172:0/1986647164 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f8005ce0 con 0x7f230806d000 2024-04-14T08:38:31.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f22effff640 1 -- 172.21.15.172:0/1986647164 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f80072f0 con 0x7f230806d000 2024-04-14T08:38:31.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f230d82d640 1 -- 172.21.15.172:0/1986647164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23080b4f80 con 0x7f230806d000 2024-04-14T08:38:31.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.154+0000 7f230d82d640 1 -- 172.21.15.172:0/1986647164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23080b5540 con 0x7f230806d000 2024-04-14T08:38:31.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.155+0000 7f230d82d640 1 -- 172.21.15.172:0/1986647164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2308075490 con 0x7f230806d000 2024-04-14T08:38:31.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.156+0000 7f22effff640 1 -- 172.21.15.172:0/1986647164 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22f801a070 con 0x7f230806d000 2024-04-14T08:38:31.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.156+0000 7f22effff640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f22f406c8f0 0x7f22f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.156+0000 7f22effff640 1 -- 172.21.15.172:0/1986647164 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f22f808b070 con 0x7f230806d000 2024-04-14T08:38:31.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.157+0000 7f23067fc640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f22f406c8f0 0x7f22f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.157+0000 7f23067fc640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f22f406c8f0 0x7f22f406edb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2308161440 tx=0x7f22fc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:31.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.161+0000 7f22effff640 1 -- 172.21.15.172:0/1986647164 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f22f8058c70 con 0x7f230806d000 2024-04-14T08:38:31.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.342+0000 7f230d82d640 1 -- 172.21.15.172:0/1986647164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23080d2820 con 0x7f230806d000 2024-04-14T08:38:31.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.342+0000 7f22effff640 1 -- 172.21.15.172:0/1986647164 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f22f805cf70 con 0x7f230806d000 2024-04-14T08:38:31.343 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:38:31.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.344+0000 7f230d82d640 1 -- 172.21.15.172:0/1986647164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f22f406c8f0 msgr2=0x7f22f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.344+0000 7f230d82d640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f22f406c8f0 0x7f22f406edb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2308161440 tx=0x7f22fc009190 comp rx=0 tx=0).stop 2024-04-14T08:38:31.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.344+0000 7f230d82d640 1 -- 172.21.15.172:0/1986647164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230806d000 msgr2=0x7f23080b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.344+0000 7f230d82d640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230806d000 0x7f23080b0b40 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f22f800c390 tx=0x7f22f800c850 comp rx=0 tx=0).stop 2024-04-14T08:38:31.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.344+0000 7f230d82d640 1 -- 172.21.15.172:0/1986647164 shutdown_connections 2024-04-14T08:38:31.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.344+0000 7f230d82d640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f22f406c8f0 0x7f22f406edb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.344+0000 7f230d82d640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23080d1a20 0x7f23081535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.344+0000 7f230d82d640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2308071250 0x7f23080b7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.344+0000 7f230d82d640 1 --2- 172.21.15.172:0/1986647164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f230806d000 0x7f23080b0b40 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.344+0000 7f230d82d640 1 -- 172.21.15.172:0/1986647164 >> 172.21.15.172:0/1986647164 conn(0x7f230808c000 msgr2=0x7f23080b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:31.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.344+0000 7f230d82d640 1 -- 172.21.15.172:0/1986647164 shutdown_connections 2024-04-14T08:38:31.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.344+0000 7f230d82d640 1 -- 172.21.15.172:0/1986647164 wait complete. 2024-04-14T08:38:31.356 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:38:31.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.461+0000 7fd1221a3640 1 Processor -- start 2024-04-14T08:38:31.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.461+0000 7fd1221a3640 1 -- start start 2024-04-14T08:38:31.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.462+0000 7fd1221a3640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd11c162710 0x7fd11c0d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.462+0000 7fd1221a3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c162ba0 0x7fd11c0d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.462+0000 7fd1221a3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd11c0cefb0 con 0x7fd11c15e430 2024-04-14T08:38:31.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.462+0000 7fd1221a3640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fd11c0cf290 con 0x7fd11c162710 2024-04-14T08:38:31.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.462+0000 7fd1221a3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd11c0cf570 con 0x7fd11c162ba0 2024-04-14T08:38:31.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.462+0000 7fd11b7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd11c15e430 0x7fd11c162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54616/0 (socket says 172.21.15.172:54616) 2024-04-14T08:38:31.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.462+0000 7fd11b7fe640 1 -- 172.21.15.172:0/1149173294 learned_addr learned my addr 172.21.15.172:0/1149173294 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:31.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037402750 0 0) 0x7fd11c0cefb0 con 0x7fd11c15e430 2024-04-14T08:38:31.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd104003520 con 0x7fd11c15e430 2024-04-14T08:38:31.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11affd640 1 --2- 172.21.15.172:0/1149173294 >> v2:172.21.15.177:3300/0 conn(0x7fd11c162710 0x7fd11c0d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11bfff640 1 --2- 172.21.15.172:0/1149173294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c162ba0 0x7fd11c0d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4117717681 0 0) 0x7fd104003520 con 0x7fd11c15e430 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c162ba0 msgr2=0x7fd11c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 --2- 172.21.15.172:0/1149173294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c162ba0 0x7fd11c0d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 >> v2:172.21.15.177:3300/0 conn(0x7fd11c162710 msgr2=0x7fd11c0d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 --2- 172.21.15.172:0/1149173294 >> v2:172.21.15.177:3300/0 conn(0x7fd11c162710 0x7fd11c0d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11c0cf850 con 0x7fd11c15e430 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fd110001690 con 0x7fd11c15e430 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 >> v1:172.21.15.172:6789/0 conn(0x7fd11c15e430 legacy=0x7fd11c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 --2- 172.21.15.172:0/1149173294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1040039b0 0x7fd104003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 --2- 172.21.15.172:0/1149173294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c162ba0 0x7fd104004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 --2- 172.21.15.172:0/1149173294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd104004d40 0x7fd1040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c0cf290 con 0x7fd1040039b0 2024-04-14T08:38:31.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c0cf570 con 0x7fd104004d40 2024-04-14T08:38:31.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd10400b9c0 con 0x7fd11c162ba0 2024-04-14T08:38:31.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11b7fe640 1 --2- 172.21.15.172:0/1149173294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1040039b0 0x7fd104003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11bfff640 1 --2- 172.21.15.172:0/1149173294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd104004d40 0x7fd1040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.463+0000 7fd11affd640 1 --2- 172.21.15.172:0/1149173294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c162ba0 0x7fd104004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.464+0000 7fd11b7fe640 1 -- 172.21.15.172:0/1149173294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c162ba0 msgr2=0x7fd104004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:31.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.464+0000 7fd11b7fe640 1 --2- 172.21.15.172:0/1149173294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c162ba0 0x7fd104004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.464+0000 7fd11b7fe640 1 -- 172.21.15.172:0/1149173294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd104004d40 msgr2=0x7fd1040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:31.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.464+0000 7fd11b7fe640 1 --2- 172.21.15.172:0/1149173294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd104004d40 0x7fd1040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.464+0000 7fd11b7fe640 1 -- 172.21.15.172:0/1149173294 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd104003520 con 0x7fd1040039b0 2024-04-14T08:38:31.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.464+0000 7fd11b7fe640 1 --2- 172.21.15.172:0/1149173294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1040039b0 0x7fd104003dd0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fd11000c210 tx=0x7fd104005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d4e04806516ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:31.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.464+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110007d60 con 0x7fd1040039b0 2024-04-14T08:38:31.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.464+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd110030070 con 0x7fd1040039b0 2024-04-14T08:38:31.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.464+0000 7fd11a7fc640 1 -- 172.21.15.172:0/1149173294 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11002cd40 con 0x7fd1040039b0 2024-04-14T08:38:31.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.465+0000 7fd1221a3640 1 -- 172.21.15.172:0/1149173294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1040039b0 msgr2=0x7fd104003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.465+0000 7fd1221a3640 1 --2- 172.21.15.172:0/1149173294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1040039b0 0x7fd104003dd0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fd11000c210 tx=0x7fd104005830 comp rx=0 tx=0).stop 2024-04-14T08:38:31.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.465+0000 7fd11bfff640 1 -- 172.21.15.172:0/1149173294 reap_dead start 2024-04-14T08:38:31.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.465+0000 7fd1221a3640 1 -- 172.21.15.172:0/1149173294 shutdown_connections 2024-04-14T08:38:31.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.465+0000 7fd1221a3640 1 -- 172.21.15.172:0/1149173294 >> 172.21.15.172:0/1149173294 conn(0x7fd11c09d560 msgr2=0x7fd11c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:31.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.465+0000 7fd1221a3640 1 -- 172.21.15.172:0/1149173294 shutdown_connections 2024-04-14T08:38:31.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.465+0000 7fd1221a3640 1 -- 172.21.15.172:0/1149173294 wait complete. 2024-04-14T08:38:31.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.466+0000 7fd1221a3640 1 Processor -- start 2024-04-14T08:38:31.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.466+0000 7fd1221a3640 1 -- start start 2024-04-14T08:38:31.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd1221a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1040039b0 0x7fd11c1f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd1221a3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd11c158a90 0x7fd11c158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd1221a3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c0ccb20 0x7fd11c0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd1221a3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c0d4e20 con 0x7fd1040039b0 2024-04-14T08:38:31.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd1221a3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c15e810 con 0x7fd11c158a90 2024-04-14T08:38:31.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd1221a3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd11c0d0050 con 0x7fd11c0ccb20 2024-04-14T08:38:31.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd11b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1040039b0 0x7fd11c1f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd11b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1040039b0 0x7fd11c1f75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35560/0 (socket says 172.21.15.172:35560) 2024-04-14T08:38:31.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd11b7fe640 1 -- 172.21.15.172:0/3729573110 learned_addr learned my addr 172.21.15.172:0/3729573110 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:31.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd11affd640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd11c158a90 0x7fd11c158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd11bfff640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c0ccb20 0x7fd11c0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd11b7fe640 1 -- 172.21.15.172:0/3729573110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c0ccb20 msgr2=0x7fd11c0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd11b7fe640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c0ccb20 0x7fd11c0ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd11b7fe640 1 -- 172.21.15.172:0/3729573110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd11c158a90 msgr2=0x7fd11c158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd11b7fe640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd11c158a90 0x7fd11c158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd11b7fe640 1 -- 172.21.15.172:0/3729573110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11c0cc340 con 0x7fd1040039b0 2024-04-14T08:38:31.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.467+0000 7fd11bfff640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c0ccb20 0x7fd11c0ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:31.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.468+0000 7fd11b7fe640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1040039b0 0x7fd11c1f75b0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd11002c550 tx=0x7fd11c0cf180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:31.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.468+0000 7fd103fff640 1 -- 172.21.15.172:0/3729573110 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110045070 con 0x7fd1040039b0 2024-04-14T08:38:31.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.468+0000 7fd103fff640 1 -- 172.21.15.172:0/3729573110 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd110004c10 con 0x7fd1040039b0 2024-04-14T08:38:31.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.468+0000 7fd1221a3640 1 -- 172.21.15.172:0/3729573110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11c159830 con 0x7fd1040039b0 2024-04-14T08:38:31.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.468+0000 7fd103fff640 1 -- 172.21.15.172:0/3729573110 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110001ca0 con 0x7fd1040039b0 2024-04-14T08:38:31.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.468+0000 7fd1221a3640 1 -- 172.21.15.172:0/3729573110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11c159df0 con 0x7fd1040039b0 2024-04-14T08:38:31.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.469+0000 7fd1221a3640 1 -- 172.21.15.172:0/3729573110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd11c068590 con 0x7fd1040039b0 2024-04-14T08:38:31.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.469+0000 7fd103fff640 1 -- 172.21.15.172:0/3729573110 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd110007070 con 0x7fd1040039b0 2024-04-14T08:38:31.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.470+0000 7fd103fff640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd10806c8f0 0x7fd10806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.470+0000 7fd103fff640 1 -- 172.21.15.172:0/3729573110 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fd1100ace20 con 0x7fd1040039b0 2024-04-14T08:38:31.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.470+0000 7fd11affd640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd10806c8f0 0x7fd10806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.470+0000 7fd11affd640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd10806c8f0 0x7fd10806edb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd11c158680 tx=0x7fd10c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:31.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.474+0000 7fd103fff640 1 -- 172.21.15.172:0/3729573110 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd11007b150 con 0x7fd1040039b0 2024-04-14T08:38:31.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.658+0000 7fd1221a3640 1 -- 172.21.15.172:0/3729573110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd11c0d18a0 con 0x7fd1040039b0 2024-04-14T08:38:31.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.659+0000 7fd103fff640 1 -- 172.21.15.172:0/3729573110 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) ==== 104+0+11 (secure 0 0 0) 0x7fd11007aa50 con 0x7fd1040039b0 2024-04-14T08:38:31.659 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:38:31.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.661+0000 7fd1221a3640 1 -- 172.21.15.172:0/3729573110 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd10806c8f0 msgr2=0x7fd10806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.661+0000 7fd1221a3640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd10806c8f0 0x7fd10806edb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd11c158680 tx=0x7fd10c008040 comp rx=0 tx=0).stop 2024-04-14T08:38:31.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.661+0000 7fd1221a3640 1 -- 172.21.15.172:0/3729573110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1040039b0 msgr2=0x7fd11c1f75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.661+0000 7fd1221a3640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1040039b0 0x7fd11c1f75b0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd11002c550 tx=0x7fd11c0cf180 comp rx=0 tx=0).stop 2024-04-14T08:38:31.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.661+0000 7fd1221a3640 1 -- 172.21.15.172:0/3729573110 shutdown_connections 2024-04-14T08:38:31.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.661+0000 7fd1221a3640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd10806c8f0 0x7fd10806edb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.661+0000 7fd1221a3640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd11c0ccb20 0x7fd11c0ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.661+0000 7fd1221a3640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd11c158a90 0x7fd11c158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.661+0000 7fd1221a3640 1 --2- 172.21.15.172:0/3729573110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1040039b0 0x7fd11c1f75b0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.661+0000 7fd1221a3640 1 -- 172.21.15.172:0/3729573110 >> 172.21.15.172:0/3729573110 conn(0x7fd11c09d560 msgr2=0x7fd11c06c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:31.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.661+0000 7fd1221a3640 1 -- 172.21.15.172:0/3729573110 shutdown_connections 2024-04-14T08:38:31.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.661+0000 7fd1221a3640 1 -- 172.21.15.172:0/3729573110 wait complete. 2024-04-14T08:38:31.672 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:38:31.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.779+0000 7f6bec7dc640 1 Processor -- start 2024-04-14T08:38:31.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.779+0000 7f6bec7dc640 1 -- start start 2024-04-14T08:38:31.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.780+0000 7f6bec7dc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6be40b23e0 0x7f6be4140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.780+0000 7f6bec7dc640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be415b620 con 0x7f6be413df30 2024-04-14T08:38:31.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.780+0000 7f6bec7dc640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be41414a0 con 0x7f6be415c380 2024-04-14T08:38:31.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.780+0000 7f6bec7dc640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f6be40cf9b0 con 0x7f6be40b23e0 2024-04-14T08:38:31.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.781+0000 7f6be9d77640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6be40b23e0 0x7f6be4140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.781+0000 7f6bea578640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6be413df30 0x7f6be40b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54620/0 (socket says 172.21.15.172:54620) 2024-04-14T08:38:31.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.781+0000 7f6bea578640 1 -- 172.21.15.172:0/3661638533 learned_addr learned my addr 172.21.15.172:0/3661638533 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:31.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.781+0000 7f6be9d77640 1 -- 172.21.15.172:0/3661638533 >> v1:172.21.15.177:6789/0 conn(0x7f6be415c380 legacy=0x7f6be4140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:31.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.781+0000 7f6be9d77640 1 -- 172.21.15.172:0/3661638533 >> v1:172.21.15.172:6789/0 conn(0x7f6be413df30 legacy=0x7f6be40b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.781+0000 7f6be9576640 1 -- 172.21.15.172:0/3661638533 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678264611 0 0) 0x7f6bd4001110 con 0x7f6be413df30 2024-04-14T08:38:31.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.781+0000 7f6be9d77640 1 -- 172.21.15.172:0/3661638533 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be415b8a0 con 0x7f6be40b23e0 2024-04-14T08:38:31.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.782+0000 7f6be9d77640 1 --2- 172.21.15.172:0/3661638533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be40b23e0 0x7f6be4140770 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6be000ba40 tx=0x7f6be002e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b8e84c9c54afb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:31.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.782+0000 7f6be9576640 1 -- 172.21.15.172:0/3661638533 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be003b070 con 0x7f6be40b23e0 2024-04-14T08:38:31.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.782+0000 7f6be9576640 1 -- 172.21.15.172:0/3661638533 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be002ee50 con 0x7f6be40b23e0 2024-04-14T08:38:31.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.782+0000 7f6be9576640 1 -- 172.21.15.172:0/3661638533 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0036660 con 0x7f6be40b23e0 2024-04-14T08:38:31.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.782+0000 7f6bec7dc640 1 -- 172.21.15.172:0/3661638533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be40b23e0 msgr2=0x7f6be4140770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.782+0000 7f6bec7dc640 1 --2- 172.21.15.172:0/3661638533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be40b23e0 0x7f6be4140770 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6be000ba40 tx=0x7f6be002e0a0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.783+0000 7f6bec7dc640 1 -- 172.21.15.172:0/3661638533 shutdown_connections 2024-04-14T08:38:31.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.783+0000 7f6bec7dc640 1 --2- 172.21.15.172:0/3661638533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be40b23e0 0x7f6be4140770 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.783+0000 7f6bec7dc640 1 -- 172.21.15.172:0/3661638533 >> 172.21.15.172:0/3661638533 conn(0x7f6be406b520 msgr2=0x7f6be406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:31.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.783+0000 7f6bec7dc640 1 -- 172.21.15.172:0/3661638533 shutdown_connections 2024-04-14T08:38:31.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.783+0000 7f6bec7dc640 1 -- 172.21.15.172:0/3661638533 wait complete. 2024-04-14T08:38:31.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.784+0000 7f6bec7dc640 1 Processor -- start 2024-04-14T08:38:31.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.784+0000 7f6bec7dc640 1 -- start start 2024-04-14T08:38:31.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.784+0000 7f6bec7dc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6be40b23e0 0x7f6be41f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.784+0000 7f6bec7dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be413df30 0x7f6be413fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.784+0000 7f6bec7dc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6be415c380 0x7f6be415d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.784+0000 7f6bec7dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6be413e310 con 0x7f6be413df30 2024-04-14T08:38:31.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.784+0000 7f6bec7dc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6be415be60 con 0x7f6be40b23e0 2024-04-14T08:38:31.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.784+0000 7f6bec7dc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6be40712d0 con 0x7f6be415c380 2024-04-14T08:38:31.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6be9d77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be413df30 0x7f6be413fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6be9d77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be413df30 0x7f6be413fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35576/0 (socket says 172.21.15.172:35576) 2024-04-14T08:38:31.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6be9d77640 1 -- 172.21.15.172:0/333147683 learned_addr learned my addr 172.21.15.172:0/333147683 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:31.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6bea578640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6be40b23e0 0x7f6be41f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6bead79640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6be415c380 0x7f6be415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6be9d77640 1 -- 172.21.15.172:0/333147683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6be415c380 msgr2=0x7f6be415d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6be9d77640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6be415c380 0x7f6be415d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6be9d77640 1 -- 172.21.15.172:0/333147683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6be40b23e0 msgr2=0x7f6be41f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6be9d77640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6be40b23e0 0x7f6be41f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6be9d77640 1 -- 172.21.15.172:0/333147683 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be40ae540 con 0x7f6be413df30 2024-04-14T08:38:31.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6bead79640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6be415c380 0x7f6be415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:31.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6be9d77640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be413df30 0x7f6be413fc60 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6be0000c00 tx=0x7f6be0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:31.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6bdaffd640 1 -- 172.21.15.172:0/333147683 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0047070 con 0x7f6be413df30 2024-04-14T08:38:31.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6bdaffd640 1 -- 172.21.15.172:0/333147683 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be003c040 con 0x7f6be413df30 2024-04-14T08:38:31.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6bec7dc640 1 -- 172.21.15.172:0/333147683 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be40ae820 con 0x7f6be413df30 2024-04-14T08:38:31.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6bdaffd640 1 -- 172.21.15.172:0/333147683 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be003b050 con 0x7f6be413df30 2024-04-14T08:38:31.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.785+0000 7f6bec7dc640 1 -- 172.21.15.172:0/333147683 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be414ece0 con 0x7f6be413df30 2024-04-14T08:38:31.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.786+0000 7f6bec7dc640 1 -- 172.21.15.172:0/333147683 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be4068590 con 0x7f6be413df30 2024-04-14T08:38:31.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.787+0000 7f6bdaffd640 1 -- 172.21.15.172:0/333147683 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6be004a070 con 0x7f6be413df30 2024-04-14T08:38:31.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.787+0000 7f6bdaffd640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6bc806c920 0x7f6bc806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:31.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.787+0000 7f6bea578640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6bc806c920 0x7f6bc806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:31.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.787+0000 7f6bdaffd640 1 -- 172.21.15.172:0/333147683 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f6be00adc20 con 0x7f6be413df30 2024-04-14T08:38:31.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.788+0000 7f6bea578640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6bc806c920 0x7f6bc806ede0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4000f60 tx=0x7f6bd4007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:31.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.792+0000 7f6bdaffd640 1 -- 172.21.15.172:0/333147683 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6be007b820 con 0x7f6be413df30 2024-04-14T08:38:31.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.970+0000 7f6bec7dc640 1 -- 172.21.15.172:0/333147683 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6be40b1950 con 0x7f6be413df30 2024-04-14T08:38:31.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.971+0000 7f6bdaffd640 1 -- 172.21.15.172:0/333147683 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) ==== 100+0+11 (secure 0 0 0) 0x7f6be007fb20 con 0x7f6be413df30 2024-04-14T08:38:31.972 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:38:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.973+0000 7f6bec7dc640 1 -- 172.21.15.172:0/333147683 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6bc806c920 msgr2=0x7f6bc806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.973+0000 7f6bec7dc640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6bc806c920 0x7f6bc806ede0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4000f60 tx=0x7f6bd4007b50 comp rx=0 tx=0).stop 2024-04-14T08:38:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.973+0000 7f6bec7dc640 1 -- 172.21.15.172:0/333147683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be413df30 msgr2=0x7f6be413fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.973+0000 7f6bec7dc640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be413df30 0x7f6be413fc60 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6be0000c00 tx=0x7f6be0004730 comp rx=0 tx=0).stop 2024-04-14T08:38:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.973+0000 7f6bec7dc640 1 -- 172.21.15.172:0/333147683 shutdown_connections 2024-04-14T08:38:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.973+0000 7f6bec7dc640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6bc806c920 0x7f6bc806ede0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.973+0000 7f6bec7dc640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6be415c380 0x7f6be415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.973+0000 7f6bec7dc640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6be413df30 0x7f6be413fc60 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.973+0000 7f6bec7dc640 1 --2- 172.21.15.172:0/333147683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6be40b23e0 0x7f6be41f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.973+0000 7f6bec7dc640 1 -- 172.21.15.172:0/333147683 >> 172.21.15.172:0/333147683 conn(0x7f6be406b520 msgr2=0x7f6be4071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.973+0000 7f6bec7dc640 1 -- 172.21.15.172:0/333147683 shutdown_connections 2024-04-14T08:38:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:31.973+0000 7f6bec7dc640 1 -- 172.21.15.172:0/333147683 wait complete. 2024-04-14T08:38:31.985 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:38:32.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.087+0000 7f99f4c36640 1 Processor -- start 2024-04-14T08:38:32.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.087+0000 7f99f4c36640 1 -- start start 2024-04-14T08:38:32.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.088+0000 7f99f4c36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99f00d2830 0x7f99f00d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.088+0000 7f99f4c36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f99f015e890 0x7f99f00d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.088+0000 7f99f4c36640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f00d3150 con 0x7f99f00cedd0 2024-04-14T08:38:32.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.088+0000 7f99f4c36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f99f00cff30 con 0x7f99f015e890 2024-04-14T08:38:32.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.088+0000 7f99f4c36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f99f00d2350 con 0x7f99f00d2830 2024-04-14T08:38:32.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.088+0000 7f99eed9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f99f00cedd0 0x7f99f01621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54622/0 (socket says 172.21.15.172:54622) 2024-04-14T08:38:32.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.088+0000 7f99eed9d640 1 -- 172.21.15.172:0/965754909 learned_addr learned my addr 172.21.15.172:0/965754909 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:32.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.088+0000 7f99ee59c640 1 --2- 172.21.15.172:0/965754909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99f00d2830 0x7f99f00d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.088+0000 7f99edd9b640 1 --2- 172.21.15.172:0/965754909 >> v2:172.21.15.177:3300/0 conn(0x7f99f015e890 0x7f99f00d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.088+0000 7f99ed59a640 1 -- 172.21.15.172:0/965754909 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858750662 0 0) 0x7f99f00d3150 con 0x7f99f00cedd0 2024-04-14T08:38:32.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.089+0000 7f99ed59a640 1 -- 172.21.15.172:0/965754909 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99d8003520 con 0x7f99f00cedd0 2024-04-14T08:38:32.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.089+0000 7f99ed59a640 1 -- 172.21.15.172:0/965754909 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 720584258 0 0) 0x7f99d8003520 con 0x7f99f00cedd0 2024-04-14T08:38:32.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.089+0000 7f99edd9b640 1 -- 172.21.15.172:0/965754909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99f00d2830 msgr2=0x7f99f00d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:32.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.089+0000 7f99edd9b640 1 --2- 172.21.15.172:0/965754909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99f00d2830 0x7f99f00d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.089+0000 7f99edd9b640 1 -- 172.21.15.172:0/965754909 >> v1:172.21.15.172:6789/0 conn(0x7f99f00cedd0 legacy=0x7f99f01621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.089+0000 7f99edd9b640 1 -- 172.21.15.172:0/965754909 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f00cf320 con 0x7f99f015e890 2024-04-14T08:38:32.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.089+0000 7f99edd9b640 1 --2- 172.21.15.172:0/965754909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99f015e890 0x7f99f00d1300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f99e00097a0 tx=0x7f99e002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40cfa2b7fc4a389e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:32.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.090+0000 7f99ed59a640 1 -- 172.21.15.172:0/965754909 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0038070 con 0x7f99f015e890 2024-04-14T08:38:32.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.090+0000 7f99ed59a640 1 -- 172.21.15.172:0/965754909 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e0031500 con 0x7f99f015e890 2024-04-14T08:38:32.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.090+0000 7f99ed59a640 1 -- 172.21.15.172:0/965754909 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e80019f0 con 0x7f99f015e890 2024-04-14T08:38:32.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.090+0000 7f99f4c36640 1 -- 172.21.15.172:0/965754909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99f015e890 msgr2=0x7f99f00d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.090+0000 7f99f4c36640 1 --2- 172.21.15.172:0/965754909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99f015e890 0x7f99f00d1300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f99e00097a0 tx=0x7f99e002b820 comp rx=0 tx=0).stop 2024-04-14T08:38:32.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.090+0000 7f99f4c36640 1 -- 172.21.15.172:0/965754909 shutdown_connections 2024-04-14T08:38:32.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.090+0000 7f99f4c36640 1 --2- 172.21.15.172:0/965754909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99f015e890 0x7f99f00d1300 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.090+0000 7f99f4c36640 1 --2- 172.21.15.172:0/965754909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99f00d2830 0x7f99f00d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.090+0000 7f99f4c36640 1 -- 172.21.15.172:0/965754909 >> 172.21.15.172:0/965754909 conn(0x7f99f009d590 msgr2=0x7f99f00d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:32.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.090+0000 7f99f4c36640 1 -- 172.21.15.172:0/965754909 shutdown_connections 2024-04-14T08:38:32.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.090+0000 7f99f4c36640 1 -- 172.21.15.172:0/965754909 wait complete. 2024-04-14T08:38:32.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.092+0000 7f99f4c36640 1 Processor -- start 2024-04-14T08:38:32.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.092+0000 7f99f4c36640 1 -- start start 2024-04-14T08:38:32.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.092+0000 7f99f4c36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f00cedd0 0x7f99f00b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.092+0000 7f99f4c36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99f00d2830 0x7f99f00b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.092+0000 7f99f4c36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99f015e890 0x7f99f00cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.092+0000 7f99f4c36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f99f00cfbc0 con 0x7f99f00cedd0 2024-04-14T08:38:32.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.092+0000 7f99f4c36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f99f00cf8e0 con 0x7f99f00d2830 2024-04-14T08:38:32.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.092+0000 7f99f4c36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f99f00d1840 con 0x7f99f015e890 2024-04-14T08:38:32.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.092+0000 7f99ee59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f00cedd0 0x7f99f00b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.092+0000 7f99ee59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f00cedd0 0x7f99f00b9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35580/0 (socket says 172.21.15.172:35580) 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.092+0000 7f99ee59c640 1 -- 172.21.15.172:0/1428440309 learned_addr learned my addr 172.21.15.172:0/1428440309 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99eed9d640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99f015e890 0x7f99f00cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99edd9b640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99f00d2830 0x7f99f00b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99ee59c640 1 -- 172.21.15.172:0/1428440309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99f015e890 msgr2=0x7f99f00cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99ee59c640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99f015e890 0x7f99f00cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99ee59c640 1 -- 172.21.15.172:0/1428440309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99f00d2830 msgr2=0x7f99f00b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99ee59c640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99f00d2830 0x7f99f00b6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99ee59c640 1 -- 172.21.15.172:0/1428440309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f00c9170 con 0x7f99f00cedd0 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99ee59c640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f00cedd0 0x7f99f00b9dd0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f99e400e870 tx=0x7f99e400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99deffd640 1 -- 172.21.15.172:0/1428440309 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e4017070 con 0x7f99f00cedd0 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99deffd640 1 -- 172.21.15.172:0/1428440309 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e400c940 con 0x7f99f00cedd0 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99f4c36640 1 -- 172.21.15.172:0/1428440309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f00c9450 con 0x7f99f00cedd0 2024-04-14T08:38:32.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99deffd640 1 -- 172.21.15.172:0/1428440309 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e400cd30 con 0x7f99f00cedd0 2024-04-14T08:38:32.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.093+0000 7f99f4c36640 1 -- 172.21.15.172:0/1428440309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f00b76c0 con 0x7f99f00cedd0 2024-04-14T08:38:32.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.094+0000 7f99f4c36640 1 -- 172.21.15.172:0/1428440309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f0068520 con 0x7f99f00cedd0 2024-04-14T08:38:32.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.095+0000 7f99deffd640 1 -- 172.21.15.172:0/1428440309 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99e401a070 con 0x7f99f00cedd0 2024-04-14T08:38:32.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.095+0000 7f99deffd640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f99d006c8f0 0x7f99d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.096+0000 7f99deffd640 1 -- 172.21.15.172:0/1428440309 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f99e408be90 con 0x7f99f00cedd0 2024-04-14T08:38:32.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.096+0000 7f99edd9b640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f99d006c8f0 0x7f99d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.096+0000 7f99edd9b640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f99d006c8f0 0x7f99d006edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f99e0035000 tx=0x7f99e0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:32.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.104+0000 7f99deffd640 1 -- 172.21.15.172:0/1428440309 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f99e4059a80 con 0x7f99f00cedd0 2024-04-14T08:38:32.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.321+0000 7f99f4c36640 1 -- 172.21.15.172:0/1428440309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99f00d4960 con 0x7f99f00cedd0 2024-04-14T08:38:32.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.322+0000 7f99deffd640 1 -- 172.21.15.172:0/1428440309 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5194 (secure 0 0 0) 0x7f99e405dd80 con 0x7f99f00cedd0 2024-04-14T08:38:32.323 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:32.323 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:37:01.115031+0000","modified":"2024-04-14T08:38:02.376656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":15,"addr":"172.21.15.172:6837/2920402198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2920402198},{"type":"v1","addr":"172.21.15.172:6837","nonce":2920402198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":5}]} 2024-04-14T08:38:32.323 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 47 2024-04-14T08:38:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.324+0000 7f99f4c36640 1 -- 172.21.15.172:0/1428440309 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f99d006c8f0 msgr2=0x7f99d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.324+0000 7f99f4c36640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f99d006c8f0 0x7f99d006edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f99e0035000 tx=0x7f99e0008030 comp rx=0 tx=0).stop 2024-04-14T08:38:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.324+0000 7f99f4c36640 1 -- 172.21.15.172:0/1428440309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f00cedd0 msgr2=0x7f99f00b9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.324+0000 7f99f4c36640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f00cedd0 0x7f99f00b9dd0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f99e400e870 tx=0x7f99e400ed30 comp rx=0 tx=0).stop 2024-04-14T08:38:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.324+0000 7f99f4c36640 1 -- 172.21.15.172:0/1428440309 shutdown_connections 2024-04-14T08:38:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.324+0000 7f99f4c36640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f99d006c8f0 0x7f99d006edb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.324+0000 7f99f4c36640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99f015e890 0x7f99f00cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.324+0000 7f99f4c36640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99f00d2830 0x7f99f00b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.324+0000 7f99f4c36640 1 --2- 172.21.15.172:0/1428440309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f00cedd0 0x7f99f00b9dd0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.324+0000 7f99f4c36640 1 -- 172.21.15.172:0/1428440309 >> 172.21.15.172:0/1428440309 conn(0x7f99f009d590 msgr2=0x7f99f00d4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.324+0000 7f99f4c36640 1 -- 172.21.15.172:0/1428440309 shutdown_connections 2024-04-14T08:38:32.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.324+0000 7f99f4c36640 1 -- 172.21.15.172:0/1428440309 wait complete. 2024-04-14T08:38:32.335 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:38:32.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.436+0000 7f4d04098640 1 Processor -- start 2024-04-14T08:38:32.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.437+0000 7f4d04098640 1 -- start start 2024-04-14T08:38:32.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.437+0000 7f4d04098640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4cfc0d2830 0x7f4cfc0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.437+0000 7f4d04098640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cfc0d3150 con 0x7f4cfc15e890 2024-04-14T08:38:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.437+0000 7f4d04098640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cfc0cf320 con 0x7f4cfc0cedd0 2024-04-14T08:38:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.437+0000 7f4d04098640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f4cfc0cff30 con 0x7f4cfc0d2830 2024-04-14T08:38:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.438+0000 7f4d01e34640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4cfc0d2830 0x7f4cfc0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.438+0000 7f4d01633640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4cfc15e890 0x7f4cfc0d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54636/0 (socket says 172.21.15.172:54636) 2024-04-14T08:38:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.438+0000 7f4d01633640 1 -- 172.21.15.172:0/3575711824 learned_addr learned my addr 172.21.15.172:0/3575711824 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.438+0000 7f4d01e34640 1 --2- 172.21.15.172:0/3575711824 >> v2:172.21.15.172:3300/0 conn(0x7f4cfc0d2830 0x7f4cfc0d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35592/0 (socket says 172.21.15.172:35592) 2024-04-14T08:38:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.438+0000 7f4d01e34640 1 -- 172.21.15.172:0/3575711824 >> v1:172.21.15.177:6789/0 conn(0x7f4cfc0cedd0 legacy=0x7f4cfc0b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.438+0000 7f4d01e34640 1 -- 172.21.15.172:0/3575711824 >> v1:172.21.15.172:6789/0 conn(0x7f4cfc15e890 legacy=0x7f4cfc0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.438+0000 7f4d00e32640 1 -- 172.21.15.172:0/3575711824 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024310620 0 0) 0x7f4cfc0d3150 con 0x7f4cfc15e890 2024-04-14T08:38:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.438+0000 7f4d01e34640 1 -- 172.21.15.172:0/3575711824 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc0cf600 con 0x7f4cfc0d2830 2024-04-14T08:38:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.439+0000 7f4d01e34640 1 --2- 172.21.15.172:0/3575711824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cfc0d2830 0x7f4cfc0d2c10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00097a0 tx=0x7f4cf002b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64d18c8a5fb923c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.439+0000 7f4d00e32640 1 -- 172.21.15.172:0/3575711824 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf002f810 con 0x7f4cfc0d2830 2024-04-14T08:38:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.439+0000 7f4d00e32640 1 -- 172.21.15.172:0/3575711824 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf002f9b0 con 0x7f4cfc0d2830 2024-04-14T08:38:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.439+0000 7f4d00e32640 1 -- 172.21.15.172:0/3575711824 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf002fd80 con 0x7f4cfc0d2830 2024-04-14T08:38:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.439+0000 7f4d04098640 1 -- 172.21.15.172:0/3575711824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cfc0d2830 msgr2=0x7f4cfc0d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.439+0000 7f4d04098640 1 --2- 172.21.15.172:0/3575711824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cfc0d2830 0x7f4cfc0d2c10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00097a0 tx=0x7f4cf002b9f0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.439+0000 7f4d04098640 1 -- 172.21.15.172:0/3575711824 shutdown_connections 2024-04-14T08:38:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.439+0000 7f4d04098640 1 --2- 172.21.15.172:0/3575711824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cfc0d2830 0x7f4cfc0d2c10 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.439+0000 7f4d04098640 1 -- 172.21.15.172:0/3575711824 >> 172.21.15.172:0/3575711824 conn(0x7f4cfc09d590 msgr2=0x7f4cfc0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.440+0000 7f4d04098640 1 -- 172.21.15.172:0/3575711824 shutdown_connections 2024-04-14T08:38:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.440+0000 7f4d04098640 1 -- 172.21.15.172:0/3575711824 wait complete. 2024-04-14T08:38:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.441+0000 7f4d04098640 1 Processor -- start 2024-04-14T08:38:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.441+0000 7f4d04098640 1 -- start start 2024-04-14T08:38:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d04098640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cfc0cedd0 0x7f4cfc1f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d04098640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cfc0d2830 0x7f4cfc1f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d04098640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cfc15e890 0x7f4cfc1f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d04098640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc0cfbc0 con 0x7f4cfc15e890 2024-04-14T08:38:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d04098640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc0d2350 con 0x7f4cfc0d2830 2024-04-14T08:38:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d04098640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4cfc0b3e20 con 0x7f4cfc0cedd0 2024-04-14T08:38:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d02635640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cfc15e890 0x7f4cfc1f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d02635640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cfc15e890 0x7f4cfc1f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35596/0 (socket says 172.21.15.172:35596) 2024-04-14T08:38:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d02635640 1 -- 172.21.15.172:0/1156315580 learned_addr learned my addr 172.21.15.172:0/1156315580 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d01633640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cfc0d2830 0x7f4cfc1f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d01e34640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cfc0cedd0 0x7f4cfc1f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d02635640 1 -- 172.21.15.172:0/1156315580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cfc0cedd0 msgr2=0x7f4cfc1f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d02635640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cfc0cedd0 0x7f4cfc1f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d02635640 1 -- 172.21.15.172:0/1156315580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cfc0d2830 msgr2=0x7f4cfc1f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d02635640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cfc0d2830 0x7f4cfc1f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d02635640 1 -- 172.21.15.172:0/1156315580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc0b5340 con 0x7f4cfc15e890 2024-04-14T08:38:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d02635640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cfc15e890 0x7f4cfc1f7ef0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400c7d0 tx=0x7f4cf400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4cee7fc640 1 -- 172.21.15.172:0/1156315580 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4003b70 con 0x7f4cfc15e890 2024-04-14T08:38:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4cee7fc640 1 -- 172.21.15.172:0/1156315580 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf4003d30 con 0x7f4cfc15e890 2024-04-14T08:38:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d04098640 1 -- 172.21.15.172:0/1156315580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cfc0b5620 con 0x7f4cfc15e890 2024-04-14T08:38:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4cee7fc640 1 -- 172.21.15.172:0/1156315580 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf400aa90 con 0x7f4cfc15e890 2024-04-14T08:38:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.443+0000 7f4d04098640 1 -- 172.21.15.172:0/1156315580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cfc0b5be0 con 0x7f4cfc15e890 2024-04-14T08:38:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.444+0000 7f4d04098640 1 -- 172.21.15.172:0/1156315580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cfc068520 con 0x7f4cfc15e890 2024-04-14T08:38:32.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.445+0000 7f4cee7fc640 1 -- 172.21.15.172:0/1156315580 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4cf400ac30 con 0x7f4cfc15e890 2024-04-14T08:38:32.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.445+0000 7f4cee7fc640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ce006c8f0 0x7f4ce006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.445+0000 7f4cee7fc640 1 -- 172.21.15.172:0/1156315580 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f4cf408bab0 con 0x7f4cfc15e890 2024-04-14T08:38:32.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.445+0000 7f4d01e34640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ce006c8f0 0x7f4ce006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.446+0000 7f4d01e34640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ce006c8f0 0x7f4ce006edb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0036000 tx=0x7f4cf0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:32.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.450+0000 7f4cee7fc640 1 -- 172.21.15.172:0/1156315580 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4cf4059740 con 0x7f4cfc15e890 2024-04-14T08:38:32.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.696+0000 7f4d04098640 1 -- 172.21.15.172:0/1156315580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cfc0d2610 con 0x7f4cfc15e890 2024-04-14T08:38:32.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.698+0000 7f4cee7fc640 1 -- 172.21.15.172:0/1156315580 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+17136 (secure 0 0 0) 0x7f4cf405da40 con 0x7f4cfc15e890 2024-04-14T08:38:32.698 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:32.699 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":75,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:37:15.356496+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:36:59.342045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-14T08:37:00.059338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/2134832221":"2024-04-15T08:37:15.356472+0000","172.21.15.172:6835/2134832221":"2024-04-15T08:37:15.356472+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:38:32.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.700+0000 7f4d04098640 1 -- 172.21.15.172:0/1156315580 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ce006c8f0 msgr2=0x7f4ce006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.700+0000 7f4d04098640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ce006c8f0 0x7f4ce006edb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0036000 tx=0x7f4cf0008030 comp rx=0 tx=0).stop 2024-04-14T08:38:32.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.700+0000 7f4d04098640 1 -- 172.21.15.172:0/1156315580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cfc15e890 msgr2=0x7f4cfc1f7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.700+0000 7f4d04098640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cfc15e890 0x7f4cfc1f7ef0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400c7d0 tx=0x7f4cf400cc90 comp rx=0 tx=0).stop 2024-04-14T08:38:32.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.700+0000 7f4d04098640 1 -- 172.21.15.172:0/1156315580 shutdown_connections 2024-04-14T08:38:32.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.700+0000 7f4d04098640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4ce006c8f0 0x7f4ce006edb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.700+0000 7f4d04098640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cfc15e890 0x7f4cfc1f7ef0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.700+0000 7f4d04098640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cfc0d2830 0x7f4cfc1f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.700+0000 7f4d04098640 1 --2- 172.21.15.172:0/1156315580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cfc0cedd0 0x7f4cfc1f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.700+0000 7f4d04098640 1 -- 172.21.15.172:0/1156315580 >> 172.21.15.172:0/1156315580 conn(0x7f4cfc09d590 msgr2=0x7f4cfc0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:32.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.700+0000 7f4d04098640 1 -- 172.21.15.172:0/1156315580 shutdown_connections 2024-04-14T08:38:32.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.700+0000 7f4d04098640 1 -- 172.21.15.172:0/1156315580 wait complete. 2024-04-14T08:38:32.711 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-14T08:38:32.712 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:38:32.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.816+0000 7fc16f11e640 1 Processor -- start 2024-04-14T08:38:32.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.816+0000 7fc16f11e640 1 -- start start 2024-04-14T08:38:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.817+0000 7fc16f11e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1680d2830 0x7fc1680d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.817+0000 7fc16f11e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1680d3150 con 0x7fc1680cedd0 2024-04-14T08:38:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.817+0000 7fc16f11e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1680cf320 con 0x7fc16815e890 2024-04-14T08:38:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.817+0000 7fc16f11e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc1680cff30 con 0x7fc1680d2830 2024-04-14T08:38:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.817+0000 7fc16ceba640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1680d2830 0x7fc1680d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.817+0000 7fc16d6bb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc1680cedd0 0x7fc1680b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54638/0 (socket says 172.21.15.172:54638) 2024-04-14T08:38:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.817+0000 7fc16ceba640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1680d2830 0x7fc1680d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35604/0 (socket says 172.21.15.172:35604) 2024-04-14T08:38:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.817+0000 7fc16d6bb640 1 -- 172.21.15.172:0/3578970326 learned_addr learned my addr 172.21.15.172:0/3578970326 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.818+0000 7fc16ceba640 1 -- 172.21.15.172:0/3578970326 >> v1:172.21.15.177:6789/0 conn(0x7fc16815e890 legacy=0x7fc1680d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.818+0000 7fc16ceba640 1 -- 172.21.15.172:0/3578970326 >> v1:172.21.15.172:6789/0 conn(0x7fc1680cedd0 legacy=0x7fc1680b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.818+0000 7fc16ceba640 1 -- 172.21.15.172:0/3578970326 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1680cf600 con 0x7fc1680d2830 2024-04-14T08:38:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.818+0000 7fc1677fe640 1 -- 172.21.15.172:0/3578970326 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 375868402 0 0) 0x7fc1680d3150 con 0x7fc1680cedd0 2024-04-14T08:38:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.819+0000 7fc16ceba640 1 --2- 172.21.15.172:0/3578970326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1680d2830 0x7fc1680d2c10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0097a0 tx=0x7fc15c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4177b6d3af2a5d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.819+0000 7fc1677fe640 1 -- 172.21.15.172:0/3578970326 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c034070 con 0x7fc1680d2830 2024-04-14T08:38:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.819+0000 7fc1677fe640 1 -- 172.21.15.172:0/3578970326 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc15c030610 con 0x7fc1680d2830 2024-04-14T08:38:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.819+0000 7fc1677fe640 1 -- 172.21.15.172:0/3578970326 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c0309e0 con 0x7fc1680d2830 2024-04-14T08:38:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.819+0000 7fc16f11e640 1 -- 172.21.15.172:0/3578970326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1680d2830 msgr2=0x7fc1680d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.819+0000 7fc16f11e640 1 --2- 172.21.15.172:0/3578970326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1680d2830 0x7fc1680d2c10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0097a0 tx=0x7fc15c02b960 comp rx=0 tx=0).stop 2024-04-14T08:38:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.819+0000 7fc16f11e640 1 -- 172.21.15.172:0/3578970326 shutdown_connections 2024-04-14T08:38:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.819+0000 7fc16f11e640 1 --2- 172.21.15.172:0/3578970326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1680d2830 0x7fc1680d2c10 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.819+0000 7fc16f11e640 1 -- 172.21.15.172:0/3578970326 >> 172.21.15.172:0/3578970326 conn(0x7fc16809d590 msgr2=0x7fc1680d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.819+0000 7fc16f11e640 1 -- 172.21.15.172:0/3578970326 shutdown_connections 2024-04-14T08:38:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.819+0000 7fc16f11e640 1 -- 172.21.15.172:0/3578970326 wait complete. 2024-04-14T08:38:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.821+0000 7fc16f11e640 1 Processor -- start 2024-04-14T08:38:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.821+0000 7fc16f11e640 1 -- start start 2024-04-14T08:38:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.822+0000 7fc16f11e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1680cedd0 0x7fc1680b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.822+0000 7fc16f11e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1680d2830 0x7fc1681585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.822+0000 7fc16f11e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc16815e890 0x7fc1680c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.822+0000 7fc16f11e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc1680cfbc0 con 0x7fc1680cedd0 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.822+0000 7fc16f11e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc1680d2350 con 0x7fc16815e890 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.822+0000 7fc16f11e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc1680b3e20 con 0x7fc1680d2830 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16ceba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1680cedd0 0x7fc1680b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16ceba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1680cedd0 0x7fc1680b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35616/0 (socket says 172.21.15.172:35616) 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16ceba640 1 -- 172.21.15.172:0/4229712307 learned_addr learned my addr 172.21.15.172:0/4229712307 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16d6bb640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc16815e890 0x7fc1680c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc167fff640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1680d2830 0x7fc1681585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16ceba640 1 -- 172.21.15.172:0/4229712307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1680d2830 msgr2=0x7fc1681585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16ceba640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1680d2830 0x7fc1681585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16ceba640 1 -- 172.21.15.172:0/4229712307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc16815e890 msgr2=0x7fc1680c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16ceba640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc16815e890 0x7fc1680c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16ceba640 1 -- 172.21.15.172:0/4229712307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1680c95f0 con 0x7fc1680cedd0 2024-04-14T08:38:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16d6bb640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc16815e890 0x7fc1680c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:32.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc167fff640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1680d2830 0x7fc1681585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:32.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16ceba640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1680cedd0 0x7fc1680b6e70 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0098c0 tx=0x7fc15c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:32.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc1657fa640 1 -- 172.21.15.172:0/4229712307 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c049070 con 0x7fc1680cedd0 2024-04-14T08:38:32.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc1657fa640 1 -- 172.21.15.172:0/4229712307 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc15c039800 con 0x7fc1680cedd0 2024-04-14T08:38:32.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16f11e640 1 -- 172.21.15.172:0/4229712307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc168160140 con 0x7fc1680cedd0 2024-04-14T08:38:32.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc1657fa640 1 -- 172.21.15.172:0/4229712307 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c034040 con 0x7fc1680cedd0 2024-04-14T08:38:32.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.823+0000 7fc16f11e640 1 -- 172.21.15.172:0/4229712307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc168160700 con 0x7fc1680cedd0 2024-04-14T08:38:32.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.824+0000 7fc16f11e640 1 -- 172.21.15.172:0/4229712307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1680d1fc0 con 0x7fc1680cedd0 2024-04-14T08:38:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.825+0000 7fc1657fa640 1 -- 172.21.15.172:0/4229712307 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc15c04c070 con 0x7fc1680cedd0 2024-04-14T08:38:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.825+0000 7fc1657fa640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc14c06c920 0x7fc14c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.826+0000 7fc1657fa640 1 -- 172.21.15.172:0/4229712307 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7fc15c0ad990 con 0x7fc1680cedd0 2024-04-14T08:38:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.826+0000 7fc167fff640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc14c06c920 0x7fc14c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.826+0000 7fc167fff640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc14c06c920 0x7fc14c06ede0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc1680cae30 tx=0x7fc154008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:32.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:32.830+0000 7fc1657fa640 1 -- 172.21.15.172:0/4229712307 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc1680d1fc0 con 0x7fc1680cedd0 2024-04-14T08:38:33.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.049+0000 7fc16f11e640 1 -- 172.21.15.172:0/4229712307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc168068520 con 0x7fc1680cedd0 2024-04-14T08:38:33.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.049+0000 7fc1657fa640 1 -- 172.21.15.172:0/4229712307 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5194 (secure 0 0 0) 0x7fc15c07bc70 con 0x7fc1680cedd0 2024-04-14T08:38:33.050 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:33.050 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:37:01.115031+0000","modified":"2024-04-14T08:38:02.376656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":15,"addr":"172.21.15.172:6837/2920402198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2920402198},{"type":"v1","addr":"172.21.15.172:6837","nonce":2920402198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":5}]} 2024-04-14T08:38:33.051 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 47 2024-04-14T08:38:33.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.051+0000 7fc16f11e640 1 -- 172.21.15.172:0/4229712307 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc14c06c920 msgr2=0x7fc14c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.051+0000 7fc16f11e640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc14c06c920 0x7fc14c06ede0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc1680cae30 tx=0x7fc154008040 comp rx=0 tx=0).stop 2024-04-14T08:38:33.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.051+0000 7fc16f11e640 1 -- 172.21.15.172:0/4229712307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1680cedd0 msgr2=0x7fc1680b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.051+0000 7fc16f11e640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1680cedd0 0x7fc1680b6e70 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0098c0 tx=0x7fc15c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.051+0000 7fc16f11e640 1 -- 172.21.15.172:0/4229712307 shutdown_connections 2024-04-14T08:38:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.051+0000 7fc16f11e640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc14c06c920 0x7fc14c06ede0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.051+0000 7fc16f11e640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc16815e890 0x7fc1680c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.051+0000 7fc16f11e640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1680d2830 0x7fc1681585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.051+0000 7fc16f11e640 1 --2- 172.21.15.172:0/4229712307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1680cedd0 0x7fc1680b6e70 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.052+0000 7fc16f11e640 1 -- 172.21.15.172:0/4229712307 >> 172.21.15.172:0/4229712307 conn(0x7fc16809d590 msgr2=0x7fc1680d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.052+0000 7fc16f11e640 1 -- 172.21.15.172:0/4229712307 shutdown_connections 2024-04-14T08:38:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.052+0000 7fc16f11e640 1 -- 172.21.15.172:0/4229712307 wait complete. 2024-04-14T08:38:33.066 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:38:33.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.174+0000 7f99d3e88640 1 Processor -- start 2024-04-14T08:38:33.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.175+0000 7f99d3e88640 1 -- start start 2024-04-14T08:38:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.175+0000 7f99d3e88640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99cc0d2830 0x7f99cc0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.175+0000 7f99d3e88640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99cc0d3150 con 0x7f99cc15e890 2024-04-14T08:38:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.175+0000 7f99d3e88640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99cc0cf320 con 0x7f99cc0cedd0 2024-04-14T08:38:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.175+0000 7f99d3e88640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f99cc0cff30 con 0x7f99cc0d2830 2024-04-14T08:38:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.176+0000 7f99d1c24640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99cc0d2830 0x7f99cc0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.176+0000 7f99d1423640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f99cc15e890 0x7f99cc0d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54652/0 (socket says 172.21.15.172:54652) 2024-04-14T08:38:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.176+0000 7f99d1423640 1 -- 172.21.15.172:0/3120209224 learned_addr learned my addr 172.21.15.172:0/3120209224 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.176+0000 7f99d0c22640 1 -- 172.21.15.172:0/3120209224 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 808617973 0 0) 0x7f99cc0d3150 con 0x7f99cc15e890 2024-04-14T08:38:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.176+0000 7f99d0c22640 1 -- 172.21.15.172:0/3120209224 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99b8003520 con 0x7f99cc15e890 2024-04-14T08:38:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.176+0000 7f99d1c24640 1 -- 172.21.15.172:0/3120209224 >> v1:172.21.15.177:6789/0 conn(0x7f99cc0cedd0 legacy=0x7f99cc0b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.176+0000 7f99d1c24640 1 -- 172.21.15.172:0/3120209224 >> v1:172.21.15.172:6789/0 conn(0x7f99cc15e890 legacy=0x7f99cc0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.176+0000 7f99d1c24640 1 -- 172.21.15.172:0/3120209224 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc0cf600 con 0x7f99cc0d2830 2024-04-14T08:38:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.177+0000 7f99d1c24640 1 --2- 172.21.15.172:0/3120209224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc0d2830 0x7f99cc0d2c10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f99c80097a0 tx=0x7f99c802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea99f38098512a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.177+0000 7f99d0c22640 1 -- 172.21.15.172:0/3120209224 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c802f810 con 0x7f99cc0d2830 2024-04-14T08:38:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.177+0000 7f99d0c22640 1 -- 172.21.15.172:0/3120209224 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c802f9b0 con 0x7f99cc0d2830 2024-04-14T08:38:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.177+0000 7f99d0c22640 1 -- 172.21.15.172:0/3120209224 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c802fd60 con 0x7f99cc0d2830 2024-04-14T08:38:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.177+0000 7f99d3e88640 1 -- 172.21.15.172:0/3120209224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc0d2830 msgr2=0x7f99cc0d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.177+0000 7f99d3e88640 1 --2- 172.21.15.172:0/3120209224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc0d2830 0x7f99cc0d2c10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f99c80097a0 tx=0x7f99c802b9f0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.177+0000 7f99d3e88640 1 -- 172.21.15.172:0/3120209224 shutdown_connections 2024-04-14T08:38:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.177+0000 7f99d3e88640 1 --2- 172.21.15.172:0/3120209224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc0d2830 0x7f99cc0d2c10 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.177+0000 7f99d3e88640 1 -- 172.21.15.172:0/3120209224 >> 172.21.15.172:0/3120209224 conn(0x7f99cc09d590 msgr2=0x7f99cc0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.177+0000 7f99d3e88640 1 -- 172.21.15.172:0/3120209224 shutdown_connections 2024-04-14T08:38:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.177+0000 7f99d3e88640 1 -- 172.21.15.172:0/3120209224 wait complete. 2024-04-14T08:38:33.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.178+0000 7f99d3e88640 1 Processor -- start 2024-04-14T08:38:33.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.179+0000 7f99d3e88640 1 -- start start 2024-04-14T08:38:33.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.179+0000 7f99d3e88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc0cedd0 0x7f99cc0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.179+0000 7f99d3e88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99cc0d2830 0x7f99cc0c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.179+0000 7f99d3e88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99cc15e890 0x7f99cc152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.179+0000 7f99d3e88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc0cfbc0 con 0x7f99cc0cedd0 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.179+0000 7f99d3e88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc0d2350 con 0x7f99cc15e890 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.179+0000 7f99d3e88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f99cc0b3e20 con 0x7f99cc0d2830 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.179+0000 7f99d1c24640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc0cedd0 0x7f99cc0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.179+0000 7f99d1c24640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc0cedd0 0x7f99cc0b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35630/0 (socket says 172.21.15.172:35630) 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.179+0000 7f99d1c24640 1 -- 172.21.15.172:0/3709128073 learned_addr learned my addr 172.21.15.172:0/3709128073 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99d1423640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99cc0d2830 0x7f99cc0c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99d2425640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99cc15e890 0x7f99cc152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99d1c24640 1 -- 172.21.15.172:0/3709128073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99cc0d2830 msgr2=0x7f99cc0c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99d1c24640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99cc0d2830 0x7f99cc0c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99d1c24640 1 -- 172.21.15.172:0/3709128073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99cc15e890 msgr2=0x7f99cc152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99d1c24640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99cc15e890 0x7f99cc152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99d1c24640 1 -- 172.21.15.172:0/3709128073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc0b4f70 con 0x7f99cc0cedd0 2024-04-14T08:38:33.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99d1423640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99cc0d2830 0x7f99cc0c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99d2425640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99cc15e890 0x7f99cc152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99d1c24640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc0cedd0 0x7f99cc0b4690 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f99c8000c00 tx=0x7f99c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99c27fc640 1 -- 172.21.15.172:0/3709128073 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8047070 con 0x7f99cc0cedd0 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99c27fc640 1 -- 172.21.15.172:0/3709128073 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c8004230 con 0x7f99cc0cedd0 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99d3e88640 1 -- 172.21.15.172:0/3709128073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99cc0b5250 con 0x7f99cc0cedd0 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99c27fc640 1 -- 172.21.15.172:0/3709128073 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8037610 con 0x7f99cc0cedd0 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.180+0000 7f99d3e88640 1 -- 172.21.15.172:0/3709128073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99cc0b5810 con 0x7f99cc0cedd0 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.181+0000 7f99d3e88640 1 -- 172.21.15.172:0/3709128073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99cc068520 con 0x7f99cc0cedd0 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.181+0000 7f99c27fc640 1 -- 172.21.15.172:0/3709128073 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99c804a070 con 0x7f99cc0cedd0 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.182+0000 7f99c27fc640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f99b006c8f0 0x7f99b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.182+0000 7f99c27fc640 1 -- 172.21.15.172:0/3709128073 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f99c80ad870 con 0x7f99cc0cedd0 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.182+0000 7f99d1423640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f99b006c8f0 0x7f99b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.182+0000 7f99d1423640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f99b006c8f0 0x7f99b006edb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0c8960 tx=0x7f99bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:33.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.186+0000 7f99c27fc640 1 -- 172.21.15.172:0/3709128073 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f99c8033030 con 0x7f99cc0cedd0 2024-04-14T08:38:33.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.427+0000 7f99d3e88640 1 -- 172.21.15.172:0/3709128073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99cc0d4960 con 0x7f99cc0cedd0 2024-04-14T08:38:33.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.427+0000 7f99c27fc640 1 -- 172.21.15.172:0/3709128073 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5194 (secure 0 0 0) 0x7f99c807bac0 con 0x7f99cc0cedd0 2024-04-14T08:38:33.428 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:33.428 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:37:01.115031+0000","modified":"2024-04-14T08:38:02.376656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":15,"addr":"172.21.15.172:6837/2920402198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2920402198},{"type":"v1","addr":"172.21.15.172:6837","nonce":2920402198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":5}]} 2024-04-14T08:38:33.429 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 47 2024-04-14T08:38:33.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.430+0000 7f99d3e88640 1 -- 172.21.15.172:0/3709128073 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f99b006c8f0 msgr2=0x7f99b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.430+0000 7f99d3e88640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f99b006c8f0 0x7f99b006edb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0c8960 tx=0x7f99bc009040 comp rx=0 tx=0).stop 2024-04-14T08:38:33.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.430+0000 7f99d3e88640 1 -- 172.21.15.172:0/3709128073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc0cedd0 msgr2=0x7f99cc0b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.430+0000 7f99d3e88640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc0cedd0 0x7f99cc0b4690 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f99c8000c00 tx=0x7f99c80046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.430+0000 7f99d3e88640 1 -- 172.21.15.172:0/3709128073 shutdown_connections 2024-04-14T08:38:33.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.430+0000 7f99d3e88640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f99b006c8f0 0x7f99b006edb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.430+0000 7f99d3e88640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99cc15e890 0x7f99cc152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.430+0000 7f99d3e88640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f99cc0d2830 0x7f99cc0c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.430+0000 7f99d3e88640 1 --2- 172.21.15.172:0/3709128073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc0cedd0 0x7f99cc0b4690 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.430+0000 7f99d3e88640 1 -- 172.21.15.172:0/3709128073 >> 172.21.15.172:0/3709128073 conn(0x7f99cc09d590 msgr2=0x7f99cc074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:33.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.430+0000 7f99d3e88640 1 -- 172.21.15.172:0/3709128073 shutdown_connections 2024-04-14T08:38:33.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.430+0000 7f99d3e88640 1 -- 172.21.15.172:0/3709128073 wait complete. 2024-04-14T08:38:33.441 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:38:33.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.548+0000 7ffaee4de640 1 Processor -- start 2024-04-14T08:38:33.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.548+0000 7ffaee4de640 1 -- start start 2024-04-14T08:38:33.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.549+0000 7ffaee4de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae815e890 0x7ffae80d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.549+0000 7ffaee4de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ffae80cedd0 0x7ffae80b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.549+0000 7ffaee4de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffae80cf320 con 0x7ffae80d2830 2024-04-14T08:38:33.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.549+0000 7ffaee4de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ffae80cff30 con 0x7ffae80cedd0 2024-04-14T08:38:33.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.549+0000 7ffaee4de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ffae80d2350 con 0x7ffae815e890 2024-04-14T08:38:33.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.550+0000 7ffaeca7b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ffae80cedd0 0x7ffae80b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.550+0000 7ffaeca7b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ffae80cedd0 0x7ffae80b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35644/0 (socket says 172.21.15.172:35644) 2024-04-14T08:38:33.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.550+0000 7ffaeca7b640 1 -- 172.21.15.172:0/1359750996 learned_addr learned my addr 172.21.15.172:0/1359750996 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:33.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.550+0000 7ffae77fe640 1 --2- 172.21.15.172:0/1359750996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae815e890 0x7ffae80d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.550+0000 7ffaeca7b640 1 -- 172.21.15.172:0/1359750996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae815e890 msgr2=0x7ffae80d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:33.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.550+0000 7ffaeca7b640 1 --2- 172.21.15.172:0/1359750996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae815e890 0x7ffae80d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.550+0000 7ffaeca7b640 1 -- 172.21.15.172:0/1359750996 >> v1:172.21.15.177:6789/0 conn(0x7ffae80d2830 legacy=0x7ffae80d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:33.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.550+0000 7ffaeca7b640 1 -- 172.21.15.172:0/1359750996 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae80cf600 con 0x7ffae80cedd0 2024-04-14T08:38:33.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.551+0000 7ffaeca7b640 1 --2- 172.21.15.172:0/1359750996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae80cedd0 0x7ffae80b3710 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009e50 tx=0x7ffae002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158756f17a9d0e0b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:33.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.551+0000 7ffae6ffd640 1 -- 172.21.15.172:0/1359750996 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae002cb00 con 0x7ffae80cedd0 2024-04-14T08:38:33.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.551+0000 7ffae6ffd640 1 -- 172.21.15.172:0/1359750996 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae002cca0 con 0x7ffae80cedd0 2024-04-14T08:38:33.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.551+0000 7ffae6ffd640 1 -- 172.21.15.172:0/1359750996 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0034610 con 0x7ffae80cedd0 2024-04-14T08:38:33.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.551+0000 7ffaee4de640 1 -- 172.21.15.172:0/1359750996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae80cedd0 msgr2=0x7ffae80b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.551+0000 7ffaee4de640 1 --2- 172.21.15.172:0/1359750996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae80cedd0 0x7ffae80b3710 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009e50 tx=0x7ffae002bf60 comp rx=0 tx=0).stop 2024-04-14T08:38:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.551+0000 7ffaee4de640 1 -- 172.21.15.172:0/1359750996 shutdown_connections 2024-04-14T08:38:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.551+0000 7ffaee4de640 1 --2- 172.21.15.172:0/1359750996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae80cedd0 0x7ffae80b3710 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.551+0000 7ffaee4de640 1 --2- 172.21.15.172:0/1359750996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae815e890 0x7ffae80d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.551+0000 7ffaee4de640 1 -- 172.21.15.172:0/1359750996 >> 172.21.15.172:0/1359750996 conn(0x7ffae809d590 msgr2=0x7ffae80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.551+0000 7ffaee4de640 1 -- 172.21.15.172:0/1359750996 shutdown_connections 2024-04-14T08:38:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.551+0000 7ffaee4de640 1 -- 172.21.15.172:0/1359750996 wait complete. 2024-04-14T08:38:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.553+0000 7ffaee4de640 1 Processor -- start 2024-04-14T08:38:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.553+0000 7ffaee4de640 1 -- start start 2024-04-14T08:38:33.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.553+0000 7ffaee4de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae80cedd0 0x7ffae81f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.553+0000 7ffaee4de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae80d2830 0x7ffae8157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.553+0000 7ffaee4de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae815e890 0x7ffae815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.553+0000 7ffaee4de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffae80d1840 con 0x7ffae815e890 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.553+0000 7ffaee4de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffae80cfbc0 con 0x7ffae80cedd0 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.553+0000 7ffaee4de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ffae80b3c50 con 0x7ffae80d2830 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffaeca7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae815e890 0x7ffae815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffaeca7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae815e890 0x7ffae815cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35654/0 (socket says 172.21.15.172:35654) 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffaeca7b640 1 -- 172.21.15.172:0/1818759639 learned_addr learned my addr 172.21.15.172:0/1818759639 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffae7fff640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae80cedd0 0x7ffae81f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffae77fe640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae80d2830 0x7ffae8157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffaeca7b640 1 -- 172.21.15.172:0/1818759639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae80d2830 msgr2=0x7ffae8157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffaeca7b640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae80d2830 0x7ffae8157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffaeca7b640 1 -- 172.21.15.172:0/1818759639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae80cedd0 msgr2=0x7ffae81f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffaeca7b640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae80cedd0 0x7ffae81f7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffaeca7b640 1 -- 172.21.15.172:0/1818759639 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae8158380 con 0x7ffae815e890 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffaeca7b640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae815e890 0x7ffae815cf50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ffae0036040 tx=0x7ffae0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffae4ff9640 1 -- 172.21.15.172:0/1818759639 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0043020 con 0x7ffae815e890 2024-04-14T08:38:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffae4ff9640 1 -- 172.21.15.172:0/1818759639 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae002ce50 con 0x7ffae815e890 2024-04-14T08:38:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffae4ff9640 1 -- 172.21.15.172:0/1818759639 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae003e650 con 0x7ffae815e890 2024-04-14T08:38:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffaee4de640 1 -- 172.21.15.172:0/1818759639 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae80c97a0 con 0x7ffae815e890 2024-04-14T08:38:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.554+0000 7ffaee4de640 1 -- 172.21.15.172:0/1818759639 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae81f85b0 con 0x7ffae815e890 2024-04-14T08:38:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.555+0000 7ffaee4de640 1 -- 172.21.15.172:0/1818759639 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae8158dd0 con 0x7ffae815e890 2024-04-14T08:38:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.556+0000 7ffae4ff9640 1 -- 172.21.15.172:0/1818759639 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffae0046020 con 0x7ffae815e890 2024-04-14T08:38:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.556+0000 7ffae4ff9640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffad406c8f0 0x7ffad406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.556+0000 7ffae4ff9640 1 -- 172.21.15.172:0/1818759639 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7ffae00ae2e0 con 0x7ffae815e890 2024-04-14T08:38:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.556+0000 7ffae7fff640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffad406c8f0 0x7ffad406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.557+0000 7ffae7fff640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffad406c8f0 0x7ffad406edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ffad80017f0 tx=0x7ffad8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:33.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.564+0000 7ffae4ff9640 1 -- 172.21.15.172:0/1818759639 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ffae8158dd0 con 0x7ffae815e890 2024-04-14T08:38:33.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.747+0000 7ffaee4de640 1 -- 172.21.15.172:0/1818759639 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffae8068520 con 0x7ffae815e890 2024-04-14T08:38:33.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.749+0000 7ffae4ff9640 1 -- 172.21.15.172:0/1818759639 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+17136 (secure 0 0 0) 0x7ffae004d030 con 0x7ffae815e890 2024-04-14T08:38:33.750 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:33.750 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":75,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:37:15.356496+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:36:59.342045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-14T08:37:00.059338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/2134832221":"2024-04-15T08:37:15.356472+0000","172.21.15.172:6835/2134832221":"2024-04-15T08:37:15.356472+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:38:33.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.752+0000 7ffaee4de640 1 -- 172.21.15.172:0/1818759639 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffad406c8f0 msgr2=0x7ffad406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.752+0000 7ffaee4de640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffad406c8f0 0x7ffad406edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ffad80017f0 tx=0x7ffad8009040 comp rx=0 tx=0).stop 2024-04-14T08:38:33.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.752+0000 7ffaee4de640 1 -- 172.21.15.172:0/1818759639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae815e890 msgr2=0x7ffae815cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.752+0000 7ffaee4de640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae815e890 0x7ffae815cf50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ffae0036040 tx=0x7ffae0004a40 comp rx=0 tx=0).stop 2024-04-14T08:38:33.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.752+0000 7ffaee4de640 1 -- 172.21.15.172:0/1818759639 shutdown_connections 2024-04-14T08:38:33.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.752+0000 7ffaee4de640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ffad406c8f0 0x7ffad406edb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.752+0000 7ffaee4de640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae815e890 0x7ffae815cf50 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.752+0000 7ffaee4de640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae80d2830 0x7ffae8157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.752+0000 7ffaee4de640 1 --2- 172.21.15.172:0/1818759639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae80cedd0 0x7ffae81f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.752+0000 7ffaee4de640 1 -- 172.21.15.172:0/1818759639 >> 172.21.15.172:0/1818759639 conn(0x7ffae809d590 msgr2=0x7ffae806c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:33.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.752+0000 7ffaee4de640 1 -- 172.21.15.172:0/1818759639 shutdown_connections 2024-04-14T08:38:33.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.752+0000 7ffaee4de640 1 -- 172.21.15.172:0/1818759639 wait complete. 2024-04-14T08:38:33.764 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:38:33.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.866+0000 7f1b94b1e640 1 Processor -- start 2024-04-14T08:38:33.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.866+0000 7f1b94b1e640 1 -- start start 2024-04-14T08:38:33.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.867+0000 7f1b94b1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1b9015e4b0 0x7f1b900cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.867+0000 7f1b94b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b900cfd70 0x7f1b900d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.867+0000 7f1b94b1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9015e890 con 0x7f1b900d0fb0 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.867+0000 7f1b94b1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f1b900b49c0 con 0x7f1b9015e4b0 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.867+0000 7f1b94b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1b900d1f30 con 0x7f1b900cfd70 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.868+0000 7f1b8e59c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1b9015e4b0 0x7f1b900cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.868+0000 7f1b85d9b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b900cfd70 0x7f1b900d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.868+0000 7f1b8ed9d640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f1b900d0fb0 0x7f1b90161fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42008/0 (socket says 172.21.15.172:42008) 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.868+0000 7f1b8ed9d640 1 -- 172.21.15.172:0/3762108353 learned_addr learned my addr 172.21.15.172:0/3762108353 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.868+0000 7f1b8e59c640 1 -- 172.21.15.172:0/3762108353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b900cfd70 msgr2=0x7f1b900d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.868+0000 7f1b8e59c640 1 --2- 172.21.15.172:0/3762108353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b900cfd70 0x7f1b900d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.868+0000 7f1b8e59c640 1 -- 172.21.15.172:0/3762108353 >> v1:172.21.15.177:6789/0 conn(0x7f1b900d0fb0 legacy=0x7f1b90161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.868+0000 7f1b8dd9b640 1 -- 172.21.15.172:0/3762108353 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2955560477 0 0) 0x7f1b9015e890 con 0x7f1b900d0fb0 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.868+0000 7f1b8e59c640 1 -- 172.21.15.172:0/3762108353 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b900d2210 con 0x7f1b9015e4b0 2024-04-14T08:38:33.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.868+0000 7f1b85d9b640 1 --2- 172.21.15.172:0/3762108353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b900cfd70 0x7f1b900d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:33.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.869+0000 7f1b8e59c640 1 --2- 172.21.15.172:0/3762108353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b9015e4b0 0x7f1b900cf830 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1b800097a0 tx=0x7f1b8002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22cb9004b98a95ca server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:33.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.869+0000 7f1b8dd9b640 1 -- 172.21.15.172:0/3762108353 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b800315a0 con 0x7f1b9015e4b0 2024-04-14T08:38:33.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.869+0000 7f1b8dd9b640 1 -- 172.21.15.172:0/3762108353 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b80037070 con 0x7f1b9015e4b0 2024-04-14T08:38:33.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.869+0000 7f1b8dd9b640 1 -- 172.21.15.172:0/3762108353 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80031960 con 0x7f1b9015e4b0 2024-04-14T08:38:33.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.869+0000 7f1b94b1e640 1 -- 172.21.15.172:0/3762108353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b9015e4b0 msgr2=0x7f1b900cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.869+0000 7f1b94b1e640 1 --2- 172.21.15.172:0/3762108353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b9015e4b0 0x7f1b900cf830 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1b800097a0 tx=0x7f1b8002b840 comp rx=0 tx=0).stop 2024-04-14T08:38:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.869+0000 7f1b94b1e640 1 -- 172.21.15.172:0/3762108353 shutdown_connections 2024-04-14T08:38:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.869+0000 7f1b94b1e640 1 --2- 172.21.15.172:0/3762108353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b900cfd70 0x7f1b900d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.869+0000 7f1b94b1e640 1 --2- 172.21.15.172:0/3762108353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b9015e4b0 0x7f1b900cf830 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.869+0000 7f1b94b1e640 1 -- 172.21.15.172:0/3762108353 >> 172.21.15.172:0/3762108353 conn(0x7f1b9009d570 msgr2=0x7f1b900d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.869+0000 7f1b94b1e640 1 -- 172.21.15.172:0/3762108353 shutdown_connections 2024-04-14T08:38:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.869+0000 7f1b94b1e640 1 -- 172.21.15.172:0/3762108353 wait complete. 2024-04-14T08:38:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.871+0000 7f1b94b1e640 1 Processor -- start 2024-04-14T08:38:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.871+0000 7f1b94b1e640 1 -- start start 2024-04-14T08:38:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.871+0000 7f1b94b1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cfd70 0x7f1b900caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.871+0000 7f1b94b1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b900d0fb0 0x7f1b9015a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.871+0000 7f1b94b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b9015e4b0 0x7f1b900b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.871+0000 7f1b94b1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1b900d2ab0 con 0x7f1b900cfd70 2024-04-14T08:38:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.871+0000 7f1b94b1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1b900d27d0 con 0x7f1b900d0fb0 2024-04-14T08:38:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.871+0000 7f1b94b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1b900d2fd0 con 0x7f1b9015e4b0 2024-04-14T08:38:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b8e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cfd70 0x7f1b900caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b8e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cfd70 0x7f1b900caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35664/0 (socket says 172.21.15.172:35664) 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b8e59c640 1 -- 172.21.15.172:0/1862106623 learned_addr learned my addr 172.21.15.172:0/1862106623 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b8ed9d640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b9015e4b0 0x7f1b900b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b85d9b640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b900d0fb0 0x7f1b9015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b8e59c640 1 -- 172.21.15.172:0/1862106623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b9015e4b0 msgr2=0x7f1b900b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b8e59c640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b9015e4b0 0x7f1b900b8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b8e59c640 1 -- 172.21.15.172:0/1862106623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b900d0fb0 msgr2=0x7f1b9015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b8e59c640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b900d0fb0 0x7f1b9015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b8e59c640 1 -- 172.21.15.172:0/1862106623 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b900caf90 con 0x7f1b900cfd70 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b8e59c640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cfd70 0x7f1b900caa50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1b80034040 tx=0x7f1b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b877fe640 1 -- 172.21.15.172:0/1862106623 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80004230 con 0x7f1b900cfd70 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b877fe640 1 -- 172.21.15.172:0/1862106623 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b800043f0 con 0x7f1b900cfd70 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b877fe640 1 -- 172.21.15.172:0/1862106623 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80003850 con 0x7f1b900cfd70 2024-04-14T08:38:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b94b1e640 1 -- 172.21.15.172:0/1862106623 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b900c82d0 con 0x7f1b900cfd70 2024-04-14T08:38:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.872+0000 7f1b94b1e640 1 -- 172.21.15.172:0/1862106623 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b900c8890 con 0x7f1b900cfd70 2024-04-14T08:38:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.873+0000 7f1b94b1e640 1 -- 172.21.15.172:0/1862106623 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b90068590 con 0x7f1b900cfd70 2024-04-14T08:38:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.874+0000 7f1b877fe640 1 -- 172.21.15.172:0/1862106623 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b800039f0 con 0x7f1b900cfd70 2024-04-14T08:38:33.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.874+0000 7f1b877fe640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1b7006c8f0 0x7f1b7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:33.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.875+0000 7f1b877fe640 1 -- 172.21.15.172:0/1862106623 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6187+0+0 (secure 0 0 0) 0x7f1b8002f030 con 0x7f1b900cfd70 2024-04-14T08:38:33.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.875+0000 7f1b85d9b640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1b7006c8f0 0x7f1b7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:33.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.875+0000 7f1b85d9b640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1b7006c8f0 0x7f1b7006edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009560 tx=0x7f1b78009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:33.879+0000 7f1b877fe640 1 -- 172.21.15.172:0/1862106623 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1b80041070 con 0x7f1b900cfd70 2024-04-14T08:38:34.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.155+0000 7f1b94b1e640 1 -- 172.21.15.172:0/1862106623 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1b900cf810 con 0x7f1b900cfd70 2024-04-14T08:38:34.156 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:38:34.156+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:38:34.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.158+0000 7f1b877fe640 1 -- 172.21.15.172:0/1862106623 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7f1b8007b7d0 con 0x7f1b900cfd70 2024-04-14T08:38:34.159 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:38:34.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.161+0000 7f1b94b1e640 1 -- 172.21.15.172:0/1862106623 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1b7006c8f0 msgr2=0x7f1b7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:34.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.161+0000 7f1b94b1e640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1b7006c8f0 0x7f1b7006edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009560 tx=0x7f1b78009190 comp rx=0 tx=0).stop 2024-04-14T08:38:34.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.161+0000 7f1b94b1e640 1 -- 172.21.15.172:0/1862106623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cfd70 msgr2=0x7f1b900caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:34.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.161+0000 7f1b94b1e640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cfd70 0x7f1b900caa50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1b80034040 tx=0x7f1b800046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:34.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.162+0000 7f1b94b1e640 1 -- 172.21.15.172:0/1862106623 shutdown_connections 2024-04-14T08:38:34.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.162+0000 7f1b94b1e640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1b7006c8f0 0x7f1b7006edb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:34.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.162+0000 7f1b94b1e640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b9015e4b0 0x7f1b900b8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:34.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.162+0000 7f1b94b1e640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b900d0fb0 0x7f1b9015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:34.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.162+0000 7f1b94b1e640 1 --2- 172.21.15.172:0/1862106623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cfd70 0x7f1b900caa50 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:34.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.162+0000 7f1b94b1e640 1 -- 172.21.15.172:0/1862106623 >> 172.21.15.172:0/1862106623 conn(0x7f1b9009d570 msgr2=0x7f1b900b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:34.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.162+0000 7f1b94b1e640 1 -- 172.21.15.172:0/1862106623 shutdown_connections 2024-04-14T08:38:34.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.162+0000 7f1b94b1e640 1 -- 172.21.15.172:0/1862106623 wait complete. 2024-04-14T08:38:34.165 INFO:tasks.ceph.mds.c.smithi172.stderr: -4182> 2024-04-14T08:37:21.372+0000 7f81e5be5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:38:34.175 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-14T08:38:34.194 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:38:34.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.280+0000 7fede995e640 1 Processor -- start 2024-04-14T08:38:34.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.280+0000 7fede995e640 1 -- start start 2024-04-14T08:38:34.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7fede995e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fede40d0a70 0x7fede40d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:34.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7fede995e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fede40b39b0 0x7fede40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7fede995e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede415e890 con 0x7fede415e4b0 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7fede995e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fede40d13f0 con 0x7fede40d0a70 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7fede995e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fede40d1f30 con 0x7fede40b39b0 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7feddbfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fede40d0a70 0x7fede40d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7feddbfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fede40d0a70 0x7fede40d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35680/0 (socket says 172.21.15.172:35680) 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7feddbfff640 1 -- 172.21.15.172:0/3488298226 learned_addr learned my addr 172.21.15.172:0/3488298226 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7fede37fe640 1 --2- 172.21.15.172:0/3488298226 >> v2:172.21.15.177:3300/0 conn(0x7fede40b39b0 0x7fede40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7feddbfff640 1 -- 172.21.15.172:0/3488298226 >> v2:172.21.15.177:3300/0 conn(0x7fede40b39b0 msgr2=0x7fede40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7feddbfff640 1 --2- 172.21.15.172:0/3488298226 >> v2:172.21.15.177:3300/0 conn(0x7fede40b39b0 0x7fede40b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7feddbfff640 1 -- 172.21.15.172:0/3488298226 >> v1:172.21.15.177:6789/0 conn(0x7fede415e4b0 legacy=0x7fede40cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7feddbfff640 1 -- 172.21.15.172:0/3488298226 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede40d2210 con 0x7fede40d0a70 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.281+0000 7fede37fe640 1 --2- 172.21.15.172:0/3488298226 >> v2:172.21.15.177:3300/0 conn(0x7fede40b39b0 0x7fede40b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.282+0000 7feddbfff640 1 --2- 172.21.15.172:0/3488298226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede40d0a70 0x7fede40d0eb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fede409da10 tx=0x7fedcc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30092d2394c0eb55 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.282+0000 7fede27fc640 1 -- 172.21.15.172:0/3488298226 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc035020 con 0x7fede40d0a70 2024-04-14T08:38:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.282+0000 7fede27fc640 1 -- 172.21.15.172:0/3488298226 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedcc031870 con 0x7fede40d0a70 2024-04-14T08:38:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.282+0000 7fede27fc640 1 -- 172.21.15.172:0/3488298226 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc031c40 con 0x7fede40d0a70 2024-04-14T08:38:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.283+0000 7fede995e640 1 -- 172.21.15.172:0/3488298226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede40d0a70 msgr2=0x7fede40d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.283+0000 7fede995e640 1 --2- 172.21.15.172:0/3488298226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede40d0a70 0x7fede40d0eb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fede409da10 tx=0x7fedcc02b910 comp rx=0 tx=0).stop 2024-04-14T08:38:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.283+0000 7fede995e640 1 -- 172.21.15.172:0/3488298226 shutdown_connections 2024-04-14T08:38:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.283+0000 7fede995e640 1 --2- 172.21.15.172:0/3488298226 >> v2:172.21.15.177:3300/0 conn(0x7fede40b39b0 0x7fede40b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.283+0000 7fede995e640 1 --2- 172.21.15.172:0/3488298226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede40d0a70 0x7fede40d0eb0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.283+0000 7fede995e640 1 -- 172.21.15.172:0/3488298226 >> 172.21.15.172:0/3488298226 conn(0x7fede409d570 msgr2=0x7fede40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.283+0000 7fede995e640 1 -- 172.21.15.172:0/3488298226 shutdown_connections 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.283+0000 7fede995e640 1 -- 172.21.15.172:0/3488298226 wait complete. 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.284+0000 7fede995e640 1 Processor -- start 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.284+0000 7fede995e640 1 -- start start 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.284+0000 7fede995e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fede40b39b0 0x7fede40c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.284+0000 7fede995e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede40d0a70 0x7fede4152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.284+0000 7fede995e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fede415e4b0 0x7fede40c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.284+0000 7fede995e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fede40d2ab0 con 0x7fede40d0a70 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.284+0000 7fede995e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fede40d27d0 con 0x7fede415e4b0 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.284+0000 7fede995e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fede40d2fd0 con 0x7fede40b39b0 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7feddbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede40d0a70 0x7fede4152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7feddbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede40d0a70 0x7fede4152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35688/0 (socket says 172.21.15.172:35688) 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7feddbfff640 1 -- 172.21.15.172:0/309843588 learned_addr learned my addr 172.21.15.172:0/309843588 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7fede37fe640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fede415e4b0 0x7fede40c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7fede2ffd640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fede40b39b0 0x7fede40c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7feddbfff640 1 -- 172.21.15.172:0/309843588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fede40b39b0 msgr2=0x7fede40c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7feddbfff640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fede40b39b0 0x7fede40c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7feddbfff640 1 -- 172.21.15.172:0/309843588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fede415e4b0 msgr2=0x7fede40c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7feddbfff640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fede415e4b0 0x7fede40c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7feddbfff640 1 -- 172.21.15.172:0/309843588 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede40c9880 con 0x7fede40d0a70 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7fede37fe640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fede415e4b0 0x7fede40c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7fede2ffd640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fede40b39b0 0x7fede40c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7feddbfff640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede40d0a70 0x7fede4152640 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fedcc02b950 tx=0x7fedcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7fede0ff9640 1 -- 172.21.15.172:0/309843588 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc040070 con 0x7fede40d0a70 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7fede0ff9640 1 -- 172.21.15.172:0/309843588 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedcc03a420 con 0x7fede40d0a70 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7fede0ff9640 1 -- 172.21.15.172:0/309843588 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc035040 con 0x7fede40d0a70 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7fede995e640 1 -- 172.21.15.172:0/309843588 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede40c8300 con 0x7fede40d0a70 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.285+0000 7fede995e640 1 -- 172.21.15.172:0/309843588 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede40c8b40 con 0x7fede40d0a70 2024-04-14T08:38:34.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.286+0000 7fede995e640 1 -- 172.21.15.172:0/309843588 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede4068590 con 0x7fede40d0a70 2024-04-14T08:38:34.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.287+0000 7fede0ff9640 1 -- 172.21.15.172:0/309843588 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedcc04b060 con 0x7fede40d0a70 2024-04-14T08:38:34.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.287+0000 7fede0ff9640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fedc406c8f0 0x7fedc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:34.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.287+0000 7fede0ff9640 1 -- 172.21.15.172:0/309843588 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6273+0+0 (secure 0 0 0) 0x7fedcc0add70 con 0x7fede40d0a70 2024-04-14T08:38:34.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.287+0000 7fede2ffd640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fedc406c8f0 0x7fedc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:34.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.288+0000 7fede2ffd640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fedc406c8f0 0x7fedc406edb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fedd4001680 tx=0x7fedd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:34.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.291+0000 7fede0ff9640 1 -- 172.21.15.172:0/309843588 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fede40c8300 con 0x7fede40d0a70 2024-04-14T08:38:34.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:34.504+0000 7fede995e640 1 -- 172.21.15.172:0/309843588 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fede40b6330 con 0x7fede40d0a70 2024-04-14T08:38:35.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.160+0000 7fede0ff9640 1 -- 172.21.15.172:0/309843588 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7fedcc07bf60 con 0x7fede40d0a70 2024-04-14T08:38:35.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.163+0000 7fede995e640 1 -- 172.21.15.172:0/309843588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fedc406c8f0 msgr2=0x7fedc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:35.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.163+0000 7fede995e640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fedc406c8f0 0x7fedc406edb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fedd4001680 tx=0x7fedd4009040 comp rx=0 tx=0).stop 2024-04-14T08:38:35.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.163+0000 7fede995e640 1 -- 172.21.15.172:0/309843588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede40d0a70 msgr2=0x7fede4152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:35.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.163+0000 7fede995e640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede40d0a70 0x7fede4152640 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fedcc02b950 tx=0x7fedcc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:35.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.163+0000 7fede995e640 1 -- 172.21.15.172:0/309843588 shutdown_connections 2024-04-14T08:38:35.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.163+0000 7fede995e640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fedc406c8f0 0x7fedc406edb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:35.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.163+0000 7fede995e640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fede415e4b0 0x7fede40c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:35.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.163+0000 7fede995e640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede40d0a70 0x7fede4152640 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:35.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.163+0000 7fede995e640 1 --2- 172.21.15.172:0/309843588 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fede40b39b0 0x7fede40c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:35.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.163+0000 7fede995e640 1 -- 172.21.15.172:0/309843588 >> 172.21.15.172:0/309843588 conn(0x7fede409d570 msgr2=0x7fede40cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:35.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.163+0000 7fede995e640 1 -- 172.21.15.172:0/309843588 shutdown_connections 2024-04-14T08:38:35.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.163+0000 7fede995e640 1 -- 172.21.15.172:0/309843588 wait complete. 2024-04-14T08:38:35.177 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-14T08:38:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.326+0000 7f3006008640 1 Processor -- start 2024-04-14T08:38:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.326+0000 7f3006008640 1 -- start start 2024-04-14T08:38:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f3006008640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f30000cc060 0x7f30000c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f3006008640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f30000ca3c0 0x7f30000ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f3006008640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f300006d9f0 con 0x7f300015c2b0 2024-04-14T08:38:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f3006008640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f30000c6bf0 con 0x7f30000cc060 2024-04-14T08:38:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f3006008640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f30000cc580 con 0x7f30000ca3c0 2024-04-14T08:38:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f2ffeffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f30000cc060 0x7f30000c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f2fff7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f300015c2b0 0x7f300006d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54662/0 (socket says 172.21.15.172:54662) 2024-04-14T08:38:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f2fff7fe640 1 -- 172.21.15.172:0/873920018 learned_addr learned my addr 172.21.15.172:0/873920018 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f2ffeffd640 1 -- 172.21.15.172:0/873920018 >> v2:172.21.15.177:3300/0 conn(0x7f30000ca3c0 msgr2=0x7f30000ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f2ffeffd640 1 --2- 172.21.15.172:0/873920018 >> v2:172.21.15.177:3300/0 conn(0x7f30000ca3c0 0x7f30000ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f2ffeffd640 1 -- 172.21.15.172:0/873920018 >> v1:172.21.15.172:6789/0 conn(0x7f300015c2b0 legacy=0x7f300006d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f2ffeffd640 1 -- 172.21.15.172:0/873920018 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3000160ae0 con 0x7f30000cc060 2024-04-14T08:38:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.327+0000 7f2ffe7fc640 1 -- 172.21.15.172:0/873920018 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1598474306 0 0) 0x7f300006d9f0 con 0x7f300015c2b0 2024-04-14T08:38:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.328+0000 7f2ffeffd640 1 --2- 172.21.15.172:0/873920018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30000cc060 0x7f30000c5ee0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f30000af970 tx=0x7f2ff402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda40a27e2a0cafd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.328+0000 7f2ffe7fc640 1 -- 172.21.15.172:0/873920018 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4035020 con 0x7f30000cc060 2024-04-14T08:38:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.328+0000 7f2ffe7fc640 1 -- 172.21.15.172:0/873920018 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff4031830 con 0x7f30000cc060 2024-04-14T08:38:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.328+0000 7f2ffe7fc640 1 -- 172.21.15.172:0/873920018 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4031c00 con 0x7f30000cc060 2024-04-14T08:38:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.328+0000 7f3006008640 1 -- 172.21.15.172:0/873920018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30000cc060 msgr2=0x7f30000c5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.328+0000 7f3006008640 1 --2- 172.21.15.172:0/873920018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30000cc060 0x7f30000c5ee0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f30000af970 tx=0x7f2ff402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:38:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.328+0000 7f3006008640 1 -- 172.21.15.172:0/873920018 shutdown_connections 2024-04-14T08:38:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.328+0000 7f3006008640 1 --2- 172.21.15.172:0/873920018 >> v2:172.21.15.177:3300/0 conn(0x7f30000ca3c0 0x7f30000ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.328+0000 7f3006008640 1 --2- 172.21.15.172:0/873920018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30000cc060 0x7f30000c5ee0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.328+0000 7f3006008640 1 -- 172.21.15.172:0/873920018 >> 172.21.15.172:0/873920018 conn(0x7f30000b1a50 msgr2=0x7f30000b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.329+0000 7f3006008640 1 -- 172.21.15.172:0/873920018 shutdown_connections 2024-04-14T08:38:35.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.329+0000 7f3006008640 1 -- 172.21.15.172:0/873920018 wait complete. 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.330+0000 7f3006008640 1 Processor -- start 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.330+0000 7f3006008640 1 -- start start 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f3006008640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30000ca3c0 0x7f300015fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f3006008640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30000cc060 0x7f30001577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f3006008640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300015c2b0 0x7f3000153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f3006008640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30001607d0 con 0x7f30000cc060 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f3006008640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f300015d6a0 con 0x7f30000ca3c0 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f3006008640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f30000b26f0 con 0x7f300015c2b0 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2ffeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30000cc060 0x7f30001577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2ffeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30000cc060 0x7f30001577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35702/0 (socket says 172.21.15.172:35702) 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2ffeffd640 1 -- 172.21.15.172:0/2614617222 learned_addr learned my addr 172.21.15.172:0/2614617222 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2fff7fe640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30000ca3c0 0x7f300015fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2ffeffd640 1 -- 172.21.15.172:0/2614617222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300015c2b0 msgr2=0x7f3000153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2ffffff640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300015c2b0 0x7f3000153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2ffeffd640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300015c2b0 0x7f3000153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2ffeffd640 1 -- 172.21.15.172:0/2614617222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30000ca3c0 msgr2=0x7f300015fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2ffeffd640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30000ca3c0 0x7f300015fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2ffeffd640 1 -- 172.21.15.172:0/2614617222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3000157d20 con 0x7f30000cc060 2024-04-14T08:38:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2fff7fe640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30000ca3c0 0x7f300015fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2ffeffd640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30000cc060 0x7f30001577e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009890 tx=0x7f2ff40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2fe7fff640 1 -- 172.21.15.172:0/2614617222 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4004230 con 0x7f30000cc060 2024-04-14T08:38:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2fe7fff640 1 -- 172.21.15.172:0/2614617222 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff40043d0 con 0x7f30000cc060 2024-04-14T08:38:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f3006008640 1 -- 172.21.15.172:0/2614617222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3000157fc0 con 0x7f30000cc060 2024-04-14T08:38:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f2fe7fff640 1 -- 172.21.15.172:0/2614617222 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff403cad0 con 0x7f30000cc060 2024-04-14T08:38:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.331+0000 7f3006008640 1 -- 172.21.15.172:0/2614617222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300015f210 con 0x7f30000cc060 2024-04-14T08:38:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.332+0000 7f3006008640 1 -- 172.21.15.172:0/2614617222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30000686a0 con 0x7f30000cc060 2024-04-14T08:38:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.333+0000 7f2fe7fff640 1 -- 172.21.15.172:0/2614617222 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ff4038020 con 0x7f30000cc060 2024-04-14T08:38:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.333+0000 7f2fe7fff640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2fec06c8f0 0x7f2fec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.333+0000 7f2fff7fe640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2fec06c8f0 0x7f2fec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.333+0000 7f2fe7fff640 1 -- 172.21.15.172:0/2614617222 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6273+0+0 (secure 0 0 0) 0x7f2ff40ae080 con 0x7f30000cc060 2024-04-14T08:38:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.334+0000 7f2fff7fe640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2fec06c8f0 0x7f2fec06edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0000f60 tx=0x7f2ff0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.338+0000 7f2fe7fff640 1 -- 172.21.15.172:0/2614617222 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2ff4041070 con 0x7f30000cc060 2024-04-14T08:38:35.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:35.512+0000 7f3006008640 1 -- 172.21.15.172:0/2614617222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f300006cf80 con 0x7f30000cc060 2024-04-14T08:38:36.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.165+0000 7f2fe7fff640 1 -- 172.21.15.172:0/2614617222 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) ==== 181+0+0 (secure 0 0 0) 0x7f2ff407c390 con 0x7f30000cc060 2024-04-14T08:38:36.166 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' removed 2024-04-14T08:38:36.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.168+0000 7f3006008640 1 -- 172.21.15.172:0/2614617222 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2fec06c8f0 msgr2=0x7f2fec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.168+0000 7f3006008640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2fec06c8f0 0x7f2fec06edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0000f60 tx=0x7f2ff0009040 comp rx=0 tx=0).stop 2024-04-14T08:38:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.168+0000 7f3006008640 1 -- 172.21.15.172:0/2614617222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30000cc060 msgr2=0x7f30001577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.168+0000 7f3006008640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30000cc060 0x7f30001577e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009890 tx=0x7f2ff40046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.168+0000 7f3006008640 1 -- 172.21.15.172:0/2614617222 shutdown_connections 2024-04-14T08:38:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.168+0000 7f3006008640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2fec06c8f0 0x7f2fec06edb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.169+0000 7f3006008640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f300015c2b0 0x7f3000153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.169+0000 7f3006008640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30000cc060 0x7f30001577e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.169+0000 7f3006008640 1 --2- 172.21.15.172:0/2614617222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30000ca3c0 0x7f300015fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.169+0000 7f3006008640 1 -- 172.21.15.172:0/2614617222 >> 172.21.15.172:0/2614617222 conn(0x7f30000b1a50 msgr2=0x7f30000af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.169+0000 7f3006008640 1 -- 172.21.15.172:0/2614617222 shutdown_connections 2024-04-14T08:38:36.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.169+0000 7f3006008640 1 -- 172.21.15.172:0/2614617222 wait complete. 2024-04-14T08:38:36.183 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-14T08:38:36.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.291+0000 7faea422d640 1 Processor -- start 2024-04-14T08:38:36.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.291+0000 7faea422d640 1 -- start start 2024-04-14T08:38:36.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.292+0000 7faea422d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fae9c0cc340 0x7fae9c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:36.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.292+0000 7faea422d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fae9c0cecc0 0x7fae9c0c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:36.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.292+0000 7faea422d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae9c0cf0f0 con 0x7fae9c15b350 2024-04-14T08:38:36.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.292+0000 7faea422d640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fae9c0cd940 con 0x7fae9c0cecc0 2024-04-14T08:38:36.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.292+0000 7faea422d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fae9c0ce4c0 con 0x7fae9c0cc340 2024-04-14T08:38:36.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.292+0000 7faea27ca640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fae9c0cecc0 0x7fae9c0c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:36.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.292+0000 7faea27ca640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fae9c0cecc0 0x7fae9c0c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35708/0 (socket says 172.21.15.172:35708) 2024-04-14T08:38:36.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.292+0000 7faea27ca640 1 -- 172.21.15.172:0/947687300 learned_addr learned my addr 172.21.15.172:0/947687300 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:36.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.292+0000 7faea27ca640 1 -- 172.21.15.172:0/947687300 >> v2:172.21.15.177:3300/0 conn(0x7fae9c0cc340 msgr2=0x7fae9c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:36.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.292+0000 7faea27ca640 1 --2- 172.21.15.172:0/947687300 >> v2:172.21.15.177:3300/0 conn(0x7fae9c0cc340 0x7fae9c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:36.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.292+0000 7faea27ca640 1 -- 172.21.15.172:0/947687300 >> v1:172.21.15.177:6789/0 conn(0x7fae9c15b350 legacy=0x7fae9c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:36.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.292+0000 7faea27ca640 1 -- 172.21.15.172:0/947687300 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9c0cdb90 con 0x7fae9c0cecc0 2024-04-14T08:38:36.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.293+0000 7faea27ca640 1 --2- 172.21.15.172:0/947687300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9c0cecc0 0x7fae9c0c4e70 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0b25f0 tx=0x7fae9402e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29481cedd5fe89 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:36.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.293+0000 7faea0fc7640 1 -- 172.21.15.172:0/947687300 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9402ea00 con 0x7fae9c0cecc0 2024-04-14T08:38:36.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.293+0000 7faea0fc7640 1 -- 172.21.15.172:0/947687300 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae9402eba0 con 0x7fae9c0cecc0 2024-04-14T08:38:36.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.293+0000 7faea0fc7640 1 -- 172.21.15.172:0/947687300 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94009600 con 0x7fae9c0cecc0 2024-04-14T08:38:36.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.294+0000 7faea422d640 1 -- 172.21.15.172:0/947687300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9c0cecc0 msgr2=0x7fae9c0c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:36.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.294+0000 7faea422d640 1 --2- 172.21.15.172:0/947687300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9c0cecc0 0x7fae9c0c4e70 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0b25f0 tx=0x7fae9402e060 comp rx=0 tx=0).stop 2024-04-14T08:38:36.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.294+0000 7faea422d640 1 -- 172.21.15.172:0/947687300 shutdown_connections 2024-04-14T08:38:36.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.294+0000 7faea422d640 1 --2- 172.21.15.172:0/947687300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9c0cecc0 0x7fae9c0c4e70 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:36.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.294+0000 7faea422d640 1 --2- 172.21.15.172:0/947687300 >> v2:172.21.15.177:3300/0 conn(0x7fae9c0cc340 0x7fae9c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:36.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.294+0000 7faea422d640 1 -- 172.21.15.172:0/947687300 >> 172.21.15.172:0/947687300 conn(0x7fae9c0b1a50 msgr2=0x7fae9c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:36.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.294+0000 7faea422d640 1 -- 172.21.15.172:0/947687300 shutdown_connections 2024-04-14T08:38:36.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.294+0000 7faea422d640 1 -- 172.21.15.172:0/947687300 wait complete. 2024-04-14T08:38:36.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.295+0000 7faea422d640 1 Processor -- start 2024-04-14T08:38:36.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.295+0000 7faea422d640 1 -- start start 2024-04-14T08:38:36.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea422d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9c0cc340 0x7fae9c15def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:36.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea422d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae9c0cecc0 0x7fae9c1f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:36.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea422d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae9c15b350 0x7fae9c1f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea422d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c0cf560 con 0x7fae9c0cc340 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea422d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c0ce150 con 0x7fae9c0cecc0 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea422d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fae9c0c53b0 con 0x7fae9c15b350 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea1fc9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9c0cc340 0x7fae9c15def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea1fc9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9c0cc340 0x7fae9c15def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35710/0 (socket says 172.21.15.172:35710) 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea1fc9640 1 -- 172.21.15.172:0/312309553 learned_addr learned my addr 172.21.15.172:0/312309553 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea1fc9640 1 -- 172.21.15.172:0/312309553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae9c15b350 msgr2=0x7fae9c1f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea1fc9640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae9c15b350 0x7fae9c1f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea1fc9640 1 -- 172.21.15.172:0/312309553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae9c0cecc0 msgr2=0x7fae9c1f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea17c8640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae9c0cecc0 0x7fae9c1f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea1fc9640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae9c0cecc0 0x7fae9c1f33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea1fc9640 1 -- 172.21.15.172:0/312309553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9c072320 con 0x7fae9c0cc340 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea17c8640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae9c0cecc0 0x7fae9c1f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:36.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.296+0000 7faea1fc9640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9c0cc340 0x7fae9c15def0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fae9000b410 tx=0x7fae9000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:36.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.297+0000 7fae8e7fc640 1 -- 172.21.15.172:0/312309553 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90015020 con 0x7fae9c0cc340 2024-04-14T08:38:36.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.297+0000 7fae8e7fc640 1 -- 172.21.15.172:0/312309553 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae90011830 con 0x7fae9c0cc340 2024-04-14T08:38:36.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.297+0000 7faea422d640 1 -- 172.21.15.172:0/312309553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9c072600 con 0x7fae9c0cc340 2024-04-14T08:38:36.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.297+0000 7fae8e7fc640 1 -- 172.21.15.172:0/312309553 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90011c20 con 0x7fae9c0cc340 2024-04-14T08:38:36.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.297+0000 7faea422d640 1 -- 172.21.15.172:0/312309553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae9c072bc0 con 0x7fae9c0cc340 2024-04-14T08:38:36.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.298+0000 7faea422d640 1 -- 172.21.15.172:0/312309553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9c0686a0 con 0x7fae9c0cc340 2024-04-14T08:38:36.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.298+0000 7fae8e7fc640 1 -- 172.21.15.172:0/312309553 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae90018020 con 0x7fae9c0cc340 2024-04-14T08:38:36.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.298+0000 7fae8e7fc640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fae8006c8f0 0x7fae8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:36.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.299+0000 7fae8e7fc640 1 -- 172.21.15.172:0/312309553 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5835+0+0 (secure 0 0 0) 0x7fae9008bab0 con 0x7fae9c0cc340 2024-04-14T08:38:36.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.299+0000 7faea17c8640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fae8006c8f0 0x7fae8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:36.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.299+0000 7faea17c8640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fae8006c8f0 0x7fae8006edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fae98006fd0 tx=0x7fae98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:36.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.303+0000 7fae8e7fc640 1 -- 172.21.15.172:0/312309553 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fae9001f020 con 0x7fae9c0cc340 2024-04-14T08:38:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:36.477+0000 7faea422d640 1 -- 172.21.15.172:0/312309553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fae9c0cf810 con 0x7fae9c0cc340 2024-04-14T08:38:37.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.167+0000 7fae8e7fc640 1 -- 172.21.15.172:0/312309553 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) ==== 169+0+0 (secure 0 0 0) 0x7fae900172c0 con 0x7fae9c0cc340 2024-04-14T08:38:37.168 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' removed 2024-04-14T08:38:37.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.170+0000 7faea422d640 1 -- 172.21.15.172:0/312309553 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fae8006c8f0 msgr2=0x7fae8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:37.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.170+0000 7faea422d640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fae8006c8f0 0x7fae8006edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fae98006fd0 tx=0x7fae98008040 comp rx=0 tx=0).stop 2024-04-14T08:38:37.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.170+0000 7faea422d640 1 -- 172.21.15.172:0/312309553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9c0cc340 msgr2=0x7fae9c15def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:37.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.170+0000 7faea422d640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9c0cc340 0x7fae9c15def0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fae9000b410 tx=0x7fae9000b8d0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.170+0000 7faea422d640 1 -- 172.21.15.172:0/312309553 shutdown_connections 2024-04-14T08:38:37.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.170+0000 7faea422d640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fae8006c8f0 0x7fae8006edb0 secure :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fae98006fd0 tx=0x7fae98008040 comp rx=0 tx=0).stop 2024-04-14T08:38:37.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.170+0000 7faea422d640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae9c15b350 0x7fae9c1f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.170+0000 7faea422d640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae9c0cecc0 0x7fae9c1f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.170+0000 7faea422d640 1 --2- 172.21.15.172:0/312309553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae9c0cc340 0x7fae9c15def0 secure :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fae9000b410 tx=0x7fae9000b8d0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.170+0000 7faea422d640 1 -- 172.21.15.172:0/312309553 >> 172.21.15.172:0/312309553 conn(0x7fae9c0b1a50 msgr2=0x7fae9c0addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:37.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.172+0000 7faea422d640 1 -- 172.21.15.172:0/312309553 shutdown_connections 2024-04-14T08:38:37.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.172+0000 7faea422d640 1 -- 172.21.15.172:0/312309553 wait complete. 2024-04-14T08:38:37.188 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-14T08:38:37.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.295+0000 7f6d5cded640 1 Processor -- start 2024-04-14T08:38:37.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.296+0000 7f6d5cded640 1 -- start start 2024-04-14T08:38:37.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.296+0000 7f6d5cded640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d5813fa70 0x7f6d580af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:37.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.296+0000 7f6d5cded640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6d580ce5b0 0x7f6d5813fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:37.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.296+0000 7f6d5cded640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d58071520 con 0x7f6d580ce150 2024-04-14T08:38:37.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.296+0000 7f6d5cded640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f6d580712d0 con 0x7f6d580ce5b0 2024-04-14T08:38:37.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.296+0000 7f6d5cded640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6d58071800 con 0x7f6d5813fa70 2024-04-14T08:38:37.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.297+0000 7f6d56d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6d580ce5b0 0x7f6d5813fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:37.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.297+0000 7f6d56d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6d580ce5b0 0x7f6d5813fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35722/0 (socket says 172.21.15.172:35722) 2024-04-14T08:38:37.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.297+0000 7f6d56d9d640 1 -- 172.21.15.172:0/1321320979 learned_addr learned my addr 172.21.15.172:0/1321320979 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:37.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.297+0000 7f6d5659c640 1 --2- 172.21.15.172:0/1321320979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d5813fa70 0x7f6d580af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:37.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.297+0000 7f6d56d9d640 1 -- 172.21.15.172:0/1321320979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d5813fa70 msgr2=0x7f6d580af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:37.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.297+0000 7f6d56d9d640 1 --2- 172.21.15.172:0/1321320979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d5813fa70 0x7f6d580af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.297+0000 7f6d56d9d640 1 -- 172.21.15.172:0/1321320979 >> v1:172.21.15.177:6789/0 conn(0x7f6d580ce150 legacy=0x7f6d580722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:37.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.297+0000 7f6d56d9d640 1 -- 172.21.15.172:0/1321320979 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d58071ae0 con 0x7f6d580ce5b0 2024-04-14T08:38:37.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.297+0000 7f6d5659c640 1 --2- 172.21.15.172:0/1321320979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d5813fa70 0x7f6d580af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:37.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.298+0000 7f6d56d9d640 1 --2- 172.21.15.172:0/1321320979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d580ce5b0 0x7f6d5813fe50 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000d150 tx=0x7f6d5002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e08c4ef0b76017 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:37.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.298+0000 7f6d55d9b640 1 -- 172.21.15.172:0/1321320979 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50034070 con 0x7f6d580ce5b0 2024-04-14T08:38:37.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.298+0000 7f6d55d9b640 1 -- 172.21.15.172:0/1321320979 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d50038400 con 0x7f6d580ce5b0 2024-04-14T08:38:37.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.298+0000 7f6d55d9b640 1 -- 172.21.15.172:0/1321320979 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d500387d0 con 0x7f6d580ce5b0 2024-04-14T08:38:37.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.298+0000 7f6d5cded640 1 -- 172.21.15.172:0/1321320979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d580ce5b0 msgr2=0x7f6d5813fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:37.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.298+0000 7f6d5cded640 1 --2- 172.21.15.172:0/1321320979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d580ce5b0 0x7f6d5813fe50 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000d150 tx=0x7f6d5002e0e0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.298+0000 7f6d5cded640 1 -- 172.21.15.172:0/1321320979 shutdown_connections 2024-04-14T08:38:37.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.298+0000 7f6d5cded640 1 --2- 172.21.15.172:0/1321320979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d580ce5b0 0x7f6d5813fe50 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.298+0000 7f6d5cded640 1 --2- 172.21.15.172:0/1321320979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d5813fa70 0x7f6d580af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.298+0000 7f6d5cded640 1 -- 172.21.15.172:0/1321320979 >> 172.21.15.172:0/1321320979 conn(0x7f6d5806b520 msgr2=0x7f6d5806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:37.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.298+0000 7f6d5cded640 1 -- 172.21.15.172:0/1321320979 shutdown_connections 2024-04-14T08:38:37.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.298+0000 7f6d5cded640 1 -- 172.21.15.172:0/1321320979 wait complete. 2024-04-14T08:38:37.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.300+0000 7f6d5cded640 1 Processor -- start 2024-04-14T08:38:37.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.300+0000 7f6d5cded640 1 -- start start 2024-04-14T08:38:37.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.300+0000 7f6d5cded640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d580ce150 0x7f6d581582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.300+0000 7f6d5cded640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d580ce5b0 0x7f6d581410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.300+0000 7f6d5cded640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d5813fa70 0x7f6d5815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.300+0000 7f6d5cded640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581442b0 con 0x7f6d5813fa70 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.300+0000 7f6d5cded640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58144130 con 0x7f6d580ce5b0 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.300+0000 7f6d5cded640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6d580729f0 con 0x7f6d580ce150 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d56d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d5813fa70 0x7f6d5815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d56d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d5813fa70 0x7f6d5815ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35728/0 (socket says 172.21.15.172:35728) 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d56d9d640 1 -- 172.21.15.172:0/3182307109 learned_addr learned my addr 172.21.15.172:0/3182307109 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d5659c640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d580ce150 0x7f6d581582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d4ffff640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d580ce5b0 0x7f6d581410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d56d9d640 1 -- 172.21.15.172:0/3182307109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d580ce150 msgr2=0x7f6d581582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d56d9d640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d580ce150 0x7f6d581582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d56d9d640 1 -- 172.21.15.172:0/3182307109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d580ce5b0 msgr2=0x7f6d581410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d56d9d640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d580ce5b0 0x7f6d581410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d56d9d640 1 -- 172.21.15.172:0/3182307109 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d581418f0 con 0x7f6d5813fa70 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d56d9d640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d5813fa70 0x7f6d5815ab70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000d270 tx=0x7f6d50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d4effd640 1 -- 172.21.15.172:0/3182307109 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50047070 con 0x7f6d5813fa70 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d4effd640 1 -- 172.21.15.172:0/3182307109 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d50037b90 con 0x7f6d5813fa70 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d4effd640 1 -- 172.21.15.172:0/3182307109 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5000b720 con 0x7f6d5813fa70 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d5cded640 1 -- 172.21.15.172:0/3182307109 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d58141bd0 con 0x7f6d5813fa70 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.301+0000 7f6d5cded640 1 -- 172.21.15.172:0/3182307109 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d58142190 con 0x7f6d5813fa70 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.302+0000 7f6d5cded640 1 -- 172.21.15.172:0/3182307109 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d58068590 con 0x7f6d5813fa70 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.303+0000 7f6d4effd640 1 -- 172.21.15.172:0/3182307109 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d5004a070 con 0x7f6d5813fa70 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.303+0000 7f6d4effd640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3806c8f0 0x7f6d3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.303+0000 7f6d4effd640 1 -- 172.21.15.172:0/3182307109 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f6d500b04d0 con 0x7f6d5813fa70 2024-04-14T08:38:37.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.303+0000 7f6d5659c640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3806c8f0 0x7f6d3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:37.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.303+0000 7f6d5659c640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3806c8f0 0x7f6d3806edb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6d48006e80 tx=0x7f6d48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:37.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.307+0000 7f6d4effd640 1 -- 172.21.15.172:0/3182307109 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6d5007e460 con 0x7f6d5813fa70 2024-04-14T08:38:37.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.483+0000 7f6d5cded640 1 -- 172.21.15.172:0/3182307109 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f6d580cc510 con 0x7f6d5813fa70 2024-04-14T08:38:37.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.486+0000 7f6d4effd640 1 -- 172.21.15.172:0/3182307109 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f6d50082760 con 0x7f6d5813fa70 2024-04-14T08:38:37.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.489+0000 7f6d5cded640 1 -- 172.21.15.172:0/3182307109 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3806c8f0 msgr2=0x7f6d3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:37.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.489+0000 7f6d5cded640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3806c8f0 0x7f6d3806edb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6d48006e80 tx=0x7f6d48008040 comp rx=0 tx=0).stop 2024-04-14T08:38:37.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.489+0000 7f6d5cded640 1 -- 172.21.15.172:0/3182307109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d5813fa70 msgr2=0x7f6d5815ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:37.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.489+0000 7f6d5cded640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d5813fa70 0x7f6d5815ab70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000d270 tx=0x7f6d50004a40 comp rx=0 tx=0).stop 2024-04-14T08:38:37.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.489+0000 7f6d5cded640 1 -- 172.21.15.172:0/3182307109 shutdown_connections 2024-04-14T08:38:37.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.489+0000 7f6d5cded640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3806c8f0 0x7f6d3806edb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.489+0000 7f6d5cded640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d5813fa70 0x7f6d5815ab70 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.489+0000 7f6d5cded640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d580ce5b0 0x7f6d581410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.489+0000 7f6d5cded640 1 --2- 172.21.15.172:0/3182307109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d580ce150 0x7f6d581582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.489+0000 7f6d5cded640 1 -- 172.21.15.172:0/3182307109 >> 172.21.15.172:0/3182307109 conn(0x7f6d5806b520 msgr2=0x7f6d58140850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:37.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.489+0000 7f6d5cded640 1 -- 172.21.15.172:0/3182307109 shutdown_connections 2024-04-14T08:38:37.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.489+0000 7f6d5cded640 1 -- 172.21.15.172:0/3182307109 wait complete. 2024-04-14T08:38:37.500 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-14T08:38:37.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.601+0000 7f676b8b7640 1 Processor -- start 2024-04-14T08:38:37.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.601+0000 7f676b8b7640 1 -- start start 2024-04-14T08:38:37.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f676b8b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f676415e5b0 0x7f67640ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:37.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f676b8b7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f67640729b0 0x7f676406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:37.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f676b8b7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67640b41d0 con 0x7f676406e160 2024-04-14T08:38:37.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f676b8b7640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f67640aebe0 con 0x7f67640729b0 2024-04-14T08:38:37.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f676b8b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f67640b4470 con 0x7f676415e5b0 2024-04-14T08:38:37.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f6768e52640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f67640729b0 0x7f676406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:37.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f6769653640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f676415e5b0 0x7f67640ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:37.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f6769e54640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f676406e160 0x7f676406c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:42042/0 (socket says 172.21.15.172:42042) 2024-04-14T08:38:37.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f6769e54640 1 -- 172.21.15.172:0/849507485 learned_addr learned my addr 172.21.15.172:0/849507485 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:37.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f6769653640 1 -- 172.21.15.172:0/849507485 >> v2:172.21.15.177:3300/0 conn(0x7f67640729b0 msgr2=0x7f676406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:37.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f6769653640 1 --2- 172.21.15.172:0/849507485 >> v2:172.21.15.177:3300/0 conn(0x7f67640729b0 0x7f676406dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f6769653640 1 -- 172.21.15.172:0/849507485 >> v1:172.21.15.177:6789/0 conn(0x7f676406e160 legacy=0x7f676406c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:37.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f6769653640 1 -- 172.21.15.172:0/849507485 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67640ae8d0 con 0x7f676415e5b0 2024-04-14T08:38:37.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.602+0000 7f6768e52640 1 --2- 172.21.15.172:0/849507485 >> v2:172.21.15.177:3300/0 conn(0x7f67640729b0 0x7f676406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:37.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.603+0000 7f6769653640 1 --2- 172.21.15.172:0/849507485 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f676415e5b0 0x7f67640ae390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f67600097a0 tx=0x7f676002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4eb44dbb041425d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:37.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.603+0000 7f675bfff640 1 -- 172.21.15.172:0/849507485 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760035020 con 0x7f676415e5b0 2024-04-14T08:38:37.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.603+0000 7f675bfff640 1 -- 172.21.15.172:0/849507485 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6760031830 con 0x7f676415e5b0 2024-04-14T08:38:37.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.604+0000 7f675bfff640 1 -- 172.21.15.172:0/849507485 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760031c00 con 0x7f676415e5b0 2024-04-14T08:38:37.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.604+0000 7f676b8b7640 1 -- 172.21.15.172:0/849507485 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f676415e5b0 msgr2=0x7f67640ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:37.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.604+0000 7f676b8b7640 1 --2- 172.21.15.172:0/849507485 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f676415e5b0 0x7f67640ae390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f67600097a0 tx=0x7f676002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.604+0000 7f676b8b7640 1 -- 172.21.15.172:0/849507485 shutdown_connections 2024-04-14T08:38:37.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.604+0000 7f676b8b7640 1 --2- 172.21.15.172:0/849507485 >> v2:172.21.15.177:3300/0 conn(0x7f67640729b0 0x7f676406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.604+0000 7f676b8b7640 1 --2- 172.21.15.172:0/849507485 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f676415e5b0 0x7f67640ae390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.604+0000 7f676b8b7640 1 -- 172.21.15.172:0/849507485 >> 172.21.15.172:0/849507485 conn(0x7f6764071ba0 msgr2=0x7f6764071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:37.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.604+0000 7f676b8b7640 1 -- 172.21.15.172:0/849507485 shutdown_connections 2024-04-14T08:38:37.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.604+0000 7f676b8b7640 1 -- 172.21.15.172:0/849507485 wait complete. 2024-04-14T08:38:37.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.605+0000 7f676b8b7640 1 Processor -- start 2024-04-14T08:38:37.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.605+0000 7f676b8b7640 1 -- start start 2024-04-14T08:38:37.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f676b8b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f676406e160 0x7f676415fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:37.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f676b8b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67640729b0 0x7f6764158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:37.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f676b8b7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f676415e5b0 0x7f67640cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:37.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f676b8b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f676406cdc0 con 0x7f67640729b0 2024-04-14T08:38:37.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f676b8b7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f676406cc40 con 0x7f676415e5b0 2024-04-14T08:38:37.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f676b8b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f67640b4c70 con 0x7f676406e160 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f6768e52640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67640729b0 0x7f6764158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f6768e52640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67640729b0 0x7f6764158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35738/0 (socket says 172.21.15.172:35738) 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f6768e52640 1 -- 172.21.15.172:0/857504636 learned_addr learned my addr 172.21.15.172:0/857504636 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f6769e54640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f676415e5b0 0x7f67640cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f6769653640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f676406e160 0x7f676415fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f6768e52640 1 -- 172.21.15.172:0/857504636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f676406e160 msgr2=0x7f676415fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f6768e52640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f676406e160 0x7f676415fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f6768e52640 1 -- 172.21.15.172:0/857504636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f676415e5b0 msgr2=0x7f67640cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f6768e52640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f676415e5b0 0x7f67640cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f6768e52640 1 -- 172.21.15.172:0/857504636 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676415ed50 con 0x7f67640729b0 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.606+0000 7f6769e54640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f676415e5b0 0x7f67640cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.607+0000 7f6768e52640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67640729b0 0x7f6764158a70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f675400ebb0 tx=0x7f675400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.607+0000 7f6759ffb640 1 -- 172.21.15.172:0/857504636 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754017070 con 0x7f67640729b0 2024-04-14T08:38:37.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.607+0000 7f6759ffb640 1 -- 172.21.15.172:0/857504636 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f675400cc30 con 0x7f67640729b0 2024-04-14T08:38:37.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.607+0000 7f676b8b7640 1 -- 172.21.15.172:0/857504636 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676415f030 con 0x7f67640729b0 2024-04-14T08:38:37.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.607+0000 7f6759ffb640 1 -- 172.21.15.172:0/857504636 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754013470 con 0x7f67640729b0 2024-04-14T08:38:37.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.607+0000 7f676b8b7640 1 -- 172.21.15.172:0/857504636 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676415f5f0 con 0x7f67640729b0 2024-04-14T08:38:37.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.608+0000 7f676b8b7640 1 -- 172.21.15.172:0/857504636 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676406dc20 con 0x7f67640729b0 2024-04-14T08:38:37.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.608+0000 7f6759ffb640 1 -- 172.21.15.172:0/857504636 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f675401a070 con 0x7f67640729b0 2024-04-14T08:38:37.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.609+0000 7f6759ffb640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f674806c8f0 0x7f674806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:37.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.609+0000 7f6759ffb640 1 -- 172.21.15.172:0/857504636 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f675408bca0 con 0x7f67640729b0 2024-04-14T08:38:37.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.609+0000 7f6769653640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f674806c8f0 0x7f674806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:37.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.609+0000 7f6769653640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f674806c8f0 0x7f674806edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6760036000 tx=0x7f676003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:37.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.613+0000 7f6759ffb640 1 -- 172.21.15.172:0/857504636 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f676406dc20 con 0x7f67640729b0 2024-04-14T08:38:37.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:37.858+0000 7f676b8b7640 1 -- 172.21.15.172:0/857504636 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f6764068c30 con 0x7f67640729b0 2024-04-14T08:38:38.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.171+0000 7f6759ffb640 1 -- 172.21.15.172:0/857504636 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v288) ==== 166+0+0 (secure 0 0 0) 0x7f6764068c30 con 0x7f67640729b0 2024-04-14T08:38:38.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.173+0000 7f676b8b7640 1 -- 172.21.15.172:0/857504636 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f674806c8f0 msgr2=0x7f674806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.173+0000 7f676b8b7640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f674806c8f0 0x7f674806edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6760036000 tx=0x7f676003c040 comp rx=0 tx=0).stop 2024-04-14T08:38:38.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.173+0000 7f676b8b7640 1 -- 172.21.15.172:0/857504636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67640729b0 msgr2=0x7f6764158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.173+0000 7f676b8b7640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67640729b0 0x7f6764158a70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f675400ebb0 tx=0x7f675400c520 comp rx=0 tx=0).stop 2024-04-14T08:38:38.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.173+0000 7f676b8b7640 1 -- 172.21.15.172:0/857504636 shutdown_connections 2024-04-14T08:38:38.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.173+0000 7f676b8b7640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f674806c8f0 0x7f674806edb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.173+0000 7f676b8b7640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f676415e5b0 0x7f67640cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.173+0000 7f676b8b7640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67640729b0 0x7f6764158a70 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.173+0000 7f676b8b7640 1 --2- 172.21.15.172:0/857504636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f676406e160 0x7f676415fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.173+0000 7f676b8b7640 1 -- 172.21.15.172:0/857504636 >> 172.21.15.172:0/857504636 conn(0x7f6764071ba0 msgr2=0x7f67640b5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:38.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.173+0000 7f676b8b7640 1 -- 172.21.15.172:0/857504636 shutdown_connections 2024-04-14T08:38:38.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.173+0000 7f676b8b7640 1 -- 172.21.15.172:0/857504636 wait complete. 2024-04-14T08:38:38.185 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-14T08:38:38.185 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-14T08:38:38.185 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-14T08:38:38.186 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:38:38.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.286+0000 7f9a85b55640 1 Processor -- start 2024-04-14T08:38:38.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.286+0000 7f9a85b55640 1 -- start start 2024-04-14T08:38:38.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.287+0000 7f9a85b55640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a800d2830 0x7f9a800d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.287+0000 7f9a85b55640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9a8015e890 0x7f9a800d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.287+0000 7f9a85b55640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a800cedd0 0x7f9a801621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.287+0000 7f9a85b55640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f9a800cff30 con 0x7f9a800cedd0 2024-04-14T08:38:38.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.287+0000 7f9a85b55640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f9a800d2350 con 0x7f9a8015e890 2024-04-14T08:38:38.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.287+0000 7f9a85b55640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9a800d3150 con 0x7f9a800d2830 2024-04-14T08:38:38.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.287+0000 7f9a7ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a800cedd0 0x7f9a801621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.287+0000 7f9a7ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a800cedd0 0x7f9a801621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35748/0 (socket says 172.21.15.172:35748) 2024-04-14T08:38:38.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.287+0000 7f9a7ffff640 1 -- 172.21.15.172:0/3002903400 learned_addr learned my addr 172.21.15.172:0/3002903400 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:38.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.288+0000 7f9a7ffff640 1 -- 172.21.15.172:0/3002903400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a800d2830 msgr2=0x7f9a800d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:38.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.288+0000 7f9a7ffff640 1 --2- 172.21.15.172:0/3002903400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a800d2830 0x7f9a800d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.288+0000 7f9a7ffff640 1 -- 172.21.15.172:0/3002903400 >> v2:172.21.15.177:3300/0 conn(0x7f9a8015e890 msgr2=0x7f9a800d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:38.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.288+0000 7f9a7ffff640 1 --2- 172.21.15.172:0/3002903400 >> v2:172.21.15.177:3300/0 conn(0x7f9a8015e890 0x7f9a800d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.288+0000 7f9a7ffff640 1 -- 172.21.15.172:0/3002903400 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a800cf320 con 0x7f9a800cedd0 2024-04-14T08:38:38.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.288+0000 7f9a7ffff640 1 --2- 172.21.15.172:0/3002903400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a800cedd0 0x7f9a801621a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9a8009dcc0 tx=0x7f9a7802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283b4c22791a560a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:38.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.289+0000 7f9a7effd640 1 -- 172.21.15.172:0/3002903400 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7802cb00 con 0x7f9a800cedd0 2024-04-14T08:38:38.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.289+0000 7f9a7effd640 1 -- 172.21.15.172:0/3002903400 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7802cca0 con 0x7f9a800cedd0 2024-04-14T08:38:38.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.289+0000 7f9a7effd640 1 -- 172.21.15.172:0/3002903400 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78034610 con 0x7f9a800cedd0 2024-04-14T08:38:38.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.289+0000 7f9a85b55640 1 -- 172.21.15.172:0/3002903400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a800cedd0 msgr2=0x7f9a801621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.289+0000 7f9a85b55640 1 --2- 172.21.15.172:0/3002903400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a800cedd0 0x7f9a801621a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9a8009dcc0 tx=0x7f9a7802bf60 comp rx=0 tx=0).stop 2024-04-14T08:38:38.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.289+0000 7f9a85b55640 1 -- 172.21.15.172:0/3002903400 shutdown_connections 2024-04-14T08:38:38.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.289+0000 7f9a85b55640 1 --2- 172.21.15.172:0/3002903400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a800cedd0 0x7f9a801621a0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.289+0000 7f9a85b55640 1 --2- 172.21.15.172:0/3002903400 >> v2:172.21.15.177:3300/0 conn(0x7f9a8015e890 0x7f9a800d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.289+0000 7f9a85b55640 1 --2- 172.21.15.172:0/3002903400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a800d2830 0x7f9a800d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.289+0000 7f9a85b55640 1 -- 172.21.15.172:0/3002903400 >> 172.21.15.172:0/3002903400 conn(0x7f9a8009d590 msgr2=0x7f9a800d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:38.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.289+0000 7f9a85b55640 1 -- 172.21.15.172:0/3002903400 shutdown_connections 2024-04-14T08:38:38.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.289+0000 7f9a85b55640 1 -- 172.21.15.172:0/3002903400 wait complete. 2024-04-14T08:38:38.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.290+0000 7f9a85b55640 1 Processor -- start 2024-04-14T08:38:38.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.291+0000 7f9a85b55640 1 -- start start 2024-04-14T08:38:38.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.291+0000 7f9a85b55640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a800cedd0 0x7f9a800cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.291+0000 7f9a85b55640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a800d2830 0x7f9a801607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.291+0000 7f9a85b55640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a8015e890 0x7f9a80160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.291+0000 7f9a85b55640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9a800cfbc0 con 0x7f9a8015e890 2024-04-14T08:38:38.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.291+0000 7f9a85b55640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9a800cf8e0 con 0x7f9a800cedd0 2024-04-14T08:38:38.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.291+0000 7f9a85b55640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9a800d1840 con 0x7f9a800d2830 2024-04-14T08:38:38.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.291+0000 7f9a7ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a8015e890 0x7f9a80160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.291+0000 7f9a7ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a8015e890 0x7f9a80160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35762/0 (socket says 172.21.15.172:35762) 2024-04-14T08:38:38.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.291+0000 7f9a7ffff640 1 -- 172.21.15.172:0/3888544794 learned_addr learned my addr 172.21.15.172:0/3888544794 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:38.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a7f7fe640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a800cedd0 0x7f9a800cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a76ffd640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a800d2830 0x7f9a801607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a7ffff640 1 -- 172.21.15.172:0/3888544794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a800d2830 msgr2=0x7f9a801607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a7ffff640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a800d2830 0x7f9a801607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a7ffff640 1 -- 172.21.15.172:0/3888544794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a800cedd0 msgr2=0x7f9a800cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a7ffff640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a800cedd0 0x7f9a800cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a7ffff640 1 -- 172.21.15.172:0/3888544794 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a800c9f80 con 0x7f9a8015e890 2024-04-14T08:38:38.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a7ffff640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a8015e890 0x7f9a80160ce0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9a78036040 tx=0x7f9a78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:38.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a7cff9640 1 -- 172.21.15.172:0/3888544794 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78043020 con 0x7f9a8015e890 2024-04-14T08:38:38.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a7cff9640 1 -- 172.21.15.172:0/3888544794 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7802ce50 con 0x7f9a8015e890 2024-04-14T08:38:38.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a85b55640 1 -- 172.21.15.172:0/3888544794 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a800ca220 con 0x7f9a8015e890 2024-04-14T08:38:38.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a7cff9640 1 -- 172.21.15.172:0/3888544794 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7803e670 con 0x7f9a8015e890 2024-04-14T08:38:38.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.292+0000 7f9a85b55640 1 -- 172.21.15.172:0/3888544794 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a800ca7e0 con 0x7f9a8015e890 2024-04-14T08:38:38.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.293+0000 7f9a85b55640 1 -- 172.21.15.172:0/3888544794 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a800b4790 con 0x7f9a8015e890 2024-04-14T08:38:38.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.294+0000 7f9a7cff9640 1 -- 172.21.15.172:0/3888544794 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a78046020 con 0x7f9a8015e890 2024-04-14T08:38:38.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.294+0000 7f9a7cff9640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a6006c8f0 0x7f9a6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.294+0000 7f9a7f7fe640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a6006c8f0 0x7f9a6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.294+0000 7f9a7cff9640 1 -- 172.21.15.172:0/3888544794 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f9a780ae230 con 0x7f9a8015e890 2024-04-14T08:38:38.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.295+0000 7f9a7f7fe640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a6006c8f0 0x7f9a6006edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000b810 tx=0x7f9a70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:38.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.299+0000 7f9a7cff9640 1 -- 172.21.15.172:0/3888544794 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9a800b4790 con 0x7f9a8015e890 2024-04-14T08:38:38.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.476+0000 7f9a85b55640 1 -- 172.21.15.172:0/3888544794 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a80068520 con 0x7f9a8015e890 2024-04-14T08:38:38.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.476+0000 7f9a7cff9640 1 -- 172.21.15.172:0/3888544794 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+13488 (secure 0 0 0) 0x7f9a7804d030 con 0x7f9a8015e890 2024-04-14T08:38:38.477 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:38.477 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":78,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:38:37.166040+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/2920402198":"2024-04-15T08:38:34.156568+0000","172.21.15.172:6835/2920402198":"2024-04-15T08:38:34.156568+0000","172.21.15.172:6837/2134832221":"2024-04-15T08:37:15.356472+0000","172.21.15.172:6835/2134832221":"2024-04-15T08:37:15.356472+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:38:38.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.478+0000 7f9a85b55640 1 -- 172.21.15.172:0/3888544794 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a6006c8f0 msgr2=0x7f9a6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.478+0000 7f9a85b55640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a6006c8f0 0x7f9a6006edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000b810 tx=0x7f9a70008040 comp rx=0 tx=0).stop 2024-04-14T08:38:38.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.478+0000 7f9a85b55640 1 -- 172.21.15.172:0/3888544794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a8015e890 msgr2=0x7f9a80160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.478+0000 7f9a85b55640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a8015e890 0x7f9a80160ce0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9a78036040 tx=0x7f9a78004a40 comp rx=0 tx=0).stop 2024-04-14T08:38:38.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.478+0000 7f9a85b55640 1 -- 172.21.15.172:0/3888544794 shutdown_connections 2024-04-14T08:38:38.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.478+0000 7f9a85b55640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a6006c8f0 0x7f9a6006edb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.478+0000 7f9a85b55640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a8015e890 0x7f9a80160ce0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.478+0000 7f9a85b55640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a800d2830 0x7f9a801607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.478+0000 7f9a85b55640 1 --2- 172.21.15.172:0/3888544794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a800cedd0 0x7f9a800cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.478+0000 7f9a85b55640 1 -- 172.21.15.172:0/3888544794 >> 172.21.15.172:0/3888544794 conn(0x7f9a8009d590 msgr2=0x7f9a8006c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:38.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.478+0000 7f9a85b55640 1 -- 172.21.15.172:0/3888544794 shutdown_connections 2024-04-14T08:38:38.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.478+0000 7f9a85b55640 1 -- 172.21.15.172:0/3888544794 wait complete. 2024-04-14T08:38:38.490 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:38:38.491 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:38:38.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.593+0000 7f21d4cc0640 1 Processor -- start 2024-04-14T08:38:38.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.593+0000 7f21d4cc0640 1 -- start start 2024-04-14T08:38:38.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.594+0000 7f21d4cc0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0159c20 0x7f21d00ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.594+0000 7f21d4cc0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f21d015c700 0x7f21d01603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.594+0000 7f21d4cc0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21d015cf10 con 0x7f21d0160930 2024-04-14T08:38:38.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.594+0000 7f21d4cc0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f21d015fc80 con 0x7f21d015c700 2024-04-14T08:38:38.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.594+0000 7f21d4cc0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f21d015ff60 con 0x7f21d0159c20 2024-04-14T08:38:38.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.594+0000 7f21ced9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f21d0160930 0x7f21d0072450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54678/0 (socket says 172.21.15.172:54678) 2024-04-14T08:38:38.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.594+0000 7f21ced9d640 1 -- 172.21.15.172:0/1757727800 learned_addr learned my addr 172.21.15.172:0/1757727800 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.594+0000 7f21c7fff640 1 --2- 172.21.15.172:0/1757727800 >> v2:172.21.15.177:3300/0 conn(0x7f21d015c700 0x7f21d01603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.594+0000 7f21ce59c640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0159c20 0x7f21d00ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.594+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 163173545 0 0) 0x7f21d015cf10 con 0x7f21d0160930 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.594+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21bc003520 con 0x7f21d0160930 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.594+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 272580667 0 0) 0x7f21bc003520 con 0x7f21d0160930 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0159c20 msgr2=0x7f21d00ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0159c20 0x7f21d00ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 >> v2:172.21.15.177:3300/0 conn(0x7f21d015c700 msgr2=0x7f21d01603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 --2- 172.21.15.172:0/1757727800 >> v2:172.21.15.177:3300/0 conn(0x7f21d015c700 0x7f21d01603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d015d1f0 con 0x7f21d0160930 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21ce59c640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0159c20 0x7f21d00ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f21c80019f0 con 0x7f21d0160930 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 >> v1:172.21.15.172:6789/0 conn(0x7f21d0160930 legacy=0x7f21d0072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21bc0039b0 0x7f21bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0159c20 0x7f21bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21bc004d40 0x7f21bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21d015fc80 con 0x7f21bc004d40 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f21d015ff60 con 0x7f21bc0039b0 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f21bc00b9c0 con 0x7f21d0159c20 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21ced9d640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21bc004d40 0x7f21bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21ce59c640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21bc0039b0 0x7f21bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21c7fff640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0159c20 0x7f21bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21ced9d640 1 -- 172.21.15.172:0/1757727800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0159c20 msgr2=0x7f21bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21ced9d640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0159c20 0x7f21bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21ced9d640 1 -- 172.21.15.172:0/1757727800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21bc0039b0 msgr2=0x7f21bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21ced9d640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21bc0039b0 0x7f21bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.595+0000 7f21ced9d640 1 -- 172.21.15.172:0/1757727800 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21bc003520 con 0x7f21bc004d40 2024-04-14T08:38:38.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.596+0000 7f21ced9d640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21bc004d40 0x7f21bc0051c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f21c800a8f0 tx=0x7f21bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c0a7e15f9c576e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:38.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.596+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c800a4c0 con 0x7f21bc004d40 2024-04-14T08:38:38.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.596+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c8034650 con 0x7f21bc004d40 2024-04-14T08:38:38.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.596+0000 7f21cdd9b640 1 -- 172.21.15.172:0/1757727800 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c8034a40 con 0x7f21bc004d40 2024-04-14T08:38:38.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.596+0000 7f21d4cc0640 1 -- 172.21.15.172:0/1757727800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21bc004d40 msgr2=0x7f21bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.596+0000 7f21d4cc0640 1 --2- 172.21.15.172:0/1757727800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21bc004d40 0x7f21bc0051c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f21c800a8f0 tx=0x7f21bc005830 comp rx=0 tx=0).stop 2024-04-14T08:38:38.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.596+0000 7f21ced9d640 1 -- 172.21.15.172:0/1757727800 reap_dead start 2024-04-14T08:38:38.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.596+0000 7f21d4cc0640 1 -- 172.21.15.172:0/1757727800 shutdown_connections 2024-04-14T08:38:38.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.596+0000 7f21d4cc0640 1 -- 172.21.15.172:0/1757727800 >> 172.21.15.172:0/1757727800 conn(0x7f21d006b520 msgr2=0x7f21d006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:38.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.597+0000 7f21d4cc0640 1 -- 172.21.15.172:0/1757727800 shutdown_connections 2024-04-14T08:38:38.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.597+0000 7f21d4cc0640 1 -- 172.21.15.172:0/1757727800 wait complete. 2024-04-14T08:38:38.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.598+0000 7f21d4cc0640 1 Processor -- start 2024-04-14T08:38:38.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.598+0000 7f21d4cc0640 1 -- start start 2024-04-14T08:38:38.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21d4cc0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21bc004d40 0x7f21d014ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21d4cc0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21d00ad8e0 0x7f21d00add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21d4cc0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0157b10 0x7f21d0157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21d4cc0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21d015cb80 con 0x7f21bc004d40 2024-04-14T08:38:38.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21d4cc0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f21d015a000 con 0x7f21d00ad8e0 2024-04-14T08:38:38.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21d4cc0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f21d0072b60 con 0x7f21d0157b10 2024-04-14T08:38:38.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21ce59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21bc004d40 0x7f21d014ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21ce59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21bc004d40 0x7f21d014ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35786/0 (socket says 172.21.15.172:35786) 2024-04-14T08:38:38.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21ce59c640 1 -- 172.21.15.172:0/823239925 learned_addr learned my addr 172.21.15.172:0/823239925 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21c7fff640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21d00ad8e0 0x7f21d00add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21ced9d640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0157b10 0x7f21d0157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21ce59c640 1 -- 172.21.15.172:0/823239925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0157b10 msgr2=0x7f21d0157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21ce59c640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0157b10 0x7f21d0157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21ce59c640 1 -- 172.21.15.172:0/823239925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21d00ad8e0 msgr2=0x7f21d00add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21ce59c640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21d00ad8e0 0x7f21d00add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21ce59c640 1 -- 172.21.15.172:0/823239925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d014b260 con 0x7f21bc004d40 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21ced9d640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0157b10 0x7f21d0157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21c7fff640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21d00ad8e0 0x7f21d00add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21ce59c640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21bc004d40 0x7f21d014ad20 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f21c0011960 tx=0x7f21c0011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21c6ffd640 1 -- 172.21.15.172:0/823239925 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c000faf0 con 0x7f21bc004d40 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21c6ffd640 1 -- 172.21.15.172:0/823239925 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c000fc90 con 0x7f21bc004d40 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21d4cc0640 1 -- 172.21.15.172:0/823239925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d00b32e0 con 0x7f21bc004d40 2024-04-14T08:38:38.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21c6ffd640 1 -- 172.21.15.172:0/823239925 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c0004030 con 0x7f21bc004d40 2024-04-14T08:38:38.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.599+0000 7f21d4cc0640 1 -- 172.21.15.172:0/823239925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d00ae730 con 0x7f21bc004d40 2024-04-14T08:38:38.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.600+0000 7f21d4cc0640 1 -- 172.21.15.172:0/823239925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d0068590 con 0x7f21bc004d40 2024-04-14T08:38:38.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.601+0000 7f21c6ffd640 1 -- 172.21.15.172:0/823239925 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21c0012040 con 0x7f21bc004d40 2024-04-14T08:38:38.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.601+0000 7f21c6ffd640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f21b006c920 0x7f21b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.601+0000 7f21c6ffd640 1 -- 172.21.15.172:0/823239925 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f21c008bd90 con 0x7f21bc004d40 2024-04-14T08:38:38.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.601+0000 7f21c7fff640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f21b006c920 0x7f21b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.602+0000 7f21c7fff640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f21b006c920 0x7f21b006ede0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f21d015d980 tx=0x7f21b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:38.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.606+0000 7f21c6ffd640 1 -- 172.21.15.172:0/823239925 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f21d014b260 con 0x7f21bc004d40 2024-04-14T08:38:38.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.778+0000 7f21d4cc0640 1 -- 172.21.15.172:0/823239925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f21d0071e50 con 0x7f21bc004d40 2024-04-14T08:38:38.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.778+0000 7f21c6ffd640 1 -- 172.21.15.172:0/823239925 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f21c0019070 con 0x7f21bc004d40 2024-04-14T08:38:38.779 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:38:38.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.780+0000 7f21d4cc0640 1 -- 172.21.15.172:0/823239925 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f21b006c920 msgr2=0x7f21b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.780+0000 7f21d4cc0640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f21b006c920 0x7f21b006ede0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f21d015d980 tx=0x7f21b8008040 comp rx=0 tx=0).stop 2024-04-14T08:38:38.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.780+0000 7f21d4cc0640 1 -- 172.21.15.172:0/823239925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21bc004d40 msgr2=0x7f21d014ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.780+0000 7f21d4cc0640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21bc004d40 0x7f21d014ad20 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f21c0011960 tx=0x7f21c0011e20 comp rx=0 tx=0).stop 2024-04-14T08:38:38.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.780+0000 7f21d4cc0640 1 -- 172.21.15.172:0/823239925 shutdown_connections 2024-04-14T08:38:38.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.780+0000 7f21d4cc0640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f21b006c920 0x7f21b006ede0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.780+0000 7f21d4cc0640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21d0157b10 0x7f21d0157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.780+0000 7f21d4cc0640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21d00ad8e0 0x7f21d00add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.780+0000 7f21d4cc0640 1 --2- 172.21.15.172:0/823239925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21bc004d40 0x7f21d014ad20 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.780+0000 7f21d4cc0640 1 -- 172.21.15.172:0/823239925 >> 172.21.15.172:0/823239925 conn(0x7f21d006b520 msgr2=0x7f21d00cd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:38.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.780+0000 7f21d4cc0640 1 -- 172.21.15.172:0/823239925 shutdown_connections 2024-04-14T08:38:38.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.780+0000 7f21d4cc0640 1 -- 172.21.15.172:0/823239925 wait complete. 2024-04-14T08:38:38.792 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-14T08:38:38.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.897+0000 7f15895bb640 1 Processor -- start 2024-04-14T08:38:38.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.897+0000 7f15895bb640 1 -- start start 2024-04-14T08:38:38.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f15895bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f158406b520 0x7f158406fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f15895bb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f15840b3dd0 0x7f158406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f15895bb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f158406be40 con 0x7f158406e130 2024-04-14T08:38:38.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f15895bb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f1584072d80 con 0x7f158406b520 2024-04-14T08:38:38.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f15895bb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f15840b4540 con 0x7f15840b3dd0 2024-04-14T08:38:38.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f1582ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f158406b520 0x7f158406fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f15837fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f158406e130 0x7f158406e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54694/0 (socket says 172.21.15.172:54694) 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f1582ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f158406b520 0x7f158406fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35792/0 (socket says 172.21.15.172:35792) 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f15837fe640 1 -- 172.21.15.172:0/3551565287 learned_addr learned my addr 172.21.15.172:0/3551565287 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f1582ffd640 1 -- 172.21.15.172:0/3551565287 >> v2:172.21.15.177:3300/0 conn(0x7f15840b3dd0 msgr2=0x7f158406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f1582ffd640 1 --2- 172.21.15.172:0/3551565287 >> v2:172.21.15.177:3300/0 conn(0x7f15840b3dd0 0x7f158406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f1582ffd640 1 -- 172.21.15.172:0/3551565287 >> v1:172.21.15.172:6789/0 conn(0x7f158406e130 legacy=0x7f158406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f1582ffd640 1 -- 172.21.15.172:0/3551565287 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1584072450 con 0x7f158406b520 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.898+0000 7f15827fc640 1 -- 172.21.15.172:0/3551565287 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 514365524 0 0) 0x7f158406be40 con 0x7f158406e130 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.899+0000 7f1582ffd640 1 --2- 172.21.15.172:0/3551565287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f158406b520 0x7f158406fdf0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1584072360 tx=0x7f157402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4f527cfa2481ef server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.899+0000 7f15827fc640 1 -- 172.21.15.172:0/3551565287 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574035020 con 0x7f158406b520 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.899+0000 7f15827fc640 1 -- 172.21.15.172:0/3551565287 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1574031830 con 0x7f158406b520 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.899+0000 7f15827fc640 1 -- 172.21.15.172:0/3551565287 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574031c00 con 0x7f158406b520 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.900+0000 7f15895bb640 1 -- 172.21.15.172:0/3551565287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f158406b520 msgr2=0x7f158406fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.900+0000 7f15895bb640 1 --2- 172.21.15.172:0/3551565287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f158406b520 0x7f158406fdf0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1584072360 tx=0x7f157402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.900+0000 7f15895bb640 1 -- 172.21.15.172:0/3551565287 shutdown_connections 2024-04-14T08:38:38.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.900+0000 7f15895bb640 1 --2- 172.21.15.172:0/3551565287 >> v2:172.21.15.177:3300/0 conn(0x7f15840b3dd0 0x7f158406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.900+0000 7f15895bb640 1 --2- 172.21.15.172:0/3551565287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f158406b520 0x7f158406fdf0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.900+0000 7f15895bb640 1 -- 172.21.15.172:0/3551565287 >> 172.21.15.172:0/3551565287 conn(0x7f15840d0f20 msgr2=0x7f15840d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:38.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.900+0000 7f15895bb640 1 -- 172.21.15.172:0/3551565287 shutdown_connections 2024-04-14T08:38:38.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.900+0000 7f15895bb640 1 -- 172.21.15.172:0/3551565287 wait complete. 2024-04-14T08:38:38.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.901+0000 7f15895bb640 1 Processor -- start 2024-04-14T08:38:38.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.901+0000 7f15895bb640 1 -- start start 2024-04-14T08:38:38.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.901+0000 7f15895bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f158406b520 0x7f1584160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.901+0000 7f15895bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f158406e130 0x7f15841435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f15895bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15840b3dd0 0x7f1584143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f15895bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15840b4250 con 0x7f158406b520 2024-04-14T08:38:38.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f15895bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1584072a10 con 0x7f15840b3dd0 2024-04-14T08:38:38.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f15895bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1584073210 con 0x7f158406e130 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f1582ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f158406b520 0x7f1584160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f1582ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f158406b520 0x7f1584160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35798/0 (socket says 172.21.15.172:35798) 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f1582ffd640 1 -- 172.21.15.172:0/2537917685 learned_addr learned my addr 172.21.15.172:0/2537917685 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f15837fe640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15840b3dd0 0x7f1584143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f157bfff640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f158406e130 0x7f15841435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f1582ffd640 1 -- 172.21.15.172:0/2537917685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f158406e130 msgr2=0x7f15841435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f1582ffd640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f158406e130 0x7f15841435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f1582ffd640 1 -- 172.21.15.172:0/2537917685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15840b3dd0 msgr2=0x7f1584143b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f1582ffd640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15840b3dd0 0x7f1584143b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f1582ffd640 1 -- 172.21.15.172:0/2537917685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158414fcb0 con 0x7f158406b520 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f157bfff640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f158406e130 0x7f15841435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f1582ffd640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f158406b520 0x7f1584160b40 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1574000c00 tx=0x7f15740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f157b7fe640 1 -- 172.21.15.172:0/2537917685 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574040070 con 0x7f158406b520 2024-04-14T08:38:38.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f157b7fe640 1 -- 172.21.15.172:0/2537917685 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1574039940 con 0x7f158406b520 2024-04-14T08:38:38.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f15895bb640 1 -- 172.21.15.172:0/2537917685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158414ff50 con 0x7f158406b520 2024-04-14T08:38:38.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f157b7fe640 1 -- 172.21.15.172:0/2537917685 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574035040 con 0x7f158406b520 2024-04-14T08:38:38.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.902+0000 7f15895bb640 1 -- 172.21.15.172:0/2537917685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1584150510 con 0x7f158406b520 2024-04-14T08:38:38.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.903+0000 7f15895bb640 1 -- 172.21.15.172:0/2537917685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1584068520 con 0x7f158406b520 2024-04-14T08:38:38.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.904+0000 7f157b7fe640 1 -- 172.21.15.172:0/2537917685 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f157404b060 con 0x7f158406b520 2024-04-14T08:38:38.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.904+0000 7f157b7fe640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f156406c8f0 0x7f156406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:38.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.904+0000 7f157b7fe640 1 -- 172.21.15.172:0/2537917685 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f15740ad430 con 0x7f158406b520 2024-04-14T08:38:38.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.904+0000 7f157bfff640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f156406c8f0 0x7f156406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:38.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.905+0000 7f157bfff640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f156406c8f0 0x7f156406edb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f156c006fd0 tx=0x7f156c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:38.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:38.909+0000 7f157b7fe640 1 -- 172.21.15.172:0/2537917685 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f157407b980 con 0x7f158406b520 2024-04-14T08:38:39.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.152+0000 7f15895bb640 1 -- 172.21.15.172:0/2537917685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f15840aedb0 con 0x7f158406b520 2024-04-14T08:38:39.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.173+0000 7f157b7fe640 1 -- 172.21.15.172:0/2537917685 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v289) ==== 170+0+0 (secure 0 0 0) 0x7f157407b280 con 0x7f158406b520 2024-04-14T08:38:39.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.175+0000 7f15895bb640 1 -- 172.21.15.172:0/2537917685 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f156406c8f0 msgr2=0x7f156406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:39.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.175+0000 7f15895bb640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f156406c8f0 0x7f156406edb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f156c006fd0 tx=0x7f156c008040 comp rx=0 tx=0).stop 2024-04-14T08:38:39.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.175+0000 7f15895bb640 1 -- 172.21.15.172:0/2537917685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f158406b520 msgr2=0x7f1584160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:39.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.175+0000 7f15895bb640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f158406b520 0x7f1584160b40 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1574000c00 tx=0x7f15740046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.175+0000 7f15895bb640 1 -- 172.21.15.172:0/2537917685 shutdown_connections 2024-04-14T08:38:39.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.175+0000 7f15895bb640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f156406c8f0 0x7f156406edb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.175+0000 7f15895bb640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15840b3dd0 0x7f1584143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.175+0000 7f15895bb640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f158406e130 0x7f15841435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.175+0000 7f15895bb640 1 --2- 172.21.15.172:0/2537917685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f158406b520 0x7f1584160b40 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.175+0000 7f15895bb640 1 -- 172.21.15.172:0/2537917685 >> 172.21.15.172:0/2537917685 conn(0x7f15840d0f20 msgr2=0x7f158406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:39.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.175+0000 7f15895bb640 1 -- 172.21.15.172:0/2537917685 shutdown_connections 2024-04-14T08:38:39.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.175+0000 7f15895bb640 1 -- 172.21.15.172:0/2537917685 wait complete. 2024-04-14T08:38:39.187 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-14T08:38:39.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.291+0000 7f5e6e75e640 1 Processor -- start 2024-04-14T08:38:39.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.291+0000 7f5e6e75e640 1 -- start start 2024-04-14T08:38:39.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.291+0000 7f5e6e75e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6813df30 0x7f5e680b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.291+0000 7f5e6e75e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5e6815c380 0x7f5e680b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.291+0000 7f5e6e75e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e6815b620 con 0x7f5e68140770 2024-04-14T08:38:39.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.291+0000 7f5e6e75e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5e680cf9b0 con 0x7f5e6815c380 2024-04-14T08:38:39.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.291+0000 7f5e6e75e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e681414a0 con 0x7f5e6813df30 2024-04-14T08:38:39.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e6ccfb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5e68140770 0x7f5e68140c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54704/0 (socket says 172.21.15.172:54704) 2024-04-14T08:38:39.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e6ccfb640 1 -- 172.21.15.172:0/1705262027 learned_addr learned my addr 172.21.15.172:0/1705262027 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:39.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e67fff640 1 --2- 172.21.15.172:0/1705262027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6813df30 0x7f5e680b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:39.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e677fe640 1 --2- 172.21.15.172:0/1705262027 >> v2:172.21.15.177:3300/0 conn(0x7f5e6815c380 0x7f5e680b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:39.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3285922558 0 0) 0x7f5e60001490 con 0x7f5e68140770 2024-04-14T08:38:39.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e50003520 con 0x7f5e68140770 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4072263354 0 0) 0x7f5e50003520 con 0x7f5e68140770 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6813df30 msgr2=0x7f5e680b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 --2- 172.21.15.172:0/1705262027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6813df30 0x7f5e680b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 >> v2:172.21.15.177:3300/0 conn(0x7f5e6815c380 msgr2=0x7f5e680b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 --2- 172.21.15.172:0/1705262027 >> v2:172.21.15.177:3300/0 conn(0x7f5e6815c380 0x7f5e680b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6815b8a0 con 0x7f5e68140770 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f5e60001bc0 con 0x7f5e68140770 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 >> v1:172.21.15.172:6789/0 conn(0x7f5e68140770 legacy=0x7f5e68140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 --2- 172.21.15.172:0/1705262027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6813df30 0x7f5e500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 --2- 172.21.15.172:0/1705262027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e50003f30 0x7f5e50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 --2- 172.21.15.172:0/1705262027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e50004d40 0x7f5e500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e680cf9b0 con 0x7f5e50004d40 2024-04-14T08:38:39.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e681414a0 con 0x7f5e50003f30 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e5000b9c0 con 0x7f5e6813df30 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.292+0000 7f5e6ccfb640 1 --2- 172.21.15.172:0/1705262027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e50004d40 0x7f5e500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.293+0000 7f5e677fe640 1 --2- 172.21.15.172:0/1705262027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e50003f30 0x7f5e50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.293+0000 7f5e67fff640 1 --2- 172.21.15.172:0/1705262027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6813df30 0x7f5e500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.293+0000 7f5e6ccfb640 1 -- 172.21.15.172:0/1705262027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6813df30 msgr2=0x7f5e500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.293+0000 7f5e6ccfb640 1 --2- 172.21.15.172:0/1705262027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e6813df30 0x7f5e500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.293+0000 7f5e6ccfb640 1 -- 172.21.15.172:0/1705262027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e50003f30 msgr2=0x7f5e50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.293+0000 7f5e6ccfb640 1 --2- 172.21.15.172:0/1705262027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e50003f30 0x7f5e50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.293+0000 7f5e6ccfb640 1 -- 172.21.15.172:0/1705262027 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e50003520 con 0x7f5e50004d40 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.293+0000 7f5e6ccfb640 1 --2- 172.21.15.172:0/1705262027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e50004d40 0x7f5e500051c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000aa30 tx=0x7f5e50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fac4b7f7275e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.294+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60038070 con 0x7f5e50004d40 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.294+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e60007290 con 0x7f5e50004d40 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.294+0000 7f5e66ffd640 1 -- 172.21.15.172:0/1705262027 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6000a650 con 0x7f5e50004d40 2024-04-14T08:38:39.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.294+0000 7f5e6e75e640 1 -- 172.21.15.172:0/1705262027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e50004d40 msgr2=0x7f5e500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:39.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.294+0000 7f5e6e75e640 1 --2- 172.21.15.172:0/1705262027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e50004d40 0x7f5e500051c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000aa30 tx=0x7f5e50005830 comp rx=0 tx=0).stop 2024-04-14T08:38:39.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.294+0000 7f5e6ccfb640 1 -- 172.21.15.172:0/1705262027 reap_dead start 2024-04-14T08:38:39.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.294+0000 7f5e6e75e640 1 -- 172.21.15.172:0/1705262027 shutdown_connections 2024-04-14T08:38:39.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.294+0000 7f5e6e75e640 1 -- 172.21.15.172:0/1705262027 >> 172.21.15.172:0/1705262027 conn(0x7f5e6806b520 msgr2=0x7f5e6806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:39.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.294+0000 7f5e6e75e640 1 -- 172.21.15.172:0/1705262027 shutdown_connections 2024-04-14T08:38:39.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.294+0000 7f5e6e75e640 1 -- 172.21.15.172:0/1705262027 wait complete. 2024-04-14T08:38:39.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.295+0000 7f5e6e75e640 1 Processor -- start 2024-04-14T08:38:39.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.295+0000 7f5e6e75e640 1 -- start start 2024-04-14T08:38:39.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.296+0000 7f5e6e75e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e681f20d0 0x7f5e681f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.296+0000 7f5e6e75e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e681f24b0 0x7f5e68152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.296+0000 7f5e6e75e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e681f8f30 0x7f5e680712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.296+0000 7f5e6e75e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6813e310 con 0x7f5e681f24b0 2024-04-14T08:38:39.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.296+0000 7f5e6e75e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6815be60 con 0x7f5e681f20d0 2024-04-14T08:38:39.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.296+0000 7f5e6e75e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e680b1cd0 con 0x7f5e681f8f30 2024-04-14T08:38:39.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.296+0000 7f5e677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e681f24b0 0x7f5e68152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:39.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.296+0000 7f5e677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e681f24b0 0x7f5e68152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35818/0 (socket says 172.21.15.172:35818) 2024-04-14T08:38:39.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.296+0000 7f5e677fe640 1 -- 172.21.15.172:0/4277921635 learned_addr learned my addr 172.21.15.172:0/4277921635 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.296+0000 7f5e67fff640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e681f20d0 0x7f5e681f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e6ccfb640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e681f8f30 0x7f5e680712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e677fe640 1 -- 172.21.15.172:0/4277921635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e681f8f30 msgr2=0x7f5e680712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e677fe640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e681f8f30 0x7f5e680712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e677fe640 1 -- 172.21.15.172:0/4277921635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e681f20d0 msgr2=0x7f5e681f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e677fe640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e681f20d0 0x7f5e681f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e677fe640 1 -- 172.21.15.172:0/4277921635 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e681f4d50 con 0x7f5e681f24b0 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e6ccfb640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e681f8f30 0x7f5e680712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e67fff640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e681f20d0 0x7f5e681f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e677fe640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e681f24b0 0x7f5e68152890 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5e58011a70 tx=0x7f5e58011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e64ff9640 1 -- 172.21.15.172:0/4277921635 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800fb50 con 0x7f5e681f24b0 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e64ff9640 1 -- 172.21.15.172:0/4277921635 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e5800fcf0 con 0x7f5e681f24b0 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e6e75e640 1 -- 172.21.15.172:0/4277921635 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e681530e0 con 0x7f5e681f24b0 2024-04-14T08:38:39.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e64ff9640 1 -- 172.21.15.172:0/4277921635 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800fea0 con 0x7f5e681f24b0 2024-04-14T08:38:39.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.297+0000 7f5e6e75e640 1 -- 172.21.15.172:0/4277921635 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e681f4ff0 con 0x7f5e681f24b0 2024-04-14T08:38:39.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.298+0000 7f5e6e75e640 1 -- 172.21.15.172:0/4277921635 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e680750a0 con 0x7f5e681f24b0 2024-04-14T08:38:39.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.298+0000 7f5e64ff9640 1 -- 172.21.15.172:0/4277921635 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e58012040 con 0x7f5e681f24b0 2024-04-14T08:38:39.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.299+0000 7f5e64ff9640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e5406c8f0 0x7f5e5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.299+0000 7f5e64ff9640 1 -- 172.21.15.172:0/4277921635 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f5e5808c0e0 con 0x7f5e681f24b0 2024-04-14T08:38:39.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.299+0000 7f5e67fff640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e5406c8f0 0x7f5e5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:39.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.299+0000 7f5e67fff640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e5406c8f0 0x7f5e5406edb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5e680b1900 tx=0x7f5e5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:39.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.303+0000 7f5e64ff9640 1 -- 172.21.15.172:0/4277921635 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5e680750a0 con 0x7f5e681f24b0 2024-04-14T08:38:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.480+0000 7f5e6e75e640 1 -- 172.21.15.172:0/4277921635 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5e680b2f60 con 0x7f5e681f24b0 2024-04-14T08:38:39.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.485+0000 7f5e64ff9640 1 -- 172.21.15.172:0/4277921635 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f5e680b2f60 con 0x7f5e681f24b0 2024-04-14T08:38:39.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.488+0000 7f5e6e75e640 1 -- 172.21.15.172:0/4277921635 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e5406c8f0 msgr2=0x7f5e5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:39.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.488+0000 7f5e6e75e640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e5406c8f0 0x7f5e5406edb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5e680b1900 tx=0x7f5e5c008040 comp rx=0 tx=0).stop 2024-04-14T08:38:39.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.488+0000 7f5e6e75e640 1 -- 172.21.15.172:0/4277921635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e681f24b0 msgr2=0x7f5e68152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:39.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.488+0000 7f5e6e75e640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e681f24b0 0x7f5e68152890 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5e58011a70 tx=0x7f5e58011f30 comp rx=0 tx=0).stop 2024-04-14T08:38:39.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.488+0000 7f5e6e75e640 1 -- 172.21.15.172:0/4277921635 shutdown_connections 2024-04-14T08:38:39.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.488+0000 7f5e6e75e640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e5406c8f0 0x7f5e5406edb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.488+0000 7f5e6e75e640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e681f8f30 0x7f5e680712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.488+0000 7f5e6e75e640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e681f24b0 0x7f5e68152890 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.488+0000 7f5e6e75e640 1 --2- 172.21.15.172:0/4277921635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e681f20d0 0x7f5e681f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.488+0000 7f5e6e75e640 1 -- 172.21.15.172:0/4277921635 >> 172.21.15.172:0/4277921635 conn(0x7f5e6806b520 msgr2=0x7f5e681599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:39.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.488+0000 7f5e6e75e640 1 -- 172.21.15.172:0/4277921635 shutdown_connections 2024-04-14T08:38:39.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.488+0000 7f5e6e75e640 1 -- 172.21.15.172:0/4277921635 wait complete. 2024-04-14T08:38:39.499 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:38:39.499 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:38:39.543 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:38:39.596 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:38:39.596 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:38:39.596 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:39.611 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:38:39.612 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:38:39.612 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:39.629 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:38:39.629 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:38:39.629 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:39.667 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:38:39.667 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:38:39.668 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:39.686 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:38:39.686 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-14T08:38:39.686 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:38:39.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.807+0000 7fa708a36640 1 Processor -- start 2024-04-14T08:38:39.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.807+0000 7fa708a36640 1 -- start start 2024-04-14T08:38:39.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.808+0000 7fa708a36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa70415e890 0x7fa7040d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.808+0000 7fa708a36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa7040cedd0 0x7fa7040b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.808+0000 7fa708a36640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7040cf320 con 0x7fa7040d2830 2024-04-14T08:38:39.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.808+0000 7fa708a36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa7040cff30 con 0x7fa70415e890 2024-04-14T08:38:39.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.808+0000 7fa708a36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa7040d2350 con 0x7fa7040cedd0 2024-04-14T08:38:39.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.808+0000 7fa701d9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa70415e890 0x7fa7040d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:39.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.809+0000 7fa701d9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa70415e890 0x7fa7040d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49566/0 (socket says 172.21.15.172:49566) 2024-04-14T08:38:39.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.808+0000 7fa70259c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa7040d2830 0x7fa7040d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45666/0 (socket says 172.21.15.172:45666) 2024-04-14T08:38:39.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.809+0000 7fa701d9b640 1 -- 172.21.15.172:0/3024498633 learned_addr learned my addr 172.21.15.172:0/3024498633 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:39.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.809+0000 7fa701d9b640 1 -- 172.21.15.172:0/3024498633 >> v2:172.21.15.177:3300/0 conn(0x7fa7040cedd0 msgr2=0x7fa7040b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:39.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.809+0000 7fa701d9b640 1 --2- 172.21.15.172:0/3024498633 >> v2:172.21.15.177:3300/0 conn(0x7fa7040cedd0 0x7fa7040b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.809+0000 7fa701d9b640 1 -- 172.21.15.172:0/3024498633 >> v1:172.21.15.172:6789/0 conn(0x7fa7040d2830 legacy=0x7fa7040d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:39.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.809+0000 7fa701d9b640 1 -- 172.21.15.172:0/3024498633 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7040cf600 con 0x7fa70415e890 2024-04-14T08:38:39.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.809+0000 7fa70159a640 1 -- 172.21.15.172:0/3024498633 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1377066399 0 0) 0x7fa7040cf320 con 0x7fa7040d2830 2024-04-14T08:38:39.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.810+0000 7fa701d9b640 1 --2- 172.21.15.172:0/3024498633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa70415e890 0x7fa7040d1300 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa70408dbe0 tx=0x7fa6f802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4b20791d5915a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:39.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.810+0000 7fa70159a640 1 -- 172.21.15.172:0/3024498633 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f802f850 con 0x7fa70415e890 2024-04-14T08:38:39.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.810+0000 7fa70159a640 1 -- 172.21.15.172:0/3024498633 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f802f9f0 con 0x7fa70415e890 2024-04-14T08:38:39.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.810+0000 7fa70159a640 1 -- 172.21.15.172:0/3024498633 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f802fdc0 con 0x7fa70415e890 2024-04-14T08:38:39.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.810+0000 7fa708a36640 1 -- 172.21.15.172:0/3024498633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa70415e890 msgr2=0x7fa7040d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.810+0000 7fa708a36640 1 --2- 172.21.15.172:0/3024498633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa70415e890 0x7fa7040d1300 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa70408dbe0 tx=0x7fa6f802ba30 comp rx=0 tx=0).stop 2024-04-14T08:38:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.810+0000 7fa708a36640 1 -- 172.21.15.172:0/3024498633 shutdown_connections 2024-04-14T08:38:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.810+0000 7fa708a36640 1 --2- 172.21.15.172:0/3024498633 >> v2:172.21.15.177:3300/0 conn(0x7fa7040cedd0 0x7fa7040b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.810+0000 7fa708a36640 1 --2- 172.21.15.172:0/3024498633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa70415e890 0x7fa7040d1300 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.810+0000 7fa708a36640 1 -- 172.21.15.172:0/3024498633 >> 172.21.15.172:0/3024498633 conn(0x7fa70409d590 msgr2=0x7fa7040d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.810+0000 7fa708a36640 1 -- 172.21.15.172:0/3024498633 shutdown_connections 2024-04-14T08:38:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.810+0000 7fa708a36640 1 -- 172.21.15.172:0/3024498633 wait complete. 2024-04-14T08:38:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.812+0000 7fa708a36640 1 Processor -- start 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.812+0000 7fa708a36640 1 -- start start 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.812+0000 7fa708a36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7040cedd0 0x7fa7040b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.812+0000 7fa708a36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7040d2830 0x7fa7040b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.812+0000 7fa708a36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa70415e890 0x7fa70415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.812+0000 7fa708a36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa7040d1840 con 0x7fa7040d2830 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.812+0000 7fa708a36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa7040cfbc0 con 0x7fa70415e890 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.812+0000 7fa708a36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa7040b3c50 con 0x7fa7040cedd0 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.812+0000 7fa701d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7040d2830 0x7fa7040b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa701d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7040d2830 0x7fa7040b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49574/0 (socket says 172.21.15.172:49574) 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa701d9b640 1 -- 172.21.15.172:0/3768284075 learned_addr learned my addr 172.21.15.172:0/3768284075 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa701d9b640 1 -- 172.21.15.172:0/3768284075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7040cedd0 msgr2=0x7fa7040b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa701d9b640 1 --2- 172.21.15.172:0/3768284075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7040cedd0 0x7fa7040b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa701d9b640 1 -- 172.21.15.172:0/3768284075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa70415e890 msgr2=0x7fa70415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa701d9b640 1 --2- 172.21.15.172:0/3768284075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa70415e890 0x7fa70415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa701d9b640 1 -- 172.21.15.172:0/3768284075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7041f53a0 con 0x7fa7040d2830 2024-04-14T08:38:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa701d9b640 1 --2- 172.21.15.172:0/3768284075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7040d2830 0x7fa7040b1a70 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8000c00 tx=0x7fa6f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa6f2ffd640 1 -- 172.21.15.172:0/3768284075 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8037d10 con 0x7fa7040d2830 2024-04-14T08:38:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa6f2ffd640 1 -- 172.21.15.172:0/3768284075 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f8046070 con 0x7fa7040d2830 2024-04-14T08:38:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa708a36640 1 -- 172.21.15.172:0/3768284075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70415fb40 con 0x7fa7040d2830 2024-04-14T08:38:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa6f2ffd640 1 -- 172.21.15.172:0/3768284075 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8041640 con 0x7fa7040d2830 2024-04-14T08:38:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.813+0000 7fa708a36640 1 -- 172.21.15.172:0/3768284075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7040b73a0 con 0x7fa7040d2830 2024-04-14T08:38:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.814+0000 7fa708a36640 1 -- 172.21.15.172:0/3768284075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa704068520 con 0x7fa7040d2830 2024-04-14T08:38:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.815+0000 7fa6f2ffd640 1 -- 172.21.15.172:0/3768284075 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6f8040070 con 0x7fa7040d2830 2024-04-14T08:38:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.815+0000 7fa6f2ffd640 1 --2- 172.21.15.172:0/3768284075 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6e406c8f0 0x7fa6e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.815+0000 7fa6f2ffd640 1 -- 172.21.15.172:0/3768284075 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7fa6f80ae190 con 0x7fa7040d2830 2024-04-14T08:38:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.815+0000 7fa70259c640 1 --2- 172.21.15.172:0/3768284075 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6e406c8f0 0x7fa6e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.816+0000 7fa70259c640 1 --2- 172.21.15.172:0/3768284075 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6e406c8f0 0x7fa6e406edb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4001560 tx=0x7fa6f4007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:39.820+0000 7fa6f2ffd640 1 -- 172.21.15.172:0/3768284075 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa6f8033030 con 0x7fa7040d2830 2024-04-14T08:38:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.033+0000 7fa708a36640 1 -- 172.21.15.172:0/3768284075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7040d2610 con 0x7fa7040d2830 2024-04-14T08:38:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.033+0000 7fa6f2ffd640 1 -- 172.21.15.172:0/3768284075 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+3878 (secure 0 0 0) 0x7fa6f807c770 con 0x7fa7040d2830 2024-04-14T08:38:40.034 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:40.034 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2700124897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2700124897},{"type":"v1","addr":"172.21.15.177:6834","nonce":2700124897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/541906092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":541906092},{"type":"v1","addr":"172.21.15.177:6835","nonce":541906092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6750,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/4267663691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4267663691},{"type":"v1","addr":"172.21.15.172:6836","nonce":4267663691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":7395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1963656915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1963656915},{"type":"v1","addr":"172.21.15.172:6837","nonce":1963656915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[]} 2024-04-14T08:38:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 49 2024-04-14T08:38:40.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.035+0000 7fa708a36640 1 -- 172.21.15.172:0/3768284075 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6e406c8f0 msgr2=0x7fa6e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:40.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.035+0000 7fa708a36640 1 --2- 172.21.15.172:0/3768284075 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6e406c8f0 0x7fa6e406edb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4001560 tx=0x7fa6f4007ca0 comp rx=0 tx=0).stop 2024-04-14T08:38:40.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.035+0000 7fa708a36640 1 -- 172.21.15.172:0/3768284075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7040d2830 msgr2=0x7fa7040b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:40.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.035+0000 7fa708a36640 1 --2- 172.21.15.172:0/3768284075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7040d2830 0x7fa7040b1a70 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8000c00 tx=0x7fa6f8004730 comp rx=0 tx=0).stop 2024-04-14T08:38:40.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.036+0000 7fa708a36640 1 -- 172.21.15.172:0/3768284075 shutdown_connections 2024-04-14T08:38:40.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.036+0000 7fa708a36640 1 --2- 172.21.15.172:0/3768284075 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa6e406c8f0 0x7fa6e406edb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:40.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.036+0000 7fa708a36640 1 --2- 172.21.15.172:0/3768284075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa70415e890 0x7fa70415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:40.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.036+0000 7fa708a36640 1 --2- 172.21.15.172:0/3768284075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7040d2830 0x7fa7040b1a70 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:40.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.036+0000 7fa708a36640 1 --2- 172.21.15.172:0/3768284075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7040cedd0 0x7fa7040b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:40.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.036+0000 7fa708a36640 1 -- 172.21.15.172:0/3768284075 >> 172.21.15.172:0/3768284075 conn(0x7fa70409d590 msgr2=0x7fa7040d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:40.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.036+0000 7fa708a36640 1 -- 172.21.15.172:0/3768284075 shutdown_connections 2024-04-14T08:38:40.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.036+0000 7fa708a36640 1 -- 172.21.15.172:0/3768284075 wait complete. 2024-04-14T08:38:40.047 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-14T08:38:40.047 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-14T08:38:40.048 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-14T08:38:40.048 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:38:40.048 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-14T08:38:40.048 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-14T08:38:40.048 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-14T08:38:40.048 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:38:40.048 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-14T08:38:40.049 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-14T08:38:40.049 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-14T08:38:40.049 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:38:40.049 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-14T08:38:40.049 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-14T08:38:40.049 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-14T08:38:40.049 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:38:40.056 INFO:tasks.ceph.mds.a:Stopped 2024-04-14T08:38:40.057 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-14T08:38:40.057 INFO:tasks.ceph.mds.c:Stopped 2024-04-14T08:38:40.057 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-14T08:38:40.061 INFO:tasks.ceph.mds.b:Stopped 2024-04-14T08:38:40.061 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-14T08:38:40.062 INFO:tasks.ceph.mds.d:Stopped 2024-04-14T08:38:40.062 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-14T08:38:40.099 INFO:tasks.ceph.mds.a:Started 2024-04-14T08:38:40.099 DEBUG:teuthology.parallel:result is None 2024-04-14T08:38:40.100 INFO:tasks.ceph.mds.c:Started 2024-04-14T08:38:40.100 DEBUG:teuthology.parallel:result is None 2024-04-14T08:38:40.103 INFO:tasks.ceph.mds.b:Started 2024-04-14T08:38:40.103 DEBUG:teuthology.parallel:result is None 2024-04-14T08:38:40.104 INFO:tasks.ceph.mds.d:Started 2024-04-14T08:38:40.104 DEBUG:teuthology.parallel:result is None 2024-04-14T08:38:40.104 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-14T08:38:40.178 INFO:tasks.ceph.mds.a.smithi172.stdout:starting mds.a at 2024-04-14T08:38:40.179 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:38:40.189 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-04-14T08:38:40.189 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-04-14T08:38:40.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.212+0000 7f7241b1e640 1 Processor -- start 2024-04-14T08:38:40.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.212+0000 7f7241b1e640 1 -- start start 2024-04-14T08:38:40.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f7241b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f723c162710 0x7f723c0d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:40.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f7241b1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f723c162b70 0x7f723c0d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:40.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f7241b1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f723c0cefb0 con 0x7f723c15e430 2024-04-14T08:38:40.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f7241b1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f723c0cf290 con 0x7f723c162b70 2024-04-14T08:38:40.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f7241b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f723c0cf570 con 0x7f723c162710 2024-04-14T08:38:40.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f723bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f723c162b70 0x7f723c0d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:40.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f723bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f723c162b70 0x7f723c0d2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49612/0 (socket says 172.21.15.172:49612) 2024-04-14T08:38:40.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f723bfff640 1 -- 172.21.15.172:0/3901943444 learned_addr learned my addr 172.21.15.172:0/3901943444 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:40.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f723affd640 1 --2- 172.21.15.172:0/3901943444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f723c162710 0x7f723c0d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:40.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f723bfff640 1 -- 172.21.15.172:0/3901943444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f723c162710 msgr2=0x7f723c0d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:40.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f723bfff640 1 --2- 172.21.15.172:0/3901943444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f723c162710 0x7f723c0d1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:40.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f723bfff640 1 -- 172.21.15.172:0/3901943444 >> v1:172.21.15.177:6789/0 conn(0x7f723c15e430 legacy=0x7f723c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:40.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.213+0000 7f723bfff640 1 -- 172.21.15.172:0/3901943444 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c0cf850 con 0x7f723c162b70 2024-04-14T08:38:40.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.214+0000 7f723bfff640 1 --2- 172.21.15.172:0/3901943444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c162b70 0x7f723c0d2330 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7234009e50 tx=0x7f723402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a288d087fe449339 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:40.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.214+0000 7f723a7fc640 1 -- 172.21.15.172:0/3901943444 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723402ca70 con 0x7f723c162b70 2024-04-14T08:38:40.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.214+0000 7f723a7fc640 1 -- 172.21.15.172:0/3901943444 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f723402cc10 con 0x7f723c162b70 2024-04-14T08:38:40.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.214+0000 7f723a7fc640 1 -- 172.21.15.172:0/3901943444 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340343f0 con 0x7f723c162b70 2024-04-14T08:38:40.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.215+0000 7f7241b1e640 1 -- 172.21.15.172:0/3901943444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c162b70 msgr2=0x7f723c0d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:40.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.215+0000 7f7241b1e640 1 --2- 172.21.15.172:0/3901943444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c162b70 0x7f723c0d2330 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7234009e50 tx=0x7f723402bed0 comp rx=0 tx=0).stop 2024-04-14T08:38:40.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.215+0000 7f7241b1e640 1 -- 172.21.15.172:0/3901943444 shutdown_connections 2024-04-14T08:38:40.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.215+0000 7f7241b1e640 1 --2- 172.21.15.172:0/3901943444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c162b70 0x7f723c0d2330 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:40.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.215+0000 7f7241b1e640 1 --2- 172.21.15.172:0/3901943444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f723c162710 0x7f723c0d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:40.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.215+0000 7f7241b1e640 1 -- 172.21.15.172:0/3901943444 >> 172.21.15.172:0/3901943444 conn(0x7f723c09d560 msgr2=0x7f723c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:40.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.215+0000 7f7241b1e640 1 -- 172.21.15.172:0/3901943444 shutdown_connections 2024-04-14T08:38:40.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.215+0000 7f7241b1e640 1 -- 172.21.15.172:0/3901943444 wait complete. 2024-04-14T08:38:40.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.216+0000 7f7241b1e640 1 Processor -- start 2024-04-14T08:38:40.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.216+0000 7f7241b1e640 1 -- start start 2024-04-14T08:38:40.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f7241b1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f723c15e430 0x7f723c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:40.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f7241b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f723c162710 0x7f723c0b8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:40.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f7241b1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c162b70 0x7f723c1f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:40.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f7241b1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f723c0d4e20 con 0x7f723c162b70 2024-04-14T08:38:40.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f7241b1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f723c15e810 con 0x7f723c15e430 2024-04-14T08:38:40.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f7241b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f723c0d0050 con 0x7f723c162710 2024-04-14T08:38:40.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f723bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c162b70 0x7f723c1f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:40.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f723bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c162b70 0x7f723c1f3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49626/0 (socket says 172.21.15.172:49626) 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f723bfff640 1 -- 172.21.15.172:0/725775624 learned_addr learned my addr 172.21.15.172:0/725775624 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f723b7fe640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f723c15e430 0x7f723c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f723affd640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f723c162710 0x7f723c0b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f723bfff640 1 -- 172.21.15.172:0/725775624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f723c162710 msgr2=0x7f723c0b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f723bfff640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f723c162710 0x7f723c0b8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f723bfff640 1 -- 172.21.15.172:0/725775624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f723c15e430 msgr2=0x7f723c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f723bfff640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f723c15e430 0x7f723c0b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f723bfff640 1 -- 172.21.15.172:0/725775624 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c0b3f50 con 0x7f723c162b70 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f723bfff640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c162b70 0x7f723c1f3690 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7234036040 tx=0x7f7234004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f7223fff640 1 -- 172.21.15.172:0/725775624 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234043020 con 0x7f723c162b70 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f7223fff640 1 -- 172.21.15.172:0/725775624 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f723403e450 con 0x7f723c162b70 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.217+0000 7f7241b1e640 1 -- 172.21.15.172:0/725775624 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723c0cb390 con 0x7f723c162b70 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.218+0000 7f7223fff640 1 -- 172.21.15.172:0/725775624 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723403e840 con 0x7f723c162b70 2024-04-14T08:38:40.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.218+0000 7f7241b1e640 1 -- 172.21.15.172:0/725775624 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723c0cb910 con 0x7f723c162b70 2024-04-14T08:38:40.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.218+0000 7f7241b1e640 1 -- 172.21.15.172:0/725775624 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723c068590 con 0x7f723c162b70 2024-04-14T08:38:40.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.219+0000 7f7223fff640 1 -- 172.21.15.172:0/725775624 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7234005d10 con 0x7f723c162b70 2024-04-14T08:38:40.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.219+0000 7f7223fff640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f722806c8f0 0x7f722806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:40.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.220+0000 7f723b7fe640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f722806c8f0 0x7f722806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:40.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.220+0000 7f7223fff640 1 -- 172.21.15.172:0/725775624 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5421+0+0 (secure 0 0 0) 0x7f72340ae430 con 0x7f723c162b70 2024-04-14T08:38:40.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.220+0000 7f723b7fe640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f722806c8f0 0x7f722806edb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f722c004f60 tx=0x7f722c007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:40.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.224+0000 7f7223fff640 1 -- 172.21.15.172:0/725775624 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f723407c3c0 con 0x7f723c162b70 2024-04-14T08:38:40.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:40.408+0000 7f7241b1e640 1 -- 172.21.15.172:0/725775624 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f723c0d18a0 con 0x7f723c162b70 2024-04-14T08:38:41.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.193+0000 7f7223fff640 1 -- 172.21.15.172:0/725775624 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) ==== 97+0+0 (secure 0 0 0) 0x7f72340806c0 con 0x7f723c162b70 2024-04-14T08:38:41.194 INFO:teuthology.orchestra.run.smithi172.stderr: removed all blocklist entries 2024-04-14T08:38:41.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.196+0000 7f7241b1e640 1 -- 172.21.15.172:0/725775624 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f722806c8f0 msgr2=0x7f722806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.196+0000 7f7241b1e640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f722806c8f0 0x7f722806edb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f722c004f60 tx=0x7f722c007e30 comp rx=0 tx=0).stop 2024-04-14T08:38:41.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.196+0000 7f7241b1e640 1 -- 172.21.15.172:0/725775624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c162b70 msgr2=0x7f723c1f3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.196+0000 7f7241b1e640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c162b70 0x7f723c1f3690 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7234036040 tx=0x7f7234004a40 comp rx=0 tx=0).stop 2024-04-14T08:38:41.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.196+0000 7f7241b1e640 1 -- 172.21.15.172:0/725775624 shutdown_connections 2024-04-14T08:38:41.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.196+0000 7f7241b1e640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f722806c8f0 0x7f722806edb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.196+0000 7f7241b1e640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c162b70 0x7f723c1f3690 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.196+0000 7f7241b1e640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f723c162710 0x7f723c0b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.196+0000 7f7241b1e640 1 --2- 172.21.15.172:0/725775624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f723c15e430 0x7f723c0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.196+0000 7f7241b1e640 1 -- 172.21.15.172:0/725775624 >> 172.21.15.172:0/725775624 conn(0x7f723c09d560 msgr2=0x7f723c0b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:41.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.196+0000 7f7241b1e640 1 -- 172.21.15.172:0/725775624 shutdown_connections 2024-04-14T08:38:41.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.196+0000 7f7241b1e640 1 -- 172.21.15.172:0/725775624 wait complete. 2024-04-14T08:38:41.209 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-14T08:38:41.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.310+0000 7fb2dfbfe640 1 Processor -- start 2024-04-14T08:38:41.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.311+0000 7fb2dfbfe640 1 -- start start 2024-04-14T08:38:41.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.312+0000 7fb2dfbfe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb2d8072970 0x7fb2d815b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.312+0000 7fb2dfbfe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d80b3360 con 0x7fb2d815a690 2024-04-14T08:38:41.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.312+0000 7fb2dfbfe640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d80b3640 con 0x7fb2d8072470 2024-04-14T08:38:41.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.312+0000 7fb2dfbfe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb2d80b3920 con 0x7fb2d8072970 2024-04-14T08:38:41.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.312+0000 7fb2de19b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb2d8072970 0x7fb2d815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.312+0000 7fb2dd99a640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb2d815a690 0x7fb2d8071d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45688/0 (socket says 172.21.15.172:45688) 2024-04-14T08:38:41.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.312+0000 7fb2dd99a640 1 -- 172.21.15.172:0/3940790586 learned_addr learned my addr 172.21.15.172:0/3940790586 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:41.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.312+0000 7fb2de19b640 1 -- 172.21.15.172:0/3940790586 >> v1:172.21.15.177:6789/0 conn(0x7fb2d8072470 legacy=0x7fb2d815b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:41.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.312+0000 7fb2de19b640 1 -- 172.21.15.172:0/3940790586 >> v1:172.21.15.172:6789/0 conn(0x7fb2d815a690 legacy=0x7fb2d8071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.312+0000 7fb2de19b640 1 -- 172.21.15.172:0/3940790586 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d80b3c00 con 0x7fb2d8072970 2024-04-14T08:38:41.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.313+0000 7fb2de19b640 1 --2- 172.21.15.172:0/3940790586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d8072970 0x7fb2d815b940 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000aea0 tx=0x7fb2d002cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cf8e9ef7fcedd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:41.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.313+0000 7fb2dc998640 1 -- 172.21.15.172:0/3940790586 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d002dac0 con 0x7fb2d8072970 2024-04-14T08:38:41.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.313+0000 7fb2dc998640 1 -- 172.21.15.172:0/3940790586 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d002dc60 con 0x7fb2d8072970 2024-04-14T08:38:41.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.313+0000 7fb2dc998640 1 -- 172.21.15.172:0/3940790586 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0008660 con 0x7fb2d8072970 2024-04-14T08:38:41.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.313+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/3940790586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d8072970 msgr2=0x7fb2d815b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.313+0000 7fb2dfbfe640 1 --2- 172.21.15.172:0/3940790586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d8072970 0x7fb2d815b940 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000aea0 tx=0x7fb2d002cf20 comp rx=0 tx=0).stop 2024-04-14T08:38:41.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.313+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/3940790586 shutdown_connections 2024-04-14T08:38:41.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.313+0000 7fb2dfbfe640 1 --2- 172.21.15.172:0/3940790586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d8072970 0x7fb2d815b940 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.313+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/3940790586 >> 172.21.15.172:0/3940790586 conn(0x7fb2d806ec30 msgr2=0x7fb2d806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:41.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.314+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/3940790586 shutdown_connections 2024-04-14T08:38:41.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.314+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/3940790586 wait complete. 2024-04-14T08:38:41.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.315+0000 7fb2dfbfe640 1 Processor -- start 2024-04-14T08:38:41.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.315+0000 7fb2dfbfe640 1 -- start start 2024-04-14T08:38:41.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.315+0000 7fb2dfbfe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d8072470 0x7fb2d81574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.315+0000 7fb2dfbfe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d8072970 0x7fb2d814fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.315+0000 7fb2dfbfe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d815a690 0x7fb2d8155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.315+0000 7fb2dfbfe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d815abf0 con 0x7fb2d8072970 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.315+0000 7fb2dfbfe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d815aa70 con 0x7fb2d8072470 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.315+0000 7fb2dfbfe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d815ad70 con 0x7fb2d815a690 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.315+0000 7fb2dd199640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d8072970 0x7fb2d814fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.315+0000 7fb2dd199640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d8072970 0x7fb2d814fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49636/0 (socket says 172.21.15.172:49636) 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.315+0000 7fb2dd199640 1 -- 172.21.15.172:0/2942954398 learned_addr learned my addr 172.21.15.172:0/2942954398 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2dd99a640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d8072470 0x7fb2d81574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2de19b640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d815a690 0x7fb2d8155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2dd199640 1 -- 172.21.15.172:0/2942954398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d815a690 msgr2=0x7fb2d8155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2dd199640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d815a690 0x7fb2d8155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2dd199640 1 -- 172.21.15.172:0/2942954398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d8072470 msgr2=0x7fb2d81574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2dd199640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d8072470 0x7fb2d81574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2dd199640 1 -- 172.21.15.172:0/2942954398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d8155e50 con 0x7fb2d8072970 2024-04-14T08:38:41.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2de19b640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d815a690 0x7fb2d8155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:41.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2dd199640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d8072970 0x7fb2d814fa30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c430 tx=0x7fb2d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:41.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2ce7fc640 1 -- 172.21.15.172:0/2942954398 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4011790 con 0x7fb2d8072970 2024-04-14T08:38:41.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2ce7fc640 1 -- 172.21.15.172:0/2942954398 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d4017070 con 0x7fb2d8072970 2024-04-14T08:38:41.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2ce7fc640 1 -- 172.21.15.172:0/2942954398 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4011bb0 con 0x7fb2d8072970 2024-04-14T08:38:41.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/2942954398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d81560f0 con 0x7fb2d8072970 2024-04-14T08:38:41.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.316+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/2942954398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d81566b0 con 0x7fb2d8072970 2024-04-14T08:38:41.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.317+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/2942954398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d8068520 con 0x7fb2d8072970 2024-04-14T08:38:41.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.317+0000 7fb2ce7fc640 1 -- 172.21.15.172:0/2942954398 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2d401c460 con 0x7fb2d8072970 2024-04-14T08:38:41.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.318+0000 7fb2ce7fc640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb2bc06c8f0 0x7fb2bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.318+0000 7fb2ce7fc640 1 -- 172.21.15.172:0/2942954398 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5249+0+0 (secure 0 0 0) 0x7fb2d408b720 con 0x7fb2d8072970 2024-04-14T08:38:41.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.318+0000 7fb2dd99a640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb2bc06c8f0 0x7fb2bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.318+0000 7fb2dd99a640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb2bc06c8f0 0x7fb2bc06edb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8007fd0 tx=0x7fb2c8007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:41.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.323+0000 7fb2ce7fc640 1 -- 172.21.15.172:0/2942954398 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb2d4059da0 con 0x7fb2d8072970 2024-04-14T08:38:41.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.564+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/2942954398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb2d806b520 con 0x7fb2d8072970 2024-04-14T08:38:41.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.565+0000 7fb2ce7fc640 1 -- 172.21.15.172:0/2942954398 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7fb2d40596a0 con 0x7fb2d8072970 2024-04-14T08:38:41.565 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:41.565 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:38:41.565 INFO:teuthology.orchestra.run.smithi172.stdout: "auth_dump": [ 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.a", 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmnNCRDRAAz7aU3JD7iiaTQH0cNotfaw==", 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.b", 2024-04-14T08:38:41.566 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtm1on4GRAAms1he+O4jWu01z0ZLLgL0A==", 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.c", 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmb7z0FBAAGVu3+TWshDGvd7/fLbgJ5A==", 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:38:41.567 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.d", 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmEEx0IRAAXiFiOK1RE/0bfyu7KI/3ng==", 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.568 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.0", 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB/lBtmnsWuCRAAqcuQhsrzpGVfrlgwTnFsOw==", 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.1", 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCClBtme9lPBxAAFB1E3g8qwZxPCdHiFogp5Q==", 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:38:41.569 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.2", 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCFlBtm966YBRAAWXZGq/0TuUMZTsyeR4xWww==", 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.570 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.3", 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCIlBtmHpktBBAAUbddNHESwyNvvp8UE/SaCw==", 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.4", 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCOlBtm+//7ChAA7rApbKll2vBGcDlW8q+kyg==", 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.571 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.5", 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCRlBtmVU4YChAAysdUosQ+TnHtjnRl1FOISA==", 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.6", 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCUlBtmnE7+BxAAqBn8YMwecA2NcZz63eBm7Q==", 2024-04-14T08:38:41.572 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.7", 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCXlBtmN8DhBRAA6CHQ943t6qJCL77jcbOv4w==", 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.573 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.0", 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA==", 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow r", 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.1", 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmp4nkLhAAhYKm8VcykVbPSohvK11I9w==", 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.574 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.2", 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmJoV9KxAAp6zlRX+gvhASQBHjpF9ewQ==", 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.575 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.3", 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmeC+pNBAASi7W4jlo6imMMzNEwLrPDA==", 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.admin", 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB6lBtmgnWqLBAAQ0fgarM8au3zFuMNnQIvVw==", 2024-04-14T08:38:41.576 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.x", 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmm5IbChAAeVcVs3Zt6+lU0wghjCcwnQ==", 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.577 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:38:41.578 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:41.578 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.y", 2024-04-14T08:38:41.578 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmfddYBhAAh8zgf4W3WWmNYWNAA7WuRw==", 2024-04-14T08:38:41.578 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:38:41.578 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:38:41.578 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:38:41.578 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:38:41.578 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.578 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:41.578 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-04-14T08:38:41.578 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:38:41.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.566+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/2942954398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb2bc06c8f0 msgr2=0x7fb2bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.566+0000 7fb2dfbfe640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb2bc06c8f0 0x7fb2bc06edb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8007fd0 tx=0x7fb2c8007e80 comp rx=0 tx=0).stop 2024-04-14T08:38:41.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.566+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/2942954398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d8072970 msgr2=0x7fb2d814fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.566+0000 7fb2dfbfe640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d8072970 0x7fb2d814fa30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c430 tx=0x7fb2d400c8f0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.566+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/2942954398 shutdown_connections 2024-04-14T08:38:41.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.566+0000 7fb2dfbfe640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb2bc06c8f0 0x7fb2bc06edb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.566+0000 7fb2dfbfe640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2d815a690 0x7fb2d8155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.566+0000 7fb2dfbfe640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2d8072970 0x7fb2d814fa30 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.566+0000 7fb2dfbfe640 1 --2- 172.21.15.172:0/2942954398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2d8072470 0x7fb2d81574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.566+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/2942954398 >> 172.21.15.172:0/2942954398 conn(0x7fb2d806ec30 msgr2=0x7fb2d80d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:41.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.567+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/2942954398 shutdown_connections 2024-04-14T08:38:41.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.567+0000 7fb2dfbfe640 1 -- 172.21.15.172:0/2942954398 wait complete. 2024-04-14T08:38:41.581 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:38:41.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.681+0000 7fcee4684640 1 Processor -- start 2024-04-14T08:38:41.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.681+0000 7fcee4684640 1 -- start start 2024-04-14T08:38:41.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.682+0000 7fcee4684640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcedc0d2830 0x7fcedc0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.682+0000 7fcee4684640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcedc15e890 0x7fcedc0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.682+0000 7fcee4684640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcedc0d3150 con 0x7fcedc0cedd0 2024-04-14T08:38:41.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.682+0000 7fcee4684640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fcedc0cff30 con 0x7fcedc15e890 2024-04-14T08:38:41.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.682+0000 7fcee4684640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fcedc0d2350 con 0x7fcedc0d2830 2024-04-14T08:38:41.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.682+0000 7fcee1c1f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcedc15e890 0x7fcedc0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.682+0000 7fcee1c1f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcedc15e890 0x7fcedc0d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49650/0 (socket says 172.21.15.172:49650) 2024-04-14T08:38:41.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.682+0000 7fcee1c1f640 1 -- 172.21.15.172:0/4036550998 learned_addr learned my addr 172.21.15.172:0/4036550998 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:41.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.683+0000 7fcee2420640 1 --2- 172.21.15.172:0/4036550998 >> v2:172.21.15.177:3300/0 conn(0x7fcedc0d2830 0x7fcedc0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.683+0000 7fcee1c1f640 1 -- 172.21.15.172:0/4036550998 >> v2:172.21.15.177:3300/0 conn(0x7fcedc0d2830 msgr2=0x7fcedc0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:41.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.683+0000 7fcee1c1f640 1 --2- 172.21.15.172:0/4036550998 >> v2:172.21.15.177:3300/0 conn(0x7fcedc0d2830 0x7fcedc0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.683+0000 7fcee1c1f640 1 -- 172.21.15.172:0/4036550998 >> v1:172.21.15.177:6789/0 conn(0x7fcedc0cedd0 legacy=0x7fcedc1621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.683+0000 7fcee1c1f640 1 -- 172.21.15.172:0/4036550998 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc0cf320 con 0x7fcedc15e890 2024-04-14T08:38:41.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.683+0000 7fcee1c1f640 1 --2- 172.21.15.172:0/4036550998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcedc15e890 0x7fcedc0d1300 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcedc08c830 tx=0x7fcecc02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91c2d4a4d55b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:41.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.684+0000 7fcee141e640 1 -- 172.21.15.172:0/4036550998 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc02f850 con 0x7fcedc15e890 2024-04-14T08:38:41.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.684+0000 7fcee141e640 1 -- 172.21.15.172:0/4036550998 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcecc02f9f0 con 0x7fcedc15e890 2024-04-14T08:38:41.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.684+0000 7fcee141e640 1 -- 172.21.15.172:0/4036550998 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc02fdc0 con 0x7fcedc15e890 2024-04-14T08:38:41.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.684+0000 7fcee4684640 1 -- 172.21.15.172:0/4036550998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcedc15e890 msgr2=0x7fcedc0d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.684+0000 7fcee4684640 1 --2- 172.21.15.172:0/4036550998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcedc15e890 0x7fcedc0d1300 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcedc08c830 tx=0x7fcecc02ba30 comp rx=0 tx=0).stop 2024-04-14T08:38:41.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.684+0000 7fcee4684640 1 -- 172.21.15.172:0/4036550998 shutdown_connections 2024-04-14T08:38:41.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.684+0000 7fcee4684640 1 --2- 172.21.15.172:0/4036550998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcedc15e890 0x7fcedc0d1300 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.684+0000 7fcee4684640 1 --2- 172.21.15.172:0/4036550998 >> v2:172.21.15.177:3300/0 conn(0x7fcedc0d2830 0x7fcedc0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.684+0000 7fcee4684640 1 -- 172.21.15.172:0/4036550998 >> 172.21.15.172:0/4036550998 conn(0x7fcedc09d590 msgr2=0x7fcedc0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:41.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.684+0000 7fcee4684640 1 -- 172.21.15.172:0/4036550998 shutdown_connections 2024-04-14T08:38:41.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.684+0000 7fcee4684640 1 -- 172.21.15.172:0/4036550998 wait complete. 2024-04-14T08:38:41.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.685+0000 7fcee4684640 1 Processor -- start 2024-04-14T08:38:41.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.686+0000 7fcee4684640 1 -- start start 2024-04-14T08:38:41.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.686+0000 7fcee4684640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcedc0cedd0 0x7fcedc15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.686+0000 7fcee4684640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcedc0d2830 0x7fcedc15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.686+0000 7fcee4684640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcedc15e890 0x7fcedc1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.686+0000 7fcee4684640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc0cfbc0 con 0x7fcedc0d2830 2024-04-14T08:38:41.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.686+0000 7fcee4684640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc0cf8e0 con 0x7fcedc0cedd0 2024-04-14T08:38:41.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.686+0000 7fcee4684640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcedc0d1840 con 0x7fcedc15e890 2024-04-14T08:38:41.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.686+0000 7fcee1c1f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcedc0d2830 0x7fcedc15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.686+0000 7fcee1c1f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcedc0d2830 0x7fcedc15cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49652/0 (socket says 172.21.15.172:49652) 2024-04-14T08:38:41.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.686+0000 7fcee1c1f640 1 -- 172.21.15.172:0/190727438 learned_addr learned my addr 172.21.15.172:0/190727438 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:41.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.686+0000 7fcee2420640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcedc0cedd0 0x7fcedc15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.686+0000 7fcee2c21640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcedc15e890 0x7fcedc1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.687+0000 7fcee1c1f640 1 -- 172.21.15.172:0/190727438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcedc15e890 msgr2=0x7fcedc1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.687+0000 7fcee1c1f640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcedc15e890 0x7fcedc1614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.687+0000 7fcee1c1f640 1 -- 172.21.15.172:0/190727438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcedc0cedd0 msgr2=0x7fcedc15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.687+0000 7fcee1c1f640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcedc0cedd0 0x7fcedc15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.687+0000 7fcee1c1f640 1 -- 172.21.15.172:0/190727438 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc160400 con 0x7fcedc0d2830 2024-04-14T08:38:41.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.687+0000 7fcee1c1f640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcedc0d2830 0x7fcedc15cb20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fcecc02ba70 tx=0x7fcecc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:41.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.687+0000 7fced2ffd640 1 -- 172.21.15.172:0/190727438 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc037d10 con 0x7fcedc0d2830 2024-04-14T08:38:41.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.687+0000 7fced2ffd640 1 -- 172.21.15.172:0/190727438 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcecc046070 con 0x7fcedc0d2830 2024-04-14T08:38:41.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.687+0000 7fced2ffd640 1 -- 172.21.15.172:0/190727438 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc041640 con 0x7fcedc0d2830 2024-04-14T08:38:41.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.687+0000 7fcee4684640 1 -- 172.21.15.172:0/190727438 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcedc1606e0 con 0x7fcedc0d2830 2024-04-14T08:38:41.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.687+0000 7fcee4684640 1 -- 172.21.15.172:0/190727438 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcedc160ca0 con 0x7fcedc0d2830 2024-04-14T08:38:41.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.688+0000 7fcee4684640 1 -- 172.21.15.172:0/190727438 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcedc068520 con 0x7fcedc0d2830 2024-04-14T08:38:41.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.689+0000 7fced2ffd640 1 -- 172.21.15.172:0/190727438 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcecc040070 con 0x7fcedc0d2830 2024-04-14T08:38:41.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.689+0000 7fced2ffd640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcec006c8f0 0x7fcec006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.689+0000 7fced2ffd640 1 -- 172.21.15.172:0/190727438 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5249+0+0 (secure 0 0 0) 0x7fcecc0ade80 con 0x7fcedc0d2830 2024-04-14T08:38:41.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.689+0000 7fcee2420640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcec006c8f0 0x7fcec006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.689+0000 7fcee2420640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcec006c8f0 0x7fcec006edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0cae30 tx=0x7fced800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:41.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.693+0000 7fced2ffd640 1 -- 172.21.15.172:0/190727438 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcecc033030 con 0x7fcedc0d2830 2024-04-14T08:38:41.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.870+0000 7fcee4684640 1 -- 172.21.15.172:0/190727438 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcedc0d3460 con 0x7fcedc0d2830 2024-04-14T08:38:41.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.871+0000 7fced2ffd640 1 -- 172.21.15.172:0/190727438 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+13225 (secure 0 0 0) 0x7fcecc07bfd0 con 0x7fcedc0d2830 2024-04-14T08:38:41.872 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:41.872 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":82,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:38:41.202667+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:38:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.873+0000 7fcee4684640 1 -- 172.21.15.172:0/190727438 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcec006c8f0 msgr2=0x7fcec006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.873+0000 7fcee4684640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcec006c8f0 0x7fcec006edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0cae30 tx=0x7fced800b040 comp rx=0 tx=0).stop 2024-04-14T08:38:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.873+0000 7fcee4684640 1 -- 172.21.15.172:0/190727438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcedc0d2830 msgr2=0x7fcedc15cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.873+0000 7fcee4684640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcedc0d2830 0x7fcedc15cb20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fcecc02ba70 tx=0x7fcecc004730 comp rx=0 tx=0).stop 2024-04-14T08:38:41.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.873+0000 7fcee4684640 1 -- 172.21.15.172:0/190727438 shutdown_connections 2024-04-14T08:38:41.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.873+0000 7fcee4684640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcec006c8f0 0x7fcec006edb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.873+0000 7fcee4684640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcedc15e890 0x7fcedc1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.873+0000 7fcee4684640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcedc0d2830 0x7fcedc15cb20 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.873+0000 7fcee4684640 1 --2- 172.21.15.172:0/190727438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcedc0cedd0 0x7fcedc15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.873+0000 7fcee4684640 1 -- 172.21.15.172:0/190727438 >> 172.21.15.172:0/190727438 conn(0x7fcedc09d590 msgr2=0x7fcedc0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:41.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.873+0000 7fcee4684640 1 -- 172.21.15.172:0/190727438 shutdown_connections 2024-04-14T08:38:41.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.873+0000 7fcee4684640 1 -- 172.21.15.172:0/190727438 wait complete. 2024-04-14T08:38:41.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:38:41.885 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:38:41.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.989+0000 7fd53dd1e640 1 Processor -- start 2024-04-14T08:38:41.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.989+0000 7fd53dd1e640 1 -- start start 2024-04-14T08:38:41.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd53dd1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd53813df30 0x7fd5380b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd53dd1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538140e80 0x7fd5380b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd53dd1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd53815b620 con 0x7fd53815c380 2024-04-14T08:38:41.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd53dd1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fd5380cf9b0 con 0x7fd53813df30 2024-04-14T08:38:41.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd53dd1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd5381414a0 con 0x7fd538140e80 2024-04-14T08:38:41.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd536ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd53815c380 0x7fd538140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45702/0 (socket says 172.21.15.172:45702) 2024-04-14T08:38:41.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd536ffd640 1 -- 172.21.15.172:0/1347963741 learned_addr learned my addr 172.21.15.172:0/1347963741 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:41.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd537fff640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538140e80 0x7fd5380b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd5377fe640 1 --2- 172.21.15.172:0/1347963741 >> v2:172.21.15.177:3300/0 conn(0x7fd53813df30 0x7fd5380b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 933195063 0 0) 0x7fd52c001110 con 0x7fd53815c380 2024-04-14T08:38:41.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd520005730 con 0x7fd53815c380 2024-04-14T08:38:41.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 232383362 0 0) 0x7fd520005730 con 0x7fd53815c380 2024-04-14T08:38:41.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538140e80 msgr2=0x7fd5380b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:41.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd5367fc640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538140e80 0x7fd5380b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 >> v2:172.21.15.177:3300/0 conn(0x7fd53813df30 msgr2=0x7fd5380b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:41.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.990+0000 7fd5367fc640 1 --2- 172.21.15.172:0/1347963741 >> v2:172.21.15.177:3300/0 conn(0x7fd53813df30 0x7fd5380b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53815b8a0 con 0x7fd53815c380 2024-04-14T08:38:41.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fd52c001840 con 0x7fd53815c380 2024-04-14T08:38:41.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 >> v1:172.21.15.172:6789/0 conn(0x7fd53815c380 legacy=0x7fd538140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5367fc640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520005bc0 0x7fd520005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5367fc640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd520006a10 0x7fd520006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5367fc640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538140e80 0x7fd5200073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:41.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd5380cf9b0 con 0x7fd520005bc0 2024-04-14T08:38:41.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381414a0 con 0x7fd520006a10 2024-04-14T08:38:41.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd52000dbd0 con 0x7fd538140e80 2024-04-14T08:38:41.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5377fe640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520005bc0 0x7fd520005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd537fff640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538140e80 0x7fd5200073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd536ffd640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd520006a10 0x7fd520006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:41.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5377fe640 1 -- 172.21.15.172:0/1347963741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538140e80 msgr2=0x7fd5200073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:41.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5377fe640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538140e80 0x7fd5200073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5377fe640 1 -- 172.21.15.172:0/1347963741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd520006a10 msgr2=0x7fd520006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:41.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5377fe640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd520006a10 0x7fd520006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:41.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd5377fe640 1 -- 172.21.15.172:0/1347963741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd520005730 con 0x7fd520005bc0 2024-04-14T08:38:41.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.991+0000 7fd536ffd640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd520006a10 0x7fd520006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:41.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.992+0000 7fd5377fe640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520005bc0 0x7fd520005fe0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fd528009ac0 tx=0x7fd520007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5eebf0f02cd608 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:41.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.992+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528039070 con 0x7fd520005bc0 2024-04-14T08:38:41.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.992+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd52802c710 con 0x7fd520005bc0 2024-04-14T08:38:41.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.992+0000 7fd5367fc640 1 -- 172.21.15.172:0/1347963741 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528009690 con 0x7fd520005bc0 2024-04-14T08:38:41.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.992+0000 7fd53dd1e640 1 -- 172.21.15.172:0/1347963741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520005bc0 msgr2=0x7fd520005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:41.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.992+0000 7fd53dd1e640 1 --2- 172.21.15.172:0/1347963741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520005bc0 0x7fd520005fe0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fd528009ac0 tx=0x7fd520007a40 comp rx=0 tx=0).stop 2024-04-14T08:38:41.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.992+0000 7fd537fff640 1 -- 172.21.15.172:0/1347963741 reap_dead start 2024-04-14T08:38:41.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.992+0000 7fd53dd1e640 1 -- 172.21.15.172:0/1347963741 shutdown_connections 2024-04-14T08:38:41.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.992+0000 7fd53dd1e640 1 -- 172.21.15.172:0/1347963741 >> 172.21.15.172:0/1347963741 conn(0x7fd53806b520 msgr2=0x7fd53806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:41.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.992+0000 7fd53dd1e640 1 -- 172.21.15.172:0/1347963741 shutdown_connections 2024-04-14T08:38:41.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.992+0000 7fd53dd1e640 1 -- 172.21.15.172:0/1347963741 wait complete. 2024-04-14T08:38:42.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.994+0000 7fd53dd1e640 1 Processor -- start 2024-04-14T08:38:42.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.994+0000 7fd53dd1e640 1 -- start start 2024-04-14T08:38:42.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.994+0000 7fd53dd1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520005bc0 0x7fd538072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.994+0000 7fd53dd1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538138780 0x7fd538137f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.994+0000 7fd53dd1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd538158340 0x7fd5380b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.994+0000 7fd53dd1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd53813e310 con 0x7fd520005bc0 2024-04-14T08:38:42.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.994+0000 7fd53dd1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd53815be60 con 0x7fd538158340 2024-04-14T08:38:42.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.994+0000 7fd53dd1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd5380b2210 con 0x7fd538138780 2024-04-14T08:38:42.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd5377fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520005bc0 0x7fd538072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd5377fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520005bc0 0x7fd538072560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49670/0 (socket says 172.21.15.172:49670) 2024-04-14T08:38:42.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd5377fe640 1 -- 172.21.15.172:0/2727149390 learned_addr learned my addr 172.21.15.172:0/2727149390 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:42.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd536ffd640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538138780 0x7fd538137f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd537fff640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd538158340 0x7fd5380b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd5377fe640 1 -- 172.21.15.172:0/2727149390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538138780 msgr2=0x7fd538137f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd5377fe640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538138780 0x7fd538137f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd5377fe640 1 -- 172.21.15.172:0/2727149390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd538158340 msgr2=0x7fd5380b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd5377fe640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd538158340 0x7fd5380b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd5377fe640 1 -- 172.21.15.172:0/2727149390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd538155df0 con 0x7fd520005bc0 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd537fff640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd538158340 0x7fd5380b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd5377fe640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520005bc0 0x7fd538072560 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd528000c00 tx=0x7fd528004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd51ffff640 1 -- 172.21.15.172:0/2727149390 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528039050 con 0x7fd520005bc0 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd51ffff640 1 -- 172.21.15.172:0/2727149390 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd52803a040 con 0x7fd520005bc0 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd53dd1e640 1 -- 172.21.15.172:0/2727149390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5381560d0 con 0x7fd520005bc0 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd51ffff640 1 -- 172.21.15.172:0/2727149390 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528005bd0 con 0x7fd520005bc0 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.995+0000 7fd53dd1e640 1 -- 172.21.15.172:0/2727149390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd538157e20 con 0x7fd520005bc0 2024-04-14T08:38:42.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.996+0000 7fd53dd1e640 1 -- 172.21.15.172:0/2727149390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd538068590 con 0x7fd520005bc0 2024-04-14T08:38:42.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.996+0000 7fd51ffff640 1 -- 172.21.15.172:0/2727149390 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd528047020 con 0x7fd520005bc0 2024-04-14T08:38:42.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.997+0000 7fd51ffff640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd52406c8f0 0x7fd52406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.997+0000 7fd51ffff640 1 -- 172.21.15.172:0/2727149390 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5249+0+0 (secure 0 0 0) 0x7fd5280ad0b0 con 0x7fd520005bc0 2024-04-14T08:38:42.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.997+0000 7fd536ffd640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd52406c8f0 0x7fd52406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:41.997+0000 7fd536ffd640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd52406c8f0 0x7fd52406edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd5381f35a0 tx=0x7fd52c007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:42.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.001+0000 7fd51ffff640 1 -- 172.21.15.172:0/2727149390 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd52807b200 con 0x7fd520005bc0 2024-04-14T08:38:42.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.176+0000 7fd53dd1e640 1 -- 172.21.15.172:0/2727149390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd5380b25e0 con 0x7fd520005bc0 2024-04-14T08:38:42.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.176+0000 7fd51ffff640 1 -- 172.21.15.172:0/2727149390 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fd52807f500 con 0x7fd520005bc0 2024-04-14T08:38:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:38:42.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.178+0000 7fd53dd1e640 1 -- 172.21.15.172:0/2727149390 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd52406c8f0 msgr2=0x7fd52406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.178+0000 7fd53dd1e640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd52406c8f0 0x7fd52406edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd5381f35a0 tx=0x7fd52c007040 comp rx=0 tx=0).stop 2024-04-14T08:38:42.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.178+0000 7fd53dd1e640 1 -- 172.21.15.172:0/2727149390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520005bc0 msgr2=0x7fd538072560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.178+0000 7fd53dd1e640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520005bc0 0x7fd538072560 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd528000c00 tx=0x7fd528004730 comp rx=0 tx=0).stop 2024-04-14T08:38:42.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.178+0000 7fd53dd1e640 1 -- 172.21.15.172:0/2727149390 shutdown_connections 2024-04-14T08:38:42.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.178+0000 7fd53dd1e640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd52406c8f0 0x7fd52406edb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.178+0000 7fd53dd1e640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd538158340 0x7fd5380b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.178+0000 7fd53dd1e640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd538138780 0x7fd538137f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.178+0000 7fd53dd1e640 1 --2- 172.21.15.172:0/2727149390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520005bc0 0x7fd538072560 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.178+0000 7fd53dd1e640 1 -- 172.21.15.172:0/2727149390 >> 172.21.15.172:0/2727149390 conn(0x7fd53806b520 msgr2=0x7fd53815a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:42.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.178+0000 7fd53dd1e640 1 -- 172.21.15.172:0/2727149390 shutdown_connections 2024-04-14T08:38:42.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.179+0000 7fd53dd1e640 1 -- 172.21.15.172:0/2727149390 wait complete. 2024-04-14T08:38:42.191 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-14T08:38:42.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.291+0000 7fb0a8a36640 1 Processor -- start 2024-04-14T08:38:42.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.291+0000 7fb0a8a36640 1 -- start start 2024-04-14T08:38:42.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.292+0000 7fb0a8a36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb0a415f400 0x7fb0a415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.292+0000 7fb0a8a36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0a415d9d0 0x7fb0a415d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.292+0000 7fb0a8a36640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0a413f600 con 0x7fb0a40b1b80 2024-04-14T08:38:42.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.292+0000 7fb0a8a36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb0a4140690 con 0x7fb0a415f400 2024-04-14T08:38:42.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.292+0000 7fb0a8a36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a413fa80 con 0x7fb0a415d9d0 2024-04-14T08:38:42.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.292+0000 7fb0a259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb0a415f400 0x7fb0a415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.292+0000 7fb0a259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb0a415f400 0x7fb0a415f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49678/0 (socket says 172.21.15.172:49678) 2024-04-14T08:38:42.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.292+0000 7fb0a259c640 1 -- 172.21.15.172:0/345819063 learned_addr learned my addr 172.21.15.172:0/345819063 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:42.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.293+0000 7fb0a1d9b640 1 --2- 172.21.15.172:0/345819063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0a415d9d0 0x7fb0a415d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.293+0000 7fb0a259c640 1 -- 172.21.15.172:0/345819063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0a415d9d0 msgr2=0x7fb0a415d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:42.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.293+0000 7fb0a259c640 1 --2- 172.21.15.172:0/345819063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0a415d9d0 0x7fb0a415d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.293+0000 7fb0a259c640 1 -- 172.21.15.172:0/345819063 >> v1:172.21.15.177:6789/0 conn(0x7fb0a40b1b80 legacy=0x7fb0a4140b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:42.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.293+0000 7fb0a259c640 1 -- 172.21.15.172:0/345819063 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a413fd60 con 0x7fb0a415f400 2024-04-14T08:38:42.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.293+0000 7fb0a1d9b640 1 --2- 172.21.15.172:0/345819063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0a415d9d0 0x7fb0a415d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:42.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.293+0000 7fb0a259c640 1 --2- 172.21.15.172:0/345819063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a415f400 0x7fb0a415f7e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40b5200 tx=0x7fb09402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76c99e25826d06f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:42.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.294+0000 7fb0a159a640 1 -- 172.21.15.172:0/345819063 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09402ca30 con 0x7fb0a415f400 2024-04-14T08:38:42.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.294+0000 7fb0a159a640 1 -- 172.21.15.172:0/345819063 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb09402cbd0 con 0x7fb0a415f400 2024-04-14T08:38:42.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.294+0000 7fb0a159a640 1 -- 172.21.15.172:0/345819063 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0940343f0 con 0x7fb0a415f400 2024-04-14T08:38:42.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.294+0000 7fb0a8a36640 1 -- 172.21.15.172:0/345819063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a415f400 msgr2=0x7fb0a415f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.294+0000 7fb0a8a36640 1 --2- 172.21.15.172:0/345819063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a415f400 0x7fb0a415f7e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40b5200 tx=0x7fb09402be90 comp rx=0 tx=0).stop 2024-04-14T08:38:42.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.294+0000 7fb0a8a36640 1 -- 172.21.15.172:0/345819063 shutdown_connections 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.294+0000 7fb0a8a36640 1 --2- 172.21.15.172:0/345819063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0a415d9d0 0x7fb0a415d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.294+0000 7fb0a8a36640 1 --2- 172.21.15.172:0/345819063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a415f400 0x7fb0a415f7e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.294+0000 7fb0a8a36640 1 -- 172.21.15.172:0/345819063 >> 172.21.15.172:0/345819063 conn(0x7fb0a406ec30 msgr2=0x7fb0a406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.294+0000 7fb0a8a36640 1 -- 172.21.15.172:0/345819063 shutdown_connections 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.294+0000 7fb0a8a36640 1 -- 172.21.15.172:0/345819063 wait complete. 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.295+0000 7fb0a8a36640 1 Processor -- start 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.295+0000 7fb0a8a36640 1 -- start start 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.296+0000 7fb0a8a36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0a40b1b80 0x7fb0a4134dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.296+0000 7fb0a8a36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a415d9d0 0x7fb0a40ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.296+0000 7fb0a8a36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0a415f400 0x7fb0a4136610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.296+0000 7fb0a8a36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a415de30 con 0x7fb0a415d9d0 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.296+0000 7fb0a8a36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a4140320 con 0x7fb0a40b1b80 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.296+0000 7fb0a8a36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a415d6d0 con 0x7fb0a415f400 2024-04-14T08:38:42.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.296+0000 7fb0a1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a415d9d0 0x7fb0a40ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.296+0000 7fb0a1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a415d9d0 0x7fb0a40ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49684/0 (socket says 172.21.15.172:49684) 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.296+0000 7fb0a1d9b640 1 -- 172.21.15.172:0/1543722053 learned_addr learned my addr 172.21.15.172:0/1543722053 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.296+0000 7fb0a259c640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0a40b1b80 0x7fb0a4134dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.296+0000 7fb0a2d9d640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0a415f400 0x7fb0a4136610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.297+0000 7fb0a1d9b640 1 -- 172.21.15.172:0/1543722053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0a415f400 msgr2=0x7fb0a4136610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.297+0000 7fb0a1d9b640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0a415f400 0x7fb0a4136610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.297+0000 7fb0a1d9b640 1 -- 172.21.15.172:0/1543722053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0a40b1b80 msgr2=0x7fb0a4134dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.297+0000 7fb0a1d9b640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0a40b1b80 0x7fb0a4134dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.297+0000 7fb0a1d9b640 1 -- 172.21.15.172:0/1543722053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a41517e0 con 0x7fb0a415d9d0 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.297+0000 7fb0a1d9b640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a415d9d0 0x7fb0a40ae7e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb09800c7f0 tx=0x7fb09800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.297+0000 7fb092ffd640 1 -- 172.21.15.172:0/1543722053 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb098015070 con 0x7fb0a415d9d0 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.297+0000 7fb092ffd640 1 -- 172.21.15.172:0/1543722053 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb098007a30 con 0x7fb0a415d9d0 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.297+0000 7fb0a8a36640 1 -- 172.21.15.172:0/1543722053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a4151ac0 con 0x7fb0a415d9d0 2024-04-14T08:38:42.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.297+0000 7fb092ffd640 1 -- 172.21.15.172:0/1543722053 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb098007e00 con 0x7fb0a415d9d0 2024-04-14T08:38:42.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.297+0000 7fb0a8a36640 1 -- 172.21.15.172:0/1543722053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a4152080 con 0x7fb0a415d9d0 2024-04-14T08:38:42.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.298+0000 7fb0a8a36640 1 -- 172.21.15.172:0/1543722053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a4068520 con 0x7fb0a415d9d0 2024-04-14T08:38:42.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.298+0000 7fb092ffd640 1 -- 172.21.15.172:0/1543722053 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb098018070 con 0x7fb0a415d9d0 2024-04-14T08:38:42.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.299+0000 7fb092ffd640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb08406c8f0 0x7fb08406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.299+0000 7fb0a259c640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb08406c8f0 0x7fb08406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.299+0000 7fb092ffd640 1 -- 172.21.15.172:0/1543722053 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5249+0+0 (secure 0 0 0) 0x7fb09808b810 con 0x7fb0a415d9d0 2024-04-14T08:38:42.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.299+0000 7fb0a259c640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb08406c8f0 0x7fb08406edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb094035000 tx=0x7fb094030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:42.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.303+0000 7fb092ffd640 1 -- 172.21.15.172:0/1543722053 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb098059840 con 0x7fb0a415d9d0 2024-04-14T08:38:42.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.480+0000 7fb0a8a36640 1 -- 172.21.15.172:0/1543722053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb0a40aed20 con 0x7fb0a415d9d0 2024-04-14T08:38:42.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.480+0000 7fb092ffd640 1 -- 172.21.15.172:0/1543722053 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7fb09805db40 con 0x7fb0a415d9d0 2024-04-14T08:38:42.481 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:42.482 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-04-14T08:38:42.482 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:38:42.482 INFO:teuthology.orchestra.run.smithi172.stdout: "poolnum": 1, 2024-04-14T08:38:42.482 INFO:teuthology.orchestra.run.smithi172.stdout: "poolname": ".mgr" 2024-04-14T08:38:42.482 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:38:42.482 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-04-14T08:38:42.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.482+0000 7fb0a8a36640 1 -- 172.21.15.172:0/1543722053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb08406c8f0 msgr2=0x7fb08406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.482+0000 7fb0a8a36640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb08406c8f0 0x7fb08406edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb094035000 tx=0x7fb094030000 comp rx=0 tx=0).stop 2024-04-14T08:38:42.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.482+0000 7fb0a8a36640 1 -- 172.21.15.172:0/1543722053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a415d9d0 msgr2=0x7fb0a40ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.482+0000 7fb0a8a36640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a415d9d0 0x7fb0a40ae7e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb09800c7f0 tx=0x7fb09800cbf0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.482+0000 7fb0a8a36640 1 -- 172.21.15.172:0/1543722053 shutdown_connections 2024-04-14T08:38:42.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.482+0000 7fb0a8a36640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb08406c8f0 0x7fb08406edb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.482+0000 7fb0a8a36640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0a415f400 0x7fb0a4136610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.482+0000 7fb0a8a36640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0a415d9d0 0x7fb0a40ae7e0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.482+0000 7fb0a8a36640 1 --2- 172.21.15.172:0/1543722053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0a40b1b80 0x7fb0a4134dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.482+0000 7fb0a8a36640 1 -- 172.21.15.172:0/1543722053 >> 172.21.15.172:0/1543722053 conn(0x7fb0a406ec30 msgr2=0x7fb0a413e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:42.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.483+0000 7fb0a8a36640 1 -- 172.21.15.172:0/1543722053 shutdown_connections 2024-04-14T08:38:42.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.483+0000 7fb0a8a36640 1 -- 172.21.15.172:0/1543722053 wait complete. 2024-04-14T08:38:42.494 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-14T08:38:42.494 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-14T08:38:42.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.598+0000 7f50a01fe640 1 Processor -- start 2024-04-14T08:38:42.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.599+0000 7f50a01fe640 1 -- start start 2024-04-14T08:38:42.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.599+0000 7f50a01fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f50980d2cf0 0x7f50981695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.599+0000 7f50a01fe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50980d1bb0 con 0x7f5098161d00 2024-04-14T08:38:42.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.599+0000 7f50a01fe640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50980d0df0 con 0x7f50980b30a0 2024-04-14T08:38:42.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.599+0000 7f50a01fe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f50980d0ba0 con 0x7f50980d2cf0 2024-04-14T08:38:42.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.600+0000 7f509df9a640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5098161d00 0x7f50980b5490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45714/0 (socket says 172.21.15.172:45714) 2024-04-14T08:38:42.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.600+0000 7f509df9a640 1 -- 172.21.15.172:0/4168855196 learned_addr learned my addr 172.21.15.172:0/4168855196 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:42.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.600+0000 7f509e79b640 1 --2- 172.21.15.172:0/4168855196 >> v2:172.21.15.177:3300/0 conn(0x7f50980d2cf0 0x7f50981695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.600+0000 7f509cf98640 1 -- 172.21.15.172:0/4168855196 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3810463362 0 0) 0x7f50980d1bb0 con 0x7f5098161d00 2024-04-14T08:38:42.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.600+0000 7f509cf98640 1 -- 172.21.15.172:0/4168855196 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5084003520 con 0x7f5098161d00 2024-04-14T08:38:42.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.600+0000 7f509cf98640 1 -- 172.21.15.172:0/4168855196 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1820480816 0 0) 0x7f5084003520 con 0x7f5098161d00 2024-04-14T08:38:42.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.600+0000 7f509e79b640 1 -- 172.21.15.172:0/4168855196 >> v1:172.21.15.177:6789/0 conn(0x7f50980b30a0 legacy=0x7f50980d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.600+0000 7f509e79b640 1 -- 172.21.15.172:0/4168855196 >> v1:172.21.15.172:6789/0 conn(0x7f5098161d00 legacy=0x7f50980b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.600+0000 7f509e79b640 1 -- 172.21.15.172:0/4168855196 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50980d10d0 con 0x7f50980d2cf0 2024-04-14T08:38:42.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.601+0000 7f509e79b640 1 --2- 172.21.15.172:0/4168855196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50980d2cf0 0x7f50981695f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5090009d90 tx=0x7f509002bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfd0b1ce5d5376d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:42.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.601+0000 7f509cf98640 1 -- 172.21.15.172:0/4168855196 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090039070 con 0x7f50980d2cf0 2024-04-14T08:38:42.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.601+0000 7f509cf98640 1 -- 172.21.15.172:0/4168855196 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509002c8b0 con 0x7f50980d2cf0 2024-04-14T08:38:42.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.601+0000 7f509cf98640 1 -- 172.21.15.172:0/4168855196 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094001670 con 0x7f50980d2cf0 2024-04-14T08:38:42.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.601+0000 7f50a01fe640 1 -- 172.21.15.172:0/4168855196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50980d2cf0 msgr2=0x7f50981695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.601+0000 7f50a01fe640 1 --2- 172.21.15.172:0/4168855196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50980d2cf0 0x7f50981695f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5090009d90 tx=0x7f509002bd80 comp rx=0 tx=0).stop 2024-04-14T08:38:42.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.601+0000 7f50a01fe640 1 -- 172.21.15.172:0/4168855196 shutdown_connections 2024-04-14T08:38:42.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.601+0000 7f50a01fe640 1 --2- 172.21.15.172:0/4168855196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50980d2cf0 0x7f50981695f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.602+0000 7f50a01fe640 1 -- 172.21.15.172:0/4168855196 >> 172.21.15.172:0/4168855196 conn(0x7f509809d570 msgr2=0x7f50980d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:42.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.602+0000 7f50a01fe640 1 -- 172.21.15.172:0/4168855196 shutdown_connections 2024-04-14T08:38:42.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.602+0000 7f50a01fe640 1 -- 172.21.15.172:0/4168855196 wait complete. 2024-04-14T08:38:42.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.603+0000 7f50a01fe640 1 Processor -- start 2024-04-14T08:38:42.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.603+0000 7f50a01fe640 1 -- start start 2024-04-14T08:38:42.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.603+0000 7f50a01fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50980b30a0 0x7f5098154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.603+0000 7f50a01fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50980d2cf0 0x7f50980b9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.603+0000 7f50a01fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5098161d00 0x7f50980b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.603+0000 7f50a01fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f50980d33b0 con 0x7f5098161d00 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.603+0000 7f50a01fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50980d1720 con 0x7f50980b30a0 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.603+0000 7f50a01fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f50980cff10 con 0x7f50980d2cf0 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509e79b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5098161d00 0x7f50980b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509e79b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5098161d00 0x7f50980b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49688/0 (socket says 172.21.15.172:49688) 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509e79b640 1 -- 172.21.15.172:0/1150186040 learned_addr learned my addr 172.21.15.172:0/1150186040 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509df9a640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50980b30a0 0x7f5098154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509d799640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50980d2cf0 0x7f50980b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509e79b640 1 -- 172.21.15.172:0/1150186040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50980d2cf0 msgr2=0x7f50980b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509e79b640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50980d2cf0 0x7f50980b9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509e79b640 1 -- 172.21.15.172:0/1150186040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50980b30a0 msgr2=0x7f5098154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:42.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509e79b640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50980b30a0 0x7f5098154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509e79b640 1 -- 172.21.15.172:0/1150186040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50981602b0 con 0x7f5098161d00 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509d799640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50980d2cf0 0x7f50980b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509df9a640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50980b30a0 0x7f5098154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f509e79b640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5098161d00 0x7f50980b5d00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5090037040 tx=0x7f5090004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f508e7fc640 1 -- 172.21.15.172:0/1150186040 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090044020 con 0x7f5098161d00 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f508e7fc640 1 -- 172.21.15.172:0/1150186040 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5090034d10 con 0x7f5098161d00 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f50a01fe640 1 -- 172.21.15.172:0/1150186040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5098160590 con 0x7f5098161d00 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f508e7fc640 1 -- 172.21.15.172:0/1150186040 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090039040 con 0x7f5098161d00 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.604+0000 7f50a01fe640 1 -- 172.21.15.172:0/1150186040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5098160b50 con 0x7f5098161d00 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.605+0000 7f50a01fe640 1 -- 172.21.15.172:0/1150186040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50980686a0 con 0x7f5098161d00 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.606+0000 7f508e7fc640 1 -- 172.21.15.172:0/1150186040 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5090047020 con 0x7f5098161d00 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.606+0000 7f508e7fc640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f507c06c8f0 0x7f507c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.606+0000 7f508e7fc640 1 -- 172.21.15.172:0/1150186040 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5249+0+0 (secure 0 0 0) 0x7f50900ae1d0 con 0x7f5098161d00 2024-04-14T08:38:42.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.606+0000 7f509df9a640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f507c06c8f0 0x7f507c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:42.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.607+0000 7f509df9a640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f507c06c8f0 0x7f507c06edb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5094005880 tx=0x7f5094009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:42.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.611+0000 7f508e7fc640 1 -- 172.21.15.172:0/1150186040 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f509004e030 con 0x7f5098161d00 2024-04-14T08:38:42.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:42.785+0000 7f50a01fe640 1 -- 172.21.15.172:0/1150186040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5098161520 con 0x7f5098161d00 2024-04-14T08:38:43.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.198+0000 7f508e7fc640 1 -- 172.21.15.172:0/1150186040 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f509007c730 con 0x7f5098161d00 2024-04-14T08:38:43.200 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' created 2024-04-14T08:38:43.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.201+0000 7f50a01fe640 1 -- 172.21.15.172:0/1150186040 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f507c06c8f0 msgr2=0x7f507c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:43.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.201+0000 7f50a01fe640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f507c06c8f0 0x7f507c06edb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5094005880 tx=0x7f5094009040 comp rx=0 tx=0).stop 2024-04-14T08:38:43.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.201+0000 7f50a01fe640 1 -- 172.21.15.172:0/1150186040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5098161d00 msgr2=0x7f50980b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:43.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.201+0000 7f50a01fe640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5098161d00 0x7f50980b5d00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5090037040 tx=0x7f5090004a40 comp rx=0 tx=0).stop 2024-04-14T08:38:43.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.201+0000 7f50a01fe640 1 -- 172.21.15.172:0/1150186040 shutdown_connections 2024-04-14T08:38:43.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.201+0000 7f50a01fe640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f507c06c8f0 0x7f507c06edb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:43.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.201+0000 7f50a01fe640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5098161d00 0x7f50980b5d00 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:43.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.201+0000 7f50a01fe640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50980d2cf0 0x7f50980b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:43.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.201+0000 7f50a01fe640 1 --2- 172.21.15.172:0/1150186040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50980b30a0 0x7f5098154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:43.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.201+0000 7f50a01fe640 1 -- 172.21.15.172:0/1150186040 >> 172.21.15.172:0/1150186040 conn(0x7f509809d570 msgr2=0x7f50980cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:43.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.201+0000 7f50a01fe640 1 -- 172.21.15.172:0/1150186040 shutdown_connections 2024-04-14T08:38:43.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.201+0000 7f50a01fe640 1 -- 172.21.15.172:0/1150186040 wait complete. 2024-04-14T08:38:43.217 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-14T08:38:43.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.323+0000 7fd6d21fe640 1 Processor -- start 2024-04-14T08:38:43.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.323+0000 7fd6d21fe640 1 -- start start 2024-04-14T08:38:43.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6d21fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd6cc13df30 0x7fd6cc0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:43.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6d21fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6cc15c380 0x7fd6cc0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:43.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6d21fe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6cc15b620 con 0x7fd6cc140770 2024-04-14T08:38:43.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6d21fe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd6cc0cf9b0 con 0x7fd6cc13df30 2024-04-14T08:38:43.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6d21fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd6cc1414a0 con 0x7fd6cc15c380 2024-04-14T08:38:43.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6cb7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd6cc13df30 0x7fd6cc0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:43.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6cbfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd6cc140770 0x7fd6cc140c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45716/0 (socket says 172.21.15.172:45716) 2024-04-14T08:38:43.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6cbfff640 1 -- 172.21.15.172:0/2269577878 learned_addr learned my addr 172.21.15.172:0/2269577878 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:43.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6cb7fe640 1 -- 172.21.15.172:0/2269577878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6cc15c380 msgr2=0x7fd6cc0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:43.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6cb7fe640 1 --2- 172.21.15.172:0/2269577878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6cc15c380 0x7fd6cc0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:43.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6cb7fe640 1 -- 172.21.15.172:0/2269577878 >> v1:172.21.15.172:6789/0 conn(0x7fd6cc140770 legacy=0x7fd6cc140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:43.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6cb7fe640 1 -- 172.21.15.172:0/2269577878 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc15b8a0 con 0x7fd6cc13df30 2024-04-14T08:38:43.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.324+0000 7fd6ca7fc640 1 -- 172.21.15.172:0/2269577878 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3772840056 0 0) 0x7fd6c4001490 con 0x7fd6cc140770 2024-04-14T08:38:43.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.325+0000 7fd6cb7fe640 1 --2- 172.21.15.172:0/2269577878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6cc13df30 0x7fd6cc0b1cd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0097d0 tx=0x7fd6bc02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa1c316eb989257 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:43.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.325+0000 7fd6ca7fc640 1 -- 172.21.15.172:0/2269577878 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc035020 con 0x7fd6cc13df30 2024-04-14T08:38:43.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.325+0000 7fd6ca7fc640 1 -- 172.21.15.172:0/2269577878 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6bc031800 con 0x7fd6cc13df30 2024-04-14T08:38:43.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.325+0000 7fd6ca7fc640 1 -- 172.21.15.172:0/2269577878 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc031bd0 con 0x7fd6cc13df30 2024-04-14T08:38:43.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.325+0000 7fd6d21fe640 1 -- 172.21.15.172:0/2269577878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6cc13df30 msgr2=0x7fd6cc0b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:43.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.325+0000 7fd6d21fe640 1 --2- 172.21.15.172:0/2269577878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6cc13df30 0x7fd6cc0b1cd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0097d0 tx=0x7fd6bc02b8a0 comp rx=0 tx=0).stop 2024-04-14T08:38:43.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.326+0000 7fd6d21fe640 1 -- 172.21.15.172:0/2269577878 shutdown_connections 2024-04-14T08:38:43.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.326+0000 7fd6d21fe640 1 --2- 172.21.15.172:0/2269577878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6cc15c380 0x7fd6cc0b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:43.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.326+0000 7fd6d21fe640 1 --2- 172.21.15.172:0/2269577878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6cc13df30 0x7fd6cc0b1cd0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:43.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.326+0000 7fd6d21fe640 1 -- 172.21.15.172:0/2269577878 >> 172.21.15.172:0/2269577878 conn(0x7fd6cc06b520 msgr2=0x7fd6cc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:43.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.326+0000 7fd6d21fe640 1 -- 172.21.15.172:0/2269577878 shutdown_connections 2024-04-14T08:38:43.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.326+0000 7fd6d21fe640 1 -- 172.21.15.172:0/2269577878 wait complete. 2024-04-14T08:38:43.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.327+0000 7fd6d21fe640 1 Processor -- start 2024-04-14T08:38:43.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.327+0000 7fd6d21fe640 1 -- start start 2024-04-14T08:38:43.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6d21fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6cc13df30 0x7fd6cc137ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:43.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6d21fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6cc140770 0x7fd6cc166ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:43.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6d21fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6cc15c380 0x7fd6cc150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:43.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6d21fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc13e310 con 0x7fd6cc13df30 2024-04-14T08:38:43.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6d21fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc15be60 con 0x7fd6cc15c380 2024-04-14T08:38:43.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6d21fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd6cc0ce9a0 con 0x7fd6cc140770 2024-04-14T08:38:43.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6cb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6cc13df30 0x7fd6cc137ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6cb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6cc13df30 0x7fd6cc137ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49702/0 (socket says 172.21.15.172:49702) 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6cb7fe640 1 -- 172.21.15.172:0/3095063535 learned_addr learned my addr 172.21.15.172:0/3095063535 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6cb7fe640 1 -- 172.21.15.172:0/3095063535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6cc140770 msgr2=0x7fd6cc166ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6cb7fe640 1 --2- 172.21.15.172:0/3095063535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6cc140770 0x7fd6cc166ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6cb7fe640 1 -- 172.21.15.172:0/3095063535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6cc15c380 msgr2=0x7fd6cc150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6cb7fe640 1 --2- 172.21.15.172:0/3095063535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6cc15c380 0x7fd6cc150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6cb7fe640 1 -- 172.21.15.172:0/3095063535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc0ae990 con 0x7fd6cc13df30 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6cb7fe640 1 --2- 172.21.15.172:0/3095063535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6cc13df30 0x7fd6cc137ff0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0098f0 tx=0x7fd6bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6b3fff640 1 -- 172.21.15.172:0/3095063535 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc03cd80 con 0x7fd6cc13df30 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6b3fff640 1 -- 172.21.15.172:0/3095063535 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6bc004030 con 0x7fd6cc13df30 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6d21fe640 1 -- 172.21.15.172:0/3095063535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6cc0af1f0 con 0x7fd6cc13df30 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6b3fff640 1 -- 172.21.15.172:0/3095063535 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc004420 con 0x7fd6cc13df30 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.328+0000 7fd6d21fe640 1 -- 172.21.15.172:0/3095063535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6cc0aef10 con 0x7fd6cc13df30 2024-04-14T08:38:43.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.329+0000 7fd6d21fe640 1 -- 172.21.15.172:0/3095063535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6cc068590 con 0x7fd6cc13df30 2024-04-14T08:38:43.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.330+0000 7fd6b3fff640 1 -- 172.21.15.172:0/3095063535 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6bc038020 con 0x7fd6cc13df30 2024-04-14T08:38:43.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.330+0000 7fd6b3fff640 1 --2- 172.21.15.172:0/3095063535 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd6b806c8f0 0x7fd6b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:43.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.330+0000 7fd6b3fff640 1 -- 172.21.15.172:0/3095063535 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5619+0+0 (secure 0 0 0) 0x7fd6bc0ae050 con 0x7fd6cc13df30 2024-04-14T08:38:43.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.330+0000 7fd6caffd640 1 --2- 172.21.15.172:0/3095063535 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd6b806c8f0 0x7fd6b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:43.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.330+0000 7fd6caffd640 1 --2- 172.21.15.172:0/3095063535 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd6b806c8f0 0x7fd6b806edb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc1f7860 tx=0x7fd6c0006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:43.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.335+0000 7fd6b3fff640 1 -- 172.21.15.172:0/3095063535 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd6bc041070 con 0x7fd6cc13df30 2024-04-14T08:38:43.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:43.511+0000 7fd6d21fe640 1 -- 172.21.15.172:0/3095063535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd6cc13e7d0 con 0x7fd6cc13df30 2024-04-14T08:38:44.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.205+0000 7fd6b3fff640 1 -- 172.21.15.172:0/3095063535 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7fd6bc07c4d0 con 0x7fd6cc13df30 2024-04-14T08:38:44.206 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' created 2024-04-14T08:38:44.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.208+0000 7fd6d21fe640 1 -- 172.21.15.172:0/3095063535 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd6b806c8f0 msgr2=0x7fd6b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:44.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.208+0000 7fd6d21fe640 1 --2- 172.21.15.172:0/3095063535 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd6b806c8f0 0x7fd6b806edb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc1f7860 tx=0x7fd6c0006bf0 comp rx=0 tx=0).stop 2024-04-14T08:38:44.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.208+0000 7fd6d21fe640 1 -- 172.21.15.172:0/3095063535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6cc13df30 msgr2=0x7fd6cc137ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.208+0000 7fd6d21fe640 1 --2- 172.21.15.172:0/3095063535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6cc13df30 0x7fd6cc137ff0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0098f0 tx=0x7fd6bc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.208+0000 7fd6d21fe640 1 -- 172.21.15.172:0/3095063535 shutdown_connections 2024-04-14T08:38:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.208+0000 7fd6d21fe640 1 --2- 172.21.15.172:0/3095063535 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd6b806c8f0 0x7fd6b806edb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.208+0000 7fd6d21fe640 1 --2- 172.21.15.172:0/3095063535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6cc15c380 0x7fd6cc150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.208+0000 7fd6d21fe640 1 --2- 172.21.15.172:0/3095063535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6cc140770 0x7fd6cc166ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.208+0000 7fd6d21fe640 1 --2- 172.21.15.172:0/3095063535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6cc13df30 0x7fd6cc137ff0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.208+0000 7fd6d21fe640 1 -- 172.21.15.172:0/3095063535 >> 172.21.15.172:0/3095063535 conn(0x7fd6cc06b520 msgr2=0x7fd6cc071940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.208+0000 7fd6d21fe640 1 -- 172.21.15.172:0/3095063535 shutdown_connections 2024-04-14T08:38:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.208+0000 7fd6d21fe640 1 -- 172.21.15.172:0/3095063535 wait complete. 2024-04-14T08:38:44.235 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-14T08:38:44.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.342+0000 7f37c962d640 1 Processor -- start 2024-04-14T08:38:44.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.342+0000 7f37c962d640 1 -- start start 2024-04-14T08:38:44.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c962d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f37c415e430 0x7f37c4162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:44.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c962d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c4162540 0x7f37c41629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:44.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c962d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37c40cefb0 con 0x7f37c40b3a30 2024-04-14T08:38:44.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c962d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f37c40cf290 con 0x7f37c415e430 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c962d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f37c40cf570 con 0x7f37c4162540 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c37fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f37c40b3a30 0x7f37c40b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45718/0 (socket says 172.21.15.172:45718) 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c37fe640 1 -- 172.21.15.172:0/1069766743 learned_addr learned my addr 172.21.15.172:0/1069766743 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c27fc640 1 --2- 172.21.15.172:0/1069766743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c4162540 0x7f37c41629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c2ffd640 1 --2- 172.21.15.172:0/1069766743 >> v2:172.21.15.177:3300/0 conn(0x7f37c415e430 0x7f37c4162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c1ffb640 1 -- 172.21.15.172:0/1069766743 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2079908915 0 0) 0x7f37c40cefb0 con 0x7f37c40b3a30 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c1ffb640 1 -- 172.21.15.172:0/1069766743 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37ac003520 con 0x7f37c40b3a30 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c1ffb640 1 -- 172.21.15.172:0/1069766743 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 789025281 0 0) 0x7f37ac003520 con 0x7f37c40b3a30 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c2ffd640 1 -- 172.21.15.172:0/1069766743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c4162540 msgr2=0x7f37c41629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c2ffd640 1 --2- 172.21.15.172:0/1069766743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c4162540 0x7f37c41629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c2ffd640 1 -- 172.21.15.172:0/1069766743 >> v1:172.21.15.172:6789/0 conn(0x7f37c40b3a30 legacy=0x7f37c40b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c2ffd640 1 -- 172.21.15.172:0/1069766743 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c40cf850 con 0x7f37c415e430 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.343+0000 7f37c27fc640 1 --2- 172.21.15.172:0/1069766743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c4162540 0x7f37c41629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:44.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.344+0000 7f37c2ffd640 1 --2- 172.21.15.172:0/1069766743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c415e430 0x7f37c4162000 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009d90 tx=0x7f37b402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=646d63bd01aa3ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:44.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.344+0000 7f37c1ffb640 1 -- 172.21.15.172:0/1069766743 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4039070 con 0x7f37c415e430 2024-04-14T08:38:44.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.344+0000 7f37c1ffb640 1 -- 172.21.15.172:0/1069766743 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b402cc60 con 0x7f37c415e430 2024-04-14T08:38:44.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.344+0000 7f37c1ffb640 1 -- 172.21.15.172:0/1069766743 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4034640 con 0x7f37c415e430 2024-04-14T08:38:44.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.345+0000 7f37c962d640 1 -- 172.21.15.172:0/1069766743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c415e430 msgr2=0x7f37c4162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:44.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.345+0000 7f37c962d640 1 --2- 172.21.15.172:0/1069766743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c415e430 0x7f37c4162000 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009d90 tx=0x7f37b402beb0 comp rx=0 tx=0).stop 2024-04-14T08:38:44.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.345+0000 7f37c962d640 1 -- 172.21.15.172:0/1069766743 shutdown_connections 2024-04-14T08:38:44.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.345+0000 7f37c962d640 1 --2- 172.21.15.172:0/1069766743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c4162540 0x7f37c41629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:44.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.345+0000 7f37c962d640 1 --2- 172.21.15.172:0/1069766743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c415e430 0x7f37c4162000 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:44.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.345+0000 7f37c962d640 1 -- 172.21.15.172:0/1069766743 >> 172.21.15.172:0/1069766743 conn(0x7f37c409d560 msgr2=0x7f37c40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:44.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.345+0000 7f37c962d640 1 -- 172.21.15.172:0/1069766743 shutdown_connections 2024-04-14T08:38:44.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.345+0000 7f37c962d640 1 -- 172.21.15.172:0/1069766743 wait complete. 2024-04-14T08:38:44.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.346+0000 7f37c962d640 1 Processor -- start 2024-04-14T08:38:44.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.346+0000 7f37c962d640 1 -- start start 2024-04-14T08:38:44.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c962d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b3a30 0x7f37c4159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:44.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c962d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37c415e430 0x7f37c40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:44.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c962d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c4162540 0x7f37c40b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:44.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c962d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37c40d4e20 con 0x7f37c415e430 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c962d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f37c415e810 con 0x7f37c4162540 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c962d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f37c40d0050 con 0x7f37c40b3a30 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37c415e430 0x7f37c40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37c415e430 0x7f37c40b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49706/0 (socket says 172.21.15.172:49706) 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c27fc640 1 -- 172.21.15.172:0/2065864431 learned_addr learned my addr 172.21.15.172:0/2065864431 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c37fe640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c4162540 0x7f37c40b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c2ffd640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b3a30 0x7f37c4159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c27fc640 1 -- 172.21.15.172:0/2065864431 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b3a30 msgr2=0x7f37c4159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c27fc640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b3a30 0x7f37c4159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c27fc640 1 -- 172.21.15.172:0/2065864431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c4162540 msgr2=0x7f37c40b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c27fc640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c4162540 0x7f37c40b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.347+0000 7f37c27fc640 1 -- 172.21.15.172:0/2065864431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c40cd0c0 con 0x7f37c415e430 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.348+0000 7f37c27fc640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37c415e430 0x7f37c40b4eb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f37b800ebe0 tx=0x7f37b800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:44.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.348+0000 7f37ab7fe640 1 -- 172.21.15.172:0/2065864431 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8017070 con 0x7f37c415e430 2024-04-14T08:38:44.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.348+0000 7f37ab7fe640 1 -- 172.21.15.172:0/2065864431 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b800cc30 con 0x7f37c415e430 2024-04-14T08:38:44.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.348+0000 7f37c962d640 1 -- 172.21.15.172:0/2065864431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c40cd3a0 con 0x7f37c415e430 2024-04-14T08:38:44.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.348+0000 7f37ab7fe640 1 -- 172.21.15.172:0/2065864431 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8013470 con 0x7f37c415e430 2024-04-14T08:38:44.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.348+0000 7f37c962d640 1 -- 172.21.15.172:0/2065864431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c40cd960 con 0x7f37c415e430 2024-04-14T08:38:44.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.349+0000 7f37c962d640 1 -- 172.21.15.172:0/2065864431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c4068590 con 0x7f37c415e430 2024-04-14T08:38:44.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.349+0000 7f37ab7fe640 1 -- 172.21.15.172:0/2065864431 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37b801a070 con 0x7f37c415e430 2024-04-14T08:38:44.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.350+0000 7f37ab7fe640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f37b006c8f0 0x7f37b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:44.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.350+0000 7f37ab7fe640 1 -- 172.21.15.172:0/2065864431 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6001+0+0 (secure 0 0 0) 0x7f37b808be60 con 0x7f37c415e430 2024-04-14T08:38:44.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.350+0000 7f37c2ffd640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f37b006c8f0 0x7f37b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:44.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.350+0000 7f37c2ffd640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f37b006c8f0 0x7f37b006edb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f37b4036000 tx=0x7f37b4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:44.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.354+0000 7f37ab7fe640 1 -- 172.21.15.172:0/2065864431 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f37b8059ba0 con 0x7f37c415e430 2024-04-14T08:38:44.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:44.570+0000 7f37c962d640 1 -- 172.21.15.172:0/2065864431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f37c40d18a0 con 0x7f37c415e430 2024-04-14T08:38:45.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.222+0000 7f37ab7fe640 1 -- 172.21.15.172:0/2065864431 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v52) ==== 174+0+0 (secure 0 0 0) 0x7f37b805dea0 con 0x7f37c415e430 2024-04-14T08:38:45.224 INFO:teuthology.orchestra.run.smithi172.stderr:new fs with metadata pool 13 and data pool 14 2024-04-14T08:38:45.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.229+0000 7f37c962d640 1 -- 172.21.15.172:0/2065864431 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f37b006c8f0 msgr2=0x7f37b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.229+0000 7f37c962d640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f37b006c8f0 0x7f37b006edb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f37b4036000 tx=0x7f37b4030000 comp rx=0 tx=0).stop 2024-04-14T08:38:45.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.229+0000 7f37c962d640 1 -- 172.21.15.172:0/2065864431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37c415e430 msgr2=0x7f37c40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.229+0000 7f37c962d640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37c415e430 0x7f37c40b4eb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f37b800ebe0 tx=0x7f37b800c520 comp rx=0 tx=0).stop 2024-04-14T08:38:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.230+0000 7f37c962d640 1 -- 172.21.15.172:0/2065864431 shutdown_connections 2024-04-14T08:38:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.230+0000 7f37c962d640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f37b006c8f0 0x7f37b006edb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.230+0000 7f37c962d640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c4162540 0x7f37c40b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.230+0000 7f37c962d640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37c415e430 0x7f37c40b4eb0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.230+0000 7f37c962d640 1 --2- 172.21.15.172:0/2065864431 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b3a30 0x7f37c4159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.231+0000 7f37c962d640 1 -- 172.21.15.172:0/2065864431 >> 172.21.15.172:0/2065864431 conn(0x7f37c409d560 msgr2=0x7f37c415c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.231+0000 7f37c962d640 1 -- 172.21.15.172:0/2065864431 shutdown_connections 2024-04-14T08:38:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.231+0000 7f37c962d640 1 -- 172.21.15.172:0/2065864431 wait complete. 2024-04-14T08:38:45.250 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:38:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.352+0000 7fb5ec636640 1 Processor -- start 2024-04-14T08:38:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.352+0000 7fb5ec636640 1 -- start start 2024-04-14T08:38:45.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5ec636640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb5e40b1df0 0x7fb5e40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5ec636640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e415c770 con 0x7fb5e40b4040 2024-04-14T08:38:45.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5ec636640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e40b3430 con 0x7fb5e40b21d0 2024-04-14T08:38:45.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5ec636640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e40b3700 con 0x7fb5e40b1df0 2024-04-14T08:38:45.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5eabd3640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb5e40b4040 0x7fb5e40b4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45728/0 (socket says 172.21.15.172:45728) 2024-04-14T08:38:45.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5eabd3640 1 -- 172.21.15.172:0/3514850520 learned_addr learned my addr 172.21.15.172:0/3514850520 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3917260294 0 0) 0x7fb5e415c770 con 0x7fb5e40b4040 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5ea3d2640 1 --2- 172.21.15.172:0/3514850520 >> v2:172.21.15.177:3300/0 conn(0x7fb5e40b1df0 0x7fb5e40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5d0003520 con 0x7fb5e40b4040 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1605875499 0 0) 0x7fb5d0003520 con 0x7fb5e40b4040 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 >> v2:172.21.15.177:3300/0 conn(0x7fb5e40b1df0 msgr2=0x7fb5e40b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 --2- 172.21.15.172:0/3514850520 >> v2:172.21.15.177:3300/0 conn(0x7fb5e40b1df0 0x7fb5e40b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 >> v1:172.21.15.177:6789/0 conn(0x7fb5e40b21d0 legacy=0x7fb5e415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e40b39e0 con 0x7fb5e40b4040 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fb5dc0019f0 con 0x7fb5e40b4040 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 >> v1:172.21.15.172:6789/0 conn(0x7fb5e40b4040 legacy=0x7fb5e40b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 --2- 172.21.15.172:0/3514850520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5d00039b0 0x7fb5d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 --2- 172.21.15.172:0/3514850520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5d0004800 0x7fb5d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 --2- 172.21.15.172:0/3514850520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 0x7fb5d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e40b3700 con 0x7fb5d00051c0 2024-04-14T08:38:45.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d000c060 con 0x7fb5d0004800 2024-04-14T08:38:45.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.353+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d000c260 con 0x7fb5d00039b0 2024-04-14T08:38:45.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.354+0000 7fb5eabd3640 1 --2- 172.21.15.172:0/3514850520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 0x7fb5d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.354+0000 7fb5ea3d2640 1 --2- 172.21.15.172:0/3514850520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5d00039b0 0x7fb5d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.354+0000 7fb5e9bd1640 1 --2- 172.21.15.172:0/3514850520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5d0004800 0x7fb5d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.354+0000 7fb5eabd3640 1 -- 172.21.15.172:0/3514850520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5d00039b0 msgr2=0x7fb5d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:45.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.354+0000 7fb5eabd3640 1 --2- 172.21.15.172:0/3514850520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5d00039b0 0x7fb5d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.354+0000 7fb5eabd3640 1 -- 172.21.15.172:0/3514850520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5d0004800 msgr2=0x7fb5d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:45.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.354+0000 7fb5eabd3640 1 --2- 172.21.15.172:0/3514850520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5d0004800 0x7fb5d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.354+0000 7fb5eabd3640 1 -- 172.21.15.172:0/3514850520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d0003520 con 0x7fb5d00051c0 2024-04-14T08:38:45.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.354+0000 7fb5eabd3640 1 -- 172.21.15.172:0/3514850520 reap_dead start 2024-04-14T08:38:45.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.355+0000 7fb5eabd3640 1 --2- 172.21.15.172:0/3514850520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 0x7fb5d000b9c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00a860 tx=0x7fb5d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9df627f8a20ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:45.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.355+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00a480 con 0x7fb5d00051c0 2024-04-14T08:38:45.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.355+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5dc034630 con 0x7fb5d00051c0 2024-04-14T08:38:45.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.355+0000 7fb5e93d0640 1 -- 172.21.15.172:0/3514850520 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc034a20 con 0x7fb5d00051c0 2024-04-14T08:38:45.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.355+0000 7fb5ec636640 1 -- 172.21.15.172:0/3514850520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 msgr2=0x7fb5d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.355+0000 7fb5ec636640 1 --2- 172.21.15.172:0/3514850520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 0x7fb5d000b9c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00a860 tx=0x7fb5d0005830 comp rx=0 tx=0).stop 2024-04-14T08:38:45.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.355+0000 7fb5ec636640 1 -- 172.21.15.172:0/3514850520 shutdown_connections 2024-04-14T08:38:45.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.355+0000 7fb5ec636640 1 --2- 172.21.15.172:0/3514850520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 0x7fb5d000b9c0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.355+0000 7fb5ec636640 1 -- 172.21.15.172:0/3514850520 >> 172.21.15.172:0/3514850520 conn(0x7fb5e406ec30 msgr2=0x7fb5e406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:45.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.355+0000 7fb5ec636640 1 -- 172.21.15.172:0/3514850520 shutdown_connections 2024-04-14T08:38:45.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.355+0000 7fb5ec636640 1 -- 172.21.15.172:0/3514850520 wait complete. 2024-04-14T08:38:45.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.357+0000 7fb5ec636640 1 Processor -- start 2024-04-14T08:38:45.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.357+0000 7fb5ec636640 1 -- start start 2024-04-14T08:38:45.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.357+0000 7fb5ec636640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 0x7fb5e4157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.357+0000 7fb5ec636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e413b750 0x7fb5e413bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.357+0000 7fb5ec636640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5e40ce1a0 0x7fb5e40ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.357+0000 7fb5ec636640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e415b7f0 con 0x7fb5d00051c0 2024-04-14T08:38:45.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.357+0000 7fb5ec636640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e415bb40 con 0x7fb5e40ce1a0 2024-04-14T08:38:45.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.357+0000 7fb5ec636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e40b2ef0 con 0x7fb5e413b750 2024-04-14T08:38:45.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.357+0000 7fb5ea3d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 0x7fb5e4157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.357+0000 7fb5ea3d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 0x7fb5e4157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49716/0 (socket says 172.21.15.172:49716) 2024-04-14T08:38:45.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.357+0000 7fb5ea3d2640 1 -- 172.21.15.172:0/784183975 learned_addr learned my addr 172.21.15.172:0/784183975 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:45.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5eabd3640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5e40ce1a0 0x7fb5e40ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5e9bd1640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e413b750 0x7fb5e413bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5ea3d2640 1 -- 172.21.15.172:0/784183975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e413b750 msgr2=0x7fb5e413bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5ea3d2640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e413b750 0x7fb5e413bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5ea3d2640 1 -- 172.21.15.172:0/784183975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5e40ce1a0 msgr2=0x7fb5e40ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5ea3d2640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5e40ce1a0 0x7fb5e40ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5ea3d2640 1 -- 172.21.15.172:0/784183975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e413a6b0 con 0x7fb5d00051c0 2024-04-14T08:38:45.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5ea3d2640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 0x7fb5e4157ad0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4011940 tx=0x7fb5d4011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:45.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5daffd640 1 -- 172.21.15.172:0/784183975 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d400fa90 con 0x7fb5d00051c0 2024-04-14T08:38:45.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5daffd640 1 -- 172.21.15.172:0/784183975 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d400fc30 con 0x7fb5d00051c0 2024-04-14T08:38:45.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5ec636640 1 -- 172.21.15.172:0/784183975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e413a990 con 0x7fb5d00051c0 2024-04-14T08:38:45.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5daffd640 1 -- 172.21.15.172:0/784183975 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4014440 con 0x7fb5d00051c0 2024-04-14T08:38:45.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.358+0000 7fb5ec636640 1 -- 172.21.15.172:0/784183975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e413af50 con 0x7fb5d00051c0 2024-04-14T08:38:45.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.359+0000 7fb5ec636640 1 -- 172.21.15.172:0/784183975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e4068520 con 0x7fb5d00051c0 2024-04-14T08:38:45.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.359+0000 7fb5daffd640 1 -- 172.21.15.172:0/784183975 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5d4012040 con 0x7fb5d00051c0 2024-04-14T08:38:45.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.360+0000 7fb5daffd640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb5c806c920 0x7fb5c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.360+0000 7fb5e9bd1640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb5c806c920 0x7fb5c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.360+0000 7fb5daffd640 1 -- 172.21.15.172:0/784183975 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6101+0+0 (secure 0 0 0) 0x7fb5d408c260 con 0x7fb5d00051c0 2024-04-14T08:38:45.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.360+0000 7fb5e9bd1640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb5c806c920 0x7fb5c806ede0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0000c00 tx=0x7fb5e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:45.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.364+0000 7fb5daffd640 1 -- 172.21.15.172:0/784183975 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb5d4019030 con 0x7fb5d00051c0 2024-04-14T08:38:45.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.540+0000 7fb5ec636640 1 -- 172.21.15.172:0/784183975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5e4158c40 con 0x7fb5d00051c0 2024-04-14T08:38:45.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.541+0000 7fb5daffd640 1 -- 172.21.15.172:0/784183975 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16958 (secure 0 0 0) 0x7fb5d405a590 con 0x7fb5d00051c0 2024-04-14T08:38:45.543 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:45.543 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":85,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:38:44.572093+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:38:42.790996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-14T08:38:43.517208+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:38:45.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.543+0000 7fb5ec636640 1 -- 172.21.15.172:0/784183975 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb5c806c920 msgr2=0x7fb5c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.543+0000 7fb5ec636640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb5c806c920 0x7fb5c806ede0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0000c00 tx=0x7fb5e0009040 comp rx=0 tx=0).stop 2024-04-14T08:38:45.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.543+0000 7fb5ec636640 1 -- 172.21.15.172:0/784183975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 msgr2=0x7fb5e4157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.543+0000 7fb5ec636640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 0x7fb5e4157ad0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4011940 tx=0x7fb5d4011e00 comp rx=0 tx=0).stop 2024-04-14T08:38:45.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.543+0000 7fb5ec636640 1 -- 172.21.15.172:0/784183975 shutdown_connections 2024-04-14T08:38:45.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.543+0000 7fb5ec636640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb5c806c920 0x7fb5c806ede0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.543+0000 7fb5ec636640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5e40ce1a0 0x7fb5e40ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.543+0000 7fb5ec636640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5e413b750 0x7fb5e413bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.543+0000 7fb5ec636640 1 --2- 172.21.15.172:0/784183975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d00051c0 0x7fb5e4157ad0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.543+0000 7fb5ec636640 1 -- 172.21.15.172:0/784183975 >> 172.21.15.172:0/784183975 conn(0x7fb5e406ec30 msgr2=0x7fb5e415d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:45.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.543+0000 7fb5ec636640 1 -- 172.21.15.172:0/784183975 shutdown_connections 2024-04-14T08:38:45.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.543+0000 7fb5ec636640 1 -- 172.21.15.172:0/784183975 wait complete. 2024-04-14T08:38:45.556 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:38:45.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.656+0000 7f139a508640 1 Processor -- start 2024-04-14T08:38:45.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.656+0000 7f139a508640 1 -- start start 2024-04-14T08:38:45.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.656+0000 7f139a508640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f13940d2830 0x7f13940d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.656+0000 7f139a508640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13940d3150 con 0x7f139415e890 2024-04-14T08:38:45.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.656+0000 7f139a508640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13940cf320 con 0x7f13940cedd0 2024-04-14T08:38:45.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.656+0000 7f139a508640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f13940cff30 con 0x7f13940d2830 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f13937fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f139415e890 0x7f13940d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45730/0 (socket says 172.21.15.172:45730) 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f13937fe640 1 -- 172.21.15.172:0/1756818376 learned_addr learned my addr 172.21.15.172:0/1756818376 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1393fff640 1 --2- 172.21.15.172:0/1756818376 >> v2:172.21.15.177:3300/0 conn(0x7f13940d2830 0x7f13940d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3732720653 0 0) 0x7f13940d3150 con 0x7f139415e890 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1380003520 con 0x7f139415e890 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 961654407 0 0) 0x7f1380003520 con 0x7f139415e890 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 >> v2:172.21.15.177:3300/0 conn(0x7f13940d2830 msgr2=0x7f13940d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 --2- 172.21.15.172:0/1756818376 >> v2:172.21.15.177:3300/0 conn(0x7f13940d2830 0x7f13940d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 >> v1:172.21.15.177:6789/0 conn(0x7f13940cedd0 legacy=0x7f13940b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13940cf600 con 0x7f139415e890 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f1384001670 con 0x7f139415e890 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1393fff640 1 --2- 172.21.15.172:0/1756818376 >> v2:172.21.15.177:3300/0 conn(0x7f13940d2830 0x7f13940d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:45.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 >> v1:172.21.15.172:6789/0 conn(0x7f139415e890 legacy=0x7f13940d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 --2- 172.21.15.172:0/1756818376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13800039b0 0x7f1380003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 --2- 172.21.15.172:0/1756818376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1380004800 0x7f1380004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 --2- 172.21.15.172:0/1756818376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13800051c0 0x7f138000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13940cff30 con 0x7f1380004800 2024-04-14T08:38:45.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f138000c070 con 0x7f13800039b0 2024-04-14T08:38:45.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f138000c270 con 0x7f13800051c0 2024-04-14T08:38:45.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f13937fe640 1 --2- 172.21.15.172:0/1756818376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1380004800 0x7f1380004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1393fff640 1 --2- 172.21.15.172:0/1756818376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13800039b0 0x7f1380003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.657+0000 7f1398aa5640 1 --2- 172.21.15.172:0/1756818376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13800051c0 0x7f138000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.658+0000 7f13937fe640 1 -- 172.21.15.172:0/1756818376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13800051c0 msgr2=0x7f138000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:45.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.658+0000 7f13937fe640 1 --2- 172.21.15.172:0/1756818376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f13800051c0 0x7f138000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.658+0000 7f13937fe640 1 -- 172.21.15.172:0/1756818376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13800039b0 msgr2=0x7f1380003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:45.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.658+0000 7f13937fe640 1 --2- 172.21.15.172:0/1756818376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13800039b0 0x7f1380003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.658+0000 7f1398aa5640 1 -- 172.21.15.172:0/1756818376 reap_dead start 2024-04-14T08:38:45.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.658+0000 7f13937fe640 1 -- 172.21.15.172:0/1756818376 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1380003520 con 0x7f1380004800 2024-04-14T08:38:45.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.658+0000 7f13937fe640 1 --2- 172.21.15.172:0/1756818376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1380004800 0x7f1380004c80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f138400c210 tx=0x7f1380005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa6f09ae162a3dc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:45.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.658+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384007d30 con 0x7f1380004800 2024-04-14T08:38:45.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.658+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1384030070 con 0x7f1380004800 2024-04-14T08:38:45.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.658+0000 7f1392ffd640 1 -- 172.21.15.172:0/1756818376 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384005d70 con 0x7f1380004800 2024-04-14T08:38:45.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.659+0000 7f139a508640 1 -- 172.21.15.172:0/1756818376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1380004800 msgr2=0x7f1380004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.659+0000 7f139a508640 1 --2- 172.21.15.172:0/1756818376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1380004800 0x7f1380004c80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f138400c210 tx=0x7f1380005830 comp rx=0 tx=0).stop 2024-04-14T08:38:45.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.659+0000 7f139a508640 1 -- 172.21.15.172:0/1756818376 shutdown_connections 2024-04-14T08:38:45.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.659+0000 7f139a508640 1 --2- 172.21.15.172:0/1756818376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1380004800 0x7f1380004c80 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.659+0000 7f139a508640 1 -- 172.21.15.172:0/1756818376 >> 172.21.15.172:0/1756818376 conn(0x7f139409d590 msgr2=0x7f13940d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:45.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.659+0000 7f139a508640 1 -- 172.21.15.172:0/1756818376 shutdown_connections 2024-04-14T08:38:45.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.659+0000 7f139a508640 1 -- 172.21.15.172:0/1756818376 wait complete. 2024-04-14T08:38:45.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.660+0000 7f139a508640 1 Processor -- start 2024-04-14T08:38:45.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.660+0000 7f139a508640 1 -- start start 2024-04-14T08:38:45.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f139a508640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1380004800 0x7f13940c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f139a508640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13940b62a0 0x7f13940b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f139a508640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13941544a0 0x7f13940ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f139a508640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13940cfbc0 con 0x7f13940b62a0 2024-04-14T08:38:45.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f139a508640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f13940d2350 con 0x7f13941544a0 2024-04-14T08:38:45.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f139a508640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f13940d2c10 con 0x7f1380004800 2024-04-14T08:38:45.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f13937fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13940b62a0 0x7f13940b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f13937fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13940b62a0 0x7f13940b7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49736/0 (socket says 172.21.15.172:49736) 2024-04-14T08:38:45.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f13937fe640 1 -- 172.21.15.172:0/3273676441 learned_addr learned my addr 172.21.15.172:0/3273676441 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:45.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f1393fff640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1380004800 0x7f13940c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f1398aa5640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13941544a0 0x7f13940ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f13937fe640 1 -- 172.21.15.172:0/3273676441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1380004800 msgr2=0x7f13940c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f13937fe640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1380004800 0x7f13940c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f13937fe640 1 -- 172.21.15.172:0/3273676441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13941544a0 msgr2=0x7f13940ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f13937fe640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13941544a0 0x7f13940ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f13937fe640 1 -- 172.21.15.172:0/3273676441 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13940c8a00 con 0x7f13940b62a0 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.661+0000 7f1398aa5640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13941544a0 0x7f13940ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.662+0000 7f13937fe640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13940b62a0 0x7f13940b7880 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f138400c210 tx=0x7f138402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.662+0000 7f1390ff9640 1 -- 172.21.15.172:0/3273676441 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384046070 con 0x7f13940b62a0 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.662+0000 7f1390ff9640 1 -- 172.21.15.172:0/3273676441 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1384004bf0 con 0x7f13940b62a0 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.662+0000 7f139a508640 1 -- 172.21.15.172:0/3273676441 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13940b7dc0 con 0x7f13940b62a0 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.662+0000 7f1390ff9640 1 -- 172.21.15.172:0/3273676441 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384007d10 con 0x7f13940b62a0 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.662+0000 7f139a508640 1 -- 172.21.15.172:0/3273676441 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13940c8f80 con 0x7f13940b62a0 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.663+0000 7f139a508640 1 -- 172.21.15.172:0/3273676441 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1394068520 con 0x7f13940b62a0 2024-04-14T08:38:45.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.663+0000 7f1390ff9640 1 -- 172.21.15.172:0/3273676441 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1384007070 con 0x7f13940b62a0 2024-04-14T08:38:45.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.664+0000 7f1390ff9640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f137406c920 0x7f137406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.664+0000 7f1390ff9640 1 -- 172.21.15.172:0/3273676441 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6101+0+0 (secure 0 0 0) 0x7f13840ad290 con 0x7f13940b62a0 2024-04-14T08:38:45.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.664+0000 7f1393fff640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f137406c920 0x7f137406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.664+0000 7f1393fff640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f137406c920 0x7f137406ede0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f137c009c90 tx=0x7f137c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:45.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.668+0000 7f1390ff9640 1 -- 172.21.15.172:0/3273676441 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f138407ae50 con 0x7f13940b62a0 2024-04-14T08:38:45.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.844+0000 7f139a508640 1 -- 172.21.15.172:0/3273676441 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13940d2610 con 0x7f13940b62a0 2024-04-14T08:38:45.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.846+0000 7f1390ff9640 1 -- 172.21.15.172:0/3273676441 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16958 (secure 0 0 0) 0x7f138407f150 con 0x7f13940b62a0 2024-04-14T08:38:45.848 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:45.848 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":85,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:38:44.572093+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:38:42.790996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-14T08:38:43.517208+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:38:45.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.848+0000 7f139a508640 1 -- 172.21.15.172:0/3273676441 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f137406c920 msgr2=0x7f137406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.848+0000 7f139a508640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f137406c920 0x7f137406ede0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f137c009c90 tx=0x7f137c008040 comp rx=0 tx=0).stop 2024-04-14T08:38:45.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.848+0000 7f139a508640 1 -- 172.21.15.172:0/3273676441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13940b62a0 msgr2=0x7f13940b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.848+0000 7f139a508640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13940b62a0 0x7f13940b7880 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f138400c210 tx=0x7f138402e070 comp rx=0 tx=0).stop 2024-04-14T08:38:45.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.848+0000 7f139a508640 1 -- 172.21.15.172:0/3273676441 shutdown_connections 2024-04-14T08:38:45.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.848+0000 7f139a508640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f137406c920 0x7f137406ede0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.848+0000 7f139a508640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13941544a0 0x7f13940ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.848+0000 7f139a508640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13940b62a0 0x7f13940b7880 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.848+0000 7f139a508640 1 --2- 172.21.15.172:0/3273676441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1380004800 0x7f13940c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.848+0000 7f139a508640 1 -- 172.21.15.172:0/3273676441 >> 172.21.15.172:0/3273676441 conn(0x7f139409d590 msgr2=0x7f13940d3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:45.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.848+0000 7f139a508640 1 -- 172.21.15.172:0/3273676441 shutdown_connections 2024-04-14T08:38:45.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.848+0000 7f139a508640 1 -- 172.21.15.172:0/3273676441 wait complete. 2024-04-14T08:38:45.862 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-14T08:38:45.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.967+0000 7fe249036640 1 Processor -- start 2024-04-14T08:38:45.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.967+0000 7fe249036640 1 -- start start 2024-04-14T08:38:45.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe249036640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe244162a80 0x7fe2440b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe249036640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe244161f10 0x7fe2440b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe249036640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe24406f810 0x7fe24406e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe249036640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe24415f6d0 con 0x7fe244161f10 2024-04-14T08:38:45.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe249036640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe244162e60 con 0x7fe244162a80 2024-04-14T08:38:45.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe249036640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe24415eac0 con 0x7fe24406f810 2024-04-14T08:38:45.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe23bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe244161f10 0x7fe2440b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe23bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe244161f10 0x7fe2440b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49746/0 (socket says 172.21.15.172:49746) 2024-04-14T08:38:45.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe23bfff640 1 -- 172.21.15.172:0/4227235199 learned_addr learned my addr 172.21.15.172:0/4227235199 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:45.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe242d9d640 1 --2- 172.21.15.172:0/4227235199 >> v2:172.21.15.177:3300/0 conn(0x7fe244162a80 0x7fe2440b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe24359e640 1 --2- 172.21.15.172:0/4227235199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe24406f810 0x7fe24406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe23bfff640 1 -- 172.21.15.172:0/4227235199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe24406f810 msgr2=0x7fe24406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:45.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe23bfff640 1 --2- 172.21.15.172:0/4227235199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe24406f810 0x7fe24406e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe23bfff640 1 -- 172.21.15.172:0/4227235199 >> v2:172.21.15.177:3300/0 conn(0x7fe244162a80 msgr2=0x7fe2440b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:45.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe23bfff640 1 --2- 172.21.15.172:0/4227235199 >> v2:172.21.15.177:3300/0 conn(0x7fe244162a80 0x7fe2440b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.968+0000 7fe23bfff640 1 -- 172.21.15.172:0/4227235199 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24415eda0 con 0x7fe244161f10 2024-04-14T08:38:45.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.969+0000 7fe23bfff640 1 --2- 172.21.15.172:0/4227235199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe244161f10 0x7fe2440b3e80 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe2440b3910 tx=0x7fe22c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cd0d8a0f375835 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.969+0000 7fe24259c640 1 -- 172.21.15.172:0/4227235199 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c035020 con 0x7fe244161f10 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.969+0000 7fe24259c640 1 -- 172.21.15.172:0/4227235199 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe22c031870 con 0x7fe244161f10 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.969+0000 7fe24259c640 1 -- 172.21.15.172:0/4227235199 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c031c40 con 0x7fe244161f10 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.970+0000 7fe249036640 1 -- 172.21.15.172:0/4227235199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe244161f10 msgr2=0x7fe2440b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.970+0000 7fe249036640 1 --2- 172.21.15.172:0/4227235199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe244161f10 0x7fe2440b3e80 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe2440b3910 tx=0x7fe22c02b910 comp rx=0 tx=0).stop 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.970+0000 7fe249036640 1 -- 172.21.15.172:0/4227235199 shutdown_connections 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.970+0000 7fe249036640 1 --2- 172.21.15.172:0/4227235199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe24406f810 0x7fe24406e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.970+0000 7fe249036640 1 --2- 172.21.15.172:0/4227235199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe244161f10 0x7fe2440b3e80 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.970+0000 7fe249036640 1 --2- 172.21.15.172:0/4227235199 >> v2:172.21.15.177:3300/0 conn(0x7fe244162a80 0x7fe2440b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.970+0000 7fe249036640 1 -- 172.21.15.172:0/4227235199 >> 172.21.15.172:0/4227235199 conn(0x7fe2440b29b0 msgr2=0x7fe2440d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.970+0000 7fe249036640 1 -- 172.21.15.172:0/4227235199 shutdown_connections 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.970+0000 7fe249036640 1 -- 172.21.15.172:0/4227235199 wait complete. 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.971+0000 7fe249036640 1 Processor -- start 2024-04-14T08:38:45.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.971+0000 7fe249036640 1 -- start start 2024-04-14T08:38:45.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe249036640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe24406f810 0x7fe244156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe249036640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe244161f10 0x7fe244151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe249036640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe244162a80 0x7fe24414d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe249036640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe244067310 con 0x7fe244161f10 2024-04-14T08:38:45.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe249036640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe24415f360 con 0x7fe244162a80 2024-04-14T08:38:45.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe249036640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe24406e6a0 con 0x7fe24406f810 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe23bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe244161f10 0x7fe244151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe23bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe244161f10 0x7fe244151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49748/0 (socket says 172.21.15.172:49748) 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe23bfff640 1 -- 172.21.15.172:0/3247336976 learned_addr learned my addr 172.21.15.172:0/3247336976 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe242d9d640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe24406f810 0x7fe244156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe23bfff640 1 -- 172.21.15.172:0/3247336976 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe24406f810 msgr2=0x7fe244156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe23bfff640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe24406f810 0x7fe244156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe23bfff640 1 -- 172.21.15.172:0/3247336976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe244162a80 msgr2=0x7fe24414d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe23bfff640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe244162a80 0x7fe24414d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe23bfff640 1 -- 172.21.15.172:0/3247336976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe244157290 con 0x7fe244161f10 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.972+0000 7fe23bfff640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe244161f10 0x7fe244151b20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fe22c02b950 tx=0x7fe22c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.973+0000 7fe23b7fe640 1 -- 172.21.15.172:0/3247336976 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c040070 con 0x7fe244161f10 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.973+0000 7fe23b7fe640 1 -- 172.21.15.172:0/3247336976 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe22c030d00 con 0x7fe244161f10 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.973+0000 7fe249036640 1 -- 172.21.15.172:0/3247336976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe244157570 con 0x7fe244161f10 2024-04-14T08:38:45.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.973+0000 7fe23b7fe640 1 -- 172.21.15.172:0/3247336976 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c035040 con 0x7fe244161f10 2024-04-14T08:38:45.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.973+0000 7fe249036640 1 -- 172.21.15.172:0/3247336976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe244157b30 con 0x7fe244161f10 2024-04-14T08:38:45.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.974+0000 7fe249036640 1 -- 172.21.15.172:0/3247336976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe244068520 con 0x7fe244161f10 2024-04-14T08:38:45.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.974+0000 7fe23b7fe640 1 -- 172.21.15.172:0/3247336976 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe22c04b060 con 0x7fe244161f10 2024-04-14T08:38:45.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.974+0000 7fe23b7fe640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe22406c8f0 0x7fe22406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:45.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.975+0000 7fe23b7fe640 1 -- 172.21.15.172:0/3247336976 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6101+0+0 (secure 0 0 0) 0x7fe22c0ada60 con 0x7fe244161f10 2024-04-14T08:38:45.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.975+0000 7fe242d9d640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe22406c8f0 0x7fe22406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:45.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.975+0000 7fe242d9d640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe22406c8f0 0x7fe22406edb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe234006fd0 tx=0x7fe234008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:45.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:45.979+0000 7fe23b7fe640 1 -- 172.21.15.172:0/3247336976 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe22c07bd00 con 0x7fe244161f10 2024-04-14T08:38:46.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.189+0000 7fe249036640 1 -- 172.21.15.172:0/3247336976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe2440d1290 con 0x7fe244161f10 2024-04-14T08:38:46.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.227+0000 7fe23b7fe640 1 -- 172.21.15.172:0/3247336976 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) ==== 118+0+0 (secure 0 0 0) 0x7fe22c07b600 con 0x7fe244161f10 2024-04-14T08:38:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.229+0000 7fe249036640 1 -- 172.21.15.172:0/3247336976 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe22406c8f0 msgr2=0x7fe22406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.229+0000 7fe249036640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe22406c8f0 0x7fe22406edb0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe234006fd0 tx=0x7fe234008040 comp rx=0 tx=0).stop 2024-04-14T08:38:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.229+0000 7fe249036640 1 -- 172.21.15.172:0/3247336976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe244161f10 msgr2=0x7fe244151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.229+0000 7fe249036640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe244161f10 0x7fe244151b20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fe22c02b950 tx=0x7fe22c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.229+0000 7fe249036640 1 -- 172.21.15.172:0/3247336976 shutdown_connections 2024-04-14T08:38:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.229+0000 7fe249036640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe22406c8f0 0x7fe22406edb0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.229+0000 7fe249036640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe244162a80 0x7fe24414d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.229+0000 7fe249036640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe244161f10 0x7fe244151b20 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.229+0000 7fe249036640 1 --2- 172.21.15.172:0/3247336976 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe24406f810 0x7fe244156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.229+0000 7fe249036640 1 -- 172.21.15.172:0/3247336976 >> 172.21.15.172:0/3247336976 conn(0x7fe2440b29b0 msgr2=0x7fe24406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:46.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.229+0000 7fe249036640 1 -- 172.21.15.172:0/3247336976 shutdown_connections 2024-04-14T08:38:46.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.229+0000 7fe249036640 1 -- 172.21.15.172:0/3247336976 wait complete. 2024-04-14T08:38:46.241 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-14T08:38:46.241 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-14T08:38:46.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.344+0000 7f992398c640 1 Processor -- start 2024-04-14T08:38:46.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.344+0000 7f992398c640 1 -- start start 2024-04-14T08:38:46.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f992398c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f991c0cca00 0x7f991c072ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:46.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f992398c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f991c0ceb40 0x7f991c06e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:46.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f992398c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991c072430 con 0x7f991c0b0070 2024-04-14T08:38:46.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f992398c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f991c073410 con 0x7f991c0ceb40 2024-04-14T08:38:46.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f992398c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f991c072a40 con 0x7f991c0cca00 2024-04-14T08:38:46.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f9921f29640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f991c0ceb40 0x7f991c06e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:46.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f9921f29640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f991c0ceb40 0x7f991c06e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49762/0 (socket says 172.21.15.172:49762) 2024-04-14T08:38:46.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f9921f29640 1 -- 172.21.15.172:0/3447641651 learned_addr learned my addr 172.21.15.172:0/3447641651 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:46.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f9921728640 1 --2- 172.21.15.172:0/3447641651 >> v2:172.21.15.177:3300/0 conn(0x7f991c0cca00 0x7f991c072ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:46.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f9921f29640 1 -- 172.21.15.172:0/3447641651 >> v2:172.21.15.177:3300/0 conn(0x7f991c0cca00 msgr2=0x7f991c072ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:46.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f9921f29640 1 --2- 172.21.15.172:0/3447641651 >> v2:172.21.15.177:3300/0 conn(0x7f991c0cca00 0x7f991c072ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:46.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f9921f29640 1 -- 172.21.15.172:0/3447641651 >> v1:172.21.15.177:6789/0 conn(0x7f991c0b0070 legacy=0x7f991c06b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:46.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.345+0000 7f9921f29640 1 -- 172.21.15.172:0/3447641651 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c0b0500 con 0x7f991c0ceb40 2024-04-14T08:38:46.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.346+0000 7f9921f29640 1 --2- 172.21.15.172:0/3447641651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0ceb40 0x7f991c06e110 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f991c075340 tx=0x7f991402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7892b49decd6920 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:46.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.346+0000 7f9913fff640 1 -- 172.21.15.172:0/3447641651 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914034070 con 0x7f991c0ceb40 2024-04-14T08:38:46.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.346+0000 7f9913fff640 1 -- 172.21.15.172:0/3447641651 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9914030860 con 0x7f991c0ceb40 2024-04-14T08:38:46.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.346+0000 7f9913fff640 1 -- 172.21.15.172:0/3447641651 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914030c30 con 0x7f991c0ceb40 2024-04-14T08:38:46.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.347+0000 7f992398c640 1 -- 172.21.15.172:0/3447641651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0ceb40 msgr2=0x7f991c06e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:46.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.347+0000 7f992398c640 1 --2- 172.21.15.172:0/3447641651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0ceb40 0x7f991c06e110 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f991c075340 tx=0x7f991402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:38:46.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.347+0000 7f992398c640 1 -- 172.21.15.172:0/3447641651 shutdown_connections 2024-04-14T08:38:46.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.347+0000 7f992398c640 1 --2- 172.21.15.172:0/3447641651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0ceb40 0x7f991c06e110 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:46.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.347+0000 7f992398c640 1 --2- 172.21.15.172:0/3447641651 >> v2:172.21.15.177:3300/0 conn(0x7f991c0cca00 0x7f991c072ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:46.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.347+0000 7f992398c640 1 -- 172.21.15.172:0/3447641651 >> 172.21.15.172:0/3447641651 conn(0x7f991c0cd980 msgr2=0x7f991c0cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:46.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.347+0000 7f992398c640 1 -- 172.21.15.172:0/3447641651 shutdown_connections 2024-04-14T08:38:46.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.347+0000 7f992398c640 1 -- 172.21.15.172:0/3447641651 wait complete. 2024-04-14T08:38:46.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.348+0000 7f992398c640 1 Processor -- start 2024-04-14T08:38:46.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.348+0000 7f992398c640 1 -- start start 2024-04-14T08:38:46.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f992398c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cca00 0x7f991c1e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:46.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f992398c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f991c0ceb40 0x7f991c1e9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:46.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f992398c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f991c1e9e50 0x7f991c1e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f992398c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f991c0ccde0 con 0x7f991c0cca00 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f992398c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f991c0b07e0 con 0x7f991c0ceb40 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f992398c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f991c0ccf60 con 0x7f991c1e9e50 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f9921728640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cca00 0x7f991c1e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f9921728640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cca00 0x7f991c1e6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49774/0 (socket says 172.21.15.172:49774) 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f9921728640 1 -- 172.21.15.172:0/1061823325 learned_addr learned my addr 172.21.15.172:0/1061823325 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f9920f27640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f991c0ceb40 0x7f991c1e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f9921f29640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f991c1e9e50 0x7f991c1e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f9921728640 1 -- 172.21.15.172:0/1061823325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f991c1e9e50 msgr2=0x7f991c1e74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f9921728640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f991c1e9e50 0x7f991c1e74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f9921728640 1 -- 172.21.15.172:0/1061823325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f991c0ceb40 msgr2=0x7f991c1e9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f9921728640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f991c0ceb40 0x7f991c1e9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:46.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f9921728640 1 -- 172.21.15.172:0/1061823325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c1e7120 con 0x7f991c0cca00 2024-04-14T08:38:46.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f9921728640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cca00 0x7f991c1e6be0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f991800efc0 tx=0x7f991800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:46.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.349+0000 7f9911ffb640 1 -- 172.21.15.172:0/1061823325 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918018020 con 0x7f991c0cca00 2024-04-14T08:38:46.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.350+0000 7f9911ffb640 1 -- 172.21.15.172:0/1061823325 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9918014480 con 0x7f991c0cca00 2024-04-14T08:38:46.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.350+0000 7f9911ffb640 1 -- 172.21.15.172:0/1061823325 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918014850 con 0x7f991c0cca00 2024-04-14T08:38:46.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.350+0000 7f992398c640 1 -- 172.21.15.172:0/1061823325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991c1e8220 con 0x7f991c0cca00 2024-04-14T08:38:46.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.350+0000 7f992398c640 1 -- 172.21.15.172:0/1061823325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991c1e87e0 con 0x7f991c0cca00 2024-04-14T08:38:46.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.351+0000 7f992398c640 1 -- 172.21.15.172:0/1061823325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991c1ea260 con 0x7f991c0cca00 2024-04-14T08:38:46.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.351+0000 7f9911ffb640 1 -- 172.21.15.172:0/1061823325 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9918005d10 con 0x7f991c0cca00 2024-04-14T08:38:46.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.352+0000 7f9911ffb640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f990006c8f0 0x7f990006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:46.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.352+0000 7f9920f27640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f990006c8f0 0x7f990006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:46.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.352+0000 7f9911ffb640 1 -- 172.21.15.172:0/1061823325 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f991808c670 con 0x7f991c0cca00 2024-04-14T08:38:46.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.352+0000 7f9920f27640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f990006c8f0 0x7f990006edb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f991c0c9ce0 tx=0x7f990c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:46.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.356+0000 7f9911ffb640 1 -- 172.21.15.172:0/1061823325 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f991c1ea260 con 0x7f991c0cca00 2024-04-14T08:38:46.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:46.571+0000 7f992398c640 1 -- 172.21.15.172:0/1061823325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f991c068520 con 0x7f991c0cca00 2024-04-14T08:38:47.198 INFO:tasks.ceph.mds.d.smithi177.stderr:2024-04-14T08:38:47.198+0000 7f30dddf1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:38:47.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.231+0000 7f9911ffb640 1 -- 172.21.15.172:0/1061823325 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7f9918022070 con 0x7f991c0cca00 2024-04-14T08:38:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.234+0000 7f992398c640 1 -- 172.21.15.172:0/1061823325 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f990006c8f0 msgr2=0x7f990006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.234+0000 7f992398c640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f990006c8f0 0x7f990006edb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f991c0c9ce0 tx=0x7f990c009040 comp rx=0 tx=0).stop 2024-04-14T08:38:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.234+0000 7f992398c640 1 -- 172.21.15.172:0/1061823325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cca00 msgr2=0x7f991c1e6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.234+0000 7f992398c640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cca00 0x7f991c1e6be0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f991800efc0 tx=0x7f991800c490 comp rx=0 tx=0).stop 2024-04-14T08:38:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.234+0000 7f992398c640 1 -- 172.21.15.172:0/1061823325 shutdown_connections 2024-04-14T08:38:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.234+0000 7f992398c640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f990006c8f0 0x7f990006edb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.234+0000 7f992398c640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f991c1e9e50 0x7f991c1e74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.234+0000 7f992398c640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f991c0ceb40 0x7f991c1e9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.234+0000 7f992398c640 1 --2- 172.21.15.172:0/1061823325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cca00 0x7f991c1e6be0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.234+0000 7f992398c640 1 -- 172.21.15.172:0/1061823325 >> 172.21.15.172:0/1061823325 conn(0x7f991c0cd980 msgr2=0x7f991c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:47.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.234+0000 7f992398c640 1 -- 172.21.15.172:0/1061823325 shutdown_connections 2024-04-14T08:38:47.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.234+0000 7f992398c640 1 -- 172.21.15.172:0/1061823325 wait complete. 2024-04-14T08:38:47.249 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-14T08:38:47.249 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:38:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.353+0000 7f4bda269640 1 Processor -- start 2024-04-14T08:38:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.354+0000 7f4bda269640 1 -- start start 2024-04-14T08:38:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.354+0000 7f4bda269640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4bd40cedd0 0x7f4bd41694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.354+0000 7f4bda269640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd40cf320 con 0x7f4bd415e890 2024-04-14T08:38:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.354+0000 7f4bda269640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd40cf600 con 0x7f4bd40d2830 2024-04-14T08:38:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.354+0000 7f4bda269640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd40cff30 con 0x7f4bd40cedd0 2024-04-14T08:38:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.354+0000 7f4bd8806640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4bd40cedd0 0x7f4bd41694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.354+0000 7f4bd37fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4bd415e890 0x7f4bd40d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45732/0 (socket says 172.21.15.172:45732) 2024-04-14T08:38:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.354+0000 7f4bd37fe640 1 -- 172.21.15.172:0/3090179550 learned_addr learned my addr 172.21.15.172:0/3090179550 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.355+0000 7f4bd8806640 1 -- 172.21.15.172:0/3090179550 >> v1:172.21.15.177:6789/0 conn(0x7f4bd40d2830 legacy=0x7f4bd40d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.355+0000 7f4bd8806640 1 -- 172.21.15.172:0/3090179550 >> v1:172.21.15.172:6789/0 conn(0x7f4bd415e890 legacy=0x7f4bd40d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.355+0000 7f4bd2ffd640 1 -- 172.21.15.172:0/3090179550 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3898800478 0 0) 0x7f4bd40cf320 con 0x7f4bd415e890 2024-04-14T08:38:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.355+0000 7f4bd8806640 1 -- 172.21.15.172:0/3090179550 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd40cf8e0 con 0x7f4bd40cedd0 2024-04-14T08:38:47.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.356+0000 7f4bd8806640 1 --2- 172.21.15.172:0/3090179550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd40cedd0 0x7f4bd41694d0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009d90 tx=0x7f4bcc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2813a05020b5f388 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:47.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.356+0000 7f4bd2ffd640 1 -- 172.21.15.172:0/3090179550 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc039070 con 0x7f4bd40cedd0 2024-04-14T08:38:47.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.356+0000 7f4bd2ffd640 1 -- 172.21.15.172:0/3090179550 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bcc02cb70 con 0x7f4bd40cedd0 2024-04-14T08:38:47.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.356+0000 7f4bd2ffd640 1 -- 172.21.15.172:0/3090179550 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc034440 con 0x7f4bd40cedd0 2024-04-14T08:38:47.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.356+0000 7f4bda269640 1 -- 172.21.15.172:0/3090179550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd40cedd0 msgr2=0x7f4bd41694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.356+0000 7f4bda269640 1 --2- 172.21.15.172:0/3090179550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd40cedd0 0x7f4bd41694d0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009d90 tx=0x7f4bcc02be40 comp rx=0 tx=0).stop 2024-04-14T08:38:47.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.356+0000 7f4bda269640 1 -- 172.21.15.172:0/3090179550 shutdown_connections 2024-04-14T08:38:47.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.356+0000 7f4bda269640 1 --2- 172.21.15.172:0/3090179550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd40cedd0 0x7f4bd41694d0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.356+0000 7f4bda269640 1 -- 172.21.15.172:0/3090179550 >> 172.21.15.172:0/3090179550 conn(0x7f4bd409d590 msgr2=0x7f4bd40d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:47.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.356+0000 7f4bda269640 1 -- 172.21.15.172:0/3090179550 shutdown_connections 2024-04-14T08:38:47.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.356+0000 7f4bda269640 1 -- 172.21.15.172:0/3090179550 wait complete. 2024-04-14T08:38:47.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.357+0000 7f4bda269640 1 Processor -- start 2024-04-14T08:38:47.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.358+0000 7f4bda269640 1 -- start start 2024-04-14T08:38:47.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.358+0000 7f4bda269640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bd40cedd0 0x7f4bd40b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:47.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.358+0000 7f4bda269640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bd40d2830 0x7f4bd40c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:47.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.358+0000 7f4bda269640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd415e890 0x7f4bd40b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.358+0000 7f4bda269640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd40b4720 con 0x7f4bd415e890 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.358+0000 7f4bda269640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd40d2350 con 0x7f4bd40d2830 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.358+0000 7f4bda269640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd40b48a0 con 0x7f4bd40cedd0 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.358+0000 7f4bd8806640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd415e890 0x7f4bd40b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.358+0000 7f4bd8806640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd415e890 0x7f4bd40b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49792/0 (socket says 172.21.15.172:49792) 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.358+0000 7f4bd8806640 1 -- 172.21.15.172:0/491807130 learned_addr learned my addr 172.21.15.172:0/491807130 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.358+0000 7f4bd3fff640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bd40cedd0 0x7f4bd40b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.359+0000 7f4bd37fe640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bd40d2830 0x7f4bd40c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.359+0000 7f4bd8806640 1 -- 172.21.15.172:0/491807130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bd40cedd0 msgr2=0x7f4bd40b98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.359+0000 7f4bd8806640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bd40cedd0 0x7f4bd40b98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.359+0000 7f4bd8806640 1 -- 172.21.15.172:0/491807130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bd40d2830 msgr2=0x7f4bd40c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.359+0000 7f4bd8806640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bd40d2830 0x7f4bd40c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.359+0000 7f4bd8806640 1 -- 172.21.15.172:0/491807130 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd40c96a0 con 0x7f4bd415e890 2024-04-14T08:38:47.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.359+0000 7f4bd8806640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd415e890 0x7f4bd40b2410 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc037040 tx=0x7f4bcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:47.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.359+0000 7f4bd0ff9640 1 -- 172.21.15.172:0/491807130 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc044030 con 0x7f4bd415e890 2024-04-14T08:38:47.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.359+0000 7f4bd0ff9640 1 -- 172.21.15.172:0/491807130 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bcc02c970 con 0x7f4bd415e890 2024-04-14T08:38:47.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.359+0000 7f4bda269640 1 -- 172.21.15.172:0/491807130 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd40c9980 con 0x7f4bd415e890 2024-04-14T08:38:47.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.359+0000 7f4bd0ff9640 1 -- 172.21.15.172:0/491807130 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc039040 con 0x7f4bd415e890 2024-04-14T08:38:47.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.359+0000 7f4bda269640 1 -- 172.21.15.172:0/491807130 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd40c9f40 con 0x7f4bd415e890 2024-04-14T08:38:47.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.360+0000 7f4bda269640 1 -- 172.21.15.172:0/491807130 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd4068520 con 0x7f4bd415e890 2024-04-14T08:38:47.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.361+0000 7f4bd0ff9640 1 -- 172.21.15.172:0/491807130 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4bcc047070 con 0x7f4bd415e890 2024-04-14T08:38:47.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.361+0000 7f4bd0ff9640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4bc006c8f0 0x7f4bc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:47.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.361+0000 7f4bd0ff9640 1 -- 172.21.15.172:0/491807130 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f4bcc0ae0f0 con 0x7f4bd415e890 2024-04-14T08:38:47.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.361+0000 7f4bd3fff640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4bc006c8f0 0x7f4bc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:47.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.361+0000 7f4bd3fff640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4bc006c8f0 0x7f4bc006edb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4000f60 tx=0x7f4bc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:47.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.366+0000 7f4bd0ff9640 1 -- 172.21.15.172:0/491807130 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4bcc04f020 con 0x7f4bd415e890 2024-04-14T08:38:47.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.578+0000 7f4bda269640 1 -- 172.21.15.172:0/491807130 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bd40d2610 con 0x7f4bd415e890 2024-04-14T08:38:47.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.579+0000 7f4bd0ff9640 1 -- 172.21.15.172:0/491807130 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5195 (secure 0 0 0) 0x7f4bcc07bd70 con 0x7f4bd415e890 2024-04-14T08:38:47.580 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:47.580 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7473,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1386685680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1386685680},{"type":"v1","addr":"172.21.15.172:6837","nonce":1386685680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:38:46.573174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7491":{"gid":7491,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/1124731311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1124731311},{"type":"v1","addr":"172.21.15.177:6835","nonce":1124731311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7491,"qdb_cluster":[7491]},"id":6}]} 2024-04-14T08:38:47.581 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 54 2024-04-14T08:38:47.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.580+0000 7f4bda269640 1 -- 172.21.15.172:0/491807130 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4bc006c8f0 msgr2=0x7f4bc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.580+0000 7f4bda269640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4bc006c8f0 0x7f4bc006edb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4000f60 tx=0x7f4bc4009040 comp rx=0 tx=0).stop 2024-04-14T08:38:47.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.580+0000 7f4bda269640 1 -- 172.21.15.172:0/491807130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd415e890 msgr2=0x7f4bd40b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.580+0000 7f4bda269640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd415e890 0x7f4bd40b2410 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc037040 tx=0x7f4bcc004a40 comp rx=0 tx=0).stop 2024-04-14T08:38:47.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.581+0000 7f4bda269640 1 -- 172.21.15.172:0/491807130 shutdown_connections 2024-04-14T08:38:47.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.581+0000 7f4bda269640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4bc006c8f0 0x7f4bc006edb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.581+0000 7f4bda269640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd415e890 0x7f4bd40b2410 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.581+0000 7f4bda269640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bd40d2830 0x7f4bd40c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.581+0000 7f4bda269640 1 --2- 172.21.15.172:0/491807130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bd40cedd0 0x7f4bd40b98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.581+0000 7f4bda269640 1 -- 172.21.15.172:0/491807130 >> 172.21.15.172:0/491807130 conn(0x7f4bd409d590 msgr2=0x7f4bd406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:47.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.581+0000 7f4bda269640 1 -- 172.21.15.172:0/491807130 shutdown_connections 2024-04-14T08:38:47.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.581+0000 7f4bda269640 1 -- 172.21.15.172:0/491807130 wait complete. 2024-04-14T08:38:47.595 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:38:47.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.694+0000 7fb81b480640 1 Processor -- start 2024-04-14T08:38:47.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.695+0000 7fb81b480640 1 -- start start 2024-04-14T08:38:47.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.695+0000 7fb81b480640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8140b1df0 0x7fb8140b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:47.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.695+0000 7fb81b480640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8140b4040 0x7fb8140b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:47.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.695+0000 7fb81b480640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8140b49f0 con 0x7fb8140b21d0 2024-04-14T08:38:47.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.695+0000 7fb81b480640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb81415c770 con 0x7fb8140b4040 2024-04-14T08:38:47.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.695+0000 7fb81b480640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb8140b3430 con 0x7fb8140b1df0 2024-04-14T08:38:47.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.695+0000 7fb819a1d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8140b4040 0x7fb8140b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:47.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.696+0000 7fb81921c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8140b1df0 0x7fb8140b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:47.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.696+0000 7fb819a1d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8140b4040 0x7fb8140b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49796/0 (socket says 172.21.15.172:49796) 2024-04-14T08:38:47.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.696+0000 7fb818a1b640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fb8140b21d0 0x7fb81415c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:48196/0 (socket says 172.21.15.172:48196) 2024-04-14T08:38:47.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.696+0000 7fb819a1d640 1 -- 172.21.15.172:0/525166857 learned_addr learned my addr 172.21.15.172:0/525166857 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:47.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.696+0000 7fb819a1d640 1 -- 172.21.15.172:0/525166857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8140b1df0 msgr2=0x7fb8140b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:47.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.696+0000 7fb819a1d640 1 --2- 172.21.15.172:0/525166857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8140b1df0 0x7fb8140b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.696+0000 7fb819a1d640 1 -- 172.21.15.172:0/525166857 >> v1:172.21.15.177:6789/0 conn(0x7fb8140b21d0 legacy=0x7fb81415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.696+0000 7fb819a1d640 1 -- 172.21.15.172:0/525166857 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8140b3700 con 0x7fb8140b4040 2024-04-14T08:38:47.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.697+0000 7fb819a1d640 1 --2- 172.21.15.172:0/525166857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8140b4040 0x7fb8140b44b0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009e50 tx=0x7fb80c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357c532a147f424d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:47.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.697+0000 7fb80bfff640 1 -- 172.21.15.172:0/525166857 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c02c8d0 con 0x7fb8140b4040 2024-04-14T08:38:47.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.697+0000 7fb80bfff640 1 -- 172.21.15.172:0/525166857 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb80c02ca70 con 0x7fb8140b4040 2024-04-14T08:38:47.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.697+0000 7fb80bfff640 1 -- 172.21.15.172:0/525166857 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c02ce40 con 0x7fb8140b4040 2024-04-14T08:38:47.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.697+0000 7fb81b480640 1 -- 172.21.15.172:0/525166857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8140b4040 msgr2=0x7fb8140b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.697+0000 7fb81b480640 1 --2- 172.21.15.172:0/525166857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8140b4040 0x7fb8140b44b0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009e50 tx=0x7fb80c02bdb0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.697+0000 7fb81b480640 1 -- 172.21.15.172:0/525166857 shutdown_connections 2024-04-14T08:38:47.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.697+0000 7fb81b480640 1 --2- 172.21.15.172:0/525166857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8140b4040 0x7fb8140b44b0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.697+0000 7fb81b480640 1 --2- 172.21.15.172:0/525166857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8140b1df0 0x7fb8140b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.697+0000 7fb81b480640 1 -- 172.21.15.172:0/525166857 >> 172.21.15.172:0/525166857 conn(0x7fb81406ec30 msgr2=0x7fb81406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:47.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.697+0000 7fb81b480640 1 -- 172.21.15.172:0/525166857 shutdown_connections 2024-04-14T08:38:47.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.697+0000 7fb81b480640 1 -- 172.21.15.172:0/525166857 wait complete. 2024-04-14T08:38:47.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.698+0000 7fb81b480640 1 Processor -- start 2024-04-14T08:38:47.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.698+0000 7fb81b480640 1 -- start start 2024-04-14T08:38:47.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.699+0000 7fb81b480640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8140b1df0 0x7fb8140b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:47.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.699+0000 7fb81b480640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8140b21d0 0x7fb81413d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:47.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.699+0000 7fb81b480640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8140b4040 0x7fb8140acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:47.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.699+0000 7fb81b480640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb81415bb40 con 0x7fb8140b4040 2024-04-14T08:38:47.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.699+0000 7fb81b480640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb8140b3cc0 con 0x7fb8140b1df0 2024-04-14T08:38:47.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.699+0000 7fb81b480640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb81415b7f0 con 0x7fb8140b21d0 2024-04-14T08:38:47.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.699+0000 7fb819a1d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8140b4040 0x7fb8140acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:47.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.699+0000 7fb819a1d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8140b4040 0x7fb8140acc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49798/0 (socket says 172.21.15.172:49798) 2024-04-14T08:38:47.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.699+0000 7fb819a1d640 1 -- 172.21.15.172:0/2983262899 learned_addr learned my addr 172.21.15.172:0/2983262899 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:47.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.699+0000 7fb818a1b640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8140b21d0 0x7fb81413d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:47.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.699+0000 7fb81921c640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8140b1df0 0x7fb8140b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:47.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.700+0000 7fb819a1d640 1 -- 172.21.15.172:0/2983262899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8140b21d0 msgr2=0x7fb81413d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.700+0000 7fb819a1d640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8140b21d0 0x7fb81413d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.700+0000 7fb819a1d640 1 -- 172.21.15.172:0/2983262899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8140b1df0 msgr2=0x7fb8140b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.700+0000 7fb819a1d640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8140b1df0 0x7fb8140b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.700+0000 7fb819a1d640 1 -- 172.21.15.172:0/2983262899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8140aa130 con 0x7fb8140b4040 2024-04-14T08:38:47.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.700+0000 7fb819a1d640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8140b4040 0x7fb8140acc70 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009e50 tx=0x7fb80c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:47.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.700+0000 7fb809ffb640 1 -- 172.21.15.172:0/2983262899 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c043020 con 0x7fb8140b4040 2024-04-14T08:38:47.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.700+0000 7fb809ffb640 1 -- 172.21.15.172:0/2983262899 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb80c03e450 con 0x7fb8140b4040 2024-04-14T08:38:47.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.700+0000 7fb81b480640 1 -- 172.21.15.172:0/2983262899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8140aa410 con 0x7fb8140b4040 2024-04-14T08:38:47.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.700+0000 7fb809ffb640 1 -- 172.21.15.172:0/2983262899 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c009510 con 0x7fb8140b4040 2024-04-14T08:38:47.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.700+0000 7fb81b480640 1 -- 172.21.15.172:0/2983262899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8140aa9d0 con 0x7fb8140b4040 2024-04-14T08:38:47.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.701+0000 7fb81b480640 1 -- 172.21.15.172:0/2983262899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb814068520 con 0x7fb8140b4040 2024-04-14T08:38:47.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.702+0000 7fb809ffb640 1 -- 172.21.15.172:0/2983262899 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb80c046020 con 0x7fb8140b4040 2024-04-14T08:38:47.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.702+0000 7fb809ffb640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7f806c8f0 0x7fb7f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:47.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.702+0000 7fb81921c640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7f806c8f0 0x7fb7f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:47.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.702+0000 7fb809ffb640 1 -- 172.21.15.172:0/2983262899 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fb80c0ae290 con 0x7fb8140b4040 2024-04-14T08:38:47.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.703+0000 7fb81921c640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7f806c8f0 0x7fb7f806edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb81000bde0 tx=0x7fb81000b3a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:47.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.707+0000 7fb809ffb640 1 -- 172.21.15.172:0/2983262899 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb80c04d030 con 0x7fb8140b4040 2024-04-14T08:38:47.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.883+0000 7fb81b480640 1 -- 172.21.15.172:0/2983262899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8140b4be0 con 0x7fb8140b4040 2024-04-14T08:38:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.884+0000 7fb809ffb640 1 -- 172.21.15.172:0/2983262899 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16945 (secure 0 0 0) 0x7fb80c07bf70 con 0x7fb8140b4040 2024-04-14T08:38:47.886 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:47.886 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":86,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:38:46.208211+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:38:42.790996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-14T08:38:43.517208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:38:47.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.886+0000 7fb81b480640 1 -- 172.21.15.172:0/2983262899 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7f806c8f0 msgr2=0x7fb7f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.886+0000 7fb81b480640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7f806c8f0 0x7fb7f806edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb81000bde0 tx=0x7fb81000b3a0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.886+0000 7fb81b480640 1 -- 172.21.15.172:0/2983262899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8140b4040 msgr2=0x7fb8140acc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:47.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.886+0000 7fb81b480640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8140b4040 0x7fb8140acc70 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009e50 tx=0x7fb80c004a40 comp rx=0 tx=0).stop 2024-04-14T08:38:47.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.886+0000 7fb81b480640 1 -- 172.21.15.172:0/2983262899 shutdown_connections 2024-04-14T08:38:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.886+0000 7fb81b480640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7f806c8f0 0x7fb7f806edb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.886+0000 7fb81b480640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8140b4040 0x7fb8140acc70 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.886+0000 7fb81b480640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8140b21d0 0x7fb81413d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.886+0000 7fb81b480640 1 --2- 172.21.15.172:0/2983262899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8140b1df0 0x7fb8140b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.886+0000 7fb81b480640 1 -- 172.21.15.172:0/2983262899 >> 172.21.15.172:0/2983262899 conn(0x7fb81406ec30 msgr2=0x7fb814071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.886+0000 7fb81b480640 1 -- 172.21.15.172:0/2983262899 shutdown_connections 2024-04-14T08:38:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.886+0000 7fb81b480640 1 -- 172.21.15.172:0/2983262899 wait complete. 2024-04-14T08:38:47.899 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-14T08:38:47.899 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:38:48.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.998+0000 7f46b4020640 1 Processor -- start 2024-04-14T08:38:48.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.998+0000 7f46b4020640 1 -- start start 2024-04-14T08:38:48.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.999+0000 7f46b4020640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ac0b2290 0x7f46ac0d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.999+0000 7f46b4020640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ac158f10 con 0x7f46ac158b30 2024-04-14T08:38:48.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.999+0000 7f46b4020640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ac15f7b0 con 0x7f46ac15ea80 2024-04-14T08:38:48.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.999+0000 7f46b4020640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f46ac0cfd20 con 0x7f46ac0b2290 2024-04-14T08:38:48.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.999+0000 7f46b1dbc640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f46ac158b30 0x7f46ac0b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45738/0 (socket says 172.21.15.172:45738) 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.999+0000 7f46b1dbc640 1 -- 172.21.15.172:0/426796198 learned_addr learned my addr 172.21.15.172:0/426796198 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:47.999+0000 7f46b15bb640 1 --2- 172.21.15.172:0/426796198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ac0b2290 0x7f46ac0d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.000+0000 7f46b0dba640 1 -- 172.21.15.172:0/426796198 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3830630545 0 0) 0x7f46ac158f10 con 0x7f46ac158b30 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.000+0000 7f46b0dba640 1 -- 172.21.15.172:0/426796198 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4698003520 con 0x7f46ac158b30 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.000+0000 7f46b0dba640 1 -- 172.21.15.172:0/426796198 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1327970041 0 0) 0x7f46ac15f7b0 con 0x7f46ac15ea80 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.000+0000 7f46b15bb640 1 -- 172.21.15.172:0/426796198 >> v1:172.21.15.177:6789/0 conn(0x7f46ac15ea80 legacy=0x7f46ac15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.000+0000 7f46b15bb640 1 -- 172.21.15.172:0/426796198 >> v1:172.21.15.172:6789/0 conn(0x7f46ac158b30 legacy=0x7f46ac0b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.000+0000 7f46b15bb640 1 -- 172.21.15.172:0/426796198 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac0d0000 con 0x7f46ac0b2290 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.001+0000 7f46b15bb640 1 --2- 172.21.15.172:0/426796198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ac0b2290 0x7f46ac0d0800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f469c0097d0 tx=0x7f469c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7dc88d5dfc78796 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.001+0000 7f46b0dba640 1 -- 172.21.15.172:0/426796198 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c035020 con 0x7f46ac0b2290 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.001+0000 7f46b0dba640 1 -- 172.21.15.172:0/426796198 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f469c031890 con 0x7f46ac0b2290 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.001+0000 7f46b0dba640 1 -- 172.21.15.172:0/426796198 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c031c40 con 0x7f46ac0b2290 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.001+0000 7f46b4020640 1 -- 172.21.15.172:0/426796198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ac0b2290 msgr2=0x7f46ac0d0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.001+0000 7f46b4020640 1 --2- 172.21.15.172:0/426796198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ac0b2290 0x7f46ac0d0800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f469c0097d0 tx=0x7f469c02b930 comp rx=0 tx=0).stop 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.001+0000 7f46b4020640 1 -- 172.21.15.172:0/426796198 shutdown_connections 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.001+0000 7f46b4020640 1 --2- 172.21.15.172:0/426796198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ac0b2290 0x7f46ac0d0800 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.001+0000 7f46b4020640 1 -- 172.21.15.172:0/426796198 >> 172.21.15.172:0/426796198 conn(0x7f46ac06ec30 msgr2=0x7f46ac06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.001+0000 7f46b4020640 1 -- 172.21.15.172:0/426796198 shutdown_connections 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.001+0000 7f46b4020640 1 -- 172.21.15.172:0/426796198 wait complete. 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.002+0000 7f46b4020640 1 Processor -- start 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.002+0000 7f46b4020640 1 -- start start 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.003+0000 7f46b4020640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46ac0b2290 0x7f46ac159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.003+0000 7f46b4020640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ac158b30 0x7f46ac15b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.003+0000 7f46b4020640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ac15ea80 0x7f46ac152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.003+0000 7f46b4020640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac0717d0 con 0x7f46ac0b2290 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.003+0000 7f46b4020640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac071650 con 0x7f46ac158b30 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.003+0000 7f46b4020640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f46ac071950 con 0x7f46ac15ea80 2024-04-14T08:38:48.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.003+0000 7f46b1dbc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46ac0b2290 0x7f46ac159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.003+0000 7f46b1dbc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46ac0b2290 0x7f46ac159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49804/0 (socket says 172.21.15.172:49804) 2024-04-14T08:38:48.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.003+0000 7f46b1dbc640 1 -- 172.21.15.172:0/603319884 learned_addr learned my addr 172.21.15.172:0/603319884 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:48.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.003+0000 7f46b25bd640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ac15ea80 0x7f46ac152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.003+0000 7f46b15bb640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ac158b30 0x7f46ac15b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.004+0000 7f46b1dbc640 1 -- 172.21.15.172:0/603319884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ac15ea80 msgr2=0x7f46ac152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.004+0000 7f46b1dbc640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ac15ea80 0x7f46ac152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.004+0000 7f46b1dbc640 1 -- 172.21.15.172:0/603319884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ac158b30 msgr2=0x7f46ac15b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.004+0000 7f46b1dbc640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ac158b30 0x7f46ac15b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.004+0000 7f46b1dbc640 1 -- 172.21.15.172:0/603319884 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac159140 con 0x7f46ac0b2290 2024-04-14T08:38:48.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.004+0000 7f46b1dbc640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46ac0b2290 0x7f46ac159af0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f46a800cb70 tx=0x7f46a800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:48.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.004+0000 7f46a27fc640 1 -- 172.21.15.172:0/603319884 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8017070 con 0x7f46ac0b2290 2024-04-14T08:38:48.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.004+0000 7f46a27fc640 1 -- 172.21.15.172:0/603319884 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a800ad40 con 0x7f46ac0b2290 2024-04-14T08:38:48.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.004+0000 7f46a27fc640 1 -- 172.21.15.172:0/603319884 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8012620 con 0x7f46ac0b2290 2024-04-14T08:38:48.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.004+0000 7f46b4020640 1 -- 172.21.15.172:0/603319884 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ac159420 con 0x7f46ac0b2290 2024-04-14T08:38:48.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.004+0000 7f46b4020640 1 -- 172.21.15.172:0/603319884 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ac155580 con 0x7f46ac0b2290 2024-04-14T08:38:48.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.005+0000 7f46b4020640 1 -- 172.21.15.172:0/603319884 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ac159810 con 0x7f46ac0b2290 2024-04-14T08:38:48.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.005+0000 7f46a27fc640 1 -- 172.21.15.172:0/603319884 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46a801a070 con 0x7f46ac0b2290 2024-04-14T08:38:48.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.006+0000 7f46a27fc640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f469006c8f0 0x7f469006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.006+0000 7f46a27fc640 1 -- 172.21.15.172:0/603319884 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f46a808ba10 con 0x7f46ac0b2290 2024-04-14T08:38:48.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.006+0000 7f46b15bb640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f469006c8f0 0x7f469006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.006+0000 7f46b15bb640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f469006c8f0 0x7f469006edb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f469c036000 tx=0x7f469c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:48.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.010+0000 7f46a27fc640 1 -- 172.21.15.172:0/603319884 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f46ac159810 con 0x7f46ac0b2290 2024-04-14T08:38:48.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.181+0000 7f46b4020640 1 -- 172.21.15.172:0/603319884 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46ac068520 con 0x7f469006c8f0 2024-04-14T08:38:48.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.187+0000 7f46a27fc640 1 -- 172.21.15.172:0/603319884 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247193 (secure 0 0 0) 0x7f469c0865b0 con 0x7f469006c8f0 2024-04-14T08:38:48.191 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:48.193 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:38:48.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.190+0000 7f46b4020640 1 -- 172.21.15.172:0/603319884 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f469006c8f0 msgr2=0x7f469006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.190+0000 7f46b4020640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f469006c8f0 0x7f469006edb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f469c036000 tx=0x7f469c008030 comp rx=0 tx=0).stop 2024-04-14T08:38:48.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.190+0000 7f46b4020640 1 -- 172.21.15.172:0/603319884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46ac0b2290 msgr2=0x7f46ac159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.190+0000 7f46b4020640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46ac0b2290 0x7f46ac159af0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f46a800cb70 tx=0x7f46a800a430 comp rx=0 tx=0).stop 2024-04-14T08:38:48.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.190+0000 7f46b4020640 1 -- 172.21.15.172:0/603319884 shutdown_connections 2024-04-14T08:38:48.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.190+0000 7f46b4020640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f469006c8f0 0x7f469006edb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.190+0000 7f46b4020640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f46ac15ea80 0x7f46ac152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.190+0000 7f46b4020640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ac158b30 0x7f46ac15b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.190+0000 7f46b4020640 1 --2- 172.21.15.172:0/603319884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46ac0b2290 0x7f46ac159af0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.190+0000 7f46b4020640 1 -- 172.21.15.172:0/603319884 >> 172.21.15.172:0/603319884 conn(0x7f46ac06ec30 msgr2=0x7f46ac072dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:48.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.190+0000 7f46b4020640 1 -- 172.21.15.172:0/603319884 shutdown_connections 2024-04-14T08:38:48.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.190+0000 7f46b4020640 1 -- 172.21.15.172:0/603319884 wait complete. 2024-04-14T08:38:48.207 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-04-14T08:38:47.191220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590402,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246256,"kb_used_data":9264,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754728464,"statfs":{"total":773094113280,"available":772841947136,"internally_reserved":0,"allocated":9486336,"data_stored":4952124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001105"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216414+0000","last_change":"2024-04-14T08:38:44.216414+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216414+0000","last_undegraded":"2024-04-14T08:38:44.216414+0000","last_fullsized":"2024-04-14T08:38:44.216414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217436+0000","last_change":"2024-04-14T08:38:44.217436+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217436+0000","last_undegraded":"2024-04-14T08:38:44.217436+0000","last_fullsized":"2024-04-14T08:38:44.217436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215977+0000","last_change":"2024-04-14T08:38:44.215977+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215977+0000","last_undegraded":"2024-04-14T08:38:44.215977+0000","last_fullsized":"2024-04-14T08:38:44.215977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.220133+0000","last_change":"2024-04-14T08:38:44.220133+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.220133+0000","last_undegraded":"2024-04-14T08:38:44.220133+0000","last_fullsized":"2024-04-14T08:38:44.220133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211723+0000","last_change":"2024-04-14T08:38:44.211723+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211723+0000","last_undegraded":"2024-04-14T08:38:44.211723+0000","last_fullsized":"2024-04-14T08:38:44.211723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.228102+0000","last_change":"2024-04-14T08:38:44.228102+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.228102+0000","last_undegraded":"2024-04-14T08:38:44.228102+0000","last_fullsized":"2024-04-14T08:38:44.228102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.226673+0000","last_change":"2024-04-14T08:38:45.221574+0000","last_active":"2024-04-14T08:38:45.226673+0000","last_peered":"2024-04-14T08:38:45.226673+0000","last_clean":"2024-04-14T08:38:45.226673+0000","last_became_active":"2024-04-14T08:38:45.221303+0000","last_became_peered":"2024-04-14T08:38:45.221303+0000","last_unstale":"2024-04-14T08:38:45.226673+0000","last_undegraded":"2024-04-14T08:38:45.226673+0000","last_fullsized":"2024-04-14T08:38:45.226673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:59:38.604514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.219635+0000","last_change":"2024-04-14T08:38:44.219635+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.219635+0000","last_undegraded":"2024-04-14T08:38:44.219635+0000","last_fullsized":"2024-04-14T08:38:44.219635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.212588+0000","last_change":"2024-04-14T08:38:44.212588+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.212588+0000","last_undegraded":"2024-04-14T08:38:44.212588+0000","last_fullsized":"2024-04-14T08:38:44.212588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216941+0000","last_change":"2024-04-14T08:38:44.216941+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216941+0000","last_undegraded":"2024-04-14T08:38:44.216941+0000","last_fullsized":"2024-04-14T08:38:44.216941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211782+0000","last_change":"2024-04-14T08:38:44.211782+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211782+0000","last_undegraded":"2024-04-14T08:38:44.211782+0000","last_fullsized":"2024-04-14T08:38:44.211782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216683+0000","last_change":"2024-04-14T08:38:44.216683+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216683+0000","last_undegraded":"2024-04-14T08:38:44.216683+0000","last_fullsized":"2024-04-14T08:38:44.216683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.220232+0000","last_change":"2024-04-14T08:38:44.220232+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.220232+0000","last_undegraded":"2024-04-14T08:38:44.220232+0000","last_fullsized":"2024-04-14T08:38:44.220232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.213829+0000","last_change":"2024-04-14T08:38:44.213829+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.213829+0000","last_undegraded":"2024-04-14T08:38:44.213829+0000","last_fullsized":"2024-04-14T08:38:44.213829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.219502+0000","last_change":"2024-04-14T08:38:44.219502+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.219502+0000","last_undegraded":"2024-04-14T08:38:44.219502+0000","last_fullsized":"2024-04-14T08:38:44.219502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.228755+0000","last_change":"2024-04-14T08:38:44.228755+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.228755+0000","last_undegraded":"2024-04-14T08:38:44.228755+0000","last_fullsized":"2024-04-14T08:38:44.228755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.227961+0000","last_change":"2024-04-14T08:38:44.227961+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.227961+0000","last_undegraded":"2024-04-14T08:38:44.227961+0000","last_fullsized":"2024-04-14T08:38:44.227961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215464+0000","last_change":"2024-04-14T08:38:44.215464+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215464+0000","last_undegraded":"2024-04-14T08:38:44.215464+0000","last_fullsized":"2024-04-14T08:38:44.215464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211266+0000","last_change":"2024-04-14T08:38:44.211266+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211266+0000","last_undegraded":"2024-04-14T08:38:44.211266+0000","last_fullsized":"2024-04-14T08:38:44.211266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218880+0000","last_change":"2024-04-14T08:38:44.218880+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218880+0000","last_undegraded":"2024-04-14T08:38:44.218880+0000","last_fullsized":"2024-04-14T08:38:44.218880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.212313+0000","last_change":"2024-04-14T08:38:44.212313+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.212313+0000","last_undegraded":"2024-04-14T08:38:44.212313+0000","last_fullsized":"2024-04-14T08:38:44.212313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.213369+0000","last_change":"2024-04-14T08:38:44.213369+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.213369+0000","last_undegraded":"2024-04-14T08:38:44.213369+0000","last_fullsized":"2024-04-14T08:38:44.213369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218241+0000","last_change":"2024-04-14T08:38:44.218241+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218241+0000","last_undegraded":"2024-04-14T08:38:44.218241+0000","last_fullsized":"2024-04-14T08:38:44.218241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215033+0000","last_change":"2024-04-14T08:38:44.215033+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215033+0000","last_undegraded":"2024-04-14T08:38:44.215033+0000","last_fullsized":"2024-04-14T08:38:44.215033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216786+0000","last_change":"2024-04-14T08:38:44.216786+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216786+0000","last_undegraded":"2024-04-14T08:38:44.216786+0000","last_fullsized":"2024-04-14T08:38:44.216786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.213997+0000","last_change":"2024-04-14T08:38:44.213997+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.213997+0000","last_undegraded":"2024-04-14T08:38:44.213997+0000","last_fullsized":"2024-04-14T08:38:44.213997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216757+0000","last_change":"2024-04-14T08:38:44.216757+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216757+0000","last_undegraded":"2024-04-14T08:38:44.216757+0000","last_fullsized":"2024-04-14T08:38:44.216757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218369+0000","last_change":"2024-04-14T08:38:44.218369+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218369+0000","last_undegraded":"2024-04-14T08:38:44.218369+0000","last_fullsized":"2024-04-14T08:38:44.218369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.229290+0000","last_change":"2024-04-14T08:38:45.226642+0000","last_active":"2024-04-14T08:38:45.229290+0000","last_peered":"2024-04-14T08:38:45.229290+0000","last_clean":"2024-04-14T08:38:45.229290+0000","last_became_active":"2024-04-14T08:38:45.226349+0000","last_became_peered":"2024-04-14T08:38:45.226349+0000","last_unstale":"2024-04-14T08:38:45.229290+0000","last_undegraded":"2024-04-14T08:38:45.229290+0000","last_fullsized":"2024-04-14T08:38:45.229290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:17:06.164898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211114+0000","last_change":"2024-04-14T08:38:44.211114+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211114+0000","last_undegraded":"2024-04-14T08:38:44.211114+0000","last_fullsized":"2024-04-14T08:38:44.211114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217550+0000","last_change":"2024-04-14T08:38:44.217550+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217550+0000","last_undegraded":"2024-04-14T08:38:44.217550+0000","last_fullsized":"2024-04-14T08:38:44.217550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.228329+0000","last_change":"2024-04-14T08:38:44.228329+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.228329+0000","last_undegraded":"2024-04-14T08:38:44.228329+0000","last_fullsized":"2024-04-14T08:38:44.228329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.225245+0000","last_change":"2024-04-14T08:38:44.225245+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.225245+0000","last_undegraded":"2024-04-14T08:38:44.225245+0000","last_fullsized":"2024-04-14T08:38:44.225245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228182+0000","last_change":"2024-04-14T08:38:44.227154+0000","last_active":"2024-04-14T08:38:44.228182+0000","last_peered":"2024-04-14T08:38:44.228182+0000","last_clean":"2024-04-14T08:38:44.228182+0000","last_became_active":"2024-04-14T08:38:44.226910+0000","last_became_peered":"2024-04-14T08:38:44.226910+0000","last_unstale":"2024-04-14T08:38:44.228182+0000","last_undegraded":"2024-04-14T08:38:44.228182+0000","last_fullsized":"2024-04-14T08:38:44.228182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:49:42.403181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.227247+0000","last_change":"2024-04-14T08:38:44.227247+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.227247+0000","last_undegraded":"2024-04-14T08:38:44.227247+0000","last_fullsized":"2024-04-14T08:38:44.227247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225223+0000","last_change":"2024-04-14T08:38:44.222545+0000","last_active":"2024-04-14T08:38:44.225223+0000","last_peered":"2024-04-14T08:38:44.225223+0000","last_clean":"2024-04-14T08:38:44.225223+0000","last_became_active":"2024-04-14T08:38:44.221999+0000","last_became_peered":"2024-04-14T08:38:44.221999+0000","last_unstale":"2024-04-14T08:38:44.225223+0000","last_undegraded":"2024-04-14T08:38:44.225223+0000","last_fullsized":"2024-04-14T08:38:44.225223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:37:59.212931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.235826+0000","last_change":"2024-04-14T08:38:45.228983+0000","last_active":"2024-04-14T08:38:45.235826+0000","last_peered":"2024-04-14T08:38:45.235826+0000","last_clean":"2024-04-14T08:38:45.235826+0000","last_became_active":"2024-04-14T08:38:45.228632+0000","last_became_peered":"2024-04-14T08:38:45.228632+0000","last_unstale":"2024-04-14T08:38:45.235826+0000","last_undegraded":"2024-04-14T08:38:45.235826+0000","last_fullsized":"2024-04-14T08:38:45.235826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:17:19.947504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.240108+0000","last_change":"2024-04-14T08:38:44.214063+0000","last_active":"2024-04-14T08:38:45.240108+0000","last_peered":"2024-04-14T08:38:45.240108+0000","last_clean":"2024-04-14T08:38:45.240108+0000","last_became_active":"2024-04-14T08:38:44.213612+0000","last_became_peered":"2024-04-14T08:38:44.213612+0000","last_unstale":"2024-04-14T08:38:45.240108+0000","last_undegraded":"2024-04-14T08:38:45.240108+0000","last_fullsized":"2024-04-14T08:38:45.240108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:15:04.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.227051+0000","last_change":"2024-04-14T08:38:44.227051+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.227051+0000","last_undegraded":"2024-04-14T08:38:44.227051+0000","last_fullsized":"2024-04-14T08:38:44.227051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228169+0000","last_change":"2024-04-14T08:38:44.227021+0000","last_active":"2024-04-14T08:38:44.228169+0000","last_peered":"2024-04-14T08:38:44.228169+0000","last_clean":"2024-04-14T08:38:44.228169+0000","last_became_active":"2024-04-14T08:38:44.226731+0000","last_became_peered":"2024-04-14T08:38:44.226731+0000","last_unstale":"2024-04-14T08:38:44.228169+0000","last_undegraded":"2024-04-14T08:38:44.228169+0000","last_fullsized":"2024-04-14T08:38:44.228169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:01.799418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217415+0000","last_change":"2024-04-14T08:38:44.217415+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217415+0000","last_undegraded":"2024-04-14T08:38:44.217415+0000","last_fullsized":"2024-04-14T08:38:44.217415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225249+0000","last_change":"2024-04-14T08:38:44.222167+0000","last_active":"2024-04-14T08:38:44.225249+0000","last_peered":"2024-04-14T08:38:44.225249+0000","last_clean":"2024-04-14T08:38:44.225249+0000","last_became_active":"2024-04-14T08:38:44.221892+0000","last_became_peered":"2024-04-14T08:38:44.221892+0000","last_unstale":"2024-04-14T08:38:44.225249+0000","last_undegraded":"2024-04-14T08:38:44.225249+0000","last_fullsized":"2024-04-14T08:38:44.225249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:13.870437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217818+0000","last_change":"2024-04-14T08:38:44.217818+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217818+0000","last_undegraded":"2024-04-14T08:38:44.217818+0000","last_fullsized":"2024-04-14T08:38:44.217818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226141+0000","last_change":"2024-04-14T08:38:44.222906+0000","last_active":"2024-04-14T08:38:44.226141+0000","last_peered":"2024-04-14T08:38:44.226141+0000","last_clean":"2024-04-14T08:38:44.226141+0000","last_became_active":"2024-04-14T08:38:44.222541+0000","last_became_peered":"2024-04-14T08:38:44.222541+0000","last_unstale":"2024-04-14T08:38:44.226141+0000","last_undegraded":"2024-04-14T08:38:44.226141+0000","last_fullsized":"2024-04-14T08:38:44.226141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:21:35.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.221823+0000","last_change":"2024-04-14T08:38:44.221823+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.221823+0000","last_undegraded":"2024-04-14T08:38:44.221823+0000","last_fullsized":"2024-04-14T08:38:44.221823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.223396+0000","last_change":"2024-04-14T08:38:44.218254+0000","last_active":"2024-04-14T08:38:44.223396+0000","last_peered":"2024-04-14T08:38:44.223396+0000","last_clean":"2024-04-14T08:38:44.223396+0000","last_became_active":"2024-04-14T08:38:44.217969+0000","last_became_peered":"2024-04-14T08:38:44.217969+0000","last_unstale":"2024-04-14T08:38:44.223396+0000","last_undegraded":"2024-04-14T08:38:44.223396+0000","last_fullsized":"2024-04-14T08:38:44.223396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:12:14.185232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217899+0000","last_change":"2024-04-14T08:38:44.217899+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217899+0000","last_undegraded":"2024-04-14T08:38:44.217899+0000","last_fullsized":"2024-04-14T08:38:44.217899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.214121+0000","last_change":"2024-04-14T08:38:44.231501+0000","last_active":"2024-04-14T08:38:45.214121+0000","last_peered":"2024-04-14T08:38:45.214121+0000","last_clean":"2024-04-14T08:38:45.214121+0000","last_became_active":"2024-04-14T08:38:44.231184+0000","last_became_peered":"2024-04-14T08:38:44.231184+0000","last_unstale":"2024-04-14T08:38:45.214121+0000","last_undegraded":"2024-04-14T08:38:45.214121+0000","last_fullsized":"2024-04-14T08:38:45.214121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:59:08.058016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.212060+0000","last_change":"2024-04-14T08:38:44.212060+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.212060+0000","last_undegraded":"2024-04-14T08:38:44.212060+0000","last_fullsized":"2024-04-14T08:38:44.212060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225545+0000","last_change":"2024-04-14T08:38:44.224713+0000","last_active":"2024-04-14T08:38:44.225545+0000","last_peered":"2024-04-14T08:38:44.225545+0000","last_clean":"2024-04-14T08:38:44.225545+0000","last_became_active":"2024-04-14T08:38:44.224437+0000","last_became_peered":"2024-04-14T08:38:44.224437+0000","last_unstale":"2024-04-14T08:38:44.225545+0000","last_undegraded":"2024-04-14T08:38:44.225545+0000","last_fullsized":"2024-04-14T08:38:44.225545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:37.147141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.221898+0000","last_change":"2024-04-14T08:38:45.219595+0000","last_active":"2024-04-14T08:38:45.221898+0000","last_peered":"2024-04-14T08:38:45.221898+0000","last_clean":"2024-04-14T08:38:45.221898+0000","last_became_active":"2024-04-14T08:38:45.219269+0000","last_became_peered":"2024-04-14T08:38:45.219269+0000","last_unstale":"2024-04-14T08:38:45.221898+0000","last_undegraded":"2024-04-14T08:38:45.221898+0000","last_fullsized":"2024-04-14T08:38:45.221898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:48:28.586545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.224564+0000","last_change":"2024-04-14T08:38:44.219411+0000","last_active":"2024-04-14T08:38:44.224564+0000","last_peered":"2024-04-14T08:38:44.224564+0000","last_clean":"2024-04-14T08:38:44.224564+0000","last_became_active":"2024-04-14T08:38:44.219130+0000","last_became_peered":"2024-04-14T08:38:44.219130+0000","last_unstale":"2024-04-14T08:38:44.224564+0000","last_undegraded":"2024-04-14T08:38:44.224564+0000","last_fullsized":"2024-04-14T08:38:44.224564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:43:02.184142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.214550+0000","last_change":"2024-04-14T08:38:44.214550+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.214550+0000","last_undegraded":"2024-04-14T08:38:44.214550+0000","last_fullsized":"2024-04-14T08:38:44.214550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233151+0000","last_change":"2024-04-14T08:38:44.231600+0000","last_active":"2024-04-14T08:38:44.233151+0000","last_peered":"2024-04-14T08:38:44.233151+0000","last_clean":"2024-04-14T08:38:44.233151+0000","last_became_active":"2024-04-14T08:38:44.231276+0000","last_became_peered":"2024-04-14T08:38:44.231276+0000","last_unstale":"2024-04-14T08:38:44.233151+0000","last_undegraded":"2024-04-14T08:38:44.233151+0000","last_fullsized":"2024-04-14T08:38:44.233151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:42.226080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216192+0000","last_change":"2024-04-14T08:38:44.216192+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216192+0000","last_undegraded":"2024-04-14T08:38:44.216192+0000","last_fullsized":"2024-04-14T08:38:44.216192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233087+0000","last_change":"2024-04-14T08:38:44.230303+0000","last_active":"2024-04-14T08:38:44.233087+0000","last_peered":"2024-04-14T08:38:44.233087+0000","last_clean":"2024-04-14T08:38:44.233087+0000","last_became_active":"2024-04-14T08:38:44.230051+0000","last_became_peered":"2024-04-14T08:38:44.230051+0000","last_unstale":"2024-04-14T08:38:44.233087+0000","last_undegraded":"2024-04-14T08:38:44.233087+0000","last_fullsized":"2024-04-14T08:38:44.233087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:36:12.417551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216476+0000","last_change":"2024-04-14T08:38:44.216476+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216476+0000","last_undegraded":"2024-04-14T08:38:44.216476+0000","last_fullsized":"2024-04-14T08:38:44.216476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.245313+0000","last_change":"2024-04-14T08:38:44.230951+0000","last_active":"2024-04-14T08:38:45.245313+0000","last_peered":"2024-04-14T08:38:45.245313+0000","last_clean":"2024-04-14T08:38:45.245313+0000","last_became_active":"2024-04-14T08:38:44.230499+0000","last_became_peered":"2024-04-14T08:38:44.230499+0000","last_unstale":"2024-04-14T08:38:45.245313+0000","last_undegraded":"2024-04-14T08:38:45.245313+0000","last_fullsized":"2024-04-14T08:38:45.245313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:55:56.125870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.210337+0000","last_change":"2024-04-14T08:38:44.210337+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.210337+0000","last_undegraded":"2024-04-14T08:38:44.210337+0000","last_fullsized":"2024-04-14T08:38:44.210337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233084+0000","last_change":"2024-04-14T08:38:44.230300+0000","last_active":"2024-04-14T08:38:44.233084+0000","last_peered":"2024-04-14T08:38:44.233084+0000","last_clean":"2024-04-14T08:38:44.233084+0000","last_became_active":"2024-04-14T08:38:44.230024+0000","last_became_peered":"2024-04-14T08:38:44.230024+0000","last_unstale":"2024-04-14T08:38:44.233084+0000","last_undegraded":"2024-04-14T08:38:44.233084+0000","last_fullsized":"2024-04-14T08:38:44.233084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:15:04.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216501+0000","last_change":"2024-04-14T08:38:44.216501+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216501+0000","last_undegraded":"2024-04-14T08:38:44.216501+0000","last_fullsized":"2024-04-14T08:38:44.216501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.227819+0000","last_change":"2024-04-14T08:38:44.224811+0000","last_active":"2024-04-14T08:38:44.227819+0000","last_peered":"2024-04-14T08:38:44.227819+0000","last_clean":"2024-04-14T08:38:44.227819+0000","last_became_active":"2024-04-14T08:38:44.224541+0000","last_became_peered":"2024-04-14T08:38:44.224541+0000","last_unstale":"2024-04-14T08:38:44.227819+0000","last_undegraded":"2024-04-14T08:38:44.227819+0000","last_fullsized":"2024-04-14T08:38:44.227819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:31:11.641961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217216+0000","last_change":"2024-04-14T08:38:44.217216+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217216+0000","last_undegraded":"2024-04-14T08:38:44.217216+0000","last_fullsized":"2024-04-14T08:38:44.217216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.217797+0000","last_change":"2024-04-14T08:38:44.216083+0000","last_active":"2024-04-14T08:38:44.217797+0000","last_peered":"2024-04-14T08:38:44.217797+0000","last_clean":"2024-04-14T08:38:44.217797+0000","last_became_active":"2024-04-14T08:38:44.215754+0000","last_became_peered":"2024-04-14T08:38:44.215754+0000","last_unstale":"2024-04-14T08:38:44.217797+0000","last_undegraded":"2024-04-14T08:38:44.217797+0000","last_fullsized":"2024-04-14T08:38:44.217797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:13.870437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.222718+0000","last_change":"2024-04-14T08:38:44.222718+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.222718+0000","last_undegraded":"2024-04-14T08:38:44.222718+0000","last_fullsized":"2024-04-14T08:38:44.222718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.223232+0000","last_change":"2024-04-14T08:38:44.222002+0000","last_active":"2024-04-14T08:38:44.223232+0000","last_peered":"2024-04-14T08:38:44.223232+0000","last_clean":"2024-04-14T08:38:44.223232+0000","last_became_active":"2024-04-14T08:38:44.221679+0000","last_became_peered":"2024-04-14T08:38:44.221679+0000","last_unstale":"2024-04-14T08:38:44.223232+0000","last_undegraded":"2024-04-14T08:38:44.223232+0000","last_fullsized":"2024-04-14T08:38:44.223232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:53.649370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215754+0000","last_change":"2024-04-14T08:38:44.215754+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215754+0000","last_undegraded":"2024-04-14T08:38:44.215754+0000","last_fullsized":"2024-04-14T08:38:44.215754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.241020+0000","last_change":"2024-04-14T08:38:44.220058+0000","last_active":"2024-04-14T08:38:45.241020+0000","last_peered":"2024-04-14T08:38:45.241020+0000","last_clean":"2024-04-14T08:38:45.241020+0000","last_became_active":"2024-04-14T08:38:44.219721+0000","last_became_peered":"2024-04-14T08:38:44.219721+0000","last_unstale":"2024-04-14T08:38:45.241020+0000","last_undegraded":"2024-04-14T08:38:45.241020+0000","last_fullsized":"2024-04-14T08:38:45.241020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:54:15.282969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211663+0000","last_change":"2024-04-14T08:38:44.211663+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211663+0000","last_undegraded":"2024-04-14T08:38:44.211663+0000","last_fullsized":"2024-04-14T08:38:44.211663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.224435+0000","last_change":"2024-04-14T08:38:44.221416+0000","last_active":"2024-04-14T08:38:44.224435+0000","last_peered":"2024-04-14T08:38:44.224435+0000","last_clean":"2024-04-14T08:38:44.224435+0000","last_became_active":"2024-04-14T08:38:44.221138+0000","last_became_peered":"2024-04-14T08:38:44.221138+0000","last_unstale":"2024-04-14T08:38:44.224435+0000","last_undegraded":"2024-04-14T08:38:44.224435+0000","last_fullsized":"2024-04-14T08:38:44.224435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:15.223452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.219407+0000","last_change":"2024-04-14T08:38:44.219407+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.219407+0000","last_undegraded":"2024-04-14T08:38:44.219407+0000","last_fullsized":"2024-04-14T08:38:44.219407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225070+0000","last_change":"2024-04-14T08:38:44.224441+0000","last_active":"2024-04-14T08:38:44.225070+0000","last_peered":"2024-04-14T08:38:44.225070+0000","last_clean":"2024-04-14T08:38:44.225070+0000","last_became_active":"2024-04-14T08:38:44.224137+0000","last_became_peered":"2024-04-14T08:38:44.224137+0000","last_unstale":"2024-04-14T08:38:44.225070+0000","last_undegraded":"2024-04-14T08:38:44.225070+0000","last_fullsized":"2024-04-14T08:38:44.225070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:36:44.980948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217106+0000","last_change":"2024-04-14T08:38:44.217106+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217106+0000","last_undegraded":"2024-04-14T08:38:44.217106+0000","last_fullsized":"2024-04-14T08:38:44.217106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226868+0000","last_change":"2024-04-14T08:38:44.220261+0000","last_active":"2024-04-14T08:38:44.226868+0000","last_peered":"2024-04-14T08:38:44.226868+0000","last_clean":"2024-04-14T08:38:44.226868+0000","last_became_active":"2024-04-14T08:38:44.220001+0000","last_became_peered":"2024-04-14T08:38:44.220001+0000","last_unstale":"2024-04-14T08:38:44.226868+0000","last_undegraded":"2024-04-14T08:38:44.226868+0000","last_fullsized":"2024-04-14T08:38:44.226868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:13:48.232427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215334+0000","last_change":"2024-04-14T08:38:44.215334+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215334+0000","last_undegraded":"2024-04-14T08:38:44.215334+0000","last_fullsized":"2024-04-14T08:38:44.215334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228198+0000","last_change":"2024-04-14T08:38:44.227400+0000","last_active":"2024-04-14T08:38:44.228198+0000","last_peered":"2024-04-14T08:38:44.228198+0000","last_clean":"2024-04-14T08:38:44.228198+0000","last_became_active":"2024-04-14T08:38:44.226963+0000","last_became_peered":"2024-04-14T08:38:44.226963+0000","last_unstale":"2024-04-14T08:38:44.228198+0000","last_undegraded":"2024-04-14T08:38:44.228198+0000","last_fullsized":"2024-04-14T08:38:44.228198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:56:30.743467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.221098+0000","last_change":"2024-04-14T08:38:44.221098+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.221098+0000","last_undegraded":"2024-04-14T08:38:44.221098+0000","last_fullsized":"2024-04-14T08:38:44.221098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226693+0000","last_change":"2024-04-14T08:38:44.221259+0000","last_active":"2024-04-14T08:38:44.226693+0000","last_peered":"2024-04-14T08:38:44.226693+0000","last_clean":"2024-04-14T08:38:44.226693+0000","last_became_active":"2024-04-14T08:38:44.220936+0000","last_became_peered":"2024-04-14T08:38:44.220936+0000","last_unstale":"2024-04-14T08:38:44.226693+0000","last_undegraded":"2024-04-14T08:38:44.226693+0000","last_fullsized":"2024-04-14T08:38:44.226693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:20:32.325057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215475+0000","last_change":"2024-04-14T08:38:44.215475+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215475+0000","last_undegraded":"2024-04-14T08:38:44.215475+0000","last_fullsized":"2024-04-14T08:38:44.215475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.240442+0000","last_change":"2024-04-14T08:38:44.222193+0000","last_active":"2024-04-14T08:38:45.240442+0000","last_peered":"2024-04-14T08:38:45.240442+0000","last_clean":"2024-04-14T08:38:45.240442+0000","last_became_active":"2024-04-14T08:38:44.221895+0000","last_became_peered":"2024-04-14T08:38:44.221895+0000","last_unstale":"2024-04-14T08:38:45.240442+0000","last_undegraded":"2024-04-14T08:38:45.240442+0000","last_fullsized":"2024-04-14T08:38:45.240442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:36:12.417551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.226478+0000","last_change":"2024-04-14T08:38:44.226478+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.226478+0000","last_undegraded":"2024-04-14T08:38:44.226478+0000","last_fullsized":"2024-04-14T08:38:44.226478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.222944+0000","last_change":"2024-04-14T08:38:44.220634+0000","last_active":"2024-04-14T08:38:44.222944+0000","last_peered":"2024-04-14T08:38:44.222944+0000","last_clean":"2024-04-14T08:38:44.222944+0000","last_became_active":"2024-04-14T08:38:44.220352+0000","last_became_peered":"2024-04-14T08:38:44.220352+0000","last_unstale":"2024-04-14T08:38:44.222944+0000","last_undegraded":"2024-04-14T08:38:44.222944+0000","last_fullsized":"2024-04-14T08:38:44.222944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:47:54.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218195+0000","last_change":"2024-04-14T08:38:44.218195+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218195+0000","last_undegraded":"2024-04-14T08:38:44.218195+0000","last_fullsized":"2024-04-14T08:38:44.218195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.232546+0000","last_change":"2024-04-14T08:38:44.230684+0000","last_active":"2024-04-14T08:38:44.232546+0000","last_peered":"2024-04-14T08:38:44.232546+0000","last_clean":"2024-04-14T08:38:44.232546+0000","last_became_active":"2024-04-14T08:38:44.230211+0000","last_became_peered":"2024-04-14T08:38:44.230211+0000","last_unstale":"2024-04-14T08:38:44.232546+0000","last_undegraded":"2024-04-14T08:38:44.232546+0000","last_fullsized":"2024-04-14T08:38:44.232546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:21:35.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216328+0000","last_change":"2024-04-14T08:38:44.216328+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216328+0000","last_undegraded":"2024-04-14T08:38:44.216328+0000","last_fullsized":"2024-04-14T08:38:44.216328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225413+0000","last_change":"2024-04-14T08:38:44.224050+0000","last_active":"2024-04-14T08:38:44.225413+0000","last_peered":"2024-04-14T08:38:44.225413+0000","last_clean":"2024-04-14T08:38:44.225413+0000","last_became_active":"2024-04-14T08:38:44.223667+0000","last_became_peered":"2024-04-14T08:38:44.223667+0000","last_unstale":"2024-04-14T08:38:44.225413+0000","last_undegraded":"2024-04-14T08:38:44.225413+0000","last_fullsized":"2024-04-14T08:38:44.225413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:07:20.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216642+0000","last_change":"2024-04-14T08:38:44.216642+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216642+0000","last_undegraded":"2024-04-14T08:38:44.216642+0000","last_fullsized":"2024-04-14T08:38:44.216642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233092+0000","last_change":"2024-04-14T08:38:44.230396+0000","last_active":"2024-04-14T08:38:44.233092+0000","last_peered":"2024-04-14T08:38:44.233092+0000","last_clean":"2024-04-14T08:38:44.233092+0000","last_became_active":"2024-04-14T08:38:44.230103+0000","last_became_peered":"2024-04-14T08:38:44.230103+0000","last_unstale":"2024-04-14T08:38:44.233092+0000","last_undegraded":"2024-04-14T08:38:44.233092+0000","last_fullsized":"2024-04-14T08:38:44.233092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:59:08.058016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.214047+0000","last_change":"2024-04-14T08:38:44.214047+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.214047+0000","last_undegraded":"2024-04-14T08:38:44.214047+0000","last_fullsized":"2024-04-14T08:38:44.214047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226116+0000","last_change":"2024-04-14T08:38:44.224830+0000","last_active":"2024-04-14T08:38:44.226116+0000","last_peered":"2024-04-14T08:38:44.226116+0000","last_clean":"2024-04-14T08:38:44.226116+0000","last_became_active":"2024-04-14T08:38:44.224558+0000","last_became_peered":"2024-04-14T08:38:44.224558+0000","last_unstale":"2024-04-14T08:38:44.226116+0000","last_undegraded":"2024-04-14T08:38:44.226116+0000","last_fullsized":"2024-04-14T08:38:44.226116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:35.503739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.214795+0000","last_change":"2024-04-14T08:38:44.214795+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.214795+0000","last_undegraded":"2024-04-14T08:38:44.214795+0000","last_fullsized":"2024-04-14T08:38:44.214795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225790+0000","last_change":"2024-04-14T08:38:44.224592+0000","last_active":"2024-04-14T08:38:44.225790+0000","last_peered":"2024-04-14T08:38:44.225790+0000","last_clean":"2024-04-14T08:38:44.225790+0000","last_became_active":"2024-04-14T08:38:44.224287+0000","last_became_peered":"2024-04-14T08:38:44.224287+0000","last_unstale":"2024-04-14T08:38:44.225790+0000","last_undegraded":"2024-04-14T08:38:44.225790+0000","last_fullsized":"2024-04-14T08:38:44.225790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:56:24.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.220700+0000","last_change":"2024-04-14T08:38:44.220700+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.220700+0000","last_undegraded":"2024-04-14T08:38:44.220700+0000","last_fullsized":"2024-04-14T08:38:44.220700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.214298+0000","last_change":"2024-04-14T08:38:44.219166+0000","last_active":"2024-04-14T08:38:45.214298+0000","last_peered":"2024-04-14T08:38:45.214298+0000","last_clean":"2024-04-14T08:38:45.214298+0000","last_became_active":"2024-04-14T08:38:44.218905+0000","last_became_peered":"2024-04-14T08:38:44.218905+0000","last_unstale":"2024-04-14T08:38:45.214298+0000","last_undegraded":"2024-04-14T08:38:45.214298+0000","last_fullsized":"2024-04-14T08:38:45.214298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:24:57.813701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.227888+0000","last_change":"2024-04-14T08:38:45.223615+0000","last_active":"2024-04-14T08:38:45.227888+0000","last_peered":"2024-04-14T08:38:45.227888+0000","last_clean":"2024-04-14T08:38:45.227888+0000","last_became_active":"2024-04-14T08:38:45.221967+0000","last_became_peered":"2024-04-14T08:38:45.221967+0000","last_unstale":"2024-04-14T08:38:45.227888+0000","last_undegraded":"2024-04-14T08:38:45.227888+0000","last_fullsized":"2024-04-14T08:38:45.227888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:09.277582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228434+0000","last_change":"2024-04-14T08:38:44.224166+0000","last_active":"2024-04-14T08:38:44.228434+0000","last_peered":"2024-04-14T08:38:44.228434+0000","last_clean":"2024-04-14T08:38:44.228434+0000","last_became_active":"2024-04-14T08:38:44.223244+0000","last_became_peered":"2024-04-14T08:38:44.223244+0000","last_unstale":"2024-04-14T08:38:44.228434+0000","last_undegraded":"2024-04-14T08:38:44.228434+0000","last_fullsized":"2024-04-14T08:38:44.228434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:42.226080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":301,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.214417+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:38:45.214417+0000","last_peered":"2024-04-14T08:38:45.214417+0000","last_clean":"2024-04-14T08:38:45.214417+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:38:45.214417+0000","last_undegraded":"2024-04-14T08:38:45.214417+0000","last_fullsized":"2024-04-14T08:38:45.214417+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1650688,"data_stored":1068945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":464687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1630208,"data_stored":1061722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:38:48.211 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:38:48.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.347+0000 7fc9a6daa640 1 Processor -- start 2024-04-14T08:38:48.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.347+0000 7fc9a6daa640 1 -- start start 2024-04-14T08:38:48.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.348+0000 7fc9a6daa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc9a00b21d0 0x7fc9a015c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.348+0000 7fc9a6daa640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9a00b3700 con 0x7fc9a00b1df0 2024-04-14T08:38:48.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.348+0000 7fc9a6daa640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9a00b39e0 con 0x7fc9a015c5a0 2024-04-14T08:38:48.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.348+0000 7fc9a6daa640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc9a00b3cc0 con 0x7fc9a00b21d0 2024-04-14T08:38:48.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.348+0000 7fc99ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc9a00b21d0 0x7fc9a015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.348+0000 7fc9a4b46640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc9a00b1df0 0x7fc9a00b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45752/0 (socket says 172.21.15.172:45752) 2024-04-14T08:38:48.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.348+0000 7fc9a4b46640 1 -- 172.21.15.172:0/3515230414 learned_addr learned my addr 172.21.15.172:0/3515230414 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:48.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.349+0000 7fc99ffff640 1 -- 172.21.15.172:0/3515230414 >> v1:172.21.15.177:6789/0 conn(0x7fc9a015c5a0 legacy=0x7fc9a00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.349+0000 7fc99ffff640 1 -- 172.21.15.172:0/3515230414 >> v1:172.21.15.172:6789/0 conn(0x7fc9a00b1df0 legacy=0x7fc9a00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.349+0000 7fc99ffff640 1 -- 172.21.15.172:0/3515230414 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a015bb40 con 0x7fc9a00b21d0 2024-04-14T08:38:48.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.349+0000 7fc99f7fe640 1 -- 172.21.15.172:0/3515230414 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 362985952 0 0) 0x7fc9a00b3700 con 0x7fc9a00b1df0 2024-04-14T08:38:48.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.349+0000 7fc99ffff640 1 --2- 172.21.15.172:0/3515230414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a00b21d0 0x7fc9a015c060 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0097a0 tx=0x7fc98c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a2fa54e738a313 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:48.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.349+0000 7fc99f7fe640 1 -- 172.21.15.172:0/3515230414 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c034070 con 0x7fc9a00b21d0 2024-04-14T08:38:48.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.349+0000 7fc99f7fe640 1 -- 172.21.15.172:0/3515230414 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c030610 con 0x7fc9a00b21d0 2024-04-14T08:38:48.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.349+0000 7fc99f7fe640 1 -- 172.21.15.172:0/3515230414 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0309e0 con 0x7fc9a00b21d0 2024-04-14T08:38:48.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.350+0000 7fc9a6daa640 1 -- 172.21.15.172:0/3515230414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a00b21d0 msgr2=0x7fc9a015c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.350+0000 7fc9a6daa640 1 --2- 172.21.15.172:0/3515230414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a00b21d0 0x7fc9a015c060 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0097a0 tx=0x7fc98c02b960 comp rx=0 tx=0).stop 2024-04-14T08:38:48.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.350+0000 7fc9a6daa640 1 -- 172.21.15.172:0/3515230414 shutdown_connections 2024-04-14T08:38:48.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.350+0000 7fc9a6daa640 1 --2- 172.21.15.172:0/3515230414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a00b21d0 0x7fc9a015c060 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.350+0000 7fc9a6daa640 1 -- 172.21.15.172:0/3515230414 >> 172.21.15.172:0/3515230414 conn(0x7fc9a006ec30 msgr2=0x7fc9a006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:48.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.350+0000 7fc9a6daa640 1 -- 172.21.15.172:0/3515230414 shutdown_connections 2024-04-14T08:38:48.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.350+0000 7fc9a6daa640 1 -- 172.21.15.172:0/3515230414 wait complete. 2024-04-14T08:38:48.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.351+0000 7fc9a6daa640 1 Processor -- start 2024-04-14T08:38:48.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.351+0000 7fc9a6daa640 1 -- start start 2024-04-14T08:38:48.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc9a6daa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9a00b1df0 0x7fc9a0134000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc9a6daa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a00b21d0 0x7fc9a00a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc9a6daa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9a00aa390 0x7fc9a015dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc9a6daa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a00b48d0 con 0x7fc9a00b21d0 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc9a6daa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a00b4750 con 0x7fc9a00aa390 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc9a6daa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc9a0134540 con 0x7fc9a00b1df0 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc99ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a00b21d0 0x7fc9a00a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc99ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a00b21d0 0x7fc9a00a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49812/0 (socket says 172.21.15.172:49812) 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc99ffff640 1 -- 172.21.15.172:0/2633180476 learned_addr learned my addr 172.21.15.172:0/2633180476 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc9a4b46640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9a00b1df0 0x7fc9a0134000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc9a5347640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9a00aa390 0x7fc9a015dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc99ffff640 1 -- 172.21.15.172:0/2633180476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9a00b1df0 msgr2=0x7fc9a0134000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc99ffff640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9a00b1df0 0x7fc9a0134000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc99ffff640 1 -- 172.21.15.172:0/2633180476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9a00aa390 msgr2=0x7fc9a015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc99ffff640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9a00aa390 0x7fc9a015dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.352+0000 7fc99ffff640 1 -- 172.21.15.172:0/2633180476 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a015cce0 con 0x7fc9a00b21d0 2024-04-14T08:38:48.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.353+0000 7fc99ffff640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a00b21d0 0x7fc9a00a9e50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fc98c000c00 tx=0x7fc98c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:48.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.353+0000 7fc99d7fa640 1 -- 172.21.15.172:0/2633180476 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c034040 con 0x7fc9a00b21d0 2024-04-14T08:38:48.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.353+0000 7fc99d7fa640 1 -- 172.21.15.172:0/2633180476 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c004030 con 0x7fc9a00b21d0 2024-04-14T08:38:48.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.353+0000 7fc99d7fa640 1 -- 172.21.15.172:0/2633180476 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c004420 con 0x7fc9a00b21d0 2024-04-14T08:38:48.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.353+0000 7fc9a6daa640 1 -- 172.21.15.172:0/2633180476 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a015cfc0 con 0x7fc9a00b21d0 2024-04-14T08:38:48.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.353+0000 7fc9a6daa640 1 -- 172.21.15.172:0/2633180476 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a015d580 con 0x7fc9a00b21d0 2024-04-14T08:38:48.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.354+0000 7fc9a6daa640 1 -- 172.21.15.172:0/2633180476 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a0068520 con 0x7fc9a00b21d0 2024-04-14T08:38:48.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.354+0000 7fc99d7fa640 1 -- 172.21.15.172:0/2633180476 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc98c037070 con 0x7fc9a00b21d0 2024-04-14T08:38:48.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.355+0000 7fc99d7fa640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc98406c8f0 0x7fc98406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.355+0000 7fc99d7fa640 1 -- 172.21.15.172:0/2633180476 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fc98c0ad840 con 0x7fc9a00b21d0 2024-04-14T08:38:48.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.355+0000 7fc9a4b46640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc98406c8f0 0x7fc98406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.355+0000 7fc9a4b46640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc98406c8f0 0x7fc98406edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc994007f70 tx=0x7fc994007f00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:48.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.359+0000 7fc99d7fa640 1 -- 172.21.15.172:0/2633180476 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc98c07b640 con 0x7fc9a00b21d0 2024-04-14T08:38:48.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.531+0000 7fc9a6daa640 1 -- 172.21.15.172:0/2633180476 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9a0158d30 con 0x7fc98406c8f0 2024-04-14T08:38:48.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.537+0000 7fc99d7fa640 1 -- 172.21.15.172:0/2633180476 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247193 (secure 0 0 0) 0x7fc9a0158d30 con 0x7fc98406c8f0 2024-04-14T08:38:48.541 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:48.542 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:38:48.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.540+0000 7fc9a6daa640 1 -- 172.21.15.172:0/2633180476 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc98406c8f0 msgr2=0x7fc98406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.540+0000 7fc9a6daa640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc98406c8f0 0x7fc98406edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc994007f70 tx=0x7fc994007f00 comp rx=0 tx=0).stop 2024-04-14T08:38:48.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.540+0000 7fc9a6daa640 1 -- 172.21.15.172:0/2633180476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a00b21d0 msgr2=0x7fc9a00a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.540+0000 7fc9a6daa640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a00b21d0 0x7fc9a00a9e50 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fc98c000c00 tx=0x7fc98c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.540+0000 7fc9a6daa640 1 -- 172.21.15.172:0/2633180476 shutdown_connections 2024-04-14T08:38:48.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.540+0000 7fc9a6daa640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc98406c8f0 0x7fc98406edb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.540+0000 7fc9a6daa640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9a00aa390 0x7fc9a015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.540+0000 7fc9a6daa640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a00b21d0 0x7fc9a00a9e50 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.540+0000 7fc9a6daa640 1 --2- 172.21.15.172:0/2633180476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9a00b1df0 0x7fc9a0134000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.540+0000 7fc9a6daa640 1 -- 172.21.15.172:0/2633180476 >> 172.21.15.172:0/2633180476 conn(0x7fc9a006ec30 msgr2=0x7fc9a01447f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:48.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.540+0000 7fc9a6daa640 1 -- 172.21.15.172:0/2633180476 shutdown_connections 2024-04-14T08:38:48.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.540+0000 7fc9a6daa640 1 -- 172.21.15.172:0/2633180476 wait complete. 2024-04-14T08:38:48.552 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-04-14T08:38:47.191220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590402,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246256,"kb_used_data":9264,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754728464,"statfs":{"total":773094113280,"available":772841947136,"internally_reserved":0,"allocated":9486336,"data_stored":4952124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001105"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216414+0000","last_change":"2024-04-14T08:38:44.216414+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216414+0000","last_undegraded":"2024-04-14T08:38:44.216414+0000","last_fullsized":"2024-04-14T08:38:44.216414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217436+0000","last_change":"2024-04-14T08:38:44.217436+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217436+0000","last_undegraded":"2024-04-14T08:38:44.217436+0000","last_fullsized":"2024-04-14T08:38:44.217436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215977+0000","last_change":"2024-04-14T08:38:44.215977+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215977+0000","last_undegraded":"2024-04-14T08:38:44.215977+0000","last_fullsized":"2024-04-14T08:38:44.215977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.220133+0000","last_change":"2024-04-14T08:38:44.220133+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.220133+0000","last_undegraded":"2024-04-14T08:38:44.220133+0000","last_fullsized":"2024-04-14T08:38:44.220133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211723+0000","last_change":"2024-04-14T08:38:44.211723+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211723+0000","last_undegraded":"2024-04-14T08:38:44.211723+0000","last_fullsized":"2024-04-14T08:38:44.211723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.228102+0000","last_change":"2024-04-14T08:38:44.228102+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.228102+0000","last_undegraded":"2024-04-14T08:38:44.228102+0000","last_fullsized":"2024-04-14T08:38:44.228102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.226673+0000","last_change":"2024-04-14T08:38:45.221574+0000","last_active":"2024-04-14T08:38:45.226673+0000","last_peered":"2024-04-14T08:38:45.226673+0000","last_clean":"2024-04-14T08:38:45.226673+0000","last_became_active":"2024-04-14T08:38:45.221303+0000","last_became_peered":"2024-04-14T08:38:45.221303+0000","last_unstale":"2024-04-14T08:38:45.226673+0000","last_undegraded":"2024-04-14T08:38:45.226673+0000","last_fullsized":"2024-04-14T08:38:45.226673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:59:38.604514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.219635+0000","last_change":"2024-04-14T08:38:44.219635+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.219635+0000","last_undegraded":"2024-04-14T08:38:44.219635+0000","last_fullsized":"2024-04-14T08:38:44.219635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.212588+0000","last_change":"2024-04-14T08:38:44.212588+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.212588+0000","last_undegraded":"2024-04-14T08:38:44.212588+0000","last_fullsized":"2024-04-14T08:38:44.212588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216941+0000","last_change":"2024-04-14T08:38:44.216941+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216941+0000","last_undegraded":"2024-04-14T08:38:44.216941+0000","last_fullsized":"2024-04-14T08:38:44.216941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211782+0000","last_change":"2024-04-14T08:38:44.211782+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211782+0000","last_undegraded":"2024-04-14T08:38:44.211782+0000","last_fullsized":"2024-04-14T08:38:44.211782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216683+0000","last_change":"2024-04-14T08:38:44.216683+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216683+0000","last_undegraded":"2024-04-14T08:38:44.216683+0000","last_fullsized":"2024-04-14T08:38:44.216683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.220232+0000","last_change":"2024-04-14T08:38:44.220232+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.220232+0000","last_undegraded":"2024-04-14T08:38:44.220232+0000","last_fullsized":"2024-04-14T08:38:44.220232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.213829+0000","last_change":"2024-04-14T08:38:44.213829+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.213829+0000","last_undegraded":"2024-04-14T08:38:44.213829+0000","last_fullsized":"2024-04-14T08:38:44.213829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.219502+0000","last_change":"2024-04-14T08:38:44.219502+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.219502+0000","last_undegraded":"2024-04-14T08:38:44.219502+0000","last_fullsized":"2024-04-14T08:38:44.219502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.228755+0000","last_change":"2024-04-14T08:38:44.228755+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.228755+0000","last_undegraded":"2024-04-14T08:38:44.228755+0000","last_fullsized":"2024-04-14T08:38:44.228755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.227961+0000","last_change":"2024-04-14T08:38:44.227961+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.227961+0000","last_undegraded":"2024-04-14T08:38:44.227961+0000","last_fullsized":"2024-04-14T08:38:44.227961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215464+0000","last_change":"2024-04-14T08:38:44.215464+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215464+0000","last_undegraded":"2024-04-14T08:38:44.215464+0000","last_fullsized":"2024-04-14T08:38:44.215464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211266+0000","last_change":"2024-04-14T08:38:44.211266+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211266+0000","last_undegraded":"2024-04-14T08:38:44.211266+0000","last_fullsized":"2024-04-14T08:38:44.211266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218880+0000","last_change":"2024-04-14T08:38:44.218880+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218880+0000","last_undegraded":"2024-04-14T08:38:44.218880+0000","last_fullsized":"2024-04-14T08:38:44.218880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.212313+0000","last_change":"2024-04-14T08:38:44.212313+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.212313+0000","last_undegraded":"2024-04-14T08:38:44.212313+0000","last_fullsized":"2024-04-14T08:38:44.212313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.213369+0000","last_change":"2024-04-14T08:38:44.213369+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.213369+0000","last_undegraded":"2024-04-14T08:38:44.213369+0000","last_fullsized":"2024-04-14T08:38:44.213369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218241+0000","last_change":"2024-04-14T08:38:44.218241+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218241+0000","last_undegraded":"2024-04-14T08:38:44.218241+0000","last_fullsized":"2024-04-14T08:38:44.218241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215033+0000","last_change":"2024-04-14T08:38:44.215033+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215033+0000","last_undegraded":"2024-04-14T08:38:44.215033+0000","last_fullsized":"2024-04-14T08:38:44.215033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216786+0000","last_change":"2024-04-14T08:38:44.216786+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216786+0000","last_undegraded":"2024-04-14T08:38:44.216786+0000","last_fullsized":"2024-04-14T08:38:44.216786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.213997+0000","last_change":"2024-04-14T08:38:44.213997+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.213997+0000","last_undegraded":"2024-04-14T08:38:44.213997+0000","last_fullsized":"2024-04-14T08:38:44.213997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216757+0000","last_change":"2024-04-14T08:38:44.216757+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216757+0000","last_undegraded":"2024-04-14T08:38:44.216757+0000","last_fullsized":"2024-04-14T08:38:44.216757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218369+0000","last_change":"2024-04-14T08:38:44.218369+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218369+0000","last_undegraded":"2024-04-14T08:38:44.218369+0000","last_fullsized":"2024-04-14T08:38:44.218369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.229290+0000","last_change":"2024-04-14T08:38:45.226642+0000","last_active":"2024-04-14T08:38:45.229290+0000","last_peered":"2024-04-14T08:38:45.229290+0000","last_clean":"2024-04-14T08:38:45.229290+0000","last_became_active":"2024-04-14T08:38:45.226349+0000","last_became_peered":"2024-04-14T08:38:45.226349+0000","last_unstale":"2024-04-14T08:38:45.229290+0000","last_undegraded":"2024-04-14T08:38:45.229290+0000","last_fullsized":"2024-04-14T08:38:45.229290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:17:06.164898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211114+0000","last_change":"2024-04-14T08:38:44.211114+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211114+0000","last_undegraded":"2024-04-14T08:38:44.211114+0000","last_fullsized":"2024-04-14T08:38:44.211114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217550+0000","last_change":"2024-04-14T08:38:44.217550+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217550+0000","last_undegraded":"2024-04-14T08:38:44.217550+0000","last_fullsized":"2024-04-14T08:38:44.217550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.228329+0000","last_change":"2024-04-14T08:38:44.228329+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.228329+0000","last_undegraded":"2024-04-14T08:38:44.228329+0000","last_fullsized":"2024-04-14T08:38:44.228329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.225245+0000","last_change":"2024-04-14T08:38:44.225245+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.225245+0000","last_undegraded":"2024-04-14T08:38:44.225245+0000","last_fullsized":"2024-04-14T08:38:44.225245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228182+0000","last_change":"2024-04-14T08:38:44.227154+0000","last_active":"2024-04-14T08:38:44.228182+0000","last_peered":"2024-04-14T08:38:44.228182+0000","last_clean":"2024-04-14T08:38:44.228182+0000","last_became_active":"2024-04-14T08:38:44.226910+0000","last_became_peered":"2024-04-14T08:38:44.226910+0000","last_unstale":"2024-04-14T08:38:44.228182+0000","last_undegraded":"2024-04-14T08:38:44.228182+0000","last_fullsized":"2024-04-14T08:38:44.228182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:49:42.403181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.227247+0000","last_change":"2024-04-14T08:38:44.227247+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.227247+0000","last_undegraded":"2024-04-14T08:38:44.227247+0000","last_fullsized":"2024-04-14T08:38:44.227247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225223+0000","last_change":"2024-04-14T08:38:44.222545+0000","last_active":"2024-04-14T08:38:44.225223+0000","last_peered":"2024-04-14T08:38:44.225223+0000","last_clean":"2024-04-14T08:38:44.225223+0000","last_became_active":"2024-04-14T08:38:44.221999+0000","last_became_peered":"2024-04-14T08:38:44.221999+0000","last_unstale":"2024-04-14T08:38:44.225223+0000","last_undegraded":"2024-04-14T08:38:44.225223+0000","last_fullsized":"2024-04-14T08:38:44.225223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:37:59.212931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.235826+0000","last_change":"2024-04-14T08:38:45.228983+0000","last_active":"2024-04-14T08:38:45.235826+0000","last_peered":"2024-04-14T08:38:45.235826+0000","last_clean":"2024-04-14T08:38:45.235826+0000","last_became_active":"2024-04-14T08:38:45.228632+0000","last_became_peered":"2024-04-14T08:38:45.228632+0000","last_unstale":"2024-04-14T08:38:45.235826+0000","last_undegraded":"2024-04-14T08:38:45.235826+0000","last_fullsized":"2024-04-14T08:38:45.235826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:17:19.947504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.240108+0000","last_change":"2024-04-14T08:38:44.214063+0000","last_active":"2024-04-14T08:38:45.240108+0000","last_peered":"2024-04-14T08:38:45.240108+0000","last_clean":"2024-04-14T08:38:45.240108+0000","last_became_active":"2024-04-14T08:38:44.213612+0000","last_became_peered":"2024-04-14T08:38:44.213612+0000","last_unstale":"2024-04-14T08:38:45.240108+0000","last_undegraded":"2024-04-14T08:38:45.240108+0000","last_fullsized":"2024-04-14T08:38:45.240108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:15:04.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.227051+0000","last_change":"2024-04-14T08:38:44.227051+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.227051+0000","last_undegraded":"2024-04-14T08:38:44.227051+0000","last_fullsized":"2024-04-14T08:38:44.227051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228169+0000","last_change":"2024-04-14T08:38:44.227021+0000","last_active":"2024-04-14T08:38:44.228169+0000","last_peered":"2024-04-14T08:38:44.228169+0000","last_clean":"2024-04-14T08:38:44.228169+0000","last_became_active":"2024-04-14T08:38:44.226731+0000","last_became_peered":"2024-04-14T08:38:44.226731+0000","last_unstale":"2024-04-14T08:38:44.228169+0000","last_undegraded":"2024-04-14T08:38:44.228169+0000","last_fullsized":"2024-04-14T08:38:44.228169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:01.799418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217415+0000","last_change":"2024-04-14T08:38:44.217415+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217415+0000","last_undegraded":"2024-04-14T08:38:44.217415+0000","last_fullsized":"2024-04-14T08:38:44.217415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225249+0000","last_change":"2024-04-14T08:38:44.222167+0000","last_active":"2024-04-14T08:38:44.225249+0000","last_peered":"2024-04-14T08:38:44.225249+0000","last_clean":"2024-04-14T08:38:44.225249+0000","last_became_active":"2024-04-14T08:38:44.221892+0000","last_became_peered":"2024-04-14T08:38:44.221892+0000","last_unstale":"2024-04-14T08:38:44.225249+0000","last_undegraded":"2024-04-14T08:38:44.225249+0000","last_fullsized":"2024-04-14T08:38:44.225249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:13.870437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217818+0000","last_change":"2024-04-14T08:38:44.217818+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217818+0000","last_undegraded":"2024-04-14T08:38:44.217818+0000","last_fullsized":"2024-04-14T08:38:44.217818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226141+0000","last_change":"2024-04-14T08:38:44.222906+0000","last_active":"2024-04-14T08:38:44.226141+0000","last_peered":"2024-04-14T08:38:44.226141+0000","last_clean":"2024-04-14T08:38:44.226141+0000","last_became_active":"2024-04-14T08:38:44.222541+0000","last_became_peered":"2024-04-14T08:38:44.222541+0000","last_unstale":"2024-04-14T08:38:44.226141+0000","last_undegraded":"2024-04-14T08:38:44.226141+0000","last_fullsized":"2024-04-14T08:38:44.226141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:21:35.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.221823+0000","last_change":"2024-04-14T08:38:44.221823+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.221823+0000","last_undegraded":"2024-04-14T08:38:44.221823+0000","last_fullsized":"2024-04-14T08:38:44.221823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.223396+0000","last_change":"2024-04-14T08:38:44.218254+0000","last_active":"2024-04-14T08:38:44.223396+0000","last_peered":"2024-04-14T08:38:44.223396+0000","last_clean":"2024-04-14T08:38:44.223396+0000","last_became_active":"2024-04-14T08:38:44.217969+0000","last_became_peered":"2024-04-14T08:38:44.217969+0000","last_unstale":"2024-04-14T08:38:44.223396+0000","last_undegraded":"2024-04-14T08:38:44.223396+0000","last_fullsized":"2024-04-14T08:38:44.223396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:12:14.185232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217899+0000","last_change":"2024-04-14T08:38:44.217899+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217899+0000","last_undegraded":"2024-04-14T08:38:44.217899+0000","last_fullsized":"2024-04-14T08:38:44.217899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.214121+0000","last_change":"2024-04-14T08:38:44.231501+0000","last_active":"2024-04-14T08:38:45.214121+0000","last_peered":"2024-04-14T08:38:45.214121+0000","last_clean":"2024-04-14T08:38:45.214121+0000","last_became_active":"2024-04-14T08:38:44.231184+0000","last_became_peered":"2024-04-14T08:38:44.231184+0000","last_unstale":"2024-04-14T08:38:45.214121+0000","last_undegraded":"2024-04-14T08:38:45.214121+0000","last_fullsized":"2024-04-14T08:38:45.214121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:59:08.058016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.212060+0000","last_change":"2024-04-14T08:38:44.212060+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.212060+0000","last_undegraded":"2024-04-14T08:38:44.212060+0000","last_fullsized":"2024-04-14T08:38:44.212060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225545+0000","last_change":"2024-04-14T08:38:44.224713+0000","last_active":"2024-04-14T08:38:44.225545+0000","last_peered":"2024-04-14T08:38:44.225545+0000","last_clean":"2024-04-14T08:38:44.225545+0000","last_became_active":"2024-04-14T08:38:44.224437+0000","last_became_peered":"2024-04-14T08:38:44.224437+0000","last_unstale":"2024-04-14T08:38:44.225545+0000","last_undegraded":"2024-04-14T08:38:44.225545+0000","last_fullsized":"2024-04-14T08:38:44.225545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:37.147141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.221898+0000","last_change":"2024-04-14T08:38:45.219595+0000","last_active":"2024-04-14T08:38:45.221898+0000","last_peered":"2024-04-14T08:38:45.221898+0000","last_clean":"2024-04-14T08:38:45.221898+0000","last_became_active":"2024-04-14T08:38:45.219269+0000","last_became_peered":"2024-04-14T08:38:45.219269+0000","last_unstale":"2024-04-14T08:38:45.221898+0000","last_undegraded":"2024-04-14T08:38:45.221898+0000","last_fullsized":"2024-04-14T08:38:45.221898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:48:28.586545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.224564+0000","last_change":"2024-04-14T08:38:44.219411+0000","last_active":"2024-04-14T08:38:44.224564+0000","last_peered":"2024-04-14T08:38:44.224564+0000","last_clean":"2024-04-14T08:38:44.224564+0000","last_became_active":"2024-04-14T08:38:44.219130+0000","last_became_peered":"2024-04-14T08:38:44.219130+0000","last_unstale":"2024-04-14T08:38:44.224564+0000","last_undegraded":"2024-04-14T08:38:44.224564+0000","last_fullsized":"2024-04-14T08:38:44.224564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:43:02.184142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.214550+0000","last_change":"2024-04-14T08:38:44.214550+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.214550+0000","last_undegraded":"2024-04-14T08:38:44.214550+0000","last_fullsized":"2024-04-14T08:38:44.214550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233151+0000","last_change":"2024-04-14T08:38:44.231600+0000","last_active":"2024-04-14T08:38:44.233151+0000","last_peered":"2024-04-14T08:38:44.233151+0000","last_clean":"2024-04-14T08:38:44.233151+0000","last_became_active":"2024-04-14T08:38:44.231276+0000","last_became_peered":"2024-04-14T08:38:44.231276+0000","last_unstale":"2024-04-14T08:38:44.233151+0000","last_undegraded":"2024-04-14T08:38:44.233151+0000","last_fullsized":"2024-04-14T08:38:44.233151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:42.226080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216192+0000","last_change":"2024-04-14T08:38:44.216192+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216192+0000","last_undegraded":"2024-04-14T08:38:44.216192+0000","last_fullsized":"2024-04-14T08:38:44.216192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233087+0000","last_change":"2024-04-14T08:38:44.230303+0000","last_active":"2024-04-14T08:38:44.233087+0000","last_peered":"2024-04-14T08:38:44.233087+0000","last_clean":"2024-04-14T08:38:44.233087+0000","last_became_active":"2024-04-14T08:38:44.230051+0000","last_became_peered":"2024-04-14T08:38:44.230051+0000","last_unstale":"2024-04-14T08:38:44.233087+0000","last_undegraded":"2024-04-14T08:38:44.233087+0000","last_fullsized":"2024-04-14T08:38:44.233087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:36:12.417551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216476+0000","last_change":"2024-04-14T08:38:44.216476+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216476+0000","last_undegraded":"2024-04-14T08:38:44.216476+0000","last_fullsized":"2024-04-14T08:38:44.216476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.245313+0000","last_change":"2024-04-14T08:38:44.230951+0000","last_active":"2024-04-14T08:38:45.245313+0000","last_peered":"2024-04-14T08:38:45.245313+0000","last_clean":"2024-04-14T08:38:45.245313+0000","last_became_active":"2024-04-14T08:38:44.230499+0000","last_became_peered":"2024-04-14T08:38:44.230499+0000","last_unstale":"2024-04-14T08:38:45.245313+0000","last_undegraded":"2024-04-14T08:38:45.245313+0000","last_fullsized":"2024-04-14T08:38:45.245313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:55:56.125870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.210337+0000","last_change":"2024-04-14T08:38:44.210337+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.210337+0000","last_undegraded":"2024-04-14T08:38:44.210337+0000","last_fullsized":"2024-04-14T08:38:44.210337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233084+0000","last_change":"2024-04-14T08:38:44.230300+0000","last_active":"2024-04-14T08:38:44.233084+0000","last_peered":"2024-04-14T08:38:44.233084+0000","last_clean":"2024-04-14T08:38:44.233084+0000","last_became_active":"2024-04-14T08:38:44.230024+0000","last_became_peered":"2024-04-14T08:38:44.230024+0000","last_unstale":"2024-04-14T08:38:44.233084+0000","last_undegraded":"2024-04-14T08:38:44.233084+0000","last_fullsized":"2024-04-14T08:38:44.233084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:15:04.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216501+0000","last_change":"2024-04-14T08:38:44.216501+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216501+0000","last_undegraded":"2024-04-14T08:38:44.216501+0000","last_fullsized":"2024-04-14T08:38:44.216501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.227819+0000","last_change":"2024-04-14T08:38:44.224811+0000","last_active":"2024-04-14T08:38:44.227819+0000","last_peered":"2024-04-14T08:38:44.227819+0000","last_clean":"2024-04-14T08:38:44.227819+0000","last_became_active":"2024-04-14T08:38:44.224541+0000","last_became_peered":"2024-04-14T08:38:44.224541+0000","last_unstale":"2024-04-14T08:38:44.227819+0000","last_undegraded":"2024-04-14T08:38:44.227819+0000","last_fullsized":"2024-04-14T08:38:44.227819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:31:11.641961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217216+0000","last_change":"2024-04-14T08:38:44.217216+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217216+0000","last_undegraded":"2024-04-14T08:38:44.217216+0000","last_fullsized":"2024-04-14T08:38:44.217216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.217797+0000","last_change":"2024-04-14T08:38:44.216083+0000","last_active":"2024-04-14T08:38:44.217797+0000","last_peered":"2024-04-14T08:38:44.217797+0000","last_clean":"2024-04-14T08:38:44.217797+0000","last_became_active":"2024-04-14T08:38:44.215754+0000","last_became_peered":"2024-04-14T08:38:44.215754+0000","last_unstale":"2024-04-14T08:38:44.217797+0000","last_undegraded":"2024-04-14T08:38:44.217797+0000","last_fullsized":"2024-04-14T08:38:44.217797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:13.870437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.222718+0000","last_change":"2024-04-14T08:38:44.222718+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.222718+0000","last_undegraded":"2024-04-14T08:38:44.222718+0000","last_fullsized":"2024-04-14T08:38:44.222718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.223232+0000","last_change":"2024-04-14T08:38:44.222002+0000","last_active":"2024-04-14T08:38:44.223232+0000","last_peered":"2024-04-14T08:38:44.223232+0000","last_clean":"2024-04-14T08:38:44.223232+0000","last_became_active":"2024-04-14T08:38:44.221679+0000","last_became_peered":"2024-04-14T08:38:44.221679+0000","last_unstale":"2024-04-14T08:38:44.223232+0000","last_undegraded":"2024-04-14T08:38:44.223232+0000","last_fullsized":"2024-04-14T08:38:44.223232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:53.649370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215754+0000","last_change":"2024-04-14T08:38:44.215754+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215754+0000","last_undegraded":"2024-04-14T08:38:44.215754+0000","last_fullsized":"2024-04-14T08:38:44.215754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.241020+0000","last_change":"2024-04-14T08:38:44.220058+0000","last_active":"2024-04-14T08:38:45.241020+0000","last_peered":"2024-04-14T08:38:45.241020+0000","last_clean":"2024-04-14T08:38:45.241020+0000","last_became_active":"2024-04-14T08:38:44.219721+0000","last_became_peered":"2024-04-14T08:38:44.219721+0000","last_unstale":"2024-04-14T08:38:45.241020+0000","last_undegraded":"2024-04-14T08:38:45.241020+0000","last_fullsized":"2024-04-14T08:38:45.241020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:54:15.282969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211663+0000","last_change":"2024-04-14T08:38:44.211663+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211663+0000","last_undegraded":"2024-04-14T08:38:44.211663+0000","last_fullsized":"2024-04-14T08:38:44.211663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.224435+0000","last_change":"2024-04-14T08:38:44.221416+0000","last_active":"2024-04-14T08:38:44.224435+0000","last_peered":"2024-04-14T08:38:44.224435+0000","last_clean":"2024-04-14T08:38:44.224435+0000","last_became_active":"2024-04-14T08:38:44.221138+0000","last_became_peered":"2024-04-14T08:38:44.221138+0000","last_unstale":"2024-04-14T08:38:44.224435+0000","last_undegraded":"2024-04-14T08:38:44.224435+0000","last_fullsized":"2024-04-14T08:38:44.224435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:15.223452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.219407+0000","last_change":"2024-04-14T08:38:44.219407+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.219407+0000","last_undegraded":"2024-04-14T08:38:44.219407+0000","last_fullsized":"2024-04-14T08:38:44.219407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225070+0000","last_change":"2024-04-14T08:38:44.224441+0000","last_active":"2024-04-14T08:38:44.225070+0000","last_peered":"2024-04-14T08:38:44.225070+0000","last_clean":"2024-04-14T08:38:44.225070+0000","last_became_active":"2024-04-14T08:38:44.224137+0000","last_became_peered":"2024-04-14T08:38:44.224137+0000","last_unstale":"2024-04-14T08:38:44.225070+0000","last_undegraded":"2024-04-14T08:38:44.225070+0000","last_fullsized":"2024-04-14T08:38:44.225070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:36:44.980948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217106+0000","last_change":"2024-04-14T08:38:44.217106+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217106+0000","last_undegraded":"2024-04-14T08:38:44.217106+0000","last_fullsized":"2024-04-14T08:38:44.217106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226868+0000","last_change":"2024-04-14T08:38:44.220261+0000","last_active":"2024-04-14T08:38:44.226868+0000","last_peered":"2024-04-14T08:38:44.226868+0000","last_clean":"2024-04-14T08:38:44.226868+0000","last_became_active":"2024-04-14T08:38:44.220001+0000","last_became_peered":"2024-04-14T08:38:44.220001+0000","last_unstale":"2024-04-14T08:38:44.226868+0000","last_undegraded":"2024-04-14T08:38:44.226868+0000","last_fullsized":"2024-04-14T08:38:44.226868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:13:48.232427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215334+0000","last_change":"2024-04-14T08:38:44.215334+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215334+0000","last_undegraded":"2024-04-14T08:38:44.215334+0000","last_fullsized":"2024-04-14T08:38:44.215334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228198+0000","last_change":"2024-04-14T08:38:44.227400+0000","last_active":"2024-04-14T08:38:44.228198+0000","last_peered":"2024-04-14T08:38:44.228198+0000","last_clean":"2024-04-14T08:38:44.228198+0000","last_became_active":"2024-04-14T08:38:44.226963+0000","last_became_peered":"2024-04-14T08:38:44.226963+0000","last_unstale":"2024-04-14T08:38:44.228198+0000","last_undegraded":"2024-04-14T08:38:44.228198+0000","last_fullsized":"2024-04-14T08:38:44.228198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:56:30.743467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.221098+0000","last_change":"2024-04-14T08:38:44.221098+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.221098+0000","last_undegraded":"2024-04-14T08:38:44.221098+0000","last_fullsized":"2024-04-14T08:38:44.221098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226693+0000","last_change":"2024-04-14T08:38:44.221259+0000","last_active":"2024-04-14T08:38:44.226693+0000","last_peered":"2024-04-14T08:38:44.226693+0000","last_clean":"2024-04-14T08:38:44.226693+0000","last_became_active":"2024-04-14T08:38:44.220936+0000","last_became_peered":"2024-04-14T08:38:44.220936+0000","last_unstale":"2024-04-14T08:38:44.226693+0000","last_undegraded":"2024-04-14T08:38:44.226693+0000","last_fullsized":"2024-04-14T08:38:44.226693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:20:32.325057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215475+0000","last_change":"2024-04-14T08:38:44.215475+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215475+0000","last_undegraded":"2024-04-14T08:38:44.215475+0000","last_fullsized":"2024-04-14T08:38:44.215475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.240442+0000","last_change":"2024-04-14T08:38:44.222193+0000","last_active":"2024-04-14T08:38:45.240442+0000","last_peered":"2024-04-14T08:38:45.240442+0000","last_clean":"2024-04-14T08:38:45.240442+0000","last_became_active":"2024-04-14T08:38:44.221895+0000","last_became_peered":"2024-04-14T08:38:44.221895+0000","last_unstale":"2024-04-14T08:38:45.240442+0000","last_undegraded":"2024-04-14T08:38:45.240442+0000","last_fullsized":"2024-04-14T08:38:45.240442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:36:12.417551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.226478+0000","last_change":"2024-04-14T08:38:44.226478+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.226478+0000","last_undegraded":"2024-04-14T08:38:44.226478+0000","last_fullsized":"2024-04-14T08:38:44.226478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.222944+0000","last_change":"2024-04-14T08:38:44.220634+0000","last_active":"2024-04-14T08:38:44.222944+0000","last_peered":"2024-04-14T08:38:44.222944+0000","last_clean":"2024-04-14T08:38:44.222944+0000","last_became_active":"2024-04-14T08:38:44.220352+0000","last_became_peered":"2024-04-14T08:38:44.220352+0000","last_unstale":"2024-04-14T08:38:44.222944+0000","last_undegraded":"2024-04-14T08:38:44.222944+0000","last_fullsized":"2024-04-14T08:38:44.222944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:47:54.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218195+0000","last_change":"2024-04-14T08:38:44.218195+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218195+0000","last_undegraded":"2024-04-14T08:38:44.218195+0000","last_fullsized":"2024-04-14T08:38:44.218195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.232546+0000","last_change":"2024-04-14T08:38:44.230684+0000","last_active":"2024-04-14T08:38:44.232546+0000","last_peered":"2024-04-14T08:38:44.232546+0000","last_clean":"2024-04-14T08:38:44.232546+0000","last_became_active":"2024-04-14T08:38:44.230211+0000","last_became_peered":"2024-04-14T08:38:44.230211+0000","last_unstale":"2024-04-14T08:38:44.232546+0000","last_undegraded":"2024-04-14T08:38:44.232546+0000","last_fullsized":"2024-04-14T08:38:44.232546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:21:35.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216328+0000","last_change":"2024-04-14T08:38:44.216328+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216328+0000","last_undegraded":"2024-04-14T08:38:44.216328+0000","last_fullsized":"2024-04-14T08:38:44.216328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225413+0000","last_change":"2024-04-14T08:38:44.224050+0000","last_active":"2024-04-14T08:38:44.225413+0000","last_peered":"2024-04-14T08:38:44.225413+0000","last_clean":"2024-04-14T08:38:44.225413+0000","last_became_active":"2024-04-14T08:38:44.223667+0000","last_became_peered":"2024-04-14T08:38:44.223667+0000","last_unstale":"2024-04-14T08:38:44.225413+0000","last_undegraded":"2024-04-14T08:38:44.225413+0000","last_fullsized":"2024-04-14T08:38:44.225413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:07:20.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216642+0000","last_change":"2024-04-14T08:38:44.216642+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216642+0000","last_undegraded":"2024-04-14T08:38:44.216642+0000","last_fullsized":"2024-04-14T08:38:44.216642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233092+0000","last_change":"2024-04-14T08:38:44.230396+0000","last_active":"2024-04-14T08:38:44.233092+0000","last_peered":"2024-04-14T08:38:44.233092+0000","last_clean":"2024-04-14T08:38:44.233092+0000","last_became_active":"2024-04-14T08:38:44.230103+0000","last_became_peered":"2024-04-14T08:38:44.230103+0000","last_unstale":"2024-04-14T08:38:44.233092+0000","last_undegraded":"2024-04-14T08:38:44.233092+0000","last_fullsized":"2024-04-14T08:38:44.233092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:59:08.058016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.214047+0000","last_change":"2024-04-14T08:38:44.214047+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.214047+0000","last_undegraded":"2024-04-14T08:38:44.214047+0000","last_fullsized":"2024-04-14T08:38:44.214047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226116+0000","last_change":"2024-04-14T08:38:44.224830+0000","last_active":"2024-04-14T08:38:44.226116+0000","last_peered":"2024-04-14T08:38:44.226116+0000","last_clean":"2024-04-14T08:38:44.226116+0000","last_became_active":"2024-04-14T08:38:44.224558+0000","last_became_peered":"2024-04-14T08:38:44.224558+0000","last_unstale":"2024-04-14T08:38:44.226116+0000","last_undegraded":"2024-04-14T08:38:44.226116+0000","last_fullsized":"2024-04-14T08:38:44.226116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:35.503739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.214795+0000","last_change":"2024-04-14T08:38:44.214795+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.214795+0000","last_undegraded":"2024-04-14T08:38:44.214795+0000","last_fullsized":"2024-04-14T08:38:44.214795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225790+0000","last_change":"2024-04-14T08:38:44.224592+0000","last_active":"2024-04-14T08:38:44.225790+0000","last_peered":"2024-04-14T08:38:44.225790+0000","last_clean":"2024-04-14T08:38:44.225790+0000","last_became_active":"2024-04-14T08:38:44.224287+0000","last_became_peered":"2024-04-14T08:38:44.224287+0000","last_unstale":"2024-04-14T08:38:44.225790+0000","last_undegraded":"2024-04-14T08:38:44.225790+0000","last_fullsized":"2024-04-14T08:38:44.225790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:56:24.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.220700+0000","last_change":"2024-04-14T08:38:44.220700+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.220700+0000","last_undegraded":"2024-04-14T08:38:44.220700+0000","last_fullsized":"2024-04-14T08:38:44.220700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.214298+0000","last_change":"2024-04-14T08:38:44.219166+0000","last_active":"2024-04-14T08:38:45.214298+0000","last_peered":"2024-04-14T08:38:45.214298+0000","last_clean":"2024-04-14T08:38:45.214298+0000","last_became_active":"2024-04-14T08:38:44.218905+0000","last_became_peered":"2024-04-14T08:38:44.218905+0000","last_unstale":"2024-04-14T08:38:45.214298+0000","last_undegraded":"2024-04-14T08:38:45.214298+0000","last_fullsized":"2024-04-14T08:38:45.214298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:24:57.813701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.227888+0000","last_change":"2024-04-14T08:38:45.223615+0000","last_active":"2024-04-14T08:38:45.227888+0000","last_peered":"2024-04-14T08:38:45.227888+0000","last_clean":"2024-04-14T08:38:45.227888+0000","last_became_active":"2024-04-14T08:38:45.221967+0000","last_became_peered":"2024-04-14T08:38:45.221967+0000","last_unstale":"2024-04-14T08:38:45.227888+0000","last_undegraded":"2024-04-14T08:38:45.227888+0000","last_fullsized":"2024-04-14T08:38:45.227888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:09.277582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228434+0000","last_change":"2024-04-14T08:38:44.224166+0000","last_active":"2024-04-14T08:38:44.228434+0000","last_peered":"2024-04-14T08:38:44.228434+0000","last_clean":"2024-04-14T08:38:44.228434+0000","last_became_active":"2024-04-14T08:38:44.223244+0000","last_became_peered":"2024-04-14T08:38:44.223244+0000","last_unstale":"2024-04-14T08:38:44.228434+0000","last_undegraded":"2024-04-14T08:38:44.228434+0000","last_fullsized":"2024-04-14T08:38:44.228434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:42.226080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":301,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.214417+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:38:45.214417+0000","last_peered":"2024-04-14T08:38:45.214417+0000","last_clean":"2024-04-14T08:38:45.214417+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:38:45.214417+0000","last_undegraded":"2024-04-14T08:38:45.214417+0000","last_fullsized":"2024-04-14T08:38:45.214417+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1650688,"data_stored":1068945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":464687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1630208,"data_stored":1061722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:38:48.556 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:38:48.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.690+0000 7fc5d2a2d640 1 Processor -- start 2024-04-14T08:38:48.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.691+0000 7fc5d2a2d640 1 -- start start 2024-04-14T08:38:48.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.691+0000 7fc5d2a2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5cc0b7a60 0x7fc5cc0ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.691+0000 7fc5d2a2d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc5cc06dc40 0x7fc5cc0cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.691+0000 7fc5d2a2d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5cc0d2b10 con 0x7fc5cc08c4c0 2024-04-14T08:38:48.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.691+0000 7fc5d2a2d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc5cc0d0ad0 con 0x7fc5cc06dc40 2024-04-14T08:38:48.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.691+0000 7fc5d2a2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc5cc0d0db0 con 0x7fc5cc0b7a60 2024-04-14T08:38:48.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.691+0000 7fc5cbfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5cc0b7a60 0x7fc5cc0ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.691+0000 7fc5d0fca640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc5cc06dc40 0x7fc5cc0cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.692+0000 7fc5cb7fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fc5cc08c4c0 0x7fc5cc0cd410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:48216/0 (socket says 172.21.15.172:48216) 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.692+0000 7fc5cb7fe640 1 -- 172.21.15.172:0/52695121 learned_addr learned my addr 172.21.15.172:0/52695121 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.692+0000 7fc5caffd640 1 -- 172.21.15.172:0/52695121 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3996747775 0 0) 0x7fc5cc0d2b10 con 0x7fc5cc08c4c0 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.692+0000 7fc5caffd640 1 -- 172.21.15.172:0/52695121 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5b4003520 con 0x7fc5cc08c4c0 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.692+0000 7fc5cbfff640 1 -- 172.21.15.172:0/52695121 >> v2:172.21.15.177:3300/0 conn(0x7fc5cc06dc40 msgr2=0x7fc5cc0cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.692+0000 7fc5cbfff640 1 --2- 172.21.15.172:0/52695121 >> v2:172.21.15.177:3300/0 conn(0x7fc5cc06dc40 0x7fc5cc0cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.692+0000 7fc5cbfff640 1 -- 172.21.15.172:0/52695121 >> v1:172.21.15.177:6789/0 conn(0x7fc5cc08c4c0 legacy=0x7fc5cc0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.692+0000 7fc5cbfff640 1 -- 172.21.15.172:0/52695121 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc0d1090 con 0x7fc5cc0b7a60 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.692+0000 7fc5d0fca640 1 --2- 172.21.15.172:0/52695121 >> v2:172.21.15.177:3300/0 conn(0x7fc5cc06dc40 0x7fc5cc0cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.693+0000 7fc5cbfff640 1 --2- 172.21.15.172:0/52695121 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5cc0b7a60 0x7fc5cc0ce8c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0097a0 tx=0x7fc5bc02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3491d5605258c89e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.693+0000 7fc5caffd640 1 -- 172.21.15.172:0/52695121 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0315a0 con 0x7fc5cc0b7a60 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.693+0000 7fc5caffd640 1 -- 172.21.15.172:0/52695121 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5bc037070 con 0x7fc5cc0b7a60 2024-04-14T08:38:48.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.693+0000 7fc5caffd640 1 -- 172.21.15.172:0/52695121 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc031960 con 0x7fc5cc0b7a60 2024-04-14T08:38:48.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.693+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/52695121 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5cc0b7a60 msgr2=0x7fc5cc0ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.693+0000 7fc5d2a2d640 1 --2- 172.21.15.172:0/52695121 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5cc0b7a60 0x7fc5cc0ce8c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0097a0 tx=0x7fc5bc02b840 comp rx=0 tx=0).stop 2024-04-14T08:38:48.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.693+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/52695121 shutdown_connections 2024-04-14T08:38:48.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.693+0000 7fc5d2a2d640 1 --2- 172.21.15.172:0/52695121 >> v2:172.21.15.177:3300/0 conn(0x7fc5cc06dc40 0x7fc5cc0cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.693+0000 7fc5d2a2d640 1 --2- 172.21.15.172:0/52695121 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5cc0b7a60 0x7fc5cc0ce8c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.693+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/52695121 >> 172.21.15.172:0/52695121 conn(0x7fc5cc08c000 msgr2=0x7fc5cc0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:48.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.693+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/52695121 shutdown_connections 2024-04-14T08:38:48.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.693+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/52695121 wait complete. 2024-04-14T08:38:48.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.694+0000 7fc5d2a2d640 1 Processor -- start 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.695+0000 7fc5d2a2d640 1 -- start start 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.695+0000 7fc5d2a2d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5cc06dc40 0x7fc5cc15c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.695+0000 7fc5d2a2d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5cc08c4c0 0x7fc5cc06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.695+0000 7fc5d2a2d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5cc0b7a60 0x7fc5cc14bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.695+0000 7fc5d2a2d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc0d2750 con 0x7fc5cc0b7a60 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.695+0000 7fc5d2a2d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc0b7e40 con 0x7fc5cc06dc40 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.695+0000 7fc5d2a2d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc5cc0d28d0 con 0x7fc5cc08c4c0 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.695+0000 7fc5cbfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5cc06dc40 0x7fc5cc15c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.695+0000 7fc5d0fca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5cc0b7a60 0x7fc5cc14bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5cb7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5cc08c4c0 0x7fc5cc06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5cbfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5cc06dc40 0x7fc5cc15c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:41528/0 (socket says 172.21.15.172:41528) 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5cbfff640 1 -- 172.21.15.172:0/2417435102 learned_addr learned my addr 172.21.15.172:0/2417435102 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5d0fca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5cc0b7a60 0x7fc5cc14bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49822/0 (socket says 172.21.15.172:49822) 2024-04-14T08:38:48.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5cbfff640 1 -- 172.21.15.172:0/2417435102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5cc08c4c0 msgr2=0x7fc5cc06fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5cbfff640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5cc08c4c0 0x7fc5cc06fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5cbfff640 1 -- 172.21.15.172:0/2417435102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5cc0b7a60 msgr2=0x7fc5cc14bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5cbfff640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5cc0b7a60 0x7fc5cc14bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5cbfff640 1 -- 172.21.15.172:0/2417435102 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc15b580 con 0x7fc5cc06dc40 2024-04-14T08:38:48.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5d0fca640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5cc0b7a60 0x7fc5cc14bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:48.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5cbfff640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5cc06dc40 0x7fc5cc15c620 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc02b880 tx=0x7fc5bc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:48.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5c8ff9640 1 -- 172.21.15.172:0/2417435102 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc032630 con 0x7fc5cc06dc40 2024-04-14T08:38:48.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5c8ff9640 1 -- 172.21.15.172:0/2417435102 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5bc032810 con 0x7fc5cc06dc40 2024-04-14T08:38:48.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/2417435102 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc15b860 con 0x7fc5cc06dc40 2024-04-14T08:38:48.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5c8ff9640 1 -- 172.21.15.172:0/2417435102 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc032c00 con 0x7fc5cc06dc40 2024-04-14T08:38:48.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.696+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/2417435102 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc15be20 con 0x7fc5cc06dc40 2024-04-14T08:38:48.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.697+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/2417435102 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5cc075490 con 0x7fc5cc06dc40 2024-04-14T08:38:48.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.698+0000 7fc5c8ff9640 1 -- 172.21.15.172:0/2417435102 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5bc04d3f0 con 0x7fc5cc06dc40 2024-04-14T08:38:48.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.698+0000 7fc5c8ff9640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc5b806c8f0 0x7fc5b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:48.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.698+0000 7fc5c8ff9640 1 -- 172.21.15.172:0/2417435102 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fc5bc02f030 con 0x7fc5cc06dc40 2024-04-14T08:38:48.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.698+0000 7fc5cb7fe640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc5b806c8f0 0x7fc5b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:48.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.698+0000 7fc5cb7fe640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc5b806c8f0 0x7fc5b806edb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc1f76c0 tx=0x7fc5c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:48.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.703+0000 7fc5c8ff9640 1 -- 172.21.15.172:0/2417435102 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc5bc040070 con 0x7fc5cc06dc40 2024-04-14T08:38:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.873+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/2417435102 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5cc06f4f0 con 0x7fc5b806c8f0 2024-04-14T08:38:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.878+0000 7fc5c8ff9640 1 -- 172.21.15.172:0/2417435102 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247193 (secure 0 0 0) 0x7fc5cc06f4f0 con 0x7fc5b806c8f0 2024-04-14T08:38:48.883 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:48.884 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:38:48.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.881+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/2417435102 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc5b806c8f0 msgr2=0x7fc5b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.882+0000 7fc5d2a2d640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc5b806c8f0 0x7fc5b806edb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc1f76c0 tx=0x7fc5c0009040 comp rx=0 tx=0).stop 2024-04-14T08:38:48.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.882+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/2417435102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5cc06dc40 msgr2=0x7fc5cc15c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.882+0000 7fc5d2a2d640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5cc06dc40 0x7fc5cc15c620 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc02b880 tx=0x7fc5bc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.882+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/2417435102 shutdown_connections 2024-04-14T08:38:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.882+0000 7fc5d2a2d640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc5b806c8f0 0x7fc5b806edb0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.882+0000 7fc5d2a2d640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5cc0b7a60 0x7fc5cc14bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.882+0000 7fc5d2a2d640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5cc08c4c0 0x7fc5cc06fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.882+0000 7fc5d2a2d640 1 --2- 172.21.15.172:0/2417435102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5cc06dc40 0x7fc5cc15c620 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.882+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/2417435102 >> 172.21.15.172:0/2417435102 conn(0x7fc5cc08c000 msgr2=0x7fc5cc0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.882+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/2417435102 shutdown_connections 2024-04-14T08:38:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:48.882+0000 7fc5d2a2d640 1 -- 172.21.15.172:0/2417435102 wait complete. 2024-04-14T08:38:48.894 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-04-14T08:38:47.191220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590402,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246256,"kb_used_data":9264,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754728464,"statfs":{"total":773094113280,"available":772841947136,"internally_reserved":0,"allocated":9486336,"data_stored":4952124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001105"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216414+0000","last_change":"2024-04-14T08:38:44.216414+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216414+0000","last_undegraded":"2024-04-14T08:38:44.216414+0000","last_fullsized":"2024-04-14T08:38:44.216414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217436+0000","last_change":"2024-04-14T08:38:44.217436+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217436+0000","last_undegraded":"2024-04-14T08:38:44.217436+0000","last_fullsized":"2024-04-14T08:38:44.217436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215977+0000","last_change":"2024-04-14T08:38:44.215977+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215977+0000","last_undegraded":"2024-04-14T08:38:44.215977+0000","last_fullsized":"2024-04-14T08:38:44.215977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.220133+0000","last_change":"2024-04-14T08:38:44.220133+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.220133+0000","last_undegraded":"2024-04-14T08:38:44.220133+0000","last_fullsized":"2024-04-14T08:38:44.220133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211723+0000","last_change":"2024-04-14T08:38:44.211723+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211723+0000","last_undegraded":"2024-04-14T08:38:44.211723+0000","last_fullsized":"2024-04-14T08:38:44.211723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.228102+0000","last_change":"2024-04-14T08:38:44.228102+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.228102+0000","last_undegraded":"2024-04-14T08:38:44.228102+0000","last_fullsized":"2024-04-14T08:38:44.228102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.226673+0000","last_change":"2024-04-14T08:38:45.221574+0000","last_active":"2024-04-14T08:38:45.226673+0000","last_peered":"2024-04-14T08:38:45.226673+0000","last_clean":"2024-04-14T08:38:45.226673+0000","last_became_active":"2024-04-14T08:38:45.221303+0000","last_became_peered":"2024-04-14T08:38:45.221303+0000","last_unstale":"2024-04-14T08:38:45.226673+0000","last_undegraded":"2024-04-14T08:38:45.226673+0000","last_fullsized":"2024-04-14T08:38:45.226673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:59:38.604514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.219635+0000","last_change":"2024-04-14T08:38:44.219635+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.219635+0000","last_undegraded":"2024-04-14T08:38:44.219635+0000","last_fullsized":"2024-04-14T08:38:44.219635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.212588+0000","last_change":"2024-04-14T08:38:44.212588+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.212588+0000","last_undegraded":"2024-04-14T08:38:44.212588+0000","last_fullsized":"2024-04-14T08:38:44.212588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216941+0000","last_change":"2024-04-14T08:38:44.216941+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216941+0000","last_undegraded":"2024-04-14T08:38:44.216941+0000","last_fullsized":"2024-04-14T08:38:44.216941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211782+0000","last_change":"2024-04-14T08:38:44.211782+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211782+0000","last_undegraded":"2024-04-14T08:38:44.211782+0000","last_fullsized":"2024-04-14T08:38:44.211782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216683+0000","last_change":"2024-04-14T08:38:44.216683+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216683+0000","last_undegraded":"2024-04-14T08:38:44.216683+0000","last_fullsized":"2024-04-14T08:38:44.216683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.220232+0000","last_change":"2024-04-14T08:38:44.220232+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.220232+0000","last_undegraded":"2024-04-14T08:38:44.220232+0000","last_fullsized":"2024-04-14T08:38:44.220232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.213829+0000","last_change":"2024-04-14T08:38:44.213829+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.213829+0000","last_undegraded":"2024-04-14T08:38:44.213829+0000","last_fullsized":"2024-04-14T08:38:44.213829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.219502+0000","last_change":"2024-04-14T08:38:44.219502+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.219502+0000","last_undegraded":"2024-04-14T08:38:44.219502+0000","last_fullsized":"2024-04-14T08:38:44.219502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.228755+0000","last_change":"2024-04-14T08:38:44.228755+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.228755+0000","last_undegraded":"2024-04-14T08:38:44.228755+0000","last_fullsized":"2024-04-14T08:38:44.228755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.227961+0000","last_change":"2024-04-14T08:38:44.227961+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.227961+0000","last_undegraded":"2024-04-14T08:38:44.227961+0000","last_fullsized":"2024-04-14T08:38:44.227961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215464+0000","last_change":"2024-04-14T08:38:44.215464+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215464+0000","last_undegraded":"2024-04-14T08:38:44.215464+0000","last_fullsized":"2024-04-14T08:38:44.215464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211266+0000","last_change":"2024-04-14T08:38:44.211266+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211266+0000","last_undegraded":"2024-04-14T08:38:44.211266+0000","last_fullsized":"2024-04-14T08:38:44.211266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218880+0000","last_change":"2024-04-14T08:38:44.218880+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218880+0000","last_undegraded":"2024-04-14T08:38:44.218880+0000","last_fullsized":"2024-04-14T08:38:44.218880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.212313+0000","last_change":"2024-04-14T08:38:44.212313+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.212313+0000","last_undegraded":"2024-04-14T08:38:44.212313+0000","last_fullsized":"2024-04-14T08:38:44.212313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.213369+0000","last_change":"2024-04-14T08:38:44.213369+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.213369+0000","last_undegraded":"2024-04-14T08:38:44.213369+0000","last_fullsized":"2024-04-14T08:38:44.213369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218241+0000","last_change":"2024-04-14T08:38:44.218241+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218241+0000","last_undegraded":"2024-04-14T08:38:44.218241+0000","last_fullsized":"2024-04-14T08:38:44.218241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215033+0000","last_change":"2024-04-14T08:38:44.215033+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215033+0000","last_undegraded":"2024-04-14T08:38:44.215033+0000","last_fullsized":"2024-04-14T08:38:44.215033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216786+0000","last_change":"2024-04-14T08:38:44.216786+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216786+0000","last_undegraded":"2024-04-14T08:38:44.216786+0000","last_fullsized":"2024-04-14T08:38:44.216786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.213997+0000","last_change":"2024-04-14T08:38:44.213997+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.213997+0000","last_undegraded":"2024-04-14T08:38:44.213997+0000","last_fullsized":"2024-04-14T08:38:44.213997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216757+0000","last_change":"2024-04-14T08:38:44.216757+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216757+0000","last_undegraded":"2024-04-14T08:38:44.216757+0000","last_fullsized":"2024-04-14T08:38:44.216757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218369+0000","last_change":"2024-04-14T08:38:44.218369+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218369+0000","last_undegraded":"2024-04-14T08:38:44.218369+0000","last_fullsized":"2024-04-14T08:38:44.218369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.229290+0000","last_change":"2024-04-14T08:38:45.226642+0000","last_active":"2024-04-14T08:38:45.229290+0000","last_peered":"2024-04-14T08:38:45.229290+0000","last_clean":"2024-04-14T08:38:45.229290+0000","last_became_active":"2024-04-14T08:38:45.226349+0000","last_became_peered":"2024-04-14T08:38:45.226349+0000","last_unstale":"2024-04-14T08:38:45.229290+0000","last_undegraded":"2024-04-14T08:38:45.229290+0000","last_fullsized":"2024-04-14T08:38:45.229290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:17:06.164898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211114+0000","last_change":"2024-04-14T08:38:44.211114+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211114+0000","last_undegraded":"2024-04-14T08:38:44.211114+0000","last_fullsized":"2024-04-14T08:38:44.211114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217550+0000","last_change":"2024-04-14T08:38:44.217550+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217550+0000","last_undegraded":"2024-04-14T08:38:44.217550+0000","last_fullsized":"2024-04-14T08:38:44.217550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.228329+0000","last_change":"2024-04-14T08:38:44.228329+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.228329+0000","last_undegraded":"2024-04-14T08:38:44.228329+0000","last_fullsized":"2024-04-14T08:38:44.228329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.225245+0000","last_change":"2024-04-14T08:38:44.225245+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.225245+0000","last_undegraded":"2024-04-14T08:38:44.225245+0000","last_fullsized":"2024-04-14T08:38:44.225245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228182+0000","last_change":"2024-04-14T08:38:44.227154+0000","last_active":"2024-04-14T08:38:44.228182+0000","last_peered":"2024-04-14T08:38:44.228182+0000","last_clean":"2024-04-14T08:38:44.228182+0000","last_became_active":"2024-04-14T08:38:44.226910+0000","last_became_peered":"2024-04-14T08:38:44.226910+0000","last_unstale":"2024-04-14T08:38:44.228182+0000","last_undegraded":"2024-04-14T08:38:44.228182+0000","last_fullsized":"2024-04-14T08:38:44.228182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:49:42.403181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.227247+0000","last_change":"2024-04-14T08:38:44.227247+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.227247+0000","last_undegraded":"2024-04-14T08:38:44.227247+0000","last_fullsized":"2024-04-14T08:38:44.227247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225223+0000","last_change":"2024-04-14T08:38:44.222545+0000","last_active":"2024-04-14T08:38:44.225223+0000","last_peered":"2024-04-14T08:38:44.225223+0000","last_clean":"2024-04-14T08:38:44.225223+0000","last_became_active":"2024-04-14T08:38:44.221999+0000","last_became_peered":"2024-04-14T08:38:44.221999+0000","last_unstale":"2024-04-14T08:38:44.225223+0000","last_undegraded":"2024-04-14T08:38:44.225223+0000","last_fullsized":"2024-04-14T08:38:44.225223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:37:59.212931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.235826+0000","last_change":"2024-04-14T08:38:45.228983+0000","last_active":"2024-04-14T08:38:45.235826+0000","last_peered":"2024-04-14T08:38:45.235826+0000","last_clean":"2024-04-14T08:38:45.235826+0000","last_became_active":"2024-04-14T08:38:45.228632+0000","last_became_peered":"2024-04-14T08:38:45.228632+0000","last_unstale":"2024-04-14T08:38:45.235826+0000","last_undegraded":"2024-04-14T08:38:45.235826+0000","last_fullsized":"2024-04-14T08:38:45.235826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:17:19.947504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.240108+0000","last_change":"2024-04-14T08:38:44.214063+0000","last_active":"2024-04-14T08:38:45.240108+0000","last_peered":"2024-04-14T08:38:45.240108+0000","last_clean":"2024-04-14T08:38:45.240108+0000","last_became_active":"2024-04-14T08:38:44.213612+0000","last_became_peered":"2024-04-14T08:38:44.213612+0000","last_unstale":"2024-04-14T08:38:45.240108+0000","last_undegraded":"2024-04-14T08:38:45.240108+0000","last_fullsized":"2024-04-14T08:38:45.240108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:15:04.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.227051+0000","last_change":"2024-04-14T08:38:44.227051+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.227051+0000","last_undegraded":"2024-04-14T08:38:44.227051+0000","last_fullsized":"2024-04-14T08:38:44.227051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228169+0000","last_change":"2024-04-14T08:38:44.227021+0000","last_active":"2024-04-14T08:38:44.228169+0000","last_peered":"2024-04-14T08:38:44.228169+0000","last_clean":"2024-04-14T08:38:44.228169+0000","last_became_active":"2024-04-14T08:38:44.226731+0000","last_became_peered":"2024-04-14T08:38:44.226731+0000","last_unstale":"2024-04-14T08:38:44.228169+0000","last_undegraded":"2024-04-14T08:38:44.228169+0000","last_fullsized":"2024-04-14T08:38:44.228169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:01.799418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217415+0000","last_change":"2024-04-14T08:38:44.217415+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217415+0000","last_undegraded":"2024-04-14T08:38:44.217415+0000","last_fullsized":"2024-04-14T08:38:44.217415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225249+0000","last_change":"2024-04-14T08:38:44.222167+0000","last_active":"2024-04-14T08:38:44.225249+0000","last_peered":"2024-04-14T08:38:44.225249+0000","last_clean":"2024-04-14T08:38:44.225249+0000","last_became_active":"2024-04-14T08:38:44.221892+0000","last_became_peered":"2024-04-14T08:38:44.221892+0000","last_unstale":"2024-04-14T08:38:44.225249+0000","last_undegraded":"2024-04-14T08:38:44.225249+0000","last_fullsized":"2024-04-14T08:38:44.225249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:13.870437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217818+0000","last_change":"2024-04-14T08:38:44.217818+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217818+0000","last_undegraded":"2024-04-14T08:38:44.217818+0000","last_fullsized":"2024-04-14T08:38:44.217818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226141+0000","last_change":"2024-04-14T08:38:44.222906+0000","last_active":"2024-04-14T08:38:44.226141+0000","last_peered":"2024-04-14T08:38:44.226141+0000","last_clean":"2024-04-14T08:38:44.226141+0000","last_became_active":"2024-04-14T08:38:44.222541+0000","last_became_peered":"2024-04-14T08:38:44.222541+0000","last_unstale":"2024-04-14T08:38:44.226141+0000","last_undegraded":"2024-04-14T08:38:44.226141+0000","last_fullsized":"2024-04-14T08:38:44.226141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:21:35.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.221823+0000","last_change":"2024-04-14T08:38:44.221823+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.221823+0000","last_undegraded":"2024-04-14T08:38:44.221823+0000","last_fullsized":"2024-04-14T08:38:44.221823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.223396+0000","last_change":"2024-04-14T08:38:44.218254+0000","last_active":"2024-04-14T08:38:44.223396+0000","last_peered":"2024-04-14T08:38:44.223396+0000","last_clean":"2024-04-14T08:38:44.223396+0000","last_became_active":"2024-04-14T08:38:44.217969+0000","last_became_peered":"2024-04-14T08:38:44.217969+0000","last_unstale":"2024-04-14T08:38:44.223396+0000","last_undegraded":"2024-04-14T08:38:44.223396+0000","last_fullsized":"2024-04-14T08:38:44.223396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:12:14.185232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217899+0000","last_change":"2024-04-14T08:38:44.217899+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217899+0000","last_undegraded":"2024-04-14T08:38:44.217899+0000","last_fullsized":"2024-04-14T08:38:44.217899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.214121+0000","last_change":"2024-04-14T08:38:44.231501+0000","last_active":"2024-04-14T08:38:45.214121+0000","last_peered":"2024-04-14T08:38:45.214121+0000","last_clean":"2024-04-14T08:38:45.214121+0000","last_became_active":"2024-04-14T08:38:44.231184+0000","last_became_peered":"2024-04-14T08:38:44.231184+0000","last_unstale":"2024-04-14T08:38:45.214121+0000","last_undegraded":"2024-04-14T08:38:45.214121+0000","last_fullsized":"2024-04-14T08:38:45.214121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:59:08.058016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.212060+0000","last_change":"2024-04-14T08:38:44.212060+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.212060+0000","last_undegraded":"2024-04-14T08:38:44.212060+0000","last_fullsized":"2024-04-14T08:38:44.212060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225545+0000","last_change":"2024-04-14T08:38:44.224713+0000","last_active":"2024-04-14T08:38:44.225545+0000","last_peered":"2024-04-14T08:38:44.225545+0000","last_clean":"2024-04-14T08:38:44.225545+0000","last_became_active":"2024-04-14T08:38:44.224437+0000","last_became_peered":"2024-04-14T08:38:44.224437+0000","last_unstale":"2024-04-14T08:38:44.225545+0000","last_undegraded":"2024-04-14T08:38:44.225545+0000","last_fullsized":"2024-04-14T08:38:44.225545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:37.147141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.221898+0000","last_change":"2024-04-14T08:38:45.219595+0000","last_active":"2024-04-14T08:38:45.221898+0000","last_peered":"2024-04-14T08:38:45.221898+0000","last_clean":"2024-04-14T08:38:45.221898+0000","last_became_active":"2024-04-14T08:38:45.219269+0000","last_became_peered":"2024-04-14T08:38:45.219269+0000","last_unstale":"2024-04-14T08:38:45.221898+0000","last_undegraded":"2024-04-14T08:38:45.221898+0000","last_fullsized":"2024-04-14T08:38:45.221898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:48:28.586545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.224564+0000","last_change":"2024-04-14T08:38:44.219411+0000","last_active":"2024-04-14T08:38:44.224564+0000","last_peered":"2024-04-14T08:38:44.224564+0000","last_clean":"2024-04-14T08:38:44.224564+0000","last_became_active":"2024-04-14T08:38:44.219130+0000","last_became_peered":"2024-04-14T08:38:44.219130+0000","last_unstale":"2024-04-14T08:38:44.224564+0000","last_undegraded":"2024-04-14T08:38:44.224564+0000","last_fullsized":"2024-04-14T08:38:44.224564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:43:02.184142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.214550+0000","last_change":"2024-04-14T08:38:44.214550+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.214550+0000","last_undegraded":"2024-04-14T08:38:44.214550+0000","last_fullsized":"2024-04-14T08:38:44.214550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233151+0000","last_change":"2024-04-14T08:38:44.231600+0000","last_active":"2024-04-14T08:38:44.233151+0000","last_peered":"2024-04-14T08:38:44.233151+0000","last_clean":"2024-04-14T08:38:44.233151+0000","last_became_active":"2024-04-14T08:38:44.231276+0000","last_became_peered":"2024-04-14T08:38:44.231276+0000","last_unstale":"2024-04-14T08:38:44.233151+0000","last_undegraded":"2024-04-14T08:38:44.233151+0000","last_fullsized":"2024-04-14T08:38:44.233151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:42.226080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216192+0000","last_change":"2024-04-14T08:38:44.216192+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216192+0000","last_undegraded":"2024-04-14T08:38:44.216192+0000","last_fullsized":"2024-04-14T08:38:44.216192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233087+0000","last_change":"2024-04-14T08:38:44.230303+0000","last_active":"2024-04-14T08:38:44.233087+0000","last_peered":"2024-04-14T08:38:44.233087+0000","last_clean":"2024-04-14T08:38:44.233087+0000","last_became_active":"2024-04-14T08:38:44.230051+0000","last_became_peered":"2024-04-14T08:38:44.230051+0000","last_unstale":"2024-04-14T08:38:44.233087+0000","last_undegraded":"2024-04-14T08:38:44.233087+0000","last_fullsized":"2024-04-14T08:38:44.233087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:36:12.417551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216476+0000","last_change":"2024-04-14T08:38:44.216476+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216476+0000","last_undegraded":"2024-04-14T08:38:44.216476+0000","last_fullsized":"2024-04-14T08:38:44.216476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.245313+0000","last_change":"2024-04-14T08:38:44.230951+0000","last_active":"2024-04-14T08:38:45.245313+0000","last_peered":"2024-04-14T08:38:45.245313+0000","last_clean":"2024-04-14T08:38:45.245313+0000","last_became_active":"2024-04-14T08:38:44.230499+0000","last_became_peered":"2024-04-14T08:38:44.230499+0000","last_unstale":"2024-04-14T08:38:45.245313+0000","last_undegraded":"2024-04-14T08:38:45.245313+0000","last_fullsized":"2024-04-14T08:38:45.245313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:55:56.125870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.210337+0000","last_change":"2024-04-14T08:38:44.210337+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.210337+0000","last_undegraded":"2024-04-14T08:38:44.210337+0000","last_fullsized":"2024-04-14T08:38:44.210337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233084+0000","last_change":"2024-04-14T08:38:44.230300+0000","last_active":"2024-04-14T08:38:44.233084+0000","last_peered":"2024-04-14T08:38:44.233084+0000","last_clean":"2024-04-14T08:38:44.233084+0000","last_became_active":"2024-04-14T08:38:44.230024+0000","last_became_peered":"2024-04-14T08:38:44.230024+0000","last_unstale":"2024-04-14T08:38:44.233084+0000","last_undegraded":"2024-04-14T08:38:44.233084+0000","last_fullsized":"2024-04-14T08:38:44.233084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:15:04.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216501+0000","last_change":"2024-04-14T08:38:44.216501+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216501+0000","last_undegraded":"2024-04-14T08:38:44.216501+0000","last_fullsized":"2024-04-14T08:38:44.216501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.227819+0000","last_change":"2024-04-14T08:38:44.224811+0000","last_active":"2024-04-14T08:38:44.227819+0000","last_peered":"2024-04-14T08:38:44.227819+0000","last_clean":"2024-04-14T08:38:44.227819+0000","last_became_active":"2024-04-14T08:38:44.224541+0000","last_became_peered":"2024-04-14T08:38:44.224541+0000","last_unstale":"2024-04-14T08:38:44.227819+0000","last_undegraded":"2024-04-14T08:38:44.227819+0000","last_fullsized":"2024-04-14T08:38:44.227819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:31:11.641961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217216+0000","last_change":"2024-04-14T08:38:44.217216+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217216+0000","last_undegraded":"2024-04-14T08:38:44.217216+0000","last_fullsized":"2024-04-14T08:38:44.217216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.217797+0000","last_change":"2024-04-14T08:38:44.216083+0000","last_active":"2024-04-14T08:38:44.217797+0000","last_peered":"2024-04-14T08:38:44.217797+0000","last_clean":"2024-04-14T08:38:44.217797+0000","last_became_active":"2024-04-14T08:38:44.215754+0000","last_became_peered":"2024-04-14T08:38:44.215754+0000","last_unstale":"2024-04-14T08:38:44.217797+0000","last_undegraded":"2024-04-14T08:38:44.217797+0000","last_fullsized":"2024-04-14T08:38:44.217797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:13.870437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.222718+0000","last_change":"2024-04-14T08:38:44.222718+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.222718+0000","last_undegraded":"2024-04-14T08:38:44.222718+0000","last_fullsized":"2024-04-14T08:38:44.222718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.223232+0000","last_change":"2024-04-14T08:38:44.222002+0000","last_active":"2024-04-14T08:38:44.223232+0000","last_peered":"2024-04-14T08:38:44.223232+0000","last_clean":"2024-04-14T08:38:44.223232+0000","last_became_active":"2024-04-14T08:38:44.221679+0000","last_became_peered":"2024-04-14T08:38:44.221679+0000","last_unstale":"2024-04-14T08:38:44.223232+0000","last_undegraded":"2024-04-14T08:38:44.223232+0000","last_fullsized":"2024-04-14T08:38:44.223232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:53.649370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215754+0000","last_change":"2024-04-14T08:38:44.215754+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215754+0000","last_undegraded":"2024-04-14T08:38:44.215754+0000","last_fullsized":"2024-04-14T08:38:44.215754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.241020+0000","last_change":"2024-04-14T08:38:44.220058+0000","last_active":"2024-04-14T08:38:45.241020+0000","last_peered":"2024-04-14T08:38:45.241020+0000","last_clean":"2024-04-14T08:38:45.241020+0000","last_became_active":"2024-04-14T08:38:44.219721+0000","last_became_peered":"2024-04-14T08:38:44.219721+0000","last_unstale":"2024-04-14T08:38:45.241020+0000","last_undegraded":"2024-04-14T08:38:45.241020+0000","last_fullsized":"2024-04-14T08:38:45.241020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:54:15.282969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211663+0000","last_change":"2024-04-14T08:38:44.211663+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211663+0000","last_undegraded":"2024-04-14T08:38:44.211663+0000","last_fullsized":"2024-04-14T08:38:44.211663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.224435+0000","last_change":"2024-04-14T08:38:44.221416+0000","last_active":"2024-04-14T08:38:44.224435+0000","last_peered":"2024-04-14T08:38:44.224435+0000","last_clean":"2024-04-14T08:38:44.224435+0000","last_became_active":"2024-04-14T08:38:44.221138+0000","last_became_peered":"2024-04-14T08:38:44.221138+0000","last_unstale":"2024-04-14T08:38:44.224435+0000","last_undegraded":"2024-04-14T08:38:44.224435+0000","last_fullsized":"2024-04-14T08:38:44.224435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:15.223452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.219407+0000","last_change":"2024-04-14T08:38:44.219407+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.219407+0000","last_undegraded":"2024-04-14T08:38:44.219407+0000","last_fullsized":"2024-04-14T08:38:44.219407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225070+0000","last_change":"2024-04-14T08:38:44.224441+0000","last_active":"2024-04-14T08:38:44.225070+0000","last_peered":"2024-04-14T08:38:44.225070+0000","last_clean":"2024-04-14T08:38:44.225070+0000","last_became_active":"2024-04-14T08:38:44.224137+0000","last_became_peered":"2024-04-14T08:38:44.224137+0000","last_unstale":"2024-04-14T08:38:44.225070+0000","last_undegraded":"2024-04-14T08:38:44.225070+0000","last_fullsized":"2024-04-14T08:38:44.225070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:36:44.980948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217106+0000","last_change":"2024-04-14T08:38:44.217106+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217106+0000","last_undegraded":"2024-04-14T08:38:44.217106+0000","last_fullsized":"2024-04-14T08:38:44.217106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226868+0000","last_change":"2024-04-14T08:38:44.220261+0000","last_active":"2024-04-14T08:38:44.226868+0000","last_peered":"2024-04-14T08:38:44.226868+0000","last_clean":"2024-04-14T08:38:44.226868+0000","last_became_active":"2024-04-14T08:38:44.220001+0000","last_became_peered":"2024-04-14T08:38:44.220001+0000","last_unstale":"2024-04-14T08:38:44.226868+0000","last_undegraded":"2024-04-14T08:38:44.226868+0000","last_fullsized":"2024-04-14T08:38:44.226868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:13:48.232427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215334+0000","last_change":"2024-04-14T08:38:44.215334+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215334+0000","last_undegraded":"2024-04-14T08:38:44.215334+0000","last_fullsized":"2024-04-14T08:38:44.215334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228198+0000","last_change":"2024-04-14T08:38:44.227400+0000","last_active":"2024-04-14T08:38:44.228198+0000","last_peered":"2024-04-14T08:38:44.228198+0000","last_clean":"2024-04-14T08:38:44.228198+0000","last_became_active":"2024-04-14T08:38:44.226963+0000","last_became_peered":"2024-04-14T08:38:44.226963+0000","last_unstale":"2024-04-14T08:38:44.228198+0000","last_undegraded":"2024-04-14T08:38:44.228198+0000","last_fullsized":"2024-04-14T08:38:44.228198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:56:30.743467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.221098+0000","last_change":"2024-04-14T08:38:44.221098+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.221098+0000","last_undegraded":"2024-04-14T08:38:44.221098+0000","last_fullsized":"2024-04-14T08:38:44.221098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226693+0000","last_change":"2024-04-14T08:38:44.221259+0000","last_active":"2024-04-14T08:38:44.226693+0000","last_peered":"2024-04-14T08:38:44.226693+0000","last_clean":"2024-04-14T08:38:44.226693+0000","last_became_active":"2024-04-14T08:38:44.220936+0000","last_became_peered":"2024-04-14T08:38:44.220936+0000","last_unstale":"2024-04-14T08:38:44.226693+0000","last_undegraded":"2024-04-14T08:38:44.226693+0000","last_fullsized":"2024-04-14T08:38:44.226693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:20:32.325057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215475+0000","last_change":"2024-04-14T08:38:44.215475+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215475+0000","last_undegraded":"2024-04-14T08:38:44.215475+0000","last_fullsized":"2024-04-14T08:38:44.215475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.240442+0000","last_change":"2024-04-14T08:38:44.222193+0000","last_active":"2024-04-14T08:38:45.240442+0000","last_peered":"2024-04-14T08:38:45.240442+0000","last_clean":"2024-04-14T08:38:45.240442+0000","last_became_active":"2024-04-14T08:38:44.221895+0000","last_became_peered":"2024-04-14T08:38:44.221895+0000","last_unstale":"2024-04-14T08:38:45.240442+0000","last_undegraded":"2024-04-14T08:38:45.240442+0000","last_fullsized":"2024-04-14T08:38:45.240442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:36:12.417551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.226478+0000","last_change":"2024-04-14T08:38:44.226478+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.226478+0000","last_undegraded":"2024-04-14T08:38:44.226478+0000","last_fullsized":"2024-04-14T08:38:44.226478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.222944+0000","last_change":"2024-04-14T08:38:44.220634+0000","last_active":"2024-04-14T08:38:44.222944+0000","last_peered":"2024-04-14T08:38:44.222944+0000","last_clean":"2024-04-14T08:38:44.222944+0000","last_became_active":"2024-04-14T08:38:44.220352+0000","last_became_peered":"2024-04-14T08:38:44.220352+0000","last_unstale":"2024-04-14T08:38:44.222944+0000","last_undegraded":"2024-04-14T08:38:44.222944+0000","last_fullsized":"2024-04-14T08:38:44.222944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:47:54.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218195+0000","last_change":"2024-04-14T08:38:44.218195+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218195+0000","last_undegraded":"2024-04-14T08:38:44.218195+0000","last_fullsized":"2024-04-14T08:38:44.218195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.232546+0000","last_change":"2024-04-14T08:38:44.230684+0000","last_active":"2024-04-14T08:38:44.232546+0000","last_peered":"2024-04-14T08:38:44.232546+0000","last_clean":"2024-04-14T08:38:44.232546+0000","last_became_active":"2024-04-14T08:38:44.230211+0000","last_became_peered":"2024-04-14T08:38:44.230211+0000","last_unstale":"2024-04-14T08:38:44.232546+0000","last_undegraded":"2024-04-14T08:38:44.232546+0000","last_fullsized":"2024-04-14T08:38:44.232546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:21:35.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216328+0000","last_change":"2024-04-14T08:38:44.216328+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216328+0000","last_undegraded":"2024-04-14T08:38:44.216328+0000","last_fullsized":"2024-04-14T08:38:44.216328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225413+0000","last_change":"2024-04-14T08:38:44.224050+0000","last_active":"2024-04-14T08:38:44.225413+0000","last_peered":"2024-04-14T08:38:44.225413+0000","last_clean":"2024-04-14T08:38:44.225413+0000","last_became_active":"2024-04-14T08:38:44.223667+0000","last_became_peered":"2024-04-14T08:38:44.223667+0000","last_unstale":"2024-04-14T08:38:44.225413+0000","last_undegraded":"2024-04-14T08:38:44.225413+0000","last_fullsized":"2024-04-14T08:38:44.225413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:07:20.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216642+0000","last_change":"2024-04-14T08:38:44.216642+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216642+0000","last_undegraded":"2024-04-14T08:38:44.216642+0000","last_fullsized":"2024-04-14T08:38:44.216642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233092+0000","last_change":"2024-04-14T08:38:44.230396+0000","last_active":"2024-04-14T08:38:44.233092+0000","last_peered":"2024-04-14T08:38:44.233092+0000","last_clean":"2024-04-14T08:38:44.233092+0000","last_became_active":"2024-04-14T08:38:44.230103+0000","last_became_peered":"2024-04-14T08:38:44.230103+0000","last_unstale":"2024-04-14T08:38:44.233092+0000","last_undegraded":"2024-04-14T08:38:44.233092+0000","last_fullsized":"2024-04-14T08:38:44.233092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:59:08.058016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.214047+0000","last_change":"2024-04-14T08:38:44.214047+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.214047+0000","last_undegraded":"2024-04-14T08:38:44.214047+0000","last_fullsized":"2024-04-14T08:38:44.214047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226116+0000","last_change":"2024-04-14T08:38:44.224830+0000","last_active":"2024-04-14T08:38:44.226116+0000","last_peered":"2024-04-14T08:38:44.226116+0000","last_clean":"2024-04-14T08:38:44.226116+0000","last_became_active":"2024-04-14T08:38:44.224558+0000","last_became_peered":"2024-04-14T08:38:44.224558+0000","last_unstale":"2024-04-14T08:38:44.226116+0000","last_undegraded":"2024-04-14T08:38:44.226116+0000","last_fullsized":"2024-04-14T08:38:44.226116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:35.503739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.214795+0000","last_change":"2024-04-14T08:38:44.214795+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.214795+0000","last_undegraded":"2024-04-14T08:38:44.214795+0000","last_fullsized":"2024-04-14T08:38:44.214795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225790+0000","last_change":"2024-04-14T08:38:44.224592+0000","last_active":"2024-04-14T08:38:44.225790+0000","last_peered":"2024-04-14T08:38:44.225790+0000","last_clean":"2024-04-14T08:38:44.225790+0000","last_became_active":"2024-04-14T08:38:44.224287+0000","last_became_peered":"2024-04-14T08:38:44.224287+0000","last_unstale":"2024-04-14T08:38:44.225790+0000","last_undegraded":"2024-04-14T08:38:44.225790+0000","last_fullsized":"2024-04-14T08:38:44.225790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:56:24.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.220700+0000","last_change":"2024-04-14T08:38:44.220700+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.220700+0000","last_undegraded":"2024-04-14T08:38:44.220700+0000","last_fullsized":"2024-04-14T08:38:44.220700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.214298+0000","last_change":"2024-04-14T08:38:44.219166+0000","last_active":"2024-04-14T08:38:45.214298+0000","last_peered":"2024-04-14T08:38:45.214298+0000","last_clean":"2024-04-14T08:38:45.214298+0000","last_became_active":"2024-04-14T08:38:44.218905+0000","last_became_peered":"2024-04-14T08:38:44.218905+0000","last_unstale":"2024-04-14T08:38:45.214298+0000","last_undegraded":"2024-04-14T08:38:45.214298+0000","last_fullsized":"2024-04-14T08:38:45.214298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:24:57.813701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.227888+0000","last_change":"2024-04-14T08:38:45.223615+0000","last_active":"2024-04-14T08:38:45.227888+0000","last_peered":"2024-04-14T08:38:45.227888+0000","last_clean":"2024-04-14T08:38:45.227888+0000","last_became_active":"2024-04-14T08:38:45.221967+0000","last_became_peered":"2024-04-14T08:38:45.221967+0000","last_unstale":"2024-04-14T08:38:45.227888+0000","last_undegraded":"2024-04-14T08:38:45.227888+0000","last_fullsized":"2024-04-14T08:38:45.227888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:09.277582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228434+0000","last_change":"2024-04-14T08:38:44.224166+0000","last_active":"2024-04-14T08:38:44.228434+0000","last_peered":"2024-04-14T08:38:44.228434+0000","last_clean":"2024-04-14T08:38:44.228434+0000","last_became_active":"2024-04-14T08:38:44.223244+0000","last_became_peered":"2024-04-14T08:38:44.223244+0000","last_unstale":"2024-04-14T08:38:44.228434+0000","last_undegraded":"2024-04-14T08:38:44.228434+0000","last_fullsized":"2024-04-14T08:38:44.228434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:42.226080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":301,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.214417+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:38:45.214417+0000","last_peered":"2024-04-14T08:38:45.214417+0000","last_clean":"2024-04-14T08:38:45.214417+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:38:45.214417+0000","last_undegraded":"2024-04-14T08:38:45.214417+0000","last_fullsized":"2024-04-14T08:38:45.214417+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1650688,"data_stored":1068945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":464687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1630208,"data_stored":1061722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:38:48.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3f is not active+clean 2024-04-14T08:38:48.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.216414+0000', 'last_change': '2024-04-14T08:38:44.216414+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.216414+0000', 'last_undegraded': '2024-04-14T08:38:44.216414+0000', 'last_fullsized': '2024-04-14T08:38:44.216414+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:38:48.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3e is not active+clean 2024-04-14T08:38:48.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.217436+0000', 'last_change': '2024-04-14T08:38:44.217436+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.217436+0000', 'last_undegraded': '2024-04-14T08:38:44.217436+0000', 'last_fullsized': '2024-04-14T08:38:44.217436+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:38:48.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3d is not active+clean 2024-04-14T08:38:48.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.215977+0000', 'last_change': '2024-04-14T08:38:44.215977+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.215977+0000', 'last_undegraded': '2024-04-14T08:38:44.215977+0000', 'last_fullsized': '2024-04-14T08:38:44.215977+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:38:48.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3c is not active+clean 2024-04-14T08:38:48.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.220133+0000', 'last_change': '2024-04-14T08:38:44.220133+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.220133+0000', 'last_undegraded': '2024-04-14T08:38:44.220133+0000', 'last_fullsized': '2024-04-14T08:38:44.220133+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:38:48.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3b is not active+clean 2024-04-14T08:38:48.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.211723+0000', 'last_change': '2024-04-14T08:38:44.211723+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.211723+0000', 'last_undegraded': '2024-04-14T08:38:44.211723+0000', 'last_fullsized': '2024-04-14T08:38:44.211723+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:38:48.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3a is not active+clean 2024-04-14T08:38:48.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.228102+0000', 'last_change': '2024-04-14T08:38:44.228102+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.228102+0000', 'last_undegraded': '2024-04-14T08:38:44.228102+0000', 'last_fullsized': '2024-04-14T08:38:44.228102+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.38 is not active+clean 2024-04-14T08:38:48.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.219635+0000', 'last_change': '2024-04-14T08:38:44.219635+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.219635+0000', 'last_undegraded': '2024-04-14T08:38:44.219635+0000', 'last_fullsized': '2024-04-14T08:38:44.219635+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:38:48.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.37 is not active+clean 2024-04-14T08:38:48.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.212588+0000', 'last_change': '2024-04-14T08:38:44.212588+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.212588+0000', 'last_undegraded': '2024-04-14T08:38:44.212588+0000', 'last_fullsized': '2024-04-14T08:38:44.212588+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:38:48.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.36 is not active+clean 2024-04-14T08:38:48.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.216941+0000', 'last_change': '2024-04-14T08:38:44.216941+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.216941+0000', 'last_undegraded': '2024-04-14T08:38:44.216941+0000', 'last_fullsized': '2024-04-14T08:38:44.216941+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:38:48.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.35 is not active+clean 2024-04-14T08:38:48.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.211782+0000', 'last_change': '2024-04-14T08:38:44.211782+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.211782+0000', 'last_undegraded': '2024-04-14T08:38:44.211782+0000', 'last_fullsized': '2024-04-14T08:38:44.211782+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:38:48.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.34 is not active+clean 2024-04-14T08:38:48.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.216683+0000', 'last_change': '2024-04-14T08:38:44.216683+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.216683+0000', 'last_undegraded': '2024-04-14T08:38:44.216683+0000', 'last_fullsized': '2024-04-14T08:38:44.216683+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:38:48.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.33 is not active+clean 2024-04-14T08:38:48.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.220232+0000', 'last_change': '2024-04-14T08:38:44.220232+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.220232+0000', 'last_undegraded': '2024-04-14T08:38:44.220232+0000', 'last_fullsized': '2024-04-14T08:38:44.220232+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:38:48.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.32 is not active+clean 2024-04-14T08:38:48.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.213829+0000', 'last_change': '2024-04-14T08:38:44.213829+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.213829+0000', 'last_undegraded': '2024-04-14T08:38:44.213829+0000', 'last_fullsized': '2024-04-14T08:38:44.213829+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:38:48.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.31 is not active+clean 2024-04-14T08:38:48.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.219502+0000', 'last_change': '2024-04-14T08:38:44.219502+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.219502+0000', 'last_undegraded': '2024-04-14T08:38:44.219502+0000', 'last_fullsized': '2024-04-14T08:38:44.219502+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:38:48.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.30 is not active+clean 2024-04-14T08:38:48.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.228755+0000', 'last_change': '2024-04-14T08:38:44.228755+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.228755+0000', 'last_undegraded': '2024-04-14T08:38:44.228755+0000', 'last_fullsized': '2024-04-14T08:38:44.228755+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2f is not active+clean 2024-04-14T08:38:48.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.227961+0000', 'last_change': '2024-04-14T08:38:44.227961+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.227961+0000', 'last_undegraded': '2024-04-14T08:38:44.227961+0000', 'last_fullsized': '2024-04-14T08:38:44.227961+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2e is not active+clean 2024-04-14T08:38:48.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.215464+0000', 'last_change': '2024-04-14T08:38:44.215464+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.215464+0000', 'last_undegraded': '2024-04-14T08:38:44.215464+0000', 'last_fullsized': '2024-04-14T08:38:44.215464+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:38:48.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2d is not active+clean 2024-04-14T08:38:48.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.211266+0000', 'last_change': '2024-04-14T08:38:44.211266+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.211266+0000', 'last_undegraded': '2024-04-14T08:38:44.211266+0000', 'last_fullsized': '2024-04-14T08:38:44.211266+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:38:48.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2c is not active+clean 2024-04-14T08:38:48.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.218880+0000', 'last_change': '2024-04-14T08:38:44.218880+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.218880+0000', 'last_undegraded': '2024-04-14T08:38:44.218880+0000', 'last_fullsized': '2024-04-14T08:38:44.218880+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:38:48.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2b is not active+clean 2024-04-14T08:38:48.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.212313+0000', 'last_change': '2024-04-14T08:38:44.212313+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.212313+0000', 'last_undegraded': '2024-04-14T08:38:44.212313+0000', 'last_fullsized': '2024-04-14T08:38:44.212313+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:38:48.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2a is not active+clean 2024-04-14T08:38:48.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.213369+0000', 'last_change': '2024-04-14T08:38:44.213369+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.213369+0000', 'last_undegraded': '2024-04-14T08:38:44.213369+0000', 'last_fullsized': '2024-04-14T08:38:44.213369+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:38:48.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2024-04-14T08:38:48.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.218241+0000', 'last_change': '2024-04-14T08:38:44.218241+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.218241+0000', 'last_undegraded': '2024-04-14T08:38:44.218241+0000', 'last_fullsized': '2024-04-14T08:38:44.218241+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:38:48.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.28 is not active+clean 2024-04-14T08:38:48.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.215033+0000', 'last_change': '2024-04-14T08:38:44.215033+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.215033+0000', 'last_undegraded': '2024-04-14T08:38:44.215033+0000', 'last_fullsized': '2024-04-14T08:38:44.215033+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:38:48.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.27 is not active+clean 2024-04-14T08:38:48.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.216786+0000', 'last_change': '2024-04-14T08:38:44.216786+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.216786+0000', 'last_undegraded': '2024-04-14T08:38:44.216786+0000', 'last_fullsized': '2024-04-14T08:38:44.216786+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:38:48.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.26 is not active+clean 2024-04-14T08:38:48.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.213997+0000', 'last_change': '2024-04-14T08:38:44.213997+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.213997+0000', 'last_undegraded': '2024-04-14T08:38:44.213997+0000', 'last_fullsized': '2024-04-14T08:38:44.213997+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:38:48.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.25 is not active+clean 2024-04-14T08:38:48.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.216757+0000', 'last_change': '2024-04-14T08:38:44.216757+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.216757+0000', 'last_undegraded': '2024-04-14T08:38:44.216757+0000', 'last_fullsized': '2024-04-14T08:38:44.216757+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:38:48.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.24 is not active+clean 2024-04-14T08:38:48.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.218369+0000', 'last_change': '2024-04-14T08:38:44.218369+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.218369+0000', 'last_undegraded': '2024-04-14T08:38:44.218369+0000', 'last_fullsized': '2024-04-14T08:38:44.218369+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:38:48.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.22 is not active+clean 2024-04-14T08:38:48.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.211114+0000', 'last_change': '2024-04-14T08:38:44.211114+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.211114+0000', 'last_undegraded': '2024-04-14T08:38:44.211114+0000', 'last_fullsized': '2024-04-14T08:38:44.211114+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:38:48.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.21 is not active+clean 2024-04-14T08:38:48.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.217550+0000', 'last_change': '2024-04-14T08:38:44.217550+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.217550+0000', 'last_undegraded': '2024-04-14T08:38:44.217550+0000', 'last_fullsized': '2024-04-14T08:38:44.217550+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:38:48.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.20 is not active+clean 2024-04-14T08:38:48.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.228329+0000', 'last_change': '2024-04-14T08:38:44.228329+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.228329+0000', 'last_undegraded': '2024-04-14T08:38:44.228329+0000', 'last_fullsized': '2024-04-14T08:38:44.228329+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-04-14T08:38:48.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.225245+0000', 'last_change': '2024-04-14T08:38:44.225245+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.225245+0000', 'last_undegraded': '2024-04-14T08:38:44.225245+0000', 'last_fullsized': '2024-04-14T08:38:44.225245+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-04-14T08:38:48.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.227247+0000', 'last_change': '2024-04-14T08:38:44.227247+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.227247+0000', 'last_undegraded': '2024-04-14T08:38:44.227247+0000', 'last_fullsized': '2024-04-14T08:38:44.227247+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-04-14T08:38:48.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.227051+0000', 'last_change': '2024-04-14T08:38:44.227051+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.227051+0000', 'last_undegraded': '2024-04-14T08:38:44.227051+0000', 'last_fullsized': '2024-04-14T08:38:44.227051+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2024-04-14T08:38:48.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.217415+0000', 'last_change': '2024-04-14T08:38:44.217415+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.217415+0000', 'last_undegraded': '2024-04-14T08:38:44.217415+0000', 'last_fullsized': '2024-04-14T08:38:44.217415+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:38:48.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-04-14T08:38:48.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.217818+0000', 'last_change': '2024-04-14T08:38:44.217818+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.217818+0000', 'last_undegraded': '2024-04-14T08:38:44.217818+0000', 'last_fullsized': '2024-04-14T08:38:44.217818+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:38:48.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-04-14T08:38:48.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.221823+0000', 'last_change': '2024-04-14T08:38:44.221823+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.221823+0000', 'last_undegraded': '2024-04-14T08:38:44.221823+0000', 'last_fullsized': '2024-04-14T08:38:44.221823+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2024-04-14T08:38:48.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.217899+0000', 'last_change': '2024-04-14T08:38:44.217899+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.217899+0000', 'last_undegraded': '2024-04-14T08:38:44.217899+0000', 'last_fullsized': '2024-04-14T08:38:44.217899+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:38:48.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-04-14T08:38:48.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.212060+0000', 'last_change': '2024-04-14T08:38:44.212060+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.212060+0000', 'last_undegraded': '2024-04-14T08:38:44.212060+0000', 'last_fullsized': '2024-04-14T08:38:44.212060+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:38:48.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-04-14T08:38:48.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.214550+0000', 'last_change': '2024-04-14T08:38:44.214550+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.214550+0000', 'last_undegraded': '2024-04-14T08:38:44.214550+0000', 'last_fullsized': '2024-04-14T08:38:44.214550+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:38:48.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2024-04-14T08:38:48.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.216192+0000', 'last_change': '2024-04-14T08:38:44.216192+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.216192+0000', 'last_undegraded': '2024-04-14T08:38:44.216192+0000', 'last_fullsized': '2024-04-14T08:38:44.216192+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:38:48.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2024-04-14T08:38:48.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.216476+0000', 'last_change': '2024-04-14T08:38:44.216476+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.216476+0000', 'last_undegraded': '2024-04-14T08:38:44.216476+0000', 'last_fullsized': '2024-04-14T08:38:44.216476+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:38:48.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-04-14T08:38:48.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.210337+0000', 'last_change': '2024-04-14T08:38:44.210337+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.210337+0000', 'last_undegraded': '2024-04-14T08:38:44.210337+0000', 'last_fullsized': '2024-04-14T08:38:44.210337+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:38:48.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-04-14T08:38:48.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.216501+0000', 'last_change': '2024-04-14T08:38:44.216501+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.216501+0000', 'last_undegraded': '2024-04-14T08:38:44.216501+0000', 'last_fullsized': '2024-04-14T08:38:44.216501+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:38:48.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-04-14T08:38:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.217216+0000', 'last_change': '2024-04-14T08:38:44.217216+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.217216+0000', 'last_undegraded': '2024-04-14T08:38:44.217216+0000', 'last_fullsized': '2024-04-14T08:38:44.217216+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:38:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-04-14T08:38:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.222718+0000', 'last_change': '2024-04-14T08:38:44.222718+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.222718+0000', 'last_undegraded': '2024-04-14T08:38:44.222718+0000', 'last_fullsized': '2024-04-14T08:38:44.222718+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-04-14T08:38:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.215754+0000', 'last_change': '2024-04-14T08:38:44.215754+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.215754+0000', 'last_undegraded': '2024-04-14T08:38:44.215754+0000', 'last_fullsized': '2024-04-14T08:38:44.215754+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:38:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-04-14T08:38:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.211663+0000', 'last_change': '2024-04-14T08:38:44.211663+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.211663+0000', 'last_undegraded': '2024-04-14T08:38:44.211663+0000', 'last_fullsized': '2024-04-14T08:38:44.211663+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:38:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-04-14T08:38:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.219407+0000', 'last_change': '2024-04-14T08:38:44.219407+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.219407+0000', 'last_undegraded': '2024-04-14T08:38:44.219407+0000', 'last_fullsized': '2024-04-14T08:38:44.219407+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-04-14T08:38:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.217106+0000', 'last_change': '2024-04-14T08:38:44.217106+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.217106+0000', 'last_undegraded': '2024-04-14T08:38:44.217106+0000', 'last_fullsized': '2024-04-14T08:38:44.217106+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:38:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-04-14T08:38:48.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.215334+0000', 'last_change': '2024-04-14T08:38:44.215334+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.215334+0000', 'last_undegraded': '2024-04-14T08:38:44.215334+0000', 'last_fullsized': '2024-04-14T08:38:44.215334+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:38:48.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-04-14T08:38:48.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.221098+0000', 'last_change': '2024-04-14T08:38:44.221098+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.221098+0000', 'last_undegraded': '2024-04-14T08:38:44.221098+0000', 'last_fullsized': '2024-04-14T08:38:44.221098+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-04-14T08:38:48.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.215475+0000', 'last_change': '2024-04-14T08:38:44.215475+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.215475+0000', 'last_undegraded': '2024-04-14T08:38:44.215475+0000', 'last_fullsized': '2024-04-14T08:38:44.215475+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:38:48.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-04-14T08:38:48.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.226478+0000', 'last_change': '2024-04-14T08:38:44.226478+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.226478+0000', 'last_undegraded': '2024-04-14T08:38:44.226478+0000', 'last_fullsized': '2024-04-14T08:38:44.226478+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-04-14T08:38:48.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.218195+0000', 'last_change': '2024-04-14T08:38:44.218195+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.218195+0000', 'last_undegraded': '2024-04-14T08:38:44.218195+0000', 'last_fullsized': '2024-04-14T08:38:44.218195+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-04-14T08:38:48.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.216328+0000', 'last_change': '2024-04-14T08:38:44.216328+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.216328+0000', 'last_undegraded': '2024-04-14T08:38:44.216328+0000', 'last_fullsized': '2024-04-14T08:38:44.216328+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:38:48.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-04-14T08:38:48.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.216642+0000', 'last_change': '2024-04-14T08:38:44.216642+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.216642+0000', 'last_undegraded': '2024-04-14T08:38:44.216642+0000', 'last_fullsized': '2024-04-14T08:38:44.216642+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:38:48.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-04-14T08:38:48.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.214047+0000', 'last_change': '2024-04-14T08:38:44.214047+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.214047+0000', 'last_undegraded': '2024-04-14T08:38:44.214047+0000', 'last_fullsized': '2024-04-14T08:38:44.214047+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:38:48.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-04-14T08:38:48.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.214795+0000', 'last_change': '2024-04-14T08:38:44.214795+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.214795+0000', 'last_undegraded': '2024-04-14T08:38:44.214795+0000', 'last_fullsized': '2024-04-14T08:38:44.214795+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:38:48.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-04-14T08:38:48.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:38:44.220700+0000', 'last_change': '2024-04-14T08:38:44.220700+0000', 'last_active': '2024-04-14T08:38:44.200740+0000', 'last_peered': '2024-04-14T08:38:44.200740+0000', 'last_clean': '2024-04-14T08:38:44.200740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:38:44.220700+0000', 'last_undegraded': '2024-04-14T08:38:44.220700+0000', 'last_fullsized': '2024-04-14T08:38:44.220700+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'last_clean_scrub_stamp': '2024-04-14T08:38:44.200740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:38:48.910 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-14T08:38:49.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.033+0000 7fda4d308640 1 Processor -- start 2024-04-14T08:38:49.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.033+0000 7fda4d308640 1 -- start start 2024-04-14T08:38:49.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda4d308640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda4815f7e0 0x7fda4815d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda4d308640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fda480b1b80 0x7fda480b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda4d308640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda4813f600 con 0x7fda4815f400 2024-04-14T08:38:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda4d308640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fda480b2520 con 0x7fda480b1b80 2024-04-14T08:38:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda4d308640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fda48140690 con 0x7fda4815f7e0 2024-04-14T08:38:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda477fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fda480b1b80 0x7fda480b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda46ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fda4815f400 0x7fda4815d190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45768/0 (socket says 172.21.15.172:45768) 2024-04-14T08:38:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda477fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fda480b1b80 0x7fda480b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49834/0 (socket says 172.21.15.172:49834) 2024-04-14T08:38:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda46ffd640 1 -- 172.21.15.172:0/2066387338 learned_addr learned my addr 172.21.15.172:0/2066387338 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda467fc640 1 --2- 172.21.15.172:0/2066387338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda4815f7e0 0x7fda4815d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda477fe640 1 -- 172.21.15.172:0/2066387338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda4815f7e0 msgr2=0x7fda4815d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda477fe640 1 --2- 172.21.15.172:0/2066387338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda4815f7e0 0x7fda4815d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda477fe640 1 -- 172.21.15.172:0/2066387338 >> v1:172.21.15.172:6789/0 conn(0x7fda4815f400 legacy=0x7fda4815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda477fe640 1 -- 172.21.15.172:0/2066387338 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4813fa80 con 0x7fda480b1b80 2024-04-14T08:38:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda45ffb640 1 -- 172.21.15.172:0/2066387338 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367652482 0 0) 0x7fda38001110 con 0x7fda4815f400 2024-04-14T08:38:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.034+0000 7fda467fc640 1 --2- 172.21.15.172:0/2066387338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda4815f7e0 0x7fda4815d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.035+0000 7fda477fe640 1 --2- 172.21.15.172:0/2066387338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda480b1b80 0x7fda480b1fe0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fda40009d90 tx=0x7fda4002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f8749fb18000e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.035+0000 7fda45ffb640 1 -- 172.21.15.172:0/2066387338 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40039070 con 0x7fda480b1b80 2024-04-14T08:38:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.035+0000 7fda45ffb640 1 -- 172.21.15.172:0/2066387338 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4002cc50 con 0x7fda480b1b80 2024-04-14T08:38:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.035+0000 7fda45ffb640 1 -- 172.21.15.172:0/2066387338 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40034440 con 0x7fda480b1b80 2024-04-14T08:38:49.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.036+0000 7fda4d308640 1 -- 172.21.15.172:0/2066387338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda480b1b80 msgr2=0x7fda480b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.036+0000 7fda4d308640 1 --2- 172.21.15.172:0/2066387338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda480b1b80 0x7fda480b1fe0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fda40009d90 tx=0x7fda4002bea0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.036+0000 7fda4d308640 1 -- 172.21.15.172:0/2066387338 shutdown_connections 2024-04-14T08:38:49.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.036+0000 7fda4d308640 1 --2- 172.21.15.172:0/2066387338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda4815f7e0 0x7fda4815d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.036+0000 7fda4d308640 1 --2- 172.21.15.172:0/2066387338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda480b1b80 0x7fda480b1fe0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.036+0000 7fda4d308640 1 -- 172.21.15.172:0/2066387338 >> 172.21.15.172:0/2066387338 conn(0x7fda4806ec30 msgr2=0x7fda4806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:49.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.036+0000 7fda4d308640 1 -- 172.21.15.172:0/2066387338 shutdown_connections 2024-04-14T08:38:49.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.036+0000 7fda4d308640 1 -- 172.21.15.172:0/2066387338 wait complete. 2024-04-14T08:38:49.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.037+0000 7fda4d308640 1 Processor -- start 2024-04-14T08:38:49.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.037+0000 7fda4d308640 1 -- start start 2024-04-14T08:38:49.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.038+0000 7fda4d308640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda480b1b80 0x7fda481f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:49.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.038+0000 7fda4d308640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4815f400 0x7fda481f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.038+0000 7fda4d308640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda4815f7e0 0x7fda481e5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.038+0000 7fda4d308640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda48140040 con 0x7fda4815f7e0 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.038+0000 7fda4d308640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fda4815df10 con 0x7fda4815f400 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.038+0000 7fda4d308640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fda48140320 con 0x7fda480b1b80 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.038+0000 7fda477fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda4815f7e0 0x7fda481e5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.038+0000 7fda477fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda4815f7e0 0x7fda481e5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49850/0 (socket says 172.21.15.172:49850) 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.038+0000 7fda477fe640 1 -- 172.21.15.172:0/3716762741 learned_addr learned my addr 172.21.15.172:0/3716762741 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.038+0000 7fda467fc640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4815f400 0x7fda481f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.038+0000 7fda46ffd640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda480b1b80 0x7fda481f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.039+0000 7fda477fe640 1 -- 172.21.15.172:0/3716762741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda480b1b80 msgr2=0x7fda481f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.039+0000 7fda477fe640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda480b1b80 0x7fda481f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.039+0000 7fda477fe640 1 -- 172.21.15.172:0/3716762741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4815f400 msgr2=0x7fda481f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.039+0000 7fda477fe640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4815f400 0x7fda481f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.039+0000 7fda477fe640 1 -- 172.21.15.172:0/3716762741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda481f28e0 con 0x7fda4815f7e0 2024-04-14T08:38:49.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.039+0000 7fda477fe640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda4815f7e0 0x7fda481e5360 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fda40009eb0 tx=0x7fda40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:49.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.039+0000 7fda2f7fe640 1 -- 172.21.15.172:0/3716762741 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40039050 con 0x7fda4815f7e0 2024-04-14T08:38:49.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.039+0000 7fda2f7fe640 1 -- 172.21.15.172:0/3716762741 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda40045070 con 0x7fda4815f7e0 2024-04-14T08:38:49.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.039+0000 7fda2f7fe640 1 -- 172.21.15.172:0/3716762741 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40009450 con 0x7fda4815f7e0 2024-04-14T08:38:49.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.039+0000 7fda4d308640 1 -- 172.21.15.172:0/3716762741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda481e5d40 con 0x7fda4815f7e0 2024-04-14T08:38:49.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.039+0000 7fda4d308640 1 -- 172.21.15.172:0/3716762741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda48153d70 con 0x7fda4815f7e0 2024-04-14T08:38:49.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.040+0000 7fda4d308640 1 -- 172.21.15.172:0/3716762741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda48068520 con 0x7fda4815f7e0 2024-04-14T08:38:49.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.040+0000 7fda2f7fe640 1 -- 172.21.15.172:0/3716762741 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda4003e610 con 0x7fda4815f7e0 2024-04-14T08:38:49.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.041+0000 7fda2f7fe640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda3406c8f0 0x7fda3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:49.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.041+0000 7fda2f7fe640 1 -- 172.21.15.172:0/3716762741 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fda400ae540 con 0x7fda4815f7e0 2024-04-14T08:38:49.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.041+0000 7fda46ffd640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda3406c8f0 0x7fda3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:49.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.041+0000 7fda46ffd640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda3406c8f0 0x7fda3406edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fda380013b0 tx=0x7fda38007b90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:49.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.045+0000 7fda2f7fe640 1 -- 172.21.15.172:0/3716762741 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fda4007c220 con 0x7fda4815f7e0 2024-04-14T08:38:49.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.291+0000 7fda4d308640 1 -- 172.21.15.172:0/3716762741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fda480d0110 con 0x7fda4815f7e0 2024-04-14T08:38:49.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.291+0000 7fda2f7fe640 1 -- 172.21.15.172:0/3716762741 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1260 (secure 0 0 0) 0x7fda40080520 con 0x7fda4815f7e0 2024-04-14T08:38:49.293 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:49.293 INFO:teuthology.orchestra.run.smithi172.stdout:{"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":380,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":86,"num_osds":8,"num_up_osds":8,"osd_up_since":1713083560,"num_in_osds":8,"osd_in_since":1713083551,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":59},{"state_name":"active+clean","count":38}],"num_pgs":97,"num_pools":3,"num_objects":7,"data_bytes":590402,"bytes_used":252166144,"bytes_avail":772841947136,"bytes_total":773094113280,"inactive_pgs_ratio":0.60824739933013916,"write_bytes_sec":1196,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":54,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7491}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-14T08:38:07.183549+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-14T08:38:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.293+0000 7fda4d308640 1 -- 172.21.15.172:0/3716762741 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda3406c8f0 msgr2=0x7fda3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.293+0000 7fda4d308640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda3406c8f0 0x7fda3406edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fda380013b0 tx=0x7fda38007b90 comp rx=0 tx=0).stop 2024-04-14T08:38:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.293+0000 7fda4d308640 1 -- 172.21.15.172:0/3716762741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda4815f7e0 msgr2=0x7fda481e5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.293+0000 7fda4d308640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda4815f7e0 0x7fda481e5360 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fda40009eb0 tx=0x7fda40004a40 comp rx=0 tx=0).stop 2024-04-14T08:38:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.293+0000 7fda4d308640 1 -- 172.21.15.172:0/3716762741 shutdown_connections 2024-04-14T08:38:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.293+0000 7fda4d308640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda3406c8f0 0x7fda3406edb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.293+0000 7fda4d308640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda4815f7e0 0x7fda481e5360 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.293+0000 7fda4d308640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4815f400 0x7fda481f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.293+0000 7fda4d308640 1 --2- 172.21.15.172:0/3716762741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda480b1b80 0x7fda481f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.293+0000 7fda4d308640 1 -- 172.21.15.172:0/3716762741 >> 172.21.15.172:0/3716762741 conn(0x7fda4806ec30 msgr2=0x7fda48071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.293+0000 7fda4d308640 1 -- 172.21.15.172:0/3716762741 shutdown_connections 2024-04-14T08:38:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.293+0000 7fda4d308640 1 -- 172.21.15.172:0/3716762741 wait complete. 2024-04-14T08:38:49.305 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-14T08:38:49.306 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:38:49.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.404+0000 7fb7fe7c8640 1 Processor -- start 2024-04-14T08:38:49.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.404+0000 7fb7fe7c8640 1 -- start start 2024-04-14T08:38:49.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7fe7c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7f80b4040 0x7fb7f80b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:49.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7fe7c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f80b49f0 con 0x7fb7f80b21d0 2024-04-14T08:38:49.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7fe7c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f815c770 con 0x7fb7f80b1df0 2024-04-14T08:38:49.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7fe7c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f80b3700 con 0x7fb7f80b4040 2024-04-14T08:38:49.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7f77fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb7f80b21d0 0x7fb7f815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45780/0 (socket says 172.21.15.172:45780) 2024-04-14T08:38:49.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7f77fe640 1 -- 172.21.15.172:0/699027362 learned_addr learned my addr 172.21.15.172:0/699027362 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:49.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7fcd65640 1 --2- 172.21.15.172:0/699027362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7f80b4040 0x7fb7f80b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:49.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7f6ffd640 1 -- 172.21.15.172:0/699027362 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 962514783 0 0) 0x7fb7f80b49f0 con 0x7fb7f80b21d0 2024-04-14T08:38:49.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7f6ffd640 1 -- 172.21.15.172:0/699027362 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7e0003520 con 0x7fb7f80b21d0 2024-04-14T08:38:49.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7f6ffd640 1 -- 172.21.15.172:0/699027362 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3343929019 0 0) 0x7fb7f815c770 con 0x7fb7f80b1df0 2024-04-14T08:38:49.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7f6ffd640 1 -- 172.21.15.172:0/699027362 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7f80b49f0 con 0x7fb7f80b1df0 2024-04-14T08:38:49.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7fcd65640 1 -- 172.21.15.172:0/699027362 >> v1:172.21.15.177:6789/0 conn(0x7fb7f80b1df0 legacy=0x7fb7f80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7fcd65640 1 -- 172.21.15.172:0/699027362 >> v1:172.21.15.172:6789/0 conn(0x7fb7f80b21d0 legacy=0x7fb7f815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7f6ffd640 1 -- 172.21.15.172:0/699027362 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1259909812 0 0) 0x7fb7e0003520 con 0x7fb7f80b21d0 2024-04-14T08:38:49.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.405+0000 7fb7fcd65640 1 -- 172.21.15.172:0/699027362 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f80b39e0 con 0x7fb7f80b4040 2024-04-14T08:38:49.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.406+0000 7fb7fcd65640 1 --2- 172.21.15.172:0/699027362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7f80b4040 0x7fb7f80b44b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009dc0 tx=0x7fb7f002bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d898dff6d6d2a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:49.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.406+0000 7fb7f6ffd640 1 -- 172.21.15.172:0/699027362 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0039070 con 0x7fb7f80b4040 2024-04-14T08:38:49.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.406+0000 7fb7f6ffd640 1 -- 172.21.15.172:0/699027362 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f002c910 con 0x7fb7f80b4040 2024-04-14T08:38:49.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.406+0000 7fb7f6ffd640 1 -- 172.21.15.172:0/699027362 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec001670 con 0x7fb7f80b4040 2024-04-14T08:38:49.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.407+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/699027362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7f80b4040 msgr2=0x7fb7f80b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.407+0000 7fb7fe7c8640 1 --2- 172.21.15.172:0/699027362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7f80b4040 0x7fb7f80b44b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009dc0 tx=0x7fb7f002bde0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.407+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/699027362 shutdown_connections 2024-04-14T08:38:49.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.407+0000 7fb7fe7c8640 1 --2- 172.21.15.172:0/699027362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7f80b4040 0x7fb7f80b44b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.407+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/699027362 >> 172.21.15.172:0/699027362 conn(0x7fb7f806ec30 msgr2=0x7fb7f806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:49.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.407+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/699027362 shutdown_connections 2024-04-14T08:38:49.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.407+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/699027362 wait complete. 2024-04-14T08:38:49.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.408+0000 7fb7fe7c8640 1 Processor -- start 2024-04-14T08:38:49.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.408+0000 7fb7fe7c8640 1 -- start start 2024-04-14T08:38:49.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7fe7c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7f80b1df0 0x7fb7f8151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:49.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7fe7c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7f80b21d0 0x7fb7f80b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:49.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7fe7c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7f80b4040 0x7fb7f8156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:49.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7fe7c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f815b7f0 con 0x7fb7f80b21d0 2024-04-14T08:38:49.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7fe7c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f815bb40 con 0x7fb7f80b4040 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7fe7c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f81552d0 con 0x7fb7f80b1df0 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7f80b21d0 0x7fb7f80b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7f80b21d0 0x7fb7f80b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49852/0 (socket says 172.21.15.172:49852) 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7f77fe640 1 -- 172.21.15.172:0/1152657125 learned_addr learned my addr 172.21.15.172:0/1152657125 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7f7fff640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7f80b1df0 0x7fb7f8151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7fcd65640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7f80b4040 0x7fb7f8156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7f77fe640 1 -- 172.21.15.172:0/1152657125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7f80b1df0 msgr2=0x7fb7f8151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7f77fe640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7f80b1df0 0x7fb7f8151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7f77fe640 1 -- 172.21.15.172:0/1152657125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7f80b4040 msgr2=0x7fb7f8156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7f77fe640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7f80b4040 0x7fb7f8156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.409+0000 7fb7f77fe640 1 -- 172.21.15.172:0/1152657125 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f80aa6b0 con 0x7fb7f80b21d0 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.410+0000 7fb7f77fe640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7f80b21d0 0x7fb7f80b03c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c750 tx=0x7fb7ec00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:49.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.410+0000 7fb7f4ff9640 1 -- 172.21.15.172:0/1152657125 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec017070 con 0x7fb7f80b21d0 2024-04-14T08:38:49.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.410+0000 7fb7f4ff9640 1 -- 172.21.15.172:0/1152657125 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7ec005ad0 con 0x7fb7f80b21d0 2024-04-14T08:38:49.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.410+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/1152657125 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f80aa950 con 0x7fb7f80b21d0 2024-04-14T08:38:49.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.410+0000 7fb7f4ff9640 1 -- 172.21.15.172:0/1152657125 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec007070 con 0x7fb7f80b21d0 2024-04-14T08:38:49.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.410+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/1152657125 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f81556c0 con 0x7fb7f80b21d0 2024-04-14T08:38:49.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.411+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/1152657125 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f8067dd0 con 0x7fb7f80b21d0 2024-04-14T08:38:49.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.411+0000 7fb7f4ff9640 1 -- 172.21.15.172:0/1152657125 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7ec01a070 con 0x7fb7f80b21d0 2024-04-14T08:38:49.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.412+0000 7fb7f4ff9640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7e406c8f0 0x7fb7e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:49.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.412+0000 7fb7f4ff9640 1 -- 172.21.15.172:0/1152657125 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fb7ec08b0b0 con 0x7fb7f80b21d0 2024-04-14T08:38:49.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.412+0000 7fb7f7fff640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7e406c8f0 0x7fb7e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:49.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.412+0000 7fb7f7fff640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7e406c8f0 0x7fb7e406edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80011e0 tx=0x7fb7e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:49.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.416+0000 7fb7f4ff9640 1 -- 172.21.15.172:0/1152657125 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb7f8067dd0 con 0x7fb7f80b21d0 2024-04-14T08:38:49.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.588+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/1152657125 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb7f8068520 con 0x7fb7e406c8f0 2024-04-14T08:38:49.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.593+0000 7fb7f4ff9640 1 -- 172.21.15.172:0/1152657125 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247193 (secure 0 0 0) 0x7fb7f8068520 con 0x7fb7e406c8f0 2024-04-14T08:38:49.597 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:49.599 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:38:49.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.597+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/1152657125 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7e406c8f0 msgr2=0x7fb7e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.597+0000 7fb7fe7c8640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7e406c8f0 0x7fb7e406edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80011e0 tx=0x7fb7e8009040 comp rx=0 tx=0).stop 2024-04-14T08:38:49.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.597+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/1152657125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7f80b21d0 msgr2=0x7fb7f80b03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:49.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.597+0000 7fb7fe7c8640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7f80b21d0 0x7fb7f80b03c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c750 tx=0x7fb7ec00cc10 comp rx=0 tx=0).stop 2024-04-14T08:38:49.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.597+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/1152657125 shutdown_connections 2024-04-14T08:38:49.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.597+0000 7fb7fe7c8640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7e406c8f0 0x7fb7e406edb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.597+0000 7fb7fe7c8640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7f80b4040 0x7fb7f8156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.597+0000 7fb7fe7c8640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7f80b21d0 0x7fb7f80b03c0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.597+0000 7fb7fe7c8640 1 --2- 172.21.15.172:0/1152657125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7f80b1df0 0x7fb7f8151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:49.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.597+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/1152657125 >> 172.21.15.172:0/1152657125 conn(0x7fb7f806ec30 msgr2=0x7fb7f8074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:49.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.597+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/1152657125 shutdown_connections 2024-04-14T08:38:49.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:49.597+0000 7fb7fe7c8640 1 -- 172.21.15.172:0/1152657125 wait complete. 2024-04-14T08:38:49.609 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-04-14T08:38:49.191611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590402,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246256,"kb_used_data":9264,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754728464,"statfs":{"total":773094113280,"available":772841947136,"internally_reserved":0,"allocated":9486336,"data_stored":4952124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.989183"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216414+0000","last_change":"2024-04-14T08:38:44.216414+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216414+0000","last_undegraded":"2024-04-14T08:38:44.216414+0000","last_fullsized":"2024-04-14T08:38:44.216414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217436+0000","last_change":"2024-04-14T08:38:44.217436+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217436+0000","last_undegraded":"2024-04-14T08:38:44.217436+0000","last_fullsized":"2024-04-14T08:38:44.217436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215977+0000","last_change":"2024-04-14T08:38:44.215977+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215977+0000","last_undegraded":"2024-04-14T08:38:44.215977+0000","last_fullsized":"2024-04-14T08:38:44.215977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.220133+0000","last_change":"2024-04-14T08:38:44.220133+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.220133+0000","last_undegraded":"2024-04-14T08:38:44.220133+0000","last_fullsized":"2024-04-14T08:38:44.220133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211723+0000","last_change":"2024-04-14T08:38:44.211723+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211723+0000","last_undegraded":"2024-04-14T08:38:44.211723+0000","last_fullsized":"2024-04-14T08:38:44.211723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.228102+0000","last_change":"2024-04-14T08:38:44.228102+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.228102+0000","last_undegraded":"2024-04-14T08:38:44.228102+0000","last_fullsized":"2024-04-14T08:38:44.228102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.226673+0000","last_change":"2024-04-14T08:38:45.221574+0000","last_active":"2024-04-14T08:38:45.226673+0000","last_peered":"2024-04-14T08:38:45.226673+0000","last_clean":"2024-04-14T08:38:45.226673+0000","last_became_active":"2024-04-14T08:38:45.221303+0000","last_became_peered":"2024-04-14T08:38:45.221303+0000","last_unstale":"2024-04-14T08:38:45.226673+0000","last_undegraded":"2024-04-14T08:38:45.226673+0000","last_fullsized":"2024-04-14T08:38:45.226673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:59:38.604514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.219635+0000","last_change":"2024-04-14T08:38:44.219635+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.219635+0000","last_undegraded":"2024-04-14T08:38:44.219635+0000","last_fullsized":"2024-04-14T08:38:44.219635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.212588+0000","last_change":"2024-04-14T08:38:44.212588+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.212588+0000","last_undegraded":"2024-04-14T08:38:44.212588+0000","last_fullsized":"2024-04-14T08:38:44.212588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216941+0000","last_change":"2024-04-14T08:38:44.216941+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216941+0000","last_undegraded":"2024-04-14T08:38:44.216941+0000","last_fullsized":"2024-04-14T08:38:44.216941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211782+0000","last_change":"2024-04-14T08:38:44.211782+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211782+0000","last_undegraded":"2024-04-14T08:38:44.211782+0000","last_fullsized":"2024-04-14T08:38:44.211782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216683+0000","last_change":"2024-04-14T08:38:44.216683+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216683+0000","last_undegraded":"2024-04-14T08:38:44.216683+0000","last_fullsized":"2024-04-14T08:38:44.216683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.220232+0000","last_change":"2024-04-14T08:38:44.220232+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.220232+0000","last_undegraded":"2024-04-14T08:38:44.220232+0000","last_fullsized":"2024-04-14T08:38:44.220232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.213829+0000","last_change":"2024-04-14T08:38:44.213829+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.213829+0000","last_undegraded":"2024-04-14T08:38:44.213829+0000","last_fullsized":"2024-04-14T08:38:44.213829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.219502+0000","last_change":"2024-04-14T08:38:44.219502+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.219502+0000","last_undegraded":"2024-04-14T08:38:44.219502+0000","last_fullsized":"2024-04-14T08:38:44.219502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.228755+0000","last_change":"2024-04-14T08:38:44.228755+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.228755+0000","last_undegraded":"2024-04-14T08:38:44.228755+0000","last_fullsized":"2024-04-14T08:38:44.228755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.227961+0000","last_change":"2024-04-14T08:38:44.227961+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.227961+0000","last_undegraded":"2024-04-14T08:38:44.227961+0000","last_fullsized":"2024-04-14T08:38:44.227961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215464+0000","last_change":"2024-04-14T08:38:44.215464+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215464+0000","last_undegraded":"2024-04-14T08:38:44.215464+0000","last_fullsized":"2024-04-14T08:38:44.215464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211266+0000","last_change":"2024-04-14T08:38:44.211266+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211266+0000","last_undegraded":"2024-04-14T08:38:44.211266+0000","last_fullsized":"2024-04-14T08:38:44.211266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218880+0000","last_change":"2024-04-14T08:38:44.218880+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218880+0000","last_undegraded":"2024-04-14T08:38:44.218880+0000","last_fullsized":"2024-04-14T08:38:44.218880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.212313+0000","last_change":"2024-04-14T08:38:44.212313+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.212313+0000","last_undegraded":"2024-04-14T08:38:44.212313+0000","last_fullsized":"2024-04-14T08:38:44.212313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.213369+0000","last_change":"2024-04-14T08:38:44.213369+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.213369+0000","last_undegraded":"2024-04-14T08:38:44.213369+0000","last_fullsized":"2024-04-14T08:38:44.213369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218241+0000","last_change":"2024-04-14T08:38:44.218241+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218241+0000","last_undegraded":"2024-04-14T08:38:44.218241+0000","last_fullsized":"2024-04-14T08:38:44.218241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215033+0000","last_change":"2024-04-14T08:38:44.215033+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215033+0000","last_undegraded":"2024-04-14T08:38:44.215033+0000","last_fullsized":"2024-04-14T08:38:44.215033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216786+0000","last_change":"2024-04-14T08:38:44.216786+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216786+0000","last_undegraded":"2024-04-14T08:38:44.216786+0000","last_fullsized":"2024-04-14T08:38:44.216786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.213997+0000","last_change":"2024-04-14T08:38:44.213997+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.213997+0000","last_undegraded":"2024-04-14T08:38:44.213997+0000","last_fullsized":"2024-04-14T08:38:44.213997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216757+0000","last_change":"2024-04-14T08:38:44.216757+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216757+0000","last_undegraded":"2024-04-14T08:38:44.216757+0000","last_fullsized":"2024-04-14T08:38:44.216757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218369+0000","last_change":"2024-04-14T08:38:44.218369+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218369+0000","last_undegraded":"2024-04-14T08:38:44.218369+0000","last_fullsized":"2024-04-14T08:38:44.218369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.229290+0000","last_change":"2024-04-14T08:38:45.226642+0000","last_active":"2024-04-14T08:38:45.229290+0000","last_peered":"2024-04-14T08:38:45.229290+0000","last_clean":"2024-04-14T08:38:45.229290+0000","last_became_active":"2024-04-14T08:38:45.226349+0000","last_became_peered":"2024-04-14T08:38:45.226349+0000","last_unstale":"2024-04-14T08:38:45.229290+0000","last_undegraded":"2024-04-14T08:38:45.229290+0000","last_fullsized":"2024-04-14T08:38:45.229290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:17:06.164898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211114+0000","last_change":"2024-04-14T08:38:44.211114+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211114+0000","last_undegraded":"2024-04-14T08:38:44.211114+0000","last_fullsized":"2024-04-14T08:38:44.211114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217550+0000","last_change":"2024-04-14T08:38:44.217550+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217550+0000","last_undegraded":"2024-04-14T08:38:44.217550+0000","last_fullsized":"2024-04-14T08:38:44.217550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.228329+0000","last_change":"2024-04-14T08:38:44.228329+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.228329+0000","last_undegraded":"2024-04-14T08:38:44.228329+0000","last_fullsized":"2024-04-14T08:38:44.228329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.225245+0000","last_change":"2024-04-14T08:38:44.225245+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.225245+0000","last_undegraded":"2024-04-14T08:38:44.225245+0000","last_fullsized":"2024-04-14T08:38:44.225245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228182+0000","last_change":"2024-04-14T08:38:44.227154+0000","last_active":"2024-04-14T08:38:44.228182+0000","last_peered":"2024-04-14T08:38:44.228182+0000","last_clean":"2024-04-14T08:38:44.228182+0000","last_became_active":"2024-04-14T08:38:44.226910+0000","last_became_peered":"2024-04-14T08:38:44.226910+0000","last_unstale":"2024-04-14T08:38:44.228182+0000","last_undegraded":"2024-04-14T08:38:44.228182+0000","last_fullsized":"2024-04-14T08:38:44.228182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:49:42.403181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.227247+0000","last_change":"2024-04-14T08:38:44.227247+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.227247+0000","last_undegraded":"2024-04-14T08:38:44.227247+0000","last_fullsized":"2024-04-14T08:38:44.227247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225223+0000","last_change":"2024-04-14T08:38:44.222545+0000","last_active":"2024-04-14T08:38:44.225223+0000","last_peered":"2024-04-14T08:38:44.225223+0000","last_clean":"2024-04-14T08:38:44.225223+0000","last_became_active":"2024-04-14T08:38:44.221999+0000","last_became_peered":"2024-04-14T08:38:44.221999+0000","last_unstale":"2024-04-14T08:38:44.225223+0000","last_undegraded":"2024-04-14T08:38:44.225223+0000","last_fullsized":"2024-04-14T08:38:44.225223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:37:59.212931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.235826+0000","last_change":"2024-04-14T08:38:45.228983+0000","last_active":"2024-04-14T08:38:45.235826+0000","last_peered":"2024-04-14T08:38:45.235826+0000","last_clean":"2024-04-14T08:38:45.235826+0000","last_became_active":"2024-04-14T08:38:45.228632+0000","last_became_peered":"2024-04-14T08:38:45.228632+0000","last_unstale":"2024-04-14T08:38:45.235826+0000","last_undegraded":"2024-04-14T08:38:45.235826+0000","last_fullsized":"2024-04-14T08:38:45.235826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:17:19.947504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.240108+0000","last_change":"2024-04-14T08:38:44.214063+0000","last_active":"2024-04-14T08:38:45.240108+0000","last_peered":"2024-04-14T08:38:45.240108+0000","last_clean":"2024-04-14T08:38:45.240108+0000","last_became_active":"2024-04-14T08:38:44.213612+0000","last_became_peered":"2024-04-14T08:38:44.213612+0000","last_unstale":"2024-04-14T08:38:45.240108+0000","last_undegraded":"2024-04-14T08:38:45.240108+0000","last_fullsized":"2024-04-14T08:38:45.240108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:15:04.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.227051+0000","last_change":"2024-04-14T08:38:44.227051+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.227051+0000","last_undegraded":"2024-04-14T08:38:44.227051+0000","last_fullsized":"2024-04-14T08:38:44.227051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228169+0000","last_change":"2024-04-14T08:38:44.227021+0000","last_active":"2024-04-14T08:38:44.228169+0000","last_peered":"2024-04-14T08:38:44.228169+0000","last_clean":"2024-04-14T08:38:44.228169+0000","last_became_active":"2024-04-14T08:38:44.226731+0000","last_became_peered":"2024-04-14T08:38:44.226731+0000","last_unstale":"2024-04-14T08:38:44.228169+0000","last_undegraded":"2024-04-14T08:38:44.228169+0000","last_fullsized":"2024-04-14T08:38:44.228169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:33:01.799418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217415+0000","last_change":"2024-04-14T08:38:44.217415+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217415+0000","last_undegraded":"2024-04-14T08:38:44.217415+0000","last_fullsized":"2024-04-14T08:38:44.217415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225249+0000","last_change":"2024-04-14T08:38:44.222167+0000","last_active":"2024-04-14T08:38:44.225249+0000","last_peered":"2024-04-14T08:38:44.225249+0000","last_clean":"2024-04-14T08:38:44.225249+0000","last_became_active":"2024-04-14T08:38:44.221892+0000","last_became_peered":"2024-04-14T08:38:44.221892+0000","last_unstale":"2024-04-14T08:38:44.225249+0000","last_undegraded":"2024-04-14T08:38:44.225249+0000","last_fullsized":"2024-04-14T08:38:44.225249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:13.870437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217818+0000","last_change":"2024-04-14T08:38:44.217818+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217818+0000","last_undegraded":"2024-04-14T08:38:44.217818+0000","last_fullsized":"2024-04-14T08:38:44.217818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226141+0000","last_change":"2024-04-14T08:38:44.222906+0000","last_active":"2024-04-14T08:38:44.226141+0000","last_peered":"2024-04-14T08:38:44.226141+0000","last_clean":"2024-04-14T08:38:44.226141+0000","last_became_active":"2024-04-14T08:38:44.222541+0000","last_became_peered":"2024-04-14T08:38:44.222541+0000","last_unstale":"2024-04-14T08:38:44.226141+0000","last_undegraded":"2024-04-14T08:38:44.226141+0000","last_fullsized":"2024-04-14T08:38:44.226141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:21:35.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.221823+0000","last_change":"2024-04-14T08:38:44.221823+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.221823+0000","last_undegraded":"2024-04-14T08:38:44.221823+0000","last_fullsized":"2024-04-14T08:38:44.221823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.223396+0000","last_change":"2024-04-14T08:38:44.218254+0000","last_active":"2024-04-14T08:38:44.223396+0000","last_peered":"2024-04-14T08:38:44.223396+0000","last_clean":"2024-04-14T08:38:44.223396+0000","last_became_active":"2024-04-14T08:38:44.217969+0000","last_became_peered":"2024-04-14T08:38:44.217969+0000","last_unstale":"2024-04-14T08:38:44.223396+0000","last_undegraded":"2024-04-14T08:38:44.223396+0000","last_fullsized":"2024-04-14T08:38:44.223396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:12:14.185232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217899+0000","last_change":"2024-04-14T08:38:44.217899+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217899+0000","last_undegraded":"2024-04-14T08:38:44.217899+0000","last_fullsized":"2024-04-14T08:38:44.217899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.214121+0000","last_change":"2024-04-14T08:38:44.231501+0000","last_active":"2024-04-14T08:38:45.214121+0000","last_peered":"2024-04-14T08:38:45.214121+0000","last_clean":"2024-04-14T08:38:45.214121+0000","last_became_active":"2024-04-14T08:38:44.231184+0000","last_became_peered":"2024-04-14T08:38:44.231184+0000","last_unstale":"2024-04-14T08:38:45.214121+0000","last_undegraded":"2024-04-14T08:38:45.214121+0000","last_fullsized":"2024-04-14T08:38:45.214121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:59:08.058016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.212060+0000","last_change":"2024-04-14T08:38:44.212060+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.212060+0000","last_undegraded":"2024-04-14T08:38:44.212060+0000","last_fullsized":"2024-04-14T08:38:44.212060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225545+0000","last_change":"2024-04-14T08:38:44.224713+0000","last_active":"2024-04-14T08:38:44.225545+0000","last_peered":"2024-04-14T08:38:44.225545+0000","last_clean":"2024-04-14T08:38:44.225545+0000","last_became_active":"2024-04-14T08:38:44.224437+0000","last_became_peered":"2024-04-14T08:38:44.224437+0000","last_unstale":"2024-04-14T08:38:44.225545+0000","last_undegraded":"2024-04-14T08:38:44.225545+0000","last_fullsized":"2024-04-14T08:38:44.225545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:37.147141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.221898+0000","last_change":"2024-04-14T08:38:45.219595+0000","last_active":"2024-04-14T08:38:45.221898+0000","last_peered":"2024-04-14T08:38:45.221898+0000","last_clean":"2024-04-14T08:38:45.221898+0000","last_became_active":"2024-04-14T08:38:45.219269+0000","last_became_peered":"2024-04-14T08:38:45.219269+0000","last_unstale":"2024-04-14T08:38:45.221898+0000","last_undegraded":"2024-04-14T08:38:45.221898+0000","last_fullsized":"2024-04-14T08:38:45.221898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:48:28.586545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.224564+0000","last_change":"2024-04-14T08:38:44.219411+0000","last_active":"2024-04-14T08:38:44.224564+0000","last_peered":"2024-04-14T08:38:44.224564+0000","last_clean":"2024-04-14T08:38:44.224564+0000","last_became_active":"2024-04-14T08:38:44.219130+0000","last_became_peered":"2024-04-14T08:38:44.219130+0000","last_unstale":"2024-04-14T08:38:44.224564+0000","last_undegraded":"2024-04-14T08:38:44.224564+0000","last_fullsized":"2024-04-14T08:38:44.224564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:43:02.184142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.214550+0000","last_change":"2024-04-14T08:38:44.214550+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.214550+0000","last_undegraded":"2024-04-14T08:38:44.214550+0000","last_fullsized":"2024-04-14T08:38:44.214550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233151+0000","last_change":"2024-04-14T08:38:44.231600+0000","last_active":"2024-04-14T08:38:44.233151+0000","last_peered":"2024-04-14T08:38:44.233151+0000","last_clean":"2024-04-14T08:38:44.233151+0000","last_became_active":"2024-04-14T08:38:44.231276+0000","last_became_peered":"2024-04-14T08:38:44.231276+0000","last_unstale":"2024-04-14T08:38:44.233151+0000","last_undegraded":"2024-04-14T08:38:44.233151+0000","last_fullsized":"2024-04-14T08:38:44.233151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:42.226080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216192+0000","last_change":"2024-04-14T08:38:44.216192+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216192+0000","last_undegraded":"2024-04-14T08:38:44.216192+0000","last_fullsized":"2024-04-14T08:38:44.216192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233087+0000","last_change":"2024-04-14T08:38:44.230303+0000","last_active":"2024-04-14T08:38:44.233087+0000","last_peered":"2024-04-14T08:38:44.233087+0000","last_clean":"2024-04-14T08:38:44.233087+0000","last_became_active":"2024-04-14T08:38:44.230051+0000","last_became_peered":"2024-04-14T08:38:44.230051+0000","last_unstale":"2024-04-14T08:38:44.233087+0000","last_undegraded":"2024-04-14T08:38:44.233087+0000","last_fullsized":"2024-04-14T08:38:44.233087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:36:12.417551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216476+0000","last_change":"2024-04-14T08:38:44.216476+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216476+0000","last_undegraded":"2024-04-14T08:38:44.216476+0000","last_fullsized":"2024-04-14T08:38:44.216476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.245313+0000","last_change":"2024-04-14T08:38:44.230951+0000","last_active":"2024-04-14T08:38:45.245313+0000","last_peered":"2024-04-14T08:38:45.245313+0000","last_clean":"2024-04-14T08:38:45.245313+0000","last_became_active":"2024-04-14T08:38:44.230499+0000","last_became_peered":"2024-04-14T08:38:44.230499+0000","last_unstale":"2024-04-14T08:38:45.245313+0000","last_undegraded":"2024-04-14T08:38:45.245313+0000","last_fullsized":"2024-04-14T08:38:45.245313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:55:56.125870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.210337+0000","last_change":"2024-04-14T08:38:44.210337+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.210337+0000","last_undegraded":"2024-04-14T08:38:44.210337+0000","last_fullsized":"2024-04-14T08:38:44.210337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233084+0000","last_change":"2024-04-14T08:38:44.230300+0000","last_active":"2024-04-14T08:38:44.233084+0000","last_peered":"2024-04-14T08:38:44.233084+0000","last_clean":"2024-04-14T08:38:44.233084+0000","last_became_active":"2024-04-14T08:38:44.230024+0000","last_became_peered":"2024-04-14T08:38:44.230024+0000","last_unstale":"2024-04-14T08:38:44.233084+0000","last_undegraded":"2024-04-14T08:38:44.233084+0000","last_fullsized":"2024-04-14T08:38:44.233084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:15:04.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216501+0000","last_change":"2024-04-14T08:38:44.216501+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216501+0000","last_undegraded":"2024-04-14T08:38:44.216501+0000","last_fullsized":"2024-04-14T08:38:44.216501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.227819+0000","last_change":"2024-04-14T08:38:44.224811+0000","last_active":"2024-04-14T08:38:44.227819+0000","last_peered":"2024-04-14T08:38:44.227819+0000","last_clean":"2024-04-14T08:38:44.227819+0000","last_became_active":"2024-04-14T08:38:44.224541+0000","last_became_peered":"2024-04-14T08:38:44.224541+0000","last_unstale":"2024-04-14T08:38:44.227819+0000","last_undegraded":"2024-04-14T08:38:44.227819+0000","last_fullsized":"2024-04-14T08:38:44.227819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:31:11.641961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217216+0000","last_change":"2024-04-14T08:38:44.217216+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217216+0000","last_undegraded":"2024-04-14T08:38:44.217216+0000","last_fullsized":"2024-04-14T08:38:44.217216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.217797+0000","last_change":"2024-04-14T08:38:44.216083+0000","last_active":"2024-04-14T08:38:44.217797+0000","last_peered":"2024-04-14T08:38:44.217797+0000","last_clean":"2024-04-14T08:38:44.217797+0000","last_became_active":"2024-04-14T08:38:44.215754+0000","last_became_peered":"2024-04-14T08:38:44.215754+0000","last_unstale":"2024-04-14T08:38:44.217797+0000","last_undegraded":"2024-04-14T08:38:44.217797+0000","last_fullsized":"2024-04-14T08:38:44.217797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:30:13.870437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.222718+0000","last_change":"2024-04-14T08:38:44.222718+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.222718+0000","last_undegraded":"2024-04-14T08:38:44.222718+0000","last_fullsized":"2024-04-14T08:38:44.222718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.223232+0000","last_change":"2024-04-14T08:38:44.222002+0000","last_active":"2024-04-14T08:38:44.223232+0000","last_peered":"2024-04-14T08:38:44.223232+0000","last_clean":"2024-04-14T08:38:44.223232+0000","last_became_active":"2024-04-14T08:38:44.221679+0000","last_became_peered":"2024-04-14T08:38:44.221679+0000","last_unstale":"2024-04-14T08:38:44.223232+0000","last_undegraded":"2024-04-14T08:38:44.223232+0000","last_fullsized":"2024-04-14T08:38:44.223232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:53.649370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215754+0000","last_change":"2024-04-14T08:38:44.215754+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215754+0000","last_undegraded":"2024-04-14T08:38:44.215754+0000","last_fullsized":"2024-04-14T08:38:44.215754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.241020+0000","last_change":"2024-04-14T08:38:44.220058+0000","last_active":"2024-04-14T08:38:45.241020+0000","last_peered":"2024-04-14T08:38:45.241020+0000","last_clean":"2024-04-14T08:38:45.241020+0000","last_became_active":"2024-04-14T08:38:44.219721+0000","last_became_peered":"2024-04-14T08:38:44.219721+0000","last_unstale":"2024-04-14T08:38:45.241020+0000","last_undegraded":"2024-04-14T08:38:45.241020+0000","last_fullsized":"2024-04-14T08:38:45.241020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:54:15.282969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.211663+0000","last_change":"2024-04-14T08:38:44.211663+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.211663+0000","last_undegraded":"2024-04-14T08:38:44.211663+0000","last_fullsized":"2024-04-14T08:38:44.211663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.224435+0000","last_change":"2024-04-14T08:38:44.221416+0000","last_active":"2024-04-14T08:38:44.224435+0000","last_peered":"2024-04-14T08:38:44.224435+0000","last_clean":"2024-04-14T08:38:44.224435+0000","last_became_active":"2024-04-14T08:38:44.221138+0000","last_became_peered":"2024-04-14T08:38:44.221138+0000","last_unstale":"2024-04-14T08:38:44.224435+0000","last_undegraded":"2024-04-14T08:38:44.224435+0000","last_fullsized":"2024-04-14T08:38:44.224435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:15.223452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.219407+0000","last_change":"2024-04-14T08:38:44.219407+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.219407+0000","last_undegraded":"2024-04-14T08:38:44.219407+0000","last_fullsized":"2024-04-14T08:38:44.219407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225070+0000","last_change":"2024-04-14T08:38:44.224441+0000","last_active":"2024-04-14T08:38:44.225070+0000","last_peered":"2024-04-14T08:38:44.225070+0000","last_clean":"2024-04-14T08:38:44.225070+0000","last_became_active":"2024-04-14T08:38:44.224137+0000","last_became_peered":"2024-04-14T08:38:44.224137+0000","last_unstale":"2024-04-14T08:38:44.225070+0000","last_undegraded":"2024-04-14T08:38:44.225070+0000","last_fullsized":"2024-04-14T08:38:44.225070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:36:44.980948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.217106+0000","last_change":"2024-04-14T08:38:44.217106+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.217106+0000","last_undegraded":"2024-04-14T08:38:44.217106+0000","last_fullsized":"2024-04-14T08:38:44.217106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226868+0000","last_change":"2024-04-14T08:38:44.220261+0000","last_active":"2024-04-14T08:38:44.226868+0000","last_peered":"2024-04-14T08:38:44.226868+0000","last_clean":"2024-04-14T08:38:44.226868+0000","last_became_active":"2024-04-14T08:38:44.220001+0000","last_became_peered":"2024-04-14T08:38:44.220001+0000","last_unstale":"2024-04-14T08:38:44.226868+0000","last_undegraded":"2024-04-14T08:38:44.226868+0000","last_fullsized":"2024-04-14T08:38:44.226868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:13:48.232427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215334+0000","last_change":"2024-04-14T08:38:44.215334+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215334+0000","last_undegraded":"2024-04-14T08:38:44.215334+0000","last_fullsized":"2024-04-14T08:38:44.215334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228198+0000","last_change":"2024-04-14T08:38:44.227400+0000","last_active":"2024-04-14T08:38:44.228198+0000","last_peered":"2024-04-14T08:38:44.228198+0000","last_clean":"2024-04-14T08:38:44.228198+0000","last_became_active":"2024-04-14T08:38:44.226963+0000","last_became_peered":"2024-04-14T08:38:44.226963+0000","last_unstale":"2024-04-14T08:38:44.228198+0000","last_undegraded":"2024-04-14T08:38:44.228198+0000","last_fullsized":"2024-04-14T08:38:44.228198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:56:30.743467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.221098+0000","last_change":"2024-04-14T08:38:44.221098+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.221098+0000","last_undegraded":"2024-04-14T08:38:44.221098+0000","last_fullsized":"2024-04-14T08:38:44.221098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226693+0000","last_change":"2024-04-14T08:38:44.221259+0000","last_active":"2024-04-14T08:38:44.226693+0000","last_peered":"2024-04-14T08:38:44.226693+0000","last_clean":"2024-04-14T08:38:44.226693+0000","last_became_active":"2024-04-14T08:38:44.220936+0000","last_became_peered":"2024-04-14T08:38:44.220936+0000","last_unstale":"2024-04-14T08:38:44.226693+0000","last_undegraded":"2024-04-14T08:38:44.226693+0000","last_fullsized":"2024-04-14T08:38:44.226693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:20:32.325057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.215475+0000","last_change":"2024-04-14T08:38:44.215475+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.215475+0000","last_undegraded":"2024-04-14T08:38:44.215475+0000","last_fullsized":"2024-04-14T08:38:44.215475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"85'2","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.240442+0000","last_change":"2024-04-14T08:38:44.222193+0000","last_active":"2024-04-14T08:38:45.240442+0000","last_peered":"2024-04-14T08:38:45.240442+0000","last_clean":"2024-04-14T08:38:45.240442+0000","last_became_active":"2024-04-14T08:38:44.221895+0000","last_became_peered":"2024-04-14T08:38:44.221895+0000","last_unstale":"2024-04-14T08:38:45.240442+0000","last_undegraded":"2024-04-14T08:38:45.240442+0000","last_fullsized":"2024-04-14T08:38:45.240442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:36:12.417551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.226478+0000","last_change":"2024-04-14T08:38:44.226478+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.226478+0000","last_undegraded":"2024-04-14T08:38:44.226478+0000","last_fullsized":"2024-04-14T08:38:44.226478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.222944+0000","last_change":"2024-04-14T08:38:44.220634+0000","last_active":"2024-04-14T08:38:44.222944+0000","last_peered":"2024-04-14T08:38:44.222944+0000","last_clean":"2024-04-14T08:38:44.222944+0000","last_became_active":"2024-04-14T08:38:44.220352+0000","last_became_peered":"2024-04-14T08:38:44.220352+0000","last_unstale":"2024-04-14T08:38:44.222944+0000","last_undegraded":"2024-04-14T08:38:44.222944+0000","last_fullsized":"2024-04-14T08:38:44.222944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:47:54.300323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.218195+0000","last_change":"2024-04-14T08:38:44.218195+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.218195+0000","last_undegraded":"2024-04-14T08:38:44.218195+0000","last_fullsized":"2024-04-14T08:38:44.218195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.232546+0000","last_change":"2024-04-14T08:38:44.230684+0000","last_active":"2024-04-14T08:38:44.232546+0000","last_peered":"2024-04-14T08:38:44.232546+0000","last_clean":"2024-04-14T08:38:44.232546+0000","last_became_active":"2024-04-14T08:38:44.230211+0000","last_became_peered":"2024-04-14T08:38:44.230211+0000","last_unstale":"2024-04-14T08:38:44.232546+0000","last_undegraded":"2024-04-14T08:38:44.232546+0000","last_fullsized":"2024-04-14T08:38:44.232546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:21:35.631704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216328+0000","last_change":"2024-04-14T08:38:44.216328+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216328+0000","last_undegraded":"2024-04-14T08:38:44.216328+0000","last_fullsized":"2024-04-14T08:38:44.216328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225413+0000","last_change":"2024-04-14T08:38:44.224050+0000","last_active":"2024-04-14T08:38:44.225413+0000","last_peered":"2024-04-14T08:38:44.225413+0000","last_clean":"2024-04-14T08:38:44.225413+0000","last_became_active":"2024-04-14T08:38:44.223667+0000","last_became_peered":"2024-04-14T08:38:44.223667+0000","last_unstale":"2024-04-14T08:38:44.225413+0000","last_undegraded":"2024-04-14T08:38:44.225413+0000","last_fullsized":"2024-04-14T08:38:44.225413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:07:20.771457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.216642+0000","last_change":"2024-04-14T08:38:44.216642+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.216642+0000","last_undegraded":"2024-04-14T08:38:44.216642+0000","last_fullsized":"2024-04-14T08:38:44.216642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.233092+0000","last_change":"2024-04-14T08:38:44.230396+0000","last_active":"2024-04-14T08:38:44.233092+0000","last_peered":"2024-04-14T08:38:44.233092+0000","last_clean":"2024-04-14T08:38:44.233092+0000","last_became_active":"2024-04-14T08:38:44.230103+0000","last_became_peered":"2024-04-14T08:38:44.230103+0000","last_unstale":"2024-04-14T08:38:44.233092+0000","last_undegraded":"2024-04-14T08:38:44.233092+0000","last_fullsized":"2024-04-14T08:38:44.233092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:59:08.058016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.214047+0000","last_change":"2024-04-14T08:38:44.214047+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.214047+0000","last_undegraded":"2024-04-14T08:38:44.214047+0000","last_fullsized":"2024-04-14T08:38:44.214047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.226116+0000","last_change":"2024-04-14T08:38:44.224830+0000","last_active":"2024-04-14T08:38:44.226116+0000","last_peered":"2024-04-14T08:38:44.226116+0000","last_clean":"2024-04-14T08:38:44.226116+0000","last_became_active":"2024-04-14T08:38:44.224558+0000","last_became_peered":"2024-04-14T08:38:44.224558+0000","last_unstale":"2024-04-14T08:38:44.226116+0000","last_undegraded":"2024-04-14T08:38:44.226116+0000","last_fullsized":"2024-04-14T08:38:44.226116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:35.503739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.214795+0000","last_change":"2024-04-14T08:38:44.214795+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.214795+0000","last_undegraded":"2024-04-14T08:38:44.214795+0000","last_fullsized":"2024-04-14T08:38:44.214795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.225790+0000","last_change":"2024-04-14T08:38:44.224592+0000","last_active":"2024-04-14T08:38:44.225790+0000","last_peered":"2024-04-14T08:38:44.225790+0000","last_clean":"2024-04-14T08:38:44.225790+0000","last_became_active":"2024-04-14T08:38:44.224287+0000","last_became_peered":"2024-04-14T08:38:44.224287+0000","last_unstale":"2024-04-14T08:38:44.225790+0000","last_undegraded":"2024-04-14T08:38:44.225790+0000","last_fullsized":"2024-04-14T08:38:44.225790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:56:24.536114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-04-14T08:38:44.220700+0000","last_change":"2024-04-14T08:38:44.220700+0000","last_active":"2024-04-14T08:38:44.200740+0000","last_peered":"2024-04-14T08:38:44.200740+0000","last_clean":"2024-04-14T08:38:44.200740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:38:44.220700+0000","last_undegraded":"2024-04-14T08:38:44.220700+0000","last_fullsized":"2024-04-14T08:38:44.220700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.214298+0000","last_change":"2024-04-14T08:38:44.219166+0000","last_active":"2024-04-14T08:38:45.214298+0000","last_peered":"2024-04-14T08:38:45.214298+0000","last_clean":"2024-04-14T08:38:45.214298+0000","last_became_active":"2024-04-14T08:38:44.218905+0000","last_became_peered":"2024-04-14T08:38:44.218905+0000","last_unstale":"2024-04-14T08:38:45.214298+0000","last_undegraded":"2024-04-14T08:38:45.214298+0000","last_fullsized":"2024-04-14T08:38:45.214298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:24:57.813701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.227888+0000","last_change":"2024-04-14T08:38:45.223615+0000","last_active":"2024-04-14T08:38:45.227888+0000","last_peered":"2024-04-14T08:38:45.227888+0000","last_clean":"2024-04-14T08:38:45.227888+0000","last_became_active":"2024-04-14T08:38:45.221967+0000","last_became_peered":"2024-04-14T08:38:45.221967+0000","last_unstale":"2024-04-14T08:38:45.227888+0000","last_undegraded":"2024-04-14T08:38:45.227888+0000","last_fullsized":"2024-04-14T08:38:45.227888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:09.277582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-14T08:38:44.228434+0000","last_change":"2024-04-14T08:38:44.224166+0000","last_active":"2024-04-14T08:38:44.228434+0000","last_peered":"2024-04-14T08:38:44.228434+0000","last_clean":"2024-04-14T08:38:44.228434+0000","last_became_active":"2024-04-14T08:38:44.223244+0000","last_became_peered":"2024-04-14T08:38:44.223244+0000","last_unstale":"2024-04-14T08:38:44.228434+0000","last_undegraded":"2024-04-14T08:38:44.228434+0000","last_fullsized":"2024-04-14T08:38:44.228434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:05:42.226080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":301,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-14T08:38:45.214417+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:38:45.214417+0000","last_peered":"2024-04-14T08:38:45.214417+0000","last_clean":"2024-04-14T08:38:45.214417+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:38:45.214417+0000","last_undegraded":"2024-04-14T08:38:45.214417+0000","last_fullsized":"2024-04-14T08:38:45.214417+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1650688,"data_stored":1068945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":471354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":464687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1630208,"data_stored":1061722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:38:52.614 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:38:52.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.725+0000 7f64ede77640 1 Processor -- start 2024-04-14T08:38:52.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.726+0000 7f64ede77640 1 -- start start 2024-04-14T08:38:52.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.726+0000 7f64ede77640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f64e80d2830 0x7f64e80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:52.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.726+0000 7f64ede77640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64e815e890 0x7f64e80d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:52.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.726+0000 7f64ede77640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e80d3150 con 0x7f64e80cedd0 2024-04-14T08:38:52.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.726+0000 7f64ede77640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f64e80cff30 con 0x7f64e80d2830 2024-04-14T08:38:52.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.726+0000 7f64ede77640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f64e80d2350 con 0x7f64e815e890 2024-04-14T08:38:52.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.727+0000 7f64e6ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64e815e890 0x7f64e80d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:52.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.727+0000 7f64e77fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f64e80d2830 0x7f64e80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:52.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.727+0000 7f64e7fff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f64e80cedd0 0x7f64e81621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:34324/0 (socket says 172.21.15.172:34324) 2024-04-14T08:38:52.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.727+0000 7f64e77fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f64e80d2830 0x7f64e80d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:37738/0 (socket says 172.21.15.172:37738) 2024-04-14T08:38:52.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.727+0000 7f64e7fff640 1 -- 172.21.15.172:0/141472844 learned_addr learned my addr 172.21.15.172:0/141472844 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:52.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.727+0000 7f64e6ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64e815e890 0x7f64e80d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:47766/0 (socket says 172.21.15.172:47766) 2024-04-14T08:38:52.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.727+0000 7f64e6ffd640 1 -- 172.21.15.172:0/141472844 >> v2:172.21.15.177:3300/0 conn(0x7f64e80d2830 msgr2=0x7f64e80d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:52.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.727+0000 7f64e6ffd640 1 --2- 172.21.15.172:0/141472844 >> v2:172.21.15.177:3300/0 conn(0x7f64e80d2830 0x7f64e80d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:52.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.727+0000 7f64e6ffd640 1 -- 172.21.15.172:0/141472844 >> v1:172.21.15.177:6789/0 conn(0x7f64e80cedd0 legacy=0x7f64e81621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:52.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.727+0000 7f64e67fc640 1 -- 172.21.15.172:0/141472844 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 968748501 0 0) 0x7f64e80d3150 con 0x7f64e80cedd0 2024-04-14T08:38:52.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.727+0000 7f64e6ffd640 1 -- 172.21.15.172:0/141472844 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e80cf320 con 0x7f64e815e890 2024-04-14T08:38:52.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.727+0000 7f64e77fe640 1 --2- 172.21.15.172:0/141472844 >> v2:172.21.15.177:3300/0 conn(0x7f64e80d2830 0x7f64e80d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:52.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.728+0000 7f64e6ffd640 1 --2- 172.21.15.172:0/141472844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64e815e890 0x7f64e80d1300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0097a0 tx=0x7f64dc02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f197c84d01631735 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:52.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.728+0000 7f64e67fc640 1 -- 172.21.15.172:0/141472844 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc034070 con 0x7f64e815e890 2024-04-14T08:38:52.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.728+0000 7f64e67fc640 1 -- 172.21.15.172:0/141472844 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64dc030610 con 0x7f64e815e890 2024-04-14T08:38:52.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.728+0000 7f64e67fc640 1 -- 172.21.15.172:0/141472844 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0309e0 con 0x7f64e815e890 2024-04-14T08:38:52.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.728+0000 7f64ede77640 1 -- 172.21.15.172:0/141472844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64e815e890 msgr2=0x7f64e80d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:52.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.728+0000 7f64ede77640 1 --2- 172.21.15.172:0/141472844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64e815e890 0x7f64e80d1300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0097a0 tx=0x7f64dc02b960 comp rx=0 tx=0).stop 2024-04-14T08:38:52.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.728+0000 7f64ede77640 1 -- 172.21.15.172:0/141472844 shutdown_connections 2024-04-14T08:38:52.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.728+0000 7f64ede77640 1 --2- 172.21.15.172:0/141472844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64e815e890 0x7f64e80d1300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:52.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.728+0000 7f64ede77640 1 --2- 172.21.15.172:0/141472844 >> v2:172.21.15.177:3300/0 conn(0x7f64e80d2830 0x7f64e80d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:52.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.728+0000 7f64ede77640 1 -- 172.21.15.172:0/141472844 >> 172.21.15.172:0/141472844 conn(0x7f64e809d590 msgr2=0x7f64e80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:52.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.728+0000 7f64ede77640 1 -- 172.21.15.172:0/141472844 shutdown_connections 2024-04-14T08:38:52.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.728+0000 7f64ede77640 1 -- 172.21.15.172:0/141472844 wait complete. 2024-04-14T08:38:52.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.730+0000 7f64ede77640 1 Processor -- start 2024-04-14T08:38:52.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.730+0000 7f64ede77640 1 -- start start 2024-04-14T08:38:52.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64ede77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e80cedd0 0x7f64e80b99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:52.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64ede77640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64e80d2830 0x7f64e80cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:52.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64ede77640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64e815e890 0x7f64e81542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:52.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64ede77640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64e80cfbc0 con 0x7f64e80cedd0 2024-04-14T08:38:52.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64ede77640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f64e80cf8e0 con 0x7f64e80d2830 2024-04-14T08:38:52.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64ede77640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f64e80d1840 con 0x7f64e815e890 2024-04-14T08:38:52.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e80cedd0 0x7f64e80b99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e80cedd0 0x7f64e80b99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37928/0 (socket says 172.21.15.172:37928) 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64e77fe640 1 -- 172.21.15.172:0/2826978510 learned_addr learned my addr 172.21.15.172:0/2826978510 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64e7fff640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64e815e890 0x7f64e81542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64e6ffd640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64e80d2830 0x7f64e80cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64e77fe640 1 -- 172.21.15.172:0/2826978510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64e815e890 msgr2=0x7f64e81542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64e77fe640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64e815e890 0x7f64e81542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64e77fe640 1 -- 172.21.15.172:0/2826978510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64e80d2830 msgr2=0x7f64e80cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64e77fe640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64e80d2830 0x7f64e80cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64e77fe640 1 -- 172.21.15.172:0/2826978510 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e80c9a30 con 0x7f64e80cedd0 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64e77fe640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e80cedd0 0x7f64e80b99f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f64d800e960 tx=0x7f64d800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64cffff640 1 -- 172.21.15.172:0/2826978510 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8017070 con 0x7f64e80cedd0 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64cffff640 1 -- 172.21.15.172:0/2826978510 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d800ca00 con 0x7f64e80cedd0 2024-04-14T08:38:52.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64ede77640 1 -- 172.21.15.172:0/2826978510 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e80c9d10 con 0x7f64e80cedd0 2024-04-14T08:38:52.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64cffff640 1 -- 172.21.15.172:0/2826978510 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d800cdd0 con 0x7f64e80cedd0 2024-04-14T08:38:52.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.731+0000 7f64ede77640 1 -- 172.21.15.172:0/2826978510 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e80b7500 con 0x7f64e80cedd0 2024-04-14T08:38:52.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.732+0000 7f64ede77640 1 -- 172.21.15.172:0/2826978510 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e80d1f80 con 0x7f64e80cedd0 2024-04-14T08:38:52.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.733+0000 7f64cffff640 1 -- 172.21.15.172:0/2826978510 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64d801a070 con 0x7f64e80cedd0 2024-04-14T08:38:52.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.733+0000 7f64cffff640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64d406c8f0 0x7f64d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:52.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.734+0000 7f64e6ffd640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64d406c8f0 0x7f64d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:52.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.734+0000 7f64cffff640 1 -- 172.21.15.172:0/2826978510 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f64d808bea0 con 0x7f64e80cedd0 2024-04-14T08:38:52.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.734+0000 7f64e6ffd640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64d406c8f0 0x7f64d406edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f64dc036000 tx=0x7f64dc03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:52.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.738+0000 7f64cffff640 1 -- 172.21.15.172:0/2826978510 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f64e80d1f80 con 0x7f64e80cedd0 2024-04-14T08:38:52.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.907+0000 7f64ede77640 1 -- 172.21.15.172:0/2826978510 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64e8068520 con 0x7f64d406c8f0 2024-04-14T08:38:52.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.913+0000 7f64cffff640 1 -- 172.21.15.172:0/2826978510 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252062 (secure 0 0 0) 0x7f64e8068520 con 0x7f64d406c8f0 2024-04-14T08:38:52.917 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:52.919 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:38:52.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.916+0000 7f64ede77640 1 -- 172.21.15.172:0/2826978510 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64d406c8f0 msgr2=0x7f64d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:52.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.916+0000 7f64ede77640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64d406c8f0 0x7f64d406edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f64dc036000 tx=0x7f64dc03b040 comp rx=0 tx=0).stop 2024-04-14T08:38:52.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.916+0000 7f64ede77640 1 -- 172.21.15.172:0/2826978510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e80cedd0 msgr2=0x7f64e80b99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:52.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.916+0000 7f64ede77640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e80cedd0 0x7f64e80b99f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f64d800e960 tx=0x7f64d800ee20 comp rx=0 tx=0).stop 2024-04-14T08:38:52.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.916+0000 7f64ede77640 1 -- 172.21.15.172:0/2826978510 shutdown_connections 2024-04-14T08:38:52.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.916+0000 7f64ede77640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64d406c8f0 0x7f64d406edb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:52.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.916+0000 7f64ede77640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64e815e890 0x7f64e81542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:52.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.916+0000 7f64ede77640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64e80d2830 0x7f64e80cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:52.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.916+0000 7f64ede77640 1 --2- 172.21.15.172:0/2826978510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64e80cedd0 0x7f64e80b99f0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:52.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.916+0000 7f64ede77640 1 -- 172.21.15.172:0/2826978510 >> 172.21.15.172:0/2826978510 conn(0x7f64e809d590 msgr2=0x7f64e80d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:52.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.916+0000 7f64ede77640 1 -- 172.21.15.172:0/2826978510 shutdown_connections 2024-04-14T08:38:52.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:52.916+0000 7f64ede77640 1 -- 172.21.15.172:0/2826978510 wait complete. 2024-04-14T08:38:52.929 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-04-14T08:38:51.191918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250648,"kb_used_data":9512,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724072,"statfs":{"total":773094113280,"available":772837449728,"internally_reserved":0,"allocated":9740288,"data_stored":5067260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.982607"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426652+0000","last_change":"2024-04-14T08:38:45.233789+0000","last_active":"2024-04-14T08:38:46.426652+0000","last_peered":"2024-04-14T08:38:46.426652+0000","last_clean":"2024-04-14T08:38:46.426652+0000","last_became_active":"2024-04-14T08:38:45.233446+0000","last_became_peered":"2024-04-14T08:38:45.233446+0000","last_unstale":"2024-04-14T08:38:46.426652+0000","last_undegraded":"2024-04-14T08:38:46.426652+0000","last_fullsized":"2024-04-14T08:38:46.426652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:09.277582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425738+0000","last_change":"2024-04-14T08:38:45.228056+0000","last_active":"2024-04-14T08:38:46.425738+0000","last_peered":"2024-04-14T08:38:46.425738+0000","last_clean":"2024-04-14T08:38:46.425738+0000","last_became_active":"2024-04-14T08:38:45.227782+0000","last_became_peered":"2024-04-14T08:38:45.227782+0000","last_unstale":"2024-04-14T08:38:46.425738+0000","last_undegraded":"2024-04-14T08:38:46.425738+0000","last_fullsized":"2024-04-14T08:38:46.425738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:36:13.425186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425899+0000","last_change":"2024-04-14T08:38:45.231393+0000","last_active":"2024-04-14T08:38:46.425899+0000","last_peered":"2024-04-14T08:38:46.425899+0000","last_clean":"2024-04-14T08:38:46.425899+0000","last_became_active":"2024-04-14T08:38:45.231131+0000","last_became_peered":"2024-04-14T08:38:45.231131+0000","last_unstale":"2024-04-14T08:38:46.425899+0000","last_undegraded":"2024-04-14T08:38:46.425899+0000","last_fullsized":"2024-04-14T08:38:46.425899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:15:05.563225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427667+0000","last_change":"2024-04-14T08:38:45.227931+0000","last_active":"2024-04-14T08:38:46.427667+0000","last_peered":"2024-04-14T08:38:46.427667+0000","last_clean":"2024-04-14T08:38:46.427667+0000","last_became_active":"2024-04-14T08:38:45.227656+0000","last_became_peered":"2024-04-14T08:38:45.227656+0000","last_unstale":"2024-04-14T08:38:46.427667+0000","last_undegraded":"2024-04-14T08:38:46.427667+0000","last_fullsized":"2024-04-14T08:38:46.427667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:12:15.192867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215914+0000","last_change":"2024-04-14T08:38:45.222666+0000","last_active":"2024-04-14T08:38:46.215914+0000","last_peered":"2024-04-14T08:38:46.215914+0000","last_clean":"2024-04-14T08:38:46.215914+0000","last_became_active":"2024-04-14T08:38:45.222392+0000","last_became_peered":"2024-04-14T08:38:45.222392+0000","last_unstale":"2024-04-14T08:38:46.215914+0000","last_undegraded":"2024-04-14T08:38:46.215914+0000","last_fullsized":"2024-04-14T08:38:46.215914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:55:57.133505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425740+0000","last_change":"2024-04-14T08:38:45.229675+0000","last_active":"2024-04-14T08:38:46.425740+0000","last_peered":"2024-04-14T08:38:46.425740+0000","last_clean":"2024-04-14T08:38:46.425740+0000","last_became_active":"2024-04-14T08:38:45.229243+0000","last_became_peered":"2024-04-14T08:38:45.229243+0000","last_unstale":"2024-04-14T08:38:46.425740+0000","last_undegraded":"2024-04-14T08:38:46.425740+0000","last_fullsized":"2024-04-14T08:38:46.425740+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:00:29.417849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.214764+0000","last_change":"2024-04-14T08:38:45.221574+0000","last_active":"2024-04-14T08:38:46.214764+0000","last_peered":"2024-04-14T08:38:46.214764+0000","last_clean":"2024-04-14T08:38:46.214764+0000","last_became_active":"2024-04-14T08:38:45.221303+0000","last_became_peered":"2024-04-14T08:38:45.221303+0000","last_unstale":"2024-04-14T08:38:46.214764+0000","last_undegraded":"2024-04-14T08:38:46.214764+0000","last_fullsized":"2024-04-14T08:38:46.214764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:28:05.374995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427293+0000","last_change":"2024-04-14T08:38:45.220229+0000","last_active":"2024-04-14T08:38:46.427293+0000","last_peered":"2024-04-14T08:38:46.427293+0000","last_clean":"2024-04-14T08:38:46.427293+0000","last_became_active":"2024-04-14T08:38:45.219373+0000","last_became_peered":"2024-04-14T08:38:45.219373+0000","last_unstale":"2024-04-14T08:38:46.427293+0000","last_undegraded":"2024-04-14T08:38:46.427293+0000","last_fullsized":"2024-04-14T08:38:46.427293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:00:18.008264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215148+0000","last_change":"2024-04-14T08:38:45.224114+0000","last_active":"2024-04-14T08:38:46.215148+0000","last_peered":"2024-04-14T08:38:46.215148+0000","last_clean":"2024-04-14T08:38:46.215148+0000","last_became_active":"2024-04-14T08:38:45.223327+0000","last_became_peered":"2024-04-14T08:38:45.223327+0000","last_unstale":"2024-04-14T08:38:46.215148+0000","last_undegraded":"2024-04-14T08:38:46.215148+0000","last_fullsized":"2024-04-14T08:38:46.215148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:35:24.830631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.212214+0000","last_change":"2024-04-14T08:38:45.231727+0000","last_active":"2024-04-14T08:38:46.212214+0000","last_peered":"2024-04-14T08:38:46.212214+0000","last_clean":"2024-04-14T08:38:46.212214+0000","last_became_active":"2024-04-14T08:38:45.230876+0000","last_became_peered":"2024-04-14T08:38:45.230876+0000","last_unstale":"2024-04-14T08:38:46.212214+0000","last_undegraded":"2024-04-14T08:38:46.212214+0000","last_fullsized":"2024-04-14T08:38:46.212214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:36:52.350808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215841+0000","last_change":"2024-04-14T08:38:45.230715+0000","last_active":"2024-04-14T08:38:46.215841+0000","last_peered":"2024-04-14T08:38:46.215841+0000","last_clean":"2024-04-14T08:38:46.215841+0000","last_became_active":"2024-04-14T08:38:45.230371+0000","last_became_peered":"2024-04-14T08:38:45.230371+0000","last_unstale":"2024-04-14T08:38:46.215841+0000","last_undegraded":"2024-04-14T08:38:46.215841+0000","last_fullsized":"2024-04-14T08:38:46.215841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:28:58.100549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.212143+0000","last_change":"2024-04-14T08:38:45.224538+0000","last_active":"2024-04-14T08:38:46.212143+0000","last_peered":"2024-04-14T08:38:46.212143+0000","last_clean":"2024-04-14T08:38:46.212143+0000","last_became_active":"2024-04-14T08:38:45.224260+0000","last_became_peered":"2024-04-14T08:38:45.224260+0000","last_unstale":"2024-04-14T08:38:46.212143+0000","last_undegraded":"2024-04-14T08:38:46.212143+0000","last_fullsized":"2024-04-14T08:38:46.212143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:33:58.755793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427499+0000","last_change":"2024-04-14T08:38:45.218347+0000","last_active":"2024-04-14T08:38:46.427499+0000","last_peered":"2024-04-14T08:38:46.427499+0000","last_clean":"2024-04-14T08:38:46.427499+0000","last_became_active":"2024-04-14T08:38:45.218028+0000","last_became_peered":"2024-04-14T08:38:45.218028+0000","last_unstale":"2024-04-14T08:38:46.427499+0000","last_undegraded":"2024-04-14T08:38:46.427499+0000","last_fullsized":"2024-04-14T08:38:46.427499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:34:25.404380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215832+0000","last_change":"2024-04-14T08:38:45.230765+0000","last_active":"2024-04-14T08:38:46.215832+0000","last_peered":"2024-04-14T08:38:46.215832+0000","last_clean":"2024-04-14T08:38:46.215832+0000","last_became_active":"2024-04-14T08:38:45.227840+0000","last_became_peered":"2024-04-14T08:38:45.227840+0000","last_unstale":"2024-04-14T08:38:46.215832+0000","last_undegraded":"2024-04-14T08:38:46.215832+0000","last_fullsized":"2024-04-14T08:38:46.215832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:52:14.298996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427068+0000","last_change":"2024-04-14T08:38:45.223083+0000","last_active":"2024-04-14T08:38:46.427068+0000","last_peered":"2024-04-14T08:38:46.427068+0000","last_clean":"2024-04-14T08:38:46.427068+0000","last_became_active":"2024-04-14T08:38:45.222800+0000","last_became_peered":"2024-04-14T08:38:45.222800+0000","last_unstale":"2024-04-14T08:38:46.427068+0000","last_undegraded":"2024-04-14T08:38:46.427068+0000","last_fullsized":"2024-04-14T08:38:46.427068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:56:31.751102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426113+0000","last_change":"2024-04-14T08:38:45.222706+0000","last_active":"2024-04-14T08:38:46.426113+0000","last_peered":"2024-04-14T08:38:46.426113+0000","last_clean":"2024-04-14T08:38:46.426113+0000","last_became_active":"2024-04-14T08:38:45.222457+0000","last_became_peered":"2024-04-14T08:38:45.222457+0000","last_unstale":"2024-04-14T08:38:46.426113+0000","last_undegraded":"2024-04-14T08:38:46.426113+0000","last_fullsized":"2024-04-14T08:38:46.426113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:36:21.955027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426873+0000","last_change":"2024-04-14T08:38:45.231411+0000","last_active":"2024-04-14T08:38:46.426873+0000","last_peered":"2024-04-14T08:38:46.426873+0000","last_clean":"2024-04-14T08:38:46.426873+0000","last_became_active":"2024-04-14T08:38:45.230597+0000","last_became_peered":"2024-04-14T08:38:45.230597+0000","last_unstale":"2024-04-14T08:38:46.426873+0000","last_undegraded":"2024-04-14T08:38:46.426873+0000","last_fullsized":"2024-04-14T08:38:46.426873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:07:45.540501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426131+0000","last_change":"2024-04-14T08:38:45.221861+0000","last_active":"2024-04-14T08:38:46.426131+0000","last_peered":"2024-04-14T08:38:46.426131+0000","last_clean":"2024-04-14T08:38:46.426131+0000","last_became_active":"2024-04-14T08:38:45.221586+0000","last_became_peered":"2024-04-14T08:38:45.221586+0000","last_unstale":"2024-04-14T08:38:46.426131+0000","last_undegraded":"2024-04-14T08:38:46.426131+0000","last_fullsized":"2024-04-14T08:38:46.426131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:24:58.821336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215837+0000","last_change":"2024-04-14T08:38:45.225885+0000","last_active":"2024-04-14T08:38:46.215837+0000","last_peered":"2024-04-14T08:38:46.215837+0000","last_clean":"2024-04-14T08:38:46.215837+0000","last_became_active":"2024-04-14T08:38:45.225626+0000","last_became_peered":"2024-04-14T08:38:45.225626+0000","last_unstale":"2024-04-14T08:38:46.215837+0000","last_undegraded":"2024-04-14T08:38:46.215837+0000","last_fullsized":"2024-04-14T08:38:46.215837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:23.852025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427749+0000","last_change":"2024-04-14T08:38:45.223854+0000","last_active":"2024-04-14T08:38:46.427749+0000","last_peered":"2024-04-14T08:38:46.427749+0000","last_clean":"2024-04-14T08:38:46.427749+0000","last_became_active":"2024-04-14T08:38:45.223533+0000","last_became_peered":"2024-04-14T08:38:45.223533+0000","last_unstale":"2024-04-14T08:38:46.427749+0000","last_undegraded":"2024-04-14T08:38:46.427749+0000","last_fullsized":"2024-04-14T08:38:46.427749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:35:51.821657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215688+0000","last_change":"2024-04-14T08:38:45.221970+0000","last_active":"2024-04-14T08:38:46.215688+0000","last_peered":"2024-04-14T08:38:46.215688+0000","last_clean":"2024-04-14T08:38:46.215688+0000","last_became_active":"2024-04-14T08:38:45.221055+0000","last_became_peered":"2024-04-14T08:38:45.221055+0000","last_unstale":"2024-04-14T08:38:46.215688+0000","last_undegraded":"2024-04-14T08:38:46.215688+0000","last_fullsized":"2024-04-14T08:38:46.215688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:35:51.821657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215599+0000","last_change":"2024-04-14T08:38:45.228069+0000","last_active":"2024-04-14T08:38:46.215599+0000","last_peered":"2024-04-14T08:38:46.215599+0000","last_clean":"2024-04-14T08:38:46.215599+0000","last_became_active":"2024-04-14T08:38:45.227254+0000","last_became_peered":"2024-04-14T08:38:45.227254+0000","last_unstale":"2024-04-14T08:38:46.215599+0000","last_undegraded":"2024-04-14T08:38:46.215599+0000","last_fullsized":"2024-04-14T08:38:46.215599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:12:15.192867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425465+0000","last_change":"2024-04-14T08:38:45.227783+0000","last_active":"2024-04-14T08:38:46.425465+0000","last_peered":"2024-04-14T08:38:46.425465+0000","last_clean":"2024-04-14T08:38:46.425465+0000","last_became_active":"2024-04-14T08:38:45.227512+0000","last_became_peered":"2024-04-14T08:38:45.227512+0000","last_unstale":"2024-04-14T08:38:46.425465+0000","last_undegraded":"2024-04-14T08:38:46.425465+0000","last_fullsized":"2024-04-14T08:38:46.425465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:43:03.191778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425981+0000","last_change":"2024-04-14T08:38:45.231835+0000","last_active":"2024-04-14T08:38:46.425981+0000","last_peered":"2024-04-14T08:38:46.425981+0000","last_clean":"2024-04-14T08:38:46.425981+0000","last_became_active":"2024-04-14T08:38:45.230957+0000","last_became_peered":"2024-04-14T08:38:45.230957+0000","last_unstale":"2024-04-14T08:38:46.425981+0000","last_undegraded":"2024-04-14T08:38:46.425981+0000","last_fullsized":"2024-04-14T08:38:46.425981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:59:09.065652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425650+0000","last_change":"2024-04-14T08:38:45.228819+0000","last_active":"2024-04-14T08:38:46.425650+0000","last_peered":"2024-04-14T08:38:46.425650+0000","last_clean":"2024-04-14T08:38:46.425650+0000","last_became_active":"2024-04-14T08:38:45.228577+0000","last_became_peered":"2024-04-14T08:38:45.228577+0000","last_unstale":"2024-04-14T08:38:46.425650+0000","last_undegraded":"2024-04-14T08:38:46.425650+0000","last_fullsized":"2024-04-14T08:38:46.425650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:54:16.290605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215864+0000","last_change":"2024-04-14T08:38:45.231095+0000","last_active":"2024-04-14T08:38:46.215864+0000","last_peered":"2024-04-14T08:38:46.215864+0000","last_clean":"2024-04-14T08:38:46.215864+0000","last_became_active":"2024-04-14T08:38:45.230806+0000","last_became_peered":"2024-04-14T08:38:45.230806+0000","last_unstale":"2024-04-14T08:38:46.215864+0000","last_undegraded":"2024-04-14T08:38:46.215864+0000","last_fullsized":"2024-04-14T08:38:46.215864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:43:44.976330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.212150+0000","last_change":"2024-04-14T08:38:45.228082+0000","last_active":"2024-04-14T08:38:46.212150+0000","last_peered":"2024-04-14T08:38:46.212150+0000","last_clean":"2024-04-14T08:38:46.212150+0000","last_became_active":"2024-04-14T08:38:45.227671+0000","last_became_peered":"2024-04-14T08:38:45.227671+0000","last_unstale":"2024-04-14T08:38:46.212150+0000","last_undegraded":"2024-04-14T08:38:46.212150+0000","last_fullsized":"2024-04-14T08:38:46.212150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:55:56.004867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427761+0000","last_change":"2024-04-14T08:38:45.225081+0000","last_active":"2024-04-14T08:38:46.427761+0000","last_peered":"2024-04-14T08:38:46.427761+0000","last_clean":"2024-04-14T08:38:46.427761+0000","last_became_active":"2024-04-14T08:38:45.224820+0000","last_became_peered":"2024-04-14T08:38:45.224820+0000","last_unstale":"2024-04-14T08:38:46.427761+0000","last_undegraded":"2024-04-14T08:38:46.427761+0000","last_fullsized":"2024-04-14T08:38:46.427761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:43:03.191778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215117+0000","last_change":"2024-04-14T08:38:45.226642+0000","last_active":"2024-04-14T08:38:46.215117+0000","last_peered":"2024-04-14T08:38:46.215117+0000","last_clean":"2024-04-14T08:38:46.215117+0000","last_became_active":"2024-04-14T08:38:45.226349+0000","last_became_peered":"2024-04-14T08:38:45.226349+0000","last_unstale":"2024-04-14T08:38:46.215117+0000","last_undegraded":"2024-04-14T08:38:46.215117+0000","last_fullsized":"2024-04-14T08:38:46.215117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:13:29.671740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215906+0000","last_change":"2024-04-14T08:38:45.222837+0000","last_active":"2024-04-14T08:38:46.215906+0000","last_peered":"2024-04-14T08:38:46.215906+0000","last_clean":"2024-04-14T08:38:46.215906+0000","last_became_active":"2024-04-14T08:38:45.222408+0000","last_became_peered":"2024-04-14T08:38:45.222408+0000","last_unstale":"2024-04-14T08:38:46.215906+0000","last_undegraded":"2024-04-14T08:38:46.215906+0000","last_fullsized":"2024-04-14T08:38:46.215906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:54:16.290605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425305+0000","last_change":"2024-04-14T08:38:45.230468+0000","last_active":"2024-04-14T08:38:46.425305+0000","last_peered":"2024-04-14T08:38:46.425305+0000","last_clean":"2024-04-14T08:38:46.425305+0000","last_became_active":"2024-04-14T08:38:45.230157+0000","last_became_peered":"2024-04-14T08:38:45.230157+0000","last_unstale":"2024-04-14T08:38:46.425305+0000","last_undegraded":"2024-04-14T08:38:46.425305+0000","last_fullsized":"2024-04-14T08:38:46.425305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:28:58.100549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425695+0000","last_change":"2024-04-14T08:38:45.222089+0000","last_active":"2024-04-14T08:38:46.425695+0000","last_peered":"2024-04-14T08:38:46.425695+0000","last_clean":"2024-04-14T08:38:46.425695+0000","last_became_active":"2024-04-14T08:38:45.221809+0000","last_became_peered":"2024-04-14T08:38:45.221809+0000","last_unstale":"2024-04-14T08:38:46.425695+0000","last_undegraded":"2024-04-14T08:38:46.425695+0000","last_fullsized":"2024-04-14T08:38:46.425695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:22:26.178452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426852+0000","last_change":"2024-04-14T08:38:45.226061+0000","last_active":"2024-04-14T08:38:46.426852+0000","last_peered":"2024-04-14T08:38:46.426852+0000","last_clean":"2024-04-14T08:38:46.426852+0000","last_became_active":"2024-04-14T08:38:45.225798+0000","last_became_peered":"2024-04-14T08:38:45.225798+0000","last_unstale":"2024-04-14T08:38:46.426852+0000","last_undegraded":"2024-04-14T08:38:46.426852+0000","last_fullsized":"2024-04-14T08:38:46.426852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:41:22.730616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"85'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425900+0000","last_change":"2024-04-14T08:38:44.227154+0000","last_active":"2024-04-14T08:38:46.425900+0000","last_peered":"2024-04-14T08:38:46.425900+0000","last_clean":"2024-04-14T08:38:46.425900+0000","last_became_active":"2024-04-14T08:38:44.226910+0000","last_became_peered":"2024-04-14T08:38:44.226910+0000","last_unstale":"2024-04-14T08:38:46.425900+0000","last_undegraded":"2024-04-14T08:38:46.425900+0000","last_fullsized":"2024-04-14T08:38:46.425900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:36:44.980948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426941+0000","last_change":"2024-04-14T08:38:45.225198+0000","last_active":"2024-04-14T08:38:46.426941+0000","last_peered":"2024-04-14T08:38:46.426941+0000","last_clean":"2024-04-14T08:38:46.426941+0000","last_became_active":"2024-04-14T08:38:45.224926+0000","last_became_peered":"2024-04-14T08:38:45.224926+0000","last_unstale":"2024-04-14T08:38:46.426941+0000","last_undegraded":"2024-04-14T08:38:46.426941+0000","last_fullsized":"2024-04-14T08:38:46.426941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:44:06.902986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.216010+0000","last_change":"2024-04-14T08:38:44.222545+0000","last_active":"2024-04-14T08:38:46.216010+0000","last_peered":"2024-04-14T08:38:46.216010+0000","last_clean":"2024-04-14T08:38:46.216010+0000","last_became_active":"2024-04-14T08:38:44.221999+0000","last_became_peered":"2024-04-14T08:38:44.221999+0000","last_unstale":"2024-04-14T08:38:46.216010+0000","last_undegraded":"2024-04-14T08:38:46.216010+0000","last_fullsized":"2024-04-14T08:38:46.216010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:27:28.589254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.214643+0000","last_change":"2024-04-14T08:38:45.228983+0000","last_active":"2024-04-14T08:38:46.214643+0000","last_peered":"2024-04-14T08:38:46.214643+0000","last_clean":"2024-04-14T08:38:46.214643+0000","last_became_active":"2024-04-14T08:38:45.228632+0000","last_became_peered":"2024-04-14T08:38:45.228632+0000","last_unstale":"2024-04-14T08:38:46.214643+0000","last_undegraded":"2024-04-14T08:38:46.214643+0000","last_fullsized":"2024-04-14T08:38:46.214643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:15:27.883896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215317+0000","last_change":"2024-04-14T08:38:44.214063+0000","last_active":"2024-04-14T08:38:46.215317+0000","last_peered":"2024-04-14T08:38:46.215317+0000","last_clean":"2024-04-14T08:38:46.215317+0000","last_became_active":"2024-04-14T08:38:44.213612+0000","last_became_peered":"2024-04-14T08:38:44.213612+0000","last_unstale":"2024-04-14T08:38:46.215317+0000","last_undegraded":"2024-04-14T08:38:46.215317+0000","last_fullsized":"2024-04-14T08:38:46.215317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:15:04.555589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427004+0000","last_change":"2024-04-14T08:38:45.228110+0000","last_active":"2024-04-14T08:38:46.427004+0000","last_peered":"2024-04-14T08:38:46.427004+0000","last_clean":"2024-04-14T08:38:46.427004+0000","last_became_active":"2024-04-14T08:38:45.227854+0000","last_became_peered":"2024-04-14T08:38:45.227854+0000","last_unstale":"2024-04-14T08:38:46.427004+0000","last_undegraded":"2024-04-14T08:38:46.427004+0000","last_fullsized":"2024-04-14T08:38:46.427004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:58:54.130406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426081+0000","last_change":"2024-04-14T08:38:44.227021+0000","last_active":"2024-04-14T08:38:46.426081+0000","last_peered":"2024-04-14T08:38:46.426081+0000","last_clean":"2024-04-14T08:38:46.426081+0000","last_became_active":"2024-04-14T08:38:44.226731+0000","last_became_peered":"2024-04-14T08:38:44.226731+0000","last_unstale":"2024-04-14T08:38:46.426081+0000","last_undegraded":"2024-04-14T08:38:46.426081+0000","last_fullsized":"2024-04-14T08:38:46.426081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:06:35.503739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427364+0000","last_change":"2024-04-14T08:38:45.225836+0000","last_active":"2024-04-14T08:38:46.427364+0000","last_peered":"2024-04-14T08:38:46.427364+0000","last_clean":"2024-04-14T08:38:46.427364+0000","last_became_active":"2024-04-14T08:38:45.225548+0000","last_became_peered":"2024-04-14T08:38:45.225548+0000","last_unstale":"2024-04-14T08:38:46.427364+0000","last_undegraded":"2024-04-14T08:38:46.427364+0000","last_fullsized":"2024-04-14T08:38:46.427364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:14:10.441689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215913+0000","last_change":"2024-04-14T08:38:44.222167+0000","last_active":"2024-04-14T08:38:46.215913+0000","last_peered":"2024-04-14T08:38:46.215913+0000","last_clean":"2024-04-14T08:38:46.215913+0000","last_became_active":"2024-04-14T08:38:44.221892+0000","last_became_peered":"2024-04-14T08:38:44.221892+0000","last_unstale":"2024-04-14T08:38:46.215913+0000","last_undegraded":"2024-04-14T08:38:46.215913+0000","last_fullsized":"2024-04-14T08:38:46.215913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:34:53.649370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425599+0000","last_change":"2024-04-14T08:38:45.238405+0000","last_active":"2024-04-14T08:38:46.425599+0000","last_peered":"2024-04-14T08:38:46.425599+0000","last_clean":"2024-04-14T08:38:46.425599+0000","last_became_active":"2024-04-14T08:38:45.238072+0000","last_became_peered":"2024-04-14T08:38:45.238072+0000","last_unstale":"2024-04-14T08:38:46.425599+0000","last_undegraded":"2024-04-14T08:38:46.425599+0000","last_fullsized":"2024-04-14T08:38:46.425599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:52:14.298996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427614+0000","last_change":"2024-04-14T08:38:44.222906+0000","last_active":"2024-04-14T08:38:46.427614+0000","last_peered":"2024-04-14T08:38:46.427614+0000","last_clean":"2024-04-14T08:38:46.427614+0000","last_became_active":"2024-04-14T08:38:44.222541+0000","last_became_peered":"2024-04-14T08:38:44.222541+0000","last_unstale":"2024-04-14T08:38:46.427614+0000","last_undegraded":"2024-04-14T08:38:46.427614+0000","last_fullsized":"2024-04-14T08:38:46.427614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:32.465724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425881+0000","last_change":"2024-04-14T08:38:45.229457+0000","last_active":"2024-04-14T08:38:46.425881+0000","last_peered":"2024-04-14T08:38:46.425881+0000","last_clean":"2024-04-14T08:38:46.425881+0000","last_became_active":"2024-04-14T08:38:45.229172+0000","last_became_peered":"2024-04-14T08:38:45.229172+0000","last_unstale":"2024-04-14T08:38:46.425881+0000","last_undegraded":"2024-04-14T08:38:46.425881+0000","last_fullsized":"2024-04-14T08:38:46.425881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:19:33.799390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426411+0000","last_change":"2024-04-14T08:38:44.218254+0000","last_active":"2024-04-14T08:38:46.426411+0000","last_peered":"2024-04-14T08:38:46.426411+0000","last_clean":"2024-04-14T08:38:46.426411+0000","last_became_active":"2024-04-14T08:38:44.217969+0000","last_became_peered":"2024-04-14T08:38:44.217969+0000","last_unstale":"2024-04-14T08:38:46.426411+0000","last_undegraded":"2024-04-14T08:38:46.426411+0000","last_fullsized":"2024-04-14T08:38:46.426411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:35:50.814022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427536+0000","last_change":"2024-04-14T08:38:45.228554+0000","last_active":"2024-04-14T08:38:46.427536+0000","last_peered":"2024-04-14T08:38:46.427536+0000","last_clean":"2024-04-14T08:38:46.427536+0000","last_became_active":"2024-04-14T08:38:45.228293+0000","last_became_peered":"2024-04-14T08:38:45.228293+0000","last_unstale":"2024-04-14T08:38:46.427536+0000","last_undegraded":"2024-04-14T08:38:46.427536+0000","last_fullsized":"2024-04-14T08:38:46.427536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:35:24.830631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.214568+0000","last_change":"2024-04-14T08:38:44.231501+0000","last_active":"2024-04-14T08:38:46.214568+0000","last_peered":"2024-04-14T08:38:46.214568+0000","last_clean":"2024-04-14T08:38:46.214568+0000","last_became_active":"2024-04-14T08:38:44.231184+0000","last_became_peered":"2024-04-14T08:38:44.231184+0000","last_unstale":"2024-04-14T08:38:46.214568+0000","last_undegraded":"2024-04-14T08:38:46.214568+0000","last_fullsized":"2024-04-14T08:38:46.214568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:59:08.058016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215612+0000","last_change":"2024-04-14T08:38:45.231654+0000","last_active":"2024-04-14T08:38:46.215612+0000","last_peered":"2024-04-14T08:38:46.215612+0000","last_clean":"2024-04-14T08:38:46.215612+0000","last_became_active":"2024-04-14T08:38:45.231165+0000","last_became_peered":"2024-04-14T08:38:45.231165+0000","last_unstale":"2024-04-14T08:38:46.215612+0000","last_undegraded":"2024-04-14T08:38:46.215612+0000","last_fullsized":"2024-04-14T08:38:46.215612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:43:03.191778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425738+0000","last_change":"2024-04-14T08:38:44.224713+0000","last_active":"2024-04-14T08:38:46.425738+0000","last_peered":"2024-04-14T08:38:46.425738+0000","last_clean":"2024-04-14T08:38:46.425738+0000","last_became_active":"2024-04-14T08:38:44.224437+0000","last_became_peered":"2024-04-14T08:38:44.224437+0000","last_unstale":"2024-04-14T08:38:46.425738+0000","last_undegraded":"2024-04-14T08:38:46.425738+0000","last_fullsized":"2024-04-14T08:38:46.425738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:46:42.922320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215220+0000","last_change":"2024-04-14T08:38:45.219595+0000","last_active":"2024-04-14T08:38:46.215220+0000","last_peered":"2024-04-14T08:38:46.215220+0000","last_clean":"2024-04-14T08:38:46.215220+0000","last_became_active":"2024-04-14T08:38:45.219269+0000","last_became_peered":"2024-04-14T08:38:45.219269+0000","last_unstale":"2024-04-14T08:38:46.215220+0000","last_undegraded":"2024-04-14T08:38:46.215220+0000","last_fullsized":"2024-04-14T08:38:46.215220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:48:59.329012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426402+0000","last_change":"2024-04-14T08:38:44.219411+0000","last_active":"2024-04-14T08:38:46.426402+0000","last_peered":"2024-04-14T08:38:46.426402+0000","last_clean":"2024-04-14T08:38:46.426402+0000","last_became_active":"2024-04-14T08:38:44.219130+0000","last_became_peered":"2024-04-14T08:38:44.219130+0000","last_unstale":"2024-04-14T08:38:46.426402+0000","last_undegraded":"2024-04-14T08:38:46.426402+0000","last_fullsized":"2024-04-14T08:38:46.426402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:29:41.073106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426304+0000","last_change":"2024-04-14T08:38:45.239856+0000","last_active":"2024-04-14T08:38:46.426304+0000","last_peered":"2024-04-14T08:38:46.426304+0000","last_clean":"2024-04-14T08:38:46.426304+0000","last_became_active":"2024-04-14T08:38:45.239587+0000","last_became_peered":"2024-04-14T08:38:45.239587+0000","last_unstale":"2024-04-14T08:38:46.426304+0000","last_undegraded":"2024-04-14T08:38:46.426304+0000","last_fullsized":"2024-04-14T08:38:46.426304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:48:28.586545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"85'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.216018+0000","last_change":"2024-04-14T08:38:44.231600+0000","last_active":"2024-04-14T08:38:46.216018+0000","last_peered":"2024-04-14T08:38:46.216018+0000","last_clean":"2024-04-14T08:38:46.216018+0000","last_became_active":"2024-04-14T08:38:44.231276+0000","last_became_peered":"2024-04-14T08:38:44.231276+0000","last_unstale":"2024-04-14T08:38:46.216018+0000","last_undegraded":"2024-04-14T08:38:46.216018+0000","last_fullsized":"2024-04-14T08:38:46.216018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:32.465724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427929+0000","last_change":"2024-04-14T08:38:45.223782+0000","last_active":"2024-04-14T08:38:46.427929+0000","last_peered":"2024-04-14T08:38:46.427929+0000","last_clean":"2024-04-14T08:38:46.427929+0000","last_became_active":"2024-04-14T08:38:45.223161+0000","last_became_peered":"2024-04-14T08:38:45.223161+0000","last_unstale":"2024-04-14T08:38:46.427929+0000","last_undegraded":"2024-04-14T08:38:46.427929+0000","last_fullsized":"2024-04-14T08:38:46.427929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:29:42.080742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"85'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426988+0000","last_change":"2024-04-14T08:38:44.230303+0000","last_active":"2024-04-14T08:38:46.426988+0000","last_peered":"2024-04-14T08:38:46.426988+0000","last_clean":"2024-04-14T08:38:46.426988+0000","last_became_active":"2024-04-14T08:38:44.230051+0000","last_became_peered":"2024-04-14T08:38:44.230051+0000","last_unstale":"2024-04-14T08:38:46.426988+0000","last_undegraded":"2024-04-14T08:38:46.426988+0000","last_fullsized":"2024-04-14T08:38:46.426988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:48:27.578909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427524+0000","last_change":"2024-04-14T08:38:45.220109+0000","last_active":"2024-04-14T08:38:46.427524+0000","last_peered":"2024-04-14T08:38:46.427524+0000","last_clean":"2024-04-14T08:38:46.427524+0000","last_became_active":"2024-04-14T08:38:45.219039+0000","last_became_peered":"2024-04-14T08:38:45.219039+0000","last_unstale":"2024-04-14T08:38:46.427524+0000","last_undegraded":"2024-04-14T08:38:46.427524+0000","last_fullsized":"2024-04-14T08:38:46.427524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:46:43.929956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215121+0000","last_change":"2024-04-14T08:38:44.230951+0000","last_active":"2024-04-14T08:38:46.215121+0000","last_peered":"2024-04-14T08:38:46.215121+0000","last_clean":"2024-04-14T08:38:46.215121+0000","last_became_active":"2024-04-14T08:38:44.230499+0000","last_became_peered":"2024-04-14T08:38:44.230499+0000","last_unstale":"2024-04-14T08:38:46.215121+0000","last_undegraded":"2024-04-14T08:38:46.215121+0000","last_fullsized":"2024-04-14T08:38:46.215121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:55:56.125870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.214738+0000","last_change":"2024-04-14T08:38:45.231284+0000","last_active":"2024-04-14T08:38:46.214738+0000","last_peered":"2024-04-14T08:38:46.214738+0000","last_clean":"2024-04-14T08:38:46.214738+0000","last_became_active":"2024-04-14T08:38:45.231016+0000","last_became_peered":"2024-04-14T08:38:45.231016+0000","last_unstale":"2024-04-14T08:38:46.214738+0000","last_undegraded":"2024-04-14T08:38:46.214738+0000","last_fullsized":"2024-04-14T08:38:46.214738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:46:43.929956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425847+0000","last_change":"2024-04-14T08:38:44.230300+0000","last_active":"2024-04-14T08:38:46.425847+0000","last_peered":"2024-04-14T08:38:46.425847+0000","last_clean":"2024-04-14T08:38:46.425847+0000","last_became_active":"2024-04-14T08:38:44.230024+0000","last_became_peered":"2024-04-14T08:38:44.230024+0000","last_unstale":"2024-04-14T08:38:46.425847+0000","last_undegraded":"2024-04-14T08:38:46.425847+0000","last_fullsized":"2024-04-14T08:38:46.425847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:24:57.813701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.211958+0000","last_change":"2024-04-14T08:38:45.231507+0000","last_active":"2024-04-14T08:38:46.211958+0000","last_peered":"2024-04-14T08:38:46.211958+0000","last_clean":"2024-04-14T08:38:46.211958+0000","last_became_active":"2024-04-14T08:38:45.231256+0000","last_became_peered":"2024-04-14T08:38:45.231256+0000","last_unstale":"2024-04-14T08:38:46.211958+0000","last_undegraded":"2024-04-14T08:38:46.211958+0000","last_fullsized":"2024-04-14T08:38:46.211958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:46:09.260855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427253+0000","last_change":"2024-04-14T08:38:44.224811+0000","last_active":"2024-04-14T08:38:46.427253+0000","last_peered":"2024-04-14T08:38:46.427253+0000","last_clean":"2024-04-14T08:38:46.427253+0000","last_became_active":"2024-04-14T08:38:44.224541+0000","last_became_peered":"2024-04-14T08:38:44.224541+0000","last_unstale":"2024-04-14T08:38:46.427253+0000","last_undegraded":"2024-04-14T08:38:46.427253+0000","last_fullsized":"2024-04-14T08:38:46.427253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:41:55.125251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.212237+0000","last_change":"2024-04-14T08:38:45.236938+0000","last_active":"2024-04-14T08:38:46.212237+0000","last_peered":"2024-04-14T08:38:46.212237+0000","last_clean":"2024-04-14T08:38:46.212237+0000","last_became_active":"2024-04-14T08:38:45.236643+0000","last_became_peered":"2024-04-14T08:38:45.236643+0000","last_unstale":"2024-04-14T08:38:46.212237+0000","last_undegraded":"2024-04-14T08:38:46.212237+0000","last_fullsized":"2024-04-14T08:38:46.212237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:21:05.832464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"85'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427530+0000","last_change":"2024-04-14T08:38:44.216083+0000","last_active":"2024-04-14T08:38:46.427530+0000","last_peered":"2024-04-14T08:38:46.427530+0000","last_clean":"2024-04-14T08:38:46.427530+0000","last_became_active":"2024-04-14T08:38:44.215754+0000","last_became_peered":"2024-04-14T08:38:44.215754+0000","last_unstale":"2024-04-14T08:38:46.427530+0000","last_undegraded":"2024-04-14T08:38:46.427530+0000","last_fullsized":"2024-04-14T08:38:46.427530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:27:28.589254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427007+0000","last_change":"2024-04-14T08:38:45.226724+0000","last_active":"2024-04-14T08:38:46.427007+0000","last_peered":"2024-04-14T08:38:46.427007+0000","last_clean":"2024-04-14T08:38:46.427007+0000","last_became_active":"2024-04-14T08:38:45.226439+0000","last_became_peered":"2024-04-14T08:38:45.226439+0000","last_unstale":"2024-04-14T08:38:46.427007+0000","last_undegraded":"2024-04-14T08:38:46.427007+0000","last_fullsized":"2024-04-14T08:38:46.427007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:38:51.955043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425985+0000","last_change":"2024-04-14T08:38:44.222002+0000","last_active":"2024-04-14T08:38:46.425985+0000","last_peered":"2024-04-14T08:38:46.425985+0000","last_clean":"2024-04-14T08:38:46.425985+0000","last_became_active":"2024-04-14T08:38:44.221679+0000","last_became_peered":"2024-04-14T08:38:44.221679+0000","last_unstale":"2024-04-14T08:38:46.425985+0000","last_undegraded":"2024-04-14T08:38:46.425985+0000","last_fullsized":"2024-04-14T08:38:46.425985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:37.147141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425553+0000","last_change":"2024-04-14T08:38:45.223449+0000","last_active":"2024-04-14T08:38:46.425553+0000","last_peered":"2024-04-14T08:38:46.425553+0000","last_clean":"2024-04-14T08:38:46.425553+0000","last_became_active":"2024-04-14T08:38:45.223097+0000","last_became_peered":"2024-04-14T08:38:45.223097+0000","last_unstale":"2024-04-14T08:38:46.425553+0000","last_undegraded":"2024-04-14T08:38:46.425553+0000","last_fullsized":"2024-04-14T08:38:46.425553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:29:42.080742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215004+0000","last_change":"2024-04-14T08:38:44.220058+0000","last_active":"2024-04-14T08:38:46.215004+0000","last_peered":"2024-04-14T08:38:46.215004+0000","last_clean":"2024-04-14T08:38:46.215004+0000","last_became_active":"2024-04-14T08:38:44.219721+0000","last_became_peered":"2024-04-14T08:38:44.219721+0000","last_unstale":"2024-04-14T08:38:46.215004+0000","last_undegraded":"2024-04-14T08:38:46.215004+0000","last_fullsized":"2024-04-14T08:38:46.215004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:54:15.282969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.215726+0000","last_change":"2024-04-14T08:38:45.225365+0000","last_active":"2024-04-14T08:38:46.215726+0000","last_peered":"2024-04-14T08:38:46.215726+0000","last_clean":"2024-04-14T08:38:46.215726+0000","last_became_active":"2024-04-14T08:38:45.224472+0000","last_became_peered":"2024-04-14T08:38:45.224472+0000","last_unstale":"2024-04-14T08:38:46.215726+0000","last_undegraded":"2024-04-14T08:38:46.215726+0000","last_fullsized":"2024-04-14T08:38:46.215726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:29:42.080742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.216110+0000","last_change":"2024-04-14T08:38:44.221416+0000","last_active":"2024-04-14T08:38:46.216110+0000","last_peered":"2024-04-14T08:38:46.216110+0000","last_clean":"2024-04-14T08:38:46.216110+0000","last_became_active":"2024-04-14T08:38:44.221138+0000","last_became_peered":"2024-04-14T08:38:44.221138+0000","last_unstale":"2024-04-14T08:38:46.216110+0000","last_undegraded":"2024-04-14T08:38:46.216110+0000","last_fullsized":"2024-04-14T08:38:46.216110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:41:55.125251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425858+0000","last_change":"2024-04-14T08:38:45.228805+0000","last_active":"2024-04-14T08:38:46.425858+0000","last_peered":"2024-04-14T08:38:46.425858+0000","last_clean":"2024-04-14T08:38:46.425858+0000","last_became_active":"2024-04-14T08:38:45.228551+0000","last_became_peered":"2024-04-14T08:38:45.228551+0000","last_unstale":"2024-04-14T08:38:46.425858+0000","last_undegraded":"2024-04-14T08:38:46.425858+0000","last_fullsized":"2024-04-14T08:38:46.425858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:51.856770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425313+0000","last_change":"2024-04-14T08:38:44.224441+0000","last_active":"2024-04-14T08:38:46.425313+0000","last_peered":"2024-04-14T08:38:46.425313+0000","last_clean":"2024-04-14T08:38:46.425313+0000","last_became_active":"2024-04-14T08:38:44.224137+0000","last_became_peered":"2024-04-14T08:38:44.224137+0000","last_unstale":"2024-04-14T08:38:46.425313+0000","last_undegraded":"2024-04-14T08:38:46.425313+0000","last_fullsized":"2024-04-14T08:38:46.425313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:14:09.434053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425305+0000","last_change":"2024-04-14T08:38:45.228056+0000","last_active":"2024-04-14T08:38:46.425305+0000","last_peered":"2024-04-14T08:38:46.425305+0000","last_clean":"2024-04-14T08:38:46.425305+0000","last_became_active":"2024-04-14T08:38:45.227776+0000","last_became_peered":"2024-04-14T08:38:45.227776+0000","last_unstale":"2024-04-14T08:38:46.425305+0000","last_undegraded":"2024-04-14T08:38:46.425305+0000","last_fullsized":"2024-04-14T08:38:46.425305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:23.852025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.211964+0000","last_change":"2024-04-14T08:38:44.220261+0000","last_active":"2024-04-14T08:38:46.211964+0000","last_peered":"2024-04-14T08:38:46.211964+0000","last_clean":"2024-04-14T08:38:46.211964+0000","last_became_active":"2024-04-14T08:38:44.220001+0000","last_became_peered":"2024-04-14T08:38:44.220001+0000","last_unstale":"2024-04-14T08:38:46.211964+0000","last_undegraded":"2024-04-14T08:38:46.211964+0000","last_fullsized":"2024-04-14T08:38:46.211964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:46:03.601356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425410+0000","last_change":"2024-04-14T08:38:45.230523+0000","last_active":"2024-04-14T08:38:46.425410+0000","last_peered":"2024-04-14T08:38:46.425410+0000","last_clean":"2024-04-14T08:38:46.425410+0000","last_became_active":"2024-04-14T08:38:45.230140+0000","last_became_peered":"2024-04-14T08:38:45.230140+0000","last_unstale":"2024-04-14T08:38:46.425410+0000","last_undegraded":"2024-04-14T08:38:46.425410+0000","last_fullsized":"2024-04-14T08:38:46.425410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:43:44.976330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425425+0000","last_change":"2024-04-14T08:38:44.227400+0000","last_active":"2024-04-14T08:38:46.425425+0000","last_peered":"2024-04-14T08:38:46.425425+0000","last_clean":"2024-04-14T08:38:46.425425+0000","last_became_active":"2024-04-14T08:38:44.226963+0000","last_became_peered":"2024-04-14T08:38:44.226963+0000","last_unstale":"2024-04-14T08:38:46.425425+0000","last_undegraded":"2024-04-14T08:38:46.425425+0000","last_fullsized":"2024-04-14T08:38:46.425425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:00:17.000628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425934+0000","last_change":"2024-04-14T08:38:45.224514+0000","last_active":"2024-04-14T08:38:46.425934+0000","last_peered":"2024-04-14T08:38:46.425934+0000","last_clean":"2024-04-14T08:38:46.425934+0000","last_became_active":"2024-04-14T08:38:45.224210+0000","last_became_peered":"2024-04-14T08:38:45.224210+0000","last_unstale":"2024-04-14T08:38:46.425934+0000","last_undegraded":"2024-04-14T08:38:46.425934+0000","last_fullsized":"2024-04-14T08:38:46.425934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:51:27.297852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.211964+0000","last_change":"2024-04-14T08:38:44.221259+0000","last_active":"2024-04-14T08:38:46.211964+0000","last_peered":"2024-04-14T08:38:46.211964+0000","last_clean":"2024-04-14T08:38:46.211964+0000","last_became_active":"2024-04-14T08:38:44.220936+0000","last_became_peered":"2024-04-14T08:38:44.220936+0000","last_unstale":"2024-04-14T08:38:46.211964+0000","last_undegraded":"2024-04-14T08:38:46.211964+0000","last_fullsized":"2024-04-14T08:38:46.211964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:21:02.763219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426127+0000","last_change":"2024-04-14T08:38:45.227242+0000","last_active":"2024-04-14T08:38:46.426127+0000","last_peered":"2024-04-14T08:38:46.426127+0000","last_clean":"2024-04-14T08:38:46.426127+0000","last_became_active":"2024-04-14T08:38:45.226976+0000","last_became_peered":"2024-04-14T08:38:45.226976+0000","last_unstale":"2024-04-14T08:38:46.426127+0000","last_undegraded":"2024-04-14T08:38:46.426127+0000","last_fullsized":"2024-04-14T08:38:46.426127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:11:23.852025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.214994+0000","last_change":"2024-04-14T08:38:44.222193+0000","last_active":"2024-04-14T08:38:46.214994+0000","last_peered":"2024-04-14T08:38:46.214994+0000","last_clean":"2024-04-14T08:38:46.214994+0000","last_became_active":"2024-04-14T08:38:44.221895+0000","last_became_peered":"2024-04-14T08:38:44.221895+0000","last_unstale":"2024-04-14T08:38:46.214994+0000","last_undegraded":"2024-04-14T08:38:46.214994+0000","last_fullsized":"2024-04-14T08:38:46.214994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:36:12.417551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427092+0000","last_change":"2024-04-14T08:38:45.227336+0000","last_active":"2024-04-14T08:38:46.427092+0000","last_peered":"2024-04-14T08:38:46.427092+0000","last_clean":"2024-04-14T08:38:46.427092+0000","last_became_active":"2024-04-14T08:38:45.227086+0000","last_became_peered":"2024-04-14T08:38:45.227086+0000","last_unstale":"2024-04-14T08:38:46.427092+0000","last_undegraded":"2024-04-14T08:38:46.427092+0000","last_fullsized":"2024-04-14T08:38:46.427092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:18.995318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.216037+0000","last_change":"2024-04-14T08:38:44.220634+0000","last_active":"2024-04-14T08:38:46.216037+0000","last_peered":"2024-04-14T08:38:46.216037+0000","last_clean":"2024-04-14T08:38:46.216037+0000","last_became_active":"2024-04-14T08:38:44.220352+0000","last_became_peered":"2024-04-14T08:38:44.220352+0000","last_unstale":"2024-04-14T08:38:46.216037+0000","last_undegraded":"2024-04-14T08:38:46.216037+0000","last_fullsized":"2024-04-14T08:38:46.216037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:39:12.731967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426834+0000","last_change":"2024-04-14T08:38:45.225949+0000","last_active":"2024-04-14T08:38:46.426834+0000","last_peered":"2024-04-14T08:38:46.426834+0000","last_clean":"2024-04-14T08:38:46.426834+0000","last_became_active":"2024-04-14T08:38:45.225670+0000","last_became_peered":"2024-04-14T08:38:45.225670+0000","last_unstale":"2024-04-14T08:38:46.426834+0000","last_undegraded":"2024-04-14T08:38:46.426834+0000","last_fullsized":"2024-04-14T08:38:46.426834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:41:41.197641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.216123+0000","last_change":"2024-04-14T08:38:44.230684+0000","last_active":"2024-04-14T08:38:46.216123+0000","last_peered":"2024-04-14T08:38:46.216123+0000","last_clean":"2024-04-14T08:38:46.216123+0000","last_became_active":"2024-04-14T08:38:44.230211+0000","last_became_peered":"2024-04-14T08:38:44.230211+0000","last_unstale":"2024-04-14T08:38:46.216123+0000","last_undegraded":"2024-04-14T08:38:46.216123+0000","last_fullsized":"2024-04-14T08:38:46.216123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:12:06.216424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425692+0000","last_change":"2024-04-14T08:38:45.232276+0000","last_active":"2024-04-14T08:38:46.425692+0000","last_peered":"2024-04-14T08:38:46.425692+0000","last_clean":"2024-04-14T08:38:46.425692+0000","last_became_active":"2024-04-14T08:38:45.231877+0000","last_became_peered":"2024-04-14T08:38:45.231877+0000","last_unstale":"2024-04-14T08:38:46.425692+0000","last_undegraded":"2024-04-14T08:38:46.425692+0000","last_fullsized":"2024-04-14T08:38:46.425692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:55:57.133505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426128+0000","last_change":"2024-04-14T08:38:44.224050+0000","last_active":"2024-04-14T08:38:46.426128+0000","last_peered":"2024-04-14T08:38:46.426128+0000","last_clean":"2024-04-14T08:38:46.426128+0000","last_became_active":"2024-04-14T08:38:44.223667+0000","last_became_peered":"2024-04-14T08:38:44.223667+0000","last_unstale":"2024-04-14T08:38:46.426128+0000","last_undegraded":"2024-04-14T08:38:46.426128+0000","last_fullsized":"2024-04-14T08:38:46.426128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:56:30.743467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.211957+0000","last_change":"2024-04-14T08:38:45.222775+0000","last_active":"2024-04-14T08:38:46.211957+0000","last_peered":"2024-04-14T08:38:46.211957+0000","last_clean":"2024-04-14T08:38:46.211957+0000","last_became_active":"2024-04-14T08:38:45.222140+0000","last_became_peered":"2024-04-14T08:38:45.222140+0000","last_unstale":"2024-04-14T08:38:46.211957+0000","last_undegraded":"2024-04-14T08:38:46.211957+0000","last_fullsized":"2024-04-14T08:38:46.211957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:36:01.807476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426697+0000","last_change":"2024-04-14T08:38:44.230396+0000","last_active":"2024-04-14T08:38:46.426697+0000","last_peered":"2024-04-14T08:38:46.426697+0000","last_clean":"2024-04-14T08:38:46.426697+0000","last_became_active":"2024-04-14T08:38:44.230103+0000","last_became_peered":"2024-04-14T08:38:44.230103+0000","last_unstale":"2024-04-14T08:38:46.426697+0000","last_undegraded":"2024-04-14T08:38:46.426697+0000","last_fullsized":"2024-04-14T08:38:46.426697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:59:37.596879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426356+0000","last_change":"2024-04-14T08:38:45.233056+0000","last_active":"2024-04-14T08:38:46.426356+0000","last_peered":"2024-04-14T08:38:46.426356+0000","last_clean":"2024-04-14T08:38:46.426356+0000","last_became_active":"2024-04-14T08:38:45.232806+0000","last_became_peered":"2024-04-14T08:38:45.232806+0000","last_unstale":"2024-04-14T08:38:46.426356+0000","last_undegraded":"2024-04-14T08:38:46.426356+0000","last_fullsized":"2024-04-14T08:38:46.426356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:59:38.604514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425484+0000","last_change":"2024-04-14T08:38:44.224830+0000","last_active":"2024-04-14T08:38:46.425484+0000","last_peered":"2024-04-14T08:38:46.425484+0000","last_clean":"2024-04-14T08:38:46.425484+0000","last_became_active":"2024-04-14T08:38:44.224558+0000","last_became_peered":"2024-04-14T08:38:44.224558+0000","last_unstale":"2024-04-14T08:38:46.425484+0000","last_undegraded":"2024-04-14T08:38:46.425484+0000","last_fullsized":"2024-04-14T08:38:46.425484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:34:24.396744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425476+0000","last_change":"2024-04-14T08:38:45.225752+0000","last_active":"2024-04-14T08:38:46.425476+0000","last_peered":"2024-04-14T08:38:46.425476+0000","last_clean":"2024-04-14T08:38:46.425476+0000","last_became_active":"2024-04-14T08:38:45.225461+0000","last_became_peered":"2024-04-14T08:38:45.225461+0000","last_unstale":"2024-04-14T08:38:46.425476+0000","last_undegraded":"2024-04-14T08:38:46.425476+0000","last_fullsized":"2024-04-14T08:38:46.425476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:35:51.821657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.425730+0000","last_change":"2024-04-14T08:38:44.224592+0000","last_active":"2024-04-14T08:38:46.425730+0000","last_peered":"2024-04-14T08:38:46.425730+0000","last_clean":"2024-04-14T08:38:46.425730+0000","last_became_active":"2024-04-14T08:38:44.224287+0000","last_became_peered":"2024-04-14T08:38:44.224287+0000","last_unstale":"2024-04-14T08:38:46.425730+0000","last_undegraded":"2024-04-14T08:38:46.425730+0000","last_fullsized":"2024-04-14T08:38:46.425730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:00:17.000628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.426662+0000","last_change":"2024-04-14T08:38:45.238151+0000","last_active":"2024-04-14T08:38:46.426662+0000","last_peered":"2024-04-14T08:38:46.426662+0000","last_clean":"2024-04-14T08:38:46.426662+0000","last_became_active":"2024-04-14T08:38:45.237877+0000","last_became_peered":"2024-04-14T08:38:45.237877+0000","last_unstale":"2024-04-14T08:38:46.426662+0000","last_undegraded":"2024-04-14T08:38:46.426662+0000","last_fullsized":"2024-04-14T08:38:46.426662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:52:13.450656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.214633+0000","last_change":"2024-04-14T08:38:44.219166+0000","last_active":"2024-04-14T08:38:46.214633+0000","last_peered":"2024-04-14T08:38:46.214633+0000","last_clean":"2024-04-14T08:38:46.214633+0000","last_became_active":"2024-04-14T08:38:44.218905+0000","last_became_peered":"2024-04-14T08:38:44.218905+0000","last_unstale":"2024-04-14T08:38:46.214633+0000","last_undegraded":"2024-04-14T08:38:46.214633+0000","last_fullsized":"2024-04-14T08:38:46.214633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:24:57.813701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.214891+0000","last_change":"2024-04-14T08:38:45.223615+0000","last_active":"2024-04-14T08:38:46.214891+0000","last_peered":"2024-04-14T08:38:46.214891+0000","last_clean":"2024-04-14T08:38:46.214891+0000","last_became_active":"2024-04-14T08:38:45.221967+0000","last_became_peered":"2024-04-14T08:38:45.221967+0000","last_unstale":"2024-04-14T08:38:46.214891+0000","last_undegraded":"2024-04-14T08:38:46.214891+0000","last_fullsized":"2024-04-14T08:38:46.214891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:44.200740+0000","last_clean_scrub_stamp":"2024-04-14T08:38:44.200740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:26.996997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.427817+0000","last_change":"2024-04-14T08:38:44.224166+0000","last_active":"2024-04-14T08:38:46.427817+0000","last_peered":"2024-04-14T08:38:46.427817+0000","last_clean":"2024-04-14T08:38:46.427817+0000","last_became_active":"2024-04-14T08:38:44.223244+0000","last_became_peered":"2024-04-14T08:38:44.223244+0000","last_unstale":"2024-04-14T08:38:46.427817+0000","last_undegraded":"2024-04-14T08:38:46.427817+0000","last_fullsized":"2024-04-14T08:38:46.427817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:38:43.193104+0000","last_clean_scrub_stamp":"2024-04-14T08:38:43.193104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:12:06.216424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":303,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-14T08:38:46.214731+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:38:46.214731+0000","last_peered":"2024-04-14T08:38:46.214731+0000","last_clean":"2024-04-14T08:38:46.214731+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:38:46.214731+0000","last_undegraded":"2024-04-14T08:38:46.214731+0000","last_fullsized":"2024-04-14T08:38:46.214731+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1662976,"data_stored":1075637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343764,"statfs":{"total":96636764160,"available":96608014336,"internally_reserved":0,"allocated":1081344,"data_stored":487429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":485213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1069056,"data_stored":485373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1060864,"data_stored":485213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574925,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1060864,"data_stored":485213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1064960,"data_stored":485283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1679360,"data_stored":1077899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:38:52.933 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-14T08:38:52.933 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-14T08:38:53.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.075+0000 7fdde660f640 1 Processor -- start 2024-04-14T08:38:53.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.075+0000 7fdde660f640 1 -- start start 2024-04-14T08:38:53.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.076+0000 7fdde660f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdde0162ba0 0x7fdde00d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:53.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.076+0000 7fdde660f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdde00cefb0 con 0x7fdde015e430 2024-04-14T08:38:53.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.076+0000 7fdde660f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdde00cf290 con 0x7fdde0162710 2024-04-14T08:38:53.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.076+0000 7fdde660f640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fdde00cf570 con 0x7fdde0162ba0 2024-04-14T08:38:53.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.076+0000 7fdddffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fdde015e430 0x7fdde0162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44870/0 (socket says 172.21.15.172:44870) 2024-04-14T08:38:53.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.076+0000 7fdddffff640 1 -- 172.21.15.172:0/2448732648 learned_addr learned my addr 172.21.15.172:0/2448732648 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:53.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.076+0000 7fdde4bac640 1 --2- 172.21.15.172:0/2448732648 >> v2:172.21.15.177:3300/0 conn(0x7fdde0162ba0 0x7fdde00d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:53.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.077+0000 7fdddeffd640 1 -- 172.21.15.172:0/2448732648 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025541768 0 0) 0x7fdde00cefb0 con 0x7fdde015e430 2024-04-14T08:38:53.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.077+0000 7fdddeffd640 1 -- 172.21.15.172:0/2448732648 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc8003520 con 0x7fdde015e430 2024-04-14T08:38:53.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.077+0000 7fdddeffd640 1 -- 172.21.15.172:0/2448732648 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1823568802 0 0) 0x7fdde00cf290 con 0x7fdde0162710 2024-04-14T08:38:53.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.077+0000 7fdddeffd640 1 -- 172.21.15.172:0/2448732648 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdde00cefb0 con 0x7fdde0162710 2024-04-14T08:38:53.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.077+0000 7fdde4bac640 1 -- 172.21.15.172:0/2448732648 >> v1:172.21.15.177:6789/0 conn(0x7fdde0162710 legacy=0x7fdde00d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.077+0000 7fdde4bac640 1 -- 172.21.15.172:0/2448732648 >> v1:172.21.15.172:6789/0 conn(0x7fdde015e430 legacy=0x7fdde0162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.077+0000 7fdde4bac640 1 -- 172.21.15.172:0/2448732648 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde00cf850 con 0x7fdde0162ba0 2024-04-14T08:38:53.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.077+0000 7fdde4bac640 1 --2- 172.21.15.172:0/2448732648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdde0162ba0 0x7fdde00d2500 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009d90 tx=0x7fddd802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f6f2acd590be693 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:53.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.078+0000 7fdddeffd640 1 -- 172.21.15.172:0/2448732648 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8039070 con 0x7fdde0162ba0 2024-04-14T08:38:53.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.078+0000 7fdddeffd640 1 -- 172.21.15.172:0/2448732648 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd802cd00 con 0x7fdde0162ba0 2024-04-14T08:38:53.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.078+0000 7fdddeffd640 1 -- 172.21.15.172:0/2448732648 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8034640 con 0x7fdde0162ba0 2024-04-14T08:38:53.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.078+0000 7fdde660f640 1 -- 172.21.15.172:0/2448732648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdde0162ba0 msgr2=0x7fdde00d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.078+0000 7fdde660f640 1 --2- 172.21.15.172:0/2448732648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdde0162ba0 0x7fdde00d2500 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009d90 tx=0x7fddd802bf50 comp rx=0 tx=0).stop 2024-04-14T08:38:53.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.078+0000 7fdde660f640 1 -- 172.21.15.172:0/2448732648 shutdown_connections 2024-04-14T08:38:53.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.078+0000 7fdde660f640 1 --2- 172.21.15.172:0/2448732648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdde0162ba0 0x7fdde00d2500 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.078+0000 7fdde660f640 1 -- 172.21.15.172:0/2448732648 >> 172.21.15.172:0/2448732648 conn(0x7fdde009d560 msgr2=0x7fdde00d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:53.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.078+0000 7fdde660f640 1 -- 172.21.15.172:0/2448732648 shutdown_connections 2024-04-14T08:38:53.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.078+0000 7fdde660f640 1 -- 172.21.15.172:0/2448732648 wait complete. 2024-04-14T08:38:53.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.079+0000 7fdde660f640 1 Processor -- start 2024-04-14T08:38:53.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.080+0000 7fdde660f640 1 -- start start 2024-04-14T08:38:53.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.080+0000 7fdde660f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdde015e430 0x7fdde0160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:53.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.080+0000 7fdde660f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdde0162710 0x7fdde01612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:53.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.080+0000 7fdde660f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdde0162ba0 0x7fdde01f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:53.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.080+0000 7fdde660f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdde00d4e20 con 0x7fdde015e430 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.080+0000 7fdde660f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdde015e810 con 0x7fdde0162ba0 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.080+0000 7fdde660f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdde00d0050 con 0x7fdde0162710 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.080+0000 7fdddffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdde015e430 0x7fdde0160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.080+0000 7fdddffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdde015e430 0x7fdde0160d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37932/0 (socket says 172.21.15.172:37932) 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.080+0000 7fdddffff640 1 -- 172.21.15.172:0/2421195593 learned_addr learned my addr 172.21.15.172:0/2421195593 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.080+0000 7fdde4bac640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdde0162ba0 0x7fdde01f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.080+0000 7fdddf7fe640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdde0162710 0x7fdde01612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.081+0000 7fdddffff640 1 -- 172.21.15.172:0/2421195593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdde0162710 msgr2=0x7fdde01612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.081+0000 7fdddffff640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdde0162710 0x7fdde01612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.081+0000 7fdddffff640 1 -- 172.21.15.172:0/2421195593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdde0162ba0 msgr2=0x7fdde01f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.081+0000 7fdddffff640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdde0162ba0 0x7fdde01f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.081+0000 7fdddffff640 1 -- 172.21.15.172:0/2421195593 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde01617f0 con 0x7fdde015e430 2024-04-14T08:38:53.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.081+0000 7fdde4bac640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdde0162ba0 0x7fdde01f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:53.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.081+0000 7fdddffff640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdde015e430 0x7fdde0160d70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fddd000c410 tx=0x7fddd000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:53.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.081+0000 7fdddcff9640 1 -- 172.21.15.172:0/2421195593 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0017070 con 0x7fdde015e430 2024-04-14T08:38:53.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.081+0000 7fdddcff9640 1 -- 172.21.15.172:0/2421195593 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd0005ce0 con 0x7fdde015e430 2024-04-14T08:38:53.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.081+0000 7fdde660f640 1 -- 172.21.15.172:0/2421195593 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde01f4550 con 0x7fdde015e430 2024-04-14T08:38:53.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.081+0000 7fdddcff9640 1 -- 172.21.15.172:0/2421195593 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd00072d0 con 0x7fdde015e430 2024-04-14T08:38:53.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.081+0000 7fdde660f640 1 -- 172.21.15.172:0/2421195593 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde01f4b10 con 0x7fdde015e430 2024-04-14T08:38:53.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.082+0000 7fdde660f640 1 -- 172.21.15.172:0/2421195593 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde0068590 con 0x7fdde015e430 2024-04-14T08:38:53.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.082+0000 7fdddcff9640 1 -- 172.21.15.172:0/2421195593 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddd001a070 con 0x7fdde015e430 2024-04-14T08:38:53.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.083+0000 7fdddcff9640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fddcc06c8f0 0x7fddcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:53.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.083+0000 7fdddf7fe640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fddcc06c8f0 0x7fddcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:53.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.083+0000 7fdddcff9640 1 -- 172.21.15.172:0/2421195593 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fddd008b080 con 0x7fdde015e430 2024-04-14T08:38:53.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.083+0000 7fdddf7fe640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fddcc06c8f0 0x7fddcc06edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fddd4007fd0 tx=0x7fddd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:53.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.087+0000 7fdddcff9640 1 -- 172.21.15.172:0/2421195593 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fddd0058c50 con 0x7fdde015e430 2024-04-14T08:38:53.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.335+0000 7fdde660f640 1 -- 172.21.15.172:0/2421195593 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fdde0074210 con 0x7fdde015e430 2024-04-14T08:38:53.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.337+0000 7fdddcff9640 1 -- 172.21.15.172:0/2421195593 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fddd005cf50 con 0x7fdde015e430 2024-04-14T08:38:53.338 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2024-04-14T08:38:53.339 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA== 2024-04-14T08:38:53.339 INFO:teuthology.orchestra.run.smithi172.stdout: caps mds = "allow" 2024-04-14T08:38:53.339 INFO:teuthology.orchestra.run.smithi172.stdout: caps mon = "allow r" 2024-04-14T08:38:53.339 INFO:teuthology.orchestra.run.smithi172.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-14T08:38:53.339 INFO:teuthology.orchestra.run.smithi172.stderr:updated caps for client.0 2024-04-14T08:38:53.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.339+0000 7fdde660f640 1 -- 172.21.15.172:0/2421195593 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fddcc06c8f0 msgr2=0x7fddcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.339+0000 7fdde660f640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fddcc06c8f0 0x7fddcc06edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fddd4007fd0 tx=0x7fddd4009040 comp rx=0 tx=0).stop 2024-04-14T08:38:53.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.339+0000 7fdde660f640 1 -- 172.21.15.172:0/2421195593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdde015e430 msgr2=0x7fdde0160d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.339+0000 7fdde660f640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdde015e430 0x7fdde0160d70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fddd000c410 tx=0x7fddd000c8d0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.339+0000 7fdde660f640 1 -- 172.21.15.172:0/2421195593 shutdown_connections 2024-04-14T08:38:53.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.339+0000 7fdde660f640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fddcc06c8f0 0x7fddcc06edb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.339+0000 7fdde660f640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdde0162ba0 0x7fdde01f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.339+0000 7fdde660f640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdde0162710 0x7fdde01612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.339+0000 7fdde660f640 1 --2- 172.21.15.172:0/2421195593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdde015e430 0x7fdde0160d70 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.339+0000 7fdde660f640 1 -- 172.21.15.172:0/2421195593 >> 172.21.15.172:0/2421195593 conn(0x7fdde009d560 msgr2=0x7fdde0161bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:53.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.339+0000 7fdde660f640 1 -- 172.21.15.172:0/2421195593 shutdown_connections 2024-04-14T08:38:53.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.339+0000 7fdde660f640 1 -- 172.21.15.172:0/2421195593 wait complete. 2024-04-14T08:38:53.352 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:38:53.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.450+0000 7f1a3b498640 1 Processor -- start 2024-04-14T08:38:53.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.451+0000 7f1a3b498640 1 -- start start 2024-04-14T08:38:53.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.451+0000 7f1a3b498640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a34072470 0x7f1a3415b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:53.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.451+0000 7f1a3b498640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a340b3360 con 0x7f1a3415a690 2024-04-14T08:38:53.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.451+0000 7f1a3b498640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a340b3640 con 0x7f1a34072970 2024-04-14T08:38:53.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.451+0000 7f1a3b498640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1a340b3920 con 0x7f1a34072470 2024-04-14T08:38:53.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.452+0000 7f1a39234640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1a3415a690 0x7f1a34071d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44878/0 (socket says 172.21.15.172:44878) 2024-04-14T08:38:53.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.452+0000 7f1a39234640 1 -- 172.21.15.172:0/537131482 learned_addr learned my addr 172.21.15.172:0/537131482 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:53.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.452+0000 7f1a38a33640 1 --2- 172.21.15.172:0/537131482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a34072470 0x7f1a3415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:53.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.452+0000 7f1a2bfff640 1 -- 172.21.15.172:0/537131482 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3179151158 0 0) 0x7f1a340b3360 con 0x7f1a3415a690 2024-04-14T08:38:53.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.452+0000 7f1a2bfff640 1 -- 172.21.15.172:0/537131482 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a20003520 con 0x7f1a3415a690 2024-04-14T08:38:53.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.452+0000 7f1a2bfff640 1 -- 172.21.15.172:0/537131482 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 729402540 0 0) 0x7f1a340b3640 con 0x7f1a34072970 2024-04-14T08:38:53.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.452+0000 7f1a2bfff640 1 -- 172.21.15.172:0/537131482 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a340b3360 con 0x7f1a34072970 2024-04-14T08:38:53.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.452+0000 7f1a38a33640 1 -- 172.21.15.172:0/537131482 >> v1:172.21.15.177:6789/0 conn(0x7f1a34072970 legacy=0x7f1a3415b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.452+0000 7f1a38a33640 1 -- 172.21.15.172:0/537131482 >> v1:172.21.15.172:6789/0 conn(0x7f1a3415a690 legacy=0x7f1a34071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.452+0000 7f1a38a33640 1 -- 172.21.15.172:0/537131482 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a340b3c00 con 0x7f1a34072470 2024-04-14T08:38:53.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.453+0000 7f1a38a33640 1 --2- 172.21.15.172:0/537131482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a34072470 0x7f1a3415b230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1a300097d0 tx=0x7f1a3002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7f417a534aba249 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:53.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.453+0000 7f1a2bfff640 1 -- 172.21.15.172:0/537131482 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30035020 con 0x7f1a34072470 2024-04-14T08:38:53.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.453+0000 7f1a2bfff640 1 -- 172.21.15.172:0/537131482 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a30031800 con 0x7f1a34072470 2024-04-14T08:38:53.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.453+0000 7f1a2bfff640 1 -- 172.21.15.172:0/537131482 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30031bd0 con 0x7f1a34072470 2024-04-14T08:38:53.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.453+0000 7f1a3b498640 1 -- 172.21.15.172:0/537131482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a34072470 msgr2=0x7f1a3415b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.453+0000 7f1a3b498640 1 --2- 172.21.15.172:0/537131482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a34072470 0x7f1a3415b230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1a300097d0 tx=0x7f1a3002b8a0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.453+0000 7f1a3b498640 1 -- 172.21.15.172:0/537131482 shutdown_connections 2024-04-14T08:38:53.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.453+0000 7f1a3b498640 1 --2- 172.21.15.172:0/537131482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a34072470 0x7f1a3415b230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.453+0000 7f1a3b498640 1 -- 172.21.15.172:0/537131482 >> 172.21.15.172:0/537131482 conn(0x7f1a3406ec30 msgr2=0x7f1a3406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:53.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.453+0000 7f1a3b498640 1 -- 172.21.15.172:0/537131482 shutdown_connections 2024-04-14T08:38:53.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.453+0000 7f1a3b498640 1 -- 172.21.15.172:0/537131482 wait complete. 2024-04-14T08:38:53.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.455+0000 7f1a3b498640 1 Processor -- start 2024-04-14T08:38:53.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.455+0000 7f1a3b498640 1 -- start start 2024-04-14T08:38:53.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.455+0000 7f1a3b498640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a34072470 0x7f1a34155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:53.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.455+0000 7f1a3b498640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a34072970 0x7f1a340ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:53.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.455+0000 7f1a3b498640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a3415a690 0x7f1a3414f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:53.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.455+0000 7f1a3b498640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3415aa70 con 0x7f1a34072470 2024-04-14T08:38:53.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.455+0000 7f1a3b498640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1a340b3100 con 0x7f1a34072970 2024-04-14T08:38:53.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.455+0000 7f1a3b498640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3415abf0 con 0x7f1a3415a690 2024-04-14T08:38:53.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.455+0000 7f1a39234640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a34072470 0x7f1a34155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:53.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a39234640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a34072470 0x7f1a34155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37940/0 (socket says 172.21.15.172:37940) 2024-04-14T08:38:53.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a39234640 1 -- 172.21.15.172:0/997660615 learned_addr learned my addr 172.21.15.172:0/997660615 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:53.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a39a35640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a3415a690 0x7f1a3414f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:53.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a38a33640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a34072970 0x7f1a340ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:53.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a39234640 1 -- 172.21.15.172:0/997660615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a3415a690 msgr2=0x7f1a3414f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a39234640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a3415a690 0x7f1a3414f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a39234640 1 -- 172.21.15.172:0/997660615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a34072970 msgr2=0x7f1a340ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a39234640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a34072970 0x7f1a340ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a39234640 1 -- 172.21.15.172:0/997660615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a341559d0 con 0x7f1a34072470 2024-04-14T08:38:53.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a39a35640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a3415a690 0x7f1a3414f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:53.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a39234640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a34072470 0x7f1a34155490 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400c410 tx=0x7f1a2400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:53.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a29ffb640 1 -- 172.21.15.172:0/997660615 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24017070 con 0x7f1a34072470 2024-04-14T08:38:53.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a29ffb640 1 -- 172.21.15.172:0/997660615 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a24005ce0 con 0x7f1a34072470 2024-04-14T08:38:53.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a3b498640 1 -- 172.21.15.172:0/997660615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a340cd150 con 0x7f1a34072470 2024-04-14T08:38:53.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a29ffb640 1 -- 172.21.15.172:0/997660615 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a240072f0 con 0x7f1a34072470 2024-04-14T08:38:53.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.456+0000 7f1a3b498640 1 -- 172.21.15.172:0/997660615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a340cd710 con 0x7f1a34072470 2024-04-14T08:38:53.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.457+0000 7f1a3b498640 1 -- 172.21.15.172:0/997660615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a34068520 con 0x7f1a34072470 2024-04-14T08:38:53.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.458+0000 7f1a29ffb640 1 -- 172.21.15.172:0/997660615 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a2401a070 con 0x7f1a34072470 2024-04-14T08:38:53.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.458+0000 7f1a29ffb640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1a1806c8f0 0x7f1a1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:53.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.458+0000 7f1a38a33640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1a1806c8f0 0x7f1a1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:53.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.458+0000 7f1a29ffb640 1 -- 172.21.15.172:0/997660615 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f1a2408b0b0 con 0x7f1a34072470 2024-04-14T08:38:53.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.459+0000 7f1a38a33640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1a1806c8f0 0x7f1a1806edb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1a30036000 tx=0x7f1a30008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:53.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.463+0000 7f1a29ffb640 1 -- 172.21.15.172:0/997660615 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1a341559d0 con 0x7f1a34072470 2024-04-14T08:38:53.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.675+0000 7f1a3b498640 1 -- 172.21.15.172:0/997660615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a340cc6b0 con 0x7f1a34072470 2024-04-14T08:38:53.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.676+0000 7f1a29ffb640 1 -- 172.21.15.172:0/997660615 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5195 (secure 0 0 0) 0x7f1a24058dc0 con 0x7f1a34072470 2024-04-14T08:38:53.677 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:53.677 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7473,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1386685680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1386685680},{"type":"v1","addr":"172.21.15.172:6837","nonce":1386685680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:38:46.573174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7491":{"gid":7491,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/1124731311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1124731311},{"type":"v1","addr":"172.21.15.177:6835","nonce":1124731311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7491,"qdb_cluster":[7491]},"id":6}]} 2024-04-14T08:38:53.678 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 54 2024-04-14T08:38:53.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.678+0000 7f1a3b498640 1 -- 172.21.15.172:0/997660615 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1a1806c8f0 msgr2=0x7f1a1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.678+0000 7f1a3b498640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1a1806c8f0 0x7f1a1806edb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1a30036000 tx=0x7f1a30008030 comp rx=0 tx=0).stop 2024-04-14T08:38:53.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.678+0000 7f1a3b498640 1 -- 172.21.15.172:0/997660615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a34072470 msgr2=0x7f1a34155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:53.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.678+0000 7f1a3b498640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a34072470 0x7f1a34155490 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400c410 tx=0x7f1a2400c8d0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.678+0000 7f1a3b498640 1 -- 172.21.15.172:0/997660615 shutdown_connections 2024-04-14T08:38:53.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.678+0000 7f1a3b498640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1a1806c8f0 0x7f1a1806edb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.678+0000 7f1a3b498640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a3415a690 0x7f1a3414f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.678+0000 7f1a3b498640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a34072970 0x7f1a340ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.678+0000 7f1a3b498640 1 --2- 172.21.15.172:0/997660615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a34072470 0x7f1a34155490 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:53.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.678+0000 7f1a3b498640 1 -- 172.21.15.172:0/997660615 >> 172.21.15.172:0/997660615 conn(0x7f1a3406ec30 msgr2=0x7f1a34159810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:53.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.678+0000 7f1a3b498640 1 -- 172.21.15.172:0/997660615 shutdown_connections 2024-04-14T08:38:53.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:53.678+0000 7f1a3b498640 1 -- 172.21.15.172:0/997660615 wait complete. 2024-04-14T08:38:54.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:38:44.572424+0000', 'modified': '2024-04-14T08:38:46.573174+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7491}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7491': {'gid': 7491, 'name': 'd', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6835/1124731311', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6833', 'nonce': 1124731311}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 1124731311}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7491, 'qdb_cluster': [7491]} 2024-04-14T08:38:54.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:38:54.093 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-14T08:38:54.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.195+0000 7f0e10311640 1 Processor -- start 2024-04-14T08:38:54.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.195+0000 7f0e10311640 1 -- start start 2024-04-14T08:38:54.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.196+0000 7f0e10311640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0e080af640 0x7f0e080ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.196+0000 7f0e10311640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e080d0480 con 0x7f0e080af260 2024-04-14T08:38:54.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.196+0000 7f0e10311640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e080d0720 con 0x7f0e080cd4e0 2024-04-14T08:38:54.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.196+0000 7f0e10311640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0e080d0a00 con 0x7f0e080af640 2024-04-14T08:38:54.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.196+0000 7f0e0d8ac640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0e080af640 0x7f0e080ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.196+0000 7f0e0e0ad640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f0e080af260 0x7f0e080722d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44892/0 (socket says 172.21.15.172:44892) 2024-04-14T08:38:54.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.196+0000 7f0e0e0ad640 1 -- 172.21.15.172:0/2594585508 learned_addr learned my addr 172.21.15.172:0/2594585508 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:54.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.196+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/2594585508 >> v1:172.21.15.177:6789/0 conn(0x7f0e080cd4e0 legacy=0x7f0e08160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.196+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/2594585508 >> v1:172.21.15.172:6789/0 conn(0x7f0e080af260 legacy=0x7f0e080722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.196+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/2594585508 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e080d0ce0 con 0x7f0e080af640 2024-04-14T08:38:54.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.196+0000 7f0e0d0ab640 1 -- 172.21.15.172:0/2594585508 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3431680647 0 0) 0x7f0e080d0480 con 0x7f0e080af260 2024-04-14T08:38:54.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.197+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/2594585508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e080af640 0x7f0e080ccfa0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0df80097a0 tx=0x7f0df802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff8cb11edd7486d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:54.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.197+0000 7f0e0d0ab640 1 -- 172.21.15.172:0/2594585508 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80315a0 con 0x7f0e080af640 2024-04-14T08:38:54.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.197+0000 7f0e0d0ab640 1 -- 172.21.15.172:0/2594585508 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df8037070 con 0x7f0e080af640 2024-04-14T08:38:54.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.197+0000 7f0e0d0ab640 1 -- 172.21.15.172:0/2594585508 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8031960 con 0x7f0e080af640 2024-04-14T08:38:54.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.198+0000 7f0e10311640 1 -- 172.21.15.172:0/2594585508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e080af640 msgr2=0x7f0e080ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.198+0000 7f0e10311640 1 --2- 172.21.15.172:0/2594585508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e080af640 0x7f0e080ccfa0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0df80097a0 tx=0x7f0df802b840 comp rx=0 tx=0).stop 2024-04-14T08:38:54.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.198+0000 7f0e10311640 1 -- 172.21.15.172:0/2594585508 shutdown_connections 2024-04-14T08:38:54.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.198+0000 7f0e10311640 1 --2- 172.21.15.172:0/2594585508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e080af640 0x7f0e080ccfa0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.198+0000 7f0e10311640 1 -- 172.21.15.172:0/2594585508 >> 172.21.15.172:0/2594585508 conn(0x7f0e080cda00 msgr2=0x7f0e080cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:54.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.198+0000 7f0e10311640 1 -- 172.21.15.172:0/2594585508 shutdown_connections 2024-04-14T08:38:54.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.198+0000 7f0e10311640 1 -- 172.21.15.172:0/2594585508 wait complete. 2024-04-14T08:38:54.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.199+0000 7f0e10311640 1 Processor -- start 2024-04-14T08:38:54.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.199+0000 7f0e10311640 1 -- start start 2024-04-14T08:38:54.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e10311640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e080af260 0x7f0e08156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e10311640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e080af640 0x7f0e0815f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e10311640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e080cd4e0 0x7f0e0815f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e10311640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e080cb790 con 0x7f0e080af260 2024-04-14T08:38:54.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e10311640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0e080729e0 con 0x7f0e080cd4e0 2024-04-14T08:38:54.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e10311640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0e080cb910 con 0x7f0e080af640 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e0e0ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e080af260 0x7f0e08156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e0e0ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e080af260 0x7f0e08156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37946/0 (socket says 172.21.15.172:37946) 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e0e0ad640 1 -- 172.21.15.172:0/384079047 learned_addr learned my addr 172.21.15.172:0/384079047 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e080af640 0x7f0e0815f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e0e8ae640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e080cd4e0 0x7f0e0815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e0e0ad640 1 -- 172.21.15.172:0/384079047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e080af640 msgr2=0x7f0e0815f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e080af640 0x7f0e0815f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e0e0ad640 1 -- 172.21.15.172:0/384079047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e080cd4e0 msgr2=0x7f0e0815f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e080cd4e0 0x7f0e0815f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e0e0ad640 1 -- 172.21.15.172:0/384079047 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e080b6850 con 0x7f0e080af260 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e080af260 0x7f0e08156600 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c4a0 tx=0x7f0e0400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0dfeffd640 1 -- 172.21.15.172:0/384079047 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04018070 con 0x7f0e080af260 2024-04-14T08:38:54.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0dfeffd640 1 -- 172.21.15.172:0/384079047 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e040074e0 con 0x7f0e080af260 2024-04-14T08:38:54.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e10311640 1 -- 172.21.15.172:0/384079047 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e080b3bb0 con 0x7f0e080af260 2024-04-14T08:38:54.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0dfeffd640 1 -- 172.21.15.172:0/384079047 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04004a20 con 0x7f0e080af260 2024-04-14T08:38:54.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.200+0000 7f0e10311640 1 -- 172.21.15.172:0/384079047 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0815b0d0 con 0x7f0e080af260 2024-04-14T08:38:54.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.202+0000 7f0dfeffd640 1 -- 172.21.15.172:0/384079047 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e0401b070 con 0x7f0e080af260 2024-04-14T08:38:54.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.202+0000 7f0dfeffd640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0dec06c920 0x7f0dec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.202+0000 7f0dfeffd640 1 -- 172.21.15.172:0/384079047 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f0e0408af60 con 0x7f0e080af260 2024-04-14T08:38:54.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.202+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0dec06c920 0x7f0dec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.203+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0dec06c920 0x7f0dec06ede0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0df8036000 tx=0x7f0df803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:54.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.203+0000 7f0e131cc740 1 Processor -- start 2024-04-14T08:38:54.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.203+0000 7f0e131cc740 1 -- start start 2024-04-14T08:38:54.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e131cc740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f9431ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e131cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f9434a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e131cc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f9431b040 0x563f9434ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e131cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563f94284030 con 0x563f94347f40 2024-04-14T08:38:54.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e131cc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563f9422c620 con 0x563f9431b040 2024-04-14T08:38:54.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e131cc740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563f9427ffe0 con 0x563f94347b60 2024-04-14T08:38:54.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e0d8ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f9434a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e0d8ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f9434a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37948/0 (socket says 172.21.15.172:37948) 2024-04-14T08:38:54.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/2337217758 learned_addr learned my addr 172.21.15.172:0/2337217758 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:54.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/2337217758 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f9431ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e0e8ae640 1 --2- 172.21.15.172:0/2337217758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f9431b040 0x563f9434ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/2337217758 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 msgr2=0x563f9431ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:54.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/2337217758 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f9431ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/2337217758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f9431b040 msgr2=0x563f9434ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.204+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/2337217758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f9431b040 0x563f9434ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.205+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/2337217758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f93f982d0 con 0x563f94347f40 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.205+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/2337217758 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f9431ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.205+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/2337217758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f9434a8f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009560 tx=0x7f0df8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9b490831454396 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.205+0000 7f0debfff640 1 -- 172.21.15.172:0/2337217758 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8031d40 con 0x563f94347f40 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.205+0000 7f0debfff640 1 -- 172.21.15.172:0/2337217758 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df8037040 con 0x563f94347f40 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.205+0000 7f0debfff640 1 -- 172.21.15.172:0/2337217758 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80506c0 con 0x563f94347f40 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.205+0000 7f0e131cc740 1 -- 172.21.15.172:0/2337217758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 msgr2=0x563f9434a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.205+0000 7f0e131cc740 1 --2- 172.21.15.172:0/2337217758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f9434a8f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009560 tx=0x7f0df8030000 comp rx=0 tx=0).stop 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.206+0000 7f0e131cc740 1 -- 172.21.15.172:0/2337217758 shutdown_connections 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.206+0000 7f0e131cc740 1 --2- 172.21.15.172:0/2337217758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f9431b040 0x563f9434ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.206+0000 7f0e131cc740 1 --2- 172.21.15.172:0/2337217758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f9434a8f0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.206+0000 7f0e131cc740 1 --2- 172.21.15.172:0/2337217758 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f9431ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.206+0000 7f0e131cc740 1 -- 172.21.15.172:0/2337217758 >> 172.21.15.172:0/2337217758 conn(0x563f9431cd70 msgr2=0x563f943658f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.206+0000 7f0e131cc740 1 -- 172.21.15.172:0/2337217758 shutdown_connections 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.206+0000 7f0e131cc740 1 -- 172.21.15.172:0/2337217758 wait complete. 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.206+0000 7f0e131cc740 1 -- start start 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.206+0000 7f0e131cc740 1 Processor -- start 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.207+0000 7f0e131cc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94347b60 0x563f9434e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.207+0000 7f0e131cc740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347f40 0x563f9433a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.207+0000 7f0e131cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94351430 0x563f943518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.207+0000 7f0e131cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563f9431f410 con 0x563f94351430 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.207+0000 7f0e131cc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563f9423f9e0 con 0x563f94347b60 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.207+0000 7f0e131cc740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563f94321c50 con 0x563f94347f40 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.207+0000 7f0e131cc740 10 client.0 _ll_register_callbacks cb 0x563f9427c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.207+0000 7f0e0e8ae640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94351430 0x563f943518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.207+0000 7f0e0e8ae640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94351430 0x563f943518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37958/0 (socket says 172.21.15.172:37958) 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.207+0000 7f0e0e8ae640 1 -- 172.21.15.172:0/925464965 learned_addr learned my addr 172.21.15.172:0/925464965 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.207+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94347b60 0x563f9434e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.207+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347f40 0x563f9433a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e0e8ae640 1 -- 172.21.15.172:0/925464965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347f40 msgr2=0x563f9433a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e0e8ae640 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347f40 0x563f9433a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e0e8ae640 1 -- 172.21.15.172:0/925464965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94347b60 msgr2=0x563f9434e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e0e8ae640 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94347b60 0x563f9434e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e0e8ae640 1 -- 172.21.15.172:0/925464965 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f940626b0 con 0x563f94351430 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e0e8ae640 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94351430 0x563f943518b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0e00012570 tx=0x7f0e00012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31689761cdb39e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0de97fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0de97fa640 1 -- 172.21.15.172:0/925464965 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00006d60 con 0x563f94351430 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0de97fa640 1 -- 172.21.15.172:0/925464965 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f0e00003970 con 0x563f94351430 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e131cc740 1 -- 172.21.15.172:0/925464965 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df8037040 con 0x563f94351430 2024-04-14T08:38:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0de97fa640 1 -- 172.21.15.172:0/925464965 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0e00003b70 con 0x563f94351430 2024-04-14T08:38:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e131cc740 10 client.7647 fetch_fsmap learned FSMap version 54 2024-04-14T08:38:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e131cc740 1 -- 172.21.15.172:0/925464965 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x563f9434cf50 con 0x563f94351430 2024-04-14T08:38:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0de97fa640 1 -- 172.21.15.172:0/925464965 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df8037040 con 0x563f94351430 2024-04-14T08:38:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0de97fa640 1 -- 172.21.15.172:0/925464965 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 3494+0+0 (secure 0 0 0) 0x7f0e00005110 con 0x563f94351430 2024-04-14T08:38:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e131cc740 10 client.7647 fetch_fsmap finished waiting for FSMap version 54 2024-04-14T08:38:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e131cc740 10 client.7647 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:38:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e131cc740 20 client.7647 populate_metadata read hostname 'smithi172' 2024-04-14T08:38:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e131cc740 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.177:6833/1124731311,v1:172.21.15.177:6835/1124731311] conn(0x563f94379720 0x563f9434fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e131cc740 4 client.7647 mds_command: new command op to 7491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:38:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.208+0000 7f0e131cc740 1 -- 172.21.15.172:0/925464965 --> [v2:172.21.15.177:6833/1124731311,v1:172.21.15.177:6835/1124731311] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f9422bcc0 con 0x563f94379720 2024-04-14T08:38:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.209+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.177:6833/1124731311,v1:172.21.15.177:6835/1124731311] conn(0x563f94379720 0x563f9434fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.209+0000 7f0de97fa640 10 client.7647 ms_handle_connect on v2:172.21.15.177:6833/1124731311 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.209+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.177:6833/1124731311,v1:172.21.15.177:6835/1124731311] conn(0x563f94379720 0x563f9434fee0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d40711aa6818c4c3 server_cookie=20ee77f98c53ffb3 in_seq=0 out_seq=0 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0de97fa640 1 -- 172.21.15.172:0/925464965 <== mds.0 v2:172.21.15.177:6833/1124731311 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f9422bcc0 con 0x563f94379720 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0de97fa640 10 client.7647 handle_command_reply: tid=0 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0e131cc740 1 client.7647 shutdown 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0e131cc740 1 -- 172.21.15.172:0/925464965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94351430 msgr2=0x563f943518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0e131cc740 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94351430 0x563f943518b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0e00012570 tx=0x7f0e00012a30 comp rx=0 tx=0).stop 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0e131cc740 1 -- 172.21.15.172:0/925464965 shutdown_connections 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0e131cc740 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.177:6833/1124731311,v1:172.21.15.177:6835/1124731311] conn(0x563f94379720 0x563f9434fee0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0e131cc740 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94351430 0x563f943518b0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0e131cc740 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347f40 0x563f9433a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0de97fa640 0 client.7647 ms_handle_reset on v2:172.21.15.177:6833/1124731311 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0e131cc740 1 --2- 172.21.15.172:0/925464965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94347b60 0x563f9434e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0e131cc740 1 -- 172.21.15.172:0/925464965 >> 172.21.15.172:0/925464965 conn(0x563f9431b040 msgr2=0x563f9431ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:54.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0e131cc740 1 -- 172.21.15.172:0/925464965 shutdown_connections 2024-04-14T08:38:54.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0e131cc740 1 -- 172.21.15.172:0/925464965 wait complete. 2024-04-14T08:38:54.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.210+0000 7f0e131cc740 20 client.7647 trim_cache size 0 max 16384 2024-04-14T08:38:54.227 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:38:54.229 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:38:54.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.228+0000 7f0e131cc740 1 Processor -- start 2024-04-14T08:38:54.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.228+0000 7f0e131cc740 1 -- start start 2024-04-14T08:38:54.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e131cc740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f94346290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e131cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f943467d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e131cc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94351430 0x563f94388400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e131cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081fff60 con 0x563f94347f40 2024-04-14T08:38:54.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e131cc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0815a490 con 0x563f94351430 2024-04-14T08:38:54.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e131cc740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0e081ff9d0 con 0x563f94347b60 2024-04-14T08:38:54.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e0d8ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f943467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e0d8ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f943467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37964/0 (socket says 172.21.15.172:37964) 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/3848772131 learned_addr learned my addr 172.21.15.172:0/3848772131 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/3848772131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f94346290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e0e8ae640 1 --2- 172.21.15.172:0/3848772131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94351430 0x563f94388400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/3848772131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 msgr2=0x563f94346290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/3848772131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f94346290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/3848772131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94351430 msgr2=0x563f94388400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/3848772131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94351430 0x563f94388400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/3848772131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f94321640 con 0x563f94347f40 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/3848772131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f94346290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/3848772131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f943467d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0df802be50 tx=0x7f0df8031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b185bb6abfb4079 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0deb7fe640 1 -- 172.21.15.172:0/3848772131 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df805f070 con 0x563f94347f40 2024-04-14T08:38:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0deb7fe640 1 -- 172.21.15.172:0/3848772131 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df8037040 con 0x563f94347f40 2024-04-14T08:38:54.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.229+0000 7f0deb7fe640 1 -- 172.21.15.172:0/3848772131 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8050480 con 0x563f94347f40 2024-04-14T08:38:54.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.230+0000 7f0e131cc740 1 -- 172.21.15.172:0/3848772131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 msgr2=0x563f943467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.230+0000 7f0e131cc740 1 --2- 172.21.15.172:0/3848772131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f943467d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0df802be50 tx=0x7f0df8031000 comp rx=0 tx=0).stop 2024-04-14T08:38:54.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.230+0000 7f0e131cc740 1 -- 172.21.15.172:0/3848772131 shutdown_connections 2024-04-14T08:38:54.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.230+0000 7f0e131cc740 1 --2- 172.21.15.172:0/3848772131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94351430 0x563f94388400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.230+0000 7f0e131cc740 1 --2- 172.21.15.172:0/3848772131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f943467d0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.230+0000 7f0e131cc740 1 --2- 172.21.15.172:0/3848772131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f94346290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.230+0000 7f0e131cc740 1 -- 172.21.15.172:0/3848772131 >> 172.21.15.172:0/3848772131 conn(0x563f9431b040 msgr2=0x563f94315e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:54.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.230+0000 7f0e131cc740 1 -- 172.21.15.172:0/3848772131 shutdown_connections 2024-04-14T08:38:54.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.230+0000 7f0e131cc740 1 -- 172.21.15.172:0/3848772131 wait complete. 2024-04-14T08:38:54.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.231+0000 7f0e131cc740 1 -- start start 2024-04-14T08:38:54.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.231+0000 7f0e131cc740 1 Processor -- start 2024-04-14T08:38:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e131cc740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f943dfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e131cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f9437c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e131cc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94351430 0x563f9437f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e131cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563f943795a0 con 0x563f94347f40 2024-04-14T08:38:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e131cc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563f94350660 con 0x563f94351430 2024-04-14T08:38:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e131cc740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x563f94379bf0 con 0x563f94347b60 2024-04-14T08:38:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e131cc740 10 client.0 _ll_register_callbacks cb 0x7f0e00001760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:38:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e0d8ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f9437c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e0d8ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f9437c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37974/0 (socket says 172.21.15.172:37974) 2024-04-14T08:38:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/117418080 learned_addr learned my addr 172.21.15.172:0/117418080 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f943dfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e0e8ae640 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94351430 0x563f9437f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/117418080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 msgr2=0x563f943dfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f943dfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/117418080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94351430 msgr2=0x563f9437f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94351430 0x563f9437f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e0d8ac640 1 -- 172.21.15.172:0/117418080 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f9434e210 con 0x563f94347f40 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e0d8ac640 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f9437c7c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0df802b880 tx=0x7f0df8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4d44381642dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0de97fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0de97fa640 1 -- 172.21.15.172:0/117418080 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8069060 con 0x563f94347f40 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0de97fa640 1 -- 172.21.15.172:0/117418080 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f0df8050a30 con 0x563f94347f40 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0e131cc740 1 -- 172.21.15.172:0/117418080 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df8037040 con 0x563f94347f40 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.232+0000 7f0de97fa640 1 -- 172.21.15.172:0/117418080 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0df805f070 con 0x563f94347f40 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.233+0000 7f0e131cc740 10 client.7653 fetch_fsmap learned FSMap version 54 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.233+0000 7f0e131cc740 1 -- 172.21.15.172:0/117418080 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x563f9437c4c0 con 0x563f94347f40 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.233+0000 7f0de97fa640 1 -- 172.21.15.172:0/117418080 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df8037040 con 0x563f94347f40 2024-04-14T08:38:54.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.233+0000 7f0de97fa640 1 -- 172.21.15.172:0/117418080 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 3494+0+0 (secure 0 0 0) 0x7f0df8031070 con 0x563f94347f40 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.233+0000 7f0e131cc740 10 client.7653 fetch_fsmap finished waiting for FSMap version 54 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.233+0000 7f0e131cc740 10 client.7653 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.233+0000 7f0e131cc740 20 client.7653 populate_metadata read hostname 'smithi172' 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.233+0000 7f0e131cc740 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.177:6833/1124731311,v1:172.21.15.177:6835/1124731311] conn(0x563f943e37b0 0x563f94367bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.233+0000 7f0e131cc740 4 client.7653 mds_command: new command op to 7491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.233+0000 7f0e131cc740 1 -- 172.21.15.172:0/117418080 --> [v2:172.21.15.177:6833/1124731311,v1:172.21.15.177:6835/1124731311] -- command(tid 0: {"prefix": "status"}) -- 0x563f94291570 con 0x563f943e37b0 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.233+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.177:6833/1124731311,v1:172.21.15.177:6835/1124731311] conn(0x563f943e37b0 0x563f94367bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.233+0000 7f0de97fa640 10 client.7653 ms_handle_connect on v2:172.21.15.177:6833/1124731311 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e0e0ad640 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.177:6833/1124731311,v1:172.21.15.177:6835/1124731311] conn(0x563f943e37b0 0x563f94367bd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4497d6f286e3868e server_cookie=8334a0c1db7d588 in_seq=0 out_seq=0 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0de97fa640 1 -- 172.21.15.172:0/117418080 <== mds.0 v2:172.21.15.177:6833/1124731311 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x563f94291570 con 0x563f943e37b0 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0de97fa640 10 client.7653 handle_command_reply: tid=0 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e131cc740 1 client.7653 shutdown 2024-04-14T08:38:54.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e131cc740 1 -- 172.21.15.172:0/117418080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 msgr2=0x563f9437c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e131cc740 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f9437c7c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0df802b880 tx=0x7f0df8004730 comp rx=0 tx=0).stop 2024-04-14T08:38:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e131cc740 1 -- 172.21.15.172:0/117418080 shutdown_connections 2024-04-14T08:38:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e131cc740 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.177:6833/1124731311,v1:172.21.15.177:6835/1124731311] conn(0x563f943e37b0 0x563f94367bd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e131cc740 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563f94351430 0x563f9437f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e131cc740 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f94347f40 0x563f9437c7c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0de97fa640 0 client.7653 ms_handle_reset on v2:172.21.15.177:6833/1124731311 2024-04-14T08:38:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e131cc740 1 --2- 172.21.15.172:0/117418080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563f94347b60 0x563f943dfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e131cc740 1 -- 172.21.15.172:0/117418080 >> 172.21.15.172:0/117418080 conn(0x563f9431b040 msgr2=0x563f943657e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e131cc740 1 -- 172.21.15.172:0/117418080 shutdown_connections 2024-04-14T08:38:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e131cc740 1 -- 172.21.15.172:0/117418080 wait complete. 2024-04-14T08:38:54.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.234+0000 7f0e131cc740 20 client.7653 trim_cache size 0 max 16384 2024-04-14T08:38:54.244 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:38:54.244 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:38:54.244 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:38:54.244 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 7491, 2024-04-14T08:38:54.245 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:38:54.245 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:38:54.245 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:38:54.245 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 9.0133243929999995, 2024-04-14T08:38:54.245 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 54, 2024-04-14T08:38:54.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 86, 2024-04-14T08:38:54.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 86, 2024-04-14T08:38:54.245 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 14.04503933 2024-04-14T08:38:54.245 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:38:54.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.235+0000 7f0e10311640 1 -- 172.21.15.172:0/384079047 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0dec06c920 msgr2=0x7f0dec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.235+0000 7f0e10311640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0dec06c920 0x7f0dec06ede0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0df8036000 tx=0x7f0df803d040 comp rx=0 tx=0).stop 2024-04-14T08:38:54.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.235+0000 7f0e10311640 1 -- 172.21.15.172:0/384079047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e080af260 msgr2=0x7f0e08156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.235+0000 7f0e10311640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e080af260 0x7f0e08156600 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c4a0 tx=0x7f0e0400c960 comp rx=0 tx=0).stop 2024-04-14T08:38:54.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.235+0000 7f0e10311640 1 -- 172.21.15.172:0/384079047 shutdown_connections 2024-04-14T08:38:54.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.235+0000 7f0e10311640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0dec06c920 0x7f0dec06ede0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.235+0000 7f0e10311640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e080cd4e0 0x7f0e0815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.235+0000 7f0e10311640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e080af640 0x7f0e0815f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.235+0000 7f0e10311640 1 --2- 172.21.15.172:0/384079047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e080af260 0x7f0e08156600 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.235+0000 7f0e10311640 1 -- 172.21.15.172:0/384079047 >> 172.21.15.172:0/384079047 conn(0x7f0e080cda00 msgr2=0x7f0e080722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:54.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.235+0000 7f0e10311640 1 -- 172.21.15.172:0/384079047 shutdown_connections 2024-04-14T08:38:54.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.235+0000 7f0e10311640 1 -- 172.21.15.172:0/384079047 wait complete. 2024-04-14T08:38:54.249 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:38:54.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.389+0000 7f22c437b640 1 Processor -- start 2024-04-14T08:38:54.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.389+0000 7f22c437b640 1 -- start start 2024-04-14T08:38:54.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c437b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f22bc0b21d0 0x7f22bc15c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c437b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f22bc15c5a0 0x7f22bc0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c437b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22bc0b3700 con 0x7f22bc0b1df0 2024-04-14T08:38:54.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c437b640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f22bc0b39e0 con 0x7f22bc0b21d0 2024-04-14T08:38:54.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c437b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f22bc0b3cc0 con 0x7f22bc15c5a0 2024-04-14T08:38:54.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c1916640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f22bc0b21d0 0x7f22bc15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c1916640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f22bc0b21d0 0x7f22bc15c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37982/0 (socket says 172.21.15.172:37982) 2024-04-14T08:38:54.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c1916640 1 -- 172.21.15.172:0/4246300345 learned_addr learned my addr 172.21.15.172:0/4246300345 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:54.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c2918640 1 --2- 172.21.15.172:0/4246300345 >> v2:172.21.15.177:3300/0 conn(0x7f22bc15c5a0 0x7f22bc0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c1916640 1 -- 172.21.15.172:0/4246300345 >> v2:172.21.15.177:3300/0 conn(0x7f22bc15c5a0 msgr2=0x7f22bc0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:54.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c1916640 1 --2- 172.21.15.172:0/4246300345 >> v2:172.21.15.177:3300/0 conn(0x7f22bc15c5a0 0x7f22bc0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c1916640 1 -- 172.21.15.172:0/4246300345 >> v1:172.21.15.177:6789/0 conn(0x7f22bc0b1df0 legacy=0x7f22bc0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.390+0000 7f22c1916640 1 -- 172.21.15.172:0/4246300345 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc15bb40 con 0x7f22bc0b21d0 2024-04-14T08:38:54.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.391+0000 7f22c1916640 1 --2- 172.21.15.172:0/4246300345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc0b21d0 0x7f22bc15c060 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0b3fa0 tx=0x7f22ac02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3d80d6ceb915f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:54.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.391+0000 7f22c1115640 1 -- 172.21.15.172:0/4246300345 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac034070 con 0x7f22bc0b21d0 2024-04-14T08:38:54.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.391+0000 7f22c1115640 1 -- 172.21.15.172:0/4246300345 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ac030860 con 0x7f22bc0b21d0 2024-04-14T08:38:54.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.391+0000 7f22c1115640 1 -- 172.21.15.172:0/4246300345 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac030c30 con 0x7f22bc0b21d0 2024-04-14T08:38:54.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.391+0000 7f22c437b640 1 -- 172.21.15.172:0/4246300345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc0b21d0 msgr2=0x7f22bc15c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.391+0000 7f22c437b640 1 --2- 172.21.15.172:0/4246300345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc0b21d0 0x7f22bc15c060 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0b3fa0 tx=0x7f22ac02b9a0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.392+0000 7f22c437b640 1 -- 172.21.15.172:0/4246300345 shutdown_connections 2024-04-14T08:38:54.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.392+0000 7f22c437b640 1 --2- 172.21.15.172:0/4246300345 >> v2:172.21.15.177:3300/0 conn(0x7f22bc15c5a0 0x7f22bc0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.392+0000 7f22c437b640 1 --2- 172.21.15.172:0/4246300345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc0b21d0 0x7f22bc15c060 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.392+0000 7f22c437b640 1 -- 172.21.15.172:0/4246300345 >> 172.21.15.172:0/4246300345 conn(0x7f22bc06ec30 msgr2=0x7f22bc06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:54.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.392+0000 7f22c437b640 1 -- 172.21.15.172:0/4246300345 shutdown_connections 2024-04-14T08:38:54.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.392+0000 7f22c437b640 1 -- 172.21.15.172:0/4246300345 wait complete. 2024-04-14T08:38:54.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.393+0000 7f22c437b640 1 Processor -- start 2024-04-14T08:38:54.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.393+0000 7f22c437b640 1 -- start start 2024-04-14T08:38:54.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.393+0000 7f22c437b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22bc0b1df0 0x7f22bc157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.393+0000 7f22c437b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc0b21d0 0x7f22bc1f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.393+0000 7f22c437b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22bc1f7f10 0x7f22bc0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.393+0000 7f22c437b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc0b4700 con 0x7f22bc0b21d0 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.393+0000 7f22c437b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc0b4580 con 0x7f22bc0b1df0 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.393+0000 7f22c437b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f22bc0b4880 con 0x7f22bc1f7f10 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c1916640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc0b21d0 0x7f22bc1f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c1916640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc0b21d0 0x7f22bc1f79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37990/0 (socket says 172.21.15.172:37990) 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c1916640 1 -- 172.21.15.172:0/2812579571 learned_addr learned my addr 172.21.15.172:0/2812579571 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c2117640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22bc0b1df0 0x7f22bc157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c2918640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22bc1f7f10 0x7f22bc0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c1916640 1 -- 172.21.15.172:0/2812579571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22bc1f7f10 msgr2=0x7f22bc0cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c1916640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22bc1f7f10 0x7f22bc0cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c1916640 1 -- 172.21.15.172:0/2812579571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22bc0b1df0 msgr2=0x7f22bc157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c1916640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22bc0b1df0 0x7f22bc157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c1916640 1 -- 172.21.15.172:0/2812579571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc1f8420 con 0x7f22bc0b21d0 2024-04-14T08:38:54.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c1916640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc0b21d0 0x7f22bc1f79d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f22ac000c00 tx=0x7f22ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22b2ffd640 1 -- 172.21.15.172:0/2812579571 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac034040 con 0x7f22bc0b21d0 2024-04-14T08:38:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22b2ffd640 1 -- 172.21.15.172:0/2812579571 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ac039d90 con 0x7f22bc0b21d0 2024-04-14T08:38:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22b2ffd640 1 -- 172.21.15.172:0/2812579571 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac043610 con 0x7f22bc0b21d0 2024-04-14T08:38:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c437b640 1 -- 172.21.15.172:0/2812579571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22bc1f8700 con 0x7f22bc0b21d0 2024-04-14T08:38:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.394+0000 7f22c437b640 1 -- 172.21.15.172:0/2812579571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22bc1e4cb0 con 0x7f22bc0b21d0 2024-04-14T08:38:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.395+0000 7f22c437b640 1 -- 172.21.15.172:0/2812579571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22bc1f8900 con 0x7f22bc0b21d0 2024-04-14T08:38:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.396+0000 7f22b2ffd640 1 -- 172.21.15.172:0/2812579571 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22ac037070 con 0x7f22bc0b21d0 2024-04-14T08:38:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.396+0000 7f22b2ffd640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f22a006c8f0 0x7f22a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.396+0000 7f22b2ffd640 1 -- 172.21.15.172:0/2812579571 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f22ac0adca0 con 0x7f22bc0b21d0 2024-04-14T08:38:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.396+0000 7f22c2117640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f22a006c8f0 0x7f22a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.396+0000 7f22c2117640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f22a006c8f0 0x7f22a006edb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f22b80016e0 tx=0x7f22b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:54.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.400+0000 7f22b2ffd640 1 -- 172.21.15.172:0/2812579571 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f22ac03e070 con 0x7f22bc0b21d0 2024-04-14T08:38:54.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.611+0000 7f22c437b640 1 -- 172.21.15.172:0/2812579571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f22bc068520 con 0x7f22bc0b21d0 2024-04-14T08:38:54.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.612+0000 7f22b2ffd640 1 -- 172.21.15.172:0/2812579571 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) ==== 53+0+73 (secure 0 0 0) 0x7f22ac07b8f0 con 0x7f22bc0b21d0 2024-04-14T08:38:54.613 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:38:54.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.613+0000 7f22c437b640 1 -- 172.21.15.172:0/2812579571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f22a006c8f0 msgr2=0x7f22a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.613+0000 7f22c437b640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f22a006c8f0 0x7f22a006edb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f22b80016e0 tx=0x7f22b8009040 comp rx=0 tx=0).stop 2024-04-14T08:38:54.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.613+0000 7f22c437b640 1 -- 172.21.15.172:0/2812579571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc0b21d0 msgr2=0x7f22bc1f79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:54.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.613+0000 7f22c437b640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc0b21d0 0x7f22bc1f79d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f22ac000c00 tx=0x7f22ac0046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.614+0000 7f22c437b640 1 -- 172.21.15.172:0/2812579571 shutdown_connections 2024-04-14T08:38:54.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.614+0000 7f22c437b640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f22a006c8f0 0x7f22a006edb0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.614+0000 7f22c437b640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22bc1f7f10 0x7f22bc0cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.614+0000 7f22c437b640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22bc0b21d0 0x7f22bc1f79d0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.614+0000 7f22c437b640 1 --2- 172.21.15.172:0/2812579571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22bc0b1df0 0x7f22bc157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:54.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.614+0000 7f22c437b640 1 -- 172.21.15.172:0/2812579571 >> 172.21.15.172:0/2812579571 conn(0x7f22bc06ec30 msgr2=0x7f22bc159330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:54.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.614+0000 7f22c437b640 1 -- 172.21.15.172:0/2812579571 shutdown_connections 2024-04-14T08:38:54.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54.614+0000 7f22c437b640 1 -- 172.21.15.172:0/2812579571 wait complete. 2024-04-14T08:38:54.626 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:38:54.626 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:38:54.627 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:38:54.627 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:38:54.627 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:38:54.627 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:38:54.627 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:38:54.627 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:38:54.627 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:38:54.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:54.628 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:38:54.644 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:38:54.644 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:38:54.644 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:38:54.644 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:54.644 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:38:54.644 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:54.644 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:38:54.644 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:38:54.644 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42335sec preferred_lft 42335sec 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:38:54.645 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:38:54.646 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:38:54.646 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:54.646 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:38:54.646 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:54.646 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:38:54.646 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:38:54.646 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:38:54.646 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:54.646 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:38:54.646 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:38:54.646 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:38:54.646 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:38:54.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:54.647 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:38:54.710 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:38:54.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:54.711 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:38:54.767 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:38:54.767 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:38:54.767 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:38:54.767 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:38:54.767 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:38:54.768 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:38:54.768 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:38:54.768 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:38:54.768 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:38:54.768 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:38:54.768 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:38:54.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:38:54.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:38:54.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:54.902 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:38:54.917 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:38:54.918 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:38:54.918 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:38:54.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:54.918 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:38:54.985 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:38:54.985 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:38:55.040 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:38:55.040 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:38:55.041 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:38:55.095 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:38:55.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:55.096 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:38:55.180 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:38:55.180 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:38:55.180 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:38:55.189 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:38:55.189 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:38:55.189 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:38:55.189 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:38:55.189 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:38:55.194 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:38:55.194 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:38:55.194 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:38:55.195 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:38:55.195 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:38:55.195 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:38:55.195 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:38:55.195 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:38:55.195 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:38:55.195 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:38:55.195 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:38:55.195 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:38:55.196 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:38:55.197 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:38:55.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:55.197 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:38:55.231 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:38:55.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:38:55.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:38:55.332 INFO:teuthology.orchestra.run.smithi172.stdout:client6419 2024-04-14T08:38:55.332 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:38:55.333 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:55.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:55.348 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-14T08:38:55.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:55.408 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-14T08:38:55.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:55.464 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-14T08:38:55.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:55.520 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-14T08:38:55.582 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:38:55.582 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:38:55.582 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:38:55.582 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:38:55.582 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-14T08:38:55.582 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:38:55.688 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627776 2024-04-14T08:38:55.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:38:55.697 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:38:55.697 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:55.753 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:38:55.753 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:38:55.809 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:38:55.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.948+0000 7f1fd6cb7640 1 Processor -- start 2024-04-14T08:38:55.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.948+0000 7f1fd6cb7640 1 -- start start 2024-04-14T08:38:55.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.949+0000 7f1fd6cb7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1fd00b1df0 0x7f1fd00b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:55.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.949+0000 7f1fd6cb7640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fd015c770 con 0x7f1fd00b4040 2024-04-14T08:38:55.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.949+0000 7f1fd6cb7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fd00b3430 con 0x7f1fd00b21d0 2024-04-14T08:38:55.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.949+0000 7f1fd6cb7640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f1fd00b3700 con 0x7f1fd00b1df0 2024-04-14T08:38:55.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.949+0000 7f1fd5254640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1fd00b4040 0x7f1fd00b4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44900/0 (socket says 172.21.15.172:44900) 2024-04-14T08:38:55.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.949+0000 7f1fd5254640 1 -- 172.21.15.172:0/2460343716 learned_addr learned my addr 172.21.15.172:0/2460343716 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:55.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.949+0000 7f1fd4a53640 1 --2- 172.21.15.172:0/2460343716 >> v2:172.21.15.177:3300/0 conn(0x7f1fd00b1df0 0x7f1fd00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:55.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.949+0000 7f1fcf7fe640 1 -- 172.21.15.172:0/2460343716 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2141790406 0 0) 0x7f1fd015c770 con 0x7f1fd00b4040 2024-04-14T08:38:55.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.949+0000 7f1fcf7fe640 1 -- 172.21.15.172:0/2460343716 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fb8003520 con 0x7f1fd00b4040 2024-04-14T08:38:55.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.949+0000 7f1fcf7fe640 1 -- 172.21.15.172:0/2460343716 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1149618014 0 0) 0x7f1fd00b3430 con 0x7f1fd00b21d0 2024-04-14T08:38:55.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.949+0000 7f1fcf7fe640 1 -- 172.21.15.172:0/2460343716 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fd015c770 con 0x7f1fd00b21d0 2024-04-14T08:38:55.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.950+0000 7f1fd4a53640 1 -- 172.21.15.172:0/2460343716 >> v1:172.21.15.177:6789/0 conn(0x7f1fd00b21d0 legacy=0x7f1fd015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:55.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.950+0000 7f1fd4a53640 1 -- 172.21.15.172:0/2460343716 >> v1:172.21.15.172:6789/0 conn(0x7f1fd00b4040 legacy=0x7f1fd00b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:55.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.950+0000 7f1fd4a53640 1 -- 172.21.15.172:0/2460343716 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd00b39e0 con 0x7f1fd00b1df0 2024-04-14T08:38:55.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.950+0000 7f1fd4a53640 1 --2- 172.21.15.172:0/2460343716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd00b1df0 0x7f1fd00b2ef0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00097a0 tx=0x7f1fc002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b932212eaf7b6fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:55.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.950+0000 7f1fcf7fe640 1 -- 172.21.15.172:0/2460343716 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0034070 con 0x7f1fd00b1df0 2024-04-14T08:38:55.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.951+0000 7f1fcf7fe640 1 -- 172.21.15.172:0/2460343716 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc0030610 con 0x7f1fd00b1df0 2024-04-14T08:38:55.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.951+0000 7f1fcf7fe640 1 -- 172.21.15.172:0/2460343716 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00309c0 con 0x7f1fd00b1df0 2024-04-14T08:38:55.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.951+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2460343716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd00b1df0 msgr2=0x7f1fd00b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:55.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.951+0000 7f1fd6cb7640 1 --2- 172.21.15.172:0/2460343716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd00b1df0 0x7f1fd00b2ef0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00097a0 tx=0x7f1fc002b960 comp rx=0 tx=0).stop 2024-04-14T08:38:55.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.951+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2460343716 shutdown_connections 2024-04-14T08:38:55.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.951+0000 7f1fd6cb7640 1 --2- 172.21.15.172:0/2460343716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd00b1df0 0x7f1fd00b2ef0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.951+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2460343716 >> 172.21.15.172:0/2460343716 conn(0x7f1fd006ec30 msgr2=0x7f1fd006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.951+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2460343716 shutdown_connections 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.951+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2460343716 wait complete. 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.952+0000 7f1fd6cb7640 1 Processor -- start 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.952+0000 7f1fd6cb7640 1 -- start start 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd6cb7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fd00b1df0 0x7f1fd00aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd6cb7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd00b21d0 0x7f1fd0131850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd6cb7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fd00b4040 0x7f1fd0131d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd6cb7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd015b7f0 con 0x7f1fd00b1df0 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd6cb7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd015bb40 con 0x7f1fd00b21d0 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd6cb7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd01552d0 con 0x7f1fd00b4040 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd4a53640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fd00b1df0 0x7f1fd00aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd4a53640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fd00b1df0 0x7f1fd00aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37998/0 (socket says 172.21.15.172:37998) 2024-04-14T08:38:55.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd4a53640 1 -- 172.21.15.172:0/2038967172 learned_addr learned my addr 172.21.15.172:0/2038967172 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fcffff640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd00b21d0 0x7f1fd0131850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd5254640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fd00b4040 0x7f1fd0131d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd4a53640 1 -- 172.21.15.172:0/2038967172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fd00b4040 msgr2=0x7f1fd0131d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd4a53640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fd00b4040 0x7f1fd0131d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd4a53640 1 -- 172.21.15.172:0/2038967172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd00b21d0 msgr2=0x7f1fd0131850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.953+0000 7f1fd4a53640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd00b21d0 0x7f1fd0131850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.954+0000 7f1fd4a53640 1 -- 172.21.15.172:0/2038967172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd013bb50 con 0x7f1fd00b1df0 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.954+0000 7f1fd5254640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fd00b4040 0x7f1fd0131d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.954+0000 7f1fd4a53640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fd00b1df0 0x7f1fd00aecd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0000c00 tx=0x7f1fc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.954+0000 7f1fcd7fa640 1 -- 172.21.15.172:0/2038967172 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0048070 con 0x7f1fd00b1df0 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.954+0000 7f1fcd7fa640 1 -- 172.21.15.172:0/2038967172 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc00044c0 con 0x7f1fd00b1df0 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.954+0000 7f1fcd7fa640 1 -- 172.21.15.172:0/2038967172 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0034050 con 0x7f1fd00b1df0 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.954+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2038967172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd00af210 con 0x7f1fd00b1df0 2024-04-14T08:38:55.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.954+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2038967172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd00adf10 con 0x7f1fd00b1df0 2024-04-14T08:38:55.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.955+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2038967172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd0068520 con 0x7f1fd00b1df0 2024-04-14T08:38:55.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.955+0000 7f1fcd7fa640 1 -- 172.21.15.172:0/2038967172 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fc004b070 con 0x7f1fd00b1df0 2024-04-14T08:38:55.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.956+0000 7f1fcd7fa640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:55.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.956+0000 7f1fcd7fa640 1 -- 172.21.15.172:0/2038967172 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f1fc00ad790 con 0x7f1fd00b1df0 2024-04-14T08:38:55.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.956+0000 7f1fcffff640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:55.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.956+0000 7f1fcffff640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4007fd0 tx=0x7f1fc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:55.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:55.960+0000 7f1fcd7fa640 1 -- 172.21.15.172:0/2038967172 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1fd013bb50 con 0x7f1fd00b1df0 2024-04-14T08:38:56.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.137+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2038967172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fd015e950 con 0x7f1fd00b1df0 2024-04-14T08:38:56.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.138+0000 7f1fcd7fa640 1 -- 172.21.15.172:0/2038967172 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16945 (secure 0 0 0) 0x7f1fc007b3e0 con 0x7f1fd00b1df0 2024-04-14T08:38:56.140 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:56.140 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":86,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:38:46.208211+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:38:42.790996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-14T08:38:43.517208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:38:56.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.140+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2038967172 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1fbc06c8f0 msgr2=0x7f1fbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.140+0000 7f1fd6cb7640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4007fd0 tx=0x7f1fc4009040 comp rx=0 tx=0).stop 2024-04-14T08:38:56.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.140+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2038967172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fd00b1df0 msgr2=0x7f1fd00aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.140+0000 7f1fd6cb7640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fd00b1df0 0x7f1fd00aecd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0000c00 tx=0x7f1fc00046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.141+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2038967172 shutdown_connections 2024-04-14T08:38:56.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.141+0000 7f1fd6cb7640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1fbc06c8f0 0x7f1fbc06edb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.141+0000 7f1fd6cb7640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fd00b4040 0x7f1fd0131d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.141+0000 7f1fd6cb7640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd00b21d0 0x7f1fd0131850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.141+0000 7f1fd6cb7640 1 --2- 172.21.15.172:0/2038967172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fd00b1df0 0x7f1fd00aecd0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.141+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2038967172 >> 172.21.15.172:0/2038967172 conn(0x7f1fd006ec30 msgr2=0x7f1fd015db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:56.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.141+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2038967172 shutdown_connections 2024-04-14T08:38:56.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.141+0000 7f1fd6cb7640 1 -- 172.21.15.172:0/2038967172 wait complete. 2024-04-14T08:38:56.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:38:42.790996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:38:43.517208+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:38:56.154 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:38:56.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.257+0000 7f9c451bb640 1 Processor -- start 2024-04-14T08:38:56.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.257+0000 7f9c451bb640 1 -- start start 2024-04-14T08:38:56.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c451bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c4015c5f0 0x7f9c400703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c451bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9c40070910 0x7f9c400cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c451bb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c400cb570 con 0x7f9c4006b520 2024-04-14T08:38:56.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c451bb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f9c400cb810 con 0x7f9c40070910 2024-04-14T08:38:56.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c451bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9c400cbaf0 con 0x7f9c4015c5f0 2024-04-14T08:38:56.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c3f59e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9c40070910 0x7f9c400cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c3f59e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9c40070910 0x7f9c400cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38014/0 (socket says 172.21.15.172:38014) 2024-04-14T08:38:56.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c3f59e640 1 -- 172.21.15.172:0/4285721170 learned_addr learned my addr 172.21.15.172:0/4285721170 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:56.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c3e59c640 1 --2- 172.21.15.172:0/4285721170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c4015c5f0 0x7f9c400703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c3f59e640 1 -- 172.21.15.172:0/4285721170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c4015c5f0 msgr2=0x7f9c400703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:56.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c3f59e640 1 --2- 172.21.15.172:0/4285721170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c4015c5f0 0x7f9c400703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c3f59e640 1 -- 172.21.15.172:0/4285721170 >> v1:172.21.15.177:6789/0 conn(0x7f9c4006b520 legacy=0x7f9c4006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.258+0000 7f9c3f59e640 1 -- 172.21.15.172:0/4285721170 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c400cbdd0 con 0x7f9c40070910 2024-04-14T08:38:56.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.259+0000 7f9c3f59e640 1 --2- 172.21.15.172:0/4285721170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c40070910 0x7f9c400cc5d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009e50 tx=0x7f9c3802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415689b22b7d3a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:56.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.259+0000 7f9c3dd9b640 1 -- 172.21.15.172:0/4285721170 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3802cb00 con 0x7f9c40070910 2024-04-14T08:38:56.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.259+0000 7f9c3dd9b640 1 -- 172.21.15.172:0/4285721170 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c3802cca0 con 0x7f9c40070910 2024-04-14T08:38:56.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.259+0000 7f9c3dd9b640 1 -- 172.21.15.172:0/4285721170 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38034610 con 0x7f9c40070910 2024-04-14T08:38:56.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.260+0000 7f9c451bb640 1 -- 172.21.15.172:0/4285721170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c40070910 msgr2=0x7f9c400cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.260+0000 7f9c451bb640 1 --2- 172.21.15.172:0/4285721170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c40070910 0x7f9c400cc5d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009e50 tx=0x7f9c3802bf60 comp rx=0 tx=0).stop 2024-04-14T08:38:56.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.260+0000 7f9c451bb640 1 -- 172.21.15.172:0/4285721170 shutdown_connections 2024-04-14T08:38:56.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.260+0000 7f9c451bb640 1 --2- 172.21.15.172:0/4285721170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c40070910 0x7f9c400cc5d0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.260+0000 7f9c451bb640 1 --2- 172.21.15.172:0/4285721170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c4015c5f0 0x7f9c400703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.260+0000 7f9c451bb640 1 -- 172.21.15.172:0/4285721170 >> 172.21.15.172:0/4285721170 conn(0x7f9c400d0f20 msgr2=0x7f9c400d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:56.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.260+0000 7f9c451bb640 1 -- 172.21.15.172:0/4285721170 shutdown_connections 2024-04-14T08:38:56.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.260+0000 7f9c451bb640 1 -- 172.21.15.172:0/4285721170 wait complete. 2024-04-14T08:38:56.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.261+0000 7f9c451bb640 1 Processor -- start 2024-04-14T08:38:56.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.261+0000 7f9c451bb640 1 -- start start 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c451bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c4006b520 0x7f9c401f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c451bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c40070910 0x7f9c400c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c451bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c4015c5f0 0x7f9c400c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c451bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9c400ccb10 con 0x7f9c4015c5f0 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c451bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4006e2a0 con 0x7f9c4006b520 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c451bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9c400ccc90 con 0x7f9c40070910 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c3f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c4015c5f0 0x7f9c400c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c3f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c4015c5f0 0x7f9c400c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38030/0 (socket says 172.21.15.172:38030) 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c3f59e640 1 -- 172.21.15.172:0/395708709 learned_addr learned my addr 172.21.15.172:0/395708709 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c3e59c640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c40070910 0x7f9c400c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c3ed9d640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c4006b520 0x7f9c401f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c3e59c640 1 -- 172.21.15.172:0/395708709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c4006b520 msgr2=0x7f9c401f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c3e59c640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c4006b520 0x7f9c401f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c3e59c640 1 -- 172.21.15.172:0/395708709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c4015c5f0 msgr2=0x7f9c400c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c3e59c640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c4015c5f0 0x7f9c400c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.262+0000 7f9c3e59c640 1 -- 172.21.15.172:0/395708709 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c400a2000 con 0x7f9c40070910 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.263+0000 7f9c3f59e640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c4015c5f0 0x7f9c400c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.263+0000 7f9c3e59c640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c40070910 0x7f9c400c7ff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000e840 tx=0x7f9c3000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.263+0000 7f9c277fe640 1 -- 172.21.15.172:0/395708709 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30017070 con 0x7f9c40070910 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.263+0000 7f9c277fe640 1 -- 172.21.15.172:0/395708709 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c3000c730 con 0x7f9c40070910 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.263+0000 7f9c451bb640 1 -- 172.21.15.172:0/395708709 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c400a0930 con 0x7f9c40070910 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.263+0000 7f9c277fe640 1 -- 172.21.15.172:0/395708709 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000cb00 con 0x7f9c40070910 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.263+0000 7f9c451bb640 1 -- 172.21.15.172:0/395708709 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c400a0ef0 con 0x7f9c40070910 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.264+0000 7f9c451bb640 1 -- 172.21.15.172:0/395708709 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c40075490 con 0x7f9c40070910 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.264+0000 7f9c277fe640 1 -- 172.21.15.172:0/395708709 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c3001a070 con 0x7f9c40070910 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.265+0000 7f9c277fe640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9c2c06c8f0 0x7f9c2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.265+0000 7f9c277fe640 1 -- 172.21.15.172:0/395708709 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f9c3008ba00 con 0x7f9c40070910 2024-04-14T08:38:56.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.265+0000 7f9c3ed9d640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9c2c06c8f0 0x7f9c2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.265+0000 7f9c3ed9d640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9c2c06c8f0 0x7f9c2c06edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9c400b6ff0 tx=0x7f9c34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:56.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.269+0000 7f9c277fe640 1 -- 172.21.15.172:0/395708709 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9c30059640 con 0x7f9c40070910 2024-04-14T08:38:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.443+0000 7f9c451bb640 1 -- 172.21.15.172:0/395708709 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9c4006bd10 con 0x7f9c40070910 2024-04-14T08:38:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.443+0000 7f9c277fe640 1 -- 172.21.15.172:0/395708709 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f9c3005d940 con 0x7f9c40070910 2024-04-14T08:38:56.445 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:38:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.445+0000 7f9c451bb640 1 -- 172.21.15.172:0/395708709 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9c2c06c8f0 msgr2=0x7f9c2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.445+0000 7f9c451bb640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9c2c06c8f0 0x7f9c2c06edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9c400b6ff0 tx=0x7f9c34009040 comp rx=0 tx=0).stop 2024-04-14T08:38:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.445+0000 7f9c451bb640 1 -- 172.21.15.172:0/395708709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c40070910 msgr2=0x7f9c400c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.445+0000 7f9c451bb640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c40070910 0x7f9c400c7ff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000e840 tx=0x7f9c3000ed00 comp rx=0 tx=0).stop 2024-04-14T08:38:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.445+0000 7f9c451bb640 1 -- 172.21.15.172:0/395708709 shutdown_connections 2024-04-14T08:38:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.445+0000 7f9c451bb640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9c2c06c8f0 0x7f9c2c06edb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.445+0000 7f9c451bb640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c4015c5f0 0x7f9c400c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.445+0000 7f9c451bb640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c40070910 0x7f9c400c7ff0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.445+0000 7f9c451bb640 1 --2- 172.21.15.172:0/395708709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c4006b520 0x7f9c401f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.445+0000 7f9c451bb640 1 -- 172.21.15.172:0/395708709 >> 172.21.15.172:0/395708709 conn(0x7f9c400d0f20 msgr2=0x7f9c400721f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.446+0000 7f9c451bb640 1 -- 172.21.15.172:0/395708709 shutdown_connections 2024-04-14T08:38:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.446+0000 7f9c451bb640 1 -- 172.21.15.172:0/395708709 wait complete. 2024-04-14T08:38:56.458 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:38:56.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.561+0000 7fe2f13a3640 1 Processor -- start 2024-04-14T08:38:56.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.561+0000 7fe2f13a3640 1 -- start start 2024-04-14T08:38:56.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2f13a3640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe2ec15e3c0 0x7fe2ec06c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2f13a3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe2ec0d1250 0x7fe2ec06d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2f13a3640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ec06cb70 con 0x7fe2ec06d040 2024-04-14T08:38:56.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2f13a3640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe2ec06ce40 con 0x7fe2ec0d1250 2024-04-14T08:38:56.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2f13a3640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe2ec06d9c0 con 0x7fe2ec15e3c0 2024-04-14T08:38:56.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2eb7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe2ec0d1250 0x7fe2ec06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2eb7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe2ec0d1250 0x7fe2ec06d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38042/0 (socket says 172.21.15.172:38042) 2024-04-14T08:38:56.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2eb7fe640 1 -- 172.21.15.172:0/650410577 learned_addr learned my addr 172.21.15.172:0/650410577 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:56.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2eaffd640 1 --2- 172.21.15.172:0/650410577 >> v2:172.21.15.177:3300/0 conn(0x7fe2ec15e3c0 0x7fe2ec06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2eb7fe640 1 -- 172.21.15.172:0/650410577 >> v2:172.21.15.177:3300/0 conn(0x7fe2ec15e3c0 msgr2=0x7fe2ec06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:56.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2eb7fe640 1 --2- 172.21.15.172:0/650410577 >> v2:172.21.15.177:3300/0 conn(0x7fe2ec15e3c0 0x7fe2ec06c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2eb7fe640 1 -- 172.21.15.172:0/650410577 >> v1:172.21.15.177:6789/0 conn(0x7fe2ec06d040 legacy=0x7fe2ec0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.562+0000 7fe2eb7fe640 1 -- 172.21.15.172:0/650410577 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec0cc780 con 0x7fe2ec0d1250 2024-04-14T08:38:56.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.563+0000 7fe2eb7fe640 1 --2- 172.21.15.172:0/650410577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2ec0d1250 0x7fe2ec06d480 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec072e40 tx=0x7fe2e402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5fec63e7b24536 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:56.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.563+0000 7fe2e9ffb640 1 -- 172.21.15.172:0/650410577 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e402cb20 con 0x7fe2ec0d1250 2024-04-14T08:38:56.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.563+0000 7fe2e9ffb640 1 -- 172.21.15.172:0/650410577 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e402ccc0 con 0x7fe2ec0d1250 2024-04-14T08:38:56.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.563+0000 7fe2e9ffb640 1 -- 172.21.15.172:0/650410577 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4034610 con 0x7fe2ec0d1250 2024-04-14T08:38:56.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.564+0000 7fe2f13a3640 1 -- 172.21.15.172:0/650410577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2ec0d1250 msgr2=0x7fe2ec06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.564+0000 7fe2f13a3640 1 --2- 172.21.15.172:0/650410577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2ec0d1250 0x7fe2ec06d480 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec072e40 tx=0x7fe2e402bf80 comp rx=0 tx=0).stop 2024-04-14T08:38:56.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.564+0000 7fe2f13a3640 1 -- 172.21.15.172:0/650410577 shutdown_connections 2024-04-14T08:38:56.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.564+0000 7fe2f13a3640 1 --2- 172.21.15.172:0/650410577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2ec0d1250 0x7fe2ec06d480 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.564+0000 7fe2f13a3640 1 --2- 172.21.15.172:0/650410577 >> v2:172.21.15.177:3300/0 conn(0x7fe2ec15e3c0 0x7fe2ec06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.564+0000 7fe2f13a3640 1 -- 172.21.15.172:0/650410577 >> 172.21.15.172:0/650410577 conn(0x7fe2ec0b3af0 msgr2=0x7fe2ec0b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:56.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.564+0000 7fe2f13a3640 1 -- 172.21.15.172:0/650410577 shutdown_connections 2024-04-14T08:38:56.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.564+0000 7fe2f13a3640 1 -- 172.21.15.172:0/650410577 wait complete. 2024-04-14T08:38:56.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.565+0000 7fe2f13a3640 1 Processor -- start 2024-04-14T08:38:56.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.565+0000 7fe2f13a3640 1 -- start start 2024-04-14T08:38:56.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2f13a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2ec06d040 0x7fe2ec0b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2f13a3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2ec0d1250 0x7fe2ec160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2f13a3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2ec15e3c0 0x7fe2ec1614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2f13a3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec0cd020 con 0x7fe2ec06d040 2024-04-14T08:38:56.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2f13a3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec0ccd40 con 0x7fe2ec0d1250 2024-04-14T08:38:56.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2f13a3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ec15e7a0 con 0x7fe2ec15e3c0 2024-04-14T08:38:56.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2eaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2ec06d040 0x7fe2ec0b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2eaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2ec06d040 0x7fe2ec0b77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38048/0 (socket says 172.21.15.172:38048) 2024-04-14T08:38:56.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2eaffd640 1 -- 172.21.15.172:0/79648551 learned_addr learned my addr 172.21.15.172:0/79648551 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:56.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2eb7fe640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2ec15e3c0 0x7fe2ec1614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2ea7fc640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2ec0d1250 0x7fe2ec160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2eaffd640 1 -- 172.21.15.172:0/79648551 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2ec15e3c0 msgr2=0x7fe2ec1614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2eaffd640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2ec15e3c0 0x7fe2ec1614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2eaffd640 1 -- 172.21.15.172:0/79648551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2ec0d1250 msgr2=0x7fe2ec160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2eaffd640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2ec0d1250 0x7fe2ec160f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.566+0000 7fe2eaffd640 1 -- 172.21.15.172:0/79648551 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec1f42c0 con 0x7fe2ec06d040 2024-04-14T08:38:56.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.567+0000 7fe2eaffd640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2ec06d040 0x7fe2ec0b77e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00e870 tx=0x7fe2dc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:56.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.567+0000 7fe2d37fe640 1 -- 172.21.15.172:0/79648551 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc017070 con 0x7fe2ec06d040 2024-04-14T08:38:56.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.567+0000 7fe2d37fe640 1 -- 172.21.15.172:0/79648551 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc00c940 con 0x7fe2ec06d040 2024-04-14T08:38:56.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.567+0000 7fe2f13a3640 1 -- 172.21.15.172:0/79648551 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ec1f4580 con 0x7fe2ec06d040 2024-04-14T08:38:56.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.567+0000 7fe2d37fe640 1 -- 172.21.15.172:0/79648551 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc00cd30 con 0x7fe2ec06d040 2024-04-14T08:38:56.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.567+0000 7fe2f13a3640 1 -- 172.21.15.172:0/79648551 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ec1f4b40 con 0x7fe2ec06d040 2024-04-14T08:38:56.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.568+0000 7fe2f13a3640 1 -- 172.21.15.172:0/79648551 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2ec068590 con 0x7fe2ec06d040 2024-04-14T08:38:56.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.568+0000 7fe2d37fe640 1 -- 172.21.15.172:0/79648551 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2dc01a070 con 0x7fe2ec06d040 2024-04-14T08:38:56.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.569+0000 7fe2d37fe640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe2d806c8f0 0x7fe2d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.569+0000 7fe2d37fe640 1 -- 172.21.15.172:0/79648551 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fe2dc08bb00 con 0x7fe2ec06d040 2024-04-14T08:38:56.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.569+0000 7fe2ea7fc640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe2d806c8f0 0x7fe2d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.569+0000 7fe2ea7fc640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe2d806c8f0 0x7fe2d806edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0b58d0 tx=0x7fe2e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:56.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.573+0000 7fe2d37fe640 1 -- 172.21.15.172:0/79648551 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe2dc059750 con 0x7fe2ec06d040 2024-04-14T08:38:56.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.748+0000 7fe2f13a3640 1 -- 172.21.15.172:0/79648551 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe2ec0c8d20 con 0x7fe2ec06d040 2024-04-14T08:38:56.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.749+0000 7fe2d37fe640 1 -- 172.21.15.172:0/79648551 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7fe2dc05da50 con 0x7fe2ec06d040 2024-04-14T08:38:56.750 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:38:56.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.751+0000 7fe2f13a3640 1 -- 172.21.15.172:0/79648551 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe2d806c8f0 msgr2=0x7fe2d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.751+0000 7fe2f13a3640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe2d806c8f0 0x7fe2d806edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0b58d0 tx=0x7fe2e0009040 comp rx=0 tx=0).stop 2024-04-14T08:38:56.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.751+0000 7fe2f13a3640 1 -- 172.21.15.172:0/79648551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2ec06d040 msgr2=0x7fe2ec0b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.751+0000 7fe2f13a3640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2ec06d040 0x7fe2ec0b77e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00e870 tx=0x7fe2dc00ed30 comp rx=0 tx=0).stop 2024-04-14T08:38:56.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.751+0000 7fe2f13a3640 1 -- 172.21.15.172:0/79648551 shutdown_connections 2024-04-14T08:38:56.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.751+0000 7fe2f13a3640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe2d806c8f0 0x7fe2d806edb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.751+0000 7fe2f13a3640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2ec15e3c0 0x7fe2ec1614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.751+0000 7fe2f13a3640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2ec0d1250 0x7fe2ec160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.751+0000 7fe2f13a3640 1 --2- 172.21.15.172:0/79648551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2ec06d040 0x7fe2ec0b77e0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.751+0000 7fe2f13a3640 1 -- 172.21.15.172:0/79648551 >> 172.21.15.172:0/79648551 conn(0x7fe2ec0b3af0 msgr2=0x7fe2ec0afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:56.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.751+0000 7fe2f13a3640 1 -- 172.21.15.172:0/79648551 shutdown_connections 2024-04-14T08:38:56.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.751+0000 7fe2f13a3640 1 -- 172.21.15.172:0/79648551 wait complete. 2024-04-14T08:38:56.763 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:38:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.865+0000 7efe72836640 1 Processor -- start 2024-04-14T08:38:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.865+0000 7efe72836640 1 -- start start 2024-04-14T08:38:56.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.866+0000 7efe72836640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efe6c162ba0 0x7efe6c0d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.866+0000 7efe72836640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe6c0cefb0 con 0x7efe6c162710 2024-04-14T08:38:56.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.866+0000 7efe72836640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe6c0cf290 con 0x7efe6c15e430 2024-04-14T08:38:56.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.866+0000 7efe72836640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7efe6c0cf570 con 0x7efe6c162ba0 2024-04-14T08:38:56.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.866+0000 7efe70dd3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efe6c162ba0 0x7efe6c0d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.866+0000 7efe6b7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7efe6c162710 0x7efe6c0d1df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44914/0 (socket says 172.21.15.172:44914) 2024-04-14T08:38:56.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.866+0000 7efe6b7fe640 1 -- 172.21.15.172:0/2093294671 learned_addr learned my addr 172.21.15.172:0/2093294671 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:56.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.867+0000 7efe70dd3640 1 -- 172.21.15.172:0/2093294671 >> v1:172.21.15.177:6789/0 conn(0x7efe6c15e430 legacy=0x7efe6c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.867+0000 7efe70dd3640 1 -- 172.21.15.172:0/2093294671 >> v1:172.21.15.172:6789/0 conn(0x7efe6c162710 legacy=0x7efe6c0d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.867+0000 7efe6affd640 1 -- 172.21.15.172:0/2093294671 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 833255550 0 0) 0x7efe6c0cf290 con 0x7efe6c15e430 2024-04-14T08:38:56.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.867+0000 7efe70dd3640 1 -- 172.21.15.172:0/2093294671 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c0cf850 con 0x7efe6c162ba0 2024-04-14T08:38:56.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.867+0000 7efe70dd3640 1 --2- 172.21.15.172:0/2093294671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe6c162ba0 0x7efe6c0d2500 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7efe64009d90 tx=0x7efe6402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266bcb66c43ee9d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:56.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.867+0000 7efe6affd640 1 -- 172.21.15.172:0/2093294671 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64039070 con 0x7efe6c162ba0 2024-04-14T08:38:56.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.867+0000 7efe6affd640 1 -- 172.21.15.172:0/2093294671 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6402cab0 con 0x7efe6c162ba0 2024-04-14T08:38:56.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.867+0000 7efe6affd640 1 -- 172.21.15.172:0/2093294671 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64034440 con 0x7efe6c162ba0 2024-04-14T08:38:56.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.868+0000 7efe72836640 1 -- 172.21.15.172:0/2093294671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe6c162ba0 msgr2=0x7efe6c0d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.868+0000 7efe72836640 1 --2- 172.21.15.172:0/2093294671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe6c162ba0 0x7efe6c0d2500 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7efe64009d90 tx=0x7efe6402bd80 comp rx=0 tx=0).stop 2024-04-14T08:38:56.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.868+0000 7efe72836640 1 -- 172.21.15.172:0/2093294671 shutdown_connections 2024-04-14T08:38:56.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.868+0000 7efe72836640 1 --2- 172.21.15.172:0/2093294671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe6c162ba0 0x7efe6c0d2500 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.868+0000 7efe72836640 1 -- 172.21.15.172:0/2093294671 >> 172.21.15.172:0/2093294671 conn(0x7efe6c09d560 msgr2=0x7efe6c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:56.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.868+0000 7efe72836640 1 -- 172.21.15.172:0/2093294671 shutdown_connections 2024-04-14T08:38:56.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.868+0000 7efe72836640 1 -- 172.21.15.172:0/2093294671 wait complete. 2024-04-14T08:38:56.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.869+0000 7efe72836640 1 Processor -- start 2024-04-14T08:38:56.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.869+0000 7efe72836640 1 -- start start 2024-04-14T08:38:56.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.870+0000 7efe72836640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe6c15e430 0x7efe6c1594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.870+0000 7efe72836640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe6c162710 0x7efe6c158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.870+0000 7efe72836640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe6c162ba0 0x7efe6c0b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.870+0000 7efe72836640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c0d4e20 con 0x7efe6c15e430 2024-04-14T08:38:56.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.870+0000 7efe72836640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c15e810 con 0x7efe6c162ba0 2024-04-14T08:38:56.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.870+0000 7efe72836640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efe6c0d0050 con 0x7efe6c162710 2024-04-14T08:38:56.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.870+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe6c15e430 0x7efe6c1594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.870+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe6c15e430 0x7efe6c1594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38074/0 (socket says 172.21.15.172:38074) 2024-04-14T08:38:56.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.870+0000 7efe6bfff640 1 -- 172.21.15.172:0/1758553826 learned_addr learned my addr 172.21.15.172:0/1758553826 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:56.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.870+0000 7efe6b7fe640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe6c162710 0x7efe6c158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.870+0000 7efe70dd3640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe6c162ba0 0x7efe6c0b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.871+0000 7efe6bfff640 1 -- 172.21.15.172:0/1758553826 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe6c162710 msgr2=0x7efe6c158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.871+0000 7efe6bfff640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe6c162710 0x7efe6c158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.871+0000 7efe6bfff640 1 -- 172.21.15.172:0/1758553826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe6c162ba0 msgr2=0x7efe6c0b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.871+0000 7efe6bfff640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe6c162ba0 0x7efe6c0b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.871+0000 7efe6bfff640 1 -- 172.21.15.172:0/1758553826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c0cc8d0 con 0x7efe6c15e430 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.871+0000 7efe6bfff640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe6c15e430 0x7efe6c1594f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00c410 tx=0x7efe5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.871+0000 7efe68ff9640 1 -- 172.21.15.172:0/1758553826 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c017070 con 0x7efe6c15e430 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.871+0000 7efe68ff9640 1 -- 172.21.15.172:0/1758553826 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5c0035f0 con 0x7efe6c15e430 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.871+0000 7efe72836640 1 -- 172.21.15.172:0/1758553826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c0ccbb0 con 0x7efe6c15e430 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.871+0000 7efe68ff9640 1 -- 172.21.15.172:0/1758553826 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0039e0 con 0x7efe6c15e430 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.871+0000 7efe72836640 1 -- 172.21.15.172:0/1758553826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c0cd170 con 0x7efe6c15e430 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.872+0000 7efe72836640 1 -- 172.21.15.172:0/1758553826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6c075040 con 0x7efe6c15e430 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.872+0000 7efe68ff9640 1 -- 172.21.15.172:0/1758553826 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe5c01a070 con 0x7efe6c15e430 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.873+0000 7efe68ff9640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efe5806c8f0 0x7efe5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.873+0000 7efe68ff9640 1 -- 172.21.15.172:0/1758553826 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7efe5c08afb0 con 0x7efe6c15e430 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.873+0000 7efe6b7fe640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efe5806c8f0 0x7efe5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:56.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.873+0000 7efe6b7fe640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efe5806c8f0 0x7efe5806edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0b5650 tx=0x7efe60007e20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:56.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:56.877+0000 7efe68ff9640 1 -- 172.21.15.172:0/1758553826 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efe6c075040 con 0x7efe6c15e430 2024-04-14T08:38:57.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.052+0000 7efe72836640 1 -- 172.21.15.172:0/1758553826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efe6c068590 con 0x7efe6c15e430 2024-04-14T08:38:57.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.052+0000 7efe68ff9640 1 -- 172.21.15.172:0/1758553826 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7efe5c058c90 con 0x7efe6c15e430 2024-04-14T08:38:57.054 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:38:57.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.054+0000 7efe72836640 1 -- 172.21.15.172:0/1758553826 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efe5806c8f0 msgr2=0x7efe5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.054+0000 7efe72836640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efe5806c8f0 0x7efe5806edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0b5650 tx=0x7efe60007e20 comp rx=0 tx=0).stop 2024-04-14T08:38:57.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.054+0000 7efe72836640 1 -- 172.21.15.172:0/1758553826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe6c15e430 msgr2=0x7efe6c1594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.054+0000 7efe72836640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe6c15e430 0x7efe6c1594f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00c410 tx=0x7efe5c00c8d0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.054+0000 7efe72836640 1 -- 172.21.15.172:0/1758553826 shutdown_connections 2024-04-14T08:38:57.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.054+0000 7efe72836640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efe5806c8f0 0x7efe5806edb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.054+0000 7efe72836640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe6c162ba0 0x7efe6c0b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.054+0000 7efe72836640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe6c162710 0x7efe6c158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.054+0000 7efe72836640 1 --2- 172.21.15.172:0/1758553826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe6c15e430 0x7efe6c1594f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.054+0000 7efe72836640 1 -- 172.21.15.172:0/1758553826 >> 172.21.15.172:0/1758553826 conn(0x7efe6c09d560 msgr2=0x7efe6c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:57.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.054+0000 7efe72836640 1 -- 172.21.15.172:0/1758553826 shutdown_connections 2024-04-14T08:38:57.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.054+0000 7efe72836640 1 -- 172.21.15.172:0/1758553826 wait complete. 2024-04-14T08:38:57.067 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:38:57.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.167+0000 7fdecf5e0640 1 Processor -- start 2024-04-14T08:38:57.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.167+0000 7fdecf5e0640 1 -- start start 2024-04-14T08:38:57.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.168+0000 7fdecf5e0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdec806bc30 0x7fdec806fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:57.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.168+0000 7fdecf5e0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdec8073090 0x7fdec80b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:57.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.168+0000 7fdecf5e0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdec80ceee0 con 0x7fdec80ccca0 2024-04-14T08:38:57.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.168+0000 7fdecf5e0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fdec8073490 con 0x7fdec806bc30 2024-04-14T08:38:57.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.168+0000 7fdecf5e0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fdec8070150 con 0x7fdec8073090 2024-04-14T08:38:57.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.168+0000 7fdeccb7b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdec806bc30 0x7fdec806fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:57.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.168+0000 7fdecd37c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fdec80ccca0 0x7fdec806b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44922/0 (socket says 172.21.15.172:44922) 2024-04-14T08:38:57.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.168+0000 7fdecd37c640 1 -- 172.21.15.172:0/1954144777 learned_addr learned my addr 172.21.15.172:0/1954144777 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:57.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.168+0000 7fdecdb7d640 1 --2- 172.21.15.172:0/1954144777 >> v2:172.21.15.177:3300/0 conn(0x7fdec8073090 0x7fdec80b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:57.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.169+0000 7fdeccb7b640 1 -- 172.21.15.172:0/1954144777 >> v2:172.21.15.177:3300/0 conn(0x7fdec8073090 msgr2=0x7fdec80b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:57.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.169+0000 7fdeccb7b640 1 --2- 172.21.15.172:0/1954144777 >> v2:172.21.15.177:3300/0 conn(0x7fdec8073090 0x7fdec80b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.169+0000 7fdeccb7b640 1 -- 172.21.15.172:0/1954144777 >> v1:172.21.15.172:6789/0 conn(0x7fdec80ccca0 legacy=0x7fdec806b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.169+0000 7fdeccb7b640 1 -- 172.21.15.172:0/1954144777 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec80722d0 con 0x7fdec806bc30 2024-04-14T08:38:57.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.169+0000 7fdebffff640 1 -- 172.21.15.172:0/1954144777 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340172673 0 0) 0x7fdec80ceee0 con 0x7fdec80ccca0 2024-04-14T08:38:57.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.169+0000 7fdeccb7b640 1 --2- 172.21.15.172:0/1954144777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec806bc30 0x7fdec806fb70 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fdec8069000 tx=0x7fdeb802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bebe7f5aa21c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:57.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.169+0000 7fdebffff640 1 -- 172.21.15.172:0/1954144777 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb80315a0 con 0x7fdec806bc30 2024-04-14T08:38:57.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.169+0000 7fdebffff640 1 -- 172.21.15.172:0/1954144777 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb8037070 con 0x7fdec806bc30 2024-04-14T08:38:57.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.170+0000 7fdebffff640 1 -- 172.21.15.172:0/1954144777 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8031960 con 0x7fdec806bc30 2024-04-14T08:38:57.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.170+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1954144777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec806bc30 msgr2=0x7fdec806fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.170+0000 7fdecf5e0640 1 --2- 172.21.15.172:0/1954144777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec806bc30 0x7fdec806fb70 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fdec8069000 tx=0x7fdeb802b840 comp rx=0 tx=0).stop 2024-04-14T08:38:57.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.170+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1954144777 shutdown_connections 2024-04-14T08:38:57.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.170+0000 7fdecf5e0640 1 --2- 172.21.15.172:0/1954144777 >> v2:172.21.15.177:3300/0 conn(0x7fdec8073090 0x7fdec80b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.170+0000 7fdecf5e0640 1 --2- 172.21.15.172:0/1954144777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec806bc30 0x7fdec806fb70 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.170+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1954144777 >> 172.21.15.172:0/1954144777 conn(0x7fdec80cda00 msgr2=0x7fdec80cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:57.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.170+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1954144777 shutdown_connections 2024-04-14T08:38:57.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.170+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1954144777 wait complete. 2024-04-14T08:38:57.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.171+0000 7fdecf5e0640 1 Processor -- start 2024-04-14T08:38:57.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.171+0000 7fdecf5e0640 1 -- start start 2024-04-14T08:38:57.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecf5e0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec806bc30 0x7fdec815f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:57.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecf5e0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec8073090 0x7fdec814e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:57.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecf5e0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdec80ccca0 0x7fdec814e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:57.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecf5e0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdec80cd080 con 0x7fdec806bc30 2024-04-14T08:38:57.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecf5e0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8072890 con 0x7fdec80ccca0 2024-04-14T08:38:57.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecf5e0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdec806e150 con 0x7fdec8073090 2024-04-14T08:38:57.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecd37c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec806bc30 0x7fdec815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:57.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecd37c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec806bc30 0x7fdec815f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38100/0 (socket says 172.21.15.172:38100) 2024-04-14T08:38:57.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecd37c640 1 -- 172.21.15.172:0/1088229882 learned_addr learned my addr 172.21.15.172:0/1088229882 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:57.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdeccb7b640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec8073090 0x7fdec814e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecdb7d640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdec80ccca0 0x7fdec814e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecd37c640 1 -- 172.21.15.172:0/1088229882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec8073090 msgr2=0x7fdec814e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecd37c640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec8073090 0x7fdec814e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecd37c640 1 -- 172.21.15.172:0/1088229882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdec80ccca0 msgr2=0x7fdec814e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecd37c640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdec80ccca0 0x7fdec814e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.172+0000 7fdecd37c640 1 -- 172.21.15.172:0/1088229882 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec815e410 con 0x7fdec806bc30 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.173+0000 7fdecd37c640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec806bc30 0x7fdec815f990 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c7c0 tx=0x7fdec400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.173+0000 7fdebdffb640 1 -- 172.21.15.172:0/1088229882 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4018070 con 0x7fdec806bc30 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.173+0000 7fdebdffb640 1 -- 172.21.15.172:0/1088229882 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec400a640 con 0x7fdec806bc30 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.173+0000 7fdebdffb640 1 -- 172.21.15.172:0/1088229882 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec400aa30 con 0x7fdec806bc30 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.173+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1088229882 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec80adee0 con 0x7fdec806bc30 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.173+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1088229882 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec80ae4a0 con 0x7fdec806bc30 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.174+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1088229882 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec8068590 con 0x7fdec806bc30 2024-04-14T08:38:57.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.174+0000 7fdebdffb640 1 -- 172.21.15.172:0/1088229882 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdec401b070 con 0x7fdec806bc30 2024-04-14T08:38:57.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.174+0000 7fdebdffb640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdeac06c8f0 0x7fdeac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:57.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.175+0000 7fdebdffb640 1 -- 172.21.15.172:0/1088229882 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fdec408b1e0 con 0x7fdec806bc30 2024-04-14T08:38:57.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.175+0000 7fdeccb7b640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdeac06c8f0 0x7fdeac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:57.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.175+0000 7fdeccb7b640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdeac06c8f0 0x7fdeac06edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8036000 tx=0x7fdeb803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:57.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.179+0000 7fdebdffb640 1 -- 172.21.15.172:0/1088229882 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdec4058ec0 con 0x7fdec806bc30 2024-04-14T08:38:57.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.367+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1088229882 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdec806c120 con 0x7fdec806bc30 2024-04-14T08:38:57.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.368+0000 7fdebdffb640 1 -- 172.21.15.172:0/1088229882 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v86) ==== 80+0+0 (secure 0 0 0) 0x7fdec405d1c0 con 0x7fdec806bc30 2024-04-14T08:38:57.369 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:38:57.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.369+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1088229882 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdeac06c8f0 msgr2=0x7fdeac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.369+0000 7fdecf5e0640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdeac06c8f0 0x7fdeac06edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8036000 tx=0x7fdeb803d040 comp rx=0 tx=0).stop 2024-04-14T08:38:57.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.369+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1088229882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec806bc30 msgr2=0x7fdec815f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.369+0000 7fdecf5e0640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec806bc30 0x7fdec815f990 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c7c0 tx=0x7fdec400cc80 comp rx=0 tx=0).stop 2024-04-14T08:38:57.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.370+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1088229882 shutdown_connections 2024-04-14T08:38:57.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.370+0000 7fdecf5e0640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdeac06c8f0 0x7fdeac06edb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.370+0000 7fdecf5e0640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdec80ccca0 0x7fdec814e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.370+0000 7fdecf5e0640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdec8073090 0x7fdec814e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.370+0000 7fdecf5e0640 1 --2- 172.21.15.172:0/1088229882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdec806bc30 0x7fdec815f990 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.370+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1088229882 >> 172.21.15.172:0/1088229882 conn(0x7fdec80cda00 msgr2=0x7fdec806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:57.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.370+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1088229882 shutdown_connections 2024-04-14T08:38:57.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.370+0000 7fdecf5e0640 1 -- 172.21.15.172:0/1088229882 wait complete. 2024-04-14T08:38:57.383 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:38:57.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:57.383 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:38:57.432 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:38:57.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:38:57.433 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:38:57.455 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:38:57.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.595+0000 7fd3c9236640 1 Processor -- start 2024-04-14T08:38:57.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.595+0000 7fd3c9236640 1 -- start start 2024-04-14T08:38:57.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3c9236640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40b1df0 0x7fd3c40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:57.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3c9236640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd3c40b4040 0x7fd3c40b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:57.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3c9236640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3c40b49f0 con 0x7fd3c40b21d0 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3c9236640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c415c770 con 0x7fd3c40b4040 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3c9236640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c40b3430 con 0x7fd3c40b1df0 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3c359e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd3c40b4040 0x7fd3c40b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3bbfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd3c40b21d0 0x7fd3c415c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44932/0 (socket says 172.21.15.172:44932) 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3bbfff640 1 -- 172.21.15.172:0/3283505557 learned_addr learned my addr 172.21.15.172:0/3283505557 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3c2d9d640 1 --2- 172.21.15.172:0/3283505557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40b1df0 0x7fd3c40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3c359e640 1 -- 172.21.15.172:0/3283505557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40b1df0 msgr2=0x7fd3c40b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3c359e640 1 --2- 172.21.15.172:0/3283505557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40b1df0 0x7fd3c40b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3c359e640 1 -- 172.21.15.172:0/3283505557 >> v1:172.21.15.172:6789/0 conn(0x7fd3c40b21d0 legacy=0x7fd3c415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3c359e640 1 -- 172.21.15.172:0/3283505557 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c40b3700 con 0x7fd3c40b4040 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.596+0000 7fd3c259c640 1 -- 172.21.15.172:0/3283505557 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3491054765 0 0) 0x7fd3c40b49f0 con 0x7fd3c40b21d0 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.597+0000 7fd3c359e640 1 --2- 172.21.15.172:0/3283505557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3c40b4040 0x7fd3c40b44b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009d90 tx=0x7fd3bc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0737754b7a9aa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:57.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.597+0000 7fd3c259c640 1 -- 172.21.15.172:0/3283505557 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc039070 con 0x7fd3c40b4040 2024-04-14T08:38:57.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.597+0000 7fd3c259c640 1 -- 172.21.15.172:0/3283505557 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3bc02cc50 con 0x7fd3c40b4040 2024-04-14T08:38:57.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.597+0000 7fd3c259c640 1 -- 172.21.15.172:0/3283505557 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc034440 con 0x7fd3c40b4040 2024-04-14T08:38:57.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.597+0000 7fd3c9236640 1 -- 172.21.15.172:0/3283505557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3c40b4040 msgr2=0x7fd3c40b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.598+0000 7fd3c9236640 1 --2- 172.21.15.172:0/3283505557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3c40b4040 0x7fd3c40b44b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009d90 tx=0x7fd3bc02bea0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.598+0000 7fd3c9236640 1 -- 172.21.15.172:0/3283505557 shutdown_connections 2024-04-14T08:38:57.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.598+0000 7fd3c9236640 1 --2- 172.21.15.172:0/3283505557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3c40b4040 0x7fd3c40b44b0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.598+0000 7fd3c9236640 1 --2- 172.21.15.172:0/3283505557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40b1df0 0x7fd3c40b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.598+0000 7fd3c9236640 1 -- 172.21.15.172:0/3283505557 >> 172.21.15.172:0/3283505557 conn(0x7fd3c406ec30 msgr2=0x7fd3c406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:57.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.598+0000 7fd3c9236640 1 -- 172.21.15.172:0/3283505557 shutdown_connections 2024-04-14T08:38:57.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.598+0000 7fd3c9236640 1 -- 172.21.15.172:0/3283505557 wait complete. 2024-04-14T08:38:57.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.599+0000 7fd3c9236640 1 Processor -- start 2024-04-14T08:38:57.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.599+0000 7fd3c9236640 1 -- start start 2024-04-14T08:38:57.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c9236640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3c40b1df0 0x7fd3c4136880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:57.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c9236640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3c40b21d0 0x7fd3c4138520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:57.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c9236640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40b4040 0x7fd3c41579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:57.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c9236640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c415bb40 con 0x7fd3c40b1df0 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c9236640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c40b3cc0 con 0x7fd3c40b21d0 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c9236640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c415b7f0 con 0x7fd3c40b4040 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3c40b1df0 0x7fd3c4136880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3c40b1df0 0x7fd3c4136880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38132/0 (socket says 172.21.15.172:38132) 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c2d9d640 1 -- 172.21.15.172:0/79306121 learned_addr learned my addr 172.21.15.172:0/79306121 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c359e640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40b4040 0x7fd3c41579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3bbfff640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3c40b21d0 0x7fd3c4138520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c2d9d640 1 -- 172.21.15.172:0/79306121 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40b4040 msgr2=0x7fd3c41579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c2d9d640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40b4040 0x7fd3c41579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c2d9d640 1 -- 172.21.15.172:0/79306121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3c40b21d0 msgr2=0x7fd3c4138520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c2d9d640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3c40b21d0 0x7fd3c4138520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c2d9d640 1 -- 172.21.15.172:0/79306121 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c4138a60 con 0x7fd3c40b1df0 2024-04-14T08:38:57.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3bbfff640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3c40b21d0 0x7fd3c4138520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:57.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c2d9d640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3c40b1df0 0x7fd3c4136880 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400e870 tx=0x7fd3b400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:57.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3bb7fe640 1 -- 172.21.15.172:0/79306121 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4017070 con 0x7fd3c40b1df0 2024-04-14T08:38:57.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3bb7fe640 1 -- 172.21.15.172:0/79306121 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b400c940 con 0x7fd3c40b1df0 2024-04-14T08:38:57.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c9236640 1 -- 172.21.15.172:0/79306121 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c4137480 con 0x7fd3c40b1df0 2024-04-14T08:38:57.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3bb7fe640 1 -- 172.21.15.172:0/79306121 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b400cd30 con 0x7fd3c40b1df0 2024-04-14T08:38:57.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.600+0000 7fd3c9236640 1 -- 172.21.15.172:0/79306121 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c4137a40 con 0x7fd3c40b1df0 2024-04-14T08:38:57.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.601+0000 7fd3c9236640 1 -- 172.21.15.172:0/79306121 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c4068520 con 0x7fd3c40b1df0 2024-04-14T08:38:57.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.602+0000 7fd3bb7fe640 1 -- 172.21.15.172:0/79306121 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3b401a070 con 0x7fd3c40b1df0 2024-04-14T08:38:57.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.602+0000 7fd3bb7fe640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd3a406c8f0 0x7fd3a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:57.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.602+0000 7fd3bbfff640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd3a406c8f0 0x7fd3a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:57.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.602+0000 7fd3bb7fe640 1 -- 172.21.15.172:0/79306121 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fd3b408ba60 con 0x7fd3c40b1df0 2024-04-14T08:38:57.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.603+0000 7fd3bbfff640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd3a406c8f0 0x7fd3a406edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40cd520 tx=0x7fd3ac007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:57.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.607+0000 7fd3bb7fe640 1 -- 172.21.15.172:0/79306121 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd3b4059740 con 0x7fd3c40b1df0 2024-04-14T08:38:57.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.820+0000 7fd3c9236640 1 -- 172.21.15.172:0/79306121 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3c40d07d0 con 0x7fd3c40b1df0 2024-04-14T08:38:57.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.820+0000 7fd3bb7fe640 1 -- 172.21.15.172:0/79306121 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5195 (secure 0 0 0) 0x7fd3b405da40 con 0x7fd3c40b1df0 2024-04-14T08:38:57.822 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:57.822 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7473,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1386685680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1386685680},{"type":"v1","addr":"172.21.15.172:6837","nonce":1386685680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:38:46.573174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7491":{"gid":7491,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/1124731311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1124731311},{"type":"v1","addr":"172.21.15.177:6835","nonce":1124731311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7491,"qdb_cluster":[7491]},"id":6}]} 2024-04-14T08:38:57.822 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 54 2024-04-14T08:38:57.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.822+0000 7fd3c9236640 1 -- 172.21.15.172:0/79306121 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd3a406c8f0 msgr2=0x7fd3a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.822+0000 7fd3c9236640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd3a406c8f0 0x7fd3a406edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40cd520 tx=0x7fd3ac007c10 comp rx=0 tx=0).stop 2024-04-14T08:38:57.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.822+0000 7fd3c9236640 1 -- 172.21.15.172:0/79306121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3c40b1df0 msgr2=0x7fd3c4136880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:57.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.822+0000 7fd3c9236640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3c40b1df0 0x7fd3c4136880 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400e870 tx=0x7fd3b400ed30 comp rx=0 tx=0).stop 2024-04-14T08:38:57.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.822+0000 7fd3c9236640 1 -- 172.21.15.172:0/79306121 shutdown_connections 2024-04-14T08:38:57.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.822+0000 7fd3c9236640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd3a406c8f0 0x7fd3a406edb0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.822+0000 7fd3c9236640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd3c40b4040 0x7fd3c41579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.822+0000 7fd3c9236640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3c40b21d0 0x7fd3c4138520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.822+0000 7fd3c9236640 1 --2- 172.21.15.172:0/79306121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3c40b1df0 0x7fd3c4136880 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:57.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.822+0000 7fd3c9236640 1 -- 172.21.15.172:0/79306121 >> 172.21.15.172:0/79306121 conn(0x7fd3c406ec30 msgr2=0x7fd3c415ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:57.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.822+0000 7fd3c9236640 1 -- 172.21.15.172:0/79306121 shutdown_connections 2024-04-14T08:38:57.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:57.822+0000 7fd3c9236640 1 -- 172.21.15.172:0/79306121 wait complete. 2024-04-14T08:38:57.892 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-14T08:38:58.059 INFO:teuthology.orchestra.run.smithi177.stdout:{"message":"","return_code":0} 2024-04-14T08:38:58.059 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-14T08:38:58.059 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:38:58.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.161+0000 7fd87fa36640 1 Processor -- start 2024-04-14T08:38:58.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.161+0000 7fd87fa36640 1 -- start start 2024-04-14T08:38:58.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd87fa36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8780729f0 0x7fd878165490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd87fa36640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8780ce860 con 0x7fd8780af740 2024-04-14T08:38:58.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd87fa36640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd878071520 con 0x7fd87813fa70 2024-04-14T08:38:58.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd87fa36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd8780712d0 con 0x7fd8780729f0 2024-04-14T08:38:58.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd87cfd1640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd8780af740 0x7fd8780722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44944/0 (socket says 172.21.15.172:44944) 2024-04-14T08:38:58.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd87cfd1640 1 -- 172.21.15.172:0/324051006 learned_addr learned my addr 172.21.15.172:0/324051006 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:38:58.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd87dfd3640 1 --2- 172.21.15.172:0/324051006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8780729f0 0x7fd878165490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd86ffff640 1 -- 172.21.15.172:0/324051006 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3521590796 0 0) 0x7fd868001110 con 0x7fd8780af740 2024-04-14T08:38:58.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd86ffff640 1 -- 172.21.15.172:0/324051006 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd864003520 con 0x7fd8780af740 2024-04-14T08:38:58.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd86ffff640 1 -- 172.21.15.172:0/324051006 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 474177 0 0) 0x7fd878071520 con 0x7fd87813fa70 2024-04-14T08:38:58.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd86ffff640 1 -- 172.21.15.172:0/324051006 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd868001110 con 0x7fd87813fa70 2024-04-14T08:38:58.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd87dfd3640 1 -- 172.21.15.172:0/324051006 >> v1:172.21.15.177:6789/0 conn(0x7fd87813fa70 legacy=0x7fd8780ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd87dfd3640 1 -- 172.21.15.172:0/324051006 >> v1:172.21.15.172:6789/0 conn(0x7fd8780af740 legacy=0x7fd8780722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.162+0000 7fd87dfd3640 1 -- 172.21.15.172:0/324051006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd878071800 con 0x7fd8780729f0 2024-04-14T08:38:58.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.163+0000 7fd87dfd3640 1 --2- 172.21.15.172:0/324051006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8780729f0 0x7fd878165490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd87000aed0 tx=0x7fd87002be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3abfccc6213b8471 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.163+0000 7fd86ffff640 1 -- 172.21.15.172:0/324051006 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87002ca20 con 0x7fd8780729f0 2024-04-14T08:38:58.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.163+0000 7fd86ffff640 1 -- 172.21.15.172:0/324051006 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd87002cbc0 con 0x7fd8780729f0 2024-04-14T08:38:58.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.163+0000 7fd86ffff640 1 -- 172.21.15.172:0/324051006 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870008440 con 0x7fd8780729f0 2024-04-14T08:38:58.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.164+0000 7fd87fa36640 1 -- 172.21.15.172:0/324051006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8780729f0 msgr2=0x7fd878165490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.164+0000 7fd87fa36640 1 --2- 172.21.15.172:0/324051006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8780729f0 0x7fd878165490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd87000aed0 tx=0x7fd87002be80 comp rx=0 tx=0).stop 2024-04-14T08:38:58.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.164+0000 7fd87fa36640 1 -- 172.21.15.172:0/324051006 shutdown_connections 2024-04-14T08:38:58.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.164+0000 7fd87fa36640 1 --2- 172.21.15.172:0/324051006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8780729f0 0x7fd878165490 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.164+0000 7fd87fa36640 1 -- 172.21.15.172:0/324051006 >> 172.21.15.172:0/324051006 conn(0x7fd87806b520 msgr2=0x7fd87806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:58.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.164+0000 7fd87fa36640 1 -- 172.21.15.172:0/324051006 shutdown_connections 2024-04-14T08:38:58.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.164+0000 7fd87fa36640 1 -- 172.21.15.172:0/324051006 wait complete. 2024-04-14T08:38:58.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.165+0000 7fd87fa36640 1 Processor -- start 2024-04-14T08:38:58.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.165+0000 7fd87fa36640 1 -- start start 2024-04-14T08:38:58.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87fa36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8780729f0 0x7fd8780aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87fa36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8780af740 0x7fd878141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87fa36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd87813fa70 0x7fd8780b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87fa36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd878144130 con 0x7fd8780af740 2024-04-14T08:38:58.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87fa36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd878071dc0 con 0x7fd8780729f0 2024-04-14T08:38:58.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87fa36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd8781442b0 con 0x7fd87813fa70 2024-04-14T08:38:58.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87cfd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8780af740 0x7fd878141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87cfd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8780af740 0x7fd878141a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38138/0 (socket says 172.21.15.172:38138) 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87cfd1640 1 -- 172.21.15.172:0/1252880800 learned_addr learned my addr 172.21.15.172:0/1252880800 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87d7d2640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8780729f0 0x7fd8780aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87dfd3640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd87813fa70 0x7fd8780b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87cfd1640 1 -- 172.21.15.172:0/1252880800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd87813fa70 msgr2=0x7fd8780b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87cfd1640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd87813fa70 0x7fd8780b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87cfd1640 1 -- 172.21.15.172:0/1252880800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8780729f0 msgr2=0x7fd8780aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87cfd1640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8780729f0 0x7fd8780aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87cfd1640 1 -- 172.21.15.172:0/1252880800 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8781ef8e0 con 0x7fd8780af740 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.166+0000 7fd87cfd1640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8780af740 0x7fd878141a60 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fd86800c680 tx=0x7fd86800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.167+0000 7fd86dffb640 1 -- 172.21.15.172:0/1252880800 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868017070 con 0x7fd8780af740 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.167+0000 7fd86dffb640 1 -- 172.21.15.172:0/1252880800 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd868005ab0 con 0x7fd8780af740 2024-04-14T08:38:58.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.167+0000 7fd87fa36640 1 -- 172.21.15.172:0/1252880800 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd878142420 con 0x7fd8780af740 2024-04-14T08:38:58.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.167+0000 7fd86dffb640 1 -- 172.21.15.172:0/1252880800 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868007070 con 0x7fd8780af740 2024-04-14T08:38:58.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.167+0000 7fd87fa36640 1 -- 172.21.15.172:0/1252880800 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8781429e0 con 0x7fd8780af740 2024-04-14T08:38:58.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.168+0000 7fd87fa36640 1 -- 172.21.15.172:0/1252880800 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd878068590 con 0x7fd8780af740 2024-04-14T08:38:58.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.168+0000 7fd86dffb640 1 -- 172.21.15.172:0/1252880800 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd86801a070 con 0x7fd8780af740 2024-04-14T08:38:58.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.169+0000 7fd86dffb640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd85c06c920 0x7fd85c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.169+0000 7fd87d7d2640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd85c06c920 0x7fd85c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.169+0000 7fd86dffb640 1 -- 172.21.15.172:0/1252880800 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fd86808b2a0 con 0x7fd8780af740 2024-04-14T08:38:58.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.169+0000 7fd87d7d2640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd85c06c920 0x7fd85c06ede0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd874007fd0 tx=0x7fd874009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.173+0000 7fd86dffb640 1 -- 172.21.15.172:0/1252880800 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd8780ce860 con 0x7fd8780af740 2024-04-14T08:38:58.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.386+0000 7fd87fa36640 1 -- 172.21.15.172:0/1252880800 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd87806c530 con 0x7fd8780af740 2024-04-14T08:38:58.388 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:38:58.387+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:38:58.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.390+0000 7fd86dffb640 1 -- 172.21.15.172:0/1252880800 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) ==== 161+0+0 (secure 0 0 0) 0x7fd868058f10 con 0x7fd8780af740 2024-04-14T08:38:58.391 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:38:58.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.392+0000 7fd87fa36640 1 -- 172.21.15.172:0/1252880800 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd85c06c920 msgr2=0x7fd85c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.392+0000 7fd87fa36640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd85c06c920 0x7fd85c06ede0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd874007fd0 tx=0x7fd874009040 comp rx=0 tx=0).stop 2024-04-14T08:38:58.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.392+0000 7fd87fa36640 1 -- 172.21.15.172:0/1252880800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8780af740 msgr2=0x7fd878141a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.392+0000 7fd87fa36640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8780af740 0x7fd878141a60 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fd86800c680 tx=0x7fd86800cb40 comp rx=0 tx=0).stop 2024-04-14T08:38:58.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.392+0000 7fd87fa36640 1 -- 172.21.15.172:0/1252880800 shutdown_connections 2024-04-14T08:38:58.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.392+0000 7fd87fa36640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd85c06c920 0x7fd85c06ede0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.392+0000 7fd87fa36640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd87813fa70 0x7fd8780b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.392+0000 7fd87fa36640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8780af740 0x7fd878141a60 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.392+0000 7fd87fa36640 1 --2- 172.21.15.172:0/1252880800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8780729f0 0x7fd8780aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.392+0000 7fd87fa36640 1 -- 172.21.15.172:0/1252880800 >> 172.21.15.172:0/1252880800 conn(0x7fd87806b520 msgr2=0x7fd878159f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:58.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.392+0000 7fd87fa36640 1 -- 172.21.15.172:0/1252880800 shutdown_connections 2024-04-14T08:38:58.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.392+0000 7fd87fa36640 1 -- 172.21.15.172:0/1252880800 wait complete. 2024-04-14T08:38:58.397 INFO:tasks.ceph.mds.d.smithi177.stderr: -5159> 2024-04-14T08:38:47.198+0000 7f30dddf1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:38:58.405 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-14T08:38:58.427 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-04-14T08:38:58.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.445+0000 7fa7b003af40 1 Processor -- start 2024-04-14T08:38:58.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.445+0000 7fa7b003af40 1 -- start start 2024-04-14T08:38:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7b003af40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621396057c0 0x5621396ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7b003af40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562139606520 0x5621396dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7b003af40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5621396da710 0x5621396dcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7b003af40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5621395ad3b0 con 0x562139606520 2024-04-14T08:38:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7b003af40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5621395b1400 con 0x5621396057c0 2024-04-14T08:38:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7b003af40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5621396db1c0 con 0x5621396da710 2024-04-14T08:38:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7ac9a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562139606520 0x5621396dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7ac9a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562139606520 0x5621396dbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38150/0 (socket says 172.21.15.172:38150) 2024-04-14T08:38:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7ac9a2640 1 -- 172.21.15.172:0/1259846727 learned_addr learned my addr 172.21.15.172:0/1259846727 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7ad1a3640 1 --2- 172.21.15.172:0/1259846727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621396057c0 0x5621396ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7ad9a4640 1 --2- 172.21.15.172:0/1259846727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5621396da710 0x5621396dcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7ac9a2640 1 -- 172.21.15.172:0/1259846727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5621396da710 msgr2=0x5621396dcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7ac9a2640 1 --2- 172.21.15.172:0/1259846727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5621396da710 0x5621396dcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7ac9a2640 1 -- 172.21.15.172:0/1259846727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621396057c0 msgr2=0x5621396ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7ac9a2640 1 --2- 172.21.15.172:0/1259846727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621396057c0 0x5621396ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7ac9a2640 1 -- 172.21.15.172:0/1259846727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562139650730 con 0x562139606520 2024-04-14T08:38:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.446+0000 7fa7ad1a3640 1 --2- 172.21.15.172:0/1259846727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621396057c0 0x5621396ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa7ac9a2640 1 --2- 172.21.15.172:0/1259846727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562139606520 0x5621396dbdd0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4055040 tx=0x7fa7a404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a92afdbbe64c681 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa79f7fe640 1 -- 172.21.15.172:0/1259846727 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4057070 con 0x562139606520 2024-04-14T08:38:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa79f7fe640 1 -- 172.21.15.172:0/1259846727 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a4061070 con 0x562139606520 2024-04-14T08:38:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa79f7fe640 1 -- 172.21.15.172:0/1259846727 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4052840 con 0x562139606520 2024-04-14T08:38:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa7b003af40 1 -- 172.21.15.172:0/1259846727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562139606520 msgr2=0x5621396dbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa7b003af40 1 --2- 172.21.15.172:0/1259846727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562139606520 0x5621396dbdd0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4055040 tx=0x7fa7a404c000 comp rx=0 tx=0).stop 2024-04-14T08:38:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa7b003af40 1 -- 172.21.15.172:0/1259846727 shutdown_connections 2024-04-14T08:38:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa7b003af40 1 --2- 172.21.15.172:0/1259846727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5621396da710 0x5621396dcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa7b003af40 1 --2- 172.21.15.172:0/1259846727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562139606520 0x5621396dbdd0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa7b003af40 1 --2- 172.21.15.172:0/1259846727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621396057c0 0x5621396ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa7b003af40 1 -- 172.21.15.172:0/1259846727 >> 172.21.15.172:0/1259846727 conn(0x562139630c50 msgr2=0x562139607f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa7b003af40 1 -- 172.21.15.172:0/1259846727 shutdown_connections 2024-04-14T08:38:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.447+0000 7fa7b003af40 1 -- 172.21.15.172:0/1259846727 wait complete. 2024-04-14T08:38:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.448+0000 7fa7b003af40 1 Processor -- start 2024-04-14T08:38:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.448+0000 7fa7b003af40 1 -- start start 2024-04-14T08:38:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7b003af40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621396057c0 0x5621396c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7b003af40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562139606520 0x5621396c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7b003af40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621396da710 0x5621396c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7b003af40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562139636270 con 0x5621396057c0 2024-04-14T08:38:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7b003af40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5621396cdd40 con 0x5621396da710 2024-04-14T08:38:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7b003af40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5621396d0ac0 con 0x562139606520 2024-04-14T08:38:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ad1a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621396057c0 0x5621396c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ad1a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621396057c0 0x5621396c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38160/0 (socket says 172.21.15.172:38160) 2024-04-14T08:38:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ad1a3640 1 -- 172.21.15.172:0/2777053827 learned_addr learned my addr 172.21.15.172:0/2777053827 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ac9a2640 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562139606520 0x5621396c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ad9a4640 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621396da710 0x5621396c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ad1a3640 1 -- 172.21.15.172:0/2777053827 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562139606520 msgr2=0x5621396c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ad1a3640 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562139606520 0x5621396c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ad1a3640 1 -- 172.21.15.172:0/2777053827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621396da710 msgr2=0x5621396c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ad1a3640 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621396da710 0x5621396c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ad1a3640 1 -- 172.21.15.172:0/2777053827 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621396c7ae0 con 0x5621396057c0 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ac9a2640 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562139606520 0x5621396c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ad9a4640 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621396da710 0x5621396c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7ad1a3640 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621396057c0 0x5621396c6bc0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000ddb0 tx=0x7fa7a000b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa79e7fc640 1 -- 172.21.15.172:0/2777053827 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0015020 con 0x5621396057c0 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa79e7fc640 1 -- 172.21.15.172:0/2777053827 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a0011660 con 0x5621396057c0 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7b003af40 1 -- 172.21.15.172:0/2777053827 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5621396e4b60 con 0x5621396057c0 2024-04-14T08:38:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa79e7fc640 1 -- 172.21.15.172:0/2777053827 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0011a50 con 0x5621396057c0 2024-04-14T08:38:58.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.449+0000 7fa7b003af40 1 -- 172.21.15.172:0/2777053827 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621396c7100 con 0x5621396057c0 2024-04-14T08:38:58.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.451+0000 7fa79e7fc640 1 -- 172.21.15.172:0/2777053827 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7a0018020 con 0x5621396057c0 2024-04-14T08:38:58.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.451+0000 7fa79e7fc640 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa79006b2d0 0x7fa79006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.451+0000 7fa79e7fc640 1 -- 172.21.15.172:0/2777053827 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7fa7a008aef0 con 0x5621396057c0 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.451+0000 7fa7ac9a2640 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa79006b2d0 0x7fa79006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.451+0000 7fa7b003af40 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5621396cea20 0x5621396e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.451+0000 7fa7b003af40 1 -- 172.21.15.172:0/2777053827 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e87) -- 0x5621396e4390 con 0x5621396cea20 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.451+0000 7fa7ad9a4640 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5621396cea20 0x5621396e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.452+0000 7fa7ac9a2640 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa79006b2d0 0x7fa79006d790 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4060000 tx=0x7fa7a404f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.452+0000 7fa7ad9a4640 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5621396cea20 0x5621396e3e50 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.455+0000 7fa7ad9a4640 1 -- 172.21.15.172:0/2777053827 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v87'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa7a80044c0 con 0x5621396cea20 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.455+0000 7fa7b003af40 1 -- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5621396cea20 msgr2=0x5621396e3e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.455+0000 7fa7b003af40 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5621396cea20 0x5621396e3e50 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.455+0000 7fa7b003af40 1 -- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa79006b2d0 msgr2=0x7fa79006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.455+0000 7fa7b003af40 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa79006b2d0 0x7fa79006d790 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4060000 tx=0x7fa7a404f000 comp rx=0 tx=0).stop 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.455+0000 7fa7b003af40 1 -- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621396057c0 msgr2=0x5621396c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.455+0000 7fa7b003af40 1 --2- 172.21.15.172:0/2777053827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621396057c0 0x5621396c6bc0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000ddb0 tx=0x7fa7a000b590 comp rx=0 tx=0).stop 2024-04-14T08:38:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.455+0000 7fa7ad9a4640 1 -- 172.21.15.172:0/2777053827 reap_dead start 2024-04-14T08:38:58.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.455+0000 7fa7b003af40 1 -- 172.21.15.172:0/2777053827 shutdown_connections 2024-04-14T08:38:58.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.455+0000 7fa7b003af40 1 -- 172.21.15.172:0/2777053827 >> 172.21.15.172:0/2777053827 conn(0x562139630c50 msgr2=0x562139607f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:58.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.455+0000 7fa7b003af40 1 -- 172.21.15.172:0/2777053827 shutdown_connections 2024-04-14T08:38:58.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.455+0000 7fa7b003af40 1 -- 172.21.15.172:0/2777053827 wait complete. 2024-04-14T08:38:58.461 DEBUG:teuthology.orchestra.run.smithi172:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-14T08:38:58.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf15493280 1 -- start start 2024-04-14T08:38:58.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf15493280 1 Processor -- start 2024-04-14T08:38:58.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf15493280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d988e400 0x5613d9872b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf15493280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d988f000 0x5613d9872580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf15493280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988f400 0x5613d9872000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf15493280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5613d98ac000 con 0x5613d988f400 2024-04-14T08:38:58.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf15493280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5613d98ac180 con 0x5613d988f000 2024-04-14T08:38:58.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf15493280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5613d98ac300 con 0x5613d988e400 2024-04-14T08:38:58.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf141fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988f400 0x5613d9872000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf141fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988f400 0x5613d9872000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38178/0 (socket says 172.21.15.172:38178) 2024-04-14T08:38:58.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf139fc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d988e400 0x5613d9872b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf131fb640 1 --2- 172.21.15.172:0/415619352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d988f000 0x5613d9872580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.541+0000 7fbf141fd640 1 -- 172.21.15.172:0/415619352 learned_addr learned my addr 172.21.15.172:0/415619352 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf141fd640 1 -- 172.21.15.172:0/415619352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d988e400 msgr2=0x5613d9872b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf141fd640 1 --2- 172.21.15.172:0/415619352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d988e400 0x5613d9872b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf141fd640 1 -- 172.21.15.172:0/415619352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d988f000 msgr2=0x5613d9872580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf141fd640 1 --2- 172.21.15.172:0/415619352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d988f000 0x5613d9872580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf141fd640 1 -- 172.21.15.172:0/415619352 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d8ae4780 con 0x5613d988f400 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf131fb640 1 --2- 172.21.15.172:0/415619352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d988f000 0x5613d9872580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf141fd640 1 --2- 172.21.15.172:0/415619352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988f400 0x5613d9872000 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x5613d98d74a0 tx=0x5613d9842060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351ec0d1978faad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf111f7640 1 -- 172.21.15.172:0/415619352 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613d8ae29c0 con 0x5613d988f400 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf111f7640 1 -- 172.21.15.172:0/415619352 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5613d8ad8e00 con 0x5613d988f400 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf111f7640 1 -- 172.21.15.172:0/415619352 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613d8ae21a0 con 0x5613d988f400 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf15493280 1 -- 172.21.15.172:0/415619352 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d8ae5860 con 0x5613d988f400 2024-04-14T08:38:58.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf15493280 1 -- 172.21.15.172:0/415619352 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5613d8ae5a40 con 0x5613d988f400 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf15493280 4 waiting for MDS map... 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf111f7640 1 -- 172.21.15.172:0/415619352 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x5613d997ca00 con 0x5613d988f400 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf111f7640 1 -- 172.21.15.172:0/415619352 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 55) ==== 2885+0+0 (secure 0 0 0) 0x5613d8b09180 con 0x5613d988f400 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf15493280 4 Got MDS map 55 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.542+0000 7fbf15493280 10 main: JournalTool::main 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.543+0000 7fbf15493280 4 main: JournalTool: connecting to RADOS... 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.543+0000 7fbf15493280 1 Processor -- start 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.543+0000 7fbf15493280 1 -- start start 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf15493280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988fc00 0x5613d9873600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf15493280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d99d2000 0x5613d9873b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf15493280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99d2400 0x5613d9874100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf15493280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5613d98ac480 con 0x5613d988fc00 2024-04-14T08:38:58.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf15493280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5613d98ac600 con 0x5613d99d2400 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf15493280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5613d98ac780 con 0x5613d99d2000 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf131fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988fc00 0x5613d9873600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf131fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988fc00 0x5613d9873600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38182/0 (socket says 172.21.15.172:38182) 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf131fb640 1 -- 172.21.15.172:0/1337550766 learned_addr learned my addr 172.21.15.172:0/1337550766 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf139fc640 1 --2- 172.21.15.172:0/1337550766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d99d2000 0x5613d9873b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf131fb640 1 --2- 172.21.15.172:0/1337550766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99d2400 0x5613d9874100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf131fb640 1 -- 172.21.15.172:0/1337550766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d99d2000 msgr2=0x5613d9873b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf131fb640 1 --2- 172.21.15.172:0/1337550766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d99d2000 0x5613d9873b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf131fb640 1 -- 172.21.15.172:0/1337550766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99d2400 msgr2=0x5613d9874100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf131fb640 1 --2- 172.21.15.172:0/1337550766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99d2400 0x5613d9874100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf131fb640 1 -- 172.21.15.172:0/1337550766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d8ae5c20 con 0x5613d988fc00 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf139fc640 1 --2- 172.21.15.172:0/1337550766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d99d2000 0x5613d9873b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:38:58.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf131fb640 1 --2- 172.21.15.172:0/1337550766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988fc00 0x5613d9873600 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x5613d98d69c0 tx=0x5613d9840900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a953c2f2cfbabc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf0f1f3640 1 -- 172.21.15.172:0/1337550766 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613d8ae2340 con 0x5613d988fc00 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf0f1f3640 1 -- 172.21.15.172:0/1337550766 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5613d8ad8a80 con 0x5613d988fc00 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.544+0000 7fbf0f1f3640 1 -- 172.21.15.172:0/1337550766 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613d8ae29c0 con 0x5613d988fc00 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.545+0000 7fbf15493280 1 -- 172.21.15.172:0/1337550766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988fc00 msgr2=0x5613d9873600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.545+0000 7fbf15493280 1 --2- 172.21.15.172:0/1337550766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988fc00 0x5613d9873600 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x5613d98d69c0 tx=0x5613d9840900 comp rx=0 tx=0).stop 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.545+0000 7fbf15493280 1 -- 172.21.15.172:0/1337550766 shutdown_connections 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.545+0000 7fbf15493280 1 --2- 172.21.15.172:0/1337550766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99d2400 0x5613d9874100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.545+0000 7fbf15493280 1 --2- 172.21.15.172:0/1337550766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d99d2000 0x5613d9873b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.545+0000 7fbf15493280 1 --2- 172.21.15.172:0/1337550766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988fc00 0x5613d9873600 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.545+0000 7fbf15493280 1 -- 172.21.15.172:0/1337550766 >> 172.21.15.172:0/1337550766 conn(0x5613d988f800 msgr2=0x5613d8b99000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.545+0000 7fbf15493280 1 -- 172.21.15.172:0/1337550766 shutdown_connections 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.545+0000 7fbf15493280 1 -- 172.21.15.172:0/1337550766 wait complete. 2024-04-14T08:38:58.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.545+0000 7fbf15493280 1 Processor -- start 2024-04-14T08:38:58.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.545+0000 7fbf15493280 1 -- start start 2024-04-14T08:38:58.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf15493280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d988f800 0x5613d9873600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf15493280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99da000 0x5613d9873b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf15493280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99da400 0x5613d9874100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf15493280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5613d98adc80 con 0x5613d99da400 2024-04-14T08:38:58.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf15493280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5613d98adb00 con 0x5613d99da000 2024-04-14T08:38:58.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf15493280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5613d98ad980 con 0x5613d988f800 2024-04-14T08:38:58.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf131fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99da400 0x5613d9874100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf139fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99da000 0x5613d9873b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf131fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99da400 0x5613d9874100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38188/0 (socket says 172.21.15.172:38188) 2024-04-14T08:38:58.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf139fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99da000 0x5613d9873b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:37946/0 (socket says 172.21.15.172:37946) 2024-04-14T08:38:58.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf131fb640 1 -- 172.21.15.172:0/1969605401 learned_addr learned my addr 172.21.15.172:0/1969605401 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:58.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf131fb640 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d988f800 0x5613d9873600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf131fb640 1 -- 172.21.15.172:0/1969605401 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d988f800 msgr2=0x5613d9873600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf131fb640 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d988f800 0x5613d9873600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf131fb640 1 -- 172.21.15.172:0/1969605401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99da000 msgr2=0x5613d9873b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf131fb640 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99da000 0x5613d9873b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf131fb640 1 -- 172.21.15.172:0/1969605401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d8ae5680 con 0x5613d99da400 2024-04-14T08:38:58.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf139fc640 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99da000 0x5613d9873b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:58.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf131fb640 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99da400 0x5613d9874100 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x5613d997e960 tx=0x5613d9840960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf0d9f0640 1 -- 172.21.15.172:0/1969605401 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613d8ae21a0 con 0x5613d99da400 2024-04-14T08:38:58.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf0d9f0640 1 -- 172.21.15.172:0/1969605401 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5613d8ad8c40 con 0x5613d99da400 2024-04-14T08:38:58.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf15493280 1 -- 172.21.15.172:0/1969605401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5613d8ae54a0 con 0x5613d99da400 2024-04-14T08:38:58.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.546+0000 7fbf0d9f0640 1 -- 172.21.15.172:0/1969605401 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613d8ae2000 con 0x5613d99da400 2024-04-14T08:38:58.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.547+0000 7fbf15493280 1 -- 172.21.15.172:0/1969605401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d8ae43c0 con 0x5613d99da400 2024-04-14T08:38:58.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.547+0000 7fbf15493280 4 main: JournalTool: resolving pool 13 2024-04-14T08:38:58.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.548+0000 7fbf0d9f0640 1 -- 172.21.15.172:0/1969605401 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5613d99d2c00 con 0x5613d99da400 2024-04-14T08:38:58.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.548+0000 7fbf0d9f0640 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5613d9a08400 0x5613d9874680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.548+0000 7fbf0d9f0640 1 -- 172.21.15.172:0/1969605401 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x5613d8c1dc00 con 0x5613d99da400 2024-04-14T08:38:58.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.548+0000 7fbf15493280 4 main: JournalTool: creating IoCtx.. 2024-04-14T08:38:58.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.548+0000 7fbf15493280 4 main: Executing for rank 0 2024-04-14T08:38:58.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.548+0000 7fbf139fc640 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5613d9a08400 0x5613d9874680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.548+0000 7fbf139fc640 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5613d9a08400 0x5613d9874680 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x5613d8bb0990 tx=0x5613d8dd2a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf15493280 1 -- start start 2024-04-14T08:38:58.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf15493280 1 Processor -- start 2024-04-14T08:38:58.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf15493280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99dac00 0x5613d9874c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf15493280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99db000 0x5613d9875180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf15493280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d99db400 0x5613d9875700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf15493280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5613d98ad680 con 0x5613d99db000 2024-04-14T08:38:58.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf15493280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5613d98ad500 con 0x5613d99dac00 2024-04-14T08:38:58.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf15493280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5613d98ac480 con 0x5613d99db400 2024-04-14T08:38:58.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf139fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99db000 0x5613d9875180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf139fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99db000 0x5613d9875180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38202/0 (socket says 172.21.15.172:38202) 2024-04-14T08:38:58.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf139fc640 1 -- 172.21.15.172:0/2857109670 learned_addr learned my addr 172.21.15.172:0/2857109670 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:58.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf131fb640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d99db400 0x5613d9875700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.549+0000 7fbf141fd640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99dac00 0x5613d9874c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf139fc640 1 -- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d99db400 msgr2=0x5613d9875700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf139fc640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d99db400 0x5613d9875700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf139fc640 1 -- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99dac00 msgr2=0x5613d9874c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf139fc640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99dac00 0x5613d9874c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf139fc640 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d8ae5c20 con 0x5613d99db000 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf139fc640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99db000 0x5613d9875180 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x5613d8dfb590 tx=0x5613d8dd2ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4268163e757972b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf0a9ea640 1 -- 172.21.15.172:0/2857109670 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613d8dc4d00 con 0x5613d99db000 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf0a9ea640 1 -- 172.21.15.172:0/2857109670 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5613d8ad8540 con 0x5613d99db000 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf0a9ea640 1 -- 172.21.15.172:0/2857109670 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613d8dc5ba0 con 0x5613d99db000 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d8ae4780 con 0x5613d99db000 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5613d8ae5860 con 0x5613d99db000 2024-04-14T08:38:58.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf15493280 4 waiting for MDS map... 2024-04-14T08:38:58.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf0a9ea640 1 -- 172.21.15.172:0/2857109670 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x5613d8e92e00 con 0x5613d99db000 2024-04-14T08:38:58.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf0a9ea640 1 -- 172.21.15.172:0/2857109670 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 55) ==== 2885+0+0 (secure 0 0 0) 0x5613d8b09500 con 0x5613d99db000 2024-04-14T08:38:58.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf15493280 4 Got MDS map 55 2024-04-14T08:38:58.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5613d99db800 0x5613d8eb0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5613d99dbc00 con 0x5613d99db800 2024-04-14T08:38:58.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.550+0000 7fbf141fd640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5613d99db800 0x5613d8eb0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.551+0000 7fbf141fd640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5613d99db800 0x5613d8eb0000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=86ca77da11dd88d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.553+0000 7fbf141fd640 1 -- 172.21.15.172:0/2857109670 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5613d8bae000 con 0x5613d99db800 2024-04-14T08:38:58.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.553+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5613d8ea5400 0x5613d8eb0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.553+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e87) -- 0x5613d8ea5800 con 0x5613d8ea5400 2024-04-14T08:38:58.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.553+0000 7fbf131fb640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5613d8ea5400 0x5613d8eb0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.553+0000 7fbf131fb640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5613d8ea5400 0x5613d8eb0580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=d2f1d116b7eaabc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.554+0000 7fbf131fb640 1 -- 172.21.15.172:0/2857109670 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5613d8bae500 con 0x5613d8ea5400 2024-04-14T08:38:58.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.554+0000 7fbf091e7640 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87) -- 0x5613d8ebc000 con 0x5613d8ea5400 2024-04-14T08:38:58.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.554+0000 7fbf091e7640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5613d8ebc400 0x5613d8eb1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.554+0000 7fbf091e7640 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e87) -- 0x5613d8ebc800 con 0x5613d8ebc400 2024-04-14T08:38:58.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.554+0000 7fbf141fd640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5613d8ebc400 0x5613d8eb1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.554+0000 7fbf131fb640 1 -- 172.21.15.172:0/2857109670 <== osd.0 v2:172.21.15.172:6800/3761711279 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x5613d8bae500 con 0x5613d8ea5400 2024-04-14T08:38:58.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.554+0000 7fbf141fd640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5613d8ebc400 0x5613d8eb1b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=af58321ed1271b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.554+0000 7fbf141fd640 1 -- 172.21.15.172:0/2857109670 <== osd.4 v2:172.21.15.177:6820/4180733146 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5613d8bae000 con 0x5613d8ebc400 2024-04-14T08:38:58.566 INFO:teuthology.orchestra.run.smithi172.stdout:old journal was 4212724~2605 2024-04-14T08:38:58.566 INFO:teuthology.orchestra.run.smithi172.stdout:new journal start will be 8388608 (4173279 bytes past old end) 2024-04-14T08:38:58.566 INFO:teuthology.orchestra.run.smithi172.stdout:writing journal head 2024-04-14T08:38:58.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.555+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5613d8ea5c00 con 0x5613d8ea5400 2024-04-14T08:38:58.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf131fb640 1 -- 172.21.15.172:0/2857109670 <== osd.0 v2:172.21.15.172:6800/3761711279 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v87'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5613d8bae500 con 0x5613d8ea5400 2024-04-14T08:38:58.567 INFO:teuthology.orchestra.run.smithi172.stdout:writing EResetJournal entry 2024-04-14T08:38:58.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5613d8ebcc00 con 0x5613d8ebc400 2024-04-14T08:38:58.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x5613d8ebd000 0x5613d8eb0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5613d8ebd400 con 0x5613d8ebd000 2024-04-14T08:38:58.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5613d8ebd800 con 0x5613d99db800 2024-04-14T08:38:58.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5613d8ebdc00 0x5613d8eb1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5613d8ebc800 con 0x5613d8ebdc00 2024-04-14T08:38:58.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5613d8ebcc00 con 0x5613d8ebc400 2024-04-14T08:38:58.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5613d99dbc00 0x5613d8eb1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5613d98af800 con 0x5613d99dbc00 2024-04-14T08:38:58.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf131fb640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5613d8ebdc00 0x5613d8eb1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf139fc640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x5613d8ebd000 0x5613d8eb0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf141fd640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5613d99dbc00 0x5613d8eb1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf131fb640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5613d8ebdc00 0x5613d8eb1080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=264bfd37748b4320 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.556+0000 7fbf139fc640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x5613d8ebd000 0x5613d8eb0b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=2531b375af33a9ea server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.557+0000 7fbf141fd640 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5613d99dbc00 0x5613d8eb1600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=58be708c1aa58e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.557+0000 7fbf131fb640 1 -- 172.21.15.172:0/2857109670 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(9 200.00000005 [delete] v87'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5613d8bae500 con 0x5613d8ebdc00 2024-04-14T08:38:58.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.558+0000 7fbf141fd640 1 -- 172.21.15.172:0/2857109670 <== osd.4 v2:172.21.15.177:6820/4180733146 2 ==== osd_op_reply(6 200.00000002 [delete] v87'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5613d8bae000 con 0x5613d8ebc400 2024-04-14T08:38:58.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.558+0000 7fbf141fd640 1 -- 172.21.15.172:0/2857109670 <== osd.6 v2:172.21.15.177:6800/2961907928 2 ==== osd_op_reply(8 200.00000004 [delete] v87'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5613d8bae000 con 0x5613d99db800 2024-04-14T08:38:58.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.558+0000 7fbf141fd640 1 -- 172.21.15.172:0/2857109670 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== osd_op_reply(11 200.00000007 [delete] v87'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5613d8bae000 con 0x5613d99dbc00 2024-04-14T08:38:58.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.559+0000 7fbf141fd640 1 -- 172.21.15.172:0/2857109670 <== osd.4 v2:172.21.15.177:6820/4180733146 3 ==== osd_op_reply(10 200.00000006 [delete] v87'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5613d8bae000 con 0x5613d8ebc400 2024-04-14T08:38:58.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.560+0000 7fbf139fc640 1 -- 172.21.15.172:0/2857109670 <== osd.5 v2:172.21.15.177:6816/3172735715 1 ==== osd_op_reply(7 200.00000003 [delete] v87'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5613d8bae780 con 0x5613d8ebd000 2024-04-14T08:38:58.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.560+0000 7fbf091e7640 1 -- 172.21.15.172:0/2857109670 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87) -- 0x5613d8eca000 con 0x5613d8ebc400 2024-04-14T08:38:58.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf141fd640 1 -- 172.21.15.172:0/2857109670 <== osd.4 v2:172.21.15.177:6820/4180733146 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v87'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5613d8bae000 con 0x5613d8ebc400 2024-04-14T08:38:58.571 INFO:teuthology.orchestra.run.smithi172.stdout:done 2024-04-14T08:38:58.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5613d8ea5400 msgr2=0x5613d8eb0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5613d8ea5400 0x5613d8eb0580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5613d8ebdc00 msgr2=0x5613d8eb1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5613d8ebdc00 0x5613d8eb1080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5613d8ebc400 msgr2=0x5613d8eb1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5613d8ebc400 0x5613d8eb1b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x5613d8ebd000 msgr2=0x5613d8eb0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x5613d8ebd000 0x5613d8eb0b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5613d99db800 msgr2=0x5613d8eb0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5613d99db800 0x5613d8eb0000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf131fb640 1 -- 172.21.15.172:0/2857109670 reap_dead start 2024-04-14T08:38:58.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5613d99dbc00 msgr2=0x5613d8eb1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5613d99dbc00 0x5613d8eb1600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf131fb640 1 -- 172.21.15.172:0/2857109670 reap_dead start 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99db000 msgr2=0x5613d9875180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99db000 0x5613d9875180 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x5613d8dfb590 tx=0x5613d8dd2ae0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 shutdown_connections 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5613d99dbc00 0x5613d8eb1600 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 --2- 172.21.15.172:0/2857109670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99db000 0x5613d9875180 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 >> 172.21.15.172:0/2857109670 conn(0x5613d99da800 msgr2=0x5613d8b99800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 shutdown_connections 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.561+0000 7fbf15493280 1 -- 172.21.15.172:0/2857109670 wait complete. 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 -- 172.21.15.172:0/1969605401 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5613d9a08400 msgr2=0x5613d9874680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5613d9a08400 0x5613d9874680 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x5613d8bb0990 tx=0x5613d8dd2a80 comp rx=0 tx=0).stop 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 -- 172.21.15.172:0/1969605401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99da400 msgr2=0x5613d9874100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99da400 0x5613d9874100 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x5613d997e960 tx=0x5613d9840960 comp rx=0 tx=0).stop 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 -- 172.21.15.172:0/1969605401 shutdown_connections 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5613d9a08400 0x5613d9874680 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d99da400 0x5613d9874100 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d99da000 0x5613d9873b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 --2- 172.21.15.172:0/1969605401 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d988f800 0x5613d9873600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 -- 172.21.15.172:0/1969605401 >> 172.21.15.172:0/1969605401 conn(0x5613d988fc00 msgr2=0x5613d8b99000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 -- 172.21.15.172:0/1969605401 shutdown_connections 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 -- 172.21.15.172:0/1969605401 wait complete. 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 -- 172.21.15.172:0/415619352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988f400 msgr2=0x5613d9872000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 --2- 172.21.15.172:0/415619352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988f400 0x5613d9872000 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x5613d98d74a0 tx=0x5613d9842060 comp rx=0 tx=0).stop 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 -- 172.21.15.172:0/415619352 shutdown_connections 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 --2- 172.21.15.172:0/415619352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d988f400 0x5613d9872000 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 --2- 172.21.15.172:0/415619352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613d988f000 0x5613d9872580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 --2- 172.21.15.172:0/415619352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5613d988e400 0x5613d9872b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 -- 172.21.15.172:0/415619352 >> 172.21.15.172:0/415619352 conn(0x5613d8aad000 msgr2=0x5613d8b98000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:58.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 -- 172.21.15.172:0/415619352 shutdown_connections 2024-04-14T08:38:58.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.562+0000 7fbf15493280 1 -- 172.21.15.172:0/415619352 wait complete. 2024-04-14T08:38:58.577 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.116298, result: old journal was 4212724~2605 new journal start will be 8388608 (4173279 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-14T08:38:58.577 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-14T08:38:58.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.717+0000 7fea8e436640 1 Processor -- start 2024-04-14T08:38:58.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.717+0000 7fea8e436640 1 -- start start 2024-04-14T08:38:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea8e436640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fea880afa60 0x7fea8815d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea8e436640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fea880cd590 0x7fea88141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea8e436640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea8815d670 con 0x7fea880cd100 2024-04-14T08:38:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea8e436640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fea880d04c0 con 0x7fea880cd590 2024-04-14T08:38:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea8e436640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fea880cda00 con 0x7fea880afa60 2024-04-14T08:38:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea8c9d3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fea880cd590 0x7fea88141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea8c9d3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fea880cd590 0x7fea88141fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38212/0 (socket says 172.21.15.172:38212) 2024-04-14T08:38:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea8c9d3640 1 -- 172.21.15.172:0/354496752 learned_addr learned my addr 172.21.15.172:0/354496752 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea87fff640 1 --2- 172.21.15.172:0/354496752 >> v2:172.21.15.177:3300/0 conn(0x7fea880afa60 0x7fea8815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea8c9d3640 1 -- 172.21.15.172:0/354496752 >> v2:172.21.15.177:3300/0 conn(0x7fea880afa60 msgr2=0x7fea8815d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea8c9d3640 1 --2- 172.21.15.172:0/354496752 >> v2:172.21.15.177:3300/0 conn(0x7fea880afa60 0x7fea8815d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea8c9d3640 1 -- 172.21.15.172:0/354496752 >> v1:172.21.15.177:6789/0 conn(0x7fea880cd100 legacy=0x7fea881418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:58.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.718+0000 7fea8c9d3640 1 -- 172.21.15.172:0/354496752 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8815ffd0 con 0x7fea880cd590 2024-04-14T08:38:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.719+0000 7fea8c9d3640 1 --2- 172.21.15.172:0/354496752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea880cd590 0x7fea88141fc0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fea880b5a70 tx=0x7fea8002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a725e5cab58dc4ad server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.719+0000 7fea86ffd640 1 -- 172.21.15.172:0/354496752 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8002c9f0 con 0x7fea880cd590 2024-04-14T08:38:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.719+0000 7fea86ffd640 1 -- 172.21.15.172:0/354496752 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8002cb90 con 0x7fea880cd590 2024-04-14T08:38:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.719+0000 7fea86ffd640 1 -- 172.21.15.172:0/354496752 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800343f0 con 0x7fea880cd590 2024-04-14T08:38:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.719+0000 7fea8e436640 1 -- 172.21.15.172:0/354496752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea880cd590 msgr2=0x7fea88141fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.719+0000 7fea8e436640 1 --2- 172.21.15.172:0/354496752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea880cd590 0x7fea88141fc0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fea880b5a70 tx=0x7fea8002be50 comp rx=0 tx=0).stop 2024-04-14T08:38:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.719+0000 7fea8e436640 1 -- 172.21.15.172:0/354496752 shutdown_connections 2024-04-14T08:38:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.719+0000 7fea8e436640 1 --2- 172.21.15.172:0/354496752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea880cd590 0x7fea88141fc0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.719+0000 7fea8e436640 1 --2- 172.21.15.172:0/354496752 >> v2:172.21.15.177:3300/0 conn(0x7fea880afa60 0x7fea8815d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.719+0000 7fea8e436640 1 -- 172.21.15.172:0/354496752 >> 172.21.15.172:0/354496752 conn(0x7fea880b29b0 msgr2=0x7fea880d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.720+0000 7fea8e436640 1 -- 172.21.15.172:0/354496752 shutdown_connections 2024-04-14T08:38:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.720+0000 7fea8e436640 1 -- 172.21.15.172:0/354496752 wait complete. 2024-04-14T08:38:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.721+0000 7fea8e436640 1 Processor -- start 2024-04-14T08:38:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.721+0000 7fea8e436640 1 -- start start 2024-04-14T08:38:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.721+0000 7fea8e436640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea880afa60 0x7fea880b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.721+0000 7fea8e436640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea880cd100 0x7fea8813fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.721+0000 7fea8e436640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea880cd590 0x7fea88141320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.721+0000 7fea8e436640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea88160870 con 0x7fea880cd590 2024-04-14T08:38:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.721+0000 7fea8e436640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fea88160590 con 0x7fea880cd100 2024-04-14T08:38:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.721+0000 7fea8e436640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fea88160b50 con 0x7fea880afa60 2024-04-14T08:38:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea8c9d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea880cd590 0x7fea88141320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea8c9d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea880cd590 0x7fea88141320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38214/0 (socket says 172.21.15.172:38214) 2024-04-14T08:38:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea8c9d3640 1 -- 172.21.15.172:0/1808502 learned_addr learned my addr 172.21.15.172:0/1808502 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea87fff640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea880afa60 0x7fea880b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea877fe640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea880cd100 0x7fea8813fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea8c9d3640 1 -- 172.21.15.172:0/1808502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea880afa60 msgr2=0x7fea880b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea8c9d3640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea880afa60 0x7fea880b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea8c9d3640 1 -- 172.21.15.172:0/1808502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea880cd100 msgr2=0x7fea8813fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea8c9d3640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea880cd100 0x7fea8813fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea8c9d3640 1 -- 172.21.15.172:0/1808502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8813a8c0 con 0x7fea880cd590 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea877fe640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea880cd100 0x7fea8813fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea8c9d3640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea880cd590 0x7fea88141320 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fea80036040 tx=0x7fea80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea84ff9640 1 -- 172.21.15.172:0/1808502 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80043020 con 0x7fea880cd590 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea84ff9640 1 -- 172.21.15.172:0/1808502 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8003e450 con 0x7fea880cd590 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea8e436640 1 -- 172.21.15.172:0/1808502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea88140280 con 0x7fea880cd590 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea84ff9640 1 -- 172.21.15.172:0/1808502 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8003e860 con 0x7fea880cd590 2024-04-14T08:38:58.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.722+0000 7fea8e436640 1 -- 172.21.15.172:0/1808502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea88140840 con 0x7fea880cd590 2024-04-14T08:38:58.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.723+0000 7fea8e436640 1 -- 172.21.15.172:0/1808502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea88068520 con 0x7fea880cd590 2024-04-14T08:38:58.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.724+0000 7fea84ff9640 1 -- 172.21.15.172:0/1808502 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea80046020 con 0x7fea880cd590 2024-04-14T08:38:58.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.724+0000 7fea84ff9640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fea7406c8f0 0x7fea7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:58.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.724+0000 7fea84ff9640 1 -- 172.21.15.172:0/1808502 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7fea800ae610 con 0x7fea880cd590 2024-04-14T08:38:58.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.724+0000 7fea87fff640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fea7406c8f0 0x7fea7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:58.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.725+0000 7fea87fff640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fea7406c8f0 0x7fea7406edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea78006fd0 tx=0x7fea78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.728+0000 7fea84ff9640 1 -- 172.21.15.172:0/1808502 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fea8004d030 con 0x7fea880cd590 2024-04-14T08:38:58.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:58.943+0000 7fea8e436640 1 -- 172.21.15.172:0/1808502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fea880d0ed0 con 0x7fea880cd590 2024-04-14T08:38:59.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.396+0000 7fea84ff9640 1 -- 172.21.15.172:0/1808502 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v56) ==== 162+0+0 (secure 0 0 0) 0x7fea8815d670 con 0x7fea880cd590 2024-04-14T08:38:59.398 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-14T08:38:59.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.399+0000 7fea8e436640 1 -- 172.21.15.172:0/1808502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fea7406c8f0 msgr2=0x7fea7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:59.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.399+0000 7fea8e436640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fea7406c8f0 0x7fea7406edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea78006fd0 tx=0x7fea78008040 comp rx=0 tx=0).stop 2024-04-14T08:38:59.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.399+0000 7fea8e436640 1 -- 172.21.15.172:0/1808502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea880cd590 msgr2=0x7fea88141320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:59.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.399+0000 7fea8e436640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea880cd590 0x7fea88141320 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fea80036040 tx=0x7fea80004a40 comp rx=0 tx=0).stop 2024-04-14T08:38:59.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.399+0000 7fea8e436640 1 -- 172.21.15.172:0/1808502 shutdown_connections 2024-04-14T08:38:59.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.399+0000 7fea8e436640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fea7406c8f0 0x7fea7406edb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.399+0000 7fea8e436640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea880cd590 0x7fea88141320 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.399+0000 7fea8e436640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea880cd100 0x7fea8813fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.399+0000 7fea8e436640 1 --2- 172.21.15.172:0/1808502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea880afa60 0x7fea880b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.399+0000 7fea8e436640 1 -- 172.21.15.172:0/1808502 >> 172.21.15.172:0/1808502 conn(0x7fea880b29b0 msgr2=0x7fea8806c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:59.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.399+0000 7fea8e436640 1 -- 172.21.15.172:0/1808502 shutdown_connections 2024-04-14T08:38:59.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.399+0000 7fea8e436640 1 -- 172.21.15.172:0/1808502 wait complete. 2024-04-14T08:38:59.413 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:38:59.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.512+0000 7f9a63891640 1 Processor -- start 2024-04-14T08:38:59.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.512+0000 7f9a63891640 1 -- start start 2024-04-14T08:38:59.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.513+0000 7f9a63891640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a5c06b520 0x7f9a5c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:59.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.513+0000 7f9a63891640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9a5c0b5b50 0x7f9a5c15c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:59.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.513+0000 7f9a63891640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a5c06be40 con 0x7f9a5c15c990 2024-04-14T08:38:59.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.513+0000 7f9a63891640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f9a5c0ae650 con 0x7f9a5c0b5b50 2024-04-14T08:38:59.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.513+0000 7f9a63891640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5c0addb0 con 0x7f9a5c06b520 2024-04-14T08:38:59.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.513+0000 7f9a6162d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a5c06b520 0x7f9a5c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:59.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.513+0000 7f9a60e2c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9a5c0b5b50 0x7f9a5c15c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:59.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.513+0000 7f9a6162d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a5c06b520 0x7f9a5c06b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:47984/0 (socket says 172.21.15.172:47984) 2024-04-14T08:38:59.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.513+0000 7f9a6162d640 1 -- 172.21.15.172:0/1001285582 learned_addr learned my addr 172.21.15.172:0/1001285582 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:59.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.514+0000 7f9a6162d640 1 -- 172.21.15.172:0/1001285582 >> v2:172.21.15.177:3300/0 conn(0x7f9a5c0b5b50 msgr2=0x7f9a5c15c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:38:59.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.514+0000 7f9a6162d640 1 --2- 172.21.15.172:0/1001285582 >> v2:172.21.15.177:3300/0 conn(0x7f9a5c0b5b50 0x7f9a5c15c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.514+0000 7f9a6162d640 1 -- 172.21.15.172:0/1001285582 >> v1:172.21.15.177:6789/0 conn(0x7f9a5c15c990 legacy=0x7f9a5c0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:59.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.514+0000 7f9a6162d640 1 -- 172.21.15.172:0/1001285582 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5c0ae090 con 0x7f9a5c06b520 2024-04-14T08:38:59.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.514+0000 7f9a60e2c640 1 --2- 172.21.15.172:0/1001285582 >> v2:172.21.15.177:3300/0 conn(0x7f9a5c0b5b50 0x7f9a5c15c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:38:59.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.515+0000 7f9a6162d640 1 --2- 172.21.15.172:0/1001285582 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a5c06b520 0x7f9a5c06b900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009870 tx=0x7f9a4c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6973494379d26c6c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:59.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.515+0000 7f9a53fff640 1 -- 172.21.15.172:0/1001285582 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c034070 con 0x7f9a5c06b520 2024-04-14T08:38:59.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.515+0000 7f9a53fff640 1 -- 172.21.15.172:0/1001285582 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a4c030840 con 0x7f9a5c06b520 2024-04-14T08:38:59.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.515+0000 7f9a53fff640 1 -- 172.21.15.172:0/1001285582 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c030c10 con 0x7f9a5c06b520 2024-04-14T08:38:59.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.515+0000 7f9a63891640 1 -- 172.21.15.172:0/1001285582 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a5c06b520 msgr2=0x7f9a5c06b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:59.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.515+0000 7f9a63891640 1 --2- 172.21.15.172:0/1001285582 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a5c06b520 0x7f9a5c06b900 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009870 tx=0x7f9a4c02b980 comp rx=0 tx=0).stop 2024-04-14T08:38:59.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.515+0000 7f9a63891640 1 -- 172.21.15.172:0/1001285582 shutdown_connections 2024-04-14T08:38:59.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.515+0000 7f9a63891640 1 --2- 172.21.15.172:0/1001285582 >> v2:172.21.15.177:3300/0 conn(0x7f9a5c0b5b50 0x7f9a5c15c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.515+0000 7f9a63891640 1 --2- 172.21.15.172:0/1001285582 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a5c06b520 0x7f9a5c06b900 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.515+0000 7f9a63891640 1 -- 172.21.15.172:0/1001285582 >> 172.21.15.172:0/1001285582 conn(0x7f9a5c0d0f20 msgr2=0x7f9a5c0d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:59.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.515+0000 7f9a63891640 1 -- 172.21.15.172:0/1001285582 shutdown_connections 2024-04-14T08:38:59.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.515+0000 7f9a63891640 1 -- 172.21.15.172:0/1001285582 wait complete. 2024-04-14T08:38:59.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a63891640 1 Processor -- start 2024-04-14T08:38:59.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a63891640 1 -- start start 2024-04-14T08:38:59.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a63891640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a5c06b520 0x7f9a5c15b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a63891640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a5c0b5b50 0x7f9a5c146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a63891640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a5c151ed0 0x7f9a5c155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a63891640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c09eda0 con 0x7f9a5c06b520 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a63891640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c0b3f70 con 0x7f9a5c0b5b50 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a63891640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5c09ef20 con 0x7f9a5c151ed0 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a6162d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a5c06b520 0x7f9a5c15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a6162d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a5c06b520 0x7f9a5c15b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38228/0 (socket says 172.21.15.172:38228) 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a6162d640 1 -- 172.21.15.172:0/1040869916 learned_addr learned my addr 172.21.15.172:0/1040869916 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a61e2e640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a5c151ed0 0x7f9a5c155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.517+0000 7f9a60e2c640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a5c0b5b50 0x7f9a5c146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.518+0000 7f9a6162d640 1 -- 172.21.15.172:0/1040869916 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a5c151ed0 msgr2=0x7f9a5c155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.518+0000 7f9a6162d640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a5c151ed0 0x7f9a5c155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.518+0000 7f9a6162d640 1 -- 172.21.15.172:0/1040869916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a5c0b5b50 msgr2=0x7f9a5c146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:59.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.518+0000 7f9a6162d640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a5c0b5b50 0x7f9a5c146a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.518+0000 7f9a6162d640 1 -- 172.21.15.172:0/1040869916 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5c1523d0 con 0x7f9a5c06b520 2024-04-14T08:38:59.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.518+0000 7f9a6162d640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a5c06b520 0x7f9a5c15b730 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c000c00 tx=0x7f9a4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:59.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.518+0000 7f9a51ffb640 1 -- 172.21.15.172:0/1040869916 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c034040 con 0x7f9a5c06b520 2024-04-14T08:38:59.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.518+0000 7f9a51ffb640 1 -- 172.21.15.172:0/1040869916 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a4c039d90 con 0x7f9a5c06b520 2024-04-14T08:38:59.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.518+0000 7f9a63891640 1 -- 172.21.15.172:0/1040869916 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5c1526b0 con 0x7f9a5c06b520 2024-04-14T08:38:59.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.518+0000 7f9a51ffb640 1 -- 172.21.15.172:0/1040869916 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c043630 con 0x7f9a5c06b520 2024-04-14T08:38:59.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.518+0000 7f9a63891640 1 -- 172.21.15.172:0/1040869916 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5c152c70 con 0x7f9a5c06b520 2024-04-14T08:38:59.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.519+0000 7f9a63891640 1 -- 172.21.15.172:0/1040869916 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a5c068520 con 0x7f9a5c06b520 2024-04-14T08:38:59.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.520+0000 7f9a51ffb640 1 -- 172.21.15.172:0/1040869916 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a4c04a060 con 0x7f9a5c06b520 2024-04-14T08:38:59.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.520+0000 7f9a51ffb640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a4006c8f0 0x7f9a4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:38:59.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.520+0000 7f9a51ffb640 1 -- 172.21.15.172:0/1040869916 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f9a4c0adb60 con 0x7f9a5c06b520 2024-04-14T08:38:59.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.520+0000 7f9a60e2c640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a4006c8f0 0x7f9a4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:38:59.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.520+0000 7f9a60e2c640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a4006c8f0 0x7f9a4006edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009560 tx=0x7f9a58009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:38:59.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.525+0000 7f9a51ffb640 1 -- 172.21.15.172:0/1040869916 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9a4c03e070 con 0x7f9a5c06b520 2024-04-14T08:38:59.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.736+0000 7f9a63891640 1 -- 172.21.15.172:0/1040869916 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a5c15f7f0 con 0x7f9a5c06b520 2024-04-14T08:38:59.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.737+0000 7f9a51ffb640 1 -- 172.21.15.172:0/1040869916 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5189 (secure 0 0 0) 0x7f9a4c07b770 con 0x7f9a5c06b520 2024-04-14T08:38:59.739 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:38:59.739 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:38:59.400300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7473":{"gid":7473,"name":"c","rank":0,"incarnation":57,"state":"up:replay","state_seq":1,"addr":"172.21.15.172:6837/1386685680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1386685680},{"type":"v1","addr":"172.21.15.172:6837","nonce":1386685680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-14T08:38:59.739 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 57 2024-04-14T08:38:59.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.739+0000 7f9a63891640 1 -- 172.21.15.172:0/1040869916 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a4006c8f0 msgr2=0x7f9a4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:59.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.739+0000 7f9a63891640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a4006c8f0 0x7f9a4006edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009560 tx=0x7f9a58009190 comp rx=0 tx=0).stop 2024-04-14T08:38:59.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.739+0000 7f9a63891640 1 -- 172.21.15.172:0/1040869916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a5c06b520 msgr2=0x7f9a5c15b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:38:59.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.739+0000 7f9a63891640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a5c06b520 0x7f9a5c15b730 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c000c00 tx=0x7f9a4c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.739+0000 7f9a63891640 1 -- 172.21.15.172:0/1040869916 shutdown_connections 2024-04-14T08:38:59.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.739+0000 7f9a63891640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a4006c8f0 0x7f9a4006edb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.739+0000 7f9a63891640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a5c151ed0 0x7f9a5c155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.739+0000 7f9a63891640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a5c0b5b50 0x7f9a5c146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.739+0000 7f9a63891640 1 --2- 172.21.15.172:0/1040869916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a5c06b520 0x7f9a5c15b730 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:38:59.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.739+0000 7f9a63891640 1 -- 172.21.15.172:0/1040869916 >> 172.21.15.172:0/1040869916 conn(0x7f9a5c0d0f20 msgr2=0x7f9a5c0afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:38:59.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.739+0000 7f9a63891640 1 -- 172.21.15.172:0/1040869916 shutdown_connections 2024-04-14T08:38:59.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:38:59.739+0000 7f9a63891640 1 -- 172.21.15.172:0/1040869916 wait complete. 2024-04-14T08:39:00.154 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:38:44.572424+0000', 'modified': '2024-04-14T08:38:59.400300+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7473': {'gid': 7473, 'name': 'c', 'rank': 0, 'incarnation': 57, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.172:6837/1386685680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 1386685680}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 1386685680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-14T08:39:00.154 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7473:up:replay 2024-04-14T08:39:01.156 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:39:01.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.270+0000 7fe8ad3c8640 1 Processor -- start 2024-04-14T08:39:01.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.270+0000 7fe8ad3c8640 1 -- start start 2024-04-14T08:39:01.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.271+0000 7fe8ad3c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe8a815c150 0x7fe8a80d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:01.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.271+0000 7fe8ad3c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe8a8162700 0x7fe8a80d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:01.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.271+0000 7fe8ad3c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8a80d0200 con 0x7fe8a80d1840 2024-04-14T08:39:01.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.271+0000 7fe8ad3c8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe8a80d1cb0 con 0x7fe8a8162700 2024-04-14T08:39:01.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.271+0000 7fe8ad3c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe8a80cef10 con 0x7fe8a815c150 2024-04-14T08:39:01.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.271+0000 7fe8a77fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe8a8162700 0x7fe8a80d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:01.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.271+0000 7fe8a77fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe8a8162700 0x7fe8a80d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58976/0 (socket says 172.21.15.172:58976) 2024-04-14T08:39:01.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.271+0000 7fe8a77fe640 1 -- 172.21.15.172:0/1332563747 learned_addr learned my addr 172.21.15.172:0/1332563747 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:01.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.272+0000 7fe8a6ffd640 1 --2- 172.21.15.172:0/1332563747 >> v2:172.21.15.177:3300/0 conn(0x7fe8a815c150 0x7fe8a80d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:01.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.272+0000 7fe8a77fe640 1 -- 172.21.15.172:0/1332563747 >> v2:172.21.15.177:3300/0 conn(0x7fe8a815c150 msgr2=0x7fe8a80d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:01.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.272+0000 7fe8a77fe640 1 --2- 172.21.15.172:0/1332563747 >> v2:172.21.15.177:3300/0 conn(0x7fe8a815c150 0x7fe8a80d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:01.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.272+0000 7fe8a77fe640 1 -- 172.21.15.172:0/1332563747 >> v1:172.21.15.177:6789/0 conn(0x7fe8a80d1840 legacy=0x7fe8a8161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:01.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.272+0000 7fe8a77fe640 1 -- 172.21.15.172:0/1332563747 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a80cf160 con 0x7fe8a8162700 2024-04-14T08:39:01.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.272+0000 7fe8a77fe640 1 --2- 172.21.15.172:0/1332563747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8a8162700 0x7fe8a80d1e80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe8a806bf30 tx=0x7fe8a002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4d1fefc5a68e40 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:01.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.273+0000 7fe8a67fc640 1 -- 172.21.15.172:0/1332563747 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a002cb20 con 0x7fe8a8162700 2024-04-14T08:39:01.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.273+0000 7fe8a67fc640 1 -- 172.21.15.172:0/1332563747 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a002ccc0 con 0x7fe8a8162700 2024-04-14T08:39:01.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.273+0000 7fe8a67fc640 1 -- 172.21.15.172:0/1332563747 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0034610 con 0x7fe8a8162700 2024-04-14T08:39:01.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.273+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/1332563747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8a8162700 msgr2=0x7fe8a80d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:01.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.273+0000 7fe8ad3c8640 1 --2- 172.21.15.172:0/1332563747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8a8162700 0x7fe8a80d1e80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe8a806bf30 tx=0x7fe8a002bf80 comp rx=0 tx=0).stop 2024-04-14T08:39:01.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.273+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/1332563747 shutdown_connections 2024-04-14T08:39:01.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.273+0000 7fe8ad3c8640 1 --2- 172.21.15.172:0/1332563747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8a8162700 0x7fe8a80d1e80 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:01.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.273+0000 7fe8ad3c8640 1 --2- 172.21.15.172:0/1332563747 >> v2:172.21.15.177:3300/0 conn(0x7fe8a815c150 0x7fe8a80d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:01.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.273+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/1332563747 >> 172.21.15.172:0/1332563747 conn(0x7fe8a809d580 msgr2=0x7fe8a80d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:01.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.273+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/1332563747 shutdown_connections 2024-04-14T08:39:01.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.273+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/1332563747 wait complete. 2024-04-14T08:39:01.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.274+0000 7fe8ad3c8640 1 Processor -- start 2024-04-14T08:39:01.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.275+0000 7fe8ad3c8640 1 -- start start 2024-04-14T08:39:01.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.275+0000 7fe8ad3c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8a80d1840 0x7fe8a815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:01.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.275+0000 7fe8ad3c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8a815c150 0x7fe8a80b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:01.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.275+0000 7fe8ad3c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8a8162700 0x7fe8a80ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:01.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.275+0000 7fe8ad3c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a80cfa00 con 0x7fe8a815c150 2024-04-14T08:39:01.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.275+0000 7fe8ad3c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a80cf720 con 0x7fe8a8162700 2024-04-14T08:39:01.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.275+0000 7fe8ad3c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a80cfce0 con 0x7fe8a80d1840 2024-04-14T08:39:01.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.275+0000 7fe89ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8a815c150 0x7fe8a80b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:01.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.275+0000 7fe89ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8a815c150 0x7fe8a80b93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58980/0 (socket says 172.21.15.172:58980) 2024-04-14T08:39:01.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.275+0000 7fe89ffff640 1 -- 172.21.15.172:0/3954167067 learned_addr learned my addr 172.21.15.172:0/3954167067 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:01.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.275+0000 7fe8a77fe640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8a8162700 0x7fe8a80ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:01.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.275+0000 7fe8a6ffd640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8a80d1840 0x7fe8a815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:01.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.276+0000 7fe89ffff640 1 -- 172.21.15.172:0/3954167067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8a80d1840 msgr2=0x7fe8a815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:01.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.276+0000 7fe89ffff640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8a80d1840 0x7fe8a815cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:01.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.276+0000 7fe89ffff640 1 -- 172.21.15.172:0/3954167067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8a8162700 msgr2=0x7fe8a80ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:01.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.276+0000 7fe89ffff640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8a8162700 0x7fe8a80ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:01.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.276+0000 7fe89ffff640 1 -- 172.21.15.172:0/3954167067 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a8151730 con 0x7fe8a815c150 2024-04-14T08:39:01.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.276+0000 7fe89ffff640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8a815c150 0x7fe8a80b93a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe89000c430 tx=0x7fe89000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:01.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.276+0000 7fe89f7fe640 1 -- 172.21.15.172:0/3954167067 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890017070 con 0x7fe8a815c150 2024-04-14T08:39:01.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.276+0000 7fe89f7fe640 1 -- 172.21.15.172:0/3954167067 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8900052d0 con 0x7fe8a815c150 2024-04-14T08:39:01.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.276+0000 7fe89f7fe640 1 -- 172.21.15.172:0/3954167067 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890003850 con 0x7fe8a815c150 2024-04-14T08:39:01.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.276+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/3954167067 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a80cb420 con 0x7fe8a815c150 2024-04-14T08:39:01.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.276+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/3954167067 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a80cb9e0 con 0x7fe8a815c150 2024-04-14T08:39:01.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.277+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/3954167067 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a814af40 con 0x7fe8a815c150 2024-04-14T08:39:01.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.278+0000 7fe89f7fe640 1 -- 172.21.15.172:0/3954167067 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe89001a070 con 0x7fe8a815c150 2024-04-14T08:39:01.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.278+0000 7fe89f7fe640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe88806c920 0x7fe88806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:01.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.278+0000 7fe89f7fe640 1 -- 172.21.15.172:0/3954167067 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7fe89008af80 con 0x7fe8a815c150 2024-04-14T08:39:01.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.278+0000 7fe8a6ffd640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe88806c920 0x7fe88806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:01.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.278+0000 7fe8a6ffd640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe88806c920 0x7fe88806ede0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe898006fd0 tx=0x7fe898008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:01.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.282+0000 7fe89f7fe640 1 -- 172.21.15.172:0/3954167067 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe8a814af40 con 0x7fe8a815c150 2024-04-14T08:39:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.496+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/3954167067 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8a8075490 con 0x7fe8a815c150 2024-04-14T08:39:01.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.496+0000 7fe89f7fe640 1 -- 172.21.15.172:0/3954167067 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5189 (secure 0 0 0) 0x7fe890058c10 con 0x7fe8a815c150 2024-04-14T08:39:01.498 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:39:01.498 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:39:00.407488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7473":{"gid":7473,"name":"c","rank":0,"incarnation":57,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.172:6837/1386685680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1386685680},{"type":"v1","addr":"172.21.15.172:6837","nonce":1386685680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-14T08:39:01.498 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 59 2024-04-14T08:39:01.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.498+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/3954167067 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe88806c920 msgr2=0x7fe88806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:01.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.498+0000 7fe8ad3c8640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe88806c920 0x7fe88806ede0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe898006fd0 tx=0x7fe898008040 comp rx=0 tx=0).stop 2024-04-14T08:39:01.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.498+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/3954167067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8a815c150 msgr2=0x7fe8a80b93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:01.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.498+0000 7fe8ad3c8640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8a815c150 0x7fe8a80b93a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe89000c430 tx=0x7fe89000c8f0 comp rx=0 tx=0).stop 2024-04-14T08:39:01.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.498+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/3954167067 shutdown_connections 2024-04-14T08:39:01.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.498+0000 7fe8ad3c8640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe88806c920 0x7fe88806ede0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:01.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.498+0000 7fe8ad3c8640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8a8162700 0x7fe8a80ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:01.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.498+0000 7fe8ad3c8640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8a815c150 0x7fe8a80b93a0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:01.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.498+0000 7fe8ad3c8640 1 --2- 172.21.15.172:0/3954167067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8a80d1840 0x7fe8a815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:01.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.498+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/3954167067 >> 172.21.15.172:0/3954167067 conn(0x7fe8a809d580 msgr2=0x7fe8a80b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:01.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.498+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/3954167067 shutdown_connections 2024-04-14T08:39:01.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:01.498+0000 7fe8ad3c8640 1 -- 172.21.15.172:0/3954167067 wait complete. 2024-04-14T08:39:01.914 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:38:44.572424+0000', 'modified': '2024-04-14T08:39:00.407488+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7473': {'gid': 7473, 'name': 'c', 'rank': 0, 'incarnation': 57, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.172:6837/1386685680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 1386685680}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 1386685680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-14T08:39:01.914 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7473:up:rejoin 2024-04-14T08:39:02.918 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:39:03.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.034+0000 7f0863e1c640 1 Processor -- start 2024-04-14T08:39:03.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.034+0000 7f0863e1c640 1 -- start start 2024-04-14T08:39:03.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.035+0000 7f0863e1c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f085c072470 0x7f085c15b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.035+0000 7f0863e1c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f085c072970 0x7f085c15b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.035+0000 7f0863e1c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f085c0b3360 con 0x7f085c15a690 2024-04-14T08:39:03.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.035+0000 7f0863e1c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f085c0b3640 con 0x7f085c072970 2024-04-14T08:39:03.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.035+0000 7f0863e1c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f085c0b3920 con 0x7f085c072470 2024-04-14T08:39:03.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.035+0000 7f08623b9640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f085c072970 0x7f085c15b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.036+0000 7f08623b9640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f085c072970 0x7f085c15b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58990/0 (socket says 172.21.15.172:58990) 2024-04-14T08:39:03.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.036+0000 7f08623b9640 1 -- 172.21.15.172:0/385423326 learned_addr learned my addr 172.21.15.172:0/385423326 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:03.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.036+0000 7f08613b7640 1 --2- 172.21.15.172:0/385423326 >> v2:172.21.15.177:3300/0 conn(0x7f085c072470 0x7f085c15b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.036+0000 7f08623b9640 1 -- 172.21.15.172:0/385423326 >> v2:172.21.15.177:3300/0 conn(0x7f085c072470 msgr2=0x7f085c15b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.036+0000 7f08623b9640 1 --2- 172.21.15.172:0/385423326 >> v2:172.21.15.177:3300/0 conn(0x7f085c072470 0x7f085c15b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.036+0000 7f08623b9640 1 -- 172.21.15.172:0/385423326 >> v1:172.21.15.177:6789/0 conn(0x7f085c15a690 legacy=0x7f085c071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.036+0000 7f08623b9640 1 -- 172.21.15.172:0/385423326 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c0b3c00 con 0x7f085c072970 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.036+0000 7f08613b7640 1 --2- 172.21.15.172:0/385423326 >> v2:172.21.15.177:3300/0 conn(0x7f085c072470 0x7f085c15b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.036+0000 7f08623b9640 1 --2- 172.21.15.172:0/385423326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f085c072970 0x7f085c15b770 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f085c0cc3a0 tx=0x7f085402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad151459d184aebf server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.037+0000 7f0860bb6640 1 -- 172.21.15.172:0/385423326 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085402c9f0 con 0x7f085c072970 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.037+0000 7f0860bb6640 1 -- 172.21.15.172:0/385423326 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085402cb90 con 0x7f085c072970 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.037+0000 7f0860bb6640 1 -- 172.21.15.172:0/385423326 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08540343f0 con 0x7f085c072970 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.037+0000 7f0863e1c640 1 -- 172.21.15.172:0/385423326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f085c072970 msgr2=0x7f085c15b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.037+0000 7f0863e1c640 1 --2- 172.21.15.172:0/385423326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f085c072970 0x7f085c15b770 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f085c0cc3a0 tx=0x7f085402be50 comp rx=0 tx=0).stop 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.037+0000 7f0863e1c640 1 -- 172.21.15.172:0/385423326 shutdown_connections 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.037+0000 7f0863e1c640 1 --2- 172.21.15.172:0/385423326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f085c072970 0x7f085c15b770 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.037+0000 7f0863e1c640 1 --2- 172.21.15.172:0/385423326 >> v2:172.21.15.177:3300/0 conn(0x7f085c072470 0x7f085c15b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.037+0000 7f0863e1c640 1 -- 172.21.15.172:0/385423326 >> 172.21.15.172:0/385423326 conn(0x7f085c06ec30 msgr2=0x7f085c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.037+0000 7f0863e1c640 1 -- 172.21.15.172:0/385423326 shutdown_connections 2024-04-14T08:39:03.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.037+0000 7f0863e1c640 1 -- 172.21.15.172:0/385423326 wait complete. 2024-04-14T08:39:03.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.039+0000 7f0863e1c640 1 Processor -- start 2024-04-14T08:39:03.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.039+0000 7f0863e1c640 1 -- start start 2024-04-14T08:39:03.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.040+0000 7f0863e1c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f085c072470 0x7f085c153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.040+0000 7f0863e1c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f085c072970 0x7f085c152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.040+0000 7f0863e1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f085c15a690 0x7f085c14e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.040+0000 7f0863e1c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f085c15aa70 con 0x7f085c15a690 2024-04-14T08:39:03.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.040+0000 7f0863e1c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f085c0b3100 con 0x7f085c072970 2024-04-14T08:39:03.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.040+0000 7f0863e1c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f085c15abf0 con 0x7f085c072470 2024-04-14T08:39:03.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.040+0000 7f08623b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f085c15a690 0x7f085c14e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.040+0000 7f08623b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f085c15a690 0x7f085c14e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58994/0 (socket says 172.21.15.172:58994) 2024-04-14T08:39:03.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f08623b9640 1 -- 172.21.15.172:0/1338448024 learned_addr learned my addr 172.21.15.172:0/1338448024 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f08613b7640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f085c072970 0x7f085c152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f0861bb8640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f085c072470 0x7f085c153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f08623b9640 1 -- 172.21.15.172:0/1338448024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f085c072470 msgr2=0x7f085c153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f08623b9640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f085c072470 0x7f085c153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f08623b9640 1 -- 172.21.15.172:0/1338448024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f085c072970 msgr2=0x7f085c152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f08623b9640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f085c072970 0x7f085c152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f08623b9640 1 -- 172.21.15.172:0/1338448024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c152ba0 con 0x7f085c15a690 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f08623b9640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f085c15a690 0x7f085c14e540 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f085402c460 tx=0x7f0854004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f08527fc640 1 -- 172.21.15.172:0/1338448024 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854043020 con 0x7f085c15a690 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f08527fc640 1 -- 172.21.15.172:0/1338448024 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085403e450 con 0x7f085c15a690 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f0863e1c640 1 -- 172.21.15.172:0/1338448024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c152e80 con 0x7f085c15a690 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f08527fc640 1 -- 172.21.15.172:0/1338448024 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854009510 con 0x7f085c15a690 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.041+0000 7f0863e1c640 1 -- 172.21.15.172:0/1338448024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c14f040 con 0x7f085c15a690 2024-04-14T08:39:03.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.042+0000 7f0863e1c640 1 -- 172.21.15.172:0/1338448024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f085c068520 con 0x7f085c15a690 2024-04-14T08:39:03.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.043+0000 7f08527fc640 1 -- 172.21.15.172:0/1338448024 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0854046020 con 0x7f085c15a690 2024-04-14T08:39:03.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.044+0000 7f08527fc640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f084006c8f0 0x7f084006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.044+0000 7f0861bb8640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f084006c8f0 0x7f084006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.044+0000 7f08527fc640 1 -- 172.21.15.172:0/1338448024 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f08540ae470 con 0x7f085c15a690 2024-04-14T08:39:03.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.044+0000 7f0861bb8640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f084006c8f0 0x7f084006edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0858006fd0 tx=0x7f0858009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.048+0000 7f08527fc640 1 -- 172.21.15.172:0/1338448024 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f085404d030 con 0x7f085c15a690 2024-04-14T08:39:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.265+0000 7f0863e1c640 1 -- 172.21.15.172:0/1338448024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f085c14f210 con 0x7f085c15a690 2024-04-14T08:39:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.265+0000 7f08527fc640 1 -- 172.21.15.172:0/1338448024 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5196 (secure 0 0 0) 0x7f085407c100 con 0x7f085c15a690 2024-04-14T08:39:03.267 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:39:03.267 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:39:02.413289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7473":{"gid":7473,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6837/1386685680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1386685680},{"type":"v1","addr":"172.21.15.172:6837","nonce":1386685680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7473,"qdb_cluster":[7473]},"id":6}]} 2024-04-14T08:39:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 60 2024-04-14T08:39:03.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.267+0000 7f0863e1c640 1 -- 172.21.15.172:0/1338448024 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f084006c8f0 msgr2=0x7f084006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.267+0000 7f0863e1c640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f084006c8f0 0x7f084006edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0858006fd0 tx=0x7f0858009060 comp rx=0 tx=0).stop 2024-04-14T08:39:03.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.267+0000 7f0863e1c640 1 -- 172.21.15.172:0/1338448024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f085c15a690 msgr2=0x7f085c14e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.267+0000 7f0863e1c640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f085c15a690 0x7f085c14e540 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f085402c460 tx=0x7f0854004a40 comp rx=0 tx=0).stop 2024-04-14T08:39:03.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.267+0000 7f0863e1c640 1 -- 172.21.15.172:0/1338448024 shutdown_connections 2024-04-14T08:39:03.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.267+0000 7f0863e1c640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f084006c8f0 0x7f084006edb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.267+0000 7f0863e1c640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f085c15a690 0x7f085c14e540 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.267+0000 7f0863e1c640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f085c072970 0x7f085c152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.267+0000 7f0863e1c640 1 --2- 172.21.15.172:0/1338448024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f085c072470 0x7f085c153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.267+0000 7f0863e1c640 1 -- 172.21.15.172:0/1338448024 >> 172.21.15.172:0/1338448024 conn(0x7f085c06ec30 msgr2=0x7f085c151940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:03.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.267+0000 7f0863e1c640 1 -- 172.21.15.172:0/1338448024 shutdown_connections 2024-04-14T08:39:03.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.267+0000 7f0863e1c640 1 -- 172.21.15.172:0/1338448024 wait complete. 2024-04-14T08:39:03.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:38:44.572424+0000', 'modified': '2024-04-14T08:39:02.413289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7473': {'gid': 7473, 'name': 'c', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.172:6837/1386685680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 1386685680}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 1386685680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7473, 'qdb_cluster': [7473]} 2024-04-14T08:39:03.687 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:39:03.687 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-14T08:39:03.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.789+0000 7f6f21daa640 1 Processor -- start 2024-04-14T08:39:03.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.789+0000 7f6f21daa640 1 -- start start 2024-04-14T08:39:03.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.789+0000 7f6f21daa640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6f1c162710 0x7f6f1c0d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.789+0000 7f6f21daa640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f1c0cefb0 con 0x7f6f1c162ba0 2024-04-14T08:39:03.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.789+0000 7f6f21daa640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f1c0cf290 con 0x7f6f1c15e430 2024-04-14T08:39:03.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.789+0000 7f6f21daa640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f6f1c0cf570 con 0x7f6f1c162710 2024-04-14T08:39:03.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6f1c162ba0 0x7f6f1c0d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43336/0 (socket says 172.21.15.172:43336) 2024-04-14T08:39:03.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1bfff640 1 -- 172.21.15.172:0/177815967 learned_addr learned my addr 172.21.15.172:0/177815967 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:39:03.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2386994877 0 0) 0x7f6f1c0cefb0 con 0x7f6f1c162ba0 2024-04-14T08:39:03.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f13fff640 1 --2- 172.21.15.172:0/177815967 >> v2:172.21.15.177:3300/0 conn(0x7f6f1c162710 0x7f6f1c0d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f08003520 con 0x7f6f1c162ba0 2024-04-14T08:39:03.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 283831936 0 0) 0x7f6f08003520 con 0x7f6f1c162ba0 2024-04-14T08:39:03.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 >> v2:172.21.15.177:3300/0 conn(0x7f6f1c162710 msgr2=0x7f6f1c0d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 --2- 172.21.15.172:0/177815967 >> v2:172.21.15.177:3300/0 conn(0x7f6f1c162710 0x7f6f1c0d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 >> v1:172.21.15.177:6789/0 conn(0x7f6f1c15e430 legacy=0x7f6f1c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c0cf850 con 0x7f6f1c162ba0 2024-04-14T08:39:03.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f6f140019f0 con 0x7f6f1c162ba0 2024-04-14T08:39:03.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 >> v1:172.21.15.172:6789/0 conn(0x7f6f1c162ba0 legacy=0x7f6f1c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 --2- 172.21.15.172:0/177815967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f080039b0 0x7f6f08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 --2- 172.21.15.172:0/177815967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f08004800 0x7f6f08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 --2- 172.21.15.172:0/177815967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f080051c0 0x7f6f0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c0cf570 con 0x7f6f08004800 2024-04-14T08:39:03.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0800c060 con 0x7f6f080039b0 2024-04-14T08:39:03.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0800c260 con 0x7f6f080051c0 2024-04-14T08:39:03.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f13fff640 1 --2- 172.21.15.172:0/177815967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f08004800 0x7f6f08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/177815967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f080039b0 0x7f6f08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.790+0000 7f6f1bfff640 1 --2- 172.21.15.172:0/177815967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f080051c0 0x7f6f0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.791+0000 7f6f13fff640 1 -- 172.21.15.172:0/177815967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f080051c0 msgr2=0x7f6f0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.791+0000 7f6f13fff640 1 --2- 172.21.15.172:0/177815967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f080051c0 0x7f6f0800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.791+0000 7f6f13fff640 1 -- 172.21.15.172:0/177815967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f080039b0 msgr2=0x7f6f08003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.791+0000 7f6f13fff640 1 --2- 172.21.15.172:0/177815967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f080039b0 0x7f6f08003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.791+0000 7f6f13fff640 1 -- 172.21.15.172:0/177815967 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f08003520 con 0x7f6f08004800 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.791+0000 7f6f1bfff640 1 -- 172.21.15.172:0/177815967 reap_dead start 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.791+0000 7f6f13fff640 1 --2- 172.21.15.172:0/177815967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f08004800 0x7f6f08004c80 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009ac0 tx=0x7f6f08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e905472913bff server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.791+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04039070 con 0x7f6f08004800 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.791+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f0402c6b0 con 0x7f6f08004800 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.791+0000 7f6f1affd640 1 -- 172.21.15.172:0/177815967 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04009690 con 0x7f6f08004800 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.792+0000 7f6f21daa640 1 -- 172.21.15.172:0/177815967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f08004800 msgr2=0x7f6f08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.792+0000 7f6f21daa640 1 --2- 172.21.15.172:0/177815967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f08004800 0x7f6f08004c80 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009ac0 tx=0x7f6f08005830 comp rx=0 tx=0).stop 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.792+0000 7f6f21daa640 1 -- 172.21.15.172:0/177815967 shutdown_connections 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.792+0000 7f6f21daa640 1 --2- 172.21.15.172:0/177815967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f08004800 0x7f6f08004c80 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.792+0000 7f6f21daa640 1 -- 172.21.15.172:0/177815967 >> 172.21.15.172:0/177815967 conn(0x7f6f1c09d560 msgr2=0x7f6f1c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.792+0000 7f6f21daa640 1 -- 172.21.15.172:0/177815967 shutdown_connections 2024-04-14T08:39:03.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.792+0000 7f6f21daa640 1 -- 172.21.15.172:0/177815967 wait complete. 2024-04-14T08:39:03.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.794+0000 7f6f21daa640 1 Processor -- start 2024-04-14T08:39:03.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.794+0000 7f6f21daa640 1 -- start start 2024-04-14T08:39:03.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f21daa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f08004800 0x7f6f1c1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f21daa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c0b5560 0x7f6f1c1e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f21daa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f1c0b8310 0x7f6f1c1586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f21daa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c0d4e20 con 0x7f6f1c0b5560 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f21daa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c15e810 con 0x7f6f1c0b8310 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f21daa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1c0d0050 con 0x7f6f08004800 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f13fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c0b5560 0x7f6f1c1e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f13fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c0b5560 0x7f6f1c1e98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59016/0 (socket says 172.21.15.172:59016) 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f13fff640 1 -- 172.21.15.172:0/2812688246 learned_addr learned my addr 172.21.15.172:0/2812688246 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f08004800 0x7f6f1c1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f1bfff640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f1c0b8310 0x7f6f1c1586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f13fff640 1 -- 172.21.15.172:0/2812688246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f08004800 msgr2=0x7f6f1c1592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f13fff640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f08004800 0x7f6f1c1592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f13fff640 1 -- 172.21.15.172:0/2812688246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f1c0b8310 msgr2=0x7f6f1c1586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f13fff640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f1c0b8310 0x7f6f1c1586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.795+0000 7f6f13fff640 1 -- 172.21.15.172:0/2812688246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c0b44c0 con 0x7f6f1c0b5560 2024-04-14T08:39:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.796+0000 7f6f13fff640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c0b5560 0x7f6f1c1e98b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6f04000c00 tx=0x7f6f04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.796+0000 7f6f18ff9640 1 -- 172.21.15.172:0/2812688246 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04042020 con 0x7f6f1c0b5560 2024-04-14T08:39:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.796+0000 7f6f18ff9640 1 -- 172.21.15.172:0/2812688246 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f0403a040 con 0x7f6f1c0b5560 2024-04-14T08:39:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.796+0000 7f6f21daa640 1 -- 172.21.15.172:0/2812688246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c0b47a0 con 0x7f6f1c0b5560 2024-04-14T08:39:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.796+0000 7f6f18ff9640 1 -- 172.21.15.172:0/2812688246 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04039070 con 0x7f6f1c0b5560 2024-04-14T08:39:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.796+0000 7f6f21daa640 1 -- 172.21.15.172:0/2812688246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c0b4d60 con 0x7f6f1c0b5560 2024-04-14T08:39:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.797+0000 7f6f18ff9640 1 -- 172.21.15.172:0/2812688246 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f04045020 con 0x7f6f1c0b5560 2024-04-14T08:39:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.798+0000 7f6f18ff9640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6efc06c8f0 0x7f6efc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.798+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6efc06c8f0 0x7f6efc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.798+0000 7f6f18ff9640 1 -- 172.21.15.172:0/2812688246 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f6f040ad1c0 con 0x7f6f1c0b5560 2024-04-14T08:39:03.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.798+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6efc06c8f0 0x7f6efc06edb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c000c00 tx=0x7f6f0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.799+0000 7f6f24b8d740 1 Processor -- start 2024-04-14T08:39:03.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.799+0000 7f6f24b8d740 1 -- start start 2024-04-14T08:39:03.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f24b8d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9db8b60 0x558be9d8bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f24b8d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9dbb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f24b8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9d8c040 0x558be9dbbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f24b8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558be9cf5030 con 0x558be9d8c040 2024-04-14T08:39:03.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f24b8d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558be9c9d620 con 0x558be9db8f40 2024-04-14T08:39:03.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f24b8d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x558be9cf0fe0 con 0x558be9db8b60 2024-04-14T08:39:03.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f1bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9d8c040 0x558be9dbbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f1bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9d8c040 0x558be9dbbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59032/0 (socket says 172.21.15.172:59032) 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f1bfff640 1 -- 172.21.15.172:0/472332403 learned_addr learned my addr 172.21.15.172:0/472332403 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/472332403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9db8b60 0x558be9d8bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f13fff640 1 --2- 172.21.15.172:0/472332403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9dbb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f1bfff640 1 -- 172.21.15.172:0/472332403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9db8b60 msgr2=0x558be9d8bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f1bfff640 1 --2- 172.21.15.172:0/472332403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9db8b60 0x558be9d8bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f1bfff640 1 -- 172.21.15.172:0/472332403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 msgr2=0x558be9dbb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f1bfff640 1 --2- 172.21.15.172:0/472332403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9dbb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f1bfff640 1 -- 172.21.15.172:0/472332403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be9a092d0 con 0x558be9d8c040 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f1bfff640 1 --2- 172.21.15.172:0/472332403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9d8c040 0x558be9dbbe30 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0c7d20 tx=0x7f6f14005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab316d16c71ec8cb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f117fa640 1 -- 172.21.15.172:0/472332403 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1401a070 con 0x558be9d8c040 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f117fa640 1 -- 172.21.15.172:0/472332403 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f14007610 con 0x558be9d8c040 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.800+0000 7f6f117fa640 1 -- 172.21.15.172:0/472332403 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f140079c0 con 0x558be9d8c040 2024-04-14T08:39:03.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.801+0000 7f6f24b8d740 1 -- 172.21.15.172:0/472332403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9d8c040 msgr2=0x558be9dbbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.801+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/472332403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9d8c040 0x558be9dbbe30 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0c7d20 tx=0x7f6f14005f40 comp rx=0 tx=0).stop 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.801+0000 7f6f24b8d740 1 -- 172.21.15.172:0/472332403 shutdown_connections 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.801+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/472332403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9d8c040 0x558be9dbbe30 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.801+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/472332403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9dbb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.801+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/472332403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9db8b60 0x558be9d8bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.801+0000 7f6f24b8d740 1 -- 172.21.15.172:0/472332403 >> 172.21.15.172:0/472332403 conn(0x558be9d8dd70 msgr2=0x558be9dd68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.801+0000 7f6f24b8d740 1 -- 172.21.15.172:0/472332403 shutdown_connections 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.801+0000 7f6f24b8d740 1 -- 172.21.15.172:0/472332403 wait complete. 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.802+0000 7f6f24b8d740 1 -- start start 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.802+0000 7f6f24b8d740 1 Processor -- start 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f24b8d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8b60 0x558be9de69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f24b8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8f40 0x558be9dbe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f24b8d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9dd7000 0x558be9dd58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f24b8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558be9d90410 con 0x558be9db8f40 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f24b8d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558be9cb09e0 con 0x558be9db8b60 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f24b8d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x558be9d92c50 con 0x558be9dd7000 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f24b8d740 10 client.0 _ll_register_callbacks cb 0x558be9ced080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f13fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8f40 0x558be9dbe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f13fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8f40 0x558be9dbe180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59046/0 (socket says 172.21.15.172:59046) 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f13fff640 1 -- 172.21.15.172:0/1139694834 learned_addr learned my addr 172.21.15.172:0/1139694834 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f1bfff640 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9dd7000 0x558be9dd58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8b60 0x558be9de69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f13fff640 1 -- 172.21.15.172:0/1139694834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9dd7000 msgr2=0x558be9dd58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f13fff640 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9dd7000 0x558be9dd58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f13fff640 1 -- 172.21.15.172:0/1139694834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8b60 msgr2=0x558be9de69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f13fff640 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8b60 0x558be9de69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f13fff640 1 -- 172.21.15.172:0/1139694834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be9ad36b0 con 0x558be9db8f40 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f13fff640 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8f40 0x558be9dbe180 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6f0407aa10 tx=0x7f6f0407b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad6403aa4cd81b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6efa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6efa7fc640 1 -- 172.21.15.172:0/1139694834 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0407f0c0 con 0x558be9db8f40 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6efa7fc640 1 -- 172.21.15.172:0/1139694834 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f6f04083460 con 0x558be9db8f40 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.803+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1139694834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f14007610 con 0x558be9db8f40 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6efa7fc640 1 -- 172.21.15.172:0/1139694834 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f6f040838f0 con 0x558be9db8f40 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6f24b8d740 10 client.7773 fetch_fsmap learned FSMap version 60 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1139694834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x558be9dd4af0 con 0x558be9db8f40 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6efa7fc640 1 -- 172.21.15.172:0/1139694834 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f14007610 con 0x558be9db8f40 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6efa7fc640 1 -- 172.21.15.172:0/1139694834 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 60) ==== 3494+0+0 (secure 0 0 0) 0x7f6f0407a160 con 0x558be9db8f40 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6f24b8d740 10 client.7773 fetch_fsmap finished waiting for FSMap version 60 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6f24b8d740 10 client.7773 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:39:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6f24b8d740 20 client.7773 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x558be9dea3c0 0x558be9dea7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6f24b8d740 4 client.7773 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1139694834 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558be9c9ccc0 con 0x558be9dea3c0 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x558be9dea3c0 0x558be9dea7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6efa7fc640 10 client.7773 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.804+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x558be9dea3c0 0x558be9dea7a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdcef75de66bfec5 server_cookie=2cbb41aa40bbaa52 in_seq=0 out_seq=0 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6efa7fc640 1 -- 172.21.15.172:0/1139694834 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558be9c9ccc0 con 0x558be9dea3c0 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6efa7fc640 10 client.7773 handle_command_reply: tid=0 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6f24b8d740 1 client.7773 shutdown 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1139694834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8f40 msgr2=0x558be9dbe180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8f40 0x558be9dbe180 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6f0407aa10 tx=0x7f6f0407b410 comp rx=0 tx=0).stop 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1139694834 shutdown_connections 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x558be9dea3c0 0x558be9dea7a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9dd7000 0x558be9dd58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6efa7fc640 0 client.7773 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8f40 0x558be9dbe180 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/1139694834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8b60 0x558be9de69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1139694834 >> 172.21.15.172:0/1139694834 conn(0x558be9d8c040 msgr2=0x558be9d8fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1139694834 shutdown_connections 2024-04-14T08:39:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1139694834 wait complete. 2024-04-14T08:39:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.805+0000 7f6f24b8d740 20 client.7773 trim_cache size 0 max 16384 2024-04-14T08:39:03.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.830+0000 7f6f24b8d740 1 Processor -- start 2024-04-14T08:39:03.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.830+0000 7f6f24b8d740 1 -- start start 2024-04-14T08:39:03.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f24b8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8b60 0x558be9dd67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f24b8d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9dbb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f24b8d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9dd7000 0x558be9dbbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f24b8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c200a30 con 0x558be9db8b60 2024-04-14T08:39:03.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f24b8d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1ffdc0 con 0x558be9db8f40 2024-04-14T08:39:03.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f24b8d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1c2003e0 con 0x558be9dd7000 2024-04-14T08:39:03.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f1b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8b60 0x558be9dd67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f1b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8b60 0x558be9dd67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59060/0 (socket says 172.21.15.172:59060) 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f1b7fe640 1 -- 172.21.15.172:0/1583499183 learned_addr learned my addr 172.21.15.172:0/1583499183 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f1bfff640 1 --2- 172.21.15.172:0/1583499183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9dd7000 0x558be9dbbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f13fff640 1 --2- 172.21.15.172:0/1583499183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9dbb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f1b7fe640 1 -- 172.21.15.172:0/1583499183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9dd7000 msgr2=0x558be9dbbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/1583499183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9dd7000 0x558be9dbbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f1b7fe640 1 -- 172.21.15.172:0/1583499183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 msgr2=0x558be9dbb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/1583499183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9dbb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.831+0000 7f6f1b7fe640 1 -- 172.21.15.172:0/1583499183 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be9dd50b0 con 0x558be9db8b60 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.832+0000 7f6f13fff640 1 --2- 172.21.15.172:0/1583499183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9dbb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.832+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/1583499183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8b60 0x558be9dd67c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x558be9db72a0 tx=0x7f6f0c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935241c4ad70f2f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.832+0000 7f6f10ff9640 1 -- 172.21.15.172:0/1583499183 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c02bc10 con 0x558be9db8b60 2024-04-14T08:39:03.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.832+0000 7f6f10ff9640 1 -- 172.21.15.172:0/1583499183 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f0c02bdb0 con 0x558be9db8b60 2024-04-14T08:39:03.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.832+0000 7f6f10ff9640 1 -- 172.21.15.172:0/1583499183 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c016b70 con 0x558be9db8b60 2024-04-14T08:39:03.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.833+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1583499183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8b60 msgr2=0x558be9dd67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.833+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/1583499183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8b60 0x558be9dd67c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x558be9db72a0 tx=0x7f6f0c014000 comp rx=0 tx=0).stop 2024-04-14T08:39:03.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.833+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1583499183 shutdown_connections 2024-04-14T08:39:03.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.833+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/1583499183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9dd7000 0x558be9dbbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.833+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/1583499183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9dbb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.833+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/1583499183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9db8b60 0x558be9dd67c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.833+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1583499183 >> 172.21.15.172:0/1583499183 conn(0x558be9d8c040 msgr2=0x558be9dbe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:03.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.833+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1583499183 shutdown_connections 2024-04-14T08:39:03.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.833+0000 7f6f24b8d740 1 -- 172.21.15.172:0/1583499183 wait complete. 2024-04-14T08:39:03.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.833+0000 7f6f24b8d740 1 -- start start 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.833+0000 7f6f24b8d740 1 Processor -- start 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.834+0000 7f6f24b8d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9db8b60 0x558be9db6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.834+0000 7f6f24b8d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9d90620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.834+0000 7f6f24b8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9dd7000 0x558be9d90b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.834+0000 7f6f24b8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558be9d90410 con 0x558be9dd7000 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.834+0000 7f6f24b8d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558be9cb09e0 con 0x558be9db8f40 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.834+0000 7f6f24b8d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x558be9dbd3e0 con 0x558be9db8b60 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.834+0000 7f6f24b8d740 10 client.0 _ll_register_callbacks cb 0x7f6f0407a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.834+0000 7f6f1bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9dd7000 0x558be9d90b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.834+0000 7f6f1bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9dd7000 0x558be9d90b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59068/0 (socket says 172.21.15.172:59068) 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.834+0000 7f6f1bfff640 1 -- 172.21.15.172:0/165462814 learned_addr learned my addr 172.21.15.172:0/165462814 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.834+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9db8b60 0x558be9db6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.834+0000 7f6f13fff640 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9d90620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f1bfff640 1 -- 172.21.15.172:0/165462814 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9db8b60 msgr2=0x558be9db6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f1bfff640 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9db8b60 0x558be9db6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f1bfff640 1 -- 172.21.15.172:0/165462814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 msgr2=0x558be9d90620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f1bfff640 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9d90620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f1bfff640 1 -- 172.21.15.172:0/165462814 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be9dc0b50 con 0x558be9dd7000 2024-04-14T08:39:03.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f13fff640 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9d90620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:39:03.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f1bfff640 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9dd7000 0x558be9d90b60 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f6f14002230 tx=0x7f6f14013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c695fc8827997d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6efa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:03.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6efa7fc640 1 -- 172.21.15.172:0/165462814 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14025030 con 0x558be9dd7000 2024-04-14T08:39:03.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6efa7fc640 1 -- 172.21.15.172:0/165462814 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f6f140078e0 con 0x558be9dd7000 2024-04-14T08:39:03.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f24b8d740 1 -- 172.21.15.172:0/165462814 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f0c02bdb0 con 0x558be9dd7000 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6efa7fc640 1 -- 172.21.15.172:0/165462814 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f6f14031050 con 0x558be9dd7000 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f24b8d740 10 client.7779 fetch_fsmap learned FSMap version 60 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f24b8d740 1 -- 172.21.15.172:0/165462814 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x558be9d93000 con 0x558be9dd7000 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6efa7fc640 1 -- 172.21.15.172:0/165462814 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f0c02bdb0 con 0x558be9dd7000 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6efa7fc640 1 -- 172.21.15.172:0/165462814 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 60) ==== 3494+0+0 (secure 0 0 0) 0x7f6f14034020 con 0x558be9dd7000 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f24b8d740 10 client.7779 fetch_fsmap finished waiting for FSMap version 60 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f24b8d740 10 client.7779 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f24b8d740 20 client.7779 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x558be9db72d0 0x558be9db76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f24b8d740 4 client.7779 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.835+0000 7f6f24b8d740 1 -- 172.21.15.172:0/165462814 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "status"}) -- 0x558be9d02570 con 0x558be9db72d0 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x558be9db72d0 0x558be9db76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6efa7fc640 10 client.7779 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:03.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f1b7fe640 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x558be9db72d0 0x558be9db76b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a353a645046b846 server_cookie=ade32f4d84e31475 in_seq=0 out_seq=0 2024-04-14T08:39:03.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6efa7fc640 1 -- 172.21.15.172:0/165462814 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x558be9d02570 con 0x558be9db72d0 2024-04-14T08:39:03.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6efa7fc640 10 client.7779 handle_command_reply: tid=0 2024-04-14T08:39:03.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f24b8d740 1 client.7779 shutdown 2024-04-14T08:39:03.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f24b8d740 1 -- 172.21.15.172:0/165462814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9dd7000 msgr2=0x558be9d90b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9dd7000 0x558be9d90b60 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f6f14002230 tx=0x7f6f14013000 comp rx=0 tx=0).stop 2024-04-14T08:39:03.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f24b8d740 1 -- 172.21.15.172:0/165462814 shutdown_connections 2024-04-14T08:39:03.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x558be9db72d0 0x558be9db76b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558be9dd7000 0x558be9d90b60 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558be9db8f40 0x558be9d90620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f24b8d740 1 --2- 172.21.15.172:0/165462814 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558be9db8b60 0x558be9db6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f24b8d740 1 -- 172.21.15.172:0/165462814 >> 172.21.15.172:0/165462814 conn(0x558be9d8c040 msgr2=0x558be9dd9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:03.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6efa7fc640 0 client.7779 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:03.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f24b8d740 1 -- 172.21.15.172:0/165462814 shutdown_connections 2024-04-14T08:39:03.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f24b8d740 1 -- 172.21.15.172:0/165462814 wait complete. 2024-04-14T08:39:03.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.836+0000 7f6f24b8d740 20 client.7779 trim_cache size 0 max 16384 2024-04-14T08:39:03.847 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:39:03.847 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:39:03.847 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:39:03.847 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 7473, 2024-04-14T08:39:03.847 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:39:03.847 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:39:03.848 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:39:03.848 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 4.432807983, 2024-04-14T08:39:03.848 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 60, 2024-04-14T08:39:03.848 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 87, 2024-04-14T08:39:03.848 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 87, 2024-04-14T08:39:03.848 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 23.657226879 2024-04-14T08:39:03.848 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:39:03.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.837+0000 7f6f21daa640 1 -- 172.21.15.172:0/2812688246 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6efc06c8f0 msgr2=0x7f6efc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.837+0000 7f6f21daa640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6efc06c8f0 0x7f6efc06edb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c000c00 tx=0x7f6f0c009040 comp rx=0 tx=0).stop 2024-04-14T08:39:03.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.837+0000 7f6f21daa640 1 -- 172.21.15.172:0/2812688246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c0b5560 msgr2=0x7f6f1c1e98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.837+0000 7f6f21daa640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c0b5560 0x7f6f1c1e98b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6f04000c00 tx=0x7f6f04004730 comp rx=0 tx=0).stop 2024-04-14T08:39:03.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.837+0000 7f6f21daa640 1 -- 172.21.15.172:0/2812688246 shutdown_connections 2024-04-14T08:39:03.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.837+0000 7f6f21daa640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6efc06c8f0 0x7f6efc06edb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.837+0000 7f6f21daa640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f1c0b8310 0x7f6f1c1586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.837+0000 7f6f21daa640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f1c0b5560 0x7f6f1c1e98b0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.837+0000 7f6f21daa640 1 --2- 172.21.15.172:0/2812688246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f08004800 0x7f6f1c1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.837+0000 7f6f21daa640 1 -- 172.21.15.172:0/2812688246 >> 172.21.15.172:0/2812688246 conn(0x7f6f1c09d560 msgr2=0x7f6f1c15c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:03.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.837+0000 7f6f21daa640 1 -- 172.21.15.172:0/2812688246 shutdown_connections 2024-04-14T08:39:03.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.837+0000 7f6f21daa640 1 -- 172.21.15.172:0/2812688246 wait complete. 2024-04-14T08:39:03.851 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:39:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.950+0000 7f7472a36640 1 Processor -- start 2024-04-14T08:39:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.950+0000 7f7472a36640 1 -- start start 2024-04-14T08:39:03.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f7472a36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f746c15c150 0x7f746c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f7472a36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f746c0d1840 0x7f746c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f7472a36640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f746c0d0200 con 0x7f746c162530 2024-04-14T08:39:03.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f7472a36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f746c0d1cb0 con 0x7f746c0d1840 2024-04-14T08:39:03.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f7472a36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f746c162980 con 0x7f746c15c150 2024-04-14T08:39:03.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f746b7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f746c0d1840 0x7f746c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f746b7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f746c0d1840 0x7f746c161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59078/0 (socket says 172.21.15.172:59078) 2024-04-14T08:39:03.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f746b7fe640 1 -- 172.21.15.172:0/1544037760 learned_addr learned my addr 172.21.15.172:0/1544037760 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:03.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f746bfff640 1 --2- 172.21.15.172:0/1544037760 >> v2:172.21.15.177:3300/0 conn(0x7f746c15c150 0x7f746c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f746b7fe640 1 -- 172.21.15.172:0/1544037760 >> v2:172.21.15.177:3300/0 conn(0x7f746c15c150 msgr2=0x7f746c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f746b7fe640 1 --2- 172.21.15.172:0/1544037760 >> v2:172.21.15.177:3300/0 conn(0x7f746c15c150 0x7f746c0d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f746b7fe640 1 -- 172.21.15.172:0/1544037760 >> v1:172.21.15.177:6789/0 conn(0x7f746c162530 legacy=0x7f746c15ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:03.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f746b7fe640 1 -- 172.21.15.172:0/1544037760 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c0cf160 con 0x7f746c0d1840 2024-04-14T08:39:03.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.952+0000 7f746bfff640 1 --2- 172.21.15.172:0/1544037760 >> v2:172.21.15.177:3300/0 conn(0x7f746c15c150 0x7f746c0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.953+0000 7f746b7fe640 1 --2- 172.21.15.172:0/1544037760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f746c0d1840 0x7f746c161ff0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f746c06bf30 tx=0x7f746002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33341b1677144c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.953+0000 7f746affd640 1 -- 172.21.15.172:0/1544037760 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460035020 con 0x7f746c0d1840 2024-04-14T08:39:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.953+0000 7f746affd640 1 -- 172.21.15.172:0/1544037760 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7460031870 con 0x7f746c0d1840 2024-04-14T08:39:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.953+0000 7f746affd640 1 -- 172.21.15.172:0/1544037760 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460031c40 con 0x7f746c0d1840 2024-04-14T08:39:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.954+0000 7f7472a36640 1 -- 172.21.15.172:0/1544037760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f746c0d1840 msgr2=0x7f746c161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.954+0000 7f7472a36640 1 --2- 172.21.15.172:0/1544037760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f746c0d1840 0x7f746c161ff0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f746c06bf30 tx=0x7f746002b910 comp rx=0 tx=0).stop 2024-04-14T08:39:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.954+0000 7f7472a36640 1 -- 172.21.15.172:0/1544037760 shutdown_connections 2024-04-14T08:39:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.954+0000 7f7472a36640 1 --2- 172.21.15.172:0/1544037760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f746c0d1840 0x7f746c161ff0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.954+0000 7f7472a36640 1 --2- 172.21.15.172:0/1544037760 >> v2:172.21.15.177:3300/0 conn(0x7f746c15c150 0x7f746c0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.954+0000 7f7472a36640 1 -- 172.21.15.172:0/1544037760 >> 172.21.15.172:0/1544037760 conn(0x7f746c09d580 msgr2=0x7f746c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.954+0000 7f7472a36640 1 -- 172.21.15.172:0/1544037760 shutdown_connections 2024-04-14T08:39:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.954+0000 7f7472a36640 1 -- 172.21.15.172:0/1544037760 wait complete. 2024-04-14T08:39:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.955+0000 7f7472a36640 1 Processor -- start 2024-04-14T08:39:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.955+0000 7f7472a36640 1 -- start start 2024-04-14T08:39:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f7472a36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f746c0d1840 0x7f746c15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f7472a36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746c15c150 0x7f746c1e8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f7472a36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746c162530 0x7f746c1e93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f7472a36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f746c0cf720 con 0x7f746c0d1840 2024-04-14T08:39:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f7472a36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f746c0cef10 con 0x7f746c15c150 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f7472a36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f746c0cfa00 con 0x7f746c162530 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f746bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f746c0d1840 0x7f746c15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f746bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f746c0d1840 0x7f746c15a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59088/0 (socket says 172.21.15.172:59088) 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f746bfff640 1 -- 172.21.15.172:0/2846810599 learned_addr learned my addr 172.21.15.172:0/2846810599 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f746bfff640 1 -- 172.21.15.172:0/2846810599 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746c162530 msgr2=0x7f746c1e93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f746bfff640 1 --2- 172.21.15.172:0/2846810599 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746c162530 0x7f746c1e93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f746bfff640 1 -- 172.21.15.172:0/2846810599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746c15c150 msgr2=0x7f746c1e8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f746bfff640 1 --2- 172.21.15.172:0/2846810599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746c15c150 0x7f746c1e8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.956+0000 7f746bfff640 1 -- 172.21.15.172:0/2846810599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c15e740 con 0x7f746c0d1840 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.957+0000 7f746bfff640 1 --2- 172.21.15.172:0/2846810599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f746c0d1840 0x7f746c15a950 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f745c00c820 tx=0x7f745c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.957+0000 7f7468ff9640 1 -- 172.21.15.172:0/2846810599 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c015070 con 0x7f746c0d1840 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.957+0000 7f7468ff9640 1 -- 172.21.15.172:0/2846810599 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f745c007a30 con 0x7f746c0d1840 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.957+0000 7f7468ff9640 1 -- 172.21.15.172:0/2846810599 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c007e00 con 0x7f746c0d1840 2024-04-14T08:39:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.957+0000 7f7472a36640 1 -- 172.21.15.172:0/2846810599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746c15ea20 con 0x7f746c0d1840 2024-04-14T08:39:03.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.957+0000 7f7472a36640 1 -- 172.21.15.172:0/2846810599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746c15efe0 con 0x7f746c0d1840 2024-04-14T08:39:03.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.958+0000 7f7472a36640 1 -- 172.21.15.172:0/2846810599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f746c075490 con 0x7f746c0d1840 2024-04-14T08:39:03.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.958+0000 7f7468ff9640 1 -- 172.21.15.172:0/2846810599 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f745c018070 con 0x7f746c0d1840 2024-04-14T08:39:03.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.959+0000 7f7468ff9640 1 --2- 172.21.15.172:0/2846810599 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f745806c8f0 0x7f745806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:03.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.959+0000 7f7468ff9640 1 -- 172.21.15.172:0/2846810599 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f745c08bd70 con 0x7f746c0d1840 2024-04-14T08:39:03.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.959+0000 7f746b7fe640 1 --2- 172.21.15.172:0/2846810599 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f745806c8f0 0x7f745806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:03.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.959+0000 7f746b7fe640 1 --2- 172.21.15.172:0/2846810599 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f745806c8f0 0x7f745806edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7460036000 tx=0x7f7460030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:03.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:03.967+0000 7f7468ff9640 1 -- 172.21.15.172:0/2846810599 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f745c059f40 con 0x7f746c0d1840 2024-04-14T08:39:04.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.204+0000 7f7472a36640 1 -- 172.21.15.172:0/2846810599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f746c0cfcd0 con 0x7f746c0d1840 2024-04-14T08:39:04.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.205+0000 7f7468ff9640 1 -- 172.21.15.172:0/2846810599 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) ==== 53+0+73 (secure 0 0 0) 0x7f745c059840 con 0x7f746c0d1840 2024-04-14T08:39:04.206 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:39:04.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.207+0000 7f7472a36640 1 -- 172.21.15.172:0/2846810599 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f745806c8f0 msgr2=0x7f745806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:04.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.207+0000 7f7472a36640 1 --2- 172.21.15.172:0/2846810599 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f745806c8f0 0x7f745806edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7460036000 tx=0x7f7460030000 comp rx=0 tx=0).stop 2024-04-14T08:39:04.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.207+0000 7f7472a36640 1 -- 172.21.15.172:0/2846810599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f746c0d1840 msgr2=0x7f746c15a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:04.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.207+0000 7f7472a36640 1 --2- 172.21.15.172:0/2846810599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f746c0d1840 0x7f746c15a950 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f745c00c820 tx=0x7f745c00cc20 comp rx=0 tx=0).stop 2024-04-14T08:39:04.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.207+0000 7f7472a36640 1 -- 172.21.15.172:0/2846810599 shutdown_connections 2024-04-14T08:39:04.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.207+0000 7f7472a36640 1 --2- 172.21.15.172:0/2846810599 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f745806c8f0 0x7f745806edb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:04.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.207+0000 7f7472a36640 1 --2- 172.21.15.172:0/2846810599 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f746c162530 0x7f746c1e93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:04.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.207+0000 7f7472a36640 1 --2- 172.21.15.172:0/2846810599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746c15c150 0x7f746c1e8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:04.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.207+0000 7f7472a36640 1 --2- 172.21.15.172:0/2846810599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f746c0d1840 0x7f746c15a950 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:04.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.207+0000 7f7472a36640 1 -- 172.21.15.172:0/2846810599 >> 172.21.15.172:0/2846810599 conn(0x7f746c09d580 msgr2=0x7f746c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:04.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.207+0000 7f7472a36640 1 -- 172.21.15.172:0/2846810599 shutdown_connections 2024-04-14T08:39:04.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04.207+0000 7f7472a36640 1 -- 172.21.15.172:0/2846810599 wait complete. 2024-04-14T08:39:04.219 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:39:04.219 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:39:04.219 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:39:04.220 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:39:04.220 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:39:04.220 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:39:04.220 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:39:04.220 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:39:04.220 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:39:04.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:04.220 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:39:04.235 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:39:04.236 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:39:04.236 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:39:04.236 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:39:04.236 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:39:04.236 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:39:04.236 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:39:04.236 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:39:04.236 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:39:04.236 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:39:04.236 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:39:04.236 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:39:04.236 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42325sec preferred_lft 42325sec 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:39:04.237 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:39:04.238 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:39:04.238 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:39:04.238 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:39:04.238 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:39:04.238 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:39:04.238 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:39:04.238 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:39:04.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:04.238 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:39:04.302 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:39:04.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:04.304 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:39:04.360 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:39:04.360 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:39:04.360 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:39:04.360 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:39:04.360 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:39:04.361 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:39:04.361 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:39:04.361 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:39:04.361 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:39:04.361 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:39:04.361 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:39:04.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:39:04.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:39:04.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:04.499 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:39:04.513 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:39:04.514 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:39:04.514 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:39:04.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:04.514 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:39:04.580 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:39:04.581 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:39:04.636 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:39:04.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:39:04.636 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:39:04.690 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:39:04.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:04.691 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:39:04.775 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:39:04.775 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:39:04.775 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:39:04.783 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:39:04.783 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:39:04.783 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:39:04.783 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:39:04.783 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:39:04.789 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:39:04.789 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:39:04.789 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:39:04.790 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:39:04.790 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:39:04.790 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:39:04.790 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:39:04.790 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:39:04.790 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:39:04.790 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:39:04.791 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:39:04.791 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:39:04.791 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:39:04.792 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:39:04.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:04.793 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:39:04.823 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:39:04.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:39:04.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:39:04.919 INFO:teuthology.orchestra.run.smithi172.stdout:client7264 2024-04-14T08:39:04.919 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:39:04.920 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:39:04.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:04.935 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-14T08:39:04.993 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:39:04.992+0000 7effbcfe9640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-14T08:39:04.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:39:04.995 INFO:teuthology.orchestra.run.smithi172.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-14T08:39:04.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:04.995 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-14T08:39:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.141+0000 7fc58995e640 1 Processor -- start 2024-04-14T08:39:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.141+0000 7fc58995e640 1 -- start start 2024-04-14T08:39:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.141+0000 7fc58995e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5840b1100 0x7fc5840d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.141+0000 7fc58995e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5840b4670 con 0x7fc58409aa70 2024-04-14T08:39:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.141+0000 7fc58995e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5840b4950 con 0x7fc58406f4d0 2024-04-14T08:39:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.141+0000 7fc58995e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc58415eee0 con 0x7fc5840b1100 2024-04-14T08:39:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc582ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc58409aa70 0x7fc58409ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43350/0 (socket says 172.21.15.172:43350) 2024-04-14T08:39:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc582ffd640 1 -- 172.21.15.172:0/3698557808 learned_addr learned my addr 172.21.15.172:0/3698557808 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:39:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc57bfff640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5840b1100 0x7fc5840d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 903527006 0 0) 0x7fc5840b4670 con 0x7fc58409aa70 2024-04-14T08:39:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc570003520 con 0x7fc58409aa70 2024-04-14T08:39:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1607889402 0 0) 0x7fc5840b4950 con 0x7fc58406f4d0 2024-04-14T08:39:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5840b4670 con 0x7fc58406f4d0 2024-04-14T08:39:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 282664830 0 0) 0x7fc570003520 con 0x7fc58409aa70 2024-04-14T08:39:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5840b1100 msgr2=0x7fc5840d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5840b1100 0x7fc5840d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 >> v1:172.21.15.177:6789/0 conn(0x7fc58406f4d0 legacy=0x7fc5840d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58415f1c0 con 0x7fc58409aa70 2024-04-14T08:39:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fc574001670 con 0x7fc58409aa70 2024-04-14T08:39:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 >> v1:172.21.15.172:6789/0 conn(0x7fc58409aa70 legacy=0x7fc58409ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc570003b70 0x7fc570003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc570004800 0x7fc570004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5700051c0 0x7fc57000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc58415eee0 con 0x7fc570004800 2024-04-14T08:39:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc57000c030 con 0x7fc5700051c0 2024-04-14T08:39:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc57000c230 con 0x7fc570003b70 2024-04-14T08:39:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc57bfff640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5840b1100 0x7fc5840d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:39:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.142+0000 7fc57bfff640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc570004800 0x7fc570004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.143+0000 7fc582ffd640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc570003b70 0x7fc570003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.143+0000 7fc5837fe640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5700051c0 0x7fc57000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.143+0000 7fc57bfff640 1 -- 172.21.15.172:0/3698557808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc570003b70 msgr2=0x7fc570003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.143+0000 7fc57bfff640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc570003b70 0x7fc570003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.143+0000 7fc57bfff640 1 -- 172.21.15.172:0/3698557808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5700051c0 msgr2=0x7fc57000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.143+0000 7fc57bfff640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5700051c0 0x7fc57000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.143+0000 7fc57bfff640 1 -- 172.21.15.172:0/3698557808 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc570003520 con 0x7fc570004800 2024-04-14T08:39:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.143+0000 7fc57bfff640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc570004800 0x7fc570004c80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc5840a0180 tx=0x7fc570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cfd0f31dff6539 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.144+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0097d0 con 0x7fc570004800 2024-04-14T08:39:05.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.144+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56c004030 con 0x7fc570004800 2024-04-14T08:39:05.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.144+0000 7fc5827fc640 1 -- 172.21.15.172:0/3698557808 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c004400 con 0x7fc570004800 2024-04-14T08:39:05.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.144+0000 7fc58995e640 1 -- 172.21.15.172:0/3698557808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc570004800 msgr2=0x7fc570004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.144+0000 7fc58995e640 1 --2- 172.21.15.172:0/3698557808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc570004800 0x7fc570004c80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc5840a0180 tx=0x7fc570005830 comp rx=0 tx=0).stop 2024-04-14T08:39:05.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.144+0000 7fc5837fe640 1 -- 172.21.15.172:0/3698557808 reap_dead start 2024-04-14T08:39:05.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.144+0000 7fc58995e640 1 -- 172.21.15.172:0/3698557808 shutdown_connections 2024-04-14T08:39:05.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.144+0000 7fc58995e640 1 -- 172.21.15.172:0/3698557808 >> 172.21.15.172:0/3698557808 conn(0x7fc5840b3b50 msgr2=0x7fc5840b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:05.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.144+0000 7fc58995e640 1 -- 172.21.15.172:0/3698557808 shutdown_connections 2024-04-14T08:39:05.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.144+0000 7fc58995e640 1 -- 172.21.15.172:0/3698557808 wait complete. 2024-04-14T08:39:05.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.146+0000 7fc58995e640 1 Processor -- start 2024-04-14T08:39:05.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.146+0000 7fc58995e640 1 -- start start 2024-04-14T08:39:05.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.147+0000 7fc58995e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc570004800 0x7fc5841f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.147+0000 7fc58995e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5841f74c0 0x7fc584159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.147+0000 7fc58995e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5840d05f0 0x7fc5840d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.147+0000 7fc58995e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc58415fb40 con 0x7fc570004800 2024-04-14T08:39:05.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.147+0000 7fc58995e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc58415f9c0 con 0x7fc5840d05f0 2024-04-14T08:39:05.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.147+0000 7fc58995e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc5840d01d0 con 0x7fc5841f74c0 2024-04-14T08:39:05.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.147+0000 7fc582ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc570004800 0x7fc5841f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.147+0000 7fc582ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc570004800 0x7fc5841f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59106/0 (socket says 172.21.15.172:59106) 2024-04-14T08:39:05.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.147+0000 7fc582ffd640 1 -- 172.21.15.172:0/3944317600 learned_addr learned my addr 172.21.15.172:0/3944317600 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:05.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc5837fe640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5840d05f0 0x7fc5840d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc57bfff640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5841f74c0 0x7fc584159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc582ffd640 1 -- 172.21.15.172:0/3944317600 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5841f74c0 msgr2=0x7fc584159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc582ffd640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5841f74c0 0x7fc584159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc582ffd640 1 -- 172.21.15.172:0/3944317600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5840d05f0 msgr2=0x7fc5840d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc582ffd640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5840d05f0 0x7fc5840d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc582ffd640 1 -- 172.21.15.172:0/3944317600 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5841f4e90 con 0x7fc570004800 2024-04-14T08:39:05.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc582ffd640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc570004800 0x7fc5841f4950 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc574004e80 tx=0x7fc574007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc580ff9640 1 -- 172.21.15.172:0/3944317600 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57400f7b0 con 0x7fc570004800 2024-04-14T08:39:05.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc580ff9640 1 -- 172.21.15.172:0/3944317600 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc574018070 con 0x7fc570004800 2024-04-14T08:39:05.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc58995e640 1 -- 172.21.15.172:0/3944317600 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5841f7a30 con 0x7fc570004800 2024-04-14T08:39:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc580ff9640 1 -- 172.21.15.172:0/3944317600 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57400fbb0 con 0x7fc570004800 2024-04-14T08:39:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.148+0000 7fc58995e640 1 -- 172.21.15.172:0/3944317600 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5841f7ff0 con 0x7fc570004800 2024-04-14T08:39:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.150+0000 7fc580ff9640 1 -- 172.21.15.172:0/3944317600 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc57400fd50 con 0x7fc570004800 2024-04-14T08:39:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.151+0000 7fc580ff9640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc56406c8f0 0x7fc56406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.151+0000 7fc580ff9640 1 -- 172.21.15.172:0/3944317600 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7fc57408bc80 con 0x7fc570004800 2024-04-14T08:39:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.151+0000 7fc57bfff640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc56406c8f0 0x7fc56406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.151+0000 7fc57bfff640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc56406c8f0 0x7fc56406edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc56c035000 tx=0x7fc56c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.151+0000 7fc58c835740 1 Processor -- start 2024-04-14T08:39:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.151+0000 7fc58c835740 1 -- start start 2024-04-14T08:39:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.152+0000 7fc58c835740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e83f0 0x55dab77bb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.152+0000 7fc58c835740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e87d0 0x55dab77eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.152+0000 7fc58c835740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77bb8d0 0x55dab77eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.152+0000 7fc58c835740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dab7724400 con 0x55dab77bb8d0 2024-04-14T08:39:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.152+0000 7fc58c835740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55dab76c2a10 con 0x55dab77e87d0 2024-04-14T08:39:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.152+0000 7fc58c835740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55dab77203b0 con 0x55dab77e83f0 2024-04-14T08:39:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.152+0000 7fc5837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77bb8d0 0x55dab77eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e87d0 0x55dab77eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc582ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e83f0 0x55dab77bb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc5837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77bb8d0 0x55dab77eb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59114/0 (socket says 172.21.15.172:59114) 2024-04-14T08:39:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc5837fe640 1 -- 172.21.15.172:0/3096755178 learned_addr learned my addr 172.21.15.172:0/3096755178 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc5837fe640 1 -- 172.21.15.172:0/3096755178 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e83f0 msgr2=0x55dab77bb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc5837fe640 1 --2- 172.21.15.172:0/3096755178 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e83f0 0x55dab77bb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc5837fe640 1 -- 172.21.15.172:0/3096755178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e87d0 msgr2=0x55dab77eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc5837fe640 1 --2- 172.21.15.172:0/3096755178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e87d0 0x55dab77eb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc5837fe640 1 -- 172.21.15.172:0/3096755178 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab76ec930 con 0x55dab77bb8d0 2024-04-14T08:39:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc582ffd640 1 --2- 172.21.15.172:0/3096755178 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e83f0 0x55dab77bb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:39:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc57bfff640 1 --2- 172.21.15.172:0/3096755178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e87d0 0x55dab77eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:39:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc5837fe640 1 --2- 172.21.15.172:0/3096755178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77bb8d0 0x55dab77eb6c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc584072c00 tx=0x7fc57c001ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ec4ad7cc9b9788 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc5797fa640 1 -- 172.21.15.172:0/3096755178 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c01a070 con 0x55dab77bb8d0 2024-04-14T08:39:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc5797fa640 1 -- 172.21.15.172:0/3096755178 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c005520 con 0x55dab77bb8d0 2024-04-14T08:39:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.153+0000 7fc5797fa640 1 -- 172.21.15.172:0/3096755178 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c015430 con 0x55dab77bb8d0 2024-04-14T08:39:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.154+0000 7fc58c835740 1 -- 172.21.15.172:0/3096755178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77bb8d0 msgr2=0x55dab77eb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.154+0000 7fc58c835740 1 --2- 172.21.15.172:0/3096755178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77bb8d0 0x55dab77eb6c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc584072c00 tx=0x7fc57c001ea0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.154+0000 7fc58c835740 1 -- 172.21.15.172:0/3096755178 shutdown_connections 2024-04-14T08:39:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.154+0000 7fc58c835740 1 --2- 172.21.15.172:0/3096755178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77bb8d0 0x55dab77eb6c0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.154+0000 7fc58c835740 1 --2- 172.21.15.172:0/3096755178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e87d0 0x55dab77eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.154+0000 7fc58c835740 1 --2- 172.21.15.172:0/3096755178 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e83f0 0x55dab77bb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.154+0000 7fc58c835740 1 -- 172.21.15.172:0/3096755178 >> 172.21.15.172:0/3096755178 conn(0x55dab77bd600 msgr2=0x55dab7806180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.154+0000 7fc58c835740 1 -- 172.21.15.172:0/3096755178 shutdown_connections 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.154+0000 7fc58c835740 1 -- 172.21.15.172:0/3096755178 wait complete. 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.155+0000 7fc58c835740 1 -- start start 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.155+0000 7fc58c835740 1 Processor -- start 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc58c835740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e83f0 0x55dab77ecca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc58c835740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e87d0 0x55dab78193f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc58c835740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77ec320 0x55dab77ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc58c835740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dab77bfca0 con 0x55dab77ec320 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc58c835740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55dab76c06e0 con 0x55dab77e83f0 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc58c835740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55dab77c24e0 con 0x55dab77e87d0 2024-04-14T08:39:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc58c835740 10 client.0 _ll_register_callbacks cb 0x55dab7437b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc5837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77ec320 0x55dab77ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc5837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77ec320 0x55dab77ee8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59126/0 (socket says 172.21.15.172:59126) 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc5837fe640 1 -- 172.21.15.172:0/3121051732 learned_addr learned my addr 172.21.15.172:0/3121051732 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc582ffd640 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e83f0 0x55dab77ecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc57bfff640 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e87d0 0x55dab78193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc5837fe640 1 -- 172.21.15.172:0/3121051732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e87d0 msgr2=0x55dab78193f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc5837fe640 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e87d0 0x55dab78193f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc5837fe640 1 -- 172.21.15.172:0/3121051732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e83f0 msgr2=0x55dab77ecca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc5837fe640 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e83f0 0x55dab77ecca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc5837fe640 1 -- 172.21.15.172:0/3121051732 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab75047b0 con 0x55dab77ec320 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc582ffd640 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e83f0 0x55dab77ecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc57bfff640 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e87d0 0x55dab78193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.156+0000 7fc5837fe640 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77ec320 0x55dab77ee8e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc57c005900 tx=0x7fc57c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7755288a925e5a2f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc5627fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc5627fc640 1 -- 172.21.15.172:0/3121051732 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c01a040 con 0x55dab77ec320 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc5627fc640 1 -- 172.21.15.172:0/3121051732 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7fc57c007040 con 0x55dab77ec320 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc58c835740 1 -- 172.21.15.172:0/3121051732 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc57c005520 con 0x55dab77ec320 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc5627fc640 1 -- 172.21.15.172:0/3121051732 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fc57c02a050 con 0x55dab77ec320 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc58c835740 10 client.7797 fetch_fsmap learned FSMap version 60 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc58c835740 1 -- 172.21.15.172:0/3121051732 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55dab7564f40 con 0x55dab77ec320 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc5627fc640 1 -- 172.21.15.172:0/3121051732 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c005520 con 0x55dab77ec320 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc5627fc640 1 -- 172.21.15.172:0/3121051732 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 60) ==== 3494+0+0 (secure 0 0 0) 0x7fc57c02d020 con 0x55dab77ec320 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc58c835740 10 client.7797 fetch_fsmap finished waiting for FSMap version 60 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc58c835740 10 client.7797 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc58c835740 20 client.7797 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc58c835740 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55dab7817a80 0x55dab7817e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc58c835740 4 client.7797 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc58c835740 1 -- 172.21.15.172:0/3121051732 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dab7731940 con 0x55dab7817a80 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.157+0000 7fc582ffd640 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55dab7817a80 0x55dab7817e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc582ffd640 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55dab7817a80 0x55dab7817e90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b018f7edae33e21 server_cookie=dbc646acfd7e66ee in_seq=0 out_seq=0 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc5627fc640 10 client.7797 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc5627fc640 1 -- 172.21.15.172:0/3121051732 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dab7731940 con 0x55dab7817a80 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc5627fc640 10 client.7797 handle_command_reply: tid=0 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc58c835740 1 client.7797 shutdown 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc58c835740 1 -- 172.21.15.172:0/3121051732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77ec320 msgr2=0x55dab77ee8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc58c835740 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77ec320 0x55dab77ee8e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc57c005900 tx=0x7fc57c001650 comp rx=0 tx=0).stop 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc58c835740 1 -- 172.21.15.172:0/3121051732 shutdown_connections 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc58c835740 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55dab7817a80 0x55dab7817e90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc58c835740 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77ec320 0x55dab77ee8e0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc58c835740 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e87d0 0x55dab78193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc5627fc640 0 client.7797 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:05.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc58c835740 1 --2- 172.21.15.172:0/3121051732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e83f0 0x55dab77ecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc58c835740 1 -- 172.21.15.172:0/3121051732 >> 172.21.15.172:0/3121051732 conn(0x55dab77bb8d0 msgr2=0x55dab77bf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:05.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc58c835740 1 -- 172.21.15.172:0/3121051732 shutdown_connections 2024-04-14T08:39:05.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.158+0000 7fc58c835740 1 -- 172.21.15.172:0/3121051732 wait complete. 2024-04-14T08:39:05.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.159+0000 7fc58c835740 20 client.7797 trim_cache size 0 max 16384 2024-04-14T08:39:05.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.176+0000 7fc58c835740 1 Processor -- start 2024-04-14T08:39:05.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.177+0000 7fc58c835740 1 -- start start 2024-04-14T08:39:05.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.177+0000 7fc58c835740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e83f0 0x55dab7816ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.177+0000 7fc58c835740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e87d0 0x55dab7815cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.177+0000 7fc58c835740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77ec320 0x55dab7806140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.177+0000 7fc58c835740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc584201560 con 0x55dab77e83f0 2024-04-14T08:39:05.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.177+0000 7fc58c835740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc5842005e0 con 0x55dab77e87d0 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.177+0000 7fc58c835740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc5841ffa60 con 0x55dab77ec320 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.177+0000 7fc582ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e83f0 0x55dab7816ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc582ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e83f0 0x55dab7816ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59134/0 (socket says 172.21.15.172:59134) 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc582ffd640 1 -- 172.21.15.172:0/2076084084 learned_addr learned my addr 172.21.15.172:0/2076084084 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc57bfff640 1 --2- 172.21.15.172:0/2076084084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e87d0 0x55dab7815cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc5837fe640 1 --2- 172.21.15.172:0/2076084084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77ec320 0x55dab7806140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc582ffd640 1 -- 172.21.15.172:0/2076084084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77ec320 msgr2=0x55dab7806140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc582ffd640 1 --2- 172.21.15.172:0/2076084084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77ec320 0x55dab7806140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc582ffd640 1 -- 172.21.15.172:0/2076084084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e87d0 msgr2=0x55dab7815cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc582ffd640 1 --2- 172.21.15.172:0/2076084084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e87d0 0x55dab7815cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc582ffd640 1 -- 172.21.15.172:0/2076084084 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab77eec50 con 0x55dab77e83f0 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc582ffd640 1 --2- 172.21.15.172:0/2076084084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e83f0 0x55dab7816ed0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc574089fd0 tx=0x7fc574009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf096ccdab6cd2f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc578ff9640 1 -- 172.21.15.172:0/2076084084 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5740623b0 con 0x55dab77e83f0 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc578ff9640 1 -- 172.21.15.172:0/2076084084 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc574060070 con 0x55dab77e83f0 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.178+0000 7fc578ff9640 1 -- 172.21.15.172:0/2076084084 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574057ad0 con 0x55dab77e83f0 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.179+0000 7fc58c835740 1 -- 172.21.15.172:0/2076084084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e83f0 msgr2=0x55dab7816ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.179+0000 7fc58c835740 1 --2- 172.21.15.172:0/2076084084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e83f0 0x55dab7816ed0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc574089fd0 tx=0x7fc574009040 comp rx=0 tx=0).stop 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.179+0000 7fc58c835740 1 -- 172.21.15.172:0/2076084084 shutdown_connections 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.179+0000 7fc58c835740 1 --2- 172.21.15.172:0/2076084084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77ec320 0x55dab7806140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.179+0000 7fc58c835740 1 --2- 172.21.15.172:0/2076084084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77e87d0 0x55dab7815cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.179+0000 7fc58c835740 1 --2- 172.21.15.172:0/2076084084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e83f0 0x55dab7816ed0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.179+0000 7fc58c835740 1 -- 172.21.15.172:0/2076084084 >> 172.21.15.172:0/2076084084 conn(0x55dab77bb8d0 msgr2=0x55dab77e2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.179+0000 7fc58c835740 1 -- 172.21.15.172:0/2076084084 shutdown_connections 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.179+0000 7fc58c835740 1 -- 172.21.15.172:0/2076084084 wait complete. 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.179+0000 7fc58c835740 1 -- start start 2024-04-14T08:39:05.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc58c835740 1 Processor -- start 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc58c835740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e83f0 0x55dab78066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc58c835740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e87d0 0x55dab787f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc58c835740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77ec320 0x55dab77e6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc58c835740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dab77bcea0 con 0x55dab77e87d0 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc58c835740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841fff90 con 0x55dab77ec320 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc58c835740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55dab77ee290 con 0x55dab77e83f0 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc58c835740 10 client.0 _ll_register_callbacks cb 0x7fc57c018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e87d0 0x55dab787f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc57bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e87d0 0x55dab787f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59140/0 (socket says 172.21.15.172:59140) 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc57bfff640 1 -- 172.21.15.172:0/1419848531 learned_addr learned my addr 172.21.15.172:0/1419848531 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc582ffd640 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e83f0 0x55dab78066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.180+0000 7fc5837fe640 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77ec320 0x55dab77e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc57bfff640 1 -- 172.21.15.172:0/1419848531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e83f0 msgr2=0x55dab78066e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc57bfff640 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e83f0 0x55dab78066e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc57bfff640 1 -- 172.21.15.172:0/1419848531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77ec320 msgr2=0x55dab77e6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc57bfff640 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77ec320 0x55dab77e6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc57bfff640 1 -- 172.21.15.172:0/1419848531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab73f80a0 con 0x55dab77e87d0 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc57bfff640 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e87d0 0x55dab787f2d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009bf0 tx=0x7fc56c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb989fd534210ba server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc5627fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc5627fc640 1 -- 172.21.15.172:0/1419848531 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c043040 con 0x55dab77e87d0 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc5627fc640 1 -- 172.21.15.172:0/1419848531 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7fc56c02c910 con 0x55dab77e87d0 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc58c835740 1 -- 172.21.15.172:0/1419848531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc574060070 con 0x55dab77e87d0 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc5627fc640 1 -- 172.21.15.172:0/1419848531 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fc56c05e050 con 0x55dab77e87d0 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc58c835740 10 client.7803 fetch_fsmap learned FSMap version 60 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc58c835740 1 -- 172.21.15.172:0/1419848531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55dab7818910 con 0x55dab77e87d0 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.181+0000 7fc5627fc640 1 -- 172.21.15.172:0/1419848531 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc574060070 con 0x55dab77e87d0 2024-04-14T08:39:05.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.182+0000 7fc5627fc640 1 -- 172.21.15.172:0/1419848531 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 60) ==== 3494+0+0 (secure 0 0 0) 0x7fc56c061020 con 0x55dab77e87d0 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.182+0000 7fc58c835740 10 client.7803 fetch_fsmap finished waiting for FSMap version 60 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.182+0000 7fc58c835740 10 client.7803 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.182+0000 7fc58c835740 20 client.7803 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.182+0000 7fc58c835740 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55dab77e7e10 0x55dab7807f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.182+0000 7fc58c835740 4 client.7803 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.182+0000 7fc58c835740 1 -- 172.21.15.172:0/1419848531 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x55dab77bb4e0 con 0x55dab77e7e10 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.182+0000 7fc582ffd640 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55dab77e7e10 0x55dab7807f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.182+0000 7fc5627fc640 10 client.7803 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.182+0000 7fc582ffd640 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55dab77e7e10 0x55dab7807f70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20721edbc6c9ae2 server_cookie=f1df9821557ee86c in_seq=0 out_seq=0 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc5627fc640 1 -- 172.21.15.172:0/1419848531 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dab77bb4e0 con 0x55dab77e7e10 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc5627fc640 10 client.7803 handle_command_reply: tid=0 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58c835740 1 client.7803 shutdown 2024-04-14T08:39:05.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58c835740 1 -- 172.21.15.172:0/1419848531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e87d0 msgr2=0x55dab787f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58c835740 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e87d0 0x55dab787f2d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009bf0 tx=0x7fc56c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58c835740 1 -- 172.21.15.172:0/1419848531 shutdown_connections 2024-04-14T08:39:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58c835740 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55dab77e7e10 0x55dab7807f70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58c835740 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dab77ec320 0x55dab77e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58c835740 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dab77e87d0 0x55dab787f2d0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58c835740 1 --2- 172.21.15.172:0/1419848531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55dab77e83f0 0x55dab78066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58c835740 1 -- 172.21.15.172:0/1419848531 >> 172.21.15.172:0/1419848531 conn(0x55dab77bb8d0 msgr2=0x55dab77e2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:05.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc5627fc640 0 client.7803 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:05.203 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:39:05.183+0000 7effbcfe9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-14T08:38:55.461911+0000 1=1+0) n(v0 rc2024-04-14T08:38:55.578911+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x555fea025b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-14T08:38:55.461911+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-14T08:38:55.578911+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-14T08:39:05.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58c835740 1 -- 172.21.15.172:0/1419848531 shutdown_connections 2024-04-14T08:39:05.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58c835740 1 -- 172.21.15.172:0/1419848531 wait complete. 2024-04-14T08:39:05.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58c835740 20 client.7803 trim_cache size 0 max 16384 2024-04-14T08:39:05.204 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:39:05.205 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-04-14T08:39:05.205 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "b017c06e-5628-4d03-a4ce-39439a7f35a9", 2024-04-14T08:39:05.205 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-04-14T08:39:05.205 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:39:05.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58995e640 1 -- 172.21.15.172:0/3944317600 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc56406c8f0 msgr2=0x7fc56406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.183+0000 7fc58995e640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc56406c8f0 0x7fc56406edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc56c035000 tx=0x7fc56c030000 comp rx=0 tx=0).stop 2024-04-14T08:39:05.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.184+0000 7fc58995e640 1 -- 172.21.15.172:0/3944317600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc570004800 msgr2=0x7fc5841f4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.184+0000 7fc58995e640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc570004800 0x7fc5841f4950 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc574004e80 tx=0x7fc574007430 comp rx=0 tx=0).stop 2024-04-14T08:39:05.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.184+0000 7fc58995e640 1 -- 172.21.15.172:0/3944317600 shutdown_connections 2024-04-14T08:39:05.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.184+0000 7fc58995e640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc56406c8f0 0x7fc56406edb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.184+0000 7fc58995e640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5840d05f0 0x7fc5840d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.184+0000 7fc58995e640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc5841f74c0 0x7fc584159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.184+0000 7fc58995e640 1 --2- 172.21.15.172:0/3944317600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc570004800 0x7fc5841f4950 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.184+0000 7fc58995e640 1 -- 172.21.15.172:0/3944317600 >> 172.21.15.172:0/3944317600 conn(0x7fc5840b3b50 msgr2=0x7fc5840b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:05.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.184+0000 7fc58995e640 1 -- 172.21.15.172:0/3944317600 shutdown_connections 2024-04-14T08:39:05.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.184+0000 7fc58995e640 1 -- 172.21.15.172:0/3944317600 wait complete. 2024-04-14T08:39:05.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:05.207 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-14T08:39:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.310+0000 7f0973c08640 1 Processor -- start 2024-04-14T08:39:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.310+0000 7f0973c08640 1 -- start start 2024-04-14T08:39:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.311+0000 7f0973c08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c15e430 0x7f096c162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.311+0000 7f0973c08640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f096c0d2e00 0x7f096c1629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.311+0000 7f0973c08640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f096c0cefb0 con 0x7f096c162540 2024-04-14T08:39:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.311+0000 7f0973c08640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f096c0cf290 con 0x7f096c0d2e00 2024-04-14T08:39:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.311+0000 7f0973c08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f096c0cf570 con 0x7f096c15e430 2024-04-14T08:39:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.311+0000 7f09711a3640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f096c162540 0x7f096c0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43352/0 (socket says 172.21.15.172:43352) 2024-04-14T08:39:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.312+0000 7f09711a3640 1 -- 172.21.15.172:0/1378349680 learned_addr learned my addr 172.21.15.172:0/1378349680 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:39:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.312+0000 7f09721a5640 1 --2- 172.21.15.172:0/1378349680 >> v2:172.21.15.177:3300/0 conn(0x7f096c0d2e00 0x7f096c1629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.312+0000 7f09719a4640 1 --2- 172.21.15.172:0/1378349680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c15e430 0x7f096c162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.312+0000 7f09709a2640 1 -- 172.21.15.172:0/1378349680 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2918046768 0 0) 0x7f096c0cefb0 con 0x7f096c162540 2024-04-14T08:39:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.312+0000 7f09709a2640 1 -- 172.21.15.172:0/1378349680 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0958003520 con 0x7f096c162540 2024-04-14T08:39:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.312+0000 7f09721a5640 1 -- 172.21.15.172:0/1378349680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c15e430 msgr2=0x7f096c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.312+0000 7f09721a5640 1 --2- 172.21.15.172:0/1378349680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c15e430 0x7f096c162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.312+0000 7f09721a5640 1 -- 172.21.15.172:0/1378349680 >> v1:172.21.15.172:6789/0 conn(0x7f096c162540 legacy=0x7f096c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.312+0000 7f09721a5640 1 -- 172.21.15.172:0/1378349680 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c0cf850 con 0x7f096c0d2e00 2024-04-14T08:39:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.312+0000 7f09719a4640 1 --2- 172.21.15.172:0/1378349680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c15e430 0x7f096c162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:39:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.313+0000 7f09721a5640 1 --2- 172.21.15.172:0/1378349680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c0d2e00 0x7f096c1629a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0964009d90 tx=0x7f096402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bca149570f4df0f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.313+0000 7f09709a2640 1 -- 172.21.15.172:0/1378349680 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964039070 con 0x7f096c0d2e00 2024-04-14T08:39:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.313+0000 7f09709a2640 1 -- 172.21.15.172:0/1378349680 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096402cc50 con 0x7f096c0d2e00 2024-04-14T08:39:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.313+0000 7f09709a2640 1 -- 172.21.15.172:0/1378349680 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964034440 con 0x7f096c0d2e00 2024-04-14T08:39:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.313+0000 7f0973c08640 1 -- 172.21.15.172:0/1378349680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c0d2e00 msgr2=0x7f096c1629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.313+0000 7f0973c08640 1 --2- 172.21.15.172:0/1378349680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c0d2e00 0x7f096c1629a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0964009d90 tx=0x7f096402bea0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.314+0000 7f0973c08640 1 -- 172.21.15.172:0/1378349680 shutdown_connections 2024-04-14T08:39:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.314+0000 7f0973c08640 1 --2- 172.21.15.172:0/1378349680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c0d2e00 0x7f096c1629a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.314+0000 7f0973c08640 1 --2- 172.21.15.172:0/1378349680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c15e430 0x7f096c162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.314+0000 7f0973c08640 1 -- 172.21.15.172:0/1378349680 >> 172.21.15.172:0/1378349680 conn(0x7f096c09d560 msgr2=0x7f096c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.314+0000 7f0973c08640 1 -- 172.21.15.172:0/1378349680 shutdown_connections 2024-04-14T08:39:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.314+0000 7f0973c08640 1 -- 172.21.15.172:0/1378349680 wait complete. 2024-04-14T08:39:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.315+0000 7f0973c08640 1 Processor -- start 2024-04-14T08:39:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.315+0000 7f0973c08640 1 -- start start 2024-04-14T08:39:05.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.315+0000 7f0973c08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096c0d2e00 0x7f096c1f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.315+0000 7f0973c08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c15e430 0x7f096c1f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.315+0000 7f0973c08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c162540 0x7f096c15b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.315+0000 7f0973c08640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f096c0d4e20 con 0x7f096c0d2e00 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.315+0000 7f0973c08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f096c15e810 con 0x7f096c162540 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.315+0000 7f0973c08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f096c0d0050 con 0x7f096c15e430 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09719a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096c0d2e00 0x7f096c1f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09719a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096c0d2e00 0x7f096c1f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59152/0 (socket says 172.21.15.172:59152) 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09719a4640 1 -- 172.21.15.172:0/1020798702 learned_addr learned my addr 172.21.15.172:0/1020798702 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09711a3640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c15e430 0x7f096c1f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09721a5640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c162540 0x7f096c15b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09719a4640 1 -- 172.21.15.172:0/1020798702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c15e430 msgr2=0x7f096c1f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09719a4640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c15e430 0x7f096c1f7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09719a4640 1 -- 172.21.15.172:0/1020798702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c162540 msgr2=0x7f096c15b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09719a4640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c162540 0x7f096c15b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09719a4640 1 -- 172.21.15.172:0/1020798702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c1f4560 con 0x7f096c0d2e00 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09711a3640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c15e430 0x7f096c1f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09721a5640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c162540 0x7f096c15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09719a4640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096c0d2e00 0x7f096c1f51b0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f096800e960 tx=0x7f096800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09627fc640 1 -- 172.21.15.172:0/1020798702 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968017070 con 0x7f096c0d2e00 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09627fc640 1 -- 172.21.15.172:0/1020798702 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096800ca00 con 0x7f096c0d2e00 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f09627fc640 1 -- 172.21.15.172:0/1020798702 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096800cdd0 con 0x7f096c0d2e00 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f0973c08640 1 -- 172.21.15.172:0/1020798702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096c1f4840 con 0x7f096c0d2e00 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.316+0000 7f0973c08640 1 -- 172.21.15.172:0/1020798702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096c1f4e00 con 0x7f096c0d2e00 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.318+0000 7f09627fc640 1 -- 172.21.15.172:0/1020798702 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f096801a070 con 0x7f096c0d2e00 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.318+0000 7f09627fc640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f095006c8f0 0x7f095006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.318+0000 7f09627fc640 1 -- 172.21.15.172:0/1020798702 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f096808bbe0 con 0x7f096c0d2e00 2024-04-14T08:39:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.318+0000 7f09711a3640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f095006c8f0 0x7f095006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.319+0000 7f09711a3640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f095006c8f0 0x7f095006edb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f095c007fd0 tx=0x7f095c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.320+0000 7f09769ea740 1 Processor -- start 2024-04-14T08:39:05.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.320+0000 7f09769ea740 1 -- start start 2024-04-14T08:39:05.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09769ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 0x5582f9530910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09769ea740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955dd50 0x5582f9560700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09769ea740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f9530e50 0x5582f9560c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09769ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5582f949a800 con 0x5582f955d970 2024-04-14T08:39:05.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09769ea740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5582f94427f0 con 0x5582f955dd50 2024-04-14T08:39:05.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09769ea740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5582f94967b0 con 0x5582f9530e50 2024-04-14T08:39:05.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09719a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 0x5582f9530910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09719a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 0x5582f9530910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59158/0 (socket says 172.21.15.172:59158) 2024-04-14T08:39:05.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09719a4640 1 -- 172.21.15.172:0/887886575 learned_addr learned my addr 172.21.15.172:0/887886575 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:05.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09719a4640 1 -- 172.21.15.172:0/887886575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f9530e50 msgr2=0x5582f9560c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09719a4640 1 --2- 172.21.15.172:0/887886575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f9530e50 0x5582f9560c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09719a4640 1 -- 172.21.15.172:0/887886575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955dd50 msgr2=0x5582f9560700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09719a4640 1 --2- 172.21.15.172:0/887886575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955dd50 0x5582f9560700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.321+0000 7f09719a4640 1 -- 172.21.15.172:0/887886575 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f92787d0 con 0x5582f955d970 2024-04-14T08:39:05.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.322+0000 7f09719a4640 1 --2- 172.21.15.172:0/887886575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 0x5582f9530910 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f0968089f30 tx=0x7f0968060180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b439abc19b92707d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.322+0000 7f094f7fe640 1 -- 172.21.15.172:0/887886575 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968021050 con 0x5582f955d970 2024-04-14T08:39:05.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.322+0000 7f094f7fe640 1 -- 172.21.15.172:0/887886575 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0968062320 con 0x5582f955d970 2024-04-14T08:39:05.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.322+0000 7f094f7fe640 1 -- 172.21.15.172:0/887886575 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968057a40 con 0x5582f955d970 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.322+0000 7f09769ea740 1 -- 172.21.15.172:0/887886575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 msgr2=0x5582f9530910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.322+0000 7f09769ea740 1 --2- 172.21.15.172:0/887886575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 0x5582f9530910 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f0968089f30 tx=0x7f0968060180 comp rx=0 tx=0).stop 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.322+0000 7f09769ea740 1 -- 172.21.15.172:0/887886575 shutdown_connections 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.322+0000 7f09769ea740 1 --2- 172.21.15.172:0/887886575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f9530e50 0x5582f9560c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.322+0000 7f09769ea740 1 --2- 172.21.15.172:0/887886575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955dd50 0x5582f9560700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.322+0000 7f09769ea740 1 --2- 172.21.15.172:0/887886575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 0x5582f9530910 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.322+0000 7f09769ea740 1 -- 172.21.15.172:0/887886575 >> 172.21.15.172:0/887886575 conn(0x5582f9532b80 msgr2=0x5582f957b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.322+0000 7f09769ea740 1 -- 172.21.15.172:0/887886575 shutdown_connections 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.323+0000 7f09769ea740 1 -- 172.21.15.172:0/887886575 wait complete. 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.323+0000 7f09769ea740 1 -- start start 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.323+0000 7f09769ea740 1 Processor -- start 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09769ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 0x5582f9562fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09769ea740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f955dd50 0x5582f954bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09769ea740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f954c280 0x5582f9551650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09769ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5582f949a800 con 0x5582f955d970 2024-04-14T08:39:05.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09769ea740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5582f9455d00 con 0x5582f954c280 2024-04-14T08:39:05.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09769ea740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5582f9535120 con 0x5582f955dd50 2024-04-14T08:39:05.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09769ea740 10 client.0 _ll_register_callbacks cb 0x5582f94556f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:05.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09719a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 0x5582f9562fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09719a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 0x5582f9562fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59162/0 (socket says 172.21.15.172:59162) 2024-04-14T08:39:05.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09719a4640 1 -- 172.21.15.172:0/2285585950 learned_addr learned my addr 172.21.15.172:0/2285585950 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:05.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09721a5640 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f954c280 0x5582f9551650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09711a3640 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f955dd50 0x5582f954bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09719a4640 1 -- 172.21.15.172:0/2285585950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f955dd50 msgr2=0x5582f954bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09719a4640 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f955dd50 0x5582f954bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09719a4640 1 -- 172.21.15.172:0/2285585950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f954c280 msgr2=0x5582f9551650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09719a4640 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f954c280 0x5582f9551650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.324+0000 7f09719a4640 1 -- 172.21.15.172:0/2285585950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f91ae3f0 con 0x5582f955d970 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f09719a4640 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 0x5582f9562fc0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f096800e930 tx=0x7f0968000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53471c1af5f14cd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f094cff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f094cff9640 1 -- 172.21.15.172:0/2285585950 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968058b90 con 0x5582f955d970 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f094cff9640 1 -- 172.21.15.172:0/2285585950 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f0968058d30 con 0x5582f955d970 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f09769ea740 1 -- 172.21.15.172:0/2285585950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0968062320 con 0x5582f955d970 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f094cff9640 1 -- 172.21.15.172:0/2285585950 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f0968021020 con 0x5582f955d970 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f09769ea740 10 client.7815 fetch_fsmap learned FSMap version 60 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f09769ea740 1 -- 172.21.15.172:0/2285585950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x5582f916e0f0 con 0x5582f955d970 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f094cff9640 1 -- 172.21.15.172:0/2285585950 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0968062320 con 0x5582f955d970 2024-04-14T08:39:05.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f094cff9640 1 -- 172.21.15.172:0/2285585950 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 60) ==== 3494+0+0 (secure 0 0 0) 0x7f0968024060 con 0x5582f955d970 2024-04-14T08:39:05.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f09769ea740 10 client.7815 fetch_fsmap finished waiting for FSMap version 60 2024-04-14T08:39:05.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f09769ea740 10 client.7815 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:39:05.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f09769ea740 20 client.7815 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:05.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f09769ea740 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x5582f9576200 0x5582f95765e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f09769ea740 4 client.7815 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:39:05.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f09769ea740 1 -- 172.21.15.172:0/2285585950 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582f94a7d40 con 0x5582f9576200 2024-04-14T08:39:05.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.325+0000 7f09711a3640 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x5582f9576200 0x5582f95765e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f094cff9640 10 client.7815 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:05.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f09711a3640 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x5582f9576200 0x5582f95765e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=285a958d04b70e73 server_cookie=9b65f372970ea330 in_seq=0 out_seq=0 2024-04-14T08:39:05.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f094cff9640 1 -- 172.21.15.172:0/2285585950 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582f94a7d40 con 0x5582f9576200 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f094cff9640 10 client.7815 handle_command_reply: tid=0 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f09769ea740 1 client.7815 shutdown 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f09769ea740 1 -- 172.21.15.172:0/2285585950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 msgr2=0x5582f9562fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f09769ea740 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 0x5582f9562fc0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f096800e930 tx=0x7f0968000bc0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f09769ea740 1 -- 172.21.15.172:0/2285585950 shutdown_connections 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f09769ea740 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x5582f9576200 0x5582f95765e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f09769ea740 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f954c280 0x5582f9551650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f09769ea740 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f955dd50 0x5582f954bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f094cff9640 0 client.7815 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f09769ea740 1 --2- 172.21.15.172:0/2285585950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955d970 0x5582f9562fc0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.326+0000 7f09769ea740 1 -- 172.21.15.172:0/2285585950 >> 172.21.15.172:0/2285585950 conn(0x5582f9530e50 msgr2=0x5582f95606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.327+0000 7f09769ea740 1 -- 172.21.15.172:0/2285585950 shutdown_connections 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.327+0000 7f09769ea740 1 -- 172.21.15.172:0/2285585950 wait complete. 2024-04-14T08:39:05.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.327+0000 7f09769ea740 20 client.7815 trim_cache size 0 max 16384 2024-04-14T08:39:05.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.344+0000 7f09769ea740 1 Processor -- start 2024-04-14T08:39:05.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.344+0000 7f09769ea740 1 -- start start 2024-04-14T08:39:05.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09769ea740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f954c280 0x5582f9563fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09769ea740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955d970 0x5582f9575660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09769ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955dd50 0x5582f959ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09769ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f096c201050 con 0x5582f955dd50 2024-04-14T08:39:05.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09769ea740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f096c0b85b0 con 0x5582f955d970 2024-04-14T08:39:05.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09769ea740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f096c161440 con 0x5582f954c280 2024-04-14T08:39:05.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09721a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955dd50 0x5582f959ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09721a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955dd50 0x5582f959ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59176/0 (socket says 172.21.15.172:59176) 2024-04-14T08:39:05.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09721a5640 1 -- 172.21.15.172:0/3383825620 learned_addr learned my addr 172.21.15.172:0/3383825620 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:05.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09719a4640 1 --2- 172.21.15.172:0/3383825620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f954c280 0x5582f9563fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09711a3640 1 --2- 172.21.15.172:0/3383825620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955d970 0x5582f9575660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09721a5640 1 -- 172.21.15.172:0/3383825620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f954c280 msgr2=0x5582f9563fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09721a5640 1 --2- 172.21.15.172:0/3383825620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f954c280 0x5582f9563fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09721a5640 1 -- 172.21.15.172:0/3383825620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955d970 msgr2=0x5582f9575660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09721a5640 1 --2- 172.21.15.172:0/3383825620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955d970 0x5582f9575660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.345+0000 7f09721a5640 1 -- 172.21.15.172:0/3383825620 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f9576950 con 0x5582f955dd50 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f09721a5640 1 --2- 172.21.15.172:0/3383825620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955dd50 0x5582f959ea70 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f0964009eb0 tx=0x7f096402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61bef290f78788d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f094effd640 1 -- 172.21.15.172:0/3383825620 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964045030 con 0x5582f955dd50 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f094effd640 1 -- 172.21.15.172:0/3383825620 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096403a040 con 0x5582f955dd50 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f094effd640 1 -- 172.21.15.172:0/3383825620 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964039070 con 0x5582f955dd50 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f09769ea740 1 -- 172.21.15.172:0/3383825620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955dd50 msgr2=0x5582f959ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f09769ea740 1 --2- 172.21.15.172:0/3383825620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955dd50 0x5582f959ea70 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f0964009eb0 tx=0x7f096402e070 comp rx=0 tx=0).stop 2024-04-14T08:39:05.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f09769ea740 1 -- 172.21.15.172:0/3383825620 shutdown_connections 2024-04-14T08:39:05.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f09769ea740 1 --2- 172.21.15.172:0/3383825620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f955dd50 0x5582f959ea70 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f09769ea740 1 --2- 172.21.15.172:0/3383825620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955d970 0x5582f9575660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f09769ea740 1 --2- 172.21.15.172:0/3383825620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f954c280 0x5582f9563fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f09769ea740 1 -- 172.21.15.172:0/3383825620 >> 172.21.15.172:0/3383825620 conn(0x5582f9530e50 msgr2=0x5582f952c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:05.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f09769ea740 1 -- 172.21.15.172:0/3383825620 shutdown_connections 2024-04-14T08:39:05.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.346+0000 7f09769ea740 1 -- 172.21.15.172:0/3383825620 wait complete. 2024-04-14T08:39:05.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.347+0000 7f09769ea740 1 -- start start 2024-04-14T08:39:05.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.347+0000 7f09769ea740 1 Processor -- start 2024-04-14T08:39:05.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09769ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f954c280 0x5582f9560580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09769ea740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f955d970 0x5582f95f4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09769ea740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955dd50 0x5582f95f4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09769ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5582f9576720 con 0x5582f954c280 2024-04-14T08:39:05.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09769ea740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f096c0b7f60 con 0x5582f955dd50 2024-04-14T08:39:05.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09769ea740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5582f95745c0 con 0x5582f955d970 2024-04-14T08:39:05.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09769ea740 10 client.0 _ll_register_callbacks cb 0x7f096805dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:05.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09719a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f954c280 0x5582f9560580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09719a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f954c280 0x5582f9560580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59180/0 (socket says 172.21.15.172:59180) 2024-04-14T08:39:05.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09719a4640 1 -- 172.21.15.172:0/3928792957 learned_addr learned my addr 172.21.15.172:0/3928792957 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:05.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09711a3640 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f955d970 0x5582f95f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09721a5640 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955dd50 0x5582f95f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09719a4640 1 -- 172.21.15.172:0/3928792957 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f955d970 msgr2=0x5582f95f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09719a4640 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f955d970 0x5582f95f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09719a4640 1 -- 172.21.15.172:0/3928792957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955dd50 msgr2=0x5582f95f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:05.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09719a4640 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955dd50 0x5582f95f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09719a4640 1 -- 172.21.15.172:0/3928792957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f9571ff0 con 0x5582f954c280 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09721a5640 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955dd50 0x5582f95f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f09719a4640 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f954c280 0x5582f9560580 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f096808bf00 tx=0x7f096800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8468ac76d684d11d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f094cff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f094cff9640 1 -- 172.21.15.172:0/3928792957 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09680559e0 con 0x5582f954c280 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.348+0000 7f094cff9640 1 -- 172.21.15.172:0/3928792957 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f0968081ce0 con 0x5582f954c280 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f09769ea740 1 -- 172.21.15.172:0/3928792957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f096403a040 con 0x5582f954c280 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f094cff9640 1 -- 172.21.15.172:0/3928792957 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f0968055800 con 0x5582f954c280 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f09769ea740 10 client.7821 fetch_fsmap learned FSMap version 60 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f09769ea740 1 -- 172.21.15.172:0/3928792957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x5582f9571b30 con 0x5582f954c280 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f094cff9640 1 -- 172.21.15.172:0/3928792957 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096403a040 con 0x5582f954c280 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f094cff9640 1 -- 172.21.15.172:0/3928792957 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 60) ==== 3494+0+0 (secure 0 0 0) 0x7f0968058600 con 0x5582f954c280 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f09769ea740 10 client.7821 fetch_fsmap finished waiting for FSMap version 60 2024-04-14T08:39:05.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f09769ea740 10 client.7821 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f09769ea740 20 client.7821 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f09769ea740 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x5582f952c1c0 0x5582f955bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f09769ea740 4 client.7821 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f09769ea740 1 -- 172.21.15.172:0/3928792957 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582f9577a70 con 0x5582f952c1c0 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f09711a3640 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x5582f952c1c0 0x5582f955bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f094cff9640 10 client.7821 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.349+0000 7f09711a3640 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x5582f952c1c0 0x5582f955bf70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8440581d27e8be7 server_cookie=a81ff1a171e436d4 in_seq=0 out_seq=0 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f094cff9640 1 -- 172.21.15.172:0/3928792957 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x5582f9577a70 con 0x5582f952c1c0 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f094cff9640 10 client.7821 handle_command_reply: tid=0 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f09769ea740 1 client.7821 shutdown 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f09769ea740 1 -- 172.21.15.172:0/3928792957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f954c280 msgr2=0x5582f9560580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f09769ea740 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f954c280 0x5582f9560580 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f096808bf00 tx=0x7f096800efd0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f09769ea740 1 -- 172.21.15.172:0/3928792957 shutdown_connections 2024-04-14T08:39:05.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f09769ea740 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x5582f952c1c0 0x5582f955bf70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f09769ea740 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582f955dd50 0x5582f95f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f09769ea740 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5582f955d970 0x5582f95f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f09769ea740 1 --2- 172.21.15.172:0/3928792957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582f954c280 0x5582f9560580 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f094cff9640 0 client.7821 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:05.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f09769ea740 1 -- 172.21.15.172:0/3928792957 >> 172.21.15.172:0/3928792957 conn(0x5582f9530e50 msgr2=0x5582f9563940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:05.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f09769ea740 1 -- 172.21.15.172:0/3928792957 shutdown_connections 2024-04-14T08:39:05.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f09769ea740 1 -- 172.21.15.172:0/3928792957 wait complete. 2024-04-14T08:39:05.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f09769ea740 20 client.7821 trim_cache size 0 max 16384 2024-04-14T08:39:05.361 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:39:05.361 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-14T08:39:05.361 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-04-14T08:39:05.361 INFO:teuthology.orchestra.run.smithi172.stdout: "b017c06e-5628-4d03-a4ce-39439a7f35a9": { 2024-04-14T08:39:05.361 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/dir", 2024-04-14T08:39:05.362 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "b017c06e-5628-4d03-a4ce-39439a7f35a9", 2024-04-14T08:39:05.362 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive" 2024-04-14T08:39:05.362 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:39:05.362 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:39:05.362 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:39:05.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f0973c08640 1 -- 172.21.15.172:0/1020798702 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f095006c8f0 msgr2=0x7f095006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f0973c08640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f095006c8f0 0x7f095006edb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f095c007fd0 tx=0x7f095c009040 comp rx=0 tx=0).stop 2024-04-14T08:39:05.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f0973c08640 1 -- 172.21.15.172:0/1020798702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096c0d2e00 msgr2=0x7f096c1f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:05.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f0973c08640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096c0d2e00 0x7f096c1f51b0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f096800e960 tx=0x7f096800ee20 comp rx=0 tx=0).stop 2024-04-14T08:39:05.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f0973c08640 1 -- 172.21.15.172:0/1020798702 shutdown_connections 2024-04-14T08:39:05.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f0973c08640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f095006c8f0 0x7f095006edb0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f0973c08640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c162540 0x7f096c15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f0973c08640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c15e430 0x7f096c1f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f0973c08640 1 --2- 172.21.15.172:0/1020798702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f096c0d2e00 0x7f096c1f51b0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:05.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.350+0000 7f0973c08640 1 -- 172.21.15.172:0/1020798702 >> 172.21.15.172:0/1020798702 conn(0x7f096c09d560 msgr2=0x7f096c0ce530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:05.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.351+0000 7f0973c08640 1 -- 172.21.15.172:0/1020798702 shutdown_connections 2024-04-14T08:39:05.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:05.351+0000 7f0973c08640 1 -- 172.21.15.172:0/1020798702 wait complete. 2024-04-14T08:39:05.366 INFO:tasks.cephfs.filesystem:scrub status for tag:b017c06e-5628-4d03-a4ce-39439a7f35a9 - {'path': '/dir', 'tag': 'b017c06e-5628-4d03-a4ce-39439a7f35a9', 'options': 'recursive'} 2024-04-14T08:39:05.424 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:39:05.423+0000 7f0f2508f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-14T08:39:07.198 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:39:07.196+0000 7effc0ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:39:24.355 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:39:24.359 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:39:35.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:35.367 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-14T08:39:35.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.472+0000 7f8b16836640 1 Processor -- start 2024-04-14T08:39:35.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.472+0000 7f8b16836640 1 -- start start 2024-04-14T08:39:35.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.472+0000 7f8b16836640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8b1015e430 0x7f8b10162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.472+0000 7f8b16836640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8b100d2e00 0x7f8b101629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.472+0000 7f8b16836640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b100cefb0 con 0x7f8b10162540 2024-04-14T08:39:35.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.472+0000 7f8b16836640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f8b100cf290 con 0x7f8b100d2e00 2024-04-14T08:39:35.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.472+0000 7f8b16836640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f8b100cf570 con 0x7f8b1015e430 2024-04-14T08:39:35.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.473+0000 7f8b14dd3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8b100d2e00 0x7f8b101629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.473+0000 7f8b0f7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f8b10162540 0x7f8b100b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:55992/0 (socket says 172.21.15.172:55992) 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.473+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/2627626141 learned_addr learned my addr 172.21.15.172:0/2627626141 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.473+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/2627626141 >> v2:172.21.15.177:3300/0 conn(0x7f8b1015e430 0x7f8b10162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.473+0000 7f8b14dd3640 1 -- 172.21.15.172:0/2627626141 >> v2:172.21.15.177:3300/0 conn(0x7f8b1015e430 msgr2=0x7f8b10162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.473+0000 7f8b14dd3640 1 --2- 172.21.15.172:0/2627626141 >> v2:172.21.15.177:3300/0 conn(0x7f8b1015e430 0x7f8b10162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.473+0000 7f8b14dd3640 1 -- 172.21.15.172:0/2627626141 >> v1:172.21.15.172:6789/0 conn(0x7f8b10162540 legacy=0x7f8b100b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.473+0000 7f8b0effd640 1 -- 172.21.15.172:0/2627626141 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1757615572 0 0) 0x7f8b100cefb0 con 0x7f8b10162540 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.473+0000 7f8b14dd3640 1 -- 172.21.15.172:0/2627626141 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b100cf850 con 0x7f8b100d2e00 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.473+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/2627626141 >> v2:172.21.15.177:3300/0 conn(0x7f8b1015e430 0x7f8b10162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.474+0000 7f8b14dd3640 1 --2- 172.21.15.172:0/2627626141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b100d2e00 0x7f8b101629d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8b100b6db0 tx=0x7f8b0802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acc18fb2fcad8e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.474+0000 7f8b0effd640 1 -- 172.21.15.172:0/2627626141 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b08039070 con 0x7f8b100d2e00 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.474+0000 7f8b0effd640 1 -- 172.21.15.172:0/2627626141 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b0802cc80 con 0x7f8b100d2e00 2024-04-14T08:39:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.474+0000 7f8b0effd640 1 -- 172.21.15.172:0/2627626141 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b08034660 con 0x7f8b100d2e00 2024-04-14T08:39:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.474+0000 7f8b16836640 1 -- 172.21.15.172:0/2627626141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b100d2e00 msgr2=0x7f8b101629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.474+0000 7f8b16836640 1 --2- 172.21.15.172:0/2627626141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b100d2e00 0x7f8b101629d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8b100b6db0 tx=0x7f8b0802bed0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.474+0000 7f8b16836640 1 -- 172.21.15.172:0/2627626141 shutdown_connections 2024-04-14T08:39:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.474+0000 7f8b16836640 1 --2- 172.21.15.172:0/2627626141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b100d2e00 0x7f8b101629d0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.474+0000 7f8b16836640 1 --2- 172.21.15.172:0/2627626141 >> v2:172.21.15.177:3300/0 conn(0x7f8b1015e430 0x7f8b10162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.474+0000 7f8b16836640 1 -- 172.21.15.172:0/2627626141 >> 172.21.15.172:0/2627626141 conn(0x7f8b1009d560 msgr2=0x7f8b100d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.475+0000 7f8b16836640 1 -- 172.21.15.172:0/2627626141 shutdown_connections 2024-04-14T08:39:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.475+0000 7f8b16836640 1 -- 172.21.15.172:0/2627626141 wait complete. 2024-04-14T08:39:35.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.476+0000 7f8b16836640 1 Processor -- start 2024-04-14T08:39:35.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.476+0000 7f8b16836640 1 -- start start 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.476+0000 7f8b16836640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b100d2e00 0x7f8b100b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.476+0000 7f8b16836640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b1015e430 0x7f8b100c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.476+0000 7f8b16836640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b10162540 0x7f8b100b9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b16836640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8b100d4e20 con 0x7f8b1015e430 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b16836640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1015e810 con 0x7f8b100d2e00 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b16836640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8b100d0050 con 0x7f8b10162540 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b1015e430 0x7f8b100c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b1015e430 0x7f8b100c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57114/0 (socket says 172.21.15.172:57114) 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/788526292 learned_addr learned my addr 172.21.15.172:0/788526292 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b100d2e00 0x7f8b100b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b14dd3640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b10162540 0x7f8b100b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/788526292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b10162540 msgr2=0x7f8b100b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b10162540 0x7f8b100b9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/788526292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b100d2e00 msgr2=0x7f8b100b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b100d2e00 0x7f8b100b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/788526292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b100c9ca0 con 0x7f8b1015e430 2024-04-14T08:39:35.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b14dd3640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b10162540 0x7f8b100b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:35.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b1015e430 0x7f8b100c9760 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400c430 tx=0x7f8b0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:35.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0cff9640 1 -- 172.21.15.172:0/788526292 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040074e0 con 0x7f8b1015e430 2024-04-14T08:39:35.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0cff9640 1 -- 172.21.15.172:0/788526292 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b040047e0 con 0x7f8b1015e430 2024-04-14T08:39:35.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b0cff9640 1 -- 172.21.15.172:0/788526292 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04004bd0 con 0x7f8b1015e430 2024-04-14T08:39:35.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b16836640 1 -- 172.21.15.172:0/788526292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b100c9f20 con 0x7f8b1015e430 2024-04-14T08:39:35.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.477+0000 7f8b16836640 1 -- 172.21.15.172:0/788526292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b100ca4e0 con 0x7f8b1015e430 2024-04-14T08:39:35.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.479+0000 7f8b0cff9640 1 -- 172.21.15.172:0/788526292 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b04011070 con 0x7f8b1015e430 2024-04-14T08:39:35.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.480+0000 7f8b0cff9640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8afc06c8f0 0x7f8afc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.480+0000 7f8b0cff9640 1 -- 172.21.15.172:0/788526292 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6303+0+0 (secure 0 0 0) 0x7f8b0408b990 con 0x7f8b1015e430 2024-04-14T08:39:35.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.480+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8afc06c8f0 0x7f8afc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.480+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8afc06c8f0 0x7f8afc06edb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8b101f5160 tx=0x7f8b00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.481+0000 7f8b1961f740 1 Processor -- start 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.481+0000 7f8b1961f740 1 -- start start 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b1961f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8d970 0x56255df60910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b1961f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8dd50 0x56255df90700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b1961f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df60e50 0x56255df90c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b1961f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56255deca800 con 0x56255df8dd50 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b1961f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56255de727f0 con 0x56255df8d970 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b1961f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56255dec67b0 con 0x56255df60e50 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8dd50 0x56255df90700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8dd50 0x56255df90700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57130/0 (socket says 172.21.15.172:57130) 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/4191134523 learned_addr learned my addr 172.21.15.172:0/4191134523 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/4191134523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8d970 0x56255df60910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b14dd3640 1 --2- 172.21.15.172:0/4191134523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df60e50 0x56255df90c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/4191134523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df60e50 msgr2=0x56255df90c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/4191134523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df60e50 0x56255df90c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/4191134523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8d970 msgr2=0x56255df60910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/4191134523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8d970 0x56255df60910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/4191134523 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255dca87d0 con 0x56255df8dd50 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.482+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/4191134523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8d970 0x56255df60910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.483+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/4191134523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8dd50 0x56255df90700 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8b04089c10 tx=0x7f8b04058e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3b5242da41d3cb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.483+0000 7f8af5ffb640 1 -- 172.21.15.172:0/4191134523 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0405d820 con 0x56255df8dd50 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.483+0000 7f8af5ffb640 1 -- 172.21.15.172:0/4191134523 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b04062050 con 0x56255df8dd50 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.483+0000 7f8af5ffb640 1 -- 172.21.15.172:0/4191134523 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04004d70 con 0x56255df8dd50 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.483+0000 7f8b1961f740 1 -- 172.21.15.172:0/4191134523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8dd50 msgr2=0x56255df90700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.483+0000 7f8b1961f740 1 --2- 172.21.15.172:0/4191134523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8dd50 0x56255df90700 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8b04089c10 tx=0x7f8b04058e50 comp rx=0 tx=0).stop 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.484+0000 7f8b1961f740 1 -- 172.21.15.172:0/4191134523 shutdown_connections 2024-04-14T08:39:35.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.484+0000 7f8b1961f740 1 --2- 172.21.15.172:0/4191134523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df60e50 0x56255df90c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.484+0000 7f8b1961f740 1 --2- 172.21.15.172:0/4191134523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8dd50 0x56255df90700 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.484+0000 7f8b1961f740 1 --2- 172.21.15.172:0/4191134523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8d970 0x56255df60910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.484+0000 7f8b1961f740 1 -- 172.21.15.172:0/4191134523 >> 172.21.15.172:0/4191134523 conn(0x56255df62b80 msgr2=0x56255dfab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:35.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.484+0000 7f8b1961f740 1 -- 172.21.15.172:0/4191134523 shutdown_connections 2024-04-14T08:39:35.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.484+0000 7f8b1961f740 1 -- 172.21.15.172:0/4191134523 wait complete. 2024-04-14T08:39:35.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.484+0000 7f8b1961f740 1 -- start start 2024-04-14T08:39:35.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.485+0000 7f8b1961f740 1 Processor -- start 2024-04-14T08:39:35.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b1961f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8d970 0x56255df81b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b1961f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8dd50 0x56255df93630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b1961f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df927f0 0x56255dfa5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b1961f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56255deca800 con 0x56255df8d970 2024-04-14T08:39:35.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b1961f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56255de85d00 con 0x56255df927f0 2024-04-14T08:39:35.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b1961f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56255df65120 con 0x56255df8dd50 2024-04-14T08:39:35.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b1961f740 10 client.0 _ll_register_callbacks cb 0x56255de856f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:35.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b0f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8dd50 0x56255df93630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b0f7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8dd50 0x56255df93630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:52782/0 (socket says 172.21.15.172:52782) 2024-04-14T08:39:35.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/3321895291 learned_addr learned my addr 172.21.15.172:0/3321895291 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:35.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8d970 0x56255df81b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b14dd3640 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df927f0 0x56255dfa5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/3321895291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df927f0 msgr2=0x56255dfa5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:35.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df927f0 0x56255dfa5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/3321895291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8d970 msgr2=0x56255df81b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:35.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8d970 0x56255df81b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.486+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/3321895291 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x56255dbde3f0 con 0x56255df8dd50 2024-04-14T08:39:35.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.487+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8dd50 0x56255df93630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8b04007fd0 tx=0x7f8b04000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=999e98780fdfe3ce server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:35.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.487+0000 7f8aeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.177:3301/0 2024-04-14T08:39:35.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.491+0000 7f8aeeffd640 1 -- 172.21.15.172:0/3321895291 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040588c0 con 0x56255df8dd50 2024-04-14T08:39:35.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.491+0000 7f8aeeffd640 1 -- 172.21.15.172:0/3321895291 <== mon.2 v2:172.21.15.177:3301/0 2 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f8b04058a60 con 0x56255df8dd50 2024-04-14T08:39:35.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.491+0000 7f8b1961f740 1 -- 172.21.15.172:0/3321895291 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b04062050 con 0x56255df8dd50 2024-04-14T08:39:35.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.491+0000 7f8aeeffd640 1 -- 172.21.15.172:0/3321895291 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8b040551c0 con 0x56255df8dd50 2024-04-14T08:39:35.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.491+0000 7f8b1961f740 10 client.6536 fetch_fsmap learned FSMap version 61 2024-04-14T08:39:35.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.491+0000 7f8b1961f740 1 -- 172.21.15.172:0/3321895291 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x56255db9e0f0 con 0x56255df8dd50 2024-04-14T08:39:35.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.492+0000 7f8aeeffd640 1 -- 172.21.15.172:0/3321895291 <== mon.2 v2:172.21.15.177:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b04062050 con 0x56255df8dd50 2024-04-14T08:39:35.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.492+0000 7f8aeeffd640 1 -- 172.21.15.172:0/3321895291 <== mon.2 v2:172.21.15.177:3301/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f8b04055360 con 0x56255df8dd50 2024-04-14T08:39:35.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.492+0000 7f8b1961f740 10 client.6536 fetch_fsmap finished waiting for FSMap version 61 2024-04-14T08:39:35.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.492+0000 7f8b1961f740 10 client.6536 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:39:35.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.492+0000 7f8b1961f740 20 client.6536 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:35.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.492+0000 7f8b1961f740 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56255dfbe850 0x56255dfbec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.492+0000 7f8b1961f740 4 client.6536 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:39:35.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.492+0000 7f8b1961f740 1 -- 172.21.15.172:0/3321895291 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56255ded7d40 con 0x56255dfbe850 2024-04-14T08:39:35.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.492+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56255dfbe850 0x56255dfbec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.492+0000 7f8aeeffd640 10 client.6536 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:35.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.492+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56255dfbe850 0x56255dfbec50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dab5d638454b269 server_cookie=c5c6bd33de4766be in_seq=0 out_seq=0 2024-04-14T08:39:35.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.493+0000 7f8aeeffd640 1 -- 172.21.15.172:0/3321895291 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56255ded7d40 con 0x56255dfbe850 2024-04-14T08:39:35.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.493+0000 7f8aeeffd640 10 client.6536 handle_command_reply: tid=0 2024-04-14T08:39:35.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.493+0000 7f8b1961f740 1 client.6536 shutdown 2024-04-14T08:39:35.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.493+0000 7f8b1961f740 1 -- 172.21.15.172:0/3321895291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8dd50 msgr2=0x56255df93630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.493+0000 7f8b1961f740 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8dd50 0x56255df93630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8b04007fd0 tx=0x7f8b04000ed0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.493+0000 7f8b1961f740 1 -- 172.21.15.172:0/3321895291 shutdown_connections 2024-04-14T08:39:35.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.493+0000 7f8b1961f740 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56255dfbe850 0x56255dfbec50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.493+0000 7f8b1961f740 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df927f0 0x56255dfa5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.493+0000 7f8b1961f740 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8dd50 0x56255df93630 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.493+0000 7f8aeeffd640 0 client.6536 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:35.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.493+0000 7f8b1961f740 1 --2- 172.21.15.172:0/3321895291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8d970 0x56255df81b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.494+0000 7f8b1961f740 1 -- 172.21.15.172:0/3321895291 >> 172.21.15.172:0/3321895291 conn(0x56255df60e50 msgr2=0x56255df64930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:35.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.494+0000 7f8b1961f740 1 -- 172.21.15.172:0/3321895291 shutdown_connections 2024-04-14T08:39:35.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.494+0000 7f8b1961f740 1 -- 172.21.15.172:0/3321895291 wait complete. 2024-04-14T08:39:35.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.494+0000 7f8b1961f740 20 client.6536 trim_cache size 0 max 16384 2024-04-14T08:39:35.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.511+0000 7f8b1961f740 1 Processor -- start 2024-04-14T08:39:35.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.511+0000 7f8b1961f740 1 -- start start 2024-04-14T08:39:35.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b1961f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df60e50 0x56255dfaf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b1961f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8d970 0x56255df64470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b1961f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8dd50 0x56255df649b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b1961f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10201110 con 0x56255df60e50 2024-04-14T08:39:35.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b1961f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10200130 con 0x56255df8dd50 2024-04-14T08:39:35.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b1961f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8b10160fb0 con 0x56255df8d970 2024-04-14T08:39:35.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b0ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df60e50 0x56255dfaf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b0ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df60e50 0x56255dfaf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57146/0 (socket says 172.21.15.172:57146) 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b0ffff640 1 -- 172.21.15.172:0/687492899 learned_addr learned my addr 172.21.15.172:0/687492899 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b0ffff640 1 -- 172.21.15.172:0/687492899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8d970 msgr2=0x56255df64470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/687492899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8d970 0x56255df64470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b14dd3640 1 --2- 172.21.15.172:0/687492899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8dd50 0x56255df649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/687492899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8d970 0x56255df64470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b0ffff640 1 -- 172.21.15.172:0/687492899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8dd50 msgr2=0x56255df649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/687492899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8dd50 0x56255df649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b0ffff640 1 -- 172.21.15.172:0/687492899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255df62860 con 0x56255df60e50 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/687492899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8d970 0x56255df64470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b14dd3640 1 --2- 172.21.15.172:0/687492899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8dd50 0x56255df649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/687492899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df60e50 0x56255dfaf580 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000a830 tx=0x7f8b00015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f520af2695192f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8af57fa640 1 -- 172.21.15.172:0/687492899 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0001b050 con 0x56255df60e50 2024-04-14T08:39:35.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8af57fa640 1 -- 172.21.15.172:0/687492899 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b0001c040 con 0x56255df60e50 2024-04-14T08:39:35.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.512+0000 7f8af57fa640 1 -- 172.21.15.172:0/687492899 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00014b30 con 0x56255df60e50 2024-04-14T08:39:35.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.513+0000 7f8b1961f740 1 -- 172.21.15.172:0/687492899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df60e50 msgr2=0x56255dfaf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.513+0000 7f8b1961f740 1 --2- 172.21.15.172:0/687492899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df60e50 0x56255dfaf580 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000a830 tx=0x7f8b00015000 comp rx=0 tx=0).stop 2024-04-14T08:39:35.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.513+0000 7f8b1961f740 1 -- 172.21.15.172:0/687492899 shutdown_connections 2024-04-14T08:39:35.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.513+0000 7f8b1961f740 1 --2- 172.21.15.172:0/687492899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8dd50 0x56255df649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.513+0000 7f8b1961f740 1 --2- 172.21.15.172:0/687492899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df8d970 0x56255df64470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.513+0000 7f8b1961f740 1 --2- 172.21.15.172:0/687492899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df60e50 0x56255dfaf580 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.513+0000 7f8b1961f740 1 -- 172.21.15.172:0/687492899 >> 172.21.15.172:0/687492899 conn(0x56255dfa52b0 msgr2=0x56255df5f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:35.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.513+0000 7f8b1961f740 1 -- 172.21.15.172:0/687492899 shutdown_connections 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.513+0000 7f8b1961f740 1 -- 172.21.15.172:0/687492899 wait complete. 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.514+0000 7f8b1961f740 1 -- start start 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.514+0000 7f8b1961f740 1 Processor -- start 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b1961f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df60e50 0x56255df938c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b1961f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8d970 0x56255dfabd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b1961f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8dd50 0x56255df65610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b1961f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56255df81120 con 0x56255df8d970 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b1961f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8b101ffb10 con 0x56255df8dd50 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b1961f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56255dfa6f10 con 0x56255df60e50 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b1961f740 10 client.0 _ll_register_callbacks cb 0x7f8b04059530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8d970 0x56255dfabd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8d970 0x56255dfabd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57148/0 (socket says 172.21.15.172:57148) 2024-04-14T08:39:35.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/3650909280 learned_addr learned my addr 172.21.15.172:0/3650909280 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b14dd3640 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8dd50 0x56255df65610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df60e50 0x56255df938c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/3650909280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df60e50 msgr2=0x56255df938c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df60e50 0x56255df938c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/3650909280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8dd50 msgr2=0x56255df65610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8dd50 0x56255df65610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b0f7fe640 1 -- 172.21.15.172:0/3650909280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56255dfbaa00 con 0x56255df8d970 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df60e50 0x56255df938c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b0f7fe640 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8d970 0x56255dfabd80 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8b040593b0 tx=0x7f8b04019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84836d258f0aca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8aeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8aeeffd640 1 -- 172.21.15.172:0/3650909280 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040586c0 con 0x56255df8d970 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8aeeffd640 1 -- 172.21.15.172:0/3650909280 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f8b0405e9b0 con 0x56255df8d970 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.515+0000 7f8b1961f740 1 -- 172.21.15.172:0/3650909280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b0001c040 con 0x56255df8d970 2024-04-14T08:39:35.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8aeeffd640 1 -- 172.21.15.172:0/3650909280 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8b04058ad0 con 0x56255df8d970 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8b1961f740 10 client.7842 fetch_fsmap learned FSMap version 61 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8b1961f740 1 -- 172.21.15.172:0/3650909280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x56255dfba690 con 0x56255df8d970 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8aeeffd640 1 -- 172.21.15.172:0/3650909280 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b0001c040 con 0x56255df8d970 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8aeeffd640 1 -- 172.21.15.172:0/3650909280 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f8b0405ebb0 con 0x56255df8d970 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8b1961f740 10 client.7842 fetch_fsmap finished waiting for FSMap version 61 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8b1961f740 10 client.7842 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8b1961f740 20 client.7842 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8b1961f740 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56255df927f0 0x56255df81120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8b1961f740 4 client.7842 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8b1961f740 1 -- 172.21.15.172:0/3650909280 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56255ded2090 con 0x56255df927f0 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56255df927f0 0x56255df81120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8aeeffd640 10 client.7842 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.516+0000 7f8b0ffff640 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56255df927f0 0x56255df81120 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b2f102870e1cfee server_cookie=356cb6039291219c in_seq=0 out_seq=0 2024-04-14T08:39:35.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8aeeffd640 1 -- 172.21.15.172:0/3650909280 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56255ded2090 con 0x56255df927f0 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8aeeffd640 10 client.7842 handle_command_reply: tid=0 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8b1961f740 1 client.7842 shutdown 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8b1961f740 1 -- 172.21.15.172:0/3650909280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8d970 msgr2=0x56255dfabd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8b1961f740 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8d970 0x56255dfabd80 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8b040593b0 tx=0x7f8b04019000 comp rx=0 tx=0).stop 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8b1961f740 1 -- 172.21.15.172:0/3650909280 shutdown_connections 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8b1961f740 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56255df927f0 0x56255df81120 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8b1961f740 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56255df8dd50 0x56255df65610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8aeeffd640 0 client.7842 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8b1961f740 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255df8d970 0x56255dfabd80 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8b1961f740 1 --2- 172.21.15.172:0/3650909280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56255df60e50 0x56255df938c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8b1961f740 1 -- 172.21.15.172:0/3650909280 >> 172.21.15.172:0/3650909280 conn(0x56255dfa52b0 msgr2=0x56255df64470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8b1961f740 1 -- 172.21.15.172:0/3650909280 shutdown_connections 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8b1961f740 1 -- 172.21.15.172:0/3650909280 wait complete. 2024-04-14T08:39:35.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.517+0000 7f8b1961f740 20 client.7842 trim_cache size 0 max 16384 2024-04-14T08:39:35.530 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:39:35.530 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-04-14T08:39:35.530 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-04-14T08:39:35.530 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:39:35.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.518+0000 7f8b16836640 1 -- 172.21.15.172:0/788526292 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8afc06c8f0 msgr2=0x7f8afc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.518+0000 7f8b16836640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8afc06c8f0 0x7f8afc06edb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8b101f5160 tx=0x7f8b00008040 comp rx=0 tx=0).stop 2024-04-14T08:39:35.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.518+0000 7f8b16836640 1 -- 172.21.15.172:0/788526292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b1015e430 msgr2=0x7f8b100c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.518+0000 7f8b16836640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b1015e430 0x7f8b100c9760 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400c430 tx=0x7f8b0400c8f0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.518+0000 7f8b16836640 1 -- 172.21.15.172:0/788526292 shutdown_connections 2024-04-14T08:39:35.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.518+0000 7f8b16836640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8afc06c8f0 0x7f8afc06edb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.518+0000 7f8b16836640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b10162540 0x7f8b100b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.518+0000 7f8b16836640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b1015e430 0x7f8b100c9760 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.518+0000 7f8b16836640 1 --2- 172.21.15.172:0/788526292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b100d2e00 0x7f8b100b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.518+0000 7f8b16836640 1 -- 172.21.15.172:0/788526292 >> 172.21.15.172:0/788526292 conn(0x7f8b1009d560 msgr2=0x7f8b100b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:35.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.518+0000 7f8b16836640 1 -- 172.21.15.172:0/788526292 shutdown_connections 2024-04-14T08:39:35.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.518+0000 7f8b16836640 1 -- 172.21.15.172:0/788526292 wait complete. 2024-04-14T08:39:35.532 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-14T08:39:35.533 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:39:35.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.633+0000 7ff5312de640 1 Processor -- start 2024-04-14T08:39:35.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.633+0000 7ff5312de640 1 -- start start 2024-04-14T08:39:35.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff5312de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff52c0d2830 0x7ff52c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff5312de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff52c15e890 0x7ff52c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff5312de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff52c0d3150 con 0x7ff52c0cedd0 2024-04-14T08:39:35.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff5312de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff52c0cff30 con 0x7ff52c15e890 2024-04-14T08:39:35.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff5312de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff52c0d2350 con 0x7ff52c0d2830 2024-04-14T08:39:35.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff52a7fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff52c15e890 0x7ff52c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff52a7fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff52c15e890 0x7ff52c0d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57158/0 (socket says 172.21.15.172:57158) 2024-04-14T08:39:35.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff52b7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff52c0cedd0 0x7ff52c1621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56006/0 (socket says 172.21.15.172:56006) 2024-04-14T08:39:35.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff52a7fc640 1 -- 172.21.15.172:0/3087683733 learned_addr learned my addr 172.21.15.172:0/3087683733 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:35.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff52affd640 1 --2- 172.21.15.172:0/3087683733 >> v2:172.21.15.177:3300/0 conn(0x7ff52c0d2830 0x7ff52c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff52a7fc640 1 -- 172.21.15.172:0/3087683733 >> v2:172.21.15.177:3300/0 conn(0x7ff52c0d2830 msgr2=0x7ff52c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:35.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff52a7fc640 1 --2- 172.21.15.172:0/3087683733 >> v2:172.21.15.177:3300/0 conn(0x7ff52c0d2830 0x7ff52c0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff52a7fc640 1 -- 172.21.15.172:0/3087683733 >> v1:172.21.15.172:6789/0 conn(0x7ff52c0cedd0 legacy=0x7ff52c1621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff52a7fc640 1 -- 172.21.15.172:0/3087683733 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c0cf320 con 0x7ff52c15e890 2024-04-14T08:39:35.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.634+0000 7ff529ffb640 1 -- 172.21.15.172:0/3087683733 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3155795298 0 0) 0x7ff52c0d3150 con 0x7ff52c0cedd0 2024-04-14T08:39:35.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.635+0000 7ff52a7fc640 1 --2- 172.21.15.172:0/3087683733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff52c15e890 0x7ff52c0d1300 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ff52c08c830 tx=0x7ff52002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aea0e23ece5d5d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:35.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.635+0000 7ff529ffb640 1 -- 172.21.15.172:0/3087683733 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52002f850 con 0x7ff52c15e890 2024-04-14T08:39:35.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.635+0000 7ff529ffb640 1 -- 172.21.15.172:0/3087683733 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff52002f9f0 con 0x7ff52c15e890 2024-04-14T08:39:35.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.635+0000 7ff529ffb640 1 -- 172.21.15.172:0/3087683733 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52002fdc0 con 0x7ff52c15e890 2024-04-14T08:39:35.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.636+0000 7ff5312de640 1 -- 172.21.15.172:0/3087683733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff52c15e890 msgr2=0x7ff52c0d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.636+0000 7ff5312de640 1 --2- 172.21.15.172:0/3087683733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff52c15e890 0x7ff52c0d1300 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ff52c08c830 tx=0x7ff52002ba30 comp rx=0 tx=0).stop 2024-04-14T08:39:35.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.636+0000 7ff5312de640 1 -- 172.21.15.172:0/3087683733 shutdown_connections 2024-04-14T08:39:35.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.636+0000 7ff5312de640 1 --2- 172.21.15.172:0/3087683733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff52c15e890 0x7ff52c0d1300 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.636+0000 7ff5312de640 1 --2- 172.21.15.172:0/3087683733 >> v2:172.21.15.177:3300/0 conn(0x7ff52c0d2830 0x7ff52c0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.636+0000 7ff5312de640 1 -- 172.21.15.172:0/3087683733 >> 172.21.15.172:0/3087683733 conn(0x7ff52c09d590 msgr2=0x7ff52c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.636+0000 7ff5312de640 1 -- 172.21.15.172:0/3087683733 shutdown_connections 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.636+0000 7ff5312de640 1 -- 172.21.15.172:0/3087683733 wait complete. 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.637+0000 7ff5312de640 1 Processor -- start 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.637+0000 7ff5312de640 1 -- start start 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.638+0000 7ff5312de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff52c0cedd0 0x7ff52c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.638+0000 7ff5312de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff52c0d2830 0x7ff52c1f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.638+0000 7ff5312de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff52c15e890 0x7ff52c0b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.638+0000 7ff5312de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c0cfbc0 con 0x7ff52c0d2830 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.638+0000 7ff5312de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c0cf8e0 con 0x7ff52c0cedd0 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.638+0000 7ff5312de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff52c0d1840 con 0x7ff52c15e890 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.638+0000 7ff52a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff52c0d2830 0x7ff52c1f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.638+0000 7ff52a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff52c0d2830 0x7ff52c1f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57170/0 (socket says 172.21.15.172:57170) 2024-04-14T08:39:35.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.638+0000 7ff52a7fc640 1 -- 172.21.15.172:0/2859059749 learned_addr learned my addr 172.21.15.172:0/2859059749 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.638+0000 7ff52b7fe640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff52c15e890 0x7ff52c0b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.638+0000 7ff52affd640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff52c0cedd0 0x7ff52c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff52a7fc640 1 -- 172.21.15.172:0/2859059749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff52c15e890 msgr2=0x7ff52c0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff52a7fc640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff52c15e890 0x7ff52c0b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff52a7fc640 1 -- 172.21.15.172:0/2859059749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff52c0cedd0 msgr2=0x7ff52c15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff52a7fc640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff52c0cedd0 0x7ff52c15aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff52a7fc640 1 -- 172.21.15.172:0/2859059749 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c1f4e40 con 0x7ff52c0d2830 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff52b7fe640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff52c15e890 0x7ff52c0b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff52affd640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff52c0cedd0 0x7ff52c15aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff52a7fc640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff52c0d2830 0x7ff52c1f5080 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff520000c00 tx=0x7ff520004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff5137fe640 1 -- 172.21.15.172:0/2859059749 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520047070 con 0x7ff52c0d2830 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff5137fe640 1 -- 172.21.15.172:0/2859059749 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff520037be0 con 0x7ff52c0d2830 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff5312de640 1 -- 172.21.15.172:0/2859059749 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52c160040 con 0x7ff52c0d2830 2024-04-14T08:39:35.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff5137fe640 1 -- 172.21.15.172:0/2859059749 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520004030 con 0x7ff52c0d2830 2024-04-14T08:39:35.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.639+0000 7ff5312de640 1 -- 172.21.15.172:0/2859059749 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52c160600 con 0x7ff52c0d2830 2024-04-14T08:39:35.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.640+0000 7ff5312de640 1 -- 172.21.15.172:0/2859059749 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff52c068520 con 0x7ff52c0d2830 2024-04-14T08:39:35.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.641+0000 7ff5137fe640 1 -- 172.21.15.172:0/2859059749 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff52004a070 con 0x7ff52c0d2830 2024-04-14T08:39:35.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.641+0000 7ff5137fe640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff51806c8f0 0x7ff51806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:35.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.641+0000 7ff5137fe640 1 -- 172.21.15.172:0/2859059749 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7ff5200addd0 con 0x7ff52c0d2830 2024-04-14T08:39:35.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.641+0000 7ff52affd640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff51806c8f0 0x7ff51806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:35.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.642+0000 7ff52affd640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff51806c8f0 0x7ff51806edb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009fd0 tx=0x7ff51c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:35.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.648+0000 7ff5137fe640 1 -- 172.21.15.172:0/2859059749 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff520033030 con 0x7ff52c0d2830 2024-04-14T08:39:35.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.878+0000 7ff5312de640 1 -- 172.21.15.172:0/2859059749 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff52c0d4960 con 0x7ff52c0d2830 2024-04-14T08:39:35.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.878+0000 7ff5137fe640 1 -- 172.21.15.172:0/2859059749 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5196 (secure 0 0 0) 0x7ff5200af090 con 0x7ff52c0d2830 2024-04-14T08:39:35.880 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:39:35.880 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:39:05.413377+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7473":{"gid":7473,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6837/1386685680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1386685680},{"type":"v1","addr":"172.21.15.172:6837","nonce":1386685680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7473,"qdb_cluster":[7473]},"id":6}]} 2024-04-14T08:39:35.880 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 61 2024-04-14T08:39:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.880+0000 7ff5312de640 1 -- 172.21.15.172:0/2859059749 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff51806c8f0 msgr2=0x7ff51806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.880+0000 7ff5312de640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff51806c8f0 0x7ff51806edb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009fd0 tx=0x7ff51c00b040 comp rx=0 tx=0).stop 2024-04-14T08:39:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.880+0000 7ff5312de640 1 -- 172.21.15.172:0/2859059749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff52c0d2830 msgr2=0x7ff52c1f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.880+0000 7ff5312de640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff52c0d2830 0x7ff52c1f5080 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ff520000c00 tx=0x7ff520004730 comp rx=0 tx=0).stop 2024-04-14T08:39:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.880+0000 7ff5312de640 1 -- 172.21.15.172:0/2859059749 shutdown_connections 2024-04-14T08:39:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.880+0000 7ff5312de640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff51806c8f0 0x7ff51806edb0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.880+0000 7ff5312de640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff52c15e890 0x7ff52c0b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.880+0000 7ff5312de640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff52c0d2830 0x7ff52c1f5080 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.880+0000 7ff5312de640 1 --2- 172.21.15.172:0/2859059749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff52c0cedd0 0x7ff52c15aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.880+0000 7ff5312de640 1 -- 172.21.15.172:0/2859059749 >> 172.21.15.172:0/2859059749 conn(0x7ff52c09d590 msgr2=0x7ff52c0d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.880+0000 7ff5312de640 1 -- 172.21.15.172:0/2859059749 shutdown_connections 2024-04-14T08:39:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.880+0000 7ff5312de640 1 -- 172.21.15.172:0/2859059749 wait complete. 2024-04-14T08:39:35.896 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-14T08:39:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:35.999+0000 7f9105dbb640 1 Processor -- start 2024-04-14T08:39:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.000+0000 7f9105dbb640 1 -- start start 2024-04-14T08:39:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.001+0000 7f9105dbb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f910015e590 0x7f91001629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.001+0000 7f9105dbb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91000b3750 con 0x7f91000b65a0 2024-04-14T08:39:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.001+0000 7f9105dbb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91000cef20 con 0x7f91000cea60 2024-04-14T08:39:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.001+0000 7f9105dbb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f91000cf200 con 0x7f910015e590 2024-04-14T08:39:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.001+0000 7f90ff7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f910015e590 0x7f91001629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.001+0000 7f90fffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f91000b65a0 0x7f91000d2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56020/0 (socket says 172.21.15.172:56020) 2024-04-14T08:39:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.001+0000 7f90fffff640 1 -- 172.21.15.172:0/1743102873 learned_addr learned my addr 172.21.15.172:0/1743102873 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:39:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.001+0000 7f90ff7fe640 1 -- 172.21.15.172:0/1743102873 >> v1:172.21.15.177:6789/0 conn(0x7f91000cea60 legacy=0x7f91000b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.001+0000 7f90ff7fe640 1 -- 172.21.15.172:0/1743102873 >> v1:172.21.15.172:6789/0 conn(0x7f91000b65a0 legacy=0x7f91000d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.001+0000 7f90fe7fc640 1 -- 172.21.15.172:0/1743102873 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1914390535 0 0) 0x7f91000b3750 con 0x7f91000b65a0 2024-04-14T08:39:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.001+0000 7f90ff7fe640 1 -- 172.21.15.172:0/1743102873 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91000cf4e0 con 0x7f910015e590 2024-04-14T08:39:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.002+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/1743102873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f910015e590 0x7f91001629c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f90f00097a0 tx=0x7f90f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467644b2e35a4d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.002+0000 7f90fe7fc640 1 -- 172.21.15.172:0/1743102873 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0035020 con 0x7f910015e590 2024-04-14T08:39:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.002+0000 7f90fe7fc640 1 -- 172.21.15.172:0/1743102873 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f0031830 con 0x7f910015e590 2024-04-14T08:39:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.002+0000 7f90fe7fc640 1 -- 172.21.15.172:0/1743102873 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0031c00 con 0x7f910015e590 2024-04-14T08:39:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.003+0000 7f9105dbb640 1 -- 172.21.15.172:0/1743102873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f910015e590 msgr2=0x7f91001629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.003+0000 7f9105dbb640 1 --2- 172.21.15.172:0/1743102873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f910015e590 0x7f91001629c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f90f00097a0 tx=0x7f90f002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.003+0000 7f9105dbb640 1 -- 172.21.15.172:0/1743102873 shutdown_connections 2024-04-14T08:39:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.003+0000 7f9105dbb640 1 --2- 172.21.15.172:0/1743102873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f910015e590 0x7f91001629c0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.003+0000 7f9105dbb640 1 -- 172.21.15.172:0/1743102873 >> 172.21.15.172:0/1743102873 conn(0x7f91000d0e90 msgr2=0x7f91000d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.003+0000 7f9105dbb640 1 -- 172.21.15.172:0/1743102873 shutdown_connections 2024-04-14T08:39:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.003+0000 7f9105dbb640 1 -- 172.21.15.172:0/1743102873 wait complete. 2024-04-14T08:39:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.005+0000 7f9105dbb640 1 Processor -- start 2024-04-14T08:39:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.005+0000 7f9105dbb640 1 -- start start 2024-04-14T08:39:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.006+0000 7f9105dbb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91000b65a0 0x7f91000ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.006+0000 7f9105dbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91000cea60 0x7f910015f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.006+0000 7f9105dbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f910015e590 0x7f91001578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.006+0000 7f9105dbb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f91000d2ff0 con 0x7f91000cea60 2024-04-14T08:39:36.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.006+0000 7f9105dbb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f91000d2e70 con 0x7f91000b65a0 2024-04-14T08:39:36.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.006+0000 7f9105dbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f91000d3170 con 0x7f910015e590 2024-04-14T08:39:36.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.006+0000 7f90feffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91000cea60 0x7f910015f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.006+0000 7f90feffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91000cea60 0x7f910015f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57188/0 (socket says 172.21.15.172:57188) 2024-04-14T08:39:36.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.006+0000 7f90feffd640 1 -- 172.21.15.172:0/2993116701 learned_addr learned my addr 172.21.15.172:0/2993116701 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.006+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91000b65a0 0x7f91000ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.006+0000 7f90fffff640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f910015e590 0x7f91001578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.007+0000 7f90feffd640 1 -- 172.21.15.172:0/2993116701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f910015e590 msgr2=0x7f91001578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.007+0000 7f90feffd640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f910015e590 0x7f91001578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.007+0000 7f90feffd640 1 -- 172.21.15.172:0/2993116701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91000b65a0 msgr2=0x7f91000ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.007+0000 7f90feffd640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91000b65a0 0x7f91000ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.007+0000 7f90feffd640 1 -- 172.21.15.172:0/2993116701 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9100160050 con 0x7f91000cea60 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.007+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91000b65a0 0x7f91000ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.007+0000 7f90feffd640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91000cea60 0x7f910015f900 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f90f400c430 tx=0x7f90f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.007+0000 7f90e7fff640 1 -- 172.21.15.172:0/2993116701 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4011790 con 0x7f91000cea60 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.007+0000 7f90e7fff640 1 -- 172.21.15.172:0/2993116701 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f4017070 con 0x7f91000cea60 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.007+0000 7f9105dbb640 1 -- 172.21.15.172:0/2993116701 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9100160330 con 0x7f91000cea60 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.007+0000 7f90e7fff640 1 -- 172.21.15.172:0/2993116701 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4011b70 con 0x7f91000cea60 2024-04-14T08:39:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.007+0000 7f9105dbb640 1 -- 172.21.15.172:0/2993116701 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91001608f0 con 0x7f91000cea60 2024-04-14T08:39:36.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.008+0000 7f90e7fff640 1 -- 172.21.15.172:0/2993116701 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90f4011d10 con 0x7f91000cea60 2024-04-14T08:39:36.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.009+0000 7f90e7fff640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f90ec06c920 0x7f90ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.009+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f90ec06c920 0x7f90ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.009+0000 7f90e7fff640 1 -- 172.21.15.172:0/2993116701 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f90f408bcb0 con 0x7f91000cea60 2024-04-14T08:39:36.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.009+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f90ec06c920 0x7f90ec06ede0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f90f0036000 tx=0x7f90f003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.010+0000 7f9108ba3740 1 Processor -- start 2024-04-14T08:39:36.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.010+0000 7f9108ba3740 1 -- start start 2024-04-14T08:39:36.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.011+0000 7f9108ba3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d45514dd90 0x55d45514e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.011+0000 7f9108ba3740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d455153bb0 0x55d455151de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.011+0000 7f9108ba3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d45519b4a0 0x55d45519b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.011+0000 7f9108ba3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d4551155a0 con 0x55d45519b4a0 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.011+0000 7f9108ba3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d4550b7680 con 0x55d45514dd90 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.011+0000 7f9108ba3740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d455055860 con 0x55d455153bb0 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.011+0000 7f90fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d45519b4a0 0x55d45519b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.011+0000 7f90fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d45519b4a0 0x55d45519b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57192/0 (socket says 172.21.15.172:57192) 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.011+0000 7f90fffff640 1 -- 172.21.15.172:0/1481422362 learned_addr learned my addr 172.21.15.172:0/1481422362 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.011+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/1481422362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d45514dd90 0x55d45514e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.011+0000 7f90feffd640 1 --2- 172.21.15.172:0/1481422362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d455153bb0 0x55d455151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f90fffff640 1 -- 172.21.15.172:0/1481422362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d455153bb0 msgr2=0x55d455151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f90fffff640 1 --2- 172.21.15.172:0/1481422362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d455153bb0 0x55d455151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f90fffff640 1 -- 172.21.15.172:0/1481422362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d45514dd90 msgr2=0x55d45514e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f90fffff640 1 --2- 172.21.15.172:0/1481422362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d45514dd90 0x55d45514e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f90fffff640 1 -- 172.21.15.172:0/1481422362 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4550841a0 con 0x55d45519b4a0 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f90feffd640 1 --2- 172.21.15.172:0/1481422362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d455153bb0 0x55d455151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/1481422362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d45514dd90 0x55d45514e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f90fffff640 1 --2- 172.21.15.172:0/1481422362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d45519b4a0 0x55d45519b920 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f90f8007290 tx=0x7f90f8007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519657e784758f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f90e57fa640 1 -- 172.21.15.172:0/1481422362 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8019070 con 0x55d45519b4a0 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f90e57fa640 1 -- 172.21.15.172:0/1481422362 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f80079a0 con 0x55d45519b4a0 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f90e57fa640 1 -- 172.21.15.172:0/1481422362 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8001c10 con 0x55d45519b4a0 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f9108ba3740 1 -- 172.21.15.172:0/1481422362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d45519b4a0 msgr2=0x55d45519b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.012+0000 7f9108ba3740 1 --2- 172.21.15.172:0/1481422362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d45519b4a0 0x55d45519b920 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f90f8007290 tx=0x7f90f8007750 comp rx=0 tx=0).stop 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.013+0000 7f9108ba3740 1 -- 172.21.15.172:0/1481422362 shutdown_connections 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.013+0000 7f9108ba3740 1 --2- 172.21.15.172:0/1481422362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d45519b4a0 0x55d45519b920 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.013+0000 7f9108ba3740 1 --2- 172.21.15.172:0/1481422362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d455153bb0 0x55d455151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.013+0000 7f9108ba3740 1 --2- 172.21.15.172:0/1481422362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d45514dd90 0x55d45514e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.013+0000 7f9108ba3740 1 -- 172.21.15.172:0/1481422362 >> 172.21.15.172:0/1481422362 conn(0x55d45514fe10 msgr2=0x55d4551516d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.013+0000 7f9108ba3740 1 -- 172.21.15.172:0/1481422362 shutdown_connections 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.013+0000 7f9108ba3740 1 -- 172.21.15.172:0/1481422362 wait complete. 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.013+0000 7f9108ba3740 1 -- start start 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.013+0000 7f9108ba3740 1 Processor -- start 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f9108ba3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d455153bb0 0x55d45517f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f9108ba3740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d45519b4a0 0x55d4551ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f9108ba3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d4551725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f9108ba3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d4550531d0 con 0x55d4551acc10 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f9108ba3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d4550b3630 con 0x55d455153bb0 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f9108ba3740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d45517a500 con 0x55d45519b4a0 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f9108ba3740 10 client.0 _ll_register_callbacks cb 0x55d455076010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f90fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d4551725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f90fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d4551725b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57208/0 (socket says 172.21.15.172:57208) 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f90fffff640 1 -- 172.21.15.172:0/3366100391 learned_addr learned my addr 172.21.15.172:0/3366100391 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d455153bb0 0x55d45517f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f90feffd640 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d45519b4a0 0x55d4551ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f90fffff640 1 -- 172.21.15.172:0/3366100391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d45519b4a0 msgr2=0x55d4551ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f90fffff640 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d45519b4a0 0x55d4551ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f90fffff640 1 -- 172.21.15.172:0/3366100391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d455153bb0 msgr2=0x55d45517f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f90fffff640 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d455153bb0 0x55d45517f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f90fffff640 1 -- 172.21.15.172:0/3366100391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d454e977c0 con 0x55d4551acc10 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d455153bb0 0x55d45517f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.014+0000 7f90feffd640 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d45519b4a0 0x55d4551ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f90fffff640 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d4551725b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f90f8001a00 tx=0x7f90f8007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25d454ca96fe7f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f90de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f90de7fc640 1 -- 172.21.15.172:0/3366100391 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8023060 con 0x55d4551acc10 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f90de7fc640 1 -- 172.21.15.172:0/3366100391 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f90f8001f60 con 0x55d4551acc10 2024-04-14T08:39:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f9108ba3740 1 -- 172.21.15.172:0/3366100391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90f80079a0 con 0x55d4551acc10 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f90de7fc640 1 -- 172.21.15.172:0/3366100391 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f90f8019070 con 0x55d4551acc10 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f9108ba3740 10 client.7866 fetch_fsmap learned FSMap version 61 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f9108ba3740 1 -- 172.21.15.172:0/3366100391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55d454ef7f50 con 0x55d4551acc10 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f90de7fc640 1 -- 172.21.15.172:0/3366100391 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f80079a0 con 0x55d4551acc10 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f90de7fc640 1 -- 172.21.15.172:0/3366100391 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f90f8005030 con 0x55d4551acc10 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f9108ba3740 10 client.7866 fetch_fsmap finished waiting for FSMap version 61 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f9108ba3740 10 client.7866 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f9108ba3740 20 client.7866 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f9108ba3740 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55d455172fc0 0x55d455190c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f9108ba3740 4 client.7866 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f9108ba3740 1 -- 172.21.15.172:0/3366100391 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4550caeb0 con 0x55d455172fc0 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.015+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55d455172fc0 0x55d455190c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.016+0000 7f90de7fc640 10 client.7866 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.016+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55d455172fc0 0x55d455190c20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8531af64ed4b904 server_cookie=890701e0b1fec47e in_seq=0 out_seq=0 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.016+0000 7f90de7fc640 1 -- 172.21.15.172:0/3366100391 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4550caeb0 con 0x55d455172fc0 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.016+0000 7f90de7fc640 10 client.7866 handle_command_reply: tid=0 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.016+0000 7f9108ba3740 1 client.7866 shutdown 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.016+0000 7f9108ba3740 1 -- 172.21.15.172:0/3366100391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 msgr2=0x55d4551725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.016+0000 7f9108ba3740 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d4551725b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f90f8001a00 tx=0x7f90f8007d90 comp rx=0 tx=0).stop 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.017+0000 7f9108ba3740 1 -- 172.21.15.172:0/3366100391 shutdown_connections 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.017+0000 7f9108ba3740 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55d455172fc0 0x55d455190c20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.017+0000 7f9108ba3740 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d4551725b0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.017+0000 7f9108ba3740 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d45519b4a0 0x55d4551ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.017+0000 7f9108ba3740 1 --2- 172.21.15.172:0/3366100391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d455153bb0 0x55d45517f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.017+0000 7f9108ba3740 1 -- 172.21.15.172:0/3366100391 >> 172.21.15.172:0/3366100391 conn(0x55d45514dd90 msgr2=0x55d455199f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.017+0000 7f90de7fc640 0 client.7866 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.017+0000 7f9108ba3740 1 -- 172.21.15.172:0/3366100391 shutdown_connections 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.017+0000 7f9108ba3740 1 -- 172.21.15.172:0/3366100391 wait complete. 2024-04-14T08:39:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.017+0000 7f9108ba3740 20 client.7866 trim_cache size 0 max 16384 2024-04-14T08:39:36.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.030+0000 7f9108ba3740 1 Processor -- start 2024-04-14T08:39:36.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.030+0000 7f9108ba3740 1 -- start start 2024-04-14T08:39:36.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f9108ba3740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d455153bb0 0x55d45517a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f9108ba3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d45519b4a0 0x55d45517ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f9108ba3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d455199f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f9108ba3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f91001e9f00 con 0x55d4551acc10 2024-04-14T08:39:36.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f9108ba3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f91001e9260 con 0x55d45519b4a0 2024-04-14T08:39:36.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f9108ba3740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f91001e98b0 con 0x55d455153bb0 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f90fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d455199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f90fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d455199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57214/0 (socket says 172.21.15.172:57214) 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f90fffff640 1 -- 172.21.15.172:0/1798346336 learned_addr learned my addr 172.21.15.172:0/1798346336 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/1798346336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d455153bb0 0x55d45517a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f90feffd640 1 --2- 172.21.15.172:0/1798346336 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d45519b4a0 0x55d45517ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f90fffff640 1 -- 172.21.15.172:0/1798346336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d455153bb0 msgr2=0x55d45517a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f90fffff640 1 --2- 172.21.15.172:0/1798346336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d455153bb0 0x55d45517a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f90fffff640 1 -- 172.21.15.172:0/1798346336 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d45519b4a0 msgr2=0x55d45517ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f90fffff640 1 --2- 172.21.15.172:0/1798346336 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d45519b4a0 0x55d45517ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.031+0000 7f90fffff640 1 -- 172.21.15.172:0/1798346336 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d45517fa20 con 0x55d4551acc10 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f90fffff640 1 --2- 172.21.15.172:0/1798346336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d455199f10 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f90f8008fb0 tx=0x7f90f801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f62827d5b424f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f90e4ff9640 1 -- 172.21.15.172:0/1798346336 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f80079a0 con 0x55d4551acc10 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f90e4ff9640 1 -- 172.21.15.172:0/1798346336 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f800b040 con 0x55d4551acc10 2024-04-14T08:39:36.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f90e4ff9640 1 -- 172.21.15.172:0/1798346336 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8005530 con 0x55d4551acc10 2024-04-14T08:39:36.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f9108ba3740 1 -- 172.21.15.172:0/1798346336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 msgr2=0x55d455199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f9108ba3740 1 --2- 172.21.15.172:0/1798346336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d455199f10 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f90f8008fb0 tx=0x7f90f801c000 comp rx=0 tx=0).stop 2024-04-14T08:39:36.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f9108ba3740 1 -- 172.21.15.172:0/1798346336 shutdown_connections 2024-04-14T08:39:36.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f9108ba3740 1 --2- 172.21.15.172:0/1798346336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d455199f10 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f9108ba3740 1 --2- 172.21.15.172:0/1798346336 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d45519b4a0 0x55d45517ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f9108ba3740 1 --2- 172.21.15.172:0/1798346336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d455153bb0 0x55d45517a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f9108ba3740 1 -- 172.21.15.172:0/1798346336 >> 172.21.15.172:0/1798346336 conn(0x55d45514dd90 msgr2=0x55d4551982a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f9108ba3740 1 -- 172.21.15.172:0/1798346336 shutdown_connections 2024-04-14T08:39:36.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.032+0000 7f9108ba3740 1 -- 172.21.15.172:0/1798346336 wait complete. 2024-04-14T08:39:36.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.033+0000 7f9108ba3740 1 -- start start 2024-04-14T08:39:36.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.033+0000 7f9108ba3740 1 Processor -- start 2024-04-14T08:39:36.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f9108ba3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d455153bb0 0x55d455215b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f9108ba3740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d45519b4a0 0x55d45517bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f9108ba3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d45517a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f9108ba3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d45517e040 con 0x55d4551acc10 2024-04-14T08:39:36.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f9108ba3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d45517c820 con 0x55d455153bb0 2024-04-14T08:39:36.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f9108ba3740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d45517cfc0 con 0x55d45519b4a0 2024-04-14T08:39:36.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f9108ba3740 10 client.0 _ll_register_callbacks cb 0x7f90f8017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:36.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f90fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d45517a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f90fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d45517a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57216/0 (socket says 172.21.15.172:57216) 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f90fffff640 1 -- 172.21.15.172:0/210250569 learned_addr learned my addr 172.21.15.172:0/210250569 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d455153bb0 0x55d455215b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f90feffd640 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d45519b4a0 0x55d45517bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f90fffff640 1 -- 172.21.15.172:0/210250569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d45519b4a0 msgr2=0x55d45517bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f90fffff640 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d45519b4a0 0x55d45517bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f90fffff640 1 -- 172.21.15.172:0/210250569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d455153bb0 msgr2=0x55d455215b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f90fffff640 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d455153bb0 0x55d455215b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f90fffff640 1 -- 172.21.15.172:0/210250569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d454d8b0b0 con 0x55d4551acc10 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f90fffff640 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d45517a950 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f90f80054b0 tx=0x7f90f8005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32830905277ee18e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.034+0000 7f90de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f90de7fc640 1 -- 172.21.15.172:0/210250569 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8023040 con 0x55d4551acc10 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f90de7fc640 1 -- 172.21.15.172:0/210250569 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f90f8005940 con 0x55d4551acc10 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f9108ba3740 1 -- 172.21.15.172:0/210250569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90f800b040 con 0x55d4551acc10 2024-04-14T08:39:36.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f90de7fc640 1 -- 172.21.15.172:0/210250569 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f90f8046050 con 0x55d4551acc10 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f9108ba3740 10 client.7872 fetch_fsmap learned FSMap version 61 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f9108ba3740 1 -- 172.21.15.172:0/210250569 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55d455152ec0 con 0x55d4551acc10 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f90de7fc640 1 -- 172.21.15.172:0/210250569 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f800b040 con 0x55d4551acc10 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f90de7fc640 1 -- 172.21.15.172:0/210250569 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f90f8049020 con 0x55d4551acc10 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f9108ba3740 10 client.7872 fetch_fsmap finished waiting for FSMap version 61 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f9108ba3740 10 client.7872 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f9108ba3740 20 client.7872 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f9108ba3740 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55d45517ae90 0x55d45518e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f9108ba3740 4 client.7872 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f9108ba3740 1 -- 172.21.15.172:0/210250569 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55d4550584a0 con 0x55d45517ae90 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55d45517ae90 0x55d45518e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f90ff7fe640 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55d45517ae90 0x55d45518e4b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7621718a42c672f4 server_cookie=d8de3eb06a5727c in_seq=0 out_seq=0 2024-04-14T08:39:36.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.035+0000 7f90de7fc640 10 client.7872 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f90de7fc640 1 -- 172.21.15.172:0/210250569 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+419 (crc 0 0 0) 0x55d4550584a0 con 0x55d45517ae90 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f90de7fc640 10 client.7872 handle_command_reply: tid=0 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9108ba3740 1 client.7872 shutdown 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9108ba3740 1 -- 172.21.15.172:0/210250569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 msgr2=0x55d45517a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9108ba3740 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d45517a950 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f90f80054b0 tx=0x7f90f8005d30 comp rx=0 tx=0).stop 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9108ba3740 1 -- 172.21.15.172:0/210250569 shutdown_connections 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9108ba3740 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55d45517ae90 0x55d45518e4b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9108ba3740 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4551acc10 0x55d45517a950 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9108ba3740 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d45519b4a0 0x55d45517bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f90de7fc640 0 client.7872 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9108ba3740 1 --2- 172.21.15.172:0/210250569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d455153bb0 0x55d455215b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9108ba3740 1 -- 172.21.15.172:0/210250569 >> 172.21.15.172:0/210250569 conn(0x55d45514dd90 msgr2=0x55d45517ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9108ba3740 1 -- 172.21.15.172:0/210250569 shutdown_connections 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9108ba3740 1 -- 172.21.15.172:0/210250569 wait complete. 2024-04-14T08:39:36.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9108ba3740 20 client.7872 trim_cache size 0 max 16384 2024-04-14T08:39:36.044 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:39:36.044 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-04-14T08:39:36.044 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:39:36.044 INFO:teuthology.orchestra.run.smithi172.stdout: "damage_type": "backtrace", 2024-04-14T08:39:36.044 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 884217824, 2024-04-14T08:39:36.044 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 1099511627777, 2024-04-14T08:39:36.044 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/testdir/hardlink" 2024-04-14T08:39:36.044 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:39:36.044 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:39:36.044 INFO:teuthology.orchestra.run.smithi172.stdout: "damage_type": "backtrace", 2024-04-14T08:39:36.044 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 2012753192, 2024-04-14T08:39:36.045 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 1099511627776, 2024-04-14T08:39:36.045 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/dir" 2024-04-14T08:39:36.045 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:39:36.045 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:39:36.045 INFO:teuthology.orchestra.run.smithi172.stdout: "damage_type": "dir_frag", 2024-04-14T08:39:36.045 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 2971787147, 2024-04-14T08:39:36.045 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 1099511627776, 2024-04-14T08:39:36.045 INFO:teuthology.orchestra.run.smithi172.stdout: "frag": "*", 2024-04-14T08:39:36.045 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/dir" 2024-04-14T08:39:36.045 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:39:36.045 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-04-14T08:39:36.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9105dbb640 1 -- 172.21.15.172:0/2993116701 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f90ec06c920 msgr2=0x7f90ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9105dbb640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f90ec06c920 0x7f90ec06ede0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f90f0036000 tx=0x7f90f003c040 comp rx=0 tx=0).stop 2024-04-14T08:39:36.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9105dbb640 1 -- 172.21.15.172:0/2993116701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91000cea60 msgr2=0x7f910015f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9105dbb640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91000cea60 0x7f910015f900 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f90f400c430 tx=0x7f90f400c8f0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9105dbb640 1 -- 172.21.15.172:0/2993116701 shutdown_connections 2024-04-14T08:39:36.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9105dbb640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f90ec06c920 0x7f90ec06ede0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9105dbb640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f910015e590 0x7f91001578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9105dbb640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91000cea60 0x7f910015f900 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9105dbb640 1 --2- 172.21.15.172:0/2993116701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91000b65a0 0x7f91000ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.036+0000 7f9105dbb640 1 -- 172.21.15.172:0/2993116701 >> 172.21.15.172:0/2993116701 conn(0x7f91000d0e90 msgr2=0x7f91000b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.037+0000 7f9105dbb640 1 -- 172.21.15.172:0/2993116701 shutdown_connections 2024-04-14T08:39:36.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.037+0000 7f9105dbb640 1 -- 172.21.15.172:0/2993116701 wait complete. 2024-04-14T08:39:36.049 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-14T08:39:36.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.147+0000 7f58919fe640 1 Processor -- start 2024-04-14T08:39:36.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.147+0000 7f58919fe640 1 -- start start 2024-04-14T08:39:36.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f58919fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f588c0b7a60 0x7f588c0ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f58919fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f588c06dc40 0x7f588c0cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f58919fe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f588c0d2b10 con 0x7f588c08c4c0 2024-04-14T08:39:36.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f58919fe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f588c0d0ad0 con 0x7f588c06dc40 2024-04-14T08:39:36.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f58919fe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f588c0d0db0 con 0x7f588c0b7a60 2024-04-14T08:39:36.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f588b7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f588c06dc40 0x7f588c0cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f588a7fc640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f588c08c4c0 0x7f588c0cd410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56036/0 (socket says 172.21.15.172:56036) 2024-04-14T08:39:36.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f588a7fc640 1 -- 172.21.15.172:0/30343562 learned_addr learned my addr 172.21.15.172:0/30343562 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:39:36.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f588affd640 1 --2- 172.21.15.172:0/30343562 >> v2:172.21.15.177:3300/0 conn(0x7f588c0b7a60 0x7f588c0ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f588b7fe640 1 -- 172.21.15.172:0/30343562 >> v2:172.21.15.177:3300/0 conn(0x7f588c0b7a60 msgr2=0x7f588c0ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f588b7fe640 1 --2- 172.21.15.172:0/30343562 >> v2:172.21.15.177:3300/0 conn(0x7f588c0b7a60 0x7f588c0ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f588b7fe640 1 -- 172.21.15.172:0/30343562 >> v1:172.21.15.172:6789/0 conn(0x7f588c08c4c0 legacy=0x7f588c0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.148+0000 7f588b7fe640 1 -- 172.21.15.172:0/30343562 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c0d1090 con 0x7f588c06dc40 2024-04-14T08:39:36.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.149+0000 7f588b7fe640 1 --2- 172.21.15.172:0/30343562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f588c06dc40 0x7f588c0cee00 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f588c069160 tx=0x7f588402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64174418d6cddc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.149+0000 7f5889ffb640 1 -- 172.21.15.172:0/30343562 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884039070 con 0x7f588c06dc40 2024-04-14T08:39:36.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.149+0000 7f5889ffb640 1 -- 172.21.15.172:0/30343562 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588402cbf0 con 0x7f588c06dc40 2024-04-14T08:39:36.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.149+0000 7f5889ffb640 1 -- 172.21.15.172:0/30343562 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884034440 con 0x7f588c06dc40 2024-04-14T08:39:36.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.150+0000 7f58919fe640 1 -- 172.21.15.172:0/30343562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f588c06dc40 msgr2=0x7f588c0cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.150+0000 7f58919fe640 1 --2- 172.21.15.172:0/30343562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f588c06dc40 0x7f588c0cee00 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f588c069160 tx=0x7f588402be40 comp rx=0 tx=0).stop 2024-04-14T08:39:36.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.150+0000 7f58919fe640 1 -- 172.21.15.172:0/30343562 shutdown_connections 2024-04-14T08:39:36.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.150+0000 7f58919fe640 1 --2- 172.21.15.172:0/30343562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f588c06dc40 0x7f588c0cee00 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.150+0000 7f58919fe640 1 --2- 172.21.15.172:0/30343562 >> v2:172.21.15.177:3300/0 conn(0x7f588c0b7a60 0x7f588c0ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.150+0000 7f58919fe640 1 -- 172.21.15.172:0/30343562 >> 172.21.15.172:0/30343562 conn(0x7f588c08c000 msgr2=0x7f588c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.150+0000 7f58919fe640 1 -- 172.21.15.172:0/30343562 shutdown_connections 2024-04-14T08:39:36.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.150+0000 7f58919fe640 1 -- 172.21.15.172:0/30343562 wait complete. 2024-04-14T08:39:36.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.151+0000 7f58919fe640 1 Processor -- start 2024-04-14T08:39:36.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.151+0000 7f58919fe640 1 -- start start 2024-04-14T08:39:36.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f58919fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f588c06dc40 0x7f588c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f58919fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f588c08c4c0 0x7f588c160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f58919fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f588c0b7a60 0x7f588c1532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f58919fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f588c0d2750 con 0x7f588c0b7a60 2024-04-14T08:39:36.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f58919fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f588c0b7e40 con 0x7f588c06dc40 2024-04-14T08:39:36.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f58919fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f588c0d28d0 con 0x7f588c08c4c0 2024-04-14T08:39:36.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f588b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f588c0b7a60 0x7f588c1532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f588b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f588c0b7a60 0x7f588c1532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57240/0 (socket says 172.21.15.172:57240) 2024-04-14T08:39:36.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f588b7fe640 1 -- 172.21.15.172:0/2546807483 learned_addr learned my addr 172.21.15.172:0/2546807483 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f588a7fc640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f588c08c4c0 0x7f588c160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f588affd640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f588c06dc40 0x7f588c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f588b7fe640 1 -- 172.21.15.172:0/2546807483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f588c08c4c0 msgr2=0x7f588c160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f588b7fe640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f588c08c4c0 0x7f588c160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f588b7fe640 1 -- 172.21.15.172:0/2546807483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f588c06dc40 msgr2=0x7f588c15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f588b7fe640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f588c06dc40 0x7f588c15cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f588b7fe640 1 -- 172.21.15.172:0/2546807483 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c161180 con 0x7f588c0b7a60 2024-04-14T08:39:36.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.152+0000 7f588affd640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f588c06dc40 0x7f588c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:36.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.153+0000 7f588b7fe640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f588c0b7a60 0x7f588c1532c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5884009e80 tx=0x7f5884004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.153+0000 7f5883fff640 1 -- 172.21.15.172:0/2546807483 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884039050 con 0x7f588c0b7a60 2024-04-14T08:39:36.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.153+0000 7f5883fff640 1 -- 172.21.15.172:0/2546807483 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5884045070 con 0x7f588c0b7a60 2024-04-14T08:39:36.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.153+0000 7f58919fe640 1 -- 172.21.15.172:0/2546807483 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588c161420 con 0x7f588c0b7a60 2024-04-14T08:39:36.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.153+0000 7f5883fff640 1 -- 172.21.15.172:0/2546807483 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588403e830 con 0x7f588c0b7a60 2024-04-14T08:39:36.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.153+0000 7f58919fe640 1 -- 172.21.15.172:0/2546807483 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588c09bfc0 con 0x7f588c0b7a60 2024-04-14T08:39:36.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.154+0000 7f58919fe640 1 -- 172.21.15.172:0/2546807483 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f588c075490 con 0x7f588c0b7a60 2024-04-14T08:39:36.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.154+0000 7f5883fff640 1 -- 172.21.15.172:0/2546807483 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f588403e9d0 con 0x7f588c0b7a60 2024-04-14T08:39:36.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.155+0000 7f5883fff640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f586c06c920 0x7f586c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.155+0000 7f588affd640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f586c06c920 0x7f586c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.155+0000 7f5883fff640 1 -- 172.21.15.172:0/2546807483 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f58840aea90 con 0x7f588c0b7a60 2024-04-14T08:39:36.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.155+0000 7f588affd640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f586c06c920 0x7f586c06ede0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5874006fd0 tx=0x7f5874008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.159+0000 7f5883fff640 1 -- 172.21.15.172:0/2546807483 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5884032e20 con 0x7f588c0b7a60 2024-04-14T08:39:36.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.381+0000 7f58919fe640 1 -- 172.21.15.172:0/2546807483 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f588c0b4720 con 0x7f588c0b7a60 2024-04-14T08:39:36.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.381+0000 7f5883fff640 1 -- 172.21.15.172:0/2546807483 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f588407cc70 con 0x7f588c0b7a60 2024-04-14T08:39:36.383 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:39:36.383 INFO:teuthology.orchestra.run.smithi172.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-14T08:39:36.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.384+0000 7f58919fe640 1 -- 172.21.15.172:0/2546807483 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f586c06c920 msgr2=0x7f586c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.384+0000 7f58919fe640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f586c06c920 0x7f586c06ede0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5874006fd0 tx=0x7f5874008040 comp rx=0 tx=0).stop 2024-04-14T08:39:36.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.384+0000 7f58919fe640 1 -- 172.21.15.172:0/2546807483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f588c0b7a60 msgr2=0x7f588c1532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.384+0000 7f58919fe640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f588c0b7a60 0x7f588c1532c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5884009e80 tx=0x7f5884004a40 comp rx=0 tx=0).stop 2024-04-14T08:39:36.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.384+0000 7f58919fe640 1 -- 172.21.15.172:0/2546807483 shutdown_connections 2024-04-14T08:39:36.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.384+0000 7f58919fe640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f586c06c920 0x7f586c06ede0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.384+0000 7f58919fe640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f588c0b7a60 0x7f588c1532c0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.384+0000 7f58919fe640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f588c08c4c0 0x7f588c160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.384+0000 7f58919fe640 1 --2- 172.21.15.172:0/2546807483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f588c06dc40 0x7f588c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.384+0000 7f58919fe640 1 -- 172.21.15.172:0/2546807483 >> 172.21.15.172:0/2546807483 conn(0x7f588c08c000 msgr2=0x7f588c0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.384+0000 7f58919fe640 1 -- 172.21.15.172:0/2546807483 shutdown_connections 2024-04-14T08:39:36.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.384+0000 7f58919fe640 1 -- 172.21.15.172:0/2546807483 wait complete. 2024-04-14T08:39:36.396 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-14T08:39:36.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:36.396 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-14T08:39:36.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.501+0000 7fa1ce2de640 1 Processor -- start 2024-04-14T08:39:36.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.501+0000 7fa1ce2de640 1 -- start start 2024-04-14T08:39:36.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.502+0000 7fa1ce2de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1c809b180 0x7fa1c80d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.502+0000 7fa1ce2de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1c815fe70 con 0x7fa1c815fa90 2024-04-14T08:39:36.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.502+0000 7fa1ce2de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1c80b31e0 con 0x7fa1c80b0ec0 2024-04-14T08:39:36.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.502+0000 7fa1ce2de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c80ce560 con 0x7fa1c809b180 2024-04-14T08:39:36.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.502+0000 7fa1c7fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa1c815fa90 0x7fa1c80d0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56052/0 (socket says 172.21.15.172:56052) 2024-04-14T08:39:36.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.502+0000 7fa1c7fff640 1 -- 172.21.15.172:0/4126995063 learned_addr learned my addr 172.21.15.172:0/4126995063 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.502+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1c809b180 0x7fa1c80d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.502+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2744019092 0 0) 0x7fa1b0001110 con 0x7fa1c815fa90 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1b4003520 con 0x7fa1c815fa90 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2535527585 0 0) 0x7fa1c80b31e0 con 0x7fa1c80b0ec0 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1b0001110 con 0x7fa1c80b0ec0 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3487828282 0 0) 0x7fa1b4003520 con 0x7fa1c815fa90 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1c809b180 msgr2=0x7fa1c80d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1c809b180 0x7fa1c80d2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 >> v1:172.21.15.177:6789/0 conn(0x7fa1c80b0ec0 legacy=0x7fa1c809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c80b3480 con 0x7fa1c815fa90 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fa1b0001840 con 0x7fa1c815fa90 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1c809b180 0x7fa1c80d2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 >> v1:172.21.15.172:6789/0 conn(0x7fa1c815fa90 legacy=0x7fa1c80d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1b4003b70 0x7fa1b4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1b4004800 0x7fa1b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1c809b180 0x7fa1b40051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c80ce560 con 0x7fa1b4003b70 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b400bb00 con 0x7fa1b4004800 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b400bd00 con 0x7fa1c809b180 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1b4003b70 0x7fa1b4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c77fe640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1b4004800 0x7fa1b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1c809b180 0x7fa1b40051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c7fff640 1 -- 172.21.15.172:0/4126995063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1c809b180 msgr2=0x7fa1b40051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1c809b180 0x7fa1b40051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c7fff640 1 -- 172.21.15.172:0/4126995063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1b4004800 msgr2=0x7fa1b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1b4004800 0x7fa1b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.503+0000 7fa1c7fff640 1 -- 172.21.15.172:0/4126995063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b4003520 con 0x7fa1b4003b70 2024-04-14T08:39:36.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.504+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1b4003b70 0x7fa1b4003f70 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000c2a0 tx=0x7fa1b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c5bb847c1e4ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.505+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0007de0 con 0x7fa1b4003b70 2024-04-14T08:39:36.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.505+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b0030070 con 0x7fa1b4003b70 2024-04-14T08:39:36.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.505+0000 7fa1c6ffd640 1 -- 172.21.15.172:0/4126995063 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b002cdf0 con 0x7fa1b4003b70 2024-04-14T08:39:36.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.505+0000 7fa1ce2de640 1 -- 172.21.15.172:0/4126995063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1b4003b70 msgr2=0x7fa1b4003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.505+0000 7fa1ce2de640 1 --2- 172.21.15.172:0/4126995063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1b4003b70 0x7fa1b4003f70 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000c2a0 tx=0x7fa1b4005830 comp rx=0 tx=0).stop 2024-04-14T08:39:36.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.505+0000 7fa1cc87b640 1 -- 172.21.15.172:0/4126995063 reap_dead start 2024-04-14T08:39:36.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.505+0000 7fa1ce2de640 1 -- 172.21.15.172:0/4126995063 shutdown_connections 2024-04-14T08:39:36.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.505+0000 7fa1ce2de640 1 -- 172.21.15.172:0/4126995063 >> 172.21.15.172:0/4126995063 conn(0x7fa1c80d2040 msgr2=0x7fa1c80d2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.505+0000 7fa1ce2de640 1 -- 172.21.15.172:0/4126995063 shutdown_connections 2024-04-14T08:39:36.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.505+0000 7fa1ce2de640 1 -- 172.21.15.172:0/4126995063 wait complete. 2024-04-14T08:39:36.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.506+0000 7fa1ce2de640 1 Processor -- start 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.507+0000 7fa1ce2de640 1 -- start start 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.507+0000 7fa1ce2de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1b4003b70 0x7fa1c8157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.507+0000 7fa1ce2de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1c8158f60 0x7fa1c81593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.507+0000 7fa1ce2de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1c80caf60 0x7fa1c80cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.507+0000 7fa1ce2de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c80ce840 con 0x7fa1c80caf60 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.507+0000 7fa1ce2de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c809b7c0 con 0x7fa1c8158f60 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.507+0000 7fa1ce2de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c80d2b30 con 0x7fa1b4003b70 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1cc87b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1c80caf60 0x7fa1c80cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1cc87b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1c80caf60 0x7fa1c80cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57250/0 (socket says 172.21.15.172:57250) 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1cc87b640 1 -- 172.21.15.172:0/447025761 learned_addr learned my addr 172.21.15.172:0/447025761 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1c77fe640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1c8158f60 0x7fa1c81593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1b4003b70 0x7fa1c8157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1cc87b640 1 -- 172.21.15.172:0/447025761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1b4003b70 msgr2=0x7fa1c8157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1b4003b70 0x7fa1c8157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1cc87b640 1 -- 172.21.15.172:0/447025761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1c8158f60 msgr2=0x7fa1c81593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1c8158f60 0x7fa1c81593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1cc87b640 1 -- 172.21.15.172:0/447025761 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c8157ac0 con 0x7fa1c80caf60 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1b4003b70 0x7fa1c8157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1c80caf60 0x7fa1c80cb3e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009ed0 tx=0x7fa1c000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1c4ff9640 1 -- 172.21.15.172:0/447025761 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0015430 con 0x7fa1c80caf60 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1c4ff9640 1 -- 172.21.15.172:0/447025761 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c001d070 con 0x7fa1c80caf60 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1ce2de640 1 -- 172.21.15.172:0/447025761 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c80cbce0 con 0x7fa1c80caf60 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1c4ff9640 1 -- 172.21.15.172:0/447025761 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0015810 con 0x7fa1c80caf60 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.508+0000 7fa1ce2de640 1 -- 172.21.15.172:0/447025761 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c80b4f30 con 0x7fa1c80caf60 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.510+0000 7fa1c4ff9640 1 -- 172.21.15.172:0/447025761 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1c00159b0 con 0x7fa1c80caf60 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.510+0000 7fa1c4ff9640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa1a806c8f0 0x7fa1a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.510+0000 7fa1c4ff9640 1 -- 172.21.15.172:0/447025761 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7fa1c008c7c0 con 0x7fa1c80caf60 2024-04-14T08:39:36.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.510+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa1a806c8f0 0x7fa1a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.511+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa1a806c8f0 0x7fa1a806edb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0034000 tx=0x7fa1b0009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.512+0000 7fa1d11b5740 1 Processor -- start 2024-04-14T08:39:36.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.512+0000 7fa1d11b5740 1 -- start start 2024-04-14T08:39:36.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1d11b5740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 0x5592859246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1d11b5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f4300 0x559285921550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1d11b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 0x559285921a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1d11b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55928585e740 con 0x5592858f7e60 2024-04-14T08:39:36.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1d11b5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559285806930 con 0x5592858f4300 2024-04-14T08:39:36.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1d11b5740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55928585a890 con 0x5592859242e0 2024-04-14T08:39:36.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1cc87b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 0x559285921a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1cc87b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 0x559285921a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57262/0 (socket says 172.21.15.172:57262) 2024-04-14T08:39:36.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1cc87b640 1 -- 172.21.15.172:0/1754372740 learned_addr learned my addr 172.21.15.172:0/1754372740 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/1754372740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 0x5592859246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1c77fe640 1 --2- 172.21.15.172:0/1754372740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f4300 0x559285921550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1cc87b640 1 -- 172.21.15.172:0/1754372740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 msgr2=0x5592859246c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/1754372740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 0x5592859246c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1cc87b640 1 -- 172.21.15.172:0/1754372740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f4300 msgr2=0x559285921550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/1754372740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f4300 0x559285921550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1cc87b640 1 -- 172.21.15.172:0/1754372740 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928563a7e0 con 0x5592858f7e60 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1c77fe640 1 --2- 172.21.15.172:0/1754372740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f4300 0x559285921550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.513+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/1754372740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 0x559285921a90 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0002230 tx=0x7fa1c001f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbf8bd6e9f0d2a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.514+0000 7fa1bdffb640 1 -- 172.21.15.172:0/1754372740 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c005e650 con 0x5592858f7e60 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.514+0000 7fa1bdffb640 1 -- 172.21.15.172:0/1754372740 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c0062e80 con 0x5592858f7e60 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.514+0000 7fa1bdffb640 1 -- 172.21.15.172:0/1754372740 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0056910 con 0x5592858f7e60 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.514+0000 7fa1d11b5740 1 -- 172.21.15.172:0/1754372740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 msgr2=0x559285921a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.514+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/1754372740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 0x559285921a90 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0002230 tx=0x7fa1c001f310 comp rx=0 tx=0).stop 2024-04-14T08:39:36.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.514+0000 7fa1d11b5740 1 -- 172.21.15.172:0/1754372740 shutdown_connections 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.514+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/1754372740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 0x559285921a90 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.514+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/1754372740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f4300 0x559285921550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.514+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/1754372740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 0x5592859246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.514+0000 7fa1d11b5740 1 -- 172.21.15.172:0/1754372740 >> 172.21.15.172:0/1754372740 conn(0x5592858f6570 msgr2=0x55928593f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.514+0000 7fa1d11b5740 1 -- 172.21.15.172:0/1754372740 shutdown_connections 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.515+0000 7fa1d11b5740 1 -- 172.21.15.172:0/1754372740 wait complete. 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.515+0000 7fa1d11b5740 1 -- start start 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.515+0000 7fa1d11b5740 1 Processor -- start 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1d11b5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f7e60 0x559285927630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1d11b5740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 0x55928594f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1d11b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 0x5592859225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1d11b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5592858f9010 con 0x559285950650 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1d11b5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559285819cf0 con 0x5592858f7e60 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1d11b5740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5592858f8400 con 0x5592859242e0 2024-04-14T08:39:36.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1d11b5740 10 client.0 _ll_register_callbacks cb 0x5592857e69f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1cc87b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 0x5592859225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1cc87b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 0x5592859225f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57278/0 (socket says 172.21.15.172:57278) 2024-04-14T08:39:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1cc87b640 1 -- 172.21.15.172:0/3655614035 learned_addr learned my addr 172.21.15.172:0/3655614035 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f7e60 0x559285927630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1cc87b640 1 -- 172.21.15.172:0/3655614035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 msgr2=0x55928594f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1c77fe640 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 0x55928594f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 0x55928594f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1cc87b640 1 -- 172.21.15.172:0/3655614035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f7e60 msgr2=0x559285927630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f7e60 0x559285927630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1cc87b640 1 -- 172.21.15.172:0/3655614035 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928569af70 con 0x559285950650 2024-04-14T08:39:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f7e60 0x559285927630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:36.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 0x5592859225f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa1c001f4c0 tx=0x7fa1c0007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22da970beeb4628 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1a6ffd640 1 -- 172.21.15.172:0/3655614035 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c008a060 con 0x559285950650 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1a6ffd640 1 -- 172.21.15.172:0/3655614035 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7fa1c0062bf0 con 0x559285950650 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1d11b5740 1 -- 172.21.15.172:0/3655614035 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c0062e80 con 0x559285950650 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1a6ffd640 1 -- 172.21.15.172:0/3655614035 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa1c005e160 con 0x559285950650 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1d11b5740 10 client.7893 fetch_fsmap learned FSMap version 61 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.517+0000 7fa1d11b5740 1 -- 172.21.15.172:0/3655614035 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x559285532170 con 0x559285950650 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.518+0000 7fa1a6ffd640 1 -- 172.21.15.172:0/3655614035 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c0062e80 con 0x559285950650 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.518+0000 7fa1a6ffd640 1 -- 172.21.15.172:0/3655614035 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7fa1c0079bc0 con 0x559285950650 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.518+0000 7fa1d11b5740 10 client.7893 fetch_fsmap finished waiting for FSMap version 61 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.518+0000 7fa1d11b5740 10 client.7893 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.518+0000 7fa1d11b5740 20 client.7893 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.518+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55928591c750 0x55928591cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.518+0000 7fa1d11b5740 4 client.7893 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:39:36.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.518+0000 7fa1d11b5740 1 -- 172.21.15.172:0/3655614035 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55928586bc80 con 0x55928591c750 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.518+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55928591c750 0x55928591cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.518+0000 7fa1a6ffd640 10 client.7893 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.518+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55928591c750 0x55928591cb60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a333f07fc11777f3 server_cookie=d9fecab2e563a0d7 in_seq=0 out_seq=0 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1a6ffd640 1 -- 172.21.15.172:0/3655614035 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55928586bc80 con 0x55928591c750 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1a6ffd640 10 client.7893 handle_command_reply: tid=0 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1d11b5740 1 client.7893 shutdown 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1d11b5740 1 -- 172.21.15.172:0/3655614035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 msgr2=0x5592859225f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 0x5592859225f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa1c001f4c0 tx=0x7fa1c0007040 comp rx=0 tx=0).stop 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1d11b5740 1 -- 172.21.15.172:0/3655614035 shutdown_connections 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55928591c750 0x55928591cb60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 0x5592859225f0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 0x55928594f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/3655614035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f7e60 0x559285927630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1d11b5740 1 -- 172.21.15.172:0/3655614035 >> 172.21.15.172:0/3655614035 conn(0x5592858f4300 msgr2=0x559285940600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1a6ffd640 0 client.7893 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:36.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1d11b5740 1 -- 172.21.15.172:0/3655614035 shutdown_connections 2024-04-14T08:39:36.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1d11b5740 1 -- 172.21.15.172:0/3655614035 wait complete. 2024-04-14T08:39:36.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.519+0000 7fa1d11b5740 20 client.7893 trim_cache size 0 max 16384 2024-04-14T08:39:36.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.553+0000 7fa1d11b5740 1 Processor -- start 2024-04-14T08:39:36.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.553+0000 7fa1d11b5740 1 -- start start 2024-04-14T08:39:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1d11b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 0x55928594f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1d11b5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592859242e0 0x5592859268c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1d11b5740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559285950650 0x559285950c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1d11b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c8201aa0 con 0x5592858f7e60 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1d11b5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c8200aa0 con 0x5592859242e0 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1d11b5740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c81ffe30 con 0x559285950650 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1c7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 0x55928594f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1c7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 0x55928594f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57284/0 (socket says 172.21.15.172:57284) 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1c7fff640 1 -- 172.21.15.172:0/961263549 learned_addr learned my addr 172.21.15.172:0/961263549 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1c7fff640 1 -- 172.21.15.172:0/961263549 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559285950650 msgr2=0x559285950c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/961263549 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559285950650 0x559285950c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1c7fff640 1 -- 172.21.15.172:0/961263549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592859242e0 msgr2=0x5592859268c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/961263549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592859242e0 0x5592859268c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1c7fff640 1 -- 172.21.15.172:0/961263549 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559285923e50 con 0x5592858f7e60 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.554+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/961263549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 0x55928594f820 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00010e0 tx=0x7fa1b002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6434fbb89421dbdd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.555+0000 7fa1bd7fa640 1 -- 172.21.15.172:0/961263549 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b004f480 con 0x5592858f7e60 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.555+0000 7fa1bd7fa640 1 -- 172.21.15.172:0/961263549 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b0054070 con 0x5592858f7e60 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.555+0000 7fa1bd7fa640 1 -- 172.21.15.172:0/961263549 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b004f820 con 0x5592858f7e60 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.555+0000 7fa1d11b5740 1 -- 172.21.15.172:0/961263549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 msgr2=0x55928594f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.555+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/961263549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 0x55928594f820 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00010e0 tx=0x7fa1b002f000 comp rx=0 tx=0).stop 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.555+0000 7fa1d11b5740 1 -- 172.21.15.172:0/961263549 shutdown_connections 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.555+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/961263549 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559285950650 0x559285950c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.555+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/961263549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592859242e0 0x5592859268c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.555+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/961263549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592858f7e60 0x55928594f820 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.555+0000 7fa1d11b5740 1 -- 172.21.15.172:0/961263549 >> 172.21.15.172:0/961263549 conn(0x5592858f4300 msgr2=0x559285942600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.555+0000 7fa1d11b5740 1 -- 172.21.15.172:0/961263549 shutdown_connections 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.555+0000 7fa1d11b5740 1 -- 172.21.15.172:0/961263549 wait complete. 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.556+0000 7fa1d11b5740 1 -- start start 2024-04-14T08:39:36.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.556+0000 7fa1d11b5740 1 Processor -- start 2024-04-14T08:39:36.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.557+0000 7fa1d11b5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f7e60 0x559285926a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.557+0000 7fa1d11b5740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 0x5592859b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.557+0000 7fa1d11b5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 0x55928591fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.557+0000 7fa1d11b5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c8200450 con 0x559285950650 2024-04-14T08:39:36.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.557+0000 7fa1d11b5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55928594fe50 con 0x5592858f7e60 2024-04-14T08:39:36.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.557+0000 7fa1d11b5740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x559285926fd0 con 0x5592859242e0 2024-04-14T08:39:36.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.557+0000 7fa1d11b5740 10 client.0 _ll_register_callbacks cb 0x7fa1c0072070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:36.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1cc87b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 0x55928591fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1cc87b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 0x55928591fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57296/0 (socket says 172.21.15.172:57296) 2024-04-14T08:39:36.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1cc87b640 1 -- 172.21.15.172:0/215677987 learned_addr learned my addr 172.21.15.172:0/215677987 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f7e60 0x559285926a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1c77fe640 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 0x5592859b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1cc87b640 1 -- 172.21.15.172:0/215677987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 msgr2=0x5592859b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 0x5592859b8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1cc87b640 1 -- 172.21.15.172:0/215677987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f7e60 msgr2=0x559285926a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f7e60 0x559285926a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1cc87b640 1 -- 172.21.15.172:0/215677987 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592858fa060 con 0x559285950650 2024-04-14T08:39:36.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f7e60 0x559285926a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:39:36.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1cc87b640 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 0x55928591fcf0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0058550 tx=0x7fa1c008a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185cae48f7fa16be server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:36.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1a6ffd640 1 -- 172.21.15.172:0/215677987 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0062bf0 con 0x559285950650 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1a6ffd640 1 -- 172.21.15.172:0/215677987 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7fa1c0085d00 con 0x559285950650 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1d11b5740 1 -- 172.21.15.172:0/215677987 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b0054070 con 0x559285950650 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1a6ffd640 1 -- 172.21.15.172:0/215677987 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa1c007e070 con 0x559285950650 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1d11b5740 10 client.7899 fetch_fsmap learned FSMap version 61 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1d11b5740 1 -- 172.21.15.172:0/215677987 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x5592859b9330 con 0x559285950650 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.558+0000 7fa1a6ffd640 1 -- 172.21.15.172:0/215677987 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b0054070 con 0x559285950650 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1a6ffd640 1 -- 172.21.15.172:0/215677987 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7fa1c0055ff0 con 0x559285950650 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1d11b5740 10 client.7899 fetch_fsmap finished waiting for FSMap version 61 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1d11b5740 10 client.7899 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1d11b5740 20 client.7899 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x559285920230 0x5592858f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1d11b5740 4 client.7899 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1d11b5740 1 -- 172.21.15.172:0/215677987 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x559285865fd0 con 0x559285920230 2024-04-14T08:39:36.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x559285920230 0x5592858f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1a6ffd640 10 client.7899 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:36.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1c7fff640 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x559285920230 0x5592858f31f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=286f8d5fc59768e1 server_cookie=76d5ea89d0ddf7a in_seq=0 out_seq=0 2024-04-14T08:39:36.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1a6ffd640 1 -- 172.21.15.172:0/215677987 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559285865fd0 con 0x559285920230 2024-04-14T08:39:36.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1a6ffd640 10 client.7899 handle_command_reply: tid=0 2024-04-14T08:39:36.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.559+0000 7fa1d11b5740 1 client.7899 shutdown 2024-04-14T08:39:36.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1d11b5740 1 -- 172.21.15.172:0/215677987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 msgr2=0x55928591fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 0x55928591fcf0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0058550 tx=0x7fa1c008a970 comp rx=0 tx=0).stop 2024-04-14T08:39:36.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1d11b5740 1 -- 172.21.15.172:0/215677987 shutdown_connections 2024-04-14T08:39:36.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x559285920230 0x5592858f31f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559285950650 0x55928591fcf0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5592859242e0 0x5592859b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1d11b5740 1 --2- 172.21.15.172:0/215677987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592858f7e60 0x559285926a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1d11b5740 1 -- 172.21.15.172:0/215677987 >> 172.21.15.172:0/215677987 conn(0x5592858f4300 msgr2=0x55928593f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1a6ffd640 0 client.7899 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:36.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1d11b5740 1 -- 172.21.15.172:0/215677987 shutdown_connections 2024-04-14T08:39:36.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1d11b5740 1 -- 172.21.15.172:0/215677987 wait complete. 2024-04-14T08:39:36.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1d11b5740 20 client.7899 trim_cache size 0 max 16384 2024-04-14T08:39:36.569 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:39:36.569 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-04-14T08:39:36.569 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "5d9c155c-09eb-490d-9116-ab6230b56dfe", 2024-04-14T08:39:36.569 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-04-14T08:39:36.569 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:39:36.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1ce2de640 1 -- 172.21.15.172:0/447025761 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa1a806c8f0 msgr2=0x7fa1a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1ce2de640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa1a806c8f0 0x7fa1a806edb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0034000 tx=0x7fa1b0009030 comp rx=0 tx=0).stop 2024-04-14T08:39:36.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1ce2de640 1 -- 172.21.15.172:0/447025761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1c80caf60 msgr2=0x7fa1c80cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1ce2de640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1c80caf60 0x7fa1c80cb3e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009ed0 tx=0x7fa1c000f580 comp rx=0 tx=0).stop 2024-04-14T08:39:36.570 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:39:36.560+0000 7effbcfe9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-04-14T08:38:55.461911+0000 1=1+0)->f(v1 m2024-04-14T08:38:55.461911+0000) n(v0 rc2024-04-14T08:38:55.578911+0000 2=1+1)->n(v0 rc2024-04-14T08:38:55.578911+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x555fea025b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-14T08:38:55.461911+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-14T08:38:55.578911+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-14T08:39:36.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1ce2de640 1 -- 172.21.15.172:0/447025761 shutdown_connections 2024-04-14T08:39:36.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1ce2de640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa1a806c8f0 0x7fa1a806edb0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1ce2de640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1c80caf60 0x7fa1c80cb3e0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.560+0000 7fa1ce2de640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1c8158f60 0x7fa1c81593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.561+0000 7fa1ce2de640 1 --2- 172.21.15.172:0/447025761 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa1b4003b70 0x7fa1c8157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.561+0000 7fa1ce2de640 1 -- 172.21.15.172:0/447025761 >> 172.21.15.172:0/447025761 conn(0x7fa1c80d2040 msgr2=0x7fa1c80cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.561+0000 7fa1ce2de640 1 -- 172.21.15.172:0/447025761 shutdown_connections 2024-04-14T08:39:36.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.561+0000 7fa1ce2de640 1 -- 172.21.15.172:0/447025761 wait complete. 2024-04-14T08:39:36.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:39:36.574 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-14T08:39:36.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.702+0000 7f4a64188640 1 Processor -- start 2024-04-14T08:39:36.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.702+0000 7f4a64188640 1 -- start start 2024-04-14T08:39:36.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a64188640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c0d1a20 0x7f4a5c06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a64188640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4a5c06d000 0x7f4a5c06d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a64188640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a5c0701b0 con 0x7f4a5c071250 2024-04-14T08:39:36.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a64188640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f4a5c06d9d0 con 0x7f4a5c06d000 2024-04-14T08:39:36.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a64188640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5c070dc0 con 0x7f4a5c0d1a20 2024-04-14T08:39:36.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a62725640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4a5c06d000 0x7f4a5c06d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a62725640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4a5c06d000 0x7f4a5c06d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57304/0 (socket says 172.21.15.172:57304) 2024-04-14T08:39:36.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a62725640 1 -- 172.21.15.172:0/3398885824 learned_addr learned my addr 172.21.15.172:0/3398885824 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a61f24640 1 --2- 172.21.15.172:0/3398885824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c0d1a20 0x7f4a5c06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a62725640 1 -- 172.21.15.172:0/3398885824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c0d1a20 msgr2=0x7f4a5c06fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a62725640 1 --2- 172.21.15.172:0/3398885824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c0d1a20 0x7f4a5c06fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a62725640 1 -- 172.21.15.172:0/3398885824 >> v1:172.21.15.177:6789/0 conn(0x7f4a5c071250 legacy=0x7f4a5c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.703+0000 7f4a62725640 1 -- 172.21.15.172:0/3398885824 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c070490 con 0x7f4a5c06d000 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.704+0000 7f4a62725640 1 --2- 172.21.15.172:0/3398885824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a5c06d000 0x7f4a5c06d490 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009e50 tx=0x7f4a5402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0018433a729fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.704+0000 7f4a60f22640 1 -- 172.21.15.172:0/3398885824 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5402ca70 con 0x7f4a5c06d000 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.704+0000 7f4a60f22640 1 -- 172.21.15.172:0/3398885824 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5402cc10 con 0x7f4a5c06d000 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.704+0000 7f4a60f22640 1 -- 172.21.15.172:0/3398885824 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a540343f0 con 0x7f4a5c06d000 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.705+0000 7f4a64188640 1 -- 172.21.15.172:0/3398885824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a5c06d000 msgr2=0x7f4a5c06d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.705+0000 7f4a64188640 1 --2- 172.21.15.172:0/3398885824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a5c06d000 0x7f4a5c06d490 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009e50 tx=0x7f4a5402bed0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.705+0000 7f4a64188640 1 -- 172.21.15.172:0/3398885824 shutdown_connections 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.705+0000 7f4a64188640 1 --2- 172.21.15.172:0/3398885824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a5c06d000 0x7f4a5c06d490 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.705+0000 7f4a64188640 1 --2- 172.21.15.172:0/3398885824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c0d1a20 0x7f4a5c06fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.705+0000 7f4a64188640 1 -- 172.21.15.172:0/3398885824 >> 172.21.15.172:0/3398885824 conn(0x7f4a5c08c000 msgr2=0x7f4a5c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.705+0000 7f4a64188640 1 -- 172.21.15.172:0/3398885824 shutdown_connections 2024-04-14T08:39:36.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.705+0000 7f4a64188640 1 -- 172.21.15.172:0/3398885824 wait complete. 2024-04-14T08:39:36.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.706+0000 7f4a64188640 1 Processor -- start 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.706+0000 7f4a64188640 1 -- start start 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a64188640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c06d000 0x7f4a5c0b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a64188640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a5c071250 0x7f4a5c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a64188640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a5c0d1a20 0x7f4a5c0c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a64188640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c0d1e00 con 0x7f4a5c071250 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a64188640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c070a50 con 0x7f4a5c0d1a20 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a64188640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5c0b5270 con 0x7f4a5c06d000 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a61723640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a5c071250 0x7f4a5c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a61723640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a5c071250 0x7f4a5c0b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57310/0 (socket says 172.21.15.172:57310) 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a61723640 1 -- 172.21.15.172:0/4039062490 learned_addr learned my addr 172.21.15.172:0/4039062490 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a62725640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a5c0d1a20 0x7f4a5c0c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a61f24640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c06d000 0x7f4a5c0b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a61723640 1 -- 172.21.15.172:0/4039062490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c06d000 msgr2=0x7f4a5c0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a61723640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c06d000 0x7f4a5c0b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a61723640 1 -- 172.21.15.172:0/4039062490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a5c0d1a20 msgr2=0x7f4a5c0c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a61723640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a5c0d1a20 0x7f4a5c0c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a61723640 1 -- 172.21.15.172:0/4039062490 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c0c9d10 con 0x7f4a5c071250 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.707+0000 7f4a62725640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a5c0d1a20 0x7f4a5c0c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.708+0000 7f4a61723640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a5c071250 0x7f4a5c0b4d30 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00c440 tx=0x7f4a4c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.708+0000 7f4a527fc640 1 -- 172.21.15.172:0/4039062490 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c0052d0 con 0x7f4a5c071250 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.708+0000 7f4a527fc640 1 -- 172.21.15.172:0/4039062490 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a4c005470 con 0x7f4a5c071250 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.708+0000 7f4a64188640 1 -- 172.21.15.172:0/4039062490 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5c0c9a30 con 0x7f4a5c071250 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.708+0000 7f4a527fc640 1 -- 172.21.15.172:0/4039062490 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c007260 con 0x7f4a5c071250 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.708+0000 7f4a64188640 1 -- 172.21.15.172:0/4039062490 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a5c0af100 con 0x7f4a5c071250 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.709+0000 7f4a527fc640 1 -- 172.21.15.172:0/4039062490 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a4c007400 con 0x7f4a5c071250 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.710+0000 7f4a527fc640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4a4006c8f0 0x7f4a4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.710+0000 7f4a527fc640 1 -- 172.21.15.172:0/4039062490 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f4a4c08b260 con 0x7f4a5c071250 2024-04-14T08:39:36.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.710+0000 7f4a61f24640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4a4006c8f0 0x7f4a4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.710+0000 7f4a61f24640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4a4006c8f0 0x7f4a4006edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009ad0 tx=0x7f4a58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.711+0000 7f4a66f6a740 1 Processor -- start 2024-04-14T08:39:36.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.711+0000 7f4a66f6a740 1 -- start start 2024-04-14T08:39:36.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.711+0000 7f4a66f6a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329970 0x55f28b2fc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.711+0000 7f4a66f6a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b329d50 0x55f28b32c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.711+0000 7f4a66f6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b2fce50 0x55f28b32cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.711+0000 7f4a66f6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f28b266800 con 0x55f28b2fce50 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.711+0000 7f4a66f6a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f28b20e7f0 con 0x55f28b329d50 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.711+0000 7f4a66f6a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f28b2627b0 con 0x55f28b329970 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a62725640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b2fce50 0x55f28b32cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a62725640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b2fce50 0x55f28b32cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57316/0 (socket says 172.21.15.172:57316) 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a62725640 1 -- 172.21.15.172:0/3552432737 learned_addr learned my addr 172.21.15.172:0/3552432737 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a61723640 1 --2- 172.21.15.172:0/3552432737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b329d50 0x55f28b32c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a61f24640 1 --2- 172.21.15.172:0/3552432737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329970 0x55f28b2fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a62725640 1 -- 172.21.15.172:0/3552432737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329970 msgr2=0x55f28b2fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a62725640 1 --2- 172.21.15.172:0/3552432737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329970 0x55f28b2fc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a62725640 1 -- 172.21.15.172:0/3552432737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b329d50 msgr2=0x55f28b32c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a62725640 1 --2- 172.21.15.172:0/3552432737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b329d50 0x55f28b32c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a62725640 1 -- 172.21.15.172:0/3552432737 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f28b0447d0 con 0x55f28b2fce50 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a62725640 1 --2- 172.21.15.172:0/3552432737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b2fce50 0x55f28b32cc40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f4a54036040 tx=0x7f4a5402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac7c168f5b70079 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a3f7fe640 1 -- 172.21.15.172:0/3552432737 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54045070 con 0x55f28b2fce50 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a3f7fe640 1 -- 172.21.15.172:0/3552432737 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5402ca70 con 0x55f28b2fce50 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.712+0000 7f4a3f7fe640 1 -- 172.21.15.172:0/3552432737 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5402cc20 con 0x55f28b2fce50 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.713+0000 7f4a66f6a740 1 -- 172.21.15.172:0/3552432737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b2fce50 msgr2=0x55f28b32cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.713+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/3552432737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b2fce50 0x55f28b32cc40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f4a54036040 tx=0x7f4a5402e000 comp rx=0 tx=0).stop 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.713+0000 7f4a66f6a740 1 -- 172.21.15.172:0/3552432737 shutdown_connections 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.713+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/3552432737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b2fce50 0x55f28b32cc40 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.713+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/3552432737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b329d50 0x55f28b32c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.713+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/3552432737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329970 0x55f28b2fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.713+0000 7f4a66f6a740 1 -- 172.21.15.172:0/3552432737 >> 172.21.15.172:0/3552432737 conn(0x55f28b2feb80 msgr2=0x55f28b347700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.713+0000 7f4a66f6a740 1 -- 172.21.15.172:0/3552432737 shutdown_connections 2024-04-14T08:39:36.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.713+0000 7f4a66f6a740 1 -- 172.21.15.172:0/3552432737 wait complete. 2024-04-14T08:39:36.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.714+0000 7f4a66f6a740 1 -- start start 2024-04-14T08:39:36.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.714+0000 7f4a66f6a740 1 Processor -- start 2024-04-14T08:39:36.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a66f6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329970 0x55f28b32fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a66f6a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329d50 0x55f28b310b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a66f6a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b311060 0x55f28b325940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a66f6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f28b301120 con 0x55f28b329970 2024-04-14T08:39:36.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a66f6a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f28b221d00 con 0x55f28b311060 2024-04-14T08:39:36.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a66f6a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f28b300510 con 0x55f28b329d50 2024-04-14T08:39:36.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a66f6a740 10 client.0 _ll_register_callbacks cb 0x55f28b2216f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:36.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a61f24640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329970 0x55f28b32fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a61f24640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329970 0x55f28b32fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57320/0 (socket says 172.21.15.172:57320) 2024-04-14T08:39:36.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a61f24640 1 -- 172.21.15.172:0/842957226 learned_addr learned my addr 172.21.15.172:0/842957226 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a62725640 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b311060 0x55f28b325940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a61723640 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329d50 0x55f28b310b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a61f24640 1 -- 172.21.15.172:0/842957226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329d50 msgr2=0x55f28b310b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a61f24640 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329d50 0x55f28b310b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a61f24640 1 -- 172.21.15.172:0/842957226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b311060 msgr2=0x55f28b325940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a61f24640 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b311060 0x55f28b325940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a61f24640 1 -- 172.21.15.172:0/842957226 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f28af7a3f0 con 0x55f28b329970 2024-04-14T08:39:36.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a62725640 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b311060 0x55f28b325940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:39:36.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a61f24640 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329970 0x55f28b32fbc0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4a580156e0 tx=0x7f4a580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76528bea719fb76 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:36.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.715+0000 7f4a3cff9640 1 -- 172.21.15.172:0/842957226 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58025070 con 0x55f28b329970 2024-04-14T08:39:36.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a3cff9640 1 -- 172.21.15.172:0/842957226 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f4a58014620 con 0x55f28b329970 2024-04-14T08:39:36.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a66f6a740 1 -- 172.21.15.172:0/842957226 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a5402ca70 con 0x55f28b329970 2024-04-14T08:39:36.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a3cff9640 1 -- 172.21.15.172:0/842957226 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f4a58030050 con 0x55f28b329970 2024-04-14T08:39:36.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a66f6a740 10 client.7911 fetch_fsmap learned FSMap version 61 2024-04-14T08:39:36.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a66f6a740 1 -- 172.21.15.172:0/842957226 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55f28af3a0f0 con 0x55f28b329970 2024-04-14T08:39:36.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a3cff9640 1 -- 172.21.15.172:0/842957226 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5402ca70 con 0x55f28b329970 2024-04-14T08:39:36.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a3cff9640 1 -- 172.21.15.172:0/842957226 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f4a58033020 con 0x55f28b329970 2024-04-14T08:39:36.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a66f6a740 10 client.7911 fetch_fsmap finished waiting for FSMap version 61 2024-04-14T08:39:36.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a66f6a740 10 client.7911 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:39:36.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a66f6a740 20 client.7911 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:36.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55f28b3283b0 0x55f28b3287d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a66f6a740 4 client.7911 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:39:36.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a66f6a740 1 -- 172.21.15.172:0/842957226 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f28b273d40 con 0x55f28b3283b0 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a61723640 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55f28b3283b0 0x55f28b3287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a3cff9640 10 client.7911 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.716+0000 7f4a61723640 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55f28b3283b0 0x55f28b3287d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6622c2d7513dd2 server_cookie=7a82dd9c613ceaa6 in_seq=0 out_seq=0 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a3cff9640 1 -- 172.21.15.172:0/842957226 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f28b273d40 con 0x55f28b3283b0 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a3cff9640 10 client.7911 handle_command_reply: tid=0 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a66f6a740 1 client.7911 shutdown 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a66f6a740 1 -- 172.21.15.172:0/842957226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329970 msgr2=0x55f28b32fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329970 0x55f28b32fbc0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4a580156e0 tx=0x7f4a580046c0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a66f6a740 1 -- 172.21.15.172:0/842957226 shutdown_connections 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55f28b3283b0 0x55f28b3287d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b311060 0x55f28b325940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329d50 0x55f28b310b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/842957226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329970 0x55f28b32fbc0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a66f6a740 1 -- 172.21.15.172:0/842957226 >> 172.21.15.172:0/842957226 conn(0x55f28b2fce50 msgr2=0x55f28b32c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a3cff9640 0 client.7911 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:36.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a66f6a740 1 -- 172.21.15.172:0/842957226 shutdown_connections 2024-04-14T08:39:36.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a66f6a740 1 -- 172.21.15.172:0/842957226 wait complete. 2024-04-14T08:39:36.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.717+0000 7f4a66f6a740 20 client.7911 trim_cache size 0 max 16384 2024-04-14T08:39:36.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.751+0000 7f4a66f6a740 1 Processor -- start 2024-04-14T08:39:36.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.751+0000 7f4a66f6a740 1 -- start start 2024-04-14T08:39:36.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a66f6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b311060 0x55f28b3220b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a66f6a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b329970 0x55f28b300630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a66f6a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329d50 0x55f28b3bb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a66f6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c201300 con 0x55f28b311060 2024-04-14T08:39:36.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a66f6a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c200300 con 0x55f28b329970 2024-04-14T08:39:36.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a66f6a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5c1f8330 con 0x55f28b329d50 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a61f24640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b311060 0x55f28b3220b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a61f24640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b311060 0x55f28b3220b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57324/0 (socket says 172.21.15.172:57324) 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a61f24640 1 -- 172.21.15.172:0/30669531 learned_addr learned my addr 172.21.15.172:0/30669531 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a61723640 1 --2- 172.21.15.172:0/30669531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b329970 0x55f28b300630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a62725640 1 --2- 172.21.15.172:0/30669531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329d50 0x55f28b3bb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a61f24640 1 -- 172.21.15.172:0/30669531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329d50 msgr2=0x55f28b3bb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a61f24640 1 --2- 172.21.15.172:0/30669531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329d50 0x55f28b3bb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a61f24640 1 -- 172.21.15.172:0/30669531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b329970 msgr2=0x55f28b300630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a61f24640 1 --2- 172.21.15.172:0/30669531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b329970 0x55f28b300630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a61f24640 1 -- 172.21.15.172:0/30669531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f28b321e10 con 0x55f28b311060 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a61723640 1 --2- 172.21.15.172:0/30669531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b329970 0x55f28b300630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.752+0000 7f4a62725640 1 --2- 172.21.15.172:0/30669531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329d50 0x55f28b3bb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:39:36.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a61f24640 1 --2- 172.21.15.172:0/30669531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b311060 0x55f28b3220b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4a580116d0 tx=0x7f4a58021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af390b64d7750ae server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a3effd640 1 -- 172.21.15.172:0/30669531 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58025040 con 0x55f28b311060 2024-04-14T08:39:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a3effd640 1 -- 172.21.15.172:0/30669531 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5801a030 con 0x55f28b311060 2024-04-14T08:39:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a3effd640 1 -- 172.21.15.172:0/30669531 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58014910 con 0x55f28b311060 2024-04-14T08:39:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a66f6a740 1 -- 172.21.15.172:0/30669531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b311060 msgr2=0x55f28b3220b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/30669531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b311060 0x55f28b3220b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4a580116d0 tx=0x7f4a58021000 comp rx=0 tx=0).stop 2024-04-14T08:39:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a66f6a740 1 -- 172.21.15.172:0/30669531 shutdown_connections 2024-04-14T08:39:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/30669531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329d50 0x55f28b3bb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/30669531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b329970 0x55f28b300630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/30669531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b311060 0x55f28b3220b0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a66f6a740 1 -- 172.21.15.172:0/30669531 >> 172.21.15.172:0/30669531 conn(0x55f28b2fce50 msgr2=0x55f28b303930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a66f6a740 1 -- 172.21.15.172:0/30669531 shutdown_connections 2024-04-14T08:39:36.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.753+0000 7f4a66f6a740 1 -- 172.21.15.172:0/30669531 wait complete. 2024-04-14T08:39:36.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.754+0000 7f4a66f6a740 1 -- start start 2024-04-14T08:39:36.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.754+0000 7f4a66f6a740 1 Processor -- start 2024-04-14T08:39:36.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.755+0000 7f4a66f6a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b311060 0x55f28b35a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.755+0000 7f4a66f6a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329970 0x55f28b3b8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.755+0000 7f4a66f6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329d50 0x55f28b301100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.755+0000 7f4a66f6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f28b325b60 con 0x55f28b329d50 2024-04-14T08:39:36.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.755+0000 7f4a66f6a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1ffd10 con 0x55f28b311060 2024-04-14T08:39:36.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.755+0000 7f4a66f6a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f28b2f85f0 con 0x55f28b329970 2024-04-14T08:39:36.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.755+0000 7f4a66f6a740 10 client.0 _ll_register_callbacks cb 0x7f4a58011f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:39:36.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.755+0000 7f4a62725640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329d50 0x55f28b301100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a62725640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329d50 0x55f28b301100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57332/0 (socket says 172.21.15.172:57332) 2024-04-14T08:39:36.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a62725640 1 -- 172.21.15.172:0/810938797 learned_addr learned my addr 172.21.15.172:0/810938797 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:39:36.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a61f24640 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b311060 0x55f28b35a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a61723640 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329970 0x55f28b3b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a62725640 1 -- 172.21.15.172:0/810938797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329970 msgr2=0x55f28b3b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a62725640 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329970 0x55f28b3b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a62725640 1 -- 172.21.15.172:0/810938797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b311060 msgr2=0x55f28b35a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:39:36.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a62725640 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b311060 0x55f28b35a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a62725640 1 -- 172.21.15.172:0/810938797 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f28b301e10 con 0x55f28b329d50 2024-04-14T08:39:36.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a62725640 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329d50 0x55f28b301100 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4a54000f80 tx=0x7f4a54004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8c3d90180e320a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:39:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:39:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a3cff9640 1 -- 172.21.15.172:0/810938797 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54045040 con 0x55f28b329d50 2024-04-14T08:39:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a3cff9640 1 -- 172.21.15.172:0/810938797 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f4a54040470 con 0x55f28b329d50 2024-04-14T08:39:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a66f6a740 1 -- 172.21.15.172:0/810938797 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a5801a030 con 0x55f28b329d50 2024-04-14T08:39:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a3cff9640 1 -- 172.21.15.172:0/810938797 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f4a54055050 con 0x55f28b329d50 2024-04-14T08:39:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a66f6a740 10 client.7917 fetch_fsmap learned FSMap version 61 2024-04-14T08:39:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a66f6a740 1 -- 172.21.15.172:0/810938797 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55f28b3472f0 con 0x55f28b329d50 2024-04-14T08:39:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.756+0000 7f4a3cff9640 1 -- 172.21.15.172:0/810938797 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5801a030 con 0x55f28b329d50 2024-04-14T08:39:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a3cff9640 1 -- 172.21.15.172:0/810938797 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f4a54058020 con 0x55f28b329d50 2024-04-14T08:39:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 10 client.7917 fetch_fsmap finished waiting for FSMap version 61 2024-04-14T08:39:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 10 client.7917 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:39:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 20 client.7917 populate_metadata read hostname 'smithi172' 2024-04-14T08:39:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55f28b347810 0x55f28b347c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:39:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 4 client.7917 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:39:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 1 -- 172.21.15.172:0/810938797 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f28b327d30 con 0x55f28b347810 2024-04-14T08:39:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a61f24640 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55f28b347810 0x55f28b347c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:39:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a3cff9640 10 client.7917 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a61f24640 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55f28b347810 0x55f28b347c20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c44cc07904b0e7d2 server_cookie=eb954f3000b237a9 in_seq=0 out_seq=0 2024-04-14T08:39:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a3cff9640 1 -- 172.21.15.172:0/810938797 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55f28b327d30 con 0x55f28b347810 2024-04-14T08:39:36.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a3cff9640 10 client.7917 handle_command_reply: tid=0 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 1 client.7917 shutdown 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 1 -- 172.21.15.172:0/810938797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329d50 msgr2=0x55f28b301100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329d50 0x55f28b301100 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4a54000f80 tx=0x7f4a54004ab0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 1 -- 172.21.15.172:0/810938797 shutdown_connections 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55f28b347810 0x55f28b347c20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f28b329d50 0x55f28b301100 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f28b329970 0x55f28b3b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a3cff9640 0 client.7917 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 1 --2- 172.21.15.172:0/810938797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f28b311060 0x55f28b35a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.757+0000 7f4a66f6a740 1 -- 172.21.15.172:0/810938797 >> 172.21.15.172:0/810938797 conn(0x55f28b2fce50 msgr2=0x55f28b325210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a66f6a740 1 -- 172.21.15.172:0/810938797 shutdown_connections 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a66f6a740 1 -- 172.21.15.172:0/810938797 wait complete. 2024-04-14T08:39:36.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a66f6a740 20 client.7917 trim_cache size 0 max 16384 2024-04-14T08:39:36.771 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:39:36.771 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-14T08:39:36.771 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-04-14T08:39:36.771 INFO:teuthology.orchestra.run.smithi172.stdout: "5d9c155c-09eb-490d-9116-ab6230b56dfe": { 2024-04-14T08:39:36.771 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/dir", 2024-04-14T08:39:36.771 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "5d9c155c-09eb-490d-9116-ab6230b56dfe", 2024-04-14T08:39:36.771 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,repair" 2024-04-14T08:39:36.771 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:39:36.771 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:39:36.771 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:39:36.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a64188640 1 -- 172.21.15.172:0/4039062490 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4a4006c8f0 msgr2=0x7f4a4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a64188640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4a4006c8f0 0x7f4a4006edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009ad0 tx=0x7f4a58008040 comp rx=0 tx=0).stop 2024-04-14T08:39:36.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a64188640 1 -- 172.21.15.172:0/4039062490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a5c071250 msgr2=0x7f4a5c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:39:36.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a64188640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a5c071250 0x7f4a5c0b4d30 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00c440 tx=0x7f4a4c001e70 comp rx=0 tx=0).stop 2024-04-14T08:39:36.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a64188640 1 -- 172.21.15.172:0/4039062490 shutdown_connections 2024-04-14T08:39:36.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a64188640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4a4006c8f0 0x7f4a4006edb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a64188640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a5c0d1a20 0x7f4a5c0c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a64188640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a5c071250 0x7f4a5c0b4d30 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a64188640 1 --2- 172.21.15.172:0/4039062490 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c06d000 0x7f4a5c0b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:39:36.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a64188640 1 -- 172.21.15.172:0/4039062490 >> 172.21.15.172:0/4039062490 conn(0x7f4a5c08c000 msgr2=0x7f4a5c0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:39:36.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a64188640 1 -- 172.21.15.172:0/4039062490 shutdown_connections 2024-04-14T08:39:36.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:39:36.758+0000 7f4a64188640 1 -- 172.21.15.172:0/4039062490 wait complete. 2024-04-14T08:39:36.774 INFO:tasks.cephfs.filesystem:scrub status for tag:5d9c155c-09eb-490d-9116-ab6230b56dfe - {'path': '/dir', 'tag': '5d9c155c-09eb-490d-9116-ab6230b56dfe', 'options': 'recursive,repair'} 2024-04-14T08:39:54.526 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:39:54.529 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:40:00.000 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:39:59.999+0000 7f0f2508f640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-04-14T08:40:06.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:06.776 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-14T08:40:06.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.885+0000 7f0b75bbb640 1 Processor -- start 2024-04-14T08:40:06.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.885+0000 7f0b75bbb640 1 -- start start 2024-04-14T08:40:06.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b75bbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7015e430 0x7f0b70162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b75bbb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0b700d2e00 0x7f0b701629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b75bbb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b700cefb0 con 0x7f0b70162540 2024-04-14T08:40:06.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b75bbb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f0b700cf290 con 0x7f0b700d2e00 2024-04-14T08:40:06.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b75bbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0b700cf570 con 0x7f0b7015e430 2024-04-14T08:40:06.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b6effd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f0b70162540 0x7f0b700b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47662/0 (socket says 172.21.15.172:47662) 2024-04-14T08:40:06.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b6effd640 1 -- 172.21.15.172:0/390795720 learned_addr learned my addr 172.21.15.172:0/390795720 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:06.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/390795720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7015e430 0x7f0b70162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/390795720 >> v2:172.21.15.177:3300/0 conn(0x7f0b700d2e00 0x7f0b701629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b6e7fc640 1 -- 172.21.15.172:0/390795720 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 646610821 0 0) 0x7f0b700cefb0 con 0x7f0b70162540 2024-04-14T08:40:06.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b6e7fc640 1 -- 172.21.15.172:0/390795720 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b58003520 con 0x7f0b70162540 2024-04-14T08:40:06.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b6e7fc640 1 -- 172.21.15.172:0/390795720 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4047616332 0 0) 0x7f0b58003520 con 0x7f0b70162540 2024-04-14T08:40:06.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b6ffff640 1 -- 172.21.15.172:0/390795720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7015e430 msgr2=0x7f0b70162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:06.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/390795720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7015e430 0x7f0b70162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b6ffff640 1 -- 172.21.15.172:0/390795720 >> v1:172.21.15.172:6789/0 conn(0x7f0b70162540 legacy=0x7f0b700b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:06.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b6ffff640 1 -- 172.21.15.172:0/390795720 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b700cf850 con 0x7f0b700d2e00 2024-04-14T08:40:06.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.886+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/390795720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7015e430 0x7f0b70162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:40:06.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.887+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/390795720 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b700d2e00 0x7f0b701629a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009d90 tx=0x7f0b6802bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f9cf714b5cfef9f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:06.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.887+0000 7f0b6e7fc640 1 -- 172.21.15.172:0/390795720 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68039070 con 0x7f0b700d2e00 2024-04-14T08:40:06.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.887+0000 7f0b6e7fc640 1 -- 172.21.15.172:0/390795720 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b6802c8a0 con 0x7f0b700d2e00 2024-04-14T08:40:06.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.887+0000 7f0b6e7fc640 1 -- 172.21.15.172:0/390795720 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64001670 con 0x7f0b700d2e00 2024-04-14T08:40:06.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.888+0000 7f0b75bbb640 1 -- 172.21.15.172:0/390795720 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b700d2e00 msgr2=0x7f0b701629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:06.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.888+0000 7f0b75bbb640 1 --2- 172.21.15.172:0/390795720 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b700d2e00 0x7f0b701629a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009d90 tx=0x7f0b6802bd70 comp rx=0 tx=0).stop 2024-04-14T08:40:06.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.888+0000 7f0b75bbb640 1 -- 172.21.15.172:0/390795720 shutdown_connections 2024-04-14T08:40:06.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.888+0000 7f0b75bbb640 1 --2- 172.21.15.172:0/390795720 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b700d2e00 0x7f0b701629a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.888+0000 7f0b75bbb640 1 --2- 172.21.15.172:0/390795720 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7015e430 0x7f0b70162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.888+0000 7f0b75bbb640 1 -- 172.21.15.172:0/390795720 >> 172.21.15.172:0/390795720 conn(0x7f0b7009d560 msgr2=0x7f0b700d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:06.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.888+0000 7f0b75bbb640 1 -- 172.21.15.172:0/390795720 shutdown_connections 2024-04-14T08:40:06.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.888+0000 7f0b75bbb640 1 -- 172.21.15.172:0/390795720 wait complete. 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.889+0000 7f0b75bbb640 1 Processor -- start 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.890+0000 7f0b75bbb640 1 -- start start 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.890+0000 7f0b75bbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b700d2e00 0x7f0b700ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.890+0000 7f0b75bbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b7015e430 0x7f0b700c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.890+0000 7f0b75bbb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b70162540 0x7f0b701613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.890+0000 7f0b75bbb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b700d4e20 con 0x7f0b7015e430 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.890+0000 7f0b75bbb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7015e810 con 0x7f0b70162540 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.890+0000 7f0b75bbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0b700d0050 con 0x7f0b700d2e00 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.890+0000 7f0b6effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b7015e430 0x7f0b700c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.890+0000 7f0b6effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b7015e430 0x7f0b700c9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38870/0 (socket says 172.21.15.172:38870) 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.890+0000 7f0b6effd640 1 -- 172.21.15.172:0/836247448 learned_addr learned my addr 172.21.15.172:0/836247448 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.890+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b70162540 0x7f0b701613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.890+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b700d2e00 0x7f0b700ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.891+0000 7f0b6ffff640 1 -- 172.21.15.172:0/836247448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b700d2e00 msgr2=0x7f0b700ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.891+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b700d2e00 0x7f0b700ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.891+0000 7f0b6ffff640 1 -- 172.21.15.172:0/836247448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b7015e430 msgr2=0x7f0b700c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.891+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b7015e430 0x7f0b700c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.891+0000 7f0b6ffff640 1 -- 172.21.15.172:0/836247448 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b700c96d0 con 0x7f0b70162540 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.891+0000 7f0b6effd640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b7015e430 0x7f0b700c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.891+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b70162540 0x7f0b701613a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0b68033000 tx=0x7f0b68004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.891+0000 7f0b57fff640 1 -- 172.21.15.172:0/836247448 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68039050 con 0x7f0b70162540 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.891+0000 7f0b57fff640 1 -- 172.21.15.172:0/836247448 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b68045070 con 0x7f0b70162540 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.891+0000 7f0b75bbb640 1 -- 172.21.15.172:0/836247448 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b700c99b0 con 0x7f0b70162540 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.891+0000 7f0b57fff640 1 -- 172.21.15.172:0/836247448 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68009450 con 0x7f0b70162540 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.891+0000 7f0b75bbb640 1 -- 172.21.15.172:0/836247448 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b700c9f70 con 0x7f0b70162540 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.893+0000 7f0b57fff640 1 -- 172.21.15.172:0/836247448 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b68040690 con 0x7f0b70162540 2024-04-14T08:40:06.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.893+0000 7f0b57fff640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.893+0000 7f0b57fff640 1 -- 172.21.15.172:0/836247448 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f0b680ae500 con 0x7f0b70162540 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.893+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.894+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009560 tx=0x7f0b60009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.894+0000 7f0b7899f740 1 Processor -- start 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.894+0000 7f0b7899f740 1 -- start start 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.895+0000 7f0b7899f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 0x56306bfe2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.895+0000 7f0b7899f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00fd50 0x56306c012700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.895+0000 7f0b7899f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306bfe2e50 0x56306c012c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.895+0000 7f0b7899f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56306bf4c800 con 0x56306c00f970 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.895+0000 7f0b7899f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56306bef47f0 con 0x56306bfe2e50 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.895+0000 7f0b7899f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56306bf487b0 con 0x56306c00fd50 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.895+0000 7f0b6f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 0x56306bfe2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.895+0000 7f0b6f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 0x56306bfe2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38884/0 (socket says 172.21.15.172:38884) 2024-04-14T08:40:06.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.895+0000 7f0b6f7fe640 1 -- 172.21.15.172:0/4128330229 learned_addr learned my addr 172.21.15.172:0/4128330229 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.895+0000 7f0b6effd640 1 --2- 172.21.15.172:0/4128330229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00fd50 0x56306c012700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.896+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/4128330229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306bfe2e50 0x56306c012c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.896+0000 7f0b6f7fe640 1 -- 172.21.15.172:0/4128330229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00fd50 msgr2=0x56306c012700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.896+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/4128330229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00fd50 0x56306c012700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.896+0000 7f0b6f7fe640 1 -- 172.21.15.172:0/4128330229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306bfe2e50 msgr2=0x56306c012c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.896+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/4128330229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306bfe2e50 0x56306c012c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.896+0000 7f0b6f7fe640 1 -- 172.21.15.172:0/4128330229 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56306bd2a7d0 con 0x56306c00f970 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.896+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/4128330229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 0x56306bfe2910 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f0b60011ef0 tx=0x7f0b60014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3669e974ed1515ef server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.896+0000 7f0b557fa640 1 -- 172.21.15.172:0/4128330229 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6001b070 con 0x56306c00f970 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.896+0000 7f0b557fa640 1 -- 172.21.15.172:0/4128330229 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b60027070 con 0x56306c00f970 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.896+0000 7f0b557fa640 1 -- 172.21.15.172:0/4128330229 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60003a20 con 0x56306c00f970 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.897+0000 7f0b7899f740 1 -- 172.21.15.172:0/4128330229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 msgr2=0x56306bfe2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:06.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.897+0000 7f0b7899f740 1 --2- 172.21.15.172:0/4128330229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 0x56306bfe2910 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f0b60011ef0 tx=0x7f0b60014000 comp rx=0 tx=0).stop 2024-04-14T08:40:06.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.897+0000 7f0b7899f740 1 -- 172.21.15.172:0/4128330229 shutdown_connections 2024-04-14T08:40:06.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.897+0000 7f0b7899f740 1 --2- 172.21.15.172:0/4128330229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306bfe2e50 0x56306c012c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.897+0000 7f0b7899f740 1 --2- 172.21.15.172:0/4128330229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00fd50 0x56306c012700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.897+0000 7f0b7899f740 1 --2- 172.21.15.172:0/4128330229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 0x56306bfe2910 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.897+0000 7f0b7899f740 1 -- 172.21.15.172:0/4128330229 >> 172.21.15.172:0/4128330229 conn(0x56306bfe4b80 msgr2=0x56306c02d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:06.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.897+0000 7f0b7899f740 1 -- 172.21.15.172:0/4128330229 shutdown_connections 2024-04-14T08:40:06.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.897+0000 7f0b7899f740 1 -- 172.21.15.172:0/4128330229 wait complete. 2024-04-14T08:40:06.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.898+0000 7f0b7899f740 1 -- start start 2024-04-14T08:40:06.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.898+0000 7f0b7899f740 1 Processor -- start 2024-04-14T08:40:06.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.898+0000 7f0b7899f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c00f970 0x56306c00d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.898+0000 7f0b7899f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00fd50 0x56306c015f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.898+0000 7f0b7899f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c0262e0 0x56306c026760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.898+0000 7f0b7899f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56306bfe7120 con 0x56306c0262e0 2024-04-14T08:40:06.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.898+0000 7f0b7899f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56306bf07d00 con 0x56306c00f970 2024-04-14T08:40:06.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.898+0000 7f0b7899f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56306bfe6510 con 0x56306c00fd50 2024-04-14T08:40:06.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b7899f740 10 client.0 _ll_register_callbacks cb 0x56306bf076f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:06.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b6ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c0262e0 0x56306c026760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b6ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c0262e0 0x56306c026760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38898/0 (socket says 172.21.15.172:38898) 2024-04-14T08:40:06.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b6ffff640 1 -- 172.21.15.172:0/1024264829 learned_addr learned my addr 172.21.15.172:0/1024264829 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:06.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b6effd640 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00fd50 0x56306c015f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c00f970 0x56306c00d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b6ffff640 1 -- 172.21.15.172:0/1024264829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00fd50 msgr2=0x56306c015f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:06.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00fd50 0x56306c015f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b6ffff640 1 -- 172.21.15.172:0/1024264829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c00f970 msgr2=0x56306c00d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:06.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c00f970 0x56306c00d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b6ffff640 1 -- 172.21.15.172:0/1024264829 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56306bc603f0 con 0x56306c0262e0 2024-04-14T08:40:06.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c0262e0 0x56306c026760 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f0b68037040 tx=0x7f0b6807bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76cf4d566cb6344 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:06.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:06.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b4e7fc640 1 -- 172.21.15.172:0/1024264829 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68080300 con 0x56306c0262e0 2024-04-14T08:40:06.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b4e7fc640 1 -- 172.21.15.172:0/1024264829 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f0b68084730 con 0x56306c0262e0 2024-04-14T08:40:06.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024264829 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b60027070 con 0x56306c0262e0 2024-04-14T08:40:06.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.899+0000 7f0b4e7fc640 1 -- 172.21.15.172:0/1024264829 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f0b68084bc0 con 0x56306c0262e0 2024-04-14T08:40:06.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b7899f740 10 client.7929 fetch_fsmap learned FSMap version 61 2024-04-14T08:40:06.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024264829 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x56306bc200f0 con 0x56306c0262e0 2024-04-14T08:40:06.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b4e7fc640 1 -- 172.21.15.172:0/1024264829 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b60027070 con 0x56306c0262e0 2024-04-14T08:40:06.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b4e7fc640 1 -- 172.21.15.172:0/1024264829 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f0b6807f040 con 0x56306c0262e0 2024-04-14T08:40:06.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b7899f740 10 client.7929 fetch_fsmap finished waiting for FSMap version 61 2024-04-14T08:40:06.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b7899f740 10 client.7929 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:40:06.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b7899f740 20 client.7929 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:06.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b7899f740 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56306c03d310 0x56306c025950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b7899f740 4 client.7929 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:40:06.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024264829 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56306bf59d40 con 0x56306c03d310 2024-04-14T08:40:06.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56306c03d310 0x56306c025950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b4e7fc640 10 client.7929 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:40:06.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.900+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56306c03d310 0x56306c025950 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f5410e5130a4c5 server_cookie=24089f59540942fc in_seq=0 out_seq=0 2024-04-14T08:40:06.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b4e7fc640 1 -- 172.21.15.172:0/1024264829 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56306bf59d40 con 0x56306c03d310 2024-04-14T08:40:06.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b4e7fc640 10 client.7929 handle_command_reply: tid=0 2024-04-14T08:40:06.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b7899f740 1 client.7929 shutdown 2024-04-14T08:40:06.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024264829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c0262e0 msgr2=0x56306c026760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:06.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b7899f740 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c0262e0 0x56306c026760 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f0b68037040 tx=0x7f0b6807bc10 comp rx=0 tx=0).stop 2024-04-14T08:40:06.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024264829 shutdown_connections 2024-04-14T08:40:06.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b7899f740 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56306c03d310 0x56306c025950 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b7899f740 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c0262e0 0x56306c026760 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b7899f740 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00fd50 0x56306c015f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b4e7fc640 0 client.7929 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:40:06.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b7899f740 1 --2- 172.21.15.172:0/1024264829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c00f970 0x56306c00d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024264829 >> 172.21.15.172:0/1024264829 conn(0x56306bfe2e50 msgr2=0x56306c0126f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:06.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024264829 shutdown_connections 2024-04-14T08:40:06.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024264829 wait complete. 2024-04-14T08:40:06.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.901+0000 7f0b7899f740 20 client.7929 trim_cache size 0 max 16384 2024-04-14T08:40:06.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.919+0000 7f0b7899f740 1 Processor -- start 2024-04-14T08:40:06.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.919+0000 7f0b7899f740 1 -- start start 2024-04-14T08:40:06.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b7899f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00f970 0x56306c0510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b7899f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00fd50 0x56306bfe9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b7899f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c0262e0 0x56306c031030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b7899f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b70200f40 con 0x56306c00fd50 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b7899f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0b701fff70 con 0x56306c0262e0 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b7899f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0b700cbfa0 con 0x56306c00f970 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b6effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00fd50 0x56306bfe9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b6effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00fd50 0x56306bfe9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38914/0 (socket says 172.21.15.172:38914) 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b6effd640 1 -- 172.21.15.172:0/3330985730 learned_addr learned my addr 172.21.15.172:0/3330985730 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b6effd640 1 -- 172.21.15.172:0/3330985730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00f970 msgr2=0x56306c0510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/3330985730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00f970 0x56306c0510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/3330985730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c0262e0 0x56306c031030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b6effd640 1 --2- 172.21.15.172:0/3330985730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00f970 0x56306c0510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b6effd640 1 -- 172.21.15.172:0/3330985730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c0262e0 msgr2=0x56306c031030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b6effd640 1 --2- 172.21.15.172:0/3330985730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c0262e0 0x56306c031030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b6effd640 1 -- 172.21.15.172:0/3330985730 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56306bfe85c0 con 0x56306c00fd50 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.920+0000 7f0b6effd640 1 --2- 172.21.15.172:0/3330985730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00fd50 0x56306bfe9930 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f0b64015410 tx=0x7f0b640158d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f74705dad404e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:06.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.921+0000 7f0b54ff9640 1 -- 172.21.15.172:0/3330985730 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64005ba0 con 0x56306c00fd50 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.921+0000 7f0b54ff9640 1 -- 172.21.15.172:0/3330985730 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b64005d40 con 0x56306c00fd50 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.921+0000 7f0b54ff9640 1 -- 172.21.15.172:0/3330985730 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b64007330 con 0x56306c00fd50 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.921+0000 7f0b7899f740 1 -- 172.21.15.172:0/3330985730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00fd50 msgr2=0x56306bfe9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.921+0000 7f0b7899f740 1 --2- 172.21.15.172:0/3330985730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00fd50 0x56306bfe9930 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f0b64015410 tx=0x7f0b640158d0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.921+0000 7f0b7899f740 1 -- 172.21.15.172:0/3330985730 shutdown_connections 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.921+0000 7f0b7899f740 1 --2- 172.21.15.172:0/3330985730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c0262e0 0x56306c031030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.921+0000 7f0b7899f740 1 --2- 172.21.15.172:0/3330985730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00fd50 0x56306bfe9930 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.921+0000 7f0b7899f740 1 --2- 172.21.15.172:0/3330985730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c00f970 0x56306c0510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.921+0000 7f0b7899f740 1 -- 172.21.15.172:0/3330985730 >> 172.21.15.172:0/3330985730 conn(0x56306bfe2e50 msgr2=0x56306c016580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.921+0000 7f0b7899f740 1 -- 172.21.15.172:0/3330985730 shutdown_connections 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.921+0000 7f0b7899f740 1 -- 172.21.15.172:0/3330985730 wait complete. 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.922+0000 7f0b7899f740 1 -- start start 2024-04-14T08:40:06.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.922+0000 7f0b7899f740 1 Processor -- start 2024-04-14T08:40:06.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b7899f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 0x56306c0470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b7899f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c00fd50 0x56306bfe1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b7899f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c0262e0 0x56306bfe1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b7899f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b700c8a10 con 0x56306c00f970 2024-04-14T08:40:06.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b7899f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56306c00f5a0 con 0x56306c00fd50 2024-04-14T08:40:06.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b7899f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56306c0a7d40 con 0x56306c0262e0 2024-04-14T08:40:06.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b7899f740 10 client.0 _ll_register_callbacks cb 0x7f0b68078390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:06.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b6f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 0x56306c0470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b6f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 0x56306c0470c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38926/0 (socket says 172.21.15.172:38926) 2024-04-14T08:40:06.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b6f7fe640 1 -- 172.21.15.172:0/1024117756 learned_addr learned my addr 172.21.15.172:0/1024117756 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:06.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b6ffff640 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c0262e0 0x56306bfe1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b6effd640 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c00fd50 0x56306bfe1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b6f7fe640 1 -- 172.21.15.172:0/1024117756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c0262e0 msgr2=0x56306bfe1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:06.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c0262e0 0x56306bfe1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b6f7fe640 1 -- 172.21.15.172:0/1024117756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c00fd50 msgr2=0x56306bfe1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:06.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c00fd50 0x56306bfe1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b6f7fe640 1 -- 172.21.15.172:0/1024117756 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56306c03d6f0 con 0x56306c00f970 2024-04-14T08:40:06.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b6f7fe640 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 0x56306c0470c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0b60011e90 tx=0x7f0b60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad8c70910857d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:06.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b4e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:06.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b4e7fc640 1 -- 172.21.15.172:0/1024117756 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6001b040 con 0x56306c00f970 2024-04-14T08:40:06.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b4e7fc640 1 -- 172.21.15.172:0/1024117756 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f0b60011a60 con 0x56306c00f970 2024-04-14T08:40:06.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.923+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024117756 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b64005d40 con 0x56306c00f970 2024-04-14T08:40:06.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b4e7fc640 1 -- 172.21.15.172:0/1024117756 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f0b60046050 con 0x56306c00f970 2024-04-14T08:40:06.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b7899f740 10 client.7935 fetch_fsmap learned FSMap version 61 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024117756 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x56306c0268a0 con 0x56306c00f970 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b4e7fc640 1 -- 172.21.15.172:0/1024117756 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b64005d40 con 0x56306c00f970 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b4e7fc640 1 -- 172.21.15.172:0/1024117756 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f0b60049020 con 0x56306c00f970 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b7899f740 10 client.7935 fetch_fsmap finished waiting for FSMap version 61 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b7899f740 10 client.7935 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b7899f740 20 client.7935 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b7899f740 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56306c0a6ce0 0x56306c0a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b7899f740 4 client.7935 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024117756 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56306bf54090 con 0x56306c0a6ce0 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b6effd640 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56306c0a6ce0 0x56306c0a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b4e7fc640 10 client.7935 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.924+0000 7f0b6effd640 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56306c0a6ce0 0x56306c0a70f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f129ed8c2aecd9 server_cookie=f1b6d54a6e02911c in_seq=0 out_seq=0 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b4e7fc640 1 -- 172.21.15.172:0/1024117756 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56306bf54090 con 0x56306c0a6ce0 2024-04-14T08:40:06.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b4e7fc640 10 client.7935 handle_command_reply: tid=0 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b7899f740 1 client.7935 shutdown 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024117756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 msgr2=0x56306c0470c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b7899f740 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 0x56306c0470c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0b60011e90 tx=0x7f0b60004730 comp rx=0 tx=0).stop 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024117756 shutdown_connections 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b7899f740 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x56306c0a6ce0 0x56306c0a70f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b7899f740 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56306c0262e0 0x56306bfe1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b7899f740 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56306c00fd50 0x56306bfe1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b7899f740 1 --2- 172.21.15.172:0/1024117756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56306c00f970 0x56306c0470c0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024117756 >> 172.21.15.172:0/1024117756 conn(0x56306bfe2e50 msgr2=0x56306c025950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b4e7fc640 0 client.7935 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024117756 shutdown_connections 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b7899f740 1 -- 172.21.15.172:0/1024117756 wait complete. 2024-04-14T08:40:06.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b7899f740 20 client.7935 trim_cache size 0 max 16384 2024-04-14T08:40:06.933 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:40:06.933 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-04-14T08:40:06.933 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-04-14T08:40:06.933 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:40:06.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b75bbb640 1 -- 172.21.15.172:0/836247448 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0b5c06c8f0 msgr2=0x7f0b5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:06.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b75bbb640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009560 tx=0x7f0b60009190 comp rx=0 tx=0).stop 2024-04-14T08:40:06.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b75bbb640 1 -- 172.21.15.172:0/836247448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b70162540 msgr2=0x7f0b701613a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:06.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.925+0000 7f0b75bbb640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b70162540 0x7f0b701613a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0b68033000 tx=0x7f0b68004a40 comp rx=0 tx=0).stop 2024-04-14T08:40:06.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.926+0000 7f0b75bbb640 1 -- 172.21.15.172:0/836247448 shutdown_connections 2024-04-14T08:40:06.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.926+0000 7f0b75bbb640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.926+0000 7f0b75bbb640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b70162540 0x7f0b701613a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.926+0000 7f0b75bbb640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b7015e430 0x7f0b700c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.926+0000 7f0b75bbb640 1 --2- 172.21.15.172:0/836247448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b700d2e00 0x7f0b700ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:06.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.926+0000 7f0b75bbb640 1 -- 172.21.15.172:0/836247448 >> 172.21.15.172:0/836247448 conn(0x7f0b7009d560 msgr2=0x7f0b700b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:06.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.926+0000 7f0b75bbb640 1 -- 172.21.15.172:0/836247448 shutdown_connections 2024-04-14T08:40:06.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:06.926+0000 7f0b75bbb640 1 -- 172.21.15.172:0/836247448 wait complete. 2024-04-14T08:40:06.937 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-14T08:40:06.937 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:40:07.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.039+0000 7fc07cd1e640 1 Processor -- start 2024-04-14T08:40:07.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.039+0000 7fc07cd1e640 1 -- start start 2024-04-14T08:40:07.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc07cd1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc0780b7a60 0x7fc0780ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc07cd1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0780d2b10 con 0x7fc07808c4c0 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc07cd1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0780d0ad0 con 0x7fc07806dc40 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc07cd1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc0780d0db0 con 0x7fc0780b7a60 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc06dd9b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc07808c4c0 0x7fc0780cd410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47678/0 (socket says 172.21.15.172:47678) 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc06dd9b640 1 -- 172.21.15.172:0/680855200 learned_addr learned my addr 172.21.15.172:0/680855200 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3124608171 0 0) 0x7fc0780d2b10 con 0x7fc07808c4c0 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc058003520 con 0x7fc07808c4c0 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc07659c640 1 --2- 172.21.15.172:0/680855200 >> v2:172.21.15.177:3300/0 conn(0x7fc0780b7a60 0x7fc0780ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1612490808 0 0) 0x7fc058003520 con 0x7fc07808c4c0 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 >> v2:172.21.15.177:3300/0 conn(0x7fc0780b7a60 msgr2=0x7fc0780ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc075d9b640 1 --2- 172.21.15.172:0/680855200 >> v2:172.21.15.177:3300/0 conn(0x7fc0780b7a60 0x7fc0780ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 >> v1:172.21.15.177:6789/0 conn(0x7fc07806dc40 legacy=0x7fc0780cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0780d1090 con 0x7fc07808c4c0 2024-04-14T08:40:07.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fc060001670 con 0x7fc07808c4c0 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc07659c640 1 --2- 172.21.15.172:0/680855200 >> v2:172.21.15.177:3300/0 conn(0x7fc0780b7a60 0x7fc0780ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 >> v1:172.21.15.172:6789/0 conn(0x7fc07808c4c0 legacy=0x7fc0780cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.040+0000 7fc075d9b640 1 --2- 172.21.15.172:0/680855200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0580039b0 0x7fc058003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc075d9b640 1 --2- 172.21.15.172:0/680855200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc058004800 0x7fc058004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc075d9b640 1 --2- 172.21.15.172:0/680855200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0580052e0 0x7fc05800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0780d0db0 con 0x7fc0580039b0 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc05800c1b0 con 0x7fc058004800 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc05800c3b0 con 0x7fc0580052e0 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc07659c640 1 --2- 172.21.15.172:0/680855200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0580039b0 0x7fc058003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc06dd9b640 1 --2- 172.21.15.172:0/680855200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc058004800 0x7fc058004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc076d9d640 1 --2- 172.21.15.172:0/680855200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0580052e0 0x7fc05800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc07659c640 1 -- 172.21.15.172:0/680855200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0580052e0 msgr2=0x7fc05800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc07659c640 1 --2- 172.21.15.172:0/680855200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0580052e0 0x7fc05800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc07659c640 1 -- 172.21.15.172:0/680855200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc058004800 msgr2=0x7fc058004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc07659c640 1 --2- 172.21.15.172:0/680855200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc058004800 0x7fc058004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc07659c640 1 -- 172.21.15.172:0/680855200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc058003520 con 0x7fc0580039b0 2024-04-14T08:40:07.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.041+0000 7fc076d9d640 1 -- 172.21.15.172:0/680855200 reap_dead start 2024-04-14T08:40:07.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.042+0000 7fc07659c640 1 --2- 172.21.15.172:0/680855200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0580039b0 0x7fc058003dd0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fc068006fd0 tx=0x7fc058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7bdefb68ea8db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.042+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06800f040 con 0x7fc0580039b0 2024-04-14T08:40:07.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.042+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc068004030 con 0x7fc0580039b0 2024-04-14T08:40:07.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.042+0000 7fc075d9b640 1 -- 172.21.15.172:0/680855200 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068004440 con 0x7fc0580039b0 2024-04-14T08:40:07.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.042+0000 7fc07cd1e640 1 -- 172.21.15.172:0/680855200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0580039b0 msgr2=0x7fc058003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.042+0000 7fc07cd1e640 1 --2- 172.21.15.172:0/680855200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0580039b0 0x7fc058003dd0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fc068006fd0 tx=0x7fc058005830 comp rx=0 tx=0).stop 2024-04-14T08:40:07.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.042+0000 7fc07cd1e640 1 -- 172.21.15.172:0/680855200 shutdown_connections 2024-04-14T08:40:07.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.042+0000 7fc07cd1e640 1 --2- 172.21.15.172:0/680855200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0580039b0 0x7fc058003dd0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.042+0000 7fc07cd1e640 1 -- 172.21.15.172:0/680855200 >> 172.21.15.172:0/680855200 conn(0x7fc07808c000 msgr2=0x7fc0780d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:07.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.042+0000 7fc07cd1e640 1 -- 172.21.15.172:0/680855200 shutdown_connections 2024-04-14T08:40:07.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.042+0000 7fc07cd1e640 1 -- 172.21.15.172:0/680855200 wait complete. 2024-04-14T08:40:07.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.044+0000 7fc07cd1e640 1 Processor -- start 2024-04-14T08:40:07.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.044+0000 7fc07cd1e640 1 -- start start 2024-04-14T08:40:07.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc07cd1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0580039b0 0x7fc07814e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc07cd1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc078150d50 0x7fc0781511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc07cd1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc07814fb20 0x7fc07814ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc07cd1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0780d2750 con 0x7fc078150d50 2024-04-14T08:40:07.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc07cd1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0780b7e40 con 0x7fc07814fb20 2024-04-14T08:40:07.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc07cd1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc0780d28d0 con 0x7fc0580039b0 2024-04-14T08:40:07.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc06dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc078150d50 0x7fc0781511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc06dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc078150d50 0x7fc0781511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38950/0 (socket says 172.21.15.172:38950) 2024-04-14T08:40:07.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc06dd9b640 1 -- 172.21.15.172:0/543212738 learned_addr learned my addr 172.21.15.172:0/543212738 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:07.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc06dd9b640 1 -- 172.21.15.172:0/543212738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0580039b0 msgr2=0x7fc07814e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc07659c640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0580039b0 0x7fc07814e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc076d9d640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc07814fb20 0x7fc07814ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc06dd9b640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0580039b0 0x7fc07814e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc06dd9b640 1 -- 172.21.15.172:0/543212738 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc07814fb20 msgr2=0x7fc07814ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc06dd9b640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc07814fb20 0x7fc07814ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc06dd9b640 1 -- 172.21.15.172:0/543212738 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0780cab70 con 0x7fc078150d50 2024-04-14T08:40:07.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc06dd9b640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc078150d50 0x7fc0781511d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc0600072c0 tx=0x7fc060007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc06f7fe640 1 -- 172.21.15.172:0/543212738 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06000f8e0 con 0x7fc078150d50 2024-04-14T08:40:07.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc06f7fe640 1 -- 172.21.15.172:0/543212738 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc060018070 con 0x7fc078150d50 2024-04-14T08:40:07.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc07cd1e640 1 -- 172.21.15.172:0/543212738 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07814ed60 con 0x7fc078150d50 2024-04-14T08:40:07.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc06f7fe640 1 -- 172.21.15.172:0/543212738 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06000fce0 con 0x7fc078150d50 2024-04-14T08:40:07.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.045+0000 7fc07cd1e640 1 -- 172.21.15.172:0/543212738 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07814f320 con 0x7fc078150d50 2024-04-14T08:40:07.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.046+0000 7fc07cd1e640 1 -- 172.21.15.172:0/543212738 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc078075490 con 0x7fc078150d50 2024-04-14T08:40:07.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.047+0000 7fc06f7fe640 1 -- 172.21.15.172:0/543212738 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc06001d460 con 0x7fc078150d50 2024-04-14T08:40:07.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.048+0000 7fc06f7fe640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05c06c8f0 0x7fc05c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.048+0000 7fc06f7fe640 1 -- 172.21.15.172:0/543212738 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7fc06008be80 con 0x7fc078150d50 2024-04-14T08:40:07.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.048+0000 7fc07659c640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05c06c8f0 0x7fc05c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.048+0000 7fc07659c640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05c06c8f0 0x7fc05c06edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc068035000 tx=0x7fc06803a040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.052+0000 7fc06f7fe640 1 -- 172.21.15.172:0/543212738 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc06005a060 con 0x7fc078150d50 2024-04-14T08:40:07.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.266+0000 7fc07cd1e640 1 -- 172.21.15.172:0/543212738 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0780d2db0 con 0x7fc078150d50 2024-04-14T08:40:07.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.266+0000 7fc06f7fe640 1 -- 172.21.15.172:0/543212738 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5196 (secure 0 0 0) 0x7fc060059960 con 0x7fc078150d50 2024-04-14T08:40:07.267 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:07.267 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:39:05.413377+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7473":{"gid":7473,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6837/1386685680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1386685680},{"type":"v1","addr":"172.21.15.172:6837","nonce":1386685680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7473,"qdb_cluster":[7473]},"id":6}]} 2024-04-14T08:40:07.268 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 61 2024-04-14T08:40:07.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.268+0000 7fc07cd1e640 1 -- 172.21.15.172:0/543212738 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05c06c8f0 msgr2=0x7fc05c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.268+0000 7fc07cd1e640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05c06c8f0 0x7fc05c06edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc068035000 tx=0x7fc06803a040 comp rx=0 tx=0).stop 2024-04-14T08:40:07.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.268+0000 7fc07cd1e640 1 -- 172.21.15.172:0/543212738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc078150d50 msgr2=0x7fc0781511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.268+0000 7fc07cd1e640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc078150d50 0x7fc0781511d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc0600072c0 tx=0x7fc060007780 comp rx=0 tx=0).stop 2024-04-14T08:40:07.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.268+0000 7fc07cd1e640 1 -- 172.21.15.172:0/543212738 shutdown_connections 2024-04-14T08:40:07.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.268+0000 7fc07cd1e640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05c06c8f0 0x7fc05c06edb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.268+0000 7fc07cd1e640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc07814fb20 0x7fc07814ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.268+0000 7fc07cd1e640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc078150d50 0x7fc0781511d0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.268+0000 7fc07cd1e640 1 --2- 172.21.15.172:0/543212738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0580039b0 0x7fc07814e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.268+0000 7fc07cd1e640 1 -- 172.21.15.172:0/543212738 >> 172.21.15.172:0/543212738 conn(0x7fc07808c000 msgr2=0x7fc078160e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:07.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.268+0000 7fc07cd1e640 1 -- 172.21.15.172:0/543212738 shutdown_connections 2024-04-14T08:40:07.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.268+0000 7fc07cd1e640 1 -- 172.21.15.172:0/543212738 wait complete. 2024-04-14T08:40:07.280 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-14T08:40:07.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.387+0000 7f14ed4c0640 1 Processor -- start 2024-04-14T08:40:07.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.387+0000 7f14ed4c0640 1 -- start start 2024-04-14T08:40:07.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.387+0000 7f14ed4c0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f14e8161ce0 0x7f14e80d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.387+0000 7f14ed4c0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f14e80d1320 0x7f14e80b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.387+0000 7f14ed4c0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14e80cedc0 con 0x7f14e80d2d40 2024-04-14T08:40:07.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.387+0000 7f14ed4c0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f14e80d31a0 con 0x7f14e80d1320 2024-04-14T08:40:07.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.387+0000 7f14ed4c0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f14e80d00e0 con 0x7f14e8161ce0 2024-04-14T08:40:07.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.387+0000 7f14e77fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f14e80d1320 0x7f14e80b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.388+0000 7f14e77fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f14e80d1320 0x7f14e80b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38960/0 (socket says 172.21.15.172:38960) 2024-04-14T08:40:07.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.388+0000 7f14e77fe640 1 -- 172.21.15.172:0/638588666 learned_addr learned my addr 172.21.15.172:0/638588666 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:07.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.388+0000 7f14e6ffd640 1 --2- 172.21.15.172:0/638588666 >> v2:172.21.15.177:3300/0 conn(0x7f14e8161ce0 0x7f14e80d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.388+0000 7f14e77fe640 1 -- 172.21.15.172:0/638588666 >> v2:172.21.15.177:3300/0 conn(0x7f14e8161ce0 msgr2=0x7f14e80d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.388+0000 7f14e77fe640 1 --2- 172.21.15.172:0/638588666 >> v2:172.21.15.177:3300/0 conn(0x7f14e8161ce0 0x7f14e80d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.388+0000 7f14e77fe640 1 -- 172.21.15.172:0/638588666 >> v1:172.21.15.177:6789/0 conn(0x7f14e80d2d40 legacy=0x7f14e80d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.388+0000 7f14e77fe640 1 -- 172.21.15.172:0/638588666 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e80cf040 con 0x7f14e80d1320 2024-04-14T08:40:07.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.389+0000 7f14e77fe640 1 --2- 172.21.15.172:0/638588666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e80d1320 0x7f14e80b0e50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f14e80d33a0 tx=0x7f14e002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e635f9182a26b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.389+0000 7f14e67fc640 1 -- 172.21.15.172:0/638588666 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e002cb20 con 0x7f14e80d1320 2024-04-14T08:40:07.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.389+0000 7f14e67fc640 1 -- 172.21.15.172:0/638588666 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e002ccc0 con 0x7f14e80d1320 2024-04-14T08:40:07.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.389+0000 7f14e67fc640 1 -- 172.21.15.172:0/638588666 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0034610 con 0x7f14e80d1320 2024-04-14T08:40:07.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.389+0000 7f14ed4c0640 1 -- 172.21.15.172:0/638588666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e80d1320 msgr2=0x7f14e80b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.389+0000 7f14ed4c0640 1 --2- 172.21.15.172:0/638588666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e80d1320 0x7f14e80b0e50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f14e80d33a0 tx=0x7f14e002bf80 comp rx=0 tx=0).stop 2024-04-14T08:40:07.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.389+0000 7f14ed4c0640 1 -- 172.21.15.172:0/638588666 shutdown_connections 2024-04-14T08:40:07.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.389+0000 7f14ed4c0640 1 --2- 172.21.15.172:0/638588666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e80d1320 0x7f14e80b0e50 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.389+0000 7f14ed4c0640 1 --2- 172.21.15.172:0/638588666 >> v2:172.21.15.177:3300/0 conn(0x7f14e8161ce0 0x7f14e80d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.389+0000 7f14ed4c0640 1 -- 172.21.15.172:0/638588666 >> 172.21.15.172:0/638588666 conn(0x7f14e809d560 msgr2=0x7f14e80d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:07.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.389+0000 7f14ed4c0640 1 -- 172.21.15.172:0/638588666 shutdown_connections 2024-04-14T08:40:07.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.389+0000 7f14ed4c0640 1 -- 172.21.15.172:0/638588666 wait complete. 2024-04-14T08:40:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.391+0000 7f14ed4c0640 1 Processor -- start 2024-04-14T08:40:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.391+0000 7f14ed4c0640 1 -- start start 2024-04-14T08:40:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14ed4c0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14e80d1320 0x7f14e81f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14ed4c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e80d2d40 0x7f14e80cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14ed4c0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14e8161ce0 0x7f14e80c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14ed4c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f14e80cf8e0 con 0x7f14e80d2d40 2024-04-14T08:40:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14ed4c0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f14e80cf600 con 0x7f14e8161ce0 2024-04-14T08:40:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14ed4c0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f14e80cfbc0 con 0x7f14e80d1320 2024-04-14T08:40:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14dffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e80d2d40 0x7f14e80cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14dffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e80d2d40 0x7f14e80cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38972/0 (socket says 172.21.15.172:38972) 2024-04-14T08:40:07.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14dffff640 1 -- 172.21.15.172:0/2602354663 learned_addr learned my addr 172.21.15.172:0/2602354663 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:07.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14e77fe640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14e8161ce0 0x7f14e80c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14e6ffd640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14e80d1320 0x7f14e81f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14dffff640 1 -- 172.21.15.172:0/2602354663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14e80d1320 msgr2=0x7f14e81f55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14dffff640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14e80d1320 0x7f14e81f55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14dffff640 1 -- 172.21.15.172:0/2602354663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14e8161ce0 msgr2=0x7f14e80c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14dffff640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14e8161ce0 0x7f14e80c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14dffff640 1 -- 172.21.15.172:0/2602354663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e80c9280 con 0x7f14e80d2d40 2024-04-14T08:40:07.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14e77fe640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14e8161ce0 0x7f14e80c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:07.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14dffff640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e80d2d40 0x7f14e80cdae0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f14d000c3c0 tx=0x7f14d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14df7fe640 1 -- 172.21.15.172:0/2602354663 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d00052d0 con 0x7f14e80d2d40 2024-04-14T08:40:07.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14df7fe640 1 -- 172.21.15.172:0/2602354663 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d0005490 con 0x7f14e80d2d40 2024-04-14T08:40:07.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14ed4c0640 1 -- 172.21.15.172:0/2602354663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e81f4af0 con 0x7f14e80d2d40 2024-04-14T08:40:07.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14df7fe640 1 -- 172.21.15.172:0/2602354663 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0007260 con 0x7f14e80d2d40 2024-04-14T08:40:07.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.392+0000 7f14ed4c0640 1 -- 172.21.15.172:0/2602354663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e80c8e90 con 0x7f14e80d2d40 2024-04-14T08:40:07.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.394+0000 7f14df7fe640 1 -- 172.21.15.172:0/2602354663 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14d0007400 con 0x7f14e80d2d40 2024-04-14T08:40:07.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.395+0000 7f14df7fe640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f14c806c920 0x7f14c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.395+0000 7f14df7fe640 1 -- 172.21.15.172:0/2602354663 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f14d008b1c0 con 0x7f14e80d2d40 2024-04-14T08:40:07.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.395+0000 7f14e6ffd640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f14c806c920 0x7f14c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.395+0000 7f14e6ffd640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f14c806c920 0x7f14c806ede0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f14d8006fd0 tx=0x7f14d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.395+0000 7f14f0380740 1 Processor -- start 2024-04-14T08:40:07.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.396+0000 7f14f0380740 1 -- start start 2024-04-14T08:40:07.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.396+0000 7f14f0380740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac08097d90 0x55ac08098170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.396+0000 7f14f0380740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac0809dbb0 0x55ac0809bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.396+0000 7f14f0380740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080e54a0 0x55ac080e5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.396+0000 7f14f0380740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ac0805f5a0 con 0x55ac080e54a0 2024-04-14T08:40:07.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.396+0000 7f14f0380740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ac08001680 con 0x55ac0809dbb0 2024-04-14T08:40:07.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.396+0000 7f14f0380740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ac07f9f860 con 0x55ac08097d90 2024-04-14T08:40:07.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080e54a0 0x55ac080e5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14e6ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac08097d90 0x55ac08098170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14dffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac0809dbb0 0x55ac0809bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080e54a0 0x55ac080e5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38986/0 (socket says 172.21.15.172:38986) 2024-04-14T08:40:07.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14e77fe640 1 -- 172.21.15.172:0/1573503444 learned_addr learned my addr 172.21.15.172:0/1573503444 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:07.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14e77fe640 1 -- 172.21.15.172:0/1573503444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac08097d90 msgr2=0x55ac08098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14e77fe640 1 --2- 172.21.15.172:0/1573503444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac08097d90 0x55ac08098170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14e77fe640 1 -- 172.21.15.172:0/1573503444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac0809dbb0 msgr2=0x55ac0809bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14e77fe640 1 --2- 172.21.15.172:0/1573503444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac0809dbb0 0x55ac0809bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14e77fe640 1 -- 172.21.15.172:0/1573503444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac07fce1a0 con 0x55ac080e54a0 2024-04-14T08:40:07.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14e77fe640 1 --2- 172.21.15.172:0/1573503444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080e54a0 0x55ac080e5920 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009f40 tx=0x7f14e002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461d8bf5e32a63fb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14dcff9640 1 -- 172.21.15.172:0/1573503444 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e002cb20 con 0x55ac080e54a0 2024-04-14T08:40:07.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14dcff9640 1 -- 172.21.15.172:0/1573503444 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e002ccc0 con 0x55ac080e54a0 2024-04-14T08:40:07.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.397+0000 7f14dcff9640 1 -- 172.21.15.172:0/1573503444 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e003e630 con 0x55ac080e54a0 2024-04-14T08:40:07.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.398+0000 7f14f0380740 1 -- 172.21.15.172:0/1573503444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080e54a0 msgr2=0x55ac080e5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.398+0000 7f14f0380740 1 --2- 172.21.15.172:0/1573503444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080e54a0 0x55ac080e5920 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009f40 tx=0x7f14e002e000 comp rx=0 tx=0).stop 2024-04-14T08:40:07.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.398+0000 7f14f0380740 1 -- 172.21.15.172:0/1573503444 shutdown_connections 2024-04-14T08:40:07.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.398+0000 7f14f0380740 1 --2- 172.21.15.172:0/1573503444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080e54a0 0x55ac080e5920 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.398+0000 7f14f0380740 1 --2- 172.21.15.172:0/1573503444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac0809dbb0 0x55ac0809bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.398+0000 7f14f0380740 1 --2- 172.21.15.172:0/1573503444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac08097d90 0x55ac08098170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.398+0000 7f14f0380740 1 -- 172.21.15.172:0/1573503444 >> 172.21.15.172:0/1573503444 conn(0x55ac08099e10 msgr2=0x55ac0809b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:07.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.398+0000 7f14f0380740 1 -- 172.21.15.172:0/1573503444 shutdown_connections 2024-04-14T08:40:07.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.398+0000 7f14f0380740 1 -- 172.21.15.172:0/1573503444 wait complete. 2024-04-14T08:40:07.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.398+0000 7f14f0380740 1 -- start start 2024-04-14T08:40:07.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14f0380740 1 Processor -- start 2024-04-14T08:40:07.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14f0380740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac0809dbb0 0x55ac080ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14f0380740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac080e54a0 0x55ac080bcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14f0380740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac080ca910 0x55ac080dd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14f0380740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ac07f9d1d0 con 0x55ac0809dbb0 2024-04-14T08:40:07.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14f0380740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ac07ffd630 con 0x55ac080e54a0 2024-04-14T08:40:07.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14f0380740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ac080c4500 con 0x55ac080ca910 2024-04-14T08:40:07.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14f0380740 10 client.0 _ll_register_callbacks cb 0x55ac07fc0010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:07.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14e6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac0809dbb0 0x55ac080ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14e6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac0809dbb0 0x55ac080ca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38994/0 (socket says 172.21.15.172:38994) 2024-04-14T08:40:07.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14e6ffd640 1 -- 172.21.15.172:0/2625430189 learned_addr learned my addr 172.21.15.172:0/2625430189 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:07.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14e77fe640 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac080ca910 0x55ac080dd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.399+0000 7f14dffff640 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac080e54a0 0x55ac080bcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14e6ffd640 1 -- 172.21.15.172:0/2625430189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac080ca910 msgr2=0x55ac080dd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14e6ffd640 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac080ca910 0x55ac080dd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14e6ffd640 1 -- 172.21.15.172:0/2625430189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac080e54a0 msgr2=0x55ac080bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14e6ffd640 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac080e54a0 0x55ac080bcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14e6ffd640 1 -- 172.21.15.172:0/2625430189 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac07de17c0 con 0x55ac0809dbb0 2024-04-14T08:40:07.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14e6ffd640 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac0809dbb0 0x55ac080ca3d0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f14d8014770 tx=0x7f14d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c031af90fe816f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:07.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14c5ffb640 1 -- 172.21.15.172:0/2625430189 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8025070 con 0x55ac0809dbb0 2024-04-14T08:40:07.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14c5ffb640 1 -- 172.21.15.172:0/2625430189 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f14d80158e0 con 0x55ac0809dbb0 2024-04-14T08:40:07.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14f0380740 1 -- 172.21.15.172:0/2625430189 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e002ccc0 con 0x55ac0809dbb0 2024-04-14T08:40:07.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14c5ffb640 1 -- 172.21.15.172:0/2625430189 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f14d8030050 con 0x55ac0809dbb0 2024-04-14T08:40:07.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14f0380740 10 client.7953 fetch_fsmap learned FSMap version 61 2024-04-14T08:40:07.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14f0380740 1 -- 172.21.15.172:0/2625430189 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55ac07e41f50 con 0x55ac0809dbb0 2024-04-14T08:40:07.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14c5ffb640 1 -- 172.21.15.172:0/2625430189 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e002ccc0 con 0x55ac0809dbb0 2024-04-14T08:40:07.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14c5ffb640 1 -- 172.21.15.172:0/2625430189 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f14d8033020 con 0x55ac0809dbb0 2024-04-14T08:40:07.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14f0380740 10 client.7953 fetch_fsmap finished waiting for FSMap version 61 2024-04-14T08:40:07.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14f0380740 10 client.7953 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:40:07.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14f0380740 20 client.7953 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:07.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.400+0000 7f14f0380740 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55ac080fcc40 0x55ac08100f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.401+0000 7f14f0380740 4 client.7953 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:40:07.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.401+0000 7f14f0380740 1 -- 172.21.15.172:0/2625430189 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac08014eb0 con 0x55ac080fcc40 2024-04-14T08:40:07.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.401+0000 7f14dffff640 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55ac080fcc40 0x55ac08100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.401+0000 7f14c5ffb640 10 client.7953 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:40:07.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.401+0000 7f14dffff640 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55ac080fcc40 0x55ac08100f10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27ecf30fcb1c20a server_cookie=ee208a927c54cc3e in_seq=0 out_seq=0 2024-04-14T08:40:07.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.401+0000 7f14c5ffb640 1 -- 172.21.15.172:0/2625430189 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac08014eb0 con 0x55ac080fcc40 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.401+0000 7f14c5ffb640 10 client.7953 handle_command_reply: tid=0 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.401+0000 7f14f0380740 1 client.7953 shutdown 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.402+0000 7f14f0380740 1 -- 172.21.15.172:0/2625430189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac0809dbb0 msgr2=0x55ac080ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.402+0000 7f14f0380740 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac0809dbb0 0x55ac080ca3d0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f14d8014770 tx=0x7f14d80046c0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.402+0000 7f14f0380740 1 -- 172.21.15.172:0/2625430189 shutdown_connections 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.402+0000 7f14f0380740 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55ac080fcc40 0x55ac08100f10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.402+0000 7f14f0380740 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac080ca910 0x55ac080dd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.402+0000 7f14f0380740 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac080e54a0 0x55ac080bcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.402+0000 7f14c5ffb640 0 client.7953 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.402+0000 7f14f0380740 1 --2- 172.21.15.172:0/2625430189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac0809dbb0 0x55ac080ca3d0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.402+0000 7f14f0380740 1 -- 172.21.15.172:0/2625430189 >> 172.21.15.172:0/2625430189 conn(0x55ac08097d90 msgr2=0x55ac080e3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.402+0000 7f14f0380740 1 -- 172.21.15.172:0/2625430189 shutdown_connections 2024-04-14T08:40:07.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.402+0000 7f14f0380740 1 -- 172.21.15.172:0/2625430189 wait complete. 2024-04-14T08:40:07.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.402+0000 7f14f0380740 20 client.7953 trim_cache size 0 max 16384 2024-04-14T08:40:07.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.415+0000 7f14f0380740 1 Processor -- start 2024-04-14T08:40:07.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.415+0000 7f14f0380740 1 -- start start 2024-04-14T08:40:07.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14f0380740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac0809dbb0 0x55ac080c4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14f0380740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 0x55ac080de040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14f0380740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac080e54a0 0x55ac080de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14f0380740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f14e82004c0 con 0x55ac080ca910 2024-04-14T08:40:07.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14f0380740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f14e80b7bd0 con 0x55ac080e54a0 2024-04-14T08:40:07.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14f0380740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f14e80b8cb0 con 0x55ac0809dbb0 2024-04-14T08:40:07.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14dffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 0x55ac080de040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14dffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 0x55ac080de040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38996/0 (socket says 172.21.15.172:38996) 2024-04-14T08:40:07.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14dffff640 1 -- 172.21.15.172:0/458998005 learned_addr learned my addr 172.21.15.172:0/458998005 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:07.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14e77fe640 1 --2- 172.21.15.172:0/458998005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac080e54a0 0x55ac080de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14dffff640 1 -- 172.21.15.172:0/458998005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac0809dbb0 msgr2=0x55ac080c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14e6ffd640 1 --2- 172.21.15.172:0/458998005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac0809dbb0 0x55ac080c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14dffff640 1 --2- 172.21.15.172:0/458998005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac0809dbb0 0x55ac080c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14dffff640 1 -- 172.21.15.172:0/458998005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac080e54a0 msgr2=0x55ac080de580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14dffff640 1 --2- 172.21.15.172:0/458998005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac080e54a0 0x55ac080de580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14dffff640 1 -- 172.21.15.172:0/458998005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac080987b0 con 0x55ac080ca910 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14e77fe640 1 --2- 172.21.15.172:0/458998005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac080e54a0 0x55ac080de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14e6ffd640 1 --2- 172.21.15.172:0/458998005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac0809dbb0 0x55ac080c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14dffff640 1 --2- 172.21.15.172:0/458998005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 0x55ac080de040 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f14d0089530 tx=0x7f14d0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e6054d4937ff45 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14c7fff640 1 -- 172.21.15.172:0/458998005 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0056dd0 con 0x55ac080ca910 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14c7fff640 1 -- 172.21.15.172:0/458998005 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d0057030 con 0x55ac080ca910 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.416+0000 7f14c7fff640 1 -- 172.21.15.172:0/458998005 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0071120 con 0x55ac080ca910 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.417+0000 7f14f0380740 1 -- 172.21.15.172:0/458998005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 msgr2=0x55ac080de040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.417+0000 7f14f0380740 1 --2- 172.21.15.172:0/458998005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 0x55ac080de040 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f14d0089530 tx=0x7f14d0000ed0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.417+0000 7f14f0380740 1 -- 172.21.15.172:0/458998005 shutdown_connections 2024-04-14T08:40:07.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.417+0000 7f14f0380740 1 --2- 172.21.15.172:0/458998005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac080e54a0 0x55ac080de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.417+0000 7f14f0380740 1 --2- 172.21.15.172:0/458998005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 0x55ac080de040 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.417+0000 7f14f0380740 1 --2- 172.21.15.172:0/458998005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac0809dbb0 0x55ac080c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.417+0000 7f14f0380740 1 -- 172.21.15.172:0/458998005 >> 172.21.15.172:0/458998005 conn(0x55ac08097d90 msgr2=0x55ac080ca1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.417+0000 7f14f0380740 1 -- 172.21.15.172:0/458998005 shutdown_connections 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.417+0000 7f14f0380740 1 -- 172.21.15.172:0/458998005 wait complete. 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.417+0000 7f14f0380740 1 -- start start 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14f0380740 1 Processor -- start 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14f0380740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac0809dbb0 0x55ac080e6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14f0380740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 0x55ac080e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14f0380740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac080e54a0 0x55ac0809b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14f0380740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ac080fcac0 con 0x55ac080ca910 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14f0380740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ac080fc620 con 0x55ac0809dbb0 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14f0380740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ac080fd050 con 0x55ac080e54a0 2024-04-14T08:40:07.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14f0380740 10 client.0 _ll_register_callbacks cb 0x7f14d8006df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14dffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 0x55ac080e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14dffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 0x55ac080e68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38998/0 (socket says 172.21.15.172:38998) 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14dffff640 1 -- 172.21.15.172:0/4029598378 learned_addr learned my addr 172.21.15.172:0/4029598378 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14e6ffd640 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac0809dbb0 0x55ac080e6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.418+0000 7f14e77fe640 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac080e54a0 0x55ac0809b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14dffff640 1 -- 172.21.15.172:0/4029598378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac080e54a0 msgr2=0x55ac0809b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14dffff640 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac080e54a0 0x55ac0809b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14dffff640 1 -- 172.21.15.172:0/4029598378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac0809dbb0 msgr2=0x55ac080e6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14dffff640 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac0809dbb0 0x55ac080e6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14dffff640 1 -- 172.21.15.172:0/4029598378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac07cd50b0 con 0x55ac080ca910 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14dffff640 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 0x55ac080e68d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f14d00613e0 tx=0x7f14d000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6065311ba58af0c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:07.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14c5ffb640 1 -- 172.21.15.172:0/4029598378 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0076060 con 0x55ac080ca910 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14c5ffb640 1 -- 172.21.15.172:0/4029598378 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f14d0058180 con 0x55ac080ca910 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14f0380740 1 -- 172.21.15.172:0/4029598378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d0057030 con 0x55ac080ca910 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14c5ffb640 1 -- 172.21.15.172:0/4029598378 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f14d0058380 con 0x55ac080ca910 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14f0380740 10 client.7959 fetch_fsmap learned FSMap version 61 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14f0380740 1 -- 172.21.15.172:0/4029598378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55ac080f19e0 con 0x55ac080ca910 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14c5ffb640 1 -- 172.21.15.172:0/4029598378 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d0057030 con 0x55ac080ca910 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14c5ffb640 1 -- 172.21.15.172:0/4029598378 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f14d005f2e0 con 0x55ac080ca910 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14f0380740 10 client.7959 fetch_fsmap finished waiting for FSMap version 61 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14f0380740 10 client.7959 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14f0380740 20 client.7959 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14f0380740 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55ac0809bae0 0x55ac080e1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14f0380740 4 client.7959 mds_command: new command op to 7473 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.419+0000 7f14f0380740 1 -- 172.21.15.172:0/4029598378 --> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55ac07fa24a0 con 0x55ac0809bae0 2024-04-14T08:40:07.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14e6ffd640 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55ac0809bae0 0x55ac080e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14c5ffb640 10 client.7959 ms_handle_connect on v2:172.21.15.172:6835/1386685680 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14e6ffd640 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55ac0809bae0 0x55ac080e1590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e9e75629004319 server_cookie=3d49b6df6fe3fd45 in_seq=0 out_seq=0 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14c5ffb640 1 -- 172.21.15.172:0/4029598378 <== mds.0 v2:172.21.15.172:6835/1386685680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+143 (crc 0 0 0) 0x55ac07fa24a0 con 0x55ac0809bae0 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14c5ffb640 10 client.7959 handle_command_reply: tid=0 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14f0380740 1 client.7959 shutdown 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14f0380740 1 -- 172.21.15.172:0/4029598378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 msgr2=0x55ac080e68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14f0380740 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 0x55ac080e68d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f14d00613e0 tx=0x7f14d000a000 comp rx=0 tx=0).stop 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14f0380740 1 -- 172.21.15.172:0/4029598378 shutdown_connections 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14f0380740 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.172:6835/1386685680,v1:172.21.15.172:6837/1386685680] conn(0x55ac0809bae0 0x55ac080e1590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14f0380740 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ac080e54a0 0x55ac0809b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14f0380740 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac080ca910 0x55ac080e68d0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14f0380740 1 --2- 172.21.15.172:0/4029598378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac0809dbb0 0x55ac080e6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14c5ffb640 0 client.7959 ms_handle_reset on v2:172.21.15.172:6835/1386685680 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14f0380740 1 -- 172.21.15.172:0/4029598378 >> 172.21.15.172:0/4029598378 conn(0x55ac08097d90 msgr2=0x55ac080c3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:07.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14f0380740 1 -- 172.21.15.172:0/4029598378 shutdown_connections 2024-04-14T08:40:07.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14f0380740 1 -- 172.21.15.172:0/4029598378 wait complete. 2024-04-14T08:40:07.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.420+0000 7f14f0380740 20 client.7959 trim_cache size 0 max 16384 2024-04-14T08:40:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:07.429 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-04-14T08:40:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:40:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: "damage_type": "backtrace", 2024-04-14T08:40:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 884217824, 2024-04-14T08:40:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 1099511627777, 2024-04-14T08:40:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/testdir/hardlink" 2024-04-14T08:40:07.429 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:40:07.430 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-04-14T08:40:07.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.421+0000 7f14ed4c0640 1 -- 172.21.15.172:0/2602354663 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f14c806c920 msgr2=0x7f14c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.421+0000 7f14ed4c0640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f14c806c920 0x7f14c806ede0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f14d8006fd0 tx=0x7f14d8008040 comp rx=0 tx=0).stop 2024-04-14T08:40:07.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.421+0000 7f14ed4c0640 1 -- 172.21.15.172:0/2602354663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e80d2d40 msgr2=0x7f14e80cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.421+0000 7f14ed4c0640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e80d2d40 0x7f14e80cdae0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f14d000c3c0 tx=0x7f14d000c880 comp rx=0 tx=0).stop 2024-04-14T08:40:07.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.421+0000 7f14ed4c0640 1 -- 172.21.15.172:0/2602354663 shutdown_connections 2024-04-14T08:40:07.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.421+0000 7f14ed4c0640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f14c806c920 0x7f14c806ede0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.421+0000 7f14ed4c0640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14e8161ce0 0x7f14e80c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.421+0000 7f14ed4c0640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e80d2d40 0x7f14e80cdae0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.421+0000 7f14ed4c0640 1 --2- 172.21.15.172:0/2602354663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14e80d1320 0x7f14e81f55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.421+0000 7f14ed4c0640 1 -- 172.21.15.172:0/2602354663 >> 172.21.15.172:0/2602354663 conn(0x7f14e809d560 msgr2=0x7f14e806c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:07.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.421+0000 7f14ed4c0640 1 -- 172.21.15.172:0/2602354663 shutdown_connections 2024-04-14T08:40:07.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.421+0000 7f14ed4c0640 1 -- 172.21.15.172:0/2602354663 wait complete. 2024-04-14T08:40:07.432 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:40:07.432 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:40:07.448 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:40:07.448 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:40:07.504 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:40:07.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.683+0000 7f9e1981c640 1 Processor -- start 2024-04-14T08:40:07.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.683+0000 7f9e1981c640 1 -- start start 2024-04-14T08:40:07.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.684+0000 7f9e1981c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1415c710 0x7f9e140d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.684+0000 7f9e1981c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9e140cde10 0x7f9e140ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.684+0000 7f9e1981c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e140aef60 con 0x7f9e140ced80 2024-04-14T08:40:07.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.684+0000 7f9e1981c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f9e140aed10 con 0x7f9e140cde10 2024-04-14T08:40:07.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.684+0000 7f9e1981c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9e140af890 con 0x7f9e1415c710 2024-04-14T08:40:07.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.684+0000 7f9e137fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9e140ced80 0x7f9e140ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47694/0 (socket says 172.21.15.172:47694) 2024-04-14T08:40:07.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.684+0000 7f9e137fe640 1 -- 172.21.15.172:0/1551158744 learned_addr learned my addr 172.21.15.172:0/1551158744 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:07.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.684+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3279641039 0 0) 0x7f9e140aef60 con 0x7f9e140ced80 2024-04-14T08:40:07.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.684+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9dfc003520 con 0x7f9e140ced80 2024-04-14T08:40:07.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.684+0000 7f9e127fc640 1 --2- 172.21.15.172:0/1551158744 >> v2:172.21.15.177:3300/0 conn(0x7f9e140cde10 0x7f9e140ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.684+0000 7f9e12ffd640 1 --2- 172.21.15.172:0/1551158744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1415c710 0x7f9e140d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2175949827 0 0) 0x7f9dfc003520 con 0x7f9e140ced80 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1415c710 msgr2=0x7f9e140d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 --2- 172.21.15.172:0/1551158744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1415c710 0x7f9e140d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 >> v2:172.21.15.177:3300/0 conn(0x7f9e140cde10 msgr2=0x7f9e140ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 --2- 172.21.15.172:0/1551158744 >> v2:172.21.15.177:3300/0 conn(0x7f9e140cde10 0x7f9e140ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e140af240 con 0x7f9e140ced80 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f9e04001670 con 0x7f9e140ced80 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 >> v1:172.21.15.172:6789/0 conn(0x7f9e140ced80 legacy=0x7f9e140ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 --2- 172.21.15.172:0/1551158744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dfc0039b0 0x7f9dfc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 --2- 172.21.15.172:0/1551158744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1415c710 0x7f9dfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 --2- 172.21.15.172:0/1551158744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dfc004d40 0x7f9dfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9e140aed10 con 0x7f9dfc0039b0 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9e140af890 con 0x7f9dfc004d40 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9dfc00b9c0 con 0x7f9e1415c710 2024-04-14T08:40:07.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e12ffd640 1 --2- 172.21.15.172:0/1551158744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dfc0039b0 0x7f9dfc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e127fc640 1 --2- 172.21.15.172:0/1551158744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1415c710 0x7f9dfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e137fe640 1 --2- 172.21.15.172:0/1551158744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dfc004d40 0x7f9dfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e12ffd640 1 -- 172.21.15.172:0/1551158744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1415c710 msgr2=0x7f9dfc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e12ffd640 1 --2- 172.21.15.172:0/1551158744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1415c710 0x7f9dfc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e12ffd640 1 -- 172.21.15.172:0/1551158744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dfc004d40 msgr2=0x7f9dfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e12ffd640 1 --2- 172.21.15.172:0/1551158744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dfc004d40 0x7f9dfc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.685+0000 7f9e12ffd640 1 -- 172.21.15.172:0/1551158744 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dfc003520 con 0x7f9dfc0039b0 2024-04-14T08:40:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.686+0000 7f9e12ffd640 1 --2- 172.21.15.172:0/1551158744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dfc0039b0 0x7f9dfc003dd0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009fc0 tx=0x7f9dfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb564f497188e8cf server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.686+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c039070 con 0x7f9dfc0039b0 2024-04-14T08:40:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.686+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e0c02cc20 con 0x7f9dfc0039b0 2024-04-14T08:40:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.686+0000 7f9e11ffb640 1 -- 172.21.15.172:0/1551158744 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c009c10 con 0x7f9dfc0039b0 2024-04-14T08:40:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.687+0000 7f9e1981c640 1 -- 172.21.15.172:0/1551158744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dfc0039b0 msgr2=0x7f9dfc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.687+0000 7f9e1981c640 1 --2- 172.21.15.172:0/1551158744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dfc0039b0 0x7f9dfc003dd0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009fc0 tx=0x7f9dfc005830 comp rx=0 tx=0).stop 2024-04-14T08:40:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.687+0000 7f9e137fe640 1 -- 172.21.15.172:0/1551158744 reap_dead start 2024-04-14T08:40:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.687+0000 7f9e1981c640 1 -- 172.21.15.172:0/1551158744 shutdown_connections 2024-04-14T08:40:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.687+0000 7f9e1981c640 1 -- 172.21.15.172:0/1551158744 >> 172.21.15.172:0/1551158744 conn(0x7f9e1408c000 msgr2=0x7f9e140d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.687+0000 7f9e1981c640 1 -- 172.21.15.172:0/1551158744 shutdown_connections 2024-04-14T08:40:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.687+0000 7f9e1981c640 1 -- 172.21.15.172:0/1551158744 wait complete. 2024-04-14T08:40:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.688+0000 7f9e1981c640 1 Processor -- start 2024-04-14T08:40:07.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.688+0000 7f9e1981c640 1 -- start start 2024-04-14T08:40:07.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e1981c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dfc0039b0 0x7f9e140c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e1981c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e140b6650 0x7f9e140c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e1981c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e141581a0 0x7f9e14158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e1981c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9e140b0d30 con 0x7f9e141581a0 2024-04-14T08:40:07.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e1981c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1415caf0 con 0x7f9e140b6650 2024-04-14T08:40:07.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e1981c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9e140b0eb0 con 0x7f9dfc0039b0 2024-04-14T08:40:07.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e137fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e141581a0 0x7f9e14158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e137fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e141581a0 0x7f9e14158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39022/0 (socket says 172.21.15.172:39022) 2024-04-14T08:40:07.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e137fe640 1 -- 172.21.15.172:0/213115339 learned_addr learned my addr 172.21.15.172:0/213115339 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:07.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e127fc640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e140b6650 0x7f9e140c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e12ffd640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dfc0039b0 0x7f9e140c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e137fe640 1 -- 172.21.15.172:0/213115339 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dfc0039b0 msgr2=0x7f9e140c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e137fe640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dfc0039b0 0x7f9e140c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e137fe640 1 -- 172.21.15.172:0/213115339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e140b6650 msgr2=0x7f9e140c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e137fe640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e140b6650 0x7f9e140c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e137fe640 1 -- 172.21.15.172:0/213115339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e14158b60 con 0x7f9e141581a0 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e137fe640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e141581a0 0x7f9e14158620 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f9e14073100 tx=0x7f9e04007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9dfbfff640 1 -- 172.21.15.172:0/213115339 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0400f910 con 0x7f9e141581a0 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9dfbfff640 1 -- 172.21.15.172:0/213115339 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e04018070 con 0x7f9e141581a0 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9e1981c640 1 -- 172.21.15.172:0/213115339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e140c69b0 con 0x7f9e141581a0 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.689+0000 7f9dfbfff640 1 -- 172.21.15.172:0/213115339 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0400fd30 con 0x7f9e141581a0 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.690+0000 7f9e1981c640 1 -- 172.21.15.172:0/213115339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e140ca880 con 0x7f9e141581a0 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.690+0000 7f9e1981c640 1 -- 172.21.15.172:0/213115339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e140d0400 con 0x7f9e141581a0 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.691+0000 7f9dfbfff640 1 -- 172.21.15.172:0/213115339 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e0401d460 con 0x7f9e141581a0 2024-04-14T08:40:07.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.691+0000 7f9dfbfff640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9e0006c920 0x7f9e0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:07.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.691+0000 7f9dfbfff640 1 -- 172.21.15.172:0/213115339 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f9e0408bc80 con 0x7f9e141581a0 2024-04-14T08:40:07.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.691+0000 7f9e12ffd640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9e0006c920 0x7f9e0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:07.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.692+0000 7f9e12ffd640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9e0006c920 0x7f9e0006ede0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c036000 tx=0x7f9e0c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:07.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.696+0000 7f9dfbfff640 1 -- 172.21.15.172:0/213115339 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9e140d0400 con 0x7f9e141581a0 2024-04-14T08:40:07.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.879+0000 7f9e1981c640 1 -- 172.21.15.172:0/213115339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e14075490 con 0x7f9e141581a0 2024-04-14T08:40:07.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.880+0000 7f9dfbfff640 1 -- 172.21.15.172:0/213115339 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+17269 (secure 0 0 0) 0x7f9e04059dd0 con 0x7f9e141581a0 2024-04-14T08:40:07.881 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:07.881 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":89,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:39:35.479728+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:38:42.790996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-14T08:38:43.517208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/1124731311":"2024-04-15T08:38:58.388334+0000","172.21.15.177:6833/1124731311":"2024-04-15T08:38:58.388334+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:40:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.882+0000 7f9e1981c640 1 -- 172.21.15.172:0/213115339 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9e0006c920 msgr2=0x7f9e0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.882+0000 7f9e1981c640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9e0006c920 0x7f9e0006ede0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c036000 tx=0x7f9e0c02e000 comp rx=0 tx=0).stop 2024-04-14T08:40:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.882+0000 7f9e1981c640 1 -- 172.21.15.172:0/213115339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e141581a0 msgr2=0x7f9e14158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.882+0000 7f9e1981c640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e141581a0 0x7f9e14158620 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f9e14073100 tx=0x7f9e04007720 comp rx=0 tx=0).stop 2024-04-14T08:40:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.882+0000 7f9e1981c640 1 -- 172.21.15.172:0/213115339 shutdown_connections 2024-04-14T08:40:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.882+0000 7f9e1981c640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9e0006c920 0x7f9e0006ede0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.882+0000 7f9e1981c640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e141581a0 0x7f9e14158620 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.882+0000 7f9e1981c640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e140b6650 0x7f9e140c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.882+0000 7f9e1981c640 1 --2- 172.21.15.172:0/213115339 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dfc0039b0 0x7f9e140c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.882+0000 7f9e1981c640 1 -- 172.21.15.172:0/213115339 >> 172.21.15.172:0/213115339 conn(0x7f9e1408c000 msgr2=0x7f9e1406f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.882+0000 7f9e1981c640 1 -- 172.21.15.172:0/213115339 shutdown_connections 2024-04-14T08:40:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.882+0000 7f9e1981c640 1 -- 172.21.15.172:0/213115339 wait complete. 2024-04-14T08:40:07.894 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:38:42.790996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:38:43.517208+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:40:07.894 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:40:07.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.998+0000 7f32bf5e0640 1 Processor -- start 2024-04-14T08:40:07.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:07.999+0000 7f32bf5e0640 1 -- start start 2024-04-14T08:40:08.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bf5e0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b815e430 0x7f32b8162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bf5e0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f32b8162540 0x7f32b81629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bf5e0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b80cefb0 con 0x7f32b80b3a30 2024-04-14T08:40:08.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bf5e0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f32b80cf290 con 0x7f32b8162540 2024-04-14T08:40:08.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bf5e0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f32b80cf570 con 0x7f32b815e430 2024-04-14T08:40:08.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bdb7d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f32b80b3a30 0x7f32b80b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47710/0 (socket says 172.21.15.172:47710) 2024-04-14T08:40:08.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bcb7b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f32b8162540 0x7f32b81629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bdb7d640 1 -- 172.21.15.172:0/3501985421 learned_addr learned my addr 172.21.15.172:0/3501985421 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:08.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bd37c640 1 --2- 172.21.15.172:0/3501985421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b815e430 0x7f32b8162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bcb7b640 1 -- 172.21.15.172:0/3501985421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b815e430 msgr2=0x7f32b8162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:08.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bcb7b640 1 --2- 172.21.15.172:0/3501985421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b815e430 0x7f32b8162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bcb7b640 1 -- 172.21.15.172:0/3501985421 >> v1:172.21.15.172:6789/0 conn(0x7f32b80b3a30 legacy=0x7f32b80b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32bcb7b640 1 -- 172.21.15.172:0/3501985421 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b80cf850 con 0x7f32b8162540 2024-04-14T08:40:08.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.000+0000 7f32affff640 1 -- 172.21.15.172:0/3501985421 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3123575660 0 0) 0x7f32b80cefb0 con 0x7f32b80b3a30 2024-04-14T08:40:08.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.001+0000 7f32bcb7b640 1 --2- 172.21.15.172:0/3501985421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b8162540 0x7f32b81629a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f32b40097a0 tx=0x7f32b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f39dc298297c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:08.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.001+0000 7f32affff640 1 -- 172.21.15.172:0/3501985421 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4035020 con 0x7f32b8162540 2024-04-14T08:40:08.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.001+0000 7f32affff640 1 -- 172.21.15.172:0/3501985421 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b4031830 con 0x7f32b8162540 2024-04-14T08:40:08.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.001+0000 7f32affff640 1 -- 172.21.15.172:0/3501985421 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4031c00 con 0x7f32b8162540 2024-04-14T08:40:08.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.002+0000 7f32bf5e0640 1 -- 172.21.15.172:0/3501985421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b8162540 msgr2=0x7f32b81629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.002+0000 7f32bf5e0640 1 --2- 172.21.15.172:0/3501985421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b8162540 0x7f32b81629a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f32b40097a0 tx=0x7f32b402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.002+0000 7f32bf5e0640 1 -- 172.21.15.172:0/3501985421 shutdown_connections 2024-04-14T08:40:08.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.002+0000 7f32bf5e0640 1 --2- 172.21.15.172:0/3501985421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b8162540 0x7f32b81629a0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.002+0000 7f32bf5e0640 1 --2- 172.21.15.172:0/3501985421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b815e430 0x7f32b8162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.002+0000 7f32bf5e0640 1 -- 172.21.15.172:0/3501985421 >> 172.21.15.172:0/3501985421 conn(0x7f32b809d560 msgr2=0x7f32b80d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:08.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.002+0000 7f32bf5e0640 1 -- 172.21.15.172:0/3501985421 shutdown_connections 2024-04-14T08:40:08.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.002+0000 7f32bf5e0640 1 -- 172.21.15.172:0/3501985421 wait complete. 2024-04-14T08:40:08.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.003+0000 7f32bf5e0640 1 Processor -- start 2024-04-14T08:40:08.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.003+0000 7f32bf5e0640 1 -- start start 2024-04-14T08:40:08.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bf5e0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b80b3a30 0x7f32b80cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bf5e0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b815e430 0x7f32b814b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bf5e0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b8162540 0x7f32b80c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bf5e0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32b80d4e20 con 0x7f32b80b3a30 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bf5e0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f32b815e810 con 0x7f32b8162540 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bf5e0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f32b80d0050 con 0x7f32b815e430 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bd37c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b80b3a30 0x7f32b80cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bd37c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b80b3a30 0x7f32b80cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39032/0 (socket says 172.21.15.172:39032) 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bd37c640 1 -- 172.21.15.172:0/145549633 learned_addr learned my addr 172.21.15.172:0/145549633 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bd37c640 1 -- 172.21.15.172:0/145549633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b815e430 msgr2=0x7f32b814b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bd37c640 1 --2- 172.21.15.172:0/145549633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b815e430 0x7f32b814b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bd37c640 1 -- 172.21.15.172:0/145549633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b8162540 msgr2=0x7f32b80c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bd37c640 1 --2- 172.21.15.172:0/145549633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b8162540 0x7f32b80c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bd37c640 1 -- 172.21.15.172:0/145549633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b80b7e00 con 0x7f32b80b3a30 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.004+0000 7f32bd37c640 1 --2- 172.21.15.172:0/145549633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b80b3a30 0x7f32b80cab80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f32a800d800 tx=0x7f32a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.005+0000 7f32adffb640 1 -- 172.21.15.172:0/145549633 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a800b6c0 con 0x7f32b80b3a30 2024-04-14T08:40:08.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.005+0000 7f32adffb640 1 -- 172.21.15.172:0/145549633 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a8019070 con 0x7f32b80b3a30 2024-04-14T08:40:08.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.005+0000 7f32bf5e0640 1 -- 172.21.15.172:0/145549633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b814e410 con 0x7f32b80b3a30 2024-04-14T08:40:08.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.005+0000 7f32adffb640 1 -- 172.21.15.172:0/145549633 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a800ba80 con 0x7f32b80b3a30 2024-04-14T08:40:08.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.005+0000 7f32bf5e0640 1 -- 172.21.15.172:0/145549633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b814e9d0 con 0x7f32b80b3a30 2024-04-14T08:40:08.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.006+0000 7f32bf5e0640 1 -- 172.21.15.172:0/145549633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b8068590 con 0x7f32b80b3a30 2024-04-14T08:40:08.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.006+0000 7f32adffb640 1 -- 172.21.15.172:0/145549633 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32a800bc20 con 0x7f32b80b3a30 2024-04-14T08:40:08.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.007+0000 7f32adffb640 1 --2- 172.21.15.172:0/145549633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f329c06c920 0x7f329c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.007+0000 7f32adffb640 1 -- 172.21.15.172:0/145549633 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f32a808c280 con 0x7f32b80b3a30 2024-04-14T08:40:08.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.007+0000 7f32bcb7b640 1 --2- 172.21.15.172:0/145549633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f329c06c920 0x7f329c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.007+0000 7f32bcb7b640 1 --2- 172.21.15.172:0/145549633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f329c06c920 0x7f329c06ede0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f32b4036000 tx=0x7f32b4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:08.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.015+0000 7f32adffb640 1 -- 172.21.15.172:0/145549633 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f32a805a3e0 con 0x7f32b80b3a30 2024-04-14T08:40:08.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.193+0000 7f32bf5e0640 1 -- 172.21.15.172:0/145549633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32b8161fe0 con 0x7f32b80b3a30 2024-04-14T08:40:08.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.194+0000 7f32adffb640 1 -- 172.21.15.172:0/145549633 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f32a8059ce0 con 0x7f32b80b3a30 2024-04-14T08:40:08.194 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:40:08.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.196+0000 7f32bf5e0640 1 -- 172.21.15.172:0/145549633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f329c06c920 msgr2=0x7f329c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.196+0000 7f32bf5e0640 1 --2- 172.21.15.172:0/145549633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f329c06c920 0x7f329c06ede0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f32b4036000 tx=0x7f32b4030000 comp rx=0 tx=0).stop 2024-04-14T08:40:08.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.196+0000 7f32bf5e0640 1 -- 172.21.15.172:0/145549633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b80b3a30 msgr2=0x7f32b80cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.196+0000 7f32bf5e0640 1 --2- 172.21.15.172:0/145549633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b80b3a30 0x7f32b80cab80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f32a800d800 tx=0x7f32a800dcc0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.196+0000 7f32bf5e0640 1 -- 172.21.15.172:0/145549633 shutdown_connections 2024-04-14T08:40:08.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.196+0000 7f32bf5e0640 1 --2- 172.21.15.172:0/145549633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f329c06c920 0x7f329c06ede0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.196+0000 7f32bf5e0640 1 --2- 172.21.15.172:0/145549633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32b8162540 0x7f32b80c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.196+0000 7f32bf5e0640 1 --2- 172.21.15.172:0/145549633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32b815e430 0x7f32b814b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.196+0000 7f32bf5e0640 1 --2- 172.21.15.172:0/145549633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32b80b3a30 0x7f32b80cab80 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.196+0000 7f32bf5e0640 1 -- 172.21.15.172:0/145549633 >> 172.21.15.172:0/145549633 conn(0x7f32b809d560 msgr2=0x7f32b806c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:08.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.196+0000 7f32bf5e0640 1 -- 172.21.15.172:0/145549633 shutdown_connections 2024-04-14T08:40:08.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.196+0000 7f32bf5e0640 1 -- 172.21.15.172:0/145549633 wait complete. 2024-04-14T08:40:08.208 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:40:08.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.316+0000 7fec36236640 1 Processor -- start 2024-04-14T08:40:08.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.316+0000 7fec36236640 1 -- start start 2024-04-14T08:40:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.317+0000 7fec36236640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fec3015e430 0x7fec30162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.317+0000 7fec36236640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec30162540 0x7fec301629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.317+0000 7fec36236640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec300cefb0 con 0x7fec300b3a30 2024-04-14T08:40:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.317+0000 7fec36236640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fec300cf290 con 0x7fec3015e430 2024-04-14T08:40:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.317+0000 7fec36236640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fec300cf570 con 0x7fec30162540 2024-04-14T08:40:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.317+0000 7fec2f7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fec3015e430 0x7fec30162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.317+0000 7fec2ffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fec300b3a30 0x7fec300b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47716/0 (socket says 172.21.15.172:47716) 2024-04-14T08:40:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.317+0000 7fec2f7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fec3015e430 0x7fec30162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39040/0 (socket says 172.21.15.172:39040) 2024-04-14T08:40:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.317+0000 7fec2ffff640 1 -- 172.21.15.172:0/1724029599 learned_addr learned my addr 172.21.15.172:0/1724029599 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.317+0000 7fec27fff640 1 --2- 172.21.15.172:0/1724029599 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec30162540 0x7fec301629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.318+0000 7fec2f7fe640 1 -- 172.21.15.172:0/1724029599 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec30162540 msgr2=0x7fec301629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:08.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.318+0000 7fec2f7fe640 1 --2- 172.21.15.172:0/1724029599 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec30162540 0x7fec301629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.318+0000 7fec2f7fe640 1 -- 172.21.15.172:0/1724029599 >> v1:172.21.15.172:6789/0 conn(0x7fec300b3a30 legacy=0x7fec300b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.318+0000 7fec2f7fe640 1 -- 172.21.15.172:0/1724029599 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec300cf850 con 0x7fec3015e430 2024-04-14T08:40:08.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.318+0000 7fec2effd640 1 -- 172.21.15.172:0/1724029599 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2509814337 0 0) 0x7fec300cefb0 con 0x7fec300b3a30 2024-04-14T08:40:08.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.318+0000 7fec2f7fe640 1 --2- 172.21.15.172:0/1724029599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec3015e430 0x7fec30162000 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fec200097a0 tx=0x7fec2002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e145866131e0343f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:08.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.319+0000 7fec2effd640 1 -- 172.21.15.172:0/1724029599 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20035020 con 0x7fec3015e430 2024-04-14T08:40:08.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.319+0000 7fec2effd640 1 -- 172.21.15.172:0/1724029599 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec20031830 con 0x7fec3015e430 2024-04-14T08:40:08.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.319+0000 7fec2effd640 1 -- 172.21.15.172:0/1724029599 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20031c00 con 0x7fec3015e430 2024-04-14T08:40:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.319+0000 7fec36236640 1 -- 172.21.15.172:0/1724029599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec3015e430 msgr2=0x7fec30162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.319+0000 7fec36236640 1 --2- 172.21.15.172:0/1724029599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec3015e430 0x7fec30162000 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fec200097a0 tx=0x7fec2002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.319+0000 7fec36236640 1 -- 172.21.15.172:0/1724029599 shutdown_connections 2024-04-14T08:40:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.319+0000 7fec36236640 1 --2- 172.21.15.172:0/1724029599 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec30162540 0x7fec301629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.319+0000 7fec36236640 1 --2- 172.21.15.172:0/1724029599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec3015e430 0x7fec30162000 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.319+0000 7fec36236640 1 -- 172.21.15.172:0/1724029599 >> 172.21.15.172:0/1724029599 conn(0x7fec3009d560 msgr2=0x7fec300d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.319+0000 7fec36236640 1 -- 172.21.15.172:0/1724029599 shutdown_connections 2024-04-14T08:40:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.319+0000 7fec36236640 1 -- 172.21.15.172:0/1724029599 wait complete. 2024-04-14T08:40:08.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.321+0000 7fec36236640 1 Processor -- start 2024-04-14T08:40:08.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.321+0000 7fec36236640 1 -- start start 2024-04-14T08:40:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.322+0000 7fec36236640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec300b3a30 0x7fec301f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.322+0000 7fec36236640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec3015e430 0x7fec301608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.322+0000 7fec36236640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec30162540 0x7fec30160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.322+0000 7fec36236640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fec300d4e20 con 0x7fec30162540 2024-04-14T08:40:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.322+0000 7fec36236640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fec3015e810 con 0x7fec300b3a30 2024-04-14T08:40:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.322+0000 7fec36236640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fec300d0050 con 0x7fec3015e430 2024-04-14T08:40:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.322+0000 7fec2ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec30162540 0x7fec30160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.322+0000 7fec2ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec30162540 0x7fec30160de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39046/0 (socket says 172.21.15.172:39046) 2024-04-14T08:40:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.322+0000 7fec2ffff640 1 -- 172.21.15.172:0/2871038928 learned_addr learned my addr 172.21.15.172:0/2871038928 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.322+0000 7fec2f7fe640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec300b3a30 0x7fec301f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.322+0000 7fec27fff640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec3015e430 0x7fec301608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.323+0000 7fec2ffff640 1 -- 172.21.15.172:0/2871038928 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec3015e430 msgr2=0x7fec301608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.323+0000 7fec2ffff640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec3015e430 0x7fec301608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.323+0000 7fec2ffff640 1 -- 172.21.15.172:0/2871038928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec300b3a30 msgr2=0x7fec301f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.323+0000 7fec2ffff640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec300b3a30 0x7fec301f4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.323+0000 7fec2ffff640 1 -- 172.21.15.172:0/2871038928 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec300cddd0 con 0x7fec30162540 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.323+0000 7fec2ffff640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec30162540 0x7fec30160de0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fec2800c780 tx=0x7fec2800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.323+0000 7fec2cff9640 1 -- 172.21.15.172:0/2871038928 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28018070 con 0x7fec30162540 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.323+0000 7fec2cff9640 1 -- 172.21.15.172:0/2871038928 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2800a640 con 0x7fec30162540 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.323+0000 7fec36236640 1 -- 172.21.15.172:0/2871038928 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec301f53a0 con 0x7fec30162540 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.323+0000 7fec2cff9640 1 -- 172.21.15.172:0/2871038928 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2800aa30 con 0x7fec30162540 2024-04-14T08:40:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.323+0000 7fec36236640 1 -- 172.21.15.172:0/2871038928 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3015f130 con 0x7fec30162540 2024-04-14T08:40:08.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.324+0000 7fec36236640 1 -- 172.21.15.172:0/2871038928 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec30068590 con 0x7fec30162540 2024-04-14T08:40:08.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.325+0000 7fec2cff9640 1 -- 172.21.15.172:0/2871038928 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec2801b070 con 0x7fec30162540 2024-04-14T08:40:08.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.325+0000 7fec2cff9640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec1006c8f0 0x7fec1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.325+0000 7fec2f7fe640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec1006c8f0 0x7fec1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.325+0000 7fec2cff9640 1 -- 172.21.15.172:0/2871038928 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7fec2808b2a0 con 0x7fec30162540 2024-04-14T08:40:08.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.326+0000 7fec2f7fe640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec1006c8f0 0x7fec1006edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fec20036000 tx=0x7fec20008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:08.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.329+0000 7fec2cff9640 1 -- 172.21.15.172:0/2871038928 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fec301f53a0 con 0x7fec30162540 2024-04-14T08:40:08.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.509+0000 7fec36236640 1 -- 172.21.15.172:0/2871038928 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fec300d18a0 con 0x7fec30162540 2024-04-14T08:40:08.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.509+0000 7fec2cff9640 1 -- 172.21.15.172:0/2871038928 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fec300cddd0 con 0x7fec30162540 2024-04-14T08:40:08.510 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:40:08.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.511+0000 7fec36236640 1 -- 172.21.15.172:0/2871038928 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec1006c8f0 msgr2=0x7fec1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.511+0000 7fec36236640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec1006c8f0 0x7fec1006edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fec20036000 tx=0x7fec20008030 comp rx=0 tx=0).stop 2024-04-14T08:40:08.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.511+0000 7fec36236640 1 -- 172.21.15.172:0/2871038928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec30162540 msgr2=0x7fec30160de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.511+0000 7fec36236640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec30162540 0x7fec30160de0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fec2800c780 tx=0x7fec2800cc40 comp rx=0 tx=0).stop 2024-04-14T08:40:08.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.511+0000 7fec36236640 1 -- 172.21.15.172:0/2871038928 shutdown_connections 2024-04-14T08:40:08.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.511+0000 7fec36236640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fec1006c8f0 0x7fec1006edb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.511+0000 7fec36236640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec30162540 0x7fec30160de0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.511+0000 7fec36236640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec3015e430 0x7fec301608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.511+0000 7fec36236640 1 --2- 172.21.15.172:0/2871038928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec300b3a30 0x7fec301f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.511+0000 7fec36236640 1 -- 172.21.15.172:0/2871038928 >> 172.21.15.172:0/2871038928 conn(0x7fec3009d560 msgr2=0x7fec300d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:08.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.511+0000 7fec36236640 1 -- 172.21.15.172:0/2871038928 shutdown_connections 2024-04-14T08:40:08.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.511+0000 7fec36236640 1 -- 172.21.15.172:0/2871038928 wait complete. 2024-04-14T08:40:08.523 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:40:08.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.628+0000 7f9141755640 1 Processor -- start 2024-04-14T08:40:08.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.628+0000 7f9141755640 1 -- start start 2024-04-14T08:40:08.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f9141755640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f913c0d1a20 0x7f913c06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f9141755640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f913c071250 0x7f913c0cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f9141755640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913c06d000 0x7f913c06d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f9141755640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f913c06d9d0 con 0x7f913c0d1a20 2024-04-14T08:40:08.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f9141755640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f913c070dc0 con 0x7f913c071250 2024-04-14T08:40:08.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f9141755640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f913c0701b0 con 0x7f913c06d000 2024-04-14T08:40:08.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f913affd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f913c0d1a20 0x7f913c06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f913affd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f913c0d1a20 0x7f913c06fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39062/0 (socket says 172.21.15.172:39062) 2024-04-14T08:40:08.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f913affd640 1 -- 172.21.15.172:0/4241831626 learned_addr learned my addr 172.21.15.172:0/4241831626 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:08.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f913b7fe640 1 --2- 172.21.15.172:0/4241831626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913c06d000 0x7f913c06d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f913a7fc640 1 --2- 172.21.15.172:0/4241831626 >> v2:172.21.15.177:3300/0 conn(0x7f913c071250 0x7f913c0cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f913affd640 1 -- 172.21.15.172:0/4241831626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913c06d000 msgr2=0x7f913c06d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:08.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f913affd640 1 --2- 172.21.15.172:0/4241831626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913c06d000 0x7f913c06d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f913affd640 1 -- 172.21.15.172:0/4241831626 >> v2:172.21.15.177:3300/0 conn(0x7f913c071250 msgr2=0x7f913c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:08.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f913affd640 1 --2- 172.21.15.172:0/4241831626 >> v2:172.21.15.177:3300/0 conn(0x7f913c071250 0x7f913c0cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.630+0000 7f913affd640 1 -- 172.21.15.172:0/4241831626 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c070490 con 0x7f913c0d1a20 2024-04-14T08:40:08.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.631+0000 7f913affd640 1 --2- 172.21.15.172:0/4241831626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c0d1a20 0x7f913c06fb80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f913c0c5bf0 tx=0x7f912c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d315976590ba1cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:08.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.631+0000 7f9139ffb640 1 -- 172.21.15.172:0/4241831626 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c02e070 con 0x7f913c0d1a20 2024-04-14T08:40:08.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.631+0000 7f9139ffb640 1 -- 172.21.15.172:0/4241831626 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912c037070 con 0x7f913c0d1a20 2024-04-14T08:40:08.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.631+0000 7f9139ffb640 1 -- 172.21.15.172:0/4241831626 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c031a00 con 0x7f913c0d1a20 2024-04-14T08:40:08.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.632+0000 7f9141755640 1 -- 172.21.15.172:0/4241831626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c0d1a20 msgr2=0x7f913c06fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.632+0000 7f9141755640 1 --2- 172.21.15.172:0/4241831626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c0d1a20 0x7f913c06fb80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f913c0c5bf0 tx=0x7f912c02b880 comp rx=0 tx=0).stop 2024-04-14T08:40:08.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.632+0000 7f9141755640 1 -- 172.21.15.172:0/4241831626 shutdown_connections 2024-04-14T08:40:08.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.632+0000 7f9141755640 1 --2- 172.21.15.172:0/4241831626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913c06d000 0x7f913c06d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.632+0000 7f9141755640 1 --2- 172.21.15.172:0/4241831626 >> v2:172.21.15.177:3300/0 conn(0x7f913c071250 0x7f913c0cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.632+0000 7f9141755640 1 --2- 172.21.15.172:0/4241831626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c0d1a20 0x7f913c06fb80 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.632+0000 7f9141755640 1 -- 172.21.15.172:0/4241831626 >> 172.21.15.172:0/4241831626 conn(0x7f913c08c000 msgr2=0x7f913c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:08.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.632+0000 7f9141755640 1 -- 172.21.15.172:0/4241831626 shutdown_connections 2024-04-14T08:40:08.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.632+0000 7f9141755640 1 -- 172.21.15.172:0/4241831626 wait complete. 2024-04-14T08:40:08.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.633+0000 7f9141755640 1 Processor -- start 2024-04-14T08:40:08.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.633+0000 7f9141755640 1 -- start start 2024-04-14T08:40:08.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f9141755640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f913c06d000 0x7f913c0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f9141755640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913c071250 0x7f913c0b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f9141755640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c0d1a20 0x7f913c15e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f9141755640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f913c0cfa70 con 0x7f913c0d1a20 2024-04-14T08:40:08.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f9141755640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f913c070a50 con 0x7f913c06d000 2024-04-14T08:40:08.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f9141755640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f913c0d1e00 con 0x7f913c071250 2024-04-14T08:40:08.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f913b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c0d1a20 0x7f913c15e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f913b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c0d1a20 0x7f913c15e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39070/0 (socket says 172.21.15.172:39070) 2024-04-14T08:40:08.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f913b7fe640 1 -- 172.21.15.172:0/1480586686 learned_addr learned my addr 172.21.15.172:0/1480586686 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:08.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f913a7fc640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913c071250 0x7f913c0b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f913affd640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f913c06d000 0x7f913c0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f913b7fe640 1 -- 172.21.15.172:0/1480586686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913c071250 msgr2=0x7f913c0b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f913b7fe640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913c071250 0x7f913c0b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f913b7fe640 1 -- 172.21.15.172:0/1480586686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f913c06d000 msgr2=0x7f913c0ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f913b7fe640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f913c06d000 0x7f913c0ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.634+0000 7f913b7fe640 1 -- 172.21.15.172:0/1480586686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c1f4f50 con 0x7f913c0d1a20 2024-04-14T08:40:08.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.635+0000 7f913b7fe640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c0d1a20 0x7f913c15e630 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9134007e60 tx=0x7f913400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:08.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.635+0000 7f91237fe640 1 -- 172.21.15.172:0/1480586686 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134014420 con 0x7f913c0d1a20 2024-04-14T08:40:08.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.635+0000 7f91237fe640 1 -- 172.21.15.172:0/1480586686 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9134019070 con 0x7f913c0d1a20 2024-04-14T08:40:08.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.635+0000 7f91237fe640 1 -- 172.21.15.172:0/1480586686 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134014820 con 0x7f913c0d1a20 2024-04-14T08:40:08.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.635+0000 7f9141755640 1 -- 172.21.15.172:0/1480586686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913c1f5230 con 0x7f913c0d1a20 2024-04-14T08:40:08.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.635+0000 7f9141755640 1 -- 172.21.15.172:0/1480586686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913c1f57f0 con 0x7f913c0d1a20 2024-04-14T08:40:08.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.636+0000 7f9141755640 1 -- 172.21.15.172:0/1480586686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f913c06e390 con 0x7f913c0d1a20 2024-04-14T08:40:08.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.636+0000 7f91237fe640 1 -- 172.21.15.172:0/1480586686 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91340149c0 con 0x7f913c0d1a20 2024-04-14T08:40:08.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.637+0000 7f91237fe640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f912806c8f0 0x7f912806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.637+0000 7f91237fe640 1 -- 172.21.15.172:0/1480586686 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f913408c3d0 con 0x7f913c0d1a20 2024-04-14T08:40:08.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.637+0000 7f913affd640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f912806c8f0 0x7f912806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.637+0000 7f913affd640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f912806c8f0 0x7f912806edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f912c036000 tx=0x7f912c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:08.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.645+0000 7f91237fe640 1 -- 172.21.15.172:0/1480586686 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f913c06e390 con 0x7f913c0d1a20 2024-04-14T08:40:08.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.825+0000 7f9141755640 1 -- 172.21.15.172:0/1480586686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f913c0cd210 con 0x7f913c0d1a20 2024-04-14T08:40:08.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.825+0000 7f91237fe640 1 -- 172.21.15.172:0/1480586686 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f9134059ff0 con 0x7f913c0d1a20 2024-04-14T08:40:08.826 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:40:08.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.827+0000 7f9141755640 1 -- 172.21.15.172:0/1480586686 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f912806c8f0 msgr2=0x7f912806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.827+0000 7f9141755640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f912806c8f0 0x7f912806edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f912c036000 tx=0x7f912c03d040 comp rx=0 tx=0).stop 2024-04-14T08:40:08.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.827+0000 7f9141755640 1 -- 172.21.15.172:0/1480586686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c0d1a20 msgr2=0x7f913c15e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.827+0000 7f9141755640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c0d1a20 0x7f913c15e630 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9134007e60 tx=0x7f913400e4b0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.827+0000 7f9141755640 1 -- 172.21.15.172:0/1480586686 shutdown_connections 2024-04-14T08:40:08.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.827+0000 7f9141755640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f912806c8f0 0x7f912806edb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.827+0000 7f9141755640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c0d1a20 0x7f913c15e630 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.827+0000 7f9141755640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f913c071250 0x7f913c0b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.827+0000 7f9141755640 1 --2- 172.21.15.172:0/1480586686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f913c06d000 0x7f913c0ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.827+0000 7f9141755640 1 -- 172.21.15.172:0/1480586686 >> 172.21.15.172:0/1480586686 conn(0x7f913c08c000 msgr2=0x7f913c0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:08.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.827+0000 7f9141755640 1 -- 172.21.15.172:0/1480586686 shutdown_connections 2024-04-14T08:40:08.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.827+0000 7f9141755640 1 -- 172.21.15.172:0/1480586686 wait complete. 2024-04-14T08:40:08.839 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:40:08.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.941+0000 7fd2c617b640 1 Processor -- start 2024-04-14T08:40:08.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.941+0000 7fd2c617b640 1 -- start start 2024-04-14T08:40:08.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.942+0000 7fd2c617b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd2c015f460 0x7fd2c00b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.942+0000 7fd2c617b640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c015edd0 con 0x7fd2c00b3bf0 2024-04-14T08:40:08.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.942+0000 7fd2c617b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c006c100 con 0x7fd2c006bc30 2024-04-14T08:40:08.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.942+0000 7fd2c617b640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd2c0141960 con 0x7fd2c015f460 2024-04-14T08:40:08.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.942+0000 7fd2bf7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd2c015f460 0x7fd2c00b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.942+0000 7fd2beffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd2c00b3bf0 0x7fd2c006b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47722/0 (socket says 172.21.15.172:47722) 2024-04-14T08:40:08.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.942+0000 7fd2beffd640 1 -- 172.21.15.172:0/1012115093 learned_addr learned my addr 172.21.15.172:0/1012115093 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:08.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.943+0000 7fd2bf7fe640 1 -- 172.21.15.172:0/1012115093 >> v1:172.21.15.177:6789/0 conn(0x7fd2c006bc30 legacy=0x7fd2c01683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.943+0000 7fd2bf7fe640 1 -- 172.21.15.172:0/1012115093 >> v1:172.21.15.172:6789/0 conn(0x7fd2c00b3bf0 legacy=0x7fd2c006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.943+0000 7fd2bf7fe640 1 -- 172.21.15.172:0/1012115093 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c006c3e0 con 0x7fd2c015f460 2024-04-14T08:40:08.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.943+0000 7fd2be7fc640 1 -- 172.21.15.172:0/1012115093 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2450195069 0 0) 0x7fd2c015edd0 con 0x7fd2c00b3bf0 2024-04-14T08:40:08.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.944+0000 7fd2bf7fe640 1 --2- 172.21.15.172:0/1012115093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c015f460 0x7fd2c00b36b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009d90 tx=0x7fd2b002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9c7afcec2d725f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:08.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.944+0000 7fd2be7fc640 1 -- 172.21.15.172:0/1012115093 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0039070 con 0x7fd2c015f460 2024-04-14T08:40:08.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.944+0000 7fd2be7fc640 1 -- 172.21.15.172:0/1012115093 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b002cc60 con 0x7fd2c015f460 2024-04-14T08:40:08.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.944+0000 7fd2be7fc640 1 -- 172.21.15.172:0/1012115093 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0034660 con 0x7fd2c015f460 2024-04-14T08:40:08.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.945+0000 7fd2c617b640 1 -- 172.21.15.172:0/1012115093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c015f460 msgr2=0x7fd2c00b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.945+0000 7fd2c617b640 1 --2- 172.21.15.172:0/1012115093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c015f460 0x7fd2c00b36b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009d90 tx=0x7fd2b002beb0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.945+0000 7fd2c617b640 1 -- 172.21.15.172:0/1012115093 shutdown_connections 2024-04-14T08:40:08.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.945+0000 7fd2c617b640 1 --2- 172.21.15.172:0/1012115093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c015f460 0x7fd2c00b36b0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.945+0000 7fd2c617b640 1 -- 172.21.15.172:0/1012115093 >> 172.21.15.172:0/1012115093 conn(0x7fd2c00d0780 msgr2=0x7fd2c00d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:08.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.945+0000 7fd2c617b640 1 -- 172.21.15.172:0/1012115093 shutdown_connections 2024-04-14T08:40:08.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.945+0000 7fd2c617b640 1 -- 172.21.15.172:0/1012115093 wait complete. 2024-04-14T08:40:08.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.946+0000 7fd2c617b640 1 Processor -- start 2024-04-14T08:40:08.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.947+0000 7fd2c617b640 1 -- start start 2024-04-14T08:40:08.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2c617b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c006bc30 0x7fd2c01f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2c617b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c00b3bf0 0x7fd2c01f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2c617b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c015f460 0x7fd2c0153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2c617b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c015f840 con 0x7fd2c015f460 2024-04-14T08:40:08.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2c617b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c006c9a0 con 0x7fd2c00b3bf0 2024-04-14T08:40:08.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2c617b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c015f9c0 con 0x7fd2c006bc30 2024-04-14T08:40:08.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2bffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c015f460 0x7fd2c0153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2bffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c015f460 0x7fd2c0153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39096/0 (socket says 172.21.15.172:39096) 2024-04-14T08:40:08.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2bffff640 1 -- 172.21.15.172:0/3555124325 learned_addr learned my addr 172.21.15.172:0/3555124325 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:08.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2beffd640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c00b3bf0 0x7fd2c01f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2bf7fe640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c006bc30 0x7fd2c01f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2bffff640 1 -- 172.21.15.172:0/3555124325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c006bc30 msgr2=0x7fd2c01f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2bffff640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c006bc30 0x7fd2c01f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2bffff640 1 -- 172.21.15.172:0/3555124325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c00b3bf0 msgr2=0x7fd2c01f1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:08.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2bffff640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c00b3bf0 0x7fd2c01f1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:08.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2bffff640 1 -- 172.21.15.172:0/3555124325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c0154940 con 0x7fd2c015f460 2024-04-14T08:40:08.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2bffff640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c015f460 0x7fd2c0153b70 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800c3f0 tx=0x7fd2b800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:08.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2a7fff640 1 -- 172.21.15.172:0/3555124325 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8017070 con 0x7fd2c015f460 2024-04-14T08:40:08.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2a7fff640 1 -- 172.21.15.172:0/3555124325 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b80052d0 con 0x7fd2c015f460 2024-04-14T08:40:08.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.948+0000 7fd2c617b640 1 -- 172.21.15.172:0/3555124325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c0155570 con 0x7fd2c015f460 2024-04-14T08:40:08.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.949+0000 7fd2a7fff640 1 -- 172.21.15.172:0/3555124325 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8003830 con 0x7fd2c015f460 2024-04-14T08:40:08.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.949+0000 7fd2c617b640 1 -- 172.21.15.172:0/3555124325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c0155b30 con 0x7fd2c015f460 2024-04-14T08:40:08.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.950+0000 7fd2c617b640 1 -- 172.21.15.172:0/3555124325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c0068590 con 0x7fd2c015f460 2024-04-14T08:40:08.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.950+0000 7fd2a7fff640 1 -- 172.21.15.172:0/3555124325 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2b801a070 con 0x7fd2c015f460 2024-04-14T08:40:08.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.951+0000 7fd2a7fff640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd2ac06c8f0 0x7fd2ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:08.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.951+0000 7fd2bf7fe640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd2ac06c8f0 0x7fd2ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:08.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.951+0000 7fd2a7fff640 1 -- 172.21.15.172:0/3555124325 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7fd2b808af00 con 0x7fd2c015f460 2024-04-14T08:40:08.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.951+0000 7fd2bf7fe640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd2ac06c8f0 0x7fd2ac06edb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0036000 tx=0x7fd2b0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:08.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:08.955+0000 7fd2a7fff640 1 -- 172.21.15.172:0/3555124325 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd2b8058be0 con 0x7fd2c015f460 2024-04-14T08:40:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.149+0000 7fd2c617b640 1 -- 172.21.15.172:0/3555124325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd2c015d690 con 0x7fd2c015f460 2024-04-14T08:40:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.150+0000 7fd2a7fff640 1 -- 172.21.15.172:0/3555124325 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v89) ==== 80+0+124 (secure 0 0 0) 0x7fd2b805cee0 con 0x7fd2c015f460 2024-04-14T08:40:09.150 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.177:6835/1124731311 2024-04-15T08:38:58.388334+0000 2024-04-14T08:40:09.151 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.177:6833/1124731311 2024-04-15T08:38:58.388334+0000 2024-04-14T08:40:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:listed 2 entries 2024-04-14T08:40:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.152+0000 7fd2c617b640 1 -- 172.21.15.172:0/3555124325 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd2ac06c8f0 msgr2=0x7fd2ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.152+0000 7fd2c617b640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd2ac06c8f0 0x7fd2ac06edb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0036000 tx=0x7fd2b0030000 comp rx=0 tx=0).stop 2024-04-14T08:40:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.152+0000 7fd2c617b640 1 -- 172.21.15.172:0/3555124325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c015f460 msgr2=0x7fd2c0153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.152+0000 7fd2c617b640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c015f460 0x7fd2c0153b70 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800c3f0 tx=0x7fd2b800c7f0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.152+0000 7fd2c617b640 1 -- 172.21.15.172:0/3555124325 shutdown_connections 2024-04-14T08:40:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.152+0000 7fd2c617b640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd2ac06c8f0 0x7fd2ac06edb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.152+0000 7fd2c617b640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c015f460 0x7fd2c0153b70 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.152+0000 7fd2c617b640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2c00b3bf0 0x7fd2c01f1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.152+0000 7fd2c617b640 1 --2- 172.21.15.172:0/3555124325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2c006bc30 0x7fd2c01f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.152+0000 7fd2c617b640 1 -- 172.21.15.172:0/3555124325 >> 172.21.15.172:0/3555124325 conn(0x7fd2c00d0780 msgr2=0x7fd2c00ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.153+0000 7fd2c617b640 1 -- 172.21.15.172:0/3555124325 shutdown_connections 2024-04-14T08:40:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.153+0000 7fd2c617b640 1 -- 172.21.15.172:0/3555124325 wait complete. 2024-04-14T08:40:09.164 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:40:09.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:09.164 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:40:09.195 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:40:09.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:09.196 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:40:09.251 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:40:09.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.393+0000 7f5e0e11e640 1 Processor -- start 2024-04-14T08:40:09.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.393+0000 7f5e0e11e640 1 -- start start 2024-04-14T08:40:09.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e0e11e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e0815c710 0x7f5e080d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e0e11e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e080aef60 con 0x7f5e080cde10 2024-04-14T08:40:09.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e0e11e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e080af240 con 0x7f5e080cef50 2024-04-14T08:40:09.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e0e11e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e080aed10 con 0x7f5e0815c710 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e06ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5e080cde10 0x7f5e080ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:47724/0 (socket says 172.21.15.172:47724) 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e06ffd640 1 -- 172.21.15.172:0/1367682792 learned_addr learned my addr 172.21.15.172:0/1367682792 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e077fe640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e0815c710 0x7f5e080d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3609030090 0 0) 0x7f5e080aef60 con 0x7f5e080cde10 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5df0003520 con 0x7f5e080cde10 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1478597993 0 0) 0x7f5df0003520 con 0x7f5e080cde10 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e0815c710 msgr2=0x7f5e080d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e067fc640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e0815c710 0x7f5e080d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 >> v1:172.21.15.177:6789/0 conn(0x7f5e080cef50 legacy=0x7f5e080afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e080af520 con 0x7f5e080cde10 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f5dfc001670 con 0x7f5e080cde10 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.394+0000 7f5e077fe640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e0815c710 0x7f5e080d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:40:09.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 >> v1:172.21.15.172:6789/0 conn(0x7f5e080cde10 legacy=0x7f5e080ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e067fc640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5df00039e0 0x7f5df0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e067fc640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df0004390 0x7f5df0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e067fc640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e0815c710 0x7f5df0005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e080aed10 con 0x7f5df0004390 2024-04-14T08:40:09.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5df000b9c0 con 0x7f5df00039e0 2024-04-14T08:40:09.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5df000bb70 con 0x7f5e0815c710 2024-04-14T08:40:09.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e06ffd640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df0004390 0x7f5df0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e077fe640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5df00039e0 0x7f5df0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e07fff640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e0815c710 0x7f5df0005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e06ffd640 1 -- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e0815c710 msgr2=0x7f5df0005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:09.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e06ffd640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e0815c710 0x7f5df0005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e06ffd640 1 -- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5df00039e0 msgr2=0x7f5df0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:09.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e06ffd640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5df00039e0 0x7f5df0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e06ffd640 1 -- 172.21.15.172:0/1367682792 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df0003520 con 0x7f5df0004390 2024-04-14T08:40:09.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.395+0000 7f5e07fff640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e0815c710 0x7f5df0005080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:09.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.396+0000 7f5e06ffd640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df0004390 0x7f5df0004860 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f5e080ce250 tx=0x7f5df0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd034fe1d4d9d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:09.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.396+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc007ce0 con 0x7f5df0004390 2024-04-14T08:40:09.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.396+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dfc030070 con 0x7f5df0004390 2024-04-14T08:40:09.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.396+0000 7f5e067fc640 1 -- 172.21.15.172:0/1367682792 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc005d30 con 0x7f5df0004390 2024-04-14T08:40:09.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.396+0000 7f5e0e11e640 1 -- 172.21.15.172:0/1367682792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df0004390 msgr2=0x7f5df0004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.396+0000 7f5e0e11e640 1 --2- 172.21.15.172:0/1367682792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df0004390 0x7f5df0004860 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f5e080ce250 tx=0x7f5df0005830 comp rx=0 tx=0).stop 2024-04-14T08:40:09.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.396+0000 7f5e07fff640 1 -- 172.21.15.172:0/1367682792 reap_dead start 2024-04-14T08:40:09.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.396+0000 7f5e0e11e640 1 -- 172.21.15.172:0/1367682792 shutdown_connections 2024-04-14T08:40:09.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.396+0000 7f5e0e11e640 1 -- 172.21.15.172:0/1367682792 >> 172.21.15.172:0/1367682792 conn(0x7f5e0808c000 msgr2=0x7f5e080d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:09.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.396+0000 7f5e0e11e640 1 -- 172.21.15.172:0/1367682792 shutdown_connections 2024-04-14T08:40:09.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.396+0000 7f5e0e11e640 1 -- 172.21.15.172:0/1367682792 wait complete. 2024-04-14T08:40:09.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.398+0000 7f5e0e11e640 1 Processor -- start 2024-04-14T08:40:09.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.398+0000 7f5e0e11e640 1 -- start start 2024-04-14T08:40:09.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.398+0000 7f5e0e11e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5df0004390 0x7f5e080b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.398+0000 7f5e0e11e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e081f53a0 0x7f5e080afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.398+0000 7f5e0e11e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e080b0160 0x7f5e081f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.398+0000 7f5e0e11e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e080d0950 con 0x7f5e081f53a0 2024-04-14T08:40:09.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.398+0000 7f5e0e11e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e080af890 con 0x7f5e080b0160 2024-04-14T08:40:09.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.398+0000 7f5e0e11e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e080d0ad0 con 0x7f5df0004390 2024-04-14T08:40:09.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e06ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e081f53a0 0x7f5e080afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e06ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e081f53a0 0x7f5e080afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39116/0 (socket says 172.21.15.172:39116) 2024-04-14T08:40:09.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e06ffd640 1 -- 172.21.15.172:0/330088409 learned_addr learned my addr 172.21.15.172:0/330088409 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:09.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e077fe640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5df0004390 0x7f5e080b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e07fff640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e080b0160 0x7f5e081f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e06ffd640 1 -- 172.21.15.172:0/330088409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5df0004390 msgr2=0x7f5e080b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e06ffd640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5df0004390 0x7f5e080b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e06ffd640 1 -- 172.21.15.172:0/330088409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e080b0160 msgr2=0x7f5e081f0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e06ffd640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e080b0160 0x7f5e081f0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e06ffd640 1 -- 172.21.15.172:0/330088409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e080cac80 con 0x7f5e081f53a0 2024-04-14T08:40:09.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e077fe640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5df0004390 0x7f5e080b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:09.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e07fff640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e080b0160 0x7f5e081f0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:09.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e06ffd640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e081f53a0 0x7f5e080afc20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0018f0 tx=0x7f5e0806f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:09.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5deffff640 1 -- 172.21.15.172:0/330088409 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc007ce0 con 0x7f5e081f53a0 2024-04-14T08:40:09.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5deffff640 1 -- 172.21.15.172:0/330088409 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dfc004bf0 con 0x7f5e081f53a0 2024-04-14T08:40:09.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e0e11e640 1 -- 172.21.15.172:0/330088409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e080caf60 con 0x7f5e081f53a0 2024-04-14T08:40:09.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5deffff640 1 -- 172.21.15.172:0/330088409 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc007070 con 0x7f5e081f53a0 2024-04-14T08:40:09.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.399+0000 7f5e0e11e640 1 -- 172.21.15.172:0/330088409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e080cbf40 con 0x7f5e081f53a0 2024-04-14T08:40:09.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.400+0000 7f5e0e11e640 1 -- 172.21.15.172:0/330088409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e08075490 con 0x7f5e081f53a0 2024-04-14T08:40:09.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.401+0000 7f5deffff640 1 -- 172.21.15.172:0/330088409 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dfc047020 con 0x7f5e081f53a0 2024-04-14T08:40:09.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.401+0000 7f5deffff640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5df406c8f0 0x7f5df406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.401+0000 7f5deffff640 1 -- 172.21.15.172:0/330088409 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7f5dfc0acf50 con 0x7f5e081f53a0 2024-04-14T08:40:09.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.401+0000 7f5e077fe640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5df406c8f0 0x7f5df406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.402+0000 7f5e077fe640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5df406c8f0 0x7f5df406edb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009bd0 tx=0x7f5df8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:09.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.406+0000 7f5deffff640 1 -- 172.21.15.172:0/330088409 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5dfc07b1c0 con 0x7f5e081f53a0 2024-04-14T08:40:09.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.621+0000 7f5e0e11e640 1 -- 172.21.15.172:0/330088409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e0815d150 con 0x7f5e081f53a0 2024-04-14T08:40:09.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.621+0000 7f5deffff640 1 -- 172.21.15.172:0/330088409 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5196 (secure 0 0 0) 0x7f5dfc07aac0 con 0x7f5e081f53a0 2024-04-14T08:40:09.623 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:09.623 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:39:05.413377+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7473":{"gid":7473,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6837/1386685680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1386685680},{"type":"v1","addr":"172.21.15.172:6837","nonce":1386685680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7473,"qdb_cluster":[7473]},"id":6}]} 2024-04-14T08:40:09.624 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 61 2024-04-14T08:40:09.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.623+0000 7f5e0e11e640 1 -- 172.21.15.172:0/330088409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5df406c8f0 msgr2=0x7f5df406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.623+0000 7f5e0e11e640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5df406c8f0 0x7f5df406edb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009bd0 tx=0x7f5df8008040 comp rx=0 tx=0).stop 2024-04-14T08:40:09.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.623+0000 7f5e0e11e640 1 -- 172.21.15.172:0/330088409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e081f53a0 msgr2=0x7f5e080afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.623+0000 7f5e0e11e640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e081f53a0 0x7f5e080afc20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0018f0 tx=0x7f5e0806f330 comp rx=0 tx=0).stop 2024-04-14T08:40:09.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.623+0000 7f5e0e11e640 1 -- 172.21.15.172:0/330088409 shutdown_connections 2024-04-14T08:40:09.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.623+0000 7f5e0e11e640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5df406c8f0 0x7f5df406edb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.623+0000 7f5e0e11e640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e080b0160 0x7f5e081f0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.623+0000 7f5e0e11e640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e081f53a0 0x7f5e080afc20 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.623+0000 7f5e0e11e640 1 --2- 172.21.15.172:0/330088409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5df0004390 0x7f5e080b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.623+0000 7f5e0e11e640 1 -- 172.21.15.172:0/330088409 >> 172.21.15.172:0/330088409 conn(0x7f5e0808c000 msgr2=0x7f5e08070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:09.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.624+0000 7f5e0e11e640 1 -- 172.21.15.172:0/330088409 shutdown_connections 2024-04-14T08:40:09.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.624+0000 7f5e0e11e640 1 -- 172.21.15.172:0/330088409 wait complete. 2024-04-14T08:40:09.639 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-14T08:40:09.772 INFO:teuthology.orchestra.run.smithi172.stdout:{"message":"","return_code":0} 2024-04-14T08:40:09.773 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-14T08:40:09.773 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:40:09.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.874+0000 7ff73b7b4640 1 Processor -- start 2024-04-14T08:40:09.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.875+0000 7ff73b7b4640 1 -- start start 2024-04-14T08:40:09.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.875+0000 7ff73b7b4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7340d0cc0 0x7ff7340d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.875+0000 7ff73b7b4640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff7340d4e20 0x7ff734161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.875+0000 7ff73b7b4640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7340d15e0 con 0x7ff7340d49c0 2024-04-14T08:40:09.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.875+0000 7ff73b7b4640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff7340ceec0 con 0x7ff7340d4e20 2024-04-14T08:40:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.875+0000 7ff73b7b4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff7340cfd00 con 0x7ff7340d0cc0 2024-04-14T08:40:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff738d4f640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff7340d49c0 0x7ff734160f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50306/0 (socket says 172.21.15.172:50306) 2024-04-14T08:40:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff738d4f640 1 -- 172.21.15.172:0/2579420981 learned_addr learned my addr 172.21.15.172:0/2579420981 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff739550640 1 --2- 172.21.15.172:0/2579420981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7340d0cc0 0x7ff7340d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff739d51640 1 --2- 172.21.15.172:0/2579420981 >> v2:172.21.15.177:3300/0 conn(0x7ff7340d4e20 0x7ff734161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1883125708 0 0) 0x7ff7340d15e0 con 0x7ff7340d49c0 2024-04-14T08:40:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff720003520 con 0x7ff7340d49c0 2024-04-14T08:40:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1510878801 0 0) 0x7ff720003520 con 0x7ff7340d49c0 2024-04-14T08:40:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7340d0cc0 msgr2=0x7ff7340d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 --2- 172.21.15.172:0/2579420981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7340d0cc0 0x7ff7340d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 >> v2:172.21.15.177:3300/0 conn(0x7ff7340d4e20 msgr2=0x7ff734161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 --2- 172.21.15.172:0/2579420981 >> v2:172.21.15.177:3300/0 conn(0x7ff7340d4e20 0x7ff734161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7340cf0f0 con 0x7ff7340d49c0 2024-04-14T08:40:09.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7ff724001670 con 0x7ff7340d49c0 2024-04-14T08:40:09.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 >> v1:172.21.15.172:6789/0 conn(0x7ff7340d49c0 legacy=0x7ff734160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 --2- 172.21.15.172:0/2579420981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7200039b0 0x7ff720003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 --2- 172.21.15.172:0/2579420981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7340d0cc0 0x7ff720004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 --2- 172.21.15.172:0/2579420981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff720004d40 0x7ff7200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7340cfd00 con 0x7ff720004d40 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff72000b9c0 con 0x7ff7200039b0 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff72000bb40 con 0x7ff7340d0cc0 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.876+0000 7ff739d51640 1 --2- 172.21.15.172:0/2579420981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff720004d40 0x7ff7200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.877+0000 7ff739550640 1 --2- 172.21.15.172:0/2579420981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7200039b0 0x7ff720003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.877+0000 7ff738d4f640 1 --2- 172.21.15.172:0/2579420981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7340d0cc0 0x7ff720004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.877+0000 7ff739d51640 1 -- 172.21.15.172:0/2579420981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7340d0cc0 msgr2=0x7ff720004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.877+0000 7ff739d51640 1 --2- 172.21.15.172:0/2579420981 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7340d0cc0 0x7ff720004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.877+0000 7ff739d51640 1 -- 172.21.15.172:0/2579420981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7200039b0 msgr2=0x7ff720003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.877+0000 7ff739d51640 1 --2- 172.21.15.172:0/2579420981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7200039b0 0x7ff720003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.877+0000 7ff739d51640 1 -- 172.21.15.172:0/2579420981 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff720003520 con 0x7ff720004d40 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.877+0000 7ff739d51640 1 --2- 172.21.15.172:0/2579420981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff720004d40 0x7ff7200051c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009ac0 tx=0x7ff720005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43583e411f5e8d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.878+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c039070 con 0x7ff720004d40 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.878+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff72c02c710 con 0x7ff720004d40 2024-04-14T08:40:09.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.878+0000 7ff72bfff640 1 -- 172.21.15.172:0/2579420981 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c009690 con 0x7ff720004d40 2024-04-14T08:40:09.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.878+0000 7ff73b7b4640 1 -- 172.21.15.172:0/2579420981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff720004d40 msgr2=0x7ff7200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.878+0000 7ff73b7b4640 1 --2- 172.21.15.172:0/2579420981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff720004d40 0x7ff7200051c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009ac0 tx=0x7ff720005830 comp rx=0 tx=0).stop 2024-04-14T08:40:09.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.878+0000 7ff739d51640 1 -- 172.21.15.172:0/2579420981 reap_dead start 2024-04-14T08:40:09.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.878+0000 7ff73b7b4640 1 -- 172.21.15.172:0/2579420981 shutdown_connections 2024-04-14T08:40:09.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.878+0000 7ff73b7b4640 1 -- 172.21.15.172:0/2579420981 >> 172.21.15.172:0/2579420981 conn(0x7ff73409d580 msgr2=0x7ff7340d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:09.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.878+0000 7ff73b7b4640 1 -- 172.21.15.172:0/2579420981 shutdown_connections 2024-04-14T08:40:09.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.878+0000 7ff73b7b4640 1 -- 172.21.15.172:0/2579420981 wait complete. 2024-04-14T08:40:09.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.879+0000 7ff73b7b4640 1 Processor -- start 2024-04-14T08:40:09.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.879+0000 7ff73b7b4640 1 -- start start 2024-04-14T08:40:09.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.880+0000 7ff73b7b4640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff720004d40 0x7ff7341e3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.880+0000 7ff73b7b4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7341e2460 0x7ff7340cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.880+0000 7ff73b7b4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff73409f150 0x7ff7340ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.880+0000 7ff73b7b4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7340cf990 con 0x7ff73409f150 2024-04-14T08:40:09.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.880+0000 7ff73b7b4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff7340cf6b0 con 0x7ff7341e2460 2024-04-14T08:40:09.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.880+0000 7ff73b7b4640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff7340d32b0 con 0x7ff720004d40 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.880+0000 7ff739d51640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff73409f150 0x7ff7340ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.880+0000 7ff739d51640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff73409f150 0x7ff7340ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41950/0 (socket says 172.21.15.172:41950) 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.880+0000 7ff739d51640 1 -- 172.21.15.172:0/3675667607 learned_addr learned my addr 172.21.15.172:0/3675667607 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.880+0000 7ff739550640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff720004d40 0x7ff7341e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.880+0000 7ff738d4f640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7341e2460 0x7ff7340cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.881+0000 7ff739d51640 1 -- 172.21.15.172:0/3675667607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff720004d40 msgr2=0x7ff7341e3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.881+0000 7ff739d51640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff720004d40 0x7ff7341e3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.881+0000 7ff739d51640 1 -- 172.21.15.172:0/3675667607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7341e2460 msgr2=0x7ff7340cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.881+0000 7ff739d51640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7341e2460 0x7ff7340cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.881+0000 7ff739d51640 1 -- 172.21.15.172:0/3675667607 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7341e3cb0 con 0x7ff73409f150 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.881+0000 7ff738d4f640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7341e2460 0x7ff7340cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.881+0000 7ff739d51640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff73409f150 0x7ff7340ccaa0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009ac0 tx=0x7ff72c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.881+0000 7ff729ffb640 1 -- 172.21.15.172:0/3675667607 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c039070 con 0x7ff73409f150 2024-04-14T08:40:09.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.881+0000 7ff729ffb640 1 -- 172.21.15.172:0/3675667607 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff72c043070 con 0x7ff73409f150 2024-04-14T08:40:09.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.881+0000 7ff729ffb640 1 -- 172.21.15.172:0/3675667607 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c009690 con 0x7ff73409f150 2024-04-14T08:40:09.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.881+0000 7ff73b7b4640 1 -- 172.21.15.172:0/3675667607 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7340c7c80 con 0x7ff73409f150 2024-04-14T08:40:09.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.881+0000 7ff73b7b4640 1 -- 172.21.15.172:0/3675667607 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7340c8240 con 0x7ff73409f150 2024-04-14T08:40:09.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.882+0000 7ff73b7b4640 1 -- 172.21.15.172:0/3675667607 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff734075490 con 0x7ff73409f150 2024-04-14T08:40:09.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.883+0000 7ff729ffb640 1 -- 172.21.15.172:0/3675667607 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff72c003850 con 0x7ff73409f150 2024-04-14T08:40:09.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.883+0000 7ff729ffb640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff71806c8f0 0x7ff71806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:09.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.883+0000 7ff729ffb640 1 -- 172.21.15.172:0/3675667607 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6303+0+0 (secure 0 0 0) 0x7ff72c0acda0 con 0x7ff73409f150 2024-04-14T08:40:09.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.883+0000 7ff739550640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff71806c8f0 0x7ff71806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:09.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.884+0000 7ff739550640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff71806c8f0 0x7ff71806edb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff730014450 tx=0x7ff73000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:09.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:09.892+0000 7ff729ffb640 1 -- 172.21.15.172:0/3675667607 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff72c04c020 con 0x7ff73409f150 2024-04-14T08:40:10.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.106+0000 7ff73b7b4640 1 -- 172.21.15.172:0/3675667607 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff7340d0aa0 con 0x7ff73409f150 2024-04-14T08:40:10.237 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:40:10.107+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:40:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.108+0000 7ff729ffb640 1 -- 172.21.15.172:0/3675667607 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7ff72c07a9c0 con 0x7ff73409f150 2024-04-14T08:40:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:40:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.111+0000 7ff73b7b4640 1 -- 172.21.15.172:0/3675667607 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff71806c8f0 msgr2=0x7ff71806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.111+0000 7ff73b7b4640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff71806c8f0 0x7ff71806edb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff730014450 tx=0x7ff73000b040 comp rx=0 tx=0).stop 2024-04-14T08:40:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.111+0000 7ff73b7b4640 1 -- 172.21.15.172:0/3675667607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff73409f150 msgr2=0x7ff7340ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.111+0000 7ff73b7b4640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff73409f150 0x7ff7340ccaa0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff72c009ac0 tx=0x7ff72c004730 comp rx=0 tx=0).stop 2024-04-14T08:40:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.111+0000 7ff73b7b4640 1 -- 172.21.15.172:0/3675667607 shutdown_connections 2024-04-14T08:40:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.111+0000 7ff73b7b4640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff71806c8f0 0x7ff71806edb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.111+0000 7ff73b7b4640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff73409f150 0x7ff7340ccaa0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.111+0000 7ff73b7b4640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7341e2460 0x7ff7340cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.111+0000 7ff73b7b4640 1 --2- 172.21.15.172:0/3675667607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff720004d40 0x7ff7341e3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.111+0000 7ff73b7b4640 1 -- 172.21.15.172:0/3675667607 >> 172.21.15.172:0/3675667607 conn(0x7ff73409d580 msgr2=0x7ff7341e5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.112+0000 7ff73b7b4640 1 -- 172.21.15.172:0/3675667607 shutdown_connections 2024-04-14T08:40:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.112+0000 7ff73b7b4640 1 -- 172.21.15.172:0/3675667607 wait complete. 2024-04-14T08:40:10.240 INFO:tasks.ceph.mds.c.smithi172.stderr: -4236> 2024-04-14T08:39:04.992+0000 7effbcfe9640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-14T08:40:10.240 INFO:tasks.ceph.mds.c.smithi172.stderr: -4053> 2024-04-14T08:39:05.183+0000 7effbcfe9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-14T08:38:55.461911+0000 1=1+0) n(v0 rc2024-04-14T08:38:55.578911+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x555fea025b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-14T08:38:55.461911+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-14T08:38:55.578911+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-14T08:40:10.240 INFO:tasks.ceph.mds.c.smithi172.stderr: -3878> 2024-04-14T08:39:07.196+0000 7effc0ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:40:10.240 INFO:tasks.ceph.mds.c.smithi172.stderr: -2197> 2024-04-14T08:39:36.560+0000 7effbcfe9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-04-14T08:38:55.461911+0000 1=1+0)->f(v1 m2024-04-14T08:38:55.461911+0000) n(v0 rc2024-04-14T08:38:55.578911+0000 2=1+1)->n(v0 rc2024-04-14T08:38:55.578911+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x555fea025b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-14T08:38:55.461911+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-14T08:38:55.578911+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-14T08:40:10.241 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:40:10.241 DEBUG:teuthology.orchestra.run.smithi172:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-14T08:40:10.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.280+0000 7f9136823740 1 -- start start 2024-04-14T08:40:10.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.280+0000 7f9136823740 1 Processor -- start 2024-04-14T08:40:10.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9136823740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc10438c00 0x55cc105a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9136823740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc10439800 0x55cc105a4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9136823740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc10439c00 0x55cc105a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9136823740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cc1120e600 con 0x55cc10438c00 2024-04-14T08:40:10.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9136823740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cc1120e480 con 0x55cc10439c00 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9136823740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55cc1120e300 con 0x55cc10439800 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f91343fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc10438c00 0x55cc105a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f91343fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc10438c00 0x55cc105a4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41972/0 (socket says 172.21.15.172:41972) 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f91343fc640 1 -- 172.21.15.172:0/1396632656 learned_addr learned my addr 172.21.15.172:0/1396632656 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9133bfb640 1 --2- 172.21.15.172:0/1396632656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc10439800 0x55cc105a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9134bfd640 1 --2- 172.21.15.172:0/1396632656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc10439c00 0x55cc105a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f91343fc640 1 -- 172.21.15.172:0/1396632656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc10439800 msgr2=0x55cc105a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f91343fc640 1 --2- 172.21.15.172:0/1396632656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc10439800 0x55cc105a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f91343fc640 1 -- 172.21.15.172:0/1396632656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc10439c00 msgr2=0x55cc105a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f91343fc640 1 --2- 172.21.15.172:0/1396632656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc10439c00 0x55cc105a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f91343fc640 1 -- 172.21.15.172:0/1396632656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc10465680 con 0x55cc10438c00 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9133bfb640 1 --2- 172.21.15.172:0/1396632656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc10439800 0x55cc105a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:10.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9134bfd640 1 --2- 172.21.15.172:0/1396632656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc10439c00 0x55cc105a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f91343fc640 1 --2- 172.21.15.172:0/1396632656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc10438c00 0x55cc105a4000 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x55cc11245020 tx=0x55cc111d2a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d88c4418631174 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9131bf7640 1 -- 172.21.15.172:0/1396632656 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cc1046e9c0 con 0x55cc10438c00 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9131bf7640 1 -- 172.21.15.172:0/1396632656 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55cc10484a80 con 0x55cc10438c00 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9131bf7640 1 -- 172.21.15.172:0/1396632656 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cc1046e4e0 con 0x55cc10438c00 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9136823740 1 -- 172.21.15.172:0/1396632656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc10465860 con 0x55cc10438c00 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9136823740 1 -- 172.21.15.172:0/1396632656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55cc104652c0 con 0x55cc10438c00 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.281+0000 7f9136823740 4 waiting for MDS map... 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.282+0000 7f9131bf7640 1 -- 172.21.15.172:0/1396632656 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x55cc1130a600 con 0x55cc10438c00 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.282+0000 7f9131bf7640 1 -- 172.21.15.172:0/1396632656 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x55cc10493180 con 0x55cc10438c00 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.282+0000 7f9136823740 4 Got MDS map 62 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.282+0000 7f9136823740 4 datascan.main: Using metadata pool output 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.282+0000 7f9136823740 4 datascan.main: connecting to RADOS... 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.282+0000 7f9136823740 1 Processor -- start 2024-04-14T08:40:10.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.282+0000 7f9136823740 1 -- start start 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9136823740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364000 0x55cc105a5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9136823740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc11364400 0x55cc105a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9136823740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc11364800 0x55cc105a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9136823740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cc1120e180 con 0x55cc11364000 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9136823740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cc1045f800 con 0x55cc11364800 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9136823740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55cc1045ea80 con 0x55cc11364400 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9134bfd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364000 0x55cc105a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9134bfd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364000 0x55cc105a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41986/0 (socket says 172.21.15.172:41986) 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9134bfd640 1 -- 172.21.15.172:0/1628155625 learned_addr learned my addr 172.21.15.172:0/1628155625 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9133bfb640 1 --2- 172.21.15.172:0/1628155625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc11364800 0x55cc105a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f91343fc640 1 --2- 172.21.15.172:0/1628155625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc11364400 0x55cc105a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9134bfd640 1 -- 172.21.15.172:0/1628155625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc11364400 msgr2=0x55cc105a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9134bfd640 1 --2- 172.21.15.172:0/1628155625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc11364400 0x55cc105a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9134bfd640 1 -- 172.21.15.172:0/1628155625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc11364800 msgr2=0x55cc105a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9134bfd640 1 --2- 172.21.15.172:0/1628155625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc11364800 0x55cc105a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9134bfd640 1 -- 172.21.15.172:0/1628155625 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc104654a0 con 0x55cc11364000 2024-04-14T08:40:10.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.283+0000 7f9134bfd640 1 --2- 172.21.15.172:0/1628155625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364000 0x55cc105a5080 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x55cc11226c90 tx=0x55cc1122b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b2c05c401bf99b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.284+0000 7f912fbf3640 1 -- 172.21.15.172:0/1628155625 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cc1046e9c0 con 0x55cc11364000 2024-04-14T08:40:10.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.284+0000 7f912fbf3640 1 -- 172.21.15.172:0/1628155625 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55cc104841c0 con 0x55cc11364000 2024-04-14T08:40:10.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.284+0000 7f912fbf3640 1 -- 172.21.15.172:0/1628155625 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cc1046e4e0 con 0x55cc11364000 2024-04-14T08:40:10.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.284+0000 7f9136823740 1 -- 172.21.15.172:0/1628155625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364000 msgr2=0x55cc105a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.284+0000 7f9136823740 1 --2- 172.21.15.172:0/1628155625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364000 0x55cc105a5080 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x55cc11226c90 tx=0x55cc1122b800 comp rx=0 tx=0).stop 2024-04-14T08:40:10.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.284+0000 7f9136823740 1 -- 172.21.15.172:0/1628155625 shutdown_connections 2024-04-14T08:40:10.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.284+0000 7f9136823740 1 --2- 172.21.15.172:0/1628155625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc11364800 0x55cc105a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.284+0000 7f9136823740 1 --2- 172.21.15.172:0/1628155625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc11364400 0x55cc105a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.284+0000 7f9136823740 1 --2- 172.21.15.172:0/1628155625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364000 0x55cc105a5080 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.284+0000 7f9136823740 1 -- 172.21.15.172:0/1628155625 >> 172.21.15.172:0/1628155625 conn(0x55cc11211c00 msgr2=0x55cc10526800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:10.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.284+0000 7f9136823740 1 -- 172.21.15.172:0/1628155625 shutdown_connections 2024-04-14T08:40:10.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.284+0000 7f9136823740 1 -- 172.21.15.172:0/1628155625 wait complete. 2024-04-14T08:40:10.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9136823740 1 Processor -- start 2024-04-14T08:40:10.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9136823740 1 -- start start 2024-04-14T08:40:10.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9136823740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc11364000 0x55cc105a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9136823740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc11364400 0x55cc105a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9136823740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364800 0x55cc105a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9136823740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cc1120f380 con 0x55cc11364800 2024-04-14T08:40:10.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9136823740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cc1120e180 con 0x55cc11364000 2024-04-14T08:40:10.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9136823740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55cc1120f500 con 0x55cc11364400 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9133bfb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364800 0x55cc105a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9133bfb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364800 0x55cc105a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42002/0 (socket says 172.21.15.172:42002) 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 learned_addr learned my addr 172.21.15.172:0/2791991075 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f91343fc640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc11364400 0x55cc105a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9134bfd640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc11364000 0x55cc105a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc11364400 msgr2=0x55cc105a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9133bfb640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc11364400 0x55cc105a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc11364000 msgr2=0x55cc105a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9133bfb640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc11364000 0x55cc105a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.285+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc104650e0 con 0x55cc11364800 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.286+0000 7f9134bfd640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc11364000 0x55cc105a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.286+0000 7f9133bfb640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364800 0x55cc105a5b80 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x55cc11227980 tx=0x55cc1122b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.286+0000 7f912e3f0640 1 -- 172.21.15.172:0/2791991075 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cc1046e9c0 con 0x55cc11364800 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.286+0000 7f912e3f0640 1 -- 172.21.15.172:0/2791991075 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55cc10484540 con 0x55cc11364800 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.286+0000 7f912e3f0640 1 -- 172.21.15.172:0/2791991075 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cc1046e4e0 con 0x55cc11364800 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.286+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cc10465a40 con 0x55cc11364800 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.286+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc104643c0 con 0x55cc11364800 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.286+0000 7f9136823740 4 datascan.init: resolving metadata pool 13 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.287+0000 7f912e3f0640 1 -- 172.21.15.172:0/2791991075 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55cc11380c00 con 0x55cc11364800 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.287+0000 7f912e3f0640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x55cc11382c00 0x55cc105a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.287+0000 7f912e3f0640 1 -- 172.21.15.172:0/2791991075 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x55cc1136b800 con 0x55cc11364800 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.287+0000 7f9136823740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.287+0000 7f9136823740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.287+0000 7f9136823740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.287+0000 7f9134bfd640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x55cc11382c00 0x55cc105a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.287+0000 7f9136823740 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x55cc11364c00 0x55cc105a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.288+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11365000 con 0x55cc11364c00 2024-04-14T08:40:10.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.288+0000 7f91343fc640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x55cc11364c00 0x55cc105a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.288+0000 7f9134bfd640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x55cc11382c00 0x55cc105a6100 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55cc11226c90 tx=0x55cc111d4180 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.288+0000 7f91343fc640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x55cc11364c00 0x55cc105a6680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.290+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.2 v2:172.21.15.172:6816/1609902500 1 ==== osd_op_reply(1 [pgnls start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11364c00 2024-04-14T08:40:10.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.290+0000 7f912f3f2640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55cc11383000 0x55cc107c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.290+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11383400 con 0x55cc11383000 2024-04-14T08:40:10.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.290+0000 7f9134bfd640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55cc11383000 0x55cc107c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.290+0000 7f9134bfd640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55cc11383000 0x55cc107c8000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.292+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== osd_op_reply(2 [pgnls start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.292+0000 7f912fbf3640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55cc11383c00 0x55cc107c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.292+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d0000 con 0x55cc11383c00 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.292+0000 7f91343fc640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55cc11383c00 0x55cc107c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.293+0000 7f91343fc640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55cc11383c00 0x55cc107c8580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.294+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(3 [pgnls start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11383c00 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.294+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11383800 con 0x55cc11383000 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.294+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 2 ==== osd_op_reply(4 [pgnls start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.295+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d0400 con 0x55cc11383000 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.295+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 3 ==== osd_op_reply(5 [pgnls start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.295+0000 7f912f3f2640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x55cc11383400 0x55cc107c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.295+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11383800 con 0x55cc11383400 2024-04-14T08:40:10.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.295+0000 7f9133bfb640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x55cc11383400 0x55cc107c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.295+0000 7f9133bfb640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x55cc11383400 0x55cc107c8b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.295+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.3 v2:172.21.15.172:6808/2305139122 1 ==== osd_op_reply(6 [pgnls start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0780 con 0x55cc11383400 2024-04-14T08:40:10.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.295+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d0400 con 0x55cc11383c00 2024-04-14T08:40:10.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.296+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.1 v2:172.21.15.172:6823/1219922421 2 ==== osd_op_reply(7 [pgnls start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11383c00 2024-04-14T08:40:10.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.296+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11365c00 con 0x55cc11383c00 2024-04-14T08:40:10.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.296+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.1 v2:172.21.15.172:6823/1219922421 3 ==== osd_op_reply(8 [pgnls start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11383c00 2024-04-14T08:40:10.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.296+0000 7f912fbf3640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x55cc107d0800 0x55cc107c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.296+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d0c00 con 0x55cc107d0800 2024-04-14T08:40:10.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.296+0000 7f9134bfd640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x55cc107d0800 0x55cc107c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.297+0000 7f9134bfd640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x55cc107d0800 0x55cc107c9600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.299+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== osd_op_reply(9 [pgnls start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc107d0800 2024-04-14T08:40:10.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.299+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d1000 con 0x55cc11383c00 2024-04-14T08:40:10.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.300+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.1 v2:172.21.15.172:6823/1219922421 4 ==== osd_op_reply(10 [pgnls start_epoch 90 out=49b] v86'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11383c00 2024-04-14T08:40:10.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.300+0000 7f912fbf3640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x55cc107d0000 0x55cc107c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.300+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d0400 con 0x55cc107d0000 2024-04-14T08:40:10.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.300+0000 7f91343fc640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x55cc107d0000 0x55cc107c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.300+0000 7f91343fc640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x55cc107d0000 0x55cc107c9b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.301+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.4 v2:172.21.15.177:6820/4180733146 1 ==== osd_op_reply(11 [pgnls start_epoch 90 out=49b] v86'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc107d0000 2024-04-14T08:40:10.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.301+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d1400 con 0x55cc11364c00 2024-04-14T08:40:10.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.301+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.2 v2:172.21.15.172:6816/1609902500 2 ==== osd_op_reply(12 [pgnls start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11364c00 2024-04-14T08:40:10.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.301+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11365000 con 0x55cc11383000 2024-04-14T08:40:10.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.301+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 4 ==== osd_op_reply(13 [pgnls start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.301+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d1800 con 0x55cc11383c00 2024-04-14T08:40:10.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.301+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.1 v2:172.21.15.172:6823/1219922421 5 ==== osd_op_reply(14 [pgnls start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11383c00 2024-04-14T08:40:10.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.301+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11365c00 con 0x55cc11383000 2024-04-14T08:40:10.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.302+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 5 ==== osd_op_reply(15 [pgnls start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.302+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d1c00 con 0x55cc11383c00 2024-04-14T08:40:10.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.302+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.1 v2:172.21.15.172:6823/1219922421 6 ==== osd_op_reply(16 [pgnls start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11383c00 2024-04-14T08:40:10.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.302+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d1000 con 0x55cc107d0800 2024-04-14T08:40:10.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.302+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.6 v2:172.21.15.177:6800/2961907928 2 ==== osd_op_reply(17 [pgnls start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc107d0800 2024-04-14T08:40:10.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.302+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e6000 con 0x55cc107d0000 2024-04-14T08:40:10.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.303+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.4 v2:172.21.15.177:6820/4180733146 2 ==== osd_op_reply(18 [pgnls start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc107d0000 2024-04-14T08:40:10.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.303+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e7400 con 0x55cc11364c00 2024-04-14T08:40:10.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.303+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.2 v2:172.21.15.172:6816/1609902500 3 ==== osd_op_reply(19 [pgnls start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11364c00 2024-04-14T08:40:10.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.303+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e6400 con 0x55cc107d0000 2024-04-14T08:40:10.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.303+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.4 v2:172.21.15.177:6820/4180733146 3 ==== osd_op_reply(20 [pgnls start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc107d0000 2024-04-14T08:40:10.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.303+0000 7f912fbf3640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55cc107e7800 0x55cc107ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.303+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e7c00 con 0x55cc107e7800 2024-04-14T08:40:10.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.303+0000 7f9133bfb640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55cc107e7800 0x55cc107ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.304+0000 7f9133bfb640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55cc107e7800 0x55cc107ca100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.306+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.5 v2:172.21.15.177:6816/3172735715 1 ==== osd_op_reply(21 [pgnls start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0780 con 0x55cc107e7800 2024-04-14T08:40:10.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.306+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e6800 con 0x55cc107d0000 2024-04-14T08:40:10.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.306+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.4 v2:172.21.15.177:6820/4180733146 4 ==== osd_op_reply(22 [pgnls start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc107d0000 2024-04-14T08:40:10.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.306+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e6000 con 0x55cc11383000 2024-04-14T08:40:10.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.306+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 6 ==== osd_op_reply(23 [pgnls start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.307+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e6c00 con 0x55cc11383000 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.307+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 7 ==== osd_op_reply(24 [pgnls start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.307+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e7400 con 0x55cc11383000 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.307+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 8 ==== osd_op_reply(25 [pgnls start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.307+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e7000 con 0x55cc107d0000 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.307+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.4 v2:172.21.15.177:6820/4180733146 5 ==== osd_op_reply(26 [pgnls start_epoch 90 out=49b] v86'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc107d0000 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.307+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d0400 con 0x55cc11383400 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.308+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.3 v2:172.21.15.172:6808/2305139122 2 ==== osd_op_reply(27 [pgnls start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0780 con 0x55cc11383400 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.308+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e6000 con 0x55cc11364c00 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.308+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.2 v2:172.21.15.172:6816/1609902500 4 ==== osd_op_reply(28 [pgnls start_epoch 90 out=49b] v86'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11364c00 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.308+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d1400 con 0x55cc107d0000 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.308+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.4 v2:172.21.15.177:6820/4180733146 6 ==== osd_op_reply(29 [pgnls start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc107d0000 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.308+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d0c00 con 0x55cc11383400 2024-04-14T08:40:10.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.308+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.3 v2:172.21.15.172:6808/2305139122 3 ==== osd_op_reply(30 [pgnls start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0780 con 0x55cc11383400 2024-04-14T08:40:10.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.308+0000 7f912fbf3640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55cc107d1800 0x55cc107ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.309+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d1c00 con 0x55cc107d1800 2024-04-14T08:40:10.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.309+0000 7f9134bfd640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55cc107d1800 0x55cc107ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.309+0000 7f9134bfd640 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55cc107d1800 0x55cc107ca680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.309+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== osd_op_reply(31 [pgnls start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc107d1800 2024-04-14T08:40:10.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.310+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d1000 con 0x55cc107e7800 2024-04-14T08:40:10.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.310+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.5 v2:172.21.15.177:6816/3172735715 2 ==== osd_op_reply(32 [pgnls start_epoch 90 out=49b] v86'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0780 con 0x55cc107e7800 2024-04-14T08:40:10.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.310+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d0400 con 0x55cc11383400 2024-04-14T08:40:10.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.310+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.3 v2:172.21.15.172:6808/2305139122 4 ==== osd_op_reply(33 [pgnls start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0780 con 0x55cc11383400 2024-04-14T08:40:10.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.310+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11365000 con 0x55cc107e7800 2024-04-14T08:40:10.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.311+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.5 v2:172.21.15.177:6816/3172735715 3 ==== osd_op_reply(34 [pgnls start_epoch 90 out=49b] v86'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0780 con 0x55cc107e7800 2024-04-14T08:40:10.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.311+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d0c00 con 0x55cc107d0000 2024-04-14T08:40:10.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.311+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.4 v2:172.21.15.177:6820/4180733146 7 ==== osd_op_reply(35 [pgnls start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc107d0000 2024-04-14T08:40:10.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.311+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11365c00 con 0x55cc11383c00 2024-04-14T08:40:10.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.311+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.1 v2:172.21.15.172:6823/1219922421 7 ==== osd_op_reply(36 [pgnls start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11383c00 2024-04-14T08:40:10.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.311+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e7c00 con 0x55cc107e7800 2024-04-14T08:40:10.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.312+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.5 v2:172.21.15.177:6816/3172735715 4 ==== osd_op_reply(37 [pgnls start_epoch 90 out=49b] v86'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0780 con 0x55cc107e7800 2024-04-14T08:40:10.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.312+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11365c00 con 0x55cc107d0800 2024-04-14T08:40:10.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.312+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.6 v2:172.21.15.177:6800/2961907928 3 ==== osd_op_reply(38 [pgnls start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc107d0800 2024-04-14T08:40:10.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.312+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11383800 con 0x55cc107d0800 2024-04-14T08:40:10.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.312+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.6 v2:172.21.15.177:6800/2961907928 4 ==== osd_op_reply(39 [pgnls start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc107d0800 2024-04-14T08:40:10.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.312+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d1400 con 0x55cc107d1800 2024-04-14T08:40:10.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.313+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.7 v2:172.21.15.177:6808/3609985207 2 ==== osd_op_reply(40 [pgnls start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc107d1800 2024-04-14T08:40:10.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.313+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d1c00 con 0x55cc11383000 2024-04-14T08:40:10.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.313+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 9 ==== osd_op_reply(41 [pgnls start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.313+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d0c00 con 0x55cc11364c00 2024-04-14T08:40:10.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.313+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.2 v2:172.21.15.172:6816/1609902500 5 ==== osd_op_reply(42 [pgnls start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11364c00 2024-04-14T08:40:10.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.313+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d1400 con 0x55cc107d1800 2024-04-14T08:40:10.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.313+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.7 v2:172.21.15.177:6808/3609985207 3 ==== osd_op_reply(43 [pgnls start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc107d1800 2024-04-14T08:40:10.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.314+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e6400 con 0x55cc107d0000 2024-04-14T08:40:10.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.314+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.4 v2:172.21.15.177:6820/4180733146 8 ==== osd_op_reply(44 [pgnls start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc107d0000 2024-04-14T08:40:10.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.314+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11383800 con 0x55cc107d0800 2024-04-14T08:40:10.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.314+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.6 v2:172.21.15.177:6800/2961907928 5 ==== osd_op_reply(45 [pgnls start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc107d0800 2024-04-14T08:40:10.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.314+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e6800 con 0x55cc107d0000 2024-04-14T08:40:10.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.314+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.4 v2:172.21.15.177:6820/4180733146 9 ==== osd_op_reply(46 [pgnls start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc107d0000 2024-04-14T08:40:10.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.314+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11365c00 con 0x55cc11383000 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.315+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 10 ==== osd_op_reply(47 [pgnls start_epoch 90 out=49b] v86'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.315+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e7000 con 0x55cc11383400 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.315+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.3 v2:172.21.15.172:6808/2305139122 5 ==== osd_op_reply(48 [pgnls start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0780 con 0x55cc11383400 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.315+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e6c00 con 0x55cc11383000 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.315+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 11 ==== osd_op_reply(49 [pgnls start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.315+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e6000 con 0x55cc107d1800 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.315+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.7 v2:172.21.15.177:6808/3609985207 4 ==== osd_op_reply(50 [pgnls start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc107d1800 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.315+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e7400 con 0x55cc11364c00 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.316+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.2 v2:172.21.15.172:6816/1609902500 6 ==== osd_op_reply(51 [pgnls start_epoch 90 out=49b] v86'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11364c00 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.316+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e7c00 con 0x55cc107e7800 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.316+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.5 v2:172.21.15.177:6816/3172735715 5 ==== osd_op_reply(52 [pgnls start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0780 con 0x55cc107e7800 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.316+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107f6000 con 0x55cc11383000 2024-04-14T08:40:10.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.316+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 12 ==== osd_op_reply(53 [pgnls start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.316+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e7000 con 0x55cc107d0000 2024-04-14T08:40:10.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.317+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.4 v2:172.21.15.177:6820/4180733146 10 ==== osd_op_reply(54 [pgnls start_epoch 90 out=49b] v86'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc107d0000 2024-04-14T08:40:10.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.317+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107f6400 con 0x55cc11383c00 2024-04-14T08:40:10.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.317+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.1 v2:172.21.15.172:6823/1219922421 8 ==== osd_op_reply(55 [pgnls start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11383c00 2024-04-14T08:40:10.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.317+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11365000 con 0x55cc107d0000 2024-04-14T08:40:10.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.317+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.4 v2:172.21.15.177:6820/4180733146 11 ==== osd_op_reply(56 [pgnls start_epoch 90 out=49b] v86'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc107d0000 2024-04-14T08:40:10.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.317+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107f6800 con 0x55cc107e7800 2024-04-14T08:40:10.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.318+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.5 v2:172.21.15.177:6816/3172735715 6 ==== osd_op_reply(57 [pgnls start_epoch 90 out=81b] v86'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55cc111c0780 con 0x55cc107e7800 2024-04-14T08:40:10.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.318+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e6c00 con 0x55cc107e7800 2024-04-14T08:40:10.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.318+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.5 v2:172.21.15.177:6816/3172735715 7 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55cc111c0780 con 0x55cc107e7800 2024-04-14T08:40:10.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.318+0000 7f9136823740 10 datascan.operator(): handling object 1099511627777.0 2024-04-14T08:40:10.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.318+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x55cc107e6000 con 0x55cc107e7800 2024-04-14T08:40:10.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.320+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.5 v2:172.21.15.177:6816/3172735715 8 ==== osd_op_reply(59 10000000001.00000000 [call] v90'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55cc111c0780 con 0x55cc107e7800 2024-04-14T08:40:10.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.320+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc11365c00 con 0x55cc107e7800 2024-04-14T08:40:10.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.320+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.5 v2:172.21.15.177:6816/3172735715 9 ==== osd_op_reply(60 [pgnls start_epoch 90 out=49b] v86'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0780 con 0x55cc107e7800 2024-04-14T08:40:10.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.320+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107f6c00 con 0x55cc11383c00 2024-04-14T08:40:10.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.320+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.1 v2:172.21.15.172:6823/1219922421 9 ==== osd_op_reply(61 [pgnls start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc11383c00 2024-04-14T08:40:10.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.320+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d1000 con 0x55cc107d0000 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.321+0000 7f91343fc640 1 -- 172.21.15.172:0/2791991075 <== osd.4 v2:172.21.15.177:6820/4180733146 12 ==== osd_op_reply(62 [pgnls start_epoch 90 out=49b] v86'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0000 con 0x55cc107d0000 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.321+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107f7000 con 0x55cc107d1800 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.321+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.7 v2:172.21.15.177:6808/3609985207 5 ==== osd_op_reply(63 [pgnls start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc107d1800 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.321+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107d0400 con 0x55cc11383000 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 13 ==== osd_op_reply(64 [pgnls start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f912fbf3640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107f7400 con 0x55cc11383000 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9134bfd640 1 -- 172.21.15.172:0/2791991075 <== osd.0 v2:172.21.15.172:6800/3761711279 14 ==== osd_op_reply(65 [pgnls start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0500 con 0x55cc11383000 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f912f3f2640 1 -- 172.21.15.172:0/2791991075 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55cc107f6400 con 0x55cc11383400 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 <== osd.3 v2:172.21.15.172:6808/2305139122 6 ==== osd_op_reply(66 [pgnls start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55cc111c0780 con 0x55cc11383400 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55cc11383000 msgr2=0x55cc107c8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55cc11383000 0x55cc107c8000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55cc11383c00 msgr2=0x55cc107c8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55cc11383c00 0x55cc107c8580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x55cc11364c00 msgr2=0x55cc105a6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x55cc11364c00 0x55cc105a6680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x55cc11383400 msgr2=0x55cc107c8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x55cc11383400 0x55cc107c8b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 reap_dead start 2024-04-14T08:40:10.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x55cc107d0000 msgr2=0x55cc107c9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x55cc107d0000 0x55cc107c9b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 reap_dead start 2024-04-14T08:40:10.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55cc107e7800 msgr2=0x55cc107ca100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55cc107e7800 0x55cc107ca100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x55cc107d0800 msgr2=0x55cc107c9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x55cc107d0800 0x55cc107c9600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55cc107d1800 msgr2=0x55cc107ca680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55cc107d1800 0x55cc107ca680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x55cc11382c00 msgr2=0x55cc105a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x55cc11382c00 0x55cc105a6100 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55cc11226c90 tx=0x55cc111d4180 comp rx=0 tx=0).stop 2024-04-14T08:40:10.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364800 msgr2=0x55cc105a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9136823740 1 --2- 172.21.15.172:0/2791991075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc11364800 0x55cc105a5b80 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x55cc11227980 tx=0x55cc1122b920 comp rx=0 tx=0).stop 2024-04-14T08:40:10.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.322+0000 7f9133bfb640 1 -- 172.21.15.172:0/2791991075 reap_dead start 2024-04-14T08:40:10.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 shutdown_connections 2024-04-14T08:40:10.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 >> 172.21.15.172:0/2791991075 conn(0x55cc11211c00 msgr2=0x55cc10526800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:10.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 shutdown_connections 2024-04-14T08:40:10.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 -- 172.21.15.172:0/2791991075 wait complete. 2024-04-14T08:40:10.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 -- 172.21.15.172:0/1396632656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc10438c00 msgr2=0x55cc105a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 --2- 172.21.15.172:0/1396632656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc10438c00 0x55cc105a4000 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x55cc11245020 tx=0x55cc111d2a80 comp rx=0 tx=0).stop 2024-04-14T08:40:10.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 -- 172.21.15.172:0/1396632656 shutdown_connections 2024-04-14T08:40:10.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 --2- 172.21.15.172:0/1396632656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cc10439c00 0x55cc105a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 --2- 172.21.15.172:0/1396632656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cc10439800 0x55cc105a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 --2- 172.21.15.172:0/1396632656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cc10438c00 0x55cc105a4000 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 -- 172.21.15.172:0/1396632656 >> 172.21.15.172:0/1396632656 conn(0x55cc10439000 msgr2=0x55cc10527000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:10.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 -- 172.21.15.172:0/1396632656 shutdown_connections 2024-04-14T08:40:10.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.323+0000 7f9136823740 1 -- 172.21.15.172:0/1396632656 wait complete. 2024-04-14T08:40:10.335 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.094257, result: 2024-04-14T08:40:10.336 DEBUG:teuthology.orchestra.run.smithi172:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-14T08:40:10.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.410+0000 7f4624031740 1 -- start start 2024-04-14T08:40:10.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.410+0000 7f4624031740 1 Processor -- start 2024-04-14T08:40:10.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.410+0000 7f4624031740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b789ec00 0x5619b866e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.410+0000 7f4624031740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b789f800 0x5619b866e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.410+0000 7f4624031740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b789fc00 0x5619b866eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.410+0000 7f4624031740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5619b8760180 con 0x5619b789ec00 2024-04-14T08:40:10.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.410+0000 7f4624031740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5619b8760300 con 0x5619b789f800 2024-04-14T08:40:10.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.410+0000 7f4624031740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5619b8760480 con 0x5619b789fc00 2024-04-14T08:40:10.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f46215fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b789ec00 0x5619b866e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f46215fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b789ec00 0x5619b866e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42026/0 (socket says 172.21.15.172:42026) 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f46215fc640 1 -- 172.21.15.172:0/3470129427 learned_addr learned my addr 172.21.15.172:0/3470129427 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f4620dfb640 1 --2- 172.21.15.172:0/3470129427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b789f800 0x5619b866e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f4621dfd640 1 --2- 172.21.15.172:0/3470129427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b789fc00 0x5619b866eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f46215fc640 1 -- 172.21.15.172:0/3470129427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b789fc00 msgr2=0x5619b866eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f46215fc640 1 --2- 172.21.15.172:0/3470129427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b789fc00 0x5619b866eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f46215fc640 1 -- 172.21.15.172:0/3470129427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b789f800 msgr2=0x5619b866e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f46215fc640 1 --2- 172.21.15.172:0/3470129427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b789f800 0x5619b866e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f46215fc640 1 -- 172.21.15.172:0/3470129427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b78cb680 con 0x5619b789ec00 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f46215fc640 1 --2- 172.21.15.172:0/3470129427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b789ec00 0x5619b866e000 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x5619b86bf080 tx=0x5619b8638600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11da96dc0462838f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f461edf7640 1 -- 172.21.15.172:0/3470129427 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5619b78d49c0 con 0x5619b789ec00 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f461edf7640 1 -- 172.21.15.172:0/3470129427 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5619b78eae00 con 0x5619b789ec00 2024-04-14T08:40:10.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f461edf7640 1 -- 172.21.15.172:0/3470129427 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5619b78d44e0 con 0x5619b789ec00 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f4624031740 1 -- 172.21.15.172:0/3470129427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619b78cb4a0 con 0x5619b789ec00 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f4624031740 1 -- 172.21.15.172:0/3470129427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5619b78cba40 con 0x5619b789ec00 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f4624031740 4 waiting for MDS map... 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f461edf7640 1 -- 172.21.15.172:0/3470129427 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x5619b8782600 con 0x5619b789ec00 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f461edf7640 1 -- 172.21.15.172:0/3470129427 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x5619b78f9180 con 0x5619b789ec00 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.411+0000 7f4624031740 4 Got MDS map 62 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.412+0000 7f4624031740 4 datascan.main: Using metadata pool output 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.412+0000 7f4624031740 4 datascan.main: connecting to RADOS... 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.412+0000 7f4624031740 1 Processor -- start 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.412+0000 7f4624031740 1 -- start start 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f4624031740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b87ce000 0x5619b866f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f4624031740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 0x5619b866f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f4624031740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b87ce800 0x5619b866fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f4624031740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5619b8760600 con 0x5619b87ce400 2024-04-14T08:40:10.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f4624031740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5619b8760780 con 0x5619b87ce000 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f4624031740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5619b8760900 con 0x5619b87ce800 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f46215fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 0x5619b866f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f46215fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 0x5619b866f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42032/0 (socket says 172.21.15.172:42032) 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f46215fc640 1 -- 172.21.15.172:0/1634638672 learned_addr learned my addr 172.21.15.172:0/1634638672 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f4620dfb640 1 --2- 172.21.15.172:0/1634638672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b87ce800 0x5619b866fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f4621dfd640 1 --2- 172.21.15.172:0/1634638672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b87ce000 0x5619b866f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f46215fc640 1 -- 172.21.15.172:0/1634638672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b87ce800 msgr2=0x5619b866fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f46215fc640 1 --2- 172.21.15.172:0/1634638672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b87ce800 0x5619b866fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f46215fc640 1 -- 172.21.15.172:0/1634638672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b87ce000 msgr2=0x5619b866f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f46215fc640 1 --2- 172.21.15.172:0/1634638672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b87ce000 0x5619b866f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f46215fc640 1 -- 172.21.15.172:0/1634638672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b78cb860 con 0x5619b87ce400 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f4621dfd640 1 --2- 172.21.15.172:0/1634638672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b87ce000 0x5619b866f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f46215fc640 1 --2- 172.21.15.172:0/1634638672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 0x5619b866f600 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x5619b86bfdd0 tx=0x5619b8638a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88cbf125e7983ef server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f461cdf3640 1 -- 172.21.15.172:0/1634638672 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5619b78d4b60 con 0x5619b87ce400 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f461cdf3640 1 -- 172.21.15.172:0/1634638672 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5619b78eac40 con 0x5619b87ce400 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.413+0000 7f461cdf3640 1 -- 172.21.15.172:0/1634638672 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5619b78d4340 con 0x5619b87ce400 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.414+0000 7f4624031740 1 -- 172.21.15.172:0/1634638672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 msgr2=0x5619b866f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.414+0000 7f4624031740 1 --2- 172.21.15.172:0/1634638672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 0x5619b866f600 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x5619b86bfdd0 tx=0x5619b8638a80 comp rx=0 tx=0).stop 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.414+0000 7f4624031740 1 -- 172.21.15.172:0/1634638672 shutdown_connections 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.414+0000 7f4624031740 1 --2- 172.21.15.172:0/1634638672 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b87ce800 0x5619b866fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.414+0000 7f4624031740 1 --2- 172.21.15.172:0/1634638672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 0x5619b866f600 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.414+0000 7f4624031740 1 --2- 172.21.15.172:0/1634638672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b87ce000 0x5619b866f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.414+0000 7f4624031740 1 -- 172.21.15.172:0/1634638672 >> 172.21.15.172:0/1634638672 conn(0x5619b867fc00 msgr2=0x5619b7983000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.414+0000 7f4624031740 1 -- 172.21.15.172:0/1634638672 shutdown_connections 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.414+0000 7f4624031740 1 -- 172.21.15.172:0/1634638672 wait complete. 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.414+0000 7f4624031740 1 Processor -- start 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.414+0000 7f4624031740 1 -- start start 2024-04-14T08:40:10.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f4624031740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b87ce000 0x5619b866f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f4624031740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 0x5619b866f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f4624031740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b87ce800 0x5619b866fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f4624031740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5619b8761500 con 0x5619b87ce400 2024-04-14T08:40:10.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f4624031740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5619b8760a80 con 0x5619b87ce800 2024-04-14T08:40:10.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f4624031740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5619b8761680 con 0x5619b87ce000 2024-04-14T08:40:10.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f46215fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 0x5619b866f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f46215fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 0x5619b866f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42038/0 (socket says 172.21.15.172:42038) 2024-04-14T08:40:10.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 learned_addr learned my addr 172.21.15.172:0/203885123 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:10.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f4621dfd640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b87ce000 0x5619b866f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f4620dfb640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b87ce800 0x5619b866fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b87ce000 msgr2=0x5619b866f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f46215fc640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b87ce000 0x5619b866f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b87ce800 msgr2=0x5619b866fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f46215fc640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b87ce800 0x5619b866fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b78cb2c0 con 0x5619b87ce400 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f4621dfd640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b87ce000 0x5619b866f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f46215fc640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 0x5619b866f600 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x5619b86a2f90 tx=0x5619b863aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f461b5f0640 1 -- 172.21.15.172:0/203885123 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5619b78d4b60 con 0x5619b87ce400 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f461b5f0640 1 -- 172.21.15.172:0/203885123 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5619b78eaa80 con 0x5619b87ce400 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5619b78cb0e0 con 0x5619b87ce400 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f461b5f0640 1 -- 172.21.15.172:0/203885123 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5619b78d4340 con 0x5619b87ce400 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619b78ca3c0 con 0x5619b87ce400 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.415+0000 7f4624031740 4 datascan.init: resolving metadata pool 13 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.417+0000 7f461b5f0640 1 -- 172.21.15.172:0/203885123 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5619b87dec00 con 0x5619b87ce400 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.417+0000 7f461b5f0640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5619b7c12000 0x5619b8670100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.417+0000 7f461b5f0640 1 -- 172.21.15.172:0/203885123 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x5619b87d4c00 con 0x5619b87ce400 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.417+0000 7f4621dfd640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5619b7c12000 0x5619b8670100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.417+0000 7f4624031740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.417+0000 7f4624031740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.417+0000 7f4624031740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.417+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5619b87cec00 0x5619b8670680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.417+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5619b87cf000 con 0x5619b87cec00 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.417+0000 7f4620dfb640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5619b87cec00 0x5619b8670680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.417+0000 7f4621dfd640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5619b7c12000 0x5619b8670100 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5619b8626de0 tx=0x5619b7ba4a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.418+0000 7f4620dfb640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5619b87cec00 0x5619b8670680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.418+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.418+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5619b7c13000 0x5619b8670c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.418+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c13400 con 0x5619b7c13000 2024-04-14T08:40:10.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.418+0000 7f4621dfd640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5619b7c13000 0x5619b8670c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.419+0000 7f4621dfd640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5619b7c13000 0x5619b8670c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.419+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c13000 2024-04-14T08:40:10.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.419+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x5619b7c13800 0x5619b8671180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.419+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c13c00 con 0x5619b7c13800 2024-04-14T08:40:10.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.419+0000 7f46215fc640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x5619b7c13800 0x5619b8671180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.419+0000 7f46215fc640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x5619b7c13800 0x5619b8671180 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.420+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.2 v2:172.21.15.172:6816/1609902500 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c13800 2024-04-14T08:40:10.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.420+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c3e000 con 0x5619b87cec00 2024-04-14T08:40:10.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.420+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.420+0000 7f461cdf3640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5619b87cf000 0x5619b7c44000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.420+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48000 con 0x5619b87cf000 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.420+0000 7f4620dfb640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5619b87cf000 0x5619b7c44000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.421+0000 7f4620dfb640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5619b87cf000 0x5619b7c44000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.421+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.421+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48800 con 0x5619b87cec00 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.421+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.421+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48400 con 0x5619b87cec00 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.421+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.421+0000 7f461c5f2640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x5619b7c48c00 0x5619b7c44580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.421+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48000 con 0x5619b7c48c00 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.421+0000 7f4621dfd640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x5619b7c48c00 0x5619b7c44580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.422+0000 7f4621dfd640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x5619b7c48c00 0x5619b7c44580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.422+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.3 v2:172.21.15.172:6808/2305139122 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c48c00 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.422+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49000 con 0x5619b87cf000 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.422+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.422+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48800 con 0x5619b87cf000 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.422+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.422+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49400 con 0x5619b7c13000 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.423+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.6 v2:172.21.15.177:6800/2961907928 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c13000 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.423+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48400 con 0x5619b87cf000 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.423+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 90 out=49b] v86'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.423+0000 7f461cdf3640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5619b7c49800 0x5619b7c44b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.423+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48000 con 0x5619b7c49800 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.423+0000 7f46215fc640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5619b7c49800 0x5619b7c44b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.424+0000 7f46215fc640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5619b7c49800 0x5619b7c44b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.424+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 90 out=49b] v86'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.424+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49c00 con 0x5619b7c13800 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.424+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.2 v2:172.21.15.172:6816/1609902500 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c13800 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.424+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49400 con 0x5619b87cec00 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.425+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.425+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49000 con 0x5619b87cf000 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.425+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 5 ==== osd_op_reply(16 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.425+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c12c00 con 0x5619b87cec00 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.425+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.425+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48800 con 0x5619b87cf000 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.425+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.425+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c13400 con 0x5619b7c13000 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.426+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.6 v2:172.21.15.177:6800/2961907928 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c13000 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.426+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48400 con 0x5619b7c49800 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.426+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.426+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54000 con 0x5619b7c13800 2024-04-14T08:40:10.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.426+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.2 v2:172.21.15.172:6816/1609902500 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c13800 2024-04-14T08:40:10.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.426+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49400 con 0x5619b7c49800 2024-04-14T08:40:10.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.427+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.427+0000 7f461cdf3640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x5619b7c54400 0x5619b7c45080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.427+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54800 con 0x5619b7c54400 2024-04-14T08:40:10.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.427+0000 7f4620dfb640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x5619b7c54400 0x5619b7c45080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.427+0000 7f4620dfb640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x5619b7c54400 0x5619b7c45080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.428+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.428+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54000 con 0x5619b7c49800 2024-04-14T08:40:10.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.428+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.428+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54c00 con 0x5619b87cec00 2024-04-14T08:40:10.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.428+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.428+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48000 con 0x5619b87cec00 2024-04-14T08:40:10.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.428+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.428+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c55000 con 0x5619b87cec00 2024-04-14T08:40:10.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.429+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.429+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49c00 con 0x5619b7c49800 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.429+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 90 out=49b] v86'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.429+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54800 con 0x5619b7c48c00 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.429+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.3 v2:172.21.15.172:6808/2305139122 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c48c00 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.429+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48400 con 0x5619b7c13800 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.430+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.2 v2:172.21.15.172:6816/1609902500 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 90 out=49b] v86'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c13800 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.430+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54c00 con 0x5619b7c49800 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.430+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.430+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49400 con 0x5619b7c48c00 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.430+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.3 v2:172.21.15.172:6808/2305139122 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c48c00 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.430+0000 7f461cdf3640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5619b7c55400 0x5619b7c45600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.430+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49000 con 0x5619b7c55400 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.431+0000 7f4621dfd640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5619b7c55400 0x5619b7c45600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.431+0000 7f4621dfd640 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5619b7c55400 0x5619b7c45600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.431+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c55400 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.431+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c13c00 con 0x5619b7c54400 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.432+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 90 out=49b] v86'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.432+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48800 con 0x5619b7c48c00 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.432+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.3 v2:172.21.15.172:6808/2305139122 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c48c00 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.432+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54800 con 0x5619b7c54400 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.432+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 90 out=49b] v86'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.432+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c12c00 con 0x5619b7c49800 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.432+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.432+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c55800 con 0x5619b87cf000 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.433+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 7 ==== osd_op_reply(38 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.433+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60000 con 0x5619b7c54400 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.433+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 4 ==== osd_op_reply(39 [pgnls-filter start_epoch 90 out=49b] v86'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.433+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c55c00 con 0x5619b7c13000 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.433+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.6 v2:172.21.15.177:6800/2961907928 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c13000 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.433+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60400 con 0x5619b7c13000 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.434+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.6 v2:172.21.15.177:6800/2961907928 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c13000 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.434+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49400 con 0x5619b7c55400 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.434+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.7 v2:172.21.15.177:6808/3609985207 2 ==== osd_op_reply(42 [pgnls-filter start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c55400 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.434+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60800 con 0x5619b87cec00 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.434+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.434+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49000 con 0x5619b7c13800 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.435+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.2 v2:172.21.15.172:6816/1609902500 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c13800 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.435+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60c00 con 0x5619b7c55400 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.435+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.7 v2:172.21.15.177:6808/3609985207 3 ==== osd_op_reply(45 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c55400 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.435+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48800 con 0x5619b7c49800 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.435+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.435+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c61000 con 0x5619b7c13000 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.436+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.6 v2:172.21.15.177:6800/2961907928 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c13000 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.436+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c13400 con 0x5619b7c49800 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.436+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.436+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c61400 con 0x5619b87cec00 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.436+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 90 out=49b] v86'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.436+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54000 con 0x5619b7c48c00 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.437+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.3 v2:172.21.15.172:6808/2305139122 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c48c00 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.437+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c61800 con 0x5619b87cec00 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.437+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.437+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c12c00 con 0x5619b7c55400 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.437+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.7 v2:172.21.15.177:6808/3609985207 4 ==== osd_op_reply(52 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c55400 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.437+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60400 con 0x5619b7c13800 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.438+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.2 v2:172.21.15.172:6816/1609902500 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 90 out=49b] v86'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c13800 2024-04-14T08:40:10.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.438+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c13c00 con 0x5619b7c54400 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.438+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 5 ==== osd_op_reply(54 [pgnls-filter start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.438+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c55c00 con 0x5619b87cec00 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.438+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.438+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49c00 con 0x5619b7c49800 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.438+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 90 out=49b] v86'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.439+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54000 con 0x5619b87cf000 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.439+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 8 ==== osd_op_reply(57 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.439+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48400 con 0x5619b7c49800 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.439+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 90 out=49b] v86'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.439+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 90 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54c00 con 0x5619b7c54400 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.439+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 6 ==== osd_op_reply(59 [pgnls-filter start_epoch 90 out=81b] v90'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.440+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60800 con 0x5619b7c13800 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.440+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.2 v2:172.21.15.172:6816/1609902500 7 ==== osd_op_reply(60 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c13800 2024-04-14T08:40:10.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.440+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c55000 con 0x5619b87cec00 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.440+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.440+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49000 con 0x5619b87cf000 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.440+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.440+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54800 con 0x5619b87cec00 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.441+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.441+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48800 con 0x5619b87cec00 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.441+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.441+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c55800 con 0x5619b7c48c00 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.441+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.3 v2:172.21.15.172:6808/2305139122 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c48c00 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.441+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48000 con 0x5619b87cf000 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.441+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 10 ==== osd_op_reply(66 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.441+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49400 con 0x5619b87cf000 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.442+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 11 ==== osd_op_reply(67 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.442+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60c00 con 0x5619b7c13000 2024-04-14T08:40:10.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.442+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.6 v2:172.21.15.177:6800/2961907928 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c13000 2024-04-14T08:40:10.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.442+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c62000 con 0x5619b87cf000 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.443+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 90 out=49b] v86'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.443+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c61000 con 0x5619b7c49800 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.443+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 90 out=49b] v86'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.443+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c62400 con 0x5619b7c13800 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.444+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.2 v2:172.21.15.172:6816/1609902500 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c13800 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.444+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60000 con 0x5619b87cec00 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.444+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.444+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c62800 con 0x5619b87cf000 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.444+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 13 ==== osd_op_reply(73 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.444+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c61c00 con 0x5619b87cec00 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.444+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.444+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c62c00 con 0x5619b87cf000 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.445+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.445+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60000 con 0x5619b7c13000 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.445+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.6 v2:172.21.15.177:6800/2961907928 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c13000 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.445+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c63000 con 0x5619b7c49800 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.445+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.445+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c62000 con 0x5619b7c13800 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.446+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.2 v2:172.21.15.172:6816/1609902500 9 ==== osd_op_reply(78 [pgnls-filter start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c13800 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.446+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c63400 con 0x5619b7c49800 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.446+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.446+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49000 con 0x5619b7c54400 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.446+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 7 ==== osd_op_reply(80 [pgnls-filter start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.446+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c63800 con 0x5619b7c49800 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.447+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.447+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48800 con 0x5619b87cec00 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.447+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.447+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c63c00 con 0x5619b87cec00 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.447+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.447+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48000 con 0x5619b87cec00 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.448+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.448+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c62400 con 0x5619b7c49800 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.448+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 90 out=49b] v86'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.448+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49400 con 0x5619b7c48c00 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.448+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.3 v2:172.21.15.172:6808/2305139122 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c48c00 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.448+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c63000 con 0x5619b7c13800 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.448+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.2 v2:172.21.15.172:6816/1609902500 10 ==== osd_op_reply(87 [pgnls-filter start_epoch 90 out=49b] v86'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c13800 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.448+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c55000 con 0x5619b7c49800 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.449+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.449+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60400 con 0x5619b7c48c00 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.449+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.3 v2:172.21.15.172:6808/2305139122 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c48c00 2024-04-14T08:40:10.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.449+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54800 con 0x5619b7c55400 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.449+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.7 v2:172.21.15.177:6808/3609985207 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c55400 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.449+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60800 con 0x5619b7c54400 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.450+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 8 ==== osd_op_reply(91 [pgnls-filter start_epoch 90 out=49b] v86'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.450+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c55800 con 0x5619b7c48c00 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.450+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.3 v2:172.21.15.172:6808/2305139122 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c48c00 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.450+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c61000 con 0x5619b7c54400 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.450+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 9 ==== osd_op_reply(93 [pgnls-filter start_epoch 90 out=49b] v86'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.450+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54800 con 0x5619b7c49800 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.451+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.451+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49c00 con 0x5619b87cf000 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.451+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 15 ==== osd_op_reply(95 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.451+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60c00 con 0x5619b7c54400 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.451+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 90 out=49b] v86'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.451+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48400 con 0x5619b7c13000 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.452+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.6 v2:172.21.15.177:6800/2961907928 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c13000 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.452+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60000 con 0x5619b7c13000 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.452+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.6 v2:172.21.15.177:6800/2961907928 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c13000 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.452+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c13400 con 0x5619b7c55400 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.452+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.7 v2:172.21.15.177:6808/3609985207 6 ==== osd_op_reply(99 [pgnls-filter start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c55400 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.452+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60400 con 0x5619b87cec00 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.452+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 90 out=49b] v86'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.453+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c61c00 con 0x5619b7c13800 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.453+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.2 v2:172.21.15.172:6816/1609902500 11 ==== osd_op_reply(101 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c13800 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.453+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49400 con 0x5619b7c55400 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.453+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.7 v2:172.21.15.177:6808/3609985207 7 ==== osd_op_reply(102 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c55400 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.453+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60800 con 0x5619b7c49800 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.453+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.453+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c12c00 con 0x5619b7c13000 2024-04-14T08:40:10.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.454+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.6 v2:172.21.15.177:6800/2961907928 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c13000 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.454+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c61000 con 0x5619b7c49800 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.454+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.454+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c61c00 con 0x5619b87cec00 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.455+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 90 out=49b] v86'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.455+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49000 con 0x5619b7c48c00 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.455+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.3 v2:172.21.15.172:6808/2305139122 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c48c00 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.455+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60800 con 0x5619b87cec00 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.455+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.455+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48800 con 0x5619b7c55400 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.455+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.7 v2:172.21.15.177:6808/3609985207 8 ==== osd_op_reply(109 [pgnls-filter start_epoch 90 out=49b] v86'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c55400 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.455+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c55000 con 0x5619b7c13800 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.456+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.2 v2:172.21.15.172:6816/1609902500 12 ==== osd_op_reply(110 [pgnls-filter start_epoch 90 out=49b] v86'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c13800 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.456+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48000 con 0x5619b7c54400 2024-04-14T08:40:10.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.456+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 11 ==== osd_op_reply(111 [pgnls-filter start_epoch 90 out=49b] v86'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.456+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54800 con 0x5619b87cec00 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.456+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.456+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c62800 con 0x5619b7c49800 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.457+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 90 out=49b] v86'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.457+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c62000 con 0x5619b87cf000 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.457+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 16 ==== osd_op_reply(114 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.457+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c62c00 con 0x5619b7c49800 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.457+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 90 out=49b] v86'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.457+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c63400 con 0x5619b7c54400 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.458+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 12 ==== osd_op_reply(116 [pgnls-filter start_epoch 90 out=81b] v90'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.458+0000 7f4624031740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.458+0000 7f4624031740 10 datascan.operator(): handling object 10000000001.0 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.458+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c61800 con 0x5619b7c54400 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.458+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 13 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.458+0000 7f4624031740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.458+0000 7f4624031740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.458+0000 7f4624031740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.458+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c55c00 con 0x5619b87cf000 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.458+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 17 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.458+0000 7f4624031740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.458+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54000 con 0x5619b7c13000 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.6 v2:172.21.15.177:6800/2961907928 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) ==== 154+0+581 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c13000 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4624031740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4624031740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4624031740 20 datascan.get_frag_of: frag is * 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c54c00 con 0x5619b87cf000 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 18 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4624031740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-14T08:40:10.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4624031740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c13c00 con 0x5619b87cf000 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 19 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4624031740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4624031740 20 datascan.read_dentry: file_head not found in result 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4624031740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.459+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60c00 con 0x5619b87cf000 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.461+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 20 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v90'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.461+0000 7f4624031740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.461+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c60400 con 0x5619b7c54400 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.461+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.5 v2:172.21.15.177:6816/3172735715 14 ==== osd_op_reply(123 [pgnls-filter start_epoch 90 out=49b] v86'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b7c54400 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.461+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c63800 con 0x5619b87cf000 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.461+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.1 v2:172.21.15.172:6823/1219922421 21 ==== osd_op_reply(124 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cf000 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.461+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c63c00 con 0x5619b7c49800 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.461+0000 7f46215fc640 1 -- 172.21.15.172:0/203885123 <== osd.4 v2:172.21.15.177:6820/4180733146 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 90 out=49b] v86'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620780 con 0x5619b7c49800 2024-04-14T08:40:10.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.461+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c62400 con 0x5619b7c55400 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.462+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.7 v2:172.21.15.177:6808/3609985207 9 ==== osd_op_reply(126 [pgnls-filter start_epoch 90 out=49b] v86'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c55400 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.462+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c49c00 con 0x5619b87cec00 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.462+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.462+0000 7f461c5f2640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c63000 con 0x5619b87cec00 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.462+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 <== osd.0 v2:172.21.15.172:6800/3761711279 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 90 out=49b] v86'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620000 con 0x5619b87cec00 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.462+0000 7f461cdf3640 1 -- 172.21.15.172:0/203885123 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 90 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x5619b7c48000 con 0x5619b7c48c00 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4621dfd640 1 -- 172.21.15.172:0/203885123 <== osd.3 v2:172.21.15.172:6808/2305139122 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 90 out=49b] v86'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5619b8620500 con 0x5619b7c48c00 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5619b87cec00 msgr2=0x5619b8670680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5619b87cec00 0x5619b8670680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5619b87cf000 msgr2=0x5619b7c44000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x5619b87cf000 0x5619b7c44000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x5619b7c13800 msgr2=0x5619b8671180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x5619b7c13800 0x5619b8671180 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x5619b7c48c00 msgr2=0x5619b7c44580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x5619b7c48c00 0x5619b7c44580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5619b7c49800 msgr2=0x5619b7c44b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5619b7c49800 0x5619b7c44b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x5619b7c54400 msgr2=0x5619b7c45080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x5619b7c54400 0x5619b7c45080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5619b7c13000 msgr2=0x5619b8670c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5619b7c13000 0x5619b8670c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5619b7c55400 msgr2=0x5619b7c45600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 reap_dead start 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5619b7c55400 0x5619b7c45600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 reap_dead start 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4620dfb640 1 -- 172.21.15.172:0/203885123 reap_dead start 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5619b7c12000 msgr2=0x5619b8670100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5619b7c12000 0x5619b8670100 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5619b8626de0 tx=0x5619b7ba4a80 comp rx=0 tx=0).stop 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 msgr2=0x5619b866f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 0x5619b866f600 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x5619b86a2f90 tx=0x5619b863aae0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 shutdown_connections 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5619b7c12000 0x5619b8670100 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/203885123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b87ce400 0x5619b866f600 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 >> 172.21.15.172:0/203885123 conn(0x5619b867fc00 msgr2=0x5619b7983000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 shutdown_connections 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/203885123 wait complete. 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/3470129427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b789ec00 msgr2=0x5619b866e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/3470129427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b789ec00 0x5619b866e000 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x5619b86bf080 tx=0x5619b8638600 comp rx=0 tx=0).stop 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/3470129427 shutdown_connections 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/3470129427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5619b789fc00 0x5619b866eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/3470129427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619b789f800 0x5619b866e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 --2- 172.21.15.172:0/3470129427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619b789ec00 0x5619b866e000 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/3470129427 >> 172.21.15.172:0/3470129427 conn(0x5619b789f000 msgr2=0x5619b7982800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/3470129427 shutdown_connections 2024-04-14T08:40:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.463+0000 7f4624031740 1 -- 172.21.15.172:0/3470129427 wait complete. 2024-04-14T08:40:10.484 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.148201, result: 2024-04-14T08:40:10.484 DEBUG:teuthology.orchestra.run.smithi172:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-14T08:40:10.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.550+0000 7fa342a17740 1 -- start start 2024-04-14T08:40:10.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.550+0000 7fa342a17740 1 Processor -- start 2024-04-14T08:40:10.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa342a17740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c74ae800 0x55f4c6850b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa342a17740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c74afc00 0x55f4c6851b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa342a17740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c74af400 0x55f4c6851080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa342a17740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f4c74c6480 con 0x55f4c74af400 2024-04-14T08:40:10.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa342a17740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f4c74c6600 con 0x55f4c74ae800 2024-04-14T08:40:10.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa342a17740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f4c74c6780 con 0x55f4c74afc00 2024-04-14T08:40:10.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa340bfd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c74af400 0x55f4c6851080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa340bfd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c74af400 0x55f4c6851080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42054/0 (socket says 172.21.15.172:42054) 2024-04-14T08:40:10.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa340bfd640 1 -- 172.21.15.172:0/32654052 learned_addr learned my addr 172.21.15.172:0/32654052 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:10.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa3403fc640 1 --2- 172.21.15.172:0/32654052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c74ae800 0x55f4c6850b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/32654052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c74afc00 0x55f4c6851b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa340bfd640 1 -- 172.21.15.172:0/32654052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c74afc00 msgr2=0x55f4c6851b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa340bfd640 1 --2- 172.21.15.172:0/32654052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c74afc00 0x55f4c6851b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa340bfd640 1 -- 172.21.15.172:0/32654052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c74ae800 msgr2=0x55f4c6850b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa340bfd640 1 --2- 172.21.15.172:0/32654052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c74ae800 0x55f4c6850b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa340bfd640 1 -- 172.21.15.172:0/32654052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c6711680 con 0x55f4c74af400 2024-04-14T08:40:10.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa3403fc640 1 --2- 172.21.15.172:0/32654052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c74ae800 0x55f4c6850b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:10.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.551+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/32654052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c74afc00 0x55f4c6851b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:10.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.552+0000 7fa340bfd640 1 --2- 172.21.15.172:0/32654052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c74af400 0x55f4c6851080 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x55f4c67da480 tx=0x55f4c7480120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec09b54eb2995871 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.552+0000 7fa33dbf7640 1 -- 172.21.15.172:0/32654052 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f4c671ab60 con 0x55f4c74af400 2024-04-14T08:40:10.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.552+0000 7fa33dbf7640 1 -- 172.21.15.172:0/32654052 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f4c67301c0 con 0x55f4c74af400 2024-04-14T08:40:10.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.552+0000 7fa33dbf7640 1 -- 172.21.15.172:0/32654052 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f4c671a1a0 con 0x55f4c74af400 2024-04-14T08:40:10.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.552+0000 7fa342a17740 1 -- 172.21.15.172:0/32654052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c6710b40 con 0x55f4c74af400 2024-04-14T08:40:10.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.552+0000 7fa342a17740 1 -- 172.21.15.172:0/32654052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55f4c67103c0 con 0x55f4c74af400 2024-04-14T08:40:10.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.552+0000 7fa342a17740 4 waiting for MDS map... 2024-04-14T08:40:10.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.552+0000 7fa33dbf7640 1 -- 172.21.15.172:0/32654052 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x55f4c74a8400 con 0x55f4c74af400 2024-04-14T08:40:10.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.552+0000 7fa33dbf7640 1 -- 172.21.15.172:0/32654052 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x55f4c673f180 con 0x55f4c74af400 2024-04-14T08:40:10.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.552+0000 7fa342a17740 4 Got MDS map 62 2024-04-14T08:40:10.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.552+0000 7fa342a17740 4 datascan.main: Using metadata pool output 2024-04-14T08:40:10.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.552+0000 7fa342a17740 4 datascan.main: connecting to RADOS... 2024-04-14T08:40:10.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa342a17740 1 Processor -- start 2024-04-14T08:40:10.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa342a17740 1 -- start start 2024-04-14T08:40:10.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa342a17740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c75be400 0x55f4c6850580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa342a17740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c75be800 0x55f4c6852100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa342a17740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75bec00 0x55f4c6852680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa342a17740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f4c670bb00 con 0x55f4c75bec00 2024-04-14T08:40:10.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa342a17740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f4c74c6f00 con 0x55f4c75be400 2024-04-14T08:40:10.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa342a17740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f4c74c7080 con 0x55f4c75be800 2024-04-14T08:40:10.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa33fbfb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75bec00 0x55f4c6852680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa33fbfb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75bec00 0x55f4c6852680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42060/0 (socket says 172.21.15.172:42060) 2024-04-14T08:40:10.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3708335006 learned_addr learned my addr 172.21.15.172:0/3708335006 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:10.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa3403fc640 1 --2- 172.21.15.172:0/3708335006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c75be800 0x55f4c6852100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.553+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3708335006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c75be400 0x55f4c6850580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3708335006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c75be800 msgr2=0x55f4c6852100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3708335006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c75be800 0x55f4c6852100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3708335006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c75be400 msgr2=0x55f4c6850580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3708335006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c75be400 0x55f4c6850580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3708335006 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c67114a0 con 0x55f4c75bec00 2024-04-14T08:40:10.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3708335006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75bec00 0x55f4c6852680 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x55f4c75baea0 tx=0x55f4c75c38c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf9578501df2b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa33bbf3640 1 -- 172.21.15.172:0/3708335006 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f4c671ab60 con 0x55f4c75bec00 2024-04-14T08:40:10.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa33bbf3640 1 -- 172.21.15.172:0/3708335006 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f4c6730c40 con 0x55f4c75bec00 2024-04-14T08:40:10.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa33bbf3640 1 -- 172.21.15.172:0/3708335006 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f4c671a1a0 con 0x55f4c75bec00 2024-04-14T08:40:10.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa342a17740 1 -- 172.21.15.172:0/3708335006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75bec00 msgr2=0x55f4c6852680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa342a17740 1 --2- 172.21.15.172:0/3708335006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75bec00 0x55f4c6852680 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x55f4c75baea0 tx=0x55f4c75c38c0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa342a17740 1 -- 172.21.15.172:0/3708335006 shutdown_connections 2024-04-14T08:40:10.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa342a17740 1 --2- 172.21.15.172:0/3708335006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75bec00 0x55f4c6852680 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa342a17740 1 --2- 172.21.15.172:0/3708335006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c75be800 0x55f4c6852100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa342a17740 1 --2- 172.21.15.172:0/3708335006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c75be400 0x55f4c6850580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa342a17740 1 -- 172.21.15.172:0/3708335006 >> 172.21.15.172:0/3708335006 conn(0x55f4c74af800 msgr2=0x55f4c67d5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:10.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa342a17740 1 -- 172.21.15.172:0/3708335006 shutdown_connections 2024-04-14T08:40:10.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.554+0000 7fa342a17740 1 -- 172.21.15.172:0/3708335006 wait complete. 2024-04-14T08:40:10.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.555+0000 7fa342a17740 1 Processor -- start 2024-04-14T08:40:10.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.555+0000 7fa342a17740 1 -- start start 2024-04-14T08:40:10.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.555+0000 7fa342a17740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c75be400 0x55f4c6850580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.555+0000 7fa342a17740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75be800 0x55f4c6852100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.555+0000 7fa342a17740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c75bec00 0x55f4c6852680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.555+0000 7fa342a17740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f4c74c7380 con 0x55f4c75be800 2024-04-14T08:40:10.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.555+0000 7fa342a17740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f4c74c7200 con 0x55f4c75bec00 2024-04-14T08:40:10.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.555+0000 7fa342a17740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f4c74c7500 con 0x55f4c75be400 2024-04-14T08:40:10.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.555+0000 7fa3403fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75be800 0x55f4c6852100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.555+0000 7fa3403fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75be800 0x55f4c6852100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42076/0 (socket says 172.21.15.172:42076) 2024-04-14T08:40:10.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 learned_addr learned my addr 172.21.15.172:0/3086526642 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:10.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c75be400 0x55f4c6850580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c75bec00 0x55f4c6852680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c75be400 msgr2=0x55f4c6850580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa3403fc640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c75be400 0x55f4c6850580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c75bec00 msgr2=0x55f4c6852680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa3403fc640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c75bec00 0x55f4c6852680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c6711680 con 0x55f4c75be800 2024-04-14T08:40:10.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa3403fc640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75be800 0x55f4c6852100 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x55f4c75bbb90 tx=0x55f4c75c3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa33a3f0640 1 -- 172.21.15.172:0/3086526642 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f4c671ab60 con 0x55f4c75be800 2024-04-14T08:40:10.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa33a3f0640 1 -- 172.21.15.172:0/3086526642 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f4c6731340 con 0x55f4c75be800 2024-04-14T08:40:10.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f4c67112c0 con 0x55f4c75be800 2024-04-14T08:40:10.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa33a3f0640 1 -- 172.21.15.172:0/3086526642 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f4c671a1a0 con 0x55f4c75be800 2024-04-14T08:40:10.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c6711a40 con 0x55f4c75be800 2024-04-14T08:40:10.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.556+0000 7fa342a17740 4 datascan.init: resolving metadata pool 13 2024-04-14T08:40:10.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.557+0000 7fa33a3f0640 1 -- 172.21.15.172:0/3086526642 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55f4c7632c00 con 0x55f4c75be800 2024-04-14T08:40:10.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.557+0000 7fa33a3f0640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x55f4c7646c00 0x55f4c6853700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.557+0000 7fa33a3f0640 1 -- 172.21.15.172:0/3086526642 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x55f4c7630c00 con 0x55f4c75be800 2024-04-14T08:40:10.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.558+0000 7fa342a17740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-14T08:40:10.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.558+0000 7fa342a17740 4 datascan.main: resolving metadata pool 13 2024-04-14T08:40:10.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.558+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x55f4c7646c00 0x55f4c6853700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.558+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55f4c75bf000 0x55f4c6852c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.558+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c75bf400 con 0x55f4c75bf000 2024-04-14T08:40:10.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.558+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55f4c75bf000 0x55f4c6852c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.558+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x55f4c7646c00 0x55f4c6853700 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55f4c75baff0 tx=0x55f4c75c3800 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.558+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55f4c75bf000 0x55f4c6852c00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.559+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== osd_op_reply(1 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.559+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55f4c75bf400 0x55f4c6a7e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.559+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c7647000 con 0x55f4c75bf400 2024-04-14T08:40:10.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.559+0000 7fa340bfd640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55f4c75bf400 0x55f4c6a7e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.559+0000 7fa340bfd640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55f4c75bf400 0x55f4c6a7e000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.559+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(2 [pgnls start_epoch 90 out=177b] v90'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.559+0000 7fa340bfd640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x55f4c7647000 0x55f4c6851600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.559+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c74ae000 con 0x55f4c7647000 2024-04-14T08:40:10.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.560+0000 7fa3403fc640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x55f4c7647000 0x55f4c6851600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.560+0000 7fa3403fc640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x55f4c7647000 0x55f4c6851600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.560+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.2 v2:172.21.15.172:6816/1609902500 1 ==== osd_op_reply(3 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647000 2024-04-14T08:40:10.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.560+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c74ae000 con 0x55f4c75bf000 2024-04-14T08:40:10.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.560+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 2 ==== osd_op_reply(4 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.560+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x55f4c74ae000 0x55f4c6a7e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.560+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c7647400 con 0x55f4c74ae000 2024-04-14T08:40:10.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.561+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x55f4c74ae000 0x55f4c6a7e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.561+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x55f4c74ae000 0x55f4c6a7e580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.561+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== osd_op_reply(5 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.561+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55f4c7647400 0x55f4c6a7eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.561+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c7647800 con 0x55f4c7647400 2024-04-14T08:40:10.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.561+0000 7fa340bfd640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55f4c7647400 0x55f4c6a7eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.562+0000 7fa340bfd640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55f4c7647400 0x55f4c6a7eb00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.562+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== osd_op_reply(6 [pgnls start_epoch 90 out=149b] v89'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c7647400 2024-04-14T08:40:10.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.562+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c7647800 con 0x55f4c75bf400 2024-04-14T08:40:10.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.562+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 2 ==== osd_op_reply(7 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.562+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c7647800 con 0x55f4c74ae000 2024-04-14T08:40:10.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.563+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 2 ==== osd_op_reply(8 [pgnls start_epoch 90 out=73b] v89'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.563+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55f4c7647800 0x55f4c6a7f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.563+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c7647c00 con 0x55f4c7647800 2024-04-14T08:40:10.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.563+0000 7fa3403fc640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55f4c7647800 0x55f4c6a7f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.563+0000 7fa3403fc640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55f4c7647800 0x55f4c6a7f080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.563+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.5 v2:172.21.15.177:6816/3172735715 1 ==== osd_op_reply(9 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647800 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.563+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c7647c00 con 0x55f4c7647400 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.564+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.0 v2:172.21.15.172:6800/3761711279 2 ==== osd_op_reply(10 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c7647400 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.564+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c7647c00 con 0x55f4c7647000 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.564+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.2 v2:172.21.15.172:6816/1609902500 2 ==== osd_op_reply(11 [pgnls start_epoch 90 out=81b] v89'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647000 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.564+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c7647c00 con 0x55f4c75bf400 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.564+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 3 ==== osd_op_reply(12 [pgnls start_epoch 90 out=49b] v87'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.564+0000 7fa340bfd640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x55f4c7647c00 0x55f4c6850000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.564+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8a000 con 0x55f4c7647c00 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.564+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x55f4c7647c00 0x55f4c6850000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.565+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x55f4c7647c00 0x55f4c6850000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.565+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 1 ==== osd_op_reply(13 [pgnls start_epoch 90 out=77b] v89'7 uv7 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.565+0000 7fa33fbfb640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x55f4c6a8a000 0x55f4c6a7f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.565+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c6a8a000 2024-04-14T08:40:10.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.565+0000 7fa340bfd640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x55f4c6a8a000 0x55f4c6a7f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.565+0000 7fa340bfd640 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x55f4c6a8a000 0x55f4c6a7f600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.566+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.3 v2:172.21.15.172:6808/2305139122 1 ==== osd_op_reply(14 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c6a8a000 2024-04-14T08:40:10.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.566+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c75bf000 2024-04-14T08:40:10.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.566+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 3 ==== osd_op_reply(15 [pgnls start_epoch 90 out=73b] v89'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.566+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c7647c00 2024-04-14T08:40:10.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.566+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 2 ==== osd_op_reply(16 [pgnls start_epoch 90 out=49b] v87'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.566+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c74ae000 2024-04-14T08:40:10.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.566+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 3 ==== osd_op_reply(17 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.566+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c75bf400 2024-04-14T08:40:10.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.567+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 4 ==== osd_op_reply(18 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.567+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c7647800 2024-04-14T08:40:10.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.567+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.5 v2:172.21.15.177:6816/3172735715 2 ==== osd_op_reply(19 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647800 2024-04-14T08:40:10.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.567+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c6a8a000 2024-04-14T08:40:10.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.567+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.3 v2:172.21.15.172:6808/2305139122 2 ==== osd_op_reply(20 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c6a8a000 2024-04-14T08:40:10.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.567+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c7647c00 2024-04-14T08:40:10.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.568+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 3 ==== osd_op_reply(21 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.568+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c7647c00 2024-04-14T08:40:10.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.568+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 4 ==== osd_op_reply(22 [pgnls start_epoch 90 out=97b] v89'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.568+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c7647c00 2024-04-14T08:40:10.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.568+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 5 ==== osd_op_reply(23 [pgnls start_epoch 90 out=74b] v85'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.568+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c75bf000 2024-04-14T08:40:10.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.568+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 4 ==== osd_op_reply(24 [pgnls start_epoch 90 out=73b] v89'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.568+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c7647400 2024-04-14T08:40:10.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.569+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.0 v2:172.21.15.172:6800/3761711279 3 ==== osd_op_reply(25 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c7647400 2024-04-14T08:40:10.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.569+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c75bf000 2024-04-14T08:40:10.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.569+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 5 ==== osd_op_reply(26 [pgnls start_epoch 90 out=98b] v89'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.569+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c7647800 2024-04-14T08:40:10.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.569+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.5 v2:172.21.15.177:6816/3172735715 3 ==== osd_op_reply(27 [pgnls start_epoch 90 out=76b] v89'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647800 2024-04-14T08:40:10.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.569+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c7647c00 2024-04-14T08:40:10.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.570+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 6 ==== osd_op_reply(28 [pgnls start_epoch 90 out=73b] v89'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.570+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c75bf000 2024-04-14T08:40:10.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.570+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 6 ==== osd_op_reply(29 [pgnls start_epoch 90 out=73b] v89'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.570+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c7647800 2024-04-14T08:40:10.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.570+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.5 v2:172.21.15.177:6816/3172735715 4 ==== osd_op_reply(30 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647800 2024-04-14T08:40:10.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.570+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c74ae000 2024-04-14T08:40:10.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.570+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 4 ==== osd_op_reply(31 [pgnls start_epoch 90 out=73b] v85'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.570+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c74ae000 2024-04-14T08:40:10.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.571+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 5 ==== osd_op_reply(32 [pgnls start_epoch 90 out=101b] v89'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.571+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-14T08:40:10.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.571+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98000 con 0x55f4c75bf400 2024-04-14T08:40:10.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.571+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.571+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-14T08:40:10.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.571+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98400 con 0x55f4c75bf400 2024-04-14T08:40:10.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-14T08:40:10.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c75bf400 2024-04-14T08:40:10.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 20 datascan.scan_links: injected ino detected 2024-04-14T08:40:10.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-14T08:40:10.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98c00 con 0x55f4c75bf400 2024-04-14T08:40:10.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-14T08:40:10.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a99000 con 0x55f4c75bf400 2024-04-14T08:40:10.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-14T08:40:10.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-14T08:40:10.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-14T08:40:10.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-14T08:40:10.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-14T08:40:10.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-14T08:40:10.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-14T08:40:10.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a99400 con 0x55f4c7647400 2024-04-14T08:40:10.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.0 v2:172.21.15.172:6800/3761711279 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c7647400 2024-04-14T08:40:10.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-14T08:40:10.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.572+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98000 con 0x55f4c74ae000 2024-04-14T08:40:10.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.573+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.573+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-14T08:40:10.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.573+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98400 con 0x55f4c7647000 2024-04-14T08:40:10.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.573+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.2 v2:172.21.15.172:6816/1609902500 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647000 2024-04-14T08:40:10.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.573+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-14T08:40:10.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.573+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-14T08:40:10.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.573+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-14T08:40:10.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.573+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b400 con 0x55f4c75bf000 2024-04-14T08:40:10.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-14T08:40:10.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b000 con 0x55f4c7647c00 2024-04-14T08:40:10.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-14T08:40:10.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-14T08:40:10.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-14T08:40:10.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-14T08:40:10.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-14T08:40:10.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8ac00 con 0x55f4c75bf000 2024-04-14T08:40:10.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-14T08:40:10.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-14T08:40:10.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-14T08:40:10.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.574+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8a800 con 0x55f4c75bf000 2024-04-14T08:40:10.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-14T08:40:10.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-14T08:40:10.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-14T08:40:10.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-14T08:40:10.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-14T08:40:10.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8a400 con 0x55f4c75bf000 2024-04-14T08:40:10.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-14T08:40:10.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-14T08:40:10.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-14T08:40:10.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-14T08:40:10.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-14T08:40:10.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98000 con 0x55f4c74ae000 2024-04-14T08:40:10.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.575+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c75bf000 2024-04-14T08:40:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.576+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 11 ==== osd_op_reply(47 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.576+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c75bf400 2024-04-14T08:40:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.576+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 10 ==== osd_op_reply(48 [pgnls start_epoch 90 out=177b] v90'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.576+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647000 2024-04-14T08:40:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.576+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.2 v2:172.21.15.172:6816/1609902500 4 ==== osd_op_reply(49 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647000 2024-04-14T08:40:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.576+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c75bf000 2024-04-14T08:40:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.577+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 12 ==== osd_op_reply(50 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.577+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c74ae000 2024-04-14T08:40:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.577+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 8 ==== osd_op_reply(51 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.577+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647400 2024-04-14T08:40:10.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.577+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.0 v2:172.21.15.172:6800/3761711279 5 ==== osd_op_reply(52 [pgnls start_epoch 90 out=149b] v89'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c7647400 2024-04-14T08:40:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.577+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c75bf400 2024-04-14T08:40:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.577+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 11 ==== osd_op_reply(53 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.577+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c74ae000 2024-04-14T08:40:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.578+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 9 ==== osd_op_reply(54 [pgnls start_epoch 90 out=73b] v89'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.578+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647800 2024-04-14T08:40:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.578+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.5 v2:172.21.15.177:6816/3172735715 5 ==== osd_op_reply(55 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647800 2024-04-14T08:40:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.578+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647400 2024-04-14T08:40:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.578+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.0 v2:172.21.15.172:6800/3761711279 6 ==== osd_op_reply(56 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c7647400 2024-04-14T08:40:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.578+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647000 2024-04-14T08:40:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.578+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.2 v2:172.21.15.172:6816/1609902500 5 ==== osd_op_reply(57 [pgnls start_epoch 90 out=81b] v89'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647000 2024-04-14T08:40:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.578+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c75bf400 2024-04-14T08:40:10.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.579+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 12 ==== osd_op_reply(58 [pgnls start_epoch 90 out=49b] v87'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.579+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647c00 2024-04-14T08:40:10.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.579+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 8 ==== osd_op_reply(59 [pgnls start_epoch 90 out=77b] v89'7 uv7 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.579+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c6a8a000 2024-04-14T08:40:10.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.579+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.3 v2:172.21.15.172:6808/2305139122 3 ==== osd_op_reply(60 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c6a8a000 2024-04-14T08:40:10.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.579+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c75bf000 2024-04-14T08:40:10.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.580+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 13 ==== osd_op_reply(61 [pgnls start_epoch 90 out=73b] v89'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.580+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647c00 2024-04-14T08:40:10.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.580+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 9 ==== osd_op_reply(62 [pgnls start_epoch 90 out=49b] v87'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.580+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c74ae000 2024-04-14T08:40:10.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.580+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 10 ==== osd_op_reply(63 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.580+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c75bf400 2024-04-14T08:40:10.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.580+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 13 ==== osd_op_reply(64 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.580+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647800 2024-04-14T08:40:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.581+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.5 v2:172.21.15.177:6816/3172735715 6 ==== osd_op_reply(65 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647800 2024-04-14T08:40:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.581+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c6a8a000 2024-04-14T08:40:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.581+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.3 v2:172.21.15.172:6808/2305139122 4 ==== osd_op_reply(66 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c6a8a000 2024-04-14T08:40:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.581+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647c00 2024-04-14T08:40:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.581+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 10 ==== osd_op_reply(67 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.581+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647c00 2024-04-14T08:40:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.582+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 11 ==== osd_op_reply(68 [pgnls start_epoch 90 out=97b] v89'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.582+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647c00 2024-04-14T08:40:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.582+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 12 ==== osd_op_reply(69 [pgnls start_epoch 90 out=74b] v85'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.582+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c75bf000 2024-04-14T08:40:10.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.582+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 14 ==== osd_op_reply(70 [pgnls start_epoch 90 out=73b] v89'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.582+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647400 2024-04-14T08:40:10.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.582+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.0 v2:172.21.15.172:6800/3761711279 7 ==== osd_op_reply(71 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c7647400 2024-04-14T08:40:10.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.582+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c75bf000 2024-04-14T08:40:10.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.583+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 15 ==== osd_op_reply(72 [pgnls start_epoch 90 out=98b] v89'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.583+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647800 2024-04-14T08:40:10.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.583+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.5 v2:172.21.15.177:6816/3172735715 7 ==== osd_op_reply(73 [pgnls start_epoch 90 out=76b] v89'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647800 2024-04-14T08:40:10.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.583+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647c00 2024-04-14T08:40:10.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.583+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 13 ==== osd_op_reply(74 [pgnls start_epoch 90 out=73b] v89'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.583+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c75bf000 2024-04-14T08:40:10.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.584+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 16 ==== osd_op_reply(75 [pgnls start_epoch 90 out=73b] v89'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.584+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c7647800 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.584+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.5 v2:172.21.15.177:6816/3172735715 8 ==== osd_op_reply(76 [pgnls start_epoch 90 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647800 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.584+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c74ae000 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.584+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 11 ==== osd_op_reply(77 [pgnls start_epoch 90 out=73b] v85'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.584+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 90 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c74ae000 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.584+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 12 ==== osd_op_reply(78 [pgnls start_epoch 90 out=101b] v89'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.584+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8bc00 con 0x55f4c75bf400 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c75bf400 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-14T08:40:10.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b400 con 0x55f4c75bf400 2024-04-14T08:40:10.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa342a17740 20 datascan.scan_links: injected ino detected 2024-04-14T08:40:10.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa342a17740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-14T08:40:10.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa342a17740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-14T08:40:10.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-14T08:40:10.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b000 con 0x55f4c75bf400 2024-04-14T08:40:10.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-14T08:40:10.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.585+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8ac00 con 0x55f4c75bf400 2024-04-14T08:40:10.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.586+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.586+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-14T08:40:10.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.586+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-14T08:40:10.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.586+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-14T08:40:10.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.586+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-14T08:40:10.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.586+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-14T08:40:10.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.586+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-14T08:40:10.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.586+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-14T08:40:10.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.586+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8a800 con 0x55f4c7647400 2024-04-14T08:40:10.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.586+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.0 v2:172.21.15.172:6800/3761711279 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c7647400 2024-04-14T08:40:10.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.586+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-14T08:40:10.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.586+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98c00 con 0x55f4c74ae000 2024-04-14T08:40:10.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.587+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.587+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-14T08:40:10.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.587+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a99000 con 0x55f4c7647000 2024-04-14T08:40:10.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.587+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 <== osd.2 v2:172.21.15.172:6816/1609902500 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x55f4c746c780 con 0x55f4c7647000 2024-04-14T08:40:10.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.587+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-14T08:40:10.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.587+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-14T08:40:10.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.587+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-14T08:40:10.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.587+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a99400 con 0x55f4c75bf000 2024-04-14T08:40:10.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.587+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-14T08:40:10.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8bc00 con 0x55f4c7647c00 2024-04-14T08:40:10.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-14T08:40:10.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-14T08:40:10.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-14T08:40:10.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-14T08:40:10.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-14T08:40:10.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b800 con 0x55f4c75bf000 2024-04-14T08:40:10.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-14T08:40:10.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-14T08:40:10.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-14T08:40:10.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b400 con 0x55f4c75bf000 2024-04-14T08:40:10.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-14T08:40:10.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-14T08:40:10.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.588+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8b000 con 0x55f4c75bf000 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8ac00 con 0x55f4c74ae000 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.6 v2:172.21.15.177:6800/2961907928 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c74ae000 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-14T08:40:10.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-14T08:40:10.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-14T08:40:10.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-14T08:40:10.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a8a800 con 0x55f4c75bf400 2024-04-14T08:40:10.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-14T08:40:10.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.589+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98000 con 0x55f4c75bf400 2024-04-14T08:40:10.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.591+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v90'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.591+0000 7fa342a17740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-14T08:40:10.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.591+0000 7fa342a17740 10 datascan.scan_links: processing 1 injected_inos 2024-04-14T08:40:10.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.591+0000 7fa342a17740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-14T08:40:10.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.591+0000 7fa342a17740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-14T08:40:10.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.591+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98800 con 0x55f4c75bf400 2024-04-14T08:40:10.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.591+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.591+0000 7fa342a17740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-14T08:40:10.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.591+0000 7fa342a17740 20 datascan.scan_links: first is now 2 2024-04-14T08:40:10.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.591+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a98c00 con 0x55f4c75bf400 2024-04-14T08:40:10.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.592+0000 7fa340bfd640 1 -- 172.21.15.172:0/3086526642 <== osd.1 v2:172.21.15.172:6823/1219922421 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v90'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55f4c746c500 con 0x55f4c75bf400 2024-04-14T08:40:10.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.592+0000 7fa342a17740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-14T08:40:10.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.592+0000 7fa342a17740 10 datascan.scan_links: updating inotable 2024-04-14T08:40:10.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.592+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a99400 con 0x55f4c75bf000 2024-04-14T08:40:10.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.593+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.593+0000 7fa342a17740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-14T08:40:10.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.593+0000 7fa342a17740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-14T08:40:10.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.593+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a99800 con 0x55f4c75bf000 2024-04-14T08:40:10.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.594+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.7 v2:172.21.15.177:6808/3609985207 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v90'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c75bf000 2024-04-14T08:40:10.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.594+0000 7fa342a17740 10 datascan.scan_links: updating snaptable 2024-04-14T08:40:10.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.594+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) -- 0x55f4c6a99c00 con 0x55f4c7647c00 2024-04-14T08:40:10.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa33fbfb640 1 -- 172.21.15.172:0/3086526642 <== osd.4 v2:172.21.15.177:6820/4180733146 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x55f4c746c000 con 0x55f4c7647c00 2024-04-14T08:40:10.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55f4c7647400 msgr2=0x55f4c6a7eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x55f4c7647400 0x55f4c6a7eb00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55f4c75bf400 msgr2=0x55f4c6a7e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55f4c75bf400 0x55f4c6a7e000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x55f4c7647000 msgr2=0x55f4c6851600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x55f4c7647000 0x55f4c6851600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x55f4c6a8a000 msgr2=0x55f4c6a7f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x55f4c6a8a000 0x55f4c6a7f600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x55f4c7647c00 msgr2=0x55f4c6850000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x55f4c7647c00 0x55f4c6850000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 reap_dead start 2024-04-14T08:40:10.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55f4c7647800 msgr2=0x55f4c6a7f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x55f4c7647800 0x55f4c6a7f080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x55f4c74ae000 msgr2=0x55f4c6a7e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x55f4c74ae000 0x55f4c6a7e580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55f4c75bf000 msgr2=0x55f4c6852c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x55f4c75bf000 0x55f4c6852c00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 reap_dead start 2024-04-14T08:40:10.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa3403fc640 1 -- 172.21.15.172:0/3086526642 reap_dead start 2024-04-14T08:40:10.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x55f4c7646c00 msgr2=0x55f4c6853700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x55f4c7646c00 0x55f4c6853700 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55f4c75baff0 tx=0x55f4c75c3800 comp rx=0 tx=0).stop 2024-04-14T08:40:10.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75be800 msgr2=0x55f4c6852100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75be800 0x55f4c6852100 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x55f4c75bbb90 tx=0x55f4c75c3920 comp rx=0 tx=0).stop 2024-04-14T08:40:10.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 shutdown_connections 2024-04-14T08:40:10.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x55f4c7646c00 0x55f4c6853700 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/3086526642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c75be800 0x55f4c6852100 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 >> 172.21.15.172:0/3086526642 conn(0x55f4c74af800 msgr2=0x55f4c67d5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:10.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 shutdown_connections 2024-04-14T08:40:10.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/3086526642 wait complete. 2024-04-14T08:40:10.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/32654052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c74af400 msgr2=0x55f4c6851080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/32654052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c74af400 0x55f4c6851080 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x55f4c67da480 tx=0x55f4c7480120 comp rx=0 tx=0).stop 2024-04-14T08:40:10.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/32654052 shutdown_connections 2024-04-14T08:40:10.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/32654052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c74af400 0x55f4c6851080 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/32654052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f4c74afc00 0x55f4c6851b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 --2- 172.21.15.172:0/32654052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f4c74ae800 0x55f4c6850b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.595+0000 7fa342a17740 1 -- 172.21.15.172:0/32654052 >> 172.21.15.172:0/32654052 conn(0x55f4c66e5000 msgr2=0x55f4c67d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:10.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.596+0000 7fa342a17740 1 -- 172.21.15.172:0/32654052 shutdown_connections 2024-04-14T08:40:10.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.596+0000 7fa342a17740 1 -- 172.21.15.172:0/32654052 wait complete. 2024-04-14T08:40:10.631 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.146499, result: 2024-04-14T08:40:10.631 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-14T08:40:10.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.746+0000 7f8cf266d640 1 Processor -- start 2024-04-14T08:40:10.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.746+0000 7f8cf266d640 1 -- start start 2024-04-14T08:40:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.746+0000 7f8cf266d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8cec161ce0 0x7f8cec0d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.746+0000 7f8cf266d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cec0cedc0 con 0x7f8cec0d1320 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.746+0000 7f8cf266d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cec0d00e0 con 0x7f8cec0d2d40 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.746+0000 7f8cf266d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f8cec0d31a0 con 0x7f8cec161ce0 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8cf0c0a640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f8cec0d1320 0x7f8cec0b0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50332/0 (socket says 172.21.15.172:50332) 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8cf0c0a640 1 -- 172.21.15.172:0/230344397 learned_addr learned my addr 172.21.15.172:0/230344397 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2740650918 0 0) 0x7f8ce4001490 con 0x7f8cec0d1320 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cd4003520 con 0x7f8cec0d1320 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8cebfff640 1 --2- 172.21.15.172:0/230344397 >> v2:172.21.15.177:3300/0 conn(0x7f8cec161ce0 0x7f8cec0d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2453980017 0 0) 0x7f8cd4003520 con 0x7f8cec0d1320 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 >> v2:172.21.15.177:3300/0 conn(0x7f8cec161ce0 msgr2=0x7f8cec0d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 --2- 172.21.15.172:0/230344397 >> v2:172.21.15.177:3300/0 conn(0x7f8cec161ce0 0x7f8cec0d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 >> v1:172.21.15.177:6789/0 conn(0x7f8cec0d2d40 legacy=0x7f8cec0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cec0cf040 con 0x7f8cec0d1320 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f8ce4001bc0 con 0x7f8cec0d1320 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 >> v1:172.21.15.172:6789/0 conn(0x7f8cec0d1320 legacy=0x7f8cec0b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 --2- 172.21.15.172:0/230344397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd40039b0 0x7f8cd4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 --2- 172.21.15.172:0/230344397 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd4004800 0x7f8cd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 --2- 172.21.15.172:0/230344397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 0x7f8cd400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd400c030 con 0x7f8cd40051c0 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd400c1e0 con 0x7f8cd40039b0 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd400c3e0 con 0x7f8cd4004800 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8cebfff640 1 --2- 172.21.15.172:0/230344397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd40039b0 0x7f8cd4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8cf0c0a640 1 --2- 172.21.15.172:0/230344397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 0x7f8cd400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.747+0000 7f8ceb7fe640 1 --2- 172.21.15.172:0/230344397 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd4004800 0x7f8cd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.748+0000 7f8cf0c0a640 1 -- 172.21.15.172:0/230344397 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd4004800 msgr2=0x7f8cd4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.748+0000 7f8cf0c0a640 1 --2- 172.21.15.172:0/230344397 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd4004800 0x7f8cd4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.748+0000 7f8cf0c0a640 1 -- 172.21.15.172:0/230344397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd40039b0 msgr2=0x7f8cd4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.748+0000 7f8cf0c0a640 1 --2- 172.21.15.172:0/230344397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd40039b0 0x7f8cd4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.748+0000 7f8cf0c0a640 1 -- 172.21.15.172:0/230344397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd4003520 con 0x7f8cd40051c0 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.748+0000 7f8cf0c0a640 1 -- 172.21.15.172:0/230344397 reap_dead start 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.748+0000 7f8cf0c0a640 1 --2- 172.21.15.172:0/230344397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 0x7f8cd400b9c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400aa50 tx=0x7f8cd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ee822ba38fa449 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.748+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4038070 con 0x7f8cd40051c0 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.749+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce4033650 con 0x7f8cd40051c0 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.749+0000 7f8ceaffd640 1 -- 172.21.15.172:0/230344397 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400a670 con 0x7f8cd40051c0 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.749+0000 7f8cf266d640 1 -- 172.21.15.172:0/230344397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 msgr2=0x7f8cd400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.749+0000 7f8cf266d640 1 --2- 172.21.15.172:0/230344397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 0x7f8cd400b9c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400aa50 tx=0x7f8cd4005830 comp rx=0 tx=0).stop 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.749+0000 7f8cf266d640 1 -- 172.21.15.172:0/230344397 shutdown_connections 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.749+0000 7f8cf266d640 1 --2- 172.21.15.172:0/230344397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 0x7f8cd400b9c0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.749+0000 7f8cf266d640 1 -- 172.21.15.172:0/230344397 >> 172.21.15.172:0/230344397 conn(0x7f8cec09d560 msgr2=0x7f8cec0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.749+0000 7f8cf266d640 1 -- 172.21.15.172:0/230344397 shutdown_connections 2024-04-14T08:40:10.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.749+0000 7f8cf266d640 1 -- 172.21.15.172:0/230344397 wait complete. 2024-04-14T08:40:10.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.750+0000 7f8cf266d640 1 Processor -- start 2024-04-14T08:40:10.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.750+0000 7f8cf266d640 1 -- start start 2024-04-14T08:40:10.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cf266d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 0x7f8cec1f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cf266d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cec1606d0 0x7f8cec15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cf266d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cec15e290 0x7f8cec0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cf266d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec0cf8e0 con 0x7f8cd40051c0 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cf266d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec0cf600 con 0x7f8cec1606d0 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cf266d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8cec0cfbc0 con 0x7f8cec15e290 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 0x7f8cec1f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cebfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 0x7f8cec1f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42098/0 (socket says 172.21.15.172:42098) 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cebfff640 1 -- 172.21.15.172:0/3261596348 learned_addr learned my addr 172.21.15.172:0/3261596348 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cf0c0a640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cec15e290 0x7f8cec0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8ceb7fe640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cec1606d0 0x7f8cec15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cebfff640 1 -- 172.21.15.172:0/3261596348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cec15e290 msgr2=0x7f8cec0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cebfff640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cec15e290 0x7f8cec0cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cebfff640 1 -- 172.21.15.172:0/3261596348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cec1606d0 msgr2=0x7f8cec15dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:10.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cebfff640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cec1606d0 0x7f8cec15dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:10.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cebfff640 1 -- 172.21.15.172:0/3261596348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cec160f20 con 0x7f8cd40051c0 2024-04-14T08:40:10.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8cf0c0a640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cec15e290 0x7f8cec0cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:10.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.751+0000 7f8ceb7fe640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cec1606d0 0x7f8cec15dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:10.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.752+0000 7f8cebfff640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 0x7f8cec1f53e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc011910 tx=0x7f8cdc011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.752+0000 7f8ce8ff9640 1 -- 172.21.15.172:0/3261596348 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc00faf0 con 0x7f8cd40051c0 2024-04-14T08:40:10.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.752+0000 7f8ce8ff9640 1 -- 172.21.15.172:0/3261596348 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cdc00fc90 con 0x7f8cd40051c0 2024-04-14T08:40:10.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.752+0000 7f8cf266d640 1 -- 172.21.15.172:0/3261596348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cec161200 con 0x7f8cd40051c0 2024-04-14T08:40:10.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.752+0000 7f8ce8ff9640 1 -- 172.21.15.172:0/3261596348 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc014440 con 0x7f8cd40051c0 2024-04-14T08:40:10.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.752+0000 7f8cf266d640 1 -- 172.21.15.172:0/3261596348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cec0b8380 con 0x7f8cd40051c0 2024-04-14T08:40:10.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.753+0000 7f8cf266d640 1 -- 172.21.15.172:0/3261596348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cec0b7e90 con 0x7f8cd40051c0 2024-04-14T08:40:10.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.753+0000 7f8ce8ff9640 1 -- 172.21.15.172:0/3261596348 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8cdc012040 con 0x7f8cd40051c0 2024-04-14T08:40:10.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.754+0000 7f8ce8ff9640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8cd806c8f0 0x7f8cd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:10.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.754+0000 7f8ce8ff9640 1 -- 172.21.15.172:0/3261596348 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f8cdc08c3f0 con 0x7f8cd40051c0 2024-04-14T08:40:10.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.754+0000 7f8ceb7fe640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8cd806c8f0 0x7f8cd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:10.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.754+0000 7f8ceb7fe640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8cd806c8f0 0x7f8cd806edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0000c00 tx=0x7f8ce0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:10.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.758+0000 7f8ce8ff9640 1 -- 172.21.15.172:0/3261596348 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8cdc019030 con 0x7f8cd40051c0 2024-04-14T08:40:10.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:10.971+0000 7f8cf266d640 1 -- 172.21.15.172:0/3261596348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f8cec075490 con 0x7f8cd40051c0 2024-04-14T08:40:11.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.109+0000 7f8ce8ff9640 1 -- 172.21.15.172:0/3261596348 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v63) ==== 162+0+0 (secure 0 0 0) 0x7f8cdc05a600 con 0x7f8cd40051c0 2024-04-14T08:40:11.110 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-14T08:40:11.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.112+0000 7f8cf266d640 1 -- 172.21.15.172:0/3261596348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8cd806c8f0 msgr2=0x7f8cd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:11.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.112+0000 7f8cf266d640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8cd806c8f0 0x7f8cd806edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0000c00 tx=0x7f8ce0009040 comp rx=0 tx=0).stop 2024-04-14T08:40:11.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.112+0000 7f8cf266d640 1 -- 172.21.15.172:0/3261596348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 msgr2=0x7f8cec1f53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:11.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.112+0000 7f8cf266d640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 0x7f8cec1f53e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc011910 tx=0x7f8cdc011dd0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.112+0000 7f8cf266d640 1 -- 172.21.15.172:0/3261596348 shutdown_connections 2024-04-14T08:40:11.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.112+0000 7f8cf266d640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8cd806c8f0 0x7f8cd806edb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.112+0000 7f8cf266d640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cec15e290 0x7f8cec0cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.112+0000 7f8cf266d640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cec1606d0 0x7f8cec15dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.112+0000 7f8cf266d640 1 --2- 172.21.15.172:0/3261596348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cd40051c0 0x7f8cec1f53e0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.112+0000 7f8cf266d640 1 -- 172.21.15.172:0/3261596348 >> 172.21.15.172:0/3261596348 conn(0x7f8cec09d560 msgr2=0x7f8cec0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:11.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.112+0000 7f8cf266d640 1 -- 172.21.15.172:0/3261596348 shutdown_connections 2024-04-14T08:40:11.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.112+0000 7f8cf266d640 1 -- 172.21.15.172:0/3261596348 wait complete. 2024-04-14T08:40:11.125 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:40:11.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.226+0000 7f9a2ea36640 1 Processor -- start 2024-04-14T08:40:11.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.226+0000 7f9a2ea36640 1 -- start start 2024-04-14T08:40:11.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.226+0000 7f9a2ea36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a280d2830 0x7f9a280d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:11.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.226+0000 7f9a2ea36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9a2815e890 0x7f9a280d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:11.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.226+0000 7f9a2ea36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a280cedd0 0x7f9a281621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:11.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.226+0000 7f9a2ea36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f9a280cff30 con 0x7f9a280d2830 2024-04-14T08:40:11.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.226+0000 7f9a2ea36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f9a280d2350 con 0x7f9a2815e890 2024-04-14T08:40:11.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.226+0000 7f9a2ea36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9a280d3150 con 0x7f9a280cedd0 2024-04-14T08:40:11.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.226+0000 7f9a27fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a280d2830 0x7f9a280d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:11.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.227+0000 7f9a27fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a280d2830 0x7f9a280d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42112/0 (socket says 172.21.15.172:42112) 2024-04-14T08:40:11.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.227+0000 7f9a27fff640 1 -- 172.21.15.172:0/84200752 learned_addr learned my addr 172.21.15.172:0/84200752 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:11.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.227+0000 7f9a2cfd3640 1 --2- 172.21.15.172:0/84200752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a280cedd0 0x7f9a281621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:11.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.227+0000 7f9a277fe640 1 --2- 172.21.15.172:0/84200752 >> v2:172.21.15.177:3300/0 conn(0x7f9a2815e890 0x7f9a280d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:11.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.227+0000 7f9a27fff640 1 -- 172.21.15.172:0/84200752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a280cedd0 msgr2=0x7f9a281621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:11.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.227+0000 7f9a27fff640 1 --2- 172.21.15.172:0/84200752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a280cedd0 0x7f9a281621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.227+0000 7f9a27fff640 1 -- 172.21.15.172:0/84200752 >> v2:172.21.15.177:3300/0 conn(0x7f9a2815e890 msgr2=0x7f9a280d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:11.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.227+0000 7f9a27fff640 1 --2- 172.21.15.172:0/84200752 >> v2:172.21.15.177:3300/0 conn(0x7f9a2815e890 0x7f9a280d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.227+0000 7f9a27fff640 1 -- 172.21.15.172:0/84200752 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a280cf320 con 0x7f9a280d2830 2024-04-14T08:40:11.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.227+0000 7f9a27fff640 1 --2- 172.21.15.172:0/84200752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a280d2830 0x7f9a280d2c10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f9a28162100 tx=0x7f9a1c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bbb8fc1fef283 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:11.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.228+0000 7f9a26ffd640 1 -- 172.21.15.172:0/84200752 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c034070 con 0x7f9a280d2830 2024-04-14T08:40:11.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.228+0000 7f9a26ffd640 1 -- 172.21.15.172:0/84200752 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a1c030860 con 0x7f9a280d2830 2024-04-14T08:40:11.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.228+0000 7f9a26ffd640 1 -- 172.21.15.172:0/84200752 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c030c30 con 0x7f9a280d2830 2024-04-14T08:40:11.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.228+0000 7f9a2ea36640 1 -- 172.21.15.172:0/84200752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a280d2830 msgr2=0x7f9a280d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:11.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.228+0000 7f9a2ea36640 1 --2- 172.21.15.172:0/84200752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a280d2830 0x7f9a280d2c10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f9a28162100 tx=0x7f9a1c02b9a0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.228+0000 7f9a2ea36640 1 -- 172.21.15.172:0/84200752 shutdown_connections 2024-04-14T08:40:11.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.228+0000 7f9a2ea36640 1 --2- 172.21.15.172:0/84200752 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a280cedd0 0x7f9a281621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.228+0000 7f9a2ea36640 1 --2- 172.21.15.172:0/84200752 >> v2:172.21.15.177:3300/0 conn(0x7f9a2815e890 0x7f9a280d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.228+0000 7f9a2ea36640 1 --2- 172.21.15.172:0/84200752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a280d2830 0x7f9a280d2c10 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.228+0000 7f9a2ea36640 1 -- 172.21.15.172:0/84200752 >> 172.21.15.172:0/84200752 conn(0x7f9a2809d590 msgr2=0x7f9a280d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:11.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.228+0000 7f9a2ea36640 1 -- 172.21.15.172:0/84200752 shutdown_connections 2024-04-14T08:40:11.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.228+0000 7f9a2ea36640 1 -- 172.21.15.172:0/84200752 wait complete. 2024-04-14T08:40:11.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.229+0000 7f9a2ea36640 1 Processor -- start 2024-04-14T08:40:11.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.230+0000 7f9a2ea36640 1 -- start start 2024-04-14T08:40:11.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.230+0000 7f9a2ea36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a280cedd0 0x7f9a2814a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:11.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.230+0000 7f9a2ea36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a280d2830 0x7f9a280b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:11.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.230+0000 7f9a2ea36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a2815e890 0x7f9a280b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:11.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.230+0000 7f9a2ea36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9a280cfbc0 con 0x7f9a280cedd0 2024-04-14T08:40:11.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.230+0000 7f9a2ea36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9a280cf8e0 con 0x7f9a280d2830 2024-04-14T08:40:11.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.230+0000 7f9a2ea36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9a280d1840 con 0x7f9a2815e890 2024-04-14T08:40:11.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.230+0000 7f9a27fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a280cedd0 0x7f9a2814a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:11.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.230+0000 7f9a27fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a280cedd0 0x7f9a2814a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42116/0 (socket says 172.21.15.172:42116) 2024-04-14T08:40:11.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.230+0000 7f9a27fff640 1 -- 172.21.15.172:0/1229060435 learned_addr learned my addr 172.21.15.172:0/1229060435 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:11.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.230+0000 7f9a2cfd3640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a2815e890 0x7f9a280b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.230+0000 7f9a277fe640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a280d2830 0x7f9a280b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a27fff640 1 -- 172.21.15.172:0/1229060435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a2815e890 msgr2=0x7f9a280b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a27fff640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a2815e890 0x7f9a280b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a27fff640 1 -- 172.21.15.172:0/1229060435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a280d2830 msgr2=0x7f9a280b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a27fff640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a280d2830 0x7f9a280b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a27fff640 1 -- 172.21.15.172:0/1229060435 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2814ab90 con 0x7f9a280cedd0 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a2cfd3640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a2815e890 0x7f9a280b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a277fe640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a280d2830 0x7f9a280b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a27fff640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a280cedd0 0x7f9a2814a650 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c000c00 tx=0x7f9a1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a24ff9640 1 -- 172.21.15.172:0/1229060435 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c034040 con 0x7f9a280cedd0 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a24ff9640 1 -- 172.21.15.172:0/1229060435 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a1c039d90 con 0x7f9a280cedd0 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a24ff9640 1 -- 172.21.15.172:0/1229060435 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c043610 con 0x7f9a280cedd0 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a2ea36640 1 -- 172.21.15.172:0/1229060435 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a281f49a0 con 0x7f9a280cedd0 2024-04-14T08:40:11.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.231+0000 7f9a2ea36640 1 -- 172.21.15.172:0/1229060435 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a281f4f60 con 0x7f9a280cedd0 2024-04-14T08:40:11.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.232+0000 7f9a2ea36640 1 -- 172.21.15.172:0/1229060435 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a28068520 con 0x7f9a280cedd0 2024-04-14T08:40:11.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.233+0000 7f9a24ff9640 1 -- 172.21.15.172:0/1229060435 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a1c037070 con 0x7f9a280cedd0 2024-04-14T08:40:11.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.233+0000 7f9a24ff9640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a1406c8f0 0x7f9a1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:11.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.233+0000 7f9a24ff9640 1 -- 172.21.15.172:0/1229060435 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f9a1c0ae020 con 0x7f9a280cedd0 2024-04-14T08:40:11.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.233+0000 7f9a277fe640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a1406c8f0 0x7f9a1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:11.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.233+0000 7f9a277fe640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a1406c8f0 0x7f9a1406edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9a281f2280 tx=0x7f9a18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:11.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.237+0000 7f9a24ff9640 1 -- 172.21.15.172:0/1229060435 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9a2814ab90 con 0x7f9a280cedd0 2024-04-14T08:40:11.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.449+0000 7f9a2ea36640 1 -- 172.21.15.172:0/1229060435 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a280d2610 con 0x7f9a280cedd0 2024-04-14T08:40:11.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.450+0000 7f9a24ff9640 1 -- 172.21.15.172:0/1229060435 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5189 (secure 0 0 0) 0x7f9a1c03e070 con 0x7f9a280cedd0 2024-04-14T08:40:11.450 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:11.451 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:40:11.112582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8013":{"gid":8013,"name":"c","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.172:6837/3091771014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3091771014},{"type":"v1","addr":"172.21.15.172:6837","nonce":3091771014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-14T08:40:11.451 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 64 2024-04-14T08:40:11.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.452+0000 7f9a2ea36640 1 -- 172.21.15.172:0/1229060435 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a1406c8f0 msgr2=0x7f9a1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:11.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.452+0000 7f9a2ea36640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a1406c8f0 0x7f9a1406edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9a281f2280 tx=0x7f9a18008040 comp rx=0 tx=0).stop 2024-04-14T08:40:11.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.452+0000 7f9a2ea36640 1 -- 172.21.15.172:0/1229060435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a280cedd0 msgr2=0x7f9a2814a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:11.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.452+0000 7f9a2ea36640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a280cedd0 0x7f9a2814a650 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c000c00 tx=0x7f9a1c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.452+0000 7f9a2ea36640 1 -- 172.21.15.172:0/1229060435 shutdown_connections 2024-04-14T08:40:11.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.452+0000 7f9a2ea36640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a1406c8f0 0x7f9a1406edb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.452+0000 7f9a2ea36640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a2815e890 0x7f9a280b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.452+0000 7f9a2ea36640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a280d2830 0x7f9a280b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.452+0000 7f9a2ea36640 1 --2- 172.21.15.172:0/1229060435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a280cedd0 0x7f9a2814a650 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:11.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.452+0000 7f9a2ea36640 1 -- 172.21.15.172:0/1229060435 >> 172.21.15.172:0/1229060435 conn(0x7f9a2809d590 msgr2=0x7f9a2815c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:11.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.452+0000 7f9a2ea36640 1 -- 172.21.15.172:0/1229060435 shutdown_connections 2024-04-14T08:40:11.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:11.452+0000 7f9a2ea36640 1 -- 172.21.15.172:0/1229060435 wait complete. 2024-04-14T08:40:11.866 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:38:44.572424+0000', 'modified': '2024-04-14T08:40:11.112582+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8013}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8013': {'gid': 8013, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.172:6837/3091771014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 3091771014}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 3091771014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-14T08:40:11.866 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8013:up:replay 2024-04-14T08:40:12.867 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:40:12.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.978+0000 7f5e536b7640 1 Processor -- start 2024-04-14T08:40:12.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.978+0000 7f5e536b7640 1 -- start start 2024-04-14T08:40:12.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.979+0000 7f5e536b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c15f400 0x7f5e4c15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:12.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.979+0000 7f5e536b7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5e4c0b1b80 0x7f5e4c0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:12.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.979+0000 7f5e536b7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e4c13f600 con 0x7f5e4c15d9d0 2024-04-14T08:40:12.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.979+0000 7f5e536b7640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5e4c0b2520 con 0x7f5e4c0b1b80 2024-04-14T08:40:12.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.979+0000 7f5e536b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e4c140690 con 0x7f5e4c15f400 2024-04-14T08:40:12.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.979+0000 7f5e51453640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c15f400 0x7f5e4c15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:12.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.979+0000 7f5e51c54640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5e4c0b1b80 0x7f5e4c0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:12.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.979+0000 7f5e50c52640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f5e4c15d9d0 0x7f5e4c15d190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:59500/0 (socket says 172.21.15.172:59500) 2024-04-14T08:40:12.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.979+0000 7f5e51453640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c15f400 0x7f5e4c15f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:38056/0 (socket says 172.21.15.172:38056) 2024-04-14T08:40:12.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.979+0000 7f5e50c52640 1 -- 172.21.15.172:0/3945160614 learned_addr learned my addr 172.21.15.172:0/3945160614 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:12.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.980+0000 7f5e51453640 1 -- 172.21.15.172:0/3945160614 >> v2:172.21.15.177:3300/0 conn(0x7f5e4c0b1b80 msgr2=0x7f5e4c0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:12.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.980+0000 7f5e51453640 1 --2- 172.21.15.172:0/3945160614 >> v2:172.21.15.177:3300/0 conn(0x7f5e4c0b1b80 0x7f5e4c0b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:12.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.980+0000 7f5e51453640 1 -- 172.21.15.172:0/3945160614 >> v1:172.21.15.177:6789/0 conn(0x7f5e4c15d9d0 legacy=0x7f5e4c15d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:12.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.980+0000 7f5e51453640 1 -- 172.21.15.172:0/3945160614 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c13fa80 con 0x7f5e4c15f400 2024-04-14T08:40:12.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.980+0000 7f5e51c54640 1 --2- 172.21.15.172:0/3945160614 >> v2:172.21.15.177:3300/0 conn(0x7f5e4c0b1b80 0x7f5e4c0b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:40:12.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.981+0000 7f5e51453640 1 --2- 172.21.15.172:0/3945160614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c15f400 0x7f5e4c15f7e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009d90 tx=0x7f5e4802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d4bbc84795d0df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:12.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.981+0000 7f5e43fff640 1 -- 172.21.15.172:0/3945160614 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48039070 con 0x7f5e4c15f400 2024-04-14T08:40:12.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.981+0000 7f5e43fff640 1 -- 172.21.15.172:0/3945160614 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e4802cc50 con 0x7f5e4c15f400 2024-04-14T08:40:12.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.981+0000 7f5e43fff640 1 -- 172.21.15.172:0/3945160614 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48034440 con 0x7f5e4c15f400 2024-04-14T08:40:12.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.981+0000 7f5e536b7640 1 -- 172.21.15.172:0/3945160614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c15f400 msgr2=0x7f5e4c15f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:12.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.981+0000 7f5e536b7640 1 --2- 172.21.15.172:0/3945160614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c15f400 0x7f5e4c15f7e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009d90 tx=0x7f5e4802bea0 comp rx=0 tx=0).stop 2024-04-14T08:40:12.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.981+0000 7f5e536b7640 1 -- 172.21.15.172:0/3945160614 shutdown_connections 2024-04-14T08:40:12.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.981+0000 7f5e536b7640 1 --2- 172.21.15.172:0/3945160614 >> v2:172.21.15.177:3300/0 conn(0x7f5e4c0b1b80 0x7f5e4c0b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:12.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.981+0000 7f5e536b7640 1 --2- 172.21.15.172:0/3945160614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c15f400 0x7f5e4c15f7e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:12.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.981+0000 7f5e536b7640 1 -- 172.21.15.172:0/3945160614 >> 172.21.15.172:0/3945160614 conn(0x7f5e4c06ec30 msgr2=0x7f5e4c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:12.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.982+0000 7f5e536b7640 1 -- 172.21.15.172:0/3945160614 shutdown_connections 2024-04-14T08:40:12.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.982+0000 7f5e536b7640 1 -- 172.21.15.172:0/3945160614 wait complete. 2024-04-14T08:40:12.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.983+0000 7f5e536b7640 1 Processor -- start 2024-04-14T08:40:12.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.983+0000 7f5e536b7640 1 -- start start 2024-04-14T08:40:12.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.983+0000 7f5e536b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c0b1b80 0x7f5e4c152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:12.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.983+0000 7f5e536b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c15d9d0 0x7f5e4c154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:12.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.983+0000 7f5e536b7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e4c15f400 0x7f5e4c15b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:12.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.983+0000 7f5e536b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c140320 con 0x7f5e4c15d9d0 2024-04-14T08:40:12.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.983+0000 7f5e536b7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c140040 con 0x7f5e4c15f400 2024-04-14T08:40:12.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.983+0000 7f5e536b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e4c15de30 con 0x7f5e4c0b1b80 2024-04-14T08:40:12.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e50c52640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c15d9d0 0x7f5e4c154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:12.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e51453640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c0b1b80 0x7f5e4c152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:12.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e50c52640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c15d9d0 0x7f5e4c154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42124/0 (socket says 172.21.15.172:42124) 2024-04-14T08:40:12.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e51453640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c0b1b80 0x7f5e4c152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:38058/0 (socket says 172.21.15.172:38058) 2024-04-14T08:40:12.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e50c52640 1 -- 172.21.15.172:0/1113567518 learned_addr learned my addr 172.21.15.172:0/1113567518 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:12.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e51c54640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e4c15f400 0x7f5e4c15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:12.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e50c52640 1 -- 172.21.15.172:0/1113567518 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c0b1b80 msgr2=0x7f5e4c152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:12.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e50c52640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c0b1b80 0x7f5e4c152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:12.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e50c52640 1 -- 172.21.15.172:0/1113567518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e4c15f400 msgr2=0x7f5e4c15b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:12.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e50c52640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e4c15f400 0x7f5e4c15b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:12.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e50c52640 1 -- 172.21.15.172:0/1113567518 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c152b60 con 0x7f5e4c15d9d0 2024-04-14T08:40:12.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e50c52640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c15d9d0 0x7f5e4c154510 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00c400 tx=0x7f5e3c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:12.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e41ffb640 1 -- 172.21.15.172:0/1113567518 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c017070 con 0x7f5e4c15d9d0 2024-04-14T08:40:12.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e41ffb640 1 -- 172.21.15.172:0/1113567518 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c005ce0 con 0x7f5e4c15d9d0 2024-04-14T08:40:12.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e536b7640 1 -- 172.21.15.172:0/1113567518 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4c1f41a0 con 0x7f5e4c15d9d0 2024-04-14T08:40:12.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.984+0000 7f5e41ffb640 1 -- 172.21.15.172:0/1113567518 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c007310 con 0x7f5e4c15d9d0 2024-04-14T08:40:12.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.985+0000 7f5e536b7640 1 -- 172.21.15.172:0/1113567518 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4c13b130 con 0x7f5e4c15d9d0 2024-04-14T08:40:12.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.986+0000 7f5e536b7640 1 -- 172.21.15.172:0/1113567518 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e4c1f4370 con 0x7f5e4c15d9d0 2024-04-14T08:40:12.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.986+0000 7f5e41ffb640 1 -- 172.21.15.172:0/1113567518 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e3c01a070 con 0x7f5e4c15d9d0 2024-04-14T08:40:12.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.986+0000 7f5e41ffb640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e3006c8f0 0x7f5e3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:12.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.986+0000 7f5e41ffb640 1 -- 172.21.15.172:0/1113567518 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f5e3c08afe0 con 0x7f5e4c15d9d0 2024-04-14T08:40:12.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.986+0000 7f5e51453640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e3006c8f0 0x7f5e3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:12.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.987+0000 7f5e51453640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e3006c8f0 0x7f5e3006edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5e48036000 tx=0x7f5e48030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:12.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:12.991+0000 7f5e41ffb640 1 -- 172.21.15.172:0/1113567518 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5e4c13f600 con 0x7f5e4c15d9d0 2024-04-14T08:40:13.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.203+0000 7f5e536b7640 1 -- 172.21.15.172:0/1113567518 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e4c156b50 con 0x7f5e4c15d9d0 2024-04-14T08:40:13.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.204+0000 7f5e41ffb640 1 -- 172.21.15.172:0/1113567518 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5189 (secure 0 0 0) 0x7f5e3c058ba0 con 0x7f5e4c15d9d0 2024-04-14T08:40:13.204 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:13.205 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:40:12.122085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8013":{"gid":8013,"name":"c","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.172:6837/3091771014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3091771014},{"type":"v1","addr":"172.21.15.172:6837","nonce":3091771014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-14T08:40:13.205 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 66 2024-04-14T08:40:13.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.205+0000 7f5e536b7640 1 -- 172.21.15.172:0/1113567518 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e3006c8f0 msgr2=0x7f5e3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:13.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.205+0000 7f5e536b7640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e3006c8f0 0x7f5e3006edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5e48036000 tx=0x7f5e48030000 comp rx=0 tx=0).stop 2024-04-14T08:40:13.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.205+0000 7f5e536b7640 1 -- 172.21.15.172:0/1113567518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c15d9d0 msgr2=0x7f5e4c154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:13.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.205+0000 7f5e536b7640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c15d9d0 0x7f5e4c154510 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00c400 tx=0x7f5e3c00c800 comp rx=0 tx=0).stop 2024-04-14T08:40:13.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.206+0000 7f5e536b7640 1 -- 172.21.15.172:0/1113567518 shutdown_connections 2024-04-14T08:40:13.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.206+0000 7f5e536b7640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e3006c8f0 0x7f5e3006edb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:13.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.206+0000 7f5e536b7640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e4c15f400 0x7f5e4c15b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:13.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.206+0000 7f5e536b7640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c15d9d0 0x7f5e4c154510 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:13.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.206+0000 7f5e536b7640 1 --2- 172.21.15.172:0/1113567518 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e4c0b1b80 0x7f5e4c152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:13.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.206+0000 7f5e536b7640 1 -- 172.21.15.172:0/1113567518 >> 172.21.15.172:0/1113567518 conn(0x7f5e4c06ec30 msgr2=0x7f5e4c15eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:13.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.206+0000 7f5e536b7640 1 -- 172.21.15.172:0/1113567518 shutdown_connections 2024-04-14T08:40:13.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:13.206+0000 7f5e536b7640 1 -- 172.21.15.172:0/1113567518 wait complete. 2024-04-14T08:40:13.620 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:38:44.572424+0000', 'modified': '2024-04-14T08:40:12.122085+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8013}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8013': {'gid': 8013, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.172:6837/3091771014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 3091771014}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 3091771014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-14T08:40:13.620 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8013:up:rejoin 2024-04-14T08:40:14.621 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:40:14.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.723+0000 7fa1963a3640 1 Processor -- start 2024-04-14T08:40:14.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.723+0000 7fa1963a3640 1 -- start start 2024-04-14T08:40:14.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa1963a3640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa1900d1a70 0x7fa1900b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:14.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa1963a3640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1900d1ed0 con 0x7fa19015e390 2024-04-14T08:40:14.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa1963a3640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa19015e770 con 0x7fa1900b4260 2024-04-14T08:40:14.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa1963a3640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa1900d0080 con 0x7fa1900d1a70 2024-04-14T08:40:14.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa18ffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa19015e390 0x7fa19015f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50342/0 (socket says 172.21.15.172:50342) 2024-04-14T08:40:14.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa18ffff640 1 -- 172.21.15.172:0/2317925693 learned_addr learned my addr 172.21.15.172:0/2317925693 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:14.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa18f7fe640 1 --2- 172.21.15.172:0/2317925693 >> v2:172.21.15.177:3300/0 conn(0x7fa1900d1a70 0x7fa1900b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:14.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1132186782 0 0) 0x7fa178001110 con 0x7fa19015e390 2024-04-14T08:40:14.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa17c003520 con 0x7fa19015e390 2024-04-14T08:40:14.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1859389486 0 0) 0x7fa17c003520 con 0x7fa19015e390 2024-04-14T08:40:14.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 >> v2:172.21.15.177:3300/0 conn(0x7fa1900d1a70 msgr2=0x7fa1900b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:14.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa18effd640 1 --2- 172.21.15.172:0/2317925693 >> v2:172.21.15.177:3300/0 conn(0x7fa1900d1a70 0x7fa1900b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:14.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 >> v1:172.21.15.177:6789/0 conn(0x7fa1900b4260 legacy=0x7fa19006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:14.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19006dba0 con 0x7fa19015e390 2024-04-14T08:40:14.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.724+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fa178001840 con 0x7fa19015e390 2024-04-14T08:40:14.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 >> v1:172.21.15.172:6789/0 conn(0x7fa19015e390 legacy=0x7fa19015f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:14.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa18effd640 1 --2- 172.21.15.172:0/2317925693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa17c0039b0 0x7fa17c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:14.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa18effd640 1 --2- 172.21.15.172:0/2317925693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa17c0043b0 0x7fa17c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa18effd640 1 --2- 172.21.15.172:0/2317925693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa17c005120 0x7fa17c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1900d0080 con 0x7fa17c005120 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c00c060 con 0x7fa17c0043b0 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa17c00c260 con 0x7fa17c0039b0 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa194940640 1 --2- 172.21.15.172:0/2317925693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa17c005120 0x7fa17c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa18ffff640 1 --2- 172.21.15.172:0/2317925693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa17c0039b0 0x7fa17c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa18f7fe640 1 --2- 172.21.15.172:0/2317925693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa17c0043b0 0x7fa17c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa194940640 1 -- 172.21.15.172:0/2317925693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa17c0039b0 msgr2=0x7fa17c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa194940640 1 --2- 172.21.15.172:0/2317925693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa17c0039b0 0x7fa17c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa194940640 1 -- 172.21.15.172:0/2317925693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa17c0043b0 msgr2=0x7fa17c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa194940640 1 --2- 172.21.15.172:0/2317925693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa17c0043b0 0x7fa17c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.725+0000 7fa194940640 1 -- 172.21.15.172:0/2317925693 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c003520 con 0x7fa17c005120 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.726+0000 7fa194940640 1 -- 172.21.15.172:0/2317925693 reap_dead start 2024-04-14T08:40:14.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.726+0000 7fa194940640 1 --2- 172.21.15.172:0/2317925693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa17c005120 0x7fa17c00b9c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa188002010 tx=0x7fa17c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06047ccc09670f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:14.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.726+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188001be0 con 0x7fa17c005120 2024-04-14T08:40:14.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.726+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa188003b70 con 0x7fa17c005120 2024-04-14T08:40:14.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.726+0000 7fa18effd640 1 -- 172.21.15.172:0/2317925693 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188005110 con 0x7fa17c005120 2024-04-14T08:40:14.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.726+0000 7fa1963a3640 1 -- 172.21.15.172:0/2317925693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa17c005120 msgr2=0x7fa17c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:14.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.726+0000 7fa1963a3640 1 --2- 172.21.15.172:0/2317925693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa17c005120 0x7fa17c00b9c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa188002010 tx=0x7fa17c005830 comp rx=0 tx=0).stop 2024-04-14T08:40:14.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.726+0000 7fa1963a3640 1 -- 172.21.15.172:0/2317925693 shutdown_connections 2024-04-14T08:40:14.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.726+0000 7fa1963a3640 1 --2- 172.21.15.172:0/2317925693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa17c005120 0x7fa17c00b9c0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:14.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.726+0000 7fa1963a3640 1 -- 172.21.15.172:0/2317925693 >> 172.21.15.172:0/2317925693 conn(0x7fa19006f820 msgr2=0x7fa19006fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:14.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.726+0000 7fa1963a3640 1 -- 172.21.15.172:0/2317925693 shutdown_connections 2024-04-14T08:40:14.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.726+0000 7fa1963a3640 1 -- 172.21.15.172:0/2317925693 wait complete. 2024-04-14T08:40:14.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.728+0000 7fa1963a3640 1 Processor -- start 2024-04-14T08:40:14.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.728+0000 7fa1963a3640 1 -- start start 2024-04-14T08:40:14.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.728+0000 7fa1963a3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa17c005120 0x7fa190156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:14.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.728+0000 7fa1963a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa19015b6a0 0x7fa19015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:14.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.728+0000 7fa1963a3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa19015d560 0x7fa1901e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:14.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.728+0000 7fa1963a3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1900d0510 con 0x7fa19015b6a0 2024-04-14T08:40:14.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.728+0000 7fa1963a3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1900b4690 con 0x7fa19015d560 2024-04-14T08:40:14.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.728+0000 7fa1963a3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa1900b3d20 con 0x7fa17c005120 2024-04-14T08:40:14.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa19015b6a0 0x7fa19015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:14.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa19015b6a0 0x7fa19015d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42136/0 (socket says 172.21.15.172:42136) 2024-04-14T08:40:14.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18f7fe640 1 -- 172.21.15.172:0/1657325082 learned_addr learned my addr 172.21.15.172:0/1657325082 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:14.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18ffff640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa17c005120 0x7fa190156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:14.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa194940640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa19015d560 0x7fa1901e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:14.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18f7fe640 1 -- 172.21.15.172:0/1657325082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa17c005120 msgr2=0x7fa190156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:14.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18f7fe640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa17c005120 0x7fa190156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:14.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18f7fe640 1 -- 172.21.15.172:0/1657325082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa19015d560 msgr2=0x7fa1901e8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:14.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18f7fe640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa19015d560 0x7fa1901e8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:14.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18f7fe640 1 -- 172.21.15.172:0/1657325082 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19015a600 con 0x7fa19015b6a0 2024-04-14T08:40:14.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18f7fe640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa19015b6a0 0x7fa19015d020 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fa1800118e0 tx=0x7fa180011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:14.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18cff9640 1 -- 172.21.15.172:0/1657325082 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18000f9d0 con 0x7fa19015b6a0 2024-04-14T08:40:14.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18cff9640 1 -- 172.21.15.172:0/1657325082 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa18000fb70 con 0x7fa19015b6a0 2024-04-14T08:40:14.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa18cff9640 1 -- 172.21.15.172:0/1657325082 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180014440 con 0x7fa19015b6a0 2024-04-14T08:40:14.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa1963a3640 1 -- 172.21.15.172:0/1657325082 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19015a8e0 con 0x7fa19015b6a0 2024-04-14T08:40:14.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.729+0000 7fa1963a3640 1 -- 172.21.15.172:0/1657325082 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa19015aea0 con 0x7fa19015b6a0 2024-04-14T08:40:14.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.730+0000 7fa1963a3640 1 -- 172.21.15.172:0/1657325082 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa190068520 con 0x7fa19015b6a0 2024-04-14T08:40:14.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.731+0000 7fa18cff9640 1 -- 172.21.15.172:0/1657325082 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa180012040 con 0x7fa19015b6a0 2024-04-14T08:40:14.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.731+0000 7fa18cff9640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa17006c920 0x7fa17006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:14.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.731+0000 7fa18cff9640 1 -- 172.21.15.172:0/1657325082 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fa18008c1a0 con 0x7fa19015b6a0 2024-04-14T08:40:14.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.731+0000 7fa18ffff640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa17006c920 0x7fa17006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:14.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.732+0000 7fa18ffff640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa17006c920 0x7fa17006ede0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa178000ed0 tx=0x7fa178007d70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:14.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.736+0000 7fa18cff9640 1 -- 172.21.15.172:0/1657325082 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa180019030 con 0x7fa19015b6a0 2024-04-14T08:40:14.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.956+0000 7fa1963a3640 1 -- 172.21.15.172:0/1657325082 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa19015d940 con 0x7fa19015b6a0 2024-04-14T08:40:14.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.956+0000 7fa18cff9640 1 -- 172.21.15.172:0/1657325082 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5196 (secure 0 0 0) 0x7fa18005a320 con 0x7fa19015b6a0 2024-04-14T08:40:14.957 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:14.957 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:40:14.122456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8013":{"gid":8013,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.172:6837/3091771014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3091771014},{"type":"v1","addr":"172.21.15.172:6837","nonce":3091771014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8013,"qdb_cluster":[8013]},"id":6}]} 2024-04-14T08:40:14.957 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 67 2024-04-14T08:40:14.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.958+0000 7fa1963a3640 1 -- 172.21.15.172:0/1657325082 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa17006c920 msgr2=0x7fa17006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:14.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.958+0000 7fa1963a3640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa17006c920 0x7fa17006ede0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa178000ed0 tx=0x7fa178007d70 comp rx=0 tx=0).stop 2024-04-14T08:40:14.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.958+0000 7fa1963a3640 1 -- 172.21.15.172:0/1657325082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa19015b6a0 msgr2=0x7fa19015d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:14.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.958+0000 7fa1963a3640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa19015b6a0 0x7fa19015d020 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fa1800118e0 tx=0x7fa180011ce0 comp rx=0 tx=0).stop 2024-04-14T08:40:14.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.958+0000 7fa1963a3640 1 -- 172.21.15.172:0/1657325082 shutdown_connections 2024-04-14T08:40:14.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.958+0000 7fa1963a3640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa17006c920 0x7fa17006ede0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:14.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.958+0000 7fa1963a3640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa19015d560 0x7fa1901e8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:14.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.958+0000 7fa1963a3640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa19015b6a0 0x7fa19015d020 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:14.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.958+0000 7fa1963a3640 1 --2- 172.21.15.172:0/1657325082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa17c005120 0x7fa190156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:14.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.958+0000 7fa1963a3640 1 -- 172.21.15.172:0/1657325082 >> 172.21.15.172:0/1657325082 conn(0x7fa19006f820 msgr2=0x7fa19006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:14.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.958+0000 7fa1963a3640 1 -- 172.21.15.172:0/1657325082 shutdown_connections 2024-04-14T08:40:14.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:14.958+0000 7fa1963a3640 1 -- 172.21.15.172:0/1657325082 wait complete. 2024-04-14T08:40:15.373 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:38:44.572424+0000', 'modified': '2024-04-14T08:40:14.122456+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8013}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8013': {'gid': 8013, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.172:6837/3091771014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 3091771014}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 3091771014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8013, 'qdb_cluster': [8013]} 2024-04-14T08:40:15.373 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:40:15.374 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-14T08:40:15.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.476+0000 7f532a795640 1 Processor -- start 2024-04-14T08:40:15.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.477+0000 7f532a795640 1 -- start start 2024-04-14T08:40:15.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.477+0000 7f532a795640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5324162710 0x7f53240d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.477+0000 7f532a795640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53240cefb0 con 0x7f532415e430 2024-04-14T08:40:15.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.477+0000 7f532a795640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53240cf290 con 0x7f5324162ba0 2024-04-14T08:40:15.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.477+0000 7f532a795640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f53240cf570 con 0x7f5324162710 2024-04-14T08:40:15.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.477+0000 7f5323fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f532415e430 0x7f5324162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50356/0 (socket says 172.21.15.172:50356) 2024-04-14T08:40:15.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.477+0000 7f5323fff640 1 -- 172.21.15.172:0/1954998286 learned_addr learned my addr 172.21.15.172:0/1954998286 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:15.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f53237fe640 1 --2- 172.21.15.172:0/1954998286 >> v2:172.21.15.177:3300/0 conn(0x7f5324162710 0x7f53240d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3247704816 0 0) 0x7f53240cefb0 con 0x7f532415e430 2024-04-14T08:40:15.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f530c003520 con 0x7f532415e430 2024-04-14T08:40:15.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3308462516 0 0) 0x7f530c003520 con 0x7f532415e430 2024-04-14T08:40:15.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 >> v2:172.21.15.177:3300/0 conn(0x7f5324162710 msgr2=0x7f53240d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 --2- 172.21.15.172:0/1954998286 >> v2:172.21.15.177:3300/0 conn(0x7f5324162710 0x7f53240d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 >> v1:172.21.15.177:6789/0 conn(0x7f5324162ba0 legacy=0x7f53240d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53240cf850 con 0x7f532415e430 2024-04-14T08:40:15.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f5314001670 con 0x7f532415e430 2024-04-14T08:40:15.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 >> v1:172.21.15.172:6789/0 conn(0x7f532415e430 legacy=0x7f5324162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 --2- 172.21.15.172:0/1954998286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f530c0039b0 0x7f530c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 --2- 172.21.15.172:0/1954998286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f530c0043b0 0x7f530c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 --2- 172.21.15.172:0/1954998286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f530c005080 0x7f530c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53240cf570 con 0x7f530c0039b0 2024-04-14T08:40:15.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f530c00bfe0 con 0x7f530c005080 2024-04-14T08:40:15.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f530c00c1e0 con 0x7f530c0043b0 2024-04-14T08:40:15.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5323fff640 1 --2- 172.21.15.172:0/1954998286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f530c0039b0 0x7f530c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f53237fe640 1 --2- 172.21.15.172:0/1954998286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f530c0043b0 0x7f530c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5328d32640 1 --2- 172.21.15.172:0/1954998286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f530c005080 0x7f530c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5323fff640 1 -- 172.21.15.172:0/1954998286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f530c0043b0 msgr2=0x7f530c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5323fff640 1 --2- 172.21.15.172:0/1954998286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f530c0043b0 0x7f530c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5323fff640 1 -- 172.21.15.172:0/1954998286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f530c005080 msgr2=0x7f530c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5323fff640 1 --2- 172.21.15.172:0/1954998286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f530c005080 0x7f530c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5323fff640 1 -- 172.21.15.172:0/1954998286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530c003520 con 0x7f530c0039b0 2024-04-14T08:40:15.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.478+0000 7f5328d32640 1 -- 172.21.15.172:0/1954998286 reap_dead start 2024-04-14T08:40:15.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.479+0000 7f5323fff640 1 --2- 172.21.15.172:0/1954998286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f530c0039b0 0x7f530c003d90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f531400c240 tx=0x7f530c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868724d90aee0743 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:15.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.479+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314007d30 con 0x7f530c0039b0 2024-04-14T08:40:15.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.479+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5314030070 con 0x7f530c0039b0 2024-04-14T08:40:15.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.479+0000 7f5322ffd640 1 -- 172.21.15.172:0/1954998286 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531402cda0 con 0x7f530c0039b0 2024-04-14T08:40:15.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.480+0000 7f532a795640 1 -- 172.21.15.172:0/1954998286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f530c0039b0 msgr2=0x7f530c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.480+0000 7f532a795640 1 --2- 172.21.15.172:0/1954998286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f530c0039b0 0x7f530c003d90 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f531400c240 tx=0x7f530c005830 comp rx=0 tx=0).stop 2024-04-14T08:40:15.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.480+0000 7f532a795640 1 -- 172.21.15.172:0/1954998286 shutdown_connections 2024-04-14T08:40:15.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.480+0000 7f532a795640 1 --2- 172.21.15.172:0/1954998286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f530c0039b0 0x7f530c003d90 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.480+0000 7f532a795640 1 -- 172.21.15.172:0/1954998286 >> 172.21.15.172:0/1954998286 conn(0x7f532409d560 msgr2=0x7f53240d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:15.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.480+0000 7f532a795640 1 -- 172.21.15.172:0/1954998286 shutdown_connections 2024-04-14T08:40:15.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.480+0000 7f532a795640 1 -- 172.21.15.172:0/1954998286 wait complete. 2024-04-14T08:40:15.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.481+0000 7f532a795640 1 Processor -- start 2024-04-14T08:40:15.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.481+0000 7f532a795640 1 -- start start 2024-04-14T08:40:15.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f532a795640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f530c0039b0 0x7f53240b7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f532a795640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53240cc340 0x7f53240cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f532a795640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53240b9470 0x7f53240b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f532a795640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53240d4e20 con 0x7f53240b9470 2024-04-14T08:40:15.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f532a795640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f532415e810 con 0x7f530c0039b0 2024-04-14T08:40:15.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f532a795640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f53240d0050 con 0x7f53240cc340 2024-04-14T08:40:15.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f5328d32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53240b9470 0x7f53240b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f5328d32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53240b9470 0x7f53240b98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42162/0 (socket says 172.21.15.172:42162) 2024-04-14T08:40:15.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f5328d32640 1 -- 172.21.15.172:0/2588347465 learned_addr learned my addr 172.21.15.172:0/2588347465 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:15.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f53237fe640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53240cc340 0x7f53240cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f5323fff640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f530c0039b0 0x7f53240b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f5328d32640 1 -- 172.21.15.172:0/2588347465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53240cc340 msgr2=0x7f53240cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f5328d32640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53240cc340 0x7f53240cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f5328d32640 1 -- 172.21.15.172:0/2588347465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f530c0039b0 msgr2=0x7f53240b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f5328d32640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f530c0039b0 0x7f53240b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f5328d32640 1 -- 172.21.15.172:0/2588347465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53240cd000 con 0x7f53240b9470 2024-04-14T08:40:15.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.482+0000 7f53237fe640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53240cc340 0x7f53240cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:15.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.483+0000 7f5328d32640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53240b9470 0x7f53240b98f0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f53240a2180 tx=0x7f531c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:15.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.483+0000 7f5320ff9640 1 -- 172.21.15.172:0/2588347465 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c003970 con 0x7f53240b9470 2024-04-14T08:40:15.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.483+0000 7f5320ff9640 1 -- 172.21.15.172:0/2588347465 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531c018070 con 0x7f53240b9470 2024-04-14T08:40:15.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.483+0000 7f532a795640 1 -- 172.21.15.172:0/2588347465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53240cd2e0 con 0x7f53240b9470 2024-04-14T08:40:15.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.483+0000 7f5320ff9640 1 -- 172.21.15.172:0/2588347465 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c003d50 con 0x7f53240b9470 2024-04-14T08:40:15.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.483+0000 7f532a795640 1 -- 172.21.15.172:0/2588347465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53240cd8a0 con 0x7f53240b9470 2024-04-14T08:40:15.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.484+0000 7f5320ff9640 1 -- 172.21.15.172:0/2588347465 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f531c0050d0 con 0x7f53240b9470 2024-04-14T08:40:15.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.485+0000 7f5320ff9640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f531006c8f0 0x7f531006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.485+0000 7f5320ff9640 1 -- 172.21.15.172:0/2588347465 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f531c08bbc0 con 0x7f53240b9470 2024-04-14T08:40:15.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.485+0000 7f5323fff640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f531006c8f0 0x7f531006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.485+0000 7f5323fff640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f531006c8f0 0x7f531006edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5314034000 tx=0x7f5314009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:15.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.486+0000 7f532d577740 1 Processor -- start 2024-04-14T08:40:15.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.486+0000 7f532d577740 1 -- start start 2024-04-14T08:40:15.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.486+0000 7f532d577740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 0x55ddb31e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.486+0000 7f532d577740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320df40 0x55ddb32108f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f532d577740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb31e1040 0x55ddb3210e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f532d577740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ddb314a030 con 0x55ddb31e1040 2024-04-14T08:40:15.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f532d577740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ddb30f2620 con 0x55ddb320db60 2024-04-14T08:40:15.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f532d577740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ddb3145fe0 con 0x55ddb320df40 2024-04-14T08:40:15.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5328d32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb31e1040 0x55ddb3210e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5328d32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb31e1040 0x55ddb3210e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42176/0 (socket says 172.21.15.172:42176) 2024-04-14T08:40:15.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5328d32640 1 -- 172.21.15.172:0/1677838007 learned_addr learned my addr 172.21.15.172:0/1677838007 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:15.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5323fff640 1 --2- 172.21.15.172:0/1677838007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 0x55ddb31e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f53237fe640 1 --2- 172.21.15.172:0/1677838007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320df40 0x55ddb32108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5328d32640 1 -- 172.21.15.172:0/1677838007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320df40 msgr2=0x55ddb32108f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5328d32640 1 --2- 172.21.15.172:0/1677838007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320df40 0x55ddb32108f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5328d32640 1 -- 172.21.15.172:0/1677838007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 msgr2=0x55ddb31e0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5328d32640 1 --2- 172.21.15.172:0/1677838007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 0x55ddb31e0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5328d32640 1 -- 172.21.15.172:0/1677838007 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb2e5e2d0 con 0x55ddb31e1040 2024-04-14T08:40:15.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5328d32640 1 --2- 172.21.15.172:0/1677838007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb31e1040 0x55ddb3210e30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f531c089db0 tx=0x7f531c059d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fca7b52cda38141 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:15.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5309ffb640 1 -- 172.21.15.172:0/1677838007 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c088070 con 0x55ddb31e1040 2024-04-14T08:40:15.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5309ffb640 1 -- 172.21.15.172:0/1677838007 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531c05d9f0 con 0x55ddb31e1040 2024-04-14T08:40:15.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.487+0000 7f5309ffb640 1 -- 172.21.15.172:0/1677838007 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c062220 con 0x55ddb31e1040 2024-04-14T08:40:15.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.488+0000 7f532d577740 1 -- 172.21.15.172:0/1677838007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb31e1040 msgr2=0x55ddb3210e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.488+0000 7f532d577740 1 --2- 172.21.15.172:0/1677838007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb31e1040 0x55ddb3210e30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f531c089db0 tx=0x7f531c059d40 comp rx=0 tx=0).stop 2024-04-14T08:40:15.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.488+0000 7f532d577740 1 -- 172.21.15.172:0/1677838007 shutdown_connections 2024-04-14T08:40:15.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.488+0000 7f532d577740 1 --2- 172.21.15.172:0/1677838007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb31e1040 0x55ddb3210e30 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.488+0000 7f532d577740 1 --2- 172.21.15.172:0/1677838007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320df40 0x55ddb32108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.488+0000 7f532d577740 1 --2- 172.21.15.172:0/1677838007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 0x55ddb31e0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.488+0000 7f532d577740 1 -- 172.21.15.172:0/1677838007 >> 172.21.15.172:0/1677838007 conn(0x55ddb31e2d70 msgr2=0x55ddb322b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:15.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.488+0000 7f532d577740 1 -- 172.21.15.172:0/1677838007 shutdown_connections 2024-04-14T08:40:15.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.488+0000 7f532d577740 1 -- 172.21.15.172:0/1677838007 wait complete. 2024-04-14T08:40:15.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.489+0000 7f532d577740 1 -- start start 2024-04-14T08:40:15.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.489+0000 7f532d577740 1 Processor -- start 2024-04-14T08:40:15.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f532d577740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 0x55ddb3206960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f532d577740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320df40 0x55ddb3206ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f532d577740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 0x55ddb3212a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f532d577740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ddb31e5410 con 0x55ddb3208a20 2024-04-14T08:40:15.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f532d577740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ddb31059e0 con 0x55ddb320db60 2024-04-14T08:40:15.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f532d577740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ddb31e7c50 con 0x55ddb320df40 2024-04-14T08:40:15.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f532d577740 10 client.0 _ll_register_callbacks cb 0x55ddb3142080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:15.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5328d32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 0x55ddb3212a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5328d32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 0x55ddb3212a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42182/0 (socket says 172.21.15.172:42182) 2024-04-14T08:40:15.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5328d32640 1 -- 172.21.15.172:0/674855750 learned_addr learned my addr 172.21.15.172:0/674855750 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:15.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f53237fe640 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320df40 0x55ddb3206ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5323fff640 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 0x55ddb3206960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5328d32640 1 -- 172.21.15.172:0/674855750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320df40 msgr2=0x55ddb3206ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5328d32640 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320df40 0x55ddb3206ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5328d32640 1 -- 172.21.15.172:0/674855750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 msgr2=0x55ddb3206960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5328d32640 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 0x55ddb3206960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5328d32640 1 -- 172.21.15.172:0/674855750 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb2f286b0 con 0x55ddb3208a20 2024-04-14T08:40:15.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5323fff640 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 0x55ddb3206960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:15.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5328d32640 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 0x55ddb3212a70 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f531c059800 tx=0x7f531c001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d82b9f06cf030f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:15.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5302ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:15.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5302ffd640 1 -- 172.21.15.172:0/674855750 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c088030 con 0x55ddb3208a20 2024-04-14T08:40:15.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5302ffd640 1 -- 172.21.15.172:0/674855750 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f531c055cb0 con 0x55ddb3208a20 2024-04-14T08:40:15.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f532d577740 1 -- 172.21.15.172:0/674855750 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f531c05d9f0 con 0x55ddb3208a20 2024-04-14T08:40:15.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f5302ffd640 1 -- 172.21.15.172:0/674855750 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f531c05eb80 con 0x55ddb3208a20 2024-04-14T08:40:15.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f532d577740 10 client.8088 fetch_fsmap learned FSMap version 67 2024-04-14T08:40:15.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.490+0000 7f532d577740 1 -- 172.21.15.172:0/674855750 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55ddb3212550 con 0x55ddb3208a20 2024-04-14T08:40:15.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.491+0000 7f5302ffd640 1 -- 172.21.15.172:0/674855750 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531c05d9f0 con 0x55ddb3208a20 2024-04-14T08:40:15.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.491+0000 7f5302ffd640 1 -- 172.21.15.172:0/674855750 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f531c058a90 con 0x55ddb3208a20 2024-04-14T08:40:15.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.491+0000 7f532d577740 10 client.8088 fetch_fsmap finished waiting for FSMap version 67 2024-04-14T08:40:15.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.491+0000 7f532d577740 10 client.8088 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:40:15.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.491+0000 7f532d577740 20 client.8088 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:15.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.491+0000 7f532d577740 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55ddb323c180 0x55ddb323c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.491+0000 7f532d577740 4 client.8088 mds_command: new command op to 8013 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:40:15.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.491+0000 7f532d577740 1 -- 172.21.15.172:0/674855750 --> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddb30f1cc0 con 0x55ddb323c180 2024-04-14T08:40:15.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.491+0000 7f5323fff640 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55ddb323c180 0x55ddb323c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.491+0000 7f5302ffd640 10 client.8088 ms_handle_connect on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:15.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.491+0000 7f5323fff640 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55ddb323c180 0x55ddb323c560 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4984b68c29f9fda server_cookie=8d44bb5e01c19f9f in_seq=0 out_seq=0 2024-04-14T08:40:15.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.492+0000 7f5302ffd640 1 -- 172.21.15.172:0/674855750 <== mds.0 v2:172.21.15.172:6835/3091771014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ddb30f1cc0 con 0x55ddb323c180 2024-04-14T08:40:15.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.492+0000 7f5302ffd640 10 client.8088 handle_command_reply: tid=0 2024-04-14T08:40:15.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.492+0000 7f532d577740 1 client.8088 shutdown 2024-04-14T08:40:15.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.492+0000 7f532d577740 1 -- 172.21.15.172:0/674855750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 msgr2=0x55ddb3212a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.492+0000 7f532d577740 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 0x55ddb3212a70 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f531c059800 tx=0x7f531c001310 comp rx=0 tx=0).stop 2024-04-14T08:40:15.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.492+0000 7f532d577740 1 -- 172.21.15.172:0/674855750 shutdown_connections 2024-04-14T08:40:15.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.492+0000 7f532d577740 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55ddb323c180 0x55ddb323c560 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.492+0000 7f532d577740 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 0x55ddb3212a70 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.492+0000 7f5302ffd640 0 client.8088 ms_handle_reset on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:15.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.492+0000 7f532d577740 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320df40 0x55ddb3206ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.492+0000 7f532d577740 1 --2- 172.21.15.172:0/674855750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 0x55ddb3206960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.492+0000 7f532d577740 1 -- 172.21.15.172:0/674855750 >> 172.21.15.172:0/674855750 conn(0x55ddb31e1040 msgr2=0x55ddb31e4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:15.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.493+0000 7f532d577740 1 -- 172.21.15.172:0/674855750 shutdown_connections 2024-04-14T08:40:15.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.493+0000 7f532d577740 1 -- 172.21.15.172:0/674855750 wait complete. 2024-04-14T08:40:15.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.493+0000 7f532d577740 20 client.8088 trim_cache size 0 max 16384 2024-04-14T08:40:15.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.527+0000 7f532d577740 1 Processor -- start 2024-04-14T08:40:15.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.527+0000 7f532d577740 1 -- start start 2024-04-14T08:40:15.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.528+0000 7f532d577740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 0x55ddb3207a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.528+0000 7f532d577740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320db60 0x55ddb323e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.528+0000 7f532d577740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320df40 0x55ddb31e4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.528+0000 7f532d577740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5324200880 con 0x55ddb3208a20 2024-04-14T08:40:15.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f532d577740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f53241ffbe0 con 0x55ddb320df40 2024-04-14T08:40:15.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f532d577740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5324200230 con 0x55ddb320db60 2024-04-14T08:40:15.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f5323fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 0x55ddb3207a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f5323fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 0x55ddb3207a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42192/0 (socket says 172.21.15.172:42192) 2024-04-14T08:40:15.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f5323fff640 1 -- 172.21.15.172:0/1317423025 learned_addr learned my addr 172.21.15.172:0/1317423025 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:15.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f53237fe640 1 --2- 172.21.15.172:0/1317423025 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320db60 0x55ddb323e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f5328d32640 1 --2- 172.21.15.172:0/1317423025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320df40 0x55ddb31e4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f5323fff640 1 -- 172.21.15.172:0/1317423025 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320db60 msgr2=0x55ddb323e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f5323fff640 1 --2- 172.21.15.172:0/1317423025 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320db60 0x55ddb323e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f5323fff640 1 -- 172.21.15.172:0/1317423025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320df40 msgr2=0x55ddb31e4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f5323fff640 1 --2- 172.21.15.172:0/1317423025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320df40 0x55ddb31e4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f5323fff640 1 -- 172.21.15.172:0/1317423025 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb31e3a00 con 0x55ddb3208a20 2024-04-14T08:40:15.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f5328d32640 1 --2- 172.21.15.172:0/1317423025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320df40 0x55ddb31e4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:15.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f5323fff640 1 --2- 172.21.15.172:0/1317423025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 0x55ddb3207a80 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f531400c330 tx=0x7f5314007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d3043118270bbb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:15.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f53097fa640 1 -- 172.21.15.172:0/1317423025 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314056070 con 0x55ddb3208a20 2024-04-14T08:40:15.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f53097fa640 1 -- 172.21.15.172:0/1317423025 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5314030040 con 0x55ddb3208a20 2024-04-14T08:40:15.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.529+0000 7f53097fa640 1 -- 172.21.15.172:0/1317423025 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53140072b0 con 0x55ddb3208a20 2024-04-14T08:40:15.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.530+0000 7f532d577740 1 -- 172.21.15.172:0/1317423025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 msgr2=0x55ddb3207a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.530+0000 7f532d577740 1 --2- 172.21.15.172:0/1317423025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 0x55ddb3207a80 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f531400c330 tx=0x7f5314007000 comp rx=0 tx=0).stop 2024-04-14T08:40:15.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.530+0000 7f532d577740 1 -- 172.21.15.172:0/1317423025 shutdown_connections 2024-04-14T08:40:15.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.530+0000 7f532d577740 1 --2- 172.21.15.172:0/1317423025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320df40 0x55ddb31e4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.530+0000 7f532d577740 1 --2- 172.21.15.172:0/1317423025 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb320db60 0x55ddb323e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.530+0000 7f532d577740 1 --2- 172.21.15.172:0/1317423025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb3208a20 0x55ddb3207a80 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.530+0000 7f532d577740 1 -- 172.21.15.172:0/1317423025 >> 172.21.15.172:0/1317423025 conn(0x55ddb31e1040 msgr2=0x55ddb31dbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:15.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.530+0000 7f532d577740 1 -- 172.21.15.172:0/1317423025 shutdown_connections 2024-04-14T08:40:15.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.530+0000 7f532d577740 1 -- 172.21.15.172:0/1317423025 wait complete. 2024-04-14T08:40:15.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.531+0000 7f532d577740 1 -- start start 2024-04-14T08:40:15.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.531+0000 7f532d577740 1 Processor -- start 2024-04-14T08:40:15.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.532+0000 7f532d577740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb3208a20 0x55ddb32a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.532+0000 7f532d577740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 0x55ddb323a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.532+0000 7f532d577740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb320df40 0x55ddb31e5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.532+0000 7f532d577740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ddb3213af0 con 0x55ddb320df40 2024-04-14T08:40:15.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.532+0000 7f532d577740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ddb31e5410 con 0x55ddb320db60 2024-04-14T08:40:15.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.532+0000 7f532d577740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ddb32131f0 con 0x55ddb3208a20 2024-04-14T08:40:15.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.532+0000 7f532d577740 10 client.0 _ll_register_callbacks cb 0x7f531c086070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:15.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.532+0000 7f5328d32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb320df40 0x55ddb31e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.532+0000 7f5328d32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb320df40 0x55ddb31e5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42194/0 (socket says 172.21.15.172:42194) 2024-04-14T08:40:15.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.532+0000 7f5328d32640 1 -- 172.21.15.172:0/2014770192 learned_addr learned my addr 172.21.15.172:0/2014770192 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:15.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.532+0000 7f5328d32640 1 -- 172.21.15.172:0/2014770192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb3208a20 msgr2=0x55ddb32a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f5328d32640 1 --2- 172.21.15.172:0/2014770192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb3208a20 0x55ddb32a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f5328d32640 1 -- 172.21.15.172:0/2014770192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 msgr2=0x55ddb323a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f5328d32640 1 --2- 172.21.15.172:0/2014770192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 0x55ddb323a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f5328d32640 1 -- 172.21.15.172:0/2014770192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb324e890 con 0x55ddb320df40 2024-04-14T08:40:15.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f5328d32640 1 --2- 172.21.15.172:0/2014770192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb320df40 0x55ddb31e5590 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f531c0050a0 tx=0x7f531c0015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e94ab816abf4624 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:15.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f5302ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:15.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f5302ffd640 1 -- 172.21.15.172:0/2014770192 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c088070 con 0x55ddb320df40 2024-04-14T08:40:15.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f5302ffd640 1 -- 172.21.15.172:0/2014770192 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f531c058710 con 0x55ddb320df40 2024-04-14T08:40:15.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f532d577740 1 -- 172.21.15.172:0/2014770192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5314030040 con 0x55ddb320df40 2024-04-14T08:40:15.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f5302ffd640 1 -- 172.21.15.172:0/2014770192 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f531c0558e0 con 0x55ddb320df40 2024-04-14T08:40:15.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f532d577740 10 client.8094 fetch_fsmap learned FSMap version 67 2024-04-14T08:40:15.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f532d577740 1 -- 172.21.15.172:0/2014770192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55ddb324f930 con 0x55ddb320df40 2024-04-14T08:40:15.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f5302ffd640 1 -- 172.21.15.172:0/2014770192 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5314030040 con 0x55ddb320df40 2024-04-14T08:40:15.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f5302ffd640 1 -- 172.21.15.172:0/2014770192 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f531c0053c0 con 0x55ddb320df40 2024-04-14T08:40:15.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f532d577740 10 client.8094 fetch_fsmap finished waiting for FSMap version 67 2024-04-14T08:40:15.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f532d577740 10 client.8094 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:40:15.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f532d577740 20 client.8094 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:15.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f532d577740 1 --2- 172.21.15.172:0/2014770192 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55ddb32141d0 0x55ddb322ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f532d577740 4 client.8094 mds_command: new command op to 8013 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:40:15.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f532d577740 1 -- 172.21.15.172:0/2014770192 --> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] -- command(tid 0: {"prefix": "status"}) -- 0x55ddb3157570 con 0x55ddb32141d0 2024-04-14T08:40:15.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.533+0000 7f5323fff640 1 --2- 172.21.15.172:0/2014770192 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55ddb32141d0 0x55ddb322ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f5302ffd640 10 client.8094 ms_handle_connect on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:15.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f5323fff640 1 --2- 172.21.15.172:0/2014770192 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55ddb32141d0 0x55ddb322ea10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1312c86a2031972 server_cookie=9a991b32892f1e38 in_seq=0 out_seq=0 2024-04-14T08:40:15.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f5302ffd640 1 -- 172.21.15.172:0/2014770192 <== mds.0 v2:172.21.15.172:6835/3091771014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ddb3157570 con 0x55ddb32141d0 2024-04-14T08:40:15.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f5302ffd640 10 client.8094 handle_command_reply: tid=0 2024-04-14T08:40:15.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f532d577740 1 client.8094 shutdown 2024-04-14T08:40:15.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f532d577740 1 -- 172.21.15.172:0/2014770192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb320df40 msgr2=0x55ddb31e5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f532d577740 1 --2- 172.21.15.172:0/2014770192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb320df40 0x55ddb31e5590 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f531c0050a0 tx=0x7f531c0015b0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f532d577740 1 -- 172.21.15.172:0/2014770192 shutdown_connections 2024-04-14T08:40:15.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f532d577740 1 --2- 172.21.15.172:0/2014770192 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55ddb32141d0 0x55ddb322ea10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f532d577740 1 --2- 172.21.15.172:0/2014770192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ddb320df40 0x55ddb31e5590 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f532d577740 1 --2- 172.21.15.172:0/2014770192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ddb320db60 0x55ddb323a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f532d577740 1 --2- 172.21.15.172:0/2014770192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ddb3208a20 0x55ddb32a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f532d577740 1 -- 172.21.15.172:0/2014770192 >> 172.21.15.172:0/2014770192 conn(0x55ddb31e1040 msgr2=0x55ddb323ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:15.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f5302ffd640 0 client.8094 ms_handle_reset on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:15.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f532d577740 1 -- 172.21.15.172:0/2014770192 shutdown_connections 2024-04-14T08:40:15.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f532d577740 1 -- 172.21.15.172:0/2014770192 wait complete. 2024-04-14T08:40:15.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.534+0000 7f532d577740 20 client.8094 trim_cache size 0 max 16384 2024-04-14T08:40:15.545 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:40:15.545 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:40:15.545 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:40:15.545 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 8013, 2024-04-14T08:40:15.545 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:40:15.545 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:40:15.545 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:40:15.545 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 4.4184483370000001, 2024-04-14T08:40:15.545 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 67, 2024-04-14T08:40:15.545 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 90, 2024-04-14T08:40:15.545 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 90, 2024-04-14T08:40:15.546 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 5.3906733339999997 2024-04-14T08:40:15.546 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:40:15.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.535+0000 7f532a795640 1 -- 172.21.15.172:0/2588347465 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f531006c8f0 msgr2=0x7f531006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.535+0000 7f532a795640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f531006c8f0 0x7f531006edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5314034000 tx=0x7f5314009030 comp rx=0 tx=0).stop 2024-04-14T08:40:15.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.535+0000 7f532a795640 1 -- 172.21.15.172:0/2588347465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53240b9470 msgr2=0x7f53240b98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.535+0000 7f532a795640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53240b9470 0x7f53240b98f0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f53240a2180 tx=0x7f531c00eb90 comp rx=0 tx=0).stop 2024-04-14T08:40:15.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.535+0000 7f532a795640 1 -- 172.21.15.172:0/2588347465 shutdown_connections 2024-04-14T08:40:15.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.535+0000 7f532a795640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f531006c8f0 0x7f531006edb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.535+0000 7f532a795640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53240b9470 0x7f53240b98f0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.535+0000 7f532a795640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53240cc340 0x7f53240cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.535+0000 7f532a795640 1 --2- 172.21.15.172:0/2588347465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f530c0039b0 0x7f53240b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.535+0000 7f532a795640 1 -- 172.21.15.172:0/2588347465 >> 172.21.15.172:0/2588347465 conn(0x7f532409d560 msgr2=0x7f532406c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:15.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.535+0000 7f532a795640 1 -- 172.21.15.172:0/2588347465 shutdown_connections 2024-04-14T08:40:15.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.535+0000 7f532a795640 1 -- 172.21.15.172:0/2588347465 wait complete. 2024-04-14T08:40:15.549 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:40:15.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.656+0000 7fbb1c58c640 1 Processor -- start 2024-04-14T08:40:15.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.656+0000 7fbb1c58c640 1 -- start start 2024-04-14T08:40:15.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb1c58c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb140d1a10 0x7fbb14161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb1c58c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb140d0200 con 0x7fbb14162530 2024-04-14T08:40:15.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb1c58c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb140cf160 con 0x7fbb1415c150 2024-04-14T08:40:15.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb1c58c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbb140cef10 con 0x7fbb140d1a10 2024-04-14T08:40:15.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb1ab29640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fbb14162530 0x7fbb140d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50372/0 (socket says 172.21.15.172:50372) 2024-04-14T08:40:15.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb1ab29640 1 -- 172.21.15.172:0/3548877842 learned_addr learned my addr 172.21.15.172:0/3548877842 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:15.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb19b27640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb140d1a10 0x7fbb14161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1782648856 0 0) 0x7fbb140d0200 con 0x7fbb14162530 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb00003520 con 0x7fbb14162530 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 185565283 0 0) 0x7fbb00003520 con 0x7fbb14162530 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb140d1a10 msgr2=0x7fbb14161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb19326640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb140d1a10 0x7fbb14161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 >> v1:172.21.15.177:6789/0 conn(0x7fbb1415c150 legacy=0x7fbb140d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb140cf440 con 0x7fbb14162530 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fbb0c001a10 con 0x7fbb14162530 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 >> v1:172.21.15.172:6789/0 conn(0x7fbb14162530 legacy=0x7fbb140d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.657+0000 7fbb19326640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb000039e0 0x7fbb00003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb19326640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb140d1a10 0x7fbb00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb19326640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb00004d40 0x7fbb000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbb140cef10 con 0x7fbb000039e0 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0000b9c0 con 0x7fbb00004d40 2024-04-14T08:40:15.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0000bbc0 con 0x7fbb140d1a10 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb1a328640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb000039e0 0x7fbb00003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb19b27640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb140d1a10 0x7fbb00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb1ab29640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb00004d40 0x7fbb000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb1a328640 1 -- 172.21.15.172:0/3548877842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb140d1a10 msgr2=0x7fbb00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb1a328640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb140d1a10 0x7fbb00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb1a328640 1 -- 172.21.15.172:0/3548877842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb00004d40 msgr2=0x7fbb000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb1a328640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb00004d40 0x7fbb000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb1a328640 1 -- 172.21.15.172:0/3548877842 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb00003520 con 0x7fbb000039e0 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.658+0000 7fbb1ab29640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb00004d40 0x7fbb000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.659+0000 7fbb1a328640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb000039e0 0x7fbb00003dc0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fbb10001860 tx=0x7fbb00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2b794b8da40814 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.659+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10038070 con 0x7fbb000039e0 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.659+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb10033660 con 0x7fbb000039e0 2024-04-14T08:40:15.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.659+0000 7fbb19326640 1 -- 172.21.15.172:0/3548877842 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10001430 con 0x7fbb000039e0 2024-04-14T08:40:15.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.659+0000 7fbb1c58c640 1 -- 172.21.15.172:0/3548877842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb000039e0 msgr2=0x7fbb00003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.659+0000 7fbb1c58c640 1 --2- 172.21.15.172:0/3548877842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb000039e0 0x7fbb00003dc0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fbb10001860 tx=0x7fbb00005830 comp rx=0 tx=0).stop 2024-04-14T08:40:15.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.659+0000 7fbb1ab29640 1 -- 172.21.15.172:0/3548877842 reap_dead start 2024-04-14T08:40:15.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.659+0000 7fbb1c58c640 1 -- 172.21.15.172:0/3548877842 shutdown_connections 2024-04-14T08:40:15.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.659+0000 7fbb1c58c640 1 -- 172.21.15.172:0/3548877842 >> 172.21.15.172:0/3548877842 conn(0x7fbb1409d580 msgr2=0x7fbb140d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:15.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.659+0000 7fbb1c58c640 1 -- 172.21.15.172:0/3548877842 shutdown_connections 2024-04-14T08:40:15.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.659+0000 7fbb1c58c640 1 -- 172.21.15.172:0/3548877842 wait complete. 2024-04-14T08:40:15.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb1c58c640 1 Processor -- start 2024-04-14T08:40:15.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb1c58c640 1 -- start start 2024-04-14T08:40:15.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb1c58c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb000039e0 0x7fbb140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb1c58c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb1415b0d0 0x7fbb1415b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb1c58c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb141e8ea0 0x7fbb141e9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb1c58c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbb140cfa00 con 0x7fbb1415b0d0 2024-04-14T08:40:15.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb1c58c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14162960 con 0x7fbb000039e0 2024-04-14T08:40:15.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb1c58c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbb140cfce0 con 0x7fbb141e8ea0 2024-04-14T08:40:15.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb19b27640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb1415b0d0 0x7fbb1415b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb19b27640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb1415b0d0 0x7fbb1415b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42226/0 (socket says 172.21.15.172:42226) 2024-04-14T08:40:15.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb19b27640 1 -- 172.21.15.172:0/360692124 learned_addr learned my addr 172.21.15.172:0/360692124 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:15.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb1ab29640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb141e8ea0 0x7fbb141e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.661+0000 7fbb1a328640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb000039e0 0x7fbb140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.662+0000 7fbb19b27640 1 -- 172.21.15.172:0/360692124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb141e8ea0 msgr2=0x7fbb141e9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.662+0000 7fbb19b27640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb141e8ea0 0x7fbb141e9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.662+0000 7fbb19b27640 1 -- 172.21.15.172:0/360692124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb000039e0 msgr2=0x7fbb140b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:15.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.662+0000 7fbb19b27640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb000039e0 0x7fbb140b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:15.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.662+0000 7fbb19b27640 1 -- 172.21.15.172:0/360692124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb140c9cd0 con 0x7fbb1415b0d0 2024-04-14T08:40:15.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.662+0000 7fbb19b27640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb1415b0d0 0x7fbb1415b550 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fbb040119d0 tx=0x7fbb04011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:15.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.662+0000 7fbb0affd640 1 -- 172.21.15.172:0/360692124 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400faf0 con 0x7fbb1415b0d0 2024-04-14T08:40:15.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.662+0000 7fbb0affd640 1 -- 172.21.15.172:0/360692124 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0400fc90 con 0x7fbb1415b0d0 2024-04-14T08:40:15.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.662+0000 7fbb1c58c640 1 -- 172.21.15.172:0/360692124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb140c7ce0 con 0x7fbb1415b0d0 2024-04-14T08:40:15.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.662+0000 7fbb0affd640 1 -- 172.21.15.172:0/360692124 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04014440 con 0x7fbb1415b0d0 2024-04-14T08:40:15.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.662+0000 7fbb1c58c640 1 -- 172.21.15.172:0/360692124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb140c82a0 con 0x7fbb1415b0d0 2024-04-14T08:40:15.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.663+0000 7fbb1c58c640 1 -- 172.21.15.172:0/360692124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb14075490 con 0x7fbb1415b0d0 2024-04-14T08:40:15.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.664+0000 7fbb0affd640 1 -- 172.21.15.172:0/360692124 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb04012040 con 0x7fbb1415b0d0 2024-04-14T08:40:15.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.664+0000 7fbb0affd640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbaf806c8f0 0x7fbaf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:15.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.664+0000 7fbb0affd640 1 -- 172.21.15.172:0/360692124 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fbb0408c2c0 con 0x7fbb1415b0d0 2024-04-14T08:40:15.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.664+0000 7fbb1a328640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbaf806c8f0 0x7fbaf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:15.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.664+0000 7fbb1a328640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbaf806c8f0 0x7fbaf806edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbb10035000 tx=0x7fbb10005040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:15.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:15.668+0000 7fbb0affd640 1 -- 172.21.15.172:0/360692124 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbb04019030 con 0x7fbb1415b0d0 2024-04-14T08:40:16.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.032+0000 7fbb1c58c640 1 -- 172.21.15.172:0/360692124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbb14074250 con 0x7fbb1415b0d0 2024-04-14T08:40:16.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.032+0000 7fbb0affd640 1 -- 172.21.15.172:0/360692124 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7fbb0405a440 con 0x7fbb1415b0d0 2024-04-14T08:40:16.033 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:40:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.035+0000 7fbb1c58c640 1 -- 172.21.15.172:0/360692124 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbaf806c8f0 msgr2=0x7fbaf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.035+0000 7fbb1c58c640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbaf806c8f0 0x7fbaf806edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbb10035000 tx=0x7fbb10005040 comp rx=0 tx=0).stop 2024-04-14T08:40:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.035+0000 7fbb1c58c640 1 -- 172.21.15.172:0/360692124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb1415b0d0 msgr2=0x7fbb1415b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.035+0000 7fbb1c58c640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb1415b0d0 0x7fbb1415b550 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fbb040119d0 tx=0x7fbb04011dd0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.035+0000 7fbb1c58c640 1 -- 172.21.15.172:0/360692124 shutdown_connections 2024-04-14T08:40:16.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.035+0000 7fbb1c58c640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbaf806c8f0 0x7fbaf806edb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.035+0000 7fbb1c58c640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb141e8ea0 0x7fbb141e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.035+0000 7fbb1c58c640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb1415b0d0 0x7fbb1415b550 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.035+0000 7fbb1c58c640 1 --2- 172.21.15.172:0/360692124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb000039e0 0x7fbb140b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.035+0000 7fbb1c58c640 1 -- 172.21.15.172:0/360692124 >> 172.21.15.172:0/360692124 conn(0x7fbb1409d580 msgr2=0x7fbb140d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:16.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.035+0000 7fbb1c58c640 1 -- 172.21.15.172:0/360692124 shutdown_connections 2024-04-14T08:40:16.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.035+0000 7fbb1c58c640 1 -- 172.21.15.172:0/360692124 wait complete. 2024-04-14T08:40:16.047 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:40:16.047 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:40:16.047 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:40:16.047 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:40:16.047 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:40:16.047 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:40:16.047 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:40:16.047 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:40:16.047 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:40:16.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:16.048 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:40:16.065 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:40:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:40:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:40:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:40:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:40:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:40:16.066 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:40:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:40:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:40:16.066 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:40:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:40:16.066 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:40:16.066 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42253sec preferred_lft 42253sec 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:40:16.067 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:40:16.068 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:40:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:40:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:40:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:40:16.068 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:40:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:40:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:40:16.068 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:40:16.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:16.068 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:40:16.117 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:40:16.116+0000 7fd70e1f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:40:16.140 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:40:16.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:16.145 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:40:16.202 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:40:16.202 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:40:16.202 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:40:16.202 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:40:16.202 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:40:16.203 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:40:16.203 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:40:16.203 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:40:16.203 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:40:16.203 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:40:16.203 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:40:16.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:40:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:40:16.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:16.341 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:40:16.356 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:40:16.356 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:40:16.356 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:40:16.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:16.356 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:40:16.424 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:40:16.424 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:40:16.480 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:40:16.480 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:40:16.481 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:40:16.535 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:40:16.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:16.535 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:40:16.620 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:40:16.620 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:40:16.620 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:40:16.629 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:40:16.629 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:40:16.629 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:40:16.629 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:40:16.629 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:40:16.639 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:40:16.640 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:40:16.640 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:40:16.640 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:40:16.640 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:40:16.640 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:40:16.640 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:40:16.640 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:40:16.644 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:40:16.645 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:40:16.645 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:40:16.645 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:40:16.645 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:40:16.645 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:40:16.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:16.645 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:40:16.713 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:40:16.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:40:16.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:40:16.809 INFO:teuthology.orchestra.run.smithi172.stdout:client6710 2024-04-14T08:40:16.809 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:40:16.810 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:40:16.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:16.825 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-14T08:40:16.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.973+0000 7fd42b0de640 1 Processor -- start 2024-04-14T08:40:16.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.973+0000 7fd42b0de640 1 -- start start 2024-04-14T08:40:16.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.974+0000 7fd42b0de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd424161d00 0x7fd4240b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.974+0000 7fd42b0de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240d2cf0 0x7fd4240cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.974+0000 7fd42b0de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4240d1bb0 con 0x7fd4240b5490 2024-04-14T08:40:16.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.974+0000 7fd42b0de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fd4240d0ba0 con 0x7fd424161d00 2024-04-14T08:40:16.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.974+0000 7fd42b0de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd4240d1720 con 0x7fd4240d2cf0 2024-04-14T08:40:16.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.974+0000 7fd423fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd4240b5490 0x7fd4240d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50374/0 (socket says 172.21.15.172:50374) 2024-04-14T08:40:16.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.974+0000 7fd423fff640 1 -- 172.21.15.172:0/2805485435 learned_addr learned my addr 172.21.15.172:0/2805485435 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:16.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.974+0000 7fd428e7a640 1 --2- 172.21.15.172:0/2805485435 >> v2:172.21.15.177:3300/0 conn(0x7fd424161d00 0x7fd4240b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:16.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.974+0000 7fd42967b640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240d2cf0 0x7fd4240cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:16.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.974+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3784481408 0 0) 0x7fd4240d1bb0 con 0x7fd4240b5490 2024-04-14T08:40:16.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.974+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd414003520 con 0x7fd4240b5490 2024-04-14T08:40:16.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1112737461 0 0) 0x7fd414003520 con 0x7fd4240b5490 2024-04-14T08:40:16.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240d2cf0 msgr2=0x7fd4240cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:16.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240d2cf0 0x7fd4240cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 >> v2:172.21.15.177:3300/0 conn(0x7fd424161d00 msgr2=0x7fd4240b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:16.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 --2- 172.21.15.172:0/2805485435 >> v2:172.21.15.177:3300/0 conn(0x7fd424161d00 0x7fd4240b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4240d0df0 con 0x7fd4240b5490 2024-04-14T08:40:16.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd42967b640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240d2cf0 0x7fd4240cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:40:16.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fd410001670 con 0x7fd4240b5490 2024-04-14T08:40:16.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 >> v1:172.21.15.172:6789/0 conn(0x7fd4240b5490 legacy=0x7fd4240d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:16.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4140039b0 0x7fd414003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414004800 0x7fd414004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240d2cf0 0x7fd4140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd4240d0ba0 con 0x7fd414004800 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd4240d1720 con 0x7fd4140039b0 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd41400b9c0 con 0x7fd4240d2cf0 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd428e7a640 1 --2- 172.21.15.172:0/2805485435 >> v2:172.21.15.177:3300/0 conn(0x7fd424161d00 0x7fd4240b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd423fff640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414004800 0x7fd414004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd42967b640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240d2cf0 0x7fd4140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd428e7a640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4140039b0 0x7fd414003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd423fff640 1 -- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240d2cf0 msgr2=0x7fd4140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd423fff640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240d2cf0 0x7fd4140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd423fff640 1 -- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4140039b0 msgr2=0x7fd414003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd423fff640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4140039b0 0x7fd414003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd423fff640 1 -- 172.21.15.172:0/2805485435 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd414003520 con 0x7fd414004800 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.975+0000 7fd428e7a640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4140039b0 0x7fd414003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:16.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.976+0000 7fd423fff640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414004800 0x7fd414004c80 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd41000c210 tx=0x7fd414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c43805d1660e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:16.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.976+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410007d50 con 0x7fd414004800 2024-04-14T08:40:16.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.977+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd410030070 con 0x7fd414004800 2024-04-14T08:40:16.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.977+0000 7fd4237fe640 1 -- 172.21.15.172:0/2805485435 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41002cd80 con 0x7fd414004800 2024-04-14T08:40:16.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.977+0000 7fd42b0de640 1 -- 172.21.15.172:0/2805485435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414004800 msgr2=0x7fd414004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:16.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.977+0000 7fd42b0de640 1 --2- 172.21.15.172:0/2805485435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd414004800 0x7fd414004c80 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd41000c210 tx=0x7fd414005830 comp rx=0 tx=0).stop 2024-04-14T08:40:16.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.977+0000 7fd42967b640 1 -- 172.21.15.172:0/2805485435 reap_dead start 2024-04-14T08:40:16.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.977+0000 7fd42b0de640 1 -- 172.21.15.172:0/2805485435 shutdown_connections 2024-04-14T08:40:16.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.977+0000 7fd42b0de640 1 -- 172.21.15.172:0/2805485435 >> 172.21.15.172:0/2805485435 conn(0x7fd42409d570 msgr2=0x7fd4240d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:16.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.978+0000 7fd42b0de640 1 -- 172.21.15.172:0/2805485435 shutdown_connections 2024-04-14T08:40:16.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.978+0000 7fd42b0de640 1 -- 172.21.15.172:0/2805485435 wait complete. 2024-04-14T08:40:16.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.979+0000 7fd42b0de640 1 Processor -- start 2024-04-14T08:40:16.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.980+0000 7fd42b0de640 1 -- start start 2024-04-14T08:40:16.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42b0de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd414004800 0x7fd4240c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42b0de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240c9450 0x7fd4240cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42b0de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4240b13a0 0x7fd42414f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42b0de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd4240d13b0 con 0x7fd4240b13a0 2024-04-14T08:40:16.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42b0de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd4240d33b0 con 0x7fd414004800 2024-04-14T08:40:16.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42b0de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd4240b5940 con 0x7fd4240c9450 2024-04-14T08:40:16.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42967b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4240b13a0 0x7fd42414f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:16.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42967b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4240b13a0 0x7fd42414f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42256/0 (socket says 172.21.15.172:42256) 2024-04-14T08:40:16.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42967b640 1 -- 172.21.15.172:0/3634762363 learned_addr learned my addr 172.21.15.172:0/3634762363 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:16.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd423fff640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240c9450 0x7fd4240cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:16.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd428e7a640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd414004800 0x7fd4240c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:16.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42967b640 1 -- 172.21.15.172:0/3634762363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240c9450 msgr2=0x7fd4240cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:16.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42967b640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240c9450 0x7fd4240cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42967b640 1 -- 172.21.15.172:0/3634762363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd414004800 msgr2=0x7fd4240c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:16.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42967b640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd414004800 0x7fd4240c7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.981+0000 7fd42967b640 1 -- 172.21.15.172:0/3634762363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4240c82b0 con 0x7fd4240b13a0 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.982+0000 7fd42967b640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4240b13a0 0x7fd42414f280 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd42408d6c0 tx=0x7fd41c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.982+0000 7fd4217fa640 1 -- 172.21.15.172:0/3634762363 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c00fda0 con 0x7fd4240b13a0 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.982+0000 7fd4217fa640 1 -- 172.21.15.172:0/3634762363 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41c01d070 con 0x7fd4240b13a0 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.982+0000 7fd42b0de640 1 -- 172.21.15.172:0/3634762363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4240cc9b0 con 0x7fd4240b13a0 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.982+0000 7fd4217fa640 1 -- 172.21.15.172:0/3634762363 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c015670 con 0x7fd4240b13a0 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.982+0000 7fd42b0de640 1 -- 172.21.15.172:0/3634762363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4240ccf70 con 0x7fd4240b13a0 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.984+0000 7fd4217fa640 1 -- 172.21.15.172:0/3634762363 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd41c015810 con 0x7fd4240b13a0 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.984+0000 7fd4217fa640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd40806c920 0x7fd40806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.984+0000 7fd4217fa640 1 -- 172.21.15.172:0/3634762363 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fd41c08c6c0 con 0x7fd4240b13a0 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.984+0000 7fd428e7a640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd40806c920 0x7fd40806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.985+0000 7fd428e7a640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd40806c920 0x7fd40806ede0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd418009ba0 tx=0x7fd418008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.985+0000 7fd42dfa1740 1 Processor -- start 2024-04-14T08:40:16.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.985+0000 7fd42dfa1740 1 -- start start 2024-04-14T08:40:16.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.986+0000 7fd42dfa1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586583482e0 0x5586583486c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.986+0000 7fd42dfa1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558658318300 0x558658345550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.986+0000 7fd42dfa1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55865831be60 0x558658345a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.986+0000 7fd42dfa1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558658282740 con 0x5586583482e0 2024-04-14T08:40:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.986+0000 7fd42dfa1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55865822a930 con 0x558658318300 2024-04-14T08:40:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.986+0000 7fd42dfa1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55865827e890 con 0x55865831be60 2024-04-14T08:40:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.986+0000 7fd428e7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586583482e0 0x5586583486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.986+0000 7fd428e7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586583482e0 0x5586583486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42258/0 (socket says 172.21.15.172:42258) 2024-04-14T08:40:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.986+0000 7fd428e7a640 1 -- 172.21.15.172:0/189059919 learned_addr learned my addr 172.21.15.172:0/189059919 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.987+0000 7fd42967b640 1 --2- 172.21.15.172:0/189059919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55865831be60 0x558658345a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:16.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.987+0000 7fd423fff640 1 --2- 172.21.15.172:0/189059919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558658318300 0x558658345550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.987+0000 7fd428e7a640 1 -- 172.21.15.172:0/189059919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55865831be60 msgr2=0x558658345a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.987+0000 7fd428e7a640 1 --2- 172.21.15.172:0/189059919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55865831be60 0x558658345a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.987+0000 7fd428e7a640 1 -- 172.21.15.172:0/189059919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558658318300 msgr2=0x558658345550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.987+0000 7fd428e7a640 1 --2- 172.21.15.172:0/189059919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558658318300 0x558658345550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.987+0000 7fd428e7a640 1 -- 172.21.15.172:0/189059919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865805e7e0 con 0x5586583482e0 2024-04-14T08:40:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.987+0000 7fd42967b640 1 --2- 172.21.15.172:0/189059919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55865831be60 0x558658345a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:40:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.987+0000 7fd428e7a640 1 --2- 172.21.15.172:0/189059919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586583482e0 0x5586583486c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd418006f80 tx=0x7fd418016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab27cc70d7450bf server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.987+0000 7fd40e7fc640 1 -- 172.21.15.172:0/189059919 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418023640 con 0x5586583482e0 2024-04-14T08:40:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.987+0000 7fd40e7fc640 1 -- 172.21.15.172:0/189059919 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd418028070 con 0x5586583482e0 2024-04-14T08:40:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.987+0000 7fd40e7fc640 1 -- 172.21.15.172:0/189059919 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4180239e0 con 0x5586583482e0 2024-04-14T08:40:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.988+0000 7fd42dfa1740 1 -- 172.21.15.172:0/189059919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586583482e0 msgr2=0x5586583486c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:16.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.988+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/189059919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586583482e0 0x5586583486c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd418006f80 tx=0x7fd418016000 comp rx=0 tx=0).stop 2024-04-14T08:40:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.988+0000 7fd42dfa1740 1 -- 172.21.15.172:0/189059919 shutdown_connections 2024-04-14T08:40:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.988+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/189059919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55865831be60 0x558658345a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.988+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/189059919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558658318300 0x558658345550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.988+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/189059919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586583482e0 0x5586583486c0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.988+0000 7fd42dfa1740 1 -- 172.21.15.172:0/189059919 >> 172.21.15.172:0/189059919 conn(0x55865831a570 msgr2=0x5586583630f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.988+0000 7fd42dfa1740 1 -- 172.21.15.172:0/189059919 shutdown_connections 2024-04-14T08:40:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.988+0000 7fd42dfa1740 1 -- 172.21.15.172:0/189059919 wait complete. 2024-04-14T08:40:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.989+0000 7fd42dfa1740 1 -- start start 2024-04-14T08:40:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.989+0000 7fd42dfa1740 1 Processor -- start 2024-04-14T08:40:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd42dfa1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 0x558658348e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd42dfa1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586583482e0 0x558658341db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:16.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd42dfa1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55865833f1f0 0x5586583422f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd42dfa1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55865831d010 con 0x55865831be60 2024-04-14T08:40:17.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd42dfa1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55865823dcf0 con 0x55865833f1f0 2024-04-14T08:40:17.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd42dfa1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55865831c400 con 0x5586583482e0 2024-04-14T08:40:17.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd42dfa1740 10 client.0 _ll_register_callbacks cb 0x55865820a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:17.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd428e7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 0x558658348e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd428e7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 0x558658348e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42262/0 (socket says 172.21.15.172:42262) 2024-04-14T08:40:17.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd428e7a640 1 -- 172.21.15.172:0/1023340280 learned_addr learned my addr 172.21.15.172:0/1023340280 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:17.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd42967b640 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55865833f1f0 0x5586583422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd423fff640 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586583482e0 0x558658341db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd428e7a640 1 -- 172.21.15.172:0/1023340280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586583482e0 msgr2=0x558658341db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd428e7a640 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586583482e0 0x558658341db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd428e7a640 1 -- 172.21.15.172:0/1023340280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55865833f1f0 msgr2=0x5586583422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd428e7a640 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55865833f1f0 0x5586583422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd428e7a640 1 -- 172.21.15.172:0/1023340280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586580bef70 con 0x55865831be60 2024-04-14T08:40:17.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd428e7a640 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 0x558658348e50 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fd418006f50 tx=0x7fd418004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e544343c2c9598c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:17.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.990+0000 7fd4037fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:17.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd4037fe640 1 -- 172.21.15.172:0/1023340280 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418023850 con 0x55865831be60 2024-04-14T08:40:17.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd4037fe640 1 -- 172.21.15.172:0/1023340280 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fd418039db0 con 0x55865831be60 2024-04-14T08:40:17.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd42dfa1740 1 -- 172.21.15.172:0/1023340280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd418028070 con 0x55865831be60 2024-04-14T08:40:17.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd4037fe640 1 -- 172.21.15.172:0/1023340280 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd4180239f0 con 0x55865831be60 2024-04-14T08:40:17.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd42dfa1740 10 client.8112 fetch_fsmap learned FSMap version 67 2024-04-14T08:40:17.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd42dfa1740 1 -- 172.21.15.172:0/1023340280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x558657f56170 con 0x55865831be60 2024-04-14T08:40:17.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd4037fe640 1 -- 172.21.15.172:0/1023340280 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd418028070 con 0x55865831be60 2024-04-14T08:40:17.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd4037fe640 1 -- 172.21.15.172:0/1023340280 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fd418023b90 con 0x55865831be60 2024-04-14T08:40:17.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd42dfa1740 10 client.8112 fetch_fsmap finished waiting for FSMap version 67 2024-04-14T08:40:17.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd42dfa1740 10 client.8112 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:40:17.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd42dfa1740 20 client.8112 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:17.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55865834bb10 0x55865834bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd42dfa1740 4 client.8112 mds_command: new command op to 8013 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:40:17.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd42dfa1740 1 -- 172.21.15.172:0/1023340280 --> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55865828fc80 con 0x55865834bb10 2024-04-14T08:40:17.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.991+0000 7fd423fff640 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55865834bb10 0x55865834bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.992+0000 7fd423fff640 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55865834bb10 0x55865834bf10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0030b2fafd1fec server_cookie=793311822aba9bca in_seq=0 out_seq=0 2024-04-14T08:40:17.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.992+0000 7fd4037fe640 10 client.8112 ms_handle_connect on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:17.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.992+0000 7fd4037fe640 1 -- 172.21.15.172:0/1023340280 <== mds.0 v2:172.21.15.172:6835/3091771014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55865828fc80 con 0x55865834bb10 2024-04-14T08:40:17.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.992+0000 7fd4037fe640 10 client.8112 handle_command_reply: tid=0 2024-04-14T08:40:17.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.992+0000 7fd42dfa1740 1 client.8112 shutdown 2024-04-14T08:40:17.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.992+0000 7fd42dfa1740 1 -- 172.21.15.172:0/1023340280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 msgr2=0x558658348e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.992+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 0x558658348e50 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fd418006f50 tx=0x7fd418004730 comp rx=0 tx=0).stop 2024-04-14T08:40:17.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.992+0000 7fd42dfa1740 1 -- 172.21.15.172:0/1023340280 shutdown_connections 2024-04-14T08:40:17.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.992+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55865834bb10 0x55865834bf10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.992+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55865833f1f0 0x5586583422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.993+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586583482e0 0x558658341db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.993+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/1023340280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 0x558658348e50 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.993+0000 7fd42dfa1740 1 -- 172.21.15.172:0/1023340280 >> 172.21.15.172:0/1023340280 conn(0x558658318300 msgr2=0x55865831d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:17.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.993+0000 7fd4037fe640 0 client.8112 ms_handle_reset on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:17.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.993+0000 7fd42dfa1740 1 -- 172.21.15.172:0/1023340280 shutdown_connections 2024-04-14T08:40:17.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.993+0000 7fd42dfa1740 1 -- 172.21.15.172:0/1023340280 wait complete. 2024-04-14T08:40:17.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:16.993+0000 7fd42dfa1740 20 client.8112 trim_cache size 0 max 16384 2024-04-14T08:40:17.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.010+0000 7fd42dfa1740 1 Processor -- start 2024-04-14T08:40:17.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.011+0000 7fd42dfa1740 1 -- start start 2024-04-14T08:40:17.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.011+0000 7fd42dfa1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55865831be60 0x558658340f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.011+0000 7fd42dfa1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865833f1f0 0x5586583494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.011+0000 7fd42dfa1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586583482e0 0x5586583499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.011+0000 7fd42dfa1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd424201ab0 con 0x55865833f1f0 2024-04-14T08:40:17.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.011+0000 7fd42dfa1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd424200ab0 con 0x55865831be60 2024-04-14T08:40:17.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.011+0000 7fd42dfa1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd4241ffe60 con 0x5586583482e0 2024-04-14T08:40:17.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.011+0000 7fd423fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865833f1f0 0x5586583494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.011+0000 7fd423fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865833f1f0 0x5586583494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42276/0 (socket says 172.21.15.172:42276) 2024-04-14T08:40:17.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.011+0000 7fd423fff640 1 -- 172.21.15.172:0/3688817611 learned_addr learned my addr 172.21.15.172:0/3688817611 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:17.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.012+0000 7fd428e7a640 1 --2- 172.21.15.172:0/3688817611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55865831be60 0x558658340f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.012+0000 7fd42967b640 1 --2- 172.21.15.172:0/3688817611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586583482e0 0x5586583499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.012+0000 7fd423fff640 1 -- 172.21.15.172:0/3688817611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586583482e0 msgr2=0x5586583499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.012+0000 7fd423fff640 1 --2- 172.21.15.172:0/3688817611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586583482e0 0x5586583499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.012+0000 7fd423fff640 1 -- 172.21.15.172:0/3688817611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55865831be60 msgr2=0x558658340f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.012+0000 7fd423fff640 1 --2- 172.21.15.172:0/3688817611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55865831be60 0x558658340f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.012+0000 7fd423fff640 1 -- 172.21.15.172:0/3688817611 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865831f940 con 0x55865833f1f0 2024-04-14T08:40:17.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.012+0000 7fd428e7a640 1 --2- 172.21.15.172:0/3688817611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55865831be60 0x558658340f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:17.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.012+0000 7fd423fff640 1 --2- 172.21.15.172:0/3688817611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865833f1f0 0x5586583494a0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd41002c590 tx=0x7fd41002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656a02828ff0bb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:17.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.012+0000 7fd40dffb640 1 -- 172.21.15.172:0/3688817611 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41003a070 con 0x55865833f1f0 2024-04-14T08:40:17.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.012+0000 7fd40dffb640 1 -- 172.21.15.172:0/3688817611 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd410030040 con 0x55865833f1f0 2024-04-14T08:40:17.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.012+0000 7fd40dffb640 1 -- 172.21.15.172:0/3688817611 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41002cb20 con 0x55865833f1f0 2024-04-14T08:40:17.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.013+0000 7fd42dfa1740 1 -- 172.21.15.172:0/3688817611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865833f1f0 msgr2=0x5586583494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.013+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/3688817611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865833f1f0 0x5586583494a0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd41002c590 tx=0x7fd41002e000 comp rx=0 tx=0).stop 2024-04-14T08:40:17.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.013+0000 7fd42dfa1740 1 -- 172.21.15.172:0/3688817611 shutdown_connections 2024-04-14T08:40:17.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.013+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/3688817611 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586583482e0 0x5586583499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.013+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/3688817611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865833f1f0 0x5586583494a0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.013+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/3688817611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55865831be60 0x558658340f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.013+0000 7fd42dfa1740 1 -- 172.21.15.172:0/3688817611 >> 172.21.15.172:0/3688817611 conn(0x558658318300 msgr2=0x5586583171f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:17.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.013+0000 7fd42dfa1740 1 -- 172.21.15.172:0/3688817611 shutdown_connections 2024-04-14T08:40:17.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.013+0000 7fd42dfa1740 1 -- 172.21.15.172:0/3688817611 wait complete. 2024-04-14T08:40:17.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.013+0000 7fd42dfa1740 1 -- start start 2024-04-14T08:40:17.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.014+0000 7fd42dfa1740 1 Processor -- start 2024-04-14T08:40:17.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.014+0000 7fd42dfa1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 0x558658371770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.014+0000 7fd42dfa1740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55865833f1f0 0x558658371cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.014+0000 7fd42dfa1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586583482e0 0x55865833c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.014+0000 7fd42dfa1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd424200460 con 0x55865831be60 2024-04-14T08:40:17.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.014+0000 7fd42dfa1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558658343460 con 0x5586583482e0 2024-04-14T08:40:17.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.014+0000 7fd42dfa1740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5586583438c0 con 0x55865833f1f0 2024-04-14T08:40:17.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd428e7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 0x558658371770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd42dfa1740 10 client.0 _ll_register_callbacks cb 0x7fd4140036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:17.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd428e7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 0x558658371770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42290/0 (socket says 172.21.15.172:42290) 2024-04-14T08:40:17.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd428e7a640 1 -- 172.21.15.172:0/834689511 learned_addr learned my addr 172.21.15.172:0/834689511 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:17.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd42967b640 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586583482e0 0x55865833c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd423fff640 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55865833f1f0 0x558658371cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd428e7a640 1 -- 172.21.15.172:0/834689511 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55865833f1f0 msgr2=0x558658371cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd428e7a640 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55865833f1f0 0x558658371cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd428e7a640 1 -- 172.21.15.172:0/834689511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586583482e0 msgr2=0x55865833c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd428e7a640 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586583482e0 0x55865833c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd428e7a640 1 -- 172.21.15.172:0/834689511 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865831d1c0 con 0x55865831be60 2024-04-14T08:40:17.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd428e7a640 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 0x558658371770 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd418006f50 tx=0x7fd418019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f31367833240bf server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:17.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd4037fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:17.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd4037fe640 1 -- 172.21.15.172:0/834689511 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418023850 con 0x55865831be60 2024-04-14T08:40:17.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd4037fe640 1 -- 172.21.15.172:0/834689511 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fd41803a9a0 con 0x55865831be60 2024-04-14T08:40:17.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd42dfa1740 1 -- 172.21.15.172:0/834689511 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd410030040 con 0x55865831be60 2024-04-14T08:40:17.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.015+0000 7fd4037fe640 1 -- 172.21.15.172:0/834689511 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fd41803aba0 con 0x55865831be60 2024-04-14T08:40:17.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd42dfa1740 10 client.8118 fetch_fsmap learned FSMap version 67 2024-04-14T08:40:17.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd42dfa1740 1 -- 172.21.15.172:0/834689511 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55865833d230 con 0x55865831be60 2024-04-14T08:40:17.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd4037fe640 1 -- 172.21.15.172:0/834689511 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd410030040 con 0x55865831be60 2024-04-14T08:40:17.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd4037fe640 1 -- 172.21.15.172:0/834689511 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fd418038860 con 0x55865831be60 2024-04-14T08:40:17.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd42dfa1740 10 client.8118 fetch_fsmap finished waiting for FSMap version 67 2024-04-14T08:40:17.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd42dfa1740 10 client.8118 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:40:17.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd42dfa1740 20 client.8118 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:17.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x5586583dca80 0x55865833c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd42dfa1740 4 client.8118 mds_command: new command op to 8013 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-14T08:40:17.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd42dfa1740 1 -- 172.21.15.172:0/834689511 --> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x558658289fd0 con 0x5586583dca80 2024-04-14T08:40:17.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd423fff640 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x5586583dca80 0x55865833c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd4037fe640 10 client.8118 ms_handle_connect on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:17.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.016+0000 7fd423fff640 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x5586583dca80 0x55865833c6d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6865e68ed2714405 server_cookie=cdb66d87d0cba4d0 in_seq=0 out_seq=0 2024-04-14T08:40:17.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd4037fe640 1 -- 172.21.15.172:0/834689511 <== mds.0 v2:172.21.15.172:6835/3091771014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558658289fd0 con 0x5586583dca80 2024-04-14T08:40:17.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd4037fe640 10 client.8118 handle_command_reply: tid=0 2024-04-14T08:40:17.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd42dfa1740 1 client.8118 shutdown 2024-04-14T08:40:17.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd42dfa1740 1 -- 172.21.15.172:0/834689511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 msgr2=0x558658371770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 0x558658371770 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd418006f50 tx=0x7fd418019000 comp rx=0 tx=0).stop 2024-04-14T08:40:17.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd42dfa1740 1 -- 172.21.15.172:0/834689511 shutdown_connections 2024-04-14T08:40:17.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x5586583dca80 0x55865833c6d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586583482e0 0x55865833c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55865833f1f0 0x558658371cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd4037fe640 0 client.8118 ms_handle_reset on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:17.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd42dfa1740 1 --2- 172.21.15.172:0/834689511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55865831be60 0x558658371770 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd42dfa1740 1 -- 172.21.15.172:0/834689511 >> 172.21.15.172:0/834689511 conn(0x558658318300 msgr2=0x55865834bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:17.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd42dfa1740 1 -- 172.21.15.172:0/834689511 shutdown_connections 2024-04-14T08:40:17.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.017+0000 7fd42dfa1740 1 -- 172.21.15.172:0/834689511 wait complete. 2024-04-14T08:40:17.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42dfa1740 20 client.8118 trim_cache size 0 max 16384 2024-04-14T08:40:17.026 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:40:17.026 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-04-14T08:40:17.026 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "41f64c90-aad5-412f-8799-812a0c5f81f3", 2024-04-14T08:40:17.026 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-04-14T08:40:17.026 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:40:17.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42b0de640 1 -- 172.21.15.172:0/3634762363 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd40806c920 msgr2=0x7fd40806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42b0de640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd40806c920 0x7fd40806ede0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd418009ba0 tx=0x7fd418008040 comp rx=0 tx=0).stop 2024-04-14T08:40:17.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42b0de640 1 -- 172.21.15.172:0/3634762363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4240b13a0 msgr2=0x7fd42414f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42b0de640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4240b13a0 0x7fd42414f280 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd42408d6c0 tx=0x7fd41c009f90 comp rx=0 tx=0).stop 2024-04-14T08:40:17.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42b0de640 1 -- 172.21.15.172:0/3634762363 shutdown_connections 2024-04-14T08:40:17.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42b0de640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd40806c920 0x7fd40806ede0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42b0de640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4240b13a0 0x7fd42414f280 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42b0de640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4240c9450 0x7fd4240cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42b0de640 1 --2- 172.21.15.172:0/3634762363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd414004800 0x7fd4240c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42b0de640 1 -- 172.21.15.172:0/3634762363 >> 172.21.15.172:0/3634762363 conn(0x7fd42409d570 msgr2=0x7fd42406d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:17.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42b0de640 1 -- 172.21.15.172:0/3634762363 shutdown_connections 2024-04-14T08:40:17.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.018+0000 7fd42b0de640 1 -- 172.21.15.172:0/3634762363 wait complete. 2024-04-14T08:40:17.029 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:40:17.019+0000 7fd70a1e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x559da32a8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-14T08:40:17.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:17.033 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-14T08:40:17.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.146+0000 7f9ed6b1e640 1 Processor -- start 2024-04-14T08:40:17.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.146+0000 7f9ed6b1e640 1 -- start start 2024-04-14T08:40:17.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.147+0000 7f9ed6b1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9ed00ccdc0 0x7f9ed00cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.147+0000 7f9ed6b1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9ed00cffb0 0x7f9ed00cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.147+0000 7f9ed6b1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ed00cd6e0 con 0x7f9ed00d1a10 2024-04-14T08:40:17.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.147+0000 7f9ed6b1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f9ed00d0610 con 0x7f9ed00cffb0 2024-04-14T08:40:17.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.147+0000 7f9ed6b1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f9ed00ced00 con 0x7f9ed00ccdc0 2024-04-14T08:40:17.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.147+0000 7f9ed50bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9ed00cffb0 0x7f9ed00cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.147+0000 7f9ed50bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9ed00cffb0 0x7f9ed00cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42302/0 (socket says 172.21.15.172:42302) 2024-04-14T08:40:17.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.147+0000 7f9ed50bb640 1 -- 172.21.15.172:0/2666411636 learned_addr learned my addr 172.21.15.172:0/2666411636 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:17.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.147+0000 7f9ed48ba640 1 --2- 172.21.15.172:0/2666411636 >> v2:172.21.15.177:3300/0 conn(0x7f9ed00ccdc0 0x7f9ed00cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.148+0000 7f9ed50bb640 1 -- 172.21.15.172:0/2666411636 >> v2:172.21.15.177:3300/0 conn(0x7f9ed00ccdc0 msgr2=0x7f9ed00cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.148+0000 7f9ed50bb640 1 --2- 172.21.15.172:0/2666411636 >> v2:172.21.15.177:3300/0 conn(0x7f9ed00ccdc0 0x7f9ed00cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.148+0000 7f9ed50bb640 1 -- 172.21.15.172:0/2666411636 >> v1:172.21.15.177:6789/0 conn(0x7f9ed00d1a10 legacy=0x7f9ed015eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.148+0000 7f9ed50bb640 1 -- 172.21.15.172:0/2666411636 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed00d0860 con 0x7f9ed00cffb0 2024-04-14T08:40:17.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.148+0000 7f9ed50bb640 1 --2- 172.21.15.172:0/2666411636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed00cffb0 0x7f9ed00cf190 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0074610 tx=0x7f9ec802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77299e2c3dfa40d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:17.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.148+0000 7f9ecf7fe640 1 -- 172.21.15.172:0/2666411636 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec802ca80 con 0x7f9ed00cffb0 2024-04-14T08:40:17.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.149+0000 7f9ecf7fe640 1 -- 172.21.15.172:0/2666411636 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec802cc20 con 0x7f9ed00cffb0 2024-04-14T08:40:17.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.149+0000 7f9ecf7fe640 1 -- 172.21.15.172:0/2666411636 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80343f0 con 0x7f9ed00cffb0 2024-04-14T08:40:17.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.149+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/2666411636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed00cffb0 msgr2=0x7f9ed00cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.149+0000 7f9ed6b1e640 1 --2- 172.21.15.172:0/2666411636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed00cffb0 0x7f9ed00cf190 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0074610 tx=0x7f9ec802bee0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.149+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/2666411636 shutdown_connections 2024-04-14T08:40:17.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.149+0000 7f9ed6b1e640 1 --2- 172.21.15.172:0/2666411636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed00cffb0 0x7f9ed00cf190 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.149+0000 7f9ed6b1e640 1 --2- 172.21.15.172:0/2666411636 >> v2:172.21.15.177:3300/0 conn(0x7f9ed00ccdc0 0x7f9ed00cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.149+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/2666411636 >> 172.21.15.172:0/2666411636 conn(0x7f9ed00d0f20 msgr2=0x7f9ed00d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:17.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.149+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/2666411636 shutdown_connections 2024-04-14T08:40:17.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.149+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/2666411636 wait complete. 2024-04-14T08:40:17.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.151+0000 7f9ed6b1e640 1 Processor -- start 2024-04-14T08:40:17.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.151+0000 7f9ed6b1e640 1 -- start start 2024-04-14T08:40:17.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.152+0000 7f9ed6b1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed00ccdc0 0x7f9ed00ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.152+0000 7f9ed6b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ed00cffb0 0x7f9ed006edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.152+0000 7f9ed6b1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed00d1a10 0x7f9ed006f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.152+0000 7f9ed6b1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed00cb730 con 0x7f9ed00d1a10 2024-04-14T08:40:17.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.152+0000 7f9ed6b1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed015f1f0 con 0x7f9ed00ccdc0 2024-04-14T08:40:17.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.152+0000 7f9ed6b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed00cf6d0 con 0x7f9ed00cffb0 2024-04-14T08:40:17.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ed50bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed00d1a10 0x7f9ed006f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ed50bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed00d1a10 0x7f9ed006f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42306/0 (socket says 172.21.15.172:42306) 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ed50bb640 1 -- 172.21.15.172:0/3797422052 learned_addr learned my addr 172.21.15.172:0/3797422052 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ed48ba640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed00ccdc0 0x7f9ed00ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ecffff640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ed00cffb0 0x7f9ed006edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ed50bb640 1 -- 172.21.15.172:0/3797422052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ed00cffb0 msgr2=0x7f9ed006edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ed50bb640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ed00cffb0 0x7f9ed006edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ed50bb640 1 -- 172.21.15.172:0/3797422052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed00ccdc0 msgr2=0x7f9ed00ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ed50bb640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed00ccdc0 0x7f9ed00ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ed50bb640 1 -- 172.21.15.172:0/3797422052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed015db40 con 0x7f9ed00d1a10 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ed50bb640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed00d1a10 0x7f9ed006f310 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8036040 tx=0x7f9ec8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ecd7fa640 1 -- 172.21.15.172:0/3797422052 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8043020 con 0x7f9ed00d1a10 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ecd7fa640 1 -- 172.21.15.172:0/3797422052 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec803e450 con 0x7f9ed00d1a10 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/3797422052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed00ca940 con 0x7f9ed00d1a10 2024-04-14T08:40:17.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ecd7fa640 1 -- 172.21.15.172:0/3797422052 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8009510 con 0x7f9ed00d1a10 2024-04-14T08:40:17.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.153+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/3797422052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed01f4970 con 0x7f9ed00d1a10 2024-04-14T08:40:17.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.155+0000 7f9ecd7fa640 1 -- 172.21.15.172:0/3797422052 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ec8046020 con 0x7f9ed00d1a10 2024-04-14T08:40:17.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.155+0000 7f9ecd7fa640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9eb406c8f0 0x7f9eb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.156+0000 7f9ecd7fa640 1 -- 172.21.15.172:0/3797422052 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f9ec80ae4d0 con 0x7f9ed00d1a10 2024-04-14T08:40:17.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.156+0000 7f9ed48ba640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9eb406c8f0 0x7f9eb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.156+0000 7f9ed48ba640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9eb406c8f0 0x7f9eb406edb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4006fd0 tx=0x7f9ec4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:17.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.156+0000 7f9ed99f8740 1 Processor -- start 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.156+0000 7f9ed99f8740 1 -- start start 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.157+0000 7f9ed99f8740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 0x558afce52910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.157+0000 7f9ed99f8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce82700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.157+0000 7f9ed99f8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce52e50 0x558afce82c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.157+0000 7f9ed99f8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558afcdbc800 con 0x558afce7fd50 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.157+0000 7f9ed99f8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558afcd647f0 con 0x558afce52e50 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.157+0000 7f9ed99f8740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x558afcdb87b0 con 0x558afce7f970 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.157+0000 7f9ecffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce82700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.157+0000 7f9ecffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce82700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42320/0 (socket says 172.21.15.172:42320) 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.157+0000 7f9ecffff640 1 -- 172.21.15.172:0/1446380558 learned_addr learned my addr 172.21.15.172:0/1446380558 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.158+0000 7f9ed48ba640 1 --2- 172.21.15.172:0/1446380558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 0x558afce52910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.158+0000 7f9ed50bb640 1 --2- 172.21.15.172:0/1446380558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce52e50 0x558afce82c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.158+0000 7f9ecffff640 1 -- 172.21.15.172:0/1446380558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 msgr2=0x558afce52910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.158+0000 7f9ecffff640 1 --2- 172.21.15.172:0/1446380558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 0x558afce52910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.158+0000 7f9ecffff640 1 -- 172.21.15.172:0/1446380558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce52e50 msgr2=0x558afce82c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.158+0000 7f9ecffff640 1 --2- 172.21.15.172:0/1446380558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce52e50 0x558afce82c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.158+0000 7f9ecffff640 1 -- 172.21.15.172:0/1446380558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558afcb9a7d0 con 0x558afce7fd50 2024-04-14T08:40:17.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.158+0000 7f9ecffff640 1 --2- 172.21.15.172:0/1446380558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce82700 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0017f0 tx=0x7f9ebc007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45d6faa73294cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:17.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.158+0000 7f9eba7fc640 1 -- 172.21.15.172:0/1446380558 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc01a070 con 0x558afce7fd50 2024-04-14T08:40:17.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.158+0000 7f9eba7fc640 1 -- 172.21.15.172:0/1446380558 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc005320 con 0x558afce7fd50 2024-04-14T08:40:17.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.159+0000 7f9ed99f8740 1 -- 172.21.15.172:0/1446380558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 msgr2=0x558afce82700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.159+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/1446380558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce82700 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0017f0 tx=0x7f9ebc007e50 comp rx=0 tx=0).stop 2024-04-14T08:40:17.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.159+0000 7f9ed99f8740 1 -- 172.21.15.172:0/1446380558 shutdown_connections 2024-04-14T08:40:17.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.159+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/1446380558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce52e50 0x558afce82c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.159+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/1446380558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce82700 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.159+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/1446380558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 0x558afce52910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.159+0000 7f9ed99f8740 1 -- 172.21.15.172:0/1446380558 >> 172.21.15.172:0/1446380558 conn(0x558afce54b80 msgr2=0x558afce9d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.159+0000 7f9ed99f8740 1 -- 172.21.15.172:0/1446380558 shutdown_connections 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.159+0000 7f9ed99f8740 1 -- 172.21.15.172:0/1446380558 wait complete. 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.160+0000 7f9ed99f8740 1 -- start start 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.160+0000 7f9ed99f8740 1 Processor -- start 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ed99f8740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 0x558afce97b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ed99f8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce98040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ed99f8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce73830 0x558afce73c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ed99f8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558afce57120 con 0x558afce7fd50 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ed99f8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558afcd77d00 con 0x558afce73830 2024-04-14T08:40:17.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ed99f8740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x558afce56510 con 0x558afce7f970 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ed99f8740 10 client.0 _ll_register_callbacks cb 0x558afcd776f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ecffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce98040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ecffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce98040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42334/0 (socket says 172.21.15.172:42334) 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ecffff640 1 -- 172.21.15.172:0/2218374880 learned_addr learned my addr 172.21.15.172:0/2218374880 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ed48ba640 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 0x558afce97b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ed50bb640 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce73830 0x558afce73c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ecffff640 1 -- 172.21.15.172:0/2218374880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 msgr2=0x558afce97b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ecffff640 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 0x558afce97b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ecffff640 1 -- 172.21.15.172:0/2218374880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce73830 msgr2=0x558afce73c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ecffff640 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce73830 0x558afce73c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ecffff640 1 -- 172.21.15.172:0/2218374880 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558afcad03f0 con 0x558afce7fd50 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ecffff640 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce98040 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc001140 tx=0x7f9ebc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2bf5e7f78c96ba server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9eaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9eaf7fe640 1 -- 172.21.15.172:0/2218374880 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc026060 con 0x558afce7fd50 2024-04-14T08:40:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9eaf7fe640 1 -- 172.21.15.172:0/2218374880 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f9ebc006d20 con 0x558afce7fd50 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.161+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2218374880 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ebc005320 con 0x558afce7fd50 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9eaf7fe640 1 -- 172.21.15.172:0/2218374880 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f9ebc02c050 con 0x558afce7fd50 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9ed99f8740 10 client.8130 fetch_fsmap learned FSMap version 67 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2218374880 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x558afca900f0 con 0x558afce7fd50 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9eaf7fe640 1 -- 172.21.15.172:0/2218374880 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc005320 con 0x558afce7fd50 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9eaf7fe640 1 -- 172.21.15.172:0/2218374880 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f9ebc02f020 con 0x558afce7fd50 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9ed99f8740 10 client.8130 fetch_fsmap finished waiting for FSMap version 67 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9ed99f8740 10 client.8130 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9ed99f8740 20 client.8130 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x558afce95970 0x558afce95d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9ed99f8740 4 client.8130 mds_command: new command op to 8013 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2218374880 --> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558afcdc9d40 con 0x558afce95970 2024-04-14T08:40:17.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9ed48ba640 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x558afce95970 0x558afce95d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9eaf7fe640 10 client.8130 ms_handle_connect on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.162+0000 7f9ed48ba640 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x558afce95970 0x558afce95d50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c67d42337929016 server_cookie=78893d8c122c325a in_seq=0 out_seq=0 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.163+0000 7f9eaf7fe640 1 -- 172.21.15.172:0/2218374880 <== mds.0 v2:172.21.15.172:6835/3091771014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558afcdc9d40 con 0x558afce95970 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.163+0000 7f9eaf7fe640 10 client.8130 handle_command_reply: tid=0 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.163+0000 7f9ed99f8740 1 client.8130 shutdown 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.164+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2218374880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 msgr2=0x558afce98040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.164+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce98040 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc001140 tx=0x7f9ebc000bc0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.164+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2218374880 shutdown_connections 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.164+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x558afce95970 0x558afce95d50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.164+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce73830 0x558afce73c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.164+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce98040 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.164+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/2218374880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 0x558afce97b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.164+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2218374880 >> 172.21.15.172:0/2218374880 conn(0x558afce52e50 msgr2=0x558afce826f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:17.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.164+0000 7f9eaf7fe640 0 client.8130 ms_handle_reset on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:17.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.164+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2218374880 shutdown_connections 2024-04-14T08:40:17.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.164+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2218374880 wait complete. 2024-04-14T08:40:17.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.164+0000 7f9ed99f8740 20 client.8130 trim_cache size 0 max 16384 2024-04-14T08:40:17.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.200+0000 7f9ed99f8740 1 Processor -- start 2024-04-14T08:40:17.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.200+0000 7f9ed99f8740 1 -- start start 2024-04-14T08:40:17.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.201+0000 7f9ed99f8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce73830 0x558afce95970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.201+0000 7f9ed99f8740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 0x558afce95eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.201+0000 7f9ed99f8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce7e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.201+0000 7f9ed99f8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed0201350 con 0x558afce7fd50 2024-04-14T08:40:17.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.201+0000 7f9ed99f8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed0200410 con 0x558afce73830 2024-04-14T08:40:17.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.201+0000 7f9ed99f8740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed00b50a0 con 0x558afce7f970 2024-04-14T08:40:17.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9ed50bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce7e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9ed48ba640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce73830 0x558afce95970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9ecffff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 0x558afce95eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9ed50bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce7e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42338/0 (socket says 172.21.15.172:42338) 2024-04-14T08:40:17.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9ed50bb640 1 -- 172.21.15.172:0/1422774316 learned_addr learned my addr 172.21.15.172:0/1422774316 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:17.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9ed50bb640 1 -- 172.21.15.172:0/1422774316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 msgr2=0x558afce95eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9ed50bb640 1 --2- 172.21.15.172:0/1422774316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 0x558afce95eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9ed50bb640 1 -- 172.21.15.172:0/1422774316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce73830 msgr2=0x558afce95970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9ed50bb640 1 --2- 172.21.15.172:0/1422774316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce73830 0x558afce95970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9ed50bb640 1 -- 172.21.15.172:0/1422774316 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558afce74000 con 0x558afce7fd50 2024-04-14T08:40:17.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9ed50bb640 1 --2- 172.21.15.172:0/1422774316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce7e360 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f9ec802c4f0 tx=0x7f9ec803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157e629662fbdab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:17.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9eb9ffb640 1 -- 172.21.15.172:0/1422774316 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec807c010 con 0x558afce7fd50 2024-04-14T08:40:17.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9eb9ffb640 1 -- 172.21.15.172:0/1422774316 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec809b040 con 0x558afce7fd50 2024-04-14T08:40:17.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.202+0000 7f9eb9ffb640 1 -- 172.21.15.172:0/1422774316 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8080310 con 0x558afce7fd50 2024-04-14T08:40:17.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.203+0000 7f9ed99f8740 1 -- 172.21.15.172:0/1422774316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 msgr2=0x558afce7e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.203+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/1422774316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce7e360 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f9ec802c4f0 tx=0x7f9ec803c000 comp rx=0 tx=0).stop 2024-04-14T08:40:17.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.203+0000 7f9ed99f8740 1 -- 172.21.15.172:0/1422774316 shutdown_connections 2024-04-14T08:40:17.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.203+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/1422774316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7fd50 0x558afce7e360 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.203+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/1422774316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce7f970 0x558afce95eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.203+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/1422774316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce73830 0x558afce95970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.203+0000 7f9ed99f8740 1 -- 172.21.15.172:0/1422774316 >> 172.21.15.172:0/1422774316 conn(0x558afce52e50 msgr2=0x558afce96550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:17.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.203+0000 7f9ed99f8740 1 -- 172.21.15.172:0/1422774316 shutdown_connections 2024-04-14T08:40:17.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.203+0000 7f9ed99f8740 1 -- 172.21.15.172:0/1422774316 wait complete. 2024-04-14T08:40:17.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.204+0000 7f9ed99f8740 1 -- start start 2024-04-14T08:40:17.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.204+0000 7f9ed99f8740 1 Processor -- start 2024-04-14T08:40:17.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ed99f8740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce73830 0x558afcf14430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ed99f8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7f970 0x558afcebee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ed99f8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce7fd50 0x558afcebf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ed99f8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558afce98f70 con 0x558afce7f970 2024-04-14T08:40:17.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ed99f8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed01ffdc0 con 0x558afce7fd50 2024-04-14T08:40:17.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ed99f8740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x558afcec0770 con 0x558afce73830 2024-04-14T08:40:17.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ed99f8740 10 client.0 _ll_register_callbacks cb 0x7f9ebc007740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ecffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7f970 0x558afcebee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ecffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7f970 0x558afcebee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42352/0 (socket says 172.21.15.172:42352) 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ecffff640 1 -- 172.21.15.172:0/2497009739 learned_addr learned my addr 172.21.15.172:0/2497009739 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ed48ba640 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce73830 0x558afcf14430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ed50bb640 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce7fd50 0x558afcebf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ecffff640 1 -- 172.21.15.172:0/2497009739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce73830 msgr2=0x558afcf14430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ecffff640 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce73830 0x558afcf14430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ecffff640 1 -- 172.21.15.172:0/2497009739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce7fd50 msgr2=0x558afcebf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ecffff640 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce7fd50 0x558afcebf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ecffff640 1 -- 172.21.15.172:0/2497009739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558afce95f10 con 0x558afce7f970 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ecffff640 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7f970 0x558afcebee50 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0011b0 tx=0x7f9ebc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f42921c1a1b4e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9eaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9eaf7fe640 1 -- 172.21.15.172:0/2497009739 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc034070 con 0x558afce7f970 2024-04-14T08:40:17.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9eaf7fe640 1 -- 172.21.15.172:0/2497009739 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f9ebc006b20 con 0x558afce7f970 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.206+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2497009739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec809b040 con 0x558afce7f970 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9eaf7fe640 1 -- 172.21.15.172:0/2497009739 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f9ebc03f050 con 0x558afce7f970 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9ed99f8740 10 client.8136 fetch_fsmap learned FSMap version 67 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2497009739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x558afcebfcf0 con 0x558afce7f970 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9eaf7fe640 1 -- 172.21.15.172:0/2497009739 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec809b040 con 0x558afce7f970 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9eaf7fe640 1 -- 172.21.15.172:0/2497009739 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f9ebc042020 con 0x558afce7f970 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9ed99f8740 10 client.8136 fetch_fsmap finished waiting for FSMap version 67 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9ed99f8740 10 client.8136 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9ed99f8740 20 client.8136 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x558afce82790 0x558afceaf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9ed99f8740 4 client.8136 mds_command: new command op to 8013 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2497009739 --> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558afce99690 con 0x558afce82790 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9ed48ba640 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x558afce82790 0x558afceaf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.207+0000 7f9eaf7fe640 10 client.8136 ms_handle_connect on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:17.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed48ba640 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x558afce82790 0x558afceaf4a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90253c86b8beb1c7 server_cookie=e46525f7c7ae5c74 in_seq=0 out_seq=0 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9eaf7fe640 1 -- 172.21.15.172:0/2497009739 <== mds.0 v2:172.21.15.172:6835/3091771014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x558afce99690 con 0x558afce82790 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9eaf7fe640 10 client.8136 handle_command_reply: tid=0 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed99f8740 1 client.8136 shutdown 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2497009739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7f970 msgr2=0x558afcebee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7f970 0x558afcebee50 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0011b0 tx=0x7f9ebc009040 comp rx=0 tx=0).stop 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2497009739 shutdown_connections 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x558afce82790 0x558afceaf4a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558afce7fd50 0x558afcebf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558afce7f970 0x558afcebee50 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed99f8740 1 --2- 172.21.15.172:0/2497009739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558afce73830 0x558afcf14430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2497009739 >> 172.21.15.172:0/2497009739 conn(0x558afce52e50 msgr2=0x558afce9dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9eaf7fe640 0 client.8136 ms_handle_reset on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2497009739 shutdown_connections 2024-04-14T08:40:17.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed99f8740 1 -- 172.21.15.172:0/2497009739 wait complete. 2024-04-14T08:40:17.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.208+0000 7f9ed99f8740 20 client.8136 trim_cache size 0 max 16384 2024-04-14T08:40:17.214 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:40:17.214 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-14T08:40:17.214 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-04-14T08:40:17.215 INFO:teuthology.orchestra.run.smithi172.stdout: "41f64c90-aad5-412f-8799-812a0c5f81f3": { 2024-04-14T08:40:17.215 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/dir", 2024-04-14T08:40:17.215 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "41f64c90-aad5-412f-8799-812a0c5f81f3", 2024-04-14T08:40:17.215 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,repair" 2024-04-14T08:40:17.215 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:40:17.215 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:40:17.215 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:40:17.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.209+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/3797422052 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9eb406c8f0 msgr2=0x7f9eb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.209+0000 7f9ed6b1e640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9eb406c8f0 0x7f9eb406edb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4006fd0 tx=0x7f9ec4008040 comp rx=0 tx=0).stop 2024-04-14T08:40:17.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.209+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/3797422052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed00d1a10 msgr2=0x7f9ed006f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:17.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.209+0000 7f9ed6b1e640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed00d1a10 0x7f9ed006f310 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8036040 tx=0x7f9ec8004a40 comp rx=0 tx=0).stop 2024-04-14T08:40:17.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.209+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/3797422052 shutdown_connections 2024-04-14T08:40:17.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.209+0000 7f9ed6b1e640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9eb406c8f0 0x7f9eb406edb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.209+0000 7f9ed6b1e640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed00d1a10 0x7f9ed006f310 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.209+0000 7f9ed6b1e640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ed00cffb0 0x7f9ed006edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.209+0000 7f9ed6b1e640 1 --2- 172.21.15.172:0/3797422052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ed00ccdc0 0x7f9ed00ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:17.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.209+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/3797422052 >> 172.21.15.172:0/3797422052 conn(0x7f9ed00d0f20 msgr2=0x7f9ed00d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:17.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.209+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/3797422052 shutdown_connections 2024-04-14T08:40:17.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:17.209+0000 7f9ed6b1e640 1 -- 172.21.15.172:0/3797422052 wait complete. 2024-04-14T08:40:17.221 INFO:tasks.cephfs.filesystem:scrub status for tag:41f64c90-aad5-412f-8799-812a0c5f81f3 - {'path': '/dir', 'tag': '41f64c90-aad5-412f-8799-812a0c5f81f3', 'options': 'recursive,repair'} 2024-04-14T08:40:24.667 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:40:24.670 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:40:47.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:47.223 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-14T08:40:47.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.327+0000 7f69326b0640 1 Processor -- start 2024-04-14T08:40:47.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.328+0000 7f69326b0640 1 -- start start 2024-04-14T08:40:47.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.329+0000 7f69326b0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f692c162710 0x7f692c0d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.329+0000 7f69326b0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f692c0cefb0 con 0x7f692c15e430 2024-04-14T08:40:47.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.329+0000 7f69326b0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f692c0cf290 con 0x7f692c162ba0 2024-04-14T08:40:47.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.329+0000 7f69326b0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f692c0cf570 con 0x7f692c162710 2024-04-14T08:40:47.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.329+0000 7f692bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f692c15e430 0x7f692c162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:41618/0 (socket says 172.21.15.172:41618) 2024-04-14T08:40:47.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.329+0000 7f692bfff640 1 -- 172.21.15.172:0/2372241535 learned_addr learned my addr 172.21.15.172:0/2372241535 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:47.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.329+0000 7f692b7fe640 1 --2- 172.21.15.172:0/2372241535 >> v2:172.21.15.177:3300/0 conn(0x7f692c162710 0x7f692c0d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.329+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3541647205 0 0) 0x7f692c0cefb0 con 0x7f692c15e430 2024-04-14T08:40:47.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.329+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6914003520 con 0x7f692c15e430 2024-04-14T08:40:47.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3208666481 0 0) 0x7f692c0cf290 con 0x7f692c162ba0 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f692c0cefb0 con 0x7f692c162ba0 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3835775853 0 0) 0x7f6914003520 con 0x7f692c15e430 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 >> v2:172.21.15.177:3300/0 conn(0x7f692c162710 msgr2=0x7f692c0d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 --2- 172.21.15.172:0/2372241535 >> v2:172.21.15.177:3300/0 conn(0x7f692c162710 0x7f692c0d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 >> v1:172.21.15.177:6789/0 conn(0x7f692c162ba0 legacy=0x7f692c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c0cf850 con 0x7f692c15e430 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f691c001670 con 0x7f692c15e430 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 >> v1:172.21.15.172:6789/0 conn(0x7f692c15e430 legacy=0x7f692c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 --2- 172.21.15.172:0/2372241535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6914003b90 0x7f6914003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 --2- 172.21.15.172:0/2372241535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6914004800 0x7f6914004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 --2- 172.21.15.172:0/2372241535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69140051c0 0x7f691400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f692c0cf570 con 0x7f6914004800 2024-04-14T08:40:47.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f691400c090 con 0x7f6914003b90 2024-04-14T08:40:47.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f691400c290 con 0x7f69140051c0 2024-04-14T08:40:47.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692b7fe640 1 --2- 172.21.15.172:0/2372241535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6914004800 0x7f6914004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692bfff640 1 --2- 172.21.15.172:0/2372241535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6914003b90 0x7f6914003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f6930c4d640 1 --2- 172.21.15.172:0/2372241535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69140051c0 0x7f691400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692b7fe640 1 -- 172.21.15.172:0/2372241535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69140051c0 msgr2=0x7f691400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692b7fe640 1 --2- 172.21.15.172:0/2372241535 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69140051c0 0x7f691400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692b7fe640 1 -- 172.21.15.172:0/2372241535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6914003b90 msgr2=0x7f6914003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692b7fe640 1 --2- 172.21.15.172:0/2372241535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6914003b90 0x7f6914003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f692b7fe640 1 -- 172.21.15.172:0/2372241535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6914003520 con 0x7f6914004800 2024-04-14T08:40:47.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.330+0000 7f6930c4d640 1 -- 172.21.15.172:0/2372241535 reap_dead start 2024-04-14T08:40:47.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.331+0000 7f692b7fe640 1 --2- 172.21.15.172:0/2372241535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6914004800 0x7f6914004c80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f6920009ac0 tx=0x7f6914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1deb0f82eee2e535 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.331+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920039070 con 0x7f6914004800 2024-04-14T08:40:47.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.331+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692002c6b0 con 0x7f6914004800 2024-04-14T08:40:47.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.331+0000 7f692affd640 1 -- 172.21.15.172:0/2372241535 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920009690 con 0x7f6914004800 2024-04-14T08:40:47.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.331+0000 7f69326b0640 1 -- 172.21.15.172:0/2372241535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6914004800 msgr2=0x7f6914004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.331+0000 7f69326b0640 1 --2- 172.21.15.172:0/2372241535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6914004800 0x7f6914004c80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f6920009ac0 tx=0x7f6914005830 comp rx=0 tx=0).stop 2024-04-14T08:40:47.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.332+0000 7f69326b0640 1 -- 172.21.15.172:0/2372241535 shutdown_connections 2024-04-14T08:40:47.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.332+0000 7f69326b0640 1 --2- 172.21.15.172:0/2372241535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6914004800 0x7f6914004c80 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.332+0000 7f69326b0640 1 -- 172.21.15.172:0/2372241535 >> 172.21.15.172:0/2372241535 conn(0x7f692c09d560 msgr2=0x7f692c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.332+0000 7f69326b0640 1 -- 172.21.15.172:0/2372241535 shutdown_connections 2024-04-14T08:40:47.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.332+0000 7f69326b0640 1 -- 172.21.15.172:0/2372241535 wait complete. 2024-04-14T08:40:47.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.333+0000 7f69326b0640 1 Processor -- start 2024-04-14T08:40:47.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.333+0000 7f69326b0640 1 -- start start 2024-04-14T08:40:47.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f69326b0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6914004800 0x7f692c14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f69326b0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f692c160b60 0x7f692c160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f69326b0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f692c0cb360 0x7f692c0cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f69326b0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f692c0d4e20 con 0x7f692c0cb360 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f69326b0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f692c15e810 con 0x7f692c160b60 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f69326b0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f692c0d0050 con 0x7f6914004800 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f6930c4d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f692c0cb360 0x7f692c0cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f6930c4d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f692c0cb360 0x7f692c0cb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42164/0 (socket says 172.21.15.172:42164) 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f6930c4d640 1 -- 172.21.15.172:0/3912848293 learned_addr learned my addr 172.21.15.172:0/3912848293 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f692b7fe640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f692c160b60 0x7f692c160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f692bfff640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6914004800 0x7f692c14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f6930c4d640 1 -- 172.21.15.172:0/3912848293 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6914004800 msgr2=0x7f692c14f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f6930c4d640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6914004800 0x7f692c14f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f6930c4d640 1 -- 172.21.15.172:0/3912848293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f692c160b60 msgr2=0x7f692c160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f6930c4d640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f692c160b60 0x7f692c160fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f6930c4d640 1 -- 172.21.15.172:0/3912848293 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c0cc0d0 con 0x7f692c0cb360 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f692b7fe640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f692c160b60 0x7f692c160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f6930c4d640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f692c0cb360 0x7f692c0cb7e0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f692c0a2180 tx=0x7f692400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f6928ff9640 1 -- 172.21.15.172:0/3912848293 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924019070 con 0x7f692c0cb360 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f6928ff9640 1 -- 172.21.15.172:0/3912848293 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6924006b50 con 0x7f692c0cb360 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f69326b0640 1 -- 172.21.15.172:0/3912848293 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692c0cc3b0 con 0x7f692c0cb360 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f6928ff9640 1 -- 172.21.15.172:0/3912848293 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924003970 con 0x7f692c0cb360 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.334+0000 7f69326b0640 1 -- 172.21.15.172:0/3912848293 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692c0cc970 con 0x7f692c0cb360 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.336+0000 7f6928ff9640 1 -- 172.21.15.172:0/3912848293 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f692401c070 con 0x7f692c0cb360 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.336+0000 7f6928ff9640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f691806c8f0 0x7f691806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.336+0000 7f6928ff9640 1 -- 172.21.15.172:0/3912848293 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f692408b570 con 0x7f692c0cb360 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.336+0000 7f692bfff640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f691806c8f0 0x7f691806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.337+0000 7f692bfff640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f691806c8f0 0x7f691806edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f691c000c00 tx=0x7f691c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.337+0000 7f6935570740 1 Processor -- start 2024-04-14T08:40:47.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.337+0000 7f6935570740 1 -- start start 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6935570740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6970 0x55e392cb9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6935570740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392ce6d50 0x55e392ce9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6935570740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392cb9e50 0x55e392ce9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6935570740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e392c23800 con 0x55e392cb9e50 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6935570740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e392bcb7f0 con 0x55e392ce6970 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6935570740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e392c1f7b0 con 0x55e392ce6d50 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6930c4d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392cb9e50 0x55e392ce9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6930c4d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392cb9e50 0x55e392ce9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42178/0 (socket says 172.21.15.172:42178) 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6930c4d640 1 -- 172.21.15.172:0/1581326078 learned_addr learned my addr 172.21.15.172:0/1581326078 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f692bfff640 1 --2- 172.21.15.172:0/1581326078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6970 0x55e392cb9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f692b7fe640 1 --2- 172.21.15.172:0/1581326078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392ce6d50 0x55e392ce9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6930c4d640 1 -- 172.21.15.172:0/1581326078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392ce6d50 msgr2=0x55e392ce9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6930c4d640 1 --2- 172.21.15.172:0/1581326078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392ce6d50 0x55e392ce9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6930c4d640 1 -- 172.21.15.172:0/1581326078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6970 msgr2=0x55e392cb9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6930c4d640 1 --2- 172.21.15.172:0/1581326078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6970 0x55e392cb9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f6930c4d640 1 -- 172.21.15.172:0/1581326078 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e392a017d0 con 0x55e392cb9e50 2024-04-14T08:40:47.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.338+0000 7f692bfff640 1 --2- 172.21.15.172:0/1581326078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6970 0x55e392cb9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:47.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.339+0000 7f6930c4d640 1 --2- 172.21.15.172:0/1581326078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392cb9e50 0x55e392ce9c40 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f6924089820 tx=0x7f69240598c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf1e64026c386a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.339+0000 7f6911ffb640 1 -- 172.21.15.172:0/1581326078 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924061c60 con 0x55e392cb9e50 2024-04-14T08:40:47.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.339+0000 7f6911ffb640 1 -- 172.21.15.172:0/1581326078 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6924057380 con 0x55e392cb9e50 2024-04-14T08:40:47.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.339+0000 7f6911ffb640 1 -- 172.21.15.172:0/1581326078 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69240556f0 con 0x55e392cb9e50 2024-04-14T08:40:47.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.339+0000 7f6935570740 1 -- 172.21.15.172:0/1581326078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392cb9e50 msgr2=0x55e392ce9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.340+0000 7f6935570740 1 --2- 172.21.15.172:0/1581326078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392cb9e50 0x55e392ce9c40 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f6924089820 tx=0x7f69240598c0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.340+0000 7f6935570740 1 -- 172.21.15.172:0/1581326078 shutdown_connections 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.340+0000 7f6935570740 1 --2- 172.21.15.172:0/1581326078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392cb9e50 0x55e392ce9c40 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.340+0000 7f6935570740 1 --2- 172.21.15.172:0/1581326078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392ce6d50 0x55e392ce9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.340+0000 7f6935570740 1 --2- 172.21.15.172:0/1581326078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6970 0x55e392cb9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.340+0000 7f6935570740 1 -- 172.21.15.172:0/1581326078 >> 172.21.15.172:0/1581326078 conn(0x55e392cbbb80 msgr2=0x55e392d04700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.340+0000 7f6935570740 1 -- 172.21.15.172:0/1581326078 shutdown_connections 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.340+0000 7f6935570740 1 -- 172.21.15.172:0/1581326078 wait complete. 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.340+0000 7f6935570740 1 -- start start 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.341+0000 7f6935570740 1 Processor -- start 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f6935570740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 0x55e392cebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f6935570740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392ce6d50 0x55e392cd63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f6935570740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392cd5d80 0x55e392cd68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f6935570740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e392cbe120 con 0x55e392ce6970 2024-04-14T08:40:47.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f6935570740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e392bded00 con 0x55e392cd5d80 2024-04-14T08:40:47.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f6935570740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e392cbd510 con 0x55e392ce6d50 2024-04-14T08:40:47.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f6935570740 10 client.0 _ll_register_callbacks cb 0x55e392bde6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:47.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f692bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 0x55e392cebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f692bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 0x55e392cebd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42184/0 (socket says 172.21.15.172:42184) 2024-04-14T08:40:47.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f692bfff640 1 -- 172.21.15.172:0/2901120922 learned_addr learned my addr 172.21.15.172:0/2901120922 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f692bfff640 1 -- 172.21.15.172:0/2901120922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392ce6d50 msgr2=0x55e392cd63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f6930c4d640 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392cd5d80 0x55e392cd68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f692b7fe640 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392ce6d50 0x55e392cd63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f692bfff640 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392ce6d50 0x55e392cd63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f692bfff640 1 -- 172.21.15.172:0/2901120922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392cd5d80 msgr2=0x55e392cd68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f692bfff640 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392cd5d80 0x55e392cd68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f692bfff640 1 -- 172.21.15.172:0/2901120922 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3929373f0 con 0x55e392ce6970 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.342+0000 7f6930c4d640 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392cd5d80 0x55e392cd68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f692bfff640 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 0x55e392cebd00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f691c0157d0 tx=0x7f691c001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef8f299981cb901 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f690affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f690affd640 1 -- 172.21.15.172:0/2901120922 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c029060 con 0x55e392ce6970 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f690affd640 1 -- 172.21.15.172:0/2901120922 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f691c024da0 con 0x55e392ce6970 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f6935570740 1 -- 172.21.15.172:0/2901120922 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6924057380 con 0x55e392ce6970 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f690affd640 1 -- 172.21.15.172:0/2901120922 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f691c0145e0 con 0x55e392ce6970 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f6935570740 10 client.8148 fetch_fsmap learned FSMap version 67 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f6935570740 1 -- 172.21.15.172:0/2901120922 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55e3928f70f0 con 0x55e392ce6970 2024-04-14T08:40:47.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f690affd640 1 -- 172.21.15.172:0/2901120922 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6924057380 con 0x55e392ce6970 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f690affd640 1 -- 172.21.15.172:0/2901120922 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f691c014780 con 0x55e392ce6970 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f6935570740 10 client.8148 fetch_fsmap finished waiting for FSMap version 67 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f6935570740 10 client.8148 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f6935570740 20 client.8148 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f6935570740 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55e392cd6e50 0x55e392d21730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f6935570740 4 client.8148 mds_command: new command op to 8013 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f6935570740 1 -- 172.21.15.172:0/2901120922 --> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e392c30d40 con 0x55e392cd6e50 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.343+0000 7f692b7fe640 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55e392cd6e50 0x55e392d21730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.344+0000 7f690affd640 10 client.8148 ms_handle_connect on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.344+0000 7f692b7fe640 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55e392cd6e50 0x55e392d21730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88483a8cdad92972 server_cookie=83b1c8f70b3ccf07 in_seq=0 out_seq=0 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.344+0000 7f690affd640 1 -- 172.21.15.172:0/2901120922 <== mds.0 v2:172.21.15.172:6835/3091771014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e392c30d40 con 0x55e392cd6e50 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.344+0000 7f690affd640 10 client.8148 handle_command_reply: tid=0 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.344+0000 7f6935570740 1 client.8148 shutdown 2024-04-14T08:40:47.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.344+0000 7f6935570740 1 -- 172.21.15.172:0/2901120922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 msgr2=0x55e392cebd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.344+0000 7f6935570740 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 0x55e392cebd00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f691c0157d0 tx=0x7f691c001d90 comp rx=0 tx=0).stop 2024-04-14T08:40:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.344+0000 7f6935570740 1 -- 172.21.15.172:0/2901120922 shutdown_connections 2024-04-14T08:40:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.344+0000 7f6935570740 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55e392cd6e50 0x55e392d21730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.345+0000 7f6935570740 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392cd5d80 0x55e392cd68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.345+0000 7f6935570740 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392ce6d50 0x55e392cd63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.345+0000 7f690affd640 0 client.8148 ms_handle_reset on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.345+0000 7f6935570740 1 --2- 172.21.15.172:0/2901120922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 0x55e392cebd00 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.345+0000 7f6935570740 1 -- 172.21.15.172:0/2901120922 >> 172.21.15.172:0/2901120922 conn(0x55e392cb9e50 msgr2=0x55e392d07b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.345+0000 7f6935570740 1 -- 172.21.15.172:0/2901120922 shutdown_connections 2024-04-14T08:40:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.345+0000 7f6935570740 1 -- 172.21.15.172:0/2901120922 wait complete. 2024-04-14T08:40:47.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.345+0000 7f6935570740 20 client.8148 trim_cache size 0 max 16384 2024-04-14T08:40:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.362+0000 7f6935570740 1 Processor -- start 2024-04-14T08:40:47.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.362+0000 7f6935570740 1 -- start start 2024-04-14T08:40:47.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6935570740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392cd5d80 0x55e392d26b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6935570740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6970 0x55e392cb50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6935570740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6d50 0x55e392cbd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6935570740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f692c201b70 con 0x55e392ce6d50 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6935570740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f692c200b70 con 0x55e392ce6970 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6935570740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f692c1fff00 con 0x55e392cd5d80 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6930c4d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6d50 0x55e392cbd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6930c4d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6d50 0x55e392cbd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42196/0 (socket says 172.21.15.172:42196) 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6930c4d640 1 -- 172.21.15.172:0/2039506643 learned_addr learned my addr 172.21.15.172:0/2039506643 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6930c4d640 1 -- 172.21.15.172:0/2039506643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392cd5d80 msgr2=0x55e392d26b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6930c4d640 1 --2- 172.21.15.172:0/2039506643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392cd5d80 0x55e392d26b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6930c4d640 1 -- 172.21.15.172:0/2039506643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6970 msgr2=0x55e392cb50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f692b7fe640 1 --2- 172.21.15.172:0/2039506643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6970 0x55e392cb50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6930c4d640 1 --2- 172.21.15.172:0/2039506643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6970 0x55e392cb50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6930c4d640 1 -- 172.21.15.172:0/2039506643 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e392c31470 con 0x55e392ce6d50 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f6930c4d640 1 --2- 172.21.15.172:0/2039506643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6d50 0x55e392cbd8b0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f69240898d0 tx=0x7f692408b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b2954e9c08aaed server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f69117fa640 1 -- 172.21.15.172:0/2039506643 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69240556f0 con 0x55e392ce6d50 2024-04-14T08:40:47.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f69117fa640 1 -- 172.21.15.172:0/2039506643 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692405d070 con 0x55e392ce6d50 2024-04-14T08:40:47.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.363+0000 7f69117fa640 1 -- 172.21.15.172:0/2039506643 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924055890 con 0x55e392ce6d50 2024-04-14T08:40:47.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.364+0000 7f6935570740 1 -- 172.21.15.172:0/2039506643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6d50 msgr2=0x55e392cbd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.364+0000 7f6935570740 1 --2- 172.21.15.172:0/2039506643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6d50 0x55e392cbd8b0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f69240898d0 tx=0x7f692408b050 comp rx=0 tx=0).stop 2024-04-14T08:40:47.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.364+0000 7f6935570740 1 -- 172.21.15.172:0/2039506643 shutdown_connections 2024-04-14T08:40:47.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.364+0000 7f6935570740 1 --2- 172.21.15.172:0/2039506643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6d50 0x55e392cbd8b0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.364+0000 7f6935570740 1 --2- 172.21.15.172:0/2039506643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6970 0x55e392cb50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.364+0000 7f6935570740 1 --2- 172.21.15.172:0/2039506643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392cd5d80 0x55e392d26b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.364+0000 7f6935570740 1 -- 172.21.15.172:0/2039506643 >> 172.21.15.172:0/2039506643 conn(0x55e392cb9e50 msgr2=0x55e392cec2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.364+0000 7f6935570740 1 -- 172.21.15.172:0/2039506643 shutdown_connections 2024-04-14T08:40:47.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.364+0000 7f6935570740 1 -- 172.21.15.172:0/2039506643 wait complete. 2024-04-14T08:40:47.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.365+0000 7f6935570740 1 -- start start 2024-04-14T08:40:47.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.365+0000 7f6935570740 1 Processor -- start 2024-04-14T08:40:47.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.365+0000 7f6935570740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392cd5d80 0x55e392d771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.365+0000 7f6935570740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 0x55e392d1dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f6935570740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6d50 0x55e392cd61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f6935570740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e392cc00e0 con 0x55e392ce6970 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f6935570740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f692c200520 con 0x55e392ce6d50 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f6935570740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e392d16660 con 0x55e392cd5d80 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f6935570740 10 client.0 _ll_register_callbacks cb 0x7f691c0073d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 0x55e392d1dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 0x55e392d1dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42200/0 (socket says 172.21.15.172:42200) 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f692b7fe640 1 -- 172.21.15.172:0/3798567388 learned_addr learned my addr 172.21.15.172:0/3798567388 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f692bfff640 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392cd5d80 0x55e392d771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f6930c4d640 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6d50 0x55e392cd61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f692b7fe640 1 -- 172.21.15.172:0/3798567388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392cd5d80 msgr2=0x55e392d771f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f692b7fe640 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392cd5d80 0x55e392d771f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f692b7fe640 1 -- 172.21.15.172:0/3798567388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6d50 msgr2=0x55e392cd61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f692b7fe640 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6d50 0x55e392cd61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f692b7fe640 1 -- 172.21.15.172:0/3798567388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e392d137e0 con 0x55e392ce6970 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f692b7fe640 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 0x55e392d1dca0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f6920009be0 tx=0x7f692002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7783799ed33f47cc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f690affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f690affd640 1 -- 172.21.15.172:0/3798567388 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692004e060 con 0x55e392ce6970 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f690affd640 1 -- 172.21.15.172:0/3798567388 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f69200040c0 con 0x55e392ce6970 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f6935570740 1 -- 172.21.15.172:0/3798567388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f692405d070 con 0x55e392ce6970 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f690affd640 1 -- 172.21.15.172:0/3798567388 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f6920054050 con 0x55e392ce6970 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f6935570740 10 client.8154 fetch_fsmap learned FSMap version 67 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.366+0000 7f6935570740 1 -- 172.21.15.172:0/3798567388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55e392cec2c0 con 0x55e392ce6970 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f690affd640 1 -- 172.21.15.172:0/3798567388 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692405d070 con 0x55e392ce6970 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f690affd640 1 -- 172.21.15.172:0/3798567388 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f6920057020 con 0x55e392ce6970 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f6935570740 10 client.8154 fetch_fsmap finished waiting for FSMap version 67 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f6935570740 10 client.8154 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-14T08:40:47.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f6935570740 20 client.8154 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f6935570740 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55e392cd6750 0x55e392d26980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f6935570740 4 client.8154 mds_command: new command op to 8013 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f6935570740 1 -- 172.21.15.172:0/3798567388 --> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e392cfe5f0 con 0x55e392cd6750 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f692bfff640 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55e392cd6750 0x55e392d26980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f690affd640 10 client.8154 ms_handle_connect on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f692bfff640 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55e392cd6750 0x55e392d26980 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d735fc68d73b72 server_cookie=3f0f3a558541a2d8 in_seq=0 out_seq=0 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f690affd640 1 -- 172.21.15.172:0/3798567388 <== mds.0 v2:172.21.15.172:6835/3091771014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e392cfe5f0 con 0x55e392cd6750 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f690affd640 10 client.8154 handle_command_reply: tid=0 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.367+0000 7f6935570740 1 client.8154 shutdown 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f6935570740 1 -- 172.21.15.172:0/3798567388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 msgr2=0x55e392d1dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f6935570740 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 0x55e392d1dca0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f6920009be0 tx=0x7f692002e070 comp rx=0 tx=0).stop 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f6935570740 1 -- 172.21.15.172:0/3798567388 shutdown_connections 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f6935570740 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x55e392cd6750 0x55e392d26980 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f6935570740 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e392ce6d50 0x55e392cd61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f6935570740 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e392ce6970 0x55e392d1dca0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f690affd640 0 client.8154 ms_handle_reset on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:47.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f6935570740 1 --2- 172.21.15.172:0/3798567388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e392cd5d80 0x55e392d771f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f6935570740 1 -- 172.21.15.172:0/3798567388 >> 172.21.15.172:0/3798567388 conn(0x55e392cb9e50 msgr2=0x55e392cbd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f6935570740 1 -- 172.21.15.172:0/3798567388 shutdown_connections 2024-04-14T08:40:47.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f6935570740 1 -- 172.21.15.172:0/3798567388 wait complete. 2024-04-14T08:40:47.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f6935570740 20 client.8154 trim_cache size 0 max 16384 2024-04-14T08:40:47.374 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:40:47.374 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-04-14T08:40:47.375 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-04-14T08:40:47.375 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:40:47.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f69326b0640 1 -- 172.21.15.172:0/3912848293 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f691806c8f0 msgr2=0x7f691806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f69326b0640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f691806c8f0 0x7f691806edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f691c000c00 tx=0x7f691c009040 comp rx=0 tx=0).stop 2024-04-14T08:40:47.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f69326b0640 1 -- 172.21.15.172:0/3912848293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f692c0cb360 msgr2=0x7f692c0cb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f69326b0640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f692c0cb360 0x7f692c0cb7e0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f692c0a2180 tx=0x7f692400eb90 comp rx=0 tx=0).stop 2024-04-14T08:40:47.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f69326b0640 1 -- 172.21.15.172:0/3912848293 shutdown_connections 2024-04-14T08:40:47.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f69326b0640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f691806c8f0 0x7f691806edb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f69326b0640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f692c0cb360 0x7f692c0cb7e0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f69326b0640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f692c160b60 0x7f692c160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f69326b0640 1 --2- 172.21.15.172:0/3912848293 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6914004800 0x7f692c14f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f69326b0640 1 -- 172.21.15.172:0/3912848293 >> 172.21.15.172:0/3912848293 conn(0x7f692c09d560 msgr2=0x7f692c0b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f69326b0640 1 -- 172.21.15.172:0/3912848293 shutdown_connections 2024-04-14T08:40:47.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.368+0000 7f69326b0640 1 -- 172.21.15.172:0/3912848293 wait complete. 2024-04-14T08:40:47.380 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-14T08:40:47.380 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:40:47.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.481+0000 7fe97b755640 1 Processor -- start 2024-04-14T08:40:47.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.481+0000 7fe97b755640 1 -- start start 2024-04-14T08:40:47.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.482+0000 7fe97b755640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe97415e890 0x7fe9740d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.482+0000 7fe97b755640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9740cedd0 0x7fe9740b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.482+0000 7fe97b755640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9740cf320 con 0x7fe9740d2830 2024-04-14T08:40:47.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.482+0000 7fe97b755640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe9740cff30 con 0x7fe97415e890 2024-04-14T08:40:47.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.482+0000 7fe97b755640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe9740d2350 con 0x7fe9740cedd0 2024-04-14T08:40:47.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.482+0000 7fe978cf0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe97415e890 0x7fe9740d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.482+0000 7fe978cf0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe97415e890 0x7fe9740d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42212/0 (socket says 172.21.15.172:42212) 2024-04-14T08:40:47.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.482+0000 7fe978cf0640 1 -- 172.21.15.172:0/2717231443 learned_addr learned my addr 172.21.15.172:0/2717231443 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.482+0000 7fe978cf0640 1 -- 172.21.15.172:0/2717231443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9740cedd0 msgr2=0x7fe9740b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.482+0000 7fe978cf0640 1 --2- 172.21.15.172:0/2717231443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9740cedd0 0x7fe9740b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.482+0000 7fe978cf0640 1 -- 172.21.15.172:0/2717231443 >> v1:172.21.15.177:6789/0 conn(0x7fe9740d2830 legacy=0x7fe9740d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.482+0000 7fe978cf0640 1 -- 172.21.15.172:0/2717231443 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9740cf600 con 0x7fe97415e890 2024-04-14T08:40:47.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.483+0000 7fe978cf0640 1 --2- 172.21.15.172:0/2717231443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe97415e890 0x7fe9740d1300 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe964009870 tx=0x7fe96402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce3800e044aefcd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.483+0000 7fe96bfff640 1 -- 172.21.15.172:0/2717231443 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964034070 con 0x7fe97415e890 2024-04-14T08:40:47.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.483+0000 7fe96bfff640 1 -- 172.21.15.172:0/2717231443 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe964030840 con 0x7fe97415e890 2024-04-14T08:40:47.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.483+0000 7fe96bfff640 1 -- 172.21.15.172:0/2717231443 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964030c10 con 0x7fe97415e890 2024-04-14T08:40:47.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.484+0000 7fe97b755640 1 -- 172.21.15.172:0/2717231443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe97415e890 msgr2=0x7fe9740d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.484+0000 7fe97b755640 1 --2- 172.21.15.172:0/2717231443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe97415e890 0x7fe9740d1300 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe964009870 tx=0x7fe96402b980 comp rx=0 tx=0).stop 2024-04-14T08:40:47.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.484+0000 7fe97b755640 1 -- 172.21.15.172:0/2717231443 shutdown_connections 2024-04-14T08:40:47.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.484+0000 7fe97b755640 1 --2- 172.21.15.172:0/2717231443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9740cedd0 0x7fe9740b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.484+0000 7fe97b755640 1 --2- 172.21.15.172:0/2717231443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe97415e890 0x7fe9740d1300 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.484+0000 7fe97b755640 1 -- 172.21.15.172:0/2717231443 >> 172.21.15.172:0/2717231443 conn(0x7fe97409d590 msgr2=0x7fe9740d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.484+0000 7fe97b755640 1 -- 172.21.15.172:0/2717231443 shutdown_connections 2024-04-14T08:40:47.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.484+0000 7fe97b755640 1 -- 172.21.15.172:0/2717231443 wait complete. 2024-04-14T08:40:47.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.486+0000 7fe97b755640 1 Processor -- start 2024-04-14T08:40:47.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.486+0000 7fe97b755640 1 -- start start 2024-04-14T08:40:47.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe97b755640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9740cedd0 0x7fe974160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe97b755640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9740d2830 0x7fe974159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe97b755640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe97415e890 0x7fe97415c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe97b755640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9740d1840 con 0x7fe97415e890 2024-04-14T08:40:47.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe97b755640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe9740cfbc0 con 0x7fe9740cedd0 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe97b755640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe9740b3c50 con 0x7fe9740d2830 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe979cf2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe97415e890 0x7fe97415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe979cf2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe97415e890 0x7fe97415c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42220/0 (socket says 172.21.15.172:42220) 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe979cf2640 1 -- 172.21.15.172:0/1406239994 learned_addr learned my addr 172.21.15.172:0/1406239994 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe978cf0640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9740d2830 0x7fe974159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe9794f1640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9740cedd0 0x7fe974160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe979cf2640 1 -- 172.21.15.172:0/1406239994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9740d2830 msgr2=0x7fe974159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe979cf2640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9740d2830 0x7fe974159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe979cf2640 1 -- 172.21.15.172:0/1406239994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9740cedd0 msgr2=0x7fe974160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe979cf2640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9740cedd0 0x7fe974160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe979cf2640 1 -- 172.21.15.172:0/1406239994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe974160740 con 0x7fe97415e890 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe979cf2640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe97415e890 0x7fe97415c710 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00b970 tx=0x7fe96c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe969ffb640 1 -- 172.21.15.172:0/1406239994 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c019070 con 0x7fe97415e890 2024-04-14T08:40:47.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe969ffb640 1 -- 172.21.15.172:0/1406239994 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96c00cb60 con 0x7fe97415e890 2024-04-14T08:40:47.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe97b755640 1 -- 172.21.15.172:0/1406239994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe974160a20 con 0x7fe97415e890 2024-04-14T08:40:47.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.487+0000 7fe969ffb640 1 -- 172.21.15.172:0/1406239994 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c014440 con 0x7fe97415e890 2024-04-14T08:40:47.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.488+0000 7fe97b755640 1 -- 172.21.15.172:0/1406239994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe974160fe0 con 0x7fe97415e890 2024-04-14T08:40:47.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.489+0000 7fe97b755640 1 -- 172.21.15.172:0/1406239994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe974068520 con 0x7fe97415e890 2024-04-14T08:40:47.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.489+0000 7fe969ffb640 1 -- 172.21.15.172:0/1406239994 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe96c01c070 con 0x7fe97415e890 2024-04-14T08:40:47.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.489+0000 7fe969ffb640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe95806c8f0 0x7fe95806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.490+0000 7fe969ffb640 1 -- 172.21.15.172:0/1406239994 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fe96c08bac0 con 0x7fe97415e890 2024-04-14T08:40:47.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.490+0000 7fe9794f1640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe95806c8f0 0x7fe95806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.490+0000 7fe9794f1640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe95806c8f0 0x7fe95806edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe9700017f0 tx=0x7fe970009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.494+0000 7fe969ffb640 1 -- 172.21.15.172:0/1406239994 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe96c0595f0 con 0x7fe97415e890 2024-04-14T08:40:47.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.715+0000 7fe97b755640 1 -- 172.21.15.172:0/1406239994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9740d2610 con 0x7fe97415e890 2024-04-14T08:40:47.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.716+0000 7fe969ffb640 1 -- 172.21.15.172:0/1406239994 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5196 (secure 0 0 0) 0x7fe96c05d8f0 con 0x7fe97415e890 2024-04-14T08:40:47.716 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:47.717 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:40:14.122456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8013":{"gid":8013,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.172:6837/3091771014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3091771014},{"type":"v1","addr":"172.21.15.172:6837","nonce":3091771014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8013,"qdb_cluster":[8013]},"id":6}]} 2024-04-14T08:40:47.717 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 67 2024-04-14T08:40:47.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.718+0000 7fe97b755640 1 -- 172.21.15.172:0/1406239994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe95806c8f0 msgr2=0x7fe95806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.718+0000 7fe97b755640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe95806c8f0 0x7fe95806edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe9700017f0 tx=0x7fe970009040 comp rx=0 tx=0).stop 2024-04-14T08:40:47.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.718+0000 7fe97b755640 1 -- 172.21.15.172:0/1406239994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe97415e890 msgr2=0x7fe97415c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.718+0000 7fe97b755640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe97415e890 0x7fe97415c710 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00b970 tx=0x7fe96c00be30 comp rx=0 tx=0).stop 2024-04-14T08:40:47.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.718+0000 7fe97b755640 1 -- 172.21.15.172:0/1406239994 shutdown_connections 2024-04-14T08:40:47.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.718+0000 7fe97b755640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe95806c8f0 0x7fe95806edb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.718+0000 7fe97b755640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe97415e890 0x7fe97415c710 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.718+0000 7fe97b755640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9740d2830 0x7fe974159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.718+0000 7fe97b755640 1 --2- 172.21.15.172:0/1406239994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9740cedd0 0x7fe974160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.718+0000 7fe97b755640 1 -- 172.21.15.172:0/1406239994 >> 172.21.15.172:0/1406239994 conn(0x7fe97409d590 msgr2=0x7fe9740d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.718+0000 7fe97b755640 1 -- 172.21.15.172:0/1406239994 shutdown_connections 2024-04-14T08:40:47.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.718+0000 7fe97b755640 1 -- 172.21.15.172:0/1406239994 wait complete. 2024-04-14T08:40:47.730 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-14T08:40:47.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.834+0000 7fa6293aa640 1 Processor -- start 2024-04-14T08:40:47.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.834+0000 7fa6293aa640 1 -- start start 2024-04-14T08:40:47.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa6293aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62415e590 0x7fa6241629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa6293aa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa6240cea60 0x7fa6240b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa6293aa640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6240b3750 con 0x7fa6240b63d0 2024-04-14T08:40:47.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa6293aa640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa6240cef20 con 0x7fa6240cea60 2024-04-14T08:40:47.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa6293aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa6240cf200 con 0x7fa62415e590 2024-04-14T08:40:47.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa61bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa6240cea60 0x7fa6240b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa6237fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa6240b63d0 0x7fa62406c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:41620/0 (socket says 172.21.15.172:41620) 2024-04-14T08:40:47.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa6237fe640 1 -- 172.21.15.172:0/4211057288 learned_addr learned my addr 172.21.15.172:0/4211057288 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:47.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa622ffd640 1 --2- 172.21.15.172:0/4211057288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62415e590 0x7fa6241629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa61bfff640 1 -- 172.21.15.172:0/4211057288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62415e590 msgr2=0x7fa6241629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa61bfff640 1 --2- 172.21.15.172:0/4211057288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62415e590 0x7fa6241629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa61bfff640 1 -- 172.21.15.172:0/4211057288 >> v1:172.21.15.172:6789/0 conn(0x7fa6240b63d0 legacy=0x7fa62406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.835+0000 7fa61bfff640 1 -- 172.21.15.172:0/4211057288 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6240cf4e0 con 0x7fa6240cea60 2024-04-14T08:40:47.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.836+0000 7fa61bfff640 1 --2- 172.21.15.172:0/4211057288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6240cea60 0x7fa6240b5e90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0097a0 tx=0x7fa60c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f93a1290c4f328 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.836+0000 7fa6227fc640 1 -- 172.21.15.172:0/4211057288 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c035020 con 0x7fa6240cea60 2024-04-14T08:40:47.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.836+0000 7fa6227fc640 1 -- 172.21.15.172:0/4211057288 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa60c031830 con 0x7fa6240cea60 2024-04-14T08:40:47.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.836+0000 7fa6227fc640 1 -- 172.21.15.172:0/4211057288 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c031c00 con 0x7fa6240cea60 2024-04-14T08:40:47.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.837+0000 7fa6293aa640 1 -- 172.21.15.172:0/4211057288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6240cea60 msgr2=0x7fa6240b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.837+0000 7fa6293aa640 1 --2- 172.21.15.172:0/4211057288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6240cea60 0x7fa6240b5e90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0097a0 tx=0x7fa60c02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.837+0000 7fa6293aa640 1 -- 172.21.15.172:0/4211057288 shutdown_connections 2024-04-14T08:40:47.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.837+0000 7fa6293aa640 1 --2- 172.21.15.172:0/4211057288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6240cea60 0x7fa6240b5e90 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.837+0000 7fa6293aa640 1 --2- 172.21.15.172:0/4211057288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62415e590 0x7fa6241629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.837+0000 7fa6293aa640 1 -- 172.21.15.172:0/4211057288 >> 172.21.15.172:0/4211057288 conn(0x7fa6240d0e90 msgr2=0x7fa6240d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.837+0000 7fa6293aa640 1 -- 172.21.15.172:0/4211057288 shutdown_connections 2024-04-14T08:40:47.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.837+0000 7fa6293aa640 1 -- 172.21.15.172:0/4211057288 wait complete. 2024-04-14T08:40:47.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.838+0000 7fa6293aa640 1 Processor -- start 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.838+0000 7fa6293aa640 1 -- start start 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6293aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6240b63d0 0x7fa6240ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6293aa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6240cea60 0x7fa6240b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6293aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa62415e590 0x7fa62414f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6293aa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6240d38f0 con 0x7fa62415e590 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6293aa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa6240d3770 con 0x7fa6240cea60 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6293aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa6240d3a70 con 0x7fa6240b63d0 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa62415e590 0x7fa62414f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa62415e590 0x7fa62414f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42238/0 (socket says 172.21.15.172:42238) 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6237fe640 1 -- 172.21.15.172:0/1747448171 learned_addr learned my addr 172.21.15.172:0/1747448171 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa622ffd640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6240b63d0 0x7fa6240ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa61bfff640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6240cea60 0x7fa6240b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6237fe640 1 -- 172.21.15.172:0/1747448171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6240b63d0 msgr2=0x7fa6240ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6237fe640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6240b63d0 0x7fa6240ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6237fe640 1 -- 172.21.15.172:0/1747448171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6240cea60 msgr2=0x7fa6240b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6237fe640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6240cea60 0x7fa6240b7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6237fe640 1 -- 172.21.15.172:0/1747448171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6240cd510 con 0x7fa62415e590 2024-04-14T08:40:47.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa61bfff640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6240cea60 0x7fa6240b7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:47.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.839+0000 7fa6237fe640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa62415e590 0x7fa62414f1a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00cbc0 tx=0x7fa61c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.840+0000 7fa61b7fe640 1 -- 172.21.15.172:0/1747448171 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c017070 con 0x7fa62415e590 2024-04-14T08:40:47.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.840+0000 7fa61b7fe640 1 -- 172.21.15.172:0/1747448171 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c00ad40 con 0x7fa62415e590 2024-04-14T08:40:47.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.840+0000 7fa6293aa640 1 -- 172.21.15.172:0/1747448171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6240c7fe0 con 0x7fa62415e590 2024-04-14T08:40:47.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.840+0000 7fa61b7fe640 1 -- 172.21.15.172:0/1747448171 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c012600 con 0x7fa62415e590 2024-04-14T08:40:47.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.840+0000 7fa6293aa640 1 -- 172.21.15.172:0/1747448171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6240cd140 con 0x7fa62415e590 2024-04-14T08:40:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.842+0000 7fa61b7fe640 1 -- 172.21.15.172:0/1747448171 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa61c01a070 con 0x7fa62415e590 2024-04-14T08:40:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.842+0000 7fa61b7fe640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa60406c8f0 0x7fa60406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.842+0000 7fa61b7fe640 1 -- 172.21.15.172:0/1747448171 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fa61c08bad0 con 0x7fa62415e590 2024-04-14T08:40:47.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.842+0000 7fa622ffd640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa60406c8f0 0x7fa60406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.843+0000 7fa622ffd640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa60406c8f0 0x7fa60406edb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa614009aa0 tx=0x7fa614009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.843+0000 7fa62c18d740 1 Processor -- start 2024-04-14T08:40:47.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.843+0000 7fa62c18d740 1 -- start start 2024-04-14T08:40:47.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa62c18d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d26d90 0x5610a8d27170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa62c18d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d2cbb0 0x5610a8d2ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa62c18d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d74920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa62c18d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5610a8cee5a0 con 0x5610a8d744a0 2024-04-14T08:40:47.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa62c18d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5610a8c90680 con 0x5610a8d2cbb0 2024-04-14T08:40:47.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa62c18d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5610a8c2e860 con 0x5610a8d26d90 2024-04-14T08:40:47.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d74920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d74920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42248/0 (socket says 172.21.15.172:42248) 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa6237fe640 1 -- 172.21.15.172:0/2071525026 learned_addr learned my addr 172.21.15.172:0/2071525026 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa6237fe640 1 -- 172.21.15.172:0/2071525026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d26d90 msgr2=0x5610a8d27170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa6237fe640 1 --2- 172.21.15.172:0/2071525026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d26d90 0x5610a8d27170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa6237fe640 1 -- 172.21.15.172:0/2071525026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d2cbb0 msgr2=0x5610a8d2ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa61bfff640 1 --2- 172.21.15.172:0/2071525026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d2cbb0 0x5610a8d2ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa6237fe640 1 --2- 172.21.15.172:0/2071525026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d2cbb0 0x5610a8d2ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa6237fe640 1 -- 172.21.15.172:0/2071525026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a8c5d1a0 con 0x5610a8d744a0 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa6237fe640 1 --2- 172.21.15.172:0/2071525026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d74920 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa61c089d50 tx=0x7fa61c0596c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fe5202b77d4d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.844+0000 7fa618ff9640 1 -- 172.21.15.172:0/2071525026 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c08b030 con 0x5610a8d744a0 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.845+0000 7fa618ff9640 1 -- 172.21.15.172:0/2071525026 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c057900 con 0x5610a8d744a0 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.845+0000 7fa618ff9640 1 -- 172.21.15.172:0/2071525026 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c061ca0 con 0x5610a8d744a0 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.845+0000 7fa62c18d740 1 -- 172.21.15.172:0/2071525026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 msgr2=0x5610a8d74920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.845+0000 7fa62c18d740 1 --2- 172.21.15.172:0/2071525026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d74920 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa61c089d50 tx=0x7fa61c0596c0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.845+0000 7fa62c18d740 1 -- 172.21.15.172:0/2071525026 shutdown_connections 2024-04-14T08:40:47.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.845+0000 7fa62c18d740 1 --2- 172.21.15.172:0/2071525026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d74920 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.845+0000 7fa62c18d740 1 --2- 172.21.15.172:0/2071525026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d2cbb0 0x5610a8d2ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.845+0000 7fa62c18d740 1 --2- 172.21.15.172:0/2071525026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d26d90 0x5610a8d27170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.845+0000 7fa62c18d740 1 -- 172.21.15.172:0/2071525026 >> 172.21.15.172:0/2071525026 conn(0x5610a8d28e10 msgr2=0x5610a8d2a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.845+0000 7fa62c18d740 1 -- 172.21.15.172:0/2071525026 shutdown_connections 2024-04-14T08:40:47.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.845+0000 7fa62c18d740 1 -- 172.21.15.172:0/2071525026 wait complete. 2024-04-14T08:40:47.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.846+0000 7fa62c18d740 1 -- start start 2024-04-14T08:40:47.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.846+0000 7fa62c18d740 1 Processor -- start 2024-04-14T08:40:47.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa62c18d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d2cbb0 0x5610a8d4bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa62c18d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d744a0 0x5610a8d6c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa62c18d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8d56b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa62c18d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5610a8c2c1d0 con 0x5610a8d2cbb0 2024-04-14T08:40:47.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa62c18d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5610a8c8c630 con 0x5610a8d566d0 2024-04-14T08:40:47.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa62c18d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5610a8d53500 con 0x5610a8d744a0 2024-04-14T08:40:47.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa62c18d740 10 client.0 _ll_register_callbacks cb 0x5610a8c4f010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:47.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa622ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d2cbb0 0x5610a8d4bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa622ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d2cbb0 0x5610a8d4bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42254/0 (socket says 172.21.15.172:42254) 2024-04-14T08:40:47.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa622ffd640 1 -- 172.21.15.172:0/1875721083 learned_addr learned my addr 172.21.15.172:0/1875721083 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa6237fe640 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8d56b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa61bfff640 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d744a0 0x5610a8d6c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa622ffd640 1 -- 172.21.15.172:0/1875721083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d744a0 msgr2=0x5610a8d6c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa622ffd640 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d744a0 0x5610a8d6c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa622ffd640 1 -- 172.21.15.172:0/1875721083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 msgr2=0x5610a8d56b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa622ffd640 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8d56b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa622ffd640 1 -- 172.21.15.172:0/1875721083 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a8a707c0 con 0x5610a8d2cbb0 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa6237fe640 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8d56b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa622ffd640 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d2cbb0 0x5610a8d4bfa0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa614011f80 tx=0x7fa614011fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68896123846cf23c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa601ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa601ffb640 1 -- 172.21.15.172:0/1875721083 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614003d20 con 0x5610a8d2cbb0 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa62c18d740 1 -- 172.21.15.172:0/1875721083 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61c057900 con 0x5610a8d2cbb0 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.847+0000 7fa601ffb640 1 -- 172.21.15.172:0/1875721083 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fa614014440 con 0x5610a8d2cbb0 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa601ffb640 1 -- 172.21.15.172:0/1875721083 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fa614014640 con 0x5610a8d2cbb0 2024-04-14T08:40:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa62c18d740 10 client.8172 fetch_fsmap learned FSMap version 67 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa62c18d740 1 -- 172.21.15.172:0/1875721083 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5610a8ad0f50 con 0x5610a8d2cbb0 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa601ffb640 1 -- 172.21.15.172:0/1875721083 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c057900 con 0x5610a8d2cbb0 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa601ffb640 1 -- 172.21.15.172:0/1875721083 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fa614031080 con 0x5610a8d2cbb0 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa62c18d740 10 client.8172 fetch_fsmap finished waiting for FSMap version 67 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa62c18d740 10 client.8172 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa62c18d740 20 client.8172 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa62c18d740 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x5610a8d59c00 0x5610a8d59fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa62c18d740 4 client.8172 mds_command: new command op to 8013 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa62c18d740 1 -- 172.21.15.172:0/1875721083 --> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610a8ca3eb0 con 0x5610a8d59c00 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa61bfff640 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x5610a8d59c00 0x5610a8d59fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa601ffb640 10 client.8172 ms_handle_connect on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.848+0000 7fa61bfff640 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x5610a8d59c00 0x5610a8d59fe0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5600c45240b0562a server_cookie=72f1d069f78e3aca in_seq=0 out_seq=0 2024-04-14T08:40:47.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa601ffb640 1 -- 172.21.15.172:0/1875721083 <== mds.0 v2:172.21.15.172:6835/3091771014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610a8ca3eb0 con 0x5610a8d59c00 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa601ffb640 10 client.8172 handle_command_reply: tid=0 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa62c18d740 1 client.8172 shutdown 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa62c18d740 1 -- 172.21.15.172:0/1875721083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d2cbb0 msgr2=0x5610a8d4bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa62c18d740 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d2cbb0 0x5610a8d4bfa0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa614011f80 tx=0x7fa614011fb0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa62c18d740 1 -- 172.21.15.172:0/1875721083 shutdown_connections 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa62c18d740 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x5610a8d59c00 0x5610a8d59fe0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa62c18d740 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8d56b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa62c18d740 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d744a0 0x5610a8d6c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa601ffb640 0 client.8172 ms_handle_reset on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa62c18d740 1 --2- 172.21.15.172:0/1875721083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d2cbb0 0x5610a8d4bfa0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa62c18d740 1 -- 172.21.15.172:0/1875721083 >> 172.21.15.172:0/1875721083 conn(0x5610a8d26d90 msgr2=0x5610a8d72450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa62c18d740 1 -- 172.21.15.172:0/1875721083 shutdown_connections 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa62c18d740 1 -- 172.21.15.172:0/1875721083 wait complete. 2024-04-14T08:40:47.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.849+0000 7fa62c18d740 20 client.8172 trim_cache size 0 max 16384 2024-04-14T08:40:47.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.875+0000 7fa62c18d740 1 Processor -- start 2024-04-14T08:40:47.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.875+0000 7fa62c18d740 1 -- start start 2024-04-14T08:40:47.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa62c18d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d2cbb0 0x5610a8d52ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa62c18d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8d72f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa62c18d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d73450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa62c18d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa624201530 con 0x5610a8d744a0 2024-04-14T08:40:47.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa62c18d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa624200530 con 0x5610a8d566d0 2024-04-14T08:40:47.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa62c18d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa62415f960 con 0x5610a8d2cbb0 2024-04-14T08:40:47.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d73450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d73450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42260/0 (socket says 172.21.15.172:42260) 2024-04-14T08:40:47.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa6237fe640 1 -- 172.21.15.172:0/3178571501 learned_addr learned my addr 172.21.15.172:0/3178571501 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa6237fe640 1 -- 172.21.15.172:0/3178571501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d2cbb0 msgr2=0x5610a8d52ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa6237fe640 1 --2- 172.21.15.172:0/3178571501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d2cbb0 0x5610a8d52ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa61bfff640 1 --2- 172.21.15.172:0/3178571501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8d72f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa6237fe640 1 -- 172.21.15.172:0/3178571501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 msgr2=0x5610a8d72f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa6237fe640 1 --2- 172.21.15.172:0/3178571501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8d72f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa6237fe640 1 -- 172.21.15.172:0/3178571501 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a89640b0 con 0x5610a8d744a0 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa6237fe640 1 --2- 172.21.15.172:0/3178571501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d73450 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa61c055a60 tx=0x7fa61c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9123c8b8c41110e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.876+0000 7fa603fff640 1 -- 172.21.15.172:0/3178571501 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c087020 con 0x5610a8d744a0 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.877+0000 7fa603fff640 1 -- 172.21.15.172:0/3178571501 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c055c70 con 0x5610a8d744a0 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.877+0000 7fa603fff640 1 -- 172.21.15.172:0/3178571501 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c08b050 con 0x5610a8d744a0 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.877+0000 7fa62c18d740 1 -- 172.21.15.172:0/3178571501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 msgr2=0x5610a8d73450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.877+0000 7fa62c18d740 1 --2- 172.21.15.172:0/3178571501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d73450 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa61c055a60 tx=0x7fa61c01e000 comp rx=0 tx=0).stop 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.877+0000 7fa62c18d740 1 -- 172.21.15.172:0/3178571501 shutdown_connections 2024-04-14T08:40:47.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.877+0000 7fa62c18d740 1 --2- 172.21.15.172:0/3178571501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d73450 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.877+0000 7fa62c18d740 1 --2- 172.21.15.172:0/3178571501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8d72f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.877+0000 7fa62c18d740 1 --2- 172.21.15.172:0/3178571501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d2cbb0 0x5610a8d52ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.877+0000 7fa62c18d740 1 -- 172.21.15.172:0/3178571501 >> 172.21.15.172:0/3178571501 conn(0x5610a8d26d90 msgr2=0x5610a8d56b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.877+0000 7fa62c18d740 1 -- 172.21.15.172:0/3178571501 shutdown_connections 2024-04-14T08:40:47.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.877+0000 7fa62c18d740 1 -- 172.21.15.172:0/3178571501 wait complete. 2024-04-14T08:40:47.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.878+0000 7fa62c18d740 1 -- start start 2024-04-14T08:40:47.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.878+0000 7fa62c18d740 1 Processor -- start 2024-04-14T08:40:47.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.878+0000 7fa62c18d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d2cbb0 0x5610a8d8be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.878+0000 7fa62c18d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8de8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.878+0000 7fa62c18d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d25a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.878+0000 7fa62c18d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5610a8d6cc40 con 0x5610a8d744a0 2024-04-14T08:40:47.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.878+0000 7fa62c18d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa6241ffee0 con 0x5610a8d566d0 2024-04-14T08:40:47.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.878+0000 7fa62c18d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5610a8d6d490 con 0x5610a8d2cbb0 2024-04-14T08:40:47.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa62c18d740 10 client.0 _ll_register_callbacks cb 0x7fa614009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:40:47.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d25a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d25a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42268/0 (socket says 172.21.15.172:42268) 2024-04-14T08:40:47.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa6237fe640 1 -- 172.21.15.172:0/1239102700 learned_addr learned my addr 172.21.15.172:0/1239102700 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa622ffd640 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d2cbb0 0x5610a8d8be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa61bfff640 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8de8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa6237fe640 1 -- 172.21.15.172:0/1239102700 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d2cbb0 msgr2=0x5610a8d8be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa6237fe640 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d2cbb0 0x5610a8d8be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa6237fe640 1 -- 172.21.15.172:0/1239102700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 msgr2=0x5610a8de8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa6237fe640 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8de8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa6237fe640 1 -- 172.21.15.172:0/1239102700 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a8d2c840 con 0x5610a8d744a0 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa622ffd640 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d2cbb0 0x5610a8d8be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa61bfff640 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8de8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa6237fe640 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d25a30 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00a930 tx=0x7fa61c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f17f1ec1db909f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa601ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa601ffb640 1 -- 172.21.15.172:0/1239102700 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c087070 con 0x5610a8d744a0 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa601ffb640 1 -- 172.21.15.172:0/1239102700 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fa61c004d60 con 0x5610a8d744a0 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa62c18d740 1 -- 172.21.15.172:0/1239102700 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61c055c70 con 0x5610a8d744a0 2024-04-14T08:40:47.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa601ffb640 1 -- 172.21.15.172:0/1239102700 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fa61c08b030 con 0x5610a8d744a0 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa62c18d740 10 client.8178 fetch_fsmap learned FSMap version 67 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa62c18d740 1 -- 172.21.15.172:0/1239102700 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5610a8debee0 con 0x5610a8d744a0 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.879+0000 7fa601ffb640 1 -- 172.21.15.172:0/1239102700 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c055c70 con 0x5610a8d744a0 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa601ffb640 1 -- 172.21.15.172:0/1239102700 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fa61c058a50 con 0x5610a8d744a0 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 10 client.8178 fetch_fsmap finished waiting for FSMap version 67 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 10 client.8178 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 20 client.8178 populate_metadata read hostname 'smithi172' 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x5610a8d72f10 0x5610a8d732f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 4 client.8178 mds_command: new command op to 8013 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 1 -- 172.21.15.172:0/1239102700 --> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5610a8d6e2a0 con 0x5610a8d72f10 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa622ffd640 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x5610a8d72f10 0x5610a8d732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa601ffb640 10 client.8178 ms_handle_connect on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa622ffd640 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x5610a8d72f10 0x5610a8d732f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eadfeb9e7a4ffa60 server_cookie=6b6c5f37020e6155 in_seq=0 out_seq=0 2024-04-14T08:40:47.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa601ffb640 1 -- 172.21.15.172:0/1239102700 <== mds.0 v2:172.21.15.172:6835/3091771014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610a8d6e2a0 con 0x5610a8d72f10 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa601ffb640 10 client.8178 handle_command_reply: tid=0 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 1 client.8178 shutdown 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 1 -- 172.21.15.172:0/1239102700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 msgr2=0x5610a8d25a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d25a30 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00a930 tx=0x7fa61c001250 comp rx=0 tx=0).stop 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 1 -- 172.21.15.172:0/1239102700 shutdown_connections 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.172:6835/3091771014,v1:172.21.15.172:6837/3091771014] conn(0x5610a8d72f10 0x5610a8d732f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610a8d744a0 0x5610a8d25a30 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5610a8d566d0 0x5610a8de8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 1 --2- 172.21.15.172:0/1239102700 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5610a8d2cbb0 0x5610a8d8be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 1 -- 172.21.15.172:0/1239102700 >> 172.21.15.172:0/1239102700 conn(0x5610a8d26d90 msgr2=0x5610a8d55e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa601ffb640 0 client.8178 ms_handle_reset on v2:172.21.15.172:6835/3091771014 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.880+0000 7fa62c18d740 1 -- 172.21.15.172:0/1239102700 shutdown_connections 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa62c18d740 1 -- 172.21.15.172:0/1239102700 wait complete. 2024-04-14T08:40:47.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa62c18d740 20 client.8178 trim_cache size 0 max 16384 2024-04-14T08:40:47.888 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:47.888 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-04-14T08:40:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa6293aa640 1 -- 172.21.15.172:0/1747448171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa60406c8f0 msgr2=0x7fa60406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa6293aa640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa60406c8f0 0x7fa60406edb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa614009aa0 tx=0x7fa614009190 comp rx=0 tx=0).stop 2024-04-14T08:40:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa6293aa640 1 -- 172.21.15.172:0/1747448171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa62415e590 msgr2=0x7fa62414f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa6293aa640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa62415e590 0x7fa62414f1a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00cbc0 tx=0x7fa61c00a430 comp rx=0 tx=0).stop 2024-04-14T08:40:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa6293aa640 1 -- 172.21.15.172:0/1747448171 shutdown_connections 2024-04-14T08:40:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa6293aa640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa60406c8f0 0x7fa60406edb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa6293aa640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa62415e590 0x7fa62414f1a0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa6293aa640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6240cea60 0x7fa6240b7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa6293aa640 1 --2- 172.21.15.172:0/1747448171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6240b63d0 0x7fa6240ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa6293aa640 1 -- 172.21.15.172:0/1747448171 >> 172.21.15.172:0/1747448171 conn(0x7fa6240d0e90 msgr2=0x7fa62415c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:47.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa6293aa640 1 -- 172.21.15.172:0/1747448171 shutdown_connections 2024-04-14T08:40:47.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.881+0000 7fa6293aa640 1 -- 172.21.15.172:0/1747448171 wait complete. 2024-04-14T08:40:47.896 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-14T08:40:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.995+0000 7f63ebd1e640 1 Processor -- start 2024-04-14T08:40:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.995+0000 7f63ebd1e640 1 -- start start 2024-04-14T08:40:47.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63ebd1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f63e415a690 0x7f63e4071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63ebd1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f63e4072970 0x7f63e40722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:47.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63ebd1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63e40b3360 con 0x7f63e415aa70 2024-04-14T08:40:47.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63ebd1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f63e40b3640 con 0x7f63e415a690 2024-04-14T08:40:47.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63ebd1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f63e40b3920 con 0x7f63e4072970 2024-04-14T08:40:47.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63e9aba640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f63e415a690 0x7f63e4071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63e9aba640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f63e415a690 0x7f63e4071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42270/0 (socket says 172.21.15.172:42270) 2024-04-14T08:40:47.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63e9aba640 1 -- 172.21.15.172:0/3459442210 learned_addr learned my addr 172.21.15.172:0/3459442210 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:47.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63e92b9640 1 --2- 172.21.15.172:0/3459442210 >> v2:172.21.15.177:3300/0 conn(0x7f63e4072970 0x7f63e40722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:47.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63e9aba640 1 -- 172.21.15.172:0/3459442210 >> v2:172.21.15.177:3300/0 conn(0x7f63e4072970 msgr2=0x7f63e40722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63e9aba640 1 --2- 172.21.15.172:0/3459442210 >> v2:172.21.15.177:3300/0 conn(0x7f63e4072970 0x7f63e40722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:47.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63e9aba640 1 -- 172.21.15.172:0/3459442210 >> v1:172.21.15.177:6789/0 conn(0x7f63e415aa70 legacy=0x7f63e415e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:47.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63e9aba640 1 -- 172.21.15.172:0/3459442210 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e40b3c00 con 0x7f63e415a690 2024-04-14T08:40:47.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.996+0000 7f63e92b9640 1 --2- 172.21.15.172:0/3459442210 >> v2:172.21.15.177:3300/0 conn(0x7f63e4072970 0x7f63e40722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:47.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.997+0000 7f63e9aba640 1 --2- 172.21.15.172:0/3459442210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e415a690 0x7f63e4071d60 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f63e40b2900 tx=0x7f63e002e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802cfd08927eccfa server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:47.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.997+0000 7f63e8ab8640 1 -- 172.21.15.172:0/3459442210 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e003b070 con 0x7f63e415a690 2024-04-14T08:40:47.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.997+0000 7f63e8ab8640 1 -- 172.21.15.172:0/3459442210 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e002ee00 con 0x7f63e415a690 2024-04-14T08:40:47.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.997+0000 7f63e8ab8640 1 -- 172.21.15.172:0/3459442210 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0036660 con 0x7f63e415a690 2024-04-14T08:40:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.998+0000 7f63ebd1e640 1 -- 172.21.15.172:0/3459442210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e415a690 msgr2=0x7f63e4071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.998+0000 7f63ebd1e640 1 --2- 172.21.15.172:0/3459442210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e415a690 0x7f63e4071d60 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f63e40b2900 tx=0x7f63e002e050 comp rx=0 tx=0).stop 2024-04-14T08:40:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.998+0000 7f63ebd1e640 1 -- 172.21.15.172:0/3459442210 shutdown_connections 2024-04-14T08:40:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.998+0000 7f63ebd1e640 1 --2- 172.21.15.172:0/3459442210 >> v2:172.21.15.177:3300/0 conn(0x7f63e4072970 0x7f63e40722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.998+0000 7f63ebd1e640 1 --2- 172.21.15.172:0/3459442210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e415a690 0x7f63e4071d60 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.998+0000 7f63ebd1e640 1 -- 172.21.15.172:0/3459442210 >> 172.21.15.172:0/3459442210 conn(0x7f63e406ec30 msgr2=0x7f63e406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.998+0000 7f63ebd1e640 1 -- 172.21.15.172:0/3459442210 shutdown_connections 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.998+0000 7f63ebd1e640 1 -- 172.21.15.172:0/3459442210 wait complete. 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.999+0000 7f63ebd1e640 1 Processor -- start 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:47.999+0000 7f63ebd1e640 1 -- start start 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63ebd1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e4072970 0x7f63e4158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63ebd1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63e415a690 0x7f63e41f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63ebd1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63e415aa70 0x7f63e40c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63ebd1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f63e415c240 con 0x7f63e4072970 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63ebd1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f63e40b3100 con 0x7f63e415a690 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63ebd1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f63e415c3c0 con 0x7f63e415aa70 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63e9aba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e4072970 0x7f63e4158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63e9aba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e4072970 0x7f63e4158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42282/0 (socket says 172.21.15.172:42282) 2024-04-14T08:40:48.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63e9aba640 1 -- 172.21.15.172:0/313735728 learned_addr learned my addr 172.21.15.172:0/313735728 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:48.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63e92b9640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63e415a690 0x7f63e41f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63ea2bb640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63e415aa70 0x7f63e40c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63e9aba640 1 -- 172.21.15.172:0/313735728 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63e415aa70 msgr2=0x7f63e40c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63e9aba640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63e415aa70 0x7f63e40c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63e9aba640 1 -- 172.21.15.172:0/313735728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63e415a690 msgr2=0x7f63e41f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63e9aba640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63e415a690 0x7f63e41f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63e9aba640 1 -- 172.21.15.172:0/313735728 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e41f4f60 con 0x7f63e4072970 2024-04-14T08:40:48.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63e9aba640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e4072970 0x7f63e4158950 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f63e000b2c0 tx=0x7f63e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:48.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63da7fc640 1 -- 172.21.15.172:0/313735728 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e003b040 con 0x7f63e4072970 2024-04-14T08:40:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63da7fc640 1 -- 172.21.15.172:0/313735728 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e0041450 con 0x7f63e4072970 2024-04-14T08:40:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63da7fc640 1 -- 172.21.15.172:0/313735728 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0041820 con 0x7f63e4072970 2024-04-14T08:40:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63ebd1e640 1 -- 172.21.15.172:0/313735728 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e41f5240 con 0x7f63e4072970 2024-04-14T08:40:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.000+0000 7f63ebd1e640 1 -- 172.21.15.172:0/313735728 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e40c9270 con 0x7f63e4072970 2024-04-14T08:40:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.001+0000 7f63ebd1e640 1 -- 172.21.15.172:0/313735728 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e4068520 con 0x7f63e4072970 2024-04-14T08:40:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.002+0000 7f63da7fc640 1 -- 172.21.15.172:0/313735728 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63e0048020 con 0x7f63e4072970 2024-04-14T08:40:48.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.002+0000 7f63da7fc640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f63c806c920 0x7f63c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.002+0000 7f63e92b9640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f63c806c920 0x7f63c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.002+0000 7f63da7fc640 1 -- 172.21.15.172:0/313735728 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f63e00add40 con 0x7f63e4072970 2024-04-14T08:40:48.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.003+0000 7f63e92b9640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f63c806c920 0x7f63c806ede0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f63e4072dc0 tx=0x7f63d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:48.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.007+0000 7f63da7fc640 1 -- 172.21.15.172:0/313735728 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f63e004f020 con 0x7f63e4072970 2024-04-14T08:40:48.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.222+0000 7f63ebd1e640 1 -- 172.21.15.172:0/313735728 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f63e40d0790 con 0x7f63e4072970 2024-04-14T08:40:48.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.222+0000 7f63da7fc640 1 -- 172.21.15.172:0/313735728 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f63e00419c0 con 0x7f63e4072970 2024-04-14T08:40:48.223 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:48.223 INFO:teuthology.orchestra.run.smithi172.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-14T08:40:48.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.224+0000 7f63ebd1e640 1 -- 172.21.15.172:0/313735728 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f63c806c920 msgr2=0x7f63c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.224+0000 7f63ebd1e640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f63c806c920 0x7f63c806ede0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f63e4072dc0 tx=0x7f63d4008040 comp rx=0 tx=0).stop 2024-04-14T08:40:48.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.224+0000 7f63ebd1e640 1 -- 172.21.15.172:0/313735728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e4072970 msgr2=0x7f63e4158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.224+0000 7f63ebd1e640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e4072970 0x7f63e4158950 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f63e000b2c0 tx=0x7f63e0004730 comp rx=0 tx=0).stop 2024-04-14T08:40:48.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.224+0000 7f63ebd1e640 1 -- 172.21.15.172:0/313735728 shutdown_connections 2024-04-14T08:40:48.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.224+0000 7f63ebd1e640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f63c806c920 0x7f63c806ede0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.224+0000 7f63ebd1e640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63e415aa70 0x7f63e40c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.224+0000 7f63ebd1e640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63e415a690 0x7f63e41f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.224+0000 7f63ebd1e640 1 --2- 172.21.15.172:0/313735728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e4072970 0x7f63e4158950 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.224+0000 7f63ebd1e640 1 -- 172.21.15.172:0/313735728 >> 172.21.15.172:0/313735728 conn(0x7f63e406ec30 msgr2=0x7f63e40cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:48.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.224+0000 7f63ebd1e640 1 -- 172.21.15.172:0/313735728 shutdown_connections 2024-04-14T08:40:48.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.224+0000 7f63ebd1e640 1 -- 172.21.15.172:0/313735728 wait complete. 2024-04-14T08:40:48.239 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-14T08:40:48.240 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:40:48.240 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:40:48.286 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:40:48.333 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:40:48.333 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:40:48.333 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:40:48.333 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:40:48.349 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:40:48.349 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:40:48.404 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:40:48.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.545+0000 7f8fff06d640 1 Processor -- start 2024-04-14T08:40:48.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.545+0000 7f8fff06d640 1 -- start start 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.545+0000 7f8fff06d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8ff80b21d0 0x7f8ff815c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.545+0000 7f8fff06d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8ff815c5a0 0x7f8ff80b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.545+0000 7f8fff06d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff80b3700 con 0x7f8ff80b1df0 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.545+0000 7f8fff06d640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff80b39e0 con 0x7f8ff815c5a0 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.545+0000 7f8fff06d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff80b3cc0 con 0x7f8ff80b21d0 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.546+0000 7f8ffd60a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8ff815c5a0 0x7f8ff80b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.546+0000 7f8ffd60a640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8ff815c5a0 0x7f8ff80b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42296/0 (socket says 172.21.15.172:42296) 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.546+0000 7f8ffd60a640 1 -- 172.21.15.172:0/4200980267 learned_addr learned my addr 172.21.15.172:0/4200980267 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.546+0000 7f8ffd60a640 1 -- 172.21.15.172:0/4200980267 >> v2:172.21.15.177:3300/0 conn(0x7f8ff80b21d0 msgr2=0x7f8ff815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.546+0000 7f8ffd60a640 1 --2- 172.21.15.172:0/4200980267 >> v2:172.21.15.177:3300/0 conn(0x7f8ff80b21d0 0x7f8ff815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.546+0000 7f8ffd60a640 1 -- 172.21.15.172:0/4200980267 >> v1:172.21.15.177:6789/0 conn(0x7f8ff80b1df0 legacy=0x7f8ff80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.546+0000 7f8ffd60a640 1 -- 172.21.15.172:0/4200980267 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff815bb40 con 0x7f8ff815c5a0 2024-04-14T08:40:48.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.547+0000 7f8ffd60a640 1 --2- 172.21.15.172:0/4200980267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ff815c5a0 0x7f8ff80b4040 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f8ff806c0d0 tx=0x7f8ff002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a30f3107ea26a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:48.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.547+0000 7f8ff77fe640 1 -- 172.21.15.172:0/4200980267 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff002ca80 con 0x7f8ff815c5a0 2024-04-14T08:40:48.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.547+0000 7f8ff77fe640 1 -- 172.21.15.172:0/4200980267 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff002cc20 con 0x7f8ff815c5a0 2024-04-14T08:40:48.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.547+0000 7f8ff77fe640 1 -- 172.21.15.172:0/4200980267 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00343f0 con 0x7f8ff815c5a0 2024-04-14T08:40:48.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.547+0000 7f8fff06d640 1 -- 172.21.15.172:0/4200980267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ff815c5a0 msgr2=0x7f8ff80b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.547+0000 7f8fff06d640 1 --2- 172.21.15.172:0/4200980267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ff815c5a0 0x7f8ff80b4040 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f8ff806c0d0 tx=0x7f8ff002bee0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.547+0000 7f8fff06d640 1 -- 172.21.15.172:0/4200980267 shutdown_connections 2024-04-14T08:40:48.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.547+0000 7f8fff06d640 1 --2- 172.21.15.172:0/4200980267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ff815c5a0 0x7f8ff80b4040 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.547+0000 7f8fff06d640 1 --2- 172.21.15.172:0/4200980267 >> v2:172.21.15.177:3300/0 conn(0x7f8ff80b21d0 0x7f8ff815c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.547+0000 7f8fff06d640 1 -- 172.21.15.172:0/4200980267 >> 172.21.15.172:0/4200980267 conn(0x7f8ff806ec30 msgr2=0x7f8ff806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:48.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.547+0000 7f8fff06d640 1 -- 172.21.15.172:0/4200980267 shutdown_connections 2024-04-14T08:40:48.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.547+0000 7f8fff06d640 1 -- 172.21.15.172:0/4200980267 wait complete. 2024-04-14T08:40:48.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.548+0000 7f8fff06d640 1 Processor -- start 2024-04-14T08:40:48.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.549+0000 7f8fff06d640 1 -- start start 2024-04-14T08:40:48.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.549+0000 7f8fff06d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff80b1df0 0x7f8ff8157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.549+0000 7f8fff06d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ff80b21d0 0x7f8ff80cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.549+0000 7f8fff06d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff80d03c0 0x7f8ff8133340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.549+0000 7f8fff06d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff80b4700 con 0x7f8ff80b21d0 2024-04-14T08:40:48.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.549+0000 7f8fff06d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff80b4580 con 0x7f8ff80d03c0 2024-04-14T08:40:48.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.549+0000 7f8fff06d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff80b4880 con 0x7f8ff80b1df0 2024-04-14T08:40:48.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ff7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ff80b21d0 0x7f8ff80cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ff7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ff80b21d0 0x7f8ff80cfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42308/0 (socket says 172.21.15.172:42308) 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ff7fff640 1 -- 172.21.15.172:0/1819846119 learned_addr learned my addr 172.21.15.172:0/1819846119 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ffd60a640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff80d03c0 0x7f8ff8133340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ffce09640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff80b1df0 0x7f8ff8157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ff7fff640 1 -- 172.21.15.172:0/1819846119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff80b1df0 msgr2=0x7f8ff8157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ff7fff640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff80b1df0 0x7f8ff8157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ff7fff640 1 -- 172.21.15.172:0/1819846119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff80d03c0 msgr2=0x7f8ff8133340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ff7fff640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff80d03c0 0x7f8ff8133340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ff7fff640 1 -- 172.21.15.172:0/1819846119 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff81351b0 con 0x7f8ff80b21d0 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ffd60a640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff80d03c0 0x7f8ff8133340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ff7fff640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ff80b21d0 0x7f8ff80cfe80 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400b410 tx=0x7f8fe400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ff57fa640 1 -- 172.21.15.172:0/1819846119 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4015020 con 0x7f8ff80b21d0 2024-04-14T08:40:48.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ff57fa640 1 -- 172.21.15.172:0/1819846119 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe4011830 con 0x7f8ff80b21d0 2024-04-14T08:40:48.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8fff06d640 1 -- 172.21.15.172:0/1819846119 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff8134110 con 0x7f8ff80b21d0 2024-04-14T08:40:48.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8ff57fa640 1 -- 172.21.15.172:0/1819846119 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4011c20 con 0x7f8ff80b21d0 2024-04-14T08:40:48.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.550+0000 7f8fff06d640 1 -- 172.21.15.172:0/1819846119 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff81346d0 con 0x7f8ff80b21d0 2024-04-14T08:40:48.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.551+0000 7f8fff06d640 1 -- 172.21.15.172:0/1819846119 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff80cc4b0 con 0x7f8ff80b21d0 2024-04-14T08:40:48.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.552+0000 7f8ff57fa640 1 -- 172.21.15.172:0/1819846119 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fe4018020 con 0x7f8ff80b21d0 2024-04-14T08:40:48.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.552+0000 7f8ff57fa640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8fdc06c8f0 0x7f8fdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.552+0000 7f8ff57fa640 1 -- 172.21.15.172:0/1819846119 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f8fe408bd10 con 0x7f8ff80b21d0 2024-04-14T08:40:48.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.552+0000 7f8ffce09640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8fdc06c8f0 0x7f8fdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.553+0000 7f8ffce09640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8fdc06c8f0 0x7f8fdc06edb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00a750 tx=0x7f8fec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:48.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.557+0000 7f8ff57fa640 1 -- 172.21.15.172:0/1819846119 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8ff80cc4b0 con 0x7f8ff80b21d0 2024-04-14T08:40:48.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.740+0000 7f8fff06d640 1 -- 172.21.15.172:0/1819846119 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ff8068520 con 0x7f8ff80b21d0 2024-04-14T08:40:48.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.741+0000 7f8ff57fa640 1 -- 172.21.15.172:0/1819846119 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+17401 (secure 0 0 0) 0x7f8fe401f020 con 0x7f8ff80b21d0 2024-04-14T08:40:48.742 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:48.742 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":90,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:40:10.107688+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:38:42.790996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-14T08:38:43.517208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/1386685680":"2024-04-15T08:40:10.107666+0000","172.21.15.172:6835/1386685680":"2024-04-15T08:40:10.107666+0000","172.21.15.177:6835/1124731311":"2024-04-15T08:38:58.388334+0000","172.21.15.177:6833/1124731311":"2024-04-15T08:38:58.388334+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:40:48.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.743+0000 7f8fff06d640 1 -- 172.21.15.172:0/1819846119 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8fdc06c8f0 msgr2=0x7f8fdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.743+0000 7f8fff06d640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8fdc06c8f0 0x7f8fdc06edb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00a750 tx=0x7f8fec008040 comp rx=0 tx=0).stop 2024-04-14T08:40:48.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.743+0000 7f8fff06d640 1 -- 172.21.15.172:0/1819846119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ff80b21d0 msgr2=0x7f8ff80cfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.743+0000 7f8fff06d640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ff80b21d0 0x7f8ff80cfe80 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400b410 tx=0x7f8fe400b8d0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.743+0000 7f8fff06d640 1 -- 172.21.15.172:0/1819846119 shutdown_connections 2024-04-14T08:40:48.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.743+0000 7f8fff06d640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8fdc06c8f0 0x7f8fdc06edb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.743+0000 7f8fff06d640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff80d03c0 0x7f8ff8133340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.743+0000 7f8fff06d640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ff80b21d0 0x7f8ff80cfe80 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.743+0000 7f8fff06d640 1 --2- 172.21.15.172:0/1819846119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff80b1df0 0x7f8ff8157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.743+0000 7f8fff06d640 1 -- 172.21.15.172:0/1819846119 >> 172.21.15.172:0/1819846119 conn(0x7f8ff806ec30 msgr2=0x7f8ff80b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:48.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.743+0000 7f8fff06d640 1 -- 172.21.15.172:0/1819846119 shutdown_connections 2024-04-14T08:40:48.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.743+0000 7f8fff06d640 1 -- 172.21.15.172:0/1819846119 wait complete. 2024-04-14T08:40:48.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:38:42.790996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:38:43.517208+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:40:48.756 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:40:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.862+0000 7f60f091e640 1 Processor -- start 2024-04-14T08:40:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.862+0000 7f60f091e640 1 -- start start 2024-04-14T08:40:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60f091e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f60ec06fb70 0x7f60ec073090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60f091e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60ec0ceee0 con 0x7f60ec06bc30 2024-04-14T08:40:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60f091e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60ec070150 con 0x7f60ec0ccca0 2024-04-14T08:40:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60f091e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f60ec0722d0 con 0x7f60ec06fb70 2024-04-14T08:40:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60e3fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f60ec06bc30 0x7f60ec06e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:41628/0 (socket says 172.21.15.172:41628) 2024-04-14T08:40:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60e3fff640 1 -- 172.21.15.172:0/1520745912 learned_addr learned my addr 172.21.15.172:0/1520745912 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60ead9d640 1 --2- 172.21.15.172:0/1520745912 >> v2:172.21.15.177:3300/0 conn(0x7f60ec06fb70 0x7f60ec073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2808164226 0 0) 0x7f60ec0ceee0 con 0x7f60ec06bc30 2024-04-14T08:40:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60d8003520 con 0x7f60ec06bc30 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1814933750 0 0) 0x7f60d8003520 con 0x7f60ec06bc30 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 >> v2:172.21.15.177:3300/0 conn(0x7f60ec06fb70 msgr2=0x7f60ec073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60e9d9b640 1 --2- 172.21.15.172:0/1520745912 >> v2:172.21.15.177:3300/0 conn(0x7f60ec06fb70 0x7f60ec073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 >> v1:172.21.15.177:6789/0 conn(0x7f60ec0ccca0 legacy=0x7f60ec06b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec0725b0 con 0x7f60ec06bc30 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f60d4001670 con 0x7f60ec06bc30 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 >> v1:172.21.15.172:6789/0 conn(0x7f60ec06bc30 legacy=0x7f60ec06e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.863+0000 7f60e9d9b640 1 --2- 172.21.15.172:0/1520745912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 0x7f60d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60e9d9b640 1 --2- 172.21.15.172:0/1520745912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60d8004800 0x7f60d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60e9d9b640 1 --2- 172.21.15.172:0/1520745912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60d80051c0 0x7f60d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec0722d0 con 0x7f60d80039b0 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f60d800c160 con 0x7f60d80051c0 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f60d800c360 con 0x7f60d8004800 2024-04-14T08:40:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60ea59c640 1 --2- 172.21.15.172:0/1520745912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 0x7f60d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60ead9d640 1 --2- 172.21.15.172:0/1520745912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60d80051c0 0x7f60d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60e3fff640 1 --2- 172.21.15.172:0/1520745912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60d8004800 0x7f60d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60ea59c640 1 -- 172.21.15.172:0/1520745912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60d8004800 msgr2=0x7f60d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60ea59c640 1 --2- 172.21.15.172:0/1520745912 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60d8004800 0x7f60d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60ea59c640 1 -- 172.21.15.172:0/1520745912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60d80051c0 msgr2=0x7f60d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60ea59c640 1 --2- 172.21.15.172:0/1520745912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60d80051c0 0x7f60d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60ead9d640 1 -- 172.21.15.172:0/1520745912 reap_dead start 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.864+0000 7f60ea59c640 1 -- 172.21.15.172:0/1520745912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d8003520 con 0x7f60d80039b0 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.865+0000 7f60ea59c640 1 --2- 172.21.15.172:0/1520745912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 0x7f60d8003dd0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f60dc001750 tx=0x7f60d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ae83c119e0318f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.865+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc001370 con 0x7f60d80039b0 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.865+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60dc0052d0 con 0x7f60d80039b0 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.865+0000 7f60e9d9b640 1 -- 172.21.15.172:0/1520745912 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc003830 con 0x7f60d80039b0 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.865+0000 7f60f091e640 1 -- 172.21.15.172:0/1520745912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 msgr2=0x7f60d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.865+0000 7f60f091e640 1 --2- 172.21.15.172:0/1520745912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 0x7f60d8003dd0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f60dc001750 tx=0x7f60d8005830 comp rx=0 tx=0).stop 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.865+0000 7f60f091e640 1 -- 172.21.15.172:0/1520745912 shutdown_connections 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.865+0000 7f60f091e640 1 --2- 172.21.15.172:0/1520745912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 0x7f60d8003dd0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.865+0000 7f60f091e640 1 -- 172.21.15.172:0/1520745912 >> 172.21.15.172:0/1520745912 conn(0x7f60ec0cda00 msgr2=0x7f60ec0cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.865+0000 7f60f091e640 1 -- 172.21.15.172:0/1520745912 shutdown_connections 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.865+0000 7f60f091e640 1 -- 172.21.15.172:0/1520745912 wait complete. 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.867+0000 7f60f091e640 1 Processor -- start 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.867+0000 7f60f091e640 1 -- start start 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.867+0000 7f60f091e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 0x7f60ec0c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.867+0000 7f60f091e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60ec1523a0 0x7f60ec1f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.867+0000 7f60f091e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60ec0b4260 0x7f60ec0b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.867+0000 7f60f091e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec073090 con 0x7f60d80039b0 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.867+0000 7f60f091e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec0cd080 con 0x7f60ec0b4260 2024-04-14T08:40:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.867+0000 7f60f091e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f60ec073210 con 0x7f60ec1523a0 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.867+0000 7f60ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 0x7f60ec0c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.867+0000 7f60ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 0x7f60ec0c4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42322/0 (socket says 172.21.15.172:42322) 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.867+0000 7f60ea59c640 1 -- 172.21.15.172:0/2700431549 learned_addr learned my addr 172.21.15.172:0/2700431549 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60e3fff640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60ec1523a0 0x7f60ec1f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60ead9d640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60ec0b4260 0x7f60ec0b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60ea59c640 1 -- 172.21.15.172:0/2700431549 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60ec1523a0 msgr2=0x7f60ec1f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60ea59c640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60ec1523a0 0x7f60ec1f3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60ea59c640 1 -- 172.21.15.172:0/2700431549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60ec0b4260 msgr2=0x7f60ec0b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60ea59c640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60ec0b4260 0x7f60ec0b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60ea59c640 1 -- 172.21.15.172:0/2700431549 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec0b3aa0 con 0x7f60d80039b0 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60ea59c640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 0x7f60ec0c4a90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f60dc005040 tx=0x7f60dc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60e2ffd640 1 -- 172.21.15.172:0/2700431549 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc041070 con 0x7f60d80039b0 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60e2ffd640 1 -- 172.21.15.172:0/2700431549 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60dc005480 con 0x7f60d80039b0 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60e2ffd640 1 -- 172.21.15.172:0/2700431549 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc007a30 con 0x7f60d80039b0 2024-04-14T08:40:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60f091e640 1 -- 172.21.15.172:0/2700431549 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ec1f4730 con 0x7f60d80039b0 2024-04-14T08:40:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.868+0000 7f60f091e640 1 -- 172.21.15.172:0/2700431549 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ec1f4cf0 con 0x7f60d80039b0 2024-04-14T08:40:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.869+0000 7f60f091e640 1 -- 172.21.15.172:0/2700431549 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ec068590 con 0x7f60d80039b0 2024-04-14T08:40:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.870+0000 7f60e2ffd640 1 -- 172.21.15.172:0/2700431549 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60dc044070 con 0x7f60d80039b0 2024-04-14T08:40:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.870+0000 7f60e2ffd640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f60cc06c920 0x7f60cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.871+0000 7f60e3fff640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f60cc06c920 0x7f60cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.871+0000 7f60e2ffd640 1 -- 172.21.15.172:0/2700431549 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f60dc0ad8a0 con 0x7f60d80039b0 2024-04-14T08:40:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.871+0000 7f60e3fff640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f60cc06c920 0x7f60cc06ede0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0b2510 tx=0x7f60d4007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:48.879+0000 7f60e2ffd640 1 -- 172.21.15.172:0/2700431549 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f60dc07b4f0 con 0x7f60d80039b0 2024-04-14T08:40:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.058+0000 7f60f091e640 1 -- 172.21.15.172:0/2700431549 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f60ec06f250 con 0x7f60d80039b0 2024-04-14T08:40:49.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.059+0000 7f60e2ffd640 1 -- 172.21.15.172:0/2700431549 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f60dc07f7f0 con 0x7f60d80039b0 2024-04-14T08:40:49.059 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:40:49.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.060+0000 7f60f091e640 1 -- 172.21.15.172:0/2700431549 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f60cc06c920 msgr2=0x7f60cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.060+0000 7f60f091e640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f60cc06c920 0x7f60cc06ede0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0b2510 tx=0x7f60d4007040 comp rx=0 tx=0).stop 2024-04-14T08:40:49.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.060+0000 7f60f091e640 1 -- 172.21.15.172:0/2700431549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 msgr2=0x7f60ec0c4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.060+0000 7f60f091e640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 0x7f60ec0c4a90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f60dc005040 tx=0x7f60dc001120 comp rx=0 tx=0).stop 2024-04-14T08:40:49.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.061+0000 7f60f091e640 1 -- 172.21.15.172:0/2700431549 shutdown_connections 2024-04-14T08:40:49.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.061+0000 7f60f091e640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f60cc06c920 0x7f60cc06ede0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.061+0000 7f60f091e640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60ec0b4260 0x7f60ec0b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.061+0000 7f60f091e640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60ec1523a0 0x7f60ec1f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.061+0000 7f60f091e640 1 --2- 172.21.15.172:0/2700431549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60d80039b0 0x7f60ec0c4a90 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.061+0000 7f60f091e640 1 -- 172.21.15.172:0/2700431549 >> 172.21.15.172:0/2700431549 conn(0x7f60ec0cda00 msgr2=0x7f60ec0addb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:49.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.061+0000 7f60f091e640 1 -- 172.21.15.172:0/2700431549 shutdown_connections 2024-04-14T08:40:49.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.061+0000 7f60f091e640 1 -- 172.21.15.172:0/2700431549 wait complete. 2024-04-14T08:40:49.073 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:40:49.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.178+0000 7fd527436640 1 Processor -- start 2024-04-14T08:40:49.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.178+0000 7fd527436640 1 -- start start 2024-04-14T08:40:49.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd527436640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd520159c20 0x7fd5200ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd527436640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd52015c700 0x7fd5201603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd527436640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd52015cf10 con 0x7fd520160930 2024-04-14T08:40:49.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd527436640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd52015fc80 con 0x7fd520159c20 2024-04-14T08:40:49.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd527436640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd52015ff60 con 0x7fd52015c700 2024-04-14T08:40:49.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd5251d2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd520159c20 0x7fd5200ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd5251d2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd520159c20 0x7fd5200ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42328/0 (socket says 172.21.15.172:42328) 2024-04-14T08:40:49.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd5259d3640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd520160930 0x7fd520072450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:41634/0 (socket says 172.21.15.172:41634) 2024-04-14T08:40:49.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd5251d2640 1 -- 172.21.15.172:0/948835825 learned_addr learned my addr 172.21.15.172:0/948835825 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:49.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd5251d2640 1 -- 172.21.15.172:0/948835825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd52015c700 msgr2=0x7fd5201603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:49.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd5249d1640 1 --2- 172.21.15.172:0/948835825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd52015c700 0x7fd5201603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd517fff640 1 -- 172.21.15.172:0/948835825 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 555614132 0 0) 0x7fd52015cf10 con 0x7fd520160930 2024-04-14T08:40:49.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd5251d2640 1 --2- 172.21.15.172:0/948835825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd52015c700 0x7fd5201603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd5251d2640 1 -- 172.21.15.172:0/948835825 >> v1:172.21.15.172:6789/0 conn(0x7fd520160930 legacy=0x7fd520072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd5251d2640 1 -- 172.21.15.172:0/948835825 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52015d1f0 con 0x7fd520159c20 2024-04-14T08:40:49.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.179+0000 7fd5249d1640 1 --2- 172.21.15.172:0/948835825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd52015c700 0x7fd5201603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:49.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.180+0000 7fd5251d2640 1 --2- 172.21.15.172:0/948835825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520159c20 0x7fd5200ce200 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009870 tx=0x7fd51c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6dc9ef9a12a0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:49.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.180+0000 7fd517fff640 1 -- 172.21.15.172:0/948835825 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c034070 con 0x7fd520159c20 2024-04-14T08:40:49.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.180+0000 7fd517fff640 1 -- 172.21.15.172:0/948835825 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51c030840 con 0x7fd520159c20 2024-04-14T08:40:49.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.180+0000 7fd517fff640 1 -- 172.21.15.172:0/948835825 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c030c10 con 0x7fd520159c20 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.181+0000 7fd527436640 1 -- 172.21.15.172:0/948835825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520159c20 msgr2=0x7fd5200ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.181+0000 7fd527436640 1 --2- 172.21.15.172:0/948835825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520159c20 0x7fd5200ce200 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009870 tx=0x7fd51c02b980 comp rx=0 tx=0).stop 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.181+0000 7fd527436640 1 -- 172.21.15.172:0/948835825 shutdown_connections 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.181+0000 7fd527436640 1 --2- 172.21.15.172:0/948835825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd52015c700 0x7fd5201603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.181+0000 7fd527436640 1 --2- 172.21.15.172:0/948835825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd520159c20 0x7fd5200ce200 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.181+0000 7fd527436640 1 -- 172.21.15.172:0/948835825 >> 172.21.15.172:0/948835825 conn(0x7fd52006b520 msgr2=0x7fd52006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.181+0000 7fd527436640 1 -- 172.21.15.172:0/948835825 shutdown_connections 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.181+0000 7fd527436640 1 -- 172.21.15.172:0/948835825 wait complete. 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.182+0000 7fd527436640 1 Processor -- start 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.182+0000 7fd527436640 1 -- start start 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd527436640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd520159c20 0x7fd5200aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd527436640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd52015c700 0x7fd5200cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd527436640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5201f62d0 0x7fd5200c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd527436640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd52015cb80 con 0x7fd5201f62d0 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd527436640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd52015a000 con 0x7fd520159c20 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd527436640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd520072b60 con 0x7fd52015c700 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd5259d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5201f62d0 0x7fd5200c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd5259d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5201f62d0 0x7fd5200c4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42336/0 (socket says 172.21.15.172:42336) 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd5259d3640 1 -- 172.21.15.172:0/3831551341 learned_addr learned my addr 172.21.15.172:0/3831551341 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd5249d1640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd52015c700 0x7fd5200cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd5251d2640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd520159c20 0x7fd5200aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd5259d3640 1 -- 172.21.15.172:0/3831551341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd52015c700 msgr2=0x7fd5200cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd5259d3640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd52015c700 0x7fd5200cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd5259d3640 1 -- 172.21.15.172:0/3831551341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd520159c20 msgr2=0x7fd5200aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd5259d3640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd520159c20 0x7fd5200aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd5259d3640 1 -- 172.21.15.172:0/3831551341 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5200aae90 con 0x7fd5201f62d0 2024-04-14T08:40:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd5259d3640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5201f62d0 0x7fd5200c4fb0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c6d0 tx=0x7fd51800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd515ffb640 1 -- 172.21.15.172:0/3831551341 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518018070 con 0x7fd5201f62d0 2024-04-14T08:40:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd515ffb640 1 -- 172.21.15.172:0/3831551341 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd518004d60 con 0x7fd5201f62d0 2024-04-14T08:40:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd527436640 1 -- 172.21.15.172:0/3831551341 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5200ab170 con 0x7fd5201f62d0 2024-04-14T08:40:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.183+0000 7fd515ffb640 1 -- 172.21.15.172:0/3831551341 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51800a880 con 0x7fd5201f62d0 2024-04-14T08:40:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.184+0000 7fd527436640 1 -- 172.21.15.172:0/3831551341 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5200aba10 con 0x7fd5201f62d0 2024-04-14T08:40:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.185+0000 7fd527436640 1 -- 172.21.15.172:0/3831551341 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd520068590 con 0x7fd5201f62d0 2024-04-14T08:40:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.185+0000 7fd515ffb640 1 -- 172.21.15.172:0/3831551341 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd51801b070 con 0x7fd5201f62d0 2024-04-14T08:40:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.185+0000 7fd515ffb640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd50406c8f0 0x7fd50406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.186+0000 7fd5251d2640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd50406c8f0 0x7fd50406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.186+0000 7fd515ffb640 1 -- 172.21.15.172:0/3831551341 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fd51808b590 con 0x7fd5201f62d0 2024-04-14T08:40:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.186+0000 7fd5251d2640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd50406c8f0 0x7fd50406edb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd51c036000 tx=0x7fd51c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.190+0000 7fd515ffb640 1 -- 172.21.15.172:0/3831551341 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd518059710 con 0x7fd5201f62d0 2024-04-14T08:40:49.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.370+0000 7fd527436640 1 -- 172.21.15.172:0/3831551341 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd520071e50 con 0x7fd5201f62d0 2024-04-14T08:40:49.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.371+0000 7fd515ffb640 1 -- 172.21.15.172:0/3831551341 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fd518059010 con 0x7fd5201f62d0 2024-04-14T08:40:49.372 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:40:49.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.373+0000 7fd527436640 1 -- 172.21.15.172:0/3831551341 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd50406c8f0 msgr2=0x7fd50406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.373+0000 7fd527436640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd50406c8f0 0x7fd50406edb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd51c036000 tx=0x7fd51c008030 comp rx=0 tx=0).stop 2024-04-14T08:40:49.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.373+0000 7fd527436640 1 -- 172.21.15.172:0/3831551341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5201f62d0 msgr2=0x7fd5200c4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.373+0000 7fd527436640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5201f62d0 0x7fd5200c4fb0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c6d0 tx=0x7fd51800cb90 comp rx=0 tx=0).stop 2024-04-14T08:40:49.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.373+0000 7fd527436640 1 -- 172.21.15.172:0/3831551341 shutdown_connections 2024-04-14T08:40:49.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.373+0000 7fd527436640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd50406c8f0 0x7fd50406edb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.373+0000 7fd527436640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5201f62d0 0x7fd5200c4fb0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.373+0000 7fd527436640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd52015c700 0x7fd5200cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.373+0000 7fd527436640 1 --2- 172.21.15.172:0/3831551341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd520159c20 0x7fd5200aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.373+0000 7fd527436640 1 -- 172.21.15.172:0/3831551341 >> 172.21.15.172:0/3831551341 conn(0x7fd52006b520 msgr2=0x7fd5200cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:49.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.373+0000 7fd527436640 1 -- 172.21.15.172:0/3831551341 shutdown_connections 2024-04-14T08:40:49.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.373+0000 7fd527436640 1 -- 172.21.15.172:0/3831551341 wait complete. 2024-04-14T08:40:49.385 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:40:49.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.490+0000 7f50b6e88640 1 Processor -- start 2024-04-14T08:40:49.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.491+0000 7f50b6e88640 1 -- start start 2024-04-14T08:40:49.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.491+0000 7f50b6e88640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f50b0071250 0x7f50b006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.491+0000 7f50b6e88640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50b006d000 0x7f50b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.491+0000 7f50b6e88640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50b00701b0 con 0x7f50b00d1a20 2024-04-14T08:40:49.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.491+0000 7f50b6e88640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f50b0070dc0 con 0x7f50b006d000 2024-04-14T08:40:49.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.491+0000 7f50b6e88640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f50b006d990 con 0x7f50b0071250 2024-04-14T08:40:49.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.491+0000 7f50b5425640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50b006d000 0x7f50b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.491+0000 7f50b4c24640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f50b00d1a20 0x7f50b00cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:41638/0 (socket says 172.21.15.172:41638) 2024-04-14T08:40:49.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.492+0000 7f50b4c24640 1 -- 172.21.15.172:0/3831468393 learned_addr learned my addr 172.21.15.172:0/3831468393 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:49.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.492+0000 7f50affff640 1 --2- 172.21.15.172:0/3831468393 >> v2:172.21.15.177:3300/0 conn(0x7f50b0071250 0x7f50b006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.492+0000 7f50b5425640 1 -- 172.21.15.172:0/3831468393 >> v2:172.21.15.177:3300/0 conn(0x7f50b0071250 msgr2=0x7f50b006fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:49.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.492+0000 7f50b5425640 1 --2- 172.21.15.172:0/3831468393 >> v2:172.21.15.177:3300/0 conn(0x7f50b0071250 0x7f50b006fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.492+0000 7f50b5425640 1 -- 172.21.15.172:0/3831468393 >> v1:172.21.15.172:6789/0 conn(0x7f50b00d1a20 legacy=0x7f50b00cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.492+0000 7f50b5425640 1 -- 172.21.15.172:0/3831468393 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b0070490 con 0x7f50b006d000 2024-04-14T08:40:49.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.492+0000 7f50af7fe640 1 -- 172.21.15.172:0/3831468393 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2622893724 0 0) 0x7f50b00701b0 con 0x7f50b00d1a20 2024-04-14T08:40:49.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.493+0000 7f50b5425640 1 --2- 172.21.15.172:0/3831468393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b006d000 0x7f50b006d450 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f50b00c5bf0 tx=0x7f50a802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5380cb710b91f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:49.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.493+0000 7f50af7fe640 1 -- 172.21.15.172:0/3831468393 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8039070 con 0x7f50b006d000 2024-04-14T08:40:49.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.493+0000 7f50af7fe640 1 -- 172.21.15.172:0/3831468393 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a802cbf0 con 0x7f50b006d000 2024-04-14T08:40:49.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.493+0000 7f50af7fe640 1 -- 172.21.15.172:0/3831468393 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8034440 con 0x7f50b006d000 2024-04-14T08:40:49.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.493+0000 7f50b6e88640 1 -- 172.21.15.172:0/3831468393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b006d000 msgr2=0x7f50b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.493+0000 7f50b6e88640 1 --2- 172.21.15.172:0/3831468393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b006d000 0x7f50b006d450 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f50b00c5bf0 tx=0x7f50a802be40 comp rx=0 tx=0).stop 2024-04-14T08:40:49.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.493+0000 7f50b6e88640 1 -- 172.21.15.172:0/3831468393 shutdown_connections 2024-04-14T08:40:49.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.493+0000 7f50b6e88640 1 --2- 172.21.15.172:0/3831468393 >> v2:172.21.15.177:3300/0 conn(0x7f50b0071250 0x7f50b006fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.493+0000 7f50b6e88640 1 --2- 172.21.15.172:0/3831468393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b006d000 0x7f50b006d450 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.493+0000 7f50b6e88640 1 -- 172.21.15.172:0/3831468393 >> 172.21.15.172:0/3831468393 conn(0x7f50b008c000 msgr2=0x7f50b00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:49.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.493+0000 7f50b6e88640 1 -- 172.21.15.172:0/3831468393 shutdown_connections 2024-04-14T08:40:49.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.493+0000 7f50b6e88640 1 -- 172.21.15.172:0/3831468393 wait complete. 2024-04-14T08:40:49.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.495+0000 7f50b6e88640 1 Processor -- start 2024-04-14T08:40:49.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.495+0000 7f50b6e88640 1 -- start start 2024-04-14T08:40:49.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b6e88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50b006d000 0x7f50b00c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b6e88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50b0071250 0x7f50b014b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b6e88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b00d1a20 0x7f50b00ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b6e88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f50b00d1e00 con 0x7f50b00d1a20 2024-04-14T08:40:49.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b6e88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50b0070a50 con 0x7f50b006d000 2024-04-14T08:40:49.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b6e88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f50b00cada0 con 0x7f50b0071250 2024-04-14T08:40:49.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b5425640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b00d1a20 0x7f50b00ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b5425640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b00d1a20 0x7f50b00ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42362/0 (socket says 172.21.15.172:42362) 2024-04-14T08:40:49.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b5425640 1 -- 172.21.15.172:0/657205651 learned_addr learned my addr 172.21.15.172:0/657205651 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:49.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50affff640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50b0071250 0x7f50b014b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b4c24640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50b006d000 0x7f50b00c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b5425640 1 -- 172.21.15.172:0/657205651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50b0071250 msgr2=0x7f50b014b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b5425640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50b0071250 0x7f50b014b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b5425640 1 -- 172.21.15.172:0/657205651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50b006d000 msgr2=0x7f50b00c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b5425640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50b006d000 0x7f50b00c91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b5425640 1 -- 172.21.15.172:0/657205651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b00c97c0 con 0x7f50b00d1a20 2024-04-14T08:40:49.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b5425640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b00d1a20 0x7f50b00ca860 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f50a8037040 tx=0x7f50a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:49.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50ad7fa640 1 -- 172.21.15.172:0/657205651 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8039050 con 0x7f50b00d1a20 2024-04-14T08:40:49.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50ad7fa640 1 -- 172.21.15.172:0/657205651 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a8045070 con 0x7f50b00d1a20 2024-04-14T08:40:49.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50ad7fa640 1 -- 172.21.15.172:0/657205651 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a803e830 con 0x7f50b00d1a20 2024-04-14T08:40:49.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.497+0000 7f50b6e88640 1 -- 172.21.15.172:0/657205651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b00c9aa0 con 0x7f50b00d1a20 2024-04-14T08:40:49.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.498+0000 7f50b6e88640 1 -- 172.21.15.172:0/657205651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b00ca060 con 0x7f50b00d1a20 2024-04-14T08:40:49.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.498+0000 7f50b6e88640 1 -- 172.21.15.172:0/657205651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b0075490 con 0x7f50b00d1a20 2024-04-14T08:40:49.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.499+0000 7f50ad7fa640 1 -- 172.21.15.172:0/657205651 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50a803e9d0 con 0x7f50b00d1a20 2024-04-14T08:40:49.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.499+0000 7f50ad7fa640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f509406c8f0 0x7f509406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.499+0000 7f50ad7fa640 1 -- 172.21.15.172:0/657205651 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f50a80aeb30 con 0x7f50b00d1a20 2024-04-14T08:40:49.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.499+0000 7f50b4c24640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f509406c8f0 0x7f509406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.500+0000 7f50b4c24640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f509406c8f0 0x7f509406edb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f50a4000f60 tx=0x7f50a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:49.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.504+0000 7f50ad7fa640 1 -- 172.21.15.172:0/657205651 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f50a807ccb0 con 0x7f50b00d1a20 2024-04-14T08:40:49.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.683+0000 7f50b6e88640 1 -- 172.21.15.172:0/657205651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f50b00d2820 con 0x7f50b00d1a20 2024-04-14T08:40:49.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.683+0000 7f50ad7fa640 1 -- 172.21.15.172:0/657205651 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f50a807c5b0 con 0x7f50b00d1a20 2024-04-14T08:40:49.684 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:40:49.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.685+0000 7f50b6e88640 1 -- 172.21.15.172:0/657205651 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f509406c8f0 msgr2=0x7f509406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.685+0000 7f50b6e88640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f509406c8f0 0x7f509406edb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f50a4000f60 tx=0x7f50a4009040 comp rx=0 tx=0).stop 2024-04-14T08:40:49.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.685+0000 7f50b6e88640 1 -- 172.21.15.172:0/657205651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b00d1a20 msgr2=0x7f50b00ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.685+0000 7f50b6e88640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b00d1a20 0x7f50b00ca860 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f50a8037040 tx=0x7f50a8004a40 comp rx=0 tx=0).stop 2024-04-14T08:40:49.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.685+0000 7f50b6e88640 1 -- 172.21.15.172:0/657205651 shutdown_connections 2024-04-14T08:40:49.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.685+0000 7f50b6e88640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f509406c8f0 0x7f509406edb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.685+0000 7f50b6e88640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b00d1a20 0x7f50b00ca860 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.685+0000 7f50b6e88640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50b0071250 0x7f50b014b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.685+0000 7f50b6e88640 1 --2- 172.21.15.172:0/657205651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50b006d000 0x7f50b00c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.685+0000 7f50b6e88640 1 -- 172.21.15.172:0/657205651 >> 172.21.15.172:0/657205651 conn(0x7f50b008c000 msgr2=0x7f50b00ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:49.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.685+0000 7f50b6e88640 1 -- 172.21.15.172:0/657205651 shutdown_connections 2024-04-14T08:40:49.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.685+0000 7f50b6e88640 1 -- 172.21.15.172:0/657205651 wait complete. 2024-04-14T08:40:49.697 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:40:49.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.798+0000 7fb64fc88640 1 Processor -- start 2024-04-14T08:40:49.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.799+0000 7fb64fc88640 1 -- start start 2024-04-14T08:40:49.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.799+0000 7fb64fc88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb64815b2c0 0x7fb6480cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.799+0000 7fb64fc88640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb6480cb780 0x7fb6480cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.799+0000 7fb64fc88640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6480cfae0 con 0x7fb6480cc3c0 2024-04-14T08:40:49.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.799+0000 7fb64fc88640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb6480cfdc0 con 0x7fb6480cb780 2024-04-14T08:40:49.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.799+0000 7fb64fc88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb6480d00a0 con 0x7fb64815b2c0 2024-04-14T08:40:49.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.799+0000 7fb64d223640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb6480cb780 0x7fb6480cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.799+0000 7fb64d223640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb6480cb780 0x7fb6480cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54032/0 (socket says 172.21.15.172:54032) 2024-04-14T08:40:49.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.800+0000 7fb64d223640 1 -- 172.21.15.172:0/2023476424 learned_addr learned my addr 172.21.15.172:0/2023476424 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:49.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.800+0000 7fb64da24640 1 --2- 172.21.15.172:0/2023476424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb64815b2c0 0x7fb6480cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.800+0000 7fb64d223640 1 -- 172.21.15.172:0/2023476424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb64815b2c0 msgr2=0x7fb6480cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.800+0000 7fb64d223640 1 --2- 172.21.15.172:0/2023476424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb64815b2c0 0x7fb6480cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.800+0000 7fb64d223640 1 -- 172.21.15.172:0/2023476424 >> v1:172.21.15.177:6789/0 conn(0x7fb6480cc3c0 legacy=0x7fb6480722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.800+0000 7fb64d223640 1 -- 172.21.15.172:0/2023476424 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6480d0380 con 0x7fb6480cb780 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.801+0000 7fb64d223640 1 --2- 172.21.15.172:0/2023476424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6480cb780 0x7fb6480cbe80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009870 tx=0x7fb63c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae9d20b2e146026 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.801+0000 7fb64ca22640 1 -- 172.21.15.172:0/2023476424 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c02f840 con 0x7fb6480cb780 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.801+0000 7fb64ca22640 1 -- 172.21.15.172:0/2023476424 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb63c02f9e0 con 0x7fb6480cb780 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.801+0000 7fb64ca22640 1 -- 172.21.15.172:0/2023476424 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c02fdb0 con 0x7fb6480cb780 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.801+0000 7fb64fc88640 1 -- 172.21.15.172:0/2023476424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6480cb780 msgr2=0x7fb6480cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.801+0000 7fb64fc88640 1 --2- 172.21.15.172:0/2023476424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6480cb780 0x7fb6480cbe80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009870 tx=0x7fb63c02ba20 comp rx=0 tx=0).stop 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.801+0000 7fb64fc88640 1 -- 172.21.15.172:0/2023476424 shutdown_connections 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.801+0000 7fb64fc88640 1 --2- 172.21.15.172:0/2023476424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6480cb780 0x7fb6480cbe80 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.801+0000 7fb64fc88640 1 --2- 172.21.15.172:0/2023476424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb64815b2c0 0x7fb6480cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.801+0000 7fb64fc88640 1 -- 172.21.15.172:0/2023476424 >> 172.21.15.172:0/2023476424 conn(0x7fb6480d0f20 msgr2=0x7fb6480d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:49.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.801+0000 7fb64fc88640 1 -- 172.21.15.172:0/2023476424 shutdown_connections 2024-04-14T08:40:49.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.801+0000 7fb64fc88640 1 -- 172.21.15.172:0/2023476424 wait complete. 2024-04-14T08:40:49.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.803+0000 7fb64fc88640 1 Processor -- start 2024-04-14T08:40:49.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.803+0000 7fb64fc88640 1 -- start start 2024-04-14T08:40:49.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.804+0000 7fb64fc88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6480cb780 0x7fb64814ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.804+0000 7fb64fc88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6480cc3c0 0x7fb6481f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.804+0000 7fb64fc88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb64815b2c0 0x7fb64814ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.804+0000 7fb64fc88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb64815b6a0 con 0x7fb6480cc3c0 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.804+0000 7fb64fc88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb6480734c0 con 0x7fb6480cb780 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.804+0000 7fb64fc88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb64815b820 con 0x7fb64815b2c0 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64d223640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6480cc3c0 0x7fb6481f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64d223640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6480cc3c0 0x7fb6481f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54042/0 (socket says 172.21.15.172:54042) 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64d223640 1 -- 172.21.15.172:0/2988495427 learned_addr learned my addr 172.21.15.172:0/2988495427 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64da24640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6480cb780 0x7fb64814ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64e225640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb64815b2c0 0x7fb64814ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64d223640 1 -- 172.21.15.172:0/2988495427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb64815b2c0 msgr2=0x7fb64814ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64d223640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb64815b2c0 0x7fb64814ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64d223640 1 -- 172.21.15.172:0/2988495427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6480cb780 msgr2=0x7fb64814ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:49.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64d223640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6480cb780 0x7fb64814ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:49.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64d223640 1 -- 172.21.15.172:0/2988495427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64814f250 con 0x7fb6480cc3c0 2024-04-14T08:40:49.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64da24640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6480cb780 0x7fb64814ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:49.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64d223640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6480cc3c0 0x7fb6481f2500 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb63c000c00 tx=0x7fb63c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:49.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb6467fc640 1 -- 172.21.15.172:0/2988495427 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c037bc0 con 0x7fb6480cc3c0 2024-04-14T08:40:49.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb6467fc640 1 -- 172.21.15.172:0/2988495427 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb63c046070 con 0x7fb6480cc3c0 2024-04-14T08:40:49.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64fc88640 1 -- 172.21.15.172:0/2988495427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64814f4c0 con 0x7fb6480cc3c0 2024-04-14T08:40:49.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb6467fc640 1 -- 172.21.15.172:0/2988495427 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c041400 con 0x7fb6480cc3c0 2024-04-14T08:40:49.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.805+0000 7fb64fc88640 1 -- 172.21.15.172:0/2988495427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64814fa80 con 0x7fb6480cc3c0 2024-04-14T08:40:49.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.806+0000 7fb64fc88640 1 -- 172.21.15.172:0/2988495427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64814fc80 con 0x7fb6480cc3c0 2024-04-14T08:40:49.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.807+0000 7fb6467fc640 1 -- 172.21.15.172:0/2988495427 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb63c040070 con 0x7fb6480cc3c0 2024-04-14T08:40:49.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.807+0000 7fb6467fc640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb63406c920 0x7fb63406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:49.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.807+0000 7fb64da24640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb63406c920 0x7fb63406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:49.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.807+0000 7fb6467fc640 1 -- 172.21.15.172:0/2988495427 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fb63c0ae400 con 0x7fb6480cc3c0 2024-04-14T08:40:49.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.808+0000 7fb64da24640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb63406c920 0x7fb63406ede0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb638009a20 tx=0x7fb638008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:49.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:49.812+0000 7fb6467fc640 1 -- 172.21.15.172:0/2988495427 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb63c033030 con 0x7fb6480cc3c0 2024-04-14T08:40:50.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.006+0000 7fb64fc88640 1 -- 172.21.15.172:0/2988495427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb6480cbf00 con 0x7fb6480cc3c0 2024-04-14T08:40:50.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.007+0000 7fb6467fc640 1 -- 172.21.15.172:0/2988495427 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v90) ==== 80+0+248 (secure 0 0 0) 0x7fb63c07bf30 con 0x7fb6480cc3c0 2024-04-14T08:40:50.007 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6837/1386685680 2024-04-15T08:40:10.107666+0000 2024-04-14T08:40:50.007 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6835/1386685680 2024-04-15T08:40:10.107666+0000 2024-04-14T08:40:50.008 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.177:6835/1124731311 2024-04-15T08:38:58.388334+0000 2024-04-14T08:40:50.008 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.177:6833/1124731311 2024-04-15T08:38:58.388334+0000 2024-04-14T08:40:50.008 INFO:teuthology.orchestra.run.smithi172.stderr:listed 4 entries 2024-04-14T08:40:50.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.009+0000 7fb64fc88640 1 -- 172.21.15.172:0/2988495427 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb63406c920 msgr2=0x7fb63406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.009+0000 7fb64fc88640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb63406c920 0x7fb63406ede0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb638009a20 tx=0x7fb638008040 comp rx=0 tx=0).stop 2024-04-14T08:40:50.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.009+0000 7fb64fc88640 1 -- 172.21.15.172:0/2988495427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6480cc3c0 msgr2=0x7fb6481f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.009+0000 7fb64fc88640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6480cc3c0 0x7fb6481f2500 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb63c000c00 tx=0x7fb63c004730 comp rx=0 tx=0).stop 2024-04-14T08:40:50.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.009+0000 7fb64fc88640 1 -- 172.21.15.172:0/2988495427 shutdown_connections 2024-04-14T08:40:50.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.009+0000 7fb64fc88640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb63406c920 0x7fb63406ede0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.009+0000 7fb64fc88640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb64815b2c0 0x7fb64814ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.009+0000 7fb64fc88640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6480cc3c0 0x7fb6481f2500 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.009+0000 7fb64fc88640 1 --2- 172.21.15.172:0/2988495427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6480cb780 0x7fb64814ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.009+0000 7fb64fc88640 1 -- 172.21.15.172:0/2988495427 >> 172.21.15.172:0/2988495427 conn(0x7fb6480d0f20 msgr2=0x7fb6480b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:50.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.009+0000 7fb64fc88640 1 -- 172.21.15.172:0/2988495427 shutdown_connections 2024-04-14T08:40:50.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.009+0000 7fb64fc88640 1 -- 172.21.15.172:0/2988495427 wait complete. 2024-04-14T08:40:50.020 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:40:50.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:50.021 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:40:50.051 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:40:50.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:40:50.051 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:40:50.106 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:40:50.106 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:40:50.106 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:40:50.126 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:40:50.126 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:40:50.127 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:40:50.127 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:40:50.165 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:40:50.165 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:40:50.165 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:40:50.165 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:40:50.186 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:40:50.186 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:40:50.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.307+0000 7f272be08640 1 Processor -- start 2024-04-14T08:40:50.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.307+0000 7f272be08640 1 -- start start 2024-04-14T08:40:50.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.308+0000 7f272be08640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f27240d2830 0x7f27240d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.308+0000 7f272be08640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f272415e890 0x7f27240d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.308+0000 7f272be08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27240cedd0 0x7f27241621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.308+0000 7f272be08640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f27240cff30 con 0x7f272415e890 2024-04-14T08:40:50.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.308+0000 7f272be08640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f27240d2350 con 0x7f27240d2830 2024-04-14T08:40:50.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.308+0000 7f272be08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f27240d3150 con 0x7f27240cedd0 2024-04-14T08:40:50.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.308+0000 7f27293a3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f272415e890 0x7f27240d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.308+0000 7f27293a3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f272415e890 0x7f27240d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54056/0 (socket says 172.21.15.172:54056) 2024-04-14T08:40:50.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.308+0000 7f27293a3640 1 -- 172.21.15.172:0/2853015130 learned_addr learned my addr 172.21.15.172:0/2853015130 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:50.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.309+0000 7f2729ba4640 1 --2- 172.21.15.172:0/2853015130 >> v2:172.21.15.177:3300/0 conn(0x7f27240d2830 0x7f27240d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.309+0000 7f272a3a5640 1 --2- 172.21.15.172:0/2853015130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27240cedd0 0x7f27241621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.309+0000 7f27293a3640 1 -- 172.21.15.172:0/2853015130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27240cedd0 msgr2=0x7f27241621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:50.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.309+0000 7f27293a3640 1 --2- 172.21.15.172:0/2853015130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27240cedd0 0x7f27241621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.309+0000 7f27293a3640 1 -- 172.21.15.172:0/2853015130 >> v2:172.21.15.177:3300/0 conn(0x7f27240d2830 msgr2=0x7f27240d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:50.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.309+0000 7f27293a3640 1 --2- 172.21.15.172:0/2853015130 >> v2:172.21.15.177:3300/0 conn(0x7f27240d2830 0x7f27240d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.309+0000 7f27293a3640 1 -- 172.21.15.172:0/2853015130 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27240cf320 con 0x7f272415e890 2024-04-14T08:40:50.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.309+0000 7f27293a3640 1 --2- 172.21.15.172:0/2853015130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272415e890 0x7f27240d1300 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f272408c830 tx=0x7f271802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d601314278a67ce server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:50.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.310+0000 7f2728ba2640 1 -- 172.21.15.172:0/2853015130 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718034070 con 0x7f272415e890 2024-04-14T08:40:50.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.310+0000 7f2728ba2640 1 -- 172.21.15.172:0/2853015130 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2718030860 con 0x7f272415e890 2024-04-14T08:40:50.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.310+0000 7f2728ba2640 1 -- 172.21.15.172:0/2853015130 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718030c30 con 0x7f272415e890 2024-04-14T08:40:50.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.310+0000 7f272be08640 1 -- 172.21.15.172:0/2853015130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272415e890 msgr2=0x7f27240d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.310+0000 7f272be08640 1 --2- 172.21.15.172:0/2853015130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272415e890 0x7f27240d1300 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f272408c830 tx=0x7f271802b9a0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.310+0000 7f272be08640 1 -- 172.21.15.172:0/2853015130 shutdown_connections 2024-04-14T08:40:50.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.310+0000 7f272be08640 1 --2- 172.21.15.172:0/2853015130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27240cedd0 0x7f27241621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.310+0000 7f272be08640 1 --2- 172.21.15.172:0/2853015130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272415e890 0x7f27240d1300 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.310+0000 7f272be08640 1 --2- 172.21.15.172:0/2853015130 >> v2:172.21.15.177:3300/0 conn(0x7f27240d2830 0x7f27240d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.310+0000 7f272be08640 1 -- 172.21.15.172:0/2853015130 >> 172.21.15.172:0/2853015130 conn(0x7f272409d590 msgr2=0x7f27240d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:50.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.310+0000 7f272be08640 1 -- 172.21.15.172:0/2853015130 shutdown_connections 2024-04-14T08:40:50.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.310+0000 7f272be08640 1 -- 172.21.15.172:0/2853015130 wait complete. 2024-04-14T08:40:50.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.311+0000 7f272be08640 1 Processor -- start 2024-04-14T08:40:50.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.311+0000 7f272be08640 1 -- start start 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.312+0000 7f272be08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27240cedd0 0x7f27240b9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.312+0000 7f272be08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27240d2830 0x7f27240b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.312+0000 7f272be08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272415e890 0x7f27241612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.312+0000 7f272be08640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27240cfbc0 con 0x7f272415e890 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.312+0000 7f272be08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f27240cf8e0 con 0x7f27240cedd0 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.312+0000 7f272be08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f27240d1840 con 0x7f27240d2830 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.312+0000 7f272a3a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272415e890 0x7f27241612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.312+0000 7f272a3a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272415e890 0x7f27241612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54070/0 (socket says 172.21.15.172:54070) 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.312+0000 7f272a3a5640 1 -- 172.21.15.172:0/1388304183 learned_addr learned my addr 172.21.15.172:0/1388304183 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.312+0000 7f27293a3640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27240d2830 0x7f27240b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.312+0000 7f2729ba4640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27240cedd0 0x7f27240b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.313+0000 7f272a3a5640 1 -- 172.21.15.172:0/1388304183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27240d2830 msgr2=0x7f27240b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.313+0000 7f272a3a5640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27240d2830 0x7f27240b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.313+0000 7f272a3a5640 1 -- 172.21.15.172:0/1388304183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27240cedd0 msgr2=0x7f27240b9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.313+0000 7f272a3a5640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27240cedd0 0x7f27240b9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.313+0000 7f272a3a5640 1 -- 172.21.15.172:0/1388304183 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27240b9c90 con 0x7f272415e890 2024-04-14T08:40:50.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.313+0000 7f272a3a5640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272415e890 0x7f27241612a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f271c007e90 tx=0x7f271c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:50.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.313+0000 7f27167fc640 1 -- 172.21.15.172:0/1388304183 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c014420 con 0x7f272415e890 2024-04-14T08:40:50.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.313+0000 7f27167fc640 1 -- 172.21.15.172:0/1388304183 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271c019070 con 0x7f272415e890 2024-04-14T08:40:50.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.313+0000 7f27167fc640 1 -- 172.21.15.172:0/1388304183 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c014800 con 0x7f272415e890 2024-04-14T08:40:50.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.313+0000 7f272be08640 1 -- 172.21.15.172:0/1388304183 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27241597e0 con 0x7f272415e890 2024-04-14T08:40:50.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.313+0000 7f272be08640 1 -- 172.21.15.172:0/1388304183 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27241607c0 con 0x7f272415e890 2024-04-14T08:40:50.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.314+0000 7f272be08640 1 -- 172.21.15.172:0/1388304183 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2724160260 con 0x7f272415e890 2024-04-14T08:40:50.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.315+0000 7f27167fc640 1 -- 172.21.15.172:0/1388304183 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f271c0149a0 con 0x7f272415e890 2024-04-14T08:40:50.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.315+0000 7f27167fc640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f270806c8f0 0x7f270806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.315+0000 7f27167fc640 1 -- 172.21.15.172:0/1388304183 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f271c08c4b0 con 0x7f272415e890 2024-04-14T08:40:50.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.315+0000 7f2729ba4640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f270806c8f0 0x7f270806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.315+0000 7f2729ba4640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f270806c8f0 0x7f270806edb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2720007920 tx=0x7f2720008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:50.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.320+0000 7f27167fc640 1 -- 172.21.15.172:0/1388304183 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2724160260 con 0x7f272415e890 2024-04-14T08:40:50.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.565+0000 7f272be08640 1 -- 172.21.15.172:0/1388304183 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27240b9ea0 con 0x7f272415e890 2024-04-14T08:40:50.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.566+0000 7f27167fc640 1 -- 172.21.15.172:0/1388304183 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5196 (secure 0 0 0) 0x7f27240b9c90 con 0x7f272415e890 2024-04-14T08:40:50.566 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:50.566 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:40:14.122456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8013":{"gid":8013,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.172:6837/3091771014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3091771014},{"type":"v1","addr":"172.21.15.172:6837","nonce":3091771014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8013,"qdb_cluster":[8013]},"id":6}]} 2024-04-14T08:40:50.567 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 67 2024-04-14T08:40:50.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.567+0000 7f272be08640 1 -- 172.21.15.172:0/1388304183 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f270806c8f0 msgr2=0x7f270806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.567+0000 7f272be08640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f270806c8f0 0x7f270806edb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2720007920 tx=0x7f2720008040 comp rx=0 tx=0).stop 2024-04-14T08:40:50.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.567+0000 7f272be08640 1 -- 172.21.15.172:0/1388304183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272415e890 msgr2=0x7f27241612a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.567+0000 7f272be08640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272415e890 0x7f27241612a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f271c007e90 tx=0x7f271c00e4b0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.567+0000 7f272be08640 1 -- 172.21.15.172:0/1388304183 shutdown_connections 2024-04-14T08:40:50.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.567+0000 7f272be08640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f270806c8f0 0x7f270806edb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.567+0000 7f272be08640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f272415e890 0x7f27241612a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.567+0000 7f272be08640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27240d2830 0x7f27240b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.567+0000 7f272be08640 1 --2- 172.21.15.172:0/1388304183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27240cedd0 0x7f27240b9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.567+0000 7f272be08640 1 -- 172.21.15.172:0/1388304183 >> 172.21.15.172:0/1388304183 conn(0x7f272409d590 msgr2=0x7f27240d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:50.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.568+0000 7f272be08640 1 -- 172.21.15.172:0/1388304183 shutdown_connections 2024-04-14T08:40:50.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.568+0000 7f272be08640 1 -- 172.21.15.172:0/1388304183 wait complete. 2024-04-14T08:40:50.579 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:40:50.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.681+0000 7fd46dbe0640 1 Processor -- start 2024-04-14T08:40:50.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.681+0000 7fd46dbe0640 1 -- start start 2024-04-14T08:40:50.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.681+0000 7fd46dbe0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4680d1a10 0x7fd468161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.681+0000 7fd46dbe0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd468162530 0x7fd4680d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.681+0000 7fd46dbe0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4680d0200 con 0x7fd46815c150 2024-04-14T08:40:50.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.681+0000 7fd46dbe0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd4680cef10 con 0x7fd468162530 2024-04-14T08:40:50.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.681+0000 7fd46dbe0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd468162960 con 0x7fd4680d1a10 2024-04-14T08:40:50.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.682+0000 7fd467fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd468162530 0x7fd4680d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.682+0000 7fd4677fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd46815c150 0x7fd4680d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50402/0 (socket says 172.21.15.172:50402) 2024-04-14T08:40:50.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.682+0000 7fd4677fe640 1 -- 172.21.15.172:0/1478565102 learned_addr learned my addr 172.21.15.172:0/1478565102 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:50.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.682+0000 7fd45ffff640 1 --2- 172.21.15.172:0/1478565102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4680d1a10 0x7fd468161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.682+0000 7fd467fff640 1 -- 172.21.15.172:0/1478565102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4680d1a10 msgr2=0x7fd468161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:50.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.682+0000 7fd467fff640 1 --2- 172.21.15.172:0/1478565102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4680d1a10 0x7fd468161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.682+0000 7fd467fff640 1 -- 172.21.15.172:0/1478565102 >> v1:172.21.15.172:6789/0 conn(0x7fd46815c150 legacy=0x7fd4680d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.682+0000 7fd467fff640 1 -- 172.21.15.172:0/1478565102 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4680cf160 con 0x7fd468162530 2024-04-14T08:40:50.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.682+0000 7fd466ffd640 1 -- 172.21.15.172:0/1478565102 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4149763514 0 0) 0x7fd4680d0200 con 0x7fd46815c150 2024-04-14T08:40:50.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.683+0000 7fd467fff640 1 --2- 172.21.15.172:0/1478565102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd468162530 0x7fd4680d1e80 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd460009d90 tx=0x7fd46002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7267384263a20851 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:50.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.683+0000 7fd466ffd640 1 -- 172.21.15.172:0/1478565102 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460039070 con 0x7fd468162530 2024-04-14T08:40:50.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.683+0000 7fd466ffd640 1 -- 172.21.15.172:0/1478565102 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd46002cc50 con 0x7fd468162530 2024-04-14T08:40:50.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.683+0000 7fd466ffd640 1 -- 172.21.15.172:0/1478565102 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460034440 con 0x7fd468162530 2024-04-14T08:40:50.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.683+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1478565102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd468162530 msgr2=0x7fd4680d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.683+0000 7fd46dbe0640 1 --2- 172.21.15.172:0/1478565102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd468162530 0x7fd4680d1e80 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd460009d90 tx=0x7fd46002bea0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.683+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1478565102 shutdown_connections 2024-04-14T08:40:50.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.683+0000 7fd46dbe0640 1 --2- 172.21.15.172:0/1478565102 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4680d1a10 0x7fd468161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.684+0000 7fd46dbe0640 1 --2- 172.21.15.172:0/1478565102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd468162530 0x7fd4680d1e80 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.684+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1478565102 >> 172.21.15.172:0/1478565102 conn(0x7fd46809d580 msgr2=0x7fd4680d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:50.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.684+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1478565102 shutdown_connections 2024-04-14T08:40:50.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.684+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1478565102 wait complete. 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.685+0000 7fd46dbe0640 1 Processor -- start 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.685+0000 7fd46dbe0640 1 -- start start 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.685+0000 7fd46dbe0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4680d1a10 0x7fd4680b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.685+0000 7fd46dbe0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46815c150 0x7fd468160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.685+0000 7fd46dbe0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd468162530 0x7fd46815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.685+0000 7fd46dbe0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd4680cfa00 con 0x7fd46815c150 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.685+0000 7fd46dbe0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd4680cf720 con 0x7fd4680d1a10 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.685+0000 7fd46dbe0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd4680cfce0 con 0x7fd468162530 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd45ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46815c150 0x7fd468160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd45ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46815c150 0x7fd468160f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54090/0 (socket says 172.21.15.172:54090) 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd45ffff640 1 -- 172.21.15.172:0/1513047528 learned_addr learned my addr 172.21.15.172:0/1513047528 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd4677fe640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4680d1a10 0x7fd4680b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd467fff640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd468162530 0x7fd46815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd45ffff640 1 -- 172.21.15.172:0/1513047528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd468162530 msgr2=0x7fd46815acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd45ffff640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd468162530 0x7fd46815acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd45ffff640 1 -- 172.21.15.172:0/1513047528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4680d1a10 msgr2=0x7fd4680b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd45ffff640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4680d1a10 0x7fd4680b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd45ffff640 1 -- 172.21.15.172:0/1513047528 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4680b5e20 con 0x7fd46815c150 2024-04-14T08:40:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd467fff640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd468162530 0x7fd46815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd45ffff640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46815c150 0x7fd468160f90 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e9e0 tx=0x7fd45000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd464ff9640 1 -- 172.21.15.172:0/1513047528 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450017070 con 0x7fd46815c150 2024-04-14T08:40:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd464ff9640 1 -- 172.21.15.172:0/1513047528 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd45000c9d0 con 0x7fd46815c150 2024-04-14T08:40:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1513047528 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4680b41b0 con 0x7fd46815c150 2024-04-14T08:40:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd464ff9640 1 -- 172.21.15.172:0/1513047528 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45000cdc0 con 0x7fd46815c150 2024-04-14T08:40:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.686+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1513047528 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4680b4770 con 0x7fd46815c150 2024-04-14T08:40:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.687+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1513047528 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd468075490 con 0x7fd46815c150 2024-04-14T08:40:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.688+0000 7fd464ff9640 1 -- 172.21.15.172:0/1513047528 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd45001a070 con 0x7fd46815c150 2024-04-14T08:40:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.688+0000 7fd464ff9640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd44806c8f0 0x7fd44806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.688+0000 7fd464ff9640 1 -- 172.21.15.172:0/1513047528 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fd45008c0f0 con 0x7fd46815c150 2024-04-14T08:40:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.688+0000 7fd4677fe640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd44806c8f0 0x7fd44806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:50.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.688+0000 7fd4677fe640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd44806c8f0 0x7fd44806edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd458000f60 tx=0x7fd458009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:50.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.692+0000 7fd464ff9640 1 -- 172.21.15.172:0/1513047528 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd450059c20 con 0x7fd46815c150 2024-04-14T08:40:50.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.898+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1513047528 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd468162d30 con 0x7fd46815c150 2024-04-14T08:40:50.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.899+0000 7fd464ff9640 1 -- 172.21.15.172:0/1513047528 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+17401 (secure 0 0 0) 0x7fd45005df20 con 0x7fd46815c150 2024-04-14T08:40:50.900 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:50.900 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":90,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:40:10.107688+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:38:42.790996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-14T08:38:43.517208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/1386685680":"2024-04-15T08:40:10.107666+0000","172.21.15.172:6835/1386685680":"2024-04-15T08:40:10.107666+0000","172.21.15.177:6835/1124731311":"2024-04-15T08:38:58.388334+0000","172.21.15.177:6833/1124731311":"2024-04-15T08:38:58.388334+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:40:50.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.901+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1513047528 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd44806c8f0 msgr2=0x7fd44806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.901+0000 7fd46dbe0640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd44806c8f0 0x7fd44806edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd458000f60 tx=0x7fd458009040 comp rx=0 tx=0).stop 2024-04-14T08:40:50.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.901+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1513047528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46815c150 msgr2=0x7fd468160f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:50.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.901+0000 7fd46dbe0640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46815c150 0x7fd468160f90 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e9e0 tx=0x7fd45000eea0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.901+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1513047528 shutdown_connections 2024-04-14T08:40:50.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.901+0000 7fd46dbe0640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd44806c8f0 0x7fd44806edb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.901+0000 7fd46dbe0640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd468162530 0x7fd46815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.901+0000 7fd46dbe0640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46815c150 0x7fd468160f90 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.901+0000 7fd46dbe0640 1 --2- 172.21.15.172:0/1513047528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4680d1a10 0x7fd4680b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:50.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.901+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1513047528 >> 172.21.15.172:0/1513047528 conn(0x7fd46809d580 msgr2=0x7fd4680cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:50.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.902+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1513047528 shutdown_connections 2024-04-14T08:40:50.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:50.902+0000 7fd46dbe0640 1 -- 172.21.15.172:0/1513047528 wait complete. 2024-04-14T08:40:50.914 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:38:42.790996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:38:43.517208+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:40:50.914 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:40:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.021+0000 7fc8a717b640 1 Processor -- start 2024-04-14T08:40:51.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.021+0000 7fc8a717b640 1 -- start start 2024-04-14T08:40:51.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.022+0000 7fc8a717b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8a015c420 0x7fc8a00b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.022+0000 7fc8a717b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc8a00ceb00 0x7fc8a00cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.022+0000 7fc8a717b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8a00cf450 con 0x7fc8a006f3e0 2024-04-14T08:40:51.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.022+0000 7fc8a717b640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc8a00d30a0 con 0x7fc8a015c420 2024-04-14T08:40:51.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.022+0000 7fc8a717b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc8a006f870 con 0x7fc8a00ceb00 2024-04-14T08:40:51.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.022+0000 7fc8a4f17640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8a015c420 0x7fc8a00b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.022+0000 7fc8a4f17640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8a015c420 0x7fc8a00b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54098/0 (socket says 172.21.15.172:54098) 2024-04-14T08:40:51.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.022+0000 7fc8a4f17640 1 -- 172.21.15.172:0/2027048370 learned_addr learned my addr 172.21.15.172:0/2027048370 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:51.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.022+0000 7fc8a5718640 1 --2- 172.21.15.172:0/2027048370 >> v2:172.21.15.177:3300/0 conn(0x7fc8a00ceb00 0x7fc8a00cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.023+0000 7fc8a4f17640 1 -- 172.21.15.172:0/2027048370 >> v2:172.21.15.177:3300/0 conn(0x7fc8a00ceb00 msgr2=0x7fc8a00cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:51.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.023+0000 7fc8a4f17640 1 --2- 172.21.15.172:0/2027048370 >> v2:172.21.15.177:3300/0 conn(0x7fc8a00ceb00 0x7fc8a00cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.023+0000 7fc8a4f17640 1 -- 172.21.15.172:0/2027048370 >> v1:172.21.15.177:6789/0 conn(0x7fc8a006f3e0 legacy=0x7fc8a00d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.023+0000 7fc8a4f17640 1 -- 172.21.15.172:0/2027048370 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a00ccdd0 con 0x7fc8a015c420 2024-04-14T08:40:51.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.023+0000 7fc8a5718640 1 --2- 172.21.15.172:0/2027048370 >> v2:172.21.15.177:3300/0 conn(0x7fc8a00ceb00 0x7fc8a00cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:51.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.024+0000 7fc8a4f17640 1 --2- 172.21.15.172:0/2027048370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8a015c420 0x7fc8a00b1510 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00cc6f0 tx=0x7fc89802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cafa6766cf689e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:51.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.024+0000 7fc89f7fe640 1 -- 172.21.15.172:0/2027048370 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89802cb20 con 0x7fc8a015c420 2024-04-14T08:40:51.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.024+0000 7fc89f7fe640 1 -- 172.21.15.172:0/2027048370 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc89802ccc0 con 0x7fc8a015c420 2024-04-14T08:40:51.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.024+0000 7fc89f7fe640 1 -- 172.21.15.172:0/2027048370 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc898034610 con 0x7fc8a015c420 2024-04-14T08:40:51.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.024+0000 7fc8a717b640 1 -- 172.21.15.172:0/2027048370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8a015c420 msgr2=0x7fc8a00b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.024+0000 7fc8a717b640 1 --2- 172.21.15.172:0/2027048370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8a015c420 0x7fc8a00b1510 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00cc6f0 tx=0x7fc89802bf80 comp rx=0 tx=0).stop 2024-04-14T08:40:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.024+0000 7fc8a717b640 1 -- 172.21.15.172:0/2027048370 shutdown_connections 2024-04-14T08:40:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.024+0000 7fc8a717b640 1 --2- 172.21.15.172:0/2027048370 >> v2:172.21.15.177:3300/0 conn(0x7fc8a00ceb00 0x7fc8a00cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.024+0000 7fc8a717b640 1 --2- 172.21.15.172:0/2027048370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8a015c420 0x7fc8a00b1510 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.024+0000 7fc8a717b640 1 -- 172.21.15.172:0/2027048370 >> 172.21.15.172:0/2027048370 conn(0x7fc8a008c000 msgr2=0x7fc8a00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.024+0000 7fc8a717b640 1 -- 172.21.15.172:0/2027048370 shutdown_connections 2024-04-14T08:40:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.024+0000 7fc8a717b640 1 -- 172.21.15.172:0/2027048370 wait complete. 2024-04-14T08:40:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.026+0000 7fc8a717b640 1 Processor -- start 2024-04-14T08:40:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.026+0000 7fc8a717b640 1 -- start start 2024-04-14T08:40:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.026+0000 7fc8a717b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8a00ceb00 0x7fc8a0156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.026+0000 7fc8a717b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8a015c420 0x7fc8a00c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.026+0000 7fc8a717b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8a00b6d40 0x7fc8a00b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.026+0000 7fc8a717b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a00cd070 con 0x7fc8a00b6d40 2024-04-14T08:40:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.026+0000 7fc8a717b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a00d2dc0 con 0x7fc8a00ceb00 2024-04-14T08:40:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.026+0000 7fc8a717b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a00b11e0 con 0x7fc8a015c420 2024-04-14T08:40:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc8a5718640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8a00b6d40 0x7fc8a00b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc8a5718640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8a00b6d40 0x7fc8a00b71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54114/0 (socket says 172.21.15.172:54114) 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc8a5718640 1 -- 172.21.15.172:0/3118040946 learned_addr learned my addr 172.21.15.172:0/3118040946 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc8a4f17640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8a00ceb00 0x7fc8a0156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc89ffff640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8a015c420 0x7fc8a00c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc8a5718640 1 -- 172.21.15.172:0/3118040946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8a015c420 msgr2=0x7fc8a00c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc8a5718640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8a015c420 0x7fc8a00c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc8a5718640 1 -- 172.21.15.172:0/3118040946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8a00ceb00 msgr2=0x7fc8a0156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc8a5718640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8a00ceb00 0x7fc8a0156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc8a5718640 1 -- 172.21.15.172:0/3118040946 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a00c6bf0 con 0x7fc8a00b6d40 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc8a5718640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8a00b6d40 0x7fc8a00b71f0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e7e0 tx=0x7fc89000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc89d7fa640 1 -- 172.21.15.172:0/3118040946 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89000f040 con 0x7fc8a00b6d40 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc89d7fa640 1 -- 172.21.15.172:0/3118040946 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc890004030 con 0x7fc8a00b6d40 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc8a717b640 1 -- 172.21.15.172:0/3118040946 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a00caeb0 con 0x7fc8a00b6d40 2024-04-14T08:40:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc89d7fa640 1 -- 172.21.15.172:0/3118040946 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8900043e0 con 0x7fc8a00b6d40 2024-04-14T08:40:51.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.027+0000 7fc8a717b640 1 -- 172.21.15.172:0/3118040946 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a00b1590 con 0x7fc8a00b6d40 2024-04-14T08:40:51.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.029+0000 7fc8a717b640 1 -- 172.21.15.172:0/3118040946 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a0075490 con 0x7fc8a00b6d40 2024-04-14T08:40:51.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.029+0000 7fc89d7fa640 1 -- 172.21.15.172:0/3118040946 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc89000c6d0 con 0x7fc8a00b6d40 2024-04-14T08:40:51.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.029+0000 7fc89d7fa640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc88c06c920 0x7fc88c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.029+0000 7fc89d7fa640 1 -- 172.21.15.172:0/3118040946 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fc89008bd50 con 0x7fc8a00b6d40 2024-04-14T08:40:51.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.029+0000 7fc8a4f17640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc88c06c920 0x7fc88c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.030+0000 7fc8a4f17640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc88c06c920 0x7fc88c06ede0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc898035000 tx=0x7fc898030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:51.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.034+0000 7fc89d7fa640 1 -- 172.21.15.172:0/3118040946 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc89008f030 con 0x7fc8a00b6d40 2024-04-14T08:40:51.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.376+0000 7fc8a717b640 1 -- 172.21.15.172:0/3118040946 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8a00d2820 con 0x7fc8a00b6d40 2024-04-14T08:40:51.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.377+0000 7fc89d7fa640 1 -- 172.21.15.172:0/3118040946 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fc890017020 con 0x7fc8a00b6d40 2024-04-14T08:40:51.379 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:40:51.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.378+0000 7fc8a717b640 1 -- 172.21.15.172:0/3118040946 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc88c06c920 msgr2=0x7fc88c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.378+0000 7fc8a717b640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc88c06c920 0x7fc88c06ede0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc898035000 tx=0x7fc898030000 comp rx=0 tx=0).stop 2024-04-14T08:40:51.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.378+0000 7fc8a717b640 1 -- 172.21.15.172:0/3118040946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8a00b6d40 msgr2=0x7fc8a00b71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.378+0000 7fc8a717b640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8a00b6d40 0x7fc8a00b71f0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e7e0 tx=0x7fc89000eca0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.379+0000 7fc8a717b640 1 -- 172.21.15.172:0/3118040946 shutdown_connections 2024-04-14T08:40:51.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.379+0000 7fc8a717b640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc88c06c920 0x7fc88c06ede0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.379+0000 7fc8a717b640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8a00b6d40 0x7fc8a00b71f0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.379+0000 7fc8a717b640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8a015c420 0x7fc8a00c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.379+0000 7fc8a717b640 1 --2- 172.21.15.172:0/3118040946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8a00ceb00 0x7fc8a0156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.379+0000 7fc8a717b640 1 -- 172.21.15.172:0/3118040946 >> 172.21.15.172:0/3118040946 conn(0x7fc8a008c000 msgr2=0x7fc8a00d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:51.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.379+0000 7fc8a717b640 1 -- 172.21.15.172:0/3118040946 shutdown_connections 2024-04-14T08:40:51.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.379+0000 7fc8a717b640 1 -- 172.21.15.172:0/3118040946 wait complete. 2024-04-14T08:40:51.391 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:40:51.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.495+0000 7f30fcd15640 1 Processor -- start 2024-04-14T08:40:51.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.495+0000 7f30fcd15640 1 -- start start 2024-04-14T08:40:51.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.496+0000 7f30fcd15640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f30f806f3e0 0x7f30f80b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.496+0000 7f30fcd15640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f80ccdd0 con 0x7f30f80ceb00 2024-04-14T08:40:51.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.496+0000 7f30fcd15640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f80d2ae0 con 0x7f30f815c420 2024-04-14T08:40:51.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.496+0000 7f30fcd15640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f30f80d30a0 con 0x7f30f806f3e0 2024-04-14T08:40:51.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.496+0000 7f30f6d9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f30f80ceb00 0x7f30f80cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50410/0 (socket says 172.21.15.172:50410) 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.496+0000 7f30f6d9d640 1 -- 172.21.15.172:0/2750575643 learned_addr learned my addr 172.21.15.172:0/2750575643 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.496+0000 7f30effff640 1 --2- 172.21.15.172:0/2750575643 >> v2:172.21.15.177:3300/0 conn(0x7f30f806f3e0 0x7f30f80b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.496+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2572260572 0 0) 0x7f30f80ccdd0 con 0x7f30f80ceb00 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.496+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30e4003520 con 0x7f30f80ceb00 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3417265800 0 0) 0x7f30e4003520 con 0x7f30f80ceb00 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 >> v2:172.21.15.177:3300/0 conn(0x7f30f806f3e0 msgr2=0x7f30f80b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 --2- 172.21.15.172:0/2750575643 >> v2:172.21.15.177:3300/0 conn(0x7f30f806f3e0 0x7f30f80b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 >> v1:172.21.15.177:6789/0 conn(0x7f30f815c420 legacy=0x7f30f80d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f80d2dc0 con 0x7f30f80ceb00 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f30f00019f0 con 0x7f30f80ceb00 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 >> v1:172.21.15.172:6789/0 conn(0x7f30f80ceb00 legacy=0x7f30f80cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 --2- 172.21.15.172:0/2750575643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e40039b0 0x7f30e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 --2- 172.21.15.172:0/2750575643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e4004800 0x7f30e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 --2- 172.21.15.172:0/2750575643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e40051c0 0x7f30e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30f80d30a0 con 0x7f30e40039b0 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f30e400c060 con 0x7f30e4004800 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f30e400c260 con 0x7f30e40051c0 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f659c640 1 --2- 172.21.15.172:0/2750575643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e40039b0 0x7f30e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30effff640 1 --2- 172.21.15.172:0/2750575643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e4004800 0x7f30e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f6d9d640 1 --2- 172.21.15.172:0/2750575643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e40051c0 0x7f30e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f659c640 1 -- 172.21.15.172:0/2750575643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e40051c0 msgr2=0x7f30e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f659c640 1 --2- 172.21.15.172:0/2750575643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e40051c0 0x7f30e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f659c640 1 -- 172.21.15.172:0/2750575643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e4004800 msgr2=0x7f30e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f659c640 1 --2- 172.21.15.172:0/2750575643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e4004800 0x7f30e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f659c640 1 -- 172.21.15.172:0/2750575643 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e4003520 con 0x7f30e40039b0 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.497+0000 7f30f6d9d640 1 -- 172.21.15.172:0/2750575643 reap_dead start 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.498+0000 7f30f659c640 1 --2- 172.21.15.172:0/2750575643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e40039b0 0x7f30e4003dd0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f30e8001750 tx=0x7f30e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a000a72c95c9d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:51.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.498+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8001370 con 0x7f30e40039b0 2024-04-14T08:40:51.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.498+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e80052d0 con 0x7f30e40039b0 2024-04-14T08:40:51.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.498+0000 7f30f5d9b640 1 -- 172.21.15.172:0/2750575643 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8003830 con 0x7f30e40039b0 2024-04-14T08:40:51.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.498+0000 7f30fcd15640 1 -- 172.21.15.172:0/2750575643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e40039b0 msgr2=0x7f30e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.499+0000 7f30fcd15640 1 --2- 172.21.15.172:0/2750575643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e40039b0 0x7f30e4003dd0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f30e8001750 tx=0x7f30e4005830 comp rx=0 tx=0).stop 2024-04-14T08:40:51.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.499+0000 7f30fcd15640 1 -- 172.21.15.172:0/2750575643 shutdown_connections 2024-04-14T08:40:51.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.499+0000 7f30fcd15640 1 --2- 172.21.15.172:0/2750575643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e40039b0 0x7f30e4003dd0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.499+0000 7f30fcd15640 1 -- 172.21.15.172:0/2750575643 >> 172.21.15.172:0/2750575643 conn(0x7f30f808c000 msgr2=0x7f30f80d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:51.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.499+0000 7f30fcd15640 1 -- 172.21.15.172:0/2750575643 shutdown_connections 2024-04-14T08:40:51.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.499+0000 7f30fcd15640 1 -- 172.21.15.172:0/2750575643 wait complete. 2024-04-14T08:40:51.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.500+0000 7f30fcd15640 1 Processor -- start 2024-04-14T08:40:51.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.500+0000 7f30fcd15640 1 -- start start 2024-04-14T08:40:51.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30fcd15640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e40039b0 0x7f30f81f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30fcd15640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30f81f7f40 0x7f30f815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30fcd15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f80c7c10 0x7f30f80c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30fcd15640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30f80b11e0 con 0x7f30f80c7c10 2024-04-14T08:40:51.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30fcd15640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f30f806f870 con 0x7f30e40039b0 2024-04-14T08:40:51.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30fcd15640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f30f815c800 con 0x7f30f81f7f40 2024-04-14T08:40:51.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30f6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f80c7c10 0x7f30f80c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30f6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f80c7c10 0x7f30f80c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54132/0 (socket says 172.21.15.172:54132) 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30f6d9d640 1 -- 172.21.15.172:0/4044918227 learned_addr learned my addr 172.21.15.172:0/4044918227 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30effff640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30f81f7f40 0x7f30f815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30f659c640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e40039b0 0x7f30f81f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30f6d9d640 1 -- 172.21.15.172:0/4044918227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30f81f7f40 msgr2=0x7f30f815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30f6d9d640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30f81f7f40 0x7f30f815add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30f6d9d640 1 -- 172.21.15.172:0/4044918227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e40039b0 msgr2=0x7f30f81f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30f6d9d640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e40039b0 0x7f30f81f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30f6d9d640 1 -- 172.21.15.172:0/4044918227 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f81f83c0 con 0x7f30f80c7c10 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30effff640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30f81f7f40 0x7f30f815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30f659c640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e40039b0 0x7f30f81f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30f6d9d640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f80c7c10 0x7f30f80c80a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f30f0000f80 tx=0x7f30f0007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:51.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30eeffd640 1 -- 172.21.15.172:0/4044918227 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f000ab20 con 0x7f30f80c7c10 2024-04-14T08:40:51.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30eeffd640 1 -- 172.21.15.172:0/4044918227 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f0019070 con 0x7f30f80c7c10 2024-04-14T08:40:51.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30fcd15640 1 -- 172.21.15.172:0/4044918227 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f80c85e0 con 0x7f30f80c7c10 2024-04-14T08:40:51.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30eeffd640 1 -- 172.21.15.172:0/4044918227 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f00133f0 con 0x7f30f80c7c10 2024-04-14T08:40:51.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.501+0000 7f30fcd15640 1 -- 172.21.15.172:0/4044918227 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f81f4870 con 0x7f30f80c7c10 2024-04-14T08:40:51.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.502+0000 7f30fcd15640 1 -- 172.21.15.172:0/4044918227 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f8075490 con 0x7f30f80c7c10 2024-04-14T08:40:51.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.503+0000 7f30eeffd640 1 -- 172.21.15.172:0/4044918227 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30f0013590 con 0x7f30f80c7c10 2024-04-14T08:40:51.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.503+0000 7f30eeffd640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30d806c8f0 0x7f30d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.503+0000 7f30eeffd640 1 -- 172.21.15.172:0/4044918227 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f30f008c380 con 0x7f30f80c7c10 2024-04-14T08:40:51.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.503+0000 7f30f659c640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30d806c8f0 0x7f30d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.504+0000 7f30f659c640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30d806c8f0 0x7f30d806edb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f30e8033000 tx=0x7f30e8009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:51.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.508+0000 7f30eeffd640 1 -- 172.21.15.172:0/4044918227 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f30f81f83c0 con 0x7f30f80c7c10 2024-04-14T08:40:51.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.688+0000 7f30fcd15640 1 -- 172.21.15.172:0/4044918227 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f30f808c510 con 0x7f30f80c7c10 2024-04-14T08:40:51.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.688+0000 7f30eeffd640 1 -- 172.21.15.172:0/4044918227 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f30f005a500 con 0x7f30f80c7c10 2024-04-14T08:40:51.689 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:40:51.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.690+0000 7f30fcd15640 1 -- 172.21.15.172:0/4044918227 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30d806c8f0 msgr2=0x7f30d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.690+0000 7f30fcd15640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30d806c8f0 0x7f30d806edb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f30e8033000 tx=0x7f30e8009030 comp rx=0 tx=0).stop 2024-04-14T08:40:51.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.690+0000 7f30fcd15640 1 -- 172.21.15.172:0/4044918227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f80c7c10 msgr2=0x7f30f80c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.690+0000 7f30fcd15640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f80c7c10 0x7f30f80c80a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f30f0000f80 tx=0x7f30f0007b00 comp rx=0 tx=0).stop 2024-04-14T08:40:51.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.690+0000 7f30fcd15640 1 -- 172.21.15.172:0/4044918227 shutdown_connections 2024-04-14T08:40:51.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.690+0000 7f30fcd15640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f30d806c8f0 0x7f30d806edb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.690+0000 7f30fcd15640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30f80c7c10 0x7f30f80c80a0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.690+0000 7f30fcd15640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30f81f7f40 0x7f30f815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.690+0000 7f30fcd15640 1 --2- 172.21.15.172:0/4044918227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e40039b0 0x7f30f81f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.690+0000 7f30fcd15640 1 -- 172.21.15.172:0/4044918227 >> 172.21.15.172:0/4044918227 conn(0x7f30f808c000 msgr2=0x7f30f80af720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:51.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.690+0000 7f30fcd15640 1 -- 172.21.15.172:0/4044918227 shutdown_connections 2024-04-14T08:40:51.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.690+0000 7f30fcd15640 1 -- 172.21.15.172:0/4044918227 wait complete. 2024-04-14T08:40:51.701 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:40:51.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.806+0000 7fe0f3555640 1 Processor -- start 2024-04-14T08:40:51.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.806+0000 7fe0f3555640 1 -- start start 2024-04-14T08:40:51.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.807+0000 7fe0f3555640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe0ec15c420 0x7fe0ec0b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.807+0000 7fe0f3555640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe0ec06f3e0 0x7fe0ec0d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.807+0000 7fe0f3555640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0ec0d0550 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.807+0000 7fe0f3555640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe0ec0d30a0 con 0x7fe0ec15c420 2024-04-14T08:40:51.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.807+0000 7fe0f3555640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe0ec06f870 con 0x7fe0ec06f3e0 2024-04-14T08:40:51.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.807+0000 7fe0f12f1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe0ec15c420 0x7fe0ec0b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.807+0000 7fe0f1af2640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe0ec0ceb00 0x7fe0ec0cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50418/0 (socket says 172.21.15.172:50418) 2024-04-14T08:40:51.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.807+0000 7fe0f12f1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe0ec15c420 0x7fe0ec0b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54142/0 (socket says 172.21.15.172:54142) 2024-04-14T08:40:51.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.807+0000 7fe0f1af2640 1 -- 172.21.15.172:0/1226646900 learned_addr learned my addr 172.21.15.172:0/1226646900 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:51.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.808+0000 7fe0f0af0640 1 --2- 172.21.15.172:0/1226646900 >> v2:172.21.15.177:3300/0 conn(0x7fe0ec06f3e0 0x7fe0ec0d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.808+0000 7fe0f12f1640 1 -- 172.21.15.172:0/1226646900 >> v2:172.21.15.177:3300/0 conn(0x7fe0ec06f3e0 msgr2=0x7fe0ec0d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:51.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.808+0000 7fe0f12f1640 1 --2- 172.21.15.172:0/1226646900 >> v2:172.21.15.177:3300/0 conn(0x7fe0ec06f3e0 0x7fe0ec0d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.808+0000 7fe0f12f1640 1 -- 172.21.15.172:0/1226646900 >> v1:172.21.15.172:6789/0 conn(0x7fe0ec0ceb00 legacy=0x7fe0ec0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.808+0000 7fe0f12f1640 1 -- 172.21.15.172:0/1226646900 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec0ccdd0 con 0x7fe0ec15c420 2024-04-14T08:40:51.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.808+0000 7fe0e3fff640 1 -- 172.21.15.172:0/1226646900 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2835711275 0 0) 0x7fe0ec0d0550 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.809+0000 7fe0f12f1640 1 --2- 172.21.15.172:0/1226646900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec15c420 0x7fe0ec0b1510 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0740b0 tx=0x7fe0e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1124176e6f30ee server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:51.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.809+0000 7fe0e3fff640 1 -- 172.21.15.172:0/1226646900 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8035020 con 0x7fe0ec15c420 2024-04-14T08:40:51.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.809+0000 7fe0e3fff640 1 -- 172.21.15.172:0/1226646900 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e8031830 con 0x7fe0ec15c420 2024-04-14T08:40:51.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.809+0000 7fe0e3fff640 1 -- 172.21.15.172:0/1226646900 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8031c00 con 0x7fe0ec15c420 2024-04-14T08:40:51.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.809+0000 7fe0f3555640 1 -- 172.21.15.172:0/1226646900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec15c420 msgr2=0x7fe0ec0b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.809+0000 7fe0f3555640 1 --2- 172.21.15.172:0/1226646900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec15c420 0x7fe0ec0b1510 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0740b0 tx=0x7fe0e802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.809+0000 7fe0f3555640 1 -- 172.21.15.172:0/1226646900 shutdown_connections 2024-04-14T08:40:51.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.809+0000 7fe0f3555640 1 --2- 172.21.15.172:0/1226646900 >> v2:172.21.15.177:3300/0 conn(0x7fe0ec06f3e0 0x7fe0ec0d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.809+0000 7fe0f3555640 1 --2- 172.21.15.172:0/1226646900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec15c420 0x7fe0ec0b1510 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.809+0000 7fe0f3555640 1 -- 172.21.15.172:0/1226646900 >> 172.21.15.172:0/1226646900 conn(0x7fe0ec08c000 msgr2=0x7fe0ec0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:51.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.809+0000 7fe0f3555640 1 -- 172.21.15.172:0/1226646900 shutdown_connections 2024-04-14T08:40:51.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.809+0000 7fe0f3555640 1 -- 172.21.15.172:0/1226646900 wait complete. 2024-04-14T08:40:51.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.810+0000 7fe0f3555640 1 Processor -- start 2024-04-14T08:40:51.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.810+0000 7fe0f3555640 1 -- start start 2024-04-14T08:40:51.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.811+0000 7fe0f3555640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec0ceb00 0x7fe0ec1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.811+0000 7fe0f3555640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ec15c420 0x7fe0ec06c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.811+0000 7fe0f3555640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0ec1f4b20 0x7fe0ec0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.811+0000 7fe0f3555640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec0cd070 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.811+0000 7fe0f3555640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec0d2dc0 con 0x7fe0ec1f4b20 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.811+0000 7fe0f3555640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe0ec0b11e0 con 0x7fe0ec15c420 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.811+0000 7fe0f12f1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec0ceb00 0x7fe0ec1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.811+0000 7fe0f12f1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec0ceb00 0x7fe0ec1591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54150/0 (socket says 172.21.15.172:54150) 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.811+0000 7fe0f12f1640 1 -- 172.21.15.172:0/3254649331 learned_addr learned my addr 172.21.15.172:0/3254649331 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.811+0000 7fe0f1af2640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0ec1f4b20 0x7fe0ec0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.811+0000 7fe0f0af0640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ec15c420 0x7fe0ec06c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.811+0000 7fe0f12f1640 1 -- 172.21.15.172:0/3254649331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ec15c420 msgr2=0x7fe0ec06c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.812+0000 7fe0f12f1640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ec15c420 0x7fe0ec06c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.812+0000 7fe0f12f1640 1 -- 172.21.15.172:0/3254649331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0ec1f4b20 msgr2=0x7fe0ec0b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:51.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.812+0000 7fe0f12f1640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0ec1f4b20 0x7fe0ec0b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.812+0000 7fe0f12f1640 1 -- 172.21.15.172:0/3254649331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec0c52f0 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.812+0000 7fe0f1af2640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0ec1f4b20 0x7fe0ec0b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.812+0000 7fe0f0af0640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ec15c420 0x7fe0ec06c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.812+0000 7fe0f12f1640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec0ceb00 0x7fe0ec1591d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8000c00 tx=0x7fe0e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.812+0000 7fe0e1ffb640 1 -- 172.21.15.172:0/3254649331 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8040070 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.812+0000 7fe0e1ffb640 1 -- 172.21.15.172:0/3254649331 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e8039a20 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.812+0000 7fe0f3555640 1 -- 172.21.15.172:0/3254649331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ec0b17a0 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.812+0000 7fe0e1ffb640 1 -- 172.21.15.172:0/3254649331 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8035040 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.812+0000 7fe0f3555640 1 -- 172.21.15.172:0/3254649331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ec0c6a80 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.813+0000 7fe0f3555640 1 -- 172.21.15.172:0/3254649331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0ec075490 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.813+0000 7fe0e1ffb640 1 -- 172.21.15.172:0/3254649331 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0e8038020 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.814+0000 7fe0e1ffb640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe0d006c920 0x7fe0d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.814+0000 7fe0e1ffb640 1 -- 172.21.15.172:0/3254649331 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fe0e80ad840 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.814+0000 7fe0f0af0640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe0d006c920 0x7fe0d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:51.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.814+0000 7fe0f0af0640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe0d006c920 0x7fe0d006ede0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec159710 tx=0x7fe0dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:51.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.818+0000 7fe0e1ffb640 1 -- 172.21.15.172:0/3254649331 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe0e807b9c0 con 0x7fe0ec0ceb00 2024-04-14T08:40:51.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.999+0000 7fe0f3555640 1 -- 172.21.15.172:0/3254649331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe0ec0d2820 con 0x7fe0ec0ceb00 2024-04-14T08:40:52.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:51.999+0000 7fe0e1ffb640 1 -- 172.21.15.172:0/3254649331 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fe0e807b2c0 con 0x7fe0ec0ceb00 2024-04-14T08:40:52.000 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:40:52.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.001+0000 7fe0f3555640 1 -- 172.21.15.172:0/3254649331 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe0d006c920 msgr2=0x7fe0d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.001+0000 7fe0f3555640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe0d006c920 0x7fe0d006ede0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec159710 tx=0x7fe0dc008040 comp rx=0 tx=0).stop 2024-04-14T08:40:52.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.001+0000 7fe0f3555640 1 -- 172.21.15.172:0/3254649331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec0ceb00 msgr2=0x7fe0ec1591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.001+0000 7fe0f3555640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec0ceb00 0x7fe0ec1591d0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8000c00 tx=0x7fe0e80046c0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.001+0000 7fe0f3555640 1 -- 172.21.15.172:0/3254649331 shutdown_connections 2024-04-14T08:40:52.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.001+0000 7fe0f3555640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe0d006c920 0x7fe0d006ede0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.001+0000 7fe0f3555640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0ec1f4b20 0x7fe0ec0b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.001+0000 7fe0f3555640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0ec15c420 0x7fe0ec06c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.001+0000 7fe0f3555640 1 --2- 172.21.15.172:0/3254649331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec0ceb00 0x7fe0ec1591d0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.001+0000 7fe0f3555640 1 -- 172.21.15.172:0/3254649331 >> 172.21.15.172:0/3254649331 conn(0x7fe0ec08c000 msgr2=0x7fe0ec0b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:52.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.001+0000 7fe0f3555640 1 -- 172.21.15.172:0/3254649331 shutdown_connections 2024-04-14T08:40:52.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.001+0000 7fe0f3555640 1 -- 172.21.15.172:0/3254649331 wait complete. 2024-04-14T08:40:52.013 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:40:52.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.115+0000 7f616ccde640 1 Processor -- start 2024-04-14T08:40:52.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.115+0000 7f616ccde640 1 -- start start 2024-04-14T08:40:52.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f616ccde640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61680b7a60 0x7f61680ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f616ccde640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f616808c4c0 0x7f61680cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f616ccde640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f61680cd410 0x7f61680cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f616ccde640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f61680d2b10 con 0x7f61680cd410 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f616ccde640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f61680d0ad0 con 0x7f616808c4c0 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f616ccde640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f61680d0db0 con 0x7f61680b7a60 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f6166d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f61680cd410 0x7f61680cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f6166d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f61680cd410 0x7f61680cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54160/0 (socket says 172.21.15.172:54160) 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f6166d9d640 1 -- 172.21.15.172:0/3567108245 learned_addr learned my addr 172.21.15.172:0/3567108245 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f6165d9b640 1 --2- 172.21.15.172:0/3567108245 >> v2:172.21.15.177:3300/0 conn(0x7f616808c4c0 0x7f61680cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f616659c640 1 --2- 172.21.15.172:0/3567108245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61680b7a60 0x7f61680ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f6166d9d640 1 -- 172.21.15.172:0/3567108245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61680b7a60 msgr2=0x7f61680ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f6166d9d640 1 --2- 172.21.15.172:0/3567108245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61680b7a60 0x7f61680ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f6166d9d640 1 -- 172.21.15.172:0/3567108245 >> v2:172.21.15.177:3300/0 conn(0x7f616808c4c0 msgr2=0x7f61680cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:52.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f6166d9d640 1 --2- 172.21.15.172:0/3567108245 >> v2:172.21.15.177:3300/0 conn(0x7f616808c4c0 0x7f61680cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.116+0000 7f6166d9d640 1 -- 172.21.15.172:0/3567108245 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61680d1090 con 0x7f61680cd410 2024-04-14T08:40:52.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.117+0000 7f6166d9d640 1 --2- 172.21.15.172:0/3567108245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61680cd410 0x7f61680cd8a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f616806c500 tx=0x7f616002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec844a7c107eceb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:52.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.117+0000 7f616559a640 1 -- 172.21.15.172:0/3567108245 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616002c9e0 con 0x7f61680cd410 2024-04-14T08:40:52.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.117+0000 7f616559a640 1 -- 172.21.15.172:0/3567108245 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616002cb80 con 0x7f61680cd410 2024-04-14T08:40:52.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.117+0000 7f616559a640 1 -- 172.21.15.172:0/3567108245 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61600343f0 con 0x7f61680cd410 2024-04-14T08:40:52.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.117+0000 7f616ccde640 1 -- 172.21.15.172:0/3567108245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61680cd410 msgr2=0x7f61680cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.118+0000 7f616ccde640 1 --2- 172.21.15.172:0/3567108245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61680cd410 0x7f61680cd8a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f616806c500 tx=0x7f616002be40 comp rx=0 tx=0).stop 2024-04-14T08:40:52.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.118+0000 7f616ccde640 1 -- 172.21.15.172:0/3567108245 shutdown_connections 2024-04-14T08:40:52.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.118+0000 7f616ccde640 1 --2- 172.21.15.172:0/3567108245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61680cd410 0x7f61680cd8a0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.118+0000 7f616ccde640 1 --2- 172.21.15.172:0/3567108245 >> v2:172.21.15.177:3300/0 conn(0x7f616808c4c0 0x7f61680cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.118+0000 7f616ccde640 1 --2- 172.21.15.172:0/3567108245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61680b7a60 0x7f61680ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.118+0000 7f616ccde640 1 -- 172.21.15.172:0/3567108245 >> 172.21.15.172:0/3567108245 conn(0x7f616808c000 msgr2=0x7f61680d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:52.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.118+0000 7f616ccde640 1 -- 172.21.15.172:0/3567108245 shutdown_connections 2024-04-14T08:40:52.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.118+0000 7f616ccde640 1 -- 172.21.15.172:0/3567108245 wait complete. 2024-04-14T08:40:52.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.119+0000 7f616ccde640 1 Processor -- start 2024-04-14T08:40:52.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.119+0000 7f616ccde640 1 -- start start 2024-04-14T08:40:52.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f616ccde640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f616808c4c0 0x7f61681f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f616ccde640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61680b7a60 0x7f616809e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f616ccde640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61680cd410 0x7f6168157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f616ccde640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61680d2750 con 0x7f61680cd410 2024-04-14T08:40:52.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f616ccde640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61680b7e40 con 0x7f616808c4c0 2024-04-14T08:40:52.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f616ccde640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f61680d28d0 con 0x7f61680b7a60 2024-04-14T08:40:52.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6166d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61680cd410 0x7f6168157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6166d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61680cd410 0x7f6168157530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54170/0 (socket says 172.21.15.172:54170) 2024-04-14T08:40:52.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6166d9d640 1 -- 172.21.15.172:0/1027672437 learned_addr learned my addr 172.21.15.172:0/1027672437 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:52.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6165d9b640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61680b7a60 0x7f616809e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f616659c640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f616808c4c0 0x7f61681f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6166d9d640 1 -- 172.21.15.172:0/1027672437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61680b7a60 msgr2=0x7f616809e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6166d9d640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61680b7a60 0x7f616809e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6166d9d640 1 -- 172.21.15.172:0/1027672437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f616808c4c0 msgr2=0x7f61681f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6166d9d640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f616808c4c0 0x7f61681f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6166d9d640 1 -- 172.21.15.172:0/1027672437 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616809fce0 con 0x7f61680cd410 2024-04-14T08:40:52.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6165d9b640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61680b7a60 0x7f616809e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:52.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f616659c640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f616808c4c0 0x7f61681f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:52.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6166d9d640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61680cd410 0x7f6168157530 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6160009f40 tx=0x7f6160004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:52.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6156ffd640 1 -- 172.21.15.172:0/1027672437 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160043020 con 0x7f61680cd410 2024-04-14T08:40:52.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6156ffd640 1 -- 172.21.15.172:0/1027672437 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616003e450 con 0x7f61680cd410 2024-04-14T08:40:52.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f6156ffd640 1 -- 172.21.15.172:0/1027672437 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160009510 con 0x7f61680cd410 2024-04-14T08:40:52.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.120+0000 7f616ccde640 1 -- 172.21.15.172:0/1027672437 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616809e970 con 0x7f61680cd410 2024-04-14T08:40:52.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.121+0000 7f616ccde640 1 -- 172.21.15.172:0/1027672437 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616809f200 con 0x7f61680cd410 2024-04-14T08:40:52.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.122+0000 7f616ccde640 1 -- 172.21.15.172:0/1027672437 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6168075490 con 0x7f61680cd410 2024-04-14T08:40:52.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.122+0000 7f6156ffd640 1 -- 172.21.15.172:0/1027672437 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6160046020 con 0x7f61680cd410 2024-04-14T08:40:52.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.123+0000 7f6156ffd640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f614806c920 0x7f614806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.123+0000 7f6156ffd640 1 -- 172.21.15.172:0/1027672437 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f61600ae3d0 con 0x7f61680cd410 2024-04-14T08:40:52.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.123+0000 7f616659c640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f614806c920 0x7f614806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.123+0000 7f616659c640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f614806c920 0x7f614806ede0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6158006fd0 tx=0x7f6158008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:52.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.127+0000 7f6156ffd640 1 -- 172.21.15.172:0/1027672437 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f616004d070 con 0x7f61680cd410 2024-04-14T08:40:52.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.385+0000 7f616ccde640 1 -- 172.21.15.172:0/1027672437 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61680c8ac0 con 0x7f61680cd410 2024-04-14T08:40:52.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.385+0000 7f6156ffd640 1 -- 172.21.15.172:0/1027672437 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5196 (secure 0 0 0) 0x7f616007c550 con 0x7f61680cd410 2024-04-14T08:40:52.386 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:52.386 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:40:14.122456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8013":{"gid":8013,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.172:6837/3091771014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3091771014},{"type":"v1","addr":"172.21.15.172:6837","nonce":3091771014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8013,"qdb_cluster":[8013]},"id":6}]} 2024-04-14T08:40:52.387 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 67 2024-04-14T08:40:52.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.387+0000 7f616ccde640 1 -- 172.21.15.172:0/1027672437 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f614806c920 msgr2=0x7f614806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.387+0000 7f616ccde640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f614806c920 0x7f614806ede0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6158006fd0 tx=0x7f6158008040 comp rx=0 tx=0).stop 2024-04-14T08:40:52.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.387+0000 7f616ccde640 1 -- 172.21.15.172:0/1027672437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61680cd410 msgr2=0x7f6168157530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.387+0000 7f616ccde640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61680cd410 0x7f6168157530 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6160009f40 tx=0x7f6160004a40 comp rx=0 tx=0).stop 2024-04-14T08:40:52.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.387+0000 7f616ccde640 1 -- 172.21.15.172:0/1027672437 shutdown_connections 2024-04-14T08:40:52.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.387+0000 7f616ccde640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f614806c920 0x7f614806ede0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.387+0000 7f616ccde640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61680cd410 0x7f6168157530 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.387+0000 7f616ccde640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61680b7a60 0x7f616809e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.387+0000 7f616ccde640 1 --2- 172.21.15.172:0/1027672437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f616808c4c0 0x7f61681f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.387+0000 7f616ccde640 1 -- 172.21.15.172:0/1027672437 >> 172.21.15.172:0/1027672437 conn(0x7f616808c000 msgr2=0x7f61680b52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:52.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.388+0000 7f616ccde640 1 -- 172.21.15.172:0/1027672437 shutdown_connections 2024-04-14T08:40:52.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.388+0000 7f616ccde640 1 -- 172.21.15.172:0/1027672437 wait complete. 2024-04-14T08:40:52.400 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:40:52.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.502+0000 7ff2696c0640 1 Processor -- start 2024-04-14T08:40:52.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.502+0000 7ff2696c0640 1 -- start start 2024-04-14T08:40:52.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff2696c0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff26406fdf0 0x7ff2640b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff2696c0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff264070270 con 0x7ff26415e890 2024-04-14T08:40:52.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff2696c0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2640b4540 con 0x7ff26406b520 2024-04-14T08:40:52.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff2696c0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff2640b4310 con 0x7ff26406fdf0 2024-04-14T08:40:52.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff2637fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff26415e890 0x7ff26415ed20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50430/0 (socket says 172.21.15.172:50430) 2024-04-14T08:40:52.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff2637fe640 1 -- 172.21.15.172:0/2044739236 learned_addr learned my addr 172.21.15.172:0/2044739236 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:52.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff25bfff640 1 --2- 172.21.15.172:0/2044739236 >> v2:172.21.15.177:3300/0 conn(0x7ff26406fdf0 0x7ff2640b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff2627fc640 1 -- 172.21.15.172:0/2044739236 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3891437431 0 0) 0x7ff264070270 con 0x7ff26415e890 2024-04-14T08:40:52.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff2627fc640 1 -- 172.21.15.172:0/2044739236 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff250003520 con 0x7ff26415e890 2024-04-14T08:40:52.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff25bfff640 1 -- 172.21.15.172:0/2044739236 >> v1:172.21.15.177:6789/0 conn(0x7ff26406b520 legacy=0x7ff26406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff25bfff640 1 -- 172.21.15.172:0/2044739236 >> v1:172.21.15.172:6789/0 conn(0x7ff26415e890 legacy=0x7ff26415ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff2627fc640 1 -- 172.21.15.172:0/2044739236 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1173411103 0 0) 0x7ff250003520 con 0x7ff26415e890 2024-04-14T08:40:52.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.503+0000 7ff25bfff640 1 -- 172.21.15.172:0/2044739236 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff264072450 con 0x7ff26406fdf0 2024-04-14T08:40:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.504+0000 7ff25bfff640 1 --2- 172.21.15.172:0/2044739236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff26406fdf0 0x7ff2640b3dd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0097a0 tx=0x7ff24c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19924a9e40790933 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.504+0000 7ff2627fc640 1 -- 172.21.15.172:0/2044739236 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c038070 con 0x7ff26406fdf0 2024-04-14T08:40:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.504+0000 7ff2627fc640 1 -- 172.21.15.172:0/2044739236 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff24c031500 con 0x7ff26406fdf0 2024-04-14T08:40:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.504+0000 7ff2627fc640 1 -- 172.21.15.172:0/2044739236 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0019f0 con 0x7ff26406fdf0 2024-04-14T08:40:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.505+0000 7ff2696c0640 1 -- 172.21.15.172:0/2044739236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff26406fdf0 msgr2=0x7ff2640b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.505+0000 7ff2696c0640 1 --2- 172.21.15.172:0/2044739236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff26406fdf0 0x7ff2640b3dd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0097a0 tx=0x7ff24c02b820 comp rx=0 tx=0).stop 2024-04-14T08:40:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.505+0000 7ff2696c0640 1 -- 172.21.15.172:0/2044739236 shutdown_connections 2024-04-14T08:40:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.505+0000 7ff2696c0640 1 --2- 172.21.15.172:0/2044739236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff26406fdf0 0x7ff2640b3dd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.505+0000 7ff2696c0640 1 -- 172.21.15.172:0/2044739236 >> 172.21.15.172:0/2044739236 conn(0x7ff2640d0f20 msgr2=0x7ff2640d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.505+0000 7ff2696c0640 1 -- 172.21.15.172:0/2044739236 shutdown_connections 2024-04-14T08:40:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.505+0000 7ff2696c0640 1 -- 172.21.15.172:0/2044739236 wait complete. 2024-04-14T08:40:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.506+0000 7ff2696c0640 1 Processor -- start 2024-04-14T08:40:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.506+0000 7ff2696c0640 1 -- start start 2024-04-14T08:40:52.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff2696c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff26406b520 0x7ff26413d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff2696c0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff26406fdf0 0x7ff26413a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff2696c0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff26415e890 0x7ff2641ea110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff2696c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff264072a10 con 0x7ff26406b520 2024-04-14T08:40:52.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff2696c0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff264072d80 con 0x7ff26406fdf0 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff2696c0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff264073210 con 0x7ff26415e890 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff262ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff26406b520 0x7ff26413d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff262ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff26406b520 0x7ff26413d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54178/0 (socket says 172.21.15.172:54178) 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff262ffd640 1 -- 172.21.15.172:0/1908095604 learned_addr learned my addr 172.21.15.172:0/1908095604 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff25bfff640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff26406fdf0 0x7ff26413a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff2637fe640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff26415e890 0x7ff2641ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff262ffd640 1 -- 172.21.15.172:0/1908095604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff26415e890 msgr2=0x7ff2641ea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff262ffd640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff26415e890 0x7ff2641ea110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff262ffd640 1 -- 172.21.15.172:0/1908095604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff26406fdf0 msgr2=0x7ff26413a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff262ffd640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff26406fdf0 0x7ff26413a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff262ffd640 1 -- 172.21.15.172:0/1908095604 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26413da80 con 0x7ff26406b520 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.507+0000 7ff2637fe640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff26415e890 0x7ff2641ea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:52.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.508+0000 7ff262ffd640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff26406b520 0x7ff26413d540 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c400 tx=0x7ff25400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:52.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.508+0000 7ff25b7fe640 1 -- 172.21.15.172:0/1908095604 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254017070 con 0x7ff26406b520 2024-04-14T08:40:52.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.508+0000 7ff25b7fe640 1 -- 172.21.15.172:0/1908095604 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2540052d0 con 0x7ff26406b520 2024-04-14T08:40:52.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.508+0000 7ff2696c0640 1 -- 172.21.15.172:0/1908095604 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26413a790 con 0x7ff26406b520 2024-04-14T08:40:52.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.508+0000 7ff25b7fe640 1 -- 172.21.15.172:0/1908095604 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254003830 con 0x7ff26406b520 2024-04-14T08:40:52.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.508+0000 7ff2696c0640 1 -- 172.21.15.172:0/1908095604 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff264150ac0 con 0x7ff26406b520 2024-04-14T08:40:52.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.509+0000 7ff2696c0640 1 -- 172.21.15.172:0/1908095604 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff264068520 con 0x7ff26406b520 2024-04-14T08:40:52.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.509+0000 7ff25b7fe640 1 -- 172.21.15.172:0/1908095604 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff25401a070 con 0x7ff26406b520 2024-04-14T08:40:52.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.510+0000 7ff25b7fe640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff24406c8f0 0x7ff24406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.510+0000 7ff25b7fe640 1 -- 172.21.15.172:0/1908095604 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7ff25408b070 con 0x7ff26406b520 2024-04-14T08:40:52.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.510+0000 7ff25bfff640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff24406c8f0 0x7ff24406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.510+0000 7ff25bfff640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff24406c8f0 0x7ff24406edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff24c035000 tx=0x7ff24c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:52.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.514+0000 7ff25b7fe640 1 -- 172.21.15.172:0/1908095604 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff26413da80 con 0x7ff26406b520 2024-04-14T08:40:52.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.697+0000 7ff2696c0640 1 -- 172.21.15.172:0/1908095604 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff26415e2d0 con 0x7ff26406b520 2024-04-14T08:40:52.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.698+0000 7ff25b7fe640 1 -- 172.21.15.172:0/1908095604 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+17401 (secure 0 0 0) 0x7ff254058ba0 con 0x7ff26406b520 2024-04-14T08:40:52.698 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:52.699 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":90,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:40:10.107688+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:38:42.790996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-14T08:38:43.517208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/1386685680":"2024-04-15T08:40:10.107666+0000","172.21.15.172:6835/1386685680":"2024-04-15T08:40:10.107666+0000","172.21.15.177:6835/1124731311":"2024-04-15T08:38:58.388334+0000","172.21.15.177:6833/1124731311":"2024-04-15T08:38:58.388334+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:40:52.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.700+0000 7ff2696c0640 1 -- 172.21.15.172:0/1908095604 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff24406c8f0 msgr2=0x7ff24406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.700+0000 7ff2696c0640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff24406c8f0 0x7ff24406edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff24c035000 tx=0x7ff24c008030 comp rx=0 tx=0).stop 2024-04-14T08:40:52.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.700+0000 7ff2696c0640 1 -- 172.21.15.172:0/1908095604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff26406b520 msgr2=0x7ff26413d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.700+0000 7ff2696c0640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff26406b520 0x7ff26413d540 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff25400c400 tx=0x7ff25400c8c0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.700+0000 7ff2696c0640 1 -- 172.21.15.172:0/1908095604 shutdown_connections 2024-04-14T08:40:52.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.700+0000 7ff2696c0640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff24406c8f0 0x7ff24406edb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.700+0000 7ff2696c0640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff26415e890 0x7ff2641ea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.700+0000 7ff2696c0640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff26406fdf0 0x7ff26413a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.700+0000 7ff2696c0640 1 --2- 172.21.15.172:0/1908095604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff26406b520 0x7ff26413d540 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.700+0000 7ff2696c0640 1 -- 172.21.15.172:0/1908095604 >> 172.21.15.172:0/1908095604 conn(0x7ff2640d0f20 msgr2=0x7ff2640addb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:52.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.700+0000 7ff2696c0640 1 -- 172.21.15.172:0/1908095604 shutdown_connections 2024-04-14T08:40:52.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.700+0000 7ff2696c0640 1 -- 172.21.15.172:0/1908095604 wait complete. 2024-04-14T08:40:52.712 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-14T08:40:52.712 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:40:52.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.817+0000 7f4ce971e640 1 Processor -- start 2024-04-14T08:40:52.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.817+0000 7f4ce971e640 1 -- start start 2024-04-14T08:40:52.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4ce971e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4ce40b21d0 0x7f4ce415c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4ce971e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ce415c5a0 0x7f4ce40b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4ce971e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ce40b3700 con 0x7f4ce40b1df0 2024-04-14T08:40:52.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4ce971e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f4ce40b39e0 con 0x7f4ce40b21d0 2024-04-14T08:40:52.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4ce971e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce40b3cc0 con 0x7f4ce415c5a0 2024-04-14T08:40:52.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4ce2ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4ce40b1df0 0x7f4ce40b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50432/0 (socket says 172.21.15.172:50432) 2024-04-14T08:40:52.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4ce2ffd640 1 -- 172.21.15.172:0/2256694078 learned_addr learned my addr 172.21.15.172:0/2256694078 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:52.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4ce37fe640 1 --2- 172.21.15.172:0/2256694078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ce415c5a0 0x7f4ce40b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4cdbfff640 1 --2- 172.21.15.172:0/2256694078 >> v2:172.21.15.177:3300/0 conn(0x7f4ce40b21d0 0x7f4ce415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4ce27fc640 1 -- 172.21.15.172:0/2256694078 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3452985471 0 0) 0x7f4ce40b3700 con 0x7f4ce40b1df0 2024-04-14T08:40:52.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4ce27fc640 1 -- 172.21.15.172:0/2256694078 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cd0003520 con 0x7f4ce40b1df0 2024-04-14T08:40:52.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4cdbfff640 1 -- 172.21.15.172:0/2256694078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ce415c5a0 msgr2=0x7f4ce40b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:52.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4cdbfff640 1 --2- 172.21.15.172:0/2256694078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ce415c5a0 0x7f4ce40b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4cdbfff640 1 -- 172.21.15.172:0/2256694078 >> v1:172.21.15.172:6789/0 conn(0x7f4ce40b1df0 legacy=0x7f4ce40b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.818+0000 7f4cdbfff640 1 -- 172.21.15.172:0/2256694078 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce415bb40 con 0x7f4ce40b21d0 2024-04-14T08:40:52.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.819+0000 7f4cdbfff640 1 --2- 172.21.15.172:0/2256694078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce40b21d0 0x7f4ce415c060 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0097a0 tx=0x7f4ccc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2912ba3674f12b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:52.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.819+0000 7f4ce27fc640 1 -- 172.21.15.172:0/2256694078 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc035020 con 0x7f4ce40b21d0 2024-04-14T08:40:52.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.819+0000 7f4ce27fc640 1 -- 172.21.15.172:0/2256694078 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ccc031830 con 0x7f4ce40b21d0 2024-04-14T08:40:52.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.819+0000 7f4ce27fc640 1 -- 172.21.15.172:0/2256694078 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc031c00 con 0x7f4ce40b21d0 2024-04-14T08:40:52.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.820+0000 7f4ce971e640 1 -- 172.21.15.172:0/2256694078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce40b21d0 msgr2=0x7f4ce415c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.820+0000 7f4ce971e640 1 --2- 172.21.15.172:0/2256694078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce40b21d0 0x7f4ce415c060 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0097a0 tx=0x7f4ccc02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.820+0000 7f4ce971e640 1 -- 172.21.15.172:0/2256694078 shutdown_connections 2024-04-14T08:40:52.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.820+0000 7f4ce971e640 1 --2- 172.21.15.172:0/2256694078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ce415c5a0 0x7f4ce40b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.820+0000 7f4ce971e640 1 --2- 172.21.15.172:0/2256694078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce40b21d0 0x7f4ce415c060 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.820+0000 7f4ce971e640 1 -- 172.21.15.172:0/2256694078 >> 172.21.15.172:0/2256694078 conn(0x7f4ce406ec30 msgr2=0x7f4ce406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:52.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.820+0000 7f4ce971e640 1 -- 172.21.15.172:0/2256694078 shutdown_connections 2024-04-14T08:40:52.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.820+0000 7f4ce971e640 1 -- 172.21.15.172:0/2256694078 wait complete. 2024-04-14T08:40:52.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.821+0000 7f4ce971e640 1 Processor -- start 2024-04-14T08:40:52.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.821+0000 7f4ce971e640 1 -- start start 2024-04-14T08:40:52.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4ce971e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ce40b1df0 0x7f4ce4141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4ce971e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce40b21d0 0x7f4ce40adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4ce971e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce413f410 0x7f4ce413f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4ce971e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce40b4700 con 0x7f4ce40b21d0 2024-04-14T08:40:52.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4ce971e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce40b4580 con 0x7f4ce413f410 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4ce971e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce40b4880 con 0x7f4ce40b1df0 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4cdbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce40b21d0 0x7f4ce40adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4cdbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce40b21d0 0x7f4ce40adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54190/0 (socket says 172.21.15.172:54190) 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4cdbfff640 1 -- 172.21.15.172:0/3416979139 learned_addr learned my addr 172.21.15.172:0/3416979139 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4ce2ffd640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ce40b1df0 0x7f4ce4141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4ce37fe640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce413f410 0x7f4ce413f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4cdbfff640 1 -- 172.21.15.172:0/3416979139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ce40b1df0 msgr2=0x7f4ce4141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4cdbfff640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ce40b1df0 0x7f4ce4141190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4cdbfff640 1 -- 172.21.15.172:0/3416979139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce413f410 msgr2=0x7f4ce413f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4cdbfff640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce413f410 0x7f4ce413f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4cdbfff640 1 -- 172.21.15.172:0/3416979139 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce40ced20 con 0x7f4ce40b21d0 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.822+0000 7f4cdbfff640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce40b21d0 0x7f4ce40adb50 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc02bee0 tx=0x7f4ccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.823+0000 7f4cdb7fe640 1 -- 172.21.15.172:0/3416979139 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc035040 con 0x7f4ce40b21d0 2024-04-14T08:40:52.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.823+0000 7f4cdb7fe640 1 -- 172.21.15.172:0/3416979139 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ccc03b920 con 0x7f4ce40b21d0 2024-04-14T08:40:52.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.823+0000 7f4ce971e640 1 -- 172.21.15.172:0/3416979139 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce40cf000 con 0x7f4ce40b21d0 2024-04-14T08:40:52.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.823+0000 7f4cdb7fe640 1 -- 172.21.15.172:0/3416979139 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc03bd10 con 0x7f4ce40b21d0 2024-04-14T08:40:52.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.823+0000 7f4ce971e640 1 -- 172.21.15.172:0/3416979139 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce40cf5c0 con 0x7f4ce40b21d0 2024-04-14T08:40:52.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.824+0000 7f4ce971e640 1 -- 172.21.15.172:0/3416979139 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce4068520 con 0x7f4ce40b21d0 2024-04-14T08:40:52.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.824+0000 7f4cdb7fe640 1 -- 172.21.15.172:0/3416979139 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ccc038020 con 0x7f4ce40b21d0 2024-04-14T08:40:52.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.824+0000 7f4cdb7fe640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4cc406c8f0 0x7f4cc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:52.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.825+0000 7f4cdb7fe640 1 -- 172.21.15.172:0/3416979139 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f4ccc0adec0 con 0x7f4ce40b21d0 2024-04-14T08:40:52.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.825+0000 7f4ce2ffd640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4cc406c8f0 0x7f4cc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:52.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.825+0000 7f4ce2ffd640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4cc406c8f0 0x7f4cc406edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4005880 tx=0x7f4cd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:52.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:52.829+0000 7f4cdb7fe640 1 -- 172.21.15.172:0/3416979139 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4ccc07c040 con 0x7f4ce40b21d0 2024-04-14T08:40:53.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.042+0000 7f4ce971e640 1 -- 172.21.15.172:0/3416979139 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ce40cc240 con 0x7f4ce40b21d0 2024-04-14T08:40:53.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.042+0000 7f4cdb7fe640 1 -- 172.21.15.172:0/3416979139 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5196 (secure 0 0 0) 0x7f4ccc07b940 con 0x7f4ce40b21d0 2024-04-14T08:40:53.043 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:53.043 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:40:14.122456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8013":{"gid":8013,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.172:6837/3091771014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3091771014},{"type":"v1","addr":"172.21.15.172:6837","nonce":3091771014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8013,"qdb_cluster":[8013]},"id":6}]} 2024-04-14T08:40:53.043 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 67 2024-04-14T08:40:53.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.044+0000 7f4ce971e640 1 -- 172.21.15.172:0/3416979139 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4cc406c8f0 msgr2=0x7f4cc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.044+0000 7f4ce971e640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4cc406c8f0 0x7f4cc406edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4005880 tx=0x7f4cd4009040 comp rx=0 tx=0).stop 2024-04-14T08:40:53.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.044+0000 7f4ce971e640 1 -- 172.21.15.172:0/3416979139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce40b21d0 msgr2=0x7f4ce40adb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.044+0000 7f4ce971e640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce40b21d0 0x7f4ce40adb50 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc02bee0 tx=0x7f4ccc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.044+0000 7f4ce971e640 1 -- 172.21.15.172:0/3416979139 shutdown_connections 2024-04-14T08:40:53.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.044+0000 7f4ce971e640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4cc406c8f0 0x7f4cc406edb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.044+0000 7f4ce971e640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce413f410 0x7f4ce413f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.044+0000 7f4ce971e640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ce40b21d0 0x7f4ce40adb50 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.044+0000 7f4ce971e640 1 --2- 172.21.15.172:0/3416979139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ce40b1df0 0x7f4ce4141190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.044+0000 7f4ce971e640 1 -- 172.21.15.172:0/3416979139 >> 172.21.15.172:0/3416979139 conn(0x7f4ce406ec30 msgr2=0x7f4ce40ae990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:53.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.044+0000 7f4ce971e640 1 -- 172.21.15.172:0/3416979139 shutdown_connections 2024-04-14T08:40:53.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.044+0000 7f4ce971e640 1 -- 172.21.15.172:0/3416979139 wait complete. 2024-04-14T08:40:53.056 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:40:53.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.157+0000 7f855ab15640 1 Processor -- start 2024-04-14T08:40:53.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.157+0000 7f855ab15640 1 -- start start 2024-04-14T08:40:53.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.158+0000 7f855ab15640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f855408c4c0 0x7f855406d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.158+0000 7f855ab15640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85540d2b10 con 0x7f85540cefd0 2024-04-14T08:40:53.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.158+0000 7f855ab15640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85540ce420 con 0x7f85540b7a60 2024-04-14T08:40:53.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.158+0000 7f855ab15640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f85540d0ad0 con 0x7f855408c4c0 2024-04-14T08:40:53.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.158+0000 7f85590b2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f855408c4c0 0x7f855406d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.158+0000 7f85590b2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f855408c4c0 0x7f855406d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54192/0 (socket says 172.21.15.172:54192) 2024-04-14T08:40:53.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.158+0000 7f854bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f85540cefd0 0x7f855406cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50440/0 (socket says 172.21.15.172:50440) 2024-04-14T08:40:53.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.158+0000 7f85590b2640 1 -- 172.21.15.172:0/3636810416 learned_addr learned my addr 172.21.15.172:0/3636810416 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:53.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.158+0000 7f85590b2640 1 -- 172.21.15.172:0/3636810416 >> v1:172.21.15.177:6789/0 conn(0x7f85540b7a60 legacy=0x7f85540ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:53.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.158+0000 7f85590b2640 1 -- 172.21.15.172:0/3636810416 >> v1:172.21.15.172:6789/0 conn(0x7f85540cefd0 legacy=0x7f855406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.158+0000 7f854b7fe640 1 -- 172.21.15.172:0/3636810416 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 426646772 0 0) 0x7f85540d2b10 con 0x7f85540cefd0 2024-04-14T08:40:53.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.158+0000 7f85590b2640 1 -- 172.21.15.172:0/3636810416 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85540d0db0 con 0x7f855408c4c0 2024-04-14T08:40:53.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.159+0000 7f85590b2640 1 --2- 172.21.15.172:0/3636810416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f855408c4c0 0x7f855406d340 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f854c009e50 tx=0x7f854c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc84a8e380c89e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:53.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.159+0000 7f854b7fe640 1 -- 172.21.15.172:0/3636810416 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c02cb20 con 0x7f855408c4c0 2024-04-14T08:40:53.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.159+0000 7f854b7fe640 1 -- 172.21.15.172:0/3636810416 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854c02ccc0 con 0x7f855408c4c0 2024-04-14T08:40:53.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.159+0000 7f854b7fe640 1 -- 172.21.15.172:0/3636810416 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c034610 con 0x7f855408c4c0 2024-04-14T08:40:53.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.160+0000 7f855ab15640 1 -- 172.21.15.172:0/3636810416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f855408c4c0 msgr2=0x7f855406d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.160+0000 7f855ab15640 1 --2- 172.21.15.172:0/3636810416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f855408c4c0 0x7f855406d340 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f854c009e50 tx=0x7f854c02bf80 comp rx=0 tx=0).stop 2024-04-14T08:40:53.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.160+0000 7f855ab15640 1 -- 172.21.15.172:0/3636810416 shutdown_connections 2024-04-14T08:40:53.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.160+0000 7f855ab15640 1 --2- 172.21.15.172:0/3636810416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f855408c4c0 0x7f855406d340 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.160+0000 7f855ab15640 1 -- 172.21.15.172:0/3636810416 >> 172.21.15.172:0/3636810416 conn(0x7f855408c000 msgr2=0x7f85540d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:53.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.160+0000 7f855ab15640 1 -- 172.21.15.172:0/3636810416 shutdown_connections 2024-04-14T08:40:53.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.160+0000 7f855ab15640 1 -- 172.21.15.172:0/3636810416 wait complete. 2024-04-14T08:40:53.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.161+0000 7f855ab15640 1 Processor -- start 2024-04-14T08:40:53.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.161+0000 7f855ab15640 1 -- start start 2024-04-14T08:40:53.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f855ab15640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855408c4c0 0x7f855415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f855ab15640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85540b7a60 0x7f855415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f855ab15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85540cefd0 0x7f855409e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f855ab15640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85540b7e40 con 0x7f85540cefd0 2024-04-14T08:40:53.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f855ab15640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f85540b4e90 con 0x7f85540b7a60 2024-04-14T08:40:53.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f855ab15640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f85540d2750 con 0x7f855408c4c0 2024-04-14T08:40:53.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f85590b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85540cefd0 0x7f855409e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f85590b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85540cefd0 0x7f855409e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54200/0 (socket says 172.21.15.172:54200) 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f85590b2640 1 -- 172.21.15.172:0/3248824459 learned_addr learned my addr 172.21.15.172:0/3248824459 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f85588b1640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855408c4c0 0x7f855415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f854bfff640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85540b7a60 0x7f855415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f85590b2640 1 -- 172.21.15.172:0/3248824459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855408c4c0 msgr2=0x7f855415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f85590b2640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855408c4c0 0x7f855415da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f85590b2640 1 -- 172.21.15.172:0/3248824459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85540b7a60 msgr2=0x7f855415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f85590b2640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85540b7a60 0x7f855415e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f85590b2640 1 -- 172.21.15.172:0/3248824459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855415df90 con 0x7f85540cefd0 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f85590b2640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85540cefd0 0x7f855409e9c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f854c009e50 tx=0x7f854c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.162+0000 7f85497fa640 1 -- 172.21.15.172:0/3248824459 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c043020 con 0x7f85540cefd0 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.163+0000 7f85497fa640 1 -- 172.21.15.172:0/3248824459 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854c03e450 con 0x7f85540cefd0 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.163+0000 7f855ab15640 1 -- 172.21.15.172:0/3248824459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85540cc7c0 con 0x7f85540cefd0 2024-04-14T08:40:53.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.163+0000 7f85497fa640 1 -- 172.21.15.172:0/3248824459 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c03e820 con 0x7f85540cefd0 2024-04-14T08:40:53.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.163+0000 7f855ab15640 1 -- 172.21.15.172:0/3248824459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855409e1c0 con 0x7f85540cefd0 2024-04-14T08:40:53.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.164+0000 7f855ab15640 1 -- 172.21.15.172:0/3248824459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8554075490 con 0x7f85540cefd0 2024-04-14T08:40:53.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.164+0000 7f85497fa640 1 -- 172.21.15.172:0/3248824459 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f854c046020 con 0x7f85540cefd0 2024-04-14T08:40:53.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.164+0000 7f85497fa640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f854006c920 0x7f854006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.164+0000 7f85497fa640 1 -- 172.21.15.172:0/3248824459 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f854c0ae240 con 0x7f85540cefd0 2024-04-14T08:40:53.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.164+0000 7f85588b1640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f854006c920 0x7f854006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.165+0000 7f85588b1640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f854006c920 0x7f854006ede0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8550000f60 tx=0x7f8550009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:53.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.169+0000 7f85497fa640 1 -- 172.21.15.172:0/3248824459 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f854c04d070 con 0x7f85540cefd0 2024-04-14T08:40:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.388+0000 7f855ab15640 1 -- 172.21.15.172:0/3248824459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f855415e720 con 0x7f85540cefd0 2024-04-14T08:40:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.388+0000 7f85497fa640 1 -- 172.21.15.172:0/3248824459 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5196 (secure 0 0 0) 0x7f854c07be00 con 0x7f85540cefd0 2024-04-14T08:40:53.389 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:53.389 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:38:44.572424+0000","modified":"2024-04-14T08:40:14.122456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8013},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8013":{"gid":8013,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.172:6837/3091771014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":3091771014},{"type":"v1","addr":"172.21.15.172:6837","nonce":3091771014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8013,"qdb_cluster":[8013]},"id":6}]} 2024-04-14T08:40:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 67 2024-04-14T08:40:53.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.390+0000 7f855ab15640 1 -- 172.21.15.172:0/3248824459 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f854006c920 msgr2=0x7f854006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.390+0000 7f855ab15640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f854006c920 0x7f854006ede0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8550000f60 tx=0x7f8550009040 comp rx=0 tx=0).stop 2024-04-14T08:40:53.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.390+0000 7f855ab15640 1 -- 172.21.15.172:0/3248824459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85540cefd0 msgr2=0x7f855409e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.390+0000 7f855ab15640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85540cefd0 0x7f855409e9c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f854c009e50 tx=0x7f854c004a40 comp rx=0 tx=0).stop 2024-04-14T08:40:53.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.390+0000 7f855ab15640 1 -- 172.21.15.172:0/3248824459 shutdown_connections 2024-04-14T08:40:53.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.390+0000 7f855ab15640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f854006c920 0x7f854006ede0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.390+0000 7f855ab15640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85540cefd0 0x7f855409e9c0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.390+0000 7f855ab15640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85540b7a60 0x7f855415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.390+0000 7f855ab15640 1 --2- 172.21.15.172:0/3248824459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f855408c4c0 0x7f855415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.390+0000 7f855ab15640 1 -- 172.21.15.172:0/3248824459 >> 172.21.15.172:0/3248824459 conn(0x7f855408c000 msgr2=0x7f85540d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:53.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.390+0000 7f855ab15640 1 -- 172.21.15.172:0/3248824459 shutdown_connections 2024-04-14T08:40:53.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.390+0000 7f855ab15640 1 -- 172.21.15.172:0/3248824459 wait complete. 2024-04-14T08:40:53.402 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:40:53.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.504+0000 7f5214e36640 1 Processor -- start 2024-04-14T08:40:53.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.504+0000 7f5214e36640 1 -- start start 2024-04-14T08:40:53.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.505+0000 7f5214e36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f52100d1a10 0x7f5210161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.505+0000 7f5214e36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5210162530 0x7f52100d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.505+0000 7f5214e36640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52100d0200 con 0x7f521015c150 2024-04-14T08:40:53.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.505+0000 7f5214e36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f5210162980 con 0x7f5210162530 2024-04-14T08:40:53.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.505+0000 7f5214e36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f52100cef10 con 0x7f52100d1a10 2024-04-14T08:40:53.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.505+0000 7f520ed9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5210162530 0x7f52100d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.505+0000 7f520ed9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5210162530 0x7f52100d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54210/0 (socket says 172.21.15.172:54210) 2024-04-14T08:40:53.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.505+0000 7f520ed9d640 1 -- 172.21.15.172:0/3611717276 learned_addr learned my addr 172.21.15.172:0/3611717276 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:53.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.505+0000 7f5207fff640 1 --2- 172.21.15.172:0/3611717276 >> v2:172.21.15.177:3300/0 conn(0x7f52100d1a10 0x7f5210161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.505+0000 7f520ed9d640 1 -- 172.21.15.172:0/3611717276 >> v2:172.21.15.177:3300/0 conn(0x7f52100d1a10 msgr2=0x7f5210161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:53.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.505+0000 7f520ed9d640 1 --2- 172.21.15.172:0/3611717276 >> v2:172.21.15.177:3300/0 conn(0x7f52100d1a10 0x7f5210161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.505+0000 7f520ed9d640 1 -- 172.21.15.172:0/3611717276 >> v1:172.21.15.177:6789/0 conn(0x7f521015c150 legacy=0x7f52100d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.506+0000 7f520ed9d640 1 -- 172.21.15.172:0/3611717276 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52100cf160 con 0x7f5210162530 2024-04-14T08:40:53.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.506+0000 7f520ed9d640 1 --2- 172.21.15.172:0/3611717276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5210162530 0x7f52100d1e80 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f52100ba330 tx=0x7f520802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43cdd7a2eed9c89e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:53.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.506+0000 7f520dd9b640 1 -- 172.21.15.172:0/3611717276 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520802cb20 con 0x7f5210162530 2024-04-14T08:40:53.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.506+0000 7f520dd9b640 1 -- 172.21.15.172:0/3611717276 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520802ccc0 con 0x7f5210162530 2024-04-14T08:40:53.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.506+0000 7f520dd9b640 1 -- 172.21.15.172:0/3611717276 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208034610 con 0x7f5210162530 2024-04-14T08:40:53.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.507+0000 7f5214e36640 1 -- 172.21.15.172:0/3611717276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5210162530 msgr2=0x7f52100d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.507+0000 7f5214e36640 1 --2- 172.21.15.172:0/3611717276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5210162530 0x7f52100d1e80 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f52100ba330 tx=0x7f520802bf80 comp rx=0 tx=0).stop 2024-04-14T08:40:53.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.507+0000 7f5214e36640 1 -- 172.21.15.172:0/3611717276 shutdown_connections 2024-04-14T08:40:53.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.507+0000 7f5214e36640 1 --2- 172.21.15.172:0/3611717276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5210162530 0x7f52100d1e80 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.507+0000 7f5214e36640 1 --2- 172.21.15.172:0/3611717276 >> v2:172.21.15.177:3300/0 conn(0x7f52100d1a10 0x7f5210161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.507+0000 7f5214e36640 1 -- 172.21.15.172:0/3611717276 >> 172.21.15.172:0/3611717276 conn(0x7f521009d580 msgr2=0x7f52100d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:53.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.507+0000 7f5214e36640 1 -- 172.21.15.172:0/3611717276 shutdown_connections 2024-04-14T08:40:53.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.507+0000 7f5214e36640 1 -- 172.21.15.172:0/3611717276 wait complete. 2024-04-14T08:40:53.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.508+0000 7f5214e36640 1 Processor -- start 2024-04-14T08:40:53.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.508+0000 7f5214e36640 1 -- start start 2024-04-14T08:40:53.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.509+0000 7f5214e36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52100d1a10 0x7f52100b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.509+0000 7f5214e36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f521015c150 0x7f52101f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.509+0000 7f5214e36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5210162530 0x7f52101f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.509+0000 7f5214e36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52100cfa00 con 0x7f5210162530 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.509+0000 7f5214e36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f52100cf720 con 0x7f521015c150 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.509+0000 7f5214e36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f52100cfce0 con 0x7f52100d1a10 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.509+0000 7f520ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5210162530 0x7f52101f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.509+0000 7f520ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5210162530 0x7f52101f7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54216/0 (socket says 172.21.15.172:54216) 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.509+0000 7f520ed9d640 1 -- 172.21.15.172:0/3597610459 learned_addr learned my addr 172.21.15.172:0/3597610459 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.509+0000 7f5207fff640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f521015c150 0x7f52101f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.509+0000 7f520e59c640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52100d1a10 0x7f52100b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.510+0000 7f520ed9d640 1 -- 172.21.15.172:0/3597610459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52100d1a10 msgr2=0x7f52100b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.510+0000 7f520ed9d640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52100d1a10 0x7f52100b7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.510+0000 7f520ed9d640 1 -- 172.21.15.172:0/3597610459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f521015c150 msgr2=0x7f52101f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.510+0000 7f520ed9d640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f521015c150 0x7f52101f75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.510+0000 7f520ed9d640 1 -- 172.21.15.172:0/3597610459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52100c80b0 con 0x7f5210162530 2024-04-14T08:40:53.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.510+0000 7f520ed9d640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5210162530 0x7f52101f7af0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f5208009f40 tx=0x7f5208004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:53.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.510+0000 7f5206ffd640 1 -- 172.21.15.172:0/3597610459 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208043020 con 0x7f5210162530 2024-04-14T08:40:53.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.510+0000 7f5206ffd640 1 -- 172.21.15.172:0/3597610459 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520803e450 con 0x7f5210162530 2024-04-14T08:40:53.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.510+0000 7f5214e36640 1 -- 172.21.15.172:0/3597610459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52100c8390 con 0x7f5210162530 2024-04-14T08:40:53.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.510+0000 7f5206ffd640 1 -- 172.21.15.172:0/3597610459 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208009510 con 0x7f5210162530 2024-04-14T08:40:53.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.510+0000 7f5214e36640 1 -- 172.21.15.172:0/3597610459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52100c8950 con 0x7f5210162530 2024-04-14T08:40:53.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.511+0000 7f5214e36640 1 -- 172.21.15.172:0/3597610459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5210075490 con 0x7f5210162530 2024-04-14T08:40:53.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.512+0000 7f5206ffd640 1 -- 172.21.15.172:0/3597610459 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5208046020 con 0x7f5210162530 2024-04-14T08:40:53.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.512+0000 7f5206ffd640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f51f006c8f0 0x7f51f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.512+0000 7f5206ffd640 1 -- 172.21.15.172:0/3597610459 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7f52080ae420 con 0x7f5210162530 2024-04-14T08:40:53.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.512+0000 7f520e59c640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f51f006c8f0 0x7f51f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.512+0000 7f520e59c640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f51f006c8f0 0x7f51f006edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f52100ce890 tx=0x7f5200009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:53.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.521+0000 7f5206ffd640 1 -- 172.21.15.172:0/3597610459 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f520804d030 con 0x7f5210162530 2024-04-14T08:40:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.703+0000 7f5214e36640 1 -- 172.21.15.172:0/3597610459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52100d49c0 con 0x7f5210162530 2024-04-14T08:40:53.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.704+0000 7f5206ffd640 1 -- 172.21.15.172:0/3597610459 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+17401 (secure 0 0 0) 0x7f520807bfe0 con 0x7f5210162530 2024-04-14T08:40:53.705 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:53.705 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":90,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:40:10.107688+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:38:42.790996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-14T08:38:43.517208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/1386685680":"2024-04-15T08:40:10.107666+0000","172.21.15.172:6835/1386685680":"2024-04-15T08:40:10.107666+0000","172.21.15.177:6835/1124731311":"2024-04-15T08:38:58.388334+0000","172.21.15.177:6833/1124731311":"2024-04-15T08:38:58.388334+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:40:53.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.707+0000 7f5214e36640 1 -- 172.21.15.172:0/3597610459 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f51f006c8f0 msgr2=0x7f51f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.707+0000 7f5214e36640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f51f006c8f0 0x7f51f006edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f52100ce890 tx=0x7f5200009040 comp rx=0 tx=0).stop 2024-04-14T08:40:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.707+0000 7f5214e36640 1 -- 172.21.15.172:0/3597610459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5210162530 msgr2=0x7f52101f7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.707+0000 7f5214e36640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5210162530 0x7f52101f7af0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f5208009f40 tx=0x7f5208004a40 comp rx=0 tx=0).stop 2024-04-14T08:40:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.707+0000 7f5214e36640 1 -- 172.21.15.172:0/3597610459 shutdown_connections 2024-04-14T08:40:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.707+0000 7f5214e36640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f51f006c8f0 0x7f51f006edb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.707+0000 7f5214e36640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5210162530 0x7f52101f7af0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.707+0000 7f5214e36640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f521015c150 0x7f52101f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.707+0000 7f5214e36640 1 --2- 172.21.15.172:0/3597610459 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52100d1a10 0x7f52100b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.707+0000 7f5214e36640 1 -- 172.21.15.172:0/3597610459 >> 172.21.15.172:0/3597610459 conn(0x7f521009d580 msgr2=0x7f521006d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.707+0000 7f5214e36640 1 -- 172.21.15.172:0/3597610459 shutdown_connections 2024-04-14T08:40:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.707+0000 7f5214e36640 1 -- 172.21.15.172:0/3597610459 wait complete. 2024-04-14T08:40:53.719 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:40:53.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.821+0000 7fcb5df1e640 1 Processor -- start 2024-04-14T08:40:53.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.822+0000 7fcb5df1e640 1 -- start start 2024-04-14T08:40:53.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.823+0000 7fcb5df1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb580cf210 0x7fcb580b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.823+0000 7fcb5df1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb580ccdd0 con 0x7fcb5815c420 2024-04-14T08:40:53.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.823+0000 7fcb5df1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb580d2ae0 con 0x7fcb5806f3e0 2024-04-14T08:40:53.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.823+0000 7fcb5df1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fcb580d30a0 con 0x7fcb580cf210 2024-04-14T08:40:53.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.823+0000 7fcb57fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb580cf210 0x7fcb580b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.823+0000 7fcb577fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fcb5815c420 0x7fcb580d0010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50450/0 (socket says 172.21.15.172:50450) 2024-04-14T08:40:53.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.823+0000 7fcb577fe640 1 -- 172.21.15.172:0/2154663042 learned_addr learned my addr 172.21.15.172:0/2154663042 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:53.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.823+0000 7fcb57fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb580cf210 0x7fcb580b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54226/0 (socket says 172.21.15.172:54226) 2024-04-14T08:40:53.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.823+0000 7fcb57fff640 1 -- 172.21.15.172:0/2154663042 >> v1:172.21.15.177:6789/0 conn(0x7fcb5806f3e0 legacy=0x7fcb580ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:53.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.823+0000 7fcb57fff640 1 -- 172.21.15.172:0/2154663042 >> v1:172.21.15.172:6789/0 conn(0x7fcb5815c420 legacy=0x7fcb580d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.823+0000 7fcb57fff640 1 -- 172.21.15.172:0/2154663042 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb580d2dc0 con 0x7fcb580cf210 2024-04-14T08:40:53.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.824+0000 7fcb57fff640 1 --2- 172.21.15.172:0/2154663042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb580cf210 0x7fcb580b1510 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fcb500097a0 tx=0x7fcb5002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712895137a05ce74 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:53.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.824+0000 7fcb567fc640 1 -- 172.21.15.172:0/2154663042 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50035020 con 0x7fcb580cf210 2024-04-14T08:40:53.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.824+0000 7fcb567fc640 1 -- 172.21.15.172:0/2154663042 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb50031830 con 0x7fcb580cf210 2024-04-14T08:40:53.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.824+0000 7fcb567fc640 1 -- 172.21.15.172:0/2154663042 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50031c00 con 0x7fcb580cf210 2024-04-14T08:40:53.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.825+0000 7fcb5df1e640 1 -- 172.21.15.172:0/2154663042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb580cf210 msgr2=0x7fcb580b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.825+0000 7fcb5df1e640 1 --2- 172.21.15.172:0/2154663042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb580cf210 0x7fcb580b1510 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fcb500097a0 tx=0x7fcb5002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.825+0000 7fcb5df1e640 1 -- 172.21.15.172:0/2154663042 shutdown_connections 2024-04-14T08:40:53.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.825+0000 7fcb5df1e640 1 --2- 172.21.15.172:0/2154663042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb580cf210 0x7fcb580b1510 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.825+0000 7fcb5df1e640 1 -- 172.21.15.172:0/2154663042 >> 172.21.15.172:0/2154663042 conn(0x7fcb5808c000 msgr2=0x7fcb580d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:53.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.825+0000 7fcb5df1e640 1 -- 172.21.15.172:0/2154663042 shutdown_connections 2024-04-14T08:40:53.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.825+0000 7fcb5df1e640 1 -- 172.21.15.172:0/2154663042 wait complete. 2024-04-14T08:40:53.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.826+0000 7fcb5df1e640 1 Processor -- start 2024-04-14T08:40:53.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.826+0000 7fcb5df1e640 1 -- start start 2024-04-14T08:40:53.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb5df1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb580cf210 0x7fcb5815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb5df1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5815c420 0x7fcb580b7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb5df1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb580c7e30 0x7fcb580c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb5df1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcb580b11e0 con 0x7fcb580c7e30 2024-04-14T08:40:53.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb5df1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5806f870 con 0x7fcb5815c420 2024-04-14T08:40:53.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb5df1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5815c800 con 0x7fcb580cf210 2024-04-14T08:40:53.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb580c7e30 0x7fcb580c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb580c7e30 0x7fcb580c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54228/0 (socket says 172.21.15.172:54228) 2024-04-14T08:40:53.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb57fff640 1 -- 172.21.15.172:0/1633122374 learned_addr learned my addr 172.21.15.172:0/1633122374 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:53.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb57fff640 1 -- 172.21.15.172:0/1633122374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb580cf210 msgr2=0x7fcb5815aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb577fe640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb580cf210 0x7fcb5815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb56ffd640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5815c420 0x7fcb580b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb57fff640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb580cf210 0x7fcb5815aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb57fff640 1 -- 172.21.15.172:0/1633122374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5815c420 msgr2=0x7fcb580b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:53.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb57fff640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5815c420 0x7fcb580b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:53.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb57fff640 1 -- 172.21.15.172:0/1633122374 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb580b70a0 con 0x7fcb580c7e30 2024-04-14T08:40:53.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.827+0000 7fcb56ffd640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5815c420 0x7fcb580b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:53.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.828+0000 7fcb57fff640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb580c7e30 0x7fcb580c82e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fcb500098c0 tx=0x7fcb500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:53.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.828+0000 7fcb4ffff640 1 -- 172.21.15.172:0/1633122374 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50041070 con 0x7fcb580c7e30 2024-04-14T08:40:53.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.828+0000 7fcb4ffff640 1 -- 172.21.15.172:0/1633122374 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb5003a8d0 con 0x7fcb580c7e30 2024-04-14T08:40:53.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.828+0000 7fcb5df1e640 1 -- 172.21.15.172:0/1633122374 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb580cb8b0 con 0x7fcb580c7e30 2024-04-14T08:40:53.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.828+0000 7fcb4ffff640 1 -- 172.21.15.172:0/1633122374 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50035040 con 0x7fcb580c7e30 2024-04-14T08:40:53.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.828+0000 7fcb5df1e640 1 -- 172.21.15.172:0/1633122374 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb580cbe70 con 0x7fcb580c7e30 2024-04-14T08:40:53.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.829+0000 7fcb5df1e640 1 -- 172.21.15.172:0/1633122374 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb58075490 con 0x7fcb580c7e30 2024-04-14T08:40:53.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.830+0000 7fcb4ffff640 1 -- 172.21.15.172:0/1633122374 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb50038020 con 0x7fcb580c7e30 2024-04-14T08:40:53.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.830+0000 7fcb4ffff640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb3806c8f0 0x7fcb3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:53.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.830+0000 7fcb4ffff640 1 -- 172.21.15.172:0/1633122374 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6389+0+0 (secure 0 0 0) 0x7fcb500ad8c0 con 0x7fcb580c7e30 2024-04-14T08:40:53.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.830+0000 7fcb577fe640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb3806c8f0 0x7fcb3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:53.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.830+0000 7fcb577fe640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb3806c8f0 0x7fcb3806edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcb480012e0 tx=0x7fcb48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:53.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:53.839+0000 7fcb4ffff640 1 -- 172.21.15.172:0/1633122374 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcb5007ba40 con 0x7fcb580c7e30 2024-04-14T08:40:54.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.085+0000 7fcb5df1e640 1 -- 172.21.15.172:0/1633122374 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcb580b7440 con 0x7fcb580c7e30 2024-04-14T08:40:54.086 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:40:54.086+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:40:54.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.087+0000 7fcb4ffff640 1 -- 172.21.15.172:0/1633122374 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7fcb5007b340 con 0x7fcb580c7e30 2024-04-14T08:40:54.088 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:40:54.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.090+0000 7fcb5df1e640 1 -- 172.21.15.172:0/1633122374 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb3806c8f0 msgr2=0x7fcb3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:54.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.090+0000 7fcb5df1e640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb3806c8f0 0x7fcb3806edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcb480012e0 tx=0x7fcb48009040 comp rx=0 tx=0).stop 2024-04-14T08:40:54.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.090+0000 7fcb5df1e640 1 -- 172.21.15.172:0/1633122374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb580c7e30 msgr2=0x7fcb580c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:54.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.090+0000 7fcb5df1e640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb580c7e30 0x7fcb580c82e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fcb500098c0 tx=0x7fcb500046c0 comp rx=0 tx=0).stop 2024-04-14T08:40:54.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.090+0000 7fcb5df1e640 1 -- 172.21.15.172:0/1633122374 shutdown_connections 2024-04-14T08:40:54.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.090+0000 7fcb5df1e640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb3806c8f0 0x7fcb3806edb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:54.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.090+0000 7fcb5df1e640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb580c7e30 0x7fcb580c82e0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:54.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.090+0000 7fcb5df1e640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb5815c420 0x7fcb580b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:54.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.090+0000 7fcb5df1e640 1 --2- 172.21.15.172:0/1633122374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb580cf210 0x7fcb5815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:54.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.090+0000 7fcb5df1e640 1 -- 172.21.15.172:0/1633122374 >> 172.21.15.172:0/1633122374 conn(0x7fcb5808c000 msgr2=0x7fcb580b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:54.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.090+0000 7fcb5df1e640 1 -- 172.21.15.172:0/1633122374 shutdown_connections 2024-04-14T08:40:54.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.090+0000 7fcb5df1e640 1 -- 172.21.15.172:0/1633122374 wait complete. 2024-04-14T08:40:54.092 INFO:tasks.ceph.mds.c.smithi172.stderr: -2369> 2024-04-14T08:40:16.116+0000 7fd70e1f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:40:54.093 INFO:tasks.ceph.mds.c.smithi172.stderr: -1914> 2024-04-14T08:40:17.019+0000 7fd70a1e9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x559da32a8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-14T08:40:54.102 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-14T08:40:54.120 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:40:54.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.206+0000 7fc84fe88640 1 Processor -- start 2024-04-14T08:40:54.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.206+0000 7fc84fe88640 1 -- start start 2024-04-14T08:40:54.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.207+0000 7fc84fe88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc84813fa70 0x7fc8480af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:54.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.207+0000 7fc84fe88640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8480ce150 0x7fc8480722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:54.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.207+0000 7fc84fe88640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc8480ce5b0 0x7fc848072820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:54.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.207+0000 7fc84fe88640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc8480712d0 con 0x7fc8480ce150 2024-04-14T08:40:54.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.207+0000 7fc84fe88640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc848072d60 con 0x7fc8480ce5b0 2024-04-14T08:40:54.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.207+0000 7fc84fe88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc848071520 con 0x7fc84813fa70 2024-04-14T08:40:54.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.207+0000 7fc84d423640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8480ce150 0x7fc8480722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:54.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.207+0000 7fc84d423640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8480ce150 0x7fc8480722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54260/0 (socket says 172.21.15.172:54260) 2024-04-14T08:40:54.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.207+0000 7fc84d423640 1 -- 172.21.15.172:0/1675125263 learned_addr learned my addr 172.21.15.172:0/1675125263 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:54.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.207+0000 7fc84dc24640 1 --2- 172.21.15.172:0/1675125263 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc84813fa70 0x7fc8480af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:54.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.207+0000 7fc84e425640 1 --2- 172.21.15.172:0/1675125263 >> v2:172.21.15.177:3300/0 conn(0x7fc8480ce5b0 0x7fc848072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:54.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.208+0000 7fc84d423640 1 -- 172.21.15.172:0/1675125263 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc84813fa70 msgr2=0x7fc8480af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:54.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.208+0000 7fc84d423640 1 --2- 172.21.15.172:0/1675125263 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc84813fa70 0x7fc8480af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:54.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.208+0000 7fc84d423640 1 -- 172.21.15.172:0/1675125263 >> v2:172.21.15.177:3300/0 conn(0x7fc8480ce5b0 msgr2=0x7fc848072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:54.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.208+0000 7fc84d423640 1 --2- 172.21.15.172:0/1675125263 >> v2:172.21.15.177:3300/0 conn(0x7fc8480ce5b0 0x7fc848072820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:54.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.208+0000 7fc84d423640 1 -- 172.21.15.172:0/1675125263 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc848071800 con 0x7fc8480ce150 2024-04-14T08:40:54.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.208+0000 7fc84e425640 1 --2- 172.21.15.172:0/1675125263 >> v2:172.21.15.177:3300/0 conn(0x7fc8480ce5b0 0x7fc848072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:54.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.208+0000 7fc84d423640 1 --2- 172.21.15.172:0/1675125263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8480ce150 0x7fc8480722e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fc84815e250 tx=0x7fc83802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6914660bd0201c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:54.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.208+0000 7fc84cc22640 1 -- 172.21.15.172:0/1675125263 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83802e070 con 0x7fc8480ce150 2024-04-14T08:40:54.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.208+0000 7fc84cc22640 1 -- 172.21.15.172:0/1675125263 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc838037070 con 0x7fc8480ce150 2024-04-14T08:40:54.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.208+0000 7fc84cc22640 1 -- 172.21.15.172:0/1675125263 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838031a00 con 0x7fc8480ce150 2024-04-14T08:40:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.209+0000 7fc84fe88640 1 -- 172.21.15.172:0/1675125263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8480ce150 msgr2=0x7fc8480722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:54.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.209+0000 7fc84fe88640 1 --2- 172.21.15.172:0/1675125263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8480ce150 0x7fc8480722e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fc84815e250 tx=0x7fc83802b880 comp rx=0 tx=0).stop 2024-04-14T08:40:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.209+0000 7fc84fe88640 1 -- 172.21.15.172:0/1675125263 shutdown_connections 2024-04-14T08:40:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.209+0000 7fc84fe88640 1 --2- 172.21.15.172:0/1675125263 >> v2:172.21.15.177:3300/0 conn(0x7fc8480ce5b0 0x7fc848072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.209+0000 7fc84fe88640 1 --2- 172.21.15.172:0/1675125263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8480ce150 0x7fc8480722e0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.209+0000 7fc84fe88640 1 --2- 172.21.15.172:0/1675125263 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc84813fa70 0x7fc8480af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.209+0000 7fc84fe88640 1 -- 172.21.15.172:0/1675125263 >> 172.21.15.172:0/1675125263 conn(0x7fc84806b520 msgr2=0x7fc84806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:54.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.209+0000 7fc84fe88640 1 -- 172.21.15.172:0/1675125263 shutdown_connections 2024-04-14T08:40:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.209+0000 7fc84fe88640 1 -- 172.21.15.172:0/1675125263 wait complete. 2024-04-14T08:40:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.210+0000 7fc84fe88640 1 Processor -- start 2024-04-14T08:40:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.210+0000 7fc84fe88640 1 -- start start 2024-04-14T08:40:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84fe88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8480ce150 0x7fc848159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:54.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84fe88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8480ce5b0 0x7fc8480a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:54.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84fe88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc84813fa70 0x7fc8480ac340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:54.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84fe88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc848144130 con 0x7fc84813fa70 2024-04-14T08:40:54.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84fe88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc848071dc0 con 0x7fc8480ce150 2024-04-14T08:40:54.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84fe88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc8481442b0 con 0x7fc8480ce5b0 2024-04-14T08:40:54.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84e425640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc84813fa70 0x7fc8480ac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:54.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84e425640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc84813fa70 0x7fc8480ac340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54272/0 (socket says 172.21.15.172:54272) 2024-04-14T08:40:54.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84e425640 1 -- 172.21.15.172:0/1572118966 learned_addr learned my addr 172.21.15.172:0/1572118966 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:54.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84dc24640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8480ce150 0x7fc848159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:54.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84d423640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8480ce5b0 0x7fc8480a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:54.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84e425640 1 -- 172.21.15.172:0/1572118966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8480ce5b0 msgr2=0x7fc8480a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:54.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84e425640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8480ce5b0 0x7fc8480a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:54.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84e425640 1 -- 172.21.15.172:0/1572118966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8480ce150 msgr2=0x7fc848159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:54.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84e425640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8480ce150 0x7fc848159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:54.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.211+0000 7fc84e425640 1 -- 172.21.15.172:0/1572118966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8480aa9c0 con 0x7fc84813fa70 2024-04-14T08:40:54.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.212+0000 7fc84e425640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc84813fa70 0x7fc8480ac340 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc840009fd0 tx=0x7fc84000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:54.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.212+0000 7fc83e7fc640 1 -- 172.21.15.172:0/1572118966 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84001a070 con 0x7fc84813fa70 2024-04-14T08:40:54.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.212+0000 7fc83e7fc640 1 -- 172.21.15.172:0/1572118966 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc840003bb0 con 0x7fc84813fa70 2024-04-14T08:40:54.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.212+0000 7fc84fe88640 1 -- 172.21.15.172:0/1572118966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8480aaca0 con 0x7fc84813fa70 2024-04-14T08:40:54.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.212+0000 7fc83e7fc640 1 -- 172.21.15.172:0/1572118966 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840004030 con 0x7fc84813fa70 2024-04-14T08:40:54.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.212+0000 7fc84fe88640 1 -- 172.21.15.172:0/1572118966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8480ab260 con 0x7fc84813fa70 2024-04-14T08:40:54.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.213+0000 7fc84fe88640 1 -- 172.21.15.172:0/1572118966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc848068590 con 0x7fc84813fa70 2024-04-14T08:40:54.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.213+0000 7fc83e7fc640 1 -- 172.21.15.172:0/1572118966 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc84001d070 con 0x7fc84813fa70 2024-04-14T08:40:54.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.214+0000 7fc83e7fc640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc82c06c8f0 0x7fc82c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:54.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.214+0000 7fc83e7fc640 1 -- 172.21.15.172:0/1572118966 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6475+0+0 (secure 0 0 0) 0x7fc84008c100 con 0x7fc84813fa70 2024-04-14T08:40:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.214+0000 7fc84dc24640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc82c06c8f0 0x7fc82c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.214+0000 7fc84dc24640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc82c06c8f0 0x7fc82c06edb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc844009a20 tx=0x7fc844009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:54.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.218+0000 7fc83e7fc640 1 -- 172.21.15.172:0/1572118966 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc840010070 con 0x7fc84813fa70 2024-04-14T08:40:54.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:54.431+0000 7fc84fe88640 1 -- 172.21.15.172:0/1572118966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc84806c530 con 0x7fc84813fa70 2024-04-14T08:40:55.033 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:40:55.036 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:40:55.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.089+0000 7fc83e7fc640 1 -- 172.21.15.172:0/1572118966 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7fc840059be0 con 0x7fc84813fa70 2024-04-14T08:40:55.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.091+0000 7fc84fe88640 1 -- 172.21.15.172:0/1572118966 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc82c06c8f0 msgr2=0x7fc82c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:55.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.091+0000 7fc84fe88640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc82c06c8f0 0x7fc82c06edb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc844009a20 tx=0x7fc844009190 comp rx=0 tx=0).stop 2024-04-14T08:40:55.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.091+0000 7fc84fe88640 1 -- 172.21.15.172:0/1572118966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc84813fa70 msgr2=0x7fc8480ac340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:55.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.091+0000 7fc84fe88640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc84813fa70 0x7fc8480ac340 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc840009fd0 tx=0x7fc84000d560 comp rx=0 tx=0).stop 2024-04-14T08:40:55.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.091+0000 7fc84fe88640 1 -- 172.21.15.172:0/1572118966 shutdown_connections 2024-04-14T08:40:55.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.091+0000 7fc84fe88640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc82c06c8f0 0x7fc82c06edb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:55.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.091+0000 7fc84fe88640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc84813fa70 0x7fc8480ac340 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:55.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.091+0000 7fc84fe88640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8480ce5b0 0x7fc8480a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:55.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.091+0000 7fc84fe88640 1 --2- 172.21.15.172:0/1572118966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8480ce150 0x7fc848159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:55.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.091+0000 7fc84fe88640 1 -- 172.21.15.172:0/1572118966 >> 172.21.15.172:0/1572118966 conn(0x7fc84806b520 msgr2=0x7fc8480cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:55.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.091+0000 7fc84fe88640 1 -- 172.21.15.172:0/1572118966 shutdown_connections 2024-04-14T08:40:55.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.091+0000 7fc84fe88640 1 -- 172.21.15.172:0/1572118966 wait complete. 2024-04-14T08:40:55.104 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-14T08:40:55.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.251+0000 7fe14f636640 1 Processor -- start 2024-04-14T08:40:55.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.251+0000 7fe14f636640 1 -- start start 2024-04-14T08:40:55.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14f636640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe14815c2b0 0x7fe1480c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:55.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14f636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1480cc060 0x7fe14806d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:55.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14f636640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1480c6bf0 con 0x7fe1480ca3c0 2024-04-14T08:40:55.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14f636640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe1480cc580 con 0x7fe14815c2b0 2024-04-14T08:40:55.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14f636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe148160ae0 con 0x7fe1480cc060 2024-04-14T08:40:55.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14cbd1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1480cc060 0x7fe14806d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:55.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14dbd3640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fe1480ca3c0 0x7fe1480c4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:53652/0 (socket says 172.21.15.172:53652) 2024-04-14T08:40:55.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14d3d2640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe14815c2b0 0x7fe1480c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:55.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14dbd3640 1 -- 172.21.15.172:0/3190704248 learned_addr learned my addr 172.21.15.172:0/3190704248 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:55.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe13ffff640 1 -- 172.21.15.172:0/3190704248 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3571904111 0 0) 0x7fe1480c6bf0 con 0x7fe1480ca3c0 2024-04-14T08:40:55.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe13ffff640 1 -- 172.21.15.172:0/3190704248 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe134003520 con 0x7fe1480ca3c0 2024-04-14T08:40:55.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14cbd1640 1 -- 172.21.15.172:0/3190704248 >> v2:172.21.15.177:3300/0 conn(0x7fe14815c2b0 msgr2=0x7fe1480c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:55.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14cbd1640 1 --2- 172.21.15.172:0/3190704248 >> v2:172.21.15.177:3300/0 conn(0x7fe14815c2b0 0x7fe1480c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:55.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14cbd1640 1 -- 172.21.15.172:0/3190704248 >> v1:172.21.15.177:6789/0 conn(0x7fe1480ca3c0 legacy=0x7fe1480c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:55.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14cbd1640 1 -- 172.21.15.172:0/3190704248 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1480c66d0 con 0x7fe1480cc060 2024-04-14T08:40:55.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.252+0000 7fe14d3d2640 1 --2- 172.21.15.172:0/3190704248 >> v2:172.21.15.177:3300/0 conn(0x7fe14815c2b0 0x7fe1480c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:40:55.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.253+0000 7fe14cbd1640 1 --2- 172.21.15.172:0/3190704248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1480cc060 0x7fe14806d2e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe1380097a0 tx=0x7fe13802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a71a046c9de0ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:55.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.253+0000 7fe13ffff640 1 -- 172.21.15.172:0/3190704248 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138035020 con 0x7fe1480cc060 2024-04-14T08:40:55.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.253+0000 7fe13ffff640 1 -- 172.21.15.172:0/3190704248 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe138031830 con 0x7fe1480cc060 2024-04-14T08:40:55.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.253+0000 7fe13ffff640 1 -- 172.21.15.172:0/3190704248 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138031be0 con 0x7fe1480cc060 2024-04-14T08:40:55.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.254+0000 7fe14f636640 1 -- 172.21.15.172:0/3190704248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1480cc060 msgr2=0x7fe14806d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:55.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.254+0000 7fe14f636640 1 --2- 172.21.15.172:0/3190704248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1480cc060 0x7fe14806d2e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe1380097a0 tx=0x7fe13802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:40:55.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.254+0000 7fe14f636640 1 -- 172.21.15.172:0/3190704248 shutdown_connections 2024-04-14T08:40:55.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.254+0000 7fe14f636640 1 --2- 172.21.15.172:0/3190704248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1480cc060 0x7fe14806d2e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:55.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.254+0000 7fe14f636640 1 --2- 172.21.15.172:0/3190704248 >> v2:172.21.15.177:3300/0 conn(0x7fe14815c2b0 0x7fe1480c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:55.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.254+0000 7fe14f636640 1 -- 172.21.15.172:0/3190704248 >> 172.21.15.172:0/3190704248 conn(0x7fe1480b1a50 msgr2=0x7fe1480b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:55.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.254+0000 7fe14f636640 1 -- 172.21.15.172:0/3190704248 shutdown_connections 2024-04-14T08:40:55.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.254+0000 7fe14f636640 1 -- 172.21.15.172:0/3190704248 wait complete. 2024-04-14T08:40:55.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.255+0000 7fe14f636640 1 Processor -- start 2024-04-14T08:40:55.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.255+0000 7fe14f636640 1 -- start start 2024-04-14T08:40:55.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14f636640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1480ca3c0 0x7fe1481f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:55.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14f636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1480cc060 0x7fe148153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14f636640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe14815c2b0 0x7fe148151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14f636640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1480b26f0 con 0x7fe1480ca3c0 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14f636640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1481607d0 con 0x7fe14815c2b0 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14f636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe1480b2870 con 0x7fe1480cc060 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14d3d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1480ca3c0 0x7fe1481f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14d3d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1480ca3c0 0x7fe1481f52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54288/0 (socket says 172.21.15.172:54288) 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14d3d2640 1 -- 172.21.15.172:0/1633433385 learned_addr learned my addr 172.21.15.172:0/1633433385 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14cbd1640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1480cc060 0x7fe148153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14dbd3640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe14815c2b0 0x7fe148151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14d3d2640 1 -- 172.21.15.172:0/1633433385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1480cc060 msgr2=0x7fe148153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14d3d2640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1480cc060 0x7fe148153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14d3d2640 1 -- 172.21.15.172:0/1633433385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe14815c2b0 msgr2=0x7fe148151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14d3d2640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe14815c2b0 0x7fe148151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:55.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14d3d2640 1 -- 172.21.15.172:0/1633433385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1481598a0 con 0x7fe1480ca3c0 2024-04-14T08:40:55.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14dbd3640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe14815c2b0 0x7fe148151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:55.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14cbd1640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1480cc060 0x7fe148153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:55.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe14d3d2640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1480ca3c0 0x7fe1481f52e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fe14400ebe0 tx=0x7fe14400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:55.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.256+0000 7fe13dffb640 1 -- 172.21.15.172:0/1633433385 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144017070 con 0x7fe1480ca3c0 2024-04-14T08:40:55.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.257+0000 7fe13dffb640 1 -- 172.21.15.172:0/1633433385 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe14400cc30 con 0x7fe1480ca3c0 2024-04-14T08:40:55.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.257+0000 7fe14f636640 1 -- 172.21.15.172:0/1633433385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1481f5b30 con 0x7fe1480ca3c0 2024-04-14T08:40:55.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.257+0000 7fe13dffb640 1 -- 172.21.15.172:0/1633433385 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144013470 con 0x7fe1480ca3c0 2024-04-14T08:40:55.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.257+0000 7fe14f636640 1 -- 172.21.15.172:0/1633433385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1481f60f0 con 0x7fe1480ca3c0 2024-04-14T08:40:55.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.258+0000 7fe14f636640 1 -- 172.21.15.172:0/1633433385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1480686a0 con 0x7fe1480ca3c0 2024-04-14T08:40:55.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.258+0000 7fe13dffb640 1 -- 172.21.15.172:0/1633433385 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe14401a070 con 0x7fe1480ca3c0 2024-04-14T08:40:55.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.259+0000 7fe13dffb640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe12c06c8f0 0x7fe12c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:55.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.259+0000 7fe14cbd1640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe12c06c8f0 0x7fe12c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:55.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.259+0000 7fe13dffb640 1 -- 172.21.15.172:0/1633433385 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6475+0+0 (secure 0 0 0) 0x7fe14408bdd0 con 0x7fe1480ca3c0 2024-04-14T08:40:55.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.259+0000 7fe14cbd1640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe12c06c8f0 0x7fe12c06edb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe138036000 tx=0x7fe138008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:55.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.263+0000 7fe13dffb640 1 -- 172.21.15.172:0/1633433385 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe144059f00 con 0x7fe1480ca3c0 2024-04-14T08:40:55.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:55.439+0000 7fe14f636640 1 -- 172.21.15.172:0/1633433385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe14806ef60 con 0x7fe1480ca3c0 2024-04-14T08:40:56.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.093+0000 7fe13dffb640 1 -- 172.21.15.172:0/1633433385 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) ==== 181+0+0 (secure 0 0 0) 0x7fe1440600d0 con 0x7fe1480ca3c0 2024-04-14T08:40:56.094 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' removed 2024-04-14T08:40:56.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.096+0000 7fe14f636640 1 -- 172.21.15.172:0/1633433385 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe12c06c8f0 msgr2=0x7fe12c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:56.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.096+0000 7fe14f636640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe12c06c8f0 0x7fe12c06edb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe138036000 tx=0x7fe138008030 comp rx=0 tx=0).stop 2024-04-14T08:40:56.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.096+0000 7fe14f636640 1 -- 172.21.15.172:0/1633433385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1480ca3c0 msgr2=0x7fe1481f52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:56.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.096+0000 7fe14f636640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1480ca3c0 0x7fe1481f52e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fe14400ebe0 tx=0x7fe14400c520 comp rx=0 tx=0).stop 2024-04-14T08:40:56.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.096+0000 7fe14f636640 1 -- 172.21.15.172:0/1633433385 shutdown_connections 2024-04-14T08:40:56.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.096+0000 7fe14f636640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe12c06c8f0 0x7fe12c06edb0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:56.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.096+0000 7fe14f636640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe14815c2b0 0x7fe148151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:56.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.096+0000 7fe14f636640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1480cc060 0x7fe148153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:56.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.096+0000 7fe14f636640 1 --2- 172.21.15.172:0/1633433385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1480ca3c0 0x7fe1481f52e0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:56.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.096+0000 7fe14f636640 1 -- 172.21.15.172:0/1633433385 >> 172.21.15.172:0/1633433385 conn(0x7fe1480b1a50 msgr2=0x7fe1480b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:56.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.096+0000 7fe14f636640 1 -- 172.21.15.172:0/1633433385 shutdown_connections 2024-04-14T08:40:56.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.097+0000 7fe14f636640 1 -- 172.21.15.172:0/1633433385 wait complete. 2024-04-14T08:40:56.110 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-14T08:40:56.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.218+0000 7fc6f451e640 1 Processor -- start 2024-04-14T08:40:56.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.218+0000 7fc6f451e640 1 -- start start 2024-04-14T08:40:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f451e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6ec15a930 0x7fc6ec06ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f451e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc6ec15ebc0 0x7fc6ec0ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f451e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ec06dc10 con 0x7fc6ec0ce350 2024-04-14T08:40:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f451e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc6ec0cecf0 con 0x7fc6ec15a930 2024-04-14T08:40:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f451e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc6ec06e820 con 0x7fc6ec15ebc0 2024-04-14T08:40:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f22ba640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6ec15a930 0x7fc6ec06ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f1ab9640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc6ec0ce350 0x7fc6ec15e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50468/0 (socket says 172.21.15.172:50468) 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f1ab9640 1 -- 172.21.15.172:0/1037466755 learned_addr learned my addr 172.21.15.172:0/1037466755 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f22ba640 1 -- 172.21.15.172:0/1037466755 >> v2:172.21.15.177:3300/0 conn(0x7fc6ec15ebc0 msgr2=0x7fc6ec0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f22ba640 1 --2- 172.21.15.172:0/1037466755 >> v2:172.21.15.177:3300/0 conn(0x7fc6ec15ebc0 0x7fc6ec0ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f22ba640 1 -- 172.21.15.172:0/1037466755 >> v1:172.21.15.172:6789/0 conn(0x7fc6ec0ce350 legacy=0x7fc6ec15e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f12b8640 1 -- 172.21.15.172:0/1037466755 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2575431450 0 0) 0x7fc6ec06dc10 con 0x7fc6ec0ce350 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.219+0000 7fc6f22ba640 1 -- 172.21.15.172:0/1037466755 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec06def0 con 0x7fc6ec15a930 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.220+0000 7fc6f22ba640 1 --2- 172.21.15.172:0/1037466755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec15a930 0x7fc6ec06ecb0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0c6dc0 tx=0x7fc6e802e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5374ed42f564e00 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.220+0000 7fc6f12b8640 1 -- 172.21.15.172:0/1037466755 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e803b070 con 0x7fc6ec15a930 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.220+0000 7fc6f12b8640 1 -- 172.21.15.172:0/1037466755 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e802edc0 con 0x7fc6ec15a930 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.220+0000 7fc6f12b8640 1 -- 172.21.15.172:0/1037466755 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8036660 con 0x7fc6ec15a930 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.221+0000 7fc6f451e640 1 -- 172.21.15.172:0/1037466755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec15a930 msgr2=0x7fc6ec06ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.221+0000 7fc6f451e640 1 --2- 172.21.15.172:0/1037466755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec15a930 0x7fc6ec06ecb0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0c6dc0 tx=0x7fc6e802e010 comp rx=0 tx=0).stop 2024-04-14T08:40:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.221+0000 7fc6f451e640 1 -- 172.21.15.172:0/1037466755 shutdown_connections 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.221+0000 7fc6f451e640 1 --2- 172.21.15.172:0/1037466755 >> v2:172.21.15.177:3300/0 conn(0x7fc6ec15ebc0 0x7fc6ec0ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.221+0000 7fc6f451e640 1 --2- 172.21.15.172:0/1037466755 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec15a930 0x7fc6ec06ecb0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.221+0000 7fc6f451e640 1 -- 172.21.15.172:0/1037466755 >> 172.21.15.172:0/1037466755 conn(0x7fc6ec0b1a50 msgr2=0x7fc6ec0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.221+0000 7fc6f451e640 1 -- 172.21.15.172:0/1037466755 shutdown_connections 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.221+0000 7fc6f451e640 1 -- 172.21.15.172:0/1037466755 wait complete. 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.222+0000 7fc6f451e640 1 Processor -- start 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.222+0000 7fc6f451e640 1 -- start start 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f451e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6ec0ce350 0x7fc6ec0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f451e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec15a930 0x7fc6ec0c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f451e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6ec15ebc0 0x7fc6ec15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f451e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec15ad10 con 0x7fc6ec15a930 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f451e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec06e4b0 con 0x7fc6ec15ebc0 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f451e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ec06d2e0 con 0x7fc6ec0ce350 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f1ab9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec15a930 0x7fc6ec0c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f1ab9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec15a930 0x7fc6ec0c56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54298/0 (socket says 172.21.15.172:54298) 2024-04-14T08:40:56.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f1ab9640 1 -- 172.21.15.172:0/2942087156 learned_addr learned my addr 172.21.15.172:0/2942087156 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:56.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f22ba640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6ec0ce350 0x7fc6ec0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:56.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f2abb640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6ec15ebc0 0x7fc6ec15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:56.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f1ab9640 1 -- 172.21.15.172:0/2942087156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6ec0ce350 msgr2=0x7fc6ec0c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:56.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f1ab9640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6ec0ce350 0x7fc6ec0c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:56.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f1ab9640 1 -- 172.21.15.172:0/2942087156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6ec15ebc0 msgr2=0x7fc6ec15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:56.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f1ab9640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6ec15ebc0 0x7fc6ec15ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:56.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f1ab9640 1 -- 172.21.15.172:0/2942087156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec0adc00 con 0x7fc6ec15a930 2024-04-14T08:40:56.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f2abb640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6ec15ebc0 0x7fc6ec15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f1ab9640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec15a930 0x7fc6ec0c56e0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00c4d0 tx=0x7fc6dc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6e2ffd640 1 -- 172.21.15.172:0/2942087156 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc018070 con 0x7fc6ec15a930 2024-04-14T08:40:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6e2ffd640 1 -- 172.21.15.172:0/2942087156 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6dc007500 con 0x7fc6ec15a930 2024-04-14T08:40:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f451e640 1 -- 172.21.15.172:0/2942087156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ec0aed10 con 0x7fc6ec15a930 2024-04-14T08:40:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6e2ffd640 1 -- 172.21.15.172:0/2942087156 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc004a20 con 0x7fc6ec15a930 2024-04-14T08:40:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.223+0000 7fc6f451e640 1 -- 172.21.15.172:0/2942087156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ec0ad120 con 0x7fc6ec15a930 2024-04-14T08:40:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.224+0000 7fc6f451e640 1 -- 172.21.15.172:0/2942087156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ec0686a0 con 0x7fc6ec15a930 2024-04-14T08:40:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.225+0000 7fc6e2ffd640 1 -- 172.21.15.172:0/2942087156 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6dc01b070 con 0x7fc6ec15a930 2024-04-14T08:40:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.225+0000 7fc6e2ffd640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc6d006c920 0x7fc6d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.225+0000 7fc6e2ffd640 1 -- 172.21.15.172:0/2942087156 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6037+0+0 (secure 0 0 0) 0x7fc6dc08af90 con 0x7fc6ec15a930 2024-04-14T08:40:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.225+0000 7fc6f22ba640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc6d006c920 0x7fc6d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:56.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.226+0000 7fc6f22ba640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc6d006c920 0x7fc6d006ede0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8038000 tx=0x7fc6e8032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:56.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.230+0000 7fc6e2ffd640 1 -- 172.21.15.172:0/2942087156 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc6dc059270 con 0x7fc6ec15a930 2024-04-14T08:40:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:56.408+0000 7fc6f451e640 1 -- 172.21.15.172:0/2942087156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc6ec0cb980 con 0x7fc6ec15a930 2024-04-14T08:40:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.094+0000 7fc6e2ffd640 1 -- 172.21.15.172:0/2942087156 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) ==== 169+0+0 (secure 0 0 0) 0x7fc6dc058b70 con 0x7fc6ec15a930 2024-04-14T08:40:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' removed 2024-04-14T08:40:57.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.097+0000 7fc6f451e640 1 -- 172.21.15.172:0/2942087156 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc6d006c920 msgr2=0x7fc6d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:57.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.097+0000 7fc6f451e640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc6d006c920 0x7fc6d006ede0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8038000 tx=0x7fc6e8032000 comp rx=0 tx=0).stop 2024-04-14T08:40:57.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.097+0000 7fc6f451e640 1 -- 172.21.15.172:0/2942087156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec15a930 msgr2=0x7fc6ec0c56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:57.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.097+0000 7fc6f451e640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec15a930 0x7fc6ec0c56e0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00c4d0 tx=0x7fc6dc00c990 comp rx=0 tx=0).stop 2024-04-14T08:40:57.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.098+0000 7fc6f451e640 1 -- 172.21.15.172:0/2942087156 shutdown_connections 2024-04-14T08:40:57.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.098+0000 7fc6f451e640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc6d006c920 0x7fc6d006ede0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.098+0000 7fc6f451e640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6ec15ebc0 0x7fc6ec15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.098+0000 7fc6f451e640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec15a930 0x7fc6ec0c56e0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.098+0000 7fc6f451e640 1 --2- 172.21.15.172:0/2942087156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6ec0ce350 0x7fc6ec0c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.098+0000 7fc6f451e640 1 -- 172.21.15.172:0/2942087156 >> 172.21.15.172:0/2942087156 conn(0x7fc6ec0b1a50 msgr2=0x7fc6ec0719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:57.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.098+0000 7fc6f451e640 1 -- 172.21.15.172:0/2942087156 shutdown_connections 2024-04-14T08:40:57.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.098+0000 7fc6f451e640 1 -- 172.21.15.172:0/2942087156 wait complete. 2024-04-14T08:40:57.111 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-14T08:40:57.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.213+0000 7f4d43098640 1 Processor -- start 2024-04-14T08:40:57.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.214+0000 7f4d43098640 1 -- start start 2024-04-14T08:40:57.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.214+0000 7f4d43098640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d3c162710 0x7f4d3c0d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:57.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.214+0000 7f4d43098640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d3c162ba0 0x7f4d3c0d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:57.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.214+0000 7f4d43098640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d3c0cefb0 con 0x7f4d3c15e430 2024-04-14T08:40:57.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.214+0000 7f4d43098640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f4d3c0cf290 con 0x7f4d3c162710 2024-04-14T08:40:57.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.214+0000 7f4d43098640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4d3c0cf570 con 0x7f4d3c162ba0 2024-04-14T08:40:57.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.215+0000 7f4d3bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d3c162710 0x7f4d3c0d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:57.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.215+0000 7f4d40e34640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4d3c15e430 0x7f4d3c162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50480/0 (socket says 172.21.15.172:50480) 2024-04-14T08:40:57.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.215+0000 7f4d3bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d3c162710 0x7f4d3c0d1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54314/0 (socket says 172.21.15.172:54314) 2024-04-14T08:40:57.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.215+0000 7f4d40e34640 1 -- 172.21.15.172:0/1762184037 learned_addr learned my addr 172.21.15.172:0/1762184037 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:57.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.215+0000 7f4d41635640 1 --2- 172.21.15.172:0/1762184037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d3c162ba0 0x7f4d3c0d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:57.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.215+0000 7f4d3bfff640 1 -- 172.21.15.172:0/1762184037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d3c162ba0 msgr2=0x7f4d3c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:57.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.215+0000 7f4d3bfff640 1 --2- 172.21.15.172:0/1762184037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d3c162ba0 0x7f4d3c0d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.215+0000 7f4d3bfff640 1 -- 172.21.15.172:0/1762184037 >> v1:172.21.15.172:6789/0 conn(0x7f4d3c15e430 legacy=0x7f4d3c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:57.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.215+0000 7f4d3bfff640 1 -- 172.21.15.172:0/1762184037 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3c0cf850 con 0x7f4d3c162710 2024-04-14T08:40:57.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.215+0000 7f4d3b7fe640 1 -- 172.21.15.172:0/1762184037 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2460498184 0 0) 0x7f4d3c0cefb0 con 0x7f4d3c15e430 2024-04-14T08:40:57.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.215+0000 7f4d41635640 1 --2- 172.21.15.172:0/1762184037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d3c162ba0 0x7f4d3c0d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:57.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.216+0000 7f4d3bfff640 1 --2- 172.21.15.172:0/1762184037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d3c162710 0x7f4d3c0d1df0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f4d280097a0 tx=0x7f4d2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22c6af4bb950dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:57.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.216+0000 7f4d3b7fe640 1 -- 172.21.15.172:0/1762184037 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28035020 con 0x7f4d3c162710 2024-04-14T08:40:57.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.216+0000 7f4d3b7fe640 1 -- 172.21.15.172:0/1762184037 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d28031830 con 0x7f4d3c162710 2024-04-14T08:40:57.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.216+0000 7f4d3b7fe640 1 -- 172.21.15.172:0/1762184037 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28031c00 con 0x7f4d3c162710 2024-04-14T08:40:57.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.216+0000 7f4d43098640 1 -- 172.21.15.172:0/1762184037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d3c162710 msgr2=0x7f4d3c0d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.216+0000 7f4d43098640 1 --2- 172.21.15.172:0/1762184037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d3c162710 0x7f4d3c0d1df0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f4d280097a0 tx=0x7f4d2802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.216+0000 7f4d43098640 1 -- 172.21.15.172:0/1762184037 shutdown_connections 2024-04-14T08:40:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.216+0000 7f4d43098640 1 --2- 172.21.15.172:0/1762184037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d3c162ba0 0x7f4d3c0d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.216+0000 7f4d43098640 1 --2- 172.21.15.172:0/1762184037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d3c162710 0x7f4d3c0d1df0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.216+0000 7f4d43098640 1 -- 172.21.15.172:0/1762184037 >> 172.21.15.172:0/1762184037 conn(0x7f4d3c09d560 msgr2=0x7f4d3c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.216+0000 7f4d43098640 1 -- 172.21.15.172:0/1762184037 shutdown_connections 2024-04-14T08:40:57.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.216+0000 7f4d43098640 1 -- 172.21.15.172:0/1762184037 wait complete. 2024-04-14T08:40:57.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.218+0000 7f4d43098640 1 Processor -- start 2024-04-14T08:40:57.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.218+0000 7f4d43098640 1 -- start start 2024-04-14T08:40:57.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.218+0000 7f4d43098640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d3c15e430 0x7f4d3c0b9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:57.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.218+0000 7f4d43098640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d3c162710 0x7f4d3c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:57.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.218+0000 7f4d43098640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d3c162ba0 0x7f4d3c1f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:57.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.218+0000 7f4d43098640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c0d4e20 con 0x7f4d3c162ba0 2024-04-14T08:40:57.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.218+0000 7f4d43098640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c15e810 con 0x7f4d3c15e430 2024-04-14T08:40:57.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.218+0000 7f4d43098640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4d3c0d0050 con 0x7f4d3c162710 2024-04-14T08:40:57.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d41635640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d3c162ba0 0x7f4d3c1f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d41635640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d3c162ba0 0x7f4d3c1f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54318/0 (socket says 172.21.15.172:54318) 2024-04-14T08:40:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d41635640 1 -- 172.21.15.172:0/1882573110 learned_addr learned my addr 172.21.15.172:0/1882573110 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d3bfff640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d3c162710 0x7f4d3c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d40e34640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d3c15e430 0x7f4d3c0b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d41635640 1 -- 172.21.15.172:0/1882573110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d3c162710 msgr2=0x7f4d3c0b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d41635640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d3c162710 0x7f4d3c0b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d41635640 1 -- 172.21.15.172:0/1882573110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d3c15e430 msgr2=0x7f4d3c0b9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d41635640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d3c15e430 0x7f4d3c0b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d41635640 1 -- 172.21.15.172:0/1882573110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3c0ce0f0 con 0x7f4d3c162ba0 2024-04-14T08:40:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d3bfff640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d3c162710 0x7f4d3c0b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d41635640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d3c162ba0 0x7f4d3c1f51f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f4d34007ef0 tx=0x7f4d3400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d397fa640 1 -- 172.21.15.172:0/1882573110 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3401a070 con 0x7f4d3c162ba0 2024-04-14T08:40:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d397fa640 1 -- 172.21.15.172:0/1882573110 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d34003b90 con 0x7f4d3c162ba0 2024-04-14T08:40:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d43098640 1 -- 172.21.15.172:0/1882573110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3c0b7fb0 con 0x7f4d3c162ba0 2024-04-14T08:40:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d397fa640 1 -- 172.21.15.172:0/1882573110 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34004030 con 0x7f4d3c162ba0 2024-04-14T08:40:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.219+0000 7f4d43098640 1 -- 172.21.15.172:0/1882573110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d3c0b8570 con 0x7f4d3c162ba0 2024-04-14T08:40:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.220+0000 7f4d43098640 1 -- 172.21.15.172:0/1882573110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d3c068590 con 0x7f4d3c162ba0 2024-04-14T08:40:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.221+0000 7f4d397fa640 1 -- 172.21.15.172:0/1882573110 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d3401d070 con 0x7f4d3c162ba0 2024-04-14T08:40:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.221+0000 7f4d397fa640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d2006c8f0 0x7f4d2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.221+0000 7f4d397fa640 1 -- 172.21.15.172:0/1882573110 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5507+0+0 (secure 0 0 0) 0x7f4d3408bc90 con 0x7f4d3c162ba0 2024-04-14T08:40:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.221+0000 7f4d40e34640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d2006c8f0 0x7f4d2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.221+0000 7f4d40e34640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d2006c8f0 0x7f4d2006edb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4d30000f60 tx=0x7f4d30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.225+0000 7f4d397fa640 1 -- 172.21.15.172:0/1882573110 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4d34010070 con 0x7f4d3c162ba0 2024-04-14T08:40:57.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.401+0000 7f4d43098640 1 -- 172.21.15.172:0/1882573110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f4d3c15dca0 con 0x7f4d3c162ba0 2024-04-14T08:40:57.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.404+0000 7f4d397fa640 1 -- 172.21.15.172:0/1882573110 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f4d34059b30 con 0x7f4d3c162ba0 2024-04-14T08:40:57.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.406+0000 7f4d43098640 1 -- 172.21.15.172:0/1882573110 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d2006c8f0 msgr2=0x7f4d2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:57.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.406+0000 7f4d43098640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d2006c8f0 0x7f4d2006edb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4d30000f60 tx=0x7f4d30009040 comp rx=0 tx=0).stop 2024-04-14T08:40:57.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.406+0000 7f4d43098640 1 -- 172.21.15.172:0/1882573110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d3c162ba0 msgr2=0x7f4d3c1f51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:57.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.406+0000 7f4d43098640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d3c162ba0 0x7f4d3c1f51f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f4d34007ef0 tx=0x7f4d3400d4a0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.407+0000 7f4d43098640 1 -- 172.21.15.172:0/1882573110 shutdown_connections 2024-04-14T08:40:57.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.407+0000 7f4d43098640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4d2006c8f0 0x7f4d2006edb0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.407+0000 7f4d43098640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d3c162ba0 0x7f4d3c1f51f0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.407+0000 7f4d43098640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d3c162710 0x7f4d3c0b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.407+0000 7f4d43098640 1 --2- 172.21.15.172:0/1882573110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d3c15e430 0x7f4d3c0b9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.407+0000 7f4d43098640 1 -- 172.21.15.172:0/1882573110 >> 172.21.15.172:0/1882573110 conn(0x7f4d3c09d560 msgr2=0x7f4d3c0b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:57.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.407+0000 7f4d43098640 1 -- 172.21.15.172:0/1882573110 shutdown_connections 2024-04-14T08:40:57.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.407+0000 7f4d43098640 1 -- 172.21.15.172:0/1882573110 wait complete. 2024-04-14T08:40:57.420 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-14T08:40:57.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.530+0000 7fc1be1cf640 1 Processor -- start 2024-04-14T08:40:57.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.530+0000 7fc1be1cf640 1 -- start start 2024-04-14T08:40:57.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1be1cf640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc1b815a690 0x7fc1b8071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:57.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1be1cf640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1b8072970 0x7fc1b80722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:57.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1be1cf640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1b80b3360 con 0x7fc1b815aa70 2024-04-14T08:40:57.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1be1cf640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc1b80b3640 con 0x7fc1b8072970 2024-04-14T08:40:57.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1be1cf640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc1b80b3920 con 0x7fc1b815a690 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1b6ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1b8072970 0x7fc1b80722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1b6ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1b8072970 0x7fc1b80722a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54330/0 (socket says 172.21.15.172:54330) 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1b6ffd640 1 -- 172.21.15.172:0/1612976183 learned_addr learned my addr 172.21.15.172:0/1612976183 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1b77fe640 1 --2- 172.21.15.172:0/1612976183 >> v2:172.21.15.177:3300/0 conn(0x7fc1b815a690 0x7fc1b8071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1b6ffd640 1 -- 172.21.15.172:0/1612976183 >> v2:172.21.15.177:3300/0 conn(0x7fc1b815a690 msgr2=0x7fc1b8071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1b6ffd640 1 --2- 172.21.15.172:0/1612976183 >> v2:172.21.15.177:3300/0 conn(0x7fc1b815a690 0x7fc1b8071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1b6ffd640 1 -- 172.21.15.172:0/1612976183 >> v1:172.21.15.177:6789/0 conn(0x7fc1b815aa70 legacy=0x7fc1b815e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1b6ffd640 1 -- 172.21.15.172:0/1612976183 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b80b3c00 con 0x7fc1b8072970 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.531+0000 7fc1b77fe640 1 --2- 172.21.15.172:0/1612976183 >> v2:172.21.15.177:3300/0 conn(0x7fc1b815a690 0x7fc1b8071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.532+0000 7fc1b6ffd640 1 --2- 172.21.15.172:0/1612976183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1b8072970 0x7fc1b80722a0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc1b806ce50 tx=0x7fc1ac02e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d908e28ac8820c4b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.532+0000 7fc1b67fc640 1 -- 172.21.15.172:0/1612976183 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac03b070 con 0x7fc1b8072970 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.532+0000 7fc1b67fc640 1 -- 172.21.15.172:0/1612976183 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1ac02ee00 con 0x7fc1b8072970 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.532+0000 7fc1b67fc640 1 -- 172.21.15.172:0/1612976183 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac036660 con 0x7fc1b8072970 2024-04-14T08:40:57.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.532+0000 7fc1be1cf640 1 -- 172.21.15.172:0/1612976183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1b8072970 msgr2=0x7fc1b80722a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:57.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.532+0000 7fc1be1cf640 1 --2- 172.21.15.172:0/1612976183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1b8072970 0x7fc1b80722a0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc1b806ce50 tx=0x7fc1ac02e050 comp rx=0 tx=0).stop 2024-04-14T08:40:57.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.532+0000 7fc1be1cf640 1 -- 172.21.15.172:0/1612976183 shutdown_connections 2024-04-14T08:40:57.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.532+0000 7fc1be1cf640 1 --2- 172.21.15.172:0/1612976183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1b8072970 0x7fc1b80722a0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.532+0000 7fc1be1cf640 1 --2- 172.21.15.172:0/1612976183 >> v2:172.21.15.177:3300/0 conn(0x7fc1b815a690 0x7fc1b8071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.532+0000 7fc1be1cf640 1 -- 172.21.15.172:0/1612976183 >> 172.21.15.172:0/1612976183 conn(0x7fc1b806ec30 msgr2=0x7fc1b806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:57.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.533+0000 7fc1be1cf640 1 -- 172.21.15.172:0/1612976183 shutdown_connections 2024-04-14T08:40:57.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.533+0000 7fc1be1cf640 1 -- 172.21.15.172:0/1612976183 wait complete. 2024-04-14T08:40:57.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.534+0000 7fc1be1cf640 1 Processor -- start 2024-04-14T08:40:57.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.534+0000 7fc1be1cf640 1 -- start start 2024-04-14T08:40:57.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.534+0000 7fc1be1cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1b8072970 0x7fc1b80b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:57.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1be1cf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1b815a690 0x7fc1b81f4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1be1cf640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1b815aa70 0x7fc1b81f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1be1cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b815c240 con 0x7fc1b8072970 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1be1cf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b80b3100 con 0x7fc1b815a690 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1be1cf640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b815c3c0 con 0x7fc1b815aa70 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1b77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1b8072970 0x7fc1b80b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1b77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1b8072970 0x7fc1b80b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54344/0 (socket says 172.21.15.172:54344) 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1b77fe640 1 -- 172.21.15.172:0/350103030 learned_addr learned my addr 172.21.15.172:0/350103030 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1b6ffd640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1b815a690 0x7fc1b81f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1b7fff640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1b815aa70 0x7fc1b81f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1b77fe640 1 -- 172.21.15.172:0/350103030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1b815aa70 msgr2=0x7fc1b81f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1b77fe640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1b815aa70 0x7fc1b81f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1b77fe640 1 -- 172.21.15.172:0/350103030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1b815a690 msgr2=0x7fc1b81f4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:57.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1b77fe640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1b815a690 0x7fc1b81f4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1b77fe640 1 -- 172.21.15.172:0/350103030 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b81f4dc0 con 0x7fc1b8072970 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1b7fff640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1b815aa70 0x7fc1b81f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1b77fe640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1b8072970 0x7fc1b80b1e40 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800c820 tx=0x7fc1a800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc19ffff640 1 -- 172.21.15.172:0/350103030 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8015070 con 0x7fc1b8072970 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc19ffff640 1 -- 172.21.15.172:0/350103030 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1a8007a30 con 0x7fc1b8072970 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1be1cf640 1 -- 172.21.15.172:0/350103030 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b81f50a0 con 0x7fc1b8072970 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc19ffff640 1 -- 172.21.15.172:0/350103030 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8007e00 con 0x7fc1b8072970 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.535+0000 7fc1be1cf640 1 -- 172.21.15.172:0/350103030 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b81501f0 con 0x7fc1b8072970 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.536+0000 7fc1be1cf640 1 -- 172.21.15.172:0/350103030 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b81506e0 con 0x7fc1b8072970 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.537+0000 7fc19ffff640 1 -- 172.21.15.172:0/350103030 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1a8018070 con 0x7fc1b8072970 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.537+0000 7fc19ffff640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1a406c8f0 0x7fc1a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.537+0000 7fc19ffff640 1 -- 172.21.15.172:0/350103030 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5507+0+0 (secure 0 0 0) 0x7fc1a808b9d0 con 0x7fc1b8072970 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.537+0000 7fc1b6ffd640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1a406c8f0 0x7fc1a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:57.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.538+0000 7fc1b6ffd640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1a406c8f0 0x7fc1a406edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac038000 tx=0x7fc1ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:57.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.542+0000 7fc19ffff640 1 -- 172.21.15.172:0/350103030 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc1a8059f50 con 0x7fc1b8072970 2024-04-14T08:40:57.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:57.786+0000 7fc1be1cf640 1 -- 172.21.15.172:0/350103030 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7fc1b8068520 con 0x7fc1b8072970 2024-04-14T08:40:58.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.098+0000 7fc19ffff640 1 -- 172.21.15.172:0/350103030 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v384) ==== 167+0+0 (secure 0 0 0) 0x7fc1a8059850 con 0x7fc1b8072970 2024-04-14T08:40:58.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.100+0000 7fc1be1cf640 1 -- 172.21.15.172:0/350103030 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1a406c8f0 msgr2=0x7fc1a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.101+0000 7fc1be1cf640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1a406c8f0 0x7fc1a406edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac038000 tx=0x7fc1ac008040 comp rx=0 tx=0).stop 2024-04-14T08:40:58.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.101+0000 7fc1be1cf640 1 -- 172.21.15.172:0/350103030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1b8072970 msgr2=0x7fc1b80b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.101+0000 7fc1be1cf640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1b8072970 0x7fc1b80b1e40 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800c820 tx=0x7fc1a800cc20 comp rx=0 tx=0).stop 2024-04-14T08:40:58.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.101+0000 7fc1be1cf640 1 -- 172.21.15.172:0/350103030 shutdown_connections 2024-04-14T08:40:58.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.101+0000 7fc1be1cf640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1a406c8f0 0x7fc1a406edb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.101+0000 7fc1be1cf640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1b815aa70 0x7fc1b81f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.101+0000 7fc1be1cf640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1b815a690 0x7fc1b81f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.101+0000 7fc1be1cf640 1 --2- 172.21.15.172:0/350103030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1b8072970 0x7fc1b80b1e40 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.101+0000 7fc1be1cf640 1 -- 172.21.15.172:0/350103030 >> 172.21.15.172:0/350103030 conn(0x7fc1b806ec30 msgr2=0x7fc1b80ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:58.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.101+0000 7fc1be1cf640 1 -- 172.21.15.172:0/350103030 shutdown_connections 2024-04-14T08:40:58.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.101+0000 7fc1be1cf640 1 -- 172.21.15.172:0/350103030 wait complete. 2024-04-14T08:40:58.113 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-14T08:40:58.113 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-14T08:40:58.113 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:40:58.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.214+0000 7f34342de640 1 Processor -- start 2024-04-14T08:40:58.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.214+0000 7f34342de640 1 -- start start 2024-04-14T08:40:58.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.215+0000 7f34342de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0b1df0 0x7f342c0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.215+0000 7f34342de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f342c0b21d0 0x7f342c15c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.215+0000 7f34342de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f342c0b3430 con 0x7f342c15c5a0 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.215+0000 7f34342de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f342c0b3700 con 0x7f342c0b21d0 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.215+0000 7f34342de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f342c0b39e0 con 0x7f342c0b1df0 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.215+0000 7f343287b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f342c15c5a0 0x7f342c0b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50494/0 (socket says 172.21.15.172:50494) 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.215+0000 7f343287b640 1 -- 172.21.15.172:0/4273877640 learned_addr learned my addr 172.21.15.172:0/4273877640 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.215+0000 7f343207a640 1 --2- 172.21.15.172:0/4273877640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0b1df0 0x7f342c0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.215+0000 7f3431879640 1 --2- 172.21.15.172:0/4273877640 >> v2:172.21.15.177:3300/0 conn(0x7f342c0b21d0 0x7f342c15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.215+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2050089146 0 0) 0x7f342c0b3430 con 0x7f342c15c5a0 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3418003520 con 0x7f342c15c5a0 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3592233425 0 0) 0x7f3418003520 con 0x7f342c15c5a0 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0b1df0 msgr2=0x7f342c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 --2- 172.21.15.172:0/4273877640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0b1df0 0x7f342c0b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 >> v2:172.21.15.177:3300/0 conn(0x7f342c0b21d0 msgr2=0x7f342c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:58.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 --2- 172.21.15.172:0/4273877640 >> v2:172.21.15.177:3300/0 conn(0x7f342c0b21d0 0x7f342c15c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c0b3cc0 con 0x7f342c15c5a0 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f3424001670 con 0x7f342c15c5a0 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 >> v1:172.21.15.172:6789/0 conn(0x7f342c15c5a0 legacy=0x7f342c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 --2- 172.21.15.172:0/4273877640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0b1df0 0x7f34180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 --2- 172.21.15.172:0/4273877640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3418003f30 0x7f3418004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 --2- 172.21.15.172:0/4273877640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3418004d40 0x7f34180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f342c0b3700 con 0x7f3418003f30 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f342c0b39e0 con 0x7f3418004d40 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f341800b9c0 con 0x7f342c0b1df0 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431879640 1 --2- 172.21.15.172:0/4273877640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3418003f30 0x7f3418004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f343207a640 1 --2- 172.21.15.172:0/4273877640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0b1df0 0x7f34180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f343287b640 1 --2- 172.21.15.172:0/4273877640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3418004d40 0x7f34180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431879640 1 -- 172.21.15.172:0/4273877640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0b1df0 msgr2=0x7f34180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431879640 1 --2- 172.21.15.172:0/4273877640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0b1df0 0x7f34180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431879640 1 -- 172.21.15.172:0/4273877640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3418004d40 msgr2=0x7f34180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431879640 1 --2- 172.21.15.172:0/4273877640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3418004d40 0x7f34180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.216+0000 7f3431879640 1 -- 172.21.15.172:0/4273877640 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3418003520 con 0x7f3418003f30 2024-04-14T08:40:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.217+0000 7f3431879640 1 --2- 172.21.15.172:0/4273877640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3418003f30 0x7f3418004800 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f341c009650 tx=0x7f3418005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e3641a65f4d63d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.217+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c03b070 con 0x7f3418003f30 2024-04-14T08:40:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.217+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341c02e8e0 con 0x7f3418003f30 2024-04-14T08:40:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.217+0000 7f3431078640 1 -- 172.21.15.172:0/4273877640 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c009690 con 0x7f3418003f30 2024-04-14T08:40:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.218+0000 7f34342de640 1 -- 172.21.15.172:0/4273877640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3418003f30 msgr2=0x7f3418004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.218+0000 7f34342de640 1 --2- 172.21.15.172:0/4273877640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3418003f30 0x7f3418004800 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f341c009650 tx=0x7f3418005830 comp rx=0 tx=0).stop 2024-04-14T08:40:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.218+0000 7f343287b640 1 -- 172.21.15.172:0/4273877640 reap_dead start 2024-04-14T08:40:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.218+0000 7f34342de640 1 -- 172.21.15.172:0/4273877640 shutdown_connections 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.218+0000 7f34342de640 1 -- 172.21.15.172:0/4273877640 >> 172.21.15.172:0/4273877640 conn(0x7f342c06ec30 msgr2=0x7f342c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.218+0000 7f34342de640 1 -- 172.21.15.172:0/4273877640 shutdown_connections 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.218+0000 7f34342de640 1 -- 172.21.15.172:0/4273877640 wait complete. 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.219+0000 7f34342de640 1 Processor -- start 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.219+0000 7f34342de640 1 -- start start 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f34342de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3418003f30 0x7f342c13d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f34342de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0aa520 0x7f342c0aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f34342de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f342c0afb40 0x7f342c0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f34342de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f342c0b4750 con 0x7f3418003f30 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f34342de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f342c15b7f0 con 0x7f342c0afb40 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f34342de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f342c0b48d0 con 0x7f342c0aa520 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f343207a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3418003f30 0x7f342c13d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f343207a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3418003f30 0x7f342c13d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54358/0 (socket says 172.21.15.172:54358) 2024-04-14T08:40:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f343207a640 1 -- 172.21.15.172:0/1895561677 learned_addr learned my addr 172.21.15.172:0/1895561677 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f3431879640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0aa520 0x7f342c0aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f343287b640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f342c0afb40 0x7f342c0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f343207a640 1 -- 172.21.15.172:0/1895561677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0aa520 msgr2=0x7f342c0aa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f343207a640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0aa520 0x7f342c0aa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f343207a640 1 -- 172.21.15.172:0/1895561677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f342c0afb40 msgr2=0x7f342c0afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f343207a640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f342c0afb40 0x7f342c0afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f343207a640 1 -- 172.21.15.172:0/1895561677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c13dd30 con 0x7f3418003f30 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f3431879640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0aa520 0x7f342c0aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f343287b640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f342c0afb40 0x7f342c0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f343207a640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3418003f30 0x7f342c13d7f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3428011e30 tx=0x7f342800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f3422ffd640 1 -- 172.21.15.172:0/1895561677 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3428015630 con 0x7f3418003f30 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f3422ffd640 1 -- 172.21.15.172:0/1895561677 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34280157d0 con 0x7f3418003f30 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f34342de640 1 -- 172.21.15.172:0/1895561677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c0aafe0 con 0x7f3418003f30 2024-04-14T08:40:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f3422ffd640 1 -- 172.21.15.172:0/1895561677 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3428015ba0 con 0x7f3418003f30 2024-04-14T08:40:58.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.220+0000 7f34342de640 1 -- 172.21.15.172:0/1895561677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c0ab5a0 con 0x7f3418003f30 2024-04-14T08:40:58.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.222+0000 7f34342de640 1 -- 172.21.15.172:0/1895561677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c068520 con 0x7f3418003f30 2024-04-14T08:40:58.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.222+0000 7f3422ffd640 1 -- 172.21.15.172:0/1895561677 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f342801d400 con 0x7f3418003f30 2024-04-14T08:40:58.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.222+0000 7f3422ffd640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f341006c8f0 0x7f341006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.222+0000 7f3422ffd640 1 -- 172.21.15.172:0/1895561677 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5507+0+0 (secure 0 0 0) 0x7f342808cba0 con 0x7f3418003f30 2024-04-14T08:40:58.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.222+0000 7f3431879640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f341006c8f0 0x7f341006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.223+0000 7f3431879640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f341006c8f0 0x7f341006edb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f341c038000 tx=0x7f341c032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:58.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.227+0000 7f3422ffd640 1 -- 172.21.15.172:0/1895561677 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3428019030 con 0x7f3418003f30 2024-04-14T08:40:58.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.402+0000 7f34342de640 1 -- 172.21.15.172:0/1895561677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f342c0b1870 con 0x7f3418003f30 2024-04-14T08:40:58.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.403+0000 7f3422ffd640 1 -- 172.21.15.172:0/1895561677 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+13620 (secure 0 0 0) 0x7f3428010040 con 0x7f3418003f30 2024-04-14T08:40:58.404 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:58.404 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":93,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:40:57.093520+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6835/3091771014":"2024-04-15T08:40:54.086728+0000","172.21.15.172:6837/1386685680":"2024-04-15T08:40:10.107666+0000","172.21.15.172:6837/3091771014":"2024-04-15T08:40:54.086728+0000","172.21.15.172:6835/1386685680":"2024-04-15T08:40:10.107666+0000","172.21.15.177:6835/1124731311":"2024-04-15T08:38:58.388334+0000","172.21.15.177:6833/1124731311":"2024-04-15T08:38:58.388334+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:40:58.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.405+0000 7f34342de640 1 -- 172.21.15.172:0/1895561677 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f341006c8f0 msgr2=0x7f341006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.405+0000 7f34342de640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f341006c8f0 0x7f341006edb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f341c038000 tx=0x7f341c032000 comp rx=0 tx=0).stop 2024-04-14T08:40:58.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.405+0000 7f34342de640 1 -- 172.21.15.172:0/1895561677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3418003f30 msgr2=0x7f342c13d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.405+0000 7f34342de640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3418003f30 0x7f342c13d7f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3428011e30 tx=0x7f342800f580 comp rx=0 tx=0).stop 2024-04-14T08:40:58.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.405+0000 7f34342de640 1 -- 172.21.15.172:0/1895561677 shutdown_connections 2024-04-14T08:40:58.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.405+0000 7f34342de640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f341006c8f0 0x7f341006edb0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.405+0000 7f34342de640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f342c0afb40 0x7f342c0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.405+0000 7f34342de640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f342c0aa520 0x7f342c0aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.405+0000 7f34342de640 1 --2- 172.21.15.172:0/1895561677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3418003f30 0x7f342c13d7f0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.405+0000 7f34342de640 1 -- 172.21.15.172:0/1895561677 >> 172.21.15.172:0/1895561677 conn(0x7f342c06ec30 msgr2=0x7f342c0728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:58.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.405+0000 7f34342de640 1 -- 172.21.15.172:0/1895561677 shutdown_connections 2024-04-14T08:40:58.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.405+0000 7f34342de640 1 -- 172.21.15.172:0/1895561677 wait complete. 2024-04-14T08:40:58.417 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:40:58.417 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:40:58.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.520+0000 7fd2704c0640 1 Processor -- start 2024-04-14T08:40:58.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.520+0000 7fd2704c0640 1 -- start start 2024-04-14T08:40:58.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd2704c0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2680ccca0 0x7fd26806fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd2704c0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd268073090 0x7fd2680b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd2704c0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2680734f0 con 0x7fd26806b520 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd2704c0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fd2680ceee0 con 0x7fd268073090 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd2704c0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd268070150 con 0x7fd2680ccca0 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd26da5b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd268073090 0x7fd2680b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd26da5b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd268073090 0x7fd2680b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54372/0 (socket says 172.21.15.172:54372) 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd26da5b640 1 -- 172.21.15.172:0/2857351503 learned_addr learned my addr 172.21.15.172:0/2857351503 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd26e25c640 1 --2- 172.21.15.172:0/2857351503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2680ccca0 0x7fd26806fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd26da5b640 1 -- 172.21.15.172:0/2857351503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2680ccca0 msgr2=0x7fd26806fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd26da5b640 1 --2- 172.21.15.172:0/2857351503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2680ccca0 0x7fd26806fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd26da5b640 1 -- 172.21.15.172:0/2857351503 >> v1:172.21.15.172:6789/0 conn(0x7fd26806b520 legacy=0x7fd26806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd26da5b640 1 -- 172.21.15.172:0/2857351503 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2680722d0 con 0x7fd268073090 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd26d25a640 1 -- 172.21.15.172:0/2857351503 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3576089215 0 0) 0x7fd2680734f0 con 0x7fd26806b520 2024-04-14T08:40:58.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.521+0000 7fd26e25c640 1 --2- 172.21.15.172:0/2857351503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2680ccca0 0x7fd26806fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:58.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.522+0000 7fd26da5b640 1 --2- 172.21.15.172:0/2857351503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268073090 0x7fd2680b0190 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fd258009870 tx=0x7fd25802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2c91ed139239e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:58.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.522+0000 7fd26d25a640 1 -- 172.21.15.172:0/2857351503 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258035020 con 0x7fd268073090 2024-04-14T08:40:58.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.522+0000 7fd26d25a640 1 -- 172.21.15.172:0/2857351503 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd258031640 con 0x7fd268073090 2024-04-14T08:40:58.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.522+0000 7fd26d25a640 1 -- 172.21.15.172:0/2857351503 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25802d040 con 0x7fd268073090 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.523+0000 7fd2704c0640 1 -- 172.21.15.172:0/2857351503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268073090 msgr2=0x7fd2680b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.523+0000 7fd2704c0640 1 --2- 172.21.15.172:0/2857351503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268073090 0x7fd2680b0190 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fd258009870 tx=0x7fd25802b8f0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.523+0000 7fd2704c0640 1 -- 172.21.15.172:0/2857351503 shutdown_connections 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.523+0000 7fd2704c0640 1 --2- 172.21.15.172:0/2857351503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268073090 0x7fd2680b0190 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.523+0000 7fd2704c0640 1 --2- 172.21.15.172:0/2857351503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2680ccca0 0x7fd26806fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.523+0000 7fd2704c0640 1 -- 172.21.15.172:0/2857351503 >> 172.21.15.172:0/2857351503 conn(0x7fd2680cda00 msgr2=0x7fd2680cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.523+0000 7fd2704c0640 1 -- 172.21.15.172:0/2857351503 shutdown_connections 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.523+0000 7fd2704c0640 1 -- 172.21.15.172:0/2857351503 wait complete. 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.524+0000 7fd2704c0640 1 Processor -- start 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.524+0000 7fd2704c0640 1 -- start start 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd2704c0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd26806b520 0x7fd2681f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd2704c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268073090 0x7fd2681595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd2704c0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2680ccca0 0x7fd2681f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd2704c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd2680cd080 con 0x7fd268073090 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd2704c0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd268072890 con 0x7fd26806b520 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd2704c0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd26806b950 con 0x7fd2680ccca0 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd26da5b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268073090 0x7fd2681595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd26da5b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268073090 0x7fd2681595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54374/0 (socket says 172.21.15.172:54374) 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd26da5b640 1 -- 172.21.15.172:0/1679249502 learned_addr learned my addr 172.21.15.172:0/1679249502 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd26ea5d640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2680ccca0 0x7fd2681f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd26e25c640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd26806b520 0x7fd2681f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd26da5b640 1 -- 172.21.15.172:0/1679249502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2680ccca0 msgr2=0x7fd2681f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd26da5b640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2680ccca0 0x7fd2681f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd26da5b640 1 -- 172.21.15.172:0/1679249502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd26806b520 msgr2=0x7fd2681f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd26da5b640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd26806b520 0x7fd2681f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd26da5b640 1 -- 172.21.15.172:0/1679249502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2680c4c50 con 0x7fd268073090 2024-04-14T08:40:58.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd26ea5d640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2680ccca0 0x7fd2681f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:58.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd26da5b640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268073090 0x7fd2681595a0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fd2580090b0 tx=0x7fd2580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:58.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd25effd640 1 -- 172.21.15.172:0/1679249502 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258040070 con 0x7fd268073090 2024-04-14T08:40:58.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd25effd640 1 -- 172.21.15.172:0/1679249502 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2580399f0 con 0x7fd268073090 2024-04-14T08:40:58.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd2704c0640 1 -- 172.21.15.172:0/1679249502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2680c4f30 con 0x7fd268073090 2024-04-14T08:40:58.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.525+0000 7fd25effd640 1 -- 172.21.15.172:0/1679249502 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258035040 con 0x7fd268073090 2024-04-14T08:40:58.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.526+0000 7fd2704c0640 1 -- 172.21.15.172:0/1679249502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2680c54f0 con 0x7fd268073090 2024-04-14T08:40:58.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.527+0000 7fd2704c0640 1 -- 172.21.15.172:0/1679249502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd268068590 con 0x7fd268073090 2024-04-14T08:40:58.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.527+0000 7fd25effd640 1 -- 172.21.15.172:0/1679249502 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd258038020 con 0x7fd268073090 2024-04-14T08:40:58.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.527+0000 7fd25effd640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd24c06c8f0 0x7fd24c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.528+0000 7fd26e25c640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd24c06c8f0 0x7fd24c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.528+0000 7fd25effd640 1 -- 172.21.15.172:0/1679249502 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5507+0+0 (secure 0 0 0) 0x7fd2580ad490 con 0x7fd268073090 2024-04-14T08:40:58.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.528+0000 7fd26e25c640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd24c06c8f0 0x7fd24c06edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd264003fd0 tx=0x7fd2640039f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:58.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.532+0000 7fd25effd640 1 -- 172.21.15.172:0/1679249502 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd25807ba10 con 0x7fd268073090 2024-04-14T08:40:58.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.704+0000 7fd2704c0640 1 -- 172.21.15.172:0/1679249502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd26806c120 con 0x7fd268073090 2024-04-14T08:40:58.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.705+0000 7fd25effd640 1 -- 172.21.15.172:0/1679249502 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fd25807b310 con 0x7fd268073090 2024-04-14T08:40:58.705 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:40:58.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.707+0000 7fd2704c0640 1 -- 172.21.15.172:0/1679249502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd24c06c8f0 msgr2=0x7fd24c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.707+0000 7fd2704c0640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd24c06c8f0 0x7fd24c06edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd264003fd0 tx=0x7fd2640039f0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.707+0000 7fd2704c0640 1 -- 172.21.15.172:0/1679249502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268073090 msgr2=0x7fd2681595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.707+0000 7fd2704c0640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268073090 0x7fd2681595a0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fd2580090b0 tx=0x7fd2580046c0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.707+0000 7fd2704c0640 1 -- 172.21.15.172:0/1679249502 shutdown_connections 2024-04-14T08:40:58.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.707+0000 7fd2704c0640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd24c06c8f0 0x7fd24c06edb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.707+0000 7fd2704c0640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2680ccca0 0x7fd2681f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.707+0000 7fd2704c0640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd268073090 0x7fd2681595a0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.707+0000 7fd2704c0640 1 --2- 172.21.15.172:0/1679249502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd26806b520 0x7fd2681f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.707+0000 7fd2704c0640 1 -- 172.21.15.172:0/1679249502 >> 172.21.15.172:0/1679249502 conn(0x7fd2680cda00 msgr2=0x7fd268161e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:58.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.707+0000 7fd2704c0640 1 -- 172.21.15.172:0/1679249502 shutdown_connections 2024-04-14T08:40:58.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.707+0000 7fd2704c0640 1 -- 172.21.15.172:0/1679249502 wait complete. 2024-04-14T08:40:58.718 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-14T08:40:58.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.818+0000 7f01b3836640 1 Processor -- start 2024-04-14T08:40:58.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.819+0000 7f01b3836640 1 -- start start 2024-04-14T08:40:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.819+0000 7f01b3836640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f01ac0cedd0 0x7f01ac1694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.819+0000 7f01b3836640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac0cf320 con 0x7f01ac0d2830 2024-04-14T08:40:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.819+0000 7f01b3836640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac0cf600 con 0x7f01ac15e890 2024-04-14T08:40:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.819+0000 7f01b3836640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f01ac0cff30 con 0x7f01ac0cedd0 2024-04-14T08:40:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.820+0000 7f01b15d2640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f01ac0d2830 0x7f01ac0d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50510/0 (socket says 172.21.15.172:50510) 2024-04-14T08:40:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.820+0000 7f01b15d2640 1 -- 172.21.15.172:0/2169328457 learned_addr learned my addr 172.21.15.172:0/2169328457 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.820+0000 7f01b1dd3640 1 --2- 172.21.15.172:0/2169328457 >> v2:172.21.15.177:3300/0 conn(0x7f01ac0cedd0 0x7f01ac1694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.820+0000 7f01a3fff640 1 -- 172.21.15.172:0/2169328457 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1894611041 0 0) 0x7f01ac0cf320 con 0x7f01ac0d2830 2024-04-14T08:40:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.820+0000 7f01a3fff640 1 -- 172.21.15.172:0/2169328457 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0198003520 con 0x7f01ac0d2830 2024-04-14T08:40:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.820+0000 7f01b1dd3640 1 -- 172.21.15.172:0/2169328457 >> v1:172.21.15.177:6789/0 conn(0x7f01ac15e890 legacy=0x7f01ac0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.820+0000 7f01b1dd3640 1 -- 172.21.15.172:0/2169328457 >> v1:172.21.15.172:6789/0 conn(0x7f01ac0d2830 legacy=0x7f01ac0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.820+0000 7f01b1dd3640 1 -- 172.21.15.172:0/2169328457 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac0cf8e0 con 0x7f01ac0cedd0 2024-04-14T08:40:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.821+0000 7f01b1dd3640 1 --2- 172.21.15.172:0/2169328457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ac0cedd0 0x7f01ac1694d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009d90 tx=0x7f01a402bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=298424ba9f010a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.821+0000 7f01a3fff640 1 -- 172.21.15.172:0/2169328457 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4039070 con 0x7f01ac0cedd0 2024-04-14T08:40:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.821+0000 7f01a3fff640 1 -- 172.21.15.172:0/2169328457 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a402cd80 con 0x7f01ac0cedd0 2024-04-14T08:40:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.821+0000 7f01a3fff640 1 -- 172.21.15.172:0/2169328457 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4034640 con 0x7f01ac0cedd0 2024-04-14T08:40:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.821+0000 7f01b3836640 1 -- 172.21.15.172:0/2169328457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ac0cedd0 msgr2=0x7f01ac1694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.821+0000 7f01b3836640 1 --2- 172.21.15.172:0/2169328457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ac0cedd0 0x7f01ac1694d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009d90 tx=0x7f01a402bfd0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.821+0000 7f01b3836640 1 -- 172.21.15.172:0/2169328457 shutdown_connections 2024-04-14T08:40:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.821+0000 7f01b3836640 1 --2- 172.21.15.172:0/2169328457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ac0cedd0 0x7f01ac1694d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.821+0000 7f01b3836640 1 -- 172.21.15.172:0/2169328457 >> 172.21.15.172:0/2169328457 conn(0x7f01ac09d590 msgr2=0x7f01ac0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.822+0000 7f01b3836640 1 -- 172.21.15.172:0/2169328457 shutdown_connections 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.822+0000 7f01b3836640 1 -- 172.21.15.172:0/2169328457 wait complete. 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.823+0000 7f01b3836640 1 Processor -- start 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.823+0000 7f01b3836640 1 -- start start 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.823+0000 7f01b3836640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01ac0cedd0 0x7f01ac0cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.823+0000 7f01b3836640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ac0d2830 0x7f01ac1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.823+0000 7f01b3836640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01ac15e890 0x7f01ac15d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.823+0000 7f01b3836640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac0b4720 con 0x7f01ac0cedd0 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.823+0000 7f01b3836640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac0d2350 con 0x7f01ac0d2830 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.823+0000 7f01b3836640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f01ac0b48a0 con 0x7f01ac15e890 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.823+0000 7f01b15d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01ac0cedd0 0x7f01ac0cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01b15d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01ac0cedd0 0x7f01ac0cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54376/0 (socket says 172.21.15.172:54376) 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01b15d2640 1 -- 172.21.15.172:0/2453040649 learned_addr learned my addr 172.21.15.172:0/2453040649 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01b1dd3640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01ac15e890 0x7f01ac15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01b0dd1640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ac0d2830 0x7f01ac1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01b15d2640 1 -- 172.21.15.172:0/2453040649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01ac15e890 msgr2=0x7f01ac15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01b15d2640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01ac15e890 0x7f01ac15d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01b15d2640 1 -- 172.21.15.172:0/2453040649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ac0d2830 msgr2=0x7f01ac1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01b15d2640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ac0d2830 0x7f01ac1542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01b15d2640 1 -- 172.21.15.172:0/2453040649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac154b20 con 0x7f01ac0cedd0 2024-04-14T08:40:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01b15d2640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01ac0cedd0 0x7f01ac0cb320 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f01a800c410 tx=0x7f01a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01a1ffb640 1 -- 172.21.15.172:0/2453040649 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8017070 con 0x7f01ac0cedd0 2024-04-14T08:40:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01a1ffb640 1 -- 172.21.15.172:0/2453040649 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a8005ce0 con 0x7f01ac0cedd0 2024-04-14T08:40:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01a1ffb640 1 -- 172.21.15.172:0/2453040649 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a80072f0 con 0x7f01ac0cedd0 2024-04-14T08:40:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01b3836640 1 -- 172.21.15.172:0/2453040649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac0cdb70 con 0x7f01ac0cedd0 2024-04-14T08:40:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.824+0000 7f01b3836640 1 -- 172.21.15.172:0/2453040649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac0cb900 con 0x7f01ac0cedd0 2024-04-14T08:40:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.825+0000 7f01b3836640 1 -- 172.21.15.172:0/2453040649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac09da60 con 0x7f01ac0cedd0 2024-04-14T08:40:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.826+0000 7f01a1ffb640 1 -- 172.21.15.172:0/2453040649 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01a801a070 con 0x7f01ac0cedd0 2024-04-14T08:40:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.826+0000 7f01a1ffb640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f019006c8f0 0x7f019006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.826+0000 7f01a1ffb640 1 -- 172.21.15.172:0/2453040649 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5507+0+0 (secure 0 0 0) 0x7f01a808add0 con 0x7f01ac0cedd0 2024-04-14T08:40:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.826+0000 7f01b0dd1640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f019006c8f0 0x7f019006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.827+0000 7f01b0dd1640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f019006c8f0 0x7f019006edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f01ac1618c0 tx=0x7f019c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:58.831+0000 7f01a1ffb640 1 -- 172.21.15.172:0/2453040649 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f01ac09da60 con 0x7f01ac0cedd0 2024-04-14T08:40:59.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.071+0000 7f01b3836640 1 -- 172.21.15.172:0/2453040649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f01ac068520 con 0x7f01ac0cedd0 2024-04-14T08:40:59.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.099+0000 7f01a1ffb640 1 -- 172.21.15.172:0/2453040649 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v385) ==== 149+0+0 (secure 0 0 0) 0x7f01a808c0d0 con 0x7f01ac0cedd0 2024-04-14T08:40:59.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.101+0000 7f01b3836640 1 -- 172.21.15.172:0/2453040649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f019006c8f0 msgr2=0x7f019006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.101+0000 7f01b3836640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f019006c8f0 0x7f019006edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f01ac1618c0 tx=0x7f019c009040 comp rx=0 tx=0).stop 2024-04-14T08:40:59.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.101+0000 7f01b3836640 1 -- 172.21.15.172:0/2453040649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01ac0cedd0 msgr2=0x7f01ac0cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.101+0000 7f01b3836640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01ac0cedd0 0x7f01ac0cb320 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f01a800c410 tx=0x7f01a800c8d0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.102+0000 7f01b3836640 1 -- 172.21.15.172:0/2453040649 shutdown_connections 2024-04-14T08:40:59.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.102+0000 7f01b3836640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f019006c8f0 0x7f019006edb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.102+0000 7f01b3836640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01ac15e890 0x7f01ac15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.102+0000 7f01b3836640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ac0d2830 0x7f01ac1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.102+0000 7f01b3836640 1 --2- 172.21.15.172:0/2453040649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01ac0cedd0 0x7f01ac0cb320 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.102+0000 7f01b3836640 1 -- 172.21.15.172:0/2453040649 >> 172.21.15.172:0/2453040649 conn(0x7f01ac09d590 msgr2=0x7f01ac074a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:59.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.102+0000 7f01b3836640 1 -- 172.21.15.172:0/2453040649 shutdown_connections 2024-04-14T08:40:59.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.102+0000 7f01b3836640 1 -- 172.21.15.172:0/2453040649 wait complete. 2024-04-14T08:40:59.113 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-14T08:40:59.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.218+0000 7fe45315e640 1 Processor -- start 2024-04-14T08:40:59.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.218+0000 7fe45315e640 1 -- start start 2024-04-14T08:40:59.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.219+0000 7fe45315e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe44c15f460 0x7fe44c0b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.219+0000 7fe45315e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c0b3bf0 0x7fe44c06b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.219+0000 7fe45315e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe44c15edd0 con 0x7fe44c06ba60 2024-04-14T08:40:59.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.219+0000 7fe45315e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe44c141960 con 0x7fe44c15f460 2024-04-14T08:40:59.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.219+0000 7fe45315e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe44c06c100 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.219+0000 7fe44bfff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c0b3bf0 0x7fe44c06b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.219+0000 7fe450efa640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe44c15f460 0x7fe44c0b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.219+0000 7fe4516fb640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fe44c06ba60 0x7fe44c162030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:53682/0 (socket says 172.21.15.172:53682) 2024-04-14T08:40:59.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.219+0000 7fe4516fb640 1 -- 172.21.15.172:0/394728878 learned_addr learned my addr 172.21.15.172:0/394728878 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:59.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.220+0000 7fe44b7fe640 1 -- 172.21.15.172:0/394728878 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3791509681 0 0) 0x7fe44c15edd0 con 0x7fe44c06ba60 2024-04-14T08:40:59.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.220+0000 7fe44b7fe640 1 -- 172.21.15.172:0/394728878 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe43c003520 con 0x7fe44c06ba60 2024-04-14T08:40:59.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.220+0000 7fe44bfff640 1 -- 172.21.15.172:0/394728878 >> v2:172.21.15.177:3300/0 conn(0x7fe44c15f460 msgr2=0x7fe44c0b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:59.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.220+0000 7fe44bfff640 1 --2- 172.21.15.172:0/394728878 >> v2:172.21.15.177:3300/0 conn(0x7fe44c15f460 0x7fe44c0b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.220+0000 7fe44bfff640 1 -- 172.21.15.172:0/394728878 >> v1:172.21.15.177:6789/0 conn(0x7fe44c06ba60 legacy=0x7fe44c162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.220+0000 7fe44bfff640 1 -- 172.21.15.172:0/394728878 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c06c3e0 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.220+0000 7fe450efa640 1 --2- 172.21.15.172:0/394728878 >> v2:172.21.15.177:3300/0 conn(0x7fe44c15f460 0x7fe44c0b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:40:59.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.220+0000 7fe44bfff640 1 --2- 172.21.15.172:0/394728878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c0b3bf0 0x7fe44c06b520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe4380097a0 tx=0x7fe43802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf1f9cdf0c96e8c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:59.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.221+0000 7fe44b7fe640 1 -- 172.21.15.172:0/394728878 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4380315a0 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.221+0000 7fe44b7fe640 1 -- 172.21.15.172:0/394728878 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe438037070 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.221+0000 7fe44b7fe640 1 -- 172.21.15.172:0/394728878 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438031940 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.221+0000 7fe45315e640 1 -- 172.21.15.172:0/394728878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c0b3bf0 msgr2=0x7fe44c06b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.221+0000 7fe45315e640 1 --2- 172.21.15.172:0/394728878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c0b3bf0 0x7fe44c06b520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe4380097a0 tx=0x7fe43802b840 comp rx=0 tx=0).stop 2024-04-14T08:40:59.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.221+0000 7fe45315e640 1 -- 172.21.15.172:0/394728878 shutdown_connections 2024-04-14T08:40:59.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.221+0000 7fe45315e640 1 --2- 172.21.15.172:0/394728878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c0b3bf0 0x7fe44c06b520 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.221+0000 7fe45315e640 1 --2- 172.21.15.172:0/394728878 >> v2:172.21.15.177:3300/0 conn(0x7fe44c15f460 0x7fe44c0b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.221+0000 7fe45315e640 1 -- 172.21.15.172:0/394728878 >> 172.21.15.172:0/394728878 conn(0x7fe44c0d0780 msgr2=0x7fe44c0d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:59.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.221+0000 7fe45315e640 1 -- 172.21.15.172:0/394728878 shutdown_connections 2024-04-14T08:40:59.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.221+0000 7fe45315e640 1 -- 172.21.15.172:0/394728878 wait complete. 2024-04-14T08:40:59.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.222+0000 7fe45315e640 1 Processor -- start 2024-04-14T08:40:59.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.222+0000 7fe45315e640 1 -- start start 2024-04-14T08:40:59.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.223+0000 7fe45315e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c06ba60 0x7fe44c158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.223+0000 7fe45315e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe44c0b3bf0 0x7fe44c15a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.223+0000 7fe45315e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44c15f460 0x7fe44c15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.223+0000 7fe45315e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c15f840 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.223+0000 7fe45315e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c06c9a0 con 0x7fe44c15f460 2024-04-14T08:40:59.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.223+0000 7fe45315e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe44c15f9c0 con 0x7fe44c06ba60 2024-04-14T08:40:59.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.223+0000 7fe44bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe44c0b3bf0 0x7fe44c15a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.223+0000 7fe44bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe44c0b3bf0 0x7fe44c15a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54380/0 (socket says 172.21.15.172:54380) 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.223+0000 7fe44bfff640 1 -- 172.21.15.172:0/1089293975 learned_addr learned my addr 172.21.15.172:0/1089293975 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.223+0000 7fe450efa640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c06ba60 0x7fe44c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.224+0000 7fe4516fb640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44c15f460 0x7fe44c15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.224+0000 7fe44bfff640 1 -- 172.21.15.172:0/1089293975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c06ba60 msgr2=0x7fe44c158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.224+0000 7fe44bfff640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c06ba60 0x7fe44c158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.224+0000 7fe44bfff640 1 -- 172.21.15.172:0/1089293975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44c15f460 msgr2=0x7fe44c15ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.224+0000 7fe44bfff640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44c15f460 0x7fe44c15ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.224+0000 7fe44bfff640 1 -- 172.21.15.172:0/1089293975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c158c10 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.224+0000 7fe44bfff640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe44c0b3bf0 0x7fe44c15a6f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fe438000c00 tx=0x7fe4380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.224+0000 7fe4497fa640 1 -- 172.21.15.172:0/1089293975 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438032630 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.224+0000 7fe4497fa640 1 -- 172.21.15.172:0/1089293975 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4380327d0 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.224+0000 7fe45315e640 1 -- 172.21.15.172:0/1089293975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c158ef0 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.224+0000 7fe4497fa640 1 -- 172.21.15.172:0/1089293975 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438032bc0 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.224+0000 7fe45315e640 1 -- 172.21.15.172:0/1089293975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c1585b0 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.225+0000 7fe45315e640 1 -- 172.21.15.172:0/1089293975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c068590 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.226+0000 7fe4497fa640 1 -- 172.21.15.172:0/1089293975 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe43804d3f0 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.226+0000 7fe4497fa640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe43006c8f0 0x7fe43006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.226+0000 7fe4497fa640 1 -- 172.21.15.172:0/1089293975 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5507+0+0 (secure 0 0 0) 0x7fe43802f030 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.226+0000 7fe450efa640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe43006c8f0 0x7fe43006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.227+0000 7fe450efa640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe43006c8f0 0x7fe43006edb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe44c157400 tx=0x7fe440009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:59.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.231+0000 7fe4497fa640 1 -- 172.21.15.172:0/1089293975 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe438040070 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.404+0000 7fe45315e640 1 -- 172.21.15.172:0/1089293975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe44c0afd10 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.407+0000 7fe4497fa640 1 -- 172.21.15.172:0/1089293975 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fe4380ad0c0 con 0x7fe44c0b3bf0 2024-04-14T08:40:59.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.410+0000 7fe45315e640 1 -- 172.21.15.172:0/1089293975 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe43006c8f0 msgr2=0x7fe43006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.410+0000 7fe45315e640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe43006c8f0 0x7fe43006edb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe44c157400 tx=0x7fe440009190 comp rx=0 tx=0).stop 2024-04-14T08:40:59.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.410+0000 7fe45315e640 1 -- 172.21.15.172:0/1089293975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe44c0b3bf0 msgr2=0x7fe44c15a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.410+0000 7fe45315e640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe44c0b3bf0 0x7fe44c15a6f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fe438000c00 tx=0x7fe4380046c0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.410+0000 7fe45315e640 1 -- 172.21.15.172:0/1089293975 shutdown_connections 2024-04-14T08:40:59.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.410+0000 7fe45315e640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe43006c8f0 0x7fe43006edb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.410+0000 7fe45315e640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe44c15f460 0x7fe44c15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.410+0000 7fe45315e640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe44c0b3bf0 0x7fe44c15a6f0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.410+0000 7fe45315e640 1 --2- 172.21.15.172:0/1089293975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe44c06ba60 0x7fe44c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.410+0000 7fe45315e640 1 -- 172.21.15.172:0/1089293975 >> 172.21.15.172:0/1089293975 conn(0x7fe44c0d0780 msgr2=0x7fe44c06dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:59.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.410+0000 7fe45315e640 1 -- 172.21.15.172:0/1089293975 shutdown_connections 2024-04-14T08:40:59.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.410+0000 7fe45315e640 1 -- 172.21.15.172:0/1089293975 wait complete. 2024-04-14T08:40:59.421 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:40:59.422 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:40:59.459 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:40:59.508 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:40:59.508 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:40:59.508 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:40:59.524 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:40:59.525 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:40:59.525 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:40:59.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:40:59.543 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:40:59.543 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:40:59.580 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:40:59.580 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:40:59.580 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:40:59.600 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:40:59.600 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-14T08:40:59.600 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:40:59.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.721+0000 7fe9ae3cf640 1 Processor -- start 2024-04-14T08:40:59.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.721+0000 7fe9ae3cf640 1 -- start start 2024-04-14T08:40:59.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9ae3cf640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe9a806b520 0x7fe9a806fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9ae3cf640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a80b3dd0 0x7fe9a806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9ae3cf640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9a806be40 con 0x7fe9a806e130 2024-04-14T08:40:59.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9ae3cf640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a8072d80 con 0x7fe9a806b520 2024-04-14T08:40:59.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9ae3cf640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a80b4540 con 0x7fe9a80b3dd0 2024-04-14T08:40:59.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9ac96c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe9a806e130 0x7fe9a806e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58910/0 (socket says 172.21.15.172:58910) 2024-04-14T08:40:59.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9ac96c640 1 -- 172.21.15.172:0/2984768536 learned_addr learned my addr 172.21.15.172:0/2984768536 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:40:59.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9a77fe640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a80b3dd0 0x7fe9a806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4229682583 0 0) 0x7fe9a806be40 con 0x7fe9a806e130 2024-04-14T08:40:59.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9a7fff640 1 --2- 172.21.15.172:0/2984768536 >> v2:172.21.15.177:3300/0 conn(0x7fe9a806b520 0x7fe9a806fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe990003520 con 0x7fe9a806e130 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1958803245 0 0) 0x7fe990003520 con 0x7fe9a806e130 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a80b3dd0 msgr2=0x7fe9a806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9a6ffd640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a80b3dd0 0x7fe9a806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 >> v2:172.21.15.177:3300/0 conn(0x7fe9a806b520 msgr2=0x7fe9a806fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9a6ffd640 1 --2- 172.21.15.172:0/2984768536 >> v2:172.21.15.177:3300/0 conn(0x7fe9a806b520 0x7fe9a806fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a8072450 con 0x7fe9a806e130 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.722+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fe9a00019f0 con 0x7fe9a806e130 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 >> v1:172.21.15.172:6789/0 conn(0x7fe9a806e130 legacy=0x7fe9a806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9a6ffd640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9900039b0 0x7fe990003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9a6ffd640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a80b3dd0 0x7fe990004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9a6ffd640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe990004d40 0x7fe9900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8072d80 con 0x7fe990004d40 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a80b4540 con 0x7fe9900039b0 2024-04-14T08:40:59.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe99000b9c0 con 0x7fe9a80b3dd0 2024-04-14T08:40:59.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9ac96c640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe990004d40 0x7fe9900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9a7fff640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9900039b0 0x7fe990003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9a77fe640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a80b3dd0 0x7fe990004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9ac96c640 1 -- 172.21.15.172:0/2984768536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a80b3dd0 msgr2=0x7fe990004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:59.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9ac96c640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a80b3dd0 0x7fe990004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9ac96c640 1 -- 172.21.15.172:0/2984768536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9900039b0 msgr2=0x7fe990003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:40:59.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9ac96c640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9900039b0 0x7fe990003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9ac96c640 1 -- 172.21.15.172:0/2984768536 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe990003520 con 0x7fe990004d40 2024-04-14T08:40:59.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.723+0000 7fe9a77fe640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a80b3dd0 0x7fe990004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:40:59.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.724+0000 7fe9ac96c640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe990004d40 0x7fe9900051c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000a8b0 tx=0x7fe990005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf15193ba8371d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:59.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.724+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000a4d0 con 0x7fe990004d40 2024-04-14T08:40:59.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.724+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a00072d0 con 0x7fe990004d40 2024-04-14T08:40:59.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.724+0000 7fe9a6ffd640 1 -- 172.21.15.172:0/2984768536 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a00076c0 con 0x7fe990004d40 2024-04-14T08:40:59.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.724+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/2984768536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe990004d40 msgr2=0x7fe9900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.724+0000 7fe9ae3cf640 1 --2- 172.21.15.172:0/2984768536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe990004d40 0x7fe9900051c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000a8b0 tx=0x7fe990005830 comp rx=0 tx=0).stop 2024-04-14T08:40:59.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.724+0000 7fe9ac96c640 1 -- 172.21.15.172:0/2984768536 reap_dead start 2024-04-14T08:40:59.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.724+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/2984768536 shutdown_connections 2024-04-14T08:40:59.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.724+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/2984768536 >> 172.21.15.172:0/2984768536 conn(0x7fe9a80d0f20 msgr2=0x7fe9a80d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:59.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.724+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/2984768536 shutdown_connections 2024-04-14T08:40:59.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.724+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/2984768536 wait complete. 2024-04-14T08:40:59.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.726+0000 7fe9ae3cf640 1 Processor -- start 2024-04-14T08:40:59.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.726+0000 7fe9ae3cf640 1 -- start start 2024-04-14T08:40:59.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.726+0000 7fe9ae3cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe990004d40 0x7fe9a81599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9ae3cf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9a8161080 0x7fe9a8144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9ae3cf640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a815b330 0x7fe9a815b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9ae3cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a80b4250 con 0x7fe990004d40 2024-04-14T08:40:59.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9ae3cf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8072a10 con 0x7fe9a8161080 2024-04-14T08:40:59.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9ae3cf640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a8073210 con 0x7fe9a815b330 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe990004d40 0x7fe9a81599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe990004d40 0x7fe9a81599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36320/0 (socket says 172.21.15.172:36320) 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a7fff640 1 -- 172.21.15.172:0/1095461675 learned_addr learned my addr 172.21.15.172:0/1095461675 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9ac96c640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a815b330 0x7fe9a815b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a77fe640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9a8161080 0x7fe9a8144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a7fff640 1 -- 172.21.15.172:0/1095461675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a815b330 msgr2=0x7fe9a815b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a7fff640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a815b330 0x7fe9a815b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a7fff640 1 -- 172.21.15.172:0/1095461675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9a8161080 msgr2=0x7fe9a8144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a7fff640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9a8161080 0x7fe9a8144820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a7fff640 1 -- 172.21.15.172:0/1095461675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a8161940 con 0x7fe990004d40 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a77fe640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9a8161080 0x7fe9a8144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a7fff640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe990004d40 0x7fe9a81599d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe998011910 tx=0x7fe998011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a4ff9640 1 -- 172.21.15.172:0/1095461675 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99800faf0 con 0x7fe990004d40 2024-04-14T08:40:59.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a4ff9640 1 -- 172.21.15.172:0/1095461675 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99800fc90 con 0x7fe990004d40 2024-04-14T08:40:59.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/1095461675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a8161c20 con 0x7fe990004d40 2024-04-14T08:40:59.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9a4ff9640 1 -- 172.21.15.172:0/1095461675 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998014440 con 0x7fe990004d40 2024-04-14T08:40:59.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.727+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/1095461675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a81621e0 con 0x7fe990004d40 2024-04-14T08:40:59.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.728+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/1095461675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a8068520 con 0x7fe990004d40 2024-04-14T08:40:59.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.729+0000 7fe9a4ff9640 1 -- 172.21.15.172:0/1095461675 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe998012040 con 0x7fe990004d40 2024-04-14T08:40:59.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.729+0000 7fe9a4ff9640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe99406c8f0 0x7fe99406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:40:59.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.729+0000 7fe9a4ff9640 1 -- 172.21.15.172:0/1095461675 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5507+0+0 (secure 0 0 0) 0x7fe99808c160 con 0x7fe990004d40 2024-04-14T08:40:59.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.729+0000 7fe9a77fe640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe99406c8f0 0x7fe99406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:40:59.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.730+0000 7fe9a77fe640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe99406c8f0 0x7fe99406edb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80701c0 tx=0x7fe99c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:40:59.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.734+0000 7fe9a4ff9640 1 -- 172.21.15.172:0/1095461675 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe998019030 con 0x7fe990004d40 2024-04-14T08:40:59.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.945+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/1095461675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9a80b1500 con 0x7fe990004d40 2024-04-14T08:40:59.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.946+0000 7fe9a4ff9640 1 -- 172.21.15.172:0/1095461675 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+3881 (secure 0 0 0) 0x7fe99805a5b0 con 0x7fe990004d40 2024-04-14T08:40:59.947 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:40:59.947 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":69,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6985,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1408657889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1408657889},{"type":"v1","addr":"172.21.15.177:6834","nonce":1408657889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2877654265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":2877654265},{"type":"v1","addr":"172.21.15.177:6835","nonce":2877654265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/2582715324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2582715324},{"type":"v1","addr":"172.21.15.172:6836","nonce":2582715324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":8304,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2640008890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2640008890},{"type":"v1","addr":"172.21.15.172:6837","nonce":2640008890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-04-14T08:40:59.947 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 69 2024-04-14T08:40:59.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.948+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/1095461675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe99406c8f0 msgr2=0x7fe99406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.948+0000 7fe9ae3cf640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe99406c8f0 0x7fe99406edb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80701c0 tx=0x7fe99c008040 comp rx=0 tx=0).stop 2024-04-14T08:40:59.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.948+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/1095461675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe990004d40 msgr2=0x7fe9a81599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:40:59.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.948+0000 7fe9ae3cf640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe990004d40 0x7fe9a81599d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe998011910 tx=0x7fe998011dd0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.948+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/1095461675 shutdown_connections 2024-04-14T08:40:59.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.948+0000 7fe9ae3cf640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe99406c8f0 0x7fe99406edb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.948+0000 7fe9ae3cf640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9a815b330 0x7fe9a815b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.948+0000 7fe9ae3cf640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9a8161080 0x7fe9a8144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.948+0000 7fe9ae3cf640 1 --2- 172.21.15.172:0/1095461675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe990004d40 0x7fe9a81599d0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:40:59.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.948+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/1095461675 >> 172.21.15.172:0/1095461675 conn(0x7fe9a80d0f20 msgr2=0x7fe9a8153d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:40:59.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.948+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/1095461675 shutdown_connections 2024-04-14T08:40:59.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:40:59.948+0000 7fe9ae3cf640 1 -- 172.21.15.172:0/1095461675 wait complete. 2024-04-14T08:40:59.960 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-14T08:40:59.960 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-14T08:40:59.961 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-14T08:40:59.961 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:40:59.961 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-14T08:40:59.961 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-14T08:40:59.961 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-14T08:40:59.961 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:40:59.961 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-14T08:40:59.961 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-14T08:40:59.962 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-14T08:40:59.962 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:40:59.962 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-14T08:40:59.962 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-14T08:40:59.962 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-14T08:40:59.962 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:40:59.970 INFO:tasks.ceph.mds.a:Stopped 2024-04-14T08:40:59.970 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-14T08:40:59.971 INFO:tasks.ceph.mds.c:Stopped 2024-04-14T08:40:59.971 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-14T08:40:59.974 INFO:tasks.ceph.mds.b:Stopped 2024-04-14T08:40:59.974 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-14T08:40:59.978 INFO:tasks.ceph.mds.d:Stopped 2024-04-14T08:40:59.979 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-14T08:41:00.013 INFO:tasks.ceph.mds.a:Started 2024-04-14T08:41:00.013 DEBUG:teuthology.parallel:result is None 2024-04-14T08:41:00.014 INFO:tasks.ceph.mds.c:Started 2024-04-14T08:41:00.015 DEBUG:teuthology.parallel:result is None 2024-04-14T08:41:00.017 INFO:tasks.ceph.mds.b:Started 2024-04-14T08:41:00.017 DEBUG:teuthology.parallel:result is None 2024-04-14T08:41:00.018 INFO:tasks.ceph.mds.d:Started 2024-04-14T08:41:00.018 DEBUG:teuthology.parallel:result is None 2024-04-14T08:41:00.018 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-14T08:41:00.092 INFO:tasks.ceph.mds.a.smithi172.stdout:starting mds.a at 2024-04-14T08:41:00.093 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:41:00.093 INFO:tasks.ceph.mds.a.smithi172.stderr:2024-04-14T08:41:00.093+0000 7fa0713fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-14T08:41:00.105 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-04-14T08:41:00.108 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-04-14T08:41:00.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.125+0000 7f96a4ade640 1 Processor -- start 2024-04-14T08:41:00.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.125+0000 7f96a4ade640 1 -- start start 2024-04-14T08:41:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.126+0000 7f96a4ade640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f96a00d49c0 0x7f96a00d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.126+0000 7f96a4ade640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96a00cf0f0 con 0x7f96a00d0cc0 2024-04-14T08:41:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.126+0000 7f96a4ade640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96a00cf3d0 con 0x7f96a0160f30 2024-04-14T08:41:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.126+0000 7f96a4ade640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f96a00ceec0 con 0x7f96a00d49c0 2024-04-14T08:41:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.126+0000 7f969e59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f96a00d0cc0 0x7f96a00d10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58928/0 (socket says 172.21.15.172:58928) 2024-04-14T08:41:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.126+0000 7f969e59c640 1 -- 172.21.15.172:0/1081986865 learned_addr learned my addr 172.21.15.172:0/1081986865 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.126+0000 7f9695d9b640 1 --2- 172.21.15.172:0/1081986865 >> v2:172.21.15.172:3300/0 conn(0x7f96a00d49c0 0x7f96a00d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.126+0000 7f9695d9b640 1 -- 172.21.15.172:0/1081986865 >> v1:172.21.15.177:6789/0 conn(0x7f96a0160f30 legacy=0x7f96a01613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:00.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.126+0000 7f9695d9b640 1 -- 172.21.15.172:0/1081986865 >> v1:172.21.15.172:6789/0 conn(0x7f96a00d0cc0 legacy=0x7f96a00d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:00.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.126+0000 7f969dd9b640 1 -- 172.21.15.172:0/1081986865 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 405753790 0 0) 0x7f96a00cf0f0 con 0x7f96a00d0cc0 2024-04-14T08:41:00.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.126+0000 7f9695d9b640 1 -- 172.21.15.172:0/1081986865 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a00cf6b0 con 0x7f96a00d49c0 2024-04-14T08:41:00.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.127+0000 7f9695d9b640 1 --2- 172.21.15.172:0/1081986865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96a00d49c0 0x7f96a00d4e50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f96880097a0 tx=0x7f968802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f41a91f028c7a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:00.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.127+0000 7f969dd9b640 1 -- 172.21.15.172:0/1081986865 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688035020 con 0x7f96a00d49c0 2024-04-14T08:41:00.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.127+0000 7f969dd9b640 1 -- 172.21.15.172:0/1081986865 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9688031830 con 0x7f96a00d49c0 2024-04-14T08:41:00.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.127+0000 7f969dd9b640 1 -- 172.21.15.172:0/1081986865 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688031c00 con 0x7f96a00d49c0 2024-04-14T08:41:00.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.128+0000 7f96a4ade640 1 -- 172.21.15.172:0/1081986865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96a00d49c0 msgr2=0x7f96a00d4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:00.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.128+0000 7f96a4ade640 1 --2- 172.21.15.172:0/1081986865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96a00d49c0 0x7f96a00d4e50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f96880097a0 tx=0x7f968802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:41:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.128+0000 7f96a4ade640 1 -- 172.21.15.172:0/1081986865 shutdown_connections 2024-04-14T08:41:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.128+0000 7f96a4ade640 1 --2- 172.21.15.172:0/1081986865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96a00d49c0 0x7f96a00d4e50 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.128+0000 7f96a4ade640 1 -- 172.21.15.172:0/1081986865 >> 172.21.15.172:0/1081986865 conn(0x7f96a009d580 msgr2=0x7f96a00d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.128+0000 7f96a4ade640 1 -- 172.21.15.172:0/1081986865 shutdown_connections 2024-04-14T08:41:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.128+0000 7f96a4ade640 1 -- 172.21.15.172:0/1081986865 wait complete. 2024-04-14T08:41:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.130+0000 7f96a4ade640 1 Processor -- start 2024-04-14T08:41:00.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.130+0000 7f96a4ade640 1 -- start start 2024-04-14T08:41:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.130+0000 7f96a4ade640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96a00d0cc0 0x7f96a009ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f96a4ade640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96a00d49c0 0x7f96a01f1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f96a4ade640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96a0160f30 0x7f96a01f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f96a4ade640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96a00d32b0 con 0x7f96a0160f30 2024-04-14T08:41:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f96a4ade640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f96a00cfd00 con 0x7f96a00d0cc0 2024-04-14T08:41:00.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f96a4ade640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f96a00d0190 con 0x7f96a00d49c0 2024-04-14T08:41:00.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f969ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96a0160f30 0x7f96a01f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:00.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f969ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96a0160f30 0x7f96a01f7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36374/0 (socket says 172.21.15.172:36374) 2024-04-14T08:41:00.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f969ed9d640 1 -- 172.21.15.172:0/4115114978 learned_addr learned my addr 172.21.15.172:0/4115114978 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:00.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f9695d9b640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96a00d49c0 0x7f96a01f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:00.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f969ed9d640 1 -- 172.21.15.172:0/4115114978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96a00d49c0 msgr2=0x7f96a01f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:00.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f969ed9d640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96a00d49c0 0x7f96a01f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:00.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f969ed9d640 1 -- 172.21.15.172:0/4115114978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96a00d0cc0 msgr2=0x7f96a009ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:00.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f969ed9d640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96a00d0cc0 0x7f96a009ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:00.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f969ed9d640 1 -- 172.21.15.172:0/4115114978 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a009f270 con 0x7f96a0160f30 2024-04-14T08:41:00.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f969ed9d640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96a0160f30 0x7f96a01f7670 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f969800b970 tx=0x7f969800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:00.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f96977fe640 1 -- 172.21.15.172:0/4115114978 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698019070 con 0x7f96a0160f30 2024-04-14T08:41:00.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f96977fe640 1 -- 172.21.15.172:0/4115114978 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f969800cb40 con 0x7f96a0160f30 2024-04-14T08:41:00.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f96977fe640 1 -- 172.21.15.172:0/4115114978 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698014440 con 0x7f96a0160f30 2024-04-14T08:41:00.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f96a4ade640 1 -- 172.21.15.172:0/4115114978 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a009f550 con 0x7f96a0160f30 2024-04-14T08:41:00.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.131+0000 7f96a4ade640 1 -- 172.21.15.172:0/4115114978 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a009fb10 con 0x7f96a0160f30 2024-04-14T08:41:00.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.132+0000 7f96a4ade640 1 -- 172.21.15.172:0/4115114978 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a009f000 con 0x7f96a0160f30 2024-04-14T08:41:00.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.133+0000 7f96977fe640 1 -- 172.21.15.172:0/4115114978 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f969801c070 con 0x7f96a0160f30 2024-04-14T08:41:00.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.133+0000 7f96977fe640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f968006c8f0 0x7f968006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:00.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.133+0000 7f96977fe640 1 -- 172.21.15.172:0/4115114978 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5507+0+0 (secure 0 0 0) 0x7f969808b7d0 con 0x7f96a0160f30 2024-04-14T08:41:00.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.133+0000 7f969e59c640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f968006c8f0 0x7f968006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:00.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.134+0000 7f969e59c640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f968006c8f0 0x7f968006edb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9690007fd0 tx=0x7f9690007900 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:00.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.138+0000 7f96977fe640 1 -- 172.21.15.172:0/4115114978 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9698059ca0 con 0x7f96a0160f30 2024-04-14T08:41:00.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:00.325+0000 7f96a4ade640 1 -- 172.21.15.172:0/4115114978 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f96a0075490 con 0x7f96a0160f30 2024-04-14T08:41:01.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.115+0000 7f96977fe640 1 -- 172.21.15.172:0/4115114978 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) ==== 97+0+0 (secure 0 0 0) 0x7f96980595a0 con 0x7f96a0160f30 2024-04-14T08:41:01.115 INFO:teuthology.orchestra.run.smithi172.stderr: removed all blocklist entries 2024-04-14T08:41:01.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.118+0000 7f96a4ade640 1 -- 172.21.15.172:0/4115114978 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f968006c8f0 msgr2=0x7f968006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.118+0000 7f96a4ade640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f968006c8f0 0x7f968006edb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9690007fd0 tx=0x7f9690007900 comp rx=0 tx=0).stop 2024-04-14T08:41:01.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.118+0000 7f96a4ade640 1 -- 172.21.15.172:0/4115114978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96a0160f30 msgr2=0x7f96a01f7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.118+0000 7f96a4ade640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96a0160f30 0x7f96a01f7670 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f969800b970 tx=0x7f969800be30 comp rx=0 tx=0).stop 2024-04-14T08:41:01.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.118+0000 7f96a4ade640 1 -- 172.21.15.172:0/4115114978 shutdown_connections 2024-04-14T08:41:01.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.118+0000 7f96a4ade640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f968006c8f0 0x7f968006edb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.118+0000 7f96a4ade640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96a0160f30 0x7f96a01f7670 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.118+0000 7f96a4ade640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96a00d49c0 0x7f96a01f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.118+0000 7f96a4ade640 1 --2- 172.21.15.172:0/4115114978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96a00d0cc0 0x7f96a009ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.118+0000 7f96a4ade640 1 -- 172.21.15.172:0/4115114978 >> 172.21.15.172:0/4115114978 conn(0x7f96a009d580 msgr2=0x7f96a0074a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:01.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.118+0000 7f96a4ade640 1 -- 172.21.15.172:0/4115114978 shutdown_connections 2024-04-14T08:41:01.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.118+0000 7f96a4ade640 1 -- 172.21.15.172:0/4115114978 wait complete. 2024-04-14T08:41:01.132 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-14T08:41:01.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.238+0000 7fbdf7c98640 1 Processor -- start 2024-04-14T08:41:01.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.238+0000 7fbdf7c98640 1 -- start start 2024-04-14T08:41:01.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.239+0000 7fbdf7c98640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf00b21d0 0x7fbdf015c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.239+0000 7fbdf7c98640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdf00b3700 con 0x7fbdf00b1df0 2024-04-14T08:41:01.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.239+0000 7fbdf7c98640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdf00b39e0 con 0x7fbdf015c5a0 2024-04-14T08:41:01.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.239+0000 7fbdf7c98640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf00b3cc0 con 0x7fbdf00b21d0 2024-04-14T08:41:01.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.239+0000 7fbdf5a34640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fbdf00b1df0 0x7fbdf00b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58942/0 (socket says 172.21.15.172:58942) 2024-04-14T08:41:01.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.239+0000 7fbdf5a34640 1 -- 172.21.15.172:0/3278150576 learned_addr learned my addr 172.21.15.172:0/3278150576 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:01.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.239+0000 7fbdf5233640 1 --2- 172.21.15.172:0/3278150576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf00b21d0 0x7fbdf015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.239+0000 7fbdf4a32640 1 -- 172.21.15.172:0/3278150576 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1670203953 0 0) 0x7fbdf00b3700 con 0x7fbdf00b1df0 2024-04-14T08:41:01.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.239+0000 7fbdf4a32640 1 -- 172.21.15.172:0/3278150576 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbddc003520 con 0x7fbdf00b1df0 2024-04-14T08:41:01.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.239+0000 7fbdf4a32640 1 -- 172.21.15.172:0/3278150576 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2779457569 0 0) 0x7fbdf00b39e0 con 0x7fbdf015c5a0 2024-04-14T08:41:01.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.239+0000 7fbdf4a32640 1 -- 172.21.15.172:0/3278150576 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdf00b3700 con 0x7fbdf015c5a0 2024-04-14T08:41:01.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.240+0000 7fbdf4a32640 1 -- 172.21.15.172:0/3278150576 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3370066604 0 0) 0x7fbddc003520 con 0x7fbdf00b1df0 2024-04-14T08:41:01.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.240+0000 7fbdf5233640 1 -- 172.21.15.172:0/3278150576 >> v1:172.21.15.177:6789/0 conn(0x7fbdf015c5a0 legacy=0x7fbdf00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.240+0000 7fbdf5233640 1 -- 172.21.15.172:0/3278150576 >> v1:172.21.15.172:6789/0 conn(0x7fbdf00b1df0 legacy=0x7fbdf00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.240+0000 7fbdf5233640 1 -- 172.21.15.172:0/3278150576 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf015bb40 con 0x7fbdf00b21d0 2024-04-14T08:41:01.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.240+0000 7fbdf5233640 1 --2- 172.21.15.172:0/3278150576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf00b21d0 0x7fbdf015c060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbde00097d0 tx=0x7fbde002b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9fabbdf30a0cb21 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:01.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.241+0000 7fbdf4a32640 1 -- 172.21.15.172:0/3278150576 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0037070 con 0x7fbdf00b21d0 2024-04-14T08:41:01.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.241+0000 7fbdf4a32640 1 -- 172.21.15.172:0/3278150576 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde0030510 con 0x7fbdf00b21d0 2024-04-14T08:41:01.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.241+0000 7fbdf4a32640 1 -- 172.21.15.172:0/3278150576 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde80019f0 con 0x7fbdf00b21d0 2024-04-14T08:41:01.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.241+0000 7fbdf7c98640 1 -- 172.21.15.172:0/3278150576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf00b21d0 msgr2=0x7fbdf015c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.241+0000 7fbdf7c98640 1 --2- 172.21.15.172:0/3278150576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf00b21d0 0x7fbdf015c060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbde00097d0 tx=0x7fbde002b800 comp rx=0 tx=0).stop 2024-04-14T08:41:01.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.241+0000 7fbdf7c98640 1 -- 172.21.15.172:0/3278150576 shutdown_connections 2024-04-14T08:41:01.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.241+0000 7fbdf7c98640 1 --2- 172.21.15.172:0/3278150576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf00b21d0 0x7fbdf015c060 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.241+0000 7fbdf7c98640 1 -- 172.21.15.172:0/3278150576 >> 172.21.15.172:0/3278150576 conn(0x7fbdf006ec30 msgr2=0x7fbdf006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:01.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.241+0000 7fbdf7c98640 1 -- 172.21.15.172:0/3278150576 shutdown_connections 2024-04-14T08:41:01.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.241+0000 7fbdf7c98640 1 -- 172.21.15.172:0/3278150576 wait complete. 2024-04-14T08:41:01.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.242+0000 7fbdf7c98640 1 Processor -- start 2024-04-14T08:41:01.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.242+0000 7fbdf7c98640 1 -- start start 2024-04-14T08:41:01.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf7c98640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf00b1df0 0x7fbdf0144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf7c98640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf00b21d0 0x7fbdf00a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf7c98640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf00a9a10 0x7fbdf00ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf7c98640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf00b48d0 con 0x7fbdf00b1df0 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf7c98640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf00b4750 con 0x7fbdf00b21d0 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf7c98640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf01447a0 con 0x7fbdf00a9a10 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf5a34640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf00b1df0 0x7fbdf0144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf5a34640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf00b1df0 0x7fbdf0144260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36390/0 (socket says 172.21.15.172:36390) 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf5a34640 1 -- 172.21.15.172:0/1866075050 learned_addr learned my addr 172.21.15.172:0/1866075050 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf5233640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf00b21d0 0x7fbdf00a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf6235640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf00a9a10 0x7fbdf00ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf5a34640 1 -- 172.21.15.172:0/1866075050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf00a9a10 msgr2=0x7fbdf00ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf5a34640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf00a9a10 0x7fbdf00ae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf5a34640 1 -- 172.21.15.172:0/1866075050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf00b21d0 msgr2=0x7fbdf00a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf5a34640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf00b21d0 0x7fbdf00a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.243+0000 7fbdf5a34640 1 -- 172.21.15.172:0/1866075050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf00a9fd0 con 0x7fbdf00b1df0 2024-04-14T08:41:01.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.244+0000 7fbdf5a34640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf00b1df0 0x7fbdf0144260 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fbde8002000 tx=0x7fbde800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:01.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.244+0000 7fbde67fc640 1 -- 172.21.15.172:0/1866075050 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde8012400 con 0x7fbdf00b1df0 2024-04-14T08:41:01.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.244+0000 7fbde67fc640 1 -- 172.21.15.172:0/1866075050 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde80125a0 con 0x7fbdf00b1df0 2024-04-14T08:41:01.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.244+0000 7fbdf7c98640 1 -- 172.21.15.172:0/1866075050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf00aa2b0 con 0x7fbdf00b1df0 2024-04-14T08:41:01.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.244+0000 7fbde67fc640 1 -- 172.21.15.172:0/1866075050 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde8012990 con 0x7fbdf00b1df0 2024-04-14T08:41:01.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.244+0000 7fbdf7c98640 1 -- 172.21.15.172:0/1866075050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf00aa870 con 0x7fbdf00b1df0 2024-04-14T08:41:01.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.245+0000 7fbdf7c98640 1 -- 172.21.15.172:0/1866075050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf0067dd0 con 0x7fbdf00b1df0 2024-04-14T08:41:01.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.245+0000 7fbde67fc640 1 -- 172.21.15.172:0/1866075050 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbde8012b30 con 0x7fbdf00b1df0 2024-04-14T08:41:01.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.246+0000 7fbde67fc640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbdd406c8f0 0x7fbdd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.246+0000 7fbde67fc640 1 -- 172.21.15.172:0/1866075050 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5249+0+0 (secure 0 0 0) 0x7fbde808c260 con 0x7fbdf00b1df0 2024-04-14T08:41:01.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.246+0000 7fbdf5233640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbdd406c8f0 0x7fbdd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.246+0000 7fbdf5233640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbdd406c8f0 0x7fbdd406edb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbde0034000 tx=0x7fbde0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:01.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.250+0000 7fbde67fc640 1 -- 172.21.15.172:0/1866075050 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbdf0067dd0 con 0x7fbdf00b1df0 2024-04-14T08:41:01.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.497+0000 7fbdf7c98640 1 -- 172.21.15.172:0/1866075050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fbdf0068520 con 0x7fbdf00b1df0 2024-04-14T08:41:01.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.497+0000 7fbde67fc640 1 -- 172.21.15.172:0/1866075050 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fbde80182b0 con 0x7fbdf00b1df0 2024-04-14T08:41:01.498 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:01.498 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:41:01.498 INFO:teuthology.orchestra.run.smithi172.stdout: "auth_dump": [ 2024-04-14T08:41:01.498 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.498 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.a", 2024-04-14T08:41:01.498 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmnNCRDRAAz7aU3JD7iiaTQH0cNotfaw==", 2024-04-14T08:41:01.499 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.499 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:41:01.499 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:41:01.499 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:41:01.499 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.499 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.499 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.499 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.499 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.b", 2024-04-14T08:41:01.499 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtm1on4GRAAms1he+O4jWu01z0ZLLgL0A==", 2024-04-14T08:41:01.499 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.500 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:41:01.500 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:41:01.500 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:41:01.500 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.500 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.500 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.500 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.500 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.c", 2024-04-14T08:41:01.500 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmb7z0FBAAGVu3+TWshDGvd7/fLbgJ5A==", 2024-04-14T08:41:01.500 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.500 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:41:01.500 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:41:01.501 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:41:01.501 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.501 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.501 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.501 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.501 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.d", 2024-04-14T08:41:01.501 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmEEx0IRAAXiFiOK1RE/0bfyu7KI/3ng==", 2024-04-14T08:41:01.501 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.501 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:41:01.501 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.0", 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB/lBtmnsWuCRAAqcuQhsrzpGVfrlgwTnFsOw==", 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.502 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.1", 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCClBtme9lPBxAAFB1E3g8qwZxPCdHiFogp5Q==", 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.2", 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCFlBtm966YBRAAWXZGq/0TuUMZTsyeR4xWww==", 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:41:01.503 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.3", 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCIlBtmHpktBBAAUbddNHESwyNvvp8UE/SaCw==", 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.504 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.4", 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCOlBtm+//7ChAA7rApbKll2vBGcDlW8q+kyg==", 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.5", 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCRlBtmVU4YChAAysdUosQ+TnHtjnRl1FOISA==", 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:41:01.505 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.6", 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCUlBtmnE7+BxAAqBn8YMwecA2NcZz63eBm7Q==", 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.506 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.7", 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCXlBtmN8DhBRAA6CHQ943t6qJCL77jcbOv4w==", 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.0", 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA==", 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.507 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow r", 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.1", 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmp4nkLhAAhYKm8VcykVbPSohvK11I9w==", 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:41:01.508 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.2", 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmJoV9KxAAp6zlRX+gvhASQBHjpF9ewQ==", 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.509 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.3", 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmeC+pNBAASi7W4jlo6imMMzNEwLrPDA==", 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.admin", 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB6lBtmgnWqLBAAQ0fgarM8au3zFuMNnQIvVw==", 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.510 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.x", 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmm5IbChAAeVcVs3Zt6+lU0wghjCcwnQ==", 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.511 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:41:01.512 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:01.512 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.y", 2024-04-14T08:41:01.512 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmfddYBhAAh8zgf4W3WWmNYWNAA7WuRw==", 2024-04-14T08:41:01.512 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:41:01.512 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:41:01.512 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:41:01.512 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:41:01.512 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.512 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:01.512 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-04-14T08:41:01.512 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:41:01.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.499+0000 7fbdf7c98640 1 -- 172.21.15.172:0/1866075050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbdd406c8f0 msgr2=0x7fbdd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.499+0000 7fbdf7c98640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbdd406c8f0 0x7fbdd406edb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbde0034000 tx=0x7fbde0008030 comp rx=0 tx=0).stop 2024-04-14T08:41:01.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.499+0000 7fbdf7c98640 1 -- 172.21.15.172:0/1866075050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf00b1df0 msgr2=0x7fbdf0144260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.499+0000 7fbdf7c98640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf00b1df0 0x7fbdf0144260 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fbde8002000 tx=0x7fbde800a430 comp rx=0 tx=0).stop 2024-04-14T08:41:01.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.499+0000 7fbdf7c98640 1 -- 172.21.15.172:0/1866075050 shutdown_connections 2024-04-14T08:41:01.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.499+0000 7fbdf7c98640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbdd406c8f0 0x7fbdd406edb0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.499+0000 7fbdf7c98640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbdf00a9a10 0x7fbdf00ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.499+0000 7fbdf7c98640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdf00b21d0 0x7fbdf00a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.499+0000 7fbdf7c98640 1 --2- 172.21.15.172:0/1866075050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdf00b1df0 0x7fbdf0144260 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.499+0000 7fbdf7c98640 1 -- 172.21.15.172:0/1866075050 >> 172.21.15.172:0/1866075050 conn(0x7fbdf006ec30 msgr2=0x7fbdf0074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:01.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.499+0000 7fbdf7c98640 1 -- 172.21.15.172:0/1866075050 shutdown_connections 2024-04-14T08:41:01.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.499+0000 7fbdf7c98640 1 -- 172.21.15.172:0/1866075050 wait complete. 2024-04-14T08:41:01.515 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:41:01.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.616+0000 7f76b0dc8640 1 Processor -- start 2024-04-14T08:41:01.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.616+0000 7f76b0dc8640 1 -- start start 2024-04-14T08:41:01.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76b0dc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac0b2290 0x7f76ac163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76b0dc8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ac167390 con 0x7f76ac163670 2024-04-14T08:41:01.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76b0dc8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ac1665d0 con 0x7f76ac071620 2024-04-14T08:41:01.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76b0dc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac1668b0 con 0x7f76ac0b2290 2024-04-14T08:41:01.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76aa59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f76ac163670 0x7f76ac0b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58954/0 (socket says 172.21.15.172:58954) 2024-04-14T08:41:01.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76aa59c640 1 -- 172.21.15.172:0/3637564215 learned_addr learned my addr 172.21.15.172:0/3637564215 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:01.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2485905795 0 0) 0x7f76ac167390 con 0x7f76ac163670 2024-04-14T08:41:01.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76a3fff640 1 --2- 172.21.15.172:0/3637564215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac0b2290 0x7f76ac163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7698003520 con 0x7f76ac163670 2024-04-14T08:41:01.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1081773481 0 0) 0x7f7698003520 con 0x7f76ac163670 2024-04-14T08:41:01.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac0b2290 msgr2=0x7f76ac163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:01.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76a9d9b640 1 --2- 172.21.15.172:0/3637564215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac0b2290 0x7f76ac163a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 >> v1:172.21.15.177:6789/0 conn(0x7f76ac071620 legacy=0x7f76ac071ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac166b90 con 0x7f76ac163670 2024-04-14T08:41:01.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f769c001670 con 0x7f76ac163670 2024-04-14T08:41:01.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.617+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 >> v1:172.21.15.172:6789/0 conn(0x7f76ac163670 legacy=0x7f76ac0b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76a9d9b640 1 --2- 172.21.15.172:0/3637564215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac0b2290 0x7f76980039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76a9d9b640 1 --2- 172.21.15.172:0/3637564215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7698003f60 0x7f7698004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76a9d9b640 1 --2- 172.21.15.172:0/3637564215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7698004d40 0x7f76980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac1668b0 con 0x7f7698004d40 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f769800baa0 con 0x7f7698003f60 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f769800bca0 con 0x7f76ac0b2290 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76aad9d640 1 --2- 172.21.15.172:0/3637564215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7698004d40 0x7f76980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76a3fff640 1 --2- 172.21.15.172:0/3637564215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7698003f60 0x7f7698004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76aa59c640 1 --2- 172.21.15.172:0/3637564215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac0b2290 0x7f76980039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76aad9d640 1 -- 172.21.15.172:0/3637564215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac0b2290 msgr2=0x7f76980039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76aad9d640 1 --2- 172.21.15.172:0/3637564215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac0b2290 0x7f76980039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76aad9d640 1 -- 172.21.15.172:0/3637564215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7698003f60 msgr2=0x7f7698004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76aad9d640 1 --2- 172.21.15.172:0/3637564215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7698003f60 0x7f7698004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.618+0000 7f76aad9d640 1 -- 172.21.15.172:0/3637564215 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7698003520 con 0x7f7698004d40 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.619+0000 7f76aad9d640 1 --2- 172.21.15.172:0/3637564215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7698004d40 0x7f76980051c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f76a4001e50 tx=0x7f7698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23f37123f77db69 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.619+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4001a70 con 0x7f7698004d40 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.619+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a4003b70 con 0x7f7698004d40 2024-04-14T08:41:01.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.619+0000 7f76a9d9b640 1 -- 172.21.15.172:0/3637564215 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a40050d0 con 0x7f7698004d40 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.619+0000 7f76b0dc8640 1 -- 172.21.15.172:0/3637564215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7698004d40 msgr2=0x7f76980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.619+0000 7f76b0dc8640 1 --2- 172.21.15.172:0/3637564215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7698004d40 0x7f76980051c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f76a4001e50 tx=0x7f7698005830 comp rx=0 tx=0).stop 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.619+0000 7f76aad9d640 1 -- 172.21.15.172:0/3637564215 reap_dead start 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.619+0000 7f76b0dc8640 1 -- 172.21.15.172:0/3637564215 shutdown_connections 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.619+0000 7f76b0dc8640 1 -- 172.21.15.172:0/3637564215 >> 172.21.15.172:0/3637564215 conn(0x7f76ac06ec30 msgr2=0x7f76ac06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.619+0000 7f76b0dc8640 1 -- 172.21.15.172:0/3637564215 shutdown_connections 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.619+0000 7f76b0dc8640 1 -- 172.21.15.172:0/3637564215 wait complete. 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.621+0000 7f76b0dc8640 1 Processor -- start 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.621+0000 7f76b0dc8640 1 -- start start 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.621+0000 7f76b0dc8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76ac14b070 0x7f76ac1edec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.621+0000 7f76b0dc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac1ec2c0 0x7f76ac1ee400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.621+0000 7f76b0dc8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76ac1ee940 0x7f76ac15a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.621+0000 7f76b0dc8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0b4210 con 0x7f76ac14b070 2024-04-14T08:41:01.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.621+0000 7f76b0dc8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0722c0 con 0x7f76ac1ee940 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.621+0000 7f76b0dc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac163a50 con 0x7f76ac1ec2c0 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.621+0000 7f76aa59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76ac14b070 0x7f76ac1edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76aa59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76ac14b070 0x7f76ac1edec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36406/0 (socket says 172.21.15.172:36406) 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76aa59c640 1 -- 172.21.15.172:0/2890767014 learned_addr learned my addr 172.21.15.172:0/2890767014 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76a3fff640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac1ec2c0 0x7f76ac1ee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76aad9d640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76ac1ee940 0x7f76ac15a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76aa59c640 1 -- 172.21.15.172:0/2890767014 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac1ec2c0 msgr2=0x7f76ac1ee400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76aa59c640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac1ec2c0 0x7f76ac1ee400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76aa59c640 1 -- 172.21.15.172:0/2890767014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76ac1ee940 msgr2=0x7f76ac15a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76aa59c640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76ac1ee940 0x7f76ac15a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76aa59c640 1 -- 172.21.15.172:0/2890767014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac15cca0 con 0x7f76ac14b070 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76aad9d640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76ac1ee940 0x7f76ac15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:41:01.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76aa59c640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76ac14b070 0x7f76ac1edec0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f769c0072f0 tx=0x7f769c0076f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:01.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76a2ffd640 1 -- 172.21.15.172:0/2890767014 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c01a070 con 0x7f76ac14b070 2024-04-14T08:41:01.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76a2ffd640 1 -- 172.21.15.172:0/2890767014 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f769c00f9c0 con 0x7f76ac14b070 2024-04-14T08:41:01.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76b0dc8640 1 -- 172.21.15.172:0/2890767014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac1ef180 con 0x7f76ac14b070 2024-04-14T08:41:01.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76a2ffd640 1 -- 172.21.15.172:0/2890767014 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c00fd70 con 0x7f76ac14b070 2024-04-14T08:41:01.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.622+0000 7f76b0dc8640 1 -- 172.21.15.172:0/2890767014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac1ef740 con 0x7f76ac14b070 2024-04-14T08:41:01.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.623+0000 7f76b0dc8640 1 -- 172.21.15.172:0/2890767014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ac068520 con 0x7f76ac14b070 2024-04-14T08:41:01.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.624+0000 7f76a2ffd640 1 -- 172.21.15.172:0/2890767014 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f769c01d070 con 0x7f76ac14b070 2024-04-14T08:41:01.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.624+0000 7f76a2ffd640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f768c06c8f0 0x7f768c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.624+0000 7f76a2ffd640 1 -- 172.21.15.172:0/2890767014 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5249+0+0 (secure 0 0 0) 0x7f769c08b580 con 0x7f76ac14b070 2024-04-14T08:41:01.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.624+0000 7f76a3fff640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f768c06c8f0 0x7f768c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.624+0000 7f76a3fff640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f768c06c8f0 0x7f768c06edb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f76ac1fc0a0 tx=0x7f7694009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:01.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.628+0000 7f76a2ffd640 1 -- 172.21.15.172:0/2890767014 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f769c059ae0 con 0x7f76ac14b070 2024-04-14T08:41:01.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.805+0000 7f76b0dc8640 1 -- 172.21.15.172:0/2890767014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76ac06c2c0 con 0x7f76ac14b070 2024-04-14T08:41:01.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.806+0000 7f76a2ffd640 1 -- 172.21.15.172:0/2890767014 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+13225 (secure 0 0 0) 0x7f769c0593e0 con 0x7f76ac14b070 2024-04-14T08:41:01.807 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:01.807 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":97,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:41:01.112133+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:41:01.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.808+0000 7f76b0dc8640 1 -- 172.21.15.172:0/2890767014 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f768c06c8f0 msgr2=0x7f768c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.808+0000 7f76b0dc8640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f768c06c8f0 0x7f768c06edb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f76ac1fc0a0 tx=0x7f7694009190 comp rx=0 tx=0).stop 2024-04-14T08:41:01.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.808+0000 7f76b0dc8640 1 -- 172.21.15.172:0/2890767014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76ac14b070 msgr2=0x7f76ac1edec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.808+0000 7f76b0dc8640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76ac14b070 0x7f76ac1edec0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f769c0072f0 tx=0x7f769c0076f0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.808+0000 7f76b0dc8640 1 -- 172.21.15.172:0/2890767014 shutdown_connections 2024-04-14T08:41:01.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.808+0000 7f76b0dc8640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f768c06c8f0 0x7f768c06edb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.808+0000 7f76b0dc8640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76ac1ee940 0x7f76ac15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.808+0000 7f76b0dc8640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76ac1ec2c0 0x7f76ac1ee400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.808+0000 7f76b0dc8640 1 --2- 172.21.15.172:0/2890767014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76ac14b070 0x7f76ac1edec0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.808+0000 7f76b0dc8640 1 -- 172.21.15.172:0/2890767014 >> 172.21.15.172:0/2890767014 conn(0x7f76ac06ec30 msgr2=0x7f76ac0b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:01.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.808+0000 7f76b0dc8640 1 -- 172.21.15.172:0/2890767014 shutdown_connections 2024-04-14T08:41:01.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.808+0000 7f76b0dc8640 1 -- 172.21.15.172:0/2890767014 wait complete. 2024-04-14T08:41:01.821 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:41:01.821 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:41:01.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.925+0000 7f307a62d640 1 Processor -- start 2024-04-14T08:41:01.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.925+0000 7f307a62d640 1 -- start start 2024-04-14T08:41:01.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.926+0000 7f307a62d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f307413df30 0x7f30740b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.926+0000 7f307a62d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f307415b620 con 0x7f307415c380 2024-04-14T08:41:01.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.926+0000 7f307a62d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30741414a0 con 0x7f3074140e80 2024-04-14T08:41:01.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.926+0000 7f307a62d640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f307415b8a0 con 0x7f307413df30 2024-04-14T08:41:01.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.926+0000 7f3073fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f307413df30 0x7f30740b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.926+0000 7f30737fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f307415c380 0x7f3074140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58956/0 (socket says 172.21.15.172:58956) 2024-04-14T08:41:01.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.926+0000 7f30737fe640 1 -- 172.21.15.172:0/877887538 learned_addr learned my addr 172.21.15.172:0/877887538 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:01.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.926+0000 7f3072ffd640 1 -- 172.21.15.172:0/877887538 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1431951184 0 0) 0x7f3064001110 con 0x7f307415c380 2024-04-14T08:41:01.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.926+0000 7f3072ffd640 1 -- 172.21.15.172:0/877887538 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f305c003520 con 0x7f307415c380 2024-04-14T08:41:01.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.926+0000 7f3073fff640 1 -- 172.21.15.172:0/877887538 >> v1:172.21.15.177:6789/0 conn(0x7f3074140e80 legacy=0x7f30740ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:01.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.926+0000 7f3073fff640 1 -- 172.21.15.172:0/877887538 >> v1:172.21.15.172:6789/0 conn(0x7f307415c380 legacy=0x7f3074140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.926+0000 7f3073fff640 1 -- 172.21.15.172:0/877887538 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307415bb80 con 0x7f307413df30 2024-04-14T08:41:01.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.927+0000 7f3073fff640 1 --2- 172.21.15.172:0/877887538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f307413df30 0x7f30740b1cd0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f30680097a0 tx=0x7f306802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593ce2b3f39e0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:01.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.927+0000 7f3072ffd640 1 -- 172.21.15.172:0/877887538 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068035020 con 0x7f307413df30 2024-04-14T08:41:01.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.927+0000 7f3072ffd640 1 -- 172.21.15.172:0/877887538 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3068031830 con 0x7f307413df30 2024-04-14T08:41:01.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.927+0000 7f3072ffd640 1 -- 172.21.15.172:0/877887538 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068031c00 con 0x7f307413df30 2024-04-14T08:41:01.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.928+0000 7f307a62d640 1 -- 172.21.15.172:0/877887538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f307413df30 msgr2=0x7f30740b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.928+0000 7f307a62d640 1 --2- 172.21.15.172:0/877887538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f307413df30 0x7f30740b1cd0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f30680097a0 tx=0x7f306802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.928+0000 7f307a62d640 1 -- 172.21.15.172:0/877887538 shutdown_connections 2024-04-14T08:41:01.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.928+0000 7f307a62d640 1 --2- 172.21.15.172:0/877887538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f307413df30 0x7f30740b1cd0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.928+0000 7f307a62d640 1 -- 172.21.15.172:0/877887538 >> 172.21.15.172:0/877887538 conn(0x7f307406b520 msgr2=0x7f307406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:01.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.928+0000 7f307a62d640 1 -- 172.21.15.172:0/877887538 shutdown_connections 2024-04-14T08:41:01.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.928+0000 7f307a62d640 1 -- 172.21.15.172:0/877887538 wait complete. 2024-04-14T08:41:01.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.929+0000 7f307a62d640 1 Processor -- start 2024-04-14T08:41:01.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.929+0000 7f307a62d640 1 -- start start 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f307a62d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f307413df30 0x7f3074157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f307a62d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3074140e80 0x7f30740b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f307a62d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f307415c380 0x7f30740b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f307a62d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30740b2210 con 0x7f3074140e80 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f307a62d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f307413e310 con 0x7f307413df30 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f307a62d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f30740b2390 con 0x7f307415c380 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f30737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3074140e80 0x7f30740b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f30737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3074140e80 0x7f30740b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36432/0 (socket says 172.21.15.172:36432) 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f30737fe640 1 -- 172.21.15.172:0/3200360164 learned_addr learned my addr 172.21.15.172:0/3200360164 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f3073fff640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f307413df30 0x7f3074157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f3078bca640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f307415c380 0x7f30740b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f30737fe640 1 -- 172.21.15.172:0/3200360164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f307415c380 msgr2=0x7f30740b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f30737fe640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f307415c380 0x7f30740b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f30737fe640 1 -- 172.21.15.172:0/3200360164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f307413df30 msgr2=0x7f3074157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:01.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f30737fe640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f307413df30 0x7f3074157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:01.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.930+0000 7f30737fe640 1 -- 172.21.15.172:0/3200360164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3074150f70 con 0x7f3074140e80 2024-04-14T08:41:01.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.931+0000 7f30737fe640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3074140e80 0x7f30740b0ee0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f306400c3d0 tx=0x7f306400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:01.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.931+0000 7f3070ff9640 1 -- 172.21.15.172:0/3200360164 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064017070 con 0x7f3074140e80 2024-04-14T08:41:01.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.931+0000 7f3070ff9640 1 -- 172.21.15.172:0/3200360164 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3064005ce0 con 0x7f3074140e80 2024-04-14T08:41:01.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.931+0000 7f307a62d640 1 -- 172.21.15.172:0/3200360164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3074157db0 con 0x7f3074140e80 2024-04-14T08:41:01.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.931+0000 7f3070ff9640 1 -- 172.21.15.172:0/3200360164 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064007310 con 0x7f3074140e80 2024-04-14T08:41:01.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.931+0000 7f307a62d640 1 -- 172.21.15.172:0/3200360164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3074158360 con 0x7f3074140e80 2024-04-14T08:41:01.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.932+0000 7f307a62d640 1 -- 172.21.15.172:0/3200360164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3074068590 con 0x7f3074140e80 2024-04-14T08:41:01.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.932+0000 7f3070ff9640 1 -- 172.21.15.172:0/3200360164 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f306401a070 con 0x7f3074140e80 2024-04-14T08:41:01.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.933+0000 7f3070ff9640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f306006c8f0 0x7f306006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:01.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.933+0000 7f3073fff640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f306006c8f0 0x7f306006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:01.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.933+0000 7f3070ff9640 1 -- 172.21.15.172:0/3200360164 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5249+0+0 (secure 0 0 0) 0x7f306408ace0 con 0x7f3074140e80 2024-04-14T08:41:01.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.933+0000 7f3073fff640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f306006c8f0 0x7f306006edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3068036000 tx=0x7f3068008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:01.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:01.937+0000 7f3070ff9640 1 -- 172.21.15.172:0/3200360164 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f307415b620 con 0x7f3074140e80 2024-04-14T08:41:02.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.111+0000 7f307a62d640 1 -- 172.21.15.172:0/3200360164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3074155570 con 0x7f3074140e80 2024-04-14T08:41:02.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.112+0000 7f3070ff9640 1 -- 172.21.15.172:0/3200360164 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f3064059360 con 0x7f3074140e80 2024-04-14T08:41:02.112 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:41:02.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.114+0000 7f307a62d640 1 -- 172.21.15.172:0/3200360164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f306006c8f0 msgr2=0x7f306006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:02.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.114+0000 7f307a62d640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f306006c8f0 0x7f306006edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3068036000 tx=0x7f3068008030 comp rx=0 tx=0).stop 2024-04-14T08:41:02.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.114+0000 7f307a62d640 1 -- 172.21.15.172:0/3200360164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3074140e80 msgr2=0x7f30740b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:02.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.114+0000 7f307a62d640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3074140e80 0x7f30740b0ee0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f306400c3d0 tx=0x7f306400c890 comp rx=0 tx=0).stop 2024-04-14T08:41:02.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.114+0000 7f307a62d640 1 -- 172.21.15.172:0/3200360164 shutdown_connections 2024-04-14T08:41:02.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.114+0000 7f307a62d640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f306006c8f0 0x7f306006edb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.114+0000 7f307a62d640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f307415c380 0x7f30740b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.114+0000 7f307a62d640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3074140e80 0x7f30740b0ee0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.115+0000 7f307a62d640 1 --2- 172.21.15.172:0/3200360164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f307413df30 0x7f3074157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.115+0000 7f307a62d640 1 -- 172.21.15.172:0/3200360164 >> 172.21.15.172:0/3200360164 conn(0x7f307406b520 msgr2=0x7f307415eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:02.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.115+0000 7f307a62d640 1 -- 172.21.15.172:0/3200360164 shutdown_connections 2024-04-14T08:41:02.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.115+0000 7f307a62d640 1 -- 172.21.15.172:0/3200360164 wait complete. 2024-04-14T08:41:02.127 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-14T08:41:02.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.227+0000 7f11294de640 1 Processor -- start 2024-04-14T08:41:02.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.227+0000 7f11294de640 1 -- start start 2024-04-14T08:41:02.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f11294de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11240d2830 0x7f11240d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f11294de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f112415e890 0x7f11240d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f11294de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11240cedd0 0x7f11241621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f11294de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f11240cff30 con 0x7f11240d2830 2024-04-14T08:41:02.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f11294de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f11240d2350 con 0x7f112415e890 2024-04-14T08:41:02.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f11294de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11240d3150 con 0x7f11240cedd0 2024-04-14T08:41:02.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f1122ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11240d2830 0x7f11240d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f1122ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11240d2830 0x7f11240d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36444/0 (socket says 172.21.15.172:36444) 2024-04-14T08:41:02.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f1122ffd640 1 -- 172.21.15.172:0/2733273677 learned_addr learned my addr 172.21.15.172:0/2733273677 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:02.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f11237fe640 1 --2- 172.21.15.172:0/2733273677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11240cedd0 0x7f11241621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f11227fc640 1 --2- 172.21.15.172:0/2733273677 >> v2:172.21.15.177:3300/0 conn(0x7f112415e890 0x7f11240d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f1122ffd640 1 -- 172.21.15.172:0/2733273677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11240cedd0 msgr2=0x7f11241621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:02.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f1122ffd640 1 --2- 172.21.15.172:0/2733273677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11240cedd0 0x7f11241621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f1122ffd640 1 -- 172.21.15.172:0/2733273677 >> v2:172.21.15.177:3300/0 conn(0x7f112415e890 msgr2=0x7f11240d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:02.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f1122ffd640 1 --2- 172.21.15.172:0/2733273677 >> v2:172.21.15.177:3300/0 conn(0x7f112415e890 0x7f11240d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.228+0000 7f1122ffd640 1 -- 172.21.15.172:0/2733273677 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11240cf320 con 0x7f11240d2830 2024-04-14T08:41:02.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.229+0000 7f1122ffd640 1 --2- 172.21.15.172:0/2733273677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240d2830 0x7f11240d2c10 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f1124162100 tx=0x7f111402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd791a5add81928f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:02.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.229+0000 7f1121ffb640 1 -- 172.21.15.172:0/2733273677 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114034070 con 0x7f11240d2830 2024-04-14T08:41:02.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.229+0000 7f1121ffb640 1 -- 172.21.15.172:0/2733273677 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1114030860 con 0x7f11240d2830 2024-04-14T08:41:02.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.229+0000 7f1121ffb640 1 -- 172.21.15.172:0/2733273677 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114030c30 con 0x7f11240d2830 2024-04-14T08:41:02.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.230+0000 7f11294de640 1 -- 172.21.15.172:0/2733273677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240d2830 msgr2=0x7f11240d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:02.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.230+0000 7f11294de640 1 --2- 172.21.15.172:0/2733273677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240d2830 0x7f11240d2c10 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f1124162100 tx=0x7f111402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.230+0000 7f11294de640 1 -- 172.21.15.172:0/2733273677 shutdown_connections 2024-04-14T08:41:02.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.230+0000 7f11294de640 1 --2- 172.21.15.172:0/2733273677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11240cedd0 0x7f11241621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.230+0000 7f11294de640 1 --2- 172.21.15.172:0/2733273677 >> v2:172.21.15.177:3300/0 conn(0x7f112415e890 0x7f11240d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.230+0000 7f11294de640 1 --2- 172.21.15.172:0/2733273677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240d2830 0x7f11240d2c10 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.230+0000 7f11294de640 1 -- 172.21.15.172:0/2733273677 >> 172.21.15.172:0/2733273677 conn(0x7f112409d590 msgr2=0x7f11240d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:02.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.230+0000 7f11294de640 1 -- 172.21.15.172:0/2733273677 shutdown_connections 2024-04-14T08:41:02.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.230+0000 7f11294de640 1 -- 172.21.15.172:0/2733273677 wait complete. 2024-04-14T08:41:02.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.231+0000 7f11294de640 1 Processor -- start 2024-04-14T08:41:02.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.231+0000 7f11294de640 1 -- start start 2024-04-14T08:41:02.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11294de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11240cedd0 0x7f112414a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11294de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240d2830 0x7f11240b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11294de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112415e890 0x7f11240b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11294de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11240cfbc0 con 0x7f11240d2830 2024-04-14T08:41:02.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11294de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f11240cf8e0 con 0x7f112415e890 2024-04-14T08:41:02.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11294de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11240d1840 con 0x7f11240cedd0 2024-04-14T08:41:02.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11227fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240d2830 0x7f11240b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11227fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240d2830 0x7f11240b36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36458/0 (socket says 172.21.15.172:36458) 2024-04-14T08:41:02.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11227fc640 1 -- 172.21.15.172:0/3915198238 learned_addr learned my addr 172.21.15.172:0/3915198238 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:02.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11237fe640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112415e890 0x7f11240b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f1122ffd640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11240cedd0 0x7f112414a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11227fc640 1 -- 172.21.15.172:0/3915198238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11240cedd0 msgr2=0x7f112414a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:02.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11227fc640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11240cedd0 0x7f112414a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11227fc640 1 -- 172.21.15.172:0/3915198238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112415e890 msgr2=0x7f11240b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:02.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11227fc640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112415e890 0x7f11240b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11227fc640 1 -- 172.21.15.172:0/3915198238 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112414ab90 con 0x7f11240d2830 2024-04-14T08:41:02.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.232+0000 7f11237fe640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112415e890 0x7f11240b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:41:02.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.233+0000 7f11227fc640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240d2830 0x7f11240b36f0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f111800d830 tx=0x7f111800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:02.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.233+0000 7f110b7fe640 1 -- 172.21.15.172:0/3915198238 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111800b6c0 con 0x7f11240d2830 2024-04-14T08:41:02.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.233+0000 7f110b7fe640 1 -- 172.21.15.172:0/3915198238 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1118019070 con 0x7f11240d2830 2024-04-14T08:41:02.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.233+0000 7f11294de640 1 -- 172.21.15.172:0/3915198238 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11241f49a0 con 0x7f11240d2830 2024-04-14T08:41:02.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.233+0000 7f110b7fe640 1 -- 172.21.15.172:0/3915198238 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111800ba80 con 0x7f11240d2830 2024-04-14T08:41:02.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.233+0000 7f11294de640 1 -- 172.21.15.172:0/3915198238 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11241f4f60 con 0x7f11240d2830 2024-04-14T08:41:02.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.234+0000 7f11294de640 1 -- 172.21.15.172:0/3915198238 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1124068520 con 0x7f11240d2830 2024-04-14T08:41:02.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.234+0000 7f110b7fe640 1 -- 172.21.15.172:0/3915198238 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f111801e460 con 0x7f11240d2830 2024-04-14T08:41:02.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.235+0000 7f110b7fe640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f111006c8f0 0x7f111006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.235+0000 7f110b7fe640 1 -- 172.21.15.172:0/3915198238 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5249+0+0 (secure 0 0 0) 0x7f111808bf50 con 0x7f11240d2830 2024-04-14T08:41:02.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.235+0000 7f1122ffd640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f111006c8f0 0x7f111006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.235+0000 7f1122ffd640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f111006c8f0 0x7f111006edb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1114036000 tx=0x7f111403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:02.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.239+0000 7f110b7fe640 1 -- 172.21.15.172:0/3915198238 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f112414ab90 con 0x7f11240d2830 2024-04-14T08:41:02.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.415+0000 7f11294de640 1 -- 172.21.15.172:0/3915198238 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f11240d2610 con 0x7f11240d2830 2024-04-14T08:41:02.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.416+0000 7f110b7fe640 1 -- 172.21.15.172:0/3915198238 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v97) ==== 84+0+65 (secure 0 0 0) 0x7f111805a540 con 0x7f11240d2830 2024-04-14T08:41:02.416 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:02.416 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-04-14T08:41:02.416 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:41:02.417 INFO:teuthology.orchestra.run.smithi172.stdout: "poolnum": 1, 2024-04-14T08:41:02.417 INFO:teuthology.orchestra.run.smithi172.stdout: "poolname": ".mgr" 2024-04-14T08:41:02.417 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:02.417 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-04-14T08:41:02.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.417+0000 7f11294de640 1 -- 172.21.15.172:0/3915198238 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f111006c8f0 msgr2=0x7f111006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:02.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.417+0000 7f11294de640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f111006c8f0 0x7f111006edb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1114036000 tx=0x7f111403b040 comp rx=0 tx=0).stop 2024-04-14T08:41:02.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.418+0000 7f11294de640 1 -- 172.21.15.172:0/3915198238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240d2830 msgr2=0x7f11240b36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:02.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.418+0000 7f11294de640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240d2830 0x7f11240b36f0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f111800d830 tx=0x7f111800dcf0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.418+0000 7f11294de640 1 -- 172.21.15.172:0/3915198238 shutdown_connections 2024-04-14T08:41:02.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.418+0000 7f11294de640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f111006c8f0 0x7f111006edb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.418+0000 7f11294de640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f112415e890 0x7f11240b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.418+0000 7f11294de640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11240d2830 0x7f11240b36f0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.418+0000 7f11294de640 1 --2- 172.21.15.172:0/3915198238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f11240cedd0 0x7f112414a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.418+0000 7f11294de640 1 -- 172.21.15.172:0/3915198238 >> 172.21.15.172:0/3915198238 conn(0x7f112409d590 msgr2=0x7f112415c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:02.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.418+0000 7f11294de640 1 -- 172.21.15.172:0/3915198238 shutdown_connections 2024-04-14T08:41:02.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.418+0000 7f11294de640 1 -- 172.21.15.172:0/3915198238 wait complete. 2024-04-14T08:41:02.429 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-14T08:41:02.429 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-14T08:41:02.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.535+0000 7ff636ede640 1 Processor -- start 2024-04-14T08:41:02.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.535+0000 7ff636ede640 1 -- start start 2024-04-14T08:41:02.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff636ede640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff630159d50 0x7ff63015c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff636ede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63015cc60 0x7ff6300ade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff636ede640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff6300ae360 0x7ff6301602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff636ede640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff63015a130 con 0x7ff6300ae360 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff636ede640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff6300b0810 con 0x7ff630159d50 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff636ede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff63015fe40 con 0x7ff63015cc60 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff63547b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff6300ae360 0x7ff6301602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff63547b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff6300ae360 0x7ff6301602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36460/0 (socket says 172.21.15.172:36460) 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff63547b640 1 -- 172.21.15.172:0/2134348533 learned_addr learned my addr 172.21.15.172:0/2134348533 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff62ffff640 1 --2- 172.21.15.172:0/2134348533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63015cc60 0x7ff6300ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff634c7a640 1 --2- 172.21.15.172:0/2134348533 >> v2:172.21.15.177:3300/0 conn(0x7ff630159d50 0x7ff63015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff63547b640 1 -- 172.21.15.172:0/2134348533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63015cc60 msgr2=0x7ff6300ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff63547b640 1 --2- 172.21.15.172:0/2134348533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63015cc60 0x7ff6300ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff63547b640 1 -- 172.21.15.172:0/2134348533 >> v2:172.21.15.177:3300/0 conn(0x7ff630159d50 msgr2=0x7ff63015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff63547b640 1 --2- 172.21.15.172:0/2134348533 >> v2:172.21.15.177:3300/0 conn(0x7ff630159d50 0x7ff63015c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff63547b640 1 -- 172.21.15.172:0/2134348533 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6300aff70 con 0x7ff6300ae360 2024-04-14T08:41:02.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff62ffff640 1 --2- 172.21.15.172:0/2134348533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63015cc60 0x7ff6300ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.536+0000 7ff634c7a640 1 --2- 172.21.15.172:0/2134348533 >> v2:172.21.15.177:3300/0 conn(0x7ff630159d50 0x7ff63015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.537+0000 7ff63547b640 1 --2- 172.21.15.172:0/2134348533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6300ae360 0x7ff6301602d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7ff630068c30 tx=0x7ff62802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5558e74c2a101e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.537+0000 7ff62f7fe640 1 -- 172.21.15.172:0/2134348533 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62802c9e0 con 0x7ff6300ae360 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.537+0000 7ff62f7fe640 1 -- 172.21.15.172:0/2134348533 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62802cb80 con 0x7ff6300ae360 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.537+0000 7ff62f7fe640 1 -- 172.21.15.172:0/2134348533 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280343f0 con 0x7ff6300ae360 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.538+0000 7ff636ede640 1 -- 172.21.15.172:0/2134348533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6300ae360 msgr2=0x7ff6301602d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.538+0000 7ff636ede640 1 --2- 172.21.15.172:0/2134348533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6300ae360 0x7ff6301602d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7ff630068c30 tx=0x7ff62802be40 comp rx=0 tx=0).stop 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.538+0000 7ff636ede640 1 -- 172.21.15.172:0/2134348533 shutdown_connections 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.538+0000 7ff636ede640 1 --2- 172.21.15.172:0/2134348533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6300ae360 0x7ff6301602d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.538+0000 7ff636ede640 1 --2- 172.21.15.172:0/2134348533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63015cc60 0x7ff6300ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.538+0000 7ff636ede640 1 --2- 172.21.15.172:0/2134348533 >> v2:172.21.15.177:3300/0 conn(0x7ff630159d50 0x7ff63015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.538+0000 7ff636ede640 1 -- 172.21.15.172:0/2134348533 >> 172.21.15.172:0/2134348533 conn(0x7ff6300d0f20 msgr2=0x7ff6300d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.538+0000 7ff636ede640 1 -- 172.21.15.172:0/2134348533 shutdown_connections 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.538+0000 7ff636ede640 1 -- 172.21.15.172:0/2134348533 wait complete. 2024-04-14T08:41:02.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.539+0000 7ff636ede640 1 Processor -- start 2024-04-14T08:41:02.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.539+0000 7ff636ede640 1 -- start start 2024-04-14T08:41:02.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff636ede640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6300ae360 0x7ff6301e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff636ede640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff630159d50 0x7ff6301f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff636ede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63015cc60 0x7ff6301f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff636ede640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff6300afb40 con 0x7ff630159d50 2024-04-14T08:41:02.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff636ede640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff6300b0530 con 0x7ff6300ae360 2024-04-14T08:41:02.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff636ede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff6300afcc0 con 0x7ff63015cc60 2024-04-14T08:41:02.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff62ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff630159d50 0x7ff6301f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff62ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff630159d50 0x7ff6301f2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36474/0 (socket says 172.21.15.172:36474) 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff62ffff640 1 -- 172.21.15.172:0/806352196 learned_addr learned my addr 172.21.15.172:0/806352196 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff63547b640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63015cc60 0x7ff6301f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff634c7a640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6300ae360 0x7ff6301e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff62ffff640 1 -- 172.21.15.172:0/806352196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63015cc60 msgr2=0x7ff6301f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff62ffff640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63015cc60 0x7ff6301f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff62ffff640 1 -- 172.21.15.172:0/806352196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6300ae360 msgr2=0x7ff6301e69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff62ffff640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6300ae360 0x7ff6301e69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff62ffff640 1 -- 172.21.15.172:0/806352196 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6301e7100 con 0x7ff630159d50 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff63547b640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63015cc60 0x7ff6301f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff634c7a640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6300ae360 0x7ff6301e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.540+0000 7ff62ffff640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff630159d50 0x7ff6301f2910 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00c7f0 tx=0x7ff61c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.541+0000 7ff62d7fa640 1 -- 172.21.15.172:0/806352196 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c015070 con 0x7ff630159d50 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.541+0000 7ff62d7fa640 1 -- 172.21.15.172:0/806352196 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61c007a30 con 0x7ff630159d50 2024-04-14T08:41:02.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.541+0000 7ff636ede640 1 -- 172.21.15.172:0/806352196 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6301e73e0 con 0x7ff630159d50 2024-04-14T08:41:02.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.541+0000 7ff62d7fa640 1 -- 172.21.15.172:0/806352196 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c007e00 con 0x7ff630159d50 2024-04-14T08:41:02.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.541+0000 7ff636ede640 1 -- 172.21.15.172:0/806352196 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff630142100 con 0x7ff630159d50 2024-04-14T08:41:02.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.542+0000 7ff636ede640 1 -- 172.21.15.172:0/806352196 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63015c040 con 0x7ff630159d50 2024-04-14T08:41:02.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.542+0000 7ff62d7fa640 1 -- 172.21.15.172:0/806352196 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff61c018070 con 0x7ff630159d50 2024-04-14T08:41:02.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.543+0000 7ff62d7fa640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff61406c8f0 0x7ff61406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:02.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.543+0000 7ff62d7fa640 1 -- 172.21.15.172:0/806352196 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5249+0+0 (secure 0 0 0) 0x7ff61c08b970 con 0x7ff630159d50 2024-04-14T08:41:02.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.543+0000 7ff634c7a640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff61406c8f0 0x7ff61406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:02.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.543+0000 7ff634c7a640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff61406c8f0 0x7ff61406edb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff6240088b0 tx=0x7ff624008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:02.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.547+0000 7ff62d7fa640 1 -- 172.21.15.172:0/806352196 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff63015c040 con 0x7ff630159d50 2024-04-14T08:41:02.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:02.721+0000 7ff636ede640 1 -- 172.21.15.172:0/806352196 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff630067dd0 con 0x7ff630159d50 2024-04-14T08:41:03.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.124+0000 7ff62d7fa640 1 -- 172.21.15.172:0/806352196 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v98) ==== 138+0+0 (secure 0 0 0) 0x7ff630067dd0 con 0x7ff630159d50 2024-04-14T08:41:03.125 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' created 2024-04-14T08:41:03.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.126+0000 7ff636ede640 1 -- 172.21.15.172:0/806352196 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff61406c8f0 msgr2=0x7ff61406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:03.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.126+0000 7ff636ede640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff61406c8f0 0x7ff61406edb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff6240088b0 tx=0x7ff624008040 comp rx=0 tx=0).stop 2024-04-14T08:41:03.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.126+0000 7ff636ede640 1 -- 172.21.15.172:0/806352196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff630159d50 msgr2=0x7ff6301f2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:03.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.126+0000 7ff636ede640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff630159d50 0x7ff6301f2910 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00c7f0 tx=0x7ff61c00cbf0 comp rx=0 tx=0).stop 2024-04-14T08:41:03.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.127+0000 7ff636ede640 1 -- 172.21.15.172:0/806352196 shutdown_connections 2024-04-14T08:41:03.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.127+0000 7ff636ede640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff61406c8f0 0x7ff61406edb0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:03.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.127+0000 7ff636ede640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff63015cc60 0x7ff6301f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:03.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.127+0000 7ff636ede640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff630159d50 0x7ff6301f2910 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:03.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.127+0000 7ff636ede640 1 --2- 172.21.15.172:0/806352196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6300ae360 0x7ff6301e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:03.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.127+0000 7ff636ede640 1 -- 172.21.15.172:0/806352196 >> 172.21.15.172:0/806352196 conn(0x7ff6300d0f20 msgr2=0x7ff6300d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:03.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.127+0000 7ff636ede640 1 -- 172.21.15.172:0/806352196 shutdown_connections 2024-04-14T08:41:03.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.127+0000 7ff636ede640 1 -- 172.21.15.172:0/806352196 wait complete. 2024-04-14T08:41:03.142 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-14T08:41:03.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.250+0000 7f7b4f55e640 1 Processor -- start 2024-04-14T08:41:03.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.250+0000 7f7b4f55e640 1 -- start start 2024-04-14T08:41:03.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4f55e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7b480ce200 0x7f7b481603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:03.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4f55e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7b48160930 0x7f7b48072450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:03.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4f55e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b4815cf10 con 0x7f7b48159c20 2024-04-14T08:41:03.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4f55e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f7b4815fc80 con 0x7f7b48160930 2024-04-14T08:41:03.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4f55e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f7b4815ff60 con 0x7f7b480ce200 2024-04-14T08:41:03.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4dafb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7b48160930 0x7f7b48072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:03.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4dafb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7b48160930 0x7f7b48072450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36484/0 (socket says 172.21.15.172:36484) 2024-04-14T08:41:03.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4dafb640 1 -- 172.21.15.172:0/3744643050 learned_addr learned my addr 172.21.15.172:0/3744643050 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:03.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4caf9640 1 --2- 172.21.15.172:0/3744643050 >> v2:172.21.15.177:3300/0 conn(0x7f7b480ce200 0x7f7b481603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:03.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4dafb640 1 -- 172.21.15.172:0/3744643050 >> v2:172.21.15.177:3300/0 conn(0x7f7b480ce200 msgr2=0x7f7b481603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:03.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4dafb640 1 --2- 172.21.15.172:0/3744643050 >> v2:172.21.15.177:3300/0 conn(0x7f7b480ce200 0x7f7b481603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:03.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4dafb640 1 -- 172.21.15.172:0/3744643050 >> v1:172.21.15.177:6789/0 conn(0x7f7b48159c20 legacy=0x7f7b4815c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:03.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.251+0000 7f7b4dafb640 1 -- 172.21.15.172:0/3744643050 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4815d1f0 con 0x7f7b48160930 2024-04-14T08:41:03.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.252+0000 7f7b4dafb640 1 --2- 172.21.15.172:0/3744643050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b48160930 0x7f7b48072450 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f7b480ccfd0 tx=0x7f7b4002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26c3bdbd5ee5a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:03.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.252+0000 7f7b3ffff640 1 -- 172.21.15.172:0/3744643050 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40035020 con 0x7f7b48160930 2024-04-14T08:41:03.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.252+0000 7f7b3ffff640 1 -- 172.21.15.172:0/3744643050 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b40031870 con 0x7f7b48160930 2024-04-14T08:41:03.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.252+0000 7f7b3ffff640 1 -- 172.21.15.172:0/3744643050 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40031c40 con 0x7f7b48160930 2024-04-14T08:41:03.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.253+0000 7f7b4f55e640 1 -- 172.21.15.172:0/3744643050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b48160930 msgr2=0x7f7b48072450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:03.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.253+0000 7f7b4f55e640 1 --2- 172.21.15.172:0/3744643050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b48160930 0x7f7b48072450 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f7b480ccfd0 tx=0x7f7b4002b910 comp rx=0 tx=0).stop 2024-04-14T08:41:03.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.253+0000 7f7b4f55e640 1 -- 172.21.15.172:0/3744643050 shutdown_connections 2024-04-14T08:41:03.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.253+0000 7f7b4f55e640 1 --2- 172.21.15.172:0/3744643050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b48160930 0x7f7b48072450 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:03.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.253+0000 7f7b4f55e640 1 --2- 172.21.15.172:0/3744643050 >> v2:172.21.15.177:3300/0 conn(0x7f7b480ce200 0x7f7b481603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:03.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.253+0000 7f7b4f55e640 1 -- 172.21.15.172:0/3744643050 >> 172.21.15.172:0/3744643050 conn(0x7f7b4806b520 msgr2=0x7f7b4806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:03.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.253+0000 7f7b4f55e640 1 -- 172.21.15.172:0/3744643050 shutdown_connections 2024-04-14T08:41:03.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.253+0000 7f7b4f55e640 1 -- 172.21.15.172:0/3744643050 wait complete. 2024-04-14T08:41:03.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.254+0000 7f7b4f55e640 1 Processor -- start 2024-04-14T08:41:03.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.254+0000 7f7b4f55e640 1 -- start start 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4f55e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b480ce200 0x7f7b48151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4f55e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b48159c20 0x7f7b48154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4f55e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b481548b0 0x7f7b480cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4f55e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48072990 con 0x7f7b480ce200 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4f55e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4815a000 con 0x7f7b48159c20 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4f55e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7b48072b10 con 0x7f7b481548b0 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4d2fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b480ce200 0x7f7b48151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4d2fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b480ce200 0x7f7b48151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36500/0 (socket says 172.21.15.172:36500) 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4d2fa640 1 -- 172.21.15.172:0/4271309283 learned_addr learned my addr 172.21.15.172:0/4271309283 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4d2fa640 1 -- 172.21.15.172:0/4271309283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b481548b0 msgr2=0x7f7b480cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4d2fa640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b481548b0 0x7f7b480cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4d2fa640 1 -- 172.21.15.172:0/4271309283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b48159c20 msgr2=0x7f7b48154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4caf9640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b48159c20 0x7f7b48154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4d2fa640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b48159c20 0x7f7b48154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4d2fa640 1 -- 172.21.15.172:0/4271309283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b48154d80 con 0x7f7b480ce200 2024-04-14T08:41:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.255+0000 7f7b4d2fa640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b480ce200 0x7f7b48151160 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400c7a0 tx=0x7f7b4400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.256+0000 7f7b3dffb640 1 -- 172.21.15.172:0/4271309283 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44017070 con 0x7f7b480ce200 2024-04-14T08:41:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.256+0000 7f7b3dffb640 1 -- 172.21.15.172:0/4271309283 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b44003da0 con 0x7f7b480ce200 2024-04-14T08:41:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.256+0000 7f7b4f55e640 1 -- 172.21.15.172:0/4271309283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b48155060 con 0x7f7b480ce200 2024-04-14T08:41:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.256+0000 7f7b3dffb640 1 -- 172.21.15.172:0/4271309283 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b440052b0 con 0x7f7b480ce200 2024-04-14T08:41:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.256+0000 7f7b4f55e640 1 -- 172.21.15.172:0/4271309283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b481e5bc0 con 0x7f7b480ce200 2024-04-14T08:41:03.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.257+0000 7f7b4f55e640 1 -- 172.21.15.172:0/4271309283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b48068590 con 0x7f7b480ce200 2024-04-14T08:41:03.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.257+0000 7f7b3dffb640 1 -- 172.21.15.172:0/4271309283 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b4401a070 con 0x7f7b480ce200 2024-04-14T08:41:03.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.258+0000 7f7b3dffb640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7b2c06c8f0 0x7f7b2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:03.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.258+0000 7f7b3dffb640 1 -- 172.21.15.172:0/4271309283 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5619+0+0 (secure 0 0 0) 0x7f7b4408b210 con 0x7f7b480ce200 2024-04-14T08:41:03.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.258+0000 7f7b4caf9640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7b2c06c8f0 0x7f7b2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:03.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.258+0000 7f7b4caf9640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7b2c06c8f0 0x7f7b2c06edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7b38006fd0 tx=0x7f7b38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:03.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.262+0000 7f7b3dffb640 1 -- 172.21.15.172:0/4271309283 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7b44059690 con 0x7f7b480ce200 2024-04-14T08:41:03.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:03.437+0000 7f7b4f55e640 1 -- 172.21.15.172:0/4271309283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7b48155610 con 0x7f7b480ce200 2024-04-14T08:41:04.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.134+0000 7f7b3dffb640 1 -- 172.21.15.172:0/4271309283 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v99) ==== 170+0+0 (secure 0 0 0) 0x7f7b44058f90 con 0x7f7b480ce200 2024-04-14T08:41:04.135 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' created 2024-04-14T08:41:04.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.136+0000 7f7b4f55e640 1 -- 172.21.15.172:0/4271309283 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7b2c06c8f0 msgr2=0x7f7b2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:04.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.136+0000 7f7b4f55e640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7b2c06c8f0 0x7f7b2c06edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7b38006fd0 tx=0x7f7b38008040 comp rx=0 tx=0).stop 2024-04-14T08:41:04.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.136+0000 7f7b4f55e640 1 -- 172.21.15.172:0/4271309283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b480ce200 msgr2=0x7f7b48151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:04.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.136+0000 7f7b4f55e640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b480ce200 0x7f7b48151160 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400c7a0 tx=0x7f7b4400cc60 comp rx=0 tx=0).stop 2024-04-14T08:41:04.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.136+0000 7f7b4f55e640 1 -- 172.21.15.172:0/4271309283 shutdown_connections 2024-04-14T08:41:04.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.136+0000 7f7b4f55e640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7b2c06c8f0 0x7f7b2c06edb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:04.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.136+0000 7f7b4f55e640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b481548b0 0x7f7b480cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:04.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.136+0000 7f7b4f55e640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b48159c20 0x7f7b48154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:04.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.136+0000 7f7b4f55e640 1 --2- 172.21.15.172:0/4271309283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b480ce200 0x7f7b48151160 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:04.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.137+0000 7f7b4f55e640 1 -- 172.21.15.172:0/4271309283 >> 172.21.15.172:0/4271309283 conn(0x7f7b4806b520 msgr2=0x7f7b4815a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:04.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.137+0000 7f7b4f55e640 1 -- 172.21.15.172:0/4271309283 shutdown_connections 2024-04-14T08:41:04.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.137+0000 7f7b4f55e640 1 -- 172.21.15.172:0/4271309283 wait complete. 2024-04-14T08:41:04.162 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-14T08:41:04.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.267+0000 7f2549e84640 1 Processor -- start 2024-04-14T08:41:04.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.267+0000 7f2549e84640 1 -- start start 2024-04-14T08:41:04.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.267+0000 7f2549e84640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f25440d1a20 0x7f254406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:04.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f2549e84640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f254406d000 0x7f254406d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:04.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f2549e84640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25440701b0 con 0x7f2544071250 2024-04-14T08:41:04.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f2549e84640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2544070dc0 con 0x7f25440d1a20 2024-04-14T08:41:04.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f2549e84640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f254406d990 con 0x7f254406d000 2024-04-14T08:41:04.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f25437fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f25440d1a20 0x7f254406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:04.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f25437fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f25440d1a20 0x7f254406fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36514/0 (socket says 172.21.15.172:36514) 2024-04-14T08:41:04.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f25437fe640 1 -- 172.21.15.172:0/3871436791 learned_addr learned my addr 172.21.15.172:0/3871436791 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:04.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f2543fff640 1 --2- 172.21.15.172:0/3871436791 >> v2:172.21.15.177:3300/0 conn(0x7f254406d000 0x7f254406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:04.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f25437fe640 1 -- 172.21.15.172:0/3871436791 >> v2:172.21.15.177:3300/0 conn(0x7f254406d000 msgr2=0x7f254406d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:04.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f25437fe640 1 --2- 172.21.15.172:0/3871436791 >> v2:172.21.15.177:3300/0 conn(0x7f254406d000 0x7f254406d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:04.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f25437fe640 1 -- 172.21.15.172:0/3871436791 >> v1:172.21.15.177:6789/0 conn(0x7f2544071250 legacy=0x7f25440cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:04.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f25437fe640 1 -- 172.21.15.172:0/3871436791 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2544070490 con 0x7f25440d1a20 2024-04-14T08:41:04.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.268+0000 7f2543fff640 1 --2- 172.21.15.172:0/3871436791 >> v2:172.21.15.177:3300/0 conn(0x7f254406d000 0x7f254406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:41:04.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.269+0000 7f25437fe640 1 --2- 172.21.15.172:0/3871436791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25440d1a20 0x7f254406fb80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f2544068f40 tx=0x7f253402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38a5b7232cdd28c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:04.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.269+0000 7f2542ffd640 1 -- 172.21.15.172:0/3871436791 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253402e070 con 0x7f25440d1a20 2024-04-14T08:41:04.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.269+0000 7f2542ffd640 1 -- 172.21.15.172:0/3871436791 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2534037070 con 0x7f25440d1a20 2024-04-14T08:41:04.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.269+0000 7f2542ffd640 1 -- 172.21.15.172:0/3871436791 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534031a00 con 0x7f25440d1a20 2024-04-14T08:41:04.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.269+0000 7f2549e84640 1 -- 172.21.15.172:0/3871436791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25440d1a20 msgr2=0x7f254406fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:04.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.269+0000 7f2549e84640 1 --2- 172.21.15.172:0/3871436791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25440d1a20 0x7f254406fb80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f2544068f40 tx=0x7f253402b880 comp rx=0 tx=0).stop 2024-04-14T08:41:04.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.270+0000 7f2549e84640 1 -- 172.21.15.172:0/3871436791 shutdown_connections 2024-04-14T08:41:04.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.270+0000 7f2549e84640 1 --2- 172.21.15.172:0/3871436791 >> v2:172.21.15.177:3300/0 conn(0x7f254406d000 0x7f254406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:04.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.270+0000 7f2549e84640 1 --2- 172.21.15.172:0/3871436791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25440d1a20 0x7f254406fb80 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:04.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.270+0000 7f2549e84640 1 -- 172.21.15.172:0/3871436791 >> 172.21.15.172:0/3871436791 conn(0x7f254408c000 msgr2=0x7f25440d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:04.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.270+0000 7f2549e84640 1 -- 172.21.15.172:0/3871436791 shutdown_connections 2024-04-14T08:41:04.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.270+0000 7f2549e84640 1 -- 172.21.15.172:0/3871436791 wait complete. 2024-04-14T08:41:04.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.271+0000 7f2549e84640 1 Processor -- start 2024-04-14T08:41:04.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.271+0000 7f2549e84640 1 -- start start 2024-04-14T08:41:04.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.271+0000 7f2549e84640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f254406d000 0x7f25441f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:04.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.271+0000 7f2549e84640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2544071250 0x7f25440d0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:04.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.271+0000 7f2549e84640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25440d1a20 0x7f25440ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:04.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.271+0000 7f2549e84640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f25440d1e00 con 0x7f25440d1a20 2024-04-14T08:41:04.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.271+0000 7f2549e84640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2544070a50 con 0x7f254406d000 2024-04-14T08:41:04.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.271+0000 7f2549e84640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f25440c7010 con 0x7f2544071250 2024-04-14T08:41:04.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2543fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25440d1a20 0x7f25440ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:04.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2543fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25440d1a20 0x7f25440ca530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36528/0 (socket says 172.21.15.172:36528) 2024-04-14T08:41:04.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2543fff640 1 -- 172.21.15.172:0/3766679131 learned_addr learned my addr 172.21.15.172:0/3766679131 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f25437fe640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f254406d000 0x7f25441f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f253bfff640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2544071250 0x7f25440d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2543fff640 1 -- 172.21.15.172:0/3766679131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2544071250 msgr2=0x7f25440d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2543fff640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2544071250 0x7f25440d0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2543fff640 1 -- 172.21.15.172:0/3766679131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f254406d000 msgr2=0x7f25441f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2543fff640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f254406d000 0x7f25441f4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2543fff640 1 -- 172.21.15.172:0/3766679131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25440c6d10 con 0x7f25440d1a20 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2543fff640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25440d1a20 0x7f25440ca530 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f253c00cef0 tx=0x7f253c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2540ff9640 1 -- 172.21.15.172:0/3766679131 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c01a070 con 0x7f25440d1a20 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2540ff9640 1 -- 172.21.15.172:0/3766679131 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253c00d860 con 0x7f25440d1a20 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2549e84640 1 -- 172.21.15.172:0/3766679131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25441f4c30 con 0x7f25440d1a20 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2540ff9640 1 -- 172.21.15.172:0/3766679131 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c00dc50 con 0x7f25440d1a20 2024-04-14T08:41:04.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.272+0000 7f2549e84640 1 -- 172.21.15.172:0/3766679131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25441f4840 con 0x7f25440d1a20 2024-04-14T08:41:04.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.273+0000 7f2549e84640 1 -- 172.21.15.172:0/3766679131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2544075490 con 0x7f25440d1a20 2024-04-14T08:41:04.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.274+0000 7f2540ff9640 1 -- 172.21.15.172:0/3766679131 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f253c01d070 con 0x7f25440d1a20 2024-04-14T08:41:04.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.274+0000 7f2540ff9640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f252406c8f0 0x7f252406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:04.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.274+0000 7f2540ff9640 1 -- 172.21.15.172:0/3766679131 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6001+0+0 (secure 0 0 0) 0x7f253c08c3c0 con 0x7f25440d1a20 2024-04-14T08:41:04.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.274+0000 7f25437fe640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f252406c8f0 0x7f252406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:04.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.275+0000 7f25437fe640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f252406c8f0 0x7f252406edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2534036000 tx=0x7f253403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:04.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.279+0000 7f2540ff9640 1 -- 172.21.15.172:0/3766679131 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f253c05a1c0 con 0x7f25440d1a20 2024-04-14T08:41:04.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:04.491+0000 7f2549e84640 1 -- 172.21.15.172:0/3766679131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f25440d2820 con 0x7f25440d1a20 2024-04-14T08:41:05.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.136+0000 7f2540ff9640 1 -- 172.21.15.172:0/3766679131 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v72) ==== 174+0+0 (secure 0 0 0) 0x7f253c05e4c0 con 0x7f25440d1a20 2024-04-14T08:41:05.137 INFO:teuthology.orchestra.run.smithi172.stderr:new fs with metadata pool 15 and data pool 16 2024-04-14T08:41:05.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.139+0000 7f2549e84640 1 -- 172.21.15.172:0/3766679131 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f252406c8f0 msgr2=0x7f252406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.139+0000 7f2549e84640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f252406c8f0 0x7f252406edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2534036000 tx=0x7f253403d040 comp rx=0 tx=0).stop 2024-04-14T08:41:05.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.139+0000 7f2549e84640 1 -- 172.21.15.172:0/3766679131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25440d1a20 msgr2=0x7f25440ca530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.139+0000 7f2549e84640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25440d1a20 0x7f25440ca530 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f253c00cef0 tx=0x7f253c00a710 comp rx=0 tx=0).stop 2024-04-14T08:41:05.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.139+0000 7f2549e84640 1 -- 172.21.15.172:0/3766679131 shutdown_connections 2024-04-14T08:41:05.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.140+0000 7f2549e84640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f252406c8f0 0x7f252406edb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.140+0000 7f2549e84640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25440d1a20 0x7f25440ca530 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.140+0000 7f2549e84640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2544071250 0x7f25440d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.140+0000 7f2549e84640 1 --2- 172.21.15.172:0/3766679131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f254406d000 0x7f25441f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.140+0000 7f2549e84640 1 -- 172.21.15.172:0/3766679131 >> 172.21.15.172:0/3766679131 conn(0x7f254408c000 msgr2=0x7f25440cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:05.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.140+0000 7f2549e84640 1 -- 172.21.15.172:0/3766679131 shutdown_connections 2024-04-14T08:41:05.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.141+0000 7f2549e84640 1 -- 172.21.15.172:0/3766679131 wait complete. 2024-04-14T08:41:05.164 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:41:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.268+0000 7fb0eddbb640 1 Processor -- start 2024-04-14T08:41:05.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.268+0000 7fb0eddbb640 1 -- start start 2024-04-14T08:41:05.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0eddbb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb0e80b21d0 0x7fb0e815c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0eddbb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0e80b3700 con 0x7fb0e815c5a0 2024-04-14T08:41:05.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0eddbb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0e80b39e0 con 0x7fb0e80b1df0 2024-04-14T08:41:05.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0eddbb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb0e80b3cc0 con 0x7fb0e80b21d0 2024-04-14T08:41:05.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e7fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb0e815c5a0 0x7fb0e80b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58964/0 (socket says 172.21.15.172:58964) 2024-04-14T08:41:05.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e7fff640 1 -- 172.21.15.172:0/733957208 learned_addr learned my addr 172.21.15.172:0/733957208 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:05.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e6ffd640 1 --2- 172.21.15.172:0/733957208 >> v2:172.21.15.177:3300/0 conn(0x7fb0e80b21d0 0x7fb0e815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4201351033 0 0) 0x7fb0e80b3700 con 0x7fb0e815c5a0 2024-04-14T08:41:05.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0d0003520 con 0x7fb0e815c5a0 2024-04-14T08:41:05.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2484957996 0 0) 0x7fb0d0003520 con 0x7fb0e815c5a0 2024-04-14T08:41:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 >> v2:172.21.15.177:3300/0 conn(0x7fb0e80b21d0 msgr2=0x7fb0e815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 --2- 172.21.15.172:0/733957208 >> v2:172.21.15.177:3300/0 conn(0x7fb0e80b21d0 0x7fb0e815c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 >> v1:172.21.15.177:6789/0 conn(0x7fb0e80b1df0 legacy=0x7fb0e80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e815bb40 con 0x7fb0e815c5a0 2024-04-14T08:41:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fb0e00019f0 con 0x7fb0e815c5a0 2024-04-14T08:41:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 >> v1:172.21.15.172:6789/0 conn(0x7fb0e815c5a0 legacy=0x7fb0e80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 --2- 172.21.15.172:0/733957208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d00039b0 0x7fb0d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 --2- 172.21.15.172:0/733957208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0004800 0x7fb0d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 --2- 172.21.15.172:0/733957208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d00051c0 0x7fb0d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e80b3cc0 con 0x7fb0d0004800 2024-04-14T08:41:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.269+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d000c060 con 0x7fb0d00051c0 2024-04-14T08:41:05.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.270+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d000c260 con 0x7fb0d00039b0 2024-04-14T08:41:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.270+0000 7fb0e6ffd640 1 --2- 172.21.15.172:0/733957208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0004800 0x7fb0d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.270+0000 7fb0e7fff640 1 --2- 172.21.15.172:0/733957208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d00051c0 0x7fb0d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.270+0000 7fb0e77fe640 1 --2- 172.21.15.172:0/733957208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d00039b0 0x7fb0d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.270+0000 7fb0e6ffd640 1 -- 172.21.15.172:0/733957208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d00039b0 msgr2=0x7fb0d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.270+0000 7fb0e6ffd640 1 --2- 172.21.15.172:0/733957208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d00039b0 0x7fb0d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.270+0000 7fb0e6ffd640 1 -- 172.21.15.172:0/733957208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d00051c0 msgr2=0x7fb0d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.270+0000 7fb0e6ffd640 1 --2- 172.21.15.172:0/733957208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d00051c0 0x7fb0d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.270+0000 7fb0e6ffd640 1 -- 172.21.15.172:0/733957208 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d0003520 con 0x7fb0d0004800 2024-04-14T08:41:05.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.270+0000 7fb0e7fff640 1 -- 172.21.15.172:0/733957208 reap_dead start 2024-04-14T08:41:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.271+0000 7fb0e6ffd640 1 --2- 172.21.15.172:0/733957208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0004800 0x7fb0d0004c80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009a70 tx=0x7fb0d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f678c7b68d1fed35 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.271+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc039070 con 0x7fb0d0004800 2024-04-14T08:41:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.271+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0dc02c680 con 0x7fb0d0004800 2024-04-14T08:41:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.271+0000 7fb0e67fc640 1 -- 172.21.15.172:0/733957208 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc009670 con 0x7fb0d0004800 2024-04-14T08:41:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.271+0000 7fb0eddbb640 1 -- 172.21.15.172:0/733957208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0004800 msgr2=0x7fb0d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.271+0000 7fb0eddbb640 1 --2- 172.21.15.172:0/733957208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0004800 0x7fb0d0004c80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009a70 tx=0x7fb0d0005830 comp rx=0 tx=0).stop 2024-04-14T08:41:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.271+0000 7fb0eddbb640 1 -- 172.21.15.172:0/733957208 shutdown_connections 2024-04-14T08:41:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.271+0000 7fb0eddbb640 1 --2- 172.21.15.172:0/733957208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0004800 0x7fb0d0004c80 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.271+0000 7fb0eddbb640 1 -- 172.21.15.172:0/733957208 >> 172.21.15.172:0/733957208 conn(0x7fb0e806ec30 msgr2=0x7fb0e806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.271+0000 7fb0eddbb640 1 -- 172.21.15.172:0/733957208 shutdown_connections 2024-04-14T08:41:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.271+0000 7fb0eddbb640 1 -- 172.21.15.172:0/733957208 wait complete. 2024-04-14T08:41:05.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.272+0000 7fb0eddbb640 1 Processor -- start 2024-04-14T08:41:05.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.273+0000 7fb0eddbb640 1 -- start start 2024-04-14T08:41:05.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.273+0000 7fb0eddbb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d0004800 0x7fb0e80d0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.273+0000 7fb0eddbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0e80cdd00 0x7fb0e813c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.273+0000 7fb0eddbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0e80ce180 0x7fb0e813c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.273+0000 7fb0eddbb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e815c1e0 con 0x7fb0e80ce180 2024-04-14T08:41:05.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.273+0000 7fb0eddbb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e815c060 con 0x7fb0d0004800 2024-04-14T08:41:05.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.273+0000 7fb0eddbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e815c360 con 0x7fb0e80cdd00 2024-04-14T08:41:05.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.273+0000 7fb0e7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0e80ce180 0x7fb0e813c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0e7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0e80ce180 0x7fb0e813c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36544/0 (socket says 172.21.15.172:36544) 2024-04-14T08:41:05.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0e7fff640 1 -- 172.21.15.172:0/3876933471 learned_addr learned my addr 172.21.15.172:0/3876933471 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:05.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0e6ffd640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0e80cdd00 0x7fb0e813c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0e77fe640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d0004800 0x7fb0e80d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0e7fff640 1 -- 172.21.15.172:0/3876933471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0e80cdd00 msgr2=0x7fb0e813c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0e7fff640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0e80cdd00 0x7fb0e813c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0e7fff640 1 -- 172.21.15.172:0/3876933471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d0004800 msgr2=0x7fb0e80d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0e7fff640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d0004800 0x7fb0e80d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0e7fff640 1 -- 172.21.15.172:0/3876933471 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e81f4160 con 0x7fb0e80ce180 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0e7fff640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0e80ce180 0x7fb0e813c720 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80ccd50 tx=0x7fb0e0007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0cffff640 1 -- 172.21.15.172:0/3876933471 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000aaa0 con 0x7fb0e80ce180 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0cffff640 1 -- 172.21.15.172:0/3876933471 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e0019070 con 0x7fb0e80ce180 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0eddbb640 1 -- 172.21.15.172:0/3876933471 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e80ce750 con 0x7fb0e80ce180 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0cffff640 1 -- 172.21.15.172:0/3876933471 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00133f0 con 0x7fb0e80ce180 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.274+0000 7fb0eddbb640 1 -- 172.21.15.172:0/3876933471 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e80ced10 con 0x7fb0e80ce180 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.275+0000 7fb0eddbb640 1 -- 172.21.15.172:0/3876933471 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e8068520 con 0x7fb0e80ce180 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.276+0000 7fb0cffff640 1 -- 172.21.15.172:0/3876933471 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0e0013590 con 0x7fb0e80ce180 2024-04-14T08:41:05.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.276+0000 7fb0cffff640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0d406c8f0 0x7fb0d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.276+0000 7fb0cffff640 1 -- 172.21.15.172:0/3876933471 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fb0e008bfd0 con 0x7fb0e80ce180 2024-04-14T08:41:05.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.276+0000 7fb0e77fe640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0d406c8f0 0x7fb0d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.276+0000 7fb0e77fe640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0d406c8f0 0x7fb0d406edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8000c00 tx=0x7fb0d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:05.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.281+0000 7fb0cffff640 1 -- 172.21.15.172:0/3876933471 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb0e0059c20 con 0x7fb0e80ce180 2024-04-14T08:41:05.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.457+0000 7fb0eddbb640 1 -- 172.21.15.172:0/3876933471 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0e8071510 con 0x7fb0e80ce180 2024-04-14T08:41:05.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.458+0000 7fb0cffff640 1 -- 172.21.15.172:0/3876933471 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16949 (secure 0 0 0) 0x7fb0e005df20 con 0x7fb0e80ce180 2024-04-14T08:41:05.459 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:05.459 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":100,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:41:04.492454+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:41:02.725765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-14T08:41:03.442157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:41:05.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.460+0000 7fb0eddbb640 1 -- 172.21.15.172:0/3876933471 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0d406c8f0 msgr2=0x7fb0d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.460+0000 7fb0eddbb640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0d406c8f0 0x7fb0d406edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8000c00 tx=0x7fb0d8009040 comp rx=0 tx=0).stop 2024-04-14T08:41:05.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.460+0000 7fb0eddbb640 1 -- 172.21.15.172:0/3876933471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0e80ce180 msgr2=0x7fb0e813c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.460+0000 7fb0eddbb640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0e80ce180 0x7fb0e813c720 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80ccd50 tx=0x7fb0e0007a90 comp rx=0 tx=0).stop 2024-04-14T08:41:05.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.460+0000 7fb0eddbb640 1 -- 172.21.15.172:0/3876933471 shutdown_connections 2024-04-14T08:41:05.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.460+0000 7fb0eddbb640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0d406c8f0 0x7fb0d406edb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.460+0000 7fb0eddbb640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0e80ce180 0x7fb0e813c720 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.460+0000 7fb0eddbb640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0e80cdd00 0x7fb0e813c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.460+0000 7fb0eddbb640 1 --2- 172.21.15.172:0/3876933471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d0004800 0x7fb0e80d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.460+0000 7fb0eddbb640 1 -- 172.21.15.172:0/3876933471 >> 172.21.15.172:0/3876933471 conn(0x7fb0e806ec30 msgr2=0x7fb0e806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:05.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.461+0000 7fb0eddbb640 1 -- 172.21.15.172:0/3876933471 shutdown_connections 2024-04-14T08:41:05.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.461+0000 7fb0eddbb640 1 -- 172.21.15.172:0/3876933471 wait complete. 2024-04-14T08:41:05.472 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:41:05.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.573+0000 7f0c6aa88640 1 Processor -- start 2024-04-14T08:41:05.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.573+0000 7f0c6aa88640 1 -- start start 2024-04-14T08:41:05.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.574+0000 7f0c6aa88640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0c6415c150 0x7f0c640d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.574+0000 7f0c6aa88640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0c640d1840 0x7f0c64161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.574+0000 7f0c6aa88640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c640d0200 con 0x7f0c64162530 2024-04-14T08:41:05.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.574+0000 7f0c6aa88640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0c640d1cb0 con 0x7f0c640d1840 2024-04-14T08:41:05.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.574+0000 7f0c6aa88640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f0c64162980 con 0x7f0c6415c150 2024-04-14T08:41:05.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.574+0000 7f0c63fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0c640d1840 0x7f0c64161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.574+0000 7f0c69025640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f0c64162530 0x7f0c6415ea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58976/0 (socket says 172.21.15.172:58976) 2024-04-14T08:41:05.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.574+0000 7f0c69025640 1 -- 172.21.15.172:0/3085135266 learned_addr learned my addr 172.21.15.172:0/3085135266 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:05.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.575+0000 7f0c68824640 1 --2- 172.21.15.172:0/3085135266 >> v2:172.21.15.177:3300/0 conn(0x7f0c6415c150 0x7f0c640d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.575+0000 7f0c637fe640 1 -- 172.21.15.172:0/3085135266 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 232170294 0 0) 0x7f0c640d0200 con 0x7f0c64162530 2024-04-14T08:41:05.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.575+0000 7f0c637fe640 1 -- 172.21.15.172:0/3085135266 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c4c003520 con 0x7f0c64162530 2024-04-14T08:41:05.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.575+0000 7f0c63fff640 1 -- 172.21.15.172:0/3085135266 >> v2:172.21.15.177:3300/0 conn(0x7f0c6415c150 msgr2=0x7f0c640d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:05.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.575+0000 7f0c63fff640 1 --2- 172.21.15.172:0/3085135266 >> v2:172.21.15.177:3300/0 conn(0x7f0c6415c150 0x7f0c640d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.575+0000 7f0c63fff640 1 -- 172.21.15.172:0/3085135266 >> v1:172.21.15.172:6789/0 conn(0x7f0c64162530 legacy=0x7f0c6415ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.575+0000 7f0c63fff640 1 -- 172.21.15.172:0/3085135266 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c640cf160 con 0x7f0c640d1840 2024-04-14T08:41:05.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.575+0000 7f0c63fff640 1 --2- 172.21.15.172:0/3085135266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c640d1840 0x7f0c64161ff0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f0c6406bf30 tx=0x7f0c5802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6603d4a672ee5c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:05.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.575+0000 7f0c637fe640 1 -- 172.21.15.172:0/3085135266 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58035020 con 0x7f0c640d1840 2024-04-14T08:41:05.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.576+0000 7f0c637fe640 1 -- 172.21.15.172:0/3085135266 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c58031830 con 0x7f0c640d1840 2024-04-14T08:41:05.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.576+0000 7f0c637fe640 1 -- 172.21.15.172:0/3085135266 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58031c00 con 0x7f0c640d1840 2024-04-14T08:41:05.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.576+0000 7f0c6aa88640 1 -- 172.21.15.172:0/3085135266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c640d1840 msgr2=0x7f0c64161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.576+0000 7f0c6aa88640 1 --2- 172.21.15.172:0/3085135266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c640d1840 0x7f0c64161ff0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f0c6406bf30 tx=0x7f0c5802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.576+0000 7f0c6aa88640 1 -- 172.21.15.172:0/3085135266 shutdown_connections 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.576+0000 7f0c6aa88640 1 --2- 172.21.15.172:0/3085135266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c640d1840 0x7f0c64161ff0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.576+0000 7f0c6aa88640 1 --2- 172.21.15.172:0/3085135266 >> v2:172.21.15.177:3300/0 conn(0x7f0c6415c150 0x7f0c640d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.576+0000 7f0c6aa88640 1 -- 172.21.15.172:0/3085135266 >> 172.21.15.172:0/3085135266 conn(0x7f0c6409d580 msgr2=0x7f0c640d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.576+0000 7f0c6aa88640 1 -- 172.21.15.172:0/3085135266 shutdown_connections 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.576+0000 7f0c6aa88640 1 -- 172.21.15.172:0/3085135266 wait complete. 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.577+0000 7f0c6aa88640 1 Processor -- start 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.577+0000 7f0c6aa88640 1 -- start start 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c6aa88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c640d1840 0x7f0c640d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c6aa88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c6415c150 0x7f0c640b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c6aa88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c64162530 0x7f0c6415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c6aa88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0c640cf720 con 0x7f0c640d1840 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c6aa88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0c640cef10 con 0x7f0c64162530 2024-04-14T08:41:05.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c6aa88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0c640cfa00 con 0x7f0c6415c150 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c68824640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c640d1840 0x7f0c640d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c68824640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c640d1840 0x7f0c640d20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36560/0 (socket says 172.21.15.172:36560) 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c68824640 1 -- 172.21.15.172:0/793561774 learned_addr learned my addr 172.21.15.172:0/793561774 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c63fff640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c6415c150 0x7f0c640b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c69025640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c64162530 0x7f0c6415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c68824640 1 -- 172.21.15.172:0/793561774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c6415c150 msgr2=0x7f0c640b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c68824640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c6415c150 0x7f0c640b6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c68824640 1 -- 172.21.15.172:0/793561774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c64162530 msgr2=0x7f0c6415a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c68824640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c64162530 0x7f0c6415a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c68824640 1 -- 172.21.15.172:0/793561774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c640cc9d0 con 0x7f0c640d1840 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c63fff640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c6415c150 0x7f0c640b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.578+0000 7f0c69025640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c64162530 0x7f0c6415a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:41:05.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.579+0000 7f0c68824640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c640d1840 0x7f0c640d20a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400d800 tx=0x7f0c5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.579+0000 7f0c617fa640 1 -- 172.21.15.172:0/793561774 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400b6c0 con 0x7f0c640d1840 2024-04-14T08:41:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.579+0000 7f0c617fa640 1 -- 172.21.15.172:0/793561774 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c54019070 con 0x7f0c640d1840 2024-04-14T08:41:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.579+0000 7f0c6aa88640 1 -- 172.21.15.172:0/793561774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c640b7950 con 0x7f0c640d1840 2024-04-14T08:41:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.579+0000 7f0c617fa640 1 -- 172.21.15.172:0/793561774 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400ba80 con 0x7f0c640d1840 2024-04-14T08:41:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.579+0000 7f0c6aa88640 1 -- 172.21.15.172:0/793561774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c640b81f0 con 0x7f0c640d1840 2024-04-14T08:41:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.580+0000 7f0c6aa88640 1 -- 172.21.15.172:0/793561774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c64075490 con 0x7f0c640d1840 2024-04-14T08:41:05.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.580+0000 7f0c617fa640 1 -- 172.21.15.172:0/793561774 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c5400bc20 con 0x7f0c640d1840 2024-04-14T08:41:05.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.581+0000 7f0c617fa640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0c5006c8f0 0x7f0c5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.581+0000 7f0c617fa640 1 -- 172.21.15.172:0/793561774 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f0c5408c180 con 0x7f0c640d1840 2024-04-14T08:41:05.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.581+0000 7f0c63fff640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0c5006c8f0 0x7f0c5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.581+0000 7f0c63fff640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0c5006c8f0 0x7f0c5006edb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0c58036000 tx=0x7f0c58008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:05.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.585+0000 7f0c617fa640 1 -- 172.21.15.172:0/793561774 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0c5401c360 con 0x7f0c640d1840 2024-04-14T08:41:05.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.761+0000 7f0c6aa88640 1 -- 172.21.15.172:0/793561774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c640d49c0 con 0x7f0c640d1840 2024-04-14T08:41:05.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.762+0000 7f0c617fa640 1 -- 172.21.15.172:0/793561774 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16949 (secure 0 0 0) 0x7f0c5405a420 con 0x7f0c640d1840 2024-04-14T08:41:05.763 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:05.763 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":100,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:41:04.492454+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:41:02.725765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-14T08:41:03.442157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:41:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.764+0000 7f0c6aa88640 1 -- 172.21.15.172:0/793561774 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0c5006c8f0 msgr2=0x7f0c5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.764+0000 7f0c6aa88640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0c5006c8f0 0x7f0c5006edb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0c58036000 tx=0x7f0c58008030 comp rx=0 tx=0).stop 2024-04-14T08:41:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.764+0000 7f0c6aa88640 1 -- 172.21.15.172:0/793561774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c640d1840 msgr2=0x7f0c640d20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.764+0000 7f0c6aa88640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c640d1840 0x7f0c640d20a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400d800 tx=0x7f0c5400dcc0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.764+0000 7f0c6aa88640 1 -- 172.21.15.172:0/793561774 shutdown_connections 2024-04-14T08:41:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.764+0000 7f0c6aa88640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0c5006c8f0 0x7f0c5006edb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.764+0000 7f0c6aa88640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c64162530 0x7f0c6415a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.764+0000 7f0c6aa88640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c6415c150 0x7f0c640b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.764+0000 7f0c6aa88640 1 --2- 172.21.15.172:0/793561774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c640d1840 0x7f0c640d20a0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.764+0000 7f0c6aa88640 1 -- 172.21.15.172:0/793561774 >> 172.21.15.172:0/793561774 conn(0x7f0c6409d580 msgr2=0x7f0c640b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.764+0000 7f0c6aa88640 1 -- 172.21.15.172:0/793561774 shutdown_connections 2024-04-14T08:41:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.764+0000 7f0c6aa88640 1 -- 172.21.15.172:0/793561774 wait complete. 2024-04-14T08:41:05.776 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-14T08:41:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.879+0000 7f475c95e640 1 Processor -- start 2024-04-14T08:41:05.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.879+0000 7f475c95e640 1 -- start start 2024-04-14T08:41:05.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.880+0000 7f475c95e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f47580b65a0 0x7f47581629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.880+0000 7f475c95e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47580b3750 con 0x7f475815e590 2024-04-14T08:41:05.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.880+0000 7f475c95e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47580cef20 con 0x7f47580cea60 2024-04-14T08:41:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.880+0000 7f475c95e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f47580cf200 con 0x7f47580b65a0 2024-04-14T08:41:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.880+0000 7f475659c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f475815e590 0x7f47580b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58990/0 (socket says 172.21.15.172:58990) 2024-04-14T08:41:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.880+0000 7f475659c640 1 -- 172.21.15.172:0/726967119 learned_addr learned my addr 172.21.15.172:0/726967119 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.881+0000 7f4755d9b640 1 --2- 172.21.15.172:0/726967119 >> v2:172.21.15.177:3300/0 conn(0x7f47580b65a0 0x7f47581629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.881+0000 7f475559a640 1 -- 172.21.15.172:0/726967119 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 45191166 0 0) 0x7f47580b3750 con 0x7f475815e590 2024-04-14T08:41:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.881+0000 7f475559a640 1 -- 172.21.15.172:0/726967119 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4740003520 con 0x7f475815e590 2024-04-14T08:41:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.881+0000 7f475559a640 1 -- 172.21.15.172:0/726967119 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1878327117 0 0) 0x7f47580cef20 con 0x7f47580cea60 2024-04-14T08:41:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.881+0000 7f475559a640 1 -- 172.21.15.172:0/726967119 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47580b3750 con 0x7f47580cea60 2024-04-14T08:41:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.881+0000 7f475559a640 1 -- 172.21.15.172:0/726967119 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1048451385 0 0) 0x7f4740003520 con 0x7f475815e590 2024-04-14T08:41:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.881+0000 7f4755d9b640 1 -- 172.21.15.172:0/726967119 >> v1:172.21.15.177:6789/0 conn(0x7f47580cea60 legacy=0x7f47580d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.881+0000 7f4755d9b640 1 -- 172.21.15.172:0/726967119 >> v1:172.21.15.172:6789/0 conn(0x7f475815e590 legacy=0x7f47580b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.881+0000 7f4755d9b640 1 -- 172.21.15.172:0/726967119 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47580cf4e0 con 0x7f47580b65a0 2024-04-14T08:41:05.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.882+0000 7f4755d9b640 1 --2- 172.21.15.172:0/726967119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47580b65a0 0x7f47581629c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f474c0097a0 tx=0x7f474c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48f9e13d767ee5bb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:05.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.882+0000 7f475559a640 1 -- 172.21.15.172:0/726967119 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c038070 con 0x7f47580b65a0 2024-04-14T08:41:05.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.882+0000 7f475559a640 1 -- 172.21.15.172:0/726967119 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474c031500 con 0x7f47580b65a0 2024-04-14T08:41:05.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.882+0000 7f475559a640 1 -- 172.21.15.172:0/726967119 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47480019f0 con 0x7f47580b65a0 2024-04-14T08:41:05.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.882+0000 7f475c95e640 1 -- 172.21.15.172:0/726967119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47580b65a0 msgr2=0x7f47581629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.882+0000 7f475c95e640 1 --2- 172.21.15.172:0/726967119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47580b65a0 0x7f47581629c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f474c0097a0 tx=0x7f474c02b820 comp rx=0 tx=0).stop 2024-04-14T08:41:05.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.883+0000 7f475c95e640 1 -- 172.21.15.172:0/726967119 shutdown_connections 2024-04-14T08:41:05.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.883+0000 7f475c95e640 1 --2- 172.21.15.172:0/726967119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47580b65a0 0x7f47581629c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.883+0000 7f475c95e640 1 -- 172.21.15.172:0/726967119 >> 172.21.15.172:0/726967119 conn(0x7f47580d0e90 msgr2=0x7f47580d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:05.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.883+0000 7f475c95e640 1 -- 172.21.15.172:0/726967119 shutdown_connections 2024-04-14T08:41:05.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.883+0000 7f475c95e640 1 -- 172.21.15.172:0/726967119 wait complete. 2024-04-14T08:41:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.884+0000 7f475c95e640 1 Processor -- start 2024-04-14T08:41:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.884+0000 7f475c95e640 1 -- start start 2024-04-14T08:41:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f475c95e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47580b65a0 0x7f47581f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f475c95e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47580cea60 0x7f475815f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f475c95e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475815e590 0x7f475815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f475c95e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f47580d2ff0 con 0x7f475815e590 2024-04-14T08:41:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f475c95e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f47580d2e70 con 0x7f47580cea60 2024-04-14T08:41:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f475c95e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f47580d3170 con 0x7f47580b65a0 2024-04-14T08:41:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4756d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475815e590 0x7f475815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4756d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475815e590 0x7f475815f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36572/0 (socket says 172.21.15.172:36572) 2024-04-14T08:41:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4756d9d640 1 -- 172.21.15.172:0/3905572810 learned_addr learned my addr 172.21.15.172:0/3905572810 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f475659c640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47580b65a0 0x7f47581f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4755d9b640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47580cea60 0x7f475815f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4756d9d640 1 -- 172.21.15.172:0/3905572810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47580b65a0 msgr2=0x7f47581f47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4756d9d640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47580b65a0 0x7f47581f47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4756d9d640 1 -- 172.21.15.172:0/3905572810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47580cea60 msgr2=0x7f475815f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4756d9d640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47580cea60 0x7f475815f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4756d9d640 1 -- 172.21.15.172:0/3905572810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47580cc340 con 0x7f475815e590 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4756d9d640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475815e590 0x7f475815f9d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f475000c460 tx=0x7f475000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4746ffd640 1 -- 172.21.15.172:0/3905572810 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750017070 con 0x7f475815e590 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4746ffd640 1 -- 172.21.15.172:0/3905572810 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47500052d0 con 0x7f475815e590 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f475c95e640 1 -- 172.21.15.172:0/3905572810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47580cb580 con 0x7f475815e590 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f4746ffd640 1 -- 172.21.15.172:0/3905572810 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750003830 con 0x7f475815e590 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.885+0000 7f475c95e640 1 -- 172.21.15.172:0/3905572810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47580cbb40 con 0x7f475815e590 2024-04-14T08:41:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.887+0000 7f475c95e640 1 -- 172.21.15.172:0/3905572810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4758068520 con 0x7f475815e590 2024-04-14T08:41:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.887+0000 7f4746ffd640 1 -- 172.21.15.172:0/3905572810 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f475001a070 con 0x7f475815e590 2024-04-14T08:41:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.887+0000 7f4746ffd640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f473806c8f0 0x7f473806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.887+0000 7f4746ffd640 1 -- 172.21.15.172:0/3905572810 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f475008afa0 con 0x7f475815e590 2024-04-14T08:41:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.887+0000 7f475659c640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f473806c8f0 0x7f473806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.888+0000 7f475659c640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f473806c8f0 0x7f473806edb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4748001560 tx=0x7f4748009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:05.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:05.892+0000 7f4746ffd640 1 -- 172.21.15.172:0/3905572810 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4750058c00 con 0x7f475815e590 2024-04-14T08:41:06.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.103+0000 7f475c95e640 1 -- 172.21.15.172:0/3905572810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f475815dd40 con 0x7f475815e590 2024-04-14T08:41:06.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.147+0000 7f4746ffd640 1 -- 172.21.15.172:0/3905572810 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) ==== 118+0+0 (secure 0 0 0) 0x7f475005cf00 con 0x7f475815e590 2024-04-14T08:41:06.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.149+0000 7f475c95e640 1 -- 172.21.15.172:0/3905572810 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f473806c8f0 msgr2=0x7f473806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:06.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.149+0000 7f475c95e640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f473806c8f0 0x7f473806edb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4748001560 tx=0x7f4748009040 comp rx=0 tx=0).stop 2024-04-14T08:41:06.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.149+0000 7f475c95e640 1 -- 172.21.15.172:0/3905572810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475815e590 msgr2=0x7f475815f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:06.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.149+0000 7f475c95e640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475815e590 0x7f475815f9d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f475000c460 tx=0x7f475000c920 comp rx=0 tx=0).stop 2024-04-14T08:41:06.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.149+0000 7f475c95e640 1 -- 172.21.15.172:0/3905572810 shutdown_connections 2024-04-14T08:41:06.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.149+0000 7f475c95e640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f473806c8f0 0x7f473806edb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:06.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.149+0000 7f475c95e640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f475815e590 0x7f475815f9d0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:06.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.149+0000 7f475c95e640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47580cea60 0x7f475815f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:06.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.149+0000 7f475c95e640 1 --2- 172.21.15.172:0/3905572810 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f47580b65a0 0x7f47581f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:06.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.149+0000 7f475c95e640 1 -- 172.21.15.172:0/3905572810 >> 172.21.15.172:0/3905572810 conn(0x7f47580d0e90 msgr2=0x7f475806c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:06.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.149+0000 7f475c95e640 1 -- 172.21.15.172:0/3905572810 shutdown_connections 2024-04-14T08:41:06.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.149+0000 7f475c95e640 1 -- 172.21.15.172:0/3905572810 wait complete. 2024-04-14T08:41:06.161 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-14T08:41:06.161 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-14T08:41:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.266+0000 7ff671de0640 1 Processor -- start 2024-04-14T08:41:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.266+0000 7ff671de0640 1 -- start start 2024-04-14T08:41:06.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff671de0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff66c15b770 0x7ff66c0cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:06.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff671de0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff66c15bb50 0x7ff66c15bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:06.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff671de0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff66c0ce390 con 0x7ff66c0cf430 2024-04-14T08:41:06.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff671de0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff66c0ce670 con 0x7ff66c15bb50 2024-04-14T08:41:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff671de0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff66c0ce950 con 0x7ff66c15b770 2024-04-14T08:41:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff66bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff66c15bb50 0x7ff66c15bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff66bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff66c15bb50 0x7ff66c15bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36584/0 (socket says 172.21.15.172:36584) 2024-04-14T08:41:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff66bfff640 1 -- 172.21.15.172:0/1430409280 learned_addr learned my addr 172.21.15.172:0/1430409280 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff66b7fe640 1 --2- 172.21.15.172:0/1430409280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff66c15b770 0x7ff66c0cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff66bfff640 1 -- 172.21.15.172:0/1430409280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff66c15b770 msgr2=0x7ff66c0cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff66bfff640 1 --2- 172.21.15.172:0/1430409280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff66c15b770 0x7ff66c0cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff66bfff640 1 -- 172.21.15.172:0/1430409280 >> v1:172.21.15.177:6789/0 conn(0x7ff66c0cf430 legacy=0x7ff66c0731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.267+0000 7ff66bfff640 1 -- 172.21.15.172:0/1430409280 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c0cec30 con 0x7ff66c15bb50 2024-04-14T08:41:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.268+0000 7ff66bfff640 1 --2- 172.21.15.172:0/1430409280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff66c15bb50 0x7ff66c15bfe0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7ff664009e50 tx=0x7ff66402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8ea19071eb6c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.268+0000 7ff66a7fc640 1 -- 172.21.15.172:0/1430409280 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66402ca70 con 0x7ff66c15bb50 2024-04-14T08:41:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.268+0000 7ff66a7fc640 1 -- 172.21.15.172:0/1430409280 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff66402cc10 con 0x7ff66c15bb50 2024-04-14T08:41:06.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.268+0000 7ff66a7fc640 1 -- 172.21.15.172:0/1430409280 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6640343f0 con 0x7ff66c15bb50 2024-04-14T08:41:06.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.269+0000 7ff671de0640 1 -- 172.21.15.172:0/1430409280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff66c15bb50 msgr2=0x7ff66c15bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:06.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.269+0000 7ff671de0640 1 --2- 172.21.15.172:0/1430409280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff66c15bb50 0x7ff66c15bfe0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7ff664009e50 tx=0x7ff66402bed0 comp rx=0 tx=0).stop 2024-04-14T08:41:06.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.269+0000 7ff671de0640 1 -- 172.21.15.172:0/1430409280 shutdown_connections 2024-04-14T08:41:06.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.269+0000 7ff671de0640 1 --2- 172.21.15.172:0/1430409280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff66c15bb50 0x7ff66c15bfe0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.269+0000 7ff671de0640 1 --2- 172.21.15.172:0/1430409280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff66c15b770 0x7ff66c0cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.269+0000 7ff671de0640 1 -- 172.21.15.172:0/1430409280 >> 172.21.15.172:0/1430409280 conn(0x7ff66c0b1c60 msgr2=0x7ff66c0b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.269+0000 7ff671de0640 1 -- 172.21.15.172:0/1430409280 shutdown_connections 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.269+0000 7ff671de0640 1 -- 172.21.15.172:0/1430409280 wait complete. 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.270+0000 7ff671de0640 1 Processor -- start 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.270+0000 7ff671de0640 1 -- start start 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff671de0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff66c0cf430 0x7ff66c15a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff671de0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff66c15b770 0x7ff66c141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff671de0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff66c15bb50 0x7ff66c15e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff671de0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c15c8c0 con 0x7ff66c0cf430 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff671de0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c15c520 con 0x7ff66c15bb50 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff671de0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff66c15ca40 con 0x7ff66c15b770 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff66b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff66c0cf430 0x7ff66c15a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:06.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff66b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff66c0cf430 0x7ff66c15a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36594/0 (socket says 172.21.15.172:36594) 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff66b7fe640 1 -- 172.21.15.172:0/803061938 learned_addr learned my addr 172.21.15.172:0/803061938 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff66affd640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff66c15b770 0x7ff66c141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff66bfff640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff66c15bb50 0x7ff66c15e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff66b7fe640 1 -- 172.21.15.172:0/803061938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff66c15b770 msgr2=0x7ff66c141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff66b7fe640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff66c15b770 0x7ff66c141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff66b7fe640 1 -- 172.21.15.172:0/803061938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff66c15bb50 msgr2=0x7ff66c15e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff66b7fe640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff66c15bb50 0x7ff66c15e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff66b7fe640 1 -- 172.21.15.172:0/803061938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c1420e0 con 0x7ff66c0cf430 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff66b7fe640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff66c0cf430 0x7ff66c15a110 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00e9b0 tx=0x7ff65c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.271+0000 7ff653fff640 1 -- 172.21.15.172:0/803061938 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c017070 con 0x7ff66c0cf430 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.272+0000 7ff653fff640 1 -- 172.21.15.172:0/803061938 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65c00c9a0 con 0x7ff66c0cf430 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.272+0000 7ff653fff640 1 -- 172.21.15.172:0/803061938 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c00cd90 con 0x7ff66c0cf430 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.272+0000 7ff671de0640 1 -- 172.21.15.172:0/803061938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c0b4300 con 0x7ff66c0cf430 2024-04-14T08:41:06.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.272+0000 7ff671de0640 1 -- 172.21.15.172:0/803061938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c0b48c0 con 0x7ff66c0cf430 2024-04-14T08:41:06.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.272+0000 7ff671de0640 1 -- 172.21.15.172:0/803061938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66c068520 con 0x7ff66c0cf430 2024-04-14T08:41:06.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.273+0000 7ff653fff640 1 -- 172.21.15.172:0/803061938 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff65c01a070 con 0x7ff66c0cf430 2024-04-14T08:41:06.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.273+0000 7ff653fff640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff65806c8f0 0x7ff65806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:06.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.274+0000 7ff653fff640 1 -- 172.21.15.172:0/803061938 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7ff65c08bf70 con 0x7ff66c0cf430 2024-04-14T08:41:06.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.274+0000 7ff66affd640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff65806c8f0 0x7ff65806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:06.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.274+0000 7ff66affd640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff65806c8f0 0x7ff65806edb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff6600016e0 tx=0x7ff660009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:06.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.278+0000 7ff653fff640 1 -- 172.21.15.172:0/803061938 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff65c059c50 con 0x7ff66c0cf430 2024-04-14T08:41:06.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:06.492+0000 7ff671de0640 1 -- 172.21.15.172:0/803061938 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff66c135b70 con 0x7ff66c0cf430 2024-04-14T08:41:07.116 INFO:tasks.ceph.mds.b.smithi177.stderr:2024-04-14T08:41:07.116+0000 7fbc777f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:41:07.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.151+0000 7ff653fff640 1 -- 172.21.15.172:0/803061938 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) ==== 122+0+0 (secure 0 0 0) 0x7ff65c05df50 con 0x7ff66c0cf430 2024-04-14T08:41:07.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.154+0000 7ff671de0640 1 -- 172.21.15.172:0/803061938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff65806c8f0 msgr2=0x7ff65806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.154+0000 7ff671de0640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff65806c8f0 0x7ff65806edb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff6600016e0 tx=0x7ff660009040 comp rx=0 tx=0).stop 2024-04-14T08:41:07.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.154+0000 7ff671de0640 1 -- 172.21.15.172:0/803061938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff66c0cf430 msgr2=0x7ff66c15a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.154+0000 7ff671de0640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff66c0cf430 0x7ff66c15a110 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00e9b0 tx=0x7ff65c00ee70 comp rx=0 tx=0).stop 2024-04-14T08:41:07.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.154+0000 7ff671de0640 1 -- 172.21.15.172:0/803061938 shutdown_connections 2024-04-14T08:41:07.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.154+0000 7ff671de0640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff65806c8f0 0x7ff65806edb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.154+0000 7ff671de0640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff66c15bb50 0x7ff66c15e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.154+0000 7ff671de0640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff66c15b770 0x7ff66c141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.154+0000 7ff671de0640 1 --2- 172.21.15.172:0/803061938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff66c0cf430 0x7ff66c15a110 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.154+0000 7ff671de0640 1 -- 172.21.15.172:0/803061938 >> 172.21.15.172:0/803061938 conn(0x7ff66c0b1c60 msgr2=0x7ff66c142a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.155+0000 7ff671de0640 1 -- 172.21.15.172:0/803061938 shutdown_connections 2024-04-14T08:41:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.155+0000 7ff671de0640 1 -- 172.21.15.172:0/803061938 wait complete. 2024-04-14T08:41:07.168 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-14T08:41:07.168 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:41:07.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.273+0000 7effba3ed640 1 Processor -- start 2024-04-14T08:41:07.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.273+0000 7effba3ed640 1 -- start start 2024-04-14T08:41:07.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.273+0000 7effba3ed640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7effb40b1df0 0x7effb40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effba3ed640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb40b21d0 0x7effb415c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effba3ed640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb40b3430 con 0x7effb415c5a0 2024-04-14T08:41:07.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effba3ed640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7effb40b3700 con 0x7effb40b1df0 2024-04-14T08:41:07.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effba3ed640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7effb40b39e0 con 0x7effb40b21d0 2024-04-14T08:41:07.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effb3fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7effb40b1df0 0x7effb40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effb3fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7effb40b1df0 0x7effb40b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36608/0 (socket says 172.21.15.172:36608) 2024-04-14T08:41:07.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effb37fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb40b21d0 0x7effb415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effb3fff640 1 -- 172.21.15.172:0/1050866884 learned_addr learned my addr 172.21.15.172:0/1050866884 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:07.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effb3fff640 1 -- 172.21.15.172:0/1050866884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb40b21d0 msgr2=0x7effb415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:07.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effb3fff640 1 --2- 172.21.15.172:0/1050866884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb40b21d0 0x7effb415c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effb3fff640 1 -- 172.21.15.172:0/1050866884 >> v1:172.21.15.177:6789/0 conn(0x7effb415c5a0 legacy=0x7effb40b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effb2ffd640 1 -- 172.21.15.172:0/1050866884 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1771636455 0 0) 0x7effb40b3430 con 0x7effb415c5a0 2024-04-14T08:41:07.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.274+0000 7effb3fff640 1 -- 172.21.15.172:0/1050866884 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb40b3cc0 con 0x7effb40b1df0 2024-04-14T08:41:07.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.275+0000 7effb3fff640 1 --2- 172.21.15.172:0/1050866884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb40b1df0 0x7effb40b2ef0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7effa400bb10 tx=0x7effa402e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a45cc2cafaa23bd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:07.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.275+0000 7effb2ffd640 1 -- 172.21.15.172:0/1050866884 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa403b070 con 0x7effb40b1df0 2024-04-14T08:41:07.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.275+0000 7effb2ffd640 1 -- 172.21.15.172:0/1050866884 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa4036440 con 0x7effb40b1df0 2024-04-14T08:41:07.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.275+0000 7effb2ffd640 1 -- 172.21.15.172:0/1050866884 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4036810 con 0x7effb40b1df0 2024-04-14T08:41:07.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.275+0000 7effba3ed640 1 -- 172.21.15.172:0/1050866884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb40b1df0 msgr2=0x7effb40b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.275+0000 7effba3ed640 1 --2- 172.21.15.172:0/1050866884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb40b1df0 0x7effb40b2ef0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7effa400bb10 tx=0x7effa402e170 comp rx=0 tx=0).stop 2024-04-14T08:41:07.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.276+0000 7effba3ed640 1 -- 172.21.15.172:0/1050866884 shutdown_connections 2024-04-14T08:41:07.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.276+0000 7effba3ed640 1 --2- 172.21.15.172:0/1050866884 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb40b21d0 0x7effb415c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.276+0000 7effba3ed640 1 --2- 172.21.15.172:0/1050866884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb40b1df0 0x7effb40b2ef0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.276+0000 7effba3ed640 1 -- 172.21.15.172:0/1050866884 >> 172.21.15.172:0/1050866884 conn(0x7effb406ec30 msgr2=0x7effb406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:07.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.276+0000 7effba3ed640 1 -- 172.21.15.172:0/1050866884 shutdown_connections 2024-04-14T08:41:07.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.276+0000 7effba3ed640 1 -- 172.21.15.172:0/1050866884 wait complete. 2024-04-14T08:41:07.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.277+0000 7effba3ed640 1 Processor -- start 2024-04-14T08:41:07.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.277+0000 7effba3ed640 1 -- start start 2024-04-14T08:41:07.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.277+0000 7effba3ed640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb40b1df0 0x7effb413eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effba3ed640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb40b21d0 0x7effb40aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effba3ed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb4157f90 0x7effb41418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effba3ed640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7effb40b4750 con 0x7effb4157f90 2024-04-14T08:41:07.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effba3ed640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7effb415b7f0 con 0x7effb40b21d0 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effba3ed640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7effb40b48d0 con 0x7effb40b1df0 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb898a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb4157f90 0x7effb41418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb898a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb4157f90 0x7effb41418d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36622/0 (socket says 172.21.15.172:36622) 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb898a640 1 -- 172.21.15.172:0/2482928495 learned_addr learned my addr 172.21.15.172:0/2482928495 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb3fff640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb40b1df0 0x7effb413eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb37fe640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb40b21d0 0x7effb40aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb898a640 1 -- 172.21.15.172:0/2482928495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb40b1df0 msgr2=0x7effb413eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb898a640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb40b1df0 0x7effb413eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb898a640 1 -- 172.21.15.172:0/2482928495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb40b21d0 msgr2=0x7effb40aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb898a640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb40b21d0 0x7effb40aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb898a640 1 -- 172.21.15.172:0/2482928495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb413f420 con 0x7effb4157f90 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb898a640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb4157f90 0x7effb41418d0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7effac00c780 tx=0x7effac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb0ff9640 1 -- 172.21.15.172:0/2482928495 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac003b70 con 0x7effb4157f90 2024-04-14T08:41:07.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb0ff9640 1 -- 172.21.15.172:0/2482928495 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac003d30 con 0x7effb4157f90 2024-04-14T08:41:07.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effba3ed640 1 -- 172.21.15.172:0/2482928495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb40ad8e0 con 0x7effb4157f90 2024-04-14T08:41:07.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effb0ff9640 1 -- 172.21.15.172:0/2482928495 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0052b0 con 0x7effb4157f90 2024-04-14T08:41:07.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.278+0000 7effba3ed640 1 -- 172.21.15.172:0/2482928495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb40adea0 con 0x7effb4157f90 2024-04-14T08:41:07.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.279+0000 7effba3ed640 1 -- 172.21.15.172:0/2482928495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb4068520 con 0x7effb4157f90 2024-04-14T08:41:07.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.280+0000 7effb0ff9640 1 -- 172.21.15.172:0/2482928495 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effac005450 con 0x7effb4157f90 2024-04-14T08:41:07.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.280+0000 7effb0ff9640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effa006c8f0 0x7effa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.280+0000 7effb0ff9640 1 -- 172.21.15.172:0/2482928495 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7effac08b9d0 con 0x7effb4157f90 2024-04-14T08:41:07.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.280+0000 7effb3fff640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effa006c8f0 0x7effa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.281+0000 7effb3fff640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effa006c8f0 0x7effa006edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7effa4038000 tx=0x7effa4032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:07.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.285+0000 7effb0ff9640 1 -- 172.21.15.172:0/2482928495 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7effac059620 con 0x7effb4157f90 2024-04-14T08:41:07.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.496+0000 7effba3ed640 1 -- 172.21.15.172:0/2482928495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effb40af1a0 con 0x7effb4157f90 2024-04-14T08:41:07.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.497+0000 7effb0ff9640 1 -- 172.21.15.172:0/2482928495 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5195 (secure 0 0 0) 0x7effac05d920 con 0x7effb4157f90 2024-04-14T08:41:07.498 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:07.498 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/4028911489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4028911489},{"type":"v1","addr":"172.21.15.172:6837","nonce":4028911489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:41:04.492773+0000","modified":"2024-04-14T08:41:06.492980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/2602337717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2602337717},{"type":"v1","addr":"172.21.15.177:6835","nonce":2602337717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8394,"qdb_cluster":[8394]},"id":7}]} 2024-04-14T08:41:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 74 2024-04-14T08:41:07.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.498+0000 7effba3ed640 1 -- 172.21.15.172:0/2482928495 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effa006c8f0 msgr2=0x7effa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.498+0000 7effba3ed640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effa006c8f0 0x7effa006edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7effa4038000 tx=0x7effa4032000 comp rx=0 tx=0).stop 2024-04-14T08:41:07.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.498+0000 7effba3ed640 1 -- 172.21.15.172:0/2482928495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb4157f90 msgr2=0x7effb41418d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.499+0000 7effba3ed640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb4157f90 0x7effb41418d0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7effac00c780 tx=0x7effac00cc40 comp rx=0 tx=0).stop 2024-04-14T08:41:07.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.499+0000 7effba3ed640 1 -- 172.21.15.172:0/2482928495 shutdown_connections 2024-04-14T08:41:07.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.499+0000 7effba3ed640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effa006c8f0 0x7effa006edb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.499+0000 7effba3ed640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effb4157f90 0x7effb41418d0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.499+0000 7effba3ed640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb40b21d0 0x7effb40aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.499+0000 7effba3ed640 1 --2- 172.21.15.172:0/2482928495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb40b1df0 0x7effb413eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.499+0000 7effba3ed640 1 -- 172.21.15.172:0/2482928495 >> 172.21.15.172:0/2482928495 conn(0x7effb406ec30 msgr2=0x7effb41596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:07.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.499+0000 7effba3ed640 1 -- 172.21.15.172:0/2482928495 shutdown_connections 2024-04-14T08:41:07.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.499+0000 7effba3ed640 1 -- 172.21.15.172:0/2482928495 wait complete. 2024-04-14T08:41:07.511 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:41:07.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.611+0000 7f28dba88640 1 Processor -- start 2024-04-14T08:41:07.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.611+0000 7f28dba88640 1 -- start start 2024-04-14T08:41:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28dba88640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f28d415e390 0x7f28d40d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28dba88640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f28d415f3a0 0x7f28d40b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28dba88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28d40b4260 0x7f28d4161f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28dba88640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f28d40b46d0 con 0x7f28d415f3a0 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28dba88640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f28d40d0080 con 0x7f28d415e390 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28dba88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f28d415e770 con 0x7f28d40b4260 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28d9023640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f28d415f3a0 0x7f28d40b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28d9023640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f28d415f3a0 0x7f28d40b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36636/0 (socket says 172.21.15.172:36636) 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28d9023640 1 -- 172.21.15.172:0/980987177 learned_addr learned my addr 172.21.15.172:0/980987177 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28da025640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28d40b4260 0x7f28d4161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28d9824640 1 --2- 172.21.15.172:0/980987177 >> v2:172.21.15.177:3300/0 conn(0x7f28d415e390 0x7f28d40d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28d9023640 1 -- 172.21.15.172:0/980987177 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28d40b4260 msgr2=0x7f28d4161f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28d9023640 1 --2- 172.21.15.172:0/980987177 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28d40b4260 0x7f28d4161f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28d9023640 1 -- 172.21.15.172:0/980987177 >> v2:172.21.15.177:3300/0 conn(0x7f28d415e390 msgr2=0x7f28d40d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28d9023640 1 --2- 172.21.15.172:0/980987177 >> v2:172.21.15.177:3300/0 conn(0x7f28d415e390 0x7f28d40d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.612+0000 7f28d9023640 1 -- 172.21.15.172:0/980987177 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d406dba0 con 0x7f28d415f3a0 2024-04-14T08:41:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.613+0000 7f28d9023640 1 --2- 172.21.15.172:0/980987177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28d415f3a0 0x7f28d40b3d20 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f28d4070690 tx=0x7f28c402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8c6dc148c27dca server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.613+0000 7f28d8822640 1 -- 172.21.15.172:0/980987177 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c402e070 con 0x7f28d415f3a0 2024-04-14T08:41:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.613+0000 7f28d8822640 1 -- 172.21.15.172:0/980987177 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c4037070 con 0x7f28d415f3a0 2024-04-14T08:41:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.613+0000 7f28d8822640 1 -- 172.21.15.172:0/980987177 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4031a00 con 0x7f28d415f3a0 2024-04-14T08:41:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.614+0000 7f28dba88640 1 -- 172.21.15.172:0/980987177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28d415f3a0 msgr2=0x7f28d40b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.614+0000 7f28dba88640 1 --2- 172.21.15.172:0/980987177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28d415f3a0 0x7f28d40b3d20 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f28d4070690 tx=0x7f28c402b880 comp rx=0 tx=0).stop 2024-04-14T08:41:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.614+0000 7f28dba88640 1 -- 172.21.15.172:0/980987177 shutdown_connections 2024-04-14T08:41:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.614+0000 7f28dba88640 1 --2- 172.21.15.172:0/980987177 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28d40b4260 0x7f28d4161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.614+0000 7f28dba88640 1 --2- 172.21.15.172:0/980987177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28d415f3a0 0x7f28d40b3d20 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.614+0000 7f28dba88640 1 --2- 172.21.15.172:0/980987177 >> v2:172.21.15.177:3300/0 conn(0x7f28d415e390 0x7f28d40d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.614+0000 7f28dba88640 1 -- 172.21.15.172:0/980987177 >> 172.21.15.172:0/980987177 conn(0x7f28d406f820 msgr2=0x7f28d406fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.614+0000 7f28dba88640 1 -- 172.21.15.172:0/980987177 shutdown_connections 2024-04-14T08:41:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.614+0000 7f28dba88640 1 -- 172.21.15.172:0/980987177 wait complete. 2024-04-14T08:41:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.615+0000 7f28dba88640 1 Processor -- start 2024-04-14T08:41:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.615+0000 7f28dba88640 1 -- start start 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28dba88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28d40b4260 0x7f28d4158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28dba88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28d415e390 0x7f28d40aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28dba88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28d415f3a0 0x7f28d4158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28dba88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f28d40d0510 con 0x7f28d40b4260 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28dba88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f28d415f800 con 0x7f28d415e390 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28dba88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f28d4162450 con 0x7f28d415f3a0 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28d9824640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28d40b4260 0x7f28d4158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28d9824640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28d40b4260 0x7f28d4158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36640/0 (socket says 172.21.15.172:36640) 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28d9824640 1 -- 172.21.15.172:0/3081790320 learned_addr learned my addr 172.21.15.172:0/3081790320 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28d9023640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28d415e390 0x7f28d40aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28da025640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28d415f3a0 0x7f28d4158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28d9824640 1 -- 172.21.15.172:0/3081790320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28d415f3a0 msgr2=0x7f28d4158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28d9824640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28d415f3a0 0x7f28d4158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28d9824640 1 -- 172.21.15.172:0/3081790320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28d415e390 msgr2=0x7f28d40aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28d9824640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28d415e390 0x7f28d40aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28d9824640 1 -- 172.21.15.172:0/3081790320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d415a750 con 0x7f28d40b4260 2024-04-14T08:41:07.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28d9824640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28d40b4260 0x7f28d4158120 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f28d000e9b0 tx=0x7f28d000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:07.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28ca7fc640 1 -- 172.21.15.172:0/3081790320 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0017070 con 0x7f28d40b4260 2024-04-14T08:41:07.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28ca7fc640 1 -- 172.21.15.172:0/3081790320 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d000c9a0 con 0x7f28d40b4260 2024-04-14T08:41:07.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28dba88640 1 -- 172.21.15.172:0/3081790320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d415aa30 con 0x7f28d40b4260 2024-04-14T08:41:07.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28ca7fc640 1 -- 172.21.15.172:0/3081790320 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d000cd90 con 0x7f28d40b4260 2024-04-14T08:41:07.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.616+0000 7f28dba88640 1 -- 172.21.15.172:0/3081790320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d415aff0 con 0x7f28d40b4260 2024-04-14T08:41:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.617+0000 7f28dba88640 1 -- 172.21.15.172:0/3081790320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d4068520 con 0x7f28d40b4260 2024-04-14T08:41:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.618+0000 7f28ca7fc640 1 -- 172.21.15.172:0/3081790320 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28d001a070 con 0x7f28d40b4260 2024-04-14T08:41:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.618+0000 7f28ca7fc640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f28b806c8f0 0x7f28b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.618+0000 7f28ca7fc640 1 -- 172.21.15.172:0/3081790320 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f28d008bee0 con 0x7f28d40b4260 2024-04-14T08:41:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.618+0000 7f28d9023640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f28b806c8f0 0x7f28b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.619+0000 7f28d9023640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f28b806c8f0 0x7f28b806edb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f28c4036000 tx=0x7f28c403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.623+0000 7f28ca7fc640 1 -- 172.21.15.172:0/3081790320 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f28d0059bc0 con 0x7f28d40b4260 2024-04-14T08:41:07.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.797+0000 7f28dba88640 1 -- 172.21.15.172:0/3081790320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28d4071840 con 0x7f28d40b4260 2024-04-14T08:41:07.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.798+0000 7f28ca7fc640 1 -- 172.21.15.172:0/3081790320 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16936 (secure 0 0 0) 0x7f28d005dec0 con 0x7f28d40b4260 2024-04-14T08:41:07.800 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:07.800 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":101,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:41:06.143425+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:41:02.725765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-14T08:41:03.442157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:41:07.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.800+0000 7f28dba88640 1 -- 172.21.15.172:0/3081790320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f28b806c8f0 msgr2=0x7f28b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.800+0000 7f28dba88640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f28b806c8f0 0x7f28b806edb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f28c4036000 tx=0x7f28c403d040 comp rx=0 tx=0).stop 2024-04-14T08:41:07.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.800+0000 7f28dba88640 1 -- 172.21.15.172:0/3081790320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28d40b4260 msgr2=0x7f28d4158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.800+0000 7f28dba88640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28d40b4260 0x7f28d4158120 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f28d000e9b0 tx=0x7f28d000ee70 comp rx=0 tx=0).stop 2024-04-14T08:41:07.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.800+0000 7f28dba88640 1 -- 172.21.15.172:0/3081790320 shutdown_connections 2024-04-14T08:41:07.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.800+0000 7f28dba88640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f28b806c8f0 0x7f28b806edb0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.800+0000 7f28dba88640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28d415f3a0 0x7f28d4158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.800+0000 7f28dba88640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28d415e390 0x7f28d40aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.800+0000 7f28dba88640 1 --2- 172.21.15.172:0/3081790320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28d40b4260 0x7f28d4158120 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.800+0000 7f28dba88640 1 -- 172.21.15.172:0/3081790320 >> 172.21.15.172:0/3081790320 conn(0x7f28d406f820 msgr2=0x7f28d41596c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:07.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.800+0000 7f28dba88640 1 -- 172.21.15.172:0/3081790320 shutdown_connections 2024-04-14T08:41:07.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.800+0000 7f28dba88640 1 -- 172.21.15.172:0/3081790320 wait complete. 2024-04-14T08:41:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-14T08:41:07.813 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:41:07.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.912+0000 7f2bc3c80640 1 Processor -- start 2024-04-14T08:41:07.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.912+0000 7f2bc3c80640 1 -- start start 2024-04-14T08:41:07.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.913+0000 7f2bc3c80640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2bbc0b1df0 0x7f2bbc0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.913+0000 7f2bc3c80640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bbc0b21d0 0x7f2bbc15c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.913+0000 7f2bc3c80640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2bbc15c5a0 0x7f2bbc0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.913+0000 7f2bc3c80640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2bbc0b3430 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.913+0000 7f2bc3c80640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2bbc0b3700 con 0x7f2bbc15c5a0 2024-04-14T08:41:07.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.913+0000 7f2bc3c80640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2bbc0b39e0 con 0x7f2bbc0b21d0 2024-04-14T08:41:07.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.913+0000 7f2bc1a1c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2bbc0b1df0 0x7f2bbc0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.913+0000 7f2bc121b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bbc0b21d0 0x7f2bbc15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.913+0000 7f2bc221d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2bbc15c5a0 0x7f2bbc0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.913+0000 7f2bc1a1c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2bbc0b1df0 0x7f2bbc0b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36646/0 (socket says 172.21.15.172:36646) 2024-04-14T08:41:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.913+0000 7f2bc1a1c640 1 -- 172.21.15.172:0/1408347918 learned_addr learned my addr 172.21.15.172:0/1408347918 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.913+0000 7f2bc121b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bbc0b21d0 0x7f2bbc15c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:41918/0 (socket says 172.21.15.172:41918) 2024-04-14T08:41:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.914+0000 7f2bc1a1c640 1 -- 172.21.15.172:0/1408347918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bbc0b21d0 msgr2=0x7f2bbc15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.914+0000 7f2bc1a1c640 1 --2- 172.21.15.172:0/1408347918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bbc0b21d0 0x7f2bbc15c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.914+0000 7f2bc1a1c640 1 -- 172.21.15.172:0/1408347918 >> v2:172.21.15.177:3300/0 conn(0x7f2bbc15c5a0 msgr2=0x7f2bbc0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.914+0000 7f2bc1a1c640 1 --2- 172.21.15.172:0/1408347918 >> v2:172.21.15.177:3300/0 conn(0x7f2bbc15c5a0 0x7f2bbc0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.914+0000 7f2bc1a1c640 1 -- 172.21.15.172:0/1408347918 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc0b3cc0 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.914+0000 7f2bc121b640 1 --2- 172.21.15.172:0/1408347918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bbc0b21d0 0x7f2bbc15c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.914+0000 7f2bc221d640 1 --2- 172.21.15.172:0/1408347918 >> v2:172.21.15.177:3300/0 conn(0x7f2bbc15c5a0 0x7f2bbc0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.914+0000 7f2bc1a1c640 1 --2- 172.21.15.172:0/1408347918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bbc0b1df0 0x7f2bbc0b2ef0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0b3910 tx=0x7f2bb802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b637ac6d297bd75 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.914+0000 7f2bc0a1a640 1 -- 172.21.15.172:0/1408347918 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8035020 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.914+0000 7f2bc0a1a640 1 -- 172.21.15.172:0/1408347918 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb8031640 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.914+0000 7f2bc0a1a640 1 -- 172.21.15.172:0/1408347918 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb802d040 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.915+0000 7f2bc3c80640 1 -- 172.21.15.172:0/1408347918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bbc0b1df0 msgr2=0x7f2bbc0b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.915+0000 7f2bc3c80640 1 --2- 172.21.15.172:0/1408347918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bbc0b1df0 0x7f2bbc0b2ef0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0b3910 tx=0x7f2bb802b8f0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.915+0000 7f2bc3c80640 1 -- 172.21.15.172:0/1408347918 shutdown_connections 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.915+0000 7f2bc3c80640 1 --2- 172.21.15.172:0/1408347918 >> v2:172.21.15.177:3300/0 conn(0x7f2bbc15c5a0 0x7f2bbc0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.915+0000 7f2bc3c80640 1 --2- 172.21.15.172:0/1408347918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bbc0b21d0 0x7f2bbc15c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.915+0000 7f2bc3c80640 1 --2- 172.21.15.172:0/1408347918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bbc0b1df0 0x7f2bbc0b2ef0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.915+0000 7f2bc3c80640 1 -- 172.21.15.172:0/1408347918 >> 172.21.15.172:0/1408347918 conn(0x7f2bbc06ec30 msgr2=0x7f2bbc06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.915+0000 7f2bc3c80640 1 -- 172.21.15.172:0/1408347918 shutdown_connections 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.915+0000 7f2bc3c80640 1 -- 172.21.15.172:0/1408347918 wait complete. 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.916+0000 7f2bc3c80640 1 Processor -- start 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.916+0000 7f2bc3c80640 1 -- start start 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc3c80640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bbc0b1df0 0x7f2bbc157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc3c80640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bbc0b21d0 0x7f2bbc137f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc3c80640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bbc13a390 0x7f2bbc13b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc3c80640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc0b4580 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc3c80640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc15b7f0 con 0x7f2bbc13a390 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc3c80640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2bbc0b4700 con 0x7f2bbc0b21d0 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc1a1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bbc0b1df0 0x7f2bbc157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc1a1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bbc0b1df0 0x7f2bbc157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36662/0 (socket says 172.21.15.172:36662) 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc1a1c640 1 -- 172.21.15.172:0/3837641360 learned_addr learned my addr 172.21.15.172:0/3837641360 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc121b640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bbc0b21d0 0x7f2bbc137f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc221d640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bbc13a390 0x7f2bbc13b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc1a1c640 1 -- 172.21.15.172:0/3837641360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bbc0b21d0 msgr2=0x7f2bbc137f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc1a1c640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bbc0b21d0 0x7f2bbc137f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc1a1c640 1 -- 172.21.15.172:0/3837641360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bbc13a390 msgr2=0x7f2bbc13b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc1a1c640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bbc13a390 0x7f2bbc13b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.917+0000 7f2bc1a1c640 1 -- 172.21.15.172:0/3837641360 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc1384b0 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.918+0000 7f2bc1a1c640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bbc0b1df0 0x7f2bbc157dd0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8000c00 tx=0x7f2bb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.918+0000 7f2bb27fc640 1 -- 172.21.15.172:0/3837641360 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb803aa60 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.918+0000 7f2bb27fc640 1 -- 172.21.15.172:0/3837641360 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb803ac20 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.918+0000 7f2bc3c80640 1 -- 172.21.15.172:0/3837641360 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bbc13a8e0 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.918+0000 7f2bb27fc640 1 -- 172.21.15.172:0/3837641360 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80443f0 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.918+0000 7f2bc3c80640 1 -- 172.21.15.172:0/3837641360 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bbc13aea0 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.919+0000 7f2bc3c80640 1 -- 172.21.15.172:0/3837641360 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bbc0b1950 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.919+0000 7f2bb27fc640 1 -- 172.21.15.172:0/3837641360 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bb804b060 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.919+0000 7f2bb27fc640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ba006c8f0 0x7f2ba006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:07.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.920+0000 7f2bb27fc640 1 -- 172.21.15.172:0/3837641360 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f2bb80ad760 con 0x7f2bbc0b1df0 2024-04-14T08:41:07.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.920+0000 7f2bc121b640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ba006c8f0 0x7f2ba006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:07.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.920+0000 7f2bc121b640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ba006c8f0 0x7f2ba006edb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009560 tx=0x7f2bac009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:07.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:07.924+0000 7f2bb27fc640 1 -- 172.21.15.172:0/3837641360 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2bbc0b1950 con 0x7f2bbc0b1df0 2024-04-14T08:41:08.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.095+0000 7f2bc3c80640 1 -- 172.21.15.172:0/3837641360 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2bbc068520 con 0x7f2ba006c8f0 2024-04-14T08:41:08.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.101+0000 7f2bb27fc640 1 -- 172.21.15.172:0/3837641360 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252194 (secure 0 0 0) 0x7f2bbc068520 con 0x7f2ba006c8f0 2024-04-14T08:41:08.105 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:08.107 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:41:08.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.104+0000 7f2bc3c80640 1 -- 172.21.15.172:0/3837641360 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ba006c8f0 msgr2=0x7f2ba006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.104+0000 7f2bc3c80640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ba006c8f0 0x7f2ba006edb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009560 tx=0x7f2bac009190 comp rx=0 tx=0).stop 2024-04-14T08:41:08.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.104+0000 7f2bc3c80640 1 -- 172.21.15.172:0/3837641360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bbc0b1df0 msgr2=0x7f2bbc157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.104+0000 7f2bc3c80640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bbc0b1df0 0x7f2bbc157dd0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8000c00 tx=0x7f2bb80046c0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.104+0000 7f2bc3c80640 1 -- 172.21.15.172:0/3837641360 shutdown_connections 2024-04-14T08:41:08.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.104+0000 7f2bc3c80640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2ba006c8f0 0x7f2ba006edb0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.104+0000 7f2bc3c80640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bbc13a390 0x7f2bbc13b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.104+0000 7f2bc3c80640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2bbc0b21d0 0x7f2bbc137f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.104+0000 7f2bc3c80640 1 --2- 172.21.15.172:0/3837641360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2bbc0b1df0 0x7f2bbc157dd0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.104+0000 7f2bc3c80640 1 -- 172.21.15.172:0/3837641360 >> 172.21.15.172:0/3837641360 conn(0x7f2bbc06ec30 msgr2=0x7f2bbc1595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:08.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.104+0000 7f2bc3c80640 1 -- 172.21.15.172:0/3837641360 shutdown_connections 2024-04-14T08:41:08.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.104+0000 7f2bc3c80640 1 -- 172.21.15.172:0/3837641360 wait complete. 2024-04-14T08:41:08.116 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-04-14T08:41:07.217726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260224,"kb_used_data":10780,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714496,"statfs":{"total":773094113280,"available":772827643904,"internally_reserved":0,"allocated":11038720,"data_stored":5722431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.089958"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162606+0000","last_change":"2024-04-14T08:41:05.156855+0000","last_active":"2024-04-14T08:41:05.162606+0000","last_peered":"2024-04-14T08:41:05.162606+0000","last_clean":"2024-04-14T08:41:05.162606+0000","last_became_active":"2024-04-14T08:41:05.156547+0000","last_became_peered":"2024-04-14T08:41:05.156547+0000","last_unstale":"2024-04-14T08:41:05.162606+0000","last_undegraded":"2024-04-14T08:41:05.162606+0000","last_fullsized":"2024-04-14T08:41:05.162606+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:20:39.314348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161756+0000","last_change":"2024-04-14T08:41:05.158746+0000","last_active":"2024-04-14T08:41:05.161756+0000","last_peered":"2024-04-14T08:41:05.161756+0000","last_clean":"2024-04-14T08:41:05.161756+0000","last_became_active":"2024-04-14T08:41:05.158296+0000","last_became_peered":"2024-04-14T08:41:05.158296+0000","last_unstale":"2024-04-14T08:41:05.161756+0000","last_undegraded":"2024-04-14T08:41:05.161756+0000","last_fullsized":"2024-04-14T08:41:05.161756+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:03:28.321597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166623+0000","last_change":"2024-04-14T08:41:05.163762+0000","last_active":"2024-04-14T08:41:05.166623+0000","last_peered":"2024-04-14T08:41:05.166623+0000","last_clean":"2024-04-14T08:41:05.166623+0000","last_became_active":"2024-04-14T08:41:05.163476+0000","last_became_peered":"2024-04-14T08:41:05.163476+0000","last_unstale":"2024-04-14T08:41:05.166623+0000","last_undegraded":"2024-04-14T08:41:05.166623+0000","last_fullsized":"2024-04-14T08:41:05.166623+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:41:09.337397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.165848+0000","last_change":"2024-04-14T08:41:05.163054+0000","last_active":"2024-04-14T08:41:05.165848+0000","last_peered":"2024-04-14T08:41:05.165848+0000","last_clean":"2024-04-14T08:41:05.165848+0000","last_became_active":"2024-04-14T08:41:05.162804+0000","last_became_peered":"2024-04-14T08:41:05.162804+0000","last_unstale":"2024-04-14T08:41:05.165848+0000","last_undegraded":"2024-04-14T08:41:05.165848+0000","last_fullsized":"2024-04-14T08:41:05.165848+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:36:36.291914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167319+0000","last_change":"2024-04-14T08:41:05.160856+0000","last_active":"2024-04-14T08:41:05.167319+0000","last_peered":"2024-04-14T08:41:05.167319+0000","last_clean":"2024-04-14T08:41:05.167319+0000","last_became_active":"2024-04-14T08:41:05.160576+0000","last_became_peered":"2024-04-14T08:41:05.160576+0000","last_unstale":"2024-04-14T08:41:05.167319+0000","last_undegraded":"2024-04-14T08:41:05.167319+0000","last_fullsized":"2024-04-14T08:41:05.167319+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:16:26.973756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170740+0000","last_change":"2024-04-14T08:41:05.167420+0000","last_active":"2024-04-14T08:41:05.170740+0000","last_peered":"2024-04-14T08:41:05.170740+0000","last_clean":"2024-04-14T08:41:05.170740+0000","last_became_active":"2024-04-14T08:41:05.167143+0000","last_became_peered":"2024-04-14T08:41:05.167143+0000","last_unstale":"2024-04-14T08:41:05.170740+0000","last_undegraded":"2024-04-14T08:41:05.170740+0000","last_fullsized":"2024-04-14T08:41:05.170740+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:27:59.897940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170729+0000","last_change":"2024-04-14T08:41:05.166977+0000","last_active":"2024-04-14T08:41:05.170729+0000","last_peered":"2024-04-14T08:41:05.170729+0000","last_clean":"2024-04-14T08:41:05.170729+0000","last_became_active":"2024-04-14T08:41:05.166674+0000","last_became_peered":"2024-04-14T08:41:05.166674+0000","last_unstale":"2024-04-14T08:41:05.170729+0000","last_undegraded":"2024-04-14T08:41:05.170729+0000","last_fullsized":"2024-04-14T08:41:05.170729+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:29:54.737024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.165700+0000","last_change":"2024-04-14T08:41:05.154902+0000","last_active":"2024-04-14T08:41:05.165700+0000","last_peered":"2024-04-14T08:41:05.165700+0000","last_clean":"2024-04-14T08:41:05.165700+0000","last_became_active":"2024-04-14T08:41:05.154621+0000","last_became_peered":"2024-04-14T08:41:05.154621+0000","last_unstale":"2024-04-14T08:41:05.165700+0000","last_undegraded":"2024-04-14T08:41:05.165700+0000","last_fullsized":"2024-04-14T08:41:05.165700+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:24:46.921747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162270+0000","last_change":"2024-04-14T08:41:05.157665+0000","last_active":"2024-04-14T08:41:05.162270+0000","last_peered":"2024-04-14T08:41:05.162270+0000","last_clean":"2024-04-14T08:41:05.162270+0000","last_became_active":"2024-04-14T08:41:05.157391+0000","last_became_peered":"2024-04-14T08:41:05.157391+0000","last_unstale":"2024-04-14T08:41:05.162270+0000","last_undegraded":"2024-04-14T08:41:05.162270+0000","last_fullsized":"2024-04-14T08:41:05.162270+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:13:53.752730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167422+0000","last_change":"2024-04-14T08:41:05.160560+0000","last_active":"2024-04-14T08:41:05.167422+0000","last_peered":"2024-04-14T08:41:05.167422+0000","last_clean":"2024-04-14T08:41:05.167422+0000","last_became_active":"2024-04-14T08:41:05.159424+0000","last_became_peered":"2024-04-14T08:41:05.159424+0000","last_unstale":"2024-04-14T08:41:05.167422+0000","last_undegraded":"2024-04-14T08:41:05.167422+0000","last_fullsized":"2024-04-14T08:41:05.167422+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:34:09.480206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167698+0000","last_change":"2024-04-14T08:41:05.163318+0000","last_active":"2024-04-14T08:41:05.167698+0000","last_peered":"2024-04-14T08:41:05.167698+0000","last_clean":"2024-04-14T08:41:05.167698+0000","last_became_active":"2024-04-14T08:41:05.162996+0000","last_became_peered":"2024-04-14T08:41:05.162996+0000","last_unstale":"2024-04-14T08:41:05.167698+0000","last_undegraded":"2024-04-14T08:41:05.167698+0000","last_fullsized":"2024-04-14T08:41:05.167698+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:05:58.415982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161956+0000","last_change":"2024-04-14T08:41:05.157945+0000","last_active":"2024-04-14T08:41:05.161956+0000","last_peered":"2024-04-14T08:41:05.161956+0000","last_clean":"2024-04-14T08:41:05.161956+0000","last_became_active":"2024-04-14T08:41:05.157669+0000","last_became_peered":"2024-04-14T08:41:05.157669+0000","last_unstale":"2024-04-14T08:41:05.161956+0000","last_undegraded":"2024-04-14T08:41:05.161956+0000","last_fullsized":"2024-04-14T08:41:05.161956+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:29:54.737024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164760+0000","last_change":"2024-04-14T08:41:05.161069+0000","last_active":"2024-04-14T08:41:05.164760+0000","last_peered":"2024-04-14T08:41:05.164760+0000","last_clean":"2024-04-14T08:41:05.164760+0000","last_became_active":"2024-04-14T08:41:05.160809+0000","last_became_peered":"2024-04-14T08:41:05.160809+0000","last_unstale":"2024-04-14T08:41:05.164760+0000","last_undegraded":"2024-04-14T08:41:05.164760+0000","last_fullsized":"2024-04-14T08:41:05.164760+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:29:54.737024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.150585+0000","last_change":"2024-04-14T08:41:05.150968+0000","last_active":"2024-04-14T08:41:06.150585+0000","last_peered":"2024-04-14T08:41:06.150585+0000","last_clean":"2024-04-14T08:41:06.150585+0000","last_became_active":"2024-04-14T08:41:05.150676+0000","last_became_peered":"2024-04-14T08:41:05.150676+0000","last_unstale":"2024-04-14T08:41:06.150585+0000","last_undegraded":"2024-04-14T08:41:06.150585+0000","last_fullsized":"2024-04-14T08:41:06.150585+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:31:00.277915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.150459+0000","last_change":"2024-04-14T08:41:05.161651+0000","last_active":"2024-04-14T08:41:06.150459+0000","last_peered":"2024-04-14T08:41:06.150459+0000","last_clean":"2024-04-14T08:41:06.150459+0000","last_became_active":"2024-04-14T08:41:05.161343+0000","last_became_peered":"2024-04-14T08:41:05.161343+0000","last_unstale":"2024-04-14T08:41:06.150459+0000","last_undegraded":"2024-04-14T08:41:06.150459+0000","last_fullsized":"2024-04-14T08:41:06.150459+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:16:41.150351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.150349+0000","last_change":"2024-04-14T08:41:05.151419+0000","last_active":"2024-04-14T08:41:06.150349+0000","last_peered":"2024-04-14T08:41:06.150349+0000","last_clean":"2024-04-14T08:41:06.150349+0000","last_became_active":"2024-04-14T08:41:05.151092+0000","last_became_peered":"2024-04-14T08:41:05.151092+0000","last_unstale":"2024-04-14T08:41:06.150349+0000","last_undegraded":"2024-04-14T08:41:06.150349+0000","last_fullsized":"2024-04-14T08:41:06.150349+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:40:18.013039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166494+0000","last_change":"2024-04-14T08:41:05.163930+0000","last_active":"2024-04-14T08:41:05.166494+0000","last_peered":"2024-04-14T08:41:05.166494+0000","last_clean":"2024-04-14T08:41:05.166494+0000","last_became_active":"2024-04-14T08:41:05.163673+0000","last_became_peered":"2024-04-14T08:41:05.163673+0000","last_unstale":"2024-04-14T08:41:05.166494+0000","last_undegraded":"2024-04-14T08:41:05.166494+0000","last_fullsized":"2024-04-14T08:41:05.166494+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:19:53.201897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170729+0000","last_change":"2024-04-14T08:41:05.166883+0000","last_active":"2024-04-14T08:41:05.170729+0000","last_peered":"2024-04-14T08:41:05.170729+0000","last_clean":"2024-04-14T08:41:05.170729+0000","last_became_active":"2024-04-14T08:41:05.166631+0000","last_became_peered":"2024-04-14T08:41:05.166631+0000","last_unstale":"2024-04-14T08:41:05.170729+0000","last_undegraded":"2024-04-14T08:41:05.170729+0000","last_fullsized":"2024-04-14T08:41:05.170729+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:23:53.194822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162783+0000","last_change":"2024-04-14T08:41:05.156842+0000","last_active":"2024-04-14T08:41:05.162783+0000","last_peered":"2024-04-14T08:41:05.162783+0000","last_clean":"2024-04-14T08:41:05.162783+0000","last_became_active":"2024-04-14T08:41:05.156580+0000","last_became_peered":"2024-04-14T08:41:05.156580+0000","last_unstale":"2024-04-14T08:41:05.162783+0000","last_undegraded":"2024-04-14T08:41:05.162783+0000","last_fullsized":"2024-04-14T08:41:05.162783+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:20.273023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161974+0000","last_change":"2024-04-14T08:41:05.158026+0000","last_active":"2024-04-14T08:41:05.161974+0000","last_peered":"2024-04-14T08:41:05.161974+0000","last_clean":"2024-04-14T08:41:05.161974+0000","last_became_active":"2024-04-14T08:41:05.157721+0000","last_became_peered":"2024-04-14T08:41:05.157721+0000","last_unstale":"2024-04-14T08:41:05.161974+0000","last_undegraded":"2024-04-14T08:41:05.161974+0000","last_fullsized":"2024-04-14T08:41:05.161974+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:27:59.897940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166081+0000","last_change":"2024-04-14T08:41:05.159179+0000","last_active":"2024-04-14T08:41:05.166081+0000","last_peered":"2024-04-14T08:41:05.166081+0000","last_clean":"2024-04-14T08:41:05.166081+0000","last_became_active":"2024-04-14T08:41:05.158920+0000","last_became_peered":"2024-04-14T08:41:05.158920+0000","last_unstale":"2024-04-14T08:41:05.166081+0000","last_undegraded":"2024-04-14T08:41:05.166081+0000","last_fullsized":"2024-04-14T08:41:05.166081+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:53.061286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167273+0000","last_change":"2024-04-14T08:41:05.162547+0000","last_active":"2024-04-14T08:41:05.167273+0000","last_peered":"2024-04-14T08:41:05.167273+0000","last_clean":"2024-04-14T08:41:05.167273+0000","last_became_active":"2024-04-14T08:41:05.162298+0000","last_became_peered":"2024-04-14T08:41:05.162298+0000","last_unstale":"2024-04-14T08:41:05.167273+0000","last_undegraded":"2024-04-14T08:41:05.167273+0000","last_fullsized":"2024-04-14T08:41:05.167273+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:13:53.752730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166436+0000","last_change":"2024-04-14T08:41:05.162802+0000","last_active":"2024-04-14T08:41:05.166436+0000","last_peered":"2024-04-14T08:41:05.166436+0000","last_clean":"2024-04-14T08:41:05.166436+0000","last_became_active":"2024-04-14T08:41:05.162529+0000","last_became_peered":"2024-04-14T08:41:05.162529+0000","last_unstale":"2024-04-14T08:41:05.166436+0000","last_undegraded":"2024-04-14T08:41:05.166436+0000","last_fullsized":"2024-04-14T08:41:05.166436+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:43:42.680257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170935+0000","last_change":"2024-04-14T08:41:05.165557+0000","last_active":"2024-04-14T08:41:05.170935+0000","last_peered":"2024-04-14T08:41:05.170935+0000","last_clean":"2024-04-14T08:41:05.170935+0000","last_became_active":"2024-04-14T08:41:05.165285+0000","last_became_peered":"2024-04-14T08:41:05.165285+0000","last_unstale":"2024-04-14T08:41:05.170935+0000","last_undegraded":"2024-04-14T08:41:05.170935+0000","last_fullsized":"2024-04-14T08:41:05.170935+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:05:58.415982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170597+0000","last_change":"2024-04-14T08:41:05.166693+0000","last_active":"2024-04-14T08:41:05.170597+0000","last_peered":"2024-04-14T08:41:05.170597+0000","last_clean":"2024-04-14T08:41:05.170597+0000","last_became_active":"2024-04-14T08:41:05.165164+0000","last_became_peered":"2024-04-14T08:41:05.165164+0000","last_unstale":"2024-04-14T08:41:05.170597+0000","last_undegraded":"2024-04-14T08:41:05.170597+0000","last_fullsized":"2024-04-14T08:41:05.170597+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:41:29.147856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164295+0000","last_change":"2024-04-14T08:41:05.160757+0000","last_active":"2024-04-14T08:41:05.164295+0000","last_peered":"2024-04-14T08:41:05.164295+0000","last_clean":"2024-04-14T08:41:05.164295+0000","last_became_active":"2024-04-14T08:41:05.160438+0000","last_became_peered":"2024-04-14T08:41:05.160438+0000","last_unstale":"2024-04-14T08:41:05.164295+0000","last_undegraded":"2024-04-14T08:41:05.164295+0000","last_fullsized":"2024-04-14T08:41:05.164295+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:27:32.491311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167721+0000","last_change":"2024-04-14T08:41:05.163760+0000","last_active":"2024-04-14T08:41:05.167721+0000","last_peered":"2024-04-14T08:41:05.167721+0000","last_clean":"2024-04-14T08:41:05.167721+0000","last_became_active":"2024-04-14T08:41:05.163154+0000","last_became_peered":"2024-04-14T08:41:05.163154+0000","last_unstale":"2024-04-14T08:41:05.167721+0000","last_undegraded":"2024-04-14T08:41:05.167721+0000","last_fullsized":"2024-04-14T08:41:05.167721+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:02:07.977713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162633+0000","last_change":"2024-04-14T08:41:05.158599+0000","last_active":"2024-04-14T08:41:05.162633+0000","last_peered":"2024-04-14T08:41:05.162633+0000","last_clean":"2024-04-14T08:41:05.162633+0000","last_became_active":"2024-04-14T08:41:05.158282+0000","last_became_peered":"2024-04-14T08:41:05.158282+0000","last_unstale":"2024-04-14T08:41:05.162633+0000","last_undegraded":"2024-04-14T08:41:05.162633+0000","last_fullsized":"2024-04-14T08:41:05.162633+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:05:58.415982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.163801+0000","last_change":"2024-04-14T08:41:05.152092+0000","last_active":"2024-04-14T08:41:05.163801+0000","last_peered":"2024-04-14T08:41:05.163801+0000","last_clean":"2024-04-14T08:41:05.163801+0000","last_became_active":"2024-04-14T08:41:05.151835+0000","last_became_peered":"2024-04-14T08:41:05.151835+0000","last_unstale":"2024-04-14T08:41:05.163801+0000","last_undegraded":"2024-04-14T08:41:05.163801+0000","last_fullsized":"2024-04-14T08:41:05.163801+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:30:25.299745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162000+0000","last_change":"2024-04-14T08:41:05.158723+0000","last_active":"2024-04-14T08:41:05.162000+0000","last_peered":"2024-04-14T08:41:05.162000+0000","last_clean":"2024-04-14T08:41:05.162000+0000","last_became_active":"2024-04-14T08:41:05.158454+0000","last_became_peered":"2024-04-14T08:41:05.158454+0000","last_unstale":"2024-04-14T08:41:05.162000+0000","last_undegraded":"2024-04-14T08:41:05.162000+0000","last_fullsized":"2024-04-14T08:41:05.162000+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:36:36.291914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170666+0000","last_change":"2024-04-14T08:41:05.165822+0000","last_active":"2024-04-14T08:41:05.170666+0000","last_peered":"2024-04-14T08:41:05.170666+0000","last_clean":"2024-04-14T08:41:05.170666+0000","last_became_active":"2024-04-14T08:41:05.165232+0000","last_became_peered":"2024-04-14T08:41:05.165232+0000","last_unstale":"2024-04-14T08:41:05.170666+0000","last_undegraded":"2024-04-14T08:41:05.170666+0000","last_fullsized":"2024-04-14T08:41:05.170666+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:47:10.299310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166337+0000","last_change":"2024-04-14T08:41:05.163181+0000","last_active":"2024-04-14T08:41:05.166337+0000","last_peered":"2024-04-14T08:41:05.166337+0000","last_clean":"2024-04-14T08:41:05.166337+0000","last_became_active":"2024-04-14T08:41:05.162908+0000","last_became_peered":"2024-04-14T08:41:05.162908+0000","last_unstale":"2024-04-14T08:41:05.166337+0000","last_undegraded":"2024-04-14T08:41:05.166337+0000","last_fullsized":"2024-04-14T08:41:05.166337+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:28:05.473331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162819+0000","last_change":"2024-04-14T08:41:05.153004+0000","last_active":"2024-04-14T08:41:05.162819+0000","last_peered":"2024-04-14T08:41:05.162819+0000","last_clean":"2024-04-14T08:41:05.162819+0000","last_became_active":"2024-04-14T08:41:05.152620+0000","last_became_peered":"2024-04-14T08:41:05.152620+0000","last_unstale":"2024-04-14T08:41:05.162819+0000","last_undegraded":"2024-04-14T08:41:05.162819+0000","last_fullsized":"2024-04-14T08:41:05.162819+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:05:21.134598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"100'4","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164264+0000","last_change":"2024-04-14T08:41:04.154347+0000","last_active":"2024-04-14T08:41:05.164264+0000","last_peered":"2024-04-14T08:41:05.164264+0000","last_clean":"2024-04-14T08:41:05.164264+0000","last_became_active":"2024-04-14T08:41:04.154039+0000","last_became_peered":"2024-04-14T08:41:04.154039+0000","last_unstale":"2024-04-14T08:41:05.164264+0000","last_undegraded":"2024-04-14T08:41:05.164264+0000","last_fullsized":"2024-04-14T08:41:05.164264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:02:06.973563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.165854+0000","last_change":"2024-04-14T08:41:05.155717+0000","last_active":"2024-04-14T08:41:05.165854+0000","last_peered":"2024-04-14T08:41:05.165854+0000","last_clean":"2024-04-14T08:41:05.165854+0000","last_became_active":"2024-04-14T08:41:05.155466+0000","last_became_peered":"2024-04-14T08:41:05.155466+0000","last_unstale":"2024-04-14T08:41:05.165854+0000","last_undegraded":"2024-04-14T08:41:05.165854+0000","last_fullsized":"2024-04-14T08:41:05.165854+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:51:19.253762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"100'1","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.168300+0000","last_change":"2024-04-14T08:41:04.149699+0000","last_active":"2024-04-14T08:41:05.168300+0000","last_peered":"2024-04-14T08:41:05.168300+0000","last_clean":"2024-04-14T08:41:05.168300+0000","last_became_active":"2024-04-14T08:41:04.149002+0000","last_became_peered":"2024-04-14T08:41:04.149002+0000","last_unstale":"2024-04-14T08:41:05.168300+0000","last_undegraded":"2024-04-14T08:41:05.168300+0000","last_fullsized":"2024-04-14T08:41:05.168300+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:25.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161960+0000","last_change":"2024-04-14T08:41:05.156669+0000","last_active":"2024-04-14T08:41:05.161960+0000","last_peered":"2024-04-14T08:41:05.161960+0000","last_clean":"2024-04-14T08:41:05.161960+0000","last_became_active":"2024-04-14T08:41:05.156397+0000","last_became_peered":"2024-04-14T08:41:05.156397+0000","last_unstale":"2024-04-14T08:41:05.161960+0000","last_undegraded":"2024-04-14T08:41:05.161960+0000","last_fullsized":"2024-04-14T08:41:05.161960+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:47:10.299310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"100'2","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.159125+0000","last_change":"2024-04-14T08:41:04.151445+0000","last_active":"2024-04-14T08:41:05.159125+0000","last_peered":"2024-04-14T08:41:05.159125+0000","last_clean":"2024-04-14T08:41:05.159125+0000","last_became_active":"2024-04-14T08:41:04.151195+0000","last_became_peered":"2024-04-14T08:41:04.151195+0000","last_unstale":"2024-04-14T08:41:05.159125+0000","last_undegraded":"2024-04-14T08:41:05.159125+0000","last_fullsized":"2024-04-14T08:41:05.159125+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:05:57.411833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.168280+0000","last_change":"2024-04-14T08:41:05.164986+0000","last_active":"2024-04-14T08:41:05.168280+0000","last_peered":"2024-04-14T08:41:05.168280+0000","last_clean":"2024-04-14T08:41:05.168280+0000","last_became_active":"2024-04-14T08:41:05.164551+0000","last_became_peered":"2024-04-14T08:41:05.164551+0000","last_unstale":"2024-04-14T08:41:05.168280+0000","last_undegraded":"2024-04-14T08:41:05.168280+0000","last_fullsized":"2024-04-14T08:41:05.168280+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:45:47.080994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"100'2","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167153+0000","last_change":"2024-04-14T08:41:04.158464+0000","last_active":"2024-04-14T08:41:05.167153+0000","last_peered":"2024-04-14T08:41:05.167153+0000","last_clean":"2024-04-14T08:41:05.167153+0000","last_became_active":"2024-04-14T08:41:04.158165+0000","last_became_peered":"2024-04-14T08:41:04.158165+0000","last_unstale":"2024-04-14T08:41:05.167153+0000","last_undegraded":"2024-04-14T08:41:05.167153+0000","last_fullsized":"2024-04-14T08:41:05.167153+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:34:32.173549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.168403+0000","last_change":"2024-04-14T08:41:05.167654+0000","last_active":"2024-04-14T08:41:05.168403+0000","last_peered":"2024-04-14T08:41:05.168403+0000","last_clean":"2024-04-14T08:41:05.168403+0000","last_became_active":"2024-04-14T08:41:05.167339+0000","last_became_peered":"2024-04-14T08:41:05.167339+0000","last_unstale":"2024-04-14T08:41:05.168403+0000","last_undegraded":"2024-04-14T08:41:05.168403+0000","last_fullsized":"2024-04-14T08:41:05.168403+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:27:32.491311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.150873+0000","last_change":"2024-04-14T08:41:04.149377+0000","last_active":"2024-04-14T08:41:06.150873+0000","last_peered":"2024-04-14T08:41:06.150873+0000","last_clean":"2024-04-14T08:41:06.150873+0000","last_became_active":"2024-04-14T08:41:04.149116+0000","last_became_peered":"2024-04-14T08:41:04.149116+0000","last_unstale":"2024-04-14T08:41:06.150873+0000","last_undegraded":"2024-04-14T08:41:06.150873+0000","last_fullsized":"2024-04-14T08:41:06.150873+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:30:10.017018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162759+0000","last_change":"2024-04-14T08:41:05.156286+0000","last_active":"2024-04-14T08:41:05.162759+0000","last_peered":"2024-04-14T08:41:05.162759+0000","last_clean":"2024-04-14T08:41:05.162759+0000","last_became_active":"2024-04-14T08:41:05.155982+0000","last_became_peered":"2024-04-14T08:41:05.155982+0000","last_unstale":"2024-04-14T08:41:05.162759+0000","last_undegraded":"2024-04-14T08:41:05.162759+0000","last_fullsized":"2024-04-14T08:41:05.162759+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:02:00.221376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"100'1","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164764+0000","last_change":"2024-04-14T08:41:04.154092+0000","last_active":"2024-04-14T08:41:05.164764+0000","last_peered":"2024-04-14T08:41:05.164764+0000","last_clean":"2024-04-14T08:41:05.164764+0000","last_became_active":"2024-04-14T08:41:04.153804+0000","last_became_peered":"2024-04-14T08:41:04.153804+0000","last_unstale":"2024-04-14T08:41:05.164764+0000","last_undegraded":"2024-04-14T08:41:05.164764+0000","last_fullsized":"2024-04-14T08:41:05.164764+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:34:08.476057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161777+0000","last_change":"2024-04-14T08:41:05.159308+0000","last_active":"2024-04-14T08:41:05.161777+0000","last_peered":"2024-04-14T08:41:05.161777+0000","last_clean":"2024-04-14T08:41:05.161777+0000","last_became_active":"2024-04-14T08:41:05.158909+0000","last_became_peered":"2024-04-14T08:41:05.158909+0000","last_unstale":"2024-04-14T08:41:05.161777+0000","last_undegraded":"2024-04-14T08:41:05.161777+0000","last_fullsized":"2024-04-14T08:41:05.161777+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:02:07.977713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.142295+0000","last_change":"2024-04-14T08:41:04.151411+0000","last_active":"2024-04-14T08:41:05.142295+0000","last_peered":"2024-04-14T08:41:05.142295+0000","last_clean":"2024-04-14T08:41:05.142295+0000","last_became_active":"2024-04-14T08:41:04.151049+0000","last_became_peered":"2024-04-14T08:41:04.151049+0000","last_unstale":"2024-04-14T08:41:05.142295+0000","last_undegraded":"2024-04-14T08:41:05.142295+0000","last_fullsized":"2024-04-14T08:41:05.142295+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:02:06.973563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.160071+0000","last_change":"2024-04-14T08:41:05.157697+0000","last_active":"2024-04-14T08:41:05.160071+0000","last_peered":"2024-04-14T08:41:05.160071+0000","last_clean":"2024-04-14T08:41:05.160071+0000","last_became_active":"2024-04-14T08:41:05.157446+0000","last_became_peered":"2024-04-14T08:41:05.157446+0000","last_unstale":"2024-04-14T08:41:05.160071+0000","last_undegraded":"2024-04-14T08:41:05.160071+0000","last_fullsized":"2024-04-14T08:41:05.160071+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:47:10.299310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.143653+0000","last_change":"2024-04-14T08:41:04.155786+0000","last_active":"2024-04-14T08:41:05.143653+0000","last_peered":"2024-04-14T08:41:05.143653+0000","last_clean":"2024-04-14T08:41:05.143653+0000","last_became_active":"2024-04-14T08:41:04.155469+0000","last_became_peered":"2024-04-14T08:41:04.155469+0000","last_unstale":"2024-04-14T08:41:05.143653+0000","last_undegraded":"2024-04-14T08:41:05.143653+0000","last_fullsized":"2024-04-14T08:41:05.143653+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:03:27.317448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162875+0000","last_change":"2024-04-14T08:41:05.156486+0000","last_active":"2024-04-14T08:41:05.162875+0000","last_peered":"2024-04-14T08:41:05.162875+0000","last_clean":"2024-04-14T08:41:05.162875+0000","last_became_active":"2024-04-14T08:41:05.156145+0000","last_became_peered":"2024-04-14T08:41:05.156145+0000","last_unstale":"2024-04-14T08:41:05.162875+0000","last_undegraded":"2024-04-14T08:41:05.162875+0000","last_fullsized":"2024-04-14T08:41:05.162875+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:54:28.030276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.141602+0000","last_change":"2024-04-14T08:41:04.152113+0000","last_active":"2024-04-14T08:41:05.141602+0000","last_peered":"2024-04-14T08:41:05.141602+0000","last_clean":"2024-04-14T08:41:05.141602+0000","last_became_active":"2024-04-14T08:41:04.151821+0000","last_became_peered":"2024-04-14T08:41:04.151821+0000","last_unstale":"2024-04-14T08:41:05.141602+0000","last_undegraded":"2024-04-14T08:41:05.141602+0000","last_fullsized":"2024-04-14T08:41:05.141602+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:03:27.317448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166949+0000","last_change":"2024-04-14T08:41:05.154435+0000","last_active":"2024-04-14T08:41:05.166949+0000","last_peered":"2024-04-14T08:41:05.166949+0000","last_clean":"2024-04-14T08:41:05.166949+0000","last_became_active":"2024-04-14T08:41:05.154144+0000","last_became_peered":"2024-04-14T08:41:05.154144+0000","last_unstale":"2024-04-14T08:41:05.166949+0000","last_undegraded":"2024-04-14T08:41:05.166949+0000","last_fullsized":"2024-04-14T08:41:05.166949+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:23:54.351934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.144059+0000","last_change":"2024-04-14T08:41:04.159556+0000","last_active":"2024-04-14T08:41:05.144059+0000","last_peered":"2024-04-14T08:41:05.144059+0000","last_clean":"2024-04-14T08:41:05.144059+0000","last_became_active":"2024-04-14T08:41:04.159323+0000","last_became_peered":"2024-04-14T08:41:04.159323+0000","last_unstale":"2024-04-14T08:41:05.144059+0000","last_undegraded":"2024-04-14T08:41:05.144059+0000","last_fullsized":"2024-04-14T08:41:05.144059+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:18:51.764627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.165954+0000","last_change":"2024-04-14T08:41:05.156838+0000","last_active":"2024-04-14T08:41:05.165954+0000","last_peered":"2024-04-14T08:41:05.165954+0000","last_clean":"2024-04-14T08:41:05.165954+0000","last_became_active":"2024-04-14T08:41:05.156207+0000","last_became_peered":"2024-04-14T08:41:05.156207+0000","last_unstale":"2024-04-14T08:41:05.165954+0000","last_undegraded":"2024-04-14T08:41:05.165954+0000","last_fullsized":"2024-04-14T08:41:05.165954+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:33:37.146832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"100'3","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.150940+0000","last_change":"2024-04-14T08:41:04.156229+0000","last_active":"2024-04-14T08:41:06.150940+0000","last_peered":"2024-04-14T08:41:06.150940+0000","last_clean":"2024-04-14T08:41:06.150940+0000","last_became_active":"2024-04-14T08:41:04.155893+0000","last_became_peered":"2024-04-14T08:41:04.155893+0000","last_unstale":"2024-04-14T08:41:06.150940+0000","last_undegraded":"2024-04-14T08:41:06.150940+0000","last_fullsized":"2024-04-14T08:41:06.150940+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:41:28.143707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167011+0000","last_change":"2024-04-14T08:41:05.157846+0000","last_active":"2024-04-14T08:41:05.167011+0000","last_peered":"2024-04-14T08:41:05.167011+0000","last_clean":"2024-04-14T08:41:05.167011+0000","last_became_active":"2024-04-14T08:41:05.157476+0000","last_became_peered":"2024-04-14T08:41:05.157476+0000","last_unstale":"2024-04-14T08:41:05.167011+0000","last_undegraded":"2024-04-14T08:41:05.167011+0000","last_fullsized":"2024-04-14T08:41:05.167011+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:04:51.866864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"100'6","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170697+0000","last_change":"2024-04-14T08:41:04.159163+0000","last_active":"2024-04-14T08:41:05.170697+0000","last_peered":"2024-04-14T08:41:05.170697+0000","last_clean":"2024-04-14T08:41:05.170697+0000","last_became_active":"2024-04-14T08:41:04.158859+0000","last_became_peered":"2024-04-14T08:41:04.158859+0000","last_unstale":"2024-04-14T08:41:05.170697+0000","last_undegraded":"2024-04-14T08:41:05.170697+0000","last_fullsized":"2024-04-14T08:41:05.170697+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:23:53.347784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164682+0000","last_change":"2024-04-14T08:41:05.162043+0000","last_active":"2024-04-14T08:41:05.164682+0000","last_peered":"2024-04-14T08:41:05.164682+0000","last_clean":"2024-04-14T08:41:05.164682+0000","last_became_active":"2024-04-14T08:41:05.161757+0000","last_became_peered":"2024-04-14T08:41:05.161757+0000","last_unstale":"2024-04-14T08:41:05.164682+0000","last_undegraded":"2024-04-14T08:41:05.164682+0000","last_fullsized":"2024-04-14T08:41:05.164682+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:13:53.752730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"100'2","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.169032+0000","last_change":"2024-04-14T08:41:04.153066+0000","last_active":"2024-04-14T08:41:05.169032+0000","last_peered":"2024-04-14T08:41:05.169032+0000","last_clean":"2024-04-14T08:41:05.169032+0000","last_became_active":"2024-04-14T08:41:04.152797+0000","last_became_peered":"2024-04-14T08:41:04.152797+0000","last_unstale":"2024-04-14T08:41:05.169032+0000","last_undegraded":"2024-04-14T08:41:05.169032+0000","last_fullsized":"2024-04-14T08:41:05.169032+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:34.256219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161157+0000","last_change":"2024-04-14T08:41:05.156664+0000","last_active":"2024-04-14T08:41:05.161157+0000","last_peered":"2024-04-14T08:41:05.161157+0000","last_clean":"2024-04-14T08:41:05.161157+0000","last_became_active":"2024-04-14T08:41:05.156339+0000","last_became_peered":"2024-04-14T08:41:05.156339+0000","last_unstale":"2024-04-14T08:41:05.161157+0000","last_undegraded":"2024-04-14T08:41:05.161157+0000","last_fullsized":"2024-04-14T08:41:05.161157+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:16:26.973756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.145564+0000","last_change":"2024-04-14T08:41:04.148022+0000","last_active":"2024-04-14T08:41:05.145564+0000","last_peered":"2024-04-14T08:41:05.145564+0000","last_clean":"2024-04-14T08:41:05.145564+0000","last_became_active":"2024-04-14T08:41:04.147766+0000","last_became_peered":"2024-04-14T08:41:04.147766+0000","last_unstale":"2024-04-14T08:41:05.145564+0000","last_undegraded":"2024-04-14T08:41:05.145564+0000","last_fullsized":"2024-04-14T08:41:05.145564+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:45:46.076845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166420+0000","last_change":"2024-04-14T08:41:05.163876+0000","last_active":"2024-04-14T08:41:05.166420+0000","last_peered":"2024-04-14T08:41:05.166420+0000","last_clean":"2024-04-14T08:41:05.166420+0000","last_became_active":"2024-04-14T08:41:05.163583+0000","last_became_peered":"2024-04-14T08:41:05.163583+0000","last_unstale":"2024-04-14T08:41:05.166420+0000","last_undegraded":"2024-04-14T08:41:05.166420+0000","last_fullsized":"2024-04-14T08:41:05.166420+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:29:18.350492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.139488+0000","last_change":"2024-04-14T08:41:04.148341+0000","last_active":"2024-04-14T08:41:05.139488+0000","last_peered":"2024-04-14T08:41:05.139488+0000","last_clean":"2024-04-14T08:41:05.139488+0000","last_became_active":"2024-04-14T08:41:04.147735+0000","last_became_peered":"2024-04-14T08:41:04.147735+0000","last_unstale":"2024-04-14T08:41:05.139488+0000","last_undegraded":"2024-04-14T08:41:05.139488+0000","last_fullsized":"2024-04-14T08:41:05.139488+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:27:17.741937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166512+0000","last_change":"2024-04-14T08:41:05.164591+0000","last_active":"2024-04-14T08:41:05.166512+0000","last_peered":"2024-04-14T08:41:05.166512+0000","last_clean":"2024-04-14T08:41:05.166512+0000","last_became_active":"2024-04-14T08:41:05.164310+0000","last_became_peered":"2024-04-14T08:41:05.164310+0000","last_unstale":"2024-04-14T08:41:05.166512+0000","last_undegraded":"2024-04-14T08:41:05.166512+0000","last_fullsized":"2024-04-14T08:41:05.166512+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:19:19.705118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"100'8","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.159647+0000","last_change":"2024-04-14T08:41:04.152016+0000","last_active":"2024-04-14T08:41:05.159647+0000","last_peered":"2024-04-14T08:41:05.159647+0000","last_clean":"2024-04-14T08:41:05.159647+0000","last_became_active":"2024-04-14T08:41:04.151760+0000","last_became_peered":"2024-04-14T08:41:04.151760+0000","last_unstale":"2024-04-14T08:41:05.159647+0000","last_undegraded":"2024-04-14T08:41:05.159647+0000","last_fullsized":"2024-04-14T08:41:05.159647+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:45:46.076845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.154546+0000","last_change":"2024-04-14T08:41:05.149470+0000","last_active":"2024-04-14T08:41:05.154546+0000","last_peered":"2024-04-14T08:41:05.154546+0000","last_clean":"2024-04-14T08:41:05.154546+0000","last_became_active":"2024-04-14T08:41:05.149175+0000","last_became_peered":"2024-04-14T08:41:05.149175+0000","last_unstale":"2024-04-14T08:41:05.154546+0000","last_undegraded":"2024-04-14T08:41:05.154546+0000","last_fullsized":"2024-04-14T08:41:05.154546+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:17:47.808646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"100'1","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164513+0000","last_change":"2024-04-14T08:41:04.149516+0000","last_active":"2024-04-14T08:41:05.164513+0000","last_peered":"2024-04-14T08:41:05.164513+0000","last_clean":"2024-04-14T08:41:05.164513+0000","last_became_active":"2024-04-14T08:41:04.149238+0000","last_became_peered":"2024-04-14T08:41:04.149238+0000","last_unstale":"2024-04-14T08:41:05.164513+0000","last_undegraded":"2024-04-14T08:41:05.164513+0000","last_fullsized":"2024-04-14T08:41:05.164513+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:34:08.476057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170589+0000","last_change":"2024-04-14T08:41:05.165446+0000","last_active":"2024-04-14T08:41:05.170589+0000","last_peered":"2024-04-14T08:41:05.170589+0000","last_clean":"2024-04-14T08:41:05.170589+0000","last_became_active":"2024-04-14T08:41:05.165161+0000","last_became_peered":"2024-04-14T08:41:05.165161+0000","last_unstale":"2024-04-14T08:41:05.170589+0000","last_undegraded":"2024-04-14T08:41:05.170589+0000","last_fullsized":"2024-04-14T08:41:05.170589+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:45:47.080994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.151078+0000","last_change":"2024-04-14T08:41:04.147164+0000","last_active":"2024-04-14T08:41:06.151078+0000","last_peered":"2024-04-14T08:41:06.151078+0000","last_clean":"2024-04-14T08:41:06.151078+0000","last_became_active":"2024-04-14T08:41:04.146880+0000","last_became_peered":"2024-04-14T08:41:04.146880+0000","last_unstale":"2024-04-14T08:41:06.151078+0000","last_undegraded":"2024-04-14T08:41:06.151078+0000","last_fullsized":"2024-04-14T08:41:06.151078+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:39:32.284645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167139+0000","last_change":"2024-04-14T08:41:05.159342+0000","last_active":"2024-04-14T08:41:05.167139+0000","last_peered":"2024-04-14T08:41:05.167139+0000","last_clean":"2024-04-14T08:41:05.167139+0000","last_became_active":"2024-04-14T08:41:05.159107+0000","last_became_peered":"2024-04-14T08:41:05.159107+0000","last_unstale":"2024-04-14T08:41:05.167139+0000","last_undegraded":"2024-04-14T08:41:05.167139+0000","last_fullsized":"2024-04-14T08:41:05.167139+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:21:20.731388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.157172+0000","last_change":"2024-04-14T08:41:04.157007+0000","last_active":"2024-04-14T08:41:05.157172+0000","last_peered":"2024-04-14T08:41:05.157172+0000","last_clean":"2024-04-14T08:41:05.157172+0000","last_became_active":"2024-04-14T08:41:04.156742+0000","last_became_peered":"2024-04-14T08:41:04.156742+0000","last_unstale":"2024-04-14T08:41:05.157172+0000","last_undegraded":"2024-04-14T08:41:05.157172+0000","last_fullsized":"2024-04-14T08:41:05.157172+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:16:25.969607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.159641+0000","last_change":"2024-04-14T08:41:05.152608+0000","last_active":"2024-04-14T08:41:05.159641+0000","last_peered":"2024-04-14T08:41:05.159641+0000","last_clean":"2024-04-14T08:41:05.159641+0000","last_became_active":"2024-04-14T08:41:05.151471+0000","last_became_peered":"2024-04-14T08:41:05.151471+0000","last_unstale":"2024-04-14T08:41:05.159641+0000","last_undegraded":"2024-04-14T08:41:05.159641+0000","last_fullsized":"2024-04-14T08:41:05.159641+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:03:31.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"100'2","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167130+0000","last_change":"2024-04-14T08:41:04.156830+0000","last_active":"2024-04-14T08:41:05.167130+0000","last_peered":"2024-04-14T08:41:05.167130+0000","last_clean":"2024-04-14T08:41:05.167130+0000","last_became_active":"2024-04-14T08:41:04.156558+0000","last_became_peered":"2024-04-14T08:41:04.156558+0000","last_unstale":"2024-04-14T08:41:05.167130+0000","last_undegraded":"2024-04-14T08:41:05.167130+0000","last_fullsized":"2024-04-14T08:41:05.167130+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:23:28.198182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.168591+0000","last_change":"2024-04-14T08:41:05.166489+0000","last_active":"2024-04-14T08:41:05.168591+0000","last_peered":"2024-04-14T08:41:05.168591+0000","last_clean":"2024-04-14T08:41:05.168591+0000","last_became_active":"2024-04-14T08:41:05.165706+0000","last_became_peered":"2024-04-14T08:41:05.165706+0000","last_unstale":"2024-04-14T08:41:05.168591+0000","last_undegraded":"2024-04-14T08:41:05.168591+0000","last_fullsized":"2024-04-14T08:41:05.168591+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:31:05.489822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.139142+0000","last_change":"2024-04-14T08:41:04.147901+0000","last_active":"2024-04-14T08:41:05.139142+0000","last_peered":"2024-04-14T08:41:05.139142+0000","last_clean":"2024-04-14T08:41:05.139142+0000","last_became_active":"2024-04-14T08:41:04.147625+0000","last_became_peered":"2024-04-14T08:41:04.147625+0000","last_unstale":"2024-04-14T08:41:05.139142+0000","last_undegraded":"2024-04-14T08:41:05.139142+0000","last_fullsized":"2024-04-14T08:41:05.139142+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:24.483825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164730+0000","last_change":"2024-04-14T08:41:05.161588+0000","last_active":"2024-04-14T08:41:05.164730+0000","last_peered":"2024-04-14T08:41:05.164730+0000","last_clean":"2024-04-14T08:41:05.164730+0000","last_became_active":"2024-04-14T08:41:05.161359+0000","last_became_peered":"2024-04-14T08:41:05.161359+0000","last_unstale":"2024-04-14T08:41:05.164730+0000","last_undegraded":"2024-04-14T08:41:05.164730+0000","last_fullsized":"2024-04-14T08:41:05.164730+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:27:59.897940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.148925+0000","last_change":"2024-04-14T08:41:04.153035+0000","last_active":"2024-04-14T08:41:05.148925+0000","last_peered":"2024-04-14T08:41:05.148925+0000","last_clean":"2024-04-14T08:41:05.148925+0000","last_became_active":"2024-04-14T08:41:04.152814+0000","last_became_peered":"2024-04-14T08:41:04.152814+0000","last_unstale":"2024-04-14T08:41:05.148925+0000","last_undegraded":"2024-04-14T08:41:05.148925+0000","last_fullsized":"2024-04-14T08:41:05.148925+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:04:50.862715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.165705+0000","last_change":"2024-04-14T08:41:05.155051+0000","last_active":"2024-04-14T08:41:05.165705+0000","last_peered":"2024-04-14T08:41:05.165705+0000","last_clean":"2024-04-14T08:41:05.165705+0000","last_became_active":"2024-04-14T08:41:05.154333+0000","last_became_peered":"2024-04-14T08:41:05.154333+0000","last_unstale":"2024-04-14T08:41:05.165705+0000","last_undegraded":"2024-04-14T08:41:05.165705+0000","last_fullsized":"2024-04-14T08:41:05.165705+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:15:49.596490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.139956+0000","last_change":"2024-04-14T08:41:04.151672+0000","last_active":"2024-04-14T08:41:05.139956+0000","last_peered":"2024-04-14T08:41:05.139956+0000","last_clean":"2024-04-14T08:41:05.139956+0000","last_became_active":"2024-04-14T08:41:04.151356+0000","last_became_peered":"2024-04-14T08:41:04.151356+0000","last_unstale":"2024-04-14T08:41:05.139956+0000","last_undegraded":"2024-04-14T08:41:05.139956+0000","last_fullsized":"2024-04-14T08:41:05.139956+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:50:47.507145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162856+0000","last_change":"2024-04-14T08:41:05.155529+0000","last_active":"2024-04-14T08:41:05.162856+0000","last_peered":"2024-04-14T08:41:05.162856+0000","last_clean":"2024-04-14T08:41:05.162856+0000","last_became_active":"2024-04-14T08:41:05.155133+0000","last_became_peered":"2024-04-14T08:41:05.155133+0000","last_unstale":"2024-04-14T08:41:05.162856+0000","last_undegraded":"2024-04-14T08:41:05.162856+0000","last_fullsized":"2024-04-14T08:41:05.162856+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:36:53.910896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"100'2","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.168261+0000","last_change":"2024-04-14T08:41:04.153068+0000","last_active":"2024-04-14T08:41:05.168261+0000","last_peered":"2024-04-14T08:41:05.168261+0000","last_clean":"2024-04-14T08:41:05.168261+0000","last_became_active":"2024-04-14T08:41:04.152818+0000","last_became_peered":"2024-04-14T08:41:04.152818+0000","last_unstale":"2024-04-14T08:41:05.168261+0000","last_undegraded":"2024-04-14T08:41:05.168261+0000","last_fullsized":"2024-04-14T08:41:05.168261+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:09:50.573062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161918+0000","last_change":"2024-04-14T08:41:05.157262+0000","last_active":"2024-04-14T08:41:05.161918+0000","last_peered":"2024-04-14T08:41:05.161918+0000","last_clean":"2024-04-14T08:41:05.161918+0000","last_became_active":"2024-04-14T08:41:05.157005+0000","last_became_peered":"2024-04-14T08:41:05.157005+0000","last_unstale":"2024-04-14T08:41:05.161918+0000","last_undegraded":"2024-04-14T08:41:05.161918+0000","last_fullsized":"2024-04-14T08:41:05.161918+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:23:53.194822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.144264+0000","last_change":"2024-04-14T08:41:04.159621+0000","last_active":"2024-04-14T08:41:05.144264+0000","last_peered":"2024-04-14T08:41:05.144264+0000","last_clean":"2024-04-14T08:41:05.144264+0000","last_became_active":"2024-04-14T08:41:04.159445+0000","last_became_peered":"2024-04-14T08:41:04.159445+0000","last_unstale":"2024-04-14T08:41:05.144264+0000","last_undegraded":"2024-04-14T08:41:05.144264+0000","last_fullsized":"2024-04-14T08:41:05.144264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:36:27.884380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.160388+0000","last_change":"2024-04-14T08:41:05.158755+0000","last_active":"2024-04-14T08:41:05.160388+0000","last_peered":"2024-04-14T08:41:05.160388+0000","last_clean":"2024-04-14T08:41:05.160388+0000","last_became_active":"2024-04-14T08:41:05.158493+0000","last_became_peered":"2024-04-14T08:41:05.158493+0000","last_unstale":"2024-04-14T08:41:05.160388+0000","last_undegraded":"2024-04-14T08:41:05.160388+0000","last_fullsized":"2024-04-14T08:41:05.160388+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:23:53.194822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.148691+0000","last_change":"2024-04-14T08:41:04.149338+0000","last_active":"2024-04-14T08:41:05.148691+0000","last_peered":"2024-04-14T08:41:05.148691+0000","last_clean":"2024-04-14T08:41:05.148691+0000","last_became_active":"2024-04-14T08:41:04.148983+0000","last_became_peered":"2024-04-14T08:41:04.148983+0000","last_unstale":"2024-04-14T08:41:05.148691+0000","last_undegraded":"2024-04-14T08:41:05.148691+0000","last_fullsized":"2024-04-14T08:41:05.148691+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:34.256219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170691+0000","last_change":"2024-04-14T08:41:05.167482+0000","last_active":"2024-04-14T08:41:05.170691+0000","last_peered":"2024-04-14T08:41:05.170691+0000","last_clean":"2024-04-14T08:41:05.170691+0000","last_became_active":"2024-04-14T08:41:05.167159+0000","last_became_peered":"2024-04-14T08:41:05.167159+0000","last_unstale":"2024-04-14T08:41:05.170691+0000","last_undegraded":"2024-04-14T08:41:05.170691+0000","last_fullsized":"2024-04-14T08:41:05.170691+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:23:53.194822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.150663+0000","last_change":"2024-04-14T08:41:04.154343+0000","last_active":"2024-04-14T08:41:05.150663+0000","last_peered":"2024-04-14T08:41:05.150663+0000","last_clean":"2024-04-14T08:41:05.150663+0000","last_became_active":"2024-04-14T08:41:04.154106+0000","last_became_peered":"2024-04-14T08:41:04.154106+0000","last_unstale":"2024-04-14T08:41:05.150663+0000","last_undegraded":"2024-04-14T08:41:05.150663+0000","last_fullsized":"2024-04-14T08:41:05.150663+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:21:08.066477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.165824+0000","last_change":"2024-04-14T08:41:05.155780+0000","last_active":"2024-04-14T08:41:05.165824+0000","last_peered":"2024-04-14T08:41:05.165824+0000","last_clean":"2024-04-14T08:41:05.165824+0000","last_became_active":"2024-04-14T08:41:05.155540+0000","last_became_peered":"2024-04-14T08:41:05.155540+0000","last_unstale":"2024-04-14T08:41:05.165824+0000","last_undegraded":"2024-04-14T08:41:05.165824+0000","last_fullsized":"2024-04-14T08:41:05.165824+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:52:39.232361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.141060+0000","last_change":"2024-04-14T08:41:04.154091+0000","last_active":"2024-04-14T08:41:05.141060+0000","last_peered":"2024-04-14T08:41:05.141060+0000","last_clean":"2024-04-14T08:41:05.141060+0000","last_became_active":"2024-04-14T08:41:04.153829+0000","last_became_peered":"2024-04-14T08:41:04.153829+0000","last_unstale":"2024-04-14T08:41:05.141060+0000","last_undegraded":"2024-04-14T08:41:05.141060+0000","last_fullsized":"2024-04-14T08:41:05.141060+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:13:52.748581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.171009+0000","last_change":"2024-04-14T08:41:05.166856+0000","last_active":"2024-04-14T08:41:05.171009+0000","last_peered":"2024-04-14T08:41:05.171009+0000","last_clean":"2024-04-14T08:41:05.171009+0000","last_became_active":"2024-04-14T08:41:05.166486+0000","last_became_peered":"2024-04-14T08:41:05.166486+0000","last_unstale":"2024-04-14T08:41:05.171009+0000","last_undegraded":"2024-04-14T08:41:05.171009+0000","last_fullsized":"2024-04-14T08:41:05.171009+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:20:52.790502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.139279+0000","last_change":"2024-04-14T08:41:04.156898+0000","last_active":"2024-04-14T08:41:05.139279+0000","last_peered":"2024-04-14T08:41:05.139279+0000","last_clean":"2024-04-14T08:41:05.139279+0000","last_became_active":"2024-04-14T08:41:04.156587+0000","last_became_peered":"2024-04-14T08:41:04.156587+0000","last_unstale":"2024-04-14T08:41:05.139279+0000","last_undegraded":"2024-04-14T08:41:05.139279+0000","last_fullsized":"2024-04-14T08:41:05.139279+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:01:27.986252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.151269+0000","last_change":"2024-04-14T08:41:05.157345+0000","last_active":"2024-04-14T08:41:06.151269+0000","last_peered":"2024-04-14T08:41:06.151269+0000","last_clean":"2024-04-14T08:41:06.151269+0000","last_became_active":"2024-04-14T08:41:05.157078+0000","last_became_peered":"2024-04-14T08:41:05.157078+0000","last_unstale":"2024-04-14T08:41:06.151269+0000","last_undegraded":"2024-04-14T08:41:06.151269+0000","last_fullsized":"2024-04-14T08:41:06.151269+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:41:47.211392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.149091+0000","last_change":"2024-04-14T08:41:04.148649+0000","last_active":"2024-04-14T08:41:05.149091+0000","last_peered":"2024-04-14T08:41:05.149091+0000","last_clean":"2024-04-14T08:41:05.149091+0000","last_became_active":"2024-04-14T08:41:04.148362+0000","last_became_peered":"2024-04-14T08:41:04.148362+0000","last_unstale":"2024-04-14T08:41:05.149091+0000","last_undegraded":"2024-04-14T08:41:05.149091+0000","last_fullsized":"2024-04-14T08:41:05.149091+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:21:19.727239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167703+0000","last_change":"2024-04-14T08:41:05.163443+0000","last_active":"2024-04-14T08:41:05.167703+0000","last_peered":"2024-04-14T08:41:05.167703+0000","last_clean":"2024-04-14T08:41:05.167703+0000","last_became_active":"2024-04-14T08:41:05.163161+0000","last_became_peered":"2024-04-14T08:41:05.163161+0000","last_unstale":"2024-04-14T08:41:05.167703+0000","last_undegraded":"2024-04-14T08:41:05.167703+0000","last_fullsized":"2024-04-14T08:41:05.167703+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:03:28.321597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.140110+0000","last_change":"2024-04-14T08:41:04.157049+0000","last_active":"2024-04-14T08:41:05.140110+0000","last_peered":"2024-04-14T08:41:05.140110+0000","last_clean":"2024-04-14T08:41:05.140110+0000","last_became_active":"2024-04-14T08:41:04.156773+0000","last_became_peered":"2024-04-14T08:41:04.156773+0000","last_unstale":"2024-04-14T08:41:05.140110+0000","last_undegraded":"2024-04-14T08:41:05.140110+0000","last_fullsized":"2024-04-14T08:41:05.140110+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:01:57.525115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.168428+0000","last_change":"2024-04-14T08:41:05.166136+0000","last_active":"2024-04-14T08:41:05.168428+0000","last_peered":"2024-04-14T08:41:05.168428+0000","last_clean":"2024-04-14T08:41:05.168428+0000","last_became_active":"2024-04-14T08:41:05.165758+0000","last_became_peered":"2024-04-14T08:41:05.165758+0000","last_unstale":"2024-04-14T08:41:05.168428+0000","last_undegraded":"2024-04-14T08:41:05.168428+0000","last_fullsized":"2024-04-14T08:41:05.168428+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:47:10.299310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"100'2","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166001+0000","last_change":"2024-04-14T08:41:04.149919+0000","last_active":"2024-04-14T08:41:05.166001+0000","last_peered":"2024-04-14T08:41:05.166001+0000","last_clean":"2024-04-14T08:41:05.166001+0000","last_became_active":"2024-04-14T08:41:04.149443+0000","last_became_peered":"2024-04-14T08:41:04.149443+0000","last_unstale":"2024-04-14T08:41:05.166001+0000","last_undegraded":"2024-04-14T08:41:05.166001+0000","last_fullsized":"2024-04-14T08:41:05.166001+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:19:38.868105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":334,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.150262+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:41:06.150262+0000","last_peered":"2024-04-14T08:41:06.150262+0000","last_clean":"2024-04-14T08:41:06.150262+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:41:06.150262+0000","last_undegraded":"2024-04-14T08:41:06.150262+0000","last_fullsized":"2024-04-14T08:41:06.150262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1843200,"data_stored":1164491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":566796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":566796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":566796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":566796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":566796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":566796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1822720,"data_stored":1157164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:41:08.122 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:41:08.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.255+0000 7f7043cd1640 1 Processor -- start 2024-04-14T08:41:08.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.256+0000 7f7043cd1640 1 -- start start 2024-04-14T08:41:08.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.256+0000 7f7043cd1640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f703c15c150 0x7f703c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.256+0000 7f7043cd1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f703c0d1840 0x7f703c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.256+0000 7f7043cd1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f703c162530 0x7f703c15ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.256+0000 7f7043cd1640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f703c0cef10 con 0x7f703c162530 2024-04-14T08:41:08.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.256+0000 7f7043cd1640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f703c0d1c80 con 0x7f703c15c150 2024-04-14T08:41:08.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.256+0000 7f7043cd1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f703c162940 con 0x7f703c0d1840 2024-04-14T08:41:08.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.257+0000 7f704226e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f703c162530 0x7f703c15ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.257+0000 7f704126c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f703c0d1840 0x7f703c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.257+0000 7f704226e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f703c162530 0x7f703c15ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36674/0 (socket says 172.21.15.172:36674) 2024-04-14T08:41:08.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.257+0000 7f7041a6d640 1 --2- 172.21.15.172:0/1137952738 >> v2:172.21.15.177:3300/0 conn(0x7f703c15c150 0x7f703c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.257+0000 7f704226e640 1 -- 172.21.15.172:0/1137952738 learned_addr learned my addr 172.21.15.172:0/1137952738 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.257+0000 7f704126c640 1 -- 172.21.15.172:0/1137952738 >> v2:172.21.15.177:3300/0 conn(0x7f703c15c150 msgr2=0x7f703c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.257+0000 7f704126c640 1 --2- 172.21.15.172:0/1137952738 >> v2:172.21.15.177:3300/0 conn(0x7f703c15c150 0x7f703c0d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.257+0000 7f704126c640 1 -- 172.21.15.172:0/1137952738 >> v2:172.21.15.172:3300/0 conn(0x7f703c162530 msgr2=0x7f703c15ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.257+0000 7f704126c640 1 --2- 172.21.15.172:0/1137952738 >> v2:172.21.15.172:3300/0 conn(0x7f703c162530 0x7f703c15ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.257+0000 7f704126c640 1 -- 172.21.15.172:0/1137952738 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c0d0200 con 0x7f703c0d1840 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.257+0000 7f704226e640 1 --2- 172.21.15.172:0/1137952738 >> v2:172.21.15.172:3300/0 conn(0x7f703c162530 0x7f703c15ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.258+0000 7f704126c640 1 --2- 172.21.15.172:0/1137952738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f703c0d1840 0x7f703c161ff0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f703c0d03d0 tx=0x7f703802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7394fe94a759533a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.258+0000 7f7040a6b640 1 -- 172.21.15.172:0/1137952738 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038031780 con 0x7f703c0d1840 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.258+0000 7f7040a6b640 1 -- 172.21.15.172:0/1137952738 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7038037070 con 0x7f703c0d1840 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.258+0000 7f7040a6b640 1 -- 172.21.15.172:0/1137952738 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038031b40 con 0x7f703c0d1840 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.258+0000 7f7043cd1640 1 -- 172.21.15.172:0/1137952738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f703c0d1840 msgr2=0x7f703c161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.258+0000 7f7043cd1640 1 --2- 172.21.15.172:0/1137952738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f703c0d1840 0x7f703c161ff0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f703c0d03d0 tx=0x7f703802b820 comp rx=0 tx=0).stop 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.258+0000 7f7043cd1640 1 -- 172.21.15.172:0/1137952738 shutdown_connections 2024-04-14T08:41:08.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.258+0000 7f7043cd1640 1 --2- 172.21.15.172:0/1137952738 >> v2:172.21.15.172:3300/0 conn(0x7f703c162530 0x7f703c15ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.258+0000 7f7043cd1640 1 --2- 172.21.15.172:0/1137952738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f703c0d1840 0x7f703c161ff0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.258+0000 7f7043cd1640 1 --2- 172.21.15.172:0/1137952738 >> v2:172.21.15.177:3300/0 conn(0x7f703c15c150 0x7f703c0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.258+0000 7f7043cd1640 1 -- 172.21.15.172:0/1137952738 >> 172.21.15.172:0/1137952738 conn(0x7f703c09d580 msgr2=0x7f703c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:08.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.259+0000 7f7043cd1640 1 -- 172.21.15.172:0/1137952738 shutdown_connections 2024-04-14T08:41:08.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.259+0000 7f7043cd1640 1 -- 172.21.15.172:0/1137952738 wait complete. 2024-04-14T08:41:08.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.260+0000 7f7043cd1640 1 Processor -- start 2024-04-14T08:41:08.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.260+0000 7f7043cd1640 1 -- start start 2024-04-14T08:41:08.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.260+0000 7f7043cd1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f703c0d1840 0x7f703c155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.260+0000 7f7043cd1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f703c15c150 0x7f703c0b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.260+0000 7f7043cd1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c162530 0x7f703c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.260+0000 7f7043cd1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f703c0cf720 con 0x7f703c162530 2024-04-14T08:41:08.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.260+0000 7f7043cd1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f703c0cf440 con 0x7f703c15c150 2024-04-14T08:41:08.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.260+0000 7f7043cd1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f703c0cfa00 con 0x7f703c0d1840 2024-04-14T08:41:08.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f704226e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c162530 0x7f703c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f704226e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c162530 0x7f703c06d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36690/0 (socket says 172.21.15.172:36690) 2024-04-14T08:41:08.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f704226e640 1 -- 172.21.15.172:0/265782111 learned_addr learned my addr 172.21.15.172:0/265782111 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:08.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f704126c640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f703c15c150 0x7f703c0b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f7041a6d640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f703c0d1840 0x7f703c155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f704226e640 1 -- 172.21.15.172:0/265782111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f703c0d1840 msgr2=0x7f703c155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f704226e640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f703c0d1840 0x7f703c155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f704226e640 1 -- 172.21.15.172:0/265782111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f703c15c150 msgr2=0x7f703c0b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f704226e640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f703c15c150 0x7f703c0b6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f704226e640 1 -- 172.21.15.172:0/265782111 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c1556c0 con 0x7f703c162530 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f704226e640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c162530 0x7f703c06d480 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f703400efc0 tx=0x7f703400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f70327fc640 1 -- 172.21.15.172:0/265782111 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703400cbe0 con 0x7f703c162530 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f70327fc640 1 -- 172.21.15.172:0/265782111 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703400cd80 con 0x7f703c162530 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f7043cd1640 1 -- 172.21.15.172:0/265782111 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c1e8a30 con 0x7f703c162530 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f70327fc640 1 -- 172.21.15.172:0/265782111 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034003bb0 con 0x7f703c162530 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.261+0000 7f7043cd1640 1 -- 172.21.15.172:0/265782111 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c0cc160 con 0x7f703c162530 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.262+0000 7f7043cd1640 1 -- 172.21.15.172:0/265782111 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703c075490 con 0x7f703c162530 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.263+0000 7f70327fc640 1 -- 172.21.15.172:0/265782111 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f703401b030 con 0x7f703c162530 2024-04-14T08:41:08.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.263+0000 7f70327fc640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f702006c8f0 0x7f702006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.263+0000 7f7041a6d640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f702006c8f0 0x7f702006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.263+0000 7f70327fc640 1 -- 172.21.15.172:0/265782111 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f703408bf50 con 0x7f703c162530 2024-04-14T08:41:08.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.264+0000 7f7041a6d640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f702006c8f0 0x7f702006edb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f702c009ac0 tx=0x7f702c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:08.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.268+0000 7f70327fc640 1 -- 172.21.15.172:0/265782111 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7034022020 con 0x7f703c162530 2024-04-14T08:41:08.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.440+0000 7f7043cd1640 1 -- 172.21.15.172:0/265782111 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f703c155c40 con 0x7f702006c8f0 2024-04-14T08:41:08.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.446+0000 7f70327fc640 1 -- 172.21.15.172:0/265782111 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252194 (secure 0 0 0) 0x7f703c155c40 con 0x7f702006c8f0 2024-04-14T08:41:08.450 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:08.451 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:41:08.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.449+0000 7f7043cd1640 1 -- 172.21.15.172:0/265782111 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f702006c8f0 msgr2=0x7f702006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.449+0000 7f7043cd1640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f702006c8f0 0x7f702006edb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f702c009ac0 tx=0x7f702c009190 comp rx=0 tx=0).stop 2024-04-14T08:41:08.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.449+0000 7f7043cd1640 1 -- 172.21.15.172:0/265782111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c162530 msgr2=0x7f703c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.449+0000 7f7043cd1640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c162530 0x7f703c06d480 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f703400efc0 tx=0x7f703400c490 comp rx=0 tx=0).stop 2024-04-14T08:41:08.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.449+0000 7f7043cd1640 1 -- 172.21.15.172:0/265782111 shutdown_connections 2024-04-14T08:41:08.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.449+0000 7f7043cd1640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f702006c8f0 0x7f702006edb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.449+0000 7f7043cd1640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f703c162530 0x7f703c06d480 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.449+0000 7f7043cd1640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f703c15c150 0x7f703c0b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.449+0000 7f7043cd1640 1 --2- 172.21.15.172:0/265782111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f703c0d1840 0x7f703c155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.449+0000 7f7043cd1640 1 -- 172.21.15.172:0/265782111 >> 172.21.15.172:0/265782111 conn(0x7f703c09d580 msgr2=0x7f703c15fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:08.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.449+0000 7f7043cd1640 1 -- 172.21.15.172:0/265782111 shutdown_connections 2024-04-14T08:41:08.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.449+0000 7f7043cd1640 1 -- 172.21.15.172:0/265782111 wait complete. 2024-04-14T08:41:08.462 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-04-14T08:41:07.217726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260224,"kb_used_data":10780,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714496,"statfs":{"total":773094113280,"available":772827643904,"internally_reserved":0,"allocated":11038720,"data_stored":5722431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.089958"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162606+0000","last_change":"2024-04-14T08:41:05.156855+0000","last_active":"2024-04-14T08:41:05.162606+0000","last_peered":"2024-04-14T08:41:05.162606+0000","last_clean":"2024-04-14T08:41:05.162606+0000","last_became_active":"2024-04-14T08:41:05.156547+0000","last_became_peered":"2024-04-14T08:41:05.156547+0000","last_unstale":"2024-04-14T08:41:05.162606+0000","last_undegraded":"2024-04-14T08:41:05.162606+0000","last_fullsized":"2024-04-14T08:41:05.162606+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:20:39.314348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161756+0000","last_change":"2024-04-14T08:41:05.158746+0000","last_active":"2024-04-14T08:41:05.161756+0000","last_peered":"2024-04-14T08:41:05.161756+0000","last_clean":"2024-04-14T08:41:05.161756+0000","last_became_active":"2024-04-14T08:41:05.158296+0000","last_became_peered":"2024-04-14T08:41:05.158296+0000","last_unstale":"2024-04-14T08:41:05.161756+0000","last_undegraded":"2024-04-14T08:41:05.161756+0000","last_fullsized":"2024-04-14T08:41:05.161756+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:03:28.321597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166623+0000","last_change":"2024-04-14T08:41:05.163762+0000","last_active":"2024-04-14T08:41:05.166623+0000","last_peered":"2024-04-14T08:41:05.166623+0000","last_clean":"2024-04-14T08:41:05.166623+0000","last_became_active":"2024-04-14T08:41:05.163476+0000","last_became_peered":"2024-04-14T08:41:05.163476+0000","last_unstale":"2024-04-14T08:41:05.166623+0000","last_undegraded":"2024-04-14T08:41:05.166623+0000","last_fullsized":"2024-04-14T08:41:05.166623+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:41:09.337397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.165848+0000","last_change":"2024-04-14T08:41:05.163054+0000","last_active":"2024-04-14T08:41:05.165848+0000","last_peered":"2024-04-14T08:41:05.165848+0000","last_clean":"2024-04-14T08:41:05.165848+0000","last_became_active":"2024-04-14T08:41:05.162804+0000","last_became_peered":"2024-04-14T08:41:05.162804+0000","last_unstale":"2024-04-14T08:41:05.165848+0000","last_undegraded":"2024-04-14T08:41:05.165848+0000","last_fullsized":"2024-04-14T08:41:05.165848+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:36:36.291914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167319+0000","last_change":"2024-04-14T08:41:05.160856+0000","last_active":"2024-04-14T08:41:05.167319+0000","last_peered":"2024-04-14T08:41:05.167319+0000","last_clean":"2024-04-14T08:41:05.167319+0000","last_became_active":"2024-04-14T08:41:05.160576+0000","last_became_peered":"2024-04-14T08:41:05.160576+0000","last_unstale":"2024-04-14T08:41:05.167319+0000","last_undegraded":"2024-04-14T08:41:05.167319+0000","last_fullsized":"2024-04-14T08:41:05.167319+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:16:26.973756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170740+0000","last_change":"2024-04-14T08:41:05.167420+0000","last_active":"2024-04-14T08:41:05.170740+0000","last_peered":"2024-04-14T08:41:05.170740+0000","last_clean":"2024-04-14T08:41:05.170740+0000","last_became_active":"2024-04-14T08:41:05.167143+0000","last_became_peered":"2024-04-14T08:41:05.167143+0000","last_unstale":"2024-04-14T08:41:05.170740+0000","last_undegraded":"2024-04-14T08:41:05.170740+0000","last_fullsized":"2024-04-14T08:41:05.170740+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:27:59.897940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170729+0000","last_change":"2024-04-14T08:41:05.166977+0000","last_active":"2024-04-14T08:41:05.170729+0000","last_peered":"2024-04-14T08:41:05.170729+0000","last_clean":"2024-04-14T08:41:05.170729+0000","last_became_active":"2024-04-14T08:41:05.166674+0000","last_became_peered":"2024-04-14T08:41:05.166674+0000","last_unstale":"2024-04-14T08:41:05.170729+0000","last_undegraded":"2024-04-14T08:41:05.170729+0000","last_fullsized":"2024-04-14T08:41:05.170729+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:29:54.737024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.165700+0000","last_change":"2024-04-14T08:41:05.154902+0000","last_active":"2024-04-14T08:41:05.165700+0000","last_peered":"2024-04-14T08:41:05.165700+0000","last_clean":"2024-04-14T08:41:05.165700+0000","last_became_active":"2024-04-14T08:41:05.154621+0000","last_became_peered":"2024-04-14T08:41:05.154621+0000","last_unstale":"2024-04-14T08:41:05.165700+0000","last_undegraded":"2024-04-14T08:41:05.165700+0000","last_fullsized":"2024-04-14T08:41:05.165700+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:24:46.921747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162270+0000","last_change":"2024-04-14T08:41:05.157665+0000","last_active":"2024-04-14T08:41:05.162270+0000","last_peered":"2024-04-14T08:41:05.162270+0000","last_clean":"2024-04-14T08:41:05.162270+0000","last_became_active":"2024-04-14T08:41:05.157391+0000","last_became_peered":"2024-04-14T08:41:05.157391+0000","last_unstale":"2024-04-14T08:41:05.162270+0000","last_undegraded":"2024-04-14T08:41:05.162270+0000","last_fullsized":"2024-04-14T08:41:05.162270+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:13:53.752730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167422+0000","last_change":"2024-04-14T08:41:05.160560+0000","last_active":"2024-04-14T08:41:05.167422+0000","last_peered":"2024-04-14T08:41:05.167422+0000","last_clean":"2024-04-14T08:41:05.167422+0000","last_became_active":"2024-04-14T08:41:05.159424+0000","last_became_peered":"2024-04-14T08:41:05.159424+0000","last_unstale":"2024-04-14T08:41:05.167422+0000","last_undegraded":"2024-04-14T08:41:05.167422+0000","last_fullsized":"2024-04-14T08:41:05.167422+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:34:09.480206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167698+0000","last_change":"2024-04-14T08:41:05.163318+0000","last_active":"2024-04-14T08:41:05.167698+0000","last_peered":"2024-04-14T08:41:05.167698+0000","last_clean":"2024-04-14T08:41:05.167698+0000","last_became_active":"2024-04-14T08:41:05.162996+0000","last_became_peered":"2024-04-14T08:41:05.162996+0000","last_unstale":"2024-04-14T08:41:05.167698+0000","last_undegraded":"2024-04-14T08:41:05.167698+0000","last_fullsized":"2024-04-14T08:41:05.167698+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:05:58.415982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161956+0000","last_change":"2024-04-14T08:41:05.157945+0000","last_active":"2024-04-14T08:41:05.161956+0000","last_peered":"2024-04-14T08:41:05.161956+0000","last_clean":"2024-04-14T08:41:05.161956+0000","last_became_active":"2024-04-14T08:41:05.157669+0000","last_became_peered":"2024-04-14T08:41:05.157669+0000","last_unstale":"2024-04-14T08:41:05.161956+0000","last_undegraded":"2024-04-14T08:41:05.161956+0000","last_fullsized":"2024-04-14T08:41:05.161956+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:29:54.737024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164760+0000","last_change":"2024-04-14T08:41:05.161069+0000","last_active":"2024-04-14T08:41:05.164760+0000","last_peered":"2024-04-14T08:41:05.164760+0000","last_clean":"2024-04-14T08:41:05.164760+0000","last_became_active":"2024-04-14T08:41:05.160809+0000","last_became_peered":"2024-04-14T08:41:05.160809+0000","last_unstale":"2024-04-14T08:41:05.164760+0000","last_undegraded":"2024-04-14T08:41:05.164760+0000","last_fullsized":"2024-04-14T08:41:05.164760+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:29:54.737024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.150585+0000","last_change":"2024-04-14T08:41:05.150968+0000","last_active":"2024-04-14T08:41:06.150585+0000","last_peered":"2024-04-14T08:41:06.150585+0000","last_clean":"2024-04-14T08:41:06.150585+0000","last_became_active":"2024-04-14T08:41:05.150676+0000","last_became_peered":"2024-04-14T08:41:05.150676+0000","last_unstale":"2024-04-14T08:41:06.150585+0000","last_undegraded":"2024-04-14T08:41:06.150585+0000","last_fullsized":"2024-04-14T08:41:06.150585+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:31:00.277915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.150459+0000","last_change":"2024-04-14T08:41:05.161651+0000","last_active":"2024-04-14T08:41:06.150459+0000","last_peered":"2024-04-14T08:41:06.150459+0000","last_clean":"2024-04-14T08:41:06.150459+0000","last_became_active":"2024-04-14T08:41:05.161343+0000","last_became_peered":"2024-04-14T08:41:05.161343+0000","last_unstale":"2024-04-14T08:41:06.150459+0000","last_undegraded":"2024-04-14T08:41:06.150459+0000","last_fullsized":"2024-04-14T08:41:06.150459+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:16:41.150351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.150349+0000","last_change":"2024-04-14T08:41:05.151419+0000","last_active":"2024-04-14T08:41:06.150349+0000","last_peered":"2024-04-14T08:41:06.150349+0000","last_clean":"2024-04-14T08:41:06.150349+0000","last_became_active":"2024-04-14T08:41:05.151092+0000","last_became_peered":"2024-04-14T08:41:05.151092+0000","last_unstale":"2024-04-14T08:41:06.150349+0000","last_undegraded":"2024-04-14T08:41:06.150349+0000","last_fullsized":"2024-04-14T08:41:06.150349+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:40:18.013039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166494+0000","last_change":"2024-04-14T08:41:05.163930+0000","last_active":"2024-04-14T08:41:05.166494+0000","last_peered":"2024-04-14T08:41:05.166494+0000","last_clean":"2024-04-14T08:41:05.166494+0000","last_became_active":"2024-04-14T08:41:05.163673+0000","last_became_peered":"2024-04-14T08:41:05.163673+0000","last_unstale":"2024-04-14T08:41:05.166494+0000","last_undegraded":"2024-04-14T08:41:05.166494+0000","last_fullsized":"2024-04-14T08:41:05.166494+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:19:53.201897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170729+0000","last_change":"2024-04-14T08:41:05.166883+0000","last_active":"2024-04-14T08:41:05.170729+0000","last_peered":"2024-04-14T08:41:05.170729+0000","last_clean":"2024-04-14T08:41:05.170729+0000","last_became_active":"2024-04-14T08:41:05.166631+0000","last_became_peered":"2024-04-14T08:41:05.166631+0000","last_unstale":"2024-04-14T08:41:05.170729+0000","last_undegraded":"2024-04-14T08:41:05.170729+0000","last_fullsized":"2024-04-14T08:41:05.170729+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:23:53.194822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162783+0000","last_change":"2024-04-14T08:41:05.156842+0000","last_active":"2024-04-14T08:41:05.162783+0000","last_peered":"2024-04-14T08:41:05.162783+0000","last_clean":"2024-04-14T08:41:05.162783+0000","last_became_active":"2024-04-14T08:41:05.156580+0000","last_became_peered":"2024-04-14T08:41:05.156580+0000","last_unstale":"2024-04-14T08:41:05.162783+0000","last_undegraded":"2024-04-14T08:41:05.162783+0000","last_fullsized":"2024-04-14T08:41:05.162783+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:06:20.273023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161974+0000","last_change":"2024-04-14T08:41:05.158026+0000","last_active":"2024-04-14T08:41:05.161974+0000","last_peered":"2024-04-14T08:41:05.161974+0000","last_clean":"2024-04-14T08:41:05.161974+0000","last_became_active":"2024-04-14T08:41:05.157721+0000","last_became_peered":"2024-04-14T08:41:05.157721+0000","last_unstale":"2024-04-14T08:41:05.161974+0000","last_undegraded":"2024-04-14T08:41:05.161974+0000","last_fullsized":"2024-04-14T08:41:05.161974+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:27:59.897940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166081+0000","last_change":"2024-04-14T08:41:05.159179+0000","last_active":"2024-04-14T08:41:05.166081+0000","last_peered":"2024-04-14T08:41:05.166081+0000","last_clean":"2024-04-14T08:41:05.166081+0000","last_became_active":"2024-04-14T08:41:05.158920+0000","last_became_peered":"2024-04-14T08:41:05.158920+0000","last_unstale":"2024-04-14T08:41:05.166081+0000","last_undegraded":"2024-04-14T08:41:05.166081+0000","last_fullsized":"2024-04-14T08:41:05.166081+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:53.061286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167273+0000","last_change":"2024-04-14T08:41:05.162547+0000","last_active":"2024-04-14T08:41:05.167273+0000","last_peered":"2024-04-14T08:41:05.167273+0000","last_clean":"2024-04-14T08:41:05.167273+0000","last_became_active":"2024-04-14T08:41:05.162298+0000","last_became_peered":"2024-04-14T08:41:05.162298+0000","last_unstale":"2024-04-14T08:41:05.167273+0000","last_undegraded":"2024-04-14T08:41:05.167273+0000","last_fullsized":"2024-04-14T08:41:05.167273+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:13:53.752730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166436+0000","last_change":"2024-04-14T08:41:05.162802+0000","last_active":"2024-04-14T08:41:05.166436+0000","last_peered":"2024-04-14T08:41:05.166436+0000","last_clean":"2024-04-14T08:41:05.166436+0000","last_became_active":"2024-04-14T08:41:05.162529+0000","last_became_peered":"2024-04-14T08:41:05.162529+0000","last_unstale":"2024-04-14T08:41:05.166436+0000","last_undegraded":"2024-04-14T08:41:05.166436+0000","last_fullsized":"2024-04-14T08:41:05.166436+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:43:42.680257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170935+0000","last_change":"2024-04-14T08:41:05.165557+0000","last_active":"2024-04-14T08:41:05.170935+0000","last_peered":"2024-04-14T08:41:05.170935+0000","last_clean":"2024-04-14T08:41:05.170935+0000","last_became_active":"2024-04-14T08:41:05.165285+0000","last_became_peered":"2024-04-14T08:41:05.165285+0000","last_unstale":"2024-04-14T08:41:05.170935+0000","last_undegraded":"2024-04-14T08:41:05.170935+0000","last_fullsized":"2024-04-14T08:41:05.170935+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:05:58.415982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170597+0000","last_change":"2024-04-14T08:41:05.166693+0000","last_active":"2024-04-14T08:41:05.170597+0000","last_peered":"2024-04-14T08:41:05.170597+0000","last_clean":"2024-04-14T08:41:05.170597+0000","last_became_active":"2024-04-14T08:41:05.165164+0000","last_became_peered":"2024-04-14T08:41:05.165164+0000","last_unstale":"2024-04-14T08:41:05.170597+0000","last_undegraded":"2024-04-14T08:41:05.170597+0000","last_fullsized":"2024-04-14T08:41:05.170597+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:41:29.147856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164295+0000","last_change":"2024-04-14T08:41:05.160757+0000","last_active":"2024-04-14T08:41:05.164295+0000","last_peered":"2024-04-14T08:41:05.164295+0000","last_clean":"2024-04-14T08:41:05.164295+0000","last_became_active":"2024-04-14T08:41:05.160438+0000","last_became_peered":"2024-04-14T08:41:05.160438+0000","last_unstale":"2024-04-14T08:41:05.164295+0000","last_undegraded":"2024-04-14T08:41:05.164295+0000","last_fullsized":"2024-04-14T08:41:05.164295+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:27:32.491311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167721+0000","last_change":"2024-04-14T08:41:05.163760+0000","last_active":"2024-04-14T08:41:05.167721+0000","last_peered":"2024-04-14T08:41:05.167721+0000","last_clean":"2024-04-14T08:41:05.167721+0000","last_became_active":"2024-04-14T08:41:05.163154+0000","last_became_peered":"2024-04-14T08:41:05.163154+0000","last_unstale":"2024-04-14T08:41:05.167721+0000","last_undegraded":"2024-04-14T08:41:05.167721+0000","last_fullsized":"2024-04-14T08:41:05.167721+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:02:07.977713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162633+0000","last_change":"2024-04-14T08:41:05.158599+0000","last_active":"2024-04-14T08:41:05.162633+0000","last_peered":"2024-04-14T08:41:05.162633+0000","last_clean":"2024-04-14T08:41:05.162633+0000","last_became_active":"2024-04-14T08:41:05.158282+0000","last_became_peered":"2024-04-14T08:41:05.158282+0000","last_unstale":"2024-04-14T08:41:05.162633+0000","last_undegraded":"2024-04-14T08:41:05.162633+0000","last_fullsized":"2024-04-14T08:41:05.162633+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:05:58.415982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.163801+0000","last_change":"2024-04-14T08:41:05.152092+0000","last_active":"2024-04-14T08:41:05.163801+0000","last_peered":"2024-04-14T08:41:05.163801+0000","last_clean":"2024-04-14T08:41:05.163801+0000","last_became_active":"2024-04-14T08:41:05.151835+0000","last_became_peered":"2024-04-14T08:41:05.151835+0000","last_unstale":"2024-04-14T08:41:05.163801+0000","last_undegraded":"2024-04-14T08:41:05.163801+0000","last_fullsized":"2024-04-14T08:41:05.163801+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:30:25.299745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162000+0000","last_change":"2024-04-14T08:41:05.158723+0000","last_active":"2024-04-14T08:41:05.162000+0000","last_peered":"2024-04-14T08:41:05.162000+0000","last_clean":"2024-04-14T08:41:05.162000+0000","last_became_active":"2024-04-14T08:41:05.158454+0000","last_became_peered":"2024-04-14T08:41:05.158454+0000","last_unstale":"2024-04-14T08:41:05.162000+0000","last_undegraded":"2024-04-14T08:41:05.162000+0000","last_fullsized":"2024-04-14T08:41:05.162000+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:36:36.291914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170666+0000","last_change":"2024-04-14T08:41:05.165822+0000","last_active":"2024-04-14T08:41:05.170666+0000","last_peered":"2024-04-14T08:41:05.170666+0000","last_clean":"2024-04-14T08:41:05.170666+0000","last_became_active":"2024-04-14T08:41:05.165232+0000","last_became_peered":"2024-04-14T08:41:05.165232+0000","last_unstale":"2024-04-14T08:41:05.170666+0000","last_undegraded":"2024-04-14T08:41:05.170666+0000","last_fullsized":"2024-04-14T08:41:05.170666+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:47:10.299310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166337+0000","last_change":"2024-04-14T08:41:05.163181+0000","last_active":"2024-04-14T08:41:05.166337+0000","last_peered":"2024-04-14T08:41:05.166337+0000","last_clean":"2024-04-14T08:41:05.166337+0000","last_became_active":"2024-04-14T08:41:05.162908+0000","last_became_peered":"2024-04-14T08:41:05.162908+0000","last_unstale":"2024-04-14T08:41:05.166337+0000","last_undegraded":"2024-04-14T08:41:05.166337+0000","last_fullsized":"2024-04-14T08:41:05.166337+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:28:05.473331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162819+0000","last_change":"2024-04-14T08:41:05.153004+0000","last_active":"2024-04-14T08:41:05.162819+0000","last_peered":"2024-04-14T08:41:05.162819+0000","last_clean":"2024-04-14T08:41:05.162819+0000","last_became_active":"2024-04-14T08:41:05.152620+0000","last_became_peered":"2024-04-14T08:41:05.152620+0000","last_unstale":"2024-04-14T08:41:05.162819+0000","last_undegraded":"2024-04-14T08:41:05.162819+0000","last_fullsized":"2024-04-14T08:41:05.162819+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:05:21.134598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"100'4","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164264+0000","last_change":"2024-04-14T08:41:04.154347+0000","last_active":"2024-04-14T08:41:05.164264+0000","last_peered":"2024-04-14T08:41:05.164264+0000","last_clean":"2024-04-14T08:41:05.164264+0000","last_became_active":"2024-04-14T08:41:04.154039+0000","last_became_peered":"2024-04-14T08:41:04.154039+0000","last_unstale":"2024-04-14T08:41:05.164264+0000","last_undegraded":"2024-04-14T08:41:05.164264+0000","last_fullsized":"2024-04-14T08:41:05.164264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:02:06.973563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.165854+0000","last_change":"2024-04-14T08:41:05.155717+0000","last_active":"2024-04-14T08:41:05.165854+0000","last_peered":"2024-04-14T08:41:05.165854+0000","last_clean":"2024-04-14T08:41:05.165854+0000","last_became_active":"2024-04-14T08:41:05.155466+0000","last_became_peered":"2024-04-14T08:41:05.155466+0000","last_unstale":"2024-04-14T08:41:05.165854+0000","last_undegraded":"2024-04-14T08:41:05.165854+0000","last_fullsized":"2024-04-14T08:41:05.165854+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:51:19.253762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"100'1","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.168300+0000","last_change":"2024-04-14T08:41:04.149699+0000","last_active":"2024-04-14T08:41:05.168300+0000","last_peered":"2024-04-14T08:41:05.168300+0000","last_clean":"2024-04-14T08:41:05.168300+0000","last_became_active":"2024-04-14T08:41:04.149002+0000","last_became_peered":"2024-04-14T08:41:04.149002+0000","last_unstale":"2024-04-14T08:41:05.168300+0000","last_undegraded":"2024-04-14T08:41:05.168300+0000","last_fullsized":"2024-04-14T08:41:05.168300+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:19:25.085498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161960+0000","last_change":"2024-04-14T08:41:05.156669+0000","last_active":"2024-04-14T08:41:05.161960+0000","last_peered":"2024-04-14T08:41:05.161960+0000","last_clean":"2024-04-14T08:41:05.161960+0000","last_became_active":"2024-04-14T08:41:05.156397+0000","last_became_peered":"2024-04-14T08:41:05.156397+0000","last_unstale":"2024-04-14T08:41:05.161960+0000","last_undegraded":"2024-04-14T08:41:05.161960+0000","last_fullsized":"2024-04-14T08:41:05.161960+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:47:10.299310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"100'2","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.159125+0000","last_change":"2024-04-14T08:41:04.151445+0000","last_active":"2024-04-14T08:41:05.159125+0000","last_peered":"2024-04-14T08:41:05.159125+0000","last_clean":"2024-04-14T08:41:05.159125+0000","last_became_active":"2024-04-14T08:41:04.151195+0000","last_became_peered":"2024-04-14T08:41:04.151195+0000","last_unstale":"2024-04-14T08:41:05.159125+0000","last_undegraded":"2024-04-14T08:41:05.159125+0000","last_fullsized":"2024-04-14T08:41:05.159125+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:05:57.411833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.168280+0000","last_change":"2024-04-14T08:41:05.164986+0000","last_active":"2024-04-14T08:41:05.168280+0000","last_peered":"2024-04-14T08:41:05.168280+0000","last_clean":"2024-04-14T08:41:05.168280+0000","last_became_active":"2024-04-14T08:41:05.164551+0000","last_became_peered":"2024-04-14T08:41:05.164551+0000","last_unstale":"2024-04-14T08:41:05.168280+0000","last_undegraded":"2024-04-14T08:41:05.168280+0000","last_fullsized":"2024-04-14T08:41:05.168280+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:45:47.080994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"100'2","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167153+0000","last_change":"2024-04-14T08:41:04.158464+0000","last_active":"2024-04-14T08:41:05.167153+0000","last_peered":"2024-04-14T08:41:05.167153+0000","last_clean":"2024-04-14T08:41:05.167153+0000","last_became_active":"2024-04-14T08:41:04.158165+0000","last_became_peered":"2024-04-14T08:41:04.158165+0000","last_unstale":"2024-04-14T08:41:05.167153+0000","last_undegraded":"2024-04-14T08:41:05.167153+0000","last_fullsized":"2024-04-14T08:41:05.167153+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:34:32.173549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.168403+0000","last_change":"2024-04-14T08:41:05.167654+0000","last_active":"2024-04-14T08:41:05.168403+0000","last_peered":"2024-04-14T08:41:05.168403+0000","last_clean":"2024-04-14T08:41:05.168403+0000","last_became_active":"2024-04-14T08:41:05.167339+0000","last_became_peered":"2024-04-14T08:41:05.167339+0000","last_unstale":"2024-04-14T08:41:05.168403+0000","last_undegraded":"2024-04-14T08:41:05.168403+0000","last_fullsized":"2024-04-14T08:41:05.168403+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:27:32.491311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.150873+0000","last_change":"2024-04-14T08:41:04.149377+0000","last_active":"2024-04-14T08:41:06.150873+0000","last_peered":"2024-04-14T08:41:06.150873+0000","last_clean":"2024-04-14T08:41:06.150873+0000","last_became_active":"2024-04-14T08:41:04.149116+0000","last_became_peered":"2024-04-14T08:41:04.149116+0000","last_unstale":"2024-04-14T08:41:06.150873+0000","last_undegraded":"2024-04-14T08:41:06.150873+0000","last_fullsized":"2024-04-14T08:41:06.150873+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:30:10.017018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162759+0000","last_change":"2024-04-14T08:41:05.156286+0000","last_active":"2024-04-14T08:41:05.162759+0000","last_peered":"2024-04-14T08:41:05.162759+0000","last_clean":"2024-04-14T08:41:05.162759+0000","last_became_active":"2024-04-14T08:41:05.155982+0000","last_became_peered":"2024-04-14T08:41:05.155982+0000","last_unstale":"2024-04-14T08:41:05.162759+0000","last_undegraded":"2024-04-14T08:41:05.162759+0000","last_fullsized":"2024-04-14T08:41:05.162759+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:02:00.221376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"100'1","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164764+0000","last_change":"2024-04-14T08:41:04.154092+0000","last_active":"2024-04-14T08:41:05.164764+0000","last_peered":"2024-04-14T08:41:05.164764+0000","last_clean":"2024-04-14T08:41:05.164764+0000","last_became_active":"2024-04-14T08:41:04.153804+0000","last_became_peered":"2024-04-14T08:41:04.153804+0000","last_unstale":"2024-04-14T08:41:05.164764+0000","last_undegraded":"2024-04-14T08:41:05.164764+0000","last_fullsized":"2024-04-14T08:41:05.164764+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:34:08.476057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161777+0000","last_change":"2024-04-14T08:41:05.159308+0000","last_active":"2024-04-14T08:41:05.161777+0000","last_peered":"2024-04-14T08:41:05.161777+0000","last_clean":"2024-04-14T08:41:05.161777+0000","last_became_active":"2024-04-14T08:41:05.158909+0000","last_became_peered":"2024-04-14T08:41:05.158909+0000","last_unstale":"2024-04-14T08:41:05.161777+0000","last_undegraded":"2024-04-14T08:41:05.161777+0000","last_fullsized":"2024-04-14T08:41:05.161777+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:02:07.977713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.142295+0000","last_change":"2024-04-14T08:41:04.151411+0000","last_active":"2024-04-14T08:41:05.142295+0000","last_peered":"2024-04-14T08:41:05.142295+0000","last_clean":"2024-04-14T08:41:05.142295+0000","last_became_active":"2024-04-14T08:41:04.151049+0000","last_became_peered":"2024-04-14T08:41:04.151049+0000","last_unstale":"2024-04-14T08:41:05.142295+0000","last_undegraded":"2024-04-14T08:41:05.142295+0000","last_fullsized":"2024-04-14T08:41:05.142295+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:02:06.973563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.160071+0000","last_change":"2024-04-14T08:41:05.157697+0000","last_active":"2024-04-14T08:41:05.160071+0000","last_peered":"2024-04-14T08:41:05.160071+0000","last_clean":"2024-04-14T08:41:05.160071+0000","last_became_active":"2024-04-14T08:41:05.157446+0000","last_became_peered":"2024-04-14T08:41:05.157446+0000","last_unstale":"2024-04-14T08:41:05.160071+0000","last_undegraded":"2024-04-14T08:41:05.160071+0000","last_fullsized":"2024-04-14T08:41:05.160071+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:47:10.299310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.143653+0000","last_change":"2024-04-14T08:41:04.155786+0000","last_active":"2024-04-14T08:41:05.143653+0000","last_peered":"2024-04-14T08:41:05.143653+0000","last_clean":"2024-04-14T08:41:05.143653+0000","last_became_active":"2024-04-14T08:41:04.155469+0000","last_became_peered":"2024-04-14T08:41:04.155469+0000","last_unstale":"2024-04-14T08:41:05.143653+0000","last_undegraded":"2024-04-14T08:41:05.143653+0000","last_fullsized":"2024-04-14T08:41:05.143653+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:03:27.317448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162875+0000","last_change":"2024-04-14T08:41:05.156486+0000","last_active":"2024-04-14T08:41:05.162875+0000","last_peered":"2024-04-14T08:41:05.162875+0000","last_clean":"2024-04-14T08:41:05.162875+0000","last_became_active":"2024-04-14T08:41:05.156145+0000","last_became_peered":"2024-04-14T08:41:05.156145+0000","last_unstale":"2024-04-14T08:41:05.162875+0000","last_undegraded":"2024-04-14T08:41:05.162875+0000","last_fullsized":"2024-04-14T08:41:05.162875+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:54:28.030276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.141602+0000","last_change":"2024-04-14T08:41:04.152113+0000","last_active":"2024-04-14T08:41:05.141602+0000","last_peered":"2024-04-14T08:41:05.141602+0000","last_clean":"2024-04-14T08:41:05.141602+0000","last_became_active":"2024-04-14T08:41:04.151821+0000","last_became_peered":"2024-04-14T08:41:04.151821+0000","last_unstale":"2024-04-14T08:41:05.141602+0000","last_undegraded":"2024-04-14T08:41:05.141602+0000","last_fullsized":"2024-04-14T08:41:05.141602+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:03:27.317448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166949+0000","last_change":"2024-04-14T08:41:05.154435+0000","last_active":"2024-04-14T08:41:05.166949+0000","last_peered":"2024-04-14T08:41:05.166949+0000","last_clean":"2024-04-14T08:41:05.166949+0000","last_became_active":"2024-04-14T08:41:05.154144+0000","last_became_peered":"2024-04-14T08:41:05.154144+0000","last_unstale":"2024-04-14T08:41:05.166949+0000","last_undegraded":"2024-04-14T08:41:05.166949+0000","last_fullsized":"2024-04-14T08:41:05.166949+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:23:54.351934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.144059+0000","last_change":"2024-04-14T08:41:04.159556+0000","last_active":"2024-04-14T08:41:05.144059+0000","last_peered":"2024-04-14T08:41:05.144059+0000","last_clean":"2024-04-14T08:41:05.144059+0000","last_became_active":"2024-04-14T08:41:04.159323+0000","last_became_peered":"2024-04-14T08:41:04.159323+0000","last_unstale":"2024-04-14T08:41:05.144059+0000","last_undegraded":"2024-04-14T08:41:05.144059+0000","last_fullsized":"2024-04-14T08:41:05.144059+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:18:51.764627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.165954+0000","last_change":"2024-04-14T08:41:05.156838+0000","last_active":"2024-04-14T08:41:05.165954+0000","last_peered":"2024-04-14T08:41:05.165954+0000","last_clean":"2024-04-14T08:41:05.165954+0000","last_became_active":"2024-04-14T08:41:05.156207+0000","last_became_peered":"2024-04-14T08:41:05.156207+0000","last_unstale":"2024-04-14T08:41:05.165954+0000","last_undegraded":"2024-04-14T08:41:05.165954+0000","last_fullsized":"2024-04-14T08:41:05.165954+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:33:37.146832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"100'3","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.150940+0000","last_change":"2024-04-14T08:41:04.156229+0000","last_active":"2024-04-14T08:41:06.150940+0000","last_peered":"2024-04-14T08:41:06.150940+0000","last_clean":"2024-04-14T08:41:06.150940+0000","last_became_active":"2024-04-14T08:41:04.155893+0000","last_became_peered":"2024-04-14T08:41:04.155893+0000","last_unstale":"2024-04-14T08:41:06.150940+0000","last_undegraded":"2024-04-14T08:41:06.150940+0000","last_fullsized":"2024-04-14T08:41:06.150940+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:41:28.143707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167011+0000","last_change":"2024-04-14T08:41:05.157846+0000","last_active":"2024-04-14T08:41:05.167011+0000","last_peered":"2024-04-14T08:41:05.167011+0000","last_clean":"2024-04-14T08:41:05.167011+0000","last_became_active":"2024-04-14T08:41:05.157476+0000","last_became_peered":"2024-04-14T08:41:05.157476+0000","last_unstale":"2024-04-14T08:41:05.167011+0000","last_undegraded":"2024-04-14T08:41:05.167011+0000","last_fullsized":"2024-04-14T08:41:05.167011+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:04:51.866864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"100'6","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170697+0000","last_change":"2024-04-14T08:41:04.159163+0000","last_active":"2024-04-14T08:41:05.170697+0000","last_peered":"2024-04-14T08:41:05.170697+0000","last_clean":"2024-04-14T08:41:05.170697+0000","last_became_active":"2024-04-14T08:41:04.158859+0000","last_became_peered":"2024-04-14T08:41:04.158859+0000","last_unstale":"2024-04-14T08:41:05.170697+0000","last_undegraded":"2024-04-14T08:41:05.170697+0000","last_fullsized":"2024-04-14T08:41:05.170697+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:23:53.347784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164682+0000","last_change":"2024-04-14T08:41:05.162043+0000","last_active":"2024-04-14T08:41:05.164682+0000","last_peered":"2024-04-14T08:41:05.164682+0000","last_clean":"2024-04-14T08:41:05.164682+0000","last_became_active":"2024-04-14T08:41:05.161757+0000","last_became_peered":"2024-04-14T08:41:05.161757+0000","last_unstale":"2024-04-14T08:41:05.164682+0000","last_undegraded":"2024-04-14T08:41:05.164682+0000","last_fullsized":"2024-04-14T08:41:05.164682+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:13:53.752730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"100'2","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.169032+0000","last_change":"2024-04-14T08:41:04.153066+0000","last_active":"2024-04-14T08:41:05.169032+0000","last_peered":"2024-04-14T08:41:05.169032+0000","last_clean":"2024-04-14T08:41:05.169032+0000","last_became_active":"2024-04-14T08:41:04.152797+0000","last_became_peered":"2024-04-14T08:41:04.152797+0000","last_unstale":"2024-04-14T08:41:05.169032+0000","last_undegraded":"2024-04-14T08:41:05.169032+0000","last_fullsized":"2024-04-14T08:41:05.169032+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:34.256219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161157+0000","last_change":"2024-04-14T08:41:05.156664+0000","last_active":"2024-04-14T08:41:05.161157+0000","last_peered":"2024-04-14T08:41:05.161157+0000","last_clean":"2024-04-14T08:41:05.161157+0000","last_became_active":"2024-04-14T08:41:05.156339+0000","last_became_peered":"2024-04-14T08:41:05.156339+0000","last_unstale":"2024-04-14T08:41:05.161157+0000","last_undegraded":"2024-04-14T08:41:05.161157+0000","last_fullsized":"2024-04-14T08:41:05.161157+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:16:26.973756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.145564+0000","last_change":"2024-04-14T08:41:04.148022+0000","last_active":"2024-04-14T08:41:05.145564+0000","last_peered":"2024-04-14T08:41:05.145564+0000","last_clean":"2024-04-14T08:41:05.145564+0000","last_became_active":"2024-04-14T08:41:04.147766+0000","last_became_peered":"2024-04-14T08:41:04.147766+0000","last_unstale":"2024-04-14T08:41:05.145564+0000","last_undegraded":"2024-04-14T08:41:05.145564+0000","last_fullsized":"2024-04-14T08:41:05.145564+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:45:46.076845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166420+0000","last_change":"2024-04-14T08:41:05.163876+0000","last_active":"2024-04-14T08:41:05.166420+0000","last_peered":"2024-04-14T08:41:05.166420+0000","last_clean":"2024-04-14T08:41:05.166420+0000","last_became_active":"2024-04-14T08:41:05.163583+0000","last_became_peered":"2024-04-14T08:41:05.163583+0000","last_unstale":"2024-04-14T08:41:05.166420+0000","last_undegraded":"2024-04-14T08:41:05.166420+0000","last_fullsized":"2024-04-14T08:41:05.166420+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:29:18.350492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.139488+0000","last_change":"2024-04-14T08:41:04.148341+0000","last_active":"2024-04-14T08:41:05.139488+0000","last_peered":"2024-04-14T08:41:05.139488+0000","last_clean":"2024-04-14T08:41:05.139488+0000","last_became_active":"2024-04-14T08:41:04.147735+0000","last_became_peered":"2024-04-14T08:41:04.147735+0000","last_unstale":"2024-04-14T08:41:05.139488+0000","last_undegraded":"2024-04-14T08:41:05.139488+0000","last_fullsized":"2024-04-14T08:41:05.139488+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:27:17.741937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166512+0000","last_change":"2024-04-14T08:41:05.164591+0000","last_active":"2024-04-14T08:41:05.166512+0000","last_peered":"2024-04-14T08:41:05.166512+0000","last_clean":"2024-04-14T08:41:05.166512+0000","last_became_active":"2024-04-14T08:41:05.164310+0000","last_became_peered":"2024-04-14T08:41:05.164310+0000","last_unstale":"2024-04-14T08:41:05.166512+0000","last_undegraded":"2024-04-14T08:41:05.166512+0000","last_fullsized":"2024-04-14T08:41:05.166512+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:19:19.705118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"100'8","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.159647+0000","last_change":"2024-04-14T08:41:04.152016+0000","last_active":"2024-04-14T08:41:05.159647+0000","last_peered":"2024-04-14T08:41:05.159647+0000","last_clean":"2024-04-14T08:41:05.159647+0000","last_became_active":"2024-04-14T08:41:04.151760+0000","last_became_peered":"2024-04-14T08:41:04.151760+0000","last_unstale":"2024-04-14T08:41:05.159647+0000","last_undegraded":"2024-04-14T08:41:05.159647+0000","last_fullsized":"2024-04-14T08:41:05.159647+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:45:46.076845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.154546+0000","last_change":"2024-04-14T08:41:05.149470+0000","last_active":"2024-04-14T08:41:05.154546+0000","last_peered":"2024-04-14T08:41:05.154546+0000","last_clean":"2024-04-14T08:41:05.154546+0000","last_became_active":"2024-04-14T08:41:05.149175+0000","last_became_peered":"2024-04-14T08:41:05.149175+0000","last_unstale":"2024-04-14T08:41:05.154546+0000","last_undegraded":"2024-04-14T08:41:05.154546+0000","last_fullsized":"2024-04-14T08:41:05.154546+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:17:47.808646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"100'1","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164513+0000","last_change":"2024-04-14T08:41:04.149516+0000","last_active":"2024-04-14T08:41:05.164513+0000","last_peered":"2024-04-14T08:41:05.164513+0000","last_clean":"2024-04-14T08:41:05.164513+0000","last_became_active":"2024-04-14T08:41:04.149238+0000","last_became_peered":"2024-04-14T08:41:04.149238+0000","last_unstale":"2024-04-14T08:41:05.164513+0000","last_undegraded":"2024-04-14T08:41:05.164513+0000","last_fullsized":"2024-04-14T08:41:05.164513+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:34:08.476057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170589+0000","last_change":"2024-04-14T08:41:05.165446+0000","last_active":"2024-04-14T08:41:05.170589+0000","last_peered":"2024-04-14T08:41:05.170589+0000","last_clean":"2024-04-14T08:41:05.170589+0000","last_became_active":"2024-04-14T08:41:05.165161+0000","last_became_peered":"2024-04-14T08:41:05.165161+0000","last_unstale":"2024-04-14T08:41:05.170589+0000","last_undegraded":"2024-04-14T08:41:05.170589+0000","last_fullsized":"2024-04-14T08:41:05.170589+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:45:47.080994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.151078+0000","last_change":"2024-04-14T08:41:04.147164+0000","last_active":"2024-04-14T08:41:06.151078+0000","last_peered":"2024-04-14T08:41:06.151078+0000","last_clean":"2024-04-14T08:41:06.151078+0000","last_became_active":"2024-04-14T08:41:04.146880+0000","last_became_peered":"2024-04-14T08:41:04.146880+0000","last_unstale":"2024-04-14T08:41:06.151078+0000","last_undegraded":"2024-04-14T08:41:06.151078+0000","last_fullsized":"2024-04-14T08:41:06.151078+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:39:32.284645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167139+0000","last_change":"2024-04-14T08:41:05.159342+0000","last_active":"2024-04-14T08:41:05.167139+0000","last_peered":"2024-04-14T08:41:05.167139+0000","last_clean":"2024-04-14T08:41:05.167139+0000","last_became_active":"2024-04-14T08:41:05.159107+0000","last_became_peered":"2024-04-14T08:41:05.159107+0000","last_unstale":"2024-04-14T08:41:05.167139+0000","last_undegraded":"2024-04-14T08:41:05.167139+0000","last_fullsized":"2024-04-14T08:41:05.167139+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:21:20.731388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.157172+0000","last_change":"2024-04-14T08:41:04.157007+0000","last_active":"2024-04-14T08:41:05.157172+0000","last_peered":"2024-04-14T08:41:05.157172+0000","last_clean":"2024-04-14T08:41:05.157172+0000","last_became_active":"2024-04-14T08:41:04.156742+0000","last_became_peered":"2024-04-14T08:41:04.156742+0000","last_unstale":"2024-04-14T08:41:05.157172+0000","last_undegraded":"2024-04-14T08:41:05.157172+0000","last_fullsized":"2024-04-14T08:41:05.157172+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:16:25.969607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.159641+0000","last_change":"2024-04-14T08:41:05.152608+0000","last_active":"2024-04-14T08:41:05.159641+0000","last_peered":"2024-04-14T08:41:05.159641+0000","last_clean":"2024-04-14T08:41:05.159641+0000","last_became_active":"2024-04-14T08:41:05.151471+0000","last_became_peered":"2024-04-14T08:41:05.151471+0000","last_unstale":"2024-04-14T08:41:05.159641+0000","last_undegraded":"2024-04-14T08:41:05.159641+0000","last_fullsized":"2024-04-14T08:41:05.159641+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:03:31.058051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"100'2","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167130+0000","last_change":"2024-04-14T08:41:04.156830+0000","last_active":"2024-04-14T08:41:05.167130+0000","last_peered":"2024-04-14T08:41:05.167130+0000","last_clean":"2024-04-14T08:41:05.167130+0000","last_became_active":"2024-04-14T08:41:04.156558+0000","last_became_peered":"2024-04-14T08:41:04.156558+0000","last_unstale":"2024-04-14T08:41:05.167130+0000","last_undegraded":"2024-04-14T08:41:05.167130+0000","last_fullsized":"2024-04-14T08:41:05.167130+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:23:28.198182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.168591+0000","last_change":"2024-04-14T08:41:05.166489+0000","last_active":"2024-04-14T08:41:05.168591+0000","last_peered":"2024-04-14T08:41:05.168591+0000","last_clean":"2024-04-14T08:41:05.168591+0000","last_became_active":"2024-04-14T08:41:05.165706+0000","last_became_peered":"2024-04-14T08:41:05.165706+0000","last_unstale":"2024-04-14T08:41:05.168591+0000","last_undegraded":"2024-04-14T08:41:05.168591+0000","last_fullsized":"2024-04-14T08:41:05.168591+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:31:05.489822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.139142+0000","last_change":"2024-04-14T08:41:04.147901+0000","last_active":"2024-04-14T08:41:05.139142+0000","last_peered":"2024-04-14T08:41:05.139142+0000","last_clean":"2024-04-14T08:41:05.139142+0000","last_became_active":"2024-04-14T08:41:04.147625+0000","last_became_peered":"2024-04-14T08:41:04.147625+0000","last_unstale":"2024-04-14T08:41:05.139142+0000","last_undegraded":"2024-04-14T08:41:05.139142+0000","last_fullsized":"2024-04-14T08:41:05.139142+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:24.483825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.164730+0000","last_change":"2024-04-14T08:41:05.161588+0000","last_active":"2024-04-14T08:41:05.164730+0000","last_peered":"2024-04-14T08:41:05.164730+0000","last_clean":"2024-04-14T08:41:05.164730+0000","last_became_active":"2024-04-14T08:41:05.161359+0000","last_became_peered":"2024-04-14T08:41:05.161359+0000","last_unstale":"2024-04-14T08:41:05.164730+0000","last_undegraded":"2024-04-14T08:41:05.164730+0000","last_fullsized":"2024-04-14T08:41:05.164730+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:27:59.897940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.148925+0000","last_change":"2024-04-14T08:41:04.153035+0000","last_active":"2024-04-14T08:41:05.148925+0000","last_peered":"2024-04-14T08:41:05.148925+0000","last_clean":"2024-04-14T08:41:05.148925+0000","last_became_active":"2024-04-14T08:41:04.152814+0000","last_became_peered":"2024-04-14T08:41:04.152814+0000","last_unstale":"2024-04-14T08:41:05.148925+0000","last_undegraded":"2024-04-14T08:41:05.148925+0000","last_fullsized":"2024-04-14T08:41:05.148925+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:04:50.862715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.165705+0000","last_change":"2024-04-14T08:41:05.155051+0000","last_active":"2024-04-14T08:41:05.165705+0000","last_peered":"2024-04-14T08:41:05.165705+0000","last_clean":"2024-04-14T08:41:05.165705+0000","last_became_active":"2024-04-14T08:41:05.154333+0000","last_became_peered":"2024-04-14T08:41:05.154333+0000","last_unstale":"2024-04-14T08:41:05.165705+0000","last_undegraded":"2024-04-14T08:41:05.165705+0000","last_fullsized":"2024-04-14T08:41:05.165705+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:15:49.596490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.139956+0000","last_change":"2024-04-14T08:41:04.151672+0000","last_active":"2024-04-14T08:41:05.139956+0000","last_peered":"2024-04-14T08:41:05.139956+0000","last_clean":"2024-04-14T08:41:05.139956+0000","last_became_active":"2024-04-14T08:41:04.151356+0000","last_became_peered":"2024-04-14T08:41:04.151356+0000","last_unstale":"2024-04-14T08:41:05.139956+0000","last_undegraded":"2024-04-14T08:41:05.139956+0000","last_fullsized":"2024-04-14T08:41:05.139956+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:50:47.507145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.162856+0000","last_change":"2024-04-14T08:41:05.155529+0000","last_active":"2024-04-14T08:41:05.162856+0000","last_peered":"2024-04-14T08:41:05.162856+0000","last_clean":"2024-04-14T08:41:05.162856+0000","last_became_active":"2024-04-14T08:41:05.155133+0000","last_became_peered":"2024-04-14T08:41:05.155133+0000","last_unstale":"2024-04-14T08:41:05.162856+0000","last_undegraded":"2024-04-14T08:41:05.162856+0000","last_fullsized":"2024-04-14T08:41:05.162856+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:36:53.910896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"100'2","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.168261+0000","last_change":"2024-04-14T08:41:04.153068+0000","last_active":"2024-04-14T08:41:05.168261+0000","last_peered":"2024-04-14T08:41:05.168261+0000","last_clean":"2024-04-14T08:41:05.168261+0000","last_became_active":"2024-04-14T08:41:04.152818+0000","last_became_peered":"2024-04-14T08:41:04.152818+0000","last_unstale":"2024-04-14T08:41:05.168261+0000","last_undegraded":"2024-04-14T08:41:05.168261+0000","last_fullsized":"2024-04-14T08:41:05.168261+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:09:50.573062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.161918+0000","last_change":"2024-04-14T08:41:05.157262+0000","last_active":"2024-04-14T08:41:05.161918+0000","last_peered":"2024-04-14T08:41:05.161918+0000","last_clean":"2024-04-14T08:41:05.161918+0000","last_became_active":"2024-04-14T08:41:05.157005+0000","last_became_peered":"2024-04-14T08:41:05.157005+0000","last_unstale":"2024-04-14T08:41:05.161918+0000","last_undegraded":"2024-04-14T08:41:05.161918+0000","last_fullsized":"2024-04-14T08:41:05.161918+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:23:53.194822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.144264+0000","last_change":"2024-04-14T08:41:04.159621+0000","last_active":"2024-04-14T08:41:05.144264+0000","last_peered":"2024-04-14T08:41:05.144264+0000","last_clean":"2024-04-14T08:41:05.144264+0000","last_became_active":"2024-04-14T08:41:04.159445+0000","last_became_peered":"2024-04-14T08:41:04.159445+0000","last_unstale":"2024-04-14T08:41:05.144264+0000","last_undegraded":"2024-04-14T08:41:05.144264+0000","last_fullsized":"2024-04-14T08:41:05.144264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:36:27.884380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.160388+0000","last_change":"2024-04-14T08:41:05.158755+0000","last_active":"2024-04-14T08:41:05.160388+0000","last_peered":"2024-04-14T08:41:05.160388+0000","last_clean":"2024-04-14T08:41:05.160388+0000","last_became_active":"2024-04-14T08:41:05.158493+0000","last_became_peered":"2024-04-14T08:41:05.158493+0000","last_unstale":"2024-04-14T08:41:05.160388+0000","last_undegraded":"2024-04-14T08:41:05.160388+0000","last_fullsized":"2024-04-14T08:41:05.160388+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:23:53.194822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.148691+0000","last_change":"2024-04-14T08:41:04.149338+0000","last_active":"2024-04-14T08:41:05.148691+0000","last_peered":"2024-04-14T08:41:05.148691+0000","last_clean":"2024-04-14T08:41:05.148691+0000","last_became_active":"2024-04-14T08:41:04.148983+0000","last_became_peered":"2024-04-14T08:41:04.148983+0000","last_unstale":"2024-04-14T08:41:05.148691+0000","last_undegraded":"2024-04-14T08:41:05.148691+0000","last_fullsized":"2024-04-14T08:41:05.148691+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:46:34.256219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.170691+0000","last_change":"2024-04-14T08:41:05.167482+0000","last_active":"2024-04-14T08:41:05.170691+0000","last_peered":"2024-04-14T08:41:05.170691+0000","last_clean":"2024-04-14T08:41:05.170691+0000","last_became_active":"2024-04-14T08:41:05.167159+0000","last_became_peered":"2024-04-14T08:41:05.167159+0000","last_unstale":"2024-04-14T08:41:05.170691+0000","last_undegraded":"2024-04-14T08:41:05.170691+0000","last_fullsized":"2024-04-14T08:41:05.170691+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:23:53.194822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.150663+0000","last_change":"2024-04-14T08:41:04.154343+0000","last_active":"2024-04-14T08:41:05.150663+0000","last_peered":"2024-04-14T08:41:05.150663+0000","last_clean":"2024-04-14T08:41:05.150663+0000","last_became_active":"2024-04-14T08:41:04.154106+0000","last_became_peered":"2024-04-14T08:41:04.154106+0000","last_unstale":"2024-04-14T08:41:05.150663+0000","last_undegraded":"2024-04-14T08:41:05.150663+0000","last_fullsized":"2024-04-14T08:41:05.150663+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:21:08.066477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.165824+0000","last_change":"2024-04-14T08:41:05.155780+0000","last_active":"2024-04-14T08:41:05.165824+0000","last_peered":"2024-04-14T08:41:05.165824+0000","last_clean":"2024-04-14T08:41:05.165824+0000","last_became_active":"2024-04-14T08:41:05.155540+0000","last_became_peered":"2024-04-14T08:41:05.155540+0000","last_unstale":"2024-04-14T08:41:05.165824+0000","last_undegraded":"2024-04-14T08:41:05.165824+0000","last_fullsized":"2024-04-14T08:41:05.165824+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:52:39.232361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.141060+0000","last_change":"2024-04-14T08:41:04.154091+0000","last_active":"2024-04-14T08:41:05.141060+0000","last_peered":"2024-04-14T08:41:05.141060+0000","last_clean":"2024-04-14T08:41:05.141060+0000","last_became_active":"2024-04-14T08:41:04.153829+0000","last_became_peered":"2024-04-14T08:41:04.153829+0000","last_unstale":"2024-04-14T08:41:05.141060+0000","last_undegraded":"2024-04-14T08:41:05.141060+0000","last_fullsized":"2024-04-14T08:41:05.141060+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:13:52.748581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.171009+0000","last_change":"2024-04-14T08:41:05.166856+0000","last_active":"2024-04-14T08:41:05.171009+0000","last_peered":"2024-04-14T08:41:05.171009+0000","last_clean":"2024-04-14T08:41:05.171009+0000","last_became_active":"2024-04-14T08:41:05.166486+0000","last_became_peered":"2024-04-14T08:41:05.166486+0000","last_unstale":"2024-04-14T08:41:05.171009+0000","last_undegraded":"2024-04-14T08:41:05.171009+0000","last_fullsized":"2024-04-14T08:41:05.171009+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:20:52.790502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.139279+0000","last_change":"2024-04-14T08:41:04.156898+0000","last_active":"2024-04-14T08:41:05.139279+0000","last_peered":"2024-04-14T08:41:05.139279+0000","last_clean":"2024-04-14T08:41:05.139279+0000","last_became_active":"2024-04-14T08:41:04.156587+0000","last_became_peered":"2024-04-14T08:41:04.156587+0000","last_unstale":"2024-04-14T08:41:05.139279+0000","last_undegraded":"2024-04-14T08:41:05.139279+0000","last_fullsized":"2024-04-14T08:41:05.139279+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:01:27.986252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.151269+0000","last_change":"2024-04-14T08:41:05.157345+0000","last_active":"2024-04-14T08:41:06.151269+0000","last_peered":"2024-04-14T08:41:06.151269+0000","last_clean":"2024-04-14T08:41:06.151269+0000","last_became_active":"2024-04-14T08:41:05.157078+0000","last_became_peered":"2024-04-14T08:41:05.157078+0000","last_unstale":"2024-04-14T08:41:06.151269+0000","last_undegraded":"2024-04-14T08:41:06.151269+0000","last_fullsized":"2024-04-14T08:41:06.151269+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:41:47.211392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.149091+0000","last_change":"2024-04-14T08:41:04.148649+0000","last_active":"2024-04-14T08:41:05.149091+0000","last_peered":"2024-04-14T08:41:05.149091+0000","last_clean":"2024-04-14T08:41:05.149091+0000","last_became_active":"2024-04-14T08:41:04.148362+0000","last_became_peered":"2024-04-14T08:41:04.148362+0000","last_unstale":"2024-04-14T08:41:05.149091+0000","last_undegraded":"2024-04-14T08:41:05.149091+0000","last_fullsized":"2024-04-14T08:41:05.149091+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:21:19.727239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.167703+0000","last_change":"2024-04-14T08:41:05.163443+0000","last_active":"2024-04-14T08:41:05.167703+0000","last_peered":"2024-04-14T08:41:05.167703+0000","last_clean":"2024-04-14T08:41:05.167703+0000","last_became_active":"2024-04-14T08:41:05.163161+0000","last_became_peered":"2024-04-14T08:41:05.163161+0000","last_unstale":"2024-04-14T08:41:05.167703+0000","last_undegraded":"2024-04-14T08:41:05.167703+0000","last_fullsized":"2024-04-14T08:41:05.167703+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:03:28.321597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.140110+0000","last_change":"2024-04-14T08:41:04.157049+0000","last_active":"2024-04-14T08:41:05.140110+0000","last_peered":"2024-04-14T08:41:05.140110+0000","last_clean":"2024-04-14T08:41:05.140110+0000","last_became_active":"2024-04-14T08:41:04.156773+0000","last_became_peered":"2024-04-14T08:41:04.156773+0000","last_unstale":"2024-04-14T08:41:05.140110+0000","last_undegraded":"2024-04-14T08:41:05.140110+0000","last_fullsized":"2024-04-14T08:41:05.140110+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:01:57.525115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.168428+0000","last_change":"2024-04-14T08:41:05.166136+0000","last_active":"2024-04-14T08:41:05.168428+0000","last_peered":"2024-04-14T08:41:05.168428+0000","last_clean":"2024-04-14T08:41:05.168428+0000","last_became_active":"2024-04-14T08:41:05.165758+0000","last_became_peered":"2024-04-14T08:41:05.165758+0000","last_unstale":"2024-04-14T08:41:05.168428+0000","last_undegraded":"2024-04-14T08:41:05.168428+0000","last_fullsized":"2024-04-14T08:41:05.168428+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:04.125490+0000","last_clean_scrub_stamp":"2024-04-14T08:41:04.125490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:47:10.299310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"100'2","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-14T08:41:05.166001+0000","last_change":"2024-04-14T08:41:04.149919+0000","last_active":"2024-04-14T08:41:05.166001+0000","last_peered":"2024-04-14T08:41:05.166001+0000","last_clean":"2024-04-14T08:41:05.166001+0000","last_became_active":"2024-04-14T08:41:04.149443+0000","last_became_peered":"2024-04-14T08:41:04.149443+0000","last_unstale":"2024-04-14T08:41:05.166001+0000","last_undegraded":"2024-04-14T08:41:05.166001+0000","last_fullsized":"2024-04-14T08:41:05.166001+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:41:03.121340+0000","last_clean_scrub_stamp":"2024-04-14T08:41:03.121340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:19:38.868105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":334,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-14T08:41:06.150262+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:41:06.150262+0000","last_peered":"2024-04-14T08:41:06.150262+0000","last_clean":"2024-04-14T08:41:06.150262+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:41:06.150262+0000","last_undegraded":"2024-04-14T08:41:06.150262+0000","last_fullsized":"2024-04-14T08:41:06.150262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1843200,"data_stored":1164491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":566796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":566796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":566796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":566796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":566796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1228800,"data_stored":566796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1822720,"data_stored":1157164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:41:08.465 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-14T08:41:08.466 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-14T08:41:08.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.609+0000 7f175311e640 1 Processor -- start 2024-04-14T08:41:08.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.609+0000 7f175311e640 1 -- start start 2024-04-14T08:41:08.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.610+0000 7f175311e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f174c15c380 0x7f174c140e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.610+0000 7f175311e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f174c15b620 con 0x7f174c0b23e0 2024-04-14T08:41:08.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.610+0000 7f175311e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f174c1414a0 con 0x7f174c13df30 2024-04-14T08:41:08.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.610+0000 7f175311e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f174c0cf9b0 con 0x7f174c15c380 2024-04-14T08:41:08.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.610+0000 7f174bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f174c0b23e0 0x7f174c140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:58992/0 (socket says 172.21.15.172:58992) 2024-04-14T08:41:08.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.610+0000 7f174bfff640 1 -- 172.21.15.172:0/3401251534 learned_addr learned my addr 172.21.15.172:0/3401251534 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:08.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.610+0000 7f17516bb640 1 --2- 172.21.15.172:0/3401251534 >> v2:172.21.15.177:3300/0 conn(0x7f174c15c380 0x7f174c140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.611+0000 7f174b7fe640 1 -- 172.21.15.172:0/3401251534 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2754838256 0 0) 0x7f1740001110 con 0x7f174c0b23e0 2024-04-14T08:41:08.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.611+0000 7f174b7fe640 1 -- 172.21.15.172:0/3401251534 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1734003520 con 0x7f174c0b23e0 2024-04-14T08:41:08.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.611+0000 7f17516bb640 1 -- 172.21.15.172:0/3401251534 >> v1:172.21.15.177:6789/0 conn(0x7f174c13df30 legacy=0x7f174c0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.611+0000 7f174b7fe640 1 -- 172.21.15.172:0/3401251534 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2170786613 0 0) 0x7f174c1414a0 con 0x7f174c13df30 2024-04-14T08:41:08.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.611+0000 7f17516bb640 1 -- 172.21.15.172:0/3401251534 >> v1:172.21.15.172:6789/0 conn(0x7f174c0b23e0 legacy=0x7f174c140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.611+0000 7f17516bb640 1 -- 172.21.15.172:0/3401251534 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c15b8a0 con 0x7f174c15c380 2024-04-14T08:41:08.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.611+0000 7f17516bb640 1 --2- 172.21.15.172:0/3401251534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f174c15c380 0x7f174c140e80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1744009d90 tx=0x7f174402bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9b0b45c9962858c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:08.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.612+0000 7f174b7fe640 1 -- 172.21.15.172:0/3401251534 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744039070 con 0x7f174c15c380 2024-04-14T08:41:08.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.612+0000 7f174b7fe640 1 -- 172.21.15.172:0/3401251534 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f174402cd00 con 0x7f174c15c380 2024-04-14T08:41:08.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.612+0000 7f174b7fe640 1 -- 172.21.15.172:0/3401251534 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744034660 con 0x7f174c15c380 2024-04-14T08:41:08.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.612+0000 7f175311e640 1 -- 172.21.15.172:0/3401251534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f174c15c380 msgr2=0x7f174c140e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.612+0000 7f175311e640 1 --2- 172.21.15.172:0/3401251534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f174c15c380 0x7f174c140e80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1744009d90 tx=0x7f174402bf50 comp rx=0 tx=0).stop 2024-04-14T08:41:08.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.612+0000 7f175311e640 1 -- 172.21.15.172:0/3401251534 shutdown_connections 2024-04-14T08:41:08.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.612+0000 7f175311e640 1 --2- 172.21.15.172:0/3401251534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f174c15c380 0x7f174c140e80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.612+0000 7f175311e640 1 -- 172.21.15.172:0/3401251534 >> 172.21.15.172:0/3401251534 conn(0x7f174c06b520 msgr2=0x7f174c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:08.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.612+0000 7f175311e640 1 -- 172.21.15.172:0/3401251534 shutdown_connections 2024-04-14T08:41:08.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.612+0000 7f175311e640 1 -- 172.21.15.172:0/3401251534 wait complete. 2024-04-14T08:41:08.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.613+0000 7f175311e640 1 Processor -- start 2024-04-14T08:41:08.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.613+0000 7f175311e640 1 -- start start 2024-04-14T08:41:08.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.614+0000 7f175311e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f174c0b23e0 0x7f174c154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.614+0000 7f175311e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f174c13df30 0x7f174c152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.614+0000 7f175311e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c15c380 0x7f174c13ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.614+0000 7f175311e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f174c13e310 con 0x7f174c15c380 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.614+0000 7f175311e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f174c15be60 con 0x7f174c0b23e0 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.614+0000 7f175311e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f174c0712d0 con 0x7f174c13df30 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.614+0000 7f17516bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c15c380 0x7f174c13ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.614+0000 7f17516bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c15c380 0x7f174c13ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36704/0 (socket says 172.21.15.172:36704) 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.614+0000 7f17516bb640 1 -- 172.21.15.172:0/316017571 learned_addr learned my addr 172.21.15.172:0/316017571 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.614+0000 7f1750eba640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f174c0b23e0 0x7f174c154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.614+0000 7f174bfff640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f174c13df30 0x7f174c152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.615+0000 7f17516bb640 1 -- 172.21.15.172:0/316017571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f174c13df30 msgr2=0x7f174c152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.615+0000 7f17516bb640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f174c13df30 0x7f174c152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.615+0000 7f17516bb640 1 -- 172.21.15.172:0/316017571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f174c0b23e0 msgr2=0x7f174c154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.615+0000 7f17516bb640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f174c0b23e0 0x7f174c154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.615+0000 7f17516bb640 1 -- 172.21.15.172:0/316017571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c155060 con 0x7f174c15c380 2024-04-14T08:41:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.615+0000 7f1750eba640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f174c0b23e0 0x7f174c154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.615+0000 7f17516bb640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c15c380 0x7f174c13ea80 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f1744037040 tx=0x7f1744004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.615+0000 7f17497fa640 1 -- 172.21.15.172:0/316017571 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744039050 con 0x7f174c15c380 2024-04-14T08:41:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.615+0000 7f17497fa640 1 -- 172.21.15.172:0/316017571 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1744045070 con 0x7f174c15c380 2024-04-14T08:41:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.615+0000 7f175311e640 1 -- 172.21.15.172:0/316017571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c0afc60 con 0x7f174c15c380 2024-04-14T08:41:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.615+0000 7f17497fa640 1 -- 172.21.15.172:0/316017571 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744009450 con 0x7f174c15c380 2024-04-14T08:41:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.615+0000 7f175311e640 1 -- 172.21.15.172:0/316017571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c0b0220 con 0x7f174c15c380 2024-04-14T08:41:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.616+0000 7f175311e640 1 -- 172.21.15.172:0/316017571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174c068590 con 0x7f174c15c380 2024-04-14T08:41:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.616+0000 7f17497fa640 1 -- 172.21.15.172:0/316017571 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f174403e830 con 0x7f174c15c380 2024-04-14T08:41:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.617+0000 7f17497fa640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f173806c920 0x7f173806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.617+0000 7f1750eba640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f173806c920 0x7f173806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.617+0000 7f17497fa640 1 -- 172.21.15.172:0/316017571 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f17440ae8c0 con 0x7f174c15c380 2024-04-14T08:41:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.617+0000 7f1750eba640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f173806c920 0x7f173806ede0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f173c000f60 tx=0x7f173c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:08.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.621+0000 7f17497fa640 1 -- 172.21.15.172:0/316017571 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f174407c630 con 0x7f174c15c380 2024-04-14T08:41:08.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.862+0000 7f175311e640 1 -- 172.21.15.172:0/316017571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f174c0b1f60 con 0x7f174c15c380 2024-04-14T08:41:08.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.864+0000 7f17497fa640 1 -- 172.21.15.172:0/316017571 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f1744080930 con 0x7f174c15c380 2024-04-14T08:41:08.866 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2024-04-14T08:41:08.866 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA== 2024-04-14T08:41:08.866 INFO:teuthology.orchestra.run.smithi172.stdout: caps mds = "allow" 2024-04-14T08:41:08.866 INFO:teuthology.orchestra.run.smithi172.stdout: caps mon = "allow r" 2024-04-14T08:41:08.866 INFO:teuthology.orchestra.run.smithi172.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-14T08:41:08.866 INFO:teuthology.orchestra.run.smithi172.stderr:updated caps for client.0 2024-04-14T08:41:08.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.866+0000 7f175311e640 1 -- 172.21.15.172:0/316017571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f173806c920 msgr2=0x7f173806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.866+0000 7f175311e640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f173806c920 0x7f173806ede0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f173c000f60 tx=0x7f173c009040 comp rx=0 tx=0).stop 2024-04-14T08:41:08.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.866+0000 7f175311e640 1 -- 172.21.15.172:0/316017571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c15c380 msgr2=0x7f174c13ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.866+0000 7f175311e640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c15c380 0x7f174c13ea80 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f1744037040 tx=0x7f1744004a40 comp rx=0 tx=0).stop 2024-04-14T08:41:08.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.866+0000 7f175311e640 1 -- 172.21.15.172:0/316017571 shutdown_connections 2024-04-14T08:41:08.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.866+0000 7f175311e640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f173806c920 0x7f173806ede0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.866+0000 7f175311e640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f174c15c380 0x7f174c13ea80 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.866+0000 7f175311e640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f174c13df30 0x7f174c152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.866+0000 7f175311e640 1 --2- 172.21.15.172:0/316017571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f174c0b23e0 0x7f174c154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.866+0000 7f175311e640 1 -- 172.21.15.172:0/316017571 >> 172.21.15.172:0/316017571 conn(0x7f174c06b520 msgr2=0x7f174c15ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:08.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.866+0000 7f175311e640 1 -- 172.21.15.172:0/316017571 shutdown_connections 2024-04-14T08:41:08.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.866+0000 7f175311e640 1 -- 172.21.15.172:0/316017571 wait complete. 2024-04-14T08:41:08.879 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:41:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.978+0000 7fed2d06d640 1 Processor -- start 2024-04-14T08:41:08.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.978+0000 7fed2d06d640 1 -- start start 2024-04-14T08:41:08.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed2d06d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fed2815a690 0x7fed28071d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed2d06d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed2815c240 0x7fed280722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed2d06d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed280b3360 con 0x7fed28072970 2024-04-14T08:41:08.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed2d06d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fed280b3640 con 0x7fed2815a690 2024-04-14T08:41:08.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed2d06d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fed280b3920 con 0x7fed2815c240 2024-04-14T08:41:08.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed1e59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fed28072970 0x7fed2815e440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:59004/0 (socket says 172.21.15.172:59004) 2024-04-14T08:41:08.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed1e59c640 1 -- 172.21.15.172:0/2924227044 learned_addr learned my addr 172.21.15.172:0/2924227044 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:08.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed2759e640 1 --2- 172.21.15.172:0/2924227044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed2815c240 0x7fed280722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed26d9d640 1 --2- 172.21.15.172:0/2924227044 >> v2:172.21.15.177:3300/0 conn(0x7fed2815a690 0x7fed28071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed2659c640 1 -- 172.21.15.172:0/2924227044 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2018960047 0 0) 0x7fed280b3360 con 0x7fed28072970 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed2659c640 1 -- 172.21.15.172:0/2924227044 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed14003520 con 0x7fed28072970 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed2759e640 1 -- 172.21.15.172:0/2924227044 >> v2:172.21.15.177:3300/0 conn(0x7fed2815a690 msgr2=0x7fed28071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed2759e640 1 --2- 172.21.15.172:0/2924227044 >> v2:172.21.15.177:3300/0 conn(0x7fed2815a690 0x7fed28071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed2759e640 1 -- 172.21.15.172:0/2924227044 >> v1:172.21.15.172:6789/0 conn(0x7fed28072970 legacy=0x7fed2815e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed2759e640 1 -- 172.21.15.172:0/2924227044 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed280b3c00 con 0x7fed2815c240 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.979+0000 7fed26d9d640 1 --2- 172.21.15.172:0/2924227044 >> v2:172.21.15.177:3300/0 conn(0x7fed2815a690 0x7fed28071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.980+0000 7fed2759e640 1 --2- 172.21.15.172:0/2924227044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed2815c240 0x7fed280722a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed20009d90 tx=0x7fed2002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a61d21ffecd48b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.980+0000 7fed2659c640 1 -- 172.21.15.172:0/2924227044 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20039070 con 0x7fed2815c240 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.980+0000 7fed2659c640 1 -- 172.21.15.172:0/2924227044 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed2002cbc0 con 0x7fed2815c240 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.980+0000 7fed2659c640 1 -- 172.21.15.172:0/2924227044 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20034440 con 0x7fed2815c240 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.981+0000 7fed2d06d640 1 -- 172.21.15.172:0/2924227044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed2815c240 msgr2=0x7fed280722a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.981+0000 7fed2d06d640 1 --2- 172.21.15.172:0/2924227044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed2815c240 0x7fed280722a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed20009d90 tx=0x7fed2002be10 comp rx=0 tx=0).stop 2024-04-14T08:41:08.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.981+0000 7fed2d06d640 1 -- 172.21.15.172:0/2924227044 shutdown_connections 2024-04-14T08:41:08.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.981+0000 7fed2d06d640 1 --2- 172.21.15.172:0/2924227044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed2815c240 0x7fed280722a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.981+0000 7fed2d06d640 1 --2- 172.21.15.172:0/2924227044 >> v2:172.21.15.177:3300/0 conn(0x7fed2815a690 0x7fed28071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.981+0000 7fed2d06d640 1 -- 172.21.15.172:0/2924227044 >> 172.21.15.172:0/2924227044 conn(0x7fed2806ec30 msgr2=0x7fed2806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:08.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.981+0000 7fed2d06d640 1 -- 172.21.15.172:0/2924227044 shutdown_connections 2024-04-14T08:41:08.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.981+0000 7fed2d06d640 1 -- 172.21.15.172:0/2924227044 wait complete. 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.982+0000 7fed2d06d640 1 Processor -- start 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.982+0000 7fed2d06d640 1 -- start start 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed2d06d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed28072970 0x7fed28158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed2d06d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed2815a690 0x7fed2814c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed2d06d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed2815c240 0x7fed2814acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed2d06d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fed2815aa70 con 0x7fed2815a690 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed2d06d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fed280b3100 con 0x7fed2815c240 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed2d06d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fed2815abf0 con 0x7fed28072970 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed1e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed2815a690 0x7fed2814c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed1e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed2815a690 0x7fed2814c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36712/0 (socket says 172.21.15.172:36712) 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed1e59c640 1 -- 172.21.15.172:0/3047214390 learned_addr learned my addr 172.21.15.172:0/3047214390 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed26d9d640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed28072970 0x7fed28158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed2759e640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed2815c240 0x7fed2814acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed1e59c640 1 -- 172.21.15.172:0/3047214390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed28072970 msgr2=0x7fed28158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed1e59c640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed28072970 0x7fed28158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed1e59c640 1 -- 172.21.15.172:0/3047214390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed2815c240 msgr2=0x7fed2814acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed1e59c640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed2815c240 0x7fed2814acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed1e59c640 1 -- 172.21.15.172:0/3047214390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2814b210 con 0x7fed2815a690 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed1e59c640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed2815a690 0x7fed2814c1a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fed1000c430 tx=0x7fed1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.983+0000 7fed1ffff640 1 -- 172.21.15.172:0/3047214390 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10017070 con 0x7fed2815a690 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.984+0000 7fed1ffff640 1 -- 172.21.15.172:0/3047214390 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed10005d00 con 0x7fed2815a690 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.984+0000 7fed1ffff640 1 -- 172.21.15.172:0/3047214390 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10007310 con 0x7fed2815a690 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.984+0000 7fed2d06d640 1 -- 172.21.15.172:0/3047214390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2814e610 con 0x7fed2815a690 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.984+0000 7fed2d06d640 1 -- 172.21.15.172:0/3047214390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2814ebd0 con 0x7fed2815a690 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.985+0000 7fed2d06d640 1 -- 172.21.15.172:0/3047214390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed28068520 con 0x7fed2815a690 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.985+0000 7fed1ffff640 1 -- 172.21.15.172:0/3047214390 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed1001a070 con 0x7fed2815a690 2024-04-14T08:41:08.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.985+0000 7fed1ffff640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fed0806c8f0 0x7fed0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:08.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.986+0000 7fed26d9d640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fed0806c8f0 0x7fed0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:08.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.986+0000 7fed1ffff640 1 -- 172.21.15.172:0/3047214390 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7fed1008b050 con 0x7fed2815a690 2024-04-14T08:41:08.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.986+0000 7fed26d9d640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fed0806c8f0 0x7fed0806edb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fed28155080 tx=0x7fed18009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:08.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:08.990+0000 7fed1ffff640 1 -- 172.21.15.172:0/3047214390 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fed10058cb0 con 0x7fed2815a690 2024-04-14T08:41:09.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.202+0000 7fed2d06d640 1 -- 172.21.15.172:0/3047214390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed280cc6b0 con 0x7fed2815a690 2024-04-14T08:41:09.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.203+0000 7fed1ffff640 1 -- 172.21.15.172:0/3047214390 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5195 (secure 0 0 0) 0x7fed10076e20 con 0x7fed2815a690 2024-04-14T08:41:09.204 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:09.204 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/4028911489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4028911489},{"type":"v1","addr":"172.21.15.172:6837","nonce":4028911489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:41:04.492773+0000","modified":"2024-04-14T08:41:06.492980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/2602337717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2602337717},{"type":"v1","addr":"172.21.15.177:6835","nonce":2602337717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8394,"qdb_cluster":[8394]},"id":7}]} 2024-04-14T08:41:09.205 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 74 2024-04-14T08:41:09.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.205+0000 7fed2d06d640 1 -- 172.21.15.172:0/3047214390 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fed0806c8f0 msgr2=0x7fed0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.205+0000 7fed2d06d640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fed0806c8f0 0x7fed0806edb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fed28155080 tx=0x7fed18009190 comp rx=0 tx=0).stop 2024-04-14T08:41:09.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.205+0000 7fed2d06d640 1 -- 172.21.15.172:0/3047214390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed2815a690 msgr2=0x7fed2814c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.205+0000 7fed2d06d640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed2815a690 0x7fed2814c1a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fed1000c430 tx=0x7fed1000c8f0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.205+0000 7fed2d06d640 1 -- 172.21.15.172:0/3047214390 shutdown_connections 2024-04-14T08:41:09.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.205+0000 7fed2d06d640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fed0806c8f0 0x7fed0806edb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.205+0000 7fed2d06d640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed2815c240 0x7fed2814acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.205+0000 7fed2d06d640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed2815a690 0x7fed2814c1a0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.205+0000 7fed2d06d640 1 --2- 172.21.15.172:0/3047214390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed28072970 0x7fed28158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.205+0000 7fed2d06d640 1 -- 172.21.15.172:0/3047214390 >> 172.21.15.172:0/3047214390 conn(0x7fed2806ec30 msgr2=0x7fed28159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:09.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.205+0000 7fed2d06d640 1 -- 172.21.15.172:0/3047214390 shutdown_connections 2024-04-14T08:41:09.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.205+0000 7fed2d06d640 1 -- 172.21.15.172:0/3047214390 wait complete. 2024-04-14T08:41:09.620 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:41:04.492773+0000', 'modified': '2024-04-14T08:41:06.492980+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8394}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8394': {'gid': 8394, 'name': 'b', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6835/2602337717', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6834', 'nonce': 2602337717}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 2602337717}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8394, 'qdb_cluster': [8394]} 2024-04-14T08:41:09.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:41:09.621 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-14T08:41:09.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.721+0000 7f0bd495e640 1 Processor -- start 2024-04-14T08:41:09.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.722+0000 7f0bd495e640 1 -- start start 2024-04-14T08:41:09.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.722+0000 7f0bd495e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0bd013df30 0x7f0bd00b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.722+0000 7f0bd495e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0bd015c380 0x7f0bd00b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.722+0000 7f0bd495e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd015b620 con 0x7f0bd0140770 2024-04-14T08:41:09.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.722+0000 7f0bd495e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0bd00cf9b0 con 0x7f0bd013df30 2024-04-14T08:41:09.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.722+0000 7f0bd495e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f0bd01414a0 con 0x7f0bd015c380 2024-04-14T08:41:09.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.722+0000 7f0bce59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0bd013df30 0x7f0bd00b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.723+0000 7f0bce59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0bd013df30 0x7f0bd00b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44762/0 (socket says 172.21.15.172:44762) 2024-04-14T08:41:09.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.723+0000 7f0bce59c640 1 -- 172.21.15.172:0/1341301819 learned_addr learned my addr 172.21.15.172:0/1341301819 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:09.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.723+0000 7f0bc7fff640 1 --2- 172.21.15.172:0/1341301819 >> v2:172.21.15.177:3300/0 conn(0x7f0bd015c380 0x7f0bd00b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.723+0000 7f0bce59c640 1 -- 172.21.15.172:0/1341301819 >> v2:172.21.15.177:3300/0 conn(0x7f0bd015c380 msgr2=0x7f0bd00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:09.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.723+0000 7f0bce59c640 1 --2- 172.21.15.172:0/1341301819 >> v2:172.21.15.177:3300/0 conn(0x7f0bd015c380 0x7f0bd00b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.723+0000 7f0bce59c640 1 -- 172.21.15.172:0/1341301819 >> v1:172.21.15.177:6789/0 conn(0x7f0bd0140770 legacy=0x7f0bd0140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.723+0000 7f0bce59c640 1 -- 172.21.15.172:0/1341301819 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd015b8a0 con 0x7f0bd013df30 2024-04-14T08:41:09.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.723+0000 7f0bce59c640 1 --2- 172.21.15.172:0/1341301819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bd013df30 0x7f0bd00b1cd0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f0bd006c4d0 tx=0x7f0bc002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98d516c93876e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:09.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.724+0000 7f0bcdd9b640 1 -- 172.21.15.172:0/1341301819 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0034070 con 0x7f0bd013df30 2024-04-14T08:41:09.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.724+0000 7f0bcdd9b640 1 -- 172.21.15.172:0/1341301819 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc0030860 con 0x7f0bd013df30 2024-04-14T08:41:09.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.724+0000 7f0bcdd9b640 1 -- 172.21.15.172:0/1341301819 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0030c30 con 0x7f0bd013df30 2024-04-14T08:41:09.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.724+0000 7f0bd495e640 1 -- 172.21.15.172:0/1341301819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bd013df30 msgr2=0x7f0bd00b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.724+0000 7f0bd495e640 1 --2- 172.21.15.172:0/1341301819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bd013df30 0x7f0bd00b1cd0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f0bd006c4d0 tx=0x7f0bc002b9a0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.724+0000 7f0bd495e640 1 -- 172.21.15.172:0/1341301819 shutdown_connections 2024-04-14T08:41:09.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.724+0000 7f0bd495e640 1 --2- 172.21.15.172:0/1341301819 >> v2:172.21.15.177:3300/0 conn(0x7f0bd015c380 0x7f0bd00b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.724+0000 7f0bd495e640 1 --2- 172.21.15.172:0/1341301819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bd013df30 0x7f0bd00b1cd0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.724+0000 7f0bd495e640 1 -- 172.21.15.172:0/1341301819 >> 172.21.15.172:0/1341301819 conn(0x7f0bd006b520 msgr2=0x7f0bd006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:09.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.724+0000 7f0bd495e640 1 -- 172.21.15.172:0/1341301819 shutdown_connections 2024-04-14T08:41:09.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.724+0000 7f0bd495e640 1 -- 172.21.15.172:0/1341301819 wait complete. 2024-04-14T08:41:09.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.726+0000 7f0bd495e640 1 Processor -- start 2024-04-14T08:41:09.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.726+0000 7f0bd495e640 1 -- start start 2024-04-14T08:41:09.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.726+0000 7f0bd495e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bd013df30 0x7f0bd01517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.726+0000 7f0bd495e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bd0140770 0x7f0bd00b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.726+0000 7f0bd495e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bd015c380 0x7f0bd00b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.726+0000 7f0bd495e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd013e310 con 0x7f0bd0140770 2024-04-14T08:41:09.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.726+0000 7f0bd495e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd015be60 con 0x7f0bd015c380 2024-04-14T08:41:09.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.726+0000 7f0bd495e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd00ce9a0 con 0x7f0bd013df30 2024-04-14T08:41:09.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.726+0000 7f0bc7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bd0140770 0x7f0bd00b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.726+0000 7f0bc7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bd0140770 0x7f0bd00b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44772/0 (socket says 172.21.15.172:44772) 2024-04-14T08:41:09.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.726+0000 7f0bc7fff640 1 -- 172.21.15.172:0/3377234167 learned_addr learned my addr 172.21.15.172:0/3377234167 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:09.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.726+0000 7f0bce59c640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bd013df30 0x7f0bd01517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.727+0000 7f0bced9d640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bd015c380 0x7f0bd00b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.727+0000 7f0bc7fff640 1 -- 172.21.15.172:0/3377234167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bd013df30 msgr2=0x7f0bd01517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.727+0000 7f0bc7fff640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bd013df30 0x7f0bd01517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.727+0000 7f0bc7fff640 1 -- 172.21.15.172:0/3377234167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bd015c380 msgr2=0x7f0bd00b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.727+0000 7f0bc7fff640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bd015c380 0x7f0bd00b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.727+0000 7f0bc7fff640 1 -- 172.21.15.172:0/3377234167 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd0153470 con 0x7f0bd0140770 2024-04-14T08:41:09.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.727+0000 7f0bc7fff640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bd0140770 0x7f0bd00b4b20 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800e870 tx=0x7f0bb800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:09.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.727+0000 7f0bc6ffd640 1 -- 172.21.15.172:0/3377234167 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8017070 con 0x7f0bd0140770 2024-04-14T08:41:09.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.727+0000 7f0bc6ffd640 1 -- 172.21.15.172:0/3377234167 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb800c940 con 0x7f0bd0140770 2024-04-14T08:41:09.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.727+0000 7f0bd495e640 1 -- 172.21.15.172:0/3377234167 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd00adc60 con 0x7f0bd0140770 2024-04-14T08:41:09.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.727+0000 7f0bc6ffd640 1 -- 172.21.15.172:0/3377234167 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800cd30 con 0x7f0bd0140770 2024-04-14T08:41:09.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.727+0000 7f0bd495e640 1 -- 172.21.15.172:0/3377234167 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd0153750 con 0x7f0bd0140770 2024-04-14T08:41:09.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.728+0000 7f0bc6ffd640 1 -- 172.21.15.172:0/3377234167 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bb801a070 con 0x7f0bd0140770 2024-04-14T08:41:09.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.729+0000 7f0bc6ffd640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0bb006c8f0 0x7f0bb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.729+0000 7f0bc6ffd640 1 -- 172.21.15.172:0/3377234167 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f0bb808bf20 con 0x7f0bd0140770 2024-04-14T08:41:09.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.729+0000 7f0bce59c640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0bb006c8f0 0x7f0bb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.729+0000 7f0bce59c640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0bb006c8f0 0x7f0bb006edb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0036000 tx=0x7f0bc0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:09.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.730+0000 7f0bd7825740 1 Processor -- start 2024-04-14T08:41:09.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.730+0000 7f0bd7825740 1 -- start start 2024-04-14T08:41:09.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bd7825740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 0x55d849142b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bd7825740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d84916ff40 0x55d8491728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bd7825740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d849143040 0x55d849172e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bd7825740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d8490ac030 con 0x55d84916fb60 2024-04-14T08:41:09.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bd7825740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d849054620 con 0x55d849143040 2024-04-14T08:41:09.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bd7825740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d8490a7fe0 con 0x55d84916ff40 2024-04-14T08:41:09.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bce59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 0x55d849142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bce59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 0x55d849142b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44780/0 (socket says 172.21.15.172:44780) 2024-04-14T08:41:09.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bce59c640 1 -- 172.21.15.172:0/584071821 learned_addr learned my addr 172.21.15.172:0/584071821 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:09.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bced9d640 1 --2- 172.21.15.172:0/584071821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d849143040 0x55d849172e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bc7fff640 1 --2- 172.21.15.172:0/584071821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d84916ff40 0x55d8491728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bce59c640 1 -- 172.21.15.172:0/584071821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d84916ff40 msgr2=0x55d8491728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:09.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bce59c640 1 --2- 172.21.15.172:0/584071821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d84916ff40 0x55d8491728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bce59c640 1 -- 172.21.15.172:0/584071821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d849143040 msgr2=0x55d849172e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:09.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bce59c640 1 --2- 172.21.15.172:0/584071821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d849143040 0x55d849172e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bce59c640 1 -- 172.21.15.172:0/584071821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d848dc02d0 con 0x55d84916fb60 2024-04-14T08:41:09.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0bce59c640 1 --2- 172.21.15.172:0/584071821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 0x55d849142b00 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009930 tx=0x7f0bc0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad06d22681890a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:09.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0baffff640 1 -- 172.21.15.172:0/584071821 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0054070 con 0x55d84916fb60 2024-04-14T08:41:09.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0baffff640 1 -- 172.21.15.172:0/584071821 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc0044970 con 0x55d84916fb60 2024-04-14T08:41:09.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.731+0000 7f0baffff640 1 -- 172.21.15.172:0/584071821 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0034040 con 0x55d84916fb60 2024-04-14T08:41:09.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.732+0000 7f0bd7825740 1 -- 172.21.15.172:0/584071821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 msgr2=0x55d849142b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.732+0000 7f0bd7825740 1 --2- 172.21.15.172:0/584071821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 0x55d849142b00 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009930 tx=0x7f0bc0030000 comp rx=0 tx=0).stop 2024-04-14T08:41:09.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.732+0000 7f0bd7825740 1 -- 172.21.15.172:0/584071821 shutdown_connections 2024-04-14T08:41:09.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.732+0000 7f0bd7825740 1 --2- 172.21.15.172:0/584071821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d849143040 0x55d849172e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.732+0000 7f0bd7825740 1 --2- 172.21.15.172:0/584071821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d84916ff40 0x55d8491728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.732+0000 7f0bd7825740 1 --2- 172.21.15.172:0/584071821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 0x55d849142b00 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.732+0000 7f0bd7825740 1 -- 172.21.15.172:0/584071821 >> 172.21.15.172:0/584071821 conn(0x55d849144d70 msgr2=0x55d84918d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:09.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.732+0000 7f0bd7825740 1 -- 172.21.15.172:0/584071821 shutdown_connections 2024-04-14T08:41:09.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.732+0000 7f0bd7825740 1 -- 172.21.15.172:0/584071821 wait complete. 2024-04-14T08:41:09.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.733+0000 7f0bd7825740 1 -- start start 2024-04-14T08:41:09.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.733+0000 7f0bd7825740 1 Processor -- start 2024-04-14T08:41:09.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bd7825740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d84916fb60 0x55d84917b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bd7825740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916ff40 0x55d84916f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bd7825740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8491773e0 0x55d849177860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bd7825740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d849147410 con 0x55d8491773e0 2024-04-14T08:41:09.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bd7825740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d8490679e0 con 0x55d84916ff40 2024-04-14T08:41:09.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bd7825740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d849149c50 con 0x55d84916fb60 2024-04-14T08:41:09.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bd7825740 10 client.0 _ll_register_callbacks cb 0x55d8490a4080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:09.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bced9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8491773e0 0x55d849177860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bced9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8491773e0 0x55d849177860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44796/0 (socket says 172.21.15.172:44796) 2024-04-14T08:41:09.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bced9d640 1 -- 172.21.15.172:0/3583883044 learned_addr learned my addr 172.21.15.172:0/3583883044 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:09.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bce59c640 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d84916fb60 0x55d84917b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bc7fff640 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916ff40 0x55d84916f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bced9d640 1 -- 172.21.15.172:0/3583883044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d84916fb60 msgr2=0x55d84917b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bced9d640 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d84916fb60 0x55d84917b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bced9d640 1 -- 172.21.15.172:0/3583883044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916ff40 msgr2=0x55d84916f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bced9d640 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916ff40 0x55d84916f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bced9d640 1 -- 172.21.15.172:0/3583883044 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d848e8a6b0 con 0x55d8491773e0 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bced9d640 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8491773e0 0x55d849177860 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8012620 tx=0x7f0bc8012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7acb59338da64db server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bad7fa640 1 -- 172.21.15.172:0/3583883044 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8019020 con 0x55d8491773e0 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bad7fa640 1 -- 172.21.15.172:0/3583883044 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f0bc8003b80 con 0x55d8491773e0 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.734+0000 7f0bd7825740 1 -- 172.21.15.172:0/3583883044 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc0044970 con 0x55d8491773e0 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bad7fa640 1 -- 172.21.15.172:0/3583883044 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f0bc8023050 con 0x55d8491773e0 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bd7825740 10 client.8523 fetch_fsmap learned FSMap version 74 2024-04-14T08:41:09.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bd7825740 1 -- 172.21.15.172:0/3583883044 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55d84917ae50 con 0x55d8491773e0 2024-04-14T08:41:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bad7fa640 1 -- 172.21.15.172:0/3583883044 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc0044970 con 0x55d8491773e0 2024-04-14T08:41:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bad7fa640 1 -- 172.21.15.172:0/3583883044 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f0bc8026020 con 0x55d8491773e0 2024-04-14T08:41:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bd7825740 10 client.8523 fetch_fsmap finished waiting for FSMap version 74 2024-04-14T08:41:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bd7825740 10 client.8523 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-14T08:41:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bd7825740 20 client.8523 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bd7825740 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d849178af0 0x55d8491a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bd7825740 4 client.8523 mds_command: new command op to 8394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:41:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bd7825740 1 -- 172.21.15.172:0/3583883044 --> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d849053cc0 con 0x55d849178af0 2024-04-14T08:41:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bce59c640 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d849178af0 0x55d8491a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bad7fa640 10 client.8523 ms_handle_connect on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.735+0000 7f0bce59c640 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d849178af0 0x55d8491a1ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=987618cd00d67a30 server_cookie=45921d0961194d8f in_seq=0 out_seq=0 2024-04-14T08:41:09.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.736+0000 7f0bad7fa640 1 -- 172.21.15.172:0/3583883044 <== mds.0 v2:172.21.15.177:6834/2602337717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d849053cc0 con 0x55d849178af0 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.736+0000 7f0bad7fa640 10 client.8523 handle_command_reply: tid=0 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.736+0000 7f0bd7825740 1 client.8523 shutdown 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.736+0000 7f0bd7825740 1 -- 172.21.15.172:0/3583883044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8491773e0 msgr2=0x55d849177860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.736+0000 7f0bd7825740 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8491773e0 0x55d849177860 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8012620 tx=0x7f0bc8012ae0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.737+0000 7f0bd7825740 1 -- 172.21.15.172:0/3583883044 shutdown_connections 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.737+0000 7f0bd7825740 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d849178af0 0x55d8491a1ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.737+0000 7f0bd7825740 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8491773e0 0x55d849177860 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.737+0000 7f0bd7825740 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916ff40 0x55d84916f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.737+0000 7f0bd7825740 1 --2- 172.21.15.172:0/3583883044 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d84916fb60 0x55d84917b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.737+0000 7f0bad7fa640 0 client.8523 ms_handle_reset on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.737+0000 7f0bd7825740 1 -- 172.21.15.172:0/3583883044 >> 172.21.15.172:0/3583883044 conn(0x55d849143040 msgr2=0x55d849146b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.737+0000 7f0bd7825740 1 -- 172.21.15.172:0/3583883044 shutdown_connections 2024-04-14T08:41:09.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.737+0000 7f0bd7825740 1 -- 172.21.15.172:0/3583883044 wait complete. 2024-04-14T08:41:09.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.737+0000 7f0bd7825740 20 client.8523 trim_cache size 0 max 16384 2024-04-14T08:41:09.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bd7825740 1 Processor -- start 2024-04-14T08:41:09.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bd7825740 1 -- start start 2024-04-14T08:41:09.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bd7825740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916fb60 0x55d84916ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bd7825740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916ff40 0x55d84917b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bd7825740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d8491773e0 0x55d84919a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bd7825740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd0200060 con 0x55d84916ff40 2024-04-14T08:41:09.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bd7825740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd00af110 con 0x55d84916fb60 2024-04-14T08:41:09.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bd7825740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd01ffa10 con 0x55d8491773e0 2024-04-14T08:41:09.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bc7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916ff40 0x55d84917b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bc7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916ff40 0x55d84917b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44798/0 (socket says 172.21.15.172:44798) 2024-04-14T08:41:09.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bc7fff640 1 -- 172.21.15.172:0/3558197991 learned_addr learned my addr 172.21.15.172:0/3558197991 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:09.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bced9d640 1 --2- 172.21.15.172:0/3558197991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d8491773e0 0x55d84919a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.755+0000 7f0bce59c640 1 --2- 172.21.15.172:0/3558197991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916fb60 0x55d84916ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bc7fff640 1 -- 172.21.15.172:0/3558197991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d8491773e0 msgr2=0x55d84919a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bc7fff640 1 --2- 172.21.15.172:0/3558197991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d8491773e0 0x55d84919a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bc7fff640 1 -- 172.21.15.172:0/3558197991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916fb60 msgr2=0x55d84916ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bc7fff640 1 --2- 172.21.15.172:0/3558197991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916fb60 0x55d84916ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bc7fff640 1 -- 172.21.15.172:0/3558197991 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d84917aaa0 con 0x55d84916ff40 2024-04-14T08:41:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bc7fff640 1 --2- 172.21.15.172:0/3558197991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916ff40 0x55d84917b820 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f0bb808a1c0 tx=0x7f0bb8060040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62072169fc53599 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0baf7fe640 1 -- 172.21.15.172:0/3558197991 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8021040 con 0x55d84916ff40 2024-04-14T08:41:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0baf7fe640 1 -- 172.21.15.172:0/3558197991 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb808a330 con 0x55d84916ff40 2024-04-14T08:41:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0baf7fe640 1 -- 172.21.15.172:0/3558197991 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8062170 con 0x55d84916ff40 2024-04-14T08:41:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bd7825740 1 -- 172.21.15.172:0/3558197991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916ff40 msgr2=0x55d84917b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bd7825740 1 --2- 172.21.15.172:0/3558197991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916ff40 0x55d84917b820 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f0bb808a1c0 tx=0x7f0bb8060040 comp rx=0 tx=0).stop 2024-04-14T08:41:09.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bd7825740 1 -- 172.21.15.172:0/3558197991 shutdown_connections 2024-04-14T08:41:09.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bd7825740 1 --2- 172.21.15.172:0/3558197991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d8491773e0 0x55d84919a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bd7825740 1 --2- 172.21.15.172:0/3558197991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916ff40 0x55d84917b820 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bd7825740 1 --2- 172.21.15.172:0/3558197991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916fb60 0x55d84916ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bd7825740 1 -- 172.21.15.172:0/3558197991 >> 172.21.15.172:0/3558197991 conn(0x55d849143040 msgr2=0x55d84913de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:09.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.756+0000 7f0bd7825740 1 -- 172.21.15.172:0/3558197991 shutdown_connections 2024-04-14T08:41:09.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.757+0000 7f0bd7825740 1 -- 172.21.15.172:0/3558197991 wait complete. 2024-04-14T08:41:09.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.757+0000 7f0bd7825740 1 -- start start 2024-04-14T08:41:09.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.757+0000 7f0bd7825740 1 Processor -- start 2024-04-14T08:41:09.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bd7825740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 0x55d849210440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bd7825740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916ff40 0x55d84917b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bd7825740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d8491773e0 0x55d84917bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bd7825740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d849178970 con 0x55d84916fb60 2024-04-14T08:41:09.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bd7825740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d84917b130 con 0x55d84916ff40 2024-04-14T08:41:09.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bd7825740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d849178fc0 con 0x55d8491773e0 2024-04-14T08:41:09.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bd7825740 10 client.0 _ll_register_callbacks cb 0x7f0bc80122c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:09.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bce59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 0x55d849210440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bce59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 0x55d849210440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44800/0 (socket says 172.21.15.172:44800) 2024-04-14T08:41:09.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bce59c640 1 -- 172.21.15.172:0/191545045 learned_addr learned my addr 172.21.15.172:0/191545045 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:09.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bc7fff640 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916ff40 0x55d84917b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bced9d640 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d8491773e0 0x55d84917bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bce59c640 1 -- 172.21.15.172:0/191545045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d8491773e0 msgr2=0x55d84917bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:09.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bce59c640 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d8491773e0 0x55d84917bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bce59c640 1 -- 172.21.15.172:0/191545045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916ff40 msgr2=0x55d84917b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:09.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bce59c640 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916ff40 0x55d84917b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bce59c640 1 -- 172.21.15.172:0/191545045 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d849197c70 con 0x55d84916fb60 2024-04-14T08:41:09.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bce59c640 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 0x55d849210440 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f0bc004e000 tx=0x7f0bc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f14739cf32a385 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:09.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:41:09.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.758+0000 7f0bad7fa640 1 -- 172.21.15.172:0/191545045 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0061070 con 0x55d84916fb60 2024-04-14T08:41:09.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bad7fa640 1 -- 172.21.15.172:0/191545045 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f0bc0030780 con 0x55d84916fb60 2024-04-14T08:41:09.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bd7825740 1 -- 172.21.15.172:0/191545045 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb808a330 con 0x55d84916fb60 2024-04-14T08:41:09.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bad7fa640 1 -- 172.21.15.172:0/191545045 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f0bc0054040 con 0x55d84916fb60 2024-04-14T08:41:09.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bd7825740 10 client.8529 fetch_fsmap learned FSMap version 74 2024-04-14T08:41:09.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bd7825740 1 -- 172.21.15.172:0/191545045 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55d849210af0 con 0x55d84916fb60 2024-04-14T08:41:09.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bad7fa640 1 -- 172.21.15.172:0/191545045 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb808a330 con 0x55d84916fb60 2024-04-14T08:41:09.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bad7fa640 1 -- 172.21.15.172:0/191545045 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f0bc0005990 con 0x55d84916fb60 2024-04-14T08:41:09.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bd7825740 10 client.8529 fetch_fsmap finished waiting for FSMap version 74 2024-04-14T08:41:09.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bd7825740 10 client.8529 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-14T08:41:09.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bd7825740 20 client.8529 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:09.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bd7825740 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d8491912f0 0x55d8491738f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bd7825740 4 client.8529 mds_command: new command op to 8394 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:41:09.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bd7825740 1 -- 172.21.15.172:0/191545045 --> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] -- command(tid 0: {"prefix": "status"}) -- 0x55d8490b9570 con 0x55d8491912f0 2024-04-14T08:41:09.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bc7fff640 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d8491912f0 0x55d8491738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bad7fa640 10 client.8529 ms_handle_connect on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:09.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.759+0000 7f0bc7fff640 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d8491912f0 0x55d8491738f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2182b42111f59a36 server_cookie=7bcc6cefe4d8149b in_seq=0 out_seq=0 2024-04-14T08:41:09.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bad7fa640 1 -- 172.21.15.172:0/191545045 <== mds.0 v2:172.21.15.177:6834/2602337717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55d8490b9570 con 0x55d8491912f0 2024-04-14T08:41:09.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bad7fa640 10 client.8529 handle_command_reply: tid=0 2024-04-14T08:41:09.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd7825740 1 client.8529 shutdown 2024-04-14T08:41:09.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd7825740 1 -- 172.21.15.172:0/191545045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 msgr2=0x55d849210440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd7825740 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 0x55d849210440 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f0bc004e000 tx=0x7f0bc0004730 comp rx=0 tx=0).stop 2024-04-14T08:41:09.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd7825740 1 -- 172.21.15.172:0/191545045 shutdown_connections 2024-04-14T08:41:09.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd7825740 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d8491912f0 0x55d8491738f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd7825740 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d8491773e0 0x55d84917bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd7825740 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d84916ff40 0x55d84917b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bad7fa640 0 client.8529 ms_handle_reset on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:09.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd7825740 1 --2- 172.21.15.172:0/191545045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d84916fb60 0x55d849210440 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd7825740 1 -- 172.21.15.172:0/191545045 >> 172.21.15.172:0/191545045 conn(0x55d849143040 msgr2=0x55d84916ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:09.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd7825740 1 -- 172.21.15.172:0/191545045 shutdown_connections 2024-04-14T08:41:09.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd7825740 1 -- 172.21.15.172:0/191545045 wait complete. 2024-04-14T08:41:09.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd7825740 20 client.8529 trim_cache size 0 max 16384 2024-04-14T08:41:09.771 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:41:09.771 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:41:09.771 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:41:09.771 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 8394, 2024-04-14T08:41:09.771 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:41:09.771 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:41:09.771 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:41:09.772 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 4.624110333, 2024-04-14T08:41:09.772 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 74, 2024-04-14T08:41:09.772 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 101, 2024-04-14T08:41:09.772 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 101, 2024-04-14T08:41:09.772 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 9.6506838029999997 2024-04-14T08:41:09.772 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:41:09.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd495e640 1 -- 172.21.15.172:0/3377234167 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0bb006c8f0 msgr2=0x7f0bb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd495e640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0bb006c8f0 0x7f0bb006edb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0036000 tx=0x7f0bc0008030 comp rx=0 tx=0).stop 2024-04-14T08:41:09.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd495e640 1 -- 172.21.15.172:0/3377234167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bd0140770 msgr2=0x7f0bd00b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.760+0000 7f0bd495e640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bd0140770 0x7f0bd00b4b20 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800e870 tx=0x7f0bb800ed30 comp rx=0 tx=0).stop 2024-04-14T08:41:09.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.761+0000 7f0bd495e640 1 -- 172.21.15.172:0/3377234167 shutdown_connections 2024-04-14T08:41:09.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.761+0000 7f0bd495e640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0bb006c8f0 0x7f0bb006edb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.761+0000 7f0bd495e640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bd015c380 0x7f0bd00b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.761+0000 7f0bd495e640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bd0140770 0x7f0bd00b4b20 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.761+0000 7f0bd495e640 1 --2- 172.21.15.172:0/3377234167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0bd013df30 0x7f0bd01517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.761+0000 7f0bd495e640 1 -- 172.21.15.172:0/3377234167 >> 172.21.15.172:0/3377234167 conn(0x7f0bd006b520 msgr2=0x7f0bd00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:09.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.761+0000 7f0bd495e640 1 -- 172.21.15.172:0/3377234167 shutdown_connections 2024-04-14T08:41:09.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.761+0000 7f0bd495e640 1 -- 172.21.15.172:0/3377234167 wait complete. 2024-04-14T08:41:09.775 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:41:09.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.874+0000 7f4ec691e640 1 Processor -- start 2024-04-14T08:41:09.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.874+0000 7f4ec691e640 1 -- start start 2024-04-14T08:41:09.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.875+0000 7f4ec691e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec00d1a70 0x7f4ec00b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.875+0000 7f4ec691e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4ec00b4260 0x7f4ec006c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.875+0000 7f4ec691e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec00d1ed0 con 0x7f4ec015e390 2024-04-14T08:41:09.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.875+0000 7f4ec691e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec00d0080 con 0x7f4ec00b4260 2024-04-14T08:41:09.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.875+0000 7f4ec691e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec015e770 con 0x7f4ec00d1a70 2024-04-14T08:41:09.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.875+0000 7f4ebffff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f4ec015e390 0x7f4ec015f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:49144/0 (socket says 172.21.15.172:49144) 2024-04-14T08:41:09.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.875+0000 7f4ebffff640 1 -- 172.21.15.172:0/959149747 learned_addr learned my addr 172.21.15.172:0/959149747 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.875+0000 7f4ec4ebb640 1 --2- 172.21.15.172:0/959149747 >> v2:172.21.15.177:3300/0 conn(0x7f4ec00b4260 0x7f4ec006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.875+0000 7f4ebf7fe640 1 --2- 172.21.15.172:0/959149747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec00d1a70 0x7f4ec00b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.876+0000 7f4ebeffd640 1 -- 172.21.15.172:0/959149747 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4005425220 0 0) 0x7f4eb0001110 con 0x7f4ec015e390 2024-04-14T08:41:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.876+0000 7f4ebeffd640 1 -- 172.21.15.172:0/959149747 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ea8003520 con 0x7f4ec015e390 2024-04-14T08:41:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.876+0000 7f4ebf7fe640 1 -- 172.21.15.172:0/959149747 >> v2:172.21.15.177:3300/0 conn(0x7f4ec00b4260 msgr2=0x7f4ec006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.876+0000 7f4ebf7fe640 1 --2- 172.21.15.172:0/959149747 >> v2:172.21.15.177:3300/0 conn(0x7f4ec00b4260 0x7f4ec006c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.876+0000 7f4ebf7fe640 1 -- 172.21.15.172:0/959149747 >> v1:172.21.15.177:6789/0 conn(0x7f4ec015e390 legacy=0x7f4ec015f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.876+0000 7f4ebf7fe640 1 -- 172.21.15.172:0/959149747 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec006dba0 con 0x7f4ec00d1a70 2024-04-14T08:41:09.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.876+0000 7f4ec4ebb640 1 --2- 172.21.15.172:0/959149747 >> v2:172.21.15.177:3300/0 conn(0x7f4ec00b4260 0x7f4ec006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.876+0000 7f4ebf7fe640 1 --2- 172.21.15.172:0/959149747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec00d1a70 0x7f4ec00b3d20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40097a0 tx=0x7f4eb402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17fb9d54b03e53d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.877+0000 7f4ebeffd640 1 -- 172.21.15.172:0/959149747 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40315a0 con 0x7f4ec00d1a70 2024-04-14T08:41:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.877+0000 7f4ebeffd640 1 -- 172.21.15.172:0/959149747 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb4037070 con 0x7f4ec00d1a70 2024-04-14T08:41:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.877+0000 7f4ebeffd640 1 -- 172.21.15.172:0/959149747 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4031960 con 0x7f4ec00d1a70 2024-04-14T08:41:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.877+0000 7f4ec691e640 1 -- 172.21.15.172:0/959149747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec00d1a70 msgr2=0x7f4ec00b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.877+0000 7f4ec691e640 1 --2- 172.21.15.172:0/959149747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec00d1a70 0x7f4ec00b3d20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40097a0 tx=0x7f4eb402b840 comp rx=0 tx=0).stop 2024-04-14T08:41:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.877+0000 7f4ec691e640 1 -- 172.21.15.172:0/959149747 shutdown_connections 2024-04-14T08:41:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.877+0000 7f4ec691e640 1 --2- 172.21.15.172:0/959149747 >> v2:172.21.15.177:3300/0 conn(0x7f4ec00b4260 0x7f4ec006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.877+0000 7f4ec691e640 1 --2- 172.21.15.172:0/959149747 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec00d1a70 0x7f4ec00b3d20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.877+0000 7f4ec691e640 1 -- 172.21.15.172:0/959149747 >> 172.21.15.172:0/959149747 conn(0x7f4ec006f820 msgr2=0x7f4ec006fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:09.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.877+0000 7f4ec691e640 1 -- 172.21.15.172:0/959149747 shutdown_connections 2024-04-14T08:41:09.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.877+0000 7f4ec691e640 1 -- 172.21.15.172:0/959149747 wait complete. 2024-04-14T08:41:09.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.878+0000 7f4ec691e640 1 Processor -- start 2024-04-14T08:41:09.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.878+0000 7f4ec691e640 1 -- start start 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.879+0000 7f4ec691e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ec00b4260 0x7f4ec00b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.879+0000 7f4ec691e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec00d1a70 0x7f4ec014d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.879+0000 7f4ec691e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ec015e390 0x7f4ec01f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.879+0000 7f4ec691e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec006cb70 con 0x7f4ec00b4260 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.879+0000 7f4ec691e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec00d0510 con 0x7f4ec015e390 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.879+0000 7f4ec691e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec006ccf0 con 0x7f4ec00d1a70 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.879+0000 7f4ebffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ec00b4260 0x7f4ec00b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.879+0000 7f4ebffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ec00b4260 0x7f4ec00b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44814/0 (socket says 172.21.15.172:44814) 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.879+0000 7f4ebffff640 1 -- 172.21.15.172:0/1628451191 learned_addr learned my addr 172.21.15.172:0/1628451191 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.879+0000 7f4ec4ebb640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ec015e390 0x7f4ec01f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.879+0000 7f4ebf7fe640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec00d1a70 0x7f4ec014d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.880+0000 7f4ebffff640 1 -- 172.21.15.172:0/1628451191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec00d1a70 msgr2=0x7f4ec014d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.880+0000 7f4ebffff640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec00d1a70 0x7f4ec014d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.880+0000 7f4ebffff640 1 -- 172.21.15.172:0/1628451191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ec015e390 msgr2=0x7f4ec01f64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.880+0000 7f4ebffff640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ec015e390 0x7f4ec01f64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.880+0000 7f4ebffff640 1 -- 172.21.15.172:0/1628451191 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec01f5720 con 0x7f4ec00b4260 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.880+0000 7f4ebffff640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ec00b4260 0x7f4ec00b13f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000c3d0 tx=0x7f4eb000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.880+0000 7f4ebcff9640 1 -- 172.21.15.172:0/1628451191 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0017070 con 0x7f4ec00b4260 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.880+0000 7f4ebcff9640 1 -- 172.21.15.172:0/1628451191 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb0005ce0 con 0x7f4ec00b4260 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.880+0000 7f4ebcff9640 1 -- 172.21.15.172:0/1628451191 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0007310 con 0x7f4ec00b4260 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.880+0000 7f4ec691e640 1 -- 172.21.15.172:0/1628451191 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec01f5a00 con 0x7f4ec00b4260 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.880+0000 7f4ec691e640 1 -- 172.21.15.172:0/1628451191 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec01f5fc0 con 0x7f4ec00b4260 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.881+0000 7f4ec691e640 1 -- 172.21.15.172:0/1628451191 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec0068520 con 0x7f4ec00b4260 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.881+0000 7f4ebcff9640 1 -- 172.21.15.172:0/1628451191 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4eb001a070 con 0x7f4ec00b4260 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.882+0000 7f4ebcff9640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4eac06c920 0x7f4eac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:09.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.882+0000 7f4ebcff9640 1 -- 172.21.15.172:0/1628451191 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f4eb008b000 con 0x7f4ec00b4260 2024-04-14T08:41:09.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.882+0000 7f4ebf7fe640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4eac06c920 0x7f4eac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:09.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.882+0000 7f4ebf7fe640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4eac06c920 0x7f4eac06ede0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4036000 tx=0x7f4eb403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:09.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:09.886+0000 7f4ebcff9640 1 -- 172.21.15.172:0/1628451191 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4ec00d1ed0 con 0x7f4ec00b4260 2024-04-14T08:41:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.098+0000 7f4ec691e640 1 -- 172.21.15.172:0/1628451191 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4ec0071840 con 0x7f4ec00b4260 2024-04-14T08:41:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.098+0000 7f4ebcff9640 1 -- 172.21.15.172:0/1628451191 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f4eb0058bc0 con 0x7f4ec00b4260 2024-04-14T08:41:10.100 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:41:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.100+0000 7f4ec691e640 1 -- 172.21.15.172:0/1628451191 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4eac06c920 msgr2=0x7f4eac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.100+0000 7f4ec691e640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4eac06c920 0x7f4eac06ede0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4036000 tx=0x7f4eb403d040 comp rx=0 tx=0).stop 2024-04-14T08:41:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.100+0000 7f4ec691e640 1 -- 172.21.15.172:0/1628451191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ec00b4260 msgr2=0x7f4ec00b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.100+0000 7f4ec691e640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ec00b4260 0x7f4ec00b13f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000c3d0 tx=0x7f4eb000c7d0 comp rx=0 tx=0).stop 2024-04-14T08:41:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.100+0000 7f4ec691e640 1 -- 172.21.15.172:0/1628451191 shutdown_connections 2024-04-14T08:41:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.100+0000 7f4ec691e640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4eac06c920 0x7f4eac06ede0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.100+0000 7f4ec691e640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ec015e390 0x7f4ec01f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.100+0000 7f4ec691e640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec00d1a70 0x7f4ec014d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.100+0000 7f4ec691e640 1 --2- 172.21.15.172:0/1628451191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ec00b4260 0x7f4ec00b13f0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.100+0000 7f4ec691e640 1 -- 172.21.15.172:0/1628451191 >> 172.21.15.172:0/1628451191 conn(0x7f4ec006f820 msgr2=0x7f4ec00b71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.100+0000 7f4ec691e640 1 -- 172.21.15.172:0/1628451191 shutdown_connections 2024-04-14T08:41:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10.100+0000 7f4ec691e640 1 -- 172.21.15.172:0/1628451191 wait complete. 2024-04-14T08:41:10.112 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:41:10.112 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:41:10.112 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:41:10.113 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:41:10.113 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:41:10.113 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:41:10.113 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:41:10.113 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:41:10.113 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:41:10.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:10.113 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:41:10.129 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:41:10.129 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:41:10.129 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:41:10.129 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:10.129 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:41:10.129 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42199sec preferred_lft 42199sec 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:41:10.130 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:41:10.131 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:10.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:10.132 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:41:10.198 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:41:10.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:10.201 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:41:10.257 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:41:10.257 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:41:10.258 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:41:10.258 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:41:10.258 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:41:10.258 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:41:10.258 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:41:10.258 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:41:10.258 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:41:10.259 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:41:10.259 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:41:10.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:41:10.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:41:10.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:10.393 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:41:10.408 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:41:10.408 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:41:10.408 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:41:10.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:10.408 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:41:10.476 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:41:10.476 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:41:10.531 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:41:10.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:41:10.532 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:41:10.586 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:41:10.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:10.587 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:41:10.671 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:41:10.672 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:41:10.672 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:41:10.680 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:41:10.680 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:41:10.680 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:41:10.680 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:41:10.680 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:41:10.698 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:41:10.698 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:41:10.698 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:41:10.698 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:41:10.698 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:41:10.698 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:41:10.698 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:41:10.699 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:41:10.699 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:41:10.699 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:41:10.699 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:41:10.699 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:41:10.699 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:41:10.700 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:41:10.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:10.701 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:41:10.728 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:41:10.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:41:10.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:41:10.829 INFO:teuthology.orchestra.run.smithi172.stdout:client8535 2024-04-14T08:41:10.829 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:41:10.829 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:41:10.844 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:41:10.844 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:41:10.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:10.900 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-14T08:41:10.996 INFO:teuthology.orchestra.run.smithi172.stderr:6+0 records in 2024-04-14T08:41:10.996 INFO:teuthology.orchestra.run.smithi172.stderr:6+0 records out 2024-04-14T08:41:10.996 INFO:teuthology.orchestra.run.smithi172.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0371723 s, 169 MB/s 2024-04-14T08:41:10.997 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:41:10.997 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:41:11.012 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:41:11.013 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:41:11.068 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:41:11.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.208+0000 7f87a5d9c640 1 Processor -- start 2024-04-14T08:41:11.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.208+0000 7f87a5d9c640 1 -- start start 2024-04-14T08:41:11.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.209+0000 7f87a5d9c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87a015f400 0x7f87a015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.209+0000 7f87a5d9c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f87a00b1b80 0x7f87a00b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.209+0000 7f87a5d9c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87a013f600 con 0x7f87a015d9d0 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.209+0000 7f87a5d9c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f87a00b2520 con 0x7f87a00b1b80 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.209+0000 7f87a5d9c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f87a0140690 con 0x7f87a015f400 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.209+0000 7f879ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f87a00b1b80 0x7f87a00b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.209+0000 7f879ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f87a00b1b80 0x7f87a00b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44830/0 (socket says 172.21.15.172:44830) 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.209+0000 7f879ffff640 1 -- 172.21.15.172:0/2857534078 learned_addr learned my addr 172.21.15.172:0/2857534078 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.209+0000 7f879f7fe640 1 --2- 172.21.15.172:0/2857534078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87a015f400 0x7f87a015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.210+0000 7f879ffff640 1 -- 172.21.15.172:0/2857534078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87a015f400 msgr2=0x7f87a015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.210+0000 7f879ffff640 1 --2- 172.21.15.172:0/2857534078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87a015f400 0x7f87a015f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.210+0000 7f879ffff640 1 -- 172.21.15.172:0/2857534078 >> v1:172.21.15.177:6789/0 conn(0x7f87a015d9d0 legacy=0x7f87a015d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.210+0000 7f879ffff640 1 -- 172.21.15.172:0/2857534078 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a013fa80 con 0x7f87a00b1b80 2024-04-14T08:41:11.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.210+0000 7f879e7fc640 1 -- 172.21.15.172:0/2857534078 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1939482105 0 0) 0x7f8794001110 con 0x7f87a015d9d0 2024-04-14T08:41:11.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.210+0000 7f879ffff640 1 --2- 172.21.15.172:0/2857534078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a00b1b80 0x7f87a00b1fe0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8798009e50 tx=0x7f879802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39b03ccddc417be server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:11.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.210+0000 7f879e7fc640 1 -- 172.21.15.172:0/2857534078 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879802cb00 con 0x7f87a00b1b80 2024-04-14T08:41:11.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.210+0000 7f879e7fc640 1 -- 172.21.15.172:0/2857534078 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879802cca0 con 0x7f87a00b1b80 2024-04-14T08:41:11.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.210+0000 7f879e7fc640 1 -- 172.21.15.172:0/2857534078 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798034610 con 0x7f87a00b1b80 2024-04-14T08:41:11.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.211+0000 7f87a5d9c640 1 -- 172.21.15.172:0/2857534078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a00b1b80 msgr2=0x7f87a00b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.211+0000 7f87a5d9c640 1 --2- 172.21.15.172:0/2857534078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a00b1b80 0x7f87a00b1fe0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8798009e50 tx=0x7f879802bf60 comp rx=0 tx=0).stop 2024-04-14T08:41:11.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.211+0000 7f87a5d9c640 1 -- 172.21.15.172:0/2857534078 shutdown_connections 2024-04-14T08:41:11.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.211+0000 7f87a5d9c640 1 --2- 172.21.15.172:0/2857534078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a00b1b80 0x7f87a00b1fe0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.211+0000 7f87a5d9c640 1 --2- 172.21.15.172:0/2857534078 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87a015f400 0x7f87a015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.211+0000 7f87a5d9c640 1 -- 172.21.15.172:0/2857534078 >> 172.21.15.172:0/2857534078 conn(0x7f87a006ec30 msgr2=0x7f87a006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:11.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.211+0000 7f87a5d9c640 1 -- 172.21.15.172:0/2857534078 shutdown_connections 2024-04-14T08:41:11.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.211+0000 7f87a5d9c640 1 -- 172.21.15.172:0/2857534078 wait complete. 2024-04-14T08:41:11.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.212+0000 7f87a5d9c640 1 Processor -- start 2024-04-14T08:41:11.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.212+0000 7f87a5d9c640 1 -- start start 2024-04-14T08:41:11.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.213+0000 7f87a5d9c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87a00b1b80 0x7f87a0154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.213+0000 7f87a5d9c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87a015d9d0 0x7f87a0156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.213+0000 7f87a5d9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a015f400 0x7f87a0158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.213+0000 7f87a5d9c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0140320 con 0x7f87a015f400 2024-04-14T08:41:11.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.213+0000 7f87a5d9c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0140040 con 0x7f87a00b1b80 2024-04-14T08:41:11.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.213+0000 7f87a5d9c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f87a015de30 con 0x7f87a015d9d0 2024-04-14T08:41:11.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.213+0000 7f879ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a015f400 0x7f87a0158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.213+0000 7f879ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a015f400 0x7f87a0158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44842/0 (socket says 172.21.15.172:44842) 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.213+0000 7f879ffff640 1 -- 172.21.15.172:0/909805601 learned_addr learned my addr 172.21.15.172:0/909805601 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.213+0000 7f879effd640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87a015d9d0 0x7f87a0156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.213+0000 7f879f7fe640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87a00b1b80 0x7f87a0154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.213+0000 7f879ffff640 1 -- 172.21.15.172:0/909805601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87a015d9d0 msgr2=0x7f87a0156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.214+0000 7f879ffff640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87a015d9d0 0x7f87a0156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.214+0000 7f879ffff640 1 -- 172.21.15.172:0/909805601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87a00b1b80 msgr2=0x7f87a0154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.214+0000 7f879ffff640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87a00b1b80 0x7f87a0154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.214+0000 7f879ffff640 1 -- 172.21.15.172:0/909805601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a0155240 con 0x7f87a015f400 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.214+0000 7f879f7fe640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87a00b1b80 0x7f87a0154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.214+0000 7f879ffff640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a015f400 0x7f87a0158d00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8798036040 tx=0x7f8798004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.214+0000 7f8787fff640 1 -- 172.21.15.172:0/909805601 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798043020 con 0x7f87a015f400 2024-04-14T08:41:11.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.214+0000 7f8787fff640 1 -- 172.21.15.172:0/909805601 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879802ce50 con 0x7f87a015f400 2024-04-14T08:41:11.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.214+0000 7f87a5d9c640 1 -- 172.21.15.172:0/909805601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a015bb50 con 0x7f87a015f400 2024-04-14T08:41:11.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.214+0000 7f8787fff640 1 -- 172.21.15.172:0/909805601 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798009510 con 0x7f87a015f400 2024-04-14T08:41:11.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.214+0000 7f87a5d9c640 1 -- 172.21.15.172:0/909805601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a015c0d0 con 0x7f87a015f400 2024-04-14T08:41:11.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.215+0000 7f87a5d9c640 1 -- 172.21.15.172:0/909805601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a0068520 con 0x7f87a015f400 2024-04-14T08:41:11.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.215+0000 7f8787fff640 1 -- 172.21.15.172:0/909805601 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8798046020 con 0x7f87a015f400 2024-04-14T08:41:11.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.216+0000 7f8787fff640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f878c06c8f0 0x7f878c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.216+0000 7f8787fff640 1 -- 172.21.15.172:0/909805601 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f87980ae180 con 0x7f87a015f400 2024-04-14T08:41:11.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.216+0000 7f879f7fe640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f878c06c8f0 0x7f878c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.216+0000 7f879f7fe640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f878c06c8f0 0x7f878c06edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8790009e90 tx=0x7f8790009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:11.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.220+0000 7f8787fff640 1 -- 172.21.15.172:0/909805601 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f879804d030 con 0x7f87a015f400 2024-04-14T08:41:11.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.398+0000 7f87a5d9c640 1 -- 172.21.15.172:0/909805601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87a006c2c0 con 0x7f87a015f400 2024-04-14T08:41:11.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.399+0000 7f8787fff640 1 -- 172.21.15.172:0/909805601 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16936 (secure 0 0 0) 0x7f879807bde0 con 0x7f87a015f400 2024-04-14T08:41:11.401 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:11.401 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":101,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:41:06.143425+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:41:02.725765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-14T08:41:03.442157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:41:11.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.401+0000 7f87a5d9c640 1 -- 172.21.15.172:0/909805601 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f878c06c8f0 msgr2=0x7f878c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.401+0000 7f87a5d9c640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f878c06c8f0 0x7f878c06edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8790009e90 tx=0x7f8790009190 comp rx=0 tx=0).stop 2024-04-14T08:41:11.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.401+0000 7f87a5d9c640 1 -- 172.21.15.172:0/909805601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a015f400 msgr2=0x7f87a0158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.401+0000 7f87a5d9c640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a015f400 0x7f87a0158d00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8798036040 tx=0x7f8798004a40 comp rx=0 tx=0).stop 2024-04-14T08:41:11.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.401+0000 7f87a5d9c640 1 -- 172.21.15.172:0/909805601 shutdown_connections 2024-04-14T08:41:11.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.401+0000 7f87a5d9c640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f878c06c8f0 0x7f878c06edb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.401+0000 7f87a5d9c640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87a015f400 0x7f87a0158d00 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.401+0000 7f87a5d9c640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87a015d9d0 0x7f87a0156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.401+0000 7f87a5d9c640 1 --2- 172.21.15.172:0/909805601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87a00b1b80 0x7f87a0154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.401+0000 7f87a5d9c640 1 -- 172.21.15.172:0/909805601 >> 172.21.15.172:0/909805601 conn(0x7f87a006ec30 msgr2=0x7f87a015e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:11.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.402+0000 7f87a5d9c640 1 -- 172.21.15.172:0/909805601 shutdown_connections 2024-04-14T08:41:11.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.402+0000 7f87a5d9c640 1 -- 172.21.15.172:0/909805601 wait complete. 2024-04-14T08:41:11.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:41:02.725765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:41:03.442157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:41:11.415 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:41:11.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.518+0000 7fcc0f75e640 1 Processor -- start 2024-04-14T08:41:11.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.518+0000 7fcc0f75e640 1 -- start start 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0f75e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcc08159c20 0x7fcc080ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0f75e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcc0815c700 0x7fcc081603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0f75e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc0815cf10 con 0x7fcc08160930 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0f75e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fcc0815fc80 con 0x7fcc08159c20 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0f75e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fcc0815ff60 con 0x7fcc0815c700 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0d4fa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcc08159c20 0x7fcc080ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0d4fa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcc08159c20 0x7fcc080ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44856/0 (socket says 172.21.15.172:44856) 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0d4fa640 1 -- 172.21.15.172:0/532020470 learned_addr learned my addr 172.21.15.172:0/532020470 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0ccf9640 1 --2- 172.21.15.172:0/532020470 >> v2:172.21.15.177:3300/0 conn(0x7fcc0815c700 0x7fcc081603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0d4fa640 1 -- 172.21.15.172:0/532020470 >> v2:172.21.15.177:3300/0 conn(0x7fcc0815c700 msgr2=0x7fcc081603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0d4fa640 1 --2- 172.21.15.172:0/532020470 >> v2:172.21.15.177:3300/0 conn(0x7fcc0815c700 0x7fcc081603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0d4fa640 1 -- 172.21.15.172:0/532020470 >> v1:172.21.15.177:6789/0 conn(0x7fcc08160930 legacy=0x7fcc08072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.519+0000 7fcc0d4fa640 1 -- 172.21.15.172:0/532020470 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0815d1f0 con 0x7fcc08159c20 2024-04-14T08:41:11.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.520+0000 7fcc0d4fa640 1 --2- 172.21.15.172:0/532020470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc08159c20 0x7fcc080ce200 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fcc08072190 tx=0x7fcc0402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43a84320a93991b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:11.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.520+0000 7fcbfffff640 1 -- 172.21.15.172:0/532020470 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04034070 con 0x7fcc08159c20 2024-04-14T08:41:11.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.520+0000 7fcbfffff640 1 -- 172.21.15.172:0/532020470 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc04030860 con 0x7fcc08159c20 2024-04-14T08:41:11.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.520+0000 7fcbfffff640 1 -- 172.21.15.172:0/532020470 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04030c30 con 0x7fcc08159c20 2024-04-14T08:41:11.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.521+0000 7fcc0f75e640 1 -- 172.21.15.172:0/532020470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc08159c20 msgr2=0x7fcc080ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.521+0000 7fcc0f75e640 1 --2- 172.21.15.172:0/532020470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc08159c20 0x7fcc080ce200 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fcc08072190 tx=0x7fcc0402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.521+0000 7fcc0f75e640 1 -- 172.21.15.172:0/532020470 shutdown_connections 2024-04-14T08:41:11.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.521+0000 7fcc0f75e640 1 --2- 172.21.15.172:0/532020470 >> v2:172.21.15.177:3300/0 conn(0x7fcc0815c700 0x7fcc081603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.521+0000 7fcc0f75e640 1 --2- 172.21.15.172:0/532020470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc08159c20 0x7fcc080ce200 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.521+0000 7fcc0f75e640 1 -- 172.21.15.172:0/532020470 >> 172.21.15.172:0/532020470 conn(0x7fcc0806b520 msgr2=0x7fcc0806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:11.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.521+0000 7fcc0f75e640 1 -- 172.21.15.172:0/532020470 shutdown_connections 2024-04-14T08:41:11.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.521+0000 7fcc0f75e640 1 -- 172.21.15.172:0/532020470 wait complete. 2024-04-14T08:41:11.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.522+0000 7fcc0f75e640 1 Processor -- start 2024-04-14T08:41:11.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.522+0000 7fcc0f75e640 1 -- start start 2024-04-14T08:41:11.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.522+0000 7fcc0f75e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc08159c20 0x7fcc08154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.522+0000 7fcc0f75e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc0815c700 0x7fcc0815b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0f75e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc081528f0 0x7fcc080b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0f75e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0815cb80 con 0x7fcc081528f0 2024-04-14T08:41:11.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0f75e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0815a000 con 0x7fcc0815c700 2024-04-14T08:41:11.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0f75e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcc08072b60 con 0x7fcc08159c20 2024-04-14T08:41:11.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0dcfb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc081528f0 0x7fcc080b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0dcfb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc081528f0 0x7fcc080b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44866/0 (socket says 172.21.15.172:44866) 2024-04-14T08:41:11.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0dcfb640 1 -- 172.21.15.172:0/1821613982 learned_addr learned my addr 172.21.15.172:0/1821613982 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:11.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0ccf9640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc0815c700 0x7fcc0815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0d4fa640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc08159c20 0x7fcc08154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0dcfb640 1 -- 172.21.15.172:0/1821613982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc08159c20 msgr2=0x7fcc08154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0dcfb640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc08159c20 0x7fcc08154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0dcfb640 1 -- 172.21.15.172:0/1821613982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc0815c700 msgr2=0x7fcc0815b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0dcfb640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc0815c700 0x7fcc0815b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0dcfb640 1 -- 172.21.15.172:0/1821613982 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0815bc70 con 0x7fcc081528f0 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0dcfb640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc081528f0 0x7fcc080b1e80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000c6b0 tx=0x7fcc0000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcbfdffb640 1 -- 172.21.15.172:0/1821613982 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00003b70 con 0x7fcc081528f0 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcbfdffb640 1 -- 172.21.15.172:0/1821613982 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc00003d30 con 0x7fcc081528f0 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0f75e640 1 -- 172.21.15.172:0/1821613982 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc081547b0 con 0x7fcc081528f0 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcbfdffb640 1 -- 172.21.15.172:0/1821613982 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc00005310 con 0x7fcc081528f0 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.523+0000 7fcc0f75e640 1 -- 172.21.15.172:0/1821613982 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc08154d70 con 0x7fcc081528f0 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.525+0000 7fcc0f75e640 1 -- 172.21.15.172:0/1821613982 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc08068590 con 0x7fcc081528f0 2024-04-14T08:41:11.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.525+0000 7fcbfdffb640 1 -- 172.21.15.172:0/1821613982 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc000054b0 con 0x7fcc081528f0 2024-04-14T08:41:11.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.525+0000 7fcbfdffb640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcbec06c920 0x7fcbec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.525+0000 7fcbfdffb640 1 -- 172.21.15.172:0/1821613982 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7fcc0008ba70 con 0x7fcc081528f0 2024-04-14T08:41:11.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.525+0000 7fcc0d4fa640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcbec06c920 0x7fcbec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.526+0000 7fcc0d4fa640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcbec06c920 0x7fcbec06ede0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcc04036000 tx=0x7fcc04008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:11.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.530+0000 7fcbfdffb640 1 -- 172.21.15.172:0/1821613982 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcc00059630 con 0x7fcc081528f0 2024-04-14T08:41:11.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.702+0000 7fcc0f75e640 1 -- 172.21.15.172:0/1821613982 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcc08071e50 con 0x7fcc081528f0 2024-04-14T08:41:11.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.703+0000 7fcbfdffb640 1 -- 172.21.15.172:0/1821613982 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7fcc0005d930 con 0x7fcc081528f0 2024-04-14T08:41:11.704 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:41:11.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.705+0000 7fcc0f75e640 1 -- 172.21.15.172:0/1821613982 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcbec06c920 msgr2=0x7fcbec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.705+0000 7fcc0f75e640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcbec06c920 0x7fcbec06ede0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcc04036000 tx=0x7fcc04008030 comp rx=0 tx=0).stop 2024-04-14T08:41:11.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.705+0000 7fcc0f75e640 1 -- 172.21.15.172:0/1821613982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc081528f0 msgr2=0x7fcc080b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.705+0000 7fcc0f75e640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc081528f0 0x7fcc080b1e80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000c6b0 tx=0x7fcc0000cb70 comp rx=0 tx=0).stop 2024-04-14T08:41:11.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.705+0000 7fcc0f75e640 1 -- 172.21.15.172:0/1821613982 shutdown_connections 2024-04-14T08:41:11.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.705+0000 7fcc0f75e640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcbec06c920 0x7fcbec06ede0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.705+0000 7fcc0f75e640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc081528f0 0x7fcc080b1e80 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.705+0000 7fcc0f75e640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc0815c700 0x7fcc0815b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.705+0000 7fcc0f75e640 1 --2- 172.21.15.172:0/1821613982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc08159c20 0x7fcc08154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.705+0000 7fcc0f75e640 1 -- 172.21.15.172:0/1821613982 >> 172.21.15.172:0/1821613982 conn(0x7fcc0806b520 msgr2=0x7fcc081581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:11.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.705+0000 7fcc0f75e640 1 -- 172.21.15.172:0/1821613982 shutdown_connections 2024-04-14T08:41:11.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.705+0000 7fcc0f75e640 1 -- 172.21.15.172:0/1821613982 wait complete. 2024-04-14T08:41:11.717 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:41:11.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.820+0000 7f8c46060640 1 Processor -- start 2024-04-14T08:41:11.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.820+0000 7f8c46060640 1 -- start start 2024-04-14T08:41:11.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c46060640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8c40140fa0 0x7f8c40162840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c46060640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8c40141380 0x7f8c400b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c46060640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c400cf9b0 con 0x7f8c400b2210 2024-04-14T08:41:11.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c46060640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f8c400b2680 con 0x7f8c40141380 2024-04-14T08:41:11.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c46060640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f8c40162d80 con 0x7f8c40140fa0 2024-04-14T08:41:11.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c3effd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8c40141380 0x7f8c400b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c3ffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f8c400b2210 0x7f8c400adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:60404/0 (socket says 172.21.15.172:60404) 2024-04-14T08:41:11.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c3effd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8c40141380 0x7f8c400b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44868/0 (socket says 172.21.15.172:44868) 2024-04-14T08:41:11.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c3ffff640 1 -- 172.21.15.172:0/2300534453 learned_addr learned my addr 172.21.15.172:0/2300534453 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:11.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c3f7fe640 1 --2- 172.21.15.172:0/2300534453 >> v2:172.21.15.177:3300/0 conn(0x7f8c40140fa0 0x7f8c40162840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c3effd640 1 -- 172.21.15.172:0/2300534453 >> v2:172.21.15.177:3300/0 conn(0x7f8c40140fa0 msgr2=0x7f8c40162840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:11.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c3effd640 1 --2- 172.21.15.172:0/2300534453 >> v2:172.21.15.177:3300/0 conn(0x7f8c40140fa0 0x7f8c40162840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c3effd640 1 -- 172.21.15.172:0/2300534453 >> v1:172.21.15.172:6789/0 conn(0x7f8c400b2210 legacy=0x7f8c400adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c3e7fc640 1 -- 172.21.15.172:0/2300534453 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3496037638 0 0) 0x7f8c400cf9b0 con 0x7f8c400b2210 2024-04-14T08:41:11.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.821+0000 7f8c3effd640 1 -- 172.21.15.172:0/2300534453 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4015ecc0 con 0x7f8c40141380 2024-04-14T08:41:11.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.822+0000 7f8c3effd640 1 --2- 172.21.15.172:0/2300534453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c40141380 0x7f8c400b1cd0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f8c4006f660 tx=0x7f8c3402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0be4ce858798ba server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:11.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.822+0000 7f8c3e7fc640 1 -- 172.21.15.172:0/2300534453 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c340315a0 con 0x7f8c40141380 2024-04-14T08:41:11.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.822+0000 7f8c3e7fc640 1 -- 172.21.15.172:0/2300534453 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c34037070 con 0x7f8c40141380 2024-04-14T08:41:11.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.822+0000 7f8c3e7fc640 1 -- 172.21.15.172:0/2300534453 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34031960 con 0x7f8c40141380 2024-04-14T08:41:11.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.823+0000 7f8c46060640 1 -- 172.21.15.172:0/2300534453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c40141380 msgr2=0x7f8c400b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.823+0000 7f8c46060640 1 --2- 172.21.15.172:0/2300534453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c40141380 0x7f8c400b1cd0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f8c4006f660 tx=0x7f8c3402b840 comp rx=0 tx=0).stop 2024-04-14T08:41:11.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.823+0000 7f8c46060640 1 -- 172.21.15.172:0/2300534453 shutdown_connections 2024-04-14T08:41:11.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.823+0000 7f8c46060640 1 --2- 172.21.15.172:0/2300534453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c40141380 0x7f8c400b1cd0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.823+0000 7f8c46060640 1 --2- 172.21.15.172:0/2300534453 >> v2:172.21.15.177:3300/0 conn(0x7f8c40140fa0 0x7f8c40162840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.823+0000 7f8c46060640 1 -- 172.21.15.172:0/2300534453 >> 172.21.15.172:0/2300534453 conn(0x7f8c4006b520 msgr2=0x7f8c4006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:11.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.823+0000 7f8c46060640 1 -- 172.21.15.172:0/2300534453 shutdown_connections 2024-04-14T08:41:11.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.823+0000 7f8c46060640 1 -- 172.21.15.172:0/2300534453 wait complete. 2024-04-14T08:41:11.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.824+0000 7f8c46060640 1 Processor -- start 2024-04-14T08:41:11.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.824+0000 7f8c46060640 1 -- start start 2024-04-14T08:41:11.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c46060640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c400b2210 0x7f8c40157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c46060640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c40140fa0 0x7f8c40071770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c46060640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c40141380 0x7f8c40071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c46060640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4015f560 con 0x7f8c400b2210 2024-04-14T08:41:11.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c46060640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4015f280 con 0x7f8c40141380 2024-04-14T08:41:11.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c46060640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8c400af3e0 con 0x7f8c40140fa0 2024-04-14T08:41:11.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c3f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c400b2210 0x7f8c40157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c3f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c400b2210 0x7f8c40157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44874/0 (socket says 172.21.15.172:44874) 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c3f7fe640 1 -- 172.21.15.172:0/2931346 learned_addr learned my addr 172.21.15.172:0/2931346 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c3ffff640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c40141380 0x7f8c40071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c3effd640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c40140fa0 0x7f8c40071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c3f7fe640 1 -- 172.21.15.172:0/2931346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c40140fa0 msgr2=0x7f8c40071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c3f7fe640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c40140fa0 0x7f8c40071770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c3f7fe640 1 -- 172.21.15.172:0/2931346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c40141380 msgr2=0x7f8c40071cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c3f7fe640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c40141380 0x7f8c40071cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c3f7fe640 1 -- 172.21.15.172:0/2931346 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c401f7a20 con 0x7f8c400b2210 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c3effd640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c40140fa0 0x7f8c40071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c3f7fe640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c400b2210 0x7f8c40157c20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000e960 tx=0x7f8c3000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.825+0000 7f8c27fff640 1 -- 172.21.15.172:0/2931346 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30017070 con 0x7f8c400b2210 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.826+0000 7f8c27fff640 1 -- 172.21.15.172:0/2931346 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3000ca00 con 0x7f8c400b2210 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.826+0000 7f8c46060640 1 -- 172.21.15.172:0/2931346 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c40139320 con 0x7f8c400b2210 2024-04-14T08:41:11.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.826+0000 7f8c27fff640 1 -- 172.21.15.172:0/2931346 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3000cdf0 con 0x7f8c400b2210 2024-04-14T08:41:11.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.826+0000 7f8c46060640 1 -- 172.21.15.172:0/2931346 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c401f7fa0 con 0x7f8c400b2210 2024-04-14T08:41:11.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.827+0000 7f8c46060640 1 -- 172.21.15.172:0/2931346 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c40068590 con 0x7f8c400b2210 2024-04-14T08:41:11.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.827+0000 7f8c27fff640 1 -- 172.21.15.172:0/2931346 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c3001a070 con 0x7f8c400b2210 2024-04-14T08:41:11.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.827+0000 7f8c27fff640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8c2c06c920 0x7f8c2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:11.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.828+0000 7f8c3effd640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8c2c06c920 0x7f8c2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:11.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.828+0000 7f8c27fff640 1 -- 172.21.15.172:0/2931346 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f8c3008bc20 con 0x7f8c400b2210 2024-04-14T08:41:11.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.828+0000 7f8c3effd640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8c2c06c920 0x7f8c2c06ede0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8c34036000 tx=0x7f8c3403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:11.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:11.832+0000 7f8c27fff640 1 -- 172.21.15.172:0/2931346 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8c30059900 con 0x7f8c400b2210 2024-04-14T08:41:12.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.009+0000 7f8c46060640 1 -- 172.21.15.172:0/2931346 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8c400b5f50 con 0x7f8c400b2210 2024-04-14T08:41:12.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.010+0000 7f8c27fff640 1 -- 172.21.15.172:0/2931346 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f8c3005dc00 con 0x7f8c400b2210 2024-04-14T08:41:12.012 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:41:12.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.012+0000 7f8c46060640 1 -- 172.21.15.172:0/2931346 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8c2c06c920 msgr2=0x7f8c2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.012+0000 7f8c46060640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8c2c06c920 0x7f8c2c06ede0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8c34036000 tx=0x7f8c3403d040 comp rx=0 tx=0).stop 2024-04-14T08:41:12.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.012+0000 7f8c46060640 1 -- 172.21.15.172:0/2931346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c400b2210 msgr2=0x7f8c40157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.012+0000 7f8c46060640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c400b2210 0x7f8c40157c20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000e960 tx=0x7f8c3000ee20 comp rx=0 tx=0).stop 2024-04-14T08:41:12.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.012+0000 7f8c46060640 1 -- 172.21.15.172:0/2931346 shutdown_connections 2024-04-14T08:41:12.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.012+0000 7f8c46060640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8c2c06c920 0x7f8c2c06ede0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.012+0000 7f8c46060640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c40141380 0x7f8c40071cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.012+0000 7f8c46060640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c40140fa0 0x7f8c40071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.012+0000 7f8c46060640 1 --2- 172.21.15.172:0/2931346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c400b2210 0x7f8c40157c20 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.012+0000 7f8c46060640 1 -- 172.21.15.172:0/2931346 >> 172.21.15.172:0/2931346 conn(0x7f8c4006b520 msgr2=0x7f8c4013f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:12.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.012+0000 7f8c46060640 1 -- 172.21.15.172:0/2931346 shutdown_connections 2024-04-14T08:41:12.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.012+0000 7f8c46060640 1 -- 172.21.15.172:0/2931346 wait complete. 2024-04-14T08:41:12.025 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:41:12.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.128+0000 7f1ffdffe640 1 Processor -- start 2024-04-14T08:41:12.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.128+0000 7f1ffdffe640 1 -- start start 2024-04-14T08:41:12.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ffdffe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ff813df30 0x7f1ff80b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ffdffe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1ff815c380 0x7f1ff80b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ffdffe640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff815b620 con 0x7f1ff8140770 2024-04-14T08:41:12.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ffdffe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff80cf9b0 con 0x7f1ff813df30 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ffdffe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff81414a0 con 0x7f1ff815c380 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ff77fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ff813df30 0x7f1ff80b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ff77fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ff813df30 0x7f1ff80b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44880/0 (socket says 172.21.15.172:44880) 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ff77fe640 1 -- 172.21.15.172:0/2276923661 learned_addr learned my addr 172.21.15.172:0/2276923661 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ff6ffd640 1 --2- 172.21.15.172:0/2276923661 >> v2:172.21.15.177:3300/0 conn(0x7f1ff815c380 0x7f1ff80b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ff6ffd640 1 -- 172.21.15.172:0/2276923661 >> v2:172.21.15.172:3300/0 conn(0x7f1ff813df30 msgr2=0x7f1ff80b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ff6ffd640 1 --2- 172.21.15.172:0/2276923661 >> v2:172.21.15.172:3300/0 conn(0x7f1ff813df30 0x7f1ff80b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ff6ffd640 1 -- 172.21.15.172:0/2276923661 >> v1:172.21.15.177:6789/0 conn(0x7f1ff8140770 legacy=0x7f1ff8140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ff6ffd640 1 -- 172.21.15.172:0/2276923661 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff815b8a0 con 0x7f1ff815c380 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ff77fe640 1 --2- 172.21.15.172:0/2276923661 >> v2:172.21.15.172:3300/0 conn(0x7f1ff813df30 0x7f1ff80b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.129+0000 7f1ff67fc640 1 -- 172.21.15.172:0/2276923661 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 949454779 0 0) 0x7f1ff0001490 con 0x7f1ff8140770 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.130+0000 7f1ff6ffd640 1 --2- 172.21.15.172:0/2276923661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff815c380 0x7f1ff80b2210 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1ff806c4d0 tx=0x7f1fe802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caff09702f906b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.130+0000 7f1ff67fc640 1 -- 172.21.15.172:0/2276923661 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8035020 con 0x7f1ff815c380 2024-04-14T08:41:12.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.130+0000 7f1ff67fc640 1 -- 172.21.15.172:0/2276923661 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe8031830 con 0x7f1ff815c380 2024-04-14T08:41:12.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.130+0000 7f1ff67fc640 1 -- 172.21.15.172:0/2276923661 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8031c00 con 0x7f1ff815c380 2024-04-14T08:41:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.131+0000 7f1ffdffe640 1 -- 172.21.15.172:0/2276923661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff815c380 msgr2=0x7f1ff80b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.131+0000 7f1ffdffe640 1 --2- 172.21.15.172:0/2276923661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff815c380 0x7f1ff80b2210 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1ff806c4d0 tx=0x7f1fe802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.131+0000 7f1ffdffe640 1 -- 172.21.15.172:0/2276923661 shutdown_connections 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.131+0000 7f1ffdffe640 1 --2- 172.21.15.172:0/2276923661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff815c380 0x7f1ff80b2210 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.131+0000 7f1ffdffe640 1 --2- 172.21.15.172:0/2276923661 >> v2:172.21.15.172:3300/0 conn(0x7f1ff813df30 0x7f1ff80b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.131+0000 7f1ffdffe640 1 -- 172.21.15.172:0/2276923661 >> 172.21.15.172:0/2276923661 conn(0x7f1ff806b520 msgr2=0x7f1ff806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.131+0000 7f1ffdffe640 1 -- 172.21.15.172:0/2276923661 shutdown_connections 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.131+0000 7f1ffdffe640 1 -- 172.21.15.172:0/2276923661 wait complete. 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.132+0000 7f1ffdffe640 1 Processor -- start 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.132+0000 7f1ffdffe640 1 -- start start 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ffdffe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff813df30 0x7f1ff81580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ffdffe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ff8140770 0x7f1ff815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ffdffe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff815c380 0x7f1ff80b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ffdffe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff813e310 con 0x7f1ff8140770 2024-04-14T08:41:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ffdffe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff815be60 con 0x7f1ff813df30 2024-04-14T08:41:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ffdffe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff80ce9a0 con 0x7f1ff815c380 2024-04-14T08:41:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ff6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ff8140770 0x7f1ff815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ff6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ff8140770 0x7f1ff815a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44886/0 (socket says 172.21.15.172:44886) 2024-04-14T08:41:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ff6ffd640 1 -- 172.21.15.172:0/1991461209 learned_addr learned my addr 172.21.15.172:0/1991461209 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ff7fff640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff815c380 0x7f1ff80b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ff77fe640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff813df30 0x7f1ff81580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ff6ffd640 1 -- 172.21.15.172:0/1991461209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff815c380 msgr2=0x7f1ff80b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ff6ffd640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff815c380 0x7f1ff80b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ff6ffd640 1 -- 172.21.15.172:0/1991461209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff813df30 msgr2=0x7f1ff81580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ff6ffd640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff813df30 0x7f1ff81580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ff6ffd640 1 -- 172.21.15.172:0/1991461209 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff80b0c20 con 0x7f1ff8140770 2024-04-14T08:41:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ff6ffd640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ff8140770 0x7f1ff815a960 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8034040 tx=0x7f1fe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1fdffff640 1 -- 172.21.15.172:0/1991461209 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8004230 con 0x7f1ff8140770 2024-04-14T08:41:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1fdffff640 1 -- 172.21.15.172:0/1991461209 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe80043d0 con 0x7f1ff8140770 2024-04-14T08:41:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1fdffff640 1 -- 172.21.15.172:0/1991461209 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe803a870 con 0x7f1ff8140770 2024-04-14T08:41:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ffdffe640 1 -- 172.21.15.172:0/1991461209 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff80b0f00 con 0x7f1ff8140770 2024-04-14T08:41:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.133+0000 7f1ffdffe640 1 -- 172.21.15.172:0/1991461209 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff80b0920 con 0x7f1ff8140770 2024-04-14T08:41:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.134+0000 7f1ffdffe640 1 -- 172.21.15.172:0/1991461209 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff8068590 con 0x7f1ff8140770 2024-04-14T08:41:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.135+0000 7f1fdffff640 1 -- 172.21.15.172:0/1991461209 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fe8038020 con 0x7f1ff8140770 2024-04-14T08:41:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.135+0000 7f1fdffff640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1fe406c8f0 0x7f1fe406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.135+0000 7f1fdffff640 1 -- 172.21.15.172:0/1991461209 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f1fe80ad840 con 0x7f1ff8140770 2024-04-14T08:41:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.135+0000 7f1ff77fe640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1fe406c8f0 0x7f1fe406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.136+0000 7f1ff77fe640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1fe406c8f0 0x7f1fe406edb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009630 tx=0x7f1fec009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.140+0000 7f1fdffff640 1 -- 172.21.15.172:0/1991461209 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1fe8040070 con 0x7f1ff8140770 2024-04-14T08:41:12.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.316+0000 7f1ffdffe640 1 -- 172.21.15.172:0/1991461209 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1ff81544d0 con 0x7f1ff8140770 2024-04-14T08:41:12.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.316+0000 7f1fdffff640 1 -- 172.21.15.172:0/1991461209 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f1fe807b4a0 con 0x7f1ff8140770 2024-04-14T08:41:12.318 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:41:12.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.318+0000 7f1ffdffe640 1 -- 172.21.15.172:0/1991461209 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1fe406c8f0 msgr2=0x7f1fe406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.318+0000 7f1ffdffe640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1fe406c8f0 0x7f1fe406edb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009630 tx=0x7f1fec009240 comp rx=0 tx=0).stop 2024-04-14T08:41:12.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.318+0000 7f1ffdffe640 1 -- 172.21.15.172:0/1991461209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ff8140770 msgr2=0x7f1ff815a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.318+0000 7f1ffdffe640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ff8140770 0x7f1ff815a960 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8034040 tx=0x7f1fe80046c0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.318+0000 7f1ffdffe640 1 -- 172.21.15.172:0/1991461209 shutdown_connections 2024-04-14T08:41:12.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.318+0000 7f1ffdffe640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1fe406c8f0 0x7f1fe406edb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.318+0000 7f1ffdffe640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff815c380 0x7f1ff80b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.318+0000 7f1ffdffe640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ff8140770 0x7f1ff815a960 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.318+0000 7f1ffdffe640 1 --2- 172.21.15.172:0/1991461209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff813df30 0x7f1ff81580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.318+0000 7f1ffdffe640 1 -- 172.21.15.172:0/1991461209 >> 172.21.15.172:0/1991461209 conn(0x7f1ff806b520 msgr2=0x7f1ff815e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:12.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.318+0000 7f1ffdffe640 1 -- 172.21.15.172:0/1991461209 shutdown_connections 2024-04-14T08:41:12.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.318+0000 7f1ffdffe640 1 -- 172.21.15.172:0/1991461209 wait complete. 2024-04-14T08:41:12.331 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:41:12.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.433+0000 7ff409636640 1 Processor -- start 2024-04-14T08:41:12.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.433+0000 7ff409636640 1 -- start start 2024-04-14T08:41:12.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.433+0000 7ff409636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff40415e430 0x7ff404162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.433+0000 7ff409636640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff404162540 0x7ff4041629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff409636640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4040cefb0 con 0x7ff4040b3a30 2024-04-14T08:41:12.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff409636640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff4040cf290 con 0x7ff404162540 2024-04-14T08:41:12.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff409636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff4040cf570 con 0x7ff40415e430 2024-04-14T08:41:12.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff4027fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff404162540 0x7ff4041629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff402ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff40415e430 0x7ff404162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff4037fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff4040b3a30 0x7ff4040b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:60416/0 (socket says 172.21.15.172:60416) 2024-04-14T08:41:12.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff4037fe640 1 -- 172.21.15.172:0/2142948244 learned_addr learned my addr 172.21.15.172:0/2142948244 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff4027fc640 1 -- 172.21.15.172:0/2142948244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff40415e430 msgr2=0x7ff404162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff4027fc640 1 --2- 172.21.15.172:0/2142948244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff40415e430 0x7ff404162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff4027fc640 1 -- 172.21.15.172:0/2142948244 >> v1:172.21.15.172:6789/0 conn(0x7ff4040b3a30 legacy=0x7ff4040b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff4027fc640 1 -- 172.21.15.172:0/2142948244 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4040cf850 con 0x7ff404162540 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff401ffb640 1 -- 172.21.15.172:0/2142948244 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1616470511 0 0) 0x7ff4040cefb0 con 0x7ff4040b3a30 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.434+0000 7ff402ffd640 1 --2- 172.21.15.172:0/2142948244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff40415e430 0x7ff404162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.435+0000 7ff4027fc640 1 --2- 172.21.15.172:0/2142948244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff404162540 0x7ff4041629a0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80097a0 tx=0x7ff3f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e8419726b3e0ba server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.435+0000 7ff401ffb640 1 -- 172.21.15.172:0/2142948244 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8035020 con 0x7ff404162540 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.435+0000 7ff401ffb640 1 -- 172.21.15.172:0/2142948244 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f8031830 con 0x7ff404162540 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.435+0000 7ff401ffb640 1 -- 172.21.15.172:0/2142948244 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8031c00 con 0x7ff404162540 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.435+0000 7ff409636640 1 -- 172.21.15.172:0/2142948244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff404162540 msgr2=0x7ff4041629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.435+0000 7ff409636640 1 --2- 172.21.15.172:0/2142948244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff404162540 0x7ff4041629a0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80097a0 tx=0x7ff3f802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.435+0000 7ff409636640 1 -- 172.21.15.172:0/2142948244 shutdown_connections 2024-04-14T08:41:12.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.435+0000 7ff409636640 1 --2- 172.21.15.172:0/2142948244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff404162540 0x7ff4041629a0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.435+0000 7ff409636640 1 --2- 172.21.15.172:0/2142948244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff40415e430 0x7ff404162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.435+0000 7ff409636640 1 -- 172.21.15.172:0/2142948244 >> 172.21.15.172:0/2142948244 conn(0x7ff40409d560 msgr2=0x7ff4040d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:12.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.435+0000 7ff409636640 1 -- 172.21.15.172:0/2142948244 shutdown_connections 2024-04-14T08:41:12.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.435+0000 7ff409636640 1 -- 172.21.15.172:0/2142948244 wait complete. 2024-04-14T08:41:12.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.437+0000 7ff409636640 1 Processor -- start 2024-04-14T08:41:12.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.437+0000 7ff409636640 1 -- start start 2024-04-14T08:41:12.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.437+0000 7ff409636640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040b3a30 0x7ff4041f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.437+0000 7ff409636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff40415e430 0x7ff4041f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.437+0000 7ff409636640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff404162540 0x7ff4040b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.437+0000 7ff409636640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff4040d4e20 con 0x7ff4040b3a30 2024-04-14T08:41:12.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.437+0000 7ff409636640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff40415e810 con 0x7ff404162540 2024-04-14T08:41:12.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.437+0000 7ff409636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff4040d0050 con 0x7ff40415e430 2024-04-14T08:41:12.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff402ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040b3a30 0x7ff4041f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff402ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040b3a30 0x7ff4041f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44900/0 (socket says 172.21.15.172:44900) 2024-04-14T08:41:12.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff402ffd640 1 -- 172.21.15.172:0/2250267259 learned_addr learned my addr 172.21.15.172:0/2250267259 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:12.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff4027fc640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff40415e430 0x7ff4041f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff4037fe640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff404162540 0x7ff4040b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff402ffd640 1 -- 172.21.15.172:0/2250267259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff40415e430 msgr2=0x7ff4041f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff402ffd640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff40415e430 0x7ff4041f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff402ffd640 1 -- 172.21.15.172:0/2250267259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff404162540 msgr2=0x7ff4040b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff402ffd640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff404162540 0x7ff4040b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff402ffd640 1 -- 172.21.15.172:0/2250267259 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4040b9c40 con 0x7ff4040b3a30 2024-04-14T08:41:12.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff402ffd640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040b3a30 0x7ff4041f5300 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400ebe0 tx=0x7ff3f400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:12.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff3eb7fe640 1 -- 172.21.15.172:0/2250267259 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4017070 con 0x7ff4040b3a30 2024-04-14T08:41:12.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff3eb7fe640 1 -- 172.21.15.172:0/2250267259 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f400cc30 con 0x7ff4040b3a30 2024-04-14T08:41:12.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff409636640 1 -- 172.21.15.172:0/2250267259 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4040b50c0 con 0x7ff4040b3a30 2024-04-14T08:41:12.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff3eb7fe640 1 -- 172.21.15.172:0/2250267259 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4013470 con 0x7ff4040b3a30 2024-04-14T08:41:12.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.438+0000 7ff409636640 1 -- 172.21.15.172:0/2250267259 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4040b5680 con 0x7ff4040b3a30 2024-04-14T08:41:12.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.439+0000 7ff409636640 1 -- 172.21.15.172:0/2250267259 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4040b4eb0 con 0x7ff4040b3a30 2024-04-14T08:41:12.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.440+0000 7ff3eb7fe640 1 -- 172.21.15.172:0/2250267259 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3f401a070 con 0x7ff4040b3a30 2024-04-14T08:41:12.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.440+0000 7ff3eb7fe640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff3f006c920 0x7ff3f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.440+0000 7ff3eb7fe640 1 -- 172.21.15.172:0/2250267259 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7ff3f408bee0 con 0x7ff4040b3a30 2024-04-14T08:41:12.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.440+0000 7ff4027fc640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff3f006c920 0x7ff3f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.441+0000 7ff4027fc640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff3f006c920 0x7ff3f006ede0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8036000 tx=0x7ff3f803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:12.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.444+0000 7ff3eb7fe640 1 -- 172.21.15.172:0/2250267259 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff3f4059b30 con 0x7ff4040b3a30 2024-04-14T08:41:12.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.629+0000 7ff409636640 1 -- 172.21.15.172:0/2250267259 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff404068590 con 0x7ff4040b3a30 2024-04-14T08:41:12.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.629+0000 7ff3eb7fe640 1 -- 172.21.15.172:0/2250267259 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) ==== 80+0+0 (secure 0 0 0) 0x7ff3f405de30 con 0x7ff4040b3a30 2024-04-14T08:41:12.631 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:41:12.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.631+0000 7ff409636640 1 -- 172.21.15.172:0/2250267259 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff3f006c920 msgr2=0x7ff3f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.631+0000 7ff409636640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff3f006c920 0x7ff3f006ede0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8036000 tx=0x7ff3f803c040 comp rx=0 tx=0).stop 2024-04-14T08:41:12.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.631+0000 7ff409636640 1 -- 172.21.15.172:0/2250267259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040b3a30 msgr2=0x7ff4041f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.631+0000 7ff409636640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040b3a30 0x7ff4041f5300 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400ebe0 tx=0x7ff3f400c520 comp rx=0 tx=0).stop 2024-04-14T08:41:12.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.631+0000 7ff409636640 1 -- 172.21.15.172:0/2250267259 shutdown_connections 2024-04-14T08:41:12.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.631+0000 7ff409636640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff3f006c920 0x7ff3f006ede0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.631+0000 7ff409636640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff404162540 0x7ff4040b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.631+0000 7ff409636640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff40415e430 0x7ff4041f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.631+0000 7ff409636640 1 --2- 172.21.15.172:0/2250267259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040b3a30 0x7ff4041f5300 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.631+0000 7ff409636640 1 -- 172.21.15.172:0/2250267259 >> 172.21.15.172:0/2250267259 conn(0x7ff40409d560 msgr2=0x7ff4040d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:12.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.631+0000 7ff409636640 1 -- 172.21.15.172:0/2250267259 shutdown_connections 2024-04-14T08:41:12.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.631+0000 7ff409636640 1 -- 172.21.15.172:0/2250267259 wait complete. 2024-04-14T08:41:12.643 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:41:12.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:12.644 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:41:12.678 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:41:12.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:12.678 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:41:12.734 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:41:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.874+0000 7f364b3aa640 1 Processor -- start 2024-04-14T08:41:12.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.874+0000 7f364b3aa640 1 -- start start 2024-04-14T08:41:12.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f364b3aa640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f36440b1df0 0x7f36440b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f364b3aa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f36440b21d0 0x7f364415c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f364b3aa640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36440b3430 con 0x7f364415c5a0 2024-04-14T08:41:12.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f364b3aa640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f36440b3700 con 0x7f36440b21d0 2024-04-14T08:41:12.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f364b3aa640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f36440b39e0 con 0x7f36440b1df0 2024-04-14T08:41:12.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f3648945640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f36440b21d0 0x7f364415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f3649947640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f364415c5a0 0x7f36440b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:60422/0 (socket says 172.21.15.172:60422) 2024-04-14T08:41:12.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f3649947640 1 -- 172.21.15.172:0/3627571167 learned_addr learned my addr 172.21.15.172:0/3627571167 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:12.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f3649146640 1 --2- 172.21.15.172:0/3627571167 >> v2:172.21.15.177:3300/0 conn(0x7f36440b1df0 0x7f36440b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f3648945640 1 -- 172.21.15.172:0/3627571167 >> v2:172.21.15.177:3300/0 conn(0x7f36440b1df0 msgr2=0x7f36440b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:12.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f3648945640 1 --2- 172.21.15.172:0/3627571167 >> v2:172.21.15.177:3300/0 conn(0x7f36440b1df0 0x7f36440b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f3648945640 1 -- 172.21.15.172:0/3627571167 >> v1:172.21.15.172:6789/0 conn(0x7f364415c5a0 legacy=0x7f36440b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f363bfff640 1 -- 172.21.15.172:0/3627571167 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3121804094 0 0) 0x7f36440b3430 con 0x7f364415c5a0 2024-04-14T08:41:12.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f3648945640 1 -- 172.21.15.172:0/3627571167 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36440b3cc0 con 0x7f36440b21d0 2024-04-14T08:41:12.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.875+0000 7f3649146640 1 --2- 172.21.15.172:0/3627571167 >> v2:172.21.15.177:3300/0 conn(0x7f36440b1df0 0x7f36440b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:12.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.876+0000 7f3648945640 1 --2- 172.21.15.172:0/3627571167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36440b21d0 0x7f364415c060 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f364406f4b0 tx=0x7f363402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e090229e2b649427 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:12.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.876+0000 7f363bfff640 1 -- 172.21.15.172:0/3627571167 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36340315a0 con 0x7f36440b21d0 2024-04-14T08:41:12.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.876+0000 7f363bfff640 1 -- 172.21.15.172:0/3627571167 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3634037070 con 0x7f36440b21d0 2024-04-14T08:41:12.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.876+0000 7f363bfff640 1 -- 172.21.15.172:0/3627571167 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634031960 con 0x7f36440b21d0 2024-04-14T08:41:12.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.877+0000 7f364b3aa640 1 -- 172.21.15.172:0/3627571167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36440b21d0 msgr2=0x7f364415c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.877+0000 7f364b3aa640 1 --2- 172.21.15.172:0/3627571167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36440b21d0 0x7f364415c060 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f364406f4b0 tx=0x7f363402b840 comp rx=0 tx=0).stop 2024-04-14T08:41:12.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.877+0000 7f364b3aa640 1 -- 172.21.15.172:0/3627571167 shutdown_connections 2024-04-14T08:41:12.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.877+0000 7f364b3aa640 1 --2- 172.21.15.172:0/3627571167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36440b21d0 0x7f364415c060 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.877+0000 7f364b3aa640 1 --2- 172.21.15.172:0/3627571167 >> v2:172.21.15.177:3300/0 conn(0x7f36440b1df0 0x7f36440b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.877+0000 7f364b3aa640 1 -- 172.21.15.172:0/3627571167 >> 172.21.15.172:0/3627571167 conn(0x7f364406ec30 msgr2=0x7f364406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.877+0000 7f364b3aa640 1 -- 172.21.15.172:0/3627571167 shutdown_connections 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.877+0000 7f364b3aa640 1 -- 172.21.15.172:0/3627571167 wait complete. 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.878+0000 7f364b3aa640 1 Processor -- start 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.878+0000 7f364b3aa640 1 -- start start 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f364b3aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36440b1df0 0x7f364413ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f364b3aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36440b21d0 0x7f364415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f364b3aa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36440cebf0 0x7f36440cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f364b3aa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36440b4750 con 0x7f36440b1df0 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f364b3aa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f364415b7f0 con 0x7f36440cebf0 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f364b3aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f36440b48d0 con 0x7f36440b21d0 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3649146640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36440b1df0 0x7f364413ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3649146640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36440b1df0 0x7f364413ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44926/0 (socket says 172.21.15.172:44926) 2024-04-14T08:41:12.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3649146640 1 -- 172.21.15.172:0/2870675685 learned_addr learned my addr 172.21.15.172:0/2870675685 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:12.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3649947640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36440cebf0 0x7f36440cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3648945640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36440b21d0 0x7f364415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3649146640 1 -- 172.21.15.172:0/2870675685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36440b21d0 msgr2=0x7f364415a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3649146640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36440b21d0 0x7f364415a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3649146640 1 -- 172.21.15.172:0/2870675685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36440cebf0 msgr2=0x7f36440cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:12.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3649146640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36440cebf0 0x7f36440cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:12.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3649146640 1 -- 172.21.15.172:0/2870675685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36440cf580 con 0x7f36440b1df0 2024-04-14T08:41:12.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3648945640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36440b21d0 0x7f364415a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:41:12.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3649146640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36440b1df0 0x7f364413ac00 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f364000e990 tx=0x7f364000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:12.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3639ffb640 1 -- 172.21.15.172:0/2870675685 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640017070 con 0x7f36440b1df0 2024-04-14T08:41:12.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3639ffb640 1 -- 172.21.15.172:0/2870675685 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f364000c9e0 con 0x7f36440b1df0 2024-04-14T08:41:12.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f364b3aa640 1 -- 172.21.15.172:0/2870675685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364413b450 con 0x7f36440b1df0 2024-04-14T08:41:12.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.879+0000 7f3639ffb640 1 -- 172.21.15.172:0/2870675685 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364000cdb0 con 0x7f36440b1df0 2024-04-14T08:41:12.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.880+0000 7f364b3aa640 1 -- 172.21.15.172:0/2870675685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36440cf820 con 0x7f36440b1df0 2024-04-14T08:41:12.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.881+0000 7f364b3aa640 1 -- 172.21.15.172:0/2870675685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36440b2890 con 0x7f36440b1df0 2024-04-14T08:41:12.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.881+0000 7f3639ffb640 1 -- 172.21.15.172:0/2870675685 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f364001a070 con 0x7f36440b1df0 2024-04-14T08:41:12.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.881+0000 7f3639ffb640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f362806c920 0x7f362806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:12.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.881+0000 7f3639ffb640 1 -- 172.21.15.172:0/2870675685 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f364008ba10 con 0x7f36440b1df0 2024-04-14T08:41:12.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.882+0000 7f3648945640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f362806c920 0x7f362806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:12.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.882+0000 7f3648945640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f362806c920 0x7f362806ede0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3634036000 tx=0x7f363403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:12.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:12.886+0000 7f3639ffb640 1 -- 172.21.15.172:0/2870675685 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f36440b2890 con 0x7f36440b1df0 2024-04-14T08:41:13.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.097+0000 7f364b3aa640 1 -- 172.21.15.172:0/2870675685 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f364413b1a0 con 0x7f36440b1df0 2024-04-14T08:41:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.098+0000 7f3639ffb640 1 -- 172.21.15.172:0/2870675685 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5195 (secure 0 0 0) 0x7f36400596f0 con 0x7f36440b1df0 2024-04-14T08:41:13.099 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:13.099 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/4028911489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4028911489},{"type":"v1","addr":"172.21.15.172:6837","nonce":4028911489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:41:04.492773+0000","modified":"2024-04-14T08:41:06.492980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/2602337717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2602337717},{"type":"v1","addr":"172.21.15.177:6835","nonce":2602337717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8394,"qdb_cluster":[8394]},"id":7}]} 2024-04-14T08:41:13.099 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 74 2024-04-14T08:41:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.100+0000 7f364b3aa640 1 -- 172.21.15.172:0/2870675685 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f362806c920 msgr2=0x7f362806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.100+0000 7f364b3aa640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f362806c920 0x7f362806ede0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3634036000 tx=0x7f363403d040 comp rx=0 tx=0).stop 2024-04-14T08:41:13.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.100+0000 7f364b3aa640 1 -- 172.21.15.172:0/2870675685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36440b1df0 msgr2=0x7f364413ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.100+0000 7f364b3aa640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36440b1df0 0x7f364413ac00 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f364000e990 tx=0x7f364000ee50 comp rx=0 tx=0).stop 2024-04-14T08:41:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.100+0000 7f364b3aa640 1 -- 172.21.15.172:0/2870675685 shutdown_connections 2024-04-14T08:41:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.100+0000 7f364b3aa640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f362806c920 0x7f362806ede0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.100+0000 7f364b3aa640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36440cebf0 0x7f36440cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.100+0000 7f364b3aa640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36440b21d0 0x7f364415a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.100+0000 7f364b3aa640 1 --2- 172.21.15.172:0/2870675685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36440b1df0 0x7f364413ac00 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.100+0000 7f364b3aa640 1 -- 172.21.15.172:0/2870675685 >> 172.21.15.172:0/2870675685 conn(0x7f364406ec30 msgr2=0x7f36441588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.100+0000 7f364b3aa640 1 -- 172.21.15.172:0/2870675685 shutdown_connections 2024-04-14T08:41:13.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.100+0000 7f364b3aa640 1 -- 172.21.15.172:0/2870675685 wait complete. 2024-04-14T08:41:13.112 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-14T08:41:13.247 INFO:teuthology.orchestra.run.smithi177.stdout:{"message":"","return_code":0} 2024-04-14T08:41:13.247 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-14T08:41:13.247 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:41:13.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.347+0000 7feb84ca2640 1 Processor -- start 2024-04-14T08:41:13.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.347+0000 7feb84ca2640 1 -- start start 2024-04-14T08:41:13.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb84ca2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb800b1df0 0x7feb800b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb84ca2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb800b21d0 0x7feb8015c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb84ca2640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7feb8015c5a0 0x7feb800b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb84ca2640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7feb800b3430 con 0x7feb800b21d0 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb84ca2640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7feb800b3700 con 0x7feb8015c5a0 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb84ca2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7feb800b39e0 con 0x7feb800b1df0 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb77fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb800b21d0 0x7feb8015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb77fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb800b21d0 0x7feb8015c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44930/0 (socket says 172.21.15.172:44930) 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb77fff640 1 -- 172.21.15.172:0/2090483428 learned_addr learned my addr 172.21.15.172:0/2090483428 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb7ed9d640 1 --2- 172.21.15.172:0/2090483428 >> v2:172.21.15.177:3300/0 conn(0x7feb8015c5a0 0x7feb800b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb7e59c640 1 --2- 172.21.15.172:0/2090483428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb800b1df0 0x7feb800b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb77fff640 1 -- 172.21.15.172:0/2090483428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb800b1df0 msgr2=0x7feb800b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb77fff640 1 --2- 172.21.15.172:0/2090483428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb800b1df0 0x7feb800b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb77fff640 1 -- 172.21.15.172:0/2090483428 >> v2:172.21.15.177:3300/0 conn(0x7feb8015c5a0 msgr2=0x7feb800b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:13.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb77fff640 1 --2- 172.21.15.172:0/2090483428 >> v2:172.21.15.177:3300/0 conn(0x7feb8015c5a0 0x7feb800b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.348+0000 7feb77fff640 1 -- 172.21.15.172:0/2090483428 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb800b3cc0 con 0x7feb800b21d0 2024-04-14T08:41:13.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.349+0000 7feb77fff640 1 --2- 172.21.15.172:0/2090483428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb800b21d0 0x7feb8015c060 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7feb8006c0d0 tx=0x7feb6802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65dc409d26ee335 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:13.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.349+0000 7feb7dd9b640 1 -- 172.21.15.172:0/2090483428 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68035020 con 0x7feb800b21d0 2024-04-14T08:41:13.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.349+0000 7feb7dd9b640 1 -- 172.21.15.172:0/2090483428 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb68031870 con 0x7feb800b21d0 2024-04-14T08:41:13.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.349+0000 7feb7dd9b640 1 -- 172.21.15.172:0/2090483428 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68031c40 con 0x7feb800b21d0 2024-04-14T08:41:13.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.349+0000 7feb84ca2640 1 -- 172.21.15.172:0/2090483428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb800b21d0 msgr2=0x7feb8015c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.349+0000 7feb84ca2640 1 --2- 172.21.15.172:0/2090483428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb800b21d0 0x7feb8015c060 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7feb8006c0d0 tx=0x7feb6802b910 comp rx=0 tx=0).stop 2024-04-14T08:41:13.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.350+0000 7feb84ca2640 1 -- 172.21.15.172:0/2090483428 shutdown_connections 2024-04-14T08:41:13.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.350+0000 7feb84ca2640 1 --2- 172.21.15.172:0/2090483428 >> v2:172.21.15.177:3300/0 conn(0x7feb8015c5a0 0x7feb800b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.350+0000 7feb84ca2640 1 --2- 172.21.15.172:0/2090483428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb800b21d0 0x7feb8015c060 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.350+0000 7feb84ca2640 1 --2- 172.21.15.172:0/2090483428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb800b1df0 0x7feb800b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.350+0000 7feb84ca2640 1 -- 172.21.15.172:0/2090483428 >> 172.21.15.172:0/2090483428 conn(0x7feb8006ec30 msgr2=0x7feb8006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:13.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.350+0000 7feb84ca2640 1 -- 172.21.15.172:0/2090483428 shutdown_connections 2024-04-14T08:41:13.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.350+0000 7feb84ca2640 1 -- 172.21.15.172:0/2090483428 wait complete. 2024-04-14T08:41:13.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.351+0000 7feb84ca2640 1 Processor -- start 2024-04-14T08:41:13.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.351+0000 7feb84ca2640 1 -- start start 2024-04-14T08:41:13.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb84ca2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb800b1df0 0x7feb800ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb84ca2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb800b21d0 0x7feb80143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb84ca2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80136bb0 0x7feb8015dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb84ca2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb800b4580 con 0x7feb80136bb0 2024-04-14T08:41:13.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb84ca2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feb8015b7f0 con 0x7feb800b21d0 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb84ca2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7feb800b4700 con 0x7feb800b1df0 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb7ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80136bb0 0x7feb8015dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb7ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80136bb0 0x7feb8015dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44940/0 (socket says 172.21.15.172:44940) 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb7ed9d640 1 -- 172.21.15.172:0/2562333795 learned_addr learned my addr 172.21.15.172:0/2562333795 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb77fff640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb800b21d0 0x7feb80143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb7e59c640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb800b1df0 0x7feb800ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb7ed9d640 1 -- 172.21.15.172:0/2562333795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb800b1df0 msgr2=0x7feb800ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb7ed9d640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb800b1df0 0x7feb800ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb7ed9d640 1 -- 172.21.15.172:0/2562333795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb800b21d0 msgr2=0x7feb80143d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb7ed9d640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb800b21d0 0x7feb80143d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb7ed9d640 1 -- 172.21.15.172:0/2562333795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb800ce790 con 0x7feb80136bb0 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb7ed9d640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80136bb0 0x7feb8015dd80 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7feb7800efc0 tx=0x7feb7800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb76ffd640 1 -- 172.21.15.172:0/2562333795 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78018020 con 0x7feb80136bb0 2024-04-14T08:41:13.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb76ffd640 1 -- 172.21.15.172:0/2562333795 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb78014480 con 0x7feb80136bb0 2024-04-14T08:41:13.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb84ca2640 1 -- 172.21.15.172:0/2562333795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb800cea70 con 0x7feb80136bb0 2024-04-14T08:41:13.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb76ffd640 1 -- 172.21.15.172:0/2562333795 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78009e60 con 0x7feb80136bb0 2024-04-14T08:41:13.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.352+0000 7feb84ca2640 1 -- 172.21.15.172:0/2562333795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb800cf030 con 0x7feb80136bb0 2024-04-14T08:41:13.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.353+0000 7feb84ca2640 1 -- 172.21.15.172:0/2562333795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb80068520 con 0x7feb80136bb0 2024-04-14T08:41:13.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.354+0000 7feb76ffd640 1 -- 172.21.15.172:0/2562333795 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb78005d10 con 0x7feb80136bb0 2024-04-14T08:41:13.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.354+0000 7feb76ffd640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb6006c8f0 0x7feb6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.354+0000 7feb76ffd640 1 -- 172.21.15.172:0/2562333795 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7feb7808c8a0 con 0x7feb80136bb0 2024-04-14T08:41:13.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.354+0000 7feb7e59c640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb6006c8f0 0x7feb6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.355+0000 7feb7e59c640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb6006c8f0 0x7feb6006edb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7feb70009ac0 tx=0x7feb70009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:13.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.358+0000 7feb76ffd640 1 -- 172.21.15.172:0/2562333795 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7feb78022030 con 0x7feb80136bb0 2024-04-14T08:41:13.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.569+0000 7feb84ca2640 1 -- 172.21.15.172:0/2562333795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb80134cc0 con 0x7feb80136bb0 2024-04-14T08:41:13.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.570+0000 7feb76ffd640 1 -- 172.21.15.172:0/2562333795 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5195 (secure 0 0 0) 0x7feb78010040 con 0x7feb80136bb0 2024-04-14T08:41:13.571 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:13.571 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/4028911489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4028911489},{"type":"v1","addr":"172.21.15.172:6837","nonce":4028911489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:41:04.492773+0000","modified":"2024-04-14T08:41:06.492980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/2602337717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2602337717},{"type":"v1","addr":"172.21.15.177:6835","nonce":2602337717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8394,"qdb_cluster":[8394]},"id":7}]} 2024-04-14T08:41:13.572 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 74 2024-04-14T08:41:13.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.572+0000 7feb84ca2640 1 -- 172.21.15.172:0/2562333795 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb6006c8f0 msgr2=0x7feb6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.572+0000 7feb84ca2640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb6006c8f0 0x7feb6006edb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7feb70009ac0 tx=0x7feb70009190 comp rx=0 tx=0).stop 2024-04-14T08:41:13.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.572+0000 7feb84ca2640 1 -- 172.21.15.172:0/2562333795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80136bb0 msgr2=0x7feb8015dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.572+0000 7feb84ca2640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80136bb0 0x7feb8015dd80 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7feb7800efc0 tx=0x7feb7800c490 comp rx=0 tx=0).stop 2024-04-14T08:41:13.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.572+0000 7feb84ca2640 1 -- 172.21.15.172:0/2562333795 shutdown_connections 2024-04-14T08:41:13.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.572+0000 7feb84ca2640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb6006c8f0 0x7feb6006edb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.572+0000 7feb84ca2640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb80136bb0 0x7feb8015dd80 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.572+0000 7feb84ca2640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb800b21d0 0x7feb80143d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.572+0000 7feb84ca2640 1 --2- 172.21.15.172:0/2562333795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb800b1df0 0x7feb800ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.572+0000 7feb84ca2640 1 -- 172.21.15.172:0/2562333795 >> 172.21.15.172:0/2562333795 conn(0x7feb8006ec30 msgr2=0x7feb80071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:13.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.572+0000 7feb84ca2640 1 -- 172.21.15.172:0/2562333795 shutdown_connections 2024-04-14T08:41:13.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.572+0000 7feb84ca2640 1 -- 172.21.15.172:0/2562333795 wait complete. 2024-04-14T08:41:13.585 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-14T08:41:13.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.625+0000 7f698be35f40 1 Processor -- start 2024-04-14T08:41:13.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.625+0000 7f698be35f40 1 -- start start 2024-04-14T08:41:13.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f698be35f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f2eca8e500 0x55f2ecb698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f698be35f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2ecad9510 0x55f2ecb48bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f698be35f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2ecb67a80 0x55f2ecb654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f698be35f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f2eca363b0 con 0x55f2ecb67a80 2024-04-14T08:41:13.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f698be35f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f2eca3a400 con 0x55f2ecad9510 2024-04-14T08:41:13.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f698be35f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f2ecad8870 con 0x55f2eca8e500 2024-04-14T08:41:13.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f69895a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2ecb67a80 0x55f2ecb654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f6988da3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f2eca8e500 0x55f2ecb698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f69895a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2ecb67a80 0x55f2ecb654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44958/0 (socket says 172.21.15.172:44958) 2024-04-14T08:41:13.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f69895a4640 1 -- 172.21.15.172:0/2975262236 learned_addr learned my addr 172.21.15.172:0/2975262236 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:13.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f6983fff640 1 --2- 172.21.15.172:0/2975262236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2ecad9510 0x55f2ecb48bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f69895a4640 1 -- 172.21.15.172:0/2975262236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f2eca8e500 msgr2=0x55f2ecb698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f69895a4640 1 --2- 172.21.15.172:0/2975262236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f2eca8e500 0x55f2ecb698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f69895a4640 1 -- 172.21.15.172:0/2975262236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2ecad9510 msgr2=0x55f2ecb48bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f69895a4640 1 --2- 172.21.15.172:0/2975262236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2ecad9510 0x55f2ecb48bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f69895a4640 1 -- 172.21.15.172:0/2975262236 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ecb60a90 con 0x55f2ecb67a80 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.626+0000 7f69895a4640 1 --2- 172.21.15.172:0/2975262236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2ecb67a80 0x55f2ecb654a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f6984001650 tx=0x7f6984001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f190a5d958b2074b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.627+0000 7f6982ffd640 1 -- 172.21.15.172:0/2975262236 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984016070 con 0x55f2ecb67a80 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.627+0000 7f6982ffd640 1 -- 172.21.15.172:0/2975262236 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698400f710 con 0x55f2ecb67a80 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.627+0000 7f6982ffd640 1 -- 172.21.15.172:0/2975262236 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698400fb00 con 0x55f2ecb67a80 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.627+0000 7f698be35f40 1 -- 172.21.15.172:0/2975262236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2ecb67a80 msgr2=0x55f2ecb654a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.627+0000 7f698be35f40 1 --2- 172.21.15.172:0/2975262236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2ecb67a80 0x55f2ecb654a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f6984001650 tx=0x7f6984001b10 comp rx=0 tx=0).stop 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.627+0000 7f698be35f40 1 -- 172.21.15.172:0/2975262236 shutdown_connections 2024-04-14T08:41:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.627+0000 7f698be35f40 1 --2- 172.21.15.172:0/2975262236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2ecb67a80 0x55f2ecb654a0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.627+0000 7f698be35f40 1 --2- 172.21.15.172:0/2975262236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2ecad9510 0x55f2ecb48bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.627+0000 7f698be35f40 1 --2- 172.21.15.172:0/2975262236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f2eca8e500 0x55f2ecb698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.627+0000 7f698be35f40 1 -- 172.21.15.172:0/2975262236 >> 172.21.15.172:0/2975262236 conn(0x55f2ecabba50 msgr2=0x55f2ecabb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.627+0000 7f698be35f40 1 -- 172.21.15.172:0/2975262236 shutdown_connections 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.627+0000 7f698be35f40 1 -- 172.21.15.172:0/2975262236 wait complete. 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.628+0000 7f698be35f40 1 Processor -- start 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.628+0000 7f698be35f40 1 -- start start 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.628+0000 7f698be35f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2eca8e500 0x55f2ecb64130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.628+0000 7f698be35f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f2ecad9510 0x55f2ecb49120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.628+0000 7f698be35f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2ecb67a80 0x55f2ecb5ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.628+0000 7f698be35f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f2eca91420 con 0x55f2eca8e500 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.628+0000 7f698be35f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f2eca91140 con 0x55f2ecb67a80 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.628+0000 7f698be35f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55f2eca91700 con 0x55f2ecad9510 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.628+0000 7f6988da3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2eca8e500 0x55f2ecb64130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.628+0000 7f6988da3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2eca8e500 0x55f2ecb64130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44974/0 (socket says 172.21.15.172:44974) 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.628+0000 7f6988da3640 1 -- 172.21.15.172:0/388005245 learned_addr learned my addr 172.21.15.172:0/388005245 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f6983fff640 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f2ecad9510 0x55f2ecb49120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f69895a4640 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2ecb67a80 0x55f2ecb5ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f6988da3640 1 -- 172.21.15.172:0/388005245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f2ecad9510 msgr2=0x55f2ecb49120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f6988da3640 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f2ecad9510 0x55f2ecb49120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f6988da3640 1 -- 172.21.15.172:0/388005245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2ecb67a80 msgr2=0x55f2ecb5ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f6988da3640 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2ecb67a80 0x55f2ecb5ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f6988da3640 1 -- 172.21.15.172:0/388005245 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ecb4c7b0 con 0x55f2eca8e500 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f6988da3640 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2eca8e500 0x55f2ecb64130 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f697c00a4f0 tx=0x7f697c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f6981ffb640 1 -- 172.21.15.172:0/388005245 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c061030 con 0x55f2eca8e500 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f6981ffb640 1 -- 172.21.15.172:0/388005245 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697c052c10 con 0x55f2eca8e500 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f6981ffb640 1 -- 172.21.15.172:0/388005245 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c057040 con 0x55f2eca8e500 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f698be35f40 1 -- 172.21.15.172:0/388005245 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f2ecb63950 con 0x55f2eca8e500 2024-04-14T08:41:13.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.629+0000 7f698be35f40 1 -- 172.21.15.172:0/388005245 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2ecb4cd30 con 0x55f2eca8e500 2024-04-14T08:41:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.630+0000 7f6981ffb640 1 -- 172.21.15.172:0/388005245 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f697c064030 con 0x55f2eca8e500 2024-04-14T08:41:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.631+0000 7f6981ffb640 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f696c06b2d0 0x7f696c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.631+0000 7f6981ffb640 1 -- 172.21.15.172:0/388005245 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f697c0cc260 con 0x55f2eca8e500 2024-04-14T08:41:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.631+0000 7f6983fff640 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f696c06b2d0 0x7f696c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.631+0000 7f698be35f40 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55f2ecb68910 0x55f2ecb4a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.631+0000 7f698be35f40 1 -- 172.21.15.172:0/388005245 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x55f2ecb604c0 con 0x55f2ecb68910 2024-04-14T08:41:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.631+0000 7f69895a4640 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55f2ecb68910 0x55f2ecb4a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.631+0000 7f6983fff640 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f696c06b2d0 0x7f696c06d790 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55f2ecb49b50 tx=0x7f69740016f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f69895a4640 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55f2ecb68910 0x55f2ecb4a000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f69895a4640 1 -- 172.21.15.172:0/388005245 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f698400f070 con 0x55f2ecb68910 2024-04-14T08:41:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f698be35f40 1 -- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55f2ecb68910 msgr2=0x55f2ecb4a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f698be35f40 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55f2ecb68910 0x55f2ecb4a000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f698be35f40 1 -- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f696c06b2d0 msgr2=0x7f696c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f698be35f40 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f696c06b2d0 0x7f696c06d790 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55f2ecb49b50 tx=0x7f69740016f0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f698be35f40 1 -- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2eca8e500 msgr2=0x55f2ecb64130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f698be35f40 1 --2- 172.21.15.172:0/388005245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2eca8e500 0x55f2ecb64130 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f697c00a4f0 tx=0x7f697c002fb0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f69895a4640 1 -- 172.21.15.172:0/388005245 reap_dead start 2024-04-14T08:41:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f698be35f40 1 -- 172.21.15.172:0/388005245 shutdown_connections 2024-04-14T08:41:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f698be35f40 1 -- 172.21.15.172:0/388005245 >> 172.21.15.172:0/388005245 conn(0x55f2ecabba50 msgr2=0x55f2ecabb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f698be35f40 1 -- 172.21.15.172:0/388005245 shutdown_connections 2024-04-14T08:41:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.632+0000 7f698be35f40 1 -- 172.21.15.172:0/388005245 wait complete. 2024-04-14T08:41:13.639 INFO:teuthology.orchestra.run.smithi172.stdout:����� 2024-04-14T08:41:13.639 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:41:13.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.777+0000 7f6d54836640 1 Processor -- start 2024-04-14T08:41:13.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.777+0000 7f6d54836640 1 -- start start 2024-04-14T08:41:13.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.778+0000 7f6d54836640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6d500b4040 0x7f6d500b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.778+0000 7f6d54836640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5015c770 con 0x7f6d500b1df0 2024-04-14T08:41:13.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.778+0000 7f6d54836640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d500b3700 con 0x7f6d500b21d0 2024-04-14T08:41:13.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.778+0000 7f6d54836640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f6d500b39e0 con 0x7f6d500b4040 2024-04-14T08:41:13.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.778+0000 7f6d4e59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6d500b1df0 0x7f6d500b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:60434/0 (socket says 172.21.15.172:60434) 2024-04-14T08:41:13.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.778+0000 7f6d4e59c640 1 -- 172.21.15.172:0/3313268416 learned_addr learned my addr 172.21.15.172:0/3313268416 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:13.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.778+0000 7f6d4ed9d640 1 --2- 172.21.15.172:0/3313268416 >> v2:172.21.15.177:3300/0 conn(0x7f6d500b4040 0x7f6d500b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.778+0000 7f6d4d59a640 1 -- 172.21.15.172:0/3313268416 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3550517473 0 0) 0x7f6d5015c770 con 0x7f6d500b1df0 2024-04-14T08:41:13.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.778+0000 7f6d4d59a640 1 -- 172.21.15.172:0/3313268416 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d38003520 con 0x7f6d500b1df0 2024-04-14T08:41:13.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.778+0000 7f6d4d59a640 1 -- 172.21.15.172:0/3313268416 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 298382845 0 0) 0x7f6d500b3700 con 0x7f6d500b21d0 2024-04-14T08:41:13.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.778+0000 7f6d4d59a640 1 -- 172.21.15.172:0/3313268416 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d5015c770 con 0x7f6d500b21d0 2024-04-14T08:41:13.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.779+0000 7f6d4ed9d640 1 -- 172.21.15.172:0/3313268416 >> v1:172.21.15.177:6789/0 conn(0x7f6d500b21d0 legacy=0x7f6d5015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.779+0000 7f6d4ed9d640 1 -- 172.21.15.172:0/3313268416 >> v1:172.21.15.172:6789/0 conn(0x7f6d500b1df0 legacy=0x7f6d500b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.779+0000 7f6d4ed9d640 1 -- 172.21.15.172:0/3313268416 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d500b3cc0 con 0x7f6d500b4040 2024-04-14T08:41:13.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.779+0000 7f6d4ed9d640 1 --2- 172.21.15.172:0/3313268416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d500b4040 0x7f6d500b4460 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6d480097a0 tx=0x7f6d4802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3448dfbfe790a66e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:13.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.779+0000 7f6d4d59a640 1 -- 172.21.15.172:0/3313268416 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48034070 con 0x7f6d500b4040 2024-04-14T08:41:13.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.779+0000 7f6d4d59a640 1 -- 172.21.15.172:0/3313268416 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d48030610 con 0x7f6d500b4040 2024-04-14T08:41:13.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.779+0000 7f6d4d59a640 1 -- 172.21.15.172:0/3313268416 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d480309e0 con 0x7f6d500b4040 2024-04-14T08:41:13.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.780+0000 7f6d54836640 1 -- 172.21.15.172:0/3313268416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d500b4040 msgr2=0x7f6d500b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.780+0000 7f6d54836640 1 --2- 172.21.15.172:0/3313268416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d500b4040 0x7f6d500b4460 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6d480097a0 tx=0x7f6d4802b960 comp rx=0 tx=0).stop 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.780+0000 7f6d54836640 1 -- 172.21.15.172:0/3313268416 shutdown_connections 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.780+0000 7f6d54836640 1 --2- 172.21.15.172:0/3313268416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d500b4040 0x7f6d500b4460 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.780+0000 7f6d54836640 1 -- 172.21.15.172:0/3313268416 >> 172.21.15.172:0/3313268416 conn(0x7f6d5006ec30 msgr2=0x7f6d5006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.780+0000 7f6d54836640 1 -- 172.21.15.172:0/3313268416 shutdown_connections 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.780+0000 7f6d54836640 1 -- 172.21.15.172:0/3313268416 wait complete. 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.781+0000 7f6d54836640 1 Processor -- start 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.781+0000 7f6d54836640 1 -- start start 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.782+0000 7f6d54836640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d500b1df0 0x7f6d50157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.782+0000 7f6d54836640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d500b21d0 0x7f6d500cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.782+0000 7f6d54836640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d500b4040 0x7f6d500cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.782+0000 7f6d54836640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5015b7f0 con 0x7f6d500b21d0 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.782+0000 7f6d54836640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6d500b49a0 con 0x7f6d500b1df0 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.782+0000 7f6d54836640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6d501552d0 con 0x7f6d500b4040 2024-04-14T08:41:13.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.782+0000 7f6d4dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d500b21d0 0x7f6d500cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.782+0000 7f6d4e59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d500b1df0 0x7f6d50157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.782+0000 7f6d4dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d500b21d0 0x7f6d500cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44986/0 (socket says 172.21.15.172:44986) 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.782+0000 7f6d4dd9b640 1 -- 172.21.15.172:0/3604870939 learned_addr learned my addr 172.21.15.172:0/3604870939 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.782+0000 7f6d4ed9d640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d500b4040 0x7f6d500cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.783+0000 7f6d4dd9b640 1 -- 172.21.15.172:0/3604870939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d500b4040 msgr2=0x7f6d500cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.783+0000 7f6d4dd9b640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d500b4040 0x7f6d500cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.783+0000 7f6d4dd9b640 1 -- 172.21.15.172:0/3604870939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d500b1df0 msgr2=0x7f6d50157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.783+0000 7f6d4dd9b640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d500b1df0 0x7f6d50157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.783+0000 7f6d4dd9b640 1 -- 172.21.15.172:0/3604870939 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d500ce3b0 con 0x7f6d500b21d0 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.783+0000 7f6d4dd9b640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d500b21d0 0x7f6d500cd760 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c450 tx=0x7f6d4400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.783+0000 7f6d3effd640 1 -- 172.21.15.172:0/3604870939 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d440052d0 con 0x7f6d500b21d0 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.783+0000 7f6d3effd640 1 -- 172.21.15.172:0/3604870939 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d44005470 con 0x7f6d500b21d0 2024-04-14T08:41:13.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.783+0000 7f6d54836640 1 -- 172.21.15.172:0/3604870939 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d500ce690 con 0x7f6d500b21d0 2024-04-14T08:41:13.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.783+0000 7f6d3effd640 1 -- 172.21.15.172:0/3604870939 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44012870 con 0x7f6d500b21d0 2024-04-14T08:41:13.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.783+0000 7f6d54836640 1 -- 172.21.15.172:0/3604870939 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d500cec50 con 0x7f6d500b21d0 2024-04-14T08:41:13.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.784+0000 7f6d54836640 1 -- 172.21.15.172:0/3604870939 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d50068520 con 0x7f6d500b21d0 2024-04-14T08:41:13.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.784+0000 7f6d3effd640 1 -- 172.21.15.172:0/3604870939 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d44012a10 con 0x7f6d500b21d0 2024-04-14T08:41:13.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.785+0000 7f6d3effd640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3006c8f0 0x7f6d3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:13.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.785+0000 7f6d3effd640 1 -- 172.21.15.172:0/3604870939 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f6d4408b7e0 con 0x7f6d500b21d0 2024-04-14T08:41:13.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.785+0000 7f6d4e59c640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3006c8f0 0x7f6d3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:13.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.785+0000 7f6d4e59c640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3006c8f0 0x7f6d3006edb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6d400012e0 tx=0x7f6d40001ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:13.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:13.789+0000 7f6d3effd640 1 -- 172.21.15.172:0/3604870939 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6d44059500 con 0x7f6d500b21d0 2024-04-14T08:41:14.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.003+0000 7f6d54836640 1 -- 172.21.15.172:0/3604870939 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6d500b0bf0 con 0x7f6d500b21d0 2024-04-14T08:41:14.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.003+0000 7f6d3effd640 1 -- 172.21.15.172:0/3604870939 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f6d4405d800 con 0x7f6d500b21d0 2024-04-14T08:41:14.005 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:41:14.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.005+0000 7f6d54836640 1 -- 172.21.15.172:0/3604870939 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3006c8f0 msgr2=0x7f6d3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:14.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.005+0000 7f6d54836640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3006c8f0 0x7f6d3006edb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6d400012e0 tx=0x7f6d40001ab0 comp rx=0 tx=0).stop 2024-04-14T08:41:14.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.005+0000 7f6d54836640 1 -- 172.21.15.172:0/3604870939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d500b21d0 msgr2=0x7f6d500cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:14.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.005+0000 7f6d54836640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d500b21d0 0x7f6d500cd760 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c450 tx=0x7f6d4400c910 comp rx=0 tx=0).stop 2024-04-14T08:41:14.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.005+0000 7f6d54836640 1 -- 172.21.15.172:0/3604870939 shutdown_connections 2024-04-14T08:41:14.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.005+0000 7f6d54836640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6d3006c8f0 0x7f6d3006edb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:14.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.005+0000 7f6d54836640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d500b4040 0x7f6d500cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:14.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.005+0000 7f6d54836640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d500b21d0 0x7f6d500cd760 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:14.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.005+0000 7f6d54836640 1 --2- 172.21.15.172:0/3604870939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d500b1df0 0x7f6d50157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:14.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.005+0000 7f6d54836640 1 -- 172.21.15.172:0/3604870939 >> 172.21.15.172:0/3604870939 conn(0x7f6d5006ec30 msgr2=0x7f6d50071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:14.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.005+0000 7f6d54836640 1 -- 172.21.15.172:0/3604870939 shutdown_connections 2024-04-14T08:41:14.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14.005+0000 7f6d54836640 1 -- 172.21.15.172:0/3604870939 wait complete. 2024-04-14T08:41:14.018 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:41:14.018 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:41:14.018 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:41:14.018 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:41:14.018 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:41:14.018 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:41:14.019 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:41:14.019 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:41:14.019 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:41:14.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:14.019 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:41:14.035 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:41:14.035 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:41:14.035 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:41:14.035 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:14.035 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:41:14.035 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:14.035 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:41:14.035 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:41:14.035 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42195sec preferred_lft 42195sec 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:41:14.036 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:41:14.037 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:14.037 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:41:14.037 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:14.037 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:41:14.037 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:41:14.037 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:41:14.037 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:14.037 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:41:14.037 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:41:14.037 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:41:14.037 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:41:14.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:14.038 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:41:14.101 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:41:14.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:14.102 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:41:14.159 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:41:14.159 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:41:14.159 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:41:14.159 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:41:14.159 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:41:14.159 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:41:14.159 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:41:14.159 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:41:14.160 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:41:14.160 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:41:14.160 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:41:14.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:41:14.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:41:14.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:14.296 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:41:14.314 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:41:14.314 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:41:14.315 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:41:14.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:14.315 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:41:14.381 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:41:14.381 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:41:14.437 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:41:14.437 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:41:14.437 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:41:14.492 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:41:14.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:14.492 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:41:14.577 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:41:14.577 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:41:14.578 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:41:14.586 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:41:14.586 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:41:14.586 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:41:14.586 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:41:14.586 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:41:14.590 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:41:14.590 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:41:14.590 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:41:14.591 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:41:14.591 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:41:14.591 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:41:14.591 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:41:14.591 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:41:14.593 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:41:14.593 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:41:14.593 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:41:14.593 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:41:14.594 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:41:14.594 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:41:14.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:14.594 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:41:14.622 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:41:14.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:41:14.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:41:14.719 INFO:teuthology.orchestra.run.smithi172.stdout:client8607 2024-04-14T08:41:14.719 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:41:14.719 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:41:14.735 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:41:14.735 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:41:14.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:14.790 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-14T08:41:14.882 INFO:teuthology.orchestra.run.smithi172.stderr:6+0 records in 2024-04-14T08:41:14.882 INFO:teuthology.orchestra.run.smithi172.stderr:6+0 records out 2024-04-14T08:41:14.882 INFO:teuthology.orchestra.run.smithi172.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.034156 s, 184 MB/s 2024-04-14T08:41:14.882 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:41:14.882 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:41:14.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:14.898 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-14T08:41:14.991 INFO:teuthology.orchestra.run.smithi172.stderr:6+0 records in 2024-04-14T08:41:14.991 INFO:teuthology.orchestra.run.smithi172.stderr:6+0 records out 2024-04-14T08:41:14.991 INFO:teuthology.orchestra.run.smithi172.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0360072 s, 175 MB/s 2024-04-14T08:41:14.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:14.992 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-14T08:41:15.008 INFO:teuthology.orchestra.run.smithi172.stdout:./ 2024-04-14T08:41:15.008 INFO:teuthology.orchestra.run.smithi172.stdout:./file2_sixmegs 2024-04-14T08:41:15.008 INFO:teuthology.orchestra.run.smithi172.stdout:./file3_sixmegs 2024-04-14T08:41:15.008 INFO:teuthology.orchestra.run.smithi172.stdout:./file1_sixmegs 2024-04-14T08:41:15.009 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:41:15.009 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:41:15.009 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:41:15.009 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:41:15.009 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-14T08:41:15.009 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:41:15.113 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:41:15.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:41:15.123 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:41:15.123 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:41:15.123 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:41:15.123 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:41:15.123 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-14T08:41:15.123 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:41:15.229 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627777 2024-04-14T08:41:15.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:41:15.238 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:41:15.238 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:41:15.238 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:41:15.239 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:41:15.239 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-14T08:41:15.239 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:41:15.343 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627778 2024-04-14T08:41:15.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:41:15.352 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:41:15.352 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:41:15.353 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:41:15.353 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:41:15.353 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-14T08:41:15.353 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:41:15.457 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627776 2024-04-14T08:41:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:41:15.467 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:41:15.467 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:41:15.523 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:41:15.523 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:41:15.579 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:41:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.717+0000 7f0a4b8de640 1 Processor -- start 2024-04-14T08:41:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.718+0000 7f0a4b8de640 1 -- start start 2024-04-14T08:41:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.718+0000 7f0a4b8de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0a44072470 0x7f0a4415b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.718+0000 7f0a4b8de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0a44072970 0x7f0a4415b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.718+0000 7f0a4b8de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a440b3360 con 0x7f0a4415a690 2024-04-14T08:41:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.718+0000 7f0a4b8de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0a440b3640 con 0x7f0a44072970 2024-04-14T08:41:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.718+0000 7f0a4b8de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f0a440b3920 con 0x7f0a44072470 2024-04-14T08:41:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.718+0000 7f0a49e7b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0a44072970 0x7f0a4415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.718+0000 7f0a48e79640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0a44072470 0x7f0a4415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.719+0000 7f0a48e79640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0a44072470 0x7f0a4415b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:57934/0 (socket says 172.21.15.172:57934) 2024-04-14T08:41:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.719+0000 7f0a48e79640 1 -- 172.21.15.172:0/3996318278 learned_addr learned my addr 172.21.15.172:0/3996318278 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.719+0000 7f0a49e7b640 1 --2- 172.21.15.172:0/3996318278 >> v2:172.21.15.172:3300/0 conn(0x7f0a44072970 0x7f0a4415b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44996/0 (socket says 172.21.15.172:44996) 2024-04-14T08:41:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.719+0000 7f0a4967a640 1 --1- 172.21.15.172:0/3996318278 >> v1:172.21.15.177:6789/0 conn(0x7f0a4415a690 0x7f0a44071d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:49206/0 (socket says 172.21.15.172:49206) 2024-04-14T08:41:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.719+0000 7f0a48e79640 1 -- 172.21.15.172:0/3996318278 >> v2:172.21.15.172:3300/0 conn(0x7f0a44072970 msgr2=0x7f0a4415b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.719+0000 7f0a48e79640 1 --2- 172.21.15.172:0/3996318278 >> v2:172.21.15.172:3300/0 conn(0x7f0a44072970 0x7f0a4415b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.719+0000 7f0a48e79640 1 -- 172.21.15.172:0/3996318278 >> v1:172.21.15.177:6789/0 conn(0x7f0a4415a690 legacy=0x7f0a44071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.719+0000 7f0a48e79640 1 -- 172.21.15.172:0/3996318278 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a440b3c00 con 0x7f0a44072470 2024-04-14T08:41:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.719+0000 7f0a3bfff640 1 -- 172.21.15.172:0/3996318278 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2277686287 0 0) 0x7f0a440b3360 con 0x7f0a4415a690 2024-04-14T08:41:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.719+0000 7f0a49e7b640 1 --2- 172.21.15.172:0/3996318278 >> v2:172.21.15.172:3300/0 conn(0x7f0a44072970 0x7f0a4415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.720+0000 7f0a48e79640 1 --2- 172.21.15.172:0/3996318278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a44072470 0x7f0a4415b230 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0a440cc3a0 tx=0x7f0a3402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c33871945072e984 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.720+0000 7f0a3bfff640 1 -- 172.21.15.172:0/3996318278 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34035020 con 0x7f0a44072470 2024-04-14T08:41:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.720+0000 7f0a3bfff640 1 -- 172.21.15.172:0/3996318278 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a34031870 con 0x7f0a44072470 2024-04-14T08:41:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.720+0000 7f0a3bfff640 1 -- 172.21.15.172:0/3996318278 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34031c40 con 0x7f0a44072470 2024-04-14T08:41:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.720+0000 7f0a4b8de640 1 -- 172.21.15.172:0/3996318278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a44072470 msgr2=0x7f0a4415b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.720+0000 7f0a4b8de640 1 --2- 172.21.15.172:0/3996318278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a44072470 0x7f0a4415b230 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0a440cc3a0 tx=0x7f0a3402b910 comp rx=0 tx=0).stop 2024-04-14T08:41:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.720+0000 7f0a4b8de640 1 -- 172.21.15.172:0/3996318278 shutdown_connections 2024-04-14T08:41:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.720+0000 7f0a4b8de640 1 --2- 172.21.15.172:0/3996318278 >> v2:172.21.15.172:3300/0 conn(0x7f0a44072970 0x7f0a4415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.720+0000 7f0a4b8de640 1 --2- 172.21.15.172:0/3996318278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a44072470 0x7f0a4415b230 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.720+0000 7f0a4b8de640 1 -- 172.21.15.172:0/3996318278 >> 172.21.15.172:0/3996318278 conn(0x7f0a4406ec30 msgr2=0x7f0a4406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.720+0000 7f0a4b8de640 1 -- 172.21.15.172:0/3996318278 shutdown_connections 2024-04-14T08:41:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.720+0000 7f0a4b8de640 1 -- 172.21.15.172:0/3996318278 wait complete. 2024-04-14T08:41:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.721+0000 7f0a4b8de640 1 Processor -- start 2024-04-14T08:41:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.722+0000 7f0a4b8de640 1 -- start start 2024-04-14T08:41:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.722+0000 7f0a4b8de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a44072470 0x7f0a441457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.722+0000 7f0a4b8de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a44072970 0x7f0a44142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.722+0000 7f0a4b8de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a4415a690 0x7f0a44146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.722+0000 7f0a4b8de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4415aa70 con 0x7f0a44072970 2024-04-14T08:41:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.722+0000 7f0a4b8de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0a440b3100 con 0x7f0a4415a690 2024-04-14T08:41:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.722+0000 7f0a4b8de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4415abf0 con 0x7f0a44072470 2024-04-14T08:41:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.722+0000 7f0a48e79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a44072970 0x7f0a44142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.722+0000 7f0a48e79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a44072970 0x7f0a44142bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45004/0 (socket says 172.21.15.172:45004) 2024-04-14T08:41:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.722+0000 7f0a48e79640 1 -- 172.21.15.172:0/2045274628 learned_addr learned my addr 172.21.15.172:0/2045274628 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.722+0000 7f0a49e7b640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a4415a690 0x7f0a44146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.722+0000 7f0a4967a640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a44072470 0x7f0a441457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.723+0000 7f0a48e79640 1 -- 172.21.15.172:0/2045274628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a44072470 msgr2=0x7f0a441457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:15.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.723+0000 7f0a48e79640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a44072470 0x7f0a441457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:15.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.723+0000 7f0a48e79640 1 -- 172.21.15.172:0/2045274628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a4415a690 msgr2=0x7f0a44146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:15.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.723+0000 7f0a48e79640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a4415a690 0x7f0a44146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:15.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.723+0000 7f0a48e79640 1 -- 172.21.15.172:0/2045274628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a441430f0 con 0x7f0a44072970 2024-04-14T08:41:15.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.723+0000 7f0a48e79640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a44072970 0x7f0a44142bb0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009960 tx=0x7f0a340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:15.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.723+0000 7f0a39ffb640 1 -- 172.21.15.172:0/2045274628 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34041070 con 0x7f0a44072970 2024-04-14T08:41:15.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.723+0000 7f0a39ffb640 1 -- 172.21.15.172:0/2045274628 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a3403ad00 con 0x7f0a44072970 2024-04-14T08:41:15.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.723+0000 7f0a39ffb640 1 -- 172.21.15.172:0/2045274628 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a34035040 con 0x7f0a44072970 2024-04-14T08:41:15.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.723+0000 7f0a4b8de640 1 -- 172.21.15.172:0/2045274628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a441475b0 con 0x7f0a44072970 2024-04-14T08:41:15.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.723+0000 7f0a4b8de640 1 -- 172.21.15.172:0/2045274628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a44167b40 con 0x7f0a44072970 2024-04-14T08:41:15.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.724+0000 7f0a4b8de640 1 -- 172.21.15.172:0/2045274628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a44068520 con 0x7f0a44072970 2024-04-14T08:41:15.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.725+0000 7f0a39ffb640 1 -- 172.21.15.172:0/2045274628 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a34038020 con 0x7f0a44072970 2024-04-14T08:41:15.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.725+0000 7f0a39ffb640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a2806c920 0x7f0a2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:15.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.725+0000 7f0a4967a640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a2806c920 0x7f0a2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:15.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.725+0000 7f0a39ffb640 1 -- 172.21.15.172:0/2045274628 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f0a340ad8d0 con 0x7f0a44072970 2024-04-14T08:41:15.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.726+0000 7f0a4967a640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a2806c920 0x7f0a2806ede0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0a400012e0 tx=0x7f0a40001630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:15.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.730+0000 7f0a39ffb640 1 -- 172.21.15.172:0/2045274628 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0a441430f0 con 0x7f0a44072970 2024-04-14T08:41:15.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.906+0000 7f0a4b8de640 1 -- 172.21.15.172:0/2045274628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a440ceea0 con 0x7f0a44072970 2024-04-14T08:41:15.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.907+0000 7f0a39ffb640 1 -- 172.21.15.172:0/2045274628 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16936 (secure 0 0 0) 0x7f0a3407bc00 con 0x7f0a44072970 2024-04-14T08:41:15.908 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:15.909 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":101,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:41:06.143425+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:41:02.725765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-14T08:41:03.442157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:41:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.909+0000 7f0a4b8de640 1 -- 172.21.15.172:0/2045274628 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a2806c920 msgr2=0x7f0a2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.909+0000 7f0a4b8de640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a2806c920 0x7f0a2806ede0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0a400012e0 tx=0x7f0a40001630 comp rx=0 tx=0).stop 2024-04-14T08:41:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.909+0000 7f0a4b8de640 1 -- 172.21.15.172:0/2045274628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a44072970 msgr2=0x7f0a44142bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.909+0000 7f0a4b8de640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a44072970 0x7f0a44142bb0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009960 tx=0x7f0a340046c0 comp rx=0 tx=0).stop 2024-04-14T08:41:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.909+0000 7f0a4b8de640 1 -- 172.21.15.172:0/2045274628 shutdown_connections 2024-04-14T08:41:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.909+0000 7f0a4b8de640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a2806c920 0x7f0a2806ede0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.909+0000 7f0a4b8de640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a4415a690 0x7f0a44146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:15.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.909+0000 7f0a4b8de640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a44072970 0x7f0a44142bb0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:15.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.909+0000 7f0a4b8de640 1 --2- 172.21.15.172:0/2045274628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a44072470 0x7f0a441457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:15.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.909+0000 7f0a4b8de640 1 -- 172.21.15.172:0/2045274628 >> 172.21.15.172:0/2045274628 conn(0x7f0a4406ec30 msgr2=0x7f0a440cf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:15.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.909+0000 7f0a4b8de640 1 -- 172.21.15.172:0/2045274628 shutdown_connections 2024-04-14T08:41:15.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:15.909+0000 7f0a4b8de640 1 -- 172.21.15.172:0/2045274628 wait complete. 2024-04-14T08:41:15.922 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:41:02.725765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:41:03.442157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:41:15.922 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:41:16.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.026+0000 7f5b4a1c8640 1 Processor -- start 2024-04-14T08:41:16.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.026+0000 7f5b4a1c8640 1 -- start start 2024-04-14T08:41:16.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.026+0000 7f5b4a1c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b4406fb70 0x7f5b44073090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.026+0000 7f5b4a1c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b440ceee0 con 0x7f5b440ccca0 2024-04-14T08:41:16.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.026+0000 7f5b4a1c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b44070150 con 0x7f5b4406bc30 2024-04-14T08:41:16.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.026+0000 7f5b4a1c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5b440722d0 con 0x7f5b4406fb70 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.027+0000 7f5b437fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5b440ccca0 0x7f5b4406b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:60446/0 (socket says 172.21.15.172:60446) 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.027+0000 7f5b437fe640 1 -- 172.21.15.172:0/2422583646 learned_addr learned my addr 172.21.15.172:0/2422583646 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.027+0000 7f5b43fff640 1 --2- 172.21.15.172:0/2422583646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b4406fb70 0x7f5b44073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.027+0000 7f5b427fc640 1 -- 172.21.15.172:0/2422583646 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3205383094 0 0) 0x7f5b440ceee0 con 0x7f5b440ccca0 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.027+0000 7f5b427fc640 1 -- 172.21.15.172:0/2422583646 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b2c003520 con 0x7f5b440ccca0 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.027+0000 7f5b427fc640 1 -- 172.21.15.172:0/2422583646 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2183966089 0 0) 0x7f5b44070150 con 0x7f5b4406bc30 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.027+0000 7f5b427fc640 1 -- 172.21.15.172:0/2422583646 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b440ceee0 con 0x7f5b4406bc30 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.027+0000 7f5b43fff640 1 -- 172.21.15.172:0/2422583646 >> v1:172.21.15.177:6789/0 conn(0x7f5b4406bc30 legacy=0x7f5b4406e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.027+0000 7f5b43fff640 1 -- 172.21.15.172:0/2422583646 >> v1:172.21.15.172:6789/0 conn(0x7f5b440ccca0 legacy=0x7f5b4406b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.027+0000 7f5b43fff640 1 -- 172.21.15.172:0/2422583646 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b440725b0 con 0x7f5b4406fb70 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.028+0000 7f5b43fff640 1 --2- 172.21.15.172:0/2422583646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b4406fb70 0x7f5b44073090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009dc0 tx=0x7f5b3c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd127cbf1d30dcfd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.028+0000 7f5b427fc640 1 -- 172.21.15.172:0/2422583646 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c039070 con 0x7f5b4406fb70 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.028+0000 7f5b427fc640 1 -- 172.21.15.172:0/2422583646 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3c02cc40 con 0x7f5b4406fb70 2024-04-14T08:41:16.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.028+0000 7f5b427fc640 1 -- 172.21.15.172:0/2422583646 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c034440 con 0x7f5b4406fb70 2024-04-14T08:41:16.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.028+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2422583646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b4406fb70 msgr2=0x7f5b44073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.028+0000 7f5b4a1c8640 1 --2- 172.21.15.172:0/2422583646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b4406fb70 0x7f5b44073090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009dc0 tx=0x7f5b3c02be90 comp rx=0 tx=0).stop 2024-04-14T08:41:16.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.028+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2422583646 shutdown_connections 2024-04-14T08:41:16.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.028+0000 7f5b4a1c8640 1 --2- 172.21.15.172:0/2422583646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b4406fb70 0x7f5b44073090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.028+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2422583646 >> 172.21.15.172:0/2422583646 conn(0x7f5b440cda00 msgr2=0x7f5b440cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:16.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.028+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2422583646 shutdown_connections 2024-04-14T08:41:16.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.028+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2422583646 wait complete. 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.030+0000 7f5b4a1c8640 1 Processor -- start 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.030+0000 7f5b4a1c8640 1 -- start start 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.030+0000 7f5b4a1c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b4406bc30 0x7f5b440c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.030+0000 7f5b4a1c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b4406fb70 0x7f5b44155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.030+0000 7f5b4a1c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b440ccca0 0x7f5b440b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.030+0000 7f5b4a1c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b440b0190 con 0x7f5b440ccca0 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.030+0000 7f5b4a1c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b440cd080 con 0x7f5b4406fb70 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.030+0000 7f5b4a1c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5b440b0310 con 0x7f5b4406bc30 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b43fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b440ccca0 0x7f5b440b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b4406bc30 0x7f5b440c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b43fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b440ccca0 0x7f5b440b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45018/0 (socket says 172.21.15.172:45018) 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b43fff640 1 -- 172.21.15.172:0/2342928432 learned_addr learned my addr 172.21.15.172:0/2342928432 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:16.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b42ffd640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b4406fb70 0x7f5b44155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b43fff640 1 -- 172.21.15.172:0/2342928432 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b4406bc30 msgr2=0x7f5b440c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b43fff640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b4406bc30 0x7f5b440c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b43fff640 1 -- 172.21.15.172:0/2342928432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b4406fb70 msgr2=0x7f5b44155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b43fff640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b4406fb70 0x7f5b44155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b43fff640 1 -- 172.21.15.172:0/2342928432 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b44155870 con 0x7f5b440ccca0 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b43fff640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b440ccca0 0x7f5b440b51c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c037040 tx=0x7f5b3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b2bfff640 1 -- 172.21.15.172:0/2342928432 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c039050 con 0x7f5b440ccca0 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b2bfff640 1 -- 172.21.15.172:0/2342928432 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3c045070 con 0x7f5b440ccca0 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2342928432 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b441f7fc0 con 0x7f5b440ccca0 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b2bfff640 1 -- 172.21.15.172:0/2342928432 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c040690 con 0x7f5b440ccca0 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.031+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2342928432 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b441f8580 con 0x7f5b440ccca0 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.032+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2342928432 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b44068590 con 0x7f5b440ccca0 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.033+0000 7f5b2bfff640 1 -- 172.21.15.172:0/2342928432 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b3c040830 con 0x7f5b440ccca0 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.033+0000 7f5b2bfff640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5b3006c8f0 0x7f5b3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.033+0000 7f5b2bfff640 1 -- 172.21.15.172:0/2342928432 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f5b3c0ae590 con 0x7f5b440ccca0 2024-04-14T08:41:16.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.033+0000 7f5b437fe640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5b3006c8f0 0x7f5b3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.034+0000 7f5b437fe640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5b3006c8f0 0x7f5b3006edb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5b34000f60 tx=0x7f5b34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:16.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.037+0000 7f5b2bfff640 1 -- 172.21.15.172:0/2342928432 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5b3c07c160 con 0x7f5b440ccca0 2024-04-14T08:41:16.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.211+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2342928432 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b4406c120 con 0x7f5b440ccca0 2024-04-14T08:41:16.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.212+0000 7f5b2bfff640 1 -- 172.21.15.172:0/2342928432 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f5b3c080460 con 0x7f5b440ccca0 2024-04-14T08:41:16.213 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:41:16.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.214+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2342928432 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5b3006c8f0 msgr2=0x7f5b3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.214+0000 7f5b4a1c8640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5b3006c8f0 0x7f5b3006edb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5b34000f60 tx=0x7f5b34009040 comp rx=0 tx=0).stop 2024-04-14T08:41:16.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.214+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2342928432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b440ccca0 msgr2=0x7f5b440b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.214+0000 7f5b4a1c8640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b440ccca0 0x7f5b440b51c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c037040 tx=0x7f5b3c004a40 comp rx=0 tx=0).stop 2024-04-14T08:41:16.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.214+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2342928432 shutdown_connections 2024-04-14T08:41:16.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.214+0000 7f5b4a1c8640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5b3006c8f0 0x7f5b3006edb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.214+0000 7f5b4a1c8640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b440ccca0 0x7f5b440b51c0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.214+0000 7f5b4a1c8640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b4406fb70 0x7f5b44155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.214+0000 7f5b4a1c8640 1 --2- 172.21.15.172:0/2342928432 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5b4406bc30 0x7f5b440c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.214+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2342928432 >> 172.21.15.172:0/2342928432 conn(0x7f5b440cda00 msgr2=0x7f5b440af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:16.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.214+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2342928432 shutdown_connections 2024-04-14T08:41:16.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.214+0000 7f5b4a1c8640 1 -- 172.21.15.172:0/2342928432 wait complete. 2024-04-14T08:41:16.226 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:41:16.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.329+0000 7f107cede640 1 Processor -- start 2024-04-14T08:41:16.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.329+0000 7f107cede640 1 -- start start 2024-04-14T08:41:16.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f107cede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107815e430 0x7f1078162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f107cede640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10780cefb0 con 0x7f10780d2e00 2024-04-14T08:41:16.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f107cede640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10780cf290 con 0x7f1078162540 2024-04-14T08:41:16.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f107cede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f10780cf570 con 0x7f107815e430 2024-04-14T08:41:16.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f1076d9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f10780d2e00 0x7f10780b4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:60454/0 (socket says 172.21.15.172:60454) 2024-04-14T08:41:16.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f1076d9d640 1 -- 172.21.15.172:0/817879998 learned_addr learned my addr 172.21.15.172:0/817879998 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f107659c640 1 --2- 172.21.15.172:0/817879998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107815e430 0x7f1078162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f1075d9b640 1 -- 172.21.15.172:0/817879998 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3037571030 0 0) 0x7f10780cefb0 con 0x7f10780d2e00 2024-04-14T08:41:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f1075d9b640 1 -- 172.21.15.172:0/817879998 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1064003520 con 0x7f10780d2e00 2024-04-14T08:41:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f1075d9b640 1 -- 172.21.15.172:0/817879998 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2594128436 0 0) 0x7f10780cf290 con 0x7f1078162540 2024-04-14T08:41:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f1075d9b640 1 -- 172.21.15.172:0/817879998 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10780cefb0 con 0x7f1078162540 2024-04-14T08:41:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f107659c640 1 -- 172.21.15.172:0/817879998 >> v1:172.21.15.177:6789/0 conn(0x7f1078162540 legacy=0x7f10780b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f107659c640 1 -- 172.21.15.172:0/817879998 >> v1:172.21.15.172:6789/0 conn(0x7f10780d2e00 legacy=0x7f10780b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.330+0000 7f107659c640 1 -- 172.21.15.172:0/817879998 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10780cf850 con 0x7f107815e430 2024-04-14T08:41:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.331+0000 7f107659c640 1 --2- 172.21.15.172:0/817879998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107815e430 0x7f1078162000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f10680097d0 tx=0x7f106802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f97f1a0e2919d99 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.331+0000 7f1075d9b640 1 -- 172.21.15.172:0/817879998 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068035020 con 0x7f107815e430 2024-04-14T08:41:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.331+0000 7f1075d9b640 1 -- 172.21.15.172:0/817879998 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1068031890 con 0x7f107815e430 2024-04-14T08:41:16.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.331+0000 7f1075d9b640 1 -- 172.21.15.172:0/817879998 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068031c40 con 0x7f107815e430 2024-04-14T08:41:16.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.331+0000 7f107cede640 1 -- 172.21.15.172:0/817879998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107815e430 msgr2=0x7f1078162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.331+0000 7f107cede640 1 --2- 172.21.15.172:0/817879998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107815e430 0x7f1078162000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f10680097d0 tx=0x7f106802b930 comp rx=0 tx=0).stop 2024-04-14T08:41:16.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.332+0000 7f107cede640 1 -- 172.21.15.172:0/817879998 shutdown_connections 2024-04-14T08:41:16.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.332+0000 7f107cede640 1 --2- 172.21.15.172:0/817879998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107815e430 0x7f1078162000 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.332+0000 7f107cede640 1 -- 172.21.15.172:0/817879998 >> 172.21.15.172:0/817879998 conn(0x7f107809d560 msgr2=0x7f10780d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:16.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.332+0000 7f107cede640 1 -- 172.21.15.172:0/817879998 shutdown_connections 2024-04-14T08:41:16.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.332+0000 7f107cede640 1 -- 172.21.15.172:0/817879998 wait complete. 2024-04-14T08:41:16.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.333+0000 7f107cede640 1 Processor -- start 2024-04-14T08:41:16.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.333+0000 7f107cede640 1 -- start start 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107cede640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10780d2e00 0x7f10780c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107cede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107815e430 0x7f10780cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107cede640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1078162540 0x7f10780b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107cede640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10780d4e20 con 0x7f10780d2e00 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107cede640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f107815e810 con 0x7f1078162540 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107cede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f10780b4850 con 0x7f107815e430 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10780d2e00 0x7f10780c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10780d2e00 0x7f10780c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45030/0 (socket says 172.21.15.172:45030) 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107659c640 1 -- 172.21.15.172:0/3972007041 learned_addr learned my addr 172.21.15.172:0/3972007041 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f106ffff640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107815e430 0x7f10780cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f1076d9d640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1078162540 0x7f10780b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107659c640 1 -- 172.21.15.172:0/3972007041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107815e430 msgr2=0x7f10780cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107659c640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107815e430 0x7f10780cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107659c640 1 -- 172.21.15.172:0/3972007041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1078162540 msgr2=0x7f10780b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107659c640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1078162540 0x7f10780b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107659c640 1 -- 172.21.15.172:0/3972007041 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10780ca3a0 con 0x7f10780d2e00 2024-04-14T08:41:16.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107659c640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10780d2e00 0x7f10780c9e60 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f10680098f0 tx=0x7f10680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:16.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f106effd640 1 -- 172.21.15.172:0/3972007041 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068035040 con 0x7f10780d2e00 2024-04-14T08:41:16.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f106effd640 1 -- 172.21.15.172:0/3972007041 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1068030b10 con 0x7f10780d2e00 2024-04-14T08:41:16.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f106effd640 1 -- 172.21.15.172:0/3972007041 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068044430 con 0x7f10780d2e00 2024-04-14T08:41:16.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107cede640 1 -- 172.21.15.172:0/3972007041 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10781f7f20 con 0x7f10780d2e00 2024-04-14T08:41:16.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.334+0000 7f107cede640 1 -- 172.21.15.172:0/3972007041 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10781f84e0 con 0x7f10780d2e00 2024-04-14T08:41:16.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.335+0000 7f107cede640 1 -- 172.21.15.172:0/3972007041 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1078068590 con 0x7f10780d2e00 2024-04-14T08:41:16.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.336+0000 7f106effd640 1 -- 172.21.15.172:0/3972007041 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1068038020 con 0x7f10780d2e00 2024-04-14T08:41:16.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.336+0000 7f106effd640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f105806c920 0x7f105806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.336+0000 7f106effd640 1 -- 172.21.15.172:0/3972007041 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f10680ae110 con 0x7f10780d2e00 2024-04-14T08:41:16.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.336+0000 7f106ffff640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f105806c920 0x7f105806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.337+0000 7f106ffff640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f105806c920 0x7f105806ede0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1060007fd0 tx=0x7f1060009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:16.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.340+0000 7f106effd640 1 -- 172.21.15.172:0/3972007041 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f10780ca3a0 con 0x7f10780d2e00 2024-04-14T08:41:16.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.518+0000 7f107cede640 1 -- 172.21.15.172:0/3972007041 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f107806ca30 con 0x7f10780d2e00 2024-04-14T08:41:16.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.518+0000 7f106effd640 1 -- 172.21.15.172:0/3972007041 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f106802d040 con 0x7f10780d2e00 2024-04-14T08:41:16.520 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:41:16.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.520+0000 7f107cede640 1 -- 172.21.15.172:0/3972007041 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f105806c920 msgr2=0x7f105806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.520+0000 7f107cede640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f105806c920 0x7f105806ede0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1060007fd0 tx=0x7f1060009040 comp rx=0 tx=0).stop 2024-04-14T08:41:16.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.520+0000 7f107cede640 1 -- 172.21.15.172:0/3972007041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10780d2e00 msgr2=0x7f10780c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.520+0000 7f107cede640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10780d2e00 0x7f10780c9e60 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f10680098f0 tx=0x7f10680046c0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.520+0000 7f107cede640 1 -- 172.21.15.172:0/3972007041 shutdown_connections 2024-04-14T08:41:16.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.520+0000 7f107cede640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f105806c920 0x7f105806ede0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.520+0000 7f107cede640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1078162540 0x7f10780b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.520+0000 7f107cede640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107815e430 0x7f10780cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.520+0000 7f107cede640 1 --2- 172.21.15.172:0/3972007041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10780d2e00 0x7f10780c9e60 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.520+0000 7f107cede640 1 -- 172.21.15.172:0/3972007041 >> 172.21.15.172:0/3972007041 conn(0x7f107809d560 msgr2=0x7f10780b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:16.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.520+0000 7f107cede640 1 -- 172.21.15.172:0/3972007041 shutdown_connections 2024-04-14T08:41:16.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.520+0000 7f107cede640 1 -- 172.21.15.172:0/3972007041 wait complete. 2024-04-14T08:41:16.533 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:41:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.635+0000 7f5ecbfa3640 1 Processor -- start 2024-04-14T08:41:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.636+0000 7f5ecbfa3640 1 -- start start 2024-04-14T08:41:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.636+0000 7f5ecbfa3640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5ec415e430 0x7f5ec4162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.636+0000 7f5ecbfa3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ec4162540 0x7f5ec41629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.636+0000 7f5ecbfa3640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ec40b3a30 0x7f5ec40b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.636+0000 7f5ecbfa3640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f5ec40cefb0 con 0x7f5ec40b3a30 2024-04-14T08:41:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.636+0000 7f5ecbfa3640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5ec40cf290 con 0x7f5ec415e430 2024-04-14T08:41:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.636+0000 7f5ecbfa3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec40cf570 con 0x7f5ec4162540 2024-04-14T08:41:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.637+0000 7f5eca540640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ec40b3a30 0x7f5ec40b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.637+0000 7f5eca540640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ec40b3a30 0x7f5ec40b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45042/0 (socket says 172.21.15.172:45042) 2024-04-14T08:41:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.637+0000 7f5eca540640 1 -- 172.21.15.172:0/1410902396 learned_addr learned my addr 172.21.15.172:0/1410902396 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.637+0000 7f5ec953e640 1 --2- 172.21.15.172:0/1410902396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ec4162540 0x7f5ec41629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.637+0000 7f5ec9d3f640 1 --2- 172.21.15.172:0/1410902396 >> v2:172.21.15.177:3300/0 conn(0x7f5ec415e430 0x7f5ec4162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.637+0000 7f5eca540640 1 -- 172.21.15.172:0/1410902396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ec4162540 msgr2=0x7f5ec41629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.637+0000 7f5eca540640 1 --2- 172.21.15.172:0/1410902396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ec4162540 0x7f5ec41629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.637+0000 7f5eca540640 1 -- 172.21.15.172:0/1410902396 >> v2:172.21.15.177:3300/0 conn(0x7f5ec415e430 msgr2=0x7f5ec4162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.637+0000 7f5eca540640 1 --2- 172.21.15.172:0/1410902396 >> v2:172.21.15.177:3300/0 conn(0x7f5ec415e430 0x7f5ec4162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.637+0000 7f5eca540640 1 -- 172.21.15.172:0/1410902396 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec40cf850 con 0x7f5ec40b3a30 2024-04-14T08:41:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5eca540640 1 --2- 172.21.15.172:0/1410902396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec40b3a30 0x7f5ec40b3e60 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40b6db0 tx=0x7f5eb402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb169c0f26a0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5ec8d3d640 1 -- 172.21.15.172:0/1410902396 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4035020 con 0x7f5ec40b3a30 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5ec8d3d640 1 -- 172.21.15.172:0/1410902396 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb4031870 con 0x7f5ec40b3a30 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5ec8d3d640 1 -- 172.21.15.172:0/1410902396 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4031c40 con 0x7f5ec40b3a30 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/1410902396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec40b3a30 msgr2=0x7f5ec40b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5ecbfa3640 1 --2- 172.21.15.172:0/1410902396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec40b3a30 0x7f5ec40b3e60 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40b6db0 tx=0x7f5eb402b910 comp rx=0 tx=0).stop 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/1410902396 shutdown_connections 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5ecbfa3640 1 --2- 172.21.15.172:0/1410902396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec40b3a30 0x7f5ec40b3e60 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5ecbfa3640 1 --2- 172.21.15.172:0/1410902396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ec4162540 0x7f5ec41629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5ecbfa3640 1 --2- 172.21.15.172:0/1410902396 >> v2:172.21.15.177:3300/0 conn(0x7f5ec415e430 0x7f5ec4162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/1410902396 >> 172.21.15.172:0/1410902396 conn(0x7f5ec409d560 msgr2=0x7f5ec40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/1410902396 shutdown_connections 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.638+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/1410902396 wait complete. 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.639+0000 7f5ecbfa3640 1 Processor -- start 2024-04-14T08:41:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.640+0000 7f5ecbfa3640 1 -- start start 2024-04-14T08:41:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.640+0000 7f5ecbfa3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ec40b3a30 0x7f5ec41f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.640+0000 7f5ecbfa3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ec415e430 0x7f5ec41e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.640+0000 7f5ecbfa3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec4162540 0x7f5ec40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.640+0000 7f5ecbfa3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec40d4e20 con 0x7f5ec4162540 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.640+0000 7f5ecbfa3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec415e810 con 0x7f5ec415e430 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.640+0000 7f5ecbfa3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec40d0050 con 0x7f5ec40b3a30 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.640+0000 7f5eca540640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec4162540 0x7f5ec40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.640+0000 7f5eca540640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec4162540 0x7f5ec40b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45054/0 (socket says 172.21.15.172:45054) 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.640+0000 7f5eca540640 1 -- 172.21.15.172:0/4207502731 learned_addr learned my addr 172.21.15.172:0/4207502731 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.640+0000 7f5ec9d3f640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ec40b3a30 0x7f5ec41f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.640+0000 7f5ec953e640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ec415e430 0x7f5ec41e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.641+0000 7f5eca540640 1 -- 172.21.15.172:0/4207502731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ec40b3a30 msgr2=0x7f5ec41f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.641+0000 7f5eca540640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ec40b3a30 0x7f5ec41f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.641+0000 7f5eca540640 1 -- 172.21.15.172:0/4207502731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ec415e430 msgr2=0x7f5ec41e88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.641+0000 7f5eca540640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ec415e430 0x7f5ec41e88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.641+0000 7f5eca540640 1 -- 172.21.15.172:0/4207502731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec40c8f70 con 0x7f5ec4162540 2024-04-14T08:41:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.641+0000 7f5eca540640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec4162540 0x7f5ec40b4e60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40090b0 tx=0x7f5eb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.641+0000 7f5ec27fc640 1 -- 172.21.15.172:0/4207502731 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4040070 con 0x7f5ec4162540 2024-04-14T08:41:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.641+0000 7f5ec27fc640 1 -- 172.21.15.172:0/4207502731 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb4030c20 con 0x7f5ec4162540 2024-04-14T08:41:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.641+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/4207502731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec40c9250 con 0x7f5ec4162540 2024-04-14T08:41:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.641+0000 7f5ec27fc640 1 -- 172.21.15.172:0/4207502731 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4035040 con 0x7f5ec4162540 2024-04-14T08:41:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.641+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/4207502731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec41f3f40 con 0x7f5ec4162540 2024-04-14T08:41:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.642+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/4207502731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec4068590 con 0x7f5ec4162540 2024-04-14T08:41:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.642+0000 7f5ec27fc640 1 -- 172.21.15.172:0/4207502731 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5eb404b060 con 0x7f5ec4162540 2024-04-14T08:41:16.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.643+0000 7f5ec27fc640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5eb006c920 0x7f5eb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.643+0000 7f5ec27fc640 1 -- 172.21.15.172:0/4207502731 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f5eb40ad670 con 0x7f5ec4162540 2024-04-14T08:41:16.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.643+0000 7f5ec9d3f640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5eb006c920 0x7f5eb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.643+0000 7f5ec9d3f640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5eb006c920 0x7f5eb006ede0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40c8220 tx=0x7f5ebc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:16.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.647+0000 7f5ec27fc640 1 -- 172.21.15.172:0/4207502731 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5eb407b350 con 0x7f5ec4162540 2024-04-14T08:41:16.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.821+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/4207502731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5ec40d18a0 con 0x7f5ec4162540 2024-04-14T08:41:16.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.821+0000 7f5ec27fc640 1 -- 172.21.15.172:0/4207502731 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f5eb407f650 con 0x7f5ec4162540 2024-04-14T08:41:16.823 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:41:16.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.823+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/4207502731 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5eb006c920 msgr2=0x7f5eb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.823+0000 7f5ecbfa3640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5eb006c920 0x7f5eb006ede0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40c8220 tx=0x7f5ebc009190 comp rx=0 tx=0).stop 2024-04-14T08:41:16.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.823+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/4207502731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec4162540 msgr2=0x7f5ec40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.823+0000 7f5ecbfa3640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec4162540 0x7f5ec40b4e60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40090b0 tx=0x7f5eb40046c0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.823+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/4207502731 shutdown_connections 2024-04-14T08:41:16.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.823+0000 7f5ecbfa3640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5eb006c920 0x7f5eb006ede0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.823+0000 7f5ecbfa3640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec4162540 0x7f5ec40b4e60 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.823+0000 7f5ecbfa3640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ec415e430 0x7f5ec41e88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.823+0000 7f5ecbfa3640 1 --2- 172.21.15.172:0/4207502731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ec40b3a30 0x7f5ec41f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.823+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/4207502731 >> 172.21.15.172:0/4207502731 conn(0x7f5ec409d560 msgr2=0x7f5ec40b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:16.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.823+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/4207502731 shutdown_connections 2024-04-14T08:41:16.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.824+0000 7f5ecbfa3640 1 -- 172.21.15.172:0/4207502731 wait complete. 2024-04-14T08:41:16.836 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:41:16.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.938+0000 7f9b39715640 1 Processor -- start 2024-04-14T08:41:16.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.938+0000 7f9b39715640 1 -- start start 2024-04-14T08:41:16.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.939+0000 7f9b39715640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9b3406d040 0x7f9b340b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.939+0000 7f9b39715640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b340d12e0 con 0x7f9b3415e820 2024-04-14T08:41:16.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.939+0000 7f9b39715640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b340d1580 con 0x7f9b340d0d30 2024-04-14T08:41:16.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.939+0000 7f9b39715640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f9b3406cb90 con 0x7f9b3406d040 2024-04-14T08:41:16.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.939+0000 7f9b32ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9b3406d040 0x7f9b340b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.939+0000 7f9b2a7fc640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9b3415e820 0x7f9b3406d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:60464/0 (socket says 172.21.15.172:60464) 2024-04-14T08:41:16.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.939+0000 7f9b2a7fc640 1 -- 172.21.15.172:0/2989202560 learned_addr learned my addr 172.21.15.172:0/2989202560 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:16.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.939+0000 7f9b32ffd640 1 -- 172.21.15.172:0/2989202560 >> v1:172.21.15.177:6789/0 conn(0x7f9b340d0d30 legacy=0x7f9b340cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.939+0000 7f9b32ffd640 1 -- 172.21.15.172:0/2989202560 >> v1:172.21.15.172:6789/0 conn(0x7f9b3415e820 legacy=0x7f9b3406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.939+0000 7f9b32ffd640 1 -- 172.21.15.172:0/2989202560 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b340cc8b0 con 0x7f9b3406d040 2024-04-14T08:41:16.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.940+0000 7f9b32ffd640 1 --2- 172.21.15.172:0/2989202560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3406d040 0x7f9b340b4620 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f9b240097a0 tx=0x7f9b2402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513cb1bd8a80b334 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:16.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.940+0000 7f9b327fc640 1 -- 172.21.15.172:0/2989202560 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24034070 con 0x7f9b3406d040 2024-04-14T08:41:16.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.940+0000 7f9b327fc640 1 -- 172.21.15.172:0/2989202560 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b24030610 con 0x7f9b3406d040 2024-04-14T08:41:16.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.940+0000 7f9b327fc640 1 -- 172.21.15.172:0/2989202560 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b240309e0 con 0x7f9b3406d040 2024-04-14T08:41:16.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.941+0000 7f9b39715640 1 -- 172.21.15.172:0/2989202560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3406d040 msgr2=0x7f9b340b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.941+0000 7f9b39715640 1 --2- 172.21.15.172:0/2989202560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3406d040 0x7f9b340b4620 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f9b240097a0 tx=0x7f9b2402b960 comp rx=0 tx=0).stop 2024-04-14T08:41:16.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.941+0000 7f9b39715640 1 -- 172.21.15.172:0/2989202560 shutdown_connections 2024-04-14T08:41:16.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.941+0000 7f9b39715640 1 --2- 172.21.15.172:0/2989202560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3406d040 0x7f9b340b4620 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.941+0000 7f9b39715640 1 -- 172.21.15.172:0/2989202560 >> 172.21.15.172:0/2989202560 conn(0x7f9b340b3b00 msgr2=0x7f9b340b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:16.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.941+0000 7f9b39715640 1 -- 172.21.15.172:0/2989202560 shutdown_connections 2024-04-14T08:41:16.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.941+0000 7f9b39715640 1 -- 172.21.15.172:0/2989202560 wait complete. 2024-04-14T08:41:16.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.942+0000 7f9b39715640 1 Processor -- start 2024-04-14T08:41:16.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.942+0000 7f9b39715640 1 -- start start 2024-04-14T08:41:16.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b39715640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3406d040 0x7f9b340c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b39715640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b340d0d30 0x7f9b3414a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b39715640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b3415e820 0x7f9b341e5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b39715640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9b340cdb90 con 0x7f9b3406d040 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b39715640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9b340cda10 con 0x7f9b340d0d30 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b39715640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9b340cdd10 con 0x7f9b3415e820 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b32ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3406d040 0x7f9b340c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b32ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3406d040 0x7f9b340c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45072/0 (socket says 172.21.15.172:45072) 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b32ffd640 1 -- 172.21.15.172:0/2635420598 learned_addr learned my addr 172.21.15.172:0/2635420598 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b2a7fc640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b340d0d30 0x7f9b3414a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b337fe640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b3415e820 0x7f9b341e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b32ffd640 1 -- 172.21.15.172:0/2635420598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b3415e820 msgr2=0x7f9b341e5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b32ffd640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b3415e820 0x7f9b341e5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b32ffd640 1 -- 172.21.15.172:0/2635420598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b340d0d30 msgr2=0x7f9b3414a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b32ffd640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b340d0d30 0x7f9b3414a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:16.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b32ffd640 1 -- 172.21.15.172:0/2635420598 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b341e6f00 con 0x7f9b3406d040 2024-04-14T08:41:16.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b32ffd640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3406d040 0x7f9b340c98b0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f9b2402bf70 tx=0x7f9b240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:16.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.943+0000 7f9b2bfff640 1 -- 172.21.15.172:0/2635420598 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24034040 con 0x7f9b3406d040 2024-04-14T08:41:16.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.944+0000 7f9b2bfff640 1 -- 172.21.15.172:0/2635420598 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b24039a10 con 0x7f9b3406d040 2024-04-14T08:41:16.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.944+0000 7f9b39715640 1 -- 172.21.15.172:0/2635420598 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b341e3ee0 con 0x7f9b3406d040 2024-04-14T08:41:16.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.944+0000 7f9b2bfff640 1 -- 172.21.15.172:0/2635420598 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24039e20 con 0x7f9b3406d040 2024-04-14T08:41:16.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.944+0000 7f9b39715640 1 -- 172.21.15.172:0/2635420598 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b341e6700 con 0x7f9b3406d040 2024-04-14T08:41:16.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.945+0000 7f9b39715640 1 -- 172.21.15.172:0/2635420598 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b34068590 con 0x7f9b3406d040 2024-04-14T08:41:16.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.945+0000 7f9b2bfff640 1 -- 172.21.15.172:0/2635420598 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b24037070 con 0x7f9b3406d040 2024-04-14T08:41:16.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.945+0000 7f9b2bfff640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9b1406c8f0 0x7f9b1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:16.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.946+0000 7f9b2bfff640 1 -- 172.21.15.172:0/2635420598 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f9b240add70 con 0x7f9b3406d040 2024-04-14T08:41:16.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.946+0000 7f9b2a7fc640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9b1406c8f0 0x7f9b1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:16.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.946+0000 7f9b2a7fc640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9b1406c8f0 0x7f9b1406edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9b341e7a60 tx=0x7f9b1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:16.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:16.950+0000 7f9b2bfff640 1 -- 172.21.15.172:0/2635420598 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9b341e6f00 con 0x7f9b3406d040 2024-04-14T08:41:17.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.137+0000 7f9b39715640 1 -- 172.21.15.172:0/2635420598 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9b3415bcd0 con 0x7f9b3406d040 2024-04-14T08:41:17.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.137+0000 7f9b2bfff640 1 -- 172.21.15.172:0/2635420598 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) ==== 80+0+0 (secure 0 0 0) 0x7f9b2407b930 con 0x7f9b3406d040 2024-04-14T08:41:17.139 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:41:17.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.139+0000 7f9b39715640 1 -- 172.21.15.172:0/2635420598 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9b1406c8f0 msgr2=0x7f9b1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.139+0000 7f9b39715640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9b1406c8f0 0x7f9b1406edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9b341e7a60 tx=0x7f9b1c009040 comp rx=0 tx=0).stop 2024-04-14T08:41:17.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.139+0000 7f9b39715640 1 -- 172.21.15.172:0/2635420598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3406d040 msgr2=0x7f9b340c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.139+0000 7f9b39715640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3406d040 0x7f9b340c98b0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f9b2402bf70 tx=0x7f9b240046c0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.139+0000 7f9b39715640 1 -- 172.21.15.172:0/2635420598 shutdown_connections 2024-04-14T08:41:17.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.139+0000 7f9b39715640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9b1406c8f0 0x7f9b1406edb0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.139+0000 7f9b39715640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b3415e820 0x7f9b341e5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.139+0000 7f9b39715640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b340d0d30 0x7f9b3414a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.139+0000 7f9b39715640 1 --2- 172.21.15.172:0/2635420598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3406d040 0x7f9b340c98b0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.139+0000 7f9b39715640 1 -- 172.21.15.172:0/2635420598 >> 172.21.15.172:0/2635420598 conn(0x7f9b340b3b00 msgr2=0x7f9b3415bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.139+0000 7f9b39715640 1 -- 172.21.15.172:0/2635420598 shutdown_connections 2024-04-14T08:41:17.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.139+0000 7f9b39715640 1 -- 172.21.15.172:0/2635420598 wait complete. 2024-04-14T08:41:17.152 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:41:17.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:17.152 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:41:17.199 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:41:17.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:17.199 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:41:17.225 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:41:17.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.366+0000 7ff399c95640 1 Processor -- start 2024-04-14T08:41:17.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.366+0000 7ff399c95640 1 -- start start 2024-04-14T08:41:17.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff399c95640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff39415e5b0 0x7ff3940ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff399c95640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39406c530 0x7ff39406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff399c95640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3940b41d0 con 0x7ff3940729b0 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff399c95640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff39406cf00 con 0x7ff39415e5b0 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff399c95640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff3940aebe0 con 0x7ff39406c530 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff3937fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff39415e5b0 0x7ff3940ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff3937fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff39415e5b0 0x7ff3940ae390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45086/0 (socket says 172.21.15.172:45086) 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff3937fe640 1 -- 172.21.15.172:0/1591176920 learned_addr learned my addr 172.21.15.172:0/1591176920 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff393fff640 1 --2- 172.21.15.172:0/1591176920 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39406c530 0x7ff39406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff3937fe640 1 -- 172.21.15.172:0/1591176920 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39406c530 msgr2=0x7ff39406c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff3937fe640 1 --2- 172.21.15.172:0/1591176920 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39406c530 0x7ff39406c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff3937fe640 1 -- 172.21.15.172:0/1591176920 >> v1:172.21.15.177:6789/0 conn(0x7ff3940729b0 legacy=0x7ff39406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff3927fc640 1 -- 172.21.15.172:0/1591176920 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1751018171 0 0) 0x7ff3940b41d0 con 0x7ff3940729b0 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.367+0000 7ff3937fe640 1 -- 172.21.15.172:0/1591176920 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3940b4470 con 0x7ff39415e5b0 2024-04-14T08:41:17.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.368+0000 7ff3937fe640 1 --2- 172.21.15.172:0/1591176920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff39415e5b0 0x7ff3940ae390 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7ff3880098a0 tx=0x7ff38802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2772f0ae3863480f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.368+0000 7ff3927fc640 1 -- 172.21.15.172:0/1591176920 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388034070 con 0x7ff39415e5b0 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.368+0000 7ff3927fc640 1 -- 172.21.15.172:0/1591176920 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff388030840 con 0x7ff39415e5b0 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.368+0000 7ff3927fc640 1 -- 172.21.15.172:0/1591176920 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388030c10 con 0x7ff39415e5b0 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.368+0000 7ff399c95640 1 -- 172.21.15.172:0/1591176920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff39415e5b0 msgr2=0x7ff3940ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.368+0000 7ff399c95640 1 --2- 172.21.15.172:0/1591176920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff39415e5b0 0x7ff3940ae390 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7ff3880098a0 tx=0x7ff38802b980 comp rx=0 tx=0).stop 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.369+0000 7ff399c95640 1 -- 172.21.15.172:0/1591176920 shutdown_connections 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.369+0000 7ff399c95640 1 --2- 172.21.15.172:0/1591176920 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39406c530 0x7ff39406c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.369+0000 7ff399c95640 1 --2- 172.21.15.172:0/1591176920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff39415e5b0 0x7ff3940ae390 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.369+0000 7ff399c95640 1 -- 172.21.15.172:0/1591176920 >> 172.21.15.172:0/1591176920 conn(0x7ff394071ba0 msgr2=0x7ff394071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.369+0000 7ff399c95640 1 -- 172.21.15.172:0/1591176920 shutdown_connections 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.369+0000 7ff399c95640 1 -- 172.21.15.172:0/1591176920 wait complete. 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.370+0000 7ff399c95640 1 Processor -- start 2024-04-14T08:41:17.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.370+0000 7ff399c95640 1 -- start start 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff399c95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff39406c530 0x7ff394160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff399c95640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3940729b0 0x7ff3940ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff399c95640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39415e5b0 0x7ff3940cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff399c95640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff3940aeec0 con 0x7ff39406c530 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff399c95640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff39406e330 con 0x7ff3940729b0 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff399c95640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff3940af040 con 0x7ff39415e5b0 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff39406c530 0x7ff394160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff39406c530 0x7ff394160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45092/0 (socket says 172.21.15.172:45092) 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff3937fe640 1 -- 172.21.15.172:0/3805918013 learned_addr learned my addr 172.21.15.172:0/3805918013 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff393fff640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39415e5b0 0x7ff3940cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff392ffd640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3940729b0 0x7ff3940ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff3937fe640 1 -- 172.21.15.172:0/3805918013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39415e5b0 msgr2=0x7ff3940cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff3937fe640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39415e5b0 0x7ff3940cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff3937fe640 1 -- 172.21.15.172:0/3805918013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3940729b0 msgr2=0x7ff3940ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff3937fe640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3940729b0 0x7ff3940ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff3937fe640 1 -- 172.21.15.172:0/3805918013 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3940cd560 con 0x7ff39406c530 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff3937fe640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff39406c530 0x7ff394160ce0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff388000c00 tx=0x7ff3880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff37bfff640 1 -- 172.21.15.172:0/3805918013 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388034040 con 0x7ff39406c530 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff37bfff640 1 -- 172.21.15.172:0/3805918013 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff388039d90 con 0x7ff39406c530 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff399c95640 1 -- 172.21.15.172:0/3805918013 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3940cd8a0 con 0x7ff39406c530 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff37bfff640 1 -- 172.21.15.172:0/3805918013 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388043630 con 0x7ff39406c530 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.371+0000 7ff399c95640 1 -- 172.21.15.172:0/3805918013 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3940ce0e0 con 0x7ff39406c530 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.372+0000 7ff399c95640 1 -- 172.21.15.172:0/3805918013 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3940cda80 con 0x7ff39406c530 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.373+0000 7ff37bfff640 1 -- 172.21.15.172:0/3805918013 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff38804a060 con 0x7ff39406c530 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.373+0000 7ff37bfff640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff38006c8f0 0x7ff38006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.373+0000 7ff37bfff640 1 -- 172.21.15.172:0/3805918013 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7ff3880adb40 con 0x7ff39406c530 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.373+0000 7ff392ffd640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff38006c8f0 0x7ff38006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.374+0000 7ff392ffd640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff38006c8f0 0x7ff38006edb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff3940b16c0 tx=0x7ff384009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.378+0000 7ff37bfff640 1 -- 172.21.15.172:0/3805918013 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff3940cd8a0 con 0x7ff39406c530 2024-04-14T08:41:17.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.588+0000 7ff399c95640 1 -- 172.21.15.172:0/3805918013 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff394068520 con 0x7ff39406c530 2024-04-14T08:41:17.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.588+0000 7ff37bfff640 1 -- 172.21.15.172:0/3805918013 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5195 (secure 0 0 0) 0x7ff38803e070 con 0x7ff39406c530 2024-04-14T08:41:17.590 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:17.591 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8382,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/4028911489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4028911489},{"type":"v1","addr":"172.21.15.172:6837","nonce":4028911489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:41:04.492773+0000","modified":"2024-04-14T08:41:06.492980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/2602337717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2602337717},{"type":"v1","addr":"172.21.15.177:6835","nonce":2602337717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8394,"qdb_cluster":[8394]},"id":7}]} 2024-04-14T08:41:17.591 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 74 2024-04-14T08:41:17.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.590+0000 7ff399c95640 1 -- 172.21.15.172:0/3805918013 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff38006c8f0 msgr2=0x7ff38006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.590+0000 7ff399c95640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff38006c8f0 0x7ff38006edb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff3940b16c0 tx=0x7ff384009040 comp rx=0 tx=0).stop 2024-04-14T08:41:17.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.590+0000 7ff399c95640 1 -- 172.21.15.172:0/3805918013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff39406c530 msgr2=0x7ff394160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.590+0000 7ff399c95640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff39406c530 0x7ff394160ce0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff388000c00 tx=0x7ff3880046c0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.590+0000 7ff399c95640 1 -- 172.21.15.172:0/3805918013 shutdown_connections 2024-04-14T08:41:17.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.590+0000 7ff399c95640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff38006c8f0 0x7ff38006edb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.590+0000 7ff399c95640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39415e5b0 0x7ff3940cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.590+0000 7ff399c95640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3940729b0 0x7ff3940ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.590+0000 7ff399c95640 1 --2- 172.21.15.172:0/3805918013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff39406c530 0x7ff394160ce0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.590+0000 7ff399c95640 1 -- 172.21.15.172:0/3805918013 >> 172.21.15.172:0/3805918013 conn(0x7ff394071ba0 msgr2=0x7ff39415c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:17.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.591+0000 7ff399c95640 1 -- 172.21.15.172:0/3805918013 shutdown_connections 2024-04-14T08:41:17.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.591+0000 7ff399c95640 1 -- 172.21.15.172:0/3805918013 wait complete. 2024-04-14T08:41:17.603 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-14T08:41:17.730 INFO:teuthology.orchestra.run.smithi177.stdout:{"message":"","return_code":0} 2024-04-14T08:41:17.730 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-14T08:41:17.730 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:41:17.730 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:41:17.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:41:17.746 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:41:17.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:17.746 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:41:17.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:41:17.801 INFO:teuthology.orchestra.run.smithi172.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-14T08:41:17.801 DEBUG:teuthology.orchestra.run.smithi172:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-14T08:41:17.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:17.844 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-14T08:41:17.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.949+0000 7f98dfe36640 1 Processor -- start 2024-04-14T08:41:17.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.949+0000 7f98dfe36640 1 -- start start 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98dfe36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f98d80b3ba0 0x7f98d80d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98dfe36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98d80b3110 0x7f98d80d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98dfe36640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d80d1c20 con 0x7f98d8161c00 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98dfe36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f98d80cf1a0 con 0x7f98d80b3ba0 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98dfe36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f98d80d1380 con 0x7f98d80b3110 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98ddbd2640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f98d8161c00 0x7f98d80d0ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:60472/0 (socket says 172.21.15.172:60472) 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98ddbd2640 1 -- 172.21.15.172:0/3200105892 learned_addr learned my addr 172.21.15.172:0/3200105892 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98dd3d1640 1 --2- 172.21.15.172:0/3200105892 >> v2:172.21.15.177:3300/0 conn(0x7f98d80b3ba0 0x7f98d80d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98de3d3640 1 --2- 172.21.15.172:0/3200105892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98d80b3110 0x7f98d80d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98dcbd0640 1 -- 172.21.15.172:0/3200105892 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2878737285 0 0) 0x7f98d80d1c20 con 0x7f98d8161c00 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98dcbd0640 1 -- 172.21.15.172:0/3200105892 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98c4003520 con 0x7f98d8161c00 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98de3d3640 1 -- 172.21.15.172:0/3200105892 >> v2:172.21.15.177:3300/0 conn(0x7f98d80b3ba0 msgr2=0x7f98d80d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98de3d3640 1 --2- 172.21.15.172:0/3200105892 >> v2:172.21.15.177:3300/0 conn(0x7f98d80b3ba0 0x7f98d80d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98de3d3640 1 -- 172.21.15.172:0/3200105892 >> v1:172.21.15.172:6789/0 conn(0x7f98d8161c00 legacy=0x7f98d80d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98de3d3640 1 -- 172.21.15.172:0/3200105892 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d80d1660 con 0x7f98d80b3110 2024-04-14T08:41:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.950+0000 7f98dd3d1640 1 --2- 172.21.15.172:0/3200105892 >> v2:172.21.15.177:3300/0 conn(0x7f98d80b3ba0 0x7f98d80d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.951+0000 7f98de3d3640 1 --2- 172.21.15.172:0/3200105892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98d80b3110 0x7f98d80d2b20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009d90 tx=0x7f98d002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=641c0d424ae64a02 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.951+0000 7f98dcbd0640 1 -- 172.21.15.172:0/3200105892 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0039070 con 0x7f98d80b3110 2024-04-14T08:41:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.951+0000 7f98dcbd0640 1 -- 172.21.15.172:0/3200105892 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d002cc50 con 0x7f98d80b3110 2024-04-14T08:41:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.951+0000 7f98dcbd0640 1 -- 172.21.15.172:0/3200105892 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0034440 con 0x7f98d80b3110 2024-04-14T08:41:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.952+0000 7f98dfe36640 1 -- 172.21.15.172:0/3200105892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98d80b3110 msgr2=0x7f98d80d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.952+0000 7f98dfe36640 1 --2- 172.21.15.172:0/3200105892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98d80b3110 0x7f98d80d2b20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009d90 tx=0x7f98d002bea0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.952+0000 7f98dfe36640 1 -- 172.21.15.172:0/3200105892 shutdown_connections 2024-04-14T08:41:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.952+0000 7f98dfe36640 1 --2- 172.21.15.172:0/3200105892 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98d80b3110 0x7f98d80d2b20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.952+0000 7f98dfe36640 1 --2- 172.21.15.172:0/3200105892 >> v2:172.21.15.177:3300/0 conn(0x7f98d80b3ba0 0x7f98d80d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.952+0000 7f98dfe36640 1 -- 172.21.15.172:0/3200105892 >> 172.21.15.172:0/3200105892 conn(0x7f98d809d570 msgr2=0x7f98d80d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.952+0000 7f98dfe36640 1 -- 172.21.15.172:0/3200105892 shutdown_connections 2024-04-14T08:41:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.952+0000 7f98dfe36640 1 -- 172.21.15.172:0/3200105892 wait complete. 2024-04-14T08:41:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.953+0000 7f98dfe36640 1 Processor -- start 2024-04-14T08:41:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.953+0000 7f98dfe36640 1 -- start start 2024-04-14T08:41:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98dfe36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98d80b3110 0x7f98d81f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98dfe36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98d80b3ba0 0x7f98d80b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98dfe36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98d8161c00 0x7f98d80b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98dfe36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98d80cf630 con 0x7f98d8161c00 2024-04-14T08:41:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98dfe36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98d8161fe0 con 0x7f98d80b3ba0 2024-04-14T08:41:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98dfe36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f98d80cf7b0 con 0x7f98d80b3110 2024-04-14T08:41:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98de3d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98d8161c00 0x7f98d80b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98de3d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98d8161c00 0x7f98d80b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45094/0 (socket says 172.21.15.172:45094) 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98de3d3640 1 -- 172.21.15.172:0/385630484 learned_addr learned my addr 172.21.15.172:0/385630484 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98ddbd2640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98d80b3110 0x7f98d81f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98dd3d1640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98d80b3ba0 0x7f98d80b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98de3d3640 1 -- 172.21.15.172:0/385630484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98d80b3110 msgr2=0x7f98d81f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98de3d3640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98d80b3110 0x7f98d81f7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98de3d3640 1 -- 172.21.15.172:0/385630484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98d80b3ba0 msgr2=0x7f98d80b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98de3d3640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98d80b3ba0 0x7f98d80b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.954+0000 7f98de3d3640 1 -- 172.21.15.172:0/385630484 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d81f4080 con 0x7f98d8161c00 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.955+0000 7f98de3d3640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98d8161c00 0x7f98d80b4750 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009d90 tx=0x7f98d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.955+0000 7f98ce7fc640 1 -- 172.21.15.172:0/385630484 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0039050 con 0x7f98d8161c00 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.955+0000 7f98ce7fc640 1 -- 172.21.15.172:0/385630484 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d0045070 con 0x7f98d8161c00 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.955+0000 7f98dfe36640 1 -- 172.21.15.172:0/385630484 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d8152400 con 0x7f98d8161c00 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.955+0000 7f98ce7fc640 1 -- 172.21.15.172:0/385630484 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0009450 con 0x7f98d8161c00 2024-04-14T08:41:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.955+0000 7f98dfe36640 1 -- 172.21.15.172:0/385630484 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d81f7e20 con 0x7f98d8161c00 2024-04-14T08:41:17.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.957+0000 7f98ce7fc640 1 -- 172.21.15.172:0/385630484 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98d003e610 con 0x7f98d8161c00 2024-04-14T08:41:17.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.957+0000 7f98ce7fc640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f98bc06c8f0 0x7f98bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.957+0000 7f98ce7fc640 1 -- 172.21.15.172:0/385630484 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f98d00ae5b0 con 0x7f98d8161c00 2024-04-14T08:41:17.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.957+0000 7f98ddbd2640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f98bc06c8f0 0x7f98bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.958+0000 7f98ddbd2640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f98bc06c8f0 0x7f98bc06edb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005880 tx=0x7f98d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.958+0000 7f98e2c3b740 1 Processor -- start 2024-04-14T08:41:17.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.958+0000 7f98e2c3b740 1 -- start start 2024-04-14T08:41:17.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98e2c3b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b478d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98e2c3b740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b475d300 0x5614b478a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98e2c3b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b4760e60 0x5614b478aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98e2c3b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5614b46c7740 con 0x5614b478d2e0 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98e2c3b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5614b466f930 con 0x5614b4760e60 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98e2c3b740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5614b46c3890 con 0x5614b475d300 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98ddbd2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b478d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98ddbd2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b478d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45110/0 (socket says 172.21.15.172:45110) 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98ddbd2640 1 -- 172.21.15.172:0/732442393 learned_addr learned my addr 172.21.15.172:0/732442393 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98de3d3640 1 --2- 172.21.15.172:0/732442393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b4760e60 0x5614b478aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98dd3d1640 1 --2- 172.21.15.172:0/732442393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b475d300 0x5614b478a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98de3d3640 1 -- 172.21.15.172:0/732442393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b475d300 msgr2=0x5614b478a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98de3d3640 1 --2- 172.21.15.172:0/732442393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b475d300 0x5614b478a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98de3d3640 1 -- 172.21.15.172:0/732442393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 msgr2=0x5614b478d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98de3d3640 1 --2- 172.21.15.172:0/732442393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b478d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98de3d3640 1 -- 172.21.15.172:0/732442393 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b44a37e0 con 0x5614b4760e60 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98ddbd2640 1 --2- 172.21.15.172:0/732442393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b478d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.959+0000 7f98de3d3640 1 --2- 172.21.15.172:0/732442393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b4760e60 0x5614b478aa90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f98d00ac9b0 tx=0x7f98d007c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f4ff5e937f4563d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.960+0000 7f98bb7fe640 1 -- 172.21.15.172:0/732442393 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0084e00 con 0x5614b4760e60 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.960+0000 7f98bb7fe640 1 -- 172.21.15.172:0/732442393 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d003a040 con 0x5614b4760e60 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.960+0000 7f98bb7fe640 1 -- 172.21.15.172:0/732442393 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0078940 con 0x5614b4760e60 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.960+0000 7f98e2c3b740 1 -- 172.21.15.172:0/732442393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b4760e60 msgr2=0x5614b478aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.960+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/732442393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b4760e60 0x5614b478aa90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f98d00ac9b0 tx=0x7f98d007c2d0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.960+0000 7f98e2c3b740 1 -- 172.21.15.172:0/732442393 shutdown_connections 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.960+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/732442393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b4760e60 0x5614b478aa90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.960+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/732442393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b475d300 0x5614b478a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.960+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/732442393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b478d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.960+0000 7f98e2c3b740 1 -- 172.21.15.172:0/732442393 >> 172.21.15.172:0/732442393 conn(0x5614b475f570 msgr2=0x5614b47a80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.960+0000 7f98e2c3b740 1 -- 172.21.15.172:0/732442393 shutdown_connections 2024-04-14T08:41:17.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.960+0000 7f98e2c3b740 1 -- 172.21.15.172:0/732442393 wait complete. 2024-04-14T08:41:17.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.961+0000 7f98e2c3b740 1 -- start start 2024-04-14T08:41:17.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.961+0000 7f98e2c3b740 1 Processor -- start 2024-04-14T08:41:17.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98e2c3b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b4760e60 0x5614b4782a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98e2c3b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b4786f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98e2c3b740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b47a5ca0 0x5614b47874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98e2c3b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5614b466f930 con 0x5614b478d2e0 2024-04-14T08:41:17.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98e2c3b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5614b4682cf0 con 0x5614b4760e60 2024-04-14T08:41:17.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98e2c3b740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5614b4762010 con 0x5614b47a5ca0 2024-04-14T08:41:17.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98e2c3b740 10 client.0 _ll_register_callbacks cb 0x5614b464f9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:17.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98dd3d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b4786f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98ddbd2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b4760e60 0x5614b4782a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98dd3d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b4786f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45112/0 (socket says 172.21.15.172:45112) 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98dd3d1640 1 -- 172.21.15.172:0/2330557024 learned_addr learned my addr 172.21.15.172:0/2330557024 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98de3d3640 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b47a5ca0 0x5614b47874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98ddbd2640 1 -- 172.21.15.172:0/2330557024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b47a5ca0 msgr2=0x5614b47874d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98ddbd2640 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b47a5ca0 0x5614b47874d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98ddbd2640 1 -- 172.21.15.172:0/2330557024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 msgr2=0x5614b4786f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98ddbd2640 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b4786f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98ddbd2640 1 -- 172.21.15.172:0/2330557024 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b4503f70 con 0x5614b4760e60 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.962+0000 7f98dd3d1640 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b4786f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98ddbd2640 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b4760e60 0x5614b4782a20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f98d4007ed0 tx=0x7f98d4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91d29d7d6d7b1fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98b8ff9640 1 -- 172.21.15.172:0/2330557024 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4013680 con 0x5614b4760e60 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98e2c3b740 1 -- 172.21.15.172:0/2330557024 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d003a040 con 0x5614b4760e60 2024-04-14T08:41:17.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98b8ff9640 1 -- 172.21.15.172:0/2330557024 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f98d4010040 con 0x5614b4760e60 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98b8ff9640 1 -- 172.21.15.172:0/2330557024 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f98d402e050 con 0x5614b4760e60 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98e2c3b740 10 client.7948 fetch_fsmap learned FSMap version 74 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98e2c3b740 1 -- 172.21.15.172:0/2330557024 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5614b439b170 con 0x5614b4760e60 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98b8ff9640 1 -- 172.21.15.172:0/2330557024 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d003a040 con 0x5614b4760e60 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98b8ff9640 1 -- 172.21.15.172:0/2330557024 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f98d4031020 con 0x5614b4760e60 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98e2c3b740 10 client.7948 fetch_fsmap finished waiting for FSMap version 74 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98e2c3b740 10 client.7948 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98e2c3b740 20 client.7948 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x5614b47bc220 0x5614b47bc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98e2c3b740 4 client.7948 mds_command: new command op to 8394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98e2c3b740 1 -- 172.21.15.172:0/2330557024 --> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614b46d4c80 con 0x5614b47bc220 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.963+0000 7f98dd3d1640 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x5614b47bc220 0x5614b47bc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.964+0000 7f98dd3d1640 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x5614b47bc220 0x5614b47bc600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f00725a18a564d87 server_cookie=f85fbeadca43f934 in_seq=0 out_seq=0 2024-04-14T08:41:17.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.964+0000 7f98b8ff9640 10 client.7948 ms_handle_connect on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.964+0000 7f98b8ff9640 1 -- 172.21.15.172:0/2330557024 <== mds.0 v2:172.21.15.177:6834/2602337717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614b46d4c80 con 0x5614b47bc220 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.964+0000 7f98b8ff9640 10 client.7948 handle_command_reply: tid=0 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.964+0000 7f98e2c3b740 1 client.7948 shutdown 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.965+0000 7f98e2c3b740 1 -- 172.21.15.172:0/2330557024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b4760e60 msgr2=0x5614b4782a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.965+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b4760e60 0x5614b4782a20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f98d4007ed0 tx=0x7f98d4000bc0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.965+0000 7f98e2c3b740 1 -- 172.21.15.172:0/2330557024 shutdown_connections 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.965+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x5614b47bc220 0x5614b47bc600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.965+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b47a5ca0 0x5614b47874d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.965+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b4786f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.965+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/2330557024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b4760e60 0x5614b4782a20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.965+0000 7f98b8ff9640 0 client.7948 ms_handle_reset on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.965+0000 7f98e2c3b740 1 -- 172.21.15.172:0/2330557024 >> 172.21.15.172:0/2330557024 conn(0x5614b475d300 msgr2=0x5614b47ab600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.965+0000 7f98e2c3b740 1 -- 172.21.15.172:0/2330557024 shutdown_connections 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.965+0000 7f98e2c3b740 1 -- 172.21.15.172:0/2330557024 wait complete. 2024-04-14T08:41:17.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.965+0000 7f98e2c3b740 20 client.7948 trim_cache size 0 max 16384 2024-04-14T08:41:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.968+0000 7effc4588640 1 Processor -- start 2024-04-14T08:41:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.968+0000 7effc4588640 1 -- start start 2024-04-14T08:41:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc4588640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effbc0b3330 0x7effbc0cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc4588640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7effbc0b3710 0x7effbc0d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc4588640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7effbc0d0c20 0x7effbc0d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc4588640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7effbc0cf410 con 0x7effbc0b3710 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc4588640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7effbc0cf6f0 con 0x7effbc0d0c20 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc4588640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7effbc0cf9d0 con 0x7effbc0b3330 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc1b23640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7effbc0b3710 0x7effbc0d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc1b23640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7effbc0b3710 0x7effbc0d2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45116/0 (socket says 172.21.15.172:45116) 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc1b23640 1 -- 172.21.15.172:0/1655482626 learned_addr learned my addr 172.21.15.172:0/1655482626 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc2b25640 1 --2- 172.21.15.172:0/1655482626 >> v2:172.21.15.177:3300/0 conn(0x7effbc0d0c20 0x7effbc0d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc2324640 1 --2- 172.21.15.172:0/1655482626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effbc0b3330 0x7effbc0cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc2b25640 1 -- 172.21.15.172:0/1655482626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effbc0b3330 msgr2=0x7effbc0cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc2b25640 1 --2- 172.21.15.172:0/1655482626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effbc0b3330 0x7effbc0cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc2b25640 1 -- 172.21.15.172:0/1655482626 >> v2:172.21.15.172:3300/0 conn(0x7effbc0b3710 msgr2=0x7effbc0d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc2b25640 1 --2- 172.21.15.172:0/1655482626 >> v2:172.21.15.172:3300/0 conn(0x7effbc0b3710 0x7effbc0d2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc2b25640 1 -- 172.21.15.172:0/1655482626 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc0cfcb0 con 0x7effbc0d0c20 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.969+0000 7effc1b23640 1 --2- 172.21.15.172:0/1655482626 >> v2:172.21.15.172:3300/0 conn(0x7effbc0b3710 0x7effbc0d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.970+0000 7effc2b25640 1 --2- 172.21.15.172:0/1655482626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effbc0d0c20 0x7effbc0d10b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7effbc06bf70 tx=0x7effb802bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=884f3ac2f1fb1cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.970+0000 7effc1322640 1 -- 172.21.15.172:0/1655482626 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8039070 con 0x7effbc0d0c20 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.970+0000 7effc1322640 1 -- 172.21.15.172:0/1655482626 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb802cb40 con 0x7effbc0d0c20 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.970+0000 7effc1322640 1 -- 172.21.15.172:0/1655482626 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8034440 con 0x7effbc0d0c20 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.971+0000 7effc4588640 1 -- 172.21.15.172:0/1655482626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effbc0d0c20 msgr2=0x7effbc0d10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.971+0000 7effc4588640 1 --2- 172.21.15.172:0/1655482626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effbc0d0c20 0x7effbc0d10b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7effbc06bf70 tx=0x7effb802bd90 comp rx=0 tx=0).stop 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.971+0000 7effc4588640 1 -- 172.21.15.172:0/1655482626 shutdown_connections 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.971+0000 7effc4588640 1 --2- 172.21.15.172:0/1655482626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effbc0d0c20 0x7effbc0d10b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.971+0000 7effc4588640 1 --2- 172.21.15.172:0/1655482626 >> v2:172.21.15.172:3300/0 conn(0x7effbc0b3710 0x7effbc0d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.971+0000 7effc4588640 1 --2- 172.21.15.172:0/1655482626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effbc0b3330 0x7effbc0cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.971+0000 7effc4588640 1 -- 172.21.15.172:0/1655482626 >> 172.21.15.172:0/1655482626 conn(0x7effbc09d580 msgr2=0x7effbc0d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.971+0000 7effc4588640 1 -- 172.21.15.172:0/1655482626 shutdown_connections 2024-04-14T08:41:17.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.971+0000 7effc4588640 1 -- 172.21.15.172:0/1655482626 wait complete. 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.972+0000 7effc4588640 1 Processor -- start 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.972+0000 7effc4588640 1 -- start start 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.973+0000 7effc4588640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effbc0b3330 0x7effbc0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.973+0000 7effc4588640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effbc0b3710 0x7effbc15b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.973+0000 7effc4588640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effbc0d0c20 0x7effbc15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.973+0000 7effc4588640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0d3100 con 0x7effbc0d0c20 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.973+0000 7effc4588640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0cff90 con 0x7effbc0b3330 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.973+0000 7effc4588640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7effbc0d3280 con 0x7effbc0b3710 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.973+0000 7effc2b25640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effbc0d0c20 0x7effbc15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.973+0000 7effc2324640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effbc0b3330 0x7effbc0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.973+0000 7effc2b25640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effbc0d0c20 0x7effbc15cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45118/0 (socket says 172.21.15.172:45118) 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.973+0000 7effc1b23640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effbc0b3710 0x7effbc15b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.973+0000 7effc2b25640 1 -- 172.21.15.172:0/1541404582 learned_addr learned my addr 172.21.15.172:0/1541404582 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effc2324640 1 -- 172.21.15.172:0/1541404582 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effbc0b3710 msgr2=0x7effbc15b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effc2324640 1 --2- 172.21.15.172:0/1541404582 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effbc0b3710 0x7effbc15b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effc2324640 1 -- 172.21.15.172:0/1541404582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effbc0d0c20 msgr2=0x7effbc15cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effc2324640 1 --2- 172.21.15.172:0/1541404582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effbc0d0c20 0x7effbc15cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effc2324640 1 -- 172.21.15.172:0/1541404582 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc15d250 con 0x7effbc0b3330 2024-04-14T08:41:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effc1b23640 1 --2- 172.21.15.172:0/1541404582 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effbc0b3710 0x7effbc15b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effc2b25640 1 --2- 172.21.15.172:0/1541404582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effbc0d0c20 0x7effbc15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effc2324640 1 --2- 172.21.15.172:0/1541404582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effbc0b3330 0x7effbc0cdec0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7effb400e870 tx=0x7effb400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effb2ffd640 1 -- 172.21.15.172:0/1541404582 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4017070 con 0x7effbc0b3330 2024-04-14T08:41:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effb2ffd640 1 -- 172.21.15.172:0/1541404582 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb400c940 con 0x7effbc0b3330 2024-04-14T08:41:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effc4588640 1 -- 172.21.15.172:0/1541404582 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc15e170 con 0x7effbc0b3330 2024-04-14T08:41:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effb2ffd640 1 -- 172.21.15.172:0/1541404582 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb400ccf0 con 0x7effbc0b3330 2024-04-14T08:41:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.974+0000 7effc4588640 1 -- 172.21.15.172:0/1541404582 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc15c510 con 0x7effbc0b3330 2024-04-14T08:41:17.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.975+0000 7effc4588640 1 -- 172.21.15.172:0/1541404582 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7effbc068590 con 0x7effbc0b3330 2024-04-14T08:41:17.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.976+0000 7effb2ffd640 1 -- 172.21.15.172:0/1541404582 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effb401a070 con 0x7effbc0b3330 2024-04-14T08:41:17.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.976+0000 7effb2ffd640 1 --2- 172.21.15.172:0/1541404582 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effa006c920 0x7effa006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.976+0000 7effb2ffd640 1 -- 172.21.15.172:0/1541404582 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7effb408bf10 con 0x7effbc0b3330 2024-04-14T08:41:17.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.976+0000 7effc1b23640 1 --2- 172.21.15.172:0/1541404582 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effa006c920 0x7effa006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.976+0000 7effb2ffd640 1 -- 172.21.15.172:0/1541404582 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+511 (secure 0 0 0) 0x7effb408c3c0 con 0x7effbc0b3330 2024-04-14T08:41:17.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.976+0000 7effc4588640 1 -- 172.21.15.172:0/1541404582 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7effbc0d4950 con 0x7effbc0b3330 2024-04-14T08:41:17.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.976+0000 7effc1b23640 1 --2- 172.21.15.172:0/1541404582 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7effa006c920 0x7effa006ede0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7effbc0b1150 tx=0x7effac009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.984+0000 7f98e2c3b740 1 Processor -- start 2024-04-14T08:41:17.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.984+0000 7f98e2c3b740 1 -- start start 2024-04-14T08:41:17.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98e2c3b740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b4760e60 0x5614b478a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98e2c3b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b478d2e0 0x5614b4781bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98e2c3b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b47a5ca0 0x5614b47ba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98e2c3b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98d8200fb0 con 0x5614b47a5ca0 2024-04-14T08:41:17.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98e2c3b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98d81fffe0 con 0x5614b478d2e0 2024-04-14T08:41:17.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98e2c3b740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f98d80c87c0 con 0x5614b4760e60 2024-04-14T08:41:17.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98de3d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b47a5ca0 0x5614b47ba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98de3d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b47a5ca0 0x5614b47ba940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45134/0 (socket says 172.21.15.172:45134) 2024-04-14T08:41:17.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98de3d3640 1 -- 172.21.15.172:0/1603315211 learned_addr learned my addr 172.21.15.172:0/1603315211 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:17.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98dd3d1640 1 --2- 172.21.15.172:0/1603315211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b478d2e0 0x5614b4781bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98ddbd2640 1 --2- 172.21.15.172:0/1603315211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b4760e60 0x5614b478a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98de3d3640 1 -- 172.21.15.172:0/1603315211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b4760e60 msgr2=0x5614b478a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98de3d3640 1 --2- 172.21.15.172:0/1603315211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b4760e60 0x5614b478a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98de3d3640 1 -- 172.21.15.172:0/1603315211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b478d2e0 msgr2=0x5614b4781bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98de3d3640 1 --2- 172.21.15.172:0/1603315211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b478d2e0 0x5614b4781bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98de3d3640 1 -- 172.21.15.172:0/1603315211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b464d950 con 0x5614b47a5ca0 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98dd3d1640 1 --2- 172.21.15.172:0/1603315211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b478d2e0 0x5614b4781bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98de3d3640 1 --2- 172.21.15.172:0/1603315211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b47a5ca0 0x5614b47ba940 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f98d0080790 tx=0x7f98d004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1724503245c3541 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98baffd640 1 -- 172.21.15.172:0/1603315211 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0084e00 con 0x5614b47a5ca0 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98baffd640 1 -- 172.21.15.172:0/1603315211 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d007a5d0 con 0x5614b47a5ca0 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.985+0000 7f98baffd640 1 -- 172.21.15.172:0/1603315211 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0084970 con 0x5614b47a5ca0 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.986+0000 7f98e2c3b740 1 -- 172.21.15.172:0/1603315211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b47a5ca0 msgr2=0x5614b47ba940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.986+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/1603315211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b47a5ca0 0x5614b47ba940 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f98d0080790 tx=0x7f98d004c000 comp rx=0 tx=0).stop 2024-04-14T08:41:17.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.986+0000 7f98e2c3b740 1 -- 172.21.15.172:0/1603315211 shutdown_connections 2024-04-14T08:41:17.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.986+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/1603315211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b47a5ca0 0x5614b47ba940 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.986+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/1603315211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b478d2e0 0x5614b4781bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.986+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/1603315211 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b4760e60 0x5614b478a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.986+0000 7f98e2c3b740 1 -- 172.21.15.172:0/1603315211 >> 172.21.15.172:0/1603315211 conn(0x5614b475d300 msgr2=0x5614b4787db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:17.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.986+0000 7f98e2c3b740 1 -- 172.21.15.172:0/1603315211 shutdown_connections 2024-04-14T08:41:17.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.986+0000 7f98e2c3b740 1 -- 172.21.15.172:0/1603315211 wait complete. 2024-04-14T08:41:17.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.987+0000 7f98e2c3b740 1 -- start start 2024-04-14T08:41:17.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.987+0000 7f98e2c3b740 1 Processor -- start 2024-04-14T08:41:17.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.987+0000 7f98e2c3b740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b4760e60 0x5614b481be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.987+0000 7f98e2c3b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b4762970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.987+0000 7f98e2c3b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b47a5ca0 0x5614b4823040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.987+0000 7f98e2c3b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5614b46d4c80 con 0x5614b478d2e0 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.987+0000 7f98e2c3b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98d81ff990 con 0x5614b47a5ca0 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.987+0000 7f98e2c3b740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5614b4761530 con 0x5614b4760e60 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98e2c3b740 10 client.0 _ll_register_callbacks cb 0x7f98d40079d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98dd3d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b4762970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98dd3d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b4762970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45140/0 (socket says 172.21.15.172:45140) 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98dd3d1640 1 -- 172.21.15.172:0/20139482 learned_addr learned my addr 172.21.15.172:0/20139482 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98ddbd2640 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b4760e60 0x5614b481be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98de3d3640 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b47a5ca0 0x5614b4823040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98dd3d1640 1 -- 172.21.15.172:0/20139482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b4760e60 msgr2=0x5614b481be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98dd3d1640 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b4760e60 0x5614b481be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98dd3d1640 1 -- 172.21.15.172:0/20139482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b47a5ca0 msgr2=0x5614b4823040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98dd3d1640 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b47a5ca0 0x5614b4823040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98dd3d1640 1 -- 172.21.15.172:0/20139482 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b4761fd0 con 0x5614b478d2e0 2024-04-14T08:41:17.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98dd3d1640 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b4762970 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x5614b47bcc00 tx=0x7f98c8015960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761e0b4a84782470 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:17.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:41:17.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98b8ff9640 1 -- 172.21.15.172:0/20139482 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8022720 con 0x5614b478d2e0 2024-04-14T08:41:17.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98b8ff9640 1 -- 172.21.15.172:0/20139482 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f98c80228c0 con 0x5614b478d2e0 2024-04-14T08:41:17.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98e2c3b740 1 -- 172.21.15.172:0/20139482 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d007a5d0 con 0x5614b478d2e0 2024-04-14T08:41:17.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98b8ff9640 1 -- 172.21.15.172:0/20139482 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f98c8014410 con 0x5614b478d2e0 2024-04-14T08:41:17.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98e2c3b740 10 client.8670 fetch_fsmap learned FSMap version 74 2024-04-14T08:41:17.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98e2c3b740 1 -- 172.21.15.172:0/20139482 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5614b4823be0 con 0x5614b478d2e0 2024-04-14T08:41:17.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.988+0000 7f98b8ff9640 1 -- 172.21.15.172:0/20139482 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d007a5d0 con 0x5614b478d2e0 2024-04-14T08:41:17.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.989+0000 7f98b8ff9640 1 -- 172.21.15.172:0/20139482 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f98c80145b0 con 0x5614b478d2e0 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.989+0000 7f98e2c3b740 10 client.8670 fetch_fsmap finished waiting for FSMap version 74 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.989+0000 7f98e2c3b740 10 client.8670 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.989+0000 7f98e2c3b740 20 client.8670 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.989+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x5614b4787260 0x5614b4823580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.989+0000 7f98e2c3b740 4 client.8670 mds_command: new command op to 8394 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.989+0000 7f98e2c3b740 1 -- 172.21.15.172:0/20139482 --> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x5614b475db00 con 0x5614b4787260 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.989+0000 7f98ddbd2640 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x5614b4787260 0x5614b4823580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.989+0000 7f98ddbd2640 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x5614b4787260 0x5614b4823580 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14947aed6dd367ea server_cookie=3b45f7766fdbebda in_seq=0 out_seq=0 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.989+0000 7f98b8ff9640 10 client.8670 ms_handle_connect on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98b8ff9640 1 -- 172.21.15.172:0/20139482 <== mds.0 v2:172.21.15.177:6834/2602337717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5614b475db00 con 0x5614b4787260 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98b8ff9640 10 client.8670 handle_command_reply: tid=0 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98e2c3b740 1 client.8670 shutdown 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98e2c3b740 1 -- 172.21.15.172:0/20139482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 msgr2=0x5614b4762970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:17.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b4762970 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x5614b47bcc00 tx=0x7f98c8015960 comp rx=0 tx=0).stop 2024-04-14T08:41:17.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98e2c3b740 1 -- 172.21.15.172:0/20139482 shutdown_connections 2024-04-14T08:41:17.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x5614b4787260 0x5614b4823580 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614b47a5ca0 0x5614b4823040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614b478d2e0 0x5614b4762970 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98e2c3b740 1 --2- 172.21.15.172:0/20139482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5614b4760e60 0x5614b481be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:17.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98e2c3b740 1 -- 172.21.15.172:0/20139482 >> 172.21.15.172:0/20139482 conn(0x5614b475d300 msgr2=0x5614b4787ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:17.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98b8ff9640 0 client.8670 ms_handle_reset on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:17.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98e2c3b740 1 -- 172.21.15.172:0/20139482 shutdown_connections 2024-04-14T08:41:17.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98e2c3b740 1 -- 172.21.15.172:0/20139482 wait complete. 2024-04-14T08:41:17.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98e2c3b740 20 client.8670 trim_cache size 0 max 16384 2024-04-14T08:41:17.999 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:41:18.000 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-04-14T08:41:18.000 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "331b9465-e788-45dc-be77-038daa4989eb", 2024-04-14T08:41:18.000 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-04-14T08:41:18.000 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:41:18.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.990+0000 7f98dfe36640 1 -- 172.21.15.172:0/385630484 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f98bc06c8f0 msgr2=0x7f98bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:18.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.991+0000 7f98dfe36640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f98bc06c8f0 0x7f98bc06edb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005880 tx=0x7f98d4009040 comp rx=0 tx=0).stop 2024-04-14T08:41:18.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.991+0000 7f98dfe36640 1 -- 172.21.15.172:0/385630484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98d8161c00 msgr2=0x7f98d80b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:18.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.991+0000 7f98dfe36640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98d8161c00 0x7f98d80b4750 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009d90 tx=0x7f98d0004a40 comp rx=0 tx=0).stop 2024-04-14T08:41:18.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.991+0000 7f98dfe36640 1 -- 172.21.15.172:0/385630484 shutdown_connections 2024-04-14T08:41:18.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.991+0000 7f98dfe36640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f98bc06c8f0 0x7f98bc06edb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.991+0000 7f98dfe36640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98d8161c00 0x7f98d80b4750 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.991+0000 7f98dfe36640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98d80b3ba0 0x7f98d80b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.991+0000 7f98dfe36640 1 --2- 172.21.15.172:0/385630484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98d80b3110 0x7f98d81f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.991+0000 7f98dfe36640 1 -- 172.21.15.172:0/385630484 >> 172.21.15.172:0/385630484 conn(0x7f98d809d570 msgr2=0x7f98d8162820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:18.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.991+0000 7f98dfe36640 1 -- 172.21.15.172:0/385630484 shutdown_connections 2024-04-14T08:41:18.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:17.991+0000 7f98dfe36640 1 -- 172.21.15.172:0/385630484 wait complete. 2024-04-14T08:41:18.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:18.003 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-14T08:41:18.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.104+0000 7f16db4d5640 1 Processor -- start 2024-04-14T08:41:18.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.104+0000 7f16db4d5640 1 -- start start 2024-04-14T08:41:18.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16db4d5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d4159c20 0x7f16d40ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16db4d5640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f16d415c700 0x7f16d41603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16db4d5640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d415cf10 con 0x7f16d4160930 2024-04-14T08:41:18.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16db4d5640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f16d415fc80 con 0x7f16d415c700 2024-04-14T08:41:18.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16db4d5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f16d415ff60 con 0x7f16d4159c20 2024-04-14T08:41:18.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16d9a72640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f16d4160930 0x7f16d4072450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:60474/0 (socket says 172.21.15.172:60474) 2024-04-14T08:41:18.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16d9a72640 1 -- 172.21.15.172:0/599732005 learned_addr learned my addr 172.21.15.172:0/599732005 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:18.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16d9271640 1 --2- 172.21.15.172:0/599732005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d4159c20 0x7f16d40ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16d8a70640 1 --2- 172.21.15.172:0/599732005 >> v2:172.21.15.177:3300/0 conn(0x7f16d415c700 0x7f16d41603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16cbfff640 1 -- 172.21.15.172:0/599732005 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 343359639 0 0) 0x7f16d415cf10 con 0x7f16d4160930 2024-04-14T08:41:18.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16cbfff640 1 -- 172.21.15.172:0/599732005 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16c0003520 con 0x7f16d4160930 2024-04-14T08:41:18.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16d9271640 1 -- 172.21.15.172:0/599732005 >> v2:172.21.15.177:3300/0 conn(0x7f16d415c700 msgr2=0x7f16d41603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:18.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16d9271640 1 --2- 172.21.15.172:0/599732005 >> v2:172.21.15.177:3300/0 conn(0x7f16d415c700 0x7f16d41603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16d9271640 1 -- 172.21.15.172:0/599732005 >> v1:172.21.15.172:6789/0 conn(0x7f16d4160930 legacy=0x7f16d4072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:18.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16d9271640 1 -- 172.21.15.172:0/599732005 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d415d1f0 con 0x7f16d4159c20 2024-04-14T08:41:18.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.105+0000 7f16d8a70640 1 --2- 172.21.15.172:0/599732005 >> v2:172.21.15.177:3300/0 conn(0x7f16d415c700 0x7f16d41603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:18.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.106+0000 7f16d9271640 1 --2- 172.21.15.172:0/599732005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d4159c20 0x7f16d40ce200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009d90 tx=0x7f16cc02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5edc2aaa957b3e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:18.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.106+0000 7f16cbfff640 1 -- 172.21.15.172:0/599732005 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc039070 con 0x7f16d4159c20 2024-04-14T08:41:18.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.106+0000 7f16cbfff640 1 -- 172.21.15.172:0/599732005 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16cc02cc50 con 0x7f16d4159c20 2024-04-14T08:41:18.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.106+0000 7f16cbfff640 1 -- 172.21.15.172:0/599732005 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc034440 con 0x7f16d4159c20 2024-04-14T08:41:18.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.107+0000 7f16db4d5640 1 -- 172.21.15.172:0/599732005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d4159c20 msgr2=0x7f16d40ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:18.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.107+0000 7f16db4d5640 1 --2- 172.21.15.172:0/599732005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d4159c20 0x7f16d40ce200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009d90 tx=0x7f16cc02bea0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.107+0000 7f16db4d5640 1 -- 172.21.15.172:0/599732005 shutdown_connections 2024-04-14T08:41:18.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.107+0000 7f16db4d5640 1 --2- 172.21.15.172:0/599732005 >> v2:172.21.15.177:3300/0 conn(0x7f16d415c700 0x7f16d41603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.107+0000 7f16db4d5640 1 --2- 172.21.15.172:0/599732005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d4159c20 0x7f16d40ce200 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.107+0000 7f16db4d5640 1 -- 172.21.15.172:0/599732005 >> 172.21.15.172:0/599732005 conn(0x7f16d406b520 msgr2=0x7f16d406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:18.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.107+0000 7f16db4d5640 1 -- 172.21.15.172:0/599732005 shutdown_connections 2024-04-14T08:41:18.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.107+0000 7f16db4d5640 1 -- 172.21.15.172:0/599732005 wait complete. 2024-04-14T08:41:18.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.108+0000 7f16db4d5640 1 Processor -- start 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.108+0000 7f16db4d5640 1 -- start start 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16db4d5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d4159c20 0x7f16d40b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16db4d5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d415c700 0x7f16d41401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16db4d5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16d40b25d0 0x7f16d4152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16db4d5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16d415cb80 con 0x7f16d4159c20 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16db4d5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f16d415a000 con 0x7f16d40b25d0 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16db4d5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f16d4072b60 con 0x7f16d415c700 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16d9271640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d4159c20 0x7f16d40b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16d9271640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d4159c20 0x7f16d40b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45148/0 (socket says 172.21.15.172:45148) 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16d9271640 1 -- 172.21.15.172:0/2058887120 learned_addr learned my addr 172.21.15.172:0/2058887120 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16d8a70640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d415c700 0x7f16d41401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16d9a72640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16d40b25d0 0x7f16d4152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16d9271640 1 -- 172.21.15.172:0/2058887120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d415c700 msgr2=0x7f16d41401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16d9271640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d415c700 0x7f16d41401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16d9271640 1 -- 172.21.15.172:0/2058887120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16d40b25d0 msgr2=0x7f16d4152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16d9271640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16d40b25d0 0x7f16d4152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16d9271640 1 -- 172.21.15.172:0/2058887120 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d40b1530 con 0x7f16d4159c20 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.109+0000 7f16d9271640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d4159c20 0x7f16d40b0d00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009d90 tx=0x7f16cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.110+0000 7f16c9ffb640 1 -- 172.21.15.172:0/2058887120 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc039050 con 0x7f16d4159c20 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.110+0000 7f16c9ffb640 1 -- 172.21.15.172:0/2058887120 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16cc045070 con 0x7f16d4159c20 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.110+0000 7f16db4d5640 1 -- 172.21.15.172:0/2058887120 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d40b1810 con 0x7f16d4159c20 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.110+0000 7f16c9ffb640 1 -- 172.21.15.172:0/2058887120 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc009450 con 0x7f16d4159c20 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.110+0000 7f16db4d5640 1 -- 172.21.15.172:0/2058887120 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d40b1dd0 con 0x7f16d4159c20 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.111+0000 7f16c9ffb640 1 -- 172.21.15.172:0/2058887120 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16cc03e610 con 0x7f16d4159c20 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.112+0000 7f16c9ffb640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f16b806c8f0 0x7f16b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.112+0000 7f16c9ffb640 1 -- 172.21.15.172:0/2058887120 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f16cc0ae5e0 con 0x7f16d4159c20 2024-04-14T08:41:18.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.112+0000 7f16d8a70640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f16b806c8f0 0x7f16b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.112+0000 7f16d8a70640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f16b806c8f0 0x7f16b806edb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009560 tx=0x7f16c4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:18.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16de395740 1 Processor -- start 2024-04-14T08:41:18.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16de395740 1 -- start start 2024-04-14T08:41:18.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16de395740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c9470a4f0 0x558c947080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16de395740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946e9d30 0x558c946c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16de395740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946c1f60 0x558c946ec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16de395740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558c94626800 con 0x558c946e9d30 2024-04-14T08:41:18.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16de395740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558c945ce7f0 con 0x558c946c1f60 2024-04-14T08:41:18.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16de395740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x558c946227b0 con 0x558c9470a4f0 2024-04-14T08:41:18.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16d8a70640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946e9d30 0x558c946c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16d8a70640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946e9d30 0x558c946c1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45162/0 (socket says 172.21.15.172:45162) 2024-04-14T08:41:18.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16d8a70640 1 -- 172.21.15.172:0/2510565257 learned_addr learned my addr 172.21.15.172:0/2510565257 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:18.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16d9271640 1 --2- 172.21.15.172:0/2510565257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c9470a4f0 0x558c947080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.113+0000 7f16d9a72640 1 --2- 172.21.15.172:0/2510565257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946c1f60 0x558c946ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.114+0000 7f16d8a70640 1 -- 172.21.15.172:0/2510565257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c9470a4f0 msgr2=0x558c947080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:18.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.114+0000 7f16d8a70640 1 --2- 172.21.15.172:0/2510565257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c9470a4f0 0x558c947080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.114+0000 7f16d8a70640 1 -- 172.21.15.172:0/2510565257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946c1f60 msgr2=0x558c946ec700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.114+0000 7f16d8a70640 1 --2- 172.21.15.172:0/2510565257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946c1f60 0x558c946ec700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.114+0000 7f16d8a70640 1 -- 172.21.15.172:0/2510565257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c944047d0 con 0x558c946e9d30 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.114+0000 7f16d8a70640 1 --2- 172.21.15.172:0/2510565257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946e9d30 0x558c946c1a20 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f16c4011e90 tx=0x7f16c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606afbde1dd11351 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.114+0000 7f16beffd640 1 -- 172.21.15.172:0/2510565257 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c401b070 con 0x558c946e9d30 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.114+0000 7f16beffd640 1 -- 172.21.15.172:0/2510565257 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16c4027070 con 0x558c946e9d30 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.114+0000 7f16beffd640 1 -- 172.21.15.172:0/2510565257 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4003a20 con 0x558c946e9d30 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.114+0000 7f16de395740 1 -- 172.21.15.172:0/2510565257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946e9d30 msgr2=0x558c946c1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.115+0000 7f16de395740 1 --2- 172.21.15.172:0/2510565257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946e9d30 0x558c946c1a20 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f16c4011e90 tx=0x7f16c4014000 comp rx=0 tx=0).stop 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.115+0000 7f16de395740 1 -- 172.21.15.172:0/2510565257 shutdown_connections 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.115+0000 7f16de395740 1 --2- 172.21.15.172:0/2510565257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946c1f60 0x558c946ec700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.115+0000 7f16de395740 1 --2- 172.21.15.172:0/2510565257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946e9d30 0x558c946c1a20 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.115+0000 7f16de395740 1 --2- 172.21.15.172:0/2510565257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c9470a4f0 0x558c947080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.115+0000 7f16de395740 1 -- 172.21.15.172:0/2510565257 >> 172.21.15.172:0/2510565257 conn(0x558c946beb80 msgr2=0x558c94707700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.115+0000 7f16de395740 1 -- 172.21.15.172:0/2510565257 shutdown_connections 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.115+0000 7f16de395740 1 -- 172.21.15.172:0/2510565257 wait complete. 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.115+0000 7f16de395740 1 -- start start 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.115+0000 7f16de395740 1 Processor -- start 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16de395740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c946e9d30 0x558c946ea260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16de395740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c9470a4f0 0x558c947132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16de395740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 0x558c947137e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16de395740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558c946c0e80 con 0x558c946ea7a0 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16de395740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558c945e1d00 con 0x558c9470a4f0 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16de395740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x558c946bd1b0 con 0x558c946e9d30 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16de395740 10 client.0 _ll_register_callbacks cb 0x558c945e16f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16d9a72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 0x558c947137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16d9271640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c946e9d30 0x558c946ea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16d9a72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 0x558c947137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45166/0 (socket says 172.21.15.172:45166) 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16d9271640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c946e9d30 0x558c946ea260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:37796/0 (socket says 172.21.15.172:37796) 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16d9a72640 1 -- 172.21.15.172:0/1326939055 learned_addr learned my addr 172.21.15.172:0/1326939055 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16d8a70640 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c9470a4f0 0x558c947132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16d9271640 1 -- 172.21.15.172:0/1326939055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c9470a4f0 msgr2=0x558c947132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16d9271640 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c9470a4f0 0x558c947132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16d9271640 1 -- 172.21.15.172:0/1326939055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 msgr2=0x558c947137e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16d9271640 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 0x558c947137e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16d9271640 1 -- 172.21.15.172:0/1326939055 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x558c9433a3f0 con 0x558c946e9d30 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.116+0000 7f16d9a72640 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 0x558c947137e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16d9271640 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c946e9d30 0x558c946ea260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0ac9d0 tx=0x7f16cc0806b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f2339c4e6bfd9b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.177:3301/0 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16b3fff640 1 -- 172.21.15.172:0/1326939055 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc07a6b0 con 0x558c946e9d30 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16de395740 1 -- 172.21.15.172:0/1326939055 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c4027070 con 0x558c946e9d30 2024-04-14T08:41:18.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16b3fff640 1 -- 172.21.15.172:0/1326939055 <== mon.2 v2:172.21.15.177:3301/0 2 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f16cc084990 con 0x558c946e9d30 2024-04-14T08:41:18.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16b3fff640 1 -- 172.21.15.172:0/1326939055 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f16cc09b050 con 0x558c946e9d30 2024-04-14T08:41:18.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16de395740 10 client.7064 fetch_fsmap learned FSMap version 74 2024-04-14T08:41:18.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16de395740 1 -- 172.21.15.172:0/1326939055 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x558c942fa0f0 con 0x558c946e9d30 2024-04-14T08:41:18.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16b3fff640 1 -- 172.21.15.172:0/1326939055 <== mon.2 v2:172.21.15.177:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16c4027070 con 0x558c946e9d30 2024-04-14T08:41:18.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16b3fff640 1 -- 172.21.15.172:0/1326939055 <== mon.2 v2:172.21.15.177:3301/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f16cc07b740 con 0x558c946e9d30 2024-04-14T08:41:18.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16de395740 10 client.7064 fetch_fsmap finished waiting for FSMap version 74 2024-04-14T08:41:18.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16de395740 10 client.7064 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-14T08:41:18.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16de395740 20 client.7064 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:18.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16de395740 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x558c946e5350 0x558c946e57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16de395740 4 client.7064 mds_command: new command op to 8394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:41:18.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16de395740 1 -- 172.21.15.172:0/1326939055 --> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c94633d40 con 0x558c946e5350 2024-04-14T08:41:18.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.117+0000 7f16d8a70640 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x558c946e5350 0x558c946e57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.118+0000 7f16b3fff640 10 client.7064 ms_handle_connect on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:18.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.118+0000 7f16d8a70640 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x558c946e5350 0x558c946e57a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a947b562a08e90cd server_cookie=23c915f6be5e24c2 in_seq=0 out_seq=0 2024-04-14T08:41:18.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.118+0000 7f16b3fff640 1 -- 172.21.15.172:0/1326939055 <== mds.0 v2:172.21.15.177:6834/2602337717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c94633d40 con 0x558c946e5350 2024-04-14T08:41:18.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.118+0000 7f16b3fff640 10 client.7064 handle_command_reply: tid=0 2024-04-14T08:41:18.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.118+0000 7f16de395740 1 client.7064 shutdown 2024-04-14T08:41:18.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.118+0000 7f16de395740 1 -- 172.21.15.172:0/1326939055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c946e9d30 msgr2=0x558c946ea260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:18.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.118+0000 7f16de395740 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c946e9d30 0x558c946ea260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0ac9d0 tx=0x7f16cc0806b0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.119+0000 7f16de395740 1 -- 172.21.15.172:0/1326939055 shutdown_connections 2024-04-14T08:41:18.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.119+0000 7f16de395740 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x558c946e5350 0x558c946e57a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.119+0000 7f16de395740 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 0x558c947137e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.119+0000 7f16b3fff640 0 client.7064 ms_handle_reset on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:18.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.119+0000 7f16de395740 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c9470a4f0 0x558c947132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.119+0000 7f16de395740 1 --2- 172.21.15.172:0/1326939055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c946e9d30 0x558c946ea260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.119+0000 7f16de395740 1 -- 172.21.15.172:0/1326939055 >> 172.21.15.172:0/1326939055 conn(0x558c946c1f60 msgr2=0x558c94707620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:18.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.119+0000 7f16de395740 1 -- 172.21.15.172:0/1326939055 shutdown_connections 2024-04-14T08:41:18.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.119+0000 7f16de395740 1 -- 172.21.15.172:0/1326939055 wait complete. 2024-04-14T08:41:18.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.119+0000 7f16de395740 20 client.7064 trim_cache size 0 max 16384 2024-04-14T08:41:18.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.136+0000 7f16de395740 1 Processor -- start 2024-04-14T08:41:18.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.136+0000 7f16de395740 1 -- start start 2024-04-14T08:41:18.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16de395740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946e9d30 0x558c946c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16de395740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c946ea7a0 0x558c946bb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16de395740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c9470a4f0 0x558c946bbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16de395740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16d41fc9f0 con 0x558c9470a4f0 2024-04-14T08:41:18.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16de395740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f16d40b45c0 con 0x558c946e9d30 2024-04-14T08:41:18.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16de395740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f16d41438c0 con 0x558c946ea7a0 2024-04-14T08:41:18.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16d9a72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c9470a4f0 0x558c946bbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16d9a72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c9470a4f0 0x558c946bbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45170/0 (socket says 172.21.15.172:45170) 2024-04-14T08:41:18.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16d9a72640 1 -- 172.21.15.172:0/780779374 learned_addr learned my addr 172.21.15.172:0/780779374 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:18.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16d8a70640 1 --2- 172.21.15.172:0/780779374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c946ea7a0 0x558c946bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16d9271640 1 --2- 172.21.15.172:0/780779374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946e9d30 0x558c946c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16d9a72640 1 -- 172.21.15.172:0/780779374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c946ea7a0 msgr2=0x558c946bb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16d9a72640 1 --2- 172.21.15.172:0/780779374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c946ea7a0 0x558c946bb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16d9a72640 1 -- 172.21.15.172:0/780779374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946e9d30 msgr2=0x558c946c2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16d9a72640 1 --2- 172.21.15.172:0/780779374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946e9d30 0x558c946c2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16d9a72640 1 -- 172.21.15.172:0/780779374 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c946ea2d0 con 0x558c9470a4f0 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16d9a72640 1 --2- 172.21.15.172:0/780779374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c9470a4f0 0x558c946bbec0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f16d0007480 tx=0x7f16d0007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af1c4b4018d6282 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16be7fc640 1 -- 172.21.15.172:0/780779374 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0015d00 con 0x558c9470a4f0 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16be7fc640 1 -- 172.21.15.172:0/780779374 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d001a030 con 0x558c9470a4f0 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.137+0000 7f16be7fc640 1 -- 172.21.15.172:0/780779374 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0015ea0 con 0x558c9470a4f0 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.138+0000 7f16de395740 1 -- 172.21.15.172:0/780779374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c9470a4f0 msgr2=0x558c946bbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.138+0000 7f16de395740 1 --2- 172.21.15.172:0/780779374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c9470a4f0 0x558c946bbec0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f16d0007480 tx=0x7f16d0007940 comp rx=0 tx=0).stop 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.138+0000 7f16de395740 1 -- 172.21.15.172:0/780779374 shutdown_connections 2024-04-14T08:41:18.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.138+0000 7f16de395740 1 --2- 172.21.15.172:0/780779374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c9470a4f0 0x558c946bbec0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.138+0000 7f16de395740 1 --2- 172.21.15.172:0/780779374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c946ea7a0 0x558c946bb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.138+0000 7f16de395740 1 --2- 172.21.15.172:0/780779374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946e9d30 0x558c946c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.138+0000 7f16de395740 1 -- 172.21.15.172:0/780779374 >> 172.21.15.172:0/780779374 conn(0x558c946c1f60 msgr2=0x558c946eedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:18.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.138+0000 7f16de395740 1 -- 172.21.15.172:0/780779374 shutdown_connections 2024-04-14T08:41:18.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.138+0000 7f16de395740 1 -- 172.21.15.172:0/780779374 wait complete. 2024-04-14T08:41:18.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.139+0000 7f16de395740 1 -- start start 2024-04-14T08:41:18.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.139+0000 7f16de395740 1 Processor -- start 2024-04-14T08:41:18.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.139+0000 7f16de395740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946e9d30 0x558c94785360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.139+0000 7f16de395740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 0x558c9472df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.139+0000 7f16de395740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c9470a4f0 0x558c9471a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.139+0000 7f16de395740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558c946eb920 con 0x558c946ea7a0 2024-04-14T08:41:18.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16de395740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f16d41417a0 con 0x558c946e9d30 2024-04-14T08:41:18.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16de395740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x558c946b82b0 con 0x558c9470a4f0 2024-04-14T08:41:18.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16de395740 10 client.0 _ll_register_callbacks cb 0x7f16cc07bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:18.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16d8a70640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 0x558c9472df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16d8a70640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 0x558c9472df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45176/0 (socket says 172.21.15.172:45176) 2024-04-14T08:41:18.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16d8a70640 1 -- 172.21.15.172:0/2025712106 learned_addr learned my addr 172.21.15.172:0/2025712106 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:18.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16d9a72640 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c9470a4f0 0x558c9471a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16d9271640 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946e9d30 0x558c94785360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16d8a70640 1 -- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c9470a4f0 msgr2=0x558c9471a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:18.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16d8a70640 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c9470a4f0 0x558c9471a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16d8a70640 1 -- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946e9d30 msgr2=0x558c94785360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:18.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16d8a70640 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946e9d30 0x558c94785360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16d8a70640 1 -- 172.21.15.172:0/2025712106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c946e13c0 con 0x558c946ea7a0 2024-04-14T08:41:18.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16d9271640 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946e9d30 0x558c94785360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:41:18.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16d8a70640 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 0x558c9472df40 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f16c4024040 tx=0x7f16c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82e66e99c175ed server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:18.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:41:18.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16b3fff640 1 -- 172.21.15.172:0/2025712106 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c401b040 con 0x558c946ea7a0 2024-04-14T08:41:18.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16b3fff640 1 -- 172.21.15.172:0/2025712106 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 6101+0+0 (secure 0 0 0) 0x7f16c4042c70 con 0x558c946ea7a0 2024-04-14T08:41:18.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16de395740 1 -- 172.21.15.172:0/2025712106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d001a030 con 0x558c946ea7a0 2024-04-14T08:41:18.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16b3fff640 1 -- 172.21.15.172:0/2025712106 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f16c4049050 con 0x558c946ea7a0 2024-04-14T08:41:18.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16de395740 10 client.8688 fetch_fsmap learned FSMap version 74 2024-04-14T08:41:18.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16de395740 1 -- 172.21.15.172:0/2025712106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x558c946c1aa0 con 0x558c946ea7a0 2024-04-14T08:41:18.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.140+0000 7f16b3fff640 1 -- 172.21.15.172:0/2025712106 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d001a030 con 0x558c946ea7a0 2024-04-14T08:41:18.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16b3fff640 1 -- 172.21.15.172:0/2025712106 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f16c404c020 con 0x558c946ea7a0 2024-04-14T08:41:18.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 10 client.8688 fetch_fsmap finished waiting for FSMap version 74 2024-04-14T08:41:18.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 10 client.8688 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-14T08:41:18.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 20 client.8688 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:18.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x558c946c0680 0x558c9471a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:18.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 4 client.8688 mds_command: new command op to 8394 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:41:18.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 1 -- 172.21.15.172:0/2025712106 --> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c9462e090 con 0x558c946c0680 2024-04-14T08:41:18.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16d9271640 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x558c946c0680 0x558c9471a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:18.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16b3fff640 10 client.8688 ms_handle_connect on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:18.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16d9271640 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x558c946c0680 0x558c9471a900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b79872a816f82c server_cookie=4f2c93133226944d in_seq=0 out_seq=0 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16b3fff640 1 -- 172.21.15.172:0/2025712106 <== mds.0 v2:172.21.15.177:6834/2602337717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x558c9462e090 con 0x558c946c0680 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16b3fff640 10 client.8688 handle_command_reply: tid=0 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 1 client.8688 shutdown 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 1 -- 172.21.15.172:0/2025712106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 msgr2=0x558c9472df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 0x558c9472df40 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f16c4024040 tx=0x7f16c4004730 comp rx=0 tx=0).stop 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 1 -- 172.21.15.172:0/2025712106 shutdown_connections 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x558c946c0680 0x558c9471a900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558c9470a4f0 0x558c9471a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c946ea7a0 0x558c9472df40 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 1 --2- 172.21.15.172:0/2025712106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558c946e9d30 0x558c94785360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16b3fff640 0 client.8688 ms_handle_reset on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.141+0000 7f16de395740 1 -- 172.21.15.172:0/2025712106 >> 172.21.15.172:0/2025712106 conn(0x558c946c1f60 msgr2=0x558c946c1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16de395740 1 -- 172.21.15.172:0/2025712106 shutdown_connections 2024-04-14T08:41:18.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16de395740 1 -- 172.21.15.172:0/2025712106 wait complete. 2024-04-14T08:41:18.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16de395740 20 client.8688 trim_cache size 0 max 16384 2024-04-14T08:41:18.151 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:41:18.151 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-14T08:41:18.151 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-04-14T08:41:18.151 INFO:teuthology.orchestra.run.smithi172.stdout: "331b9465-e788-45dc-be77-038daa4989eb": { 2024-04-14T08:41:18.151 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-04-14T08:41:18.152 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "331b9465-e788-45dc-be77-038daa4989eb", 2024-04-14T08:41:18.152 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,repair" 2024-04-14T08:41:18.152 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:18.152 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:18.152 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:41:18.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16db4d5640 1 -- 172.21.15.172:0/2058887120 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f16b806c8f0 msgr2=0x7f16b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:18.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16db4d5640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f16b806c8f0 0x7f16b806edb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009560 tx=0x7f16c4009190 comp rx=0 tx=0).stop 2024-04-14T08:41:18.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16db4d5640 1 -- 172.21.15.172:0/2058887120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d4159c20 msgr2=0x7f16d40b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:18.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16db4d5640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d4159c20 0x7f16d40b0d00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009d90 tx=0x7f16cc004a40 comp rx=0 tx=0).stop 2024-04-14T08:41:18.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16db4d5640 1 -- 172.21.15.172:0/2058887120 shutdown_connections 2024-04-14T08:41:18.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16db4d5640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f16b806c8f0 0x7f16b806edb0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16db4d5640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16d40b25d0 0x7f16d4152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16db4d5640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16d415c700 0x7f16d41401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16db4d5640 1 --2- 172.21.15.172:0/2058887120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d4159c20 0x7f16d40b0d00 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:18.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16db4d5640 1 -- 172.21.15.172:0/2058887120 >> 172.21.15.172:0/2058887120 conn(0x7f16d406b520 msgr2=0x7f16d415a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:18.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16db4d5640 1 -- 172.21.15.172:0/2058887120 shutdown_connections 2024-04-14T08:41:18.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.142+0000 7f16db4d5640 1 -- 172.21.15.172:0/2058887120 wait complete. 2024-04-14T08:41:18.155 INFO:tasks.cephfs.filesystem:scrub status for tag:331b9465-e788-45dc-be77-038daa4989eb - {'path': '/', 'tag': '331b9465-e788-45dc-be77-038daa4989eb', 'options': 'recursive,repair'} 2024-04-14T08:41:18.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:18.295+0000 7effb2ffd640 1 -- 172.21.15.172:0/1541404582 <== mon.1 v2:172.21.15.177:3300/0 7 ==== log(4 entries from seq 1 at 2024-04-14T08:41:17.991444+0000) ==== 695+0+0 (secure 0 0 0) 0x7effb40790b0 con 0x7effbc0b3330 2024-04-14T08:41:18.297 INFO:teuthology.orchestra.run.smithi172.stdout: cluster: 2024-04-14T08:41:18.297 INFO:teuthology.orchestra.run.smithi172.stdout: id: 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:41:18.297 INFO:teuthology.orchestra.run.smithi172.stdout: health: HEALTH_OK 2024-04-14T08:41:18.297 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:18.297 INFO:teuthology.orchestra.run.smithi172.stdout: services: 2024-04-14T08:41:18.297 INFO:teuthology.orchestra.run.smithi172.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2024-04-14T08:41:18.297 INFO:teuthology.orchestra.run.smithi172.stdout: mgr: y(active, since 8m), standbys: x 2024-04-14T08:41:18.297 INFO:teuthology.orchestra.run.smithi172.stdout: mds: 1/1 daemons up, 3 standby 2024-04-14T08:41:18.297 INFO:teuthology.orchestra.run.smithi172.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 8m) 2024-04-14T08:41:18.297 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout: data: 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout: volumes: 1/1 healthy 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout: pools: 3 pools, 97 pgs 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout: objects: 30 objects, 19 MiB 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout: usage: 276 MiB used, 720 GiB / 720 GiB avail 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout: pgs: 97 active+clean 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout: io: 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout: client: 92 B/s rd, 1.6 MiB/s wr, 0 op/s rd, 4 op/s wr 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:17.991444+0000 mds.b [INF] scrub queued for path: / 2024-04-14T08:41:18.298 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:17.991447+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-04-14T08:41:18.299 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:17.991451+0000 mds.b [INF] scrub summary: active paths [/] 2024-04-14T08:41:18.299 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:17.992553+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-04-14T08:41:21.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:21.304+0000 7effb2ffd640 1 -- 172.21.15.172:0/1541404582 <== mon.1 v2:172.21.15.177:3300/0 8 ==== log(1 entries from seq 5 at 2024-04-14T08:41:20.113435+0000) ==== 189+0+0 (secure 0 0 0) 0x7effb408a3c0 con 0x7effbc0b3330 2024-04-14T08:41:21.306 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:20.113435+0000 mds.b [INF] scrub summary: idle 2024-04-14T08:41:25.097 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:41:25.101 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:41:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:35.327+0000 7effb2ffd640 1 -- 172.21.15.172:0/1541404582 <== mon.1 v2:172.21.15.177:3300/0 9 ==== log(1 entries from seq 1148 at 2024-04-14T08:41:34.488021+0000) ==== 323+0+0 (secure 0 0 0) 0x7effb4057b80 con 0x7effbc0b3330 2024-04-14T08:41:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:36.341+0000 7effb2ffd640 1 -- 172.21.15.172:0/1541404582 <== mon.1 v2:172.21.15.177:3300/0 10 ==== log(1 entries from seq 1149 at 2024-04-14T08:41:35.336287+0000) ==== 325+0+0 (secure 0 0 0) 0x7effb4057de0 con 0x7effbc0b3330 2024-04-14T08:41:48.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:48.157 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-14T08:41:48.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.270+0000 7f946211e640 1 Processor -- start 2024-04-14T08:41:48.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.270+0000 7f946211e640 1 -- start start 2024-04-14T08:41:48.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.271+0000 7f946211e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945c15e590 0x7f945c1629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.271+0000 7f946211e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f945c0b65a0 0x7f945c0d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.271+0000 7f946211e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f945c0b3750 con 0x7f945c0cea60 2024-04-14T08:41:48.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.271+0000 7f946211e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f945c0cef20 con 0x7f945c0b65a0 2024-04-14T08:41:48.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.271+0000 7f946211e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f945c0cf200 con 0x7f945c15e590 2024-04-14T08:41:48.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.271+0000 7f945bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f945c0b65a0 0x7f945c0d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.271+0000 7f945affd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f945c0cea60 0x7f945c0b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44190/0 (socket says 172.21.15.172:44190) 2024-04-14T08:41:48.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.271+0000 7f945affd640 1 -- 172.21.15.172:0/2821315249 learned_addr learned my addr 172.21.15.172:0/2821315249 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:48.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.272+0000 7f945b7fe640 1 --2- 172.21.15.172:0/2821315249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945c15e590 0x7f945c1629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.272+0000 7f945bfff640 1 -- 172.21.15.172:0/2821315249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945c15e590 msgr2=0x7f945c1629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.272+0000 7f945bfff640 1 --2- 172.21.15.172:0/2821315249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945c15e590 0x7f945c1629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.272+0000 7f945bfff640 1 -- 172.21.15.172:0/2821315249 >> v1:172.21.15.172:6789/0 conn(0x7f945c0cea60 legacy=0x7f945c0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.272+0000 7f945bfff640 1 -- 172.21.15.172:0/2821315249 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c0cf4e0 con 0x7f945c0b65a0 2024-04-14T08:41:48.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.272+0000 7f945a7fc640 1 -- 172.21.15.172:0/2821315249 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2001680928 0 0) 0x7f945c0b3750 con 0x7f945c0cea60 2024-04-14T08:41:48.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.273+0000 7f945bfff640 1 --2- 172.21.15.172:0/2821315249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0b65a0 0x7f945c0d2760 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9454009d90 tx=0x7f945402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5525428cae6e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.273+0000 7f945a7fc640 1 -- 172.21.15.172:0/2821315249 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454039070 con 0x7f945c0b65a0 2024-04-14T08:41:48.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.273+0000 7f945a7fc640 1 -- 172.21.15.172:0/2821315249 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945402cc50 con 0x7f945c0b65a0 2024-04-14T08:41:48.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.273+0000 7f945a7fc640 1 -- 172.21.15.172:0/2821315249 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454034440 con 0x7f945c0b65a0 2024-04-14T08:41:48.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.273+0000 7f946211e640 1 -- 172.21.15.172:0/2821315249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0b65a0 msgr2=0x7f945c0d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.273+0000 7f946211e640 1 --2- 172.21.15.172:0/2821315249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0b65a0 0x7f945c0d2760 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9454009d90 tx=0x7f945402bea0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.273+0000 7f946211e640 1 -- 172.21.15.172:0/2821315249 shutdown_connections 2024-04-14T08:41:48.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.273+0000 7f946211e640 1 --2- 172.21.15.172:0/2821315249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0b65a0 0x7f945c0d2760 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.273+0000 7f946211e640 1 --2- 172.21.15.172:0/2821315249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945c15e590 0x7f945c1629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.273+0000 7f946211e640 1 -- 172.21.15.172:0/2821315249 >> 172.21.15.172:0/2821315249 conn(0x7f945c0d0e90 msgr2=0x7f945c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.273+0000 7f946211e640 1 -- 172.21.15.172:0/2821315249 shutdown_connections 2024-04-14T08:41:48.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.273+0000 7f946211e640 1 -- 172.21.15.172:0/2821315249 wait complete. 2024-04-14T08:41:48.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.274+0000 7f946211e640 1 Processor -- start 2024-04-14T08:41:48.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.275+0000 7f946211e640 1 -- start start 2024-04-14T08:41:48.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.275+0000 7f946211e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945c0b65a0 0x7f945c06c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.275+0000 7f946211e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0cea60 0x7f945c0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.275+0000 7f946211e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f945c15e590 0x7f945c0b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.275+0000 7f946211e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f945c0d2e20 con 0x7f945c0cea60 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.275+0000 7f946211e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f945c0d2ca0 con 0x7f945c15e590 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.275+0000 7f946211e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f945c0d2fa0 con 0x7f945c0b65a0 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f945affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0cea60 0x7f945c0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f945affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0cea60 0x7f945c0b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56618/0 (socket says 172.21.15.172:56618) 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f945affd640 1 -- 172.21.15.172:0/907841960 learned_addr learned my addr 172.21.15.172:0/907841960 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f945b7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945c0b65a0 0x7f945c06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f945bfff640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f945c15e590 0x7f945c0b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f945affd640 1 -- 172.21.15.172:0/907841960 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945c0b65a0 msgr2=0x7f945c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f945affd640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945c0b65a0 0x7f945c06c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f945affd640 1 -- 172.21.15.172:0/907841960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f945c15e590 msgr2=0x7f945c0b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f945affd640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f945c15e590 0x7f945c0b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f945affd640 1 -- 172.21.15.172:0/907841960 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c1f84f0 con 0x7f945c0cea60 2024-04-14T08:41:48.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f945affd640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0cea60 0x7f945c0b2200 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f945000c500 tx=0x7f945000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f9443fff640 1 -- 172.21.15.172:0/907841960 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450018070 con 0x7f945c0cea60 2024-04-14T08:41:48.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f9443fff640 1 -- 172.21.15.172:0/907841960 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9450007580 con 0x7f945c0cea60 2024-04-14T08:41:48.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f946211e640 1 -- 172.21.15.172:0/907841960 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945c1f7730 con 0x7f945c0cea60 2024-04-14T08:41:48.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f9443fff640 1 -- 172.21.15.172:0/907841960 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450004a20 con 0x7f945c0cea60 2024-04-14T08:41:48.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.276+0000 7f946211e640 1 -- 172.21.15.172:0/907841960 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945c1f7cf0 con 0x7f945c0cea60 2024-04-14T08:41:48.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.278+0000 7f9443fff640 1 -- 172.21.15.172:0/907841960 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f945001b070 con 0x7f945c0cea60 2024-04-14T08:41:48.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.279+0000 7f9443fff640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f944806c8f0 0x7f944806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.279+0000 7f9443fff640 1 -- 172.21.15.172:0/907841960 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6130+0+0 (secure 0 0 0) 0x7f945008af20 con 0x7f945c0cea60 2024-04-14T08:41:48.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.279+0000 7f945b7fe640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f944806c8f0 0x7f944806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.279+0000 7f945b7fe640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f944806c8f0 0x7f944806edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f945c06d520 tx=0x7f944c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.279+0000 7f9465006740 1 Processor -- start 2024-04-14T08:41:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.279+0000 7f9465006740 1 -- start start 2024-04-14T08:41:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.280+0000 7f9465006740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7b970 0x55d4e3e4e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.280+0000 7f9465006740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7bd50 0x55d4e3e7e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.280+0000 7f9465006740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e4ee50 0x55d4e3e7ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.280+0000 7f9465006740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d4e3db8800 con 0x55d4e3e4ee50 2024-04-14T08:41:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.280+0000 7f9465006740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d4e3d607f0 con 0x55d4e3e7bd50 2024-04-14T08:41:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.280+0000 7f9465006740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d4e3db47b0 con 0x55d4e3e7b970 2024-04-14T08:41:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.280+0000 7f945bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e4ee50 0x55d4e3e7ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.280+0000 7f945bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e4ee50 0x55d4e3e7ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56634/0 (socket says 172.21.15.172:56634) 2024-04-14T08:41:48.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.280+0000 7f945bfff640 1 -- 172.21.15.172:0/901486989 learned_addr learned my addr 172.21.15.172:0/901486989 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.280+0000 7f945b7fe640 1 --2- 172.21.15.172:0/901486989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7b970 0x55d4e3e4e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.280+0000 7f945affd640 1 --2- 172.21.15.172:0/901486989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7bd50 0x55d4e3e7e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.281+0000 7f945bfff640 1 -- 172.21.15.172:0/901486989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7b970 msgr2=0x55d4e3e4e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.281+0000 7f945bfff640 1 --2- 172.21.15.172:0/901486989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7b970 0x55d4e3e4e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.281+0000 7f945bfff640 1 -- 172.21.15.172:0/901486989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7bd50 msgr2=0x55d4e3e7e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.281+0000 7f945bfff640 1 --2- 172.21.15.172:0/901486989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7bd50 0x55d4e3e7e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.281+0000 7f945bfff640 1 -- 172.21.15.172:0/901486989 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e3b967d0 con 0x55d4e3e4ee50 2024-04-14T08:41:48.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.281+0000 7f945b7fe640 1 --2- 172.21.15.172:0/901486989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7b970 0x55d4e3e4e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:48.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.281+0000 7f945bfff640 1 --2- 172.21.15.172:0/901486989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e4ee50 0x55d4e3e7ec40 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f9454037040 tx=0x7f945402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1f94a5c835b167 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.281+0000 7f94417fa640 1 -- 172.21.15.172:0/901486989 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454039040 con 0x55d4e3e4ee50 2024-04-14T08:41:48.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.281+0000 7f94417fa640 1 -- 172.21.15.172:0/901486989 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945403a040 con 0x55d4e3e4ee50 2024-04-14T08:41:48.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.281+0000 7f94417fa640 1 -- 172.21.15.172:0/901486989 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454034440 con 0x55d4e3e4ee50 2024-04-14T08:41:48.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.282+0000 7f9465006740 1 -- 172.21.15.172:0/901486989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e4ee50 msgr2=0x55d4e3e7ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.282+0000 7f9465006740 1 --2- 172.21.15.172:0/901486989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e4ee50 0x55d4e3e7ec40 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f9454037040 tx=0x7f945402e000 comp rx=0 tx=0).stop 2024-04-14T08:41:48.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.282+0000 7f9465006740 1 -- 172.21.15.172:0/901486989 shutdown_connections 2024-04-14T08:41:48.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.282+0000 7f9465006740 1 --2- 172.21.15.172:0/901486989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e4ee50 0x55d4e3e7ec40 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.282+0000 7f9465006740 1 --2- 172.21.15.172:0/901486989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7bd50 0x55d4e3e7e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.282+0000 7f9465006740 1 --2- 172.21.15.172:0/901486989 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7b970 0x55d4e3e4e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.282+0000 7f9465006740 1 -- 172.21.15.172:0/901486989 >> 172.21.15.172:0/901486989 conn(0x55d4e3e50b80 msgr2=0x55d4e3e99700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.282+0000 7f9465006740 1 -- 172.21.15.172:0/901486989 shutdown_connections 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.282+0000 7f9465006740 1 -- 172.21.15.172:0/901486989 wait complete. 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.282+0000 7f9465006740 1 -- start start 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.282+0000 7f9465006740 1 Processor -- start 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.283+0000 7f9465006740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7b970 0x55d4e3e79ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.283+0000 7f9465006740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e80ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.283+0000 7f9465006740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 0x55d4e3e936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.283+0000 7f9465006740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d4e3e53120 con 0x55d4e3e80ff0 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.283+0000 7f9465006740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d4e3d73d00 con 0x55d4e3e7b970 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.283+0000 7f9465006740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d4e3e52510 con 0x55d4e3e7bd50 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.283+0000 7f9465006740 10 client.0 _ll_register_callbacks cb 0x55d4e3d736f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:48.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.283+0000 7f945bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 0x55d4e3e936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.283+0000 7f945bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 0x55d4e3e936c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56648/0 (socket says 172.21.15.172:56648) 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.283+0000 7f945bfff640 1 -- 172.21.15.172:0/428424856 learned_addr learned my addr 172.21.15.172:0/428424856 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.283+0000 7f945affd640 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e80ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.283+0000 7f945b7fe640 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7b970 0x55d4e3e79ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f945bfff640 1 -- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 msgr2=0x55d4e3e80ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f945bfff640 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e80ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f945bfff640 1 -- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7b970 msgr2=0x55d4e3e79ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f945bfff640 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7b970 0x55d4e3e79ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f945bfff640 1 -- 172.21.15.172:0/428424856 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e3acc3f0 con 0x55d4e3e80ff0 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f945affd640 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e80ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f945bfff640 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 0x55d4e3e936c0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f9454009e80 tx=0x7f9454004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a6e7b35ed03eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f943a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f943a7fc640 1 -- 172.21.15.172:0/428424856 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945404d030 con 0x55d4e3e80ff0 2024-04-14T08:41:48.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f943a7fc640 1 -- 172.21.15.172:0/428424856 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 6130+0+0 (secure 0 0 0) 0x7f9454003ba0 con 0x55d4e3e80ff0 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f9465006740 1 -- 172.21.15.172:0/428424856 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945403a040 con 0x55d4e3e80ff0 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f943a7fc640 1 -- 172.21.15.172:0/428424856 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f9454056050 con 0x55d4e3e80ff0 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f9465006740 10 client.8703 fetch_fsmap learned FSMap version 74 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f9465006740 1 -- 172.21.15.172:0/428424856 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55d4e3a8c0f0 con 0x55d4e3e80ff0 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f943a7fc640 1 -- 172.21.15.172:0/428424856 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945403a040 con 0x55d4e3e80ff0 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f943a7fc640 1 -- 172.21.15.172:0/428424856 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f9454059020 con 0x55d4e3e80ff0 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f9465006740 10 client.8703 fetch_fsmap finished waiting for FSMap version 74 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f9465006740 10 client.8703 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f9465006740 20 client.8703 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f9465006740 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d4e3eacd50 0x55d4e3ead130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f9465006740 4 client.8703 mds_command: new command op to 8394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.284+0000 7f9465006740 1 -- 172.21.15.172:0/428424856 --> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4e3dc5d40 con 0x55d4e3eacd50 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.285+0000 7f945b7fe640 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d4e3eacd50 0x55d4e3ead130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.285+0000 7f943a7fc640 10 client.8703 ms_handle_connect on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.285+0000 7f945b7fe640 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d4e3eacd50 0x55d4e3ead130 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0df8cd605b40112 server_cookie=cf70d3bb32f94a53 in_seq=0 out_seq=0 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f943a7fc640 1 -- 172.21.15.172:0/428424856 <== mds.0 v2:172.21.15.177:6834/2602337717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4e3dc5d40 con 0x55d4e3eacd50 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f943a7fc640 10 client.8703 handle_command_reply: tid=0 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f9465006740 1 client.8703 shutdown 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f9465006740 1 -- 172.21.15.172:0/428424856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 msgr2=0x55d4e3e936c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f9465006740 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 0x55d4e3e936c0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f9454009e80 tx=0x7f9454004ab0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f9465006740 1 -- 172.21.15.172:0/428424856 shutdown_connections 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f9465006740 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d4e3eacd50 0x55d4e3ead130 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f9465006740 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 0x55d4e3e936c0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f9465006740 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e80ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f943a7fc640 0 client.8703 ms_handle_reset on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f9465006740 1 --2- 172.21.15.172:0/428424856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7b970 0x55d4e3e79ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f9465006740 1 -- 172.21.15.172:0/428424856 >> 172.21.15.172:0/428424856 conn(0x55d4e3e4ee50 msgr2=0x55d4e3e9cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f9465006740 1 -- 172.21.15.172:0/428424856 shutdown_connections 2024-04-14T08:41:48.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f9465006740 1 -- 172.21.15.172:0/428424856 wait complete. 2024-04-14T08:41:48.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.286+0000 7f9465006740 20 client.8703 trim_cache size 0 max 16384 2024-04-14T08:41:48.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.303+0000 7f9465006740 1 Processor -- start 2024-04-14T08:41:48.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.303+0000 7f9465006740 1 -- start start 2024-04-14T08:41:48.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.304+0000 7f9465006740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7b970 0x55d4e3eab310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.304+0000 7f9465006740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e99c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.304+0000 7f9465006740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 0x55d4e3eabf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.304+0000 7f9465006740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d4e3e829d0 con 0x55d4e3e80ff0 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.304+0000 7f9465006740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f945c200f20 con 0x55d4e3e7b970 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.304+0000 7f9465006740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f945c1fff60 con 0x55d4e3e7bd50 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.304+0000 7f945bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 0x55d4e3eabf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.304+0000 7f945b7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7b970 0x55d4e3eab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.304+0000 7f945bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 0x55d4e3eabf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56652/0 (socket says 172.21.15.172:56652) 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.304+0000 7f945affd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e99c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.304+0000 7f945bfff640 1 -- 172.21.15.172:0/2241655808 learned_addr learned my addr 172.21.15.172:0/2241655808 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.305+0000 7f945bfff640 1 -- 172.21.15.172:0/2241655808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 msgr2=0x55d4e3e99c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.305+0000 7f945bfff640 1 --2- 172.21.15.172:0/2241655808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e99c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.305+0000 7f945bfff640 1 -- 172.21.15.172:0/2241655808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7b970 msgr2=0x55d4e3eab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.305+0000 7f945bfff640 1 --2- 172.21.15.172:0/2241655808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7b970 0x55d4e3eab310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.305+0000 7f945bfff640 1 -- 172.21.15.172:0/2241655808 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e3e4a8a0 con 0x55d4e3e80ff0 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.305+0000 7f945bfff640 1 --2- 172.21.15.172:0/2241655808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 0x55d4e3eabf70 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f945402c4b0 tx=0x7f9454043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29aeee2090a0b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.305+0000 7f9440ff9640 1 -- 172.21.15.172:0/2241655808 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945404d070 con 0x55d4e3e80ff0 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.305+0000 7f9440ff9640 1 -- 172.21.15.172:0/2241655808 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9454034640 con 0x55d4e3e80ff0 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.305+0000 7f9440ff9640 1 -- 172.21.15.172:0/2241655808 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454039040 con 0x55d4e3e80ff0 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.306+0000 7f9465006740 1 -- 172.21.15.172:0/2241655808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 msgr2=0x55d4e3eabf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.306+0000 7f9465006740 1 --2- 172.21.15.172:0/2241655808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 0x55d4e3eabf70 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f945402c4b0 tx=0x7f9454043000 comp rx=0 tx=0).stop 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.306+0000 7f9465006740 1 -- 172.21.15.172:0/2241655808 shutdown_connections 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.306+0000 7f9465006740 1 --2- 172.21.15.172:0/2241655808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e80ff0 0x55d4e3eabf70 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.306+0000 7f9465006740 1 --2- 172.21.15.172:0/2241655808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e99c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.306+0000 7f9465006740 1 --2- 172.21.15.172:0/2241655808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e7b970 0x55d4e3eab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.306+0000 7f9465006740 1 -- 172.21.15.172:0/2241655808 >> 172.21.15.172:0/2241655808 conn(0x55d4e3e4ee50 msgr2=0x55d4e3ebcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.306+0000 7f9465006740 1 -- 172.21.15.172:0/2241655808 shutdown_connections 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.306+0000 7f9465006740 1 -- 172.21.15.172:0/2241655808 wait complete. 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.306+0000 7f9465006740 1 -- start start 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.306+0000 7f9465006740 1 Processor -- start 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f9465006740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e7b970 0x55d4e3f12ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f9465006740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e7fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f9465006740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e80ff0 0x55d4e3ea7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f9465006740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1ff990 con 0x55d4e3e7b970 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f9465006740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f945c15fac0 con 0x55d4e3e80ff0 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f9465006740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d4e3e7a930 con 0x55d4e3e7bd50 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f9465006740 10 client.0 _ll_register_callbacks cb 0x7f944c009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f945b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e7b970 0x55d4e3f12ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f945b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e7b970 0x55d4e3f12ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56654/0 (socket says 172.21.15.172:56654) 2024-04-14T08:41:48.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f945b7fe640 1 -- 172.21.15.172:0/3742095489 learned_addr learned my addr 172.21.15.172:0/3742095489 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f945affd640 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e7fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f945bfff640 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e80ff0 0x55d4e3ea7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f945b7fe640 1 -- 172.21.15.172:0/3742095489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 msgr2=0x55d4e3e7fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f945b7fe640 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e7fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f945b7fe640 1 -- 172.21.15.172:0/3742095489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e80ff0 msgr2=0x55d4e3ea7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f945b7fe640 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e80ff0 0x55d4e3ea7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.307+0000 7f945b7fe640 1 -- 172.21.15.172:0/3742095489 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e3e7e200 con 0x55d4e3e7b970 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f945b7fe640 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e7b970 0x55d4e3f12ee0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x55d4e3ead730 tx=0x7f944c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90e3cfde756fa25 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f943a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f943a7fc640 1 -- 172.21.15.172:0/3742095489 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c023b90 con 0x55d4e3e7b970 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f943a7fc640 1 -- 172.21.15.172:0/3742095489 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 6130+0+0 (secure 0 0 0) 0x7f944c023d30 con 0x55d4e3e7b970 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f9465006740 1 -- 172.21.15.172:0/3742095489 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9454034640 con 0x55d4e3e7b970 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f943a7fc640 1 -- 172.21.15.172:0/3742095489 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f944c03d050 con 0x55d4e3e7b970 2024-04-14T08:41:48.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f9465006740 10 client.8709 fetch_fsmap learned FSMap version 74 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f9465006740 1 -- 172.21.15.172:0/3742095489 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55d4e3ea8360 con 0x55d4e3e7b970 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f943a7fc640 1 -- 172.21.15.172:0/3742095489 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9454034640 con 0x55d4e3e7b970 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f943a7fc640 1 -- 172.21.15.172:0/3742095489 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f944c020040 con 0x55d4e3e7b970 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f9465006740 10 client.8709 fetch_fsmap finished waiting for FSMap version 74 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f9465006740 10 client.8709 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f9465006740 20 client.8709 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f9465006740 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d4e3e79cf0 0x55d4e3e7a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f9465006740 4 client.8709 mds_command: new command op to 8394 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f9465006740 1 -- 172.21.15.172:0/3742095489 --> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d4e3dc0090 con 0x55d4e3e79cf0 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.308+0000 7f945affd640 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d4e3e79cf0 0x55d4e3e7a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f945affd640 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d4e3e79cf0 0x55d4e3e7a110 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64ab8600df5c7a3 server_cookie=f4ee4a8e3d0c4611 in_seq=0 out_seq=0 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f943a7fc640 10 client.8709 ms_handle_connect on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f943a7fc640 1 -- 172.21.15.172:0/3742095489 <== mds.0 v2:172.21.15.177:6834/2602337717 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d4e3dc0090 con 0x55d4e3e79cf0 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f943a7fc640 10 client.8709 handle_command_reply: tid=0 2024-04-14T08:41:48.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f9465006740 1 client.8709 shutdown 2024-04-14T08:41:48.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f9465006740 1 -- 172.21.15.172:0/3742095489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e7b970 msgr2=0x55d4e3f12ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f9465006740 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e7b970 0x55d4e3f12ee0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x55d4e3ead730 tx=0x7f944c004730 comp rx=0 tx=0).stop 2024-04-14T08:41:48.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f9465006740 1 -- 172.21.15.172:0/3742095489 shutdown_connections 2024-04-14T08:41:48.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f9465006740 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.177:6834/2602337717,v1:172.21.15.177:6835/2602337717] conn(0x55d4e3e79cf0 0x55d4e3e7a110 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f9465006740 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4e3e80ff0 0x55d4e3ea7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f943a7fc640 0 client.8709 ms_handle_reset on v2:172.21.15.177:6834/2602337717 2024-04-14T08:41:48.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f9465006740 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d4e3e7bd50 0x55d4e3e7fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f9465006740 1 --2- 172.21.15.172:0/3742095489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4e3e7b970 0x55d4e3f12ee0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f9465006740 1 -- 172.21.15.172:0/3742095489 >> 172.21.15.172:0/3742095489 conn(0x55d4e3e4ee50 msgr2=0x55d4e3ebcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f9465006740 1 -- 172.21.15.172:0/3742095489 shutdown_connections 2024-04-14T08:41:48.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f9465006740 1 -- 172.21.15.172:0/3742095489 wait complete. 2024-04-14T08:41:48.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.309+0000 7f9465006740 20 client.8709 trim_cache size 0 max 16384 2024-04-14T08:41:48.319 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:41:48.319 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-04-14T08:41:48.319 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-04-14T08:41:48.319 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:41:48.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.310+0000 7f946211e640 1 -- 172.21.15.172:0/907841960 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f944806c8f0 msgr2=0x7f944806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.310+0000 7f946211e640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f944806c8f0 0x7f944806edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f945c06d520 tx=0x7f944c009190 comp rx=0 tx=0).stop 2024-04-14T08:41:48.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.310+0000 7f946211e640 1 -- 172.21.15.172:0/907841960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0cea60 msgr2=0x7f945c0b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.310+0000 7f946211e640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0cea60 0x7f945c0b2200 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f945000c500 tx=0x7f945000c9c0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.310+0000 7f946211e640 1 -- 172.21.15.172:0/907841960 shutdown_connections 2024-04-14T08:41:48.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.310+0000 7f946211e640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f944806c8f0 0x7f944806edb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.310+0000 7f946211e640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f945c15e590 0x7f945c0b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.310+0000 7f946211e640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0cea60 0x7f945c0b2200 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.310+0000 7f946211e640 1 --2- 172.21.15.172:0/907841960 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f945c0b65a0 0x7f945c06c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.310+0000 7f946211e640 1 -- 172.21.15.172:0/907841960 >> 172.21.15.172:0/907841960 conn(0x7f945c0d0e90 msgr2=0x7f945c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.310+0000 7f946211e640 1 -- 172.21.15.172:0/907841960 shutdown_connections 2024-04-14T08:41:48.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.310+0000 7f946211e640 1 -- 172.21.15.172:0/907841960 wait complete. 2024-04-14T08:41:48.322 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-14T08:41:48.423 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:41:48.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.524+0000 7fb49257b640 1 Processor -- start 2024-04-14T08:41:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.524+0000 7fb49257b640 1 -- start start 2024-04-14T08:41:48.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb49257b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb48c0ccca0 0x7fb48c06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb49257b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb48c06bc30 0x7fb48c0b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb49257b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb48c0ceee0 con 0x7fb48c073090 2024-04-14T08:41:48.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb49257b640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb48c073520 con 0x7fb48c0ccca0 2024-04-14T08:41:48.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb49257b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb48c070150 con 0x7fb48c06bc30 2024-04-14T08:41:48.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb48bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb48c0ccca0 0x7fb48c06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb490b18640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb48c06bc30 0x7fb48c0b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb48bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb48c0ccca0 0x7fb48c06fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56666/0 (socket says 172.21.15.172:56666) 2024-04-14T08:41:48.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb48bfff640 1 -- 172.21.15.172:0/1023936765 learned_addr learned my addr 172.21.15.172:0/1023936765 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb48bfff640 1 -- 172.21.15.172:0/1023936765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb48c06bc30 msgr2=0x7fb48c0b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb48bfff640 1 --2- 172.21.15.172:0/1023936765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb48c06bc30 0x7fb48c0b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb48bfff640 1 -- 172.21.15.172:0/1023936765 >> v1:172.21.15.177:6789/0 conn(0x7fb48c073090 legacy=0x7fb48c06b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb48bfff640 1 -- 172.21.15.172:0/1023936765 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c0722d0 con 0x7fb48c0ccca0 2024-04-14T08:41:48.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.525+0000 7fb48affd640 1 -- 172.21.15.172:0/1023936765 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 365068591 0 0) 0x7fb48c0ceee0 con 0x7fb48c073090 2024-04-14T08:41:48.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.526+0000 7fb48bfff640 1 --2- 172.21.15.172:0/1023936765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c0ccca0 0x7fb48c06fb70 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009870 tx=0x7fb47c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916a8fe710732e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.526+0000 7fb48affd640 1 -- 172.21.15.172:0/1023936765 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c035020 con 0x7fb48c0ccca0 2024-04-14T08:41:48.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.526+0000 7fb48affd640 1 -- 172.21.15.172:0/1023936765 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb47c031640 con 0x7fb48c0ccca0 2024-04-14T08:41:48.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.526+0000 7fb48affd640 1 -- 172.21.15.172:0/1023936765 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c02d040 con 0x7fb48c0ccca0 2024-04-14T08:41:48.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.527+0000 7fb49257b640 1 -- 172.21.15.172:0/1023936765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c0ccca0 msgr2=0x7fb48c06fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.527+0000 7fb49257b640 1 --2- 172.21.15.172:0/1023936765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c0ccca0 0x7fb48c06fb70 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009870 tx=0x7fb47c02b8f0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.527+0000 7fb49257b640 1 -- 172.21.15.172:0/1023936765 shutdown_connections 2024-04-14T08:41:48.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.527+0000 7fb49257b640 1 --2- 172.21.15.172:0/1023936765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb48c06bc30 0x7fb48c0b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.527+0000 7fb49257b640 1 --2- 172.21.15.172:0/1023936765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c0ccca0 0x7fb48c06fb70 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.527+0000 7fb49257b640 1 -- 172.21.15.172:0/1023936765 >> 172.21.15.172:0/1023936765 conn(0x7fb48c0cda00 msgr2=0x7fb48c0cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.527+0000 7fb49257b640 1 -- 172.21.15.172:0/1023936765 shutdown_connections 2024-04-14T08:41:48.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.527+0000 7fb49257b640 1 -- 172.21.15.172:0/1023936765 wait complete. 2024-04-14T08:41:48.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.528+0000 7fb49257b640 1 Processor -- start 2024-04-14T08:41:48.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.528+0000 7fb49257b640 1 -- start start 2024-04-14T08:41:48.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb49257b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c06bc30 0x7fb48c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb49257b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb48c073090 0x7fb48c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb49257b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb48c0ccca0 0x7fb48c0af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb49257b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c0cd080 con 0x7fb48c06bc30 2024-04-14T08:41:48.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb49257b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c072890 con 0x7fb48c0ccca0 2024-04-14T08:41:48.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb49257b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb48c06e150 con 0x7fb48c073090 2024-04-14T08:41:48.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb48bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c06bc30 0x7fb48c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb48bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c06bc30 0x7fb48c15bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56682/0 (socket says 172.21.15.172:56682) 2024-04-14T08:41:48.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb48bfff640 1 -- 172.21.15.172:0/3584390298 learned_addr learned my addr 172.21.15.172:0/3584390298 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb48b7fe640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb48c073090 0x7fb48c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb490b18640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb48c0ccca0 0x7fb48c0af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb48bfff640 1 -- 172.21.15.172:0/3584390298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb48c073090 msgr2=0x7fb48c0aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb48bfff640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb48c073090 0x7fb48c0aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb48bfff640 1 -- 172.21.15.172:0/3584390298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb48c0ccca0 msgr2=0x7fb48c0af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb48bfff640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb48c0ccca0 0x7fb48c0af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb48bfff640 1 -- 172.21.15.172:0/3584390298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c15c530 con 0x7fb48c06bc30 2024-04-14T08:41:48.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb490b18640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb48c0ccca0 0x7fb48c0af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:41:48.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb48bfff640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c06bc30 0x7fb48c15bff0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fb47c02b930 tx=0x7fb47c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb488ff9640 1 -- 172.21.15.172:0/3584390298 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c040070 con 0x7fb48c06bc30 2024-04-14T08:41:48.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb488ff9640 1 -- 172.21.15.172:0/3584390298 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb47c03e070 con 0x7fb48c06bc30 2024-04-14T08:41:48.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb488ff9640 1 -- 172.21.15.172:0/3584390298 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c035040 con 0x7fb48c06bc30 2024-04-14T08:41:48.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.529+0000 7fb49257b640 1 -- 172.21.15.172:0/3584390298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48c0b00e0 con 0x7fb48c06bc30 2024-04-14T08:41:48.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.530+0000 7fb49257b640 1 -- 172.21.15.172:0/3584390298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48c0ae130 con 0x7fb48c06bc30 2024-04-14T08:41:48.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.530+0000 7fb49257b640 1 -- 172.21.15.172:0/3584390298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb48c068590 con 0x7fb48c06bc30 2024-04-14T08:41:48.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.531+0000 7fb488ff9640 1 -- 172.21.15.172:0/3584390298 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb47c038020 con 0x7fb48c06bc30 2024-04-14T08:41:48.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.531+0000 7fb488ff9640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb47806c8f0 0x7fb47806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.531+0000 7fb488ff9640 1 -- 172.21.15.172:0/3584390298 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6130+0+0 (secure 0 0 0) 0x7fb47c0ad5d0 con 0x7fb48c06bc30 2024-04-14T08:41:48.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.531+0000 7fb48b7fe640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb47806c8f0 0x7fb47806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.532+0000 7fb48b7fe640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb47806c8f0 0x7fb47806edb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb48c161950 tx=0x7fb480009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.536+0000 7fb488ff9640 1 -- 172.21.15.172:0/3584390298 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb47c07b7c0 con 0x7fb48c06bc30 2024-04-14T08:41:48.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.746+0000 7fb49257b640 1 -- 172.21.15.172:0/3584390298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb48c0c4610 con 0x7fb48c06bc30 2024-04-14T08:41:48.748 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:41:48.747+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:41:48.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.748+0000 7fb488ff9640 1 -- 172.21.15.172:0/3584390298 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 161+0+0 (secure 0 0 0) 0x7fb47c07b0c0 con 0x7fb48c06bc30 2024-04-14T08:41:48.750 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:41:48.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.751+0000 7fb49257b640 1 -- 172.21.15.172:0/3584390298 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb47806c8f0 msgr2=0x7fb47806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.751+0000 7fb49257b640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb47806c8f0 0x7fb47806edb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb48c161950 tx=0x7fb480009040 comp rx=0 tx=0).stop 2024-04-14T08:41:48.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.751+0000 7fb49257b640 1 -- 172.21.15.172:0/3584390298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c06bc30 msgr2=0x7fb48c15bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.751+0000 7fb49257b640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c06bc30 0x7fb48c15bff0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fb47c02b930 tx=0x7fb47c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.751+0000 7fb49257b640 1 -- 172.21.15.172:0/3584390298 shutdown_connections 2024-04-14T08:41:48.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.751+0000 7fb49257b640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb47806c8f0 0x7fb47806edb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.751+0000 7fb49257b640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb48c0ccca0 0x7fb48c0af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.751+0000 7fb49257b640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb48c073090 0x7fb48c0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.751+0000 7fb49257b640 1 --2- 172.21.15.172:0/3584390298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb48c06bc30 0x7fb48c15bff0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.751+0000 7fb49257b640 1 -- 172.21.15.172:0/3584390298 >> 172.21.15.172:0/3584390298 conn(0x7fb48c0cda00 msgr2=0x7fb48c0b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.752+0000 7fb49257b640 1 -- 172.21.15.172:0/3584390298 shutdown_connections 2024-04-14T08:41:48.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.752+0000 7fb49257b640 1 -- 172.21.15.172:0/3584390298 wait complete. 2024-04-14T08:41:48.755 INFO:tasks.ceph.mds.b.smithi177.stderr: -7172> 2024-04-14T08:41:07.116+0000 7fbc777f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:41:48.764 DEBUG:teuthology.orchestra.run.smithi172:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-14T08:41:48.786 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-04-14T08:41:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.804+0000 7ffb47160280 1 -- start start 2024-04-14T08:41:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.804+0000 7ffb47160280 1 Processor -- start 2024-04-14T08:41:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb47160280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe8750c00 0x564fe88be000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb47160280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe8751800 0x564fe88be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb47160280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe8751c00 0x564fe88beb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb47160280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564fe9520180 con 0x564fe8751c00 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb47160280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564fe9520300 con 0x564fe8750c00 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb47160280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x564fe9520480 con 0x564fe8751800 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb45dfd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe8751c00 0x564fe88beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb45dfd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe8751c00 0x564fe88beb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56698/0 (socket says 172.21.15.172:56698) 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb45dfd640 1 -- 172.21.15.172:0/3420242424 learned_addr learned my addr 172.21.15.172:0/3420242424 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb44dfb640 1 --2- 172.21.15.172:0/3420242424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe8751800 0x564fe88be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb455fc640 1 --2- 172.21.15.172:0/3420242424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe8750c00 0x564fe88be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb45dfd640 1 -- 172.21.15.172:0/3420242424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe8751800 msgr2=0x564fe88be580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb45dfd640 1 --2- 172.21.15.172:0/3420242424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe8751800 0x564fe88be580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb45dfd640 1 -- 172.21.15.172:0/3420242424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe8750c00 msgr2=0x564fe88be000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb45dfd640 1 --2- 172.21.15.172:0/3420242424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe8750c00 0x564fe88be000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.805+0000 7ffb45dfd640 1 -- 172.21.15.172:0/3420242424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe8788780 con 0x564fe8751c00 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.806+0000 7ffb45dfd640 1 --2- 172.21.15.172:0/3420242424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe8751c00 0x564fe88beb00 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x564fe9567380 tx=0x564fe94e75c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4dd6644d36ccdb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.806+0000 7ffb42df7640 1 -- 172.21.15.172:0/3420242424 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564fe87864e0 con 0x564fe8751c00 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.806+0000 7ffb42df7640 1 -- 172.21.15.172:0/3420242424 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564fe877cfc0 con 0x564fe8751c00 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.806+0000 7ffb42df7640 1 -- 172.21.15.172:0/3420242424 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564fe8786b60 con 0x564fe8751c00 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.806+0000 7ffb47160280 1 -- 172.21.15.172:0/3420242424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe87894a0 con 0x564fe8751c00 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.806+0000 7ffb47160280 1 -- 172.21.15.172:0/3420242424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x564fe8788000 con 0x564fe8751c00 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.806+0000 7ffb47160280 4 waiting for MDS map... 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.806+0000 7ffb42df7640 1 -- 172.21.15.172:0/3420242424 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x564fe88c6600 con 0x564fe8751c00 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.806+0000 7ffb42df7640 1 -- 172.21.15.172:0/3420242424 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 75) ==== 2885+0+0 (secure 0 0 0) 0x564fe87ad180 con 0x564fe8751c00 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.806+0000 7ffb47160280 4 Got MDS map 75 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.806+0000 7ffb47160280 10 main: JournalTool::main 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.806+0000 7ffb47160280 4 main: JournalTool: connecting to RADOS... 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb47160280 1 Processor -- start 2024-04-14T08:41:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb47160280 1 -- start start 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb47160280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe9607400 0x564fe88bf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb47160280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607800 0x564fe88bf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb47160280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe9607c00 0x564fe88bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb47160280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564fe9520600 con 0x564fe9607800 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb47160280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564fe9520780 con 0x564fe9607c00 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb47160280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x564fe9520900 con 0x564fe9607400 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb455fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607800 0x564fe88bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb455fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607800 0x564fe88bf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56702/0 (socket says 172.21.15.172:56702) 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb455fc640 1 -- 172.21.15.172:0/1608938046 learned_addr learned my addr 172.21.15.172:0/1608938046 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb44dfb640 1 --2- 172.21.15.172:0/1608938046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe9607400 0x564fe88bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.807+0000 7ffb44dfb640 1 --2- 172.21.15.172:0/1608938046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe9607c00 0x564fe88bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb455fc640 1 -- 172.21.15.172:0/1608938046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe9607400 msgr2=0x564fe88bf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb455fc640 1 --2- 172.21.15.172:0/1608938046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe9607400 0x564fe88bf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb455fc640 1 -- 172.21.15.172:0/1608938046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe9607c00 msgr2=0x564fe88bfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb455fc640 1 --2- 172.21.15.172:0/1608938046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe9607c00 0x564fe88bfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb455fc640 1 -- 172.21.15.172:0/1608938046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe87883c0 con 0x564fe9607800 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb455fc640 1 --2- 172.21.15.172:0/1608938046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607800 0x564fe88bf600 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x564fe962c690 tx=0x564fe962baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59bf1b1cbc4baee server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb40df3640 1 -- 172.21.15.172:0/1608938046 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564fe87864e0 con 0x564fe9607800 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb40df3640 1 -- 172.21.15.172:0/1608938046 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564fe877ca80 con 0x564fe9607800 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb40df3640 1 -- 172.21.15.172:0/1608938046 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564fe8786b60 con 0x564fe9607800 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb47160280 1 -- 172.21.15.172:0/1608938046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607800 msgr2=0x564fe88bf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb47160280 1 --2- 172.21.15.172:0/1608938046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607800 0x564fe88bf600 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x564fe962c690 tx=0x564fe962baa0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb47160280 1 -- 172.21.15.172:0/1608938046 shutdown_connections 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb47160280 1 --2- 172.21.15.172:0/1608938046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe9607c00 0x564fe88bfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb47160280 1 --2- 172.21.15.172:0/1608938046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607800 0x564fe88bf600 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb47160280 1 --2- 172.21.15.172:0/1608938046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe9607400 0x564fe88bf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb47160280 1 -- 172.21.15.172:0/1608938046 >> 172.21.15.172:0/1608938046 conn(0x564fe9607000 msgr2=0x564fe8851000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb47160280 1 -- 172.21.15.172:0/1608938046 shutdown_connections 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.808+0000 7ffb47160280 1 -- 172.21.15.172:0/1608938046 wait complete. 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.809+0000 7ffb47160280 1 Processor -- start 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.809+0000 7ffb47160280 1 -- start start 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.809+0000 7ffb47160280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe9607400 0x564fe88bf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.809+0000 7ffb47160280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe9607800 0x564fe88bf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.809+0000 7ffb47160280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607c00 0x564fe88bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.809+0000 7ffb47160280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564fe9521b00 con 0x564fe9607c00 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.809+0000 7ffb47160280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564fe9521980 con 0x564fe9607800 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.809+0000 7ffb47160280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x564fe9521800 con 0x564fe9607400 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb44dfb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe9607400 0x564fe88bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb44dfb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607c00 0x564fe88bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb455fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe9607800 0x564fe88bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb44dfb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607c00 0x564fe88bfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56712/0 (socket says 172.21.15.172:56712) 2024-04-14T08:41:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb44dfb640 1 -- 172.21.15.172:0/2851898338 learned_addr learned my addr 172.21.15.172:0/2851898338 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb44dfb640 1 -- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe9607400 msgr2=0x564fe88bf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb44dfb640 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe9607400 0x564fe88bf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb44dfb640 1 -- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe9607800 msgr2=0x564fe88bf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb44dfb640 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe9607800 0x564fe88bf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb44dfb640 1 -- 172.21.15.172:0/2851898338 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe8788780 con 0x564fe9607c00 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb44dfb640 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607c00 0x564fe88bfb80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x564fe9566ae0 tx=0x564fe94e7200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb3f5f0640 1 -- 172.21.15.172:0/2851898338 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564fe87869c0 con 0x564fe9607c00 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb3f5f0640 1 -- 172.21.15.172:0/2851898338 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564fe877ce00 con 0x564fe9607c00 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564fe87894a0 con 0x564fe9607c00 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb3f5f0640 1 -- 172.21.15.172:0/2851898338 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564fe87864e0 con 0x564fe9607c00 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe8788960 con 0x564fe9607c00 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.810+0000 7ffb47160280 4 main: JournalTool: resolving pool 15 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.811+0000 7ffb3f5f0640 1 -- 172.21.15.172:0/2851898338 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x564fe9602c00 con 0x564fe9607c00 2024-04-14T08:41:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.812+0000 7ffb3f5f0640 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x564fe960ec00 0x564fe88c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.812+0000 7ffb3f5f0640 1 -- 172.21.15.172:0/2851898338 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x564fe9610a00 con 0x564fe9607c00 2024-04-14T08:41:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.812+0000 7ffb455fc640 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x564fe960ec00 0x564fe88c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.812+0000 7ffb47160280 4 main: JournalTool: creating IoCtx.. 2024-04-14T08:41:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.812+0000 7ffb47160280 4 main: Executing for rank 0 2024-04-14T08:41:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.812+0000 7ffb47160280 4 Filtering by inode '1099511627777' 2024-04-14T08:41:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.812+0000 7ffb47160280 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x564fe960f000 0x564fe88c0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.812+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104) -- 0x564fe960f400 con 0x564fe960f000 2024-04-14T08:41:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.812+0000 7ffb455fc640 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x564fe960ec00 0x564fe88c0100 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x564fe962c690 tx=0x564fe94e7500 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.812+0000 7ffb44dfb640 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x564fe960f000 0x564fe88c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.812+0000 7ffb44dfb640 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x564fe960f000 0x564fe88c0680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.814+0000 7ffb44dfb640 1 -- 172.21.15.172:0/2851898338 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x564fe8844000 con 0x564fe960f000 2024-04-14T08:41:48.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.814+0000 7ffb47160280 4 Pointer 400.00000000 is readable 2024-04-14T08:41:48.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.814+0000 7ffb47160280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-14T08:41:48.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.814+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104) -- 0x564fe960f800 con 0x564fe960f000 2024-04-14T08:41:48.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb44dfb640 1 -- 172.21.15.172:0/2851898338 <== osd.6 v2:172.21.15.177:6800/2961907928 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x564fe8844000 con 0x564fe960f000 2024-04-14T08:41:48.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb47160280 10 Header 0x400000 0x407d21 0x40809d 2024-04-14T08:41:48.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb47160280 10 Starting journal scan from offset 0x407d21 2024-04-14T08:41:48.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104) -- 0x564fe960fc00 con 0x564fe960f000 2024-04-14T08:41:48.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb44dfb640 1 -- 172.21.15.172:0/2851898338 <== osd.6 v2:172.21.15.177:6800/2961907928 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x564fe8844000 con 0x564fe960f000 2024-04-14T08:41:48.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb47160280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-14T08:41:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb47160280 10 Parsing data, 0x37c bytes available 2024-04-14T08:41:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb47160280 10 Attempting decode at 0x407d21 2024-04-14T08:41:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb47160280 10 Consumed 0x37c bytes 2024-04-14T08:41:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb47160280 10 Valid entry at 0x407d21 2024-04-14T08:41:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb47160280 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x564fe8a54000 0x564fe88c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104) -- 0x564fe8a54400 con 0x564fe8a54000 2024-04-14T08:41:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.815+0000 7ffb45dfd640 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x564fe8a54000 0x564fe88c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb45dfd640 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x564fe8a54000 0x564fe88c0c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb45dfd640 1 -- 172.21.15.172:0/2851898338 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564fe8844500 con 0x564fe8a54000 2024-04-14T08:41:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 4 Reached end of journal objects 2024-04-14T08:41:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 4 Scanned objects, 0 missing, 1 valid 2024-04-14T08:41:48.824 INFO:teuthology.orchestra.run.smithi172.stdout:Events by type: 2024-04-14T08:41:48.825 INFO:teuthology.orchestra.run.smithi172.stdout:Errors: 0 2024-04-14T08:41:48.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 4 Events scanned, 0 gaps 2024-04-14T08:41:48.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 4 Found 1 valid events 2024-04-14T08:41:48.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 4 Selected 0 events events for processing 2024-04-14T08:41:48.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 4 0 events 2024-04-14T08:41:48.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x564fe960f000 msgr2=0x564fe88c0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x564fe960f000 0x564fe88c0680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x564fe8a54000 msgr2=0x564fe88c0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x564fe8a54000 0x564fe88c0c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x564fe960ec00 msgr2=0x564fe88c0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x564fe960ec00 0x564fe88c0100 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x564fe962c690 tx=0x564fe94e7500 comp rx=0 tx=0).stop 2024-04-14T08:41:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607c00 msgr2=0x564fe88bfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 --2- 172.21.15.172:0/2851898338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe9607c00 0x564fe88bfb80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x564fe9566ae0 tx=0x564fe94e7200 comp rx=0 tx=0).stop 2024-04-14T08:41:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb455fc640 1 -- 172.21.15.172:0/2851898338 reap_dead start 2024-04-14T08:41:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb455fc640 1 -- 172.21.15.172:0/2851898338 reap_dead start 2024-04-14T08:41:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 shutdown_connections 2024-04-14T08:41:48.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 >> 172.21.15.172:0/2851898338 conn(0x564fe9607000 msgr2=0x564fe8851000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 shutdown_connections 2024-04-14T08:41:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 -- 172.21.15.172:0/2851898338 wait complete. 2024-04-14T08:41:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.816+0000 7ffb47160280 1 -- 172.21.15.172:0/3420242424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe8751c00 msgr2=0x564fe88beb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.817+0000 7ffb47160280 1 --2- 172.21.15.172:0/3420242424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe8751c00 0x564fe88beb00 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x564fe9567380 tx=0x564fe94e75c0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.817+0000 7ffb47160280 1 -- 172.21.15.172:0/3420242424 shutdown_connections 2024-04-14T08:41:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.817+0000 7ffb47160280 1 --2- 172.21.15.172:0/3420242424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564fe8751c00 0x564fe88beb00 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.817+0000 7ffb47160280 1 --2- 172.21.15.172:0/3420242424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x564fe8751800 0x564fe88be580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.817+0000 7ffb47160280 1 --2- 172.21.15.172:0/3420242424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564fe8750c00 0x564fe88be000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.817+0000 7ffb47160280 1 -- 172.21.15.172:0/3420242424 >> 172.21.15.172:0/3420242424 conn(0x564fe8751000 msgr2=0x564fe8850800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.817+0000 7ffb47160280 1 -- 172.21.15.172:0/3420242424 shutdown_connections 2024-04-14T08:41:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.817+0000 7ffb47160280 1 -- 172.21.15.172:0/3420242424 wait complete. 2024-04-14T08:41:48.830 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.065552, result: Events by type: Errors: 0 2024-04-14T08:41:48.830 DEBUG:teuthology.orchestra.run.smithi172:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-14T08:41:48.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.908+0000 7f66a6741280 1 -- start start 2024-04-14T08:41:48.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.908+0000 7f66a6741280 1 Processor -- start 2024-04-14T08:41:48.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a6741280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e812cc00 0x5576e8eee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a6741280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e812d800 0x5576e8eee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a6741280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e812dc00 0x5576e8eeeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a6741280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5576e8f04180 con 0x5576e812dc00 2024-04-14T08:41:48.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a6741280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5576e8f04300 con 0x5576e812d800 2024-04-14T08:41:48.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a6741280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5576e8f04480 con 0x5576e812cc00 2024-04-14T08:41:48.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a53fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e812dc00 0x5576e8eeeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a53fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e812dc00 0x5576e8eeeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56736/0 (socket says 172.21.15.172:56736) 2024-04-14T08:41:48.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a53fd640 1 -- 172.21.15.172:0/3384962828 learned_addr learned my addr 172.21.15.172:0/3384962828 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a4bfc640 1 --2- 172.21.15.172:0/3384962828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e812cc00 0x5576e8eee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a43fb640 1 --2- 172.21.15.172:0/3384962828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e812d800 0x5576e8eee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a53fd640 1 -- 172.21.15.172:0/3384962828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e812cc00 msgr2=0x5576e8eee000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a53fd640 1 --2- 172.21.15.172:0/3384962828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e812cc00 0x5576e8eee000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a53fd640 1 -- 172.21.15.172:0/3384962828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e812d800 msgr2=0x5576e8eee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a53fd640 1 --2- 172.21.15.172:0/3384962828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e812d800 0x5576e8eee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a53fd640 1 -- 172.21.15.172:0/3384962828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e8164780 con 0x5576e812dc00 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a53fd640 1 --2- 172.21.15.172:0/3384962828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e812dc00 0x5576e8eeeb00 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x5576e8f4d380 tx=0x5576e8ec22a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df982dbf81820a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a23f7640 1 -- 172.21.15.172:0/3384962828 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576e81624e0 con 0x5576e812dc00 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a23f7640 1 -- 172.21.15.172:0/3384962828 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5576e8158fc0 con 0x5576e812dc00 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a23f7640 1 -- 172.21.15.172:0/3384962828 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576e8162b60 con 0x5576e812dc00 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a6741280 1 -- 172.21.15.172:0/3384962828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e81654a0 con 0x5576e812dc00 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a6741280 1 -- 172.21.15.172:0/3384962828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5576e8164000 con 0x5576e812dc00 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.909+0000 7f66a6741280 4 waiting for MDS map... 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.910+0000 7f66a23f7640 1 -- 172.21.15.172:0/3384962828 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x5576e9006600 con 0x5576e812dc00 2024-04-14T08:41:48.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.910+0000 7f66a23f7640 1 -- 172.21.15.172:0/3384962828 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 75) ==== 2885+0+0 (secure 0 0 0) 0x5576e8189180 con 0x5576e812dc00 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.910+0000 7f66a6741280 4 Got MDS map 75 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.910+0000 7f66a6741280 10 main: JournalTool::main 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.910+0000 7f66a6741280 4 main: JournalTool: connecting to RADOS... 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.910+0000 7f66a6741280 1 Processor -- start 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.910+0000 7f66a6741280 1 -- start start 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a6741280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e8fef400 0x5576e8eef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a6741280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fef800 0x5576e8eef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a6741280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e8fefc00 0x5576e8eefb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a6741280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5576e8f04600 con 0x5576e8fef800 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a6741280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5576e8f04780 con 0x5576e8fef400 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a6741280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5576e8f04900 con 0x5576e8fefc00 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a4bfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fef800 0x5576e8eef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a4bfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fef800 0x5576e8eef600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56748/0 (socket says 172.21.15.172:56748) 2024-04-14T08:41:48.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a43fb640 1 --2- 172.21.15.172:0/2056199062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e8fef400 0x5576e8eef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a4bfc640 1 -- 172.21.15.172:0/2056199062 learned_addr learned my addr 172.21.15.172:0/2056199062 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a43fb640 1 --2- 172.21.15.172:0/2056199062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e8fefc00 0x5576e8eefb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a4bfc640 1 -- 172.21.15.172:0/2056199062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e8fefc00 msgr2=0x5576e8eefb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a4bfc640 1 --2- 172.21.15.172:0/2056199062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e8fefc00 0x5576e8eefb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a4bfc640 1 -- 172.21.15.172:0/2056199062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e8fef400 msgr2=0x5576e8eef080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:48.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a4bfc640 1 --2- 172.21.15.172:0/2056199062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e8fef400 0x5576e8eef080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a4bfc640 1 -- 172.21.15.172:0/2056199062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e81643c0 con 0x5576e8fef800 2024-04-14T08:41:48.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.911+0000 7f66a4bfc640 1 --2- 172.21.15.172:0/2056199062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fef800 0x5576e8eef600 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x5576e8f4cab0 tx=0x5576e8ec05a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8603f43a43209bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.912+0000 7f66a03f3640 1 -- 172.21.15.172:0/2056199062 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576e81629c0 con 0x5576e8fef800 2024-04-14T08:41:48.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.912+0000 7f66a03f3640 1 -- 172.21.15.172:0/2056199062 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5576e8158e00 con 0x5576e8fef800 2024-04-14T08:41:48.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.912+0000 7f66a03f3640 1 -- 172.21.15.172:0/2056199062 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576e81624e0 con 0x5576e8fef800 2024-04-14T08:41:48.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.912+0000 7f66a6741280 1 -- 172.21.15.172:0/2056199062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fef800 msgr2=0x5576e8eef600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.912+0000 7f66a6741280 1 --2- 172.21.15.172:0/2056199062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fef800 0x5576e8eef600 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x5576e8f4cab0 tx=0x5576e8ec05a0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.912+0000 7f66a6741280 1 -- 172.21.15.172:0/2056199062 shutdown_connections 2024-04-14T08:41:48.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.912+0000 7f66a6741280 1 --2- 172.21.15.172:0/2056199062 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e8fefc00 0x5576e8eefb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.912+0000 7f66a6741280 1 --2- 172.21.15.172:0/2056199062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fef800 0x5576e8eef600 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.912+0000 7f66a6741280 1 --2- 172.21.15.172:0/2056199062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e8fef400 0x5576e8eef080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.912+0000 7f66a6741280 1 -- 172.21.15.172:0/2056199062 >> 172.21.15.172:0/2056199062 conn(0x5576e8fef000 msgr2=0x5576e8219000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.912+0000 7f66a6741280 1 -- 172.21.15.172:0/2056199062 shutdown_connections 2024-04-14T08:41:48.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.912+0000 7f66a6741280 1 -- 172.21.15.172:0/2056199062 wait complete. 2024-04-14T08:41:48.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a6741280 1 Processor -- start 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a6741280 1 -- start start 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a6741280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e8fef400 0x5576e8eef080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a6741280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e8fef800 0x5576e8eef600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a6741280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fefc00 0x5576e8eefb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a6741280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5576e8f05980 con 0x5576e8fefc00 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a6741280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5576e8f05800 con 0x5576e8fef400 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a6741280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5576e8f05680 con 0x5576e8fef800 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a43fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fefc00 0x5576e8eefb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a4bfc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e8fef800 0x5576e8eef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a43fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fefc00 0x5576e8eefb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56752/0 (socket says 172.21.15.172:56752) 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a43fb640 1 -- 172.21.15.172:0/2493573397 learned_addr learned my addr 172.21.15.172:0/2493573397 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.913+0000 7f66a43fb640 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e8fef400 0x5576e8eef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f66a4bfc640 1 -- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e8fef400 msgr2=0x5576e8eef080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f66a4bfc640 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e8fef400 0x5576e8eef080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f66a4bfc640 1 -- 172.21.15.172:0/2493573397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fefc00 msgr2=0x5576e8eefb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f66a4bfc640 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fefc00 0x5576e8eefb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f66a4bfc640 1 -- 172.21.15.172:0/2493573397 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e8164780 con 0x5576e8fef800 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f66a43fb640 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e8fefc00 0x5576e8eefb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f66a4bfc640 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e8fef800 0x5576e8eef600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x5576e9010780 tx=0x5576e8ec0b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f669ebf0640 1 -- 172.21.15.172:0/2493573397 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576e8162b60 con 0x5576e8fef800 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f669ebf0640 1 -- 172.21.15.172:0/2493573397 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5576e8158c40 con 0x5576e8fef800 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x5576e81654a0 con 0x5576e8fef800 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f669ebf0640 1 -- 172.21.15.172:0/2493573397 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576e8162000 con 0x5576e8fef800 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x5576e8164960 con 0x5576e8fef800 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.914+0000 7f66a6741280 4 main: JournalTool: resolving pool 15 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.915+0000 7f669ebf0640 1 -- 172.21.15.172:0/2493573397 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5576e906cc00 con 0x5576e8fef800 2024-04-14T08:41:48.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.916+0000 7f669ebf0640 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5576e9076400 0x5576e8ef0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.916+0000 7f669ebf0640 1 -- 172.21.15.172:0/2493573397 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x5576e9078600 con 0x5576e8fef800 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.916+0000 7f66a6741280 4 main: JournalTool: creating IoCtx.. 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.916+0000 7f66a6741280 4 main: Executing for rank 0 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.916+0000 7f66a43fb640 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5576e9076400 0x5576e8ef0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.916+0000 7f66a6741280 4 Filtering by inode '1099511627778' 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.916+0000 7f66a6741280 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5576e9076800 0x5576e8ef0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.916+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104) -- 0x5576e9076c00 con 0x5576e9076800 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.916+0000 7f66a43fb640 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5576e9076800 0x5576e8ef0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.916+0000 7f66a43fb640 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5576e9076400 0x5576e8ef0100 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5576e9011410 tx=0x5576e8ec2240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.916+0000 7f66a43fb640 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5576e9076800 0x5576e8ef0680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a43fb640 1 -- 172.21.15.172:0/2493573397 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5576e8eb0000 con 0x5576e9076800 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 4 Pointer 400.00000000 is readable 2024-04-14T08:41:48.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104) -- 0x5576e9077000 con 0x5576e9076800 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a43fb640 1 -- 172.21.15.172:0/2493573397 <== osd.6 v2:172.21.15.177:6800/2961907928 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5576e8eb0000 con 0x5576e9076800 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 10 Header 0x400000 0x407d21 0x40809d 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 10 Starting journal scan from offset 0x407d21 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104) -- 0x5576e9077400 con 0x5576e9076800 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a43fb640 1 -- 172.21.15.172:0/2493573397 <== osd.6 v2:172.21.15.177:6800/2961907928 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x5576e8eb0000 con 0x5576e9076800 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 10 Parsing data, 0x37c bytes available 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 10 Attempting decode at 0x407d21 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 10 Consumed 0x37c bytes 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 10 Valid entry at 0x407d21 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5576e9077800 0x5576e8ef0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104) -- 0x5576e84a6800 con 0x5576e9077800 2024-04-14T08:41:48.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.917+0000 7f66a53fd640 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5576e9077800 0x5576e8ef0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:48.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a53fd640 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5576e9077800 0x5576e8ef0c00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:48.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a53fd640 1 -- 172.21.15.172:0/2493573397 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5576e8eb0500 con 0x5576e9077800 2024-04-14T08:41:48.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 4 Reached end of journal objects 2024-04-14T08:41:48.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 4 Scanned objects, 0 missing, 1 valid 2024-04-14T08:41:48.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 4 Events scanned, 0 gaps 2024-04-14T08:41:48.931 INFO:teuthology.orchestra.run.smithi172.stdout:Events by type: 2024-04-14T08:41:48.931 INFO:teuthology.orchestra.run.smithi172.stdout:Errors: 0 2024-04-14T08:41:48.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 4 Found 1 valid events 2024-04-14T08:41:48.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 4 Selected 0 events events for processing 2024-04-14T08:41:48.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 4 0 events 2024-04-14T08:41:48.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5576e9076800 msgr2=0x5576e8ef0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x5576e9076800 0x5576e8ef0680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5576e9077800 msgr2=0x5576e8ef0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x5576e9077800 0x5576e8ef0c00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5576e9076400 msgr2=0x5576e8ef0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x5576e9076400 0x5576e8ef0100 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5576e9011410 tx=0x5576e8ec2240 comp rx=0 tx=0).stop 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e8fef800 msgr2=0x5576e8eef600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e8fef800 0x5576e8eef600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x5576e9010780 tx=0x5576e8ec0b40 comp rx=0 tx=0).stop 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a4bfc640 1 -- 172.21.15.172:0/2493573397 reap_dead start 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 shutdown_connections 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 --2- 172.21.15.172:0/2493573397 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e8fef800 0x5576e8eef600 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 >> 172.21.15.172:0/2493573397 conn(0x5576e8fef000 msgr2=0x5576e8219000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 shutdown_connections 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/2493573397 wait complete. 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/3384962828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e812dc00 msgr2=0x5576e8eeeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:48.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 --2- 172.21.15.172:0/3384962828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e812dc00 0x5576e8eeeb00 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x5576e8f4d380 tx=0x5576e8ec22a0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/3384962828 shutdown_connections 2024-04-14T08:41:48.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 --2- 172.21.15.172:0/3384962828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576e812dc00 0x5576e8eeeb00 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 --2- 172.21.15.172:0/3384962828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576e812d800 0x5576e8eee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 --2- 172.21.15.172:0/3384962828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5576e812cc00 0x5576e8eee000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:48.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/3384962828 >> 172.21.15.172:0/3384962828 conn(0x5576e812d000 msgr2=0x5576e8218800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:48.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/3384962828 shutdown_connections 2024-04-14T08:41:48.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:48.918+0000 7f66a6741280 1 -- 172.21.15.172:0/3384962828 wait complete. 2024-04-14T08:41:48.936 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.106350, result: Events by type: Errors: 0 2024-04-14T08:41:48.936 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-14T08:41:49.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.017+0000 7fb9d1b32f40 1 Processor -- start 2024-04-14T08:41:49.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.017+0000 7fb9d1b32f40 1 -- start start 2024-04-14T08:41:49.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.017+0000 7fb9d1b32f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d30ca397c0 0x55d30cb17bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.017+0000 7fb9d1b32f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d30ca3a6f0 0x55d30cb0c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.017+0000 7fb9d1b32f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 0x55d30cb0caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.017+0000 7fb9d1b32f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d30cb0f460 con 0x55d30cb0e710 2024-04-14T08:41:49.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.017+0000 7fb9d1b32f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d30c9e13b0 con 0x55d30ca3a6f0 2024-04-14T08:41:49.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.017+0000 7fb9d1b32f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d30cb0f700 con 0x55d30ca397c0 2024-04-14T08:41:49.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.017+0000 7fb9cfba4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 0x55d30cb0caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.017+0000 7fb9cfba4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 0x55d30cb0caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56764/0 (socket says 172.21.15.172:56764) 2024-04-14T08:41:49.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.017+0000 7fb9cfba4640 1 -- 172.21.15.172:0/1857671327 learned_addr learned my addr 172.21.15.172:0/1857671327 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:49.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9ceba2640 1 --2- 172.21.15.172:0/1857671327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d30ca3a6f0 0x55d30cb0c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9cf3a3640 1 --2- 172.21.15.172:0/1857671327 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d30ca397c0 0x55d30cb17bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9cfba4640 1 -- 172.21.15.172:0/1857671327 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d30ca397c0 msgr2=0x55d30cb17bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:49.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9cfba4640 1 --2- 172.21.15.172:0/1857671327 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d30ca397c0 0x55d30cb17bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9cfba4640 1 -- 172.21.15.172:0/1857671327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d30ca3a6f0 msgr2=0x55d30cb0c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:49.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9cfba4640 1 --2- 172.21.15.172:0/1857671327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d30ca3a6f0 0x55d30cb0c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9cfba4640 1 -- 172.21.15.172:0/1857671327 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d30cb0bdd0 con 0x55d30cb0e710 2024-04-14T08:41:49.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9cfba4640 1 --2- 172.21.15.172:0/1857671327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 0x55d30cb0caf0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8044750 tx=0x7fb9c804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395ea9fe065adb91 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:49.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9cdba0640 1 -- 172.21.15.172:0/1857671327 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8052860 con 0x55d30cb0e710 2024-04-14T08:41:49.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9cdba0640 1 -- 172.21.15.172:0/1857671327 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9c8062070 con 0x55d30cb0e710 2024-04-14T08:41:49.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9cdba0640 1 -- 172.21.15.172:0/1857671327 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8052c20 con 0x55d30cb0e710 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/1857671327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 msgr2=0x55d30cb0caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9d1b32f40 1 --2- 172.21.15.172:0/1857671327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 0x55d30cb0caf0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8044750 tx=0x7fb9c804c000 comp rx=0 tx=0).stop 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/1857671327 shutdown_connections 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9d1b32f40 1 --2- 172.21.15.172:0/1857671327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 0x55d30cb0caf0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9d1b32f40 1 --2- 172.21.15.172:0/1857671327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d30ca3a6f0 0x55d30cb0c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9d1b32f40 1 --2- 172.21.15.172:0/1857671327 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d30ca397c0 0x55d30cb17bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.018+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/1857671327 >> 172.21.15.172:0/1857671327 conn(0x55d30ca64c50 msgr2=0x55d30cb08770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.019+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/1857671327 shutdown_connections 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.019+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/1857671327 wait complete. 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.019+0000 7fb9d1b32f40 1 Processor -- start 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.019+0000 7fb9d1b32f40 1 -- start start 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9d1b32f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d30ca397c0 0x55d30cb04a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9d1b32f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d30ca3a6f0 0x55d30caff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9d1b32f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 0x55d30cb0fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9d1b32f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d30cb0d680 con 0x55d30cb0e710 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9d1b32f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d30cb04600 con 0x55d30ca3a6f0 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9d1b32f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d30cb04fd0 con 0x55d30ca397c0 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9cfba4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 0x55d30cb0fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9ceba2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d30ca3a6f0 0x55d30caff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9cf3a3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d30ca397c0 0x55d30cb04a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9cfba4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 0x55d30cb0fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56770/0 (socket says 172.21.15.172:56770) 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9cfba4640 1 -- 172.21.15.172:0/4277433997 learned_addr learned my addr 172.21.15.172:0/4277433997 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9cf3a3640 1 -- 172.21.15.172:0/4277433997 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d30ca3a6f0 msgr2=0x55d30caff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9cf3a3640 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d30ca3a6f0 0x55d30caff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9cf3a3640 1 -- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 msgr2=0x55d30cb0fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9cf3a3640 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 0x55d30cb0fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9cf3a3640 1 -- 172.21.15.172:0/4277433997 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d30cb18180 con 0x55d30ca397c0 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9ceba2640 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d30ca3a6f0 0x55d30caff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9cfba4640 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d30cb0e710 0x55d30cb0fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.020+0000 7fb9cf3a3640 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d30ca397c0 0x55d30cb04a90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009ee0 tx=0x7fb9c0008400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.021+0000 7fb9ccb9e640 1 -- 172.21.15.172:0/4277433997 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0015660 con 0x55d30ca397c0 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.021+0000 7fb9ccb9e640 1 -- 172.21.15.172:0/4277433997 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9c001a070 con 0x55d30ca397c0 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.021+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/4277433997 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d30cb18440 con 0x55d30ca397c0 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.021+0000 7fb9ccb9e640 1 -- 172.21.15.172:0/4277433997 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0015a40 con 0x55d30ca397c0 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.021+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/4277433997 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55d30cb10f50 con 0x55d30ca397c0 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.022+0000 7fb9ccb9e640 1 -- 172.21.15.172:0/4277433997 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9c0013030 con 0x55d30ca397c0 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.022+0000 7fb9ccb9e640 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9b006b2d0 0x7fb9b006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.022+0000 7fb9ccb9e640 1 -- 172.21.15.172:0/4277433997 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7fb9c008a820 con 0x55d30ca397c0 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.022+0000 7fb9ceba2640 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9b006b2d0 0x7fb9b006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.023+0000 7fb9d1b32f40 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55d30cb10430 0x55d30ca67ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.023+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/4277433997 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x55d30cb10810 con 0x55d30cb10430 2024-04-14T08:41:49.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.023+0000 7fb9cfba4640 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55d30cb10430 0x55d30ca67ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.023+0000 7fb9ceba2640 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9b006b2d0 0x7fb9b006d790 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55d30cb0fcb0 tx=0x7fb9b8001930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:49.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.023+0000 7fb9cfba4640 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55d30cb10430 0x55d30ca67ef0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:49.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.026+0000 7fb9cfba4640 1 -- 172.21.15.172:0/4277433997 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v104'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb9c80529a0 con 0x55d30cb10430 2024-04-14T08:41:49.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.026+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55d30cb10430 msgr2=0x55d30ca67ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.026+0000 7fb9d1b32f40 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55d30cb10430 0x55d30ca67ef0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.026+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9b006b2d0 msgr2=0x7fb9b006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.026+0000 7fb9d1b32f40 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb9b006b2d0 0x7fb9b006d790 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55d30cb0fcb0 tx=0x7fb9b8001930 comp rx=0 tx=0).stop 2024-04-14T08:41:49.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.026+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/4277433997 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d30ca397c0 msgr2=0x55d30cb04a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.026+0000 7fb9d1b32f40 1 --2- 172.21.15.172:0/4277433997 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d30ca397c0 0x55d30cb04a90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009ee0 tx=0x7fb9c0008400 comp rx=0 tx=0).stop 2024-04-14T08:41:49.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.026+0000 7fb9cfba4640 1 -- 172.21.15.172:0/4277433997 reap_dead start 2024-04-14T08:41:49.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.026+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/4277433997 shutdown_connections 2024-04-14T08:41:49.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.026+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/4277433997 >> 172.21.15.172:0/4277433997 conn(0x55d30ca64c50 msgr2=0x55d30cb095e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:49.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.027+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/4277433997 shutdown_connections 2024-04-14T08:41:49.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.027+0000 7fb9d1b32f40 1 -- 172.21.15.172:0/4277433997 wait complete. 2024-04-14T08:41:49.034 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-14T08:41:49.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.176+0000 7f745442d640 1 Processor -- start 2024-04-14T08:41:49.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.176+0000 7f745442d640 1 -- start start 2024-04-14T08:41:49.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.177+0000 7f745442d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f744c0cea60 0x7f744c1629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.177+0000 7f745442d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f744c0b3750 con 0x7f744c15e590 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.177+0000 7f745442d640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f744c0cef20 con 0x7f744c0b65a0 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.177+0000 7f745442d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f744c0cf200 con 0x7f744c0cea60 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.177+0000 7f74521c9640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f744c15e590 0x7f744c0b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44210/0 (socket says 172.21.15.172:44210) 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74521c9640 1 -- 172.21.15.172:0/471564851 learned_addr learned my addr 172.21.15.172:0/471564851 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74529ca640 1 --2- 172.21.15.172:0/471564851 >> v2:172.21.15.177:3300/0 conn(0x7f744c0cea60 0x7f744c1629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2990767325 0 0) 0x7f744c0b3750 con 0x7f744c15e590 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7438003520 con 0x7f744c15e590 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3150863525 0 0) 0x7f7438003520 con 0x7f744c15e590 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 >> v2:172.21.15.177:3300/0 conn(0x7f744c0cea60 msgr2=0x7f744c1629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 --2- 172.21.15.172:0/471564851 >> v2:172.21.15.177:3300/0 conn(0x7f744c0cea60 0x7f744c1629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 >> v1:172.21.15.177:6789/0 conn(0x7f744c0b65a0 legacy=0x7f744c0d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744c0cf4e0 con 0x7f744c15e590 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f74480019f0 con 0x7f744c15e590 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74529ca640 1 --2- 172.21.15.172:0/471564851 >> v2:172.21.15.177:3300/0 conn(0x7f744c0cea60 0x7f744c1629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 >> v1:172.21.15.172:6789/0 conn(0x7f744c15e590 legacy=0x7f744c0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 --2- 172.21.15.172:0/471564851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74380039b0 0x7f7438003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 --2- 172.21.15.172:0/471564851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7438004800 0x7f7438004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 --2- 172.21.15.172:0/471564851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74380051c0 0x7f743800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f744c0cf200 con 0x7f74380039b0 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f743800c060 con 0x7f7438004800 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f743800c260 con 0x7f74380051c0 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74521c9640 1 --2- 172.21.15.172:0/471564851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74380039b0 0x7f7438003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74519c8640 1 --2- 172.21.15.172:0/471564851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7438004800 0x7f7438004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74529ca640 1 --2- 172.21.15.172:0/471564851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74380051c0 0x7f743800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74521c9640 1 -- 172.21.15.172:0/471564851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74380051c0 msgr2=0x7f743800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:49.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74521c9640 1 --2- 172.21.15.172:0/471564851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74380051c0 0x7f743800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74521c9640 1 -- 172.21.15.172:0/471564851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7438004800 msgr2=0x7f7438004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74521c9640 1 --2- 172.21.15.172:0/471564851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7438004800 0x7f7438004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74521c9640 1 -- 172.21.15.172:0/471564851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7438003520 con 0x7f74380039b0 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.178+0000 7f74529ca640 1 -- 172.21.15.172:0/471564851 reap_dead start 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.179+0000 7f74521c9640 1 --2- 172.21.15.172:0/471564851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74380039b0 0x7f7438003dd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f744800a850 tx=0x7f7438005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184ac166c693f758 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.179+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744800a4a0 con 0x7f74380039b0 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.179+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7448034650 con 0x7f74380039b0 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.179+0000 7f74511c7640 1 -- 172.21.15.172:0/471564851 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448034a80 con 0x7f74380039b0 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.180+0000 7f745442d640 1 -- 172.21.15.172:0/471564851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74380039b0 msgr2=0x7f7438003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.180+0000 7f745442d640 1 --2- 172.21.15.172:0/471564851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74380039b0 0x7f7438003dd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f744800a850 tx=0x7f7438005830 comp rx=0 tx=0).stop 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.180+0000 7f745442d640 1 -- 172.21.15.172:0/471564851 shutdown_connections 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.180+0000 7f745442d640 1 --2- 172.21.15.172:0/471564851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74380039b0 0x7f7438003dd0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.180+0000 7f745442d640 1 -- 172.21.15.172:0/471564851 >> 172.21.15.172:0/471564851 conn(0x7f744c0d0e90 msgr2=0x7f744c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.180+0000 7f745442d640 1 -- 172.21.15.172:0/471564851 shutdown_connections 2024-04-14T08:41:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.180+0000 7f745442d640 1 -- 172.21.15.172:0/471564851 wait complete. 2024-04-14T08:41:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.181+0000 7f745442d640 1 Processor -- start 2024-04-14T08:41:49.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.181+0000 7f745442d640 1 -- start start 2024-04-14T08:41:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f745442d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74380039b0 0x7f744c150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f745442d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f744c14cc50 0x7f744c0d4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f745442d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f744c0d4950 0x7f744c15f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f745442d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f744c162b40 con 0x7f744c0d4950 2024-04-14T08:41:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f745442d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f744c1629c0 con 0x7f744c14cc50 2024-04-14T08:41:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f745442d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f744c162cc0 con 0x7f74380039b0 2024-04-14T08:41:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f74529ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f744c0d4950 0x7f744c15f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f74529ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f744c0d4950 0x7f744c15f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56802/0 (socket says 172.21.15.172:56802) 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f74529ca640 1 -- 172.21.15.172:0/3430032462 learned_addr learned my addr 172.21.15.172:0/3430032462 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f74521c9640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74380039b0 0x7f744c150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f74519c8640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f744c14cc50 0x7f744c0d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f74529ca640 1 -- 172.21.15.172:0/3430032462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74380039b0 msgr2=0x7f744c150700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f74529ca640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74380039b0 0x7f744c150700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f74529ca640 1 -- 172.21.15.172:0/3430032462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f744c14cc50 msgr2=0x7f744c0d4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.182+0000 7f74529ca640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f744c14cc50 0x7f744c0d4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.183+0000 7f74529ca640 1 -- 172.21.15.172:0/3430032462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744c1606d0 con 0x7f744c0d4950 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.183+0000 7f74529ca640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f744c0d4950 0x7f744c15f9c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f744c06df30 tx=0x7f7444011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.183+0000 7f7442ffd640 1 -- 172.21.15.172:0/3430032462 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444019070 con 0x7f744c0d4950 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.183+0000 7f7442ffd640 1 -- 172.21.15.172:0/3430032462 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f744400f9e0 con 0x7f744c0d4950 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.183+0000 7f745442d640 1 -- 172.21.15.172:0/3430032462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744c1609b0 con 0x7f744c0d4950 2024-04-14T08:41:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.183+0000 7f7442ffd640 1 -- 172.21.15.172:0/3430032462 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744400fdd0 con 0x7f744c0d4950 2024-04-14T08:41:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.183+0000 7f745442d640 1 -- 172.21.15.172:0/3430032462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f744c0aca20 con 0x7f744c0d4950 2024-04-14T08:41:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.184+0000 7f745442d640 1 -- 172.21.15.172:0/3430032462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f744c0acc20 con 0x7f744c0d4950 2024-04-14T08:41:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.184+0000 7f7442ffd640 1 -- 172.21.15.172:0/3430032462 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f744401c070 con 0x7f744c0d4950 2024-04-14T08:41:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.185+0000 7f7442ffd640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f743006c920 0x7f743006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.185+0000 7f7442ffd640 1 -- 172.21.15.172:0/3430032462 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7f744408bd70 con 0x7f744c0d4950 2024-04-14T08:41:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.185+0000 7f74521c9640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f743006c920 0x7f743006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.185+0000 7f74521c9640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f743006c920 0x7f743006ede0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7448034000 tx=0x7f744802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.189+0000 7f7442ffd640 1 -- 172.21.15.172:0/3430032462 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f744c0acc20 con 0x7f744c0d4950 2024-04-14T08:41:49.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.401+0000 7f745442d640 1 -- 172.21.15.172:0/3430032462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f744c068520 con 0x7f744c0d4950 2024-04-14T08:41:49.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.755+0000 7f7442ffd640 1 -- 172.21.15.172:0/3430032462 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v76) ==== 162+0+0 (secure 0 0 0) 0x7f74440599e0 con 0x7f744c0d4950 2024-04-14T08:41:49.757 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-14T08:41:49.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.757+0000 7f745442d640 1 -- 172.21.15.172:0/3430032462 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f743006c920 msgr2=0x7f743006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.757+0000 7f745442d640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f743006c920 0x7f743006ede0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7448034000 tx=0x7f744802f000 comp rx=0 tx=0).stop 2024-04-14T08:41:49.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.757+0000 7f745442d640 1 -- 172.21.15.172:0/3430032462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f744c0d4950 msgr2=0x7f744c15f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.757+0000 7f745442d640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f744c0d4950 0x7f744c15f9c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f744c06df30 tx=0x7f7444011d60 comp rx=0 tx=0).stop 2024-04-14T08:41:49.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.757+0000 7f745442d640 1 -- 172.21.15.172:0/3430032462 shutdown_connections 2024-04-14T08:41:49.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.757+0000 7f745442d640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f743006c920 0x7f743006ede0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.757+0000 7f745442d640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f744c0d4950 0x7f744c15f9c0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.757+0000 7f745442d640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f744c14cc50 0x7f744c0d4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.757+0000 7f745442d640 1 --2- 172.21.15.172:0/3430032462 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74380039b0 0x7f744c150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.757+0000 7f745442d640 1 -- 172.21.15.172:0/3430032462 >> 172.21.15.172:0/3430032462 conn(0x7f744c0d0e90 msgr2=0x7f744c0b5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:49.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.757+0000 7f745442d640 1 -- 172.21.15.172:0/3430032462 shutdown_connections 2024-04-14T08:41:49.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.757+0000 7f745442d640 1 -- 172.21.15.172:0/3430032462 wait complete. 2024-04-14T08:41:49.773 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:41:49.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.873+0000 7f3f470d5640 1 Processor -- start 2024-04-14T08:41:49.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.873+0000 7f3f470d5640 1 -- start start 2024-04-14T08:41:49.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f470d5640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3f400b1df0 0x7f3f400b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f470d5640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3f400b21d0 0x7f3f4015c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f470d5640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f400b3430 con 0x7f3f4015c5a0 2024-04-14T08:41:49.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f470d5640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f3f400b3700 con 0x7f3f400b21d0 2024-04-14T08:41:49.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f470d5640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f3f400b39e0 con 0x7f3f400b1df0 2024-04-14T08:41:49.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f3ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3f400b21d0 0x7f3f4015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f45672640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3f4015c5a0 0x7f3f400b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:57054/0 (socket says 172.21.15.172:57054) 2024-04-14T08:41:49.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f3ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3f400b21d0 0x7f3f4015c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56264/0 (socket says 172.21.15.172:56264) 2024-04-14T08:41:49.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f45672640 1 -- 172.21.15.172:0/922421554 learned_addr learned my addr 172.21.15.172:0/922421554 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:49.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f3ffff640 1 -- 172.21.15.172:0/922421554 >> v2:172.21.15.177:3300/0 conn(0x7f3f400b1df0 msgr2=0x7f3f400b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:49.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f3ffff640 1 --2- 172.21.15.172:0/922421554 >> v2:172.21.15.177:3300/0 conn(0x7f3f400b1df0 0x7f3f400b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f3ffff640 1 -- 172.21.15.172:0/922421554 >> v1:172.21.15.172:6789/0 conn(0x7f3f4015c5a0 legacy=0x7f3f400b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f3f7fe640 1 -- 172.21.15.172:0/922421554 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873036154 0 0) 0x7f3f400b3430 con 0x7f3f4015c5a0 2024-04-14T08:41:49.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.874+0000 7f3f3ffff640 1 -- 172.21.15.172:0/922421554 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f400b3cc0 con 0x7f3f400b21d0 2024-04-14T08:41:49.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.875+0000 7f3f3ffff640 1 --2- 172.21.15.172:0/922421554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f400b21d0 0x7f3f4015c060 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f3f4006f4b0 tx=0x7f3f2802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b80b960321309cb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:49.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.875+0000 7f3f3f7fe640 1 -- 172.21.15.172:0/922421554 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f280315a0 con 0x7f3f400b21d0 2024-04-14T08:41:49.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.875+0000 7f3f3f7fe640 1 -- 172.21.15.172:0/922421554 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f28037070 con 0x7f3f400b21d0 2024-04-14T08:41:49.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.875+0000 7f3f3f7fe640 1 -- 172.21.15.172:0/922421554 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28031960 con 0x7f3f400b21d0 2024-04-14T08:41:49.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.875+0000 7f3f470d5640 1 -- 172.21.15.172:0/922421554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f400b21d0 msgr2=0x7f3f4015c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.875+0000 7f3f470d5640 1 --2- 172.21.15.172:0/922421554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f400b21d0 0x7f3f4015c060 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f3f4006f4b0 tx=0x7f3f2802b840 comp rx=0 tx=0).stop 2024-04-14T08:41:49.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.876+0000 7f3f470d5640 1 -- 172.21.15.172:0/922421554 shutdown_connections 2024-04-14T08:41:49.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.876+0000 7f3f470d5640 1 --2- 172.21.15.172:0/922421554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f400b21d0 0x7f3f4015c060 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.876+0000 7f3f470d5640 1 --2- 172.21.15.172:0/922421554 >> v2:172.21.15.177:3300/0 conn(0x7f3f400b1df0 0x7f3f400b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.876+0000 7f3f470d5640 1 -- 172.21.15.172:0/922421554 >> 172.21.15.172:0/922421554 conn(0x7f3f4006ec30 msgr2=0x7f3f4006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:49.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.876+0000 7f3f470d5640 1 -- 172.21.15.172:0/922421554 shutdown_connections 2024-04-14T08:41:49.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.876+0000 7f3f470d5640 1 -- 172.21.15.172:0/922421554 wait complete. 2024-04-14T08:41:49.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.877+0000 7f3f470d5640 1 Processor -- start 2024-04-14T08:41:49.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.877+0000 7f3f470d5640 1 -- start start 2024-04-14T08:41:49.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f470d5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f400b1df0 0x7f3f400aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f470d5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f400b21d0 0x7f3f401336e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f470d5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f400cf990 0x7f3f400acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f470d5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3f400b4750 con 0x7f3f400b1df0 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f470d5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4015b7f0 con 0x7f3f400b21d0 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f470d5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3f400b48d0 con 0x7f3f400cf990 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f44e71640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f400b1df0 0x7f3f400aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f44e71640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f400b1df0 0x7f3f400aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56274/0 (socket says 172.21.15.172:56274) 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f44e71640 1 -- 172.21.15.172:0/521832682 learned_addr learned my addr 172.21.15.172:0/521832682 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f3ffff640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f400b21d0 0x7f3f401336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f45672640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f400cf990 0x7f3f400acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f44e71640 1 -- 172.21.15.172:0/521832682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f400cf990 msgr2=0x7f3f400acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f44e71640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f400cf990 0x7f3f400acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f44e71640 1 -- 172.21.15.172:0/521832682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f400b21d0 msgr2=0x7f3f401336e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:49.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f44e71640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f400b21d0 0x7f3f401336e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f44e71640 1 -- 172.21.15.172:0/521832682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f40133c20 con 0x7f3f400b1df0 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f44e71640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f400b1df0 0x7f3f400aead0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800ba50 tx=0x7f3f3800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f3d7fa640 1 -- 172.21.15.172:0/521832682 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38019070 con 0x7f3f400b1df0 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f3d7fa640 1 -- 172.21.15.172:0/521832682 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3800cc00 con 0x7f3f400b1df0 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f470d5640 1 -- 172.21.15.172:0/521832682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f400af140 con 0x7f3f400b1df0 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f3d7fa640 1 -- 172.21.15.172:0/521832682 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38014440 con 0x7f3f400b1df0 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.878+0000 7f3f470d5640 1 -- 172.21.15.172:0/521832682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f400ac570 con 0x7f3f400b1df0 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.879+0000 7f3f470d5640 1 -- 172.21.15.172:0/521832682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f40068520 con 0x7f3f400b1df0 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.880+0000 7f3f3d7fa640 1 -- 172.21.15.172:0/521832682 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f3801c070 con 0x7f3f400b1df0 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.880+0000 7f3f3d7fa640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3f2006c8f0 0x7f3f2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.880+0000 7f3f3d7fa640 1 -- 172.21.15.172:0/521832682 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7f3f3808ba50 con 0x7f3f400b1df0 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.880+0000 7f3f3ffff640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3f2006c8f0 0x7f3f2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:49.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.881+0000 7f3f3ffff640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3f2006c8f0 0x7f3f2006edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3f28036000 tx=0x7f3f2803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:49.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:49.885+0000 7f3f3d7fa640 1 -- 172.21.15.172:0/521832682 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3f38059d10 con 0x7f3f400b1df0 2024-04-14T08:41:50.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.095+0000 7f3f470d5640 1 -- 172.21.15.172:0/521832682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f4006b8a0 con 0x7f3f400b1df0 2024-04-14T08:41:50.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.096+0000 7f3f3d7fa640 1 -- 172.21.15.172:0/521832682 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5190 (secure 0 0 0) 0x7f3f38059610 con 0x7f3f400b1df0 2024-04-14T08:41:50.098 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:50.098 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8005,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3289017090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3289017090},{"type":"v1","addr":"172.21.15.177:6835","nonce":3289017090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:41:04.492773+0000","modified":"2024-04-14T08:41:49.758912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8382":{"gid":8382,"name":"a","rank":0,"incarnation":77,"state":"up:replay","state_seq":1,"addr":"172.21.15.172:6837/4028911489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4028911489},{"type":"v1","addr":"172.21.15.172:6837","nonce":4028911489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-14T08:41:50.100 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 77 2024-04-14T08:41:50.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.098+0000 7f3f470d5640 1 -- 172.21.15.172:0/521832682 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3f2006c8f0 msgr2=0x7f3f2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:50.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.098+0000 7f3f470d5640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3f2006c8f0 0x7f3f2006edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3f28036000 tx=0x7f3f2803d040 comp rx=0 tx=0).stop 2024-04-14T08:41:50.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.098+0000 7f3f470d5640 1 -- 172.21.15.172:0/521832682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f400b1df0 msgr2=0x7f3f400aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:50.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.098+0000 7f3f470d5640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f400b1df0 0x7f3f400aead0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800ba50 tx=0x7f3f3800bf10 comp rx=0 tx=0).stop 2024-04-14T08:41:50.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.098+0000 7f3f470d5640 1 -- 172.21.15.172:0/521832682 shutdown_connections 2024-04-14T08:41:50.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.098+0000 7f3f470d5640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3f2006c8f0 0x7f3f2006edb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:50.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.098+0000 7f3f470d5640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f400cf990 0x7f3f400acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:50.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.098+0000 7f3f470d5640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f400b21d0 0x7f3f401336e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:50.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.098+0000 7f3f470d5640 1 --2- 172.21.15.172:0/521832682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f400b1df0 0x7f3f400aead0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:50.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.098+0000 7f3f470d5640 1 -- 172.21.15.172:0/521832682 >> 172.21.15.172:0/521832682 conn(0x7f3f4006ec30 msgr2=0x7f3f40158860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:50.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.098+0000 7f3f470d5640 1 -- 172.21.15.172:0/521832682 shutdown_connections 2024-04-14T08:41:50.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:50.098+0000 7f3f470d5640 1 -- 172.21.15.172:0/521832682 wait complete. 2024-04-14T08:41:50.514 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:41:04.492773+0000', 'modified': '2024-04-14T08:41:49.758912+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 104, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8382}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8382': {'gid': 8382, 'name': 'a', 'rank': 0, 'incarnation': 77, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.172:6837/4028911489', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 4028911489}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 4028911489}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-14T08:41:50.514 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8382:up:replay 2024-04-14T08:41:51.516 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:41:51.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.628+0000 7f09ddbbb640 1 Processor -- start 2024-04-14T08:41:51.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.628+0000 7f09ddbbb640 1 -- start start 2024-04-14T08:41:51.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.628+0000 7f09ddbbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09d80b1df0 0x7f09d80b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:51.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.628+0000 7f09ddbbb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f09d80b21d0 0x7f09d815c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:51.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.628+0000 7f09ddbbb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09d80b3430 con 0x7f09d815c5a0 2024-04-14T08:41:51.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.628+0000 7f09ddbbb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f09d80b3700 con 0x7f09d80b21d0 2024-04-14T08:41:51.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.628+0000 7f09ddbbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f09d80b39e0 con 0x7f09d80b1df0 2024-04-14T08:41:51.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d7fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f09d815c5a0 0x7f09d80b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:57066/0 (socket says 172.21.15.172:57066) 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d7fff640 1 -- 172.21.15.172:0/1826722135 learned_addr learned my addr 172.21.15.172:0/1826722135 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d77fe640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09d80b1df0 0x7f09d80b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d6ffd640 1 --2- 172.21.15.172:0/1826722135 >> v2:172.21.15.177:3300/0 conn(0x7f09d80b21d0 0x7f09d815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3906253016 0 0) 0x7f09d80b3430 con 0x7f09d815c5a0 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09c0003520 con 0x7f09d815c5a0 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3690424052 0 0) 0x7f09c0003520 con 0x7f09d815c5a0 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09d80b1df0 msgr2=0x7f09d80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09d80b1df0 0x7f09d80b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 >> v2:172.21.15.177:3300/0 conn(0x7f09d80b21d0 msgr2=0x7f09d815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 --2- 172.21.15.172:0/1826722135 >> v2:172.21.15.177:3300/0 conn(0x7f09d80b21d0 0x7f09d815c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d80b3cc0 con 0x7f09d815c5a0 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d6ffd640 1 --2- 172.21.15.172:0/1826722135 >> v2:172.21.15.177:3300/0 conn(0x7f09d80b21d0 0x7f09d815c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f09d00019f0 con 0x7f09d815c5a0 2024-04-14T08:41:51.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 >> v1:172.21.15.172:6789/0 conn(0x7f09d815c5a0 legacy=0x7f09d80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09c00039b0 0x7f09c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c0004800 0x7f09c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09d80b1df0 0x7f09c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f09d80b3700 con 0x7f09c00039b0 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f09d80b39e0 con 0x7f09c0004800 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f09c000b9c0 con 0x7f09d80b1df0 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d77fe640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09c00039b0 0x7f09c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.629+0000 7f09d6ffd640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c0004800 0x7f09c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.630+0000 7f09d7fff640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09d80b1df0 0x7f09c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.630+0000 7f09d77fe640 1 -- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09d80b1df0 msgr2=0x7f09c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.630+0000 7f09d77fe640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09d80b1df0 0x7f09c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.630+0000 7f09d77fe640 1 -- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c0004800 msgr2=0x7f09c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.630+0000 7f09d77fe640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c0004800 0x7f09c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.630+0000 7f09d77fe640 1 -- 172.21.15.172:0/1826722135 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c0003520 con 0x7f09c00039b0 2024-04-14T08:41:51.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.630+0000 7f09d7fff640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09d80b1df0 0x7f09c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:41:51.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.630+0000 7f09d6ffd640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c0004800 0x7f09c0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:41:51.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.630+0000 7f09d77fe640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09c00039b0 0x7f09c0003dd0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009ac0 tx=0x7f09c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3423367d1711769 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:51.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.630+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8039070 con 0x7f09c00039b0 2024-04-14T08:41:51.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.631+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c802c6b0 con 0x7f09c00039b0 2024-04-14T08:41:51.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.631+0000 7f09d67fc640 1 -- 172.21.15.172:0/1826722135 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8009690 con 0x7f09c00039b0 2024-04-14T08:41:51.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.631+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1826722135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09c00039b0 msgr2=0x7f09c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:51.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.631+0000 7f09ddbbb640 1 --2- 172.21.15.172:0/1826722135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09c00039b0 0x7f09c0003dd0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009ac0 tx=0x7f09c0005830 comp rx=0 tx=0).stop 2024-04-14T08:41:51.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.631+0000 7f09d7fff640 1 -- 172.21.15.172:0/1826722135 reap_dead start 2024-04-14T08:41:51.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.631+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1826722135 shutdown_connections 2024-04-14T08:41:51.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.631+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1826722135 >> 172.21.15.172:0/1826722135 conn(0x7f09d806ec30 msgr2=0x7f09d806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:51.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.631+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1826722135 shutdown_connections 2024-04-14T08:41:51.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.631+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1826722135 wait complete. 2024-04-14T08:41:51.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.632+0000 7f09ddbbb640 1 Processor -- start 2024-04-14T08:41:51.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.632+0000 7f09ddbbb640 1 -- start start 2024-04-14T08:41:51.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.633+0000 7f09ddbbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09c00039b0 0x7f09d81f2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:51.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.633+0000 7f09ddbbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d80d00a0 0x7f09d8144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:51.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.633+0000 7f09ddbbb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09d80d0520 0x7f09d8144580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:51.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.633+0000 7f09ddbbb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f09d80b4750 con 0x7f09d80d00a0 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.633+0000 7f09ddbbb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f09d815b7f0 con 0x7f09d80d0520 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.633+0000 7f09ddbbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f09d80b48d0 con 0x7f09c00039b0 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.633+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d80d00a0 0x7f09d8144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.633+0000 7f09d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d80d00a0 0x7f09d8144040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56284/0 (socket says 172.21.15.172:56284) 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.633+0000 7f09d6ffd640 1 -- 172.21.15.172:0/1812137006 learned_addr learned my addr 172.21.15.172:0/1812137006 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.633+0000 7f09d77fe640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09c00039b0 0x7f09d81f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.633+0000 7f09d7fff640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09d80d0520 0x7f09d8144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.634+0000 7f09d6ffd640 1 -- 172.21.15.172:0/1812137006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09c00039b0 msgr2=0x7f09d81f2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.634+0000 7f09d6ffd640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09c00039b0 0x7f09d81f2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.634+0000 7f09d6ffd640 1 -- 172.21.15.172:0/1812137006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09d80d0520 msgr2=0x7f09d8144580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.634+0000 7f09d6ffd640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09d80d0520 0x7f09d8144580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.634+0000 7f09d6ffd640 1 -- 172.21.15.172:0/1812137006 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d80abaa0 con 0x7f09d80d00a0 2024-04-14T08:41:51.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.634+0000 7f09d6ffd640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d80d00a0 0x7f09d8144040 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f09cc011860 tx=0x7f09cc011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:51.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.634+0000 7f09bffff640 1 -- 172.21.15.172:0/1812137006 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00fbd0 con 0x7f09d80d00a0 2024-04-14T08:41:51.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.634+0000 7f09bffff640 1 -- 172.21.15.172:0/1812137006 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09cc00fd70 con 0x7f09d80d00a0 2024-04-14T08:41:51.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.634+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1812137006 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d80abd80 con 0x7f09d80d00a0 2024-04-14T08:41:51.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.634+0000 7f09bffff640 1 -- 172.21.15.172:0/1812137006 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc015660 con 0x7f09d80d00a0 2024-04-14T08:41:51.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.634+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1812137006 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d80ac340 con 0x7f09d80d00a0 2024-04-14T08:41:51.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.635+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1812137006 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d8068520 con 0x7f09d80d00a0 2024-04-14T08:41:51.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.636+0000 7f09bffff640 1 -- 172.21.15.172:0/1812137006 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09cc015800 con 0x7f09d80d00a0 2024-04-14T08:41:51.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.636+0000 7f09bffff640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f09c406c8f0 0x7f09c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:51.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.636+0000 7f09bffff640 1 -- 172.21.15.172:0/1812137006 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7f09cc08c7f0 con 0x7f09d80d00a0 2024-04-14T08:41:51.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.636+0000 7f09d77fe640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f09c406c8f0 0x7f09c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:51.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.636+0000 7f09d77fe640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f09c406c8f0 0x7f09c406edb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f09c8036000 tx=0x7f09c8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:51.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.640+0000 7f09bffff640 1 -- 172.21.15.172:0/1812137006 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f09cc019030 con 0x7f09d80d00a0 2024-04-14T08:41:51.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.856+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1812137006 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09d8071ad0 con 0x7f09d80d00a0 2024-04-14T08:41:51.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.856+0000 7f09bffff640 1 -- 172.21.15.172:0/1812137006 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5191 (secure 0 0 0) 0x7f09cc05a3d0 con 0x7f09d80d00a0 2024-04-14T08:41:51.858 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:51.858 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":79,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8005,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3289017090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3289017090},{"type":"v1","addr":"172.21.15.177:6835","nonce":3289017090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:41:04.492773+0000","modified":"2024-04-14T08:41:50.763749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8382":{"gid":8382,"name":"a","rank":0,"incarnation":77,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.172:6837/4028911489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4028911489},{"type":"v1","addr":"172.21.15.172:6837","nonce":4028911489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-14T08:41:51.858 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 79 2024-04-14T08:41:51.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.858+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1812137006 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f09c406c8f0 msgr2=0x7f09c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:51.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.858+0000 7f09ddbbb640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f09c406c8f0 0x7f09c406edb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f09c8036000 tx=0x7f09c8030000 comp rx=0 tx=0).stop 2024-04-14T08:41:51.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.858+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1812137006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d80d00a0 msgr2=0x7f09d8144040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:51.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.858+0000 7f09ddbbb640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d80d00a0 0x7f09d8144040 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f09cc011860 tx=0x7f09cc011d20 comp rx=0 tx=0).stop 2024-04-14T08:41:51.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.858+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1812137006 shutdown_connections 2024-04-14T08:41:51.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.858+0000 7f09ddbbb640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f09c406c8f0 0x7f09c406edb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:51.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.858+0000 7f09ddbbb640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09d80d0520 0x7f09d8144580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:51.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.858+0000 7f09ddbbb640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d80d00a0 0x7f09d8144040 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:51.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.858+0000 7f09ddbbb640 1 --2- 172.21.15.172:0/1812137006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09c00039b0 0x7f09d81f2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:51.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.858+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1812137006 >> 172.21.15.172:0/1812137006 conn(0x7f09d806ec30 msgr2=0x7f09d815d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:51.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.859+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1812137006 shutdown_connections 2024-04-14T08:41:51.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:51.859+0000 7f09ddbbb640 1 -- 172.21.15.172:0/1812137006 wait complete. 2024-04-14T08:41:52.273 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:41:04.492773+0000', 'modified': '2024-04-14T08:41:50.763749+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 104, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8382}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8382': {'gid': 8382, 'name': 'a', 'rank': 0, 'incarnation': 77, 'state': 'up:rejoin', 'state_seq': 15, 'addr': '172.21.15.172:6837/4028911489', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 4028911489}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 4028911489}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-14T08:41:52.274 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8382:up:rejoin 2024-04-14T08:41:53.274 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:41:53.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.376+0000 7fb84fede640 1 Processor -- start 2024-04-14T08:41:53.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.376+0000 7fb84fede640 1 -- start start 2024-04-14T08:41:53.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.377+0000 7fb84fede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8480cefd0 0x7fb84806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:53.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.377+0000 7fb84fede640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb84808c4c0 0x7fb84806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:53.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.377+0000 7fb84fede640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8480d2b10 con 0x7fb8480b7a60 2024-04-14T08:41:53.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.377+0000 7fb84fede640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb8480ce420 con 0x7fb84808c4c0 2024-04-14T08:41:53.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.377+0000 7fb84fede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb8480d0ad0 con 0x7fb8480cefd0 2024-04-14T08:41:53.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.377+0000 7fb84dc7a640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb8480b7a60 0x7fb8480ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:57078/0 (socket says 172.21.15.172:57078) 2024-04-14T08:41:53.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.377+0000 7fb84dc7a640 1 -- 172.21.15.172:0/573799215 learned_addr learned my addr 172.21.15.172:0/573799215 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:53.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.377+0000 7fb84d479640 1 --2- 172.21.15.172:0/573799215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8480cefd0 0x7fb84806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:53.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.377+0000 7fb84e47b640 1 --2- 172.21.15.172:0/573799215 >> v2:172.21.15.177:3300/0 conn(0x7fb84808c4c0 0x7fb84806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:53.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.377+0000 7fb84cc78640 1 -- 172.21.15.172:0/573799215 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1107842230 0 0) 0x7fb8480d2b10 con 0x7fb8480b7a60 2024-04-14T08:41:53.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.378+0000 7fb84cc78640 1 -- 172.21.15.172:0/573799215 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb830003520 con 0x7fb8480b7a60 2024-04-14T08:41:53.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.378+0000 7fb84e47b640 1 -- 172.21.15.172:0/573799215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8480cefd0 msgr2=0x7fb84806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:53.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.378+0000 7fb84e47b640 1 --2- 172.21.15.172:0/573799215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8480cefd0 0x7fb84806cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:53.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.378+0000 7fb84e47b640 1 -- 172.21.15.172:0/573799215 >> v1:172.21.15.172:6789/0 conn(0x7fb8480b7a60 legacy=0x7fb8480ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:53.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.378+0000 7fb84e47b640 1 -- 172.21.15.172:0/573799215 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8480d0db0 con 0x7fb84808c4c0 2024-04-14T08:41:53.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.378+0000 7fb84d479640 1 --2- 172.21.15.172:0/573799215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8480cefd0 0x7fb84806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:53.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.379+0000 7fb84e47b640 1 --2- 172.21.15.172:0/573799215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb84808c4c0 0x7fb84806d170 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb840009d90 tx=0x7fb84002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b70d3766ad48a9e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:53.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.379+0000 7fb84cc78640 1 -- 172.21.15.172:0/573799215 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840039070 con 0x7fb84808c4c0 2024-04-14T08:41:53.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.379+0000 7fb84cc78640 1 -- 172.21.15.172:0/573799215 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84002cbc0 con 0x7fb84808c4c0 2024-04-14T08:41:53.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.379+0000 7fb84cc78640 1 -- 172.21.15.172:0/573799215 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840034440 con 0x7fb84808c4c0 2024-04-14T08:41:53.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.380+0000 7fb84fede640 1 -- 172.21.15.172:0/573799215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb84808c4c0 msgr2=0x7fb84806d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:53.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.380+0000 7fb84fede640 1 --2- 172.21.15.172:0/573799215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb84808c4c0 0x7fb84806d170 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb840009d90 tx=0x7fb84002be10 comp rx=0 tx=0).stop 2024-04-14T08:41:53.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.380+0000 7fb84fede640 1 -- 172.21.15.172:0/573799215 shutdown_connections 2024-04-14T08:41:53.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.380+0000 7fb84fede640 1 --2- 172.21.15.172:0/573799215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb84808c4c0 0x7fb84806d170 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:53.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.380+0000 7fb84fede640 1 --2- 172.21.15.172:0/573799215 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8480cefd0 0x7fb84806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:53.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.380+0000 7fb84fede640 1 -- 172.21.15.172:0/573799215 >> 172.21.15.172:0/573799215 conn(0x7fb84808c000 msgr2=0x7fb8480d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:53.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.380+0000 7fb84fede640 1 -- 172.21.15.172:0/573799215 shutdown_connections 2024-04-14T08:41:53.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.380+0000 7fb84fede640 1 -- 172.21.15.172:0/573799215 wait complete. 2024-04-14T08:41:53.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.381+0000 7fb84fede640 1 Processor -- start 2024-04-14T08:41:53.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.381+0000 7fb84fede640 1 -- start start 2024-04-14T08:41:53.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84fede640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb84808c4c0 0x7fb848157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:53.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84fede640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8480b7a60 0x7fb8480c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84fede640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8480cefd0 0x7fb8480c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84fede640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8480b7e40 con 0x7fb84808c4c0 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84fede640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb8480b4e90 con 0x7fb8480cefd0 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84fede640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb8480d2750 con 0x7fb8480b7a60 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84dc7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb84808c4c0 0x7fb848157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84dc7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb84808c4c0 0x7fb848157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56294/0 (socket says 172.21.15.172:56294) 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84dc7a640 1 -- 172.21.15.172:0/2034323152 learned_addr learned my addr 172.21.15.172:0/2034323152 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84dc7a640 1 -- 172.21.15.172:0/2034323152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8480b7a60 msgr2=0x7fb8480c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84d479640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8480b7a60 0x7fb8480c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84e47b640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8480cefd0 0x7fb8480c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84dc7a640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8480b7a60 0x7fb8480c6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84dc7a640 1 -- 172.21.15.172:0/2034323152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8480cefd0 msgr2=0x7fb8480c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84dc7a640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8480cefd0 0x7fb8480c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:53.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84dc7a640 1 -- 172.21.15.172:0/2034323152 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb848157d90 con 0x7fb84808c4c0 2024-04-14T08:41:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84dc7a640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb84808c4c0 0x7fb848157850 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fb83800c3c0 tx=0x7fb83800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb8467fc640 1 -- 172.21.15.172:0/2034323152 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838017070 con 0x7fb84808c4c0 2024-04-14T08:41:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb8467fc640 1 -- 172.21.15.172:0/2034323152 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb838005ce0 con 0x7fb84808c4c0 2024-04-14T08:41:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb8467fc640 1 -- 172.21.15.172:0/2034323152 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8380072f0 con 0x7fb84808c4c0 2024-04-14T08:41:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.382+0000 7fb84fede640 1 -- 172.21.15.172:0/2034323152 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8480c7470 con 0x7fb84808c4c0 2024-04-14T08:41:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.383+0000 7fb84fede640 1 -- 172.21.15.172:0/2034323152 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84815b840 con 0x7fb84808c4c0 2024-04-14T08:41:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.384+0000 7fb84fede640 1 -- 172.21.15.172:0/2034323152 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb848075490 con 0x7fb84808c4c0 2024-04-14T08:41:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.384+0000 7fb8467fc640 1 -- 172.21.15.172:0/2034323152 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb83801a070 con 0x7fb84808c4c0 2024-04-14T08:41:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.385+0000 7fb8467fc640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb83406c8f0 0x7fb83406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.385+0000 7fb84d479640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb83406c8f0 0x7fb83406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.385+0000 7fb8467fc640 1 -- 172.21.15.172:0/2034323152 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7fb83808b090 con 0x7fb84808c4c0 2024-04-14T08:41:53.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.385+0000 7fb84d479640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb83406c8f0 0x7fb83406edb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb848161ad0 tx=0x7fb83c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:53.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.389+0000 7fb8467fc640 1 -- 172.21.15.172:0/2034323152 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb838059350 con 0x7fb84808c4c0 2024-04-14T08:41:53.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.608+0000 7fb84fede640 1 -- 172.21.15.172:0/2034323152 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8480d0490 con 0x7fb84808c4c0 2024-04-14T08:41:53.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.608+0000 7fb8467fc640 1 -- 172.21.15.172:0/2034323152 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5198 (secure 0 0 0) 0x7fb838058c50 con 0x7fb84808c4c0 2024-04-14T08:41:53.610 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:53.610 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":80,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8005,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3289017090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3289017090},{"type":"v1","addr":"172.21.15.177:6835","nonce":3289017090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:41:04.492773+0000","modified":"2024-04-14T08:41:52.771653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8382},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8382":{"gid":8382,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":16,"addr":"172.21.15.172:6837/4028911489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4028911489},{"type":"v1","addr":"172.21.15.172:6837","nonce":4028911489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8382,"qdb_cluster":[8382]},"id":7}]} 2024-04-14T08:41:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 80 2024-04-14T08:41:53.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.610+0000 7fb84fede640 1 -- 172.21.15.172:0/2034323152 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb83406c8f0 msgr2=0x7fb83406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:53.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.610+0000 7fb84fede640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb83406c8f0 0x7fb83406edb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb848161ad0 tx=0x7fb83c008040 comp rx=0 tx=0).stop 2024-04-14T08:41:53.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.610+0000 7fb84fede640 1 -- 172.21.15.172:0/2034323152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb84808c4c0 msgr2=0x7fb848157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:53.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.610+0000 7fb84fede640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb84808c4c0 0x7fb848157850 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fb83800c3c0 tx=0x7fb83800c880 comp rx=0 tx=0).stop 2024-04-14T08:41:53.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.610+0000 7fb84fede640 1 -- 172.21.15.172:0/2034323152 shutdown_connections 2024-04-14T08:41:53.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.611+0000 7fb84fede640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb83406c8f0 0x7fb83406edb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:53.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.611+0000 7fb84fede640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8480cefd0 0x7fb8480c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:53.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.611+0000 7fb84fede640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8480b7a60 0x7fb8480c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:53.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.611+0000 7fb84fede640 1 --2- 172.21.15.172:0/2034323152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb84808c4c0 0x7fb848157850 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:53.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.611+0000 7fb84fede640 1 -- 172.21.15.172:0/2034323152 >> 172.21.15.172:0/2034323152 conn(0x7fb84808c000 msgr2=0x7fb8480d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:53.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.611+0000 7fb84fede640 1 -- 172.21.15.172:0/2034323152 shutdown_connections 2024-04-14T08:41:53.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:53.611+0000 7fb84fede640 1 -- 172.21.15.172:0/2034323152 wait complete. 2024-04-14T08:41:54.026 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:41:04.492773+0000', 'modified': '2024-04-14T08:41:52.771653+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 104, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8382}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8382': {'gid': 8382, 'name': 'a', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.172:6837/4028911489', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 4028911489}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 4028911489}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8382, 'qdb_cluster': [8382]} 2024-04-14T08:41:54.026 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:41:54.026 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-14T08:41:54.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.130+0000 7fdae997b640 1 Processor -- start 2024-04-14T08:41:54.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.130+0000 7fdae997b640 1 -- start start 2024-04-14T08:41:54.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.131+0000 7fdae997b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae406d040 0x7fdae40b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.131+0000 7fdae997b640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdae40d0d30 0x7fdae40cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.131+0000 7fdae997b640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdae40d12e0 con 0x7fdae415e820 2024-04-14T08:41:54.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.131+0000 7fdae997b640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fdae406cb90 con 0x7fdae40d0d30 2024-04-14T08:41:54.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.131+0000 7fdae997b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdae40cc660 con 0x7fdae406d040 2024-04-14T08:41:54.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.131+0000 7fdae37fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdae40d0d30 0x7fdae40cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.131+0000 7fdae2ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae406d040 0x7fdae40b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.131+0000 7fdadbfff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fdae415e820 0x7fdae406d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:59854/0 (socket says 172.21.15.172:59854) 2024-04-14T08:41:54.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.131+0000 7fdadbfff640 1 -- 172.21.15.172:0/2385333449 learned_addr learned my addr 172.21.15.172:0/2385333449 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:54.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.131+0000 7fdae37fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdae40d0d30 0x7fdae40cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:50350/0 (socket says 172.21.15.172:50350) 2024-04-14T08:41:54.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.131+0000 7fdae2ffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae406d040 0x7fdae40b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:34636/0 (socket says 172.21.15.172:34636) 2024-04-14T08:41:54.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.132+0000 7fdae27fc640 1 -- 172.21.15.172:0/2385333449 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268865760 0 0) 0x7fdae40d12e0 con 0x7fdae415e820 2024-04-14T08:41:54.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.132+0000 7fdae27fc640 1 -- 172.21.15.172:0/2385333449 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdad0003520 con 0x7fdae415e820 2024-04-14T08:41:54.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.132+0000 7fdae2ffd640 1 -- 172.21.15.172:0/2385333449 >> v2:172.21.15.177:3300/0 conn(0x7fdae40d0d30 msgr2=0x7fdae40cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.132+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/2385333449 >> v2:172.21.15.177:3300/0 conn(0x7fdae40d0d30 0x7fdae40cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.132+0000 7fdae2ffd640 1 -- 172.21.15.172:0/2385333449 >> v1:172.21.15.177:6789/0 conn(0x7fdae415e820 legacy=0x7fdae406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.132+0000 7fdae2ffd640 1 -- 172.21.15.172:0/2385333449 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae40d1580 con 0x7fdae406d040 2024-04-14T08:41:54.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.132+0000 7fdae37fe640 1 --2- 172.21.15.172:0/2385333449 >> v2:172.21.15.177:3300/0 conn(0x7fdae40d0d30 0x7fdae40cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:41:54.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.132+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/2385333449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae406d040 0x7fdae40b4620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdad40097a0 tx=0x7fdad402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=488f3efdc6a53ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.133+0000 7fdae27fc640 1 -- 172.21.15.172:0/2385333449 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4035020 con 0x7fdae406d040 2024-04-14T08:41:54.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.133+0000 7fdae27fc640 1 -- 172.21.15.172:0/2385333449 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad4031830 con 0x7fdae406d040 2024-04-14T08:41:54.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.133+0000 7fdae27fc640 1 -- 172.21.15.172:0/2385333449 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4031be0 con 0x7fdae406d040 2024-04-14T08:41:54.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.133+0000 7fdae997b640 1 -- 172.21.15.172:0/2385333449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae406d040 msgr2=0x7fdae40b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.133+0000 7fdae997b640 1 --2- 172.21.15.172:0/2385333449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae406d040 0x7fdae40b4620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdad40097a0 tx=0x7fdad402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.133+0000 7fdae997b640 1 -- 172.21.15.172:0/2385333449 shutdown_connections 2024-04-14T08:41:54.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.133+0000 7fdae997b640 1 --2- 172.21.15.172:0/2385333449 >> v2:172.21.15.177:3300/0 conn(0x7fdae40d0d30 0x7fdae40cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.133+0000 7fdae997b640 1 --2- 172.21.15.172:0/2385333449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae406d040 0x7fdae40b4620 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.133+0000 7fdae997b640 1 -- 172.21.15.172:0/2385333449 >> 172.21.15.172:0/2385333449 conn(0x7fdae40b3b00 msgr2=0x7fdae40b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.133+0000 7fdae997b640 1 -- 172.21.15.172:0/2385333449 shutdown_connections 2024-04-14T08:41:54.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.133+0000 7fdae997b640 1 -- 172.21.15.172:0/2385333449 wait complete. 2024-04-14T08:41:54.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.134+0000 7fdae997b640 1 Processor -- start 2024-04-14T08:41:54.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.134+0000 7fdae997b640 1 -- start start 2024-04-14T08:41:54.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.135+0000 7fdae997b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdae406d040 0x7fdae409bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.135+0000 7fdae997b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdae40d0d30 0x7fdae41f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.135+0000 7fdae997b640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae415e820 0x7fdae40701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.135+0000 7fdae997b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdae40cd5f0 con 0x7fdae40d0d30 2024-04-14T08:41:54.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.135+0000 7fdae997b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdae40ccc20 con 0x7fdae406d040 2024-04-14T08:41:54.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.135+0000 7fdae997b640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdae40cd770 con 0x7fdae415e820 2024-04-14T08:41:54.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.135+0000 7fdadbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdae40d0d30 0x7fdae41f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.135+0000 7fdadbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdae40d0d30 0x7fdae41f50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56300/0 (socket says 172.21.15.172:56300) 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.135+0000 7fdadbfff640 1 -- 172.21.15.172:0/1542283330 learned_addr learned my addr 172.21.15.172:0/1542283330 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.135+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdae406d040 0x7fdae409bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.135+0000 7fdae37fe640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae415e820 0x7fdae40701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.136+0000 7fdadbfff640 1 -- 172.21.15.172:0/1542283330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae415e820 msgr2=0x7fdae40701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.136+0000 7fdadbfff640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae415e820 0x7fdae40701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.136+0000 7fdadbfff640 1 -- 172.21.15.172:0/1542283330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdae406d040 msgr2=0x7fdae409bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.136+0000 7fdadbfff640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdae406d040 0x7fdae409bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.136+0000 7fdadbfff640 1 -- 172.21.15.172:0/1542283330 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae409c8b0 con 0x7fdae40d0d30 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.136+0000 7fdadbfff640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdae40d0d30 0x7fdae41f50d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00c390 tx=0x7fdacc00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.136+0000 7fdae0ff9640 1 -- 172.21.15.172:0/1542283330 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc017070 con 0x7fdae40d0d30 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.136+0000 7fdae0ff9640 1 -- 172.21.15.172:0/1542283330 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdacc005ce0 con 0x7fdae40d0d30 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.136+0000 7fdae997b640 1 -- 172.21.15.172:0/1542283330 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae409e280 con 0x7fdae40d0d30 2024-04-14T08:41:54.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.136+0000 7fdae0ff9640 1 -- 172.21.15.172:0/1542283330 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0072f0 con 0x7fdae40d0d30 2024-04-14T08:41:54.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.136+0000 7fdae997b640 1 -- 172.21.15.172:0/1542283330 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae409dec0 con 0x7fdae40d0d30 2024-04-14T08:41:54.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.138+0000 7fdae0ff9640 1 -- 172.21.15.172:0/1542283330 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdacc01a070 con 0x7fdae40d0d30 2024-04-14T08:41:54.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.138+0000 7fdae0ff9640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdac406c8f0 0x7fdac406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.138+0000 7fdae0ff9640 1 -- 172.21.15.172:0/1542283330 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7fdacc08afe0 con 0x7fdae40d0d30 2024-04-14T08:41:54.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.138+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdac406c8f0 0x7fdac406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.138+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdac406c8f0 0x7fdac406edb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdad4036000 tx=0x7fdad4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.139+0000 7fdaec765740 1 Processor -- start 2024-04-14T08:41:54.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.140+0000 7fdaec765740 1 -- start start 2024-04-14T08:41:54.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdaec765740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5b60 0x56159f5c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdaec765740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5f40 0x56159f5f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdaec765740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5c9040 0x56159f5f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdaec765740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56159f532030 con 0x56159f5f5b60 2024-04-14T08:41:54.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdaec765740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56159f4da620 con 0x56159f5c9040 2024-04-14T08:41:54.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdaec765740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56159f52dfe0 con 0x56159f5f5f40 2024-04-14T08:41:54.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdae2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5b60 0x56159f5c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdae2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5b60 0x56159f5c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56308/0 (socket says 172.21.15.172:56308) 2024-04-14T08:41:54.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdae2ffd640 1 -- 172.21.15.172:0/2614586104 learned_addr learned my addr 172.21.15.172:0/2614586104 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdadbfff640 1 --2- 172.21.15.172:0/2614586104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5f40 0x56159f5f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdae2ffd640 1 -- 172.21.15.172:0/2614586104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5f40 msgr2=0x56159f5f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/2614586104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5f40 0x56159f5f88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdae37fe640 1 --2- 172.21.15.172:0/2614586104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5c9040 0x56159f5f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdae2ffd640 1 -- 172.21.15.172:0/2614586104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5c9040 msgr2=0x56159f5f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/2614586104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5c9040 0x56159f5f8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdae2ffd640 1 -- 172.21.15.172:0/2614586104 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56159f2462d0 con 0x56159f5f5b60 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.141+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/2614586104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5b60 0x56159f5c8b00 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fdad402bee0 tx=0x7fdad4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdefe67f4a8351c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.142+0000 7fdad97fa640 1 -- 172.21.15.172:0/2614586104 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4035040 con 0x56159f5f5b60 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.142+0000 7fdad97fa640 1 -- 172.21.15.172:0/2614586104 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad404d070 con 0x56159f5f5b60 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.142+0000 7fdad97fa640 1 -- 172.21.15.172:0/2614586104 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4003610 con 0x56159f5f5b60 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.142+0000 7fdaec765740 1 -- 172.21.15.172:0/2614586104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5b60 msgr2=0x56159f5c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.142+0000 7fdaec765740 1 --2- 172.21.15.172:0/2614586104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5b60 0x56159f5c8b00 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fdad402bee0 tx=0x7fdad4030000 comp rx=0 tx=0).stop 2024-04-14T08:41:54.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.143+0000 7fdaec765740 1 -- 172.21.15.172:0/2614586104 shutdown_connections 2024-04-14T08:41:54.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.143+0000 7fdaec765740 1 --2- 172.21.15.172:0/2614586104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5c9040 0x56159f5f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.143+0000 7fdaec765740 1 --2- 172.21.15.172:0/2614586104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5f40 0x56159f5f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.143+0000 7fdaec765740 1 --2- 172.21.15.172:0/2614586104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5b60 0x56159f5c8b00 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.143+0000 7fdaec765740 1 -- 172.21.15.172:0/2614586104 >> 172.21.15.172:0/2614586104 conn(0x56159f5cad70 msgr2=0x56159f6138f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.143+0000 7fdaec765740 1 -- 172.21.15.172:0/2614586104 shutdown_connections 2024-04-14T08:41:54.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.143+0000 7fdaec765740 1 -- 172.21.15.172:0/2614586104 wait complete. 2024-04-14T08:41:54.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.143+0000 7fdaec765740 1 -- start start 2024-04-14T08:41:54.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.143+0000 7fdaec765740 1 Processor -- start 2024-04-14T08:41:54.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdaec765740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5b60 0x56159f5fb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdaec765740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5f40 0x56159f624140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdaec765740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 0x56159f5f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdaec765740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56159f5cd410 con 0x56159f624680 2024-04-14T08:41:54.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdaec765740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56159f4ed9e0 con 0x56159f5f5f40 2024-04-14T08:41:54.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdaec765740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56159f5cfc50 con 0x56159f5f5b60 2024-04-14T08:41:54.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdaec765740 10 client.0 _ll_register_callbacks cb 0x56159f52a080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:54.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 0x56159f5f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 0x56159f5f4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56318/0 (socket says 172.21.15.172:56318) 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdae37fe640 1 -- 172.21.15.172:0/2545835038 learned_addr learned my addr 172.21.15.172:0/2545835038 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdadbfff640 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5f40 0x56159f624140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5b60 0x56159f5fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdae37fe640 1 -- 172.21.15.172:0/2545835038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5b60 msgr2=0x56159f5fb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdae37fe640 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5b60 0x56159f5fb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdae37fe640 1 -- 172.21.15.172:0/2545835038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5f40 msgr2=0x56159f624140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdae37fe640 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5f40 0x56159f624140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.144+0000 7fdae37fe640 1 -- 172.21.15.172:0/2545835038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56159f3106b0 con 0x56159f624680 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdae37fe640 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 0x56159f5f4a60 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fdadc007da0 tx=0x7fdadc012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e43c8172f18528 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdac27fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdac27fc640 1 -- 172.21.15.172:0/2545835038 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc01c020 con 0x56159f624680 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdac27fc640 1 -- 172.21.15.172:0/2545835038 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7fdadc018670 con 0x56159f624680 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdaec765740 1 -- 172.21.15.172:0/2545835038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad404d070 con 0x56159f624680 2024-04-14T08:41:54.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdac27fc640 1 -- 172.21.15.172:0/2545835038 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fdadc026050 con 0x56159f624680 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdaec765740 10 client.8793 fetch_fsmap learned FSMap version 80 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdaec765740 1 -- 172.21.15.172:0/2545835038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) -- 0x56159f601040 con 0x56159f624680 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdac27fc640 1 -- 172.21.15.172:0/2545835038 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad404d070 con 0x56159f624680 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdac27fc640 1 -- 172.21.15.172:0/2545835038 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 80) ==== 3494+0+0 (secure 0 0 0) 0x7fdadc003990 con 0x56159f624680 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdaec765740 10 client.8793 fetch_fsmap finished waiting for FSMap version 80 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdaec765740 10 client.8793 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdaec765740 20 client.8793 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdaec765740 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56159f5feab0 0x56159f5fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdaec765740 4 client.8793 mds_command: new command op to 8382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdaec765740 1 -- 172.21.15.172:0/2545835038 --> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56159f4d9cc0 con 0x56159f5feab0 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.145+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56159f5feab0 0x56159f5fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdac27fc640 10 client.8793 ms_handle_connect on v2:172.21.15.172:6835/4028911489 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56159f5feab0 0x56159f5fdb00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ca32c2a46f8f18 server_cookie=ed2089ba31e5c322 in_seq=0 out_seq=0 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdac27fc640 1 -- 172.21.15.172:0/2545835038 <== mds.0 v2:172.21.15.172:6835/4028911489 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56159f4d9cc0 con 0x56159f5feab0 2024-04-14T08:41:54.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdac27fc640 10 client.8793 handle_command_reply: tid=0 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdaec765740 1 client.8793 shutdown 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdaec765740 1 -- 172.21.15.172:0/2545835038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 msgr2=0x56159f5f4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdaec765740 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 0x56159f5f4a60 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fdadc007da0 tx=0x7fdadc012610 comp rx=0 tx=0).stop 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdaec765740 1 -- 172.21.15.172:0/2545835038 shutdown_connections 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdaec765740 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56159f5feab0 0x56159f5fdb00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdaec765740 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 0x56159f5f4a60 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdaec765740 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5f40 0x56159f624140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdac27fc640 0 client.8793 ms_handle_reset on v2:172.21.15.172:6835/4028911489 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdaec765740 1 --2- 172.21.15.172:0/2545835038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5b60 0x56159f5fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.146+0000 7fdaec765740 1 -- 172.21.15.172:0/2545835038 >> 172.21.15.172:0/2545835038 conn(0x56159f5c9040 msgr2=0x56159f5ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.147+0000 7fdaec765740 1 -- 172.21.15.172:0/2545835038 shutdown_connections 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.147+0000 7fdaec765740 1 -- 172.21.15.172:0/2545835038 wait complete. 2024-04-14T08:41:54.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.147+0000 7fdaec765740 20 client.8793 trim_cache size 0 max 16384 2024-04-14T08:41:54.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.165+0000 7fdaec765740 1 Processor -- start 2024-04-14T08:41:54.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.165+0000 7fdaec765740 1 -- start start 2024-04-14T08:41:54.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdaec765740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5b60 0x56159f5f8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdaec765740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5f40 0x56159f617070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdaec765740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 0x56159f5f4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdaec765740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41ffea0 con 0x56159f624680 2024-04-14T08:41:54.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdaec765740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdae414ba50 con 0x56159f5f5b60 2024-04-14T08:41:54.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdaec765740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdae409f830 con 0x56159f5f5f40 2024-04-14T08:41:54.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 0x56159f5f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 0x56159f5f4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56334/0 (socket says 172.21.15.172:56334) 2024-04-14T08:41:54.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdae37fe640 1 -- 172.21.15.172:0/1357604844 learned_addr learned my addr 172.21.15.172:0/1357604844 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdadbfff640 1 --2- 172.21.15.172:0/1357604844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5f40 0x56159f617070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/1357604844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5b60 0x56159f5f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdae37fe640 1 -- 172.21.15.172:0/1357604844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5f40 msgr2=0x56159f617070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdae37fe640 1 --2- 172.21.15.172:0/1357604844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5f40 0x56159f617070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdae37fe640 1 -- 172.21.15.172:0/1357604844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5b60 msgr2=0x56159f5f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdae37fe640 1 --2- 172.21.15.172:0/1357604844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5b60 0x56159f5f8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdae37fe640 1 -- 172.21.15.172:0/1357604844 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56159f5fbff0 con 0x56159f624680 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdae37fe640 1 --2- 172.21.15.172:0/1357604844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 0x56159f5f4650 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fdadc007750 tx=0x7fdadc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460c8f4f2cf82849 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdad8ff9640 1 -- 172.21.15.172:0/1357604844 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc02f070 con 0x56159f624680 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdad8ff9640 1 -- 172.21.15.172:0/1357604844 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdadc007e90 con 0x56159f624680 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.166+0000 7fdad8ff9640 1 -- 172.21.15.172:0/1357604844 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc01c050 con 0x56159f624680 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.167+0000 7fdaec765740 1 -- 172.21.15.172:0/1357604844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 msgr2=0x56159f5f4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.167+0000 7fdaec765740 1 --2- 172.21.15.172:0/1357604844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 0x56159f5f4650 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fdadc007750 tx=0x7fdadc018000 comp rx=0 tx=0).stop 2024-04-14T08:41:54.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.167+0000 7fdaec765740 1 -- 172.21.15.172:0/1357604844 shutdown_connections 2024-04-14T08:41:54.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.167+0000 7fdaec765740 1 --2- 172.21.15.172:0/1357604844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f624680 0x56159f5f4650 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.167+0000 7fdaec765740 1 --2- 172.21.15.172:0/1357604844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f5f5f40 0x56159f617070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.167+0000 7fdaec765740 1 --2- 172.21.15.172:0/1357604844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5b60 0x56159f5f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.167+0000 7fdaec765740 1 -- 172.21.15.172:0/1357604844 >> 172.21.15.172:0/1357604844 conn(0x56159f5c9040 msgr2=0x56159f6021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.167+0000 7fdaec765740 1 -- 172.21.15.172:0/1357604844 shutdown_connections 2024-04-14T08:41:54.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.167+0000 7fdaec765740 1 -- 172.21.15.172:0/1357604844 wait complete. 2024-04-14T08:41:54.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.168+0000 7fdaec765740 1 -- start start 2024-04-14T08:41:54.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.168+0000 7fdaec765740 1 Processor -- start 2024-04-14T08:41:54.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.168+0000 7fdaec765740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5b60 0x56159f5c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.168+0000 7fdaec765740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5f40 0x56159f621fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.168+0000 7fdaec765740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f624680 0x56159f622500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdaec765740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56159f5fe930 con 0x56159f5f5f40 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdaec765740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56159f5ff1d0 con 0x56159f5f5b60 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdaec765740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56159f5fef80 con 0x56159f624680 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdaec765740 10 client.0 _ll_register_callbacks cb 0x7fdadc009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdadbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5f40 0x56159f621fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdadbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5f40 0x56159f621fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56344/0 (socket says 172.21.15.172:56344) 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdadbfff640 1 -- 172.21.15.172:0/2164888150 learned_addr learned my addr 172.21.15.172:0/2164888150 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5b60 0x56159f5c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdae37fe640 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f624680 0x56159f622500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdadbfff640 1 -- 172.21.15.172:0/2164888150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f624680 msgr2=0x56159f622500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdadbfff640 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f624680 0x56159f622500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdadbfff640 1 -- 172.21.15.172:0/2164888150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5b60 msgr2=0x56159f5c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdadbfff640 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5b60 0x56159f5c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdadbfff640 1 -- 172.21.15.172:0/2164888150 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56159f6235c0 con 0x56159f5f5f40 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5b60 0x56159f5c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdadbfff640 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5f40 0x56159f621fc0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fdacc058b60 tx=0x7fdacc058c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff6c30cc326dadb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdac27fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdac27fc640 1 -- 172.21.15.172:0/2164888150 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc05ae60 con 0x56159f5f5f40 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdac27fc640 1 -- 172.21.15.172:0/2164888150 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7fdacc007690 con 0x56159f5f5f40 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdaec765740 1 -- 172.21.15.172:0/2164888150 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdadc007e90 con 0x56159f5f5f40 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdac27fc640 1 -- 172.21.15.172:0/2164888150 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fdacc078050 con 0x56159f5f5f40 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdaec765740 10 client.8799 fetch_fsmap learned FSMap version 80 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdaec765740 1 -- 172.21.15.172:0/2164888150 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) -- 0x56159f68f4b0 con 0x56159f5f5f40 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.169+0000 7fdac27fc640 1 -- 172.21.15.172:0/2164888150 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdadc007e90 con 0x56159f5f5f40 2024-04-14T08:41:54.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdac27fc640 1 -- 172.21.15.172:0/2164888150 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 80) ==== 3494+0+0 (secure 0 0 0) 0x7fdacc058020 con 0x56159f5f5f40 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 10 client.8799 fetch_fsmap finished waiting for FSMap version 80 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 10 client.8799 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 20 client.8799 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56159f68de20 0x56159f68e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 4 client.8799 mds_command: new command op to 8382 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 -- 172.21.15.172:0/2164888150 --> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] -- command(tid 0: {"prefix": "status"}) -- 0x56159f53f570 con 0x56159f68de20 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56159f68de20 0x56159f68e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdae2ffd640 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56159f68de20 0x56159f68e200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34930d5c7258eb77 server_cookie=5832995c631c0b4e in_seq=0 out_seq=0 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdac27fc640 10 client.8799 ms_handle_connect on v2:172.21.15.172:6835/4028911489 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdac27fc640 1 -- 172.21.15.172:0/2164888150 <== mds.0 v2:172.21.15.172:6835/4028911489 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x56159f53f570 con 0x56159f68de20 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdac27fc640 10 client.8799 handle_command_reply: tid=0 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 client.8799 shutdown 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 -- 172.21.15.172:0/2164888150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5f40 msgr2=0x56159f621fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5f40 0x56159f621fc0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fdacc058b60 tx=0x7fdacc058c90 comp rx=0 tx=0).stop 2024-04-14T08:41:54.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 -- 172.21.15.172:0/2164888150 shutdown_connections 2024-04-14T08:41:54.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56159f68de20 0x56159f68e200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56159f624680 0x56159f622500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159f5f5f40 0x56159f621fc0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdac27fc640 0 client.8799 ms_handle_reset on v2:172.21.15.172:6835/4028911489 2024-04-14T08:41:54.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 --2- 172.21.15.172:0/2164888150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56159f5f5b60 0x56159f5c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 -- 172.21.15.172:0/2164888150 >> 172.21.15.172:0/2164888150 conn(0x56159f5c9040 msgr2=0x56159f5fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 -- 172.21.15.172:0/2164888150 shutdown_connections 2024-04-14T08:41:54.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.170+0000 7fdaec765740 1 -- 172.21.15.172:0/2164888150 wait complete. 2024-04-14T08:41:54.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdaec765740 20 client.8799 trim_cache size 0 max 16384 2024-04-14T08:41:54.178 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:41:54.179 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:41:54.179 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:41:54.179 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 8382, 2024-04-14T08:41:54.179 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:41:54.179 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:41:54.179 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:41:54.179 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 4.4114475649999996, 2024-04-14T08:41:54.179 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 80, 2024-04-14T08:41:54.179 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 104, 2024-04-14T08:41:54.179 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 104, 2024-04-14T08:41:54.179 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 54.077355480000001 2024-04-14T08:41:54.179 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:41:54.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdae997b640 1 -- 172.21.15.172:0/1542283330 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdac406c8f0 msgr2=0x7fdac406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdae997b640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdac406c8f0 0x7fdac406edb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdad4036000 tx=0x7fdad4008030 comp rx=0 tx=0).stop 2024-04-14T08:41:54.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdae997b640 1 -- 172.21.15.172:0/1542283330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdae40d0d30 msgr2=0x7fdae41f50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdae997b640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdae40d0d30 0x7fdae41f50d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00c390 tx=0x7fdacc00c790 comp rx=0 tx=0).stop 2024-04-14T08:41:54.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdae997b640 1 -- 172.21.15.172:0/1542283330 shutdown_connections 2024-04-14T08:41:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdae997b640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdac406c8f0 0x7fdac406edb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdae997b640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdae415e820 0x7fdae40701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdae997b640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdae40d0d30 0x7fdae41f50d0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdae997b640 1 --2- 172.21.15.172:0/1542283330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdae406d040 0x7fdae409bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdae997b640 1 -- 172.21.15.172:0/1542283330 >> 172.21.15.172:0/1542283330 conn(0x7fdae40b3b00 msgr2=0x7fdae406f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdae997b640 1 -- 172.21.15.172:0/1542283330 shutdown_connections 2024-04-14T08:41:54.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.171+0000 7fdae997b640 1 -- 172.21.15.172:0/1542283330 wait complete. 2024-04-14T08:41:54.183 DEBUG:teuthology.orchestra.run.smithi172:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-14T08:41:54.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:54.185 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-14T08:41:54.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.308+0000 7f9b336a6640 1 Processor -- start 2024-04-14T08:41:54.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.309+0000 7f9b336a6640 1 -- start start 2024-04-14T08:41:54.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.309+0000 7f9b336a6640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0d0a70 0x7f9b2c0d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.309+0000 7f9b336a6640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b2c15e890 con 0x7f9b2c0b39b0 2024-04-14T08:41:54.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.309+0000 7f9b336a6640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b2c0d1f30 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.309+0000 7f9b336a6640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9b2c0d13f0 con 0x7f9b2c0d0a70 2024-04-14T08:41:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.310+0000 7f9b31c43640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9b2c0b39b0 0x7f9b2c0b3e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:57082/0 (socket says 172.21.15.172:57082) 2024-04-14T08:41:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.310+0000 7f9b31c43640 1 -- 172.21.15.172:0/3081135017 learned_addr learned my addr 172.21.15.172:0/3081135017 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.310+0000 7f9b30c41640 1 --2- 172.21.15.172:0/3081135017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0d0a70 0x7f9b2c0d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.310+0000 7f9b23fff640 1 -- 172.21.15.172:0/3081135017 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1254020810 0 0) 0x7f9b2c15e890 con 0x7f9b2c0b39b0 2024-04-14T08:41:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.310+0000 7f9b23fff640 1 -- 172.21.15.172:0/3081135017 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b18003520 con 0x7f9b2c0b39b0 2024-04-14T08:41:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.310+0000 7f9b23fff640 1 -- 172.21.15.172:0/3081135017 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2277370578 0 0) 0x7f9b2c0d1f30 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.310+0000 7f9b30c41640 1 -- 172.21.15.172:0/3081135017 >> v1:172.21.15.177:6789/0 conn(0x7f9b2c15e4b0 legacy=0x7f9b2c0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.310+0000 7f9b30c41640 1 -- 172.21.15.172:0/3081135017 >> v1:172.21.15.172:6789/0 conn(0x7f9b2c0b39b0 legacy=0x7f9b2c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.310+0000 7f9b30c41640 1 -- 172.21.15.172:0/3081135017 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c0d2210 con 0x7f9b2c0d0a70 2024-04-14T08:41:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.311+0000 7f9b30c41640 1 --2- 172.21.15.172:0/3081135017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0d0a70 0x7f9b2c0d0eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0097d0 tx=0x7f9b1c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2bbf5787907aef server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.311+0000 7f9b23fff640 1 -- 172.21.15.172:0/3081135017 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c035020 con 0x7f9b2c0d0a70 2024-04-14T08:41:54.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.311+0000 7f9b23fff640 1 -- 172.21.15.172:0/3081135017 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1c031800 con 0x7f9b2c0d0a70 2024-04-14T08:41:54.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.311+0000 7f9b23fff640 1 -- 172.21.15.172:0/3081135017 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c031bd0 con 0x7f9b2c0d0a70 2024-04-14T08:41:54.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.312+0000 7f9b336a6640 1 -- 172.21.15.172:0/3081135017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0d0a70 msgr2=0x7f9b2c0d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.312+0000 7f9b336a6640 1 --2- 172.21.15.172:0/3081135017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0d0a70 0x7f9b2c0d0eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0097d0 tx=0x7f9b1c02b8a0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.312+0000 7f9b336a6640 1 -- 172.21.15.172:0/3081135017 shutdown_connections 2024-04-14T08:41:54.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.312+0000 7f9b336a6640 1 --2- 172.21.15.172:0/3081135017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0d0a70 0x7f9b2c0d0eb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.312+0000 7f9b336a6640 1 -- 172.21.15.172:0/3081135017 >> 172.21.15.172:0/3081135017 conn(0x7f9b2c09d570 msgr2=0x7f9b2c0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.312+0000 7f9b336a6640 1 -- 172.21.15.172:0/3081135017 shutdown_connections 2024-04-14T08:41:54.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.312+0000 7f9b336a6640 1 -- 172.21.15.172:0/3081135017 wait complete. 2024-04-14T08:41:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.313+0000 7f9b336a6640 1 Processor -- start 2024-04-14T08:41:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.313+0000 7f9b336a6640 1 -- start start 2024-04-14T08:41:54.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b336a6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b2c0b39b0 0x7f9b2c156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b336a6640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0d0a70 0x7f9b2c0cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b336a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c15e4b0 0x7f9b2c0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b336a6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c0d27d0 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b336a6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c0d24f0 con 0x7f9b2c0b39b0 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b336a6640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9b2c0d2ab0 con 0x7f9b2c0d0a70 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b31c43640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c15e4b0 0x7f9b2c0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b31c43640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c15e4b0 0x7f9b2c0c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56360/0 (socket says 172.21.15.172:56360) 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b31c43640 1 -- 172.21.15.172:0/1959802918 learned_addr learned my addr 172.21.15.172:0/1959802918 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b31442640 1 --2- 172.21.15.172:0/1959802918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b2c0b39b0 0x7f9b2c156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b30c41640 1 --2- 172.21.15.172:0/1959802918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0d0a70 0x7f9b2c0cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b31c43640 1 -- 172.21.15.172:0/1959802918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0d0a70 msgr2=0x7f9b2c0cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b31c43640 1 --2- 172.21.15.172:0/1959802918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b2c0d0a70 0x7f9b2c0cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b31c43640 1 -- 172.21.15.172:0/1959802918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b2c0b39b0 msgr2=0x7f9b2c156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b31c43640 1 --2- 172.21.15.172:0/1959802918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b2c0b39b0 0x7f9b2c156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b31c43640 1 -- 172.21.15.172:0/1959802918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c0cc980 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.314+0000 7f9b31c43640 1 --2- 172.21.15.172:0/1959802918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b2c15e4b0 0x7f9b2c0c9bf0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400cb40 tx=0x7f9b2400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.315+0000 7f9b21ffb640 1 -- 172.21.15.172:0/1959802918 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24017070 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.315+0000 7f9b21ffb640 1 -- 172.21.15.172:0/1959802918 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b2400aca0 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.315+0000 7f9b336a6640 1 -- 172.21.15.172:0/1959802918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2c0ccc60 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.315+0000 7f9b21ffb640 1 -- 172.21.15.172:0/1959802918 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24012620 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.315+0000 7f9b336a6640 1 -- 172.21.15.172:0/1959802918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2c0c7960 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.316+0000 7f9b336a6640 1 -- 172.21.15.172:0/1959802918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f9b2c068590 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.316+0000 7f9b21ffb640 1 -- 172.21.15.172:0/1959802918 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b2401a070 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.317+0000 7f9b21ffb640 1 --2- 172.21.15.172:0/1959802918 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9b1006c8f0 0x7f9b1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.317+0000 7f9b21ffb640 1 -- 172.21.15.172:0/1959802918 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7f9b2408bb00 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.317+0000 7f9b31442640 1 --2- 172.21.15.172:0/1959802918 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9b1006c8f0 0x7f9b1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.317+0000 7f9b21ffb640 1 -- 172.21.15.172:0/1959802918 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+512 (secure 0 0 0) 0x7f9b24090240 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.317+0000 7f9b336a6640 1 -- 172.21.15.172:0/1959802918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f9b2c0cee30 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.317+0000 7f9b21ffb640 1 -- 172.21.15.172:0/1959802918 <== mon.0 v2:172.21.15.172:3300/0 7 ==== log(2 entries from seq 1173 at 2024-04-14T08:41:52.771720+0000) ==== 389+0+0 (secure 0 0 0) 0x7f9b24089f40 con 0x7f9b2c15e4b0 2024-04-14T08:41:54.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.317+0000 7f9b31442640 1 --2- 172.21.15.172:0/1959802918 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9b1006c8f0 0x7f9b1006edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9b28000f60 tx=0x7f9b28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.320 INFO:teuthology.orchestra.run.smithi172.stdout: cluster: 2024-04-14T08:41:54.320 INFO:teuthology.orchestra.run.smithi172.stdout: id: 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: health: HEALTH_OK 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: services: 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: mgr: y(active, since 9m), standbys: x 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: mds: 1/1 daemons up, 3 standby 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: data: 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: volumes: 1/1 healthy 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: pools: 3 pools, 97 pgs 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: objects: 30 objects, 19 MiB 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: pgs: 97 active+clean 2024-04-14T08:41:54.321 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:54.322 INFO:teuthology.orchestra.run.smithi172.stdout: io: 2024-04-14T08:41:54.322 INFO:teuthology.orchestra.run.smithi172.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-04-14T08:41:54.322 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:54.322 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:41:54.322 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:52.771720+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-14T08:41:54.322 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:52.771747+0000 mon.a [INF] Cluster is now healthy 2024-04-14T08:41:54.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.320+0000 7fb4d0cad640 1 Processor -- start 2024-04-14T08:41:54.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.320+0000 7fb4d0cad640 1 -- start start 2024-04-14T08:41:54.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.320+0000 7fb4d0cad640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb4cc06f4d0 0x7fb4cc0d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.320+0000 7fb4d0cad640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4cc0b4670 con 0x7fb4cc0b1100 2024-04-14T08:41:54.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.320+0000 7fb4d0cad640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4cc0b4950 con 0x7fb4cc09aa70 2024-04-14T08:41:54.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.320+0000 7fb4d0cad640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb4cc15eee0 con 0x7fb4cc06f4d0 2024-04-14T08:41:54.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.321+0000 7fb4cad9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb4cc06f4d0 0x7fb4cc0d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.321+0000 7fb4c3fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb4cc0b1100 0x7fb4cc0d01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:57094/0 (socket says 172.21.15.172:57094) 2024-04-14T08:41:54.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.321+0000 7fb4c3fff640 1 -- 172.21.15.172:0/511346145 learned_addr learned my addr 172.21.15.172:0/511346145 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.321+0000 7fb4cad9d640 1 -- 172.21.15.172:0/511346145 >> v1:172.21.15.177:6789/0 conn(0x7fb4cc09aa70 legacy=0x7fb4cc09ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.321+0000 7fb4cad9d640 1 -- 172.21.15.172:0/511346145 >> v1:172.21.15.172:6789/0 conn(0x7fb4cc0b1100 legacy=0x7fb4cc0d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.321+0000 7fb4cad9d640 1 -- 172.21.15.172:0/511346145 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc15f1c0 con 0x7fb4cc06f4d0 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.322+0000 7fb4cad9d640 1 --2- 172.21.15.172:0/511346145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4cc06f4d0 0x7fb4cc0d08e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009d90 tx=0x7fb4c402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228c72b45b4beac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.322+0000 7fb4c9d9b640 1 -- 172.21.15.172:0/511346145 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4039070 con 0x7fb4cc06f4d0 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.322+0000 7fb4c9d9b640 1 -- 172.21.15.172:0/511346145 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c402cc60 con 0x7fb4cc06f4d0 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.322+0000 7fb4c9d9b640 1 -- 172.21.15.172:0/511346145 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4034660 con 0x7fb4cc06f4d0 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.322+0000 7fb4d0cad640 1 -- 172.21.15.172:0/511346145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4cc06f4d0 msgr2=0x7fb4cc0d08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.322+0000 7fb4d0cad640 1 --2- 172.21.15.172:0/511346145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4cc06f4d0 0x7fb4cc0d08e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009d90 tx=0x7fb4c402beb0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.322+0000 7fb4d0cad640 1 -- 172.21.15.172:0/511346145 shutdown_connections 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.322+0000 7fb4d0cad640 1 --2- 172.21.15.172:0/511346145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4cc06f4d0 0x7fb4cc0d08e0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.322+0000 7fb4d0cad640 1 -- 172.21.15.172:0/511346145 >> 172.21.15.172:0/511346145 conn(0x7fb4cc0b3b50 msgr2=0x7fb4cc0b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.322+0000 7fb4d0cad640 1 -- 172.21.15.172:0/511346145 shutdown_connections 2024-04-14T08:41:54.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.322+0000 7fb4d0cad640 1 -- 172.21.15.172:0/511346145 wait complete. 2024-04-14T08:41:54.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.324+0000 7fb4d0cad640 1 Processor -- start 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.324+0000 7fb4d0cad640 1 -- start start 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.324+0000 7fb4d0cad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc06f4d0 0x7fb4cc0c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.324+0000 7fb4d0cad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4cc09aa70 0x7fb4cc1f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.324+0000 7fb4d0cad640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc0b1100 0x7fb4cc159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.324+0000 7fb4d0cad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc15fb40 con 0x7fb4cc09aa70 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.324+0000 7fb4d0cad640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc15f9c0 con 0x7fb4cc0b1100 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.324+0000 7fb4d0cad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb4cc0d0e20 con 0x7fb4cc06f4d0 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.324+0000 7fb4c3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4cc09aa70 0x7fb4cc1f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.324+0000 7fb4c3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4cc09aa70 0x7fb4cc1f4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56374/0 (socket says 172.21.15.172:56374) 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.324+0000 7fb4c3fff640 1 -- 172.21.15.172:0/1815622867 learned_addr learned my addr 172.21.15.172:0/1815622867 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4cad9d640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc0b1100 0x7fb4cc159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4ca59c640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc06f4d0 0x7fb4cc0c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4c3fff640 1 -- 172.21.15.172:0/1815622867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc06f4d0 msgr2=0x7fb4cc0c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4c3fff640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc06f4d0 0x7fb4cc0c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4c3fff640 1 -- 172.21.15.172:0/1815622867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc0b1100 msgr2=0x7fb4cc159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4c3fff640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc0b1100 0x7fb4cc159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4c3fff640 1 -- 172.21.15.172:0/1815622867 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc0cd850 con 0x7fb4cc09aa70 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4c3fff640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4cc09aa70 0x7fb4cc1f4700 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400c4a0 tx=0x7fb4b400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4c2ffd640 1 -- 172.21.15.172:0/1815622867 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4017070 con 0x7fb4cc09aa70 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4c2ffd640 1 -- 172.21.15.172:0/1815622867 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b4005ce0 con 0x7fb4cc09aa70 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4d0cad640 1 -- 172.21.15.172:0/1815622867 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4cc1f4d50 con 0x7fb4cc09aa70 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4c2ffd640 1 -- 172.21.15.172:0/1815622867 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4007270 con 0x7fb4cc09aa70 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.325+0000 7fb4d0cad640 1 -- 172.21.15.172:0/1815622867 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4cc1f5310 con 0x7fb4cc09aa70 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.327+0000 7fb4c2ffd640 1 -- 172.21.15.172:0/1815622867 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4b401a070 con 0x7fb4cc09aa70 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.327+0000 7fb4c2ffd640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb4ac06c8f0 0x7fb4ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.327+0000 7fb4c2ffd640 1 -- 172.21.15.172:0/1815622867 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7fb4b408af40 con 0x7fb4cc09aa70 2024-04-14T08:41:54.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.327+0000 7fb4ca59c640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb4ac06c8f0 0x7fb4ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.328+0000 7fb4ca59c640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb4ac06c8f0 0x7fb4ac06edb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0017d0 tx=0x7fb4bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.328+0000 7fb4d3b6d740 1 Processor -- start 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.328+0000 7fb4d3b6d740 1 -- start start 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.329+0000 7fb4d3b6d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe70632e0 0x558fe70636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.329+0000 7fb4d3b6d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe7033300 0x558fe7060550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.329+0000 7fb4d3b6d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe7060a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.329+0000 7fb4d3b6d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558fe6f9d740 con 0x558fe70632e0 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.329+0000 7fb4d3b6d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558fe6f45930 con 0x558fe7033300 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.329+0000 7fb4d3b6d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x558fe6f99890 con 0x558fe7036e60 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.329+0000 7fb4ca59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe70632e0 0x558fe70636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.329+0000 7fb4ca59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe70632e0 0x558fe70636c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56382/0 (socket says 172.21.15.172:56382) 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.329+0000 7fb4ca59c640 1 -- 172.21.15.172:0/2729715741 learned_addr learned my addr 172.21.15.172:0/2729715741 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.329+0000 7fb4cad9d640 1 --2- 172.21.15.172:0/2729715741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe7060a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.329+0000 7fb4c3fff640 1 --2- 172.21.15.172:0/2729715741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe7033300 0x558fe7060550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.330+0000 7fb4ca59c640 1 -- 172.21.15.172:0/2729715741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 msgr2=0x558fe7060a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.330+0000 7fb4ca59c640 1 --2- 172.21.15.172:0/2729715741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe7060a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.330+0000 7fb4ca59c640 1 -- 172.21.15.172:0/2729715741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe7033300 msgr2=0x558fe7060550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.330+0000 7fb4ca59c640 1 --2- 172.21.15.172:0/2729715741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe7033300 0x558fe7060550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.330+0000 7fb4ca59c640 1 -- 172.21.15.172:0/2729715741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fe6d797e0 con 0x558fe70632e0 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.330+0000 7fb4ca59c640 1 --2- 172.21.15.172:0/2729715741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe70632e0 0x558fe70636c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc001eb0 tx=0x7fb4bc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3281aaeaf458f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.330+0000 7fb4abfff640 1 -- 172.21.15.172:0/2729715741 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc026070 con 0x558fe70632e0 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.330+0000 7fb4abfff640 1 -- 172.21.15.172:0/2729715741 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4bc007030 con 0x558fe70632e0 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.330+0000 7fb4abfff640 1 -- 172.21.15.172:0/2729715741 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0073e0 con 0x558fe70632e0 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.330+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/2729715741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe70632e0 msgr2=0x558fe70636c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.330+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/2729715741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe70632e0 0x558fe70636c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc001eb0 tx=0x7fb4bc012000 comp rx=0 tx=0).stop 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.331+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/2729715741 shutdown_connections 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.331+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/2729715741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe7060a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.331+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/2729715741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe7033300 0x558fe7060550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.331+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/2729715741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe70632e0 0x558fe70636c0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.331+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/2729715741 >> 172.21.15.172:0/2729715741 conn(0x558fe7035570 msgr2=0x558fe707e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.331+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/2729715741 shutdown_connections 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.331+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/2729715741 wait complete. 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.331+0000 7fb4d3b6d740 1 -- start start 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.331+0000 7fb4d3b6d740 1 Processor -- start 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.332+0000 7fb4d3b6d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe706a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.332+0000 7fb4d3b6d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe70632e0 0x558fe70664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.332+0000 7fb4d3b6d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 0x558fe708eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.332+0000 7fb4d3b6d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558fe7038010 con 0x558fe705f930 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.332+0000 7fb4d3b6d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558fe6f58cf0 con 0x558fe70632e0 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.332+0000 7fb4d3b6d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x558fe7037400 con 0x558fe7036e60 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.332+0000 7fb4d3b6d740 10 client.0 _ll_register_callbacks cb 0x558fe6f259f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.332+0000 7fb4cad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 0x558fe708eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.332+0000 7fb4cad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 0x558fe708eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56386/0 (socket says 172.21.15.172:56386) 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.332+0000 7fb4cad9d640 1 -- 172.21.15.172:0/1958279615 learned_addr learned my addr 172.21.15.172:0/1958279615 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.332+0000 7fb4ca59c640 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe706a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.332+0000 7fb4c3fff640 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe70632e0 0x558fe70664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4cad9d640 1 -- 172.21.15.172:0/1958279615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 msgr2=0x558fe706a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4cad9d640 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe706a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4cad9d640 1 -- 172.21.15.172:0/1958279615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe70632e0 msgr2=0x558fe70664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4cad9d640 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe70632e0 0x558fe70664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4cad9d640 1 -- 172.21.15.172:0/1958279615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fe6dd9f70 con 0x558fe705f930 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4cad9d640 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 0x558fe708eb90 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4037040 tx=0x7fb4c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd69b165166ae72 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4a97fa640 1 -- 172.21.15.172:0/1958279615 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4039040 con 0x558fe705f930 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4a97fa640 1 -- 172.21.15.172:0/1958279615 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7fb4c4034c40 con 0x558fe705f930 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1958279615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4bc007030 con 0x558fe705f930 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4a97fa640 1 -- 172.21.15.172:0/1958279615 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fb4c4034e40 con 0x558fe705f930 2024-04-14T08:41:54.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4d3b6d740 10 client.8820 fetch_fsmap learned FSMap version 80 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1958279615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) -- 0x558fe6c71170 con 0x558fe705f930 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4a97fa640 1 -- 172.21.15.172:0/1958279615 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4bc007030 con 0x558fe705f930 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.333+0000 7fb4a97fa640 1 -- 172.21.15.172:0/1958279615 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 80) ==== 3494+0+0 (secure 0 0 0) 0x7fb4c4041440 con 0x558fe705f930 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.334+0000 7fb4d3b6d740 10 client.8820 fetch_fsmap finished waiting for FSMap version 80 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.334+0000 7fb4d3b6d740 10 client.8820 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.334+0000 7fb4d3b6d740 20 client.8820 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.334+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x558fe705ece0 0x558fe7067a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.334+0000 7fb4d3b6d740 4 client.8820 mds_command: new command op to 8382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.334+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1958279615 --> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fe6faac80 con 0x558fe705ece0 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.334+0000 7fb4ca59c640 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x558fe705ece0 0x558fe7067a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.334+0000 7fb4a97fa640 10 client.8820 ms_handle_connect on v2:172.21.15.172:6835/4028911489 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.334+0000 7fb4ca59c640 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x558fe705ece0 0x558fe7067a20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e1234b5e23ebbe2 server_cookie=d721818bd4fd83a1 in_seq=0 out_seq=0 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4a97fa640 1 -- 172.21.15.172:0/1958279615 <== mds.0 v2:172.21.15.172:6835/4028911489 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558fe6faac80 con 0x558fe705ece0 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4a97fa640 10 client.8820 handle_command_reply: tid=0 2024-04-14T08:41:54.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4d3b6d740 1 client.8820 shutdown 2024-04-14T08:41:54.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1958279615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 msgr2=0x558fe708eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 0x558fe708eb90 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4037040 tx=0x7fb4c4004ab0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1958279615 shutdown_connections 2024-04-14T08:41:54.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x558fe705ece0 0x558fe7067a20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4a97fa640 0 client.8820 ms_handle_reset on v2:172.21.15.172:6835/4028911489 2024-04-14T08:41:54.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 0x558fe708eb90 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe70632e0 0x558fe70664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1958279615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe706a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1958279615 >> 172.21.15.172:0/1958279615 conn(0x558fe7033300 msgr2=0x558fe70393a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1958279615 shutdown_connections 2024-04-14T08:41:54.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1958279615 wait complete. 2024-04-14T08:41:54.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.335+0000 7fb4d3b6d740 20 client.8820 trim_cache size 0 max 16384 2024-04-14T08:41:54.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.353+0000 7fb4d3b6d740 1 Processor -- start 2024-04-14T08:41:54.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.353+0000 7fb4d3b6d740 1 -- start start 2024-04-14T08:41:54.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.353+0000 7fb4d3b6d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe7060090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.353+0000 7fb4d3b6d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe705ece0 0x558fe70817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.353+0000 7fb4d3b6d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 0x558fe7081d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.353+0000 7fb4d3b6d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc201140 con 0x558fe705f930 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.353+0000 7fb4d3b6d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc200180 con 0x558fe705ece0 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.353+0000 7fb4d3b6d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb4cc0c9ed0 con 0x558fe7036e60 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.353+0000 7fb4cad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 0x558fe7081d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.353+0000 7fb4cad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 0x558fe7081d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56402/0 (socket says 172.21.15.172:56402) 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.353+0000 7fb4cad9d640 1 -- 172.21.15.172:0/1994647193 learned_addr learned my addr 172.21.15.172:0/1994647193 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.354+0000 7fb4cad9d640 1 -- 172.21.15.172:0/1994647193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 msgr2=0x558fe7060090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.354+0000 7fb4cad9d640 1 --2- 172.21.15.172:0/1994647193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe7060090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.354+0000 7fb4cad9d640 1 -- 172.21.15.172:0/1994647193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe705ece0 msgr2=0x558fe70817f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.354+0000 7fb4cad9d640 1 --2- 172.21.15.172:0/1994647193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe705ece0 0x558fe70817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.354+0000 7fb4cad9d640 1 -- 172.21.15.172:0/1994647193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fe705f0c0 con 0x558fe705f930 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.354+0000 7fb4cad9d640 1 --2- 172.21.15.172:0/1994647193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 0x558fe7081d30 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fb4c402cdd0 tx=0x7fb4c4042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315a252a651433b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.354+0000 7fb4ab7fe640 1 -- 172.21.15.172:0/1994647193 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4039070 con 0x558fe705f930 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.354+0000 7fb4ab7fe640 1 -- 172.21.15.172:0/1994647193 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c403a040 con 0x558fe705f930 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.354+0000 7fb4ab7fe640 1 -- 172.21.15.172:0/1994647193 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4041990 con 0x558fe705f930 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.354+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1994647193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 msgr2=0x558fe7081d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.354+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1994647193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 0x558fe7081d30 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fb4c402cdd0 tx=0x7fb4c4042000 comp rx=0 tx=0).stop 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.355+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1994647193 shutdown_connections 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.355+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1994647193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705f930 0x558fe7081d30 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.355+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1994647193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe705ece0 0x558fe70817f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.355+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1994647193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe7060090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.355+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1994647193 >> 172.21.15.172:0/1994647193 conn(0x558fe7033300 msgr2=0x558fe702ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.355+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1994647193 shutdown_connections 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.355+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1994647193 wait complete. 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.355+0000 7fb4d3b6d740 1 -- start start 2024-04-14T08:41:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.355+0000 7fb4d3b6d740 1 Processor -- start 2024-04-14T08:41:54.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4d3b6d740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe709e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4d3b6d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705ece0 0x558fe70984f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4d3b6d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe705f930 0x558fe705db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4d3b6d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558fe7067140 con 0x558fe705ece0 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4d3b6d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc0ce880 con 0x558fe705f930 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4d3b6d740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x558fe70693f0 con 0x558fe7036e60 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4d3b6d740 10 client.0 _ll_register_callbacks cb 0x7fb4c402cca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4c3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705ece0 0x558fe70984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4c3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705ece0 0x558fe70984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56404/0 (socket says 172.21.15.172:56404) 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4c3fff640 1 -- 172.21.15.172:0/1428866172 learned_addr learned my addr 172.21.15.172:0/1428866172 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4ca59c640 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe709e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4cad9d640 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe705f930 0x558fe705db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4c3fff640 1 -- 172.21.15.172:0/1428866172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 msgr2=0x558fe709e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4c3fff640 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe709e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4c3fff640 1 -- 172.21.15.172:0/1428866172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe705f930 msgr2=0x558fe705db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4c3fff640 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe705f930 0x558fe705db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.356+0000 7fb4c3fff640 1 -- 172.21.15.172:0/1428866172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fe7066970 con 0x558fe705ece0 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4c3fff640 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705ece0 0x558fe70984f0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4089270 tx=0x7fb4b405cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580ae26a85035331 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4a97fa640 1 -- 172.21.15.172:0/1428866172 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4086070 con 0x558fe705ece0 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4a97fa640 1 -- 172.21.15.172:0/1428866172 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7fb4b4082dd0 con 0x558fe705ece0 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1428866172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c403a040 con 0x558fe705ece0 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4a97fa640 1 -- 172.21.15.172:0/1428866172 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fb4b4077050 con 0x558fe705ece0 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4d3b6d740 10 client.8826 fetch_fsmap learned FSMap version 80 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1428866172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) -- 0x558fe70395a0 con 0x558fe705ece0 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4a97fa640 1 -- 172.21.15.172:0/1428866172 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c403a040 con 0x558fe705ece0 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4a97fa640 1 -- 172.21.15.172:0/1428866172 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 80) ==== 3494+0+0 (secure 0 0 0) 0x7fb4b40611c0 con 0x558fe705ece0 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4d3b6d740 10 client.8826 fetch_fsmap finished waiting for FSMap version 80 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4d3b6d740 10 client.8826 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4d3b6d740 20 client.8826 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x558fe70632e0 0x558fe70817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4d3b6d740 4 client.8826 mds_command: new command op to 8382 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-14T08:41:54.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1428866172 --> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x558fe7033b00 con 0x558fe70632e0 2024-04-14T08:41:54.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.357+0000 7fb4ca59c640 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x558fe70632e0 0x558fe70817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.358+0000 7fb4a97fa640 10 client.8826 ms_handle_connect on v2:172.21.15.172:6835/4028911489 2024-04-14T08:41:54.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.358+0000 7fb4ca59c640 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x558fe70632e0 0x558fe70817f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94cf58a59e4eb486 server_cookie=7fa9157c6ce0b6ae in_seq=0 out_seq=0 2024-04-14T08:41:54.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.358+0000 7fb4a97fa640 1 -- 172.21.15.172:0/1428866172 <== mds.0 v2:172.21.15.172:6835/4028911489 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558fe7033b00 con 0x558fe70632e0 2024-04-14T08:41:54.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.358+0000 7fb4a97fa640 10 client.8826 handle_command_reply: tid=0 2024-04-14T08:41:54.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.358+0000 7fb4d3b6d740 1 client.8826 shutdown 2024-04-14T08:41:54.364 INFO:tasks.ceph.mds.a.smithi172.stderr:2024-04-14T08:41:54.358+0000 7fa0673e9640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-14T08:41:54.364 INFO:tasks.ceph.mds.a.smithi172.stderr:2024-04-14T08:41:54.358+0000 7fa0673e9640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-14T08:41:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.358+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1428866172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705ece0 msgr2=0x558fe70984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.358+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705ece0 0x558fe70984f0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4089270 tx=0x7fb4b405cee0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.359+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1428866172 shutdown_connections 2024-04-14T08:41:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.359+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x558fe70632e0 0x558fe70817f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.359+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558fe705f930 0x558fe705db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.359+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fe705ece0 0x558fe70984f0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.359+0000 7fb4d3b6d740 1 --2- 172.21.15.172:0/1428866172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x558fe7036e60 0x558fe709e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.359+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1428866172 >> 172.21.15.172:0/1428866172 conn(0x558fe7033300 msgr2=0x558fe702ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.366 INFO:tasks.ceph.mds.a.smithi172.stderr:2024-04-14T08:41:54.359+0000 7fa0673e9640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-14T08:41:54.366 INFO:tasks.ceph.mds.a.smithi172.stderr:2024-04-14T08:41:54.359+0000 7fa0673e9640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-14T08:41:54.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.359+0000 7fb4a97fa640 0 client.8826 ms_handle_reset on v2:172.21.15.172:6835/4028911489 2024-04-14T08:41:54.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.359+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1428866172 shutdown_connections 2024-04-14T08:41:54.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.359+0000 7fb4d3b6d740 1 -- 172.21.15.172:0/1428866172 wait complete. 2024-04-14T08:41:54.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.359+0000 7fb4d3b6d740 20 client.8826 trim_cache size 0 max 16384 2024-04-14T08:41:54.367 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:41:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-04-14T08:41:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "f46b6eaa-5b44-4fb7-b309-6c943446cfaf", 2024-04-14T08:41:54.367 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-04-14T08:41:54.367 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:41:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.360+0000 7fb4d0cad640 1 -- 172.21.15.172:0/1815622867 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb4ac06c8f0 msgr2=0x7fb4ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.360+0000 7fb4d0cad640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb4ac06c8f0 0x7fb4ac06edb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0017d0 tx=0x7fb4bc009040 comp rx=0 tx=0).stop 2024-04-14T08:41:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.360+0000 7fb4d0cad640 1 -- 172.21.15.172:0/1815622867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4cc09aa70 msgr2=0x7fb4cc1f4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.360+0000 7fb4d0cad640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4cc09aa70 0x7fb4cc1f4700 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400c4a0 tx=0x7fb4b400c960 comp rx=0 tx=0).stop 2024-04-14T08:41:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.360+0000 7fb4d0cad640 1 -- 172.21.15.172:0/1815622867 shutdown_connections 2024-04-14T08:41:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.360+0000 7fb4d0cad640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb4ac06c8f0 0x7fb4ac06edb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.360+0000 7fb4d0cad640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4cc0b1100 0x7fb4cc159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.360+0000 7fb4d0cad640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4cc09aa70 0x7fb4cc1f4700 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.360+0000 7fb4d0cad640 1 --2- 172.21.15.172:0/1815622867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4cc06f4d0 0x7fb4cc0c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.360+0000 7fb4d0cad640 1 -- 172.21.15.172:0/1815622867 >> 172.21.15.172:0/1815622867 conn(0x7fb4cc0b3b50 msgr2=0x7fb4cc15d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.360+0000 7fb4d0cad640 1 -- 172.21.15.172:0/1815622867 shutdown_connections 2024-04-14T08:41:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.360+0000 7fb4d0cad640 1 -- 172.21.15.172:0/1815622867 wait complete. 2024-04-14T08:41:54.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:41:54.372 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-14T08:41:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.480+0000 7fc867115640 1 Processor -- start 2024-04-14T08:41:54.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.480+0000 7fc867115640 1 -- start start 2024-04-14T08:41:54.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.481+0000 7fc867115640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86015e430 0x7fc860162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.481+0000 7fc867115640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8600d2e00 0x7fc8601629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.481+0000 7fc867115640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8600cefb0 con 0x7fc860162540 2024-04-14T08:41:54.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.481+0000 7fc867115640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc8600cf290 con 0x7fc8600d2e00 2024-04-14T08:41:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.481+0000 7fc867115640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc8600cf570 con 0x7fc86015e430 2024-04-14T08:41:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.482+0000 7fc8656b2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8600d2e00 0x7fc8601629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.482+0000 7fc8656b2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8600d2e00 0x7fc8601629a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56408/0 (socket says 172.21.15.172:56408) 2024-04-14T08:41:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.482+0000 7fc8656b2640 1 -- 172.21.15.172:0/3985994331 learned_addr learned my addr 172.21.15.172:0/3985994331 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.482+0000 7fc85ffff640 1 --1- 172.21.15.172:0/3985994331 >> v1:172.21.15.172:6789/0 conn(0x7fc860162540 0x7fc8600b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:57106/0 (socket says 172.21.15.172:57106) 2024-04-14T08:41:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.482+0000 7fc864eb1640 1 --2- 172.21.15.172:0/3985994331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86015e430 0x7fc860162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.482+0000 7fc8656b2640 1 -- 172.21.15.172:0/3985994331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86015e430 msgr2=0x7fc860162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.482+0000 7fc8656b2640 1 --2- 172.21.15.172:0/3985994331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86015e430 0x7fc860162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.482+0000 7fc8656b2640 1 -- 172.21.15.172:0/3985994331 >> v1:172.21.15.172:6789/0 conn(0x7fc860162540 legacy=0x7fc8600b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.482+0000 7fc85f7fe640 1 -- 172.21.15.172:0/3985994331 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2727875266 0 0) 0x7fc8600cefb0 con 0x7fc860162540 2024-04-14T08:41:54.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.482+0000 7fc8656b2640 1 -- 172.21.15.172:0/3985994331 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8600cf850 con 0x7fc8600d2e00 2024-04-14T08:41:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.483+0000 7fc8656b2640 1 --2- 172.21.15.172:0/3985994331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8600d2e00 0x7fc8601629a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fc858009e50 tx=0x7fc85802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b7811c5020505f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.483+0000 7fc85f7fe640 1 -- 172.21.15.172:0/3985994331 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85802cb00 con 0x7fc8600d2e00 2024-04-14T08:41:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.483+0000 7fc85f7fe640 1 -- 172.21.15.172:0/3985994331 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc85802cca0 con 0x7fc8600d2e00 2024-04-14T08:41:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.483+0000 7fc85f7fe640 1 -- 172.21.15.172:0/3985994331 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858034610 con 0x7fc8600d2e00 2024-04-14T08:41:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.483+0000 7fc867115640 1 -- 172.21.15.172:0/3985994331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8600d2e00 msgr2=0x7fc8601629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.483+0000 7fc867115640 1 --2- 172.21.15.172:0/3985994331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8600d2e00 0x7fc8601629a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fc858009e50 tx=0x7fc85802bf60 comp rx=0 tx=0).stop 2024-04-14T08:41:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.483+0000 7fc867115640 1 -- 172.21.15.172:0/3985994331 shutdown_connections 2024-04-14T08:41:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.483+0000 7fc867115640 1 --2- 172.21.15.172:0/3985994331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8600d2e00 0x7fc8601629a0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.483+0000 7fc867115640 1 --2- 172.21.15.172:0/3985994331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86015e430 0x7fc860162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.483+0000 7fc867115640 1 -- 172.21.15.172:0/3985994331 >> 172.21.15.172:0/3985994331 conn(0x7fc86009d560 msgr2=0x7fc8600d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.483+0000 7fc867115640 1 -- 172.21.15.172:0/3985994331 shutdown_connections 2024-04-14T08:41:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.483+0000 7fc867115640 1 -- 172.21.15.172:0/3985994331 wait complete. 2024-04-14T08:41:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.485+0000 7fc867115640 1 Processor -- start 2024-04-14T08:41:54.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.485+0000 7fc867115640 1 -- start start 2024-04-14T08:41:54.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.485+0000 7fc867115640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8600d2e00 0x7fc860155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.485+0000 7fc867115640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86015e430 0x7fc86015e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.485+0000 7fc867115640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc860162540 0x7fc8600ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.485+0000 7fc867115640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc8600d4e20 con 0x7fc8600d2e00 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.485+0000 7fc867115640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc86015e810 con 0x7fc860162540 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.485+0000 7fc867115640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc8600d0050 con 0x7fc86015e430 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc864eb1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8600d2e00 0x7fc860155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc864eb1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8600d2e00 0x7fc860155960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56414/0 (socket says 172.21.15.172:56414) 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc864eb1640 1 -- 172.21.15.172:0/1498918557 learned_addr learned my addr 172.21.15.172:0/1498918557 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc85ffff640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86015e430 0x7fc86015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc864eb1640 1 -- 172.21.15.172:0/1498918557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86015e430 msgr2=0x7fc86015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc8656b2640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc860162540 0x7fc8600ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc864eb1640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86015e430 0x7fc86015e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc864eb1640 1 -- 172.21.15.172:0/1498918557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc860162540 msgr2=0x7fc8600ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc864eb1640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc860162540 0x7fc8600ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc864eb1640 1 -- 172.21.15.172:0/1498918557 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8600b5bd0 con 0x7fc8600d2e00 2024-04-14T08:41:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc864eb1640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8600d2e00 0x7fc860155960 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc85400e9b0 tx=0x7fc85400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc85d7fa640 1 -- 172.21.15.172:0/1498918557 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854017070 con 0x7fc8600d2e00 2024-04-14T08:41:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc85d7fa640 1 -- 172.21.15.172:0/1498918557 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc85400c9a0 con 0x7fc8600d2e00 2024-04-14T08:41:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc867115640 1 -- 172.21.15.172:0/1498918557 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc860155ea0 con 0x7fc8600d2e00 2024-04-14T08:41:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc85d7fa640 1 -- 172.21.15.172:0/1498918557 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85400cd90 con 0x7fc8600d2e00 2024-04-14T08:41:54.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.486+0000 7fc867115640 1 -- 172.21.15.172:0/1498918557 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8600b81d0 con 0x7fc8600d2e00 2024-04-14T08:41:54.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.488+0000 7fc85d7fa640 1 -- 172.21.15.172:0/1498918557 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc85401a070 con 0x7fc8600d2e00 2024-04-14T08:41:54.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.488+0000 7fc85d7fa640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc84406c8f0 0x7fc84406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.488+0000 7fc85d7fa640 1 -- 172.21.15.172:0/1498918557 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7fc85408be20 con 0x7fc8600d2e00 2024-04-14T08:41:54.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.488+0000 7fc85ffff640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc84406c8f0 0x7fc84406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.489+0000 7fc85ffff640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc84406c8f0 0x7fc84406edb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc8600b9f20 tx=0x7fc84c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.489+0000 7fc869fdc740 1 Processor -- start 2024-04-14T08:41:54.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.489+0000 7fc869fdc740 1 -- start start 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc869fdc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 0x56349e50f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc869fdc740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e53cd50 0x56349e53f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc869fdc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e50fe50 0x56349e53fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc869fdc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56349e479800 con 0x56349e53c970 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc869fdc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56349e4217f0 con 0x56349e50fe50 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc869fdc740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56349e4757b0 con 0x56349e53cd50 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc864eb1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 0x56349e50f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc864eb1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 0x56349e50f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56420/0 (socket says 172.21.15.172:56420) 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc864eb1640 1 -- 172.21.15.172:0/3244576503 learned_addr learned my addr 172.21.15.172:0/3244576503 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc864eb1640 1 -- 172.21.15.172:0/3244576503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e53cd50 msgr2=0x56349e53f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc864eb1640 1 --2- 172.21.15.172:0/3244576503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e53cd50 0x56349e53f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc864eb1640 1 -- 172.21.15.172:0/3244576503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e50fe50 msgr2=0x56349e53fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc864eb1640 1 --2- 172.21.15.172:0/3244576503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e50fe50 0x56349e53fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc864eb1640 1 -- 172.21.15.172:0/3244576503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349e2577d0 con 0x56349e53c970 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc864eb1640 1 --2- 172.21.15.172:0/3244576503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 0x56349e50f910 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fc854062010 tx=0x7fc8540593f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89178806ab6e116 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc84a7fc640 1 -- 172.21.15.172:0/3244576503 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854021030 con 0x56349e53c970 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc84a7fc640 1 -- 172.21.15.172:0/3244576503 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc85405dd10 con 0x56349e53c970 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.490+0000 7fc84a7fc640 1 -- 172.21.15.172:0/3244576503 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854062540 con 0x56349e53c970 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.491+0000 7fc869fdc740 1 -- 172.21.15.172:0/3244576503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 msgr2=0x56349e50f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.491+0000 7fc869fdc740 1 --2- 172.21.15.172:0/3244576503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 0x56349e50f910 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fc854062010 tx=0x7fc8540593f0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.491+0000 7fc869fdc740 1 -- 172.21.15.172:0/3244576503 shutdown_connections 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.491+0000 7fc869fdc740 1 --2- 172.21.15.172:0/3244576503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e50fe50 0x56349e53fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.491+0000 7fc869fdc740 1 --2- 172.21.15.172:0/3244576503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e53cd50 0x56349e53f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.491+0000 7fc869fdc740 1 --2- 172.21.15.172:0/3244576503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 0x56349e50f910 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.491+0000 7fc869fdc740 1 -- 172.21.15.172:0/3244576503 >> 172.21.15.172:0/3244576503 conn(0x56349e511b80 msgr2=0x56349e55a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.491+0000 7fc869fdc740 1 -- 172.21.15.172:0/3244576503 shutdown_connections 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.491+0000 7fc869fdc740 1 -- 172.21.15.172:0/3244576503 wait complete. 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.492+0000 7fc869fdc740 1 -- start start 2024-04-14T08:41:54.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.492+0000 7fc869fdc740 1 Processor -- start 2024-04-14T08:41:54.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.492+0000 7fc869fdc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e53c970 0x56349e529bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc869fdc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 0x56349e535ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc869fdc740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e537960 0x56349e55da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc869fdc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56349e514120 con 0x56349e53cd50 2024-04-14T08:41:54.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc869fdc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56349e434d00 con 0x56349e53c970 2024-04-14T08:41:54.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc869fdc740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56349e513510 con 0x56349e537960 2024-04-14T08:41:54.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc869fdc740 10 client.0 _ll_register_callbacks cb 0x56349e4346f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:54.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc85ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 0x56349e535ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc85ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 0x56349e535ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56436/0 (socket says 172.21.15.172:56436) 2024-04-14T08:41:54.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc85ffff640 1 -- 172.21.15.172:0/462368631 learned_addr learned my addr 172.21.15.172:0/462368631 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc864eb1640 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e53c970 0x56349e529bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc8656b2640 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e537960 0x56349e55da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc85ffff640 1 -- 172.21.15.172:0/462368631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e537960 msgr2=0x56349e55da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc85ffff640 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e537960 0x56349e55da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc85ffff640 1 -- 172.21.15.172:0/462368631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e53c970 msgr2=0x56349e529bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc85ffff640 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e53c970 0x56349e529bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc85ffff640 1 -- 172.21.15.172:0/462368631 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349e18d3f0 con 0x56349e53cd50 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc85ffff640 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 0x56349e535ff0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc84c005a80 tx=0x7fc84c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af742e82ab64fa62 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc83f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc83f7fe640 1 -- 172.21.15.172:0/462368631 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c00de60 con 0x56349e53cd50 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc83f7fe640 1 -- 172.21.15.172:0/462368631 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7fc84c022cd0 con 0x56349e53cd50 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.493+0000 7fc869fdc740 1 -- 172.21.15.172:0/462368631 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc85405dd10 con 0x56349e53cd50 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc83f7fe640 1 -- 172.21.15.172:0/462368631 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fc84c012410 con 0x56349e53cd50 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc869fdc740 10 client.8841 fetch_fsmap learned FSMap version 80 2024-04-14T08:41:54.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc869fdc740 1 -- 172.21.15.172:0/462368631 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) -- 0x56349e14d0f0 con 0x56349e53cd50 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc83f7fe640 1 -- 172.21.15.172:0/462368631 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc85405dd10 con 0x56349e53cd50 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc83f7fe640 1 -- 172.21.15.172:0/462368631 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 80) ==== 3494+0+0 (secure 0 0 0) 0x7fc84c029050 con 0x56349e53cd50 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc869fdc740 10 client.8841 fetch_fsmap finished waiting for FSMap version 80 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc869fdc740 10 client.8841 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc869fdc740 20 client.8841 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc869fdc740 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56349e53ec00 0x56349e53f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc869fdc740 4 client.8841 mds_command: new command op to 8382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc869fdc740 1 -- 172.21.15.172:0/462368631 --> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56349e486d40 con 0x56349e53ec00 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc864eb1640 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56349e53ec00 0x56349e53f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc83f7fe640 10 client.8841 ms_handle_connect on v2:172.21.15.172:6835/4028911489 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.494+0000 7fc864eb1640 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56349e53ec00 0x56349e53f000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c32a46948c59bd1b server_cookie=4f8d1ca734751578 in_seq=0 out_seq=0 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.495+0000 7fc83f7fe640 1 -- 172.21.15.172:0/462368631 <== mds.0 v2:172.21.15.172:6835/4028911489 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56349e486d40 con 0x56349e53ec00 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.495+0000 7fc83f7fe640 10 client.8841 handle_command_reply: tid=0 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.495+0000 7fc869fdc740 1 client.8841 shutdown 2024-04-14T08:41:54.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.495+0000 7fc869fdc740 1 -- 172.21.15.172:0/462368631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 msgr2=0x56349e535ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.495+0000 7fc869fdc740 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 0x56349e535ff0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc84c005a80 tx=0x7fc84c005e80 comp rx=0 tx=0).stop 2024-04-14T08:41:54.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.495+0000 7fc869fdc740 1 -- 172.21.15.172:0/462368631 shutdown_connections 2024-04-14T08:41:54.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.495+0000 7fc869fdc740 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56349e53ec00 0x56349e53f000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.495+0000 7fc869fdc740 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e537960 0x56349e55da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.495+0000 7fc869fdc740 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 0x56349e535ff0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.495+0000 7fc869fdc740 1 --2- 172.21.15.172:0/462368631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e53c970 0x56349e529bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.495+0000 7fc83f7fe640 0 client.8841 ms_handle_reset on v2:172.21.15.172:6835/4028911489 2024-04-14T08:41:54.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.495+0000 7fc869fdc740 1 -- 172.21.15.172:0/462368631 >> 172.21.15.172:0/462368631 conn(0x56349e50fe50 msgr2=0x56349e514610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.496+0000 7fc869fdc740 1 -- 172.21.15.172:0/462368631 shutdown_connections 2024-04-14T08:41:54.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.496+0000 7fc869fdc740 1 -- 172.21.15.172:0/462368631 wait complete. 2024-04-14T08:41:54.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.496+0000 7fc869fdc740 20 client.8841 trim_cache size 0 max 16384 2024-04-14T08:41:54.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.513+0000 7fc869fdc740 1 Processor -- start 2024-04-14T08:41:54.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.513+0000 7fc869fdc740 1 -- start start 2024-04-14T08:41:54.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc869fdc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e537960 0x56349e529090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc869fdc740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e53c970 0x56349e515750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc869fdc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 0x56349e515c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc869fdc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc860201130 con 0x56349e53cd50 2024-04-14T08:41:54.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc869fdc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc860200150 con 0x56349e537960 2024-04-14T08:41:54.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc869fdc740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc8600c8ac0 con 0x56349e53c970 2024-04-14T08:41:54.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc8656b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 0x56349e515c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc8656b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 0x56349e515c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56444/0 (socket says 172.21.15.172:56444) 2024-04-14T08:41:54.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc8656b2640 1 -- 172.21.15.172:0/1286736838 learned_addr learned my addr 172.21.15.172:0/1286736838 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc85ffff640 1 --2- 172.21.15.172:0/1286736838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e53c970 0x56349e515750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc864eb1640 1 --2- 172.21.15.172:0/1286736838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e537960 0x56349e529090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc8656b2640 1 -- 172.21.15.172:0/1286736838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e53c970 msgr2=0x56349e515750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc8656b2640 1 --2- 172.21.15.172:0/1286736838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e53c970 0x56349e515750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc8656b2640 1 -- 172.21.15.172:0/1286736838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e537960 msgr2=0x56349e529090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc8656b2640 1 --2- 172.21.15.172:0/1286736838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e537960 0x56349e529090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc8656b2640 1 -- 172.21.15.172:0/1286736838 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349e53c320 con 0x56349e53cd50 2024-04-14T08:41:54.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.514+0000 7fc8656b2640 1 --2- 172.21.15.172:0/1286736838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 0x56349e515c90 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc858009e50 tx=0x7fc85802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d1020e9879b359 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.515+0000 7fc849ffb640 1 -- 172.21.15.172:0/1286736838 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858043070 con 0x56349e53cd50 2024-04-14T08:41:54.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.515+0000 7fc849ffb640 1 -- 172.21.15.172:0/1286736838 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc85802ce50 con 0x56349e53cd50 2024-04-14T08:41:54.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.515+0000 7fc849ffb640 1 -- 172.21.15.172:0/1286736838 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858003990 con 0x56349e53cd50 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.515+0000 7fc869fdc740 1 -- 172.21.15.172:0/1286736838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 msgr2=0x56349e515c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.515+0000 7fc869fdc740 1 --2- 172.21.15.172:0/1286736838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 0x56349e515c90 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc858009e50 tx=0x7fc85802e070 comp rx=0 tx=0).stop 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.515+0000 7fc869fdc740 1 -- 172.21.15.172:0/1286736838 shutdown_connections 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.515+0000 7fc869fdc740 1 --2- 172.21.15.172:0/1286736838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53cd50 0x56349e515c90 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.515+0000 7fc869fdc740 1 --2- 172.21.15.172:0/1286736838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e53c970 0x56349e515750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.515+0000 7fc869fdc740 1 --2- 172.21.15.172:0/1286736838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e537960 0x56349e529090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.515+0000 7fc869fdc740 1 -- 172.21.15.172:0/1286736838 >> 172.21.15.172:0/1286736838 conn(0x56349e50fe50 msgr2=0x56349e5406f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.515+0000 7fc869fdc740 1 -- 172.21.15.172:0/1286736838 shutdown_connections 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.515+0000 7fc869fdc740 1 -- 172.21.15.172:0/1286736838 wait complete. 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.516+0000 7fc869fdc740 1 -- start start 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.516+0000 7fc869fdc740 1 Processor -- start 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc869fdc740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e537960 0x56349e57e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc869fdc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 0x56349e5404e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc869fdc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e53cd50 0x56349e540a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc869fdc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56349e486d40 con 0x56349e53c970 2024-04-14T08:41:54.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc869fdc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc8600b9590 con 0x56349e53cd50 2024-04-14T08:41:54.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc869fdc740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56349e5112e0 con 0x56349e537960 2024-04-14T08:41:54.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc869fdc740 10 client.0 _ll_register_callbacks cb 0x7fc84c001270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:41:54.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc85ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 0x56349e5404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc85ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 0x56349e5404e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56460/0 (socket says 172.21.15.172:56460) 2024-04-14T08:41:54.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc85ffff640 1 -- 172.21.15.172:0/551388719 learned_addr learned my addr 172.21.15.172:0/551388719 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:41:54.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc864eb1640 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e537960 0x56349e57e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc8656b2640 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e53cd50 0x56349e540a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc85ffff640 1 -- 172.21.15.172:0/551388719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e537960 msgr2=0x56349e57e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc85ffff640 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e537960 0x56349e57e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc85ffff640 1 -- 172.21.15.172:0/551388719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e53cd50 msgr2=0x56349e540a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc85ffff640 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e53cd50 0x56349e540a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc85ffff640 1 -- 172.21.15.172:0/551388719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349e53f630 con 0x56349e53c970 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc85ffff640 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 0x56349e5404e0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0010c0 tx=0x7fc84c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f4b1924f9c13ae server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.517+0000 7fc83f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc83f7fe640 1 -- 172.21.15.172:0/551388719 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c012d90 con 0x56349e53c970 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc83f7fe640 1 -- 172.21.15.172:0/551388719 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7fc84c02b470 con 0x56349e53c970 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc869fdc740 1 -- 172.21.15.172:0/551388719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc85802ce50 con 0x56349e53c970 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc83f7fe640 1 -- 172.21.15.172:0/551388719 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fc84c00d150 con 0x56349e53c970 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc869fdc740 10 client.8847 fetch_fsmap learned FSMap version 80 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc869fdc740 1 -- 172.21.15.172:0/551388719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) -- 0x56349e535c40 con 0x56349e53c970 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc83f7fe640 1 -- 172.21.15.172:0/551388719 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc85802ce50 con 0x56349e53c970 2024-04-14T08:41:54.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc83f7fe640 1 -- 172.21.15.172:0/551388719 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 80) ==== 3494+0+0 (secure 0 0 0) 0x7fc84c029040 con 0x56349e53c970 2024-04-14T08:41:54.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc869fdc740 10 client.8847 fetch_fsmap finished waiting for FSMap version 80 2024-04-14T08:41:54.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc869fdc740 10 client.8847 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-14T08:41:54.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc869fdc740 20 client.8847 populate_metadata read hostname 'smithi172' 2024-04-14T08:41:54.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc869fdc740 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56349e5ba560 0x56349e53adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:41:54.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc869fdc740 4 client.8847 mds_command: new command op to 8382 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:41:54.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc869fdc740 1 -- 172.21.15.172:0/551388719 --> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56349e481090 con 0x56349e5ba560 2024-04-14T08:41:54.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc864eb1640 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56349e5ba560 0x56349e53adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:41:54.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.518+0000 7fc83f7fe640 10 client.8847 ms_handle_connect on v2:172.21.15.172:6835/4028911489 2024-04-14T08:41:54.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc864eb1640 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56349e5ba560 0x56349e53adb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157a91ed6e1b0ed6 server_cookie=d6c9d3e4ddc8dc01 in_seq=0 out_seq=0 2024-04-14T08:41:54.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc83f7fe640 1 -- 172.21.15.172:0/551388719 <== mds.0 v2:172.21.15.172:6835/4028911489 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x56349e481090 con 0x56349e5ba560 2024-04-14T08:41:54.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc83f7fe640 10 client.8847 handle_command_reply: tid=0 2024-04-14T08:41:54.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc869fdc740 1 client.8847 shutdown 2024-04-14T08:41:54.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc869fdc740 1 -- 172.21.15.172:0/551388719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 msgr2=0x56349e5404e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc869fdc740 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 0x56349e5404e0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0010c0 tx=0x7fc84c011000 comp rx=0 tx=0).stop 2024-04-14T08:41:54.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc869fdc740 1 -- 172.21.15.172:0/551388719 shutdown_connections 2024-04-14T08:41:54.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc869fdc740 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x56349e5ba560 0x56349e53adb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc869fdc740 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56349e53cd50 0x56349e540a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc869fdc740 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56349e53c970 0x56349e5404e0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc83f7fe640 0 client.8847 ms_handle_reset on v2:172.21.15.172:6835/4028911489 2024-04-14T08:41:54.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc869fdc740 1 --2- 172.21.15.172:0/551388719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56349e537960 0x56349e57e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc869fdc740 1 -- 172.21.15.172:0/551388719 >> 172.21.15.172:0/551388719 conn(0x56349e50fe50 msgr2=0x56349e5346b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc869fdc740 1 -- 172.21.15.172:0/551388719 shutdown_connections 2024-04-14T08:41:54.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc869fdc740 1 -- 172.21.15.172:0/551388719 wait complete. 2024-04-14T08:41:54.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.519+0000 7fc869fdc740 20 client.8847 trim_cache size 0 max 16384 2024-04-14T08:41:54.530 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:41:54.530 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-14T08:41:54.530 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-04-14T08:41:54.530 INFO:teuthology.orchestra.run.smithi172.stdout: "f46b6eaa-5b44-4fb7-b309-6c943446cfaf": { 2024-04-14T08:41:54.530 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-04-14T08:41:54.531 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "f46b6eaa-5b44-4fb7-b309-6c943446cfaf", 2024-04-14T08:41:54.531 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,repair" 2024-04-14T08:41:54.531 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:54.531 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:41:54.531 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:41:54.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.520+0000 7fc867115640 1 -- 172.21.15.172:0/1498918557 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc84406c8f0 msgr2=0x7fc84406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.520+0000 7fc867115640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc84406c8f0 0x7fc84406edb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc8600b9f20 tx=0x7fc84c009040 comp rx=0 tx=0).stop 2024-04-14T08:41:54.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.520+0000 7fc867115640 1 -- 172.21.15.172:0/1498918557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8600d2e00 msgr2=0x7fc860155960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:41:54.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.520+0000 7fc867115640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8600d2e00 0x7fc860155960 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc85400e9b0 tx=0x7fc85400ee70 comp rx=0 tx=0).stop 2024-04-14T08:41:54.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.520+0000 7fc867115640 1 -- 172.21.15.172:0/1498918557 shutdown_connections 2024-04-14T08:41:54.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.520+0000 7fc867115640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc84406c8f0 0x7fc84406edb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.520+0000 7fc867115640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc860162540 0x7fc8600ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.520+0000 7fc867115640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc86015e430 0x7fc86015e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.520+0000 7fc867115640 1 --2- 172.21.15.172:0/1498918557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8600d2e00 0x7fc860155960 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:41:54.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.520+0000 7fc867115640 1 -- 172.21.15.172:0/1498918557 >> 172.21.15.172:0/1498918557 conn(0x7fc86009d560 msgr2=0x7fc8600b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:41:54.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.520+0000 7fc867115640 1 -- 172.21.15.172:0/1498918557 shutdown_connections 2024-04-14T08:41:54.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:54.520+0000 7fc867115640 1 -- 172.21.15.172:0/1498918557 wait complete. 2024-04-14T08:41:54.533 INFO:tasks.cephfs.filesystem:scrub status for tag:f46b6eaa-5b44-4fb7-b309-6c943446cfaf - {'path': '/', 'tag': 'f46b6eaa-5b44-4fb7-b309-6c943446cfaf', 'options': 'recursive,repair'} 2024-04-14T08:41:55.226 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:41:55.228 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:41:55.416 INFO:tasks.ceph.mds.a.smithi172.stderr:2024-04-14T08:41:55.415+0000 7fa06b3f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:41:55.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:41:55.784+0000 7f9b21ffb640 1 -- 172.21.15.172:0/1959802918 <== mon.0 v2:172.21.15.172:3300/0 8 ==== log(9 entries from seq 1 at 2024-04-14T08:41:54.359871+0000) ==== 1558+0+0 (secure 0 0 0) 0x7f9b24057780 con 0x7f9b2c15e4b0 2024-04-14T08:41:55.786 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:54.359871+0000 mds.a [INF] scrub queued for path: / 2024-04-14T08:41:55.786 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:54.359874+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-14T08:41:55.786 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:54.359879+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-14T08:41:55.786 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:54.360436+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-14T08:41:55.786 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:54.360443+0000 mds.a [ERR] inode table repaired for inode: 0x10000000001 2024-04-14T08:41:55.786 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:54.360653+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-14T08:41:55.786 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:54.360660+0000 mds.a [ERR] inode table repaired for inode: 0x10000000002 2024-04-14T08:41:55.786 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:54.361121+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-14T08:41:55.786 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:41:55.097629+0000 mds.a [INF] scrub summary: idle 2024-04-14T08:42:24.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:24.534 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-14T08:42:24.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.639+0000 7f03d80ad640 1 Processor -- start 2024-04-14T08:42:24.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.639+0000 7f03d80ad640 1 -- start start 2024-04-14T08:42:24.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.640+0000 7f03d80ad640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f03d015e430 0x7f03d0162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.640+0000 7f03d80ad640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d00cefb0 con 0x7f03d0162540 2024-04-14T08:42:24.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.640+0000 7f03d80ad640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d00cf290 con 0x7f03d00d2e00 2024-04-14T08:42:24.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.640+0000 7f03d80ad640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f03d00cf570 con 0x7f03d015e430 2024-04-14T08:42:24.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.640+0000 7f03d5e49640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f03d015e430 0x7f03d0162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.640+0000 7f03d5648640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f03d0162540 0x7f03d00b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:37460/0 (socket says 172.21.15.172:37460) 2024-04-14T08:42:24.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.640+0000 7f03d5648640 1 -- 172.21.15.172:0/3621709039 learned_addr learned my addr 172.21.15.172:0/3621709039 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:24.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.641+0000 7f03d5e49640 1 -- 172.21.15.172:0/3621709039 >> v1:172.21.15.177:6789/0 conn(0x7f03d00d2e00 legacy=0x7f03d00b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.641+0000 7f03d5e49640 1 -- 172.21.15.172:0/3621709039 >> v1:172.21.15.172:6789/0 conn(0x7f03d0162540 legacy=0x7f03d00b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.641+0000 7f03d5e49640 1 -- 172.21.15.172:0/3621709039 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d00cf850 con 0x7f03d015e430 2024-04-14T08:42:24.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.641+0000 7f03d4e47640 1 -- 172.21.15.172:0/3621709039 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2968033018 0 0) 0x7f03d00cefb0 con 0x7f03d0162540 2024-04-14T08:42:24.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.642+0000 7f03d5e49640 1 --2- 172.21.15.172:0/3621709039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d015e430 0x7f03d0162000 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0097a0 tx=0x7f03cc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f816c7a75ecbef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:24.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.642+0000 7f03d4e47640 1 -- 172.21.15.172:0/3621709039 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc034070 con 0x7f03d015e430 2024-04-14T08:42:24.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.642+0000 7f03d4e47640 1 -- 172.21.15.172:0/3621709039 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03cc030610 con 0x7f03d015e430 2024-04-14T08:42:24.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.642+0000 7f03d4e47640 1 -- 172.21.15.172:0/3621709039 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc0309e0 con 0x7f03d015e430 2024-04-14T08:42:24.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.643+0000 7f03d80ad640 1 -- 172.21.15.172:0/3621709039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d015e430 msgr2=0x7f03d0162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.643+0000 7f03d80ad640 1 --2- 172.21.15.172:0/3621709039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d015e430 0x7f03d0162000 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0097a0 tx=0x7f03cc02b960 comp rx=0 tx=0).stop 2024-04-14T08:42:24.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.643+0000 7f03d80ad640 1 -- 172.21.15.172:0/3621709039 shutdown_connections 2024-04-14T08:42:24.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.643+0000 7f03d80ad640 1 --2- 172.21.15.172:0/3621709039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d015e430 0x7f03d0162000 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.643+0000 7f03d80ad640 1 -- 172.21.15.172:0/3621709039 >> 172.21.15.172:0/3621709039 conn(0x7f03d009d560 msgr2=0x7f03d00d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:24.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.643+0000 7f03d80ad640 1 -- 172.21.15.172:0/3621709039 shutdown_connections 2024-04-14T08:42:24.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.643+0000 7f03d80ad640 1 -- 172.21.15.172:0/3621709039 wait complete. 2024-04-14T08:42:24.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.644+0000 7f03d80ad640 1 Processor -- start 2024-04-14T08:42:24.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.644+0000 7f03d80ad640 1 -- start start 2024-04-14T08:42:24.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d80ad640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03d00d2e00 0x7f03d01f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d80ad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03d015e430 0x7f03d0152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d80ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d0162540 0x7f03d00b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d80ad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f03d00d4e20 con 0x7f03d0162540 2024-04-14T08:42:24.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d80ad640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f03d015e810 con 0x7f03d00d2e00 2024-04-14T08:42:24.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d80ad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f03d00b4850 con 0x7f03d015e430 2024-04-14T08:42:24.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d664a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d0162540 0x7f03d00b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d664a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d0162540 0x7f03d00b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35114/0 (socket says 172.21.15.172:35114) 2024-04-14T08:42:24.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d664a640 1 -- 172.21.15.172:0/93293560 learned_addr learned my addr 172.21.15.172:0/93293560 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:24.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d5e49640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03d00d2e00 0x7f03d01f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d5648640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03d015e430 0x7f03d0152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d664a640 1 -- 172.21.15.172:0/93293560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03d015e430 msgr2=0x7f03d0152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d664a640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03d015e430 0x7f03d0152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d664a640 1 -- 172.21.15.172:0/93293560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03d00d2e00 msgr2=0x7f03d01f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d664a640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03d00d2e00 0x7f03d01f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d664a640 1 -- 172.21.15.172:0/93293560 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d01f7fa0 con 0x7f03d0162540 2024-04-14T08:42:24.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d664a640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d0162540 0x7f03d00b1ea0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f03c800c7a0 tx=0x7f03c800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:24.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03c67fc640 1 -- 172.21.15.172:0/93293560 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8003b70 con 0x7f03d0162540 2024-04-14T08:42:24.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03c67fc640 1 -- 172.21.15.172:0/93293560 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c8003d30 con 0x7f03d0162540 2024-04-14T08:42:24.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d80ad640 1 -- 172.21.15.172:0/93293560 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d01f8280 con 0x7f03d0162540 2024-04-14T08:42:24.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03c67fc640 1 -- 172.21.15.172:0/93293560 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c800aa90 con 0x7f03d0162540 2024-04-14T08:42:24.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.645+0000 7f03d80ad640 1 -- 172.21.15.172:0/93293560 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d01f8840 con 0x7f03d0162540 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.647+0000 7f03c67fc640 1 -- 172.21.15.172:0/93293560 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03c800ac30 con 0x7f03d0162540 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.648+0000 7f03c67fc640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f03b406c8f0 0x7f03b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.648+0000 7f03c67fc640 1 -- 172.21.15.172:0/93293560 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7f03c808bb60 con 0x7f03d0162540 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.648+0000 7f03d5e49640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f03b406c8f0 0x7f03b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.648+0000 7f03daf6f740 1 Processor -- start 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.648+0000 7f03daf6f740 1 -- start start 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.649+0000 7f03d5e49640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f03b406c8f0 0x7f03b406edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f03cc036000 tx=0x7f03cc03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.649+0000 7f03daf6f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4a6970 0x556a1f479910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.649+0000 7f03daf6f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 0x556a1f4a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.649+0000 7f03daf6f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f479e50 0x556a1f4a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.649+0000 7f03daf6f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556a1f3e3800 con 0x556a1f479e50 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.649+0000 7f03daf6f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556a1f38b7f0 con 0x556a1f4a6970 2024-04-14T08:42:24.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.649+0000 7f03daf6f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x556a1f3df7b0 con 0x556a1f4a6d50 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.649+0000 7f03d664a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f479e50 0x556a1f4a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.649+0000 7f03d664a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f479e50 0x556a1f4a9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35126/0 (socket says 172.21.15.172:35126) 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.649+0000 7f03d664a640 1 -- 172.21.15.172:0/734032823 learned_addr learned my addr 172.21.15.172:0/734032823 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.649+0000 7f03d5e49640 1 --2- 172.21.15.172:0/734032823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4a6970 0x556a1f479910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.649+0000 7f03d5648640 1 --2- 172.21.15.172:0/734032823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 0x556a1f4a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.650+0000 7f03d664a640 1 -- 172.21.15.172:0/734032823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 msgr2=0x556a1f4a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.650+0000 7f03d664a640 1 --2- 172.21.15.172:0/734032823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 0x556a1f4a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.650+0000 7f03d664a640 1 -- 172.21.15.172:0/734032823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4a6970 msgr2=0x556a1f479910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.650+0000 7f03d664a640 1 --2- 172.21.15.172:0/734032823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4a6970 0x556a1f479910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.650+0000 7f03d664a640 1 -- 172.21.15.172:0/734032823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a1f1c17d0 con 0x556a1f479e50 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.650+0000 7f03d664a640 1 --2- 172.21.15.172:0/734032823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f479e50 0x556a1f4a9c40 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f03c8089e00 tx=0x7f03c8059640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7b4ec6e8f708a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.650+0000 7f03b37fe640 1 -- 172.21.15.172:0/734032823 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8088070 con 0x556a1f479e50 2024-04-14T08:42:24.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.650+0000 7f03b37fe640 1 -- 172.21.15.172:0/734032823 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c805da40 con 0x556a1f479e50 2024-04-14T08:42:24.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.650+0000 7f03b37fe640 1 -- 172.21.15.172:0/734032823 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8062270 con 0x556a1f479e50 2024-04-14T08:42:24.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.650+0000 7f03daf6f740 1 -- 172.21.15.172:0/734032823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f479e50 msgr2=0x556a1f4a9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.650+0000 7f03daf6f740 1 --2- 172.21.15.172:0/734032823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f479e50 0x556a1f4a9c40 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f03c8089e00 tx=0x7f03c8059640 comp rx=0 tx=0).stop 2024-04-14T08:42:24.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.651+0000 7f03daf6f740 1 -- 172.21.15.172:0/734032823 shutdown_connections 2024-04-14T08:42:24.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.651+0000 7f03daf6f740 1 --2- 172.21.15.172:0/734032823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f479e50 0x556a1f4a9c40 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.651+0000 7f03daf6f740 1 --2- 172.21.15.172:0/734032823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 0x556a1f4a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.651+0000 7f03daf6f740 1 --2- 172.21.15.172:0/734032823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4a6970 0x556a1f479910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.651+0000 7f03daf6f740 1 -- 172.21.15.172:0/734032823 >> 172.21.15.172:0/734032823 conn(0x556a1f47bb80 msgr2=0x556a1f4c4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:24.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.651+0000 7f03daf6f740 1 -- 172.21.15.172:0/734032823 shutdown_connections 2024-04-14T08:42:24.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.651+0000 7f03daf6f740 1 -- 172.21.15.172:0/734032823 wait complete. 2024-04-14T08:42:24.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.651+0000 7f03daf6f740 1 -- start start 2024-04-14T08:42:24.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.651+0000 7f03daf6f740 1 Processor -- start 2024-04-14T08:42:24.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.652+0000 7f03daf6f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6970 0x556a1f4d53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.652+0000 7f03daf6f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 0x556a1f4a2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.652+0000 7f03daf6f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f49e6e0 0x556a1f49eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.652+0000 7f03daf6f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556a1f47e120 con 0x556a1f4a6970 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.652+0000 7f03daf6f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556a1f39ed00 con 0x556a1f49e6e0 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.652+0000 7f03daf6f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x556a1f47d510 con 0x556a1f4a6d50 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.652+0000 7f03daf6f740 10 client.0 _ll_register_callbacks cb 0x556a1f39e6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.652+0000 7f03d5e49640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6970 0x556a1f4d53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.652+0000 7f03d5e49640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6970 0x556a1f4d53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35138/0 (socket says 172.21.15.172:35138) 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.652+0000 7f03d5e49640 1 -- 172.21.15.172:0/983179931 learned_addr learned my addr 172.21.15.172:0/983179931 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.652+0000 7f03d5648640 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 0x556a1f4a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.652+0000 7f03d664a640 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f49e6e0 0x556a1f49eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03d5e49640 1 -- 172.21.15.172:0/983179931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 msgr2=0x556a1f4a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:24.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03d5e49640 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 0x556a1f4a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03d5e49640 1 -- 172.21.15.172:0/983179931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f49e6e0 msgr2=0x556a1f49eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03d5e49640 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f49e6e0 0x556a1f49eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03d5e49640 1 -- 172.21.15.172:0/983179931 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a1f0f73f0 con 0x556a1f4a6970 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03d664a640 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f49e6e0 0x556a1f49eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03d5e49640 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6970 0x556a1f4d53c0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009890 tx=0x7f03cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bcdddb8e1f820a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03b0ff9640 1 -- 172.21.15.172:0/983179931 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc034040 con 0x556a1f4a6970 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03b0ff9640 1 -- 172.21.15.172:0/983179931 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7f03cc051910 con 0x556a1f4a6970 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03daf6f740 1 -- 172.21.15.172:0/983179931 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c805da40 con 0x556a1f4a6970 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03b0ff9640 1 -- 172.21.15.172:0/983179931 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f03cc05e050 con 0x556a1f4a6970 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03daf6f740 10 client.8862 fetch_fsmap learned FSMap version 80 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03daf6f740 1 -- 172.21.15.172:0/983179931 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) -- 0x556a1f0b70f0 con 0x556a1f4a6970 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03b0ff9640 1 -- 172.21.15.172:0/983179931 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c805da40 con 0x556a1f4a6970 2024-04-14T08:42:24.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03b0ff9640 1 -- 172.21.15.172:0/983179931 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 80) ==== 3494+0+0 (secure 0 0 0) 0x7f03cc061020 con 0x556a1f4a6970 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03daf6f740 10 client.8862 fetch_fsmap finished waiting for FSMap version 80 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03daf6f740 10 client.8862 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03daf6f740 20 client.8862 populate_metadata read hostname 'smithi172' 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03daf6f740 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x556a1f4a89d0 0x556a1f4ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03daf6f740 4 client.8862 mds_command: new command op to 8382 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.653+0000 7f03daf6f740 1 -- 172.21.15.172:0/983179931 --> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a1f3f0d40 con 0x556a1f4a89d0 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.654+0000 7f03d5648640 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x556a1f4a89d0 0x556a1f4ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.654+0000 7f03b0ff9640 10 client.8862 ms_handle_connect on v2:172.21.15.172:6835/4028911489 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.654+0000 7f03d5648640 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x556a1f4a89d0 0x556a1f4ac830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5969f19a8f434879 server_cookie=4a076ee0b3e440ed in_seq=0 out_seq=0 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.654+0000 7f03b0ff9640 1 -- 172.21.15.172:0/983179931 <== mds.0 v2:172.21.15.172:6835/4028911489 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a1f3f0d40 con 0x556a1f4a89d0 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.654+0000 7f03b0ff9640 10 client.8862 handle_command_reply: tid=0 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.654+0000 7f03daf6f740 1 client.8862 shutdown 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.655+0000 7f03daf6f740 1 -- 172.21.15.172:0/983179931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6970 msgr2=0x556a1f4d53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.655+0000 7f03daf6f740 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6970 0x556a1f4d53c0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009890 tx=0x7f03cc004730 comp rx=0 tx=0).stop 2024-04-14T08:42:24.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.655+0000 7f03daf6f740 1 -- 172.21.15.172:0/983179931 shutdown_connections 2024-04-14T08:42:24.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.655+0000 7f03daf6f740 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x556a1f4a89d0 0x556a1f4ac830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.655+0000 7f03daf6f740 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f49e6e0 0x556a1f49eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.655+0000 7f03b0ff9640 0 client.8862 ms_handle_reset on v2:172.21.15.172:6835/4028911489 2024-04-14T08:42:24.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.655+0000 7f03daf6f740 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 0x556a1f4a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.655+0000 7f03daf6f740 1 --2- 172.21.15.172:0/983179931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6970 0x556a1f4d53c0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.655+0000 7f03daf6f740 1 -- 172.21.15.172:0/983179931 >> 172.21.15.172:0/983179931 conn(0x556a1f479e50 msgr2=0x556a1f4c7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:24.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.655+0000 7f03daf6f740 1 -- 172.21.15.172:0/983179931 shutdown_connections 2024-04-14T08:42:24.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.655+0000 7f03daf6f740 1 -- 172.21.15.172:0/983179931 wait complete. 2024-04-14T08:42:24.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.655+0000 7f03daf6f740 20 client.8862 trim_cache size 0 max 16384 2024-04-14T08:42:24.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.672+0000 7f03daf6f740 1 Processor -- start 2024-04-14T08:42:24.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.672+0000 7f03daf6f740 1 -- start start 2024-04-14T08:42:24.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03daf6f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f49e6e0 0x556a1f4a7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03daf6f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4a6970 0x556a1f4a7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03daf6f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 0x556a1f4d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03daf6f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f03d0201140 con 0x556a1f49e6e0 2024-04-14T08:42:24.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03daf6f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f03d02001b0 con 0x556a1f4a6970 2024-04-14T08:42:24.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03daf6f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f03d00b7790 con 0x556a1f4a6d50 2024-04-14T08:42:24.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03d5e49640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f49e6e0 0x556a1f4a7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03d5e49640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f49e6e0 0x556a1f4a7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35142/0 (socket says 172.21.15.172:35142) 2024-04-14T08:42:24.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03d5e49640 1 -- 172.21.15.172:0/11410890 learned_addr learned my addr 172.21.15.172:0/11410890 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:24.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03d664a640 1 --2- 172.21.15.172:0/11410890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 0x556a1f4d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03d5648640 1 --2- 172.21.15.172:0/11410890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4a6970 0x556a1f4a7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03d5e49640 1 -- 172.21.15.172:0/11410890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 msgr2=0x556a1f4d2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03d5e49640 1 --2- 172.21.15.172:0/11410890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 0x556a1f4d2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03d5e49640 1 -- 172.21.15.172:0/11410890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4a6970 msgr2=0x556a1f4a7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03d5e49640 1 --2- 172.21.15.172:0/11410890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4a6970 0x556a1f4a7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03d5e49640 1 -- 172.21.15.172:0/11410890 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a1f4e9cd0 con 0x556a1f49e6e0 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03d5648640 1 --2- 172.21.15.172:0/11410890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4a6970 0x556a1f4a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.673+0000 7f03d5e49640 1 --2- 172.21.15.172:0/11410890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f49e6e0 0x556a1f4a7130 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f03cc051cc0 tx=0x7f03cc047070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a14b59682e9d397 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.674+0000 7f03b2ffd640 1 -- 172.21.15.172:0/11410890 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc066070 con 0x556a1f49e6e0 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.674+0000 7f03b2ffd640 1 -- 172.21.15.172:0/11410890 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03cc056070 con 0x556a1f49e6e0 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.674+0000 7f03b2ffd640 1 -- 172.21.15.172:0/11410890 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc034040 con 0x556a1f49e6e0 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.674+0000 7f03daf6f740 1 -- 172.21.15.172:0/11410890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f49e6e0 msgr2=0x556a1f4a7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.674+0000 7f03daf6f740 1 --2- 172.21.15.172:0/11410890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f49e6e0 0x556a1f4a7130 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f03cc051cc0 tx=0x7f03cc047070 comp rx=0 tx=0).stop 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.674+0000 7f03daf6f740 1 -- 172.21.15.172:0/11410890 shutdown_connections 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.674+0000 7f03daf6f740 1 --2- 172.21.15.172:0/11410890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6d50 0x556a1f4d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.674+0000 7f03daf6f740 1 --2- 172.21.15.172:0/11410890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4a6970 0x556a1f4a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.674+0000 7f03daf6f740 1 --2- 172.21.15.172:0/11410890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f49e6e0 0x556a1f4a7130 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.674+0000 7f03daf6f740 1 -- 172.21.15.172:0/11410890 >> 172.21.15.172:0/11410890 conn(0x556a1f479e50 msgr2=0x556a1f4ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:24.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.674+0000 7f03daf6f740 1 -- 172.21.15.172:0/11410890 shutdown_connections 2024-04-14T08:42:24.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.674+0000 7f03daf6f740 1 -- 172.21.15.172:0/11410890 wait complete. 2024-04-14T08:42:24.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.675+0000 7f03daf6f740 1 -- start start 2024-04-14T08:42:24.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.675+0000 7f03daf6f740 1 Processor -- start 2024-04-14T08:42:24.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03daf6f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6970 0x556a1f53e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03daf6f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6d50 0x556a1f4e48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03daf6f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4c7c10 0x556a1f47e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03daf6f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556a1f4c8b30 con 0x556a1f4a6d50 2024-04-14T08:42:24.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03daf6f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f03d00b7de0 con 0x556a1f4c7c10 2024-04-14T08:42:24.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03daf6f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x556a1f3f1470 con 0x556a1f4a6970 2024-04-14T08:42:24.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03daf6f740 10 client.0 _ll_register_callbacks cb 0x7f03cc039df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:42:24.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03d5648640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6d50 0x556a1f4e48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03d5648640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6d50 0x556a1f4e48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35144/0 (socket says 172.21.15.172:35144) 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03d5648640 1 -- 172.21.15.172:0/842056030 learned_addr learned my addr 172.21.15.172:0/842056030 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03d5e49640 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6970 0x556a1f53e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03d664a640 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4c7c10 0x556a1f47e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03d5648640 1 -- 172.21.15.172:0/842056030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6970 msgr2=0x556a1f53e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03d5648640 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6970 0x556a1f53e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03d5648640 1 -- 172.21.15.172:0/842056030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4c7c10 msgr2=0x556a1f47e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03d5648640 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4c7c10 0x556a1f47e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03d5648640 1 -- 172.21.15.172:0/842056030 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a1f49dfd0 con 0x556a1f4a6d50 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03d664a640 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4c7c10 0x556a1f47e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03d5648640 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6d50 0x556a1f4e48b0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f03c00048f0 tx=0x7f03c0007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14de69c7d897163 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03b0ff9640 1 -- 172.21.15.172:0/842056030 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0023060 con 0x556a1f4a6d50 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03b0ff9640 1 -- 172.21.15.172:0/842056030 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7f03c00188f0 con 0x556a1f4a6d50 2024-04-14T08:42:24.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.676+0000 7f03daf6f740 1 -- 172.21.15.172:0/842056030 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03cc056070 con 0x556a1f4a6d50 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03b0ff9640 1 -- 172.21.15.172:0/842056030 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f03c0029050 con 0x556a1f4a6d50 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03daf6f740 10 client.8868 fetch_fsmap learned FSMap version 80 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03daf6f740 1 -- 172.21.15.172:0/842056030 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) -- 0x556a1f542060 con 0x556a1f4a6d50 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03b0ff9640 1 -- 172.21.15.172:0/842056030 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03cc056070 con 0x556a1f4a6d50 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03b0ff9640 1 -- 172.21.15.172:0/842056030 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 80) ==== 3494+0+0 (secure 0 0 0) 0x7f03c002c020 con 0x556a1f4a6d50 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03daf6f740 10 client.8868 fetch_fsmap finished waiting for FSMap version 80 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03daf6f740 10 client.8868 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03daf6f740 20 client.8868 populate_metadata read hostname 'smithi172' 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03daf6f740 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x556a1f4a7840 0x556a1f47eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03daf6f740 4 client.8868 mds_command: new command op to 8382 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03daf6f740 1 -- 172.21.15.172:0/842056030 --> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a1f49f5b0 con 0x556a1f4a7840 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03d5e49640 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x556a1f4a7840 0x556a1f47eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03b0ff9640 10 client.8868 ms_handle_connect on v2:172.21.15.172:6835/4028911489 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03d5e49640 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x556a1f4a7840 0x556a1f47eb50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=429d3b72b6e01186 server_cookie=2aa679bf846cdee7 in_seq=0 out_seq=0 2024-04-14T08:42:24.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03b0ff9640 1 -- 172.21.15.172:0/842056030 <== mds.0 v2:172.21.15.172:6835/4028911489 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556a1f49f5b0 con 0x556a1f4a7840 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03b0ff9640 10 client.8868 handle_command_reply: tid=0 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.677+0000 7f03daf6f740 1 client.8868 shutdown 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03daf6f740 1 -- 172.21.15.172:0/842056030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6d50 msgr2=0x556a1f4e48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03daf6f740 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6d50 0x556a1f4e48b0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f03c00048f0 tx=0x7f03c0007e40 comp rx=0 tx=0).stop 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03daf6f740 1 -- 172.21.15.172:0/842056030 shutdown_connections 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03daf6f740 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.172:6835/4028911489,v1:172.21.15.172:6837/4028911489] conn(0x556a1f4a7840 0x556a1f47eb50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03daf6f740 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556a1f4c7c10 0x556a1f47e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03daf6f740 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a1f4a6d50 0x556a1f4e48b0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03daf6f740 1 --2- 172.21.15.172:0/842056030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556a1f4a6970 0x556a1f53e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03daf6f740 1 -- 172.21.15.172:0/842056030 >> 172.21.15.172:0/842056030 conn(0x556a1f479e50 msgr2=0x556a1f4a7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03b0ff9640 0 client.8868 ms_handle_reset on v2:172.21.15.172:6835/4028911489 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03daf6f740 1 -- 172.21.15.172:0/842056030 shutdown_connections 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03daf6f740 1 -- 172.21.15.172:0/842056030 wait complete. 2024-04-14T08:42:24.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03daf6f740 20 client.8868 trim_cache size 0 max 16384 2024-04-14T08:42:24.685 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:42:24.685 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-04-14T08:42:24.685 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-04-14T08:42:24.685 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:42:24.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03d80ad640 1 -- 172.21.15.172:0/93293560 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f03b406c8f0 msgr2=0x7f03b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03d80ad640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f03b406c8f0 0x7f03b406edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f03cc036000 tx=0x7f03cc03b040 comp rx=0 tx=0).stop 2024-04-14T08:42:24.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03d80ad640 1 -- 172.21.15.172:0/93293560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d0162540 msgr2=0x7f03d00b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03d80ad640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d0162540 0x7f03d00b1ea0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f03c800c7a0 tx=0x7f03c800cc60 comp rx=0 tx=0).stop 2024-04-14T08:42:24.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03d80ad640 1 -- 172.21.15.172:0/93293560 shutdown_connections 2024-04-14T08:42:24.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03d80ad640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f03b406c8f0 0x7f03b406edb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03d80ad640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03d0162540 0x7f03d00b1ea0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03d80ad640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03d015e430 0x7f03d0152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.678+0000 7f03d80ad640 1 --2- 172.21.15.172:0/93293560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03d00d2e00 0x7f03d01f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.679+0000 7f03d80ad640 1 -- 172.21.15.172:0/93293560 >> 172.21.15.172:0/93293560 conn(0x7f03d009d560 msgr2=0x7f03d00b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:24.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.679+0000 7f03d80ad640 1 -- 172.21.15.172:0/93293560 shutdown_connections 2024-04-14T08:42:24.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.679+0000 7f03d80ad640 1 -- 172.21.15.172:0/93293560 wait complete. 2024-04-14T08:42:24.691 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-14T08:42:24.792 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:42:24.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.895+0000 7f2358ad1640 1 Processor -- start 2024-04-14T08:42:24.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.895+0000 7f2358ad1640 1 -- start start 2024-04-14T08:42:24.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2358ad1640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2354162710 0x7f23540d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2358ad1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2354162ba0 0x7f23540d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2358ad1640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23540cefb0 con 0x7f235415e430 2024-04-14T08:42:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2358ad1640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f23540cf290 con 0x7f2354162710 2024-04-14T08:42:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2358ad1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f23540cf570 con 0x7f2354162ba0 2024-04-14T08:42:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2351d9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2354162710 0x7f23540d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2351d9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2354162710 0x7f23540d1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35146/0 (socket says 172.21.15.172:35146) 2024-04-14T08:42:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2351d9b640 1 -- 172.21.15.172:0/4105963716 learned_addr learned my addr 172.21.15.172:0/4105963716 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2352d9d640 1 --2- 172.21.15.172:0/4105963716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2354162ba0 0x7f23540d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2351d9b640 1 -- 172.21.15.172:0/4105963716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2354162ba0 msgr2=0x7f23540d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2351d9b640 1 --2- 172.21.15.172:0/4105963716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2354162ba0 0x7f23540d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2351d9b640 1 -- 172.21.15.172:0/4105963716 >> v1:172.21.15.177:6789/0 conn(0x7f235415e430 legacy=0x7f2354162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.897+0000 7f2351d9b640 1 -- 172.21.15.172:0/4105963716 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23540cf850 con 0x7f2354162710 2024-04-14T08:42:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.898+0000 7f2351d9b640 1 --2- 172.21.15.172:0/4105963716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162710 0x7f23540d1df0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f2344009870 tx=0x7f234402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0d62fc81878784 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.898+0000 7f235159a640 1 -- 172.21.15.172:0/4105963716 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344034070 con 0x7f2354162710 2024-04-14T08:42:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.898+0000 7f235159a640 1 -- 172.21.15.172:0/4105963716 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2344030840 con 0x7f2354162710 2024-04-14T08:42:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.898+0000 7f235159a640 1 -- 172.21.15.172:0/4105963716 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344030c10 con 0x7f2354162710 2024-04-14T08:42:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.899+0000 7f2358ad1640 1 -- 172.21.15.172:0/4105963716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162710 msgr2=0x7f23540d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.899+0000 7f2358ad1640 1 --2- 172.21.15.172:0/4105963716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162710 0x7f23540d1df0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f2344009870 tx=0x7f234402b980 comp rx=0 tx=0).stop 2024-04-14T08:42:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.899+0000 7f2358ad1640 1 -- 172.21.15.172:0/4105963716 shutdown_connections 2024-04-14T08:42:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.899+0000 7f2358ad1640 1 --2- 172.21.15.172:0/4105963716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2354162ba0 0x7f23540d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.899+0000 7f2358ad1640 1 --2- 172.21.15.172:0/4105963716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162710 0x7f23540d1df0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.899+0000 7f2358ad1640 1 -- 172.21.15.172:0/4105963716 >> 172.21.15.172:0/4105963716 conn(0x7f235409d560 msgr2=0x7f23540d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.899+0000 7f2358ad1640 1 -- 172.21.15.172:0/4105963716 shutdown_connections 2024-04-14T08:42:24.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.899+0000 7f2358ad1640 1 -- 172.21.15.172:0/4105963716 wait complete. 2024-04-14T08:42:24.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.900+0000 7f2358ad1640 1 Processor -- start 2024-04-14T08:42:24.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.901+0000 7f2358ad1640 1 -- start start 2024-04-14T08:42:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.901+0000 7f2358ad1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f235415e430 0x7f23540ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.901+0000 7f2358ad1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2354162710 0x7f23540b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.901+0000 7f2358ad1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162ba0 0x7f23540b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.901+0000 7f2358ad1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23540d4e20 con 0x7f2354162ba0 2024-04-14T08:42:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.901+0000 7f2358ad1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f235415e810 con 0x7f235415e430 2024-04-14T08:42:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.901+0000 7f2358ad1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f23540d0050 con 0x7f2354162710 2024-04-14T08:42:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.901+0000 7f2352d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162ba0 0x7f23540b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.901+0000 7f2352d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162ba0 0x7f23540b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35152/0 (socket says 172.21.15.172:35152) 2024-04-14T08:42:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.901+0000 7f2352d9d640 1 -- 172.21.15.172:0/606621057 learned_addr learned my addr 172.21.15.172:0/606621057 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:24.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.901+0000 7f2351d9b640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2354162710 0x7f23540b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.901+0000 7f235259c640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f235415e430 0x7f23540ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.902+0000 7f2352d9d640 1 -- 172.21.15.172:0/606621057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2354162710 msgr2=0x7f23540b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.902+0000 7f2352d9d640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2354162710 0x7f23540b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.902+0000 7f2352d9d640 1 -- 172.21.15.172:0/606621057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f235415e430 msgr2=0x7f23540ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.902+0000 7f2352d9d640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f235415e430 0x7f23540ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.902+0000 7f2352d9d640 1 -- 172.21.15.172:0/606621057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23540b8680 con 0x7f2354162ba0 2024-04-14T08:42:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.902+0000 7f2352d9d640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162ba0 0x7f23540b7e30 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f234c007e60 tx=0x7f234c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.902+0000 7f2342ffd640 1 -- 172.21.15.172:0/606621057 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c014420 con 0x7f2354162ba0 2024-04-14T08:42:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.902+0000 7f2342ffd640 1 -- 172.21.15.172:0/606621057 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234c019070 con 0x7f2354162ba0 2024-04-14T08:42:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.902+0000 7f2358ad1640 1 -- 172.21.15.172:0/606621057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23540b8960 con 0x7f2354162ba0 2024-04-14T08:42:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.902+0000 7f2342ffd640 1 -- 172.21.15.172:0/606621057 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c014820 con 0x7f2354162ba0 2024-04-14T08:42:24.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.902+0000 7f2358ad1640 1 -- 172.21.15.172:0/606621057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23540b8f20 con 0x7f2354162ba0 2024-04-14T08:42:24.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.903+0000 7f2342ffd640 1 -- 172.21.15.172:0/606621057 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f234c0149c0 con 0x7f2354162ba0 2024-04-14T08:42:24.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.903+0000 7f2358ad1640 1 -- 172.21.15.172:0/606621057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2354068590 con 0x7f2354162ba0 2024-04-14T08:42:24.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.904+0000 7f2342ffd640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f233406c8f0 0x7f233406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:24.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.904+0000 7f2342ffd640 1 -- 172.21.15.172:0/606621057 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6216+0+0 (secure 0 0 0) 0x7f234c08c380 con 0x7f2354162ba0 2024-04-14T08:42:24.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.904+0000 7f235259c640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f233406c8f0 0x7f233406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:24.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.904+0000 7f235259c640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f233406c8f0 0x7f233406edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2348000f60 tx=0x7f2348009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:24.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:24.909+0000 7f2342ffd640 1 -- 172.21.15.172:0/606621057 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f234c059ff0 con 0x7f2354162ba0 2024-04-14T08:42:25.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.144+0000 7f2358ad1640 1 -- 172.21.15.172:0/606621057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f23540b2000 con 0x7f2354162ba0 2024-04-14T08:42:25.145 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:42:25.145+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:42:25.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.146+0000 7f2342ffd640 1 -- 172.21.15.172:0/606621057 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) ==== 161+0+0 (secure 0 0 0) 0x7f234c05e2f0 con 0x7f2354162ba0 2024-04-14T08:42:25.147 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:42:25.149 INFO:tasks.ceph.mds.a.smithi172.stderr: -3018> 2024-04-14T08:41:00.093+0000 7fa0713fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-14T08:42:25.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.149+0000 7f2358ad1640 1 -- 172.21.15.172:0/606621057 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f233406c8f0 msgr2=0x7f233406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.149+0000 7f2358ad1640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f233406c8f0 0x7f233406edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2348000f60 tx=0x7f2348009040 comp rx=0 tx=0).stop 2024-04-14T08:42:25.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.149+0000 7f2358ad1640 1 -- 172.21.15.172:0/606621057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162ba0 msgr2=0x7f23540b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.149+0000 7f2358ad1640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162ba0 0x7f23540b7e30 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f234c007e60 tx=0x7f234c00e4b0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.150+0000 7f2358ad1640 1 -- 172.21.15.172:0/606621057 shutdown_connections 2024-04-14T08:42:25.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.150+0000 7f2358ad1640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f233406c8f0 0x7f233406edb0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.150+0000 7f2358ad1640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2354162ba0 0x7f23540b7e30 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.150+0000 7f2358ad1640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2354162710 0x7f23540b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.150+0000 7f2358ad1640 1 --2- 172.21.15.172:0/606621057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f235415e430 0x7f23540ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.150+0000 7f2358ad1640 1 -- 172.21.15.172:0/606621057 >> 172.21.15.172:0/606621057 conn(0x7f235409d560 msgr2=0x7f23540d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:25.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.150+0000 7f2358ad1640 1 -- 172.21.15.172:0/606621057 shutdown_connections 2024-04-14T08:42:25.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.150+0000 7f2358ad1640 1 -- 172.21.15.172:0/606621057 wait complete. 2024-04-14T08:42:25.153 INFO:tasks.ceph.mds.a.smithi172.stderr: -1152> 2024-04-14T08:41:54.358+0000 7fa0673e9640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-14T08:42:25.153 INFO:tasks.ceph.mds.a.smithi172.stderr: -1149> 2024-04-14T08:41:54.358+0000 7fa0673e9640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-14T08:42:25.153 INFO:tasks.ceph.mds.a.smithi172.stderr: -1132> 2024-04-14T08:41:54.359+0000 7fa0673e9640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-14T08:42:25.153 INFO:tasks.ceph.mds.a.smithi172.stderr: -1129> 2024-04-14T08:41:54.359+0000 7fa0673e9640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-14T08:42:25.153 INFO:tasks.ceph.mds.a.smithi172.stderr: -950> 2024-04-14T08:41:55.415+0000 7fa06b3f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:42:25.163 DEBUG:teuthology.orchestra.run.smithi172:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-14T08:42:25.179 INFO:tasks.ceph.mds.a.smithi172.stdout:starting mds.a at 2024-04-14T08:42:25.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.203+0000 7f03a3a30280 1 -- start start 2024-04-14T08:42:25.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.203+0000 7f03a3a30280 1 Processor -- start 2024-04-14T08:42:25.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a3a30280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5607028b2c00 0x560702a24000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a3a30280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607028b3800 0x560702a24580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a3a30280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5607028b3c00 0x560702a24b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a3a30280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560702a28600 con 0x5607028b3800 2024-04-14T08:42:25.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a3a30280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560702a28480 con 0x5607028b2c00 2024-04-14T08:42:25.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a3a30280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x560702a28300 con 0x5607028b3c00 2024-04-14T08:42:25.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a05fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607028b3800 0x560702a24580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a05fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607028b3800 0x560702a24580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35188/0 (socket says 172.21.15.172:35188) 2024-04-14T08:42:25.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a05fb640 1 -- 172.21.15.172:0/3371960488 learned_addr learned my addr 172.21.15.172:0/3371960488 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:25.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a0dfc640 1 --2- 172.21.15.172:0/3371960488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5607028b2c00 0x560702a24000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a15fd640 1 --2- 172.21.15.172:0/3371960488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5607028b3c00 0x560702a24b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a05fb640 1 -- 172.21.15.172:0/3371960488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5607028b3c00 msgr2=0x560702a24b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:25.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a05fb640 1 --2- 172.21.15.172:0/3371960488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5607028b3c00 0x560702a24b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a05fb640 1 -- 172.21.15.172:0/3371960488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5607028b2c00 msgr2=0x560702a24000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:25.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a05fb640 1 --2- 172.21.15.172:0/3371960488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5607028b2c00 0x560702a24000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a05fb640 1 -- 172.21.15.172:0/3371960488 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607028ea780 con 0x5607028b3800 2024-04-14T08:42:25.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a15fd640 1 --2- 172.21.15.172:0/3371960488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5607028b3c00 0x560702a24b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:25.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a0dfc640 1 --2- 172.21.15.172:0/3371960488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5607028b2c00 0x560702a24000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:25.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f03a05fb640 1 --2- 172.21.15.172:0/3371960488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607028b3800 0x560702a24580 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x5607036e0d80 tx=0x5607036482a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd191f95c494c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:25.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.204+0000 7f039e5f7640 1 -- 172.21.15.172:0/3371960488 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5607028e84e0 con 0x5607028b3800 2024-04-14T08:42:25.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.205+0000 7f039e5f7640 1 -- 172.21.15.172:0/3371960488 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5607028dee00 con 0x5607028b3800 2024-04-14T08:42:25.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.205+0000 7f039e5f7640 1 -- 172.21.15.172:0/3371960488 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5607028e8b60 con 0x5607028b3800 2024-04-14T08:42:25.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.205+0000 7f03a3a30280 1 -- 172.21.15.172:0/3371960488 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607028ea3c0 con 0x5607028b3800 2024-04-14T08:42:25.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.205+0000 7f03a3a30280 1 -- 172.21.15.172:0/3371960488 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5607028eb680 con 0x5607028b3800 2024-04-14T08:42:25.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.205+0000 7f03a3a30280 4 waiting for MDS map... 2024-04-14T08:42:25.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.205+0000 7f039e5f7640 1 -- 172.21.15.172:0/3371960488 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x560703784600 con 0x5607028b3800 2024-04-14T08:42:25.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.205+0000 7f039e5f7640 1 -- 172.21.15.172:0/3371960488 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 81) ==== 2885+0+0 (secure 0 0 0) 0x56070290f180 con 0x5607028b3800 2024-04-14T08:42:25.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.205+0000 7f03a3a30280 4 Got MDS map 81 2024-04-14T08:42:25.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.205+0000 7f03a3a30280 10 main: main 2024-04-14T08:42:25.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.205+0000 7f03a3a30280 4 main: connecting to RADOS... 2024-04-14T08:42:25.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.205+0000 7f03a3a30280 1 Processor -- start 2024-04-14T08:42:25.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.205+0000 7f03a3a30280 1 -- start start 2024-04-14T08:42:25.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a3a30280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560703787400 0x560702a25080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a3a30280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787800 0x560702a25600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a3a30280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560703787c00 0x560702a25b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a3a30280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560702a28180 con 0x560703787800 2024-04-14T08:42:25.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a3a30280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5607028d9800 con 0x560703787c00 2024-04-14T08:42:25.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a3a30280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5607028d8a80 con 0x560703787400 2024-04-14T08:42:25.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a0dfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787800 0x560702a25600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a0dfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787800 0x560702a25600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35190/0 (socket says 172.21.15.172:35190) 2024-04-14T08:42:25.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a0dfc640 1 -- 172.21.15.172:0/972704504 learned_addr learned my addr 172.21.15.172:0/972704504 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:25.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a15fd640 1 --2- 172.21.15.172:0/972704504 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560703787400 0x560702a25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a05fb640 1 --2- 172.21.15.172:0/972704504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560703787c00 0x560702a25b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a0dfc640 1 -- 172.21.15.172:0/972704504 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560703787400 msgr2=0x560702a25080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:25.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a0dfc640 1 --2- 172.21.15.172:0/972704504 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560703787400 0x560702a25080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a0dfc640 1 -- 172.21.15.172:0/972704504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560703787c00 msgr2=0x560702a25b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:25.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a0dfc640 1 --2- 172.21.15.172:0/972704504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560703787c00 0x560702a25b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a0dfc640 1 -- 172.21.15.172:0/972704504 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607028eb4a0 con 0x560703787800 2024-04-14T08:42:25.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.206+0000 7f03a0dfc640 1 --2- 172.21.15.172:0/972704504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787800 0x560702a25600 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x5607037f40f0 tx=0x5607037a1800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c25e4eae74c467 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:25.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.207+0000 7f039c5f3640 1 -- 172.21.15.172:0/972704504 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5607028e84e0 con 0x560703787800 2024-04-14T08:42:25.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.207+0000 7f039c5f3640 1 -- 172.21.15.172:0/972704504 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5607028dea80 con 0x560703787800 2024-04-14T08:42:25.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.207+0000 7f039c5f3640 1 -- 172.21.15.172:0/972704504 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5607028e8b60 con 0x560703787800 2024-04-14T08:42:25.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.207+0000 7f03a3a30280 1 -- 172.21.15.172:0/972704504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787800 msgr2=0x560702a25600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.207+0000 7f03a3a30280 1 --2- 172.21.15.172:0/972704504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787800 0x560702a25600 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x5607037f40f0 tx=0x5607037a1800 comp rx=0 tx=0).stop 2024-04-14T08:42:25.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.207+0000 7f03a3a30280 1 -- 172.21.15.172:0/972704504 shutdown_connections 2024-04-14T08:42:25.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.207+0000 7f03a3a30280 1 --2- 172.21.15.172:0/972704504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560703787c00 0x560702a25b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.207+0000 7f03a3a30280 1 --2- 172.21.15.172:0/972704504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787800 0x560702a25600 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.207+0000 7f03a3a30280 1 --2- 172.21.15.172:0/972704504 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560703787400 0x560702a25080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.207+0000 7f03a3a30280 1 -- 172.21.15.172:0/972704504 >> 172.21.15.172:0/972704504 conn(0x560703787000 msgr2=0x56070299c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:25.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.207+0000 7f03a3a30280 1 -- 172.21.15.172:0/972704504 shutdown_connections 2024-04-14T08:42:25.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.207+0000 7f03a3a30280 1 -- 172.21.15.172:0/972704504 wait complete. 2024-04-14T08:42:25.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a3a30280 1 Processor -- start 2024-04-14T08:42:25.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a3a30280 1 -- start start 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a3a30280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787400 0x560702a25080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a3a30280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560703787800 0x560702a25600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a3a30280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560703787c00 0x560702a25b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a3a30280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560702a29380 con 0x560703787400 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a3a30280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560702a28180 con 0x560703787c00 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a3a30280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x560702a29500 con 0x560703787800 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a15fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787400 0x560702a25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a15fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787400 0x560702a25080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35192/0 (socket says 172.21.15.172:35192) 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a15fd640 1 -- 172.21.15.172:0/2888434418 learned_addr learned my addr 172.21.15.172:0/2888434418 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a05fb640 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560703787c00 0x560702a25b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.208+0000 7f03a0dfc640 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560703787800 0x560702a25600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.209+0000 7f03a15fd640 1 -- 172.21.15.172:0/2888434418 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560703787800 msgr2=0x560702a25600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.209+0000 7f03a15fd640 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560703787800 0x560702a25600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.209+0000 7f03a15fd640 1 -- 172.21.15.172:0/2888434418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560703787c00 msgr2=0x560702a25b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.209+0000 7f03a15fd640 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560703787c00 0x560702a25b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.209+0000 7f03a15fd640 1 -- 172.21.15.172:0/2888434418 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607028ea780 con 0x560703787400 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.209+0000 7f03a15fd640 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787400 0x560702a25080 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x5607037f5350 tx=0x560703646b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.209+0000 7f039adf0640 1 -- 172.21.15.172:0/2888434418 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5607028e89c0 con 0x560703787400 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.209+0000 7f039adf0640 1 -- 172.21.15.172:0/2888434418 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5607028dea80 con 0x560703787400 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.209+0000 7f03a3a30280 1 -- 172.21.15.172:0/2888434418 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5607028ea3c0 con 0x560703787400 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.209+0000 7f039adf0640 1 -- 172.21.15.172:0/2888434418 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5607028e84e0 con 0x560703787400 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.209+0000 7f03a3a30280 1 -- 172.21.15.172:0/2888434418 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607028ea000 con 0x560703787400 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.209+0000 7f03a3a30280 4 main: resolving pool 15 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.210+0000 7f039adf0640 1 -- 172.21.15.172:0/2888434418 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5607036aac00 con 0x560703787400 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.210+0000 7f039adf0640 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x560703780c00 0x560702a26100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.210+0000 7f039adf0640 1 -- 172.21.15.172:0/2888434418 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x560703782c00 con 0x560703787400 2024-04-14T08:42:25.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.211+0000 7f03a3a30280 4 main: creating IoCtx.. 2024-04-14T08:42:25.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.211+0000 7f03a0dfc640 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x560703780c00 0x560702a26100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.211+0000 7f03a3a30280 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x560703781000 0x560702a26680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.211+0000 7f03a3a30280 1 -- 172.21.15.172:0/2888434418 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) -- 0x560703781400 con 0x560703781000 2024-04-14T08:42:25.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.211+0000 7f03a05fb640 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x560703781000 0x560702a26680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.211+0000 7f03a0dfc640 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x560703780c00 0x560702a26100 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x5607037f40f0 tx=0x560703648240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:25.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.211+0000 7f03a05fb640 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x560703781000 0x560702a26680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:25.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.213+0000 7f03a05fb640 1 -- 172.21.15.172:0/2888434418 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x560703636000 con 0x560703781000 2024-04-14T08:42:25.222 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:42:25.222 INFO:teuthology.orchestra.run.smithi172.stdout: "0": { 2024-04-14T08:42:25.222 INFO:teuthology.orchestra.run.smithi172.stdout: "data": { 2024-04-14T08:42:25.222 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 5, 2024-04-14T08:42:25.222 INFO:teuthology.orchestra.run.smithi172.stdout: "inotable": { 2024-04-14T08:42:25.222 INFO:teuthology.orchestra.run.smithi172.stdout: "projected_free": [ 2024-04-14T08:42:25.222 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:25.222 INFO:teuthology.orchestra.run.smithi172.stdout: "start": 1099511627779, 2024-04-14T08:42:25.222 INFO:teuthology.orchestra.run.smithi172.stdout: "len": 1099511627773 2024-04-14T08:42:25.222 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout: "free": [ 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout: "start": 1099511627779, 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout: "len": 1099511627773 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout: "result": 0 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:42:25.223 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:25.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.213+0000 7f03a3a30280 1 -- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x560703781000 msgr2=0x560702a26680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.213+0000 7f03a3a30280 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x560703781000 0x560702a26680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.213+0000 7f03a3a30280 1 -- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x560703780c00 msgr2=0x560702a26100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.213+0000 7f03a3a30280 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x560703780c00 0x560702a26100 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x5607037f40f0 tx=0x560703648240 comp rx=0 tx=0).stop 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.213+0000 7f03a3a30280 1 -- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787400 msgr2=0x560702a25080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.213+0000 7f03a3a30280 1 --2- 172.21.15.172:0/2888434418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560703787400 0x560702a25080 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x5607037f5350 tx=0x560703646b40 comp rx=0 tx=0).stop 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.213+0000 7f03a0dfc640 1 -- 172.21.15.172:0/2888434418 reap_dead start 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.213+0000 7f03a3a30280 1 -- 172.21.15.172:0/2888434418 shutdown_connections 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.213+0000 7f03a3a30280 1 -- 172.21.15.172:0/2888434418 >> 172.21.15.172:0/2888434418 conn(0x560703787000 msgr2=0x56070299c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.214+0000 7f03a3a30280 1 -- 172.21.15.172:0/2888434418 shutdown_connections 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.214+0000 7f03a3a30280 1 -- 172.21.15.172:0/2888434418 wait complete. 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.214+0000 7f03a3a30280 1 -- 172.21.15.172:0/3371960488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607028b3800 msgr2=0x560702a24580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.214+0000 7f03a3a30280 1 --2- 172.21.15.172:0/3371960488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607028b3800 0x560702a24580 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x5607036e0d80 tx=0x5607036482a0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.214+0000 7f03a3a30280 1 -- 172.21.15.172:0/3371960488 shutdown_connections 2024-04-14T08:42:25.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.214+0000 7f03a3a30280 1 --2- 172.21.15.172:0/3371960488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5607028b3c00 0x560702a24b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.214+0000 7f03a3a30280 1 --2- 172.21.15.172:0/3371960488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607028b3800 0x560702a24580 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.214+0000 7f03a3a30280 1 --2- 172.21.15.172:0/3371960488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5607028b2c00 0x560702a24000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.214+0000 7f03a3a30280 1 -- 172.21.15.172:0/3371960488 >> 172.21.15.172:0/3371960488 conn(0x5607028b3000 msgr2=0x56070299d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:25.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.214+0000 7f03a3a30280 1 -- 172.21.15.172:0/3371960488 shutdown_connections 2024-04-14T08:42:25.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.214+0000 7f03a3a30280 1 -- 172.21.15.172:0/3371960488 wait complete. 2024-04-14T08:42:25.227 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.063901, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-14T08:42:25.228 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:42:25.228 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:42:25.313 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:42:25.323 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:42:25.325 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:42:25.366 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:42:25.366 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:42:25.366 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:42:25.366 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:25.382 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:42:25.382 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:42:25.382 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:42:25.382 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:25.423 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:42:25.423 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:42:25.423 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:42:25.423 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:25.459 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:42:25.459 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:42:25.459 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:42:25.459 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:25.479 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:42:25.480 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:42:25.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.599+0000 7f197f2ad640 1 Processor -- start 2024-04-14T08:42:25.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.600+0000 7f197f2ad640 1 -- start start 2024-04-14T08:42:25.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.600+0000 7f197f2ad640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f19780d2830 0x7f19780d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.600+0000 7f197f2ad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f197815e890 0x7f19780d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.600+0000 7f197f2ad640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19780d3150 con 0x7f19780cedd0 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.600+0000 7f197f2ad640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f19780cff30 con 0x7f19780d2830 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.600+0000 7f197f2ad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f19780d2350 con 0x7f197815e890 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f197d84a640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f19780cedd0 0x7f19781621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:37488/0 (socket says 172.21.15.172:37488) 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f197d84a640 1 -- 172.21.15.172:0/1335977070 learned_addr learned my addr 172.21.15.172:0/1335977070 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f197d049640 1 --2- 172.21.15.172:0/1335977070 >> v2:172.21.15.177:3300/0 conn(0x7f19780d2830 0x7f19780d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f197c848640 1 --2- 172.21.15.172:0/1335977070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f197815e890 0x7f19780d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2476915799 0 0) 0x7f19780d3150 con 0x7f19780cedd0 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1964003520 con 0x7f19780cedd0 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2169596410 0 0) 0x7f1964003520 con 0x7f19780cedd0 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f197815e890 msgr2=0x7f19780d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 --2- 172.21.15.172:0/1335977070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f197815e890 0x7f19780d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 >> v2:172.21.15.177:3300/0 conn(0x7f19780d2830 msgr2=0x7f19780d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:25.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 --2- 172.21.15.172:0/1335977070 >> v2:172.21.15.177:3300/0 conn(0x7f19780d2830 0x7f19780d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19780cf320 con 0x7f19780cedd0 2024-04-14T08:42:25.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f19700019f0 con 0x7f19780cedd0 2024-04-14T08:42:25.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 >> v1:172.21.15.172:6789/0 conn(0x7f19780cedd0 legacy=0x7f19781621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 --2- 172.21.15.172:0/1335977070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19640039b0 0x7f1964003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 --2- 172.21.15.172:0/1335977070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f197815e890 0x7f1964004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 --2- 172.21.15.172:0/1335977070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1964004d40 0x7f19640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f19780cff30 con 0x7f1964004d40 2024-04-14T08:42:25.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f19780d2350 con 0x7f19640039b0 2024-04-14T08:42:25.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f196400b9c0 con 0x7f197815e890 2024-04-14T08:42:25.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f197d84a640 1 --2- 172.21.15.172:0/1335977070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1964004d40 0x7f19640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f197c848640 1 --2- 172.21.15.172:0/1335977070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f197815e890 0x7f1964004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.601+0000 7f197d049640 1 --2- 172.21.15.172:0/1335977070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19640039b0 0x7f1964003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.602+0000 7f197d84a640 1 -- 172.21.15.172:0/1335977070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f197815e890 msgr2=0x7f1964004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:25.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.602+0000 7f197d84a640 1 --2- 172.21.15.172:0/1335977070 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f197815e890 0x7f1964004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.602+0000 7f197d84a640 1 -- 172.21.15.172:0/1335977070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19640039b0 msgr2=0x7f1964003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:25.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.602+0000 7f197d84a640 1 --2- 172.21.15.172:0/1335977070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19640039b0 0x7f1964003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.602+0000 7f197d84a640 1 -- 172.21.15.172:0/1335977070 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1964003520 con 0x7f1964004d40 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.602+0000 7f197d84a640 1 --2- 172.21.15.172:0/1335977070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1964004d40 0x7f19640051c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f19700013d0 tx=0x7f1964005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7e2e616e38cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.602+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197000a4e0 con 0x7f1964004d40 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.602+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1970034650 con 0x7f1964004d40 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.602+0000 7f196ffff640 1 -- 172.21.15.172:0/1335977070 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970034a40 con 0x7f1964004d40 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.603+0000 7f197f2ad640 1 -- 172.21.15.172:0/1335977070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1964004d40 msgr2=0x7f19640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.603+0000 7f197f2ad640 1 --2- 172.21.15.172:0/1335977070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1964004d40 0x7f19640051c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f19700013d0 tx=0x7f1964005830 comp rx=0 tx=0).stop 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.603+0000 7f197d84a640 1 -- 172.21.15.172:0/1335977070 reap_dead start 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.603+0000 7f197f2ad640 1 -- 172.21.15.172:0/1335977070 shutdown_connections 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.603+0000 7f197f2ad640 1 -- 172.21.15.172:0/1335977070 >> 172.21.15.172:0/1335977070 conn(0x7f197809d590 msgr2=0x7f19780d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.603+0000 7f197f2ad640 1 -- 172.21.15.172:0/1335977070 shutdown_connections 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.603+0000 7f197f2ad640 1 -- 172.21.15.172:0/1335977070 wait complete. 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.604+0000 7f197f2ad640 1 Processor -- start 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.604+0000 7f197f2ad640 1 -- start start 2024-04-14T08:42:25.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197f2ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1978151b80 0x7f19780cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197f2ad640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19780cb320 0x7f19780c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197f2ad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19780c95a0 0x7f197815fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197f2ad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f19780cfbc0 con 0x7f1978151b80 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197f2ad640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f19780cf8e0 con 0x7f19780cb320 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197f2ad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f19780b4720 con 0x7f19780c95a0 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197d049640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1978151b80 0x7f19780cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197d049640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1978151b80 0x7f19780cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35220/0 (socket says 172.21.15.172:35220) 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197d049640 1 -- 172.21.15.172:0/1957035825 learned_addr learned my addr 172.21.15.172:0/1957035825 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197c848640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19780cb320 0x7f19780c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197d84a640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19780c95a0 0x7f197815fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197d049640 1 -- 172.21.15.172:0/1957035825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19780c95a0 msgr2=0x7f197815fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197d049640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19780c95a0 0x7f197815fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197d049640 1 -- 172.21.15.172:0/1957035825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19780cb320 msgr2=0x7f19780c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197d049640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19780cb320 0x7f19780c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197d049640 1 -- 172.21.15.172:0/1957035825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19780cc740 con 0x7f1978151b80 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197c848640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19780cb320 0x7f19780c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.605+0000 7f197d049640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1978151b80 0x7f19780cc200 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f1968011a70 tx=0x7f1968011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.606+0000 7f196dffb640 1 -- 172.21.15.172:0/1957035825 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196800fb50 con 0x7f1978151b80 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.606+0000 7f196dffb640 1 -- 172.21.15.172:0/1957035825 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196800fcf0 con 0x7f1978151b80 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.606+0000 7f197f2ad640 1 -- 172.21.15.172:0/1957035825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197815eef0 con 0x7f1978151b80 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.606+0000 7f196dffb640 1 -- 172.21.15.172:0/1957035825 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196800fea0 con 0x7f1978151b80 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.606+0000 7f197f2ad640 1 -- 172.21.15.172:0/1957035825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197815f4b0 con 0x7f1978151b80 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.607+0000 7f197f2ad640 1 -- 172.21.15.172:0/1957035825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1978068520 con 0x7f1978151b80 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.607+0000 7f196dffb640 1 -- 172.21.15.172:0/1957035825 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1968012040 con 0x7f1978151b80 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.608+0000 7f196dffb640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f195c06c920 0x7f195c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.608+0000 7f196dffb640 1 -- 172.21.15.172:0/1957035825 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7f196808c370 con 0x7f1978151b80 2024-04-14T08:42:25.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.608+0000 7f197c848640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f195c06c920 0x7f195c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.608+0000 7f197c848640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f195c06c920 0x7f195c06ede0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1978161570 tx=0x7f1974008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:25.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.612+0000 7f196dffb640 1 -- 172.21.15.172:0/1957035825 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1968019030 con 0x7f1978151b80 2024-04-14T08:42:25.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.823+0000 7f197f2ad640 1 -- 172.21.15.172:0/1957035825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19780d2610 con 0x7f1978151b80 2024-04-14T08:42:25.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.823+0000 7f196dffb640 1 -- 172.21.15.172:0/1957035825 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+4354 (secure 0 0 0) 0x7f196805a5e0 con 0x7f1978151b80 2024-04-14T08:42:25.824 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:25.825 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8005,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3289017090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3289017090},{"type":"v1","addr":"172.21.15.177:6835","nonce":3289017090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":81,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:41:04.492773+0000","modified":"2024-04-14T08:42:25.146002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-14T08:42:25.826 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 81 2024-04-14T08:42:25.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.825+0000 7f197f2ad640 1 -- 172.21.15.172:0/1957035825 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f195c06c920 msgr2=0x7f195c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.825+0000 7f197f2ad640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f195c06c920 0x7f195c06ede0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1978161570 tx=0x7f1974008040 comp rx=0 tx=0).stop 2024-04-14T08:42:25.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.825+0000 7f197f2ad640 1 -- 172.21.15.172:0/1957035825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1978151b80 msgr2=0x7f19780cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.825+0000 7f197f2ad640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1978151b80 0x7f19780cc200 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f1968011a70 tx=0x7f1968011f30 comp rx=0 tx=0).stop 2024-04-14T08:42:25.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.825+0000 7f197f2ad640 1 -- 172.21.15.172:0/1957035825 shutdown_connections 2024-04-14T08:42:25.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.825+0000 7f197f2ad640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f195c06c920 0x7f195c06ede0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.825+0000 7f197f2ad640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19780c95a0 0x7f197815fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.825+0000 7f197f2ad640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19780cb320 0x7f19780c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.825+0000 7f197f2ad640 1 --2- 172.21.15.172:0/1957035825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1978151b80 0x7f19780cc200 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.825+0000 7f197f2ad640 1 -- 172.21.15.172:0/1957035825 >> 172.21.15.172:0/1957035825 conn(0x7f197809d590 msgr2=0x7f197806c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.825+0000 7f197f2ad640 1 -- 172.21.15.172:0/1957035825 shutdown_connections 2024-04-14T08:42:25.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.825+0000 7f197f2ad640 1 -- 172.21.15.172:0/1957035825 wait complete. 2024-04-14T08:42:25.837 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:42:25.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.939+0000 7f94fa711640 1 Processor -- start 2024-04-14T08:42:25.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.939+0000 7f94fa711640 1 -- start start 2024-04-14T08:42:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.939+0000 7f94fa711640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f94f40b1df0 0x7f94f40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.939+0000 7f94fa711640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f415c770 con 0x7f94f40b21d0 2024-04-14T08:42:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.940+0000 7f94fa711640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f40b3430 con 0x7f94f40b4040 2024-04-14T08:42:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.940+0000 7f94fa711640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f94f40b3700 con 0x7f94f40b1df0 2024-04-14T08:42:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.940+0000 7f94f3fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f94f40b1df0 0x7f94f40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.940+0000 7f94f37fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f94f40b21d0 0x7f94f415c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:37504/0 (socket says 172.21.15.172:37504) 2024-04-14T08:42:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.940+0000 7f94f37fe640 1 -- 172.21.15.172:0/1173611108 learned_addr learned my addr 172.21.15.172:0/1173611108 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.940+0000 7f94f3fff640 1 -- 172.21.15.172:0/1173611108 >> v1:172.21.15.177:6789/0 conn(0x7f94f40b4040 legacy=0x7f94f40b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.940+0000 7f94f3fff640 1 -- 172.21.15.172:0/1173611108 >> v1:172.21.15.172:6789/0 conn(0x7f94f40b21d0 legacy=0x7f94f415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.940+0000 7f94f3fff640 1 -- 172.21.15.172:0/1173611108 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f40b39e0 con 0x7f94f40b1df0 2024-04-14T08:42:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.940+0000 7f94f2ffd640 1 -- 172.21.15.172:0/1173611108 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 185416127 0 0) 0x7f94f415c770 con 0x7f94f40b21d0 2024-04-14T08:42:25.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.941+0000 7f94f3fff640 1 --2- 172.21.15.172:0/1173611108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94f40b1df0 0x7f94f40b2ef0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f94e40097a0 tx=0x7f94e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4377f85ce4c66c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.941+0000 7f94f2ffd640 1 -- 172.21.15.172:0/1173611108 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4035020 con 0x7f94f40b1df0 2024-04-14T08:42:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.941+0000 7f94f2ffd640 1 -- 172.21.15.172:0/1173611108 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e4031830 con 0x7f94f40b1df0 2024-04-14T08:42:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.941+0000 7f94f2ffd640 1 -- 172.21.15.172:0/1173611108 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4031c00 con 0x7f94f40b1df0 2024-04-14T08:42:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.941+0000 7f94fa711640 1 -- 172.21.15.172:0/1173611108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94f40b1df0 msgr2=0x7f94f40b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.941+0000 7f94fa711640 1 --2- 172.21.15.172:0/1173611108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94f40b1df0 0x7f94f40b2ef0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f94e40097a0 tx=0x7f94e402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.941+0000 7f94fa711640 1 -- 172.21.15.172:0/1173611108 shutdown_connections 2024-04-14T08:42:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.941+0000 7f94fa711640 1 --2- 172.21.15.172:0/1173611108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94f40b1df0 0x7f94f40b2ef0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.941+0000 7f94fa711640 1 -- 172.21.15.172:0/1173611108 >> 172.21.15.172:0/1173611108 conn(0x7f94f406ec30 msgr2=0x7f94f406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.941+0000 7f94fa711640 1 -- 172.21.15.172:0/1173611108 shutdown_connections 2024-04-14T08:42:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.941+0000 7f94fa711640 1 -- 172.21.15.172:0/1173611108 wait complete. 2024-04-14T08:42:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.942+0000 7f94fa711640 1 Processor -- start 2024-04-14T08:42:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.943+0000 7f94fa711640 1 -- start start 2024-04-14T08:42:25.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.943+0000 7f94fa711640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94f40b1df0 0x7f94f40cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.943+0000 7f94fa711640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94f40b21d0 0x7f94f41f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.943+0000 7f94fa711640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94f40b4040 0x7f94f40cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.943+0000 7f94fa711640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94f415b7f0 con 0x7f94f40b4040 2024-04-14T08:42:25.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.943+0000 7f94fa711640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f94f415bb40 con 0x7f94f40b1df0 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.943+0000 7f94fa711640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f94f41552d0 con 0x7f94f40b21d0 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.943+0000 7f94f8cae640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94f40b4040 0x7f94f40cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f8cae640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94f40b4040 0x7f94f40cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35238/0 (socket says 172.21.15.172:35238) 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f8cae640 1 -- 172.21.15.172:0/124403959 learned_addr learned my addr 172.21.15.172:0/124403959 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f3fff640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94f40b1df0 0x7f94f40cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f37fe640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94f40b21d0 0x7f94f41f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f8cae640 1 -- 172.21.15.172:0/124403959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94f40b21d0 msgr2=0x7f94f41f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f8cae640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94f40b21d0 0x7f94f41f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f8cae640 1 -- 172.21.15.172:0/124403959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94f40b1df0 msgr2=0x7f94f40cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f8cae640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94f40b1df0 0x7f94f40cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f8cae640 1 -- 172.21.15.172:0/124403959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f4131a50 con 0x7f94f40b4040 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f8cae640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94f40b4040 0x7f94f40cfd80 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b990 tx=0x7f94ec00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f0ff9640 1 -- 172.21.15.172:0/124403959 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec019070 con 0x7f94f40b4040 2024-04-14T08:42:25.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f0ff9640 1 -- 172.21.15.172:0/124403959 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ec00cb40 con 0x7f94f40b4040 2024-04-14T08:42:25.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94fa711640 1 -- 172.21.15.172:0/124403959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f41f40c0 con 0x7f94f40b4040 2024-04-14T08:42:25.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94f0ff9640 1 -- 172.21.15.172:0/124403959 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec014440 con 0x7f94f40b4040 2024-04-14T08:42:25.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.944+0000 7f94fa711640 1 -- 172.21.15.172:0/124403959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f4131fa0 con 0x7f94f40b4040 2024-04-14T08:42:25.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.945+0000 7f94fa711640 1 -- 172.21.15.172:0/124403959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f4068520 con 0x7f94f40b4040 2024-04-14T08:42:25.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.946+0000 7f94f0ff9640 1 -- 172.21.15.172:0/124403959 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94ec01c070 con 0x7f94f40b4040 2024-04-14T08:42:25.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.946+0000 7f94f0ff9640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f94e006c8f0 0x7f94e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:25.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.946+0000 7f94f0ff9640 1 -- 172.21.15.172:0/124403959 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7f94ec08ba80 con 0x7f94f40b4040 2024-04-14T08:42:25.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.946+0000 7f94f3fff640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f94e006c8f0 0x7f94e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:25.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.947+0000 7f94f3fff640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f94e006c8f0 0x7f94e006edb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f94e4036000 tx=0x7f94e403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:25.951+0000 7f94f0ff9640 1 -- 172.21.15.172:0/124403959 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f94ec059c60 con 0x7f94f40b4040 2024-04-14T08:42:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.127+0000 7f94fa711640 1 -- 172.21.15.172:0/124403959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94f4132710 con 0x7f94f40b4040 2024-04-14T08:42:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.128+0000 7f94f0ff9640 1 -- 172.21.15.172:0/124403959 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+17250 (secure 0 0 0) 0x7f94ec059560 con 0x7f94f40b4040 2024-04-14T08:42:26.128 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:26.129 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":105,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:42:25.145836+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:41:02.725765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-14T08:41:03.442157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/4028911489":"2024-04-15T08:42:25.145813+0000","172.21.15.172:6835/4028911489":"2024-04-15T08:42:25.145813+0000","172.21.15.177:6835/2602337717":"2024-04-15T08:41:48.748747+0000","172.21.15.177:6834/2602337717":"2024-04-15T08:41:48.748747+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:42:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.130+0000 7f94fa711640 1 -- 172.21.15.172:0/124403959 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f94e006c8f0 msgr2=0x7f94e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.130+0000 7f94fa711640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f94e006c8f0 0x7f94e006edb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f94e4036000 tx=0x7f94e403c040 comp rx=0 tx=0).stop 2024-04-14T08:42:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.130+0000 7f94fa711640 1 -- 172.21.15.172:0/124403959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94f40b4040 msgr2=0x7f94f40cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.130+0000 7f94fa711640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94f40b4040 0x7f94f40cfd80 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b990 tx=0x7f94ec00be50 comp rx=0 tx=0).stop 2024-04-14T08:42:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.130+0000 7f94fa711640 1 -- 172.21.15.172:0/124403959 shutdown_connections 2024-04-14T08:42:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.130+0000 7f94fa711640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f94e006c8f0 0x7f94e006edb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.130+0000 7f94fa711640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94f40b4040 0x7f94f40cfd80 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.130+0000 7f94fa711640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94f40b21d0 0x7f94f41f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.130+0000 7f94fa711640 1 --2- 172.21.15.172:0/124403959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94f40b1df0 0x7f94f40cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.130+0000 7f94fa711640 1 -- 172.21.15.172:0/124403959 >> 172.21.15.172:0/124403959 conn(0x7f94f406ec30 msgr2=0x7f94f40b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.130+0000 7f94fa711640 1 -- 172.21.15.172:0/124403959 shutdown_connections 2024-04-14T08:42:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.130+0000 7f94fa711640 1 -- 172.21.15.172:0/124403959 wait complete. 2024-04-14T08:42:26.141 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:41:02.725765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:41:03.442157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:42:26.142 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:42:26.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.247+0000 7f1672a0f640 1 Processor -- start 2024-04-14T08:42:26.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.247+0000 7f1672a0f640 1 -- start start 2024-04-14T08:42:26.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.247+0000 7f1672a0f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f166c13df30 0x7f166c0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.247+0000 7f1672a0f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f166c15c380 0x7f166c0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.247+0000 7f1672a0f640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166c15b620 con 0x7f166c140770 2024-04-14T08:42:26.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.247+0000 7f1672a0f640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f166c0cf9b0 con 0x7f166c13df30 2024-04-14T08:42:26.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.247+0000 7f1672a0f640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f166c1414a0 con 0x7f166c15c380 2024-04-14T08:42:26.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.248+0000 7f166bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f166c13df30 0x7f166c0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.248+0000 7f166bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f166c13df30 0x7f166c0b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35248/0 (socket says 172.21.15.172:35248) 2024-04-14T08:42:26.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.248+0000 7f166bfff640 1 -- 172.21.15.172:0/1372278067 learned_addr learned my addr 172.21.15.172:0/1372278067 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:26.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.248+0000 7f166b7fe640 1 --2- 172.21.15.172:0/1372278067 >> v2:172.21.15.177:3300/0 conn(0x7f166c15c380 0x7f166c0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.248+0000 7f166bfff640 1 -- 172.21.15.172:0/1372278067 >> v2:172.21.15.177:3300/0 conn(0x7f166c15c380 msgr2=0x7f166c0b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:26.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.248+0000 7f166bfff640 1 --2- 172.21.15.172:0/1372278067 >> v2:172.21.15.177:3300/0 conn(0x7f166c15c380 0x7f166c0b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.248+0000 7f166bfff640 1 -- 172.21.15.172:0/1372278067 >> v1:172.21.15.177:6789/0 conn(0x7f166c140770 legacy=0x7f166c140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:26.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.248+0000 7f166bfff640 1 -- 172.21.15.172:0/1372278067 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c15b8a0 con 0x7f166c13df30 2024-04-14T08:42:26.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.248+0000 7f166b7fe640 1 --2- 172.21.15.172:0/1372278067 >> v2:172.21.15.177:3300/0 conn(0x7f166c15c380 0x7f166c0b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:26.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.249+0000 7f166bfff640 1 --2- 172.21.15.172:0/1372278067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166c13df30 0x7f166c0b1cd0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f166c06c4d0 tx=0x7f165c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7fbc421845f8e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:26.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.249+0000 7f166affd640 1 -- 172.21.15.172:0/1372278067 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c035020 con 0x7f166c13df30 2024-04-14T08:42:26.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.249+0000 7f166affd640 1 -- 172.21.15.172:0/1372278067 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c031870 con 0x7f166c13df30 2024-04-14T08:42:26.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.249+0000 7f166affd640 1 -- 172.21.15.172:0/1372278067 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c031c40 con 0x7f166c13df30 2024-04-14T08:42:26.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.249+0000 7f1672a0f640 1 -- 172.21.15.172:0/1372278067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166c13df30 msgr2=0x7f166c0b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.249+0000 7f1672a0f640 1 --2- 172.21.15.172:0/1372278067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166c13df30 0x7f166c0b1cd0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f166c06c4d0 tx=0x7f165c02b910 comp rx=0 tx=0).stop 2024-04-14T08:42:26.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.249+0000 7f1672a0f640 1 -- 172.21.15.172:0/1372278067 shutdown_connections 2024-04-14T08:42:26.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.249+0000 7f1672a0f640 1 --2- 172.21.15.172:0/1372278067 >> v2:172.21.15.177:3300/0 conn(0x7f166c15c380 0x7f166c0b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.249+0000 7f1672a0f640 1 --2- 172.21.15.172:0/1372278067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166c13df30 0x7f166c0b1cd0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.249+0000 7f1672a0f640 1 -- 172.21.15.172:0/1372278067 >> 172.21.15.172:0/1372278067 conn(0x7f166c06b520 msgr2=0x7f166c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:26.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.249+0000 7f1672a0f640 1 -- 172.21.15.172:0/1372278067 shutdown_connections 2024-04-14T08:42:26.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.249+0000 7f1672a0f640 1 -- 172.21.15.172:0/1372278067 wait complete. 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.250+0000 7f1672a0f640 1 Processor -- start 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.251+0000 7f1672a0f640 1 -- start start 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.251+0000 7f1672a0f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f166c13df30 0x7f166c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.251+0000 7f1672a0f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f166c140770 0x7f166c1f0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.251+0000 7f1672a0f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166c15c380 0x7f166c151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.251+0000 7f1672a0f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f166c13e310 con 0x7f166c15c380 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.251+0000 7f1672a0f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f166c15be60 con 0x7f166c140770 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.251+0000 7f1672a0f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f166c0ce9a0 con 0x7f166c13df30 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.251+0000 7f1670fac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166c15c380 0x7f166c151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.251+0000 7f1670fac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166c15c380 0x7f166c151370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35250/0 (socket says 172.21.15.172:35250) 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.251+0000 7f1670fac640 1 -- 172.21.15.172:0/383724811 learned_addr learned my addr 172.21.15.172:0/383724811 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f166b7fe640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f166c140770 0x7f166c1f0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f166bfff640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f166c13df30 0x7f166c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f1670fac640 1 -- 172.21.15.172:0/383724811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f166c13df30 msgr2=0x7f166c154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f1670fac640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f166c13df30 0x7f166c154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f1670fac640 1 -- 172.21.15.172:0/383724811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f166c140770 msgr2=0x7f166c1f0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f1670fac640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f166c140770 0x7f166c1f0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f1670fac640 1 -- 172.21.15.172:0/383724811 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c1f0bd0 con 0x7f166c15c380 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f166b7fe640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f166c140770 0x7f166c1f0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f166bfff640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f166c13df30 0x7f166c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f1670fac640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166c15c380 0x7f166c151370 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f166400c7b0 tx=0x7f166400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f1668ff9640 1 -- 172.21.15.172:0/383724811 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664004b60 con 0x7f166c15c380 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f1668ff9640 1 -- 172.21.15.172:0/383724811 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1664004d00 con 0x7f166c15c380 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f1668ff9640 1 -- 172.21.15.172:0/383724811 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166400aa90 con 0x7f166c15c380 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f1672a0f640 1 -- 172.21.15.172:0/383724811 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c1518b0 con 0x7f166c15c380 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.252+0000 7f1672a0f640 1 -- 172.21.15.172:0/383724811 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c155270 con 0x7f166c15c380 2024-04-14T08:42:26.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.253+0000 7f1672a0f640 1 -- 172.21.15.172:0/383724811 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166c068590 con 0x7f166c15c380 2024-04-14T08:42:26.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.253+0000 7f1668ff9640 1 -- 172.21.15.172:0/383724811 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f166400ac30 con 0x7f166c15c380 2024-04-14T08:42:26.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.254+0000 7f1668ff9640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f165806c8f0 0x7f165806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.254+0000 7f166bfff640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f165806c8f0 0x7f165806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.254+0000 7f1668ff9640 1 -- 172.21.15.172:0/383724811 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7f166408c000 con 0x7f166c15c380 2024-04-14T08:42:26.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.254+0000 7f166bfff640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f165806c8f0 0x7f165806edb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f165c036000 tx=0x7f165c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:26.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.258+0000 7f1668ff9640 1 -- 172.21.15.172:0/383724811 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1664020070 con 0x7f166c15c380 2024-04-14T08:42:26.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.433+0000 7f1672a0f640 1 -- 172.21.15.172:0/383724811 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f166c151e60 con 0x7f166c15c380 2024-04-14T08:42:26.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.433+0000 7f1668ff9640 1 -- 172.21.15.172:0/383724811 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f166c1f0bd0 con 0x7f166c15c380 2024-04-14T08:42:26.434 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:42:26.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.435+0000 7f1672a0f640 1 -- 172.21.15.172:0/383724811 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f165806c8f0 msgr2=0x7f165806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.435+0000 7f1672a0f640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f165806c8f0 0x7f165806edb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f165c036000 tx=0x7f165c03c040 comp rx=0 tx=0).stop 2024-04-14T08:42:26.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.435+0000 7f1672a0f640 1 -- 172.21.15.172:0/383724811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166c15c380 msgr2=0x7f166c151370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.435+0000 7f1672a0f640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166c15c380 0x7f166c151370 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f166400c7b0 tx=0x7f166400cc70 comp rx=0 tx=0).stop 2024-04-14T08:42:26.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.435+0000 7f1672a0f640 1 -- 172.21.15.172:0/383724811 shutdown_connections 2024-04-14T08:42:26.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.435+0000 7f1672a0f640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f165806c8f0 0x7f165806edb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.435+0000 7f1672a0f640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f166c15c380 0x7f166c151370 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.435+0000 7f1672a0f640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f166c140770 0x7f166c1f0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.435+0000 7f1672a0f640 1 --2- 172.21.15.172:0/383724811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f166c13df30 0x7f166c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.435+0000 7f1672a0f640 1 -- 172.21.15.172:0/383724811 >> 172.21.15.172:0/383724811 conn(0x7f166c06b520 msgr2=0x7f166c13e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:26.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.435+0000 7f1672a0f640 1 -- 172.21.15.172:0/383724811 shutdown_connections 2024-04-14T08:42:26.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.435+0000 7f1672a0f640 1 -- 172.21.15.172:0/383724811 wait complete. 2024-04-14T08:42:26.447 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:42:26.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.550+0000 7fa5e3fbb640 1 Processor -- start 2024-04-14T08:42:26.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.550+0000 7fa5e3fbb640 1 -- start start 2024-04-14T08:42:26.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.551+0000 7fa5e3fbb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa5dc0b23e0 0x7fa5dc140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.551+0000 7fa5e3fbb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa5dc15c380 0x7fa5dc140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.551+0000 7fa5e3fbb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5dc15b620 con 0x7fa5dc13df30 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.551+0000 7fa5e3fbb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa5dc0cf9b0 con 0x7fa5dc0b23e0 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.551+0000 7fa5e3fbb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fa5dc1411f0 con 0x7fa5dc15c380 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.551+0000 7fa5e1556640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa5dc0b23e0 0x7fa5dc140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.551+0000 7fa5e1556640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa5dc0b23e0 0x7fa5dc140770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35262/0 (socket says 172.21.15.172:35262) 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.551+0000 7fa5e1556640 1 -- 172.21.15.172:0/3728044712 learned_addr learned my addr 172.21.15.172:0/3728044712 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.551+0000 7fa5e2558640 1 --2- 172.21.15.172:0/3728044712 >> v2:172.21.15.177:3300/0 conn(0x7fa5dc15c380 0x7fa5dc140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.552+0000 7fa5e1556640 1 -- 172.21.15.172:0/3728044712 >> v2:172.21.15.177:3300/0 conn(0x7fa5dc15c380 msgr2=0x7fa5dc140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.552+0000 7fa5e1556640 1 --2- 172.21.15.172:0/3728044712 >> v2:172.21.15.177:3300/0 conn(0x7fa5dc15c380 0x7fa5dc140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.552+0000 7fa5e1556640 1 -- 172.21.15.172:0/3728044712 >> v1:172.21.15.172:6789/0 conn(0x7fa5dc13df30 legacy=0x7fa5dc0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.552+0000 7fa5e1556640 1 -- 172.21.15.172:0/3728044712 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc1414a0 con 0x7fa5dc0b23e0 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.552+0000 7fa5e1556640 1 --2- 172.21.15.172:0/3728044712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc0b23e0 0x7fa5dc140770 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc1591f0 tx=0x7fa5cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5f0ffa22ab4179 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:26.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.552+0000 7fa5e0d55640 1 -- 172.21.15.172:0/3728044712 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc034070 con 0x7fa5dc0b23e0 2024-04-14T08:42:26.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.552+0000 7fa5e0d55640 1 -- 172.21.15.172:0/3728044712 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5cc030860 con 0x7fa5dc0b23e0 2024-04-14T08:42:26.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.552+0000 7fa5e0d55640 1 -- 172.21.15.172:0/3728044712 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc030c30 con 0x7fa5dc0b23e0 2024-04-14T08:42:26.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.553+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/3728044712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc0b23e0 msgr2=0x7fa5dc140770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.553+0000 7fa5e3fbb640 1 --2- 172.21.15.172:0/3728044712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc0b23e0 0x7fa5dc140770 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc1591f0 tx=0x7fa5cc02b9a0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.553+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/3728044712 shutdown_connections 2024-04-14T08:42:26.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.553+0000 7fa5e3fbb640 1 --2- 172.21.15.172:0/3728044712 >> v2:172.21.15.177:3300/0 conn(0x7fa5dc15c380 0x7fa5dc140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.553+0000 7fa5e3fbb640 1 --2- 172.21.15.172:0/3728044712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc0b23e0 0x7fa5dc140770 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.553+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/3728044712 >> 172.21.15.172:0/3728044712 conn(0x7fa5dc06b520 msgr2=0x7fa5dc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:26.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.553+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/3728044712 shutdown_connections 2024-04-14T08:42:26.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.553+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/3728044712 wait complete. 2024-04-14T08:42:26.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.554+0000 7fa5e3fbb640 1 Processor -- start 2024-04-14T08:42:26.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.554+0000 7fa5e3fbb640 1 -- start start 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e3fbb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5dc0b23e0 0x7fa5dc136bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e3fbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5dc13df30 0x7fa5dc072220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e3fbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc15c380 0x7fa5dc072760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e3fbb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc15be60 con 0x7fa5dc15c380 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e3fbb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc15bb80 con 0x7fa5dc0b23e0 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e3fbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa5dc13e310 con 0x7fa5dc13df30 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e2558640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc15c380 0x7fa5dc072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e2558640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc15c380 0x7fa5dc072760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35264/0 (socket says 172.21.15.172:35264) 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e2558640 1 -- 172.21.15.172:0/1505355950 learned_addr learned my addr 172.21.15.172:0/1505355950 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e1556640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5dc13df30 0x7fa5dc072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e2558640 1 -- 172.21.15.172:0/1505355950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5dc13df30 msgr2=0x7fa5dc072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e2558640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5dc13df30 0x7fa5dc072220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e2558640 1 -- 172.21.15.172:0/1505355950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5dc0b23e0 msgr2=0x7fa5dc136bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e1d57640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5dc0b23e0 0x7fa5dc136bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e2558640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5dc0b23e0 0x7fa5dc136bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e2558640 1 -- 172.21.15.172:0/1505355950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc137300 con 0x7fa5dc15c380 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e1556640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5dc13df30 0x7fa5dc072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.555+0000 7fa5e1d57640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5dc0b23e0 0x7fa5dc136bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.556+0000 7fa5e2558640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc15c380 0x7fa5dc072760 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400d5d0 tx=0x7fa5d400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.556+0000 7fa5d27fc640 1 -- 172.21.15.172:0/1505355950 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40125f0 con 0x7fa5dc15c380 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.556+0000 7fa5d27fc640 1 -- 172.21.15.172:0/1505355950 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d4019070 con 0x7fa5dc15c380 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.556+0000 7fa5d27fc640 1 -- 172.21.15.172:0/1505355950 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40129d0 con 0x7fa5dc15c380 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.556+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/1505355950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5dc1375e0 con 0x7fa5dc15c380 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.556+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/1505355950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5dc137ba0 con 0x7fa5dc15c380 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.557+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/1505355950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5dc068590 con 0x7fa5dc15c380 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.557+0000 7fa5d27fc640 1 -- 172.21.15.172:0/1505355950 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5d4012b70 con 0x7fa5dc15c380 2024-04-14T08:42:26.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.558+0000 7fa5d27fc640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa5c006c8f0 0x7fa5c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.558+0000 7fa5d27fc640 1 -- 172.21.15.172:0/1505355950 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7fa5d408c280 con 0x7fa5dc15c380 2024-04-14T08:42:26.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.558+0000 7fa5e1d57640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa5c006c8f0 0x7fa5c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.558+0000 7fa5e1d57640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa5c006c8f0 0x7fa5c006edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8001d70 tx=0x7fa5d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:26.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.562+0000 7fa5d27fc640 1 -- 172.21.15.172:0/1505355950 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa5d405a3d0 con 0x7fa5dc15c380 2024-04-14T08:42:26.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.736+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/1505355950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa5dc0b38b0 con 0x7fa5dc15c380 2024-04-14T08:42:26.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.737+0000 7fa5d27fc640 1 -- 172.21.15.172:0/1505355950 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7fa5d4059cd0 con 0x7fa5dc15c380 2024-04-14T08:42:26.737 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:42:26.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.739+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/1505355950 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa5c006c8f0 msgr2=0x7fa5c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.739+0000 7fa5e3fbb640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa5c006c8f0 0x7fa5c006edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8001d70 tx=0x7fa5d8009040 comp rx=0 tx=0).stop 2024-04-14T08:42:26.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.739+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/1505355950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc15c380 msgr2=0x7fa5dc072760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.739+0000 7fa5e3fbb640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc15c380 0x7fa5dc072760 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400d5d0 tx=0x7fa5d400da90 comp rx=0 tx=0).stop 2024-04-14T08:42:26.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.739+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/1505355950 shutdown_connections 2024-04-14T08:42:26.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.739+0000 7fa5e3fbb640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa5c006c8f0 0x7fa5c006edb0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.739+0000 7fa5e3fbb640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc15c380 0x7fa5dc072760 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.739+0000 7fa5e3fbb640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5dc13df30 0x7fa5dc072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.739+0000 7fa5e3fbb640 1 --2- 172.21.15.172:0/1505355950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5dc0b23e0 0x7fa5dc136bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.739+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/1505355950 >> 172.21.15.172:0/1505355950 conn(0x7fa5dc06b520 msgr2=0x7fa5dc06d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:26.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.739+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/1505355950 shutdown_connections 2024-04-14T08:42:26.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.739+0000 7fa5e3fbb640 1 -- 172.21.15.172:0/1505355950 wait complete. 2024-04-14T08:42:26.750 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:42:26.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.854+0000 7f9fdfe20640 1 Processor -- start 2024-04-14T08:42:26.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.855+0000 7f9fdfe20640 1 -- start start 2024-04-14T08:42:26.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.855+0000 7f9fdfe20640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9fd806d710 0x7f9fd806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.855+0000 7f9fdfe20640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd80701b0 con 0x7f9fd8071250 2024-04-14T08:42:26.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.855+0000 7f9fdfe20640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd8070490 con 0x7f9fd80d1a20 2024-04-14T08:42:26.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.855+0000 7f9fdfe20640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f9fd8070dc0 con 0x7f9fd806d710 2024-04-14T08:42:26.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.855+0000 7f9fdd3bb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9fd8071250 0x7f9fd806d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:37522/0 (socket says 172.21.15.172:37522) 2024-04-14T08:42:26.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.855+0000 7f9fdd3bb640 1 -- 172.21.15.172:0/2661588748 learned_addr learned my addr 172.21.15.172:0/2661588748 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:26.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2426483212 0 0) 0x7f9fd80701b0 con 0x7f9fd8071250 2024-04-14T08:42:26.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fde3bd640 1 --2- 172.21.15.172:0/2661588748 >> v2:172.21.15.177:3300/0 conn(0x7f9fd806d710 0x7f9fd806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fc4003520 con 0x7f9fd8071250 2024-04-14T08:42:26.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3532728642 0 0) 0x7f9fc4003520 con 0x7f9fd8071250 2024-04-14T08:42:26.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 >> v2:172.21.15.177:3300/0 conn(0x7f9fd806d710 msgr2=0x7f9fd806fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:26.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 --2- 172.21.15.172:0/2661588748 >> v2:172.21.15.177:3300/0 conn(0x7f9fd806d710 0x7f9fd806fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 >> v1:172.21.15.177:6789/0 conn(0x7f9fd80d1a20 legacy=0x7f9fd80cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd8070770 con 0x7f9fd8071250 2024-04-14T08:42:26.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f9fc8001670 con 0x7f9fd8071250 2024-04-14T08:42:26.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 >> v1:172.21.15.172:6789/0 conn(0x7f9fd8071250 legacy=0x7f9fd806d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 --2- 172.21.15.172:0/2661588748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 0x7f9fc4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 --2- 172.21.15.172:0/2661588748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fc4004800 0x7f9fc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 --2- 172.21.15.172:0/2661588748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fc40051c0 0x7f9fc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd8070dc0 con 0x7f9fc40039b0 2024-04-14T08:42:26.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc400c160 con 0x7f9fc40051c0 2024-04-14T08:42:26.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc400c360 con 0x7f9fc4004800 2024-04-14T08:42:26.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fddbbc640 1 --2- 172.21.15.172:0/2661588748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 0x7f9fc4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fde3bd640 1 --2- 172.21.15.172:0/2661588748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fc40051c0 0x7f9fc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fdd3bb640 1 --2- 172.21.15.172:0/2661588748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fc4004800 0x7f9fc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fddbbc640 1 -- 172.21.15.172:0/2661588748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fc4004800 msgr2=0x7f9fc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:26.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.856+0000 7f9fddbbc640 1 --2- 172.21.15.172:0/2661588748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fc4004800 0x7f9fc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.857+0000 7f9fddbbc640 1 -- 172.21.15.172:0/2661588748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fc40051c0 msgr2=0x7f9fc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:26.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.857+0000 7f9fddbbc640 1 --2- 172.21.15.172:0/2661588748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fc40051c0 0x7f9fc400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.857+0000 7f9fddbbc640 1 -- 172.21.15.172:0/2661588748 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc4003520 con 0x7f9fc40039b0 2024-04-14T08:42:26.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.857+0000 7f9fde3bd640 1 -- 172.21.15.172:0/2661588748 reap_dead start 2024-04-14T08:42:26.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.857+0000 7f9fdd3bb640 1 --2- 172.21.15.172:0/2661588748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fc4004800 0x7f9fc4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:26.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.857+0000 7f9fddbbc640 1 --2- 172.21.15.172:0/2661588748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 0x7f9fc4003dd0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4001e50 tx=0x7f9fc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fd810c3c521605 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:26.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.857+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4001a70 con 0x7f9fc40039b0 2024-04-14T08:42:26.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.857+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd4003b70 con 0x7f9fc40039b0 2024-04-14T08:42:26.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.857+0000 7f9fdcbba640 1 -- 172.21.15.172:0/2661588748 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40050d0 con 0x7f9fc40039b0 2024-04-14T08:42:26.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.858+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2661588748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 msgr2=0x7f9fc4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.858+0000 7f9fdfe20640 1 --2- 172.21.15.172:0/2661588748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 0x7f9fc4003dd0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4001e50 tx=0x7f9fc4005830 comp rx=0 tx=0).stop 2024-04-14T08:42:26.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.858+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2661588748 shutdown_connections 2024-04-14T08:42:26.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.858+0000 7f9fdfe20640 1 --2- 172.21.15.172:0/2661588748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 0x7f9fc4003dd0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.858+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2661588748 >> 172.21.15.172:0/2661588748 conn(0x7f9fd808c000 msgr2=0x7f9fd80d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:26.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.858+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2661588748 shutdown_connections 2024-04-14T08:42:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.858+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2661588748 wait complete. 2024-04-14T08:42:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.859+0000 7f9fdfe20640 1 Processor -- start 2024-04-14T08:42:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.859+0000 7f9fdfe20640 1 -- start start 2024-04-14T08:42:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fdfe20640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 0x7f9fd8158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fdfe20640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fd815a8e0 0x7f9fd80cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fdfe20640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fd81f8250 0x7f9fd80b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fdfe20640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd80d1e00 con 0x7f9fc40039b0 2024-04-14T08:42:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fdfe20640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd806fb80 con 0x7f9fd815a8e0 2024-04-14T08:42:26.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fdfe20640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd81f7ae0 con 0x7f9fd81f8250 2024-04-14T08:42:26.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fddbbc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 0x7f9fd8158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fddbbc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 0x7f9fd8158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35268/0 (socket says 172.21.15.172:35268) 2024-04-14T08:42:26.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fddbbc640 1 -- 172.21.15.172:0/2612197251 learned_addr learned my addr 172.21.15.172:0/2612197251 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:26.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fddbbc640 1 -- 172.21.15.172:0/2612197251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fd81f8250 msgr2=0x7f9fd80b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fddbbc640 1 --2- 172.21.15.172:0/2612197251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fd81f8250 0x7f9fd80b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fddbbc640 1 -- 172.21.15.172:0/2612197251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fd815a8e0 msgr2=0x7f9fd80cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:26.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fddbbc640 1 --2- 172.21.15.172:0/2612197251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fd815a8e0 0x7f9fd80cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:26.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fddbbc640 1 -- 172.21.15.172:0/2612197251 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd815ae00 con 0x7f9fc40039b0 2024-04-14T08:42:26.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fddbbc640 1 --2- 172.21.15.172:0/2612197251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 0x7f9fd8158d40 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000f80 tx=0x7f9fd40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:26.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fce7fc640 1 -- 172.21.15.172:0/2612197251 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4003970 con 0x7f9fc40039b0 2024-04-14T08:42:26.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.860+0000 7f9fce7fc640 1 -- 172.21.15.172:0/2612197251 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd4006b50 con 0x7f9fc40039b0 2024-04-14T08:42:26.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.861+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2612197251 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd815b0e0 con 0x7f9fc40039b0 2024-04-14T08:42:26.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.861+0000 7f9fce7fc640 1 -- 172.21.15.172:0/2612197251 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4007020 con 0x7f9fc40039b0 2024-04-14T08:42:26.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.861+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2612197251 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd81f7d30 con 0x7f9fc40039b0 2024-04-14T08:42:26.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.861+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2612197251 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd8075490 con 0x7f9fc40039b0 2024-04-14T08:42:26.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.862+0000 7f9fce7fc640 1 -- 172.21.15.172:0/2612197251 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fd4049060 con 0x7f9fc40039b0 2024-04-14T08:42:26.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.862+0000 7f9fce7fc640 1 --2- 172.21.15.172:0/2612197251 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9fbc06c8f0 0x7f9fbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:26.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.862+0000 7f9fdd3bb640 1 --2- 172.21.15.172:0/2612197251 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9fbc06c8f0 0x7f9fbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:26.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.863+0000 7f9fce7fc640 1 -- 172.21.15.172:0/2612197251 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7f9fd40ad040 con 0x7f9fc40039b0 2024-04-14T08:42:26.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.863+0000 7f9fdd3bb640 1 --2- 172.21.15.172:0/2612197251 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9fbc06c8f0 0x7f9fbc06edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8000fb0 tx=0x7f9fc8001e00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:26.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:26.867+0000 7f9fce7fc640 1 -- 172.21.15.172:0/2612197251 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9fd4033020 con 0x7f9fc40039b0 2024-04-14T08:42:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.041+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2612197251 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9fd80d2820 con 0x7f9fc40039b0 2024-04-14T08:42:27.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.042+0000 7f9fce7fc640 1 -- 172.21.15.172:0/2612197251 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f9fd407b2b0 con 0x7f9fc40039b0 2024-04-14T08:42:27.042 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:42:27.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.044+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2612197251 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9fbc06c8f0 msgr2=0x7f9fbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.044+0000 7f9fdfe20640 1 --2- 172.21.15.172:0/2612197251 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9fbc06c8f0 0x7f9fbc06edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8000fb0 tx=0x7f9fc8001e00 comp rx=0 tx=0).stop 2024-04-14T08:42:27.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.044+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2612197251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 msgr2=0x7f9fd8158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.044+0000 7f9fdfe20640 1 --2- 172.21.15.172:0/2612197251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 0x7f9fd8158d40 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000f80 tx=0x7f9fd40014a0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.044+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2612197251 shutdown_connections 2024-04-14T08:42:27.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.044+0000 7f9fdfe20640 1 --2- 172.21.15.172:0/2612197251 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9fbc06c8f0 0x7f9fbc06edb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.044+0000 7f9fdfe20640 1 --2- 172.21.15.172:0/2612197251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9fd81f8250 0x7f9fd80b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.044+0000 7f9fdfe20640 1 --2- 172.21.15.172:0/2612197251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9fd815a8e0 0x7f9fd80cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.044+0000 7f9fdfe20640 1 --2- 172.21.15.172:0/2612197251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fc40039b0 0x7f9fd8158d40 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.044+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2612197251 >> 172.21.15.172:0/2612197251 conn(0x7f9fd808c000 msgr2=0x7f9fd80ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:27.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.044+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2612197251 shutdown_connections 2024-04-14T08:42:27.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.044+0000 7f9fdfe20640 1 -- 172.21.15.172:0/2612197251 wait complete. 2024-04-14T08:42:27.055 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:42:27.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.156+0000 7fe166b1e640 1 Processor -- start 2024-04-14T08:42:27.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.156+0000 7fe166b1e640 1 -- start start 2024-04-14T08:42:27.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.156+0000 7fe166b1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe1600cefd0 0x7fe16006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.156+0000 7fe166b1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe16008c4c0 0x7fe16006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.156+0000 7fe166b1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1600d2b10 con 0x7fe1600b7a60 2024-04-14T08:42:27.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.156+0000 7fe166b1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fe1600ce420 con 0x7fe1600cefd0 2024-04-14T08:42:27.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.156+0000 7fe166b1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe1600d0ad0 con 0x7fe16008c4c0 2024-04-14T08:42:27.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.156+0000 7fe15ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe1600cefd0 0x7fe16006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.156+0000 7fe1648ba640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe1600b7a60 0x7fe1600ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:37534/0 (socket says 172.21.15.172:37534) 2024-04-14T08:42:27.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.157+0000 7fe15ffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe1600cefd0 0x7fe16006cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35274/0 (socket says 172.21.15.172:35274) 2024-04-14T08:42:27.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.157+0000 7fe1648ba640 1 -- 172.21.15.172:0/655024063 learned_addr learned my addr 172.21.15.172:0/655024063 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:27.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.157+0000 7fe1650bb640 1 --2- 172.21.15.172:0/655024063 >> v2:172.21.15.177:3300/0 conn(0x7fe16008c4c0 0x7fe16006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.157+0000 7fe15ffff640 1 -- 172.21.15.172:0/655024063 >> v2:172.21.15.177:3300/0 conn(0x7fe16008c4c0 msgr2=0x7fe16006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:27.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.157+0000 7fe15ffff640 1 --2- 172.21.15.172:0/655024063 >> v2:172.21.15.177:3300/0 conn(0x7fe16008c4c0 0x7fe16006d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.157+0000 7fe15ffff640 1 -- 172.21.15.172:0/655024063 >> v1:172.21.15.172:6789/0 conn(0x7fe1600b7a60 legacy=0x7fe1600ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.157+0000 7fe15ffff640 1 -- 172.21.15.172:0/655024063 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1600d0db0 con 0x7fe1600cefd0 2024-04-14T08:42:27.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.157+0000 7fe15f7fe640 1 -- 172.21.15.172:0/655024063 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3138165810 0 0) 0x7fe1600d2b10 con 0x7fe1600b7a60 2024-04-14T08:42:27.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.157+0000 7fe1650bb640 1 --2- 172.21.15.172:0/655024063 >> v2:172.21.15.177:3300/0 conn(0x7fe16008c4c0 0x7fe16006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:27.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.158+0000 7fe15ffff640 1 --2- 172.21.15.172:0/655024063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600cefd0 0x7fe16006cc30 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe1600732d0 tx=0x7fe15402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6ac0f26d99318e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:27.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.158+0000 7fe15f7fe640 1 -- 172.21.15.172:0/655024063 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1540315a0 con 0x7fe1600cefd0 2024-04-14T08:42:27.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.158+0000 7fe15f7fe640 1 -- 172.21.15.172:0/655024063 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe154037070 con 0x7fe1600cefd0 2024-04-14T08:42:27.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.158+0000 7fe15f7fe640 1 -- 172.21.15.172:0/655024063 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154031960 con 0x7fe1600cefd0 2024-04-14T08:42:27.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.158+0000 7fe166b1e640 1 -- 172.21.15.172:0/655024063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600cefd0 msgr2=0x7fe16006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.158+0000 7fe166b1e640 1 --2- 172.21.15.172:0/655024063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600cefd0 0x7fe16006cc30 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe1600732d0 tx=0x7fe15402b840 comp rx=0 tx=0).stop 2024-04-14T08:42:27.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.158+0000 7fe166b1e640 1 -- 172.21.15.172:0/655024063 shutdown_connections 2024-04-14T08:42:27.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.158+0000 7fe166b1e640 1 --2- 172.21.15.172:0/655024063 >> v2:172.21.15.177:3300/0 conn(0x7fe16008c4c0 0x7fe16006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.158+0000 7fe166b1e640 1 --2- 172.21.15.172:0/655024063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600cefd0 0x7fe16006cc30 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.158+0000 7fe166b1e640 1 -- 172.21.15.172:0/655024063 >> 172.21.15.172:0/655024063 conn(0x7fe16008c000 msgr2=0x7fe1600d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:27.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.158+0000 7fe166b1e640 1 -- 172.21.15.172:0/655024063 shutdown_connections 2024-04-14T08:42:27.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.158+0000 7fe166b1e640 1 -- 172.21.15.172:0/655024063 wait complete. 2024-04-14T08:42:27.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.159+0000 7fe166b1e640 1 Processor -- start 2024-04-14T08:42:27.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.159+0000 7fe166b1e640 1 -- start start 2024-04-14T08:42:27.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.160+0000 7fe166b1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe16008c4c0 0x7fe16015c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.160+0000 7fe166b1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600b7a60 0x7fe160161c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.160+0000 7fe166b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1600cefd0 0x7fe1601f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.160+0000 7fe166b1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1600b7e40 con 0x7fe1600b7a60 2024-04-14T08:42:27.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.160+0000 7fe166b1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1600b4e90 con 0x7fe16008c4c0 2024-04-14T08:42:27.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.160+0000 7fe166b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe1600d2750 con 0x7fe1600cefd0 2024-04-14T08:42:27.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.160+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600b7a60 0x7fe160161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.160+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600b7a60 0x7fe160161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35286/0 (socket says 172.21.15.172:35286) 2024-04-14T08:42:27.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.160+0000 7fe15ffff640 1 -- 172.21.15.172:0/1411764558 learned_addr learned my addr 172.21.15.172:0/1411764558 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:27.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe1648ba640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe16008c4c0 0x7fe16015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe1650bb640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1600cefd0 0x7fe1601f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe15ffff640 1 -- 172.21.15.172:0/1411764558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1600cefd0 msgr2=0x7fe1601f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe15ffff640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1600cefd0 0x7fe1601f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe15ffff640 1 -- 172.21.15.172:0/1411764558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe16008c4c0 msgr2=0x7fe16015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe15ffff640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe16008c4c0 0x7fe16015c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe15ffff640 1 -- 172.21.15.172:0/1411764558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16015f220 con 0x7fe1600b7a60 2024-04-14T08:42:27.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe1648ba640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe16008c4c0 0x7fe16015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:27.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe1650bb640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1600cefd0 0x7fe1601f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:27.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe15ffff640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600b7a60 0x7fe160161c10 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe154000c00 tx=0x7fe1540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:27.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe15d7fa640 1 -- 172.21.15.172:0/1411764558 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154004230 con 0x7fe1600b7a60 2024-04-14T08:42:27.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe15d7fa640 1 -- 172.21.15.172:0/1411764558 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1540043f0 con 0x7fe1600b7a60 2024-04-14T08:42:27.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe166b1e640 1 -- 172.21.15.172:0/1411764558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16015f500 con 0x7fe1600b7a60 2024-04-14T08:42:27.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe15d7fa640 1 -- 172.21.15.172:0/1411764558 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154003850 con 0x7fe1600b7a60 2024-04-14T08:42:27.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.161+0000 7fe166b1e640 1 -- 172.21.15.172:0/1411764558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16015fac0 con 0x7fe1600b7a60 2024-04-14T08:42:27.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.162+0000 7fe166b1e640 1 -- 172.21.15.172:0/1411764558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe160075490 con 0x7fe1600b7a60 2024-04-14T08:42:27.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.163+0000 7fe15d7fa640 1 -- 172.21.15.172:0/1411764558 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1540039f0 con 0x7fe1600b7a60 2024-04-14T08:42:27.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.163+0000 7fe15d7fa640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe14c06c8f0 0x7fe14c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.163+0000 7fe1648ba640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe14c06c8f0 0x7fe14c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.163+0000 7fe15d7fa640 1 -- 172.21.15.172:0/1411764558 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7fe15402f030 con 0x7fe1600b7a60 2024-04-14T08:42:27.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.163+0000 7fe1648ba640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe14c06c8f0 0x7fe14c06edb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe150000f60 tx=0x7fe150009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:27.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.168+0000 7fe15d7fa640 1 -- 172.21.15.172:0/1411764558 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe154041070 con 0x7fe1600b7a60 2024-04-14T08:42:27.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.381+0000 7fe166b1e640 1 -- 172.21.15.172:0/1411764558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe16015fc90 con 0x7fe1600b7a60 2024-04-14T08:42:27.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.381+0000 7fe15d7fa640 1 -- 172.21.15.172:0/1411764558 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5182 (secure 0 0 0) 0x7fe15407bf50 con 0x7fe1600b7a60 2024-04-14T08:42:27.382 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:27.382 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8005,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3289017090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3289017090},{"type":"v1","addr":"172.21.15.177:6835","nonce":3289017090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8880,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1467868914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1467868914},{"type":"v1","addr":"172.21.15.172:6837","nonce":1467868914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":81,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:41:04.492773+0000","modified":"2024-04-14T08:42:25.146002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-14T08:42:27.382 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 82 2024-04-14T08:42:27.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.383+0000 7fe166b1e640 1 -- 172.21.15.172:0/1411764558 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe14c06c8f0 msgr2=0x7fe14c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.383+0000 7fe166b1e640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe14c06c8f0 0x7fe14c06edb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe150000f60 tx=0x7fe150009040 comp rx=0 tx=0).stop 2024-04-14T08:42:27.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.383+0000 7fe166b1e640 1 -- 172.21.15.172:0/1411764558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600b7a60 msgr2=0x7fe160161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.383+0000 7fe166b1e640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600b7a60 0x7fe160161c10 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe154000c00 tx=0x7fe1540046c0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.383+0000 7fe166b1e640 1 -- 172.21.15.172:0/1411764558 shutdown_connections 2024-04-14T08:42:27.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.383+0000 7fe166b1e640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe14c06c8f0 0x7fe14c06edb0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.383+0000 7fe166b1e640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1600cefd0 0x7fe1601f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.383+0000 7fe166b1e640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600b7a60 0x7fe160161c10 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.383+0000 7fe166b1e640 1 --2- 172.21.15.172:0/1411764558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe16008c4c0 0x7fe16015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.383+0000 7fe166b1e640 1 -- 172.21.15.172:0/1411764558 >> 172.21.15.172:0/1411764558 conn(0x7fe16008c000 msgr2=0x7fe160074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:27.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.384+0000 7fe166b1e640 1 -- 172.21.15.172:0/1411764558 shutdown_connections 2024-04-14T08:42:27.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.384+0000 7fe166b1e640 1 -- 172.21.15.172:0/1411764558 wait complete. 2024-04-14T08:42:27.395 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:42:27.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.495+0000 7f493c008640 1 Processor -- start 2024-04-14T08:42:27.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.495+0000 7f493c008640 1 -- start start 2024-04-14T08:42:27.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.496+0000 7f493c008640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f49340d2830 0x7f49340d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.496+0000 7f493c008640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49340d3150 con 0x7f493415e890 2024-04-14T08:42:27.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.496+0000 7f493c008640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49340cf320 con 0x7f49340cedd0 2024-04-14T08:42:27.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.496+0000 7f493c008640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f49340cff30 con 0x7f49340d2830 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.496+0000 7f49395a3640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f493415e890 0x7f49340d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:37536/0 (socket says 172.21.15.172:37536) 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.496+0000 7f49395a3640 1 -- 172.21.15.172:0/4027695663 learned_addr learned my addr 172.21.15.172:0/4027695663 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4939da4640 1 --2- 172.21.15.172:0/4027695663 >> v2:172.21.15.177:3300/0 conn(0x7f49340d2830 0x7f49340d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 879907365 0 0) 0x7f49340d3150 con 0x7f493415e890 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4920003520 con 0x7f493415e890 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1809281168 0 0) 0x7f49340cf320 con 0x7f49340cedd0 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49340d3150 con 0x7f49340cedd0 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4166502386 0 0) 0x7f4920003520 con 0x7f493415e890 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 >> v2:172.21.15.177:3300/0 conn(0x7f49340d2830 msgr2=0x7f49340d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 --2- 172.21.15.172:0/4027695663 >> v2:172.21.15.177:3300/0 conn(0x7f49340d2830 0x7f49340d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 >> v1:172.21.15.177:6789/0 conn(0x7f49340cedd0 legacy=0x7f49340b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49340cf600 con 0x7f493415e890 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f49240015f0 con 0x7f493415e890 2024-04-14T08:42:27.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 >> v1:172.21.15.172:6789/0 conn(0x7f493415e890 legacy=0x7f49340d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 --2- 172.21.15.172:0/4027695663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4920003b90 0x7f4920003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 --2- 172.21.15.172:0/4027695663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4920004800 0x7f4920004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 --2- 172.21.15.172:0/4027695663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49200051c0 0x7f492000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49340cff30 con 0x7f4920003b90 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f492000bfe0 con 0x7f4920004800 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f492000c1e0 con 0x7f49200051c0 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4939da4640 1 --2- 172.21.15.172:0/4027695663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4920003b90 0x7f4920003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f49395a3640 1 --2- 172.21.15.172:0/4027695663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4920004800 0x7f4920004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f493a5a5640 1 --2- 172.21.15.172:0/4027695663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49200051c0 0x7f492000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4939da4640 1 -- 172.21.15.172:0/4027695663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49200051c0 msgr2=0x7f492000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4939da4640 1 --2- 172.21.15.172:0/4027695663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49200051c0 0x7f492000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4939da4640 1 -- 172.21.15.172:0/4027695663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4920004800 msgr2=0x7f4920004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4939da4640 1 --2- 172.21.15.172:0/4027695663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4920004800 0x7f4920004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f4939da4640 1 -- 172.21.15.172:0/4027695663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4920003520 con 0x7f4920003b90 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f493a5a5640 1 --2- 172.21.15.172:0/4027695663 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49200051c0 0x7f492000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.497+0000 7f493a5a5640 1 -- 172.21.15.172:0/4027695663 reap_dead start 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.498+0000 7f4939da4640 1 --2- 172.21.15.172:0/4027695663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4920003b90 0x7f4920003f90 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f4930009fc0 tx=0x7f4920005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f74f927f36b4fe server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.498+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930039070 con 0x7f4920003b90 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.498+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f493002cc20 con 0x7f4920003b90 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.498+0000 7f4938da2640 1 -- 172.21.15.172:0/4027695663 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930009c10 con 0x7f4920003b90 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.499+0000 7f493c008640 1 -- 172.21.15.172:0/4027695663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4920003b90 msgr2=0x7f4920003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.499+0000 7f493c008640 1 --2- 172.21.15.172:0/4027695663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4920003b90 0x7f4920003f90 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f4930009fc0 tx=0x7f4920005830 comp rx=0 tx=0).stop 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.499+0000 7f493c008640 1 -- 172.21.15.172:0/4027695663 shutdown_connections 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.499+0000 7f493c008640 1 --2- 172.21.15.172:0/4027695663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4920003b90 0x7f4920003f90 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.499+0000 7f493c008640 1 -- 172.21.15.172:0/4027695663 >> 172.21.15.172:0/4027695663 conn(0x7f493409d590 msgr2=0x7f49340d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.499+0000 7f493c008640 1 -- 172.21.15.172:0/4027695663 shutdown_connections 2024-04-14T08:42:27.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.499+0000 7f493c008640 1 -- 172.21.15.172:0/4027695663 wait complete. 2024-04-14T08:42:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.500+0000 7f493c008640 1 Processor -- start 2024-04-14T08:42:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.500+0000 7f493c008640 1 -- start start 2024-04-14T08:42:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493c008640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4920003b90 0x7f49340c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493c008640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49341592c0 0x7f49340b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493c008640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4934160060 0x7f49340cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493c008640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49340cfbc0 con 0x7f4934160060 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493c008640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f49340d2350 con 0x7f4920003b90 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493c008640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f49340d2c10 con 0x7f49341592c0 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493a5a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4934160060 0x7f49340cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493a5a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4934160060 0x7f49340cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35306/0 (socket says 172.21.15.172:35306) 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493a5a5640 1 -- 172.21.15.172:0/3984474919 learned_addr learned my addr 172.21.15.172:0/3984474919 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f4939da4640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4920003b90 0x7f49340c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f49395a3640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49341592c0 0x7f49340b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493a5a5640 1 -- 172.21.15.172:0/3984474919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49341592c0 msgr2=0x7f49340b96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493a5a5640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49341592c0 0x7f49340b96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493a5a5640 1 -- 172.21.15.172:0/3984474919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4920003b90 msgr2=0x7f49340c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493a5a5640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4920003b90 0x7f49340c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493a5a5640 1 -- 172.21.15.172:0/3984474919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49340b8610 con 0x7f4934160060 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f4939da4640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4920003b90 0x7f49340c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493a5a5640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4934160060 0x7f49340cb320 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f493408d6e0 tx=0x7f492c00d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f492a7fc640 1 -- 172.21.15.172:0/3984474919 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c018070 con 0x7f4934160060 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f492a7fc640 1 -- 172.21.15.172:0/3984474919 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492c0035f0 con 0x7f4934160060 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f493c008640 1 -- 172.21.15.172:0/3984474919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49340b88f0 con 0x7f4934160060 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.501+0000 7f492a7fc640 1 -- 172.21.15.172:0/3984474919 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c0039e0 con 0x7f4934160060 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.502+0000 7f493c008640 1 -- 172.21.15.172:0/3984474919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49340b8eb0 con 0x7f4934160060 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.502+0000 7f493c008640 1 -- 172.21.15.172:0/3984474919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4934068520 con 0x7f4934160060 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.503+0000 7f492a7fc640 1 -- 172.21.15.172:0/3984474919 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f492c01b070 con 0x7f4934160060 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.503+0000 7f492a7fc640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f491806c8f0 0x7f491806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.503+0000 7f492a7fc640 1 -- 172.21.15.172:0/3984474919 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7f492c08b340 con 0x7f4934160060 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.503+0000 7f4939da4640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f491806c8f0 0x7f491806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.504+0000 7f4939da4640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f491806c8f0 0x7f491806edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4930036000 tx=0x7f493002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.508+0000 7f492a7fc640 1 -- 172.21.15.172:0/3984474919 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f492c0595b0 con 0x7f4934160060 2024-04-14T08:42:27.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.685+0000 7f493c008640 1 -- 172.21.15.172:0/3984474919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49340d4960 con 0x7f4934160060 2024-04-14T08:42:27.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.686+0000 7f492a7fc640 1 -- 172.21.15.172:0/3984474919 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+17250 (secure 0 0 0) 0x7f492c058eb0 con 0x7f4934160060 2024-04-14T08:42:27.686 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:27.686 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":105,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:42:25.145836+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:41:02.725765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-14T08:41:03.442157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/4028911489":"2024-04-15T08:42:25.145813+0000","172.21.15.172:6835/4028911489":"2024-04-15T08:42:25.145813+0000","172.21.15.177:6835/2602337717":"2024-04-15T08:41:48.748747+0000","172.21.15.177:6834/2602337717":"2024-04-15T08:41:48.748747+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:42:27.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.688+0000 7f493c008640 1 -- 172.21.15.172:0/3984474919 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f491806c8f0 msgr2=0x7f491806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.688+0000 7f493c008640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f491806c8f0 0x7f491806edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4930036000 tx=0x7f493002e000 comp rx=0 tx=0).stop 2024-04-14T08:42:27.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.688+0000 7f493c008640 1 -- 172.21.15.172:0/3984474919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4934160060 msgr2=0x7f49340cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.688+0000 7f493c008640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4934160060 0x7f49340cb320 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f493408d6e0 tx=0x7f492c00d880 comp rx=0 tx=0).stop 2024-04-14T08:42:27.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.688+0000 7f493c008640 1 -- 172.21.15.172:0/3984474919 shutdown_connections 2024-04-14T08:42:27.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.688+0000 7f493c008640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f491806c8f0 0x7f491806edb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.688+0000 7f493c008640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4934160060 0x7f49340cb320 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.688+0000 7f493c008640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49341592c0 0x7f49340b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.688+0000 7f493c008640 1 --2- 172.21.15.172:0/3984474919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4920003b90 0x7f49340c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.688+0000 7f493c008640 1 -- 172.21.15.172:0/3984474919 >> 172.21.15.172:0/3984474919 conn(0x7f493409d590 msgr2=0x7f49340b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:27.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.688+0000 7f493c008640 1 -- 172.21.15.172:0/3984474919 shutdown_connections 2024-04-14T08:42:27.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.688+0000 7f493c008640 1 -- 172.21.15.172:0/3984474919 wait complete. 2024-04-14T08:42:27.699 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-14T08:42:27.699 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:42:27.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.803+0000 7f2d1b688640 1 Processor -- start 2024-04-14T08:42:27.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.803+0000 7f2d1b688640 1 -- start start 2024-04-14T08:42:27.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.804+0000 7f2d1b688640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2d1406fdf0 0x7f2d140b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.804+0000 7f2d1b688640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d14070270 con 0x7f2d1415e890 2024-04-14T08:42:27.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.804+0000 7f2d1b688640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d140b4540 con 0x7f2d1406b520 2024-04-14T08:42:27.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.804+0000 7f2d1b688640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2d140b4310 con 0x7f2d1406fdf0 2024-04-14T08:42:27.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.804+0000 7f2d19c25640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2d1415e890 0x7f2d1415ed20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:37540/0 (socket says 172.21.15.172:37540) 2024-04-14T08:42:27.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.804+0000 7f2d19c25640 1 -- 172.21.15.172:0/3306090971 learned_addr learned my addr 172.21.15.172:0/3306090971 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.804+0000 7f2d18c23640 1 --2- 172.21.15.172:0/3306090971 >> v2:172.21.15.177:3300/0 conn(0x7f2d1406fdf0 0x7f2d140b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.804+0000 7f2d0bfff640 1 -- 172.21.15.172:0/3306090971 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325719873 0 0) 0x7f2d14070270 con 0x7f2d1415e890 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.805+0000 7f2d0bfff640 1 -- 172.21.15.172:0/3306090971 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d00003520 con 0x7f2d1415e890 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.805+0000 7f2d0bfff640 1 -- 172.21.15.172:0/3306090971 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 781614116 0 0) 0x7f2d140b4540 con 0x7f2d1406b520 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.805+0000 7f2d0bfff640 1 -- 172.21.15.172:0/3306090971 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d14070270 con 0x7f2d1406b520 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.805+0000 7f2d0bfff640 1 -- 172.21.15.172:0/3306090971 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2784612680 0 0) 0x7f2d00003520 con 0x7f2d1415e890 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.805+0000 7f2d18c23640 1 -- 172.21.15.172:0/3306090971 >> v1:172.21.15.177:6789/0 conn(0x7f2d1406b520 legacy=0x7f2d1406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.805+0000 7f2d18c23640 1 -- 172.21.15.172:0/3306090971 >> v1:172.21.15.172:6789/0 conn(0x7f2d1415e890 legacy=0x7f2d1415ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.805+0000 7f2d18c23640 1 -- 172.21.15.172:0/3306090971 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d14072450 con 0x7f2d1406fdf0 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.806+0000 7f2d18c23640 1 --2- 172.21.15.172:0/3306090971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1406fdf0 0x7f2d140b3dd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2d040097a0 tx=0x7f2d0402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=717e0de9cf90804d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.806+0000 7f2d0bfff640 1 -- 172.21.15.172:0/3306090971 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04038070 con 0x7f2d1406fdf0 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.806+0000 7f2d0bfff640 1 -- 172.21.15.172:0/3306090971 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d04031500 con 0x7f2d1406fdf0 2024-04-14T08:42:27.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.806+0000 7f2d0bfff640 1 -- 172.21.15.172:0/3306090971 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0019f0 con 0x7f2d1406fdf0 2024-04-14T08:42:27.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.806+0000 7f2d1b688640 1 -- 172.21.15.172:0/3306090971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1406fdf0 msgr2=0x7f2d140b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.806+0000 7f2d1b688640 1 --2- 172.21.15.172:0/3306090971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1406fdf0 0x7f2d140b3dd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2d040097a0 tx=0x7f2d0402b820 comp rx=0 tx=0).stop 2024-04-14T08:42:27.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.806+0000 7f2d1b688640 1 -- 172.21.15.172:0/3306090971 shutdown_connections 2024-04-14T08:42:27.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.806+0000 7f2d1b688640 1 --2- 172.21.15.172:0/3306090971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1406fdf0 0x7f2d140b3dd0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.806+0000 7f2d1b688640 1 -- 172.21.15.172:0/3306090971 >> 172.21.15.172:0/3306090971 conn(0x7f2d140d0f20 msgr2=0x7f2d140d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:27.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.806+0000 7f2d1b688640 1 -- 172.21.15.172:0/3306090971 shutdown_connections 2024-04-14T08:42:27.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.806+0000 7f2d1b688640 1 -- 172.21.15.172:0/3306090971 wait complete. 2024-04-14T08:42:27.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.808+0000 7f2d1b688640 1 Processor -- start 2024-04-14T08:42:27.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.808+0000 7f2d1b688640 1 -- start start 2024-04-14T08:42:27.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.808+0000 7f2d1b688640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1406b520 0x7f2d1415d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.808+0000 7f2d1b688640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1406fdf0 0x7f2d14145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.808+0000 7f2d1b688640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1415e890 0x7f2d1415a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.808+0000 7f2d1b688640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14072a10 con 0x7f2d1406b520 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.808+0000 7f2d1b688640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14072d80 con 0x7f2d1415e890 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.808+0000 7f2d1b688640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2d14073210 con 0x7f2d1406fdf0 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.808+0000 7f2d19424640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1406b520 0x7f2d1415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.808+0000 7f2d19424640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1406b520 0x7f2d1415d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35322/0 (socket says 172.21.15.172:35322) 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.808+0000 7f2d19424640 1 -- 172.21.15.172:0/3897496684 learned_addr learned my addr 172.21.15.172:0/3897496684 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.808+0000 7f2d18c23640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1406fdf0 0x7f2d14145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.809+0000 7f2d19c25640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1415e890 0x7f2d1415a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.809+0000 7f2d19424640 1 -- 172.21.15.172:0/3897496684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1406fdf0 msgr2=0x7f2d14145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.809+0000 7f2d19424640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1406fdf0 0x7f2d14145980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.809+0000 7f2d19424640 1 -- 172.21.15.172:0/3897496684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1415e890 msgr2=0x7f2d1415a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.809+0000 7f2d19424640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1415e890 0x7f2d1415a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.809+0000 7f2d19424640 1 -- 172.21.15.172:0/3897496684 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1415c660 con 0x7f2d1406b520 2024-04-14T08:42:27.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.809+0000 7f2d19424640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1406b520 0x7f2d1415d700 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000c430 tx=0x7f2d1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:27.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.809+0000 7f2d09ffb640 1 -- 172.21.15.172:0/3897496684 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10017070 con 0x7f2d1406b520 2024-04-14T08:42:27.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.809+0000 7f2d09ffb640 1 -- 172.21.15.172:0/3897496684 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d100052f0 con 0x7f2d1406b520 2024-04-14T08:42:27.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.809+0000 7f2d1b688640 1 -- 172.21.15.172:0/3897496684 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1415c940 con 0x7f2d1406b520 2024-04-14T08:42:27.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.809+0000 7f2d09ffb640 1 -- 172.21.15.172:0/3897496684 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10003830 con 0x7f2d1406b520 2024-04-14T08:42:27.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.809+0000 7f2d1b688640 1 -- 172.21.15.172:0/3897496684 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1415cf00 con 0x7f2d1406b520 2024-04-14T08:42:27.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.810+0000 7f2d1b688640 1 -- 172.21.15.172:0/3897496684 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d14068520 con 0x7f2d1406b520 2024-04-14T08:42:27.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.811+0000 7f2d09ffb640 1 -- 172.21.15.172:0/3897496684 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d1001a070 con 0x7f2d1406b520 2024-04-14T08:42:27.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.811+0000 7f2d09ffb640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2cf806c8f0 0x7f2cf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:27.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.811+0000 7f2d09ffb640 1 -- 172.21.15.172:0/3897496684 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7f2d1008b050 con 0x7f2d1406b520 2024-04-14T08:42:27.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.811+0000 7f2d18c23640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2cf806c8f0 0x7f2cf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:27.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.812+0000 7f2d18c23640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2cf806c8f0 0x7f2cf806edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2d04035000 tx=0x7f2d04008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:27.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:27.816+0000 7f2d09ffb640 1 -- 172.21.15.172:0/3897496684 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2d10058c70 con 0x7f2d1406b520 2024-04-14T08:42:28.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.030+0000 7f2d1b688640 1 -- 172.21.15.172:0/3897496684 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d1406c3b0 con 0x7f2d1406b520 2024-04-14T08:42:28.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.030+0000 7f2d09ffb640 1 -- 172.21.15.172:0/3897496684 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5182 (secure 0 0 0) 0x7f2d1005cf70 con 0x7f2d1406b520 2024-04-14T08:42:28.031 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:28.031 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8005,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3289017090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3289017090},{"type":"v1","addr":"172.21.15.177:6835","nonce":3289017090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8880,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1467868914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1467868914},{"type":"v1","addr":"172.21.15.172:6837","nonce":1467868914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":81,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:41:04.492773+0000","modified":"2024-04-14T08:42:25.146002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-14T08:42:28.031 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 82 2024-04-14T08:42:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.033+0000 7f2d1b688640 1 -- 172.21.15.172:0/3897496684 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2cf806c8f0 msgr2=0x7f2cf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.033+0000 7f2d1b688640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2cf806c8f0 0x7f2cf806edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2d04035000 tx=0x7f2d04008030 comp rx=0 tx=0).stop 2024-04-14T08:42:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.033+0000 7f2d1b688640 1 -- 172.21.15.172:0/3897496684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1406b520 msgr2=0x7f2d1415d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.033+0000 7f2d1b688640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1406b520 0x7f2d1415d700 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000c430 tx=0x7f2d1000c8f0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.033+0000 7f2d1b688640 1 -- 172.21.15.172:0/3897496684 shutdown_connections 2024-04-14T08:42:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.033+0000 7f2d1b688640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2cf806c8f0 0x7f2cf806edb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.033+0000 7f2d1b688640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1415e890 0x7f2d1415a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.033+0000 7f2d1b688640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1406fdf0 0x7f2d14145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.033+0000 7f2d1b688640 1 --2- 172.21.15.172:0/3897496684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d1406b520 0x7f2d1415d700 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.033+0000 7f2d1b688640 1 -- 172.21.15.172:0/3897496684 >> 172.21.15.172:0/3897496684 conn(0x7f2d140d0f20 msgr2=0x7f2d140b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.033+0000 7f2d1b688640 1 -- 172.21.15.172:0/3897496684 shutdown_connections 2024-04-14T08:42:28.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.033+0000 7f2d1b688640 1 -- 172.21.15.172:0/3897496684 wait complete. 2024-04-14T08:42:28.044 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:42:28.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.146+0000 7f5d51e69640 1 Processor -- start 2024-04-14T08:42:28.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.146+0000 7f5d51e69640 1 -- start start 2024-04-14T08:42:28.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.146+0000 7f5d51e69640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0cedd0 0x7f5d4c1694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.146+0000 7f5d51e69640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4c0cf320 con 0x7f5d4c0d2830 2024-04-14T08:42:28.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.146+0000 7f5d51e69640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4c0cf600 con 0x7f5d4c15e890 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.146+0000 7f5d51e69640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4c0cff30 con 0x7f5d4c0cedd0 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4b7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5d4c0d2830 0x7f5d4c0d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:37552/0 (socket says 172.21.15.172:37552) 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4b7fe640 1 -- 172.21.15.172:0/1032308350 learned_addr learned my addr 172.21.15.172:0/1032308350 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3581656193 0 0) 0x7f5d4c0cf320 con 0x7f5d4c0d2830 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4bfff640 1 --2- 172.21.15.172:0/1032308350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0cedd0 0x7f5d4c1694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d34003520 con 0x7f5d4c0d2830 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1174106188 0 0) 0x7f5d34003520 con 0x7f5d4c0d2830 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0cedd0 msgr2=0x7f5d4c1694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 --2- 172.21.15.172:0/1032308350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0cedd0 0x7f5d4c1694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 >> v1:172.21.15.177:6789/0 conn(0x7f5d4c15e890 legacy=0x7f5d4c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c0cf8e0 con 0x7f5d4c0d2830 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f5d3c001670 con 0x7f5d4c0d2830 2024-04-14T08:42:28.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 >> v1:172.21.15.172:6789/0 conn(0x7f5d4c0d2830 legacy=0x7f5d4c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 --2- 172.21.15.172:0/1032308350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d340039e0 0x7f5d34003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 --2- 172.21.15.172:0/1032308350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d34004800 0x7f5d34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 --2- 172.21.15.172:0/1032308350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0cedd0 0x7f5d340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c0cff30 con 0x7f5d34004800 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3400bb00 con 0x7f5d340039e0 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5d3400bd00 con 0x7f5d4c0cedd0 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4affd640 1 --2- 172.21.15.172:0/1032308350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d34004800 0x7f5d34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4b7fe640 1 --2- 172.21.15.172:0/1032308350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d340039e0 0x7f5d34003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.147+0000 7f5d4bfff640 1 --2- 172.21.15.172:0/1032308350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0cedd0 0x7f5d340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.148+0000 7f5d4affd640 1 -- 172.21.15.172:0/1032308350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0cedd0 msgr2=0x7f5d340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.148+0000 7f5d4affd640 1 --2- 172.21.15.172:0/1032308350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0cedd0 0x7f5d340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.148+0000 7f5d4affd640 1 -- 172.21.15.172:0/1032308350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d340039e0 msgr2=0x7f5d34003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.148+0000 7f5d4affd640 1 --2- 172.21.15.172:0/1032308350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d340039e0 0x7f5d34003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.148+0000 7f5d4affd640 1 -- 172.21.15.172:0/1032308350 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d34003520 con 0x7f5d34004800 2024-04-14T08:42:28.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.148+0000 7f5d4affd640 1 --2- 172.21.15.172:0/1032308350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d34004800 0x7f5d34004c80 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f5d40001720 tx=0x7f5d34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32ccf2d98fbe6d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:28.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.148+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40001370 con 0x7f5d34004800 2024-04-14T08:42:28.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.148+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d400052d0 con 0x7f5d34004800 2024-04-14T08:42:28.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.148+0000 7f5d4a7fc640 1 -- 172.21.15.172:0/1032308350 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40003830 con 0x7f5d34004800 2024-04-14T08:42:28.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.149+0000 7f5d51e69640 1 -- 172.21.15.172:0/1032308350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d34004800 msgr2=0x7f5d34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.149+0000 7f5d51e69640 1 --2- 172.21.15.172:0/1032308350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d34004800 0x7f5d34004c80 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f5d40001720 tx=0x7f5d34005830 comp rx=0 tx=0).stop 2024-04-14T08:42:28.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.149+0000 7f5d4bfff640 1 -- 172.21.15.172:0/1032308350 reap_dead start 2024-04-14T08:42:28.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.149+0000 7f5d51e69640 1 -- 172.21.15.172:0/1032308350 shutdown_connections 2024-04-14T08:42:28.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.149+0000 7f5d51e69640 1 -- 172.21.15.172:0/1032308350 >> 172.21.15.172:0/1032308350 conn(0x7f5d4c09d590 msgr2=0x7f5d4c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:28.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.149+0000 7f5d51e69640 1 -- 172.21.15.172:0/1032308350 shutdown_connections 2024-04-14T08:42:28.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.149+0000 7f5d51e69640 1 -- 172.21.15.172:0/1032308350 wait complete. 2024-04-14T08:42:28.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.150+0000 7f5d51e69640 1 Processor -- start 2024-04-14T08:42:28.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.150+0000 7f5d51e69640 1 -- start start 2024-04-14T08:42:28.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d51e69640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d34004800 0x7f5d4c156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d51e69640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0b1ed0 0x7f5d4c0b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d51e69640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d4c0c8b40 0x7f5d4c0c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d51e69640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c0b4720 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d51e69640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c0d2350 con 0x7f5d34004800 2024-04-14T08:42:28.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d51e69640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4c0b48a0 con 0x7f5d4c0b1ed0 2024-04-14T08:42:28.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d4bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d4c0c8b40 0x7f5d4c0c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d4bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d4c0c8b40 0x7f5d4c0c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35330/0 (socket says 172.21.15.172:35330) 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d4bfff640 1 -- 172.21.15.172:0/3496847522 learned_addr learned my addr 172.21.15.172:0/3496847522 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d4affd640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0b1ed0 0x7f5d4c0b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d4b7fe640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d34004800 0x7f5d4c156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d4bfff640 1 -- 172.21.15.172:0/3496847522 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0b1ed0 msgr2=0x7f5d4c0b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d4bfff640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0b1ed0 0x7f5d4c0b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d4bfff640 1 -- 172.21.15.172:0/3496847522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d34004800 msgr2=0x7f5d4c156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d4bfff640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d34004800 0x7f5d4c156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d4bfff640 1 -- 172.21.15.172:0/3496847522 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c160be0 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.151+0000 7f5d4bfff640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d4c0c8b40 0x7f5d4c0c8fc0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0d0530 tx=0x7f5d440115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.152+0000 7f5d33fff640 1 -- 172.21.15.172:0/3496847522 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44003990 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.152+0000 7f5d33fff640 1 -- 172.21.15.172:0/3496847522 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4401b070 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.152+0000 7f5d33fff640 1 -- 172.21.15.172:0/3496847522 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44003b70 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.152+0000 7f5d51e69640 1 -- 172.21.15.172:0/3496847522 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c160e80 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.152+0000 7f5d51e69640 1 -- 172.21.15.172:0/3496847522 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c161440 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.153+0000 7f5d51e69640 1 -- 172.21.15.172:0/3496847522 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4c068520 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.153+0000 7f5d33fff640 1 -- 172.21.15.172:0/3496847522 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d44005d10 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.153+0000 7f5d33fff640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5d3806c920 0x7f5d3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.153+0000 7f5d33fff640 1 -- 172.21.15.172:0/3496847522 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7f5d4408c0a0 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.153+0000 7f5d4b7fe640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5d3806c920 0x7f5d3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.154+0000 7f5d4b7fe640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5d3806c920 0x7f5d3806ede0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c007fd0 tx=0x7f5d3c0078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:28.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.158+0000 7f5d33fff640 1 -- 172.21.15.172:0/3496847522 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5d44059c30 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.369+0000 7f5d51e69640 1 -- 172.21.15.172:0/3496847522 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d4c0d2610 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.369+0000 7f5d33fff640 1 -- 172.21.15.172:0/3496847522 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5182 (secure 0 0 0) 0x7f5d4405df30 con 0x7f5d4c0c8b40 2024-04-14T08:42:28.370 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:28.370 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8005,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3289017090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3289017090},{"type":"v1","addr":"172.21.15.177:6835","nonce":3289017090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8880,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1467868914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1467868914},{"type":"v1","addr":"172.21.15.172:6837","nonce":1467868914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":81,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:41:04.492773+0000","modified":"2024-04-14T08:42:25.146002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-14T08:42:28.370 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 82 2024-04-14T08:42:28.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.371+0000 7f5d51e69640 1 -- 172.21.15.172:0/3496847522 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5d3806c920 msgr2=0x7f5d3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.371+0000 7f5d51e69640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5d3806c920 0x7f5d3806ede0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c007fd0 tx=0x7f5d3c0078f0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.371+0000 7f5d51e69640 1 -- 172.21.15.172:0/3496847522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d4c0c8b40 msgr2=0x7f5d4c0c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.371+0000 7f5d51e69640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d4c0c8b40 0x7f5d4c0c8fc0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0d0530 tx=0x7f5d440115a0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.371+0000 7f5d51e69640 1 -- 172.21.15.172:0/3496847522 shutdown_connections 2024-04-14T08:42:28.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.371+0000 7f5d51e69640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5d3806c920 0x7f5d3806ede0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.371+0000 7f5d51e69640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d4c0c8b40 0x7f5d4c0c8fc0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.371+0000 7f5d51e69640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d4c0b1ed0 0x7f5d4c0b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.371+0000 7f5d51e69640 1 --2- 172.21.15.172:0/3496847522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d34004800 0x7f5d4c156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.371+0000 7f5d51e69640 1 -- 172.21.15.172:0/3496847522 >> 172.21.15.172:0/3496847522 conn(0x7f5d4c09d590 msgr2=0x7f5d4c0b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:28.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.372+0000 7f5d51e69640 1 -- 172.21.15.172:0/3496847522 shutdown_connections 2024-04-14T08:42:28.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.372+0000 7f5d51e69640 1 -- 172.21.15.172:0/3496847522 wait complete. 2024-04-14T08:42:28.383 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:42:28.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.484+0000 7fda725cf640 1 Processor -- start 2024-04-14T08:42:28.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.484+0000 7fda725cf640 1 -- start start 2024-04-14T08:42:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda725cf640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fda6c160170 0x7fda6c15d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda725cf640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15d4d0 0x7fda6c0b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda725cf640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda6c13f910 con 0x7fda6c15d020 2024-04-14T08:42:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda725cf640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fda6c13fbf0 con 0x7fda6c160170 2024-04-14T08:42:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda725cf640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fda6c13fed0 con 0x7fda6c15d4d0 2024-04-14T08:42:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6b7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fda6c15d020 0x7fda6c160550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:37562/0 (socket says 172.21.15.172:37562) 2024-04-14T08:42:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6b7fe640 1 -- 172.21.15.172:0/3804707814 learned_addr learned my addr 172.21.15.172:0/3804707814 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda70b6c640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15d4d0 0x7fda6c0b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6bfff640 1 --2- 172.21.15.172:0/3804707814 >> v2:172.21.15.177:3300/0 conn(0x7fda6c160170 0x7fda6c15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 637822689 0 0) 0x7fda6c13f910 con 0x7fda6c15d020 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda54003520 con 0x7fda6c15d020 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3607829932 0 0) 0x7fda54003520 con 0x7fda6c15d020 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15d4d0 msgr2=0x7fda6c0b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15d4d0 0x7fda6c0b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 >> v2:172.21.15.177:3300/0 conn(0x7fda6c160170 msgr2=0x7fda6c15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 --2- 172.21.15.172:0/3804707814 >> v2:172.21.15.177:3300/0 conn(0x7fda6c160170 0x7fda6c15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c1401b0 con 0x7fda6c15d020 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fda5c001670 con 0x7fda6c15d020 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda70b6c640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15d4d0 0x7fda6c0b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 >> v1:172.21.15.172:6789/0 conn(0x7fda6c15d020 legacy=0x7fda6c160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda540039b0 0x7fda54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15d4d0 0x7fda54004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda54004d40 0x7fda540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c13fbf0 con 0x7fda540039b0 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c13fed0 con 0x7fda54004d40 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fda5400b9c0 con 0x7fda6c15d4d0 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.485+0000 7fda6bfff640 1 --2- 172.21.15.172:0/3804707814 >> v2:172.21.15.177:3300/0 conn(0x7fda6c160170 0x7fda6c15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.486+0000 7fda70b6c640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda54004d40 0x7fda540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.486+0000 7fda6b7fe640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15d4d0 0x7fda54004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.486+0000 7fda6bfff640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda540039b0 0x7fda54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.486+0000 7fda6bfff640 1 -- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15d4d0 msgr2=0x7fda54004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.486+0000 7fda6bfff640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15d4d0 0x7fda54004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.486+0000 7fda6bfff640 1 -- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda54004d40 msgr2=0x7fda540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.486+0000 7fda6bfff640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda54004d40 0x7fda540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.486+0000 7fda6bfff640 1 -- 172.21.15.172:0/3804707814 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda54003520 con 0x7fda540039b0 2024-04-14T08:42:28.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.486+0000 7fda70b6c640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda54004d40 0x7fda540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.487+0000 7fda6bfff640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda540039b0 0x7fda54003dd0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fda60009c70 tx=0x7fda54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ebacbafd84f8a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.487+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60009840 con 0x7fda540039b0 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.487+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda60004020 con 0x7fda540039b0 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.487+0000 7fda6affd640 1 -- 172.21.15.172:0/3804707814 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60004410 con 0x7fda540039b0 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.487+0000 7fda725cf640 1 -- 172.21.15.172:0/3804707814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda540039b0 msgr2=0x7fda54003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.487+0000 7fda725cf640 1 --2- 172.21.15.172:0/3804707814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda540039b0 0x7fda54003dd0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fda60009c70 tx=0x7fda54005830 comp rx=0 tx=0).stop 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.487+0000 7fda70b6c640 1 -- 172.21.15.172:0/3804707814 reap_dead start 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.487+0000 7fda725cf640 1 -- 172.21.15.172:0/3804707814 shutdown_connections 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.487+0000 7fda725cf640 1 -- 172.21.15.172:0/3804707814 >> 172.21.15.172:0/3804707814 conn(0x7fda6c06ec30 msgr2=0x7fda6c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.487+0000 7fda725cf640 1 -- 172.21.15.172:0/3804707814 shutdown_connections 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.487+0000 7fda725cf640 1 -- 172.21.15.172:0/3804707814 wait complete. 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.489+0000 7fda725cf640 1 Processor -- start 2024-04-14T08:42:28.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.489+0000 7fda725cf640 1 -- start start 2024-04-14T08:42:28.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.489+0000 7fda725cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda6c13da80 0x7fda6c159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.489+0000 7fda725cf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda6c13de60 0x7fda6c0a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.489+0000 7fda725cf640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15b100 0x7fda6c0a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.489+0000 7fda725cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c0b1b80 con 0x7fda6c13da80 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.489+0000 7fda725cf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c160c60 con 0x7fda6c13de60 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.489+0000 7fda725cf640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fda6c0b1d00 con 0x7fda6c15b100 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda6bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda6c13da80 0x7fda6c159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda6bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda6c13da80 0x7fda6c159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35354/0 (socket says 172.21.15.172:35354) 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda6bfff640 1 -- 172.21.15.172:0/213411204 learned_addr learned my addr 172.21.15.172:0/213411204 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda70b6c640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15b100 0x7fda6c0a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda6bfff640 1 -- 172.21.15.172:0/213411204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15b100 msgr2=0x7fda6c0a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda6b7fe640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda6c13de60 0x7fda6c0a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda6bfff640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15b100 0x7fda6c0a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda6bfff640 1 -- 172.21.15.172:0/213411204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda6c13de60 msgr2=0x7fda6c0a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda6bfff640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda6c13de60 0x7fda6c0a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda6bfff640 1 -- 172.21.15.172:0/213411204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c159dc0 con 0x7fda6c13da80 2024-04-14T08:42:28.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda6bfff640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda6c13da80 0x7fda6c159880 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fda6002c510 tx=0x7fda60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:28.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda68ff9640 1 -- 172.21.15.172:0/213411204 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60043020 con 0x7fda6c13da80 2024-04-14T08:42:28.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda68ff9640 1 -- 172.21.15.172:0/213411204 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6002c960 con 0x7fda6c13da80 2024-04-14T08:42:28.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda725cf640 1 -- 172.21.15.172:0/213411204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6c15a060 con 0x7fda6c13da80 2024-04-14T08:42:28.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda68ff9640 1 -- 172.21.15.172:0/213411204 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60006e10 con 0x7fda6c13da80 2024-04-14T08:42:28.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.490+0000 7fda725cf640 1 -- 172.21.15.172:0/213411204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6c15a620 con 0x7fda6c13da80 2024-04-14T08:42:28.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.491+0000 7fda725cf640 1 -- 172.21.15.172:0/213411204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6c068520 con 0x7fda6c13da80 2024-04-14T08:42:28.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.492+0000 7fda68ff9640 1 -- 172.21.15.172:0/213411204 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda60046020 con 0x7fda6c13da80 2024-04-14T08:42:28.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.492+0000 7fda68ff9640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda5806c8f0 0x7fda5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.492+0000 7fda68ff9640 1 -- 172.21.15.172:0/213411204 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7fda600add80 con 0x7fda6c13da80 2024-04-14T08:42:28.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.492+0000 7fda6b7fe640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda5806c8f0 0x7fda5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.492+0000 7fda6b7fe640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda5806c8f0 0x7fda5806edb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fda6c159110 tx=0x7fda5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:28.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.496+0000 7fda68ff9640 1 -- 172.21.15.172:0/213411204 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fda6004d030 con 0x7fda6c13da80 2024-04-14T08:42:28.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.673+0000 7fda725cf640 1 -- 172.21.15.172:0/213411204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda6c06c2c0 con 0x7fda6c13da80 2024-04-14T08:42:28.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.674+0000 7fda68ff9640 1 -- 172.21.15.172:0/213411204 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+17250 (secure 0 0 0) 0x7fda6007bff0 con 0x7fda6c13da80 2024-04-14T08:42:28.675 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:28.675 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":105,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:42:25.145836+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:41:02.725765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-14T08:41:03.442157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/4028911489":"2024-04-15T08:42:25.145813+0000","172.21.15.172:6835/4028911489":"2024-04-15T08:42:25.145813+0000","172.21.15.177:6835/2602337717":"2024-04-15T08:41:48.748747+0000","172.21.15.177:6834/2602337717":"2024-04-15T08:41:48.748747+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:42:28.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.676+0000 7fda725cf640 1 -- 172.21.15.172:0/213411204 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda5806c8f0 msgr2=0x7fda5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.676+0000 7fda725cf640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda5806c8f0 0x7fda5806edb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fda6c159110 tx=0x7fda5c009040 comp rx=0 tx=0).stop 2024-04-14T08:42:28.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.676+0000 7fda725cf640 1 -- 172.21.15.172:0/213411204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda6c13da80 msgr2=0x7fda6c159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.676+0000 7fda725cf640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda6c13da80 0x7fda6c159880 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fda6002c510 tx=0x7fda60004730 comp rx=0 tx=0).stop 2024-04-14T08:42:28.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.676+0000 7fda725cf640 1 -- 172.21.15.172:0/213411204 shutdown_connections 2024-04-14T08:42:28.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.676+0000 7fda725cf640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fda5806c8f0 0x7fda5806edb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.677+0000 7fda725cf640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fda6c15b100 0x7fda6c0a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.677+0000 7fda725cf640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda6c13de60 0x7fda6c0a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.677+0000 7fda725cf640 1 --2- 172.21.15.172:0/213411204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda6c13da80 0x7fda6c159880 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.677+0000 7fda725cf640 1 -- 172.21.15.172:0/213411204 >> 172.21.15.172:0/213411204 conn(0x7fda6c06ec30 msgr2=0x7fda6c0ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:28.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.677+0000 7fda725cf640 1 -- 172.21.15.172:0/213411204 shutdown_connections 2024-04-14T08:42:28.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.677+0000 7fda725cf640 1 -- 172.21.15.172:0/213411204 wait complete. 2024-04-14T08:42:28.688 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:42:28.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.790+0000 7f0a8fe08640 1 Processor -- start 2024-04-14T08:42:28.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.790+0000 7f0a8fe08640 1 -- start start 2024-04-14T08:42:28.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.791+0000 7f0a8fe08640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0a8815c380 0x7f0a88140e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.791+0000 7f0a8fe08640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a8815b620 con 0x7f0a8813df30 2024-04-14T08:42:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.791+0000 7f0a8fe08640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a881414a0 con 0x7f0a880b23e0 2024-04-14T08:42:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.791+0000 7f0a8fe08640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0a880cf9b0 con 0x7f0a8815c380 2024-04-14T08:42:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.791+0000 7f0a8e3a5640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0a8815c380 0x7f0a88140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.791+0000 7f0a8dba4640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f0a8813df30 0x7f0a880b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:37574/0 (socket says 172.21.15.172:37574) 2024-04-14T08:42:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.791+0000 7f0a8dba4640 1 -- 172.21.15.172:0/1712721462 learned_addr learned my addr 172.21.15.172:0/1712721462 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.791+0000 7f0a8e3a5640 1 -- 172.21.15.172:0/1712721462 >> v1:172.21.15.177:6789/0 conn(0x7f0a880b23e0 legacy=0x7f0a88140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.791+0000 7f0a8e3a5640 1 -- 172.21.15.172:0/1712721462 >> v1:172.21.15.172:6789/0 conn(0x7f0a8813df30 legacy=0x7f0a880b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.791+0000 7f0a8cba2640 1 -- 172.21.15.172:0/1712721462 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2622412286 0 0) 0x7f0a84001110 con 0x7f0a8813df30 2024-04-14T08:42:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.791+0000 7f0a8e3a5640 1 -- 172.21.15.172:0/1712721462 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8815b8a0 con 0x7f0a8815c380 2024-04-14T08:42:28.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.792+0000 7f0a8e3a5640 1 --2- 172.21.15.172:0/1712721462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815c380 0x7f0a88140e80 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009d90 tx=0x7f0a8002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625515007658bf1b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:28.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.792+0000 7f0a8cba2640 1 -- 172.21.15.172:0/1712721462 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80039070 con 0x7f0a8815c380 2024-04-14T08:42:28.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.792+0000 7f0a8cba2640 1 -- 172.21.15.172:0/1712721462 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8002cd00 con 0x7f0a8815c380 2024-04-14T08:42:28.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.792+0000 7f0a8cba2640 1 -- 172.21.15.172:0/1712721462 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80034660 con 0x7f0a8815c380 2024-04-14T08:42:28.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.792+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1712721462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815c380 msgr2=0x7f0a88140e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.792+0000 7f0a8fe08640 1 --2- 172.21.15.172:0/1712721462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815c380 0x7f0a88140e80 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009d90 tx=0x7f0a8002bf50 comp rx=0 tx=0).stop 2024-04-14T08:42:28.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.792+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1712721462 shutdown_connections 2024-04-14T08:42:28.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.792+0000 7f0a8fe08640 1 --2- 172.21.15.172:0/1712721462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815c380 0x7f0a88140e80 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.792+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1712721462 >> 172.21.15.172:0/1712721462 conn(0x7f0a8806b520 msgr2=0x7f0a8806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:28.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.793+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1712721462 shutdown_connections 2024-04-14T08:42:28.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.793+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1712721462 wait complete. 2024-04-14T08:42:28.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.794+0000 7f0a8fe08640 1 Processor -- start 2024-04-14T08:42:28.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.794+0000 7f0a8fe08640 1 -- start start 2024-04-14T08:42:28.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.794+0000 7f0a8fe08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a880b23e0 0x7f0a880b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.794+0000 7f0a8fe08640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a8813df30 0x7f0a88137780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.794+0000 7f0a8fe08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815c380 0x7f0a880712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.794+0000 7f0a8fe08640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8813e310 con 0x7f0a8815c380 2024-04-14T08:42:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.794+0000 7f0a8fe08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8815be60 con 0x7f0a880b23e0 2024-04-14T08:42:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.794+0000 7f0a8fe08640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0a88165fa0 con 0x7f0a8813df30 2024-04-14T08:42:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.794+0000 7f0a8e3a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815c380 0x7f0a880712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8e3a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815c380 0x7f0a880712d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35380/0 (socket says 172.21.15.172:35380) 2024-04-14T08:42:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8e3a5640 1 -- 172.21.15.172:0/1698183954 learned_addr learned my addr 172.21.15.172:0/1698183954 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8dba4640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a880b23e0 0x7f0a880b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8d3a3640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a8813df30 0x7f0a88137780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8e3a5640 1 -- 172.21.15.172:0/1698183954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a8813df30 msgr2=0x7f0a88137780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8e3a5640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a8813df30 0x7f0a88137780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8e3a5640 1 -- 172.21.15.172:0/1698183954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a880b23e0 msgr2=0x7f0a880b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8e3a5640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a880b23e0 0x7f0a880b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8e3a5640 1 -- 172.21.15.172:0/1698183954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a881593e0 con 0x7f0a8815c380 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8d3a3640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a8813df30 0x7f0a88137780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8dba4640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a880b23e0 0x7f0a880b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8e3a5640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815c380 0x7f0a880712d0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009eb0 tx=0x7f0a80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a7e7fc640 1 -- 172.21.15.172:0/1698183954 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80039050 con 0x7f0a8815c380 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a7e7fc640 1 -- 172.21.15.172:0/1698183954 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a80045070 con 0x7f0a8815c380 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a7e7fc640 1 -- 172.21.15.172:0/1698183954 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8003e830 con 0x7f0a8815c380 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1698183954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a880b5730 con 0x7f0a8815c380 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.795+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1698183954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a88150dd0 con 0x7f0a8815c380 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.796+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1698183954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a88068590 con 0x7f0a8815c380 2024-04-14T08:42:28.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.796+0000 7f0a7e7fc640 1 -- 172.21.15.172:0/1698183954 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a8003e9d0 con 0x7f0a8815c380 2024-04-14T08:42:28.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.797+0000 7f0a7e7fc640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a6c06c8f0 0x7f0a6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:28.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.797+0000 7f0a8dba4640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a6c06c8f0 0x7f0a6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:28.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.797+0000 7f0a7e7fc640 1 -- 172.21.15.172:0/1698183954 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7f0a800aead0 con 0x7f0a8815c380 2024-04-14T08:42:28.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.797+0000 7f0a8dba4640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a6c06c8f0 0x7f0a6c06edb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0a84000f60 tx=0x7f0a84007a10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:28.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:28.801+0000 7f0a7e7fc640 1 -- 172.21.15.172:0/1698183954 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0a8007ce60 con 0x7f0a8815c380 2024-04-14T08:42:29.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.014+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1698183954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0a88151010 con 0x7f0a8815c380 2024-04-14T08:42:29.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.830+0000 7f0a7e7fc640 1 -- 172.21.15.172:0/1698183954 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) ==== 161+0+0 (secure 0 0 0) 0x7f0a8007c760 con 0x7f0a8815c380 2024-04-14T08:42:29.831 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:42:29.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.833+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1698183954 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a6c06c8f0 msgr2=0x7f0a6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:29.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.833+0000 7f0a8fe08640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a6c06c8f0 0x7f0a6c06edb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0a84000f60 tx=0x7f0a84007a10 comp rx=0 tx=0).stop 2024-04-14T08:42:29.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.833+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1698183954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815c380 msgr2=0x7f0a880712d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:29.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.833+0000 7f0a8fe08640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815c380 0x7f0a880712d0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009eb0 tx=0x7f0a80004a40 comp rx=0 tx=0).stop 2024-04-14T08:42:29.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.833+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1698183954 shutdown_connections 2024-04-14T08:42:29.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.833+0000 7f0a8fe08640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0a6c06c8f0 0x7f0a6c06edb0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:29.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.833+0000 7f0a8fe08640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815c380 0x7f0a880712d0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:29.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.833+0000 7f0a8fe08640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a8813df30 0x7f0a88137780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:29.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.833+0000 7f0a8fe08640 1 --2- 172.21.15.172:0/1698183954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a880b23e0 0x7f0a880b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:29.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.833+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1698183954 >> 172.21.15.172:0/1698183954 conn(0x7f0a8806b520 msgr2=0x7f0a880723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:29.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.833+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1698183954 shutdown_connections 2024-04-14T08:42:29.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.833+0000 7f0a8fe08640 1 -- 172.21.15.172:0/1698183954 wait complete. 2024-04-14T08:42:29.845 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-14T08:42:29.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.948+0000 7f5dff588640 1 Processor -- start 2024-04-14T08:42:29.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.948+0000 7f5dff588640 1 -- start start 2024-04-14T08:42:29.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dff588640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5df80b23e0 0x7f5df8140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:29.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dff588640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5df815c380 0x7f5df8140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:29.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dff588640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df815b620 con 0x7f5df813df30 2024-04-14T08:42:29.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dff588640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f5df80cf9b0 con 0x7f5df815c380 2024-04-14T08:42:29.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dff588640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5df81411f0 con 0x7f5df80b23e0 2024-04-14T08:42:29.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dfdb25640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5df815c380 0x7f5df8140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:29.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dfdb25640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5df815c380 0x7f5df8140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57054/0 (socket says 172.21.15.172:57054) 2024-04-14T08:42:29.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dfd324640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5df813df30 0x7f5df80b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56106/0 (socket says 172.21.15.172:56106) 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dfdb25640 1 -- 172.21.15.172:0/2198355750 learned_addr learned my addr 172.21.15.172:0/2198355750 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dfdb25640 1 -- 172.21.15.172:0/2198355750 >> v2:172.21.15.177:3300/0 conn(0x7f5df80b23e0 msgr2=0x7f5df8140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dfcb23640 1 --2- 172.21.15.172:0/2198355750 >> v2:172.21.15.177:3300/0 conn(0x7f5df80b23e0 0x7f5df8140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dfdb25640 1 --2- 172.21.15.172:0/2198355750 >> v2:172.21.15.177:3300/0 conn(0x7f5df80b23e0 0x7f5df8140770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dfdb25640 1 -- 172.21.15.172:0/2198355750 >> v1:172.21.15.172:6789/0 conn(0x7f5df813df30 legacy=0x7f5df80b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5deffff640 1 -- 172.21.15.172:0/2198355750 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2600964613 0 0) 0x7f5df4001110 con 0x7f5df813df30 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dfdb25640 1 -- 172.21.15.172:0/2198355750 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df81414a0 con 0x7f5df815c380 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.949+0000 7f5dfcb23640 1 --2- 172.21.15.172:0/2198355750 >> v2:172.21.15.177:3300/0 conn(0x7f5df80b23e0 0x7f5df8140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.950+0000 7f5dfdb25640 1 --2- 172.21.15.172:0/2198355750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df815c380 0x7f5df8140cb0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5df806c4d0 tx=0x7f5df002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166554565372fd21 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.950+0000 7f5deffff640 1 -- 172.21.15.172:0/2198355750 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df002cb20 con 0x7f5df815c380 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.950+0000 7f5deffff640 1 -- 172.21.15.172:0/2198355750 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df002ccc0 con 0x7f5df815c380 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.950+0000 7f5deffff640 1 -- 172.21.15.172:0/2198355750 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0034610 con 0x7f5df815c380 2024-04-14T08:42:29.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.951+0000 7f5dff588640 1 -- 172.21.15.172:0/2198355750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df815c380 msgr2=0x7f5df8140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:29.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.951+0000 7f5dff588640 1 --2- 172.21.15.172:0/2198355750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df815c380 0x7f5df8140cb0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5df806c4d0 tx=0x7f5df002bf80 comp rx=0 tx=0).stop 2024-04-14T08:42:29.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.951+0000 7f5dff588640 1 -- 172.21.15.172:0/2198355750 shutdown_connections 2024-04-14T08:42:29.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.951+0000 7f5dff588640 1 --2- 172.21.15.172:0/2198355750 >> v2:172.21.15.177:3300/0 conn(0x7f5df80b23e0 0x7f5df8140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:29.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.951+0000 7f5dff588640 1 --2- 172.21.15.172:0/2198355750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df815c380 0x7f5df8140cb0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:29.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.951+0000 7f5dff588640 1 -- 172.21.15.172:0/2198355750 >> 172.21.15.172:0/2198355750 conn(0x7f5df806b520 msgr2=0x7f5df806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:29.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.951+0000 7f5dff588640 1 -- 172.21.15.172:0/2198355750 shutdown_connections 2024-04-14T08:42:29.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.951+0000 7f5dff588640 1 -- 172.21.15.172:0/2198355750 wait complete. 2024-04-14T08:42:29.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.952+0000 7f5dff588640 1 Processor -- start 2024-04-14T08:42:29.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.952+0000 7f5dff588640 1 -- start start 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dff588640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5df80b23e0 0x7f5df81485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dff588640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5df813df30 0x7f5df8072960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dff588640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df815c380 0x7f5df8137880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dff588640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5df815be60 con 0x7f5df815c380 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dff588640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5df815bb80 con 0x7f5df80b23e0 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dff588640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5df813e310 con 0x7f5df813df30 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dfdb25640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df815c380 0x7f5df8137880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dfdb25640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df815c380 0x7f5df8137880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57060/0 (socket says 172.21.15.172:57060) 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dfdb25640 1 -- 172.21.15.172:0/3833577795 learned_addr learned my addr 172.21.15.172:0/3833577795 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dfcb23640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5df813df30 0x7f5df8072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dfd324640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5df80b23e0 0x7f5df81485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dfdb25640 1 -- 172.21.15.172:0/3833577795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5df813df30 msgr2=0x7f5df8072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dfdb25640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5df813df30 0x7f5df8072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:29.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dfdb25640 1 -- 172.21.15.172:0/3833577795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5df80b23e0 msgr2=0x7f5df81485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dfdb25640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5df80b23e0 0x7f5df81485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dfdb25640 1 -- 172.21.15.172:0/3833577795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df80b4fb0 con 0x7f5df815c380 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dfd324640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5df80b23e0 0x7f5df81485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dfdb25640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df815c380 0x7f5df8137880 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f5df0036040 tx=0x7f5df0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dedffb640 1 -- 172.21.15.172:0/3833577795 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0043020 con 0x7f5df815c380 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dedffb640 1 -- 172.21.15.172:0/3833577795 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df003e450 con 0x7f5df815c380 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dff588640 1 -- 172.21.15.172:0/3833577795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df8155ed0 con 0x7f5df815c380 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.953+0000 7f5dedffb640 1 -- 172.21.15.172:0/3833577795 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0009510 con 0x7f5df815c380 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.954+0000 7f5dff588640 1 -- 172.21.15.172:0/3833577795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df80b5250 con 0x7f5df815c380 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.955+0000 7f5dff588640 1 -- 172.21.15.172:0/3833577795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df8068590 con 0x7f5df815c380 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.955+0000 7f5dedffb640 1 -- 172.21.15.172:0/3833577795 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5df0046020 con 0x7f5df815c380 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.956+0000 7f5dedffb640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5ddc06c8f0 0x7f5ddc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.956+0000 7f5dedffb640 1 -- 172.21.15.172:0/3833577795 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7f5df00ae5b0 con 0x7f5df815c380 2024-04-14T08:42:29.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.956+0000 7f5dfd324640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5ddc06c8f0 0x7f5ddc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:29.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.956+0000 7f5dfd324640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5ddc06c8f0 0x7f5ddc06edb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5df4000f60 tx=0x7f5df4007a10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:29.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:29.960+0000 7f5dedffb640 1 -- 172.21.15.172:0/3833577795 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5df004d030 con 0x7f5df815c380 2024-04-14T08:42:30.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.174+0000 7f5dff588640 1 -- 172.21.15.172:0/3833577795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5df80cd0a0 con 0x7f5df815c380 2024-04-14T08:42:30.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.831+0000 7f5dedffb640 1 -- 172.21.15.172:0/3833577795 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) ==== 104+0+0 (secure 0 0 0) 0x7f5df007c820 con 0x7f5df815c380 2024-04-14T08:42:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.835+0000 7f5dff588640 1 -- 172.21.15.172:0/3833577795 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5ddc06c8f0 msgr2=0x7f5ddc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:30.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.835+0000 7f5dff588640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5ddc06c8f0 0x7f5ddc06edb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5df4000f60 tx=0x7f5df4007a10 comp rx=0 tx=0).stop 2024-04-14T08:42:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.835+0000 7f5dff588640 1 -- 172.21.15.172:0/3833577795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df815c380 msgr2=0x7f5df8137880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.835+0000 7f5dff588640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df815c380 0x7f5df8137880 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f5df0036040 tx=0x7f5df0004a40 comp rx=0 tx=0).stop 2024-04-14T08:42:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.835+0000 7f5dff588640 1 -- 172.21.15.172:0/3833577795 shutdown_connections 2024-04-14T08:42:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.835+0000 7f5dff588640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5ddc06c8f0 0x7f5ddc06edb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.835+0000 7f5dff588640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5df815c380 0x7f5df8137880 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.835+0000 7f5dff588640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5df813df30 0x7f5df8072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.835+0000 7f5dff588640 1 --2- 172.21.15.172:0/3833577795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5df80b23e0 0x7f5df81485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.835+0000 7f5dff588640 1 -- 172.21.15.172:0/3833577795 >> 172.21.15.172:0/3833577795 conn(0x7f5df806b520 msgr2=0x7f5df80b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.835+0000 7f5dff588640 1 -- 172.21.15.172:0/3833577795 shutdown_connections 2024-04-14T08:42:30.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.835+0000 7f5dff588640 1 -- 172.21.15.172:0/3833577795 wait complete. 2024-04-14T08:42:30.848 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-14T08:42:30.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.956+0000 7f2a5395e640 1 Processor -- start 2024-04-14T08:42:30.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.956+0000 7f2a5395e640 1 -- start start 2024-04-14T08:42:30.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.957+0000 7f2a5395e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2a4c15b350 0x7f2a4c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:30.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.957+0000 7f2a5395e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a4c0cc170 0x7f2a4c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:30.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.957+0000 7f2a5395e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a4c0cdb90 con 0x7f2a4c0cecc0 2024-04-14T08:42:30.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.957+0000 7f2a5395e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2a4c0cf130 con 0x7f2a4c0cc170 2024-04-14T08:42:30.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.957+0000 7f2a5395e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2a4c0cd940 con 0x7f2a4c15b350 2024-04-14T08:42:30.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.957+0000 7f2a50ef9640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a4c0cc170 0x7f2a4c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:30.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.957+0000 7f2a51efb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2a4c0cecc0 0x7f2a4c15ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56122/0 (socket says 172.21.15.172:56122) 2024-04-14T08:42:30.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.957+0000 7f2a51efb640 1 -- 172.21.15.172:0/3284549174 learned_addr learned my addr 172.21.15.172:0/3284549174 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:30.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.957+0000 7f2a50ef9640 1 --2- 172.21.15.172:0/3284549174 >> v2:172.21.15.172:3300/0 conn(0x7f2a4c0cc170 0x7f2a4c0ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57068/0 (socket says 172.21.15.172:57068) 2024-04-14T08:42:30.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.957+0000 7f2a516fa640 1 --2- 172.21.15.172:0/3284549174 >> v2:172.21.15.177:3300/0 conn(0x7f2a4c15b350 0x7f2a4c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:30.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.958+0000 7f2a50ef9640 1 -- 172.21.15.172:0/3284549174 >> v2:172.21.15.177:3300/0 conn(0x7f2a4c15b350 msgr2=0x7f2a4c0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:30.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.958+0000 7f2a50ef9640 1 --2- 172.21.15.172:0/3284549174 >> v2:172.21.15.177:3300/0 conn(0x7f2a4c15b350 0x7f2a4c0cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:30.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.958+0000 7f2a50ef9640 1 -- 172.21.15.172:0/3284549174 >> v1:172.21.15.172:6789/0 conn(0x7f2a4c0cecc0 legacy=0x7f2a4c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:30.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.958+0000 7f2a50ef9640 1 -- 172.21.15.172:0/3284549174 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4c0cde70 con 0x7f2a4c0cc170 2024-04-14T08:42:30.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.958+0000 7f2a516fa640 1 --2- 172.21.15.172:0/3284549174 >> v2:172.21.15.177:3300/0 conn(0x7f2a4c15b350 0x7f2a4c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:30.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.958+0000 7f2a50ef9640 1 --2- 172.21.15.172:0/3284549174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a4c0cc170 0x7f2a4c0ce780 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c06cb20 tx=0x7f2a3c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0704333b2e5349 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:30.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.958+0000 7f2a43fff640 1 -- 172.21.15.172:0/3284549174 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c0315a0 con 0x7f2a4c0cc170 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.958+0000 7f2a43fff640 1 -- 172.21.15.172:0/3284549174 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a3c037070 con 0x7f2a4c0cc170 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.958+0000 7f2a43fff640 1 -- 172.21.15.172:0/3284549174 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c031960 con 0x7f2a4c0cc170 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.959+0000 7f2a5395e640 1 -- 172.21.15.172:0/3284549174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a4c0cc170 msgr2=0x7f2a4c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.959+0000 7f2a5395e640 1 --2- 172.21.15.172:0/3284549174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a4c0cc170 0x7f2a4c0ce780 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c06cb20 tx=0x7f2a3c02b840 comp rx=0 tx=0).stop 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.959+0000 7f2a5395e640 1 -- 172.21.15.172:0/3284549174 shutdown_connections 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.959+0000 7f2a5395e640 1 --2- 172.21.15.172:0/3284549174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a4c0cc170 0x7f2a4c0ce780 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.959+0000 7f2a5395e640 1 --2- 172.21.15.172:0/3284549174 >> v2:172.21.15.177:3300/0 conn(0x7f2a4c15b350 0x7f2a4c0cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.959+0000 7f2a5395e640 1 -- 172.21.15.172:0/3284549174 >> 172.21.15.172:0/3284549174 conn(0x7f2a4c0b1a50 msgr2=0x7f2a4c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.959+0000 7f2a5395e640 1 -- 172.21.15.172:0/3284549174 shutdown_connections 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.959+0000 7f2a5395e640 1 -- 172.21.15.172:0/3284549174 wait complete. 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.960+0000 7f2a5395e640 1 Processor -- start 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.960+0000 7f2a5395e640 1 -- start start 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a5395e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a4c0cc170 0x7f2a4c0c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:30.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a5395e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a4c0cecc0 0x7f2a4c1502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a5395e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a4c15b350 0x7f2a4c152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a5395e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c0cf560 con 0x7f2a4c0cc170 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a5395e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c0ce4c0 con 0x7f2a4c0cecc0 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a5395e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4c0c5e80 con 0x7f2a4c15b350 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a516fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a4c0cc170 0x7f2a4c0c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a516fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a4c0cc170 0x7f2a4c0c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57082/0 (socket says 172.21.15.172:57082) 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a516fa640 1 -- 172.21.15.172:0/1329043887 learned_addr learned my addr 172.21.15.172:0/1329043887 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a516fa640 1 -- 172.21.15.172:0/1329043887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a4c15b350 msgr2=0x7f2a4c152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a516fa640 1 --2- 172.21.15.172:0/1329043887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a4c15b350 0x7f2a4c152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a516fa640 1 -- 172.21.15.172:0/1329043887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a4c0cecc0 msgr2=0x7f2a4c1502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a516fa640 1 --2- 172.21.15.172:0/1329043887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a4c0cecc0 0x7f2a4c1502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a516fa640 1 -- 172.21.15.172:0/1329043887 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4c0c8c70 con 0x7f2a4c0cc170 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.961+0000 7f2a516fa640 1 --2- 172.21.15.172:0/1329043887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a4c0cc170 0x7f2a4c0c8730 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c820 tx=0x7f2a4800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:30.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.962+0000 7f2a41ffb640 1 -- 172.21.15.172:0/1329043887 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48015070 con 0x7f2a4c0cc170 2024-04-14T08:42:30.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.962+0000 7f2a41ffb640 1 -- 172.21.15.172:0/1329043887 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a48007a30 con 0x7f2a4c0cc170 2024-04-14T08:42:30.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.962+0000 7f2a5395e640 1 -- 172.21.15.172:0/1329043887 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4c150b30 con 0x7f2a4c0cc170 2024-04-14T08:42:30.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.962+0000 7f2a41ffb640 1 -- 172.21.15.172:0/1329043887 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48007de0 con 0x7f2a4c0cc170 2024-04-14T08:42:30.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.962+0000 7f2a5395e640 1 -- 172.21.15.172:0/1329043887 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4c1510f0 con 0x7f2a4c0cc170 2024-04-14T08:42:30.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.963+0000 7f2a5395e640 1 -- 172.21.15.172:0/1329043887 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a4c0686a0 con 0x7f2a4c0cc170 2024-04-14T08:42:30.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.963+0000 7f2a41ffb640 1 -- 172.21.15.172:0/1329043887 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a48018070 con 0x7f2a4c0cc170 2024-04-14T08:42:30.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.963+0000 7f2a41ffb640 1 --2- 172.21.15.172:0/1329043887 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3006c8f0 0x7f2a3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:30.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.964+0000 7f2a41ffb640 1 -- 172.21.15.172:0/1329043887 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6302+0+0 (secure 0 0 0) 0x7f2a4808bc40 con 0x7f2a4c0cc170 2024-04-14T08:42:30.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.964+0000 7f2a50ef9640 1 --2- 172.21.15.172:0/1329043887 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3006c8f0 0x7f2a3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:30.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.964+0000 7f2a50ef9640 1 --2- 172.21.15.172:0/1329043887 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3006c8f0 0x7f2a3006edb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c036000 tx=0x7f2a3c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:30.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:30.968+0000 7f2a41ffb640 1 -- 172.21.15.172:0/1329043887 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2a48059e20 con 0x7f2a4c0cc170 2024-04-14T08:42:31.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.142+0000 7f2a5395e640 1 -- 172.21.15.172:0/1329043887 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2a4c06d690 con 0x7f2a4c0cc170 2024-04-14T08:42:31.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.841+0000 7f2a41ffb640 1 -- 172.21.15.172:0/1329043887 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v106) ==== 181+0+0 (secure 0 0 0) 0x7f2a48059720 con 0x7f2a4c0cc170 2024-04-14T08:42:31.842 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' removed 2024-04-14T08:42:31.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.844+0000 7f2a5395e640 1 -- 172.21.15.172:0/1329043887 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3006c8f0 msgr2=0x7f2a3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:31.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.844+0000 7f2a5395e640 1 --2- 172.21.15.172:0/1329043887 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3006c8f0 0x7f2a3006edb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c036000 tx=0x7f2a3c030000 comp rx=0 tx=0).stop 2024-04-14T08:42:31.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.844+0000 7f2a5395e640 1 -- 172.21.15.172:0/1329043887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a4c0cc170 msgr2=0x7f2a4c0c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:31.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.844+0000 7f2a5395e640 1 --2- 172.21.15.172:0/1329043887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a4c0cc170 0x7f2a4c0c8730 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c820 tx=0x7f2a4800cce0 comp rx=0 tx=0).stop 2024-04-14T08:42:31.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.845+0000 7f2a5395e640 1 -- 172.21.15.172:0/1329043887 shutdown_connections 2024-04-14T08:42:31.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.845+0000 7f2a5395e640 1 --2- 172.21.15.172:0/1329043887 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3006c8f0 0x7f2a3006edb0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:31.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.845+0000 7f2a5395e640 1 --2- 172.21.15.172:0/1329043887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a4c15b350 0x7f2a4c152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:31.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.845+0000 7f2a5395e640 1 --2- 172.21.15.172:0/1329043887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a4c0cecc0 0x7f2a4c1502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:31.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.845+0000 7f2a5395e640 1 --2- 172.21.15.172:0/1329043887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a4c0cc170 0x7f2a4c0c8730 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:31.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.845+0000 7f2a5395e640 1 -- 172.21.15.172:0/1329043887 >> 172.21.15.172:0/1329043887 conn(0x7f2a4c0b1a50 msgr2=0x7f2a4c074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:31.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.845+0000 7f2a5395e640 1 -- 172.21.15.172:0/1329043887 shutdown_connections 2024-04-14T08:42:31.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.845+0000 7f2a5395e640 1 -- 172.21.15.172:0/1329043887 wait complete. 2024-04-14T08:42:31.859 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-14T08:42:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.968+0000 7f785d515640 1 Processor -- start 2024-04-14T08:42:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.968+0000 7f785d515640 1 -- start start 2024-04-14T08:42:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.969+0000 7f785d515640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f78580cee90 0x7f7858166060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.969+0000 7f785d515640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78580cdb90 con 0x7f78580cc340 2024-04-14T08:42:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.969+0000 7f785d515640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78580cde70 con 0x7f785815b350 2024-04-14T08:42:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.969+0000 7f785d515640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f78580cd940 con 0x7f78580cee90 2024-04-14T08:42:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.969+0000 7f784ffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f78580cc340 0x7f78580ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56136/0 (socket says 172.21.15.172:56136) 2024-04-14T08:42:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.969+0000 7f784ffff640 1 -- 172.21.15.172:0/820017496 learned_addr learned my addr 172.21.15.172:0/820017496 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.969+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2785942949 0 0) 0x7f78580cdb90 con 0x7f78580cc340 2024-04-14T08:42:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.969+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7844003520 con 0x7f78580cc340 2024-04-14T08:42:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.969+0000 7f78577fe640 1 --2- 172.21.15.172:0/820017496 >> v2:172.21.15.177:3300/0 conn(0x7f78580cee90 0x7f7858166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3270698355 0 0) 0x7f7844003520 con 0x7f78580cc340 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 >> v2:172.21.15.177:3300/0 conn(0x7f78580cee90 msgr2=0x7f7858166060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 --2- 172.21.15.172:0/820017496 >> v2:172.21.15.177:3300/0 conn(0x7f78580cee90 0x7f7858166060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 >> v1:172.21.15.177:6789/0 conn(0x7f785815b350 legacy=0x7f78580cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78580ce150 con 0x7f78580cc340 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f7840001670 con 0x7f78580cc340 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 >> v1:172.21.15.172:6789/0 conn(0x7f78580cc340 legacy=0x7f78580ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 --2- 172.21.15.172:0/820017496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78440039b0 0x7f7844003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 --2- 172.21.15.172:0/820017496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7844004800 0x7f7844004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 --2- 172.21.15.172:0/820017496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78440052e0 0x7f784400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f78580cd940 con 0x7f78440039b0 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f784400c1b0 con 0x7f7844004800 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f784400e5c0 con 0x7f78440052e0 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f7856ffd640 1 --2- 172.21.15.172:0/820017496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78440039b0 0x7f7844003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f784ffff640 1 --2- 172.21.15.172:0/820017496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7844004800 0x7f7844004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78577fe640 1 --2- 172.21.15.172:0/820017496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78440052e0 0x7f784400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f7856ffd640 1 -- 172.21.15.172:0/820017496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78440052e0 msgr2=0x7f784400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f7856ffd640 1 --2- 172.21.15.172:0/820017496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78440052e0 0x7f784400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f7856ffd640 1 -- 172.21.15.172:0/820017496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7844004800 msgr2=0x7f7844004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f7856ffd640 1 --2- 172.21.15.172:0/820017496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7844004800 0x7f7844004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f7856ffd640 1 -- 172.21.15.172:0/820017496 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7844003520 con 0x7f78440039b0 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.970+0000 7f78577fe640 1 -- 172.21.15.172:0/820017496 reap_dead start 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.971+0000 7f7856ffd640 1 --2- 172.21.15.172:0/820017496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78440039b0 0x7f7844003dd0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7848001720 tx=0x7f7844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e01add7688dca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.971+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848001370 con 0x7f78440039b0 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.971+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78480052d0 con 0x7f78440039b0 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.971+0000 7f78567fc640 1 -- 172.21.15.172:0/820017496 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848003830 con 0x7f78440039b0 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.971+0000 7f785d515640 1 -- 172.21.15.172:0/820017496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78440039b0 msgr2=0x7f7844003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.971+0000 7f785d515640 1 --2- 172.21.15.172:0/820017496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78440039b0 0x7f7844003dd0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7848001720 tx=0x7f7844005830 comp rx=0 tx=0).stop 2024-04-14T08:42:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.971+0000 7f785d515640 1 -- 172.21.15.172:0/820017496 shutdown_connections 2024-04-14T08:42:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.971+0000 7f785d515640 1 --2- 172.21.15.172:0/820017496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78440039b0 0x7f7844003dd0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.971+0000 7f785d515640 1 -- 172.21.15.172:0/820017496 >> 172.21.15.172:0/820017496 conn(0x7f78580b1a50 msgr2=0x7f78580b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.971+0000 7f785d515640 1 -- 172.21.15.172:0/820017496 shutdown_connections 2024-04-14T08:42:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.972+0000 7f785d515640 1 -- 172.21.15.172:0/820017496 wait complete. 2024-04-14T08:42:31.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.973+0000 7f785d515640 1 Processor -- start 2024-04-14T08:42:31.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.973+0000 7f785d515640 1 -- start start 2024-04-14T08:42:31.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.973+0000 7f785d515640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78440039b0 0x7f7858150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.973+0000 7f785d515640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78581587b0 0x7f7858158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.973+0000 7f785d515640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f785809cc80 0x7f785809d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.973+0000 7f785d515640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f78580ce780 con 0x7f78581587b0 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.973+0000 7f785d515640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f78580ce4c0 con 0x7f785809cc80 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.973+0000 7f785d515640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f78580ce900 con 0x7f78440039b0 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.973+0000 7f784ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78581587b0 0x7f7858158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.973+0000 7f784ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78581587b0 0x7f7858158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57098/0 (socket says 172.21.15.172:57098) 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.973+0000 7f784ffff640 1 -- 172.21.15.172:0/3778355171 learned_addr learned my addr 172.21.15.172:0/3778355171 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f7856ffd640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78440039b0 0x7f7858150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f78577fe640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f785809cc80 0x7f785809d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f784ffff640 1 -- 172.21.15.172:0/3778355171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78440039b0 msgr2=0x7f7858150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f784ffff640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78440039b0 0x7f7858150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f784ffff640 1 -- 172.21.15.172:0/3778355171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f785809cc80 msgr2=0x7f785809d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f784ffff640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f785809cc80 0x7f785809d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f784ffff640 1 -- 172.21.15.172:0/3778355171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78581f44e0 con 0x7f78581587b0 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f784ffff640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78581587b0 0x7f7858158c30 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f7840007340 tx=0x7f7840007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f784f7fe640 1 -- 172.21.15.172:0/3778355171 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784000f8c0 con 0x7f78581587b0 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f784f7fe640 1 -- 172.21.15.172:0/3778355171 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7840018070 con 0x7f78581587b0 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f785d515640 1 -- 172.21.15.172:0/3778355171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78581f47c0 con 0x7f78581587b0 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f784f7fe640 1 -- 172.21.15.172:0/3778355171 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784000fcc0 con 0x7f78581587b0 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.974+0000 7f785d515640 1 -- 172.21.15.172:0/3778355171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78581f4d80 con 0x7f78581587b0 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.975+0000 7f785d515640 1 -- 172.21.15.172:0/3778355171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78580686a0 con 0x7f78581587b0 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.975+0000 7f784f7fe640 1 -- 172.21.15.172:0/3778355171 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f784001d460 con 0x7f78581587b0 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.976+0000 7f784f7fe640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f783806c920 0x7f783806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.976+0000 7f784f7fe640 1 -- 172.21.15.172:0/3778355171 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5864+0+0 (secure 0 0 0) 0x7f784008ba40 con 0x7f78581587b0 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.976+0000 7f7856ffd640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f783806c920 0x7f783806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.976+0000 7f7856ffd640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f783806c920 0x7f783806ede0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7848033000 tx=0x7f7848009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:31.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:31.980+0000 7f784f7fe640 1 -- 172.21.15.172:0/3778355171 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f784000fe60 con 0x7f78581587b0 2024-04-14T08:42:32.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.158+0000 7f785d515640 1 -- 172.21.15.172:0/3778355171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f78580cbc00 con 0x7f78581587b0 2024-04-14T08:42:32.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.846+0000 7f784f7fe640 1 -- 172.21.15.172:0/3778355171 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v107) ==== 169+0+0 (secure 0 0 0) 0x7f784008be20 con 0x7f78581587b0 2024-04-14T08:42:32.847 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' removed 2024-04-14T08:42:32.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.850+0000 7f785d515640 1 -- 172.21.15.172:0/3778355171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f783806c920 msgr2=0x7f783806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:32.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.850+0000 7f785d515640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f783806c920 0x7f783806ede0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7848033000 tx=0x7f7848009030 comp rx=0 tx=0).stop 2024-04-14T08:42:32.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.850+0000 7f785d515640 1 -- 172.21.15.172:0/3778355171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78581587b0 msgr2=0x7f7858158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:32.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.850+0000 7f785d515640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78581587b0 0x7f7858158c30 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f7840007340 tx=0x7f7840007800 comp rx=0 tx=0).stop 2024-04-14T08:42:32.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.850+0000 7f785d515640 1 -- 172.21.15.172:0/3778355171 shutdown_connections 2024-04-14T08:42:32.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.850+0000 7f785d515640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f783806c920 0x7f783806ede0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:32.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.850+0000 7f785d515640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f785809cc80 0x7f785809d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:32.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.850+0000 7f785d515640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78581587b0 0x7f7858158c30 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:32.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.850+0000 7f785d515640 1 --2- 172.21.15.172:0/3778355171 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78440039b0 0x7f7858150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:32.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.850+0000 7f785d515640 1 -- 172.21.15.172:0/3778355171 >> 172.21.15.172:0/3778355171 conn(0x7f78580b1a50 msgr2=0x7f78580c4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:32.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.851+0000 7f785d515640 1 -- 172.21.15.172:0/3778355171 shutdown_connections 2024-04-14T08:42:32.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.851+0000 7f785d515640 1 -- 172.21.15.172:0/3778355171 wait complete. 2024-04-14T08:42:32.866 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-14T08:42:32.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.972+0000 7f5cc48c0640 1 Processor -- start 2024-04-14T08:42:32.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.972+0000 7f5cc48c0640 1 -- start start 2024-04-14T08:42:32.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.973+0000 7f5cc48c0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5cc015f460 0x7f5cc00b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:32.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.973+0000 7f5cc48c0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc015edd0 con 0x7f5cc006bc30 2024-04-14T08:42:32.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.973+0000 7f5cc48c0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc006c100 con 0x7f5cc00b3bf0 2024-04-14T08:42:32.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.973+0000 7f5cc48c0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5cc0141960 con 0x7f5cc015f460 2024-04-14T08:42:32.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.973+0000 7f5cbed9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5cc006bc30 0x7f5cc01683c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56148/0 (socket says 172.21.15.172:56148) 2024-04-14T08:42:32.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.973+0000 7f5cbed9d640 1 -- 172.21.15.172:0/3663701710 learned_addr learned my addr 172.21.15.172:0/3663701710 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbe59c640 1 --2- 172.21.15.172:0/3663701710 >> v2:172.21.15.177:3300/0 conn(0x7f5cc015f460 0x7f5cc00b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3529041416 0 0) 0x7f5cc015edd0 con 0x7f5cc006bc30 2024-04-14T08:42:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ca8003520 con 0x7f5cc006bc30 2024-04-14T08:42:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 72630467 0 0) 0x7f5ca8003520 con 0x7f5cc006bc30 2024-04-14T08:42:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 >> v2:172.21.15.177:3300/0 conn(0x7f5cc015f460 msgr2=0x7f5cc00b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:32.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 --2- 172.21.15.172:0/3663701710 >> v2:172.21.15.177:3300/0 conn(0x7f5cc015f460 0x7f5cc00b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 >> v1:172.21.15.177:6789/0 conn(0x7f5cc00b3bf0 legacy=0x7f5cc006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc006c3e0 con 0x7f5cc006bc30 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f5cb8001670 con 0x7f5cc006bc30 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 >> v1:172.21.15.172:6789/0 conn(0x7f5cc006bc30 legacy=0x7f5cc01683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 --2- 172.21.15.172:0/3663701710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ca80039b0 0x7f5ca8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 --2- 172.21.15.172:0/3663701710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 0x7f5ca8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 --2- 172.21.15.172:0/3663701710 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ca80051c0 0x7f5ca800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc0141960 con 0x7f5ca8004800 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca800c160 con 0x7f5ca80039b0 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca800c360 con 0x7f5ca80051c0 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbdd9b640 1 --2- 172.21.15.172:0/3663701710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 0x7f5ca8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbe59c640 1 --2- 172.21.15.172:0/3663701710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ca80039b0 0x7f5ca8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbed9d640 1 --2- 172.21.15.172:0/3663701710 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ca80051c0 0x7f5ca800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:32.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbdd9b640 1 -- 172.21.15.172:0/3663701710 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ca80051c0 msgr2=0x7f5ca800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbdd9b640 1 --2- 172.21.15.172:0/3663701710 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ca80051c0 0x7f5ca800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbdd9b640 1 -- 172.21.15.172:0/3663701710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ca80039b0 msgr2=0x7f5ca8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbdd9b640 1 --2- 172.21.15.172:0/3663701710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ca80039b0 0x7f5ca8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbdd9b640 1 -- 172.21.15.172:0/3663701710 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca8003520 con 0x7f5ca8004800 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.974+0000 7f5cbed9d640 1 -- 172.21.15.172:0/3663701710 reap_dead start 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.975+0000 7f5cbdd9b640 1 --2- 172.21.15.172:0/3663701710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 0x7f5ca8004c80 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4001750 tx=0x7f5ca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47250129804be212 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.975+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4001370 con 0x7f5ca8004800 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.975+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb40052d0 con 0x7f5ca8004800 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.975+0000 7f5cbd59a640 1 -- 172.21.15.172:0/3663701710 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4003830 con 0x7f5ca8004800 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.976+0000 7f5cc48c0640 1 -- 172.21.15.172:0/3663701710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 msgr2=0x7f5ca8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.976+0000 7f5cc48c0640 1 --2- 172.21.15.172:0/3663701710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 0x7f5ca8004c80 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4001750 tx=0x7f5ca8005830 comp rx=0 tx=0).stop 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.976+0000 7f5cc48c0640 1 -- 172.21.15.172:0/3663701710 shutdown_connections 2024-04-14T08:42:32.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.976+0000 7f5cc48c0640 1 --2- 172.21.15.172:0/3663701710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 0x7f5ca8004c80 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:32.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.976+0000 7f5cc48c0640 1 -- 172.21.15.172:0/3663701710 >> 172.21.15.172:0/3663701710 conn(0x7f5cc00d0780 msgr2=0x7f5cc00d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:32.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.976+0000 7f5cc48c0640 1 -- 172.21.15.172:0/3663701710 shutdown_connections 2024-04-14T08:42:32.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.976+0000 7f5cc48c0640 1 -- 172.21.15.172:0/3663701710 wait complete. 2024-04-14T08:42:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.977+0000 7f5cc48c0640 1 Processor -- start 2024-04-14T08:42:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.977+0000 7f5cc48c0640 1 -- start start 2024-04-14T08:42:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cc48c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 0x7f5cc01f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cc48c0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cc013d150 0x7f5cc01575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cc48c0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cc0157b00 0x7f5cc01f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:32.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cc48c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc015f840 con 0x7f5ca8004800 2024-04-14T08:42:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cc48c0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc006c9a0 con 0x7f5cc013d150 2024-04-14T08:42:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cc48c0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc015f9c0 con 0x7f5cc0157b00 2024-04-14T08:42:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 0x7f5cc01f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 0x7f5cc01f49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57122/0 (socket says 172.21.15.172:57122) 2024-04-14T08:42:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbe59c640 1 -- 172.21.15.172:0/4174236469 learned_addr learned my addr 172.21.15.172:0/4174236469 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbdd9b640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cc013d150 0x7f5cc01575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbed9d640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cc0157b00 0x7f5cc01f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbe59c640 1 -- 172.21.15.172:0/4174236469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cc0157b00 msgr2=0x7f5cc01f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbe59c640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cc0157b00 0x7f5cc01f7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:32.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbe59c640 1 -- 172.21.15.172:0/4174236469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cc013d150 msgr2=0x7f5cc01575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbe59c640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cc013d150 0x7f5cc01575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbe59c640 1 -- 172.21.15.172:0/4174236469 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc01f4f30 con 0x7f5ca8004800 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbdd9b640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cc013d150 0x7f5cc01575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbed9d640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cc0157b00 0x7f5cc01f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5cbe59c640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 0x7f5cc01f49f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0007f30 tx=0x7f5cb0011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.978+0000 7f5caeffd640 1 -- 172.21.15.172:0/4174236469 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb001a020 con 0x7f5ca8004800 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.979+0000 7f5caeffd640 1 -- 172.21.15.172:0/4174236469 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb0016470 con 0x7f5ca8004800 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.979+0000 7f5cc48c0640 1 -- 172.21.15.172:0/4174236469 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc0155a20 con 0x7f5ca8004800 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.979+0000 7f5caeffd640 1 -- 172.21.15.172:0/4174236469 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0016860 con 0x7f5ca8004800 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.979+0000 7f5cc48c0640 1 -- 172.21.15.172:0/4174236469 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc0155fe0 con 0x7f5ca8004800 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.980+0000 7f5cc48c0640 1 -- 172.21.15.172:0/4174236469 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc0068590 con 0x7f5ca8004800 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.980+0000 7f5caeffd640 1 -- 172.21.15.172:0/4174236469 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5cb0005d10 con 0x7f5ca8004800 2024-04-14T08:42:32.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.980+0000 7f5caeffd640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5ca006c920 0x7f5ca006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:32.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.980+0000 7f5caeffd640 1 -- 172.21.15.172:0/4174236469 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5421+0+0 (secure 0 0 0) 0x7f5cb008c930 con 0x7f5ca8004800 2024-04-14T08:42:32.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.980+0000 7f5cbdd9b640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5ca006c920 0x7f5ca006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:32.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.981+0000 7f5cbdd9b640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5ca006c920 0x7f5ca006ede0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4033000 tx=0x7f5cb4009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:32.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:32.985+0000 7f5caeffd640 1 -- 172.21.15.172:0/4174236469 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5cb0024030 con 0x7f5ca8004800 2024-04-14T08:42:33.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.161+0000 7f5cc48c0640 1 -- 172.21.15.172:0/4174236469 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5cc0067670 con 0x7f5ca8004800 2024-04-14T08:42:33.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.164+0000 7f5caeffd640 1 -- 172.21.15.172:0/4174236469 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f5cb005af10 con 0x7f5ca8004800 2024-04-14T08:42:33.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.166+0000 7f5cc48c0640 1 -- 172.21.15.172:0/4174236469 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5ca006c920 msgr2=0x7f5ca006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:33.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.166+0000 7f5cc48c0640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5ca006c920 0x7f5ca006ede0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4033000 tx=0x7f5cb4009030 comp rx=0 tx=0).stop 2024-04-14T08:42:33.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.166+0000 7f5cc48c0640 1 -- 172.21.15.172:0/4174236469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 msgr2=0x7f5cc01f49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:33.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.166+0000 7f5cc48c0640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 0x7f5cc01f49f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0007f30 tx=0x7f5cb0011660 comp rx=0 tx=0).stop 2024-04-14T08:42:33.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.166+0000 7f5cc48c0640 1 -- 172.21.15.172:0/4174236469 shutdown_connections 2024-04-14T08:42:33.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.166+0000 7f5cc48c0640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5ca006c920 0x7f5ca006ede0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.166+0000 7f5cc48c0640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cc0157b00 0x7f5cc01f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.166+0000 7f5cc48c0640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cc013d150 0x7f5cc01575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.166+0000 7f5cc48c0640 1 --2- 172.21.15.172:0/4174236469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ca8004800 0x7f5cc01f49f0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.166+0000 7f5cc48c0640 1 -- 172.21.15.172:0/4174236469 >> 172.21.15.172:0/4174236469 conn(0x7f5cc00d0780 msgr2=0x7f5cc00aed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:33.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.166+0000 7f5cc48c0640 1 -- 172.21.15.172:0/4174236469 shutdown_connections 2024-04-14T08:42:33.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.166+0000 7f5cc48c0640 1 -- 172.21.15.172:0/4174236469 wait complete. 2024-04-14T08:42:33.178 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-14T08:42:33.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.279+0000 7fdf9879c640 1 Processor -- start 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.279+0000 7fdf9879c640 1 -- start start 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.280+0000 7fdf9879c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdf9015e890 0x7fdf900d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.280+0000 7fdf9879c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf900cedd0 0x7fdf900b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.280+0000 7fdf9879c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf900cf320 con 0x7fdf900d2830 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.280+0000 7fdf9879c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fdf900cff30 con 0x7fdf9015e890 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.280+0000 7fdf9879c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdf900d2350 con 0x7fdf900cedd0 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.281+0000 7fdf96d39640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf900cedd0 0x7fdf900b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.281+0000 7fdf95d37640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdf9015e890 0x7fdf900d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.281+0000 7fdf96d39640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf900cedd0 0x7fdf900b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:38530/0 (socket says 172.21.15.172:38530) 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.281+0000 7fdf96d39640 1 -- 172.21.15.172:0/3147916290 learned_addr learned my addr 172.21.15.172:0/3147916290 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.281+0000 7fdf96d39640 1 -- 172.21.15.172:0/3147916290 >> v2:172.21.15.177:3300/0 conn(0x7fdf9015e890 msgr2=0x7fdf900d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.281+0000 7fdf96d39640 1 --2- 172.21.15.172:0/3147916290 >> v2:172.21.15.177:3300/0 conn(0x7fdf9015e890 0x7fdf900d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.281+0000 7fdf96d39640 1 -- 172.21.15.172:0/3147916290 >> v1:172.21.15.177:6789/0 conn(0x7fdf900d2830 legacy=0x7fdf900d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.281+0000 7fdf96d39640 1 -- 172.21.15.172:0/3147916290 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf900cf600 con 0x7fdf900cedd0 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.281+0000 7fdf95536640 1 -- 172.21.15.172:0/3147916290 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2393215190 0 0) 0x7fdf900cf320 con 0x7fdf900d2830 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.281+0000 7fdf95d37640 1 --2- 172.21.15.172:0/3147916290 >> v2:172.21.15.177:3300/0 conn(0x7fdf9015e890 0x7fdf900d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.282+0000 7fdf96d39640 1 --2- 172.21.15.172:0/3147916290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf900cedd0 0x7fdf900b3710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009870 tx=0x7fdf8c02ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c9d3304368c6bce server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.282+0000 7fdf95536640 1 -- 172.21.15.172:0/3147916290 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c02f840 con 0x7fdf900cedd0 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.282+0000 7fdf95536640 1 -- 172.21.15.172:0/3147916290 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c02f9e0 con 0x7fdf900cedd0 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.282+0000 7fdf95536640 1 -- 172.21.15.172:0/3147916290 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c02fdb0 con 0x7fdf900cedd0 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.283+0000 7fdf9879c640 1 -- 172.21.15.172:0/3147916290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf900cedd0 msgr2=0x7fdf900b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.283+0000 7fdf9879c640 1 --2- 172.21.15.172:0/3147916290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf900cedd0 0x7fdf900b3710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009870 tx=0x7fdf8c02ba20 comp rx=0 tx=0).stop 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.283+0000 7fdf9879c640 1 -- 172.21.15.172:0/3147916290 shutdown_connections 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.283+0000 7fdf9879c640 1 --2- 172.21.15.172:0/3147916290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf900cedd0 0x7fdf900b3710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.283+0000 7fdf9879c640 1 --2- 172.21.15.172:0/3147916290 >> v2:172.21.15.177:3300/0 conn(0x7fdf9015e890 0x7fdf900d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.283+0000 7fdf9879c640 1 -- 172.21.15.172:0/3147916290 >> 172.21.15.172:0/3147916290 conn(0x7fdf9009d590 msgr2=0x7fdf900d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:33.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.283+0000 7fdf9879c640 1 -- 172.21.15.172:0/3147916290 shutdown_connections 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.283+0000 7fdf9879c640 1 -- 172.21.15.172:0/3147916290 wait complete. 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.284+0000 7fdf9879c640 1 Processor -- start 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.284+0000 7fdf9879c640 1 -- start start 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf9879c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf900cedd0 0x7fdf900cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf9879c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf900d2830 0x7fdf900cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf9879c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf9015e890 0x7fdf900b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf9879c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdf900d1840 con 0x7fdf900cedd0 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf9879c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdf900cfbc0 con 0x7fdf900d2830 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf9879c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fdf900b3c50 con 0x7fdf9015e890 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf96538640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf900cedd0 0x7fdf900cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf96538640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf900cedd0 0x7fdf900cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57128/0 (socket says 172.21.15.172:57128) 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf96538640 1 -- 172.21.15.172:0/985176385 learned_addr learned my addr 172.21.15.172:0/985176385 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:33.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf96d39640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf9015e890 0x7fdf900b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf96538640 1 -- 172.21.15.172:0/985176385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf9015e890 msgr2=0x7fdf900b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf96538640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf9015e890 0x7fdf900b5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf96538640 1 -- 172.21.15.172:0/985176385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf900d2830 msgr2=0x7fdf900cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf96538640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf900d2830 0x7fdf900cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf96538640 1 -- 172.21.15.172:0/985176385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9015fbd0 con 0x7fdf900cedd0 2024-04-14T08:42:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf96538640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf900cedd0 0x7fdf900cd6b0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c890 tx=0x7fdf8800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.285+0000 7fdf86ffd640 1 -- 172.21.15.172:0/985176385 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88017070 con 0x7fdf900cedd0 2024-04-14T08:42:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.286+0000 7fdf86ffd640 1 -- 172.21.15.172:0/985176385 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf88003dc0 con 0x7fdf900cedd0 2024-04-14T08:42:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.286+0000 7fdf9879c640 1 -- 172.21.15.172:0/985176385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf90160340 con 0x7fdf900cedd0 2024-04-14T08:42:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.286+0000 7fdf86ffd640 1 -- 172.21.15.172:0/985176385 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880052b0 con 0x7fdf900cedd0 2024-04-14T08:42:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.286+0000 7fdf9879c640 1 -- 172.21.15.172:0/985176385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf90160900 con 0x7fdf900cedd0 2024-04-14T08:42:33.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.286+0000 7fdf9879c640 1 -- 172.21.15.172:0/985176385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf900d1630 con 0x7fdf900cedd0 2024-04-14T08:42:33.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.287+0000 7fdf86ffd640 1 -- 172.21.15.172:0/985176385 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf8801a070 con 0x7fdf900cedd0 2024-04-14T08:42:33.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.287+0000 7fdf86ffd640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdf7406c8f0 0x7fdf7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:33.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.288+0000 7fdf86ffd640 1 -- 172.21.15.172:0/985176385 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5421+0+0 (secure 0 0 0) 0x7fdf8808b150 con 0x7fdf900cedd0 2024-04-14T08:42:33.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.288+0000 7fdf95d37640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdf7406c8f0 0x7fdf7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:33.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.288+0000 7fdf95d37640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdf7406c8f0 0x7fdf7406edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fdf80006e80 tx=0x7fdf80006dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:33.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.292+0000 7fdf86ffd640 1 -- 172.21.15.172:0/985176385 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdf900d1630 con 0x7fdf900cedd0 2024-04-14T08:42:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.533+0000 7fdf9879c640 1 -- 172.21.15.172:0/985176385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fdf900d1f80 con 0x7fdf900cedd0 2024-04-14T08:42:33.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.847+0000 7fdf86ffd640 1 -- 172.21.15.172:0/985176385 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v451) ==== 146+0+0 (secure 0 0 0) 0x7fdf900d1f80 con 0x7fdf900cedd0 2024-04-14T08:42:33.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.849+0000 7fdf9879c640 1 -- 172.21.15.172:0/985176385 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdf7406c8f0 msgr2=0x7fdf7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:33.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.849+0000 7fdf9879c640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdf7406c8f0 0x7fdf7406edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fdf80006e80 tx=0x7fdf80006dd0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.849+0000 7fdf9879c640 1 -- 172.21.15.172:0/985176385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf900cedd0 msgr2=0x7fdf900cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:33.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.850+0000 7fdf9879c640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf900cedd0 0x7fdf900cd6b0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c890 tx=0x7fdf8800cd50 comp rx=0 tx=0).stop 2024-04-14T08:42:33.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.850+0000 7fdf9879c640 1 -- 172.21.15.172:0/985176385 shutdown_connections 2024-04-14T08:42:33.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.850+0000 7fdf9879c640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fdf7406c8f0 0x7fdf7406edb0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.850+0000 7fdf9879c640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf9015e890 0x7fdf900b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.850+0000 7fdf9879c640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf900d2830 0x7fdf900cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.850+0000 7fdf9879c640 1 --2- 172.21.15.172:0/985176385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf900cedd0 0x7fdf900cd6b0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.850+0000 7fdf9879c640 1 -- 172.21.15.172:0/985176385 >> 172.21.15.172:0/985176385 conn(0x7fdf9009d590 msgr2=0x7fdf900d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:33.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.850+0000 7fdf9879c640 1 -- 172.21.15.172:0/985176385 shutdown_connections 2024-04-14T08:42:33.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.850+0000 7fdf9879c640 1 -- 172.21.15.172:0/985176385 wait complete. 2024-04-14T08:42:33.861 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-14T08:42:33.861 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-14T08:42:33.862 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:42:33.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.963+0000 7f63457fe640 1 Processor -- start 2024-04-14T08:42:33.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.963+0000 7f63457fe640 1 -- start start 2024-04-14T08:42:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f63457fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634015c710 0x7f63400d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f63457fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f63400cde10 0x7f63400ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:33.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f63457fe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63400aef60 con 0x7f63400ced80 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f63457fe640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f63400aed10 con 0x7f63400cde10 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f63457fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f63400af890 con 0x7f634015c710 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f633e7fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f63400cde10 0x7f63400ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f633f7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f63400ced80 0x7f63400ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56154/0 (socket says 172.21.15.172:56154) 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f633f7fe640 1 -- 172.21.15.172:0/2689352466 learned_addr learned my addr 172.21.15.172:0/2689352466 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f633effd640 1 --2- 172.21.15.172:0/2689352466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634015c710 0x7f63400d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f633e7fc640 1 -- 172.21.15.172:0/2689352466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634015c710 msgr2=0x7f63400d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f633e7fc640 1 --2- 172.21.15.172:0/2689352466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634015c710 0x7f63400d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f633e7fc640 1 -- 172.21.15.172:0/2689352466 >> v1:172.21.15.172:6789/0 conn(0x7f63400ced80 legacy=0x7f63400ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f633dffb640 1 -- 172.21.15.172:0/2689352466 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 551931044 0 0) 0x7f63400aef60 con 0x7f63400ced80 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.964+0000 7f633e7fc640 1 -- 172.21.15.172:0/2689352466 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63400af240 con 0x7f63400cde10 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.965+0000 7f633e7fc640 1 --2- 172.21.15.172:0/2689352466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63400cde10 0x7f63400ce840 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f63340097a0 tx=0x7f633402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9172612012c5d3fc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:33.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.965+0000 7f633dffb640 1 -- 172.21.15.172:0/2689352466 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334035020 con 0x7f63400cde10 2024-04-14T08:42:33.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.965+0000 7f633dffb640 1 -- 172.21.15.172:0/2689352466 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6334031830 con 0x7f63400cde10 2024-04-14T08:42:33.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.965+0000 7f633dffb640 1 -- 172.21.15.172:0/2689352466 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334031c00 con 0x7f63400cde10 2024-04-14T08:42:33.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.966+0000 7f63457fe640 1 -- 172.21.15.172:0/2689352466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63400cde10 msgr2=0x7f63400ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:33.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.966+0000 7f63457fe640 1 --2- 172.21.15.172:0/2689352466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63400cde10 0x7f63400ce840 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f63340097a0 tx=0x7f633402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.966+0000 7f63457fe640 1 -- 172.21.15.172:0/2689352466 shutdown_connections 2024-04-14T08:42:33.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.966+0000 7f63457fe640 1 --2- 172.21.15.172:0/2689352466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63400cde10 0x7f63400ce840 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.966+0000 7f63457fe640 1 --2- 172.21.15.172:0/2689352466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634015c710 0x7f63400d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.966+0000 7f63457fe640 1 -- 172.21.15.172:0/2689352466 >> 172.21.15.172:0/2689352466 conn(0x7f634008c000 msgr2=0x7f63400d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:33.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.966+0000 7f63457fe640 1 -- 172.21.15.172:0/2689352466 shutdown_connections 2024-04-14T08:42:33.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.966+0000 7f63457fe640 1 -- 172.21.15.172:0/2689352466 wait complete. 2024-04-14T08:42:33.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.967+0000 7f63457fe640 1 Processor -- start 2024-04-14T08:42:33.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.967+0000 7f63457fe640 1 -- start start 2024-04-14T08:42:33.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f63457fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63400cde10 0x7f63400c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:33.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f63457fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63400ced80 0x7f63400cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:33.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f63457fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634015c710 0x7f63401542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:33.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f63457fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f63400d0e90 con 0x7f63400ced80 2024-04-14T08:42:33.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f63457fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f634015caf0 con 0x7f63400cde10 2024-04-14T08:42:33.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f63457fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f63400d1010 con 0x7f634015c710 2024-04-14T08:42:33.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f633e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63400ced80 0x7f63400cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:33.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f633e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63400ced80 0x7f63400cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57146/0 (socket says 172.21.15.172:57146) 2024-04-14T08:42:33.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f633e7fc640 1 -- 172.21.15.172:0/19443519 learned_addr learned my addr 172.21.15.172:0/19443519 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:33.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f633e7fc640 1 -- 172.21.15.172:0/19443519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634015c710 msgr2=0x7f63401542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:33.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f633e7fc640 1 --2- 172.21.15.172:0/19443519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634015c710 0x7f63401542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f633e7fc640 1 -- 172.21.15.172:0/19443519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63400cde10 msgr2=0x7f63400c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:33.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f633e7fc640 1 --2- 172.21.15.172:0/19443519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63400cde10 0x7f63400c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:33.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.968+0000 7f633e7fc640 1 -- 172.21.15.172:0/19443519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63400ca170 con 0x7f63400ced80 2024-04-14T08:42:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.969+0000 7f633e7fc640 1 --2- 172.21.15.172:0/19443519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63400ced80 0x7f63400cba50 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f633402bee0 tx=0x7f63340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.969+0000 7f63277fe640 1 -- 172.21.15.172:0/19443519 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334041070 con 0x7f63400ced80 2024-04-14T08:42:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.969+0000 7f63277fe640 1 -- 172.21.15.172:0/19443519 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f633403b920 con 0x7f63400ced80 2024-04-14T08:42:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.969+0000 7f63277fe640 1 -- 172.21.15.172:0/19443519 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334035040 con 0x7f63400ced80 2024-04-14T08:42:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.969+0000 7f63457fe640 1 -- 172.21.15.172:0/19443519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634006d290 con 0x7f63400ced80 2024-04-14T08:42:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.969+0000 7f63457fe640 1 -- 172.21.15.172:0/19443519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634006d850 con 0x7f63400ced80 2024-04-14T08:42:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.970+0000 7f63457fe640 1 -- 172.21.15.172:0/19443519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6340075490 con 0x7f63400ced80 2024-04-14T08:42:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.970+0000 7f63277fe640 1 -- 172.21.15.172:0/19443519 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6334038020 con 0x7f63400ced80 2024-04-14T08:42:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.970+0000 7f63277fe640 1 --2- 172.21.15.172:0/19443519 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f632c06c920 0x7f632c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.971+0000 7f63277fe640 1 -- 172.21.15.172:0/19443519 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5421+0+0 (secure 0 0 0) 0x7f63340ad380 con 0x7f63400ced80 2024-04-14T08:42:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.971+0000 7f633effd640 1 --2- 172.21.15.172:0/19443519 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f632c06c920 0x7f632c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.971+0000 7f633effd640 1 --2- 172.21.15.172:0/19443519 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f632c06c920 0x7f632c06ede0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f63300070b0 tx=0x7f6330007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:33.975+0000 7f63277fe640 1 -- 172.21.15.172:0/19443519 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f633407b8e0 con 0x7f63400ced80 2024-04-14T08:42:34.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.151+0000 7f63457fe640 1 -- 172.21.15.172:0/19443519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63400d0c70 con 0x7f63400ced80 2024-04-14T08:42:34.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.151+0000 7f63277fe640 1 -- 172.21.15.172:0/19443519 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+13490 (secure 0 0 0) 0x7f633407b1e0 con 0x7f63400ced80 2024-04-14T08:42:34.152 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:34.152 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":107,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:42:32.842062+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/4028911489":"2024-04-15T08:42:25.145813+0000","172.21.15.172:6835/4028911489":"2024-04-15T08:42:25.145813+0000","172.21.15.177:6835/2602337717":"2024-04-15T08:41:48.748747+0000","172.21.15.177:6834/2602337717":"2024-04-15T08:41:48.748747+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:42:34.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.153+0000 7f63457fe640 1 -- 172.21.15.172:0/19443519 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f632c06c920 msgr2=0x7f632c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.153+0000 7f63457fe640 1 --2- 172.21.15.172:0/19443519 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f632c06c920 0x7f632c06ede0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f63300070b0 tx=0x7f6330007040 comp rx=0 tx=0).stop 2024-04-14T08:42:34.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.153+0000 7f63457fe640 1 -- 172.21.15.172:0/19443519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63400ced80 msgr2=0x7f63400cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.153+0000 7f63457fe640 1 --2- 172.21.15.172:0/19443519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63400ced80 0x7f63400cba50 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f633402bee0 tx=0x7f63340046c0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.153+0000 7f63457fe640 1 -- 172.21.15.172:0/19443519 shutdown_connections 2024-04-14T08:42:34.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.153+0000 7f63457fe640 1 --2- 172.21.15.172:0/19443519 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f632c06c920 0x7f632c06ede0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.153+0000 7f63457fe640 1 --2- 172.21.15.172:0/19443519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f634015c710 0x7f63401542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.153+0000 7f63457fe640 1 --2- 172.21.15.172:0/19443519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63400ced80 0x7f63400cba50 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.153+0000 7f63457fe640 1 --2- 172.21.15.172:0/19443519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f63400cde10 0x7f63400c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.153+0000 7f63457fe640 1 -- 172.21.15.172:0/19443519 >> 172.21.15.172:0/19443519 conn(0x7f634008c000 msgr2=0x7f6340070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:34.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.153+0000 7f63457fe640 1 -- 172.21.15.172:0/19443519 shutdown_connections 2024-04-14T08:42:34.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.153+0000 7f63457fe640 1 -- 172.21.15.172:0/19443519 wait complete. 2024-04-14T08:42:34.165 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:42:34.165 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:42:34.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.269+0000 7f9776b1e640 1 Processor -- start 2024-04-14T08:42:34.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.269+0000 7f9776b1e640 1 -- start start 2024-04-14T08:42:34.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f9776b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97700d0890 0x7f97700b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f9776b1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f97700cfc70 0x7f97700af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f9776b1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97700aee60 con 0x7f977015c550 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f9776b1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f97700cd960 con 0x7f97700cfc70 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f9776b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f97700cdc40 con 0x7f97700d0890 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f97750bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f97700cfc70 0x7f97700af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f97750bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f97700cfc70 0x7f97700af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57156/0 (socket says 172.21.15.172:57156) 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f97750bb640 1 -- 172.21.15.172:0/3132635130 learned_addr learned my addr 172.21.15.172:0/3132635130 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f9767fff640 1 --2- 172.21.15.172:0/3132635130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97700d0890 0x7f97700b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f97750bb640 1 -- 172.21.15.172:0/3132635130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97700d0890 msgr2=0x7f97700b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f97750bb640 1 --2- 172.21.15.172:0/3132635130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97700d0890 0x7f97700b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f97750bb640 1 -- 172.21.15.172:0/3132635130 >> v1:172.21.15.177:6789/0 conn(0x7f977015c550 legacy=0x7f97700d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f97750bb640 1 -- 172.21.15.172:0/3132635130 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97700cdf20 con 0x7f97700cfc70 2024-04-14T08:42:34.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.270+0000 7f9767fff640 1 --2- 172.21.15.172:0/3132635130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97700d0890 0x7f97700b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:34.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.271+0000 7f97750bb640 1 --2- 172.21.15.172:0/3132635130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97700cfc70 0x7f97700af620 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9768009e50 tx=0x7f976802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad2c89f9d983b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:34.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.271+0000 7f97677fe640 1 -- 172.21.15.172:0/3132635130 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976802ca70 con 0x7f97700cfc70 2024-04-14T08:42:34.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.271+0000 7f97677fe640 1 -- 172.21.15.172:0/3132635130 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976802cc10 con 0x7f97700cfc70 2024-04-14T08:42:34.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.271+0000 7f97677fe640 1 -- 172.21.15.172:0/3132635130 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97680343f0 con 0x7f97700cfc70 2024-04-14T08:42:34.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.271+0000 7f9776b1e640 1 -- 172.21.15.172:0/3132635130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97700cfc70 msgr2=0x7f97700af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.271+0000 7f9776b1e640 1 --2- 172.21.15.172:0/3132635130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97700cfc70 0x7f97700af620 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9768009e50 tx=0x7f976802bed0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.272+0000 7f9776b1e640 1 -- 172.21.15.172:0/3132635130 shutdown_connections 2024-04-14T08:42:34.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.272+0000 7f9776b1e640 1 --2- 172.21.15.172:0/3132635130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97700cfc70 0x7f97700af620 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.272+0000 7f9776b1e640 1 --2- 172.21.15.172:0/3132635130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97700d0890 0x7f97700b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.272+0000 7f9776b1e640 1 -- 172.21.15.172:0/3132635130 >> 172.21.15.172:0/3132635130 conn(0x7f97700b3b00 msgr2=0x7f97700b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:34.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.272+0000 7f9776b1e640 1 -- 172.21.15.172:0/3132635130 shutdown_connections 2024-04-14T08:42:34.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.272+0000 7f9776b1e640 1 -- 172.21.15.172:0/3132635130 wait complete. 2024-04-14T08:42:34.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.273+0000 7f9776b1e640 1 Processor -- start 2024-04-14T08:42:34.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.273+0000 7f9776b1e640 1 -- start start 2024-04-14T08:42:34.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f9776b1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97700cfc70 0x7f97701f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f9776b1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97700d0890 0x7f977015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f9776b1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f977015c550 0x7f977015f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f9776b1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f97700afce0 con 0x7f97700cfc70 2024-04-14T08:42:34.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f9776b1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f97700afb60 con 0x7f977015c550 2024-04-14T08:42:34.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f9776b1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f97700afe60 con 0x7f97700d0890 2024-04-14T08:42:34.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f97748ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97700cfc70 0x7f97701f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f97748ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97700cfc70 0x7f97701f59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57166/0 (socket says 172.21.15.172:57166) 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f97748ba640 1 -- 172.21.15.172:0/2072057837 learned_addr learned my addr 172.21.15.172:0/2072057837 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f97748ba640 1 -- 172.21.15.172:0/2072057837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97700d0890 msgr2=0x7f977015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f97748ba640 1 --2- 172.21.15.172:0/2072057837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97700d0890 0x7f977015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f97748ba640 1 -- 172.21.15.172:0/2072057837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f977015c550 msgr2=0x7f977015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f97748ba640 1 --2- 172.21.15.172:0/2072057837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f977015c550 0x7f977015f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f97748ba640 1 -- 172.21.15.172:0/2072057837 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97701f2bd0 con 0x7f97700cfc70 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f97748ba640 1 --2- 172.21.15.172:0/2072057837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97700cfc70 0x7f97701f59b0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f976c00b3f0 tx=0x7f976c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f97657fa640 1 -- 172.21.15.172:0/2072057837 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c015020 con 0x7f97700cfc70 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f97657fa640 1 -- 172.21.15.172:0/2072057837 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976c011830 con 0x7f97700cfc70 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f97657fa640 1 -- 172.21.15.172:0/2072057837 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c011c20 con 0x7f97700cfc70 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f9776b1e640 1 -- 172.21.15.172:0/2072057837 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97701f1e10 con 0x7f97700cfc70 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.274+0000 7f9776b1e640 1 -- 172.21.15.172:0/2072057837 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97701f23d0 con 0x7f97700cfc70 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.275+0000 7f9776b1e640 1 -- 172.21.15.172:0/2072057837 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9770068590 con 0x7f97700cfc70 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.276+0000 7f97657fa640 1 -- 172.21.15.172:0/2072057837 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f976c018020 con 0x7f97700cfc70 2024-04-14T08:42:34.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.276+0000 7f97657fa640 1 --2- 172.21.15.172:0/2072057837 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f975c06c8f0 0x7f975c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.276+0000 7f97657fa640 1 -- 172.21.15.172:0/2072057837 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5421+0+0 (secure 0 0 0) 0x7f976c08b5e0 con 0x7f97700cfc70 2024-04-14T08:42:34.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.276+0000 7f9767fff640 1 --2- 172.21.15.172:0/2072057837 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f975c06c8f0 0x7f975c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.276+0000 7f9767fff640 1 --2- 172.21.15.172:0/2072057837 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f975c06c8f0 0x7f975c06edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9760006fd0 tx=0x7f9760008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:34.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.280+0000 7f97657fa640 1 -- 172.21.15.172:0/2072057837 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f976c01f030 con 0x7f97700cfc70 2024-04-14T08:42:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.452+0000 7f9776b1e640 1 -- 172.21.15.172:0/2072057837 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f977015bb60 con 0x7f97700cfc70 2024-04-14T08:42:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.453+0000 7f97657fa640 1 -- 172.21.15.172:0/2072057837 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7f976c0172c0 con 0x7f97700cfc70 2024-04-14T08:42:34.453 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:42:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.455+0000 7f9776b1e640 1 -- 172.21.15.172:0/2072057837 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f975c06c8f0 msgr2=0x7f975c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.455+0000 7f9776b1e640 1 --2- 172.21.15.172:0/2072057837 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f975c06c8f0 0x7f975c06edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9760006fd0 tx=0x7f9760008040 comp rx=0 tx=0).stop 2024-04-14T08:42:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.455+0000 7f9776b1e640 1 -- 172.21.15.172:0/2072057837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97700cfc70 msgr2=0x7f97701f59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.455+0000 7f9776b1e640 1 --2- 172.21.15.172:0/2072057837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97700cfc70 0x7f97701f59b0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f976c00b3f0 tx=0x7f976c00b8b0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.455+0000 7f9776b1e640 1 -- 172.21.15.172:0/2072057837 shutdown_connections 2024-04-14T08:42:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.455+0000 7f9776b1e640 1 --2- 172.21.15.172:0/2072057837 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f975c06c8f0 0x7f975c06edb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.455+0000 7f9776b1e640 1 --2- 172.21.15.172:0/2072057837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f977015c550 0x7f977015f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.455+0000 7f9776b1e640 1 --2- 172.21.15.172:0/2072057837 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97700d0890 0x7f977015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.455+0000 7f9776b1e640 1 --2- 172.21.15.172:0/2072057837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97700cfc70 0x7f97701f59b0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.455+0000 7f9776b1e640 1 -- 172.21.15.172:0/2072057837 >> 172.21.15.172:0/2072057837 conn(0x7f97700b3b00 msgr2=0x7f97700b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.455+0000 7f9776b1e640 1 -- 172.21.15.172:0/2072057837 shutdown_connections 2024-04-14T08:42:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.455+0000 7f9776b1e640 1 -- 172.21.15.172:0/2072057837 wait complete. 2024-04-14T08:42:34.466 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-14T08:42:34.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.567+0000 7fe4d7fa3640 1 Processor -- start 2024-04-14T08:42:34.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.567+0000 7fe4d7fa3640 1 -- start start 2024-04-14T08:42:34.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d7fa3640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe4d00b1df0 0x7fe4d00b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d7fa3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4d00b4040 0x7fe4d00b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d7fa3640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d00b49f0 con 0x7fe4d00b21d0 2024-04-14T08:42:34.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d7fa3640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d015c770 con 0x7fe4d00b1df0 2024-04-14T08:42:34.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d7fa3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d00b3430 con 0x7fe4d00b4040 2024-04-14T08:42:34.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d5d3f640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe4d00b1df0 0x7fe4d00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d553e640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fe4d00b21d0 0x7fe4d015c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:55092/0 (socket says 172.21.15.172:55092) 2024-04-14T08:42:34.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d553e640 1 -- 172.21.15.172:0/3717973806 learned_addr learned my addr 172.21.15.172:0/3717973806 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:34.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d6540640 1 --2- 172.21.15.172:0/3717973806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4d00b4040 0x7fe4d00b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d5d3f640 1 -- 172.21.15.172:0/3717973806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4d00b4040 msgr2=0x7fe4d00b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:34.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d5d3f640 1 --2- 172.21.15.172:0/3717973806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4d00b4040 0x7fe4d00b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d5d3f640 1 -- 172.21.15.172:0/3717973806 >> v1:172.21.15.177:6789/0 conn(0x7fe4d00b21d0 legacy=0x7fe4d015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.568+0000 7fe4d5d3f640 1 -- 172.21.15.172:0/3717973806 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d00b3700 con 0x7fe4d00b1df0 2024-04-14T08:42:34.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.569+0000 7fe4d5d3f640 1 --2- 172.21.15.172:0/3717973806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d00b1df0 0x7fe4d00b2ef0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0097a0 tx=0x7fe4cc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=726217d8c290ad4d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:34.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.569+0000 7fe4d4d3d640 1 -- 172.21.15.172:0/3717973806 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc035020 con 0x7fe4d00b1df0 2024-04-14T08:42:34.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.569+0000 7fe4d4d3d640 1 -- 172.21.15.172:0/3717973806 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc031830 con 0x7fe4d00b1df0 2024-04-14T08:42:34.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.569+0000 7fe4d4d3d640 1 -- 172.21.15.172:0/3717973806 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc031c00 con 0x7fe4d00b1df0 2024-04-14T08:42:34.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.570+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/3717973806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d00b1df0 msgr2=0x7fe4d00b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.570+0000 7fe4d7fa3640 1 --2- 172.21.15.172:0/3717973806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d00b1df0 0x7fe4d00b2ef0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0097a0 tx=0x7fe4cc02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.570+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/3717973806 shutdown_connections 2024-04-14T08:42:34.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.570+0000 7fe4d7fa3640 1 --2- 172.21.15.172:0/3717973806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4d00b4040 0x7fe4d00b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.570+0000 7fe4d7fa3640 1 --2- 172.21.15.172:0/3717973806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d00b1df0 0x7fe4d00b2ef0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.570+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/3717973806 >> 172.21.15.172:0/3717973806 conn(0x7fe4d006ec30 msgr2=0x7fe4d006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:34.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.570+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/3717973806 shutdown_connections 2024-04-14T08:42:34.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.570+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/3717973806 wait complete. 2024-04-14T08:42:34.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.571+0000 7fe4d7fa3640 1 Processor -- start 2024-04-14T08:42:34.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.571+0000 7fe4d7fa3640 1 -- start start 2024-04-14T08:42:34.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d7fa3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4d00b1df0 0x7fe4d0142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d7fa3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d00b21d0 0x7fe4d0139ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d7fa3640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4d00b4040 0x7fe4d013ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d7fa3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d015bb40 con 0x7fe4d00b1df0 2024-04-14T08:42:34.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d7fa3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d00b3cc0 con 0x7fe4d00b21d0 2024-04-14T08:42:34.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d7fa3640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d015b7f0 con 0x7fe4d00b4040 2024-04-14T08:42:34.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d5d3f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4d00b1df0 0x7fe4d0142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d5d3f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4d00b1df0 0x7fe4d0142900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57182/0 (socket says 172.21.15.172:57182) 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d5d3f640 1 -- 172.21.15.172:0/609166861 learned_addr learned my addr 172.21.15.172:0/609166861 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d6540640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4d00b4040 0x7fe4d013ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d5d3f640 1 -- 172.21.15.172:0/609166861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4d00b4040 msgr2=0x7fe4d013ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d5d3f640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4d00b4040 0x7fe4d013ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d553e640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d00b21d0 0x7fe4d0139ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d5d3f640 1 -- 172.21.15.172:0/609166861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d00b21d0 msgr2=0x7fe4d0139ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d5d3f640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d00b21d0 0x7fe4d0139ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d5d3f640 1 -- 172.21.15.172:0/609166861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d013a010 con 0x7fe4d00b1df0 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d553e640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d00b21d0 0x7fe4d0139ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.572+0000 7fe4d5d3f640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4d00b1df0 0x7fe4d0142900 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc02bee0 tx=0x7fe4cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.573+0000 7fe4c67fc640 1 -- 172.21.15.172:0/609166861 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc035040 con 0x7fe4d00b1df0 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.573+0000 7fe4c67fc640 1 -- 172.21.15.172:0/609166861 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc03b920 con 0x7fe4d00b1df0 2024-04-14T08:42:34.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.573+0000 7fe4c67fc640 1 -- 172.21.15.172:0/609166861 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc03bd10 con 0x7fe4d00b1df0 2024-04-14T08:42:34.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.573+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/609166861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d01f42f0 con 0x7fe4d00b1df0 2024-04-14T08:42:34.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.573+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/609166861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d01f48b0 con 0x7fe4d00b1df0 2024-04-14T08:42:34.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.574+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/609166861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d015c9f0 con 0x7fe4d00b1df0 2024-04-14T08:42:34.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.574+0000 7fe4c67fc640 1 -- 172.21.15.172:0/609166861 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4cc038020 con 0x7fe4d00b1df0 2024-04-14T08:42:34.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.575+0000 7fe4c67fc640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe4b406c920 0x7fe4b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.575+0000 7fe4d553e640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe4b406c920 0x7fe4b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.575+0000 7fe4c67fc640 1 -- 172.21.15.172:0/609166861 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5421+0+0 (secure 0 0 0) 0x7fe4cc0adaf0 con 0x7fe4d00b1df0 2024-04-14T08:42:34.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.575+0000 7fe4d553e640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe4b406c920 0x7fe4b406ede0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00b2c00 tx=0x7fe4c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:34.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.579+0000 7fe4c67fc640 1 -- 172.21.15.172:0/609166861 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe4cc07c050 con 0x7fe4d00b1df0 2024-04-14T08:42:34.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.822+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/609166861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7fe4d0068520 con 0x7fe4d00b1df0 2024-04-14T08:42:34.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.849+0000 7fe4c67fc640 1 -- 172.21.15.172:0/609166861 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v452) ==== 147+0+0 (secure 0 0 0) 0x7fe4cc031800 con 0x7fe4d00b1df0 2024-04-14T08:42:34.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.851+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/609166861 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe4b406c920 msgr2=0x7fe4b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.851+0000 7fe4d7fa3640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe4b406c920 0x7fe4b406ede0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00b2c00 tx=0x7fe4c0009040 comp rx=0 tx=0).stop 2024-04-14T08:42:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.851+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/609166861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4d00b1df0 msgr2=0x7fe4d0142900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.851+0000 7fe4d7fa3640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4d00b1df0 0x7fe4d0142900 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc02bee0 tx=0x7fe4cc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.851+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/609166861 shutdown_connections 2024-04-14T08:42:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.851+0000 7fe4d7fa3640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe4b406c920 0x7fe4b406ede0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.851+0000 7fe4d7fa3640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4d00b4040 0x7fe4d013ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.851+0000 7fe4d7fa3640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4d00b21d0 0x7fe4d0139ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.851+0000 7fe4d7fa3640 1 --2- 172.21.15.172:0/609166861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4d00b1df0 0x7fe4d0142900 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.851+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/609166861 >> 172.21.15.172:0/609166861 conn(0x7fe4d006ec30 msgr2=0x7fe4d0159410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.851+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/609166861 shutdown_connections 2024-04-14T08:42:34.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.851+0000 7fe4d7fa3640 1 -- 172.21.15.172:0/609166861 wait complete. 2024-04-14T08:42:34.862 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-14T08:42:34.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.965+0000 7f26efe1c640 1 Processor -- start 2024-04-14T08:42:34.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.966+0000 7f26efe1c640 1 -- start start 2024-04-14T08:42:34.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.966+0000 7f26efe1c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26e80d0cc0 0x7f26e80d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.966+0000 7f26efe1c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e80d49c0 0x7f26e80d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.966+0000 7f26efe1c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26e80d15e0 con 0x7f26e8160f30 2024-04-14T08:42:34.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.966+0000 7f26efe1c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f26e80ceec0 con 0x7f26e80d0cc0 2024-04-14T08:42:34.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.966+0000 7f26efe1c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f26e80cfd00 con 0x7f26e80d49c0 2024-04-14T08:42:34.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.967+0000 7f26edbb8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26e80d0cc0 0x7f26e80d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.967+0000 7f26edbb8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26e80d0cc0 0x7f26e80d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57184/0 (socket says 172.21.15.172:57184) 2024-04-14T08:42:34.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.967+0000 7f26edbb8640 1 -- 172.21.15.172:0/583469640 learned_addr learned my addr 172.21.15.172:0/583469640 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:34.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.967+0000 7f26ed3b7640 1 --2- 172.21.15.172:0/583469640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e80d49c0 0x7f26e80d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.967+0000 7f26edbb8640 1 -- 172.21.15.172:0/583469640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e80d49c0 msgr2=0x7f26e80d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:34.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.967+0000 7f26edbb8640 1 --2- 172.21.15.172:0/583469640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e80d49c0 0x7f26e80d4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.967+0000 7f26edbb8640 1 -- 172.21.15.172:0/583469640 >> v1:172.21.15.177:6789/0 conn(0x7f26e8160f30 legacy=0x7f26e81613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:34.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.967+0000 7f26edbb8640 1 -- 172.21.15.172:0/583469640 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e80cf0f0 con 0x7f26e80d0cc0 2024-04-14T08:42:34.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.968+0000 7f26edbb8640 1 --2- 172.21.15.172:0/583469640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26e80d0cc0 0x7f26e80d10a0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009e50 tx=0x7f26e402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1735c56ab9912b64 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:34.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.968+0000 7f26ecbb6640 1 -- 172.21.15.172:0/583469640 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e402ca70 con 0x7f26e80d0cc0 2024-04-14T08:42:34.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.968+0000 7f26ecbb6640 1 -- 172.21.15.172:0/583469640 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e402cc10 con 0x7f26e80d0cc0 2024-04-14T08:42:34.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.968+0000 7f26ecbb6640 1 -- 172.21.15.172:0/583469640 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e40343f0 con 0x7f26e80d0cc0 2024-04-14T08:42:34.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.968+0000 7f26efe1c640 1 -- 172.21.15.172:0/583469640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26e80d0cc0 msgr2=0x7f26e80d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.968+0000 7f26efe1c640 1 --2- 172.21.15.172:0/583469640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26e80d0cc0 0x7f26e80d10a0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009e50 tx=0x7f26e402bed0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.968+0000 7f26efe1c640 1 -- 172.21.15.172:0/583469640 shutdown_connections 2024-04-14T08:42:34.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.968+0000 7f26efe1c640 1 --2- 172.21.15.172:0/583469640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e80d49c0 0x7f26e80d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.968+0000 7f26efe1c640 1 --2- 172.21.15.172:0/583469640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26e80d0cc0 0x7f26e80d10a0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.968+0000 7f26efe1c640 1 -- 172.21.15.172:0/583469640 >> 172.21.15.172:0/583469640 conn(0x7f26e809d580 msgr2=0x7f26e80d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:34.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.969+0000 7f26efe1c640 1 -- 172.21.15.172:0/583469640 shutdown_connections 2024-04-14T08:42:34.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.969+0000 7f26efe1c640 1 -- 172.21.15.172:0/583469640 wait complete. 2024-04-14T08:42:34.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.970+0000 7f26efe1c640 1 Processor -- start 2024-04-14T08:42:34.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.970+0000 7f26efe1c640 1 -- start start 2024-04-14T08:42:34.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.970+0000 7f26efe1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26e80d0cc0 0x7f26e81f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.970+0000 7f26efe1c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e80d49c0 0x7f26e81f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.970+0000 7f26efe1c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e8160f30 0x7f26e81f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.970+0000 7f26efe1c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26e80cf990 con 0x7f26e80d0cc0 2024-04-14T08:42:34.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.970+0000 7f26efe1c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f26e80cf6b0 con 0x7f26e8160f30 2024-04-14T08:42:34.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.970+0000 7f26efe1c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f26e80d32b0 con 0x7f26e80d49c0 2024-04-14T08:42:34.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.970+0000 7f26edbb8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26e80d0cc0 0x7f26e81f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.970+0000 7f26edbb8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26e80d0cc0 0x7f26e81f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57198/0 (socket says 172.21.15.172:57198) 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.970+0000 7f26edbb8640 1 -- 172.21.15.172:0/1074495296 learned_addr learned my addr 172.21.15.172:0/1074495296 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26ed3b7640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e80d49c0 0x7f26e81f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26ee3b9640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e8160f30 0x7f26e81f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26edbb8640 1 -- 172.21.15.172:0/1074495296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e80d49c0 msgr2=0x7f26e81f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26edbb8640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e80d49c0 0x7f26e81f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26edbb8640 1 -- 172.21.15.172:0/1074495296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e8160f30 msgr2=0x7f26e81f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26edbb8640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e8160f30 0x7f26e81f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26edbb8640 1 -- 172.21.15.172:0/1074495296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e81f5200 con 0x7f26e80d0cc0 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26ee3b9640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e8160f30 0x7f26e81f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26ed3b7640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e80d49c0 0x7f26e81f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26edbb8640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26e80d0cc0 0x7f26e81f1e80 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009e50 tx=0x7f26e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26de7fc640 1 -- 172.21.15.172:0/1074495296 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4043020 con 0x7f26e80d0cc0 2024-04-14T08:42:34.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26de7fc640 1 -- 172.21.15.172:0/1074495296 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e403e450 con 0x7f26e80d0cc0 2024-04-14T08:42:34.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26efe1c640 1 -- 172.21.15.172:0/1074495296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e81f57e0 con 0x7f26e80d0cc0 2024-04-14T08:42:34.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26de7fc640 1 -- 172.21.15.172:0/1074495296 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e403e840 con 0x7f26e80d0cc0 2024-04-14T08:42:34.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.971+0000 7f26efe1c640 1 -- 172.21.15.172:0/1074495296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e81f5da0 con 0x7f26e80d0cc0 2024-04-14T08:42:34.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.972+0000 7f26efe1c640 1 -- 172.21.15.172:0/1074495296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e8075490 con 0x7f26e80d0cc0 2024-04-14T08:42:34.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.973+0000 7f26de7fc640 1 -- 172.21.15.172:0/1074495296 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26e4046020 con 0x7f26e80d0cc0 2024-04-14T08:42:34.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.973+0000 7f26de7fc640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f26cc06c8f0 0x7f26cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:34.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.973+0000 7f26de7fc640 1 -- 172.21.15.172:0/1074495296 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5421+0+0 (secure 0 0 0) 0x7f26e40ae310 con 0x7f26e80d0cc0 2024-04-14T08:42:34.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.973+0000 7f26ed3b7640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f26cc06c8f0 0x7f26cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:34.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.973+0000 7f26ed3b7640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f26cc06c8f0 0x7f26cc06edb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f26e80a1470 tx=0x7f26d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:34.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:34.977+0000 7f26de7fc640 1 -- 172.21.15.172:0/1074495296 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f26e404d030 con 0x7f26e80d0cc0 2024-04-14T08:42:35.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.152+0000 7f26efe1c640 1 -- 172.21.15.172:0/1074495296 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f26e80d0aa0 con 0x7f26e80d0cc0 2024-04-14T08:42:35.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.155+0000 7f26de7fc640 1 -- 172.21.15.172:0/1074495296 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f26e81f5200 con 0x7f26e80d0cc0 2024-04-14T08:42:35.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.157+0000 7f26efe1c640 1 -- 172.21.15.172:0/1074495296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f26cc06c8f0 msgr2=0x7f26cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:35.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.157+0000 7f26efe1c640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f26cc06c8f0 0x7f26cc06edb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f26e80a1470 tx=0x7f26d8008040 comp rx=0 tx=0).stop 2024-04-14T08:42:35.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.157+0000 7f26efe1c640 1 -- 172.21.15.172:0/1074495296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26e80d0cc0 msgr2=0x7f26e81f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:35.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.157+0000 7f26efe1c640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26e80d0cc0 0x7f26e81f1e80 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009e50 tx=0x7f26e4004a40 comp rx=0 tx=0).stop 2024-04-14T08:42:35.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.157+0000 7f26efe1c640 1 -- 172.21.15.172:0/1074495296 shutdown_connections 2024-04-14T08:42:35.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.157+0000 7f26efe1c640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f26cc06c8f0 0x7f26cc06edb0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.157+0000 7f26efe1c640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e8160f30 0x7f26e81f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.157+0000 7f26efe1c640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e80d49c0 0x7f26e81f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.157+0000 7f26efe1c640 1 --2- 172.21.15.172:0/1074495296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26e80d0cc0 0x7f26e81f1e80 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.157+0000 7f26efe1c640 1 -- 172.21.15.172:0/1074495296 >> 172.21.15.172:0/1074495296 conn(0x7f26e809d580 msgr2=0x7f26e8160790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:35.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.157+0000 7f26efe1c640 1 -- 172.21.15.172:0/1074495296 shutdown_connections 2024-04-14T08:42:35.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.157+0000 7f26efe1c640 1 -- 172.21.15.172:0/1074495296 wait complete. 2024-04-14T08:42:35.169 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:42:35.170 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:42:35.206 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:42:35.258 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:42:35.259 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:42:35.259 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:35.274 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:42:35.274 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:42:35.274 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:35.292 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:42:35.292 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:42:35.292 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:35.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:42:35.329 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:42:35.329 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:35.351 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:42:35.351 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-14T08:42:35.351 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:42:35.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.469+0000 7ff350469640 1 Processor -- start 2024-04-14T08:42:35.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.469+0000 7ff350469640 1 -- start start 2024-04-14T08:42:35.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.470+0000 7ff350469640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff3480d2830 0x7ff3480d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.470+0000 7ff350469640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff34815e890 0x7ff3480d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.470+0000 7ff350469640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff3480cedd0 0x7ff3481621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.470+0000 7ff350469640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff3480cff30 con 0x7ff3480cedd0 2024-04-14T08:42:35.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.470+0000 7ff350469640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff3480d2350 con 0x7ff3480d2830 2024-04-14T08:42:35.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.470+0000 7ff350469640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff3480d3150 con 0x7ff34815e890 2024-04-14T08:42:35.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.470+0000 7ff34ea06640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff3480cedd0 0x7ff3481621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.471+0000 7ff34ea06640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff3480cedd0 0x7ff3481621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57210/0 (socket says 172.21.15.172:57210) 2024-04-14T08:42:35.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.471+0000 7ff34ea06640 1 -- 172.21.15.172:0/3853025334 learned_addr learned my addr 172.21.15.172:0/3853025334 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:35.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.471+0000 7ff34e205640 1 --2- 172.21.15.172:0/3853025334 >> v2:172.21.15.177:3300/0 conn(0x7ff3480d2830 0x7ff3480d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.471+0000 7ff34da04640 1 --2- 172.21.15.172:0/3853025334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff34815e890 0x7ff3480d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.471+0000 7ff34ea06640 1 -- 172.21.15.172:0/3853025334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff34815e890 msgr2=0x7ff3480d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:35.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.471+0000 7ff34ea06640 1 --2- 172.21.15.172:0/3853025334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff34815e890 0x7ff3480d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.471+0000 7ff34ea06640 1 -- 172.21.15.172:0/3853025334 >> v2:172.21.15.177:3300/0 conn(0x7ff3480d2830 msgr2=0x7ff3480d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:35.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.471+0000 7ff34ea06640 1 --2- 172.21.15.172:0/3853025334 >> v2:172.21.15.177:3300/0 conn(0x7ff3480d2830 0x7ff3480d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.471+0000 7ff34ea06640 1 -- 172.21.15.172:0/3853025334 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3480cf320 con 0x7ff3480cedd0 2024-04-14T08:42:35.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.472+0000 7ff34ea06640 1 --2- 172.21.15.172:0/3853025334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3480cedd0 0x7ff3481621a0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff34808c830 tx=0x7ff33802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40e98eb447c2095 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:35.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.472+0000 7ff34d203640 1 -- 172.21.15.172:0/3853025334 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338034070 con 0x7ff3480cedd0 2024-04-14T08:42:35.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.472+0000 7ff34d203640 1 -- 172.21.15.172:0/3853025334 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff338030860 con 0x7ff3480cedd0 2024-04-14T08:42:35.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.472+0000 7ff34d203640 1 -- 172.21.15.172:0/3853025334 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338030c30 con 0x7ff3480cedd0 2024-04-14T08:42:35.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.472+0000 7ff350469640 1 -- 172.21.15.172:0/3853025334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3480cedd0 msgr2=0x7ff3481621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.472+0000 7ff350469640 1 --2- 172.21.15.172:0/3853025334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3480cedd0 0x7ff3481621a0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff34808c830 tx=0x7ff33802b9a0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.472+0000 7ff350469640 1 -- 172.21.15.172:0/3853025334 shutdown_connections 2024-04-14T08:42:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.472+0000 7ff350469640 1 --2- 172.21.15.172:0/3853025334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3480cedd0 0x7ff3481621a0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.472+0000 7ff350469640 1 --2- 172.21.15.172:0/3853025334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff34815e890 0x7ff3480d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.472+0000 7ff350469640 1 --2- 172.21.15.172:0/3853025334 >> v2:172.21.15.177:3300/0 conn(0x7ff3480d2830 0x7ff3480d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.472+0000 7ff350469640 1 -- 172.21.15.172:0/3853025334 >> 172.21.15.172:0/3853025334 conn(0x7ff34809d590 msgr2=0x7ff3480d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.472+0000 7ff350469640 1 -- 172.21.15.172:0/3853025334 shutdown_connections 2024-04-14T08:42:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.473+0000 7ff350469640 1 -- 172.21.15.172:0/3853025334 wait complete. 2024-04-14T08:42:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.474+0000 7ff350469640 1 Processor -- start 2024-04-14T08:42:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.474+0000 7ff350469640 1 -- start start 2024-04-14T08:42:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.474+0000 7ff350469640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3480cedd0 0x7ff348152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.474+0000 7ff350469640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3480d2830 0x7ff3480c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.474+0000 7ff350469640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff34815e890 0x7ff34806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.474+0000 7ff350469640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff3480cfbc0 con 0x7ff34815e890 2024-04-14T08:42:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.474+0000 7ff350469640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff3480cf8e0 con 0x7ff3480cedd0 2024-04-14T08:42:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.474+0000 7ff350469640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff3480d1840 con 0x7ff3480d2830 2024-04-14T08:42:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.474+0000 7ff34ea06640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff34815e890 0x7ff34806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.474+0000 7ff34ea06640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff34815e890 0x7ff34806d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57220/0 (socket says 172.21.15.172:57220) 2024-04-14T08:42:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.474+0000 7ff34ea06640 1 -- 172.21.15.172:0/1408490612 learned_addr learned my addr 172.21.15.172:0/1408490612 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff34e205640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3480cedd0 0x7ff348152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff34da04640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3480d2830 0x7ff3480c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff34ea06640 1 -- 172.21.15.172:0/1408490612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3480d2830 msgr2=0x7ff3480c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff34ea06640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3480d2830 0x7ff3480c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff34ea06640 1 -- 172.21.15.172:0/1408490612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3480cedd0 msgr2=0x7ff348152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff34ea06640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3480cedd0 0x7ff348152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff34ea06640 1 -- 172.21.15.172:0/1408490612 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3480c87e0 con 0x7ff34815e890 2024-04-14T08:42:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff34ea06640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff34815e890 0x7ff34806d360 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7ff33802b9e0 tx=0x7ff3380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff346ffd640 1 -- 172.21.15.172:0/1408490612 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338034040 con 0x7ff34815e890 2024-04-14T08:42:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff346ffd640 1 -- 172.21.15.172:0/1408490612 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff338039d90 con 0x7ff34815e890 2024-04-14T08:42:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff350469640 1 -- 172.21.15.172:0/1408490612 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3480ca210 con 0x7ff34815e890 2024-04-14T08:42:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff346ffd640 1 -- 172.21.15.172:0/1408490612 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338043630 con 0x7ff34815e890 2024-04-14T08:42:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.475+0000 7ff350469640 1 -- 172.21.15.172:0/1408490612 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3480ca7d0 con 0x7ff34815e890 2024-04-14T08:42:35.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.476+0000 7ff350469640 1 -- 172.21.15.172:0/1408490612 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff348068520 con 0x7ff34815e890 2024-04-14T08:42:35.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.477+0000 7ff346ffd640 1 -- 172.21.15.172:0/1408490612 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff338037070 con 0x7ff34815e890 2024-04-14T08:42:35.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.477+0000 7ff346ffd640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff33406c920 0x7ff33406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.477+0000 7ff346ffd640 1 -- 172.21.15.172:0/1408490612 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5421+0+0 (secure 0 0 0) 0x7ff3380ada40 con 0x7ff34815e890 2024-04-14T08:42:35.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.477+0000 7ff34e205640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff33406c920 0x7ff33406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.478+0000 7ff34e205640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff33406c920 0x7ff33406ede0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff340011340 tx=0x7ff34000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:35.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.482+0000 7ff346ffd640 1 -- 172.21.15.172:0/1408490612 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff33803e070 con 0x7ff34815e890 2024-04-14T08:42:35.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.691+0000 7ff350469640 1 -- 172.21.15.172:0/1408490612 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3480ca9a0 con 0x7ff34815e890 2024-04-14T08:42:35.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.692+0000 7ff346ffd640 1 -- 172.21.15.172:0/1408490612 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+3881 (secure 0 0 0) 0x7ff3380ade20 con 0x7ff34815e890 2024-04-14T08:42:35.692 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:35.692 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":84,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/4063885191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":4063885191},{"type":"v1","addr":"172.21.15.177:6833","nonce":4063885191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8005,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3289017090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3289017090},{"type":"v1","addr":"172.21.15.177:6835","nonce":3289017090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":8379,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/1906476761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1906476761},{"type":"v1","addr":"172.21.15.172:6836","nonce":1906476761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8880,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/1467868914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":1467868914},{"type":"v1","addr":"172.21.15.172:6837","nonce":1467868914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":82}],"filesystems":[]} 2024-04-14T08:42:35.692 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 84 2024-04-14T08:42:35.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.693+0000 7ff350469640 1 -- 172.21.15.172:0/1408490612 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff33406c920 msgr2=0x7ff33406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:35.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.693+0000 7ff350469640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff33406c920 0x7ff33406ede0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff340011340 tx=0x7ff34000b040 comp rx=0 tx=0).stop 2024-04-14T08:42:35.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.693+0000 7ff350469640 1 -- 172.21.15.172:0/1408490612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff34815e890 msgr2=0x7ff34806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:35.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.693+0000 7ff350469640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff34815e890 0x7ff34806d360 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7ff33802b9e0 tx=0x7ff3380046c0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.693+0000 7ff350469640 1 -- 172.21.15.172:0/1408490612 shutdown_connections 2024-04-14T08:42:35.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.694+0000 7ff350469640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff33406c920 0x7ff33406ede0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.694+0000 7ff350469640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff34815e890 0x7ff34806d360 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.694+0000 7ff350469640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3480d2830 0x7ff3480c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.694+0000 7ff350469640 1 --2- 172.21.15.172:0/1408490612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3480cedd0 0x7ff348152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.694+0000 7ff350469640 1 -- 172.21.15.172:0/1408490612 >> 172.21.15.172:0/1408490612 conn(0x7ff34809d590 msgr2=0x7ff3480b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:35.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.694+0000 7ff350469640 1 -- 172.21.15.172:0/1408490612 shutdown_connections 2024-04-14T08:42:35.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.694+0000 7ff350469640 1 -- 172.21.15.172:0/1408490612 wait complete. 2024-04-14T08:42:35.705 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-14T08:42:35.705 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-14T08:42:35.705 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-14T08:42:35.705 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:42:35.706 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-14T08:42:35.706 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-14T08:42:35.706 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-14T08:42:35.706 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:42:35.706 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-14T08:42:35.706 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-14T08:42:35.707 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-14T08:42:35.707 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:42:35.707 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-14T08:42:35.707 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-14T08:42:35.707 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-14T08:42:35.707 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:42:35.715 INFO:tasks.ceph.mds.a:Stopped 2024-04-14T08:42:35.715 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-14T08:42:35.716 INFO:tasks.ceph.mds.c:Stopped 2024-04-14T08:42:35.716 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-14T08:42:35.718 INFO:tasks.ceph.mds.b:Stopped 2024-04-14T08:42:35.718 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-14T08:42:35.719 INFO:tasks.ceph.mds.d:Stopped 2024-04-14T08:42:35.719 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-14T08:42:35.757 INFO:tasks.ceph.mds.a:Started 2024-04-14T08:42:35.758 DEBUG:teuthology.parallel:result is None 2024-04-14T08:42:35.758 INFO:tasks.ceph.mds.c:Started 2024-04-14T08:42:35.759 DEBUG:teuthology.parallel:result is None 2024-04-14T08:42:35.761 INFO:tasks.ceph.mds.b:Started 2024-04-14T08:42:35.762 DEBUG:teuthology.parallel:result is None 2024-04-14T08:42:35.762 INFO:tasks.ceph.mds.d:Started 2024-04-14T08:42:35.762 DEBUG:teuthology.parallel:result is None 2024-04-14T08:42:35.763 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-14T08:42:35.835 INFO:tasks.ceph.mds.a.smithi172.stdout:starting mds.a at 2024-04-14T08:42:35.835 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:42:35.844 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2024-04-14T08:42:35.844 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-04-14T08:42:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.870+0000 7f10d395e640 1 Processor -- start 2024-04-14T08:42:35.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.871+0000 7f10d395e640 1 -- start start 2024-04-14T08:42:35.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.871+0000 7f10d395e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f10cc15e4b0 0x7f10cc0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.871+0000 7f10d395e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f10cc0d1180 0x7f10cc0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.871+0000 7f10d395e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc15e890 con 0x7f10cc0cfd70 2024-04-14T08:42:35.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.871+0000 7f10d395e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f10cc0d1f30 con 0x7f10cc15e4b0 2024-04-14T08:42:35.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.871+0000 7f10d395e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f10cc0d2210 con 0x7f10cc0d1180 2024-04-14T08:42:35.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.871+0000 7f10d16fa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f10cc15e4b0 0x7f10cc0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.871+0000 7f10d0ef9640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f10cc0cfd70 0x7f10cc0d0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56180/0 (socket says 172.21.15.172:56180) 2024-04-14T08:42:35.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.871+0000 7f10d0ef9640 1 -- 172.21.15.172:0/1398196575 learned_addr learned my addr 172.21.15.172:0/1398196575 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:35.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.871+0000 7f10d16fa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f10cc15e4b0 0x7f10cc0cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57260/0 (socket says 172.21.15.172:57260) 2024-04-14T08:42:35.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.872+0000 7f10d1efb640 1 --2- 172.21.15.172:0/1398196575 >> v2:172.21.15.177:3300/0 conn(0x7f10cc0d1180 0x7f10cc0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.872+0000 7f10c3fff640 1 -- 172.21.15.172:0/1398196575 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4146639646 0 0) 0x7f10cc15e890 con 0x7f10cc0cfd70 2024-04-14T08:42:35.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.872+0000 7f10c3fff640 1 -- 172.21.15.172:0/1398196575 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10b8003520 con 0x7f10cc0cfd70 2024-04-14T08:42:35.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.872+0000 7f10d16fa640 1 -- 172.21.15.172:0/1398196575 >> v2:172.21.15.177:3300/0 conn(0x7f10cc0d1180 msgr2=0x7f10cc0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:35.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.872+0000 7f10d16fa640 1 --2- 172.21.15.172:0/1398196575 >> v2:172.21.15.177:3300/0 conn(0x7f10cc0d1180 0x7f10cc0b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.872+0000 7f10d16fa640 1 -- 172.21.15.172:0/1398196575 >> v1:172.21.15.172:6789/0 conn(0x7f10cc0cfd70 legacy=0x7f10cc0d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:35.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.872+0000 7f10d16fa640 1 -- 172.21.15.172:0/1398196575 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc0d24f0 con 0x7f10cc15e4b0 2024-04-14T08:42:35.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.873+0000 7f10d16fa640 1 --2- 172.21.15.172:0/1398196575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10cc15e4b0 0x7f10cc0cf830 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f10cc09da10 tx=0x7f10c802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92612dc80a86f581 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:35.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.873+0000 7f10c3fff640 1 -- 172.21.15.172:0/1398196575 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8039070 con 0x7f10cc15e4b0 2024-04-14T08:42:35.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.873+0000 7f10c3fff640 1 -- 172.21.15.172:0/1398196575 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c802cbf0 con 0x7f10cc15e4b0 2024-04-14T08:42:35.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.873+0000 7f10c3fff640 1 -- 172.21.15.172:0/1398196575 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8034440 con 0x7f10cc15e4b0 2024-04-14T08:42:35.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.873+0000 7f10d395e640 1 -- 172.21.15.172:0/1398196575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10cc15e4b0 msgr2=0x7f10cc0cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.873+0000 7f10d395e640 1 --2- 172.21.15.172:0/1398196575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10cc15e4b0 0x7f10cc0cf830 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f10cc09da10 tx=0x7f10c802be40 comp rx=0 tx=0).stop 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.873+0000 7f10d395e640 1 -- 172.21.15.172:0/1398196575 shutdown_connections 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.873+0000 7f10d395e640 1 --2- 172.21.15.172:0/1398196575 >> v2:172.21.15.177:3300/0 conn(0x7f10cc0d1180 0x7f10cc0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.873+0000 7f10d395e640 1 --2- 172.21.15.172:0/1398196575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10cc15e4b0 0x7f10cc0cf830 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.873+0000 7f10d395e640 1 -- 172.21.15.172:0/1398196575 >> 172.21.15.172:0/1398196575 conn(0x7f10cc09d570 msgr2=0x7f10cc0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.873+0000 7f10d395e640 1 -- 172.21.15.172:0/1398196575 shutdown_connections 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.873+0000 7f10d395e640 1 -- 172.21.15.172:0/1398196575 wait complete. 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.875+0000 7f10d395e640 1 Processor -- start 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.875+0000 7f10d395e640 1 -- start start 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.875+0000 7f10d395e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10cc0cfd70 0x7f10cc15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.875+0000 7f10d395e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10cc0d1180 0x7f10cc15d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.875+0000 7f10d395e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10cc15e4b0 0x7f10cc0c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.875+0000 7f10d395e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0d2fd0 con 0x7f10cc0cfd70 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.875+0000 7f10d395e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0d2ab0 con 0x7f10cc0d1180 2024-04-14T08:42:35.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.875+0000 7f10d395e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc0b97e0 con 0x7f10cc15e4b0 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.875+0000 7f10d16fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10cc0cfd70 0x7f10cc15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.875+0000 7f10d16fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10cc0cfd70 0x7f10cc15cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57268/0 (socket says 172.21.15.172:57268) 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.875+0000 7f10d16fa640 1 -- 172.21.15.172:0/4270944941 learned_addr learned my addr 172.21.15.172:0/4270944941 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10d1efb640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10cc15e4b0 0x7f10cc0c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10d0ef9640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10cc0d1180 0x7f10cc15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10d16fa640 1 -- 172.21.15.172:0/4270944941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10cc15e4b0 msgr2=0x7f10cc0c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10d16fa640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10cc15e4b0 0x7f10cc0c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10d16fa640 1 -- 172.21.15.172:0/4270944941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10cc0d1180 msgr2=0x7f10cc15d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10d16fa640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10cc0d1180 0x7f10cc15d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10d16fa640 1 -- 172.21.15.172:0/4270944941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc0c7ca0 con 0x7f10cc0cfd70 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10d1efb640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10cc15e4b0 0x7f10cc0c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10d0ef9640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10cc0d1180 0x7f10cc15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:35.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10d16fa640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10cc0cfd70 0x7f10cc15cc50 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009e80 tx=0x7f10c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10c1ffb640 1 -- 172.21.15.172:0/4270944941 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8039050 con 0x7f10cc0cfd70 2024-04-14T08:42:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10c1ffb640 1 -- 172.21.15.172:0/4270944941 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c8045070 con 0x7f10cc0cfd70 2024-04-14T08:42:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10c1ffb640 1 -- 172.21.15.172:0/4270944941 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8009450 con 0x7f10cc0cfd70 2024-04-14T08:42:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.876+0000 7f10d395e640 1 -- 172.21.15.172:0/4270944941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc0b8c60 con 0x7f10cc0cfd70 2024-04-14T08:42:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.877+0000 7f10d395e640 1 -- 172.21.15.172:0/4270944941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc0b9220 con 0x7f10cc0cfd70 2024-04-14T08:42:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.878+0000 7f10d395e640 1 -- 172.21.15.172:0/4270944941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc068590 con 0x7f10cc0cfd70 2024-04-14T08:42:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.878+0000 7f10c1ffb640 1 -- 172.21.15.172:0/4270944941 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10c803e830 con 0x7f10cc0cfd70 2024-04-14T08:42:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.879+0000 7f10c1ffb640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10b006c920 0x7f10b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.879+0000 7f10c1ffb640 1 -- 172.21.15.172:0/4270944941 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5421+0+0 (secure 0 0 0) 0x7f10c80ae710 con 0x7f10cc0cfd70 2024-04-14T08:42:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.879+0000 7f10d0ef9640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10b006c920 0x7f10b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.879+0000 7f10d0ef9640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10b006c920 0x7f10b006ede0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0c7970 tx=0x7f10bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:35.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:35.883+0000 7f10c1ffb640 1 -- 172.21.15.172:0/4270944941 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f10c8032e20 con 0x7f10cc0cfd70 2024-04-14T08:42:36.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.067+0000 7f10d395e640 1 -- 172.21.15.172:0/4270944941 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f10cc0d17e0 con 0x7f10cc0cfd70 2024-04-14T08:42:36.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.865+0000 7f10c1ffb640 1 -- 172.21.15.172:0/4270944941 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v112) ==== 97+0+0 (secure 0 0 0) 0x7f10c807ce10 con 0x7f10cc0cfd70 2024-04-14T08:42:36.866 INFO:teuthology.orchestra.run.smithi172.stderr: removed all blocklist entries 2024-04-14T08:42:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.868+0000 7f10d395e640 1 -- 172.21.15.172:0/4270944941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10b006c920 msgr2=0x7f10b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.868+0000 7f10d395e640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10b006c920 0x7f10b006ede0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0c7970 tx=0x7f10bc009040 comp rx=0 tx=0).stop 2024-04-14T08:42:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.868+0000 7f10d395e640 1 -- 172.21.15.172:0/4270944941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10cc0cfd70 msgr2=0x7f10cc15cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.868+0000 7f10d395e640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10cc0cfd70 0x7f10cc15cc50 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009e80 tx=0x7f10c8004a40 comp rx=0 tx=0).stop 2024-04-14T08:42:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.868+0000 7f10d395e640 1 -- 172.21.15.172:0/4270944941 shutdown_connections 2024-04-14T08:42:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.868+0000 7f10d395e640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f10b006c920 0x7f10b006ede0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.868+0000 7f10d395e640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10cc15e4b0 0x7f10cc0c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.868+0000 7f10d395e640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10cc0d1180 0x7f10cc15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.868+0000 7f10d395e640 1 --2- 172.21.15.172:0/4270944941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10cc0cfd70 0x7f10cc15cc50 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.868+0000 7f10d395e640 1 -- 172.21.15.172:0/4270944941 >> 172.21.15.172:0/4270944941 conn(0x7f10cc09d570 msgr2=0x7f10cc0d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.868+0000 7f10d395e640 1 -- 172.21.15.172:0/4270944941 shutdown_connections 2024-04-14T08:42:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.868+0000 7f10d395e640 1 -- 172.21.15.172:0/4270944941 wait complete. 2024-04-14T08:42:36.880 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-14T08:42:36.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.983+0000 7f0eca636640 1 Processor -- start 2024-04-14T08:42:36.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.983+0000 7f0eca636640 1 -- start start 2024-04-14T08:42:36.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.984+0000 7f0eca636640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0ec40b1df0 0x7f0ec40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:36.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.984+0000 7f0eca636640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0ec40b21d0 0x7f0ec415c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:36.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.984+0000 7f0eca636640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ec40b3430 con 0x7f0ec415c5a0 2024-04-14T08:42:36.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.984+0000 7f0eca636640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0ec40b3700 con 0x7f0ec40b21d0 2024-04-14T08:42:36.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.984+0000 7f0eca636640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f0ec40b39e0 con 0x7f0ec40b1df0 2024-04-14T08:42:36.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.984+0000 7f0ec37fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0ec40b21d0 0x7f0ec415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:36.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.984+0000 7f0ec37fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0ec40b21d0 0x7f0ec415c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57272/0 (socket says 172.21.15.172:57272) 2024-04-14T08:42:36.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.984+0000 7f0ec37fe640 1 -- 172.21.15.172:0/3844418137 learned_addr learned my addr 172.21.15.172:0/3844418137 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:36.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.984+0000 7f0ec37fe640 1 -- 172.21.15.172:0/3844418137 >> v2:172.21.15.177:3300/0 conn(0x7f0ec40b1df0 msgr2=0x7f0ec40b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:36.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.984+0000 7f0ec37fe640 1 --2- 172.21.15.172:0/3844418137 >> v2:172.21.15.177:3300/0 conn(0x7f0ec40b1df0 0x7f0ec40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:36.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.984+0000 7f0ec37fe640 1 -- 172.21.15.172:0/3844418137 >> v1:172.21.15.177:6789/0 conn(0x7f0ec415c5a0 legacy=0x7f0ec40b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:36.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.984+0000 7f0ec37fe640 1 -- 172.21.15.172:0/3844418137 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec40b3cc0 con 0x7f0ec40b21d0 2024-04-14T08:42:36.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.985+0000 7f0ec37fe640 1 --2- 172.21.15.172:0/3844418137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec40b21d0 0x7f0ec415c060 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f0ec406f4b0 tx=0x7f0eb802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856fe21e530f0b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:36.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.985+0000 7f0ec2ffd640 1 -- 172.21.15.172:0/3844418137 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb802e070 con 0x7f0ec40b21d0 2024-04-14T08:42:36.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.985+0000 7f0ec2ffd640 1 -- 172.21.15.172:0/3844418137 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb8037070 con 0x7f0ec40b21d0 2024-04-14T08:42:36.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.985+0000 7f0ec2ffd640 1 -- 172.21.15.172:0/3844418137 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8031a00 con 0x7f0ec40b21d0 2024-04-14T08:42:36.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.986+0000 7f0eca636640 1 -- 172.21.15.172:0/3844418137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec40b21d0 msgr2=0x7f0ec415c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:36.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.986+0000 7f0eca636640 1 --2- 172.21.15.172:0/3844418137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec40b21d0 0x7f0ec415c060 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f0ec406f4b0 tx=0x7f0eb802b880 comp rx=0 tx=0).stop 2024-04-14T08:42:36.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.986+0000 7f0eca636640 1 -- 172.21.15.172:0/3844418137 shutdown_connections 2024-04-14T08:42:36.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.986+0000 7f0eca636640 1 --2- 172.21.15.172:0/3844418137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec40b21d0 0x7f0ec415c060 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:36.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.986+0000 7f0eca636640 1 --2- 172.21.15.172:0/3844418137 >> v2:172.21.15.177:3300/0 conn(0x7f0ec40b1df0 0x7f0ec40b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:36.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.986+0000 7f0eca636640 1 -- 172.21.15.172:0/3844418137 >> 172.21.15.172:0/3844418137 conn(0x7f0ec406ec30 msgr2=0x7f0ec406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:36.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.986+0000 7f0eca636640 1 -- 172.21.15.172:0/3844418137 shutdown_connections 2024-04-14T08:42:36.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.986+0000 7f0eca636640 1 -- 172.21.15.172:0/3844418137 wait complete. 2024-04-14T08:42:36.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.987+0000 7f0eca636640 1 Processor -- start 2024-04-14T08:42:36.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.987+0000 7f0eca636640 1 -- start start 2024-04-14T08:42:36.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0eca636640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec40b1df0 0x7f0ec41f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:36.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0eca636640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ec40b21d0 0x7f0ec41f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0eca636640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ec4157810 0x7f0ec40ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0eca636640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec40b4750 con 0x7f0ec40b1df0 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0eca636640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec415b7f0 con 0x7f0ec40b21d0 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0eca636640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0ec40b48d0 con 0x7f0ec4157810 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0ec3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec40b1df0 0x7f0ec41f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0ec3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec40b1df0 0x7f0ec41f39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57282/0 (socket says 172.21.15.172:57282) 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0ec3fff640 1 -- 172.21.15.172:0/3803359200 learned_addr learned my addr 172.21.15.172:0/3803359200 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0ec37fe640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ec40b21d0 0x7f0ec41f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0ec3fff640 1 -- 172.21.15.172:0/3803359200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ec4157810 msgr2=0x7f0ec40ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0ec3fff640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ec4157810 0x7f0ec40ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0ec3fff640 1 -- 172.21.15.172:0/3803359200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ec40b21d0 msgr2=0x7f0ec41f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0ec3fff640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ec40b21d0 0x7f0ec41f2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0ec3fff640 1 -- 172.21.15.172:0/3803359200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec41f5160 con 0x7f0ec40b1df0 2024-04-14T08:42:36.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0ec3fff640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec40b1df0 0x7f0ec41f39c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400b4d0 tx=0x7f0eb400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:36.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.988+0000 7f0ec0ff9640 1 -- 172.21.15.172:0/3803359200 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4014070 con 0x7f0ec40b1df0 2024-04-14T08:42:36.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.989+0000 7f0ec0ff9640 1 -- 172.21.15.172:0/3803359200 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb4010870 con 0x7f0ec40b1df0 2024-04-14T08:42:36.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.989+0000 7f0ec0ff9640 1 -- 172.21.15.172:0/3803359200 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4010c60 con 0x7f0ec40b1df0 2024-04-14T08:42:36.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.989+0000 7f0eca636640 1 -- 172.21.15.172:0/3803359200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec41f40c0 con 0x7f0ec40b1df0 2024-04-14T08:42:36.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.989+0000 7f0eca636640 1 -- 172.21.15.172:0/3803359200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec41f4680 con 0x7f0ec40b1df0 2024-04-14T08:42:36.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.990+0000 7f0eca636640 1 -- 172.21.15.172:0/3803359200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec415ca20 con 0x7f0ec40b1df0 2024-04-14T08:42:36.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.990+0000 7f0ec0ff9640 1 -- 172.21.15.172:0/3803359200 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0eb4017070 con 0x7f0ec40b1df0 2024-04-14T08:42:36.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.990+0000 7f0ec0ff9640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0eb006c8f0 0x7f0eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:36.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.990+0000 7f0ec0ff9640 1 -- 172.21.15.172:0/3803359200 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5249+0+0 (secure 0 0 0) 0x7f0eb408b750 con 0x7f0ec40b1df0 2024-04-14T08:42:36.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.990+0000 7f0ec37fe640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0eb006c8f0 0x7f0eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:36.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.991+0000 7f0ec37fe640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0eb006c8f0 0x7f0eb006edb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8036000 tx=0x7f0eb8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:36.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:36.995+0000 7f0ec0ff9640 1 -- 172.21.15.172:0/3803359200 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0ec415ca20 con 0x7f0ec40b1df0 2024-04-14T08:42:37.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.236+0000 7f0eca636640 1 -- 172.21.15.172:0/3803359200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0ec4068520 con 0x7f0ec40b1df0 2024-04-14T08:42:37.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.237+0000 7f0ec0ff9640 1 -- 172.21.15.172:0/3803359200 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f0eb4059d40 con 0x7f0ec40b1df0 2024-04-14T08:42:37.237 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:37.237 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:42:37.237 INFO:teuthology.orchestra.run.smithi172.stdout: "auth_dump": [ 2024-04-14T08:42:37.237 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.237 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.a", 2024-04-14T08:42:37.237 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmnNCRDRAAz7aU3JD7iiaTQH0cNotfaw==", 2024-04-14T08:42:37.237 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.b", 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtm1on4GRAAms1he+O4jWu01z0ZLLgL0A==", 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:42:37.238 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.c", 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmb7z0FBAAGVu3+TWshDGvd7/fLbgJ5A==", 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.239 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mds.d", 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmEEx0IRAAXiFiOK1RE/0bfyu7KI/3ng==", 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.0", 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB/lBtmnsWuCRAAqcuQhsrzpGVfrlgwTnFsOw==", 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:42:37.240 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.1", 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCClBtme9lPBxAAFB1E3g8qwZxPCdHiFogp5Q==", 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.241 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.2", 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCFlBtm966YBRAAWXZGq/0TuUMZTsyeR4xWww==", 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.3", 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCIlBtmHpktBBAAUbddNHESwyNvvp8UE/SaCw==", 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:42:37.242 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.4", 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCOlBtm+//7ChAA7rApbKll2vBGcDlW8q+kyg==", 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.5", 2024-04-14T08:42:37.243 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCRlBtmVU4YChAAysdUosQ+TnHtjnRl1FOISA==", 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.6", 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCUlBtmnE7+BxAAqBn8YMwecA2NcZz63eBm7Q==", 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:42:37.244 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.245 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.245 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.245 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.245 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "osd.7", 2024-04-14T08:42:37.245 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQCXlBtmN8DhBRAA6CHQ943t6qJCL77jcbOv4w==", 2024-04-14T08:42:37.245 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.245 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow profile osd", 2024-04-14T08:42:37.245 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile osd", 2024-04-14T08:42:37.245 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.245 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.0", 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA==", 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow r", 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.1", 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmp4nkLhAAhYKm8VcykVbPSohvK11I9w==", 2024-04-14T08:42:37.246 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.2", 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmJoV9KxAAp6zlRX+gvhASQBHjpF9ewQ==", 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:42:37.247 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.248 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.248 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.248 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.3", 2024-04-14T08:42:37.248 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmeC+pNBAASi7W4jlo6imMMzNEwLrPDA==", 2024-04-14T08:42:37.248 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.248 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow", 2024-04-14T08:42:37.248 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow r", 2024-04-14T08:42:37.248 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow rw", 2024-04-14T08:42:37.248 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow rwx" 2024-04-14T08:42:37.248 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.248 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "client.admin", 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB6lBtmgnWqLBAAQ0fgarM8au3zFuMNnQIvVw==", 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr": "allow *", 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow *", 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.x", 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmm5IbChAAeVcVs3Zt6+lU0wghjCcwnQ==", 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.249 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: "entity": "mgr.y", 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: "key": "AQB7lBtmfddYBhAAh8zgf4W3WWmNYWNAA7WuRw==", 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": { 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": "allow *", 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: "mon": "allow profile mgr", 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: "osd": "allow *" 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:37.250 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-04-14T08:42:37.251 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:42:37.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.238+0000 7f0eca636640 1 -- 172.21.15.172:0/3803359200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0eb006c8f0 msgr2=0x7f0eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.238+0000 7f0eca636640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0eb006c8f0 0x7f0eb006edb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8036000 tx=0x7f0eb8008030 comp rx=0 tx=0).stop 2024-04-14T08:42:37.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.239+0000 7f0eca636640 1 -- 172.21.15.172:0/3803359200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec40b1df0 msgr2=0x7f0ec41f39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.239+0000 7f0eca636640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec40b1df0 0x7f0ec41f39c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400b4d0 tx=0x7f0eb400b990 comp rx=0 tx=0).stop 2024-04-14T08:42:37.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.239+0000 7f0eca636640 1 -- 172.21.15.172:0/3803359200 shutdown_connections 2024-04-14T08:42:37.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.239+0000 7f0eca636640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0eb006c8f0 0x7f0eb006edb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.239+0000 7f0eca636640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ec4157810 0x7f0ec40ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.239+0000 7f0eca636640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ec40b21d0 0x7f0ec41f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.239+0000 7f0eca636640 1 --2- 172.21.15.172:0/3803359200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec40b1df0 0x7f0ec41f39c0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.239+0000 7f0eca636640 1 -- 172.21.15.172:0/3803359200 >> 172.21.15.172:0/3803359200 conn(0x7f0ec406ec30 msgr2=0x7f0ec415cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:37.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.239+0000 7f0eca636640 1 -- 172.21.15.172:0/3803359200 shutdown_connections 2024-04-14T08:42:37.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.239+0000 7f0eca636640 1 -- 172.21.15.172:0/3803359200 wait complete. 2024-04-14T08:42:37.253 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:42:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.354+0000 7f118dac8640 1 Processor -- start 2024-04-14T08:42:37.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.354+0000 7f118dac8640 1 -- start start 2024-04-14T08:42:37.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f118dac8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1188160170 0x7f118815d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f118dac8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f118815d4d0 0x7f11880b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f118dac8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f118813f910 con 0x7f118815d020 2024-04-14T08:42:37.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f118dac8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f118813fbf0 con 0x7f1188160170 2024-04-14T08:42:37.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f118dac8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f118813fed0 con 0x7f118815d4d0 2024-04-14T08:42:37.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f11877fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1188160170 0x7f118815d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f1186ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f118815d020 0x7f1188160550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56190/0 (socket says 172.21.15.172:56190) 2024-04-14T08:42:37.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f1186ffd640 1 -- 172.21.15.172:0/3515430911 learned_addr learned my addr 172.21.15.172:0/3515430911 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:37.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f11877fe640 1 -- 172.21.15.172:0/3515430911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f118815d4d0 msgr2=0x7f11880b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:37.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f11877fe640 1 --2- 172.21.15.172:0/3515430911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f118815d4d0 0x7f11880b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f11877fe640 1 -- 172.21.15.172:0/3515430911 >> v1:172.21.15.172:6789/0 conn(0x7f118815d020 legacy=0x7f1188160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f11877fe640 1 -- 172.21.15.172:0/3515430911 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11881401b0 con 0x7f1188160170 2024-04-14T08:42:37.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.355+0000 7f11867fc640 1 -- 172.21.15.172:0/3515430911 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244999304 0 0) 0x7f118813f910 con 0x7f118815d020 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.356+0000 7f11877fe640 1 --2- 172.21.15.172:0/3515430911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1188160170 0x7f118815d980 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f117c0097a0 tx=0x7f117c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa37973ac678bada server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.356+0000 7f11867fc640 1 -- 172.21.15.172:0/3515430911 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c035020 con 0x7f1188160170 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.356+0000 7f11867fc640 1 -- 172.21.15.172:0/3515430911 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f117c031830 con 0x7f1188160170 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.356+0000 7f11867fc640 1 -- 172.21.15.172:0/3515430911 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c031c00 con 0x7f1188160170 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.356+0000 7f118dac8640 1 -- 172.21.15.172:0/3515430911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1188160170 msgr2=0x7f118815d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.356+0000 7f118dac8640 1 --2- 172.21.15.172:0/3515430911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1188160170 0x7f118815d980 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f117c0097a0 tx=0x7f117c02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.357+0000 7f118dac8640 1 -- 172.21.15.172:0/3515430911 shutdown_connections 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.357+0000 7f118dac8640 1 --2- 172.21.15.172:0/3515430911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f118815d4d0 0x7f11880b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.357+0000 7f118dac8640 1 --2- 172.21.15.172:0/3515430911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1188160170 0x7f118815d980 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.357+0000 7f118dac8640 1 -- 172.21.15.172:0/3515430911 >> 172.21.15.172:0/3515430911 conn(0x7f118806ec30 msgr2=0x7f118806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.357+0000 7f118dac8640 1 -- 172.21.15.172:0/3515430911 shutdown_connections 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.357+0000 7f118dac8640 1 -- 172.21.15.172:0/3515430911 wait complete. 2024-04-14T08:42:37.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.358+0000 7f118dac8640 1 Processor -- start 2024-04-14T08:42:37.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.358+0000 7f118dac8640 1 -- start start 2024-04-14T08:42:37.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f118dac8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f118815d020 0x7f118815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f118dac8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f118815d4d0 0x7f11880b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f118dac8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1188160170 0x7f1188154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f118dac8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11880b20c0 con 0x7f118815d020 2024-04-14T08:42:37.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f118dac8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1188160c60 con 0x7f1188160170 2024-04-14T08:42:37.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f118dac8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f11880b2240 con 0x7f118815d4d0 2024-04-14T08:42:37.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f11877fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f118815d020 0x7f118815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f11877fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f118815d020 0x7f118815a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57296/0 (socket says 172.21.15.172:57296) 2024-04-14T08:42:37.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f11877fe640 1 -- 172.21.15.172:0/3231270173 learned_addr learned my addr 172.21.15.172:0/3231270173 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:37.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f1186ffd640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f118815d4d0 0x7f11880b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f1187fff640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1188160170 0x7f1188154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f11877fe640 1 -- 172.21.15.172:0/3231270173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f118815d4d0 msgr2=0x7f11880b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f11877fe640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f118815d4d0 0x7f11880b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f11877fe640 1 -- 172.21.15.172:0/3231270173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1188160170 msgr2=0x7f1188154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f11877fe640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1188160170 0x7f1188154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f11877fe640 1 -- 172.21.15.172:0/3231270173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118815b0d0 con 0x7f118815d020 2024-04-14T08:42:37.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f1187fff640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1188160170 0x7f1188154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:37.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f11877fe640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f118815d020 0x7f118815a8a0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f117c0097a0 tx=0x7f117c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:37.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f116ffff640 1 -- 172.21.15.172:0/3231270173 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c004230 con 0x7f118815d020 2024-04-14T08:42:37.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f116ffff640 1 -- 172.21.15.172:0/3231270173 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f117c0043f0 con 0x7f118815d020 2024-04-14T08:42:37.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f118dac8640 1 -- 172.21.15.172:0/3231270173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118815b3b0 con 0x7f118815d020 2024-04-14T08:42:37.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f116ffff640 1 -- 172.21.15.172:0/3231270173 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c003850 con 0x7f118815d020 2024-04-14T08:42:37.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.359+0000 7f118dac8640 1 -- 172.21.15.172:0/3231270173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11881585d0 con 0x7f118815d020 2024-04-14T08:42:37.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.360+0000 7f118dac8640 1 -- 172.21.15.172:0/3231270173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f118815b580 con 0x7f118815d020 2024-04-14T08:42:37.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.361+0000 7f116ffff640 1 -- 172.21.15.172:0/3231270173 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f117c04c060 con 0x7f118815d020 2024-04-14T08:42:37.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.361+0000 7f116ffff640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f117406c920 0x7f117406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.361+0000 7f116ffff640 1 -- 172.21.15.172:0/3231270173 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5249+0+0 (secure 0 0 0) 0x7f117c0ad500 con 0x7f118815d020 2024-04-14T08:42:37.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.361+0000 7f1186ffd640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f117406c920 0x7f117406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.362+0000 7f1186ffd640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f117406c920 0x7f117406ede0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f118815c0d0 tx=0x7f1178007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:37.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.366+0000 7f116ffff640 1 -- 172.21.15.172:0/3231270173 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f117c040070 con 0x7f118815d020 2024-04-14T08:42:37.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.541+0000 7f118dac8640 1 -- 172.21.15.172:0/3231270173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1188068520 con 0x7f118815d020 2024-04-14T08:42:37.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.542+0000 7f116ffff640 1 -- 172.21.15.172:0/3231270173 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+13227 (secure 0 0 0) 0x7f117c07baf0 con 0x7f118815d020 2024-04-14T08:42:37.542 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:37.542 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":112,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:42:36.862636+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:42:37.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.544+0000 7f118dac8640 1 -- 172.21.15.172:0/3231270173 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f117406c920 msgr2=0x7f117406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.544+0000 7f118dac8640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f117406c920 0x7f117406ede0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f118815c0d0 tx=0x7f1178007b50 comp rx=0 tx=0).stop 2024-04-14T08:42:37.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.544+0000 7f118dac8640 1 -- 172.21.15.172:0/3231270173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f118815d020 msgr2=0x7f118815a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.544+0000 7f118dac8640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f118815d020 0x7f118815a8a0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f117c0097a0 tx=0x7f117c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.544+0000 7f118dac8640 1 -- 172.21.15.172:0/3231270173 shutdown_connections 2024-04-14T08:42:37.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.544+0000 7f118dac8640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f117406c920 0x7f117406ede0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.544+0000 7f118dac8640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1188160170 0x7f1188154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.544+0000 7f118dac8640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f118815d4d0 0x7f11880b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.544+0000 7f118dac8640 1 --2- 172.21.15.172:0/3231270173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f118815d020 0x7f118815a8a0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.544+0000 7f118dac8640 1 -- 172.21.15.172:0/3231270173 >> 172.21.15.172:0/3231270173 conn(0x7f118806ec30 msgr2=0x7f11880d0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:37.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.544+0000 7f118dac8640 1 -- 172.21.15.172:0/3231270173 shutdown_connections 2024-04-14T08:42:37.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.544+0000 7f118dac8640 1 -- 172.21.15.172:0/3231270173 wait complete. 2024-04-14T08:42:37.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:42:37.555 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:42:37.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.659+0000 7f612f8b7640 1 Processor -- start 2024-04-14T08:42:37.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.659+0000 7f612f8b7640 1 -- start start 2024-04-14T08:42:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.660+0000 7f612f8b7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f61280b23e0 0x7f6128140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.660+0000 7f612f8b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f612815c380 0x7f6128140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.660+0000 7f612f8b7640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f612815b620 con 0x7f612813df30 2024-04-14T08:42:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.660+0000 7f612f8b7640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f61280cf9b0 con 0x7f61280b23e0 2024-04-14T08:42:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.660+0000 7f612f8b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f61281411f0 con 0x7f612815c380 2024-04-14T08:42:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.660+0000 7f612d653640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f612813df30 0x7f61280b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56206/0 (socket says 172.21.15.172:56206) 2024-04-14T08:42:37.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.660+0000 7f612d653640 1 -- 172.21.15.172:0/3109236552 learned_addr learned my addr 172.21.15.172:0/3109236552 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.660+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2528586102 0 0) 0x7f6124001490 con 0x7f612813df30 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.660+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6114003520 con 0x7f612813df30 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.660+0000 7f612ce52640 1 --2- 172.21.15.172:0/3109236552 >> v2:172.21.15.177:3300/0 conn(0x7f61280b23e0 0x7f6128140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.660+0000 7f612de54640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f612815c380 0x7f6128140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 235722467 0 0) 0x7f6114003520 con 0x7f612813df30 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f612815c380 msgr2=0x7f6128140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f612815c380 0x7f6128140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 >> v2:172.21.15.177:3300/0 conn(0x7f61280b23e0 msgr2=0x7f6128140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 --2- 172.21.15.172:0/3109236552 >> v2:172.21.15.177:3300/0 conn(0x7f61280b23e0 0x7f6128140770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61281414a0 con 0x7f612813df30 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f6124001bc0 con 0x7f612813df30 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 >> v1:172.21.15.172:6789/0 conn(0x7f612813df30 legacy=0x7f61280b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61140039b0 0x7f6114003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f612815c380 0x7f6114004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6114004d40 0x7f61140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61280cf9b0 con 0x7f61140039b0 2024-04-14T08:42:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61281411f0 con 0x7f6114004d40 2024-04-14T08:42:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f611400b9c0 con 0x7f612815c380 2024-04-14T08:42:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f612d653640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61140039b0 0x7f6114003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f612ce52640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f612815c380 0x7f6114004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f612de54640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6114004d40 0x7f61140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f612d653640 1 -- 172.21.15.172:0/3109236552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f612815c380 msgr2=0x7f6114004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f612d653640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f612815c380 0x7f6114004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f612d653640 1 -- 172.21.15.172:0/3109236552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6114004d40 msgr2=0x7f61140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:37.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f612d653640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6114004d40 0x7f61140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f612d653640 1 -- 172.21.15.172:0/3109236552 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6114003520 con 0x7f61140039b0 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.661+0000 7f612de54640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6114004d40 0x7f61140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.662+0000 7f612d653640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61140039b0 0x7f6114003dd0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f612400aa70 tx=0x7f6114005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc5e33c747e00ee server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.662+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124038070 con 0x7f61140039b0 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.662+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61240072d0 con 0x7f61140039b0 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.662+0000 7f611ffff640 1 -- 172.21.15.172:0/3109236552 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612400a690 con 0x7f61140039b0 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.662+0000 7f612f8b7640 1 -- 172.21.15.172:0/3109236552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61140039b0 msgr2=0x7f6114003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.662+0000 7f612f8b7640 1 --2- 172.21.15.172:0/3109236552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61140039b0 0x7f6114003dd0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f612400aa70 tx=0x7f6114005830 comp rx=0 tx=0).stop 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.662+0000 7f612de54640 1 -- 172.21.15.172:0/3109236552 reap_dead start 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.663+0000 7f612f8b7640 1 -- 172.21.15.172:0/3109236552 shutdown_connections 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.663+0000 7f612f8b7640 1 -- 172.21.15.172:0/3109236552 >> 172.21.15.172:0/3109236552 conn(0x7f612806b520 msgr2=0x7f612806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.663+0000 7f612f8b7640 1 -- 172.21.15.172:0/3109236552 shutdown_connections 2024-04-14T08:42:37.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.663+0000 7f612f8b7640 1 -- 172.21.15.172:0/3109236552 wait complete. 2024-04-14T08:42:37.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.664+0000 7f612f8b7640 1 Processor -- start 2024-04-14T08:42:37.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.664+0000 7f612f8b7640 1 -- start start 2024-04-14T08:42:37.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.664+0000 7f612f8b7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61140039b0 0x7f6128158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.664+0000 7f612f8b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61280b4a50 0x7f6128155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.664+0000 7f612f8b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6128156200 0x7f6128156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.664+0000 7f612f8b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f612815be60 con 0x7f6128156200 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.664+0000 7f612f8b7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f612815bb80 con 0x7f61140039b0 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.664+0000 7f612f8b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f612813e310 con 0x7f61280b4a50 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612de54640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6128156200 0x7f6128156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612de54640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6128156200 0x7f6128156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57302/0 (socket says 172.21.15.172:57302) 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612de54640 1 -- 172.21.15.172:0/259635610 learned_addr learned my addr 172.21.15.172:0/259635610 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612ce52640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61280b4a50 0x7f6128155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612d653640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61140039b0 0x7f6128158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612de54640 1 -- 172.21.15.172:0/259635610 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61280b4a50 msgr2=0x7f6128155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612de54640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61280b4a50 0x7f6128155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612de54640 1 -- 172.21.15.172:0/259635610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61140039b0 msgr2=0x7f6128158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612de54640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61140039b0 0x7f6128158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612de54640 1 -- 172.21.15.172:0/259635610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61280b4fd0 con 0x7f6128156200 2024-04-14T08:42:37.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612de54640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6128156200 0x7f6128156680 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f61280cdb90 tx=0x7f6120011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:37.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f611dffb640 1 -- 172.21.15.172:0/259635610 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120004440 con 0x7f6128156200 2024-04-14T08:42:37.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f611dffb640 1 -- 172.21.15.172:0/259635610 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f612001b070 con 0x7f6128156200 2024-04-14T08:42:37.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612f8b7640 1 -- 172.21.15.172:0/259635610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61280b52b0 con 0x7f6128156200 2024-04-14T08:42:37.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f611dffb640 1 -- 172.21.15.172:0/259635610 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120003850 con 0x7f6128156200 2024-04-14T08:42:37.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.665+0000 7f612f8b7640 1 -- 172.21.15.172:0/259635610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61280b5870 con 0x7f6128156200 2024-04-14T08:42:37.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.666+0000 7f612f8b7640 1 -- 172.21.15.172:0/259635610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61280b5a70 con 0x7f6128156200 2024-04-14T08:42:37.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.667+0000 7f611dffb640 1 -- 172.21.15.172:0/259635610 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61200039f0 con 0x7f6128156200 2024-04-14T08:42:37.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.667+0000 7f611dffb640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f610c06c8f0 0x7f610c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.667+0000 7f611dffb640 1 -- 172.21.15.172:0/259635610 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5249+0+0 (secure 0 0 0) 0x7f612008b930 con 0x7f6128156200 2024-04-14T08:42:37.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.667+0000 7f612d653640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f610c06c8f0 0x7f610c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.668+0000 7f612d653640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f610c06c8f0 0x7f610c06edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6124035000 tx=0x7f6124030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:37.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.672+0000 7f611dffb640 1 -- 172.21.15.172:0/259635610 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f61280b5a70 con 0x7f6128156200 2024-04-14T08:42:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.843+0000 7f612f8b7640 1 -- 172.21.15.172:0/259635610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61280b38b0 con 0x7f6128156200 2024-04-14T08:42:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.844+0000 7f611dffb640 1 -- 172.21.15.172:0/259635610 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7f6120025020 con 0x7f6128156200 2024-04-14T08:42:37.844 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:42:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.846+0000 7f612f8b7640 1 -- 172.21.15.172:0/259635610 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f610c06c8f0 msgr2=0x7f610c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.846+0000 7f612f8b7640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f610c06c8f0 0x7f610c06edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6124035000 tx=0x7f6124030000 comp rx=0 tx=0).stop 2024-04-14T08:42:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.846+0000 7f612f8b7640 1 -- 172.21.15.172:0/259635610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6128156200 msgr2=0x7f6128156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.846+0000 7f612f8b7640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6128156200 0x7f6128156680 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f61280cdb90 tx=0x7f6120011860 comp rx=0 tx=0).stop 2024-04-14T08:42:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.846+0000 7f612f8b7640 1 -- 172.21.15.172:0/259635610 shutdown_connections 2024-04-14T08:42:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.846+0000 7f612f8b7640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f610c06c8f0 0x7f610c06edb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.846+0000 7f612f8b7640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6128156200 0x7f6128156680 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.846+0000 7f612f8b7640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61280b4a50 0x7f6128155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.846+0000 7f612f8b7640 1 --2- 172.21.15.172:0/259635610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61140039b0 0x7f6128158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.846+0000 7f612f8b7640 1 -- 172.21.15.172:0/259635610 >> 172.21.15.172:0/259635610 conn(0x7f612806b520 msgr2=0x7f6128159240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.846+0000 7f612f8b7640 1 -- 172.21.15.172:0/259635610 shutdown_connections 2024-04-14T08:42:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.846+0000 7f612f8b7640 1 -- 172.21.15.172:0/259635610 wait complete. 2024-04-14T08:42:37.857 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-14T08:42:37.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.958+0000 7f01befed640 1 Processor -- start 2024-04-14T08:42:37.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.958+0000 7f01befed640 1 -- start start 2024-04-14T08:42:37.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01befed640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f01b80b1df0 0x7f01b80b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01befed640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f01b80b21d0 0x7f01b815c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01befed640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b80b3430 con 0x7f01b815c5a0 2024-04-14T08:42:37.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01befed640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f01b80b3700 con 0x7f01b80b21d0 2024-04-14T08:42:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01befed640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f01b80b39e0 con 0x7f01b80b1df0 2024-04-14T08:42:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01b7fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f01b80b21d0 0x7f01b815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01b7fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f01b80b21d0 0x7f01b815c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57312/0 (socket says 172.21.15.172:57312) 2024-04-14T08:42:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01bd58a640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f01b815c5a0 0x7f01b80b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56222/0 (socket says 172.21.15.172:56222) 2024-04-14T08:42:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01b7fff640 1 -- 172.21.15.172:0/4075817575 learned_addr learned my addr 172.21.15.172:0/4075817575 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01bcd89640 1 --2- 172.21.15.172:0/4075817575 >> v2:172.21.15.177:3300/0 conn(0x7f01b80b1df0 0x7f01b80b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01b7fff640 1 -- 172.21.15.172:0/4075817575 >> v2:172.21.15.177:3300/0 conn(0x7f01b80b1df0 msgr2=0x7f01b80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:37.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01b7fff640 1 --2- 172.21.15.172:0/4075817575 >> v2:172.21.15.177:3300/0 conn(0x7f01b80b1df0 0x7f01b80b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01b7fff640 1 -- 172.21.15.172:0/4075817575 >> v1:172.21.15.172:6789/0 conn(0x7f01b815c5a0 legacy=0x7f01b80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01b7fff640 1 -- 172.21.15.172:0/4075817575 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b80b3cc0 con 0x7f01b80b21d0 2024-04-14T08:42:37.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.959+0000 7f01b77fe640 1 -- 172.21.15.172:0/4075817575 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3901130633 0 0) 0x7f01b80b3430 con 0x7f01b815c5a0 2024-04-14T08:42:37.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.960+0000 7f01bcd89640 1 --2- 172.21.15.172:0/4075817575 >> v2:172.21.15.177:3300/0 conn(0x7f01b80b1df0 0x7f01b80b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:37.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.960+0000 7f01b7fff640 1 --2- 172.21.15.172:0/4075817575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b80b21d0 0x7f01b815c060 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f01b806f4b0 tx=0x7f01ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf075fde1e33b505 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:37.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.960+0000 7f01b77fe640 1 -- 172.21.15.172:0/4075817575 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac035020 con 0x7f01b80b21d0 2024-04-14T08:42:37.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.960+0000 7f01b77fe640 1 -- 172.21.15.172:0/4075817575 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ac031870 con 0x7f01b80b21d0 2024-04-14T08:42:37.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.960+0000 7f01b77fe640 1 -- 172.21.15.172:0/4075817575 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac031c40 con 0x7f01b80b21d0 2024-04-14T08:42:37.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.961+0000 7f01befed640 1 -- 172.21.15.172:0/4075817575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b80b21d0 msgr2=0x7f01b815c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.961+0000 7f01befed640 1 --2- 172.21.15.172:0/4075817575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b80b21d0 0x7f01b815c060 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f01b806f4b0 tx=0x7f01ac02b910 comp rx=0 tx=0).stop 2024-04-14T08:42:37.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.961+0000 7f01befed640 1 -- 172.21.15.172:0/4075817575 shutdown_connections 2024-04-14T08:42:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.961+0000 7f01befed640 1 --2- 172.21.15.172:0/4075817575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b80b21d0 0x7f01b815c060 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.961+0000 7f01befed640 1 --2- 172.21.15.172:0/4075817575 >> v2:172.21.15.177:3300/0 conn(0x7f01b80b1df0 0x7f01b80b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.961+0000 7f01befed640 1 -- 172.21.15.172:0/4075817575 >> 172.21.15.172:0/4075817575 conn(0x7f01b806ec30 msgr2=0x7f01b806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.961+0000 7f01befed640 1 -- 172.21.15.172:0/4075817575 shutdown_connections 2024-04-14T08:42:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.961+0000 7f01befed640 1 -- 172.21.15.172:0/4075817575 wait complete. 2024-04-14T08:42:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.962+0000 7f01befed640 1 Processor -- start 2024-04-14T08:42:37.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.962+0000 7f01befed640 1 -- start start 2024-04-14T08:42:37.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01befed640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01b80b1df0 0x7f01b815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01befed640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01b80b21d0 0x7f01b8144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01befed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b81573f0 0x7f01b81f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01befed640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01b80b4750 con 0x7f01b81573f0 2024-04-14T08:42:37.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01befed640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f01b815b7f0 con 0x7f01b80b1df0 2024-04-14T08:42:37.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01befed640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f01b80b48d0 con 0x7f01b80b21d0 2024-04-14T08:42:37.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01bd58a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b81573f0 0x7f01b81f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01bd58a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b81573f0 0x7f01b81f49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57318/0 (socket says 172.21.15.172:57318) 2024-04-14T08:42:37.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01bd58a640 1 -- 172.21.15.172:0/3552787201 learned_addr learned my addr 172.21.15.172:0/3552787201 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:37.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01b7fff640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01b80b21d0 0x7f01b8144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01bcd89640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01b80b1df0 0x7f01b815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01bd58a640 1 -- 172.21.15.172:0/3552787201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01b80b21d0 msgr2=0x7f01b8144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01bd58a640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01b80b21d0 0x7f01b8144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01bd58a640 1 -- 172.21.15.172:0/3552787201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01b80b1df0 msgr2=0x7f01b815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:37.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01bd58a640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01b80b1df0 0x7f01b815a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:37.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01bd58a640 1 -- 172.21.15.172:0/3552787201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b8159440 con 0x7f01b81573f0 2024-04-14T08:42:37.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01b7fff640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01b80b21d0 0x7f01b8144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:37.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01bcd89640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01b80b1df0 0x7f01b815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:37.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.963+0000 7f01bd58a640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b81573f0 0x7f01b81f49a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f01b000c7a0 tx=0x7f01b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:37.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.964+0000 7f01b57fa640 1 -- 172.21.15.172:0/3552787201 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0018070 con 0x7f01b81573f0 2024-04-14T08:42:37.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.964+0000 7f01b57fa640 1 -- 172.21.15.172:0/3552787201 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b000a640 con 0x7f01b81573f0 2024-04-14T08:42:37.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.964+0000 7f01befed640 1 -- 172.21.15.172:0/3552787201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b8159720 con 0x7f01b81573f0 2024-04-14T08:42:37.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.964+0000 7f01b57fa640 1 -- 172.21.15.172:0/3552787201 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b000aab0 con 0x7f01b81573f0 2024-04-14T08:42:37.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.964+0000 7f01befed640 1 -- 172.21.15.172:0/3552787201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b8159ce0 con 0x7f01b81573f0 2024-04-14T08:42:37.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.965+0000 7f01befed640 1 -- 172.21.15.172:0/3552787201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b80728c0 con 0x7f01b81573f0 2024-04-14T08:42:37.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.965+0000 7f01b57fa640 1 -- 172.21.15.172:0/3552787201 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01b001b070 con 0x7f01b81573f0 2024-04-14T08:42:37.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.966+0000 7f01b57fa640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01a406c8f0 0x7f01a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:37.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.966+0000 7f01bcd89640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01a406c8f0 0x7f01a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:37.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.966+0000 7f01b57fa640 1 -- 172.21.15.172:0/3552787201 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5249+0+0 (secure 0 0 0) 0x7f01b008afc0 con 0x7f01b81573f0 2024-04-14T08:42:37.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.966+0000 7f01bcd89640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01a406c8f0 0x7f01a406edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f01a800a990 tx=0x7f01a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:37.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:37.970+0000 7f01b57fa640 1 -- 172.21.15.172:0/3552787201 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f01b80728c0 con 0x7f01b81573f0 2024-04-14T08:42:38.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.147+0000 7f01befed640 1 -- 172.21.15.172:0/3552787201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f01b8068520 con 0x7f01b81573f0 2024-04-14T08:42:38.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.147+0000 7f01b57fa640 1 -- 172.21.15.172:0/3552787201 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v112) ==== 84+0+65 (secure 0 0 0) 0x7f01b00595b0 con 0x7f01b81573f0 2024-04-14T08:42:38.147 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:38.148 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-04-14T08:42:38.148 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-04-14T08:42:38.148 INFO:teuthology.orchestra.run.smithi172.stdout: "poolnum": 1, 2024-04-14T08:42:38.148 INFO:teuthology.orchestra.run.smithi172.stdout: "poolname": ".mgr" 2024-04-14T08:42:38.148 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-04-14T08:42:38.148 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-04-14T08:42:38.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.149+0000 7f01befed640 1 -- 172.21.15.172:0/3552787201 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01a406c8f0 msgr2=0x7f01a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:38.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.149+0000 7f01befed640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01a406c8f0 0x7f01a406edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f01a800a990 tx=0x7f01a8008040 comp rx=0 tx=0).stop 2024-04-14T08:42:38.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.149+0000 7f01befed640 1 -- 172.21.15.172:0/3552787201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b81573f0 msgr2=0x7f01b81f49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:38.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.149+0000 7f01befed640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b81573f0 0x7f01b81f49a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f01b000c7a0 tx=0x7f01b000cc60 comp rx=0 tx=0).stop 2024-04-14T08:42:38.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.149+0000 7f01befed640 1 -- 172.21.15.172:0/3552787201 shutdown_connections 2024-04-14T08:42:38.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.149+0000 7f01befed640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01a406c8f0 0x7f01a406edb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.149+0000 7f01befed640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01b81573f0 0x7f01b81f49a0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.149+0000 7f01befed640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01b80b21d0 0x7f01b8144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.149+0000 7f01befed640 1 --2- 172.21.15.172:0/3552787201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01b80b1df0 0x7f01b815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.149+0000 7f01befed640 1 -- 172.21.15.172:0/3552787201 >> 172.21.15.172:0/3552787201 conn(0x7f01b806ec30 msgr2=0x7f01b806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:38.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.149+0000 7f01befed640 1 -- 172.21.15.172:0/3552787201 shutdown_connections 2024-04-14T08:42:38.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.149+0000 7f01befed640 1 -- 172.21.15.172:0/3552787201 wait complete. 2024-04-14T08:42:38.160 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-14T08:42:38.160 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-14T08:42:38.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.264+0000 7f7c19ed1640 1 Processor -- start 2024-04-14T08:42:38.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.265+0000 7f7c19ed1640 1 -- start start 2024-04-14T08:42:38.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.265+0000 7f7c19ed1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c140cc340 0x7f7c140ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:38.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.265+0000 7f7c19ed1640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c140cdb90 con 0x7f7c140cecc0 2024-04-14T08:42:38.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.265+0000 7f7c19ed1640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c140cde70 con 0x7f7c1415b350 2024-04-14T08:42:38.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.265+0000 7f7c19ed1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7c140cf150 con 0x7f7c140cc340 2024-04-14T08:42:38.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c13fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7c140cecc0 0x7f7c140c4e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56238/0 (socket says 172.21.15.172:56238) 2024-04-14T08:42:38.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c13fff640 1 -- 172.21.15.172:0/670412185 learned_addr learned my addr 172.21.15.172:0/670412185 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:38.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c12ffd640 1 --2- 172.21.15.172:0/670412185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c140cc340 0x7f7c140ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:38.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4151581948 0 0) 0x7f7c140cdb90 con 0x7f7c140cecc0 2024-04-14T08:42:38.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bfc003520 con 0x7f7c140cecc0 2024-04-14T08:42:38.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2768639107 0 0) 0x7f7c140cde70 con 0x7f7c1415b350 2024-04-14T08:42:38.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c140cdb90 con 0x7f7c1415b350 2024-04-14T08:42:38.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1957723648 0 0) 0x7f7bfc003520 con 0x7f7c140cecc0 2024-04-14T08:42:38.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c140cc340 msgr2=0x7f7c140ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 --2- 172.21.15.172:0/670412185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c140cc340 0x7f7c140ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 >> v1:172.21.15.177:6789/0 conn(0x7f7c1415b350 legacy=0x7f7c140cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c140ce150 con 0x7f7c140cecc0 2024-04-14T08:42:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f7c0c0019f0 con 0x7f7c140cecc0 2024-04-14T08:42:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 >> v1:172.21.15.172:6789/0 conn(0x7f7c140cecc0 legacy=0x7f7c140c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 --2- 172.21.15.172:0/670412185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bfc003b70 0x7f7bfc003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 --2- 172.21.15.172:0/670412185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c140cc340 0x7f7bfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 --2- 172.21.15.172:0/670412185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bfc004d40 0x7f7bfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7c140cf150 con 0x7f7bfc004d40 2024-04-14T08:42:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc00dcd0 con 0x7f7bfc003b70 2024-04-14T08:42:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc00ded0 con 0x7f7c140cc340 2024-04-14T08:42:38.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c13fff640 1 --2- 172.21.15.172:0/670412185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bfc004d40 0x7f7bfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c137fe640 1 --2- 172.21.15.172:0/670412185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bfc003b70 0x7f7bfc003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.266+0000 7f7c12ffd640 1 --2- 172.21.15.172:0/670412185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c140cc340 0x7f7bfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.267+0000 7f7c13fff640 1 -- 172.21.15.172:0/670412185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c140cc340 msgr2=0x7f7bfc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.267+0000 7f7c13fff640 1 --2- 172.21.15.172:0/670412185 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c140cc340 0x7f7bfc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.267+0000 7f7c13fff640 1 -- 172.21.15.172:0/670412185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bfc003b70 msgr2=0x7f7bfc003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.267+0000 7f7c13fff640 1 --2- 172.21.15.172:0/670412185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7bfc003b70 0x7f7bfc003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.267+0000 7f7c13fff640 1 -- 172.21.15.172:0/670412185 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc003520 con 0x7f7bfc004d40 2024-04-14T08:42:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.267+0000 7f7c13fff640 1 --2- 172.21.15.172:0/670412185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bfc004d40 0x7f7bfc0051c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00a7c0 tx=0x7f7bfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af8bee89599ce44 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.267+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c00a410 con 0x7f7bfc004d40 2024-04-14T08:42:38.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.267+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c034650 con 0x7f7bfc004d40 2024-04-14T08:42:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.267+0000 7f7c127fc640 1 -- 172.21.15.172:0/670412185 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c034a40 con 0x7f7bfc004d40 2024-04-14T08:42:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.268+0000 7f7c19ed1640 1 -- 172.21.15.172:0/670412185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bfc004d40 msgr2=0x7f7bfc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.268+0000 7f7c19ed1640 1 --2- 172.21.15.172:0/670412185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bfc004d40 0x7f7bfc0051c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00a7c0 tx=0x7f7bfc005830 comp rx=0 tx=0).stop 2024-04-14T08:42:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.268+0000 7f7c13fff640 1 -- 172.21.15.172:0/670412185 reap_dead start 2024-04-14T08:42:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.268+0000 7f7c19ed1640 1 -- 172.21.15.172:0/670412185 shutdown_connections 2024-04-14T08:42:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.268+0000 7f7c19ed1640 1 -- 172.21.15.172:0/670412185 >> 172.21.15.172:0/670412185 conn(0x7f7c140b1a50 msgr2=0x7f7c140b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.268+0000 7f7c19ed1640 1 -- 172.21.15.172:0/670412185 shutdown_connections 2024-04-14T08:42:38.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.268+0000 7f7c19ed1640 1 -- 172.21.15.172:0/670412185 wait complete. 2024-04-14T08:42:38.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.269+0000 7f7c19ed1640 1 Processor -- start 2024-04-14T08:42:38.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.269+0000 7f7c19ed1640 1 -- start start 2024-04-14T08:42:38.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.270+0000 7f7c19ed1640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bfc004d40 0x7f7c14151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:38.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.270+0000 7f7c19ed1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c14151580 0x7f7c14159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:38.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.270+0000 7f7c19ed1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415a1e0 0x7f7c141f0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:38.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.270+0000 7f7c19ed1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7c140ce4c0 con 0x7f7c1415a1e0 2024-04-14T08:42:38.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.270+0000 7f7c19ed1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7c140cd940 con 0x7f7c14151580 2024-04-14T08:42:38.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.270+0000 7f7c19ed1640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7c140ce780 con 0x7f7bfc004d40 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.270+0000 7f7c13fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415a1e0 0x7f7c141f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.270+0000 7f7c13fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415a1e0 0x7f7c141f0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57332/0 (socket says 172.21.15.172:57332) 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.270+0000 7f7c13fff640 1 -- 172.21.15.172:0/3257237259 learned_addr learned my addr 172.21.15.172:0/3257237259 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.270+0000 7f7c12ffd640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c14151580 0x7f7c14159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.270+0000 7f7c137fe640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bfc004d40 0x7f7c14151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.271+0000 7f7c13fff640 1 -- 172.21.15.172:0/3257237259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bfc004d40 msgr2=0x7f7c14151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.271+0000 7f7c13fff640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bfc004d40 0x7f7c14151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.271+0000 7f7c13fff640 1 -- 172.21.15.172:0/3257237259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c14151580 msgr2=0x7f7c14159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.271+0000 7f7c13fff640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c14151580 0x7f7c14159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.271+0000 7f7c13fff640 1 -- 172.21.15.172:0/3257237259 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c140c6ee0 con 0x7f7c1415a1e0 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.271+0000 7f7c13fff640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415a1e0 0x7f7c141f0300 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00a7c0 tx=0x7f7c140cdd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.271+0000 7f7bfbfff640 1 -- 172.21.15.172:0/3257237259 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0019f0 con 0x7f7c1415a1e0 2024-04-14T08:42:38.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.271+0000 7f7bfbfff640 1 -- 172.21.15.172:0/3257237259 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c044070 con 0x7f7c1415a1e0 2024-04-14T08:42:38.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.271+0000 7f7bfbfff640 1 -- 172.21.15.172:0/3257237259 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c034a00 con 0x7f7c1415a1e0 2024-04-14T08:42:38.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.271+0000 7f7c19ed1640 1 -- 172.21.15.172:0/3257237259 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c140c71c0 con 0x7f7c1415a1e0 2024-04-14T08:42:38.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.271+0000 7f7c19ed1640 1 -- 172.21.15.172:0/3257237259 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c140c7780 con 0x7f7c1415a1e0 2024-04-14T08:42:38.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.272+0000 7f7c19ed1640 1 -- 172.21.15.172:0/3257237259 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c140686a0 con 0x7f7c1415a1e0 2024-04-14T08:42:38.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.272+0000 7f7bfbfff640 1 -- 172.21.15.172:0/3257237259 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c0c049460 con 0x7f7c1415a1e0 2024-04-14T08:42:38.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.273+0000 7f7bfbfff640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7c0006c8f0 0x7f7c0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:38.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.273+0000 7f7bfbfff640 1 -- 172.21.15.172:0/3257237259 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5249+0+0 (secure 0 0 0) 0x7f7c0c0adee0 con 0x7f7c1415a1e0 2024-04-14T08:42:38.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.273+0000 7f7c137fe640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7c0006c8f0 0x7f7c0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:38.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.273+0000 7f7c137fe640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7c0006c8f0 0x7f7c0006edb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7c040018d0 tx=0x7f7c04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:38.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.277+0000 7f7bfbfff640 1 -- 172.21.15.172:0/3257237259 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7c0c07c590 con 0x7f7c1415a1e0 2024-04-14T08:42:38.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.451+0000 7f7c19ed1640 1 -- 172.21.15.172:0/3257237259 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7c140b6030 con 0x7f7c1415a1e0 2024-04-14T08:42:38.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.879+0000 7f7bfbfff640 1 -- 172.21.15.172:0/3257237259 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v113) ==== 138+0+0 (secure 0 0 0) 0x7f7c0c07be90 con 0x7f7c1415a1e0 2024-04-14T08:42:38.880 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' created 2024-04-14T08:42:38.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.881+0000 7f7c19ed1640 1 -- 172.21.15.172:0/3257237259 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7c0006c8f0 msgr2=0x7f7c0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:38.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.881+0000 7f7c19ed1640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7c0006c8f0 0x7f7c0006edb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7c040018d0 tx=0x7f7c04009040 comp rx=0 tx=0).stop 2024-04-14T08:42:38.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.881+0000 7f7c19ed1640 1 -- 172.21.15.172:0/3257237259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415a1e0 msgr2=0x7f7c141f0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:38.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.881+0000 7f7c19ed1640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415a1e0 0x7f7c141f0300 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00a7c0 tx=0x7f7c140cdd60 comp rx=0 tx=0).stop 2024-04-14T08:42:38.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.881+0000 7f7c19ed1640 1 -- 172.21.15.172:0/3257237259 shutdown_connections 2024-04-14T08:42:38.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.881+0000 7f7c19ed1640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7c0006c8f0 0x7f7c0006edb0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.881+0000 7f7c19ed1640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c1415a1e0 0x7f7c141f0300 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.881+0000 7f7c19ed1640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c14151580 0x7f7c14159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.881+0000 7f7c19ed1640 1 --2- 172.21.15.172:0/3257237259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bfc004d40 0x7f7c14151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:38.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.881+0000 7f7c19ed1640 1 -- 172.21.15.172:0/3257237259 >> 172.21.15.172:0/3257237259 conn(0x7f7c140b1a50 msgr2=0x7f7c1406e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:38.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.882+0000 7f7c19ed1640 1 -- 172.21.15.172:0/3257237259 shutdown_connections 2024-04-14T08:42:38.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:38.882+0000 7f7c19ed1640 1 -- 172.21.15.172:0/3257237259 wait complete. 2024-04-14T08:42:38.897 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-14T08:42:39.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.005+0000 7fe542036640 1 Processor -- start 2024-04-14T08:42:39.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.005+0000 7fe542036640 1 -- start start 2024-04-14T08:42:39.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.006+0000 7fe542036640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe53c0b65a0 0x7fe53c1629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:39.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.006+0000 7fe542036640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe53c0b3750 con 0x7fe53c0cea60 2024-04-14T08:42:39.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.006+0000 7fe542036640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe53c0cef20 con 0x7fe53c15e590 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.006+0000 7fe542036640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fe53c0cf200 con 0x7fe53c0b65a0 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.006+0000 7fe53bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe53c0cea60 0x7fe53c0d2760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:56252/0 (socket says 172.21.15.172:56252) 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.006+0000 7fe53bfff640 1 -- 172.21.15.172:0/3732519545 learned_addr learned my addr 172.21.15.172:0/3732519545 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53affd640 1 --2- 172.21.15.172:0/3732519545 >> v2:172.21.15.177:3300/0 conn(0x7fe53c0b65a0 0x7fe53c1629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 591285962 0 0) 0x7fe53c0b3750 con 0x7fe53c0cea60 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe524003520 con 0x7fe53c0cea60 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2186475957 0 0) 0x7fe524003520 con 0x7fe53c0cea60 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 >> v2:172.21.15.177:3300/0 conn(0x7fe53c0b65a0 msgr2=0x7fe53c1629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 --2- 172.21.15.172:0/3732519545 >> v2:172.21.15.177:3300/0 conn(0x7fe53c0b65a0 0x7fe53c1629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 >> v1:172.21.15.177:6789/0 conn(0x7fe53c15e590 legacy=0x7fe53c0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c0cf4e0 con 0x7fe53c0cea60 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fe5340019f0 con 0x7fe53c0cea60 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 >> v1:172.21.15.172:6789/0 conn(0x7fe53c0cea60 legacy=0x7fe53c0d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:39.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 --2- 172.21.15.172:0/3732519545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5240039b0 0x7fe524003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 --2- 172.21.15.172:0/3732519545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe524004800 0x7fe524004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 --2- 172.21.15.172:0/3732519545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5240051c0 0x7fe52400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c0cf200 con 0x7fe524004800 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe52400c060 con 0x7fe5240039b0 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe52400c260 con 0x7fe5240051c0 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53affd640 1 --2- 172.21.15.172:0/3732519545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe524004800 0x7fe524004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53b7fe640 1 --2- 172.21.15.172:0/3732519545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5240039b0 0x7fe524003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53bfff640 1 --2- 172.21.15.172:0/3732519545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5240051c0 0x7fe52400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53affd640 1 -- 172.21.15.172:0/3732519545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5240051c0 msgr2=0x7fe52400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53affd640 1 --2- 172.21.15.172:0/3732519545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5240051c0 0x7fe52400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53affd640 1 -- 172.21.15.172:0/3732519545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5240039b0 msgr2=0x7fe524003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53affd640 1 --2- 172.21.15.172:0/3732519545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5240039b0 0x7fe524003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:39.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53affd640 1 -- 172.21.15.172:0/3732519545 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe524003520 con 0x7fe524004800 2024-04-14T08:42:39.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.007+0000 7fe53bfff640 1 -- 172.21.15.172:0/3732519545 reap_dead start 2024-04-14T08:42:39.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.008+0000 7fe53affd640 1 --2- 172.21.15.172:0/3732519545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe524004800 0x7fe524004c80 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe530009ac0 tx=0x7fe524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f7a2817fbf8dc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:39.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.008+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530039070 con 0x7fe524004800 2024-04-14T08:42:39.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.008+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53002c6b0 con 0x7fe524004800 2024-04-14T08:42:39.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.008+0000 7fe53a7fc640 1 -- 172.21.15.172:0/3732519545 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530009690 con 0x7fe524004800 2024-04-14T08:42:39.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.009+0000 7fe542036640 1 -- 172.21.15.172:0/3732519545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe524004800 msgr2=0x7fe524004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:39.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.009+0000 7fe542036640 1 --2- 172.21.15.172:0/3732519545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe524004800 0x7fe524004c80 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe530009ac0 tx=0x7fe524005830 comp rx=0 tx=0).stop 2024-04-14T08:42:39.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.009+0000 7fe542036640 1 -- 172.21.15.172:0/3732519545 shutdown_connections 2024-04-14T08:42:39.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.009+0000 7fe542036640 1 --2- 172.21.15.172:0/3732519545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe524004800 0x7fe524004c80 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:39.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.009+0000 7fe542036640 1 -- 172.21.15.172:0/3732519545 >> 172.21.15.172:0/3732519545 conn(0x7fe53c0d0e90 msgr2=0x7fe53c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:39.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.009+0000 7fe542036640 1 -- 172.21.15.172:0/3732519545 shutdown_connections 2024-04-14T08:42:39.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.009+0000 7fe542036640 1 -- 172.21.15.172:0/3732519545 wait complete. 2024-04-14T08:42:39.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.010+0000 7fe542036640 1 Processor -- start 2024-04-14T08:42:39.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.010+0000 7fe542036640 1 -- start start 2024-04-14T08:42:39.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe542036640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe524004800 0x7fe53c1f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:39.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe542036640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe53c14f860 0x7fe53c0aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:39.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe542036640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53c0acda0 0x7fe53c0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:39.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe542036640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c162b40 con 0x7fe53c0acda0 2024-04-14T08:42:39.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe542036640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1629c0 con 0x7fe524004800 2024-04-14T08:42:39.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe542036640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c162cc0 con 0x7fe53c14f860 2024-04-14T08:42:39.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53c0acda0 0x7fe53c0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:39.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53c0acda0 0x7fe53c0ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57344/0 (socket says 172.21.15.172:57344) 2024-04-14T08:42:39.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe53bfff640 1 -- 172.21.15.172:0/2286863910 learned_addr learned my addr 172.21.15.172:0/2286863910 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:39.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe53b7fe640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe524004800 0x7fe53c1f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:39.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe53affd640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe53c14f860 0x7fe53c0aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:39.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe53bfff640 1 -- 172.21.15.172:0/2286863910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe53c14f860 msgr2=0x7fe53c0aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe53bfff640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe53c14f860 0x7fe53c0aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe53bfff640 1 -- 172.21.15.172:0/2286863910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe524004800 msgr2=0x7fe53c1f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe53bfff640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe524004800 0x7fe53c1f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe53bfff640 1 -- 172.21.15.172:0/2286863910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c0ab690 con 0x7fe53c0acda0 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe53b7fe640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe524004800 0x7fe53c1f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe53bfff640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53c0acda0 0x7fe53c0ad220 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe53c08d4e0 tx=0x7fe534007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.011+0000 7fe523fff640 1 -- 172.21.15.172:0/2286863910 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53400ab10 con 0x7fe53c0acda0 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.012+0000 7fe523fff640 1 -- 172.21.15.172:0/2286863910 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe534018070 con 0x7fe53c0acda0 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.012+0000 7fe542036640 1 -- 172.21.15.172:0/2286863910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53c0ab970 con 0x7fe53c0acda0 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.012+0000 7fe523fff640 1 -- 172.21.15.172:0/2286863910 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5340133f0 con 0x7fe53c0acda0 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.012+0000 7fe542036640 1 -- 172.21.15.172:0/2286863910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53c0abf30 con 0x7fe53c0acda0 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.012+0000 7fe542036640 1 -- 172.21.15.172:0/2286863910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53c068590 con 0x7fe53c0acda0 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.013+0000 7fe523fff640 1 -- 172.21.15.172:0/2286863910 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe534013590 con 0x7fe53c0acda0 2024-04-14T08:42:39.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.013+0000 7fe523fff640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe52806c8f0 0x7fe52806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:39.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.014+0000 7fe523fff640 1 -- 172.21.15.172:0/2286863910 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5619+0+0 (secure 0 0 0) 0x7fe53408c0b0 con 0x7fe53c0acda0 2024-04-14T08:42:39.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.014+0000 7fe53b7fe640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe52806c8f0 0x7fe52806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:39.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.014+0000 7fe53b7fe640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe52806c8f0 0x7fe52806edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe53c1533d0 tx=0x7fe52c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:39.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.018+0000 7fe523fff640 1 -- 172.21.15.172:0/2286863910 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe53405a530 con 0x7fe53c0acda0 2024-04-14T08:42:39.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.193+0000 7fe542036640 1 -- 172.21.15.172:0/2286863910 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe53c0d1bb0 con 0x7fe53c0acda0 2024-04-14T08:42:39.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.887+0000 7fe523fff640 1 -- 172.21.15.172:0/2286863910 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v114) ==== 170+0+0 (secure 0 0 0) 0x7fe534059e30 con 0x7fe53c0acda0 2024-04-14T08:42:39.887 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' created 2024-04-14T08:42:39.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.889+0000 7fe542036640 1 -- 172.21.15.172:0/2286863910 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe52806c8f0 msgr2=0x7fe52806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:39.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.889+0000 7fe542036640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe52806c8f0 0x7fe52806edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe53c1533d0 tx=0x7fe52c009040 comp rx=0 tx=0).stop 2024-04-14T08:42:39.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.889+0000 7fe542036640 1 -- 172.21.15.172:0/2286863910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53c0acda0 msgr2=0x7fe53c0ad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:39.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.889+0000 7fe542036640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53c0acda0 0x7fe53c0ad220 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe53c08d4e0 tx=0x7fe534007a90 comp rx=0 tx=0).stop 2024-04-14T08:42:39.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.890+0000 7fe542036640 1 -- 172.21.15.172:0/2286863910 shutdown_connections 2024-04-14T08:42:39.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.890+0000 7fe542036640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fe52806c8f0 0x7fe52806edb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:39.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.890+0000 7fe542036640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53c0acda0 0x7fe53c0ad220 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:39.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.890+0000 7fe542036640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe53c14f860 0x7fe53c0aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:39.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.890+0000 7fe542036640 1 --2- 172.21.15.172:0/2286863910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe524004800 0x7fe53c1f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:39.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.890+0000 7fe542036640 1 -- 172.21.15.172:0/2286863910 >> 172.21.15.172:0/2286863910 conn(0x7fe53c0d0e90 msgr2=0x7fe53c0d3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:39.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.890+0000 7fe542036640 1 -- 172.21.15.172:0/2286863910 shutdown_connections 2024-04-14T08:42:39.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:39.890+0000 7fe542036640 1 -- 172.21.15.172:0/2286863910 wait complete. 2024-04-14T08:42:39.913 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-14T08:42:40.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.021+0000 7f6df6495640 1 Processor -- start 2024-04-14T08:42:40.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.022+0000 7f6df6495640 1 -- start start 2024-04-14T08:42:40.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.022+0000 7f6df6495640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6df015e430 0x7f6df0162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:40.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.022+0000 7f6df6495640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6df00d2e00 0x7f6df01629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:40.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.022+0000 7f6df6495640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6df00cefb0 con 0x7f6df0162540 2024-04-14T08:42:40.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.022+0000 7f6df6495640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f6df00cf290 con 0x7f6df00d2e00 2024-04-14T08:42:40.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.022+0000 7f6df6495640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6df00cf570 con 0x7f6df015e430 2024-04-14T08:42:40.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.023+0000 7f6deffff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6df015e430 0x7f6df0162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:40.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.023+0000 7f6df4a32640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6df00d2e00 0x7f6df01629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:40.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.023+0000 7f6deffff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6df015e430 0x7f6df0162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:33434/0 (socket says 172.21.15.172:33434) 2024-04-14T08:42:40.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.023+0000 7f6def7fe640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f6df0162540 0x7f6df00b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:33620/0 (socket says 172.21.15.172:33620) 2024-04-14T08:42:40.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.023+0000 7f6deffff640 1 -- 172.21.15.172:0/854819456 learned_addr learned my addr 172.21.15.172:0/854819456 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:40.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.023+0000 7f6deffff640 1 -- 172.21.15.172:0/854819456 >> v2:172.21.15.177:3300/0 conn(0x7f6df00d2e00 msgr2=0x7f6df01629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:40.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.023+0000 7f6deffff640 1 --2- 172.21.15.172:0/854819456 >> v2:172.21.15.177:3300/0 conn(0x7f6df00d2e00 0x7f6df01629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:40.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.023+0000 7f6deffff640 1 -- 172.21.15.172:0/854819456 >> v1:172.21.15.177:6789/0 conn(0x7f6df0162540 legacy=0x7f6df00b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:40.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.023+0000 7f6deffff640 1 -- 172.21.15.172:0/854819456 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df00cf850 con 0x7f6df015e430 2024-04-14T08:42:40.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.023+0000 7f6deeffd640 1 -- 172.21.15.172:0/854819456 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2052043459 0 0) 0x7f6df00cefb0 con 0x7f6df0162540 2024-04-14T08:42:40.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.023+0000 7f6df4a32640 1 --2- 172.21.15.172:0/854819456 >> v2:172.21.15.177:3300/0 conn(0x7f6df00d2e00 0x7f6df01629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.024+0000 7f6deffff640 1 --2- 172.21.15.172:0/854819456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6df015e430 0x7f6df0162000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009870 tx=0x7f6de002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bfba67c9cf1de8e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.024+0000 7f6deeffd640 1 -- 172.21.15.172:0/854819456 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0034070 con 0x7f6df015e430 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.024+0000 7f6deeffd640 1 -- 172.21.15.172:0/854819456 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6de0030840 con 0x7f6df015e430 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.024+0000 7f6deeffd640 1 -- 172.21.15.172:0/854819456 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0030c10 con 0x7f6df015e430 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.024+0000 7f6df6495640 1 -- 172.21.15.172:0/854819456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6df015e430 msgr2=0x7f6df0162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.024+0000 7f6df6495640 1 --2- 172.21.15.172:0/854819456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6df015e430 0x7f6df0162000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009870 tx=0x7f6de002b980 comp rx=0 tx=0).stop 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.024+0000 7f6df6495640 1 -- 172.21.15.172:0/854819456 shutdown_connections 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.024+0000 7f6df6495640 1 --2- 172.21.15.172:0/854819456 >> v2:172.21.15.177:3300/0 conn(0x7f6df00d2e00 0x7f6df01629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.024+0000 7f6df6495640 1 --2- 172.21.15.172:0/854819456 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6df015e430 0x7f6df0162000 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.024+0000 7f6df6495640 1 -- 172.21.15.172:0/854819456 >> 172.21.15.172:0/854819456 conn(0x7f6df009d560 msgr2=0x7f6df00d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.024+0000 7f6df6495640 1 -- 172.21.15.172:0/854819456 shutdown_connections 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.024+0000 7f6df6495640 1 -- 172.21.15.172:0/854819456 wait complete. 2024-04-14T08:42:40.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.026+0000 7f6df6495640 1 Processor -- start 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.026+0000 7f6df6495640 1 -- start start 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.026+0000 7f6df6495640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6df00d2e00 0x7f6df00cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df6495640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6df015e430 0x7f6df01f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df6495640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6df0162540 0x7f6df00cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df6495640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6df00d4e20 con 0x7f6df0162540 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df6495640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6df015e810 con 0x7f6df00d2e00 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df6495640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f6df00d0050 con 0x7f6df015e430 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df4a32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6df0162540 0x7f6df00cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df4a32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6df0162540 0x7f6df00cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39730/0 (socket says 172.21.15.172:39730) 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df4a32640 1 -- 172.21.15.172:0/2715252994 learned_addr learned my addr 172.21.15.172:0/2715252994 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6def7fe640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6df015e430 0x7f6df01f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6deffff640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6df00d2e00 0x7f6df00cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:40.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df4a32640 1 -- 172.21.15.172:0/2715252994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6df015e430 msgr2=0x7f6df01f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df4a32640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6df015e430 0x7f6df01f5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df4a32640 1 -- 172.21.15.172:0/2715252994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6df00d2e00 msgr2=0x7f6df00cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df4a32640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6df00d2e00 0x7f6df00cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df4a32640 1 -- 172.21.15.172:0/2715252994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df00b7810 con 0x7f6df0162540 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df4a32640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6df0162540 0x7f6df00cd910 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f6de800efc0 tx=0x7f6de8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6decff9640 1 -- 172.21.15.172:0/2715252994 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de8018030 con 0x7f6df0162540 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6decff9640 1 -- 172.21.15.172:0/2715252994 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6de800cde0 con 0x7f6df0162540 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df6495640 1 -- 172.21.15.172:0/2715252994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df00b7af0 con 0x7f6df0162540 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6decff9640 1 -- 172.21.15.172:0/2715252994 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de8003bb0 con 0x7f6df0162540 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.027+0000 7f6df6495640 1 -- 172.21.15.172:0/2715252994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df00b8390 con 0x7f6df0162540 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.028+0000 7f6df6495640 1 -- 172.21.15.172:0/2715252994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df00b7cc0 con 0x7f6df0162540 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.029+0000 7f6decff9640 1 -- 172.21.15.172:0/2715252994 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6de801b070 con 0x7f6df0162540 2024-04-14T08:42:40.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.029+0000 7f6decff9640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6ddc06c8f0 0x7f6ddc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:40.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.030+0000 7f6deffff640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6ddc06c8f0 0x7f6ddc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:40.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.030+0000 7f6decff9640 1 -- 172.21.15.172:0/2715252994 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6001+0+0 (secure 0 0 0) 0x7f6de808c630 con 0x7f6df0162540 2024-04-14T08:42:40.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.030+0000 7f6deffff640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6ddc06c8f0 0x7f6ddc06edb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6de0036000 tx=0x7f6de0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:40.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.034+0000 7f6decff9640 1 -- 172.21.15.172:0/2715252994 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6de8010040 con 0x7f6df0162540 2024-04-14T08:42:40.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.250+0000 7f6df6495640 1 -- 172.21.15.172:0/2715252994 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6df00cea10 con 0x7f6df0162540 2024-04-14T08:42:40.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.889+0000 7f6decff9640 1 -- 172.21.15.172:0/2715252994 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v86) ==== 174+0+0 (secure 0 0 0) 0x7f6df00cea10 con 0x7f6df0162540 2024-04-14T08:42:40.889 INFO:teuthology.orchestra.run.smithi172.stderr:new fs with metadata pool 17 and data pool 18 2024-04-14T08:42:40.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.892+0000 7f6df6495640 1 -- 172.21.15.172:0/2715252994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6ddc06c8f0 msgr2=0x7f6ddc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:40.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.892+0000 7f6df6495640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6ddc06c8f0 0x7f6ddc06edb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6de0036000 tx=0x7f6de0008030 comp rx=0 tx=0).stop 2024-04-14T08:42:40.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.892+0000 7f6df6495640 1 -- 172.21.15.172:0/2715252994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6df0162540 msgr2=0x7f6df00cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:40.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.892+0000 7f6df6495640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6df0162540 0x7f6df00cd910 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f6de800efc0 tx=0x7f6de8009f90 comp rx=0 tx=0).stop 2024-04-14T08:42:40.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.892+0000 7f6df6495640 1 -- 172.21.15.172:0/2715252994 shutdown_connections 2024-04-14T08:42:40.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.892+0000 7f6df6495640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f6ddc06c8f0 0x7f6ddc06edb0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:40.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.892+0000 7f6df6495640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6df0162540 0x7f6df00cd910 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:40.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.892+0000 7f6df6495640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6df015e430 0x7f6df01f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:40.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.892+0000 7f6df6495640 1 --2- 172.21.15.172:0/2715252994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6df00d2e00 0x7f6df00cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:40.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.892+0000 7f6df6495640 1 -- 172.21.15.172:0/2715252994 >> 172.21.15.172:0/2715252994 conn(0x7f6df009d560 msgr2=0x7f6df015cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:40.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.892+0000 7f6df6495640 1 -- 172.21.15.172:0/2715252994 shutdown_connections 2024-04-14T08:42:40.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:40.892+0000 7f6df6495640 1 -- 172.21.15.172:0/2715252994 wait complete. 2024-04-14T08:42:40.920 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:42:41.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.026+0000 7f5c3a5c8640 1 Processor -- start 2024-04-14T08:42:41.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.026+0000 7f5c3a5c8640 1 -- start start 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c3a5c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5c34072970 0x7f5c3415b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c3a5c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c340b3360 con 0x7f5c34072470 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c3a5c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c340b3640 con 0x7f5c3415a690 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c3a5c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5c340b3920 con 0x7f5c34072970 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c337fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5c34072470 0x7f5c3415b230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34152/0 (socket says 172.21.15.172:34152) 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c337fe640 1 -- 172.21.15.172:0/863165039 learned_addr learned my addr 172.21.15.172:0/863165039 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c38b65640 1 --2- 172.21.15.172:0/863165039 >> v2:172.21.15.177:3300/0 conn(0x7f5c34072970 0x7f5c3415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c32ffd640 1 -- 172.21.15.172:0/863165039 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2839018387 0 0) 0x7f5c340b3360 con 0x7f5c34072470 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c32ffd640 1 -- 172.21.15.172:0/863165039 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c20003520 con 0x7f5c34072470 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c38b65640 1 -- 172.21.15.172:0/863165039 >> v1:172.21.15.177:6789/0 conn(0x7f5c3415a690 legacy=0x7f5c34071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c32ffd640 1 -- 172.21.15.172:0/863165039 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 383657170 0 0) 0x7f5c20003520 con 0x7f5c34072470 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c38b65640 1 -- 172.21.15.172:0/863165039 >> v1:172.21.15.172:6789/0 conn(0x7f5c34072470 legacy=0x7f5c3415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.027+0000 7f5c38b65640 1 -- 172.21.15.172:0/863165039 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c340b3c00 con 0x7f5c34072970 2024-04-14T08:42:41.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.028+0000 7f5c38b65640 1 --2- 172.21.15.172:0/863165039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c34072970 0x7f5c3415b940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009d90 tx=0x7f5c2c02bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3994218bf1fd14d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:41.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.028+0000 7f5c32ffd640 1 -- 172.21.15.172:0/863165039 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c039070 con 0x7f5c34072970 2024-04-14T08:42:41.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.028+0000 7f5c32ffd640 1 -- 172.21.15.172:0/863165039 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2c02c8b0 con 0x7f5c34072970 2024-04-14T08:42:41.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.028+0000 7f5c32ffd640 1 -- 172.21.15.172:0/863165039 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c001670 con 0x7f5c34072970 2024-04-14T08:42:41.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.029+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/863165039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c34072970 msgr2=0x7f5c3415b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.029+0000 7f5c3a5c8640 1 --2- 172.21.15.172:0/863165039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c34072970 0x7f5c3415b940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009d90 tx=0x7f5c2c02bd80 comp rx=0 tx=0).stop 2024-04-14T08:42:41.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.029+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/863165039 shutdown_connections 2024-04-14T08:42:41.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.029+0000 7f5c3a5c8640 1 --2- 172.21.15.172:0/863165039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c34072970 0x7f5c3415b940 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.029+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/863165039 >> 172.21.15.172:0/863165039 conn(0x7f5c3406ec30 msgr2=0x7f5c3406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:41.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.029+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/863165039 shutdown_connections 2024-04-14T08:42:41.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.029+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/863165039 wait complete. 2024-04-14T08:42:41.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.030+0000 7f5c3a5c8640 1 Processor -- start 2024-04-14T08:42:41.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.030+0000 7f5c3a5c8640 1 -- start start 2024-04-14T08:42:41.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c3a5c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c34072470 0x7f5c3414d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c3a5c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c34072970 0x7f5c341447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c3a5c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3415a690 0x7f5c340b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c3a5c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3415abf0 con 0x7f5c34072970 2024-04-14T08:42:41.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c3a5c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3415aa70 con 0x7f5c3415a690 2024-04-14T08:42:41.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c3a5c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5c3415ad70 con 0x7f5c34072470 2024-04-14T08:42:41.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c337fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c34072970 0x7f5c341447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c337fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c34072970 0x7f5c341447c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39746/0 (socket says 172.21.15.172:39746) 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c337fe640 1 -- 172.21.15.172:0/3962172193 learned_addr learned my addr 172.21.15.172:0/3962172193 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c38b65640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3415a690 0x7f5c340b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c33fff640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c34072470 0x7f5c3414d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c337fe640 1 -- 172.21.15.172:0/3962172193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c34072470 msgr2=0x7f5c3414d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.031+0000 7f5c337fe640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c34072470 0x7f5c3414d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.032+0000 7f5c337fe640 1 -- 172.21.15.172:0/3962172193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3415a690 msgr2=0x7f5c340b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.032+0000 7f5c337fe640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3415a690 0x7f5c340b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.032+0000 7f5c337fe640 1 -- 172.21.15.172:0/3962172193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3414a0e0 con 0x7f5c34072970 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.032+0000 7f5c337fe640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c34072970 0x7f5c341447c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00c760 tx=0x7f5c1c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.032+0000 7f5c30ff9640 1 -- 172.21.15.172:0/3962172193 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c017070 con 0x7f5c34072970 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.032+0000 7f5c30ff9640 1 -- 172.21.15.172:0/3962172193 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c1c005af0 con 0x7f5c34072970 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.032+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/3962172193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c34148530 con 0x7f5c34072970 2024-04-14T08:42:41.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.032+0000 7f5c30ff9640 1 -- 172.21.15.172:0/3962172193 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c007070 con 0x7f5c34072970 2024-04-14T08:42:41.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.032+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/3962172193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c34148af0 con 0x7f5c34072970 2024-04-14T08:42:41.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.033+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/3962172193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c340b5330 con 0x7f5c34072970 2024-04-14T08:42:41.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.033+0000 7f5c30ff9640 1 -- 172.21.15.172:0/3962172193 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c1c01a070 con 0x7f5c34072970 2024-04-14T08:42:41.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.034+0000 7f5c30ff9640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5c1406c8f0 0x7f5c1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.034+0000 7f5c30ff9640 1 -- 172.21.15.172:0/3962172193 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6101+0+0 (secure 0 0 0) 0x7f5c1c08b100 con 0x7f5c34072970 2024-04-14T08:42:41.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.034+0000 7f5c33fff640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5c1406c8f0 0x7f5c1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.034+0000 7f5c33fff640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5c1406c8f0 0x7f5c1406edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5c340cfba0 tx=0x7f5c24009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:41.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.038+0000 7f5c30ff9640 1 -- 172.21.15.172:0/3962172193 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5c1c058de0 con 0x7f5c34072970 2024-04-14T08:42:41.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.217+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/3962172193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c34068520 con 0x7f5c34072970 2024-04-14T08:42:41.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.218+0000 7f5c30ff9640 1 -- 172.21.15.172:0/3962172193 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17025 (secure 0 0 0) 0x7f5c1c05d0e0 con 0x7f5c34072970 2024-04-14T08:42:41.218 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:41.219 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":115,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:42:40.251220+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:42:38.455900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-14T08:42:39.198157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:42:41.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.220+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/3962172193 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5c1406c8f0 msgr2=0x7f5c1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.220+0000 7f5c3a5c8640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5c1406c8f0 0x7f5c1406edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5c340cfba0 tx=0x7f5c24009040 comp rx=0 tx=0).stop 2024-04-14T08:42:41.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.220+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/3962172193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c34072970 msgr2=0x7f5c341447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.220+0000 7f5c3a5c8640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c34072970 0x7f5c341447c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00c760 tx=0x7f5c1c00cc20 comp rx=0 tx=0).stop 2024-04-14T08:42:41.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.220+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/3962172193 shutdown_connections 2024-04-14T08:42:41.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.220+0000 7f5c3a5c8640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5c1406c8f0 0x7f5c1406edb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.220+0000 7f5c3a5c8640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3415a690 0x7f5c340b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.220+0000 7f5c3a5c8640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c34072970 0x7f5c341447c0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.220+0000 7f5c3a5c8640 1 --2- 172.21.15.172:0/3962172193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c34072470 0x7f5c3414d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.220+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/3962172193 >> 172.21.15.172:0/3962172193 conn(0x7f5c3406ec30 msgr2=0x7f5c34159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:41.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.220+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/3962172193 shutdown_connections 2024-04-14T08:42:41.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.220+0000 7f5c3a5c8640 1 -- 172.21.15.172:0/3962172193 wait complete. 2024-04-14T08:42:41.232 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:42:41.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.332+0000 7f122ede0640 1 Processor -- start 2024-04-14T08:42:41.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.332+0000 7f122ede0640 1 -- start start 2024-04-14T08:42:41.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.333+0000 7f122ede0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f122815e890 0x7f12280d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.333+0000 7f122ede0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12280cf320 con 0x7f12280d2830 2024-04-14T08:42:41.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.333+0000 7f122ede0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12280cf600 con 0x7f12280cedd0 2024-04-14T08:42:41.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.333+0000 7f122ede0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f12280cff30 con 0x7f122815e890 2024-04-14T08:42:41.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.333+0000 7f1227fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f122815e890 0x7f12280d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.333+0000 7f122cb7c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f12280d2830 0x7f12280d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34168/0 (socket says 172.21.15.172:34168) 2024-04-14T08:42:41.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.333+0000 7f122cb7c640 1 -- 172.21.15.172:0/2379923126 learned_addr learned my addr 172.21.15.172:0/2379923126 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:41.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.334+0000 7f1227fff640 1 -- 172.21.15.172:0/2379923126 >> v1:172.21.15.177:6789/0 conn(0x7f12280cedd0 legacy=0x7f12280b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:41.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.334+0000 7f1227fff640 1 -- 172.21.15.172:0/2379923126 >> v1:172.21.15.172:6789/0 conn(0x7f12280d2830 legacy=0x7f12280d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.334+0000 7f12277fe640 1 -- 172.21.15.172:0/2379923126 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902997822 0 0) 0x7f12280cf320 con 0x7f12280d2830 2024-04-14T08:42:41.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.334+0000 7f1227fff640 1 -- 172.21.15.172:0/2379923126 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12280cf8e0 con 0x7f122815e890 2024-04-14T08:42:41.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.334+0000 7f1227fff640 1 --2- 172.21.15.172:0/2379923126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f122815e890 0x7f12280d1300 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f121c0097a0 tx=0x7f121c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920b9de8a92c000e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:41.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.335+0000 7f12277fe640 1 -- 172.21.15.172:0/2379923126 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c034070 con 0x7f122815e890 2024-04-14T08:42:41.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.335+0000 7f12277fe640 1 -- 172.21.15.172:0/2379923126 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f121c030610 con 0x7f122815e890 2024-04-14T08:42:41.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.335+0000 7f12277fe640 1 -- 172.21.15.172:0/2379923126 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c0309e0 con 0x7f122815e890 2024-04-14T08:42:41.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.335+0000 7f122ede0640 1 -- 172.21.15.172:0/2379923126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f122815e890 msgr2=0x7f12280d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.335+0000 7f122ede0640 1 --2- 172.21.15.172:0/2379923126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f122815e890 0x7f12280d1300 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f121c0097a0 tx=0x7f121c02b960 comp rx=0 tx=0).stop 2024-04-14T08:42:41.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.335+0000 7f122ede0640 1 -- 172.21.15.172:0/2379923126 shutdown_connections 2024-04-14T08:42:41.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.335+0000 7f122ede0640 1 --2- 172.21.15.172:0/2379923126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f122815e890 0x7f12280d1300 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.335+0000 7f122ede0640 1 -- 172.21.15.172:0/2379923126 >> 172.21.15.172:0/2379923126 conn(0x7f122809d590 msgr2=0x7f12280d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:41.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.335+0000 7f122ede0640 1 -- 172.21.15.172:0/2379923126 shutdown_connections 2024-04-14T08:42:41.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.335+0000 7f122ede0640 1 -- 172.21.15.172:0/2379923126 wait complete. 2024-04-14T08:42:41.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.336+0000 7f122ede0640 1 Processor -- start 2024-04-14T08:42:41.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.336+0000 7f122ede0640 1 -- start start 2024-04-14T08:42:41.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.337+0000 7f122ede0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12280cedd0 0x7f12280b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.337+0000 7f122ede0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12280d2830 0x7f12281593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.337+0000 7f122ede0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f122815e890 0x7f1228161f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.337+0000 7f122ede0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12280d1840 con 0x7f12280cedd0 2024-04-14T08:42:41.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.337+0000 7f122ede0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f12280d2350 con 0x7f122815e890 2024-04-14T08:42:41.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.337+0000 7f122ede0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f12280b3e20 con 0x7f12280d2830 2024-04-14T08:42:41.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.337+0000 7f122cb7c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12280cedd0 0x7f12280b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.337+0000 7f122cb7c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12280cedd0 0x7f12280b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39754/0 (socket says 172.21.15.172:39754) 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.337+0000 7f122cb7c640 1 -- 172.21.15.172:0/2429670311 learned_addr learned my addr 172.21.15.172:0/2429670311 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.337+0000 7f122d37d640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f122815e890 0x7f1228161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.337+0000 7f1227fff640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12280d2830 0x7f12281593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.338+0000 7f122cb7c640 1 -- 172.21.15.172:0/2429670311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12280d2830 msgr2=0x7f12281593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.338+0000 7f122cb7c640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12280d2830 0x7f12281593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.338+0000 7f122cb7c640 1 -- 172.21.15.172:0/2429670311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f122815e890 msgr2=0x7f1228161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.338+0000 7f122cb7c640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f122815e890 0x7f1228161f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.338+0000 7f122cb7c640 1 -- 172.21.15.172:0/2429670311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12280ca6f0 con 0x7f12280cedd0 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.338+0000 7f1227fff640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12280d2830 0x7f12281593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.338+0000 7f122cb7c640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12280cedd0 0x7f12280b6e70 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f121800c4d0 tx=0x7f121800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.338+0000 7f12257fa640 1 -- 172.21.15.172:0/2429670311 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218018070 con 0x7f12280cedd0 2024-04-14T08:42:41.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.338+0000 7f12257fa640 1 -- 172.21.15.172:0/2429670311 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1218007500 con 0x7f12280cedd0 2024-04-14T08:42:41.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.338+0000 7f122ede0640 1 -- 172.21.15.172:0/2429670311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12280ca9d0 con 0x7f12280cedd0 2024-04-14T08:42:41.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.338+0000 7f12257fa640 1 -- 172.21.15.172:0/2429670311 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218004a20 con 0x7f12280cedd0 2024-04-14T08:42:41.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.338+0000 7f122ede0640 1 -- 172.21.15.172:0/2429670311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1228160220 con 0x7f12280cedd0 2024-04-14T08:42:41.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.339+0000 7f122ede0640 1 -- 172.21.15.172:0/2429670311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1228068520 con 0x7f12280cedd0 2024-04-14T08:42:41.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.340+0000 7f12257fa640 1 -- 172.21.15.172:0/2429670311 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f121801b070 con 0x7f12280cedd0 2024-04-14T08:42:41.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.340+0000 7f12257fa640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f121406c8f0 0x7f121406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.340+0000 7f12257fa640 1 -- 172.21.15.172:0/2429670311 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6101+0+0 (secure 0 0 0) 0x7f121808afe0 con 0x7f12280cedd0 2024-04-14T08:42:41.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.340+0000 7f1227fff640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f121406c8f0 0x7f121406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.340+0000 7f1227fff640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f121406c8f0 0x7f121406edb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f121c036000 tx=0x7f121c03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:41.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.345+0000 7f12257fa640 1 -- 172.21.15.172:0/2429670311 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1218058c30 con 0x7f12280cedd0 2024-04-14T08:42:41.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.520+0000 7f122ede0640 1 -- 172.21.15.172:0/2429670311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12280d2610 con 0x7f12280cedd0 2024-04-14T08:42:41.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.521+0000 7f12257fa640 1 -- 172.21.15.172:0/2429670311 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+17025 (secure 0 0 0) 0x7f121805cf30 con 0x7f12280cedd0 2024-04-14T08:42:41.521 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:41.522 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":115,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:42:40.251220+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:42:38.455900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-14T08:42:39.198157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:42:41.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.523+0000 7f122ede0640 1 -- 172.21.15.172:0/2429670311 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f121406c8f0 msgr2=0x7f121406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.523+0000 7f122ede0640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f121406c8f0 0x7f121406edb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f121c036000 tx=0x7f121c03b040 comp rx=0 tx=0).stop 2024-04-14T08:42:41.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.523+0000 7f122ede0640 1 -- 172.21.15.172:0/2429670311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12280cedd0 msgr2=0x7f12280b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.523+0000 7f122ede0640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12280cedd0 0x7f12280b6e70 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f121800c4d0 tx=0x7f121800c990 comp rx=0 tx=0).stop 2024-04-14T08:42:41.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.523+0000 7f122ede0640 1 -- 172.21.15.172:0/2429670311 shutdown_connections 2024-04-14T08:42:41.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.523+0000 7f122ede0640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f121406c8f0 0x7f121406edb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.523+0000 7f122ede0640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f122815e890 0x7f1228161f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.523+0000 7f122ede0640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12280d2830 0x7f12281593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.523+0000 7f122ede0640 1 --2- 172.21.15.172:0/2429670311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12280cedd0 0x7f12280b6e70 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.523+0000 7f122ede0640 1 -- 172.21.15.172:0/2429670311 >> 172.21.15.172:0/2429670311 conn(0x7f122809d590 msgr2=0x7f12280b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:41.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.523+0000 7f122ede0640 1 -- 172.21.15.172:0/2429670311 shutdown_connections 2024-04-14T08:42:41.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.523+0000 7f122ede0640 1 -- 172.21.15.172:0/2429670311 wait complete. 2024-04-14T08:42:41.535 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-14T08:42:41.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.638+0000 7f12267bb640 1 Processor -- start 2024-04-14T08:42:41.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.638+0000 7f12267bb640 1 -- start start 2024-04-14T08:42:41.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.639+0000 7f12267bb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f12200cca00 0x7f1220072ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.639+0000 7f12267bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12200ceb40 0x7f122006e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.639+0000 7f12267bb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1220072430 con 0x7f12200b0070 2024-04-14T08:42:41.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.639+0000 7f12267bb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f1220073410 con 0x7f12200cca00 2024-04-14T08:42:41.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.639+0000 7f12267bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1220072a40 con 0x7f12200ceb40 2024-04-14T08:42:41.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.639+0000 7f121f7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f12200b0070 0x7f122006b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34178/0 (socket says 172.21.15.172:34178) 2024-04-14T08:42:41.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.639+0000 7f121f7fe640 1 -- 172.21.15.172:0/1937099012 learned_addr learned my addr 172.21.15.172:0/1937099012 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:41.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.639+0000 7f121ffff640 1 --2- 172.21.15.172:0/1937099012 >> v2:172.21.15.177:3300/0 conn(0x7f12200cca00 0x7f1220072ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.639+0000 7f1224d58640 1 --2- 172.21.15.172:0/1937099012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12200ceb40 0x7f122006e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.640+0000 7f121effd640 1 -- 172.21.15.172:0/1937099012 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2797345033 0 0) 0x7f1220072430 con 0x7f12200b0070 2024-04-14T08:42:41.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.640+0000 7f121effd640 1 -- 172.21.15.172:0/1937099012 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1208003520 con 0x7f12200b0070 2024-04-14T08:42:41.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.640+0000 7f121ffff640 1 -- 172.21.15.172:0/1937099012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12200ceb40 msgr2=0x7f122006e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:41.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.640+0000 7f121ffff640 1 --2- 172.21.15.172:0/1937099012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12200ceb40 0x7f122006e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.640+0000 7f121ffff640 1 -- 172.21.15.172:0/1937099012 >> v1:172.21.15.172:6789/0 conn(0x7f12200b0070 legacy=0x7f122006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.640+0000 7f121ffff640 1 -- 172.21.15.172:0/1937099012 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12200b0500 con 0x7f12200cca00 2024-04-14T08:42:41.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.640+0000 7f1224d58640 1 --2- 172.21.15.172:0/1937099012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12200ceb40 0x7f122006e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:41.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.641+0000 7f121ffff640 1 --2- 172.21.15.172:0/1937099012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12200cca00 0x7f1220072ed0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f12100097a0 tx=0x7f121002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4aa69764d3c9434 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:41.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.641+0000 7f121effd640 1 -- 172.21.15.172:0/1937099012 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210035020 con 0x7f12200cca00 2024-04-14T08:42:41.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.641+0000 7f121effd640 1 -- 172.21.15.172:0/1937099012 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1210031830 con 0x7f12200cca00 2024-04-14T08:42:41.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.641+0000 7f121effd640 1 -- 172.21.15.172:0/1937099012 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210031be0 con 0x7f12200cca00 2024-04-14T08:42:41.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.641+0000 7f12267bb640 1 -- 172.21.15.172:0/1937099012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12200cca00 msgr2=0x7f1220072ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.641+0000 7f12267bb640 1 --2- 172.21.15.172:0/1937099012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12200cca00 0x7f1220072ed0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f12100097a0 tx=0x7f121002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.641+0000 7f12267bb640 1 -- 172.21.15.172:0/1937099012 shutdown_connections 2024-04-14T08:42:41.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.641+0000 7f12267bb640 1 --2- 172.21.15.172:0/1937099012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12200ceb40 0x7f122006e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.641+0000 7f12267bb640 1 --2- 172.21.15.172:0/1937099012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12200cca00 0x7f1220072ed0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.641+0000 7f12267bb640 1 -- 172.21.15.172:0/1937099012 >> 172.21.15.172:0/1937099012 conn(0x7f12200cd980 msgr2=0x7f12200cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:41.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.641+0000 7f12267bb640 1 -- 172.21.15.172:0/1937099012 shutdown_connections 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.641+0000 7f12267bb640 1 -- 172.21.15.172:0/1937099012 wait complete. 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.642+0000 7f12267bb640 1 Processor -- start 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.643+0000 7f12267bb640 1 -- start start 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.643+0000 7f12267bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12200cca00 0x7f12201dc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.643+0000 7f12267bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12200ceb40 0x7f12200c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.643+0000 7f12267bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12200c9f70 0x7f12200b9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.643+0000 7f12267bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12200ccde0 con 0x7f12200c9f70 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.643+0000 7f12267bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f12200b07e0 con 0x7f12200cca00 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.643+0000 7f12267bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f12200ccf60 con 0x7f12200ceb40 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.643+0000 7f1224d58640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12200c9f70 0x7f12200b9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.643+0000 7f1224d58640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12200c9f70 0x7f12200b9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39756/0 (socket says 172.21.15.172:39756) 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f1224d58640 1 -- 172.21.15.172:0/1137924347 learned_addr learned my addr 172.21.15.172:0/1137924347 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:41.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f121ffff640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12200cca00 0x7f12201dc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f121f7fe640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12200ceb40 0x7f12200c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f1224d58640 1 -- 172.21.15.172:0/1137924347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12200ceb40 msgr2=0x7f12200c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f1224d58640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12200ceb40 0x7f12200c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f1224d58640 1 -- 172.21.15.172:0/1137924347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12200cca00 msgr2=0x7f12201dc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f1224d58640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12200cca00 0x7f12201dc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f1224d58640 1 -- 172.21.15.172:0/1137924347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12200ca480 con 0x7f12200c9f70 2024-04-14T08:42:41.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f1224d58640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12200c9f70 0x7f12200b9e70 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f121800efb0 tx=0x7f121800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:41.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f121cff9640 1 -- 172.21.15.172:0/1137924347 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218018020 con 0x7f12200c9f70 2024-04-14T08:42:41.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f121cff9640 1 -- 172.21.15.172:0/1137924347 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1218014480 con 0x7f12200c9f70 2024-04-14T08:42:41.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f12267bb640 1 -- 172.21.15.172:0/1137924347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12200ca760 con 0x7f12200c9f70 2024-04-14T08:42:41.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f121cff9640 1 -- 172.21.15.172:0/1137924347 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218014850 con 0x7f12200c9f70 2024-04-14T08:42:41.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.644+0000 7f12267bb640 1 -- 172.21.15.172:0/1137924347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12201dc9e0 con 0x7f12200c9f70 2024-04-14T08:42:41.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.645+0000 7f12267bb640 1 -- 172.21.15.172:0/1137924347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1220068520 con 0x7f12200c9f70 2024-04-14T08:42:41.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.646+0000 7f121cff9640 1 -- 172.21.15.172:0/1137924347 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f121801b020 con 0x7f12200c9f70 2024-04-14T08:42:41.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.646+0000 7f121cff9640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f120c06c8f0 0x7f120c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:41.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.646+0000 7f121ffff640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f120c06c8f0 0x7f120c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:41.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.646+0000 7f121cff9640 1 -- 172.21.15.172:0/1137924347 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 6101+0+0 (secure 0 0 0) 0x7f121808ca30 con 0x7f12200c9f70 2024-04-14T08:42:41.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.647+0000 7f121ffff640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f120c06c8f0 0x7f120c06edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1210036000 tx=0x7f1210008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:41.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.651+0000 7f121cff9640 1 -- 172.21.15.172:0/1137924347 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1218022020 con 0x7f12200c9f70 2024-04-14T08:42:41.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.861+0000 7f12267bb640 1 -- 172.21.15.172:0/1137924347 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f122006c050 con 0x7f12200c9f70 2024-04-14T08:42:41.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.906+0000 7f121cff9640 1 -- 172.21.15.172:0/1137924347 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v87) ==== 118+0+0 (secure 0 0 0) 0x7f1218010040 con 0x7f12200c9f70 2024-04-14T08:42:41.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.909+0000 7f12267bb640 1 -- 172.21.15.172:0/1137924347 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f120c06c8f0 msgr2=0x7f120c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.909+0000 7f12267bb640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f120c06c8f0 0x7f120c06edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1210036000 tx=0x7f1210008030 comp rx=0 tx=0).stop 2024-04-14T08:42:41.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.909+0000 7f12267bb640 1 -- 172.21.15.172:0/1137924347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12200c9f70 msgr2=0x7f12200b9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:41.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.909+0000 7f12267bb640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12200c9f70 0x7f12200b9e70 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f121800efb0 tx=0x7f121800c490 comp rx=0 tx=0).stop 2024-04-14T08:42:41.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.909+0000 7f12267bb640 1 -- 172.21.15.172:0/1137924347 shutdown_connections 2024-04-14T08:42:41.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.909+0000 7f12267bb640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f120c06c8f0 0x7f120c06edb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.909+0000 7f12267bb640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12200c9f70 0x7f12200b9e70 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.909+0000 7f12267bb640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12200ceb40 0x7f12200c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.909+0000 7f12267bb640 1 --2- 172.21.15.172:0/1137924347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12200cca00 0x7f12201dc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:41.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.909+0000 7f12267bb640 1 -- 172.21.15.172:0/1137924347 >> 172.21.15.172:0/1137924347 conn(0x7f12200cd980 msgr2=0x7f12200b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:41.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.909+0000 7f12267bb640 1 -- 172.21.15.172:0/1137924347 shutdown_connections 2024-04-14T08:42:41.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:41.909+0000 7f12267bb640 1 -- 172.21.15.172:0/1137924347 wait complete. 2024-04-14T08:42:41.921 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-14T08:42:41.921 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-14T08:42:42.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.025+0000 7fcfbebc8640 1 Processor -- start 2024-04-14T08:42:42.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.025+0000 7fcfbebc8640 1 -- start start 2024-04-14T08:42:42.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.025+0000 7fcfbebc8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcfb815e590 0x7fcfb81629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:42.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.025+0000 7fcfbebc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb80b65a0 0x7fcfb80d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:42.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.025+0000 7fcfbebc8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfb80b3750 con 0x7fcfb80cea60 2024-04-14T08:42:42.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.025+0000 7fcfbebc8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fcfb80cef20 con 0x7fcfb815e590 2024-04-14T08:42:42.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.025+0000 7fcfbebc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb80cf200 con 0x7fcfb80b65a0 2024-04-14T08:42:42.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.026+0000 7fcfbc964640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcfb815e590 0x7fcfb81629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:42.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.026+0000 7fcfbc964640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcfb815e590 0x7fcfb81629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39770/0 (socket says 172.21.15.172:39770) 2024-04-14T08:42:42.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.026+0000 7fcfbc964640 1 -- 172.21.15.172:0/124467996 learned_addr learned my addr 172.21.15.172:0/124467996 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:42.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.026+0000 7fcfbd165640 1 --2- 172.21.15.172:0/124467996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb80b65a0 0x7fcfb80d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:42.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.026+0000 7fcfbc964640 1 -- 172.21.15.172:0/124467996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb80b65a0 msgr2=0x7fcfb80d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:42.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.026+0000 7fcfbc964640 1 --2- 172.21.15.172:0/124467996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb80b65a0 0x7fcfb80d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:42.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.026+0000 7fcfbc964640 1 -- 172.21.15.172:0/124467996 >> v1:172.21.15.177:6789/0 conn(0x7fcfb80cea60 legacy=0x7fcfb80b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:42.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.026+0000 7fcfbc964640 1 -- 172.21.15.172:0/124467996 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb80cf4e0 con 0x7fcfb815e590 2024-04-14T08:42:42.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.027+0000 7fcfbc964640 1 --2- 172.21.15.172:0/124467996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb815e590 0x7fcfb81629c0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009870 tx=0x7fcfac02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0eb1ace72b02ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:42.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.027+0000 7fcfb77fe640 1 -- 172.21.15.172:0/124467996 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac034070 con 0x7fcfb815e590 2024-04-14T08:42:42.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.027+0000 7fcfb77fe640 1 -- 172.21.15.172:0/124467996 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfac030840 con 0x7fcfb815e590 2024-04-14T08:42:42.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.027+0000 7fcfb77fe640 1 -- 172.21.15.172:0/124467996 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac030c10 con 0x7fcfb815e590 2024-04-14T08:42:42.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.027+0000 7fcfbebc8640 1 -- 172.21.15.172:0/124467996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb815e590 msgr2=0x7fcfb81629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:42.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.027+0000 7fcfbebc8640 1 --2- 172.21.15.172:0/124467996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb815e590 0x7fcfb81629c0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009870 tx=0x7fcfac02b980 comp rx=0 tx=0).stop 2024-04-14T08:42:42.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.027+0000 7fcfbebc8640 1 -- 172.21.15.172:0/124467996 shutdown_connections 2024-04-14T08:42:42.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.027+0000 7fcfbebc8640 1 --2- 172.21.15.172:0/124467996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb80b65a0 0x7fcfb80d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:42.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.027+0000 7fcfbebc8640 1 --2- 172.21.15.172:0/124467996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb815e590 0x7fcfb81629c0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:42.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.027+0000 7fcfbebc8640 1 -- 172.21.15.172:0/124467996 >> 172.21.15.172:0/124467996 conn(0x7fcfb80d0e90 msgr2=0x7fcfb80d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:42.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.027+0000 7fcfbebc8640 1 -- 172.21.15.172:0/124467996 shutdown_connections 2024-04-14T08:42:42.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.027+0000 7fcfbebc8640 1 -- 172.21.15.172:0/124467996 wait complete. 2024-04-14T08:42:42.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.028+0000 7fcfbebc8640 1 Processor -- start 2024-04-14T08:42:42.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.028+0000 7fcfbebc8640 1 -- start start 2024-04-14T08:42:42.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.029+0000 7fcfbebc8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80b65a0 0x7fcfb80cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:42.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.029+0000 7fcfbebc8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb80cea60 0x7fcfb80cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:42.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.029+0000 7fcfbebc8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb815e590 0x7fcfb8157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:42.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.029+0000 7fcfbebc8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb80d2e20 con 0x7fcfb80cea60 2024-04-14T08:42:42.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.029+0000 7fcfbebc8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb80d2ca0 con 0x7fcfb80b65a0 2024-04-14T08:42:42.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.029+0000 7fcfbebc8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb80d2fa0 con 0x7fcfb815e590 2024-04-14T08:42:42.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.029+0000 7fcfb7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb80cea60 0x7fcfb80cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:42.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.029+0000 7fcfb7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb80cea60 0x7fcfb80cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39786/0 (socket says 172.21.15.172:39786) 2024-04-14T08:42:42.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.029+0000 7fcfb7fff640 1 -- 172.21.15.172:0/854013271 learned_addr learned my addr 172.21.15.172:0/854013271 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:42.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.029+0000 7fcfbc964640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80b65a0 0x7fcfb80cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:42.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.029+0000 7fcfbd165640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb815e590 0x7fcfb8157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:42.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.030+0000 7fcfb7fff640 1 -- 172.21.15.172:0/854013271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb815e590 msgr2=0x7fcfb8157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:42.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.030+0000 7fcfb7fff640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb815e590 0x7fcfb8157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:42.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.030+0000 7fcfb7fff640 1 -- 172.21.15.172:0/854013271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80b65a0 msgr2=0x7fcfb80cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:42.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.030+0000 7fcfb7fff640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80b65a0 0x7fcfb80cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:42.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.030+0000 7fcfb7fff640 1 -- 172.21.15.172:0/854013271 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb81e9ef0 con 0x7fcfb80cea60 2024-04-14T08:42:42.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.030+0000 7fcfbc964640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80b65a0 0x7fcfb80cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:42.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.030+0000 7fcfb7fff640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb80cea60 0x7fcfb80cba20 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400c430 tx=0x7fcfa400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:42.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.030+0000 7fcfb57fa640 1 -- 172.21.15.172:0/854013271 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40052d0 con 0x7fcfb80cea60 2024-04-14T08:42:42.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.030+0000 7fcfb57fa640 1 -- 172.21.15.172:0/854013271 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa4005490 con 0x7fcfb80cea60 2024-04-14T08:42:42.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.030+0000 7fcfbebc8640 1 -- 172.21.15.172:0/854013271 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb81ea1d0 con 0x7fcfb80cea60 2024-04-14T08:42:42.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.030+0000 7fcfb57fa640 1 -- 172.21.15.172:0/854013271 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4007260 con 0x7fcfb80cea60 2024-04-14T08:42:42.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.030+0000 7fcfbebc8640 1 -- 172.21.15.172:0/854013271 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb81ea790 con 0x7fcfb80cea60 2024-04-14T08:42:42.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.031+0000 7fcfbebc8640 1 -- 172.21.15.172:0/854013271 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb80b4350 con 0x7fcfb80cea60 2024-04-14T08:42:42.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.032+0000 7fcfb57fa640 1 -- 172.21.15.172:0/854013271 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfa4007400 con 0x7fcfb80cea60 2024-04-14T08:42:42.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.032+0000 7fcfb57fa640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcf9c06c8f0 0x7fcf9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:42.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.032+0000 7fcfbc964640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcf9c06c8f0 0x7fcf9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:42.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.032+0000 7fcfb57fa640 1 -- 172.21.15.172:0/854013271 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7fcfa408b0e0 con 0x7fcfb80cea60 2024-04-14T08:42:42.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.033+0000 7fcfbc964640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcf9c06c8f0 0x7fcf9c06edb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcfac036000 tx=0x7fcfac03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:42.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.037+0000 7fcfb57fa640 1 -- 172.21.15.172:0/854013271 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcfb80b4350 con 0x7fcfb80cea60 2024-04-14T08:42:42.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.248+0000 7fcfbebc8640 1 -- 172.21.15.172:0/854013271 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcfb80d1bb0 con 0x7fcfb80cea60 2024-04-14T08:42:42.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.916+0000 7fcfb57fa640 1 -- 172.21.15.172:0/854013271 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v88) ==== 122+0+0 (secure 0 0 0) 0x7fcfa4058dc0 con 0x7fcfb80cea60 2024-04-14T08:42:42.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.918+0000 7fcfbebc8640 1 -- 172.21.15.172:0/854013271 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcf9c06c8f0 msgr2=0x7fcf9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:42.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.918+0000 7fcfbebc8640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcf9c06c8f0 0x7fcf9c06edb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcfac036000 tx=0x7fcfac03b040 comp rx=0 tx=0).stop 2024-04-14T08:42:42.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.918+0000 7fcfbebc8640 1 -- 172.21.15.172:0/854013271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb80cea60 msgr2=0x7fcfb80cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:42.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.918+0000 7fcfbebc8640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb80cea60 0x7fcfb80cba20 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400c430 tx=0x7fcfa400c8f0 comp rx=0 tx=0).stop 2024-04-14T08:42:42.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.919+0000 7fcfbebc8640 1 -- 172.21.15.172:0/854013271 shutdown_connections 2024-04-14T08:42:42.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.919+0000 7fcfbebc8640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcf9c06c8f0 0x7fcf9c06edb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:42.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.919+0000 7fcfbebc8640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb815e590 0x7fcfb8157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:42.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.919+0000 7fcfbebc8640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfb80cea60 0x7fcfb80cba20 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:42.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.919+0000 7fcfbebc8640 1 --2- 172.21.15.172:0/854013271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80b65a0 0x7fcfb80cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:42.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.919+0000 7fcfbebc8640 1 -- 172.21.15.172:0/854013271 >> 172.21.15.172:0/854013271 conn(0x7fcfb80d0e90 msgr2=0x7fcfb80d3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:42.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.919+0000 7fcfbebc8640 1 -- 172.21.15.172:0/854013271 shutdown_connections 2024-04-14T08:42:42.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:42.919+0000 7fcfbebc8640 1 -- 172.21.15.172:0/854013271 wait complete. 2024-04-14T08:42:42.932 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-14T08:42:42.932 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:42:43.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.039+0000 7f01dc75e640 1 Processor -- start 2024-04-14T08:42:43.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.039+0000 7f01dc75e640 1 -- start start 2024-04-14T08:42:43.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01dc75e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f01d40b4040 0x7f01d40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01dc75e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d415c770 con 0x7f01d40b1df0 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01dc75e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d40b3700 con 0x7f01d40b21d0 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01dc75e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f01d40b39e0 con 0x7f01d40b4040 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01da4fa640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f01d40b1df0 0x7f01d40b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34180/0 (socket says 172.21.15.172:34180) 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01da4fa640 1 -- 172.21.15.172:0/692282576 learned_addr learned my addr 172.21.15.172:0/692282576 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01dacfb640 1 --2- 172.21.15.172:0/692282576 >> v2:172.21.15.177:3300/0 conn(0x7f01d40b4040 0x7f01d40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01d94f8640 1 -- 172.21.15.172:0/692282576 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3091929315 0 0) 0x7f01d415c770 con 0x7f01d40b1df0 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01d94f8640 1 -- 172.21.15.172:0/692282576 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01c0003520 con 0x7f01d40b1df0 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01d94f8640 1 -- 172.21.15.172:0/692282576 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2712275205 0 0) 0x7f01d40b3700 con 0x7f01d40b21d0 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01d94f8640 1 -- 172.21.15.172:0/692282576 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01d415c770 con 0x7f01d40b21d0 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01dacfb640 1 -- 172.21.15.172:0/692282576 >> v1:172.21.15.177:6789/0 conn(0x7f01d40b21d0 legacy=0x7f01d415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01dacfb640 1 -- 172.21.15.172:0/692282576 >> v1:172.21.15.172:6789/0 conn(0x7f01d40b1df0 legacy=0x7f01d40b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.040+0000 7f01dacfb640 1 -- 172.21.15.172:0/692282576 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d40b3cc0 con 0x7f01d40b4040 2024-04-14T08:42:43.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.041+0000 7f01dacfb640 1 --2- 172.21.15.172:0/692282576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01d40b4040 0x7f01d40b4460 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009d90 tx=0x7f01cc02bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=832294b39a192871 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:43.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.041+0000 7f01d94f8640 1 -- 172.21.15.172:0/692282576 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc039070 con 0x7f01d40b4040 2024-04-14T08:42:43.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.041+0000 7f01d94f8640 1 -- 172.21.15.172:0/692282576 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc02cd00 con 0x7f01d40b4040 2024-04-14T08:42:43.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.041+0000 7f01d94f8640 1 -- 172.21.15.172:0/692282576 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc034660 con 0x7f01d40b4040 2024-04-14T08:42:43.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.042+0000 7f01dc75e640 1 -- 172.21.15.172:0/692282576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01d40b4040 msgr2=0x7f01d40b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.042+0000 7f01dc75e640 1 --2- 172.21.15.172:0/692282576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01d40b4040 0x7f01d40b4460 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009d90 tx=0x7f01cc02bf50 comp rx=0 tx=0).stop 2024-04-14T08:42:43.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.042+0000 7f01dc75e640 1 -- 172.21.15.172:0/692282576 shutdown_connections 2024-04-14T08:42:43.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.042+0000 7f01dc75e640 1 --2- 172.21.15.172:0/692282576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01d40b4040 0x7f01d40b4460 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.042+0000 7f01dc75e640 1 -- 172.21.15.172:0/692282576 >> 172.21.15.172:0/692282576 conn(0x7f01d406ec30 msgr2=0x7f01d406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:43.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.042+0000 7f01dc75e640 1 -- 172.21.15.172:0/692282576 shutdown_connections 2024-04-14T08:42:43.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.042+0000 7f01dc75e640 1 -- 172.21.15.172:0/692282576 wait complete. 2024-04-14T08:42:43.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.043+0000 7f01dc75e640 1 Processor -- start 2024-04-14T08:42:43.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.043+0000 7f01dc75e640 1 -- start start 2024-04-14T08:42:43.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01dc75e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d40b1df0 0x7f01d40d0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01dc75e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01d40b21d0 0x7f01d41f1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01dc75e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01d40b4040 0x7f01d413b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01dc75e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01d415b7f0 con 0x7f01d40b1df0 2024-04-14T08:42:43.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01dc75e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f01d40b49a0 con 0x7f01d40b21d0 2024-04-14T08:42:43.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01dc75e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f01d41552d0 con 0x7f01d40b4040 2024-04-14T08:42:43.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01da4fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d40b1df0 0x7f01d40d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01da4fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d40b1df0 0x7f01d40d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39792/0 (socket says 172.21.15.172:39792) 2024-04-14T08:42:43.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01da4fa640 1 -- 172.21.15.172:0/2569308952 learned_addr learned my addr 172.21.15.172:0/2569308952 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:43.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01dacfb640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01d40b4040 0x7f01d413b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01d9cf9640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01d40b21d0 0x7f01d41f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01da4fa640 1 -- 172.21.15.172:0/2569308952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01d40b4040 msgr2=0x7f01d413b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01da4fa640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01d40b4040 0x7f01d413b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01da4fa640 1 -- 172.21.15.172:0/2569308952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01d40b21d0 msgr2=0x7f01d41f1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01da4fa640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01d40b21d0 0x7f01d41f1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01da4fa640 1 -- 172.21.15.172:0/2569308952 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d4139c00 con 0x7f01d40b1df0 2024-04-14T08:42:43.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01dacfb640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01d40b4040 0x7f01d413b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:43.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01d9cf9640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01d40b21d0 0x7f01d41f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:43.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01da4fa640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d40b1df0 0x7f01d40d0890 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c500 tx=0x7f01d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:43.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01caffd640 1 -- 172.21.15.172:0/2569308952 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0017070 con 0x7f01d40b1df0 2024-04-14T08:42:43.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01caffd640 1 -- 172.21.15.172:0/2569308952 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d0005ce0 con 0x7f01d40b1df0 2024-04-14T08:42:43.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01dc75e640 1 -- 172.21.15.172:0/2569308952 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d4139ee0 con 0x7f01d40b1df0 2024-04-14T08:42:43.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.044+0000 7f01caffd640 1 -- 172.21.15.172:0/2569308952 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d00072f0 con 0x7f01d40b1df0 2024-04-14T08:42:43.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.045+0000 7f01dc75e640 1 -- 172.21.15.172:0/2569308952 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d4155ae0 con 0x7f01d40b1df0 2024-04-14T08:42:43.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.046+0000 7f01dc75e640 1 -- 172.21.15.172:0/2569308952 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d4068520 con 0x7f01d40b1df0 2024-04-14T08:42:43.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.046+0000 7f01caffd640 1 -- 172.21.15.172:0/2569308952 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01d001a070 con 0x7f01d40b1df0 2024-04-14T08:42:43.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.046+0000 7f01caffd640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01b806c8f0 0x7f01b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.047+0000 7f01caffd640 1 -- 172.21.15.172:0/2569308952 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f01d008af90 con 0x7f01d40b1df0 2024-04-14T08:42:43.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.047+0000 7f01d9cf9640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01b806c8f0 0x7f01b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.047+0000 7f01d9cf9640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01b806c8f0 0x7f01b806edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f01c4007fd0 tx=0x7f01c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:43.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.051+0000 7f01caffd640 1 -- 172.21.15.172:0/2569308952 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f01d0058d30 con 0x7f01d40b1df0 2024-04-14T08:42:43.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.263+0000 7f01dc75e640 1 -- 172.21.15.172:0/2569308952 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01d4072b10 con 0x7f01d40b1df0 2024-04-14T08:42:43.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.263+0000 7f01caffd640 1 -- 172.21.15.172:0/2569308952 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5192 (secure 0 0 0) 0x7f01d005d030 con 0x7f01d40b1df0 2024-04-14T08:42:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:43.264 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":88,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9048,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/744853572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":744853572},{"type":"v1","addr":"172.21.15.172:6837","nonce":744853572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:42:42.249444+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9060},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9060":{"gid":9060,"name":"d","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/3201353747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3201353747},{"type":"v1","addr":"172.21.15.177:6835","nonce":3201353747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9060,"qdb_cluster":[9060]},"id":8}]} 2024-04-14T08:42:43.264 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 88 2024-04-14T08:42:43.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.265+0000 7f01dc75e640 1 -- 172.21.15.172:0/2569308952 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01b806c8f0 msgr2=0x7f01b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.265+0000 7f01dc75e640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01b806c8f0 0x7f01b806edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f01c4007fd0 tx=0x7f01c4009040 comp rx=0 tx=0).stop 2024-04-14T08:42:43.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.265+0000 7f01dc75e640 1 -- 172.21.15.172:0/2569308952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d40b1df0 msgr2=0x7f01d40d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.265+0000 7f01dc75e640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d40b1df0 0x7f01d40d0890 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c500 tx=0x7f01d000c9c0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.265+0000 7f01dc75e640 1 -- 172.21.15.172:0/2569308952 shutdown_connections 2024-04-14T08:42:43.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.265+0000 7f01dc75e640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f01b806c8f0 0x7f01b806edb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.265+0000 7f01dc75e640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01d40b4040 0x7f01d413b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.265+0000 7f01dc75e640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01d40b21d0 0x7f01d41f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.265+0000 7f01dc75e640 1 --2- 172.21.15.172:0/2569308952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01d40b1df0 0x7f01d40d0890 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.265+0000 7f01dc75e640 1 -- 172.21.15.172:0/2569308952 >> 172.21.15.172:0/2569308952 conn(0x7f01d406ec30 msgr2=0x7f01d406b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:43.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.266+0000 7f01dc75e640 1 -- 172.21.15.172:0/2569308952 shutdown_connections 2024-04-14T08:42:43.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.266+0000 7f01dc75e640 1 -- 172.21.15.172:0/2569308952 wait complete. 2024-04-14T08:42:43.277 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:42:43.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.378+0000 7f51693aa640 1 Processor -- start 2024-04-14T08:42:43.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.378+0000 7f51693aa640 1 -- start start 2024-04-14T08:42:43.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.379+0000 7f51693aa640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f516415c150 0x7f51640d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.379+0000 7f51693aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51640d1840 0x7f5164161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.379+0000 7f51693aa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5164162530 0x7f516415ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.379+0000 7f51693aa640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f51640cef10 con 0x7f5164162530 2024-04-14T08:42:43.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.379+0000 7f51693aa640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f51640d1c80 con 0x7f516415c150 2024-04-14T08:42:43.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.379+0000 7f51693aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5164162940 con 0x7f51640d1840 2024-04-14T08:42:43.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.379+0000 7f5162ffd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f516415c150 0x7f51640d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.379+0000 7f51637fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5164162530 0x7f516415ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.380+0000 7f5162ffd640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f516415c150 0x7f51640d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33194/0 (socket says 172.21.15.172:33194) 2024-04-14T08:42:43.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.380+0000 7f51627fc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51640d1840 0x7f5164161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.380+0000 7f5162ffd640 1 -- 172.21.15.172:0/3635527103 learned_addr learned my addr 172.21.15.172:0/3635527103 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:43.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.380+0000 7f5162ffd640 1 -- 172.21.15.172:0/3635527103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51640d1840 msgr2=0x7f5164161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:43.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.380+0000 7f5162ffd640 1 --2- 172.21.15.172:0/3635527103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51640d1840 0x7f5164161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.380+0000 7f5162ffd640 1 -- 172.21.15.172:0/3635527103 >> v2:172.21.15.172:3300/0 conn(0x7f5164162530 msgr2=0x7f516415ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:43.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.380+0000 7f5162ffd640 1 --2- 172.21.15.172:0/3635527103 >> v2:172.21.15.172:3300/0 conn(0x7f5164162530 0x7f516415ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.380+0000 7f5162ffd640 1 -- 172.21.15.172:0/3635527103 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51640d0200 con 0x7f516415c150 2024-04-14T08:42:43.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.380+0000 7f51627fc640 1 --2- 172.21.15.172:0/3635527103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51640d1840 0x7f5164161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:43.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f5162ffd640 1 --2- 172.21.15.172:0/3635527103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f516415c150 0x7f51640d1300 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f51640d03d0 tx=0x7f515402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca923eb9648f4ee server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:43.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f5161ffb640 1 -- 172.21.15.172:0/3635527103 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515402ca70 con 0x7f516415c150 2024-04-14T08:42:43.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f5161ffb640 1 -- 172.21.15.172:0/3635527103 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f515402cc10 con 0x7f516415c150 2024-04-14T08:42:43.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f5161ffb640 1 -- 172.21.15.172:0/3635527103 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540343f0 con 0x7f516415c150 2024-04-14T08:42:43.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f51693aa640 1 -- 172.21.15.172:0/3635527103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f516415c150 msgr2=0x7f51640d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f51693aa640 1 --2- 172.21.15.172:0/3635527103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f516415c150 0x7f51640d1300 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f51640d03d0 tx=0x7f515402bed0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f51693aa640 1 -- 172.21.15.172:0/3635527103 shutdown_connections 2024-04-14T08:42:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f51693aa640 1 --2- 172.21.15.172:0/3635527103 >> v2:172.21.15.172:3300/0 conn(0x7f5164162530 0x7f516415ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f51693aa640 1 --2- 172.21.15.172:0/3635527103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51640d1840 0x7f5164161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f51693aa640 1 --2- 172.21.15.172:0/3635527103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f516415c150 0x7f51640d1300 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f51693aa640 1 -- 172.21.15.172:0/3635527103 >> 172.21.15.172:0/3635527103 conn(0x7f516409d580 msgr2=0x7f51640d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f51693aa640 1 -- 172.21.15.172:0/3635527103 shutdown_connections 2024-04-14T08:42:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.381+0000 7f51693aa640 1 -- 172.21.15.172:0/3635527103 wait complete. 2024-04-14T08:42:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.383+0000 7f51693aa640 1 Processor -- start 2024-04-14T08:42:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.383+0000 7f51693aa640 1 -- start start 2024-04-14T08:42:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.383+0000 7f51693aa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51640d1840 0x7f51641550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.383+0000 7f51693aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516415c150 0x7f51641f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.383+0000 7f51693aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5164162530 0x7f51641f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.383+0000 7f51693aa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f51640cf720 con 0x7f516415c150 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.383+0000 7f51693aa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f51640cf440 con 0x7f51640d1840 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.383+0000 7f51693aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f51640cfa00 con 0x7f5164162530 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.383+0000 7f51627fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516415c150 0x7f51641f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.383+0000 7f51627fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516415c150 0x7f51641f82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39808/0 (socket says 172.21.15.172:39808) 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.383+0000 7f51627fc640 1 -- 172.21.15.172:0/3649502885 learned_addr learned my addr 172.21.15.172:0/3649502885 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.383+0000 7f5162ffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51640d1840 0x7f51641550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.384+0000 7f51637fe640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5164162530 0x7f51641f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.384+0000 7f51627fc640 1 -- 172.21.15.172:0/3649502885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5164162530 msgr2=0x7f51641f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.384+0000 7f51627fc640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5164162530 0x7f51641f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.384+0000 7f51627fc640 1 -- 172.21.15.172:0/3649502885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51640d1840 msgr2=0x7f51641550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.384+0000 7f51627fc640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51640d1840 0x7f51641550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.384+0000 7f51627fc640 1 -- 172.21.15.172:0/3649502885 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51641f5380 con 0x7f516415c150 2024-04-14T08:42:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.384+0000 7f51627fc640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516415c150 0x7f51641f82c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f515800e980 tx=0x7f515800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.384+0000 7f514b7fe640 1 -- 172.21.15.172:0/3649502885 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158017070 con 0x7f516415c150 2024-04-14T08:42:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.384+0000 7f514b7fe640 1 -- 172.21.15.172:0/3649502885 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f515800ca00 con 0x7f516415c150 2024-04-14T08:42:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.384+0000 7f514b7fe640 1 -- 172.21.15.172:0/3649502885 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515800cdd0 con 0x7f516415c150 2024-04-14T08:42:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.384+0000 7f51693aa640 1 -- 172.21.15.172:0/3649502885 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51641f4dc0 con 0x7f516415c150 2024-04-14T08:42:43.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.384+0000 7f51693aa640 1 -- 172.21.15.172:0/3649502885 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51641f7ac0 con 0x7f516415c150 2024-04-14T08:42:43.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.385+0000 7f51693aa640 1 -- 172.21.15.172:0/3649502885 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5164075490 con 0x7f516415c150 2024-04-14T08:42:43.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.386+0000 7f514b7fe640 1 -- 172.21.15.172:0/3649502885 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f515801a070 con 0x7f516415c150 2024-04-14T08:42:43.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.386+0000 7f514b7fe640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f515006c8f0 0x7f515006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.386+0000 7f514b7fe640 1 -- 172.21.15.172:0/3649502885 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f515808bdc0 con 0x7f516415c150 2024-04-14T08:42:43.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.386+0000 7f5162ffd640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f515006c8f0 0x7f515006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.387+0000 7f5162ffd640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f515006c8f0 0x7f515006edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5154035000 tx=0x7f5154030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:43.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.391+0000 7f514b7fe640 1 -- 172.21.15.172:0/3649502885 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5158059aa0 con 0x7f516415c150 2024-04-14T08:42:43.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.566+0000 7f51693aa640 1 -- 172.21.15.172:0/3649502885 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51640d49c0 con 0x7f516415c150 2024-04-14T08:42:43.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.568+0000 7f514b7fe640 1 -- 172.21.15.172:0/3649502885 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+17012 (secure 0 0 0) 0x7f515805dda0 con 0x7f516415c150 2024-04-14T08:42:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:43.568 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":116,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:42:41.900871+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:42:38.455900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-14T08:42:39.198157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:42:43.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.569+0000 7f51693aa640 1 -- 172.21.15.172:0/3649502885 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f515006c8f0 msgr2=0x7f515006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.569+0000 7f51693aa640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f515006c8f0 0x7f515006edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5154035000 tx=0x7f5154030000 comp rx=0 tx=0).stop 2024-04-14T08:42:43.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.570+0000 7f51693aa640 1 -- 172.21.15.172:0/3649502885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516415c150 msgr2=0x7f51641f82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.570+0000 7f51693aa640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516415c150 0x7f51641f82c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f515800e980 tx=0x7f515800ee40 comp rx=0 tx=0).stop 2024-04-14T08:42:43.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.570+0000 7f51693aa640 1 -- 172.21.15.172:0/3649502885 shutdown_connections 2024-04-14T08:42:43.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.570+0000 7f51693aa640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f515006c8f0 0x7f515006edb0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.570+0000 7f51693aa640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5164162530 0x7f51641f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.570+0000 7f51693aa640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516415c150 0x7f51641f82c0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.570+0000 7f51693aa640 1 --2- 172.21.15.172:0/3649502885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51640d1840 0x7f51641550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.570+0000 7f51693aa640 1 -- 172.21.15.172:0/3649502885 >> 172.21.15.172:0/3649502885 conn(0x7f516409d580 msgr2=0x7f516406c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:43.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.570+0000 7f51693aa640 1 -- 172.21.15.172:0/3649502885 shutdown_connections 2024-04-14T08:42:43.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.570+0000 7f51693aa640 1 -- 172.21.15.172:0/3649502885 wait complete. 2024-04-14T08:42:43.581 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-14T08:42:43.581 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:42:43.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.683+0000 7fa539836640 1 Processor -- start 2024-04-14T08:42:43.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.684+0000 7fa539836640 1 -- start start 2024-04-14T08:42:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.684+0000 7fa539836640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5340d2830 0x7fa5340d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.684+0000 7fa539836640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa53415e890 0x7fa5340d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.684+0000 7fa539836640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5340d3150 con 0x7fa5340cedd0 2024-04-14T08:42:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.684+0000 7fa539836640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa5340cff30 con 0x7fa53415e890 2024-04-14T08:42:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.684+0000 7fa539836640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa5340d2350 con 0x7fa5340d2830 2024-04-14T08:42:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.684+0000 7fa52bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa53415e890 0x7fa5340d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.685+0000 7fa52bfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa53415e890 0x7fa5340d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39814/0 (socket says 172.21.15.172:39814) 2024-04-14T08:42:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.685+0000 7fa5337fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa5340cedd0 0x7fa5341621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34186/0 (socket says 172.21.15.172:34186) 2024-04-14T08:42:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.685+0000 7fa52bfff640 1 -- 172.21.15.172:0/1461850767 learned_addr learned my addr 172.21.15.172:0/1461850767 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.685+0000 7fa532ffd640 1 --2- 172.21.15.172:0/1461850767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5340d2830 0x7fa5340d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.685+0000 7fa52bfff640 1 -- 172.21.15.172:0/1461850767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5340d2830 msgr2=0x7fa5340d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.685+0000 7fa52bfff640 1 --2- 172.21.15.172:0/1461850767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5340d2830 0x7fa5340d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.685+0000 7fa52bfff640 1 -- 172.21.15.172:0/1461850767 >> v1:172.21.15.172:6789/0 conn(0x7fa5340cedd0 legacy=0x7fa5341621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.685+0000 7fa52bfff640 1 -- 172.21.15.172:0/1461850767 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5340cf320 con 0x7fa53415e890 2024-04-14T08:42:43.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.685+0000 7fa532ffd640 1 --2- 172.21.15.172:0/1461850767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5340d2830 0x7fa5340d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:43.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.686+0000 7fa52bfff640 1 --2- 172.21.15.172:0/1461850767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa53415e890 0x7fa5340d1300 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009870 tx=0x7fa51c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d90900805885c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:43.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.686+0000 7fa5327fc640 1 -- 172.21.15.172:0/1461850767 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c02f840 con 0x7fa53415e890 2024-04-14T08:42:43.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.686+0000 7fa5327fc640 1 -- 172.21.15.172:0/1461850767 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51c02f9e0 con 0x7fa53415e890 2024-04-14T08:42:43.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.686+0000 7fa5327fc640 1 -- 172.21.15.172:0/1461850767 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c02fdb0 con 0x7fa53415e890 2024-04-14T08:42:43.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.686+0000 7fa539836640 1 -- 172.21.15.172:0/1461850767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa53415e890 msgr2=0x7fa5340d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.686+0000 7fa539836640 1 --2- 172.21.15.172:0/1461850767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa53415e890 0x7fa5340d1300 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009870 tx=0x7fa51c02ba20 comp rx=0 tx=0).stop 2024-04-14T08:42:43.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.686+0000 7fa539836640 1 -- 172.21.15.172:0/1461850767 shutdown_connections 2024-04-14T08:42:43.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.686+0000 7fa539836640 1 --2- 172.21.15.172:0/1461850767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa53415e890 0x7fa5340d1300 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.686+0000 7fa539836640 1 --2- 172.21.15.172:0/1461850767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5340d2830 0x7fa5340d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.686+0000 7fa539836640 1 -- 172.21.15.172:0/1461850767 >> 172.21.15.172:0/1461850767 conn(0x7fa53409d590 msgr2=0x7fa5340d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:43.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.686+0000 7fa539836640 1 -- 172.21.15.172:0/1461850767 shutdown_connections 2024-04-14T08:42:43.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.686+0000 7fa539836640 1 -- 172.21.15.172:0/1461850767 wait complete. 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.687+0000 7fa539836640 1 Processor -- start 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.688+0000 7fa539836640 1 -- start start 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.688+0000 7fa539836640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5340cedd0 0x7fa534155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.688+0000 7fa539836640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5340d2830 0x7fa5340b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.688+0000 7fa539836640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa53415e890 0x7fa5340b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.688+0000 7fa539836640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa5340cfbc0 con 0x7fa53415e890 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.688+0000 7fa539836640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa5340cf8e0 con 0x7fa5340d2830 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.688+0000 7fa539836640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa5340d1840 con 0x7fa5340cedd0 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.688+0000 7fa5337fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa53415e890 0x7fa5340b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.688+0000 7fa5337fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa53415e890 0x7fa5340b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39822/0 (socket says 172.21.15.172:39822) 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.688+0000 7fa5337fe640 1 -- 172.21.15.172:0/3244534675 learned_addr learned my addr 172.21.15.172:0/3244534675 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.688+0000 7fa52bfff640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5340d2830 0x7fa5340b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.688+0000 7fa532ffd640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5340cedd0 0x7fa534155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.689+0000 7fa5337fe640 1 -- 172.21.15.172:0/3244534675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5340cedd0 msgr2=0x7fa534155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.689+0000 7fa5337fe640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5340cedd0 0x7fa534155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.689+0000 7fa5337fe640 1 -- 172.21.15.172:0/3244534675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5340d2830 msgr2=0x7fa5340b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.689+0000 7fa5337fe640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5340d2830 0x7fa5340b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.689+0000 7fa5337fe640 1 -- 172.21.15.172:0/3244534675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5340b7010 con 0x7fa53415e890 2024-04-14T08:42:43.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.689+0000 7fa5337fe640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa53415e890 0x7fa5340b24d0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00c760 tx=0x7fa52c00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:43.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.689+0000 7fa530ff9640 1 -- 172.21.15.172:0/3244534675 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c018070 con 0x7fa53415e890 2024-04-14T08:42:43.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.689+0000 7fa530ff9640 1 -- 172.21.15.172:0/3244534675 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52c004d60 con 0x7fa53415e890 2024-04-14T08:42:43.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.689+0000 7fa539836640 1 -- 172.21.15.172:0/3244534675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5340b72f0 con 0x7fa53415e890 2024-04-14T08:42:43.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.689+0000 7fa530ff9640 1 -- 172.21.15.172:0/3244534675 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00a880 con 0x7fa53415e890 2024-04-14T08:42:43.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.689+0000 7fa539836640 1 -- 172.21.15.172:0/3244534675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5340b78b0 con 0x7fa53415e890 2024-04-14T08:42:43.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.690+0000 7fa539836640 1 -- 172.21.15.172:0/3244534675 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa534068520 con 0x7fa53415e890 2024-04-14T08:42:43.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.690+0000 7fa530ff9640 1 -- 172.21.15.172:0/3244534675 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa52c01b070 con 0x7fa53415e890 2024-04-14T08:42:43.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.691+0000 7fa530ff9640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa51406c8f0 0x7fa51406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:43.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.691+0000 7fa530ff9640 1 -- 172.21.15.172:0/3244534675 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7fa52c08b0c0 con 0x7fa53415e890 2024-04-14T08:42:43.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.691+0000 7fa532ffd640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa51406c8f0 0x7fa51406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:43.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.691+0000 7fa532ffd640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa51406c8f0 0x7fa51406edb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa524009560 tx=0x7fa524009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:43.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.695+0000 7fa530ff9640 1 -- 172.21.15.172:0/3244534675 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa52c058d10 con 0x7fa53415e890 2024-04-14T08:42:43.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.870+0000 7fa539836640 1 -- 172.21.15.172:0/3244534675 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa5340b7af0 con 0x7fa51406c8f0 2024-04-14T08:42:43.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.876+0000 7fa530ff9640 1 -- 172.21.15.172:0/3244534675 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247762 (secure 0 0 0) 0x7fa524011a00 con 0x7fa51406c8f0 2024-04-14T08:42:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:42:43.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.879+0000 7fa539836640 1 -- 172.21.15.172:0/3244534675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa51406c8f0 msgr2=0x7fa51406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.879+0000 7fa539836640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa51406c8f0 0x7fa51406edb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa524009560 tx=0x7fa524009190 comp rx=0 tx=0).stop 2024-04-14T08:42:43.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.879+0000 7fa539836640 1 -- 172.21.15.172:0/3244534675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa53415e890 msgr2=0x7fa5340b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:43.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.879+0000 7fa539836640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa53415e890 0x7fa5340b24d0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00c760 tx=0x7fa52c00cb60 comp rx=0 tx=0).stop 2024-04-14T08:42:43.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.879+0000 7fa539836640 1 -- 172.21.15.172:0/3244534675 shutdown_connections 2024-04-14T08:42:43.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.879+0000 7fa539836640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa51406c8f0 0x7fa51406edb0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.879+0000 7fa539836640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa53415e890 0x7fa5340b24d0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.879+0000 7fa539836640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa5340d2830 0x7fa5340b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.879+0000 7fa539836640 1 --2- 172.21.15.172:0/3244534675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa5340cedd0 0x7fa534155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:43.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.879+0000 7fa539836640 1 -- 172.21.15.172:0/3244534675 >> 172.21.15.172:0/3244534675 conn(0x7fa53409d590 msgr2=0x7fa5340d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:43.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.879+0000 7fa539836640 1 -- 172.21.15.172:0/3244534675 shutdown_connections 2024-04-14T08:42:43.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:43.879+0000 7fa539836640 1 -- 172.21.15.172:0/3244534675 wait complete. 2024-04-14T08:42:43.890 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-04-14T08:42:43.237918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261656,"kb_used_data":12204,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713064,"statfs":{"total":773094113280,"available":772826177536,"internally_reserved":0,"allocated":12496896,"data_stored":6427089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.355032"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.891452+0000","last_change":"2024-04-14T08:42:39.891452+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.891452+0000","last_undegraded":"2024-04-14T08:42:39.891452+0000","last_fullsized":"2024-04-14T08:42:39.891452+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.906122+0000","last_change":"2024-04-14T08:42:39.906122+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.906122+0000","last_undegraded":"2024-04-14T08:42:39.906122+0000","last_fullsized":"2024-04-14T08:42:39.906122+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904781+0000","last_change":"2024-04-14T08:42:39.904781+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904781+0000","last_undegraded":"2024-04-14T08:42:39.904781+0000","last_fullsized":"2024-04-14T08:42:39.904781+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.900277+0000","last_change":"2024-04-14T08:42:40.899261+0000","last_active":"2024-04-14T08:42:40.900277+0000","last_peered":"2024-04-14T08:42:40.900277+0000","last_clean":"2024-04-14T08:42:40.900277+0000","last_became_active":"2024-04-14T08:42:40.898970+0000","last_became_peered":"2024-04-14T08:42:40.898970+0000","last_unstale":"2024-04-14T08:42:40.900277+0000","last_undegraded":"2024-04-14T08:42:40.900277+0000","last_fullsized":"2024-04-14T08:42:40.900277+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:57.005587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.891722+0000","last_change":"2024-04-14T08:42:39.891722+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.891722+0000","last_undegraded":"2024-04-14T08:42:39.891722+0000","last_fullsized":"2024-04-14T08:42:39.891722+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904215+0000","last_change":"2024-04-14T08:42:39.904215+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904215+0000","last_undegraded":"2024-04-14T08:42:39.904215+0000","last_fullsized":"2024-04-14T08:42:39.904215+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.910912+0000","last_change":"2024-04-14T08:42:39.910912+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.910912+0000","last_undegraded":"2024-04-14T08:42:39.910912+0000","last_fullsized":"2024-04-14T08:42:39.910912+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.907664+0000","last_change":"2024-04-14T08:42:39.907664+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.907664+0000","last_undegraded":"2024-04-14T08:42:39.907664+0000","last_fullsized":"2024-04-14T08:42:39.907664+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.914262+0000","last_change":"2024-04-14T08:42:40.909455+0000","last_active":"2024-04-14T08:42:40.914262+0000","last_peered":"2024-04-14T08:42:40.914262+0000","last_clean":"2024-04-14T08:42:40.914262+0000","last_became_active":"2024-04-14T08:42:40.909156+0000","last_became_peered":"2024-04-14T08:42:40.909156+0000","last_unstale":"2024-04-14T08:42:40.914262+0000","last_undegraded":"2024-04-14T08:42:40.914262+0000","last_fullsized":"2024-04-14T08:42:40.914262+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:50:25.331351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902706+0000","last_change":"2024-04-14T08:42:39.902706+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902706+0000","last_undegraded":"2024-04-14T08:42:39.902706+0000","last_fullsized":"2024-04-14T08:42:39.902706+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.892580+0000","last_change":"2024-04-14T08:42:39.892580+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.892580+0000","last_undegraded":"2024-04-14T08:42:39.892580+0000","last_fullsized":"2024-04-14T08:42:39.892580+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904944+0000","last_change":"2024-04-14T08:42:39.904944+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904944+0000","last_undegraded":"2024-04-14T08:42:39.904944+0000","last_fullsized":"2024-04-14T08:42:39.904944+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903252+0000","last_change":"2024-04-14T08:42:39.903252+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903252+0000","last_undegraded":"2024-04-14T08:42:39.903252+0000","last_fullsized":"2024-04-14T08:42:39.903252+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.909607+0000","last_change":"2024-04-14T08:42:39.909607+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.909607+0000","last_undegraded":"2024-04-14T08:42:39.909607+0000","last_fullsized":"2024-04-14T08:42:39.909607+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.893081+0000","last_change":"2024-04-14T08:42:39.893081+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.893081+0000","last_undegraded":"2024-04-14T08:42:39.893081+0000","last_fullsized":"2024-04-14T08:42:39.893081+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903738+0000","last_change":"2024-04-14T08:42:39.903738+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903738+0000","last_undegraded":"2024-04-14T08:42:39.903738+0000","last_fullsized":"2024-04-14T08:42:39.903738+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903068+0000","last_change":"2024-04-14T08:42:39.903068+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903068+0000","last_undegraded":"2024-04-14T08:42:39.903068+0000","last_fullsized":"2024-04-14T08:42:39.903068+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.905703+0000","last_change":"2024-04-14T08:42:39.905703+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.905703+0000","last_undegraded":"2024-04-14T08:42:39.905703+0000","last_fullsized":"2024-04-14T08:42:39.905703+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.896903+0000","last_change":"2024-04-14T08:42:39.896903+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.896903+0000","last_undegraded":"2024-04-14T08:42:39.896903+0000","last_fullsized":"2024-04-14T08:42:39.896903+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.908810+0000","last_change":"2024-04-14T08:42:39.908810+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.908810+0000","last_undegraded":"2024-04-14T08:42:39.908810+0000","last_fullsized":"2024-04-14T08:42:39.908810+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.913413+0000","last_change":"2024-04-14T08:42:40.906675+0000","last_active":"2024-04-14T08:42:40.913413+0000","last_peered":"2024-04-14T08:42:40.913413+0000","last_clean":"2024-04-14T08:42:40.913413+0000","last_became_active":"2024-04-14T08:42:40.904423+0000","last_became_peered":"2024-04-14T08:42:40.904423+0000","last_unstale":"2024-04-14T08:42:40.913413+0000","last_undegraded":"2024-04-14T08:42:40.913413+0000","last_fullsized":"2024-04-14T08:42:40.913413+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:42:53.101473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.892991+0000","last_change":"2024-04-14T08:42:39.892991+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.892991+0000","last_undegraded":"2024-04-14T08:42:39.892991+0000","last_fullsized":"2024-04-14T08:42:39.892991+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.921770+0000","last_change":"2024-04-14T08:42:40.919066+0000","last_active":"2024-04-14T08:42:40.921770+0000","last_peered":"2024-04-14T08:42:40.921770+0000","last_clean":"2024-04-14T08:42:40.921770+0000","last_became_active":"2024-04-14T08:42:40.918630+0000","last_became_peered":"2024-04-14T08:42:40.918630+0000","last_unstale":"2024-04-14T08:42:40.921770+0000","last_undegraded":"2024-04-14T08:42:40.921770+0000","last_fullsized":"2024-04-14T08:42:40.921770+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:53:03.886099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.899098+0000","last_change":"2024-04-14T08:42:39.899098+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.899098+0000","last_undegraded":"2024-04-14T08:42:39.899098+0000","last_fullsized":"2024-04-14T08:42:39.899098+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900930+0000","last_change":"2024-04-14T08:42:39.900930+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900930+0000","last_undegraded":"2024-04-14T08:42:39.900930+0000","last_fullsized":"2024-04-14T08:42:39.900930+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.906240+0000","last_change":"2024-04-14T08:42:39.906240+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.906240+0000","last_undegraded":"2024-04-14T08:42:39.906240+0000","last_fullsized":"2024-04-14T08:42:39.906240+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.896498+0000","last_change":"2024-04-14T08:42:39.896498+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.896498+0000","last_undegraded":"2024-04-14T08:42:39.896498+0000","last_fullsized":"2024-04-14T08:42:39.896498+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904518+0000","last_change":"2024-04-14T08:42:39.904518+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904518+0000","last_undegraded":"2024-04-14T08:42:39.904518+0000","last_fullsized":"2024-04-14T08:42:39.904518+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.905764+0000","last_change":"2024-04-14T08:42:39.905764+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.905764+0000","last_undegraded":"2024-04-14T08:42:39.905764+0000","last_fullsized":"2024-04-14T08:42:39.905764+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.909220+0000","last_change":"2024-04-14T08:42:39.909220+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.909220+0000","last_undegraded":"2024-04-14T08:42:39.909220+0000","last_fullsized":"2024-04-14T08:42:39.909220+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.899058+0000","last_change":"2024-04-14T08:42:39.899058+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.899058+0000","last_undegraded":"2024-04-14T08:42:39.899058+0000","last_fullsized":"2024-04-14T08:42:39.899058+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.898579+0000","last_change":"2024-04-14T08:42:39.898579+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.898579+0000","last_undegraded":"2024-04-14T08:42:39.898579+0000","last_fullsized":"2024-04-14T08:42:39.898579+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902637+0000","last_change":"2024-04-14T08:42:39.902637+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902637+0000","last_undegraded":"2024-04-14T08:42:39.902637+0000","last_fullsized":"2024-04-14T08:42:39.902637+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.906183+0000","last_change":"2024-04-14T08:42:39.902925+0000","last_active":"2024-04-14T08:42:39.906183+0000","last_peered":"2024-04-14T08:42:39.906183+0000","last_clean":"2024-04-14T08:42:39.906183+0000","last_became_active":"2024-04-14T08:42:39.902599+0000","last_became_peered":"2024-04-14T08:42:39.902599+0000","last_unstale":"2024-04-14T08:42:39.906183+0000","last_undegraded":"2024-04-14T08:42:39.906183+0000","last_fullsized":"2024-04-14T08:42:39.906183+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:59.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903562+0000","last_change":"2024-04-14T08:42:39.903562+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903562+0000","last_undegraded":"2024-04-14T08:42:39.903562+0000","last_fullsized":"2024-04-14T08:42:39.903562+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.904995+0000","last_change":"2024-04-14T08:42:39.898146+0000","last_active":"2024-04-14T08:42:39.904995+0000","last_peered":"2024-04-14T08:42:39.904995+0000","last_clean":"2024-04-14T08:42:39.904995+0000","last_became_active":"2024-04-14T08:42:39.897857+0000","last_became_peered":"2024-04-14T08:42:39.897857+0000","last_unstale":"2024-04-14T08:42:39.904995+0000","last_undegraded":"2024-04-14T08:42:39.904995+0000","last_fullsized":"2024-04-14T08:42:39.904995+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:45:17.425830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900508+0000","last_change":"2024-04-14T08:42:39.900508+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900508+0000","last_undegraded":"2024-04-14T08:42:39.900508+0000","last_fullsized":"2024-04-14T08:42:39.900508+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.920415+0000","last_change":"2024-04-14T08:42:39.903458+0000","last_active":"2024-04-14T08:42:40.920415+0000","last_peered":"2024-04-14T08:42:40.920415+0000","last_clean":"2024-04-14T08:42:40.920415+0000","last_became_active":"2024-04-14T08:42:39.903032+0000","last_became_peered":"2024-04-14T08:42:39.903032+0000","last_unstale":"2024-04-14T08:42:40.920415+0000","last_undegraded":"2024-04-14T08:42:40.920415+0000","last_fullsized":"2024-04-14T08:42:40.920415+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:43.408088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.908622+0000","last_change":"2024-04-14T08:42:40.904383+0000","last_active":"2024-04-14T08:42:40.908622+0000","last_peered":"2024-04-14T08:42:40.908622+0000","last_clean":"2024-04-14T08:42:40.908622+0000","last_became_active":"2024-04-14T08:42:40.904099+0000","last_became_peered":"2024-04-14T08:42:40.904099+0000","last_unstale":"2024-04-14T08:42:40.908622+0000","last_undegraded":"2024-04-14T08:42:40.908622+0000","last_fullsized":"2024-04-14T08:42:40.908622+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:37:01.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919569+0000","last_change":"2024-04-14T08:42:39.914891+0000","last_active":"2024-04-14T08:42:39.919569+0000","last_peered":"2024-04-14T08:42:39.919569+0000","last_clean":"2024-04-14T08:42:39.919569+0000","last_became_active":"2024-04-14T08:42:39.914617+0000","last_became_peered":"2024-04-14T08:42:39.914617+0000","last_unstale":"2024-04-14T08:42:39.919569+0000","last_undegraded":"2024-04-14T08:42:39.919569+0000","last_fullsized":"2024-04-14T08:42:39.919569+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:21:27.947470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902219+0000","last_change":"2024-04-14T08:42:39.902219+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902219+0000","last_undegraded":"2024-04-14T08:42:39.902219+0000","last_fullsized":"2024-04-14T08:42:39.902219+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.903615+0000","last_change":"2024-04-14T08:42:39.898512+0000","last_active":"2024-04-14T08:42:39.903615+0000","last_peered":"2024-04-14T08:42:39.903615+0000","last_clean":"2024-04-14T08:42:39.903615+0000","last_became_active":"2024-04-14T08:42:39.898183+0000","last_became_peered":"2024-04-14T08:42:39.898183+0000","last_unstale":"2024-04-14T08:42:39.903615+0000","last_undegraded":"2024-04-14T08:42:39.903615+0000","last_fullsized":"2024-04-14T08:42:39.903615+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:35:14.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.891105+0000","last_change":"2024-04-14T08:42:39.891105+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.891105+0000","last_undegraded":"2024-04-14T08:42:39.891105+0000","last_fullsized":"2024-04-14T08:42:39.891105+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919468+0000","last_change":"2024-04-14T08:42:39.915147+0000","last_active":"2024-04-14T08:42:39.919468+0000","last_peered":"2024-04-14T08:42:39.919468+0000","last_clean":"2024-04-14T08:42:39.919468+0000","last_became_active":"2024-04-14T08:42:39.914929+0000","last_became_peered":"2024-04-14T08:42:39.914929+0000","last_unstale":"2024-04-14T08:42:39.919468+0000","last_undegraded":"2024-04-14T08:42:39.919468+0000","last_fullsized":"2024-04-14T08:42:39.919468+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:54:03.062201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.908821+0000","last_change":"2024-04-14T08:42:39.908821+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.908821+0000","last_undegraded":"2024-04-14T08:42:39.908821+0000","last_fullsized":"2024-04-14T08:42:39.908821+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.908946+0000","last_change":"2024-04-14T08:42:39.904121+0000","last_active":"2024-04-14T08:42:39.908946+0000","last_peered":"2024-04-14T08:42:39.908946+0000","last_clean":"2024-04-14T08:42:39.908946+0000","last_became_active":"2024-04-14T08:42:39.903879+0000","last_became_peered":"2024-04-14T08:42:39.903879+0000","last_unstale":"2024-04-14T08:42:39.908946+0000","last_undegraded":"2024-04-14T08:42:39.908946+0000","last_fullsized":"2024-04-14T08:42:39.908946+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:25:18.927294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902699+0000","last_change":"2024-04-14T08:42:39.902699+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902699+0000","last_undegraded":"2024-04-14T08:42:39.902699+0000","last_fullsized":"2024-04-14T08:42:39.902699+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.905886+0000","last_change":"2024-04-14T08:42:39.900795+0000","last_active":"2024-04-14T08:42:39.905886+0000","last_peered":"2024-04-14T08:42:39.905886+0000","last_clean":"2024-04-14T08:42:39.905886+0000","last_became_active":"2024-04-14T08:42:39.900345+0000","last_became_peered":"2024-04-14T08:42:39.900345+0000","last_unstale":"2024-04-14T08:42:39.905886+0000","last_undegraded":"2024-04-14T08:42:39.905886+0000","last_fullsized":"2024-04-14T08:42:39.905886+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:38:16.668417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.896257+0000","last_change":"2024-04-14T08:42:39.896257+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.896257+0000","last_undegraded":"2024-04-14T08:42:39.896257+0000","last_fullsized":"2024-04-14T08:42:39.896257+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911424+0000","last_change":"2024-04-14T08:42:39.908268+0000","last_active":"2024-04-14T08:42:39.911424+0000","last_peered":"2024-04-14T08:42:39.911424+0000","last_clean":"2024-04-14T08:42:39.911424+0000","last_became_active":"2024-04-14T08:42:39.907971+0000","last_became_peered":"2024-04-14T08:42:39.907971+0000","last_unstale":"2024-04-14T08:42:39.911424+0000","last_undegraded":"2024-04-14T08:42:39.911424+0000","last_fullsized":"2024-04-14T08:42:39.911424+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:07:34.836827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900647+0000","last_change":"2024-04-14T08:42:39.900647+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900647+0000","last_undegraded":"2024-04-14T08:42:39.900647+0000","last_fullsized":"2024-04-14T08:42:39.900647+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.916586+0000","last_change":"2024-04-14T08:42:39.911711+0000","last_active":"2024-04-14T08:42:39.916586+0000","last_peered":"2024-04-14T08:42:39.916586+0000","last_clean":"2024-04-14T08:42:39.916586+0000","last_became_active":"2024-04-14T08:42:39.911408+0000","last_became_peered":"2024-04-14T08:42:39.911408+0000","last_unstale":"2024-04-14T08:42:39.916586+0000","last_undegraded":"2024-04-14T08:42:39.916586+0000","last_fullsized":"2024-04-14T08:42:39.916586+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:59.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.905288+0000","last_change":"2024-04-14T08:42:39.905288+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.905288+0000","last_undegraded":"2024-04-14T08:42:39.905288+0000","last_fullsized":"2024-04-14T08:42:39.905288+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911434+0000","last_change":"2024-04-14T08:42:39.906942+0000","last_active":"2024-04-14T08:42:39.911434+0000","last_peered":"2024-04-14T08:42:39.911434+0000","last_clean":"2024-04-14T08:42:39.911434+0000","last_became_active":"2024-04-14T08:42:39.906650+0000","last_became_peered":"2024-04-14T08:42:39.906650+0000","last_unstale":"2024-04-14T08:42:39.911434+0000","last_undegraded":"2024-04-14T08:42:39.911434+0000","last_fullsized":"2024-04-14T08:42:39.911434+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:16:05.440009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900522+0000","last_change":"2024-04-14T08:42:39.900522+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900522+0000","last_undegraded":"2024-04-14T08:42:39.900522+0000","last_fullsized":"2024-04-14T08:42:39.900522+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.922215+0000","last_change":"2024-04-14T08:42:39.917311+0000","last_active":"2024-04-14T08:42:39.922215+0000","last_peered":"2024-04-14T08:42:39.922215+0000","last_clean":"2024-04-14T08:42:39.922215+0000","last_became_active":"2024-04-14T08:42:39.917001+0000","last_became_peered":"2024-04-14T08:42:39.917001+0000","last_unstale":"2024-04-14T08:42:39.922215+0000","last_undegraded":"2024-04-14T08:42:39.922215+0000","last_fullsized":"2024-04-14T08:42:39.922215+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:25:26.756136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.896919+0000","last_change":"2024-04-14T08:42:39.896919+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.896919+0000","last_undegraded":"2024-04-14T08:42:39.896919+0000","last_fullsized":"2024-04-14T08:42:39.896919+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919614+0000","last_change":"2024-04-14T08:42:39.914644+0000","last_active":"2024-04-14T08:42:39.919614+0000","last_peered":"2024-04-14T08:42:39.919614+0000","last_clean":"2024-04-14T08:42:39.919614+0000","last_became_active":"2024-04-14T08:42:39.914350+0000","last_became_peered":"2024-04-14T08:42:39.914350+0000","last_unstale":"2024-04-14T08:42:39.919614+0000","last_undegraded":"2024-04-14T08:42:39.919614+0000","last_fullsized":"2024-04-14T08:42:39.919614+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:35:14.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902158+0000","last_change":"2024-04-14T08:42:39.902158+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902158+0000","last_undegraded":"2024-04-14T08:42:39.902158+0000","last_fullsized":"2024-04-14T08:42:39.902158+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919597+0000","last_change":"2024-04-14T08:42:39.914644+0000","last_active":"2024-04-14T08:42:39.919597+0000","last_peered":"2024-04-14T08:42:39.919597+0000","last_clean":"2024-04-14T08:42:39.919597+0000","last_became_active":"2024-04-14T08:42:39.914387+0000","last_became_peered":"2024-04-14T08:42:39.914387+0000","last_unstale":"2024-04-14T08:42:39.919597+0000","last_undegraded":"2024-04-14T08:42:39.919597+0000","last_fullsized":"2024-04-14T08:42:39.919597+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:59.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.905085+0000","last_change":"2024-04-14T08:42:39.905085+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.905085+0000","last_undegraded":"2024-04-14T08:42:39.905085+0000","last_fullsized":"2024-04-14T08:42:39.905085+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.922210+0000","last_change":"2024-04-14T08:42:39.917235+0000","last_active":"2024-04-14T08:42:39.922210+0000","last_peered":"2024-04-14T08:42:39.922210+0000","last_clean":"2024-04-14T08:42:39.922210+0000","last_became_active":"2024-04-14T08:42:39.916937+0000","last_became_peered":"2024-04-14T08:42:39.916937+0000","last_unstale":"2024-04-14T08:42:39.922210+0000","last_undegraded":"2024-04-14T08:42:39.922210+0000","last_fullsized":"2024-04-14T08:42:39.922210+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:07:34.836827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.899650+0000","last_change":"2024-04-14T08:42:39.899650+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.899650+0000","last_undegraded":"2024-04-14T08:42:39.899650+0000","last_fullsized":"2024-04-14T08:42:39.899650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911878+0000","last_change":"2024-04-14T08:42:39.907928+0000","last_active":"2024-04-14T08:42:39.911878+0000","last_peered":"2024-04-14T08:42:39.911878+0000","last_clean":"2024-04-14T08:42:39.911878+0000","last_became_active":"2024-04-14T08:42:39.907662+0000","last_became_peered":"2024-04-14T08:42:39.907662+0000","last_unstale":"2024-04-14T08:42:39.911878+0000","last_undegraded":"2024-04-14T08:42:39.911878+0000","last_fullsized":"2024-04-14T08:42:39.911878+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:42:44.082970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.892168+0000","last_change":"2024-04-14T08:42:39.892168+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.892168+0000","last_undegraded":"2024-04-14T08:42:39.892168+0000","last_fullsized":"2024-04-14T08:42:39.892168+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.910329+0000","last_change":"2024-04-14T08:42:39.903548+0000","last_active":"2024-04-14T08:42:39.910329+0000","last_peered":"2024-04-14T08:42:39.910329+0000","last_clean":"2024-04-14T08:42:39.910329+0000","last_became_active":"2024-04-14T08:42:39.903272+0000","last_became_peered":"2024-04-14T08:42:39.903272+0000","last_unstale":"2024-04-14T08:42:39.910329+0000","last_undegraded":"2024-04-14T08:42:39.910329+0000","last_fullsized":"2024-04-14T08:42:39.910329+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:31:53.521063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904059+0000","last_change":"2024-04-14T08:42:39.904059+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904059+0000","last_undegraded":"2024-04-14T08:42:39.904059+0000","last_fullsized":"2024-04-14T08:42:39.904059+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.922186+0000","last_change":"2024-04-14T08:42:39.907861+0000","last_active":"2024-04-14T08:42:40.922186+0000","last_peered":"2024-04-14T08:42:40.922186+0000","last_clean":"2024-04-14T08:42:40.922186+0000","last_became_active":"2024-04-14T08:42:39.907144+0000","last_became_peered":"2024-04-14T08:42:39.907144+0000","last_unstale":"2024-04-14T08:42:40.922186+0000","last_undegraded":"2024-04-14T08:42:40.922186+0000","last_fullsized":"2024-04-14T08:42:40.922186+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:50.027906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900070+0000","last_change":"2024-04-14T08:42:39.900070+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900070+0000","last_undegraded":"2024-04-14T08:42:39.900070+0000","last_fullsized":"2024-04-14T08:42:39.900070+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.895697+0000","last_change":"2024-04-14T08:42:39.900873+0000","last_active":"2024-04-14T08:42:40.895697+0000","last_peered":"2024-04-14T08:42:40.895697+0000","last_clean":"2024-04-14T08:42:40.895697+0000","last_became_active":"2024-04-14T08:42:39.900603+0000","last_became_peered":"2024-04-14T08:42:39.900603+0000","last_unstale":"2024-04-14T08:42:40.895697+0000","last_undegraded":"2024-04-14T08:42:40.895697+0000","last_fullsized":"2024-04-14T08:42:40.895697+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:29.063677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902777+0000","last_change":"2024-04-14T08:42:39.902777+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902777+0000","last_undegraded":"2024-04-14T08:42:39.902777+0000","last_fullsized":"2024-04-14T08:42:39.902777+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911312+0000","last_change":"2024-04-14T08:42:39.904952+0000","last_active":"2024-04-14T08:42:39.911312+0000","last_peered":"2024-04-14T08:42:39.911312+0000","last_clean":"2024-04-14T08:42:39.911312+0000","last_became_active":"2024-04-14T08:42:39.904677+0000","last_became_peered":"2024-04-14T08:42:39.904677+0000","last_unstale":"2024-04-14T08:42:39.911312+0000","last_undegraded":"2024-04-14T08:42:39.911312+0000","last_fullsized":"2024-04-14T08:42:39.911312+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:45:07.221511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904226+0000","last_change":"2024-04-14T08:42:39.904226+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904226+0000","last_undegraded":"2024-04-14T08:42:39.904226+0000","last_fullsized":"2024-04-14T08:42:39.904226+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.894138+0000","last_change":"2024-04-14T08:42:39.902053+0000","last_active":"2024-04-14T08:42:40.894138+0000","last_peered":"2024-04-14T08:42:40.894138+0000","last_clean":"2024-04-14T08:42:40.894138+0000","last_became_active":"2024-04-14T08:42:39.901709+0000","last_became_peered":"2024-04-14T08:42:39.901709+0000","last_unstale":"2024-04-14T08:42:40.894138+0000","last_undegraded":"2024-04-14T08:42:40.894138+0000","last_fullsized":"2024-04-14T08:42:40.894138+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:58:25.264866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.908418+0000","last_change":"2024-04-14T08:42:39.908418+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.908418+0000","last_undegraded":"2024-04-14T08:42:39.908418+0000","last_fullsized":"2024-04-14T08:42:39.908418+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.922233+0000","last_change":"2024-04-14T08:42:39.917517+0000","last_active":"2024-04-14T08:42:39.922233+0000","last_peered":"2024-04-14T08:42:39.922233+0000","last_clean":"2024-04-14T08:42:39.922233+0000","last_became_active":"2024-04-14T08:42:39.917246+0000","last_became_peered":"2024-04-14T08:42:39.917246+0000","last_unstale":"2024-04-14T08:42:39.922233+0000","last_undegraded":"2024-04-14T08:42:39.922233+0000","last_fullsized":"2024-04-14T08:42:39.922233+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:41.656802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.899442+0000","last_change":"2024-04-14T08:42:39.899442+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.899442+0000","last_undegraded":"2024-04-14T08:42:39.899442+0000","last_fullsized":"2024-04-14T08:42:39.899442+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.893438+0000","last_change":"2024-04-14T08:42:39.903314+0000","last_active":"2024-04-14T08:42:40.893438+0000","last_peered":"2024-04-14T08:42:40.893438+0000","last_clean":"2024-04-14T08:42:40.893438+0000","last_became_active":"2024-04-14T08:42:39.903044+0000","last_became_peered":"2024-04-14T08:42:39.903044+0000","last_unstale":"2024-04-14T08:42:40.893438+0000","last_undegraded":"2024-04-14T08:42:40.893438+0000","last_fullsized":"2024-04-14T08:42:40.893438+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:32:34.297541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903140+0000","last_change":"2024-04-14T08:42:39.903140+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903140+0000","last_undegraded":"2024-04-14T08:42:39.903140+0000","last_fullsized":"2024-04-14T08:42:39.903140+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.905903+0000","last_change":"2024-04-14T08:42:39.899765+0000","last_active":"2024-04-14T08:42:39.905903+0000","last_peered":"2024-04-14T08:42:39.905903+0000","last_clean":"2024-04-14T08:42:39.905903+0000","last_became_active":"2024-04-14T08:42:39.898985+0000","last_became_peered":"2024-04-14T08:42:39.898985+0000","last_unstale":"2024-04-14T08:42:39.905903+0000","last_undegraded":"2024-04-14T08:42:39.905903+0000","last_fullsized":"2024-04-14T08:42:39.905903+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:00:31.753726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.901623+0000","last_change":"2024-04-14T08:42:39.901623+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.901623+0000","last_undegraded":"2024-04-14T08:42:39.901623+0000","last_fullsized":"2024-04-14T08:42:39.901623+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911433+0000","last_change":"2024-04-14T08:42:39.908137+0000","last_active":"2024-04-14T08:42:39.911433+0000","last_peered":"2024-04-14T08:42:39.911433+0000","last_clean":"2024-04-14T08:42:39.911433+0000","last_became_active":"2024-04-14T08:42:39.907857+0000","last_became_peered":"2024-04-14T08:42:39.907857+0000","last_unstale":"2024-04-14T08:42:39.911433+0000","last_undegraded":"2024-04-14T08:42:39.911433+0000","last_fullsized":"2024-04-14T08:42:39.911433+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:42:20.871047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.910001+0000","last_change":"2024-04-14T08:42:40.904731+0000","last_active":"2024-04-14T08:42:40.910001+0000","last_peered":"2024-04-14T08:42:40.910001+0000","last_clean":"2024-04-14T08:42:40.910001+0000","last_became_active":"2024-04-14T08:42:40.904169+0000","last_became_peered":"2024-04-14T08:42:40.904169+0000","last_unstale":"2024-04-14T08:42:40.910001+0000","last_undegraded":"2024-04-14T08:42:40.910001+0000","last_fullsized":"2024-04-14T08:42:40.910001+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:40.822681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911401+0000","last_change":"2024-04-14T08:42:39.907401+0000","last_active":"2024-04-14T08:42:39.911401+0000","last_peered":"2024-04-14T08:42:39.911401+0000","last_clean":"2024-04-14T08:42:39.911401+0000","last_became_active":"2024-04-14T08:42:39.907124+0000","last_became_peered":"2024-04-14T08:42:39.907124+0000","last_unstale":"2024-04-14T08:42:39.911401+0000","last_undegraded":"2024-04-14T08:42:39.911401+0000","last_fullsized":"2024-04-14T08:42:39.911401+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:35:14.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904317+0000","last_change":"2024-04-14T08:42:39.904317+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904317+0000","last_undegraded":"2024-04-14T08:42:39.904317+0000","last_fullsized":"2024-04-14T08:42:39.904317+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911743+0000","last_change":"2024-04-14T08:42:39.901863+0000","last_active":"2024-04-14T08:42:39.911743+0000","last_peered":"2024-04-14T08:42:39.911743+0000","last_clean":"2024-04-14T08:42:39.911743+0000","last_became_active":"2024-04-14T08:42:39.901612+0000","last_became_peered":"2024-04-14T08:42:39.901612+0000","last_unstale":"2024-04-14T08:42:39.911743+0000","last_undegraded":"2024-04-14T08:42:39.911743+0000","last_fullsized":"2024-04-14T08:42:39.911743+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:52:47.647486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.901878+0000","last_change":"2024-04-14T08:42:39.901878+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.901878+0000","last_undegraded":"2024-04-14T08:42:39.901878+0000","last_fullsized":"2024-04-14T08:42:39.901878+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.903592+0000","last_change":"2024-04-14T08:42:39.898079+0000","last_active":"2024-04-14T08:42:39.903592+0000","last_peered":"2024-04-14T08:42:39.903592+0000","last_clean":"2024-04-14T08:42:39.903592+0000","last_became_active":"2024-04-14T08:42:39.897776+0000","last_became_peered":"2024-04-14T08:42:39.897776+0000","last_unstale":"2024-04-14T08:42:39.903592+0000","last_undegraded":"2024-04-14T08:42:39.903592+0000","last_fullsized":"2024-04-14T08:42:39.903592+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:59.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903815+0000","last_change":"2024-04-14T08:42:39.903815+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903815+0000","last_undegraded":"2024-04-14T08:42:39.903815+0000","last_fullsized":"2024-04-14T08:42:39.903815+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.908711+0000","last_change":"2024-04-14T08:42:39.907024+0000","last_active":"2024-04-14T08:42:39.908711+0000","last_peered":"2024-04-14T08:42:39.908711+0000","last_clean":"2024-04-14T08:42:39.908711+0000","last_became_active":"2024-04-14T08:42:39.906749+0000","last_became_peered":"2024-04-14T08:42:39.906749+0000","last_unstale":"2024-04-14T08:42:39.908711+0000","last_undegraded":"2024-04-14T08:42:39.908711+0000","last_fullsized":"2024-04-14T08:42:39.908711+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:42:20.871047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.914252+0000","last_change":"2024-04-14T08:42:40.909631+0000","last_active":"2024-04-14T08:42:40.914252+0000","last_peered":"2024-04-14T08:42:40.914252+0000","last_clean":"2024-04-14T08:42:40.914252+0000","last_became_active":"2024-04-14T08:42:40.909358+0000","last_became_peered":"2024-04-14T08:42:40.909358+0000","last_unstale":"2024-04-14T08:42:40.914252+0000","last_undegraded":"2024-04-14T08:42:40.914252+0000","last_fullsized":"2024-04-14T08:42:40.914252+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:56.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.900604+0000","last_change":"2024-04-14T08:42:39.899535+0000","last_active":"2024-04-14T08:42:39.900604+0000","last_peered":"2024-04-14T08:42:39.900604+0000","last_clean":"2024-04-14T08:42:39.900604+0000","last_became_active":"2024-04-14T08:42:39.899245+0000","last_became_peered":"2024-04-14T08:42:39.899245+0000","last_unstale":"2024-04-14T08:42:39.900604+0000","last_undegraded":"2024-04-14T08:42:39.900604+0000","last_fullsized":"2024-04-14T08:42:39.900604+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:55:11.795053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.901718+0000","last_change":"2024-04-14T08:42:39.901718+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.901718+0000","last_undegraded":"2024-04-14T08:42:39.901718+0000","last_fullsized":"2024-04-14T08:42:39.901718+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.893537+0000","last_change":"2024-04-14T08:42:39.903168+0000","last_active":"2024-04-14T08:42:40.893537+0000","last_peered":"2024-04-14T08:42:40.893537+0000","last_clean":"2024-04-14T08:42:40.893537+0000","last_became_active":"2024-04-14T08:42:39.902874+0000","last_became_peered":"2024-04-14T08:42:39.902874+0000","last_unstale":"2024-04-14T08:42:40.893537+0000","last_undegraded":"2024-04-14T08:42:40.893537+0000","last_fullsized":"2024-04-14T08:42:40.893537+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:34:54.764802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.916751+0000","last_change":"2024-04-14T08:42:40.913643+0000","last_active":"2024-04-14T08:42:40.916751+0000","last_peered":"2024-04-14T08:42:40.916751+0000","last_clean":"2024-04-14T08:42:40.916751+0000","last_became_active":"2024-04-14T08:42:40.912975+0000","last_became_peered":"2024-04-14T08:42:40.912975+0000","last_unstale":"2024-04-14T08:42:40.916751+0000","last_undegraded":"2024-04-14T08:42:40.916751+0000","last_fullsized":"2024-04-14T08:42:40.916751+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:28.290311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919652+0000","last_change":"2024-04-14T08:42:39.915077+0000","last_active":"2024-04-14T08:42:39.919652+0000","last_peered":"2024-04-14T08:42:39.919652+0000","last_clean":"2024-04-14T08:42:39.919652+0000","last_became_active":"2024-04-14T08:42:39.914803+0000","last_became_peered":"2024-04-14T08:42:39.914803+0000","last_unstale":"2024-04-14T08:42:39.919652+0000","last_undegraded":"2024-04-14T08:42:39.919652+0000","last_fullsized":"2024-04-14T08:42:39.919652+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:42:20.871047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":362,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.893578+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:42:40.893578+0000","last_peered":"2024-04-14T08:42:40.893578+0000","last_clean":"2024-04-14T08:42:40.893578+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:42:40.893578+0000","last_undegraded":"2024-04-14T08:42:40.893578+0000","last_fullsized":"2024-04-14T08:42:40.893578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2015232,"data_stored":1250329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1409024,"data_stored":653294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1409024,"data_stored":653294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1409024,"data_stored":653294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574971,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":659961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":659961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1409024,"data_stored":653294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2002944,"data_stored":1243662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:42:43.954 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:42:44.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.055+0000 7fae1031e640 1 Processor -- start 2024-04-14T08:42:44.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.055+0000 7fae1031e640 1 -- start start 2024-04-14T08:42:44.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae1031e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fae080729b0 0x7fae0806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae1031e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fae0806e160 0x7fae0806c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae1031e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae080b41d0 con 0x7fae0815e5b0 2024-04-14T08:42:44.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae1031e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fae080aebe0 con 0x7fae080729b0 2024-04-14T08:42:44.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae1031e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fae080b4470 con 0x7fae0806e160 2024-04-14T08:42:44.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae0e0ba640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fae0815e5b0 0x7fae080ae390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34194/0 (socket says 172.21.15.172:34194) 2024-04-14T08:42:44.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae0d8b9640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fae080729b0 0x7fae0806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae0e0ba640 1 -- 172.21.15.172:0/2954904827 learned_addr learned my addr 172.21.15.172:0/2954904827 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae0e8bb640 1 --2- 172.21.15.172:0/2954904827 >> v2:172.21.15.177:3300/0 conn(0x7fae0806e160 0x7fae0806c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae0d0b8640 1 -- 172.21.15.172:0/2954904827 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1199858246 0 0) 0x7fae080b41d0 con 0x7fae0815e5b0 2024-04-14T08:42:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae0d0b8640 1 -- 172.21.15.172:0/2954904827 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadf4003520 con 0x7fae0815e5b0 2024-04-14T08:42:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae0d8b9640 1 -- 172.21.15.172:0/2954904827 >> v2:172.21.15.177:3300/0 conn(0x7fae0806e160 msgr2=0x7fae0806c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae0d8b9640 1 --2- 172.21.15.172:0/2954904827 >> v2:172.21.15.177:3300/0 conn(0x7fae0806e160 0x7fae0806c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae0d8b9640 1 -- 172.21.15.172:0/2954904827 >> v1:172.21.15.172:6789/0 conn(0x7fae0815e5b0 legacy=0x7fae080ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.056+0000 7fae0d8b9640 1 -- 172.21.15.172:0/2954904827 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae080aeec0 con 0x7fae080729b0 2024-04-14T08:42:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.057+0000 7fae0d8b9640 1 --2- 172.21.15.172:0/2954904827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080729b0 0x7fae0806dc20 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fadf80097a0 tx=0x7fadf802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72998313ae749bee server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.057+0000 7fae0d0b8640 1 -- 172.21.15.172:0/2954904827 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8035020 con 0x7fae080729b0 2024-04-14T08:42:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.057+0000 7fae0d0b8640 1 -- 172.21.15.172:0/2954904827 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf8031830 con 0x7fae080729b0 2024-04-14T08:42:44.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.057+0000 7fae0d0b8640 1 -- 172.21.15.172:0/2954904827 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8031c00 con 0x7fae080729b0 2024-04-14T08:42:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.058+0000 7fae1031e640 1 -- 172.21.15.172:0/2954904827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080729b0 msgr2=0x7fae0806dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.058+0000 7fae1031e640 1 --2- 172.21.15.172:0/2954904827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080729b0 0x7fae0806dc20 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fadf80097a0 tx=0x7fadf802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.058+0000 7fae1031e640 1 -- 172.21.15.172:0/2954904827 shutdown_connections 2024-04-14T08:42:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.058+0000 7fae1031e640 1 --2- 172.21.15.172:0/2954904827 >> v2:172.21.15.177:3300/0 conn(0x7fae0806e160 0x7fae0806c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.058+0000 7fae1031e640 1 --2- 172.21.15.172:0/2954904827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080729b0 0x7fae0806dc20 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.058+0000 7fae1031e640 1 -- 172.21.15.172:0/2954904827 >> 172.21.15.172:0/2954904827 conn(0x7fae08071ba0 msgr2=0x7fae08071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.058+0000 7fae1031e640 1 -- 172.21.15.172:0/2954904827 shutdown_connections 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.058+0000 7fae1031e640 1 -- 172.21.15.172:0/2954904827 wait complete. 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.059+0000 7fae1031e640 1 Processor -- start 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.059+0000 7fae1031e640 1 -- start start 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae1031e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae0806e160 0x7fae08152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae1031e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080729b0 0x7fae08160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae1031e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae0815e5b0 0x7fae08160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae1031e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae0806ce00 con 0x7fae080729b0 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae1031e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fae0806cc80 con 0x7fae0815e5b0 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae1031e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fae080b4c70 con 0x7fae0806e160 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae0d8b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080729b0 0x7fae08160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae0d8b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080729b0 0x7fae08160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39832/0 (socket says 172.21.15.172:39832) 2024-04-14T08:42:44.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae0d8b9640 1 -- 172.21.15.172:0/362927847 learned_addr learned my addr 172.21.15.172:0/362927847 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae0e8bb640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae0815e5b0 0x7fae08160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae0e0ba640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae0806e160 0x7fae08152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae0d8b9640 1 -- 172.21.15.172:0/362927847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae0806e160 msgr2=0x7fae08152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae0d8b9640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae0806e160 0x7fae08152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae0d8b9640 1 -- 172.21.15.172:0/362927847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae0815e5b0 msgr2=0x7fae08160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae0d8b9640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae0815e5b0 0x7fae08160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae0d8b9640 1 -- 172.21.15.172:0/362927847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae080cda20 con 0x7fae080729b0 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae0d8b9640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080729b0 0x7fae08160590 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fadf802bee0 tx=0x7fadf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fadfeffd640 1 -- 172.21.15.172:0/362927847 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8035040 con 0x7fae080729b0 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fadfeffd640 1 -- 172.21.15.172:0/362927847 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf803b920 con 0x7fae080729b0 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae1031e640 1 -- 172.21.15.172:0/362927847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0815e990 con 0x7fae080729b0 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fadfeffd640 1 -- 172.21.15.172:0/362927847 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf803bd10 con 0x7fae080729b0 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.060+0000 7fae1031e640 1 -- 172.21.15.172:0/362927847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0815ef50 con 0x7fae080729b0 2024-04-14T08:42:44.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.061+0000 7fae1031e640 1 -- 172.21.15.172:0/362927847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae08068520 con 0x7fae080729b0 2024-04-14T08:42:44.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.062+0000 7fadfeffd640 1 -- 172.21.15.172:0/362927847 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadf8038020 con 0x7fae080729b0 2024-04-14T08:42:44.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.062+0000 7fadfeffd640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fadec06c8f0 0x7fadec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.062+0000 7fae0e0ba640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fadec06c8f0 0x7fadec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.062+0000 7fadfeffd640 1 -- 172.21.15.172:0/362927847 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7fadf80adc10 con 0x7fae080729b0 2024-04-14T08:42:44.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.063+0000 7fae0e0ba640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fadec06c8f0 0x7fadec06edb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fae04005880 tx=0x7fae04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:44.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.067+0000 7fadfeffd640 1 -- 172.21.15.172:0/362927847 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fadf807b8f0 con 0x7fae080729b0 2024-04-14T08:42:44.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.239+0000 7fae1031e640 1 -- 172.21.15.172:0/362927847 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae08072730 con 0x7fadec06c8f0 2024-04-14T08:42:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.245+0000 7fadfeffd640 1 -- 172.21.15.172:0/362927847 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247762 (secure 0 0 0) 0x7fae08072730 con 0x7fadec06c8f0 2024-04-14T08:42:44.248 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:44.249 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:42:44.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.248+0000 7fae1031e640 1 -- 172.21.15.172:0/362927847 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fadec06c8f0 msgr2=0x7fadec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.248+0000 7fae1031e640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fadec06c8f0 0x7fadec06edb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fae04005880 tx=0x7fae04009040 comp rx=0 tx=0).stop 2024-04-14T08:42:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.248+0000 7fae1031e640 1 -- 172.21.15.172:0/362927847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080729b0 msgr2=0x7fae08160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.248+0000 7fae1031e640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080729b0 0x7fae08160590 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fadf802bee0 tx=0x7fadf80046c0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.248+0000 7fae1031e640 1 -- 172.21.15.172:0/362927847 shutdown_connections 2024-04-14T08:42:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.248+0000 7fae1031e640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fadec06c8f0 0x7fadec06edb0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.248+0000 7fae1031e640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae0815e5b0 0x7fae08160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.248+0000 7fae1031e640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae080729b0 0x7fae08160590 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.248+0000 7fae1031e640 1 --2- 172.21.15.172:0/362927847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae0806e160 0x7fae08152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.248+0000 7fae1031e640 1 -- 172.21.15.172:0/362927847 >> 172.21.15.172:0/362927847 conn(0x7fae08071ba0 msgr2=0x7fae0806ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.249+0000 7fae1031e640 1 -- 172.21.15.172:0/362927847 shutdown_connections 2024-04-14T08:42:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.249+0000 7fae1031e640 1 -- 172.21.15.172:0/362927847 wait complete. 2024-04-14T08:42:44.259 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-04-14T08:42:43.237918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261656,"kb_used_data":12204,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713064,"statfs":{"total":773094113280,"available":772826177536,"internally_reserved":0,"allocated":12496896,"data_stored":6427089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.355032"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.891452+0000","last_change":"2024-04-14T08:42:39.891452+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.891452+0000","last_undegraded":"2024-04-14T08:42:39.891452+0000","last_fullsized":"2024-04-14T08:42:39.891452+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.906122+0000","last_change":"2024-04-14T08:42:39.906122+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.906122+0000","last_undegraded":"2024-04-14T08:42:39.906122+0000","last_fullsized":"2024-04-14T08:42:39.906122+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904781+0000","last_change":"2024-04-14T08:42:39.904781+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904781+0000","last_undegraded":"2024-04-14T08:42:39.904781+0000","last_fullsized":"2024-04-14T08:42:39.904781+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.900277+0000","last_change":"2024-04-14T08:42:40.899261+0000","last_active":"2024-04-14T08:42:40.900277+0000","last_peered":"2024-04-14T08:42:40.900277+0000","last_clean":"2024-04-14T08:42:40.900277+0000","last_became_active":"2024-04-14T08:42:40.898970+0000","last_became_peered":"2024-04-14T08:42:40.898970+0000","last_unstale":"2024-04-14T08:42:40.900277+0000","last_undegraded":"2024-04-14T08:42:40.900277+0000","last_fullsized":"2024-04-14T08:42:40.900277+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:57.005587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.891722+0000","last_change":"2024-04-14T08:42:39.891722+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.891722+0000","last_undegraded":"2024-04-14T08:42:39.891722+0000","last_fullsized":"2024-04-14T08:42:39.891722+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904215+0000","last_change":"2024-04-14T08:42:39.904215+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904215+0000","last_undegraded":"2024-04-14T08:42:39.904215+0000","last_fullsized":"2024-04-14T08:42:39.904215+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.910912+0000","last_change":"2024-04-14T08:42:39.910912+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.910912+0000","last_undegraded":"2024-04-14T08:42:39.910912+0000","last_fullsized":"2024-04-14T08:42:39.910912+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.907664+0000","last_change":"2024-04-14T08:42:39.907664+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.907664+0000","last_undegraded":"2024-04-14T08:42:39.907664+0000","last_fullsized":"2024-04-14T08:42:39.907664+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.914262+0000","last_change":"2024-04-14T08:42:40.909455+0000","last_active":"2024-04-14T08:42:40.914262+0000","last_peered":"2024-04-14T08:42:40.914262+0000","last_clean":"2024-04-14T08:42:40.914262+0000","last_became_active":"2024-04-14T08:42:40.909156+0000","last_became_peered":"2024-04-14T08:42:40.909156+0000","last_unstale":"2024-04-14T08:42:40.914262+0000","last_undegraded":"2024-04-14T08:42:40.914262+0000","last_fullsized":"2024-04-14T08:42:40.914262+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:50:25.331351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902706+0000","last_change":"2024-04-14T08:42:39.902706+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902706+0000","last_undegraded":"2024-04-14T08:42:39.902706+0000","last_fullsized":"2024-04-14T08:42:39.902706+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.892580+0000","last_change":"2024-04-14T08:42:39.892580+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.892580+0000","last_undegraded":"2024-04-14T08:42:39.892580+0000","last_fullsized":"2024-04-14T08:42:39.892580+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904944+0000","last_change":"2024-04-14T08:42:39.904944+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904944+0000","last_undegraded":"2024-04-14T08:42:39.904944+0000","last_fullsized":"2024-04-14T08:42:39.904944+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903252+0000","last_change":"2024-04-14T08:42:39.903252+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903252+0000","last_undegraded":"2024-04-14T08:42:39.903252+0000","last_fullsized":"2024-04-14T08:42:39.903252+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.909607+0000","last_change":"2024-04-14T08:42:39.909607+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.909607+0000","last_undegraded":"2024-04-14T08:42:39.909607+0000","last_fullsized":"2024-04-14T08:42:39.909607+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.893081+0000","last_change":"2024-04-14T08:42:39.893081+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.893081+0000","last_undegraded":"2024-04-14T08:42:39.893081+0000","last_fullsized":"2024-04-14T08:42:39.893081+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903738+0000","last_change":"2024-04-14T08:42:39.903738+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903738+0000","last_undegraded":"2024-04-14T08:42:39.903738+0000","last_fullsized":"2024-04-14T08:42:39.903738+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903068+0000","last_change":"2024-04-14T08:42:39.903068+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903068+0000","last_undegraded":"2024-04-14T08:42:39.903068+0000","last_fullsized":"2024-04-14T08:42:39.903068+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.905703+0000","last_change":"2024-04-14T08:42:39.905703+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.905703+0000","last_undegraded":"2024-04-14T08:42:39.905703+0000","last_fullsized":"2024-04-14T08:42:39.905703+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.896903+0000","last_change":"2024-04-14T08:42:39.896903+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.896903+0000","last_undegraded":"2024-04-14T08:42:39.896903+0000","last_fullsized":"2024-04-14T08:42:39.896903+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.908810+0000","last_change":"2024-04-14T08:42:39.908810+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.908810+0000","last_undegraded":"2024-04-14T08:42:39.908810+0000","last_fullsized":"2024-04-14T08:42:39.908810+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.913413+0000","last_change":"2024-04-14T08:42:40.906675+0000","last_active":"2024-04-14T08:42:40.913413+0000","last_peered":"2024-04-14T08:42:40.913413+0000","last_clean":"2024-04-14T08:42:40.913413+0000","last_became_active":"2024-04-14T08:42:40.904423+0000","last_became_peered":"2024-04-14T08:42:40.904423+0000","last_unstale":"2024-04-14T08:42:40.913413+0000","last_undegraded":"2024-04-14T08:42:40.913413+0000","last_fullsized":"2024-04-14T08:42:40.913413+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:42:53.101473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.892991+0000","last_change":"2024-04-14T08:42:39.892991+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.892991+0000","last_undegraded":"2024-04-14T08:42:39.892991+0000","last_fullsized":"2024-04-14T08:42:39.892991+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.921770+0000","last_change":"2024-04-14T08:42:40.919066+0000","last_active":"2024-04-14T08:42:40.921770+0000","last_peered":"2024-04-14T08:42:40.921770+0000","last_clean":"2024-04-14T08:42:40.921770+0000","last_became_active":"2024-04-14T08:42:40.918630+0000","last_became_peered":"2024-04-14T08:42:40.918630+0000","last_unstale":"2024-04-14T08:42:40.921770+0000","last_undegraded":"2024-04-14T08:42:40.921770+0000","last_fullsized":"2024-04-14T08:42:40.921770+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:53:03.886099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.899098+0000","last_change":"2024-04-14T08:42:39.899098+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.899098+0000","last_undegraded":"2024-04-14T08:42:39.899098+0000","last_fullsized":"2024-04-14T08:42:39.899098+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900930+0000","last_change":"2024-04-14T08:42:39.900930+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900930+0000","last_undegraded":"2024-04-14T08:42:39.900930+0000","last_fullsized":"2024-04-14T08:42:39.900930+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.906240+0000","last_change":"2024-04-14T08:42:39.906240+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.906240+0000","last_undegraded":"2024-04-14T08:42:39.906240+0000","last_fullsized":"2024-04-14T08:42:39.906240+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.896498+0000","last_change":"2024-04-14T08:42:39.896498+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.896498+0000","last_undegraded":"2024-04-14T08:42:39.896498+0000","last_fullsized":"2024-04-14T08:42:39.896498+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904518+0000","last_change":"2024-04-14T08:42:39.904518+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904518+0000","last_undegraded":"2024-04-14T08:42:39.904518+0000","last_fullsized":"2024-04-14T08:42:39.904518+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.905764+0000","last_change":"2024-04-14T08:42:39.905764+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.905764+0000","last_undegraded":"2024-04-14T08:42:39.905764+0000","last_fullsized":"2024-04-14T08:42:39.905764+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.909220+0000","last_change":"2024-04-14T08:42:39.909220+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.909220+0000","last_undegraded":"2024-04-14T08:42:39.909220+0000","last_fullsized":"2024-04-14T08:42:39.909220+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.899058+0000","last_change":"2024-04-14T08:42:39.899058+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.899058+0000","last_undegraded":"2024-04-14T08:42:39.899058+0000","last_fullsized":"2024-04-14T08:42:39.899058+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.898579+0000","last_change":"2024-04-14T08:42:39.898579+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.898579+0000","last_undegraded":"2024-04-14T08:42:39.898579+0000","last_fullsized":"2024-04-14T08:42:39.898579+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902637+0000","last_change":"2024-04-14T08:42:39.902637+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902637+0000","last_undegraded":"2024-04-14T08:42:39.902637+0000","last_fullsized":"2024-04-14T08:42:39.902637+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.906183+0000","last_change":"2024-04-14T08:42:39.902925+0000","last_active":"2024-04-14T08:42:39.906183+0000","last_peered":"2024-04-14T08:42:39.906183+0000","last_clean":"2024-04-14T08:42:39.906183+0000","last_became_active":"2024-04-14T08:42:39.902599+0000","last_became_peered":"2024-04-14T08:42:39.902599+0000","last_unstale":"2024-04-14T08:42:39.906183+0000","last_undegraded":"2024-04-14T08:42:39.906183+0000","last_fullsized":"2024-04-14T08:42:39.906183+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:59.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903562+0000","last_change":"2024-04-14T08:42:39.903562+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903562+0000","last_undegraded":"2024-04-14T08:42:39.903562+0000","last_fullsized":"2024-04-14T08:42:39.903562+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.904995+0000","last_change":"2024-04-14T08:42:39.898146+0000","last_active":"2024-04-14T08:42:39.904995+0000","last_peered":"2024-04-14T08:42:39.904995+0000","last_clean":"2024-04-14T08:42:39.904995+0000","last_became_active":"2024-04-14T08:42:39.897857+0000","last_became_peered":"2024-04-14T08:42:39.897857+0000","last_unstale":"2024-04-14T08:42:39.904995+0000","last_undegraded":"2024-04-14T08:42:39.904995+0000","last_fullsized":"2024-04-14T08:42:39.904995+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:45:17.425830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900508+0000","last_change":"2024-04-14T08:42:39.900508+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900508+0000","last_undegraded":"2024-04-14T08:42:39.900508+0000","last_fullsized":"2024-04-14T08:42:39.900508+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.920415+0000","last_change":"2024-04-14T08:42:39.903458+0000","last_active":"2024-04-14T08:42:40.920415+0000","last_peered":"2024-04-14T08:42:40.920415+0000","last_clean":"2024-04-14T08:42:40.920415+0000","last_became_active":"2024-04-14T08:42:39.903032+0000","last_became_peered":"2024-04-14T08:42:39.903032+0000","last_unstale":"2024-04-14T08:42:40.920415+0000","last_undegraded":"2024-04-14T08:42:40.920415+0000","last_fullsized":"2024-04-14T08:42:40.920415+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:43.408088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.908622+0000","last_change":"2024-04-14T08:42:40.904383+0000","last_active":"2024-04-14T08:42:40.908622+0000","last_peered":"2024-04-14T08:42:40.908622+0000","last_clean":"2024-04-14T08:42:40.908622+0000","last_became_active":"2024-04-14T08:42:40.904099+0000","last_became_peered":"2024-04-14T08:42:40.904099+0000","last_unstale":"2024-04-14T08:42:40.908622+0000","last_undegraded":"2024-04-14T08:42:40.908622+0000","last_fullsized":"2024-04-14T08:42:40.908622+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:37:01.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919569+0000","last_change":"2024-04-14T08:42:39.914891+0000","last_active":"2024-04-14T08:42:39.919569+0000","last_peered":"2024-04-14T08:42:39.919569+0000","last_clean":"2024-04-14T08:42:39.919569+0000","last_became_active":"2024-04-14T08:42:39.914617+0000","last_became_peered":"2024-04-14T08:42:39.914617+0000","last_unstale":"2024-04-14T08:42:39.919569+0000","last_undegraded":"2024-04-14T08:42:39.919569+0000","last_fullsized":"2024-04-14T08:42:39.919569+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:21:27.947470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902219+0000","last_change":"2024-04-14T08:42:39.902219+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902219+0000","last_undegraded":"2024-04-14T08:42:39.902219+0000","last_fullsized":"2024-04-14T08:42:39.902219+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.903615+0000","last_change":"2024-04-14T08:42:39.898512+0000","last_active":"2024-04-14T08:42:39.903615+0000","last_peered":"2024-04-14T08:42:39.903615+0000","last_clean":"2024-04-14T08:42:39.903615+0000","last_became_active":"2024-04-14T08:42:39.898183+0000","last_became_peered":"2024-04-14T08:42:39.898183+0000","last_unstale":"2024-04-14T08:42:39.903615+0000","last_undegraded":"2024-04-14T08:42:39.903615+0000","last_fullsized":"2024-04-14T08:42:39.903615+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:35:14.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.891105+0000","last_change":"2024-04-14T08:42:39.891105+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.891105+0000","last_undegraded":"2024-04-14T08:42:39.891105+0000","last_fullsized":"2024-04-14T08:42:39.891105+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919468+0000","last_change":"2024-04-14T08:42:39.915147+0000","last_active":"2024-04-14T08:42:39.919468+0000","last_peered":"2024-04-14T08:42:39.919468+0000","last_clean":"2024-04-14T08:42:39.919468+0000","last_became_active":"2024-04-14T08:42:39.914929+0000","last_became_peered":"2024-04-14T08:42:39.914929+0000","last_unstale":"2024-04-14T08:42:39.919468+0000","last_undegraded":"2024-04-14T08:42:39.919468+0000","last_fullsized":"2024-04-14T08:42:39.919468+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:54:03.062201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.908821+0000","last_change":"2024-04-14T08:42:39.908821+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.908821+0000","last_undegraded":"2024-04-14T08:42:39.908821+0000","last_fullsized":"2024-04-14T08:42:39.908821+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.908946+0000","last_change":"2024-04-14T08:42:39.904121+0000","last_active":"2024-04-14T08:42:39.908946+0000","last_peered":"2024-04-14T08:42:39.908946+0000","last_clean":"2024-04-14T08:42:39.908946+0000","last_became_active":"2024-04-14T08:42:39.903879+0000","last_became_peered":"2024-04-14T08:42:39.903879+0000","last_unstale":"2024-04-14T08:42:39.908946+0000","last_undegraded":"2024-04-14T08:42:39.908946+0000","last_fullsized":"2024-04-14T08:42:39.908946+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:25:18.927294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902699+0000","last_change":"2024-04-14T08:42:39.902699+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902699+0000","last_undegraded":"2024-04-14T08:42:39.902699+0000","last_fullsized":"2024-04-14T08:42:39.902699+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.905886+0000","last_change":"2024-04-14T08:42:39.900795+0000","last_active":"2024-04-14T08:42:39.905886+0000","last_peered":"2024-04-14T08:42:39.905886+0000","last_clean":"2024-04-14T08:42:39.905886+0000","last_became_active":"2024-04-14T08:42:39.900345+0000","last_became_peered":"2024-04-14T08:42:39.900345+0000","last_unstale":"2024-04-14T08:42:39.905886+0000","last_undegraded":"2024-04-14T08:42:39.905886+0000","last_fullsized":"2024-04-14T08:42:39.905886+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:38:16.668417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.896257+0000","last_change":"2024-04-14T08:42:39.896257+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.896257+0000","last_undegraded":"2024-04-14T08:42:39.896257+0000","last_fullsized":"2024-04-14T08:42:39.896257+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911424+0000","last_change":"2024-04-14T08:42:39.908268+0000","last_active":"2024-04-14T08:42:39.911424+0000","last_peered":"2024-04-14T08:42:39.911424+0000","last_clean":"2024-04-14T08:42:39.911424+0000","last_became_active":"2024-04-14T08:42:39.907971+0000","last_became_peered":"2024-04-14T08:42:39.907971+0000","last_unstale":"2024-04-14T08:42:39.911424+0000","last_undegraded":"2024-04-14T08:42:39.911424+0000","last_fullsized":"2024-04-14T08:42:39.911424+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:07:34.836827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900647+0000","last_change":"2024-04-14T08:42:39.900647+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900647+0000","last_undegraded":"2024-04-14T08:42:39.900647+0000","last_fullsized":"2024-04-14T08:42:39.900647+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.916586+0000","last_change":"2024-04-14T08:42:39.911711+0000","last_active":"2024-04-14T08:42:39.916586+0000","last_peered":"2024-04-14T08:42:39.916586+0000","last_clean":"2024-04-14T08:42:39.916586+0000","last_became_active":"2024-04-14T08:42:39.911408+0000","last_became_peered":"2024-04-14T08:42:39.911408+0000","last_unstale":"2024-04-14T08:42:39.916586+0000","last_undegraded":"2024-04-14T08:42:39.916586+0000","last_fullsized":"2024-04-14T08:42:39.916586+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:59.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.905288+0000","last_change":"2024-04-14T08:42:39.905288+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.905288+0000","last_undegraded":"2024-04-14T08:42:39.905288+0000","last_fullsized":"2024-04-14T08:42:39.905288+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911434+0000","last_change":"2024-04-14T08:42:39.906942+0000","last_active":"2024-04-14T08:42:39.911434+0000","last_peered":"2024-04-14T08:42:39.911434+0000","last_clean":"2024-04-14T08:42:39.911434+0000","last_became_active":"2024-04-14T08:42:39.906650+0000","last_became_peered":"2024-04-14T08:42:39.906650+0000","last_unstale":"2024-04-14T08:42:39.911434+0000","last_undegraded":"2024-04-14T08:42:39.911434+0000","last_fullsized":"2024-04-14T08:42:39.911434+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:16:05.440009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900522+0000","last_change":"2024-04-14T08:42:39.900522+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900522+0000","last_undegraded":"2024-04-14T08:42:39.900522+0000","last_fullsized":"2024-04-14T08:42:39.900522+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.922215+0000","last_change":"2024-04-14T08:42:39.917311+0000","last_active":"2024-04-14T08:42:39.922215+0000","last_peered":"2024-04-14T08:42:39.922215+0000","last_clean":"2024-04-14T08:42:39.922215+0000","last_became_active":"2024-04-14T08:42:39.917001+0000","last_became_peered":"2024-04-14T08:42:39.917001+0000","last_unstale":"2024-04-14T08:42:39.922215+0000","last_undegraded":"2024-04-14T08:42:39.922215+0000","last_fullsized":"2024-04-14T08:42:39.922215+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:25:26.756136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.896919+0000","last_change":"2024-04-14T08:42:39.896919+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.896919+0000","last_undegraded":"2024-04-14T08:42:39.896919+0000","last_fullsized":"2024-04-14T08:42:39.896919+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919614+0000","last_change":"2024-04-14T08:42:39.914644+0000","last_active":"2024-04-14T08:42:39.919614+0000","last_peered":"2024-04-14T08:42:39.919614+0000","last_clean":"2024-04-14T08:42:39.919614+0000","last_became_active":"2024-04-14T08:42:39.914350+0000","last_became_peered":"2024-04-14T08:42:39.914350+0000","last_unstale":"2024-04-14T08:42:39.919614+0000","last_undegraded":"2024-04-14T08:42:39.919614+0000","last_fullsized":"2024-04-14T08:42:39.919614+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:35:14.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902158+0000","last_change":"2024-04-14T08:42:39.902158+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902158+0000","last_undegraded":"2024-04-14T08:42:39.902158+0000","last_fullsized":"2024-04-14T08:42:39.902158+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919597+0000","last_change":"2024-04-14T08:42:39.914644+0000","last_active":"2024-04-14T08:42:39.919597+0000","last_peered":"2024-04-14T08:42:39.919597+0000","last_clean":"2024-04-14T08:42:39.919597+0000","last_became_active":"2024-04-14T08:42:39.914387+0000","last_became_peered":"2024-04-14T08:42:39.914387+0000","last_unstale":"2024-04-14T08:42:39.919597+0000","last_undegraded":"2024-04-14T08:42:39.919597+0000","last_fullsized":"2024-04-14T08:42:39.919597+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:59.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.905085+0000","last_change":"2024-04-14T08:42:39.905085+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.905085+0000","last_undegraded":"2024-04-14T08:42:39.905085+0000","last_fullsized":"2024-04-14T08:42:39.905085+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.922210+0000","last_change":"2024-04-14T08:42:39.917235+0000","last_active":"2024-04-14T08:42:39.922210+0000","last_peered":"2024-04-14T08:42:39.922210+0000","last_clean":"2024-04-14T08:42:39.922210+0000","last_became_active":"2024-04-14T08:42:39.916937+0000","last_became_peered":"2024-04-14T08:42:39.916937+0000","last_unstale":"2024-04-14T08:42:39.922210+0000","last_undegraded":"2024-04-14T08:42:39.922210+0000","last_fullsized":"2024-04-14T08:42:39.922210+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:07:34.836827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.899650+0000","last_change":"2024-04-14T08:42:39.899650+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.899650+0000","last_undegraded":"2024-04-14T08:42:39.899650+0000","last_fullsized":"2024-04-14T08:42:39.899650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911878+0000","last_change":"2024-04-14T08:42:39.907928+0000","last_active":"2024-04-14T08:42:39.911878+0000","last_peered":"2024-04-14T08:42:39.911878+0000","last_clean":"2024-04-14T08:42:39.911878+0000","last_became_active":"2024-04-14T08:42:39.907662+0000","last_became_peered":"2024-04-14T08:42:39.907662+0000","last_unstale":"2024-04-14T08:42:39.911878+0000","last_undegraded":"2024-04-14T08:42:39.911878+0000","last_fullsized":"2024-04-14T08:42:39.911878+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:42:44.082970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.892168+0000","last_change":"2024-04-14T08:42:39.892168+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.892168+0000","last_undegraded":"2024-04-14T08:42:39.892168+0000","last_fullsized":"2024-04-14T08:42:39.892168+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.910329+0000","last_change":"2024-04-14T08:42:39.903548+0000","last_active":"2024-04-14T08:42:39.910329+0000","last_peered":"2024-04-14T08:42:39.910329+0000","last_clean":"2024-04-14T08:42:39.910329+0000","last_became_active":"2024-04-14T08:42:39.903272+0000","last_became_peered":"2024-04-14T08:42:39.903272+0000","last_unstale":"2024-04-14T08:42:39.910329+0000","last_undegraded":"2024-04-14T08:42:39.910329+0000","last_fullsized":"2024-04-14T08:42:39.910329+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:31:53.521063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904059+0000","last_change":"2024-04-14T08:42:39.904059+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904059+0000","last_undegraded":"2024-04-14T08:42:39.904059+0000","last_fullsized":"2024-04-14T08:42:39.904059+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.922186+0000","last_change":"2024-04-14T08:42:39.907861+0000","last_active":"2024-04-14T08:42:40.922186+0000","last_peered":"2024-04-14T08:42:40.922186+0000","last_clean":"2024-04-14T08:42:40.922186+0000","last_became_active":"2024-04-14T08:42:39.907144+0000","last_became_peered":"2024-04-14T08:42:39.907144+0000","last_unstale":"2024-04-14T08:42:40.922186+0000","last_undegraded":"2024-04-14T08:42:40.922186+0000","last_fullsized":"2024-04-14T08:42:40.922186+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:50.027906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900070+0000","last_change":"2024-04-14T08:42:39.900070+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900070+0000","last_undegraded":"2024-04-14T08:42:39.900070+0000","last_fullsized":"2024-04-14T08:42:39.900070+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.895697+0000","last_change":"2024-04-14T08:42:39.900873+0000","last_active":"2024-04-14T08:42:40.895697+0000","last_peered":"2024-04-14T08:42:40.895697+0000","last_clean":"2024-04-14T08:42:40.895697+0000","last_became_active":"2024-04-14T08:42:39.900603+0000","last_became_peered":"2024-04-14T08:42:39.900603+0000","last_unstale":"2024-04-14T08:42:40.895697+0000","last_undegraded":"2024-04-14T08:42:40.895697+0000","last_fullsized":"2024-04-14T08:42:40.895697+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:29.063677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902777+0000","last_change":"2024-04-14T08:42:39.902777+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902777+0000","last_undegraded":"2024-04-14T08:42:39.902777+0000","last_fullsized":"2024-04-14T08:42:39.902777+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911312+0000","last_change":"2024-04-14T08:42:39.904952+0000","last_active":"2024-04-14T08:42:39.911312+0000","last_peered":"2024-04-14T08:42:39.911312+0000","last_clean":"2024-04-14T08:42:39.911312+0000","last_became_active":"2024-04-14T08:42:39.904677+0000","last_became_peered":"2024-04-14T08:42:39.904677+0000","last_unstale":"2024-04-14T08:42:39.911312+0000","last_undegraded":"2024-04-14T08:42:39.911312+0000","last_fullsized":"2024-04-14T08:42:39.911312+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:45:07.221511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904226+0000","last_change":"2024-04-14T08:42:39.904226+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904226+0000","last_undegraded":"2024-04-14T08:42:39.904226+0000","last_fullsized":"2024-04-14T08:42:39.904226+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.894138+0000","last_change":"2024-04-14T08:42:39.902053+0000","last_active":"2024-04-14T08:42:40.894138+0000","last_peered":"2024-04-14T08:42:40.894138+0000","last_clean":"2024-04-14T08:42:40.894138+0000","last_became_active":"2024-04-14T08:42:39.901709+0000","last_became_peered":"2024-04-14T08:42:39.901709+0000","last_unstale":"2024-04-14T08:42:40.894138+0000","last_undegraded":"2024-04-14T08:42:40.894138+0000","last_fullsized":"2024-04-14T08:42:40.894138+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:58:25.264866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.908418+0000","last_change":"2024-04-14T08:42:39.908418+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.908418+0000","last_undegraded":"2024-04-14T08:42:39.908418+0000","last_fullsized":"2024-04-14T08:42:39.908418+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.922233+0000","last_change":"2024-04-14T08:42:39.917517+0000","last_active":"2024-04-14T08:42:39.922233+0000","last_peered":"2024-04-14T08:42:39.922233+0000","last_clean":"2024-04-14T08:42:39.922233+0000","last_became_active":"2024-04-14T08:42:39.917246+0000","last_became_peered":"2024-04-14T08:42:39.917246+0000","last_unstale":"2024-04-14T08:42:39.922233+0000","last_undegraded":"2024-04-14T08:42:39.922233+0000","last_fullsized":"2024-04-14T08:42:39.922233+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:41.656802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.899442+0000","last_change":"2024-04-14T08:42:39.899442+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.899442+0000","last_undegraded":"2024-04-14T08:42:39.899442+0000","last_fullsized":"2024-04-14T08:42:39.899442+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.893438+0000","last_change":"2024-04-14T08:42:39.903314+0000","last_active":"2024-04-14T08:42:40.893438+0000","last_peered":"2024-04-14T08:42:40.893438+0000","last_clean":"2024-04-14T08:42:40.893438+0000","last_became_active":"2024-04-14T08:42:39.903044+0000","last_became_peered":"2024-04-14T08:42:39.903044+0000","last_unstale":"2024-04-14T08:42:40.893438+0000","last_undegraded":"2024-04-14T08:42:40.893438+0000","last_fullsized":"2024-04-14T08:42:40.893438+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:32:34.297541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903140+0000","last_change":"2024-04-14T08:42:39.903140+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903140+0000","last_undegraded":"2024-04-14T08:42:39.903140+0000","last_fullsized":"2024-04-14T08:42:39.903140+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.905903+0000","last_change":"2024-04-14T08:42:39.899765+0000","last_active":"2024-04-14T08:42:39.905903+0000","last_peered":"2024-04-14T08:42:39.905903+0000","last_clean":"2024-04-14T08:42:39.905903+0000","last_became_active":"2024-04-14T08:42:39.898985+0000","last_became_peered":"2024-04-14T08:42:39.898985+0000","last_unstale":"2024-04-14T08:42:39.905903+0000","last_undegraded":"2024-04-14T08:42:39.905903+0000","last_fullsized":"2024-04-14T08:42:39.905903+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:00:31.753726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.901623+0000","last_change":"2024-04-14T08:42:39.901623+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.901623+0000","last_undegraded":"2024-04-14T08:42:39.901623+0000","last_fullsized":"2024-04-14T08:42:39.901623+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911433+0000","last_change":"2024-04-14T08:42:39.908137+0000","last_active":"2024-04-14T08:42:39.911433+0000","last_peered":"2024-04-14T08:42:39.911433+0000","last_clean":"2024-04-14T08:42:39.911433+0000","last_became_active":"2024-04-14T08:42:39.907857+0000","last_became_peered":"2024-04-14T08:42:39.907857+0000","last_unstale":"2024-04-14T08:42:39.911433+0000","last_undegraded":"2024-04-14T08:42:39.911433+0000","last_fullsized":"2024-04-14T08:42:39.911433+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:42:20.871047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.910001+0000","last_change":"2024-04-14T08:42:40.904731+0000","last_active":"2024-04-14T08:42:40.910001+0000","last_peered":"2024-04-14T08:42:40.910001+0000","last_clean":"2024-04-14T08:42:40.910001+0000","last_became_active":"2024-04-14T08:42:40.904169+0000","last_became_peered":"2024-04-14T08:42:40.904169+0000","last_unstale":"2024-04-14T08:42:40.910001+0000","last_undegraded":"2024-04-14T08:42:40.910001+0000","last_fullsized":"2024-04-14T08:42:40.910001+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:40.822681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911401+0000","last_change":"2024-04-14T08:42:39.907401+0000","last_active":"2024-04-14T08:42:39.911401+0000","last_peered":"2024-04-14T08:42:39.911401+0000","last_clean":"2024-04-14T08:42:39.911401+0000","last_became_active":"2024-04-14T08:42:39.907124+0000","last_became_peered":"2024-04-14T08:42:39.907124+0000","last_unstale":"2024-04-14T08:42:39.911401+0000","last_undegraded":"2024-04-14T08:42:39.911401+0000","last_fullsized":"2024-04-14T08:42:39.911401+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:35:14.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904317+0000","last_change":"2024-04-14T08:42:39.904317+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904317+0000","last_undegraded":"2024-04-14T08:42:39.904317+0000","last_fullsized":"2024-04-14T08:42:39.904317+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911743+0000","last_change":"2024-04-14T08:42:39.901863+0000","last_active":"2024-04-14T08:42:39.911743+0000","last_peered":"2024-04-14T08:42:39.911743+0000","last_clean":"2024-04-14T08:42:39.911743+0000","last_became_active":"2024-04-14T08:42:39.901612+0000","last_became_peered":"2024-04-14T08:42:39.901612+0000","last_unstale":"2024-04-14T08:42:39.911743+0000","last_undegraded":"2024-04-14T08:42:39.911743+0000","last_fullsized":"2024-04-14T08:42:39.911743+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:52:47.647486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.901878+0000","last_change":"2024-04-14T08:42:39.901878+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.901878+0000","last_undegraded":"2024-04-14T08:42:39.901878+0000","last_fullsized":"2024-04-14T08:42:39.901878+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.903592+0000","last_change":"2024-04-14T08:42:39.898079+0000","last_active":"2024-04-14T08:42:39.903592+0000","last_peered":"2024-04-14T08:42:39.903592+0000","last_clean":"2024-04-14T08:42:39.903592+0000","last_became_active":"2024-04-14T08:42:39.897776+0000","last_became_peered":"2024-04-14T08:42:39.897776+0000","last_unstale":"2024-04-14T08:42:39.903592+0000","last_undegraded":"2024-04-14T08:42:39.903592+0000","last_fullsized":"2024-04-14T08:42:39.903592+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:59.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903815+0000","last_change":"2024-04-14T08:42:39.903815+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903815+0000","last_undegraded":"2024-04-14T08:42:39.903815+0000","last_fullsized":"2024-04-14T08:42:39.903815+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.908711+0000","last_change":"2024-04-14T08:42:39.907024+0000","last_active":"2024-04-14T08:42:39.908711+0000","last_peered":"2024-04-14T08:42:39.908711+0000","last_clean":"2024-04-14T08:42:39.908711+0000","last_became_active":"2024-04-14T08:42:39.906749+0000","last_became_peered":"2024-04-14T08:42:39.906749+0000","last_unstale":"2024-04-14T08:42:39.908711+0000","last_undegraded":"2024-04-14T08:42:39.908711+0000","last_fullsized":"2024-04-14T08:42:39.908711+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:42:20.871047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.914252+0000","last_change":"2024-04-14T08:42:40.909631+0000","last_active":"2024-04-14T08:42:40.914252+0000","last_peered":"2024-04-14T08:42:40.914252+0000","last_clean":"2024-04-14T08:42:40.914252+0000","last_became_active":"2024-04-14T08:42:40.909358+0000","last_became_peered":"2024-04-14T08:42:40.909358+0000","last_unstale":"2024-04-14T08:42:40.914252+0000","last_undegraded":"2024-04-14T08:42:40.914252+0000","last_fullsized":"2024-04-14T08:42:40.914252+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:56.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.900604+0000","last_change":"2024-04-14T08:42:39.899535+0000","last_active":"2024-04-14T08:42:39.900604+0000","last_peered":"2024-04-14T08:42:39.900604+0000","last_clean":"2024-04-14T08:42:39.900604+0000","last_became_active":"2024-04-14T08:42:39.899245+0000","last_became_peered":"2024-04-14T08:42:39.899245+0000","last_unstale":"2024-04-14T08:42:39.900604+0000","last_undegraded":"2024-04-14T08:42:39.900604+0000","last_fullsized":"2024-04-14T08:42:39.900604+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:55:11.795053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.901718+0000","last_change":"2024-04-14T08:42:39.901718+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.901718+0000","last_undegraded":"2024-04-14T08:42:39.901718+0000","last_fullsized":"2024-04-14T08:42:39.901718+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.893537+0000","last_change":"2024-04-14T08:42:39.903168+0000","last_active":"2024-04-14T08:42:40.893537+0000","last_peered":"2024-04-14T08:42:40.893537+0000","last_clean":"2024-04-14T08:42:40.893537+0000","last_became_active":"2024-04-14T08:42:39.902874+0000","last_became_peered":"2024-04-14T08:42:39.902874+0000","last_unstale":"2024-04-14T08:42:40.893537+0000","last_undegraded":"2024-04-14T08:42:40.893537+0000","last_fullsized":"2024-04-14T08:42:40.893537+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:34:54.764802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.916751+0000","last_change":"2024-04-14T08:42:40.913643+0000","last_active":"2024-04-14T08:42:40.916751+0000","last_peered":"2024-04-14T08:42:40.916751+0000","last_clean":"2024-04-14T08:42:40.916751+0000","last_became_active":"2024-04-14T08:42:40.912975+0000","last_became_peered":"2024-04-14T08:42:40.912975+0000","last_unstale":"2024-04-14T08:42:40.916751+0000","last_undegraded":"2024-04-14T08:42:40.916751+0000","last_fullsized":"2024-04-14T08:42:40.916751+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:28.290311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919652+0000","last_change":"2024-04-14T08:42:39.915077+0000","last_active":"2024-04-14T08:42:39.919652+0000","last_peered":"2024-04-14T08:42:39.919652+0000","last_clean":"2024-04-14T08:42:39.919652+0000","last_became_active":"2024-04-14T08:42:39.914803+0000","last_became_peered":"2024-04-14T08:42:39.914803+0000","last_unstale":"2024-04-14T08:42:39.919652+0000","last_undegraded":"2024-04-14T08:42:39.919652+0000","last_fullsized":"2024-04-14T08:42:39.919652+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:42:20.871047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":362,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.893578+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:42:40.893578+0000","last_peered":"2024-04-14T08:42:40.893578+0000","last_clean":"2024-04-14T08:42:40.893578+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:42:40.893578+0000","last_undegraded":"2024-04-14T08:42:40.893578+0000","last_fullsized":"2024-04-14T08:42:40.893578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2015232,"data_stored":1250329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1409024,"data_stored":653294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1409024,"data_stored":653294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1409024,"data_stored":653294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574971,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":659961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":659961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1409024,"data_stored":653294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2002944,"data_stored":1243662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:42:44.263 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:42:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.401+0000 7f10a0588640 1 Processor -- start 2024-04-14T08:42:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.401+0000 7f10a0588640 1 -- start start 2024-04-14T08:42:44.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.402+0000 7f10a0588640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f10980d2830 0x7f10980d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.402+0000 7f10a0588640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f109815e890 0x7f10980d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.402+0000 7f10a0588640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10980d3150 con 0x7f10980cedd0 2024-04-14T08:42:44.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.402+0000 7f10a0588640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f10980cff30 con 0x7f10980d2830 2024-04-14T08:42:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.402+0000 7f10a0588640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f10980d2350 con 0x7f109815e890 2024-04-14T08:42:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.402+0000 7f109e324640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f10980d2830 0x7f10980d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.403+0000 7f109e324640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f10980d2830 0x7f10980d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39844/0 (socket says 172.21.15.172:39844) 2024-04-14T08:42:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.403+0000 7f109e324640 1 -- 172.21.15.172:0/525605182 learned_addr learned my addr 172.21.15.172:0/525605182 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.403+0000 7f109eb25640 1 --1- 172.21.15.172:0/525605182 >> v1:172.21.15.177:6789/0 conn(0x7f10980cedd0 0x7f10981621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:33656/0 (socket says 172.21.15.172:33656) 2024-04-14T08:42:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.403+0000 7f109db23640 1 --2- 172.21.15.172:0/525605182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f109815e890 0x7f10980d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.403+0000 7f109e324640 1 -- 172.21.15.172:0/525605182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f109815e890 msgr2=0x7f10980d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.403+0000 7f109e324640 1 --2- 172.21.15.172:0/525605182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f109815e890 0x7f10980d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.403+0000 7f109e324640 1 -- 172.21.15.172:0/525605182 >> v1:172.21.15.177:6789/0 conn(0x7f10980cedd0 legacy=0x7f10981621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.403+0000 7f109e324640 1 -- 172.21.15.172:0/525605182 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10980cf320 con 0x7f10980d2830 2024-04-14T08:42:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.403+0000 7f109d322640 1 -- 172.21.15.172:0/525605182 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 27289616 0 0) 0x7f10980d3150 con 0x7f10980cedd0 2024-04-14T08:42:44.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.404+0000 7f109e324640 1 --2- 172.21.15.172:0/525605182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10980d2830 0x7f10980d2c10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f1094009e50 tx=0x7f109402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3367169d9c5fda0d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:44.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.404+0000 7f109d322640 1 -- 172.21.15.172:0/525605182 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109402cb90 con 0x7f10980d2830 2024-04-14T08:42:44.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.404+0000 7f109d322640 1 -- 172.21.15.172:0/525605182 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109402cd30 con 0x7f10980d2830 2024-04-14T08:42:44.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.404+0000 7f109d322640 1 -- 172.21.15.172:0/525605182 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094034610 con 0x7f10980d2830 2024-04-14T08:42:44.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.404+0000 7f10a0588640 1 -- 172.21.15.172:0/525605182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10980d2830 msgr2=0x7f10980d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.404+0000 7f10a0588640 1 --2- 172.21.15.172:0/525605182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10980d2830 0x7f10980d2c10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f1094009e50 tx=0x7f109402bff0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.404+0000 7f10a0588640 1 -- 172.21.15.172:0/525605182 shutdown_connections 2024-04-14T08:42:44.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.404+0000 7f10a0588640 1 --2- 172.21.15.172:0/525605182 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f109815e890 0x7f10980d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.404+0000 7f10a0588640 1 --2- 172.21.15.172:0/525605182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10980d2830 0x7f10980d2c10 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.404+0000 7f10a0588640 1 -- 172.21.15.172:0/525605182 >> 172.21.15.172:0/525605182 conn(0x7f109809d590 msgr2=0x7f10980d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:44.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.404+0000 7f10a0588640 1 -- 172.21.15.172:0/525605182 shutdown_connections 2024-04-14T08:42:44.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.404+0000 7f10a0588640 1 -- 172.21.15.172:0/525605182 wait complete. 2024-04-14T08:42:44.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.405+0000 7f10a0588640 1 Processor -- start 2024-04-14T08:42:44.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.406+0000 7f10a0588640 1 -- start start 2024-04-14T08:42:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.406+0000 7f10a0588640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10980cedd0 0x7f10981496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.406+0000 7f10a0588640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10980d2830 0x7f10980b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.406+0000 7f10a0588640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f109815e890 0x7f10980b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.406+0000 7f10a0588640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10980cfbc0 con 0x7f10980d2830 2024-04-14T08:42:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.406+0000 7f10a0588640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f10980cf8e0 con 0x7f109815e890 2024-04-14T08:42:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.406+0000 7f10a0588640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f10980d1840 con 0x7f10980cedd0 2024-04-14T08:42:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.406+0000 7f109db23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10980d2830 0x7f10980b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.406+0000 7f109db23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10980d2830 0x7f10980b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39858/0 (socket says 172.21.15.172:39858) 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.406+0000 7f109db23640 1 -- 172.21.15.172:0/2821824807 learned_addr learned my addr 172.21.15.172:0/2821824807 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.406+0000 7f109e324640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10980cedd0 0x7f10981496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.406+0000 7f109eb25640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f109815e890 0x7f10980b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.407+0000 7f109db23640 1 -- 172.21.15.172:0/2821824807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10980cedd0 msgr2=0x7f10981496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.407+0000 7f109db23640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10980cedd0 0x7f10981496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.407+0000 7f109db23640 1 -- 172.21.15.172:0/2821824807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f109815e890 msgr2=0x7f10980b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.407+0000 7f109db23640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f109815e890 0x7f10980b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.407+0000 7f109db23640 1 -- 172.21.15.172:0/2821824807 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10980b7910 con 0x7f10980d2830 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.407+0000 7f109db23640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10980d2830 0x7f10980b6e70 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f108800e980 tx=0x7f108800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.407+0000 7f108effd640 1 -- 172.21.15.172:0/2821824807 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088017070 con 0x7f10980d2830 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.407+0000 7f108effd640 1 -- 172.21.15.172:0/2821824807 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f108800ca00 con 0x7f10980d2830 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.407+0000 7f10a0588640 1 -- 172.21.15.172:0/2821824807 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10980b7bf0 con 0x7f10980d2830 2024-04-14T08:42:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.407+0000 7f108effd640 1 -- 172.21.15.172:0/2821824807 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108800cdd0 con 0x7f10980d2830 2024-04-14T08:42:44.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.407+0000 7f10a0588640 1 -- 172.21.15.172:0/2821824807 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10980b81b0 con 0x7f10980d2830 2024-04-14T08:42:44.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.408+0000 7f10a0588640 1 -- 172.21.15.172:0/2821824807 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1098068520 con 0x7f10980d2830 2024-04-14T08:42:44.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.409+0000 7f108effd640 1 -- 172.21.15.172:0/2821824807 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f108801a070 con 0x7f10980d2830 2024-04-14T08:42:44.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.409+0000 7f108effd640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f107c06c8f0 0x7f107c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.409+0000 7f108effd640 1 -- 172.21.15.172:0/2821824807 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f108808bf80 con 0x7f10980d2830 2024-04-14T08:42:44.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.409+0000 7f109e324640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f107c06c8f0 0x7f107c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.409+0000 7f109e324640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f107c06c8f0 0x7f107c06edb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1094035000 tx=0x7f1094030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:44.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.413+0000 7f108effd640 1 -- 172.21.15.172:0/2821824807 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1088059c60 con 0x7f10980d2830 2024-04-14T08:42:44.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.586+0000 7f10a0588640 1 -- 172.21.15.172:0/2821824807 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10980d5070 con 0x7f107c06c8f0 2024-04-14T08:42:44.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.592+0000 7f108effd640 1 -- 172.21.15.172:0/2821824807 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247762 (secure 0 0 0) 0x7f10980d5070 con 0x7f107c06c8f0 2024-04-14T08:42:44.595 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:44.596 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:42:44.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.595+0000 7f10a0588640 1 -- 172.21.15.172:0/2821824807 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f107c06c8f0 msgr2=0x7f107c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.595+0000 7f10a0588640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f107c06c8f0 0x7f107c06edb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1094035000 tx=0x7f1094030000 comp rx=0 tx=0).stop 2024-04-14T08:42:44.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.595+0000 7f10a0588640 1 -- 172.21.15.172:0/2821824807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10980d2830 msgr2=0x7f10980b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.595+0000 7f10a0588640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10980d2830 0x7f10980b6e70 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f108800e980 tx=0x7f108800ee40 comp rx=0 tx=0).stop 2024-04-14T08:42:44.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.595+0000 7f10a0588640 1 -- 172.21.15.172:0/2821824807 shutdown_connections 2024-04-14T08:42:44.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.595+0000 7f10a0588640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f107c06c8f0 0x7f107c06edb0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.595+0000 7f10a0588640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f109815e890 0x7f10980b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.595+0000 7f10a0588640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10980d2830 0x7f10980b6e70 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.595+0000 7f10a0588640 1 --2- 172.21.15.172:0/2821824807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10980cedd0 0x7f10981496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.595+0000 7f10a0588640 1 -- 172.21.15.172:0/2821824807 >> 172.21.15.172:0/2821824807 conn(0x7f109809d590 msgr2=0x7f10980b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:44.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.595+0000 7f10a0588640 1 -- 172.21.15.172:0/2821824807 shutdown_connections 2024-04-14T08:42:44.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.595+0000 7f10a0588640 1 -- 172.21.15.172:0/2821824807 wait complete. 2024-04-14T08:42:44.606 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-04-14T08:42:43.237918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261656,"kb_used_data":12204,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713064,"statfs":{"total":773094113280,"available":772826177536,"internally_reserved":0,"allocated":12496896,"data_stored":6427089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.355032"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.891452+0000","last_change":"2024-04-14T08:42:39.891452+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.891452+0000","last_undegraded":"2024-04-14T08:42:39.891452+0000","last_fullsized":"2024-04-14T08:42:39.891452+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.906122+0000","last_change":"2024-04-14T08:42:39.906122+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.906122+0000","last_undegraded":"2024-04-14T08:42:39.906122+0000","last_fullsized":"2024-04-14T08:42:39.906122+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904781+0000","last_change":"2024-04-14T08:42:39.904781+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904781+0000","last_undegraded":"2024-04-14T08:42:39.904781+0000","last_fullsized":"2024-04-14T08:42:39.904781+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.900277+0000","last_change":"2024-04-14T08:42:40.899261+0000","last_active":"2024-04-14T08:42:40.900277+0000","last_peered":"2024-04-14T08:42:40.900277+0000","last_clean":"2024-04-14T08:42:40.900277+0000","last_became_active":"2024-04-14T08:42:40.898970+0000","last_became_peered":"2024-04-14T08:42:40.898970+0000","last_unstale":"2024-04-14T08:42:40.900277+0000","last_undegraded":"2024-04-14T08:42:40.900277+0000","last_fullsized":"2024-04-14T08:42:40.900277+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:57.005587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.891722+0000","last_change":"2024-04-14T08:42:39.891722+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.891722+0000","last_undegraded":"2024-04-14T08:42:39.891722+0000","last_fullsized":"2024-04-14T08:42:39.891722+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904215+0000","last_change":"2024-04-14T08:42:39.904215+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904215+0000","last_undegraded":"2024-04-14T08:42:39.904215+0000","last_fullsized":"2024-04-14T08:42:39.904215+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.910912+0000","last_change":"2024-04-14T08:42:39.910912+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.910912+0000","last_undegraded":"2024-04-14T08:42:39.910912+0000","last_fullsized":"2024-04-14T08:42:39.910912+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.907664+0000","last_change":"2024-04-14T08:42:39.907664+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.907664+0000","last_undegraded":"2024-04-14T08:42:39.907664+0000","last_fullsized":"2024-04-14T08:42:39.907664+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.914262+0000","last_change":"2024-04-14T08:42:40.909455+0000","last_active":"2024-04-14T08:42:40.914262+0000","last_peered":"2024-04-14T08:42:40.914262+0000","last_clean":"2024-04-14T08:42:40.914262+0000","last_became_active":"2024-04-14T08:42:40.909156+0000","last_became_peered":"2024-04-14T08:42:40.909156+0000","last_unstale":"2024-04-14T08:42:40.914262+0000","last_undegraded":"2024-04-14T08:42:40.914262+0000","last_fullsized":"2024-04-14T08:42:40.914262+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:50:25.331351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902706+0000","last_change":"2024-04-14T08:42:39.902706+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902706+0000","last_undegraded":"2024-04-14T08:42:39.902706+0000","last_fullsized":"2024-04-14T08:42:39.902706+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.892580+0000","last_change":"2024-04-14T08:42:39.892580+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.892580+0000","last_undegraded":"2024-04-14T08:42:39.892580+0000","last_fullsized":"2024-04-14T08:42:39.892580+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904944+0000","last_change":"2024-04-14T08:42:39.904944+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904944+0000","last_undegraded":"2024-04-14T08:42:39.904944+0000","last_fullsized":"2024-04-14T08:42:39.904944+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903252+0000","last_change":"2024-04-14T08:42:39.903252+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903252+0000","last_undegraded":"2024-04-14T08:42:39.903252+0000","last_fullsized":"2024-04-14T08:42:39.903252+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.909607+0000","last_change":"2024-04-14T08:42:39.909607+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.909607+0000","last_undegraded":"2024-04-14T08:42:39.909607+0000","last_fullsized":"2024-04-14T08:42:39.909607+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.893081+0000","last_change":"2024-04-14T08:42:39.893081+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.893081+0000","last_undegraded":"2024-04-14T08:42:39.893081+0000","last_fullsized":"2024-04-14T08:42:39.893081+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903738+0000","last_change":"2024-04-14T08:42:39.903738+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903738+0000","last_undegraded":"2024-04-14T08:42:39.903738+0000","last_fullsized":"2024-04-14T08:42:39.903738+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903068+0000","last_change":"2024-04-14T08:42:39.903068+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903068+0000","last_undegraded":"2024-04-14T08:42:39.903068+0000","last_fullsized":"2024-04-14T08:42:39.903068+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.905703+0000","last_change":"2024-04-14T08:42:39.905703+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.905703+0000","last_undegraded":"2024-04-14T08:42:39.905703+0000","last_fullsized":"2024-04-14T08:42:39.905703+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.896903+0000","last_change":"2024-04-14T08:42:39.896903+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.896903+0000","last_undegraded":"2024-04-14T08:42:39.896903+0000","last_fullsized":"2024-04-14T08:42:39.896903+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.908810+0000","last_change":"2024-04-14T08:42:39.908810+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.908810+0000","last_undegraded":"2024-04-14T08:42:39.908810+0000","last_fullsized":"2024-04-14T08:42:39.908810+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.913413+0000","last_change":"2024-04-14T08:42:40.906675+0000","last_active":"2024-04-14T08:42:40.913413+0000","last_peered":"2024-04-14T08:42:40.913413+0000","last_clean":"2024-04-14T08:42:40.913413+0000","last_became_active":"2024-04-14T08:42:40.904423+0000","last_became_peered":"2024-04-14T08:42:40.904423+0000","last_unstale":"2024-04-14T08:42:40.913413+0000","last_undegraded":"2024-04-14T08:42:40.913413+0000","last_fullsized":"2024-04-14T08:42:40.913413+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:42:53.101473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.892991+0000","last_change":"2024-04-14T08:42:39.892991+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.892991+0000","last_undegraded":"2024-04-14T08:42:39.892991+0000","last_fullsized":"2024-04-14T08:42:39.892991+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.921770+0000","last_change":"2024-04-14T08:42:40.919066+0000","last_active":"2024-04-14T08:42:40.921770+0000","last_peered":"2024-04-14T08:42:40.921770+0000","last_clean":"2024-04-14T08:42:40.921770+0000","last_became_active":"2024-04-14T08:42:40.918630+0000","last_became_peered":"2024-04-14T08:42:40.918630+0000","last_unstale":"2024-04-14T08:42:40.921770+0000","last_undegraded":"2024-04-14T08:42:40.921770+0000","last_fullsized":"2024-04-14T08:42:40.921770+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:53:03.886099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.899098+0000","last_change":"2024-04-14T08:42:39.899098+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.899098+0000","last_undegraded":"2024-04-14T08:42:39.899098+0000","last_fullsized":"2024-04-14T08:42:39.899098+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900930+0000","last_change":"2024-04-14T08:42:39.900930+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900930+0000","last_undegraded":"2024-04-14T08:42:39.900930+0000","last_fullsized":"2024-04-14T08:42:39.900930+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.906240+0000","last_change":"2024-04-14T08:42:39.906240+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.906240+0000","last_undegraded":"2024-04-14T08:42:39.906240+0000","last_fullsized":"2024-04-14T08:42:39.906240+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.896498+0000","last_change":"2024-04-14T08:42:39.896498+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.896498+0000","last_undegraded":"2024-04-14T08:42:39.896498+0000","last_fullsized":"2024-04-14T08:42:39.896498+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904518+0000","last_change":"2024-04-14T08:42:39.904518+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904518+0000","last_undegraded":"2024-04-14T08:42:39.904518+0000","last_fullsized":"2024-04-14T08:42:39.904518+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.905764+0000","last_change":"2024-04-14T08:42:39.905764+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.905764+0000","last_undegraded":"2024-04-14T08:42:39.905764+0000","last_fullsized":"2024-04-14T08:42:39.905764+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.909220+0000","last_change":"2024-04-14T08:42:39.909220+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.909220+0000","last_undegraded":"2024-04-14T08:42:39.909220+0000","last_fullsized":"2024-04-14T08:42:39.909220+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.899058+0000","last_change":"2024-04-14T08:42:39.899058+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.899058+0000","last_undegraded":"2024-04-14T08:42:39.899058+0000","last_fullsized":"2024-04-14T08:42:39.899058+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.898579+0000","last_change":"2024-04-14T08:42:39.898579+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.898579+0000","last_undegraded":"2024-04-14T08:42:39.898579+0000","last_fullsized":"2024-04-14T08:42:39.898579+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902637+0000","last_change":"2024-04-14T08:42:39.902637+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902637+0000","last_undegraded":"2024-04-14T08:42:39.902637+0000","last_fullsized":"2024-04-14T08:42:39.902637+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.906183+0000","last_change":"2024-04-14T08:42:39.902925+0000","last_active":"2024-04-14T08:42:39.906183+0000","last_peered":"2024-04-14T08:42:39.906183+0000","last_clean":"2024-04-14T08:42:39.906183+0000","last_became_active":"2024-04-14T08:42:39.902599+0000","last_became_peered":"2024-04-14T08:42:39.902599+0000","last_unstale":"2024-04-14T08:42:39.906183+0000","last_undegraded":"2024-04-14T08:42:39.906183+0000","last_fullsized":"2024-04-14T08:42:39.906183+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:59.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903562+0000","last_change":"2024-04-14T08:42:39.903562+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903562+0000","last_undegraded":"2024-04-14T08:42:39.903562+0000","last_fullsized":"2024-04-14T08:42:39.903562+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.904995+0000","last_change":"2024-04-14T08:42:39.898146+0000","last_active":"2024-04-14T08:42:39.904995+0000","last_peered":"2024-04-14T08:42:39.904995+0000","last_clean":"2024-04-14T08:42:39.904995+0000","last_became_active":"2024-04-14T08:42:39.897857+0000","last_became_peered":"2024-04-14T08:42:39.897857+0000","last_unstale":"2024-04-14T08:42:39.904995+0000","last_undegraded":"2024-04-14T08:42:39.904995+0000","last_fullsized":"2024-04-14T08:42:39.904995+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:45:17.425830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900508+0000","last_change":"2024-04-14T08:42:39.900508+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900508+0000","last_undegraded":"2024-04-14T08:42:39.900508+0000","last_fullsized":"2024-04-14T08:42:39.900508+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.920415+0000","last_change":"2024-04-14T08:42:39.903458+0000","last_active":"2024-04-14T08:42:40.920415+0000","last_peered":"2024-04-14T08:42:40.920415+0000","last_clean":"2024-04-14T08:42:40.920415+0000","last_became_active":"2024-04-14T08:42:39.903032+0000","last_became_peered":"2024-04-14T08:42:39.903032+0000","last_unstale":"2024-04-14T08:42:40.920415+0000","last_undegraded":"2024-04-14T08:42:40.920415+0000","last_fullsized":"2024-04-14T08:42:40.920415+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:43.408088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.908622+0000","last_change":"2024-04-14T08:42:40.904383+0000","last_active":"2024-04-14T08:42:40.908622+0000","last_peered":"2024-04-14T08:42:40.908622+0000","last_clean":"2024-04-14T08:42:40.908622+0000","last_became_active":"2024-04-14T08:42:40.904099+0000","last_became_peered":"2024-04-14T08:42:40.904099+0000","last_unstale":"2024-04-14T08:42:40.908622+0000","last_undegraded":"2024-04-14T08:42:40.908622+0000","last_fullsized":"2024-04-14T08:42:40.908622+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:37:01.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919569+0000","last_change":"2024-04-14T08:42:39.914891+0000","last_active":"2024-04-14T08:42:39.919569+0000","last_peered":"2024-04-14T08:42:39.919569+0000","last_clean":"2024-04-14T08:42:39.919569+0000","last_became_active":"2024-04-14T08:42:39.914617+0000","last_became_peered":"2024-04-14T08:42:39.914617+0000","last_unstale":"2024-04-14T08:42:39.919569+0000","last_undegraded":"2024-04-14T08:42:39.919569+0000","last_fullsized":"2024-04-14T08:42:39.919569+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:21:27.947470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902219+0000","last_change":"2024-04-14T08:42:39.902219+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902219+0000","last_undegraded":"2024-04-14T08:42:39.902219+0000","last_fullsized":"2024-04-14T08:42:39.902219+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.903615+0000","last_change":"2024-04-14T08:42:39.898512+0000","last_active":"2024-04-14T08:42:39.903615+0000","last_peered":"2024-04-14T08:42:39.903615+0000","last_clean":"2024-04-14T08:42:39.903615+0000","last_became_active":"2024-04-14T08:42:39.898183+0000","last_became_peered":"2024-04-14T08:42:39.898183+0000","last_unstale":"2024-04-14T08:42:39.903615+0000","last_undegraded":"2024-04-14T08:42:39.903615+0000","last_fullsized":"2024-04-14T08:42:39.903615+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:35:14.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.891105+0000","last_change":"2024-04-14T08:42:39.891105+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.891105+0000","last_undegraded":"2024-04-14T08:42:39.891105+0000","last_fullsized":"2024-04-14T08:42:39.891105+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919468+0000","last_change":"2024-04-14T08:42:39.915147+0000","last_active":"2024-04-14T08:42:39.919468+0000","last_peered":"2024-04-14T08:42:39.919468+0000","last_clean":"2024-04-14T08:42:39.919468+0000","last_became_active":"2024-04-14T08:42:39.914929+0000","last_became_peered":"2024-04-14T08:42:39.914929+0000","last_unstale":"2024-04-14T08:42:39.919468+0000","last_undegraded":"2024-04-14T08:42:39.919468+0000","last_fullsized":"2024-04-14T08:42:39.919468+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:54:03.062201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.908821+0000","last_change":"2024-04-14T08:42:39.908821+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.908821+0000","last_undegraded":"2024-04-14T08:42:39.908821+0000","last_fullsized":"2024-04-14T08:42:39.908821+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.908946+0000","last_change":"2024-04-14T08:42:39.904121+0000","last_active":"2024-04-14T08:42:39.908946+0000","last_peered":"2024-04-14T08:42:39.908946+0000","last_clean":"2024-04-14T08:42:39.908946+0000","last_became_active":"2024-04-14T08:42:39.903879+0000","last_became_peered":"2024-04-14T08:42:39.903879+0000","last_unstale":"2024-04-14T08:42:39.908946+0000","last_undegraded":"2024-04-14T08:42:39.908946+0000","last_fullsized":"2024-04-14T08:42:39.908946+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:25:18.927294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902699+0000","last_change":"2024-04-14T08:42:39.902699+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902699+0000","last_undegraded":"2024-04-14T08:42:39.902699+0000","last_fullsized":"2024-04-14T08:42:39.902699+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.905886+0000","last_change":"2024-04-14T08:42:39.900795+0000","last_active":"2024-04-14T08:42:39.905886+0000","last_peered":"2024-04-14T08:42:39.905886+0000","last_clean":"2024-04-14T08:42:39.905886+0000","last_became_active":"2024-04-14T08:42:39.900345+0000","last_became_peered":"2024-04-14T08:42:39.900345+0000","last_unstale":"2024-04-14T08:42:39.905886+0000","last_undegraded":"2024-04-14T08:42:39.905886+0000","last_fullsized":"2024-04-14T08:42:39.905886+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:38:16.668417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.896257+0000","last_change":"2024-04-14T08:42:39.896257+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.896257+0000","last_undegraded":"2024-04-14T08:42:39.896257+0000","last_fullsized":"2024-04-14T08:42:39.896257+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911424+0000","last_change":"2024-04-14T08:42:39.908268+0000","last_active":"2024-04-14T08:42:39.911424+0000","last_peered":"2024-04-14T08:42:39.911424+0000","last_clean":"2024-04-14T08:42:39.911424+0000","last_became_active":"2024-04-14T08:42:39.907971+0000","last_became_peered":"2024-04-14T08:42:39.907971+0000","last_unstale":"2024-04-14T08:42:39.911424+0000","last_undegraded":"2024-04-14T08:42:39.911424+0000","last_fullsized":"2024-04-14T08:42:39.911424+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:07:34.836827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900647+0000","last_change":"2024-04-14T08:42:39.900647+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900647+0000","last_undegraded":"2024-04-14T08:42:39.900647+0000","last_fullsized":"2024-04-14T08:42:39.900647+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.916586+0000","last_change":"2024-04-14T08:42:39.911711+0000","last_active":"2024-04-14T08:42:39.916586+0000","last_peered":"2024-04-14T08:42:39.916586+0000","last_clean":"2024-04-14T08:42:39.916586+0000","last_became_active":"2024-04-14T08:42:39.911408+0000","last_became_peered":"2024-04-14T08:42:39.911408+0000","last_unstale":"2024-04-14T08:42:39.916586+0000","last_undegraded":"2024-04-14T08:42:39.916586+0000","last_fullsized":"2024-04-14T08:42:39.916586+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:59.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.905288+0000","last_change":"2024-04-14T08:42:39.905288+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.905288+0000","last_undegraded":"2024-04-14T08:42:39.905288+0000","last_fullsized":"2024-04-14T08:42:39.905288+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911434+0000","last_change":"2024-04-14T08:42:39.906942+0000","last_active":"2024-04-14T08:42:39.911434+0000","last_peered":"2024-04-14T08:42:39.911434+0000","last_clean":"2024-04-14T08:42:39.911434+0000","last_became_active":"2024-04-14T08:42:39.906650+0000","last_became_peered":"2024-04-14T08:42:39.906650+0000","last_unstale":"2024-04-14T08:42:39.911434+0000","last_undegraded":"2024-04-14T08:42:39.911434+0000","last_fullsized":"2024-04-14T08:42:39.911434+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:16:05.440009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900522+0000","last_change":"2024-04-14T08:42:39.900522+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900522+0000","last_undegraded":"2024-04-14T08:42:39.900522+0000","last_fullsized":"2024-04-14T08:42:39.900522+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.922215+0000","last_change":"2024-04-14T08:42:39.917311+0000","last_active":"2024-04-14T08:42:39.922215+0000","last_peered":"2024-04-14T08:42:39.922215+0000","last_clean":"2024-04-14T08:42:39.922215+0000","last_became_active":"2024-04-14T08:42:39.917001+0000","last_became_peered":"2024-04-14T08:42:39.917001+0000","last_unstale":"2024-04-14T08:42:39.922215+0000","last_undegraded":"2024-04-14T08:42:39.922215+0000","last_fullsized":"2024-04-14T08:42:39.922215+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:25:26.756136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.896919+0000","last_change":"2024-04-14T08:42:39.896919+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.896919+0000","last_undegraded":"2024-04-14T08:42:39.896919+0000","last_fullsized":"2024-04-14T08:42:39.896919+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919614+0000","last_change":"2024-04-14T08:42:39.914644+0000","last_active":"2024-04-14T08:42:39.919614+0000","last_peered":"2024-04-14T08:42:39.919614+0000","last_clean":"2024-04-14T08:42:39.919614+0000","last_became_active":"2024-04-14T08:42:39.914350+0000","last_became_peered":"2024-04-14T08:42:39.914350+0000","last_unstale":"2024-04-14T08:42:39.919614+0000","last_undegraded":"2024-04-14T08:42:39.919614+0000","last_fullsized":"2024-04-14T08:42:39.919614+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:35:14.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902158+0000","last_change":"2024-04-14T08:42:39.902158+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902158+0000","last_undegraded":"2024-04-14T08:42:39.902158+0000","last_fullsized":"2024-04-14T08:42:39.902158+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919597+0000","last_change":"2024-04-14T08:42:39.914644+0000","last_active":"2024-04-14T08:42:39.919597+0000","last_peered":"2024-04-14T08:42:39.919597+0000","last_clean":"2024-04-14T08:42:39.919597+0000","last_became_active":"2024-04-14T08:42:39.914387+0000","last_became_peered":"2024-04-14T08:42:39.914387+0000","last_unstale":"2024-04-14T08:42:39.919597+0000","last_undegraded":"2024-04-14T08:42:39.919597+0000","last_fullsized":"2024-04-14T08:42:39.919597+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:59.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.905085+0000","last_change":"2024-04-14T08:42:39.905085+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.905085+0000","last_undegraded":"2024-04-14T08:42:39.905085+0000","last_fullsized":"2024-04-14T08:42:39.905085+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.922210+0000","last_change":"2024-04-14T08:42:39.917235+0000","last_active":"2024-04-14T08:42:39.922210+0000","last_peered":"2024-04-14T08:42:39.922210+0000","last_clean":"2024-04-14T08:42:39.922210+0000","last_became_active":"2024-04-14T08:42:39.916937+0000","last_became_peered":"2024-04-14T08:42:39.916937+0000","last_unstale":"2024-04-14T08:42:39.922210+0000","last_undegraded":"2024-04-14T08:42:39.922210+0000","last_fullsized":"2024-04-14T08:42:39.922210+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:07:34.836827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.899650+0000","last_change":"2024-04-14T08:42:39.899650+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.899650+0000","last_undegraded":"2024-04-14T08:42:39.899650+0000","last_fullsized":"2024-04-14T08:42:39.899650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911878+0000","last_change":"2024-04-14T08:42:39.907928+0000","last_active":"2024-04-14T08:42:39.911878+0000","last_peered":"2024-04-14T08:42:39.911878+0000","last_clean":"2024-04-14T08:42:39.911878+0000","last_became_active":"2024-04-14T08:42:39.907662+0000","last_became_peered":"2024-04-14T08:42:39.907662+0000","last_unstale":"2024-04-14T08:42:39.911878+0000","last_undegraded":"2024-04-14T08:42:39.911878+0000","last_fullsized":"2024-04-14T08:42:39.911878+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:42:44.082970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.892168+0000","last_change":"2024-04-14T08:42:39.892168+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.892168+0000","last_undegraded":"2024-04-14T08:42:39.892168+0000","last_fullsized":"2024-04-14T08:42:39.892168+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.910329+0000","last_change":"2024-04-14T08:42:39.903548+0000","last_active":"2024-04-14T08:42:39.910329+0000","last_peered":"2024-04-14T08:42:39.910329+0000","last_clean":"2024-04-14T08:42:39.910329+0000","last_became_active":"2024-04-14T08:42:39.903272+0000","last_became_peered":"2024-04-14T08:42:39.903272+0000","last_unstale":"2024-04-14T08:42:39.910329+0000","last_undegraded":"2024-04-14T08:42:39.910329+0000","last_fullsized":"2024-04-14T08:42:39.910329+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:31:53.521063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904059+0000","last_change":"2024-04-14T08:42:39.904059+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904059+0000","last_undegraded":"2024-04-14T08:42:39.904059+0000","last_fullsized":"2024-04-14T08:42:39.904059+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.922186+0000","last_change":"2024-04-14T08:42:39.907861+0000","last_active":"2024-04-14T08:42:40.922186+0000","last_peered":"2024-04-14T08:42:40.922186+0000","last_clean":"2024-04-14T08:42:40.922186+0000","last_became_active":"2024-04-14T08:42:39.907144+0000","last_became_peered":"2024-04-14T08:42:39.907144+0000","last_unstale":"2024-04-14T08:42:40.922186+0000","last_undegraded":"2024-04-14T08:42:40.922186+0000","last_fullsized":"2024-04-14T08:42:40.922186+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:50.027906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.900070+0000","last_change":"2024-04-14T08:42:39.900070+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.900070+0000","last_undegraded":"2024-04-14T08:42:39.900070+0000","last_fullsized":"2024-04-14T08:42:39.900070+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.895697+0000","last_change":"2024-04-14T08:42:39.900873+0000","last_active":"2024-04-14T08:42:40.895697+0000","last_peered":"2024-04-14T08:42:40.895697+0000","last_clean":"2024-04-14T08:42:40.895697+0000","last_became_active":"2024-04-14T08:42:39.900603+0000","last_became_peered":"2024-04-14T08:42:39.900603+0000","last_unstale":"2024-04-14T08:42:40.895697+0000","last_undegraded":"2024-04-14T08:42:40.895697+0000","last_fullsized":"2024-04-14T08:42:40.895697+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:29.063677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.902777+0000","last_change":"2024-04-14T08:42:39.902777+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.902777+0000","last_undegraded":"2024-04-14T08:42:39.902777+0000","last_fullsized":"2024-04-14T08:42:39.902777+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911312+0000","last_change":"2024-04-14T08:42:39.904952+0000","last_active":"2024-04-14T08:42:39.911312+0000","last_peered":"2024-04-14T08:42:39.911312+0000","last_clean":"2024-04-14T08:42:39.911312+0000","last_became_active":"2024-04-14T08:42:39.904677+0000","last_became_peered":"2024-04-14T08:42:39.904677+0000","last_unstale":"2024-04-14T08:42:39.911312+0000","last_undegraded":"2024-04-14T08:42:39.911312+0000","last_fullsized":"2024-04-14T08:42:39.911312+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:45:07.221511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904226+0000","last_change":"2024-04-14T08:42:39.904226+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904226+0000","last_undegraded":"2024-04-14T08:42:39.904226+0000","last_fullsized":"2024-04-14T08:42:39.904226+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.894138+0000","last_change":"2024-04-14T08:42:39.902053+0000","last_active":"2024-04-14T08:42:40.894138+0000","last_peered":"2024-04-14T08:42:40.894138+0000","last_clean":"2024-04-14T08:42:40.894138+0000","last_became_active":"2024-04-14T08:42:39.901709+0000","last_became_peered":"2024-04-14T08:42:39.901709+0000","last_unstale":"2024-04-14T08:42:40.894138+0000","last_undegraded":"2024-04-14T08:42:40.894138+0000","last_fullsized":"2024-04-14T08:42:40.894138+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:58:25.264866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.908418+0000","last_change":"2024-04-14T08:42:39.908418+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.908418+0000","last_undegraded":"2024-04-14T08:42:39.908418+0000","last_fullsized":"2024-04-14T08:42:39.908418+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.922233+0000","last_change":"2024-04-14T08:42:39.917517+0000","last_active":"2024-04-14T08:42:39.922233+0000","last_peered":"2024-04-14T08:42:39.922233+0000","last_clean":"2024-04-14T08:42:39.922233+0000","last_became_active":"2024-04-14T08:42:39.917246+0000","last_became_peered":"2024-04-14T08:42:39.917246+0000","last_unstale":"2024-04-14T08:42:39.922233+0000","last_undegraded":"2024-04-14T08:42:39.922233+0000","last_fullsized":"2024-04-14T08:42:39.922233+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:41.656802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.899442+0000","last_change":"2024-04-14T08:42:39.899442+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.899442+0000","last_undegraded":"2024-04-14T08:42:39.899442+0000","last_fullsized":"2024-04-14T08:42:39.899442+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.893438+0000","last_change":"2024-04-14T08:42:39.903314+0000","last_active":"2024-04-14T08:42:40.893438+0000","last_peered":"2024-04-14T08:42:40.893438+0000","last_clean":"2024-04-14T08:42:40.893438+0000","last_became_active":"2024-04-14T08:42:39.903044+0000","last_became_peered":"2024-04-14T08:42:39.903044+0000","last_unstale":"2024-04-14T08:42:40.893438+0000","last_undegraded":"2024-04-14T08:42:40.893438+0000","last_fullsized":"2024-04-14T08:42:40.893438+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:32:34.297541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903140+0000","last_change":"2024-04-14T08:42:39.903140+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903140+0000","last_undegraded":"2024-04-14T08:42:39.903140+0000","last_fullsized":"2024-04-14T08:42:39.903140+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.905903+0000","last_change":"2024-04-14T08:42:39.899765+0000","last_active":"2024-04-14T08:42:39.905903+0000","last_peered":"2024-04-14T08:42:39.905903+0000","last_clean":"2024-04-14T08:42:39.905903+0000","last_became_active":"2024-04-14T08:42:39.898985+0000","last_became_peered":"2024-04-14T08:42:39.898985+0000","last_unstale":"2024-04-14T08:42:39.905903+0000","last_undegraded":"2024-04-14T08:42:39.905903+0000","last_fullsized":"2024-04-14T08:42:39.905903+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:00:31.753726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.901623+0000","last_change":"2024-04-14T08:42:39.901623+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.901623+0000","last_undegraded":"2024-04-14T08:42:39.901623+0000","last_fullsized":"2024-04-14T08:42:39.901623+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911433+0000","last_change":"2024-04-14T08:42:39.908137+0000","last_active":"2024-04-14T08:42:39.911433+0000","last_peered":"2024-04-14T08:42:39.911433+0000","last_clean":"2024-04-14T08:42:39.911433+0000","last_became_active":"2024-04-14T08:42:39.907857+0000","last_became_peered":"2024-04-14T08:42:39.907857+0000","last_unstale":"2024-04-14T08:42:39.911433+0000","last_undegraded":"2024-04-14T08:42:39.911433+0000","last_fullsized":"2024-04-14T08:42:39.911433+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:42:20.871047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.910001+0000","last_change":"2024-04-14T08:42:40.904731+0000","last_active":"2024-04-14T08:42:40.910001+0000","last_peered":"2024-04-14T08:42:40.910001+0000","last_clean":"2024-04-14T08:42:40.910001+0000","last_became_active":"2024-04-14T08:42:40.904169+0000","last_became_peered":"2024-04-14T08:42:40.904169+0000","last_unstale":"2024-04-14T08:42:40.910001+0000","last_undegraded":"2024-04-14T08:42:40.910001+0000","last_fullsized":"2024-04-14T08:42:40.910001+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:40.822681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911401+0000","last_change":"2024-04-14T08:42:39.907401+0000","last_active":"2024-04-14T08:42:39.911401+0000","last_peered":"2024-04-14T08:42:39.911401+0000","last_clean":"2024-04-14T08:42:39.911401+0000","last_became_active":"2024-04-14T08:42:39.907124+0000","last_became_peered":"2024-04-14T08:42:39.907124+0000","last_unstale":"2024-04-14T08:42:39.911401+0000","last_undegraded":"2024-04-14T08:42:39.911401+0000","last_fullsized":"2024-04-14T08:42:39.911401+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:35:14.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.904317+0000","last_change":"2024-04-14T08:42:39.904317+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.904317+0000","last_undegraded":"2024-04-14T08:42:39.904317+0000","last_fullsized":"2024-04-14T08:42:39.904317+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.911743+0000","last_change":"2024-04-14T08:42:39.901863+0000","last_active":"2024-04-14T08:42:39.911743+0000","last_peered":"2024-04-14T08:42:39.911743+0000","last_clean":"2024-04-14T08:42:39.911743+0000","last_became_active":"2024-04-14T08:42:39.901612+0000","last_became_peered":"2024-04-14T08:42:39.901612+0000","last_unstale":"2024-04-14T08:42:39.911743+0000","last_undegraded":"2024-04-14T08:42:39.911743+0000","last_fullsized":"2024-04-14T08:42:39.911743+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:52:47.647486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.901878+0000","last_change":"2024-04-14T08:42:39.901878+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.901878+0000","last_undegraded":"2024-04-14T08:42:39.901878+0000","last_fullsized":"2024-04-14T08:42:39.901878+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.903592+0000","last_change":"2024-04-14T08:42:39.898079+0000","last_active":"2024-04-14T08:42:39.903592+0000","last_peered":"2024-04-14T08:42:39.903592+0000","last_clean":"2024-04-14T08:42:39.903592+0000","last_became_active":"2024-04-14T08:42:39.897776+0000","last_became_peered":"2024-04-14T08:42:39.897776+0000","last_unstale":"2024-04-14T08:42:39.903592+0000","last_undegraded":"2024-04-14T08:42:39.903592+0000","last_fullsized":"2024-04-14T08:42:39.903592+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:02:59.064412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.903815+0000","last_change":"2024-04-14T08:42:39.903815+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.903815+0000","last_undegraded":"2024-04-14T08:42:39.903815+0000","last_fullsized":"2024-04-14T08:42:39.903815+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.908711+0000","last_change":"2024-04-14T08:42:39.907024+0000","last_active":"2024-04-14T08:42:39.908711+0000","last_peered":"2024-04-14T08:42:39.908711+0000","last_clean":"2024-04-14T08:42:39.908711+0000","last_became_active":"2024-04-14T08:42:39.906749+0000","last_became_peered":"2024-04-14T08:42:39.906749+0000","last_unstale":"2024-04-14T08:42:39.908711+0000","last_undegraded":"2024-04-14T08:42:39.908711+0000","last_fullsized":"2024-04-14T08:42:39.908711+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:42:20.871047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.914252+0000","last_change":"2024-04-14T08:42:40.909631+0000","last_active":"2024-04-14T08:42:40.914252+0000","last_peered":"2024-04-14T08:42:40.914252+0000","last_clean":"2024-04-14T08:42:40.914252+0000","last_became_active":"2024-04-14T08:42:40.909358+0000","last_became_peered":"2024-04-14T08:42:40.909358+0000","last_unstale":"2024-04-14T08:42:40.914252+0000","last_undegraded":"2024-04-14T08:42:40.914252+0000","last_fullsized":"2024-04-14T08:42:40.914252+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:56.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.900604+0000","last_change":"2024-04-14T08:42:39.899535+0000","last_active":"2024-04-14T08:42:39.900604+0000","last_peered":"2024-04-14T08:42:39.900604+0000","last_clean":"2024-04-14T08:42:39.900604+0000","last_became_active":"2024-04-14T08:42:39.899245+0000","last_became_peered":"2024-04-14T08:42:39.899245+0000","last_unstale":"2024-04-14T08:42:39.900604+0000","last_undegraded":"2024-04-14T08:42:39.900604+0000","last_fullsized":"2024-04-14T08:42:39.900604+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:55:11.795053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-14T08:42:39.901718+0000","last_change":"2024-04-14T08:42:39.901718+0000","last_active":"2024-04-14T08:42:39.880431+0000","last_peered":"2024-04-14T08:42:39.880431+0000","last_clean":"2024-04-14T08:42:39.880431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-14T08:42:39.901718+0000","last_undegraded":"2024-04-14T08:42:39.901718+0000","last_fullsized":"2024-04-14T08:42:39.901718+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.893537+0000","last_change":"2024-04-14T08:42:39.903168+0000","last_active":"2024-04-14T08:42:40.893537+0000","last_peered":"2024-04-14T08:42:40.893537+0000","last_clean":"2024-04-14T08:42:40.893537+0000","last_became_active":"2024-04-14T08:42:39.902874+0000","last_became_peered":"2024-04-14T08:42:39.902874+0000","last_unstale":"2024-04-14T08:42:40.893537+0000","last_undegraded":"2024-04-14T08:42:40.893537+0000","last_fullsized":"2024-04-14T08:42:40.893537+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:34:54.764802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.916751+0000","last_change":"2024-04-14T08:42:40.913643+0000","last_active":"2024-04-14T08:42:40.916751+0000","last_peered":"2024-04-14T08:42:40.916751+0000","last_clean":"2024-04-14T08:42:40.916751+0000","last_became_active":"2024-04-14T08:42:40.912975+0000","last_became_peered":"2024-04-14T08:42:40.912975+0000","last_unstale":"2024-04-14T08:42:40.916751+0000","last_undegraded":"2024-04-14T08:42:40.916751+0000","last_fullsized":"2024-04-14T08:42:40.916751+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:28.290311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-04-14T08:42:39.919652+0000","last_change":"2024-04-14T08:42:39.915077+0000","last_active":"2024-04-14T08:42:39.919652+0000","last_peered":"2024-04-14T08:42:39.919652+0000","last_clean":"2024-04-14T08:42:39.919652+0000","last_became_active":"2024-04-14T08:42:39.914803+0000","last_became_peered":"2024-04-14T08:42:39.914803+0000","last_unstale":"2024-04-14T08:42:39.919652+0000","last_undegraded":"2024-04-14T08:42:39.919652+0000","last_fullsized":"2024-04-14T08:42:39.919652+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:42:20.871047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":362,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.893578+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:42:40.893578+0000","last_peered":"2024-04-14T08:42:40.893578+0000","last_clean":"2024-04-14T08:42:40.893578+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:42:40.893578+0000","last_undegraded":"2024-04-14T08:42:40.893578+0000","last_fullsized":"2024-04-14T08:42:40.893578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2015232,"data_stored":1250329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1409024,"data_stored":653294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1409024,"data_stored":653294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1409024,"data_stored":653294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574971,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":659961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1421312,"data_stored":659961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1409024,"data_stored":653294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2002944,"data_stored":1243662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:42:44.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3f is not active+clean 2024-04-14T08:42:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.891452+0000', 'last_change': '2024-04-14T08:42:39.891452+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.891452+0000', 'last_undegraded': '2024-04-14T08:42:39.891452+0000', 'last_fullsized': '2024-04-14T08:42:39.891452+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:42:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3e is not active+clean 2024-04-14T08:42:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.906122+0000', 'last_change': '2024-04-14T08:42:39.906122+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.906122+0000', 'last_undegraded': '2024-04-14T08:42:39.906122+0000', 'last_fullsized': '2024-04-14T08:42:39.906122+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:42:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3d is not active+clean 2024-04-14T08:42:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.904781+0000', 'last_change': '2024-04-14T08:42:39.904781+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.904781+0000', 'last_undegraded': '2024-04-14T08:42:39.904781+0000', 'last_fullsized': '2024-04-14T08:42:39.904781+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:42:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3b is not active+clean 2024-04-14T08:42:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.891722+0000', 'last_change': '2024-04-14T08:42:39.891722+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.891722+0000', 'last_undegraded': '2024-04-14T08:42:39.891722+0000', 'last_fullsized': '2024-04-14T08:42:39.891722+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:42:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3a is not active+clean 2024-04-14T08:42:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.904215+0000', 'last_change': '2024-04-14T08:42:39.904215+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.904215+0000', 'last_undegraded': '2024-04-14T08:42:39.904215+0000', 'last_fullsized': '2024-04-14T08:42:39.904215+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:42:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.39 is not active+clean 2024-04-14T08:42:44.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.910912+0000', 'last_change': '2024-04-14T08:42:39.910912+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.910912+0000', 'last_undegraded': '2024-04-14T08:42:39.910912+0000', 'last_fullsized': '2024-04-14T08:42:39.910912+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:42:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.38 is not active+clean 2024-04-14T08:42:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.907664+0000', 'last_change': '2024-04-14T08:42:39.907664+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.907664+0000', 'last_undegraded': '2024-04-14T08:42:39.907664+0000', 'last_fullsized': '2024-04-14T08:42:39.907664+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:42:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.36 is not active+clean 2024-04-14T08:42:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.902706+0000', 'last_change': '2024-04-14T08:42:39.902706+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.902706+0000', 'last_undegraded': '2024-04-14T08:42:39.902706+0000', 'last_fullsized': '2024-04-14T08:42:39.902706+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:42:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.35 is not active+clean 2024-04-14T08:42:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.892580+0000', 'last_change': '2024-04-14T08:42:39.892580+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.892580+0000', 'last_undegraded': '2024-04-14T08:42:39.892580+0000', 'last_fullsized': '2024-04-14T08:42:39.892580+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:42:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.34 is not active+clean 2024-04-14T08:42:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.904944+0000', 'last_change': '2024-04-14T08:42:39.904944+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.904944+0000', 'last_undegraded': '2024-04-14T08:42:39.904944+0000', 'last_fullsized': '2024-04-14T08:42:39.904944+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:42:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.33 is not active+clean 2024-04-14T08:42:44.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.903252+0000', 'last_change': '2024-04-14T08:42:39.903252+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.903252+0000', 'last_undegraded': '2024-04-14T08:42:39.903252+0000', 'last_fullsized': '2024-04-14T08:42:39.903252+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:42:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.32 is not active+clean 2024-04-14T08:42:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.909607+0000', 'last_change': '2024-04-14T08:42:39.909607+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.909607+0000', 'last_undegraded': '2024-04-14T08:42:39.909607+0000', 'last_fullsized': '2024-04-14T08:42:39.909607+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:42:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.31 is not active+clean 2024-04-14T08:42:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.893081+0000', 'last_change': '2024-04-14T08:42:39.893081+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.893081+0000', 'last_undegraded': '2024-04-14T08:42:39.893081+0000', 'last_fullsized': '2024-04-14T08:42:39.893081+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:42:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.30 is not active+clean 2024-04-14T08:42:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.903738+0000', 'last_change': '2024-04-14T08:42:39.903738+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.903738+0000', 'last_undegraded': '2024-04-14T08:42:39.903738+0000', 'last_fullsized': '2024-04-14T08:42:39.903738+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:42:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2f is not active+clean 2024-04-14T08:42:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.903068+0000', 'last_change': '2024-04-14T08:42:39.903068+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.903068+0000', 'last_undegraded': '2024-04-14T08:42:39.903068+0000', 'last_fullsized': '2024-04-14T08:42:39.903068+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:42:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2e is not active+clean 2024-04-14T08:42:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.905703+0000', 'last_change': '2024-04-14T08:42:39.905703+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.905703+0000', 'last_undegraded': '2024-04-14T08:42:39.905703+0000', 'last_fullsized': '2024-04-14T08:42:39.905703+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:42:44.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2d is not active+clean 2024-04-14T08:42:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.896903+0000', 'last_change': '2024-04-14T08:42:39.896903+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.896903+0000', 'last_undegraded': '2024-04-14T08:42:39.896903+0000', 'last_fullsized': '2024-04-14T08:42:39.896903+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:42:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2c is not active+clean 2024-04-14T08:42:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.908810+0000', 'last_change': '2024-04-14T08:42:39.908810+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.908810+0000', 'last_undegraded': '2024-04-14T08:42:39.908810+0000', 'last_fullsized': '2024-04-14T08:42:39.908810+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:42:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2a is not active+clean 2024-04-14T08:42:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.892991+0000', 'last_change': '2024-04-14T08:42:39.892991+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.892991+0000', 'last_undegraded': '2024-04-14T08:42:39.892991+0000', 'last_fullsized': '2024-04-14T08:42:39.892991+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:42:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.28 is not active+clean 2024-04-14T08:42:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.899098+0000', 'last_change': '2024-04-14T08:42:39.899098+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.899098+0000', 'last_undegraded': '2024-04-14T08:42:39.899098+0000', 'last_fullsized': '2024-04-14T08:42:39.899098+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:42:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.27 is not active+clean 2024-04-14T08:42:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.900930+0000', 'last_change': '2024-04-14T08:42:39.900930+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.900930+0000', 'last_undegraded': '2024-04-14T08:42:39.900930+0000', 'last_fullsized': '2024-04-14T08:42:39.900930+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:42:44.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.26 is not active+clean 2024-04-14T08:42:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.906240+0000', 'last_change': '2024-04-14T08:42:39.906240+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.906240+0000', 'last_undegraded': '2024-04-14T08:42:39.906240+0000', 'last_fullsized': '2024-04-14T08:42:39.906240+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:42:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.25 is not active+clean 2024-04-14T08:42:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.896498+0000', 'last_change': '2024-04-14T08:42:39.896498+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.896498+0000', 'last_undegraded': '2024-04-14T08:42:39.896498+0000', 'last_fullsized': '2024-04-14T08:42:39.896498+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:42:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.24 is not active+clean 2024-04-14T08:42:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.904518+0000', 'last_change': '2024-04-14T08:42:39.904518+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.904518+0000', 'last_undegraded': '2024-04-14T08:42:39.904518+0000', 'last_fullsized': '2024-04-14T08:42:39.904518+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:42:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.23 is not active+clean 2024-04-14T08:42:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.905764+0000', 'last_change': '2024-04-14T08:42:39.905764+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.905764+0000', 'last_undegraded': '2024-04-14T08:42:39.905764+0000', 'last_fullsized': '2024-04-14T08:42:39.905764+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:42:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.22 is not active+clean 2024-04-14T08:42:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.909220+0000', 'last_change': '2024-04-14T08:42:39.909220+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.909220+0000', 'last_undegraded': '2024-04-14T08:42:39.909220+0000', 'last_fullsized': '2024-04-14T08:42:39.909220+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:42:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.21 is not active+clean 2024-04-14T08:42:44.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.899058+0000', 'last_change': '2024-04-14T08:42:39.899058+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.899058+0000', 'last_undegraded': '2024-04-14T08:42:39.899058+0000', 'last_fullsized': '2024-04-14T08:42:39.899058+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:42:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.20 is not active+clean 2024-04-14T08:42:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.898579+0000', 'last_change': '2024-04-14T08:42:39.898579+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.898579+0000', 'last_undegraded': '2024-04-14T08:42:39.898579+0000', 'last_fullsized': '2024-04-14T08:42:39.898579+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:42:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1c is not active+clean 2024-04-14T08:42:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.902637+0000', 'last_change': '2024-04-14T08:42:39.902637+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.902637+0000', 'last_undegraded': '2024-04-14T08:42:39.902637+0000', 'last_fullsized': '2024-04-14T08:42:39.902637+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:42:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1d is not active+clean 2024-04-14T08:42:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.903562+0000', 'last_change': '2024-04-14T08:42:39.903562+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.903562+0000', 'last_undegraded': '2024-04-14T08:42:39.903562+0000', 'last_fullsized': '2024-04-14T08:42:39.903562+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:42:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1e is not active+clean 2024-04-14T08:42:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.900508+0000', 'last_change': '2024-04-14T08:42:39.900508+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.900508+0000', 'last_undegraded': '2024-04-14T08:42:39.900508+0000', 'last_fullsized': '2024-04-14T08:42:39.900508+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:42:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.18 is not active+clean 2024-04-14T08:42:44.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.902219+0000', 'last_change': '2024-04-14T08:42:39.902219+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.902219+0000', 'last_undegraded': '2024-04-14T08:42:39.902219+0000', 'last_fullsized': '2024-04-14T08:42:39.902219+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:42:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.19 is not active+clean 2024-04-14T08:42:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.891105+0000', 'last_change': '2024-04-14T08:42:39.891105+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.891105+0000', 'last_undegraded': '2024-04-14T08:42:39.891105+0000', 'last_fullsized': '2024-04-14T08:42:39.891105+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:42:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1a is not active+clean 2024-04-14T08:42:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.908821+0000', 'last_change': '2024-04-14T08:42:39.908821+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.908821+0000', 'last_undegraded': '2024-04-14T08:42:39.908821+0000', 'last_fullsized': '2024-04-14T08:42:39.908821+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:42:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1b is not active+clean 2024-04-14T08:42:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.902699+0000', 'last_change': '2024-04-14T08:42:39.902699+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.902699+0000', 'last_undegraded': '2024-04-14T08:42:39.902699+0000', 'last_fullsized': '2024-04-14T08:42:39.902699+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:42:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.14 is not active+clean 2024-04-14T08:42:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.896257+0000', 'last_change': '2024-04-14T08:42:39.896257+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.896257+0000', 'last_undegraded': '2024-04-14T08:42:39.896257+0000', 'last_fullsized': '2024-04-14T08:42:39.896257+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:42:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.15 is not active+clean 2024-04-14T08:42:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.900647+0000', 'last_change': '2024-04-14T08:42:39.900647+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.900647+0000', 'last_undegraded': '2024-04-14T08:42:39.900647+0000', 'last_fullsized': '2024-04-14T08:42:39.900647+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:42:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.16 is not active+clean 2024-04-14T08:42:44.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.905288+0000', 'last_change': '2024-04-14T08:42:39.905288+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.905288+0000', 'last_undegraded': '2024-04-14T08:42:39.905288+0000', 'last_fullsized': '2024-04-14T08:42:39.905288+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:42:44.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.17 is not active+clean 2024-04-14T08:42:44.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.900522+0000', 'last_change': '2024-04-14T08:42:39.900522+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.900522+0000', 'last_undegraded': '2024-04-14T08:42:39.900522+0000', 'last_fullsized': '2024-04-14T08:42:39.900522+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:42:44.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.10 is not active+clean 2024-04-14T08:42:44.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.896919+0000', 'last_change': '2024-04-14T08:42:39.896919+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.896919+0000', 'last_undegraded': '2024-04-14T08:42:39.896919+0000', 'last_fullsized': '2024-04-14T08:42:39.896919+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-14T08:42:44.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.11 is not active+clean 2024-04-14T08:42:44.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.902158+0000', 'last_change': '2024-04-14T08:42:39.902158+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.902158+0000', 'last_undegraded': '2024-04-14T08:42:39.902158+0000', 'last_fullsized': '2024-04-14T08:42:39.902158+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:42:44.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.12 is not active+clean 2024-04-14T08:42:44.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.905085+0000', 'last_change': '2024-04-14T08:42:39.905085+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.905085+0000', 'last_undegraded': '2024-04-14T08:42:39.905085+0000', 'last_fullsized': '2024-04-14T08:42:39.905085+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:42:44.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.c is not active+clean 2024-04-14T08:42:44.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.899650+0000', 'last_change': '2024-04-14T08:42:39.899650+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.899650+0000', 'last_undegraded': '2024-04-14T08:42:39.899650+0000', 'last_fullsized': '2024-04-14T08:42:39.899650+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:42:44.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.d is not active+clean 2024-04-14T08:42:44.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.892168+0000', 'last_change': '2024-04-14T08:42:39.892168+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.892168+0000', 'last_undegraded': '2024-04-14T08:42:39.892168+0000', 'last_fullsized': '2024-04-14T08:42:39.892168+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-14T08:42:44.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.e is not active+clean 2024-04-14T08:42:44.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.904059+0000', 'last_change': '2024-04-14T08:42:39.904059+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.904059+0000', 'last_undegraded': '2024-04-14T08:42:39.904059+0000', 'last_fullsized': '2024-04-14T08:42:39.904059+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:42:44.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.f is not active+clean 2024-04-14T08:42:44.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.900070+0000', 'last_change': '2024-04-14T08:42:39.900070+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.900070+0000', 'last_undegraded': '2024-04-14T08:42:39.900070+0000', 'last_fullsized': '2024-04-14T08:42:39.900070+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:42:44.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.8 is not active+clean 2024-04-14T08:42:44.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.902777+0000', 'last_change': '2024-04-14T08:42:39.902777+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.902777+0000', 'last_undegraded': '2024-04-14T08:42:39.902777+0000', 'last_fullsized': '2024-04-14T08:42:39.902777+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:42:44.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.9 is not active+clean 2024-04-14T08:42:44.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.904226+0000', 'last_change': '2024-04-14T08:42:39.904226+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.904226+0000', 'last_undegraded': '2024-04-14T08:42:39.904226+0000', 'last_fullsized': '2024-04-14T08:42:39.904226+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:42:44.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.a is not active+clean 2024-04-14T08:42:44.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.908418+0000', 'last_change': '2024-04-14T08:42:39.908418+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.908418+0000', 'last_undegraded': '2024-04-14T08:42:39.908418+0000', 'last_fullsized': '2024-04-14T08:42:39.908418+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:42:44.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.b is not active+clean 2024-04-14T08:42:44.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.899442+0000', 'last_change': '2024-04-14T08:42:39.899442+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.899442+0000', 'last_undegraded': '2024-04-14T08:42:39.899442+0000', 'last_fullsized': '2024-04-14T08:42:39.899442+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-14T08:42:44.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.4 is not active+clean 2024-04-14T08:42:44.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.903140+0000', 'last_change': '2024-04-14T08:42:39.903140+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.903140+0000', 'last_undegraded': '2024-04-14T08:42:39.903140+0000', 'last_fullsized': '2024-04-14T08:42:39.903140+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:42:44.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.5 is not active+clean 2024-04-14T08:42:44.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.901623+0000', 'last_change': '2024-04-14T08:42:39.901623+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.901623+0000', 'last_undegraded': '2024-04-14T08:42:39.901623+0000', 'last_fullsized': '2024-04-14T08:42:39.901623+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:42:44.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.7 is not active+clean 2024-04-14T08:42:44.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.904317+0000', 'last_change': '2024-04-14T08:42:39.904317+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.904317+0000', 'last_undegraded': '2024-04-14T08:42:39.904317+0000', 'last_fullsized': '2024-04-14T08:42:39.904317+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-14T08:42:44.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.0 is not active+clean 2024-04-14T08:42:44.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.901878+0000', 'last_change': '2024-04-14T08:42:39.901878+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.901878+0000', 'last_undegraded': '2024-04-14T08:42:39.901878+0000', 'last_fullsized': '2024-04-14T08:42:39.901878+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-14T08:42:44.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1 is not active+clean 2024-04-14T08:42:44.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.903815+0000', 'last_change': '2024-04-14T08:42:39.903815+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.903815+0000', 'last_undegraded': '2024-04-14T08:42:39.903815+0000', 'last_fullsized': '2024-04-14T08:42:39.903815+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-14T08:42:44.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3 is not active+clean 2024-04-14T08:42:44.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-14T08:42:39.901718+0000', 'last_change': '2024-04-14T08:42:39.901718+0000', 'last_active': '2024-04-14T08:42:39.880431+0000', 'last_peered': '2024-04-14T08:42:39.880431+0000', 'last_clean': '2024-04-14T08:42:39.880431+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-14T08:42:39.901718+0000', 'last_undegraded': '2024-04-14T08:42:39.901718+0000', 'last_fullsized': '2024-04-14T08:42:39.901718+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'last_clean_scrub_stamp': '2024-04-14T08:42:39.880431+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-14T08:42:44.621 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-14T08:42:44.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.746+0000 7f8fb3d74640 1 Processor -- start 2024-04-14T08:42:44.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.747+0000 7f8fb3d74640 1 -- start start 2024-04-14T08:42:44.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.747+0000 7f8fb3d74640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fac0b1df0 0x7f8fac0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.747+0000 7f8fb3d74640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8fac0b21d0 0x7f8fac15c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.747+0000 7f8fb3d74640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fac0b3430 con 0x7f8fac15c5a0 2024-04-14T08:42:44.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.747+0000 7f8fb3d74640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f8fac0b3700 con 0x7f8fac0b21d0 2024-04-14T08:42:44.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.747+0000 7f8fb3d74640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8fac0b39e0 con 0x7f8fac0b1df0 2024-04-14T08:42:44.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.748+0000 7f8fb2311640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f8fac15c5a0 0x7f8fac0b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34204/0 (socket says 172.21.15.172:34204) 2024-04-14T08:42:44.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.748+0000 7f8fb2311640 1 -- 172.21.15.172:0/1490667999 learned_addr learned my addr 172.21.15.172:0/1490667999 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:44.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.748+0000 7f8fb1b10640 1 --2- 172.21.15.172:0/1490667999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fac0b1df0 0x7f8fac0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.748+0000 7f8fb130f640 1 --2- 172.21.15.172:0/1490667999 >> v2:172.21.15.177:3300/0 conn(0x7f8fac0b21d0 0x7f8fac15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.748+0000 7f8fb0b0e640 1 -- 172.21.15.172:0/1490667999 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2960920055 0 0) 0x7f8fac0b3430 con 0x7f8fac15c5a0 2024-04-14T08:42:44.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.748+0000 7f8fb0b0e640 1 -- 172.21.15.172:0/1490667999 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f98003520 con 0x7f8fac15c5a0 2024-04-14T08:42:44.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.748+0000 7f8fb130f640 1 -- 172.21.15.172:0/1490667999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fac0b1df0 msgr2=0x7f8fac0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:44.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.748+0000 7f8fb130f640 1 --2- 172.21.15.172:0/1490667999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fac0b1df0 0x7f8fac0b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.748+0000 7f8fb130f640 1 -- 172.21.15.172:0/1490667999 >> v1:172.21.15.172:6789/0 conn(0x7f8fac15c5a0 legacy=0x7f8fac0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.748+0000 7f8fb130f640 1 -- 172.21.15.172:0/1490667999 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fac0b3cc0 con 0x7f8fac0b21d0 2024-04-14T08:42:44.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.748+0000 7f8fb1b10640 1 --2- 172.21.15.172:0/1490667999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fac0b1df0 0x7f8fac0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:44.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.749+0000 7f8fb130f640 1 --2- 172.21.15.172:0/1490667999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fac0b21d0 0x7f8fac15c060 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0097a0 tx=0x7f8f9c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d804b155d81db2ee server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:44.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.749+0000 7f8fb0b0e640 1 -- 172.21.15.172:0/1490667999 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c035020 con 0x7f8fac0b21d0 2024-04-14T08:42:44.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.749+0000 7f8fb0b0e640 1 -- 172.21.15.172:0/1490667999 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f9c031830 con 0x7f8fac0b21d0 2024-04-14T08:42:44.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.749+0000 7f8fb0b0e640 1 -- 172.21.15.172:0/1490667999 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c031be0 con 0x7f8fac0b21d0 2024-04-14T08:42:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.749+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1490667999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fac0b21d0 msgr2=0x7f8fac15c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.749+0000 7f8fb3d74640 1 --2- 172.21.15.172:0/1490667999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fac0b21d0 0x7f8fac15c060 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0097a0 tx=0x7f8f9c02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.749+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1490667999 shutdown_connections 2024-04-14T08:42:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.749+0000 7f8fb3d74640 1 --2- 172.21.15.172:0/1490667999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fac0b21d0 0x7f8fac15c060 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.749+0000 7f8fb3d74640 1 --2- 172.21.15.172:0/1490667999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fac0b1df0 0x7f8fac0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.749+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1490667999 >> 172.21.15.172:0/1490667999 conn(0x7f8fac06ec30 msgr2=0x7f8fac06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.750+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1490667999 shutdown_connections 2024-04-14T08:42:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.750+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1490667999 wait complete. 2024-04-14T08:42:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb3d74640 1 Processor -- start 2024-04-14T08:42:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb3d74640 1 -- start start 2024-04-14T08:42:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb3d74640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fac0b1df0 0x7f8fac0ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb3d74640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fac0b21d0 0x7f8fac0ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb3d74640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fac157880 0x7f8fac0728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb3d74640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac0b4750 con 0x7f8fac157880 2024-04-14T08:42:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb3d74640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac15b7f0 con 0x7f8fac0b1df0 2024-04-14T08:42:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb3d74640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8fac0b48d0 con 0x7f8fac0b21d0 2024-04-14T08:42:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb2311640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fac157880 0x7f8fac0728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb1b10640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fac0b1df0 0x7f8fac0ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb1b10640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fac0b1df0 0x7f8fac0ab6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33258/0 (socket says 172.21.15.172:33258) 2024-04-14T08:42:44.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb130f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fac0b21d0 0x7f8fac0ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb1b10640 1 -- 172.21.15.172:0/1964121073 learned_addr learned my addr 172.21.15.172:0/1964121073 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:44.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.751+0000 7f8fb2311640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fac157880 0x7f8fac0728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39872/0 (socket says 172.21.15.172:39872) 2024-04-14T08:42:44.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.752+0000 7f8fb2311640 1 -- 172.21.15.172:0/1964121073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fac0b21d0 msgr2=0x7f8fac0ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.752+0000 7f8fb2311640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fac0b21d0 0x7f8fac0ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.752+0000 7f8fb2311640 1 -- 172.21.15.172:0/1964121073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fac0b1df0 msgr2=0x7f8fac0ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:44.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.752+0000 7f8fb2311640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fac0b1df0 0x7f8fac0ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:44.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.752+0000 7f8fb2311640 1 -- 172.21.15.172:0/1964121073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fac1f5660 con 0x7f8fac157880 2024-04-14T08:42:44.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.752+0000 7f8fb1b10640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fac0b1df0 0x7f8fac0ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:44.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.752+0000 7f8fb2311640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fac157880 0x7f8fac0728c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800c310 tx=0x7f8fa8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:44.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.752+0000 7f8fa27fc640 1 -- 172.21.15.172:0/1964121073 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8017070 con 0x7f8fac157880 2024-04-14T08:42:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.752+0000 7f8fa27fc640 1 -- 172.21.15.172:0/1964121073 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa8005ce0 con 0x7f8fac157880 2024-04-14T08:42:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.752+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1964121073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fac1f5940 con 0x7f8fac157880 2024-04-14T08:42:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.752+0000 7f8fa27fc640 1 -- 172.21.15.172:0/1964121073 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa80072d0 con 0x7f8fac157880 2024-04-14T08:42:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.752+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1964121073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fac1f5f00 con 0x7f8fac157880 2024-04-14T08:42:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.753+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1964121073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fac1f6140 con 0x7f8fac157880 2024-04-14T08:42:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.754+0000 7f8fa27fc640 1 -- 172.21.15.172:0/1964121073 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fa801a070 con 0x7f8fac157880 2024-04-14T08:42:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.754+0000 7f8fa27fc640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8f9006c8f0 0x7f8f9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.754+0000 7f8fa27fc640 1 -- 172.21.15.172:0/1964121073 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f8fa808ae10 con 0x7f8fac157880 2024-04-14T08:42:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.754+0000 7f8fb1b10640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8f9006c8f0 0x7f8f9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:44.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.754+0000 7f8fb1b10640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8f9006c8f0 0x7f8f9006edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009bd0 tx=0x7f8fa400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:44.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:44.759+0000 7f8fa27fc640 1 -- 172.21.15.172:0/1964121073 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8fa8058af0 con 0x7f8fac157880 2024-04-14T08:42:45.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.003+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1964121073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8fac071610 con 0x7f8fac157880 2024-04-14T08:42:45.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.004+0000 7f8fa27fc640 1 -- 172.21.15.172:0/1964121073 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1260 (secure 0 0 0) 0x7f8fa805cdf0 con 0x7f8fac157880 2024-04-14T08:42:45.004 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:45.004 INFO:teuthology.orchestra.run.smithi172.stdout:{"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":616,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":116,"num_osds":8,"num_up_osds":8,"osd_up_since":1713083560,"num_in_osds":8,"osd_in_since":1713083551,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":56},{"state_name":"active+clean","count":41}],"num_pgs":97,"num_pools":3,"num_objects":4,"data_bytes":590438,"bytes_used":267935744,"bytes_avail":772826177536,"bytes_total":773094113280,"inactive_pgs_ratio":0.57731956243515015,"write_bytes_sec":235,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":88,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":9060}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-14T08:42:25.234716+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-14T08:42:45.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.006+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1964121073 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8f9006c8f0 msgr2=0x7f8f9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:45.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.006+0000 7f8fb3d74640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8f9006c8f0 0x7f8f9006edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009bd0 tx=0x7f8fa400b040 comp rx=0 tx=0).stop 2024-04-14T08:42:45.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.006+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1964121073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fac157880 msgr2=0x7f8fac0728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:45.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.006+0000 7f8fb3d74640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fac157880 0x7f8fac0728c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800c310 tx=0x7f8fa8001e70 comp rx=0 tx=0).stop 2024-04-14T08:42:45.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.006+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1964121073 shutdown_connections 2024-04-14T08:42:45.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.006+0000 7f8fb3d74640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8f9006c8f0 0x7f8f9006edb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:45.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.006+0000 7f8fb3d74640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fac157880 0x7f8fac0728c0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:45.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.006+0000 7f8fb3d74640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fac0b21d0 0x7f8fac0ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:45.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.006+0000 7f8fb3d74640 1 --2- 172.21.15.172:0/1964121073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fac0b1df0 0x7f8fac0ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:45.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.006+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1964121073 >> 172.21.15.172:0/1964121073 conn(0x7f8fac06ec30 msgr2=0x7f8fac0b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:45.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.006+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1964121073 shutdown_connections 2024-04-14T08:42:45.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.006+0000 7f8fb3d74640 1 -- 172.21.15.172:0/1964121073 wait complete. 2024-04-14T08:42:45.018 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-14T08:42:45.018 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:42:45.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.133+0000 7f2a583bb640 1 Processor -- start 2024-04-14T08:42:45.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.133+0000 7f2a583bb640 1 -- start start 2024-04-14T08:42:45.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.134+0000 7f2a583bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a500d2830 0x7f2a500d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:45.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.134+0000 7f2a583bb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a500d3150 con 0x7f2a500cedd0 2024-04-14T08:42:45.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.134+0000 7f2a583bb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a500cf320 con 0x7f2a5015e890 2024-04-14T08:42:45.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.134+0000 7f2a583bb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2a500cff30 con 0x7f2a500d2830 2024-04-14T08:42:45.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.134+0000 7f2a56157640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a500d2830 0x7f2a500d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:45.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.135+0000 7f2a56958640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2a500cedd0 0x7f2a500b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34216/0 (socket says 172.21.15.172:34216) 2024-04-14T08:42:45.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.135+0000 7f2a56157640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a500d2830 0x7f2a500d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39886/0 (socket says 172.21.15.172:39886) 2024-04-14T08:42:45.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.135+0000 7f2a56958640 1 -- 172.21.15.172:0/1661997258 learned_addr learned my addr 172.21.15.172:0/1661997258 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:45.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.135+0000 7f2a56157640 1 -- 172.21.15.172:0/1661997258 >> v1:172.21.15.177:6789/0 conn(0x7f2a5015e890 legacy=0x7f2a500d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:45.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.135+0000 7f2a56157640 1 -- 172.21.15.172:0/1661997258 >> v1:172.21.15.172:6789/0 conn(0x7f2a500cedd0 legacy=0x7f2a500b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:45.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.135+0000 7f2a55155640 1 -- 172.21.15.172:0/1661997258 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1370054153 0 0) 0x7f2a500cf320 con 0x7f2a5015e890 2024-04-14T08:42:45.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.135+0000 7f2a56157640 1 -- 172.21.15.172:0/1661997258 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a500cf600 con 0x7f2a500d2830 2024-04-14T08:42:45.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.136+0000 7f2a56157640 1 --2- 172.21.15.172:0/1661997258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a500d2830 0x7f2a500d2c10 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f2a400097a0 tx=0x7f2a4002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf69494a54f3b447 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:45.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.136+0000 7f2a55155640 1 -- 172.21.15.172:0/1661997258 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40038070 con 0x7f2a500d2830 2024-04-14T08:42:45.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.136+0000 7f2a55155640 1 -- 172.21.15.172:0/1661997258 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a40031700 con 0x7f2a500d2830 2024-04-14T08:42:45.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.136+0000 7f2a55155640 1 -- 172.21.15.172:0/1661997258 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40031ad0 con 0x7f2a500d2830 2024-04-14T08:42:45.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.136+0000 7f2a583bb640 1 -- 172.21.15.172:0/1661997258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a500d2830 msgr2=0x7f2a500d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:45.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.136+0000 7f2a583bb640 1 --2- 172.21.15.172:0/1661997258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a500d2830 0x7f2a500d2c10 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f2a400097a0 tx=0x7f2a4002b820 comp rx=0 tx=0).stop 2024-04-14T08:42:45.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.136+0000 7f2a583bb640 1 -- 172.21.15.172:0/1661997258 shutdown_connections 2024-04-14T08:42:45.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.136+0000 7f2a583bb640 1 --2- 172.21.15.172:0/1661997258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a500d2830 0x7f2a500d2c10 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:45.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.136+0000 7f2a583bb640 1 -- 172.21.15.172:0/1661997258 >> 172.21.15.172:0/1661997258 conn(0x7f2a5009d590 msgr2=0x7f2a500d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:45.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.136+0000 7f2a583bb640 1 -- 172.21.15.172:0/1661997258 shutdown_connections 2024-04-14T08:42:45.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.136+0000 7f2a583bb640 1 -- 172.21.15.172:0/1661997258 wait complete. 2024-04-14T08:42:45.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.137+0000 7f2a583bb640 1 Processor -- start 2024-04-14T08:42:45.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.138+0000 7f2a583bb640 1 -- start start 2024-04-14T08:42:45.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.138+0000 7f2a583bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a500cedd0 0x7f2a5006d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:45.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.138+0000 7f2a583bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a500d2830 0x7f2a500b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:45.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.138+0000 7f2a583bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a5015e890 0x7f2a5015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:45.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.138+0000 7f2a583bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a500cfbc0 con 0x7f2a500cedd0 2024-04-14T08:42:45.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.138+0000 7f2a583bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a500d2350 con 0x7f2a500d2830 2024-04-14T08:42:45.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.138+0000 7f2a583bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2a500b3e20 con 0x7f2a5015e890 2024-04-14T08:42:45.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.138+0000 7f2a56157640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a500cedd0 0x7f2a5006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:45.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.138+0000 7f2a56157640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a500cedd0 0x7f2a5006d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39902/0 (socket says 172.21.15.172:39902) 2024-04-14T08:42:45.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.138+0000 7f2a56157640 1 -- 172.21.15.172:0/3295650473 learned_addr learned my addr 172.21.15.172:0/3295650473 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:45.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.138+0000 7f2a55956640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a500d2830 0x7f2a500b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:45.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.138+0000 7f2a56958640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a5015e890 0x7f2a5015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:45.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.139+0000 7f2a56157640 1 -- 172.21.15.172:0/3295650473 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a5015e890 msgr2=0x7f2a5015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:45.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.139+0000 7f2a56157640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a5015e890 0x7f2a5015ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:45.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.139+0000 7f2a56157640 1 -- 172.21.15.172:0/3295650473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a500d2830 msgr2=0x7f2a500b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:45.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.139+0000 7f2a56157640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a500d2830 0x7f2a500b76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:45.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.139+0000 7f2a56157640 1 -- 172.21.15.172:0/3295650473 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a500c9130 con 0x7f2a500cedd0 2024-04-14T08:42:45.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.139+0000 7f2a56157640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a500cedd0 0x7f2a5006d590 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f2a40000c00 tx=0x7f2a400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:45.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.139+0000 7f2a46ffd640 1 -- 172.21.15.172:0/3295650473 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40043070 con 0x7f2a500cedd0 2024-04-14T08:42:45.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.139+0000 7f2a46ffd640 1 -- 172.21.15.172:0/3295650473 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a40033a30 con 0x7f2a500cedd0 2024-04-14T08:42:45.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.139+0000 7f2a583bb640 1 -- 172.21.15.172:0/3295650473 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a500c9410 con 0x7f2a500cedd0 2024-04-14T08:42:45.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.139+0000 7f2a46ffd640 1 -- 172.21.15.172:0/3295650473 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40038040 con 0x7f2a500cedd0 2024-04-14T08:42:45.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.139+0000 7f2a583bb640 1 -- 172.21.15.172:0/3295650473 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a500c99d0 con 0x7f2a500cedd0 2024-04-14T08:42:45.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.140+0000 7f2a583bb640 1 -- 172.21.15.172:0/3295650473 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a50068520 con 0x7f2a500cedd0 2024-04-14T08:42:45.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.141+0000 7f2a46ffd640 1 -- 172.21.15.172:0/3295650473 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a40046070 con 0x7f2a500cedd0 2024-04-14T08:42:45.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.141+0000 7f2a46ffd640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3406c8f0 0x7f2a3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:45.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.141+0000 7f2a55956640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3406c8f0 0x7f2a3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:45.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.141+0000 7f2a46ffd640 1 -- 172.21.15.172:0/3295650473 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f2a400ada30 con 0x7f2a500cedd0 2024-04-14T08:42:45.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.141+0000 7f2a55956640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3406c8f0 0x7f2a3406edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2a501f7a70 tx=0x7f2a4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:45.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.146+0000 7f2a46ffd640 1 -- 172.21.15.172:0/3295650473 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2a40030070 con 0x7f2a500cedd0 2024-04-14T08:42:45.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.321+0000 7f2a583bb640 1 -- 172.21.15.172:0/3295650473 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a500d2610 con 0x7f2a3406c8f0 2024-04-14T08:42:45.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.327+0000 7f2a46ffd640 1 -- 172.21.15.172:0/3295650473 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7f2a500d2610 con 0x7f2a3406c8f0 2024-04-14T08:42:45.329 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:45.331 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:42:45.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.330+0000 7f2a583bb640 1 -- 172.21.15.172:0/3295650473 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3406c8f0 msgr2=0x7f2a3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:45.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.330+0000 7f2a583bb640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3406c8f0 0x7f2a3406edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2a501f7a70 tx=0x7f2a4c009040 comp rx=0 tx=0).stop 2024-04-14T08:42:45.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.330+0000 7f2a583bb640 1 -- 172.21.15.172:0/3295650473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a500cedd0 msgr2=0x7f2a5006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:45.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.330+0000 7f2a583bb640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a500cedd0 0x7f2a5006d590 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f2a40000c00 tx=0x7f2a400046c0 comp rx=0 tx=0).stop 2024-04-14T08:42:45.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.330+0000 7f2a583bb640 1 -- 172.21.15.172:0/3295650473 shutdown_connections 2024-04-14T08:42:45.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.330+0000 7f2a583bb640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a3406c8f0 0x7f2a3406edb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:45.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.330+0000 7f2a583bb640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a5015e890 0x7f2a5015ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:45.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.330+0000 7f2a583bb640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a500d2830 0x7f2a500b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:45.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.330+0000 7f2a583bb640 1 --2- 172.21.15.172:0/3295650473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a500cedd0 0x7f2a5006d590 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:45.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.330+0000 7f2a583bb640 1 -- 172.21.15.172:0/3295650473 >> 172.21.15.172:0/3295650473 conn(0x7f2a5009d590 msgr2=0x7f2a500b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:45.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.330+0000 7f2a583bb640 1 -- 172.21.15.172:0/3295650473 shutdown_connections 2024-04-14T08:42:45.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:45.330+0000 7f2a583bb640 1 -- 172.21.15.172:0/3295650473 wait complete. 2024-04-14T08:42:45.341 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-04-14T08:42:45.238229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261936,"kb_used_data":12480,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712784,"statfs":{"total":773094113280,"available":772825890816,"internally_reserved":0,"allocated":12779520,"data_stored":6561481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001038"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914421+0000","last_change":"2024-04-14T08:42:40.910371+0000","last_active":"2024-04-14T08:42:41.914421+0000","last_peered":"2024-04-14T08:42:41.914421+0000","last_clean":"2024-04-14T08:42:41.914421+0000","last_became_active":"2024-04-14T08:42:40.910102+0000","last_became_peered":"2024-04-14T08:42:40.910102+0000","last_unstale":"2024-04-14T08:42:41.914421+0000","last_undegraded":"2024-04-14T08:42:41.914421+0000","last_fullsized":"2024-04-14T08:42:41.914421+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:51.037275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909433+0000","last_change":"2024-04-14T08:42:40.919215+0000","last_active":"2024-04-14T08:42:41.909433+0000","last_peered":"2024-04-14T08:42:41.909433+0000","last_clean":"2024-04-14T08:42:41.909433+0000","last_became_active":"2024-04-14T08:42:40.918909+0000","last_became_peered":"2024-04-14T08:42:40.918909+0000","last_unstale":"2024-04-14T08:42:41.909433+0000","last_undegraded":"2024-04-14T08:42:41.909433+0000","last_fullsized":"2024-04-14T08:42:41.909433+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:38:30.115362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909385+0000","last_change":"2024-04-14T08:42:40.913610+0000","last_active":"2024-04-14T08:42:41.909385+0000","last_peered":"2024-04-14T08:42:41.909385+0000","last_clean":"2024-04-14T08:42:41.909385+0000","last_became_active":"2024-04-14T08:42:40.913312+0000","last_became_peered":"2024-04-14T08:42:40.913312+0000","last_unstale":"2024-04-14T08:42:41.909385+0000","last_undegraded":"2024-04-14T08:42:41.909385+0000","last_fullsized":"2024-04-14T08:42:41.909385+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:05:03.546435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.900277+0000","last_change":"2024-04-14T08:42:40.899261+0000","last_active":"2024-04-14T08:42:40.900277+0000","last_peered":"2024-04-14T08:42:40.900277+0000","last_clean":"2024-04-14T08:42:40.900277+0000","last_became_active":"2024-04-14T08:42:40.898970+0000","last_became_peered":"2024-04-14T08:42:40.898970+0000","last_unstale":"2024-04-14T08:42:40.900277+0000","last_undegraded":"2024-04-14T08:42:40.900277+0000","last_fullsized":"2024-04-14T08:42:40.900277+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:57.005587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915042+0000","last_change":"2024-04-14T08:42:40.903545+0000","last_active":"2024-04-14T08:42:41.915042+0000","last_peered":"2024-04-14T08:42:41.915042+0000","last_clean":"2024-04-14T08:42:41.915042+0000","last_became_active":"2024-04-14T08:42:40.903269+0000","last_became_peered":"2024-04-14T08:42:40.903269+0000","last_unstale":"2024-04-14T08:42:41.915042+0000","last_undegraded":"2024-04-14T08:42:41.915042+0000","last_fullsized":"2024-04-14T08:42:41.915042+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:40.822681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902585+0000","last_change":"2024-04-14T08:42:40.924183+0000","last_active":"2024-04-14T08:42:42.902585+0000","last_peered":"2024-04-14T08:42:42.902585+0000","last_clean":"2024-04-14T08:42:42.902585+0000","last_became_active":"2024-04-14T08:42:40.923882+0000","last_became_peered":"2024-04-14T08:42:40.923882+0000","last_unstale":"2024-04-14T08:42:42.902585+0000","last_undegraded":"2024-04-14T08:42:42.902585+0000","last_fullsized":"2024-04-14T08:42:42.902585+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:30:41.697931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902442+0000","last_change":"2024-04-14T08:42:40.909665+0000","last_active":"2024-04-14T08:42:42.902442+0000","last_peered":"2024-04-14T08:42:42.902442+0000","last_clean":"2024-04-14T08:42:42.902442+0000","last_became_active":"2024-04-14T08:42:40.909352+0000","last_became_peered":"2024-04-14T08:42:40.909352+0000","last_unstale":"2024-04-14T08:42:42.902442+0000","last_undegraded":"2024-04-14T08:42:42.902442+0000","last_fullsized":"2024-04-14T08:42:42.902442+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:56:34.156369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.908996+0000","last_change":"2024-04-14T08:42:40.914676+0000","last_active":"2024-04-14T08:42:41.908996+0000","last_peered":"2024-04-14T08:42:41.908996+0000","last_clean":"2024-04-14T08:42:41.908996+0000","last_became_active":"2024-04-14T08:42:40.914163+0000","last_became_peered":"2024-04-14T08:42:40.914163+0000","last_unstale":"2024-04-14T08:42:41.908996+0000","last_undegraded":"2024-04-14T08:42:41.908996+0000","last_fullsized":"2024-04-14T08:42:41.908996+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:15:02.182102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.914262+0000","last_change":"2024-04-14T08:42:40.909455+0000","last_active":"2024-04-14T08:42:40.914262+0000","last_peered":"2024-04-14T08:42:40.914262+0000","last_clean":"2024-04-14T08:42:40.914262+0000","last_became_active":"2024-04-14T08:42:40.909156+0000","last_became_peered":"2024-04-14T08:42:40.909156+0000","last_unstale":"2024-04-14T08:42:40.914262+0000","last_undegraded":"2024-04-14T08:42:40.914262+0000","last_fullsized":"2024-04-14T08:42:40.914262+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:50:25.331351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290638+0000","last_change":"2024-04-14T08:42:40.914190+0000","last_active":"2024-04-14T08:42:43.290638+0000","last_peered":"2024-04-14T08:42:43.290638+0000","last_clean":"2024-04-14T08:42:43.290638+0000","last_became_active":"2024-04-14T08:42:40.912281+0000","last_became_peered":"2024-04-14T08:42:40.912281+0000","last_unstale":"2024-04-14T08:42:43.290638+0000","last_undegraded":"2024-04-14T08:42:43.290638+0000","last_fullsized":"2024-04-14T08:42:43.290638+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:57.283646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914610+0000","last_change":"2024-04-14T08:42:40.910755+0000","last_active":"2024-04-14T08:42:41.914610+0000","last_peered":"2024-04-14T08:42:41.914610+0000","last_clean":"2024-04-14T08:42:41.914610+0000","last_became_active":"2024-04-14T08:42:40.910464+0000","last_became_peered":"2024-04-14T08:42:40.910464+0000","last_unstale":"2024-04-14T08:42:41.914610+0000","last_undegraded":"2024-04-14T08:42:41.914610+0000","last_fullsized":"2024-04-14T08:42:41.914610+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:57.005587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902797+0000","last_change":"2024-04-14T08:42:40.912985+0000","last_active":"2024-04-14T08:42:42.902797+0000","last_peered":"2024-04-14T08:42:42.902797+0000","last_clean":"2024-04-14T08:42:42.902797+0000","last_became_active":"2024-04-14T08:42:40.912339+0000","last_became_peered":"2024-04-14T08:42:40.912339+0000","last_unstale":"2024-04-14T08:42:42.902797+0000","last_undegraded":"2024-04-14T08:42:42.902797+0000","last_fullsized":"2024-04-14T08:42:42.902797+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:39:59.964083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290778+0000","last_change":"2024-04-14T08:42:40.913208+0000","last_active":"2024-04-14T08:42:43.290778+0000","last_peered":"2024-04-14T08:42:43.290778+0000","last_clean":"2024-04-14T08:42:43.290778+0000","last_became_active":"2024-04-14T08:42:40.912939+0000","last_became_peered":"2024-04-14T08:42:40.912939+0000","last_unstale":"2024-04-14T08:42:43.290778+0000","last_undegraded":"2024-04-14T08:42:43.290778+0000","last_fullsized":"2024-04-14T08:42:43.290778+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:30.073046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909414+0000","last_change":"2024-04-14T08:42:40.922514+0000","last_active":"2024-04-14T08:42:41.909414+0000","last_peered":"2024-04-14T08:42:41.909414+0000","last_clean":"2024-04-14T08:42:41.909414+0000","last_became_active":"2024-04-14T08:42:40.914727+0000","last_became_peered":"2024-04-14T08:42:40.914727+0000","last_unstale":"2024-04-14T08:42:41.909414+0000","last_undegraded":"2024-04-14T08:42:41.909414+0000","last_fullsized":"2024-04-14T08:42:41.909414+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:49.529029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915343+0000","last_change":"2024-04-14T08:42:40.911355+0000","last_active":"2024-04-14T08:42:41.915343+0000","last_peered":"2024-04-14T08:42:41.915343+0000","last_clean":"2024-04-14T08:42:41.915343+0000","last_became_active":"2024-04-14T08:42:40.911065+0000","last_became_peered":"2024-04-14T08:42:40.911065+0000","last_unstale":"2024-04-14T08:42:41.915343+0000","last_undegraded":"2024-04-14T08:42:41.915343+0000","last_fullsized":"2024-04-14T08:42:41.915343+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:42:53.101473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902361+0000","last_change":"2024-04-14T08:42:40.920994+0000","last_active":"2024-04-14T08:42:42.902361+0000","last_peered":"2024-04-14T08:42:42.902361+0000","last_clean":"2024-04-14T08:42:42.902361+0000","last_became_active":"2024-04-14T08:42:40.920704+0000","last_became_peered":"2024-04-14T08:42:40.920704+0000","last_unstale":"2024-04-14T08:42:42.902361+0000","last_undegraded":"2024-04-14T08:42:42.902361+0000","last_fullsized":"2024-04-14T08:42:42.902361+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:55:23.423410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290638+0000","last_change":"2024-04-14T08:42:40.909394+0000","last_active":"2024-04-14T08:42:43.290638+0000","last_peered":"2024-04-14T08:42:43.290638+0000","last_clean":"2024-04-14T08:42:43.290638+0000","last_became_active":"2024-04-14T08:42:40.909131+0000","last_became_peered":"2024-04-14T08:42:40.909131+0000","last_unstale":"2024-04-14T08:42:43.290638+0000","last_undegraded":"2024-04-14T08:42:43.290638+0000","last_fullsized":"2024-04-14T08:42:43.290638+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:32:35.306909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909292+0000","last_change":"2024-04-14T08:42:40.919870+0000","last_active":"2024-04-14T08:42:41.909292+0000","last_peered":"2024-04-14T08:42:41.909292+0000","last_clean":"2024-04-14T08:42:41.909292+0000","last_became_active":"2024-04-14T08:42:40.919598+0000","last_became_peered":"2024-04-14T08:42:40.919598+0000","last_unstale":"2024-04-14T08:42:41.909292+0000","last_undegraded":"2024-04-14T08:42:41.909292+0000","last_fullsized":"2024-04-14T08:42:41.909292+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:19.465728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900264+0000","last_change":"2024-04-14T08:42:40.909410+0000","last_active":"2024-04-14T08:42:42.900264+0000","last_peered":"2024-04-14T08:42:42.900264+0000","last_clean":"2024-04-14T08:42:42.900264+0000","last_became_active":"2024-04-14T08:42:40.909150+0000","last_became_peered":"2024-04-14T08:42:40.909150+0000","last_unstale":"2024-04-14T08:42:42.900264+0000","last_undegraded":"2024-04-14T08:42:42.900264+0000","last_fullsized":"2024-04-14T08:42:42.900264+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:44.417457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290995+0000","last_change":"2024-04-14T08:42:40.921455+0000","last_active":"2024-04-14T08:42:43.290995+0000","last_peered":"2024-04-14T08:42:43.290995+0000","last_clean":"2024-04-14T08:42:43.290995+0000","last_became_active":"2024-04-14T08:42:40.921182+0000","last_became_peered":"2024-04-14T08:42:40.921182+0000","last_unstale":"2024-04-14T08:42:43.290995+0000","last_undegraded":"2024-04-14T08:42:43.290995+0000","last_fullsized":"2024-04-14T08:42:43.290995+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:32:52.097791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.913413+0000","last_change":"2024-04-14T08:42:40.906675+0000","last_active":"2024-04-14T08:42:40.913413+0000","last_peered":"2024-04-14T08:42:40.913413+0000","last_clean":"2024-04-14T08:42:40.913413+0000","last_became_active":"2024-04-14T08:42:40.904423+0000","last_became_peered":"2024-04-14T08:42:40.904423+0000","last_unstale":"2024-04-14T08:42:40.913413+0000","last_undegraded":"2024-04-14T08:42:40.913413+0000","last_fullsized":"2024-04-14T08:42:40.913413+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:42:53.101473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914058+0000","last_change":"2024-04-14T08:42:40.922891+0000","last_active":"2024-04-14T08:42:41.914058+0000","last_peered":"2024-04-14T08:42:41.914058+0000","last_clean":"2024-04-14T08:42:41.914058+0000","last_became_active":"2024-04-14T08:42:40.922613+0000","last_became_peered":"2024-04-14T08:42:40.922613+0000","last_unstale":"2024-04-14T08:42:41.914058+0000","last_undegraded":"2024-04-14T08:42:41.914058+0000","last_fullsized":"2024-04-14T08:42:41.914058+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:44.417457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.921770+0000","last_change":"2024-04-14T08:42:40.919066+0000","last_active":"2024-04-14T08:42:40.921770+0000","last_peered":"2024-04-14T08:42:40.921770+0000","last_clean":"2024-04-14T08:42:40.921770+0000","last_became_active":"2024-04-14T08:42:40.918630+0000","last_became_peered":"2024-04-14T08:42:40.918630+0000","last_unstale":"2024-04-14T08:42:40.921770+0000","last_undegraded":"2024-04-14T08:42:40.921770+0000","last_fullsized":"2024-04-14T08:42:40.921770+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:53:03.886099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290470+0000","last_change":"2024-04-14T08:42:40.912776+0000","last_active":"2024-04-14T08:42:43.290470+0000","last_peered":"2024-04-14T08:42:43.290470+0000","last_clean":"2024-04-14T08:42:43.290470+0000","last_became_active":"2024-04-14T08:42:40.912504+0000","last_became_peered":"2024-04-14T08:42:40.912504+0000","last_unstale":"2024-04-14T08:42:43.290470+0000","last_undegraded":"2024-04-14T08:42:43.290470+0000","last_fullsized":"2024-04-14T08:42:43.290470+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:44.417457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290820+0000","last_change":"2024-04-14T08:42:40.910387+0000","last_active":"2024-04-14T08:42:43.290820+0000","last_peered":"2024-04-14T08:42:43.290820+0000","last_clean":"2024-04-14T08:42:43.290820+0000","last_became_active":"2024-04-14T08:42:40.910101+0000","last_became_peered":"2024-04-14T08:42:40.910101+0000","last_unstale":"2024-04-14T08:42:43.290820+0000","last_undegraded":"2024-04-14T08:42:43.290820+0000","last_fullsized":"2024-04-14T08:42:43.290820+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:56.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290987+0000","last_change":"2024-04-14T08:42:40.911692+0000","last_active":"2024-04-14T08:42:43.290987+0000","last_peered":"2024-04-14T08:42:43.290987+0000","last_clean":"2024-04-14T08:42:43.290987+0000","last_became_active":"2024-04-14T08:42:40.910075+0000","last_became_peered":"2024-04-14T08:42:40.910075+0000","last_unstale":"2024-04-14T08:42:43.290987+0000","last_undegraded":"2024-04-14T08:42:43.290987+0000","last_fullsized":"2024-04-14T08:42:43.290987+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:47:15.652845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900301+0000","last_change":"2024-04-14T08:42:40.908641+0000","last_active":"2024-04-14T08:42:42.900301+0000","last_peered":"2024-04-14T08:42:42.900301+0000","last_clean":"2024-04-14T08:42:42.900301+0000","last_became_active":"2024-04-14T08:42:40.908339+0000","last_became_peered":"2024-04-14T08:42:40.908339+0000","last_unstale":"2024-04-14T08:42:42.900301+0000","last_undegraded":"2024-04-14T08:42:42.900301+0000","last_fullsized":"2024-04-14T08:42:42.900301+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:51.037275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902875+0000","last_change":"2024-04-14T08:42:40.912688+0000","last_active":"2024-04-14T08:42:42.902875+0000","last_peered":"2024-04-14T08:42:42.902875+0000","last_clean":"2024-04-14T08:42:42.902875+0000","last_became_active":"2024-04-14T08:42:40.912007+0000","last_became_peered":"2024-04-14T08:42:40.912007+0000","last_unstale":"2024-04-14T08:42:42.902875+0000","last_undegraded":"2024-04-14T08:42:42.902875+0000","last_fullsized":"2024-04-14T08:42:42.902875+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:45:16.718608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290689+0000","last_change":"2024-04-14T08:42:40.908025+0000","last_active":"2024-04-14T08:42:43.290689+0000","last_peered":"2024-04-14T08:42:43.290689+0000","last_clean":"2024-04-14T08:42:43.290689+0000","last_became_active":"2024-04-14T08:42:40.906658+0000","last_became_peered":"2024-04-14T08:42:40.906658+0000","last_unstale":"2024-04-14T08:42:43.290689+0000","last_undegraded":"2024-04-14T08:42:43.290689+0000","last_fullsized":"2024-04-14T08:42:43.290689+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:08:55.311489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909476+0000","last_change":"2024-04-14T08:42:40.923172+0000","last_active":"2024-04-14T08:42:41.909476+0000","last_peered":"2024-04-14T08:42:41.909476+0000","last_clean":"2024-04-14T08:42:41.909476+0000","last_became_active":"2024-04-14T08:42:40.922918+0000","last_became_peered":"2024-04-14T08:42:40.922918+0000","last_unstale":"2024-04-14T08:42:41.909476+0000","last_undegraded":"2024-04-14T08:42:41.909476+0000","last_fullsized":"2024-04-14T08:42:41.909476+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:01:09.722380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290686+0000","last_change":"2024-04-14T08:42:40.920428+0000","last_active":"2024-04-14T08:42:43.290686+0000","last_peered":"2024-04-14T08:42:43.290686+0000","last_clean":"2024-04-14T08:42:43.290686+0000","last_became_active":"2024-04-14T08:42:40.920121+0000","last_became_peered":"2024-04-14T08:42:40.920121+0000","last_unstale":"2024-04-14T08:42:43.290686+0000","last_undegraded":"2024-04-14T08:42:43.290686+0000","last_fullsized":"2024-04-14T08:42:43.290686+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:40.822681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290475+0000","last_change":"2024-04-14T08:42:40.913456+0000","last_active":"2024-04-14T08:42:43.290475+0000","last_peered":"2024-04-14T08:42:43.290475+0000","last_clean":"2024-04-14T08:42:43.290475+0000","last_became_active":"2024-04-14T08:42:40.913031+0000","last_became_peered":"2024-04-14T08:42:40.913031+0000","last_unstale":"2024-04-14T08:42:43.290475+0000","last_undegraded":"2024-04-14T08:42:43.290475+0000","last_fullsized":"2024-04-14T08:42:43.290475+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:51.037275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290650+0000","last_change":"2024-04-14T08:42:40.916128+0000","last_active":"2024-04-14T08:42:43.290650+0000","last_peered":"2024-04-14T08:42:43.290650+0000","last_clean":"2024-04-14T08:42:43.290650+0000","last_became_active":"2024-04-14T08:42:40.915859+0000","last_became_peered":"2024-04-14T08:42:40.915859+0000","last_unstale":"2024-04-14T08:42:43.290650+0000","last_undegraded":"2024-04-14T08:42:43.290650+0000","last_fullsized":"2024-04-14T08:42:43.290650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:34:55.774170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"115'4","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900211+0000","last_change":"2024-04-14T08:42:39.902925+0000","last_active":"2024-04-14T08:42:42.900211+0000","last_peered":"2024-04-14T08:42:42.900211+0000","last_clean":"2024-04-14T08:42:42.900211+0000","last_became_active":"2024-04-14T08:42:39.902599+0000","last_became_peered":"2024-04-14T08:42:39.902599+0000","last_unstale":"2024-04-14T08:42:42.900211+0000","last_undegraded":"2024-04-14T08:42:42.900211+0000","last_fullsized":"2024-04-14T08:42:42.900211+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:25:26.756136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902825+0000","last_change":"2024-04-14T08:42:40.918689+0000","last_active":"2024-04-14T08:42:42.902825+0000","last_peered":"2024-04-14T08:42:42.902825+0000","last_clean":"2024-04-14T08:42:42.902825+0000","last_became_active":"2024-04-14T08:42:40.918387+0000","last_became_peered":"2024-04-14T08:42:40.918387+0000","last_unstale":"2024-04-14T08:42:42.902825+0000","last_undegraded":"2024-04-14T08:42:42.902825+0000","last_fullsized":"2024-04-14T08:42:42.902825+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:26:01.926948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290736+0000","last_change":"2024-04-14T08:42:39.898146+0000","last_active":"2024-04-14T08:42:43.290736+0000","last_peered":"2024-04-14T08:42:43.290736+0000","last_clean":"2024-04-14T08:42:43.290736+0000","last_became_active":"2024-04-14T08:42:39.897857+0000","last_became_peered":"2024-04-14T08:42:39.897857+0000","last_unstale":"2024-04-14T08:42:43.290736+0000","last_undegraded":"2024-04-14T08:42:43.290736+0000","last_fullsized":"2024-04-14T08:42:43.290736+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:22:11.033372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290693+0000","last_change":"2024-04-14T08:42:40.909220+0000","last_active":"2024-04-14T08:42:43.290693+0000","last_peered":"2024-04-14T08:42:43.290693+0000","last_clean":"2024-04-14T08:42:43.290693+0000","last_became_active":"2024-04-14T08:42:40.908884+0000","last_became_peered":"2024-04-14T08:42:40.908884+0000","last_unstale":"2024-04-14T08:42:43.290693+0000","last_undegraded":"2024-04-14T08:42:43.290693+0000","last_fullsized":"2024-04-14T08:42:43.290693+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:42:53.101473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"115'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.920415+0000","last_change":"2024-04-14T08:42:39.903458+0000","last_active":"2024-04-14T08:42:40.920415+0000","last_peered":"2024-04-14T08:42:40.920415+0000","last_clean":"2024-04-14T08:42:40.920415+0000","last_became_active":"2024-04-14T08:42:39.903032+0000","last_became_peered":"2024-04-14T08:42:39.903032+0000","last_unstale":"2024-04-14T08:42:40.920415+0000","last_undegraded":"2024-04-14T08:42:40.920415+0000","last_fullsized":"2024-04-14T08:42:40.920415+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:43.408088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.908622+0000","last_change":"2024-04-14T08:42:40.904383+0000","last_active":"2024-04-14T08:42:40.908622+0000","last_peered":"2024-04-14T08:42:40.908622+0000","last_clean":"2024-04-14T08:42:40.908622+0000","last_became_active":"2024-04-14T08:42:40.904099+0000","last_became_peered":"2024-04-14T08:42:40.904099+0000","last_unstale":"2024-04-14T08:42:40.908622+0000","last_undegraded":"2024-04-14T08:42:40.908622+0000","last_fullsized":"2024-04-14T08:42:40.908622+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:37:01.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290787+0000","last_change":"2024-04-14T08:42:39.914891+0000","last_active":"2024-04-14T08:42:43.290787+0000","last_peered":"2024-04-14T08:42:43.290787+0000","last_clean":"2024-04-14T08:42:43.290787+0000","last_became_active":"2024-04-14T08:42:39.914617+0000","last_became_peered":"2024-04-14T08:42:39.914617+0000","last_unstale":"2024-04-14T08:42:43.290787+0000","last_undegraded":"2024-04-14T08:42:43.290787+0000","last_fullsized":"2024-04-14T08:42:43.290787+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:20:54.450691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290724+0000","last_change":"2024-04-14T08:42:40.912016+0000","last_active":"2024-04-14T08:42:43.290724+0000","last_peered":"2024-04-14T08:42:43.290724+0000","last_clean":"2024-04-14T08:42:43.290724+0000","last_became_active":"2024-04-14T08:42:40.911749+0000","last_became_peered":"2024-04-14T08:42:40.911749+0000","last_unstale":"2024-04-14T08:42:43.290724+0000","last_undegraded":"2024-04-14T08:42:43.290724+0000","last_fullsized":"2024-04-14T08:42:43.290724+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:44.417457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915101+0000","last_change":"2024-04-14T08:42:39.898512+0000","last_active":"2024-04-14T08:42:41.915101+0000","last_peered":"2024-04-14T08:42:41.915101+0000","last_clean":"2024-04-14T08:42:41.915101+0000","last_became_active":"2024-04-14T08:42:39.898183+0000","last_became_peered":"2024-04-14T08:42:39.898183+0000","last_unstale":"2024-04-14T08:42:41.915101+0000","last_undegraded":"2024-04-14T08:42:41.915101+0000","last_fullsized":"2024-04-14T08:42:41.915101+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:07:34.836827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915435+0000","last_change":"2024-04-14T08:42:40.900777+0000","last_active":"2024-04-14T08:42:41.915435+0000","last_peered":"2024-04-14T08:42:41.915435+0000","last_clean":"2024-04-14T08:42:41.915435+0000","last_became_active":"2024-04-14T08:42:40.900462+0000","last_became_peered":"2024-04-14T08:42:40.900462+0000","last_unstale":"2024-04-14T08:42:41.915435+0000","last_undegraded":"2024-04-14T08:42:41.915435+0000","last_fullsized":"2024-04-14T08:42:41.915435+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:50:25.331351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902512+0000","last_change":"2024-04-14T08:42:39.915147+0000","last_active":"2024-04-14T08:42:42.902512+0000","last_peered":"2024-04-14T08:42:42.902512+0000","last_clean":"2024-04-14T08:42:42.902512+0000","last_became_active":"2024-04-14T08:42:39.914929+0000","last_became_peered":"2024-04-14T08:42:39.914929+0000","last_unstale":"2024-04-14T08:42:42.902512+0000","last_undegraded":"2024-04-14T08:42:42.902512+0000","last_fullsized":"2024-04-14T08:42:42.902512+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:41:14.539282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909526+0000","last_change":"2024-04-14T08:42:40.924817+0000","last_active":"2024-04-14T08:42:41.909526+0000","last_peered":"2024-04-14T08:42:41.909526+0000","last_clean":"2024-04-14T08:42:41.909526+0000","last_became_active":"2024-04-14T08:42:40.924440+0000","last_became_peered":"2024-04-14T08:42:40.924440+0000","last_unstale":"2024-04-14T08:42:41.909526+0000","last_undegraded":"2024-04-14T08:42:41.909526+0000","last_fullsized":"2024-04-14T08:42:41.909526+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:06:35.189673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290565+0000","last_change":"2024-04-14T08:42:39.904121+0000","last_active":"2024-04-14T08:42:43.290565+0000","last_peered":"2024-04-14T08:42:43.290565+0000","last_clean":"2024-04-14T08:42:43.290565+0000","last_became_active":"2024-04-14T08:42:39.903879+0000","last_became_peered":"2024-04-14T08:42:39.903879+0000","last_unstale":"2024-04-14T08:42:43.290565+0000","last_undegraded":"2024-04-14T08:42:43.290565+0000","last_fullsized":"2024-04-14T08:42:43.290565+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:42:20.871047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290906+0000","last_change":"2024-04-14T08:42:40.924009+0000","last_active":"2024-04-14T08:42:43.290906+0000","last_peered":"2024-04-14T08:42:43.290906+0000","last_clean":"2024-04-14T08:42:43.290906+0000","last_became_active":"2024-04-14T08:42:40.923601+0000","last_became_peered":"2024-04-14T08:42:40.923601+0000","last_unstale":"2024-04-14T08:42:43.290906+0000","last_undegraded":"2024-04-14T08:42:43.290906+0000","last_fullsized":"2024-04-14T08:42:43.290906+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:51.037275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902524+0000","last_change":"2024-04-14T08:42:39.900795+0000","last_active":"2024-04-14T08:42:42.902524+0000","last_peered":"2024-04-14T08:42:42.902524+0000","last_clean":"2024-04-14T08:42:42.902524+0000","last_became_active":"2024-04-14T08:42:39.900345+0000","last_became_peered":"2024-04-14T08:42:39.900345+0000","last_unstale":"2024-04-14T08:42:42.902524+0000","last_undegraded":"2024-04-14T08:42:42.902524+0000","last_fullsized":"2024-04-14T08:42:42.902524+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:22:52.184492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900153+0000","last_change":"2024-04-14T08:42:40.907871+0000","last_active":"2024-04-14T08:42:42.900153+0000","last_peered":"2024-04-14T08:42:42.900153+0000","last_clean":"2024-04-14T08:42:42.900153+0000","last_became_active":"2024-04-14T08:42:40.907553+0000","last_became_peered":"2024-04-14T08:42:40.907553+0000","last_unstale":"2024-04-14T08:42:42.900153+0000","last_undegraded":"2024-04-14T08:42:42.900153+0000","last_fullsized":"2024-04-14T08:42:42.900153+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:58:26.274235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900224+0000","last_change":"2024-04-14T08:42:39.908268+0000","last_active":"2024-04-14T08:42:42.900224+0000","last_peered":"2024-04-14T08:42:42.900224+0000","last_clean":"2024-04-14T08:42:42.900224+0000","last_became_active":"2024-04-14T08:42:39.907971+0000","last_became_peered":"2024-04-14T08:42:39.907971+0000","last_unstale":"2024-04-14T08:42:42.900224+0000","last_undegraded":"2024-04-14T08:42:42.900224+0000","last_fullsized":"2024-04-14T08:42:42.900224+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:47:33.457508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290674+0000","last_change":"2024-04-14T08:42:40.919109+0000","last_active":"2024-04-14T08:42:43.290674+0000","last_peered":"2024-04-14T08:42:43.290674+0000","last_clean":"2024-04-14T08:42:43.290674+0000","last_became_active":"2024-04-14T08:42:40.918802+0000","last_became_peered":"2024-04-14T08:42:40.918802+0000","last_unstale":"2024-04-14T08:42:43.290674+0000","last_undegraded":"2024-04-14T08:42:43.290674+0000","last_fullsized":"2024-04-14T08:42:43.290674+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:57.005587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290681+0000","last_change":"2024-04-14T08:42:39.911711+0000","last_active":"2024-04-14T08:42:43.290681+0000","last_peered":"2024-04-14T08:42:43.290681+0000","last_clean":"2024-04-14T08:42:43.290681+0000","last_became_active":"2024-04-14T08:42:39.911408+0000","last_became_peered":"2024-04-14T08:42:39.911408+0000","last_unstale":"2024-04-14T08:42:43.290681+0000","last_undegraded":"2024-04-14T08:42:43.290681+0000","last_fullsized":"2024-04-14T08:42:43.290681+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:35:14.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909427+0000","last_change":"2024-04-14T08:42:40.915228+0000","last_active":"2024-04-14T08:42:41.909427+0000","last_peered":"2024-04-14T08:42:41.909427+0000","last_clean":"2024-04-14T08:42:41.909427+0000","last_became_active":"2024-04-14T08:42:40.914332+0000","last_became_peered":"2024-04-14T08:42:40.914332+0000","last_unstale":"2024-04-14T08:42:41.909427+0000","last_undegraded":"2024-04-14T08:42:41.909427+0000","last_fullsized":"2024-04-14T08:42:41.909427+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:13:59.521980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"115'3","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290711+0000","last_change":"2024-04-14T08:42:39.906942+0000","last_active":"2024-04-14T08:42:43.290711+0000","last_peered":"2024-04-14T08:42:43.290711+0000","last_clean":"2024-04-14T08:42:43.290711+0000","last_became_active":"2024-04-14T08:42:39.906650+0000","last_became_peered":"2024-04-14T08:42:39.906650+0000","last_unstale":"2024-04-14T08:42:43.290711+0000","last_undegraded":"2024-04-14T08:42:43.290711+0000","last_fullsized":"2024-04-14T08:42:43.290711+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:07:34.836827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290863+0000","last_change":"2024-04-14T08:42:40.920432+0000","last_active":"2024-04-14T08:42:43.290863+0000","last_peered":"2024-04-14T08:42:43.290863+0000","last_clean":"2024-04-14T08:42:43.290863+0000","last_became_active":"2024-04-14T08:42:40.920133+0000","last_became_peered":"2024-04-14T08:42:40.920133+0000","last_unstale":"2024-04-14T08:42:43.290863+0000","last_undegraded":"2024-04-14T08:42:43.290863+0000","last_fullsized":"2024-04-14T08:42:43.290863+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:28.290311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"115'6","reported_seq":20,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909195+0000","last_change":"2024-04-14T08:42:39.917311+0000","last_active":"2024-04-14T08:42:41.909195+0000","last_peered":"2024-04-14T08:42:41.909195+0000","last_clean":"2024-04-14T08:42:41.909195+0000","last_became_active":"2024-04-14T08:42:39.917001+0000","last_became_peered":"2024-04-14T08:42:39.917001+0000","last_unstale":"2024-04-14T08:42:41.909195+0000","last_undegraded":"2024-04-14T08:42:41.909195+0000","last_fullsized":"2024-04-14T08:42:41.909195+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:58:25.264866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900171+0000","last_change":"2024-04-14T08:42:40.917029+0000","last_active":"2024-04-14T08:42:42.900171+0000","last_peered":"2024-04-14T08:42:42.900171+0000","last_clean":"2024-04-14T08:42:42.900171+0000","last_became_active":"2024-04-14T08:42:40.916763+0000","last_became_peered":"2024-04-14T08:42:40.916763+0000","last_unstale":"2024-04-14T08:42:42.900171+0000","last_undegraded":"2024-04-14T08:42:42.900171+0000","last_fullsized":"2024-04-14T08:42:42.900171+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:30.073046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909128+0000","last_change":"2024-04-14T08:42:39.914644+0000","last_active":"2024-04-14T08:42:41.909128+0000","last_peered":"2024-04-14T08:42:41.909128+0000","last_clean":"2024-04-14T08:42:41.909128+0000","last_became_active":"2024-04-14T08:42:39.914350+0000","last_became_peered":"2024-04-14T08:42:39.914350+0000","last_unstale":"2024-04-14T08:42:41.909128+0000","last_undegraded":"2024-04-14T08:42:41.909128+0000","last_fullsized":"2024-04-14T08:42:41.909128+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:29.063677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290471+0000","last_change":"2024-04-14T08:42:40.910224+0000","last_active":"2024-04-14T08:42:43.290471+0000","last_peered":"2024-04-14T08:42:43.290471+0000","last_clean":"2024-04-14T08:42:43.290471+0000","last_became_active":"2024-04-14T08:42:40.909934+0000","last_became_peered":"2024-04-14T08:42:40.909934+0000","last_unstale":"2024-04-14T08:42:43.290471+0000","last_undegraded":"2024-04-14T08:42:43.290471+0000","last_fullsized":"2024-04-14T08:42:43.290471+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:45:57.742439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909634+0000","last_change":"2024-04-14T08:42:39.914644+0000","last_active":"2024-04-14T08:42:41.909634+0000","last_peered":"2024-04-14T08:42:41.909634+0000","last_clean":"2024-04-14T08:42:41.909634+0000","last_became_active":"2024-04-14T08:42:39.914387+0000","last_became_peered":"2024-04-14T08:42:39.914387+0000","last_unstale":"2024-04-14T08:42:41.909634+0000","last_undegraded":"2024-04-14T08:42:41.909634+0000","last_fullsized":"2024-04-14T08:42:41.909634+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:56.274277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290552+0000","last_change":"2024-04-14T08:42:40.900265+0000","last_active":"2024-04-14T08:42:43.290552+0000","last_peered":"2024-04-14T08:42:43.290552+0000","last_clean":"2024-04-14T08:42:43.290552+0000","last_became_active":"2024-04-14T08:42:40.899538+0000","last_became_peered":"2024-04-14T08:42:40.899538+0000","last_unstale":"2024-04-14T08:42:43.290552+0000","last_undegraded":"2024-04-14T08:42:43.290552+0000","last_fullsized":"2024-04-14T08:42:43.290552+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:52:35.491912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909109+0000","last_change":"2024-04-14T08:42:39.917235+0000","last_active":"2024-04-14T08:42:41.909109+0000","last_peered":"2024-04-14T08:42:41.909109+0000","last_clean":"2024-04-14T08:42:41.909109+0000","last_became_active":"2024-04-14T08:42:39.916937+0000","last_became_peered":"2024-04-14T08:42:39.916937+0000","last_unstale":"2024-04-14T08:42:41.909109+0000","last_undegraded":"2024-04-14T08:42:41.909109+0000","last_fullsized":"2024-04-14T08:42:41.909109+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:48:28.600717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290659+0000","last_change":"2024-04-14T08:42:40.914873+0000","last_active":"2024-04-14T08:42:43.290659+0000","last_peered":"2024-04-14T08:42:43.290659+0000","last_clean":"2024-04-14T08:42:43.290659+0000","last_became_active":"2024-04-14T08:42:40.914598+0000","last_became_peered":"2024-04-14T08:42:40.914598+0000","last_unstale":"2024-04-14T08:42:43.290659+0000","last_undegraded":"2024-04-14T08:42:43.290659+0000","last_fullsized":"2024-04-14T08:42:43.290659+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:37:01.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290857+0000","last_change":"2024-04-14T08:42:39.907928+0000","last_active":"2024-04-14T08:42:43.290857+0000","last_peered":"2024-04-14T08:42:43.290857+0000","last_clean":"2024-04-14T08:42:43.290857+0000","last_became_active":"2024-04-14T08:42:39.907662+0000","last_became_peered":"2024-04-14T08:42:39.907662+0000","last_unstale":"2024-04-14T08:42:43.290857+0000","last_undegraded":"2024-04-14T08:42:43.290857+0000","last_fullsized":"2024-04-14T08:42:43.290857+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:32:40.235395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914775+0000","last_change":"2024-04-14T08:42:40.922938+0000","last_active":"2024-04-14T08:42:41.914775+0000","last_peered":"2024-04-14T08:42:41.914775+0000","last_clean":"2024-04-14T08:42:41.914775+0000","last_became_active":"2024-04-14T08:42:40.922638+0000","last_became_peered":"2024-04-14T08:42:40.922638+0000","last_unstale":"2024-04-14T08:42:41.914775+0000","last_undegraded":"2024-04-14T08:42:41.914775+0000","last_fullsized":"2024-04-14T08:42:41.914775+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:37:01.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.291173+0000","last_change":"2024-04-14T08:42:39.903548+0000","last_active":"2024-04-14T08:42:43.291173+0000","last_peered":"2024-04-14T08:42:43.291173+0000","last_clean":"2024-04-14T08:42:43.291173+0000","last_became_active":"2024-04-14T08:42:39.903272+0000","last_became_peered":"2024-04-14T08:42:39.903272+0000","last_unstale":"2024-04-14T08:42:43.291173+0000","last_undegraded":"2024-04-14T08:42:43.291173+0000","last_fullsized":"2024-04-14T08:42:43.291173+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:27.935436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902883+0000","last_change":"2024-04-14T08:42:40.913520+0000","last_active":"2024-04-14T08:42:42.902883+0000","last_peered":"2024-04-14T08:42:42.902883+0000","last_clean":"2024-04-14T08:42:42.902883+0000","last_became_active":"2024-04-14T08:42:40.913220+0000","last_became_peered":"2024-04-14T08:42:40.913220+0000","last_unstale":"2024-04-14T08:42:42.902883+0000","last_undegraded":"2024-04-14T08:42:42.902883+0000","last_fullsized":"2024-04-14T08:42:42.902883+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:28:20.358223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"115'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.922186+0000","last_change":"2024-04-14T08:42:39.907861+0000","last_active":"2024-04-14T08:42:40.922186+0000","last_peered":"2024-04-14T08:42:40.922186+0000","last_clean":"2024-04-14T08:42:40.922186+0000","last_became_active":"2024-04-14T08:42:39.907144+0000","last_became_peered":"2024-04-14T08:42:39.907144+0000","last_unstale":"2024-04-14T08:42:40.922186+0000","last_undegraded":"2024-04-14T08:42:40.922186+0000","last_fullsized":"2024-04-14T08:42:40.922186+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:50.027906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290893+0000","last_change":"2024-04-14T08:42:40.911807+0000","last_active":"2024-04-14T08:42:43.290893+0000","last_peered":"2024-04-14T08:42:43.290893+0000","last_clean":"2024-04-14T08:42:43.290893+0000","last_became_active":"2024-04-14T08:42:40.911503+0000","last_became_peered":"2024-04-14T08:42:40.911503+0000","last_unstale":"2024-04-14T08:42:43.290893+0000","last_undegraded":"2024-04-14T08:42:43.290893+0000","last_fullsized":"2024-04-14T08:42:43.290893+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:53:03.886099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.895697+0000","last_change":"2024-04-14T08:42:39.900873+0000","last_active":"2024-04-14T08:42:40.895697+0000","last_peered":"2024-04-14T08:42:40.895697+0000","last_clean":"2024-04-14T08:42:40.895697+0000","last_became_active":"2024-04-14T08:42:39.900603+0000","last_became_peered":"2024-04-14T08:42:39.900603+0000","last_unstale":"2024-04-14T08:42:40.895697+0000","last_undegraded":"2024-04-14T08:42:40.895697+0000","last_fullsized":"2024-04-14T08:42:40.895697+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:29.063677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902378+0000","last_change":"2024-04-14T08:42:40.914655+0000","last_active":"2024-04-14T08:42:42.902378+0000","last_peered":"2024-04-14T08:42:42.902378+0000","last_clean":"2024-04-14T08:42:42.902378+0000","last_became_active":"2024-04-14T08:42:40.914378+0000","last_became_peered":"2024-04-14T08:42:40.914378+0000","last_unstale":"2024-04-14T08:42:42.902378+0000","last_undegraded":"2024-04-14T08:42:42.902378+0000","last_fullsized":"2024-04-14T08:42:42.902378+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:35:09.462995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902387+0000","last_change":"2024-04-14T08:42:39.904952+0000","last_active":"2024-04-14T08:42:42.902387+0000","last_peered":"2024-04-14T08:42:42.902387+0000","last_clean":"2024-04-14T08:42:42.902387+0000","last_became_active":"2024-04-14T08:42:39.904677+0000","last_became_peered":"2024-04-14T08:42:39.904677+0000","last_unstale":"2024-04-14T08:42:42.902387+0000","last_undegraded":"2024-04-14T08:42:42.902387+0000","last_fullsized":"2024-04-14T08:42:42.902387+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:05:14.219045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.908978+0000","last_change":"2024-04-14T08:42:40.915072+0000","last_active":"2024-04-14T08:42:41.908978+0000","last_peered":"2024-04-14T08:42:41.908978+0000","last_clean":"2024-04-14T08:42:41.908978+0000","last_became_active":"2024-04-14T08:42:40.913471+0000","last_became_peered":"2024-04-14T08:42:40.913471+0000","last_unstale":"2024-04-14T08:42:41.908978+0000","last_undegraded":"2024-04-14T08:42:41.908978+0000","last_fullsized":"2024-04-14T08:42:41.908978+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:53:03.886099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.894138+0000","last_change":"2024-04-14T08:42:39.902053+0000","last_active":"2024-04-14T08:42:40.894138+0000","last_peered":"2024-04-14T08:42:40.894138+0000","last_clean":"2024-04-14T08:42:40.894138+0000","last_became_active":"2024-04-14T08:42:39.901709+0000","last_became_peered":"2024-04-14T08:42:39.901709+0000","last_unstale":"2024-04-14T08:42:40.894138+0000","last_undegraded":"2024-04-14T08:42:40.894138+0000","last_fullsized":"2024-04-14T08:42:40.894138+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:58:25.264866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909617+0000","last_change":"2024-04-14T08:42:40.915770+0000","last_active":"2024-04-14T08:42:41.909617+0000","last_peered":"2024-04-14T08:42:41.909617+0000","last_clean":"2024-04-14T08:42:41.909617+0000","last_became_active":"2024-04-14T08:42:40.914328+0000","last_became_peered":"2024-04-14T08:42:40.914328+0000","last_unstale":"2024-04-14T08:42:41.909617+0000","last_undegraded":"2024-04-14T08:42:41.909617+0000","last_fullsized":"2024-04-14T08:42:41.909617+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:38:12.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909161+0000","last_change":"2024-04-14T08:42:39.917517+0000","last_active":"2024-04-14T08:42:41.909161+0000","last_peered":"2024-04-14T08:42:41.909161+0000","last_clean":"2024-04-14T08:42:41.909161+0000","last_became_active":"2024-04-14T08:42:39.917246+0000","last_became_peered":"2024-04-14T08:42:39.917246+0000","last_unstale":"2024-04-14T08:42:41.909161+0000","last_undegraded":"2024-04-14T08:42:41.909161+0000","last_fullsized":"2024-04-14T08:42:41.909161+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:45:56.733070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290698+0000","last_change":"2024-04-14T08:42:40.909187+0000","last_active":"2024-04-14T08:42:43.290698+0000","last_peered":"2024-04-14T08:42:43.290698+0000","last_clean":"2024-04-14T08:42:43.290698+0000","last_became_active":"2024-04-14T08:42:40.908912+0000","last_became_peered":"2024-04-14T08:42:40.908912+0000","last_unstale":"2024-04-14T08:42:43.290698+0000","last_undegraded":"2024-04-14T08:42:43.290698+0000","last_fullsized":"2024-04-14T08:42:43.290698+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:50:25.331351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.893438+0000","last_change":"2024-04-14T08:42:39.903314+0000","last_active":"2024-04-14T08:42:40.893438+0000","last_peered":"2024-04-14T08:42:40.893438+0000","last_clean":"2024-04-14T08:42:40.893438+0000","last_became_active":"2024-04-14T08:42:39.903044+0000","last_became_peered":"2024-04-14T08:42:39.903044+0000","last_unstale":"2024-04-14T08:42:40.893438+0000","last_undegraded":"2024-04-14T08:42:40.893438+0000","last_fullsized":"2024-04-14T08:42:40.893438+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:32:34.297541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902969+0000","last_change":"2024-04-14T08:42:40.917142+0000","last_active":"2024-04-14T08:42:42.902969+0000","last_peered":"2024-04-14T08:42:42.902969+0000","last_clean":"2024-04-14T08:42:42.902969+0000","last_became_active":"2024-04-14T08:42:40.915276+0000","last_became_peered":"2024-04-14T08:42:40.915276+0000","last_unstale":"2024-04-14T08:42:42.902969+0000","last_undegraded":"2024-04-14T08:42:42.902969+0000","last_fullsized":"2024-04-14T08:42:42.902969+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:38:34.850919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902941+0000","last_change":"2024-04-14T08:42:39.899765+0000","last_active":"2024-04-14T08:42:42.902941+0000","last_peered":"2024-04-14T08:42:42.902941+0000","last_clean":"2024-04-14T08:42:42.902941+0000","last_became_active":"2024-04-14T08:42:39.898985+0000","last_became_peered":"2024-04-14T08:42:39.898985+0000","last_unstale":"2024-04-14T08:42:42.902941+0000","last_undegraded":"2024-04-14T08:42:42.902941+0000","last_fullsized":"2024-04-14T08:42:42.902941+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:01:57.414126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290731+0000","last_change":"2024-04-14T08:42:40.912641+0000","last_active":"2024-04-14T08:42:43.290731+0000","last_peered":"2024-04-14T08:42:43.290731+0000","last_clean":"2024-04-14T08:42:43.290731+0000","last_became_active":"2024-04-14T08:42:40.912388+0000","last_became_peered":"2024-04-14T08:42:40.912388+0000","last_unstale":"2024-04-14T08:42:43.290731+0000","last_undegraded":"2024-04-14T08:42:43.290731+0000","last_fullsized":"2024-04-14T08:42:43.290731+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:58:26.274235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"116'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900440+0000","last_change":"2024-04-14T08:42:39.908137+0000","last_active":"2024-04-14T08:42:42.900440+0000","last_peered":"2024-04-14T08:42:42.900440+0000","last_clean":"2024-04-14T08:42:42.900440+0000","last_became_active":"2024-04-14T08:42:39.907857+0000","last_became_peered":"2024-04-14T08:42:39.907857+0000","last_unstale":"2024-04-14T08:42:42.900440+0000","last_undegraded":"2024-04-14T08:42:42.900440+0000","last_fullsized":"2024-04-14T08:42:42.900440+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:48:28.600717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.910001+0000","last_change":"2024-04-14T08:42:40.904731+0000","last_active":"2024-04-14T08:42:40.910001+0000","last_peered":"2024-04-14T08:42:40.910001+0000","last_clean":"2024-04-14T08:42:40.910001+0000","last_became_active":"2024-04-14T08:42:40.904169+0000","last_became_peered":"2024-04-14T08:42:40.904169+0000","last_unstale":"2024-04-14T08:42:40.910001+0000","last_undegraded":"2024-04-14T08:42:40.910001+0000","last_fullsized":"2024-04-14T08:42:40.910001+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:40.822681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900334+0000","last_change":"2024-04-14T08:42:39.907401+0000","last_active":"2024-04-14T08:42:42.900334+0000","last_peered":"2024-04-14T08:42:42.900334+0000","last_clean":"2024-04-14T08:42:42.900334+0000","last_became_active":"2024-04-14T08:42:39.907124+0000","last_became_peered":"2024-04-14T08:42:39.907124+0000","last_unstale":"2024-04-14T08:42:42.900334+0000","last_undegraded":"2024-04-14T08:42:42.900334+0000","last_fullsized":"2024-04-14T08:42:42.900334+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:41.656802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.291258+0000","last_change":"2024-04-14T08:42:40.911392+0000","last_active":"2024-04-14T08:42:43.291258+0000","last_peered":"2024-04-14T08:42:43.291258+0000","last_clean":"2024-04-14T08:42:43.291258+0000","last_became_active":"2024-04-14T08:42:40.908865+0000","last_became_peered":"2024-04-14T08:42:40.908865+0000","last_unstale":"2024-04-14T08:42:43.291258+0000","last_undegraded":"2024-04-14T08:42:43.291258+0000","last_fullsized":"2024-04-14T08:42:43.291258+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:02:00.666186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.903031+0000","last_change":"2024-04-14T08:42:39.901863+0000","last_active":"2024-04-14T08:42:42.903031+0000","last_peered":"2024-04-14T08:42:42.903031+0000","last_clean":"2024-04-14T08:42:42.903031+0000","last_became_active":"2024-04-14T08:42:39.901612+0000","last_became_peered":"2024-04-14T08:42:39.901612+0000","last_unstale":"2024-04-14T08:42:42.903031+0000","last_undegraded":"2024-04-14T08:42:42.903031+0000","last_fullsized":"2024-04-14T08:42:42.903031+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:15.161276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902590+0000","last_change":"2024-04-14T08:42:40.906995+0000","last_active":"2024-04-14T08:42:42.902590+0000","last_peered":"2024-04-14T08:42:42.902590+0000","last_clean":"2024-04-14T08:42:42.902590+0000","last_became_active":"2024-04-14T08:42:40.906684+0000","last_became_peered":"2024-04-14T08:42:40.906684+0000","last_unstale":"2024-04-14T08:42:42.902590+0000","last_undegraded":"2024-04-14T08:42:42.902590+0000","last_fullsized":"2024-04-14T08:42:42.902590+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:19:18.732652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915157+0000","last_change":"2024-04-14T08:42:39.898079+0000","last_active":"2024-04-14T08:42:41.915157+0000","last_peered":"2024-04-14T08:42:41.915157+0000","last_clean":"2024-04-14T08:42:41.915157+0000","last_became_active":"2024-04-14T08:42:39.897776+0000","last_became_peered":"2024-04-14T08:42:39.897776+0000","last_unstale":"2024-04-14T08:42:41.915157+0000","last_undegraded":"2024-04-14T08:42:41.915157+0000","last_fullsized":"2024-04-14T08:42:41.915157+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:25:26.756136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909020+0000","last_change":"2024-04-14T08:42:40.913761+0000","last_active":"2024-04-14T08:42:41.909020+0000","last_peered":"2024-04-14T08:42:41.909020+0000","last_clean":"2024-04-14T08:42:41.909020+0000","last_became_active":"2024-04-14T08:42:40.913125+0000","last_became_peered":"2024-04-14T08:42:40.913125+0000","last_unstale":"2024-04-14T08:42:41.909020+0000","last_undegraded":"2024-04-14T08:42:41.909020+0000","last_fullsized":"2024-04-14T08:42:41.909020+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:01:17.366719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914168+0000","last_change":"2024-04-14T08:42:39.907024+0000","last_active":"2024-04-14T08:42:41.914168+0000","last_peered":"2024-04-14T08:42:41.914168+0000","last_clean":"2024-04-14T08:42:41.914168+0000","last_became_active":"2024-04-14T08:42:39.906749+0000","last_became_peered":"2024-04-14T08:42:39.906749+0000","last_unstale":"2024-04-14T08:42:41.914168+0000","last_undegraded":"2024-04-14T08:42:41.914168+0000","last_fullsized":"2024-04-14T08:42:41.914168+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:41.656802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.914252+0000","last_change":"2024-04-14T08:42:40.909631+0000","last_active":"2024-04-14T08:42:40.914252+0000","last_peered":"2024-04-14T08:42:40.914252+0000","last_clean":"2024-04-14T08:42:40.914252+0000","last_became_active":"2024-04-14T08:42:40.909358+0000","last_became_peered":"2024-04-14T08:42:40.909358+0000","last_unstale":"2024-04-14T08:42:40.914252+0000","last_undegraded":"2024-04-14T08:42:40.914252+0000","last_fullsized":"2024-04-14T08:42:40.914252+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:56.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902433+0000","last_change":"2024-04-14T08:42:39.899535+0000","last_active":"2024-04-14T08:42:42.902433+0000","last_peered":"2024-04-14T08:42:42.902433+0000","last_clean":"2024-04-14T08:42:42.902433+0000","last_became_active":"2024-04-14T08:42:39.899245+0000","last_became_peered":"2024-04-14T08:42:39.899245+0000","last_unstale":"2024-04-14T08:42:42.902433+0000","last_undegraded":"2024-04-14T08:42:42.902433+0000","last_fullsized":"2024-04-14T08:42:42.902433+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:17:13.730626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290960+0000","last_change":"2024-04-14T08:42:40.917261+0000","last_active":"2024-04-14T08:42:43.290960+0000","last_peered":"2024-04-14T08:42:43.290960+0000","last_clean":"2024-04-14T08:42:43.290960+0000","last_became_active":"2024-04-14T08:42:40.916998+0000","last_became_peered":"2024-04-14T08:42:40.916998+0000","last_unstale":"2024-04-14T08:42:43.290960+0000","last_undegraded":"2024-04-14T08:42:43.290960+0000","last_fullsized":"2024-04-14T08:42:43.290960+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:57.005587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.893537+0000","last_change":"2024-04-14T08:42:39.903168+0000","last_active":"2024-04-14T08:42:40.893537+0000","last_peered":"2024-04-14T08:42:40.893537+0000","last_clean":"2024-04-14T08:42:40.893537+0000","last_became_active":"2024-04-14T08:42:39.902874+0000","last_became_peered":"2024-04-14T08:42:39.902874+0000","last_unstale":"2024-04-14T08:42:40.893537+0000","last_undegraded":"2024-04-14T08:42:40.893537+0000","last_fullsized":"2024-04-14T08:42:40.893537+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:34:54.764802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.916751+0000","last_change":"2024-04-14T08:42:40.913643+0000","last_active":"2024-04-14T08:42:40.916751+0000","last_peered":"2024-04-14T08:42:40.916751+0000","last_clean":"2024-04-14T08:42:40.916751+0000","last_became_active":"2024-04-14T08:42:40.912975+0000","last_became_peered":"2024-04-14T08:42:40.912975+0000","last_unstale":"2024-04-14T08:42:40.916751+0000","last_undegraded":"2024-04-14T08:42:40.916751+0000","last_fullsized":"2024-04-14T08:42:40.916751+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:28.290311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"115'8","reported_seq":22,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909001+0000","last_change":"2024-04-14T08:42:39.915077+0000","last_active":"2024-04-14T08:42:41.909001+0000","last_peered":"2024-04-14T08:42:41.909001+0000","last_clean":"2024-04-14T08:42:41.909001+0000","last_became_active":"2024-04-14T08:42:39.914803+0000","last_became_peered":"2024-04-14T08:42:39.914803+0000","last_unstale":"2024-04-14T08:42:41.909001+0000","last_undegraded":"2024-04-14T08:42:41.909001+0000","last_fullsized":"2024-04-14T08:42:41.909001+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:47:33.457508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":362,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-14T08:42:40.893578+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:42:40.893578+0000","last_peered":"2024-04-14T08:42:40.893578+0000","last_clean":"2024-04-14T08:42:40.893578+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:42:40.893578+0000","last_undegraded":"2024-04-14T08:42:40.893578+0000","last_fullsized":"2024-04-14T08:42:40.893578+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2015232,"data_stored":1250329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574972,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1449984,"data_stored":674402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574972,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":673980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574972,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1458176,"data_stored":675432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574972,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":675466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574972,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1445888,"data_stored":673820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1449984,"data_stored":673842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574972,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33176,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338664,"statfs":{"total":96636764160,"available":96602791936,"internally_reserved":0,"allocated":2043904,"data_stored":1264210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:42:46.164 INFO:tasks.ceph.mds.d.smithi177.stderr:2024-04-14T08:42:46.163+0000 7f665cdf1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:42:48.346 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:42:48.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.459+0000 7f21d17c8640 1 Processor -- start 2024-04-14T08:42:48.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.459+0000 7f21d17c8640 1 -- start start 2024-04-14T08:42:48.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21d17c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21cc0cedd0 0x7f21cc1694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21d17c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21cc0cf320 con 0x7f21cc0d2830 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21d17c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21cc0cf600 con 0x7f21cc15e890 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21d17c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f21cc0cff30 con 0x7f21cc0cedd0 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21caffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f21cc0d2830 0x7f21cc0d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34226/0 (socket says 172.21.15.172:34226) 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21caffd640 1 -- 172.21.15.172:0/932239305 learned_addr learned my addr 172.21.15.172:0/932239305 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21cb7fe640 1 --2- 172.21.15.172:0/932239305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21cc0cedd0 0x7f21cc1694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21ca7fc640 1 -- 172.21.15.172:0/932239305 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 216041128 0 0) 0x7f21cc0cf320 con 0x7f21cc0d2830 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21ca7fc640 1 -- 172.21.15.172:0/932239305 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21b8003520 con 0x7f21cc0d2830 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21ca7fc640 1 -- 172.21.15.172:0/932239305 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 768479736 0 0) 0x7f21cc0cf600 con 0x7f21cc15e890 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21ca7fc640 1 -- 172.21.15.172:0/932239305 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21cc0cf320 con 0x7f21cc15e890 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21cb7fe640 1 -- 172.21.15.172:0/932239305 >> v1:172.21.15.177:6789/0 conn(0x7f21cc15e890 legacy=0x7f21cc0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21cb7fe640 1 -- 172.21.15.172:0/932239305 >> v1:172.21.15.172:6789/0 conn(0x7f21cc0d2830 legacy=0x7f21cc0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:48.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.460+0000 7f21cb7fe640 1 -- 172.21.15.172:0/932239305 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc0cf8e0 con 0x7f21cc0cedd0 2024-04-14T08:42:48.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.461+0000 7f21cb7fe640 1 --2- 172.21.15.172:0/932239305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21cc0cedd0 0x7f21cc1694d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009dc0 tx=0x7f21c402bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32508e94443feb56 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:48.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.461+0000 7f21ca7fc640 1 -- 172.21.15.172:0/932239305 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4039070 con 0x7f21cc0cedd0 2024-04-14T08:42:48.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.461+0000 7f21ca7fc640 1 -- 172.21.15.172:0/932239305 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c402cd50 con 0x7f21cc0cedd0 2024-04-14T08:42:48.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.461+0000 7f21ca7fc640 1 -- 172.21.15.172:0/932239305 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4034640 con 0x7f21cc0cedd0 2024-04-14T08:42:48.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.462+0000 7f21d17c8640 1 -- 172.21.15.172:0/932239305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21cc0cedd0 msgr2=0x7f21cc1694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:48.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.462+0000 7f21d17c8640 1 --2- 172.21.15.172:0/932239305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21cc0cedd0 0x7f21cc1694d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009dc0 tx=0x7f21c402bfa0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.462+0000 7f21d17c8640 1 -- 172.21.15.172:0/932239305 shutdown_connections 2024-04-14T08:42:48.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.462+0000 7f21d17c8640 1 --2- 172.21.15.172:0/932239305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21cc0cedd0 0x7f21cc1694d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.462+0000 7f21d17c8640 1 -- 172.21.15.172:0/932239305 >> 172.21.15.172:0/932239305 conn(0x7f21cc09d590 msgr2=0x7f21cc0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:48.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.462+0000 7f21d17c8640 1 -- 172.21.15.172:0/932239305 shutdown_connections 2024-04-14T08:42:48.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.462+0000 7f21d17c8640 1 -- 172.21.15.172:0/932239305 wait complete. 2024-04-14T08:42:48.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.463+0000 7f21d17c8640 1 Processor -- start 2024-04-14T08:42:48.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.463+0000 7f21d17c8640 1 -- start start 2024-04-14T08:42:48.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21d17c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21cc0cedd0 0x7f21cc0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:48.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21d17c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21cc0d2830 0x7f21cc0b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:48.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21d17c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21cc15e890 0x7f21cc1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:48.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21d17c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc0b4720 con 0x7f21cc0cedd0 2024-04-14T08:42:48.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21d17c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc0d2350 con 0x7f21cc0d2830 2024-04-14T08:42:48.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21d17c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f21cc0b48a0 con 0x7f21cc15e890 2024-04-14T08:42:48.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21caffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21cc0cedd0 0x7f21cc0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:48.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21caffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21cc0cedd0 0x7f21cc0cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39914/0 (socket says 172.21.15.172:39914) 2024-04-14T08:42:48.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21caffd640 1 -- 172.21.15.172:0/1276742731 learned_addr learned my addr 172.21.15.172:0/1276742731 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21c27fc640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21cc0d2830 0x7f21cc0b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21cb7fe640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21cc15e890 0x7f21cc1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21caffd640 1 -- 172.21.15.172:0/1276742731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21cc15e890 msgr2=0x7f21cc1540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21caffd640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21cc15e890 0x7f21cc1540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21caffd640 1 -- 172.21.15.172:0/1276742731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21cc0d2830 msgr2=0x7f21cc0b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21caffd640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21cc0d2830 0x7f21cc0b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.464+0000 7f21caffd640 1 -- 172.21.15.172:0/1276742731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc0cac70 con 0x7f21cc0cedd0 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.465+0000 7f21caffd640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21cc0cedd0 0x7f21cc0cdc20 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00c410 tx=0x7f21bc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.465+0000 7f21c3fff640 1 -- 172.21.15.172:0/1276742731 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc017070 con 0x7f21cc0cedd0 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.465+0000 7f21c3fff640 1 -- 172.21.15.172:0/1276742731 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21bc005ce0 con 0x7f21cc0cedd0 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.465+0000 7f21d17c8640 1 -- 172.21.15.172:0/1276742731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc0ce160 con 0x7f21cc0cedd0 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.465+0000 7f21c3fff640 1 -- 172.21.15.172:0/1276742731 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc0072d0 con 0x7f21cc0cedd0 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.465+0000 7f21d17c8640 1 -- 172.21.15.172:0/1276742731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc0ca6b0 con 0x7f21cc0cedd0 2024-04-14T08:42:48.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.466+0000 7f21d17c8640 1 -- 172.21.15.172:0/1276742731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc0cb570 con 0x7f21cc0cedd0 2024-04-14T08:42:48.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.466+0000 7f21c3fff640 1 -- 172.21.15.172:0/1276742731 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21bc01a070 con 0x7f21cc0cedd0 2024-04-14T08:42:48.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.467+0000 7f21c3fff640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f21ac06c8f0 0x7f21ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:48.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.467+0000 7f21c3fff640 1 -- 172.21.15.172:0/1276742731 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f21bc08af70 con 0x7f21cc0cedd0 2024-04-14T08:42:48.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.467+0000 7f21c27fc640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f21ac06c8f0 0x7f21ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:48.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.467+0000 7f21c27fc640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f21ac06c8f0 0x7f21ac06edb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f21b4007fd0 tx=0x7f21b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:48.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.471+0000 7f21c3fff640 1 -- 172.21.15.172:0/1276742731 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f21bc058c50 con 0x7f21cc0cedd0 2024-04-14T08:42:48.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.643+0000 7f21d17c8640 1 -- 172.21.15.172:0/1276742731 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21cc068520 con 0x7f21ac06c8f0 2024-04-14T08:42:48.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.649+0000 7f21c3fff640 1 -- 172.21.15.172:0/1276742731 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252488 (secure 0 0 0) 0x7f21cc068520 con 0x7f21ac06c8f0 2024-04-14T08:42:48.652 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:48.653 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:42:48.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.652+0000 7f21d17c8640 1 -- 172.21.15.172:0/1276742731 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f21ac06c8f0 msgr2=0x7f21ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:48.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.652+0000 7f21d17c8640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f21ac06c8f0 0x7f21ac06edb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f21b4007fd0 tx=0x7f21b4009040 comp rx=0 tx=0).stop 2024-04-14T08:42:48.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.652+0000 7f21d17c8640 1 -- 172.21.15.172:0/1276742731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21cc0cedd0 msgr2=0x7f21cc0cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:48.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.652+0000 7f21d17c8640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21cc0cedd0 0x7f21cc0cdc20 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00c410 tx=0x7f21bc00c8d0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.652+0000 7f21d17c8640 1 -- 172.21.15.172:0/1276742731 shutdown_connections 2024-04-14T08:42:48.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.652+0000 7f21d17c8640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f21ac06c8f0 0x7f21ac06edb0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.652+0000 7f21d17c8640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21cc15e890 0x7f21cc1540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.652+0000 7f21d17c8640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21cc0d2830 0x7f21cc0b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.652+0000 7f21d17c8640 1 --2- 172.21.15.172:0/1276742731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21cc0cedd0 0x7f21cc0cdc20 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.652+0000 7f21d17c8640 1 -- 172.21.15.172:0/1276742731 >> 172.21.15.172:0/1276742731 conn(0x7f21cc09d590 msgr2=0x7f21cc0b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:48.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.652+0000 7f21d17c8640 1 -- 172.21.15.172:0/1276742731 shutdown_connections 2024-04-14T08:42:48.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.652+0000 7f21d17c8640 1 -- 172.21.15.172:0/1276742731 wait complete. 2024-04-14T08:42:48.663 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-04-14T08:42:47.238730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261976,"kb_used_data":12520,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712744,"statfs":{"total":773094113280,"available":772825849856,"internally_reserved":0,"allocated":12820480,"data_stored":6576116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.348203"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914421+0000","last_change":"2024-04-14T08:42:40.910371+0000","last_active":"2024-04-14T08:42:41.914421+0000","last_peered":"2024-04-14T08:42:41.914421+0000","last_clean":"2024-04-14T08:42:41.914421+0000","last_became_active":"2024-04-14T08:42:40.910102+0000","last_became_peered":"2024-04-14T08:42:40.910102+0000","last_unstale":"2024-04-14T08:42:41.914421+0000","last_undegraded":"2024-04-14T08:42:41.914421+0000","last_fullsized":"2024-04-14T08:42:41.914421+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:51.037275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909433+0000","last_change":"2024-04-14T08:42:40.919215+0000","last_active":"2024-04-14T08:42:41.909433+0000","last_peered":"2024-04-14T08:42:41.909433+0000","last_clean":"2024-04-14T08:42:41.909433+0000","last_became_active":"2024-04-14T08:42:40.918909+0000","last_became_peered":"2024-04-14T08:42:40.918909+0000","last_unstale":"2024-04-14T08:42:41.909433+0000","last_undegraded":"2024-04-14T08:42:41.909433+0000","last_fullsized":"2024-04-14T08:42:41.909433+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:38:30.115362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909385+0000","last_change":"2024-04-14T08:42:40.913610+0000","last_active":"2024-04-14T08:42:41.909385+0000","last_peered":"2024-04-14T08:42:41.909385+0000","last_clean":"2024-04-14T08:42:41.909385+0000","last_became_active":"2024-04-14T08:42:40.913312+0000","last_became_peered":"2024-04-14T08:42:40.913312+0000","last_unstale":"2024-04-14T08:42:41.909385+0000","last_undegraded":"2024-04-14T08:42:41.909385+0000","last_fullsized":"2024-04-14T08:42:41.909385+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:05:03.546435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914438+0000","last_change":"2024-04-14T08:42:40.899261+0000","last_active":"2024-04-14T08:42:41.914438+0000","last_peered":"2024-04-14T08:42:41.914438+0000","last_clean":"2024-04-14T08:42:41.914438+0000","last_became_active":"2024-04-14T08:42:40.898970+0000","last_became_peered":"2024-04-14T08:42:40.898970+0000","last_unstale":"2024-04-14T08:42:41.914438+0000","last_undegraded":"2024-04-14T08:42:41.914438+0000","last_fullsized":"2024-04-14T08:42:41.914438+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:05:03.546435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915042+0000","last_change":"2024-04-14T08:42:40.903545+0000","last_active":"2024-04-14T08:42:41.915042+0000","last_peered":"2024-04-14T08:42:41.915042+0000","last_clean":"2024-04-14T08:42:41.915042+0000","last_became_active":"2024-04-14T08:42:40.903269+0000","last_became_peered":"2024-04-14T08:42:40.903269+0000","last_unstale":"2024-04-14T08:42:41.915042+0000","last_undegraded":"2024-04-14T08:42:41.915042+0000","last_fullsized":"2024-04-14T08:42:41.915042+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:40.822681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902585+0000","last_change":"2024-04-14T08:42:40.924183+0000","last_active":"2024-04-14T08:42:42.902585+0000","last_peered":"2024-04-14T08:42:42.902585+0000","last_clean":"2024-04-14T08:42:42.902585+0000","last_became_active":"2024-04-14T08:42:40.923882+0000","last_became_peered":"2024-04-14T08:42:40.923882+0000","last_unstale":"2024-04-14T08:42:42.902585+0000","last_undegraded":"2024-04-14T08:42:42.902585+0000","last_fullsized":"2024-04-14T08:42:42.902585+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:30:41.697931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902442+0000","last_change":"2024-04-14T08:42:40.909665+0000","last_active":"2024-04-14T08:42:42.902442+0000","last_peered":"2024-04-14T08:42:42.902442+0000","last_clean":"2024-04-14T08:42:42.902442+0000","last_became_active":"2024-04-14T08:42:40.909352+0000","last_became_peered":"2024-04-14T08:42:40.909352+0000","last_unstale":"2024-04-14T08:42:42.902442+0000","last_undegraded":"2024-04-14T08:42:42.902442+0000","last_fullsized":"2024-04-14T08:42:42.902442+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:56:34.156369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.908996+0000","last_change":"2024-04-14T08:42:40.914676+0000","last_active":"2024-04-14T08:42:41.908996+0000","last_peered":"2024-04-14T08:42:41.908996+0000","last_clean":"2024-04-14T08:42:41.908996+0000","last_became_active":"2024-04-14T08:42:40.914163+0000","last_became_peered":"2024-04-14T08:42:40.914163+0000","last_unstale":"2024-04-14T08:42:41.908996+0000","last_undegraded":"2024-04-14T08:42:41.908996+0000","last_fullsized":"2024-04-14T08:42:41.908996+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:15:02.182102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914225+0000","last_change":"2024-04-14T08:42:40.909455+0000","last_active":"2024-04-14T08:42:41.914225+0000","last_peered":"2024-04-14T08:42:41.914225+0000","last_clean":"2024-04-14T08:42:41.914225+0000","last_became_active":"2024-04-14T08:42:40.909156+0000","last_became_peered":"2024-04-14T08:42:40.909156+0000","last_unstale":"2024-04-14T08:42:41.914225+0000","last_undegraded":"2024-04-14T08:42:41.914225+0000","last_fullsized":"2024-04-14T08:42:41.914225+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:01:17.366719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290638+0000","last_change":"2024-04-14T08:42:40.914190+0000","last_active":"2024-04-14T08:42:43.290638+0000","last_peered":"2024-04-14T08:42:43.290638+0000","last_clean":"2024-04-14T08:42:43.290638+0000","last_became_active":"2024-04-14T08:42:40.912281+0000","last_became_peered":"2024-04-14T08:42:40.912281+0000","last_unstale":"2024-04-14T08:42:43.290638+0000","last_undegraded":"2024-04-14T08:42:43.290638+0000","last_fullsized":"2024-04-14T08:42:43.290638+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:57.283646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914610+0000","last_change":"2024-04-14T08:42:40.910755+0000","last_active":"2024-04-14T08:42:41.914610+0000","last_peered":"2024-04-14T08:42:41.914610+0000","last_clean":"2024-04-14T08:42:41.914610+0000","last_became_active":"2024-04-14T08:42:40.910464+0000","last_became_peered":"2024-04-14T08:42:40.910464+0000","last_unstale":"2024-04-14T08:42:41.914610+0000","last_undegraded":"2024-04-14T08:42:41.914610+0000","last_fullsized":"2024-04-14T08:42:41.914610+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:57.005587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902797+0000","last_change":"2024-04-14T08:42:40.912985+0000","last_active":"2024-04-14T08:42:42.902797+0000","last_peered":"2024-04-14T08:42:42.902797+0000","last_clean":"2024-04-14T08:42:42.902797+0000","last_became_active":"2024-04-14T08:42:40.912339+0000","last_became_peered":"2024-04-14T08:42:40.912339+0000","last_unstale":"2024-04-14T08:42:42.902797+0000","last_undegraded":"2024-04-14T08:42:42.902797+0000","last_fullsized":"2024-04-14T08:42:42.902797+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:39:59.964083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290778+0000","last_change":"2024-04-14T08:42:40.913208+0000","last_active":"2024-04-14T08:42:43.290778+0000","last_peered":"2024-04-14T08:42:43.290778+0000","last_clean":"2024-04-14T08:42:43.290778+0000","last_became_active":"2024-04-14T08:42:40.912939+0000","last_became_peered":"2024-04-14T08:42:40.912939+0000","last_unstale":"2024-04-14T08:42:43.290778+0000","last_undegraded":"2024-04-14T08:42:43.290778+0000","last_fullsized":"2024-04-14T08:42:43.290778+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:30.073046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909414+0000","last_change":"2024-04-14T08:42:40.922514+0000","last_active":"2024-04-14T08:42:41.909414+0000","last_peered":"2024-04-14T08:42:41.909414+0000","last_clean":"2024-04-14T08:42:41.909414+0000","last_became_active":"2024-04-14T08:42:40.914727+0000","last_became_peered":"2024-04-14T08:42:40.914727+0000","last_unstale":"2024-04-14T08:42:41.909414+0000","last_undegraded":"2024-04-14T08:42:41.909414+0000","last_fullsized":"2024-04-14T08:42:41.909414+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:49.529029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915343+0000","last_change":"2024-04-14T08:42:40.911355+0000","last_active":"2024-04-14T08:42:41.915343+0000","last_peered":"2024-04-14T08:42:41.915343+0000","last_clean":"2024-04-14T08:42:41.915343+0000","last_became_active":"2024-04-14T08:42:40.911065+0000","last_became_peered":"2024-04-14T08:42:40.911065+0000","last_unstale":"2024-04-14T08:42:41.915343+0000","last_undegraded":"2024-04-14T08:42:41.915343+0000","last_fullsized":"2024-04-14T08:42:41.915343+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:42:53.101473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902361+0000","last_change":"2024-04-14T08:42:40.920994+0000","last_active":"2024-04-14T08:42:42.902361+0000","last_peered":"2024-04-14T08:42:42.902361+0000","last_clean":"2024-04-14T08:42:42.902361+0000","last_became_active":"2024-04-14T08:42:40.920704+0000","last_became_peered":"2024-04-14T08:42:40.920704+0000","last_unstale":"2024-04-14T08:42:42.902361+0000","last_undegraded":"2024-04-14T08:42:42.902361+0000","last_fullsized":"2024-04-14T08:42:42.902361+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:55:23.423410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290638+0000","last_change":"2024-04-14T08:42:40.909394+0000","last_active":"2024-04-14T08:42:43.290638+0000","last_peered":"2024-04-14T08:42:43.290638+0000","last_clean":"2024-04-14T08:42:43.290638+0000","last_became_active":"2024-04-14T08:42:40.909131+0000","last_became_peered":"2024-04-14T08:42:40.909131+0000","last_unstale":"2024-04-14T08:42:43.290638+0000","last_undegraded":"2024-04-14T08:42:43.290638+0000","last_fullsized":"2024-04-14T08:42:43.290638+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:32:35.306909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909292+0000","last_change":"2024-04-14T08:42:40.919870+0000","last_active":"2024-04-14T08:42:41.909292+0000","last_peered":"2024-04-14T08:42:41.909292+0000","last_clean":"2024-04-14T08:42:41.909292+0000","last_became_active":"2024-04-14T08:42:40.919598+0000","last_became_peered":"2024-04-14T08:42:40.919598+0000","last_unstale":"2024-04-14T08:42:41.909292+0000","last_undegraded":"2024-04-14T08:42:41.909292+0000","last_fullsized":"2024-04-14T08:42:41.909292+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:19.465728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900264+0000","last_change":"2024-04-14T08:42:40.909410+0000","last_active":"2024-04-14T08:42:42.900264+0000","last_peered":"2024-04-14T08:42:42.900264+0000","last_clean":"2024-04-14T08:42:42.900264+0000","last_became_active":"2024-04-14T08:42:40.909150+0000","last_became_peered":"2024-04-14T08:42:40.909150+0000","last_unstale":"2024-04-14T08:42:42.900264+0000","last_undegraded":"2024-04-14T08:42:42.900264+0000","last_fullsized":"2024-04-14T08:42:42.900264+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:44.417457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290995+0000","last_change":"2024-04-14T08:42:40.921455+0000","last_active":"2024-04-14T08:42:43.290995+0000","last_peered":"2024-04-14T08:42:43.290995+0000","last_clean":"2024-04-14T08:42:43.290995+0000","last_became_active":"2024-04-14T08:42:40.921182+0000","last_became_peered":"2024-04-14T08:42:40.921182+0000","last_unstale":"2024-04-14T08:42:43.290995+0000","last_undegraded":"2024-04-14T08:42:43.290995+0000","last_fullsized":"2024-04-14T08:42:43.290995+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:32:52.097791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915131+0000","last_change":"2024-04-14T08:42:40.906675+0000","last_active":"2024-04-14T08:42:41.915131+0000","last_peered":"2024-04-14T08:42:41.915131+0000","last_clean":"2024-04-14T08:42:41.915131+0000","last_became_active":"2024-04-14T08:42:40.904423+0000","last_became_peered":"2024-04-14T08:42:40.904423+0000","last_unstale":"2024-04-14T08:42:41.915131+0000","last_undegraded":"2024-04-14T08:42:41.915131+0000","last_fullsized":"2024-04-14T08:42:41.915131+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:38:30.115362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914058+0000","last_change":"2024-04-14T08:42:40.922891+0000","last_active":"2024-04-14T08:42:41.914058+0000","last_peered":"2024-04-14T08:42:41.914058+0000","last_clean":"2024-04-14T08:42:41.914058+0000","last_became_active":"2024-04-14T08:42:40.922613+0000","last_became_peered":"2024-04-14T08:42:40.922613+0000","last_unstale":"2024-04-14T08:42:41.914058+0000","last_undegraded":"2024-04-14T08:42:41.914058+0000","last_fullsized":"2024-04-14T08:42:41.914058+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:44.417457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914689+0000","last_change":"2024-04-14T08:42:40.919066+0000","last_active":"2024-04-14T08:42:41.914689+0000","last_peered":"2024-04-14T08:42:41.914689+0000","last_clean":"2024-04-14T08:42:41.914689+0000","last_became_active":"2024-04-14T08:42:40.918630+0000","last_became_peered":"2024-04-14T08:42:40.918630+0000","last_unstale":"2024-04-14T08:42:41.914689+0000","last_undegraded":"2024-04-14T08:42:41.914689+0000","last_fullsized":"2024-04-14T08:42:41.914689+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:40:49.529029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290470+0000","last_change":"2024-04-14T08:42:40.912776+0000","last_active":"2024-04-14T08:42:43.290470+0000","last_peered":"2024-04-14T08:42:43.290470+0000","last_clean":"2024-04-14T08:42:43.290470+0000","last_became_active":"2024-04-14T08:42:40.912504+0000","last_became_peered":"2024-04-14T08:42:40.912504+0000","last_unstale":"2024-04-14T08:42:43.290470+0000","last_undegraded":"2024-04-14T08:42:43.290470+0000","last_fullsized":"2024-04-14T08:42:43.290470+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:44.417457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290820+0000","last_change":"2024-04-14T08:42:40.910387+0000","last_active":"2024-04-14T08:42:43.290820+0000","last_peered":"2024-04-14T08:42:43.290820+0000","last_clean":"2024-04-14T08:42:43.290820+0000","last_became_active":"2024-04-14T08:42:40.910101+0000","last_became_peered":"2024-04-14T08:42:40.910101+0000","last_unstale":"2024-04-14T08:42:43.290820+0000","last_undegraded":"2024-04-14T08:42:43.290820+0000","last_fullsized":"2024-04-14T08:42:43.290820+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:14:56.970195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290987+0000","last_change":"2024-04-14T08:42:40.911692+0000","last_active":"2024-04-14T08:42:43.290987+0000","last_peered":"2024-04-14T08:42:43.290987+0000","last_clean":"2024-04-14T08:42:43.290987+0000","last_became_active":"2024-04-14T08:42:40.910075+0000","last_became_peered":"2024-04-14T08:42:40.910075+0000","last_unstale":"2024-04-14T08:42:43.290987+0000","last_undegraded":"2024-04-14T08:42:43.290987+0000","last_fullsized":"2024-04-14T08:42:43.290987+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:47:15.652845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900301+0000","last_change":"2024-04-14T08:42:40.908641+0000","last_active":"2024-04-14T08:42:42.900301+0000","last_peered":"2024-04-14T08:42:42.900301+0000","last_clean":"2024-04-14T08:42:42.900301+0000","last_became_active":"2024-04-14T08:42:40.908339+0000","last_became_peered":"2024-04-14T08:42:40.908339+0000","last_unstale":"2024-04-14T08:42:42.900301+0000","last_undegraded":"2024-04-14T08:42:42.900301+0000","last_fullsized":"2024-04-14T08:42:42.900301+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:51.037275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902875+0000","last_change":"2024-04-14T08:42:40.912688+0000","last_active":"2024-04-14T08:42:42.902875+0000","last_peered":"2024-04-14T08:42:42.902875+0000","last_clean":"2024-04-14T08:42:42.902875+0000","last_became_active":"2024-04-14T08:42:40.912007+0000","last_became_peered":"2024-04-14T08:42:40.912007+0000","last_unstale":"2024-04-14T08:42:42.902875+0000","last_undegraded":"2024-04-14T08:42:42.902875+0000","last_fullsized":"2024-04-14T08:42:42.902875+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:45:16.718608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290689+0000","last_change":"2024-04-14T08:42:40.908025+0000","last_active":"2024-04-14T08:42:43.290689+0000","last_peered":"2024-04-14T08:42:43.290689+0000","last_clean":"2024-04-14T08:42:43.290689+0000","last_became_active":"2024-04-14T08:42:40.906658+0000","last_became_peered":"2024-04-14T08:42:40.906658+0000","last_unstale":"2024-04-14T08:42:43.290689+0000","last_undegraded":"2024-04-14T08:42:43.290689+0000","last_fullsized":"2024-04-14T08:42:43.290689+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:08:55.311489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909476+0000","last_change":"2024-04-14T08:42:40.923172+0000","last_active":"2024-04-14T08:42:41.909476+0000","last_peered":"2024-04-14T08:42:41.909476+0000","last_clean":"2024-04-14T08:42:41.909476+0000","last_became_active":"2024-04-14T08:42:40.922918+0000","last_became_peered":"2024-04-14T08:42:40.922918+0000","last_unstale":"2024-04-14T08:42:41.909476+0000","last_undegraded":"2024-04-14T08:42:41.909476+0000","last_fullsized":"2024-04-14T08:42:41.909476+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:01:09.722380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290686+0000","last_change":"2024-04-14T08:42:40.920428+0000","last_active":"2024-04-14T08:42:43.290686+0000","last_peered":"2024-04-14T08:42:43.290686+0000","last_clean":"2024-04-14T08:42:43.290686+0000","last_became_active":"2024-04-14T08:42:40.920121+0000","last_became_peered":"2024-04-14T08:42:40.920121+0000","last_unstale":"2024-04-14T08:42:43.290686+0000","last_undegraded":"2024-04-14T08:42:43.290686+0000","last_fullsized":"2024-04-14T08:42:43.290686+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:40.822681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290475+0000","last_change":"2024-04-14T08:42:40.913456+0000","last_active":"2024-04-14T08:42:43.290475+0000","last_peered":"2024-04-14T08:42:43.290475+0000","last_clean":"2024-04-14T08:42:43.290475+0000","last_became_active":"2024-04-14T08:42:40.913031+0000","last_became_peered":"2024-04-14T08:42:40.913031+0000","last_unstale":"2024-04-14T08:42:43.290475+0000","last_undegraded":"2024-04-14T08:42:43.290475+0000","last_fullsized":"2024-04-14T08:42:43.290475+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:51.037275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290650+0000","last_change":"2024-04-14T08:42:40.916128+0000","last_active":"2024-04-14T08:42:43.290650+0000","last_peered":"2024-04-14T08:42:43.290650+0000","last_clean":"2024-04-14T08:42:43.290650+0000","last_became_active":"2024-04-14T08:42:40.915859+0000","last_became_peered":"2024-04-14T08:42:40.915859+0000","last_unstale":"2024-04-14T08:42:43.290650+0000","last_undegraded":"2024-04-14T08:42:43.290650+0000","last_fullsized":"2024-04-14T08:42:43.290650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:34:55.774170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"115'4","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900211+0000","last_change":"2024-04-14T08:42:39.902925+0000","last_active":"2024-04-14T08:42:42.900211+0000","last_peered":"2024-04-14T08:42:42.900211+0000","last_clean":"2024-04-14T08:42:42.900211+0000","last_became_active":"2024-04-14T08:42:39.902599+0000","last_became_peered":"2024-04-14T08:42:39.902599+0000","last_unstale":"2024-04-14T08:42:42.900211+0000","last_undegraded":"2024-04-14T08:42:42.900211+0000","last_fullsized":"2024-04-14T08:42:42.900211+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:25:26.756136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902825+0000","last_change":"2024-04-14T08:42:40.918689+0000","last_active":"2024-04-14T08:42:42.902825+0000","last_peered":"2024-04-14T08:42:42.902825+0000","last_clean":"2024-04-14T08:42:42.902825+0000","last_became_active":"2024-04-14T08:42:40.918387+0000","last_became_peered":"2024-04-14T08:42:40.918387+0000","last_unstale":"2024-04-14T08:42:42.902825+0000","last_undegraded":"2024-04-14T08:42:42.902825+0000","last_fullsized":"2024-04-14T08:42:42.902825+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:26:01.926948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290736+0000","last_change":"2024-04-14T08:42:39.898146+0000","last_active":"2024-04-14T08:42:43.290736+0000","last_peered":"2024-04-14T08:42:43.290736+0000","last_clean":"2024-04-14T08:42:43.290736+0000","last_became_active":"2024-04-14T08:42:39.897857+0000","last_became_peered":"2024-04-14T08:42:39.897857+0000","last_unstale":"2024-04-14T08:42:43.290736+0000","last_undegraded":"2024-04-14T08:42:43.290736+0000","last_fullsized":"2024-04-14T08:42:43.290736+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:22:11.033372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290693+0000","last_change":"2024-04-14T08:42:40.909220+0000","last_active":"2024-04-14T08:42:43.290693+0000","last_peered":"2024-04-14T08:42:43.290693+0000","last_clean":"2024-04-14T08:42:43.290693+0000","last_became_active":"2024-04-14T08:42:40.908884+0000","last_became_peered":"2024-04-14T08:42:40.908884+0000","last_unstale":"2024-04-14T08:42:43.290693+0000","last_undegraded":"2024-04-14T08:42:43.290693+0000","last_fullsized":"2024-04-14T08:42:43.290693+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:42:53.101473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915028+0000","last_change":"2024-04-14T08:42:39.903458+0000","last_active":"2024-04-14T08:42:41.915028+0000","last_peered":"2024-04-14T08:42:41.915028+0000","last_clean":"2024-04-14T08:42:41.915028+0000","last_became_active":"2024-04-14T08:42:39.903032+0000","last_became_peered":"2024-04-14T08:42:39.903032+0000","last_unstale":"2024-04-14T08:42:41.915028+0000","last_undegraded":"2024-04-14T08:42:41.915028+0000","last_fullsized":"2024-04-14T08:42:41.915028+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:43.408088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914383+0000","last_change":"2024-04-14T08:42:40.904383+0000","last_active":"2024-04-14T08:42:41.914383+0000","last_peered":"2024-04-14T08:42:41.914383+0000","last_clean":"2024-04-14T08:42:41.914383+0000","last_became_active":"2024-04-14T08:42:40.904099+0000","last_became_peered":"2024-04-14T08:42:40.904099+0000","last_unstale":"2024-04-14T08:42:41.914383+0000","last_undegraded":"2024-04-14T08:42:41.914383+0000","last_fullsized":"2024-04-14T08:42:41.914383+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:31:19.465728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290787+0000","last_change":"2024-04-14T08:42:39.914891+0000","last_active":"2024-04-14T08:42:43.290787+0000","last_peered":"2024-04-14T08:42:43.290787+0000","last_clean":"2024-04-14T08:42:43.290787+0000","last_became_active":"2024-04-14T08:42:39.914617+0000","last_became_peered":"2024-04-14T08:42:39.914617+0000","last_unstale":"2024-04-14T08:42:43.290787+0000","last_undegraded":"2024-04-14T08:42:43.290787+0000","last_fullsized":"2024-04-14T08:42:43.290787+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:20:54.450691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290724+0000","last_change":"2024-04-14T08:42:40.912016+0000","last_active":"2024-04-14T08:42:43.290724+0000","last_peered":"2024-04-14T08:42:43.290724+0000","last_clean":"2024-04-14T08:42:43.290724+0000","last_became_active":"2024-04-14T08:42:40.911749+0000","last_became_peered":"2024-04-14T08:42:40.911749+0000","last_unstale":"2024-04-14T08:42:43.290724+0000","last_undegraded":"2024-04-14T08:42:43.290724+0000","last_fullsized":"2024-04-14T08:42:43.290724+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:12:44.417457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915101+0000","last_change":"2024-04-14T08:42:39.898512+0000","last_active":"2024-04-14T08:42:41.915101+0000","last_peered":"2024-04-14T08:42:41.915101+0000","last_clean":"2024-04-14T08:42:41.915101+0000","last_became_active":"2024-04-14T08:42:39.898183+0000","last_became_peered":"2024-04-14T08:42:39.898183+0000","last_unstale":"2024-04-14T08:42:41.915101+0000","last_undegraded":"2024-04-14T08:42:41.915101+0000","last_fullsized":"2024-04-14T08:42:41.915101+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:07:34.836827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915435+0000","last_change":"2024-04-14T08:42:40.900777+0000","last_active":"2024-04-14T08:42:41.915435+0000","last_peered":"2024-04-14T08:42:41.915435+0000","last_clean":"2024-04-14T08:42:41.915435+0000","last_became_active":"2024-04-14T08:42:40.900462+0000","last_became_peered":"2024-04-14T08:42:40.900462+0000","last_unstale":"2024-04-14T08:42:41.915435+0000","last_undegraded":"2024-04-14T08:42:41.915435+0000","last_fullsized":"2024-04-14T08:42:41.915435+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:50:25.331351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902512+0000","last_change":"2024-04-14T08:42:39.915147+0000","last_active":"2024-04-14T08:42:42.902512+0000","last_peered":"2024-04-14T08:42:42.902512+0000","last_clean":"2024-04-14T08:42:42.902512+0000","last_became_active":"2024-04-14T08:42:39.914929+0000","last_became_peered":"2024-04-14T08:42:39.914929+0000","last_unstale":"2024-04-14T08:42:42.902512+0000","last_undegraded":"2024-04-14T08:42:42.902512+0000","last_fullsized":"2024-04-14T08:42:42.902512+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:41:14.539282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909526+0000","last_change":"2024-04-14T08:42:40.924817+0000","last_active":"2024-04-14T08:42:41.909526+0000","last_peered":"2024-04-14T08:42:41.909526+0000","last_clean":"2024-04-14T08:42:41.909526+0000","last_became_active":"2024-04-14T08:42:40.924440+0000","last_became_peered":"2024-04-14T08:42:40.924440+0000","last_unstale":"2024-04-14T08:42:41.909526+0000","last_undegraded":"2024-04-14T08:42:41.909526+0000","last_fullsized":"2024-04-14T08:42:41.909526+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:06:35.189673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290565+0000","last_change":"2024-04-14T08:42:39.904121+0000","last_active":"2024-04-14T08:42:43.290565+0000","last_peered":"2024-04-14T08:42:43.290565+0000","last_clean":"2024-04-14T08:42:43.290565+0000","last_became_active":"2024-04-14T08:42:39.903879+0000","last_became_peered":"2024-04-14T08:42:39.903879+0000","last_unstale":"2024-04-14T08:42:43.290565+0000","last_undegraded":"2024-04-14T08:42:43.290565+0000","last_fullsized":"2024-04-14T08:42:43.290565+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:42:20.871047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290906+0000","last_change":"2024-04-14T08:42:40.924009+0000","last_active":"2024-04-14T08:42:43.290906+0000","last_peered":"2024-04-14T08:42:43.290906+0000","last_clean":"2024-04-14T08:42:43.290906+0000","last_became_active":"2024-04-14T08:42:40.923601+0000","last_became_peered":"2024-04-14T08:42:40.923601+0000","last_unstale":"2024-04-14T08:42:43.290906+0000","last_undegraded":"2024-04-14T08:42:43.290906+0000","last_fullsized":"2024-04-14T08:42:43.290906+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:51.037275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902524+0000","last_change":"2024-04-14T08:42:39.900795+0000","last_active":"2024-04-14T08:42:42.902524+0000","last_peered":"2024-04-14T08:42:42.902524+0000","last_clean":"2024-04-14T08:42:42.902524+0000","last_became_active":"2024-04-14T08:42:39.900345+0000","last_became_peered":"2024-04-14T08:42:39.900345+0000","last_unstale":"2024-04-14T08:42:42.902524+0000","last_undegraded":"2024-04-14T08:42:42.902524+0000","last_fullsized":"2024-04-14T08:42:42.902524+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:22:52.184492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900153+0000","last_change":"2024-04-14T08:42:40.907871+0000","last_active":"2024-04-14T08:42:42.900153+0000","last_peered":"2024-04-14T08:42:42.900153+0000","last_clean":"2024-04-14T08:42:42.900153+0000","last_became_active":"2024-04-14T08:42:40.907553+0000","last_became_peered":"2024-04-14T08:42:40.907553+0000","last_unstale":"2024-04-14T08:42:42.900153+0000","last_undegraded":"2024-04-14T08:42:42.900153+0000","last_fullsized":"2024-04-14T08:42:42.900153+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:58:26.274235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900224+0000","last_change":"2024-04-14T08:42:39.908268+0000","last_active":"2024-04-14T08:42:42.900224+0000","last_peered":"2024-04-14T08:42:42.900224+0000","last_clean":"2024-04-14T08:42:42.900224+0000","last_became_active":"2024-04-14T08:42:39.907971+0000","last_became_peered":"2024-04-14T08:42:39.907971+0000","last_unstale":"2024-04-14T08:42:42.900224+0000","last_undegraded":"2024-04-14T08:42:42.900224+0000","last_fullsized":"2024-04-14T08:42:42.900224+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:47:33.457508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290674+0000","last_change":"2024-04-14T08:42:40.919109+0000","last_active":"2024-04-14T08:42:43.290674+0000","last_peered":"2024-04-14T08:42:43.290674+0000","last_clean":"2024-04-14T08:42:43.290674+0000","last_became_active":"2024-04-14T08:42:40.918802+0000","last_became_peered":"2024-04-14T08:42:40.918802+0000","last_unstale":"2024-04-14T08:42:43.290674+0000","last_undegraded":"2024-04-14T08:42:43.290674+0000","last_fullsized":"2024-04-14T08:42:43.290674+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:57.005587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290681+0000","last_change":"2024-04-14T08:42:39.911711+0000","last_active":"2024-04-14T08:42:43.290681+0000","last_peered":"2024-04-14T08:42:43.290681+0000","last_clean":"2024-04-14T08:42:43.290681+0000","last_became_active":"2024-04-14T08:42:39.911408+0000","last_became_peered":"2024-04-14T08:42:39.911408+0000","last_unstale":"2024-04-14T08:42:43.290681+0000","last_undegraded":"2024-04-14T08:42:43.290681+0000","last_fullsized":"2024-04-14T08:42:43.290681+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:35:14.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909427+0000","last_change":"2024-04-14T08:42:40.915228+0000","last_active":"2024-04-14T08:42:41.909427+0000","last_peered":"2024-04-14T08:42:41.909427+0000","last_clean":"2024-04-14T08:42:41.909427+0000","last_became_active":"2024-04-14T08:42:40.914332+0000","last_became_peered":"2024-04-14T08:42:40.914332+0000","last_unstale":"2024-04-14T08:42:41.909427+0000","last_undegraded":"2024-04-14T08:42:41.909427+0000","last_fullsized":"2024-04-14T08:42:41.909427+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:13:59.521980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"115'3","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290711+0000","last_change":"2024-04-14T08:42:39.906942+0000","last_active":"2024-04-14T08:42:43.290711+0000","last_peered":"2024-04-14T08:42:43.290711+0000","last_clean":"2024-04-14T08:42:43.290711+0000","last_became_active":"2024-04-14T08:42:39.906650+0000","last_became_peered":"2024-04-14T08:42:39.906650+0000","last_unstale":"2024-04-14T08:42:43.290711+0000","last_undegraded":"2024-04-14T08:42:43.290711+0000","last_fullsized":"2024-04-14T08:42:43.290711+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:07:34.836827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290863+0000","last_change":"2024-04-14T08:42:40.920432+0000","last_active":"2024-04-14T08:42:43.290863+0000","last_peered":"2024-04-14T08:42:43.290863+0000","last_clean":"2024-04-14T08:42:43.290863+0000","last_became_active":"2024-04-14T08:42:40.920133+0000","last_became_peered":"2024-04-14T08:42:40.920133+0000","last_unstale":"2024-04-14T08:42:43.290863+0000","last_undegraded":"2024-04-14T08:42:43.290863+0000","last_fullsized":"2024-04-14T08:42:43.290863+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:22:28.290311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"115'6","reported_seq":20,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909195+0000","last_change":"2024-04-14T08:42:39.917311+0000","last_active":"2024-04-14T08:42:41.909195+0000","last_peered":"2024-04-14T08:42:41.909195+0000","last_clean":"2024-04-14T08:42:41.909195+0000","last_became_active":"2024-04-14T08:42:39.917001+0000","last_became_peered":"2024-04-14T08:42:39.917001+0000","last_unstale":"2024-04-14T08:42:41.909195+0000","last_undegraded":"2024-04-14T08:42:41.909195+0000","last_fullsized":"2024-04-14T08:42:41.909195+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:58:25.264866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900171+0000","last_change":"2024-04-14T08:42:40.917029+0000","last_active":"2024-04-14T08:42:42.900171+0000","last_peered":"2024-04-14T08:42:42.900171+0000","last_clean":"2024-04-14T08:42:42.900171+0000","last_became_active":"2024-04-14T08:42:40.916763+0000","last_became_peered":"2024-04-14T08:42:40.916763+0000","last_unstale":"2024-04-14T08:42:42.900171+0000","last_undegraded":"2024-04-14T08:42:42.900171+0000","last_fullsized":"2024-04-14T08:42:42.900171+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:30.073046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909128+0000","last_change":"2024-04-14T08:42:39.914644+0000","last_active":"2024-04-14T08:42:41.909128+0000","last_peered":"2024-04-14T08:42:41.909128+0000","last_clean":"2024-04-14T08:42:41.909128+0000","last_became_active":"2024-04-14T08:42:39.914350+0000","last_became_peered":"2024-04-14T08:42:39.914350+0000","last_unstale":"2024-04-14T08:42:41.909128+0000","last_undegraded":"2024-04-14T08:42:41.909128+0000","last_fullsized":"2024-04-14T08:42:41.909128+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:29.063677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290471+0000","last_change":"2024-04-14T08:42:40.910224+0000","last_active":"2024-04-14T08:42:43.290471+0000","last_peered":"2024-04-14T08:42:43.290471+0000","last_clean":"2024-04-14T08:42:43.290471+0000","last_became_active":"2024-04-14T08:42:40.909934+0000","last_became_peered":"2024-04-14T08:42:40.909934+0000","last_unstale":"2024-04-14T08:42:43.290471+0000","last_undegraded":"2024-04-14T08:42:43.290471+0000","last_fullsized":"2024-04-14T08:42:43.290471+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:45:57.742439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909634+0000","last_change":"2024-04-14T08:42:39.914644+0000","last_active":"2024-04-14T08:42:41.909634+0000","last_peered":"2024-04-14T08:42:41.909634+0000","last_clean":"2024-04-14T08:42:41.909634+0000","last_became_active":"2024-04-14T08:42:39.914387+0000","last_became_peered":"2024-04-14T08:42:39.914387+0000","last_unstale":"2024-04-14T08:42:41.909634+0000","last_undegraded":"2024-04-14T08:42:41.909634+0000","last_fullsized":"2024-04-14T08:42:41.909634+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:49:56.274277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290552+0000","last_change":"2024-04-14T08:42:40.900265+0000","last_active":"2024-04-14T08:42:43.290552+0000","last_peered":"2024-04-14T08:42:43.290552+0000","last_clean":"2024-04-14T08:42:43.290552+0000","last_became_active":"2024-04-14T08:42:40.899538+0000","last_became_peered":"2024-04-14T08:42:40.899538+0000","last_unstale":"2024-04-14T08:42:43.290552+0000","last_undegraded":"2024-04-14T08:42:43.290552+0000","last_fullsized":"2024-04-14T08:42:43.290552+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:52:35.491912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909109+0000","last_change":"2024-04-14T08:42:39.917235+0000","last_active":"2024-04-14T08:42:41.909109+0000","last_peered":"2024-04-14T08:42:41.909109+0000","last_clean":"2024-04-14T08:42:41.909109+0000","last_became_active":"2024-04-14T08:42:39.916937+0000","last_became_peered":"2024-04-14T08:42:39.916937+0000","last_unstale":"2024-04-14T08:42:41.909109+0000","last_undegraded":"2024-04-14T08:42:41.909109+0000","last_fullsized":"2024-04-14T08:42:41.909109+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:48:28.600717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290659+0000","last_change":"2024-04-14T08:42:40.914873+0000","last_active":"2024-04-14T08:42:43.290659+0000","last_peered":"2024-04-14T08:42:43.290659+0000","last_clean":"2024-04-14T08:42:43.290659+0000","last_became_active":"2024-04-14T08:42:40.914598+0000","last_became_peered":"2024-04-14T08:42:40.914598+0000","last_unstale":"2024-04-14T08:42:43.290659+0000","last_undegraded":"2024-04-14T08:42:43.290659+0000","last_fullsized":"2024-04-14T08:42:43.290659+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:37:01.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290857+0000","last_change":"2024-04-14T08:42:39.907928+0000","last_active":"2024-04-14T08:42:43.290857+0000","last_peered":"2024-04-14T08:42:43.290857+0000","last_clean":"2024-04-14T08:42:43.290857+0000","last_became_active":"2024-04-14T08:42:39.907662+0000","last_became_peered":"2024-04-14T08:42:39.907662+0000","last_unstale":"2024-04-14T08:42:43.290857+0000","last_undegraded":"2024-04-14T08:42:43.290857+0000","last_fullsized":"2024-04-14T08:42:43.290857+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:32:40.235395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914775+0000","last_change":"2024-04-14T08:42:40.922938+0000","last_active":"2024-04-14T08:42:41.914775+0000","last_peered":"2024-04-14T08:42:41.914775+0000","last_clean":"2024-04-14T08:42:41.914775+0000","last_became_active":"2024-04-14T08:42:40.922638+0000","last_became_peered":"2024-04-14T08:42:40.922638+0000","last_unstale":"2024-04-14T08:42:41.914775+0000","last_undegraded":"2024-04-14T08:42:41.914775+0000","last_fullsized":"2024-04-14T08:42:41.914775+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:37:01.426565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.291173+0000","last_change":"2024-04-14T08:42:39.903548+0000","last_active":"2024-04-14T08:42:43.291173+0000","last_peered":"2024-04-14T08:42:43.291173+0000","last_clean":"2024-04-14T08:42:43.291173+0000","last_became_active":"2024-04-14T08:42:39.903272+0000","last_became_peered":"2024-04-14T08:42:39.903272+0000","last_unstale":"2024-04-14T08:42:43.291173+0000","last_undegraded":"2024-04-14T08:42:43.291173+0000","last_fullsized":"2024-04-14T08:42:43.291173+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:17:27.935436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902883+0000","last_change":"2024-04-14T08:42:40.913520+0000","last_active":"2024-04-14T08:42:42.902883+0000","last_peered":"2024-04-14T08:42:42.902883+0000","last_clean":"2024-04-14T08:42:42.902883+0000","last_became_active":"2024-04-14T08:42:40.913220+0000","last_became_peered":"2024-04-14T08:42:40.913220+0000","last_unstale":"2024-04-14T08:42:42.902883+0000","last_undegraded":"2024-04-14T08:42:42.902883+0000","last_fullsized":"2024-04-14T08:42:42.902883+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:28:20.358223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"115'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915134+0000","last_change":"2024-04-14T08:42:39.907861+0000","last_active":"2024-04-14T08:42:41.915134+0000","last_peered":"2024-04-14T08:42:41.915134+0000","last_clean":"2024-04-14T08:42:41.915134+0000","last_became_active":"2024-04-14T08:42:39.907144+0000","last_became_peered":"2024-04-14T08:42:39.907144+0000","last_unstale":"2024-04-14T08:42:41.915134+0000","last_undegraded":"2024-04-14T08:42:41.915134+0000","last_fullsized":"2024-04-14T08:42:41.915134+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:53:50.027906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290893+0000","last_change":"2024-04-14T08:42:40.911807+0000","last_active":"2024-04-14T08:42:43.290893+0000","last_peered":"2024-04-14T08:42:43.290893+0000","last_clean":"2024-04-14T08:42:43.290893+0000","last_became_active":"2024-04-14T08:42:40.911503+0000","last_became_peered":"2024-04-14T08:42:40.911503+0000","last_unstale":"2024-04-14T08:42:43.290893+0000","last_undegraded":"2024-04-14T08:42:43.290893+0000","last_fullsized":"2024-04-14T08:42:43.290893+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:53:03.886099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914549+0000","last_change":"2024-04-14T08:42:39.900873+0000","last_active":"2024-04-14T08:42:41.914549+0000","last_peered":"2024-04-14T08:42:41.914549+0000","last_clean":"2024-04-14T08:42:41.914549+0000","last_became_active":"2024-04-14T08:42:39.900603+0000","last_became_peered":"2024-04-14T08:42:39.900603+0000","last_unstale":"2024-04-14T08:42:41.914549+0000","last_undegraded":"2024-04-14T08:42:41.914549+0000","last_fullsized":"2024-04-14T08:42:41.914549+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:59:29.063677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902378+0000","last_change":"2024-04-14T08:42:40.914655+0000","last_active":"2024-04-14T08:42:42.902378+0000","last_peered":"2024-04-14T08:42:42.902378+0000","last_clean":"2024-04-14T08:42:42.902378+0000","last_became_active":"2024-04-14T08:42:40.914378+0000","last_became_peered":"2024-04-14T08:42:40.914378+0000","last_unstale":"2024-04-14T08:42:42.902378+0000","last_undegraded":"2024-04-14T08:42:42.902378+0000","last_fullsized":"2024-04-14T08:42:42.902378+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:35:09.462995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902387+0000","last_change":"2024-04-14T08:42:39.904952+0000","last_active":"2024-04-14T08:42:42.902387+0000","last_peered":"2024-04-14T08:42:42.902387+0000","last_clean":"2024-04-14T08:42:42.902387+0000","last_became_active":"2024-04-14T08:42:39.904677+0000","last_became_peered":"2024-04-14T08:42:39.904677+0000","last_unstale":"2024-04-14T08:42:42.902387+0000","last_undegraded":"2024-04-14T08:42:42.902387+0000","last_fullsized":"2024-04-14T08:42:42.902387+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:05:14.219045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.908978+0000","last_change":"2024-04-14T08:42:40.915072+0000","last_active":"2024-04-14T08:42:41.908978+0000","last_peered":"2024-04-14T08:42:41.908978+0000","last_clean":"2024-04-14T08:42:41.908978+0000","last_became_active":"2024-04-14T08:42:40.913471+0000","last_became_peered":"2024-04-14T08:42:40.913471+0000","last_unstale":"2024-04-14T08:42:41.908978+0000","last_undegraded":"2024-04-14T08:42:41.908978+0000","last_fullsized":"2024-04-14T08:42:41.908978+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T08:53:03.886099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914237+0000","last_change":"2024-04-14T08:42:39.902053+0000","last_active":"2024-04-14T08:42:41.914237+0000","last_peered":"2024-04-14T08:42:41.914237+0000","last_clean":"2024-04-14T08:42:41.914237+0000","last_became_active":"2024-04-14T08:42:39.901709+0000","last_became_peered":"2024-04-14T08:42:39.901709+0000","last_unstale":"2024-04-14T08:42:41.914237+0000","last_undegraded":"2024-04-14T08:42:41.914237+0000","last_fullsized":"2024-04-14T08:42:41.914237+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:58:25.264866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909617+0000","last_change":"2024-04-14T08:42:40.915770+0000","last_active":"2024-04-14T08:42:41.909617+0000","last_peered":"2024-04-14T08:42:41.909617+0000","last_clean":"2024-04-14T08:42:41.909617+0000","last_became_active":"2024-04-14T08:42:40.914328+0000","last_became_peered":"2024-04-14T08:42:40.914328+0000","last_unstale":"2024-04-14T08:42:41.909617+0000","last_undegraded":"2024-04-14T08:42:41.909617+0000","last_fullsized":"2024-04-14T08:42:41.909617+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:38:12.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909161+0000","last_change":"2024-04-14T08:42:39.917517+0000","last_active":"2024-04-14T08:42:41.909161+0000","last_peered":"2024-04-14T08:42:41.909161+0000","last_clean":"2024-04-14T08:42:41.909161+0000","last_became_active":"2024-04-14T08:42:39.917246+0000","last_became_peered":"2024-04-14T08:42:39.917246+0000","last_unstale":"2024-04-14T08:42:41.909161+0000","last_undegraded":"2024-04-14T08:42:41.909161+0000","last_fullsized":"2024-04-14T08:42:41.909161+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:45:56.733070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290698+0000","last_change":"2024-04-14T08:42:40.909187+0000","last_active":"2024-04-14T08:42:43.290698+0000","last_peered":"2024-04-14T08:42:43.290698+0000","last_clean":"2024-04-14T08:42:43.290698+0000","last_became_active":"2024-04-14T08:42:40.908912+0000","last_became_peered":"2024-04-14T08:42:40.908912+0000","last_unstale":"2024-04-14T08:42:43.290698+0000","last_undegraded":"2024-04-14T08:42:43.290698+0000","last_fullsized":"2024-04-14T08:42:43.290698+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:50:25.331351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.913699+0000","last_change":"2024-04-14T08:42:39.903314+0000","last_active":"2024-04-14T08:42:41.913699+0000","last_peered":"2024-04-14T08:42:41.913699+0000","last_clean":"2024-04-14T08:42:41.913699+0000","last_became_active":"2024-04-14T08:42:39.903044+0000","last_became_peered":"2024-04-14T08:42:39.903044+0000","last_unstale":"2024-04-14T08:42:41.913699+0000","last_undegraded":"2024-04-14T08:42:41.913699+0000","last_fullsized":"2024-04-14T08:42:41.913699+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:32:34.297541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902969+0000","last_change":"2024-04-14T08:42:40.917142+0000","last_active":"2024-04-14T08:42:42.902969+0000","last_peered":"2024-04-14T08:42:42.902969+0000","last_clean":"2024-04-14T08:42:42.902969+0000","last_became_active":"2024-04-14T08:42:40.915276+0000","last_became_peered":"2024-04-14T08:42:40.915276+0000","last_unstale":"2024-04-14T08:42:42.902969+0000","last_undegraded":"2024-04-14T08:42:42.902969+0000","last_fullsized":"2024-04-14T08:42:42.902969+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:38:34.850919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"115'2","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902941+0000","last_change":"2024-04-14T08:42:39.899765+0000","last_active":"2024-04-14T08:42:42.902941+0000","last_peered":"2024-04-14T08:42:42.902941+0000","last_clean":"2024-04-14T08:42:42.902941+0000","last_became_active":"2024-04-14T08:42:39.898985+0000","last_became_peered":"2024-04-14T08:42:39.898985+0000","last_unstale":"2024-04-14T08:42:42.902941+0000","last_undegraded":"2024-04-14T08:42:42.902941+0000","last_fullsized":"2024-04-14T08:42:42.902941+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:01:57.414126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290731+0000","last_change":"2024-04-14T08:42:40.912641+0000","last_active":"2024-04-14T08:42:43.290731+0000","last_peered":"2024-04-14T08:42:43.290731+0000","last_clean":"2024-04-14T08:42:43.290731+0000","last_became_active":"2024-04-14T08:42:40.912388+0000","last_became_peered":"2024-04-14T08:42:40.912388+0000","last_unstale":"2024-04-14T08:42:43.290731+0000","last_undegraded":"2024-04-14T08:42:43.290731+0000","last_fullsized":"2024-04-14T08:42:43.290731+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:58:26.274235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"116'1","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900440+0000","last_change":"2024-04-14T08:42:39.908137+0000","last_active":"2024-04-14T08:42:42.900440+0000","last_peered":"2024-04-14T08:42:42.900440+0000","last_clean":"2024-04-14T08:42:42.900440+0000","last_became_active":"2024-04-14T08:42:39.907857+0000","last_became_peered":"2024-04-14T08:42:39.907857+0000","last_unstale":"2024-04-14T08:42:42.900440+0000","last_undegraded":"2024-04-14T08:42:42.900440+0000","last_fullsized":"2024-04-14T08:42:42.900440+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:48:28.600717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914730+0000","last_change":"2024-04-14T08:42:40.904731+0000","last_active":"2024-04-14T08:42:41.914730+0000","last_peered":"2024-04-14T08:42:41.914730+0000","last_clean":"2024-04-14T08:42:41.914730+0000","last_became_active":"2024-04-14T08:42:40.904169+0000","last_became_peered":"2024-04-14T08:42:40.904169+0000","last_unstale":"2024-04-14T08:42:41.914730+0000","last_undegraded":"2024-04-14T08:42:41.914730+0000","last_fullsized":"2024-04-14T08:42:41.914730+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:13:59.521980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.900334+0000","last_change":"2024-04-14T08:42:39.907401+0000","last_active":"2024-04-14T08:42:42.900334+0000","last_peered":"2024-04-14T08:42:42.900334+0000","last_clean":"2024-04-14T08:42:42.900334+0000","last_became_active":"2024-04-14T08:42:39.907124+0000","last_became_peered":"2024-04-14T08:42:39.907124+0000","last_unstale":"2024-04-14T08:42:42.900334+0000","last_undegraded":"2024-04-14T08:42:42.900334+0000","last_fullsized":"2024-04-14T08:42:42.900334+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:41.656802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.291258+0000","last_change":"2024-04-14T08:42:40.911392+0000","last_active":"2024-04-14T08:42:43.291258+0000","last_peered":"2024-04-14T08:42:43.291258+0000","last_clean":"2024-04-14T08:42:43.291258+0000","last_became_active":"2024-04-14T08:42:40.908865+0000","last_became_peered":"2024-04-14T08:42:40.908865+0000","last_unstale":"2024-04-14T08:42:43.291258+0000","last_undegraded":"2024-04-14T08:42:43.291258+0000","last_fullsized":"2024-04-14T08:42:43.291258+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T11:02:00.666186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.903031+0000","last_change":"2024-04-14T08:42:39.901863+0000","last_active":"2024-04-14T08:42:42.903031+0000","last_peered":"2024-04-14T08:42:42.903031+0000","last_clean":"2024-04-14T08:42:42.903031+0000","last_became_active":"2024-04-14T08:42:39.901612+0000","last_became_peered":"2024-04-14T08:42:39.901612+0000","last_unstale":"2024-04-14T08:42:42.903031+0000","last_undegraded":"2024-04-14T08:42:42.903031+0000","last_fullsized":"2024-04-14T08:42:42.903031+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T10:12:15.161276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902590+0000","last_change":"2024-04-14T08:42:40.906995+0000","last_active":"2024-04-14T08:42:42.902590+0000","last_peered":"2024-04-14T08:42:42.902590+0000","last_clean":"2024-04-14T08:42:42.902590+0000","last_became_active":"2024-04-14T08:42:40.906684+0000","last_became_peered":"2024-04-14T08:42:40.906684+0000","last_unstale":"2024-04-14T08:42:42.902590+0000","last_undegraded":"2024-04-14T08:42:42.902590+0000","last_fullsized":"2024-04-14T08:42:42.902590+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:19:18.732652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915157+0000","last_change":"2024-04-14T08:42:39.898079+0000","last_active":"2024-04-14T08:42:41.915157+0000","last_peered":"2024-04-14T08:42:41.915157+0000","last_clean":"2024-04-14T08:42:41.915157+0000","last_became_active":"2024-04-14T08:42:39.897776+0000","last_became_peered":"2024-04-14T08:42:39.897776+0000","last_unstale":"2024-04-14T08:42:41.915157+0000","last_undegraded":"2024-04-14T08:42:41.915157+0000","last_fullsized":"2024-04-14T08:42:41.915157+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:25:26.756136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909020+0000","last_change":"2024-04-14T08:42:40.913761+0000","last_active":"2024-04-14T08:42:41.909020+0000","last_peered":"2024-04-14T08:42:41.909020+0000","last_clean":"2024-04-14T08:42:41.909020+0000","last_became_active":"2024-04-14T08:42:40.913125+0000","last_became_peered":"2024-04-14T08:42:40.913125+0000","last_unstale":"2024-04-14T08:42:41.909020+0000","last_undegraded":"2024-04-14T08:42:41.909020+0000","last_fullsized":"2024-04-14T08:42:41.909020+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T13:01:17.366719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914168+0000","last_change":"2024-04-14T08:42:39.907024+0000","last_active":"2024-04-14T08:42:41.914168+0000","last_peered":"2024-04-14T08:42:41.914168+0000","last_clean":"2024-04-14T08:42:41.914168+0000","last_became_active":"2024-04-14T08:42:39.906749+0000","last_became_peered":"2024-04-14T08:42:39.906749+0000","last_unstale":"2024-04-14T08:42:41.914168+0000","last_undegraded":"2024-04-14T08:42:41.914168+0000","last_fullsized":"2024-04-14T08:42:41.914168+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:01:41.656802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914136+0000","last_change":"2024-04-14T08:42:40.909631+0000","last_active":"2024-04-14T08:42:41.914136+0000","last_peered":"2024-04-14T08:42:41.914136+0000","last_clean":"2024-04-14T08:42:41.914136+0000","last_became_active":"2024-04-14T08:42:40.909358+0000","last_became_peered":"2024-04-14T08:42:40.909358+0000","last_unstale":"2024-04-14T08:42:41.914136+0000","last_undegraded":"2024-04-14T08:42:41.914136+0000","last_fullsized":"2024-04-14T08:42:41.914136+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:15:02.182102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:42.902433+0000","last_change":"2024-04-14T08:42:39.899535+0000","last_active":"2024-04-14T08:42:42.902433+0000","last_peered":"2024-04-14T08:42:42.902433+0000","last_clean":"2024-04-14T08:42:42.902433+0000","last_became_active":"2024-04-14T08:42:39.899245+0000","last_became_peered":"2024-04-14T08:42:39.899245+0000","last_unstale":"2024-04-14T08:42:42.902433+0000","last_undegraded":"2024-04-14T08:42:42.902433+0000","last_fullsized":"2024-04-14T08:42:42.902433+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:17:13.730626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:43.290960+0000","last_change":"2024-04-14T08:42:40.917261+0000","last_active":"2024-04-14T08:42:43.290960+0000","last_peered":"2024-04-14T08:42:43.290960+0000","last_clean":"2024-04-14T08:42:43.290960+0000","last_became_active":"2024-04-14T08:42:40.916998+0000","last_became_peered":"2024-04-14T08:42:40.916998+0000","last_unstale":"2024-04-14T08:42:43.290960+0000","last_undegraded":"2024-04-14T08:42:43.290960+0000","last_fullsized":"2024-04-14T08:42:43.290960+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T09:21:57.005587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.913699+0000","last_change":"2024-04-14T08:42:39.903168+0000","last_active":"2024-04-14T08:42:41.913699+0000","last_peered":"2024-04-14T08:42:41.913699+0000","last_clean":"2024-04-14T08:42:41.913699+0000","last_became_active":"2024-04-14T08:42:39.902874+0000","last_became_peered":"2024-04-14T08:42:39.902874+0000","last_unstale":"2024-04-14T08:42:41.913699+0000","last_undegraded":"2024-04-14T08:42:41.913699+0000","last_fullsized":"2024-04-14T08:42:41.913699+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:34:54.764802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.915279+0000","last_change":"2024-04-14T08:42:40.913643+0000","last_active":"2024-04-14T08:42:41.915279+0000","last_peered":"2024-04-14T08:42:41.915279+0000","last_clean":"2024-04-14T08:42:41.915279+0000","last_became_active":"2024-04-14T08:42:40.912975+0000","last_became_peered":"2024-04-14T08:42:40.912975+0000","last_unstale":"2024-04-14T08:42:41.915279+0000","last_undegraded":"2024-04-14T08:42:41.915279+0000","last_fullsized":"2024-04-14T08:42:41.915279+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:39.880431+0000","last_clean_scrub_stamp":"2024-04-14T08:42:39.880431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:01:09.722380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"115'8","reported_seq":22,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.909001+0000","last_change":"2024-04-14T08:42:39.915077+0000","last_active":"2024-04-14T08:42:41.909001+0000","last_peered":"2024-04-14T08:42:41.909001+0000","last_clean":"2024-04-14T08:42:41.909001+0000","last_became_active":"2024-04-14T08:42:39.914803+0000","last_became_peered":"2024-04-14T08:42:39.914803+0000","last_unstale":"2024-04-14T08:42:41.909001+0000","last_undegraded":"2024-04-14T08:42:41.909001+0000","last_fullsized":"2024-04-14T08:42:41.909001+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:42:38.871062+0000","last_clean_scrub_stamp":"2024-04-14T08:42:38.871062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T12:47:33.457508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":365,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-14T08:42:41.914002+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:42:41.914002+0000","last_peered":"2024-04-14T08:42:41.914002+0000","last_clean":"2024-04-14T08:42:41.914002+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:42:41.914002+0000","last_undegraded":"2024-04-14T08:42:41.914002+0000","last_fullsized":"2024-04-14T08:42:41.914002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574972,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":2056192,"data_stored":1264964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574972,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1449984,"data_stored":674402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574972,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":673980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574972,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1458176,"data_stored":675432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574972,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":675466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574972,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1445888,"data_stored":673820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1449984,"data_stored":673842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574972,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33176,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338664,"statfs":{"total":96636764160,"available":96602791936,"internally_reserved":0,"allocated":2043904,"data_stored":1264210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:42:48.667 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-14T08:42:48.668 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-14T08:42:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.811+0000 7f7ec78b0640 1 Processor -- start 2024-04-14T08:42:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.811+0000 7f7ec78b0640 1 -- start start 2024-04-14T08:42:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.812+0000 7f7ec78b0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7ec013df30 0x7f7ec00b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.812+0000 7f7ec78b0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec0140e80 0x7f7ec00b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.812+0000 7f7ec78b0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ec015b620 con 0x7f7ec015c380 2024-04-14T08:42:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.812+0000 7f7ec78b0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f7ec00cf9b0 con 0x7f7ec013df30 2024-04-14T08:42:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.812+0000 7f7ec78b0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec01414a0 con 0x7f7ec0140e80 2024-04-14T08:42:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.812+0000 7f7ec564c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7ec013df30 0x7f7ec00b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.812+0000 7f7ec4e4b640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7ec015c380 0x7f7ec0140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34232/0 (socket says 172.21.15.172:34232) 2024-04-14T08:42:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.812+0000 7f7ec4e4b640 1 -- 172.21.15.172:0/3748244023 learned_addr learned my addr 172.21.15.172:0/3748244023 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.812+0000 7f7ec5e4d640 1 --2- 172.21.15.172:0/3748244023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec0140e80 0x7f7ec00b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.813+0000 7f7ec564c640 1 -- 172.21.15.172:0/3748244023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec0140e80 msgr2=0x7f7ec00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.813+0000 7f7ec564c640 1 --2- 172.21.15.172:0/3748244023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec0140e80 0x7f7ec00b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.813+0000 7f7ec564c640 1 -- 172.21.15.172:0/3748244023 >> v1:172.21.15.172:6789/0 conn(0x7f7ec015c380 legacy=0x7f7ec0140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.813+0000 7f7ec564c640 1 -- 172.21.15.172:0/3748244023 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec015b8a0 con 0x7f7ec013df30 2024-04-14T08:42:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.813+0000 7f7eb7fff640 1 -- 172.21.15.172:0/3748244023 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2470156276 0 0) 0x7f7eb0001110 con 0x7f7ec015c380 2024-04-14T08:42:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.813+0000 7f7ec5e4d640 1 --2- 172.21.15.172:0/3748244023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec0140e80 0x7f7ec00b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.813+0000 7f7ec564c640 1 --2- 172.21.15.172:0/3748244023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ec013df30 0x7f7ec00b1cd0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009d90 tx=0x7f7ebc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412fb8624326bbca server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.813+0000 7f7eb7fff640 1 -- 172.21.15.172:0/3748244023 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc039070 con 0x7f7ec013df30 2024-04-14T08:42:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.813+0000 7f7eb7fff640 1 -- 172.21.15.172:0/3748244023 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ebc02cc50 con 0x7f7ec013df30 2024-04-14T08:42:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.813+0000 7f7eb7fff640 1 -- 172.21.15.172:0/3748244023 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc034440 con 0x7f7ec013df30 2024-04-14T08:42:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.814+0000 7f7ec78b0640 1 -- 172.21.15.172:0/3748244023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ec013df30 msgr2=0x7f7ec00b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.814+0000 7f7ec78b0640 1 --2- 172.21.15.172:0/3748244023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ec013df30 0x7f7ec00b1cd0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009d90 tx=0x7f7ebc02bea0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.814+0000 7f7ec78b0640 1 -- 172.21.15.172:0/3748244023 shutdown_connections 2024-04-14T08:42:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.814+0000 7f7ec78b0640 1 --2- 172.21.15.172:0/3748244023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec0140e80 0x7f7ec00b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.814+0000 7f7ec78b0640 1 --2- 172.21.15.172:0/3748244023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ec013df30 0x7f7ec00b1cd0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.814+0000 7f7ec78b0640 1 -- 172.21.15.172:0/3748244023 >> 172.21.15.172:0/3748244023 conn(0x7f7ec006b520 msgr2=0x7f7ec006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.814+0000 7f7ec78b0640 1 -- 172.21.15.172:0/3748244023 shutdown_connections 2024-04-14T08:42:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.814+0000 7f7ec78b0640 1 -- 172.21.15.172:0/3748244023 wait complete. 2024-04-14T08:42:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.815+0000 7f7ec78b0640 1 Processor -- start 2024-04-14T08:42:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.815+0000 7f7ec78b0640 1 -- start start 2024-04-14T08:42:48.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec78b0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec013df30 0x7f7ec0150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec78b0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0140e80 0x7f7ec015a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec78b0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ec015c380 0x7f7ec015a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec78b0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec013e310 con 0x7f7ec015c380 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec78b0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec015be60 con 0x7f7ec0140e80 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec78b0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec00ce9a0 con 0x7f7ec013df30 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec5e4d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ec015c380 0x7f7ec015a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec5e4d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ec015c380 0x7f7ec015a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39942/0 (socket says 172.21.15.172:39942) 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec5e4d640 1 -- 172.21.15.172:0/1234453878 learned_addr learned my addr 172.21.15.172:0/1234453878 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec4e4b640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0140e80 0x7f7ec015a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec564c640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec013df30 0x7f7ec0150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec5e4d640 1 -- 172.21.15.172:0/1234453878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec013df30 msgr2=0x7f7ec0150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec5e4d640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec013df30 0x7f7ec0150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec5e4d640 1 -- 172.21.15.172:0/1234453878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0140e80 msgr2=0x7f7ec015a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec5e4d640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0140e80 0x7f7ec015a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec5e4d640 1 -- 172.21.15.172:0/1234453878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec015d490 con 0x7f7ec015c380 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.816+0000 7f7ec5e4d640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ec015c380 0x7f7ec015a7d0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800e930 tx=0x7f7eb800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.817+0000 7f7eb5ffb640 1 -- 172.21.15.172:0/1234453878 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8017070 con 0x7f7ec015c380 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.817+0000 7f7eb5ffb640 1 -- 172.21.15.172:0/1234453878 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb800ca00 con 0x7f7ec015c380 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.817+0000 7f7ec78b0640 1 -- 172.21.15.172:0/1234453878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec015d770 con 0x7f7ec015c380 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.817+0000 7f7eb5ffb640 1 -- 172.21.15.172:0/1234453878 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb800cdd0 con 0x7f7ec015c380 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.817+0000 7f7ec78b0640 1 -- 172.21.15.172:0/1234453878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec0136850 con 0x7f7ec015c380 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.818+0000 7f7ec78b0640 1 -- 172.21.15.172:0/1234453878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec0068590 con 0x7f7ec015c380 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.818+0000 7f7eb5ffb640 1 -- 172.21.15.172:0/1234453878 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7eb8013470 con 0x7f7ec015c380 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.818+0000 7f7eb5ffb640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7ea406c8f0 0x7f7ea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.819+0000 7f7eb5ffb640 1 -- 172.21.15.172:0/1234453878 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f7eb808bb90 con 0x7f7ec015c380 2024-04-14T08:42:48.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.819+0000 7f7ec564c640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7ea406c8f0 0x7f7ea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:48.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.819+0000 7f7ec564c640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7ea406c8f0 0x7f7ea406edb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc036000 tx=0x7f7ebc030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:48.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:48.823+0000 7f7eb5ffb640 1 -- 172.21.15.172:0/1234453878 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7eb80598a0 con 0x7f7ec015c380 2024-04-14T08:42:49.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.069+0000 7f7ec78b0640 1 -- 172.21.15.172:0/1234453878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7ec0071650 con 0x7f7ec015c380 2024-04-14T08:42:49.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.071+0000 7f7eb5ffb640 1 -- 172.21.15.172:0/1234453878 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f7eb80600c0 con 0x7f7ec015c380 2024-04-14T08:42:49.071 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2024-04-14T08:42:49.071 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQB7lBtmYCq1JRAAKd6E+opbcdOS+YZnbcuIFA== 2024-04-14T08:42:49.071 INFO:teuthology.orchestra.run.smithi172.stdout: caps mds = "allow" 2024-04-14T08:42:49.071 INFO:teuthology.orchestra.run.smithi172.stdout: caps mon = "allow r" 2024-04-14T08:42:49.071 INFO:teuthology.orchestra.run.smithi172.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-14T08:42:49.072 INFO:teuthology.orchestra.run.smithi172.stderr:updated caps for client.0 2024-04-14T08:42:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.073+0000 7f7ec78b0640 1 -- 172.21.15.172:0/1234453878 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7ea406c8f0 msgr2=0x7f7ea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.073+0000 7f7ec78b0640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7ea406c8f0 0x7f7ea406edb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc036000 tx=0x7f7ebc030000 comp rx=0 tx=0).stop 2024-04-14T08:42:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.073+0000 7f7ec78b0640 1 -- 172.21.15.172:0/1234453878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ec015c380 msgr2=0x7f7ec015a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.073+0000 7f7ec78b0640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ec015c380 0x7f7ec015a7d0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800e930 tx=0x7f7eb800edf0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.073+0000 7f7ec78b0640 1 -- 172.21.15.172:0/1234453878 shutdown_connections 2024-04-14T08:42:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.073+0000 7f7ec78b0640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7ea406c8f0 0x7f7ea406edb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.073+0000 7f7ec78b0640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ec015c380 0x7f7ec015a7d0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.073+0000 7f7ec78b0640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0140e80 0x7f7ec015a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.073+0000 7f7ec78b0640 1 --2- 172.21.15.172:0/1234453878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ec013df30 0x7f7ec0150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.073+0000 7f7ec78b0640 1 -- 172.21.15.172:0/1234453878 >> 172.21.15.172:0/1234453878 conn(0x7f7ec006b520 msgr2=0x7f7ec015eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.073+0000 7f7ec78b0640 1 -- 172.21.15.172:0/1234453878 shutdown_connections 2024-04-14T08:42:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.073+0000 7f7ec78b0640 1 -- 172.21.15.172:0/1234453878 wait complete. 2024-04-14T08:42:49.084 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:42:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.185+0000 7f514151e640 1 Processor -- start 2024-04-14T08:42:49.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.185+0000 7f514151e640 1 -- start start 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f514151e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f513c0d2830 0x7f513c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f514151e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f513c0cedd0 0x7f513c0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f514151e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f513c0d3150 con 0x7f513c15e890 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f514151e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f513c0cff30 con 0x7f513c0d2830 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f514151e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f513c0d2350 con 0x7f513c0cedd0 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f513affd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f513c0d2830 0x7f513c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f513a7fc640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f513c15e890 0x7f513c0d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:34244/0 (socket says 172.21.15.172:34244) 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f513a7fc640 1 -- 172.21.15.172:0/4250694659 learned_addr learned my addr 172.21.15.172:0/4250694659 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f513b7fe640 1 --2- 172.21.15.172:0/4250694659 >> v2:172.21.15.177:3300/0 conn(0x7f513c0cedd0 0x7f513c0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f513affd640 1 -- 172.21.15.172:0/4250694659 >> v2:172.21.15.177:3300/0 conn(0x7f513c0cedd0 msgr2=0x7f513c0b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f513affd640 1 --2- 172.21.15.172:0/4250694659 >> v2:172.21.15.177:3300/0 conn(0x7f513c0cedd0 0x7f513c0b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f513affd640 1 -- 172.21.15.172:0/4250694659 >> v1:172.21.15.172:6789/0 conn(0x7f513c15e890 legacy=0x7f513c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f513affd640 1 -- 172.21.15.172:0/4250694659 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513c0cf320 con 0x7f513c0d2830 2024-04-14T08:42:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.186+0000 7f5139ffb640 1 -- 172.21.15.172:0/4250694659 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 267936776 0 0) 0x7f513c0d3150 con 0x7f513c15e890 2024-04-14T08:42:49.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.187+0000 7f513affd640 1 --2- 172.21.15.172:0/4250694659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f513c0d2830 0x7f513c0d2c10 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f513c08e4b0 tx=0x7f512c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4819f9eba2671121 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.187+0000 7f5139ffb640 1 -- 172.21.15.172:0/4250694659 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c034070 con 0x7f513c0d2830 2024-04-14T08:42:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.187+0000 7f5139ffb640 1 -- 172.21.15.172:0/4250694659 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f512c030610 con 0x7f513c0d2830 2024-04-14T08:42:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.187+0000 7f5139ffb640 1 -- 172.21.15.172:0/4250694659 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c0309e0 con 0x7f513c0d2830 2024-04-14T08:42:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.187+0000 7f514151e640 1 -- 172.21.15.172:0/4250694659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f513c0d2830 msgr2=0x7f513c0d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.187+0000 7f514151e640 1 --2- 172.21.15.172:0/4250694659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f513c0d2830 0x7f513c0d2c10 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f513c08e4b0 tx=0x7f512c02b960 comp rx=0 tx=0).stop 2024-04-14T08:42:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.188+0000 7f514151e640 1 -- 172.21.15.172:0/4250694659 shutdown_connections 2024-04-14T08:42:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.188+0000 7f514151e640 1 --2- 172.21.15.172:0/4250694659 >> v2:172.21.15.177:3300/0 conn(0x7f513c0cedd0 0x7f513c0b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.188+0000 7f514151e640 1 --2- 172.21.15.172:0/4250694659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f513c0d2830 0x7f513c0d2c10 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.188+0000 7f514151e640 1 -- 172.21.15.172:0/4250694659 >> 172.21.15.172:0/4250694659 conn(0x7f513c09d590 msgr2=0x7f513c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.188+0000 7f514151e640 1 -- 172.21.15.172:0/4250694659 shutdown_connections 2024-04-14T08:42:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.188+0000 7f514151e640 1 -- 172.21.15.172:0/4250694659 wait complete. 2024-04-14T08:42:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.189+0000 7f514151e640 1 Processor -- start 2024-04-14T08:42:49.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.189+0000 7f514151e640 1 -- start start 2024-04-14T08:42:49.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f514151e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f513c0cedd0 0x7f513c0cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f514151e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f513c0d2830 0x7f513c0b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f514151e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f513c15e890 0x7f513c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f514151e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f513c0cfbc0 con 0x7f513c0d2830 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f514151e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f513c0cf8e0 con 0x7f513c15e890 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f514151e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f513c0b3c50 con 0x7f513c0cedd0 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f513a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f513c0d2830 0x7f513c0b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f513a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f513c0d2830 0x7f513c0b6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39952/0 (socket says 172.21.15.172:39952) 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f513a7fc640 1 -- 172.21.15.172:0/2216894753 learned_addr learned my addr 172.21.15.172:0/2216894753 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f513b7fe640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f513c15e890 0x7f513c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f513affd640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f513c0cedd0 0x7f513c0cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f513a7fc640 1 -- 172.21.15.172:0/2216894753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f513c0cedd0 msgr2=0x7f513c0cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f513a7fc640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f513c0cedd0 0x7f513c0cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f513a7fc640 1 -- 172.21.15.172:0/2216894753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f513c15e890 msgr2=0x7f513c0b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f513a7fc640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f513c15e890 0x7f513c0b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f513a7fc640 1 -- 172.21.15.172:0/2216894753 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513c15eca0 con 0x7f513c0d2830 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f513a7fc640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f513c0d2830 0x7f513c0b6480 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f513000c460 tx=0x7f513000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f51237fe640 1 -- 172.21.15.172:0/2216894753 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130018070 con 0x7f513c0d2830 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f51237fe640 1 -- 172.21.15.172:0/2216894753 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5130007500 con 0x7f513c0d2830 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f51237fe640 1 -- 172.21.15.172:0/2216894753 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130004a20 con 0x7f513c0d2830 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f514151e640 1 -- 172.21.15.172:0/2216894753 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f513c15ef60 con 0x7f513c0d2830 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.190+0000 7f514151e640 1 -- 172.21.15.172:0/2216894753 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f513c0c7d80 con 0x7f513c0d2830 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.191+0000 7f514151e640 1 -- 172.21.15.172:0/2216894753 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f513c0ca340 con 0x7f513c0d2830 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.192+0000 7f51237fe640 1 -- 172.21.15.172:0/2216894753 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f513001b070 con 0x7f513c0d2830 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.192+0000 7f51237fe640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f512806c920 0x7f512806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.192+0000 7f51237fe640 1 -- 172.21.15.172:0/2216894753 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f513008ae50 con 0x7f513c0d2830 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.192+0000 7f513affd640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f512806c920 0x7f512806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.193+0000 7f513affd640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f512806c920 0x7f512806ede0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f512c036000 tx=0x7f512c03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:49.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.197+0000 7f51237fe640 1 -- 172.21.15.172:0/2216894753 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f513c0ca340 con 0x7f513c0d2830 2024-04-14T08:42:49.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.409+0000 7f514151e640 1 -- 172.21.15.172:0/2216894753 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f513c068520 con 0x7f513c0d2830 2024-04-14T08:42:49.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.410+0000 7f51237fe640 1 -- 172.21.15.172:0/2216894753 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5192 (secure 0 0 0) 0x7f5130058aa0 con 0x7f513c0d2830 2024-04-14T08:42:49.410 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:49.411 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":88,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9048,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/744853572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":744853572},{"type":"v1","addr":"172.21.15.172:6837","nonce":744853572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:42:42.249444+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9060},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9060":{"gid":9060,"name":"d","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/3201353747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3201353747},{"type":"v1","addr":"172.21.15.177:6835","nonce":3201353747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9060,"qdb_cluster":[9060]},"id":8}]} 2024-04-14T08:42:49.411 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 88 2024-04-14T08:42:49.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.412+0000 7f514151e640 1 -- 172.21.15.172:0/2216894753 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f512806c920 msgr2=0x7f512806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.412+0000 7f514151e640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f512806c920 0x7f512806ede0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f512c036000 tx=0x7f512c03b040 comp rx=0 tx=0).stop 2024-04-14T08:42:49.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.412+0000 7f514151e640 1 -- 172.21.15.172:0/2216894753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f513c0d2830 msgr2=0x7f513c0b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.412+0000 7f514151e640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f513c0d2830 0x7f513c0b6480 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f513000c460 tx=0x7f513000c920 comp rx=0 tx=0).stop 2024-04-14T08:42:49.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.412+0000 7f514151e640 1 -- 172.21.15.172:0/2216894753 shutdown_connections 2024-04-14T08:42:49.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.412+0000 7f514151e640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f512806c920 0x7f512806ede0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.412+0000 7f514151e640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f513c15e890 0x7f513c0b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.412+0000 7f514151e640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f513c0d2830 0x7f513c0b6480 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.412+0000 7f514151e640 1 --2- 172.21.15.172:0/2216894753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f513c0cedd0 0x7f513c0cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.412+0000 7f514151e640 1 -- 172.21.15.172:0/2216894753 >> 172.21.15.172:0/2216894753 conn(0x7f513c09d590 msgr2=0x7f513c0d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:49.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.412+0000 7f514151e640 1 -- 172.21.15.172:0/2216894753 shutdown_connections 2024-04-14T08:42:49.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.412+0000 7f514151e640 1 -- 172.21.15.172:0/2216894753 wait complete. 2024-04-14T08:42:49.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:42:40.251538+0000', 'modified': '2024-04-14T08:42:42.249444+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9060}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9060': {'gid': 9060, 'name': 'd', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6835/3201353747', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6833', 'nonce': 3201353747}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 3201353747}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9060, 'qdb_cluster': [9060]} 2024-04-14T08:42:49.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:42:49.826 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-14T08:42:49.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.933+0000 7f36245bb640 1 Processor -- start 2024-04-14T08:42:49.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.933+0000 7f36245bb640 1 -- start start 2024-04-14T08:42:49.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.934+0000 7f36245bb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f361c15e430 0x7f361c162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.934+0000 7f36245bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f361c162540 0x7f361c1629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.934+0000 7f36245bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f361c0b3a30 0x7f361c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.934+0000 7f36245bb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f361c0cefb0 con 0x7f361c162540 2024-04-14T08:42:49.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.934+0000 7f36245bb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f361c0cf290 con 0x7f361c15e430 2024-04-14T08:42:49.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.934+0000 7f36245bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f361c0cf570 con 0x7f361c0b3a30 2024-04-14T08:42:49.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.935+0000 7f3621b56640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f361c162540 0x7f361c1629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.935+0000 7f3621b56640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f361c162540 0x7f361c1629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45274/0 (socket says 172.21.15.172:45274) 2024-04-14T08:42:49.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.935+0000 7f3621b56640 1 -- 172.21.15.172:0/898848480 learned_addr learned my addr 172.21.15.172:0/898848480 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:49.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.935+0000 7f3622357640 1 --2- 172.21.15.172:0/898848480 >> v2:172.21.15.177:3300/0 conn(0x7f361c15e430 0x7f361c162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.935+0000 7f3622b58640 1 --2- 172.21.15.172:0/898848480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f361c0b3a30 0x7f361c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.935+0000 7f3621b56640 1 -- 172.21.15.172:0/898848480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f361c0b3a30 msgr2=0x7f361c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:49.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.935+0000 7f3621b56640 1 --2- 172.21.15.172:0/898848480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f361c0b3a30 0x7f361c0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.935+0000 7f3621b56640 1 -- 172.21.15.172:0/898848480 >> v2:172.21.15.177:3300/0 conn(0x7f361c15e430 msgr2=0x7f361c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:49.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.935+0000 7f3621b56640 1 --2- 172.21.15.172:0/898848480 >> v2:172.21.15.177:3300/0 conn(0x7f361c15e430 0x7f361c162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.935+0000 7f3621b56640 1 -- 172.21.15.172:0/898848480 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c0cf850 con 0x7f361c162540 2024-04-14T08:42:49.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.936+0000 7f3621b56640 1 --2- 172.21.15.172:0/898848480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361c162540 0x7f361c1629d0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f361c0b6db0 tx=0x7f360c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba9a93070a07033 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:49.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.936+0000 7f3621355640 1 -- 172.21.15.172:0/898848480 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c035020 con 0x7f361c162540 2024-04-14T08:42:49.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.936+0000 7f3621355640 1 -- 172.21.15.172:0/898848480 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c031870 con 0x7f361c162540 2024-04-14T08:42:49.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.936+0000 7f3621355640 1 -- 172.21.15.172:0/898848480 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c031c40 con 0x7f361c162540 2024-04-14T08:42:49.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.936+0000 7f36245bb640 1 -- 172.21.15.172:0/898848480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361c162540 msgr2=0x7f361c1629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.936+0000 7f36245bb640 1 --2- 172.21.15.172:0/898848480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361c162540 0x7f361c1629d0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f361c0b6db0 tx=0x7f360c02b910 comp rx=0 tx=0).stop 2024-04-14T08:42:49.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.936+0000 7f36245bb640 1 -- 172.21.15.172:0/898848480 shutdown_connections 2024-04-14T08:42:49.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.936+0000 7f36245bb640 1 --2- 172.21.15.172:0/898848480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f361c0b3a30 0x7f361c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.936+0000 7f36245bb640 1 --2- 172.21.15.172:0/898848480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361c162540 0x7f361c1629d0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.936+0000 7f36245bb640 1 --2- 172.21.15.172:0/898848480 >> v2:172.21.15.177:3300/0 conn(0x7f361c15e430 0x7f361c162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.936+0000 7f36245bb640 1 -- 172.21.15.172:0/898848480 >> 172.21.15.172:0/898848480 conn(0x7f361c09d560 msgr2=0x7f361c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:49.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.937+0000 7f36245bb640 1 -- 172.21.15.172:0/898848480 shutdown_connections 2024-04-14T08:42:49.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.937+0000 7f36245bb640 1 -- 172.21.15.172:0/898848480 wait complete. 2024-04-14T08:42:49.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.938+0000 7f36245bb640 1 Processor -- start 2024-04-14T08:42:49.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.938+0000 7f36245bb640 1 -- start start 2024-04-14T08:42:49.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.938+0000 7f36245bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361c0b3a30 0x7f361c14e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.938+0000 7f36245bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f361c15e430 0x7f361c0b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.938+0000 7f36245bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f361c162540 0x7f361c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.938+0000 7f36245bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f361c0d4e20 con 0x7f361c0b3a30 2024-04-14T08:42:49.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.938+0000 7f36245bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f361c15e810 con 0x7f361c15e430 2024-04-14T08:42:49.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.938+0000 7f36245bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f361c0d0050 con 0x7f361c162540 2024-04-14T08:42:49.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.938+0000 7f3622357640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361c0b3a30 0x7f361c14e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.938+0000 7f3622357640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361c0b3a30 0x7f361c14e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45286/0 (socket says 172.21.15.172:45286) 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f3622357640 1 -- 172.21.15.172:0/1768893537 learned_addr learned my addr 172.21.15.172:0/1768893537 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f3621b56640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f361c15e430 0x7f361c0b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f3622b58640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f361c162540 0x7f361c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f3622357640 1 -- 172.21.15.172:0/1768893537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f361c162540 msgr2=0x7f361c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f3622357640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f361c162540 0x7f361c15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f3622357640 1 -- 172.21.15.172:0/1768893537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f361c15e430 msgr2=0x7f361c0b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f3622357640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f361c15e430 0x7f361c0b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f3622357640 1 -- 172.21.15.172:0/1768893537 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c14eba0 con 0x7f361c0b3a30 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f3622357640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361c0b3a30 0x7f361c14e660 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f361800e8a0 tx=0x7f361800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f3612ffd640 1 -- 172.21.15.172:0/1768893537 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618017070 con 0x7f361c0b3a30 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f3612ffd640 1 -- 172.21.15.172:0/1768893537 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361800c8e0 con 0x7f361c0b3a30 2024-04-14T08:42:49.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f36245bb640 1 -- 172.21.15.172:0/1768893537 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c0cb490 con 0x7f361c0b3a30 2024-04-14T08:42:49.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f3612ffd640 1 -- 172.21.15.172:0/1768893537 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361800ccd0 con 0x7f361c0b3a30 2024-04-14T08:42:49.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.939+0000 7f36245bb640 1 -- 172.21.15.172:0/1768893537 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c0cba50 con 0x7f361c0b3a30 2024-04-14T08:42:49.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.941+0000 7f3612ffd640 1 -- 172.21.15.172:0/1768893537 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f361801a070 con 0x7f361c0b3a30 2024-04-14T08:42:49.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.941+0000 7f3612ffd640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f360006c8f0 0x7f360006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.941+0000 7f3612ffd640 1 -- 172.21.15.172:0/1768893537 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f361808ba00 con 0x7f361c0b3a30 2024-04-14T08:42:49.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.941+0000 7f3621b56640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f360006c8f0 0x7f360006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.942+0000 7f3621b56640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f360006c8f0 0x7f360006edb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f360c036000 tx=0x7f360c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.942+0000 7f36273a0740 1 Processor -- start 2024-04-14T08:42:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.942+0000 7f36273a0740 1 -- start start 2024-04-14T08:42:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.943+0000 7f36273a0740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cb60 0x55ce69b5fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.943+0000 7f36273a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 0x55ce69b8f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.943+0000 7f36273a0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b60040 0x55ce69b8fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.943+0000 7f36273a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ce69ac9030 con 0x55ce69b8cf40 2024-04-14T08:42:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.943+0000 7f36273a0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ce69a71620 con 0x55ce69b60040 2024-04-14T08:42:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.943+0000 7f36273a0740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ce69ac4fe0 con 0x55ce69b8cb60 2024-04-14T08:42:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.943+0000 7f3621b56640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 0x55ce69b8f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.943+0000 7f3621b56640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 0x55ce69b8f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45302/0 (socket says 172.21.15.172:45302) 2024-04-14T08:42:49.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.943+0000 7f3622b58640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b60040 0x55ce69b8fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.943+0000 7f3621b56640 1 -- 172.21.15.172:0/685316643 learned_addr learned my addr 172.21.15.172:0/685316643 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.943+0000 7f3622357640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cb60 0x55ce69b5fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.944+0000 7f3621b56640 1 -- 172.21.15.172:0/685316643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cb60 msgr2=0x55ce69b5fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.944+0000 7f3621b56640 1 --2- 172.21.15.172:0/685316643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cb60 0x55ce69b5fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.944+0000 7f3621b56640 1 -- 172.21.15.172:0/685316643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b60040 msgr2=0x55ce69b8fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.944+0000 7f3621b56640 1 --2- 172.21.15.172:0/685316643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b60040 0x55ce69b8fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.944+0000 7f3621b56640 1 -- 172.21.15.172:0/685316643 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce697dd2d0 con 0x55ce69b8cf40 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.944+0000 7f3622357640 1 --2- 172.21.15.172:0/685316643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cb60 0x55ce69b5fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.944+0000 7f3621b56640 1 --2- 172.21.15.172:0/685316643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 0x55ce69b8f8f0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f360c009960 tx=0x7f360c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9956087459347801 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.944+0000 7f35fffff640 1 -- 172.21.15.172:0/685316643 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c053070 con 0x55ce69b8cf40 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.944+0000 7f35fffff640 1 -- 172.21.15.172:0/685316643 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c0448d0 con 0x55ce69b8cf40 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.944+0000 7f35fffff640 1 -- 172.21.15.172:0/685316643 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c035040 con 0x55ce69b8cf40 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.945+0000 7f36273a0740 1 -- 172.21.15.172:0/685316643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 msgr2=0x55ce69b8f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.945+0000 7f36273a0740 1 --2- 172.21.15.172:0/685316643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 0x55ce69b8f8f0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f360c009960 tx=0x7f360c030000 comp rx=0 tx=0).stop 2024-04-14T08:42:49.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.945+0000 7f36273a0740 1 -- 172.21.15.172:0/685316643 shutdown_connections 2024-04-14T08:42:49.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.945+0000 7f36273a0740 1 --2- 172.21.15.172:0/685316643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b60040 0x55ce69b8fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.945+0000 7f36273a0740 1 --2- 172.21.15.172:0/685316643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 0x55ce69b8f8f0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.945+0000 7f36273a0740 1 --2- 172.21.15.172:0/685316643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cb60 0x55ce69b5fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.945+0000 7f36273a0740 1 -- 172.21.15.172:0/685316643 >> 172.21.15.172:0/685316643 conn(0x55ce69b61d70 msgr2=0x55ce69baa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:49.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.945+0000 7f36273a0740 1 -- 172.21.15.172:0/685316643 shutdown_connections 2024-04-14T08:42:49.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.945+0000 7f36273a0740 1 -- 172.21.15.172:0/685316643 wait complete. 2024-04-14T08:42:49.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.945+0000 7f36273a0740 1 -- start start 2024-04-14T08:42:49.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.945+0000 7f36273a0740 1 Processor -- start 2024-04-14T08:42:49.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.946+0000 7f36273a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cb60 0x55ce69b79a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.946+0000 7f36273a0740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cf40 0x55ce69b7cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.946+0000 7f36273a0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b926e0 0x55ce69b96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.946+0000 7f36273a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ce69b64410 con 0x55ce69b8cb60 2024-04-14T08:42:49.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.946+0000 7f36273a0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ce69a849e0 con 0x55ce69b926e0 2024-04-14T08:42:49.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.946+0000 7f36273a0740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ce69b66c50 con 0x55ce69b8cf40 2024-04-14T08:42:49.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.946+0000 7f36273a0740 10 client.0 _ll_register_callbacks cb 0x55ce69ac1080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:42:49.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.946+0000 7f3622357640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cb60 0x55ce69b79a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.946+0000 7f3622357640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cb60 0x55ce69b79a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45310/0 (socket says 172.21.15.172:45310) 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.946+0000 7f3622357640 1 -- 172.21.15.172:0/2183484869 learned_addr learned my addr 172.21.15.172:0/2183484869 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.946+0000 7f3621b56640 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cf40 0x55ce69b7cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.946+0000 7f3622b58640 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b926e0 0x55ce69b96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f3622357640 1 -- 172.21.15.172:0/2183484869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cf40 msgr2=0x55ce69b7cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f3622357640 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cf40 0x55ce69b7cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f3622357640 1 -- 172.21.15.172:0/2183484869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b926e0 msgr2=0x55ce69b96a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f3622357640 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b926e0 0x55ce69b96a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f3622357640 1 -- 172.21.15.172:0/2183484869 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce698a76b0 con 0x55ce69b8cb60 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f3622357640 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cb60 0x55ce69b79a90 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f361800e870 tx=0x7f36180597a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfcff914dec2294 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f35fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f35fd7fa640 1 -- 172.21.15.172:0/2183484869 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618088060 con 0x55ce69b8cb60 2024-04-14T08:42:49.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f35fd7fa640 1 -- 172.21.15.172:0/2183484869 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f361805ed70 con 0x55ce69b8cb60 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f36273a0740 1 -- 172.21.15.172:0/2183484869 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f360c0448d0 con 0x55ce69b8cb60 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f35fd7fa640 1 -- 172.21.15.172:0/2183484869 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f3618021040 con 0x55ce69b8cb60 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f36273a0740 10 client.9228 fetch_fsmap learned FSMap version 88 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f36273a0740 1 -- 172.21.15.172:0/2183484869 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55ce69b7ad10 con 0x55ce69b8cb60 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f35fd7fa640 1 -- 172.21.15.172:0/2183484869 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c0448d0 con 0x55ce69b8cb60 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f35fd7fa640 1 -- 172.21.15.172:0/2183484869 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f3618024060 con 0x55ce69b8cb60 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f36273a0740 10 client.9228 fetch_fsmap finished waiting for FSMap version 88 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f36273a0740 10 client.9228 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f36273a0740 20 client.9228 populate_metadata read hostname 'smithi172' 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f36273a0740 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55ce69b7a5c0 0x55ce69bbec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f36273a0740 4 client.9228 mds_command: new command op to 9060 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.947+0000 7f36273a0740 1 -- 172.21.15.172:0/2183484869 --> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce69a70cc0 con 0x55ce69b7a5c0 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.948+0000 7f3621b56640 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55ce69b7a5c0 0x55ce69bbec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.948+0000 7f35fd7fa640 10 client.9228 ms_handle_connect on v2:172.21.15.177:6833/3201353747 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.948+0000 7f3621b56640 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55ce69b7a5c0 0x55ce69bbec80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2cfd5b2aa71756a server_cookie=8e31c594928077e in_seq=0 out_seq=0 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f35fd7fa640 1 -- 172.21.15.172:0/2183484869 <== mds.0 v2:172.21.15.177:6833/3201353747 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce69a70cc0 con 0x55ce69b7a5c0 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f35fd7fa640 10 client.9228 handle_command_reply: tid=0 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f36273a0740 1 client.9228 shutdown 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f36273a0740 1 -- 172.21.15.172:0/2183484869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cb60 msgr2=0x55ce69b79a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f36273a0740 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cb60 0x55ce69b79a90 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f361800e870 tx=0x7f36180597a0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f36273a0740 1 -- 172.21.15.172:0/2183484869 shutdown_connections 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f36273a0740 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55ce69b7a5c0 0x55ce69bbec80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f36273a0740 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b926e0 0x55ce69b96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f36273a0740 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cf40 0x55ce69b7cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f36273a0740 1 --2- 172.21.15.172:0/2183484869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cb60 0x55ce69b79a90 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f35fd7fa640 0 client.9228 ms_handle_reset on v2:172.21.15.177:6833/3201353747 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f36273a0740 1 -- 172.21.15.172:0/2183484869 >> 172.21.15.172:0/2183484869 conn(0x55ce69b60040 msgr2=0x55ce69b63a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:49.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f36273a0740 1 -- 172.21.15.172:0/2183484869 shutdown_connections 2024-04-14T08:42:49.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f36273a0740 1 -- 172.21.15.172:0/2183484869 wait complete. 2024-04-14T08:42:49.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.949+0000 7f36273a0740 20 client.9228 trim_cache size 0 max 16384 2024-04-14T08:42:49.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.967+0000 7f36273a0740 1 Processor -- start 2024-04-14T08:42:49.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.967+0000 7f36273a0740 1 -- start start 2024-04-14T08:42:49.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f36273a0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b8cb60 0x55ce69baddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f36273a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 0x55ce69bba460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f36273a0740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b926e0 0x55ce69b93070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f36273a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f361c200120 con 0x55ce69b8cf40 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f36273a0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f361c0cdcc0 con 0x55ce69b8cb60 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f36273a0740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f361c1ffb90 con 0x55ce69b926e0 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f3621b56640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 0x55ce69bba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f3621b56640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 0x55ce69bba460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45320/0 (socket says 172.21.15.172:45320) 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f3621b56640 1 -- 172.21.15.172:0/4273708289 learned_addr learned my addr 172.21.15.172:0/4273708289 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f3622b58640 1 --2- 172.21.15.172:0/4273708289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b926e0 0x55ce69b93070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f3622357640 1 --2- 172.21.15.172:0/4273708289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b8cb60 0x55ce69baddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f3621b56640 1 -- 172.21.15.172:0/4273708289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b926e0 msgr2=0x55ce69b93070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f3621b56640 1 --2- 172.21.15.172:0/4273708289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b926e0 0x55ce69b93070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f3621b56640 1 -- 172.21.15.172:0/4273708289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b8cb60 msgr2=0x55ce69baddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f3621b56640 1 --2- 172.21.15.172:0/4273708289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b8cb60 0x55ce69baddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f3621b56640 1 -- 172.21.15.172:0/4273708289 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce69bbeff0 con 0x55ce69b8cf40 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f3621b56640 1 --2- 172.21.15.172:0/4273708289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 0x55ce69bba460 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f360c0090b0 tx=0x7f360c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3feffb2a4919aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f35ff7fe640 1 -- 172.21.15.172:0/4273708289 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c05e070 con 0x55ce69b8cf40 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f35ff7fe640 1 -- 172.21.15.172:0/4273708289 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c009490 con 0x55ce69b8cf40 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.968+0000 7f35ff7fe640 1 -- 172.21.15.172:0/4273708289 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c053040 con 0x55ce69b8cf40 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.969+0000 7f36273a0740 1 -- 172.21.15.172:0/4273708289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 msgr2=0x55ce69bba460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.969+0000 7f36273a0740 1 --2- 172.21.15.172:0/4273708289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 0x55ce69bba460 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f360c0090b0 tx=0x7f360c009240 comp rx=0 tx=0).stop 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.969+0000 7f36273a0740 1 -- 172.21.15.172:0/4273708289 shutdown_connections 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.969+0000 7f36273a0740 1 --2- 172.21.15.172:0/4273708289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b926e0 0x55ce69b93070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.969+0000 7f36273a0740 1 --2- 172.21.15.172:0/4273708289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b8cf40 0x55ce69bba460 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.969+0000 7f36273a0740 1 --2- 172.21.15.172:0/4273708289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b8cb60 0x55ce69baddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.969+0000 7f36273a0740 1 -- 172.21.15.172:0/4273708289 >> 172.21.15.172:0/4273708289 conn(0x55ce69b60040 msgr2=0x55ce69b5ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.969+0000 7f36273a0740 1 -- 172.21.15.172:0/4273708289 shutdown_connections 2024-04-14T08:42:49.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.969+0000 7f36273a0740 1 -- 172.21.15.172:0/4273708289 wait complete. 2024-04-14T08:42:49.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.969+0000 7f36273a0740 1 -- start start 2024-04-14T08:42:49.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.970+0000 7f36273a0740 1 Processor -- start 2024-04-14T08:42:49.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.970+0000 7f36273a0740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cb60 0x55ce69c0a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.970+0000 7f36273a0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b8cf40 0x55ce69c0b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.970+0000 7f36273a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b926e0 0x55ce69b8b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.970+0000 7f36273a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ce69b7a440 con 0x55ce69b926e0 2024-04-14T08:42:49.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.970+0000 7f36273a0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ce69b91f70 con 0x55ce69b8cf40 2024-04-14T08:42:49.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.970+0000 7f36273a0740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55ce69b7aa90 con 0x55ce69b8cb60 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.970+0000 7f36273a0740 10 client.0 _ll_register_callbacks cb 0x7f36180046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.970+0000 7f3622b58640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b926e0 0x55ce69b8b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.970+0000 7f3622b58640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b926e0 0x55ce69b8b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45326/0 (socket says 172.21.15.172:45326) 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.970+0000 7f3622b58640 1 -- 172.21.15.172:0/2884820753 learned_addr learned my addr 172.21.15.172:0/2884820753 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f3622357640 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cb60 0x55ce69c0a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f3621b56640 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b8cf40 0x55ce69c0b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f3622b58640 1 -- 172.21.15.172:0/2884820753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cb60 msgr2=0x55ce69c0a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f3622b58640 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cb60 0x55ce69c0a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f3622b58640 1 -- 172.21.15.172:0/2884820753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b8cf40 msgr2=0x55ce69c0b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f3622b58640 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b8cf40 0x55ce69c0b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f3622b58640 1 -- 172.21.15.172:0/2884820753 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce69bce310 con 0x55ce69b926e0 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f3622b58640 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b926e0 0x55ce69b8b150 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x55ce69b62d00 tx=0x7f36140177c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d9c80cb6559699 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f35fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:42:49.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f35fd7fa640 1 -- 172.21.15.172:0/2884820753 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614003990 con 0x55ce69b926e0 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f35fd7fa640 1 -- 172.21.15.172:0/2884820753 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f3614023c60 con 0x55ce69b926e0 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f36273a0740 1 -- 172.21.15.172:0/2884820753 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f360c009490 con 0x55ce69b926e0 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f35fd7fa640 1 -- 172.21.15.172:0/2884820753 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f3614023e60 con 0x55ce69b926e0 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f36273a0740 10 client.9234 fetch_fsmap learned FSMap version 88 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f36273a0740 1 -- 172.21.15.172:0/2884820753 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55ce69c0b6f0 con 0x55ce69b926e0 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f35fd7fa640 1 -- 172.21.15.172:0/2884820753 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c009490 con 0x55ce69b926e0 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f35fd7fa640 1 -- 172.21.15.172:0/2884820753 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f361402a050 con 0x55ce69b926e0 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f36273a0740 10 client.9234 fetch_fsmap finished waiting for FSMap version 88 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f36273a0740 10 client.9234 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f36273a0740 20 client.9234 populate_metadata read hostname 'smithi172' 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.971+0000 7f36273a0740 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55ce69b80080 0x55ce69babab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 4 client.9234 mds_command: new command op to 9060 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 1 -- 172.21.15.172:0/2884820753 --> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] -- command(tid 0: {"prefix": "status"}) -- 0x55ce69ad6570 con 0x55ce69b80080 2024-04-14T08:42:49.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f3622357640 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55ce69b80080 0x55ce69babab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f35fd7fa640 10 client.9234 ms_handle_connect on v2:172.21.15.177:6833/3201353747 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f3622357640 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55ce69b80080 0x55ce69babab0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39befa0eeb2ab3de server_cookie=ba40bfcdf8e0a7c9 in_seq=0 out_seq=0 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f35fd7fa640 1 -- 172.21.15.172:0/2884820753 <== mds.0 v2:172.21.15.177:6833/3201353747 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55ce69ad6570 con 0x55ce69b80080 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f35fd7fa640 10 client.9234 handle_command_reply: tid=0 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 1 client.9234 shutdown 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 1 -- 172.21.15.172:0/2884820753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b926e0 msgr2=0x55ce69b8b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b926e0 0x55ce69b8b150 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x55ce69b62d00 tx=0x7f36140177c0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 1 -- 172.21.15.172:0/2884820753 shutdown_connections 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55ce69b80080 0x55ce69babab0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce69b926e0 0x55ce69b8b150 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f35fd7fa640 0 client.9234 ms_handle_reset on v2:172.21.15.177:6833/3201353747 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce69b8cf40 0x55ce69c0b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 1 --2- 172.21.15.172:0/2884820753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ce69b8cb60 0x55ce69c0a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 1 -- 172.21.15.172:0/2884820753 >> 172.21.15.172:0/2884820753 conn(0x55ce69b60040 msgr2=0x55ce69b93070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:49.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 1 -- 172.21.15.172:0/2884820753 shutdown_connections 2024-04-14T08:42:49.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.972+0000 7f36273a0740 1 -- 172.21.15.172:0/2884820753 wait complete. 2024-04-14T08:42:49.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36273a0740 20 client.9234 trim_cache size 0 max 16384 2024-04-14T08:42:49.979 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:42:49.979 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:42:49.979 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:42:49.979 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 9060, 2024-04-14T08:42:49.979 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:42:49.979 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:42:49.979 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:42:49.979 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 9.0827945270000008, 2024-04-14T08:42:49.980 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 88, 2024-04-14T08:42:49.980 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 116, 2024-04-14T08:42:49.980 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 116, 2024-04-14T08:42:49.980 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 14.126529290000001 2024-04-14T08:42:49.980 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:42:49.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36245bb640 1 -- 172.21.15.172:0/1768893537 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f360006c8f0 msgr2=0x7f360006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36245bb640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f360006c8f0 0x7f360006edb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f360c036000 tx=0x7f360c03c040 comp rx=0 tx=0).stop 2024-04-14T08:42:49.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36245bb640 1 -- 172.21.15.172:0/1768893537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361c0b3a30 msgr2=0x7f361c14e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:49.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36245bb640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361c0b3a30 0x7f361c14e660 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f361800e8a0 tx=0x7f361800ed60 comp rx=0 tx=0).stop 2024-04-14T08:42:49.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36245bb640 1 -- 172.21.15.172:0/1768893537 shutdown_connections 2024-04-14T08:42:49.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36245bb640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f360006c8f0 0x7f360006edb0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36245bb640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f361c162540 0x7f361c15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36245bb640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f361c15e430 0x7f361c0b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36245bb640 1 --2- 172.21.15.172:0/1768893537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f361c0b3a30 0x7f361c14e660 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:49.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36245bb640 1 -- 172.21.15.172:0/1768893537 >> 172.21.15.172:0/1768893537 conn(0x7f361c09d560 msgr2=0x7f361c0b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:49.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36245bb640 1 -- 172.21.15.172:0/1768893537 shutdown_connections 2024-04-14T08:42:49.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:49.973+0000 7f36245bb640 1 -- 172.21.15.172:0/1768893537 wait complete. 2024-04-14T08:42:49.984 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:42:50.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.085+0000 7f1530688640 1 Processor -- start 2024-04-14T08:42:50.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.085+0000 7f1530688640 1 -- start start 2024-04-14T08:42:50.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.086+0000 7f1530688640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15280d2830 0x7f15280d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:50.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.086+0000 7f1530688640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15280d3150 con 0x7f152815e890 2024-04-14T08:42:50.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.086+0000 7f1530688640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15280cf320 con 0x7f15280cedd0 2024-04-14T08:42:50.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.086+0000 7f1530688640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f15280cff30 con 0x7f15280d2830 2024-04-14T08:42:50.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.086+0000 7f152e424640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15280d2830 0x7f15280d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:50.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.086+0000 7f152dc23640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f152815e890 0x7f15280d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50800/0 (socket says 172.21.15.172:50800) 2024-04-14T08:42:50.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.086+0000 7f152e424640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f15280d2830 0x7f15280d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45330/0 (socket says 172.21.15.172:45330) 2024-04-14T08:42:50.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.086+0000 7f152dc23640 1 -- 172.21.15.172:0/2098193660 learned_addr learned my addr 172.21.15.172:0/2098193660 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:50.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.087+0000 7f152e424640 1 -- 172.21.15.172:0/2098193660 >> v1:172.21.15.177:6789/0 conn(0x7f15280cedd0 legacy=0x7f15280b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:50.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.087+0000 7f152e424640 1 -- 172.21.15.172:0/2098193660 >> v1:172.21.15.172:6789/0 conn(0x7f152815e890 legacy=0x7f15280d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:50.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.087+0000 7f152d422640 1 -- 172.21.15.172:0/2098193660 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 67020177 0 0) 0x7f15280cf320 con 0x7f15280cedd0 2024-04-14T08:42:50.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.087+0000 7f152e424640 1 -- 172.21.15.172:0/2098193660 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15280cf600 con 0x7f15280d2830 2024-04-14T08:42:50.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.087+0000 7f152e424640 1 --2- 172.21.15.172:0/2098193660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15280d2830 0x7f15280d2c10 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f15200097a0 tx=0x7f152002b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629fea3f5719a09e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:50.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.087+0000 7f152d422640 1 -- 172.21.15.172:0/2098193660 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152002f810 con 0x7f15280d2830 2024-04-14T08:42:50.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.087+0000 7f152d422640 1 -- 172.21.15.172:0/2098193660 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f152002f9b0 con 0x7f15280d2830 2024-04-14T08:42:50.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.088+0000 7f152d422640 1 -- 172.21.15.172:0/2098193660 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152002fd80 con 0x7f15280d2830 2024-04-14T08:42:50.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.088+0000 7f1530688640 1 -- 172.21.15.172:0/2098193660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15280d2830 msgr2=0x7f15280d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:50.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.088+0000 7f1530688640 1 --2- 172.21.15.172:0/2098193660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15280d2830 0x7f15280d2c10 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f15200097a0 tx=0x7f152002b9f0 comp rx=0 tx=0).stop 2024-04-14T08:42:50.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.088+0000 7f1530688640 1 -- 172.21.15.172:0/2098193660 shutdown_connections 2024-04-14T08:42:50.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.088+0000 7f1530688640 1 --2- 172.21.15.172:0/2098193660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15280d2830 0x7f15280d2c10 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.088+0000 7f1530688640 1 -- 172.21.15.172:0/2098193660 >> 172.21.15.172:0/2098193660 conn(0x7f152809d590 msgr2=0x7f15280d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.088+0000 7f1530688640 1 -- 172.21.15.172:0/2098193660 shutdown_connections 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.088+0000 7f1530688640 1 -- 172.21.15.172:0/2098193660 wait complete. 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.089+0000 7f1530688640 1 Processor -- start 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.089+0000 7f1530688640 1 -- start start 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f1530688640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15280cedd0 0x7f15280cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f1530688640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15280d2830 0x7f15280cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f1530688640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f152815e890 0x7f15280b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f1530688640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15280cfbc0 con 0x7f15280cedd0 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f1530688640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f15280d2350 con 0x7f15280d2830 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f1530688640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f15280b3e20 con 0x7f152815e890 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152e424640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15280cedd0 0x7f15280cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152e424640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15280cedd0 0x7f15280cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45334/0 (socket says 172.21.15.172:45334) 2024-04-14T08:42:50.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152e424640 1 -- 172.21.15.172:0/2534497702 learned_addr learned my addr 172.21.15.172:0/2534497702 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152ec25640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f152815e890 0x7f15280b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152dc23640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15280d2830 0x7f15280cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152e424640 1 -- 172.21.15.172:0/2534497702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f152815e890 msgr2=0x7f15280b86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152e424640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f152815e890 0x7f15280b86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152e424640 1 -- 172.21.15.172:0/2534497702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15280d2830 msgr2=0x7f15280cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152e424640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15280d2830 0x7f15280cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152e424640 1 -- 172.21.15.172:0/2534497702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15280cb990 con 0x7f15280cedd0 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152dc23640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15280d2830 0x7f15280cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152ec25640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f152815e890 0x7f15280b86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.090+0000 7f152e424640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15280cedd0 0x7f15280cb320 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f152002c000 tx=0x7f1520004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.091+0000 7f151effd640 1 -- 172.21.15.172:0/2534497702 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520046070 con 0x7f15280cedd0 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.091+0000 7f151effd640 1 -- 172.21.15.172:0/2534497702 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1520037c60 con 0x7f15280cedd0 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.091+0000 7f1530688640 1 -- 172.21.15.172:0/2534497702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15280cbc70 con 0x7f15280cedd0 2024-04-14T08:42:50.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.091+0000 7f151effd640 1 -- 172.21.15.172:0/2534497702 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152002fe60 con 0x7f15280cedd0 2024-04-14T08:42:50.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.091+0000 7f1530688640 1 -- 172.21.15.172:0/2534497702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15280cc230 con 0x7f15280cedd0 2024-04-14T08:42:50.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.092+0000 7f1530688640 1 -- 172.21.15.172:0/2534497702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1528068520 con 0x7f15280cedd0 2024-04-14T08:42:50.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.092+0000 7f151effd640 1 -- 172.21.15.172:0/2534497702 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1520049070 con 0x7f15280cedd0 2024-04-14T08:42:50.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.092+0000 7f151effd640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f150c06c8f0 0x7f150c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:50.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.093+0000 7f151effd640 1 -- 172.21.15.172:0/2534497702 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f15200ad890 con 0x7f15280cedd0 2024-04-14T08:42:50.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.093+0000 7f152dc23640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f150c06c8f0 0x7f150c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:50.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.093+0000 7f152dc23640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f150c06c8f0 0x7f150c06edb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f15280b69b0 tx=0x7f1518009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:50.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.097+0000 7f151effd640 1 -- 172.21.15.172:0/2534497702 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1520033030 con 0x7f15280cedd0 2024-04-14T08:42:50.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.311+0000 7f1530688640 1 -- 172.21.15.172:0/2534497702 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f15280d2610 con 0x7f15280cedd0 2024-04-14T08:42:50.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.312+0000 7f151effd640 1 -- 172.21.15.172:0/2534497702 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) ==== 53+0+73 (secure 0 0 0) 0x7f152007bbc0 con 0x7f15280cedd0 2024-04-14T08:42:50.312 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:42:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.314+0000 7f1530688640 1 -- 172.21.15.172:0/2534497702 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f150c06c8f0 msgr2=0x7f150c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.314+0000 7f1530688640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f150c06c8f0 0x7f150c06edb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f15280b69b0 tx=0x7f1518009040 comp rx=0 tx=0).stop 2024-04-14T08:42:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.314+0000 7f1530688640 1 -- 172.21.15.172:0/2534497702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15280cedd0 msgr2=0x7f15280cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.314+0000 7f1530688640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15280cedd0 0x7f15280cb320 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f152002c000 tx=0x7f1520004730 comp rx=0 tx=0).stop 2024-04-14T08:42:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.314+0000 7f1530688640 1 -- 172.21.15.172:0/2534497702 shutdown_connections 2024-04-14T08:42:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.314+0000 7f1530688640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f150c06c8f0 0x7f150c06edb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.314+0000 7f1530688640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f152815e890 0x7f15280b86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.314+0000 7f1530688640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15280d2830 0x7f15280cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:50.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.314+0000 7f1530688640 1 --2- 172.21.15.172:0/2534497702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15280cedd0 0x7f15280cb320 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:50.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.314+0000 7f1530688640 1 -- 172.21.15.172:0/2534497702 >> 172.21.15.172:0/2534497702 conn(0x7f152809d590 msgr2=0x7f15280b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:50.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.314+0000 7f1530688640 1 -- 172.21.15.172:0/2534497702 shutdown_connections 2024-04-14T08:42:50.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50.314+0000 7f1530688640 1 -- 172.21.15.172:0/2534497702 wait complete. 2024-04-14T08:42:50.325 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:42:50.325 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:42:50.325 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:42:50.325 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:42:50.325 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:42:50.325 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:42:50.325 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:42:50.325 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:42:50.326 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:42:50.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:50.326 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:42:50.341 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:42:50.341 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:42:50.341 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:42:50.341 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:50.342 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:42:50.342 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:50.342 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:42:50.342 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:42:50.342 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:42:50.342 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:42:50.342 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:42:50.342 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:42:50.342 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:42:50.342 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:42:50.343 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:42:50.343 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:42:50.343 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:42:50.343 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42099sec preferred_lft 42099sec 2024-04-14T08:42:50.343 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:42:50.343 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:50.343 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:42:50.343 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:42:50.343 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:42:50.343 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:50.343 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:42:50.344 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:50.344 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:42:50.344 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:42:50.344 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:42:50.344 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:50.344 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:42:50.344 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:42:50.344 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:42:50.344 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:50.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:50.345 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:42:50.408 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:42:50.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:50.409 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:42:50.464 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:42:50.464 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:42:50.464 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:42:50.465 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:42:50.465 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:42:50.465 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:42:50.465 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:42:50.465 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:42:50.465 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:42:50.465 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:42:50.465 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:42:50.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:42:50.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:50.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:50.602 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:42:50.617 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:42:50.617 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:42:50.617 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:42:50.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:50.617 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:42:50.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:42:50.684 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:42:50.740 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:42:50.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:42:50.740 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:42:50.795 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:42:50.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:50.795 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:42:50.880 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:42:50.880 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:42:50.880 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:42:50.888 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:42:50.888 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:42:50.888 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:42:50.888 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:42:50.888 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:42:50.904 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:42:50.904 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:42:50.904 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:42:50.904 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:42:50.904 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:42:50.905 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:42:50.905 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:42:50.905 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:42:50.906 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:42:50.906 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:42:50.906 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:42:50.907 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:42:50.907 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:42:50.907 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:42:50.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:50.907 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:42:50.935 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:42:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:42:51.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:51.041 INFO:teuthology.orchestra.run.smithi172.stdout:client9246 2024-04-14T08:42:51.041 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:42:51.042 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:51.057 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:42:51.057 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:42:51.057 DEBUG:teuthology.orchestra.run.smithi172:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-14T08:42:51.057 DEBUG:teuthology.orchestra.run.smithi172:> parent = os.path.join(mount_point, "parent") 2024-04-14T08:42:51.057 DEBUG:teuthology.orchestra.run.smithi172:> os.mkdir(parent) 2024-04-14T08:42:51.057 DEBUG:teuthology.orchestra.run.smithi172:> flushed = os.path.join(parent, "flushed") 2024-04-14T08:42:51.058 DEBUG:teuthology.orchestra.run.smithi172:> os.mkdir(flushed) 2024-04-14T08:42:51.058 DEBUG:teuthology.orchestra.run.smithi172:> for f in ["alpha", "bravo", "charlie"]: 2024-04-14T08:42:51.058 DEBUG:teuthology.orchestra.run.smithi172:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-14T08:42:51.058 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:42:51.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:51.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:51.181 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-14T08:42:51.241 INFO:teuthology.orchestra.run.smithi172.stdout:./ 2024-04-14T08:42:51.241 INFO:teuthology.orchestra.run.smithi172.stdout:./parent 2024-04-14T08:42:51.241 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/flushed 2024-04-14T08:42:51.241 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/flushed/charlie 2024-04-14T08:42:51.241 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/flushed/alpha 2024-04-14T08:42:51.241 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/flushed/bravo 2024-04-14T08:42:51.242 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:42:51.242 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:42:51.242 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:42:51.242 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:42:51.242 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-14T08:42:51.242 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:42:51.346 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:42:51.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:51.355 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:42:51.355 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:42:51.355 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:42:51.355 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:42:51.355 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-14T08:42:51.355 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:42:51.460 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627776 2024-04-14T08:42:51.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:51.469 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:42:51.469 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:42:51.469 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:42:51.470 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:42:51.470 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-14T08:42:51.470 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:42:51.575 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627777 2024-04-14T08:42:51.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:51.584 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:42:51.584 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:42:51.585 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:42:51.585 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:42:51.585 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-14T08:42:51.585 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:42:51.690 INFO:teuthology.orchestra.run.smithi172.stdout:1099511628280 2024-04-14T08:42:51.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:51.700 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:42:51.700 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:42:51.700 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:42:51.700 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:42:51.700 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-14T08:42:51.700 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:42:51.805 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627778 2024-04-14T08:42:51.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:51.814 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:42:51.815 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:42:51.815 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:42:51.815 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:42:51.815 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-14T08:42:51.815 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:42:51.921 INFO:teuthology.orchestra.run.smithi172.stdout:1099511628279 2024-04-14T08:42:51.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:51.930 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:42:51.931 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:51.987 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:42:51.987 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:52.043 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:42:52.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.184+0000 7f7e5011e640 1 Processor -- start 2024-04-14T08:42:52.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.185+0000 7f7e5011e640 1 -- start start 2024-04-14T08:42:52.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e5011e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e480b1df0 0x7f7e480b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e5011e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7e480b4040 0x7f7e480b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e5011e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e480b49f0 con 0x7f7e480b21d0 2024-04-14T08:42:52.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e5011e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f7e4815c770 con 0x7f7e480b4040 2024-04-14T08:42:52.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e5011e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7e480b3430 con 0x7f7e480b1df0 2024-04-14T08:42:52.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e4e6bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7e480b4040 0x7f7e480b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e4d6b9640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7e480b21d0 0x7f7e4815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50808/0 (socket says 172.21.15.172:50808) 2024-04-14T08:42:52.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e4d6b9640 1 -- 172.21.15.172:0/2887824625 learned_addr learned my addr 172.21.15.172:0/2887824625 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:52.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e4deba640 1 --2- 172.21.15.172:0/2887824625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e480b1df0 0x7f7e480b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e4ceb8640 1 -- 172.21.15.172:0/2887824625 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 491832836 0 0) 0x7f7e480b49f0 con 0x7f7e480b21d0 2024-04-14T08:42:52.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e4ceb8640 1 -- 172.21.15.172:0/2887824625 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e34003520 con 0x7f7e480b21d0 2024-04-14T08:42:52.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e4e6bb640 1 -- 172.21.15.172:0/2887824625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e480b1df0 msgr2=0x7f7e480b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:52.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e4e6bb640 1 --2- 172.21.15.172:0/2887824625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e480b1df0 0x7f7e480b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e4e6bb640 1 -- 172.21.15.172:0/2887824625 >> v1:172.21.15.172:6789/0 conn(0x7f7e480b21d0 legacy=0x7f7e4815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.186+0000 7f7e4e6bb640 1 -- 172.21.15.172:0/2887824625 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e480b3700 con 0x7f7e480b4040 2024-04-14T08:42:52.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.187+0000 7f7e4e6bb640 1 --2- 172.21.15.172:0/2887824625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e480b4040 0x7f7e480b44b0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009d90 tx=0x7f7e4002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f0a6ac88db1b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:52.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.187+0000 7f7e4ceb8640 1 -- 172.21.15.172:0/2887824625 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40039070 con 0x7f7e480b4040 2024-04-14T08:42:52.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.187+0000 7f7e4ceb8640 1 -- 172.21.15.172:0/2887824625 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4002cc60 con 0x7f7e480b4040 2024-04-14T08:42:52.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.187+0000 7f7e4ceb8640 1 -- 172.21.15.172:0/2887824625 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40034660 con 0x7f7e480b4040 2024-04-14T08:42:52.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.187+0000 7f7e5011e640 1 -- 172.21.15.172:0/2887824625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e480b4040 msgr2=0x7f7e480b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.187+0000 7f7e5011e640 1 --2- 172.21.15.172:0/2887824625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e480b4040 0x7f7e480b44b0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009d90 tx=0x7f7e4002beb0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.187+0000 7f7e5011e640 1 -- 172.21.15.172:0/2887824625 shutdown_connections 2024-04-14T08:42:52.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.187+0000 7f7e5011e640 1 --2- 172.21.15.172:0/2887824625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e480b4040 0x7f7e480b44b0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.187+0000 7f7e5011e640 1 --2- 172.21.15.172:0/2887824625 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e480b1df0 0x7f7e480b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.187+0000 7f7e5011e640 1 -- 172.21.15.172:0/2887824625 >> 172.21.15.172:0/2887824625 conn(0x7f7e4806ec30 msgr2=0x7f7e4806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:52.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.188+0000 7f7e5011e640 1 -- 172.21.15.172:0/2887824625 shutdown_connections 2024-04-14T08:42:52.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.188+0000 7f7e5011e640 1 -- 172.21.15.172:0/2887824625 wait complete. 2024-04-14T08:42:52.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.189+0000 7f7e5011e640 1 Processor -- start 2024-04-14T08:42:52.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.189+0000 7f7e5011e640 1 -- start start 2024-04-14T08:42:52.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.189+0000 7f7e5011e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e480b1df0 0x7f7e481f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.189+0000 7f7e5011e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e480b21d0 0x7f7e481569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.189+0000 7f7e5011e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e480b4040 0x7f7e48137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.189+0000 7f7e5011e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4815bb40 con 0x7f7e480b21d0 2024-04-14T08:42:52.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.189+0000 7f7e5011e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7e480b3cc0 con 0x7f7e480b1df0 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.189+0000 7f7e5011e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7e4815b7f0 con 0x7f7e480b4040 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.189+0000 7f7e4d6b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e480b21d0 0x7f7e481569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e4d6b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e480b21d0 0x7f7e481569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45350/0 (socket says 172.21.15.172:45350) 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e4d6b9640 1 -- 172.21.15.172:0/3612591649 learned_addr learned my addr 172.21.15.172:0/3612591649 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e4e6bb640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e480b4040 0x7f7e48137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e4deba640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e480b1df0 0x7f7e481f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e4d6b9640 1 -- 172.21.15.172:0/3612591649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e480b4040 msgr2=0x7f7e48137130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e4d6b9640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e480b4040 0x7f7e48137130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e4d6b9640 1 -- 172.21.15.172:0/3612591649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e480b1df0 msgr2=0x7f7e481f1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e4d6b9640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e480b1df0 0x7f7e481f1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e4d6b9640 1 -- 172.21.15.172:0/3612591649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e481f1980 con 0x7f7e480b21d0 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e4e6bb640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e480b4040 0x7f7e48137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e4d6b9640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e480b21d0 0x7f7e481569d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400c460 tx=0x7f7e4400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:52.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e3e7fc640 1 -- 172.21.15.172:0/3612591649 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44017070 con 0x7f7e480b21d0 2024-04-14T08:42:52.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e3e7fc640 1 -- 172.21.15.172:0/3612591649 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e44005ce0 con 0x7f7e480b21d0 2024-04-14T08:42:52.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e3e7fc640 1 -- 172.21.15.172:0/3612591649 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44007290 con 0x7f7e480b21d0 2024-04-14T08:42:52.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e5011e640 1 -- 172.21.15.172:0/3612591649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e48157cf0 con 0x7f7e480b21d0 2024-04-14T08:42:52.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.190+0000 7f7e5011e640 1 -- 172.21.15.172:0/3612591649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e48157730 con 0x7f7e480b21d0 2024-04-14T08:42:52.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.191+0000 7f7e5011e640 1 -- 172.21.15.172:0/3612591649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e48068520 con 0x7f7e480b21d0 2024-04-14T08:42:52.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.192+0000 7f7e3e7fc640 1 -- 172.21.15.172:0/3612591649 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e4401a070 con 0x7f7e480b21d0 2024-04-14T08:42:52.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.192+0000 7f7e3e7fc640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7e2c06c920 0x7f7e2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.192+0000 7f7e4deba640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7e2c06c920 0x7f7e2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.192+0000 7f7e3e7fc640 1 -- 172.21.15.172:0/3612591649 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f7e4408af10 con 0x7f7e480b21d0 2024-04-14T08:42:52.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.193+0000 7f7e4deba640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7e2c06c920 0x7f7e2c06ede0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009b40 tx=0x7f7e38009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:52.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.197+0000 7f7e3e7fc640 1 -- 172.21.15.172:0/3612591649 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7e44058bf0 con 0x7f7e480b21d0 2024-04-14T08:42:52.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.373+0000 7f7e5011e640 1 -- 172.21.15.172:0/3612591649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e48071f70 con 0x7f7e480b21d0 2024-04-14T08:42:52.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.374+0000 7f7e3e7fc640 1 -- 172.21.15.172:0/3612591649 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+17012 (secure 0 0 0) 0x7f7e4405cef0 con 0x7f7e480b21d0 2024-04-14T08:42:52.375 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:52.375 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":116,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:42:41.900871+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:42:38.455900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-14T08:42:39.198157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:42:52.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.376+0000 7f7e5011e640 1 -- 172.21.15.172:0/3612591649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7e2c06c920 msgr2=0x7f7e2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.376+0000 7f7e5011e640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7e2c06c920 0x7f7e2c06ede0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009b40 tx=0x7f7e38009190 comp rx=0 tx=0).stop 2024-04-14T08:42:52.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.376+0000 7f7e5011e640 1 -- 172.21.15.172:0/3612591649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e480b21d0 msgr2=0x7f7e481569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.376+0000 7f7e5011e640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e480b21d0 0x7f7e481569d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400c460 tx=0x7f7e4400c920 comp rx=0 tx=0).stop 2024-04-14T08:42:52.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.376+0000 7f7e5011e640 1 -- 172.21.15.172:0/3612591649 shutdown_connections 2024-04-14T08:42:52.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.376+0000 7f7e5011e640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7e2c06c920 0x7f7e2c06ede0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.376+0000 7f7e5011e640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e480b4040 0x7f7e48137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.376+0000 7f7e5011e640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e480b21d0 0x7f7e481569d0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.376+0000 7f7e5011e640 1 --2- 172.21.15.172:0/3612591649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e480b1df0 0x7f7e481f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.376+0000 7f7e5011e640 1 -- 172.21.15.172:0/3612591649 >> 172.21.15.172:0/3612591649 conn(0x7f7e4806ec30 msgr2=0x7f7e480b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:52.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.376+0000 7f7e5011e640 1 -- 172.21.15.172:0/3612591649 shutdown_connections 2024-04-14T08:42:52.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.376+0000 7f7e5011e640 1 -- 172.21.15.172:0/3612591649 wait complete. 2024-04-14T08:42:52.388 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:42:38.455900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:42:39.198157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:42:52.388 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:42:52.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.492+0000 7f55fbad5640 1 Processor -- start 2024-04-14T08:42:52.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.492+0000 7f55fbad5640 1 -- start start 2024-04-14T08:42:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55fbad5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f40b23e0 0x7f55f4140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55fbad5640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f415b620 con 0x7f55f413df30 2024-04-14T08:42:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55fbad5640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f41414a0 con 0x7f55f415c380 2024-04-14T08:42:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55fbad5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f55f40cf9b0 con 0x7f55f40b23e0 2024-04-14T08:42:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55f9871640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f55f413df30 0x7f55f40b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50820/0 (socket says 172.21.15.172:50820) 2024-04-14T08:42:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55f9871640 1 -- 172.21.15.172:0/3655781166 learned_addr learned my addr 172.21.15.172:0/3655781166 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55f9070640 1 --2- 172.21.15.172:0/3655781166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f40b23e0 0x7f55f4140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55f886f640 1 -- 172.21.15.172:0/3655781166 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1498018247 0 0) 0x7f55ec001490 con 0x7f55f413df30 2024-04-14T08:42:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55f886f640 1 -- 172.21.15.172:0/3655781166 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55e0003520 con 0x7f55f413df30 2024-04-14T08:42:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55f886f640 1 -- 172.21.15.172:0/3655781166 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2202840081 0 0) 0x7f55f41414a0 con 0x7f55f415c380 2024-04-14T08:42:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55f9070640 1 -- 172.21.15.172:0/3655781166 >> v1:172.21.15.177:6789/0 conn(0x7f55f415c380 legacy=0x7f55f4140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55f9070640 1 -- 172.21.15.172:0/3655781166 >> v1:172.21.15.172:6789/0 conn(0x7f55f413df30 legacy=0x7f55f40b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.493+0000 7f55f9070640 1 -- 172.21.15.172:0/3655781166 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f415b8a0 con 0x7f55f40b23e0 2024-04-14T08:42:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.494+0000 7f55f9070640 1 --2- 172.21.15.172:0/3655781166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f40b23e0 0x7f55f4140770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55e40097d0 tx=0x7f55e402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4321de7ebca65786 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.494+0000 7f55f886f640 1 -- 172.21.15.172:0/3655781166 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4035020 con 0x7f55f40b23e0 2024-04-14T08:42:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.494+0000 7f55f886f640 1 -- 172.21.15.172:0/3655781166 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e4031890 con 0x7f55f40b23e0 2024-04-14T08:42:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.494+0000 7f55f886f640 1 -- 172.21.15.172:0/3655781166 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4031c40 con 0x7f55f40b23e0 2024-04-14T08:42:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.495+0000 7f55fbad5640 1 -- 172.21.15.172:0/3655781166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f40b23e0 msgr2=0x7f55f4140770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.495+0000 7f55fbad5640 1 --2- 172.21.15.172:0/3655781166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f40b23e0 0x7f55f4140770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55e40097d0 tx=0x7f55e402b930 comp rx=0 tx=0).stop 2024-04-14T08:42:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.495+0000 7f55fbad5640 1 -- 172.21.15.172:0/3655781166 shutdown_connections 2024-04-14T08:42:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.495+0000 7f55fbad5640 1 --2- 172.21.15.172:0/3655781166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f40b23e0 0x7f55f4140770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.495+0000 7f55fbad5640 1 -- 172.21.15.172:0/3655781166 >> 172.21.15.172:0/3655781166 conn(0x7f55f406b520 msgr2=0x7f55f406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.495+0000 7f55fbad5640 1 -- 172.21.15.172:0/3655781166 shutdown_connections 2024-04-14T08:42:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.495+0000 7f55fbad5640 1 -- 172.21.15.172:0/3655781166 wait complete. 2024-04-14T08:42:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.496+0000 7f55fbad5640 1 Processor -- start 2024-04-14T08:42:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.496+0000 7f55fbad5640 1 -- start start 2024-04-14T08:42:52.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55fbad5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f40b23e0 0x7f55f415acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55fbad5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55f413df30 0x7f55f414a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55fbad5640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f415c380 0x7f55f4167fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55fbad5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f55f413e310 con 0x7f55f413df30 2024-04-14T08:42:52.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55fbad5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f55f415be60 con 0x7f55f40b23e0 2024-04-14T08:42:52.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55fbad5640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f55f40712d0 con 0x7f55f415c380 2024-04-14T08:42:52.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55f9070640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55f413df30 0x7f55f414a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55f9070640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55f413df30 0x7f55f414a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45364/0 (socket says 172.21.15.172:45364) 2024-04-14T08:42:52.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55f9070640 1 -- 172.21.15.172:0/3694271241 learned_addr learned my addr 172.21.15.172:0/3694271241 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:52.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55f9871640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f40b23e0 0x7f55f415acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55fa072640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f415c380 0x7f55f4167fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55f9070640 1 -- 172.21.15.172:0/3694271241 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f415c380 msgr2=0x7f55f4167fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55f9070640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f415c380 0x7f55f4167fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55f9070640 1 -- 172.21.15.172:0/3694271241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f40b23e0 msgr2=0x7f55f415acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55f9070640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f40b23e0 0x7f55f415acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55f9070640 1 -- 172.21.15.172:0/3694271241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f414a580 con 0x7f55f413df30 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.497+0000 7f55f9070640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55f413df30 0x7f55f414a040 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f55e40098f0 tx=0x7f55e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.498+0000 7f55ea7fc640 1 -- 172.21.15.172:0/3694271241 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4035040 con 0x7f55f413df30 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.498+0000 7f55ea7fc640 1 -- 172.21.15.172:0/3694271241 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e4003820 con 0x7f55f413df30 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.498+0000 7f55fbad5640 1 -- 172.21.15.172:0/3694271241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f4168810 con 0x7f55f413df30 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.498+0000 7f55ea7fc640 1 -- 172.21.15.172:0/3694271241 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4003c10 con 0x7f55f413df30 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.498+0000 7f55fbad5640 1 -- 172.21.15.172:0/3694271241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f4168dd0 con 0x7f55f413df30 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.499+0000 7f55fbad5640 1 -- 172.21.15.172:0/3694271241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f4068590 con 0x7f55f413df30 2024-04-14T08:42:52.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.499+0000 7f55ea7fc640 1 -- 172.21.15.172:0/3694271241 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55e4038020 con 0x7f55f413df30 2024-04-14T08:42:52.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.499+0000 7f55ea7fc640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f55d806c8f0 0x7f55d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.500+0000 7f55ea7fc640 1 -- 172.21.15.172:0/3694271241 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f55e40ae280 con 0x7f55f413df30 2024-04-14T08:42:52.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.500+0000 7f55f9871640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f55d806c8f0 0x7f55d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.500+0000 7f55f9871640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f55d806c8f0 0x7f55d806edb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f55ec005c00 tx=0x7f55ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:52.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.504+0000 7f55ea7fc640 1 -- 172.21.15.172:0/3694271241 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f55e402d040 con 0x7f55f413df30 2024-04-14T08:42:52.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.676+0000 7f55fbad5640 1 -- 172.21.15.172:0/3694271241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f55f4167da0 con 0x7f55f413df30 2024-04-14T08:42:52.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.676+0000 7f55ea7fc640 1 -- 172.21.15.172:0/3694271241 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) ==== 93+0+10 (secure 0 0 0) 0x7f55e407bee0 con 0x7f55f413df30 2024-04-14T08:42:52.676 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:42:52.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.678+0000 7f55fbad5640 1 -- 172.21.15.172:0/3694271241 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f55d806c8f0 msgr2=0x7f55d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.678+0000 7f55fbad5640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f55d806c8f0 0x7f55d806edb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f55ec005c00 tx=0x7f55ec009040 comp rx=0 tx=0).stop 2024-04-14T08:42:52.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.678+0000 7f55fbad5640 1 -- 172.21.15.172:0/3694271241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55f413df30 msgr2=0x7f55f414a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.678+0000 7f55fbad5640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55f413df30 0x7f55f414a040 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f55e40098f0 tx=0x7f55e40046c0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.678+0000 7f55fbad5640 1 -- 172.21.15.172:0/3694271241 shutdown_connections 2024-04-14T08:42:52.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.678+0000 7f55fbad5640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f55d806c8f0 0x7f55d806edb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.678+0000 7f55fbad5640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f415c380 0x7f55f4167fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.678+0000 7f55fbad5640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55f413df30 0x7f55f414a040 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.678+0000 7f55fbad5640 1 --2- 172.21.15.172:0/3694271241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f40b23e0 0x7f55f415acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.678+0000 7f55fbad5640 1 -- 172.21.15.172:0/3694271241 >> 172.21.15.172:0/3694271241 conn(0x7f55f406b520 msgr2=0x7f55f40b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:52.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.678+0000 7f55fbad5640 1 -- 172.21.15.172:0/3694271241 shutdown_connections 2024-04-14T08:42:52.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.678+0000 7f55fbad5640 1 -- 172.21.15.172:0/3694271241 wait complete. 2024-04-14T08:42:52.690 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:42:52.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.794+0000 7f5e2a3c8640 1 Processor -- start 2024-04-14T08:42:52.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.794+0000 7f5e2a3c8640 1 -- start start 2024-04-14T08:42:52.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e2a3c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2406bc30 0x7f5e2406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e2a3c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5e24073090 0x7f5e240b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e2a3c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e240ceee0 con 0x7f5e240ccca0 2024-04-14T08:42:52.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e2a3c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f5e240734a0 con 0x7f5e24073090 2024-04-14T08:42:52.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e2a3c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e24070150 con 0x7f5e2406bc30 2024-04-14T08:42:52.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e28965640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5e24073090 0x7f5e240b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e237fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2406bc30 0x7f5e2406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e23fff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f5e240ccca0 0x7f5e2406b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:37892/0 (socket says 172.21.15.172:37892) 2024-04-14T08:42:52.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e23fff640 1 -- 172.21.15.172:0/1716964703 learned_addr learned my addr 172.21.15.172:0/1716964703 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:52.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e237fe640 1 -- 172.21.15.172:0/1716964703 >> v2:172.21.15.177:3300/0 conn(0x7f5e24073090 msgr2=0x7f5e240b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:52.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e237fe640 1 --2- 172.21.15.172:0/1716964703 >> v2:172.21.15.177:3300/0 conn(0x7f5e24073090 0x7f5e240b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e237fe640 1 -- 172.21.15.172:0/1716964703 >> v1:172.21.15.177:6789/0 conn(0x7f5e240ccca0 legacy=0x7f5e2406b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e237fe640 1 -- 172.21.15.172:0/1716964703 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e240722d0 con 0x7f5e2406bc30 2024-04-14T08:42:52.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e22ffd640 1 -- 172.21.15.172:0/1716964703 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1955833581 0 0) 0x7f5e240ceee0 con 0x7f5e240ccca0 2024-04-14T08:42:52.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.795+0000 7f5e28965640 1 --2- 172.21.15.172:0/1716964703 >> v2:172.21.15.177:3300/0 conn(0x7f5e24073090 0x7f5e240b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:52.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.796+0000 7f5e237fe640 1 --2- 172.21.15.172:0/1716964703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2406bc30 0x7f5e2406fb70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5e180097a0 tx=0x7f5e1802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b0aa694e4541c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:52.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.796+0000 7f5e22ffd640 1 -- 172.21.15.172:0/1716964703 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e180315a0 con 0x7f5e2406bc30 2024-04-14T08:42:52.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.796+0000 7f5e22ffd640 1 -- 172.21.15.172:0/1716964703 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e18037070 con 0x7f5e2406bc30 2024-04-14T08:42:52.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.796+0000 7f5e22ffd640 1 -- 172.21.15.172:0/1716964703 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e18031960 con 0x7f5e2406bc30 2024-04-14T08:42:52.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.797+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/1716964703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2406bc30 msgr2=0x7f5e2406fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.797+0000 7f5e2a3c8640 1 --2- 172.21.15.172:0/1716964703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2406bc30 0x7f5e2406fb70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5e180097a0 tx=0x7f5e1802b840 comp rx=0 tx=0).stop 2024-04-14T08:42:52.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.797+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/1716964703 shutdown_connections 2024-04-14T08:42:52.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.797+0000 7f5e2a3c8640 1 --2- 172.21.15.172:0/1716964703 >> v2:172.21.15.177:3300/0 conn(0x7f5e24073090 0x7f5e240b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.797+0000 7f5e2a3c8640 1 --2- 172.21.15.172:0/1716964703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2406bc30 0x7f5e2406fb70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.797+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/1716964703 >> 172.21.15.172:0/1716964703 conn(0x7f5e240cda00 msgr2=0x7f5e240cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:52.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.797+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/1716964703 shutdown_connections 2024-04-14T08:42:52.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.797+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/1716964703 wait complete. 2024-04-14T08:42:52.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.798+0000 7f5e2a3c8640 1 Processor -- start 2024-04-14T08:42:52.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.798+0000 7f5e2a3c8640 1 -- start start 2024-04-14T08:42:52.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e2a3c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e2406bc30 0x7f5e240ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e2a3c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e24073090 0x7f5e240c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e2a3c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e240ccca0 0x7f5e241f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e2a3c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e240cd080 con 0x7f5e2406bc30 2024-04-14T08:42:52.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e2a3c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e24072890 con 0x7f5e240ccca0 2024-04-14T08:42:52.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e2a3c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2406e150 con 0x7f5e24073090 2024-04-14T08:42:52.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e23fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e2406bc30 0x7f5e240ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e23fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e2406bc30 0x7f5e240ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45366/0 (socket says 172.21.15.172:45366) 2024-04-14T08:42:52.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e23fff640 1 -- 172.21.15.172:0/380409210 learned_addr learned my addr 172.21.15.172:0/380409210 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:52.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e237fe640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e24073090 0x7f5e240c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e28965640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e240ccca0 0x7f5e241f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e23fff640 1 -- 172.21.15.172:0/380409210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e24073090 msgr2=0x7f5e240c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e23fff640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e24073090 0x7f5e240c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e23fff640 1 -- 172.21.15.172:0/380409210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e240ccca0 msgr2=0x7f5e241f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e23fff640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e240ccca0 0x7f5e241f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e23fff640 1 -- 172.21.15.172:0/380409210 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e240c9c50 con 0x7f5e2406bc30 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.799+0000 7f5e23fff640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e2406bc30 0x7f5e240ca080 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400c460 tx=0x7f5e1400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.800+0000 7f5e20ff9640 1 -- 172.21.15.172:0/380409210 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e140074e0 con 0x7f5e2406bc30 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.800+0000 7f5e20ff9640 1 -- 172.21.15.172:0/380409210 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e140047e0 con 0x7f5e2406bc30 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.800+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/380409210 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e240ca660 con 0x7f5e2406bc30 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.800+0000 7f5e20ff9640 1 -- 172.21.15.172:0/380409210 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14004bd0 con 0x7f5e2406bc30 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.800+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/380409210 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e240caf00 con 0x7f5e2406bc30 2024-04-14T08:42:52.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.801+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/380409210 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e24068590 con 0x7f5e2406bc30 2024-04-14T08:42:52.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.801+0000 7f5e20ff9640 1 -- 172.21.15.172:0/380409210 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e14011070 con 0x7f5e2406bc30 2024-04-14T08:42:52.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.802+0000 7f5e20ff9640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e1006c8f0 0x7f5e1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:52.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.802+0000 7f5e20ff9640 1 -- 172.21.15.172:0/380409210 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f5e1408b810 con 0x7f5e2406bc30 2024-04-14T08:42:52.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.802+0000 7f5e237fe640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e1006c8f0 0x7f5e1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:52.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.802+0000 7f5e237fe640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e1006c8f0 0x7f5e1006edb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5e18036000 tx=0x7f5e18008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:52.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.806+0000 7f5e20ff9640 1 -- 172.21.15.172:0/380409210 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5e14059ab0 con 0x7f5e2406bc30 2024-04-14T08:42:52.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.980+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/380409210 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5e2406c120 con 0x7f5e2406bc30 2024-04-14T08:42:52.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.980+0000 7f5e20ff9640 1 -- 172.21.15.172:0/380409210 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) ==== 104+0+11 (secure 0 0 0) 0x7f5e140593b0 con 0x7f5e2406bc30 2024-04-14T08:42:52.981 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:42:52.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.982+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/380409210 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e1006c8f0 msgr2=0x7f5e1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.982+0000 7f5e2a3c8640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e1006c8f0 0x7f5e1006edb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5e18036000 tx=0x7f5e18008030 comp rx=0 tx=0).stop 2024-04-14T08:42:52.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.982+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/380409210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e2406bc30 msgr2=0x7f5e240ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:52.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.982+0000 7f5e2a3c8640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e2406bc30 0x7f5e240ca080 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400c460 tx=0x7f5e1400c860 comp rx=0 tx=0).stop 2024-04-14T08:42:52.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.982+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/380409210 shutdown_connections 2024-04-14T08:42:52.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.982+0000 7f5e2a3c8640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f5e1006c8f0 0x7f5e1006edb0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.982+0000 7f5e2a3c8640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e240ccca0 0x7f5e241f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.982+0000 7f5e2a3c8640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e24073090 0x7f5e240c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.982+0000 7f5e2a3c8640 1 --2- 172.21.15.172:0/380409210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e2406bc30 0x7f5e240ca080 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:52.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.982+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/380409210 >> 172.21.15.172:0/380409210 conn(0x7f5e240cda00 msgr2=0x7f5e240b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:52.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.982+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/380409210 shutdown_connections 2024-04-14T08:42:52.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:52.982+0000 7f5e2a3c8640 1 -- 172.21.15.172:0/380409210 wait complete. 2024-04-14T08:42:52.993 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:42:53.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.097+0000 7f7a4ce98640 1 Processor -- start 2024-04-14T08:42:53.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.097+0000 7f7a4ce98640 1 -- start start 2024-04-14T08:42:53.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.098+0000 7f7a4ce98640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a480ce550 0x7f7a4806d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.098+0000 7f7a4ce98640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a480cc780 con 0x7f7a4806d580 2024-04-14T08:42:53.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.098+0000 7f7a4ce98640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a480cca60 con 0x7f7a4815e3c0 2024-04-14T08:42:53.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.098+0000 7f7a4ce98640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4806ce40 con 0x7f7a480ce550 2024-04-14T08:42:53.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.098+0000 7f7a46d9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7a4806d580 0x7f7a480d0240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50830/0 (socket says 172.21.15.172:50830) 2024-04-14T08:42:53.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.098+0000 7f7a46d9d640 1 -- 172.21.15.172:0/2943461967 learned_addr learned my addr 172.21.15.172:0/2943461967 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:53.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.098+0000 7f7a4659c640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f7a4815e3c0 0x7f7a4806c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:37902/0 (socket says 172.21.15.172:37902) 2024-04-14T08:42:53.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.098+0000 7f7a45d9b640 1 --2- 172.21.15.172:0/2943461967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a480ce550 0x7f7a4806d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.098+0000 7f7a4559a640 1 -- 172.21.15.172:0/2943461967 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1887619183 0 0) 0x7f7a480cc780 con 0x7f7a4806d580 2024-04-14T08:42:53.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.098+0000 7f7a4559a640 1 -- 172.21.15.172:0/2943461967 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a30003520 con 0x7f7a4806d580 2024-04-14T08:42:53.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.098+0000 7f7a4559a640 1 -- 172.21.15.172:0/2943461967 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 32508218 0 0) 0x7f7a480cca60 con 0x7f7a4815e3c0 2024-04-14T08:42:53.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.098+0000 7f7a4559a640 1 -- 172.21.15.172:0/2943461967 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a480cc780 con 0x7f7a4815e3c0 2024-04-14T08:42:53.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.099+0000 7f7a45d9b640 1 -- 172.21.15.172:0/2943461967 >> v1:172.21.15.177:6789/0 conn(0x7f7a4815e3c0 legacy=0x7f7a4806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.099+0000 7f7a45d9b640 1 -- 172.21.15.172:0/2943461967 >> v1:172.21.15.172:6789/0 conn(0x7f7a4806d580 legacy=0x7f7a480d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.099+0000 7f7a45d9b640 1 -- 172.21.15.172:0/2943461967 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a480ccd40 con 0x7f7a480ce550 2024-04-14T08:42:53.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.099+0000 7f7a45d9b640 1 --2- 172.21.15.172:0/2943461967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a480ce550 0x7f7a4806d040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7a380097d0 tx=0x7f7a3802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdb174dda41c31f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.099+0000 7f7a4559a640 1 -- 172.21.15.172:0/2943461967 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38035020 con 0x7f7a480ce550 2024-04-14T08:42:53.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.100+0000 7f7a4559a640 1 -- 172.21.15.172:0/2943461967 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a38031890 con 0x7f7a480ce550 2024-04-14T08:42:53.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.100+0000 7f7a4559a640 1 -- 172.21.15.172:0/2943461967 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38031c60 con 0x7f7a480ce550 2024-04-14T08:42:53.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.100+0000 7f7a4ce98640 1 -- 172.21.15.172:0/2943461967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a480ce550 msgr2=0x7f7a4806d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.100+0000 7f7a4ce98640 1 --2- 172.21.15.172:0/2943461967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a480ce550 0x7f7a4806d040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7a380097d0 tx=0x7f7a3802b930 comp rx=0 tx=0).stop 2024-04-14T08:42:53.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.100+0000 7f7a4ce98640 1 -- 172.21.15.172:0/2943461967 shutdown_connections 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.100+0000 7f7a4ce98640 1 --2- 172.21.15.172:0/2943461967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a480ce550 0x7f7a4806d040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.100+0000 7f7a4ce98640 1 -- 172.21.15.172:0/2943461967 >> 172.21.15.172:0/2943461967 conn(0x7f7a480b3af0 msgr2=0x7f7a480b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.100+0000 7f7a4ce98640 1 -- 172.21.15.172:0/2943461967 shutdown_connections 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.100+0000 7f7a4ce98640 1 -- 172.21.15.172:0/2943461967 wait complete. 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.101+0000 7f7a4ce98640 1 Processor -- start 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.102+0000 7f7a4ce98640 1 -- start start 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.102+0000 7f7a4ce98640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4806d580 0x7f7a480b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.102+0000 7f7a4ce98640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a480ce550 0x7f7a480cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.102+0000 7f7a4ce98640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4815e3c0 0x7f7a48160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.102+0000 7f7a4ce98640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7a480d0950 con 0x7f7a480ce550 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.102+0000 7f7a4ce98640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4815e7a0 con 0x7f7a4806d580 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.102+0000 7f7a4ce98640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7a480d0ad0 con 0x7f7a4815e3c0 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.102+0000 7f7a45d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a480ce550 0x7f7a480cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.102+0000 7f7a45d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a480ce550 0x7f7a480cb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45368/0 (socket says 172.21.15.172:45368) 2024-04-14T08:42:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.102+0000 7f7a45d9b640 1 -- 172.21.15.172:0/696303764 learned_addr learned my addr 172.21.15.172:0/696303764 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.102+0000 7f7a4659c640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4806d580 0x7f7a480b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.102+0000 7f7a46d9d640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4815e3c0 0x7f7a48160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.103+0000 7f7a45d9b640 1 -- 172.21.15.172:0/696303764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4815e3c0 msgr2=0x7f7a48160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.103+0000 7f7a45d9b640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4815e3c0 0x7f7a48160ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.103+0000 7f7a45d9b640 1 -- 172.21.15.172:0/696303764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4806d580 msgr2=0x7f7a480b7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.103+0000 7f7a45d9b640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4806d580 0x7f7a480b7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.103+0000 7f7a45d9b640 1 -- 172.21.15.172:0/696303764 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4815f7c0 con 0x7f7a480ce550 2024-04-14T08:42:53.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.103+0000 7f7a45d9b640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a480ce550 0x7f7a480cb060 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f7a380098f0 tx=0x7f7a380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.103+0000 7f7a36ffd640 1 -- 172.21.15.172:0/696303764 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38035040 con 0x7f7a480ce550 2024-04-14T08:42:53.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.103+0000 7f7a36ffd640 1 -- 172.21.15.172:0/696303764 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a38030d20 con 0x7f7a480ce550 2024-04-14T08:42:53.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.103+0000 7f7a4ce98640 1 -- 172.21.15.172:0/696303764 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4815faa0 con 0x7f7a480ce550 2024-04-14T08:42:53.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.103+0000 7f7a36ffd640 1 -- 172.21.15.172:0/696303764 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38006e10 con 0x7f7a480ce550 2024-04-14T08:42:53.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.103+0000 7f7a4ce98640 1 -- 172.21.15.172:0/696303764 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a48160060 con 0x7f7a480ce550 2024-04-14T08:42:53.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.104+0000 7f7a4ce98640 1 -- 172.21.15.172:0/696303764 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a48068590 con 0x7f7a480ce550 2024-04-14T08:42:53.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.104+0000 7f7a36ffd640 1 -- 172.21.15.172:0/696303764 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a38038020 con 0x7f7a480ce550 2024-04-14T08:42:53.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.105+0000 7f7a36ffd640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7a2806c920 0x7f7a2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.105+0000 7f7a4659c640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7a2806c920 0x7f7a2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.105+0000 7f7a36ffd640 1 -- 172.21.15.172:0/696303764 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f7a380ae0c0 con 0x7f7a480ce550 2024-04-14T08:42:53.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.105+0000 7f7a4659c640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7a2806c920 0x7f7a2806ede0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c000f60 tx=0x7f7a3c007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.109+0000 7f7a36ffd640 1 -- 172.21.15.172:0/696303764 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7a3802d040 con 0x7f7a480ce550 2024-04-14T08:42:53.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.286+0000 7f7a4ce98640 1 -- 172.21.15.172:0/696303764 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7a480b0ad0 con 0x7f7a480ce550 2024-04-14T08:42:53.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.286+0000 7f7a36ffd640 1 -- 172.21.15.172:0/696303764 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) ==== 100+0+11 (secure 0 0 0) 0x7f7a3807bda0 con 0x7f7a480ce550 2024-04-14T08:42:53.287 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:42:53.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.288+0000 7f7a4ce98640 1 -- 172.21.15.172:0/696303764 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7a2806c920 msgr2=0x7f7a2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.288+0000 7f7a4ce98640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7a2806c920 0x7f7a2806ede0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c000f60 tx=0x7f7a3c007cf0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.288+0000 7f7a4ce98640 1 -- 172.21.15.172:0/696303764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a480ce550 msgr2=0x7f7a480cb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.288+0000 7f7a4ce98640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a480ce550 0x7f7a480cb060 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f7a380098f0 tx=0x7f7a380046c0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.288+0000 7f7a4ce98640 1 -- 172.21.15.172:0/696303764 shutdown_connections 2024-04-14T08:42:53.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.288+0000 7f7a4ce98640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7a2806c920 0x7f7a2806ede0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.288+0000 7f7a4ce98640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4815e3c0 0x7f7a48160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.288+0000 7f7a4ce98640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a480ce550 0x7f7a480cb060 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.288+0000 7f7a4ce98640 1 --2- 172.21.15.172:0/696303764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4806d580 0x7f7a480b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.288+0000 7f7a4ce98640 1 -- 172.21.15.172:0/696303764 >> 172.21.15.172:0/696303764 conn(0x7f7a480b3af0 msgr2=0x7f7a4806fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:53.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.288+0000 7f7a4ce98640 1 -- 172.21.15.172:0/696303764 shutdown_connections 2024-04-14T08:42:53.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.288+0000 7f7a4ce98640 1 -- 172.21.15.172:0/696303764 wait complete. 2024-04-14T08:42:53.300 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:42:53.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.403+0000 7f60f96b7640 1 Processor -- start 2024-04-14T08:42:53.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.403+0000 7f60f96b7640 1 -- start start 2024-04-14T08:42:53.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f96b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60f415e4b0 0x7f60f40cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f96b7640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f60f40cfd70 0x7f60f40d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f96b7640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f60f40d0fb0 0x7f60f4161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f96b7640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f60f415e890 con 0x7f60f40d0fb0 2024-04-14T08:42:53.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f96b7640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f60f40d1f30 con 0x7f60f40cfd70 2024-04-14T08:42:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f96b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f60f40d2210 con 0x7f60f415e4b0 2024-04-14T08:42:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f37fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f60f40d0fb0 0x7f60f4161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f37fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f60f40d0fb0 0x7f60f4161fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45382/0 (socket says 172.21.15.172:45382) 2024-04-14T08:42:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f37fe640 1 -- 172.21.15.172:0/3523995964 learned_addr learned my addr 172.21.15.172:0/3523995964 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f27fc640 1 --2- 172.21.15.172:0/3523995964 >> v2:172.21.15.177:3300/0 conn(0x7f60f40cfd70 0x7f60f40d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f2ffd640 1 --2- 172.21.15.172:0/3523995964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60f415e4b0 0x7f60f40cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f37fe640 1 -- 172.21.15.172:0/3523995964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60f415e4b0 msgr2=0x7f60f40cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f37fe640 1 --2- 172.21.15.172:0/3523995964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60f415e4b0 0x7f60f40cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f37fe640 1 -- 172.21.15.172:0/3523995964 >> v2:172.21.15.177:3300/0 conn(0x7f60f40cfd70 msgr2=0x7f60f40d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f37fe640 1 --2- 172.21.15.172:0/3523995964 >> v2:172.21.15.177:3300/0 conn(0x7f60f40cfd70 0x7f60f40d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.404+0000 7f60f37fe640 1 -- 172.21.15.172:0/3523995964 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f40d24f0 con 0x7f60f40d0fb0 2024-04-14T08:42:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.405+0000 7f60f37fe640 1 --2- 172.21.15.172:0/3523995964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60f40d0fb0 0x7f60f4161fa0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f60f409dca0 tx=0x7f60ec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f23d76797974021 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.405+0000 7f60f1ffb640 1 -- 172.21.15.172:0/3523995964 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec02c9e0 con 0x7f60f40d0fb0 2024-04-14T08:42:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.405+0000 7f60f1ffb640 1 -- 172.21.15.172:0/3523995964 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ec02cb80 con 0x7f60f40d0fb0 2024-04-14T08:42:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.405+0000 7f60f1ffb640 1 -- 172.21.15.172:0/3523995964 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec0343f0 con 0x7f60f40d0fb0 2024-04-14T08:42:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.406+0000 7f60f96b7640 1 -- 172.21.15.172:0/3523995964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60f40d0fb0 msgr2=0x7f60f4161fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.406+0000 7f60f96b7640 1 --2- 172.21.15.172:0/3523995964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60f40d0fb0 0x7f60f4161fa0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f60f409dca0 tx=0x7f60ec02be40 comp rx=0 tx=0).stop 2024-04-14T08:42:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.406+0000 7f60f96b7640 1 -- 172.21.15.172:0/3523995964 shutdown_connections 2024-04-14T08:42:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.406+0000 7f60f96b7640 1 --2- 172.21.15.172:0/3523995964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60f40d0fb0 0x7f60f4161fa0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.406+0000 7f60f96b7640 1 --2- 172.21.15.172:0/3523995964 >> v2:172.21.15.177:3300/0 conn(0x7f60f40cfd70 0x7f60f40d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.406+0000 7f60f96b7640 1 --2- 172.21.15.172:0/3523995964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60f415e4b0 0x7f60f40cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.406+0000 7f60f96b7640 1 -- 172.21.15.172:0/3523995964 >> 172.21.15.172:0/3523995964 conn(0x7f60f409d570 msgr2=0x7f60f40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.406+0000 7f60f96b7640 1 -- 172.21.15.172:0/3523995964 shutdown_connections 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.406+0000 7f60f96b7640 1 -- 172.21.15.172:0/3523995964 wait complete. 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.407+0000 7f60f96b7640 1 Processor -- start 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.407+0000 7f60f96b7640 1 -- start start 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f96b7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60f40cfd70 0x7f60f40cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f96b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60f40d0fb0 0x7f60f41ea030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f96b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60f415e4b0 0x7f60f41612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f96b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60f40d2fd0 con 0x7f60f40d0fb0 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f96b7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f60f40d2ab0 con 0x7f60f40cfd70 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f96b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f60f40ce320 con 0x7f60f415e4b0 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60f40d0fb0 0x7f60f41ea030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60f40d0fb0 0x7f60f41ea030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45390/0 (socket says 172.21.15.172:45390) 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f27fc640 1 -- 172.21.15.172:0/2285441794 learned_addr learned my addr 172.21.15.172:0/2285441794 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f2ffd640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60f40cfd70 0x7f60f40cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f37fe640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60f415e4b0 0x7f60f41612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f27fc640 1 -- 172.21.15.172:0/2285441794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60f415e4b0 msgr2=0x7f60f41612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f27fc640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60f415e4b0 0x7f60f41612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f27fc640 1 -- 172.21.15.172:0/2285441794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60f40cfd70 msgr2=0x7f60f40cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f27fc640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60f40cfd70 0x7f60f40cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.408+0000 7f60f27fc640 1 -- 172.21.15.172:0/2285441794 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f40cb750 con 0x7f60f40d0fb0 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.409+0000 7f60f27fc640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60f40d0fb0 0x7f60f41ea030 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f60e800e870 tx=0x7f60e800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.409+0000 7f60db7fe640 1 -- 172.21.15.172:0/2285441794 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8017070 con 0x7f60f40d0fb0 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.409+0000 7f60db7fe640 1 -- 172.21.15.172:0/2285441794 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e800ee60 con 0x7f60f40d0fb0 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.409+0000 7f60f96b7640 1 -- 172.21.15.172:0/2285441794 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f40cba30 con 0x7f60f40d0fb0 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.409+0000 7f60db7fe640 1 -- 172.21.15.172:0/2285441794 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e800cb80 con 0x7f60f40d0fb0 2024-04-14T08:42:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.409+0000 7f60f96b7640 1 -- 172.21.15.172:0/2285441794 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f40b78c0 con 0x7f60f40d0fb0 2024-04-14T08:42:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.410+0000 7f60f96b7640 1 -- 172.21.15.172:0/2285441794 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f4068590 con 0x7f60f40d0fb0 2024-04-14T08:42:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.410+0000 7f60db7fe640 1 -- 172.21.15.172:0/2285441794 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60e801a070 con 0x7f60f40d0fb0 2024-04-14T08:42:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.411+0000 7f60db7fe640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f60e006c920 0x7f60e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.411+0000 7f60db7fe640 1 -- 172.21.15.172:0/2285441794 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f60e808bd50 con 0x7f60f40d0fb0 2024-04-14T08:42:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.411+0000 7f60f2ffd640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f60e006c920 0x7f60e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.411+0000 7f60f2ffd640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f60e006c920 0x7f60e006ede0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009b40 tx=0x7f60e4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.415+0000 7f60db7fe640 1 -- 172.21.15.172:0/2285441794 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f60e808f030 con 0x7f60f40d0fb0 2024-04-14T08:42:53.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.601+0000 7f60f96b7640 1 -- 172.21.15.172:0/2285441794 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f60f40d17e0 con 0x7f60f40d0fb0 2024-04-14T08:42:53.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.601+0000 7f60db7fe640 1 -- 172.21.15.172:0/2285441794 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v116) ==== 80+0+0 (secure 0 0 0) 0x7f60e8059a30 con 0x7f60f40d0fb0 2024-04-14T08:42:53.601 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:42:53.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.603+0000 7f60f96b7640 1 -- 172.21.15.172:0/2285441794 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f60e006c920 msgr2=0x7f60e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.603+0000 7f60f96b7640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f60e006c920 0x7f60e006ede0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009b40 tx=0x7f60e4009190 comp rx=0 tx=0).stop 2024-04-14T08:42:53.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.603+0000 7f60f96b7640 1 -- 172.21.15.172:0/2285441794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60f40d0fb0 msgr2=0x7f60f41ea030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.603+0000 7f60f96b7640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60f40d0fb0 0x7f60f41ea030 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f60e800e870 tx=0x7f60e800ed30 comp rx=0 tx=0).stop 2024-04-14T08:42:53.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.603+0000 7f60f96b7640 1 -- 172.21.15.172:0/2285441794 shutdown_connections 2024-04-14T08:42:53.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.603+0000 7f60f96b7640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f60e006c920 0x7f60e006ede0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.603+0000 7f60f96b7640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60f415e4b0 0x7f60f41612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.603+0000 7f60f96b7640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60f40d0fb0 0x7f60f41ea030 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.603+0000 7f60f96b7640 1 --2- 172.21.15.172:0/2285441794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60f40cfd70 0x7f60f40cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.603+0000 7f60f96b7640 1 -- 172.21.15.172:0/2285441794 >> 172.21.15.172:0/2285441794 conn(0x7f60f409d570 msgr2=0x7f60f40b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:53.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.603+0000 7f60f96b7640 1 -- 172.21.15.172:0/2285441794 shutdown_connections 2024-04-14T08:42:53.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.603+0000 7f60f96b7640 1 -- 172.21.15.172:0/2285441794 wait complete. 2024-04-14T08:42:53.614 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:42:53.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:53.615 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:42:53.667 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:42:53.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:53.667 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:42:53.690 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-14T08:42:53.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.832+0000 7f013ae60640 1 Processor -- start 2024-04-14T08:42:53.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.833+0000 7f013ae60640 1 -- start start 2024-04-14T08:42:53.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.833+0000 7f013ae60640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0134162710 0x7f01340d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.833+0000 7f013ae60640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01340cefb0 con 0x7f0134162ba0 2024-04-14T08:42:53.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.833+0000 7f013ae60640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01340cf290 con 0x7f013415e430 2024-04-14T08:42:53.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.833+0000 7f013ae60640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f01340cf570 con 0x7f0134162710 2024-04-14T08:42:53.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.834+0000 7f0133fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0134162710 0x7f01340d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.834+0000 7f01393fd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f0134162ba0 0x7f01340d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50838/0 (socket says 172.21.15.172:50838) 2024-04-14T08:42:53.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.834+0000 7f0133fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0134162710 0x7f01340d1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45394/0 (socket says 172.21.15.172:45394) 2024-04-14T08:42:53.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.834+0000 7f01393fd640 1 -- 172.21.15.172:0/3819733669 learned_addr learned my addr 172.21.15.172:0/3819733669 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:53.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.834+0000 7f0138bfc640 1 --1- 172.21.15.172:0/3819733669 >> v1:172.21.15.177:6789/0 conn(0x7f013415e430 0x7f0134162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:37916/0 (socket says 172.21.15.172:37916) 2024-04-14T08:42:53.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.834+0000 7f0133fff640 1 -- 172.21.15.172:0/3819733669 >> v1:172.21.15.177:6789/0 conn(0x7f013415e430 legacy=0x7f0134162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.834+0000 7f0133fff640 1 -- 172.21.15.172:0/3819733669 >> v1:172.21.15.172:6789/0 conn(0x7f0134162ba0 legacy=0x7f01340d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.834+0000 7f01337fe640 1 -- 172.21.15.172:0/3819733669 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4129212864 0 0) 0x7f01340cf290 con 0x7f013415e430 2024-04-14T08:42:53.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.834+0000 7f0133fff640 1 -- 172.21.15.172:0/3819733669 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01340cf850 con 0x7f0134162710 2024-04-14T08:42:53.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.835+0000 7f0133fff640 1 --2- 172.21.15.172:0/3819733669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0134162710 0x7f01340d1df0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f01200097a0 tx=0x7f012002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975c2c486b504d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.835+0000 7f01337fe640 1 -- 172.21.15.172:0/3819733669 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120037070 con 0x7f0134162710 2024-04-14T08:42:53.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.835+0000 7f01337fe640 1 -- 172.21.15.172:0/3819733669 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0120030710 con 0x7f0134162710 2024-04-14T08:42:53.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.835+0000 7f01337fe640 1 -- 172.21.15.172:0/3819733669 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120030ae0 con 0x7f0134162710 2024-04-14T08:42:53.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.835+0000 7f013ae60640 1 -- 172.21.15.172:0/3819733669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0134162710 msgr2=0x7f01340d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.835+0000 7f013ae60640 1 --2- 172.21.15.172:0/3819733669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0134162710 0x7f01340d1df0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f01200097a0 tx=0x7f012002b790 comp rx=0 tx=0).stop 2024-04-14T08:42:53.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.835+0000 7f013ae60640 1 -- 172.21.15.172:0/3819733669 shutdown_connections 2024-04-14T08:42:53.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.835+0000 7f013ae60640 1 --2- 172.21.15.172:0/3819733669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0134162710 0x7f01340d1df0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.835+0000 7f013ae60640 1 -- 172.21.15.172:0/3819733669 >> 172.21.15.172:0/3819733669 conn(0x7f013409d560 msgr2=0x7f01340d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:53.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.835+0000 7f013ae60640 1 -- 172.21.15.172:0/3819733669 shutdown_connections 2024-04-14T08:42:53.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.835+0000 7f013ae60640 1 -- 172.21.15.172:0/3819733669 wait complete. 2024-04-14T08:42:53.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.836+0000 7f013ae60640 1 Processor -- start 2024-04-14T08:42:53.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.837+0000 7f013ae60640 1 -- start start 2024-04-14T08:42:53.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.837+0000 7f013ae60640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f013415e430 0x7f0134152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.837+0000 7f013ae60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0134162710 0x7f01340ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.837+0000 7f013ae60640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0134162ba0 0x7f01340b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.837+0000 7f013ae60640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01340d4e20 con 0x7f0134162ba0 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.837+0000 7f013ae60640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f013415e810 con 0x7f013415e430 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.837+0000 7f013ae60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f01340d0050 con 0x7f0134162710 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.837+0000 7f01393fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0134162ba0 0x7f01340b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.837+0000 7f01393fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0134162ba0 0x7f01340b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45400/0 (socket says 172.21.15.172:45400) 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.837+0000 7f01393fd640 1 -- 172.21.15.172:0/3477031214 learned_addr learned my addr 172.21.15.172:0/3477031214 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f0138bfc640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f013415e430 0x7f0134152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f0133fff640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0134162710 0x7f01340ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f01393fd640 1 -- 172.21.15.172:0/3477031214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0134162710 msgr2=0x7f01340ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f01393fd640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0134162710 0x7f01340ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f01393fd640 1 -- 172.21.15.172:0/3477031214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f013415e430 msgr2=0x7f0134152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f01393fd640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f013415e430 0x7f0134152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f01393fd640 1 -- 172.21.15.172:0/3477031214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0134152b70 con 0x7f0134162ba0 2024-04-14T08:42:53.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f01393fd640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0134162ba0 0x7f01340b3a10 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f012c00cb20 tx=0x7f012c0020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f01317fa640 1 -- 172.21.15.172:0/3477031214 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c018070 con 0x7f0134162ba0 2024-04-14T08:42:53.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f01317fa640 1 -- 172.21.15.172:0/3477031214 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f012c00a760 con 0x7f0134162ba0 2024-04-14T08:42:53.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f013ae60640 1 -- 172.21.15.172:0/3477031214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01340c96f0 con 0x7f0134162ba0 2024-04-14T08:42:53.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f01317fa640 1 -- 172.21.15.172:0/3477031214 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c00ab30 con 0x7f0134162ba0 2024-04-14T08:42:53.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.838+0000 7f013ae60640 1 -- 172.21.15.172:0/3477031214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01340c9cb0 con 0x7f0134162ba0 2024-04-14T08:42:53.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.840+0000 7f01317fa640 1 -- 172.21.15.172:0/3477031214 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f012c01b070 con 0x7f0134162ba0 2024-04-14T08:42:53.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.840+0000 7f01317fa640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f011806c8f0 0x7f011806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.840+0000 7f01317fa640 1 -- 172.21.15.172:0/3477031214 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f012c08b390 con 0x7f0134162ba0 2024-04-14T08:42:53.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.840+0000 7f0138bfc640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f011806c8f0 0x7f011806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.841+0000 7f0138bfc640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f011806c8f0 0x7f011806edb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0128000f60 tx=0x7f0128007e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.841+0000 7f013dc52740 1 Processor -- start 2024-04-14T08:42:53.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.841+0000 7f013dc52740 1 -- start start 2024-04-14T08:42:53.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f013dc52740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 0x55e6ff5a2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f013dc52740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f013dc52740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5a2e50 0x55e6ff5d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f013dc52740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e6ff50c800 con 0x55e6ff5cf970 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f013dc52740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e6ff4b47f0 con 0x55e6ff5a2e50 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f013dc52740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e6ff5087b0 con 0x55e6ff5cfd50 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f0138bfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 0x55e6ff5a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f0138bfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 0x55e6ff5a2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45414/0 (socket says 172.21.15.172:45414) 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f0138bfc640 1 -- 172.21.15.172:0/3930730664 learned_addr learned my addr 172.21.15.172:0/3930730664 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f0133fff640 1 --2- 172.21.15.172:0/3930730664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f01393fd640 1 --2- 172.21.15.172:0/3930730664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5a2e50 0x55e6ff5d2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f0138bfc640 1 -- 172.21.15.172:0/3930730664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 msgr2=0x55e6ff5d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f0138bfc640 1 --2- 172.21.15.172:0/3930730664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5d2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f0138bfc640 1 -- 172.21.15.172:0/3930730664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5a2e50 msgr2=0x55e6ff5d2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f0138bfc640 1 --2- 172.21.15.172:0/3930730664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5a2e50 0x55e6ff5d2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f0138bfc640 1 -- 172.21.15.172:0/3930730664 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6ff2ea7d0 con 0x55e6ff5cf970 2024-04-14T08:42:53.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f0133fff640 1 --2- 172.21.15.172:0/3930730664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:53.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.842+0000 7f0138bfc640 1 --2- 172.21.15.172:0/3930730664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 0x55e6ff5a2910 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f0128001830 tx=0x7f0128013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61dcae4336f854fe server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.843+0000 7f011e7fc640 1 -- 172.21.15.172:0/3930730664 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128026070 con 0x55e6ff5cf970 2024-04-14T08:42:53.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.843+0000 7f011e7fc640 1 -- 172.21.15.172:0/3930730664 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0128001cf0 con 0x55e6ff5cf970 2024-04-14T08:42:53.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.843+0000 7f011e7fc640 1 -- 172.21.15.172:0/3930730664 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128013dc0 con 0x55e6ff5cf970 2024-04-14T08:42:53.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.843+0000 7f013dc52740 1 -- 172.21.15.172:0/3930730664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 msgr2=0x55e6ff5a2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.843+0000 7f013dc52740 1 --2- 172.21.15.172:0/3930730664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 0x55e6ff5a2910 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f0128001830 tx=0x7f0128013000 comp rx=0 tx=0).stop 2024-04-14T08:42:53.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.843+0000 7f013dc52740 1 -- 172.21.15.172:0/3930730664 shutdown_connections 2024-04-14T08:42:53.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.843+0000 7f013dc52740 1 --2- 172.21.15.172:0/3930730664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5a2e50 0x55e6ff5d2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.843+0000 7f013dc52740 1 --2- 172.21.15.172:0/3930730664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.843+0000 7f013dc52740 1 --2- 172.21.15.172:0/3930730664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 0x55e6ff5a2910 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.843+0000 7f013dc52740 1 -- 172.21.15.172:0/3930730664 >> 172.21.15.172:0/3930730664 conn(0x55e6ff5a4b80 msgr2=0x55e6ff5ed700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:53.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.843+0000 7f013dc52740 1 -- 172.21.15.172:0/3930730664 shutdown_connections 2024-04-14T08:42:53.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.843+0000 7f013dc52740 1 -- 172.21.15.172:0/3930730664 wait complete. 2024-04-14T08:42:53.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.844+0000 7f013dc52740 1 -- start start 2024-04-14T08:42:53.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.844+0000 7f013dc52740 1 Processor -- start 2024-04-14T08:42:53.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f013dc52740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 0x55e6ff5d4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f013dc52740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5fd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f013dc52740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5c2a30 0x55e6ff5e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f013dc52740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e6ff5a7120 con 0x55e6ff5cf970 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f013dc52740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e6ff4c7d00 con 0x55e6ff5c2a30 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f013dc52740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e6ff5a6510 con 0x55e6ff5cfd50 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f013dc52740 10 client.0 _ll_register_callbacks cb 0x55e6ff4c76f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f0138bfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 0x55e6ff5d4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f0138bfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 0x55e6ff5d4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45424/0 (socket says 172.21.15.172:45424) 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f0138bfc640 1 -- 172.21.15.172:0/3793214245 learned_addr learned my addr 172.21.15.172:0/3793214245 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f01393fd640 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5c2a30 0x55e6ff5e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f0133fff640 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5fd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f0138bfc640 1 -- 172.21.15.172:0/3793214245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 msgr2=0x55e6ff5fd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f0138bfc640 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5fd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f0138bfc640 1 -- 172.21.15.172:0/3793214245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5c2a30 msgr2=0x55e6ff5e6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f0138bfc640 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5c2a30 0x55e6ff5e6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f0138bfc640 1 -- 172.21.15.172:0/3793214245 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6ff2203f0 con 0x55e6ff5cf970 2024-04-14T08:42:53.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f0138bfc640 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 0x55e6ff5d4640 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f01280017d0 tx=0x7f0128000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab03bfee5a531bd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f01137fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f01137fe640 1 -- 172.21.15.172:0/3793214245 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128026070 con 0x55e6ff5cf970 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f01137fe640 1 -- 172.21.15.172:0/3793214245 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f0128021a70 con 0x55e6ff5cf970 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.845+0000 7f013dc52740 1 -- 172.21.15.172:0/3793214245 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0128001cf0 con 0x55e6ff5cf970 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f01137fe640 1 -- 172.21.15.172:0/3793214245 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0128039050 con 0x55e6ff5cf970 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f013dc52740 10 client.9291 fetch_fsmap learned FSMap version 88 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f013dc52740 1 -- 172.21.15.172:0/3793214245 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55e6ff1e00f0 con 0x55e6ff5cf970 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f01137fe640 1 -- 172.21.15.172:0/3793214245 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0128001cf0 con 0x55e6ff5cf970 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f01137fe640 1 -- 172.21.15.172:0/3793214245 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f012803c020 con 0x55e6ff5cf970 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f013dc52740 10 client.9291 fetch_fsmap finished waiting for FSMap version 88 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f013dc52740 10 client.9291 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f013dc52740 20 client.9291 populate_metadata read hostname 'smithi172' 2024-04-14T08:42:53.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f013dc52740 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55e6ff6016e0 0x55e6ff601ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f013dc52740 4 client.9291 mds_command: new command op to 9060 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f013dc52740 1 -- 172.21.15.172:0/3793214245 --> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6ff519d40 con 0x55e6ff6016e0 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f0133fff640 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55e6ff6016e0 0x55e6ff601ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f0133fff640 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55e6ff6016e0 0x55e6ff601ac0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6529b87f97b5c1d server_cookie=4d9d84f30a921d74 in_seq=0 out_seq=0 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.846+0000 7f01137fe640 10 client.9291 ms_handle_connect on v2:172.21.15.177:6833/3201353747 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.847+0000 7f01137fe640 1 -- 172.21.15.172:0/3793214245 <== mds.0 v2:172.21.15.177:6833/3201353747 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6ff519d40 con 0x55e6ff6016e0 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.847+0000 7f01137fe640 10 client.9291 handle_command_reply: tid=0 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.847+0000 7f013dc52740 1 client.9291 shutdown 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.847+0000 7f013dc52740 1 -- 172.21.15.172:0/3793214245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 msgr2=0x55e6ff5d4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.847+0000 7f013dc52740 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 0x55e6ff5d4640 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f01280017d0 tx=0x7f0128000ed0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.847+0000 7f013dc52740 1 -- 172.21.15.172:0/3793214245 shutdown_connections 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.848+0000 7f013dc52740 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55e6ff6016e0 0x55e6ff601ac0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.848+0000 7f013dc52740 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5c2a30 0x55e6ff5e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.848+0000 7f013dc52740 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5fd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.848+0000 7f01137fe640 0 client.9291 ms_handle_reset on v2:172.21.15.177:6833/3201353747 2024-04-14T08:42:53.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.848+0000 7f013dc52740 1 --2- 172.21.15.172:0/3793214245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cf970 0x55e6ff5d4640 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.848+0000 7f013dc52740 1 -- 172.21.15.172:0/3793214245 >> 172.21.15.172:0/3793214245 conn(0x55e6ff5a2e50 msgr2=0x55e6ff5d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:53.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.848+0000 7f013dc52740 1 -- 172.21.15.172:0/3793214245 shutdown_connections 2024-04-14T08:42:53.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.848+0000 7f013dc52740 1 -- 172.21.15.172:0/3793214245 wait complete. 2024-04-14T08:42:53.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.848+0000 7f013dc52740 20 client.9291 trim_cache size 0 max 16384 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.862+0000 7f013dc52740 1 Processor -- start 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.862+0000 7f013dc52740 1 -- start start 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f013dc52740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5c2a30 0x55e6ff5e3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f013dc52740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5cf970 0x55e6ff5e6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f013dc52740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5e6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f013dc52740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0134200fc0 con 0x55e6ff5c2a30 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f013dc52740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0134200030 con 0x55e6ff5cf970 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f013dc52740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f01340c8660 con 0x55e6ff5cfd50 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f0138bfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5c2a30 0x55e6ff5e3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f0138bfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5c2a30 0x55e6ff5e3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45438/0 (socket says 172.21.15.172:45438) 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f0138bfc640 1 -- 172.21.15.172:0/508532851 learned_addr learned my addr 172.21.15.172:0/508532851 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f0133fff640 1 --2- 172.21.15.172:0/508532851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5cf970 0x55e6ff5e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f01393fd640 1 --2- 172.21.15.172:0/508532851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f0138bfc640 1 -- 172.21.15.172:0/508532851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 msgr2=0x55e6ff5e6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:53.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f0138bfc640 1 --2- 172.21.15.172:0/508532851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5e6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f0138bfc640 1 -- 172.21.15.172:0/508532851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5cf970 msgr2=0x55e6ff5e6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:53.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f0138bfc640 1 --2- 172.21.15.172:0/508532851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5cf970 0x55e6ff5e6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.863+0000 7f0138bfc640 1 -- 172.21.15.172:0/508532851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6ff5a9450 con 0x55e6ff5c2a30 2024-04-14T08:42:53.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f0138bfc640 1 --2- 172.21.15.172:0/508532851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5c2a30 0x55e6ff5e3b20 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f0128007090 tx=0x7f012802b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9b6d962e04d1b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f011dffb640 1 -- 172.21.15.172:0/508532851 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01280137c0 con 0x55e6ff5c2a30 2024-04-14T08:42:53.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f011dffb640 1 -- 172.21.15.172:0/508532851 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0128027040 con 0x55e6ff5c2a30 2024-04-14T08:42:53.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f011dffb640 1 -- 172.21.15.172:0/508532851 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128021c70 con 0x55e6ff5c2a30 2024-04-14T08:42:53.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f013dc52740 1 -- 172.21.15.172:0/508532851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5c2a30 msgr2=0x55e6ff5e3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f013dc52740 1 --2- 172.21.15.172:0/508532851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5c2a30 0x55e6ff5e3b20 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f0128007090 tx=0x7f012802b000 comp rx=0 tx=0).stop 2024-04-14T08:42:53.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f013dc52740 1 -- 172.21.15.172:0/508532851 shutdown_connections 2024-04-14T08:42:53.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f013dc52740 1 --2- 172.21.15.172:0/508532851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cfd50 0x55e6ff5e6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f013dc52740 1 --2- 172.21.15.172:0/508532851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5cf970 0x55e6ff5e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f013dc52740 1 --2- 172.21.15.172:0/508532851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5c2a30 0x55e6ff5e3b20 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f013dc52740 1 -- 172.21.15.172:0/508532851 >> 172.21.15.172:0/508532851 conn(0x55e6ff5a2e50 msgr2=0x55e6ff5f0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:53.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f013dc52740 1 -- 172.21.15.172:0/508532851 shutdown_connections 2024-04-14T08:42:53.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.864+0000 7f013dc52740 1 -- 172.21.15.172:0/508532851 wait complete. 2024-04-14T08:42:53.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.865+0000 7f013dc52740 1 -- start start 2024-04-14T08:42:53.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.865+0000 7f013dc52740 1 Processor -- start 2024-04-14T08:42:53.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f013dc52740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5c2a30 0x55e6ff65dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f013dc52740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cf970 0x55e6ff5a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f013dc52740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cfd50 0x55e6ff5d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f013dc52740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e6ff610d50 con 0x55e6ff5cfd50 2024-04-14T08:42:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f013dc52740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f01341ffc50 con 0x55e6ff5c2a30 2024-04-14T08:42:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f013dc52740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55e6ff5d6b60 con 0x55e6ff5cf970 2024-04-14T08:42:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f013dc52740 10 client.0 _ll_register_callbacks cb 0x7f0128004ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:42:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f01393fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cfd50 0x55e6ff5d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f01393fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cfd50 0x55e6ff5d5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45446/0 (socket says 172.21.15.172:45446) 2024-04-14T08:42:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f01393fd640 1 -- 172.21.15.172:0/2610058465 learned_addr learned my addr 172.21.15.172:0/2610058465 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f0133fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cf970 0x55e6ff5a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f0138bfc640 1 --2- 172.21.15.172:0/2610058465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5c2a30 0x55e6ff65dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f01393fd640 1 -- 172.21.15.172:0/2610058465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cf970 msgr2=0x55e6ff5a73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:53.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f01393fd640 1 --2- 172.21.15.172:0/2610058465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cf970 0x55e6ff5a73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f01393fd640 1 -- 172.21.15.172:0/2610058465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5c2a30 msgr2=0x55e6ff65dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:53.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f01393fd640 1 --2- 172.21.15.172:0/2610058465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5c2a30 0x55e6ff65dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f01393fd640 1 -- 172.21.15.172:0/2610058465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6ff667120 con 0x55e6ff5cfd50 2024-04-14T08:42:53.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f01393fd640 1 --2- 172.21.15.172:0/2610058465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cfd50 0x55e6ff5d5c90 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f012c059000 tx=0x7f012c061cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f536893bca05891 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f01137fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:42:53.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f01137fe640 1 -- 172.21.15.172:0/2610058465 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c0556e0 con 0x55e6ff5cfd50 2024-04-14T08:42:53.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f01137fe640 1 -- 172.21.15.172:0/2610058465 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f012c089b50 con 0x55e6ff5cfd50 2024-04-14T08:42:53.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.866+0000 7f013dc52740 1 -- 172.21.15.172:0/2610058465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0128027040 con 0x55e6ff5cfd50 2024-04-14T08:42:53.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f01137fe640 1 -- 172.21.15.172:0/2610058465 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f012c077050 con 0x55e6ff5cfd50 2024-04-14T08:42:53.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f013dc52740 10 client.9297 fetch_fsmap learned FSMap version 88 2024-04-14T08:42:53.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f013dc52740 1 -- 172.21.15.172:0/2610058465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55e6ff5ecfb0 con 0x55e6ff5cfd50 2024-04-14T08:42:53.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f01137fe640 1 -- 172.21.15.172:0/2610058465 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0128027040 con 0x55e6ff5cfd50 2024-04-14T08:42:53.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f01137fe640 1 -- 172.21.15.172:0/2610058465 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f012c05b040 con 0x55e6ff5cfd50 2024-04-14T08:42:53.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f013dc52740 10 client.9297 fetch_fsmap finished waiting for FSMap version 88 2024-04-14T08:42:53.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f013dc52740 10 client.9297 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-14T08:42:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f013dc52740 20 client.9297 populate_metadata read hostname 'smithi172' 2024-04-14T08:42:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f013dc52740 1 --2- 172.21.15.172:0/2610058465 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55e6ff5d61d0 0x55e6ff66a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f013dc52740 4 client.9297 mds_command: new command op to 9060 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-14T08:42:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f013dc52740 1 -- 172.21.15.172:0/2610058465 --> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6ff514090 con 0x55e6ff5d61d0 2024-04-14T08:42:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f0138bfc640 1 --2- 172.21.15.172:0/2610058465 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55e6ff5d61d0 0x55e6ff66a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f01137fe640 10 client.9297 ms_handle_connect on v2:172.21.15.177:6833/3201353747 2024-04-14T08:42:53.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.867+0000 7f0138bfc640 1 --2- 172.21.15.172:0/2610058465 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55e6ff5d61d0 0x55e6ff66a700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e19ebe73735b9fce server_cookie=5a2d367f33f9b7f in_seq=0 out_seq=0 2024-04-14T08:42:53.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f01137fe640 1 -- 172.21.15.172:0/2610058465 <== mds.0 v2:172.21.15.177:6833/3201353747 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6ff514090 con 0x55e6ff5d61d0 2024-04-14T08:42:53.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f01137fe640 10 client.9297 handle_command_reply: tid=0 2024-04-14T08:42:53.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f013dc52740 1 client.9297 shutdown 2024-04-14T08:42:53.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f013dc52740 1 -- 172.21.15.172:0/2610058465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cfd50 msgr2=0x55e6ff5d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f013dc52740 1 --2- 172.21.15.172:0/2610058465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cfd50 0x55e6ff5d5c90 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f012c059000 tx=0x7f012c061cb0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f013dc52740 1 -- 172.21.15.172:0/2610058465 shutdown_connections 2024-04-14T08:42:53.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f013dc52740 1 --2- 172.21.15.172:0/2610058465 >> [v2:172.21.15.177:6833/3201353747,v1:172.21.15.177:6835/3201353747] conn(0x55e6ff5d61d0 0x55e6ff66a700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f013dc52740 1 --2- 172.21.15.172:0/2610058465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e6ff5cfd50 0x55e6ff5d5c90 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f013dc52740 1 --2- 172.21.15.172:0/2610058465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e6ff5cf970 0x55e6ff5a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f01137fe640 0 client.9297 ms_handle_reset on v2:172.21.15.177:6833/3201353747 2024-04-14T08:42:53.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f013dc52740 1 --2- 172.21.15.172:0/2610058465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e6ff5c2a30 0x55e6ff65dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f013dc52740 1 -- 172.21.15.172:0/2610058465 >> 172.21.15.172:0/2610058465 conn(0x55e6ff5a2e50 msgr2=0x55e6ff5f0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:53.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f013dc52740 1 -- 172.21.15.172:0/2610058465 shutdown_connections 2024-04-14T08:42:53.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f013dc52740 1 -- 172.21.15.172:0/2610058465 wait complete. 2024-04-14T08:42:53.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.881+0000 7f013dc52740 20 client.9297 trim_cache size 0 max 16384 2024-04-14T08:42:53.883 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:42:53.883 INFO:teuthology.orchestra.run.smithi172.stdout: "message": "", 2024-04-14T08:42:53.884 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0 2024-04-14T08:42:53.884 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:42:53.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.882+0000 7f013ae60640 1 -- 172.21.15.172:0/3477031214 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f011806c8f0 msgr2=0x7f011806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.882+0000 7f013ae60640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f011806c8f0 0x7f011806edb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0128000f60 tx=0x7f0128007e40 comp rx=0 tx=0).stop 2024-04-14T08:42:53.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.882+0000 7f013ae60640 1 -- 172.21.15.172:0/3477031214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0134162ba0 msgr2=0x7f01340b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.882+0000 7f013ae60640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0134162ba0 0x7f01340b3a10 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f012c00cb20 tx=0x7f012c0020f0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.882+0000 7f013ae60640 1 -- 172.21.15.172:0/3477031214 shutdown_connections 2024-04-14T08:42:53.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.882+0000 7f013ae60640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f011806c8f0 0x7f011806edb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.882+0000 7f013ae60640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0134162ba0 0x7f01340b3a10 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.882+0000 7f013ae60640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0134162710 0x7f01340ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.882+0000 7f013ae60640 1 --2- 172.21.15.172:0/3477031214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f013415e430 0x7f0134152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.882+0000 7f013ae60640 1 -- 172.21.15.172:0/3477031214 >> 172.21.15.172:0/3477031214 conn(0x7f013409d560 msgr2=0x7f01340b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:53.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.882+0000 7f013ae60640 1 -- 172.21.15.172:0/3477031214 shutdown_connections 2024-04-14T08:42:53.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.882+0000 7f013ae60640 1 -- 172.21.15.172:0/3477031214 wait complete. 2024-04-14T08:42:53.893 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:42:53.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.994+0000 7f8fc67aa640 1 Processor -- start 2024-04-14T08:42:53.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.994+0000 7f8fc67aa640 1 -- start start 2024-04-14T08:42:53.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fc67aa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8fc015e890 0x7f8fc00d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fc67aa640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8fc00cedd0 0x7f8fc00b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:53.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fc67aa640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fc00cf320 con 0x7f8fc00d2830 2024-04-14T08:42:53.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fc67aa640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc00cff30 con 0x7f8fc015e890 2024-04-14T08:42:53.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fc67aa640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc00d2350 con 0x7f8fc00cedd0 2024-04-14T08:42:53.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fbf7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8fc015e890 0x7f8fc00d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fbf7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8fc015e890 0x7f8fc00d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45450/0 (socket says 172.21.15.172:45450) 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fbf7fe640 1 -- 172.21.15.172:0/1793266363 learned_addr learned my addr 172.21.15.172:0/1793266363 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fbffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f8fc00d2830 0x7f8fc00d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50848/0 (socket says 172.21.15.172:50848) 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fc4d47640 1 --2- 172.21.15.172:0/1793266363 >> v2:172.21.15.177:3300/0 conn(0x7f8fc00cedd0 0x7f8fc00b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fbf7fe640 1 -- 172.21.15.172:0/1793266363 >> v2:172.21.15.177:3300/0 conn(0x7f8fc00cedd0 msgr2=0x7f8fc00b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fbf7fe640 1 --2- 172.21.15.172:0/1793266363 >> v2:172.21.15.177:3300/0 conn(0x7f8fc00cedd0 0x7f8fc00b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fbf7fe640 1 -- 172.21.15.172:0/1793266363 >> v1:172.21.15.172:6789/0 conn(0x7f8fc00d2830 legacy=0x7f8fc00d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fbf7fe640 1 -- 172.21.15.172:0/1793266363 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc00cf600 con 0x7f8fc015e890 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.995+0000 7f8fbeffd640 1 -- 172.21.15.172:0/1793266363 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1098815025 0 0) 0x7f8fc00cf320 con 0x7f8fc00d2830 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.996+0000 7f8fbf7fe640 1 --2- 172.21.15.172:0/1793266363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc015e890 0x7f8fc00d1300 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f8fc008dbe0 tx=0x7f8fb402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e468089543ba3f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.996+0000 7f8fbeffd640 1 -- 172.21.15.172:0/1793266363 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb402f850 con 0x7f8fc015e890 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.996+0000 7f8fbeffd640 1 -- 172.21.15.172:0/1793266363 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb402f9f0 con 0x7f8fc015e890 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.996+0000 7f8fbeffd640 1 -- 172.21.15.172:0/1793266363 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb402fdc0 con 0x7f8fc015e890 2024-04-14T08:42:53.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.997+0000 7f8fc67aa640 1 -- 172.21.15.172:0/1793266363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc015e890 msgr2=0x7f8fc00d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:53.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.997+0000 7f8fc67aa640 1 --2- 172.21.15.172:0/1793266363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc015e890 0x7f8fc00d1300 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f8fc008dbe0 tx=0x7f8fb402ba30 comp rx=0 tx=0).stop 2024-04-14T08:42:53.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.997+0000 7f8fc67aa640 1 -- 172.21.15.172:0/1793266363 shutdown_connections 2024-04-14T08:42:53.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.997+0000 7f8fc67aa640 1 --2- 172.21.15.172:0/1793266363 >> v2:172.21.15.177:3300/0 conn(0x7f8fc00cedd0 0x7f8fc00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.997+0000 7f8fc67aa640 1 --2- 172.21.15.172:0/1793266363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc015e890 0x7f8fc00d1300 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:53.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.997+0000 7f8fc67aa640 1 -- 172.21.15.172:0/1793266363 >> 172.21.15.172:0/1793266363 conn(0x7f8fc009d590 msgr2=0x7f8fc00d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:53.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.997+0000 7f8fc67aa640 1 -- 172.21.15.172:0/1793266363 shutdown_connections 2024-04-14T08:42:53.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.997+0000 7f8fc67aa640 1 -- 172.21.15.172:0/1793266363 wait complete. 2024-04-14T08:42:54.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.998+0000 7f8fc67aa640 1 Processor -- start 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.998+0000 7f8fc67aa640 1 -- start start 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fc67aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc00cedd0 0x7f8fc0160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fc67aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fc00d2830 0x7f8fc01596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fc67aa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc015e890 0x7f8fc00cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fc67aa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc00d1840 con 0x7f8fc00cedd0 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fc67aa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc00cfbc0 con 0x7f8fc015e890 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fc67aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc00b3c50 con 0x7f8fc00d2830 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fbffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc00cedd0 0x7f8fc0160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fbffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc00cedd0 0x7f8fc0160cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45460/0 (socket says 172.21.15.172:45460) 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fbffff640 1 -- 172.21.15.172:0/3665525153 learned_addr learned my addr 172.21.15.172:0/3665525153 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fc4d47640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc015e890 0x7f8fc00cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fbf7fe640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fc00d2830 0x7f8fc01596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fbffff640 1 -- 172.21.15.172:0/3665525153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fc00d2830 msgr2=0x7f8fc01596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:54.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fbffff640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fc00d2830 0x7f8fc01596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fbffff640 1 -- 172.21.15.172:0/3665525153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc015e890 msgr2=0x7f8fc00cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fbffff640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc015e890 0x7f8fc00cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fbffff640 1 -- 172.21.15.172:0/3665525153 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc015ff00 con 0x7f8fc00cedd0 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:53.999+0000 7f8fbffff640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc00cedd0 0x7f8fc0160cc0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c500 tx=0x7f8fb000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.000+0000 7f8fbcff9640 1 -- 172.21.15.172:0/3665525153 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0018070 con 0x7f8fc00cedd0 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.000+0000 7f8fbcff9640 1 -- 172.21.15.172:0/3665525153 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb0007580 con 0x7f8fc00cedd0 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.000+0000 7f8fc67aa640 1 -- 172.21.15.172:0/3665525153 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc01601e0 con 0x7f8fc00cedd0 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.000+0000 7f8fbcff9640 1 -- 172.21.15.172:0/3665525153 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0004a20 con 0x7f8fc00cedd0 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.000+0000 7f8fc67aa640 1 -- 172.21.15.172:0/3665525153 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc01607a0 con 0x7f8fc00cedd0 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.001+0000 7f8fc67aa640 1 -- 172.21.15.172:0/3665525153 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc0068520 con 0x7f8fc00cedd0 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.001+0000 7f8fbcff9640 1 -- 172.21.15.172:0/3665525153 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fb0004bc0 con 0x7f8fc00cedd0 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.001+0000 7f8fbcff9640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8fac06c8f0 0x7f8fac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.002+0000 7f8fbcff9640 1 -- 172.21.15.172:0/3665525153 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f8fb008af30 con 0x7f8fc00cedd0 2024-04-14T08:42:54.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.002+0000 7f8fbf7fe640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8fac06c8f0 0x7f8fac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:54.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.002+0000 7f8fbf7fe640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8fac06c8f0 0x7f8fac06edb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4036000 tx=0x7f8fb4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:54.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.006+0000 7f8fbcff9640 1 -- 172.21.15.172:0/3665525153 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8fb0058b90 con 0x7f8fc00cedd0 2024-04-14T08:42:54.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.222+0000 7f8fc67aa640 1 -- 172.21.15.172:0/3665525153 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8fc00d2610 con 0x7f8fc00cedd0 2024-04-14T08:42:54.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.222+0000 7f8fbcff9640 1 -- 172.21.15.172:0/3665525153 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) ==== 53+0+73 (secure 0 0 0) 0x7f8fb005ce90 con 0x7f8fc00cedd0 2024-04-14T08:42:54.223 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:42:54.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.224+0000 7f8fc67aa640 1 -- 172.21.15.172:0/3665525153 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8fac06c8f0 msgr2=0x7f8fac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.224+0000 7f8fc67aa640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8fac06c8f0 0x7f8fac06edb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4036000 tx=0x7f8fb4008030 comp rx=0 tx=0).stop 2024-04-14T08:42:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.224+0000 7f8fc67aa640 1 -- 172.21.15.172:0/3665525153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc00cedd0 msgr2=0x7f8fc0160cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.224+0000 7f8fc67aa640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc00cedd0 0x7f8fc0160cc0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c500 tx=0x7f8fb000c9c0 comp rx=0 tx=0).stop 2024-04-14T08:42:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.224+0000 7f8fc67aa640 1 -- 172.21.15.172:0/3665525153 shutdown_connections 2024-04-14T08:42:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.224+0000 7f8fc67aa640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8fac06c8f0 0x7f8fac06edb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.224+0000 7f8fc67aa640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fc015e890 0x7f8fc00cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.224+0000 7f8fc67aa640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fc00d2830 0x7f8fc01596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.224+0000 7f8fc67aa640 1 --2- 172.21.15.172:0/3665525153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc00cedd0 0x7f8fc0160cc0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.224+0000 7f8fc67aa640 1 -- 172.21.15.172:0/3665525153 >> 172.21.15.172:0/3665525153 conn(0x7f8fc009d590 msgr2=0x7f8fc006c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.224+0000 7f8fc67aa640 1 -- 172.21.15.172:0/3665525153 shutdown_connections 2024-04-14T08:42:54.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54.224+0000 7f8fc67aa640 1 -- 172.21.15.172:0/3665525153 wait complete. 2024-04-14T08:42:54.235 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:42:54.235 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:42:54.236 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:42:54.236 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:42:54.236 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:42:54.236 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:42:54.236 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:42:54.236 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:42:54.236 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:42:54.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:54.236 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:42:54.251 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:42:54.252 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42095sec preferred_lft 42095sec 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:42:54.253 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:42:54.254 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:42:54.254 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:54.254 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:42:54.254 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:42:54.254 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:42:54.254 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:42:54.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:54.254 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:42:54.318 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:42:54.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:54.319 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:42:54.375 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:42:54.375 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:42:54.375 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:42:54.375 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:42:54.376 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:42:54.376 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:42:54.376 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:42:54.376 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:42:54.376 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:42:54.376 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:42:54.376 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:42:54.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:42:54.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:54.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:54.511 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:42:54.526 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:42:54.526 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:42:54.526 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:42:54.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:54.527 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:42:54.594 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:42:54.594 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:42:54.649 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:42:54.649 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:42:54.649 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:42:54.703 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:42:54.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:54.704 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:42:54.790 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:42:54.791 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:42:54.791 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:42:54.797 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:42:54.797 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:42:54.797 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:42:54.797 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:42:54.797 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:42:54.805 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:42:54.805 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:42:54.805 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:42:54.805 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:42:54.805 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:42:54.805 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:42:54.805 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:42:54.805 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:42:54.806 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:42:54.806 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:42:54.806 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:42:54.806 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:42:54.806 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:42:54.807 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:42:54.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:54.807 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:42:54.874 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:42:54.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:42:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:54.970 INFO:teuthology.orchestra.run.smithi172.stdout:client9309 2024-04-14T08:42:54.970 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:42:54.970 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:54.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:54.986 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-14T08:42:55.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:55.045 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-14T08:42:55.124 INFO:teuthology.orchestra.run.smithi172.stderr:8+0 records in 2024-04-14T08:42:55.124 INFO:teuthology.orchestra.run.smithi172.stderr:8+0 records out 2024-04-14T08:42:55.124 INFO:teuthology.orchestra.run.smithi172.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.024109 s, 348 MB/s 2024-04-14T08:42:55.125 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:42:55.125 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:42:55.125 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:42:55.125 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:42:55.125 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-14T08:42:55.126 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:42:55.234 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627779 2024-04-14T08:42:55.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:55.514 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:42:55.514 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:42:55.514 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:42:55.514 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:42:55.514 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-14T08:42:55.514 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:42:55.515 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:42:55.568 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:42:55.625 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627780 2024-04-14T08:42:55.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:42:55.637 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:42:55.637 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:55.703 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:42:55.703 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:55.762 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:42:55.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.906+0000 7f3f0f715640 1 Processor -- start 2024-04-14T08:42:55.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.907+0000 7f3f0f715640 1 -- start start 2024-04-14T08:42:55.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.907+0000 7f3f0f715640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f080d2830 0x7f3f080d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:55.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.907+0000 7f3f0f715640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f080d3150 con 0x7f3f0815e890 2024-04-14T08:42:55.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.907+0000 7f3f0f715640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f080cf320 con 0x7f3f080cedd0 2024-04-14T08:42:55.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.907+0000 7f3f0f715640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3f080cff30 con 0x7f3f080d2830 2024-04-14T08:42:55.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.908+0000 7f3f0ccb0640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3f0815e890 0x7f3f080d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50854/0 (socket says 172.21.15.172:50854) 2024-04-14T08:42:55.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.908+0000 7f3f0ccb0640 1 -- 172.21.15.172:0/1475841230 learned_addr learned my addr 172.21.15.172:0/1475841230 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:55.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.908+0000 7f3f0d4b1640 1 --2- 172.21.15.172:0/1475841230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f080d2830 0x7f3f080d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:55.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.908+0000 7f3efffff640 1 -- 172.21.15.172:0/1475841230 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 350901266 0 0) 0x7f3f080d3150 con 0x7f3f0815e890 2024-04-14T08:42:55.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.908+0000 7f3efffff640 1 -- 172.21.15.172:0/1475841230 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ef4003520 con 0x7f3f0815e890 2024-04-14T08:42:55.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.908+0000 7f3efffff640 1 -- 172.21.15.172:0/1475841230 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898668806 0 0) 0x7f3f080cf320 con 0x7f3f080cedd0 2024-04-14T08:42:55.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.908+0000 7f3efffff640 1 -- 172.21.15.172:0/1475841230 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f080d3150 con 0x7f3f080cedd0 2024-04-14T08:42:55.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.908+0000 7f3f0d4b1640 1 -- 172.21.15.172:0/1475841230 >> v1:172.21.15.177:6789/0 conn(0x7f3f080cedd0 legacy=0x7f3f080b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:55.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.908+0000 7f3f0d4b1640 1 -- 172.21.15.172:0/1475841230 >> v1:172.21.15.172:6789/0 conn(0x7f3f0815e890 legacy=0x7f3f080d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:55.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.908+0000 7f3f0d4b1640 1 -- 172.21.15.172:0/1475841230 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f080cf600 con 0x7f3f080d2830 2024-04-14T08:42:55.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.909+0000 7f3f0d4b1640 1 --2- 172.21.15.172:0/1475841230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f080d2830 0x7f3f080d2c10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3f040097d0 tx=0x7f3f0402b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa902900b2ab3071 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:55.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.909+0000 7f3efffff640 1 -- 172.21.15.172:0/1475841230 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04034070 con 0x7f3f080d2830 2024-04-14T08:42:55.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.909+0000 7f3efffff640 1 -- 172.21.15.172:0/1475841230 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f04030880 con 0x7f3f080d2830 2024-04-14T08:42:55.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.909+0000 7f3efffff640 1 -- 172.21.15.172:0/1475841230 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04030c50 con 0x7f3f080d2830 2024-04-14T08:42:55.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.909+0000 7f3f0f715640 1 -- 172.21.15.172:0/1475841230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f080d2830 msgr2=0x7f3f080d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:55.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.909+0000 7f3f0f715640 1 --2- 172.21.15.172:0/1475841230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f080d2830 0x7f3f080d2c10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3f040097d0 tx=0x7f3f0402b9c0 comp rx=0 tx=0).stop 2024-04-14T08:42:55.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.909+0000 7f3f0f715640 1 -- 172.21.15.172:0/1475841230 shutdown_connections 2024-04-14T08:42:55.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.909+0000 7f3f0f715640 1 --2- 172.21.15.172:0/1475841230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f080d2830 0x7f3f080d2c10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:55.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.909+0000 7f3f0f715640 1 -- 172.21.15.172:0/1475841230 >> 172.21.15.172:0/1475841230 conn(0x7f3f0809d590 msgr2=0x7f3f080d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:55.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.910+0000 7f3f0f715640 1 -- 172.21.15.172:0/1475841230 shutdown_connections 2024-04-14T08:42:55.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.910+0000 7f3f0f715640 1 -- 172.21.15.172:0/1475841230 wait complete. 2024-04-14T08:42:55.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.911+0000 7f3f0f715640 1 Processor -- start 2024-04-14T08:42:55.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.911+0000 7f3f0f715640 1 -- start start 2024-04-14T08:42:55.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.911+0000 7f3f0f715640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f080cedd0 0x7f3f08152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:55.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.911+0000 7f3f0f715640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f080d2830 0x7f3f080b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:55.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.911+0000 7f3f0f715640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f0815e890 0x7f3f080b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:55.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.911+0000 7f3f0f715640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3f080cfbc0 con 0x7f3f0815e890 2024-04-14T08:42:55.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.911+0000 7f3f0f715640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f080d2350 con 0x7f3f080cedd0 2024-04-14T08:42:55.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.911+0000 7f3f0f715640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3f080b3e20 con 0x7f3f080d2830 2024-04-14T08:42:55.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0dcb2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f0815e890 0x7f3f080b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:55.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0d4b1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f080cedd0 0x7f3f08152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:55.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0ccb0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f080d2830 0x7f3f080b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:55.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0dcb2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f0815e890 0x7f3f080b5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45466/0 (socket says 172.21.15.172:45466) 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0dcb2640 1 -- 172.21.15.172:0/3846584725 learned_addr learned my addr 172.21.15.172:0/3846584725 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0d4b1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f080cedd0 0x7f3f08152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33342/0 (socket says 172.21.15.172:33342) 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0ccb0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f080d2830 0x7f3f080b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:40934/0 (socket says 172.21.15.172:40934) 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0dcb2640 1 -- 172.21.15.172:0/3846584725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f080d2830 msgr2=0x7f3f080b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0dcb2640 1 --2- 172.21.15.172:0/3846584725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f080d2830 0x7f3f080b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0dcb2640 1 -- 172.21.15.172:0/3846584725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f080cedd0 msgr2=0x7f3f08152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0dcb2640 1 --2- 172.21.15.172:0/3846584725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f080cedd0 0x7f3f08152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0dcb2640 1 -- 172.21.15.172:0/3846584725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f081f47f0 con 0x7f3f0815e890 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0dcb2640 1 --2- 172.21.15.172:0/3846584725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f0815e890 0x7f3f080b5fd0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c790 tx=0x7f3f0000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3efdffb640 1 -- 172.21.15.172:0/3846584725 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00003b70 con 0x7f3f0815e890 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3efdffb640 1 -- 172.21.15.172:0/3846584725 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f00003d10 con 0x7f3f0815e890 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3efdffb640 1 -- 172.21.15.172:0/3846584725 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000aa50 con 0x7f3f0815e890 2024-04-14T08:42:55.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0f715640 1 -- 172.21.15.172:0/3846584725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f081f4ab0 con 0x7f3f0815e890 2024-04-14T08:42:55.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.912+0000 7f3f0f715640 1 -- 172.21.15.172:0/3846584725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f081f5070 con 0x7f3f0815e890 2024-04-14T08:42:55.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.913+0000 7f3f0f715640 1 -- 172.21.15.172:0/3846584725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f08068520 con 0x7f3f0815e890 2024-04-14T08:42:55.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.914+0000 7f3efdffb640 1 -- 172.21.15.172:0/3846584725 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f0000abf0 con 0x7f3f0815e890 2024-04-14T08:42:55.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.914+0000 7f3efdffb640 1 --2- 172.21.15.172:0/3846584725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3eec06c8f0 0x7f3eec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:55.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.914+0000 7f3efdffb640 1 -- 172.21.15.172:0/3846584725 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f3f0008ba70 con 0x7f3f0815e890 2024-04-14T08:42:55.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.914+0000 7f3f0d4b1640 1 --2- 172.21.15.172:0/3846584725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3eec06c8f0 0x7f3eec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:55.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.915+0000 7f3f0d4b1640 1 --2- 172.21.15.172:0/3846584725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3eec06c8f0 0x7f3eec06edb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3f04036000 tx=0x7f3f0403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:55.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:55.919+0000 7f3efdffb640 1 -- 172.21.15.172:0/3846584725 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3f00059750 con 0x7f3f0815e890 2024-04-14T08:42:56.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.096+0000 7f3f0f715640 1 -- 172.21.15.172:0/3846584725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f08067750 con 0x7f3f0815e890 2024-04-14T08:42:56.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.097+0000 7f3efdffb640 1 -- 172.21.15.172:0/3846584725 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+17012 (secure 0 0 0) 0x7f3f0005da50 con 0x7f3f0815e890 2024-04-14T08:42:56.097 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:56.097 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":116,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:42:41.900871+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:42:38.455900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-14T08:42:39.198157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:42:56.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.099+0000 7f3f0f715640 1 -- 172.21.15.172:0/3846584725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3eec06c8f0 msgr2=0x7f3eec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.099+0000 7f3f0f715640 1 --2- 172.21.15.172:0/3846584725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3eec06c8f0 0x7f3eec06edb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3f04036000 tx=0x7f3f0403b040 comp rx=0 tx=0).stop 2024-04-14T08:42:56.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.099+0000 7f3f0f715640 1 -- 172.21.15.172:0/3846584725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f0815e890 msgr2=0x7f3f080b5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.099+0000 7f3f0f715640 1 --2- 172.21.15.172:0/3846584725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f0815e890 0x7f3f080b5fd0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c790 tx=0x7f3f0000cc50 comp rx=0 tx=0).stop 2024-04-14T08:42:56.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.099+0000 7f3f0f715640 1 -- 172.21.15.172:0/3846584725 shutdown_connections 2024-04-14T08:42:56.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.099+0000 7f3f0f715640 1 --2- 172.21.15.172:0/3846584725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3eec06c8f0 0x7f3eec06edb0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.099+0000 7f3f0f715640 1 --2- 172.21.15.172:0/3846584725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f0815e890 0x7f3f080b5fd0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.099+0000 7f3f0f715640 1 --2- 172.21.15.172:0/3846584725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3f080d2830 0x7f3f080b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.099+0000 7f3f0f715640 1 --2- 172.21.15.172:0/3846584725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f080cedd0 0x7f3f08152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.099+0000 7f3f0f715640 1 -- 172.21.15.172:0/3846584725 >> 172.21.15.172:0/3846584725 conn(0x7f3f0809d590 msgr2=0x7f3f0806d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:56.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.099+0000 7f3f0f715640 1 -- 172.21.15.172:0/3846584725 shutdown_connections 2024-04-14T08:42:56.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.099+0000 7f3f0f715640 1 -- 172.21.15.172:0/3846584725 wait complete. 2024-04-14T08:42:56.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:42:38.455900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:42:39.198157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:42:56.111 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:42:56.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.215+0000 7f990101c640 1 Processor -- start 2024-04-14T08:42:56.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.215+0000 7f990101c640 1 -- start start 2024-04-14T08:42:56.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f990101c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f98fc072450 0x7f98fc0728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f990101c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98fc160b00 con 0x7f98fc159c20 2024-04-14T08:42:56.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f990101c640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98fc15cf10 con 0x7f98fc0ce200 2024-04-14T08:42:56.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f990101c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f98fc15fc80 con 0x7f98fc072450 2024-04-14T08:42:56.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f98fad9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f98fc159c20 0x7f98fc15c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50870/0 (socket says 172.21.15.172:50870) 2024-04-14T08:42:56.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f98fad9d640 1 -- 172.21.15.172:0/4260394115 learned_addr learned my addr 172.21.15.172:0/4260394115 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:56.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f98fb59e640 1 --2- 172.21.15.172:0/4260394115 >> v2:172.21.15.177:3300/0 conn(0x7f98fc072450 0x7f98fc0728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f98fa59c640 1 -- 172.21.15.172:0/4260394115 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1867094968 0 0) 0x7f98fc160b00 con 0x7f98fc159c20 2024-04-14T08:42:56.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f98fa59c640 1 -- 172.21.15.172:0/4260394115 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98e8003520 con 0x7f98fc159c20 2024-04-14T08:42:56.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f98fb59e640 1 -- 172.21.15.172:0/4260394115 >> v1:172.21.15.177:6789/0 conn(0x7f98fc0ce200 legacy=0x7f98fc1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f98fa59c640 1 -- 172.21.15.172:0/4260394115 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1277221193 0 0) 0x7f98fc15cf10 con 0x7f98fc0ce200 2024-04-14T08:42:56.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f98fb59e640 1 -- 172.21.15.172:0/4260394115 >> v1:172.21.15.172:6789/0 conn(0x7f98fc159c20 legacy=0x7f98fc15c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.216+0000 7f98fb59e640 1 -- 172.21.15.172:0/4260394115 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc15d1f0 con 0x7f98fc072450 2024-04-14T08:42:56.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.217+0000 7f98fb59e640 1 --2- 172.21.15.172:0/4260394115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98fc072450 0x7f98fc0728c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009d90 tx=0x7f98f402bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b134175ce10bbf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:56.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.217+0000 7f98fa59c640 1 -- 172.21.15.172:0/4260394115 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4039070 con 0x7f98fc072450 2024-04-14T08:42:56.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.217+0000 7f98fa59c640 1 -- 172.21.15.172:0/4260394115 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f402cd00 con 0x7f98fc072450 2024-04-14T08:42:56.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.217+0000 7f98fa59c640 1 -- 172.21.15.172:0/4260394115 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4034660 con 0x7f98fc072450 2024-04-14T08:42:56.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.218+0000 7f990101c640 1 -- 172.21.15.172:0/4260394115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98fc072450 msgr2=0x7f98fc0728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.218+0000 7f990101c640 1 --2- 172.21.15.172:0/4260394115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98fc072450 0x7f98fc0728c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009d90 tx=0x7f98f402bf50 comp rx=0 tx=0).stop 2024-04-14T08:42:56.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.218+0000 7f990101c640 1 -- 172.21.15.172:0/4260394115 shutdown_connections 2024-04-14T08:42:56.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.218+0000 7f990101c640 1 --2- 172.21.15.172:0/4260394115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98fc072450 0x7f98fc0728c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.218+0000 7f990101c640 1 -- 172.21.15.172:0/4260394115 >> 172.21.15.172:0/4260394115 conn(0x7f98fc06b520 msgr2=0x7f98fc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:56.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.218+0000 7f990101c640 1 -- 172.21.15.172:0/4260394115 shutdown_connections 2024-04-14T08:42:56.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.218+0000 7f990101c640 1 -- 172.21.15.172:0/4260394115 wait complete. 2024-04-14T08:42:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.219+0000 7f990101c640 1 Processor -- start 2024-04-14T08:42:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.219+0000 7f990101c640 1 -- start start 2024-04-14T08:42:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f990101c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98fc072450 0x7f98fc154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f990101c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98fc0ce200 0x7f98fc0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f990101c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98fc159c20 0x7f98fc0a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f990101c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc15a000 con 0x7f98fc0ce200 2024-04-14T08:42:56.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f990101c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc15ff60 con 0x7f98fc159c20 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f990101c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f98fc0b5350 con 0x7f98fc072450 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98fc0ce200 0x7f98fc0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98fc0ce200 0x7f98fc0b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45472/0 (socket says 172.21.15.172:45472) 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98f3fff640 1 -- 172.21.15.172:0/3550136633 learned_addr learned my addr 172.21.15.172:0/3550136633 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98fb59e640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98fc159c20 0x7f98fc0a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98fad9d640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98fc072450 0x7f98fc154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98f3fff640 1 -- 172.21.15.172:0/3550136633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98fc072450 msgr2=0x7f98fc154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98f3fff640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98fc072450 0x7f98fc154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98f3fff640 1 -- 172.21.15.172:0/3550136633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98fc159c20 msgr2=0x7f98fc0a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98f3fff640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98fc159c20 0x7f98fc0a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98f3fff640 1 -- 172.21.15.172:0/3550136633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc0b2810 con 0x7f98fc0ce200 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98fb59e640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98fc159c20 0x7f98fc0a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98f3fff640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98fc0ce200 0x7f98fc0b1ee0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f98e400c4b0 tx=0x7f98e400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:56.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98f37fe640 1 -- 172.21.15.172:0/3550136633 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4017070 con 0x7f98fc0ce200 2024-04-14T08:42:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98f37fe640 1 -- 172.21.15.172:0/3550136633 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e4005480 con 0x7f98fc0ce200 2024-04-14T08:42:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f98f37fe640 1 -- 172.21.15.172:0/3550136633 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4003830 con 0x7f98fc0ce200 2024-04-14T08:42:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f990101c640 1 -- 172.21.15.172:0/3550136633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98fc0ad2a0 con 0x7f98fc0ce200 2024-04-14T08:42:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.220+0000 7f990101c640 1 -- 172.21.15.172:0/3550136633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98fc0ad860 con 0x7f98fc0ce200 2024-04-14T08:42:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.221+0000 7f990101c640 1 -- 172.21.15.172:0/3550136633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98fc0ada60 con 0x7f98fc0ce200 2024-04-14T08:42:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.222+0000 7f98f37fe640 1 -- 172.21.15.172:0/3550136633 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98e401a070 con 0x7f98fc0ce200 2024-04-14T08:42:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.222+0000 7f98f37fe640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f98dc06c8f0 0x7f98dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.222+0000 7f98f37fe640 1 -- 172.21.15.172:0/3550136633 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f98e408af40 con 0x7f98fc0ce200 2024-04-14T08:42:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.222+0000 7f98fad9d640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f98dc06c8f0 0x7f98dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.223+0000 7f98fad9d640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f98dc06c8f0 0x7f98dc06edb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f98ec005880 tx=0x7f98ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:56.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.227+0000 7f98f37fe640 1 -- 172.21.15.172:0/3550136633 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f98fc0ada60 con 0x7f98fc0ce200 2024-04-14T08:42:56.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.400+0000 7f990101c640 1 -- 172.21.15.172:0/3550136633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98fc068590 con 0x7f98fc0ce200 2024-04-14T08:42:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.401+0000 7f98f37fe640 1 -- 172.21.15.172:0/3550136633 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) ==== 93+0+10 (secure 0 0 0) 0x7f98e4058c20 con 0x7f98fc0ce200 2024-04-14T08:42:56.401 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:42:56.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.402+0000 7f990101c640 1 -- 172.21.15.172:0/3550136633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f98dc06c8f0 msgr2=0x7f98dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.402+0000 7f990101c640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f98dc06c8f0 0x7f98dc06edb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f98ec005880 tx=0x7f98ec009040 comp rx=0 tx=0).stop 2024-04-14T08:42:56.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.402+0000 7f990101c640 1 -- 172.21.15.172:0/3550136633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98fc0ce200 msgr2=0x7f98fc0b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.402+0000 7f990101c640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98fc0ce200 0x7f98fc0b1ee0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f98e400c4b0 tx=0x7f98e400c970 comp rx=0 tx=0).stop 2024-04-14T08:42:56.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.403+0000 7f990101c640 1 -- 172.21.15.172:0/3550136633 shutdown_connections 2024-04-14T08:42:56.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.403+0000 7f990101c640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f98dc06c8f0 0x7f98dc06edb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.403+0000 7f990101c640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98fc159c20 0x7f98fc0a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.403+0000 7f990101c640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98fc0ce200 0x7f98fc0b1ee0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.403+0000 7f990101c640 1 --2- 172.21.15.172:0/3550136633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98fc072450 0x7f98fc154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.403+0000 7f990101c640 1 -- 172.21.15.172:0/3550136633 >> 172.21.15.172:0/3550136633 conn(0x7f98fc06b520 msgr2=0x7f98fc15f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:56.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.403+0000 7f990101c640 1 -- 172.21.15.172:0/3550136633 shutdown_connections 2024-04-14T08:42:56.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.403+0000 7f990101c640 1 -- 172.21.15.172:0/3550136633 wait complete. 2024-04-14T08:42:56.414 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:42:56.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.518+0000 7f1d7c1c8640 1 Processor -- start 2024-04-14T08:42:56.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.519+0000 7f1d7c1c8640 1 -- start start 2024-04-14T08:42:56.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.519+0000 7f1d7c1c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1d7415e430 0x7f1d74162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.519+0000 7f1d7c1c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d740cefb0 con 0x7f1d74162540 2024-04-14T08:42:56.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.519+0000 7f1d7c1c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d740cf290 con 0x7f1d740d2e00 2024-04-14T08:42:56.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.519+0000 7f1d7c1c8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f1d740cf570 con 0x7f1d7415e430 2024-04-14T08:42:56.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.519+0000 7f1d79f64640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1d7415e430 0x7f1d74162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.520+0000 7f1d79763640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1d74162540 0x7f1d740b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50872/0 (socket says 172.21.15.172:50872) 2024-04-14T08:42:56.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.520+0000 7f1d79f64640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1d7415e430 0x7f1d74162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45482/0 (socket says 172.21.15.172:45482) 2024-04-14T08:42:56.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.520+0000 7f1d79763640 1 -- 172.21.15.172:0/3695922960 learned_addr learned my addr 172.21.15.172:0/3695922960 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:56.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.520+0000 7f1d78f62640 1 -- 172.21.15.172:0/3695922960 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3056064502 0 0) 0x7f1d740cf290 con 0x7f1d740d2e00 2024-04-14T08:42:56.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.520+0000 7f1d79f64640 1 -- 172.21.15.172:0/3695922960 >> v1:172.21.15.177:6789/0 conn(0x7f1d740d2e00 legacy=0x7f1d740b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.520+0000 7f1d79f64640 1 -- 172.21.15.172:0/3695922960 >> v1:172.21.15.172:6789/0 conn(0x7f1d74162540 legacy=0x7f1d740b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.520+0000 7f1d79f64640 1 -- 172.21.15.172:0/3695922960 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d740cf850 con 0x7f1d7415e430 2024-04-14T08:42:56.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.521+0000 7f1d79f64640 1 --2- 172.21.15.172:0/3695922960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7415e430 0x7f1d74162000 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009d90 tx=0x7f1d7002bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6994bd51a56be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:56.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.521+0000 7f1d78f62640 1 -- 172.21.15.172:0/3695922960 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70039070 con 0x7f1d7415e430 2024-04-14T08:42:56.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.521+0000 7f1d78f62640 1 -- 172.21.15.172:0/3695922960 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d7002cab0 con 0x7f1d7415e430 2024-04-14T08:42:56.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.521+0000 7f1d78f62640 1 -- 172.21.15.172:0/3695922960 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70034440 con 0x7f1d7415e430 2024-04-14T08:42:56.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.521+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/3695922960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7415e430 msgr2=0x7f1d74162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.521+0000 7f1d7c1c8640 1 --2- 172.21.15.172:0/3695922960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7415e430 0x7f1d74162000 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009d90 tx=0x7f1d7002bd80 comp rx=0 tx=0).stop 2024-04-14T08:42:56.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.521+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/3695922960 shutdown_connections 2024-04-14T08:42:56.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.521+0000 7f1d7c1c8640 1 --2- 172.21.15.172:0/3695922960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7415e430 0x7f1d74162000 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.521+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/3695922960 >> 172.21.15.172:0/3695922960 conn(0x7f1d7409d560 msgr2=0x7f1d740d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:56.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.521+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/3695922960 shutdown_connections 2024-04-14T08:42:56.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.521+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/3695922960 wait complete. 2024-04-14T08:42:56.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.522+0000 7f1d7c1c8640 1 Processor -- start 2024-04-14T08:42:56.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.522+0000 7f1d7c1c8640 1 -- start start 2024-04-14T08:42:56.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.523+0000 7f1d7c1c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d740d2e00 0x7f1d741586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.523+0000 7f1d7c1c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7415e430 0x7f1d7415e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.523+0000 7f1d7c1c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d74162540 0x7f1d741601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.523+0000 7f1d7c1c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1d740d4e20 con 0x7f1d7415e430 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.523+0000 7f1d7c1c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7415e810 con 0x7f1d74162540 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.523+0000 7f1d7c1c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1d740b4850 con 0x7f1d740d2e00 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.523+0000 7f1d79763640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7415e430 0x7f1d7415e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.523+0000 7f1d79763640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7415e430 0x7f1d7415e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45488/0 (socket says 172.21.15.172:45488) 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.523+0000 7f1d79763640 1 -- 172.21.15.172:0/2197582506 learned_addr learned my addr 172.21.15.172:0/2197582506 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.523+0000 7f1d7a765640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d74162540 0x7f1d741601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.523+0000 7f1d79f64640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d740d2e00 0x7f1d741586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.524+0000 7f1d79763640 1 -- 172.21.15.172:0/2197582506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d740d2e00 msgr2=0x7f1d741586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.524+0000 7f1d79763640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d740d2e00 0x7f1d741586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.524+0000 7f1d79763640 1 -- 172.21.15.172:0/2197582506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d74162540 msgr2=0x7f1d741601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.524+0000 7f1d79763640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d74162540 0x7f1d741601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.524+0000 7f1d79763640 1 -- 172.21.15.172:0/2197582506 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d741606e0 con 0x7f1d7415e430 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.524+0000 7f1d79f64640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d740d2e00 0x7f1d741586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:56.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.524+0000 7f1d79763640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7415e430 0x7f1d7415e990 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400c5f0 tx=0x7f1d6400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:56.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.524+0000 7f1d6a7fc640 1 -- 172.21.15.172:0/2197582506 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64018070 con 0x7f1d7415e430 2024-04-14T08:42:56.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.524+0000 7f1d6a7fc640 1 -- 172.21.15.172:0/2197582506 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d640074e0 con 0x7f1d7415e430 2024-04-14T08:42:56.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.524+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/2197582506 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7415f100 con 0x7f1d7415e430 2024-04-14T08:42:56.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.524+0000 7f1d6a7fc640 1 -- 172.21.15.172:0/2197582506 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64004a00 con 0x7f1d7415e430 2024-04-14T08:42:56.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.524+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/2197582506 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d7415f6c0 con 0x7f1d7415e430 2024-04-14T08:42:56.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.525+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/2197582506 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d740b5010 con 0x7f1d7415e430 2024-04-14T08:42:56.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.525+0000 7f1d6a7fc640 1 -- 172.21.15.172:0/2197582506 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d6401b070 con 0x7f1d7415e430 2024-04-14T08:42:56.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.526+0000 7f1d6a7fc640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1d5806c8f0 0x7f1d5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.526+0000 7f1d6a7fc640 1 -- 172.21.15.172:0/2197582506 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7f1d6408b090 con 0x7f1d7415e430 2024-04-14T08:42:56.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.526+0000 7f1d79f64640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1d5806c8f0 0x7f1d5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.526+0000 7f1d79f64640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1d5806c8f0 0x7f1d5806edb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1d70036000 tx=0x7f1d70030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:56.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.530+0000 7f1d6a7fc640 1 -- 172.21.15.172:0/2197582506 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1d64058ce0 con 0x7f1d7415e430 2024-04-14T08:42:56.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.707+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/2197582506 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1d740b3fb0 con 0x7f1d7415e430 2024-04-14T08:42:56.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.707+0000 7f1d6a7fc640 1 -- 172.21.15.172:0/2197582506 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) ==== 104+0+11 (secure 0 0 0) 0x7f1d6405cfe0 con 0x7f1d7415e430 2024-04-14T08:42:56.708 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:42:56.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.709+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/2197582506 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1d5806c8f0 msgr2=0x7f1d5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.709+0000 7f1d7c1c8640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1d5806c8f0 0x7f1d5806edb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1d70036000 tx=0x7f1d70030000 comp rx=0 tx=0).stop 2024-04-14T08:42:56.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.709+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/2197582506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7415e430 msgr2=0x7f1d7415e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.709+0000 7f1d7c1c8640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7415e430 0x7f1d7415e990 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400c5f0 tx=0x7f1d6400cab0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.709+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/2197582506 shutdown_connections 2024-04-14T08:42:56.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.709+0000 7f1d7c1c8640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1d5806c8f0 0x7f1d5806edb0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.709+0000 7f1d7c1c8640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d74162540 0x7f1d741601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.709+0000 7f1d7c1c8640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7415e430 0x7f1d7415e990 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.709+0000 7f1d7c1c8640 1 --2- 172.21.15.172:0/2197582506 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d740d2e00 0x7f1d741586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.709+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/2197582506 >> 172.21.15.172:0/2197582506 conn(0x7f1d7409d560 msgr2=0x7f1d740d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:56.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.709+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/2197582506 shutdown_connections 2024-04-14T08:42:56.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.709+0000 7f1d7c1c8640 1 -- 172.21.15.172:0/2197582506 wait complete. 2024-04-14T08:42:56.721 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:42:56.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.824+0000 7fc654ab7640 1 Processor -- start 2024-04-14T08:42:56.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.824+0000 7fc654ab7640 1 -- start start 2024-04-14T08:42:56.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.825+0000 7fc654ab7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc65015e430 0x7fc650162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.825+0000 7fc654ab7640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6500d2e00 0x7fc6501629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.825+0000 7fc654ab7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6500cefb0 con 0x7fc650162540 2024-04-14T08:42:56.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.825+0000 7fc654ab7640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc6500cf290 con 0x7fc6500d2e00 2024-04-14T08:42:56.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.825+0000 7fc654ab7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc6500cf570 con 0x7fc65015e430 2024-04-14T08:42:56.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.826+0000 7fc64ed9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6500d2e00 0x7fc6501629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.826+0000 7fc64ed9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6500d2e00 0x7fc6501629a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45504/0 (socket says 172.21.15.172:45504) 2024-04-14T08:42:56.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.826+0000 7fc64ed9d640 1 -- 172.21.15.172:0/764569033 learned_addr learned my addr 172.21.15.172:0/764569033 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:56.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.826+0000 7fc64e59c640 1 --2- 172.21.15.172:0/764569033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc65015e430 0x7fc650162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.826+0000 7fc64ed9d640 1 -- 172.21.15.172:0/764569033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc65015e430 msgr2=0x7fc650162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:56.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.826+0000 7fc64ed9d640 1 --2- 172.21.15.172:0/764569033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc65015e430 0x7fc650162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.826+0000 7fc64ed9d640 1 -- 172.21.15.172:0/764569033 >> v1:172.21.15.177:6789/0 conn(0x7fc650162540 legacy=0x7fc6500b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.826+0000 7fc64ed9d640 1 -- 172.21.15.172:0/764569033 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6500cf850 con 0x7fc6500d2e00 2024-04-14T08:42:56.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.826+0000 7fc64d59a640 1 -- 172.21.15.172:0/764569033 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 503439047 0 0) 0x7fc6500cefb0 con 0x7fc650162540 2024-04-14T08:42:56.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.827+0000 7fc64ed9d640 1 --2- 172.21.15.172:0/764569033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6500d2e00 0x7fc6501629a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fc648009e50 tx=0x7fc64802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e07fb35c3f272d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.827+0000 7fc64d59a640 1 -- 172.21.15.172:0/764569033 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64802cb00 con 0x7fc6500d2e00 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.827+0000 7fc64d59a640 1 -- 172.21.15.172:0/764569033 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc64802cca0 con 0x7fc6500d2e00 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.827+0000 7fc64d59a640 1 -- 172.21.15.172:0/764569033 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648034610 con 0x7fc6500d2e00 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.827+0000 7fc654ab7640 1 -- 172.21.15.172:0/764569033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6500d2e00 msgr2=0x7fc6501629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.827+0000 7fc654ab7640 1 --2- 172.21.15.172:0/764569033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6500d2e00 0x7fc6501629a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fc648009e50 tx=0x7fc64802bf60 comp rx=0 tx=0).stop 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.827+0000 7fc654ab7640 1 -- 172.21.15.172:0/764569033 shutdown_connections 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.827+0000 7fc654ab7640 1 --2- 172.21.15.172:0/764569033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6500d2e00 0x7fc6501629a0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.827+0000 7fc654ab7640 1 --2- 172.21.15.172:0/764569033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc65015e430 0x7fc650162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.827+0000 7fc654ab7640 1 -- 172.21.15.172:0/764569033 >> 172.21.15.172:0/764569033 conn(0x7fc65009d560 msgr2=0x7fc6500d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.827+0000 7fc654ab7640 1 -- 172.21.15.172:0/764569033 shutdown_connections 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.827+0000 7fc654ab7640 1 -- 172.21.15.172:0/764569033 wait complete. 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.829+0000 7fc654ab7640 1 Processor -- start 2024-04-14T08:42:56.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.829+0000 7fc654ab7640 1 -- start start 2024-04-14T08:42:56.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.829+0000 7fc654ab7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6500d2e00 0x7fc6501597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.829+0000 7fc654ab7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc65015e430 0x7fc65015e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.829+0000 7fc654ab7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc650162540 0x7fc6500b8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.829+0000 7fc654ab7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500d4e20 con 0x7fc650162540 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.829+0000 7fc654ab7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc65015e810 con 0x7fc65015e430 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.829+0000 7fc654ab7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc6500d0050 con 0x7fc6500d2e00 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc650162540 0x7fc6500b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64e59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6500d2e00 0x7fc6501597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64dd9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc65015e430 0x7fc65015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc650162540 0x7fc6500b8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45508/0 (socket says 172.21.15.172:45508) 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64e59c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6500d2e00 0x7fc6501597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:40958/0 (socket says 172.21.15.172:40958) 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64ed9d640 1 -- 172.21.15.172:0/448075578 learned_addr learned my addr 172.21.15.172:0/448075578 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64dd9b640 1 -- 172.21.15.172:0/448075578 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6500d2e00 msgr2=0x7fc6501597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64dd9b640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6500d2e00 0x7fc6501597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64dd9b640 1 -- 172.21.15.172:0/448075578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc650162540 msgr2=0x7fc6500b8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64dd9b640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc650162540 0x7fc6500b8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64dd9b640 1 -- 172.21.15.172:0/448075578 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6500cc8b0 con 0x7fc65015e430 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64e59c640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6500d2e00 0x7fc6501597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64ed9d640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc650162540 0x7fc6500b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc64dd9b640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc65015e430 0x7fc65015e990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc64000c410 tx=0x7fc64000c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc646ffd640 1 -- 172.21.15.172:0/448075578 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640017070 con 0x7fc65015e430 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc646ffd640 1 -- 172.21.15.172:0/448075578 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6400035f0 con 0x7fc65015e430 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc654ab7640 1 -- 172.21.15.172:0/448075578 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6500ccb50 con 0x7fc65015e430 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc646ffd640 1 -- 172.21.15.172:0/448075578 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640003a20 con 0x7fc65015e430 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.830+0000 7fc654ab7640 1 -- 172.21.15.172:0/448075578 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6500cd110 con 0x7fc65015e430 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.831+0000 7fc654ab7640 1 -- 172.21.15.172:0/448075578 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc650068590 con 0x7fc65015e430 2024-04-14T08:42:56.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.832+0000 7fc646ffd640 1 -- 172.21.15.172:0/448075578 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc64001a070 con 0x7fc65015e430 2024-04-14T08:42:56.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.832+0000 7fc646ffd640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc63006c8f0 0x7fc63006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:56.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.832+0000 7fc646ffd640 1 -- 172.21.15.172:0/448075578 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7fc64008af30 con 0x7fc65015e430 2024-04-14T08:42:56.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.832+0000 7fc64e59c640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc63006c8f0 0x7fc63006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:56.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.833+0000 7fc64e59c640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc63006c8f0 0x7fc63006edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc638009e90 tx=0x7fc638009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:56.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:56.837+0000 7fc646ffd640 1 -- 172.21.15.172:0/448075578 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc640058cd0 con 0x7fc65015e430 2024-04-14T08:42:57.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.015+0000 7fc654ab7640 1 -- 172.21.15.172:0/448075578 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc6500d18a0 con 0x7fc65015e430 2024-04-14T08:42:57.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.015+0000 7fc646ffd640 1 -- 172.21.15.172:0/448075578 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) ==== 100+0+11 (secure 0 0 0) 0x7fc64005cfd0 con 0x7fc65015e430 2024-04-14T08:42:57.015 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:42:57.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.017+0000 7fc654ab7640 1 -- 172.21.15.172:0/448075578 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc63006c8f0 msgr2=0x7fc63006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.017+0000 7fc654ab7640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc63006c8f0 0x7fc63006edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc638009e90 tx=0x7fc638009190 comp rx=0 tx=0).stop 2024-04-14T08:42:57.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.017+0000 7fc654ab7640 1 -- 172.21.15.172:0/448075578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc65015e430 msgr2=0x7fc65015e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.017+0000 7fc654ab7640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc65015e430 0x7fc65015e990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc64000c410 tx=0x7fc64000c810 comp rx=0 tx=0).stop 2024-04-14T08:42:57.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.017+0000 7fc654ab7640 1 -- 172.21.15.172:0/448075578 shutdown_connections 2024-04-14T08:42:57.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.017+0000 7fc654ab7640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc63006c8f0 0x7fc63006edb0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.017+0000 7fc654ab7640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc650162540 0x7fc6500b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.017+0000 7fc654ab7640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc65015e430 0x7fc65015e990 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.017+0000 7fc654ab7640 1 --2- 172.21.15.172:0/448075578 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6500d2e00 0x7fc6501597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.017+0000 7fc654ab7640 1 -- 172.21.15.172:0/448075578 >> 172.21.15.172:0/448075578 conn(0x7fc65009d560 msgr2=0x7fc6500d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:57.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.017+0000 7fc654ab7640 1 -- 172.21.15.172:0/448075578 shutdown_connections 2024-04-14T08:42:57.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.017+0000 7fc654ab7640 1 -- 172.21.15.172:0/448075578 wait complete. 2024-04-14T08:42:57.029 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:42:57.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.131+0000 7fcae94de640 1 Processor -- start 2024-04-14T08:42:57.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.131+0000 7fcae94de640 1 -- start start 2024-04-14T08:42:57.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.132+0000 7fcae94de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162710 0x7fcae40d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:57.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.132+0000 7fcae94de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcae4162b70 0x7fcae40d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:57.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.132+0000 7fcae94de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae40cefb0 con 0x7fcae415e430 2024-04-14T08:42:57.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.132+0000 7fcae94de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fcae40cf290 con 0x7fcae4162b70 2024-04-14T08:42:57.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.132+0000 7fcae94de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcae40cf570 con 0x7fcae4162710 2024-04-14T08:42:57.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.132+0000 7fcae2ffd640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fcae415e430 0x7fcae4162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:37966/0 (socket says 172.21.15.172:37966) 2024-04-14T08:42:57.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.132+0000 7fcae27fc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162710 0x7fcae40d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:57.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.132+0000 7fcae2ffd640 1 -- 172.21.15.172:0/3413348147 learned_addr learned my addr 172.21.15.172:0/3413348147 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:57.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.132+0000 7fcae37fe640 1 --2- 172.21.15.172:0/3413348147 >> v2:172.21.15.177:3300/0 conn(0x7fcae4162b70 0x7fcae40d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:57.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.133+0000 7fcae1ffb640 1 -- 172.21.15.172:0/3413348147 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 678933736 0 0) 0x7fcae40cefb0 con 0x7fcae415e430 2024-04-14T08:42:57.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.133+0000 7fcae1ffb640 1 -- 172.21.15.172:0/3413348147 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcad0003520 con 0x7fcae415e430 2024-04-14T08:42:57.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.133+0000 7fcae37fe640 1 -- 172.21.15.172:0/3413348147 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162710 msgr2=0x7fcae40d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:57.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.133+0000 7fcae37fe640 1 --2- 172.21.15.172:0/3413348147 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162710 0x7fcae40d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.133+0000 7fcae37fe640 1 -- 172.21.15.172:0/3413348147 >> v1:172.21.15.177:6789/0 conn(0x7fcae415e430 legacy=0x7fcae4162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.133+0000 7fcae37fe640 1 -- 172.21.15.172:0/3413348147 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae40cf850 con 0x7fcae4162b70 2024-04-14T08:42:57.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.133+0000 7fcae27fc640 1 --2- 172.21.15.172:0/3413348147 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162710 0x7fcae40d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:57.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.134+0000 7fcae37fe640 1 --2- 172.21.15.172:0/3413348147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae4162b70 0x7fcae40d2330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fcad40097a0 tx=0x7fcad402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d62956ff0f9ef5b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:57.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.134+0000 7fcae1ffb640 1 -- 172.21.15.172:0/3413348147 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4035020 con 0x7fcae4162b70 2024-04-14T08:42:57.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.134+0000 7fcae1ffb640 1 -- 172.21.15.172:0/3413348147 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad4031830 con 0x7fcae4162b70 2024-04-14T08:42:57.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.134+0000 7fcae1ffb640 1 -- 172.21.15.172:0/3413348147 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4031c00 con 0x7fcae4162b70 2024-04-14T08:42:57.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.134+0000 7fcae94de640 1 -- 172.21.15.172:0/3413348147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae4162b70 msgr2=0x7fcae40d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.134+0000 7fcae94de640 1 --2- 172.21.15.172:0/3413348147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae4162b70 0x7fcae40d2330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fcad40097a0 tx=0x7fcad402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.134+0000 7fcae94de640 1 -- 172.21.15.172:0/3413348147 shutdown_connections 2024-04-14T08:42:57.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.134+0000 7fcae94de640 1 --2- 172.21.15.172:0/3413348147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae4162b70 0x7fcae40d2330 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.134+0000 7fcae94de640 1 --2- 172.21.15.172:0/3413348147 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162710 0x7fcae40d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.134+0000 7fcae94de640 1 -- 172.21.15.172:0/3413348147 >> 172.21.15.172:0/3413348147 conn(0x7fcae409d560 msgr2=0x7fcae40d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:57.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.134+0000 7fcae94de640 1 -- 172.21.15.172:0/3413348147 shutdown_connections 2024-04-14T08:42:57.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.134+0000 7fcae94de640 1 -- 172.21.15.172:0/3413348147 wait complete. 2024-04-14T08:42:57.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.136+0000 7fcae94de640 1 Processor -- start 2024-04-14T08:42:57.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.136+0000 7fcae94de640 1 -- start start 2024-04-14T08:42:57.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.136+0000 7fcae94de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae415e430 0x7fcae41610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:57.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.136+0000 7fcae94de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae4162710 0x7fcae40b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:57.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.136+0000 7fcae94de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162b70 0x7fcae40b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:57.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.136+0000 7fcae94de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcae40d4e20 con 0x7fcae415e430 2024-04-14T08:42:57.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.136+0000 7fcae94de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcae415e810 con 0x7fcae4162710 2024-04-14T08:42:57.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.136+0000 7fcae94de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcae40d0050 con 0x7fcae4162b70 2024-04-14T08:42:57.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae415e430 0x7fcae41610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:57.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae37fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162b70 0x7fcae40b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:57.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae415e430 0x7fcae41610d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45522/0 (socket says 172.21.15.172:45522) 2024-04-14T08:42:57.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae37fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162b70 0x7fcae40b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:40978/0 (socket says 172.21.15.172:40978) 2024-04-14T08:42:57.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae2ffd640 1 -- 172.21.15.172:0/2316836550 learned_addr learned my addr 172.21.15.172:0/2316836550 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:57.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae27fc640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae4162710 0x7fcae40b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:57.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae2ffd640 1 -- 172.21.15.172:0/2316836550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162b70 msgr2=0x7fcae40b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae2ffd640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162b70 0x7fcae40b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae2ffd640 1 -- 172.21.15.172:0/2316836550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae4162710 msgr2=0x7fcae40b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae2ffd640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae4162710 0x7fcae40b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae2ffd640 1 -- 172.21.15.172:0/2316836550 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae4160050 con 0x7fcae415e430 2024-04-14T08:42:57.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae37fe640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162b70 0x7fcae40b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:57.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae2ffd640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae415e430 0x7fcae41610d0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00cb40 tx=0x7fcadc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:57.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcadb7fe640 1 -- 172.21.15.172:0/2316836550 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc017070 con 0x7fcae415e430 2024-04-14T08:42:57.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcadb7fe640 1 -- 172.21.15.172:0/2316836550 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcadc00aca0 con 0x7fcae415e430 2024-04-14T08:42:57.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae94de640 1 -- 172.21.15.172:0/2316836550 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae4160330 con 0x7fcae415e430 2024-04-14T08:42:57.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcadb7fe640 1 -- 172.21.15.172:0/2316836550 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc012620 con 0x7fcae415e430 2024-04-14T08:42:57.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.137+0000 7fcae94de640 1 -- 172.21.15.172:0/2316836550 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae41608f0 con 0x7fcae415e430 2024-04-14T08:42:57.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.138+0000 7fcae94de640 1 -- 172.21.15.172:0/2316836550 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae4068590 con 0x7fcae415e430 2024-04-14T08:42:57.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.139+0000 7fcadb7fe640 1 -- 172.21.15.172:0/2316836550 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcadc01a070 con 0x7fcae415e430 2024-04-14T08:42:57.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.139+0000 7fcadb7fe640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcac406c8f0 0x7fcac406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:57.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.139+0000 7fcadb7fe640 1 -- 172.21.15.172:0/2316836550 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7fcadc08ba80 con 0x7fcae415e430 2024-04-14T08:42:57.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.139+0000 7fcae27fc640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcac406c8f0 0x7fcac406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:57.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.140+0000 7fcae27fc640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcac406c8f0 0x7fcac406edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009560 tx=0x7fcacc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:57.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.144+0000 7fcadb7fe640 1 -- 172.21.15.172:0/2316836550 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcadc0596e0 con 0x7fcae415e430 2024-04-14T08:42:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.330+0000 7fcae94de640 1 -- 172.21.15.172:0/2316836550 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcae40d18a0 con 0x7fcae415e430 2024-04-14T08:42:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.331+0000 7fcadb7fe640 1 -- 172.21.15.172:0/2316836550 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v116) ==== 80+0+0 (secure 0 0 0) 0x7fcadc05d9e0 con 0x7fcae415e430 2024-04-14T08:42:57.331 INFO:teuthology.orchestra.run.smithi172.stderr:listed 0 entries 2024-04-14T08:42:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.333+0000 7fcae94de640 1 -- 172.21.15.172:0/2316836550 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcac406c8f0 msgr2=0x7fcac406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.333+0000 7fcae94de640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcac406c8f0 0x7fcac406edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009560 tx=0x7fcacc009190 comp rx=0 tx=0).stop 2024-04-14T08:42:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.333+0000 7fcae94de640 1 -- 172.21.15.172:0/2316836550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae415e430 msgr2=0x7fcae41610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.333+0000 7fcae94de640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae415e430 0x7fcae41610d0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00cb40 tx=0x7fcadc00a060 comp rx=0 tx=0).stop 2024-04-14T08:42:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.333+0000 7fcae94de640 1 -- 172.21.15.172:0/2316836550 shutdown_connections 2024-04-14T08:42:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.333+0000 7fcae94de640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcac406c8f0 0x7fcac406edb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.333+0000 7fcae94de640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae4162b70 0x7fcae40b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.333+0000 7fcae94de640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae4162710 0x7fcae40b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.333+0000 7fcae94de640 1 --2- 172.21.15.172:0/2316836550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcae415e430 0x7fcae41610d0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.333+0000 7fcae94de640 1 -- 172.21.15.172:0/2316836550 >> 172.21.15.172:0/2316836550 conn(0x7fcae409d560 msgr2=0x7fcae415c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:57.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.333+0000 7fcae94de640 1 -- 172.21.15.172:0/2316836550 shutdown_connections 2024-04-14T08:42:57.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.333+0000 7fcae94de640 1 -- 172.21.15.172:0/2316836550 wait complete. 2024-04-14T08:42:57.344 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:42:57.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:57.344 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:42:57.414 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:42:57.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:57.414 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:42:57.440 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:42:57.440 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:42:57.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:42:57.497 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:42:57.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:42:57.498 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:42:57.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:42:57.552 INFO:teuthology.orchestra.run.smithi172.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-14T08:42:57.552 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:42:57.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.693+0000 7efecf0c8640 1 Processor -- start 2024-04-14T08:42:57.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.693+0000 7efecf0c8640 1 -- start start 2024-04-14T08:42:57.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecf0c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efec813df30 0x7efec80b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:57.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecf0c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec815c380 0x7efec80b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:57.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecf0c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efec815b620 con 0x7efec8140770 2024-04-14T08:42:57.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecf0c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7efec80cf9b0 con 0x7efec813df30 2024-04-14T08:42:57.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecf0c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efec81414a0 con 0x7efec815c380 2024-04-14T08:42:57.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecce64640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efec813df30 0x7efec80b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:57.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecce64640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efec813df30 0x7efec80b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33404/0 (socket says 172.21.15.172:33404) 2024-04-14T08:42:57.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecd665640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7efec8140770 0x7efec8140c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:37976/0 (socket says 172.21.15.172:37976) 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efec7fff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec815c380 0x7efec80b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecce64640 1 -- 172.21.15.172:0/3495582383 learned_addr learned my addr 172.21.15.172:0/3495582383 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecce64640 1 -- 172.21.15.172:0/3495582383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec815c380 msgr2=0x7efec80b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecce64640 1 --2- 172.21.15.172:0/3495582383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec815c380 0x7efec80b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecce64640 1 -- 172.21.15.172:0/3495582383 >> v1:172.21.15.177:6789/0 conn(0x7efec8140770 legacy=0x7efec8140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efecce64640 1 -- 172.21.15.172:0/3495582383 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec815b8a0 con 0x7efec813df30 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efec77fe640 1 -- 172.21.15.172:0/3495582383 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 201417637 0 0) 0x7efec0001490 con 0x7efec8140770 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.694+0000 7efec7fff640 1 --2- 172.21.15.172:0/3495582383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec815c380 0x7efec80b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.695+0000 7efecce64640 1 --2- 172.21.15.172:0/3495582383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efec813df30 0x7efec80b1cd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efebc0098a0 tx=0x7efebc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e00c10daf0268e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.695+0000 7efec77fe640 1 -- 172.21.15.172:0/3495582383 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc034070 con 0x7efec813df30 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.695+0000 7efec77fe640 1 -- 172.21.15.172:0/3495582383 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efebc030840 con 0x7efec813df30 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.695+0000 7efec77fe640 1 -- 172.21.15.172:0/3495582383 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc030c10 con 0x7efec813df30 2024-04-14T08:42:57.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.696+0000 7efecf0c8640 1 -- 172.21.15.172:0/3495582383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efec813df30 msgr2=0x7efec80b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.696+0000 7efecf0c8640 1 --2- 172.21.15.172:0/3495582383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efec813df30 0x7efec80b1cd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efebc0098a0 tx=0x7efebc02b980 comp rx=0 tx=0).stop 2024-04-14T08:42:57.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.696+0000 7efecf0c8640 1 -- 172.21.15.172:0/3495582383 shutdown_connections 2024-04-14T08:42:57.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.696+0000 7efecf0c8640 1 --2- 172.21.15.172:0/3495582383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec815c380 0x7efec80b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.696+0000 7efecf0c8640 1 --2- 172.21.15.172:0/3495582383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efec813df30 0x7efec80b1cd0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.696+0000 7efecf0c8640 1 -- 172.21.15.172:0/3495582383 >> 172.21.15.172:0/3495582383 conn(0x7efec806b520 msgr2=0x7efec806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:57.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.696+0000 7efecf0c8640 1 -- 172.21.15.172:0/3495582383 shutdown_connections 2024-04-14T08:42:57.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.696+0000 7efecf0c8640 1 -- 172.21.15.172:0/3495582383 wait complete. 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.697+0000 7efecf0c8640 1 Processor -- start 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.697+0000 7efecf0c8640 1 -- start start 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efecf0c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec813df30 0x7efec8131760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efecf0c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8140770 0x7efec80af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efecf0c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efec815c380 0x7efec80b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efecf0c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efec813e310 con 0x7efec8140770 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efecf0c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efec815be60 con 0x7efec815c380 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efecf0c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efec80ce9a0 con 0x7efec813df30 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efec7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8140770 0x7efec80af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efec7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8140770 0x7efec80af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45526/0 (socket says 172.21.15.172:45526) 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efec7fff640 1 -- 172.21.15.172:0/3075018791 learned_addr learned my addr 172.21.15.172:0/3075018791 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efecd665640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efec815c380 0x7efec80b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:57.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efecce64640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec813df30 0x7efec8131760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:57.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efec7fff640 1 -- 172.21.15.172:0/3075018791 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec813df30 msgr2=0x7efec8131760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efec7fff640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec813df30 0x7efec8131760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efec7fff640 1 -- 172.21.15.172:0/3075018791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efec815c380 msgr2=0x7efec80b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efec7fff640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efec815c380 0x7efec80b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efec7fff640 1 -- 172.21.15.172:0/3075018791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec80b0040 con 0x7efec8140770 2024-04-14T08:42:57.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.698+0000 7efec7fff640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8140770 0x7efec80af740 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7efeb400e990 tx=0x7efeb400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:57.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.699+0000 7efec57fa640 1 -- 172.21.15.172:0/3075018791 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4017070 con 0x7efec8140770 2024-04-14T08:42:57.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.699+0000 7efec57fa640 1 -- 172.21.15.172:0/3075018791 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb400c9e0 con 0x7efec8140770 2024-04-14T08:42:57.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.699+0000 7efecf0c8640 1 -- 172.21.15.172:0/3075018791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec80b0320 con 0x7efec8140770 2024-04-14T08:42:57.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.699+0000 7efec57fa640 1 -- 172.21.15.172:0/3075018791 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb400cdb0 con 0x7efec8140770 2024-04-14T08:42:57.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.699+0000 7efecf0c8640 1 -- 172.21.15.172:0/3075018791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec80b08e0 con 0x7efec8140770 2024-04-14T08:42:57.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.699+0000 7efecf0c8640 1 -- 172.21.15.172:0/3075018791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec80b0d20 con 0x7efec8140770 2024-04-14T08:42:57.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.700+0000 7efec57fa640 1 -- 172.21.15.172:0/3075018791 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efeb401a070 con 0x7efec8140770 2024-04-14T08:42:57.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.700+0000 7efec57fa640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efeac06c8f0 0x7efeac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:57.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.700+0000 7efec57fa640 1 -- 172.21.15.172:0/3075018791 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6101+0+0 (secure 0 0 0) 0x7efeb408bdf0 con 0x7efec8140770 2024-04-14T08:42:57.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.700+0000 7efecce64640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efeac06c8f0 0x7efeac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:57.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.701+0000 7efecce64640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efeac06c8f0 0x7efeac06edb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efebc036000 tx=0x7efebc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:57.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.705+0000 7efec57fa640 1 -- 172.21.15.172:0/3075018791 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efeb4059ad0 con 0x7efec8140770 2024-04-14T08:42:57.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.914+0000 7efecf0c8640 1 -- 172.21.15.172:0/3075018791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efec8068590 con 0x7efec8140770 2024-04-14T08:42:57.915 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:42:57.915+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:42:57.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.916+0000 7efec57fa640 1 -- 172.21.15.172:0/3075018791 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) ==== 161+0+0 (secure 0 0 0) 0x7efeb405ddd0 con 0x7efec8140770 2024-04-14T08:42:57.916 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:42:57.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.919+0000 7efecf0c8640 1 -- 172.21.15.172:0/3075018791 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efeac06c8f0 msgr2=0x7efeac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.919+0000 7efecf0c8640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efeac06c8f0 0x7efeac06edb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efebc036000 tx=0x7efebc008030 comp rx=0 tx=0).stop 2024-04-14T08:42:57.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.919+0000 7efecf0c8640 1 -- 172.21.15.172:0/3075018791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8140770 msgr2=0x7efec80af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:57.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.919+0000 7efecf0c8640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8140770 0x7efec80af740 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7efeb400e990 tx=0x7efeb400ee50 comp rx=0 tx=0).stop 2024-04-14T08:42:57.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.919+0000 7efecf0c8640 1 -- 172.21.15.172:0/3075018791 shutdown_connections 2024-04-14T08:42:57.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.919+0000 7efecf0c8640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efeac06c8f0 0x7efeac06edb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.919+0000 7efecf0c8640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efec815c380 0x7efec80b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.919+0000 7efecf0c8640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8140770 0x7efec80af740 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.919+0000 7efecf0c8640 1 --2- 172.21.15.172:0/3075018791 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efec813df30 0x7efec8131760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:57.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.919+0000 7efecf0c8640 1 -- 172.21.15.172:0/3075018791 >> 172.21.15.172:0/3075018791 conn(0x7efec806b520 msgr2=0x7efec8072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:57.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.919+0000 7efecf0c8640 1 -- 172.21.15.172:0/3075018791 shutdown_connections 2024-04-14T08:42:57.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:57.919+0000 7efecf0c8640 1 -- 172.21.15.172:0/3075018791 wait complete. 2024-04-14T08:42:57.923 INFO:tasks.ceph.mds.d.smithi177.stderr: -8506> 2024-04-14T08:42:46.163+0000 7f665cdf1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:42:57.931 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-14T08:42:57.933 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-14T08:42:57.957 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2024-04-14T08:42:57.982 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-04-14T08:42:57.982 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2024-04-14T08:42:57.982 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-14T08:42:57.982 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2024-04-14T08:42:57.982 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.982 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2024-04-14T08:42:57.982 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.982 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2024-04-14T08:42:57.982 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.982 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2024-04-14T08:42:57.982 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2024-04-14T08:42:57.983 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2024-04-14T08:42:57.984 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2024-04-14T08:42:57.984 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2024-04-14T08:42:57.984 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2024-04-14T08:42:57.984 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2024-04-14T08:42:57.984 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2024-04-14T08:42:57.984 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-14T08:42:57.984 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-14T08:42:57.984 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2024-04-14T08:42:57.984 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.984 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-14T08:42:57.984 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.984 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.172,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.985 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2024-04-14T08:42:57.986 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2024-04-14T08:42:57.987 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 20 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 20 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 10 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.988 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2024-04-14T08:42:57.989 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-14T08:42:57.990 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2024-04-14T08:42:57.990 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2024-04-14T08:42:57.990 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:42:57.990 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2024-04-14T08:42:57.990 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2024-04-14T08:42:57.990 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:42:57.990 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.990 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-14T08:42:57.990 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2024-04-14T08:42:57.990 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2024-04-14T08:42:57.990 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.990 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2024-04-14T08:42:57.991 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-14T08:42:57.991 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-14T08:42:57.991 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.991 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2024-04-14T08:42:57.992 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2024-04-14T08:42:57.992 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.992 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2024-04-14T08:42:57.992 INFO:teuthology.orchestra.run.smithi172.stdout: mon down mkfs grace = 300 2024-04-14T08:42:57.992 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2024-04-14T08:42:57.992 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:57.992 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2024-04-14T08:42:57.993 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = True 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = False 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-14T08:42:57.994 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-14T08:42:57.995 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2024-04-14T08:42:57.995 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2024-04-14T08:42:57.995 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-14T08:42:57.995 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2024-04-14T08:42:57.995 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.996 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2024-04-14T08:42:57.996 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.996 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2024-04-14T08:42:57.996 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.996 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2024-04-14T08:42:57.996 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2024-04-14T08:42:57.996 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.996 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2024-04-14T08:42:57.996 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2024-04-14T08:42:57.996 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.996 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-14T08:42:57.996 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2024-04-14T08:42:57.997 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2024-04-14T08:42:57.998 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.172,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on skipped message = False 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:57.999 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug trim objects = true 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.000 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 20 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 20 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 10 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2024-04-14T08:42:58.001 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.002 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2024-04-14T08:42:58.003 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:42:58.003 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-04-14T08:42:58.003 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2024-04-14T08:42:58.003 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-04-14T08:42:58.003 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2024-04-14T08:42:58.003 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2024-04-14T08:42:58.003 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:42:58.003 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2024-04-14T08:42:58.003 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2024-04-14T08:42:58.003 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:42:58.003 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: mon down mkfs grace = 300 2024-04-14T08:42:58.004 INFO:teuthology.orchestra.run.smithi177.stdout: mon op complaint time = 120 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 600 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = "" 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2024-04-14T08:42:58.005 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout:[mds] 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds = 20 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds balancer = 20 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug frag = True 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug scatterstat = True 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout: mds op complaint time = 180 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout: mds verify scatter = False 2024-04-14T08:42:58.006 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2024-04-14T08:42:58.007 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2024-04-14T08:42:58.007 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2024-04-14T08:42:58.009 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-14T08:42:58.052 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-14T08:42:58.102 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-04-14T08:42:58.102 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2024-04-14T08:42:58.102 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-14T08:42:58.102 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2024-04-14T08:42:58.102 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.102 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2024-04-14T08:42:58.102 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.102 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2024-04-14T08:42:58.102 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.102 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2024-04-14T08:42:58.103 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2024-04-14T08:42:58.104 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.172,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2024-04-14T08:42:58.105 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2024-04-14T08:42:58.106 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2024-04-14T08:42:58.106 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.106 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2024-04-14T08:42:58.106 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2024-04-14T08:42:58.106 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.106 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2024-04-14T08:42:58.106 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2024-04-14T08:42:58.106 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2024-04-14T08:42:58.106 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2024-04-14T08:42:58.107 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2024-04-14T08:42:58.108 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2024-04-14T08:42:58.108 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2024-04-14T08:42:58.108 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 20 2024-04-14T08:42:58.108 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 20 2024-04-14T08:42:58.108 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:42:58.108 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2024-04-14T08:42:58.108 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 10 2024-04-14T08:42:58.108 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2024-04-14T08:42:58.108 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2024-04-14T08:42:58.108 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2024-04-14T08:42:58.109 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2024-04-14T08:42:58.109 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2024-04-14T08:42:58.109 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-14T08:42:58.109 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.109 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2024-04-14T08:42:58.110 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:42:58.110 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2024-04-14T08:42:58.110 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-14T08:42:58.110 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-14T08:42:58.110 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:42:58.110 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:42:58.110 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2024-04-14T08:42:58.110 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-14T08:42:58.110 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.110 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2024-04-14T08:42:58.110 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:42:58.110 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-14T08:42:58.111 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2024-04-14T08:42:58.111 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-14T08:42:58.111 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2024-04-14T08:42:58.111 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2024-04-14T08:42:58.111 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:42:58.111 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2024-04-14T08:42:58.111 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2024-04-14T08:42:58.111 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:42:58.111 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.111 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-14T08:42:58.111 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2024-04-14T08:42:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2024-04-14T08:42:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2024-04-14T08:42:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-14T08:42:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-14T08:42:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2024-04-14T08:42:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2024-04-14T08:42:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.112 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: mon down mkfs grace = 300 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:42:58.113 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2024-04-14T08:42:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-14T08:42:58.114 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-14T08:42:58.114 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2024-04-14T08:42:58.114 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2024-04-14T08:42:58.114 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2024-04-14T08:42:58.114 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-04-14T08:42:58.114 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-04-14T08:42:58.114 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-04-14T08:42:58.114 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-04-14T08:42:58.115 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2024-04-14T08:42:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2024-04-14T08:42:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2024-04-14T08:42:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:42:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2024-04-14T08:42:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = False 2024-04-14T08:42:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2024-04-14T08:42:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = False 2024-04-14T08:42:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-14T08:42:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-14T08:42:58.115 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-14T08:42:58.116 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2024-04-14T08:42:58.116 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2024-04-14T08:42:58.116 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.117 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2024-04-14T08:42:58.118 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2024-04-14T08:42:58.119 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-14T08:42:58.119 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-14T08:42:58.119 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2024-04-14T08:42:58.119 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.119 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-14T08:42:58.119 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.119 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2024-04-14T08:42:58.119 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.119 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2024-04-14T08:42:58.119 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.119 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2024-04-14T08:42:58.119 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2024-04-14T08:42:58.120 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2024-04-14T08:42:58.120 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.172,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2024-04-14T08:42:58.120 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on skipped message = False 2024-04-14T08:42:58.120 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:42:58.120 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.120 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2024-04-14T08:42:58.120 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2024-04-14T08:42:58.120 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.120 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2024-04-14T08:42:58.121 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2024-04-14T08:42:58.121 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2024-04-14T08:42:58.121 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.121 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2024-04-14T08:42:58.121 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2024-04-14T08:42:58.121 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.121 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2024-04-14T08:42:58.121 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug trim objects = true 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.122 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 20 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 20 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 10 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2024-04-14T08:42:58.123 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:42:58.124 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2024-04-14T08:42:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: mon down mkfs grace = 300 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: mon op complaint time = 120 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2024-04-14T08:42:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 600 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = "" 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2024-04-14T08:42:58.127 INFO:teuthology.orchestra.run.smithi177.stdout:[mds] 2024-04-14T08:42:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds = 20 2024-04-14T08:42:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds balancer = 20 2024-04-14T08:42:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:42:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug frag = True 2024-04-14T08:42:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug scatterstat = False 2024-04-14T08:42:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: mds op complaint time = 180 2024-04-14T08:42:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: mds verify scatter = False 2024-04-14T08:42:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2024-04-14T08:42:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2024-04-14T08:42:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2024-04-14T08:42:58.130 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-14T08:42:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.212+0000 7fb46ca4af40 1 Processor -- start 2024-04-14T08:42:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.212+0000 7fb46ca4af40 1 -- start start 2024-04-14T08:42:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46ca4af40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586d8f6ddf0 0x5586d902d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46ca4af40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586d8f6f540 0x5586d8fb11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46ca4af40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f9f500 0x5586d90d9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46ca4af40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5586d8f6e840 con 0x5586d8f9f500 2024-04-14T08:42:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46ca4af40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5586d8f163b0 con 0x5586d8f6ddf0 2024-04-14T08:42:58.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46ca4af40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5586d8f6eae0 con 0x5586d8f6f540 2024-04-14T08:42:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46a3a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f9f500 0x5586d90d9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46a3a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f9f500 0x5586d90d9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45546/0 (socket says 172.21.15.172:45546) 2024-04-14T08:42:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46a3a4640 1 -- 172.21.15.172:0/3380320230 learned_addr learned my addr 172.21.15.172:0/3380320230 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb469ba3640 1 --2- 172.21.15.172:0/3380320230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586d8f6ddf0 0x5586d902d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb4693a2640 1 --2- 172.21.15.172:0/3380320230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586d8f6f540 0x5586d8fb11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46a3a4640 1 -- 172.21.15.172:0/3380320230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586d8f6f540 msgr2=0x5586d8fb11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46a3a4640 1 --2- 172.21.15.172:0/3380320230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586d8f6f540 0x5586d8fb11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46a3a4640 1 -- 172.21.15.172:0/3380320230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586d8f6ddf0 msgr2=0x5586d902d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46a3a4640 1 --2- 172.21.15.172:0/3380320230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586d8f6ddf0 0x5586d902d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46a3a4640 1 -- 172.21.15.172:0/3380320230 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d902da00 con 0x5586d8f9f500 2024-04-14T08:42:58.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb46a3a4640 1 --2- 172.21.15.172:0/3380320230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f9f500 0x5586d90d9280 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fb46401b960 tx=0x7fb46404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2accd19f3011e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:58.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.213+0000 7fb45bfff640 1 -- 172.21.15.172:0/3380320230 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464063070 con 0x5586d8f9f500 2024-04-14T08:42:58.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.214+0000 7fb45bfff640 1 -- 172.21.15.172:0/3380320230 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb464052ae0 con 0x5586d8f9f500 2024-04-14T08:42:58.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.214+0000 7fb45bfff640 1 -- 172.21.15.172:0/3380320230 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46405e470 con 0x5586d8f9f500 2024-04-14T08:42:58.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.214+0000 7fb46ca4af40 1 -- 172.21.15.172:0/3380320230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f9f500 msgr2=0x5586d90d9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.214+0000 7fb46ca4af40 1 --2- 172.21.15.172:0/3380320230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f9f500 0x5586d90d9280 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fb46401b960 tx=0x7fb46404c000 comp rx=0 tx=0).stop 2024-04-14T08:42:58.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.214+0000 7fb46ca4af40 1 -- 172.21.15.172:0/3380320230 shutdown_connections 2024-04-14T08:42:58.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.214+0000 7fb46ca4af40 1 --2- 172.21.15.172:0/3380320230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f9f500 0x5586d90d9280 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.214+0000 7fb46ca4af40 1 --2- 172.21.15.172:0/3380320230 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586d8f6f540 0x5586d8fb11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.214+0000 7fb46ca4af40 1 --2- 172.21.15.172:0/3380320230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586d8f6ddf0 0x5586d902d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.214+0000 7fb46ca4af40 1 -- 172.21.15.172:0/3380320230 >> 172.21.15.172:0/3380320230 conn(0x5586d8f993d0 msgr2=0x5586d902cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:58.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.214+0000 7fb46ca4af40 1 -- 172.21.15.172:0/3380320230 shutdown_connections 2024-04-14T08:42:58.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.214+0000 7fb46ca4af40 1 -- 172.21.15.172:0/3380320230 wait complete. 2024-04-14T08:42:58.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.215+0000 7fb46ca4af40 1 Processor -- start 2024-04-14T08:42:58.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.215+0000 7fb46ca4af40 1 -- start start 2024-04-14T08:42:58.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.215+0000 7fb46ca4af40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f6ddf0 0x5586d8fafff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.215+0000 7fb46ca4af40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586d8f6f540 0x5586d8fb3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.215+0000 7fb46ca4af40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586d8f9f500 0x5586d8fb3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.215+0000 7fb46ca4af40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5586d9026fa0 con 0x5586d8f6ddf0 2024-04-14T08:42:58.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.215+0000 7fb46ca4af40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5586d9027940 con 0x5586d8f9f500 2024-04-14T08:42:58.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.215+0000 7fb46ca4af40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5586d9027280 con 0x5586d8f6f540 2024-04-14T08:42:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.215+0000 7fb469ba3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f6ddf0 0x5586d8fafff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.215+0000 7fb469ba3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f6ddf0 0x5586d8fafff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45558/0 (socket says 172.21.15.172:45558) 2024-04-14T08:42:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.215+0000 7fb469ba3640 1 -- 172.21.15.172:0/1230725376 learned_addr learned my addr 172.21.15.172:0/1230725376 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb4693a2640 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586d8f6f540 0x5586d8fb3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb46a3a4640 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586d8f9f500 0x5586d8fb3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb469ba3640 1 -- 172.21.15.172:0/1230725376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586d8f6f540 msgr2=0x5586d8fb3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb469ba3640 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5586d8f6f540 0x5586d8fb3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb469ba3640 1 -- 172.21.15.172:0/1230725376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586d8f9f500 msgr2=0x5586d8fb3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb469ba3640 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586d8f9f500 0x5586d8fb3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb469ba3640 1 -- 172.21.15.172:0/1230725376 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d8f9ec10 con 0x5586d8f6ddf0 2024-04-14T08:42:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb469ba3640 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f6ddf0 0x5586d8fafff0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009e80 tx=0x7fb45c009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb45affd640 1 -- 172.21.15.172:0/1230725376 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c01b070 con 0x5586d8f6ddf0 2024-04-14T08:42:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb45affd640 1 -- 172.21.15.172:0/1230725376 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45c016480 con 0x5586d8f6ddf0 2024-04-14T08:42:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb45affd640 1 -- 172.21.15.172:0/1230725376 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c016870 con 0x5586d8f6ddf0 2024-04-14T08:42:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb46ca4af40 1 -- 172.21.15.172:0/1230725376 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5586d8f9eeb0 con 0x5586d8f6ddf0 2024-04-14T08:42:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.216+0000 7fb46ca4af40 1 -- 172.21.15.172:0/1230725376 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d8f98a30 con 0x5586d8f6ddf0 2024-04-14T08:42:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.217+0000 7fb45affd640 1 -- 172.21.15.172:0/1230725376 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb45c01e070 con 0x5586d8f6ddf0 2024-04-14T08:42:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.218+0000 7fb45affd640 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb44c06b2d0 0x7fb44c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.218+0000 7fb45affd640 1 -- 172.21.15.172:0/1230725376 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7fb45c08a270 con 0x5586d8f6ddf0 2024-04-14T08:42:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.218+0000 7fb4693a2640 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb44c06b2d0 0x7fb44c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.218+0000 7fb46ca4af40 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5586d8fb0530 0x5586d9043d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.218+0000 7fb46ca4af40 1 -- 172.21.15.172:0/1230725376 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5586d8fb5950 con 0x5586d8fb0530 2024-04-14T08:42:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.218+0000 7fb46a3a4640 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5586d8fb0530 0x5586d9043d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.218+0000 7fb4693a2640 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb44c06b2d0 0x7fb44c06d790 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb460001ed0 tx=0x7fb460001bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:58.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.218+0000 7fb46a3a4640 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5586d8fb0530 0x5586d9043d40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:58.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.220+0000 7fb46a3a4640 1 -- 172.21.15.172:0/1230725376 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v117'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb46405e570 con 0x5586d8fb0530 2024-04-14T08:42:58.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.220+0000 7fb46ca4af40 1 -- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5586d8fb0530 msgr2=0x5586d9043d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.220+0000 7fb46ca4af40 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x5586d8fb0530 0x5586d9043d40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.220+0000 7fb46ca4af40 1 -- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb44c06b2d0 msgr2=0x7fb44c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.220+0000 7fb46ca4af40 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb44c06b2d0 0x7fb44c06d790 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb460001ed0 tx=0x7fb460001bc0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.220+0000 7fb46ca4af40 1 -- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f6ddf0 msgr2=0x5586d8fafff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.220+0000 7fb46ca4af40 1 --2- 172.21.15.172:0/1230725376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586d8f6ddf0 0x5586d8fafff0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009e80 tx=0x7fb45c009f70 comp rx=0 tx=0).stop 2024-04-14T08:42:58.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.220+0000 7fb46a3a4640 1 -- 172.21.15.172:0/1230725376 reap_dead start 2024-04-14T08:42:58.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.220+0000 7fb46ca4af40 1 -- 172.21.15.172:0/1230725376 shutdown_connections 2024-04-14T08:42:58.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.220+0000 7fb46ca4af40 1 -- 172.21.15.172:0/1230725376 >> 172.21.15.172:0/1230725376 conn(0x5586d8f993d0 msgr2=0x5586d902ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:58.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.220+0000 7fb46ca4af40 1 -- 172.21.15.172:0/1230725376 shutdown_connections 2024-04-14T08:42:58.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.220+0000 7fb46ca4af40 1 -- 172.21.15.172:0/1230725376 wait complete. 2024-04-14T08:42:58.229 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-14T08:42:58.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.374+0000 7faaaca60640 1 Processor -- start 2024-04-14T08:42:58.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.374+0000 7faaaca60640 1 -- start start 2024-04-14T08:42:58.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaaca60640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7faaa815b770 0x7faaa80cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaaca60640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faaa80cf430 0x7faaa80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaaca60640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaa80cf8c0 con 0x7faaa815bb50 2024-04-14T08:42:58.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaaca60640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7faaa80ce390 con 0x7faaa80cf430 2024-04-14T08:42:58.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaaca60640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7faaa80ce670 con 0x7faaa815b770 2024-04-14T08:42:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaa5d9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faaa80cf430 0x7faaa80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaa5d9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faaa80cf430 0x7faaa80731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45570/0 (socket says 172.21.15.172:45570) 2024-04-14T08:42:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaa5d9b640 1 -- 172.21.15.172:0/2636305120 learned_addr learned my addr 172.21.15.172:0/2636305120 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaa659c640 1 --2- 172.21.15.172:0/2636305120 >> v2:172.21.15.177:3300/0 conn(0x7faaa815b770 0x7faaa80cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaa5d9b640 1 -- 172.21.15.172:0/2636305120 >> v2:172.21.15.177:3300/0 conn(0x7faaa815b770 msgr2=0x7faaa80cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaa5d9b640 1 --2- 172.21.15.172:0/2636305120 >> v2:172.21.15.177:3300/0 conn(0x7faaa815b770 0x7faaa80cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaa5d9b640 1 -- 172.21.15.172:0/2636305120 >> v1:172.21.15.177:6789/0 conn(0x7faaa815bb50 legacy=0x7faaa815bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.375+0000 7faaa5d9b640 1 -- 172.21.15.172:0/2636305120 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa80ce950 con 0x7faaa80cf430 2024-04-14T08:42:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.376+0000 7faaa5d9b640 1 --2- 172.21.15.172:0/2636305120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faaa80cf430 0x7faaa80731c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009870 tx=0x7faa9c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268ef9f1cf3b7f28 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.376+0000 7faaa559a640 1 -- 172.21.15.172:0/2636305120 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c035020 con 0x7faaa80cf430 2024-04-14T08:42:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.376+0000 7faaa559a640 1 -- 172.21.15.172:0/2636305120 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa9c02d040 con 0x7faaa80cf430 2024-04-14T08:42:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.376+0000 7faaa559a640 1 -- 172.21.15.172:0/2636305120 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c031a80 con 0x7faaa80cf430 2024-04-14T08:42:58.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.377+0000 7faaaca60640 1 -- 172.21.15.172:0/2636305120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faaa80cf430 msgr2=0x7faaa80731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.377+0000 7faaaca60640 1 --2- 172.21.15.172:0/2636305120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faaa80cf430 0x7faaa80731c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009870 tx=0x7faa9c02b900 comp rx=0 tx=0).stop 2024-04-14T08:42:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.377+0000 7faaaca60640 1 -- 172.21.15.172:0/2636305120 shutdown_connections 2024-04-14T08:42:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.377+0000 7faaaca60640 1 --2- 172.21.15.172:0/2636305120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faaa80cf430 0x7faaa80731c0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.377+0000 7faaaca60640 1 --2- 172.21.15.172:0/2636305120 >> v2:172.21.15.177:3300/0 conn(0x7faaa815b770 0x7faaa80cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.377+0000 7faaaca60640 1 -- 172.21.15.172:0/2636305120 >> 172.21.15.172:0/2636305120 conn(0x7faaa80b1c60 msgr2=0x7faaa80b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.377+0000 7faaaca60640 1 -- 172.21.15.172:0/2636305120 shutdown_connections 2024-04-14T08:42:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.377+0000 7faaaca60640 1 -- 172.21.15.172:0/2636305120 wait complete. 2024-04-14T08:42:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.378+0000 7faaaca60640 1 Processor -- start 2024-04-14T08:42:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.378+0000 7faaaca60640 1 -- start start 2024-04-14T08:42:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaaca60640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faaa80cf430 0x7faaa80cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaaca60640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faaa815b770 0x7faaa80c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaaca60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faaa815bb50 0x7faaa80c5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaaca60640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faaa815c8c0 con 0x7faaa80cf430 2024-04-14T08:42:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaaca60640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faaa8073700 con 0x7faaa815b770 2024-04-14T08:42:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaaca60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7faaa815ca40 con 0x7faaa815bb50 2024-04-14T08:42:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaa659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faaa80cf430 0x7faaa80cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaa659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faaa80cf430 0x7faaa80cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45586/0 (socket says 172.21.15.172:45586) 2024-04-14T08:42:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaa659c640 1 -- 172.21.15.172:0/2198038053 learned_addr learned my addr 172.21.15.172:0/2198038053 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaa5d9b640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faaa815b770 0x7faaa80c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaa6d9d640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faaa815bb50 0x7faaa80c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaa659c640 1 -- 172.21.15.172:0/2198038053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faaa815bb50 msgr2=0x7faaa80c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaa659c640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faaa815bb50 0x7faaa80c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaa659c640 1 -- 172.21.15.172:0/2198038053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faaa815b770 msgr2=0x7faaa80c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaa659c640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faaa815b770 0x7faaa80c4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaa659c640 1 -- 172.21.15.172:0/2198038053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa813aeb0 con 0x7faaa80cf430 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.379+0000 7faaa6d9d640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faaa815bb50 0x7faaa80c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.380+0000 7faaa659c640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faaa80cf430 0x7faaa80cb2e0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7faa98009db0 tx=0x7faa9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.380+0000 7faa96ffd640 1 -- 172.21.15.172:0/2198038053 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98018020 con 0x7faaa80cf430 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.380+0000 7faa96ffd640 1 -- 172.21.15.172:0/2198038053 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa98014480 con 0x7faaa80cf430 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.380+0000 7faaaca60640 1 -- 172.21.15.172:0/2198038053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa813b190 con 0x7faaa80cf430 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.380+0000 7faa96ffd640 1 -- 172.21.15.172:0/2198038053 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98014850 con 0x7faaa80cf430 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.380+0000 7faaaca60640 1 -- 172.21.15.172:0/2198038053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa813b750 con 0x7faaa80cf430 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.381+0000 7faaaca60640 1 -- 172.21.15.172:0/2198038053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa8068520 con 0x7faaa80cf430 2024-04-14T08:42:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.381+0000 7faa96ffd640 1 -- 172.21.15.172:0/2198038053 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa98005d10 con 0x7faaa80cf430 2024-04-14T08:42:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.382+0000 7faa96ffd640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faa8806c8f0 0x7faa8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.382+0000 7faa96ffd640 1 -- 172.21.15.172:0/2198038053 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7faa9808c7a0 con 0x7faaa80cf430 2024-04-14T08:42:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.382+0000 7faaa5d9b640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faa8806c8f0 0x7faa8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.382+0000 7faaa5d9b640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faa8806c8f0 0x7faa8806edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faa9c036000 tx=0x7faa9c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:58.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.386+0000 7faa96ffd640 1 -- 172.21.15.172:0/2198038053 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7faa98022070 con 0x7faaa80cf430 2024-04-14T08:42:58.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.598+0000 7faaaca60640 1 -- 172.21.15.172:0/2198038053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7faaa815d2f0 con 0x7faaa80cf430 2024-04-14T08:42:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.827+0000 7faa96ffd640 1 -- 172.21.15.172:0/2198038053 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v90) ==== 162+0+0 (secure 0 0 0) 0x7faa98010040 con 0x7faaa80cf430 2024-04-14T08:42:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-14T08:42:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.829+0000 7faaaca60640 1 -- 172.21.15.172:0/2198038053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faa8806c8f0 msgr2=0x7faa8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.829+0000 7faaaca60640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faa8806c8f0 0x7faa8806edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faa9c036000 tx=0x7faa9c03c040 comp rx=0 tx=0).stop 2024-04-14T08:42:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.829+0000 7faaaca60640 1 -- 172.21.15.172:0/2198038053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faaa80cf430 msgr2=0x7faaa80cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.829+0000 7faaaca60640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faaa80cf430 0x7faaa80cb2e0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7faa98009db0 tx=0x7faa9800c490 comp rx=0 tx=0).stop 2024-04-14T08:42:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.829+0000 7faaaca60640 1 -- 172.21.15.172:0/2198038053 shutdown_connections 2024-04-14T08:42:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.829+0000 7faaaca60640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7faa8806c8f0 0x7faa8806edb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.829+0000 7faaaca60640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faaa815bb50 0x7faaa80c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.829+0000 7faaaca60640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faaa815b770 0x7faaa80c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.829+0000 7faaaca60640 1 --2- 172.21.15.172:0/2198038053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faaa80cf430 0x7faaa80cb2e0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.829+0000 7faaaca60640 1 -- 172.21.15.172:0/2198038053 >> 172.21.15.172:0/2198038053 conn(0x7faaa80b1c60 msgr2=0x7faaa8074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.830+0000 7faaaca60640 1 -- 172.21.15.172:0/2198038053 shutdown_connections 2024-04-14T08:42:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.830+0000 7faaaca60640 1 -- 172.21.15.172:0/2198038053 wait complete. 2024-04-14T08:42:58.843 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:42:58.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.944+0000 7f1be9555640 1 Processor -- start 2024-04-14T08:42:58.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.944+0000 7f1be9555640 1 -- start start 2024-04-14T08:42:58.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be9555640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415e790 0x7f1be40d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be9555640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1be40d2680 0x7f1be40d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be9555640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1be415eb70 con 0x7f1be40d0980 2024-04-14T08:42:58.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be9555640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f1be40d02a0 con 0x7f1be40d2680 2024-04-14T08:42:58.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be9555640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1be40d3020 con 0x7f1be415e790 2024-04-14T08:42:58.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be37fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1be40d0980 0x7f1be40d0de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:50878/0 (socket says 172.21.15.172:50878) 2024-04-14T08:42:58.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be37fe640 1 -- 172.21.15.172:0/3837670557 learned_addr learned my addr 172.21.15.172:0/3837670557 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:42:58.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2233494847 0 0) 0x7f1be415eb70 con 0x7f1be40d0980 2024-04-14T08:42:58.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1bdbfff640 1 --2- 172.21.15.172:0/3837670557 >> v2:172.21.15.177:3300/0 conn(0x7f1be40d2680 0x7f1be40d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be2ffd640 1 --2- 172.21.15.172:0/3837670557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415e790 0x7f1be40d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bd0003520 con 0x7f1be40d0980 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2473974995 0 0) 0x7f1bd0003520 con 0x7f1be40d0980 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415e790 msgr2=0x7f1be40d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be27fc640 1 --2- 172.21.15.172:0/3837670557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415e790 0x7f1be40d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 >> v2:172.21.15.177:3300/0 conn(0x7f1be40d2680 msgr2=0x7f1be40d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be27fc640 1 --2- 172.21.15.172:0/3837670557 >> v2:172.21.15.177:3300/0 conn(0x7f1be40d2680 0x7f1be40d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be40cfce0 con 0x7f1be40d0980 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f1bdc0019f0 con 0x7f1be40d0980 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.945+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 >> v1:172.21.15.172:6789/0 conn(0x7f1be40d0980 legacy=0x7f1be40d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be27fc640 1 --2- 172.21.15.172:0/3837670557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415e790 0x7f1bd00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be27fc640 1 --2- 172.21.15.172:0/3837670557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0003f60 0x7f1bd0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be27fc640 1 --2- 172.21.15.172:0/3837670557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bd0004d40 0x7f1bd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd000b9c0 con 0x7f1bd0004d40 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd000bb40 con 0x7f1bd0003f60 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd000bcc0 con 0x7f1be415e790 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be37fe640 1 --2- 172.21.15.172:0/3837670557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bd0004d40 0x7f1bd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1bdbfff640 1 --2- 172.21.15.172:0/3837670557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0003f60 0x7f1bd0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be2ffd640 1 --2- 172.21.15.172:0/3837670557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415e790 0x7f1bd00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be37fe640 1 -- 172.21.15.172:0/3837670557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415e790 msgr2=0x7f1bd00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be37fe640 1 --2- 172.21.15.172:0/3837670557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415e790 0x7f1bd00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be37fe640 1 -- 172.21.15.172:0/3837670557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0003f60 msgr2=0x7f1bd0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be37fe640 1 --2- 172.21.15.172:0/3837670557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0003f60 0x7f1bd0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.946+0000 7f1be37fe640 1 -- 172.21.15.172:0/3837670557 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd0003520 con 0x7f1bd0004d40 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.947+0000 7f1be37fe640 1 --2- 172.21.15.172:0/3837670557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bd0004d40 0x7f1bd00051c0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00a850 tx=0x7f1bd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a57d39f8eec37 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.947+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc00a4a0 con 0x7f1bd0004d40 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.947+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bdc034650 con 0x7f1bd0004d40 2024-04-14T08:42:58.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.947+0000 7f1be27fc640 1 -- 172.21.15.172:0/3837670557 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc034a80 con 0x7f1bd0004d40 2024-04-14T08:42:58.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.947+0000 7f1be9555640 1 -- 172.21.15.172:0/3837670557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bd0004d40 msgr2=0x7f1bd00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.947+0000 7f1be9555640 1 --2- 172.21.15.172:0/3837670557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bd0004d40 0x7f1bd00051c0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00a850 tx=0x7f1bd0005830 comp rx=0 tx=0).stop 2024-04-14T08:42:58.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.947+0000 7f1be37fe640 1 -- 172.21.15.172:0/3837670557 reap_dead start 2024-04-14T08:42:58.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.947+0000 7f1be9555640 1 -- 172.21.15.172:0/3837670557 shutdown_connections 2024-04-14T08:42:58.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.947+0000 7f1be9555640 1 -- 172.21.15.172:0/3837670557 >> 172.21.15.172:0/3837670557 conn(0x7f1be409d590 msgr2=0x7f1be40d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:58.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.947+0000 7f1be9555640 1 -- 172.21.15.172:0/3837670557 shutdown_connections 2024-04-14T08:42:58.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.947+0000 7f1be9555640 1 -- 172.21.15.172:0/3837670557 wait complete. 2024-04-14T08:42:58.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1be9555640 1 Processor -- start 2024-04-14T08:42:58.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1be9555640 1 -- start start 2024-04-14T08:42:58.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1be9555640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0004d40 0x7f1be41f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1be9555640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be41f4c00 0x7f1be40b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1be9555640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be40b5050 0x7f1be40b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1be9555640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1be40b1020 con 0x7f1be40b5050 2024-04-14T08:42:58.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1be9555640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1be40b1390 con 0x7f1bd0004d40 2024-04-14T08:42:58.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1be9555640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1be40b3510 con 0x7f1be41f4c00 2024-04-14T08:42:58.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1be37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be40b5050 0x7f1be40b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1be37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be40b5050 0x7f1be40b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45608/0 (socket says 172.21.15.172:45608) 2024-04-14T08:42:58.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1be37fe640 1 -- 172.21.15.172:0/2135044200 learned_addr learned my addr 172.21.15.172:0/2135044200 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1be2ffd640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0004d40 0x7f1be41f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.949+0000 7f1bdbfff640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be41f4c00 0x7f1be40b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.950+0000 7f1be37fe640 1 -- 172.21.15.172:0/2135044200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be41f4c00 msgr2=0x7f1be40b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.950+0000 7f1be37fe640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be41f4c00 0x7f1be40b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.950+0000 7f1be37fe640 1 -- 172.21.15.172:0/2135044200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0004d40 msgr2=0x7f1be41f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.950+0000 7f1be37fe640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0004d40 0x7f1be41f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.950+0000 7f1be37fe640 1 -- 172.21.15.172:0/2135044200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be41f3de0 con 0x7f1be40b5050 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.950+0000 7f1be37fe640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be40b5050 0x7f1be40b3780 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc001cf0 tx=0x7f1bdc001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.950+0000 7f1bdb7fe640 1 -- 172.21.15.172:0/2135044200 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc00a480 con 0x7f1be40b5050 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.950+0000 7f1bdb7fe640 1 -- 172.21.15.172:0/2135044200 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bdc044070 con 0x7f1be40b5050 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.950+0000 7f1be9555640 1 -- 172.21.15.172:0/2135044200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be40b4390 con 0x7f1be40b5050 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.950+0000 7f1bdb7fe640 1 -- 172.21.15.172:0/2135044200 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc034a00 con 0x7f1be40b5050 2024-04-14T08:42:58.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.950+0000 7f1be9555640 1 -- 172.21.15.172:0/2135044200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be40c7e20 con 0x7f1be40b5050 2024-04-14T08:42:58.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.951+0000 7f1be9555640 1 -- 172.21.15.172:0/2135044200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be4068520 con 0x7f1be40b5050 2024-04-14T08:42:58.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.951+0000 7f1bdb7fe640 1 -- 172.21.15.172:0/2135044200 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bdc034ba0 con 0x7f1be40b5050 2024-04-14T08:42:58.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.952+0000 7f1bdb7fe640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1bc406c920 0x7f1bc406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:42:58.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.952+0000 7f1bdb7fe640 1 -- 172.21.15.172:0/2135044200 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f1bdc0adea0 con 0x7f1be40b5050 2024-04-14T08:42:58.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.952+0000 7f1be2ffd640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1bc406c920 0x7f1bc406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:42:58.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.952+0000 7f1be2ffd640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1bc406c920 0x7f1bc406ede0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009bc0 tx=0x7f1bd4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:42:58.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:58.956+0000 7f1bdb7fe640 1 -- 172.21.15.172:0/2135044200 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1be41f3de0 con 0x7f1be40b5050 2024-04-14T08:42:59.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.168+0000 7f1be9555640 1 -- 172.21.15.172:0/2135044200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1be40b2980 con 0x7f1be40b5050 2024-04-14T08:42:59.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.169+0000 7f1bdb7fe640 1 -- 172.21.15.172:0/2135044200 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5187 (secure 0 0 0) 0x7f1bdc032e40 con 0x7f1be40b5050 2024-04-14T08:42:59.169 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:42:59.169 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":90,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3881291102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3881291102},{"type":"v1","addr":"172.21.15.177:6835","nonce":3881291102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:42:58.826627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9048":{"gid":9048,"name":"c","rank":0,"incarnation":90,"state":"up:replay","state_seq":1,"addr":"172.21.15.172:6837/744853572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":744853572},{"type":"v1","addr":"172.21.15.172:6837","nonce":744853572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-14T08:42:59.169 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 90 2024-04-14T08:42:59.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.171+0000 7f1be9555640 1 -- 172.21.15.172:0/2135044200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1bc406c920 msgr2=0x7f1bc406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:59.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.171+0000 7f1be9555640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1bc406c920 0x7f1bc406ede0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009bc0 tx=0x7f1bd4009190 comp rx=0 tx=0).stop 2024-04-14T08:42:59.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.171+0000 7f1be9555640 1 -- 172.21.15.172:0/2135044200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be40b5050 msgr2=0x7f1be40b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:42:59.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.171+0000 7f1be9555640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be40b5050 0x7f1be40b3780 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc001cf0 tx=0x7f1bdc001dc0 comp rx=0 tx=0).stop 2024-04-14T08:42:59.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.171+0000 7f1be9555640 1 -- 172.21.15.172:0/2135044200 shutdown_connections 2024-04-14T08:42:59.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.171+0000 7f1be9555640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1bc406c920 0x7f1bc406ede0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:59.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.171+0000 7f1be9555640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1be40b5050 0x7f1be40b3780 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:59.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.171+0000 7f1be9555640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be41f4c00 0x7f1be40b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:59.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.171+0000 7f1be9555640 1 --2- 172.21.15.172:0/2135044200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0004d40 0x7f1be41f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:42:59.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.171+0000 7f1be9555640 1 -- 172.21.15.172:0/2135044200 >> 172.21.15.172:0/2135044200 conn(0x7f1be409d590 msgr2=0x7f1be415d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:42:59.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.171+0000 7f1be9555640 1 -- 172.21.15.172:0/2135044200 shutdown_connections 2024-04-14T08:42:59.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:42:59.171+0000 7f1be9555640 1 -- 172.21.15.172:0/2135044200 wait complete. 2024-04-14T08:42:59.585 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:42:40.251538+0000', 'modified': '2024-04-14T08:42:58.826627+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9048}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9048': {'gid': 9048, 'name': 'c', 'rank': 0, 'incarnation': 90, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.172:6837/744853572', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 744853572}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 744853572}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-14T08:42:59.585 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9048:up:replay 2024-04-14T08:43:00.587 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:43:00.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.698+0000 7f7634d1e640 1 Processor -- start 2024-04-14T08:43:00.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.698+0000 7f7634d1e640 1 -- start start 2024-04-14T08:43:00.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.699+0000 7f7634d1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f76300b21d0 0x7f763015c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:00.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.699+0000 7f7634d1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76300b3700 con 0x7f76300b1df0 2024-04-14T08:43:00.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.699+0000 7f7634d1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76300b39e0 con 0x7f763015c5a0 2024-04-14T08:43:00.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.699+0000 7f7634d1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f76300b3cc0 con 0x7f76300b21d0 2024-04-14T08:43:00.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.699+0000 7f762dd9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f76300b21d0 0x7f763015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:00.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.699+0000 7f762e59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f76300b1df0 0x7f76300b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45778/0 (socket says 172.21.15.172:45778) 2024-04-14T08:43:00.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.699+0000 7f762e59c640 1 -- 172.21.15.172:0/1037753369 learned_addr learned my addr 172.21.15.172:0/1037753369 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:00.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.700+0000 7f762dd9b640 1 -- 172.21.15.172:0/1037753369 >> v1:172.21.15.177:6789/0 conn(0x7f763015c5a0 legacy=0x7f76300b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:00.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.700+0000 7f762dd9b640 1 -- 172.21.15.172:0/1037753369 >> v1:172.21.15.172:6789/0 conn(0x7f76300b1df0 legacy=0x7f76300b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:00.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.700+0000 7f762d59a640 1 -- 172.21.15.172:0/1037753369 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3133451484 0 0) 0x7f76300b3700 con 0x7f76300b1df0 2024-04-14T08:43:00.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.700+0000 7f762dd9b640 1 -- 172.21.15.172:0/1037753369 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763015bb40 con 0x7f76300b21d0 2024-04-14T08:43:00.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.701+0000 7f762dd9b640 1 --2- 172.21.15.172:0/1037753369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76300b21d0 0x7f763015c060 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f76240097a0 tx=0x7f762402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4207c25a8eb3c7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:00.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.701+0000 7f762d59a640 1 -- 172.21.15.172:0/1037753369 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624035020 con 0x7f76300b21d0 2024-04-14T08:43:00.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.701+0000 7f762d59a640 1 -- 172.21.15.172:0/1037753369 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7624031830 con 0x7f76300b21d0 2024-04-14T08:43:00.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.701+0000 7f762d59a640 1 -- 172.21.15.172:0/1037753369 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624031c00 con 0x7f76300b21d0 2024-04-14T08:43:00.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.701+0000 7f7634d1e640 1 -- 172.21.15.172:0/1037753369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76300b21d0 msgr2=0x7f763015c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:00.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.701+0000 7f7634d1e640 1 --2- 172.21.15.172:0/1037753369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76300b21d0 0x7f763015c060 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f76240097a0 tx=0x7f762402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:43:00.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.701+0000 7f7634d1e640 1 -- 172.21.15.172:0/1037753369 shutdown_connections 2024-04-14T08:43:00.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.701+0000 7f7634d1e640 1 --2- 172.21.15.172:0/1037753369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76300b21d0 0x7f763015c060 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:00.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.701+0000 7f7634d1e640 1 -- 172.21.15.172:0/1037753369 >> 172.21.15.172:0/1037753369 conn(0x7f763006ec30 msgr2=0x7f763006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:00.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.701+0000 7f7634d1e640 1 -- 172.21.15.172:0/1037753369 shutdown_connections 2024-04-14T08:43:00.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.701+0000 7f7634d1e640 1 -- 172.21.15.172:0/1037753369 wait complete. 2024-04-14T08:43:00.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.703+0000 7f7634d1e640 1 Processor -- start 2024-04-14T08:43:00.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.703+0000 7f7634d1e640 1 -- start start 2024-04-14T08:43:00.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.703+0000 7f7634d1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76300b1df0 0x7f76300cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:00.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.703+0000 7f7634d1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76300b21d0 0x7f763013cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:00.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.703+0000 7f7634d1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76300a9f70 0x7f76300aa420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:00.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.703+0000 7f7634d1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f76300b48d0 con 0x7f76300b1df0 2024-04-14T08:43:00.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.703+0000 7f7634d1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f76300b4750 con 0x7f76300a9f70 2024-04-14T08:43:00.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.703+0000 7f7634d1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f76301f4b80 con 0x7f76300b21d0 2024-04-14T08:43:00.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f762e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76300b1df0 0x7f76300cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:00.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f762e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76300b1df0 0x7f76300cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59382/0 (socket says 172.21.15.172:59382) 2024-04-14T08:43:00.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f762e59c640 1 -- 172.21.15.172:0/970162194 learned_addr learned my addr 172.21.15.172:0/970162194 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:00.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f762ed9d640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76300a9f70 0x7f76300aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:00.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f762dd9b640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76300b21d0 0x7f763013cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:00.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f762e59c640 1 -- 172.21.15.172:0/970162194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76300b21d0 msgr2=0x7f763013cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:00.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f762e59c640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76300b21d0 0x7f763013cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:00.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f762e59c640 1 -- 172.21.15.172:0/970162194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76300a9f70 msgr2=0x7f76300aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:00.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f762e59c640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76300a9f70 0x7f76300aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:00.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f762e59c640 1 -- 172.21.15.172:0/970162194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76300aab30 con 0x7f76300b1df0 2024-04-14T08:43:00.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f762ed9d640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76300a9f70 0x7f76300aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:00.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f762e59c640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76300b1df0 0x7f76300cf160 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f762000c890 tx=0x7f762000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:00.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f761effd640 1 -- 172.21.15.172:0/970162194 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7620018070 con 0x7f76300b1df0 2024-04-14T08:43:00.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f761effd640 1 -- 172.21.15.172:0/970162194 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f762000a640 con 0x7f76300b1df0 2024-04-14T08:43:00.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f761effd640 1 -- 172.21.15.172:0/970162194 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762000aa30 con 0x7f76300b1df0 2024-04-14T08:43:00.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f7634d1e640 1 -- 172.21.15.172:0/970162194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76301429a0 con 0x7f76300b1df0 2024-04-14T08:43:00.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.704+0000 7f7634d1e640 1 -- 172.21.15.172:0/970162194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7630142f60 con 0x7f76300b1df0 2024-04-14T08:43:00.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.705+0000 7f7634d1e640 1 -- 172.21.15.172:0/970162194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7630068520 con 0x7f76300b1df0 2024-04-14T08:43:00.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.706+0000 7f761effd640 1 -- 172.21.15.172:0/970162194 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f762001b070 con 0x7f76300b1df0 2024-04-14T08:43:00.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.706+0000 7f761effd640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f761006c8f0 0x7f761006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:00.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.707+0000 7f762dd9b640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f761006c8f0 0x7f761006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:00.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.707+0000 7f761effd640 1 -- 172.21.15.172:0/970162194 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f762008b4e0 con 0x7f76300b1df0 2024-04-14T08:43:00.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.707+0000 7f762dd9b640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f761006c8f0 0x7f761006edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7624036000 tx=0x7f762403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:00.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.711+0000 7f761effd640 1 -- 172.21.15.172:0/970162194 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7620059690 con 0x7f76300b1df0 2024-04-14T08:43:00.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.924+0000 7f7634d1e640 1 -- 172.21.15.172:0/970162194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76300732e0 con 0x7f76300b1df0 2024-04-14T08:43:00.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.925+0000 7f761effd640 1 -- 172.21.15.172:0/970162194 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5187 (secure 0 0 0) 0x7f7620058f90 con 0x7f76300b1df0 2024-04-14T08:43:00.926 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:00.926 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":92,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3881291102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3881291102},{"type":"v1","addr":"172.21.15.177:6835","nonce":3881291102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:42:59.831475+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9048":{"gid":9048,"name":"c","rank":0,"incarnation":90,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.172:6837/744853572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":744853572},{"type":"v1","addr":"172.21.15.172:6837","nonce":744853572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-14T08:43:00.926 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 92 2024-04-14T08:43:00.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.927+0000 7f7634d1e640 1 -- 172.21.15.172:0/970162194 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f761006c8f0 msgr2=0x7f761006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:00.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.927+0000 7f7634d1e640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f761006c8f0 0x7f761006edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7624036000 tx=0x7f762403c040 comp rx=0 tx=0).stop 2024-04-14T08:43:00.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.927+0000 7f7634d1e640 1 -- 172.21.15.172:0/970162194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76300b1df0 msgr2=0x7f76300cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:00.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.927+0000 7f7634d1e640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76300b1df0 0x7f76300cf160 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f762000c890 tx=0x7f762000cd50 comp rx=0 tx=0).stop 2024-04-14T08:43:00.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.927+0000 7f7634d1e640 1 -- 172.21.15.172:0/970162194 shutdown_connections 2024-04-14T08:43:00.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.927+0000 7f7634d1e640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f761006c8f0 0x7f761006edb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:00.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.927+0000 7f7634d1e640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76300a9f70 0x7f76300aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:00.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.927+0000 7f7634d1e640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f76300b21d0 0x7f763013cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:00.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.927+0000 7f7634d1e640 1 --2- 172.21.15.172:0/970162194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76300b1df0 0x7f76300cf160 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:00.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.927+0000 7f7634d1e640 1 -- 172.21.15.172:0/970162194 >> 172.21.15.172:0/970162194 conn(0x7f763006ec30 msgr2=0x7f76300b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:00.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.927+0000 7f7634d1e640 1 -- 172.21.15.172:0/970162194 shutdown_connections 2024-04-14T08:43:00.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:00.927+0000 7f7634d1e640 1 -- 172.21.15.172:0/970162194 wait complete. 2024-04-14T08:43:01.340 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:42:40.251538+0000', 'modified': '2024-04-14T08:42:59.831475+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9048}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9048': {'gid': 9048, 'name': 'c', 'rank': 0, 'incarnation': 90, 'state': 'up:rejoin', 'state_seq': 8, 'addr': '172.21.15.172:6837/744853572', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 744853572}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 744853572}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-14T08:43:01.341 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9048:up:rejoin 2024-04-14T08:43:02.341 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:43:02.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.443+0000 7fd262f1e640 1 Processor -- start 2024-04-14T08:43:02.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.443+0000 7fd262f1e640 1 -- start start 2024-04-14T08:43:02.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.444+0000 7fd262f1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd25c0cedd0 0x7fd25c1694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:02.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.444+0000 7fd262f1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd25c0cf320 con 0x7fd25c15e890 2024-04-14T08:43:02.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.444+0000 7fd262f1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd25c0cf600 con 0x7fd25c0d2830 2024-04-14T08:43:02.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.444+0000 7fd262f1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd25c0cff30 con 0x7fd25c0cedd0 2024-04-14T08:43:02.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.444+0000 7fd25bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd25c15e890 0x7fd25c0d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45780/0 (socket says 172.21.15.172:45780) 2024-04-14T08:43:02.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.444+0000 7fd25bfff640 1 -- 172.21.15.172:0/1156855769 learned_addr learned my addr 172.21.15.172:0/1156855769 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:02.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.444+0000 7fd2614bb640 1 --2- 172.21.15.172:0/1156855769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd25c0cedd0 0x7fd25c1694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:02.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.444+0000 7fd25b7fe640 1 -- 172.21.15.172:0/1156855769 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 405735917 0 0) 0x7fd25c0cf320 con 0x7fd25c15e890 2024-04-14T08:43:02.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.444+0000 7fd25b7fe640 1 -- 172.21.15.172:0/1156855769 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd24c003520 con 0x7fd25c15e890 2024-04-14T08:43:02.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.445+0000 7fd25b7fe640 1 -- 172.21.15.172:0/1156855769 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4118958045 0 0) 0x7fd25c0cf600 con 0x7fd25c0d2830 2024-04-14T08:43:02.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.445+0000 7fd25b7fe640 1 -- 172.21.15.172:0/1156855769 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd25c0cf320 con 0x7fd25c0d2830 2024-04-14T08:43:02.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.445+0000 7fd25b7fe640 1 -- 172.21.15.172:0/1156855769 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 198132390 0 0) 0x7fd24c003520 con 0x7fd25c15e890 2024-04-14T08:43:02.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.445+0000 7fd2614bb640 1 -- 172.21.15.172:0/1156855769 >> v1:172.21.15.177:6789/0 conn(0x7fd25c0d2830 legacy=0x7fd25c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:02.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.445+0000 7fd2614bb640 1 -- 172.21.15.172:0/1156855769 >> v1:172.21.15.172:6789/0 conn(0x7fd25c15e890 legacy=0x7fd25c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:02.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.445+0000 7fd2614bb640 1 -- 172.21.15.172:0/1156855769 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c0cf8e0 con 0x7fd25c0cedd0 2024-04-14T08:43:02.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.445+0000 7fd2614bb640 1 --2- 172.21.15.172:0/1156855769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd25c0cedd0 0x7fd25c1694d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd254009dc0 tx=0x7fd25402be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d1c613a6ff569ec server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:02.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.446+0000 7fd25b7fe640 1 -- 172.21.15.172:0/1156855769 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254039070 con 0x7fd25c0cedd0 2024-04-14T08:43:02.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.446+0000 7fd25b7fe640 1 -- 172.21.15.172:0/1156855769 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd25402c9a0 con 0x7fd25c0cedd0 2024-04-14T08:43:02.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.446+0000 7fd25b7fe640 1 -- 172.21.15.172:0/1156855769 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248001670 con 0x7fd25c0cedd0 2024-04-14T08:43:02.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.446+0000 7fd262f1e640 1 -- 172.21.15.172:0/1156855769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd25c0cedd0 msgr2=0x7fd25c1694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:02.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.446+0000 7fd262f1e640 1 --2- 172.21.15.172:0/1156855769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd25c0cedd0 0x7fd25c1694d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd254009dc0 tx=0x7fd25402be70 comp rx=0 tx=0).stop 2024-04-14T08:43:02.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.446+0000 7fd262f1e640 1 -- 172.21.15.172:0/1156855769 shutdown_connections 2024-04-14T08:43:02.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.446+0000 7fd262f1e640 1 --2- 172.21.15.172:0/1156855769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd25c0cedd0 0x7fd25c1694d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:02.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.446+0000 7fd262f1e640 1 -- 172.21.15.172:0/1156855769 >> 172.21.15.172:0/1156855769 conn(0x7fd25c09d590 msgr2=0x7fd25c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:02.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.446+0000 7fd262f1e640 1 -- 172.21.15.172:0/1156855769 shutdown_connections 2024-04-14T08:43:02.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.446+0000 7fd262f1e640 1 -- 172.21.15.172:0/1156855769 wait complete. 2024-04-14T08:43:02.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.447+0000 7fd262f1e640 1 Processor -- start 2024-04-14T08:43:02.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.447+0000 7fd262f1e640 1 -- start start 2024-04-14T08:43:02.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.448+0000 7fd262f1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd25c0cedd0 0x7fd25c0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:02.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.448+0000 7fd262f1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd25c0d2830 0x7fd25c160e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:02.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.448+0000 7fd262f1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd25c15e890 0x7fd25c161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:02.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.448+0000 7fd262f1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c0b4720 con 0x7fd25c15e890 2024-04-14T08:43:02.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.448+0000 7fd262f1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c0d2350 con 0x7fd25c0cedd0 2024-04-14T08:43:02.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.448+0000 7fd262f1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fd25c0b48a0 con 0x7fd25c0d2830 2024-04-14T08:43:02.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.448+0000 7fd2614bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd25c15e890 0x7fd25c161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:02.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.448+0000 7fd2614bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd25c15e890 0x7fd25c161380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59388/0 (socket says 172.21.15.172:59388) 2024-04-14T08:43:02.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.448+0000 7fd2614bb640 1 -- 172.21.15.172:0/3199650618 learned_addr learned my addr 172.21.15.172:0/3199650618 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:02.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.448+0000 7fd260cba640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd25c0cedd0 0x7fd25c0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:02.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.448+0000 7fd25bfff640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd25c0d2830 0x7fd25c160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.449+0000 7fd2614bb640 1 -- 172.21.15.172:0/3199650618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd25c0d2830 msgr2=0x7fd25c160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.449+0000 7fd2614bb640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd25c0d2830 0x7fd25c160e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.449+0000 7fd2614bb640 1 -- 172.21.15.172:0/3199650618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd25c0cedd0 msgr2=0x7fd25c0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.449+0000 7fd2614bb640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd25c0cedd0 0x7fd25c0c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.449+0000 7fd2614bb640 1 -- 172.21.15.172:0/3199650618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c14e320 con 0x7fd25c15e890 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.449+0000 7fd2614bb640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd25c15e890 0x7fd25c161380 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fd254009ee0 tx=0x7fd254004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.449+0000 7fd2597fa640 1 -- 172.21.15.172:0/3199650618 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254039050 con 0x7fd25c15e890 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.449+0000 7fd2597fa640 1 -- 172.21.15.172:0/3199650618 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd254045070 con 0x7fd25c15e890 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.449+0000 7fd262f1e640 1 -- 172.21.15.172:0/3199650618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25c14e600 con 0x7fd25c15e890 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.449+0000 7fd2597fa640 1 -- 172.21.15.172:0/3199650618 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254009450 con 0x7fd25c15e890 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.449+0000 7fd262f1e640 1 -- 172.21.15.172:0/3199650618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25c14ba00 con 0x7fd25c15e890 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.450+0000 7fd262f1e640 1 -- 172.21.15.172:0/3199650618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd25c068520 con 0x7fd25c15e890 2024-04-14T08:43:02.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.451+0000 7fd2597fa640 1 -- 172.21.15.172:0/3199650618 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd254004690 con 0x7fd25c15e890 2024-04-14T08:43:02.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.451+0000 7fd2597fa640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd24006c8f0 0x7fd24006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:02.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.451+0000 7fd2597fa640 1 -- 172.21.15.172:0/3199650618 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7fd2540ae4d0 con 0x7fd25c15e890 2024-04-14T08:43:02.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.451+0000 7fd260cba640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd24006c8f0 0x7fd24006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:02.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.451+0000 7fd260cba640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd24006c8f0 0x7fd24006edb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd2500011e0 tx=0x7fd250009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:02.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.455+0000 7fd2597fa640 1 -- 172.21.15.172:0/3199650618 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd25407c280 con 0x7fd25c15e890 2024-04-14T08:43:02.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.694+0000 7fd262f1e640 1 -- 172.21.15.172:0/3199650618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd25c0d2610 con 0x7fd25c15e890 2024-04-14T08:43:02.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.695+0000 7fd2597fa640 1 -- 172.21.15.172:0/3199650618 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5194 (secure 0 0 0) 0x7fd254080580 con 0x7fd25c15e890 2024-04-14T08:43:02.696 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:02.696 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3881291102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3881291102},{"type":"v1","addr":"172.21.15.177:6835","nonce":3881291102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:43:01.831015+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9048":{"gid":9048,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6837/744853572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":744853572},{"type":"v1","addr":"172.21.15.172:6837","nonce":744853572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9048,"qdb_cluster":[9048]},"id":8}]} 2024-04-14T08:43:02.696 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 93 2024-04-14T08:43:02.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.698+0000 7fd262f1e640 1 -- 172.21.15.172:0/3199650618 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd24006c8f0 msgr2=0x7fd24006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:02.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.698+0000 7fd262f1e640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd24006c8f0 0x7fd24006edb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd2500011e0 tx=0x7fd250009040 comp rx=0 tx=0).stop 2024-04-14T08:43:02.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.698+0000 7fd262f1e640 1 -- 172.21.15.172:0/3199650618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd25c15e890 msgr2=0x7fd25c161380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:02.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.698+0000 7fd262f1e640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd25c15e890 0x7fd25c161380 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fd254009ee0 tx=0x7fd254004a40 comp rx=0 tx=0).stop 2024-04-14T08:43:02.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.698+0000 7fd262f1e640 1 -- 172.21.15.172:0/3199650618 shutdown_connections 2024-04-14T08:43:02.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.698+0000 7fd262f1e640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd24006c8f0 0x7fd24006edb0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:02.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.698+0000 7fd262f1e640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd25c15e890 0x7fd25c161380 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:02.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.698+0000 7fd262f1e640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd25c0d2830 0x7fd25c160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:02.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.698+0000 7fd262f1e640 1 --2- 172.21.15.172:0/3199650618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd25c0cedd0 0x7fd25c0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:02.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.698+0000 7fd262f1e640 1 -- 172.21.15.172:0/3199650618 >> 172.21.15.172:0/3199650618 conn(0x7fd25c09d590 msgr2=0x7fd25c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:02.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.698+0000 7fd262f1e640 1 -- 172.21.15.172:0/3199650618 shutdown_connections 2024-04-14T08:43:02.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:02.698+0000 7fd262f1e640 1 -- 172.21.15.172:0/3199650618 wait complete. 2024-04-14T08:43:03.111 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:42:40.251538+0000', 'modified': '2024-04-14T08:43:01.831015+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9048}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9048': {'gid': 9048, 'name': 'c', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.172:6837/744853572', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 744853572}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 744853572}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9048, 'qdb_cluster': [9048]} 2024-04-14T08:43:03.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:43:03.112 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-14T08:43:03.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.216+0000 7f2a31a0f640 1 Processor -- start 2024-04-14T08:43:03.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.216+0000 7f2a31a0f640 1 -- start start 2024-04-14T08:43:03.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.217+0000 7f2a31a0f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a2c0ccca0 0x7f2a2c06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.217+0000 7f2a31a0f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a2c073090 0x7f2a2c0b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.217+0000 7f2a31a0f640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a2c0734f0 con 0x7f2a2c06b520 2024-04-14T08:43:03.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.217+0000 7f2a31a0f640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2a2c0ceee0 con 0x7f2a2c0ccca0 2024-04-14T08:43:03.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.217+0000 7f2a31a0f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2c070150 con 0x7f2a2c073090 2024-04-14T08:43:03.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.217+0000 7f2a2affd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a2c0ccca0 0x7f2a2c06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.217+0000 7f2a2b7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2a2c06b520 0x7f2a2c06e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45786/0 (socket says 172.21.15.172:45786) 2024-04-14T08:43:03.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.217+0000 7f2a2b7fe640 1 -- 172.21.15.172:0/4207928659 learned_addr learned my addr 172.21.15.172:0/4207928659 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:03.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.218+0000 7f2a2a7fc640 1 --2- 172.21.15.172:0/4207928659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a2c073090 0x7f2a2c0b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.218+0000 7f2a2affd640 1 -- 172.21.15.172:0/4207928659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a2c073090 msgr2=0x7f2a2c0b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:03.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.218+0000 7f2a2affd640 1 --2- 172.21.15.172:0/4207928659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a2c073090 0x7f2a2c0b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.218+0000 7f2a2affd640 1 -- 172.21.15.172:0/4207928659 >> v1:172.21.15.172:6789/0 conn(0x7f2a2c06b520 legacy=0x7f2a2c06e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.218+0000 7f2a2affd640 1 -- 172.21.15.172:0/4207928659 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2c0722d0 con 0x7f2a2c0ccca0 2024-04-14T08:43:03.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.218+0000 7f2a29ffb640 1 -- 172.21.15.172:0/4207928659 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1643808688 0 0) 0x7f2a2c0734f0 con 0x7f2a2c06b520 2024-04-14T08:43:03.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.218+0000 7f2a2affd640 1 --2- 172.21.15.172:0/4207928659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a2c0ccca0 0x7f2a2c06fb70 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0097a0 tx=0x7f2a1c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c057036b9fea555b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.219+0000 7f2a29ffb640 1 -- 172.21.15.172:0/4207928659 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c0315a0 con 0x7f2a2c0ccca0 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.219+0000 7f2a29ffb640 1 -- 172.21.15.172:0/4207928659 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a1c037070 con 0x7f2a2c0ccca0 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.219+0000 7f2a29ffb640 1 -- 172.21.15.172:0/4207928659 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c031960 con 0x7f2a2c0ccca0 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.219+0000 7f2a31a0f640 1 -- 172.21.15.172:0/4207928659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a2c0ccca0 msgr2=0x7f2a2c06fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.219+0000 7f2a31a0f640 1 --2- 172.21.15.172:0/4207928659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a2c0ccca0 0x7f2a2c06fb70 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0097a0 tx=0x7f2a1c02b840 comp rx=0 tx=0).stop 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.219+0000 7f2a31a0f640 1 -- 172.21.15.172:0/4207928659 shutdown_connections 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.219+0000 7f2a31a0f640 1 --2- 172.21.15.172:0/4207928659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a2c073090 0x7f2a2c0b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.219+0000 7f2a31a0f640 1 --2- 172.21.15.172:0/4207928659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a2c0ccca0 0x7f2a2c06fb70 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.219+0000 7f2a31a0f640 1 -- 172.21.15.172:0/4207928659 >> 172.21.15.172:0/4207928659 conn(0x7f2a2c0cda00 msgr2=0x7f2a2c0cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.219+0000 7f2a31a0f640 1 -- 172.21.15.172:0/4207928659 shutdown_connections 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.219+0000 7f2a31a0f640 1 -- 172.21.15.172:0/4207928659 wait complete. 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.220+0000 7f2a31a0f640 1 Processor -- start 2024-04-14T08:43:03.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.220+0000 7f2a31a0f640 1 -- start start 2024-04-14T08:43:03.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.221+0000 7f2a31a0f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a2c06b520 0x7f2a2c15f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.221+0000 7f2a31a0f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a2c073090 0x7f2a2c0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.221+0000 7f2a31a0f640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a2c0ccca0 0x7f2a2c0b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.221+0000 7f2a31a0f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c0cd080 con 0x7f2a2c06b520 2024-04-14T08:43:03.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.221+0000 7f2a31a0f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c072890 con 0x7f2a2c073090 2024-04-14T08:43:03.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.221+0000 7f2a31a0f640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2c06b950 con 0x7f2a2c0ccca0 2024-04-14T08:43:03.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.221+0000 7f2a2affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a2c06b520 0x7f2a2c15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.221+0000 7f2a2affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a2c06b520 0x7f2a2c15f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59416/0 (socket says 172.21.15.172:59416) 2024-04-14T08:43:03.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.221+0000 7f2a2affd640 1 -- 172.21.15.172:0/1864384851 learned_addr learned my addr 172.21.15.172:0/1864384851 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:03.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a2b7fe640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a2c0ccca0 0x7f2a2c0b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a2affd640 1 -- 172.21.15.172:0/1864384851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a2c0ccca0 msgr2=0x7f2a2c0b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a2a7fc640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a2c073090 0x7f2a2c0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a2affd640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a2c0ccca0 0x7f2a2c0b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a2affd640 1 -- 172.21.15.172:0/1864384851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a2c073090 msgr2=0x7f2a2c0addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a2affd640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a2c073090 0x7f2a2c0addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a2affd640 1 -- 172.21.15.172:0/1864384851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2c0ca750 con 0x7f2a2c06b520 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a2b7fe640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a2c0ccca0 0x7f2a2c0b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a2a7fc640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a2c073090 0x7f2a2c0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a2affd640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a2c06b520 0x7f2a2c15f120 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c034040 tx=0x7f2a1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a23fff640 1 -- 172.21.15.172:0/1864384851 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c004230 con 0x7f2a2c06b520 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a23fff640 1 -- 172.21.15.172:0/1864384851 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a1c0043f0 con 0x7f2a2c06b520 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a31a0f640 1 -- 172.21.15.172:0/1864384851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2c0b0a50 con 0x7f2a2c06b520 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a23fff640 1 -- 172.21.15.172:0/1864384851 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c003850 con 0x7f2a2c06b520 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.222+0000 7f2a31a0f640 1 -- 172.21.15.172:0/1864384851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2c0ae2f0 con 0x7f2a2c06b520 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.223+0000 7f2a23fff640 1 -- 172.21.15.172:0/1864384851 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a1c0039f0 con 0x7f2a2c06b520 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.224+0000 7f2a23fff640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a0c06c8f0 0x7f2a0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.224+0000 7f2a23fff640 1 -- 172.21.15.172:0/1864384851 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f2a1c02f030 con 0x7f2a2c06b520 2024-04-14T08:43:03.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.224+0000 7f2a2a7fc640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a0c06c8f0 0x7f2a0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.224+0000 7f2a2a7fc640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a0c06c8f0 0x7f2a0c06edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0cc390 tx=0x7f2a14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:03.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.225+0000 7f2a347f7740 1 Processor -- start 2024-04-14T08:43:03.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.225+0000 7f2a347f7740 1 -- start start 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a347f7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 0x55c7b84e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a347f7740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8511f40 0x55c7b85148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a347f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b84e5040 0x55c7b8514e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a347f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c7b844e030 con 0x55c7b84e5040 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a347f7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c7b83f6620 con 0x55c7b8511b60 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a347f7740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55c7b8449fe0 con 0x55c7b8511f40 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a2b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b84e5040 0x55c7b8514e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a2b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b84e5040 0x55c7b8514e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59422/0 (socket says 172.21.15.172:59422) 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a2b7fe640 1 -- 172.21.15.172:0/3263155849 learned_addr learned my addr 172.21.15.172:0/3263155849 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a2b7fe640 1 -- 172.21.15.172:0/3263155849 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8511f40 msgr2=0x55c7b85148f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a2b7fe640 1 --2- 172.21.15.172:0/3263155849 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8511f40 0x55c7b85148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a2b7fe640 1 -- 172.21.15.172:0/3263155849 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 msgr2=0x55c7b84e4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:03.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a2b7fe640 1 --2- 172.21.15.172:0/3263155849 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 0x55c7b84e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a2b7fe640 1 -- 172.21.15.172:0/3263155849 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7b81622d0 con 0x55c7b84e5040 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.226+0000 7f2a2b7fe640 1 --2- 172.21.15.172:0/3263155849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b84e5040 0x55c7b8514e30 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2a24001e80 tx=0x7f2a2400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c165289fab0ab378 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.227+0000 7f2a217fa640 1 -- 172.21.15.172:0/3263155849 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400ddd0 con 0x55c7b84e5040 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.227+0000 7f2a217fa640 1 -- 172.21.15.172:0/3263155849 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a24004b60 con 0x55c7b84e5040 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.227+0000 7f2a217fa640 1 -- 172.21.15.172:0/3263155849 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a240070d0 con 0x55c7b84e5040 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.227+0000 7f2a347f7740 1 -- 172.21.15.172:0/3263155849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b84e5040 msgr2=0x55c7b8514e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.227+0000 7f2a347f7740 1 --2- 172.21.15.172:0/3263155849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b84e5040 0x55c7b8514e30 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2a24001e80 tx=0x7f2a2400d520 comp rx=0 tx=0).stop 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.227+0000 7f2a347f7740 1 -- 172.21.15.172:0/3263155849 shutdown_connections 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.227+0000 7f2a347f7740 1 --2- 172.21.15.172:0/3263155849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b84e5040 0x55c7b8514e30 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.227+0000 7f2a347f7740 1 --2- 172.21.15.172:0/3263155849 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8511f40 0x55c7b85148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.227+0000 7f2a347f7740 1 --2- 172.21.15.172:0/3263155849 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 0x55c7b84e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.227+0000 7f2a347f7740 1 -- 172.21.15.172:0/3263155849 >> 172.21.15.172:0/3263155849 conn(0x55c7b84e6d70 msgr2=0x55c7b852f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.227+0000 7f2a347f7740 1 -- 172.21.15.172:0/3263155849 shutdown_connections 2024-04-14T08:43:03.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.227+0000 7f2a347f7740 1 -- 172.21.15.172:0/3263155849 wait complete. 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.228+0000 7f2a347f7740 1 -- start start 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.228+0000 7f2a347f7740 1 Processor -- start 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a347f7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 0x55c7b8515a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a347f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511f40 0x55c7b8540ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a347f7740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8517ea0 0x55c7b8511320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a347f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c7b84e9410 con 0x55c7b8511f40 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a347f7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c7b84099e0 con 0x55c7b8511b60 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a347f7740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55c7b84ebc50 con 0x55c7b8517ea0 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a347f7740 10 client.0 _ll_register_callbacks cb 0x55c7b8446080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a2a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511f40 0x55c7b8540ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a2a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511f40 0x55c7b8540ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59428/0 (socket says 172.21.15.172:59428) 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a2a7fc640 1 -- 172.21.15.172:0/691859882 learned_addr learned my addr 172.21.15.172:0/691859882 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a2affd640 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 0x55c7b8515a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a2b7fe640 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8517ea0 0x55c7b8511320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a2a7fc640 1 -- 172.21.15.172:0/691859882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8517ea0 msgr2=0x55c7b8511320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a2a7fc640 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8517ea0 0x55c7b8511320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a2a7fc640 1 -- 172.21.15.172:0/691859882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 msgr2=0x55c7b8515a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a2a7fc640 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 0x55c7b8515a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a2a7fc640 1 -- 172.21.15.172:0/691859882 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7b822c6b0 con 0x55c7b8511f40 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a2b7fe640 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8517ea0 0x55c7b8511320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a2a7fc640 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511f40 0x55c7b8540ae0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400eaa0 tx=0x7f2a1400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2328bc518224cb6c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a0a7fc640 1 -- 172.21.15.172:0/691859882 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1400f040 con 0x55c7b8511f40 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a0a7fc640 1 -- 172.21.15.172:0/691859882 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f2a14023cb0 con 0x55c7b8511f40 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.229+0000 7f2a347f7740 1 -- 172.21.15.172:0/691859882 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a24004b60 con 0x55c7b8511f40 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a0a7fc640 1 -- 172.21.15.172:0/691859882 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f2a14003ca0 con 0x55c7b8511f40 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a347f7740 10 client.9402 fetch_fsmap learned FSMap version 93 2024-04-14T08:43:03.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a347f7740 1 -- 172.21.15.172:0/691859882 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) -- 0x55c7b8526e20 con 0x55c7b8511f40 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a0a7fc640 1 -- 172.21.15.172:0/691859882 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a24004b60 con 0x55c7b8511f40 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a0a7fc640 1 -- 172.21.15.172:0/691859882 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 93) ==== 3494+0+0 (secure 0 0 0) 0x7f2a1402a050 con 0x55c7b8511f40 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a347f7740 10 client.9402 fetch_fsmap finished waiting for FSMap version 93 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a347f7740 10 client.9402 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a347f7740 20 client.9402 populate_metadata read hostname 'smithi172' 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a347f7740 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55c7b8543da0 0x55c7b854e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a347f7740 4 client.9402 mds_command: new command op to 9048 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a347f7740 1 -- 172.21.15.172:0/691859882 --> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7b83f5cc0 con 0x55c7b8543da0 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a2affd640 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55c7b8543da0 0x55c7b854e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a0a7fc640 10 client.9402 ms_handle_connect on v2:172.21.15.172:6835/744853572 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.230+0000 7f2a2affd640 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55c7b8543da0 0x55c7b854e9a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d8f0764bc28a20 server_cookie=c41560ee1ecb286b in_seq=0 out_seq=0 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a0a7fc640 1 -- 172.21.15.172:0/691859882 <== mds.0 v2:172.21.15.172:6835/744853572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7b83f5cc0 con 0x55c7b8543da0 2024-04-14T08:43:03.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a0a7fc640 10 client.9402 handle_command_reply: tid=0 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a347f7740 1 client.9402 shutdown 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a347f7740 1 -- 172.21.15.172:0/691859882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511f40 msgr2=0x55c7b8540ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a347f7740 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511f40 0x55c7b8540ae0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400eaa0 tx=0x7f2a1400eea0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a347f7740 1 -- 172.21.15.172:0/691859882 shutdown_connections 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a347f7740 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8517ea0 0x55c7b8511320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a347f7740 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55c7b8543da0 0x55c7b854e9a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a347f7740 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511f40 0x55c7b8540ae0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a347f7740 1 --2- 172.21.15.172:0/691859882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 0x55c7b8515a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a0a7fc640 0 client.9402 ms_handle_reset on v2:172.21.15.172:6835/744853572 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a347f7740 1 -- 172.21.15.172:0/691859882 >> 172.21.15.172:0/691859882 conn(0x55c7b84e5040 msgr2=0x55c7b84e8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a347f7740 1 -- 172.21.15.172:0/691859882 shutdown_connections 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a347f7740 1 -- 172.21.15.172:0/691859882 wait complete. 2024-04-14T08:43:03.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.231+0000 7f2a347f7740 20 client.9402 trim_cache size 0 max 16384 2024-04-14T08:43:03.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.254+0000 7f2a347f7740 1 Processor -- start 2024-04-14T08:43:03.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.254+0000 7f2a347f7740 1 -- start start 2024-04-14T08:43:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.255+0000 7f2a347f7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 0x55c7b8514390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.255+0000 7f2a347f7740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8511f40 0x55c7b8528f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.255+0000 7f2a347f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8517ea0 0x55c7b85a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.255+0000 7f2a347f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c1fff60 con 0x55c7b8517ea0 2024-04-14T08:43:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.255+0000 7f2a347f7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c0c9d60 con 0x55c7b8511b60 2024-04-14T08:43:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.255+0000 7f2a347f7740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2c1ff910 con 0x55c7b8511f40 2024-04-14T08:43:03.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a2b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8517ea0 0x55c7b85a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a2b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8517ea0 0x55c7b85a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59432/0 (socket says 172.21.15.172:59432) 2024-04-14T08:43:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a2b7fe640 1 -- 172.21.15.172:0/2736755440 learned_addr learned my addr 172.21.15.172:0/2736755440 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a2affd640 1 --2- 172.21.15.172:0/2736755440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 0x55c7b8514390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a2a7fc640 1 --2- 172.21.15.172:0/2736755440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8511f40 0x55c7b8528f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a2b7fe640 1 -- 172.21.15.172:0/2736755440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8511f40 msgr2=0x55c7b8528f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a2b7fe640 1 --2- 172.21.15.172:0/2736755440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8511f40 0x55c7b8528f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a2b7fe640 1 -- 172.21.15.172:0/2736755440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 msgr2=0x55c7b8514390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a2b7fe640 1 --2- 172.21.15.172:0/2736755440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 0x55c7b8514390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a2b7fe640 1 -- 172.21.15.172:0/2736755440 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7b8511690 con 0x55c7b8517ea0 2024-04-14T08:43:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a2affd640 1 --2- 172.21.15.172:0/2736755440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 0x55c7b8514390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:03.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a2b7fe640 1 --2- 172.21.15.172:0/2736755440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8517ea0 0x55c7b85a0f90 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f2a240012e0 tx=0x7f2a24006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d751b0130d142b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a20ff9640 1 -- 172.21.15.172:0/2736755440 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400dc50 con 0x55c7b8517ea0 2024-04-14T08:43:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a20ff9640 1 -- 172.21.15.172:0/2736755440 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a2400ddf0 con 0x55c7b8517ea0 2024-04-14T08:43:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.256+0000 7f2a20ff9640 1 -- 172.21.15.172:0/2736755440 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24004d60 con 0x55c7b8517ea0 2024-04-14T08:43:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.257+0000 7f2a347f7740 1 -- 172.21.15.172:0/2736755440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8517ea0 msgr2=0x55c7b85a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.257+0000 7f2a347f7740 1 --2- 172.21.15.172:0/2736755440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8517ea0 0x55c7b85a0f90 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f2a240012e0 tx=0x7f2a24006000 comp rx=0 tx=0).stop 2024-04-14T08:43:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.257+0000 7f2a347f7740 1 -- 172.21.15.172:0/2736755440 shutdown_connections 2024-04-14T08:43:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.257+0000 7f2a347f7740 1 --2- 172.21.15.172:0/2736755440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8517ea0 0x55c7b85a0f90 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.257+0000 7f2a347f7740 1 --2- 172.21.15.172:0/2736755440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8511f40 0x55c7b8528f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.257+0000 7f2a347f7740 1 --2- 172.21.15.172:0/2736755440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511b60 0x55c7b8514390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.257+0000 7f2a347f7740 1 -- 172.21.15.172:0/2736755440 >> 172.21.15.172:0/2736755440 conn(0x55c7b84e5040 msgr2=0x55c7b84dfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.257+0000 7f2a347f7740 1 -- 172.21.15.172:0/2736755440 shutdown_connections 2024-04-14T08:43:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.257+0000 7f2a347f7740 1 -- 172.21.15.172:0/2736755440 wait complete. 2024-04-14T08:43:03.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.258+0000 7f2a347f7740 1 -- start start 2024-04-14T08:43:03.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.258+0000 7f2a347f7740 1 Processor -- start 2024-04-14T08:43:03.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.259+0000 7f2a347f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511b60 0x55c7b85a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.259+0000 7f2a347f7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511f40 0x55c7b8525280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.259+0000 7f2a347f7740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8517ea0 0x55c7b85257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.259+0000 7f2a347f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c7b8516060 con 0x55c7b8511b60 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.259+0000 7f2a347f7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c7b8518640 con 0x55c7b8511f40 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.259+0000 7f2a347f7740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55c7b8528c10 con 0x55c7b8517ea0 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.259+0000 7f2a347f7740 10 client.0 _ll_register_callbacks cb 0x7f2a140095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.259+0000 7f2a2affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511b60 0x55c7b85a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.259+0000 7f2a2affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511b60 0x55c7b85a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59438/0 (socket says 172.21.15.172:59438) 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.259+0000 7f2a2affd640 1 -- 172.21.15.172:0/1548277026 learned_addr learned my addr 172.21.15.172:0/1548277026 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.259+0000 7f2a2b7fe640 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8517ea0 0x55c7b85257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.259+0000 7f2a2a7fc640 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511f40 0x55c7b8525280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a2affd640 1 -- 172.21.15.172:0/1548277026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8517ea0 msgr2=0x55c7b85257c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a2affd640 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8517ea0 0x55c7b85257c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a2affd640 1 -- 172.21.15.172:0/1548277026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511f40 msgr2=0x55c7b8525280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a2affd640 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511f40 0x55c7b8525280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a2affd640 1 -- 172.21.15.172:0/1548277026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7b84e0100 con 0x55c7b8511b60 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a2b7fe640 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8517ea0 0x55c7b85257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a2affd640 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511b60 0x55c7b85a00c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c004650 tx=0x7f2a1c02bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a120175f7f3f5292 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a0a7fc640 1 -- 172.21.15.172:0/1548277026 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c0a9020 con 0x55c7b8511b60 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a0a7fc640 1 -- 172.21.15.172:0/1548277026 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f2a1c077df0 con 0x55c7b8511b60 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a347f7740 1 -- 172.21.15.172:0/1548277026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a2400ddf0 con 0x55c7b8511b60 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a0a7fc640 1 -- 172.21.15.172:0/1548277026 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f2a1c07be80 con 0x55c7b8511b60 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a347f7740 10 client.9408 fetch_fsmap learned FSMap version 93 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a347f7740 1 -- 172.21.15.172:0/1548277026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) -- 0x55c7b85aa9e0 con 0x55c7b8511b60 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a0a7fc640 1 -- 172.21.15.172:0/1548277026 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a2400ddf0 con 0x55c7b8511b60 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a0a7fc640 1 -- 172.21.15.172:0/1548277026 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 93) ==== 3494+0+0 (secure 0 0 0) 0x7f2a1c031070 con 0x55c7b8511b60 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a347f7740 10 client.9408 fetch_fsmap finished waiting for FSMap version 93 2024-04-14T08:43:03.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a347f7740 10 client.9408 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a347f7740 20 client.9408 populate_metadata read hostname 'smithi172' 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a347f7740 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55c7b853df70 0x55c7b853e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a347f7740 4 client.9408 mds_command: new command op to 9048 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a347f7740 1 -- 172.21.15.172:0/1548277026 --> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] -- command(tid 0: {"prefix": "status"}) -- 0x55c7b845b570 con 0x55c7b853df70 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.260+0000 7f2a2a7fc640 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55c7b853df70 0x55c7b853e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a2a7fc640 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55c7b853df70 0x55c7b853e350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59990dd5388a41e server_cookie=1520da0b6b134126 in_seq=0 out_seq=0 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a0a7fc640 10 client.9408 ms_handle_connect on v2:172.21.15.172:6835/744853572 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a0a7fc640 1 -- 172.21.15.172:0/1548277026 <== mds.0 v2:172.21.15.172:6835/744853572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55c7b845b570 con 0x55c7b853df70 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a0a7fc640 10 client.9408 handle_command_reply: tid=0 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a347f7740 1 client.9408 shutdown 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a347f7740 1 -- 172.21.15.172:0/1548277026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511b60 msgr2=0x55c7b85a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a347f7740 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511b60 0x55c7b85a00c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c004650 tx=0x7f2a1c02bd60 comp rx=0 tx=0).stop 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a347f7740 1 -- 172.21.15.172:0/1548277026 shutdown_connections 2024-04-14T08:43:03.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a347f7740 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c7b8517ea0 0x55c7b85257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a347f7740 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7b8511f40 0x55c7b8525280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a347f7740 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55c7b853df70 0x55c7b853e350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a347f7740 1 --2- 172.21.15.172:0/1548277026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7b8511b60 0x55c7b85a00c0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a0a7fc640 0 client.9408 ms_handle_reset on v2:172.21.15.172:6835/744853572 2024-04-14T08:43:03.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a347f7740 1 -- 172.21.15.172:0/1548277026 >> 172.21.15.172:0/1548277026 conn(0x55c7b84e5040 msgr2=0x55c7b8532d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:03.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a347f7740 1 -- 172.21.15.172:0/1548277026 shutdown_connections 2024-04-14T08:43:03.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a347f7740 1 -- 172.21.15.172:0/1548277026 wait complete. 2024-04-14T08:43:03.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.261+0000 7f2a347f7740 20 client.9408 trim_cache size 0 max 16384 2024-04-14T08:43:03.268 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:43:03.268 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:43:03.268 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:43:03.269 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 9048, 2024-04-14T08:43:03.269 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:43:03.269 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:43:03.269 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:43:03.269 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 4.4335537839999999, 2024-04-14T08:43:03.269 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 93, 2024-04-14T08:43:03.269 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 117, 2024-04-14T08:43:03.269 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 117, 2024-04-14T08:43:03.269 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 27.424747333999999 2024-04-14T08:43:03.269 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:43:03.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.262+0000 7f2a31a0f640 1 -- 172.21.15.172:0/1864384851 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a0c06c8f0 msgr2=0x7f2a0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.262+0000 7f2a31a0f640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a0c06c8f0 0x7f2a0c06edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0cc390 tx=0x7f2a14008040 comp rx=0 tx=0).stop 2024-04-14T08:43:03.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.262+0000 7f2a31a0f640 1 -- 172.21.15.172:0/1864384851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a2c06b520 msgr2=0x7f2a2c15f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.262+0000 7f2a31a0f640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a2c06b520 0x7f2a2c15f120 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c034040 tx=0x7f2a1c0046c0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.262+0000 7f2a31a0f640 1 -- 172.21.15.172:0/1864384851 shutdown_connections 2024-04-14T08:43:03.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.262+0000 7f2a31a0f640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2a0c06c8f0 0x7f2a0c06edb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.262+0000 7f2a31a0f640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a2c0ccca0 0x7f2a2c0b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.262+0000 7f2a31a0f640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a2c073090 0x7f2a2c0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.262+0000 7f2a31a0f640 1 --2- 172.21.15.172:0/1864384851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a2c06b520 0x7f2a2c15f120 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.262+0000 7f2a31a0f640 1 -- 172.21.15.172:0/1864384851 >> 172.21.15.172:0/1864384851 conn(0x7f2a2c0cda00 msgr2=0x7f2a2c161410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:03.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.262+0000 7f2a31a0f640 1 -- 172.21.15.172:0/1864384851 shutdown_connections 2024-04-14T08:43:03.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.262+0000 7f2a31a0f640 1 -- 172.21.15.172:0/1864384851 wait complete. 2024-04-14T08:43:03.273 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:43:03.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.375+0000 7f3b49480640 1 Processor -- start 2024-04-14T08:43:03.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.375+0000 7f3b49480640 1 -- start start 2024-04-14T08:43:03.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b49480640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3b440b7a60 0x7f3b440ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b49480640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3b4406dc40 0x7f3b440cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b49480640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b440d2b10 con 0x7f3b4408c4c0 2024-04-14T08:43:03.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b49480640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f3b440d0ad0 con 0x7f3b440b7a60 2024-04-14T08:43:03.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b49480640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f3b440d0db0 con 0x7f3b4406dc40 2024-04-14T08:43:03.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b42ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3b440b7a60 0x7f3b440ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b427fc640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3b4408c4c0 0x7f3b440cd410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45794/0 (socket says 172.21.15.172:45794) 2024-04-14T08:43:03.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b427fc640 1 -- 172.21.15.172:0/253966023 learned_addr learned my addr 172.21.15.172:0/253966023 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:03.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b437fe640 1 --2- 172.21.15.172:0/253966023 >> v2:172.21.15.177:3300/0 conn(0x7f3b4406dc40 0x7f3b440cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b42ffd640 1 -- 172.21.15.172:0/253966023 >> v2:172.21.15.177:3300/0 conn(0x7f3b4406dc40 msgr2=0x7f3b440cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:03.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b42ffd640 1 --2- 172.21.15.172:0/253966023 >> v2:172.21.15.177:3300/0 conn(0x7f3b4406dc40 0x7f3b440cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b42ffd640 1 -- 172.21.15.172:0/253966023 >> v1:172.21.15.172:6789/0 conn(0x7f3b4408c4c0 legacy=0x7f3b440cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b42ffd640 1 -- 172.21.15.172:0/253966023 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b440d1090 con 0x7f3b440b7a60 2024-04-14T08:43:03.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.376+0000 7f3b41ffb640 1 -- 172.21.15.172:0/253966023 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2371073040 0 0) 0x7f3b440d2b10 con 0x7f3b4408c4c0 2024-04-14T08:43:03.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.377+0000 7f3b437fe640 1 --2- 172.21.15.172:0/253966023 >> v2:172.21.15.177:3300/0 conn(0x7f3b4406dc40 0x7f3b440cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:03.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.377+0000 7f3b42ffd640 1 --2- 172.21.15.172:0/253966023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b440b7a60 0x7f3b440ce8c0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3b440732d0 tx=0x7f3b3402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea04d7541f99fc76 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:03.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.377+0000 7f3b41ffb640 1 -- 172.21.15.172:0/253966023 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340315a0 con 0x7f3b440b7a60 2024-04-14T08:43:03.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.377+0000 7f3b41ffb640 1 -- 172.21.15.172:0/253966023 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b34037070 con 0x7f3b440b7a60 2024-04-14T08:43:03.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.377+0000 7f3b41ffb640 1 -- 172.21.15.172:0/253966023 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34031960 con 0x7f3b440b7a60 2024-04-14T08:43:03.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.378+0000 7f3b49480640 1 -- 172.21.15.172:0/253966023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b440b7a60 msgr2=0x7f3b440ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.378+0000 7f3b49480640 1 --2- 172.21.15.172:0/253966023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b440b7a60 0x7f3b440ce8c0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3b440732d0 tx=0x7f3b3402b840 comp rx=0 tx=0).stop 2024-04-14T08:43:03.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.378+0000 7f3b49480640 1 -- 172.21.15.172:0/253966023 shutdown_connections 2024-04-14T08:43:03.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.378+0000 7f3b49480640 1 --2- 172.21.15.172:0/253966023 >> v2:172.21.15.177:3300/0 conn(0x7f3b4406dc40 0x7f3b440cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.378+0000 7f3b49480640 1 --2- 172.21.15.172:0/253966023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b440b7a60 0x7f3b440ce8c0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.378+0000 7f3b49480640 1 -- 172.21.15.172:0/253966023 >> 172.21.15.172:0/253966023 conn(0x7f3b4408c000 msgr2=0x7f3b440d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:03.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.378+0000 7f3b49480640 1 -- 172.21.15.172:0/253966023 shutdown_connections 2024-04-14T08:43:03.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.378+0000 7f3b49480640 1 -- 172.21.15.172:0/253966023 wait complete. 2024-04-14T08:43:03.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.379+0000 7f3b49480640 1 Processor -- start 2024-04-14T08:43:03.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.379+0000 7f3b49480640 1 -- start start 2024-04-14T08:43:03.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b49480640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b4406dc40 0x7f3b440b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b49480640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b4408c4c0 0x7f3b440b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b49480640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b440b7a60 0x7f3b44148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b49480640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3b440d2750 con 0x7f3b4408c4c0 2024-04-14T08:43:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b49480640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3b440b7e40 con 0x7f3b440b7a60 2024-04-14T08:43:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b49480640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3b440d28d0 con 0x7f3b4406dc40 2024-04-14T08:43:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b427fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b4408c4c0 0x7f3b440b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b427fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b4408c4c0 0x7f3b440b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59454/0 (socket says 172.21.15.172:59454) 2024-04-14T08:43:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b427fc640 1 -- 172.21.15.172:0/266922727 learned_addr learned my addr 172.21.15.172:0/266922727 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b42ffd640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b4406dc40 0x7f3b440b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b437fe640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b440b7a60 0x7f3b44148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b427fc640 1 -- 172.21.15.172:0/266922727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b4406dc40 msgr2=0x7f3b440b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b427fc640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b4406dc40 0x7f3b440b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b427fc640 1 -- 172.21.15.172:0/266922727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b440b7a60 msgr2=0x7f3b44148900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b427fc640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b440b7a60 0x7f3b44148900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b427fc640 1 -- 172.21.15.172:0/266922727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b44159c20 con 0x7f3b4408c4c0 2024-04-14T08:43:03.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.380+0000 7f3b42ffd640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b4406dc40 0x7f3b440b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:03.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.381+0000 7f3b427fc640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b4408c4c0 0x7f3b440b4000 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800c4d0 tx=0x7f3b3800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:03.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.381+0000 7f3b2b7fe640 1 -- 172.21.15.172:0/266922727 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38018070 con 0x7f3b4408c4c0 2024-04-14T08:43:03.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.381+0000 7f3b2b7fe640 1 -- 172.21.15.172:0/266922727 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b38007500 con 0x7f3b4408c4c0 2024-04-14T08:43:03.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.381+0000 7f3b49480640 1 -- 172.21.15.172:0/266922727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b441f4550 con 0x7f3b4408c4c0 2024-04-14T08:43:03.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.381+0000 7f3b2b7fe640 1 -- 172.21.15.172:0/266922727 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38004a20 con 0x7f3b4408c4c0 2024-04-14T08:43:03.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.381+0000 7f3b49480640 1 -- 172.21.15.172:0/266922727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b441f4b10 con 0x7f3b4408c4c0 2024-04-14T08:43:03.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.382+0000 7f3b49480640 1 -- 172.21.15.172:0/266922727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b441485d0 con 0x7f3b4408c4c0 2024-04-14T08:43:03.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.382+0000 7f3b2b7fe640 1 -- 172.21.15.172:0/266922727 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b38004bc0 con 0x7f3b4408c4c0 2024-04-14T08:43:03.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.383+0000 7f3b2b7fe640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3b3006c8f0 0x7f3b3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:03.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.383+0000 7f3b42ffd640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3b3006c8f0 0x7f3b3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:03.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.383+0000 7f3b2b7fe640 1 -- 172.21.15.172:0/266922727 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f3b3808afc0 con 0x7f3b4408c4c0 2024-04-14T08:43:03.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.383+0000 7f3b42ffd640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3b3006c8f0 0x7f3b3006edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3b34036000 tx=0x7f3b34008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:03.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.387+0000 7f3b2b7fe640 1 -- 172.21.15.172:0/266922727 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3b44159c20 con 0x7f3b4408c4c0 2024-04-14T08:43:03.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.606+0000 7f3b49480640 1 -- 172.21.15.172:0/266922727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3b4406e120 con 0x7f3b4408c4c0 2024-04-14T08:43:03.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.606+0000 7f3b2b7fe640 1 -- 172.21.15.172:0/266922727 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v93) ==== 53+0+73 (secure 0 0 0) 0x7f3b38059220 con 0x7f3b4408c4c0 2024-04-14T08:43:03.607 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:43:03.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.609+0000 7f3b49480640 1 -- 172.21.15.172:0/266922727 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3b3006c8f0 msgr2=0x7f3b3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.609+0000 7f3b49480640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3b3006c8f0 0x7f3b3006edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3b34036000 tx=0x7f3b34008030 comp rx=0 tx=0).stop 2024-04-14T08:43:03.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.609+0000 7f3b49480640 1 -- 172.21.15.172:0/266922727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b4408c4c0 msgr2=0x7f3b440b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:03.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.609+0000 7f3b49480640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b4408c4c0 0x7f3b440b4000 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800c4d0 tx=0x7f3b3800c990 comp rx=0 tx=0).stop 2024-04-14T08:43:03.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.609+0000 7f3b49480640 1 -- 172.21.15.172:0/266922727 shutdown_connections 2024-04-14T08:43:03.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.609+0000 7f3b49480640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3b3006c8f0 0x7f3b3006edb0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.609+0000 7f3b49480640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b440b7a60 0x7f3b44148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.609+0000 7f3b49480640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b4408c4c0 0x7f3b440b4000 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.609+0000 7f3b49480640 1 --2- 172.21.15.172:0/266922727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b4406dc40 0x7f3b440b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:03.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.609+0000 7f3b49480640 1 -- 172.21.15.172:0/266922727 >> 172.21.15.172:0/266922727 conn(0x7f3b4408c000 msgr2=0x7f3b440cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:03.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.609+0000 7f3b49480640 1 -- 172.21.15.172:0/266922727 shutdown_connections 2024-04-14T08:43:03.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03.609+0000 7f3b49480640 1 -- 172.21.15.172:0/266922727 wait complete. 2024-04-14T08:43:03.620 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:43:03.620 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:43:03.620 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:43:03.621 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:43:03.621 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:43:03.621 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:43:03.621 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:43:03.621 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:43:03.621 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:43:03.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:03.621 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:43:03.636 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:43:03.637 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42086sec preferred_lft 42086sec 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:43:03.638 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:43:03.639 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:43:03.639 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:03.639 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:43:03.639 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:43:03.639 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:43:03.639 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:03.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:03.639 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:43:03.703 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:43:03.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:03.704 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:43:03.761 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:43:03.761 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:43:03.761 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:43:03.761 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:43:03.761 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:43:03.762 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:43:03.762 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:43:03.762 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:43:03.762 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:43:03.762 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:43:03.762 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:43:03.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:43:03.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:03.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:03.904 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:43:03.919 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:43:03.919 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:43:03.920 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:43:03.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:03.920 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:43:03.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:43:03.987 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:43:04.042 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:43:04.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:43:04.043 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:43:04.096 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:43:04.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:04.096 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:43:04.180 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:43:04.181 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:43:04.181 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:43:04.189 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:43:04.189 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:43:04.189 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:43:04.189 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:43:04.189 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:43:04.194 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:43:04.194 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:43:04.194 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:43:04.194 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:43:04.194 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:43:04.195 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:43:04.195 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:43:04.195 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:43:04.197 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:43:04.197 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:43:04.197 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:43:04.197 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:43:04.197 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:43:04.198 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:43:04.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:04.198 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:43:04.271 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:43:04.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:43:04.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:04.366 INFO:teuthology.orchestra.run.smithi172.stdout:client9420 2024-04-14T08:43:04.367 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:43:04.367 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:04.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:04.383 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-14T08:43:04.442 INFO:teuthology.orchestra.run.smithi172.stdout:./ 2024-04-14T08:43:04.443 INFO:teuthology.orchestra.run.smithi172.stdout:./parent 2024-04-14T08:43:04.443 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/flushed 2024-04-14T08:43:04.443 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/flushed/charlie 2024-04-14T08:43:04.443 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/flushed/alpha 2024-04-14T08:43:04.443 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/unflushed 2024-04-14T08:43:04.443 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/unflushed/jfile 2024-04-14T08:43:04.444 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:04.444 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:04.444 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:04.444 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:04.444 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-14T08:43:04.445 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:04.567 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:43:04.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:04.576 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:04.576 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:04.576 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:04.576 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:04.576 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-14T08:43:04.576 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:04.681 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627776 2024-04-14T08:43:04.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:04.690 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:04.690 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:04.690 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:04.691 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:04.691 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-14T08:43:04.691 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:04.796 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627777 2024-04-14T08:43:04.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:04.805 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:04.806 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:04.806 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:04.806 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:04.806 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-14T08:43:04.806 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:04.913 INFO:teuthology.orchestra.run.smithi172.stdout:1099511628280 2024-04-14T08:43:04.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:04.924 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:04.924 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:04.924 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:04.924 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:04.924 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-14T08:43:04.924 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:05.029 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627778 2024-04-14T08:43:05.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:05.039 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:05.039 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:05.039 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:05.039 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:05.039 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-14T08:43:05.039 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:05.144 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627779 2024-04-14T08:43:05.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:05.153 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:05.153 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:05.153 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:05.153 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:05.153 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-14T08:43:05.153 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:05.283 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627780 2024-04-14T08:43:05.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:05.294 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:43:05.294 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:05.350 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:43:05.350 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:05.406 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:43:05.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.546+0000 7fc2a155e640 1 Processor -- start 2024-04-14T08:43:05.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.546+0000 7fc2a155e640 1 -- start start 2024-04-14T08:43:05.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.547+0000 7fc2a155e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc29c15e890 0x7fc29c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.547+0000 7fc2a155e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29c0cedd0 0x7fc29c0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.547+0000 7fc2a155e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc29c0cf320 con 0x7fc29c0d2830 2024-04-14T08:43:05.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.547+0000 7fc2a155e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc29c0cff30 con 0x7fc29c15e890 2024-04-14T08:43:05.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.547+0000 7fc2a155e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc29c0d2350 con 0x7fc29c0cedd0 2024-04-14T08:43:05.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.548+0000 7fc29b7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29c0cedd0 0x7fc29c0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.548+0000 7fc293fff640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc29c15e890 0x7fc29c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.548+0000 7fc29affd640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fc29c0d2830 0x7fc29c0d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:39648/0 (socket says 172.21.15.172:39648) 2024-04-14T08:43:05.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.548+0000 7fc29affd640 1 -- 172.21.15.172:0/1404573063 learned_addr learned my addr 172.21.15.172:0/1404573063 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:05.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.548+0000 7fc293fff640 1 -- 172.21.15.172:0/1404573063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29c0cedd0 msgr2=0x7fc29c0b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:05.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.548+0000 7fc293fff640 1 --2- 172.21.15.172:0/1404573063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29c0cedd0 0x7fc29c0b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.548+0000 7fc293fff640 1 -- 172.21.15.172:0/1404573063 >> v1:172.21.15.177:6789/0 conn(0x7fc29c0d2830 legacy=0x7fc29c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:05.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.548+0000 7fc293fff640 1 -- 172.21.15.172:0/1404573063 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c0cf600 con 0x7fc29c15e890 2024-04-14T08:43:05.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.548+0000 7fc29b7fe640 1 --2- 172.21.15.172:0/1404573063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29c0cedd0 0x7fc29c0b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:43:05.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.549+0000 7fc293fff640 1 --2- 172.21.15.172:0/1404573063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29c15e890 0x7fc29c0d1300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc2840097a0 tx=0x7fc28402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2260c657f578c25a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:05.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.549+0000 7fc29a7fc640 1 -- 172.21.15.172:0/1404573063 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc284034070 con 0x7fc29c15e890 2024-04-14T08:43:05.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.549+0000 7fc29a7fc640 1 -- 172.21.15.172:0/1404573063 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc284030610 con 0x7fc29c15e890 2024-04-14T08:43:05.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.549+0000 7fc29a7fc640 1 -- 172.21.15.172:0/1404573063 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2840309e0 con 0x7fc29c15e890 2024-04-14T08:43:05.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.550+0000 7fc2a155e640 1 -- 172.21.15.172:0/1404573063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29c15e890 msgr2=0x7fc29c0d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:05.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.550+0000 7fc2a155e640 1 --2- 172.21.15.172:0/1404573063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29c15e890 0x7fc29c0d1300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc2840097a0 tx=0x7fc28402b960 comp rx=0 tx=0).stop 2024-04-14T08:43:05.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.550+0000 7fc2a155e640 1 -- 172.21.15.172:0/1404573063 shutdown_connections 2024-04-14T08:43:05.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.550+0000 7fc2a155e640 1 --2- 172.21.15.172:0/1404573063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29c0cedd0 0x7fc29c0b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.550+0000 7fc2a155e640 1 --2- 172.21.15.172:0/1404573063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29c15e890 0x7fc29c0d1300 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.550+0000 7fc2a155e640 1 -- 172.21.15.172:0/1404573063 >> 172.21.15.172:0/1404573063 conn(0x7fc29c09d590 msgr2=0x7fc29c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:05.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.550+0000 7fc2a155e640 1 -- 172.21.15.172:0/1404573063 shutdown_connections 2024-04-14T08:43:05.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.550+0000 7fc2a155e640 1 -- 172.21.15.172:0/1404573063 wait complete. 2024-04-14T08:43:05.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.551+0000 7fc2a155e640 1 Processor -- start 2024-04-14T08:43:05.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.552+0000 7fc2a155e640 1 -- start start 2024-04-14T08:43:05.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc2a155e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29c0cedd0 0x7fc29c157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc2a155e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29c0d2830 0x7fc29c0c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc2a155e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc29c15e890 0x7fc29c0ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc2a155e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c0d1840 con 0x7fc29c15e890 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc2a155e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c0cfbc0 con 0x7fc29c0d2830 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc2a155e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc29c0b3c50 con 0x7fc29c0cedd0 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc29b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc29c15e890 0x7fc29c0ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc29b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc29c15e890 0x7fc29c0ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59464/0 (socket says 172.21.15.172:59464) 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc29b7fe640 1 -- 172.21.15.172:0/507032054 learned_addr learned my addr 172.21.15.172:0/507032054 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc29affd640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29c0cedd0 0x7fc29c157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc293fff640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29c0d2830 0x7fc29c0c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc29b7fe640 1 -- 172.21.15.172:0/507032054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29c0cedd0 msgr2=0x7fc29c157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc29b7fe640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29c0cedd0 0x7fc29c157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc29b7fe640 1 -- 172.21.15.172:0/507032054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29c0d2830 msgr2=0x7fc29c0c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc29b7fe640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29c0d2830 0x7fc29c0c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc29b7fe640 1 -- 172.21.15.172:0/507032054 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c0c9310 con 0x7fc29c15e890 2024-04-14T08:43:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.553+0000 7fc29b7fe640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc29c15e890 0x7fc29c0ca140 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fc29400efc0 tx=0x7fc294009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:05.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.554+0000 7fc2937fe640 1 -- 172.21.15.172:0/507032054 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294018030 con 0x7fc29c15e890 2024-04-14T08:43:05.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.554+0000 7fc2937fe640 1 -- 172.21.15.172:0/507032054 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc29400cde0 con 0x7fc29c15e890 2024-04-14T08:43:05.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.554+0000 7fc2a155e640 1 -- 172.21.15.172:0/507032054 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29c0c95f0 con 0x7fc29c15e890 2024-04-14T08:43:05.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.554+0000 7fc2937fe640 1 -- 172.21.15.172:0/507032054 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294003bb0 con 0x7fc29c15e890 2024-04-14T08:43:05.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.554+0000 7fc2a155e640 1 -- 172.21.15.172:0/507032054 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29c15f6a0 con 0x7fc29c15e890 2024-04-14T08:43:05.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.555+0000 7fc2a155e640 1 -- 172.21.15.172:0/507032054 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29c0c97c0 con 0x7fc29c15e890 2024-04-14T08:43:05.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.555+0000 7fc2937fe640 1 -- 172.21.15.172:0/507032054 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc29401b070 con 0x7fc29c15e890 2024-04-14T08:43:05.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.556+0000 7fc2937fe640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc27c06c920 0x7fc27c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.556+0000 7fc29affd640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc27c06c920 0x7fc27c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.556+0000 7fc2937fe640 1 -- 172.21.15.172:0/507032054 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7fc29408c670 con 0x7fc29c15e890 2024-04-14T08:43:05.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.556+0000 7fc29affd640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc27c06c920 0x7fc27c06ede0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0a2110 tx=0x7fc28c007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:05.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.560+0000 7fc2937fe640 1 -- 172.21.15.172:0/507032054 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc294010040 con 0x7fc29c15e890 2024-04-14T08:43:05.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.737+0000 7fc2a155e640 1 -- 172.21.15.172:0/507032054 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc29c0ce690 con 0x7fc29c15e890 2024-04-14T08:43:05.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.739+0000 7fc2937fe640 1 -- 172.21.15.172:0/507032054 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+17143 (secure 0 0 0) 0x7fc29405a270 con 0x7fc29c15e890 2024-04-14T08:43:05.739 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:05.739 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":117,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:42:57.915880+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:42:38.455900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-14T08:42:39.198157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/3201353747":"2024-04-15T08:42:57.915856+0000","172.21.15.177:6833/3201353747":"2024-04-15T08:42:57.915856+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:43:05.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.741+0000 7fc2a155e640 1 -- 172.21.15.172:0/507032054 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc27c06c920 msgr2=0x7fc27c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:05.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.741+0000 7fc2a155e640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc27c06c920 0x7fc27c06ede0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0a2110 tx=0x7fc28c007e80 comp rx=0 tx=0).stop 2024-04-14T08:43:05.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.741+0000 7fc2a155e640 1 -- 172.21.15.172:0/507032054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc29c15e890 msgr2=0x7fc29c0ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:05.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.741+0000 7fc2a155e640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc29c15e890 0x7fc29c0ca140 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fc29400efc0 tx=0x7fc294009f90 comp rx=0 tx=0).stop 2024-04-14T08:43:05.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.741+0000 7fc2a155e640 1 -- 172.21.15.172:0/507032054 shutdown_connections 2024-04-14T08:43:05.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.741+0000 7fc2a155e640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc27c06c920 0x7fc27c06ede0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.741+0000 7fc2a155e640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc29c15e890 0x7fc29c0ca140 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.741+0000 7fc2a155e640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29c0d2830 0x7fc29c0c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.741+0000 7fc2a155e640 1 --2- 172.21.15.172:0/507032054 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29c0cedd0 0x7fc29c157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.741+0000 7fc2a155e640 1 -- 172.21.15.172:0/507032054 >> 172.21.15.172:0/507032054 conn(0x7fc29c09d590 msgr2=0x7fc29c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:05.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.741+0000 7fc2a155e640 1 -- 172.21.15.172:0/507032054 shutdown_connections 2024-04-14T08:43:05.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.741+0000 7fc2a155e640 1 -- 172.21.15.172:0/507032054 wait complete. 2024-04-14T08:43:05.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:42:38.455900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:42:39.198157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:43:05.753 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:43:05.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.859+0000 7f1cd5036640 1 Processor -- start 2024-04-14T08:43:05.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.859+0000 7f1cd5036640 1 -- start start 2024-04-14T08:43:05.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.860+0000 7f1cd5036640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd006d710 0x7f1cd006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.860+0000 7f1cd5036640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd00701b0 con 0x7f1cd0071250 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.860+0000 7f1cd5036640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd0070490 con 0x7f1cd00d1a20 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.860+0000 7f1cd5036640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd0070dc0 con 0x7f1cd006d710 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1cce59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1cd0071250 0x7f1cd006d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45800/0 (socket says 172.21.15.172:45800) 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1cce59c640 1 -- 172.21.15.172:0/1473080353 learned_addr learned my addr 172.21.15.172:0/1473080353 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3164372960 0 0) 0x7f1cd00701b0 con 0x7f1cd0071250 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccf59e640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd006d710 0x7f1cd006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cbc003520 con 0x7f1cd0071250 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2555953639 0 0) 0x7f1cbc003520 con 0x7f1cd0071250 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd006d710 msgr2=0x7f1cd006fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd006d710 0x7f1cd006fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 >> v1:172.21.15.177:6789/0 conn(0x7f1cd00d1a20 legacy=0x7f1cd00cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd0070770 con 0x7f1cd0071250 2024-04-14T08:43:05.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f1cb8001670 con 0x7f1cd0071250 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 >> v1:172.21.15.172:6789/0 conn(0x7f1cd0071250 legacy=0x7f1cd006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cbc0039e0 0x7f1cbc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd006d710 0x7f1cbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cbc004d40 0x7f1cbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0070dc0 con 0x7f1cbc0039e0 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc00baf0 con 0x7f1cbc004d40 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc00bcf0 con 0x7f1cd006d710 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1cced9d640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cbc0039e0 0x7f1cbc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1cce59c640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd006d710 0x7f1cbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.861+0000 7f1ccf59e640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cbc004d40 0x7f1cbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.862+0000 7f1cced9d640 1 -- 172.21.15.172:0/1473080353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd006d710 msgr2=0x7f1cbc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.862+0000 7f1cced9d640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd006d710 0x7f1cbc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.862+0000 7f1cced9d640 1 -- 172.21.15.172:0/1473080353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cbc004d40 msgr2=0x7f1cbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:05.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.862+0000 7f1cced9d640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cbc004d40 0x7f1cbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.862+0000 7f1cced9d640 1 -- 172.21.15.172:0/1473080353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc003520 con 0x7f1cbc0039e0 2024-04-14T08:43:05.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.862+0000 7f1cce59c640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd006d710 0x7f1cbc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:05.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.862+0000 7f1cced9d640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cbc0039e0 0x7f1cbc003e00 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8001e50 tx=0x7f1cbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5755603fef7132 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:05.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.862+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8001a70 con 0x7f1cbc0039e0 2024-04-14T08:43:05.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.862+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc8003b70 con 0x7f1cbc0039e0 2024-04-14T08:43:05.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.862+0000 7f1ccdd9b640 1 -- 172.21.15.172:0/1473080353 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80050d0 con 0x7f1cbc0039e0 2024-04-14T08:43:05.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.863+0000 7f1cd5036640 1 -- 172.21.15.172:0/1473080353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cbc0039e0 msgr2=0x7f1cbc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:05.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.863+0000 7f1cd5036640 1 --2- 172.21.15.172:0/1473080353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cbc0039e0 0x7f1cbc003e00 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8001e50 tx=0x7f1cbc005830 comp rx=0 tx=0).stop 2024-04-14T08:43:05.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.863+0000 7f1ccf59e640 1 -- 172.21.15.172:0/1473080353 reap_dead start 2024-04-14T08:43:05.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.863+0000 7f1cd5036640 1 -- 172.21.15.172:0/1473080353 shutdown_connections 2024-04-14T08:43:05.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.863+0000 7f1cd5036640 1 -- 172.21.15.172:0/1473080353 >> 172.21.15.172:0/1473080353 conn(0x7f1cd008c000 msgr2=0x7f1cd00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:05.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.863+0000 7f1cd5036640 1 -- 172.21.15.172:0/1473080353 shutdown_connections 2024-04-14T08:43:05.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.863+0000 7f1cd5036640 1 -- 172.21.15.172:0/1473080353 wait complete. 2024-04-14T08:43:05.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.864+0000 7f1cd5036640 1 Processor -- start 2024-04-14T08:43:05.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.865+0000 7f1cd5036640 1 -- start start 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.865+0000 7f1cd5036640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cbc0039e0 0x7f1cd014e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.865+0000 7f1cd5036640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd01f3fb0 0x7f1cd01f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.865+0000 7f1cd5036640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cd00ca740 0x7f1cd00cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.865+0000 7f1cd5036640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd00d1e00 con 0x7f1cd00ca740 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.865+0000 7f1cd5036640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd006fb80 con 0x7f1cd01f3fb0 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.865+0000 7f1cd5036640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd01f4ff0 con 0x7f1cbc0039e0 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.865+0000 7f1ccf59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cd00ca740 0x7f1cd00cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.865+0000 7f1ccf59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cd00ca740 0x7f1cd00cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59486/0 (socket says 172.21.15.172:59486) 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.865+0000 7f1ccf59e640 1 -- 172.21.15.172:0/507939238 learned_addr learned my addr 172.21.15.172:0/507939238 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.865+0000 7f1cce59c640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd01f3fb0 0x7f1cd01f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.865+0000 7f1cced9d640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cbc0039e0 0x7f1cd014e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.866+0000 7f1ccf59e640 1 -- 172.21.15.172:0/507939238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cbc0039e0 msgr2=0x7f1cd014e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:05.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.866+0000 7f1ccf59e640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cbc0039e0 0x7f1cd014e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.866+0000 7f1ccf59e640 1 -- 172.21.15.172:0/507939238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd01f3fb0 msgr2=0x7f1cd01f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.866+0000 7f1ccf59e640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd01f3fb0 0x7f1cd01f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.866+0000 7f1ccf59e640 1 -- 172.21.15.172:0/507939238 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd01f4a30 con 0x7f1cd00ca740 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.866+0000 7f1ccf59e640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cd00ca740 0x7f1cd00cabc0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f1cd006db20 tx=0x7f1cc0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.866+0000 7f1cc77fe640 1 -- 172.21.15.172:0/507939238 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0019020 con 0x7f1cd00ca740 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.866+0000 7f1cc77fe640 1 -- 172.21.15.172:0/507939238 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc0003610 con 0x7f1cd00ca740 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.866+0000 7f1cd5036640 1 -- 172.21.15.172:0/507939238 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd01f4d10 con 0x7f1cd00ca740 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.866+0000 7f1cc77fe640 1 -- 172.21.15.172:0/507939238 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0003a00 con 0x7f1cd00ca740 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.866+0000 7f1cd5036640 1 -- 172.21.15.172:0/507939238 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd0159f10 con 0x7f1cd00ca740 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.867+0000 7f1cd5036640 1 -- 172.21.15.172:0/507939238 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd015a110 con 0x7f1cd00ca740 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.867+0000 7f1cc77fe640 1 -- 172.21.15.172:0/507939238 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cc001c020 con 0x7f1cd00ca740 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.868+0000 7f1cc77fe640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1cb006c920 0x7f1cb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.868+0000 7f1cc77fe640 1 -- 172.21.15.172:0/507939238 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f1cc008bc40 con 0x7f1cd00ca740 2024-04-14T08:43:05.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.868+0000 7f1cced9d640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1cb006c920 0x7f1cb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:05.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.868+0000 7f1cced9d640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1cb006c920 0x7f1cb006ede0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8034000 tx=0x7f1cc803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:05.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:05.872+0000 7f1cc77fe640 1 -- 172.21.15.172:0/507939238 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1cd015a110 con 0x7f1cd00ca740 2024-04-14T08:43:06.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.051+0000 7f1cd5036640 1 -- 172.21.15.172:0/507939238 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1cd0075490 con 0x7f1cd00ca740 2024-04-14T08:43:06.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.051+0000 7f1cc77fe640 1 -- 172.21.15.172:0/507939238 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) ==== 93+0+10 (secure 0 0 0) 0x7f1cc0023030 con 0x7f1cd00ca740 2024-04-14T08:43:06.052 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:43:06.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.053+0000 7f1cd5036640 1 -- 172.21.15.172:0/507939238 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1cb006c920 msgr2=0x7f1cb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.053+0000 7f1cd5036640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1cb006c920 0x7f1cb006ede0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8034000 tx=0x7f1cc803d040 comp rx=0 tx=0).stop 2024-04-14T08:43:06.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.053+0000 7f1cd5036640 1 -- 172.21.15.172:0/507939238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cd00ca740 msgr2=0x7f1cd00cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.053+0000 7f1cd5036640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cd00ca740 0x7f1cd00cabc0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f1cd006db20 tx=0x7f1cc0011920 comp rx=0 tx=0).stop 2024-04-14T08:43:06.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.053+0000 7f1cd5036640 1 -- 172.21.15.172:0/507939238 shutdown_connections 2024-04-14T08:43:06.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.053+0000 7f1cd5036640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f1cb006c920 0x7f1cb006ede0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.053+0000 7f1cd5036640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cd00ca740 0x7f1cd00cabc0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.053+0000 7f1cd5036640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd01f3fb0 0x7f1cd01f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.053+0000 7f1cd5036640 1 --2- 172.21.15.172:0/507939238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cbc0039e0 0x7f1cd014e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.053+0000 7f1cd5036640 1 -- 172.21.15.172:0/507939238 >> 172.21.15.172:0/507939238 conn(0x7f1cd008c000 msgr2=0x7f1cd00ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:06.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.053+0000 7f1cd5036640 1 -- 172.21.15.172:0/507939238 shutdown_connections 2024-04-14T08:43:06.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.053+0000 7f1cd5036640 1 -- 172.21.15.172:0/507939238 wait complete. 2024-04-14T08:43:06.065 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:43:06.163 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:43:06.163+0000 7fd090ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:43:06.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.206+0000 7f0e95036640 1 Processor -- start 2024-04-14T08:43:06.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.206+0000 7f0e95036640 1 -- start start 2024-04-14T08:43:06.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.207+0000 7f0e95036640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0e900af740 0x7f0e900722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.207+0000 7f0e95036640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e90072820 0x7f0e9013fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.207+0000 7f0e95036640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e900ce860 con 0x7f0e9013fa70 2024-04-14T08:43:06.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.207+0000 7f0e95036640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f0e900712d0 con 0x7f0e900af740 2024-04-14T08:43:06.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.207+0000 7f0e95036640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0e90071520 con 0x7f0e90072820 2024-04-14T08:43:06.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.207+0000 7f0e87fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0e900af740 0x7f0e900722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.207+0000 7f0e87fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0e900af740 0x7f0e900722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59492/0 (socket says 172.21.15.172:59492) 2024-04-14T08:43:06.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.207+0000 7f0e87fff640 1 -- 172.21.15.172:0/3551750461 learned_addr learned my addr 172.21.15.172:0/3551750461 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:06.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.208+0000 7f0e8f59e640 1 --2- 172.21.15.172:0/3551750461 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e90072820 0x7f0e9013fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.208+0000 7f0e87fff640 1 -- 172.21.15.172:0/3551750461 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e90072820 msgr2=0x7f0e9013fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:06.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.208+0000 7f0e87fff640 1 --2- 172.21.15.172:0/3551750461 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e90072820 0x7f0e9013fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.208+0000 7f0e87fff640 1 -- 172.21.15.172:0/3551750461 >> v1:172.21.15.177:6789/0 conn(0x7f0e9013fa70 legacy=0x7f0e900ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.208+0000 7f0e8e59c640 1 -- 172.21.15.172:0/3551750461 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4245867090 0 0) 0x7f0e80001110 con 0x7f0e9013fa70 2024-04-14T08:43:06.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.208+0000 7f0e87fff640 1 -- 172.21.15.172:0/3551750461 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e90071800 con 0x7f0e900af740 2024-04-14T08:43:06.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.209+0000 7f0e87fff640 1 --2- 172.21.15.172:0/3551750461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e900af740 0x7f0e900722e0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009870 tx=0x7f0e7802ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4480e4a3c2430ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:06.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.209+0000 7f0e8e59c640 1 -- 172.21.15.172:0/3551750461 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78030820 con 0x7f0e900af740 2024-04-14T08:43:06.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.209+0000 7f0e8e59c640 1 -- 172.21.15.172:0/3551750461 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e780309c0 con 0x7f0e900af740 2024-04-14T08:43:06.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.209+0000 7f0e8e59c640 1 -- 172.21.15.172:0/3551750461 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78030d90 con 0x7f0e900af740 2024-04-14T08:43:06.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.209+0000 7f0e95036640 1 -- 172.21.15.172:0/3551750461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e900af740 msgr2=0x7f0e900722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.209+0000 7f0e95036640 1 --2- 172.21.15.172:0/3551750461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e900af740 0x7f0e900722e0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009870 tx=0x7f0e7802ca00 comp rx=0 tx=0).stop 2024-04-14T08:43:06.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.209+0000 7f0e95036640 1 -- 172.21.15.172:0/3551750461 shutdown_connections 2024-04-14T08:43:06.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.209+0000 7f0e95036640 1 --2- 172.21.15.172:0/3551750461 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e90072820 0x7f0e9013fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.209+0000 7f0e95036640 1 --2- 172.21.15.172:0/3551750461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e900af740 0x7f0e900722e0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.209+0000 7f0e95036640 1 -- 172.21.15.172:0/3551750461 >> 172.21.15.172:0/3551750461 conn(0x7f0e9006b520 msgr2=0x7f0e9006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:06.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.209+0000 7f0e95036640 1 -- 172.21.15.172:0/3551750461 shutdown_connections 2024-04-14T08:43:06.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.209+0000 7f0e95036640 1 -- 172.21.15.172:0/3551750461 wait complete. 2024-04-14T08:43:06.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.211+0000 7f0e95036640 1 Processor -- start 2024-04-14T08:43:06.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.211+0000 7f0e95036640 1 -- start start 2024-04-14T08:43:06.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.211+0000 7f0e95036640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e90072820 0x7f0e90157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.211+0000 7f0e95036640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e900af740 0x7f0e9013fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.211+0000 7f0e95036640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e9013fa70 0x7f0e90140390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.211+0000 7f0e95036640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90072c70 con 0x7f0e900af740 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.211+0000 7f0e95036640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90071dc0 con 0x7f0e9013fa70 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.211+0000 7f0e95036640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f0e90144130 con 0x7f0e90072820 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e87fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e900af740 0x7f0e9013fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e8ed9d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e90072820 0x7f0e90157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e87fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e900af740 0x7f0e9013fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59500/0 (socket says 172.21.15.172:59500) 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e87fff640 1 -- 172.21.15.172:0/4094458615 learned_addr learned my addr 172.21.15.172:0/4094458615 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e8f59e640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e9013fa70 0x7f0e90140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e87fff640 1 -- 172.21.15.172:0/4094458615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e90072820 msgr2=0x7f0e90157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e87fff640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e90072820 0x7f0e90157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e87fff640 1 -- 172.21.15.172:0/4094458615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e9013fa70 msgr2=0x7f0e90140390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e87fff640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e9013fa70 0x7f0e90140390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e87fff640 1 -- 172.21.15.172:0/4094458615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e90142040 con 0x7f0e900af740 2024-04-14T08:43:06.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e87fff640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e900af740 0x7f0e9013fe50 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f0e78000c00 tx=0x7f0e780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:06.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e877fe640 1 -- 172.21.15.172:0/4094458615 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78004030 con 0x7f0e900af740 2024-04-14T08:43:06.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e877fe640 1 -- 172.21.15.172:0/4094458615 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e780041d0 con 0x7f0e900af740 2024-04-14T08:43:06.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e95036640 1 -- 172.21.15.172:0/4094458615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e90142320 con 0x7f0e900af740 2024-04-14T08:43:06.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e877fe640 1 -- 172.21.15.172:0/4094458615 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78038730 con 0x7f0e900af740 2024-04-14T08:43:06.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.212+0000 7f0e95036640 1 -- 172.21.15.172:0/4094458615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e901428e0 con 0x7f0e900af740 2024-04-14T08:43:06.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.213+0000 7f0e95036640 1 -- 172.21.15.172:0/4094458615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e90068590 con 0x7f0e900af740 2024-04-14T08:43:06.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.214+0000 7f0e877fe640 1 -- 172.21.15.172:0/4094458615 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e7804a060 con 0x7f0e900af740 2024-04-14T08:43:06.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.214+0000 7f0e877fe640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0e7006c8f0 0x7f0e7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.215+0000 7f0e877fe640 1 -- 172.21.15.172:0/4094458615 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f0e780ad520 con 0x7f0e900af740 2024-04-14T08:43:06.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.215+0000 7f0e8ed9d640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0e7006c8f0 0x7f0e7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.215+0000 7f0e8ed9d640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0e7006c8f0 0x7f0e7006edb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0e80000f60 tx=0x7f0e80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:06.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.219+0000 7f0e877fe640 1 -- 172.21.15.172:0/4094458615 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0e78034030 con 0x7f0e900af740 2024-04-14T08:43:06.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.554+0000 7f0e95036640 1 -- 172.21.15.172:0/4094458615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0e90159c00 con 0x7f0e900af740 2024-04-14T08:43:06.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.554+0000 7f0e877fe640 1 -- 172.21.15.172:0/4094458615 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) ==== 104+0+11 (secure 0 0 0) 0x7f0e7807b1b0 con 0x7f0e900af740 2024-04-14T08:43:06.554 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:43:06.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.556+0000 7f0e95036640 1 -- 172.21.15.172:0/4094458615 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0e7006c8f0 msgr2=0x7f0e7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.556+0000 7f0e95036640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0e7006c8f0 0x7f0e7006edb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0e80000f60 tx=0x7f0e80009040 comp rx=0 tx=0).stop 2024-04-14T08:43:06.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.556+0000 7f0e95036640 1 -- 172.21.15.172:0/4094458615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e900af740 msgr2=0x7f0e9013fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.556+0000 7f0e95036640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e900af740 0x7f0e9013fe50 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f0e78000c00 tx=0x7f0e780046c0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.556+0000 7f0e95036640 1 -- 172.21.15.172:0/4094458615 shutdown_connections 2024-04-14T08:43:06.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.556+0000 7f0e95036640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f0e7006c8f0 0x7f0e7006edb0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.556+0000 7f0e95036640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e9013fa70 0x7f0e90140390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.556+0000 7f0e95036640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e900af740 0x7f0e9013fe50 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.556+0000 7f0e95036640 1 --2- 172.21.15.172:0/4094458615 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e90072820 0x7f0e90157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.556+0000 7f0e95036640 1 -- 172.21.15.172:0/4094458615 >> 172.21.15.172:0/4094458615 conn(0x7f0e9006b520 msgr2=0x7f0e900afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:06.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.556+0000 7f0e95036640 1 -- 172.21.15.172:0/4094458615 shutdown_connections 2024-04-14T08:43:06.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.556+0000 7f0e95036640 1 -- 172.21.15.172:0/4094458615 wait complete. 2024-04-14T08:43:06.567 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:43:06.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.671+0000 7f23a7cde640 1 Processor -- start 2024-04-14T08:43:06.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.672+0000 7f23a7cde640 1 -- start start 2024-04-14T08:43:06.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.672+0000 7f23a7cde640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a00d1a20 0x7f23a006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.672+0000 7f23a7cde640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f23a0071250 0x7f23a00cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.672+0000 7f23a7cde640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23a00701b0 con 0x7f23a006d000 2024-04-14T08:43:06.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.672+0000 7f23a7cde640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f23a0070dc0 con 0x7f23a0071250 2024-04-14T08:43:06.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.672+0000 7f23a7cde640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f23a0070490 con 0x7f23a00d1a20 2024-04-14T08:43:06.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.673+0000 7f23a5279640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f23a0071250 0x7f23a00cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.673+0000 7f23a5279640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f23a0071250 0x7f23a00cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:33272/0 (socket says 172.21.15.172:33272) 2024-04-14T08:43:06.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.673+0000 7f23a5279640 1 -- 172.21.15.172:0/3056734670 learned_addr learned my addr 172.21.15.172:0/3056734670 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:06.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.673+0000 7f23a5a7a640 1 --2- 172.21.15.172:0/3056734670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a00d1a20 0x7f23a006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.673+0000 7f23a5279640 1 -- 172.21.15.172:0/3056734670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a00d1a20 msgr2=0x7f23a006fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.673+0000 7f23a5279640 1 --2- 172.21.15.172:0/3056734670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a00d1a20 0x7f23a006fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.673+0000 7f23a5279640 1 -- 172.21.15.172:0/3056734670 >> v1:172.21.15.177:6789/0 conn(0x7f23a006d000 legacy=0x7f23a015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.673+0000 7f23a5279640 1 -- 172.21.15.172:0/3056734670 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a0070770 con 0x7f23a0071250 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.674+0000 7f23a5279640 1 --2- 172.21.15.172:0/3056734670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a0071250 0x7f23a00cf530 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f239c009870 tx=0x7f239c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cedf1788758b0ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.674+0000 7f23a4a78640 1 -- 172.21.15.172:0/3056734670 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c035020 con 0x7f23a0071250 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.674+0000 7f23a4a78640 1 -- 172.21.15.172:0/3056734670 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239c031640 con 0x7f23a0071250 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.674+0000 7f23a4a78640 1 -- 172.21.15.172:0/3056734670 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c02d040 con 0x7f23a0071250 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.675+0000 7f23a7cde640 1 -- 172.21.15.172:0/3056734670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a0071250 msgr2=0x7f23a00cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.675+0000 7f23a7cde640 1 --2- 172.21.15.172:0/3056734670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a0071250 0x7f23a00cf530 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f239c009870 tx=0x7f239c02b8f0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.675+0000 7f23a7cde640 1 -- 172.21.15.172:0/3056734670 shutdown_connections 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.675+0000 7f23a7cde640 1 --2- 172.21.15.172:0/3056734670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a0071250 0x7f23a00cf530 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.675+0000 7f23a7cde640 1 --2- 172.21.15.172:0/3056734670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a00d1a20 0x7f23a006fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.675+0000 7f23a7cde640 1 -- 172.21.15.172:0/3056734670 >> 172.21.15.172:0/3056734670 conn(0x7f23a008c000 msgr2=0x7f23a00d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:06.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.675+0000 7f23a7cde640 1 -- 172.21.15.172:0/3056734670 shutdown_connections 2024-04-14T08:43:06.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.675+0000 7f23a7cde640 1 -- 172.21.15.172:0/3056734670 wait complete. 2024-04-14T08:43:06.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.677+0000 7f23a7cde640 1 Processor -- start 2024-04-14T08:43:06.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.677+0000 7f23a7cde640 1 -- start start 2024-04-14T08:43:06.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.678+0000 7f23a7cde640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a006d000 0x7f23a01d43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.678+0000 7f23a7cde640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a0071250 0x7f23a01f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.678+0000 7f23a7cde640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a00d1a20 0x7f23a01e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.678+0000 7f23a7cde640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23a00d1e00 con 0x7f23a006d000 2024-04-14T08:43:06.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.678+0000 7f23a7cde640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f23a00cfa70 con 0x7f23a0071250 2024-04-14T08:43:06.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.678+0000 7f23a7cde640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f23a00b08b0 con 0x7f23a00d1a20 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.678+0000 7f23a5a7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a006d000 0x7f23a01d43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.678+0000 7f23a5a7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a006d000 0x7f23a01d43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59508/0 (socket says 172.21.15.172:59508) 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.678+0000 7f23a5a7a640 1 -- 172.21.15.172:0/1886392646 learned_addr learned my addr 172.21.15.172:0/1886392646 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23a5279640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a0071250 0x7f23a01f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23a627b640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a00d1a20 0x7f23a01e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23a5a7a640 1 -- 172.21.15.172:0/1886392646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a00d1a20 msgr2=0x7f23a01e20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23a5a7a640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a00d1a20 0x7f23a01e20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23a5a7a640 1 -- 172.21.15.172:0/1886392646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a0071250 msgr2=0x7f23a01f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23a5a7a640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a0071250 0x7f23a01f89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23a5a7a640 1 -- 172.21.15.172:0/1886392646 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a00b0b90 con 0x7f23a006d000 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23a5279640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a0071250 0x7f23a01f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23a627b640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a00d1a20 0x7f23a01e20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23a5a7a640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a006d000 0x7f23a01d43b0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f239400e8a0 tx=0x7f239400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:06.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23927fc640 1 -- 172.21.15.172:0/1886392646 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394017070 con 0x7f23a006d000 2024-04-14T08:43:06.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23927fc640 1 -- 172.21.15.172:0/1886392646 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239400c940 con 0x7f23a006d000 2024-04-14T08:43:06.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23a7cde640 1 -- 172.21.15.172:0/1886392646 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a01e76d0 con 0x7f23a006d000 2024-04-14T08:43:06.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23927fc640 1 -- 172.21.15.172:0/1886392646 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239400cd30 con 0x7f23a006d000 2024-04-14T08:43:06.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.679+0000 7f23a7cde640 1 -- 172.21.15.172:0/1886392646 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a01e0980 con 0x7f23a006d000 2024-04-14T08:43:06.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.680+0000 7f23a7cde640 1 -- 172.21.15.172:0/1886392646 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a01e6ae0 con 0x7f23a006d000 2024-04-14T08:43:06.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.681+0000 7f23927fc640 1 -- 172.21.15.172:0/1886392646 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f239401a070 con 0x7f23a006d000 2024-04-14T08:43:06.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.681+0000 7f23927fc640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f238406c8f0 0x7f238406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.681+0000 7f23927fc640 1 -- 172.21.15.172:0/1886392646 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f239408bf70 con 0x7f23a006d000 2024-04-14T08:43:06.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.681+0000 7f23a5279640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f238406c8f0 0x7f238406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.682+0000 7f23a5279640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f238406c8f0 0x7f238406edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f239c036000 tx=0x7f239c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:06.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.686+0000 7f23927fc640 1 -- 172.21.15.172:0/1886392646 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f23a01e6ae0 con 0x7f23a006d000 2024-04-14T08:43:06.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.867+0000 7f23a7cde640 1 -- 172.21.15.172:0/1886392646 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f23a0075490 con 0x7f23a006d000 2024-04-14T08:43:06.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.868+0000 7f23927fc640 1 -- 172.21.15.172:0/1886392646 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) ==== 100+0+11 (secure 0 0 0) 0x7f239405a1c0 con 0x7f23a006d000 2024-04-14T08:43:06.868 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:43:06.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.870+0000 7f23a7cde640 1 -- 172.21.15.172:0/1886392646 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f238406c8f0 msgr2=0x7f238406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.870+0000 7f23a7cde640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f238406c8f0 0x7f238406edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f239c036000 tx=0x7f239c008030 comp rx=0 tx=0).stop 2024-04-14T08:43:06.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.870+0000 7f23a7cde640 1 -- 172.21.15.172:0/1886392646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a006d000 msgr2=0x7f23a01d43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.870+0000 7f23a7cde640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a006d000 0x7f23a01d43b0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f239400e8a0 tx=0x7f239400ed60 comp rx=0 tx=0).stop 2024-04-14T08:43:06.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.870+0000 7f23a7cde640 1 -- 172.21.15.172:0/1886392646 shutdown_connections 2024-04-14T08:43:06.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.870+0000 7f23a7cde640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f238406c8f0 0x7f238406edb0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.870+0000 7f23a7cde640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a00d1a20 0x7f23a01e20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.870+0000 7f23a7cde640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a0071250 0x7f23a01f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.870+0000 7f23a7cde640 1 --2- 172.21.15.172:0/1886392646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23a006d000 0x7f23a01d43b0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.870+0000 7f23a7cde640 1 -- 172.21.15.172:0/1886392646 >> 172.21.15.172:0/1886392646 conn(0x7f23a008c000 msgr2=0x7f23a01e5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:06.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.870+0000 7f23a7cde640 1 -- 172.21.15.172:0/1886392646 shutdown_connections 2024-04-14T08:43:06.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.870+0000 7f23a7cde640 1 -- 172.21.15.172:0/1886392646 wait complete. 2024-04-14T08:43:06.882 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:43:06.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.983+0000 7f37001bb640 1 Processor -- start 2024-04-14T08:43:06.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.984+0000 7f37001bb640 1 -- start start 2024-04-14T08:43:06.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f37001bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f80d0a70 0x7f36f80d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f37001bb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f815e890 con 0x7f36f815e4b0 2024-04-14T08:43:06.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f37001bb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f80d1f30 con 0x7f36f80b39b0 2024-04-14T08:43:06.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f37001bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f36f80d13f0 con 0x7f36f80d0a70 2024-04-14T08:43:06.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fdf57640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f36f815e4b0 0x7f36f80cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45812/0 (socket says 172.21.15.172:45812) 2024-04-14T08:43:06.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fdf57640 1 -- 172.21.15.172:0/3004410603 learned_addr learned my addr 172.21.15.172:0/3004410603 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:06.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fd756640 1 --2- 172.21.15.172:0/3004410603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f80d0a70 0x7f36f80d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3618924978 0 0) 0x7f36f815e890 con 0x7f36f815e4b0 2024-04-14T08:43:06.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36e4003520 con 0x7f36f815e4b0 2024-04-14T08:43:06.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1783697725 0 0) 0x7f36f80d1f30 con 0x7f36f80b39b0 2024-04-14T08:43:06.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36f815e890 con 0x7f36f80b39b0 2024-04-14T08:43:06.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2119534154 0 0) 0x7f36e4003520 con 0x7f36f815e4b0 2024-04-14T08:43:06.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f80d0a70 msgr2=0x7f36f80d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:06.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 --2- 172.21.15.172:0/3004410603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f80d0a70 0x7f36f80d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 >> v1:172.21.15.177:6789/0 conn(0x7f36f80b39b0 legacy=0x7f36f80b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f80d2210 con 0x7f36f815e4b0 2024-04-14T08:43:06.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f36f4001670 con 0x7f36f815e4b0 2024-04-14T08:43:06.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 >> v1:172.21.15.172:6789/0 conn(0x7f36f815e4b0 legacy=0x7f36f80cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 --2- 172.21.15.172:0/3004410603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f80d0a70 0x7f36e4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 --2- 172.21.15.172:0/3004410603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36e4004120 0x7f36e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 --2- 172.21.15.172:0/3004410603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e4004d40 0x7f36e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36f80d13f0 con 0x7f36e4004d40 2024-04-14T08:43:06.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36e400bb80 con 0x7f36e4004120 2024-04-14T08:43:06.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.985+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f36e400bd80 con 0x7f36f80d0a70 2024-04-14T08:43:06.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.986+0000 7f36fe758640 1 --2- 172.21.15.172:0/3004410603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e4004d40 0x7f36e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.986+0000 7f36fdf57640 1 --2- 172.21.15.172:0/3004410603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f80d0a70 0x7f36e4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.986+0000 7f36fd756640 1 --2- 172.21.15.172:0/3004410603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36e4004120 0x7f36e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.986+0000 7f36fe758640 1 -- 172.21.15.172:0/3004410603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f80d0a70 msgr2=0x7f36e4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:06.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.986+0000 7f36fe758640 1 --2- 172.21.15.172:0/3004410603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f80d0a70 0x7f36e4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.986+0000 7f36fe758640 1 -- 172.21.15.172:0/3004410603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36e4004120 msgr2=0x7f36e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:06.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.986+0000 7f36fe758640 1 --2- 172.21.15.172:0/3004410603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36e4004120 0x7f36e4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.986+0000 7f36fe758640 1 -- 172.21.15.172:0/3004410603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e4003520 con 0x7f36e4004d40 2024-04-14T08:43:06.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.987+0000 7f36fe758640 1 --2- 172.21.15.172:0/3004410603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e4004d40 0x7f36e40051c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f36f0001fd0 tx=0x7f36e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5ec8c26823492e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:06.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.987+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0001bf0 con 0x7f36e4004d40 2024-04-14T08:43:06.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.987+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f0003b70 con 0x7f36e4004d40 2024-04-14T08:43:06.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.987+0000 7f36fcf55640 1 -- 172.21.15.172:0/3004410603 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f00050d0 con 0x7f36e4004d40 2024-04-14T08:43:06.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.987+0000 7f37001bb640 1 -- 172.21.15.172:0/3004410603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e4004d40 msgr2=0x7f36e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.987+0000 7f37001bb640 1 --2- 172.21.15.172:0/3004410603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36e4004d40 0x7f36e40051c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f36f0001fd0 tx=0x7f36e4005830 comp rx=0 tx=0).stop 2024-04-14T08:43:06.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.987+0000 7f36fe758640 1 -- 172.21.15.172:0/3004410603 reap_dead start 2024-04-14T08:43:06.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.987+0000 7f37001bb640 1 -- 172.21.15.172:0/3004410603 shutdown_connections 2024-04-14T08:43:06.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.987+0000 7f37001bb640 1 -- 172.21.15.172:0/3004410603 >> 172.21.15.172:0/3004410603 conn(0x7f36f809d570 msgr2=0x7f36f80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:06.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.988+0000 7f37001bb640 1 -- 172.21.15.172:0/3004410603 shutdown_connections 2024-04-14T08:43:06.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.988+0000 7f37001bb640 1 -- 172.21.15.172:0/3004410603 wait complete. 2024-04-14T08:43:06.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.989+0000 7f37001bb640 1 Processor -- start 2024-04-14T08:43:06.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.989+0000 7f37001bb640 1 -- start start 2024-04-14T08:43:06.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.989+0000 7f37001bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f81575d0 0x7f36f80b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f37001bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f81579b0 0x7f36f80cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f37001bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f806c250 0x7f36f806c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f37001bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36f80d2ab0 con 0x7f36f81575d0 2024-04-14T08:43:06.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f37001bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36f80d27d0 con 0x7f36f806c250 2024-04-14T08:43:06.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f37001bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f36f80d2fd0 con 0x7f36f81579b0 2024-04-14T08:43:06.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36fdf57640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f81575d0 0x7f36f80b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36fdf57640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f81575d0 0x7f36f80b7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59532/0 (socket says 172.21.15.172:59532) 2024-04-14T08:43:06.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36fdf57640 1 -- 172.21.15.172:0/3428036543 learned_addr learned my addr 172.21.15.172:0/3428036543 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:06.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36fd756640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f81579b0 0x7f36f80cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36fe758640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f806c250 0x7f36f806c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36fdf57640 1 -- 172.21.15.172:0/3428036543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f81579b0 msgr2=0x7f36f80cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36fdf57640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f81579b0 0x7f36f80cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36fdf57640 1 -- 172.21.15.172:0/3428036543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f806c250 msgr2=0x7f36f806c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36fdf57640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f806c250 0x7f36f806c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36fdf57640 1 -- 172.21.15.172:0/3428036543 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f80cbdb0 con 0x7f36f81575d0 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36fe758640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f806c250 0x7f36f806c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36fdf57640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f81575d0 0x7f36f80b7f30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f36f4004ee0 tx=0x7f36f4007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36ee7fc640 1 -- 172.21.15.172:0/3428036543 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f400f7b0 con 0x7f36f81575d0 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36ee7fc640 1 -- 172.21.15.172:0/3428036543 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f4018070 con 0x7f36f81575d0 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f36ee7fc640 1 -- 172.21.15.172:0/3428036543 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f400fbb0 con 0x7f36f81575d0 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f37001bb640 1 -- 172.21.15.172:0/3428036543 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f80cc090 con 0x7f36f81575d0 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.990+0000 7f37001bb640 1 -- 172.21.15.172:0/3428036543 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f80b8630 con 0x7f36f81575d0 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.991+0000 7f37001bb640 1 -- 172.21.15.172:0/3428036543 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f8159240 con 0x7f36f81575d0 2024-04-14T08:43:06.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.992+0000 7f36ee7fc640 1 -- 172.21.15.172:0/3428036543 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36f400fd50 con 0x7f36f81575d0 2024-04-14T08:43:06.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.992+0000 7f36ee7fc640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f36dc06c920 0x7f36dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:06.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.992+0000 7f36ee7fc640 1 -- 172.21.15.172:0/3428036543 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f36f408bc50 con 0x7f36f81575d0 2024-04-14T08:43:06.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.992+0000 7f36fd756640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f36dc06c920 0x7f36dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:06.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.993+0000 7f36fd756640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f36dc06c920 0x7f36dc06ede0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f36f8157db0 tx=0x7f36e8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:06.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:06.997+0000 7f36ee7fc640 1 -- 172.21.15.172:0/3428036543 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f36f8159240 con 0x7f36f81575d0 2024-04-14T08:43:07.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.190+0000 7f37001bb640 1 -- 172.21.15.172:0/3428036543 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f36f8068590 con 0x7f36f81575d0 2024-04-14T08:43:07.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.190+0000 7f36ee7fc640 1 -- 172.21.15.172:0/3428036543 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v117) ==== 80+0+124 (secure 0 0 0) 0x7f36f4091020 con 0x7f36f81575d0 2024-04-14T08:43:07.192 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.177:6835/3201353747 2024-04-15T08:42:57.915856+0000 2024-04-14T08:43:07.192 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.177:6833/3201353747 2024-04-15T08:42:57.915856+0000 2024-04-14T08:43:07.193 INFO:teuthology.orchestra.run.smithi172.stderr:listed 2 entries 2024-04-14T08:43:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.192+0000 7f37001bb640 1 -- 172.21.15.172:0/3428036543 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f36dc06c920 msgr2=0x7f36dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.192+0000 7f37001bb640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f36dc06c920 0x7f36dc06ede0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f36f8157db0 tx=0x7f36e8009190 comp rx=0 tx=0).stop 2024-04-14T08:43:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.192+0000 7f37001bb640 1 -- 172.21.15.172:0/3428036543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f81575d0 msgr2=0x7f36f80b7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.192+0000 7f37001bb640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f81575d0 0x7f36f80b7f30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f36f4004ee0 tx=0x7f36f4007430 comp rx=0 tx=0).stop 2024-04-14T08:43:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.192+0000 7f37001bb640 1 -- 172.21.15.172:0/3428036543 shutdown_connections 2024-04-14T08:43:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.192+0000 7f37001bb640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f36dc06c920 0x7f36dc06ede0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.192+0000 7f37001bb640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f806c250 0x7f36f806c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.192+0000 7f37001bb640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36f81579b0 0x7f36f80cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.192+0000 7f37001bb640 1 --2- 172.21.15.172:0/3428036543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f81575d0 0x7f36f80b7f30 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.192+0000 7f37001bb640 1 -- 172.21.15.172:0/3428036543 >> 172.21.15.172:0/3428036543 conn(0x7f36f809d570 msgr2=0x7f36f815a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.193+0000 7f37001bb640 1 -- 172.21.15.172:0/3428036543 shutdown_connections 2024-04-14T08:43:07.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.193+0000 7f37001bb640 1 -- 172.21.15.172:0/3428036543 wait complete. 2024-04-14T08:43:07.204 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:43:07.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:07.204 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:43:07.236 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:43:07.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:07.236 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:43:07.291 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:43:07.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.453+0000 7ff3e81fe640 1 Processor -- start 2024-04-14T08:43:07.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.453+0000 7ff3e81fe640 1 -- start start 2024-04-14T08:43:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e81fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff3e00d2830 0x7ff3e00d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e81fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3e00cedd0 0x7ff3e00b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e81fe640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e00d3150 con 0x7ff3e015e890 2024-04-14T08:43:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e81fe640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff3e00cff30 con 0x7ff3e00d2830 2024-04-14T08:43:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e81fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e00d2350 con 0x7ff3e00cedd0 2024-04-14T08:43:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e5799640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff3e015e890 0x7ff3e00d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45818/0 (socket says 172.21.15.172:45818) 2024-04-14T08:43:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e5799640 1 -- 172.21.15.172:0/2506863309 learned_addr learned my addr 172.21.15.172:0/2506863309 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:07.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e5f9a640 1 --2- 172.21.15.172:0/2506863309 >> v2:172.21.15.177:3300/0 conn(0x7ff3e00d2830 0x7ff3e00d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e679b640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3e00cedd0 0x7ff3e00b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1047996378 0 0) 0x7ff3e00d3150 con 0x7ff3e015e890 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3cc003520 con 0x7ff3e015e890 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 314956140 0 0) 0x7ff3cc003520 con 0x7ff3e015e890 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3e00cedd0 msgr2=0x7ff3e00b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3e00cedd0 0x7ff3e00b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 >> v2:172.21.15.177:3300/0 conn(0x7ff3e00d2830 msgr2=0x7ff3e00d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 --2- 172.21.15.172:0/2506863309 >> v2:172.21.15.177:3300/0 conn(0x7ff3e00d2830 0x7ff3e00d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e00cf320 con 0x7ff3e015e890 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7ff3d0001670 con 0x7ff3e015e890 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 >> v1:172.21.15.172:6789/0 conn(0x7ff3e015e890 legacy=0x7ff3e00d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3cc0039b0 0x7ff3cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3cc0043a0 0x7ff3cc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3cc004ef0 0x7ff3cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e00cff30 con 0x7ff3cc0043a0 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e00d2350 con 0x7ff3cc004ef0 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff3cc00bf00 con 0x7ff3cc0039b0 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.454+0000 7ff3e679b640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3e00cedd0 0x7ff3e00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.455+0000 7ff3e5799640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3cc0043a0 0x7ff3cc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.455+0000 7ff3e5f9a640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3cc0039b0 0x7ff3cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.455+0000 7ff3e679b640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3cc004ef0 0x7ff3cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.455+0000 7ff3e5799640 1 -- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3cc0039b0 msgr2=0x7ff3cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.455+0000 7ff3e5799640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3cc0039b0 0x7ff3cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.455+0000 7ff3e5799640 1 -- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3cc004ef0 msgr2=0x7ff3cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.455+0000 7ff3e5799640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3cc004ef0 0x7ff3cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.455+0000 7ff3e5799640 1 -- 172.21.15.172:0/2506863309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc003520 con 0x7ff3cc0043a0 2024-04-14T08:43:07.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.455+0000 7ff3e679b640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3cc004ef0 0x7ff3cc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.456+0000 7ff3e5799640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3cc0043a0 0x7ff3cc004880 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000c240 tx=0x7ff3cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da919bfbb102895 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.456+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0007d50 con 0x7ff3cc0043a0 2024-04-14T08:43:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.456+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d0030070 con 0x7ff3cc0043a0 2024-04-14T08:43:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.456+0000 7ff3e4f98640 1 -- 172.21.15.172:0/2506863309 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d002cdc0 con 0x7ff3cc0043a0 2024-04-14T08:43:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.457+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2506863309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3cc0043a0 msgr2=0x7ff3cc004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.457+0000 7ff3e81fe640 1 --2- 172.21.15.172:0/2506863309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3cc0043a0 0x7ff3cc004880 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000c240 tx=0x7ff3cc005830 comp rx=0 tx=0).stop 2024-04-14T08:43:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.457+0000 7ff3e679b640 1 -- 172.21.15.172:0/2506863309 reap_dead start 2024-04-14T08:43:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.457+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2506863309 shutdown_connections 2024-04-14T08:43:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.457+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2506863309 >> 172.21.15.172:0/2506863309 conn(0x7ff3e009d590 msgr2=0x7ff3e00d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.457+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2506863309 shutdown_connections 2024-04-14T08:43:07.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.457+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2506863309 wait complete. 2024-04-14T08:43:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.458+0000 7ff3e81fe640 1 Processor -- start 2024-04-14T08:43:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.458+0000 7ff3e81fe640 1 -- start start 2024-04-14T08:43:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e81fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3cc0043a0 0x7ff3e00b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e81fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3e0152770 0x7ff3e00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e81fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3e00cd850 0x7ff3e00cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e81fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e00cfbc0 con 0x7ff3cc0043a0 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e81fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e00cf8e0 con 0x7ff3e00cd850 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e81fe640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e00b3710 con 0x7ff3e0152770 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e5f9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3cc0043a0 0x7ff3e00b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e5f9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3cc0043a0 0x7ff3e00b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59548/0 (socket says 172.21.15.172:59548) 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e5f9a640 1 -- 172.21.15.172:0/2065390311 learned_addr learned my addr 172.21.15.172:0/2065390311 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e5799640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3e0152770 0x7ff3e00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e679b640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3e00cd850 0x7ff3e00cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e5f9a640 1 -- 172.21.15.172:0/2065390311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3e0152770 msgr2=0x7ff3e00cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e5f9a640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3e0152770 0x7ff3e00cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e5f9a640 1 -- 172.21.15.172:0/2065390311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3e00cd850 msgr2=0x7ff3e00cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e5f9a640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3e00cd850 0x7ff3e00cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e5f9a640 1 -- 172.21.15.172:0/2065390311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e00cdd60 con 0x7ff3cc0043a0 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e5f9a640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3cc0043a0 0x7ff3e00b4fe0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc011960 tx=0x7ff3dc011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3d67fc640 1 -- 172.21.15.172:0/2065390311 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc00faf0 con 0x7ff3cc0043a0 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3d67fc640 1 -- 172.21.15.172:0/2065390311 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3dc00fc90 con 0x7ff3cc0043a0 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3d67fc640 1 -- 172.21.15.172:0/2065390311 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc014440 con 0x7ff3cc0043a0 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.459+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2065390311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e00ce040 con 0x7ff3cc0043a0 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.460+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2065390311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e00b5520 con 0x7ff3cc0043a0 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.461+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2065390311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e0068520 con 0x7ff3cc0043a0 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.461+0000 7ff3d67fc640 1 -- 172.21.15.172:0/2065390311 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3dc012040 con 0x7ff3cc0043a0 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.461+0000 7ff3d67fc640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff3c406c8f0 0x7ff3c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.461+0000 7ff3d67fc640 1 -- 172.21.15.172:0/2065390311 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7ff3dc08c410 con 0x7ff3cc0043a0 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.461+0000 7ff3e5799640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff3c406c8f0 0x7ff3c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.462+0000 7ff3e5799640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff3c406c8f0 0x7ff3c406edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0034000 tx=0x7ff3d0007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:07.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.470+0000 7ff3d67fc640 1 -- 172.21.15.172:0/2065390311 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff3dc019030 con 0x7ff3cc0043a0 2024-04-14T08:43:07.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.684+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2065390311 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3e00d2610 con 0x7ff3cc0043a0 2024-04-14T08:43:07.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.685+0000 7ff3d67fc640 1 -- 172.21.15.172:0/2065390311 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5194 (secure 0 0 0) 0x7ff3dc05a6f0 con 0x7ff3cc0043a0 2024-04-14T08:43:07.685 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:07.685 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":93,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3881291102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3881291102},{"type":"v1","addr":"172.21.15.177:6835","nonce":3881291102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:43:01.831015+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9048},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9048":{"gid":9048,"name":"c","rank":0,"incarnation":90,"state":"up:active","state_seq":9,"addr":"172.21.15.172:6837/744853572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":744853572},{"type":"v1","addr":"172.21.15.172:6837","nonce":744853572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9048,"qdb_cluster":[9048]},"id":8}]} 2024-04-14T08:43:07.685 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 93 2024-04-14T08:43:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.687+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2065390311 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff3c406c8f0 msgr2=0x7ff3c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.687+0000 7ff3e81fe640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff3c406c8f0 0x7ff3c406edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0034000 tx=0x7ff3d0007000 comp rx=0 tx=0).stop 2024-04-14T08:43:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.687+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2065390311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3cc0043a0 msgr2=0x7ff3e00b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.687+0000 7ff3e81fe640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3cc0043a0 0x7ff3e00b4fe0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc011960 tx=0x7ff3dc011e20 comp rx=0 tx=0).stop 2024-04-14T08:43:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.687+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2065390311 shutdown_connections 2024-04-14T08:43:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.687+0000 7ff3e81fe640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff3c406c8f0 0x7ff3c406edb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.687+0000 7ff3e81fe640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3e00cd850 0x7ff3e00cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.687+0000 7ff3e81fe640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3e0152770 0x7ff3e00cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.687+0000 7ff3e81fe640 1 --2- 172.21.15.172:0/2065390311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3cc0043a0 0x7ff3e00b4fe0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.687+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2065390311 >> 172.21.15.172:0/2065390311 conn(0x7ff3e009d590 msgr2=0x7ff3e015ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.687+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2065390311 shutdown_connections 2024-04-14T08:43:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.687+0000 7ff3e81fe640 1 -- 172.21.15.172:0/2065390311 wait complete. 2024-04-14T08:43:07.702 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-04-14T08:43:07.828 INFO:teuthology.orchestra.run.smithi172.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-14T08:43:07.828 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-14T08:43:07.828 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-14T08:43:07.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.872+0000 7f58e243ff40 1 Processor -- start 2024-04-14T08:43:07.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.872+0000 7f58e243ff40 1 -- start start 2024-04-14T08:43:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e243ff40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d361973e10 0x55d361a013c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e243ff40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d3619f9fc0 0x55d361928540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e243ff40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361a00170 0x55d3619f5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e243ff40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d3619591f0 con 0x55d361a00170 2024-04-14T08:43:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e243ff40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d361959070 con 0x55d361973e10 2024-04-14T08:43:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e243ff40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d361959370 con 0x55d3619f9fc0 2024-04-14T08:43:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e053d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361a00170 0x55d3619f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e053d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361a00170 0x55d3619f5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59568/0 (socket says 172.21.15.172:59568) 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e053d640 1 -- 172.21.15.172:0/3103653260 learned_addr learned my addr 172.21.15.172:0/3103653260 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58d7fff640 1 --2- 172.21.15.172:0/3103653260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d3619f9fc0 0x55d361928540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58dfd3c640 1 --2- 172.21.15.172:0/3103653260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d361973e10 0x55d361a013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e053d640 1 -- 172.21.15.172:0/3103653260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d3619f9fc0 msgr2=0x55d361928540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e053d640 1 --2- 172.21.15.172:0/3103653260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d3619f9fc0 0x55d361928540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e053d640 1 -- 172.21.15.172:0/3103653260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d361973e10 msgr2=0x55d361a013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e053d640 1 --2- 172.21.15.172:0/3103653260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d361973e10 0x55d361a013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e053d640 1 -- 172.21.15.172:0/3103653260 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d361958a30 con 0x55d361a00170 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58e053d640 1 --2- 172.21.15.172:0/3103653260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361a00170 0x55d3619f5310 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f58d800b640 tx=0x7f58d800bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b128790ced4e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58ded3a640 1 -- 172.21.15.172:0/3103653260 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8014020 con 0x55d361a00170 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58ded3a640 1 -- 172.21.15.172:0/3103653260 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d8010820 con 0x55d361a00170 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.873+0000 7f58ded3a640 1 -- 172.21.15.172:0/3103653260 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8010c10 con 0x55d361a00170 2024-04-14T08:43:07.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.874+0000 7f58e243ff40 1 -- 172.21.15.172:0/3103653260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361a00170 msgr2=0x55d3619f5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.874+0000 7f58e243ff40 1 --2- 172.21.15.172:0/3103653260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361a00170 0x55d3619f5310 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f58d800b640 tx=0x7f58d800bb00 comp rx=0 tx=0).stop 2024-04-14T08:43:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.874+0000 7f58e243ff40 1 -- 172.21.15.172:0/3103653260 shutdown_connections 2024-04-14T08:43:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.874+0000 7f58e243ff40 1 --2- 172.21.15.172:0/3103653260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361a00170 0x55d3619f5310 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.874+0000 7f58e243ff40 1 --2- 172.21.15.172:0/3103653260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d3619f9fc0 0x55d361928540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.874+0000 7f58e243ff40 1 --2- 172.21.15.172:0/3103653260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d361973e10 0x55d361a013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.874+0000 7f58e243ff40 1 -- 172.21.15.172:0/3103653260 >> 172.21.15.172:0/3103653260 conn(0x55d361956a50 msgr2=0x55d361929000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.874+0000 7f58e243ff40 1 -- 172.21.15.172:0/3103653260 shutdown_connections 2024-04-14T08:43:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.874+0000 7f58e243ff40 1 -- 172.21.15.172:0/3103653260 wait complete. 2024-04-14T08:43:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.875+0000 7f58e243ff40 1 Processor -- start 2024-04-14T08:43:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.875+0000 7f58e243ff40 1 -- start start 2024-04-14T08:43:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58e243ff40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361973e10 0x55d3619f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58e243ff40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d3619f9fc0 0x55d3619ecf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58e243ff40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d361a00170 0x55d3619f0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58e243ff40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d3619553a0 con 0x55d361973e10 2024-04-14T08:43:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58e243ff40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d361955f90 con 0x55d361a00170 2024-04-14T08:43:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58e243ff40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55d361969ea0 con 0x55d3619f9fc0 2024-04-14T08:43:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58dfd3c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361973e10 0x55d3619f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58dfd3c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361973e10 0x55d3619f4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59578/0 (socket says 172.21.15.172:59578) 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58dfd3c640 1 -- 172.21.15.172:0/2035625545 learned_addr learned my addr 172.21.15.172:0/2035625545 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58dfd3c640 1 -- 172.21.15.172:0/2035625545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d3619f9fc0 msgr2=0x55d3619ecf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58dfd3c640 1 --2- 172.21.15.172:0/2035625545 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d3619f9fc0 0x55d3619ecf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58dfd3c640 1 -- 172.21.15.172:0/2035625545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d361a00170 msgr2=0x55d3619f0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58dfd3c640 1 --2- 172.21.15.172:0/2035625545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d361a00170 0x55d3619f0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.876+0000 7f58dfd3c640 1 -- 172.21.15.172:0/2035625545 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3619f4b50 con 0x55d361973e10 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.877+0000 7f58dfd3c640 1 --2- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361973e10 0x55d3619f4570 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f58d0055040 tx=0x7f58d0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.877+0000 7f58ddd38640 1 -- 172.21.15.172:0/2035625545 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0057040 con 0x55d361973e10 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.877+0000 7f58ddd38640 1 -- 172.21.15.172:0/2035625545 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d0052b30 con 0x55d361973e10 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.877+0000 7f58e243ff40 1 -- 172.21.15.172:0/2035625545 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d3619f4e30 con 0x55d361973e10 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.877+0000 7f58ddd38640 1 -- 172.21.15.172:0/2035625545 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d005d440 con 0x55d361973e10 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.877+0000 7f58e243ff40 1 -- 172.21.15.172:0/2035625545 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3619ed4d0 con 0x55d361973e10 2024-04-14T08:43:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.878+0000 7f58ddd38640 1 -- 172.21.15.172:0/2035625545 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58d005d5e0 con 0x55d361973e10 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.879+0000 7f58ddd38640 1 --2- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58c406b2d0 0x7f58c406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.879+0000 7f58ddd38640 1 -- 172.21.15.172:0/2035625545 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f58d00cc8d0 con 0x55d361973e10 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.879+0000 7f58d7fff640 1 --2- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58c406b2d0 0x7f58c406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.879+0000 7f58e243ff40 1 --2- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55d3619ee060 0x55d3619ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.879+0000 7f58e243ff40 1 -- 172.21.15.172:0/2035625545 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) -- 0x55d3619eea10 con 0x55d3619ee060 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.879+0000 7f58e053d640 1 --2- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55d3619ee060 0x55d3619ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.879+0000 7f58d7fff640 1 --2- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58c406b2d0 0x7f58c406d790 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x55d361a02ac0 tx=0x7f58c80058c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.879+0000 7f58e053d640 1 --2- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55d3619ee060 0x55d3619ee4d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.880+0000 7f58e053d640 1 -- 172.21.15.172:0/2035625545 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f58d80109d0 con 0x55d3619ee060 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.880+0000 7f58e243ff40 1 -- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55d3619ee060 msgr2=0x55d3619ee4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.880+0000 7f58e243ff40 1 --2- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x55d3619ee060 0x55d3619ee4d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.880+0000 7f58e243ff40 1 -- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58c406b2d0 msgr2=0x7f58c406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.880+0000 7f58e243ff40 1 --2- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f58c406b2d0 0x7f58c406d790 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x55d361a02ac0 tx=0x7f58c80058c0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.880+0000 7f58e243ff40 1 -- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361973e10 msgr2=0x55d3619f4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.880+0000 7f58e243ff40 1 --2- 172.21.15.172:0/2035625545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d361973e10 0x55d3619f4570 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f58d0055040 tx=0x7f58d0002fb0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.880+0000 7f58e053d640 1 -- 172.21.15.172:0/2035625545 reap_dead start 2024-04-14T08:43:07.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.880+0000 7f58e243ff40 1 -- 172.21.15.172:0/2035625545 shutdown_connections 2024-04-14T08:43:07.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.880+0000 7f58e243ff40 1 -- 172.21.15.172:0/2035625545 >> 172.21.15.172:0/2035625545 conn(0x55d361956a50 msgr2=0x55d361972200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:07.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.880+0000 7f58e243ff40 1 -- 172.21.15.172:0/2035625545 shutdown_connections 2024-04-14T08:43:07.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.880+0000 7f58e243ff40 1 -- 172.21.15.172:0/2035625545 wait complete. 2024-04-14T08:43:07.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:43:07.885 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-14T08:43:07.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.967+0000 7fd709756f40 1 Processor -- start 2024-04-14T08:43:07.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.967+0000 7fd709756f40 1 -- start start 2024-04-14T08:43:07.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.967+0000 7fd709756f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f168d040 0x5615f175b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.967+0000 7fd709756f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615f16a10d0 0x5615f1754120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.967+0000 7fd709756f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f175cf60 0x5615f1758fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.967+0000 7fd709756f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5615f1777ce0 con 0x5615f175cf60 2024-04-14T08:43:07.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.967+0000 7fd709756f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5615f1777a40 con 0x5615f168d040 2024-04-14T08:43:07.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.967+0000 7fd709756f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5615f1777f80 con 0x5615f16a10d0 2024-04-14T08:43:07.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.967+0000 7fd70784c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f175cf60 0x5615f1758fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.967+0000 7fd70784c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f175cf60 0x5615f1758fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59600/0 (socket says 172.21.15.172:59600) 2024-04-14T08:43:07.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.967+0000 7fd70784c640 1 -- 172.21.15.172:0/2555006778 learned_addr learned my addr 172.21.15.172:0/2555006778 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:07.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.968+0000 7fd70684a640 1 --2- 172.21.15.172:0/2555006778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615f16a10d0 0x5615f1754120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.968+0000 7fd70704b640 1 --2- 172.21.15.172:0/2555006778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f168d040 0x5615f175b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.968+0000 7fd70784c640 1 -- 172.21.15.172:0/2555006778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615f16a10d0 msgr2=0x5615f1754120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.968+0000 7fd70784c640 1 --2- 172.21.15.172:0/2555006778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615f16a10d0 0x5615f1754120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.968+0000 7fd70784c640 1 -- 172.21.15.172:0/2555006778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f168d040 msgr2=0x5615f175b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.968+0000 7fd70784c640 1 --2- 172.21.15.172:0/2555006778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f168d040 0x5615f175b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.968+0000 7fd70784c640 1 -- 172.21.15.172:0/2555006778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f16ce3a0 con 0x5615f175cf60 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.968+0000 7fd70704b640 1 --2- 172.21.15.172:0/2555006778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f168d040 0x5615f175b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.968+0000 7fd70784c640 1 --2- 172.21.15.172:0/2555006778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f175cf60 0x5615f1758fa0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fd700057040 tx=0x7fd70004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9707c35aa165504 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.968+0000 7fd705848640 1 -- 172.21.15.172:0/2555006778 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70004daf0 con 0x5615f175cf60 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.968+0000 7fd705848640 1 -- 172.21.15.172:0/2555006778 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd70004dcb0 con 0x5615f175cf60 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.968+0000 7fd705848640 1 -- 172.21.15.172:0/2555006778 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70004de60 con 0x5615f175cf60 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.969+0000 7fd709756f40 1 -- 172.21.15.172:0/2555006778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f175cf60 msgr2=0x5615f1758fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.969+0000 7fd709756f40 1 --2- 172.21.15.172:0/2555006778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f175cf60 0x5615f1758fa0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fd700057040 tx=0x7fd70004f000 comp rx=0 tx=0).stop 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.969+0000 7fd709756f40 1 -- 172.21.15.172:0/2555006778 shutdown_connections 2024-04-14T08:43:07.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.969+0000 7fd709756f40 1 --2- 172.21.15.172:0/2555006778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f175cf60 0x5615f1758fa0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.969+0000 7fd709756f40 1 --2- 172.21.15.172:0/2555006778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615f16a10d0 0x5615f1754120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.969+0000 7fd709756f40 1 --2- 172.21.15.172:0/2555006778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f168d040 0x5615f175b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.969+0000 7fd709756f40 1 -- 172.21.15.172:0/2555006778 >> 172.21.15.172:0/2555006778 conn(0x5615f16c93d0 msgr2=0x5615f16cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:07.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.969+0000 7fd709756f40 1 -- 172.21.15.172:0/2555006778 shutdown_connections 2024-04-14T08:43:07.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.969+0000 7fd709756f40 1 -- 172.21.15.172:0/2555006778 wait complete. 2024-04-14T08:43:07.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.969+0000 7fd709756f40 1 Processor -- start 2024-04-14T08:43:07.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.969+0000 7fd709756f40 1 -- start start 2024-04-14T08:43:07.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd709756f40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615f168d040 0x5615f180ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd709756f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f16a10d0 0x5615f180f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd709756f40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f175cf60 0x5615f177af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd709756f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5615f1778c10 con 0x5615f16a10d0 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd709756f40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5615f1778950 con 0x5615f175cf60 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd709756f40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x5615f175acc0 con 0x5615f168d040 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd70684a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f16a10d0 0x5615f180f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd70684a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f16a10d0 0x5615f180f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59612/0 (socket says 172.21.15.172:59612) 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd70684a640 1 -- 172.21.15.172:0/932797690 learned_addr learned my addr 172.21.15.172:0/932797690 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd70684a640 1 -- 172.21.15.172:0/932797690 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615f168d040 msgr2=0x5615f180ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd70684a640 1 --2- 172.21.15.172:0/932797690 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615f168d040 0x5615f180ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd70684a640 1 -- 172.21.15.172:0/932797690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f175cf60 msgr2=0x5615f177af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd70684a640 1 --2- 172.21.15.172:0/932797690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f175cf60 0x5615f177af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd70684a640 1 -- 172.21.15.172:0/932797690 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f1756920 con 0x5615f16a10d0 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.970+0000 7fd70684a640 1 --2- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f16a10d0 0x5615f180f440 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc009e50 tx=0x7fd6fc006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.971+0000 7fd704846640 1 -- 172.21.15.172:0/932797690 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc013400 con 0x5615f16a10d0 2024-04-14T08:43:07.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.971+0000 7fd704846640 1 -- 172.21.15.172:0/932797690 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6fc0135a0 con 0x5615f16a10d0 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.971+0000 7fd704846640 1 -- 172.21.15.172:0/932797690 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc013990 con 0x5615f16a10d0 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.971+0000 7fd709756f40 1 -- 172.21.15.172:0/932797690 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5615f17572f0 con 0x5615f16a10d0 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.971+0000 7fd709756f40 1 -- 172.21.15.172:0/932797690 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f17784b0 con 0x5615f16a10d0 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.972+0000 7fd704846640 1 -- 172.21.15.172:0/932797690 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6fc01a020 con 0x5615f16a10d0 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.972+0000 7fd704846640 1 --2- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd6f406b2d0 0x7fd6f406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.973+0000 7fd70704b640 1 --2- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd6f406b2d0 0x7fd6f406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.973+0000 7fd704846640 1 -- 172.21.15.172:0/932797690 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7fd6fc08a380 con 0x5615f16a10d0 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.973+0000 7fd709756f40 1 --2- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x5615f16e8790 0x5615f177b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.973+0000 7fd709756f40 1 -- 172.21.15.172:0/932797690 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) -- 0x5615f180f980 con 0x5615f16e8790 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.973+0000 7fd70704b640 1 --2- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd6f406b2d0 0x7fd6f406d790 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8007500 tx=0x7fd6f8005700 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.973+0000 7fd70784c640 1 --2- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x5615f16e8790 0x5615f177b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.973+0000 7fd70784c640 1 --2- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x5615f16e8790 0x5615f177b4d0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:07.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.974+0000 7fd70784c640 1 -- 172.21.15.172:0/932797690 <== osd.2 v2:172.21.15.172:6816/1609902500 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fd700065070 con 0x5615f16e8790 2024-04-14T08:43:07.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.974+0000 7fd709756f40 1 -- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x5615f16e8790 msgr2=0x5615f177b4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.974+0000 7fd709756f40 1 --2- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x5615f16e8790 0x5615f177b4d0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:07.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.974+0000 7fd709756f40 1 -- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd6f406b2d0 msgr2=0x7fd6f406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.974+0000 7fd709756f40 1 --2- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fd6f406b2d0 0x7fd6f406d790 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8007500 tx=0x7fd6f8005700 comp rx=0 tx=0).stop 2024-04-14T08:43:07.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.974+0000 7fd709756f40 1 -- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f16a10d0 msgr2=0x5615f180f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:07.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.974+0000 7fd709756f40 1 --2- 172.21.15.172:0/932797690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5615f16a10d0 0x5615f180f440 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc009e50 tx=0x7fd6fc006750 comp rx=0 tx=0).stop 2024-04-14T08:43:07.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.974+0000 7fd70784c640 1 -- 172.21.15.172:0/932797690 reap_dead start 2024-04-14T08:43:07.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.974+0000 7fd709756f40 1 -- 172.21.15.172:0/932797690 shutdown_connections 2024-04-14T08:43:07.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.974+0000 7fd709756f40 1 -- 172.21.15.172:0/932797690 >> 172.21.15.172:0/932797690 conn(0x5615f16c93d0 msgr2=0x5615f169d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:07.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.974+0000 7fd709756f40 1 -- 172.21.15.172:0/932797690 shutdown_connections 2024-04-14T08:43:07.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:07.974+0000 7fd709756f40 1 -- 172.21.15.172:0/932797690 wait complete. 2024-04-14T08:43:07.979 INFO:teuthology.orchestra.run.smithi172.stdout:mytag123 2024-04-14T08:43:07.979 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-14T08:43:08.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.062+0000 7fa4b49daf40 1 Processor -- start 2024-04-14T08:43:08.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.062+0000 7fa4b49daf40 1 -- start start 2024-04-14T08:43:08.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b49daf40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563471fc0330 0x563472092a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b49daf40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 0x563471fec250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b49daf40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563472099a80 0x563471fec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b49daf40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56347200a870 con 0x56347200b510 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b49daf40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56347200a5d0 con 0x563471fc0330 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b49daf40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56347200ab10 con 0x563472099a80 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b1a4a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 0x563471fec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b1a4a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 0x563471fec250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59628/0 (socket says 172.21.15.172:59628) 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b1a4a640 1 -- 172.21.15.172:0/4243646901 learned_addr learned my addr 172.21.15.172:0/4243646901 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b1a4a640 1 -- 172.21.15.172:0/4243646901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563472099a80 msgr2=0x563471fec790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b1a4a640 1 --2- 172.21.15.172:0/4243646901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563472099a80 0x563471fec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b1a4a640 1 -- 172.21.15.172:0/4243646901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563471fc0330 msgr2=0x563472092a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b1a4a640 1 --2- 172.21.15.172:0/4243646901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563471fc0330 0x563472092a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b1a4a640 1 -- 172.21.15.172:0/4243646901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347209bdd0 con 0x56347200b510 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b1a4a640 1 --2- 172.21.15.172:0/4243646901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 0x563471fec250 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800a770 tx=0x7fa4a800a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a7f31915f5db44 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b0a48640 1 -- 172.21.15.172:0/4243646901 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8019070 con 0x56347200b510 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b0a48640 1 -- 172.21.15.172:0/4243646901 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a8007600 con 0x56347200b510 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.064+0000 7fa4b0a48640 1 -- 172.21.15.172:0/4243646901 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80079d0 con 0x56347200b510 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.065+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4243646901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 msgr2=0x563471fec250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.065+0000 7fa4b49daf40 1 --2- 172.21.15.172:0/4243646901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 0x563471fec250 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800a770 tx=0x7fa4a800a860 comp rx=0 tx=0).stop 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.065+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4243646901 shutdown_connections 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.065+0000 7fa4b49daf40 1 --2- 172.21.15.172:0/4243646901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563472099a80 0x563471fec790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.065+0000 7fa4b49daf40 1 --2- 172.21.15.172:0/4243646901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 0x563471fec250 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.065+0000 7fa4b49daf40 1 --2- 172.21.15.172:0/4243646901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563471fc0330 0x563472092a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.065+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4243646901 >> 172.21.15.172:0/4243646901 conn(0x563471feda50 msgr2=0x56347207b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.065+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4243646901 shutdown_connections 2024-04-14T08:43:08.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.065+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4243646901 wait complete. 2024-04-14T08:43:08.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.065+0000 7fa4b49daf40 1 Processor -- start 2024-04-14T08:43:08.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.065+0000 7fa4b49daf40 1 -- start start 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b49daf40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563471fc0330 0x563472090560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b49daf40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 0x563472095aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b49daf40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563472099a80 0x563472095fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b49daf40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56347209b5b0 con 0x56347200b510 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b49daf40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563471fc4710 con 0x563472099a80 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b49daf40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x56347209b850 con 0x563471fc0330 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b1a4a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 0x563472095aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b1a4a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 0x563472095aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59636/0 (socket says 172.21.15.172:59636) 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b1a4a640 1 -- 172.21.15.172:0/4010241509 learned_addr learned my addr 172.21.15.172:0/4010241509 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b2a4c640 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563472099a80 0x563472095fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b224b640 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563471fc0330 0x563472090560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b1a4a640 1 -- 172.21.15.172:0/4010241509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563471fc0330 msgr2=0x563472090560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b1a4a640 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563471fc0330 0x563472090560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b1a4a640 1 -- 172.21.15.172:0/4010241509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563472099a80 msgr2=0x563472095fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b1a4a640 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563472099a80 0x563472095fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.066+0000 7fa4b1a4a640 1 -- 172.21.15.172:0/4010241509 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563472091e60 con 0x56347200b510 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.067+0000 7fa4b1a4a640 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 0x563472095aa0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8017040 tx=0x7fa4a8003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.067+0000 7fa4a37fe640 1 -- 172.21.15.172:0/4010241509 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8019050 con 0x56347200b510 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.067+0000 7fa4a37fe640 1 -- 172.21.15.172:0/4010241509 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a8025070 con 0x56347200b510 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.067+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4010241509 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563472090ae0 con 0x56347200b510 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.067+0000 7fa4a37fe640 1 -- 172.21.15.172:0/4010241509 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8014ac0 con 0x56347200b510 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.067+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4010241509 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634720910a0 con 0x56347200b510 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.068+0000 7fa4a37fe640 1 -- 172.21.15.172:0/4010241509 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4a8014c60 con 0x56347200b510 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.068+0000 7fa4a37fe640 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa49406b300 0x7fa49406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.069+0000 7fa4a37fe640 1 -- 172.21.15.172:0/4010241509 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7fa4a808c520 con 0x56347200b510 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.069+0000 7fa4b224b640 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa49406b300 0x7fa49406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.069+0000 7fa4b49daf40 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5634720050d0 0x563472006d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.069+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4010241509 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) -- 0x563472007290 con 0x5634720050d0 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.069+0000 7fa4b2a4c640 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5634720050d0 0x563472006d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.069+0000 7fa4b224b640 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa49406b300 0x7fa49406d7c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4054000 tx=0x7fa4a404c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.069+0000 7fa4b2a4c640 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5634720050d0 0x563472006d50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.070+0000 7fa4b2a4c640 1 -- 172.21.15.172:0/4010241509 <== osd.4 v2:172.21.15.177:6820/4180733146 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fa4ac00c6c0 con 0x5634720050d0 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.070+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4010241509 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5634720050d0 msgr2=0x563472006d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.070+0000 7fa4b49daf40 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x5634720050d0 0x563472006d50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.070+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4010241509 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa49406b300 msgr2=0x7fa49406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.070+0000 7fa4b49daf40 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa49406b300 0x7fa49406d7c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4054000 tx=0x7fa4a404c000 comp rx=0 tx=0).stop 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.070+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4010241509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 msgr2=0x563472095aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.070+0000 7fa4b49daf40 1 --2- 172.21.15.172:0/4010241509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347200b510 0x563472095aa0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8017040 tx=0x7fa4a8003020 comp rx=0 tx=0).stop 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.070+0000 7fa4b2a4c640 1 -- 172.21.15.172:0/4010241509 reap_dead start 2024-04-14T08:43:08.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.070+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4010241509 shutdown_connections 2024-04-14T08:43:08.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.070+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4010241509 >> 172.21.15.172:0/4010241509 conn(0x563471feda50 msgr2=0x563471fed1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.070+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4010241509 shutdown_connections 2024-04-14T08:43:08.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.070+0000 7fa4b49daf40 1 -- 172.21.15.172:0/4010241509 wait complete. 2024-04-14T08:43:08.075 INFO:teuthology.orchestra.run.smithi172.stdout:mytag123 2024-04-14T08:43:08.076 DEBUG:teuthology.orchestra.run.smithi172:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-14T08:43:08.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.158+0000 7f9aa581ef40 1 Processor -- start 2024-04-14T08:43:08.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.158+0000 7f9aa581ef40 1 -- start start 2024-04-14T08:43:08.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa581ef40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 0x565403e22f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa581ef40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565403d949e0 0x565403d7a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa581ef40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565403e20730 0x565403e25410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa581ef40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565403cf33b0 con 0x565403d7b3a0 2024-04-14T08:43:08.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa581ef40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x565403d4d630 con 0x565403e20730 2024-04-14T08:43:08.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa581ef40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x565403e2a0e0 con 0x565403d949e0 2024-04-14T08:43:08.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa304b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 0x565403e22f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa304b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 0x565403e22f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59650/0 (socket says 172.21.15.172:59650) 2024-04-14T08:43:08.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa304b640 1 -- 172.21.15.172:0/4193867665 learned_addr learned my addr 172.21.15.172:0/4193867665 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa284a640 1 --2- 172.21.15.172:0/4193867665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565403d949e0 0x565403d7a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa384c640 1 --2- 172.21.15.172:0/4193867665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565403e20730 0x565403e25410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa304b640 1 -- 172.21.15.172:0/4193867665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565403d949e0 msgr2=0x565403d7a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa304b640 1 --2- 172.21.15.172:0/4193867665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565403d949e0 0x565403d7a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa304b640 1 -- 172.21.15.172:0/4193867665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565403e20730 msgr2=0x565403e25410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa304b640 1 --2- 172.21.15.172:0/4193867665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565403e20730 0x565403e25410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa304b640 1 -- 172.21.15.172:0/4193867665 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565403e222b0 con 0x565403d7b3a0 2024-04-14T08:43:08.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa384c640 1 --2- 172.21.15.172:0/4193867665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565403e20730 0x565403e25410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:08.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.159+0000 7f9aa304b640 1 --2- 172.21.15.172:0/4193867665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 0x565403e22f40 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f9a940095d0 tx=0x7f9a94009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b7bbd36a3a431d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.160+0000 7f9aa1848640 1 -- 172.21.15.172:0/4193867665 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94017070 con 0x565403d7b3a0 2024-04-14T08:43:08.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.160+0000 7f9aa1848640 1 -- 172.21.15.172:0/4193867665 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a94006680 con 0x565403d7b3a0 2024-04-14T08:43:08.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.160+0000 7f9aa1848640 1 -- 172.21.15.172:0/4193867665 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94006a30 con 0x565403d7b3a0 2024-04-14T08:43:08.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.160+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4193867665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 msgr2=0x565403e22f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.160+0000 7f9aa581ef40 1 --2- 172.21.15.172:0/4193867665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 0x565403e22f40 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f9a940095d0 tx=0x7f9a94009a90 comp rx=0 tx=0).stop 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.160+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4193867665 shutdown_connections 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.160+0000 7f9aa581ef40 1 --2- 172.21.15.172:0/4193867665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565403e20730 0x565403e25410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.160+0000 7f9aa581ef40 1 --2- 172.21.15.172:0/4193867665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565403d949e0 0x565403d7a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.160+0000 7f9aa581ef40 1 --2- 172.21.15.172:0/4193867665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 0x565403e22f40 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.160+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4193867665 >> 172.21.15.172:0/4193867665 conn(0x565403d78a50 msgr2=0x565403d4ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.160+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4193867665 shutdown_connections 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.160+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4193867665 wait complete. 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.161+0000 7f9aa581ef40 1 Processor -- start 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.161+0000 7f9aa581ef40 1 -- start start 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.161+0000 7f9aa581ef40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 0x565403d4b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.161+0000 7f9aa581ef40 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565403d949e0 0x565403e2ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.161+0000 7f9aa581ef40 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565403e20730 0x565403e17580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.161+0000 7f9aa581ef40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565403e29d10 con 0x565403d7b3a0 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.161+0000 7f9aa581ef40 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x565403e1f270 con 0x565403e20730 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.161+0000 7f9aa581ef40 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x565403e2a330 con 0x565403d949e0 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa304b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 0x565403d4b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa304b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 0x565403d4b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59652/0 (socket says 172.21.15.172:59652) 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa304b640 1 -- 172.21.15.172:0/4126554852 learned_addr learned my addr 172.21.15.172:0/4126554852 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa284a640 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565403d949e0 0x565403e2ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa384c640 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565403e20730 0x565403e17580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa304b640 1 -- 172.21.15.172:0/4126554852 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565403d949e0 msgr2=0x565403e2ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa304b640 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565403d949e0 0x565403e2ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa304b640 1 -- 172.21.15.172:0/4126554852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565403e20730 msgr2=0x565403e17580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa304b640 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565403e20730 0x565403e17580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa304b640 1 -- 172.21.15.172:0/4126554852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565403e197c0 con 0x565403d7b3a0 2024-04-14T08:43:08.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa304b640 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 0x565403d4b310 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009f90 tx=0x7f9a94003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa0846640 1 -- 172.21.15.172:0/4126554852 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94017040 con 0x565403d7b3a0 2024-04-14T08:43:08.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa0846640 1 -- 172.21.15.172:0/4126554852 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a94022070 con 0x565403d7b3a0 2024-04-14T08:43:08.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4126554852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565403e18250 con 0x565403d7b3a0 2024-04-14T08:43:08.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa0846640 1 -- 172.21.15.172:0/4126554852 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9401d680 con 0x565403d7b3a0 2024-04-14T08:43:08.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.162+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4126554852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565403d90b80 con 0x565403d7b3a0 2024-04-14T08:43:08.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.164+0000 7f9aa0846640 1 -- 172.21.15.172:0/4126554852 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a9401d8e0 con 0x565403d7b3a0 2024-04-14T08:43:08.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.164+0000 7f9aa0846640 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a9006b2d0 0x7f9a9006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.164+0000 7f9aa0846640 1 -- 172.21.15.172:0/4126554852 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f9a9408ba90 con 0x565403d7b3a0 2024-04-14T08:43:08.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.164+0000 7f9aa284a640 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a9006b2d0 0x7f9a9006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.164+0000 7f9aa581ef40 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x565403e19c80 0x565403e1a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.164+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4126554852 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) -- 0x565403e1a5d0 con 0x565403e19c80 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.164+0000 7f9aa384c640 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x565403e19c80 0x565403e1a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa284a640 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a9006b2d0 0x7f9a9006d790 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x565403e25fb0 tx=0x7f9a98003650 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa384c640 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x565403e19c80 0x565403e1a090 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa384c640 1 -- 172.21.15.172:0/4126554852 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f9a9c05a040 con 0x565403e19c80 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x565403e19c80 msgr2=0x565403e1a090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa581ef40 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x565403e19c80 0x565403e1a090 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a9006b2d0 msgr2=0x7f9a9006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa581ef40 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9a9006b2d0 0x7f9a9006d790 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x565403e25fb0 tx=0x7f9a98003650 comp rx=0 tx=0).stop 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 msgr2=0x565403d4b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa581ef40 1 --2- 172.21.15.172:0/4126554852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565403d7b3a0 0x565403d4b310 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009f90 tx=0x7f9a94003020 comp rx=0 tx=0).stop 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa384c640 1 -- 172.21.15.172:0/4126554852 reap_dead start 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4126554852 shutdown_connections 2024-04-14T08:43:08.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4126554852 >> 172.21.15.172:0/4126554852 conn(0x565403d78a50 msgr2=0x565403e1d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4126554852 shutdown_connections 2024-04-14T08:43:08.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.165+0000 7f9aa581ef40 1 -- 172.21.15.172:0/4126554852 wait complete. 2024-04-14T08:43:08.171 INFO:teuthology.orchestra.run.smithi172.stdout:mytag123 2024-04-14T08:43:08.172 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-14T08:43:08.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.316+0000 7fb20591e640 1 Processor -- start 2024-04-14T08:43:08.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.316+0000 7fb20591e640 1 -- start start 2024-04-14T08:43:08.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.317+0000 7fb20591e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb2000d49c0 0x7fb2000d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.317+0000 7fb20591e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb200160f30 0x7fb2001613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.317+0000 7fb20591e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2000cf0f0 con 0x7fb2000d0cc0 2024-04-14T08:43:08.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.317+0000 7fb20591e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb2000ceec0 con 0x7fb200160f30 2024-04-14T08:43:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.317+0000 7fb20591e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb2000cfd00 con 0x7fb2000d49c0 2024-04-14T08:43:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.317+0000 7fb1ff7fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb200160f30 0x7fb2001613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.318+0000 7fb1feffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb2000d0cc0 0x7fb2000d10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45828/0 (socket says 172.21.15.172:45828) 2024-04-14T08:43:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.318+0000 7fb1feffd640 1 -- 172.21.15.172:0/3506725084 learned_addr learned my addr 172.21.15.172:0/3506725084 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.318+0000 7fb1fe7fc640 1 --2- 172.21.15.172:0/3506725084 >> v2:172.21.15.177:3300/0 conn(0x7fb2000d49c0 0x7fb2000d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.318+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/3506725084 >> v2:172.21.15.177:3300/0 conn(0x7fb2000d49c0 msgr2=0x7fb2000d4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.318+0000 7fb1ff7fe640 1 --2- 172.21.15.172:0/3506725084 >> v2:172.21.15.177:3300/0 conn(0x7fb2000d49c0 0x7fb2000d4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.318+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/3506725084 >> v1:172.21.15.172:6789/0 conn(0x7fb2000d0cc0 legacy=0x7fb2000d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.318+0000 7fb1fdffb640 1 -- 172.21.15.172:0/3506725084 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4162937551 0 0) 0x7fb2000cf0f0 con 0x7fb2000d0cc0 2024-04-14T08:43:08.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.318+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/3506725084 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2000cf3d0 con 0x7fb200160f30 2024-04-14T08:43:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.319+0000 7fb1ff7fe640 1 --2- 172.21.15.172:0/3506725084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb200160f30 0x7fb2001613a0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb20006bee0 tx=0x7fb1f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab2c20fd0fbbcab server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.319+0000 7fb1fdffb640 1 -- 172.21.15.172:0/3506725084 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8035020 con 0x7fb200160f30 2024-04-14T08:43:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.319+0000 7fb1fdffb640 1 -- 172.21.15.172:0/3506725084 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f8031830 con 0x7fb200160f30 2024-04-14T08:43:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.319+0000 7fb1fdffb640 1 -- 172.21.15.172:0/3506725084 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8031c00 con 0x7fb200160f30 2024-04-14T08:43:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.320+0000 7fb20591e640 1 -- 172.21.15.172:0/3506725084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb200160f30 msgr2=0x7fb2001613a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.320+0000 7fb20591e640 1 --2- 172.21.15.172:0/3506725084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb200160f30 0x7fb2001613a0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb20006bee0 tx=0x7fb1f802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.320+0000 7fb20591e640 1 -- 172.21.15.172:0/3506725084 shutdown_connections 2024-04-14T08:43:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.320+0000 7fb20591e640 1 --2- 172.21.15.172:0/3506725084 >> v2:172.21.15.177:3300/0 conn(0x7fb2000d49c0 0x7fb2000d4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.320+0000 7fb20591e640 1 --2- 172.21.15.172:0/3506725084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb200160f30 0x7fb2001613a0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.320+0000 7fb20591e640 1 -- 172.21.15.172:0/3506725084 >> 172.21.15.172:0/3506725084 conn(0x7fb20009d580 msgr2=0x7fb2000d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.320+0000 7fb20591e640 1 -- 172.21.15.172:0/3506725084 shutdown_connections 2024-04-14T08:43:08.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.320+0000 7fb20591e640 1 -- 172.21.15.172:0/3506725084 wait complete. 2024-04-14T08:43:08.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.321+0000 7fb20591e640 1 Processor -- start 2024-04-14T08:43:08.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.321+0000 7fb20591e640 1 -- start start 2024-04-14T08:43:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb20591e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2000d0cc0 0x7fb20006c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb20591e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2000d49c0 0x7fb2000c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb20591e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb200160f30 0x7fb2000c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb20591e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb2000d32b0 con 0x7fb200160f30 2024-04-14T08:43:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb20591e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2000cf990 con 0x7fb2000d49c0 2024-04-14T08:43:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb20591e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb2000d0190 con 0x7fb2000d0cc0 2024-04-14T08:43:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb1ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb200160f30 0x7fb2000c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb1ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb200160f30 0x7fb2000c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59664/0 (socket says 172.21.15.172:59664) 2024-04-14T08:43:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/2953065964 learned_addr learned my addr 172.21.15.172:0/2953065964 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb1feffd640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2000d0cc0 0x7fb20006c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/2953065964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2000d0cc0 msgr2=0x7fb20006c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb1fe7fc640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2000d49c0 0x7fb2000c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb1ff7fe640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2000d0cc0 0x7fb20006c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/2953065964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2000d49c0 msgr2=0x7fb2000c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb1ff7fe640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2000d49c0 0x7fb2000c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/2953065964 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2000a0f00 con 0x7fb200160f30 2024-04-14T08:43:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.322+0000 7fb1ff7fe640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb200160f30 0x7fb2000c8950 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8034040 tx=0x7fb1f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.323+0000 7fb1f7fff640 1 -- 172.21.15.172:0/2953065964 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8040070 con 0x7fb200160f30 2024-04-14T08:43:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.323+0000 7fb1f7fff640 1 -- 172.21.15.172:0/2953065964 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f803a850 con 0x7fb200160f30 2024-04-14T08:43:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.323+0000 7fb20591e640 1 -- 172.21.15.172:0/2953065964 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2000a1180 con 0x7fb200160f30 2024-04-14T08:43:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.323+0000 7fb1f7fff640 1 -- 172.21.15.172:0/2953065964 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8035040 con 0x7fb200160f30 2024-04-14T08:43:08.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.323+0000 7fb20591e640 1 -- 172.21.15.172:0/2953065964 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2000a1740 con 0x7fb200160f30 2024-04-14T08:43:08.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.324+0000 7fb1f7fff640 1 -- 172.21.15.172:0/2953065964 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1f8038020 con 0x7fb200160f30 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.324+0000 7fb1f7fff640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb1e006c8f0 0x7fb1e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.325+0000 7fb1f7fff640 1 -- 172.21.15.172:0/2953065964 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7fb1f80ad5b0 con 0x7fb200160f30 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.325+0000 7fb1feffd640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb1e006c8f0 0x7fb1e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.325+0000 7fb1feffd640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb1e006c8f0 0x7fb1e006edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00012e0 tx=0x7fb1f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.325+0000 7fb2087ee740 1 Processor -- start 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.325+0000 7fb2087ee740 1 -- start start 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.326+0000 7fb2087ee740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 0x55fc57db1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.326+0000 7fb2087ee740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57dded50 0x55fc57de1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.326+0000 7fb2087ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57db1e50 0x55fc57de1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.326+0000 7fb2087ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fc57d1b800 con 0x55fc57db1e50 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.326+0000 7fb2087ee740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fc57cc37f0 con 0x55fc57dde970 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.326+0000 7fb2087ee740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55fc57d177b0 con 0x55fc57dded50 2024-04-14T08:43:08.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.326+0000 7fb1ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57db1e50 0x55fc57de1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.326+0000 7fb1ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57db1e50 0x55fc57de1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59678/0 (socket says 172.21.15.172:59678) 2024-04-14T08:43:08.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.326+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/1938506467 learned_addr learned my addr 172.21.15.172:0/1938506467 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.327+0000 7fb1feffd640 1 --2- 172.21.15.172:0/1938506467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 0x55fc57db1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.327+0000 7fb1fe7fc640 1 --2- 172.21.15.172:0/1938506467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57dded50 0x55fc57de1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.327+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/1938506467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57dded50 msgr2=0x55fc57de1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.327+0000 7fb1ff7fe640 1 --2- 172.21.15.172:0/1938506467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57dded50 0x55fc57de1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.327+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/1938506467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 msgr2=0x55fc57db1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.327+0000 7fb1ff7fe640 1 --2- 172.21.15.172:0/1938506467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 0x55fc57db1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.327+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/1938506467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc57af97d0 con 0x55fc57db1e50 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.327+0000 7fb1ff7fe640 1 --2- 172.21.15.172:0/1938506467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57db1e50 0x55fc57de1c40 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fb1f80ab8a0 tx=0x7fb1f807ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fae9da4833797f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.327+0000 7fb1f57fa640 1 -- 172.21.15.172:0/1938506467 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f807f4b0 con 0x55fc57db1e50 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.327+0000 7fb1f57fa640 1 -- 172.21.15.172:0/1938506467 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f8083ce0 con 0x55fc57db1e50 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.327+0000 7fb1f57fa640 1 -- 172.21.15.172:0/1938506467 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f807f120 con 0x55fc57db1e50 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.328+0000 7fb2087ee740 1 -- 172.21.15.172:0/1938506467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57db1e50 msgr2=0x55fc57de1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.328+0000 7fb2087ee740 1 --2- 172.21.15.172:0/1938506467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57db1e50 0x55fc57de1c40 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fb1f80ab8a0 tx=0x7fb1f807ab60 comp rx=0 tx=0).stop 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.328+0000 7fb2087ee740 1 -- 172.21.15.172:0/1938506467 shutdown_connections 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.328+0000 7fb2087ee740 1 --2- 172.21.15.172:0/1938506467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57db1e50 0x55fc57de1c40 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.328+0000 7fb2087ee740 1 --2- 172.21.15.172:0/1938506467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57dded50 0x55fc57de1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.328+0000 7fb2087ee740 1 --2- 172.21.15.172:0/1938506467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 0x55fc57db1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.328+0000 7fb2087ee740 1 -- 172.21.15.172:0/1938506467 >> 172.21.15.172:0/1938506467 conn(0x55fc57db3b80 msgr2=0x55fc57dfc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.328+0000 7fb2087ee740 1 -- 172.21.15.172:0/1938506467 shutdown_connections 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.328+0000 7fb2087ee740 1 -- 172.21.15.172:0/1938506467 wait complete. 2024-04-14T08:43:08.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.328+0000 7fb2087ee740 1 -- start start 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.328+0000 7fb2087ee740 1 Processor -- start 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.329+0000 7fb2087ee740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 0x55fc57dced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.329+0000 7fb2087ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 0x55fc57de51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.329+0000 7fb2087ee740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57e0c190 0x55fc57ddf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.329+0000 7fb2087ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fc57d1b800 con 0x55fc57dded50 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.329+0000 7fb2087ee740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fc57cd6d00 con 0x55fc57dde970 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.329+0000 7fb2087ee740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55fc57db6120 con 0x55fc57e0c190 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.329+0000 7fb2087ee740 10 client.0 _ll_register_callbacks cb 0x55fc57cd66f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.329+0000 7fb1fe7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 0x55fc57de51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.329+0000 7fb1fe7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 0x55fc57de51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59684/0 (socket says 172.21.15.172:59684) 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.329+0000 7fb1fe7fc640 1 -- 172.21.15.172:0/2235656357 learned_addr learned my addr 172.21.15.172:0/2235656357 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.329+0000 7fb1feffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 0x55fc57dced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.329+0000 7fb1ff7fe640 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57e0c190 0x55fc57ddf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1fe7fc640 1 -- 172.21.15.172:0/2235656357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57e0c190 msgr2=0x55fc57ddf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1fe7fc640 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57e0c190 0x55fc57ddf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1fe7fc640 1 -- 172.21.15.172:0/2235656357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 msgr2=0x55fc57dced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1fe7fc640 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 0x55fc57dced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1fe7fc640 1 -- 172.21.15.172:0/2235656357 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc57a2f3f0 con 0x55fc57dded50 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1feffd640 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 0x55fc57dced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1fe7fc640 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 0x55fc57de51d0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009f20 tx=0x7fb1e80116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24d8fd97b0b4994 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1de7fc640 1 -- 172.21.15.172:0/2235656357 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8019070 con 0x55fc57dded50 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1de7fc640 1 -- 172.21.15.172:0/2235656357 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7fb1e8015870 con 0x55fc57dded50 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb2087ee740 1 -- 172.21.15.172:0/2235656357 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f8083ce0 con 0x55fc57dded50 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1de7fc640 1 -- 172.21.15.172:0/2235656357 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fb1e8024050 con 0x55fc57dded50 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb2087ee740 10 client.9501 fetch_fsmap learned FSMap version 93 2024-04-14T08:43:08.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb2087ee740 1 -- 172.21.15.172:0/2235656357 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) -- 0x55fc579ef0f0 con 0x55fc57dded50 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1de7fc640 1 -- 172.21.15.172:0/2235656357 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f8083ce0 con 0x55fc57dded50 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb1de7fc640 1 -- 172.21.15.172:0/2235656357 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 93) ==== 3494+0+0 (secure 0 0 0) 0x7fb1e8027020 con 0x55fc57dded50 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb2087ee740 10 client.9501 fetch_fsmap finished waiting for FSMap version 93 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb2087ee740 10 client.9501 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb2087ee740 20 client.9501 populate_metadata read hostname 'smithi172' 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb2087ee740 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55fc57de3ed0 0x55fc57de4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb2087ee740 4 client.9501 mds_command: new command op to 9048 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.330+0000 7fb2087ee740 1 -- 172.21.15.172:0/2235656357 --> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc57d28d40 con 0x55fc57de3ed0 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.331+0000 7fb1feffd640 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55fc57de3ed0 0x55fc57de4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.331+0000 7fb1de7fc640 10 client.9501 ms_handle_connect on v2:172.21.15.172:6835/744853572 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.331+0000 7fb1feffd640 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55fc57de3ed0 0x55fc57de4380 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f70391ce3b1142 server_cookie=a9e3d92fb402d1af in_seq=0 out_seq=0 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.331+0000 7fb1de7fc640 1 -- 172.21.15.172:0/2235656357 <== mds.0 v2:172.21.15.172:6835/744853572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc57d28d40 con 0x55fc57de3ed0 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.331+0000 7fb1de7fc640 10 client.9501 handle_command_reply: tid=0 2024-04-14T08:43:08.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.331+0000 7fb2087ee740 1 client.9501 shutdown 2024-04-14T08:43:08.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.332+0000 7fb2087ee740 1 -- 172.21.15.172:0/2235656357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 msgr2=0x55fc57de51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.332+0000 7fb2087ee740 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 0x55fc57de51d0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009f20 tx=0x7fb1e80116f0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.332+0000 7fb2087ee740 1 -- 172.21.15.172:0/2235656357 shutdown_connections 2024-04-14T08:43:08.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.332+0000 7fb2087ee740 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57e0c190 0x55fc57ddf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.332+0000 7fb2087ee740 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55fc57de3ed0 0x55fc57de4380 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.332+0000 7fb2087ee740 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 0x55fc57de51d0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.332+0000 7fb2087ee740 1 --2- 172.21.15.172:0/2235656357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 0x55fc57dced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.332+0000 7fb2087ee740 1 -- 172.21.15.172:0/2235656357 >> 172.21.15.172:0/2235656357 conn(0x55fc57db1e50 msgr2=0x55fc57db5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.332+0000 7fb1de7fc640 0 client.9501 ms_handle_reset on v2:172.21.15.172:6835/744853572 2024-04-14T08:43:08.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.332+0000 7fb2087ee740 1 -- 172.21.15.172:0/2235656357 shutdown_connections 2024-04-14T08:43:08.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.332+0000 7fb2087ee740 1 -- 172.21.15.172:0/2235656357 wait complete. 2024-04-14T08:43:08.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.332+0000 7fb2087ee740 20 client.9501 trim_cache size 0 max 16384 2024-04-14T08:43:08.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.347+0000 7fb2087ee740 1 Processor -- start 2024-04-14T08:43:08.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.347+0000 7fb2087ee740 1 -- start start 2024-04-14T08:43:08.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.347+0000 7fb2087ee740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 0x55fc57de4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.347+0000 7fb2087ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 0x55fc57de3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.347+0000 7fb2087ee740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57e0c190 0x55fc57de4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.347+0000 7fb2087ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb200200e90 con 0x55fc57dded50 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.347+0000 7fb2087ee740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001fff30 con 0x55fc57dde970 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.347+0000 7fb2087ee740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb2001f81d0 con 0x55fc57e0c190 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.348+0000 7fb1fe7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 0x55fc57de3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.348+0000 7fb1fe7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 0x55fc57de3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59686/0 (socket says 172.21.15.172:59686) 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.348+0000 7fb1fe7fc640 1 -- 172.21.15.172:0/842470746 learned_addr learned my addr 172.21.15.172:0/842470746 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.348+0000 7fb1fe7fc640 1 -- 172.21.15.172:0/842470746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57e0c190 msgr2=0x55fc57de4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.348+0000 7fb1fe7fc640 1 --2- 172.21.15.172:0/842470746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57e0c190 0x55fc57de4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.348+0000 7fb1fe7fc640 1 -- 172.21.15.172:0/842470746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 msgr2=0x55fc57de4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.348+0000 7fb1fe7fc640 1 --2- 172.21.15.172:0/842470746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 0x55fc57de4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.348+0000 7fb1fe7fc640 1 -- 172.21.15.172:0/842470746 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc57dac620 con 0x55fc57dded50 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.348+0000 7fb1fe7fc640 1 --2- 172.21.15.172:0/842470746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 0x55fc57de3ed0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8001eb0 tx=0x7fb1e8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5f19609d7bc0dc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.348+0000 7fb1f4ff9640 1 -- 172.21.15.172:0/842470746 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8000e60 con 0x55fc57dded50 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.348+0000 7fb1f4ff9640 1 -- 172.21.15.172:0/842470746 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e8005cb0 con 0x55fc57dded50 2024-04-14T08:43:08.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.348+0000 7fb1f4ff9640 1 -- 172.21.15.172:0/842470746 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8019070 con 0x55fc57dded50 2024-04-14T08:43:08.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.349+0000 7fb2087ee740 1 -- 172.21.15.172:0/842470746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 msgr2=0x55fc57de3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.349+0000 7fb2087ee740 1 --2- 172.21.15.172:0/842470746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 0x55fc57de3ed0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8001eb0 tx=0x7fb1e8016000 comp rx=0 tx=0).stop 2024-04-14T08:43:08.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.349+0000 7fb2087ee740 1 -- 172.21.15.172:0/842470746 shutdown_connections 2024-04-14T08:43:08.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.349+0000 7fb2087ee740 1 --2- 172.21.15.172:0/842470746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57e0c190 0x55fc57de4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.349+0000 7fb2087ee740 1 --2- 172.21.15.172:0/842470746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57dded50 0x55fc57de3ed0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.349+0000 7fb2087ee740 1 --2- 172.21.15.172:0/842470746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dde970 0x55fc57de4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.349+0000 7fb2087ee740 1 -- 172.21.15.172:0/842470746 >> 172.21.15.172:0/842470746 conn(0x55fc57db1e50 msgr2=0x55fc57dcebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.349+0000 7fb2087ee740 1 -- 172.21.15.172:0/842470746 shutdown_connections 2024-04-14T08:43:08.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.349+0000 7fb2087ee740 1 -- 172.21.15.172:0/842470746 wait complete. 2024-04-14T08:43:08.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.350+0000 7fb2087ee740 1 -- start start 2024-04-14T08:43:08.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.350+0000 7fb2087ee740 1 Processor -- start 2024-04-14T08:43:08.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.350+0000 7fb2087ee740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57dde970 0x55fc57e78ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.350+0000 7fb2087ee740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dded50 0x55fc57de3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.350+0000 7fb2087ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57e0c190 0x55fc57de4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.350+0000 7fb2087ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fc57e00210 con 0x55fc57e0c190 2024-04-14T08:43:08.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.350+0000 7fb2087ee740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001ffa50 con 0x55fc57dded50 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.350+0000 7fb2087ee740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x55fc57de2920 con 0x55fc57dde970 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.350+0000 7fb2087ee740 10 client.0 _ll_register_callbacks cb 0x7fb1e8009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57e0c190 0x55fc57de4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57e0c190 0x55fc57de4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59702/0 (socket says 172.21.15.172:59702) 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1feffd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57dde970 0x55fc57e78ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/1392322772 learned_addr learned my addr 172.21.15.172:0/1392322772 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1fe7fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dded50 0x55fc57de3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/1392322772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57dde970 msgr2=0x55fc57e78ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1ff7fe640 1 --2- 172.21.15.172:0/1392322772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57dde970 0x55fc57e78ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/1392322772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dded50 msgr2=0x55fc57de3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1ff7fe640 1 --2- 172.21.15.172:0/1392322772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dded50 0x55fc57de3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1ff7fe640 1 -- 172.21.15.172:0/1392322772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc57dff4f0 con 0x55fc57e0c190 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1ff7fe640 1 --2- 172.21.15.172:0/1392322772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57e0c190 0x55fc57de4410 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fb1f807b2c0 tx=0x7fb1f8031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afbb5c7dacc3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1de7fc640 1 -- 172.21.15.172:0/1392322772 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8083ce0 con 0x55fc57e0c190 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1de7fc640 1 -- 172.21.15.172:0/1392322772 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7fb1f8077770 con 0x55fc57e0c190 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb2087ee740 1 -- 172.21.15.172:0/1392322772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e8005cb0 con 0x55fc57e0c190 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1de7fc640 1 -- 172.21.15.172:0/1392322772 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fb1f803ac40 con 0x55fc57e0c190 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb2087ee740 10 client.9507 fetch_fsmap learned FSMap version 93 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb2087ee740 1 -- 172.21.15.172:0/1392322772 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) -- 0x55fc57db1130 con 0x55fc57e0c190 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.351+0000 7fb1de7fc640 1 -- 172.21.15.172:0/1392322772 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e8005cb0 con 0x55fc57e0c190 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.352+0000 7fb1de7fc640 1 -- 172.21.15.172:0/1392322772 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 93) ==== 3494+0+0 (secure 0 0 0) 0x7fb1f807a550 con 0x55fc57e0c190 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.352+0000 7fb2087ee740 10 client.9507 fetch_fsmap finished waiting for FSMap version 93 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.352+0000 7fb2087ee740 10 client.9507 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.352+0000 7fb2087ee740 20 client.9507 populate_metadata read hostname 'smithi172' 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.352+0000 7fb2087ee740 1 --2- 172.21.15.172:0/1392322772 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55fc57df3240 0x55fc57e794a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.352+0000 7fb2087ee740 4 client.9507 mds_command: new command op to 9048 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-14T08:43:08.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.352+0000 7fb2087ee740 1 -- 172.21.15.172:0/1392322772 --> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc57df49b0 con 0x55fc57df3240 2024-04-14T08:43:08.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.352+0000 7fb1feffd640 1 --2- 172.21.15.172:0/1392322772 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55fc57df3240 0x55fc57e794a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.352+0000 7fb1de7fc640 10 client.9507 ms_handle_connect on v2:172.21.15.172:6835/744853572 2024-04-14T08:43:08.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.352+0000 7fb1feffd640 1 --2- 172.21.15.172:0/1392322772 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55fc57df3240 0x55fc57e794a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=792b7ccbc7703b81 server_cookie=ec40e2b27ec6a392 in_seq=0 out_seq=0 2024-04-14T08:43:08.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb1de7fc640 1 -- 172.21.15.172:0/1392322772 <== mds.0 v2:172.21.15.172:6835/744853572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc57df49b0 con 0x55fc57df3240 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb1de7fc640 10 client.9507 handle_command_reply: tid=0 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb2087ee740 1 client.9507 shutdown 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb2087ee740 1 -- 172.21.15.172:0/1392322772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57e0c190 msgr2=0x55fc57de4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb2087ee740 1 --2- 172.21.15.172:0/1392322772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57e0c190 0x55fc57de4410 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fb1f807b2c0 tx=0x7fb1f8031f90 comp rx=0 tx=0).stop 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb2087ee740 1 -- 172.21.15.172:0/1392322772 shutdown_connections 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb2087ee740 1 --2- 172.21.15.172:0/1392322772 >> [v2:172.21.15.172:6835/744853572,v1:172.21.15.172:6837/744853572] conn(0x55fc57df3240 0x55fc57e794a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb2087ee740 1 --2- 172.21.15.172:0/1392322772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fc57e0c190 0x55fc57de4410 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb1de7fc640 0 client.9507 ms_handle_reset on v2:172.21.15.172:6835/744853572 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb2087ee740 1 --2- 172.21.15.172:0/1392322772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc57dded50 0x55fc57de3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb2087ee740 1 --2- 172.21.15.172:0/1392322772 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55fc57dde970 0x55fc57e78ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb2087ee740 1 -- 172.21.15.172:0/1392322772 >> 172.21.15.172:0/1392322772 conn(0x55fc57db1e50 msgr2=0x55fc57db7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb2087ee740 1 -- 172.21.15.172:0/1392322772 shutdown_connections 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb2087ee740 1 -- 172.21.15.172:0/1392322772 wait complete. 2024-04-14T08:43:08.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.366+0000 7fb2087ee740 20 client.9507 trim_cache size 0 max 16384 2024-04-14T08:43:08.369 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:43:08.369 INFO:teuthology.orchestra.run.smithi172.stdout: "message": "", 2024-04-14T08:43:08.369 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0 2024-04-14T08:43:08.369 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:43:08.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.367+0000 7fb20591e640 1 -- 172.21.15.172:0/2953065964 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb1e006c8f0 msgr2=0x7fb1e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.367+0000 7fb20591e640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb1e006c8f0 0x7fb1e006edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00012e0 tx=0x7fb1f0009040 comp rx=0 tx=0).stop 2024-04-14T08:43:08.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.367+0000 7fb20591e640 1 -- 172.21.15.172:0/2953065964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb200160f30 msgr2=0x7fb2000c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.367+0000 7fb20591e640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb200160f30 0x7fb2000c8950 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8034040 tx=0x7fb1f80046c0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.367+0000 7fb20591e640 1 -- 172.21.15.172:0/2953065964 shutdown_connections 2024-04-14T08:43:08.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.367+0000 7fb20591e640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb1e006c8f0 0x7fb1e006edb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.367+0000 7fb20591e640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb200160f30 0x7fb2000c8950 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.367+0000 7fb20591e640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2000d49c0 0x7fb2000c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.367+0000 7fb20591e640 1 --2- 172.21.15.172:0/2953065964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2000d0cc0 0x7fb20006c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.367+0000 7fb20591e640 1 -- 172.21.15.172:0/2953065964 >> 172.21.15.172:0/2953065964 conn(0x7fb20009d580 msgr2=0x7fb200160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.367+0000 7fb20591e640 1 -- 172.21.15.172:0/2953065964 shutdown_connections 2024-04-14T08:43:08.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.367+0000 7fb20591e640 1 -- 172.21.15.172:0/2953065964 wait complete. 2024-04-14T08:43:08.378 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:43:08.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.480+0000 7f50f96b7640 1 Processor -- start 2024-04-14T08:43:08.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.480+0000 7f50f96b7640 1 -- start start 2024-04-14T08:43:08.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.481+0000 7f50f96b7640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50f40b23e0 0x7f50f4140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.481+0000 7f50f96b7640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f415b620 con 0x7f50f415c380 2024-04-14T08:43:08.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.481+0000 7f50f96b7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f41414a0 con 0x7f50f413df30 2024-04-14T08:43:08.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.481+0000 7f50f96b7640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f50f40cf9b0 con 0x7f50f40b23e0 2024-04-14T08:43:08.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.482+0000 7f50ebfff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f50f40b23e0 0x7f50f4140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.482+0000 7f50f37fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f50f415c380 0x7f50f4140cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:45836/0 (socket says 172.21.15.172:45836) 2024-04-14T08:43:08.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.482+0000 7f50f37fe640 1 -- 172.21.15.172:0/2000288511 learned_addr learned my addr 172.21.15.172:0/2000288511 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:08.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.482+0000 7f50ebfff640 1 -- 172.21.15.172:0/2000288511 >> v1:172.21.15.177:6789/0 conn(0x7f50f413df30 legacy=0x7f50f40b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.482+0000 7f50ebfff640 1 -- 172.21.15.172:0/2000288511 >> v1:172.21.15.172:6789/0 conn(0x7f50f415c380 legacy=0x7f50f4140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.482+0000 7f50ebfff640 1 -- 172.21.15.172:0/2000288511 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f415b8a0 con 0x7f50f40b23e0 2024-04-14T08:43:08.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.482+0000 7f50f27fc640 1 -- 172.21.15.172:0/2000288511 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2551613708 0 0) 0x7f50ec001490 con 0x7f50f415c380 2024-04-14T08:43:08.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.483+0000 7f50ebfff640 1 --2- 172.21.15.172:0/2000288511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f40b23e0 0x7f50f4140770 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0097a0 tx=0x7f50dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daef4831bb1fe584 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.483+0000 7f50f27fc640 1 -- 172.21.15.172:0/2000288511 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc035020 con 0x7f50f40b23e0 2024-04-14T08:43:08.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.483+0000 7f50f27fc640 1 -- 172.21.15.172:0/2000288511 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50dc031830 con 0x7f50f40b23e0 2024-04-14T08:43:08.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.483+0000 7f50f27fc640 1 -- 172.21.15.172:0/2000288511 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc031c00 con 0x7f50f40b23e0 2024-04-14T08:43:08.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.484+0000 7f50f96b7640 1 -- 172.21.15.172:0/2000288511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f40b23e0 msgr2=0x7f50f4140770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.484+0000 7f50f96b7640 1 --2- 172.21.15.172:0/2000288511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f40b23e0 0x7f50f4140770 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0097a0 tx=0x7f50dc02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.484+0000 7f50f96b7640 1 -- 172.21.15.172:0/2000288511 shutdown_connections 2024-04-14T08:43:08.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.484+0000 7f50f96b7640 1 --2- 172.21.15.172:0/2000288511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f40b23e0 0x7f50f4140770 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.484+0000 7f50f96b7640 1 -- 172.21.15.172:0/2000288511 >> 172.21.15.172:0/2000288511 conn(0x7f50f406b520 msgr2=0x7f50f406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.484+0000 7f50f96b7640 1 -- 172.21.15.172:0/2000288511 shutdown_connections 2024-04-14T08:43:08.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.484+0000 7f50f96b7640 1 -- 172.21.15.172:0/2000288511 wait complete. 2024-04-14T08:43:08.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.485+0000 7f50f96b7640 1 Processor -- start 2024-04-14T08:43:08.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.485+0000 7f50f96b7640 1 -- start start 2024-04-14T08:43:08.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f96b7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50f40b23e0 0x7f50f41399d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f96b7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f413df30 0x7f50f40b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f96b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f415c380 0x7f50f40b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f96b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f50f413e310 con 0x7f50f415c380 2024-04-14T08:43:08.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f96b7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50f415be60 con 0x7f50f40b23e0 2024-04-14T08:43:08.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f96b7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f50f4165fa0 con 0x7f50f413df30 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f415c380 0x7f50f40b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f415c380 0x7f50f40b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59712/0 (socket says 172.21.15.172:59712) 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f37fe640 1 -- 172.21.15.172:0/2703045733 learned_addr learned my addr 172.21.15.172:0/2703045733 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f2ffd640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50f40b23e0 0x7f50f41399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50ebfff640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f413df30 0x7f50f40b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f37fe640 1 -- 172.21.15.172:0/2703045733 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f413df30 msgr2=0x7f50f40b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f37fe640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f413df30 0x7f50f40b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f37fe640 1 -- 172.21.15.172:0/2703045733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50f40b23e0 msgr2=0x7f50f41399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f37fe640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50f40b23e0 0x7f50f41399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f37fe640 1 -- 172.21.15.172:0/2703045733 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f414df50 con 0x7f50f415c380 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50f37fe640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f415c380 0x7f50f40b4010 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00c6d0 tx=0x7f50ec00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50eb7fe640 1 -- 172.21.15.172:0/2703045733 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec018070 con 0x7f50f415c380 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.486+0000 7f50eb7fe640 1 -- 172.21.15.172:0/2703045733 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec004d60 con 0x7f50f415c380 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.487+0000 7f50f96b7640 1 -- 172.21.15.172:0/2703045733 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f414e230 con 0x7f50f415c380 2024-04-14T08:43:08.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.487+0000 7f50eb7fe640 1 -- 172.21.15.172:0/2703045733 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00a880 con 0x7f50f415c380 2024-04-14T08:43:08.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.487+0000 7f50f96b7640 1 -- 172.21.15.172:0/2703045733 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f40b4550 con 0x7f50f415c380 2024-04-14T08:43:08.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.488+0000 7f50f96b7640 1 -- 172.21.15.172:0/2703045733 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f4068590 con 0x7f50f415c380 2024-04-14T08:43:08.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.488+0000 7f50eb7fe640 1 -- 172.21.15.172:0/2703045733 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50ec01b070 con 0x7f50f415c380 2024-04-14T08:43:08.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.489+0000 7f50eb7fe640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f50d406c8f0 0x7f50d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.489+0000 7f50eb7fe640 1 -- 172.21.15.172:0/2703045733 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6187+0+0 (secure 0 0 0) 0x7f50ec08b4c0 con 0x7f50f415c380 2024-04-14T08:43:08.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.489+0000 7f50f2ffd640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f50d406c8f0 0x7f50d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.489+0000 7f50f2ffd640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f50d406c8f0 0x7f50d406edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f50e4003d90 tx=0x7f50e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.493+0000 7f50eb7fe640 1 -- 172.21.15.172:0/2703045733 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f50f415b620 con 0x7f50f415c380 2024-04-14T08:43:08.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.710+0000 7f50f96b7640 1 -- 172.21.15.172:0/2703045733 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f50f40b38b0 con 0x7f50f415c380 2024-04-14T08:43:08.711 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:43:08.711+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:43:08.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.712+0000 7f50eb7fe640 1 -- 172.21.15.172:0/2703045733 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v94) ==== 161+0+0 (secure 0 0 0) 0x7f50ec059710 con 0x7f50f415c380 2024-04-14T08:43:08.712 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:43:08.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.716+0000 7f50f96b7640 1 -- 172.21.15.172:0/2703045733 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f50d406c8f0 msgr2=0x7f50d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.716+0000 7f50f96b7640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f50d406c8f0 0x7f50d406edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f50e4003d90 tx=0x7f50e4009040 comp rx=0 tx=0).stop 2024-04-14T08:43:08.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.716+0000 7f50f96b7640 1 -- 172.21.15.172:0/2703045733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f415c380 msgr2=0x7f50f40b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.716+0000 7f50f96b7640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f415c380 0x7f50f40b4010 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00c6d0 tx=0x7f50ec00cb90 comp rx=0 tx=0).stop 2024-04-14T08:43:08.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.716+0000 7f50f96b7640 1 -- 172.21.15.172:0/2703045733 shutdown_connections 2024-04-14T08:43:08.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.716+0000 7f50f96b7640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f50d406c8f0 0x7f50d406edb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.716+0000 7f50f96b7640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50f415c380 0x7f50f40b4010 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.716+0000 7f50f96b7640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f413df30 0x7f50f40b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.716+0000 7f50f96b7640 1 --2- 172.21.15.172:0/2703045733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50f40b23e0 0x7f50f41399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.716+0000 7f50f96b7640 1 -- 172.21.15.172:0/2703045733 >> 172.21.15.172:0/2703045733 conn(0x7f50f406b520 msgr2=0x7f50f4137220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.716+0000 7f50f96b7640 1 -- 172.21.15.172:0/2703045733 shutdown_connections 2024-04-14T08:43:08.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.716+0000 7f50f96b7640 1 -- 172.21.15.172:0/2703045733 wait complete. 2024-04-14T08:43:08.718 INFO:tasks.ceph.mds.c.smithi172.stderr: -946> 2024-04-14T08:43:06.163+0000 7fd090ff1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:43:08.728 DEBUG:teuthology.orchestra.run.smithi172:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-14T08:43:08.744 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:43:08.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.771+0000 7fce5160d280 1 -- start start 2024-04-14T08:43:08.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.771+0000 7fce5160d280 1 Processor -- start 2024-04-14T08:43:08.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.771+0000 7fce5160d280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565132e6cc00 0x565132fd8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.771+0000 7fce5160d280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565132e6d800 0x565132fd8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.771+0000 7fce5160d280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565132e6dc00 0x565132fd8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.771+0000 7fce5160d280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565132fe0180 con 0x565132e6d800 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.771+0000 7fce5160d280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x565132fe0300 con 0x565132e6cc00 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.771+0000 7fce5160d280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x565132fe0480 con 0x565132e6dc00 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.771+0000 7fce4e7fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565132e6d800 0x565132fd8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.771+0000 7fce4e7fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565132e6d800 0x565132fd8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59730/0 (socket says 172.21.15.172:59730) 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.771+0000 7fce4e7fb640 1 -- 172.21.15.172:0/528872570 learned_addr learned my addr 172.21.15.172:0/528872570 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4effc640 1 --2- 172.21.15.172:0/528872570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565132e6cc00 0x565132fd8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4f7fd640 1 --2- 172.21.15.172:0/528872570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565132e6dc00 0x565132fd8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4e7fb640 1 -- 172.21.15.172:0/528872570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565132e6dc00 msgr2=0x565132fd8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4e7fb640 1 --2- 172.21.15.172:0/528872570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565132e6dc00 0x565132fd8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4e7fb640 1 -- 172.21.15.172:0/528872570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565132e6cc00 msgr2=0x565132fd8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4e7fb640 1 --2- 172.21.15.172:0/528872570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565132e6cc00 0x565132fd8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4e7fb640 1 -- 172.21.15.172:0/528872570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565132ea4780 con 0x565132e6d800 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4f7fd640 1 --2- 172.21.15.172:0/528872570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565132e6dc00 0x565132fd8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:08.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4effc640 1 --2- 172.21.15.172:0/528872570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565132e6cc00 0x565132fd8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4e7fb640 1 --2- 172.21.15.172:0/528872570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565132e6d800 0x565132fd8580 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x565133c833b0 tx=0x565133c035c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbed8ddb8281f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4c7f7640 1 -- 172.21.15.172:0/528872570 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565132ea24e0 con 0x565132e6d800 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4c7f7640 1 -- 172.21.15.172:0/528872570 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565132e98a80 con 0x565132e6d800 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4c7f7640 1 -- 172.21.15.172:0/528872570 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565132ea2b60 con 0x565132e6d800 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce5160d280 1 -- 172.21.15.172:0/528872570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565132ea54a0 con 0x565132e6d800 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce5160d280 1 -- 172.21.15.172:0/528872570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x565132ea4000 con 0x565132e6d800 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce5160d280 4 waiting for MDS map... 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4c7f7640 1 -- 172.21.15.172:0/528872570 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x565133d3a600 con 0x565132e6d800 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce4c7f7640 1 -- 172.21.15.172:0/528872570 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 94) ==== 2885+0+0 (secure 0 0 0) 0x565132ec9180 con 0x565132e6d800 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce5160d280 4 Got MDS map 94 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce5160d280 10 main: JournalTool::main 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.772+0000 7fce5160d280 4 main: JournalTool: connecting to RADOS... 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.773+0000 7fce5160d280 1 Processor -- start 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.773+0000 7fce5160d280 1 -- start start 2024-04-14T08:43:08.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.773+0000 7fce5160d280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133d3d400 0x565132fd9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.773+0000 7fce5160d280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d800 0x565132fd9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.773+0000 7fce5160d280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133d3dc00 0x565132fd9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce5160d280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565132fe0600 con 0x565133d3d800 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce5160d280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x565132fe0780 con 0x565133d3dc00 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce5160d280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x565132fe0900 con 0x565133d3d400 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4effc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d800 0x565132fd9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4effc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d800 0x565132fd9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59732/0 (socket says 172.21.15.172:59732) 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4effc640 1 -- 172.21.15.172:0/1931360775 learned_addr learned my addr 172.21.15.172:0/1931360775 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4f7fd640 1 --2- 172.21.15.172:0/1931360775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133d3d400 0x565132fd9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4e7fb640 1 --2- 172.21.15.172:0/1931360775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133d3dc00 0x565132fd9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4effc640 1 -- 172.21.15.172:0/1931360775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133d3d400 msgr2=0x565132fd9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4effc640 1 --2- 172.21.15.172:0/1931360775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133d3d400 0x565132fd9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4effc640 1 -- 172.21.15.172:0/1931360775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133d3dc00 msgr2=0x565132fd9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4effc640 1 --2- 172.21.15.172:0/1931360775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133d3dc00 0x565132fd9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4effc640 1 -- 172.21.15.172:0/1931360775 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565132ea43c0 con 0x565133d3d800 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4e7fb640 1 --2- 172.21.15.172:0/1931360775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133d3dc00 0x565132fd9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4effc640 1 --2- 172.21.15.172:0/1931360775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d800 0x565132fd9600 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x565133c82ab0 tx=0x565133c033e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ccf53f5ac021e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4a7f3640 1 -- 172.21.15.172:0/1931360775 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565132ea29c0 con 0x565133d3d800 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4a7f3640 1 -- 172.21.15.172:0/1931360775 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565132e98fc0 con 0x565133d3d800 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.774+0000 7fce4a7f3640 1 -- 172.21.15.172:0/1931360775 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565132ea24e0 con 0x565133d3d800 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.775+0000 7fce5160d280 1 -- 172.21.15.172:0/1931360775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d800 msgr2=0x565132fd9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.775+0000 7fce5160d280 1 --2- 172.21.15.172:0/1931360775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d800 0x565132fd9600 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x565133c82ab0 tx=0x565133c033e0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.775+0000 7fce5160d280 1 -- 172.21.15.172:0/1931360775 shutdown_connections 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.775+0000 7fce5160d280 1 --2- 172.21.15.172:0/1931360775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133d3dc00 0x565132fd9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.775+0000 7fce5160d280 1 --2- 172.21.15.172:0/1931360775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d800 0x565132fd9600 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.775+0000 7fce5160d280 1 --2- 172.21.15.172:0/1931360775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133d3d400 0x565132fd9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.775+0000 7fce5160d280 1 -- 172.21.15.172:0/1931360775 >> 172.21.15.172:0/1931360775 conn(0x565133d3d000 msgr2=0x565132f6d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.775+0000 7fce5160d280 1 -- 172.21.15.172:0/1931360775 shutdown_connections 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.775+0000 7fce5160d280 1 -- 172.21.15.172:0/1931360775 wait complete. 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.775+0000 7fce5160d280 1 Processor -- start 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.775+0000 7fce5160d280 1 -- start start 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce5160d280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d400 0x565132fd9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce5160d280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133d3d800 0x565132fd9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce5160d280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133d3dc00 0x565132fd9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce5160d280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565132fe1b00 con 0x565133d3d400 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce5160d280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x565132fe1980 con 0x565133d3dc00 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce5160d280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x565132fe1800 con 0x565133d3d800 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4f7fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d400 0x565132fd9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4f7fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d400 0x565132fd9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59738/0 (socket says 172.21.15.172:59738) 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4f7fd640 1 -- 172.21.15.172:0/814941749 learned_addr learned my addr 172.21.15.172:0/814941749 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4effc640 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133d3d800 0x565132fd9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4e7fb640 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133d3dc00 0x565132fd9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4f7fd640 1 -- 172.21.15.172:0/814941749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133d3d800 msgr2=0x565132fd9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4f7fd640 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133d3d800 0x565132fd9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4f7fd640 1 -- 172.21.15.172:0/814941749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133d3dc00 msgr2=0x565132fd9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4f7fd640 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133d3dc00 0x565132fd9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4f7fd640 1 -- 172.21.15.172:0/814941749 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565132ea4780 con 0x565133d3d400 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4effc640 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133d3d800 0x565132fd9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4e7fb640 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133d3dc00 0x565132fd9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce4f7fd640 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d400 0x565132fd9080 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x565133d57410 tx=0x565133d59800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce48ff0640 1 -- 172.21.15.172:0/814941749 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565132ea29c0 con 0x565133d3d400 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce48ff0640 1 -- 172.21.15.172:0/814941749 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565132e98c40 con 0x565133d3d400 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.776+0000 7fce5160d280 1 -- 172.21.15.172:0/814941749 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565132ea54a0 con 0x565133d3d400 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.777+0000 7fce48ff0640 1 -- 172.21.15.172:0/814941749 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565132ea24e0 con 0x565133d3d400 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.777+0000 7fce5160d280 1 -- 172.21.15.172:0/814941749 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565132ea4960 con 0x565133d3d400 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.777+0000 7fce5160d280 4 main: JournalTool: resolving pool 17 2024-04-14T08:43:08.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.778+0000 7fce48ff0640 1 -- 172.21.15.172:0/814941749 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x565133c58c00 con 0x565133d3d400 2024-04-14T08:43:08.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.778+0000 7fce48ff0640 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x565133db8c00 0x565132fda100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.778+0000 7fce48ff0640 1 -- 172.21.15.172:0/814941749 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x565133dba600 con 0x565133d3d400 2024-04-14T08:43:08.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.778+0000 7fce4effc640 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x565133db8c00 0x565132fda100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.778+0000 7fce5160d280 4 main: JournalTool: creating IoCtx.. 2024-04-14T08:43:08.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.778+0000 7fce5160d280 4 main: Executing for rank 0 2024-04-14T08:43:08.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.779+0000 7fce4effc640 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x565133db8c00 0x565132fda100 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x565133c82ab0 tx=0x565133c032c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.779+0000 7fce5160d280 1 -- start start 2024-04-14T08:43:08.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.779+0000 7fce5160d280 1 Processor -- start 2024-04-14T08:43:08.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.779+0000 7fce5160d280 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133db9400 0x565132fda680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.779+0000 7fce5160d280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133db9800 0x565132fdac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.779+0000 7fce5160d280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133db9c00 0x565132fdb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.779+0000 7fce5160d280 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565132fe1500 con 0x565133db9400 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.779+0000 7fce5160d280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x565132fe0780 con 0x565133db9c00 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.779+0000 7fce5160d280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x565132fe0a80 con 0x565133db9800 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce4f7fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133db9400 0x565132fda680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce4f7fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133db9400 0x565132fda680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59754/0 (socket says 172.21.15.172:59754) 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce4f7fd640 1 -- 172.21.15.172:0/2721453803 learned_addr learned my addr 172.21.15.172:0/2721453803 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce4effc640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133db9800 0x565132fdac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce4e7fb640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133db9c00 0x565132fdb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce4f7fd640 1 -- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133db9800 msgr2=0x565132fdac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce4f7fd640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133db9800 0x565132fdac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce4f7fd640 1 -- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133db9c00 msgr2=0x565132fdb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce4f7fd640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133db9c00 0x565132fdb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce4f7fd640 1 -- 172.21.15.172:0/2721453803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565132ea5680 con 0x565133db9400 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce4f7fd640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133db9400 0x565132fda680 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x565133def470 tx=0x565133176180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652d214f5718bddc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce45fea640 1 -- 172.21.15.172:0/2721453803 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565133175860 con 0x565133db9400 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce45fea640 1 -- 172.21.15.172:0/2721453803 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x565132e981c0 con 0x565133db9400 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce45fea640 1 -- 172.21.15.172:0/2721453803 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x565133175a00 con 0x565133db9400 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565132ea4780 con 0x565133db9400 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x565132ea54a0 con 0x565133db9400 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce5160d280 4 waiting for MDS map... 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce45fea640 1 -- 172.21.15.172:0/2721453803 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x565133246200 con 0x565133db9400 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce45fea640 1 -- 172.21.15.172:0/2721453803 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 94) ==== 2885+0+0 (secure 0 0 0) 0x565132ec9500 con 0x565133db9400 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce5160d280 4 Got MDS map 94 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x565133252400 0x565132fdb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.780+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x565133252800 con 0x565133252400 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.781+0000 7fce4effc640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x565133252400 0x565132fdb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.781+0000 7fce4effc640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x565133252400 0x565132fdb700 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=11e0d8bf39e8f9d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.782+0000 7fce4effc640 1 -- 172.21.15.172:0/2721453803 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v118'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x565133bf0000 con 0x565133252400 2024-04-14T08:43:08.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.782+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x565133252c00 0x565133260000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.782+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x565133253000 con 0x565133252c00 2024-04-14T08:43:08.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.782+0000 7fce4e7fb640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x565133252c00 0x565133260000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.783+0000 7fce4e7fb640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x565133252c00 0x565133260000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=df46150fe70cbc23 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.786+0000 7fce4e7fb640 1 -- 172.21.15.172:0/2721453803 <== osd.3 v2:172.21.15.172:6808/2305139122 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v118'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x565133bf0500 con 0x565133252c00 2024-04-14T08:43:08.791 INFO:teuthology.orchestra.run.smithi172.stdout:writing EResetJournal entry 2024-04-14T08:43:08.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.786+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x565133253400 con 0x565133252c00 2024-04-14T08:43:08.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.786+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x565133253800 0x565133260580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.786+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x565133253c00 con 0x565133253800 2024-04-14T08:43:08.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.786+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x565133270000 0x565133260b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.786+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x565133270400 con 0x565133270000 2024-04-14T08:43:08.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.786+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x565133270800 0x565133261080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.786+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x565133270c00 con 0x565133270800 2024-04-14T08:43:08.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.786+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x565133271000 con 0x565133270000 2024-04-14T08:43:08.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.786+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x565133271400 con 0x565133252400 2024-04-14T08:43:08.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.787+0000 7fce4f7fd640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x565133253800 0x565133260580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.787+0000 7fce4effc640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x565133270000 0x565133260b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.787+0000 7fce4e7fb640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x565133270800 0x565133261080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.787+0000 7fce4effc640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x565133270000 0x565133260b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=1565b659bbb71b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.787+0000 7fce4f7fd640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x565133253800 0x565133260580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=e03cf3e3826fa269 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.787+0000 7fce4e7fb640 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x565133270800 0x565133261080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=b349ee4ba852496b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.788+0000 7fce4e7fb640 1 -- 172.21.15.172:0/2721453803 <== osd.3 v2:172.21.15.172:6808/2305139122 2 ==== osd_op_reply(3 200.00000001 [delete] v118'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x565133bf0500 con 0x565133252c00 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.788+0000 7fce4effc640 1 -- 172.21.15.172:0/2721453803 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== osd_op_reply(5 200.00000003 [delete] v118'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x565133bf0000 con 0x565133270000 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.788+0000 7fce4effc640 1 -- 172.21.15.172:0/2721453803 <== osd.1 v2:172.21.15.172:6823/1219922421 2 ==== osd_op_reply(8 200.00000006 [delete] v118'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x565133bf0000 con 0x565133252400 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.789+0000 7fce4effc640 1 -- 172.21.15.172:0/2721453803 <== osd.0 v2:172.21.15.172:6800/3761711279 2 ==== osd_op_reply(7 200.00000005 [delete] v118'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x565133bf0000 con 0x565133270000 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.789+0000 7fce4f7fd640 1 -- 172.21.15.172:0/2721453803 <== osd.4 v2:172.21.15.177:6820/4180733146 1 ==== osd_op_reply(4 200.00000002 [delete] v118'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x565133bf0780 con 0x565133253800 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.789+0000 7fce447e7640 1 -- 172.21.15.172:0/2721453803 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x565133c47400 con 0x565133252c00 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.790+0000 7fce4e7fb640 1 -- 172.21.15.172:0/2721453803 <== osd.3 v2:172.21.15.172:6808/2305139122 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v118'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x565133bf0500 con 0x565133252c00 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce4e7fb640 1 -- 172.21.15.172:0/2721453803 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== osd_op_reply(6 200.00000004 [delete] v118'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x565133bf0500 con 0x565133270800 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x565133270000 msgr2=0x565133260b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x565133270000 0x565133260b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x565133252400 msgr2=0x565132fdb700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x565133252400 0x565132fdb700 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x565133252c00 msgr2=0x565133260000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x565133252c00 0x565133260000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x565133253800 msgr2=0x565133260580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x565133253800 0x565133260580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce4e7fb640 1 -- 172.21.15.172:0/2721453803 reap_dead start 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x565133270800 msgr2=0x565133261080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x565133270800 0x565133261080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133db9400 msgr2=0x565132fda680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133db9400 0x565132fda680 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x565133def470 tx=0x565133176180 comp rx=0 tx=0).stop 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 shutdown_connections 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x565133270800 0x565133261080 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x565133253800 0x565133260580 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 --2- 172.21.15.172:0/2721453803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133db9400 0x565132fda680 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 >> 172.21.15.172:0/2721453803 conn(0x565133db9000 msgr2=0x565132f6d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 shutdown_connections 2024-04-14T08:43:08.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 -- 172.21.15.172:0/2721453803 wait complete. 2024-04-14T08:43:08.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 -- 172.21.15.172:0/814941749 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x565133db8c00 msgr2=0x565132fda100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x565133db8c00 0x565132fda100 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x565133c82ab0 tx=0x565133c032c0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 -- 172.21.15.172:0/814941749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d400 msgr2=0x565132fd9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.791+0000 7fce5160d280 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d400 0x565132fd9080 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x565133d57410 tx=0x565133d59800 comp rx=0 tx=0).stop 2024-04-14T08:43:08.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 -- 172.21.15.172:0/814941749 shutdown_connections 2024-04-14T08:43:08.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x565133db8c00 0x565132fda100 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565133d3dc00 0x565132fd9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565133d3d800 0x565132fd9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 --2- 172.21.15.172:0/814941749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565133d3d400 0x565132fd9080 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 -- 172.21.15.172:0/814941749 >> 172.21.15.172:0/814941749 conn(0x565133d3d000 msgr2=0x565132f6d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 -- 172.21.15.172:0/814941749 shutdown_connections 2024-04-14T08:43:08.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 -- 172.21.15.172:0/814941749 wait complete. 2024-04-14T08:43:08.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 -- 172.21.15.172:0/528872570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565132e6d800 msgr2=0x565132fd8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 --2- 172.21.15.172:0/528872570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565132e6d800 0x565132fd8580 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x565133c833b0 tx=0x565133c035c0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 -- 172.21.15.172:0/528872570 shutdown_connections 2024-04-14T08:43:08.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 --2- 172.21.15.172:0/528872570 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x565132e6dc00 0x565132fd8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 --2- 172.21.15.172:0/528872570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565132e6d800 0x565132fd8580 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 --2- 172.21.15.172:0/528872570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565132e6cc00 0x565132fd8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 -- 172.21.15.172:0/528872570 >> 172.21.15.172:0/528872570 conn(0x565132e6d000 msgr2=0x565132f6c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 -- 172.21.15.172:0/528872570 shutdown_connections 2024-04-14T08:43:08.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.792+0000 7fce5160d280 1 -- 172.21.15.172:0/528872570 wait complete. 2024-04-14T08:43:08.801 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.073777, result: writing EResetJournal entry 2024-04-14T08:43:08.802 DEBUG:teuthology.orchestra.run.smithi172:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-14T08:43:08.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.876+0000 7f47d142f740 1 -- start start 2024-04-14T08:43:08.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.876+0000 7f47d142f740 1 Processor -- start 2024-04-14T08:43:08.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.876+0000 7f47d142f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd62d2c00 0x559dd643e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.876+0000 7f47d142f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd62d3800 0x559dd643e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.876+0000 7f47d142f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd62d3c00 0x559dd643eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.876+0000 7f47d142f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559dd70a2600 con 0x559dd62d3c00 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.876+0000 7f47d142f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559dd70a2480 con 0x559dd62d2c00 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.876+0000 7f47d142f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x559dd70a2300 con 0x559dd62d3800 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.876+0000 7f47cf3fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd62d3c00 0x559dd643eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.876+0000 7f47cf3fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd62d3c00 0x559dd643eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59770/0 (socket says 172.21.15.172:59770) 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.876+0000 7f47cf3fd640 1 -- 172.21.15.172:0/3731908735 learned_addr learned my addr 172.21.15.172:0/3731908735 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cebfc640 1 --2- 172.21.15.172:0/3731908735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd62d2c00 0x559dd643e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/3731908735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd62d3800 0x559dd643e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cf3fd640 1 -- 172.21.15.172:0/3731908735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd62d3800 msgr2=0x559dd643e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cf3fd640 1 --2- 172.21.15.172:0/3731908735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd62d3800 0x559dd643e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cf3fd640 1 -- 172.21.15.172:0/3731908735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd62d2c00 msgr2=0x559dd643e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cf3fd640 1 --2- 172.21.15.172:0/3731908735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd62d2c00 0x559dd643e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cf3fd640 1 -- 172.21.15.172:0/3731908735 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd62ff680 con 0x559dd62d3c00 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cebfc640 1 --2- 172.21.15.172:0/3731908735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd62d2c00 0x559dd643e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cf3fd640 1 --2- 172.21.15.172:0/3731908735 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd62d3c00 0x559dd643eb00 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x559dd70d9b90 tx=0x559dd706e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f613755916ccbc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cc3f7640 1 -- 172.21.15.172:0/3731908735 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559dd63084e0 con 0x559dd62d3c00 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cc3f7640 1 -- 172.21.15.172:0/3731908735 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x559dd631ec40 con 0x559dd62d3c00 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cc3f7640 1 -- 172.21.15.172:0/3731908735 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559dd6308b60 con 0x559dd62d3c00 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47d142f740 1 -- 172.21.15.172:0/3731908735 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd62ff860 con 0x559dd62d3c00 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47d142f740 1 -- 172.21.15.172:0/3731908735 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x559dd62ff2c0 con 0x559dd62d3c00 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47d142f740 4 waiting for MDS map... 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cc3f7640 1 -- 172.21.15.172:0/3731908735 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x559dd719e600 con 0x559dd62d3c00 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47cc3f7640 1 -- 172.21.15.172:0/3731908735 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 94) ==== 2885+0+0 (secure 0 0 0) 0x559dd632d180 con 0x559dd62d3c00 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.877+0000 7f47d142f740 4 Got MDS map 94 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.878+0000 7f47d142f740 4 datascan.main: Using metadata pool output 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.878+0000 7f47d142f740 4 datascan.main: connecting to RADOS... 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.878+0000 7f47d142f740 1 Processor -- start 2024-04-14T08:43:08.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.878+0000 7f47d142f740 1 -- start start 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47d142f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 0x559dd643f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47d142f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd71a1800 0x559dd643f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47d142f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd71a1c00 0x559dd643fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47d142f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559dd70a2180 con 0x559dd71a1400 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47d142f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559dd62f9800 con 0x559dd71a1c00 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47d142f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x559dd62f8a80 con 0x559dd71a1800 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ce3fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 0x559dd643f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ce3fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 0x559dd643f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59784/0 (socket says 172.21.15.172:59784) 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2291990617 learned_addr learned my addr 172.21.15.172:0/2291990617 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47cebfc640 1 --2- 172.21.15.172:0/2291990617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd71a1800 0x559dd643f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2291990617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd71a1c00 0x559dd643fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2291990617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd71a1800 msgr2=0x559dd643f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2291990617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd71a1800 0x559dd643f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2291990617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd71a1c00 msgr2=0x559dd643fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2291990617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd71a1c00 0x559dd643fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2291990617 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd62ff4a0 con 0x559dd71a1400 2024-04-14T08:43:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2291990617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 0x559dd643f080 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x559dd718a930 tx=0x559dd71bf860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f331a5f0b469ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2291990617 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559dd63084e0 con 0x559dd71a1400 2024-04-14T08:43:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2291990617 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x559dd631e540 con 0x559dd71a1400 2024-04-14T08:43:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.879+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2291990617 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559dd6308b60 con 0x559dd71a1400 2024-04-14T08:43:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.880+0000 7f47d142f740 1 -- 172.21.15.172:0/2291990617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 msgr2=0x559dd643f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.880+0000 7f47d142f740 1 --2- 172.21.15.172:0/2291990617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 0x559dd643f080 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x559dd718a930 tx=0x559dd71bf860 comp rx=0 tx=0).stop 2024-04-14T08:43:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.880+0000 7f47d142f740 1 -- 172.21.15.172:0/2291990617 shutdown_connections 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.880+0000 7f47d142f740 1 --2- 172.21.15.172:0/2291990617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd71a1c00 0x559dd643fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.880+0000 7f47d142f740 1 --2- 172.21.15.172:0/2291990617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd71a1800 0x559dd643f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.880+0000 7f47d142f740 1 --2- 172.21.15.172:0/2291990617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 0x559dd643f080 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.880+0000 7f47d142f740 1 -- 172.21.15.172:0/2291990617 >> 172.21.15.172:0/2291990617 conn(0x559dd71a1000 msgr2=0x559dd63b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.880+0000 7f47d142f740 1 -- 172.21.15.172:0/2291990617 shutdown_connections 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.880+0000 7f47d142f740 1 -- 172.21.15.172:0/2291990617 wait complete. 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.880+0000 7f47d142f740 1 Processor -- start 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.880+0000 7f47d142f740 1 -- start start 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47d142f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 0x559dd643f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47d142f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd71a1800 0x559dd643f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47d142f740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd71a1c00 0x559dd643fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47d142f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559dd70a3380 con 0x559dd71a1400 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47d142f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559dd70a2180 con 0x559dd71a1800 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47d142f740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x559dd70a3500 con 0x559dd71a1c00 2024-04-14T08:43:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47ce3fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 0x559dd643f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47ce3fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 0x559dd643f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59800/0 (socket says 172.21.15.172:59800) 2024-04-14T08:43:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 learned_addr learned my addr 172.21.15.172:0/2823123796 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47cebfc640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd71a1800 0x559dd643f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd71a1c00 0x559dd643fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd71a1c00 msgr2=0x559dd643fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd71a1c00 0x559dd643fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd71a1800 msgr2=0x559dd643f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd71a1800 0x559dd643f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd62ff680 con 0x559dd71a1400 2024-04-14T08:43:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 0x559dd643f080 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x559dd718bbf0 tx=0x559dd71bf800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.881+0000 7f47c8bf0640 1 -- 172.21.15.172:0/2823123796 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559dd6308000 con 0x559dd71a1400 2024-04-14T08:43:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.882+0000 7f47c8bf0640 1 -- 172.21.15.172:0/2823123796 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x559dd631ea80 con 0x559dd71a1400 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.882+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559dd62ff860 con 0x559dd71a1400 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.882+0000 7f47c8bf0640 1 -- 172.21.15.172:0/2823123796 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559dd63084e0 con 0x559dd71a1400 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.882+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd62ff0e0 con 0x559dd71a1400 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.882+0000 7f47d142f740 4 datascan.init: resolving metadata pool 17 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.883+0000 7f47c8bf0640 1 -- 172.21.15.172:0/2823123796 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x559dd721ec00 con 0x559dd71a1400 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.883+0000 7f47c8bf0640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x559dd721ac00 0x559dd6440100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.883+0000 7f47c8bf0640 1 -- 172.21.15.172:0/2823123796 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x559dd721ca00 con 0x559dd71a1400 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.883+0000 7f47cebfc640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x559dd721ac00 0x559dd6440100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.883+0000 7f47d142f740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.883+0000 7f47d142f740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.883+0000 7f47d142f740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.883+0000 7f47d142f740 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x559dd721b000 0x559dd6440680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.883+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd721b400 con 0x559dd721b000 2024-04-14T08:43:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.883+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x559dd721b000 0x559dd6440680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.884+0000 7f47cebfc640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x559dd721ac00 0x559dd6440100 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x559dd705a180 tx=0x559dd706c7e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.884+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x559dd721b000 0x559dd6440680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.884+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== osd_op_reply(1 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd721b000 2024-04-14T08:43:08.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.884+0000 7f47c9bf2640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x559dd6666000 0x559dd666c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.884+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6666400 con 0x559dd6666000 2024-04-14T08:43:08.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.884+0000 7f47cf3fd640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x559dd6666000 0x559dd666c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.885+0000 7f47cf3fd640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x559dd6666000 0x559dd666c000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.887+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.5 v2:172.21.15.177:6816/3172735715 1 ==== osd_op_reply(2 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6666000 2024-04-14T08:43:08.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.887+0000 7f47ca3f3640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x559dd6666c00 0x559dd666c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.887+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667000 con 0x559dd6666c00 2024-04-14T08:43:08.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.887+0000 7f47cebfc640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x559dd6666c00 0x559dd666c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.888+0000 7f47cebfc640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x559dd6666c00 0x559dd666c580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.888+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== osd_op_reply(3 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064780 con 0x559dd6666c00 2024-04-14T08:43:08.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.888+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6666800 con 0x559dd721b000 2024-04-14T08:43:08.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.888+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.0 v2:172.21.15.172:6800/3761711279 2 ==== osd_op_reply(4 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd721b000 2024-04-14T08:43:08.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.888+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667400 con 0x559dd721b000 2024-04-14T08:43:08.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.888+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.0 v2:172.21.15.172:6800/3761711279 3 ==== osd_op_reply(5 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd721b000 2024-04-14T08:43:08.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.888+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667800 con 0x559dd6666000 2024-04-14T08:43:08.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.889+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.5 v2:172.21.15.177:6816/3172735715 2 ==== osd_op_reply(6 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6666000 2024-04-14T08:43:08.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.889+0000 7f47ca3f3640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x559dd6676000 0x559dd666cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.889+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6676400 con 0x559dd6676000 2024-04-14T08:43:08.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.889+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x559dd6676000 0x559dd666cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.889+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x559dd6676000 0x559dd666cb00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.890+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(7 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.890+0000 7f47c9bf2640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x559dd6667c00 0x559dd666d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.890+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6676400 con 0x559dd6667c00 2024-04-14T08:43:08.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.890+0000 7f47cf3fd640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x559dd6667c00 0x559dd666d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.890+0000 7f47cf3fd640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x559dd6667c00 0x559dd666d600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.890+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.3 v2:172.21.15.172:6808/2305139122 1 ==== osd_op_reply(8 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6667c00 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.890+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6676800 con 0x559dd721b000 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.890+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.0 v2:172.21.15.172:6800/3761711279 4 ==== osd_op_reply(9 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd721b000 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.891+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6666800 con 0x559dd721b000 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.891+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.0 v2:172.21.15.172:6800/3761711279 5 ==== osd_op_reply(10 [pgnls start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd721b000 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.891+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6676c00 con 0x559dd6666c00 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.891+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 <== osd.6 v2:172.21.15.177:6800/2961907928 2 ==== osd_op_reply(11 [pgnls start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064780 con 0x559dd6666c00 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.891+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667400 con 0x559dd721b000 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.892+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.0 v2:172.21.15.172:6800/3761711279 6 ==== osd_op_reply(12 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd721b000 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.892+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6677000 con 0x559dd6666000 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.892+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.5 v2:172.21.15.177:6816/3172735715 3 ==== osd_op_reply(13 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6666000 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.892+0000 7f47c9bf2640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x559dd721b400 0x559dd666db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.892+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6676400 con 0x559dd721b400 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.892+0000 7f47cebfc640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x559dd721b400 0x559dd666db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.892+0000 7f47cebfc640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x559dd721b400 0x559dd666db80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.893+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 <== osd.2 v2:172.21.15.172:6816/1609902500 1 ==== osd_op_reply(14 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064780 con 0x559dd721b400 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.893+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6677400 con 0x559dd6676000 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.893+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 2 ==== osd_op_reply(15 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.893+0000 7f47c9bf2640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x559dd6677000 0x559dd666e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.893+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6666400 con 0x559dd6677000 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.893+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x559dd6677000 0x559dd666e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.893+0000 7f47ce3fb640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x559dd6677000 0x559dd666e100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.894+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== osd_op_reply(16 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6677000 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.894+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6677800 con 0x559dd6677000 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.894+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.7 v2:172.21.15.177:6808/3609985207 2 ==== osd_op_reply(17 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6677000 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.894+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667800 con 0x559dd6667c00 2024-04-14T08:43:08.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.895+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.3 v2:172.21.15.172:6808/2305139122 2 ==== osd_op_reply(18 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6667c00 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.895+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6666800 con 0x559dd721b400 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.895+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 <== osd.2 v2:172.21.15.172:6816/1609902500 2 ==== osd_op_reply(19 [pgnls start_epoch 118 out=81b] v117'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559dd7064780 con 0x559dd721b400 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.895+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x559dd721b800 con 0x559dd721b400 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.895+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 <== osd.2 v2:172.21.15.172:6816/1609902500 3 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x559dd7064780 con 0x559dd721b400 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.895+0000 7f47d142f740 10 datascan.operator(): handling object 1099511627778.0 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.895+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x559dd721bc00 con 0x559dd721b400 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.897+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 <== osd.2 v2:172.21.15.172:6816/1609902500 4 ==== osd_op_reply(21 10000000002.00000000 [call] v118'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x559dd7064780 con 0x559dd721b400 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.897+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6676800 con 0x559dd6667c00 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.897+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.3 v2:172.21.15.172:6808/2305139122 3 ==== osd_op_reply(22 [pgnls start_epoch 118 out=49b] v116'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6667c00 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.897+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667400 con 0x559dd6667c00 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.897+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.3 v2:172.21.15.172:6808/2305139122 4 ==== osd_op_reply(23 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6667c00 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.897+0000 7f47c9bf2640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x559dd71a0800 0x559dd666e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.897+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6670000 con 0x559dd71a0800 2024-04-14T08:43:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.897+0000 7f47cf3fd640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x559dd71a0800 0x559dd666e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.898+0000 7f47cf3fd640 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x559dd71a0800 0x559dd666e680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.898+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.4 v2:172.21.15.177:6820/4180733146 1 ==== osd_op_reply(24 [pgnls start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd71a0800 2024-04-14T08:43:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.898+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6666400 con 0x559dd6667c00 2024-04-14T08:43:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.898+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.3 v2:172.21.15.172:6808/2305139122 5 ==== osd_op_reply(25 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6667c00 2024-04-14T08:43:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.898+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6670400 con 0x559dd721b000 2024-04-14T08:43:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.899+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.0 v2:172.21.15.172:6800/3761711279 7 ==== osd_op_reply(26 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd721b000 2024-04-14T08:43:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.899+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd70a5400 con 0x559dd6677000 2024-04-14T08:43:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.899+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.7 v2:172.21.15.177:6808/3609985207 3 ==== osd_op_reply(27 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6677000 2024-04-14T08:43:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.899+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6670800 con 0x559dd721b400 2024-04-14T08:43:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.899+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 <== osd.2 v2:172.21.15.172:6816/1609902500 5 ==== osd_op_reply(28 [pgnls start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064780 con 0x559dd721b400 2024-04-14T08:43:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.899+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd70a5000 con 0x559dd6667c00 2024-04-14T08:43:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.899+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.3 v2:172.21.15.172:6808/2305139122 6 ==== osd_op_reply(29 [pgnls start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6667c00 2024-04-14T08:43:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.899+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6670c00 con 0x559dd6676000 2024-04-14T08:43:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.900+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 3 ==== osd_op_reply(30 [pgnls start_epoch 118 out=81b] v116'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.900+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x559dd6677400 con 0x559dd6676000 2024-04-14T08:43:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.900+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 4 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.900+0000 7f47d142f740 10 datascan.operator(): handling object 1099511628279.0 2024-04-14T08:43:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.900+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x559dd6676c00 con 0x559dd6676000 2024-04-14T08:43:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.901+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 5 ==== osd_op_reply(32 100000001f7.00000000 [call] v118'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.901+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6676400 con 0x559dd721b000 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.901+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.0 v2:172.21.15.172:6800/3761711279 8 ==== osd_op_reply(33 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd721b000 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.901+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6671000 con 0x559dd6667c00 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.902+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.3 v2:172.21.15.172:6808/2305139122 7 ==== osd_op_reply(34 [pgnls start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6667c00 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.902+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667800 con 0x559dd6666000 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.902+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.5 v2:172.21.15.177:6816/3172735715 4 ==== osd_op_reply(35 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6666000 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.902+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6671400 con 0x559dd6667c00 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.902+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.3 v2:172.21.15.172:6808/2305139122 8 ==== osd_op_reply(36 [pgnls start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6667c00 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.902+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667400 con 0x559dd6667c00 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.903+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.3 v2:172.21.15.172:6808/2305139122 9 ==== osd_op_reply(37 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6667c00 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.903+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6670400 con 0x559dd6666000 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.903+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.5 v2:172.21.15.177:6816/3172735715 5 ==== osd_op_reply(38 [pgnls start_epoch 118 out=49b] v116'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6666000 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.903+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6677800 con 0x559dd6676000 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.903+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 6 ==== osd_op_reply(39 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.903+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6670c00 con 0x559dd71a0800 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.904+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.4 v2:172.21.15.177:6820/4180733146 2 ==== osd_op_reply(40 [pgnls start_epoch 118 out=81b] v116'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559dd7064500 con 0x559dd71a0800 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.904+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x559dd6677c00 con 0x559dd71a0800 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.904+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.4 v2:172.21.15.177:6820/4180733146 3 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x559dd7064500 con 0x559dd71a0800 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.904+0000 7f47d142f740 10 datascan.operator(): handling object 1099511627780.1 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.904+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667000 con 0x559dd6676000 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.905+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 7 ==== osd_op_reply(42 10000000004.00000000 [call] v118'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.905+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6671000 con 0x559dd6667c00 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.905+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.3 v2:172.21.15.172:6808/2305139122 10 ==== osd_op_reply(43 [pgnls start_epoch 118 out=49b] v116'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6667c00 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.906+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6670000 con 0x559dd6677000 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.906+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.7 v2:172.21.15.177:6808/3609985207 4 ==== osd_op_reply(44 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6677000 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.906+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6677400 con 0x559dd71a0800 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.906+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.4 v2:172.21.15.177:6820/4180733146 4 ==== osd_op_reply(45 [pgnls start_epoch 118 out=81b] v117'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559dd7064500 con 0x559dd71a0800 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.906+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x559dd6671400 con 0x559dd71a0800 2024-04-14T08:43:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.907+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.4 v2:172.21.15.177:6820/4180733146 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x559dd7064500 con 0x559dd71a0800 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.907+0000 7f47d142f740 10 datascan.operator(): handling object 1099511628280.0 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.907+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x559dd6670400 con 0x559dd71a0800 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.908+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.4 v2:172.21.15.177:6820/4180733146 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v118'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x559dd7064500 con 0x559dd71a0800 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.908+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6670c00 con 0x559dd721b000 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.908+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.0 v2:172.21.15.172:6800/3761711279 9 ==== osd_op_reply(48 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd721b000 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.908+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6676c00 con 0x559dd6676000 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.909+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 8 ==== osd_op_reply(49 [pgnls start_epoch 118 out=81b] v118'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.909+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x559dd6677c00 con 0x559dd6676000 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.909+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 9 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.909+0000 7f47d142f740 10 datascan.operator(): handling object 1099511627780.0 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.909+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667800 con 0x559dd6676000 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.910+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 10 ==== osd_op_reply(51 10000000004.00000000 [call] v118'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.910+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667400 con 0x559dd6666c00 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.910+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 <== osd.6 v2:172.21.15.177:6800/2961907928 3 ==== osd_op_reply(52 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064780 con 0x559dd6666c00 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.910+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6676800 con 0x559dd6666000 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.911+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.5 v2:172.21.15.177:6816/3172735715 6 ==== osd_op_reply(53 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6666000 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.911+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6671800 con 0x559dd71a0800 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.911+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.4 v2:172.21.15.177:6820/4180733146 7 ==== osd_op_reply(54 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd71a0800 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.911+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6670000 con 0x559dd71a0800 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.911+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.4 v2:172.21.15.177:6820/4180733146 8 ==== osd_op_reply(55 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd71a0800 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.911+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd70a5400 con 0x559dd6666c00 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.912+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 <== osd.6 v2:172.21.15.177:6800/2961907928 4 ==== osd_op_reply(56 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064780 con 0x559dd6666c00 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.912+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6670c00 con 0x559dd721b000 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.912+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.0 v2:172.21.15.172:6800/3761711279 10 ==== osd_op_reply(57 [pgnls start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd721b000 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.912+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd70a5000 con 0x559dd6667c00 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.912+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.3 v2:172.21.15.172:6808/2305139122 11 ==== osd_op_reply(58 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6667c00 2024-04-14T08:43:08.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.912+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6676400 con 0x559dd6676000 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.912+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 11 ==== osd_op_reply(59 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.913+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6666400 con 0x559dd721b400 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.913+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 <== osd.2 v2:172.21.15.172:6816/1609902500 6 ==== osd_op_reply(60 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064780 con 0x559dd721b400 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.913+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6677800 con 0x559dd6677000 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.913+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.7 v2:172.21.15.177:6808/3609985207 5 ==== osd_op_reply(61 [pgnls start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6677000 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.913+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd70a5000 con 0x559dd6676000 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.913+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 12 ==== osd_op_reply(62 [pgnls start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.913+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6676c00 con 0x559dd6666000 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.914+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.5 v2:172.21.15.177:6816/3172735715 7 ==== osd_op_reply(63 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6666000 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.914+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd70a5400 con 0x559dd6677000 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.914+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.7 v2:172.21.15.177:6808/3609985207 6 ==== osd_op_reply(64 [pgnls start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6677000 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.914+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6677c00 con 0x559dd6676000 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.914+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 13 ==== osd_op_reply(65 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.914+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6666800 con 0x559dd71a0800 2024-04-14T08:43:08.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.915+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.4 v2:172.21.15.177:6820/4180733146 9 ==== osd_op_reply(66 [pgnls start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd71a0800 2024-04-14T08:43:08.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.915+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667000 con 0x559dd721b400 2024-04-14T08:43:08.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.915+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 <== osd.2 v2:172.21.15.172:6816/1609902500 7 ==== osd_op_reply(67 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064780 con 0x559dd721b400 2024-04-14T08:43:08.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.915+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6667400 con 0x559dd6666000 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.915+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.5 v2:172.21.15.177:6816/3172735715 8 ==== osd_op_reply(68 [pgnls start_epoch 118 out=49b] v116'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6666000 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.915+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6671000 con 0x559dd6666000 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.916+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.5 v2:172.21.15.177:6816/3172735715 9 ==== osd_op_reply(69 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6666000 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.916+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd721b800 con 0x559dd6677000 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.916+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.7 v2:172.21.15.177:6808/3609985207 7 ==== osd_op_reply(70 [pgnls start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6677000 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.916+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6671400 con 0x559dd6666000 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.916+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.5 v2:172.21.15.177:6816/3172735715 10 ==== osd_op_reply(71 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd6666000 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.916+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd721bc00 con 0x559dd6676000 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.1 v2:172.21.15.172:6823/1219922421 14 ==== osd_op_reply(72 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6676000 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47ca3f3640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6670400 con 0x559dd6677000 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47ce3fb640 1 -- 172.21.15.172:0/2823123796 <== osd.7 v2:172.21.15.177:6808/3609985207 8 ==== osd_op_reply(73 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064000 con 0x559dd6677000 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47c9bf2640 1 -- 172.21.15.172:0/2823123796 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x559dd6677400 con 0x559dd71a0800 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47cf3fd640 1 -- 172.21.15.172:0/2823123796 <== osd.4 v2:172.21.15.177:6820/4180733146 10 ==== osd_op_reply(74 [pgnls start_epoch 118 out=49b] v116'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559dd7064500 con 0x559dd71a0800 2024-04-14T08:43:08.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x559dd721b000 msgr2=0x559dd6440680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47d142f740 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x559dd721b000 0x559dd6440680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x559dd6676000 msgr2=0x559dd666cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47d142f740 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x559dd6676000 0x559dd666cb00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x559dd721b400 msgr2=0x559dd666db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47d142f740 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x559dd721b400 0x559dd666db80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x559dd6667c00 msgr2=0x559dd666d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47d142f740 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x559dd6667c00 0x559dd666d600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 reap_dead start 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x559dd71a0800 msgr2=0x559dd666e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.917+0000 7f47d142f740 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x559dd71a0800 0x559dd666e680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 reap_dead start 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x559dd6666000 msgr2=0x559dd666c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x559dd6666000 0x559dd666c000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x559dd6666c00 msgr2=0x559dd666c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x559dd6666c00 0x559dd666c580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x559dd6677000 msgr2=0x559dd666e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x559dd6677000 0x559dd666e100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x559dd721ac00 msgr2=0x559dd6440100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x559dd721ac00 0x559dd6440100 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x559dd705a180 tx=0x559dd706c7e0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 msgr2=0x559dd643f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 --2- 172.21.15.172:0/2823123796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd71a1400 0x559dd643f080 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x559dd718bbf0 tx=0x559dd71bf800 comp rx=0 tx=0).stop 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 reap_dead start 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47cebfc640 1 -- 172.21.15.172:0/2823123796 reap_dead start 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 shutdown_connections 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 >> 172.21.15.172:0/2823123796 conn(0x559dd71a1000 msgr2=0x559dd63b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 shutdown_connections 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/2823123796 wait complete. 2024-04-14T08:43:08.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/3731908735 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd62d3c00 msgr2=0x559dd643eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:08.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 --2- 172.21.15.172:0/3731908735 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd62d3c00 0x559dd643eb00 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x559dd70d9b90 tx=0x559dd706e0c0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/3731908735 shutdown_connections 2024-04-14T08:43:08.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 --2- 172.21.15.172:0/3731908735 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559dd62d3c00 0x559dd643eb00 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 --2- 172.21.15.172:0/3731908735 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559dd62d3800 0x559dd643e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 --2- 172.21.15.172:0/3731908735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559dd62d2c00 0x559dd643e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:08.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/3731908735 >> 172.21.15.172:0/3731908735 conn(0x559dd62d3000 msgr2=0x559dd63b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:08.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/3731908735 shutdown_connections 2024-04-14T08:43:08.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:08.918+0000 7f47d142f740 1 -- 172.21.15.172:0/3731908735 wait complete. 2024-04-14T08:43:08.928 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.126209, result: 2024-04-14T08:43:08.928 DEBUG:teuthology.orchestra.run.smithi172:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-14T08:43:09.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.006+0000 7fca9bc31740 1 -- start start 2024-04-14T08:43:09.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.006+0000 7fca9bc31740 1 Processor -- start 2024-04-14T08:43:09.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.006+0000 7fca9bc31740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561464dc2c00 0x561465b86000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.006+0000 7fca9bc31740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561464dc3800 0x561465b86580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.006+0000 7fca9bc31740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561464dc3c00 0x561465b86b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.006+0000 7fca9bc31740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561465ba2180 con 0x561464dc3c00 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.006+0000 7fca9bc31740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561465ba2300 con 0x561464dc2c00 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.006+0000 7fca9bc31740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x561465ba2480 con 0x561464dc3800 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.006+0000 7fca999fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561464dc3c00 0x561465b86b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.006+0000 7fca999fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561464dc3c00 0x561465b86b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59818/0 (socket says 172.21.15.172:59818) 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.006+0000 7fca999fd640 1 -- 172.21.15.172:0/3293209453 learned_addr learned my addr 172.21.15.172:0/3293209453 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca991fc640 1 --2- 172.21.15.172:0/3293209453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561464dc2c00 0x561465b86000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca999fd640 1 -- 172.21.15.172:0/3293209453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561464dc3800 msgr2=0x561465b86580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca999fd640 1 --2- 172.21.15.172:0/3293209453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561464dc3800 0x561465b86580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca999fd640 1 -- 172.21.15.172:0/3293209453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561464dc2c00 msgr2=0x561465b86000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:09.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca999fd640 1 --2- 172.21.15.172:0/3293209453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561464dc2c00 0x561465b86000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca999fd640 1 -- 172.21.15.172:0/3293209453 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561464def680 con 0x561464dc3c00 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca999fd640 1 --2- 172.21.15.172:0/3293209453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561464dc3c00 0x561465b86b00 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x561465bdf020 tx=0x561465b5c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad9be7faf6df4ca server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca969f7640 1 -- 172.21.15.172:0/3293209453 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561464df89c0 con 0x561464dc3c00 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca969f7640 1 -- 172.21.15.172:0/3293209453 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561464e0e1c0 con 0x561464dc3c00 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca9bc31740 1 -- 172.21.15.172:0/3293209453 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561464def860 con 0x561464dc3c00 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca9bc31740 1 -- 172.21.15.172:0/3293209453 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x561464def2c0 con 0x561464dc3c00 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca9bc31740 4 waiting for MDS map... 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca969f7640 1 -- 172.21.15.172:0/3293209453 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561464df84e0 con 0x561464dc3c00 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca969f7640 1 -- 172.21.15.172:0/3293209453 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x561465c94200 con 0x561464dc3c00 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca969f7640 1 -- 172.21.15.172:0/3293209453 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 94) ==== 2885+0+0 (secure 0 0 0) 0x561464e1d180 con 0x561464dc3c00 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca9bc31740 4 Got MDS map 94 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca9bc31740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-14T08:43:09.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca9bc31740 4 datascan.main: Using metadata pool output 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.007+0000 7fca9bc31740 4 datascan.main: connecting to RADOS... 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.008+0000 7fca9bc31740 1 Processor -- start 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.008+0000 7fca9bc31740 1 -- start start 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.008+0000 7fca9bc31740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 0x561465b87080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.008+0000 7fca9bc31740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561465ce6400 0x561465b87600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.008+0000 7fca9bc31740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561465ce6800 0x561465b87b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.008+0000 7fca9bc31740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561465ba2600 con 0x561465ce6000 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.008+0000 7fca9bc31740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561465ba2780 con 0x561465ce6800 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.008+0000 7fca9bc31740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x561465ba2900 con 0x561465ce6400 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.008+0000 7fca989fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 0x561465b87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.008+0000 7fca989fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 0x561465b87080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59832/0 (socket says 172.21.15.172:59832) 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.008+0000 7fca989fb640 1 -- 172.21.15.172:0/1551621947 learned_addr learned my addr 172.21.15.172:0/1551621947 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca991fc640 1 --2- 172.21.15.172:0/1551621947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561465ce6400 0x561465b87600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca989fb640 1 --2- 172.21.15.172:0/1551621947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561465ce6800 0x561465b87b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca989fb640 1 -- 172.21.15.172:0/1551621947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561465ce6400 msgr2=0x561465b87600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca989fb640 1 --2- 172.21.15.172:0/1551621947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561465ce6400 0x561465b87600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca989fb640 1 -- 172.21.15.172:0/1551621947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561465ce6800 msgr2=0x561465b87b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca989fb640 1 --2- 172.21.15.172:0/1551621947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561465ce6800 0x561465b87b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca989fb640 1 -- 172.21.15.172:0/1551621947 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561464def4a0 con 0x561465ce6000 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca989fb640 1 --2- 172.21.15.172:0/1551621947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 0x561465b87080 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x561465c84a20 tx=0x561465c9cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f211b0b4068cfa78 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca949f3640 1 -- 172.21.15.172:0/1551621947 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561464df89c0 con 0x561465ce6000 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca949f3640 1 -- 172.21.15.172:0/1551621947 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561464e0ee00 con 0x561465ce6000 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca949f3640 1 -- 172.21.15.172:0/1551621947 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561464df84e0 con 0x561465ce6000 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca9bc31740 1 -- 172.21.15.172:0/1551621947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 msgr2=0x561465b87080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca9bc31740 1 --2- 172.21.15.172:0/1551621947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 0x561465b87080 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x561465c84a20 tx=0x561465c9cd20 comp rx=0 tx=0).stop 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca9bc31740 1 -- 172.21.15.172:0/1551621947 shutdown_connections 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca9bc31740 1 --2- 172.21.15.172:0/1551621947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561465ce6800 0x561465b87b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca9bc31740 1 --2- 172.21.15.172:0/1551621947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561465ce6400 0x561465b87600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca9bc31740 1 --2- 172.21.15.172:0/1551621947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 0x561465b87080 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca9bc31740 1 -- 172.21.15.172:0/1551621947 >> 172.21.15.172:0/1551621947 conn(0x561464eb9c00 msgr2=0x561464ea7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca9bc31740 1 -- 172.21.15.172:0/1551621947 shutdown_connections 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.009+0000 7fca9bc31740 1 -- 172.21.15.172:0/1551621947 wait complete. 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.010+0000 7fca9bc31740 1 Processor -- start 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.010+0000 7fca9bc31740 1 -- start start 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca9bc31740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 0x561465b87080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca9bc31740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561465ce6400 0x561465b87600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca9bc31740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561465ce6800 0x561465b87b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca9bc31740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561465ba3c80 con 0x561465ce6000 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca9bc31740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561465ba3b00 con 0x561465ce6400 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca9bc31740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x561465ba3980 con 0x561465ce6800 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca989fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 0x561465b87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca989fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 0x561465b87080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59838/0 (socket says 172.21.15.172:59838) 2024-04-14T08:43:09.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 learned_addr learned my addr 172.21.15.172:0/3456042641 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca991fc640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561465ce6400 0x561465b87600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca989fb640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561465ce6800 0x561465b87b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561465ce6800 msgr2=0x561465b87b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca989fb640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561465ce6800 0x561465b87b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561465ce6400 msgr2=0x561465b87600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca989fb640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561465ce6400 0x561465b87600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561464def0e0 con 0x561465ce6000 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca989fb640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 0x561465b87080 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x561465c856e0 tx=0x561465b5ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca931f0640 1 -- 172.21.15.172:0/3456042641 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561464df8000 con 0x561465ce6000 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca931f0640 1 -- 172.21.15.172:0/3456042641 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561464e0ec40 con 0x561465ce6000 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561464defa40 con 0x561465ce6000 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca931f0640 1 -- 172.21.15.172:0/3456042641 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561464df89c0 con 0x561465ce6000 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561464dee3c0 con 0x561465ce6000 2024-04-14T08:43:09.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.011+0000 7fca9bc31740 4 datascan.init: resolving metadata pool 17 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.013+0000 7fca931f0640 1 -- 172.21.15.172:0/3456042641 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x561465cf6c00 con 0x561465ce6000 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.013+0000 7fca931f0640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x56146513a000 0x561465b88100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.013+0000 7fca931f0640 1 -- 172.21.15.172:0/3456042641 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x561465ceca00 con 0x561465ce6000 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.013+0000 7fca9bc31740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.013+0000 7fca9bc31740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.013+0000 7fca9bc31740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.013+0000 7fca991fc640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x56146513a000 0x561465b88100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.013+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x561465ce6c00 0x561465b88680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.013+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x561465ce7000 con 0x561465ce6c00 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.013+0000 7fca989fb640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x561465ce6c00 0x561465b88680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.013+0000 7fca991fc640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x56146513a000 0x561465b88100 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x561465bdf8f0 tx=0x561465144000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.013+0000 7fca989fb640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x561465ce6c00 0x561465b88680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.014+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.014+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x56146513b000 0x561465b88c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.014+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x56146513b400 con 0x56146513b000 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.014+0000 7fca999fd640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x56146513b000 0x561465b88c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.014+0000 7fca999fd640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x56146513b000 0x561465b88c00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.015+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x561464ec8500 con 0x56146513b000 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.015+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x56146513b800 0x561465b89180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.015+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146513bc00 con 0x56146513b800 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.015+0000 7fca991fc640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x56146513b800 0x561465b89180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.015+0000 7fca991fc640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x56146513b800 0x561465b89180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.015+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.015+0000 7fca941f2640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x561465156000 0x56146515c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.015+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465156400 con 0x561465156000 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.015+0000 7fca989fb640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x561465156000 0x56146515c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.016+0000 7fca989fb640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x561465156000 0x56146515c000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.016+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.016+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465156c00 con 0x56146513b000 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.016+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.6 v2:172.21.15.177:6800/2961907928 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146513b000 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.016+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465156800 con 0x56146513b800 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.017+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.017+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465157000 con 0x56146513b800 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.017+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.017+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465156800 con 0x561465156000 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.017+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.017+0000 7fca949f3640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x561465157400 0x56146515c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.017+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465157000 con 0x561465157400 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.017+0000 7fca999fd640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x561465157400 0x56146515c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.018+0000 7fca999fd640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x561465157400 0x56146515c580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.018+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x561465157400 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.018+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146513ac00 con 0x561465ce6c00 2024-04-14T08:43:09.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.018+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.018+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465157800 con 0x56146513b800 2024-04-14T08:43:09.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.018+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.018+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146513bc00 con 0x56146513b800 2024-04-14T08:43:09.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.018+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.019+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465157c00 con 0x56146513b000 2024-04-14T08:43:09.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.019+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.6 v2:172.21.15.177:6800/2961907928 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146513b000 2024-04-14T08:43:09.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.019+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146513ac00 con 0x56146513b800 2024-04-14T08:43:09.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.019+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.019+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a000 con 0x561465156000 2024-04-14T08:43:09.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.019+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.019+0000 7fca941f2640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x561465156400 0x56146515cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.019+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465156800 con 0x561465156400 2024-04-14T08:43:09.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.020+0000 7fca991fc640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x561465156400 0x56146515cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.020+0000 7fca991fc640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x561465156400 0x56146515cb00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.020+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.2 v2:172.21.15.172:6816/1609902500 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x561465156400 2024-04-14T08:43:09.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.020+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a400 con 0x561465157400 2024-04-14T08:43:09.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.020+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.1 v2:172.21.15.172:6823/1219922421 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x561465157400 2024-04-14T08:43:09.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.020+0000 7fca941f2640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x561465ce7000 0x56146515d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.020+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a400 con 0x561465ce7000 2024-04-14T08:43:09.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.020+0000 7fca989fb640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x561465ce7000 0x56146515d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.021+0000 7fca989fb640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x561465ce7000 0x56146515d080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.021+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce7000 2024-04-14T08:43:09.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.021+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a800 con 0x561465ce7000 2024-04-14T08:43:09.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.021+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.7 v2:172.21.15.177:6808/3609985207 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce7000 2024-04-14T08:43:09.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.021+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465156c00 con 0x561465ce6c00 2024-04-14T08:43:09.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.022+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.022+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516ac00 con 0x561465156400 2024-04-14T08:43:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.022+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.2 v2:172.21.15.172:6816/1609902500 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 118 out=81b] v118'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x561464ec8780 con 0x561465156400 2024-04-14T08:43:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.022+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516ac00 con 0x56146513b800 2024-04-14T08:43:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.022+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.022+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516b800 con 0x561465156000 2024-04-14T08:43:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.023+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.023+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465157000 con 0x56146513b000 2024-04-14T08:43:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.023+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.6 v2:172.21.15.177:6800/2961907928 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146513b000 2024-04-14T08:43:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.023+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516bc00 con 0x56146513b800 2024-04-14T08:43:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.023+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.023+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465157c00 con 0x56146513b800 2024-04-14T08:43:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.023+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.023+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a000 con 0x561465156000 2024-04-14T08:43:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.024+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.024+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146513b400 con 0x561465157400 2024-04-14T08:43:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.024+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.1 v2:172.21.15.172:6823/1219922421 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x561465157400 2024-04-14T08:43:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.024+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a400 con 0x561465ce6c00 2024-04-14T08:43:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.024+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.024+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176000 con 0x56146513b800 2024-04-14T08:43:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.024+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.024+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a800 con 0x56146513b800 2024-04-14T08:43:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.025+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.025+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176400 con 0x56146513b000 2024-04-14T08:43:09.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.025+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.6 v2:172.21.15.177:6800/2961907928 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146513b000 2024-04-14T08:43:09.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.025+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516b800 con 0x56146513b800 2024-04-14T08:43:09.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.025+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.025+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176800 con 0x561465156000 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.025+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.025+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465156c00 con 0x561465156400 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.026+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.2 v2:172.21.15.172:6816/1609902500 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x561465156400 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.026+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176c00 con 0x561465157400 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.026+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.1 v2:172.21.15.172:6823/1219922421 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x561465157400 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.026+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176000 con 0x561465ce7000 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.026+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.7 v2:172.21.15.177:6808/3609985207 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce7000 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.026+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465177000 con 0x561465ce7000 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.026+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.7 v2:172.21.15.177:6808/3609985207 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce7000 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.026+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465177800 con 0x561465ce6c00 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.027+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.027+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465177400 con 0x561465156400 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.027+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.2 v2:172.21.15.172:6816/1609902500 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 118 out=49b] v118'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x561465156400 2024-04-14T08:43:09.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.027+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516ac00 con 0x561465ce6c00 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.027+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 118 out=49b] v116'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.027+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176800 con 0x561465ce6c00 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.027+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 7 ==== osd_op_reply(42 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.027+0000 7fca949f3640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x56146516bc00 0x56146515d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.027+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a800 con 0x56146516bc00 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.028+0000 7fca999fd640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x56146516bc00 0x56146515d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.028+0000 7fca999fd640 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x56146516bc00 0x56146515d600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.028+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.4 v2:172.21.15.177:6820/4180733146 1 ==== osd_op_reply(43 [pgnls-filter start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146516bc00 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.028+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176000 con 0x561465ce6c00 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.028+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.028+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516b800 con 0x56146513b800 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.029+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 13 ==== osd_op_reply(45 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.029+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465177000 con 0x561465ce7000 2024-04-14T08:43:09.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.029+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.7 v2:172.21.15.177:6808/3609985207 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce7000 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.029+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465157c00 con 0x561465156400 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.029+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.2 v2:172.21.15.172:6816/1609902500 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x561465156400 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.029+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465177800 con 0x561465ce6c00 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465182000 con 0x561465157400 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.1 v2:172.21.15.172:6823/1219922421 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 118 out=81b] v118'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x561464ec8500 con 0x561465157400 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca9bc31740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca9bc31740 10 datascan.operator(): handling object 100000001f7.0 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x56146516b400 con 0x561465157400 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.1 v2:172.21.15.172:6823/1219922421 6 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x561464ec8500 con 0x561465157400 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca9bc31740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca9bc31740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca9bc31740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x561465157800 con 0x56146513b800 2024-04-14T08:43:09.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 14 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca9bc31740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.030+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x561465156800 con 0x561465ce6c00 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 10 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) ==== 164+0+585 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca9bc31740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca9bc31740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca9bc31740 20 datascan.get_frag_of: frag is * 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x56146513bc00 con 0x56146513b800 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 15 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca9bc31740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca9bc31740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x56146513ac00 con 0x56146513b800 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 16 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca9bc31740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-14T08:43:09.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca9bc31740 20 datascan.read_dentry: bravo_head not found in result 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca9bc31740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.031+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x561465177400 con 0x56146513b800 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.033+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 17 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v118'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.033+0000 7fca9bc31740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.033+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146513bc00 con 0x56146513b800 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.033+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.033+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465182400 con 0x561465ce6c00 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.033+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 11 ==== osd_op_reply(57 [pgnls-filter start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.033+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465177c00 con 0x561465156000 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.033+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.034+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465182800 con 0x561465ce6c00 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.034+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 12 ==== osd_op_reply(59 [pgnls-filter start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.034+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a000 con 0x561465ce6c00 2024-04-14T08:43:09.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.034+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 13 ==== osd_op_reply(60 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.034+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465182c00 con 0x561465156000 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.034+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 118 out=49b] v116'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.034+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a400 con 0x561465157400 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.035+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.1 v2:172.21.15.172:6823/1219922421 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x561465157400 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.035+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465183000 con 0x56146516bc00 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.035+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.4 v2:172.21.15.177:6820/4180733146 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 118 out=49b] v116'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146516bc00 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.035+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516ac00 con 0x561465ce6c00 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.036+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 118 out=49b] v116'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.036+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465183400 con 0x561465ce7000 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.036+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.7 v2:172.21.15.177:6808/3609985207 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce7000 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.036+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176800 con 0x56146516bc00 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.037+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.4 v2:172.21.15.177:6820/4180733146 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 118 out=49b] v118'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146516bc00 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.037+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465183800 con 0x56146513b800 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.037+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.037+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176000 con 0x561465157400 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.037+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.1 v2:172.21.15.172:6823/1219922421 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 118 out=49b] v118'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x561465157400 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.037+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465183c00 con 0x56146513b000 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.037+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.6 v2:172.21.15.177:6800/2961907928 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146513b000 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.037+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465177000 con 0x561465156000 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.038+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.038+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465183000 con 0x56146516bc00 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.038+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.4 v2:172.21.15.177:6820/4180733146 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146516bc00 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.038+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465177800 con 0x56146516bc00 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.038+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.4 v2:172.21.15.177:6820/4180733146 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146516bc00 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.038+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465183c00 con 0x56146513b000 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.039+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.6 v2:172.21.15.177:6800/2961907928 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146513b000 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.039+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176400 con 0x56146513b800 2024-04-14T08:43:09.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.039+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.0 v2:172.21.15.172:6800/3761711279 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x56146513b800 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.039+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516b400 con 0x561465ce6c00 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.039+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.3 v2:172.21.15.172:6808/2305139122 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce6c00 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.039+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176c00 con 0x561465157400 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.039+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.1 v2:172.21.15.172:6823/1219922421 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x561465157400 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.039+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516b000 con 0x561465156400 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.040+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.2 v2:172.21.15.172:6816/1609902500 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x561465156400 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.040+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465156800 con 0x561465ce7000 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.040+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.7 v2:172.21.15.177:6808/3609985207 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce7000 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.040+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a400 con 0x561465157400 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.040+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.1 v2:172.21.15.172:6823/1219922421 10 ==== osd_op_reply(79 [pgnls-filter start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x561465157400 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.040+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465182800 con 0x561465156000 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.041+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.041+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516b800 con 0x561465ce7000 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.041+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.7 v2:172.21.15.177:6808/3609985207 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce7000 2024-04-14T08:43:09.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.041+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465182c00 con 0x561465157400 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.041+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.1 v2:172.21.15.172:6823/1219922421 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x561465157400 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.041+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a800 con 0x56146516bc00 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.042+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.4 v2:172.21.15.177:6820/4180733146 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146516bc00 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.042+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465183400 con 0x561465156400 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.042+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 <== osd.2 v2:172.21.15.172:6816/1609902500 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8780 con 0x561465156400 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.042+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176800 con 0x561465156000 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.042+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 118 out=49b] v116'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.042+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465182400 con 0x561465156000 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.043+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.043+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465176000 con 0x561465ce7000 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.043+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.7 v2:172.21.15.177:6808/3609985207 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce7000 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.043+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516a000 con 0x561465156000 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.043+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.5 v2:172.21.15.177:6816/3172735715 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465156000 2024-04-14T08:43:09.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.043+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465182000 con 0x561465157400 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.1 v2:172.21.15.172:6823/1219922421 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x561465157400 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca941f2640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x56146516ac00 con 0x561465ce7000 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca989fb640 1 -- 172.21.15.172:0/3456042641 <== osd.7 v2:172.21.15.177:6808/3609985207 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8000 con 0x561465ce7000 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca949f3640 1 -- 172.21.15.172:0/3456042641 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x561465183000 con 0x56146516bc00 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca999fd640 1 -- 172.21.15.172:0/3456042641 <== osd.4 v2:172.21.15.177:6820/4180733146 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 118 out=49b] v116'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561464ec8500 con 0x56146516bc00 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x56146513b800 msgr2=0x561465b89180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x56146513b800 0x561465b89180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x561465157400 msgr2=0x56146515c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x561465157400 0x56146515c580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x561465156400 msgr2=0x56146515cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x561465156400 0x56146515cb00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x561465ce6c00 msgr2=0x561465b88680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x561465ce6c00 0x561465b88680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x56146516bc00 msgr2=0x56146515d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x56146516bc00 0x56146515d600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 reap_dead start 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x561465156000 msgr2=0x56146515c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x561465156000 0x56146515c000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x56146513b000 msgr2=0x561465b88c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x56146513b000 0x561465b88c00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x561465ce7000 msgr2=0x56146515d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.044+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x561465ce7000 0x56146515d080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 reap_dead start 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x56146513a000 msgr2=0x561465b88100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca991fc640 1 -- 172.21.15.172:0/3456042641 reap_dead start 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x56146513a000 0x561465b88100 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x561465bdf8f0 tx=0x561465144000 comp rx=0 tx=0).stop 2024-04-14T08:43:09.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 msgr2=0x561465b87080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 0x561465b87080 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x561465c856e0 tx=0x561465b5ca80 comp rx=0 tx=0).stop 2024-04-14T08:43:09.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 shutdown_connections 2024-04-14T08:43:09.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x56146513a000 0x561465b88100 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3456042641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561465ce6000 0x561465b87080 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 >> 172.21.15.172:0/3456042641 conn(0x561464eb9c00 msgr2=0x561464ea7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:09.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 shutdown_connections 2024-04-14T08:43:09.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 -- 172.21.15.172:0/3456042641 wait complete. 2024-04-14T08:43:09.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 -- 172.21.15.172:0/3293209453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561464dc3c00 msgr2=0x561465b86b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3293209453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561464dc3c00 0x561465b86b00 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x561465bdf020 tx=0x561465b5c840 comp rx=0 tx=0).stop 2024-04-14T08:43:09.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 -- 172.21.15.172:0/3293209453 shutdown_connections 2024-04-14T08:43:09.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3293209453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561464dc3c00 0x561465b86b00 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3293209453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561464dc3800 0x561465b86580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 --2- 172.21.15.172:0/3293209453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561464dc2c00 0x561465b86000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 -- 172.21.15.172:0/3293209453 >> 172.21.15.172:0/3293209453 conn(0x561464dc3000 msgr2=0x561464ea6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:09.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 -- 172.21.15.172:0/3293209453 shutdown_connections 2024-04-14T08:43:09.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.045+0000 7fca9bc31740 1 -- 172.21.15.172:0/3293209453 wait complete. 2024-04-14T08:43:09.066 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.137568, result: 2024-04-14T08:43:09.066 DEBUG:teuthology.orchestra.run.smithi172:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-14T08:43:09.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.133+0000 7fe35b251740 1 -- start start 2024-04-14T08:43:09.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.133+0000 7fe35b251740 1 Processor -- start 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.133+0000 7fe35b251740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a89cb0c00 0x557a8aa84000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.133+0000 7fe35b251740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a89cb1800 0x557a8aa84580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.133+0000 7fe35b251740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a89cb1c00 0x557a8aa84b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.133+0000 7fe35b251740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557a8aa98180 con 0x557a89cb1c00 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.133+0000 7fe35b251740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557a8aa98300 con 0x557a89cb0c00 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.133+0000 7fe35b251740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x557a8aa98480 con 0x557a89cb1800 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3597fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a89cb1c00 0x557a8aa84b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3597fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a89cb1c00 0x557a8aa84b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59860/0 (socket says 172.21.15.172:59860) 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3597fd640 1 -- 172.21.15.172:0/3254383253 learned_addr learned my addr 172.21.15.172:0/3254383253 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3587fb640 1 --2- 172.21.15.172:0/3254383253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a89cb1800 0x557a8aa84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe358ffc640 1 --2- 172.21.15.172:0/3254383253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a89cb0c00 0x557a8aa84000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3597fd640 1 -- 172.21.15.172:0/3254383253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a89cb1800 msgr2=0x557a8aa84580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3597fd640 1 --2- 172.21.15.172:0/3254383253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a89cb1800 0x557a8aa84580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3597fd640 1 -- 172.21.15.172:0/3254383253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a89cb0c00 msgr2=0x557a8aa84000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3597fd640 1 --2- 172.21.15.172:0/3254383253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a89cb0c00 0x557a8aa84000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3597fd640 1 -- 172.21.15.172:0/3254383253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a89cdd680 con 0x557a89cb1c00 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3597fd640 1 --2- 172.21.15.172:0/3254383253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a89cb1c00 0x557a8aa84b00 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x557a8aad5a10 tx=0x557a8aa4c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9b2aebbfd4a10c server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3567f7640 1 -- 172.21.15.172:0/3254383253 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a89ce64e0 con 0x557a89cb1c00 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3567f7640 1 -- 172.21.15.172:0/3254383253 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557a89cfc1c0 con 0x557a89cb1c00 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3567f7640 1 -- 172.21.15.172:0/3254383253 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a89ce6b60 con 0x557a89cb1c00 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe35b251740 1 -- 172.21.15.172:0/3254383253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a89cdd860 con 0x557a89cb1c00 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe35b251740 1 -- 172.21.15.172:0/3254383253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=0}) -- 0x557a89cdd2c0 con 0x557a89cb1c00 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe35b251740 4 waiting for MDS map... 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3567f7640 1 -- 172.21.15.172:0/3254383253 <== mon.0 v2:172.21.15.172:3300/0 4 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x557a8ab94a00 con 0x557a89cb1c00 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe3567f7640 1 -- 172.21.15.172:0/3254383253 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 94) ==== 2885+0+0 (secure 0 0 0) 0x557a89d0b180 con 0x557a89cb1c00 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.134+0000 7fe35b251740 4 Got MDS map 94 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.135+0000 7fe35b251740 4 datascan.main: Using metadata pool output 2024-04-14T08:43:09.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.135+0000 7fe35b251740 4 datascan.main: connecting to RADOS... 2024-04-14T08:43:09.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.135+0000 7fe35b251740 1 Processor -- start 2024-04-14T08:43:09.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.135+0000 7fe35b251740 1 -- start start 2024-04-14T08:43:09.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe35b251740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a8ab97400 0x557a8aa85080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe35b251740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a8ab97800 0x557a8aa85600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe35b251740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 0x557a8aa85b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe35b251740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557a8aa98600 con 0x557a8ab97c00 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe35b251740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557a8aa98780 con 0x557a8ab97400 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe35b251740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x557a8aa98900 con 0x557a8ab97800 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3587fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 0x557a8aa85b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3587fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 0x557a8aa85b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59870/0 (socket says 172.21.15.172:59870) 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3587fb640 1 -- 172.21.15.172:0/4118819294 learned_addr learned my addr 172.21.15.172:0/4118819294 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe358ffc640 1 --2- 172.21.15.172:0/4118819294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a8ab97800 0x557a8aa85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3587fb640 1 --2- 172.21.15.172:0/4118819294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a8ab97400 0x557a8aa85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3587fb640 1 -- 172.21.15.172:0/4118819294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a8ab97800 msgr2=0x557a8aa85600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3587fb640 1 --2- 172.21.15.172:0/4118819294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a8ab97800 0x557a8aa85600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3587fb640 1 -- 172.21.15.172:0/4118819294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a8ab97400 msgr2=0x557a8aa85080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:09.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3587fb640 1 --2- 172.21.15.172:0/4118819294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a8ab97400 0x557a8aa85080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3587fb640 1 -- 172.21.15.172:0/4118819294 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a89cdd4a0 con 0x557a8ab97c00 2024-04-14T08:43:09.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3587fb640 1 --2- 172.21.15.172:0/4118819294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 0x557a8aa85b80 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x557a8aad5080 tx=0x557a8aa4a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af1a904f35ad4ec server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3547f3640 1 -- 172.21.15.172:0/4118819294 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a89ce69c0 con 0x557a8ab97c00 2024-04-14T08:43:09.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3547f3640 1 -- 172.21.15.172:0/4118819294 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557a89cfce00 con 0x557a8ab97c00 2024-04-14T08:43:09.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.136+0000 7fe3547f3640 1 -- 172.21.15.172:0/4118819294 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a89ce64e0 con 0x557a8ab97c00 2024-04-14T08:43:09.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.137+0000 7fe35b251740 1 -- 172.21.15.172:0/4118819294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 msgr2=0x557a8aa85b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.137+0000 7fe35b251740 1 --2- 172.21.15.172:0/4118819294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 0x557a8aa85b80 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x557a8aad5080 tx=0x557a8aa4a6c0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.137+0000 7fe35b251740 1 -- 172.21.15.172:0/4118819294 shutdown_connections 2024-04-14T08:43:09.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.137+0000 7fe35b251740 1 --2- 172.21.15.172:0/4118819294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 0x557a8aa85b80 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.137+0000 7fe35b251740 1 --2- 172.21.15.172:0/4118819294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a8ab97800 0x557a8aa85600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.137+0000 7fe35b251740 1 --2- 172.21.15.172:0/4118819294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a8ab97400 0x557a8aa85080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.137+0000 7fe35b251740 1 -- 172.21.15.172:0/4118819294 >> 172.21.15.172:0/4118819294 conn(0x557a8ab97000 msgr2=0x557a89d95000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.137+0000 7fe35b251740 1 -- 172.21.15.172:0/4118819294 shutdown_connections 2024-04-14T08:43:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.137+0000 7fe35b251740 1 -- 172.21.15.172:0/4118819294 wait complete. 2024-04-14T08:43:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.137+0000 7fe35b251740 1 Processor -- start 2024-04-14T08:43:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.137+0000 7fe35b251740 1 -- start start 2024-04-14T08:43:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe35b251740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a8ab97400 0x557a8aa85080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe35b251740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a8ab97800 0x557a8aa85600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe35b251740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 0x557a8aa85b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe35b251740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557a8aa99b00 con 0x557a8ab97c00 2024-04-14T08:43:09.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe35b251740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557a8aa99980 con 0x557a8ab97400 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe35b251740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x557a8aa99800 con 0x557a8ab97800 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe3587fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 0x557a8aa85b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe3587fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 0x557a8aa85b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59876/0 (socket says 172.21.15.172:59876) 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 learned_addr learned my addr 172.21.15.172:0/1371652405 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe358ffc640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a8ab97800 0x557a8aa85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a8ab97400 0x557a8aa85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a8ab97800 msgr2=0x557a8aa85600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a8ab97800 0x557a8aa85600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a8ab97400 msgr2=0x557a8aa85080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a8ab97400 0x557a8aa85080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a89cdd680 con 0x557a8ab97c00 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.138+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 0x557a8aa85b80 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x557a8ab82f30 tx=0x557a8aa4aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.139+0000 7fe352ff0640 1 -- 172.21.15.172:0/1371652405 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a89ce6b60 con 0x557a8ab97c00 2024-04-14T08:43:09.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.139+0000 7fe352ff0640 1 -- 172.21.15.172:0/1371652405 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557a89cfcc40 con 0x557a8ab97c00 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.139+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557a89cdd860 con 0x557a8ab97c00 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.139+0000 7fe352ff0640 1 -- 172.21.15.172:0/1371652405 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a89ce6000 con 0x557a8ab97c00 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.139+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a89cdd0e0 con 0x557a8ab97c00 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.139+0000 7fe35b251740 4 datascan.init: resolving metadata pool 17 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.140+0000 7fe352ff0640 1 -- 172.21.15.172:0/1371652405 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x557a8abf8c00 con 0x557a8ab97c00 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.140+0000 7fe352ff0640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x557a8a02c000 0x557a8aa86100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.140+0000 7fe352ff0640 1 -- 172.21.15.172:0/1371652405 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x557a8aabc600 con 0x557a8ab97c00 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.140+0000 7fe35b251740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.140+0000 7fe35b251740 4 datascan.main: resolving metadata pool 17 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.140+0000 7fe358ffc640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x557a8a02c000 0x557a8aa86100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.140+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x557a8a02c400 0x557a8aa86680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.140+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a02c800 con 0x557a8a02c400 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.140+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x557a8a02c400 0x557a8aa86680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.141+0000 7fe358ffc640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x557a8a02c000 0x557a8aa86100 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x557a8aad58f0 tx=0x557a89fc0a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.141+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x557a8a02c400 0x557a8aa86680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.141+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== osd_op_reply(1 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.141+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x557a8a02c800 0x557a8a034000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.141+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8aaa3400 con 0x557a8a02c800 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.141+0000 7fe3597fd640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x557a8a02c800 0x557a8a034000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.142+0000 7fe3597fd640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x557a8a02c800 0x557a8a034000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.142+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 1 ==== osd_op_reply(2 [pgnls start_epoch 118 out=177b] v117'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.142+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8aaa3400 con 0x557a8a02c400 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.142+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 2 ==== osd_op_reply(3 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.142+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x557a8aaa3400 0x557a8a034580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.142+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046000 con 0x557a8aaa3400 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.143+0000 7fe358ffc640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x557a8aaa3400 0x557a8a034580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.143+0000 7fe358ffc640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x557a8aaa3400 0x557a8a034580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.143+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== osd_op_reply(4 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.143+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046000 con 0x557a8aaa3400 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.143+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 2 ==== osd_op_reply(5 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.143+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046000 con 0x557a8a02c800 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.144+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 2 ==== osd_op_reply(6 [pgnls start_epoch 118 out=149b] v118'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.144+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046000 con 0x557a8a02c400 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.144+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 3 ==== osd_op_reply(7 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.144+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x557a8a046000 0x557a8a034b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.144+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046400 con 0x557a8a046000 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.144+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x557a8a046000 0x557a8a034b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.144+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x557a8a046000 0x557a8a034b00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.145+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== osd_op_reply(8 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a046000 2024-04-14T08:43:09.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.145+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x557a8a046400 0x557a8a035080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.145+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046800 con 0x557a8a046400 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.145+0000 7fe3597fd640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x557a8a046400 0x557a8a035080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.145+0000 7fe3597fd640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x557a8a046400 0x557a8a035080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.146+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.4 v2:172.21.15.177:6820/4180733146 1 ==== osd_op_reply(9 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a046400 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.146+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046800 con 0x557a8a02c800 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.146+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 3 ==== osd_op_reply(10 [pgnls start_epoch 118 out=81b] v117'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.146+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046800 con 0x557a8a02c400 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.146+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 4 ==== osd_op_reply(11 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.146+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046800 con 0x557a8aaa3400 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.146+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 3 ==== osd_op_reply(12 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.146+0000 7fe358ffc640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x557a8a046800 0x557a8a036100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.146+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047000 con 0x557a8a046800 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.147+0000 7fe358ffc640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x557a8a046800 0x557a8a036100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.147+0000 7fe358ffc640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x557a8a046800 0x557a8a036100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.147+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== osd_op_reply(13 [pgnls start_epoch 118 out=77b] v117'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x557a8aa32780 con 0x557a8a046800 2024-04-14T08:43:09.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.147+0000 7fe358ffc640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x557a8a047000 0x557a8a036680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.147+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047400 con 0x557a8a047000 2024-04-14T08:43:09.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.147+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x557a8a047000 0x557a8a036680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.148+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x557a8a047000 0x557a8a036680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.148+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.5 v2:172.21.15.177:6816/3172735715 1 ==== osd_op_reply(14 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a047000 2024-04-14T08:43:09.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.148+0000 7fe3587fb640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x557a8a047400 0x557a8a035600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.148+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a047400 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.148+0000 7fe3597fd640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x557a8a047400 0x557a8a035600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.148+0000 7fe3597fd640 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x557a8a047400 0x557a8a035600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.149+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.2 v2:172.21.15.172:6816/1609902500 1 ==== osd_op_reply(15 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a047400 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.149+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a046000 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.149+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.1 v2:172.21.15.172:6823/1219922421 2 ==== osd_op_reply(16 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a046000 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.149+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8aaa3400 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.149+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 4 ==== osd_op_reply(17 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.149+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a02c800 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.149+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 4 ==== osd_op_reply(18 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.149+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a02c800 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.150+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 5 ==== osd_op_reply(19 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.150+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a047000 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.150+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.5 v2:172.21.15.177:6816/3172735715 2 ==== osd_op_reply(20 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a047000 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.150+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a046800 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.150+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.6 v2:172.21.15.177:6800/2961907928 2 ==== osd_op_reply(21 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32780 con 0x557a8a046800 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.150+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a047000 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.150+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.5 v2:172.21.15.177:6816/3172735715 3 ==== osd_op_reply(22 [pgnls start_epoch 118 out=97b] v117'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a047000 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.150+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a02c400 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.151+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 5 ==== osd_op_reply(23 [pgnls start_epoch 118 out=74b] v115'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.151+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a02c400 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.151+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 6 ==== osd_op_reply(24 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.151+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a046400 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.151+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.4 v2:172.21.15.177:6820/4180733146 2 ==== osd_op_reply(25 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a046400 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.151+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a02c800 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.152+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 6 ==== osd_op_reply(26 [pgnls start_epoch 118 out=98b] v117'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.152+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8aaa3400 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.152+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 5 ==== osd_op_reply(27 [pgnls start_epoch 118 out=76b] v118'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.152+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a046400 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.152+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.4 v2:172.21.15.177:6820/4180733146 3 ==== osd_op_reply(28 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a046400 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.152+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8aaa3400 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.152+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 6 ==== osd_op_reply(29 [pgnls start_epoch 118 out=105b] v118'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.152+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a046800 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.153+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.6 v2:172.21.15.177:6800/2961907928 3 ==== osd_op_reply(30 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32780 con 0x557a8a046800 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.153+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a046000 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.153+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.1 v2:172.21.15.172:6823/1219922421 3 ==== osd_op_reply(31 [pgnls start_epoch 118 out=73b] v118'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a046000 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.153+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a046800 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.153+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.6 v2:172.21.15.177:6800/2961907928 4 ==== osd_op_reply(32 [pgnls start_epoch 118 out=101b] v118'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x557a8aa32780 con 0x557a8a046800 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.153+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.153+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a02cc00 con 0x557a8a02c800 2024-04-14T08:43:09.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.154+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.154+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.154+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a02d000 con 0x557a8a02c800 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.154+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.154+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.154+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a02d400 con 0x557a8a02c800 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.154+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.154+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.154+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a02c800 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a02c800 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-14T08:43:09.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a02cc00 con 0x557a8a02c800 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.155+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a02d000 con 0x557a8a046000 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.156+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.1 v2:172.21.15.172:6823/1219922421 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a046000 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.156+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.156+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a02d400 con 0x557a8a02c800 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.156+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 13 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) ==== 164+0+584 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.156+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.156+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.156+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.156+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8ab96800 con 0x557a8a047400 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.156+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.2 v2:172.21.15.172:6816/1609902500 2 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a047400 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.156+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.156+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05e000 con 0x557a8a047000 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.157+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.5 v2:172.21.15.177:6816/3172735715 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a047000 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.157+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.157+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.157+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.157+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.157+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.157+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05e400 con 0x557a8a02c400 2024-04-14T08:43:09.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.157+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.157+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.157+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.157+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.157+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05e800 con 0x557a8a02c800 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05ec00 con 0x557a8aaa3400 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05f000 con 0x557a8aaa3400 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) ==== 164+0+1667 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 20 datascan.scan_links: injected ino detected 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.158+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05f400 con 0x557a8a046800 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.159+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.6 v2:172.21.15.177:6800/2961907928 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x557a8aa32780 con 0x557a8a046800 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.159+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05f800 con 0x557a8a02c400 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.159+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 8 ==== osd_op_reply(48 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.159+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05f800 con 0x557a8a02c800 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.160+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 15 ==== osd_op_reply(49 [pgnls start_epoch 118 out=177b] v117'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.160+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a02c400 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.160+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 9 ==== osd_op_reply(50 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.160+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8aaa3400 2024-04-14T08:43:09.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.160+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 9 ==== osd_op_reply(51 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.160+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8aaa3400 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.160+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 10 ==== osd_op_reply(52 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.160+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a02c800 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.161+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 16 ==== osd_op_reply(53 [pgnls start_epoch 118 out=149b] v118'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.161+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a02c400 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.161+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 10 ==== osd_op_reply(54 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.161+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a046000 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.162+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.1 v2:172.21.15.172:6823/1219922421 5 ==== osd_op_reply(55 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a046000 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.162+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a046400 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.162+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.4 v2:172.21.15.177:6820/4180733146 4 ==== osd_op_reply(56 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a046400 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.162+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a02c800 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.162+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 17 ==== osd_op_reply(57 [pgnls start_epoch 118 out=81b] v117'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.162+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a02c400 2024-04-14T08:43:09.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.162+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 11 ==== osd_op_reply(58 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.162+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8aaa3400 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.163+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 11 ==== osd_op_reply(59 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.163+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a046800 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.163+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.6 v2:172.21.15.177:6800/2961907928 6 ==== osd_op_reply(60 [pgnls start_epoch 118 out=77b] v117'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x557a8aa32780 con 0x557a8a046800 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.163+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a047000 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.163+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.5 v2:172.21.15.177:6816/3172735715 5 ==== osd_op_reply(61 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a047000 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.163+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a047400 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.163+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.2 v2:172.21.15.172:6816/1609902500 3 ==== osd_op_reply(62 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a047400 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.164+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a046000 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.164+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.1 v2:172.21.15.172:6823/1219922421 6 ==== osd_op_reply(63 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a046000 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.164+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8aaa3400 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.164+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 12 ==== osd_op_reply(64 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.164+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a02c800 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.164+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 18 ==== osd_op_reply(65 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.164+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a02c800 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.164+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 19 ==== osd_op_reply(66 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.164+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a047000 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.165+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.5 v2:172.21.15.177:6816/3172735715 6 ==== osd_op_reply(67 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a047000 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.165+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a046800 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.165+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.6 v2:172.21.15.177:6800/2961907928 7 ==== osd_op_reply(68 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32780 con 0x557a8a046800 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.165+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a047000 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.165+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.5 v2:172.21.15.177:6816/3172735715 7 ==== osd_op_reply(69 [pgnls start_epoch 118 out=97b] v117'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a047000 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.165+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a02c400 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.166+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 12 ==== osd_op_reply(70 [pgnls start_epoch 118 out=74b] v115'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.166+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a02c400 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.166+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 13 ==== osd_op_reply(71 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.166+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a046400 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.166+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.4 v2:172.21.15.177:6820/4180733146 5 ==== osd_op_reply(72 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a046400 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.166+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a02c800 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.166+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 20 ==== osd_op_reply(73 [pgnls start_epoch 118 out=98b] v117'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.166+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8aaa3400 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.167+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 13 ==== osd_op_reply(74 [pgnls start_epoch 118 out=76b] v118'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.167+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a046400 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.167+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.4 v2:172.21.15.177:6820/4180733146 6 ==== osd_op_reply(75 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a046400 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.167+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8aaa3400 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.167+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 14 ==== osd_op_reply(76 [pgnls start_epoch 118 out=105b] v118'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.167+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a046800 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.167+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.6 v2:172.21.15.177:6800/2961907928 8 ==== osd_op_reply(77 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x557a8aa32780 con 0x557a8a046800 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.167+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a046000 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.168+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.1 v2:172.21.15.172:6823/1219922421 7 ==== osd_op_reply(78 [pgnls start_epoch 118 out=73b] v118'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a046000 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.168+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047c00 con 0x557a8a046800 2024-04-14T08:43:09.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.168+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.6 v2:172.21.15.177:6800/2961907928 9 ==== osd_op_reply(79 [pgnls start_epoch 118 out=101b] v118'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x557a8aa32780 con 0x557a8a046800 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.168+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.168+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05fc00 con 0x557a8a02c800 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.168+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.168+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.168+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05e800 con 0x557a8a02c800 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05f800 con 0x557a8a02c800 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05ec00 con 0x557a8a02c800 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-14T08:43:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05f000 con 0x557a8a02c800 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.169+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05f400 con 0x557a8a02c800 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05fc00 con 0x557a8a046000 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.1 v2:172.21.15.172:6823/1219922421 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a046000 2024-04-14T08:43:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05e800 con 0x557a8a02c800 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 27 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) ==== 164+0+584 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05f800 con 0x557a8a047400 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.2 v2:172.21.15.172:6816/1609902500 4 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a047400 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.170+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a047000 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.5 v2:172.21.15.177:6816/3172735715 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a047000 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-14T08:43:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047800 con 0x557a8a02c400 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8ab96800 con 0x557a8a02c800 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.171+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a02d400 con 0x557a8aaa3400 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a02cc00 con 0x557a8aaa3400 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) ==== 164+0+1667 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 20 datascan.scan_links: injected ino detected 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a046c00 con 0x557a8a046800 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.6 v2:172.21.15.177:6800/2961907928 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x557a8aa32780 con 0x557a8a046800 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-14T08:43:09.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 10 datascan.scan_links: processing 1 injected_inos 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.172+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a047800 con 0x557a8aaa3400 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.173+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.173+0000 7fe35b251740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.173+0000 7fe35b251740 20 datascan.scan_links: first is now 2 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.173+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05e000 con 0x557a8aaa3400 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.174+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 <== osd.0 v2:172.21.15.172:6800/3761711279 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v118'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x557a8aa32780 con 0x557a8aaa3400 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.174+0000 7fe35b251740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.174+0000 7fe35b251740 10 datascan.scan_links: updating inotable 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.174+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05e400 con 0x557a8a02c800 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.174+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 29 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.174+0000 7fe35b251740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-04-14T08:43:09.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.174+0000 7fe35b251740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.174+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x557a8a05fc00 con 0x557a8a02c800 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe3597fd640 1 -- 172.21.15.172:0/1371652405 <== osd.3 v2:172.21.15.172:6808/2305139122 30 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v118'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x557a8aa32500 con 0x557a8a02c800 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 10 datascan.scan_links: updating snaptable 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x557a8a02d000 con 0x557a8a02c400 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe3587fb640 1 -- 172.21.15.172:0/1371652405 <== osd.7 v2:172.21.15.177:6808/3609985207 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x557a8aa32000 con 0x557a8a02c400 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x557a8aaa3400 msgr2=0x557a8a034580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x557a8aaa3400 0x557a8a034580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x557a8a046000 msgr2=0x557a8a034b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x557a8a046000 0x557a8a034b00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x557a8a047400 msgr2=0x557a8a035600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x557a8a047400 0x557a8a035600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x557a8a02c800 msgr2=0x557a8a034000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x557a8a02c800 0x557a8a034000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x557a8a046400 msgr2=0x557a8a035080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x557a8a046400 0x557a8a035080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x557a8a047000 msgr2=0x557a8a036680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x557a8a047000 0x557a8a036680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x557a8a046800 msgr2=0x557a8a036100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x557a8a046800 0x557a8a036100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x557a8a02c400 msgr2=0x557a8aa86680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x557a8a02c400 0x557a8aa86680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 reap_dead start 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 reap_dead start 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe358ffc640 1 -- 172.21.15.172:0/1371652405 reap_dead start 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x557a8a02c000 msgr2=0x557a8aa86100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x557a8a02c000 0x557a8aa86100 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x557a8aad58f0 tx=0x557a89fc0a80 comp rx=0 tx=0).stop 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 msgr2=0x557a8aa85b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 0x557a8aa85b80 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x557a8ab82f30 tx=0x557a8aa4aa80 comp rx=0 tx=0).stop 2024-04-14T08:43:09.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 shutdown_connections 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x557a8a02c000 0x557a8aa86100 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 --2- 172.21.15.172:0/1371652405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a8ab97c00 0x557a8aa85b80 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 >> 172.21.15.172:0/1371652405 conn(0x557a8ab97000 msgr2=0x557a89d95000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 shutdown_connections 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.176+0000 7fe35b251740 1 -- 172.21.15.172:0/1371652405 wait complete. 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.177+0000 7fe35b251740 1 -- 172.21.15.172:0/3254383253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a89cb1c00 msgr2=0x557a8aa84b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.177+0000 7fe35b251740 1 --2- 172.21.15.172:0/3254383253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a89cb1c00 0x557a8aa84b00 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x557a8aad5a10 tx=0x557a8aa4c060 comp rx=0 tx=0).stop 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.177+0000 7fe35b251740 1 -- 172.21.15.172:0/3254383253 shutdown_connections 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.177+0000 7fe35b251740 1 --2- 172.21.15.172:0/3254383253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a89cb1c00 0x557a8aa84b00 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.177+0000 7fe35b251740 1 --2- 172.21.15.172:0/3254383253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x557a89cb1800 0x557a8aa84580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.177+0000 7fe35b251740 1 --2- 172.21.15.172:0/3254383253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a89cb0c00 0x557a8aa84000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.177+0000 7fe35b251740 1 -- 172.21.15.172:0/3254383253 >> 172.21.15.172:0/3254383253 conn(0x557a89cb1000 msgr2=0x557a89d94800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.177+0000 7fe35b251740 1 -- 172.21.15.172:0/3254383253 shutdown_connections 2024-04-14T08:43:09.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.177+0000 7fe35b251740 1 -- 172.21.15.172:0/3254383253 wait complete. 2024-04-14T08:43:09.199 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.133256, result: 2024-04-14T08:43:09.200 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-14T08:43:09.242 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-14T08:43:09.292 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-04-14T08:43:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2024-04-14T08:43:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-14T08:43:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2024-04-14T08:43:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2024-04-14T08:43:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2024-04-14T08:43:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2024-04-14T08:43:09.292 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2024-04-14T08:43:09.293 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2024-04-14T08:43:09.294 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.172,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2024-04-14T08:43:09.295 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.296 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 20 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 20 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:43:09.297 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 10 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:43:09.298 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2024-04-14T08:43:09.299 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: mon down mkfs grace = 300 2024-04-14T08:43:09.300 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2024-04-14T08:43:09.301 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = False 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = True 2024-04-14T08:43:09.302 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-14T08:43:09.303 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-14T08:43:09.303 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-14T08:43:09.303 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2024-04-14T08:43:09.303 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2024-04-14T08:43:09.303 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-14T08:43:09.303 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2024-04-14T08:43:09.303 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.303 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2024-04-14T08:43:09.303 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.304 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.305 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.172,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on skipped message = False 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.306 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug trim objects = true 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2024-04-14T08:43:09.307 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 20 2024-04-14T08:43:09.308 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 20 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 10 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-04-14T08:43:09.309 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:43:09.310 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.311 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: mon down mkfs grace = 300 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: mon op complaint time = 120 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 600 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = "" 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2024-04-14T08:43:09.312 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout:[mds] 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds = 20 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds balancer = 20 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug frag = True 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug scatterstat = False 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout: mds op complaint time = 180 2024-04-14T08:43:09.313 INFO:teuthology.orchestra.run.smithi177.stdout: mds verify scatter = True 2024-04-14T08:43:09.314 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2024-04-14T08:43:09.314 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2024-04-14T08:43:09.314 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2024-04-14T08:43:09.316 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-14T08:43:09.357 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-14T08:43:09.406 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-04-14T08:43:09.406 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2024-04-14T08:43:09.406 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-14T08:43:09.406 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-14T08:43:09.407 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2024-04-14T08:43:09.408 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.172,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2024-04-14T08:43:09.409 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2024-04-14T08:43:09.410 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:43:09.410 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.410 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2024-04-14T08:43:09.410 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2024-04-14T08:43:09.410 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.410 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2024-04-14T08:43:09.410 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2024-04-14T08:43:09.410 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2024-04-14T08:43:09.410 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.411 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 20 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 20 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2024-04-14T08:43:09.412 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 10 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:43:09.413 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:43:09.414 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2024-04-14T08:43:09.414 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-14T08:43:09.414 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.414 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2024-04-14T08:43:09.414 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:43:09.414 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-14T08:43:09.414 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2024-04-14T08:43:09.415 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: mon down mkfs grace = 300 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2024-04-14T08:43:09.416 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-04-14T08:43:09.417 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-04-14T08:43:09.418 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2024-04-14T08:43:09.418 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2024-04-14T08:43:09.418 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2024-04-14T08:43:09.418 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-14T08:43:09.418 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2024-04-14T08:43:09.418 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = True 2024-04-14T08:43:09.418 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2024-04-14T08:43:09.418 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = True 2024-04-14T08:43:09.418 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-14T08:43:09.418 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-14T08:43:09.418 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-14T08:43:09.419 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2024-04-14T08:43:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2024-04-14T08:43:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-14T08:43:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2024-04-14T08:43:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2024-04-14T08:43:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2024-04-14T08:43:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.419 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.420 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2024-04-14T08:43:09.421 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.172,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on skipped message = False 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = 8e7e3a2f-e38f-4857-b139-5df1f7a099dc 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2024-04-14T08:43:09.422 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug trim objects = true 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.423 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 20 2024-04-14T08:43:09.424 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 20 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 10 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:43:09.425 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2024-04-14T08:43:09.426 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.427 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: mon down mkfs grace = 300 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: mon op complaint time = 120 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 600 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:43:09.428 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = "" 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout:[mds] 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds = 20 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds balancer = 20 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2024-04-14T08:43:09.429 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug frag = True 2024-04-14T08:43:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug scatterstat = True 2024-04-14T08:43:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: mds op complaint time = 180 2024-04-14T08:43:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: mds verify scatter = True 2024-04-14T08:43:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2024-04-14T08:43:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2024-04-14T08:43:09.430 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2024-04-14T08:43:09.431 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-14T08:43:09.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.575+0000 7fbb89291640 1 Processor -- start 2024-04-14T08:43:09.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.575+0000 7fbb89291640 1 -- start start 2024-04-14T08:43:09.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.576+0000 7fbb89291640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbb840d09d0 0x7fbb840cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.576+0000 7fbb89291640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb840cd650 0x7fbb840b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.576+0000 7fbb89291640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb840d2df0 con 0x7fbb8415c4e0 2024-04-14T08:43:09.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.576+0000 7fbb89291640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fbb840cdb50 con 0x7fbb840d09d0 2024-04-14T08:43:09.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.576+0000 7fbb89291640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbb840cde30 con 0x7fbb840cd650 2024-04-14T08:43:09.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.577+0000 7fbb837fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb840cd650 0x7fbb840b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.577+0000 7fbb827fc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbb840d09d0 0x7fbb840cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.577+0000 7fbb827fc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbb840d09d0 0x7fbb840cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:59586/0 (socket says 172.21.15.172:59586) 2024-04-14T08:43:09.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.577+0000 7fbb82ffd640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fbb8415c4e0 0x7fbb840d02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:59890/0 (socket says 172.21.15.172:59890) 2024-04-14T08:43:09.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.577+0000 7fbb837fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb840cd650 0x7fbb840b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:53978/0 (socket says 172.21.15.172:53978) 2024-04-14T08:43:09.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.577+0000 7fbb827fc640 1 -- 172.21.15.172:0/3782801548 learned_addr learned my addr 172.21.15.172:0/3782801548 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:09.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.577+0000 7fbb827fc640 1 -- 172.21.15.172:0/3782801548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb840cd650 msgr2=0x7fbb840b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:09.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.577+0000 7fbb827fc640 1 --2- 172.21.15.172:0/3782801548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb840cd650 0x7fbb840b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.577+0000 7fbb827fc640 1 -- 172.21.15.172:0/3782801548 >> v1:172.21.15.177:6789/0 conn(0x7fbb8415c4e0 legacy=0x7fbb840d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.577+0000 7fbb827fc640 1 -- 172.21.15.172:0/3782801548 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb840b1990 con 0x7fbb840d09d0 2024-04-14T08:43:09.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.577+0000 7fbb81ffb640 1 -- 172.21.15.172:0/3782801548 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4218695615 0 0) 0x7fbb840d2df0 con 0x7fbb8415c4e0 2024-04-14T08:43:09.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.577+0000 7fbb837fe640 1 --2- 172.21.15.172:0/3782801548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb840cd650 0x7fbb840b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:43:09.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.578+0000 7fbb827fc640 1 --2- 172.21.15.172:0/3782801548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb840d09d0 0x7fbb840cfc00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009870 tx=0x7fbb7802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81de4f9a4d33a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.578+0000 7fbb81ffb640 1 -- 172.21.15.172:0/3782801548 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78034070 con 0x7fbb840d09d0 2024-04-14T08:43:09.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.578+0000 7fbb81ffb640 1 -- 172.21.15.172:0/3782801548 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb78030840 con 0x7fbb840d09d0 2024-04-14T08:43:09.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.578+0000 7fbb81ffb640 1 -- 172.21.15.172:0/3782801548 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78030c10 con 0x7fbb840d09d0 2024-04-14T08:43:09.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.578+0000 7fbb89291640 1 -- 172.21.15.172:0/3782801548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb840d09d0 msgr2=0x7fbb840cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.578+0000 7fbb89291640 1 --2- 172.21.15.172:0/3782801548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb840d09d0 0x7fbb840cfc00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009870 tx=0x7fbb7802b980 comp rx=0 tx=0).stop 2024-04-14T08:43:09.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.578+0000 7fbb89291640 1 -- 172.21.15.172:0/3782801548 shutdown_connections 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.578+0000 7fbb89291640 1 --2- 172.21.15.172:0/3782801548 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb840cd650 0x7fbb840b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.578+0000 7fbb89291640 1 --2- 172.21.15.172:0/3782801548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb840d09d0 0x7fbb840cfc00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.578+0000 7fbb89291640 1 -- 172.21.15.172:0/3782801548 >> 172.21.15.172:0/3782801548 conn(0x7fbb840d2030 msgr2=0x7fbb840d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.578+0000 7fbb89291640 1 -- 172.21.15.172:0/3782801548 shutdown_connections 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.578+0000 7fbb89291640 1 -- 172.21.15.172:0/3782801548 wait complete. 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.580+0000 7fbb89291640 1 Processor -- start 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.580+0000 7fbb89291640 1 -- start start 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.580+0000 7fbb89291640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb840cd650 0x7fbb84155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.580+0000 7fbb89291640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb840d09d0 0x7fbb841f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.580+0000 7fbb89291640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb8415c4e0 0x7fbb8415d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.580+0000 7fbb89291640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbb840b2b30 con 0x7fbb840cd650 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.580+0000 7fbb89291640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb840b29b0 con 0x7fbb840d09d0 2024-04-14T08:43:09.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.580+0000 7fbb89291640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbb840b2cb0 con 0x7fbb8415c4e0 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.580+0000 7fbb82ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb840cd650 0x7fbb84155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb827fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb840d09d0 0x7fbb841f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb82ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb840cd650 0x7fbb84155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46770/0 (socket says 172.21.15.172:46770) 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb82ffd640 1 -- 172.21.15.172:0/2549435035 learned_addr learned my addr 172.21.15.172:0/2549435035 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb837fe640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb8415c4e0 0x7fbb8415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb82ffd640 1 -- 172.21.15.172:0/2549435035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb8415c4e0 msgr2=0x7fbb8415d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb82ffd640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb8415c4e0 0x7fbb8415d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb82ffd640 1 -- 172.21.15.172:0/2549435035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb840d09d0 msgr2=0x7fbb841f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb82ffd640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb840d09d0 0x7fbb841f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb82ffd640 1 -- 172.21.15.172:0/2549435035 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8415e160 con 0x7fbb840cd650 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb827fc640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb840d09d0 0x7fbb841f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb837fe640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb8415c4e0 0x7fbb8415d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb82ffd640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb840cd650 0x7fbb84155720 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400c7f0 tx=0x7fbb7400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb6b7fe640 1 -- 172.21.15.172:0/2549435035 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74018070 con 0x7fbb840cd650 2024-04-14T08:43:09.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb6b7fe640 1 -- 172.21.15.172:0/2549435035 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb7400ce40 con 0x7fbb840cd650 2024-04-14T08:43:09.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb89291640 1 -- 172.21.15.172:0/2549435035 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8415e440 con 0x7fbb840cd650 2024-04-14T08:43:09.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb6b7fe640 1 -- 172.21.15.172:0/2549435035 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7400a880 con 0x7fbb840cd650 2024-04-14T08:43:09.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.581+0000 7fbb89291640 1 -- 172.21.15.172:0/2549435035 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb8415ea00 con 0x7fbb840cd650 2024-04-14T08:43:09.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.582+0000 7fbb89291640 1 -- 172.21.15.172:0/2549435035 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb840d2df0 con 0x7fbb840cd650 2024-04-14T08:43:09.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.583+0000 7fbb6b7fe640 1 -- 172.21.15.172:0/2549435035 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb7400aa20 con 0x7fbb840cd650 2024-04-14T08:43:09.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.583+0000 7fbb6b7fe640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb7006c8f0 0x7fbb7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:09.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.583+0000 7fbb6b7fe640 1 -- 172.21.15.172:0/2549435035 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7fbb7408b290 con 0x7fbb840cd650 2024-04-14T08:43:09.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.583+0000 7fbb827fc640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb7006c8f0 0x7fbb7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:09.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.583+0000 7fbb827fc640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb7006c8f0 0x7fbb7006edb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbb78036000 tx=0x7fbb78008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:09.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.587+0000 7fbb6b7fe640 1 -- 172.21.15.172:0/2549435035 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbb840d2df0 con 0x7fbb840cd650 2024-04-14T08:43:09.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:09.803+0000 7fbb89291640 1 -- 172.21.15.172:0/2549435035 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fbb840d1840 con 0x7fbb840cd650 2024-04-14T08:43:10.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.721+0000 7fbb6b7fe640 1 -- 172.21.15.172:0/2549435035 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v96) ==== 162+0+0 (secure 0 0 0) 0x7fbb74058db0 con 0x7fbb840cd650 2024-04-14T08:43:10.722 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-14T08:43:10.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.724+0000 7fbb89291640 1 -- 172.21.15.172:0/2549435035 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb7006c8f0 msgr2=0x7fbb7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:10.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.724+0000 7fbb89291640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb7006c8f0 0x7fbb7006edb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbb78036000 tx=0x7fbb78008030 comp rx=0 tx=0).stop 2024-04-14T08:43:10.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.724+0000 7fbb89291640 1 -- 172.21.15.172:0/2549435035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb840cd650 msgr2=0x7fbb84155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:10.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.724+0000 7fbb89291640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb840cd650 0x7fbb84155720 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400c7f0 tx=0x7fbb7400cbf0 comp rx=0 tx=0).stop 2024-04-14T08:43:10.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.724+0000 7fbb89291640 1 -- 172.21.15.172:0/2549435035 shutdown_connections 2024-04-14T08:43:10.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.724+0000 7fbb89291640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbb7006c8f0 0x7fbb7006edb0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:10.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.724+0000 7fbb89291640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb8415c4e0 0x7fbb8415d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:10.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.724+0000 7fbb89291640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb840d09d0 0x7fbb841f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:10.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.724+0000 7fbb89291640 1 --2- 172.21.15.172:0/2549435035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb840cd650 0x7fbb84155720 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:10.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.724+0000 7fbb89291640 1 -- 172.21.15.172:0/2549435035 >> 172.21.15.172:0/2549435035 conn(0x7fbb840d2030 msgr2=0x7fbb840cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:10.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.724+0000 7fbb89291640 1 -- 172.21.15.172:0/2549435035 shutdown_connections 2024-04-14T08:43:10.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.724+0000 7fbb89291640 1 -- 172.21.15.172:0/2549435035 wait complete. 2024-04-14T08:43:10.740 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:43:10.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.846+0000 7f8193cc0640 1 Processor -- start 2024-04-14T08:43:10.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.846+0000 7f8193cc0640 1 -- start start 2024-04-14T08:43:10.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.847+0000 7f8193cc0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f818c0b4040 0x7f818c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:10.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.847+0000 7f8193cc0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f818c15c770 con 0x7f818c0b1df0 2024-04-14T08:43:10.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.847+0000 7f8193cc0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f818c0b3700 con 0x7f818c0b21d0 2024-04-14T08:43:10.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.847+0000 7f8193cc0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f818c0b39e0 con 0x7f818c0b4040 2024-04-14T08:43:10.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.847+0000 7f819225d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f818c0b4040 0x7f818c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:10.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.847+0000 7f8191a5c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f818c0b1df0 0x7f818c0b2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54932/0 (socket says 172.21.15.172:54932) 2024-04-14T08:43:10.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.847+0000 7f8191a5c640 1 -- 172.21.15.172:0/1349342039 learned_addr learned my addr 172.21.15.172:0/1349342039 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:10.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.847+0000 7f819225d640 1 -- 172.21.15.172:0/1349342039 >> v1:172.21.15.177:6789/0 conn(0x7f818c0b21d0 legacy=0x7f818c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:10.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.847+0000 7f819225d640 1 -- 172.21.15.172:0/1349342039 >> v1:172.21.15.172:6789/0 conn(0x7f818c0b1df0 legacy=0x7f818c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:10.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.847+0000 7f8190a5a640 1 -- 172.21.15.172:0/1349342039 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4119054169 0 0) 0x7f818c15c770 con 0x7f818c0b1df0 2024-04-14T08:43:10.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.847+0000 7f819225d640 1 -- 172.21.15.172:0/1349342039 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c0b3cc0 con 0x7f818c0b4040 2024-04-14T08:43:10.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.848+0000 7f819225d640 1 --2- 172.21.15.172:0/1349342039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c0b4040 0x7f818c0b4460 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f8184009d90 tx=0x7f818402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e6b9d6bbcb24a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:10.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.848+0000 7f8190a5a640 1 -- 172.21.15.172:0/1349342039 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184039070 con 0x7f818c0b4040 2024-04-14T08:43:10.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.848+0000 7f8190a5a640 1 -- 172.21.15.172:0/1349342039 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818402cc60 con 0x7f818c0b4040 2024-04-14T08:43:10.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.848+0000 7f8190a5a640 1 -- 172.21.15.172:0/1349342039 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184034660 con 0x7f818c0b4040 2024-04-14T08:43:10.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.849+0000 7f8193cc0640 1 -- 172.21.15.172:0/1349342039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c0b4040 msgr2=0x7f818c0b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:10.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.849+0000 7f8193cc0640 1 --2- 172.21.15.172:0/1349342039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c0b4040 0x7f818c0b4460 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f8184009d90 tx=0x7f818402beb0 comp rx=0 tx=0).stop 2024-04-14T08:43:10.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.849+0000 7f8193cc0640 1 -- 172.21.15.172:0/1349342039 shutdown_connections 2024-04-14T08:43:10.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.849+0000 7f8193cc0640 1 --2- 172.21.15.172:0/1349342039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c0b4040 0x7f818c0b4460 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:10.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.849+0000 7f8193cc0640 1 -- 172.21.15.172:0/1349342039 >> 172.21.15.172:0/1349342039 conn(0x7f818c06ec30 msgr2=0x7f818c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:10.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.849+0000 7f8193cc0640 1 -- 172.21.15.172:0/1349342039 shutdown_connections 2024-04-14T08:43:10.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.849+0000 7f8193cc0640 1 -- 172.21.15.172:0/1349342039 wait complete. 2024-04-14T08:43:10.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.850+0000 7f8193cc0640 1 Processor -- start 2024-04-14T08:43:10.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.850+0000 7f8193cc0640 1 -- start start 2024-04-14T08:43:10.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f8193cc0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f818c0b1df0 0x7f818c151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:10.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f8193cc0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c0b21d0 0x7f818c130240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f8193cc0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f818c0b4040 0x7f818c133b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f8193cc0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f818c15b7f0 con 0x7f818c0b21d0 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f8193cc0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f818c0b49a0 con 0x7f818c0b1df0 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f8193cc0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f818c1552d0 con 0x7f818c0b4040 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f819125b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c0b21d0 0x7f818c130240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f819125b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c0b21d0 0x7f818c130240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46796/0 (socket says 172.21.15.172:46796) 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f819125b640 1 -- 172.21.15.172:0/3971419671 learned_addr learned my addr 172.21.15.172:0/3971419671 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f8191a5c640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f818c0b1df0 0x7f818c151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f819225d640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f818c0b4040 0x7f818c133b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f819125b640 1 -- 172.21.15.172:0/3971419671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f818c0b4040 msgr2=0x7f818c133b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f819125b640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f818c0b4040 0x7f818c133b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f819125b640 1 -- 172.21.15.172:0/3971419671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f818c0b1df0 msgr2=0x7f818c151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f819125b640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f818c0b1df0 0x7f818c151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f819125b640 1 -- 172.21.15.172:0/3971419671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c1eff10 con 0x7f818c0b21d0 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f819225d640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f818c0b4040 0x7f818c133b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:10.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f819125b640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c0b21d0 0x7f818c130240 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f817c00c400 tx=0x7f817c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:10.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f81827fc640 1 -- 172.21.15.172:0/3971419671 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c011790 con 0x7f818c0b21d0 2024-04-14T08:43:10.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f81827fc640 1 -- 172.21.15.172:0/3971419671 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f817c017070 con 0x7f818c0b21d0 2024-04-14T08:43:10.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f81827fc640 1 -- 172.21.15.172:0/3971419671 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c011bb0 con 0x7f818c0b21d0 2024-04-14T08:43:10.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.851+0000 7f8193cc0640 1 -- 172.21.15.172:0/3971419671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818c1f01f0 con 0x7f818c0b21d0 2024-04-14T08:43:10.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.852+0000 7f8193cc0640 1 -- 172.21.15.172:0/3971419671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818c1f07b0 con 0x7f818c0b21d0 2024-04-14T08:43:10.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.853+0000 7f8193cc0640 1 -- 172.21.15.172:0/3971419671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f818c068520 con 0x7f818c0b21d0 2024-04-14T08:43:10.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.853+0000 7f81827fc640 1 -- 172.21.15.172:0/3971419671 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f817c01c460 con 0x7f818c0b21d0 2024-04-14T08:43:10.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.853+0000 7f81827fc640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f817006c8f0 0x7f817006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:10.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.854+0000 7f8191a5c640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f817006c8f0 0x7f817006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:10.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.854+0000 7f81827fc640 1 -- 172.21.15.172:0/3971419671 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7f817c08bc30 con 0x7f818c0b21d0 2024-04-14T08:43:10.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.854+0000 7f8191a5c640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f817006c8f0 0x7f817006edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8188000f60 tx=0x7f8188007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:10.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:10.858+0000 7f81827fc640 1 -- 172.21.15.172:0/3971419671 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f817c059e20 con 0x7f818c0b21d0 2024-04-14T08:43:11.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.068+0000 7f8193cc0640 1 -- 172.21.15.172:0/3971419671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f818c1f0980 con 0x7f818c0b21d0 2024-04-14T08:43:11.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.068+0000 7f81827fc640 1 -- 172.21.15.172:0/3971419671 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5190 (secure 0 0 0) 0x7f817c059720 con 0x7f818c0b21d0 2024-04-14T08:43:11.069 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:11.069 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":97,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3881291102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3881291102},{"type":"v1","addr":"172.21.15.177:6835","nonce":3881291102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:43:10.724533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9519":{"gid":9519,"name":"c","rank":0,"incarnation":97,"state":"up:replay","state_seq":1,"addr":"172.21.15.172:6837/4065123823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4065123823},{"type":"v1","addr":"172.21.15.172:6837","nonce":4065123823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-14T08:43:11.069 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 97 2024-04-14T08:43:11.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.070+0000 7f8193cc0640 1 -- 172.21.15.172:0/3971419671 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f817006c8f0 msgr2=0x7f817006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:11.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.070+0000 7f8193cc0640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f817006c8f0 0x7f817006edb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8188000f60 tx=0x7f8188007b50 comp rx=0 tx=0).stop 2024-04-14T08:43:11.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.070+0000 7f8193cc0640 1 -- 172.21.15.172:0/3971419671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c0b21d0 msgr2=0x7f818c130240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:11.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.070+0000 7f8193cc0640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c0b21d0 0x7f818c130240 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f817c00c400 tx=0x7f817c00c8c0 comp rx=0 tx=0).stop 2024-04-14T08:43:11.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.071+0000 7f8193cc0640 1 -- 172.21.15.172:0/3971419671 shutdown_connections 2024-04-14T08:43:11.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.071+0000 7f8193cc0640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f817006c8f0 0x7f817006edb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:11.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.071+0000 7f8193cc0640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f818c0b4040 0x7f818c133b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:11.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.071+0000 7f8193cc0640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f818c0b21d0 0x7f818c130240 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:11.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.071+0000 7f8193cc0640 1 --2- 172.21.15.172:0/3971419671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f818c0b1df0 0x7f818c151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:11.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.071+0000 7f8193cc0640 1 -- 172.21.15.172:0/3971419671 >> 172.21.15.172:0/3971419671 conn(0x7f818c06ec30 msgr2=0x7f818c159c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:11.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.071+0000 7f8193cc0640 1 -- 172.21.15.172:0/3971419671 shutdown_connections 2024-04-14T08:43:11.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:11.071+0000 7f8193cc0640 1 -- 172.21.15.172:0/3971419671 wait complete. 2024-04-14T08:43:11.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 97, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:42:40.251538+0000', 'modified': '2024-04-14T08:43:10.724533+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9519}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9519': {'gid': 9519, 'name': 'c', 'rank': 0, 'incarnation': 97, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.172:6837/4065123823', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 4065123823}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 4065123823}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-14T08:43:11.485 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9519:up:replay 2024-04-14T08:43:12.486 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:43:12.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.598+0000 7ff5502c8640 1 Processor -- start 2024-04-14T08:43:12.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.598+0000 7ff5502c8640 1 -- start start 2024-04-14T08:43:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.599+0000 7ff5502c8640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff54815e5b0 0x7ff5480ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.599+0000 7ff5502c8640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff5480729b0 0x7ff54806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.599+0000 7ff5502c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5480b41d0 con 0x7ff54806e160 2024-04-14T08:43:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.599+0000 7ff5502c8640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff5480aebe0 con 0x7ff5480729b0 2024-04-14T08:43:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.599+0000 7ff5502c8640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7ff5480b4470 con 0x7ff54815e5b0 2024-04-14T08:43:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.599+0000 7ff54d863640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff5480729b0 0x7ff54806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.599+0000 7ff54d863640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff5480729b0 0x7ff54806dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46798/0 (socket says 172.21.15.172:46798) 2024-04-14T08:43:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.599+0000 7ff54d863640 1 -- 172.21.15.172:0/164524997 learned_addr learned my addr 172.21.15.172:0/164524997 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:12.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.599+0000 7ff54e064640 1 --2- 172.21.15.172:0/164524997 >> v2:172.21.15.177:3300/0 conn(0x7ff54815e5b0 0x7ff5480ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.600+0000 7ff54d863640 1 -- 172.21.15.172:0/164524997 >> v2:172.21.15.177:3300/0 conn(0x7ff54815e5b0 msgr2=0x7ff5480ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.600+0000 7ff54d863640 1 --2- 172.21.15.172:0/164524997 >> v2:172.21.15.177:3300/0 conn(0x7ff54815e5b0 0x7ff5480ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.600+0000 7ff54d863640 1 -- 172.21.15.172:0/164524997 >> v1:172.21.15.177:6789/0 conn(0x7ff54806e160 legacy=0x7ff54806c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.600+0000 7ff54d863640 1 -- 172.21.15.172:0/164524997 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5480ae8d0 con 0x7ff5480729b0 2024-04-14T08:43:12.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.600+0000 7ff54e064640 1 --2- 172.21.15.172:0/164524997 >> v2:172.21.15.177:3300/0 conn(0x7ff54815e5b0 0x7ff5480ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:12.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.600+0000 7ff54d863640 1 --2- 172.21.15.172:0/164524997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5480729b0 0x7ff54806dc20 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ff538009870 tx=0x7ff53802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea149c2636047fb server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.601+0000 7ff54d062640 1 -- 172.21.15.172:0/164524997 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538034070 con 0x7ff5480729b0 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.601+0000 7ff54d062640 1 -- 172.21.15.172:0/164524997 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff538030860 con 0x7ff5480729b0 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.601+0000 7ff54d062640 1 -- 172.21.15.172:0/164524997 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538030c30 con 0x7ff5480729b0 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.601+0000 7ff5502c8640 1 -- 172.21.15.172:0/164524997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5480729b0 msgr2=0x7ff54806dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.601+0000 7ff5502c8640 1 --2- 172.21.15.172:0/164524997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5480729b0 0x7ff54806dc20 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ff538009870 tx=0x7ff53802b9a0 comp rx=0 tx=0).stop 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.601+0000 7ff5502c8640 1 -- 172.21.15.172:0/164524997 shutdown_connections 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.601+0000 7ff5502c8640 1 --2- 172.21.15.172:0/164524997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5480729b0 0x7ff54806dc20 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.601+0000 7ff5502c8640 1 --2- 172.21.15.172:0/164524997 >> v2:172.21.15.177:3300/0 conn(0x7ff54815e5b0 0x7ff5480ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.601+0000 7ff5502c8640 1 -- 172.21.15.172:0/164524997 >> 172.21.15.172:0/164524997 conn(0x7ff548071ba0 msgr2=0x7ff548071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.601+0000 7ff5502c8640 1 -- 172.21.15.172:0/164524997 shutdown_connections 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.601+0000 7ff5502c8640 1 -- 172.21.15.172:0/164524997 wait complete. 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.602+0000 7ff5502c8640 1 Processor -- start 2024-04-14T08:43:12.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.603+0000 7ff5502c8640 1 -- start start 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.603+0000 7ff5502c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54806e160 0x7ff548149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.603+0000 7ff5502c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5480729b0 0x7ff5480cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.603+0000 7ff5502c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54815e5b0 0x7ff5480cf580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.603+0000 7ff5502c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff54806cdc0 con 0x7ff54806e160 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.603+0000 7ff5502c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff54806cc40 con 0x7ff5480729b0 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.603+0000 7ff5502c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff5480b4c70 con 0x7ff54815e5b0 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.603+0000 7ff54e064640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54806e160 0x7ff548149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.603+0000 7ff54e064640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54806e160 0x7ff548149260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46810/0 (socket says 172.21.15.172:46810) 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.603+0000 7ff54e064640 1 -- 172.21.15.172:0/1742186172 learned_addr learned my addr 172.21.15.172:0/1742186172 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff54d863640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5480729b0 0x7ff5480cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff54e865640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54815e5b0 0x7ff5480cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff54e064640 1 -- 172.21.15.172:0/1742186172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54815e5b0 msgr2=0x7ff5480cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff54e064640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54815e5b0 0x7ff5480cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:12.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff54e064640 1 -- 172.21.15.172:0/1742186172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5480729b0 msgr2=0x7ff5480cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff54e064640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5480729b0 0x7ff5480cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff54e064640 1 -- 172.21.15.172:0/1742186172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5481497a0 con 0x7ff54806e160 2024-04-14T08:43:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff54e064640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54806e160 0x7ff548149260 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ff544007ef0 tx=0x7ff54400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff53effd640 1 -- 172.21.15.172:0/1742186172 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54401a070 con 0x7ff54806e160 2024-04-14T08:43:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff53effd640 1 -- 172.21.15.172:0/1742186172 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff544003b90 con 0x7ff54806e160 2024-04-14T08:43:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff5502c8640 1 -- 172.21.15.172:0/1742186172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5480cffd0 con 0x7ff54806e160 2024-04-14T08:43:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff53effd640 1 -- 172.21.15.172:0/1742186172 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54400de20 con 0x7ff54806e160 2024-04-14T08:43:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.604+0000 7ff5502c8640 1 -- 172.21.15.172:0/1742186172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff548160490 con 0x7ff54806e160 2024-04-14T08:43:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.605+0000 7ff5502c8640 1 -- 172.21.15.172:0/1742186172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff548068520 con 0x7ff54806e160 2024-04-14T08:43:12.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.606+0000 7ff53effd640 1 -- 172.21.15.172:0/1742186172 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff54401d070 con 0x7ff54806e160 2024-04-14T08:43:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.606+0000 7ff53effd640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff52c06c8f0 0x7ff52c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.606+0000 7ff53effd640 1 -- 172.21.15.172:0/1742186172 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7ff54408be90 con 0x7ff54806e160 2024-04-14T08:43:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.606+0000 7ff54d863640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff52c06c8f0 0x7ff52c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:12.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.607+0000 7ff54d863640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff52c06c8f0 0x7ff52c06edb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff538036000 tx=0x7ff538008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:12.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.610+0000 7ff53effd640 1 -- 172.21.15.172:0/1742186172 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff5481497a0 con 0x7ff54806e160 2024-04-14T08:43:12.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.824+0000 7ff5502c8640 1 -- 172.21.15.172:0/1742186172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff548072790 con 0x7ff54806e160 2024-04-14T08:43:12.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.824+0000 7ff53effd640 1 -- 172.21.15.172:0/1742186172 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5190 (secure 0 0 0) 0x7ff544010070 con 0x7ff54806e160 2024-04-14T08:43:12.826 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:12.826 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3881291102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3881291102},{"type":"v1","addr":"172.21.15.177:6835","nonce":3881291102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:43:11.739153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9519":{"gid":9519,"name":"c","rank":0,"incarnation":97,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.172:6837/4065123823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4065123823},{"type":"v1","addr":"172.21.15.172:6837","nonce":4065123823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-14T08:43:12.827 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 99 2024-04-14T08:43:12.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.826+0000 7ff5502c8640 1 -- 172.21.15.172:0/1742186172 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff52c06c8f0 msgr2=0x7ff52c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:12.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.826+0000 7ff5502c8640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff52c06c8f0 0x7ff52c06edb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff538036000 tx=0x7ff538008030 comp rx=0 tx=0).stop 2024-04-14T08:43:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.826+0000 7ff5502c8640 1 -- 172.21.15.172:0/1742186172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54806e160 msgr2=0x7ff548149260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.826+0000 7ff5502c8640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54806e160 0x7ff548149260 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ff544007ef0 tx=0x7ff54400d4a0 comp rx=0 tx=0).stop 2024-04-14T08:43:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.826+0000 7ff5502c8640 1 -- 172.21.15.172:0/1742186172 shutdown_connections 2024-04-14T08:43:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.826+0000 7ff5502c8640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff52c06c8f0 0x7ff52c06edb0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.826+0000 7ff5502c8640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54815e5b0 0x7ff5480cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.826+0000 7ff5502c8640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5480729b0 0x7ff5480cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.826+0000 7ff5502c8640 1 --2- 172.21.15.172:0/1742186172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54806e160 0x7ff548149260 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.826+0000 7ff5502c8640 1 -- 172.21.15.172:0/1742186172 >> 172.21.15.172:0/1742186172 conn(0x7ff548071ba0 msgr2=0x7ff54806ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.826+0000 7ff5502c8640 1 -- 172.21.15.172:0/1742186172 shutdown_connections 2024-04-14T08:43:12.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:12.826+0000 7ff5502c8640 1 -- 172.21.15.172:0/1742186172 wait complete. 2024-04-14T08:43:13.240 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:42:40.251538+0000', 'modified': '2024-04-14T08:43:11.739153+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9519}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9519': {'gid': 9519, 'name': 'c', 'rank': 0, 'incarnation': 97, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.172:6837/4065123823', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 4065123823}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 4065123823}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-14T08:43:13.241 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9519:up:rejoin 2024-04-14T08:43:14.242 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:43:14.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.344+0000 7fc20058c640 1 Processor -- start 2024-04-14T08:43:14.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.344+0000 7fc20058c640 1 -- start start 2024-04-14T08:43:14.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc20058c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1f815c710 0x7fc1f80d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc20058c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1f80cde10 0x7fc1f80ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc20058c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc1f80ced80 0x7fc1f80ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc20058c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc1f80aed10 con 0x7fc1f815c710 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc20058c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc1f80af890 con 0x7fc1f80ced80 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc20058c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f80aef60 con 0x7fc1f80cde10 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc1fe328640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1f815c710 0x7fc1f80d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc1fe328640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc1f815c710 0x7fc1f80d0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46822/0 (socket says 172.21.15.172:46822) 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc1fe328640 1 -- 172.21.15.172:0/581560940 learned_addr learned my addr 172.21.15.172:0/581560940 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc1fe328640 1 -- 172.21.15.172:0/581560940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1f80cde10 msgr2=0x7fc1f80ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc1fe328640 1 --2- 172.21.15.172:0/581560940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1f80cde10 0x7fc1f80ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc1fe328640 1 -- 172.21.15.172:0/581560940 >> v2:172.21.15.177:3300/0 conn(0x7fc1f80ced80 msgr2=0x7fc1f80ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc1fe328640 1 --2- 172.21.15.172:0/581560940 >> v2:172.21.15.177:3300/0 conn(0x7fc1f80ced80 0x7fc1f80ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:14.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.346+0000 7fc1fe328640 1 -- 172.21.15.172:0/581560940 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f80af240 con 0x7fc1f815c710 2024-04-14T08:43:14.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.347+0000 7fc1fe328640 1 --2- 172.21.15.172:0/581560940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f815c710 0x7fc1f80d0950 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80d3140 tx=0x7fc1f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b1aff80a69197d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:14.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.347+0000 7fc1fd326640 1 -- 172.21.15.172:0/581560940 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4035020 con 0x7fc1f815c710 2024-04-14T08:43:14.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.347+0000 7fc1fd326640 1 -- 172.21.15.172:0/581560940 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f4031870 con 0x7fc1f815c710 2024-04-14T08:43:14.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.347+0000 7fc1fd326640 1 -- 172.21.15.172:0/581560940 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4031c40 con 0x7fc1f815c710 2024-04-14T08:43:14.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.348+0000 7fc20058c640 1 -- 172.21.15.172:0/581560940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f815c710 msgr2=0x7fc1f80d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:14.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.348+0000 7fc20058c640 1 --2- 172.21.15.172:0/581560940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f815c710 0x7fc1f80d0950 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80d3140 tx=0x7fc1f402b910 comp rx=0 tx=0).stop 2024-04-14T08:43:14.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.348+0000 7fc20058c640 1 -- 172.21.15.172:0/581560940 shutdown_connections 2024-04-14T08:43:14.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.348+0000 7fc20058c640 1 --2- 172.21.15.172:0/581560940 >> v2:172.21.15.177:3300/0 conn(0x7fc1f80ced80 0x7fc1f80ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:14.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.348+0000 7fc20058c640 1 --2- 172.21.15.172:0/581560940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1f80cde10 0x7fc1f80ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:14.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.348+0000 7fc20058c640 1 --2- 172.21.15.172:0/581560940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f815c710 0x7fc1f80d0950 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:14.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.348+0000 7fc20058c640 1 -- 172.21.15.172:0/581560940 >> 172.21.15.172:0/581560940 conn(0x7fc1f808c000 msgr2=0x7fc1f80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:14.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.348+0000 7fc20058c640 1 -- 172.21.15.172:0/581560940 shutdown_connections 2024-04-14T08:43:14.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.348+0000 7fc20058c640 1 -- 172.21.15.172:0/581560940 wait complete. 2024-04-14T08:43:14.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.349+0000 7fc20058c640 1 Processor -- start 2024-04-14T08:43:14.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.349+0000 7fc20058c640 1 -- start start 2024-04-14T08:43:14.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc20058c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1f80cde10 0x7fc1f80b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:14.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc20058c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1f80ced80 0x7fc1f815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:14.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc20058c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f815c710 0x7fc1f80b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:14.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc20058c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f80d0e90 con 0x7fc1f815c710 2024-04-14T08:43:14.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc20058c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f815caf0 con 0x7fc1f80ced80 2024-04-14T08:43:14.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc20058c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f80d1010 con 0x7fc1f80cde10 2024-04-14T08:43:14.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc1feb29640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f815c710 0x7fc1f80b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:14.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc1feb29640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f815c710 0x7fc1f80b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46832/0 (socket says 172.21.15.172:46832) 2024-04-14T08:43:14.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc1feb29640 1 -- 172.21.15.172:0/1335471128 learned_addr learned my addr 172.21.15.172:0/1335471128 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc1feb29640 1 -- 172.21.15.172:0/1335471128 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1f80cde10 msgr2=0x7fc1f80b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc1fe328640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1f80cde10 0x7fc1f80b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc1feb29640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1f80cde10 0x7fc1f80b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc1feb29640 1 -- 172.21.15.172:0/1335471128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1f80ced80 msgr2=0x7fc1f815cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc1feb29640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1f80ced80 0x7fc1f815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.350+0000 7fc1feb29640 1 -- 172.21.15.172:0/1335471128 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f80b6560 con 0x7fc1f815c710 2024-04-14T08:43:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.351+0000 7fc1feb29640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f815c710 0x7fc1f80b7600 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000b970 tx=0x7fc1f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.351+0000 7fc1eeffd640 1 -- 172.21.15.172:0/1335471128 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0019070 con 0x7fc1f815c710 2024-04-14T08:43:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.351+0000 7fc1eeffd640 1 -- 172.21.15.172:0/1335471128 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f000cb60 con 0x7fc1f815c710 2024-04-14T08:43:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.351+0000 7fc20058c640 1 -- 172.21.15.172:0/1335471128 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f80b6840 con 0x7fc1f815c710 2024-04-14T08:43:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.351+0000 7fc1eeffd640 1 -- 172.21.15.172:0/1335471128 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0014440 con 0x7fc1f815c710 2024-04-14T08:43:14.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.351+0000 7fc20058c640 1 -- 172.21.15.172:0/1335471128 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f80b6e00 con 0x7fc1f815c710 2024-04-14T08:43:14.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.352+0000 7fc20058c640 1 -- 172.21.15.172:0/1335471128 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f8075490 con 0x7fc1f815c710 2024-04-14T08:43:14.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.353+0000 7fc1eeffd640 1 -- 172.21.15.172:0/1335471128 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1f001c070 con 0x7fc1f815c710 2024-04-14T08:43:14.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.353+0000 7fc1eeffd640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1dc06c8f0 0x7fc1dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:14.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.353+0000 7fc1eeffd640 1 -- 172.21.15.172:0/1335471128 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7fc1f008ba70 con 0x7fc1f815c710 2024-04-14T08:43:14.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.353+0000 7fc1fe328640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1dc06c8f0 0x7fc1dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:14.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.354+0000 7fc1fe328640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1dc06c8f0 0x7fc1dc06edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4036000 tx=0x7fc1f403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:14.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.357+0000 7fc1eeffd640 1 -- 172.21.15.172:0/1335471128 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc1f0059cf0 con 0x7fc1f815c710 2024-04-14T08:43:14.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.575+0000 7fc20058c640 1 -- 172.21.15.172:0/1335471128 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1f8070e60 con 0x7fc1f815c710 2024-04-14T08:43:14.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.576+0000 7fc1eeffd640 1 -- 172.21.15.172:0/1335471128 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5199 (secure 0 0 0) 0x7fc1f00595f0 con 0x7fc1f815c710 2024-04-14T08:43:14.576 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:14.576 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3881291102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3881291102},{"type":"v1","addr":"172.21.15.177:6835","nonce":3881291102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:43:13.745318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9519":{"gid":9519,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.172:6837/4065123823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4065123823},{"type":"v1","addr":"172.21.15.172:6837","nonce":4065123823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9519,"qdb_cluster":[9519]},"id":8}]} 2024-04-14T08:43:14.577 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 100 2024-04-14T08:43:14.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.578+0000 7fc20058c640 1 -- 172.21.15.172:0/1335471128 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1dc06c8f0 msgr2=0x7fc1dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:14.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.578+0000 7fc20058c640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1dc06c8f0 0x7fc1dc06edb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4036000 tx=0x7fc1f403c040 comp rx=0 tx=0).stop 2024-04-14T08:43:14.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.578+0000 7fc20058c640 1 -- 172.21.15.172:0/1335471128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f815c710 msgr2=0x7fc1f80b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:14.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.578+0000 7fc20058c640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f815c710 0x7fc1f80b7600 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000b970 tx=0x7fc1f000be30 comp rx=0 tx=0).stop 2024-04-14T08:43:14.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.578+0000 7fc20058c640 1 -- 172.21.15.172:0/1335471128 shutdown_connections 2024-04-14T08:43:14.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.578+0000 7fc20058c640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc1dc06c8f0 0x7fc1dc06edb0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:14.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.578+0000 7fc20058c640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f815c710 0x7fc1f80b7600 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:14.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.578+0000 7fc20058c640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1f80ced80 0x7fc1f815cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:14.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.578+0000 7fc20058c640 1 --2- 172.21.15.172:0/1335471128 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1f80cde10 0x7fc1f80b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:14.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.578+0000 7fc20058c640 1 -- 172.21.15.172:0/1335471128 >> 172.21.15.172:0/1335471128 conn(0x7fc1f808c000 msgr2=0x7fc1f806f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:14.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.578+0000 7fc20058c640 1 -- 172.21.15.172:0/1335471128 shutdown_connections 2024-04-14T08:43:14.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:14.578+0000 7fc20058c640 1 -- 172.21.15.172:0/1335471128 wait complete. 2024-04-14T08:43:14.719 INFO:tasks.ceph.mds.c.smithi172.stderr:2024-04-14T08:43:14.719+0000 7f29e59f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:43:14.995 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-14T08:42:40.251538+0000', 'modified': '2024-04-14T08:43:13.745318+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9519}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9519': {'gid': 9519, 'name': 'c', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.172:6837/4065123823', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6835', 'nonce': 4065123823}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 4065123823}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9519, 'qdb_cluster': [9519]} 2024-04-14T08:43:14.996 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-14T08:43:14.996 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-14T08:43:15.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.098+0000 7fc03c220640 1 Processor -- start 2024-04-14T08:43:15.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.098+0000 7fc03c220640 1 -- start start 2024-04-14T08:43:15.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.099+0000 7fc03c220640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc034162ba0 0x7fc0340d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.099+0000 7fc03c220640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0340cefb0 con 0x7fc03415e430 2024-04-14T08:43:15.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.099+0000 7fc03c220640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0340cf290 con 0x7fc034162710 2024-04-14T08:43:15.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.099+0000 7fc03c220640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc0340cf570 con 0x7fc034162ba0 2024-04-14T08:43:15.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.099+0000 7fc03a7bd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc034162ba0 0x7fc0340d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.099+0000 7fc039fbc640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc03415e430 0x7fc034162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54934/0 (socket says 172.21.15.172:54934) 2024-04-14T08:43:15.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.099+0000 7fc03a7bd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc034162ba0 0x7fc0340d2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46844/0 (socket says 172.21.15.172:46844) 2024-04-14T08:43:15.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.099+0000 7fc039fbc640 1 -- 172.21.15.172:0/1609674164 learned_addr learned my addr 172.21.15.172:0/1609674164 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:15.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.100+0000 7fc03a7bd640 1 -- 172.21.15.172:0/1609674164 >> v1:172.21.15.177:6789/0 conn(0x7fc034162710 legacy=0x7fc0340d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:15.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.100+0000 7fc038fba640 1 -- 172.21.15.172:0/1609674164 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2946320242 0 0) 0x7fc0340cefb0 con 0x7fc03415e430 2024-04-14T08:43:15.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.100+0000 7fc03a7bd640 1 -- 172.21.15.172:0/1609674164 >> v1:172.21.15.172:6789/0 conn(0x7fc03415e430 legacy=0x7fc034162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.100+0000 7fc03a7bd640 1 -- 172.21.15.172:0/1609674164 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0340cf850 con 0x7fc034162ba0 2024-04-14T08:43:15.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.100+0000 7fc03a7bd640 1 --2- 172.21.15.172:0/1609674164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc034162ba0 0x7fc0340d2500 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009d90 tx=0x7fc02c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8a260d6f6aa9a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:15.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.101+0000 7fc038fba640 1 -- 172.21.15.172:0/1609674164 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c039070 con 0x7fc034162ba0 2024-04-14T08:43:15.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.101+0000 7fc038fba640 1 -- 172.21.15.172:0/1609674164 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c02cc60 con 0x7fc034162ba0 2024-04-14T08:43:15.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.101+0000 7fc038fba640 1 -- 172.21.15.172:0/1609674164 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c034660 con 0x7fc034162ba0 2024-04-14T08:43:15.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.101+0000 7fc03c220640 1 -- 172.21.15.172:0/1609674164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc034162ba0 msgr2=0x7fc0340d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.101+0000 7fc03c220640 1 --2- 172.21.15.172:0/1609674164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc034162ba0 0x7fc0340d2500 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009d90 tx=0x7fc02c02beb0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.101+0000 7fc03c220640 1 -- 172.21.15.172:0/1609674164 shutdown_connections 2024-04-14T08:43:15.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.101+0000 7fc03c220640 1 --2- 172.21.15.172:0/1609674164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc034162ba0 0x7fc0340d2500 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.101+0000 7fc03c220640 1 -- 172.21.15.172:0/1609674164 >> 172.21.15.172:0/1609674164 conn(0x7fc03409d560 msgr2=0x7fc0340d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:15.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.101+0000 7fc03c220640 1 -- 172.21.15.172:0/1609674164 shutdown_connections 2024-04-14T08:43:15.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.101+0000 7fc03c220640 1 -- 172.21.15.172:0/1609674164 wait complete. 2024-04-14T08:43:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.102+0000 7fc03c220640 1 Processor -- start 2024-04-14T08:43:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.102+0000 7fc03c220640 1 -- start start 2024-04-14T08:43:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc03c220640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03415e430 0x7fc03414f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc03c220640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc034162710 0x7fc034161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc03c220640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc034162ba0 0x7fc0340b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc03c220640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0340d4e20 con 0x7fc03415e430 2024-04-14T08:43:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc03c220640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc03415e810 con 0x7fc034162710 2024-04-14T08:43:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc03c220640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc0340d0050 con 0x7fc034162ba0 2024-04-14T08:43:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc039fbc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03415e430 0x7fc03414f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc039fbc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03415e430 0x7fc03414f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46860/0 (socket says 172.21.15.172:46860) 2024-04-14T08:43:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc039fbc640 1 -- 172.21.15.172:0/3776677173 learned_addr learned my addr 172.21.15.172:0/3776677173 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:15.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc0397bb640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc034162710 0x7fc034161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc039fbc640 1 -- 172.21.15.172:0/3776677173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc034162ba0 msgr2=0x7fc0340b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc039fbc640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc034162ba0 0x7fc0340b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc039fbc640 1 -- 172.21.15.172:0/3776677173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc034162710 msgr2=0x7fc034161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc039fbc640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc034162710 0x7fc034161180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc039fbc640 1 -- 172.21.15.172:0/3776677173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc034159ae0 con 0x7fc03415e430 2024-04-14T08:43:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.103+0000 7fc0397bb640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc034162710 0x7fc034161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.104+0000 7fc039fbc640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03415e430 0x7fc03414f5e0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fc02400c430 tx=0x7fc02400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.104+0000 7fc02a7fc640 1 -- 172.21.15.172:0/3776677173 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0240074e0 con 0x7fc03415e430 2024-04-14T08:43:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.104+0000 7fc02a7fc640 1 -- 172.21.15.172:0/3776677173 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0240047e0 con 0x7fc03415e430 2024-04-14T08:43:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.104+0000 7fc03c220640 1 -- 172.21.15.172:0/3776677173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0340cb6d0 con 0x7fc03415e430 2024-04-14T08:43:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.104+0000 7fc02a7fc640 1 -- 172.21.15.172:0/3776677173 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024004bd0 con 0x7fc03415e430 2024-04-14T08:43:15.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.104+0000 7fc03c220640 1 -- 172.21.15.172:0/3776677173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0340cbf70 con 0x7fc03415e430 2024-04-14T08:43:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.105+0000 7fc02a7fc640 1 -- 172.21.15.172:0/3776677173 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc024011070 con 0x7fc03415e430 2024-04-14T08:43:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.106+0000 7fc02a7fc640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc01806c8f0 0x7fc01806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.106+0000 7fc02a7fc640 1 -- 172.21.15.172:0/3776677173 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7fc02408b9a0 con 0x7fc03415e430 2024-04-14T08:43:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.106+0000 7fc0397bb640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc01806c8f0 0x7fc01806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.106+0000 7fc0397bb640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc01806c8f0 0x7fc01806edb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc0340b2300 tx=0x7fc030008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:15.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.108+0000 7fc03f00a740 1 Processor -- start 2024-04-14T08:43:15.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.108+0000 7fc03f00a740 1 -- start start 2024-04-14T08:43:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03f00a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f41b60 0x561678f14b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03f00a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 0x561678f448f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03f00a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f15040 0x561678f44e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03f00a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561678e7e030 con 0x561678f15040 2024-04-14T08:43:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03f00a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561678e26620 con 0x561678f41f40 2024-04-14T08:43:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03f00a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x561678e79fe0 con 0x561678f41b60 2024-04-14T08:43:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03a7bd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f15040 0x561678f44e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03a7bd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f15040 0x561678f44e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46864/0 (socket says 172.21.15.172:46864) 2024-04-14T08:43:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03a7bd640 1 -- 172.21.15.172:0/1671387897 learned_addr learned my addr 172.21.15.172:0/1671387897 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc039fbc640 1 --2- 172.21.15.172:0/1671387897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f41b60 0x561678f14b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc0397bb640 1 --2- 172.21.15.172:0/1671387897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 0x561678f448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03a7bd640 1 -- 172.21.15.172:0/1671387897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f41b60 msgr2=0x561678f14b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03a7bd640 1 --2- 172.21.15.172:0/1671387897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f41b60 0x561678f14b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03a7bd640 1 -- 172.21.15.172:0/1671387897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 msgr2=0x561678f448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03a7bd640 1 --2- 172.21.15.172:0/1671387897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 0x561678f448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.109+0000 7fc03a7bd640 1 -- 172.21.15.172:0/1671387897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561678b922d0 con 0x561678f15040 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.110+0000 7fc03a7bd640 1 --2- 172.21.15.172:0/1671387897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f15040 0x561678f44e30 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009eb0 tx=0x7fc02c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd31437be5c9cbb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.110+0000 7fc0177fe640 1 -- 172.21.15.172:0/1671387897 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c039040 con 0x561678f15040 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.110+0000 7fc0177fe640 1 -- 172.21.15.172:0/1671387897 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c045070 con 0x561678f15040 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.110+0000 7fc0177fe640 1 -- 172.21.15.172:0/1671387897 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c03f470 con 0x561678f15040 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.111+0000 7fc03f00a740 1 -- 172.21.15.172:0/1671387897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f15040 msgr2=0x561678f44e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.111+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1671387897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f15040 0x561678f44e30 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009eb0 tx=0x7fc02c02e070 comp rx=0 tx=0).stop 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.111+0000 7fc03f00a740 1 -- 172.21.15.172:0/1671387897 shutdown_connections 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.111+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1671387897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f15040 0x561678f44e30 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.111+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1671387897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 0x561678f448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.111+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1671387897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f41b60 0x561678f14b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.111+0000 7fc03f00a740 1 -- 172.21.15.172:0/1671387897 >> 172.21.15.172:0/1671387897 conn(0x561678f16d70 msgr2=0x561678f5f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:15.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.111+0000 7fc03f00a740 1 -- 172.21.15.172:0/1671387897 shutdown_connections 2024-04-14T08:43:15.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.111+0000 7fc03f00a740 1 -- 172.21.15.172:0/1671387897 wait complete. 2024-04-14T08:43:15.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.111+0000 7fc03f00a740 1 -- start start 2024-04-14T08:43:15.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.111+0000 7fc03f00a740 1 Processor -- start 2024-04-14T08:43:15.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.112+0000 7fc03f00a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41b60 0x561678f45de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.112+0000 7fc03f00a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 0x561678f73f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.112+0000 7fc03f00a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f4b890 0x561678f4bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.112+0000 7fc03f00a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561678f19410 con 0x561678f41b60 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.112+0000 7fc03f00a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561678e399e0 con 0x561678f41f40 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.112+0000 7fc03f00a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x561678f1bc50 con 0x561678f4b890 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.112+0000 7fc03f00a740 10 client.0 _ll_register_callbacks cb 0x561678e76080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.112+0000 7fc039fbc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41b60 0x561678f45de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.112+0000 7fc039fbc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41b60 0x561678f45de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46876/0 (socket says 172.21.15.172:46876) 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.112+0000 7fc039fbc640 1 -- 172.21.15.172:0/1819110848 learned_addr learned my addr 172.21.15.172:0/1819110848 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.112+0000 7fc0397bb640 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 0x561678f73f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.112+0000 7fc03a7bd640 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f4b890 0x561678f4bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc039fbc640 1 -- 172.21.15.172:0/1819110848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f4b890 msgr2=0x561678f4bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc039fbc640 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f4b890 0x561678f4bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc039fbc640 1 -- 172.21.15.172:0/1819110848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 msgr2=0x561678f73f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc039fbc640 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 0x561678f73f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc039fbc640 1 -- 172.21.15.172:0/1819110848 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561678c5c6b0 con 0x561678f41b60 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc039fbc640 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41b60 0x561678f45de0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fc024001120 tx=0x7fc024059b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9977262254f7166f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc014ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc014ff9640 1 -- 172.21.15.172:0/1819110848 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024062070 con 0x561678f41b60 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc014ff9640 1 -- 172.21.15.172:0/1819110848 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7fc0240588e0 con 0x561678f41b60 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc03f00a740 1 -- 172.21.15.172:0/1819110848 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc02c045070 con 0x561678f41b60 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc014ff9640 1 -- 172.21.15.172:0/1819110848 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fc024058ae0 con 0x561678f41b60 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc03f00a740 10 client.9612 fetch_fsmap learned FSMap version 100 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc03f00a740 1 -- 172.21.15.172:0/1819110848 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x561678f468b0 con 0x561678f41b60 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc014ff9640 1 -- 172.21.15.172:0/1819110848 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c045070 con 0x561678f41b60 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc014ff9640 1 -- 172.21.15.172:0/1819110848 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7fc0240551e0 con 0x561678f41b60 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc03f00a740 10 client.9612 fetch_fsmap finished waiting for FSMap version 100 2024-04-14T08:43:15.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc03f00a740 10 client.9612 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc03f00a740 20 client.9612 populate_metadata read hostname 'smithi172' 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.172:6835/4065123823,v1:172.21.15.172:6837/4065123823] conn(0x561678f7bf60 0x561678f7e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc03f00a740 4 client.9612 mds_command: new command op to 9519 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.113+0000 7fc03f00a740 1 -- 172.21.15.172:0/1819110848 --> [v2:172.21.15.172:6835/4065123823,v1:172.21.15.172:6837/4065123823] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561678e25cc0 con 0x561678f7bf60 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.114+0000 7fc0397bb640 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.172:6835/4065123823,v1:172.21.15.172:6837/4065123823] conn(0x561678f7bf60 0x561678f7e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.114+0000 7fc014ff9640 10 client.9612 ms_handle_connect on v2:172.21.15.172:6835/4065123823 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.114+0000 7fc0397bb640 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.172:6835/4065123823,v1:172.21.15.172:6837/4065123823] conn(0x561678f7bf60 0x561678f7e350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0660e40f6cec80b server_cookie=8a5fdae50cbacead in_seq=0 out_seq=0 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.114+0000 7fc014ff9640 1 -- 172.21.15.172:0/1819110848 <== mds.0 v2:172.21.15.172:6835/4065123823 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561678e25cc0 con 0x561678f7bf60 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.114+0000 7fc014ff9640 10 client.9612 handle_command_reply: tid=0 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc03f00a740 1 client.9612 shutdown 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc03f00a740 1 -- 172.21.15.172:0/1819110848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41b60 msgr2=0x561678f45de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41b60 0x561678f45de0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fc024001120 tx=0x7fc024059b90 comp rx=0 tx=0).stop 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc03f00a740 1 -- 172.21.15.172:0/1819110848 shutdown_connections 2024-04-14T08:43:15.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.172:6835/4065123823,v1:172.21.15.172:6837/4065123823] conn(0x561678f7bf60 0x561678f7e350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f4b890 0x561678f4bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc014ff9640 0 client.9612 ms_handle_reset on v2:172.21.15.172:6835/4065123823 2024-04-14T08:43:15.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 0x561678f73f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1819110848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41b60 0x561678f45de0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc03f00a740 1 -- 172.21.15.172:0/1819110848 >> 172.21.15.172:0/1819110848 conn(0x561678f15040 msgr2=0x561678f62e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:15.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc03f00a740 1 -- 172.21.15.172:0/1819110848 shutdown_connections 2024-04-14T08:43:15.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc03f00a740 1 -- 172.21.15.172:0/1819110848 wait complete. 2024-04-14T08:43:15.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.115+0000 7fc03f00a740 20 client.9612 trim_cache size 0 max 16384 2024-04-14T08:43:15.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.150+0000 7fc03f00a740 1 Processor -- start 2024-04-14T08:43:15.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.150+0000 7fc03f00a740 1 -- start start 2024-04-14T08:43:15.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.151+0000 7fc03f00a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f41b60 0x561678f5fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.151+0000 7fc03f00a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 0x561678f60050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.151+0000 7fc03f00a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f4b890 0x561678f4a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.151+0000 7fc03f00a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc034200180 con 0x561678f4b890 2024-04-14T08:43:15.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.151+0000 7fc03f00a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0340cd3d0 con 0x561678f41f40 2024-04-14T08:43:15.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.151+0000 7fc03f00a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc0340cda20 con 0x561678f41b60 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.151+0000 7fc03a7bd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f4b890 0x561678f4a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.151+0000 7fc03a7bd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f4b890 0x561678f4a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46888/0 (socket says 172.21.15.172:46888) 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.151+0000 7fc03a7bd640 1 -- 172.21.15.172:0/1933399876 learned_addr learned my addr 172.21.15.172:0/1933399876 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.151+0000 7fc039fbc640 1 --2- 172.21.15.172:0/1933399876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f41b60 0x561678f5fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.151+0000 7fc0397bb640 1 --2- 172.21.15.172:0/1933399876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 0x561678f60050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.152+0000 7fc03a7bd640 1 -- 172.21.15.172:0/1933399876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f41b60 msgr2=0x561678f5fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.152+0000 7fc03a7bd640 1 --2- 172.21.15.172:0/1933399876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f41b60 0x561678f5fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.152+0000 7fc03a7bd640 1 -- 172.21.15.172:0/1933399876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 msgr2=0x561678f60050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.152+0000 7fc03a7bd640 1 --2- 172.21.15.172:0/1933399876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 0x561678f60050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.152+0000 7fc03a7bd640 1 -- 172.21.15.172:0/1933399876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561678f46e70 con 0x561678f4b890 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.152+0000 7fc03a7bd640 1 --2- 172.21.15.172:0/1933399876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f4b890 0x561678f4a3e0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fc02c042040 tx=0x7fc02c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103413c2d52ad1a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.152+0000 7fc016ffd640 1 -- 172.21.15.172:0/1933399876 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c003b90 con 0x561678f4b890 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.152+0000 7fc016ffd640 1 -- 172.21.15.172:0/1933399876 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c045040 con 0x561678f4b890 2024-04-14T08:43:15.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.152+0000 7fc016ffd640 1 -- 172.21.15.172:0/1933399876 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c02ce20 con 0x561678f4b890 2024-04-14T08:43:15.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.153+0000 7fc03f00a740 1 -- 172.21.15.172:0/1933399876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f4b890 msgr2=0x561678f4a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.153+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1933399876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f4b890 0x561678f4a3e0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fc02c042040 tx=0x7fc02c030000 comp rx=0 tx=0).stop 2024-04-14T08:43:15.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.153+0000 7fc03f00a740 1 -- 172.21.15.172:0/1933399876 shutdown_connections 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.153+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1933399876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f4b890 0x561678f4a3e0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.153+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1933399876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41f40 0x561678f60050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.153+0000 7fc03f00a740 1 --2- 172.21.15.172:0/1933399876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f41b60 0x561678f5fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.153+0000 7fc03f00a740 1 -- 172.21.15.172:0/1933399876 >> 172.21.15.172:0/1933399876 conn(0x561678f15040 msgr2=0x561678f61f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.153+0000 7fc03f00a740 1 -- 172.21.15.172:0/1933399876 shutdown_connections 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.153+0000 7fc03f00a740 1 -- 172.21.15.172:0/1933399876 wait complete. 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.153+0000 7fc03f00a740 1 -- start start 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.153+0000 7fc03f00a740 1 Processor -- start 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.154+0000 7fc03f00a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41b60 0x561678f6de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.154+0000 7fc03f00a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41f40 0x561678f18d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.154+0000 7fc03f00a740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f4b890 0x561678f5fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.154+0000 7fc03f00a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561678f47150 con 0x561678f41f40 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.154+0000 7fc03f00a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561678f46b90 con 0x561678f41b60 2024-04-14T08:43:15.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.154+0000 7fc03f00a740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x561678f476c0 con 0x561678f4b890 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.154+0000 7fc03f00a740 10 client.0 _ll_register_callbacks cb 0x7fc024055730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.154+0000 7fc0397bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41f40 0x561678f18d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.154+0000 7fc0397bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41f40 0x561678f18d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46900/0 (socket says 172.21.15.172:46900) 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.154+0000 7fc0397bb640 1 -- 172.21.15.172:0/3618743819 learned_addr learned my addr 172.21.15.172:0/3618743819 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.154+0000 7fc039fbc640 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41b60 0x561678f6de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.154+0000 7fc03a7bd640 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f4b890 0x561678f5fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc0397bb640 1 -- 172.21.15.172:0/3618743819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f4b890 msgr2=0x561678f5fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc0397bb640 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f4b890 0x561678f5fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc0397bb640 1 -- 172.21.15.172:0/3618743819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41b60 msgr2=0x561678f6de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc0397bb640 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41b60 0x561678f6de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc0397bb640 1 -- 172.21.15.172:0/3618743819 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561678f6e380 con 0x561678f41f40 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc03a7bd640 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f4b890 0x561678f5fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc0397bb640 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41f40 0x561678f18d40 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fc0340cada0 tx=0x7fc030004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1da17df0345cdad server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:15.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc014ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc014ff9640 1 -- 172.21.15.172:0/3618743819 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030013b30 con 0x561678f41f40 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc014ff9640 1 -- 172.21.15.172:0/3618743819 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7fc030013cd0 con 0x561678f41f40 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc03f00a740 1 -- 172.21.15.172:0/3618743819 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc02c045040 con 0x561678f41f40 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc014ff9640 1 -- 172.21.15.172:0/3618743819 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fc030038050 con 0x561678f41f40 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc03f00a740 10 client.9618 fetch_fsmap learned FSMap version 100 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc03f00a740 1 -- 172.21.15.172:0/3618743819 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x561678f185d0 con 0x561678f41f40 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc014ff9640 1 -- 172.21.15.172:0/3618743819 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c045040 con 0x561678f41f40 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc014ff9640 1 -- 172.21.15.172:0/3618743819 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7fc03002e020 con 0x561678f41f40 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc03f00a740 10 client.9618 fetch_fsmap finished waiting for FSMap version 100 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc03f00a740 10 client.9618 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc03f00a740 20 client.9618 populate_metadata read hostname 'smithi172' 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc03f00a740 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.172:6835/4065123823,v1:172.21.15.172:6837/4065123823] conn(0x561678f7bf60 0x561678f3f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc03f00a740 4 client.9618 mds_command: new command op to 9519 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-14T08:43:15.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc03f00a740 1 -- 172.21.15.172:0/3618743819 --> [v2:172.21.15.172:6835/4065123823,v1:172.21.15.172:6837/4065123823] -- command(tid 0: {"prefix": "status"}) -- 0x561678e8b570 con 0x561678f7bf60 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.155+0000 7fc039fbc640 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.172:6835/4065123823,v1:172.21.15.172:6837/4065123823] conn(0x561678f7bf60 0x561678f3f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc014ff9640 10 client.9618 ms_handle_connect on v2:172.21.15.172:6835/4065123823 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc039fbc640 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.172:6835/4065123823,v1:172.21.15.172:6837/4065123823] conn(0x561678f7bf60 0x561678f3f900 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1071ead30d3fda server_cookie=4716b927ad520b6f in_seq=0 out_seq=0 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc014ff9640 1 -- 172.21.15.172:0/3618743819 <== mds.0 v2:172.21.15.172:6835/4065123823 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561678e8b570 con 0x561678f7bf60 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc014ff9640 10 client.9618 handle_command_reply: tid=0 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc03f00a740 1 client.9618 shutdown 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc03f00a740 1 -- 172.21.15.172:0/3618743819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41f40 msgr2=0x561678f18d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc03f00a740 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41f40 0x561678f18d40 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fc0340cada0 tx=0x7fc030004680 comp rx=0 tx=0).stop 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc03f00a740 1 -- 172.21.15.172:0/3618743819 shutdown_connections 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc03f00a740 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.172:6835/4065123823,v1:172.21.15.172:6837/4065123823] conn(0x561678f7bf60 0x561678f3f900 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc03f00a740 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561678f4b890 0x561678f5fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc03f00a740 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561678f41f40 0x561678f18d40 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc03f00a740 1 --2- 172.21.15.172:0/3618743819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561678f41b60 0x561678f6de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc014ff9640 0 client.9618 ms_handle_reset on v2:172.21.15.172:6835/4065123823 2024-04-14T08:43:15.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc03f00a740 1 -- 172.21.15.172:0/3618743819 >> 172.21.15.172:0/3618743819 conn(0x561678f15040 msgr2=0x561678f34400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:15.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc03f00a740 1 -- 172.21.15.172:0/3618743819 shutdown_connections 2024-04-14T08:43:15.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc03f00a740 1 -- 172.21.15.172:0/3618743819 wait complete. 2024-04-14T08:43:15.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.156+0000 7fc03f00a740 20 client.9618 trim_cache size 0 max 16384 2024-04-14T08:43:15.163 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:43:15.163 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "8e7e3a2f-e38f-4857-b139-5df1f7a099dc", 2024-04-14T08:43:15.163 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-04-14T08:43:15.163 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 9519, 2024-04-14T08:43:15.163 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-04-14T08:43:15.163 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-04-14T08:43:15.164 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-04-14T08:43:15.164 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 4.4306416569999998, 2024-04-14T08:43:15.164 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 100, 2024-04-14T08:43:15.164 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 118, 2024-04-14T08:43:15.164 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 118, 2024-04-14T08:43:15.164 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 6.410505498 2024-04-14T08:43:15.164 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:43:15.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.157+0000 7fc03c220640 1 -- 172.21.15.172:0/3776677173 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc01806c8f0 msgr2=0x7fc01806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.157+0000 7fc03c220640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc01806c8f0 0x7fc01806edb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc0340b2300 tx=0x7fc030008040 comp rx=0 tx=0).stop 2024-04-14T08:43:15.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.157+0000 7fc03c220640 1 -- 172.21.15.172:0/3776677173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03415e430 msgr2=0x7fc03414f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.157+0000 7fc03c220640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03415e430 0x7fc03414f5e0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fc02400c430 tx=0x7fc02400c8f0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.157+0000 7fc03c220640 1 -- 172.21.15.172:0/3776677173 shutdown_connections 2024-04-14T08:43:15.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.157+0000 7fc03c220640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc01806c8f0 0x7fc01806edb0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.157+0000 7fc03c220640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc034162ba0 0x7fc0340b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.157+0000 7fc03c220640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc034162710 0x7fc034161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.157+0000 7fc03c220640 1 --2- 172.21.15.172:0/3776677173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc03415e430 0x7fc03414f5e0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.157+0000 7fc03c220640 1 -- 172.21.15.172:0/3776677173 >> 172.21.15.172:0/3776677173 conn(0x7fc03409d560 msgr2=0x7fc0340b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:15.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.157+0000 7fc03c220640 1 -- 172.21.15.172:0/3776677173 shutdown_connections 2024-04-14T08:43:15.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.157+0000 7fc03c220640 1 -- 172.21.15.172:0/3776677173 wait complete. 2024-04-14T08:43:15.168 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-14T08:43:15.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.270+0000 7f757cdbb640 1 Processor -- start 2024-04-14T08:43:15.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.270+0000 7f757cdbb640 1 -- start start 2024-04-14T08:43:15.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.271+0000 7f757cdbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75780d2830 0x7f75780d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.271+0000 7f757cdbb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75780d3150 con 0x7f75780cedd0 2024-04-14T08:43:15.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.271+0000 7f757cdbb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75780cf320 con 0x7f757815e890 2024-04-14T08:43:15.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.271+0000 7f757cdbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f75780cff30 con 0x7f75780d2830 2024-04-14T08:43:15.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.271+0000 7f7576d9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f75780cedd0 0x7f75780b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54950/0 (socket says 172.21.15.172:54950) 2024-04-14T08:43:15.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.271+0000 7f7576d9d640 1 -- 172.21.15.172:0/60652138 learned_addr learned my addr 172.21.15.172:0/60652138 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.271+0000 7f757659c640 1 --2- 172.21.15.172:0/60652138 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75780d2830 0x7f75780d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.271+0000 7f7575d9b640 1 -- 172.21.15.172:0/60652138 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 721716624 0 0) 0x7f75780d3150 con 0x7f75780cedd0 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.271+0000 7f7575d9b640 1 -- 172.21.15.172:0/60652138 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7564003520 con 0x7f75780cedd0 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.271+0000 7f7575d9b640 1 -- 172.21.15.172:0/60652138 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1537874403 0 0) 0x7f75780cf320 con 0x7f757815e890 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.272+0000 7f7575d9b640 1 -- 172.21.15.172:0/60652138 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75780d3150 con 0x7f757815e890 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.272+0000 7f757659c640 1 -- 172.21.15.172:0/60652138 >> v1:172.21.15.177:6789/0 conn(0x7f757815e890 legacy=0x7f75780d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.272+0000 7f757659c640 1 -- 172.21.15.172:0/60652138 >> v1:172.21.15.172:6789/0 conn(0x7f75780cedd0 legacy=0x7f75780b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.272+0000 7f7575d9b640 1 -- 172.21.15.172:0/60652138 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 759955647 0 0) 0x7f7564003520 con 0x7f75780cedd0 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.272+0000 7f757659c640 1 -- 172.21.15.172:0/60652138 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75780cf600 con 0x7f75780d2830 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.272+0000 7f757659c640 1 --2- 172.21.15.172:0/60652138 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75780d2830 0x7f75780d2c10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75680097d0 tx=0x7f756802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da7b4186ba2c769f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.273+0000 7f7575d9b640 1 -- 172.21.15.172:0/60652138 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700019f0 con 0x7f75780d2830 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.273+0000 7f7575d9b640 1 -- 172.21.15.172:0/60652138 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7568037070 con 0x7f75780d2830 2024-04-14T08:43:15.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.273+0000 7f7575d9b640 1 -- 172.21.15.172:0/60652138 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568031780 con 0x7f75780d2830 2024-04-14T08:43:15.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.273+0000 7f757cdbb640 1 -- 172.21.15.172:0/60652138 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75780d2830 msgr2=0x7f75780d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.273+0000 7f757cdbb640 1 --2- 172.21.15.172:0/60652138 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75780d2830 0x7f75780d2c10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75680097d0 tx=0x7f756802b8a0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.273+0000 7f757cdbb640 1 -- 172.21.15.172:0/60652138 shutdown_connections 2024-04-14T08:43:15.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.273+0000 7f757cdbb640 1 --2- 172.21.15.172:0/60652138 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75780d2830 0x7f75780d2c10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.273+0000 7f757cdbb640 1 -- 172.21.15.172:0/60652138 >> 172.21.15.172:0/60652138 conn(0x7f757809d590 msgr2=0x7f75780d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:15.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.273+0000 7f757cdbb640 1 -- 172.21.15.172:0/60652138 shutdown_connections 2024-04-14T08:43:15.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.273+0000 7f757cdbb640 1 -- 172.21.15.172:0/60652138 wait complete. 2024-04-14T08:43:15.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f757cdbb640 1 Processor -- start 2024-04-14T08:43:15.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f757cdbb640 1 -- start start 2024-04-14T08:43:15.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f757cdbb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75780cedd0 0x7f757814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f757cdbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75780d2830 0x7f75780cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f757cdbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f757815e890 0x7f757815b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f757cdbb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f75780cfbc0 con 0x7f757815e890 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f757cdbb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f75780d2350 con 0x7f75780cedd0 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f757cdbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f75780b3e20 con 0x7f75780d2830 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f7576d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f757815e890 0x7f757815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f7576d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f757815e890 0x7f757815b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46916/0 (socket says 172.21.15.172:46916) 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f7576d9d640 1 -- 172.21.15.172:0/1014974678 learned_addr learned my addr 172.21.15.172:0/1014974678 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f756dd9b640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75780d2830 0x7f75780cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.275+0000 7f757659c640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75780cedd0 0x7f757814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.276+0000 7f7576d9d640 1 -- 172.21.15.172:0/1014974678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75780d2830 msgr2=0x7f75780cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.276+0000 7f7576d9d640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75780d2830 0x7f75780cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.276+0000 7f7576d9d640 1 -- 172.21.15.172:0/1014974678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75780cedd0 msgr2=0x7f757814f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.276+0000 7f7576d9d640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75780cedd0 0x7f757814f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.276+0000 7f7576d9d640 1 -- 172.21.15.172:0/1014974678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75781f4510 con 0x7f757815e890 2024-04-14T08:43:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.276+0000 7f7576d9d640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f757815e890 0x7f757815b140 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f7570001f30 tx=0x7f757000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.276+0000 7f756f7fe640 1 -- 172.21.15.172:0/1014974678 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570017070 con 0x7f757815e890 2024-04-14T08:43:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.276+0000 7f756f7fe640 1 -- 172.21.15.172:0/1014974678 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7570012400 con 0x7f757815e890 2024-04-14T08:43:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.276+0000 7f757cdbb640 1 -- 172.21.15.172:0/1014974678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757814fbe0 con 0x7f757815e890 2024-04-14T08:43:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.276+0000 7f756f7fe640 1 -- 172.21.15.172:0/1014974678 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570012830 con 0x7f757815e890 2024-04-14T08:43:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.276+0000 7f757cdbb640 1 -- 172.21.15.172:0/1014974678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75781f4a60 con 0x7f757815e890 2024-04-14T08:43:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.277+0000 7f757cdbb640 1 -- 172.21.15.172:0/1014974678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7578068520 con 0x7f757815e890 2024-04-14T08:43:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.278+0000 7f756f7fe640 1 -- 172.21.15.172:0/1014974678 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f757001a070 con 0x7f757815e890 2024-04-14T08:43:15.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.278+0000 7f756f7fe640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f755806c8f0 0x7f755806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:15.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.278+0000 7f756f7fe640 1 -- 172.21.15.172:0/1014974678 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7f757008bce0 con 0x7f757815e890 2024-04-14T08:43:15.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.278+0000 7f757659c640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f755806c8f0 0x7f755806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:15.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.279+0000 7f757659c640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f755806c8f0 0x7f755806edb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7568036000 tx=0x7f7568008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:15.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.283+0000 7f756f7fe640 1 -- 172.21.15.172:0/1014974678 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f757814fbe0 con 0x7f757815e890 2024-04-14T08:43:15.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.533+0000 7f757cdbb640 1 -- 172.21.15.172:0/1014974678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f75780d2610 con 0x7f757815e890 2024-04-14T08:43:15.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.533+0000 7f756f7fe640 1 -- 172.21.15.172:0/1014974678 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7f7570059880 con 0x7f757815e890 2024-04-14T08:43:15.534 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-14T08:43:15.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.535+0000 7f757cdbb640 1 -- 172.21.15.172:0/1014974678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f755806c8f0 msgr2=0x7f755806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.535+0000 7f757cdbb640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f755806c8f0 0x7f755806edb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7568036000 tx=0x7f7568008030 comp rx=0 tx=0).stop 2024-04-14T08:43:15.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.535+0000 7f757cdbb640 1 -- 172.21.15.172:0/1014974678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f757815e890 msgr2=0x7f757815b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:15.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.535+0000 7f757cdbb640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f757815e890 0x7f757815b140 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f7570001f30 tx=0x7f757000a430 comp rx=0 tx=0).stop 2024-04-14T08:43:15.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.535+0000 7f757cdbb640 1 -- 172.21.15.172:0/1014974678 shutdown_connections 2024-04-14T08:43:15.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.535+0000 7f757cdbb640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f755806c8f0 0x7f755806edb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.535+0000 7f757cdbb640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f757815e890 0x7f757815b140 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.535+0000 7f757cdbb640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75780d2830 0x7f75780cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.536+0000 7f757cdbb640 1 --2- 172.21.15.172:0/1014974678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75780cedd0 0x7f757814f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:15.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.536+0000 7f757cdbb640 1 -- 172.21.15.172:0/1014974678 >> 172.21.15.172:0/1014974678 conn(0x7f757809d590 msgr2=0x7f75780d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:15.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.536+0000 7f757cdbb640 1 -- 172.21.15.172:0/1014974678 shutdown_connections 2024-04-14T08:43:15.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15.536+0000 7f757cdbb640 1 -- 172.21.15.172:0/1014974678 wait complete. 2024-04-14T08:43:15.547 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-14T08:43:15.548 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-14T08:43:15.548 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-14T08:43:15.548 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-14T08:43:15.548 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-14T08:43:15.548 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-14T08:43:15.548 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-14T08:43:15.548 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-14T08:43:15.548 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:43:15.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:15.548 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-14T08:43:15.565 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-14T08:43:15.565 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-14T08:43:15.565 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-14T08:43:15.565 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:15.565 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-14T08:43:15.565 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:15.566 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:43:15.566 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-14T08:43:15.566 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-14T08:43:15.566 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:43:15.566 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-14T08:43:15.566 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-14T08:43:15.566 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-14T08:43:15.566 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:43:15.566 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-14T08:43:15.566 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:43:15.566 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42074sec preferred_lft 42074sec 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::d831:d9ff:feab:94e0/64 scope link 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 6a:98:6f:19:7c:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::6898:6fff:fe19:7c5c/64 scope link 2024-04-14T08:43:15.567 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:15.568 INFO:teuthology.orchestra.run.smithi172.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-14T08:43:15.568 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 22:36:c4:f6:4d:d9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:43:15.568 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::2036:c4ff:fef6:4dd9/64 scope link 2024-04-14T08:43:15.568 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-14T08:43:15.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:15.568 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-14T08:43:15.631 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:43:15.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:15.633 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-14T08:43:15.688 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-14T08:43:15.688 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-14T08:43:15.689 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-14T08:43:15.689 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-14T08:43:15.689 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-14T08:43:15.689 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-14T08:43:15.689 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-14T08:43:15.689 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-14T08:43:15.689 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-14T08:43:15.689 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-14T08:43:15.689 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-14T08:43:15.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-14T08:43:15.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:15.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:15.826 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:43:15.841 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:43:15.841 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:43:15.841 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:43:15.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:15.842 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-14T08:43:15.909 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:43:15.909 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-14T08:43:15.964 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-14T08:43:15.965 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-14T08:43:15.965 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:43:16.020 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-14T08:43:16.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:16.020 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:43:16.110 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:43:16.110 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:43:16.110 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:43:16.118 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:43:16.118 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:43:16.118 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:43:16.118 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:43:16.118 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:43:16.123 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-14T08:43:16.123 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-14T08:43:16.123 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-14T08:43:16.123 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:3300,172.21.15.177:3300,172.21.15.177:3301" 2024-04-14T08:43:16.123 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-14T08:43:16.123 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790" 2024-04-14T08:43:16.123 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.172:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2024-04-14T08:43:16.123 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8e7e3a2f-e38f-4857-b139-5df1f7a099dc" will pass to kernel 2024-04-14T08:43:16.124 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-14T08:43:16.124 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-14T08:43:16.124 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-14T08:43:16.124 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-14T08:43:16.124 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-14T08:43:16.125 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-14T08:43:16.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:16.126 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-14T08:43:16.153 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-14T08:43:16.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-14T08:43:16.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:16.248 INFO:teuthology.orchestra.run.smithi172.stdout:client9627 2024-04-14T08:43:16.248 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:43:16.249 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:16.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:16.265 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-14T08:43:16.324 INFO:teuthology.orchestra.run.smithi172.stdout:./ 2024-04-14T08:43:16.324 INFO:teuthology.orchestra.run.smithi172.stdout:./parent 2024-04-14T08:43:16.324 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/flushed 2024-04-14T08:43:16.324 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/flushed/charlie 2024-04-14T08:43:16.324 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/flushed/alpha 2024-04-14T08:43:16.324 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/flushed/bravo 2024-04-14T08:43:16.324 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/unflushed 2024-04-14T08:43:16.324 INFO:teuthology.orchestra.run.smithi172.stdout:./parent/unflushed/jfile 2024-04-14T08:43:16.325 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:16.325 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:16.325 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:16.325 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:16.325 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-14T08:43:16.325 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:16.448 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-14T08:43:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:16.459 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:16.459 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:16.459 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:16.459 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:16.459 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-14T08:43:16.459 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:16.576 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627776 2024-04-14T08:43:16.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:16.586 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:16.587 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:16.587 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:16.587 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:16.587 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-14T08:43:16.587 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:16.692 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627777 2024-04-14T08:43:16.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:16.702 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:16.702 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:16.702 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:16.702 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:16.702 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-14T08:43:16.702 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:16.808 INFO:teuthology.orchestra.run.smithi172.stdout:1099511628280 2024-04-14T08:43:16.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:16.817 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:16.817 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:16.817 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:16.817 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:16.817 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-14T08:43:16.817 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:16.933 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627778 2024-04-14T08:43:16.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:16.942 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:16.942 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:16.942 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:16.942 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:16.942 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-14T08:43:16.942 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:17.048 INFO:teuthology.orchestra.run.smithi172.stdout:1099511628279 2024-04-14T08:43:17.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:17.059 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:17.059 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:17.059 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:17.059 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:17.059 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-14T08:43:17.060 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:17.164 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627779 2024-04-14T08:43:17.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:17.174 DEBUG:teuthology.orchestra.run.smithi172:> stdin-killer -- python3 -c ' 2024-04-14T08:43:17.174 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-04-14T08:43:17.174 DEBUG:teuthology.orchestra.run.smithi172:> import stat 2024-04-14T08:43:17.174 DEBUG:teuthology.orchestra.run.smithi172:> 2024-04-14T08:43:17.174 DEBUG:teuthology.orchestra.run.smithi172:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-14T08:43:17.174 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-04-14T08:43:17.282 INFO:teuthology.orchestra.run.smithi172.stdout:1099511627780 2024-04-14T08:43:17.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-14T08:43:17.290 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-14T08:43:17.290 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:43:17.372 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-14T08:43:17.414 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-14T08:43:17.415 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:43:17.415 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:43:17.415 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:17.431 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:43:17.431 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:17.488 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:43:17.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.629+0000 7f7b1918c640 1 Processor -- start 2024-04-14T08:43:17.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.629+0000 7f7b1918c640 1 -- start start 2024-04-14T08:43:17.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.629+0000 7f7b1918c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7b140b7a60 0x7f7b140ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.629+0000 7f7b1918c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7b1406dc40 0x7f7b140cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b1918c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b140d2b10 con 0x7f7b1408c4c0 2024-04-14T08:43:17.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b1918c640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f7b140d0ad0 con 0x7f7b1406dc40 2024-04-14T08:43:17.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b1918c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f7b140d0db0 con 0x7f7b140b7a60 2024-04-14T08:43:17.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b1359e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7b1406dc40 0x7f7b140cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b1359e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7b1406dc40 0x7f7b140cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46920/0 (socket says 172.21.15.172:46920) 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b0a59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7b1408c4c0 0x7f7b140cd410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54954/0 (socket says 172.21.15.172:54954) 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b1359e640 1 -- 172.21.15.172:0/2911857582 learned_addr learned my addr 172.21.15.172:0/2911857582 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b12d9d640 1 --2- 172.21.15.172:0/2911857582 >> v2:172.21.15.177:3300/0 conn(0x7f7b140b7a60 0x7f7b140ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b1359e640 1 -- 172.21.15.172:0/2911857582 >> v2:172.21.15.177:3300/0 conn(0x7f7b140b7a60 msgr2=0x7f7b140ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b1359e640 1 --2- 172.21.15.172:0/2911857582 >> v2:172.21.15.177:3300/0 conn(0x7f7b140b7a60 0x7f7b140ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b1359e640 1 -- 172.21.15.172:0/2911857582 >> v1:172.21.15.172:6789/0 conn(0x7f7b1408c4c0 legacy=0x7f7b140cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b1359e640 1 -- 172.21.15.172:0/2911857582 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b140d1090 con 0x7f7b1406dc40 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b1259c640 1 -- 172.21.15.172:0/2911857582 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 354910733 0 0) 0x7f7b140d2b10 con 0x7f7b1408c4c0 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.630+0000 7f7b12d9d640 1 --2- 172.21.15.172:0/2911857582 >> v2:172.21.15.177:3300/0 conn(0x7f7b140b7a60 0x7f7b140ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.631+0000 7f7b1359e640 1 --2- 172.21.15.172:0/2911857582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1406dc40 0x7f7b140cee00 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f7b14069160 tx=0x7f7b0c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f237285b3d9c478a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.631+0000 7f7b1259c640 1 -- 172.21.15.172:0/2911857582 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c02ca80 con 0x7f7b1406dc40 2024-04-14T08:43:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.631+0000 7f7b1259c640 1 -- 172.21.15.172:0/2911857582 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0c02cc20 con 0x7f7b1406dc40 2024-04-14T08:43:17.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.631+0000 7f7b1259c640 1 -- 172.21.15.172:0/2911857582 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c0343f0 con 0x7f7b1406dc40 2024-04-14T08:43:17.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.632+0000 7f7b1918c640 1 -- 172.21.15.172:0/2911857582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1406dc40 msgr2=0x7f7b140cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:17.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.632+0000 7f7b1918c640 1 --2- 172.21.15.172:0/2911857582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1406dc40 0x7f7b140cee00 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f7b14069160 tx=0x7f7b0c02bee0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.632+0000 7f7b1918c640 1 -- 172.21.15.172:0/2911857582 shutdown_connections 2024-04-14T08:43:17.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.632+0000 7f7b1918c640 1 --2- 172.21.15.172:0/2911857582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1406dc40 0x7f7b140cee00 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.632+0000 7f7b1918c640 1 --2- 172.21.15.172:0/2911857582 >> v2:172.21.15.177:3300/0 conn(0x7f7b140b7a60 0x7f7b140ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.632+0000 7f7b1918c640 1 -- 172.21.15.172:0/2911857582 >> 172.21.15.172:0/2911857582 conn(0x7f7b1408c000 msgr2=0x7f7b140d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:17.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.632+0000 7f7b1918c640 1 -- 172.21.15.172:0/2911857582 shutdown_connections 2024-04-14T08:43:17.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.632+0000 7f7b1918c640 1 -- 172.21.15.172:0/2911857582 wait complete. 2024-04-14T08:43:17.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.633+0000 7f7b1918c640 1 Processor -- start 2024-04-14T08:43:17.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.633+0000 7f7b1918c640 1 -- start start 2024-04-14T08:43:17.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b1918c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b1406dc40 0x7f7b140b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b1918c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1408c4c0 0x7f7b1415cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b1918c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b140b7a60 0x7f7b1415f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b1918c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b140d2750 con 0x7f7b1408c4c0 2024-04-14T08:43:17.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b1918c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7b140b7e40 con 0x7f7b1406dc40 2024-04-14T08:43:17.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b1918c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7b140d28d0 con 0x7f7b140b7a60 2024-04-14T08:43:17.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b0a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1408c4c0 0x7f7b1415cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b0a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1408c4c0 0x7f7b1415cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46926/0 (socket says 172.21.15.172:46926) 2024-04-14T08:43:17.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b0a59c640 1 -- 172.21.15.172:0/2365490382 learned_addr learned my addr 172.21.15.172:0/2365490382 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:17.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b12d9d640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b1406dc40 0x7f7b140b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b1359e640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b140b7a60 0x7f7b1415f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b0a59c640 1 -- 172.21.15.172:0/2365490382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b140b7a60 msgr2=0x7f7b1415f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b0a59c640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b140b7a60 0x7f7b1415f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b0a59c640 1 -- 172.21.15.172:0/2365490382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b1406dc40 msgr2=0x7f7b140b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b0a59c640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b1406dc40 0x7f7b140b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b0a59c640 1 -- 172.21.15.172:0/2365490382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b140b2b10 con 0x7f7b1408c4c0 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.634+0000 7f7b1359e640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b140b7a60 0x7f7b1415f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.635+0000 7f7b0a59c640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1408c4c0 0x7f7b1415cee0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00c500 tx=0x7f7afc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.635+0000 7f7b0bfff640 1 -- 172.21.15.172:0/2365490382 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc018070 con 0x7f7b1408c4c0 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.635+0000 7f7b0bfff640 1 -- 172.21.15.172:0/2365490382 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7afc007580 con 0x7f7b1408c4c0 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.635+0000 7f7b1918c640 1 -- 172.21.15.172:0/2365490382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b140b1530 con 0x7f7b1408c4c0 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.635+0000 7f7b0bfff640 1 -- 172.21.15.172:0/2365490382 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc004a20 con 0x7f7b1408c4c0 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.635+0000 7f7b1918c640 1 -- 172.21.15.172:0/2365490382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b140b1af0 con 0x7f7b1408c4c0 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.636+0000 7f7b1918c640 1 -- 172.21.15.172:0/2365490382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b14075490 con 0x7f7b1408c4c0 2024-04-14T08:43:17.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.636+0000 7f7b0bfff640 1 -- 172.21.15.172:0/2365490382 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7afc01b070 con 0x7f7b1408c4c0 2024-04-14T08:43:17.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.637+0000 7f7b0bfff640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7af406c8f0 0x7f7af406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.637+0000 7f7b12d9d640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7af406c8f0 0x7f7af406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.637+0000 7f7b0bfff640 1 -- 172.21.15.172:0/2365490382 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7f7afc08b060 con 0x7f7b1408c4c0 2024-04-14T08:43:17.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.637+0000 7f7b12d9d640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7af406c8f0 0x7f7af406edb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7b141622e0 tx=0x7f7b04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:17.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.641+0000 7f7b0bfff640 1 -- 172.21.15.172:0/2365490382 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7afc0592e0 con 0x7f7b1408c4c0 2024-04-14T08:43:17.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.822+0000 7f7b1918c640 1 -- 172.21.15.172:0/2365490382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b140b6fa0 con 0x7f7b1408c4c0 2024-04-14T08:43:17.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.823+0000 7f7b0bfff640 1 -- 172.21.15.172:0/2365490382 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+17273 (secure 0 0 0) 0x7f7afc058be0 con 0x7f7b1408c4c0 2024-04-14T08:43:17.824 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:17.824 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":118,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:43:08.711666+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:42:38.455900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-14T08:42:39.198157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/744853572":"2024-04-15T08:43:08.711642+0000","172.21.15.177:6835/3201353747":"2024-04-15T08:42:57.915856+0000","172.21.15.172:6835/744853572":"2024-04-15T08:43:08.711642+0000","172.21.15.177:6833/3201353747":"2024-04-15T08:42:57.915856+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:43:17.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.826+0000 7f7b1918c640 1 -- 172.21.15.172:0/2365490382 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7af406c8f0 msgr2=0x7f7af406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:17.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.826+0000 7f7b1918c640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7af406c8f0 0x7f7af406edb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7b141622e0 tx=0x7f7b04008040 comp rx=0 tx=0).stop 2024-04-14T08:43:17.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.826+0000 7f7b1918c640 1 -- 172.21.15.172:0/2365490382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1408c4c0 msgr2=0x7f7b1415cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:17.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.826+0000 7f7b1918c640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1408c4c0 0x7f7b1415cee0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00c500 tx=0x7f7afc00c9c0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.826+0000 7f7b1918c640 1 -- 172.21.15.172:0/2365490382 shutdown_connections 2024-04-14T08:43:17.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.826+0000 7f7b1918c640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7af406c8f0 0x7f7af406edb0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.826+0000 7f7b1918c640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b140b7a60 0x7f7b1415f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.826+0000 7f7b1918c640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b1408c4c0 0x7f7b1415cee0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.826+0000 7f7b1918c640 1 --2- 172.21.15.172:0/2365490382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b1406dc40 0x7f7b140b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.826+0000 7f7b1918c640 1 -- 172.21.15.172:0/2365490382 >> 172.21.15.172:0/2365490382 conn(0x7f7b1408c000 msgr2=0x7f7b140b5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:17.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.827+0000 7f7b1918c640 1 -- 172.21.15.172:0/2365490382 shutdown_connections 2024-04-14T08:43:17.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.827+0000 7f7b1918c640 1 -- 172.21.15.172:0/2365490382 wait complete. 2024-04-14T08:43:17.838 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:42:38.455900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:42:39.198157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:43:17.838 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:43:17.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.943+0000 7f3d19288640 1 Processor -- start 2024-04-14T08:43:17.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.943+0000 7f3d19288640 1 -- start start 2024-04-14T08:43:17.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d19288640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3d140b23e0 0x7f3d140adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d19288640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d140cf9b0 con 0x7f3d14140fa0 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d19288640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d1415ecc0 con 0x7f3d14141380 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d19288640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f3d1415efa0 con 0x7f3d140b23e0 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d12ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3d14140fa0 0x7f3d14162840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54962/0 (socket says 172.21.15.172:54962) 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d12ffd640 1 -- 172.21.15.172:0/1080124542 learned_addr learned my addr 172.21.15.172:0/1080124542 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d137fe640 1 --2- 172.21.15.172:0/1080124542 >> v2:172.21.15.177:3300/0 conn(0x7f3d140b23e0 0x7f3d140adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1261073531 0 0) 0x7f3d140cf9b0 con 0x7f3d14140fa0 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d00003520 con 0x7f3d14140fa0 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 564668700 0 0) 0x7f3d00003520 con 0x7f3d14140fa0 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 >> v2:172.21.15.177:3300/0 conn(0x7f3d140b23e0 msgr2=0x7f3d140adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 --2- 172.21.15.172:0/1080124542 >> v2:172.21.15.177:3300/0 conn(0x7f3d140b23e0 0x7f3d140adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 >> v1:172.21.15.177:6789/0 conn(0x7f3d14141380 legacy=0x7f3d140b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:17.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1415f280 con 0x7f3d14140fa0 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f3d04001670 con 0x7f3d14140fa0 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 >> v1:172.21.15.172:6789/0 conn(0x7f3d14140fa0 legacy=0x7f3d14162840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 --2- 172.21.15.172:0/1080124542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d000039b0 0x7f3d00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 --2- 172.21.15.172:0/1080124542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 0x7f3d00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 --2- 172.21.15.172:0/1080124542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d000051c0 0x7f3d0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1415efa0 con 0x7f3d00004800 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0000c160 con 0x7f3d000039b0 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.944+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0000c360 con 0x7f3d000051c0 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.945+0000 7f3d0bfff640 1 --2- 172.21.15.172:0/1080124542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 0x7f3d00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.945+0000 7f3d12ffd640 1 --2- 172.21.15.172:0/1080124542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d000039b0 0x7f3d00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.945+0000 7f3d137fe640 1 --2- 172.21.15.172:0/1080124542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d000051c0 0x7f3d0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.945+0000 7f3d0bfff640 1 -- 172.21.15.172:0/1080124542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d000051c0 msgr2=0x7f3d0000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:17.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.945+0000 7f3d0bfff640 1 --2- 172.21.15.172:0/1080124542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d000051c0 0x7f3d0000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.945+0000 7f3d0bfff640 1 -- 172.21.15.172:0/1080124542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d000039b0 msgr2=0x7f3d00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.945+0000 7f3d0bfff640 1 --2- 172.21.15.172:0/1080124542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d000039b0 0x7f3d00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.945+0000 7f3d0bfff640 1 -- 172.21.15.172:0/1080124542 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d00003520 con 0x7f3d00004800 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.945+0000 7f3d137fe640 1 --2- 172.21.15.172:0/1080124542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d000051c0 0x7f3d0000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.945+0000 7f3d137fe640 1 -- 172.21.15.172:0/1080124542 reap_dead start 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.946+0000 7f3d0bfff640 1 --2- 172.21.15.172:0/1080124542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 0x7f3d00004c80 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc001720 tx=0x7f3d00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48407dbfe334412 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.946+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc001370 con 0x7f3d00004800 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.946+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cfc0052d0 con 0x7f3d00004800 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.946+0000 7f3d127fc640 1 -- 172.21.15.172:0/1080124542 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc003830 con 0x7f3d00004800 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.946+0000 7f3d19288640 1 -- 172.21.15.172:0/1080124542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 msgr2=0x7f3d00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.946+0000 7f3d19288640 1 --2- 172.21.15.172:0/1080124542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 0x7f3d00004c80 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc001720 tx=0x7f3d00005830 comp rx=0 tx=0).stop 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.946+0000 7f3d19288640 1 -- 172.21.15.172:0/1080124542 shutdown_connections 2024-04-14T08:43:17.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.946+0000 7f3d19288640 1 --2- 172.21.15.172:0/1080124542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 0x7f3d00004c80 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.946+0000 7f3d19288640 1 -- 172.21.15.172:0/1080124542 >> 172.21.15.172:0/1080124542 conn(0x7f3d1406b520 msgr2=0x7f3d1406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.946+0000 7f3d19288640 1 -- 172.21.15.172:0/1080124542 shutdown_connections 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.946+0000 7f3d19288640 1 -- 172.21.15.172:0/1080124542 wait complete. 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.948+0000 7f3d19288640 1 Processor -- start 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.948+0000 7f3d19288640 1 -- start start 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.949+0000 7f3d19288640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 0x7f3d14141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.949+0000 7f3d19288640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d14159eb0 0x7f3d1415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.949+0000 7f3d19288640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d141f6940 0x7f3d14160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.949+0000 7f3d19288640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1415e890 con 0x7f3d00004800 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.949+0000 7f3d19288640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3d140af3e0 con 0x7f3d141f6940 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.949+0000 7f3d19288640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1415ea10 con 0x7f3d14159eb0 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.949+0000 7f3d12ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 0x7f3d14141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.949+0000 7f3d12ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 0x7f3d14141840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46952/0 (socket says 172.21.15.172:46952) 2024-04-14T08:43:17.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d12ffd640 1 -- 172.21.15.172:0/667687001 learned_addr learned my addr 172.21.15.172:0/667687001 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d137fe640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d141f6940 0x7f3d14160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d0bfff640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d14159eb0 0x7f3d1415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d12ffd640 1 -- 172.21.15.172:0/667687001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d14159eb0 msgr2=0x7f3d1415b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d12ffd640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d14159eb0 0x7f3d1415b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d12ffd640 1 -- 172.21.15.172:0/667687001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d141f6940 msgr2=0x7f3d14160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d12ffd640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d141f6940 0x7f3d14160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d12ffd640 1 -- 172.21.15.172:0/667687001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1415bdd0 con 0x7f3d00004800 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d0bfff640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d14159eb0 0x7f3d1415b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d12ffd640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 0x7f3d14141840 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004f20 tx=0x7f3d040074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d0b7fe640 1 -- 172.21.15.172:0/667687001 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0400f7b0 con 0x7f3d00004800 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d0b7fe640 1 -- 172.21.15.172:0/667687001 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d04018070 con 0x7f3d00004800 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d19288640 1 -- 172.21.15.172:0/667687001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1415a330 con 0x7f3d00004800 2024-04-14T08:43:17.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d0b7fe640 1 -- 172.21.15.172:0/667687001 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0400fbb0 con 0x7f3d00004800 2024-04-14T08:43:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.950+0000 7f3d19288640 1 -- 172.21.15.172:0/667687001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d1415c330 con 0x7f3d00004800 2024-04-14T08:43:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.951+0000 7f3d19288640 1 -- 172.21.15.172:0/667687001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d14068590 con 0x7f3d00004800 2024-04-14T08:43:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.952+0000 7f3d0b7fe640 1 -- 172.21.15.172:0/667687001 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d0400fd50 con 0x7f3d00004800 2024-04-14T08:43:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.952+0000 7f3d0b7fe640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3cf406c920 0x7f3cf406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.952+0000 7f3d0b7fe640 1 -- 172.21.15.172:0/667687001 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7f3d0408c720 con 0x7f3d00004800 2024-04-14T08:43:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.952+0000 7f3d0bfff640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3cf406c920 0x7f3cf406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:17.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.953+0000 7f3d0bfff640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3cf406c920 0x7f3cf406ede0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc033000 tx=0x7f3cfc009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:17.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:17.960+0000 7f3d0b7fe640 1 -- 172.21.15.172:0/667687001 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3d1415bdd0 con 0x7f3d00004800 2024-04-14T08:43:18.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.140+0000 7f3d19288640 1 -- 172.21.15.172:0/667687001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3d1406c530 con 0x7f3d00004800 2024-04-14T08:43:18.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.140+0000 7f3d0b7fe640 1 -- 172.21.15.172:0/667687001 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7f3d0405aa00 con 0x7f3d00004800 2024-04-14T08:43:18.217 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:43:18.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.142+0000 7f3d19288640 1 -- 172.21.15.172:0/667687001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3cf406c920 msgr2=0x7f3cf406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.142+0000 7f3d19288640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3cf406c920 0x7f3cf406ede0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc033000 tx=0x7f3cfc009030 comp rx=0 tx=0).stop 2024-04-14T08:43:18.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.142+0000 7f3d19288640 1 -- 172.21.15.172:0/667687001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 msgr2=0x7f3d14141840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.142+0000 7f3d19288640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 0x7f3d14141840 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004f20 tx=0x7f3d040074f0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.142+0000 7f3d19288640 1 -- 172.21.15.172:0/667687001 shutdown_connections 2024-04-14T08:43:18.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.142+0000 7f3d19288640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f3cf406c920 0x7f3cf406ede0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.142+0000 7f3d19288640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d141f6940 0x7f3d14160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.142+0000 7f3d19288640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d14159eb0 0x7f3d1415b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.142+0000 7f3d19288640 1 --2- 172.21.15.172:0/667687001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d00004800 0x7f3d14141840 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.142+0000 7f3d19288640 1 -- 172.21.15.172:0/667687001 >> 172.21.15.172:0/667687001 conn(0x7f3d1406b520 msgr2=0x7f3d1415f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:18.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.142+0000 7f3d19288640 1 -- 172.21.15.172:0/667687001 shutdown_connections 2024-04-14T08:43:18.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.142+0000 7f3d19288640 1 -- 172.21.15.172:0/667687001 wait complete. 2024-04-14T08:43:18.221 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:43:18.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.326+0000 7fa430e36640 1 Processor -- start 2024-04-14T08:43:18.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.326+0000 7fa430e36640 1 -- start start 2024-04-14T08:43:18.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.327+0000 7fa430e36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa42c15c420 0x7fa42c0b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.327+0000 7fa430e36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa42c06f3e0 0x7fa42c0d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.327+0000 7fa430e36640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa42c0d0550 con 0x7fa42c0ceb00 2024-04-14T08:43:18.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.327+0000 7fa430e36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fa42c0d30a0 con 0x7fa42c15c420 2024-04-14T08:43:18.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.327+0000 7fa430e36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa42c06f870 con 0x7fa42c06f3e0 2024-04-14T08:43:18.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.327+0000 7fa42a59c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa42c15c420 0x7fa42c0b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.327+0000 7fa42ad9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa42c0ceb00 0x7fa42c0cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54970/0 (socket says 172.21.15.172:54970) 2024-04-14T08:43:18.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.327+0000 7fa42ad9d640 1 -- 172.21.15.172:0/1102919987 learned_addr learned my addr 172.21.15.172:0/1102919987 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:18.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.327+0000 7fa421d9b640 1 --2- 172.21.15.172:0/1102919987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa42c06f3e0 0x7fa42c0d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.327+0000 7fa42a59c640 1 -- 172.21.15.172:0/1102919987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa42c06f3e0 msgr2=0x7fa42c0d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:18.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.328+0000 7fa42a59c640 1 --2- 172.21.15.172:0/1102919987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa42c06f3e0 0x7fa42c0d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.327+0000 7fa429d9b640 1 -- 172.21.15.172:0/1102919987 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3325692800 0 0) 0x7fa42c0d0550 con 0x7fa42c0ceb00 2024-04-14T08:43:18.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.328+0000 7fa42a59c640 1 -- 172.21.15.172:0/1102919987 >> v1:172.21.15.172:6789/0 conn(0x7fa42c0ceb00 legacy=0x7fa42c0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.328+0000 7fa42a59c640 1 -- 172.21.15.172:0/1102919987 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c0ccdd0 con 0x7fa42c15c420 2024-04-14T08:43:18.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.328+0000 7fa421d9b640 1 --2- 172.21.15.172:0/1102919987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa42c06f3e0 0x7fa42c0d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:18.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.328+0000 7fa42a59c640 1 --2- 172.21.15.172:0/1102919987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa42c15c420 0x7fa42c0b1510 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0097a0 tx=0x7fa41c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630c0d544c4f841f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:18.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.328+0000 7fa429d9b640 1 -- 172.21.15.172:0/1102919987 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c035020 con 0x7fa42c15c420 2024-04-14T08:43:18.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.328+0000 7fa429d9b640 1 -- 172.21.15.172:0/1102919987 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41c031830 con 0x7fa42c15c420 2024-04-14T08:43:18.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.328+0000 7fa429d9b640 1 -- 172.21.15.172:0/1102919987 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c031c00 con 0x7fa42c15c420 2024-04-14T08:43:18.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.329+0000 7fa430e36640 1 -- 172.21.15.172:0/1102919987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa42c15c420 msgr2=0x7fa42c0b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.329+0000 7fa430e36640 1 --2- 172.21.15.172:0/1102919987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa42c15c420 0x7fa42c0b1510 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0097a0 tx=0x7fa41c02b8d0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.329+0000 7fa430e36640 1 -- 172.21.15.172:0/1102919987 shutdown_connections 2024-04-14T08:43:18.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.329+0000 7fa430e36640 1 --2- 172.21.15.172:0/1102919987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa42c06f3e0 0x7fa42c0d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.329+0000 7fa430e36640 1 --2- 172.21.15.172:0/1102919987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa42c15c420 0x7fa42c0b1510 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.329+0000 7fa430e36640 1 -- 172.21.15.172:0/1102919987 >> 172.21.15.172:0/1102919987 conn(0x7fa42c08c000 msgr2=0x7fa42c0d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:18.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.329+0000 7fa430e36640 1 -- 172.21.15.172:0/1102919987 shutdown_connections 2024-04-14T08:43:18.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.329+0000 7fa430e36640 1 -- 172.21.15.172:0/1102919987 wait complete. 2024-04-14T08:43:18.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.330+0000 7fa430e36640 1 Processor -- start 2024-04-14T08:43:18.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.331+0000 7fa430e36640 1 -- start start 2024-04-14T08:43:18.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.331+0000 7fa430e36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa42c0ceb00 0x7fa42c1e8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.331+0000 7fa430e36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa42c15c420 0x7fa42c0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.331+0000 7fa430e36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa42c0c5990 0x7fa42c0af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.331+0000 7fa430e36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c0cd070 con 0x7fa42c15c420 2024-04-14T08:43:18.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.331+0000 7fa430e36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c0d2dc0 con 0x7fa42c0c5990 2024-04-14T08:43:18.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.331+0000 7fa430e36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa42c0b11e0 con 0x7fa42c0ceb00 2024-04-14T08:43:18.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.331+0000 7fa421d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa42c15c420 0x7fa42c0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.331+0000 7fa421d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa42c15c420 0x7fa42c0c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46966/0 (socket says 172.21.15.172:46966) 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.331+0000 7fa421d9b640 1 -- 172.21.15.172:0/1237860343 learned_addr learned my addr 172.21.15.172:0/1237860343 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.331+0000 7fa42a59c640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa42c0ceb00 0x7fa42c1e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.332+0000 7fa42ad9d640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa42c0c5990 0x7fa42c0af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.332+0000 7fa421d9b640 1 -- 172.21.15.172:0/1237860343 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa42c0ceb00 msgr2=0x7fa42c1e8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.332+0000 7fa421d9b640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa42c0ceb00 0x7fa42c1e8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.332+0000 7fa421d9b640 1 -- 172.21.15.172:0/1237860343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa42c0c5990 msgr2=0x7fa42c0af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.332+0000 7fa421d9b640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa42c0c5990 0x7fa42c0af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.332+0000 7fa421d9b640 1 -- 172.21.15.172:0/1237860343 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c0c6170 con 0x7fa42c15c420 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.332+0000 7fa421d9b640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa42c15c420 0x7fa42c0c7ab0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fa41400e730 tx=0x7fa41400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.332+0000 7fa4237fe640 1 -- 172.21.15.172:0/1237860343 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41400f040 con 0x7fa42c15c420 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.332+0000 7fa4237fe640 1 -- 172.21.15.172:0/1237860343 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa414004230 con 0x7fa42c15c420 2024-04-14T08:43:18.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.332+0000 7fa430e36640 1 -- 172.21.15.172:0/1237860343 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42c0c6450 con 0x7fa42c15c420 2024-04-14T08:43:18.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.332+0000 7fa4237fe640 1 -- 172.21.15.172:0/1237860343 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41400ee40 con 0x7fa42c15c420 2024-04-14T08:43:18.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.332+0000 7fa430e36640 1 -- 172.21.15.172:0/1237860343 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42c0c6a10 con 0x7fa42c15c420 2024-04-14T08:43:18.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.333+0000 7fa430e36640 1 -- 172.21.15.172:0/1237860343 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42c075490 con 0x7fa42c15c420 2024-04-14T08:43:18.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.334+0000 7fa4237fe640 1 -- 172.21.15.172:0/1237860343 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa41400c6d0 con 0x7fa42c15c420 2024-04-14T08:43:18.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.334+0000 7fa4237fe640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa40c06c8f0 0x7fa40c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.334+0000 7fa4237fe640 1 -- 172.21.15.172:0/1237860343 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7fa41408b9b0 con 0x7fa42c15c420 2024-04-14T08:43:18.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.334+0000 7fa42a59c640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa40c06c8f0 0x7fa40c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.335+0000 7fa42a59c640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa40c06c8f0 0x7fa40c06edb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa41c036000 tx=0x7fa41c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:18.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.338+0000 7fa4237fe640 1 -- 172.21.15.172:0/1237860343 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa414017020 con 0x7fa42c15c420 2024-04-14T08:43:18.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.519+0000 7fa430e36640 1 -- 172.21.15.172:0/1237860343 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa42c0705c0 con 0x7fa42c15c420 2024-04-14T08:43:18.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.519+0000 7fa4237fe640 1 -- 172.21.15.172:0/1237860343 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7fa4140595e0 con 0x7fa42c15c420 2024-04-14T08:43:18.519 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:43:18.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.521+0000 7fa430e36640 1 -- 172.21.15.172:0/1237860343 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa40c06c8f0 msgr2=0x7fa40c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.521+0000 7fa430e36640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa40c06c8f0 0x7fa40c06edb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa41c036000 tx=0x7fa41c008030 comp rx=0 tx=0).stop 2024-04-14T08:43:18.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.521+0000 7fa430e36640 1 -- 172.21.15.172:0/1237860343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa42c15c420 msgr2=0x7fa42c0c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.521+0000 7fa430e36640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa42c15c420 0x7fa42c0c7ab0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fa41400e730 tx=0x7fa41400ebf0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.521+0000 7fa430e36640 1 -- 172.21.15.172:0/1237860343 shutdown_connections 2024-04-14T08:43:18.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.521+0000 7fa430e36640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa40c06c8f0 0x7fa40c06edb0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.521+0000 7fa430e36640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa42c0c5990 0x7fa42c0af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.521+0000 7fa430e36640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa42c15c420 0x7fa42c0c7ab0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.521+0000 7fa430e36640 1 --2- 172.21.15.172:0/1237860343 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa42c0ceb00 0x7fa42c1e8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.521+0000 7fa430e36640 1 -- 172.21.15.172:0/1237860343 >> 172.21.15.172:0/1237860343 conn(0x7fa42c08c000 msgr2=0x7fa42c15b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:18.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.521+0000 7fa430e36640 1 -- 172.21.15.172:0/1237860343 shutdown_connections 2024-04-14T08:43:18.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.521+0000 7fa430e36640 1 -- 172.21.15.172:0/1237860343 wait complete. 2024-04-14T08:43:18.532 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:43:18.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.637+0000 7f7fd1898640 1 Processor -- start 2024-04-14T08:43:18.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.637+0000 7f7fd1898640 1 -- start start 2024-04-14T08:43:18.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fd1898640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7fcc0b3dc0 0x7f7fcc06b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fd1898640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fcc06ba60 0x7f7fcc1683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fd1898640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fcc15edd0 con 0x7f7fcc15f460 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fd1898640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f7fcc141960 con 0x7f7fcc0b3dc0 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fd1898640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7fcc06c100 con 0x7f7fcc06ba60 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fca7fc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7fcc0b3dc0 0x7f7fcc06b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fcb7fe640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fcc06ba60 0x7f7fcc1683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fca7fc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7fcc0b3dc0 0x7f7fcc06b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:59768/0 (socket says 172.21.15.172:59768) 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fcaffd640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f7fcc15f460 0x7f7fcc0b36b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:59940/0 (socket says 172.21.15.172:59940) 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fca7fc640 1 -- 172.21.15.172:0/369172557 learned_addr learned my addr 172.21.15.172:0/369172557 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fcb7fe640 1 -- 172.21.15.172:0/369172557 >> v2:172.21.15.177:3300/0 conn(0x7f7fcc0b3dc0 msgr2=0x7f7fcc06b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fcb7fe640 1 --2- 172.21.15.172:0/369172557 >> v2:172.21.15.177:3300/0 conn(0x7f7fcc0b3dc0 0x7f7fcc06b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fcb7fe640 1 -- 172.21.15.172:0/369172557 >> v1:172.21.15.177:6789/0 conn(0x7f7fcc15f460 legacy=0x7f7fcc0b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.638+0000 7f7fcb7fe640 1 -- 172.21.15.172:0/369172557 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc06c3e0 con 0x7f7fcc06ba60 2024-04-14T08:43:18.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.639+0000 7f7fcb7fe640 1 --2- 172.21.15.172:0/369172557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fcc06ba60 0x7f7fcc1683c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009d90 tx=0x7f7fc402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98f5528ac029659f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:18.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.639+0000 7f7fc9ffb640 1 -- 172.21.15.172:0/369172557 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4039070 con 0x7f7fcc06ba60 2024-04-14T08:43:18.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.639+0000 7f7fc9ffb640 1 -- 172.21.15.172:0/369172557 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc402cbc0 con 0x7f7fcc06ba60 2024-04-14T08:43:18.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.639+0000 7f7fc9ffb640 1 -- 172.21.15.172:0/369172557 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4034440 con 0x7f7fcc06ba60 2024-04-14T08:43:18.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.640+0000 7f7fd1898640 1 -- 172.21.15.172:0/369172557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fcc06ba60 msgr2=0x7f7fcc1683c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.640+0000 7f7fd1898640 1 --2- 172.21.15.172:0/369172557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fcc06ba60 0x7f7fcc1683c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009d90 tx=0x7f7fc402be10 comp rx=0 tx=0).stop 2024-04-14T08:43:18.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.640+0000 7f7fd1898640 1 -- 172.21.15.172:0/369172557 shutdown_connections 2024-04-14T08:43:18.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.640+0000 7f7fd1898640 1 --2- 172.21.15.172:0/369172557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fcc06ba60 0x7f7fcc1683c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.640+0000 7f7fd1898640 1 --2- 172.21.15.172:0/369172557 >> v2:172.21.15.177:3300/0 conn(0x7f7fcc0b3dc0 0x7f7fcc06b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.640+0000 7f7fd1898640 1 -- 172.21.15.172:0/369172557 >> 172.21.15.172:0/369172557 conn(0x7f7fcc0d0780 msgr2=0x7f7fcc0d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:18.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.640+0000 7f7fd1898640 1 -- 172.21.15.172:0/369172557 shutdown_connections 2024-04-14T08:43:18.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.640+0000 7f7fd1898640 1 -- 172.21.15.172:0/369172557 wait complete. 2024-04-14T08:43:18.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.641+0000 7f7fd1898640 1 Processor -- start 2024-04-14T08:43:18.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.641+0000 7f7fd1898640 1 -- start start 2024-04-14T08:43:18.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fd1898640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc06ba60 0x7f7fcc153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fd1898640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fcc0b3dc0 0x7f7fcc1f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fd1898640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fcc15f460 0x7f7fcc1f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fd1898640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc15f840 con 0x7f7fcc06ba60 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fd1898640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc06c9a0 con 0x7f7fcc15f460 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fd1898640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7fcc15f9c0 con 0x7f7fcc0b3dc0 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fcaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc06ba60 0x7f7fcc153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fcaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc06ba60 0x7f7fcc153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46974/0 (socket says 172.21.15.172:46974) 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fcaffd640 1 -- 172.21.15.172:0/2361206942 learned_addr learned my addr 172.21.15.172:0/2361206942 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fca7fc640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fcc0b3dc0 0x7f7fcc1f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fcb7fe640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fcc15f460 0x7f7fcc1f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fcaffd640 1 -- 172.21.15.172:0/2361206942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fcc0b3dc0 msgr2=0x7f7fcc1f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fcaffd640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fcc0b3dc0 0x7f7fcc1f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fcaffd640 1 -- 172.21.15.172:0/2361206942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fcc15f460 msgr2=0x7f7fcc1f7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fcaffd640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fcc15f460 0x7f7fcc1f7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.642+0000 7f7fcaffd640 1 -- 172.21.15.172:0/2361206942 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc15fbe0 con 0x7f7fcc06ba60 2024-04-14T08:43:18.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.643+0000 7f7fcaffd640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc06ba60 0x7f7fcc153c00 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00c430 tx=0x7f7fbc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:18.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.643+0000 7f7fb3fff640 1 -- 172.21.15.172:0/2361206942 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0074e0 con 0x7f7fcc06ba60 2024-04-14T08:43:18.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.643+0000 7f7fb3fff640 1 -- 172.21.15.172:0/2361206942 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fbc007680 con 0x7f7fcc06ba60 2024-04-14T08:43:18.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.643+0000 7f7fd1898640 1 -- 172.21.15.172:0/2361206942 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fcc162400 con 0x7f7fcc06ba60 2024-04-14T08:43:18.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.643+0000 7f7fb3fff640 1 -- 172.21.15.172:0/2361206942 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc004a00 con 0x7f7fcc06ba60 2024-04-14T08:43:18.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.643+0000 7f7fd1898640 1 -- 172.21.15.172:0/2361206942 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fcc15fe50 con 0x7f7fcc06ba60 2024-04-14T08:43:18.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.644+0000 7f7fd1898640 1 -- 172.21.15.172:0/2361206942 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fcc162a80 con 0x7f7fcc06ba60 2024-04-14T08:43:18.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.644+0000 7f7fb3fff640 1 -- 172.21.15.172:0/2361206942 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fbc01a020 con 0x7f7fcc06ba60 2024-04-14T08:43:18.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.645+0000 7f7fb3fff640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7fb806c8f0 0x7f7fb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.645+0000 7f7fca7fc640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7fb806c8f0 0x7f7fb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.645+0000 7f7fb3fff640 1 -- 172.21.15.172:0/2361206942 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7f7fbc08b5f0 con 0x7f7fcc06ba60 2024-04-14T08:43:18.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.645+0000 7f7fca7fc640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7fb806c8f0 0x7f7fb806edb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc155a80 tx=0x7f7fc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:18.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.649+0000 7f7fb3fff640 1 -- 172.21.15.172:0/2361206942 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7fbc059760 con 0x7f7fcc06ba60 2024-04-14T08:43:18.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.826+0000 7f7fd1898640 1 -- 172.21.15.172:0/2361206942 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7fcc160020 con 0x7f7fcc06ba60 2024-04-14T08:43:18.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.827+0000 7f7fb3fff640 1 -- 172.21.15.172:0/2361206942 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7f7fbc059060 con 0x7f7fcc06ba60 2024-04-14T08:43:18.827 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:43:18.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.829+0000 7f7fd1898640 1 -- 172.21.15.172:0/2361206942 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7fb806c8f0 msgr2=0x7f7fb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.829+0000 7f7fd1898640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7fb806c8f0 0x7f7fb806edb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc155a80 tx=0x7f7fc0008040 comp rx=0 tx=0).stop 2024-04-14T08:43:18.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.829+0000 7f7fd1898640 1 -- 172.21.15.172:0/2361206942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc06ba60 msgr2=0x7f7fcc153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.829+0000 7f7fd1898640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc06ba60 0x7f7fcc153c00 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00c430 tx=0x7f7fbc00c8f0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.829+0000 7f7fd1898640 1 -- 172.21.15.172:0/2361206942 shutdown_connections 2024-04-14T08:43:18.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.829+0000 7f7fd1898640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f7fb806c8f0 0x7f7fb806edb0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.829+0000 7f7fd1898640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fcc15f460 0x7f7fcc1f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.829+0000 7f7fd1898640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fcc0b3dc0 0x7f7fcc1f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.829+0000 7f7fd1898640 1 --2- 172.21.15.172:0/2361206942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc06ba60 0x7f7fcc153c00 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.829+0000 7f7fd1898640 1 -- 172.21.15.172:0/2361206942 >> 172.21.15.172:0/2361206942 conn(0x7f7fcc0d0780 msgr2=0x7f7fcc15ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:18.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.829+0000 7f7fd1898640 1 -- 172.21.15.172:0/2361206942 shutdown_connections 2024-04-14T08:43:18.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.829+0000 7f7fd1898640 1 -- 172.21.15.172:0/2361206942 wait complete. 2024-04-14T08:43:18.840 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-14T08:43:18.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.942+0000 7ff6d3dcc640 1 Processor -- start 2024-04-14T08:43:18.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.942+0000 7ff6d3dcc640 1 -- start start 2024-04-14T08:43:18.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.942+0000 7ff6d3dcc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6cc13df30 0x7ff6cc0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.942+0000 7ff6d3dcc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff6cc15c380 0x7ff6cc0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.942+0000 7ff6d3dcc640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6cc15b620 con 0x7ff6cc140770 2024-04-14T08:43:18.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d3dcc640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7ff6cc0cf9b0 con 0x7ff6cc15c380 2024-04-14T08:43:18.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d3dcc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff6cc1414a0 con 0x7ff6cc13df30 2024-04-14T08:43:18.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d1367640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff6cc15c380 0x7ff6cc0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d2369640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff6cc140770 0x7ff6cc140c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:54986/0 (socket says 172.21.15.172:54986) 2024-04-14T08:43:18.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d1367640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff6cc15c380 0x7ff6cc0b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46988/0 (socket says 172.21.15.172:46988) 2024-04-14T08:43:18.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d2369640 1 -- 172.21.15.172:0/2598795154 learned_addr learned my addr 172.21.15.172:0/2598795154 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:18.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d1b68640 1 --2- 172.21.15.172:0/2598795154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6cc13df30 0x7ff6cc0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d1367640 1 -- 172.21.15.172:0/2598795154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6cc13df30 msgr2=0x7ff6cc0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:18.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d1367640 1 --2- 172.21.15.172:0/2598795154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6cc13df30 0x7ff6cc0b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d1367640 1 -- 172.21.15.172:0/2598795154 >> v1:172.21.15.172:6789/0 conn(0x7ff6cc140770 legacy=0x7ff6cc140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d0b66640 1 -- 172.21.15.172:0/2598795154 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367061248 0 0) 0x7ff6c8001110 con 0x7ff6cc140770 2024-04-14T08:43:18.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d1367640 1 -- 172.21.15.172:0/2598795154 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6cc15b8a0 con 0x7ff6cc15c380 2024-04-14T08:43:18.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.943+0000 7ff6d1b68640 1 --2- 172.21.15.172:0/2598795154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6cc13df30 0x7ff6cc0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:18.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.944+0000 7ff6d1367640 1 --2- 172.21.15.172:0/2598795154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc15c380 0x7ff6cc0b2210 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0097d0 tx=0x7ff6bc02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f9f7d1a21273fa server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:18.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.944+0000 7ff6d0b66640 1 -- 172.21.15.172:0/2598795154 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc035020 con 0x7ff6cc15c380 2024-04-14T08:43:18.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.944+0000 7ff6d0b66640 1 -- 172.21.15.172:0/2598795154 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6bc031800 con 0x7ff6cc15c380 2024-04-14T08:43:18.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.944+0000 7ff6d0b66640 1 -- 172.21.15.172:0/2598795154 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc031bd0 con 0x7ff6cc15c380 2024-04-14T08:43:18.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.944+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/2598795154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc15c380 msgr2=0x7ff6cc0b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.944+0000 7ff6d3dcc640 1 --2- 172.21.15.172:0/2598795154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc15c380 0x7ff6cc0b2210 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0097d0 tx=0x7ff6bc02b8a0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.944+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/2598795154 shutdown_connections 2024-04-14T08:43:18.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.944+0000 7ff6d3dcc640 1 --2- 172.21.15.172:0/2598795154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc15c380 0x7ff6cc0b2210 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.944+0000 7ff6d3dcc640 1 --2- 172.21.15.172:0/2598795154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6cc13df30 0x7ff6cc0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.944+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/2598795154 >> 172.21.15.172:0/2598795154 conn(0x7ff6cc06b520 msgr2=0x7ff6cc06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:18.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.945+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/2598795154 shutdown_connections 2024-04-14T08:43:18.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.945+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/2598795154 wait complete. 2024-04-14T08:43:18.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.946+0000 7ff6d3dcc640 1 Processor -- start 2024-04-14T08:43:18.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.946+0000 7ff6d3dcc640 1 -- start start 2024-04-14T08:43:18.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.946+0000 7ff6d3dcc640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6cc13df30 0x7ff6cc137a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.946+0000 7ff6d3dcc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc140770 0x7ff6cc154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.946+0000 7ff6d3dcc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6cc15c380 0x7ff6cc15a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.946+0000 7ff6d3dcc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc13e310 con 0x7ff6cc140770 2024-04-14T08:43:18.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.946+0000 7ff6d3dcc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff6cc15be60 con 0x7ff6cc15c380 2024-04-14T08:43:18.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.946+0000 7ff6d3dcc640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff6cc0ce9a0 con 0x7ff6cc13df30 2024-04-14T08:43:18.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.946+0000 7ff6d1367640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc140770 0x7ff6cc154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6d1367640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc140770 0x7ff6cc154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46998/0 (socket says 172.21.15.172:46998) 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6d1367640 1 -- 172.21.15.172:0/265061902 learned_addr learned my addr 172.21.15.172:0/265061902 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6d1b68640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6cc13df30 0x7ff6cc137a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6d2369640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6cc15c380 0x7ff6cc15a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6d1367640 1 -- 172.21.15.172:0/265061902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6cc13df30 msgr2=0x7ff6cc137a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6d1367640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6cc13df30 0x7ff6cc137a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6d1367640 1 -- 172.21.15.172:0/265061902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6cc15c380 msgr2=0x7ff6cc15a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6d1367640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6cc15c380 0x7ff6cc15a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6d1367640 1 -- 172.21.15.172:0/265061902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6cc154a70 con 0x7ff6cc140770 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6d1367640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc140770 0x7ff6cc154180 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0097d0 tx=0x7ff6bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6c27fc640 1 -- 172.21.15.172:0/265061902 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc041070 con 0x7ff6cc140770 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6c27fc640 1 -- 172.21.15.172:0/265061902 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6bc03ad00 con 0x7ff6cc140770 2024-04-14T08:43:18.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6c27fc640 1 -- 172.21.15.172:0/265061902 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc035040 con 0x7ff6cc140770 2024-04-14T08:43:18.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/265061902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6cc154d50 con 0x7ff6cc140770 2024-04-14T08:43:18.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.947+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/265061902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6cc138460 con 0x7ff6cc140770 2024-04-14T08:43:18.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.948+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/265061902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6cc158fb0 con 0x7ff6cc140770 2024-04-14T08:43:18.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.949+0000 7ff6c27fc640 1 -- 172.21.15.172:0/265061902 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6bc038020 con 0x7ff6cc140770 2024-04-14T08:43:18.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.949+0000 7ff6c27fc640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff6b006c8f0 0x7ff6b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:18.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.949+0000 7ff6c27fc640 1 -- 172.21.15.172:0/265061902 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7ff6bc0ada80 con 0x7ff6cc140770 2024-04-14T08:43:18.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.949+0000 7ff6d1b68640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff6b006c8f0 0x7ff6b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:18.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.949+0000 7ff6d1b68640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff6b006c8f0 0x7ff6b006edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40075b0 tx=0x7ff6c4007540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:18.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:18.958+0000 7ff6c27fc640 1 -- 172.21.15.172:0/265061902 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff6cc158fb0 con 0x7ff6cc140770 2024-04-14T08:43:19.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.151+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/265061902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff6cc0b6060 con 0x7ff6cc140770 2024-04-14T08:43:19.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.151+0000 7ff6c27fc640 1 -- 172.21.15.172:0/265061902 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v118) ==== 80+0+246 (secure 0 0 0) 0x7ff6cc0b6060 con 0x7ff6cc140770 2024-04-14T08:43:19.152 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6837/744853572 2024-04-15T08:43:08.711642+0000 2024-04-14T08:43:19.152 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.177:6835/3201353747 2024-04-15T08:42:57.915856+0000 2024-04-14T08:43:19.152 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6835/744853572 2024-04-15T08:43:08.711642+0000 2024-04-14T08:43:19.152 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.177:6833/3201353747 2024-04-15T08:42:57.915856+0000 2024-04-14T08:43:19.152 INFO:teuthology.orchestra.run.smithi172.stderr:listed 4 entries 2024-04-14T08:43:19.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.153+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/265061902 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff6b006c8f0 msgr2=0x7ff6b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:19.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.153+0000 7ff6d3dcc640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff6b006c8f0 0x7ff6b006edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40075b0 tx=0x7ff6c4007540 comp rx=0 tx=0).stop 2024-04-14T08:43:19.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.153+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/265061902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc140770 msgr2=0x7ff6cc154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:19.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.153+0000 7ff6d3dcc640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc140770 0x7ff6cc154180 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0097d0 tx=0x7ff6bc0046c0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.154+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/265061902 shutdown_connections 2024-04-14T08:43:19.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.154+0000 7ff6d3dcc640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff6b006c8f0 0x7ff6b006edb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.154+0000 7ff6d3dcc640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6cc15c380 0x7ff6cc15a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.154+0000 7ff6d3dcc640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc140770 0x7ff6cc154180 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.154+0000 7ff6d3dcc640 1 --2- 172.21.15.172:0/265061902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff6cc13df30 0x7ff6cc137a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.154+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/265061902 >> 172.21.15.172:0/265061902 conn(0x7ff6cc06b520 msgr2=0x7ff6cc140c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:19.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.154+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/265061902 shutdown_connections 2024-04-14T08:43:19.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.154+0000 7ff6d3dcc640 1 -- 172.21.15.172:0/265061902 wait complete. 2024-04-14T08:43:19.165 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-14T08:43:19.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:19.165 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-14T08:43:19.195 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:43:19.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:19.196 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-14T08:43:19.251 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:43:19.251 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:43:19.251 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:19.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:43:19.272 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:43:19.272 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:43:19.272 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:19.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:43:19.306 INFO:tasks.cephfs.mount:Terminating background process 2024-04-14T08:43:19.307 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:43:19.307 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:19.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:43:19.331 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:43:19.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.449+0000 7fac836a2640 1 Processor -- start 2024-04-14T08:43:19.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.449+0000 7fac836a2640 1 -- start start 2024-04-14T08:43:19.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.450+0000 7fac836a2640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fac7c15e640 0x7fac7c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.450+0000 7fac836a2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c0b0630 0x7fac7c06b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.450+0000 7fac836a2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fac7c0c5db0 0x7fac7c06ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.450+0000 7fac836a2640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fac7c15ea20 con 0x7fac7c0c5db0 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.450+0000 7fac836a2640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fac7c0d1af0 con 0x7fac7c15e640 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.450+0000 7fac836a2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fac7c0c7840 con 0x7fac7c0b0630 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.450+0000 7fac81c3f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fac7c0c5db0 0x7fac7c06ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.450+0000 7fac81c3f640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fac7c0c5db0 0x7fac7c06ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47006/0 (socket says 172.21.15.172:47006) 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.450+0000 7fac81c3f640 1 -- 172.21.15.172:0/3908722275 learned_addr learned my addr 172.21.15.172:0/3908722275 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.450+0000 7fac8143e640 1 --2- 172.21.15.172:0/3908722275 >> v2:172.21.15.177:3300/0 conn(0x7fac7c15e640 0x7fac7c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.450+0000 7fac80c3d640 1 --2- 172.21.15.172:0/3908722275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c0b0630 0x7fac7c06b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.451+0000 7fac81c3f640 1 -- 172.21.15.172:0/3908722275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c0b0630 msgr2=0x7fac7c06b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.451+0000 7fac81c3f640 1 --2- 172.21.15.172:0/3908722275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c0b0630 0x7fac7c06b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.451+0000 7fac81c3f640 1 -- 172.21.15.172:0/3908722275 >> v2:172.21.15.177:3300/0 conn(0x7fac7c15e640 msgr2=0x7fac7c0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.451+0000 7fac81c3f640 1 --2- 172.21.15.172:0/3908722275 >> v2:172.21.15.177:3300/0 conn(0x7fac7c15e640 0x7fac7c0cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.451+0000 7fac81c3f640 1 -- 172.21.15.172:0/3908722275 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7c0c7b20 con 0x7fac7c0c5db0 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.451+0000 7fac8143e640 1 --2- 172.21.15.172:0/3908722275 >> v2:172.21.15.177:3300/0 conn(0x7fac7c15e640 0x7fac7c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:19.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.451+0000 7fac80c3d640 1 --2- 172.21.15.172:0/3908722275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c0b0630 0x7fac7c06b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.451+0000 7fac81c3f640 1 --2- 172.21.15.172:0/3908722275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7c0c5db0 0x7fac7c06ba60 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fac7c074130 tx=0x7fac7402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b836fbe662ce8e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.451+0000 7fac73fff640 1 -- 172.21.15.172:0/3908722275 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7402c9e0 con 0x7fac7c0c5db0 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.451+0000 7fac73fff640 1 -- 172.21.15.172:0/3908722275 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7402cb80 con 0x7fac7c0c5db0 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.451+0000 7fac73fff640 1 -- 172.21.15.172:0/3908722275 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac740343f0 con 0x7fac7c0c5db0 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.452+0000 7fac836a2640 1 -- 172.21.15.172:0/3908722275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7c0c5db0 msgr2=0x7fac7c06ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.452+0000 7fac836a2640 1 --2- 172.21.15.172:0/3908722275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7c0c5db0 0x7fac7c06ba60 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fac7c074130 tx=0x7fac7402be40 comp rx=0 tx=0).stop 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.452+0000 7fac836a2640 1 -- 172.21.15.172:0/3908722275 shutdown_connections 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.452+0000 7fac836a2640 1 --2- 172.21.15.172:0/3908722275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7c0c5db0 0x7fac7c06ba60 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.452+0000 7fac836a2640 1 --2- 172.21.15.172:0/3908722275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c0b0630 0x7fac7c06b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.452+0000 7fac836a2640 1 --2- 172.21.15.172:0/3908722275 >> v2:172.21.15.177:3300/0 conn(0x7fac7c15e640 0x7fac7c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.452+0000 7fac836a2640 1 -- 172.21.15.172:0/3908722275 >> 172.21.15.172:0/3908722275 conn(0x7fac7c0d0f30 msgr2=0x7fac7c0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.452+0000 7fac836a2640 1 -- 172.21.15.172:0/3908722275 shutdown_connections 2024-04-14T08:43:19.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.452+0000 7fac836a2640 1 -- 172.21.15.172:0/3908722275 wait complete. 2024-04-14T08:43:19.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.453+0000 7fac836a2640 1 Processor -- start 2024-04-14T08:43:19.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.453+0000 7fac836a2640 1 -- start start 2024-04-14T08:43:19.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac836a2640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c0b0630 0x7fac7c0ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac836a2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac7c0c5db0 0x7fac7c0b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac836a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7c15e640 0x7fac7c0cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac836a2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c06c1e0 con 0x7fac7c15e640 2024-04-14T08:43:19.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac836a2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c0d1dd0 con 0x7fac7c0c5db0 2024-04-14T08:43:19.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac836a2640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fac7c162530 con 0x7fac7c0b0630 2024-04-14T08:43:19.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac81c3f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7c15e640 0x7fac7c0cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac81c3f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7c15e640 0x7fac7c0cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47014/0 (socket says 172.21.15.172:47014) 2024-04-14T08:43:19.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac81c3f640 1 -- 172.21.15.172:0/1895354286 learned_addr learned my addr 172.21.15.172:0/1895354286 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:19.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac80c3d640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac7c0c5db0 0x7fac7c0b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac8143e640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c0b0630 0x7fac7c0ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac81c3f640 1 -- 172.21.15.172:0/1895354286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c0b0630 msgr2=0x7fac7c0ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:19.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac81c3f640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c0b0630 0x7fac7c0ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac81c3f640 1 -- 172.21.15.172:0/1895354286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac7c0c5db0 msgr2=0x7fac7c0b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:19.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac81c3f640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac7c0c5db0 0x7fac7c0b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.454+0000 7fac81c3f640 1 -- 172.21.15.172:0/1895354286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7c0ca9a0 con 0x7fac7c15e640 2024-04-14T08:43:19.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.455+0000 7fac81c3f640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7c15e640 0x7fac7c0cd990 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fac74036040 tx=0x7fac74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:19.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.455+0000 7fac71ffb640 1 -- 172.21.15.172:0/1895354286 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74043020 con 0x7fac7c15e640 2024-04-14T08:43:19.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.455+0000 7fac71ffb640 1 -- 172.21.15.172:0/1895354286 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7403e450 con 0x7fac7c15e640 2024-04-14T08:43:19.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.455+0000 7fac836a2640 1 -- 172.21.15.172:0/1895354286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7c0c93c0 con 0x7fac7c15e640 2024-04-14T08:43:19.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.455+0000 7fac71ffb640 1 -- 172.21.15.172:0/1895354286 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7403e860 con 0x7fac7c15e640 2024-04-14T08:43:19.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.455+0000 7fac836a2640 1 -- 172.21.15.172:0/1895354286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac7c0c9c60 con 0x7fac7c15e640 2024-04-14T08:43:19.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.456+0000 7fac836a2640 1 -- 172.21.15.172:0/1895354286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac7c075490 con 0x7fac7c15e640 2024-04-14T08:43:19.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.457+0000 7fac71ffb640 1 -- 172.21.15.172:0/1895354286 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac74046020 con 0x7fac7c15e640 2024-04-14T08:43:19.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.457+0000 7fac71ffb640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fac6006c8f0 0x7fac6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.457+0000 7fac71ffb640 1 -- 172.21.15.172:0/1895354286 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7fac740ae700 con 0x7fac7c15e640 2024-04-14T08:43:19.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.457+0000 7fac8143e640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fac6006c8f0 0x7fac6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.457+0000 7fac8143e640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fac6006c8f0 0x7fac6006edb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00a950 tx=0x7fac6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:19.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.466+0000 7fac71ffb640 1 -- 172.21.15.172:0/1895354286 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fac7404d030 con 0x7fac7c15e640 2024-04-14T08:43:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.686+0000 7fac836a2640 1 -- 172.21.15.172:0/1895354286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac7c0b2750 con 0x7fac7c15e640 2024-04-14T08:43:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.687+0000 7fac71ffb640 1 -- 172.21.15.172:0/1895354286 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5199 (secure 0 0 0) 0x7fac7407c8f0 con 0x7fac7c15e640 2024-04-14T08:43:19.687 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:19.687 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3881291102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3881291102},{"type":"v1","addr":"172.21.15.177:6835","nonce":3881291102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:43:13.745318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9519":{"gid":9519,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.172:6837/4065123823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4065123823},{"type":"v1","addr":"172.21.15.172:6837","nonce":4065123823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9519,"qdb_cluster":[9519]},"id":8}]} 2024-04-14T08:43:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 100 2024-04-14T08:43:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.689+0000 7fac836a2640 1 -- 172.21.15.172:0/1895354286 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fac6006c8f0 msgr2=0x7fac6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.689+0000 7fac836a2640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fac6006c8f0 0x7fac6006edb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00a950 tx=0x7fac6c008040 comp rx=0 tx=0).stop 2024-04-14T08:43:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.689+0000 7fac836a2640 1 -- 172.21.15.172:0/1895354286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7c15e640 msgr2=0x7fac7c0cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.689+0000 7fac836a2640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7c15e640 0x7fac7c0cd990 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fac74036040 tx=0x7fac74004a40 comp rx=0 tx=0).stop 2024-04-14T08:43:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.689+0000 7fac836a2640 1 -- 172.21.15.172:0/1895354286 shutdown_connections 2024-04-14T08:43:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.689+0000 7fac836a2640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fac6006c8f0 0x7fac6006edb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.689+0000 7fac836a2640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac7c15e640 0x7fac7c0cd990 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.689+0000 7fac836a2640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac7c0c5db0 0x7fac7c0b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.689+0000 7fac836a2640 1 --2- 172.21.15.172:0/1895354286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c0b0630 0x7fac7c0ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.689+0000 7fac836a2640 1 -- 172.21.15.172:0/1895354286 >> 172.21.15.172:0/1895354286 conn(0x7fac7c0d0f30 msgr2=0x7fac7c0af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.689+0000 7fac836a2640 1 -- 172.21.15.172:0/1895354286 shutdown_connections 2024-04-14T08:43:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.689+0000 7fac836a2640 1 -- 172.21.15.172:0/1895354286 wait complete. 2024-04-14T08:43:19.700 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:43:19.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.802+0000 7f2cb142d640 1 Processor -- start 2024-04-14T08:43:19.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.802+0000 7f2cb142d640 1 -- start start 2024-04-14T08:43:19.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2cb142d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2cac0d2830 0x7f2cac0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2cb142d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2cac15e890 0x7f2cac0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2cb142d640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cac0d3150 con 0x7f2cac0cedd0 2024-04-14T08:43:19.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2cb142d640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f2cac0cff30 con 0x7f2cac0d2830 2024-04-14T08:43:19.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2cb142d640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2cac0d2350 con 0x7f2cac15e890 2024-04-14T08:43:19.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2caaffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2cac0d2830 0x7f2cac0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2cab7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2cac0cedd0 0x7f2cac1621a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:51896/0 (socket says 172.21.15.172:51896) 2024-04-14T08:43:19.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2cab7fe640 1 -- 172.21.15.172:0/1213697555 learned_addr learned my addr 172.21.15.172:0/1213697555 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:19.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2caa7fc640 1 --2- 172.21.15.172:0/1213697555 >> v2:172.21.15.177:3300/0 conn(0x7f2cac15e890 0x7f2cac0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2caaffd640 1 -- 172.21.15.172:0/1213697555 >> v2:172.21.15.177:3300/0 conn(0x7f2cac15e890 msgr2=0x7f2cac0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2caaffd640 1 --2- 172.21.15.172:0/1213697555 >> v2:172.21.15.177:3300/0 conn(0x7f2cac15e890 0x7f2cac0d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2caaffd640 1 -- 172.21.15.172:0/1213697555 >> v1:172.21.15.172:6789/0 conn(0x7f2cac0cedd0 legacy=0x7f2cac1621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.803+0000 7f2caaffd640 1 -- 172.21.15.172:0/1213697555 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac0cf320 con 0x7f2cac0d2830 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.805+0000 7f2caaffd640 1 --2- 172.21.15.172:0/1213697555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cac0d2830 0x7f2cac0d2c10 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2cac162100 tx=0x7f2ca002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c28f565396a4eca server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.805+0000 7f2ca9ffb640 1 -- 172.21.15.172:0/1213697555 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0034070 con 0x7f2cac0d2830 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.805+0000 7f2ca9ffb640 1 -- 172.21.15.172:0/1213697555 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca0030610 con 0x7f2cac0d2830 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.805+0000 7f2ca9ffb640 1 -- 172.21.15.172:0/1213697555 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00309e0 con 0x7f2cac0d2830 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.805+0000 7f2cb142d640 1 -- 172.21.15.172:0/1213697555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cac0d2830 msgr2=0x7f2cac0d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.805+0000 7f2cb142d640 1 --2- 172.21.15.172:0/1213697555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cac0d2830 0x7f2cac0d2c10 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2cac162100 tx=0x7f2ca002b960 comp rx=0 tx=0).stop 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.805+0000 7f2cb142d640 1 -- 172.21.15.172:0/1213697555 shutdown_connections 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.805+0000 7f2cb142d640 1 --2- 172.21.15.172:0/1213697555 >> v2:172.21.15.177:3300/0 conn(0x7f2cac15e890 0x7f2cac0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.805+0000 7f2cb142d640 1 --2- 172.21.15.172:0/1213697555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cac0d2830 0x7f2cac0d2c10 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.805+0000 7f2cb142d640 1 -- 172.21.15.172:0/1213697555 >> 172.21.15.172:0/1213697555 conn(0x7f2cac09d590 msgr2=0x7f2cac0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:19.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.805+0000 7f2cb142d640 1 -- 172.21.15.172:0/1213697555 shutdown_connections 2024-04-14T08:43:19.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.805+0000 7f2cb142d640 1 -- 172.21.15.172:0/1213697555 wait complete. 2024-04-14T08:43:19.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.807+0000 7f2cb142d640 1 Processor -- start 2024-04-14T08:43:19.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.807+0000 7f2cb142d640 1 -- start start 2024-04-14T08:43:19.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cb142d640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2cac0cedd0 0x7f2cac156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cb142d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cac0d2830 0x7f2cac0b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cb142d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cac15e890 0x7f2cac0b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cb142d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac0cfbc0 con 0x7f2cac15e890 2024-04-14T08:43:19.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cb142d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac0cf8e0 con 0x7f2cac0d2830 2024-04-14T08:43:19.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cb142d640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2cac0d1840 con 0x7f2cac0cedd0 2024-04-14T08:43:19.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cac15e890 0x7f2cac0b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cac15e890 0x7f2cac0b5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47756/0 (socket says 172.21.15.172:47756) 2024-04-14T08:43:19.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cab7fe640 1 -- 172.21.15.172:0/1249595874 learned_addr learned my addr 172.21.15.172:0/1249595874 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2caa7fc640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cac0d2830 0x7f2cac0b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cab7fe640 1 -- 172.21.15.172:0/1249595874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2cac0cedd0 msgr2=0x7f2cac156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2caaffd640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2cac0cedd0 0x7f2cac156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cab7fe640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2cac0cedd0 0x7f2cac156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cab7fe640 1 -- 172.21.15.172:0/1249595874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cac0d2830 msgr2=0x7f2cac0b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cab7fe640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cac0d2830 0x7f2cac0b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cab7fe640 1 -- 172.21.15.172:0/1249595874 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac0cb650 con 0x7f2cac15e890 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2caa7fc640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cac0d2830 0x7f2cac0b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2caaffd640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2cac0cedd0 0x7f2cac156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cab7fe640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cac15e890 0x7f2cac0b5e00 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400ca80 tx=0x7f2ca400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2c937fe640 1 -- 172.21.15.172:0/1249595874 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4017070 con 0x7f2cac15e890 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2c937fe640 1 -- 172.21.15.172:0/1249595874 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca400ac30 con 0x7f2cac15e890 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cb142d640 1 -- 172.21.15.172:0/1249595874 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac0cd400 con 0x7f2cac15e890 2024-04-14T08:43:19.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2c937fe640 1 -- 172.21.15.172:0/1249595874 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4012600 con 0x7f2cac15e890 2024-04-14T08:43:19.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.809+0000 7f2cb142d640 1 -- 172.21.15.172:0/1249595874 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac0cd9c0 con 0x7f2cac15e890 2024-04-14T08:43:19.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.810+0000 7f2cb142d640 1 -- 172.21.15.172:0/1249595874 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac068520 con 0x7f2cac15e890 2024-04-14T08:43:19.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.811+0000 7f2c937fe640 1 -- 172.21.15.172:0/1249595874 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ca401a070 con 0x7f2cac15e890 2024-04-14T08:43:19.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.812+0000 7f2c937fe640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c9806c8f0 0x7f2c9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:19.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.812+0000 7f2caaffd640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c9806c8f0 0x7f2c9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:19.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.812+0000 7f2c937fe640 1 -- 172.21.15.172:0/1249595874 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7f2ca408b8b0 con 0x7f2cac15e890 2024-04-14T08:43:19.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.812+0000 7f2caaffd640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c9806c8f0 0x7f2c9806edb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0036000 tx=0x7f2ca0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:19.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:19.816+0000 7f2c937fe640 1 -- 172.21.15.172:0/1249595874 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2cac0cb650 con 0x7f2cac15e890 2024-04-14T08:43:20.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.039+0000 7f2cb142d640 1 -- 172.21.15.172:0/1249595874 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cac0d4960 con 0x7f2cac15e890 2024-04-14T08:43:20.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.041+0000 7f2c937fe640 1 -- 172.21.15.172:0/1249595874 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+17273 (secure 0 0 0) 0x7f2ca400ae20 con 0x7f2cac15e890 2024-04-14T08:43:20.041 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:20.041 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":118,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:43:08.711666+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:42:38.455900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-14T08:42:39.198157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/744853572":"2024-04-15T08:43:08.711642+0000","172.21.15.177:6835/3201353747":"2024-04-15T08:42:57.915856+0000","172.21.15.172:6835/744853572":"2024-04-15T08:43:08.711642+0000","172.21.15.177:6833/3201353747":"2024-04-15T08:42:57.915856+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:43:20.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.043+0000 7f2cb142d640 1 -- 172.21.15.172:0/1249595874 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c9806c8f0 msgr2=0x7f2c9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.043+0000 7f2cb142d640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c9806c8f0 0x7f2c9806edb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0036000 tx=0x7f2ca0008030 comp rx=0 tx=0).stop 2024-04-14T08:43:20.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.043+0000 7f2cb142d640 1 -- 172.21.15.172:0/1249595874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cac15e890 msgr2=0x7f2cac0b5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.043+0000 7f2cb142d640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cac15e890 0x7f2cac0b5e00 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400ca80 tx=0x7f2ca400cf40 comp rx=0 tx=0).stop 2024-04-14T08:43:20.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.043+0000 7f2cb142d640 1 -- 172.21.15.172:0/1249595874 shutdown_connections 2024-04-14T08:43:20.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.043+0000 7f2cb142d640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2c9806c8f0 0x7f2c9806edb0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.043+0000 7f2cb142d640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cac15e890 0x7f2cac0b5e00 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.043+0000 7f2cb142d640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2cac0d2830 0x7f2cac0b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.043+0000 7f2cb142d640 1 --2- 172.21.15.172:0/1249595874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2cac0cedd0 0x7f2cac156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.043+0000 7f2cb142d640 1 -- 172.21.15.172:0/1249595874 >> 172.21.15.172:0/1249595874 conn(0x7f2cac09d590 msgr2=0x7f2cac0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:20.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.043+0000 7f2cb142d640 1 -- 172.21.15.172:0/1249595874 shutdown_connections 2024-04-14T08:43:20.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.043+0000 7f2cb142d640 1 -- 172.21.15.172:0/1249595874 wait complete. 2024-04-14T08:43:20.054 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T08:32:41.135898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-14T08:42:38.455900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-14T08:42:39.198157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T08:43:20.055 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-14T08:43:20.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.160+0000 7fb809084640 1 Processor -- start 2024-04-14T08:43:20.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.160+0000 7fb809084640 1 -- start start 2024-04-14T08:43:20.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.160+0000 7fb809084640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8040ccca0 0x7fb80406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.160+0000 7fb809084640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80406bc30 0x7fb8040b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.160+0000 7fb809084640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8040ceee0 con 0x7fb804073090 2024-04-14T08:43:20.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.160+0000 7fb809084640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb804073520 con 0x7fb8040ccca0 2024-04-14T08:43:20.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.160+0000 7fb809084640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb804070150 con 0x7fb80406bc30 2024-04-14T08:43:20.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.161+0000 7fb802d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8040ccca0 0x7fb80406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.161+0000 7fb802d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8040ccca0 0x7fb80406fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47768/0 (socket says 172.21.15.172:47768) 2024-04-14T08:43:20.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.161+0000 7fb802d9d640 1 -- 172.21.15.172:0/4049971336 learned_addr learned my addr 172.21.15.172:0/4049971336 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:20.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.161+0000 7fb80359e640 1 --2- 172.21.15.172:0/4049971336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80406bc30 0x7fb8040b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.161+0000 7fb802d9d640 1 -- 172.21.15.172:0/4049971336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80406bc30 msgr2=0x7fb8040b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:20.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.161+0000 7fb802d9d640 1 --2- 172.21.15.172:0/4049971336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80406bc30 0x7fb8040b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.161+0000 7fb802d9d640 1 -- 172.21.15.172:0/4049971336 >> v1:172.21.15.177:6789/0 conn(0x7fb804073090 legacy=0x7fb80406b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:20.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.161+0000 7fb802d9d640 1 -- 172.21.15.172:0/4049971336 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8040722d0 con 0x7fb8040ccca0 2024-04-14T08:43:20.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.161+0000 7fb80359e640 1 --2- 172.21.15.172:0/4049971336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80406bc30 0x7fb8040b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:20.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.162+0000 7fb802d9d640 1 --2- 172.21.15.172:0/4049971336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8040ccca0 0x7fb80406fb70 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009e50 tx=0x7fb7fc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73885b36ac190001 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:20.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.162+0000 7fb801d9b640 1 -- 172.21.15.172:0/4049971336 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc02c9e0 con 0x7fb8040ccca0 2024-04-14T08:43:20.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.162+0000 7fb801d9b640 1 -- 172.21.15.172:0/4049971336 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7fc02cb80 con 0x7fb8040ccca0 2024-04-14T08:43:20.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.162+0000 7fb801d9b640 1 -- 172.21.15.172:0/4049971336 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0343f0 con 0x7fb8040ccca0 2024-04-14T08:43:20.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.163+0000 7fb809084640 1 -- 172.21.15.172:0/4049971336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8040ccca0 msgr2=0x7fb80406fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.163+0000 7fb809084640 1 --2- 172.21.15.172:0/4049971336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8040ccca0 0x7fb80406fb70 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009e50 tx=0x7fb7fc02be40 comp rx=0 tx=0).stop 2024-04-14T08:43:20.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.163+0000 7fb809084640 1 -- 172.21.15.172:0/4049971336 shutdown_connections 2024-04-14T08:43:20.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.163+0000 7fb809084640 1 --2- 172.21.15.172:0/4049971336 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb80406bc30 0x7fb8040b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.163+0000 7fb809084640 1 --2- 172.21.15.172:0/4049971336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8040ccca0 0x7fb80406fb70 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.163+0000 7fb809084640 1 -- 172.21.15.172:0/4049971336 >> 172.21.15.172:0/4049971336 conn(0x7fb8040cda00 msgr2=0x7fb8040cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:20.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.163+0000 7fb809084640 1 -- 172.21.15.172:0/4049971336 shutdown_connections 2024-04-14T08:43:20.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.163+0000 7fb809084640 1 -- 172.21.15.172:0/4049971336 wait complete. 2024-04-14T08:43:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.164+0000 7fb809084640 1 Processor -- start 2024-04-14T08:43:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.165+0000 7fb809084640 1 -- start start 2024-04-14T08:43:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.165+0000 7fb809084640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80406bc30 0x7fb80415bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.165+0000 7fb809084640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb804073090 0x7fb8040b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.165+0000 7fb809084640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8040ccca0 0x7fb8040ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.165+0000 7fb809084640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8040cd080 con 0x7fb8040ccca0 2024-04-14T08:43:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.165+0000 7fb809084640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb804072890 con 0x7fb80406bc30 2024-04-14T08:43:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.165+0000 7fb809084640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb80406e150 con 0x7fb804073090 2024-04-14T08:43:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.165+0000 7fb80359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8040ccca0 0x7fb8040ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.165+0000 7fb80359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8040ccca0 0x7fb8040ae910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47772/0 (socket says 172.21.15.172:47772) 2024-04-14T08:43:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.165+0000 7fb80359e640 1 -- 172.21.15.172:0/325698769 learned_addr learned my addr 172.21.15.172:0/325698769 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.165+0000 7fb802d9d640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80406bc30 0x7fb80415bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.165+0000 7fb80259c640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb804073090 0x7fb8040b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.166+0000 7fb80359e640 1 -- 172.21.15.172:0/325698769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb804073090 msgr2=0x7fb8040b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.166+0000 7fb80359e640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb804073090 0x7fb8040b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.166+0000 7fb80359e640 1 -- 172.21.15.172:0/325698769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80406bc30 msgr2=0x7fb80415bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.166+0000 7fb80359e640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80406bc30 0x7fb80415bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.166+0000 7fb80359e640 1 -- 172.21.15.172:0/325698769 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8040ca920 con 0x7fb8040ccca0 2024-04-14T08:43:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.166+0000 7fb80359e640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8040ccca0 0x7fb8040ae910 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c7f0 tx=0x7fb7f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.166+0000 7fb7fb7fe640 1 -- 172.21.15.172:0/325698769 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4015070 con 0x7fb8040ccca0 2024-04-14T08:43:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.166+0000 7fb7fb7fe640 1 -- 172.21.15.172:0/325698769 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f4007820 con 0x7fb8040ccca0 2024-04-14T08:43:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.166+0000 7fb809084640 1 -- 172.21.15.172:0/325698769 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8040cac00 con 0x7fb8040ccca0 2024-04-14T08:43:20.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.166+0000 7fb7fb7fe640 1 -- 172.21.15.172:0/325698769 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4007c50 con 0x7fb8040ccca0 2024-04-14T08:43:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.166+0000 7fb809084640 1 -- 172.21.15.172:0/325698769 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8040cb4a0 con 0x7fb8040ccca0 2024-04-14T08:43:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.167+0000 7fb809084640 1 -- 172.21.15.172:0/325698769 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8040cb6a0 con 0x7fb8040ccca0 2024-04-14T08:43:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.168+0000 7fb7fb7fe640 1 -- 172.21.15.172:0/325698769 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7f4018070 con 0x7fb8040ccca0 2024-04-14T08:43:20.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.168+0000 7fb7fb7fe640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7e406c920 0x7fb7e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.168+0000 7fb7fb7fe640 1 -- 172.21.15.172:0/325698769 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7fb7f408be40 con 0x7fb8040ccca0 2024-04-14T08:43:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.168+0000 7fb802d9d640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7e406c920 0x7fb7e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.169+0000 7fb802d9d640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7e406c920 0x7fb7e406ede0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc035000 tx=0x7fb7fc030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:20.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.172+0000 7fb7fb7fe640 1 -- 172.21.15.172:0/325698769 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb8040cb6a0 con 0x7fb8040ccca0 2024-04-14T08:43:20.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.475+0000 7fb809084640 1 -- 172.21.15.172:0/325698769 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb804068590 con 0x7fb8040ccca0 2024-04-14T08:43:20.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.476+0000 7fb7fb7fe640 1 -- 172.21.15.172:0/325698769 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7fb7f40599e0 con 0x7fb8040ccca0 2024-04-14T08:43:20.476 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-04-14T08:43:20.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.478+0000 7fb809084640 1 -- 172.21.15.172:0/325698769 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7e406c920 msgr2=0x7fb7e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.478+0000 7fb809084640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7e406c920 0x7fb7e406ede0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc035000 tx=0x7fb7fc030000 comp rx=0 tx=0).stop 2024-04-14T08:43:20.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.478+0000 7fb809084640 1 -- 172.21.15.172:0/325698769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8040ccca0 msgr2=0x7fb8040ae910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.478+0000 7fb809084640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8040ccca0 0x7fb8040ae910 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c7f0 tx=0x7fb7f400ccb0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.478+0000 7fb809084640 1 -- 172.21.15.172:0/325698769 shutdown_connections 2024-04-14T08:43:20.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.478+0000 7fb809084640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb7e406c920 0x7fb7e406ede0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.478+0000 7fb809084640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8040ccca0 0x7fb8040ae910 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.478+0000 7fb809084640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb804073090 0x7fb8040b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.478+0000 7fb809084640 1 --2- 172.21.15.172:0/325698769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80406bc30 0x7fb80415bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.478+0000 7fb809084640 1 -- 172.21.15.172:0/325698769 >> 172.21.15.172:0/325698769 conn(0x7fb8040cda00 msgr2=0x7fb8040c6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:20.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.478+0000 7fb809084640 1 -- 172.21.15.172:0/325698769 shutdown_connections 2024-04-14T08:43:20.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.478+0000 7fb809084640 1 -- 172.21.15.172:0/325698769 wait complete. 2024-04-14T08:43:20.489 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-14T08:43:20.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.594+0000 7f2b252de640 1 Processor -- start 2024-04-14T08:43:20.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.594+0000 7f2b252de640 1 -- start start 2024-04-14T08:43:20.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.595+0000 7f2b252de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b200d1a20 0x7f2b2006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.595+0000 7f2b252de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2b2006d000 0x7f2b2006d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.595+0000 7f2b252de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b200701b0 con 0x7f2b20071250 2024-04-14T08:43:20.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.595+0000 7f2b252de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2b2006d9d0 con 0x7f2b2006d000 2024-04-14T08:43:20.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.595+0000 7f2b252de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2b20070dc0 con 0x7f2b200d1a20 2024-04-14T08:43:20.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.595+0000 7f2b1effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b200d1a20 0x7f2b2006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.595+0000 7f2b1f7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2b2006d000 0x7f2b2006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.596+0000 7f2b1e7fc640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f2b20071250 0x7f2b200cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:49576/0 (socket says 172.21.15.172:49576) 2024-04-14T08:43:20.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.596+0000 7f2b1f7fe640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2b2006d000 0x7f2b2006d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:47576/0 (socket says 172.21.15.172:47576) 2024-04-14T08:43:20.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.596+0000 7f2b1effd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b200d1a20 0x7f2b2006fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:50194/0 (socket says 172.21.15.172:50194) 2024-04-14T08:43:20.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.596+0000 7f2b1e7fc640 1 -- 172.21.15.172:0/308239111 learned_addr learned my addr 172.21.15.172:0/308239111 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:20.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.596+0000 7f2b1effd640 1 -- 172.21.15.172:0/308239111 >> v2:172.21.15.177:3300/0 conn(0x7f2b2006d000 msgr2=0x7f2b2006d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:20.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.596+0000 7f2b1effd640 1 --2- 172.21.15.172:0/308239111 >> v2:172.21.15.177:3300/0 conn(0x7f2b2006d000 0x7f2b2006d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.596+0000 7f2b1effd640 1 -- 172.21.15.172:0/308239111 >> v1:172.21.15.177:6789/0 conn(0x7f2b20071250 legacy=0x7f2b200cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.596+0000 7f2b1effd640 1 -- 172.21.15.172:0/308239111 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b20070490 con 0x7f2b200d1a20 2024-04-14T08:43:20.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.597+0000 7f2b1effd640 1 --2- 172.21.15.172:0/308239111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b200d1a20 0x7f2b2006fb80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2b100097a0 tx=0x7f2b1002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0b815542822d854 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:20.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.597+0000 7f2b1dffb640 1 -- 172.21.15.172:0/308239111 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b100315a0 con 0x7f2b200d1a20 2024-04-14T08:43:20.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.597+0000 7f2b1dffb640 1 -- 172.21.15.172:0/308239111 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b10037070 con 0x7f2b200d1a20 2024-04-14T08:43:20.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.597+0000 7f2b1dffb640 1 -- 172.21.15.172:0/308239111 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10031960 con 0x7f2b200d1a20 2024-04-14T08:43:20.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.597+0000 7f2b252de640 1 -- 172.21.15.172:0/308239111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b200d1a20 msgr2=0x7f2b2006fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.597+0000 7f2b252de640 1 --2- 172.21.15.172:0/308239111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b200d1a20 0x7f2b2006fb80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2b100097a0 tx=0x7f2b1002b840 comp rx=0 tx=0).stop 2024-04-14T08:43:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.597+0000 7f2b252de640 1 -- 172.21.15.172:0/308239111 shutdown_connections 2024-04-14T08:43:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.597+0000 7f2b252de640 1 --2- 172.21.15.172:0/308239111 >> v2:172.21.15.177:3300/0 conn(0x7f2b2006d000 0x7f2b2006d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.597+0000 7f2b252de640 1 --2- 172.21.15.172:0/308239111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b200d1a20 0x7f2b2006fb80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.597+0000 7f2b252de640 1 -- 172.21.15.172:0/308239111 >> 172.21.15.172:0/308239111 conn(0x7f2b2008c000 msgr2=0x7f2b200d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.597+0000 7f2b252de640 1 -- 172.21.15.172:0/308239111 shutdown_connections 2024-04-14T08:43:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.597+0000 7f2b252de640 1 -- 172.21.15.172:0/308239111 wait complete. 2024-04-14T08:43:20.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.599+0000 7f2b252de640 1 Processor -- start 2024-04-14T08:43:20.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.599+0000 7f2b252de640 1 -- start start 2024-04-14T08:43:20.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.599+0000 7f2b252de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b2006d000 0x7f2b201e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.599+0000 7f2b252de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b20071250 0x7f2b201e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.599+0000 7f2b252de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b200d1a20 0x7f2b200b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.599+0000 7f2b252de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b200d1e00 con 0x7f2b200d1a20 2024-04-14T08:43:20.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.599+0000 7f2b252de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20070a50 con 0x7f2b20071250 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b252de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2b200cab30 con 0x7f2b2006d000 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b200d1a20 0x7f2b200b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b200d1a20 0x7f2b200b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47776/0 (socket says 172.21.15.172:47776) 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b1f7fe640 1 -- 172.21.15.172:0/4203476823 learned_addr learned my addr 172.21.15.172:0/4203476823 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b1e7fc640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b20071250 0x7f2b201e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b1effd640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b2006d000 0x7f2b201e2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b1f7fe640 1 -- 172.21.15.172:0/4203476823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b2006d000 msgr2=0x7f2b201e2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b1f7fe640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b2006d000 0x7f2b201e2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b1f7fe640 1 -- 172.21.15.172:0/4203476823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b20071250 msgr2=0x7f2b201e4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b1f7fe640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b20071250 0x7f2b201e4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b1f7fe640 1 -- 172.21.15.172:0/4203476823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b200c9f20 con 0x7f2b200d1a20 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b1f7fe640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b200d1a20 0x7f2b200b4fa0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800efc0 tx=0x7f2b1800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:20.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b077fe640 1 -- 172.21.15.172:0/4203476823 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18018020 con 0x7f2b200d1a20 2024-04-14T08:43:20.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b077fe640 1 -- 172.21.15.172:0/4203476823 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b18014480 con 0x7f2b200d1a20 2024-04-14T08:43:20.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b252de640 1 -- 172.21.15.172:0/4203476823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b200ca200 con 0x7f2b200d1a20 2024-04-14T08:43:20.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b077fe640 1 -- 172.21.15.172:0/4203476823 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18014870 con 0x7f2b200d1a20 2024-04-14T08:43:20.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.600+0000 7f2b252de640 1 -- 172.21.15.172:0/4203476823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b200ca7c0 con 0x7f2b200d1a20 2024-04-14T08:43:20.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.601+0000 7f2b252de640 1 -- 172.21.15.172:0/4203476823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b20075490 con 0x7f2b200d1a20 2024-04-14T08:43:20.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.602+0000 7f2b077fe640 1 -- 172.21.15.172:0/4203476823 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b18005d10 con 0x7f2b200d1a20 2024-04-14T08:43:20.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.602+0000 7f2b077fe640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2b0c06c8f0 0x7f2b0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.602+0000 7f2b077fe640 1 -- 172.21.15.172:0/4203476823 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7f2b1808c780 con 0x7f2b200d1a20 2024-04-14T08:43:20.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.602+0000 7f2b1effd640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2b0c06c8f0 0x7f2b0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.603+0000 7f2b1effd640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2b0c06c8f0 0x7f2b0c06edb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2b10036000 tx=0x7f2b1003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.607+0000 7f2b077fe640 1 -- 172.21.15.172:0/4203476823 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2b18022070 con 0x7f2b200d1a20 2024-04-14T08:43:20.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.786+0000 7f2b252de640 1 -- 172.21.15.172:0/4203476823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2b200d2820 con 0x7f2b200d1a20 2024-04-14T08:43:20.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.786+0000 7f2b077fe640 1 -- 172.21.15.172:0/4203476823 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7f2b18010040 con 0x7f2b200d1a20 2024-04-14T08:43:20.787 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-04-14T08:43:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.788+0000 7f2b252de640 1 -- 172.21.15.172:0/4203476823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2b0c06c8f0 msgr2=0x7f2b0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.788+0000 7f2b252de640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2b0c06c8f0 0x7f2b0c06edb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2b10036000 tx=0x7f2b1003d040 comp rx=0 tx=0).stop 2024-04-14T08:43:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.788+0000 7f2b252de640 1 -- 172.21.15.172:0/4203476823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b200d1a20 msgr2=0x7f2b200b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.788+0000 7f2b252de640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b200d1a20 0x7f2b200b4fa0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800efc0 tx=0x7f2b1800c490 comp rx=0 tx=0).stop 2024-04-14T08:43:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.788+0000 7f2b252de640 1 -- 172.21.15.172:0/4203476823 shutdown_connections 2024-04-14T08:43:20.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.788+0000 7f2b252de640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2b0c06c8f0 0x7f2b0c06edb0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.788+0000 7f2b252de640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b200d1a20 0x7f2b200b4fa0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.788+0000 7f2b252de640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b20071250 0x7f2b201e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.788+0000 7f2b252de640 1 --2- 172.21.15.172:0/4203476823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b2006d000 0x7f2b201e2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.788+0000 7f2b252de640 1 -- 172.21.15.172:0/4203476823 >> 172.21.15.172:0/4203476823 conn(0x7f2b2008c000 msgr2=0x7f2b200b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.789+0000 7f2b252de640 1 -- 172.21.15.172:0/4203476823 shutdown_connections 2024-04-14T08:43:20.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.789+0000 7f2b252de640 1 -- 172.21.15.172:0/4203476823 wait complete. 2024-04-14T08:43:20.800 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-14T08:43:20.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.905+0000 7f64cea98640 1 Processor -- start 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.905+0000 7f64cea98640 1 -- start start 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.905+0000 7f64cea98640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f64c815e430 0x7f64c8162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.906+0000 7f64cea98640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f64c8162540 0x7f64c81629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.906+0000 7f64cea98640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64c80cefb0 con 0x7f64c80b3a30 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.906+0000 7f64cea98640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f64c80cf290 con 0x7f64c8162540 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.906+0000 7f64cea98640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f64c80cf570 con 0x7f64c815e430 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.906+0000 7f64c7fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f64c8162540 0x7f64c81629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.906+0000 7f64c7fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f64c8162540 0x7f64c81629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47790/0 (socket says 172.21.15.172:47790) 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.906+0000 7f64c7fff640 1 -- 172.21.15.172:0/181811789 learned_addr learned my addr 172.21.15.172:0/181811789 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.906+0000 7f64cc834640 1 --2- 172.21.15.172:0/181811789 >> v2:172.21.15.177:3300/0 conn(0x7f64c815e430 0x7f64c8162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.906+0000 7f64c7fff640 1 -- 172.21.15.172:0/181811789 >> v2:172.21.15.177:3300/0 conn(0x7f64c815e430 msgr2=0x7f64c8162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.906+0000 7f64c7fff640 1 --2- 172.21.15.172:0/181811789 >> v2:172.21.15.177:3300/0 conn(0x7f64c815e430 0x7f64c8162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.906+0000 7f64c7fff640 1 -- 172.21.15.172:0/181811789 >> v1:172.21.15.177:6789/0 conn(0x7f64c80b3a30 legacy=0x7f64c80b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.906+0000 7f64c7fff640 1 -- 172.21.15.172:0/181811789 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c80cf850 con 0x7f64c8162540 2024-04-14T08:43:20.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.907+0000 7f64c7fff640 1 --2- 172.21.15.172:0/181811789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64c8162540 0x7f64c81629d0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f64c80b6db0 tx=0x7f64b402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28fdef05cae996f server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:20.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.907+0000 7f64c77fe640 1 -- 172.21.15.172:0/181811789 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4034070 con 0x7f64c8162540 2024-04-14T08:43:20.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.907+0000 7f64c77fe640 1 -- 172.21.15.172:0/181811789 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b4030860 con 0x7f64c8162540 2024-04-14T08:43:20.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.907+0000 7f64c77fe640 1 -- 172.21.15.172:0/181811789 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4030c30 con 0x7f64c8162540 2024-04-14T08:43:20.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.908+0000 7f64cea98640 1 -- 172.21.15.172:0/181811789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64c8162540 msgr2=0x7f64c81629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.908+0000 7f64cea98640 1 --2- 172.21.15.172:0/181811789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64c8162540 0x7f64c81629d0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f64c80b6db0 tx=0x7f64b402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.908+0000 7f64cea98640 1 -- 172.21.15.172:0/181811789 shutdown_connections 2024-04-14T08:43:20.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.908+0000 7f64cea98640 1 --2- 172.21.15.172:0/181811789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64c8162540 0x7f64c81629d0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.908+0000 7f64cea98640 1 --2- 172.21.15.172:0/181811789 >> v2:172.21.15.177:3300/0 conn(0x7f64c815e430 0x7f64c8162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.908+0000 7f64cea98640 1 -- 172.21.15.172:0/181811789 >> 172.21.15.172:0/181811789 conn(0x7f64c809d560 msgr2=0x7f64c80d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:20.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.908+0000 7f64cea98640 1 -- 172.21.15.172:0/181811789 shutdown_connections 2024-04-14T08:43:20.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.908+0000 7f64cea98640 1 -- 172.21.15.172:0/181811789 wait complete. 2024-04-14T08:43:20.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.909+0000 7f64cea98640 1 Processor -- start 2024-04-14T08:43:20.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.909+0000 7f64cea98640 1 -- start start 2024-04-14T08:43:20.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.910+0000 7f64cea98640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64c80b3a30 0x7f64c80b98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.910+0000 7f64cea98640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c815e430 0x7f64c814f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.910+0000 7f64cea98640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64c8162540 0x7f64c80c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.910+0000 7f64cea98640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64c80d4e20 con 0x7f64c80b3a30 2024-04-14T08:43:20.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.910+0000 7f64cea98640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f64c815e810 con 0x7f64c815e430 2024-04-14T08:43:20.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.910+0000 7f64cea98640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f64c80d0050 con 0x7f64c8162540 2024-04-14T08:43:20.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.910+0000 7f64cc834640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64c80b3a30 0x7f64c80b98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.910+0000 7f64cc834640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64c80b3a30 0x7f64c80b98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47800/0 (socket says 172.21.15.172:47800) 2024-04-14T08:43:20.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.910+0000 7f64cc834640 1 -- 172.21.15.172:0/3973160816 learned_addr learned my addr 172.21.15.172:0/3973160816 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:20.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.910+0000 7f64c7fff640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c815e430 0x7f64c814f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.910+0000 7f64cd035640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64c8162540 0x7f64c80c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.911+0000 7f64cc834640 1 -- 172.21.15.172:0/3973160816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64c8162540 msgr2=0x7f64c80c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.911+0000 7f64cc834640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64c8162540 0x7f64c80c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.911+0000 7f64cc834640 1 -- 172.21.15.172:0/3973160816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c815e430 msgr2=0x7f64c814f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:20.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.911+0000 7f64cc834640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c815e430 0x7f64c814f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:20.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.911+0000 7f64cc834640 1 -- 172.21.15.172:0/3973160816 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c814fc40 con 0x7f64c80b3a30 2024-04-14T08:43:20.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.911+0000 7f64cc834640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64c80b3a30 0x7f64c80b98a0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00e9b0 tx=0x7f64bc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:20.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.911+0000 7f64c57fa640 1 -- 172.21.15.172:0/3973160816 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc017070 con 0x7f64c80b3a30 2024-04-14T08:43:20.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.911+0000 7f64c57fa640 1 -- 172.21.15.172:0/3973160816 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64bc00c9a0 con 0x7f64c80b3a30 2024-04-14T08:43:20.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.911+0000 7f64cea98640 1 -- 172.21.15.172:0/3973160816 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c80cc490 con 0x7f64c80b3a30 2024-04-14T08:43:20.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.911+0000 7f64c57fa640 1 -- 172.21.15.172:0/3973160816 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc00cd90 con 0x7f64c80b3a30 2024-04-14T08:43:20.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.911+0000 7f64cea98640 1 -- 172.21.15.172:0/3973160816 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c80cca50 con 0x7f64c80b3a30 2024-04-14T08:43:20.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.912+0000 7f64cea98640 1 -- 172.21.15.172:0/3973160816 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c8068590 con 0x7f64c80b3a30 2024-04-14T08:43:20.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.913+0000 7f64c57fa640 1 -- 172.21.15.172:0/3973160816 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64bc01a070 con 0x7f64c80b3a30 2024-04-14T08:43:20.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.913+0000 7f64c57fa640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64ac06c8f0 0x7f64ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:20.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.913+0000 7f64c57fa640 1 -- 172.21.15.172:0/3973160816 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7f64bc08bcf0 con 0x7f64c80b3a30 2024-04-14T08:43:20.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.913+0000 7f64c7fff640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64ac06c8f0 0x7f64ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:20.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.914+0000 7f64c7fff640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64ac06c8f0 0x7f64ac06edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f64b4036000 tx=0x7f64b403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:20.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:20.918+0000 7f64c57fa640 1 -- 172.21.15.172:0/3973160816 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f64bc090070 con 0x7f64c80b3a30 2024-04-14T08:43:21.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.175+0000 7f64cea98640 1 -- 172.21.15.172:0/3973160816 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f64c80d18a0 con 0x7f64c80b3a30 2024-04-14T08:43:21.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.175+0000 7f64c57fa640 1 -- 172.21.15.172:0/3973160816 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7f64c814fc40 con 0x7f64c80b3a30 2024-04-14T08:43:21.176 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-04-14T08:43:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.177+0000 7f64cea98640 1 -- 172.21.15.172:0/3973160816 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64ac06c8f0 msgr2=0x7f64ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.177+0000 7f64cea98640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64ac06c8f0 0x7f64ac06edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f64b4036000 tx=0x7f64b403b040 comp rx=0 tx=0).stop 2024-04-14T08:43:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.177+0000 7f64cea98640 1 -- 172.21.15.172:0/3973160816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64c80b3a30 msgr2=0x7f64c80b98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.177+0000 7f64cea98640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64c80b3a30 0x7f64c80b98a0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00e9b0 tx=0x7f64bc00ee70 comp rx=0 tx=0).stop 2024-04-14T08:43:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.178+0000 7f64cea98640 1 -- 172.21.15.172:0/3973160816 shutdown_connections 2024-04-14T08:43:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.178+0000 7f64cea98640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f64ac06c8f0 0x7f64ac06edb0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.178+0000 7f64cea98640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64c8162540 0x7f64c80c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.178+0000 7f64cea98640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c815e430 0x7f64c814f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.178+0000 7f64cea98640 1 --2- 172.21.15.172:0/3973160816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64c80b3a30 0x7f64c80b98a0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.178+0000 7f64cea98640 1 -- 172.21.15.172:0/3973160816 >> 172.21.15.172:0/3973160816 conn(0x7f64c809d560 msgr2=0x7f64c80d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:21.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.178+0000 7f64cea98640 1 -- 172.21.15.172:0/3973160816 shutdown_connections 2024-04-14T08:43:21.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.178+0000 7f64cea98640 1 -- 172.21.15.172:0/3973160816 wait complete. 2024-04-14T08:43:21.189 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:43:21.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.291+0000 7fbe548de640 1 Processor -- start 2024-04-14T08:43:21.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.291+0000 7fbe548de640 1 -- start start 2024-04-14T08:43:21.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.292+0000 7fbe548de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbe5015e890 0x7fbe500d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:21.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.292+0000 7fbe548de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe500cf320 con 0x7fbe500d2830 2024-04-14T08:43:21.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.292+0000 7fbe548de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe500cf600 con 0x7fbe500cedd0 2024-04-14T08:43:21.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.292+0000 7fbe548de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fbe500cff30 con 0x7fbe5015e890 2024-04-14T08:43:21.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.292+0000 7fbe4e59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fbe500d2830 0x7fbe500d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:51906/0 (socket says 172.21.15.172:51906) 2024-04-14T08:43:21.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.292+0000 7fbe45d9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbe5015e890 0x7fbe500d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:21.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.292+0000 7fbe4e59c640 1 -- 172.21.15.172:0/752628502 learned_addr learned my addr 172.21.15.172:0/752628502 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:21.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.292+0000 7fbe4dd9b640 1 -- 172.21.15.172:0/752628502 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2736469964 0 0) 0x7fbe500cf320 con 0x7fbe500d2830 2024-04-14T08:43:21.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.292+0000 7fbe4dd9b640 1 -- 172.21.15.172:0/752628502 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe3c003520 con 0x7fbe500d2830 2024-04-14T08:43:21.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.293+0000 7fbe45d9b640 1 -- 172.21.15.172:0/752628502 >> v1:172.21.15.177:6789/0 conn(0x7fbe500cedd0 legacy=0x7fbe500b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:21.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.293+0000 7fbe45d9b640 1 -- 172.21.15.172:0/752628502 >> v1:172.21.15.172:6789/0 conn(0x7fbe500d2830 legacy=0x7fbe500d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.293+0000 7fbe45d9b640 1 -- 172.21.15.172:0/752628502 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe500cf8e0 con 0x7fbe5015e890 2024-04-14T08:43:21.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.293+0000 7fbe45d9b640 1 --2- 172.21.15.172:0/752628502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5015e890 0x7fbe500d1300 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fbe380097a0 tx=0x7fbe3802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5499b46f1beee42 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:21.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.293+0000 7fbe4dd9b640 1 -- 172.21.15.172:0/752628502 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38034070 con 0x7fbe5015e890 2024-04-14T08:43:21.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.294+0000 7fbe4dd9b640 1 -- 172.21.15.172:0/752628502 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe38030610 con 0x7fbe5015e890 2024-04-14T08:43:21.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.294+0000 7fbe4dd9b640 1 -- 172.21.15.172:0/752628502 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe380309c0 con 0x7fbe5015e890 2024-04-14T08:43:21.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.294+0000 7fbe548de640 1 -- 172.21.15.172:0/752628502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5015e890 msgr2=0x7fbe500d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.294+0000 7fbe548de640 1 --2- 172.21.15.172:0/752628502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5015e890 0x7fbe500d1300 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fbe380097a0 tx=0x7fbe3802b960 comp rx=0 tx=0).stop 2024-04-14T08:43:21.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.294+0000 7fbe548de640 1 -- 172.21.15.172:0/752628502 shutdown_connections 2024-04-14T08:43:21.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.294+0000 7fbe548de640 1 --2- 172.21.15.172:0/752628502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5015e890 0x7fbe500d1300 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.294+0000 7fbe548de640 1 -- 172.21.15.172:0/752628502 >> 172.21.15.172:0/752628502 conn(0x7fbe5009d590 msgr2=0x7fbe500d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:21.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.294+0000 7fbe548de640 1 -- 172.21.15.172:0/752628502 shutdown_connections 2024-04-14T08:43:21.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.294+0000 7fbe548de640 1 -- 172.21.15.172:0/752628502 wait complete. 2024-04-14T08:43:21.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.296+0000 7fbe548de640 1 Processor -- start 2024-04-14T08:43:21.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.296+0000 7fbe548de640 1 -- start start 2024-04-14T08:43:21.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.296+0000 7fbe548de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500cedd0 0x7fbe5014c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:21.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.296+0000 7fbe548de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe500d2830 0x7fbe500b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:21.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.296+0000 7fbe548de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe5015e890 0x7fbe500b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:21.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.296+0000 7fbe548de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbe500d1840 con 0x7fbe500cedd0 2024-04-14T08:43:21.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.296+0000 7fbe548de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbe500d2350 con 0x7fbe500d2830 2024-04-14T08:43:21.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.296+0000 7fbe548de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbe500b3e20 con 0x7fbe5015e890 2024-04-14T08:43:21.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.296+0000 7fbe4e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500cedd0 0x7fbe5014c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:21.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.296+0000 7fbe4e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500cedd0 0x7fbe5014c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47822/0 (socket says 172.21.15.172:47822) 2024-04-14T08:43:21.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.296+0000 7fbe4e59c640 1 -- 172.21.15.172:0/4228070176 learned_addr learned my addr 172.21.15.172:0/4228070176 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:21.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe45d9b640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe500d2830 0x7fbe500b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:21.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe4e59c640 1 -- 172.21.15.172:0/4228070176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe5015e890 msgr2=0x7fbe500b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe4ed9d640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe5015e890 0x7fbe500b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:21.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe4e59c640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe5015e890 0x7fbe500b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe4e59c640 1 -- 172.21.15.172:0/4228070176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe500d2830 msgr2=0x7fbe500b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe4e59c640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe500d2830 0x7fbe500b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe4e59c640 1 -- 172.21.15.172:0/4228070176 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5015ff20 con 0x7fbe500cedd0 2024-04-14T08:43:21.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe45d9b640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe500d2830 0x7fbe500b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:21.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe4e59c640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500cedd0 0x7fbe5014c4b0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000c430 tx=0x7fbe4000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:21.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe477fe640 1 -- 172.21.15.172:0/4228070176 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40017070 con 0x7fbe500cedd0 2024-04-14T08:43:21.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe477fe640 1 -- 172.21.15.172:0/4228070176 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe40005ce0 con 0x7fbe500cedd0 2024-04-14T08:43:21.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe548de640 1 -- 172.21.15.172:0/4228070176 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe501f7ff0 con 0x7fbe500cedd0 2024-04-14T08:43:21.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe477fe640 1 -- 172.21.15.172:0/4228070176 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe400072f0 con 0x7fbe500cedd0 2024-04-14T08:43:21.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.297+0000 7fbe548de640 1 -- 172.21.15.172:0/4228070176 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe501f85b0 con 0x7fbe500cedd0 2024-04-14T08:43:21.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.298+0000 7fbe548de640 1 -- 172.21.15.172:0/4228070176 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe50068520 con 0x7fbe500cedd0 2024-04-14T08:43:21.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.299+0000 7fbe477fe640 1 -- 172.21.15.172:0/4228070176 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe4001a070 con 0x7fbe500cedd0 2024-04-14T08:43:21.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.299+0000 7fbe477fe640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbe3006c8f0 0x7fbe3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:21.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.299+0000 7fbe477fe640 1 -- 172.21.15.172:0/4228070176 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7fbe4008b1f0 con 0x7fbe500cedd0 2024-04-14T08:43:21.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.299+0000 7fbe45d9b640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbe3006c8f0 0x7fbe3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:21.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.300+0000 7fbe45d9b640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbe3006c8f0 0x7fbe3006edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbe38036000 tx=0x7fbe38008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:21.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.308+0000 7fbe477fe640 1 -- 172.21.15.172:0/4228070176 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbe400593e0 con 0x7fbe500cedd0 2024-04-14T08:43:21.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.555+0000 7fbe548de640 1 -- 172.21.15.172:0/4228070176 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe500d4960 con 0x7fbe500cedd0 2024-04-14T08:43:21.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.556+0000 7fbe477fe640 1 -- 172.21.15.172:0/4228070176 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5199 (secure 0 0 0) 0x7fbe40058ce0 con 0x7fbe500cedd0 2024-04-14T08:43:21.556 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:21.556 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3881291102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3881291102},{"type":"v1","addr":"172.21.15.177:6835","nonce":3881291102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:43:13.745318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9519":{"gid":9519,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.172:6837/4065123823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4065123823},{"type":"v1","addr":"172.21.15.172:6837","nonce":4065123823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9519,"qdb_cluster":[9519]},"id":8}]} 2024-04-14T08:43:21.556 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 100 2024-04-14T08:43:21.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.558+0000 7fbe548de640 1 -- 172.21.15.172:0/4228070176 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbe3006c8f0 msgr2=0x7fbe3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.558+0000 7fbe548de640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbe3006c8f0 0x7fbe3006edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbe38036000 tx=0x7fbe38008030 comp rx=0 tx=0).stop 2024-04-14T08:43:21.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.558+0000 7fbe548de640 1 -- 172.21.15.172:0/4228070176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500cedd0 msgr2=0x7fbe5014c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.558+0000 7fbe548de640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500cedd0 0x7fbe5014c4b0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000c430 tx=0x7fbe4000c8f0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.558+0000 7fbe548de640 1 -- 172.21.15.172:0/4228070176 shutdown_connections 2024-04-14T08:43:21.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.558+0000 7fbe548de640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbe3006c8f0 0x7fbe3006edb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.558+0000 7fbe548de640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe5015e890 0x7fbe500b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.558+0000 7fbe548de640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe500d2830 0x7fbe500b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.558+0000 7fbe548de640 1 --2- 172.21.15.172:0/4228070176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe500cedd0 0x7fbe5014c4b0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.558+0000 7fbe548de640 1 -- 172.21.15.172:0/4228070176 >> 172.21.15.172:0/4228070176 conn(0x7fbe5009d590 msgr2=0x7fbe5015c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:21.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.558+0000 7fbe548de640 1 -- 172.21.15.172:0/4228070176 shutdown_connections 2024-04-14T08:43:21.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.558+0000 7fbe548de640 1 -- 172.21.15.172:0/4228070176 wait complete. 2024-04-14T08:43:21.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:43:21.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.680+0000 7fb68211e640 1 Processor -- start 2024-04-14T08:43:21.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.680+0000 7fb68211e640 1 -- start start 2024-04-14T08:43:21.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.680+0000 7fb68211e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb67c15c710 0x7fb67c0d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:21.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.680+0000 7fb68211e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb67c0cde10 0x7fb67c0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:21.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.680+0000 7fb68211e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb67c0aef60 con 0x7fb67c0ced80 2024-04-14T08:43:21.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.680+0000 7fb68211e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb67c0cf1f0 con 0x7fb67c0cde10 2024-04-14T08:43:21.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.680+0000 7fb68211e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb67c0aed10 con 0x7fb67c15c710 2024-04-14T08:43:21.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.681+0000 7fb673fff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb67c0cde10 0x7fb67c0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:21.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.681+0000 7fb67bfff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb67c0ced80 0x7fb67c0ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:51916/0 (socket says 172.21.15.172:51916) 2024-04-14T08:43:21.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.681+0000 7fb67bfff640 1 -- 172.21.15.172:0/496790655 learned_addr learned my addr 172.21.15.172:0/496790655 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:21.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.681+0000 7fb67b7fe640 1 --2- 172.21.15.172:0/496790655 >> v2:172.21.15.177:3300/0 conn(0x7fb67c15c710 0x7fb67c0d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:21.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.681+0000 7fb673fff640 1 -- 172.21.15.172:0/496790655 >> v2:172.21.15.177:3300/0 conn(0x7fb67c15c710 msgr2=0x7fb67c0d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:21.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.681+0000 7fb673fff640 1 --2- 172.21.15.172:0/496790655 >> v2:172.21.15.177:3300/0 conn(0x7fb67c15c710 0x7fb67c0d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.681+0000 7fb673fff640 1 -- 172.21.15.172:0/496790655 >> v1:172.21.15.172:6789/0 conn(0x7fb67c0ced80 legacy=0x7fb67c0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.681+0000 7fb673fff640 1 -- 172.21.15.172:0/496790655 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67c0af240 con 0x7fb67c0cde10 2024-04-14T08:43:21.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.681+0000 7fb67affd640 1 -- 172.21.15.172:0/496790655 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 419358202 0 0) 0x7fb67c0aef60 con 0x7fb67c0ced80 2024-04-14T08:43:21.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.682+0000 7fb673fff640 1 --2- 172.21.15.172:0/496790655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb67c0cde10 0x7fb67c0ce840 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fb67c069160 tx=0x7fb66402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275aac83467d9288 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:21.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.682+0000 7fb67affd640 1 -- 172.21.15.172:0/496790655 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb664035020 con 0x7fb67c0cde10 2024-04-14T08:43:21.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.682+0000 7fb67affd640 1 -- 172.21.15.172:0/496790655 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb664031830 con 0x7fb67c0cde10 2024-04-14T08:43:21.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.682+0000 7fb67affd640 1 -- 172.21.15.172:0/496790655 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb664031c00 con 0x7fb67c0cde10 2024-04-14T08:43:21.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.683+0000 7fb68211e640 1 -- 172.21.15.172:0/496790655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb67c0cde10 msgr2=0x7fb67c0ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.683+0000 7fb68211e640 1 --2- 172.21.15.172:0/496790655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb67c0cde10 0x7fb67c0ce840 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fb67c069160 tx=0x7fb66402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.683+0000 7fb68211e640 1 -- 172.21.15.172:0/496790655 shutdown_connections 2024-04-14T08:43:21.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.683+0000 7fb68211e640 1 --2- 172.21.15.172:0/496790655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb67c0cde10 0x7fb67c0ce840 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.683+0000 7fb68211e640 1 --2- 172.21.15.172:0/496790655 >> v2:172.21.15.177:3300/0 conn(0x7fb67c15c710 0x7fb67c0d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.683+0000 7fb68211e640 1 -- 172.21.15.172:0/496790655 >> 172.21.15.172:0/496790655 conn(0x7fb67c08c000 msgr2=0x7fb67c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:21.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.683+0000 7fb68211e640 1 -- 172.21.15.172:0/496790655 shutdown_connections 2024-04-14T08:43:21.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.683+0000 7fb68211e640 1 -- 172.21.15.172:0/496790655 wait complete. 2024-04-14T08:43:21.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.685+0000 7fb68211e640 1 Processor -- start 2024-04-14T08:43:21.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.685+0000 7fb68211e640 1 -- start start 2024-04-14T08:43:21.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb68211e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb67c0cde10 0x7fb67c1f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:21.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb68211e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c0ced80 0x7fb67c157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:21.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb68211e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb67c15c710 0x7fb67c0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:21.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb68211e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c15caf0 con 0x7fb67c0cde10 2024-04-14T08:43:21.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb68211e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c0af890 con 0x7fb67c15c710 2024-04-14T08:43:21.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb68211e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb67c0d0e90 con 0x7fb67c0ced80 2024-04-14T08:43:21.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb67b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb67c0cde10 0x7fb67c1f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb67b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb67c0cde10 0x7fb67c1f58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47840/0 (socket says 172.21.15.172:47840) 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb67b7fe640 1 -- 172.21.15.172:0/2236876773 learned_addr learned my addr 172.21.15.172:0/2236876773 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb67bfff640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb67c15c710 0x7fb67c0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb673fff640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c0ced80 0x7fb67c157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb67b7fe640 1 -- 172.21.15.172:0/2236876773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c0ced80 msgr2=0x7fb67c157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb67b7fe640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c0ced80 0x7fb67c157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb67b7fe640 1 -- 172.21.15.172:0/2236876773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb67c15c710 msgr2=0x7fb67c0c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb67b7fe640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb67c15c710 0x7fb67c0c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb67b7fe640 1 -- 172.21.15.172:0/2236876773 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67c1f5440 con 0x7fb67c0cde10 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb67bfff640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb67c15c710 0x7fb67c0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb673fff640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c0ced80 0x7fb67c157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb67b7fe640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb67c0cde10 0x7fb67c1f58a0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00e870 tx=0x7fb66c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb678ff9640 1 -- 172.21.15.172:0/2236876773 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c017070 con 0x7fb67c0cde10 2024-04-14T08:43:21.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb678ff9640 1 -- 172.21.15.172:0/2236876773 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb66c00c940 con 0x7fb67c0cde10 2024-04-14T08:43:21.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.686+0000 7fb678ff9640 1 -- 172.21.15.172:0/2236876773 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c00cd30 con 0x7fb67c0cde10 2024-04-14T08:43:21.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.687+0000 7fb68211e640 1 -- 172.21.15.172:0/2236876773 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67c06c8b0 con 0x7fb67c0cde10 2024-04-14T08:43:21.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.687+0000 7fb68211e640 1 -- 172.21.15.172:0/2236876773 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb67c06ce70 con 0x7fb67c0cde10 2024-04-14T08:43:21.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.688+0000 7fb68211e640 1 -- 172.21.15.172:0/2236876773 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb67c075490 con 0x7fb67c0cde10 2024-04-14T08:43:21.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.688+0000 7fb678ff9640 1 -- 172.21.15.172:0/2236876773 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb66c01a070 con 0x7fb67c0cde10 2024-04-14T08:43:21.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.688+0000 7fb678ff9640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb65c06c8f0 0x7fb65c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:21.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.689+0000 7fb678ff9640 1 -- 172.21.15.172:0/2236876773 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7fb66c08bff0 con 0x7fb67c0cde10 2024-04-14T08:43:21.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.689+0000 7fb673fff640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb65c06c8f0 0x7fb65c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:21.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.689+0000 7fb673fff640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb65c06c8f0 0x7fb65c06edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb664036000 tx=0x7fb664008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:21.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.693+0000 7fb678ff9640 1 -- 172.21.15.172:0/2236876773 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb66c05a1d0 con 0x7fb67c0cde10 2024-04-14T08:43:21.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.890+0000 7fb68211e640 1 -- 172.21.15.172:0/2236876773 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb67c15d8e0 con 0x7fb67c0cde10 2024-04-14T08:43:21.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.891+0000 7fb678ff9640 1 -- 172.21.15.172:0/2236876773 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+17273 (secure 0 0 0) 0x7fb66c059ad0 con 0x7fb67c0cde10 2024-04-14T08:43:21.891 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:21.892 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":118,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:43:08.711666+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:42:38.455900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-14T08:42:39.198157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/744853572":"2024-04-15T08:43:08.711642+0000","172.21.15.177:6835/3201353747":"2024-04-15T08:42:57.915856+0000","172.21.15.172:6835/744853572":"2024-04-15T08:43:08.711642+0000","172.21.15.177:6833/3201353747":"2024-04-15T08:42:57.915856+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:43:21.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.893+0000 7fb68211e640 1 -- 172.21.15.172:0/2236876773 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb65c06c8f0 msgr2=0x7fb65c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.893+0000 7fb68211e640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb65c06c8f0 0x7fb65c06edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb664036000 tx=0x7fb664008030 comp rx=0 tx=0).stop 2024-04-14T08:43:21.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.893+0000 7fb68211e640 1 -- 172.21.15.172:0/2236876773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb67c0cde10 msgr2=0x7fb67c1f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:21.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.893+0000 7fb68211e640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb67c0cde10 0x7fb67c1f58a0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00e870 tx=0x7fb66c00ed30 comp rx=0 tx=0).stop 2024-04-14T08:43:21.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.893+0000 7fb68211e640 1 -- 172.21.15.172:0/2236876773 shutdown_connections 2024-04-14T08:43:21.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.893+0000 7fb68211e640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb65c06c8f0 0x7fb65c06edb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.893+0000 7fb68211e640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb67c15c710 0x7fb67c0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.893+0000 7fb68211e640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c0ced80 0x7fb67c157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.893+0000 7fb68211e640 1 --2- 172.21.15.172:0/2236876773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb67c0cde10 0x7fb67c1f58a0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:21.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.893+0000 7fb68211e640 1 -- 172.21.15.172:0/2236876773 >> 172.21.15.172:0/2236876773 conn(0x7fb67c08c000 msgr2=0x7fb67c15d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:21.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.893+0000 7fb68211e640 1 -- 172.21.15.172:0/2236876773 shutdown_connections 2024-04-14T08:43:21.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:21.893+0000 7fb68211e640 1 -- 172.21.15.172:0/2236876773 wait complete. 2024-04-14T08:43:21.905 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-14T08:43:21.905 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:43:22.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.007+0000 7f2d7d208640 1 Processor -- start 2024-04-14T08:43:22.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.007+0000 7f2d7d208640 1 -- start start 2024-04-14T08:43:22.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.008+0000 7f2d7d208640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815e890 0x7f2d780d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.008+0000 7f2d7d208640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2d780cedd0 0x7f2d780b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.008+0000 7f2d7d208640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d780cf320 con 0x7f2d780d2830 2024-04-14T08:43:22.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.008+0000 7f2d7d208640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f2d780cff30 con 0x7f2d780cedd0 2024-04-14T08:43:22.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.008+0000 7f2d7d208640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2d780d2350 con 0x7f2d7815e890 2024-04-14T08:43:22.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.008+0000 7f2d76d9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f2d780d2830 0x7f2d780d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:51928/0 (socket says 172.21.15.172:51928) 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.008+0000 7f2d76d9d640 1 -- 172.21.15.172:0/3856969666 learned_addr learned my addr 172.21.15.172:0/3856969666 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3354255175 0 0) 0x7f2d780cf320 con 0x7f2d780d2830 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7759e640 1 --2- 172.21.15.172:0/3856969666 >> v2:172.21.15.177:3300/0 conn(0x7f2d780cedd0 0x7f2d780b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d64003520 con 0x7f2d780d2830 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d6e59c640 1 --2- 172.21.15.172:0/3856969666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815e890 0x7f2d780d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3388368390 0 0) 0x7f2d64003520 con 0x7f2d780d2830 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815e890 msgr2=0x7f2d780d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 --2- 172.21.15.172:0/3856969666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815e890 0x7f2d780d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 >> v2:172.21.15.177:3300/0 conn(0x7f2d780cedd0 msgr2=0x7f2d780b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 --2- 172.21.15.172:0/3856969666 >> v2:172.21.15.177:3300/0 conn(0x7f2d780cedd0 0x7f2d780b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d780cf600 con 0x7f2d780d2830 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f2d68001670 con 0x7f2d780d2830 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 >> v1:172.21.15.172:6789/0 conn(0x7f2d780d2830 legacy=0x7f2d780d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 --2- 172.21.15.172:0/3856969666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d640039b0 0x7f2d64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 --2- 172.21.15.172:0/3856969666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815e890 0x7f2d64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 --2- 172.21.15.172:0/3856969666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d64004d40 0x7f2d640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780cff30 con 0x7f2d640039b0 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780d2350 con 0x7f2d64004d40 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6400b9c0 con 0x7f2d7815e890 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d76d9d640 1 --2- 172.21.15.172:0/3856969666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d640039b0 0x7f2d64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d6e59c640 1 --2- 172.21.15.172:0/3856969666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815e890 0x7f2d64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d7759e640 1 --2- 172.21.15.172:0/3856969666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d64004d40 0x7f2d640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d76d9d640 1 -- 172.21.15.172:0/3856969666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815e890 msgr2=0x7f2d64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d76d9d640 1 --2- 172.21.15.172:0/3856969666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d7815e890 0x7f2d64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d76d9d640 1 -- 172.21.15.172:0/3856969666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d64004d40 msgr2=0x7f2d640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d76d9d640 1 --2- 172.21.15.172:0/3856969666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d64004d40 0x7f2d640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.009+0000 7f2d76d9d640 1 -- 172.21.15.172:0/3856969666 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d64003520 con 0x7f2d640039b0 2024-04-14T08:43:22.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.010+0000 7f2d76d9d640 1 --2- 172.21.15.172:0/3856969666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d640039b0 0x7f2d64003dd0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f2d68001050 tx=0x7f2d64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85579a6ddf5b0078 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:22.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.010+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68007d70 con 0x7f2d640039b0 2024-04-14T08:43:22.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.010+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d68030070 con 0x7f2d640039b0 2024-04-14T08:43:22.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.010+0000 7f2d7659c640 1 -- 172.21.15.172:0/3856969666 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6802cd80 con 0x7f2d640039b0 2024-04-14T08:43:22.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.011+0000 7f2d7d208640 1 -- 172.21.15.172:0/3856969666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d640039b0 msgr2=0x7f2d64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.011+0000 7f2d7d208640 1 --2- 172.21.15.172:0/3856969666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d640039b0 0x7f2d64003dd0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f2d68001050 tx=0x7f2d64005830 comp rx=0 tx=0).stop 2024-04-14T08:43:22.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.011+0000 7f2d7759e640 1 -- 172.21.15.172:0/3856969666 reap_dead start 2024-04-14T08:43:22.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.011+0000 7f2d7d208640 1 -- 172.21.15.172:0/3856969666 shutdown_connections 2024-04-14T08:43:22.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.011+0000 7f2d7d208640 1 -- 172.21.15.172:0/3856969666 >> 172.21.15.172:0/3856969666 conn(0x7f2d7809d590 msgr2=0x7f2d780d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:22.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.011+0000 7f2d7d208640 1 -- 172.21.15.172:0/3856969666 shutdown_connections 2024-04-14T08:43:22.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.011+0000 7f2d7d208640 1 -- 172.21.15.172:0/3856969666 wait complete. 2024-04-14T08:43:22.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.013+0000 7f2d7d208640 1 Processor -- start 2024-04-14T08:43:22.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.013+0000 7f2d7d208640 1 -- start start 2024-04-14T08:43:22.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.014+0000 7f2d7d208640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d640039b0 0x7f2d7815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.014+0000 7f2d7d208640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d78160830 0x7f2d78155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.014+0000 7f2d7d208640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78160ef0 0x7f2d780b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.014+0000 7f2d7d208640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780d1300 con 0x7f2d78160ef0 2024-04-14T08:43:22.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.014+0000 7f2d7d208640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2d780cfbc0 con 0x7f2d78160830 2024-04-14T08:43:22.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.014+0000 7f2d7d208640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f2d780d1480 con 0x7f2d640039b0 2024-04-14T08:43:22.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d7759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78160ef0 0x7f2d780b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d7759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78160ef0 0x7f2d780b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47854/0 (socket says 172.21.15.172:47854) 2024-04-14T08:43:22.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d7759e640 1 -- 172.21.15.172:0/1424187652 learned_addr learned my addr 172.21.15.172:0/1424187652 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d76d9d640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d640039b0 0x7f2d7815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d6e59c640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d78160830 0x7f2d78155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d7759e640 1 -- 172.21.15.172:0/1424187652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d640039b0 msgr2=0x7f2d7815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d7759e640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d640039b0 0x7f2d7815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d7759e640 1 -- 172.21.15.172:0/1424187652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d78160830 msgr2=0x7f2d78155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d7759e640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d78160830 0x7f2d78155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d7759e640 1 -- 172.21.15.172:0/1424187652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d78155a80 con 0x7f2d78160ef0 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d7759e640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78160ef0 0x7f2d780b1f40 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f2d7808d6e0 tx=0x7f2d7000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d6ffff640 1 -- 172.21.15.172:0/1424187652 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000fce0 con 0x7f2d78160ef0 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d6ffff640 1 -- 172.21.15.172:0/1424187652 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d7000fe80 con 0x7f2d78160ef0 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d7d208640 1 -- 172.21.15.172:0/1424187652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d781617b0 con 0x7f2d78160ef0 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d6ffff640 1 -- 172.21.15.172:0/1424187652 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70015670 con 0x7f2d78160ef0 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.015+0000 7f2d7d208640 1 -- 172.21.15.172:0/1424187652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7815f0b0 con 0x7f2d78160ef0 2024-04-14T08:43:22.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.016+0000 7f2d7d208640 1 -- 172.21.15.172:0/1424187652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d78068520 con 0x7f2d78160ef0 2024-04-14T08:43:22.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.017+0000 7f2d6ffff640 1 -- 172.21.15.172:0/1424187652 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d70015810 con 0x7f2d78160ef0 2024-04-14T08:43:22.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.017+0000 7f2d6ffff640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d5806c920 0x7f2d5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.018+0000 7f2d76d9d640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d5806c920 0x7f2d5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.018+0000 7f2d6ffff640 1 -- 172.21.15.172:0/1424187652 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7f2d7008c910 con 0x7f2d78160ef0 2024-04-14T08:43:22.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.018+0000 7f2d76d9d640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d5806c920 0x7f2d5806ede0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2d68034000 tx=0x7f2d68007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:22.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.022+0000 7f2d6ffff640 1 -- 172.21.15.172:0/1424187652 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2d70019030 con 0x7f2d78160ef0 2024-04-14T08:43:22.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.242+0000 7f2d7d208640 1 -- 172.21.15.172:0/1424187652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d780d2610 con 0x7f2d78160ef0 2024-04-14T08:43:22.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.242+0000 7f2d6ffff640 1 -- 172.21.15.172:0/1424187652 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5199 (secure 0 0 0) 0x7f2d70010040 con 0x7f2d78160ef0 2024-04-14T08:43:22.247 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:22.247 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3881291102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3881291102},{"type":"v1","addr":"172.21.15.177:6835","nonce":3881291102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:43:13.745318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9519":{"gid":9519,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.172:6837/4065123823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4065123823},{"type":"v1","addr":"172.21.15.172:6837","nonce":4065123823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9519,"qdb_cluster":[9519]},"id":8}]} 2024-04-14T08:43:22.248 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 100 2024-04-14T08:43:22.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.244+0000 7f2d7d208640 1 -- 172.21.15.172:0/1424187652 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d5806c920 msgr2=0x7f2d5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.244+0000 7f2d7d208640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d5806c920 0x7f2d5806ede0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2d68034000 tx=0x7f2d68007000 comp rx=0 tx=0).stop 2024-04-14T08:43:22.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.244+0000 7f2d7d208640 1 -- 172.21.15.172:0/1424187652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78160ef0 msgr2=0x7f2d780b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.244+0000 7f2d7d208640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78160ef0 0x7f2d780b1f40 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f2d7808d6e0 tx=0x7f2d7000f1b0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.244+0000 7f2d7d208640 1 -- 172.21.15.172:0/1424187652 shutdown_connections 2024-04-14T08:43:22.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.244+0000 7f2d7d208640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f2d5806c920 0x7f2d5806ede0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.244+0000 7f2d7d208640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78160ef0 0x7f2d780b1f40 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.244+0000 7f2d7d208640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d78160830 0x7f2d78155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.244+0000 7f2d7d208640 1 --2- 172.21.15.172:0/1424187652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d640039b0 0x7f2d7815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.244+0000 7f2d7d208640 1 -- 172.21.15.172:0/1424187652 >> 172.21.15.172:0/1424187652 conn(0x7f2d7809d590 msgr2=0x7f2d780b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:22.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.244+0000 7f2d7d208640 1 -- 172.21.15.172:0/1424187652 shutdown_connections 2024-04-14T08:43:22.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.244+0000 7f2d7d208640 1 -- 172.21.15.172:0/1424187652 wait complete. 2024-04-14T08:43:22.259 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:43:22.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.364+0000 7f03244de640 1 Processor -- start 2024-04-14T08:43:22.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.364+0000 7f03244de640 1 -- start start 2024-04-14T08:43:22.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.365+0000 7f03244de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f031c0d2830 0x7f031c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.365+0000 7f03244de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f031c15e890 0x7f031c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.365+0000 7f03244de640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f031c0d3150 con 0x7f031c0cedd0 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.365+0000 7f03244de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f031c0cff30 con 0x7f031c15e890 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.365+0000 7f03244de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f031c0d2350 con 0x7f031c0d2830 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.365+0000 7f0321a79640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f031c15e890 0x7f031c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.365+0000 7f0321a79640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f031c15e890 0x7f031c0d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47870/0 (socket says 172.21.15.172:47870) 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.365+0000 7f0321a79640 1 -- 172.21.15.172:0/4058211968 learned_addr learned my addr 172.21.15.172:0/4058211968 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.365+0000 7f032227a640 1 --2- 172.21.15.172:0/4058211968 >> v2:172.21.15.177:3300/0 conn(0x7f031c0d2830 0x7f031c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.366+0000 7f0321a79640 1 -- 172.21.15.172:0/4058211968 >> v2:172.21.15.177:3300/0 conn(0x7f031c0d2830 msgr2=0x7f031c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.366+0000 7f0321a79640 1 --2- 172.21.15.172:0/4058211968 >> v2:172.21.15.177:3300/0 conn(0x7f031c0d2830 0x7f031c0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.366+0000 7f0321a79640 1 -- 172.21.15.172:0/4058211968 >> v1:172.21.15.177:6789/0 conn(0x7f031c0cedd0 legacy=0x7f031c1621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.366+0000 7f0321a79640 1 -- 172.21.15.172:0/4058211968 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c0cf320 con 0x7f031c15e890 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.366+0000 7f0321a79640 1 --2- 172.21.15.172:0/4058211968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c15e890 0x7f031c0d1300 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f031c08c830 tx=0x7f031802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cc9e29ea190882 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.366+0000 7f0321278640 1 -- 172.21.15.172:0/4058211968 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318034070 con 0x7f031c15e890 2024-04-14T08:43:22.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.366+0000 7f0321278640 1 -- 172.21.15.172:0/4058211968 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0318030860 con 0x7f031c15e890 2024-04-14T08:43:22.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.366+0000 7f0321278640 1 -- 172.21.15.172:0/4058211968 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318030c30 con 0x7f031c15e890 2024-04-14T08:43:22.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.367+0000 7f03244de640 1 -- 172.21.15.172:0/4058211968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c15e890 msgr2=0x7f031c0d1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.367+0000 7f03244de640 1 --2- 172.21.15.172:0/4058211968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c15e890 0x7f031c0d1300 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f031c08c830 tx=0x7f031802b9a0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.367+0000 7f03244de640 1 -- 172.21.15.172:0/4058211968 shutdown_connections 2024-04-14T08:43:22.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.367+0000 7f03244de640 1 --2- 172.21.15.172:0/4058211968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c15e890 0x7f031c0d1300 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.367+0000 7f03244de640 1 --2- 172.21.15.172:0/4058211968 >> v2:172.21.15.177:3300/0 conn(0x7f031c0d2830 0x7f031c0d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.367+0000 7f03244de640 1 -- 172.21.15.172:0/4058211968 >> 172.21.15.172:0/4058211968 conn(0x7f031c09d590 msgr2=0x7f031c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:22.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.367+0000 7f03244de640 1 -- 172.21.15.172:0/4058211968 shutdown_connections 2024-04-14T08:43:22.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.367+0000 7f03244de640 1 -- 172.21.15.172:0/4058211968 wait complete. 2024-04-14T08:43:22.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.368+0000 7f03244de640 1 Processor -- start 2024-04-14T08:43:22.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.368+0000 7f03244de640 1 -- start start 2024-04-14T08:43:22.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f03244de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f031c0cedd0 0x7f031c155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f03244de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031c0d2830 0x7f031c15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f03244de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c15e890 0x7f031c15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f03244de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f031c0cfbc0 con 0x7f031c15e890 2024-04-14T08:43:22.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f03244de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f031c0cf8e0 con 0x7f031c0cedd0 2024-04-14T08:43:22.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f03244de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f031c0d1840 con 0x7f031c0d2830 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f0322a7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c15e890 0x7f031c15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f0322a7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c15e890 0x7f031c15b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47872/0 (socket says 172.21.15.172:47872) 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f0322a7b640 1 -- 172.21.15.172:0/1131068299 learned_addr learned my addr 172.21.15.172:0/1131068299 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f032227a640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f031c0cedd0 0x7f031c155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f0321a79640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031c0d2830 0x7f031c15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f0322a7b640 1 -- 172.21.15.172:0/1131068299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031c0d2830 msgr2=0x7f031c15b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f0322a7b640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031c0d2830 0x7f031c15b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f0322a7b640 1 -- 172.21.15.172:0/1131068299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f031c0cedd0 msgr2=0x7f031c155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f0322a7b640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f031c0cedd0 0x7f031c155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f0322a7b640 1 -- 172.21.15.172:0/1131068299 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c1606f0 con 0x7f031c15e890 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f032227a640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f031c0cedd0 0x7f031c155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.369+0000 7f0322a7b640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c15e890 0x7f031c15b780 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f031400b970 tx=0x7f031400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:22.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.370+0000 7f030effd640 1 -- 172.21.15.172:0/1131068299 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314019070 con 0x7f031c15e890 2024-04-14T08:43:22.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.370+0000 7f030effd640 1 -- 172.21.15.172:0/1131068299 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f031400cb40 con 0x7f031c15e890 2024-04-14T08:43:22.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.370+0000 7f03244de640 1 -- 172.21.15.172:0/1131068299 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031c1609d0 con 0x7f031c15e890 2024-04-14T08:43:22.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.370+0000 7f030effd640 1 -- 172.21.15.172:0/1131068299 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314014440 con 0x7f031c15e890 2024-04-14T08:43:22.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.370+0000 7f03244de640 1 -- 172.21.15.172:0/1131068299 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031c0b73a0 con 0x7f031c15e890 2024-04-14T08:43:22.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.371+0000 7f03244de640 1 -- 172.21.15.172:0/1131068299 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f031c068520 con 0x7f031c15e890 2024-04-14T08:43:22.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.371+0000 7f030effd640 1 -- 172.21.15.172:0/1131068299 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f031401c070 con 0x7f031c15e890 2024-04-14T08:43:22.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.372+0000 7f030effd640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f030006c8f0 0x7f030006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.372+0000 7f032227a640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f030006c8f0 0x7f030006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.372+0000 7f030effd640 1 -- 172.21.15.172:0/1131068299 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7f031408b930 con 0x7f031c15e890 2024-04-14T08:43:22.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.372+0000 7f032227a640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f030006c8f0 0x7f030006edb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0310006fd0 tx=0x7f0310008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:22.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.376+0000 7f030effd640 1 -- 172.21.15.172:0/1131068299 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0314059bb0 con 0x7f031c15e890 2024-04-14T08:43:22.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.594+0000 7f03244de640 1 -- 172.21.15.172:0/1131068299 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f031c0d2610 con 0x7f031c15e890 2024-04-14T08:43:22.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.595+0000 7f030effd640 1 -- 172.21.15.172:0/1131068299 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5199 (secure 0 0 0) 0x7f03140594b0 con 0x7f031c15e890 2024-04-14T08:43:22.595 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:22.595 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3881291102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3881291102},{"type":"v1","addr":"172.21.15.177:6835","nonce":3881291102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-14T08:42:40.251538+0000","modified":"2024-04-14T08:43:13.745318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9519},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9519":{"gid":9519,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.172:6837/4065123823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":4065123823},{"type":"v1","addr":"172.21.15.172:6837","nonce":4065123823}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9519,"qdb_cluster":[9519]},"id":8}]} 2024-04-14T08:43:22.595 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 100 2024-04-14T08:43:22.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.596+0000 7f03244de640 1 -- 172.21.15.172:0/1131068299 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f030006c8f0 msgr2=0x7f030006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.596+0000 7f03244de640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f030006c8f0 0x7f030006edb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0310006fd0 tx=0x7f0310008040 comp rx=0 tx=0).stop 2024-04-14T08:43:22.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.596+0000 7f03244de640 1 -- 172.21.15.172:0/1131068299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c15e890 msgr2=0x7f031c15b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.596+0000 7f03244de640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c15e890 0x7f031c15b780 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f031400b970 tx=0x7f031400be30 comp rx=0 tx=0).stop 2024-04-14T08:43:22.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.597+0000 7f03244de640 1 -- 172.21.15.172:0/1131068299 shutdown_connections 2024-04-14T08:43:22.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.597+0000 7f03244de640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f030006c8f0 0x7f030006edb0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.597+0000 7f03244de640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f031c15e890 0x7f031c15b780 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.597+0000 7f03244de640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031c0d2830 0x7f031c15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.597+0000 7f03244de640 1 --2- 172.21.15.172:0/1131068299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f031c0cedd0 0x7f031c155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.597+0000 7f03244de640 1 -- 172.21.15.172:0/1131068299 >> 172.21.15.172:0/1131068299 conn(0x7f031c09d590 msgr2=0x7f031c0b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:22.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.597+0000 7f03244de640 1 -- 172.21.15.172:0/1131068299 shutdown_connections 2024-04-14T08:43:22.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.597+0000 7f03244de640 1 -- 172.21.15.172:0/1131068299 wait complete. 2024-04-14T08:43:22.608 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:43:22.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.709+0000 7fa8fc55e640 1 Processor -- start 2024-04-14T08:43:22.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.709+0000 7fa8fc55e640 1 -- start start 2024-04-14T08:43:22.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.710+0000 7fa8fc55e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f40cedd0 0x7fa8f41694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.710+0000 7fa8fc55e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8f40cf320 con 0x7fa8f40d2830 2024-04-14T08:43:22.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.710+0000 7fa8fc55e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8f40cf600 con 0x7fa8f415e890 2024-04-14T08:43:22.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.710+0000 7fa8fc55e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f40cff30 con 0x7fa8f40cedd0 2024-04-14T08:43:22.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.710+0000 7fa8fa2fa640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa8f40d2830 0x7fa8f40d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:51934/0 (socket says 172.21.15.172:51934) 2024-04-14T08:43:22.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.710+0000 7fa8fa2fa640 1 -- 172.21.15.172:0/2712442572 learned_addr learned my addr 172.21.15.172:0/2712442572 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:22.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.710+0000 7fa8faafb640 1 --2- 172.21.15.172:0/2712442572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f40cedd0 0x7fa8f41694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.710+0000 7fa8f92f8640 1 -- 172.21.15.172:0/2712442572 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2178658416 0 0) 0x7fa8f40cf320 con 0x7fa8f40d2830 2024-04-14T08:43:22.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.710+0000 7fa8f92f8640 1 -- 172.21.15.172:0/2712442572 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8e0003520 con 0x7fa8f40d2830 2024-04-14T08:43:22.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.711+0000 7fa8f92f8640 1 -- 172.21.15.172:0/2712442572 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1875943161 0 0) 0x7fa8f40cf600 con 0x7fa8f415e890 2024-04-14T08:43:22.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.711+0000 7fa8f92f8640 1 -- 172.21.15.172:0/2712442572 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8f40cf320 con 0x7fa8f415e890 2024-04-14T08:43:22.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.711+0000 7fa8f92f8640 1 -- 172.21.15.172:0/2712442572 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 646956212 0 0) 0x7fa8e0003520 con 0x7fa8f40d2830 2024-04-14T08:43:22.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.711+0000 7fa8faafb640 1 -- 172.21.15.172:0/2712442572 >> v1:172.21.15.177:6789/0 conn(0x7fa8f415e890 legacy=0x7fa8f40d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.711+0000 7fa8faafb640 1 -- 172.21.15.172:0/2712442572 >> v1:172.21.15.172:6789/0 conn(0x7fa8f40d2830 legacy=0x7fa8f40d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.711+0000 7fa8faafb640 1 -- 172.21.15.172:0/2712442572 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f40cf8e0 con 0x7fa8f40cedd0 2024-04-14T08:43:22.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.711+0000 7fa8faafb640 1 --2- 172.21.15.172:0/2712442572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f40cedd0 0x7fa8f41694d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009dc0 tx=0x7fa8ec02be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21718d67c62f7c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:22.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.712+0000 7fa8f92f8640 1 -- 172.21.15.172:0/2712442572 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec039070 con 0x7fa8f40cedd0 2024-04-14T08:43:22.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.712+0000 7fa8f92f8640 1 -- 172.21.15.172:0/2712442572 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8ec02c9a0 con 0x7fa8f40cedd0 2024-04-14T08:43:22.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.712+0000 7fa8f92f8640 1 -- 172.21.15.172:0/2712442572 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4001670 con 0x7fa8f40cedd0 2024-04-14T08:43:22.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.712+0000 7fa8fc55e640 1 -- 172.21.15.172:0/2712442572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f40cedd0 msgr2=0x7fa8f41694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.712+0000 7fa8fc55e640 1 --2- 172.21.15.172:0/2712442572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f40cedd0 0x7fa8f41694d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009dc0 tx=0x7fa8ec02be70 comp rx=0 tx=0).stop 2024-04-14T08:43:22.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.712+0000 7fa8fc55e640 1 -- 172.21.15.172:0/2712442572 shutdown_connections 2024-04-14T08:43:22.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.712+0000 7fa8fc55e640 1 --2- 172.21.15.172:0/2712442572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f40cedd0 0x7fa8f41694d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.712+0000 7fa8fc55e640 1 -- 172.21.15.172:0/2712442572 >> 172.21.15.172:0/2712442572 conn(0x7fa8f409d590 msgr2=0x7fa8f40d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:22.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.712+0000 7fa8fc55e640 1 -- 172.21.15.172:0/2712442572 shutdown_connections 2024-04-14T08:43:22.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.712+0000 7fa8fc55e640 1 -- 172.21.15.172:0/2712442572 wait complete. 2024-04-14T08:43:22.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.713+0000 7fa8fc55e640 1 Processor -- start 2024-04-14T08:43:22.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.713+0000 7fa8fc55e640 1 -- start start 2024-04-14T08:43:22.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fc55e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f40cedd0 0x7fa8f414e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fc55e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f40d2830 0x7fa8f40cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fc55e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f415e890 0x7fa8f40cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fc55e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f40b4720 con 0x7fa8f40cedd0 2024-04-14T08:43:22.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fc55e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f40d2350 con 0x7fa8f40d2830 2024-04-14T08:43:22.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fc55e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f40b48a0 con 0x7fa8f415e890 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fa2fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f40cedd0 0x7fa8f414e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fa2fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f40cedd0 0x7fa8f414e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47882/0 (socket says 172.21.15.172:47882) 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fa2fa640 1 -- 172.21.15.172:0/563386664 learned_addr learned my addr 172.21.15.172:0/563386664 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8f9af9640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f40d2830 0x7fa8f40cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8faafb640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f415e890 0x7fa8f40cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fa2fa640 1 -- 172.21.15.172:0/563386664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f415e890 msgr2=0x7fa8f40cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fa2fa640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f415e890 0x7fa8f40cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fa2fa640 1 -- 172.21.15.172:0/563386664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f40d2830 msgr2=0x7fa8f40cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fa2fa640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f40d2830 0x7fa8f40cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8fa2fa640 1 -- 172.21.15.172:0/563386664 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f414e550 con 0x7fa8f40cedd0 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.714+0000 7fa8f9af9640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f40d2830 0x7fa8f40cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.715+0000 7fa8fa2fa640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f40cedd0 0x7fa8f414e010 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400c750 tx=0x7fa8e400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.715+0000 7fa8eaffd640 1 -- 172.21.15.172:0/563386664 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4005ad0 con 0x7fa8f40cedd0 2024-04-14T08:43:22.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.715+0000 7fa8eaffd640 1 -- 172.21.15.172:0/563386664 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e4005c70 con 0x7fa8f40cedd0 2024-04-14T08:43:22.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.715+0000 7fa8fc55e640 1 -- 172.21.15.172:0/563386664 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f40b8f00 con 0x7fa8f40cedd0 2024-04-14T08:43:22.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.715+0000 7fa8eaffd640 1 -- 172.21.15.172:0/563386664 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e40072b0 con 0x7fa8f40cedd0 2024-04-14T08:43:22.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.715+0000 7fa8fc55e640 1 -- 172.21.15.172:0/563386664 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f40b94c0 con 0x7fa8f40cedd0 2024-04-14T08:43:22.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.716+0000 7fa8fc55e640 1 -- 172.21.15.172:0/563386664 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f4068520 con 0x7fa8f40cedd0 2024-04-14T08:43:22.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.716+0000 7fa8eaffd640 1 -- 172.21.15.172:0/563386664 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8e4007450 con 0x7fa8f40cedd0 2024-04-14T08:43:22.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.717+0000 7fa8eaffd640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8d806c8f0 0x7fa8d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:22.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.717+0000 7fa8eaffd640 1 -- 172.21.15.172:0/563386664 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7fa8e408bc20 con 0x7fa8f40cedd0 2024-04-14T08:43:22.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.717+0000 7fa8f9af9640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8d806c8f0 0x7fa8d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:22.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.717+0000 7fa8f9af9640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8d806c8f0 0x7fa8d806edb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40c7cf0 tx=0x7fa8f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:22.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.721+0000 7fa8eaffd640 1 -- 172.21.15.172:0/563386664 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa8e4005e20 con 0x7fa8f40cedd0 2024-04-14T08:43:22.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.904+0000 7fa8fc55e640 1 -- 172.21.15.172:0/563386664 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8f414e860 con 0x7fa8f40cedd0 2024-04-14T08:43:22.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.905+0000 7fa8eaffd640 1 -- 172.21.15.172:0/563386664 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+17273 (secure 0 0 0) 0x7fa8e4059ed0 con 0x7fa8f40cedd0 2024-04-14T08:43:22.906 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:22.907 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":118,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:43:08.711666+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-14T08:42:38.455900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-14T08:42:39.198157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/744853572":"2024-04-15T08:43:08.711642+0000","172.21.15.177:6835/3201353747":"2024-04-15T08:42:57.915856+0000","172.21.15.172:6835/744853572":"2024-04-15T08:43:08.711642+0000","172.21.15.177:6833/3201353747":"2024-04-15T08:42:57.915856+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:43:22.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.907+0000 7fa8fc55e640 1 -- 172.21.15.172:0/563386664 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8d806c8f0 msgr2=0x7fa8d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.907+0000 7fa8fc55e640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8d806c8f0 0x7fa8d806edb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40c7cf0 tx=0x7fa8f0009040 comp rx=0 tx=0).stop 2024-04-14T08:43:22.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.907+0000 7fa8fc55e640 1 -- 172.21.15.172:0/563386664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f40cedd0 msgr2=0x7fa8f414e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:22.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.907+0000 7fa8fc55e640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f40cedd0 0x7fa8f414e010 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400c750 tx=0x7fa8e400cc10 comp rx=0 tx=0).stop 2024-04-14T08:43:22.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.908+0000 7fa8fc55e640 1 -- 172.21.15.172:0/563386664 shutdown_connections 2024-04-14T08:43:22.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.908+0000 7fa8fc55e640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fa8d806c8f0 0x7fa8d806edb0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.908+0000 7fa8fc55e640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8f415e890 0x7fa8f40cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.908+0000 7fa8fc55e640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f40d2830 0x7fa8f40cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.908+0000 7fa8fc55e640 1 --2- 172.21.15.172:0/563386664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f40cedd0 0x7fa8f414e010 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:22.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.908+0000 7fa8fc55e640 1 -- 172.21.15.172:0/563386664 >> 172.21.15.172:0/563386664 conn(0x7fa8f409d590 msgr2=0x7fa8f406c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:22.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.908+0000 7fa8fc55e640 1 -- 172.21.15.172:0/563386664 shutdown_connections 2024-04-14T08:43:22.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:22.908+0000 7fa8fc55e640 1 -- 172.21.15.172:0/563386664 wait complete. 2024-04-14T08:43:22.920 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-14T08:43:23.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.040+0000 7fcb6b9bb640 1 Processor -- start 2024-04-14T08:43:23.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.041+0000 7fcb6b9bb640 1 -- start start 2024-04-14T08:43:23.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.041+0000 7fcb6b9bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6415e430 0x7fcb64162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:23.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.041+0000 7fcb6b9bb640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb640d2e00 0x7fcb641629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:23.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.041+0000 7fcb6b9bb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb640cefb0 con 0x7fcb64162540 2024-04-14T08:43:23.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.041+0000 7fcb6b9bb640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fcb640cf290 con 0x7fcb640d2e00 2024-04-14T08:43:23.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.041+0000 7fcb6b9bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcb640cf570 con 0x7fcb6415e430 2024-04-14T08:43:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.041+0000 7fcb69f58640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb640d2e00 0x7fcb641629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.041+0000 7fcb69f58640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb640d2e00 0x7fcb641629a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47890/0 (socket says 172.21.15.172:47890) 2024-04-14T08:43:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.041+0000 7fcb69f58640 1 -- 172.21.15.172:0/1840439796 learned_addr learned my addr 172.21.15.172:0/1840439796 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.042+0000 7fcb69757640 1 --2- 172.21.15.172:0/1840439796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6415e430 0x7fcb64162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.042+0000 7fcb69f58640 1 -- 172.21.15.172:0/1840439796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6415e430 msgr2=0x7fcb64162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.042+0000 7fcb69f58640 1 --2- 172.21.15.172:0/1840439796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6415e430 0x7fcb64162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.042+0000 7fcb69f58640 1 -- 172.21.15.172:0/1840439796 >> v1:172.21.15.177:6789/0 conn(0x7fcb64162540 legacy=0x7fcb640b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.042+0000 7fcb69f58640 1 -- 172.21.15.172:0/1840439796 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb640cf850 con 0x7fcb640d2e00 2024-04-14T08:43:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.042+0000 7fcb69757640 1 --2- 172.21.15.172:0/1840439796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6415e430 0x7fcb64162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.043+0000 7fcb69f58640 1 --2- 172.21.15.172:0/1840439796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb640d2e00 0x7fcb641629a0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009e50 tx=0x7fcb5c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa42db4e8d906308 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.043+0000 7fcb5bfff640 1 -- 172.21.15.172:0/1840439796 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c02cb00 con 0x7fcb640d2e00 2024-04-14T08:43:23.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.043+0000 7fcb5bfff640 1 -- 172.21.15.172:0/1840439796 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb5c02cca0 con 0x7fcb640d2e00 2024-04-14T08:43:23.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.043+0000 7fcb5bfff640 1 -- 172.21.15.172:0/1840439796 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c034610 con 0x7fcb640d2e00 2024-04-14T08:43:23.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.043+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1840439796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb640d2e00 msgr2=0x7fcb641629a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:23.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.043+0000 7fcb6b9bb640 1 --2- 172.21.15.172:0/1840439796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb640d2e00 0x7fcb641629a0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009e50 tx=0x7fcb5c02bf60 comp rx=0 tx=0).stop 2024-04-14T08:43:23.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.043+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1840439796 shutdown_connections 2024-04-14T08:43:23.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.043+0000 7fcb6b9bb640 1 --2- 172.21.15.172:0/1840439796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb640d2e00 0x7fcb641629a0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.043+0000 7fcb6b9bb640 1 --2- 172.21.15.172:0/1840439796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb6415e430 0x7fcb64162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.043+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1840439796 >> 172.21.15.172:0/1840439796 conn(0x7fcb6409d560 msgr2=0x7fcb640d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:23.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.043+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1840439796 shutdown_connections 2024-04-14T08:43:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.043+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1840439796 wait complete. 2024-04-14T08:43:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.044+0000 7fcb6b9bb640 1 Processor -- start 2024-04-14T08:43:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.045+0000 7fcb6b9bb640 1 -- start start 2024-04-14T08:43:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.045+0000 7fcb6b9bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb640d2e00 0x7fcb641591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.045+0000 7fcb6b9bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6415e430 0x7fcb640cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.045+0000 7fcb6b9bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb64162540 0x7fcb64158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.045+0000 7fcb6b9bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcb640d4e20 con 0x7fcb6415e430 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.045+0000 7fcb6b9bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6415e810 con 0x7fcb640d2e00 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.045+0000 7fcb6b9bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fcb640d0050 con 0x7fcb64162540 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.045+0000 7fcb68f56640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6415e430 0x7fcb640cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.045+0000 7fcb68f56640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6415e430 0x7fcb640cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47892/0 (socket says 172.21.15.172:47892) 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.045+0000 7fcb68f56640 1 -- 172.21.15.172:0/1606111546 learned_addr learned my addr 172.21.15.172:0/1606111546 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.045+0000 7fcb69f58640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb64162540 0x7fcb64158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.045+0000 7fcb69757640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb640d2e00 0x7fcb641591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.046+0000 7fcb68f56640 1 -- 172.21.15.172:0/1606111546 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb64162540 msgr2=0x7fcb64158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.046+0000 7fcb68f56640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb64162540 0x7fcb64158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.046+0000 7fcb68f56640 1 -- 172.21.15.172:0/1606111546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb640d2e00 msgr2=0x7fcb641591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.046+0000 7fcb68f56640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb640d2e00 0x7fcb641591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.046+0000 7fcb68f56640 1 -- 172.21.15.172:0/1606111546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb641e9020 con 0x7fcb6415e430 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.046+0000 7fcb68f56640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6415e430 0x7fcb640cdd20 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c3c0 tx=0x7fcb5400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:23.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.046+0000 7fcb59ffb640 1 -- 172.21.15.172:0/1606111546 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb540052d0 con 0x7fcb6415e430 2024-04-14T08:43:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.046+0000 7fcb59ffb640 1 -- 172.21.15.172:0/1606111546 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb54005490 con 0x7fcb6415e430 2024-04-14T08:43:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.046+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1606111546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb641e8a60 con 0x7fcb6415e430 2024-04-14T08:43:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.046+0000 7fcb59ffb640 1 -- 172.21.15.172:0/1606111546 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54007260 con 0x7fcb6415e430 2024-04-14T08:43:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.046+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1606111546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb640cceb0 con 0x7fcb6415e430 2024-04-14T08:43:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.047+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1606111546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb640cccc0 con 0x7fcb6415e430 2024-04-14T08:43:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.047+0000 7fcb59ffb640 1 -- 172.21.15.172:0/1606111546 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb54007400 con 0x7fcb6415e430 2024-04-14T08:43:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.048+0000 7fcb59ffb640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb4806c8f0 0x7fcb4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.048+0000 7fcb59ffb640 1 -- 172.21.15.172:0/1606111546 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6273+0+0 (secure 0 0 0) 0x7fcb5408b0f0 con 0x7fcb6415e430 2024-04-14T08:43:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.048+0000 7fcb69757640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb4806c8f0 0x7fcb4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.048+0000 7fcb69757640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb4806c8f0 0x7fcb4806edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcb6406d4f0 tx=0x7fcb60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:23.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.052+0000 7fcb59ffb640 1 -- 172.21.15.172:0/1606111546 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcb641e9020 con 0x7fcb6415e430 2024-04-14T08:43:23.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.304+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1606111546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcb64068590 con 0x7fcb6415e430 2024-04-14T08:43:23.305 INFO:tasks.ceph.mon.a.smithi172.stderr:2024-04-14T08:43:23.305+0000 7f0f2288a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-14T08:43:23.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.306+0000 7fcb59ffb640 1 -- 172.21.15.172:0/1606111546 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) ==== 161+0+0 (secure 0 0 0) 0x7fcb5405f020 con 0x7fcb6415e430 2024-04-14T08:43:23.306 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-14T08:43:23.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.309+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1606111546 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb4806c8f0 msgr2=0x7fcb4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:23.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.309+0000 7fcb6b9bb640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb4806c8f0 0x7fcb4806edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcb6406d4f0 tx=0x7fcb60008040 comp rx=0 tx=0).stop 2024-04-14T08:43:23.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.309+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1606111546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6415e430 msgr2=0x7fcb640cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:23.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.309+0000 7fcb6b9bb640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6415e430 0x7fcb640cdd20 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400c3c0 tx=0x7fcb5400c880 comp rx=0 tx=0).stop 2024-04-14T08:43:23.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.309+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1606111546 shutdown_connections 2024-04-14T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.309+0000 7fcb6b9bb640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fcb4806c8f0 0x7fcb4806edb0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.309+0000 7fcb6b9bb640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb64162540 0x7fcb64158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.309+0000 7fcb6b9bb640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb6415e430 0x7fcb640cdd20 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.309+0000 7fcb6b9bb640 1 --2- 172.21.15.172:0/1606111546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb640d2e00 0x7fcb641591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.309+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1606111546 >> 172.21.15.172:0/1606111546 conn(0x7fcb6409d560 msgr2=0x7fcb6406dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.309+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1606111546 shutdown_connections 2024-04-14T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.309+0000 7fcb6b9bb640 1 -- 172.21.15.172:0/1606111546 wait complete. 2024-04-14T08:43:23.311 INFO:tasks.ceph.mds.c.smithi172.stderr: -1281> 2024-04-14T08:43:14.719+0000 7f29e59f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-14T08:43:23.322 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-14T08:43:23.337 INFO:tasks.ceph.mds.c.smithi172.stdout:starting mds.c at 2024-04-14T08:43:23.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.441+0000 7efc6e236640 1 Processor -- start 2024-04-14T08:43:23.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.441+0000 7efc6e236640 1 -- start start 2024-04-14T08:43:23.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.442+0000 7efc6e236640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efc68159c20 0x7efc680ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:23.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.442+0000 7efc6e236640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc68160b00 con 0x7efc68072450 2024-04-14T08:43:23.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.442+0000 7efc6e236640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc6815cf10 con 0x7efc6815c700 2024-04-14T08:43:23.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.442+0000 7efc6e236640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7efc6815fc80 con 0x7efc68159c20 2024-04-14T08:43:23.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.442+0000 7efc67fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7efc68072450 0x7efc680b5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:51942/0 (socket says 172.21.15.172:51942) 2024-04-14T08:43:23.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.442+0000 7efc67fff640 1 -- 172.21.15.172:0/3180617942 learned_addr learned my addr 172.21.15.172:0/3180617942 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:23.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.442+0000 7efc677fe640 1 --2- 172.21.15.172:0/3180617942 >> v2:172.21.15.172:3300/0 conn(0x7efc68159c20 0x7efc680ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:23.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.442+0000 7efc677fe640 1 -- 172.21.15.172:0/3180617942 >> v1:172.21.15.177:6789/0 conn(0x7efc6815c700 legacy=0x7efc681603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:23.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.442+0000 7efc667fc640 1 -- 172.21.15.172:0/3180617942 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1750613474 0 0) 0x7efc68160b00 con 0x7efc68072450 2024-04-14T08:43:23.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.442+0000 7efc677fe640 1 -- 172.21.15.172:0/3180617942 >> v1:172.21.15.172:6789/0 conn(0x7efc68072450 legacy=0x7efc680b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:23.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.442+0000 7efc677fe640 1 -- 172.21.15.172:0/3180617942 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6815d1f0 con 0x7efc68159c20 2024-04-14T08:43:23.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.443+0000 7efc677fe640 1 --2- 172.21.15.172:0/3180617942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc68159c20 0x7efc680ce200 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7efc580097a0 tx=0x7efc5802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60512fd7e2f4c5d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:23.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.443+0000 7efc667fc640 1 -- 172.21.15.172:0/3180617942 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58035020 con 0x7efc68159c20 2024-04-14T08:43:23.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.443+0000 7efc667fc640 1 -- 172.21.15.172:0/3180617942 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc58031830 con 0x7efc68159c20 2024-04-14T08:43:23.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.443+0000 7efc667fc640 1 -- 172.21.15.172:0/3180617942 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58031c00 con 0x7efc68159c20 2024-04-14T08:43:23.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.444+0000 7efc6e236640 1 -- 172.21.15.172:0/3180617942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc68159c20 msgr2=0x7efc680ce200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:23.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.444+0000 7efc6e236640 1 --2- 172.21.15.172:0/3180617942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc68159c20 0x7efc680ce200 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7efc580097a0 tx=0x7efc5802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.444+0000 7efc6e236640 1 -- 172.21.15.172:0/3180617942 shutdown_connections 2024-04-14T08:43:23.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.444+0000 7efc6e236640 1 --2- 172.21.15.172:0/3180617942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc68159c20 0x7efc680ce200 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.444+0000 7efc6e236640 1 -- 172.21.15.172:0/3180617942 >> 172.21.15.172:0/3180617942 conn(0x7efc6806b520 msgr2=0x7efc6806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:23.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.444+0000 7efc6e236640 1 -- 172.21.15.172:0/3180617942 shutdown_connections 2024-04-14T08:43:23.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.444+0000 7efc6e236640 1 -- 172.21.15.172:0/3180617942 wait complete. 2024-04-14T08:43:23.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.445+0000 7efc6e236640 1 Processor -- start 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.445+0000 7efc6e236640 1 -- start start 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc6e236640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc68072450 0x7efc6815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc6e236640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc68159c20 0x7efc68152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc6e236640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc6815c700 0x7efc68152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc6e236640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efc6815a000 con 0x7efc68072450 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc6e236640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efc6815ff60 con 0x7efc6815c700 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc6e236640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7efc6815cb80 con 0x7efc68159c20 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc68072450 0x7efc6815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc68072450 0x7efc6815daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47924/0 (socket says 172.21.15.172:47924) 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc677fe640 1 -- 172.21.15.172:0/1143653086 learned_addr learned my addr 172.21.15.172:0/1143653086 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc66ffd640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc68159c20 0x7efc68152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc67fff640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc6815c700 0x7efc68152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:23.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc677fe640 1 -- 172.21.15.172:0/1143653086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc68159c20 msgr2=0x7efc68152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:23.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc677fe640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc68159c20 0x7efc68152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc677fe640 1 -- 172.21.15.172:0/1143653086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc6815c700 msgr2=0x7efc68152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:23.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc677fe640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc6815c700 0x7efc68152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:23.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc677fe640 1 -- 172.21.15.172:0/1143653086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc681432f0 con 0x7efc68072450 2024-04-14T08:43:23.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.446+0000 7efc677fe640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc68072450 0x7efc6815daa0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7efc580098c0 tx=0x7efc580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:23.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.447+0000 7efc4ffff640 1 -- 172.21.15.172:0/1143653086 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5803ba80 con 0x7efc68072450 2024-04-14T08:43:23.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.447+0000 7efc4ffff640 1 -- 172.21.15.172:0/1143653086 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc5803bc20 con 0x7efc68072450 2024-04-14T08:43:23.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.447+0000 7efc6e236640 1 -- 172.21.15.172:0/1143653086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc681435d0 con 0x7efc68072450 2024-04-14T08:43:23.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.447+0000 7efc4ffff640 1 -- 172.21.15.172:0/1143653086 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580453f0 con 0x7efc68072450 2024-04-14T08:43:23.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.447+0000 7efc6e236640 1 -- 172.21.15.172:0/1143653086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc68152f80 con 0x7efc68072450 2024-04-14T08:43:23.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.448+0000 7efc6e236640 1 -- 172.21.15.172:0/1143653086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc68068590 con 0x7efc68072450 2024-04-14T08:43:23.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.448+0000 7efc4ffff640 1 -- 172.21.15.172:0/1143653086 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc5804c060 con 0x7efc68072450 2024-04-14T08:43:23.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.448+0000 7efc4ffff640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efc5406c8f0 0x7efc5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:23.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.449+0000 7efc4ffff640 1 -- 172.21.15.172:0/1143653086 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6359+0+0 (secure 0 0 0) 0x7efc580adb20 con 0x7efc68072450 2024-04-14T08:43:23.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.449+0000 7efc66ffd640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efc5406c8f0 0x7efc5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:23.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.449+0000 7efc66ffd640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efc5406c8f0 0x7efc5406edb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efc5c006fd0 tx=0x7efc5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:23.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.453+0000 7efc4ffff640 1 -- 172.21.15.172:0/1143653086 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efc58040070 con 0x7efc68072450 2024-04-14T08:43:23.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:23.665+0000 7efc6e236640 1 -- 172.21.15.172:0/1143653086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7efc6815f220 con 0x7efc68072450 2024-04-14T08:43:24.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.307+0000 7efc4ffff640 1 -- 172.21.15.172:0/1143653086 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v102) ==== 104+0+0 (secure 0 0 0) 0x7efc5802f2e0 con 0x7efc68072450 2024-04-14T08:43:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.311+0000 7efc6e236640 1 -- 172.21.15.172:0/1143653086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efc5406c8f0 msgr2=0x7efc5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.311+0000 7efc6e236640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efc5406c8f0 0x7efc5406edb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efc5c006fd0 tx=0x7efc5c008040 comp rx=0 tx=0).stop 2024-04-14T08:43:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.311+0000 7efc6e236640 1 -- 172.21.15.172:0/1143653086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc68072450 msgr2=0x7efc6815daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.311+0000 7efc6e236640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc68072450 0x7efc6815daa0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7efc580098c0 tx=0x7efc580046c0 comp rx=0 tx=0).stop 2024-04-14T08:43:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.311+0000 7efc6e236640 1 -- 172.21.15.172:0/1143653086 shutdown_connections 2024-04-14T08:43:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.311+0000 7efc6e236640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7efc5406c8f0 0x7efc5406edb0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.311+0000 7efc6e236640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc6815c700 0x7efc68152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.311+0000 7efc6e236640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efc68159c20 0x7efc68152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.311+0000 7efc6e236640 1 --2- 172.21.15.172:0/1143653086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc68072450 0x7efc6815daa0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.311+0000 7efc6e236640 1 -- 172.21.15.172:0/1143653086 >> 172.21.15.172:0/1143653086 conn(0x7efc6806b520 msgr2=0x7efc6815a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.311+0000 7efc6e236640 1 -- 172.21.15.172:0/1143653086 shutdown_connections 2024-04-14T08:43:24.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.311+0000 7efc6e236640 1 -- 172.21.15.172:0/1143653086 wait complete. 2024-04-14T08:43:24.325 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-14T08:43:24.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.438+0000 7fc076dbb640 1 Processor -- start 2024-04-14T08:43:24.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.438+0000 7fc076dbb640 1 -- start start 2024-04-14T08:43:24.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc076dbb640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc07009b180 0x7fc07015d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:24.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc076dbb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0700d01a0 con 0x7fc0700cfc30 2024-04-14T08:43:24.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc076dbb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0700d0480 con 0x7fc07015c5a0 2024-04-14T08:43:24.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc076dbb640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fc07015d990 con 0x7fc07009b180 2024-04-14T08:43:24.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc075358640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc0700cfc30 0x7fc0700ccce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:51958/0 (socket says 172.21.15.172:51958) 2024-04-14T08:43:24.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc075358640 1 -- 172.21.15.172:0/2365307428 learned_addr learned my addr 172.21.15.172:0/2365307428 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:24.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06ffff640 1 --2- 172.21.15.172:0/2365307428 >> v2:172.21.15.177:3300/0 conn(0x7fc07009b180 0x7fc07015d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:24.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1465447940 0 0) 0x7fc0700d01a0 con 0x7fc0700cfc30 2024-04-14T08:43:24.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc060003520 con 0x7fc0700cfc30 2024-04-14T08:43:24.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3316530781 0 0) 0x7fc060003520 con 0x7fc0700cfc30 2024-04-14T08:43:24.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 >> v2:172.21.15.177:3300/0 conn(0x7fc07009b180 msgr2=0x7fc07015d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:24.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 --2- 172.21.15.172:0/2365307428 >> v2:172.21.15.177:3300/0 conn(0x7fc07009b180 0x7fc07015d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:24.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 >> v1:172.21.15.177:6789/0 conn(0x7fc07015c5a0 legacy=0x7fc07009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:24.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0700d0760 con 0x7fc0700cfc30 2024-04-14T08:43:24.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fc0680019f0 con 0x7fc0700cfc30 2024-04-14T08:43:24.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 >> v1:172.21.15.172:6789/0 conn(0x7fc0700cfc30 legacy=0x7fc0700ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:24.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 --2- 172.21.15.172:0/2365307428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 0x7fc060003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:24.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 --2- 172.21.15.172:0/2365307428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc060004800 0x7fc060004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:24.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 --2- 172.21.15.172:0/2365307428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0600051c0 0x7fc06000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:24.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc07015d990 con 0x7fc0600039b0 2024-04-14T08:43:24.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc06000c060 con 0x7fc0600051c0 2024-04-14T08:43:24.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.439+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc06000c260 con 0x7fc060004800 2024-04-14T08:43:24.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.440+0000 7fc074b57640 1 --2- 172.21.15.172:0/2365307428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 0x7fc060003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.440+0000 7fc075358640 1 --2- 172.21.15.172:0/2365307428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0600051c0 0x7fc06000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.440+0000 7fc06ffff640 1 --2- 172.21.15.172:0/2365307428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc060004800 0x7fc060004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.440+0000 7fc074b57640 1 -- 172.21.15.172:0/2365307428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc060004800 msgr2=0x7fc060004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.440+0000 7fc074b57640 1 --2- 172.21.15.172:0/2365307428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc060004800 0x7fc060004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.440+0000 7fc074b57640 1 -- 172.21.15.172:0/2365307428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0600051c0 msgr2=0x7fc06000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.440+0000 7fc074b57640 1 --2- 172.21.15.172:0/2365307428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0600051c0 0x7fc06000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.440+0000 7fc074b57640 1 -- 172.21.15.172:0/2365307428 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc060003520 con 0x7fc0600039b0 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.440+0000 7fc075358640 1 -- 172.21.15.172:0/2365307428 reap_dead start 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.440+0000 7fc06ffff640 1 --2- 172.21.15.172:0/2365307428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc060004800 0x7fc060004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.441+0000 7fc074b57640 1 --2- 172.21.15.172:0/2365307428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 0x7fc060003dd0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fc064001750 tx=0x7fc060005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e70c993c3ef699 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.441+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064001370 con 0x7fc0600039b0 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.441+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0640052d0 con 0x7fc0600039b0 2024-04-14T08:43:24.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.441+0000 7fc06f7fe640 1 -- 172.21.15.172:0/2365307428 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064003830 con 0x7fc0600039b0 2024-04-14T08:43:24.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.441+0000 7fc076dbb640 1 -- 172.21.15.172:0/2365307428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 msgr2=0x7fc060003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:24.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.441+0000 7fc076dbb640 1 --2- 172.21.15.172:0/2365307428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 0x7fc060003dd0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fc064001750 tx=0x7fc060005830 comp rx=0 tx=0).stop 2024-04-14T08:43:24.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.441+0000 7fc076dbb640 1 -- 172.21.15.172:0/2365307428 shutdown_connections 2024-04-14T08:43:24.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.441+0000 7fc076dbb640 1 --2- 172.21.15.172:0/2365307428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 0x7fc060003dd0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:24.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.441+0000 7fc076dbb640 1 -- 172.21.15.172:0/2365307428 >> 172.21.15.172:0/2365307428 conn(0x7fc0700b3b50 msgr2=0x7fc0700b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:24.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.441+0000 7fc076dbb640 1 -- 172.21.15.172:0/2365307428 shutdown_connections 2024-04-14T08:43:24.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.441+0000 7fc076dbb640 1 -- 172.21.15.172:0/2365307428 wait complete. 2024-04-14T08:43:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.442+0000 7fc076dbb640 1 Processor -- start 2024-04-14T08:43:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.442+0000 7fc076dbb640 1 -- start start 2024-04-14T08:43:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.443+0000 7fc076dbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 0x7fc0700b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.443+0000 7fc076dbb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc07014f440 0x7fc0700c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.443+0000 7fc076dbb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0700c6070 0x7fc0700c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.443+0000 7fc076dbb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc07015d450 con 0x7fc0600039b0 2024-04-14T08:43:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.443+0000 7fc076dbb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0700d0db0 con 0x7fc0700c6070 2024-04-14T08:43:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.443+0000 7fc076dbb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc07015d5d0 con 0x7fc07014f440 2024-04-14T08:43:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.443+0000 7fc074b57640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 0x7fc0700b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.443+0000 7fc074b57640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 0x7fc0700b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47932/0 (socket says 172.21.15.172:47932) 2024-04-14T08:43:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.443+0000 7fc074b57640 1 -- 172.21.15.172:0/4112376232 learned_addr learned my addr 172.21.15.172:0/4112376232 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc06ffff640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc07014f440 0x7fc0700c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc075358640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0700c6070 0x7fc0700c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc074b57640 1 -- 172.21.15.172:0/4112376232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc07014f440 msgr2=0x7fc0700c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc074b57640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc07014f440 0x7fc0700c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc074b57640 1 -- 172.21.15.172:0/4112376232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0700c6070 msgr2=0x7fc0700c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc074b57640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0700c6070 0x7fc0700c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc074b57640 1 -- 172.21.15.172:0/4112376232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0700c9380 con 0x7fc0600039b0 2024-04-14T08:43:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc075358640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0700c6070 0x7fc0700c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc074b57640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 0x7fc0700b1010 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fc064005040 tx=0x7fc064001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc06d7fa640 1 -- 172.21.15.172:0/4112376232 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064041020 con 0x7fc0600039b0 2024-04-14T08:43:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc06d7fa640 1 -- 172.21.15.172:0/4112376232 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc064005480 con 0x7fc0600039b0 2024-04-14T08:43:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc076dbb640 1 -- 172.21.15.172:0/4112376232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0700c9660 con 0x7fc0600039b0 2024-04-14T08:43:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc06d7fa640 1 -- 172.21.15.172:0/4112376232 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064007a30 con 0x7fc0600039b0 2024-04-14T08:43:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.444+0000 7fc076dbb640 1 -- 172.21.15.172:0/4112376232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0700c9c20 con 0x7fc0600039b0 2024-04-14T08:43:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.445+0000 7fc076dbb640 1 -- 172.21.15.172:0/4112376232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0700c9110 con 0x7fc0600039b0 2024-04-14T08:43:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.445+0000 7fc06d7fa640 1 -- 172.21.15.172:0/4112376232 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc064044020 con 0x7fc0600039b0 2024-04-14T08:43:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.446+0000 7fc06d7fa640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05406c8f0 0x7fc05406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.446+0000 7fc06d7fa640 1 -- 172.21.15.172:0/4112376232 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6359+0+0 (secure 0 0 0) 0x7fc0640ad7b0 con 0x7fc0600039b0 2024-04-14T08:43:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.446+0000 7fc06ffff640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05406c8f0 0x7fc05406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.446+0000 7fc06ffff640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05406c8f0 0x7fc05406edb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc0700cbb80 tx=0x7fc05c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.450+0000 7fc06d7fa640 1 -- 172.21.15.172:0/4112376232 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc06404b030 con 0x7fc0600039b0 2024-04-14T08:43:24.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:24.625+0000 7fc076dbb640 1 -- 172.21.15.172:0/4112376232 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc0700686a0 con 0x7fc0600039b0 2024-04-14T08:43:25.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.309+0000 7fc06d7fa640 1 -- 172.21.15.172:0/4112376232 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) ==== 181+0+0 (secure 0 0 0) 0x7fc06407b8c0 con 0x7fc0600039b0 2024-04-14T08:43:25.310 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' removed 2024-04-14T08:43:25.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.312+0000 7fc076dbb640 1 -- 172.21.15.172:0/4112376232 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05406c8f0 msgr2=0x7fc05406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:25.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.312+0000 7fc076dbb640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05406c8f0 0x7fc05406edb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc0700cbb80 tx=0x7fc05c008040 comp rx=0 tx=0).stop 2024-04-14T08:43:25.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.312+0000 7fc076dbb640 1 -- 172.21.15.172:0/4112376232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 msgr2=0x7fc0700b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:25.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.312+0000 7fc076dbb640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 0x7fc0700b1010 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fc064005040 tx=0x7fc064001120 comp rx=0 tx=0).stop 2024-04-14T08:43:25.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.313+0000 7fc076dbb640 1 -- 172.21.15.172:0/4112376232 shutdown_connections 2024-04-14T08:43:25.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.313+0000 7fc076dbb640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc05406c8f0 0x7fc05406edb0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:25.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.313+0000 7fc076dbb640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0700c6070 0x7fc0700c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:25.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.313+0000 7fc076dbb640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc07014f440 0x7fc0700c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:25.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.313+0000 7fc076dbb640 1 --2- 172.21.15.172:0/4112376232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0600039b0 0x7fc0700b1010 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:25.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.313+0000 7fc076dbb640 1 -- 172.21.15.172:0/4112376232 >> 172.21.15.172:0/4112376232 conn(0x7fc0700b3b50 msgr2=0x7fc0700b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:25.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.313+0000 7fc076dbb640 1 -- 172.21.15.172:0/4112376232 shutdown_connections 2024-04-14T08:43:25.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.313+0000 7fc076dbb640 1 -- 172.21.15.172:0/4112376232 wait complete. 2024-04-14T08:43:25.327 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-14T08:43:25.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.435+0000 7feb988c0640 1 Processor -- start 2024-04-14T08:43:25.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.435+0000 7feb988c0640 1 -- start start 2024-04-14T08:43:25.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb988c0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb9415a930 0x7feb9406ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:25.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb988c0640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7feb940ce350 0x7feb940ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:25.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb988c0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb9406dc10 con 0x7feb9415e4b0 2024-04-14T08:43:25.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb988c0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7feb940cecf0 con 0x7feb9415a930 2024-04-14T08:43:25.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb988c0640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7feb9406e820 con 0x7feb940ce350 2024-04-14T08:43:25.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb9259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb9415a930 0x7feb9406ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:25.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb9259c640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb9415a930 0x7feb9406ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47944/0 (socket says 172.21.15.172:47944) 2024-04-14T08:43:25.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb9259c640 1 -- 172.21.15.172:0/182846860 learned_addr learned my addr 172.21.15.172:0/182846860 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:25.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb8bfff640 1 --2- 172.21.15.172:0/182846860 >> v2:172.21.15.177:3300/0 conn(0x7feb940ce350 0x7feb940ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:25.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb9259c640 1 -- 172.21.15.172:0/182846860 >> v2:172.21.15.177:3300/0 conn(0x7feb940ce350 msgr2=0x7feb940ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:25.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb9259c640 1 --2- 172.21.15.172:0/182846860 >> v2:172.21.15.177:3300/0 conn(0x7feb940ce350 0x7feb940ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:25.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb9259c640 1 -- 172.21.15.172:0/182846860 >> v1:172.21.15.177:6789/0 conn(0x7feb9415e4b0 legacy=0x7feb940711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:25.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.436+0000 7feb9259c640 1 -- 172.21.15.172:0/182846860 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9406def0 con 0x7feb9415a930 2024-04-14T08:43:25.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.437+0000 7feb9259c640 1 --2- 172.21.15.172:0/182846860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9415a930 0x7feb9406ecb0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7feb9415e410 tx=0x7feb8402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6c12c66edab95d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:25.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.437+0000 7feb91d9b640 1 -- 172.21.15.172:0/182846860 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8402e070 con 0x7feb9415a930 2024-04-14T08:43:25.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.437+0000 7feb91d9b640 1 -- 172.21.15.172:0/182846860 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb84037070 con 0x7feb9415a930 2024-04-14T08:43:25.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.437+0000 7feb91d9b640 1 -- 172.21.15.172:0/182846860 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84031a00 con 0x7feb9415a930 2024-04-14T08:43:25.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.438+0000 7feb988c0640 1 -- 172.21.15.172:0/182846860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9415a930 msgr2=0x7feb9406ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:25.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.438+0000 7feb988c0640 1 --2- 172.21.15.172:0/182846860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9415a930 0x7feb9406ecb0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7feb9415e410 tx=0x7feb8402b880 comp rx=0 tx=0).stop 2024-04-14T08:43:25.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.438+0000 7feb988c0640 1 -- 172.21.15.172:0/182846860 shutdown_connections 2024-04-14T08:43:25.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.438+0000 7feb988c0640 1 --2- 172.21.15.172:0/182846860 >> v2:172.21.15.177:3300/0 conn(0x7feb940ce350 0x7feb940ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:25.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.438+0000 7feb988c0640 1 --2- 172.21.15.172:0/182846860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb9415a930 0x7feb9406ecb0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:25.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.438+0000 7feb988c0640 1 -- 172.21.15.172:0/182846860 >> 172.21.15.172:0/182846860 conn(0x7feb940b1a50 msgr2=0x7feb940b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:25.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.438+0000 7feb988c0640 1 -- 172.21.15.172:0/182846860 shutdown_connections 2024-04-14T08:43:25.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.438+0000 7feb988c0640 1 -- 172.21.15.172:0/182846860 wait complete. 2024-04-14T08:43:25.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.439+0000 7feb988c0640 1 Processor -- start 2024-04-14T08:43:25.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.439+0000 7feb988c0640 1 -- start start 2024-04-14T08:43:25.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb988c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb940ce350 0x7feb940c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:25.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb988c0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb9415a930 0x7feb940c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:25.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb988c0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9415e4b0 0x7feb94157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:25.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb988c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb9415ad10 con 0x7feb940ce350 2024-04-14T08:43:25.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb988c0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feb9406e4b0 con 0x7feb9415a930 2024-04-14T08:43:25.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb988c0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7feb9415e940 con 0x7feb9415e4b0 2024-04-14T08:43:25.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb9259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb940ce350 0x7feb940c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:25.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb9259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb940ce350 0x7feb940c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47952/0 (socket says 172.21.15.172:47952) 2024-04-14T08:43:25.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb9259c640 1 -- 172.21.15.172:0/110465475 learned_addr learned my addr 172.21.15.172:0/110465475 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:25.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb8bfff640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb9415a930 0x7feb940c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb92d9d640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9415e4b0 0x7feb94157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb9259c640 1 -- 172.21.15.172:0/110465475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9415e4b0 msgr2=0x7feb94157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb9259c640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9415e4b0 0x7feb94157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb9259c640 1 -- 172.21.15.172:0/110465475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb9415a930 msgr2=0x7feb940c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb9259c640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb9415a930 0x7feb940c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb9259c640 1 -- 172.21.15.172:0/110465475 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb941f0440 con 0x7feb940ce350 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb9259c640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb940ce350 0x7feb940c7c90 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7feb84000c00 tx=0x7feb840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb8affd640 1 -- 172.21.15.172:0/110465475 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84032880 con 0x7feb940ce350 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb8affd640 1 -- 172.21.15.172:0/110465475 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb84032a20 con 0x7feb940ce350 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb988c0640 1 -- 172.21.15.172:0/110465475 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb941f0720 con 0x7feb940ce350 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb8affd640 1 -- 172.21.15.172:0/110465475 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84032e10 con 0x7feb940ce350 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.440+0000 7feb988c0640 1 -- 172.21.15.172:0/110465475 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb941f0ce0 con 0x7feb940ce350 2024-04-14T08:43:25.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.441+0000 7feb988c0640 1 -- 172.21.15.172:0/110465475 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb940686a0 con 0x7feb940ce350 2024-04-14T08:43:25.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.442+0000 7feb8affd640 1 -- 172.21.15.172:0/110465475 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb840453f0 con 0x7feb940ce350 2024-04-14T08:43:25.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.442+0000 7feb8affd640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb7406c8f0 0x7feb7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:25.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.442+0000 7feb8affd640 1 -- 172.21.15.172:0/110465475 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5921+0+0 (secure 0 0 0) 0x7feb840adaf0 con 0x7feb940ce350 2024-04-14T08:43:25.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.442+0000 7feb8bfff640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb7406c8f0 0x7feb7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:25.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.443+0000 7feb8bfff640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb7406c8f0 0x7feb7406edb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7feb940b5cf0 tx=0x7feb7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:25.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.447+0000 7feb8affd640 1 -- 172.21.15.172:0/110465475 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7feb84040070 con 0x7feb940ce350 2024-04-14T08:43:25.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:25.623+0000 7feb988c0640 1 -- 172.21.15.172:0/110465475 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7feb941f3510 con 0x7feb940ce350 2024-04-14T08:43:25.624 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:43:25.626 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:43:26.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.311+0000 7feb8affd640 1 -- 172.21.15.172:0/110465475 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) ==== 169+0+0 (secure 0 0 0) 0x7feb8407b880 con 0x7feb940ce350 2024-04-14T08:43:26.311 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' removed 2024-04-14T08:43:26.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.315+0000 7feb988c0640 1 -- 172.21.15.172:0/110465475 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb7406c8f0 msgr2=0x7feb7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:26.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.315+0000 7feb988c0640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb7406c8f0 0x7feb7406edb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7feb940b5cf0 tx=0x7feb7c008040 comp rx=0 tx=0).stop 2024-04-14T08:43:26.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.315+0000 7feb988c0640 1 -- 172.21.15.172:0/110465475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb940ce350 msgr2=0x7feb940c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:26.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.315+0000 7feb988c0640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb940ce350 0x7feb940c7c90 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7feb84000c00 tx=0x7feb840046c0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.315+0000 7feb988c0640 1 -- 172.21.15.172:0/110465475 shutdown_connections 2024-04-14T08:43:26.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.315+0000 7feb988c0640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7feb7406c8f0 0x7feb7406edb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.315+0000 7feb988c0640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb9415e4b0 0x7feb94157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.316+0000 7feb988c0640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb9415a930 0x7feb940c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.316+0000 7feb988c0640 1 --2- 172.21.15.172:0/110465475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb940ce350 0x7feb940c7c90 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.316+0000 7feb988c0640 1 -- 172.21.15.172:0/110465475 >> 172.21.15.172:0/110465475 conn(0x7feb940b1a50 msgr2=0x7feb94158e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:26.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.316+0000 7feb988c0640 1 -- 172.21.15.172:0/110465475 shutdown_connections 2024-04-14T08:43:26.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.316+0000 7feb988c0640 1 -- 172.21.15.172:0/110465475 wait complete. 2024-04-14T08:43:26.331 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-14T08:43:26.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.438+0000 7fb47cc36640 1 Processor -- start 2024-04-14T08:43:26.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.438+0000 7fb47cc36640 1 -- start start 2024-04-14T08:43:26.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb47cc36640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb47813df30 0x7fb4780b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:26.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb47cc36640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb478140e80 0x7fb4780b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:26.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb47cc36640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb47815b620 con 0x7fb47815c380 2024-04-14T08:43:26.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb47cc36640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fb4780cf9b0 con 0x7fb478140e80 2024-04-14T08:43:26.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb47cc36640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fb4781414a0 con 0x7fb47813df30 2024-04-14T08:43:26.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb476d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb478140e80 0x7fb4780b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:26.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb476d9d640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb478140e80 0x7fb4780b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47966/0 (socket says 172.21.15.172:47966) 2024-04-14T08:43:26.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb476d9d640 1 -- 172.21.15.172:0/2076481326 learned_addr learned my addr 172.21.15.172:0/2076481326 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:26.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb47659c640 1 --2- 172.21.15.172:0/2076481326 >> v2:172.21.15.177:3300/0 conn(0x7fb47813df30 0x7fb4780b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb476d9d640 1 -- 172.21.15.172:0/2076481326 >> v2:172.21.15.177:3300/0 conn(0x7fb47813df30 msgr2=0x7fb4780b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb476d9d640 1 --2- 172.21.15.172:0/2076481326 >> v2:172.21.15.177:3300/0 conn(0x7fb47813df30 0x7fb4780b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb476d9d640 1 -- 172.21.15.172:0/2076481326 >> v1:172.21.15.177:6789/0 conn(0x7fb47815c380 legacy=0x7fb478140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb476d9d640 1 -- 172.21.15.172:0/2076481326 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47815b8a0 con 0x7fb478140e80 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.439+0000 7fb47659c640 1 --2- 172.21.15.172:0/2076481326 >> v2:172.21.15.177:3300/0 conn(0x7fb47813df30 0x7fb4780b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.440+0000 7fb476d9d640 1 --2- 172.21.15.172:0/2076481326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb478140e80 0x7fb4780b2210 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fb47806f660 tx=0x7fb47002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806299cf1137b212 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.440+0000 7fb47559a640 1 -- 172.21.15.172:0/2076481326 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47002ca80 con 0x7fb478140e80 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.440+0000 7fb47559a640 1 -- 172.21.15.172:0/2076481326 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb47002cc20 con 0x7fb478140e80 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.440+0000 7fb47559a640 1 -- 172.21.15.172:0/2076481326 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4700343f0 con 0x7fb478140e80 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.441+0000 7fb47cc36640 1 -- 172.21.15.172:0/2076481326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb478140e80 msgr2=0x7fb4780b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.441+0000 7fb47cc36640 1 --2- 172.21.15.172:0/2076481326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb478140e80 0x7fb4780b2210 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fb47806f660 tx=0x7fb47002bee0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.441+0000 7fb47cc36640 1 -- 172.21.15.172:0/2076481326 shutdown_connections 2024-04-14T08:43:26.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.441+0000 7fb47cc36640 1 --2- 172.21.15.172:0/2076481326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb478140e80 0x7fb4780b2210 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.441+0000 7fb47cc36640 1 --2- 172.21.15.172:0/2076481326 >> v2:172.21.15.177:3300/0 conn(0x7fb47813df30 0x7fb4780b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.441+0000 7fb47cc36640 1 -- 172.21.15.172:0/2076481326 >> 172.21.15.172:0/2076481326 conn(0x7fb47806b520 msgr2=0x7fb47806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:26.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.441+0000 7fb47cc36640 1 -- 172.21.15.172:0/2076481326 shutdown_connections 2024-04-14T08:43:26.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.441+0000 7fb47cc36640 1 -- 172.21.15.172:0/2076481326 wait complete. 2024-04-14T08:43:26.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.442+0000 7fb47cc36640 1 Processor -- start 2024-04-14T08:43:26.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.442+0000 7fb47cc36640 1 -- start start 2024-04-14T08:43:26.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47cc36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb47813df30 0x7fb478153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:26.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47cc36640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb478140e80 0x7fb478158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:26.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47cc36640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb47815c380 0x7fb4781395e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:26.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47cc36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb47813e310 con 0x7fb47813df30 2024-04-14T08:43:26.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47cc36640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb47815be60 con 0x7fb47815c380 2024-04-14T08:43:26.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47cc36640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb4780ce9a0 con 0x7fb478140e80 2024-04-14T08:43:26.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb47813df30 0x7fb478153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:26.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb47813df30 0x7fb478153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47978/0 (socket says 172.21.15.172:47978) 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47659c640 1 -- 172.21.15.172:0/3716302793 learned_addr learned my addr 172.21.15.172:0/3716302793 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb475d9b640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb478140e80 0x7fb478158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb476d9d640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb47815c380 0x7fb4781395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47659c640 1 -- 172.21.15.172:0/3716302793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb478140e80 msgr2=0x7fb478158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47659c640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb478140e80 0x7fb478158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47659c640 1 -- 172.21.15.172:0/3716302793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb47815c380 msgr2=0x7fb4781395e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47659c640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb47815c380 0x7fb4781395e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47659c640 1 -- 172.21.15.172:0/3716302793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb478153de0 con 0x7fb47813df30 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47659c640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb47813df30 0x7fb478153790 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fb46800c7f0 tx=0x7fb46800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb466ffd640 1 -- 172.21.15.172:0/3716302793 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468015070 con 0x7fb47813df30 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb466ffd640 1 -- 172.21.15.172:0/3716302793 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb468007a30 con 0x7fb47813df30 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb466ffd640 1 -- 172.21.15.172:0/3716302793 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468007e00 con 0x7fb47813df30 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.443+0000 7fb47cc36640 1 -- 172.21.15.172:0/3716302793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4781540c0 con 0x7fb47813df30 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.444+0000 7fb47cc36640 1 -- 172.21.15.172:0/3716302793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb478154680 con 0x7fb47813df30 2024-04-14T08:43:26.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.444+0000 7fb47cc36640 1 -- 172.21.15.172:0/3716302793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4780cd460 con 0x7fb47813df30 2024-04-14T08:43:26.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.445+0000 7fb466ffd640 1 -- 172.21.15.172:0/3716302793 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb468018070 con 0x7fb47813df30 2024-04-14T08:43:26.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.445+0000 7fb466ffd640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb45806c920 0x7fb45806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:26.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.445+0000 7fb466ffd640 1 -- 172.21.15.172:0/3716302793 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7fb46808ba70 con 0x7fb47813df30 2024-04-14T08:43:26.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.445+0000 7fb475d9b640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb45806c920 0x7fb45806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:26.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.446+0000 7fb475d9b640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb45806c920 0x7fb45806ede0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb46c006fd0 tx=0x7fb46c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:26.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.450+0000 7fb466ffd640 1 -- 172.21.15.172:0/3716302793 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb468059f60 con 0x7fb47813df30 2024-04-14T08:43:26.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.625+0000 7fb47cc36640 1 -- 172.21.15.172:0/3716302793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb4780b38b0 con 0x7fb47813df30 2024-04-14T08:43:26.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.629+0000 7fb466ffd640 1 -- 172.21.15.172:0/3716302793 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fb468059860 con 0x7fb47813df30 2024-04-14T08:43:26.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.631+0000 7fb47cc36640 1 -- 172.21.15.172:0/3716302793 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb45806c920 msgr2=0x7fb45806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:26.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.631+0000 7fb47cc36640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb45806c920 0x7fb45806ede0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb46c006fd0 tx=0x7fb46c008040 comp rx=0 tx=0).stop 2024-04-14T08:43:26.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.631+0000 7fb47cc36640 1 -- 172.21.15.172:0/3716302793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb47813df30 msgr2=0x7fb478153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:26.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.631+0000 7fb47cc36640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb47813df30 0x7fb478153790 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fb46800c7f0 tx=0x7fb46800cbf0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.631+0000 7fb47cc36640 1 -- 172.21.15.172:0/3716302793 shutdown_connections 2024-04-14T08:43:26.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.631+0000 7fb47cc36640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb45806c920 0x7fb45806ede0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.631+0000 7fb47cc36640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb47815c380 0x7fb4781395e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.631+0000 7fb47cc36640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb478140e80 0x7fb478158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.631+0000 7fb47cc36640 1 --2- 172.21.15.172:0/3716302793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb47813df30 0x7fb478153790 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.631+0000 7fb47cc36640 1 -- 172.21.15.172:0/3716302793 >> 172.21.15.172:0/3716302793 conn(0x7fb47806b520 msgr2=0x7fb47815eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:26.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.631+0000 7fb47cc36640 1 -- 172.21.15.172:0/3716302793 shutdown_connections 2024-04-14T08:43:26.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.631+0000 7fb47cc36640 1 -- 172.21.15.172:0/3716302793 wait complete. 2024-04-14T08:43:26.642 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-14T08:43:26.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.743+0000 7f69a1088640 1 Processor -- start 2024-04-14T08:43:26.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.743+0000 7f69a1088640 1 -- start start 2024-04-14T08:43:26.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.744+0000 7f69a1088640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f699c0b1df0 0x7f699c0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:26.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.744+0000 7f69a1088640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f699c0b4040 0x7f699c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:26.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.744+0000 7f69a1088640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f699c15c770 con 0x7f699c0b21d0 2024-04-14T08:43:26.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.744+0000 7f69a1088640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f699c0b3430 con 0x7f699c0b4040 2024-04-14T08:43:26.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.744+0000 7f69a1088640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f699c0b3700 con 0x7f699c0b1df0 2024-04-14T08:43:26.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.744+0000 7f699b59e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f699c0b4040 0x7f699c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:26.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.744+0000 7f699b59e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f699c0b4040 0x7f699c0b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47986/0 (socket says 172.21.15.172:47986) 2024-04-14T08:43:26.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.744+0000 7f699b59e640 1 -- 172.21.15.172:0/2914784719 learned_addr learned my addr 172.21.15.172:0/2914784719 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:26.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.744+0000 7f699ad9d640 1 --2- 172.21.15.172:0/2914784719 >> v2:172.21.15.177:3300/0 conn(0x7f699c0b1df0 0x7f699c0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:26.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.745+0000 7f699b59e640 1 -- 172.21.15.172:0/2914784719 >> v2:172.21.15.177:3300/0 conn(0x7f699c0b1df0 msgr2=0x7f699c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:26.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.745+0000 7f699b59e640 1 --2- 172.21.15.172:0/2914784719 >> v2:172.21.15.177:3300/0 conn(0x7f699c0b1df0 0x7f699c0b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.745+0000 7f699b59e640 1 -- 172.21.15.172:0/2914784719 >> v1:172.21.15.177:6789/0 conn(0x7f699c0b21d0 legacy=0x7f699c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:26.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.745+0000 7f699b59e640 1 -- 172.21.15.172:0/2914784719 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c0b39e0 con 0x7f699c0b4040 2024-04-14T08:43:26.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.745+0000 7f699ad9d640 1 --2- 172.21.15.172:0/2914784719 >> v2:172.21.15.177:3300/0 conn(0x7f699c0b1df0 0x7f699c0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:26.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.745+0000 7f699b59e640 1 --2- 172.21.15.172:0/2914784719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f699c0b4040 0x7f699c0b4460 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f699c06f4b0 tx=0x7f699402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859783e7a64debfe server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:26.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.745+0000 7f699a59c640 1 -- 172.21.15.172:0/2914784719 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699402ca80 con 0x7f699c0b4040 2024-04-14T08:43:26.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.745+0000 7f699a59c640 1 -- 172.21.15.172:0/2914784719 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699402cc20 con 0x7f699c0b4040 2024-04-14T08:43:26.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.745+0000 7f699a59c640 1 -- 172.21.15.172:0/2914784719 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69940343f0 con 0x7f699c0b4040 2024-04-14T08:43:26.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.746+0000 7f69a1088640 1 -- 172.21.15.172:0/2914784719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f699c0b4040 msgr2=0x7f699c0b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:26.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.746+0000 7f69a1088640 1 --2- 172.21.15.172:0/2914784719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f699c0b4040 0x7f699c0b4460 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f699c06f4b0 tx=0x7f699402bee0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.746+0000 7f69a1088640 1 -- 172.21.15.172:0/2914784719 shutdown_connections 2024-04-14T08:43:26.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.746+0000 7f69a1088640 1 --2- 172.21.15.172:0/2914784719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f699c0b4040 0x7f699c0b4460 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.746+0000 7f69a1088640 1 --2- 172.21.15.172:0/2914784719 >> v2:172.21.15.177:3300/0 conn(0x7f699c0b1df0 0x7f699c0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.746+0000 7f69a1088640 1 -- 172.21.15.172:0/2914784719 >> 172.21.15.172:0/2914784719 conn(0x7f699c06ec30 msgr2=0x7f699c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:26.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.746+0000 7f69a1088640 1 -- 172.21.15.172:0/2914784719 shutdown_connections 2024-04-14T08:43:26.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.746+0000 7f69a1088640 1 -- 172.21.15.172:0/2914784719 wait complete. 2024-04-14T08:43:26.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.747+0000 7f69a1088640 1 Processor -- start 2024-04-14T08:43:26.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.747+0000 7f69a1088640 1 -- start start 2024-04-14T08:43:26.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f69a1088640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f699c0b1df0 0x7f699c13a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:26.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f69a1088640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f699c0b21d0 0x7f699c153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f69a1088640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f699c0b4040 0x7f699c0af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f69a1088640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f699c0b49a0 con 0x7f699c0b4040 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f69a1088640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f699c15bb40 con 0x7f699c0b21d0 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f69a1088640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f699c15b7f0 con 0x7f699c0b1df0 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f699b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f699c0b4040 0x7f699c0af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f699b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f699c0b4040 0x7f699c0af980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47994/0 (socket says 172.21.15.172:47994) 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f699b59e640 1 -- 172.21.15.172:0/3568882730 learned_addr learned my addr 172.21.15.172:0/3568882730 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f699ad9d640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f699c0b1df0 0x7f699c13a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f6993fff640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f699c0b21d0 0x7f699c153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f699b59e640 1 -- 172.21.15.172:0/3568882730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f699c0b1df0 msgr2=0x7f699c13a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f699b59e640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f699c0b1df0 0x7f699c13a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f699b59e640 1 -- 172.21.15.172:0/3568882730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f699c0b21d0 msgr2=0x7f699c153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:26.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f699b59e640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f699c0b21d0 0x7f699c153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f699b59e640 1 -- 172.21.15.172:0/3568882730 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c0cf3b0 con 0x7f699c0b4040 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f6993fff640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f699c0b21d0 0x7f699c153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.748+0000 7f699ad9d640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f699c0b1df0 0x7f699c13a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.749+0000 7f699b59e640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f699c0b4040 0x7f699c0af980 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f6994009f40 tx=0x7f6994004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.749+0000 7f69937fe640 1 -- 172.21.15.172:0/3568882730 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994009510 con 0x7f699c0b4040 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.749+0000 7f69937fe640 1 -- 172.21.15.172:0/3568882730 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6994047480 con 0x7f699c0b4040 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.749+0000 7f69937fe640 1 -- 172.21.15.172:0/3568882730 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994047850 con 0x7f699c0b4040 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.749+0000 7f69a1088640 1 -- 172.21.15.172:0/3568882730 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c0cf6f0 con 0x7f699c0b4040 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.749+0000 7f69a1088640 1 -- 172.21.15.172:0/3568882730 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c159db0 con 0x7f699c0b4040 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.750+0000 7f69a1088640 1 -- 172.21.15.172:0/3568882730 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f699c068520 con 0x7f699c0b4040 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.750+0000 7f69937fe640 1 -- 172.21.15.172:0/3568882730 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6994005d10 con 0x7f699c0b4040 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.751+0000 7f69937fe640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f697c06c920 0x7f697c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.751+0000 7f69937fe640 1 -- 172.21.15.172:0/3568882730 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f69940affc0 con 0x7f699c0b4040 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.751+0000 7f699ad9d640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f697c06c920 0x7f697c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:26.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.751+0000 7f699ad9d640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f697c06c920 0x7f697c06ede0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f698c006fd0 tx=0x7f698c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:26.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:26.755+0000 7f69937fe640 1 -- 172.21.15.172:0/3568882730 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f699c0cf6f0 con 0x7f699c0b4040 2024-04-14T08:43:27.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.008+0000 7f69a1088640 1 -- 172.21.15.172:0/3568882730 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f699c13e060 con 0x7f699c0b4040 2024-04-14T08:43:27.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.318+0000 7f69937fe640 1 -- 172.21.15.172:0/3568882730 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v500) ==== 144+0+0 (secure 0 0 0) 0x7f699404c060 con 0x7f699c0b4040 2024-04-14T08:43:27.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.320+0000 7f69a1088640 1 -- 172.21.15.172:0/3568882730 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f697c06c920 msgr2=0x7f697c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:27.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.320+0000 7f69a1088640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f697c06c920 0x7f697c06ede0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f698c006fd0 tx=0x7f698c008040 comp rx=0 tx=0).stop 2024-04-14T08:43:27.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.321+0000 7f69a1088640 1 -- 172.21.15.172:0/3568882730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f699c0b4040 msgr2=0x7f699c0af980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:27.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.321+0000 7f69a1088640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f699c0b4040 0x7f699c0af980 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f6994009f40 tx=0x7f6994004a40 comp rx=0 tx=0).stop 2024-04-14T08:43:27.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.321+0000 7f69a1088640 1 -- 172.21.15.172:0/3568882730 shutdown_connections 2024-04-14T08:43:27.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.321+0000 7f69a1088640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f697c06c920 0x7f697c06ede0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:27.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.321+0000 7f69a1088640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f699c0b4040 0x7f699c0af980 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:27.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.321+0000 7f69a1088640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f699c0b21d0 0x7f699c153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:27.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.321+0000 7f69a1088640 1 --2- 172.21.15.172:0/3568882730 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f699c0b1df0 0x7f699c13a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:27.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.321+0000 7f69a1088640 1 -- 172.21.15.172:0/3568882730 >> 172.21.15.172:0/3568882730 conn(0x7f699c06ec30 msgr2=0x7f699c15cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:27.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.321+0000 7f69a1088640 1 -- 172.21.15.172:0/3568882730 shutdown_connections 2024-04-14T08:43:27.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:27.321+0000 7f69a1088640 1 -- 172.21.15.172:0/3568882730 wait complete. 2024-04-14T08:43:27.335 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-14T08:43:27.336 INFO:tasks.cephfs_test_runner: 2024-04-14T08:43:27.336 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-14T08:43:27.336 INFO:tasks.cephfs_test_runner:Ran 7 tests in 593.522s 2024-04-14T08:43:27.336 INFO:tasks.cephfs_test_runner: 2024-04-14T08:43:27.336 INFO:tasks.cephfs_test_runner:OK 2024-04-14T08:43:27.336 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-14T08:43:27.347 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-14T08:43:27.358 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-14T08:43:27.358 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:43:27.358 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:27.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:43:27.375 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-14T08:43:27.375 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:27.431 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20615 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-14T08:43:27.431 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:43:27.431 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:27.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:43:27.450 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-14T08:43:27.450 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-14T08:43:27.507 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056330,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3233 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-14T08:43:27.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:27.508 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:43:27.522 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-14T08:43:27.523 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-14T08:43:27.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:27.523 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-14T08:43:27.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:27.592 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-14T08:43:27.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:27.660 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-04-14T08:43:27.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:27.746 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-14T08:43:27.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:27.761 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-04-14T08:43:27.831 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2024-04-14T08:43:27.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:43:27.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:27.833 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2024-04-14T08:43:27.851 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-14T08:43:27.852 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-14T08:43:27.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:27.852 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-14T08:43:27.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:27.924 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-14T08:43:27.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:27.994 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2024-04-14T08:43:28.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:28.095 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2024-04-14T08:43:28.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-14T08:43:28.111 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2024-04-14T08:43:28.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:43:28.188 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2024-04-14T08:43:28.188 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-14T08:43:28.198 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-14T08:43:28.198 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:43:28.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.301+0000 7f6215d1e640 1 Processor -- start 2024-04-14T08:43:28.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.301+0000 7f6215d1e640 1 -- start start 2024-04-14T08:43:28.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f6215d1e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f62100b21d0 0x7f621015c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f6215d1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f621015c5a0 0x7f62100b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f6215d1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62100b3700 con 0x7f62100b1df0 2024-04-14T08:43:28.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f6215d1e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f62100b39e0 con 0x7f62100b21d0 2024-04-14T08:43:28.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f6215d1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f62100b3cc0 con 0x7f621015c5a0 2024-04-14T08:43:28.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f620effd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f62100b21d0 0x7f621015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f620effd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f62100b21d0 0x7f621015c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48004/0 (socket says 172.21.15.172:48004) 2024-04-14T08:43:28.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f620effd640 1 -- 172.21.15.172:0/837626449 learned_addr learned my addr 172.21.15.172:0/837626449 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:28.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f620ffff640 1 --2- 172.21.15.172:0/837626449 >> v2:172.21.15.177:3300/0 conn(0x7f621015c5a0 0x7f62100b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f620effd640 1 -- 172.21.15.172:0/837626449 >> v2:172.21.15.177:3300/0 conn(0x7f621015c5a0 msgr2=0x7f62100b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:28.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f620effd640 1 --2- 172.21.15.172:0/837626449 >> v2:172.21.15.177:3300/0 conn(0x7f621015c5a0 0x7f62100b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f620effd640 1 -- 172.21.15.172:0/837626449 >> v1:172.21.15.177:6789/0 conn(0x7f62100b1df0 legacy=0x7f62100b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.302+0000 7f620effd640 1 -- 172.21.15.172:0/837626449 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621015bb40 con 0x7f62100b21d0 2024-04-14T08:43:28.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.303+0000 7f620effd640 1 --2- 172.21.15.172:0/837626449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62100b21d0 0x7f621015c060 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f62100b3fa0 tx=0x7f620402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4109d428d4a4935 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:28.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.303+0000 7f620e7fc640 1 -- 172.21.15.172:0/837626449 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204034070 con 0x7f62100b21d0 2024-04-14T08:43:28.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.303+0000 7f620e7fc640 1 -- 172.21.15.172:0/837626449 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6204030860 con 0x7f62100b21d0 2024-04-14T08:43:28.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.303+0000 7f620e7fc640 1 -- 172.21.15.172:0/837626449 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204030c30 con 0x7f62100b21d0 2024-04-14T08:43:28.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.304+0000 7f6215d1e640 1 -- 172.21.15.172:0/837626449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62100b21d0 msgr2=0x7f621015c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.304+0000 7f6215d1e640 1 --2- 172.21.15.172:0/837626449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62100b21d0 0x7f621015c060 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f62100b3fa0 tx=0x7f620402b9a0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.304+0000 7f6215d1e640 1 -- 172.21.15.172:0/837626449 shutdown_connections 2024-04-14T08:43:28.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.304+0000 7f6215d1e640 1 --2- 172.21.15.172:0/837626449 >> v2:172.21.15.177:3300/0 conn(0x7f621015c5a0 0x7f62100b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.304+0000 7f6215d1e640 1 --2- 172.21.15.172:0/837626449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62100b21d0 0x7f621015c060 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.304+0000 7f6215d1e640 1 -- 172.21.15.172:0/837626449 >> 172.21.15.172:0/837626449 conn(0x7f621006ec30 msgr2=0x7f621006b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:28.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.304+0000 7f6215d1e640 1 -- 172.21.15.172:0/837626449 shutdown_connections 2024-04-14T08:43:28.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.304+0000 7f6215d1e640 1 -- 172.21.15.172:0/837626449 wait complete. 2024-04-14T08:43:28.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.305+0000 7f6215d1e640 1 Processor -- start 2024-04-14T08:43:28.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.305+0000 7f6215d1e640 1 -- start start 2024-04-14T08:43:28.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f6215d1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62100b1df0 0x7f62100ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f6215d1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62100b21d0 0x7f621015d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f6215d1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62100ae830 0x7f621015dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f6215d1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62100b4700 con 0x7f62100b1df0 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f6215d1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f62100b4580 con 0x7f62100ae830 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f6215d1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f62100b4880 con 0x7f62100b21d0 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f620f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62100b1df0 0x7f62100ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f620f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62100b1df0 0x7f62100ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48008/0 (socket says 172.21.15.172:48008) 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f620f7fe640 1 -- 172.21.15.172:0/2785827584 learned_addr learned my addr 172.21.15.172:0/2785827584 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f620ffff640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62100ae830 0x7f621015dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f620effd640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62100b21d0 0x7f621015d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f620f7fe640 1 -- 172.21.15.172:0/2785827584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62100b21d0 msgr2=0x7f621015d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f620f7fe640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62100b21d0 0x7f621015d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f620f7fe640 1 -- 172.21.15.172:0/2785827584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62100ae830 msgr2=0x7f621015dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f620f7fe640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62100ae830 0x7f621015dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f620f7fe640 1 -- 172.21.15.172:0/2785827584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62100aedd0 con 0x7f62100b1df0 2024-04-14T08:43:28.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f620f7fe640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62100b1df0 0x7f62100ae2f0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f620000c3c0 tx=0x7f620000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:28.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.306+0000 7f61f7fff640 1 -- 172.21.15.172:0/2785827584 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62000052d0 con 0x7f62100b1df0 2024-04-14T08:43:28.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.307+0000 7f61f7fff640 1 -- 172.21.15.172:0/2785827584 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6200005490 con 0x7f62100b1df0 2024-04-14T08:43:28.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.307+0000 7f6215d1e640 1 -- 172.21.15.172:0/2785827584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62100af0b0 con 0x7f62100b1df0 2024-04-14T08:43:28.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.307+0000 7f61f7fff640 1 -- 172.21.15.172:0/2785827584 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200007260 con 0x7f62100b1df0 2024-04-14T08:43:28.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.307+0000 7f6215d1e640 1 -- 172.21.15.172:0/2785827584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621013d7e0 con 0x7f62100b1df0 2024-04-14T08:43:28.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.308+0000 7f6215d1e640 1 -- 172.21.15.172:0/2785827584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6210067dd0 con 0x7f62100b1df0 2024-04-14T08:43:28.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.308+0000 7f61f7fff640 1 -- 172.21.15.172:0/2785827584 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6200007400 con 0x7f62100b1df0 2024-04-14T08:43:28.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.308+0000 7f61f7fff640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f61fc06c8f0 0x7f61fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.309+0000 7f620effd640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f61fc06c8f0 0x7f61fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.309+0000 7f61f7fff640 1 -- 172.21.15.172:0/2785827584 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f620008ae50 con 0x7f62100b1df0 2024-04-14T08:43:28.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.309+0000 7f620effd640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f61fc06c8f0 0x7f61fc06edb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6204036000 tx=0x7f620403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:28.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.313+0000 7f61f7fff640 1 -- 172.21.15.172:0/2785827584 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6210067dd0 con 0x7f62100b1df0 2024-04-14T08:43:28.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.486+0000 7f6215d1e640 1 -- 172.21.15.172:0/2785827584 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6210068520 con 0x7f61fc06c8f0 2024-04-14T08:43:28.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.487+0000 7f61f7fff640 1 -- 172.21.15.172:0/2785827584 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13436 (secure 0 0 0) 0x7f6210068520 con 0x7f61fc06c8f0 2024-04-14T08:43:28.487 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:28.487 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:43:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.489+0000 7f6215d1e640 1 -- 172.21.15.172:0/2785827584 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f61fc06c8f0 msgr2=0x7f61fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.489+0000 7f6215d1e640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f61fc06c8f0 0x7f61fc06edb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6204036000 tx=0x7f620403b040 comp rx=0 tx=0).stop 2024-04-14T08:43:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.489+0000 7f6215d1e640 1 -- 172.21.15.172:0/2785827584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62100b1df0 msgr2=0x7f62100ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.489+0000 7f6215d1e640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62100b1df0 0x7f62100ae2f0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f620000c3c0 tx=0x7f620000c880 comp rx=0 tx=0).stop 2024-04-14T08:43:28.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.489+0000 7f6215d1e640 1 -- 172.21.15.172:0/2785827584 shutdown_connections 2024-04-14T08:43:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.489+0000 7f6215d1e640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f61fc06c8f0 0x7f61fc06edb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.489+0000 7f6215d1e640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62100ae830 0x7f621015dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.489+0000 7f6215d1e640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62100b21d0 0x7f621015d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.489+0000 7f6215d1e640 1 --2- 172.21.15.172:0/2785827584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62100b1df0 0x7f62100ae2f0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.489+0000 7f6215d1e640 1 -- 172.21.15.172:0/2785827584 >> 172.21.15.172:0/2785827584 conn(0x7f621006ec30 msgr2=0x7f62100728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.489+0000 7f6215d1e640 1 -- 172.21.15.172:0/2785827584 shutdown_connections 2024-04-14T08:43:28.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.489+0000 7f6215d1e640 1 -- 172.21.15.172:0/2785827584 wait complete. 2024-04-14T08:43:28.500 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-04-14T08:43:27.246942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278624,"kb_used_data":29192,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696096,"statfs":{"total":773094113280,"available":772808802304,"internally_reserved":0,"allocated":29892608,"data_stored":23502293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390540,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-175,"num_write_kb":-8303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001289"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":373,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-14T08:43:25.313532+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:43:25.313532+0000","last_peered":"2024-04-14T08:43:25.313532+0000","last_clean":"2024-04-14T08:43:25.313532+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:43:25.313532+0000","last_undegraded":"2024-04-14T08:43:25.313532+0000","last_fullsized":"2024-04-14T08:43:25.313532+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2097152,"data_stored":1290439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36728,"kb_used_data":5548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335112,"statfs":{"total":96636764160,"available":96599154688,"internally_reserved":0,"allocated":5681152,"data_stored":4888383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36732,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335108,"statfs":{"total":96636764160,"available":96599150592,"internally_reserved":0,"allocated":5685248,"data_stored":4887961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36728,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335112,"statfs":{"total":96636764160,"available":96599154688,"internally_reserved":0,"allocated":5685248,"data_stored":4882285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1499136,"data_stored":694648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":693502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36732,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335108,"statfs":{"total":96636764160,"available":96599150592,"internally_reserved":0,"allocated":5685248,"data_stored":4887828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2072576,"data_stored":1277247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:43:28.501 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:43:28.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.602+0000 7f8df5d1e640 1 Processor -- start 2024-04-14T08:43:28.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.603+0000 7f8df5d1e640 1 -- start start 2024-04-14T08:43:28.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.603+0000 7f8df5d1e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8df015e890 0x7f8df00d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.603+0000 7f8df5d1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8df00cf320 con 0x7f8df00cedd0 2024-04-14T08:43:28.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.603+0000 7f8df5d1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8df00cf600 con 0x7f8df00d2830 2024-04-14T08:43:28.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.603+0000 7f8df5d1e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f8df00cff30 con 0x7f8df015e890 2024-04-14T08:43:28.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.603+0000 7f8deffff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f8df00cedd0 0x7f8df00b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:51972/0 (socket says 172.21.15.172:51972) 2024-04-14T08:43:28.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deffff640 1 -- 172.21.15.172:0/1602848157 learned_addr learned my addr 172.21.15.172:0/1602848157 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:28.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3767183238 0 0) 0x7f8df00cf320 con 0x7f8df00cedd0 2024-04-14T08:43:28.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ddc003520 con 0x7f8df00cedd0 2024-04-14T08:43:28.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8de6ffd640 1 --2- 172.21.15.172:0/1602848157 >> v2:172.21.15.177:3300/0 conn(0x7f8df015e890 0x7f8df00d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1688661515 0 0) 0x7f8ddc003520 con 0x7f8df00cedd0 2024-04-14T08:43:28.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 >> v2:172.21.15.177:3300/0 conn(0x7f8df015e890 msgr2=0x7f8df00d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:28.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 --2- 172.21.15.172:0/1602848157 >> v2:172.21.15.177:3300/0 conn(0x7f8df015e890 0x7f8df00d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 >> v1:172.21.15.177:6789/0 conn(0x7f8df00d2830 legacy=0x7f8df00d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df00cf8e0 con 0x7f8df00cedd0 2024-04-14T08:43:28.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f8de80019f0 con 0x7f8df00cedd0 2024-04-14T08:43:28.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 >> v1:172.21.15.172:6789/0 conn(0x7f8df00cedd0 legacy=0x7f8df00b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 --2- 172.21.15.172:0/1602848157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ddc0039b0 0x7f8ddc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 --2- 172.21.15.172:0/1602848157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 0x7f8ddc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 --2- 172.21.15.172:0/1602848157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ddc0051c0 0x7f8ddc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8df00cff30 con 0x7f8ddc004800 2024-04-14T08:43:28.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc00c070 con 0x7f8ddc0051c0 2024-04-14T08:43:28.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8ddc00c270 con 0x7f8ddc0039b0 2024-04-14T08:43:28.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8de6ffd640 1 --2- 172.21.15.172:0/1602848157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 0x7f8ddc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8deffff640 1 --2- 172.21.15.172:0/1602848157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ddc0051c0 0x7f8ddc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8def7fe640 1 --2- 172.21.15.172:0/1602848157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ddc0039b0 0x7f8ddc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8de6ffd640 1 -- 172.21.15.172:0/1602848157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ddc0039b0 msgr2=0x7f8ddc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8de6ffd640 1 --2- 172.21.15.172:0/1602848157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ddc0039b0 0x7f8ddc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8de6ffd640 1 -- 172.21.15.172:0/1602848157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ddc0051c0 msgr2=0x7f8ddc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.604+0000 7f8de6ffd640 1 --2- 172.21.15.172:0/1602848157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ddc0051c0 0x7f8ddc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.605+0000 7f8deffff640 1 -- 172.21.15.172:0/1602848157 reap_dead start 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.605+0000 7f8de6ffd640 1 -- 172.21.15.172:0/1602848157 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ddc003520 con 0x7f8ddc004800 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.605+0000 7f8de6ffd640 1 --2- 172.21.15.172:0/1602848157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 0x7f8ddc004c80 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009ac0 tx=0x7f8ddc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239ce5e394fe8617 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.605+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8039070 con 0x7f8ddc004800 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.605+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd802c6b0 con 0x7f8ddc004800 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.605+0000 7f8deeffd640 1 -- 172.21.15.172:0/1602848157 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8009690 con 0x7f8ddc004800 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.606+0000 7f8df5d1e640 1 -- 172.21.15.172:0/1602848157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 msgr2=0x7f8ddc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.606+0000 7f8df5d1e640 1 --2- 172.21.15.172:0/1602848157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 0x7f8ddc004c80 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009ac0 tx=0x7f8ddc005830 comp rx=0 tx=0).stop 2024-04-14T08:43:28.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.606+0000 7f8df5d1e640 1 -- 172.21.15.172:0/1602848157 shutdown_connections 2024-04-14T08:43:28.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.606+0000 7f8df5d1e640 1 --2- 172.21.15.172:0/1602848157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 0x7f8ddc004c80 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.606+0000 7f8df5d1e640 1 -- 172.21.15.172:0/1602848157 >> 172.21.15.172:0/1602848157 conn(0x7f8df009d590 msgr2=0x7f8df00d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:28.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.606+0000 7f8df5d1e640 1 -- 172.21.15.172:0/1602848157 shutdown_connections 2024-04-14T08:43:28.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.606+0000 7f8df5d1e640 1 -- 172.21.15.172:0/1602848157 wait complete. 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.607+0000 7f8df5d1e640 1 Processor -- start 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.607+0000 7f8df5d1e640 1 -- start start 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8df5d1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 0x7f8df015eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8df5d1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8df00b2800 0x7f8df0160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8df5d1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8df00b2c60 0x7f8df0152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8df5d1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8df00d2c10 con 0x7f8ddc004800 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8df5d1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8df00d2350 con 0x7f8df00b2800 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8df5d1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f8df00d2d90 con 0x7f8df00b2c60 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8def7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 0x7f8df015eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8def7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 0x7f8df015eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48034/0 (socket says 172.21.15.172:48034) 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8def7fe640 1 -- 172.21.15.172:0/143557330 learned_addr learned my addr 172.21.15.172:0/143557330 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8deffff640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8df00b2c60 0x7f8df0152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8de6ffd640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8df00b2800 0x7f8df0160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8def7fe640 1 -- 172.21.15.172:0/143557330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8df00b2c60 msgr2=0x7f8df0152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8def7fe640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8df00b2c60 0x7f8df0152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8def7fe640 1 -- 172.21.15.172:0/143557330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8df00b2800 msgr2=0x7f8df0160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8def7fe640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8df00b2800 0x7f8df0160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.608+0000 7f8def7fe640 1 -- 172.21.15.172:0/143557330 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df01f49e0 con 0x7f8ddc004800 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.609+0000 7f8def7fe640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 0x7f8df015eca0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f8de000f5f0 tx=0x7f8de000f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.609+0000 7f8decff9640 1 -- 172.21.15.172:0/143557330 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de00052e0 con 0x7f8ddc004800 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.609+0000 7f8decff9640 1 -- 172.21.15.172:0/143557330 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de0018070 con 0x7f8ddc004800 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.609+0000 7f8df5d1e640 1 -- 172.21.15.172:0/143557330 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df01f4cc0 con 0x7f8ddc004800 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.609+0000 7f8decff9640 1 -- 172.21.15.172:0/143557330 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0003810 con 0x7f8ddc004800 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.609+0000 7f8df5d1e640 1 -- 172.21.15.172:0/143557330 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df01f5280 con 0x7f8ddc004800 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.610+0000 7f8df5d1e640 1 -- 172.21.15.172:0/143557330 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df01f5660 con 0x7f8ddc004800 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.610+0000 7f8decff9640 1 -- 172.21.15.172:0/143557330 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8de00039b0 con 0x7f8ddc004800 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.611+0000 7f8decff9640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8dd006c920 0x7f8dd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.611+0000 7f8de6ffd640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8dd006c920 0x7f8dd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.611+0000 7f8decff9640 1 -- 172.21.15.172:0/143557330 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f8de008b670 con 0x7f8ddc004800 2024-04-14T08:43:28.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.611+0000 7f8de6ffd640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8dd006c920 0x7f8dd006ede0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8036000 tx=0x7f8dd8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:28.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.615+0000 7f8decff9640 1 -- 172.21.15.172:0/143557330 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8de0059bf0 con 0x7f8ddc004800 2024-04-14T08:43:28.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.789+0000 7f8df5d1e640 1 -- 172.21.15.172:0/143557330 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8df0068520 con 0x7f8dd006c920 2024-04-14T08:43:28.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.789+0000 7f8decff9640 1 -- 172.21.15.172:0/143557330 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13436 (secure 0 0 0) 0x7f8df0068520 con 0x7f8dd006c920 2024-04-14T08:43:28.790 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:28.790 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:43:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.791+0000 7f8df5d1e640 1 -- 172.21.15.172:0/143557330 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8dd006c920 msgr2=0x7f8dd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.791+0000 7f8df5d1e640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8dd006c920 0x7f8dd006ede0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8036000 tx=0x7f8dd8030000 comp rx=0 tx=0).stop 2024-04-14T08:43:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.791+0000 7f8df5d1e640 1 -- 172.21.15.172:0/143557330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 msgr2=0x7f8df015eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.791+0000 7f8df5d1e640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 0x7f8df015eca0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f8de000f5f0 tx=0x7f8de000f9f0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.791+0000 7f8df5d1e640 1 -- 172.21.15.172:0/143557330 shutdown_connections 2024-04-14T08:43:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.791+0000 7f8df5d1e640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f8dd006c920 0x7f8dd006ede0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.791+0000 7f8df5d1e640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8df00b2c60 0x7f8df0152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.791+0000 7f8df5d1e640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8df00b2800 0x7f8df0160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.791+0000 7f8df5d1e640 1 --2- 172.21.15.172:0/143557330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ddc004800 0x7f8df015eca0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.791+0000 7f8df5d1e640 1 -- 172.21.15.172:0/143557330 >> 172.21.15.172:0/143557330 conn(0x7f8df009d590 msgr2=0x7f8df00b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.792+0000 7f8df5d1e640 1 -- 172.21.15.172:0/143557330 shutdown_connections 2024-04-14T08:43:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.792+0000 7f8df5d1e640 1 -- 172.21.15.172:0/143557330 wait complete. 2024-04-14T08:43:28.802 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-04-14T08:43:27.246942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278624,"kb_used_data":29192,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696096,"statfs":{"total":773094113280,"available":772808802304,"internally_reserved":0,"allocated":29892608,"data_stored":23502293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390540,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-175,"num_write_kb":-8303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001289"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":373,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-14T08:43:25.313532+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:43:25.313532+0000","last_peered":"2024-04-14T08:43:25.313532+0000","last_clean":"2024-04-14T08:43:25.313532+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:43:25.313532+0000","last_undegraded":"2024-04-14T08:43:25.313532+0000","last_fullsized":"2024-04-14T08:43:25.313532+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2097152,"data_stored":1290439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36728,"kb_used_data":5548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335112,"statfs":{"total":96636764160,"available":96599154688,"internally_reserved":0,"allocated":5681152,"data_stored":4888383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36732,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335108,"statfs":{"total":96636764160,"available":96599150592,"internally_reserved":0,"allocated":5685248,"data_stored":4887961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36728,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335112,"statfs":{"total":96636764160,"available":96599154688,"internally_reserved":0,"allocated":5685248,"data_stored":4882285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1499136,"data_stored":694648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":693502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36732,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335108,"statfs":{"total":96636764160,"available":96599150592,"internally_reserved":0,"allocated":5685248,"data_stored":4887828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2072576,"data_stored":1277247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:43:28.803 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-14T08:43:28.803 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:43:28.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.904+0000 7fbf0e9c8640 1 Processor -- start 2024-04-14T08:43:28.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.904+0000 7fbf0e9c8640 1 -- start start 2024-04-14T08:43:28.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf0e9c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf080b1df0 0x7fbf080b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf0e9c8640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf0815c770 con 0x7fbf080b21d0 2024-04-14T08:43:28.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf0e9c8640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf080b3430 con 0x7fbf080b4040 2024-04-14T08:43:28.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf0e9c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbf080b3700 con 0x7fbf080b1df0 2024-04-14T08:43:28.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf077fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fbf080b21d0 0x7fbf0815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:51980/0 (socket says 172.21.15.172:51980) 2024-04-14T08:43:28.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf077fe640 1 -- 172.21.15.172:0/701297528 learned_addr learned my addr 172.21.15.172:0/701297528 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:28.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf07fff640 1 --2- 172.21.15.172:0/701297528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf080b1df0 0x7fbf080b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4100887703 0 0) 0x7fbf0815c770 con 0x7fbf080b21d0 2024-04-14T08:43:28.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbef0003520 con 0x7fbf080b21d0 2024-04-14T08:43:28.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2769262383 0 0) 0x7fbef0003520 con 0x7fbf080b21d0 2024-04-14T08:43:28.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf080b1df0 msgr2=0x7fbf080b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:28.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 --2- 172.21.15.172:0/701297528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf080b1df0 0x7fbf080b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 >> v1:172.21.15.177:6789/0 conn(0x7fbf080b4040 legacy=0x7fbf080b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf080b39e0 con 0x7fbf080b21d0 2024-04-14T08:43:28.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fbefc001670 con 0x7fbf080b21d0 2024-04-14T08:43:28.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 >> v1:172.21.15.172:6789/0 conn(0x7fbf080b21d0 legacy=0x7fbf0815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 --2- 172.21.15.172:0/701297528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf080b1df0 0x7fbef00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 --2- 172.21.15.172:0/701297528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbef0003f20 0x7fbef0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 --2- 172.21.15.172:0/701297528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef0004d40 0x7fbef00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbf080b3700 con 0x7fbef0004d40 2024-04-14T08:43:28.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbef000ba20 con 0x7fbef0003f20 2024-04-14T08:43:28.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.905+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbef000bc20 con 0x7fbf080b1df0 2024-04-14T08:43:28.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.906+0000 7fbf07fff640 1 --2- 172.21.15.172:0/701297528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf080b1df0 0x7fbef00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.906+0000 7fbf0cf65640 1 --2- 172.21.15.172:0/701297528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef0004d40 0x7fbef00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.906+0000 7fbf077fe640 1 --2- 172.21.15.172:0/701297528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbef0003f20 0x7fbef0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.906+0000 7fbf0cf65640 1 -- 172.21.15.172:0/701297528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf080b1df0 msgr2=0x7fbef00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:28.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.906+0000 7fbf0cf65640 1 --2- 172.21.15.172:0/701297528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf080b1df0 0x7fbef00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.906+0000 7fbf0cf65640 1 -- 172.21.15.172:0/701297528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbef0003f20 msgr2=0x7fbef0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:28.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.906+0000 7fbf0cf65640 1 --2- 172.21.15.172:0/701297528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbef0003f20 0x7fbef0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.906+0000 7fbf0cf65640 1 -- 172.21.15.172:0/701297528 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef0003520 con 0x7fbef0004d40 2024-04-14T08:43:28.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.907+0000 7fbf0cf65640 1 --2- 172.21.15.172:0/701297528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef0004d40 0x7fbef00051c0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fbf00001f60 tx=0x7fbef0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b87315d237ef95 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:28.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.907+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00001b30 con 0x7fbef0004d40 2024-04-14T08:43:28.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.907+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0002d710 con 0x7fbef0004d40 2024-04-14T08:43:28.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.907+0000 7fbf06ffd640 1 -- 172.21.15.172:0/701297528 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0002db00 con 0x7fbef0004d40 2024-04-14T08:43:28.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.907+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/701297528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef0004d40 msgr2=0x7fbef00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.907+0000 7fbf0e9c8640 1 --2- 172.21.15.172:0/701297528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef0004d40 0x7fbef00051c0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fbf00001f60 tx=0x7fbef0005830 comp rx=0 tx=0).stop 2024-04-14T08:43:28.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.907+0000 7fbf0cf65640 1 -- 172.21.15.172:0/701297528 reap_dead start 2024-04-14T08:43:28.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.907+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/701297528 shutdown_connections 2024-04-14T08:43:28.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.907+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/701297528 >> 172.21.15.172:0/701297528 conn(0x7fbf0806ec30 msgr2=0x7fbf0806b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:28.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.907+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/701297528 shutdown_connections 2024-04-14T08:43:28.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.907+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/701297528 wait complete. 2024-04-14T08:43:28.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.908+0000 7fbf0e9c8640 1 Processor -- start 2024-04-14T08:43:28.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.909+0000 7fbf0e9c8640 1 -- start start 2024-04-14T08:43:28.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.909+0000 7fbf0e9c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef0004d40 0x7fbf0813e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.909+0000 7fbf0e9c8640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf08157310 0x7fbf080cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.909+0000 7fbf0e9c8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf080cfe10 0x7fbf08143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.909+0000 7fbf0e9c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0815b7f0 con 0x7fbef0004d40 2024-04-14T08:43:28.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.909+0000 7fbf0e9c8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0815bb40 con 0x7fbf080cfe10 2024-04-14T08:43:28.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.909+0000 7fbf0e9c8640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fbf080b2ef0 con 0x7fbf08157310 2024-04-14T08:43:28.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.909+0000 7fbf07fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef0004d40 0x7fbf0813e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.909+0000 7fbf07fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef0004d40 0x7fbf0813e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48058/0 (socket says 172.21.15.172:48058) 2024-04-14T08:43:28.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf07fff640 1 -- 172.21.15.172:0/2336053534 learned_addr learned my addr 172.21.15.172:0/2336053534 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:28.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf077fe640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf08157310 0x7fbf080cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf0cf65640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf080cfe10 0x7fbf08143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf07fff640 1 -- 172.21.15.172:0/2336053534 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf08157310 msgr2=0x7fbf080cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf07fff640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf08157310 0x7fbf080cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf07fff640 1 -- 172.21.15.172:0/2336053534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf080cfe10 msgr2=0x7fbf08143960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:28.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf07fff640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf080cfe10 0x7fbf08143960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:28.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf07fff640 1 -- 172.21.15.172:0/2336053534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0813dcf0 con 0x7fbef0004d40 2024-04-14T08:43:28.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf07fff640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef0004d40 0x7fbf0813e790 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fbef80119a0 tx=0x7fbef8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:28.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf04ff9640 1 -- 172.21.15.172:0/2336053534 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef800faf0 con 0x7fbef0004d40 2024-04-14T08:43:28.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf04ff9640 1 -- 172.21.15.172:0/2336053534 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef800fc90 con 0x7fbef0004d40 2024-04-14T08:43:28.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/2336053534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf080d0370 con 0x7fbef0004d40 2024-04-14T08:43:28.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf04ff9640 1 -- 172.21.15.172:0/2336053534 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8014440 con 0x7fbef0004d40 2024-04-14T08:43:28.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.910+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/2336053534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf08157df0 con 0x7fbef0004d40 2024-04-14T08:43:28.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.911+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/2336053534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0813e3c0 con 0x7fbef0004d40 2024-04-14T08:43:28.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.912+0000 7fbf04ff9640 1 -- 172.21.15.172:0/2336053534 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbef8012040 con 0x7fbef0004d40 2024-04-14T08:43:28.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.912+0000 7fbf04ff9640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbef406c920 0x7fbef406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:28.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.912+0000 7fbf04ff9640 1 -- 172.21.15.172:0/2336053534 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7fbef808c120 con 0x7fbef0004d40 2024-04-14T08:43:28.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.912+0000 7fbf077fe640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbef406c920 0x7fbef406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:28.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.912+0000 7fbf077fe640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbef406c920 0x7fbef406ede0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbf081445d0 tx=0x7fbefc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:28.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:28.917+0000 7fbf04ff9640 1 -- 172.21.15.172:0/2336053534 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbef8019030 con 0x7fbef0004d40 2024-04-14T08:43:29.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.088+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/2336053534 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf08068520 con 0x7fbef406c920 2024-04-14T08:43:29.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.088+0000 7fbf04ff9640 1 -- 172.21.15.172:0/2336053534 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13436 (secure 0 0 0) 0x7fbf08068520 con 0x7fbef406c920 2024-04-14T08:43:29.089 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:29.089 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:43:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.090+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/2336053534 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbef406c920 msgr2=0x7fbef406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.090+0000 7fbf0e9c8640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbef406c920 0x7fbef406ede0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fbf081445d0 tx=0x7fbefc009040 comp rx=0 tx=0).stop 2024-04-14T08:43:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.090+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/2336053534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef0004d40 msgr2=0x7fbf0813e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.090+0000 7fbf0e9c8640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef0004d40 0x7fbf0813e790 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fbef80119a0 tx=0x7fbef8011e60 comp rx=0 tx=0).stop 2024-04-14T08:43:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.090+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/2336053534 shutdown_connections 2024-04-14T08:43:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.090+0000 7fbf0e9c8640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fbef406c920 0x7fbef406ede0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.090+0000 7fbf0e9c8640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf080cfe10 0x7fbf08143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.090+0000 7fbf0e9c8640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf08157310 0x7fbf080cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.090+0000 7fbf0e9c8640 1 --2- 172.21.15.172:0/2336053534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbef0004d40 0x7fbf0813e790 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.090+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/2336053534 >> 172.21.15.172:0/2336053534 conn(0x7fbf0806ec30 msgr2=0x7fbf08072aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.090+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/2336053534 shutdown_connections 2024-04-14T08:43:29.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.091+0000 7fbf0e9c8640 1 -- 172.21.15.172:0/2336053534 wait complete. 2024-04-14T08:43:29.101 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-04-14T08:43:27.246942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278624,"kb_used_data":29192,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696096,"statfs":{"total":773094113280,"available":772808802304,"internally_reserved":0,"allocated":29892608,"data_stored":23502293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390540,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-175,"num_write_kb":-8303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001289"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":373,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-14T08:43:25.313532+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:43:25.313532+0000","last_peered":"2024-04-14T08:43:25.313532+0000","last_clean":"2024-04-14T08:43:25.313532+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:43:25.313532+0000","last_undegraded":"2024-04-14T08:43:25.313532+0000","last_fullsized":"2024-04-14T08:43:25.313532+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2097152,"data_stored":1290439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36728,"kb_used_data":5548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335112,"statfs":{"total":96636764160,"available":96599154688,"internally_reserved":0,"allocated":5681152,"data_stored":4888383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36732,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335108,"statfs":{"total":96636764160,"available":96599150592,"internally_reserved":0,"allocated":5685248,"data_stored":4887961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36728,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335112,"statfs":{"total":96636764160,"available":96599154688,"internally_reserved":0,"allocated":5685248,"data_stored":4882285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1499136,"data_stored":694648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":693502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36732,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335108,"statfs":{"total":96636764160,"available":96599150592,"internally_reserved":0,"allocated":5685248,"data_stored":4887828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2072576,"data_stored":1277247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:43:29.102 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-14T08:43:29.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.203+0000 7ff9eff1e640 1 Processor -- start 2024-04-14T08:43:29.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.203+0000 7ff9eff1e640 1 -- start start 2024-04-14T08:43:29.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.204+0000 7ff9eff1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9e80d2830 0x7ff9e80d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:29.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.204+0000 7ff9eff1e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e80d3150 con 0x7ff9e80cedd0 2024-04-14T08:43:29.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.204+0000 7ff9eff1e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e80cf320 con 0x7ff9e815e890 2024-04-14T08:43:29.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.204+0000 7ff9eff1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e80cff30 con 0x7ff9e80d2830 2024-04-14T08:43:29.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.204+0000 7ff9ee4bb640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff9e80cedd0 0x7ff9e80b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:51988/0 (socket says 172.21.15.172:51988) 2024-04-14T08:43:29.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.204+0000 7ff9ee4bb640 1 -- 172.21.15.172:0/1735042024 learned_addr learned my addr 172.21.15.172:0/1735042024 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:29.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1850071483 0 0) 0x7ff9e80d3150 con 0x7ff9e80cedd0 2024-04-14T08:43:29.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9d4003520 con 0x7ff9e80cedd0 2024-04-14T08:43:29.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9edcba640 1 --2- 172.21.15.172:0/1735042024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9e80d2830 0x7ff9e80d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:29.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 302137236 0 0) 0x7ff9d4003520 con 0x7ff9e80cedd0 2024-04-14T08:43:29.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9e80d2830 msgr2=0x7ff9e80d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:29.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 --2- 172.21.15.172:0/1735042024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9e80d2830 0x7ff9e80d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 >> v1:172.21.15.177:6789/0 conn(0x7ff9e815e890 legacy=0x7ff9e80d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:29.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e80cf600 con 0x7ff9e80cedd0 2024-04-14T08:43:29.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7ff9e0001670 con 0x7ff9e80cedd0 2024-04-14T08:43:29.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 >> v1:172.21.15.172:6789/0 conn(0x7ff9e80cedd0 legacy=0x7ff9e80b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:29.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 --2- 172.21.15.172:0/1735042024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d40039e0 0x7ff9d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:29.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 --2- 172.21.15.172:0/1735042024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9e80d2830 0x7ff9d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:29.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 --2- 172.21.15.172:0/1735042024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9d4004d40 0x7ff9d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:29.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e80cff30 con 0x7ff9d40039e0 2024-04-14T08:43:29.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d400bb30 con 0x7ff9d4004d40 2024-04-14T08:43:29.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d400bd30 con 0x7ff9e80d2830 2024-04-14T08:43:29.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9edcba640 1 --2- 172.21.15.172:0/1735042024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d40039e0 0x7ff9d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:29.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9ee4bb640 1 --2- 172.21.15.172:0/1735042024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9d4004d40 0x7ff9d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:29.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9ed4b9640 1 --2- 172.21.15.172:0/1735042024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9e80d2830 0x7ff9d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:29.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9edcba640 1 -- 172.21.15.172:0/1735042024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9e80d2830 msgr2=0x7ff9d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:29.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9edcba640 1 --2- 172.21.15.172:0/1735042024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9e80d2830 0x7ff9d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9edcba640 1 -- 172.21.15.172:0/1735042024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9d4004d40 msgr2=0x7ff9d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:29.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9edcba640 1 --2- 172.21.15.172:0/1735042024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9d4004d40 0x7ff9d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.205+0000 7ff9edcba640 1 -- 172.21.15.172:0/1735042024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d4003520 con 0x7ff9d40039e0 2024-04-14T08:43:29.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.206+0000 7ff9edcba640 1 --2- 172.21.15.172:0/1735042024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d40039e0 0x7ff9d4003e00 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009f90 tx=0x7ff9d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfb5db6b79f20d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:29.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.206+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4039070 con 0x7ff9d40039e0 2024-04-14T08:43:29.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.206+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e402cbd0 con 0x7ff9d40039e0 2024-04-14T08:43:29.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.206+0000 7ff9eccb8640 1 -- 172.21.15.172:0/1735042024 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4009c10 con 0x7ff9d40039e0 2024-04-14T08:43:29.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.207+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1735042024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d40039e0 msgr2=0x7ff9d4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:29.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.207+0000 7ff9eff1e640 1 --2- 172.21.15.172:0/1735042024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d40039e0 0x7ff9d4003e00 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009f90 tx=0x7ff9d4005830 comp rx=0 tx=0).stop 2024-04-14T08:43:29.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.207+0000 7ff9ee4bb640 1 -- 172.21.15.172:0/1735042024 reap_dead start 2024-04-14T08:43:29.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.207+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1735042024 shutdown_connections 2024-04-14T08:43:29.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.207+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1735042024 >> 172.21.15.172:0/1735042024 conn(0x7ff9e809d590 msgr2=0x7ff9e80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:29.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.207+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1735042024 shutdown_connections 2024-04-14T08:43:29.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.207+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1735042024 wait complete. 2024-04-14T08:43:29.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.208+0000 7ff9eff1e640 1 Processor -- start 2024-04-14T08:43:29.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.208+0000 7ff9eff1e640 1 -- start start 2024-04-14T08:43:29.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9eff1e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9d40039e0 0x7ff9e8155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:29.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9eff1e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9e8161320 0x7ff9e80c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:29.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9eff1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9e81617a0 0x7ff9e80c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:29.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9eff1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e80cfbc0 con 0x7ff9e81617a0 2024-04-14T08:43:29.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9eff1e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e80d2350 con 0x7ff9d40039e0 2024-04-14T08:43:29.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9eff1e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e80d2c10 con 0x7ff9e8161320 2024-04-14T08:43:29.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9ee4bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9e81617a0 0x7ff9e80c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:29.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9ee4bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9e81617a0 0x7ff9e80c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48072/0 (socket says 172.21.15.172:48072) 2024-04-14T08:43:29.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9ee4bb640 1 -- 172.21.15.172:0/1693110388 learned_addr learned my addr 172.21.15.172:0/1693110388 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:29.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9ee4bb640 1 -- 172.21.15.172:0/1693110388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9e8161320 msgr2=0x7ff9e80c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:29.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9ee4bb640 1 --2- 172.21.15.172:0/1693110388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9e8161320 0x7ff9e80c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9ee4bb640 1 -- 172.21.15.172:0/1693110388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9d40039e0 msgr2=0x7ff9e8155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:29.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9ee4bb640 1 --2- 172.21.15.172:0/1693110388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9d40039e0 0x7ff9e8155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9ee4bb640 1 -- 172.21.15.172:0/1693110388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e80c9ef0 con 0x7ff9e81617a0 2024-04-14T08:43:29.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9ee4bb640 1 --2- 172.21.15.172:0/1693110388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9e81617a0 0x7ff9e80c82c0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7ff9e808e4b0 tx=0x7ff9e000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:29.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9de7fc640 1 -- 172.21.15.172:0/1693110388 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0019070 con 0x7ff9e81617a0 2024-04-14T08:43:29.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9de7fc640 1 -- 172.21.15.172:0/1693110388 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e000c640 con 0x7ff9e81617a0 2024-04-14T08:43:29.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9de7fc640 1 -- 172.21.15.172:0/1693110388 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000ca10 con 0x7ff9e81617a0 2024-04-14T08:43:29.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1693110388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e80ca1d0 con 0x7ff9e81617a0 2024-04-14T08:43:29.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.209+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1693110388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e80ca790 con 0x7ff9e81617a0 2024-04-14T08:43:29.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.210+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1693110388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e80d2f90 con 0x7ff9e81617a0 2024-04-14T08:43:29.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.211+0000 7ff9de7fc640 1 -- 172.21.15.172:0/1693110388 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9e001c070 con 0x7ff9e81617a0 2024-04-14T08:43:29.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.211+0000 7ff9de7fc640 1 --2- 172.21.15.172:0/1693110388 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff9cc06c8f0 0x7ff9cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:29.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.211+0000 7ff9de7fc640 1 -- 172.21.15.172:0/1693110388 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7ff9e008af30 con 0x7ff9e81617a0 2024-04-14T08:43:29.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.211+0000 7ff9edcba640 1 --2- 172.21.15.172:0/1693110388 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff9cc06c8f0 0x7ff9cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:29.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.212+0000 7ff9edcba640 1 --2- 172.21.15.172:0/1693110388 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff9cc06c8f0 0x7ff9cc06edb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4036000 tx=0x7ff9e402e070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:29.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.216+0000 7ff9de7fc640 1 -- 172.21.15.172:0/1693110388 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff9e80d2f90 con 0x7ff9e81617a0 2024-04-14T08:43:29.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.394+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1693110388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9e8068520 con 0x7ff9e81617a0 2024-04-14T08:43:29.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.395+0000 7ff9de7fc640 1 -- 172.21.15.172:0/1693110388 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+13627 (secure 0 0 0) 0x7ff9e0059420 con 0x7ff9e81617a0 2024-04-14T08:43:29.395 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:29.396 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":121,"fsid":"8e7e3a2f-e38f-4857-b139-5df1f7a099dc","created":"2024-04-14T08:32:28.961865+0000","modified":"2024-04-14T08:43:26.310295+0000","last_up_change":"2024-04-14T08:32:40.123344+0000","last_in_change":"2024-04-14T08:32:31.201031+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T08:32:41.135898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6f1b9c1-f5ac-4793-a0f4-925d05e6981c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6801","nonce":3761711279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6803","nonce":3761711279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6807","nonce":3761711279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":3761711279},{"type":"v1","addr":"172.21.15.172:6805","nonce":3761711279}]},"public_addr":"172.21.15.172:6801/3761711279","cluster_addr":"172.21.15.172:6803/3761711279","heartbeat_back_addr":"172.21.15.172:6807/3761711279","heartbeat_front_addr":"172.21.15.172:6805/3761711279","state":["exists","up"]},{"osd":1,"uuid":"689180ef-9f72-41c1-b646-a24328801759","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6823","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6825","nonce":1219922421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6827","nonce":1219922421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6831","nonce":1219922421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1219922421},{"type":"v1","addr":"172.21.15.172:6829","nonce":1219922421}]},"public_addr":"172.21.15.172:6825/1219922421","cluster_addr":"172.21.15.172:6827/1219922421","heartbeat_back_addr":"172.21.15.172:6831/1219922421","heartbeat_front_addr":"172.21.15.172:6829/1219922421","state":["exists","up"]},{"osd":2,"uuid":"245caafe-6315-445d-8e4d-5ef247dfe476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6817","nonce":1609902500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6819","nonce":1609902500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6824","nonce":1609902500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":1609902500},{"type":"v1","addr":"172.21.15.172:6821","nonce":1609902500}]},"public_addr":"172.21.15.172:6817/1609902500","cluster_addr":"172.21.15.172:6819/1609902500","heartbeat_back_addr":"172.21.15.172:6824/1609902500","heartbeat_front_addr":"172.21.15.172:6821/1609902500","state":["exists","up"]},{"osd":3,"uuid":"0889e14a-47cc-4585-ab53-9d78a1bdf5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6809","nonce":2305139122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6811","nonce":2305139122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6815","nonce":2305139122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":2305139122},{"type":"v1","addr":"172.21.15.172:6813","nonce":2305139122}]},"public_addr":"172.21.15.172:6809/2305139122","cluster_addr":"172.21.15.172:6811/2305139122","heartbeat_back_addr":"172.21.15.172:6815/2305139122","heartbeat_front_addr":"172.21.15.172:6813/2305139122","state":["exists","up"]},{"osd":4,"uuid":"39793065-1412-483e-a616-b20f815c4a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6822","nonce":4180733146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6826","nonce":4180733146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6831","nonce":4180733146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":4180733146},{"type":"v1","addr":"172.21.15.177:6829","nonce":4180733146}]},"public_addr":"172.21.15.177:6822/4180733146","cluster_addr":"172.21.15.177:6826/4180733146","heartbeat_back_addr":"172.21.15.177:6831/4180733146","heartbeat_front_addr":"172.21.15.177:6829/4180733146","state":["exists","up"]},{"osd":5,"uuid":"f593b8e5-fd62-4917-bc8d-9c695c04b8ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6817","nonce":3172735715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6819","nonce":3172735715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6827","nonce":3172735715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":3172735715},{"type":"v1","addr":"172.21.15.177:6823","nonce":3172735715}]},"public_addr":"172.21.15.177:6817/3172735715","cluster_addr":"172.21.15.177:6819/3172735715","heartbeat_back_addr":"172.21.15.177:6827/3172735715","heartbeat_front_addr":"172.21.15.177:6823/3172735715","state":["exists","up"]},{"osd":6,"uuid":"3075e364-a72d-4130-9bc9-2fcd91b77c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6801","nonce":2961907928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6803","nonce":2961907928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6807","nonce":2961907928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2961907928},{"type":"v1","addr":"172.21.15.177:6805","nonce":2961907928}]},"public_addr":"172.21.15.177:6801/2961907928","cluster_addr":"172.21.15.177:6803/2961907928","heartbeat_back_addr":"172.21.15.177:6807/2961907928","heartbeat_front_addr":"172.21.15.177:6805/2961907928","state":["exists","up"]},{"osd":7,"uuid":"69516eae-c9e6-4c79-a191-bad7981ec9ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6809","nonce":3609985207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6811","nonce":3609985207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6815","nonce":3609985207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3609985207},{"type":"v1","addr":"172.21.15.177:6813","nonce":3609985207}]},"public_addr":"172.21.15.177:6809/3609985207","cluster_addr":"172.21.15.177:6811/3609985207","heartbeat_back_addr":"172.21.15.177:6815/3609985207","heartbeat_front_addr":"172.21.15.177:6813/3609985207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.261949+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.422285+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.359928+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.373513+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.416255+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.372956+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.293273+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T08:32:36.358250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6837/4065123823":"2024-04-15T08:43:23.305449+0000","172.21.15.172:6837/744853572":"2024-04-15T08:43:08.711642+0000","172.21.15.177:6835/3201353747":"2024-04-15T08:42:57.915856+0000","172.21.15.172:6835/4065123823":"2024-04-15T08:43:23.305449+0000","172.21.15.172:6835/744853572":"2024-04-15T08:43:08.711642+0000","172.21.15.177:6833/3201353747":"2024-04-15T08:42:57.915856+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T08:43:29.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.397+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1693110388 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff9cc06c8f0 msgr2=0x7ff9cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:29.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.397+0000 7ff9eff1e640 1 --2- 172.21.15.172:0/1693110388 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff9cc06c8f0 0x7ff9cc06edb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4036000 tx=0x7ff9e402e070 comp rx=0 tx=0).stop 2024-04-14T08:43:29.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.397+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1693110388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9e81617a0 msgr2=0x7ff9e80c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:29.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.397+0000 7ff9eff1e640 1 --2- 172.21.15.172:0/1693110388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9e81617a0 0x7ff9e80c82c0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7ff9e808e4b0 tx=0x7ff9e000eb90 comp rx=0 tx=0).stop 2024-04-14T08:43:29.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.397+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1693110388 shutdown_connections 2024-04-14T08:43:29.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.397+0000 7ff9eff1e640 1 --2- 172.21.15.172:0/1693110388 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7ff9cc06c8f0 0x7ff9cc06edb0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.397+0000 7ff9eff1e640 1 --2- 172.21.15.172:0/1693110388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9e81617a0 0x7ff9e80c82c0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.397+0000 7ff9eff1e640 1 --2- 172.21.15.172:0/1693110388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff9e8161320 0x7ff9e80c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.397+0000 7ff9eff1e640 1 --2- 172.21.15.172:0/1693110388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9d40039e0 0x7ff9e8155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:29.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.397+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1693110388 >> 172.21.15.172:0/1693110388 conn(0x7ff9e809d590 msgr2=0x7ff9e81575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:29.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.397+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1693110388 shutdown_connections 2024-04-14T08:43:29.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:29.397+0000 7ff9eff1e640 1 -- 172.21.15.172:0/1693110388 wait complete. 2024-04-14T08:43:30.409 INFO:tasks.ceph:Scrubbing osd.0 2024-04-14T08:43:30.410 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-14T08:43:30.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.526+0000 7f032d1f0640 1 Processor -- start 2024-04-14T08:43:30.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.526+0000 7f032d1f0640 1 -- start start 2024-04-14T08:43:30.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.527+0000 7f032d1f0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03280cee90 0x7f0328166060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.527+0000 7f032d1f0640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03280cdb90 con 0x7f03280cc340 2024-04-14T08:43:30.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.527+0000 7f032d1f0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03280cde70 con 0x7f032815b350 2024-04-14T08:43:30.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.527+0000 7f032d1f0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f03280cd940 con 0x7f03280cee90 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.527+0000 7f032659c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f03280cc340 0x7f03280ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43148/0 (socket says 172.21.15.172:43148) 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.527+0000 7f032659c640 1 -- 172.21.15.172:0/2934749305 learned_addr learned my addr 172.21.15.172:0/2934749305 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.527+0000 7f032759e640 1 --2- 172.21.15.172:0/2934749305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03280cee90 0x7f0328166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.527+0000 7f0325d9b640 1 -- 172.21.15.172:0/2934749305 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3539182154 0 0) 0x7f03280cdb90 con 0x7f03280cc340 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.527+0000 7f0325d9b640 1 -- 172.21.15.172:0/2934749305 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0310003520 con 0x7f03280cc340 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.527+0000 7f0325d9b640 1 -- 172.21.15.172:0/2934749305 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1689915956 0 0) 0x7f03280cde70 con 0x7f032815b350 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.528+0000 7f0325d9b640 1 -- 172.21.15.172:0/2934749305 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03280cdb90 con 0x7f032815b350 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.528+0000 7f032759e640 1 -- 172.21.15.172:0/2934749305 >> v1:172.21.15.177:6789/0 conn(0x7f032815b350 legacy=0x7f03280cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.528+0000 7f032759e640 1 -- 172.21.15.172:0/2934749305 >> v1:172.21.15.172:6789/0 conn(0x7f03280cc340 legacy=0x7f03280ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.528+0000 7f032759e640 1 -- 172.21.15.172:0/2934749305 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03280ce150 con 0x7f03280cee90 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.528+0000 7f032759e640 1 --2- 172.21.15.172:0/2934749305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03280cee90 0x7f0328166060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f03280cdac0 tx=0x7f032002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2347cd6df10df22 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.528+0000 7f0325d9b640 1 -- 172.21.15.172:0/2934749305 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320035020 con 0x7f03280cee90 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.529+0000 7f0325d9b640 1 -- 172.21.15.172:0/2934749305 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0320031800 con 0x7f03280cee90 2024-04-14T08:43:30.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.529+0000 7f0325d9b640 1 -- 172.21.15.172:0/2934749305 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320031bb0 con 0x7f03280cee90 2024-04-14T08:43:30.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.529+0000 7f032d1f0640 1 -- 172.21.15.172:0/2934749305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03280cee90 msgr2=0x7f0328166060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.529+0000 7f032d1f0640 1 --2- 172.21.15.172:0/2934749305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03280cee90 0x7f0328166060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f03280cdac0 tx=0x7f032002b8a0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.529+0000 7f032d1f0640 1 -- 172.21.15.172:0/2934749305 shutdown_connections 2024-04-14T08:43:30.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.529+0000 7f032d1f0640 1 --2- 172.21.15.172:0/2934749305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03280cee90 0x7f0328166060 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.529+0000 7f032d1f0640 1 -- 172.21.15.172:0/2934749305 >> 172.21.15.172:0/2934749305 conn(0x7f03280b1a50 msgr2=0x7f03280b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:30.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.529+0000 7f032d1f0640 1 -- 172.21.15.172:0/2934749305 shutdown_connections 2024-04-14T08:43:30.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.529+0000 7f032d1f0640 1 -- 172.21.15.172:0/2934749305 wait complete. 2024-04-14T08:43:30.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.531+0000 7f032d1f0640 1 Processor -- start 2024-04-14T08:43:30.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.531+0000 7f032d1f0640 1 -- start start 2024-04-14T08:43:30.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.531+0000 7f032d1f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03280cc340 0x7f032814fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.531+0000 7f032d1f0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03280cee90 0x7f032806dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.531+0000 7f032d1f0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f032815b350 0x7f032806e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.531+0000 7f032d1f0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f03280c5e80 con 0x7f03280cc340 2024-04-14T08:43:30.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.531+0000 7f032d1f0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f03280ce4c0 con 0x7f03280cee90 2024-04-14T08:43:30.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.531+0000 7f032d1f0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f03280c6000 con 0x7f032815b350 2024-04-14T08:43:30.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.531+0000 7f0326d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03280cc340 0x7f032814fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f0326d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03280cc340 0x7f032814fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50846/0 (socket says 172.21.15.172:50846) 2024-04-14T08:43:30.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f0326d9d640 1 -- 172.21.15.172:0/1310226199 learned_addr learned my addr 172.21.15.172:0/1310226199 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:30.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f032659c640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03280cee90 0x7f032806dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f032759e640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f032815b350 0x7f032806e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f0326d9d640 1 -- 172.21.15.172:0/1310226199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f032815b350 msgr2=0x7f032806e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f0326d9d640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f032815b350 0x7f032806e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f0326d9d640 1 -- 172.21.15.172:0/1310226199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03280cee90 msgr2=0x7f032806dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f0326d9d640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03280cee90 0x7f032806dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f0326d9d640 1 -- 172.21.15.172:0/1310226199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032809cf80 con 0x7f03280cc340 2024-04-14T08:43:30.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f0326d9d640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03280cc340 0x7f032814fe70 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f031800c8a0 tx=0x7f031800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:30.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f030f7fe640 1 -- 172.21.15.172:0/1310226199 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318003b70 con 0x7f03280cc340 2024-04-14T08:43:30.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f030f7fe640 1 -- 172.21.15.172:0/1310226199 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0318003d30 con 0x7f03280cc340 2024-04-14T08:43:30.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f030f7fe640 1 -- 172.21.15.172:0/1310226199 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031800aa90 con 0x7f03280cc340 2024-04-14T08:43:30.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f032d1f0640 1 -- 172.21.15.172:0/1310226199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032809d260 con 0x7f03280cc340 2024-04-14T08:43:30.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.532+0000 7f032d1f0640 1 -- 172.21.15.172:0/1310226199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032809d820 con 0x7f03280cc340 2024-04-14T08:43:30.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.533+0000 7f032d1f0640 1 -- 172.21.15.172:0/1310226199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03280686a0 con 0x7f03280cc340 2024-04-14T08:43:30.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.534+0000 7f030f7fe640 1 -- 172.21.15.172:0/1310226199 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f031800ac30 con 0x7f03280cc340 2024-04-14T08:43:30.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.534+0000 7f030f7fe640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f031406c8f0 0x7f031406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.534+0000 7f030f7fe640 1 -- 172.21.15.172:0/1310226199 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f031808c2c0 con 0x7f03280cc340 2024-04-14T08:43:30.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.534+0000 7f032659c640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f031406c8f0 0x7f031406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.534+0000 7f030f7fe640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x7f0314072ed0 0x7f0314075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.534+0000 7f030f7fe640 1 -- 172.21.15.172:0/1310226199 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0318004ec0 con 0x7f0314072ed0 2024-04-14T08:43:30.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.534+0000 7f030f7fe640 1 -- 172.21.15.172:0/1310226199 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f031808c750 con 0x7f03280cc340 2024-04-14T08:43:30.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.535+0000 7f032759e640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x7f0314072ed0 0x7f0314075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.535+0000 7f032659c640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f031406c8f0 0x7f031406edb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f031c0011e0 tx=0x7f031c007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:30.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.535+0000 7f032759e640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x7f0314072ed0 0x7f0314075310 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:30.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.535+0000 7f030f7fe640 1 -- 172.21.15.172:0/1310226199 <== osd.0 v2:172.21.15.172:6800/3761711279 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0318004ec0 con 0x7f0314072ed0 2024-04-14T08:43:30.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.548+0000 7f032d1f0640 1 -- 172.21.15.172:0/1310226199 --> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f03280b28a0 con 0x7f0314072ed0 2024-04-14T08:43:30.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.549+0000 7f030f7fe640 1 -- 172.21.15.172:0/1310226199 <== osd.0 v2:172.21.15.172:6800/3761711279 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f03280b28a0 con 0x7f0314072ed0 2024-04-14T08:43:30.549 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:43:30.549 INFO:teuthology.orchestra.run.smithi172.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-14T08:43:30.549 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:43:30.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.549+0000 7f032d1f0640 1 -- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x7f0314072ed0 msgr2=0x7f0314075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.549+0000 7f032d1f0640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:6800/3761711279,v1:172.21.15.172:6801/3761711279] conn(0x7f0314072ed0 0x7f0314075310 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.549+0000 7f032d1f0640 1 -- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f031406c8f0 msgr2=0x7f031406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.549+0000 7f032d1f0640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f031406c8f0 0x7f031406edb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f031c0011e0 tx=0x7f031c007d30 comp rx=0 tx=0).stop 2024-04-14T08:43:30.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.549+0000 7f032d1f0640 1 -- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03280cc340 msgr2=0x7f032814fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.549+0000 7f032d1f0640 1 --2- 172.21.15.172:0/1310226199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03280cc340 0x7f032814fe70 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f031800c8a0 tx=0x7f031800cd60 comp rx=0 tx=0).stop 2024-04-14T08:43:30.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.549+0000 7f032759e640 1 -- 172.21.15.172:0/1310226199 reap_dead start 2024-04-14T08:43:30.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.549+0000 7f032d1f0640 1 -- 172.21.15.172:0/1310226199 shutdown_connections 2024-04-14T08:43:30.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.549+0000 7f032d1f0640 1 -- 172.21.15.172:0/1310226199 >> 172.21.15.172:0/1310226199 conn(0x7f03280b1a50 msgr2=0x7f03280ae7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:30.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.549+0000 7f032d1f0640 1 -- 172.21.15.172:0/1310226199 shutdown_connections 2024-04-14T08:43:30.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.549+0000 7f032d1f0640 1 -- 172.21.15.172:0/1310226199 wait complete. 2024-04-14T08:43:30.560 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-14T08:43:30.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.661+0000 7f9dd5e84640 1 Processor -- start 2024-04-14T08:43:30.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.661+0000 7f9dd5e84640 1 -- start start 2024-04-14T08:43:30.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dd5e84640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dd00b23e0 0x7f9dd0140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dd5e84640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9dd015c380 0x7f9dd0140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dd5e84640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dd015b620 con 0x7f9dd013df30 2024-04-14T08:43:30.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dd5e84640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd00cf9b0 con 0x7f9dd015c380 2024-04-14T08:43:30.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dd5e84640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd01411f0 con 0x7f9dd00b23e0 2024-04-14T08:43:30.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dcffff640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9dd015c380 0x7f9dd0140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dcf7fe640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9dd013df30 0x7f9dd00b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43152/0 (socket says 172.21.15.172:43152) 2024-04-14T08:43:30.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dcf7fe640 1 -- 172.21.15.172:0/4007237260 learned_addr learned my addr 172.21.15.172:0/4007237260 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:30.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dceffd640 1 --2- 172.21.15.172:0/4007237260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dd00b23e0 0x7f9dd0140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dcffff640 1 -- 172.21.15.172:0/4007237260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dd00b23e0 msgr2=0x7f9dd0140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:30.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dcffff640 1 --2- 172.21.15.172:0/4007237260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dd00b23e0 0x7f9dd0140770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dcffff640 1 -- 172.21.15.172:0/4007237260 >> v1:172.21.15.172:6789/0 conn(0x7f9dd013df30 legacy=0x7f9dd00b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dcffff640 1 -- 172.21.15.172:0/4007237260 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd01414a0 con 0x7f9dd015c380 2024-04-14T08:43:30.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dce7fc640 1 -- 172.21.15.172:0/4007237260 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1206402117 0 0) 0x7f9dc0001110 con 0x7f9dd013df30 2024-04-14T08:43:30.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.662+0000 7f9dceffd640 1 --2- 172.21.15.172:0/4007237260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dd00b23e0 0x7f9dd0140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:30.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.663+0000 7f9dcffff640 1 --2- 172.21.15.172:0/4007237260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dd015c380 0x7f9dd0140cb0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009d90 tx=0x7f9dc802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6961401739e9fffc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:30.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.663+0000 7f9dce7fc640 1 -- 172.21.15.172:0/4007237260 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8039070 con 0x7f9dd015c380 2024-04-14T08:43:30.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.663+0000 7f9dce7fc640 1 -- 172.21.15.172:0/4007237260 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc802cc50 con 0x7f9dd015c380 2024-04-14T08:43:30.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.663+0000 7f9dce7fc640 1 -- 172.21.15.172:0/4007237260 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8034440 con 0x7f9dd015c380 2024-04-14T08:43:30.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.664+0000 7f9dd5e84640 1 -- 172.21.15.172:0/4007237260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dd015c380 msgr2=0x7f9dd0140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.664+0000 7f9dd5e84640 1 --2- 172.21.15.172:0/4007237260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dd015c380 0x7f9dd0140cb0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009d90 tx=0x7f9dc802bea0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.664+0000 7f9dd5e84640 1 -- 172.21.15.172:0/4007237260 shutdown_connections 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.664+0000 7f9dd5e84640 1 --2- 172.21.15.172:0/4007237260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dd00b23e0 0x7f9dd0140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.664+0000 7f9dd5e84640 1 --2- 172.21.15.172:0/4007237260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dd015c380 0x7f9dd0140cb0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.664+0000 7f9dd5e84640 1 -- 172.21.15.172:0/4007237260 >> 172.21.15.172:0/4007237260 conn(0x7f9dd006b520 msgr2=0x7f9dd006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.664+0000 7f9dd5e84640 1 -- 172.21.15.172:0/4007237260 shutdown_connections 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.664+0000 7f9dd5e84640 1 -- 172.21.15.172:0/4007237260 wait complete. 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.665+0000 7f9dd5e84640 1 Processor -- start 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.665+0000 7f9dd5e84640 1 -- start start 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dd5e84640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dd00b23e0 0x7f9dd01378f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dd5e84640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dd013df30 0x7f9dd0158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dd5e84640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dd015c380 0x7f9dd013c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dd5e84640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd015be60 con 0x7f9dd015c380 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dd5e84640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd015bb80 con 0x7f9dd013df30 2024-04-14T08:43:30.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dd5e84640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd013e310 con 0x7f9dd00b23e0 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dcffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dd015c380 0x7f9dd013c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dcffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dd015c380 0x7f9dd013c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50864/0 (socket says 172.21.15.172:50864) 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dcffff640 1 -- 172.21.15.172:0/1856621979 learned_addr learned my addr 172.21.15.172:0/1856621979 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dcf7fe640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dd00b23e0 0x7f9dd01378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dceffd640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dd013df30 0x7f9dd0158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dcffff640 1 -- 172.21.15.172:0/1856621979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dd00b23e0 msgr2=0x7f9dd01378f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dcffff640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dd00b23e0 0x7f9dd01378f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dcffff640 1 -- 172.21.15.172:0/1856621979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dd013df30 msgr2=0x7f9dd0158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dcffff640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dd013df30 0x7f9dd0158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dcffff640 1 -- 172.21.15.172:0/1856621979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd0157610 con 0x7f9dd015c380 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dceffd640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dd013df30 0x7f9dd0158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dcffff640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dd015c380 0x7f9dd013c190 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009eb0 tx=0x7f9dc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9db7fff640 1 -- 172.21.15.172:0/1856621979 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8039050 con 0x7f9dd015c380 2024-04-14T08:43:30.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9db7fff640 1 -- 172.21.15.172:0/1856621979 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc8045070 con 0x7f9dd015c380 2024-04-14T08:43:30.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9db7fff640 1 -- 172.21.15.172:0/1856621979 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc803e610 con 0x7f9dd015c380 2024-04-14T08:43:30.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.666+0000 7f9dd5e84640 1 -- 172.21.15.172:0/1856621979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd01578f0 con 0x7f9dd015c380 2024-04-14T08:43:30.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.667+0000 7f9dd5e84640 1 -- 172.21.15.172:0/1856621979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd0158880 con 0x7f9dd015c380 2024-04-14T08:43:30.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.667+0000 7f9dd5e84640 1 -- 172.21.15.172:0/1856621979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd0068590 con 0x7f9dd015c380 2024-04-14T08:43:30.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.668+0000 7f9db7fff640 1 -- 172.21.15.172:0/1856621979 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9dc803e7b0 con 0x7f9dd015c380 2024-04-14T08:43:30.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.668+0000 7f9db7fff640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9dbc06c8f0 0x7f9dbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.668+0000 7f9db7fff640 1 -- 172.21.15.172:0/1856621979 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f9dc80ae670 con 0x7f9dd015c380 2024-04-14T08:43:30.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.668+0000 7f9dcf7fe640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9dbc06c8f0 0x7f9dbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.669+0000 7f9dcf7fe640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9dbc06c8f0 0x7f9dbc06edb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0007fd0 tx=0x7f9dc0007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:30.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.673+0000 7f9db7fff640 1 -- 172.21.15.172:0/1856621979 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9dc807cc80 con 0x7f9dd015c380 2024-04-14T08:43:30.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.847+0000 7f9dd5e84640 1 -- 172.21.15.172:0/1856621979 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f9dd01573c0 con 0x7f9dbc06c8f0 2024-04-14T08:43:30.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.847+0000 7f9db7fff640 1 -- 172.21.15.172:0/1856621979 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9dd01573c0 con 0x7f9dbc06c8f0 2024-04-14T08:43:30.848 INFO:teuthology.orchestra.run.smithi172.stderr:instructed osd(s) 0 to deep-scrub 2024-04-14T08:43:30.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.850+0000 7f9dd5e84640 1 -- 172.21.15.172:0/1856621979 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9dbc06c8f0 msgr2=0x7f9dbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.850+0000 7f9dd5e84640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9dbc06c8f0 0x7f9dbc06edb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0007fd0 tx=0x7f9dc0007de0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.850+0000 7f9dd5e84640 1 -- 172.21.15.172:0/1856621979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dd015c380 msgr2=0x7f9dd013c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.850+0000 7f9dd5e84640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dd015c380 0x7f9dd013c190 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009eb0 tx=0x7f9dc8004a40 comp rx=0 tx=0).stop 2024-04-14T08:43:30.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.850+0000 7f9dd5e84640 1 -- 172.21.15.172:0/1856621979 shutdown_connections 2024-04-14T08:43:30.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.850+0000 7f9dd5e84640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f9dbc06c8f0 0x7f9dbc06edb0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.850+0000 7f9dd5e84640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dd015c380 0x7f9dd013c190 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.850+0000 7f9dd5e84640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9dd013df30 0x7f9dd0158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.850+0000 7f9dd5e84640 1 --2- 172.21.15.172:0/1856621979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9dd00b23e0 0x7f9dd01378f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.850+0000 7f9dd5e84640 1 -- 172.21.15.172:0/1856621979 >> 172.21.15.172:0/1856621979 conn(0x7f9dd006b520 msgr2=0x7f9dd015eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:30.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.850+0000 7f9dd5e84640 1 -- 172.21.15.172:0/1856621979 shutdown_connections 2024-04-14T08:43:30.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.850+0000 7f9dd5e84640 1 -- 172.21.15.172:0/1856621979 wait complete. 2024-04-14T08:43:30.861 INFO:tasks.ceph:Scrubbing osd.1 2024-04-14T08:43:30.861 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-14T08:43:30.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.965+0000 7f4316e60640 1 Processor -- start 2024-04-14T08:43:30.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.966+0000 7f4316e60640 1 -- start start 2024-04-14T08:43:30.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.966+0000 7f4316e60640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f431015b350 0x7f43100cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.966+0000 7f4316e60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43100cee90 0x7f43100c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.966+0000 7f4316e60640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43100cdb90 con 0x7f43100cc170 2024-04-14T08:43:30.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.966+0000 7f4316e60640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f43100cd940 con 0x7f431015b350 2024-04-14T08:43:30.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.966+0000 7f4316e60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f43100ce4c0 con 0x7f43100cee90 2024-04-14T08:43:30.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.967+0000 7f4314bfc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f431015b350 0x7f43100cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.967+0000 7f43153fd640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43100cee90 0x7f43100c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.967+0000 7f430ffff640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f43100cc170 0x7f43100ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:36408/0 (socket says 172.21.15.172:36408) 2024-04-14T08:43:30.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.967+0000 7f430ffff640 1 -- 172.21.15.172:0/1572395173 learned_addr learned my addr 172.21.15.172:0/1572395173 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:30.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.967+0000 7f4314bfc640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f431015b350 0x7f43100cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.172:51040/0 (socket says 172.21.15.172:51040) 2024-04-14T08:43:30.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.967+0000 7f43153fd640 1 -- 172.21.15.172:0/1572395173 >> v2:172.21.15.177:3300/0 conn(0x7f431015b350 msgr2=0x7f43100cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:30.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.967+0000 7f43153fd640 1 --2- 172.21.15.172:0/1572395173 >> v2:172.21.15.177:3300/0 conn(0x7f431015b350 0x7f43100cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.967+0000 7f43153fd640 1 -- 172.21.15.172:0/1572395173 >> v1:172.21.15.177:6789/0 conn(0x7f43100cc170 legacy=0x7f43100ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.967+0000 7f43153fd640 1 -- 172.21.15.172:0/1572395173 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43100cde70 con 0x7f43100cee90 2024-04-14T08:43:30.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.967+0000 7f4314bfc640 1 --2- 172.21.15.172:0/1572395173 >> v2:172.21.15.177:3300/0 conn(0x7f431015b350 0x7f43100cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:43:30.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.967+0000 7f430f7fe640 1 -- 172.21.15.172:0/1572395173 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3171958826 0 0) 0x7f43100cdb90 con 0x7f43100cc170 2024-04-14T08:43:30.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.968+0000 7f43153fd640 1 --2- 172.21.15.172:0/1572395173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43100cee90 0x7f43100c4e70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4308009d90 tx=0x7f430802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9de3350d67d5abe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:30.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.968+0000 7f430f7fe640 1 -- 172.21.15.172:0/1572395173 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308039070 con 0x7f43100cee90 2024-04-14T08:43:30.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.968+0000 7f430f7fe640 1 -- 172.21.15.172:0/1572395173 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430802cbc0 con 0x7f43100cee90 2024-04-14T08:43:30.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.968+0000 7f430f7fe640 1 -- 172.21.15.172:0/1572395173 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308034440 con 0x7f43100cee90 2024-04-14T08:43:30.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.968+0000 7f4316e60640 1 -- 172.21.15.172:0/1572395173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43100cee90 msgr2=0x7f43100c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.968+0000 7f4316e60640 1 --2- 172.21.15.172:0/1572395173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43100cee90 0x7f43100c4e70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4308009d90 tx=0x7f430802be10 comp rx=0 tx=0).stop 2024-04-14T08:43:30.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.968+0000 7f4316e60640 1 -- 172.21.15.172:0/1572395173 shutdown_connections 2024-04-14T08:43:30.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.968+0000 7f4316e60640 1 --2- 172.21.15.172:0/1572395173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43100cee90 0x7f43100c4e70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.968+0000 7f4316e60640 1 --2- 172.21.15.172:0/1572395173 >> v2:172.21.15.177:3300/0 conn(0x7f431015b350 0x7f43100cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.968+0000 7f4316e60640 1 -- 172.21.15.172:0/1572395173 >> 172.21.15.172:0/1572395173 conn(0x7f43100b1a50 msgr2=0x7f43100b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:30.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.969+0000 7f4316e60640 1 -- 172.21.15.172:0/1572395173 shutdown_connections 2024-04-14T08:43:30.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.969+0000 7f4316e60640 1 -- 172.21.15.172:0/1572395173 wait complete. 2024-04-14T08:43:30.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.970+0000 7f4316e60640 1 Processor -- start 2024-04-14T08:43:30.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.970+0000 7f4316e60640 1 -- start start 2024-04-14T08:43:30.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.970+0000 7f4316e60640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43100cc170 0x7f431009b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.970+0000 7f4316e60640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43100cee90 0x7f43101e7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.970+0000 7f4316e60640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f431015b350 0x7f43101e80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.970+0000 7f4316e60640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43100c53b0 con 0x7f43100cc170 2024-04-14T08:43:30.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.970+0000 7f4316e60640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f43100cf560 con 0x7f43100cee90 2024-04-14T08:43:30.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.970+0000 7f4316e60640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f43100c5530 con 0x7f431015b350 2024-04-14T08:43:30.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f4314bfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43100cc170 0x7f431009b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f4314bfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43100cc170 0x7f431009b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50878/0 (socket says 172.21.15.172:50878) 2024-04-14T08:43:30.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f4314bfc640 1 -- 172.21.15.172:0/2668971001 learned_addr learned my addr 172.21.15.172:0/2668971001 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:30.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f43153fd640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f431015b350 0x7f43101e80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f430ffff640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43100cee90 0x7f43101e7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f4314bfc640 1 -- 172.21.15.172:0/2668971001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f431015b350 msgr2=0x7f43101e80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f4314bfc640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f431015b350 0x7f43101e80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f4314bfc640 1 -- 172.21.15.172:0/2668971001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43100cee90 msgr2=0x7f43101e7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f4314bfc640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43100cee90 0x7f43101e7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f4314bfc640 1 -- 172.21.15.172:0/2668971001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431009d9f0 con 0x7f43100cc170 2024-04-14T08:43:30.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f4314bfc640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43100cc170 0x7f431009b480 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f430400e990 tx=0x7f430400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:30.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f430d7fa640 1 -- 172.21.15.172:0/2668971001 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304017070 con 0x7f43100cc170 2024-04-14T08:43:30.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f430d7fa640 1 -- 172.21.15.172:0/2668971001 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430400c9e0 con 0x7f43100cc170 2024-04-14T08:43:30.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f4316e60640 1 -- 172.21.15.172:0/2668971001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43100b1070 con 0x7f43100cc170 2024-04-14T08:43:30.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f430d7fa640 1 -- 172.21.15.172:0/2668971001 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430400cdb0 con 0x7f43100cc170 2024-04-14T08:43:30.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.971+0000 7f4316e60640 1 -- 172.21.15.172:0/2668971001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431009d1f0 con 0x7f43100cc170 2024-04-14T08:43:30.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.972+0000 7f4316e60640 1 -- 172.21.15.172:0/2668971001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43100686a0 con 0x7f43100cc170 2024-04-14T08:43:30.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.973+0000 7f430d7fa640 1 -- 172.21.15.172:0/2668971001 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f430401a070 con 0x7f43100cc170 2024-04-14T08:43:30.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.973+0000 7f430d7fa640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f42f406c8f0 0x7f42f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.973+0000 7f430d7fa640 1 -- 172.21.15.172:0/2668971001 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f430408b7e0 con 0x7f43100cc170 2024-04-14T08:43:30.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.973+0000 7f430ffff640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f42f406c8f0 0x7f42f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.973+0000 7f430d7fa640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x7f42f4072ed0 0x7f42f4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:30.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.973+0000 7f430d7fa640 1 -- 172.21.15.172:0/2668971001 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f430400e020 con 0x7f42f4072ed0 2024-04-14T08:43:30.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.973+0000 7f430d7fa640 1 -- 172.21.15.172:0/2668971001 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f43040902c0 con 0x7f43100cc170 2024-04-14T08:43:30.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.974+0000 7f43153fd640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x7f42f4072ed0 0x7f42f4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:30.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.974+0000 7f430ffff640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f42f406c8f0 0x7f42f406edb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f43100b1810 tx=0x7f42fc007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:30.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.974+0000 7f43153fd640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x7f42f4072ed0 0x7f42f4075310 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:30.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.974+0000 7f430d7fa640 1 -- 172.21.15.172:0/2668971001 <== osd.1 v2:172.21.15.172:6823/1219922421 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f430400e020 con 0x7f42f4072ed0 2024-04-14T08:43:30.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.987+0000 7f4316e60640 1 -- 172.21.15.172:0/2668971001 --> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f43100b28a0 con 0x7f42f4072ed0 2024-04-14T08:43:30.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.987+0000 7f430d7fa640 1 -- 172.21.15.172:0/2668971001 <== osd.1 v2:172.21.15.172:6823/1219922421 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f43100b28a0 con 0x7f42f4072ed0 2024-04-14T08:43:30.988 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:43:30.988 INFO:teuthology.orchestra.run.smithi172.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-14T08:43:30.988 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:43:30.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.988+0000 7f4316e60640 1 -- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x7f42f4072ed0 msgr2=0x7f42f4075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.988+0000 7f4316e60640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:6823/1219922421,v1:172.21.15.172:6825/1219922421] conn(0x7f42f4072ed0 0x7f42f4075310 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:30.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.988+0000 7f4316e60640 1 -- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f42f406c8f0 msgr2=0x7f42f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.988+0000 7f4316e60640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f42f406c8f0 0x7f42f406edb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f43100b1810 tx=0x7f42fc007d50 comp rx=0 tx=0).stop 2024-04-14T08:43:30.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.988+0000 7f4316e60640 1 -- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43100cc170 msgr2=0x7f431009b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:30.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.988+0000 7f4316e60640 1 --2- 172.21.15.172:0/2668971001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43100cc170 0x7f431009b480 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f430400e990 tx=0x7f430400ee50 comp rx=0 tx=0).stop 2024-04-14T08:43:30.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.988+0000 7f43153fd640 1 -- 172.21.15.172:0/2668971001 reap_dead start 2024-04-14T08:43:30.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.988+0000 7f4316e60640 1 -- 172.21.15.172:0/2668971001 shutdown_connections 2024-04-14T08:43:30.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.988+0000 7f4316e60640 1 -- 172.21.15.172:0/2668971001 >> 172.21.15.172:0/2668971001 conn(0x7f43100b1a50 msgr2=0x7f43100ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:30.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.988+0000 7f4316e60640 1 -- 172.21.15.172:0/2668971001 shutdown_connections 2024-04-14T08:43:30.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:30.988+0000 7f4316e60640 1 -- 172.21.15.172:0/2668971001 wait complete. 2024-04-14T08:43:30.998 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-14T08:43:31.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.100+0000 7f528f836640 1 Processor -- start 2024-04-14T08:43:31.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.101+0000 7f528f836640 1 -- start start 2024-04-14T08:43:31.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.101+0000 7f528f836640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52880cffb0 0x7f52880cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.101+0000 7f528f836640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52880d0860 con 0x7f52880ccdc0 2024-04-14T08:43:31.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.101+0000 7f528f836640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52880d0b40 con 0x7f52880d1a10 2024-04-14T08:43:31.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.101+0000 7f528f836640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f52880d0610 con 0x7f52880cffb0 2024-04-14T08:43:31.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.101+0000 7f528d5d2640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f52880ccdc0 0x7f52880cd1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43156/0 (socket says 172.21.15.172:43156) 2024-04-14T08:43:31.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.101+0000 7f528d5d2640 1 -- 172.21.15.172:0/711911464 learned_addr learned my addr 172.21.15.172:0/711911464 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:31.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.102+0000 7f528ddd3640 1 --2- 172.21.15.172:0/711911464 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52880cffb0 0x7f52880cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.102+0000 7f527ffff640 1 -- 172.21.15.172:0/711911464 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1449758043 0 0) 0x7f52880d0860 con 0x7f52880ccdc0 2024-04-14T08:43:31.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.102+0000 7f527ffff640 1 -- 172.21.15.172:0/711911464 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5274003520 con 0x7f52880ccdc0 2024-04-14T08:43:31.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.102+0000 7f527ffff640 1 -- 172.21.15.172:0/711911464 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3472920774 0 0) 0x7f52880d0b40 con 0x7f52880d1a10 2024-04-14T08:43:31.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.102+0000 7f527ffff640 1 -- 172.21.15.172:0/711911464 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52880d0860 con 0x7f52880d1a10 2024-04-14T08:43:31.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.102+0000 7f528ddd3640 1 -- 172.21.15.172:0/711911464 >> v1:172.21.15.177:6789/0 conn(0x7f52880d1a10 legacy=0x7f528815eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.102+0000 7f528ddd3640 1 -- 172.21.15.172:0/711911464 >> v1:172.21.15.172:6789/0 conn(0x7f52880ccdc0 legacy=0x7f52880cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.102+0000 7f528ddd3640 1 -- 172.21.15.172:0/711911464 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528815f1f0 con 0x7f52880cffb0 2024-04-14T08:43:31.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.103+0000 7f528ddd3640 1 --2- 172.21.15.172:0/711911464 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52880cffb0 0x7f52880cf190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f52800097d0 tx=0x7f528002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b86611ce88a09b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.103+0000 7f527ffff640 1 -- 172.21.15.172:0/711911464 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280035020 con 0x7f52880cffb0 2024-04-14T08:43:31.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.103+0000 7f527ffff640 1 -- 172.21.15.172:0/711911464 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5280031800 con 0x7f52880cffb0 2024-04-14T08:43:31.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.103+0000 7f527ffff640 1 -- 172.21.15.172:0/711911464 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280031bb0 con 0x7f52880cffb0 2024-04-14T08:43:31.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.103+0000 7f528f836640 1 -- 172.21.15.172:0/711911464 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52880cffb0 msgr2=0x7f52880cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.103+0000 7f528f836640 1 --2- 172.21.15.172:0/711911464 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52880cffb0 0x7f52880cf190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f52800097d0 tx=0x7f528002b8a0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.103+0000 7f528f836640 1 -- 172.21.15.172:0/711911464 shutdown_connections 2024-04-14T08:43:31.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.103+0000 7f528f836640 1 --2- 172.21.15.172:0/711911464 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52880cffb0 0x7f52880cf190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.103+0000 7f528f836640 1 -- 172.21.15.172:0/711911464 >> 172.21.15.172:0/711911464 conn(0x7f52880d0f20 msgr2=0x7f52880d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:31.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.103+0000 7f528f836640 1 -- 172.21.15.172:0/711911464 shutdown_connections 2024-04-14T08:43:31.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.103+0000 7f528f836640 1 -- 172.21.15.172:0/711911464 wait complete. 2024-04-14T08:43:31.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.104+0000 7f528f836640 1 Processor -- start 2024-04-14T08:43:31.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.105+0000 7f528f836640 1 -- start start 2024-04-14T08:43:31.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.105+0000 7f528f836640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52880ccdc0 0x7f52880a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.105+0000 7f528f836640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52880cffb0 0x7f52880c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.105+0000 7f528f836640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52880d1a10 0x7f52880b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.105+0000 7f528f836640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52880cf6d0 con 0x7f52880d1a10 2024-04-14T08:43:31.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.105+0000 7f528f836640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f52880ced00 con 0x7f52880cffb0 2024-04-14T08:43:31.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.105+0000 7f528f836640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f52880cf850 con 0x7f52880ccdc0 2024-04-14T08:43:31.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.105+0000 7f528ddd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52880d1a10 0x7f52880b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.105+0000 7f528ddd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52880d1a10 0x7f52880b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50880/0 (socket says 172.21.15.172:50880) 2024-04-14T08:43:31.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.105+0000 7f528ddd3640 1 -- 172.21.15.172:0/1853797719 learned_addr learned my addr 172.21.15.172:0/1853797719 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:31.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.105+0000 7f528d5d2640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52880ccdc0 0x7f52880a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.105+0000 7f528cdd1640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52880cffb0 0x7f52880c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.106+0000 7f528ddd3640 1 -- 172.21.15.172:0/1853797719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52880ccdc0 msgr2=0x7f52880a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.106+0000 7f528ddd3640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52880ccdc0 0x7f52880a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.106+0000 7f528ddd3640 1 -- 172.21.15.172:0/1853797719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52880cffb0 msgr2=0x7f52880c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.106+0000 7f528ddd3640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52880cffb0 0x7f52880c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.106+0000 7f528ddd3640 1 -- 172.21.15.172:0/1853797719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52880c8a90 con 0x7f52880d1a10 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.106+0000 7f528ddd3640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52880d1a10 0x7f52880b5d50 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f528002b8e0 tx=0x7f52800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.106+0000 7f527dffb640 1 -- 172.21.15.172:0/1853797719 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280041070 con 0x7f52880d1a10 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.106+0000 7f527dffb640 1 -- 172.21.15.172:0/1853797719 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5280031e30 con 0x7f52880d1a10 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.106+0000 7f528f836640 1 -- 172.21.15.172:0/1853797719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52880c8d70 con 0x7f52880d1a10 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.106+0000 7f527dffb640 1 -- 172.21.15.172:0/1853797719 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280035040 con 0x7f52880d1a10 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.106+0000 7f528f836640 1 -- 172.21.15.172:0/1853797719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52880c9330 con 0x7f52880d1a10 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.107+0000 7f528f836640 1 -- 172.21.15.172:0/1853797719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5288075490 con 0x7f52880d1a10 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.108+0000 7f527dffb640 1 -- 172.21.15.172:0/1853797719 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5280038020 con 0x7f52880d1a10 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.108+0000 7f527dffb640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f526c06c920 0x7f526c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.108+0000 7f527dffb640 1 -- 172.21.15.172:0/1853797719 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f52800ad410 con 0x7f52880d1a10 2024-04-14T08:43:31.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.108+0000 7f528d5d2640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f526c06c920 0x7f526c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.108+0000 7f528d5d2640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f526c06c920 0x7f526c06ede0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f52840012e0 tx=0x7f5284001a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.112+0000 7f527dffb640 1 -- 172.21.15.172:0/1853797719 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f528007b230 con 0x7f52880d1a10 2024-04-14T08:43:31.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.288+0000 7f528f836640 1 -- 172.21.15.172:0/1853797719 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f52880c96a0 con 0x7f526c06c920 2024-04-14T08:43:31.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.288+0000 7f527dffb640 1 -- 172.21.15.172:0/1853797719 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f52880c96a0 con 0x7f526c06c920 2024-04-14T08:43:31.288 INFO:teuthology.orchestra.run.smithi172.stderr:instructed osd(s) 1 to deep-scrub 2024-04-14T08:43:31.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.290+0000 7f528f836640 1 -- 172.21.15.172:0/1853797719 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f526c06c920 msgr2=0x7f526c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.290+0000 7f528f836640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f526c06c920 0x7f526c06ede0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f52840012e0 tx=0x7f5284001a30 comp rx=0 tx=0).stop 2024-04-14T08:43:31.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.290+0000 7f528f836640 1 -- 172.21.15.172:0/1853797719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52880d1a10 msgr2=0x7f52880b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.290+0000 7f528f836640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52880d1a10 0x7f52880b5d50 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f528002b8e0 tx=0x7f52800046c0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.290+0000 7f528f836640 1 -- 172.21.15.172:0/1853797719 shutdown_connections 2024-04-14T08:43:31.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.290+0000 7f528f836640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f526c06c920 0x7f526c06ede0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.290+0000 7f528f836640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52880d1a10 0x7f52880b5d50 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.290+0000 7f528f836640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f52880cffb0 0x7f52880c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.290+0000 7f528f836640 1 --2- 172.21.15.172:0/1853797719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52880ccdc0 0x7f52880a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.290+0000 7f528f836640 1 -- 172.21.15.172:0/1853797719 >> 172.21.15.172:0/1853797719 conn(0x7f52880d0f20 msgr2=0x7f52880706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:31.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.290+0000 7f528f836640 1 -- 172.21.15.172:0/1853797719 shutdown_connections 2024-04-14T08:43:31.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.290+0000 7f528f836640 1 -- 172.21.15.172:0/1853797719 wait complete. 2024-04-14T08:43:31.301 INFO:tasks.ceph:Scrubbing osd.2 2024-04-14T08:43:31.301 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-14T08:43:31.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.406+0000 7f09e27aa640 1 Processor -- start 2024-04-14T08:43:31.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.407+0000 7f09e27aa640 1 -- start start 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.407+0000 7f09e27aa640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f09dc0cc340 0x7f09dc0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.407+0000 7f09e27aa640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f09dc0cecc0 0x7f09dc0c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.407+0000 7f09e27aa640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09dc0cf0f0 con 0x7f09dc15b350 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.407+0000 7f09e27aa640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f09dc0cd940 con 0x7f09dc0cecc0 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.407+0000 7f09e27aa640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f09dc0ce4c0 con 0x7f09dc0cc340 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.408+0000 7f09e0d47640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f09dc0cecc0 0x7f09dc0c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.408+0000 7f09e0d47640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f09dc0cecc0 0x7f09dc0c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50890/0 (socket says 172.21.15.172:50890) 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.408+0000 7f09e0d47640 1 -- 172.21.15.172:0/3772394542 learned_addr learned my addr 172.21.15.172:0/3772394542 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.408+0000 7f09db7fe640 1 --2- 172.21.15.172:0/3772394542 >> v2:172.21.15.177:3300/0 conn(0x7f09dc0cc340 0x7f09dc0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.408+0000 7f09e0d47640 1 -- 172.21.15.172:0/3772394542 >> v2:172.21.15.177:3300/0 conn(0x7f09dc0cc340 msgr2=0x7f09dc0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.408+0000 7f09e0d47640 1 --2- 172.21.15.172:0/3772394542 >> v2:172.21.15.177:3300/0 conn(0x7f09dc0cc340 0x7f09dc0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.408+0000 7f09e0d47640 1 -- 172.21.15.172:0/3772394542 >> v1:172.21.15.177:6789/0 conn(0x7f09dc15b350 legacy=0x7f09dc0cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.408+0000 7f09e0d47640 1 -- 172.21.15.172:0/3772394542 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc0cdb90 con 0x7f09dc0cecc0 2024-04-14T08:43:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.408+0000 7f09db7fe640 1 --2- 172.21.15.172:0/3772394542 >> v2:172.21.15.177:3300/0 conn(0x7f09dc0cc340 0x7f09dc0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.409+0000 7f09e0d47640 1 --2- 172.21.15.172:0/3772394542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09dc0cecc0 0x7f09dc0c4e70 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0b25f0 tx=0x7f09d402e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b882b4a1b839e748 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.409+0000 7f09daffd640 1 -- 172.21.15.172:0/3772394542 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d402ea00 con 0x7f09dc0cecc0 2024-04-14T08:43:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.409+0000 7f09daffd640 1 -- 172.21.15.172:0/3772394542 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d402eba0 con 0x7f09dc0cecc0 2024-04-14T08:43:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.409+0000 7f09daffd640 1 -- 172.21.15.172:0/3772394542 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4009600 con 0x7f09dc0cecc0 2024-04-14T08:43:31.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.409+0000 7f09e27aa640 1 -- 172.21.15.172:0/3772394542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09dc0cecc0 msgr2=0x7f09dc0c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.409+0000 7f09e27aa640 1 --2- 172.21.15.172:0/3772394542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09dc0cecc0 0x7f09dc0c4e70 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0b25f0 tx=0x7f09d402e060 comp rx=0 tx=0).stop 2024-04-14T08:43:31.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.409+0000 7f09e27aa640 1 -- 172.21.15.172:0/3772394542 shutdown_connections 2024-04-14T08:43:31.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.409+0000 7f09e27aa640 1 --2- 172.21.15.172:0/3772394542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09dc0cecc0 0x7f09dc0c4e70 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.409+0000 7f09e27aa640 1 --2- 172.21.15.172:0/3772394542 >> v2:172.21.15.177:3300/0 conn(0x7f09dc0cc340 0x7f09dc0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.409+0000 7f09e27aa640 1 -- 172.21.15.172:0/3772394542 >> 172.21.15.172:0/3772394542 conn(0x7f09dc0b1a50 msgr2=0x7f09dc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.409+0000 7f09e27aa640 1 -- 172.21.15.172:0/3772394542 shutdown_connections 2024-04-14T08:43:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.409+0000 7f09e27aa640 1 -- 172.21.15.172:0/3772394542 wait complete. 2024-04-14T08:43:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.410+0000 7f09e27aa640 1 Processor -- start 2024-04-14T08:43:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.411+0000 7f09e27aa640 1 -- start start 2024-04-14T08:43:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.411+0000 7f09e27aa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09dc0cc340 0x7f09dc146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.411+0000 7f09e27aa640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09dc0cecc0 0x7f09dc072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.411+0000 7f09e27aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09dc15b350 0x7f09dc072860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.411+0000 7f09e27aa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc0cf560 con 0x7f09dc15b350 2024-04-14T08:43:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.411+0000 7f09e27aa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc0ce150 con 0x7f09dc0cc340 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.411+0000 7f09e27aa640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f09dc0c53b0 con 0x7f09dc0cecc0 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.411+0000 7f09e0d47640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09dc15b350 0x7f09dc072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09e0d47640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09dc15b350 0x7f09dc072860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50902/0 (socket says 172.21.15.172:50902) 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09e0d47640 1 -- 172.21.15.172:0/92419678 learned_addr learned my addr 172.21.15.172:0/92419678 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09dbfff640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09dc0cc340 0x7f09dc146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09db7fe640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09dc0cecc0 0x7f09dc072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09e0d47640 1 -- 172.21.15.172:0/92419678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09dc0cecc0 msgr2=0x7f09dc072320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09e0d47640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09dc0cecc0 0x7f09dc072320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09e0d47640 1 -- 172.21.15.172:0/92419678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09dc0cc340 msgr2=0x7f09dc146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09e0d47640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09dc0cc340 0x7f09dc146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09e0d47640 1 -- 172.21.15.172:0/92419678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc15b9f0 con 0x7f09dc15b350 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09e0d47640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09dc15b350 0x7f09dc072860 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f09d402e670 tx=0x7f09d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09d8ff9640 1 -- 172.21.15.172:0/92419678 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4045020 con 0x7f09dc15b350 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09d8ff9640 1 -- 172.21.15.172:0/92419678 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d4040660 con 0x7f09dc15b350 2024-04-14T08:43:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09d8ff9640 1 -- 172.21.15.172:0/92419678 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d400b720 con 0x7f09dc15b350 2024-04-14T08:43:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09e27aa640 1 -- 172.21.15.172:0/92419678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09dc15de90 con 0x7f09dc15b350 2024-04-14T08:43:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.412+0000 7f09e27aa640 1 -- 172.21.15.172:0/92419678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09dc15e450 con 0x7f09dc15b350 2024-04-14T08:43:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.413+0000 7f09e27aa640 1 -- 172.21.15.172:0/92419678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09dc15e650 con 0x7f09dc15b350 2024-04-14T08:43:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.414+0000 7f09d8ff9640 1 -- 172.21.15.172:0/92419678 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09d4048020 con 0x7f09dc15b350 2024-04-14T08:43:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.414+0000 7f09d8ff9640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f09c806c8f0 0x7f09c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.414+0000 7f09d8ff9640 1 -- 172.21.15.172:0/92419678 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f09d40b05e0 con 0x7f09dc15b350 2024-04-14T08:43:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.414+0000 7f09dbfff640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f09c806c8f0 0x7f09c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.414+0000 7f09d8ff9640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x7f09c8072ed0 0x7f09c8075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.414+0000 7f09d8ff9640 1 -- 172.21.15.172:0/92419678 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09d4049e70 con 0x7f09c8072ed0 2024-04-14T08:43:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.414+0000 7f09d8ff9640 1 -- 172.21.15.172:0/92419678 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f09d40b7050 con 0x7f09dc15b350 2024-04-14T08:43:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.414+0000 7f09db7fe640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x7f09c8072ed0 0x7f09c8075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.414+0000 7f09dbfff640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f09c806c8f0 0x7f09c806edb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f09cc006fd0 tx=0x7f09cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.415+0000 7f09db7fe640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x7f09c8072ed0 0x7f09c8075310 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.415+0000 7f09d8ff9640 1 -- 172.21.15.172:0/92419678 <== osd.2 v2:172.21.15.172:6816/1609902500 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09d4049e70 con 0x7f09c8072ed0 2024-04-14T08:43:31.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.428+0000 7f09e27aa640 1 -- 172.21.15.172:0/92419678 --> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f09dc146e20 con 0x7f09c8072ed0 2024-04-14T08:43:31.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.428+0000 7f09d8ff9640 1 -- 172.21.15.172:0/92419678 <== osd.2 v2:172.21.15.172:6816/1609902500 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f09dc146e20 con 0x7f09c8072ed0 2024-04-14T08:43:31.429 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:43:31.429 INFO:teuthology.orchestra.run.smithi172.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-14T08:43:31.429 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:43:31.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.429+0000 7f09e27aa640 1 -- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x7f09c8072ed0 msgr2=0x7f09c8075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.429+0000 7f09e27aa640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:6816/1609902500,v1:172.21.15.172:6817/1609902500] conn(0x7f09c8072ed0 0x7f09c8075310 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.429+0000 7f09e27aa640 1 -- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f09c806c8f0 msgr2=0x7f09c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.429+0000 7f09e27aa640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f09c806c8f0 0x7f09c806edb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f09cc006fd0 tx=0x7f09cc008040 comp rx=0 tx=0).stop 2024-04-14T08:43:31.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.429+0000 7f09e27aa640 1 -- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09dc15b350 msgr2=0x7f09dc072860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.429+0000 7f09e27aa640 1 --2- 172.21.15.172:0/92419678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09dc15b350 0x7f09dc072860 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f09d402e670 tx=0x7f09d4004a40 comp rx=0 tx=0).stop 2024-04-14T08:43:31.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.429+0000 7f09e0d47640 1 -- 172.21.15.172:0/92419678 reap_dead start 2024-04-14T08:43:31.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.429+0000 7f09e27aa640 1 -- 172.21.15.172:0/92419678 shutdown_connections 2024-04-14T08:43:31.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.429+0000 7f09e27aa640 1 -- 172.21.15.172:0/92419678 >> 172.21.15.172:0/92419678 conn(0x7f09dc0b1a50 msgr2=0x7f09dc06dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:31.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.429+0000 7f09e27aa640 1 -- 172.21.15.172:0/92419678 shutdown_connections 2024-04-14T08:43:31.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.429+0000 7f09e27aa640 1 -- 172.21.15.172:0/92419678 wait complete. 2024-04-14T08:43:31.439 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-14T08:43:31.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.542+0000 7f864cb88640 1 Processor -- start 2024-04-14T08:43:31.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.542+0000 7f864cb88640 1 -- start start 2024-04-14T08:43:31.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.542+0000 7f864cb88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8648162710 0x7f86480d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.542+0000 7f864cb88640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8648162b70 0x7f86480d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.542+0000 7f864cb88640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86480cefb0 con 0x7f864815e430 2024-04-14T08:43:31.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.542+0000 7f864cb88640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f86480cf290 con 0x7f8648162b70 2024-04-14T08:43:31.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.542+0000 7f864cb88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f86480cf570 con 0x7f8648162710 2024-04-14T08:43:31.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.543+0000 7f863ffff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8648162710 0x7f86480d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.543+0000 7f8646d9d640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8648162b70 0x7f86480d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.543+0000 7f863ffff640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8648162710 0x7f86480d1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.172:55822/0 (socket says 172.21.15.172:55822) 2024-04-14T08:43:31.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.543+0000 7f864659c640 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f864815e430 0x7f8648162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.172:36438/0 (socket says 172.21.15.172:36438) 2024-04-14T08:43:31.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.543+0000 7f863ffff640 1 -- 172.21.15.172:0/3121904719 learned_addr learned my addr 172.21.15.172:0/3121904719 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:31.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.543+0000 7f863ffff640 1 -- 172.21.15.172:0/3121904719 >> v2:172.21.15.177:3300/0 conn(0x7f8648162b70 msgr2=0x7f86480d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:31.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.543+0000 7f863ffff640 1 --2- 172.21.15.172:0/3121904719 >> v2:172.21.15.177:3300/0 conn(0x7f8648162b70 0x7f86480d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.543+0000 7f863ffff640 1 -- 172.21.15.172:0/3121904719 >> v1:172.21.15.177:6789/0 conn(0x7f864815e430 legacy=0x7f8648162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.543+0000 7f863ffff640 1 -- 172.21.15.172:0/3121904719 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86480cf850 con 0x7f8648162710 2024-04-14T08:43:31.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.543+0000 7f8646d9d640 1 --2- 172.21.15.172:0/3121904719 >> v2:172.21.15.177:3300/0 conn(0x7f8648162b70 0x7f86480d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:43:31.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.544+0000 7f863ffff640 1 --2- 172.21.15.172:0/3121904719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8648162710 0x7f86480d1df0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8630009870 tx=0x7f863002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a29141df52c3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.544+0000 7f8645d9b640 1 -- 172.21.15.172:0/3121904719 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630034070 con 0x7f8648162710 2024-04-14T08:43:31.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.544+0000 7f8645d9b640 1 -- 172.21.15.172:0/3121904719 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8630030840 con 0x7f8648162710 2024-04-14T08:43:31.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.544+0000 7f8645d9b640 1 -- 172.21.15.172:0/3121904719 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630030c10 con 0x7f8648162710 2024-04-14T08:43:31.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.544+0000 7f864cb88640 1 -- 172.21.15.172:0/3121904719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8648162710 msgr2=0x7f86480d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.544+0000 7f864cb88640 1 --2- 172.21.15.172:0/3121904719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8648162710 0x7f86480d1df0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8630009870 tx=0x7f863002b980 comp rx=0 tx=0).stop 2024-04-14T08:43:31.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.544+0000 7f864cb88640 1 -- 172.21.15.172:0/3121904719 shutdown_connections 2024-04-14T08:43:31.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.544+0000 7f864cb88640 1 --2- 172.21.15.172:0/3121904719 >> v2:172.21.15.177:3300/0 conn(0x7f8648162b70 0x7f86480d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.544+0000 7f864cb88640 1 --2- 172.21.15.172:0/3121904719 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8648162710 0x7f86480d1df0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.544+0000 7f864cb88640 1 -- 172.21.15.172:0/3121904719 >> 172.21.15.172:0/3121904719 conn(0x7f864809d560 msgr2=0x7f86480d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:31.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.545+0000 7f864cb88640 1 -- 172.21.15.172:0/3121904719 shutdown_connections 2024-04-14T08:43:31.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.545+0000 7f864cb88640 1 -- 172.21.15.172:0/3121904719 wait complete. 2024-04-14T08:43:31.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.546+0000 7f864cb88640 1 Processor -- start 2024-04-14T08:43:31.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.546+0000 7f864cb88640 1 -- start start 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.546+0000 7f864cb88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f864815e430 0x7f86480ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.546+0000 7f864cb88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8648162710 0x7f864814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.546+0000 7f864cb88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8648162b70 0x7f86480b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.546+0000 7f864cb88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f86480d4e20 con 0x7f8648162b70 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.546+0000 7f864cb88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f864815e810 con 0x7f864815e430 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f864cb88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f86480d0050 con 0x7f8648162710 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f8646d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8648162b70 0x7f86480b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f8646d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8648162b70 0x7f86480b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50904/0 (socket says 172.21.15.172:50904) 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f8646d9d640 1 -- 172.21.15.172:0/85401959 learned_addr learned my addr 172.21.15.172:0/85401959 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f864659c640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f864815e430 0x7f86480ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f863ffff640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8648162710 0x7f864814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f8646d9d640 1 -- 172.21.15.172:0/85401959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8648162710 msgr2=0x7f864814e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f8646d9d640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8648162710 0x7f864814e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f8646d9d640 1 -- 172.21.15.172:0/85401959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f864815e430 msgr2=0x7f86480ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f8646d9d640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f864815e430 0x7f86480ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f8646d9d640 1 -- 172.21.15.172:0/85401959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86480c8d20 con 0x7f8648162b70 2024-04-14T08:43:31.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f864659c640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f864815e430 0x7f86480ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:43:31.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f8646d9d640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8648162b70 0x7f86480b5d00 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f864000efc0 tx=0x7f864000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f863effd640 1 -- 172.21.15.172:0/85401959 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640018030 con 0x7f8648162b70 2024-04-14T08:43:31.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f863effd640 1 -- 172.21.15.172:0/85401959 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f864000cdf0 con 0x7f8648162b70 2024-04-14T08:43:31.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f864cb88640 1 -- 172.21.15.172:0/85401959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86480c9000 con 0x7f8648162b70 2024-04-14T08:43:31.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f863effd640 1 -- 172.21.15.172:0/85401959 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640003bb0 con 0x7f8648162b70 2024-04-14T08:43:31.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.547+0000 7f864cb88640 1 -- 172.21.15.172:0/85401959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86480c95c0 con 0x7f8648162b70 2024-04-14T08:43:31.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.548+0000 7f864cb88640 1 -- 172.21.15.172:0/85401959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f864815aad0 con 0x7f8648162b70 2024-04-14T08:43:31.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.549+0000 7f863effd640 1 -- 172.21.15.172:0/85401959 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f864001b070 con 0x7f8648162b70 2024-04-14T08:43:31.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.549+0000 7f863effd640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f862806c8f0 0x7f862806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.549+0000 7f863effd640 1 -- 172.21.15.172:0/85401959 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f864008c2d0 con 0x7f8648162b70 2024-04-14T08:43:31.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.549+0000 7f864659c640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f862806c8f0 0x7f862806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.550+0000 7f864659c640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f862806c8f0 0x7f862806edb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8638007fd0 tx=0x7f8638007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.554+0000 7f863effd640 1 -- 172.21.15.172:0/85401959 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8640010040 con 0x7f8648162b70 2024-04-14T08:43:31.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.730+0000 7f864cb88640 1 -- 172.21.15.172:0/85401959 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f8648068590 con 0x7f862806c8f0 2024-04-14T08:43:31.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.730+0000 7f863effd640 1 -- 172.21.15.172:0/85401959 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8648068590 con 0x7f862806c8f0 2024-04-14T08:43:31.730 INFO:teuthology.orchestra.run.smithi172.stderr:instructed osd(s) 2 to deep-scrub 2024-04-14T08:43:31.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.732+0000 7f864cb88640 1 -- 172.21.15.172:0/85401959 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f862806c8f0 msgr2=0x7f862806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.732+0000 7f864cb88640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f862806c8f0 0x7f862806edb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8638007fd0 tx=0x7f8638007e80 comp rx=0 tx=0).stop 2024-04-14T08:43:31.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.732+0000 7f864cb88640 1 -- 172.21.15.172:0/85401959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8648162b70 msgr2=0x7f86480b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.732+0000 7f864cb88640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8648162b70 0x7f86480b5d00 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f864000efc0 tx=0x7f864000c490 comp rx=0 tx=0).stop 2024-04-14T08:43:31.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.732+0000 7f864cb88640 1 -- 172.21.15.172:0/85401959 shutdown_connections 2024-04-14T08:43:31.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.732+0000 7f864cb88640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f862806c8f0 0x7f862806edb0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.732+0000 7f864cb88640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8648162b70 0x7f86480b5d00 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.732+0000 7f864cb88640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8648162710 0x7f864814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.732+0000 7f864cb88640 1 --2- 172.21.15.172:0/85401959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f864815e430 0x7f86480ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.732+0000 7f864cb88640 1 -- 172.21.15.172:0/85401959 >> 172.21.15.172:0/85401959 conn(0x7f864809d560 msgr2=0x7f86480b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:31.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.732+0000 7f864cb88640 1 -- 172.21.15.172:0/85401959 shutdown_connections 2024-04-14T08:43:31.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.732+0000 7f864cb88640 1 -- 172.21.15.172:0/85401959 wait complete. 2024-04-14T08:43:31.743 INFO:tasks.ceph:Scrubbing osd.3 2024-04-14T08:43:31.743 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-14T08:43:31.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.848+0000 7f4bf0820640 1 Processor -- start 2024-04-14T08:43:31.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.848+0000 7f4bf0820640 1 -- start start 2024-04-14T08:43:31.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.849+0000 7f4bf0820640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4bec0b3dd0 0x7f4bec162870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.849+0000 7f4bf0820640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bec0b4260 con 0x7f4bec06f350 2024-04-14T08:43:31.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.849+0000 7f4bf0820640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bec0b44d0 con 0x7f4bec06b520 2024-04-14T08:43:31.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.849+0000 7f4bf0820640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f4bec162db0 con 0x7f4bec0b3dd0 2024-04-14T08:43:31.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.849+0000 7f4be9d9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4bec0b3dd0 0x7f4bec162870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.849+0000 7f4bead9d640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4bec06f350 0x7f4bec06f760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43158/0 (socket says 172.21.15.172:43158) 2024-04-14T08:43:31.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.849+0000 7f4bead9d640 1 -- 172.21.15.172:0/2539255176 learned_addr learned my addr 172.21.15.172:0/2539255176 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:31.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.849+0000 7f4be9d9b640 1 -- 172.21.15.172:0/2539255176 >> v1:172.21.15.177:6789/0 conn(0x7f4bec06b520 legacy=0x7f4bec06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.849+0000 7f4be9d9b640 1 -- 172.21.15.172:0/2539255176 >> v1:172.21.15.172:6789/0 conn(0x7f4bec06f350 legacy=0x7f4bec06f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.849+0000 7f4be9d9b640 1 -- 172.21.15.172:0/2539255176 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec15ec10 con 0x7f4bec0b3dd0 2024-04-14T08:43:31.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.849+0000 7f4be959a640 1 -- 172.21.15.172:0/2539255176 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136577030 0 0) 0x7f4bec0b4260 con 0x7f4bec06f350 2024-04-14T08:43:31.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.850+0000 7f4be9d9b640 1 --2- 172.21.15.172:0/2539255176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bec0b3dd0 0x7f4bec162870 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4be00097a0 tx=0x7f4be002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d223a378fb4cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.850+0000 7f4be959a640 1 -- 172.21.15.172:0/2539255176 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0034070 con 0x7f4bec0b3dd0 2024-04-14T08:43:31.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.850+0000 7f4be959a640 1 -- 172.21.15.172:0/2539255176 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4be0030610 con 0x7f4bec0b3dd0 2024-04-14T08:43:31.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.850+0000 7f4be959a640 1 -- 172.21.15.172:0/2539255176 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be00309e0 con 0x7f4bec0b3dd0 2024-04-14T08:43:31.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.851+0000 7f4bf0820640 1 -- 172.21.15.172:0/2539255176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bec0b3dd0 msgr2=0x7f4bec162870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.851+0000 7f4bf0820640 1 --2- 172.21.15.172:0/2539255176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bec0b3dd0 0x7f4bec162870 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4be00097a0 tx=0x7f4be002b960 comp rx=0 tx=0).stop 2024-04-14T08:43:31.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.851+0000 7f4bf0820640 1 -- 172.21.15.172:0/2539255176 shutdown_connections 2024-04-14T08:43:31.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.851+0000 7f4bf0820640 1 --2- 172.21.15.172:0/2539255176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bec0b3dd0 0x7f4bec162870 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.851+0000 7f4bf0820640 1 -- 172.21.15.172:0/2539255176 >> 172.21.15.172:0/2539255176 conn(0x7f4bec0d0f20 msgr2=0x7f4bec0d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:31.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.851+0000 7f4bf0820640 1 -- 172.21.15.172:0/2539255176 shutdown_connections 2024-04-14T08:43:31.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.851+0000 7f4bf0820640 1 -- 172.21.15.172:0/2539255176 wait complete. 2024-04-14T08:43:31.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.852+0000 7f4bf0820640 1 Processor -- start 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.852+0000 7f4bf0820640 1 -- start start 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4bf0820640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06b520 0x7f4bec143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4bf0820640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bec06f350 0x7f4bec156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4bf0820640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bec0b3dd0 0x7f4bec157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4bf0820640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec15e8c0 con 0x7f4bec06f350 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4bf0820640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec15f260 con 0x7f4bec0b3dd0 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4bf0820640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4bec15f6f0 con 0x7f4bec06b520 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4be9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bec06f350 0x7f4bec156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4be9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bec06f350 0x7f4bec156d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50924/0 (socket says 172.21.15.172:50924) 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4be9d9b640 1 -- 172.21.15.172:0/4049456571 learned_addr learned my addr 172.21.15.172:0/4049456571 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4bead9d640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bec0b3dd0 0x7f4bec157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4bea59c640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06b520 0x7f4bec143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4be9d9b640 1 -- 172.21.15.172:0/4049456571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06b520 msgr2=0x7f4bec143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4be9d9b640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06b520 0x7f4bec143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4be9d9b640 1 -- 172.21.15.172:0/4049456571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bec0b3dd0 msgr2=0x7f4bec157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4be9d9b640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bec0b3dd0 0x7f4bec157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4be9d9b640 1 -- 172.21.15.172:0/4049456571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec158380 con 0x7f4bec06f350 2024-04-14T08:43:31.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.853+0000 7f4bea59c640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bec06b520 0x7f4bec143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:31.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.854+0000 7f4be9d9b640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bec06f350 0x7f4bec156d00 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f4be00098c0 tx=0x7f4be00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.854+0000 7f4bdaffd640 1 -- 172.21.15.172:0/4049456571 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0034040 con 0x7f4bec06f350 2024-04-14T08:43:31.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.854+0000 7f4bdaffd640 1 -- 172.21.15.172:0/4049456571 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4be0039a10 con 0x7f4bec06f350 2024-04-14T08:43:31.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.854+0000 7f4bdaffd640 1 -- 172.21.15.172:0/4049456571 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0039de0 con 0x7f4bec06f350 2024-04-14T08:43:31.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.854+0000 7f4bf0820640 1 -- 172.21.15.172:0/4049456571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec158620 con 0x7f4bec06f350 2024-04-14T08:43:31.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.854+0000 7f4bf0820640 1 -- 172.21.15.172:0/4049456571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec158be0 con 0x7f4bec06f350 2024-04-14T08:43:31.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.855+0000 7f4bf0820640 1 -- 172.21.15.172:0/4049456571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4bec0686a0 con 0x7f4bec06f350 2024-04-14T08:43:31.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.855+0000 7f4bdaffd640 1 -- 172.21.15.172:0/4049456571 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4be0037070 con 0x7f4bec06f350 2024-04-14T08:43:31.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.856+0000 7f4bdaffd640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4bcc06c8f0 0x7f4bcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.856+0000 7f4bdaffd640 1 -- 172.21.15.172:0/4049456571 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f4be00ae5d0 con 0x7f4bec06f350 2024-04-14T08:43:31.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.856+0000 7f4bea59c640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4bcc06c8f0 0x7f4bcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.856+0000 7f4bdaffd640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x7f4bcc072ed0 0x7f4bcc075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.856+0000 7f4bdaffd640 1 -- 172.21.15.172:0/4049456571 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4be000af30 con 0x7f4bcc072ed0 2024-04-14T08:43:31.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.856+0000 7f4bdaffd640 1 -- 172.21.15.172:0/4049456571 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f4be00aea60 con 0x7f4bec06f350 2024-04-14T08:43:31.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.856+0000 7f4bead9d640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x7f4bcc072ed0 0x7f4bcc075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.856+0000 7f4bea59c640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4bcc06c8f0 0x7f4bcc06edb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc007fd0 tx=0x7f4bdc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.856+0000 7f4bead9d640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x7f4bcc072ed0 0x7f4bcc075310 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.857+0000 7f4bdaffd640 1 -- 172.21.15.172:0/4049456571 <== osd.3 v2:172.21.15.172:6808/2305139122 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4be000af30 con 0x7f4bcc072ed0 2024-04-14T08:43:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.870+0000 7f4bf0820640 1 -- 172.21.15.172:0/4049456571 --> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4bec06c3d0 con 0x7f4bcc072ed0 2024-04-14T08:43:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.870+0000 7f4bdaffd640 1 -- 172.21.15.172:0/4049456571 <== osd.3 v2:172.21.15.172:6808/2305139122 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4bec06c3d0 con 0x7f4bcc072ed0 2024-04-14T08:43:31.871 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:43:31.871 INFO:teuthology.orchestra.run.smithi172.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-14T08:43:31.871 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:43:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.870+0000 7f4bf0820640 1 -- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x7f4bcc072ed0 msgr2=0x7f4bcc075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.870+0000 7f4bf0820640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:6808/2305139122,v1:172.21.15.172:6809/2305139122] conn(0x7f4bcc072ed0 0x7f4bcc075310 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.870+0000 7f4bf0820640 1 -- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4bcc06c8f0 msgr2=0x7f4bcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.870+0000 7f4bf0820640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4bcc06c8f0 0x7f4bcc06edb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc007fd0 tx=0x7f4bdc009040 comp rx=0 tx=0).stop 2024-04-14T08:43:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.870+0000 7f4bf0820640 1 -- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bec06f350 msgr2=0x7f4bec156d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.870+0000 7f4bf0820640 1 --2- 172.21.15.172:0/4049456571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bec06f350 0x7f4bec156d00 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f4be00098c0 tx=0x7f4be00046c0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.870+0000 7f4bead9d640 1 -- 172.21.15.172:0/4049456571 reap_dead start 2024-04-14T08:43:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.870+0000 7f4bf0820640 1 -- 172.21.15.172:0/4049456571 shutdown_connections 2024-04-14T08:43:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.870+0000 7f4bf0820640 1 -- 172.21.15.172:0/4049456571 >> 172.21.15.172:0/4049456571 conn(0x7f4bec0d0f20 msgr2=0x7f4bec0ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.871+0000 7f4bf0820640 1 -- 172.21.15.172:0/4049456571 shutdown_connections 2024-04-14T08:43:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.871+0000 7f4bf0820640 1 -- 172.21.15.172:0/4049456571 wait complete. 2024-04-14T08:43:31.881 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-14T08:43:31.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.982+0000 7f92fdab7640 1 Processor -- start 2024-04-14T08:43:31.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.982+0000 7f92fdab7640 1 -- start start 2024-04-14T08:43:31.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.982+0000 7f92fdab7640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f92f80ce200 0x7f92f81603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.982+0000 7f92fdab7640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f815cf10 con 0x7f92f8160930 2024-04-14T08:43:31.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.982+0000 7f92fdab7640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f815d1f0 con 0x7f92f8159c20 2024-04-14T08:43:31.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.982+0000 7f92fdab7640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f92f815fc80 con 0x7f92f80ce200 2024-04-14T08:43:31.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.983+0000 7f92f6ffd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f92f80ce200 0x7f92f81603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.983+0000 7f92f7fff640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f92f8160930 0x7f92f8072450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43168/0 (socket says 172.21.15.172:43168) 2024-04-14T08:43:31.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.983+0000 7f92f7fff640 1 -- 172.21.15.172:0/414438542 learned_addr learned my addr 172.21.15.172:0/414438542 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:31.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.983+0000 7f92f6ffd640 1 -- 172.21.15.172:0/414438542 >> v1:172.21.15.177:6789/0 conn(0x7f92f8159c20 legacy=0x7f92f815c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.983+0000 7f92f6ffd640 1 -- 172.21.15.172:0/414438542 >> v1:172.21.15.172:6789/0 conn(0x7f92f8160930 legacy=0x7f92f8072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.983+0000 7f92f6ffd640 1 -- 172.21.15.172:0/414438542 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f815d4d0 con 0x7f92f80ce200 2024-04-14T08:43:31.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.983+0000 7f92f67fc640 1 -- 172.21.15.172:0/414438542 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 549279950 0 0) 0x7f92f815cf10 con 0x7f92f8160930 2024-04-14T08:43:31.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.984+0000 7f92f6ffd640 1 --2- 172.21.15.172:0/414438542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92f80ce200 0x7f92f81603f0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0097a0 tx=0x7f92ec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618311dc420869dc server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.984+0000 7f92f67fc640 1 -- 172.21.15.172:0/414438542 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec034070 con 0x7f92f80ce200 2024-04-14T08:43:31.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.984+0000 7f92f67fc640 1 -- 172.21.15.172:0/414438542 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ec030610 con 0x7f92f80ce200 2024-04-14T08:43:31.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.984+0000 7f92f67fc640 1 -- 172.21.15.172:0/414438542 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec0309e0 con 0x7f92f80ce200 2024-04-14T08:43:31.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.984+0000 7f92fdab7640 1 -- 172.21.15.172:0/414438542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92f80ce200 msgr2=0x7f92f81603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.984+0000 7f92fdab7640 1 --2- 172.21.15.172:0/414438542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92f80ce200 0x7f92f81603f0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0097a0 tx=0x7f92ec02b960 comp rx=0 tx=0).stop 2024-04-14T08:43:31.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.984+0000 7f92fdab7640 1 -- 172.21.15.172:0/414438542 shutdown_connections 2024-04-14T08:43:31.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.984+0000 7f92fdab7640 1 --2- 172.21.15.172:0/414438542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92f80ce200 0x7f92f81603f0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.984+0000 7f92fdab7640 1 -- 172.21.15.172:0/414438542 >> 172.21.15.172:0/414438542 conn(0x7f92f806b520 msgr2=0x7f92f806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:31.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.984+0000 7f92fdab7640 1 -- 172.21.15.172:0/414438542 shutdown_connections 2024-04-14T08:43:31.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.984+0000 7f92fdab7640 1 -- 172.21.15.172:0/414438542 wait complete. 2024-04-14T08:43:31.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.985+0000 7f92fdab7640 1 Processor -- start 2024-04-14T08:43:31.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.986+0000 7f92fdab7640 1 -- start start 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.986+0000 7f92fdab7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f80ce200 0x7f92f80b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.986+0000 7f92fdab7640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92f8159c20 0x7f92f80b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.986+0000 7f92fdab7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92f815f130 0x7f92f8143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.986+0000 7f92fdab7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f92f8072b60 con 0x7f92f815f130 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.986+0000 7f92fdab7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f92f815ff60 con 0x7f92f80ce200 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.986+0000 7f92fdab7640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f92f8072ce0 con 0x7f92f8159c20 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.986+0000 7f92f7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92f815f130 0x7f92f8143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92f7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92f815f130 0x7f92f8143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50956/0 (socket says 172.21.15.172:50956) 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92f7fff640 1 -- 172.21.15.172:0/3273447725 learned_addr learned my addr 172.21.15.172:0/3273447725 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92f77fe640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f80ce200 0x7f92f80b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92f6ffd640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92f8159c20 0x7f92f80b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92f7fff640 1 -- 172.21.15.172:0/3273447725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92f8159c20 msgr2=0x7f92f80b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92f7fff640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92f8159c20 0x7f92f80b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92f7fff640 1 -- 172.21.15.172:0/3273447725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f80ce200 msgr2=0x7f92f80b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:31.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92f7fff640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f80ce200 0x7f92f80b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:31.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92f7fff640 1 -- 172.21.15.172:0/3273447725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f80b1cf0 con 0x7f92f815f130 2024-04-14T08:43:31.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92f7fff640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92f815f130 0x7f92f8143fd0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f92f000caf0 tx=0x7f92f0002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92dffff640 1 -- 172.21.15.172:0/3273447725 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f000abd0 con 0x7f92f815f130 2024-04-14T08:43:31.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92dffff640 1 -- 172.21.15.172:0/3273447725 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f000ad70 con 0x7f92f815f130 2024-04-14T08:43:31.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92fdab7640 1 -- 172.21.15.172:0/3273447725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f80b1450 con 0x7f92f815f130 2024-04-14T08:43:31.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92dffff640 1 -- 172.21.15.172:0/3273447725 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0012640 con 0x7f92f815f130 2024-04-14T08:43:31.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.987+0000 7f92fdab7640 1 -- 172.21.15.172:0/3273447725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f80b1a10 con 0x7f92f815f130 2024-04-14T08:43:31.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.988+0000 7f92fdab7640 1 -- 172.21.15.172:0/3273447725 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f8068590 con 0x7f92f815f130 2024-04-14T08:43:31.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.989+0000 7f92dffff640 1 -- 172.21.15.172:0/3273447725 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92f00127e0 con 0x7f92f815f130 2024-04-14T08:43:31.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.989+0000 7f92dffff640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f92e406c8f0 0x7f92e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:31.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.989+0000 7f92f77fe640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f92e406c8f0 0x7f92e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:31.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.989+0000 7f92dffff640 1 -- 172.21.15.172:0/3273447725 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f92f008be90 con 0x7f92f815f130 2024-04-14T08:43:31.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.990+0000 7f92f77fe640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f92e406c8f0 0x7f92e406edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f92e8000f60 tx=0x7f92e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:31.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:31.994+0000 7f92dffff640 1 -- 172.21.15.172:0/3273447725 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f92f0059d30 con 0x7f92f815f130 2024-04-14T08:43:32.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.170+0000 7f92fdab7640 1 -- 172.21.15.172:0/3273447725 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f92f8071e50 con 0x7f92e406c8f0 2024-04-14T08:43:32.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.170+0000 7f92dffff640 1 -- 172.21.15.172:0/3273447725 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f92f8071e50 con 0x7f92e406c8f0 2024-04-14T08:43:32.171 INFO:teuthology.orchestra.run.smithi172.stderr:instructed osd(s) 3 to deep-scrub 2024-04-14T08:43:32.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.172+0000 7f92fdab7640 1 -- 172.21.15.172:0/3273447725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f92e406c8f0 msgr2=0x7f92e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.172+0000 7f92fdab7640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f92e406c8f0 0x7f92e406edb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f92e8000f60 tx=0x7f92e8009040 comp rx=0 tx=0).stop 2024-04-14T08:43:32.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.172+0000 7f92fdab7640 1 -- 172.21.15.172:0/3273447725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92f815f130 msgr2=0x7f92f8143fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.172+0000 7f92fdab7640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92f815f130 0x7f92f8143fd0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f92f000caf0 tx=0x7f92f0002190 comp rx=0 tx=0).stop 2024-04-14T08:43:32.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.172+0000 7f92fdab7640 1 -- 172.21.15.172:0/3273447725 shutdown_connections 2024-04-14T08:43:32.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.172+0000 7f92fdab7640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f92e406c8f0 0x7f92e406edb0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.172+0000 7f92fdab7640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92f815f130 0x7f92f8143fd0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.172+0000 7f92fdab7640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92f8159c20 0x7f92f80b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.172+0000 7f92fdab7640 1 --2- 172.21.15.172:0/3273447725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f80ce200 0x7f92f80b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.172+0000 7f92fdab7640 1 -- 172.21.15.172:0/3273447725 >> 172.21.15.172:0/3273447725 conn(0x7f92f806b520 msgr2=0x7f92f80b5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:32.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.172+0000 7f92fdab7640 1 -- 172.21.15.172:0/3273447725 shutdown_connections 2024-04-14T08:43:32.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.172+0000 7f92fdab7640 1 -- 172.21.15.172:0/3273447725 wait complete. 2024-04-14T08:43:32.184 INFO:tasks.ceph:Scrubbing osd.4 2024-04-14T08:43:32.184 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-14T08:43:32.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.288+0000 7f372ac88640 1 Processor -- start 2024-04-14T08:43:32.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.288+0000 7f372ac88640 1 -- start start 2024-04-14T08:43:32.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.289+0000 7f372ac88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372406ecb0 0x7f372415a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.289+0000 7f372ac88640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f372406d2e0 0x7f37240ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.289+0000 7f372ac88640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f372406dc10 con 0x7f372406d740 2024-04-14T08:43:32.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.289+0000 7f372ac88640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f372406e820 con 0x7f372406d2e0 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.289+0000 7f372ac88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f372406def0 con 0x7f372406ecb0 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.289+0000 7f3729225640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f372406d740 0x7f372415e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43180/0 (socket says 172.21.15.172:43180) 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.289+0000 7f3729225640 1 -- 172.21.15.172:0/3248031309 learned_addr learned my addr 172.21.15.172:0/3248031309 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.289+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2002763615 0 0) 0x7f372406dc10 con 0x7f372406d740 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.289+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f370c003520 con 0x7f372406d740 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.289+0000 7f3728a24640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372406ecb0 0x7f372415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.289+0000 7f3723fff640 1 --2- 172.21.15.172:0/3248031309 >> v2:172.21.15.177:3300/0 conn(0x7f372406d2e0 0x7f37240ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2802109381 0 0) 0x7f370c003520 con 0x7f372406d740 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372406ecb0 msgr2=0x7f372415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372406ecb0 0x7f372415a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 >> v2:172.21.15.177:3300/0 conn(0x7f372406d2e0 msgr2=0x7f37240ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 --2- 172.21.15.172:0/3248031309 >> v2:172.21.15.177:3300/0 conn(0x7f372406d2e0 0x7f37240ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372406e1d0 con 0x7f372406d740 2024-04-14T08:43:32.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f371c0019f0 con 0x7f372406d740 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 >> v1:172.21.15.172:6789/0 conn(0x7f372406d740 legacy=0x7f372415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372406ecb0 0x7f370c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f370c003f30 0x7f370c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f370c004d40 0x7f370c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f372406e820 con 0x7f370c004d40 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f372406def0 con 0x7f370c003f30 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f370c00ba10 con 0x7f372406ecb0 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f3729225640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f370c004d40 0x7f370c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f3723fff640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f370c003f30 0x7f370c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f3728a24640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372406ecb0 0x7f370c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f3729225640 1 -- 172.21.15.172:0/3248031309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372406ecb0 msgr2=0x7f370c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f3729225640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372406ecb0 0x7f370c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f3729225640 1 -- 172.21.15.172:0/3248031309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f370c003f30 msgr2=0x7f370c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f3729225640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f370c003f30 0x7f370c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f3729225640 1 -- 172.21.15.172:0/3248031309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c003520 con 0x7f370c004d40 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.290+0000 7f3723fff640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f370c003f30 0x7f370c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.291+0000 7f3729225640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f370c004d40 0x7f370c0051c0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f372406db40 tx=0x7f370c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d5e6eddd014137 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.291+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c00a4e0 con 0x7f370c004d40 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.291+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371c034650 con 0x7f370c004d40 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.291+0000 7f37237fe640 1 -- 172.21.15.172:0/3248031309 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c034a40 con 0x7f370c004d40 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.292+0000 7f372ac88640 1 -- 172.21.15.172:0/3248031309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f370c004d40 msgr2=0x7f370c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.292+0000 7f372ac88640 1 --2- 172.21.15.172:0/3248031309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f370c004d40 0x7f370c0051c0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f372406db40 tx=0x7f370c005830 comp rx=0 tx=0).stop 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.292+0000 7f3729225640 1 -- 172.21.15.172:0/3248031309 reap_dead start 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.292+0000 7f372ac88640 1 -- 172.21.15.172:0/3248031309 shutdown_connections 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.292+0000 7f372ac88640 1 -- 172.21.15.172:0/3248031309 >> 172.21.15.172:0/3248031309 conn(0x7f37240b1a50 msgr2=0x7f37240b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.292+0000 7f372ac88640 1 -- 172.21.15.172:0/3248031309 shutdown_connections 2024-04-14T08:43:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.292+0000 7f372ac88640 1 -- 172.21.15.172:0/3248031309 wait complete. 2024-04-14T08:43:32.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.293+0000 7f372ac88640 1 Processor -- start 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.293+0000 7f372ac88640 1 -- start start 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f372ac88640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3724148fc0 0x7f37241497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f372ac88640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3724072090 0x7f37240724f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f372ac88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f372414b860 0x7f372414bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f372ac88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f372415ed40 con 0x7f372414b860 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f372ac88640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f372415ebc0 con 0x7f3724072090 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f372ac88640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f37240ce890 con 0x7f3724148fc0 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f3729225640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f372414b860 0x7f372414bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f3729225640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f372414b860 0x7f372414bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50968/0 (socket says 172.21.15.172:50968) 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f3729225640 1 -- 172.21.15.172:0/1008639914 learned_addr learned my addr 172.21.15.172:0/1008639914 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f3728a24640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3724148fc0 0x7f37241497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f3723fff640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3724072090 0x7f37240724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f3729225640 1 -- 172.21.15.172:0/1008639914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3724148fc0 msgr2=0x7f37241497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f3729225640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3724148fc0 0x7f37241497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f3729225640 1 -- 172.21.15.172:0/1008639914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3724072090 msgr2=0x7f37240724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f3729225640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3724072090 0x7f37240724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f3729225640 1 -- 172.21.15.172:0/1008639914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37240ca010 con 0x7f372414b860 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f3729225640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f372414b860 0x7f372414bd10 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f371c00a0b0 tx=0x7f371c001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f37217fa640 1 -- 172.21.15.172:0/1008639914 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c039040 con 0x7f372414b860 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f37217fa640 1 -- 172.21.15.172:0/1008639914 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371c045070 con 0x7f372414b860 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f372ac88640 1 -- 172.21.15.172:0/1008639914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37240c6e40 con 0x7f372414b860 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.294+0000 7f37217fa640 1 -- 172.21.15.172:0/1008639914 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718009690 con 0x7f372414b860 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.295+0000 7f372ac88640 1 -- 172.21.15.172:0/1008639914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37240c7400 con 0x7f372414b860 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.295+0000 7f372ac88640 1 -- 172.21.15.172:0/1008639914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37240686a0 con 0x7f372414b860 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.296+0000 7f37217fa640 1 -- 172.21.15.172:0/1008639914 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f371c034a40 con 0x7f372414b860 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.296+0000 7f37217fa640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f371006c8f0 0x7f371006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.296+0000 7f3728a24640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f371006c8f0 0x7f371006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.296+0000 7f37217fa640 1 -- 172.21.15.172:0/1008639914 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f371c0add30 con 0x7f372414b860 2024-04-14T08:43:32.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.296+0000 7f37217fa640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x7f3710072ed0 0x7f3710075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.296+0000 7f37217fa640 1 -- 172.21.15.172:0/1008639914 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f371c009f00 con 0x7f3710072ed0 2024-04-14T08:43:32.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.296+0000 7f37217fa640 1 -- 172.21.15.172:0/1008639914 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f371c0b0430 con 0x7f372414b860 2024-04-14T08:43:32.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.297+0000 7f3728a24640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f371006c8f0 0x7f371006edb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3714009bc0 tx=0x7f3714009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.297+0000 7f3723fff640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x7f3710072ed0 0x7f3710075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.297+0000 7f3723fff640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x7f3710072ed0 0x7f3710075310 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.298+0000 7f37217fa640 1 -- 172.21.15.172:0/1008639914 <== osd.4 v2:172.21.15.177:6820/4180733146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f371c009f00 con 0x7f3710072ed0 2024-04-14T08:43:32.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.311+0000 7f372ac88640 1 -- 172.21.15.172:0/1008639914 --> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f37240b28a0 con 0x7f3710072ed0 2024-04-14T08:43:32.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.311+0000 7f37217fa640 1 -- 172.21.15.172:0/1008639914 <== osd.4 v2:172.21.15.177:6820/4180733146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f37240b28a0 con 0x7f3710072ed0 2024-04-14T08:43:32.315 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:43:32.316 INFO:teuthology.orchestra.run.smithi172.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-14T08:43:32.316 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:43:32.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.311+0000 7f372ac88640 1 -- 172.21.15.172:0/1008639914 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x7f3710072ed0 msgr2=0x7f3710075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.311+0000 7f372ac88640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.177:6820/4180733146,v1:172.21.15.177:6822/4180733146] conn(0x7f3710072ed0 0x7f3710075310 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.312+0000 7f372ac88640 1 -- 172.21.15.172:0/1008639914 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f371006c8f0 msgr2=0x7f371006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.312+0000 7f372ac88640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f371006c8f0 0x7f371006edb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3714009bc0 tx=0x7f3714009190 comp rx=0 tx=0).stop 2024-04-14T08:43:32.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.312+0000 7f372ac88640 1 -- 172.21.15.172:0/1008639914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f372414b860 msgr2=0x7f372414bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.312+0000 7f372ac88640 1 --2- 172.21.15.172:0/1008639914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f372414b860 0x7f372414bd10 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f371c00a0b0 tx=0x7f371c001360 comp rx=0 tx=0).stop 2024-04-14T08:43:32.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.312+0000 7f3729225640 1 -- 172.21.15.172:0/1008639914 reap_dead start 2024-04-14T08:43:32.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.312+0000 7f372ac88640 1 -- 172.21.15.172:0/1008639914 shutdown_connections 2024-04-14T08:43:32.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.312+0000 7f372ac88640 1 -- 172.21.15.172:0/1008639914 >> 172.21.15.172:0/1008639914 conn(0x7f37240b1a50 msgr2=0x7f37241599f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:32.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.312+0000 7f372ac88640 1 -- 172.21.15.172:0/1008639914 shutdown_connections 2024-04-14T08:43:32.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.312+0000 7f372ac88640 1 -- 172.21.15.172:0/1008639914 wait complete. 2024-04-14T08:43:32.323 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-14T08:43:32.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.424+0000 7fabca077640 1 Processor -- start 2024-04-14T08:43:32.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.424+0000 7fabca077640 1 -- start start 2024-04-14T08:43:32.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.425+0000 7fabca077640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fabc413df30 0x7fabc40b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.425+0000 7fabca077640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabc4140e80 0x7fabc40b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.425+0000 7fabca077640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc415b620 con 0x7fabc415c380 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.425+0000 7fabca077640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fabc40cf9b0 con 0x7fabc413df30 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.425+0000 7fabca077640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fabc41414a0 con 0x7fabc4140e80 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.425+0000 7fabc37fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fabc413df30 0x7fabc40b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.425+0000 7fabc2ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fabc415c380 0x7fabc4140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43196/0 (socket says 172.21.15.172:43196) 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.425+0000 7fabc2ffd640 1 -- 172.21.15.172:0/1592217383 learned_addr learned my addr 172.21.15.172:0/1592217383 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.426+0000 7fabc37fe640 1 -- 172.21.15.172:0/1592217383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabc4140e80 msgr2=0x7fabc40b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.426+0000 7fabc37fe640 1 --2- 172.21.15.172:0/1592217383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabc4140e80 0x7fabc40b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.426+0000 7fabc37fe640 1 -- 172.21.15.172:0/1592217383 >> v1:172.21.15.172:6789/0 conn(0x7fabc415c380 legacy=0x7fabc4140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.426+0000 7fabc37fe640 1 -- 172.21.15.172:0/1592217383 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc415b8a0 con 0x7fabc413df30 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.426+0000 7fabc27fc640 1 -- 172.21.15.172:0/1592217383 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3346646297 0 0) 0x7fabb8001110 con 0x7fabc415c380 2024-04-14T08:43:32.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.426+0000 7fabc37fe640 1 --2- 172.21.15.172:0/1592217383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc413df30 0x7fabc40b1cd0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fabb40097a0 tx=0x7fabb402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a639ceffa1b8047 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.426+0000 7fabc27fc640 1 -- 172.21.15.172:0/1592217383 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4035020 con 0x7fabc413df30 2024-04-14T08:43:32.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.426+0000 7fabc27fc640 1 -- 172.21.15.172:0/1592217383 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb4031830 con 0x7fabc413df30 2024-04-14T08:43:32.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.426+0000 7fabc27fc640 1 -- 172.21.15.172:0/1592217383 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4031c00 con 0x7fabc413df30 2024-04-14T08:43:32.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.427+0000 7fabca077640 1 -- 172.21.15.172:0/1592217383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc413df30 msgr2=0x7fabc40b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.427+0000 7fabca077640 1 --2- 172.21.15.172:0/1592217383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc413df30 0x7fabc40b1cd0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fabb40097a0 tx=0x7fabb402b8d0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.427+0000 7fabca077640 1 -- 172.21.15.172:0/1592217383 shutdown_connections 2024-04-14T08:43:32.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.427+0000 7fabca077640 1 --2- 172.21.15.172:0/1592217383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabc4140e80 0x7fabc40b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.427+0000 7fabca077640 1 --2- 172.21.15.172:0/1592217383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc413df30 0x7fabc40b1cd0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.427+0000 7fabca077640 1 -- 172.21.15.172:0/1592217383 >> 172.21.15.172:0/1592217383 conn(0x7fabc406b520 msgr2=0x7fabc406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:32.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.427+0000 7fabca077640 1 -- 172.21.15.172:0/1592217383 shutdown_connections 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.427+0000 7fabca077640 1 -- 172.21.15.172:0/1592217383 wait complete. 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.428+0000 7fabca077640 1 Processor -- start 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.428+0000 7fabca077640 1 -- start start 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabca077640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc413df30 0x7fabc4152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabca077640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc4140e80 0x7fabc4138f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabca077640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabc415c380 0x7fabc4139460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabca077640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fabc413e310 con 0x7fabc413df30 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabca077640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fabc415be60 con 0x7fabc4140e80 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabca077640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fabc40ce9a0 con 0x7fabc415c380 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc413df30 0x7fabc4152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc413df30 0x7fabc4152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50988/0 (socket says 172.21.15.172:50988) 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabc37fe640 1 -- 172.21.15.172:0/2237253729 learned_addr learned my addr 172.21.15.172:0/2237253729 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabc2ffd640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc4140e80 0x7fabc4138f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabc3fff640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabc415c380 0x7fabc4139460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabc37fe640 1 -- 172.21.15.172:0/2237253729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabc415c380 msgr2=0x7fabc4139460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabc37fe640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabc415c380 0x7fabc4139460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabc37fe640 1 -- 172.21.15.172:0/2237253729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc4140e80 msgr2=0x7fabc4138f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabc37fe640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc4140e80 0x7fabc4138f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabc37fe640 1 -- 172.21.15.172:0/2237253729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc413ae00 con 0x7fabc413df30 2024-04-14T08:43:32.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabc2ffd640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc4140e80 0x7fabc4138f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:32.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.429+0000 7fabc3fff640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabc415c380 0x7fabc4139460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:32.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.430+0000 7fabc37fe640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc413df30 0x7fabc4152740 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fabb40097a0 tx=0x7fabb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.430+0000 7fababfff640 1 -- 172.21.15.172:0/2237253729 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4004230 con 0x7fabc413df30 2024-04-14T08:43:32.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.430+0000 7fababfff640 1 -- 172.21.15.172:0/2237253729 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb40043d0 con 0x7fabc413df30 2024-04-14T08:43:32.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.430+0000 7fabca077640 1 -- 172.21.15.172:0/2237253729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc413b0e0 con 0x7fabc413df30 2024-04-14T08:43:32.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.430+0000 7fababfff640 1 -- 172.21.15.172:0/2237253729 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb403cad0 con 0x7fabc413df30 2024-04-14T08:43:32.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.430+0000 7fabca077640 1 -- 172.21.15.172:0/2237253729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc413b6a0 con 0x7fabc413df30 2024-04-14T08:43:32.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.431+0000 7fabca077640 1 -- 172.21.15.172:0/2237253729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc4068590 con 0x7fabc413df30 2024-04-14T08:43:32.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.431+0000 7fababfff640 1 -- 172.21.15.172:0/2237253729 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabb4038020 con 0x7fabc413df30 2024-04-14T08:43:32.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.432+0000 7fababfff640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fabb006c8f0 0x7fabb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.432+0000 7fababfff640 1 -- 172.21.15.172:0/2237253729 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7fabb40adcf0 con 0x7fabc413df30 2024-04-14T08:43:32.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.432+0000 7fabc2ffd640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fabb006c8f0 0x7fabb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.432+0000 7fabc2ffd640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fabb006c8f0 0x7fabb006edb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fabc40cf850 tx=0x7fabb8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.436+0000 7fababfff640 1 -- 172.21.15.172:0/2237253729 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fabb4041070 con 0x7fabc413df30 2024-04-14T08:43:32.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.609+0000 7fabca077640 1 -- 172.21.15.172:0/2237253729 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fabc4072c60 con 0x7fabb006c8f0 2024-04-14T08:43:32.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.609+0000 7fababfff640 1 -- 172.21.15.172:0/2237253729 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fabc4072c60 con 0x7fabb006c8f0 2024-04-14T08:43:32.611 INFO:teuthology.orchestra.run.smithi172.stderr:instructed osd(s) 4 to deep-scrub 2024-04-14T08:43:32.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.611+0000 7fabca077640 1 -- 172.21.15.172:0/2237253729 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fabb006c8f0 msgr2=0x7fabb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.611+0000 7fabca077640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fabb006c8f0 0x7fabb006edb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fabc40cf850 tx=0x7fabb8009040 comp rx=0 tx=0).stop 2024-04-14T08:43:32.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.611+0000 7fabca077640 1 -- 172.21.15.172:0/2237253729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc413df30 msgr2=0x7fabc4152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.611+0000 7fabca077640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc413df30 0x7fabc4152740 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fabb40097a0 tx=0x7fabb40046c0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.611+0000 7fabca077640 1 -- 172.21.15.172:0/2237253729 shutdown_connections 2024-04-14T08:43:32.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.611+0000 7fabca077640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fabb006c8f0 0x7fabb006edb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.611+0000 7fabca077640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabc415c380 0x7fabc4139460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.611+0000 7fabca077640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabc4140e80 0x7fabc4138f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.611+0000 7fabca077640 1 --2- 172.21.15.172:0/2237253729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabc413df30 0x7fabc4152740 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.611+0000 7fabca077640 1 -- 172.21.15.172:0/2237253729 >> 172.21.15.172:0/2237253729 conn(0x7fabc406b520 msgr2=0x7fabc415ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:32.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.611+0000 7fabca077640 1 -- 172.21.15.172:0/2237253729 shutdown_connections 2024-04-14T08:43:32.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.611+0000 7fabca077640 1 -- 172.21.15.172:0/2237253729 wait complete. 2024-04-14T08:43:32.623 INFO:tasks.ceph:Scrubbing osd.5 2024-04-14T08:43:32.623 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-14T08:43:32.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.727+0000 7f71188de640 1 Processor -- start 2024-04-14T08:43:32.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.727+0000 7f71188de640 1 -- start start 2024-04-14T08:43:32.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.728+0000 7f71188de640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f71140b3610 0x7f711406dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.728+0000 7f71188de640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f711409b7b0 0x7f71140cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.728+0000 7f71188de640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71140d2d50 con 0x7f711409aa70 2024-04-14T08:43:32.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.728+0000 7f71188de640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f711406cc60 con 0x7f71140b3610 2024-04-14T08:43:32.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.728+0000 7f71188de640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f711406ce90 con 0x7f711409b7b0 2024-04-14T08:43:32.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.728+0000 7f7111d9b640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f71140b3610 0x7f711406dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.728+0000 7f711259c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f711409aa70 0x7f711409ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43204/0 (socket says 172.21.15.172:43204) 2024-04-14T08:43:32.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.728+0000 7f711259c640 1 -- 172.21.15.172:0/3869705843 learned_addr learned my addr 172.21.15.172:0/3869705843 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:32.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.729+0000 7f7112d9d640 1 --2- 172.21.15.172:0/3869705843 >> v2:172.21.15.177:3300/0 conn(0x7f711409b7b0 0x7f71140cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.729+0000 7f7111d9b640 1 -- 172.21.15.172:0/3869705843 >> v2:172.21.15.177:3300/0 conn(0x7f711409b7b0 msgr2=0x7f71140cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:32.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.729+0000 7f7111d9b640 1 --2- 172.21.15.172:0/3869705843 >> v2:172.21.15.177:3300/0 conn(0x7f711409b7b0 0x7f71140cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.729+0000 7f7111d9b640 1 -- 172.21.15.172:0/3869705843 >> v1:172.21.15.172:6789/0 conn(0x7f711409aa70 legacy=0x7f711409ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.729+0000 7f7111d9b640 1 -- 172.21.15.172:0/3869705843 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711406d170 con 0x7f71140b3610 2024-04-14T08:43:32.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.729+0000 7f711159a640 1 -- 172.21.15.172:0/3869705843 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1363750459 0 0) 0x7f71140d2d50 con 0x7f711409aa70 2024-04-14T08:43:32.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.729+0000 7f7111d9b640 1 --2- 172.21.15.172:0/3869705843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71140b3610 0x7f711406dc50 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f7114073420 tx=0x7f710802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fe8d70bdf82b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.730+0000 7f711159a640 1 -- 172.21.15.172:0/3869705843 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108035020 con 0x7f71140b3610 2024-04-14T08:43:32.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.730+0000 7f711159a640 1 -- 172.21.15.172:0/3869705843 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7108031830 con 0x7f71140b3610 2024-04-14T08:43:32.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.730+0000 7f711159a640 1 -- 172.21.15.172:0/3869705843 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108031c00 con 0x7f71140b3610 2024-04-14T08:43:32.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.730+0000 7f71188de640 1 -- 172.21.15.172:0/3869705843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71140b3610 msgr2=0x7f711406dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.730+0000 7f71188de640 1 --2- 172.21.15.172:0/3869705843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71140b3610 0x7f711406dc50 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f7114073420 tx=0x7f710802b8d0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.730+0000 7f71188de640 1 -- 172.21.15.172:0/3869705843 shutdown_connections 2024-04-14T08:43:32.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.730+0000 7f71188de640 1 --2- 172.21.15.172:0/3869705843 >> v2:172.21.15.177:3300/0 conn(0x7f711409b7b0 0x7f71140cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.730+0000 7f71188de640 1 --2- 172.21.15.172:0/3869705843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71140b3610 0x7f711406dc50 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.730+0000 7f71188de640 1 -- 172.21.15.172:0/3869705843 >> 172.21.15.172:0/3869705843 conn(0x7f71140d2040 msgr2=0x7f71140d2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:32.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.730+0000 7f71188de640 1 -- 172.21.15.172:0/3869705843 shutdown_connections 2024-04-14T08:43:32.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.730+0000 7f71188de640 1 -- 172.21.15.172:0/3869705843 wait complete. 2024-04-14T08:43:32.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.731+0000 7f71188de640 1 Processor -- start 2024-04-14T08:43:32.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.732+0000 7f71188de640 1 -- start start 2024-04-14T08:43:32.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.732+0000 7f71188de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f711409aa70 0x7f711415d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.732+0000 7f71188de640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f711409b7b0 0x7f711414b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.732+0000 7f71188de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71140b3610 0x7f711414bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.732+0000 7f71188de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7114161e40 con 0x7f71140b3610 2024-04-14T08:43:32.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.732+0000 7f71188de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f71140cff40 con 0x7f711409aa70 2024-04-14T08:43:32.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.732+0000 7f71188de640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f7114161fc0 con 0x7f711409b7b0 2024-04-14T08:43:32.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.732+0000 7f7112d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71140b3610 0x7f711414bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.732+0000 7f7112d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71140b3610 0x7f711414bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51012/0 (socket says 172.21.15.172:51012) 2024-04-14T08:43:32.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.732+0000 7f7112d9d640 1 -- 172.21.15.172:0/2074335704 learned_addr learned my addr 172.21.15.172:0/2074335704 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:32.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.732+0000 7f711259c640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f711409aa70 0x7f711415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.732+0000 7f7111d9b640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f711409b7b0 0x7f711414b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.733+0000 7f7112d9d640 1 -- 172.21.15.172:0/2074335704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f711409b7b0 msgr2=0x7f711414b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.733+0000 7f7112d9d640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f711409b7b0 0x7f711414b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.733+0000 7f7112d9d640 1 -- 172.21.15.172:0/2074335704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f711409aa70 msgr2=0x7f711415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.733+0000 7f7112d9d640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f711409aa70 0x7f711415d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.733+0000 7f7112d9d640 1 -- 172.21.15.172:0/2074335704 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711415b840 con 0x7f71140b3610 2024-04-14T08:43:32.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.733+0000 7f7111d9b640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f711409b7b0 0x7f711414b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:32.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.733+0000 7f7112d9d640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71140b3610 0x7f711414bd90 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f710c00d800 tx=0x7f710c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.733+0000 7f7102ffd640 1 -- 172.21.15.172:0/2074335704 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c00b6c0 con 0x7f71140b3610 2024-04-14T08:43:32.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.733+0000 7f7102ffd640 1 -- 172.21.15.172:0/2074335704 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f710c019070 con 0x7f71140b3610 2024-04-14T08:43:32.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.733+0000 7f71188de640 1 -- 172.21.15.172:0/2074335704 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711415dc80 con 0x7f71140b3610 2024-04-14T08:43:32.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.733+0000 7f7102ffd640 1 -- 172.21.15.172:0/2074335704 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c00bac0 con 0x7f71140b3610 2024-04-14T08:43:32.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.733+0000 7f71188de640 1 -- 172.21.15.172:0/2074335704 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711415e240 con 0x7f71140b3610 2024-04-14T08:43:32.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.734+0000 7f71188de640 1 -- 172.21.15.172:0/2074335704 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f711415e900 con 0x7f71140b3610 2024-04-14T08:43:32.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.734+0000 7f7102ffd640 1 -- 172.21.15.172:0/2074335704 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f710c00bc60 con 0x7f71140b3610 2024-04-14T08:43:32.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.735+0000 7f7102ffd640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f70f406c8f0 0x7f70f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.735+0000 7f7102ffd640 1 -- 172.21.15.172:0/2074335704 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f710c08c820 con 0x7f71140b3610 2024-04-14T08:43:32.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.735+0000 7f711259c640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f70f406c8f0 0x7f70f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.735+0000 7f7102ffd640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x7f70f4072ed0 0x7f70f4075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.735+0000 7f7102ffd640 1 -- 172.21.15.172:0/2074335704 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f70f40759e0 con 0x7f70f4072ed0 2024-04-14T08:43:32.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.735+0000 7f7102ffd640 1 -- 172.21.15.172:0/2074335704 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f710c08ccb0 con 0x7f71140b3610 2024-04-14T08:43:32.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.735+0000 7f711259c640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f70f406c8f0 0x7f70f406edb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f71040012e0 tx=0x7f7104001630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.735+0000 7f7111d9b640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x7f70f4072ed0 0x7f70f4075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.736+0000 7f7111d9b640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x7f70f4072ed0 0x7f70f4075310 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.736+0000 7f7102ffd640 1 -- 172.21.15.172:0/2074335704 <== osd.5 v2:172.21.15.177:6816/3172735715 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f70f40759e0 con 0x7f70f4072ed0 2024-04-14T08:43:32.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.749+0000 7f71188de640 1 -- 172.21.15.172:0/2074335704 --> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f71140686a0 con 0x7f70f4072ed0 2024-04-14T08:43:32.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.750+0000 7f7102ffd640 1 -- 172.21.15.172:0/2074335704 <== osd.5 v2:172.21.15.177:6816/3172735715 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f71140686a0 con 0x7f70f4072ed0 2024-04-14T08:43:32.752 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:43:32.752 INFO:teuthology.orchestra.run.smithi172.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-14T08:43:32.754 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:43:32.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.750+0000 7f71188de640 1 -- 172.21.15.172:0/2074335704 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x7f70f4072ed0 msgr2=0x7f70f4075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.750+0000 7f71188de640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.177:6816/3172735715,v1:172.21.15.177:6817/3172735715] conn(0x7f70f4072ed0 0x7f70f4075310 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.750+0000 7f71188de640 1 -- 172.21.15.172:0/2074335704 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f70f406c8f0 msgr2=0x7f70f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.750+0000 7f71188de640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f70f406c8f0 0x7f70f406edb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f71040012e0 tx=0x7f7104001630 comp rx=0 tx=0).stop 2024-04-14T08:43:32.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.750+0000 7f71188de640 1 -- 172.21.15.172:0/2074335704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71140b3610 msgr2=0x7f711414bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.750+0000 7f71188de640 1 --2- 172.21.15.172:0/2074335704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71140b3610 0x7f711414bd90 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f710c00d800 tx=0x7f710c00dcc0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.750+0000 7f7112d9d640 1 -- 172.21.15.172:0/2074335704 reap_dead start 2024-04-14T08:43:32.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.751+0000 7f71188de640 1 -- 172.21.15.172:0/2074335704 shutdown_connections 2024-04-14T08:43:32.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.751+0000 7f71188de640 1 -- 172.21.15.172:0/2074335704 >> 172.21.15.172:0/2074335704 conn(0x7f71140d2040 msgr2=0x7f71140cb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:32.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.751+0000 7f71188de640 1 -- 172.21.15.172:0/2074335704 shutdown_connections 2024-04-14T08:43:32.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.751+0000 7f71188de640 1 -- 172.21.15.172:0/2074335704 wait complete. 2024-04-14T08:43:32.762 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-14T08:43:32.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.863+0000 7f639531e640 1 Processor -- start 2024-04-14T08:43:32.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.863+0000 7f639531e640 1 -- start start 2024-04-14T08:43:32.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.864+0000 7f639531e640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f639015e430 0x7f6390162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.864+0000 7f639531e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63900cefb0 con 0x7f6390162540 2024-04-14T08:43:32.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.864+0000 7f639531e640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63900cf290 con 0x7f63900d2e00 2024-04-14T08:43:32.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.864+0000 7f639531e640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f63900cf570 con 0x7f639015e430 2024-04-14T08:43:32.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.864+0000 7f638effd640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f639015e430 0x7f6390162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.864+0000 7f638e7fc640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6390162540 0x7f63900b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43214/0 (socket says 172.21.15.172:43214) 2024-04-14T08:43:32.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.864+0000 7f638e7fc640 1 -- 172.21.15.172:0/565470150 learned_addr learned my addr 172.21.15.172:0/565470150 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:32.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.864+0000 7f638effd640 1 -- 172.21.15.172:0/565470150 >> v1:172.21.15.177:6789/0 conn(0x7f63900d2e00 legacy=0x7f63900b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:32.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.864+0000 7f638effd640 1 -- 172.21.15.172:0/565470150 >> v1:172.21.15.172:6789/0 conn(0x7f6390162540 legacy=0x7f63900b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.864+0000 7f638effd640 1 -- 172.21.15.172:0/565470150 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63900cf850 con 0x7f639015e430 2024-04-14T08:43:32.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.865+0000 7f638effd640 1 --2- 172.21.15.172:0/565470150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f639015e430 0x7f6390162000 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f63800097a0 tx=0x7f638002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63c0bebe77904ee server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.865+0000 7f638dffb640 1 -- 172.21.15.172:0/565470150 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380035020 con 0x7f639015e430 2024-04-14T08:43:32.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.865+0000 7f638dffb640 1 -- 172.21.15.172:0/565470150 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6380031830 con 0x7f639015e430 2024-04-14T08:43:32.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.865+0000 7f638dffb640 1 -- 172.21.15.172:0/565470150 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6380031c00 con 0x7f639015e430 2024-04-14T08:43:32.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.866+0000 7f639531e640 1 -- 172.21.15.172:0/565470150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f639015e430 msgr2=0x7f6390162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.866+0000 7f639531e640 1 --2- 172.21.15.172:0/565470150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f639015e430 0x7f6390162000 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f63800097a0 tx=0x7f638002b8d0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.866+0000 7f639531e640 1 -- 172.21.15.172:0/565470150 shutdown_connections 2024-04-14T08:43:32.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.866+0000 7f639531e640 1 --2- 172.21.15.172:0/565470150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f639015e430 0x7f6390162000 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.866+0000 7f639531e640 1 -- 172.21.15.172:0/565470150 >> 172.21.15.172:0/565470150 conn(0x7f639009d560 msgr2=0x7f63900d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:32.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.866+0000 7f639531e640 1 -- 172.21.15.172:0/565470150 shutdown_connections 2024-04-14T08:43:32.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.866+0000 7f639531e640 1 -- 172.21.15.172:0/565470150 wait complete. 2024-04-14T08:43:32.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.867+0000 7f639531e640 1 Processor -- start 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.867+0000 7f639531e640 1 -- start start 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f639531e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63900d2e00 0x7f63900c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f639531e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f639015e430 0x7f63900b9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f639531e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6390162540 0x7f63900cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f639531e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f63900d4e20 con 0x7f6390162540 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f639531e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f639015e810 con 0x7f639015e430 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f639531e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f63900b4850 con 0x7f63900d2e00 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f638f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6390162540 0x7f63900cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f638f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6390162540 0x7f63900cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51042/0 (socket says 172.21.15.172:51042) 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f638f7fe640 1 -- 172.21.15.172:0/2275376135 learned_addr learned my addr 172.21.15.172:0/2275376135 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f638e7fc640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f639015e430 0x7f63900b9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f638effd640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63900d2e00 0x7f63900c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f638f7fe640 1 -- 172.21.15.172:0/2275376135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63900d2e00 msgr2=0x7f63900c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f638f7fe640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63900d2e00 0x7f63900c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f638f7fe640 1 -- 172.21.15.172:0/2275376135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f639015e430 msgr2=0x7f63900b9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:32.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f638f7fe640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f639015e430 0x7f63900b9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:32.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f638f7fe640 1 -- 172.21.15.172:0/2275376135 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63900b9b90 con 0x7f6390162540 2024-04-14T08:43:32.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f638e7fc640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f639015e430 0x7f63900b9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:32.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f638f7fe640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6390162540 0x7f63900cd080 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f638800c780 tx=0x7f638800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.868+0000 7f63777fe640 1 -- 172.21.15.172:0/2275376135 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6388017070 con 0x7f6390162540 2024-04-14T08:43:32.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.869+0000 7f63777fe640 1 -- 172.21.15.172:0/2275376135 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6388003da0 con 0x7f6390162540 2024-04-14T08:43:32.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.869+0000 7f639531e640 1 -- 172.21.15.172:0/2275376135 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63900c94b0 con 0x7f6390162540 2024-04-14T08:43:32.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.869+0000 7f63777fe640 1 -- 172.21.15.172:0/2275376135 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63880052b0 con 0x7f6390162540 2024-04-14T08:43:32.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.869+0000 7f639531e640 1 -- 172.21.15.172:0/2275376135 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63900c9a70 con 0x7f6390162540 2024-04-14T08:43:32.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.869+0000 7f639531e640 1 -- 172.21.15.172:0/2275376135 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6390068590 con 0x7f6390162540 2024-04-14T08:43:32.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.870+0000 7f63777fe640 1 -- 172.21.15.172:0/2275376135 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f638801a070 con 0x7f6390162540 2024-04-14T08:43:32.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.870+0000 7f63777fe640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f637c06c8f0 0x7f637c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:32.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.870+0000 7f638effd640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f637c06c8f0 0x7f637c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:32.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.871+0000 7f63777fe640 1 -- 172.21.15.172:0/2275376135 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f638808b0d0 con 0x7f6390162540 2024-04-14T08:43:32.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.871+0000 7f638effd640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f637c06c8f0 0x7f637c06edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6380036000 tx=0x7f6380008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:32.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:32.875+0000 7f63777fe640 1 -- 172.21.15.172:0/2275376135 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f63880595c0 con 0x7f6390162540 2024-04-14T08:43:33.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.048+0000 7f639531e640 1 -- 172.21.15.172:0/2275376135 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f63900d18a0 con 0x7f637c06c8f0 2024-04-14T08:43:33.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.049+0000 7f63777fe640 1 -- 172.21.15.172:0/2275376135 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f63900d18a0 con 0x7f637c06c8f0 2024-04-14T08:43:33.051 INFO:teuthology.orchestra.run.smithi172.stderr:instructed osd(s) 5 to deep-scrub 2024-04-14T08:43:33.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.051+0000 7f639531e640 1 -- 172.21.15.172:0/2275376135 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f637c06c8f0 msgr2=0x7f637c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.051+0000 7f639531e640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f637c06c8f0 0x7f637c06edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6380036000 tx=0x7f6380008030 comp rx=0 tx=0).stop 2024-04-14T08:43:33.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.051+0000 7f639531e640 1 -- 172.21.15.172:0/2275376135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6390162540 msgr2=0x7f63900cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.051+0000 7f639531e640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6390162540 0x7f63900cd080 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f638800c780 tx=0x7f638800cc40 comp rx=0 tx=0).stop 2024-04-14T08:43:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.051+0000 7f639531e640 1 -- 172.21.15.172:0/2275376135 shutdown_connections 2024-04-14T08:43:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.051+0000 7f639531e640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f637c06c8f0 0x7f637c06edb0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.051+0000 7f639531e640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6390162540 0x7f63900cd080 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.051+0000 7f639531e640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f639015e430 0x7f63900b9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.051+0000 7f639531e640 1 --2- 172.21.15.172:0/2275376135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f63900d2e00 0x7f63900c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.051+0000 7f639531e640 1 -- 172.21.15.172:0/2275376135 >> 172.21.15.172:0/2275376135 conn(0x7f639009d560 msgr2=0x7f63900b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.051+0000 7f639531e640 1 -- 172.21.15.172:0/2275376135 shutdown_connections 2024-04-14T08:43:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.051+0000 7f639531e640 1 -- 172.21.15.172:0/2275376135 wait complete. 2024-04-14T08:43:33.063 INFO:tasks.ceph:Scrubbing osd.6 2024-04-14T08:43:33.064 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-14T08:43:33.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.167+0000 7f293dd11640 1 Processor -- start 2024-04-14T08:43:33.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.167+0000 7f293dd11640 1 -- start start 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.168+0000 7f293dd11640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f293806b520 0x7f29380b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.168+0000 7f293dd11640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f293806f350 0x7f293806f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.168+0000 7f293dd11640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29380b44d0 con 0x7f2938162870 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.168+0000 7f293dd11640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f293815f260 con 0x7f293806b520 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.168+0000 7f293dd11640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f293815ec10 con 0x7f293806f350 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.168+0000 7f29377fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f293806b520 0x7f29380b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.168+0000 7f29377fe640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f293806b520 0x7f29380b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51058/0 (socket says 172.21.15.172:51058) 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.168+0000 7f29377fe640 1 -- 172.21.15.172:0/1993859941 learned_addr learned my addr 172.21.15.172:0/1993859941 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.169+0000 7f2937fff640 1 --2- 172.21.15.172:0/1993859941 >> v2:172.21.15.177:3300/0 conn(0x7f293806f350 0x7f293806f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.169+0000 7f29377fe640 1 -- 172.21.15.172:0/1993859941 >> v2:172.21.15.177:3300/0 conn(0x7f293806f350 msgr2=0x7f293806f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.169+0000 7f29377fe640 1 --2- 172.21.15.172:0/1993859941 >> v2:172.21.15.177:3300/0 conn(0x7f293806f350 0x7f293806f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.169+0000 7f29377fe640 1 -- 172.21.15.172:0/1993859941 >> v1:172.21.15.177:6789/0 conn(0x7f2938162870 legacy=0x7f293806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.169+0000 7f29377fe640 1 -- 172.21.15.172:0/1993859941 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293815eef0 con 0x7f293806b520 2024-04-14T08:43:33.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.169+0000 7f2937fff640 1 --2- 172.21.15.172:0/1993859941 >> v2:172.21.15.177:3300/0 conn(0x7f293806f350 0x7f293806f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.169+0000 7f29377fe640 1 --2- 172.21.15.172:0/1993859941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f293806b520 0x7f29380b3dd0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f2938073830 tx=0x7f292802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e841d6ccd3f990d server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.170+0000 7f29367fc640 1 -- 172.21.15.172:0/1993859941 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928034070 con 0x7f293806b520 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.170+0000 7f29367fc640 1 -- 172.21.15.172:0/1993859941 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2928030860 con 0x7f293806b520 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.170+0000 7f29367fc640 1 -- 172.21.15.172:0/1993859941 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928030c30 con 0x7f293806b520 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.170+0000 7f293dd11640 1 -- 172.21.15.172:0/1993859941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f293806b520 msgr2=0x7f29380b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.170+0000 7f293dd11640 1 --2- 172.21.15.172:0/1993859941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f293806b520 0x7f29380b3dd0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f2938073830 tx=0x7f292802b9a0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.170+0000 7f293dd11640 1 -- 172.21.15.172:0/1993859941 shutdown_connections 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.170+0000 7f293dd11640 1 --2- 172.21.15.172:0/1993859941 >> v2:172.21.15.177:3300/0 conn(0x7f293806f350 0x7f293806f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.170+0000 7f293dd11640 1 --2- 172.21.15.172:0/1993859941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f293806b520 0x7f29380b3dd0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.170+0000 7f293dd11640 1 -- 172.21.15.172:0/1993859941 >> 172.21.15.172:0/1993859941 conn(0x7f29380d0f20 msgr2=0x7f29380d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.170+0000 7f293dd11640 1 -- 172.21.15.172:0/1993859941 shutdown_connections 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.170+0000 7f293dd11640 1 -- 172.21.15.172:0/1993859941 wait complete. 2024-04-14T08:43:33.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.171+0000 7f293dd11640 1 Processor -- start 2024-04-14T08:43:33.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.171+0000 7f293dd11640 1 -- start start 2024-04-14T08:43:33.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f293dd11640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f293806b520 0x7f29381f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f293dd11640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f293806f350 0x7f2938161110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f293dd11640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938162870 0x7f2938161650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f293dd11640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f293815f6f0 con 0x7f293806b520 2024-04-14T08:43:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f293dd11640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f293815e8c0 con 0x7f293806f350 2024-04-14T08:43:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f293dd11640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f293815f870 con 0x7f2938162870 2024-04-14T08:43:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f29377fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f293806b520 0x7f29381f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f29377fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f293806b520 0x7f29381f8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51062/0 (socket says 172.21.15.172:51062) 2024-04-14T08:43:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f29377fe640 1 -- 172.21.15.172:0/1855982016 learned_addr learned my addr 172.21.15.172:0/1855982016 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f29377fe640 1 -- 172.21.15.172:0/1855982016 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938162870 msgr2=0x7f2938161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f29377fe640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2938162870 0x7f2938161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f29377fe640 1 -- 172.21.15.172:0/1855982016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f293806f350 msgr2=0x7f2938161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.172+0000 7f2936ffd640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f293806f350 0x7f2938161110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.173+0000 7f29377fe640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f293806f350 0x7f2938161110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.173+0000 7f29377fe640 1 -- 172.21.15.172:0/1855982016 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29381f8b70 con 0x7f293806b520 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.173+0000 7f29377fe640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f293806b520 0x7f29381f8630 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f292802b9e0 tx=0x7f2928004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.173+0000 7f291ffff640 1 -- 172.21.15.172:0/1855982016 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928048070 con 0x7f293806b520 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.173+0000 7f291ffff640 1 -- 172.21.15.172:0/1855982016 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2928030660 con 0x7f293806b520 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.173+0000 7f293dd11640 1 -- 172.21.15.172:0/1855982016 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29381e7fc0 con 0x7f293806b520 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.173+0000 7f291ffff640 1 -- 172.21.15.172:0/1855982016 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928034040 con 0x7f293806b520 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.173+0000 7f293dd11640 1 -- 172.21.15.172:0/1855982016 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29381e8580 con 0x7f293806b520 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.174+0000 7f293dd11640 1 -- 172.21.15.172:0/1855982016 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f29380686a0 con 0x7f293806b520 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.174+0000 7f291ffff640 1 -- 172.21.15.172:0/1855982016 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f292804b070 con 0x7f293806b520 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.175+0000 7f291ffff640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f292406c8f0 0x7f292406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.175+0000 7f291ffff640 1 -- 172.21.15.172:0/1855982016 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f29280adb80 con 0x7f293806b520 2024-04-14T08:43:33.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.175+0000 7f2936ffd640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f292406c8f0 0x7f292406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.175+0000 7f291ffff640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x7f2924072ed0 0x7f2924075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.175+0000 7f291ffff640 1 -- 172.21.15.172:0/1855982016 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f29240759e0 con 0x7f2924072ed0 2024-04-14T08:43:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.175+0000 7f291ffff640 1 -- 172.21.15.172:0/1855982016 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f29280b22a0 con 0x7f293806b520 2024-04-14T08:43:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.175+0000 7f2936ffd640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f292406c8f0 0x7f292406edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2938141910 tx=0x7f292c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.175+0000 7f2937fff640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x7f2924072ed0 0x7f2924075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.176+0000 7f2937fff640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x7f2924072ed0 0x7f2924075310 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.176+0000 7f291ffff640 1 -- 172.21.15.172:0/1855982016 <== osd.6 v2:172.21.15.177:6800/2961907928 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f29240759e0 con 0x7f2924072ed0 2024-04-14T08:43:33.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.189+0000 7f293dd11640 1 -- 172.21.15.172:0/1855982016 --> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f29381e9fc0 con 0x7f2924072ed0 2024-04-14T08:43:33.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.190+0000 7f291ffff640 1 -- 172.21.15.172:0/1855982016 <== osd.6 v2:172.21.15.177:6800/2961907928 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f29381e9fc0 con 0x7f2924072ed0 2024-04-14T08:43:33.191 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:43:33.192 INFO:teuthology.orchestra.run.smithi172.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-14T08:43:33.192 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:43:33.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.190+0000 7f293dd11640 1 -- 172.21.15.172:0/1855982016 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x7f2924072ed0 msgr2=0x7f2924075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.190+0000 7f293dd11640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.177:6800/2961907928,v1:172.21.15.177:6801/2961907928] conn(0x7f2924072ed0 0x7f2924075310 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.190+0000 7f293dd11640 1 -- 172.21.15.172:0/1855982016 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f292406c8f0 msgr2=0x7f292406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.190+0000 7f293dd11640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f292406c8f0 0x7f292406edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2938141910 tx=0x7f292c009040 comp rx=0 tx=0).stop 2024-04-14T08:43:33.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.190+0000 7f293dd11640 1 -- 172.21.15.172:0/1855982016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f293806b520 msgr2=0x7f29381f8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.190+0000 7f293dd11640 1 --2- 172.21.15.172:0/1855982016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f293806b520 0x7f29381f8630 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f292802b9e0 tx=0x7f2928004730 comp rx=0 tx=0).stop 2024-04-14T08:43:33.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.190+0000 7f2937fff640 1 -- 172.21.15.172:0/1855982016 reap_dead start 2024-04-14T08:43:33.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.190+0000 7f293dd11640 1 -- 172.21.15.172:0/1855982016 shutdown_connections 2024-04-14T08:43:33.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.190+0000 7f293dd11640 1 -- 172.21.15.172:0/1855982016 >> 172.21.15.172:0/1855982016 conn(0x7f29380d0f20 msgr2=0x7f2938074900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:33.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.191+0000 7f293dd11640 1 -- 172.21.15.172:0/1855982016 shutdown_connections 2024-04-14T08:43:33.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.191+0000 7f293dd11640 1 -- 172.21.15.172:0/1855982016 wait complete. 2024-04-14T08:43:33.202 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-14T08:43:33.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.305+0000 7f6445b8c640 1 Processor -- start 2024-04-14T08:43:33.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.305+0000 7f6445b8c640 1 -- start start 2024-04-14T08:43:33.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f6445b8c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644015f460 0x7f64400b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f6445b8c640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f644006bc30 0x7f64401683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f6445b8c640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644015edd0 con 0x7f64400b3bf0 2024-04-14T08:43:33.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f6445b8c640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f6440141960 con 0x7f644006bc30 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f6445b8c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f644006c100 con 0x7f644015f460 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643effd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f64400b3bf0 0x7f644006b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43218/0 (socket says 172.21.15.172:43218) 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643effd640 1 -- 172.21.15.172:0/171759480 learned_addr learned my addr 172.21.15.172:0/171759480 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2401120199 0 0) 0x7f644015edd0 con 0x7f64400b3bf0 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643ffff640 1 --2- 172.21.15.172:0/171759480 >> v2:172.21.15.177:3300/0 conn(0x7f644006bc30 0x7f64401683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6428003520 con 0x7f64400b3bf0 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643f7fe640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644015f460 0x7f64400b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1492697603 0 0) 0x7f6428003520 con 0x7f64400b3bf0 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644015f460 msgr2=0x7f64400b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643e7fc640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644015f460 0x7f64400b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 >> v2:172.21.15.177:3300/0 conn(0x7f644006bc30 msgr2=0x7f64401683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643e7fc640 1 --2- 172.21.15.172:0/171759480 >> v2:172.21.15.177:3300/0 conn(0x7f644006bc30 0x7f64401683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644006c3e0 con 0x7f64400b3bf0 2024-04-14T08:43:33.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f6434001670 con 0x7f64400b3bf0 2024-04-14T08:43:33.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.306+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 >> v1:172.21.15.172:6789/0 conn(0x7f64400b3bf0 legacy=0x7f644006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643e7fc640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64280039b0 0x7f6428003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643e7fc640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6428004800 0x7f6428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643e7fc640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644015f460 0x7f64280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6440141960 con 0x7f6428004800 2024-04-14T08:43:33.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f644006c100 con 0x7f64280039b0 2024-04-14T08:43:33.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f642800ba70 con 0x7f644015f460 2024-04-14T08:43:33.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643effd640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6428004800 0x7f6428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643ffff640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644015f460 0x7f64280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643f7fe640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64280039b0 0x7f6428003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643effd640 1 -- 172.21.15.172:0/171759480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644015f460 msgr2=0x7f64280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:33.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643effd640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644015f460 0x7f64280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643effd640 1 -- 172.21.15.172:0/171759480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64280039b0 msgr2=0x7f6428003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:33.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643effd640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64280039b0 0x7f6428003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643effd640 1 -- 172.21.15.172:0/171759480 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6428003520 con 0x7f6428004800 2024-04-14T08:43:33.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.307+0000 7f643ffff640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644015f460 0x7f64280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:33.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.308+0000 7f643effd640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6428004800 0x7f6428004c80 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f6434001050 tx=0x7f6428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce31d5511acae48 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.308+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434007d50 con 0x7f6428004800 2024-04-14T08:43:33.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.308+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6434030070 con 0x7f6428004800 2024-04-14T08:43:33.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.308+0000 7f643e7fc640 1 -- 172.21.15.172:0/171759480 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434005d10 con 0x7f6428004800 2024-04-14T08:43:33.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.308+0000 7f6445b8c640 1 -- 172.21.15.172:0/171759480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6428004800 msgr2=0x7f6428004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.308+0000 7f6445b8c640 1 --2- 172.21.15.172:0/171759480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6428004800 0x7f6428004c80 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f6434001050 tx=0x7f6428005830 comp rx=0 tx=0).stop 2024-04-14T08:43:33.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.308+0000 7f643ffff640 1 -- 172.21.15.172:0/171759480 reap_dead start 2024-04-14T08:43:33.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.308+0000 7f6445b8c640 1 -- 172.21.15.172:0/171759480 shutdown_connections 2024-04-14T08:43:33.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.308+0000 7f6445b8c640 1 -- 172.21.15.172:0/171759480 >> 172.21.15.172:0/171759480 conn(0x7f64400d0780 msgr2=0x7f64400d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:33.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.308+0000 7f6445b8c640 1 -- 172.21.15.172:0/171759480 shutdown_connections 2024-04-14T08:43:33.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.308+0000 7f6445b8c640 1 -- 172.21.15.172:0/171759480 wait complete. 2024-04-14T08:43:33.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.309+0000 7f6445b8c640 1 Processor -- start 2024-04-14T08:43:33.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.310+0000 7f6445b8c640 1 -- start start 2024-04-14T08:43:33.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.310+0000 7f6445b8c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6428004800 0x7f644014d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.310+0000 7f6445b8c640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644014e1e0 0x7f644014a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.310+0000 7f6445b8c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6440137b50 0x7f6440138000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.310+0000 7f6445b8c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f644015f840 con 0x7f6428004800 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.310+0000 7f6445b8c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f644006c9a0 con 0x7f6440137b50 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.310+0000 7f6445b8c640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f644015f9c0 con 0x7f644014e1e0 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.310+0000 7f643f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6428004800 0x7f644014d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.310+0000 7f643f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6428004800 0x7f644014d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51074/0 (socket says 172.21.15.172:51074) 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.310+0000 7f643f7fe640 1 -- 172.21.15.172:0/341825883 learned_addr learned my addr 172.21.15.172:0/341825883 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f643ffff640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6440137b50 0x7f6440138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f643effd640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644014e1e0 0x7f644014a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f643f7fe640 1 -- 172.21.15.172:0/341825883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644014e1e0 msgr2=0x7f644014a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f643f7fe640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644014e1e0 0x7f644014a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f643f7fe640 1 -- 172.21.15.172:0/341825883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6440137b50 msgr2=0x7f6440138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f643f7fe640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6440137b50 0x7f6440138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f643f7fe640 1 -- 172.21.15.172:0/341825883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644013ae30 con 0x7f6428004800 2024-04-14T08:43:33.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f643f7fe640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6428004800 0x7f644014d9b0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f643000b7f0 tx=0x7f643000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f6427fff640 1 -- 172.21.15.172:0/341825883 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430018070 con 0x7f6428004800 2024-04-14T08:43:33.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f6427fff640 1 -- 172.21.15.172:0/341825883 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6430010830 con 0x7f6428004800 2024-04-14T08:43:33.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f6445b8c640 1 -- 172.21.15.172:0/341825883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644014ae90 con 0x7f6428004800 2024-04-14T08:43:33.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f6427fff640 1 -- 172.21.15.172:0/341825883 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430010c00 con 0x7f6428004800 2024-04-14T08:43:33.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.311+0000 7f6445b8c640 1 -- 172.21.15.172:0/341825883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644013a070 con 0x7f6428004800 2024-04-14T08:43:33.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.312+0000 7f6445b8c640 1 -- 172.21.15.172:0/341825883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6440067670 con 0x7f6428004800 2024-04-14T08:43:33.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.312+0000 7f6427fff640 1 -- 172.21.15.172:0/341825883 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f643001b070 con 0x7f6428004800 2024-04-14T08:43:33.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.313+0000 7f6427fff640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f642c06c920 0x7f642c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.313+0000 7f6427fff640 1 -- 172.21.15.172:0/341825883 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f643008bfc0 con 0x7f6428004800 2024-04-14T08:43:33.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.313+0000 7f643effd640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f642c06c920 0x7f642c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.313+0000 7f643effd640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f642c06c920 0x7f642c06ede0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6434034000 tx=0x7f6434007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.317+0000 7f6427fff640 1 -- 172.21.15.172:0/341825883 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f643005a540 con 0x7f6428004800 2024-04-14T08:43:33.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.494+0000 7f6445b8c640 1 -- 172.21.15.172:0/341825883 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f6440068570 con 0x7f642c06c920 2024-04-14T08:43:33.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.495+0000 7f6427fff640 1 -- 172.21.15.172:0/341825883 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6440068570 con 0x7f642c06c920 2024-04-14T08:43:33.496 INFO:teuthology.orchestra.run.smithi172.stderr:instructed osd(s) 6 to deep-scrub 2024-04-14T08:43:33.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.496+0000 7f6445b8c640 1 -- 172.21.15.172:0/341825883 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f642c06c920 msgr2=0x7f642c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.496+0000 7f6445b8c640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f642c06c920 0x7f642c06ede0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6434034000 tx=0x7f6434007000 comp rx=0 tx=0).stop 2024-04-14T08:43:33.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.496+0000 7f6445b8c640 1 -- 172.21.15.172:0/341825883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6428004800 msgr2=0x7f644014d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.496+0000 7f6445b8c640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6428004800 0x7f644014d9b0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f643000b7f0 tx=0x7f643000bcb0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.497+0000 7f6445b8c640 1 -- 172.21.15.172:0/341825883 shutdown_connections 2024-04-14T08:43:33.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.497+0000 7f6445b8c640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f642c06c920 0x7f642c06ede0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.497+0000 7f6445b8c640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6440137b50 0x7f6440138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.497+0000 7f6445b8c640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f644014e1e0 0x7f644014a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.497+0000 7f6445b8c640 1 --2- 172.21.15.172:0/341825883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6428004800 0x7f644014d9b0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.497+0000 7f6445b8c640 1 -- 172.21.15.172:0/341825883 >> 172.21.15.172:0/341825883 conn(0x7f64400d0780 msgr2=0x7f644006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:33.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.497+0000 7f6445b8c640 1 -- 172.21.15.172:0/341825883 shutdown_connections 2024-04-14T08:43:33.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.497+0000 7f6445b8c640 1 -- 172.21.15.172:0/341825883 wait complete. 2024-04-14T08:43:33.510 INFO:tasks.ceph:Scrubbing osd.7 2024-04-14T08:43:33.510 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-14T08:43:33.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.614+0000 7f599c036640 1 Processor -- start 2024-04-14T08:43:33.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.614+0000 7f599c036640 1 -- start start 2024-04-14T08:43:33.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.614+0000 7f599c036640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f599415e5b0 0x7f59940ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.615+0000 7f599c036640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59940b41d0 con 0x7f59940729b0 2024-04-14T08:43:33.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.615+0000 7f599c036640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59940aebe0 con 0x7f599406c530 2024-04-14T08:43:33.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.615+0000 7f599c036640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7f59940b4470 con 0x7f599415e5b0 2024-04-14T08:43:33.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.615+0000 7f59995d1640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f59940729b0 0x7f599406dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43220/0 (socket says 172.21.15.172:43220) 2024-04-14T08:43:33.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.615+0000 7f59995d1640 1 -- 172.21.15.172:0/478997416 learned_addr learned my addr 172.21.15.172:0/478997416 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:33.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.615+0000 7f5999dd2640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f599415e5b0 0x7f59940ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.615+0000 7f5999dd2640 1 -- 172.21.15.172:0/478997416 >> v1:172.21.15.177:6789/0 conn(0x7f599406c530 legacy=0x7f59940b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.615+0000 7f5999dd2640 1 -- 172.21.15.172:0/478997416 >> v1:172.21.15.172:6789/0 conn(0x7f59940729b0 legacy=0x7f599406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.615+0000 7f5999dd2640 1 -- 172.21.15.172:0/478997416 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59940ae8d0 con 0x7f599415e5b0 2024-04-14T08:43:33.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.615+0000 7f5998dd0640 1 -- 172.21.15.172:0/478997416 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 85973396 0 0) 0x7f59940b41d0 con 0x7f59940729b0 2024-04-14T08:43:33.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.616+0000 7f5999dd2640 1 --2- 172.21.15.172:0/478997416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f599415e5b0 0x7f59940ae390 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f59900097d0 tx=0x7f599002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20df4eefc62d419 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.616+0000 7f5998dd0640 1 -- 172.21.15.172:0/478997416 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990034070 con 0x7f599415e5b0 2024-04-14T08:43:33.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.616+0000 7f5998dd0640 1 -- 172.21.15.172:0/478997416 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5990030610 con 0x7f599415e5b0 2024-04-14T08:43:33.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.616+0000 7f5998dd0640 1 -- 172.21.15.172:0/478997416 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59900309e0 con 0x7f599415e5b0 2024-04-14T08:43:33.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.616+0000 7f599c036640 1 -- 172.21.15.172:0/478997416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f599415e5b0 msgr2=0x7f59940ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.616+0000 7f599c036640 1 --2- 172.21.15.172:0/478997416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f599415e5b0 0x7f59940ae390 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f59900097d0 tx=0x7f599002b960 comp rx=0 tx=0).stop 2024-04-14T08:43:33.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.616+0000 7f599c036640 1 -- 172.21.15.172:0/478997416 shutdown_connections 2024-04-14T08:43:33.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.616+0000 7f599c036640 1 --2- 172.21.15.172:0/478997416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f599415e5b0 0x7f59940ae390 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.616+0000 7f599c036640 1 -- 172.21.15.172:0/478997416 >> 172.21.15.172:0/478997416 conn(0x7f5994071ba0 msgr2=0x7f5994071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:33.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.617+0000 7f599c036640 1 -- 172.21.15.172:0/478997416 shutdown_connections 2024-04-14T08:43:33.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.617+0000 7f599c036640 1 -- 172.21.15.172:0/478997416 wait complete. 2024-04-14T08:43:33.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.618+0000 7f599c036640 1 Processor -- start 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.618+0000 7f599c036640 1 -- start start 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.618+0000 7f599c036640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f599406c530 0x7f59940cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.618+0000 7f599c036640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59940729b0 0x7f599415eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.618+0000 7f599c036640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f599415e5b0 0x7f5994145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.618+0000 7f599c036640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f59940af040 con 0x7f59940729b0 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.618+0000 7f599c036640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f59940aeec0 con 0x7f599406c530 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.618+0000 7f599c036640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f59940b5380 con 0x7f599415e5b0 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.618+0000 7f59995d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59940729b0 0x7f599415eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.618+0000 7f59995d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59940729b0 0x7f599415eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51082/0 (socket says 172.21.15.172:51082) 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.618+0000 7f59995d1640 1 -- 172.21.15.172:0/3345846331 learned_addr learned my addr 172.21.15.172:0/3345846331 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f5999dd2640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f599406c530 0x7f59940cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f599a5d3640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f599415e5b0 0x7f5994145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f59995d1640 1 -- 172.21.15.172:0/3345846331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f599415e5b0 msgr2=0x7f5994145020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f59995d1640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f599415e5b0 0x7f5994145020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f59995d1640 1 -- 172.21.15.172:0/3345846331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f599406c530 msgr2=0x7f59940cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f59995d1640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f599406c530 0x7f59940cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f59995d1640 1 -- 172.21.15.172:0/3345846331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59941578f0 con 0x7f59940729b0 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f5999dd2640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f599406c530 0x7f59940cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f59995d1640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59940729b0 0x7f599415eb20 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f598400c4a0 tx=0x7f598400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f598a7fc640 1 -- 172.21.15.172:0/3345846331 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984018070 con 0x7f59940729b0 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f598a7fc640 1 -- 172.21.15.172:0/3345846331 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59840074e0 con 0x7f59940729b0 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f599c036640 1 -- 172.21.15.172:0/3345846331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5994157bd0 con 0x7f59940729b0 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f598a7fc640 1 -- 172.21.15.172:0/3345846331 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984004a20 con 0x7f59940729b0 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.619+0000 7f599c036640 1 -- 172.21.15.172:0/3345846331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5994158190 con 0x7f59940729b0 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.620+0000 7f599c036640 1 -- 172.21.15.172:0/3345846331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59940686a0 con 0x7f59940729b0 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.621+0000 7f598a7fc640 1 -- 172.21.15.172:0/3345846331 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f598401b070 con 0x7f59940729b0 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.621+0000 7f598a7fc640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f597806c8f0 0x7f597806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.621+0000 7f598a7fc640 1 -- 172.21.15.172:0/3345846331 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f598408b3b0 con 0x7f59940729b0 2024-04-14T08:43:33.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.621+0000 7f5999dd2640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f597806c8f0 0x7f597806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.621+0000 7f598a7fc640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x7f5978072ed0 0x7f5978075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.621+0000 7f598a7fc640 1 -- 172.21.15.172:0/3345846331 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f598401fe50 con 0x7f5978072ed0 2024-04-14T08:43:33.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.621+0000 7f598a7fc640 1 -- 172.21.15.172:0/3345846331 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f598408b840 con 0x7f59940729b0 2024-04-14T08:43:33.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.621+0000 7f599a5d3640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x7f5978072ed0 0x7f5978075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.621+0000 7f5999dd2640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f597806c8f0 0x7f597806edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5990036000 tx=0x7f599003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.622+0000 7f599a5d3640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x7f5978072ed0 0x7f5978075310 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.623+0000 7f598a7fc640 1 -- 172.21.15.172:0/3345846331 <== osd.7 v2:172.21.15.177:6808/3609985207 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f598401fe50 con 0x7f5978072ed0 2024-04-14T08:43:33.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.635+0000 7f599c036640 1 -- 172.21.15.172:0/3345846331 --> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5994072790 con 0x7f5978072ed0 2024-04-14T08:43:33.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.636+0000 7f598a7fc640 1 -- 172.21.15.172:0/3345846331 <== osd.7 v2:172.21.15.177:6808/3609985207 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5994072790 con 0x7f5978072ed0 2024-04-14T08:43:33.637 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-14T08:43:33.637 INFO:teuthology.orchestra.run.smithi172.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-14T08:43:33.637 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-14T08:43:33.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.636+0000 7f599c036640 1 -- 172.21.15.172:0/3345846331 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x7f5978072ed0 msgr2=0x7f5978075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.636+0000 7f599c036640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.177:6808/3609985207,v1:172.21.15.177:6809/3609985207] conn(0x7f5978072ed0 0x7f5978075310 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.636+0000 7f599c036640 1 -- 172.21.15.172:0/3345846331 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f597806c8f0 msgr2=0x7f597806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.636+0000 7f599c036640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f597806c8f0 0x7f597806edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5990036000 tx=0x7f599003b040 comp rx=0 tx=0).stop 2024-04-14T08:43:33.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.636+0000 7f599c036640 1 -- 172.21.15.172:0/3345846331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59940729b0 msgr2=0x7f599415eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.636+0000 7f599c036640 1 --2- 172.21.15.172:0/3345846331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59940729b0 0x7f599415eb20 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f598400c4a0 tx=0x7f598400c960 comp rx=0 tx=0).stop 2024-04-14T08:43:33.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.636+0000 7f599a5d3640 1 -- 172.21.15.172:0/3345846331 reap_dead start 2024-04-14T08:43:33.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.636+0000 7f599c036640 1 -- 172.21.15.172:0/3345846331 shutdown_connections 2024-04-14T08:43:33.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.636+0000 7f599c036640 1 -- 172.21.15.172:0/3345846331 >> 172.21.15.172:0/3345846331 conn(0x7f5994071ba0 msgr2=0x7f59940b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:33.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.636+0000 7f599c036640 1 -- 172.21.15.172:0/3345846331 shutdown_connections 2024-04-14T08:43:33.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.636+0000 7f599c036640 1 -- 172.21.15.172:0/3345846331 wait complete. 2024-04-14T08:43:33.648 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-14T08:43:33.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.749+0000 7fb0de0ad640 1 Processor -- start 2024-04-14T08:43:33.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.749+0000 7fb0de0ad640 1 -- start start 2024-04-14T08:43:33.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.750+0000 7fb0de0ad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d815e3c0 0x7fb0d806c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.750+0000 7fb0de0ad640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0d806cb70 con 0x7fb0d806d040 2024-04-14T08:43:33.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.750+0000 7fb0de0ad640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0d80cc780 con 0x7fb0d80d1250 2024-04-14T08:43:33.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.750+0000 7fb0de0ad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d806ce40 con 0x7fb0d815e3c0 2024-04-14T08:43:33.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.750+0000 7fb0d6ffd640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fb0d806d040 0x7fb0d80cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43234/0 (socket says 172.21.15.172:43234) 2024-04-14T08:43:33.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.750+0000 7fb0d6ffd640 1 -- 172.21.15.172:0/3594402834 learned_addr learned my addr 172.21.15.172:0/3594402834 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4262967690 0 0) 0x7fb0d806cb70 con 0x7fb0d806d040 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0c0003520 con 0x7fb0d806d040 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d77fe640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d815e3c0 0x7fb0d806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4216990984 0 0) 0x7fb0c0003520 con 0x7fb0d806d040 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d815e3c0 msgr2=0x7fb0d806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d815e3c0 0x7fb0d806c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 >> v1:172.21.15.177:6789/0 conn(0x7fb0d80d1250 legacy=0x7fb0d80cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d80cca60 con 0x7fb0d806d040 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fb0cc001670 con 0x7fb0d806d040 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d77fe640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d815e3c0 0x7fb0d806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 >> v1:172.21.15.172:6789/0 conn(0x7fb0d806d040 legacy=0x7fb0d80cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0c00039e0 0x7fb0c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d815e3c0 0x7fb0c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0c0004d40 0x7fb0c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d806ce40 con 0x7fb0c00039e0 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c000bad0 con 0x7fb0c0004d40 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c000bcd0 con 0x7fb0d815e3c0 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d77fe640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0c00039e0 0x7fb0c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d6ffd640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d815e3c0 0x7fb0c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d7fff640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0c0004d40 0x7fb0c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d77fe640 1 -- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d815e3c0 msgr2=0x7fb0c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d77fe640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d815e3c0 0x7fb0c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d77fe640 1 -- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0c0004d40 msgr2=0x7fb0c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d77fe640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0c0004d40 0x7fb0c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d77fe640 1 -- 172.21.15.172:0/3594402834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c0003520 con 0x7fb0c00039e0 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.751+0000 7fb0d6ffd640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0d815e3c0 0x7fb0c0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.752+0000 7fb0d77fe640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0c00039e0 0x7fb0c0003e00 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8006fd0 tx=0x7fb0c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e13a0c0bcf1a87b server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.752+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c800f040 con 0x7fb0c00039e0 2024-04-14T08:43:33.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.752+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c802e070 con 0x7fb0c00039e0 2024-04-14T08:43:33.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.752+0000 7fb0d67fc640 1 -- 172.21.15.172:0/3594402834 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8004450 con 0x7fb0c00039e0 2024-04-14T08:43:33.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.753+0000 7fb0de0ad640 1 -- 172.21.15.172:0/3594402834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0c00039e0 msgr2=0x7fb0c0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.753+0000 7fb0de0ad640 1 --2- 172.21.15.172:0/3594402834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0c00039e0 0x7fb0c0003e00 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8006fd0 tx=0x7fb0c0005830 comp rx=0 tx=0).stop 2024-04-14T08:43:33.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.753+0000 7fb0d7fff640 1 -- 172.21.15.172:0/3594402834 reap_dead start 2024-04-14T08:43:33.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.753+0000 7fb0de0ad640 1 -- 172.21.15.172:0/3594402834 shutdown_connections 2024-04-14T08:43:33.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.753+0000 7fb0de0ad640 1 -- 172.21.15.172:0/3594402834 >> 172.21.15.172:0/3594402834 conn(0x7fb0d80b3af0 msgr2=0x7fb0d80b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:33.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.753+0000 7fb0de0ad640 1 -- 172.21.15.172:0/3594402834 shutdown_connections 2024-04-14T08:43:33.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.753+0000 7fb0de0ad640 1 -- 172.21.15.172:0/3594402834 wait complete. 2024-04-14T08:43:33.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.754+0000 7fb0de0ad640 1 Processor -- start 2024-04-14T08:43:33.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.754+0000 7fb0de0ad640 1 -- start start 2024-04-14T08:43:33.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0de0ad640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0c00039e0 0x7fb0d80cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0de0ad640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d81545e0 0x7fb0d809be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0de0ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d809d6c0 0x7fb0d80b7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0de0ad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d815e7a0 con 0x7fb0d809d6c0 2024-04-14T08:43:33.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0de0ad640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d80cd020 con 0x7fb0d81545e0 2024-04-14T08:43:33.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0de0ad640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fb0d806c630 con 0x7fb0c00039e0 2024-04-14T08:43:33.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d809d6c0 0x7fb0d80b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d809d6c0 0x7fb0d80b7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51098/0 (socket says 172.21.15.172:51098) 2024-04-14T08:43:33.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0d7fff640 1 -- 172.21.15.172:0/464918541 learned_addr learned my addr 172.21.15.172:0/464918541 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:33.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0d7fff640 1 -- 172.21.15.172:0/464918541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0c00039e0 msgr2=0x7fb0d80cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0d7fff640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0c00039e0 0x7fb0d80cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0d6ffd640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d81545e0 0x7fb0d809be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0d7fff640 1 -- 172.21.15.172:0/464918541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d81545e0 msgr2=0x7fb0d809be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0d7fff640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d81545e0 0x7fb0d809be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0d7fff640 1 -- 172.21.15.172:0/464918541 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d809c380 con 0x7fb0d809d6c0 2024-04-14T08:43:33.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0d7fff640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d809d6c0 0x7fb0d80b7360 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8072e40 tx=0x7fb0d000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0bffff640 1 -- 172.21.15.172:0/464918541 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0003970 con 0x7fb0d809d6c0 2024-04-14T08:43:33.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0bffff640 1 -- 172.21.15.172:0/464918541 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d0018070 con 0x7fb0d809d6c0 2024-04-14T08:43:33.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0de0ad640 1 -- 172.21.15.172:0/464918541 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d809c620 con 0x7fb0d809d6c0 2024-04-14T08:43:33.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0bffff640 1 -- 172.21.15.172:0/464918541 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0003d50 con 0x7fb0d809d6c0 2024-04-14T08:43:33.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.755+0000 7fb0de0ad640 1 -- 172.21.15.172:0/464918541 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d809cbe0 con 0x7fb0d809d6c0 2024-04-14T08:43:33.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.756+0000 7fb0de0ad640 1 -- 172.21.15.172:0/464918541 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d80cb8c0 con 0x7fb0d809d6c0 2024-04-14T08:43:33.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.757+0000 7fb0bffff640 1 -- 172.21.15.172:0/464918541 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0d00050d0 con 0x7fb0d809d6c0 2024-04-14T08:43:33.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.757+0000 7fb0bffff640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0c406c8f0 0x7fb0c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:33.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.757+0000 7fb0bffff640 1 -- 172.21.15.172:0/464918541 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7fb0d008b790 con 0x7fb0d809d6c0 2024-04-14T08:43:33.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.757+0000 7fb0d77fe640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0c406c8f0 0x7fb0c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:33.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.758+0000 7fb0d77fe640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0c406c8f0 0x7fb0c406edb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8035000 tx=0x7fb0c8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:33.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.762+0000 7fb0bffff640 1 -- 172.21.15.172:0/464918541 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb0d80cb8c0 con 0x7fb0d809d6c0 2024-04-14T08:43:33.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.936+0000 7fb0de0ad640 1 -- 172.21.15.172:0/464918541 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fb0d8068590 con 0x7fb0c406c8f0 2024-04-14T08:43:33.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.936+0000 7fb0bffff640 1 -- 172.21.15.172:0/464918541 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb0d8068590 con 0x7fb0c406c8f0 2024-04-14T08:43:33.937 INFO:teuthology.orchestra.run.smithi172.stderr:instructed osd(s) 7 to deep-scrub 2024-04-14T08:43:33.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.938+0000 7fb0de0ad640 1 -- 172.21.15.172:0/464918541 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0c406c8f0 msgr2=0x7fb0c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.938+0000 7fb0de0ad640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0c406c8f0 0x7fb0c406edb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8035000 tx=0x7fb0c8030000 comp rx=0 tx=0).stop 2024-04-14T08:43:33.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.938+0000 7fb0de0ad640 1 -- 172.21.15.172:0/464918541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d809d6c0 msgr2=0x7fb0d80b7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:33.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.938+0000 7fb0de0ad640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d809d6c0 0x7fb0d80b7360 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8072e40 tx=0x7fb0d000eb90 comp rx=0 tx=0).stop 2024-04-14T08:43:33.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.938+0000 7fb0de0ad640 1 -- 172.21.15.172:0/464918541 shutdown_connections 2024-04-14T08:43:33.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.938+0000 7fb0de0ad640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fb0c406c8f0 0x7fb0c406edb0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.938+0000 7fb0de0ad640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d809d6c0 0x7fb0d80b7360 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.938+0000 7fb0de0ad640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0d81545e0 0x7fb0d809be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.938+0000 7fb0de0ad640 1 --2- 172.21.15.172:0/464918541 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb0c00039e0 0x7fb0d80cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:33.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.938+0000 7fb0de0ad640 1 -- 172.21.15.172:0/464918541 >> 172.21.15.172:0/464918541 conn(0x7fb0d80b3af0 msgr2=0x7fb0d8160a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:33.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.938+0000 7fb0de0ad640 1 -- 172.21.15.172:0/464918541 shutdown_connections 2024-04-14T08:43:33.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:33.938+0000 7fb0de0ad640 1 -- 172.21.15.172:0/464918541 wait complete. 2024-04-14T08:43:33.951 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:43:34.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.051+0000 7f37209bb640 1 Processor -- start 2024-04-14T08:43:34.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.051+0000 7f37209bb640 1 -- start start 2024-04-14T08:43:34.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f37209bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c0b2290 0x7f371c15d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:34.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f37209bb640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f371c13f600 con 0x7f371c15f400 2024-04-14T08:43:34.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f37209bb640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f371c13fa80 con 0x7f371c15f7e0 2024-04-14T08:43:34.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f37209bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f371c140690 con 0x7f371c0b2290 2024-04-14T08:43:34.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371a59c640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f371c15f400 0x7f371c15d190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43244/0 (socket says 172.21.15.172:43244) 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371a59c640 1 -- 172.21.15.172:0/550818253 learned_addr learned my addr 172.21.15.172:0/550818253 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371ad9d640 1 --2- 172.21.15.172:0/550818253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c0b2290 0x7f371c15d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4019369215 0 0) 0x7f3714001490 con 0x7f371c15f400 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3704003520 con 0x7f371c15f400 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1353866902 0 0) 0x7f371c13fa80 con 0x7f371c15f7e0 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3714001490 con 0x7f371c15f7e0 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 999108603 0 0) 0x7f3704003520 con 0x7f371c15f400 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c0b2290 msgr2=0x7f371c15d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371959a640 1 --2- 172.21.15.172:0/550818253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c0b2290 0x7f371c15d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 >> v1:172.21.15.177:6789/0 conn(0x7f371c15f7e0 legacy=0x7f371c0b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371c13fd60 con 0x7f371c15f400 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.052+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f3714001b40 con 0x7f371c15f400 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 >> v1:172.21.15.172:6789/0 conn(0x7f371c15f400 legacy=0x7f371c15d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:34.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f371959a640 1 --2- 172.21.15.172:0/550818253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c0b2290 0x7f3704003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:34.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f371959a640 1 --2- 172.21.15.172:0/550818253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3704004120 0x7f3704004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:34.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f371959a640 1 --2- 172.21.15.172:0/550818253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3704004d40 0x7f37040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:34.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f371c140690 con 0x7f3704004120 2024-04-14T08:43:34.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f370400bb00 con 0x7f3704004d40 2024-04-14T08:43:34.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f370400bd00 con 0x7f371c0b2290 2024-04-14T08:43:34.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f3719d9b640 1 --2- 172.21.15.172:0/550818253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3704004120 0x7f3704004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:34.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f371a59c640 1 --2- 172.21.15.172:0/550818253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c0b2290 0x7f3704003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:34.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f371ad9d640 1 --2- 172.21.15.172:0/550818253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3704004d40 0x7f37040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:34.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f3719d9b640 1 -- 172.21.15.172:0/550818253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c0b2290 msgr2=0x7f3704003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:34.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f3719d9b640 1 --2- 172.21.15.172:0/550818253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c0b2290 0x7f3704003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:34.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f3719d9b640 1 -- 172.21.15.172:0/550818253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3704004d40 msgr2=0x7f37040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:34.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f3719d9b640 1 --2- 172.21.15.172:0/550818253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3704004d40 0x7f37040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:34.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.053+0000 7f3719d9b640 1 -- 172.21.15.172:0/550818253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3704003520 con 0x7f3704004120 2024-04-14T08:43:34.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.054+0000 7f3719d9b640 1 --2- 172.21.15.172:0/550818253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3704004120 0x7f3704004800 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f37100018d0 tx=0x7f3704005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424daa68e8aad88 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:34.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.054+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37100014f0 con 0x7f3704004120 2024-04-14T08:43:34.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.054+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37100052d0 con 0x7f3704004120 2024-04-14T08:43:34.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.054+0000 7f371959a640 1 -- 172.21.15.172:0/550818253 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710003810 con 0x7f3704004120 2024-04-14T08:43:34.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.054+0000 7f37209bb640 1 -- 172.21.15.172:0/550818253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3704004120 msgr2=0x7f3704004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:34.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.054+0000 7f37209bb640 1 --2- 172.21.15.172:0/550818253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3704004120 0x7f3704004800 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f37100018d0 tx=0x7f3704005830 comp rx=0 tx=0).stop 2024-04-14T08:43:34.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.054+0000 7f371ad9d640 1 -- 172.21.15.172:0/550818253 reap_dead start 2024-04-14T08:43:34.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.054+0000 7f37209bb640 1 -- 172.21.15.172:0/550818253 shutdown_connections 2024-04-14T08:43:34.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.054+0000 7f37209bb640 1 -- 172.21.15.172:0/550818253 >> 172.21.15.172:0/550818253 conn(0x7f371c06ec30 msgr2=0x7f371c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:34.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.054+0000 7f37209bb640 1 -- 172.21.15.172:0/550818253 shutdown_connections 2024-04-14T08:43:34.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.054+0000 7f37209bb640 1 -- 172.21.15.172:0/550818253 wait complete. 2024-04-14T08:43:34.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.056+0000 7f37209bb640 1 Processor -- start 2024-04-14T08:43:34.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.056+0000 7f37209bb640 1 -- start start 2024-04-14T08:43:34.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.056+0000 7f37209bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3704004120 0x7f371c1520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:34.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.056+0000 7f37209bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f371c0cfa30 0x7f371c0cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.056+0000 7f37209bb640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c1f1ef0 0x7f371c1f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f37209bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f371c15d9d0 con 0x7f371c0cfa30 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f37209bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f371c140320 con 0x7f3704004120 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f37209bb640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f371c15db50 con 0x7f371c1f1ef0 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f3719d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f371c0cfa30 0x7f371c0cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f3719d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f371c0cfa30 0x7f371c0cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51104/0 (socket says 172.21.15.172:51104) 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f3719d9b640 1 -- 172.21.15.172:0/510320789 learned_addr learned my addr 172.21.15.172:0/510320789 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f371a59c640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3704004120 0x7f371c1520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f371ad9d640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c1f1ef0 0x7f371c1f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f3719d9b640 1 -- 172.21.15.172:0/510320789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c1f1ef0 msgr2=0x7f371c1f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f3719d9b640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c1f1ef0 0x7f371c1f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f3719d9b640 1 -- 172.21.15.172:0/510320789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3704004120 msgr2=0x7f371c1520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:34.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f3719d9b640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3704004120 0x7f371c1520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f3719d9b640 1 -- 172.21.15.172:0/510320789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371c0d03f0 con 0x7f371c0cfa30 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f371ad9d640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c1f1ef0 0x7f371c1f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f371a59c640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3704004120 0x7f371c1520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f3719d9b640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f371c0cfa30 0x7f371c0cfeb0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f37100018d0 tx=0x7f37100012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f370affd640 1 -- 172.21.15.172:0/510320789 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710042030 con 0x7f371c0cfa30 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f370affd640 1 -- 172.21.15.172:0/510320789 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37100050d0 con 0x7f371c0cfa30 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f370affd640 1 -- 172.21.15.172:0/510320789 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710007250 con 0x7f371c0cfa30 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f37209bb640 1 -- 172.21.15.172:0/510320789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371c1f28b0 con 0x7f371c0cfa30 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.057+0000 7f37209bb640 1 -- 172.21.15.172:0/510320789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371c0d0960 con 0x7f371c0cfa30 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.058+0000 7f37209bb640 1 -- 172.21.15.172:0/510320789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f371c071fb0 con 0x7f371c0cfa30 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.059+0000 7f370affd640 1 -- 172.21.15.172:0/510320789 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3710045070 con 0x7f371c0cfa30 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.059+0000 7f370affd640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f36fc06c920 0x7f36fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.059+0000 7f371a59c640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f36fc06c920 0x7f36fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.059+0000 7f370affd640 1 -- 172.21.15.172:0/510320789 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f37100ad050 con 0x7f371c0cfa30 2024-04-14T08:43:34.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.060+0000 7f371a59c640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f36fc06c920 0x7f36fc06ede0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f371c0b1ed0 tx=0x7f3714006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:34.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.064+0000 7f370affd640 1 -- 172.21.15.172:0/510320789 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f371c071fb0 con 0x7f371c0cfa30 2024-04-14T08:43:34.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.237+0000 7f37209bb640 1 -- 172.21.15.172:0/510320789 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f371c075040 con 0x7f36fc06c920 2024-04-14T08:43:34.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.238+0000 7f370affd640 1 -- 172.21.15.172:0/510320789 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13499 (secure 0 0 0) 0x7f371c075040 con 0x7f36fc06c920 2024-04-14T08:43:34.239 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:34.239 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:43:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.240+0000 7f37209bb640 1 -- 172.21.15.172:0/510320789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f36fc06c920 msgr2=0x7f36fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.240+0000 7f37209bb640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f36fc06c920 0x7f36fc06ede0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f371c0b1ed0 tx=0x7f3714006040 comp rx=0 tx=0).stop 2024-04-14T08:43:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.240+0000 7f37209bb640 1 -- 172.21.15.172:0/510320789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f371c0cfa30 msgr2=0x7f371c0cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.240+0000 7f37209bb640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f371c0cfa30 0x7f371c0cfeb0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f37100018d0 tx=0x7f37100012d0 comp rx=0 tx=0).stop 2024-04-14T08:43:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.240+0000 7f37209bb640 1 -- 172.21.15.172:0/510320789 shutdown_connections 2024-04-14T08:43:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.240+0000 7f37209bb640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f36fc06c920 0x7f36fc06ede0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.240+0000 7f37209bb640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f371c1f1ef0 0x7f371c1f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.240+0000 7f37209bb640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f371c0cfa30 0x7f371c0cfeb0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.240+0000 7f37209bb640 1 --2- 172.21.15.172:0/510320789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3704004120 0x7f371c1520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.240+0000 7f37209bb640 1 -- 172.21.15.172:0/510320789 >> 172.21.15.172:0/510320789 conn(0x7f371c06ec30 msgr2=0x7f371c06b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:34.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.240+0000 7f37209bb640 1 -- 172.21.15.172:0/510320789 shutdown_connections 2024-04-14T08:43:34.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:34.240+0000 7f37209bb640 1 -- 172.21.15.172:0/510320789 wait complete. 2024-04-14T08:43:34.252 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-04-14T08:43:33.247831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262476,"kb_used_data":13000,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712244,"statfs":{"total":773094113280,"available":772825337856,"internally_reserved":0,"allocated":13312000,"data_stored":6830280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390540,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-175,"num_write_kb":-8303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001382"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":375,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-14T08:43:26.313740+0000","last_change":"2024-04-14T08:32:43.197065+0000","last_active":"2024-04-14T08:43:26.313740+0000","last_peered":"2024-04-14T08:43:26.313740+0000","last_clean":"2024-04-14T08:43:26.313740+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:43:26.313740+0000","last_undegraded":"2024-04-14T08:43:26.313740+0000","last_fullsized":"2024-04-14T08:43:26.313740+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T08:32:42.127080+0000","last_clean_scrub_stamp":"2024-04-14T08:32:42.127080+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:54.112902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574981,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2109440,"data_stored":1296561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":706193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":706193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":706193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":706193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":706193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":706193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2109440,"data_stored":1296561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:43:34.257 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-14T08:32:42.127080+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=14, tm_hour=8, tm_min=32, tm_sec=42, tm_wday=6, tm_yday=105, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=14, tm_hour=8, tm_min=43, tm_sec=29, tm_wday=6, tm_yday=105, tm_isdst=0) 2024-04-14T08:43:34.257 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-14T08:43:54.259 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-14T08:43:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.372+0000 7fefcb75e640 1 Processor -- start 2024-04-14T08:43:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.372+0000 7fefcb75e640 1 -- start start 2024-04-14T08:43:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.373+0000 7fefcb75e640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fefc415e890 0x7fefc40d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.373+0000 7fefcb75e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc40cedd0 0x7fefc40b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.373+0000 7fefcb75e640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc40cf320 con 0x7fefc40d2830 2024-04-14T08:43:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.373+0000 7fefcb75e640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7fefc40cff30 con 0x7fefc415e890 2024-04-14T08:43:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.373+0000 7fefcb75e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fefc40d2350 con 0x7fefc40cedd0 2024-04-14T08:43:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.373+0000 7fefc94fa640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fefc40d2830 0x7fefc40d2c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44200/0 (socket says 172.21.15.172:44200) 2024-04-14T08:43:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.373+0000 7fefc94fa640 1 -- 172.21.15.172:0/2162175509 learned_addr learned my addr 172.21.15.172:0/2162175509 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefc8cf9640 1 --2- 172.21.15.172:0/2162175509 >> v2:172.21.15.177:3300/0 conn(0x7fefc415e890 0x7fefc40d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2491330073 0 0) 0x7fefc40cf320 con 0x7fefc40d2830 2024-04-14T08:43:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefc9cfb640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc40cedd0 0x7fefc40b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefb0003520 con 0x7fefc40d2830 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4047701878 0 0) 0x7fefb0003520 con 0x7fefc40d2830 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc40cedd0 msgr2=0x7fefc40b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc40cedd0 0x7fefc40b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 >> v2:172.21.15.177:3300/0 conn(0x7fefc415e890 msgr2=0x7fefc40d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 --2- 172.21.15.172:0/2162175509 >> v2:172.21.15.177:3300/0 conn(0x7fefc415e890 0x7fefc40d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc40cf600 con 0x7fefc40d2830 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7fefc0001670 con 0x7fefc40d2830 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 >> v1:172.21.15.172:6789/0 conn(0x7fefc40d2830 legacy=0x7fefc40d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefb00039b0 0x7fefb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc40cedd0 0x7fefb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb0004d40 0x7fefb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fefc40cff30 con 0x7fefb0004d40 2024-04-14T08:43:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fefc40d2350 con 0x7fefb00039b0 2024-04-14T08:43:54.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fefb000b9c0 con 0x7fefc40cedd0 2024-04-14T08:43:54.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefc94fa640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefb00039b0 0x7fefb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefc9cfb640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb0004d40 0x7fefb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefc8cf9640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc40cedd0 0x7fefb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefc9cfb640 1 -- 172.21.15.172:0/2162175509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc40cedd0 msgr2=0x7fefb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:54.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefc9cfb640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc40cedd0 0x7fefb0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefc9cfb640 1 -- 172.21.15.172:0/2162175509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefb00039b0 msgr2=0x7fefb0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:54.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefc9cfb640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefb00039b0 0x7fefb0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefc9cfb640 1 -- 172.21.15.172:0/2162175509 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb0003520 con 0x7fefb0004d40 2024-04-14T08:43:54.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.374+0000 7fefc94fa640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefb00039b0 0x7fefb0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-14T08:43:54.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.375+0000 7fefc9cfb640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb0004d40 0x7fefb00051c0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009fc0 tx=0x7fefb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51afd668178c8c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:54.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.375+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc039070 con 0x7fefb0004d40 2024-04-14T08:43:54.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.375+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefbc02cc20 con 0x7fefb0004d40 2024-04-14T08:43:54.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.375+0000 7fefbbfff640 1 -- 172.21.15.172:0/2162175509 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc009c10 con 0x7fefb0004d40 2024-04-14T08:43:54.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.376+0000 7fefcb75e640 1 -- 172.21.15.172:0/2162175509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb0004d40 msgr2=0x7fefb00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.376+0000 7fefcb75e640 1 --2- 172.21.15.172:0/2162175509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb0004d40 0x7fefb00051c0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009fc0 tx=0x7fefb0005830 comp rx=0 tx=0).stop 2024-04-14T08:43:54.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.376+0000 7fefc9cfb640 1 -- 172.21.15.172:0/2162175509 reap_dead start 2024-04-14T08:43:54.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.376+0000 7fefcb75e640 1 -- 172.21.15.172:0/2162175509 shutdown_connections 2024-04-14T08:43:54.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.376+0000 7fefcb75e640 1 -- 172.21.15.172:0/2162175509 >> 172.21.15.172:0/2162175509 conn(0x7fefc409d590 msgr2=0x7fefc40d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:54.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.376+0000 7fefcb75e640 1 -- 172.21.15.172:0/2162175509 shutdown_connections 2024-04-14T08:43:54.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.376+0000 7fefcb75e640 1 -- 172.21.15.172:0/2162175509 wait complete. 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.377+0000 7fefcb75e640 1 Processor -- start 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.377+0000 7fefcb75e640 1 -- start start 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefcb75e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb0004d40 0x7fefc40b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefcb75e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefc4159600 0x7fefc415ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefcb75e640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc41e8960 0x7fefc40c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefcb75e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fefc40d1840 con 0x7fefb0004d40 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefcb75e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fefc40cfbc0 con 0x7fefc4159600 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefcb75e640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fefc40b3710 con 0x7fefc41e8960 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefc94fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb0004d40 0x7fefc40b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefc94fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb0004d40 0x7fefc40b6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38110/0 (socket says 172.21.15.172:38110) 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefc94fa640 1 -- 172.21.15.172:0/283208257 learned_addr learned my addr 172.21.15.172:0/283208257 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefc8cf9640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefc4159600 0x7fefc415ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefc9cfb640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc41e8960 0x7fefc40c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefc94fa640 1 -- 172.21.15.172:0/283208257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc41e8960 msgr2=0x7fefc40c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefc94fa640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc41e8960 0x7fefc40c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefc94fa640 1 -- 172.21.15.172:0/283208257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefc4159600 msgr2=0x7fefc415ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefc94fa640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefc4159600 0x7fefc415ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.378+0000 7fefc94fa640 1 -- 172.21.15.172:0/283208257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc40b55d0 con 0x7fefb0004d40 2024-04-14T08:43:54.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.379+0000 7fefc94fa640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb0004d40 0x7fefc40b6670 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fefc0007d40 tx=0x7fefc0007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:54.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.381+0000 7fefb9ffb640 1 -- 172.21.15.172:0/283208257 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc000f9f0 con 0x7fefb0004d40 2024-04-14T08:43:54.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.381+0000 7fefb9ffb640 1 -- 172.21.15.172:0/283208257 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc0018070 con 0x7fefb0004d40 2024-04-14T08:43:54.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.381+0000 7fefcb75e640 1 -- 172.21.15.172:0/283208257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc40b58b0 con 0x7fefb0004d40 2024-04-14T08:43:54.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.381+0000 7fefb9ffb640 1 -- 172.21.15.172:0/283208257 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc000fdb0 con 0x7fefb0004d40 2024-04-14T08:43:54.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.381+0000 7fefcb75e640 1 -- 172.21.15.172:0/283208257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc40b5e70 con 0x7fefb0004d40 2024-04-14T08:43:54.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.382+0000 7fefcb75e640 1 -- 172.21.15.172:0/283208257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc40d30a0 con 0x7fefb0004d40 2024-04-14T08:43:54.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.383+0000 7fefb9ffb640 1 -- 172.21.15.172:0/283208257 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefc001d460 con 0x7fefb0004d40 2024-04-14T08:43:54.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.383+0000 7fefb9ffb640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fefa806c8f0 0x7fefa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.383+0000 7fefb9ffb640 1 -- 172.21.15.172:0/283208257 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7fefc008bcd0 con 0x7fefb0004d40 2024-04-14T08:43:54.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.383+0000 7fefc8cf9640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fefa806c8f0 0x7fefa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.383+0000 7fefc8cf9640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fefa806c8f0 0x7fefa806edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fefc415dc20 tx=0x7fefb4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:54.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.388+0000 7fefb9ffb640 1 -- 172.21.15.172:0/283208257 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fefc005a1c0 con 0x7fefb0004d40 2024-04-14T08:43:54.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.559+0000 7fefcb75e640 1 -- 172.21.15.172:0/283208257 --> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefc40d2610 con 0x7fefa806c8f0 2024-04-14T08:43:54.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.559+0000 7fefb9ffb640 1 -- 172.21.15.172:0/283208257 <== mgr.4104 v2:172.21.15.172:6832/1188012648 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7fefc40d2610 con 0x7fefa806c8f0 2024-04-14T08:43:54.561 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:54.561 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-04-14T08:43:54.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.561+0000 7fefcb75e640 1 -- 172.21.15.172:0/283208257 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fefa806c8f0 msgr2=0x7fefa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.561+0000 7fefcb75e640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fefa806c8f0 0x7fefa806edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fefc415dc20 tx=0x7fefb4009190 comp rx=0 tx=0).stop 2024-04-14T08:43:54.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.561+0000 7fefcb75e640 1 -- 172.21.15.172:0/283208257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb0004d40 msgr2=0x7fefc40b6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.561+0000 7fefcb75e640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb0004d40 0x7fefc40b6670 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fefc0007d40 tx=0x7fefc0007750 comp rx=0 tx=0).stop 2024-04-14T08:43:54.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.562+0000 7fefcb75e640 1 -- 172.21.15.172:0/283208257 shutdown_connections 2024-04-14T08:43:54.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.562+0000 7fefcb75e640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fefa806c8f0 0x7fefa806edb0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.562+0000 7fefcb75e640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefc41e8960 0x7fefc40c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.562+0000 7fefcb75e640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefc4159600 0x7fefc415ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.562+0000 7fefcb75e640 1 --2- 172.21.15.172:0/283208257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb0004d40 0x7fefc40b6670 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.562+0000 7fefcb75e640 1 -- 172.21.15.172:0/283208257 >> 172.21.15.172:0/283208257 conn(0x7fefc409d590 msgr2=0x7fefc40d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:54.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.562+0000 7fefcb75e640 1 -- 172.21.15.172:0/283208257 shutdown_connections 2024-04-14T08:43:54.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.562+0000 7fefcb75e640 1 -- 172.21.15.172:0/283208257 wait complete. 2024-04-14T08:43:54.573 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-04-14T08:43:53.251905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262456,"kb_used_data":13000,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712264,"statfs":{"total":773094113280,"available":772825358336,"internally_reserved":0,"allocated":13312000,"data_stored":6830280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002441"},"pg_stats":[{"pgid":"1.0","version":"121'119","reported_seq":386,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-14T08:43:34.732748+0000","last_change":"2024-04-14T08:43:34.732782+0000","last_active":"2024-04-14T08:43:34.732748+0000","last_peered":"2024-04-14T08:43:34.732748+0000","last_clean":"2024-04-14T08:43:34.732748+0000","last_became_active":"2024-04-14T08:32:43.196200+0000","last_became_peered":"2024-04-14T08:32:43.196200+0000","last_unstale":"2024-04-14T08:43:34.732748+0000","last_undegraded":"2024-04-14T08:43:34.732748+0000","last_fullsized":"2024-04-14T08:43:34.732748+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"121'119","last_scrub_stamp":"2024-04-14T08:43:34.732650+0000","last_deep_scrub":"121'119","last_deep_scrub_stamp":"2024-04-14T08:43:34.732650+0000","last_clean_scrub_stamp":"2024-04-14T08:43:34.732650+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T14:25:55.043115+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574985,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2109440,"data_stored":1296561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":706193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":706193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":706193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":706193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":706193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":706193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574985,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2109440,"data_stored":1296561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T08:43:54.574 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-14T08:43:54.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.674+0000 7fc0d15e0640 1 Processor -- start 2024-04-14T08:43:54.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.674+0000 7fc0d15e0640 1 -- start start 2024-04-14T08:43:54.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0d15e0640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc0cc0d1310 0x7fc0cc0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0d15e0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc0b4d30 0x7fc0cc0cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0d15e0640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0cc0b5a20 con 0x7fc0cc161d10 2024-04-14T08:43:54.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0d15e0640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 -- 0x7fc0cc0d2580 con 0x7fc0cc0d1310 2024-04-14T08:43:54.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0d15e0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc0cc0d2860 con 0x7fc0cc0b4d30 2024-04-14T08:43:54.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0ca7fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc0cc0d1310 0x7fc0cc0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0ca7fc640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc0cc0d1310 0x7fc0cc0cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38124/0 (socket says 172.21.15.172:38124) 2024-04-14T08:43:54.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0ca7fc640 1 -- 172.21.15.172:0/3907986392 learned_addr learned my addr 172.21.15.172:0/3907986392 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:54.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0cb7fe640 1 --2- 172.21.15.172:0/3907986392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc0b4d30 0x7fc0cc0cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0ca7fc640 1 -- 172.21.15.172:0/3907986392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc0b4d30 msgr2=0x7fc0cc0cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:54.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0ca7fc640 1 --2- 172.21.15.172:0/3907986392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc0b4d30 0x7fc0cc0cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0ca7fc640 1 -- 172.21.15.172:0/3907986392 >> v1:172.21.15.177:6789/0 conn(0x7fc0cc161d10 legacy=0x7fc0cc0d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.675+0000 7fc0ca7fc640 1 -- 172.21.15.172:0/3907986392 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc0d2b40 con 0x7fc0cc0d1310 2024-04-14T08:43:54.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.676+0000 7fc0ca7fc640 1 --2- 172.21.15.172:0/3907986392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0cc0d1310 0x7fc0cc0cee40 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009870 tx=0x7fc0c002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dd3ff877b2b569 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:54.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.676+0000 7fc0c9ffb640 1 -- 172.21.15.172:0/3907986392 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0034070 con 0x7fc0cc0d1310 2024-04-14T08:43:54.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.676+0000 7fc0c9ffb640 1 -- 172.21.15.172:0/3907986392 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c0030840 con 0x7fc0cc0d1310 2024-04-14T08:43:54.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.676+0000 7fc0c9ffb640 1 -- 172.21.15.172:0/3907986392 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0030c10 con 0x7fc0cc0d1310 2024-04-14T08:43:54.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.677+0000 7fc0d15e0640 1 -- 172.21.15.172:0/3907986392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0cc0d1310 msgr2=0x7fc0cc0cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.677+0000 7fc0d15e0640 1 --2- 172.21.15.172:0/3907986392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0cc0d1310 0x7fc0cc0cee40 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009870 tx=0x7fc0c002b980 comp rx=0 tx=0).stop 2024-04-14T08:43:54.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.677+0000 7fc0d15e0640 1 -- 172.21.15.172:0/3907986392 shutdown_connections 2024-04-14T08:43:54.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.677+0000 7fc0d15e0640 1 --2- 172.21.15.172:0/3907986392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc0b4d30 0x7fc0cc0cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.677+0000 7fc0d15e0640 1 --2- 172.21.15.172:0/3907986392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0cc0d1310 0x7fc0cc0cee40 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.677+0000 7fc0d15e0640 1 -- 172.21.15.172:0/3907986392 >> 172.21.15.172:0/3907986392 conn(0x7fc0cc09d580 msgr2=0x7fc0cc0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:54.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.677+0000 7fc0d15e0640 1 -- 172.21.15.172:0/3907986392 shutdown_connections 2024-04-14T08:43:54.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.677+0000 7fc0d15e0640 1 -- 172.21.15.172:0/3907986392 wait complete. 2024-04-14T08:43:54.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.678+0000 7fc0d15e0640 1 Processor -- start 2024-04-14T08:43:54.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.678+0000 7fc0d15e0640 1 -- start start 2024-04-14T08:43:54.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0d15e0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0cc0b4d30 0x7fc0cc1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0d15e0640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc0d1310 0x7fc0cc15f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0d15e0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0cc161d10 0x7fc0cc15f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0d15e0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc0b3000 con 0x7fc0cc161d10 2024-04-14T08:43:54.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0d15e0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc0d3100 con 0x7fc0cc0b4d30 2024-04-14T08:43:54.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0d15e0640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7fc0cc0b3180 con 0x7fc0cc0d1310 2024-04-14T08:43:54.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0cb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0cc161d10 0x7fc0cc15f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0cb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0cc161d10 0x7fc0cc15f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38132/0 (socket says 172.21.15.172:38132) 2024-04-14T08:43:54.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0cb7fe640 1 -- 172.21.15.172:0/1131478193 learned_addr learned my addr 172.21.15.172:0/1131478193 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0caffd640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0cc0b4d30 0x7fc0cc1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0ca7fc640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc0d1310 0x7fc0cc15f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0cb7fe640 1 -- 172.21.15.172:0/1131478193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc0d1310 msgr2=0x7fc0cc15f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0cb7fe640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc0d1310 0x7fc0cc15f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0cb7fe640 1 -- 172.21.15.172:0/1131478193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0cc0b4d30 msgr2=0x7fc0cc1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0cb7fe640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0cc0b4d30 0x7fc0cc1592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0cb7fe640 1 -- 172.21.15.172:0/1131478193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc0c7f20 con 0x7fc0cc161d10 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.679+0000 7fc0caffd640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0cc0b4d30 0x7fc0cc1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.680+0000 7fc0cb7fe640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0cc161d10 0x7fc0cc15f580 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4007e70 tx=0x7fc0c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.680+0000 7fc0b37fe640 1 -- 172.21.15.172:0/1131478193 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4014620 con 0x7fc0cc161d10 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.680+0000 7fc0b37fe640 1 -- 172.21.15.172:0/1131478193 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c4019070 con 0x7fc0cc161d10 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.680+0000 7fc0d15e0640 1 -- 172.21.15.172:0/1131478193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0cc0c8200 con 0x7fc0cc161d10 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.680+0000 7fc0b37fe640 1 -- 172.21.15.172:0/1131478193 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4014a20 con 0x7fc0cc161d10 2024-04-14T08:43:54.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.680+0000 7fc0d15e0640 1 -- 172.21.15.172:0/1131478193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0cc0c87c0 con 0x7fc0cc161d10 2024-04-14T08:43:54.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.681+0000 7fc0d15e0640 1 -- 172.21.15.172:0/1131478193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0cc068520 con 0x7fc0cc161d10 2024-04-14T08:43:54.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.681+0000 7fc0b37fe640 1 -- 172.21.15.172:0/1131478193 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0c4014bc0 con 0x7fc0cc161d10 2024-04-14T08:43:54.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.681+0000 7fc0b37fe640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc0b806c8f0 0x7fc0b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.682+0000 7fc0b37fe640 1 -- 172.21.15.172:0/1131478193 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7fc0c408c450 con 0x7fc0cc161d10 2024-04-14T08:43:54.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.682+0000 7fc0caffd640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc0b806c8f0 0x7fc0b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.682+0000 7fc0caffd640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc0b806c8f0 0x7fc0b806edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc007fd0 tx=0x7fc0bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:54.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.686+0000 7fc0b37fe640 1 -- 172.21.15.172:0/1131478193 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc0c4011260 con 0x7fc0cc161d10 2024-04-14T08:43:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.860+0000 7fc0d15e0640 1 -- 172.21.15.172:0/1131478193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fc0cc0d0640 con 0x7fc0cc161d10 2024-04-14T08:43:54.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.863+0000 7fc0b37fe640 1 -- 172.21.15.172:0/1131478193 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7fc0c405a940 con 0x7fc0cc161d10 2024-04-14T08:43:54.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.864+0000 7fc0b37fe640 1 -- 172.21.15.172:0/1131478193 <== mon.0 v2:172.21.15.172:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc0c405a240 con 0x7fc0cc161d10 2024-04-14T08:43:54.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.866+0000 7fc0d15e0640 1 -- 172.21.15.172:0/1131478193 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc0b806c8f0 msgr2=0x7fc0b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.866+0000 7fc0d15e0640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc0b806c8f0 0x7fc0b806edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc007fd0 tx=0x7fc0bc009040 comp rx=0 tx=0).stop 2024-04-14T08:43:54.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.866+0000 7fc0d15e0640 1 -- 172.21.15.172:0/1131478193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0cc161d10 msgr2=0x7fc0cc15f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.866+0000 7fc0d15e0640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0cc161d10 0x7fc0cc15f580 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4007e70 tx=0x7fc0c400e4b0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.866+0000 7fc0d15e0640 1 -- 172.21.15.172:0/1131478193 shutdown_connections 2024-04-14T08:43:54.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.866+0000 7fc0d15e0640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7fc0b806c8f0 0x7fc0b806edb0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.866+0000 7fc0d15e0640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0cc161d10 0x7fc0cc15f580 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.866+0000 7fc0d15e0640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc0d1310 0x7fc0cc15f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.866+0000 7fc0d15e0640 1 --2- 172.21.15.172:0/1131478193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0cc0b4d30 0x7fc0cc1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.866+0000 7fc0d15e0640 1 -- 172.21.15.172:0/1131478193 >> 172.21.15.172:0/1131478193 conn(0x7fc0cc09d580 msgr2=0x7fc0cc0cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:54.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.866+0000 7fc0d15e0640 1 -- 172.21.15.172:0/1131478193 shutdown_connections 2024-04-14T08:43:54.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.866+0000 7fc0d15e0640 1 -- 172.21.15.172:0/1131478193 wait complete. 2024-04-14T08:43:54.878 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-14T08:43:54.879 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-14T08:43:54.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.978+0000 7f4e93508640 1 Processor -- start 2024-04-14T08:43:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.979+0000 7f4e93508640 1 -- start start 2024-04-14T08:43:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.979+0000 7f4e93508640 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4e8c0d2830 0x7f4e8c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.979+0000 7f4e93508640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e8c0d3150 con 0x7f4e8c0cedd0 2024-04-14T08:43:54.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.979+0000 7f4e93508640 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e8c0cf320 con 0x7f4e8c15e890 2024-04-14T08:43:54.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.979+0000 7f4e93508640 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 -- 0x7f4e8c0cff30 con 0x7f4e8c0d2830 2024-04-14T08:43:54.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e91aa5640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4e8c0cedd0 0x7f4e8c0b3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:44214/0 (socket says 172.21.15.172:44214) 2024-04-14T08:43:54.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e91aa5640 1 -- 172.21.15.172:0/174250579 learned_addr learned my addr 172.21.15.172:0/174250579 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-14T08:43:54.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e912a4640 1 --2- 172.21.15.172:0/174250579 >> v2:172.21.15.177:3300/0 conn(0x7f4e8c0d2830 0x7f4e8c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 <== mon.0 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 297992618 0 0) 0x7f4e8c0d3150 con 0x7f4e8c0cedd0 2024-04-14T08:43:54.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e78003520 con 0x7f4e8c0cedd0 2024-04-14T08:43:54.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 <== mon.0 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4148628300 0 0) 0x7f4e78003520 con 0x7f4e8c0cedd0 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 >> v2:172.21.15.177:3300/0 conn(0x7f4e8c0d2830 msgr2=0x7f4e8c0d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 --2- 172.21.15.172:0/174250579 >> v2:172.21.15.177:3300/0 conn(0x7f4e8c0d2830 0x7f4e8c0d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 >> v1:172.21.15.177:6789/0 conn(0x7f4e8c15e890 legacy=0x7f4e8c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 --> v1:172.21.15.172:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c0cf600 con 0x7f4e8c0cedd0 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 <== mon.0 v1:172.21.15.172:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 516673318 0 0) 0x7f4e88001670 con 0x7f4e8c0cedd0 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 >> v1:172.21.15.172:6789/0 conn(0x7f4e8c0cedd0 legacy=0x7f4e8c0b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 --2- 172.21.15.172:0/174250579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e780039b0 0x7f4e78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 --2- 172.21.15.172:0/174250579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e78004800 0x7f4e78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 --2- 172.21.15.172:0/174250579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e780051c0 0x7f4e7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c0cff30 con 0x7f4e780051c0 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7800c070 con 0x7f4e780039b0 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7800c270 con 0x7f4e78004800 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e91aa5640 1 --2- 172.21.15.172:0/174250579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e780051c0 0x7f4e7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e912a4640 1 --2- 172.21.15.172:0/174250579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e780039b0 0x7f4e78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.980+0000 7f4e90aa3640 1 --2- 172.21.15.172:0/174250579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e78004800 0x7f4e78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.981+0000 7f4e91aa5640 1 -- 172.21.15.172:0/174250579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e78004800 msgr2=0x7f4e78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:54.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.981+0000 7f4e91aa5640 1 --2- 172.21.15.172:0/174250579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e78004800 0x7f4e78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.981+0000 7f4e91aa5640 1 -- 172.21.15.172:0/174250579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e780039b0 msgr2=0x7f4e78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-14T08:43:54.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.981+0000 7f4e91aa5640 1 --2- 172.21.15.172:0/174250579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e780039b0 0x7f4e78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.981+0000 7f4e91aa5640 1 -- 172.21.15.172:0/174250579 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e78003520 con 0x7f4e780051c0 2024-04-14T08:43:54.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.981+0000 7f4e91aa5640 1 -- 172.21.15.172:0/174250579 reap_dead start 2024-04-14T08:43:54.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.981+0000 7f4e91aa5640 1 --2- 172.21.15.172:0/174250579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e780051c0 0x7f4e7800b9c0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800c210 tx=0x7f4e78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22d11d9f12d2f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:54.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.981+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e88007d30 con 0x7f4e780051c0 2024-04-14T08:43:54.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.981+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4e88030070 con 0x7f4e780051c0 2024-04-14T08:43:54.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.981+0000 7f4e83fff640 1 -- 172.21.15.172:0/174250579 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8802cd80 con 0x7f4e780051c0 2024-04-14T08:43:54.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.982+0000 7f4e93508640 1 -- 172.21.15.172:0/174250579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e780051c0 msgr2=0x7f4e7800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.982+0000 7f4e93508640 1 --2- 172.21.15.172:0/174250579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e780051c0 0x7f4e7800b9c0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800c210 tx=0x7f4e78005830 comp rx=0 tx=0).stop 2024-04-14T08:43:54.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.982+0000 7f4e93508640 1 -- 172.21.15.172:0/174250579 shutdown_connections 2024-04-14T08:43:54.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.982+0000 7f4e93508640 1 --2- 172.21.15.172:0/174250579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e780051c0 0x7f4e7800b9c0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.982+0000 7f4e93508640 1 -- 172.21.15.172:0/174250579 >> 172.21.15.172:0/174250579 conn(0x7f4e8c09d590 msgr2=0x7f4e8c0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:54.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.982+0000 7f4e93508640 1 -- 172.21.15.172:0/174250579 shutdown_connections 2024-04-14T08:43:54.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.982+0000 7f4e93508640 1 -- 172.21.15.172:0/174250579 wait complete. 2024-04-14T08:43:54.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.983+0000 7f4e93508640 1 Processor -- start 2024-04-14T08:43:54.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.983+0000 7f4e93508640 1 -- start start 2024-04-14T08:43:54.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e93508640 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e780051c0 0x7f4e8c148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e93508640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e8c0c8300 0x7f4e8c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e93508640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06d8a0 0x7f4e8c0b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e93508640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c0cfbc0 con 0x7f4e8c0c8300 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e93508640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c0d2350 con 0x7f4e8c06d8a0 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e93508640 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 -- 0x7f4e8c0d2c10 con 0x7f4e780051c0 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e90aa3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e8c0c8300 0x7f4e8c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e90aa3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e8c0c8300 0x7f4e8c06d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38144/0 (socket says 172.21.15.172:38144) 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e90aa3640 1 -- 172.21.15.172:0/2956559821 learned_addr learned my addr 172.21.15.172:0/2956559821 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e91aa5640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06d8a0 0x7f4e8c0b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e90aa3640 1 -- 172.21.15.172:0/2956559821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e780051c0 msgr2=0x7f4e8c148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e912a4640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e780051c0 0x7f4e8c148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e90aa3640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e780051c0 0x7f4e8c148650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e90aa3640 1 -- 172.21.15.172:0/2956559821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06d8a0 msgr2=0x7f4e8c0b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:54.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e90aa3640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06d8a0 0x7f4e8c0b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e90aa3640 1 -- 172.21.15.172:0/2956559821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c0cb700 con 0x7f4e8c0c8300 2024-04-14T08:43:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e91aa5640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06d8a0 0x7f4e8c0b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-14T08:43:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.984+0000 7f4e90aa3640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e8c0c8300 0x7f4e8c06d360 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00f620 tx=0x7f4e7c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.985+0000 7f4e81ffb640 1 -- 172.21.15.172:0/2956559821 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0052e0 con 0x7f4e8c0c8300 2024-04-14T08:43:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.985+0000 7f4e81ffb640 1 -- 172.21.15.172:0/2956559821 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4e7c018070 con 0x7f4e8c0c8300 2024-04-14T08:43:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.985+0000 7f4e81ffb640 1 -- 172.21.15.172:0/2956559821 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c003830 con 0x7f4e8c0c8300 2024-04-14T08:43:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.985+0000 7f4e93508640 1 -- 172.21.15.172:0/2956559821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8c0cb9e0 con 0x7f4e8c0c8300 2024-04-14T08:43:54.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.985+0000 7f4e93508640 1 -- 172.21.15.172:0/2956559821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8c0cbfa0 con 0x7f4e8c0c8300 2024-04-14T08:43:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.986+0000 7f4e93508640 1 -- 172.21.15.172:0/2956559821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e8c068520 con 0x7f4e8c0c8300 2024-04-14T08:43:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.986+0000 7f4e81ffb640 1 -- 172.21.15.172:0/2956559821 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e7c0039d0 con 0x7f4e8c0c8300 2024-04-14T08:43:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.986+0000 7f4e81ffb640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4e7006c8f0 0x7f4e7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-14T08:43:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.986+0000 7f4e81ffb640 1 -- 172.21.15.172:0/2956559821 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5507+0+0 (secure 0 0 0) 0x7f4e7c08b920 con 0x7f4e8c0c8300 2024-04-14T08:43:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.987+0000 7f4e912a4640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4e7006c8f0 0x7f4e7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-14T08:43:54.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.987+0000 7f4e912a4640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4e7006c8f0 0x7f4e7006edb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c15a0c0 tx=0x7f4e8400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-14T08:43:54.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:54.991+0000 7f4e81ffb640 1 -- 172.21.15.172:0/2956559821 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4e7c059d80 con 0x7f4e8c0c8300 2024-04-14T08:43:55.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.204+0000 7f4e93508640 1 -- 172.21.15.172:0/2956559821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e8c06dc80 con 0x7f4e8c0c8300 2024-04-14T08:43:55.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.205+0000 7f4e81ffb640 1 -- 172.21.15.172:0/2956559821 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+3883 (secure 0 0 0) 0x7f4e7c059680 con 0x7f4e8c0c8300 2024-04-14T08:43:55.206 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-14T08:43:55.206 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":102,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6836/3001787810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3001787810},{"type":"v1","addr":"172.21.15.172:6836","nonce":3001787810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/3013746004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3013746004},{"type":"v1","addr":"172.21.15.177:6834","nonce":3013746004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3881291102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3881291102},{"type":"v1","addr":"172.21.15.177:6835","nonce":3881291102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90},{"gid":9741,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2123910915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6835","nonce":2123910915},{"type":"v1","addr":"172.21.15.172:6837","nonce":2123910915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[]} 2024-04-14T08:43:55.206 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 102 2024-04-14T08:43:55.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.207+0000 7f4e93508640 1 -- 172.21.15.172:0/2956559821 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4e7006c8f0 msgr2=0x7f4e7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:55.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.207+0000 7f4e93508640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4e7006c8f0 0x7f4e7006edb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c15a0c0 tx=0x7f4e8400b040 comp rx=0 tx=0).stop 2024-04-14T08:43:55.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.207+0000 7f4e93508640 1 -- 172.21.15.172:0/2956559821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e8c0c8300 msgr2=0x7f4e8c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T08:43:55.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.207+0000 7f4e93508640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e8c0c8300 0x7f4e8c06d360 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00f620 tx=0x7f4e7c00fae0 comp rx=0 tx=0).stop 2024-04-14T08:43:55.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.207+0000 7f4e93508640 1 -- 172.21.15.172:0/2956559821 shutdown_connections 2024-04-14T08:43:55.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.207+0000 7f4e93508640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.172:6832/1188012648,v1:172.21.15.172:6833/1188012648] conn(0x7f4e7006c8f0 0x7f4e7006edb0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:55.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.207+0000 7f4e93508640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06d8a0 0x7f4e8c0b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:55.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.207+0000 7f4e93508640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e8c0c8300 0x7f4e8c06d360 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:55.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.207+0000 7f4e93508640 1 --2- 172.21.15.172:0/2956559821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e780051c0 0x7f4e8c148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-14T08:43:55.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.207+0000 7f4e93508640 1 -- 172.21.15.172:0/2956559821 >> 172.21.15.172:0/2956559821 conn(0x7f4e8c09d590 msgr2=0x7f4e8c0b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T08:43:55.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.207+0000 7f4e93508640 1 -- 172.21.15.172:0/2956559821 shutdown_connections 2024-04-14T08:43:55.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-14T08:43:55.207+0000 7f4e93508640 1 -- 172.21.15.172:0/2956559821 wait complete. 2024-04-14T08:43:55.219 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-14T08:43:55.219 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-14T08:43:55.219 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-14T08:43:55.219 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.228 INFO:tasks.ceph.mds.a:Stopped 2024-04-14T08:43:55.228 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-14T08:43:55.228 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.277 INFO:tasks.ceph.mds.c:Stopped 2024-04-14T08:43:55.277 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-14T08:43:55.277 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.290 INFO:tasks.ceph.mds.b:Stopped 2024-04-14T08:43:55.290 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-14T08:43:55.290 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.342 INFO:tasks.ceph.mds.d:Stopped 2024-04-14T08:43:55.342 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-14T08:43:55.342 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-14T08:43:55.343 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.377 INFO:tasks.ceph.osd.0:Stopped 2024-04-14T08:43:55.377 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-14T08:43:55.377 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.445 INFO:tasks.ceph.osd.1:Stopped 2024-04-14T08:43:55.445 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-14T08:43:55.445 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.516 INFO:tasks.ceph.osd.2:Stopped 2024-04-14T08:43:55.516 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-14T08:43:55.517 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.588 INFO:tasks.ceph.osd.3:Stopped 2024-04-14T08:43:55.588 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-14T08:43:55.588 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.624 INFO:tasks.ceph.osd.4:Stopped 2024-04-14T08:43:55.624 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-14T08:43:55.624 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.689 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:43:55.691 INFO:tasks.ceph.osd.5:Stopped 2024-04-14T08:43:55.692 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-14T08:43:55.692 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.692 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:43:55.766 INFO:tasks.ceph.osd.6:Stopped 2024-04-14T08:43:55.767 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-14T08:43:55.767 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.834 INFO:tasks.ceph.osd.7:Stopped 2024-04-14T08:43:55.835 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-14T08:43:55.835 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-14T08:43:55.835 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.861 INFO:tasks.ceph.mgr.y:Stopped 2024-04-14T08:43:55.861 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-14T08:43:55.861 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.900 INFO:tasks.ceph.mgr.x:Stopped 2024-04-14T08:43:55.901 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-14T08:43:55.901 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-14T08:43:55.901 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.913 INFO:tasks.ceph.mon.a:Stopped 2024-04-14T08:43:55.914 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-14T08:43:55.914 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:55.952 INFO:tasks.ceph.mon.b:Stopped 2024-04-14T08:43:55.953 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-14T08:43:55.953 INFO:teuthology.orchestra.run:waiting for 300 2024-04-14T08:43:56.004 INFO:tasks.ceph.mon.c:Stopped 2024-04-14T08:43:56.004 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-14T08:43:56.004 DEBUG:teuthology.orchestra.run.smithi172:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-14T08:43:56.037 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:37:21.994098+0000 mds.c (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-04-14T08:43:56.038 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-04-14T08:43:56.038 DEBUG:teuthology.orchestra.run.smithi172:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-14T08:43:56.113 DEBUG:teuthology.orchestra.run.smithi172:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-14T08:43:56.188 INFO:teuthology.orchestra.run.smithi172.stdout:2024-04-14T08:39:04.994014+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-14T08:43:56.188 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:43:56.188 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-14T08:43:56.399 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:43:56.400 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-14T08:43:56.490 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:43:56.490 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-14T08:43:56.548 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:43:56.548 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-14T08:43:56.606 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:43:56.606 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-14T08:43:56.770 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:43:56.771 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-14T08:43:56.843 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:43:56.843 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-14T08:43:56.901 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:43:56.902 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-14T08:43:56.950 INFO:tasks.ceph:Archiving mon data... 2024-04-14T08:43:56.952 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369/data/mon.a.tgz 2024-04-14T08:43:56.952 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-04-14T08:43:56.968 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.WMa5MUvxf1 2024-04-14T08:43:56.968 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.WMa5MUvxf1 2024-04-14T08:43:57.131 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.WMa5MUvxf1 2024-04-14T08:43:57.216 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.WMa5MUvxf1 is 407KB 2024-04-14T08:43:57.317 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.WMa5MUvxf1 2024-04-14T08:43:57.332 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369/data/mon.b.tgz 2024-04-14T08:43:57.332 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-04-14T08:43:57.349 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.r7lLDtOrbD 2024-04-14T08:43:57.349 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.r7lLDtOrbD 2024-04-14T08:43:57.502 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.r7lLDtOrbD 2024-04-14T08:43:57.588 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.r7lLDtOrbD is 412KB 2024-04-14T08:43:57.684 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.r7lLDtOrbD 2024-04-14T08:43:57.699 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369/data/mon.c.tgz 2024-04-14T08:43:57.699 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2024-04-14T08:43:57.754 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.J0RlMf9prG 2024-04-14T08:43:57.754 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.J0RlMf9prG 2024-04-14T08:43:57.894 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.J0RlMf9prG 2024-04-14T08:43:57.980 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.J0RlMf9prG is 412KB 2024-04-14T08:43:58.079 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.J0RlMf9prG 2024-04-14T08:43:58.098 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-14T08:43:58.099 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-14T08:43:58.101 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-14T08:43:58.185 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-14T08:43:58.217 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-14T08:43:58.217 INFO:tasks.ceph:Archiving crash dumps... 2024-04-14T08:43:58.219 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369/remote/smithi172/crash 2024-04-14T08:43:58.220 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-14T08:43:58.255 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369/remote/smithi177/crash 2024-04-14T08:43:58.256 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-14T08:43:58.283 INFO:tasks.ceph:Shutting down logrotate 2024-04-14T08:43:58.284 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:43:58.299 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-14T08:43:58.379 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-14T08:43:58.409 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-14T08:43:58.436 INFO:tasks.ceph:Compressing logs... 2024-04-14T08:43:58.436 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-14T08:43:58.451 DEBUG:teuthology.orchestra.run.smithi177:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-14T08:43:58.532 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-14T08:43:58.532 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43838.log 2024-04-14T08:43:58.532 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38010.log 2024-04-14T08:43:58.532 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.1.43838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37850.log 2024-04-14T08:43:58.532 INFO:teuthology.orchestra.run.smithi177.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43838.log.gz 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38010.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.37850.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38010.log.gz 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph.audit.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37850.log.gz 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37930.log 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37810.log 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.37930.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.37810.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37930.log.gz 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44238.log 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37810.log.gz 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose/var/log/ceph/ceph-osd.5.log: -- /var/log/ceph/ceph-mgr.x.log 2024-04-14T08:43:58.533 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.3.44238.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44238.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.3.44234.log 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37970.log 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.3.44234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44234.log.gz 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.37970.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37970.log.gz 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr: --verbose -- /var/log/ceph/ceph-client.1.43833.log 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37890.log 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.1.43833.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43833.log.gz 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38050.log 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.37890.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37890.log.gz 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38050.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38050.log.gz 2024-04-14T08:43:58.534 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38090.log 2024-04-14T08:43:58.535 INFO:teuthology.orchestra.run.smithi177.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-14T08:43:58.535 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mon.b.log: 94.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-14T08:43:58.535 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-14T08:43:58.535 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-client.admin.38090.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-14T08:43:58.535 INFO:teuthology.orchestra.run.smithi177.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38090.log.gz 2024-04-14T08:43:58.570 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-14T08:43:58.676 INFO:teuthology.orchestra.run.smithi177.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-14T08:43:58.755 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83060.log 2024-04-14T08:43:58.756 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69783.log 2024-04-14T08:43:58.756 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.83060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67416.log 2024-04-14T08:43:58.756 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56174.log 2024-04-14T08:43:58.756 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83060.log.gz 2024-04-14T08:43:58.756 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.67416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65305.log 2024-04-14T08:43:58.756 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69783.log.gz 2024-04-14T08:43:58.756 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67416.log.gz 2024-04-14T08:43:58.756 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57505.log 2024-04-14T08:43:58.756 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46132.log 2024-04-14T08:43:58.756 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57505.log: gzip -5 --verbose -- 91.2% /var/log/ceph/ceph-client.admin.44852.log 2024-04-14T08:43:58.756 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56174.log.gz 2024-04-14T08:43:58.756 INFO:teuthology.orchestra.run.smithi172.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65305.log.gz 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46132.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57505.log.gzgzip 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.75271.log 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.44852.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46132.log.gz 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74861.log 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44852.log.gz 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67612.log 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75271.log.gz 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82724.log 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.67612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46134.log 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74861.log.gz 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60718.log 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67612.log.gz 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46134.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82724.log.gz 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82113.log 2024-04-14T08:43:58.757 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60718.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46134.log.gz 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43765.log 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82113.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60718.log.gz 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82113.log.gz 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48210.log 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mon.a.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.43765.log.gz 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59149.log 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81698.log 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48210.log.gz 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60298.log 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75444.log 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59149.log.gz 2024-04-14T08:43:58.758 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60298.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81698.log.gz 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73251.log 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49188.log 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60298.log.gz 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75444.log: /var/log/ceph/ceph-client.admin.73251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49020.log 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75444.log.gz 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49188.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73251.log.gz 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78588.log 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr: 89.4%/var/log/ceph/ceph-client.admin.49020.log: -- replaced with /var/log/ceph/ceph-client.admin.49188.log.gz 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75823.log 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49020.log.gz 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.78588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45618.log 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75823.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78588.log.gz 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68276.log 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45618.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.75823.log.gz 2024-04-14T08:43:58.759 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57192.log 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45618.log.gz 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69415.log 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57192.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68276.log.gz 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58564.log 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69415.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57192.log.gz 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51934.log 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69415.log.gz 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65348.log 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82850.log 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51934.log.gz 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53355.log 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58564.log.gz 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75294.log 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65348.log.gz 2024-04-14T08:43:58.760 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82850.log: /var/log/ceph/ceph-client.admin.53355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37520.log 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82850.log.gz 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75294.log: gzip -5 89.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.53355.log.gz -- 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr: /var/log/ceph/ceph-client.admin.46836.log 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.37520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80596.log 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.75294.log.gz 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37520.log.gz 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47534.log 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46836.log.gz 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.80596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68782.log 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.47534.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.80596.log.gz -5 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51141.log 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47534.log.gz 2024-04-14T08:43:58.761 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66846.log 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51141.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68782.log.gz 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62018.log 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51141.log.gz 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72302.log 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.62018.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66846.log.gz 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76626.log 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72302.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62018.log.gz 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45793.log 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72302.log.gz 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76924.log 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45793.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76626.log.gz 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46878.log 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48558.log 88.9% 2024-04-14T08:43:58.762 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45793.log.gz 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76924.log.gz 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62102.log 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48558.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46878.log.gz 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66422.log 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6%/var/log/ceph/ceph-client.admin.62102.log: -- replaced with /var/log/ceph/ceph-client.admin.48558.log.gz 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82472.log 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81352.log 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82472.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66422.log.gz 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62102.log.gz 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67140.log 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82472.log.gz 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77092.log 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81352.log.gz 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.67140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60256.log 2024-04-14T08:43:58.763 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71665.log 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77092.log.gz 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50181.log 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67140.log.gz 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51838.log 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60256.log.gz 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71665.log: /var/log/ceph/ceph-client.admin.50181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69535.log 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71665.log.gz 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51838.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50181.log.gz 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70778.log 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69535.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51838.log.gz 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83102.log 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.70778.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.69535.log.gz 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66636.log 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.83102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50763.log 89.0% 2024-04-14T08:43:58.764 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70778.log.gz 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66636.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.83102.log.gz --verbose 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-client.admin.61371.log 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66636.log.gz 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56838.log 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50511.log 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50763.log.gz 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61371.log: /var/log/ceph/ceph-client.admin.56838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55277.log 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61371.log.gz 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50511.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56838.log.gz 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67959.log 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55277.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50511.log.gz 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57929.log 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55277.log.gz 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.67959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53790.log 2024-04-14T08:43:58.765 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73767.log 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53790.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57929.log.gz 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74272.log 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.67959.log.gz 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53790.log.gz 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.73767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73767.log.gz 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55529.log 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51573.log 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74272.log.gz 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55235.log 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63587.log 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55235.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51573.log.gz 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53238.log 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.55529.log.gz 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55235.log.gz 2024-04-14T08:43:58.766 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.63587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51015.log 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.63587.log.gz 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63808.log 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53238.log.gz 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45366.log 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51015.log.gz 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.63808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49797.log 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56576.log 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49797.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45366.log.gz 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66762.log 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.63808.log.gz 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49797.log.gz 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67922.log 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66762.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56576.log.gz 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57421.log 2024-04-14T08:43:58.767 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7%/var/log/ceph/ceph-client.admin.67922.log: -- replaced with /var/log/ceph/ceph-client.admin.66762.log.gz 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83228.log 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57421.log: gzip -5 --verbose -- 89.4% /var/log/ceph/ceph-client.admin.58186.log 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67922.log.gz 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.83228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71334.log 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57421.log.gz 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58186.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83228.log.gz 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53176.log 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58186.log.gz 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74710.log 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53176.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71334.log.gz 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76201.log 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7%/var/log/ceph/ceph-client.admin.74710.log: -- replaced with /var/log/ceph/ceph-client.admin.53176.log.gz 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49713.log 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76201.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.74710.log.gz 2024-04-14T08:43:58.768 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50679.log 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49713.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76201.log.gz 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66972.log 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50679.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49713.log.gz 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81572.log 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50679.log.gz 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54630.log 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81572.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66972.log.gz 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65558.log 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81572.log.gz 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69191.log 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54630.log.gz 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77386.log 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69191.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65558.log.gz 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50013.log 2024-04-14T08:43:58.769 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52148.log 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69191.log.gz 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65871.log 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50013.log.gz 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65660.log 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.77386.log.gz 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.52148.log: /var/log/ceph/ceph-client.admin.65871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50721.log 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52148.log.gz 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65871.log.gz 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65703.log 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50721.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65660.log.gz 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72008.log 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65703.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50721.log.gz 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76033.log 2024-04-14T08:43:58.770 INFO:teuthology.orchestra.run.smithi172.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65703.log.gz 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61976.log 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55487.log 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72008.log.gz 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61976.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76033.log.gz 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80942.log 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0%/var/log/ceph/ceph-client.admin.55487.log: -- replaced with /var/log/ceph/ceph-client.admin.61976.log.gz 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76966.log 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.80942.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55487.log.gz 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64499.log 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.80942.log.gz 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54937.log 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76966.log.gz 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.64499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73209.log 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77050.log 88.9% 2024-04-14T08:43:58.771 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64499.log.gz 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54937.log.gz 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.73209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66253.log 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77050.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.73209.log.gz -- 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr: /var/log/ceph/ceph-client.admin.45798.log 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59341.log 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77050.log.gz 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66253.log.gz 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73575.log 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59341.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45798.log.gz 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75655.log 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.73575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68168.log 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75655.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73575.log.gz 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59341.log.gz 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68932.log 2024-04-14T08:43:58.772 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81866.log 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68168.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75655.log.gz 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65913.log 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68168.log.gz 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64729.log 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65913.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81866.log.gz 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68932.log.gz 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52274.log 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65913.log.gz 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.64729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71124.log 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64729.log.gz 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.52274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51742.log 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71124.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52274.log.gz 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71581.log 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51742.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71124.log.gz 2024-04-14T08:43:58.773 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80684.log 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51742.log.gz 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74069.log 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.80684.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71581.log.gz 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45787.log 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82808.log 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74069.log.gz 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45787.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.80684.log.gz 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46584.log 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45787.log.gz 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54163.log 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82808.log.gz 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53014.log 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46584.log.gz 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60760.log 2024-04-14T08:43:58.774 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53014.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54163.log.gz 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51657.log 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53014.log.gz/var/log/ceph/ceph-client.admin.60760.log: 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71376.log 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60760.log.gz 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68629.log 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69925.log 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51657.log.gz 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71376.log.gz 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53098.log 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72598.log 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53098.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68629.log.gz 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69925.log.gz 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46131.log 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53098.log.gz 2024-04-14T08:43:58.775 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62360.log 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72598.log.gz 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52106.log 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.62360.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46131.log.gz 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52972.log 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62360.log.gz 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.52106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61041.log 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52106.log.gz 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.52972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77344.log 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52972.log.gz 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56618.log 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56796.log 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61041.log.gz 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56618.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77344.log.gz 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63874.log 2024-04-14T08:43:58.776 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56618.log.gz 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61724.log 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.63874.log: gzip 89.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.56796.log.gz --verbose 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-client.admin.70467.log 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77176.log 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61724.log.gz 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.63874.log.gz 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.70467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59299.log 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81656.log 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77176.log: /var/log/ceph/ceph-client.admin.59299.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77176.log.gz 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75034.log 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.70467.log.gz 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81740.log 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0%/var/log/ceph/ceph-client.admin.81656.log: -- replaced with /var/log/ceph/ceph-client.admin.59299.log.gz 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65829.log 2024-04-14T08:43:58.777 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81656.log.gz 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.75034.log.gz 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54886.log 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65829.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81740.log.gz 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43017.log 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65829.log.gz 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43681.log 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54886.log.gz 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61121.log 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61934.log 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43017.log.gz 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61121.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43681.log.gz 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60550.log 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61121.log.gz 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55067.log 2024-04-14T08:43:58.778 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61934.log: /var/log/ceph/ceph-client.admin.60550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48516.log 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61934.log.gz 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55067.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60550.log.gz 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61329.log 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55067.log.gz 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54714.log 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43553.log 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48516.log.gz 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54714.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61329.log.gz 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48726.log 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54714.log.gz 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62580.log 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54373.log 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43553.log.gz 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.62580.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48726.log.gz 2024-04-14T08:43:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76882.log 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62580.log.gz 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81310.log 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56660.log 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54373.log.gz 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81310.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76882.log.gz 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83018.log 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81310.log.gz 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58951.log 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.83018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52190.log 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56660.log.gz 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58951.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83018.log.gz 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71928.log 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58951.log.gz 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.52190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49146.log 2024-04-14T08:43:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71928.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52190.log.gz 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57547.log 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49146.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71928.log.gz 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53056.log 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57547.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49146.log.gz 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82640.log 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53056.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57547.log.gz -5 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr: --verbose -- /var/log/ceph/ceph-client.0.68464.log 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53056.log.gz 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73763.log 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.68464.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.82640.log.gz 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71539.log 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68464.log.gz 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.73763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78201.log 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73763.log.gz 2024-04-14T08:43:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75613.log 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.78201.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71539.log.gzgzip 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.82556.log 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78201.log.gz 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61552.log 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75613.log.gz 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70221.log 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61552.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82556.log.gz 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.61552.log.gz 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.70221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54079.log 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70221.log.gz 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76159.log 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54079.log: gzip 87.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz -- 2024-04-14T08:43:58.782 INFO:teuthology.orchestra.run.smithi172.stderr: /var/log/ceph/ceph-client.admin.46668.log 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76159.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54079.log.gz 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45198.log 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76159.log.gz 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37787.log 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45198.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46668.log.gz 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71833.log 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45198.log.gz 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.37787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65787.log 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37787.log.gz 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70820.log 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65787.log: gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.71833.log.gz --verbose 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-client.admin.69331.log 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65787.log.gz 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.70820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80144.log 2024-04-14T08:43:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69331.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70820.log.gz 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75121.log 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69331.log.gz 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.80144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48168.log 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64918.log 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48168.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.80144.log.gz 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72682.log 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.75121.log.gz 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48168.log.gz 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.64918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46374.log 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64918.log.gz 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64066.log 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72682.log.gz 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78866.log 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.64066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76117.log 2024-04-14T08:43:58.784 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46374.log.gz 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.78866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43236.log 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78866.log.gz 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76117.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64066.log.gz 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81908.log 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76117.log.gz 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65263.log 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81908.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43236.log.gz 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55021.log 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81908.log.gz 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43807.log 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77302.log 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65263.log.gz 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55021.log.gz 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46920.log 2024-04-14T08:43:58.785 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43807.log.gz 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59952.log 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50427.log 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77302.log.gz 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46920.log.gz 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45576.log 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59952.log.gz 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50931.log 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45576.log: gzip 88.3% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.50427.log.gz -- 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr: /var/log/ceph/ceph-client.admin.66295.log 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45576.log.gz 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48474.log 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50931.log.gz 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60820.log 2024-04-14T08:43:58.786 INFO:teuthology.orchestra.run.smithi172.stderr: 88.9%/var/log/ceph/ceph-client.admin.48474.log: -- replaced with /var/log/ceph/ceph-client.admin.66295.log.gz 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44451.log 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60820.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48474.log.gz 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65002.log 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60820.log.gz 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.44451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43637.log 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53934.log 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65002.log.gz 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80210.log 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67899.log 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53934.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43637.log.gz 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.80210.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53934.log.gz 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80088.log 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.44451.log.gz 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51099.log 2024-04-14T08:43:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.67899.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80210.log.gz 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.80088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69597.log 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67899.log.gz 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.80088.log.gz 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51099.log: -- /var/log/ceph/ceph-client.admin.63713.log 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58396.log 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51099.log.gz 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.63713.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69597.log.gz 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46126.log 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63713.log.gz 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62996.log 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76840.log 88.6% 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58396.log.gz 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46126.log.gz 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.62996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71166.log 2024-04-14T08:43:58.788 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76840.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62996.log.gz 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64174.log 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71166.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76840.log.gz 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66678.log 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.64174.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71166.log.gz 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82892.log 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66678.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82892.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66678.log.gz 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78393.log 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64174.log.gz 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82892.log.gz 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48432.log 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.78393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76075.log 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78393.log.gz 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49593.log 2024-04-14T08:43:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76075.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48432.log.gz 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46710.log 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76075.log.gz 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64433.log 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55787.log 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49593.log.gz 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.64433.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46710.log.gz 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54548.log 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.55787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80500.log 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55787.log.gz 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54205.log 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64433.log.gz 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54548.log: /var/log/ceph/ceph-client.admin.80500.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54548.log.gz 2024-04-14T08:43:58.790 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56922.log 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80500.log.gz 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44952.log 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55445.log 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54205.log.gz 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56922.log.gz 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.44952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62622.log 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55445.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44952.log.gz 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50385.log 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55445.log.gz 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.62622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50097.log 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50385.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62622.log.gz 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59599.log 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50097.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50385.log.gz 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57252.log 2024-04-14T08:43:58.791 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50097.log.gz 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43933.log 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70370.log 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57252.log.gz 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59599.log.gz 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74111.log 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76798.log 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9%/var/log/ceph/ceph-client.admin.70370.log: -- replaced with /var/log/ceph/ceph-client.admin.43933.log.gz 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45036.log 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76798.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74111.log.gz 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:gzip 91.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.70370.log.gz 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62538.log 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76798.log.gz 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65390.log 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.62538.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45036.log.gz 2024-04-14T08:43:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44642.log 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62538.log.gz 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44726.log 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.44642.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65390.log.gz 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61244.log 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44642.log.gz 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.44726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74365.log 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5%/var/log/ceph/ceph-client.admin.61244.log: -- replaced with /var/log/ceph/ceph-client.admin.44726.log.gz 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69680.log 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74365.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61244.log.gz 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53976.log 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70575.log 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53976.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.74365.log.gz 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76500.log 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53976.log.gz 2024-04-14T08:43:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61808.log 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.70575.log: /var/log/ceph/ceph-client.admin.76500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47004.log 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76500.log.gz 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43405.log 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.70575.log.gz 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59449.log 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr: 87.5%/var/log/ceph/ceph-client.admin.47004.log: -- replaced with /var/log/ceph/ceph-client.admin.61808.log.gz 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43405.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.47004.log.gz 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60130.log 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.69680.log.gz 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43405.log.gz 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81614.log 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68563.log 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60130.log.gz 2024-04-14T08:43:58.794 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79096.log 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59449.log.gz 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81614.log.gz 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80341.log 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.79096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55109.log 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.80341.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79096.log.gz 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45534.log 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68563.log.gz 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79250.log 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55109.log: /var/log/ceph/ceph-client.admin.45534.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55109.log.gz 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74537.log 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45534.log.gz 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57684.log 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.79250.log: /var/log/ceph/ceph-client.admin.74537.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.80341.log.gz 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68468.log 2024-04-14T08:43:58.795 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79250.log.gz 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49839.log 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.68468.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57684.log.gz 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68468.log.gz 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- 91.0% /var/log/ceph/ceph-client.admin.46458.log 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74537.log.gz 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63380.log 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49839.log: /var/log/ceph/ceph-client.admin.46458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71418.log 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49839.log.gz 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.63380.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46458.log.gz 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63380.log.gz 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49635.log 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56086.log 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71292.log 2024-04-14T08:43:58.796 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71418.log.gz 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49635.log.gz 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52358.log 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60214.log 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.52358.log: 91.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.56086.log.gz -5 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56324.log 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71292.log.gz 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59665.log 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52358.log.gz 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60214.log.gz 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58270.log 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59665.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56324.log.gz 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72258.log 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59665.log.gz 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71749.log 2024-04-14T08:43:58.797 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58270.log.gz 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45408.log 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71749.log: gzip -5 --verbose 89.7% -- /var/log/ceph/ceph-client.admin.45324.log -- replaced with /var/log/ceph/ceph-client.admin.72258.log.gz 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71749.log.gz 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62783.log 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45324.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45408.log.gz 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48894.log 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr: 89.3%/var/log/ceph/ceph-client.admin.62783.log: -- replaced with /var/log/ceph/ceph-client.admin.45324.log.gz 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45795.log 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51699.log 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62783.log.gz 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45795.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48894.log.gz 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65432.log 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45795.log.gz 2024-04-14T08:43:58.798 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59910.log 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65432.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51699.log.gz 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56258.log 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59910.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65432.log.gz 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48768.log 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59910.log.gz 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46752.log 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63545.log 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46752.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48768.log.gz 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69967.log 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.56258.log.gz 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46752.log.gz 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.63545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71478.log 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63545.log.gz 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76712.log 2024-04-14T08:43:58.799 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71707.log 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76712.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71478.log.gz 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59868.log 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69967.log.gz 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71707.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76712.log.gz 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60862.log 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71707.log.gz 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75991.log 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60862.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59868.log.gzgzip 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.75033.log 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75991.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60862.log.gz 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57463.log 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75033.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75991.log.gz 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70998.log 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68824.log 2024-04-14T08:43:58.800 INFO:teuthology.orchestra.run.smithi172.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.75033.log.gz 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.70998.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57463.log.gz 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58974.log 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70998.log.gz 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53538.log 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68122.log 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53538.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.58974.log.gz 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66720.log 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68824.log.gz 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53538.log.gz 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79814.log 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64024.log 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68122.log.gz 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.79814.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66720.log.gz 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45799.log 2024-04-14T08:43:58.801 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.64024.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79814.log.gz 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77640.log 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45799.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64024.log.gz 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55403.log 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.77640.log: gzip -5 88.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.45799.log.gz -- 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr: /var/log/ceph/ceph-client.admin.68998.log 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77640.log.gz 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46416.log 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68998.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55403.log.gz 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65179.log 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61286.log 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.68998.log.gz 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46416.log.gz 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77974.log 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74153.log 2024-04-14T08:43:58.802 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61286.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65179.log.gz 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59083.log 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61286.log.gz 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74153.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77974.log.gz 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69457.log 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59083.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74153.log.gz 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45282.log 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51281.log 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69457.log.gz 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45282.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59083.log.gz 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45282.log.gz 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47667.log 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.1.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.51281.log.gz 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65221.log 2024-04-14T08:43:58.803 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.47667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77452.log 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65221.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47667.log.gz 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69879.log 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79977.log 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77452.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65221.log.gz 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56492.log 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77452.log.gz 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69879.log.gz 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79054.log 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.79977.log: /var/log/ceph/ceph-client.admin.56492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61415.log 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.79977.log.gz 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.79054.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56492.log.gz 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79054.log.gz 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79184.log 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59407.log 2024-04-14T08:43:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.79184.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61415.log.gz 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55783.log 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79688.log 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.79184.log.gz 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4%/var/log/ceph/ceph-client.0.55783.log: -- replaced with /var/log/ceph/ceph-client.admin.59407.log.gz 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53892.log 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55783.log.gz 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53318.log 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.79688.log: /var/log/ceph/ceph-client.admin.53892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69083.log 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79688.log.gz 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53318.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53892.log.gz 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70033.log 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60508.log 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69083.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53318.log.gz 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.70033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72724.log 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70033.log.gz 2024-04-14T08:43:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66172.log 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69083.log.gz 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62706.log 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72724.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60508.log.gz 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60904.log 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72724.log.gz 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66172.log.gz 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.62706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44041.log 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62706.log.gzgzip 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58102.log 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60904.log: /var/log/ceph/ceph-client.admin.44041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67412.log 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60904.log.gz 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58102.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44041.log.gz 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63038.log 2024-04-14T08:43:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62845.log 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.67412.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58102.log.gz 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67412.log.gz 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.63038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49551.log 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63038.log.gz 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49314.log 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.62845.log: /var/log/ceph/ceph-client.admin.49551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74819.log 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62845.log.gz 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49314.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49551.log.gz 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61202.log 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49314.log.gz 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77008.log 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45078.log 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61202.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74819.log.gz 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79772.log 2024-04-14T08:43:58.807 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61202.log.gz 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77008.log.gz 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51445.log 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45078.log: /var/log/ceph/ceph-client.admin.79772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76542.log 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45078.log.gz 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51445.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79772.log.gz 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64391.log 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51445.log.gz 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51785.log 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr:gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76542.log.gz 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64960.log 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.64391.log: /var/log/ceph/ceph-client.admin.51785.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64391.log.gz 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58480.log 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51785.log.gz 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54979.log 2024-04-14T08:43:58.808 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.64960.log: /var/log/ceph/ceph-client.admin.58480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66804.log 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64960.log.gz 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58480.log.gz 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50553.log 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73293.log 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66804.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54979.log.gz 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50553.log: gzip 87.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.66804.log.gz 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-client.admin.58058.log 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50553.log.gz 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66092.log 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.73293.log: /var/log/ceph/ceph-client.admin.58058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44810.log 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73293.log.gz 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66092.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58058.log.gz 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37876.log 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.44810.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66092.log.gz 2024-04-14T08:43:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78945.log 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44810.log.gz 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67224.log 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.37876.log: /var/log/ceph/ceph-client.admin.78945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61640.log 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37876.log.gz 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.67224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66594.log 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.78945.log.gz 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67224.log.gz 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82598.log 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66594.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61640.log.gz 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78016.log 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66594.log.gzgzip 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.78058.log 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82598.log: /var/log/ceph/ceph-client.admin.78016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47529.log 2024-04-14T08:43:58.810 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82598.log.gz 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78016.log.gz 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.78058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46794.log 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.47529.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.78058.log.gz 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60424.log 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47529.log.gz 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80014.log 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46794.log.gz 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48298.log 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53748.log 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60424.log.gz 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.80014.log: /var/log/ceph/ceph-client.admin.48298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66930.log 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.80014.log.gz 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48347.log 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48298.log.gz 2024-04-14T08:43:58.811 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66930.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53748.log.gz 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78630.log 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67857.log 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48347.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66930.log.gz 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.78630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45450.log 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48347.log.gz 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54036.log 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7%/var/log/ceph/ceph-client.admin.67857.log: -- replaced with /var/log/ceph/ceph-client.admin.78630.log.gz 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45450.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67857.log.gz 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79934.log 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52616.log 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45450.log.gz 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54036.log: /var/log/ceph/ceph-client.admin.79934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77218.log 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54036.log.gz 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.52616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65086.log 2024-04-14T08:43:58.812 INFO:teuthology.orchestra.run.smithi172.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79934.log.gz 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52616.log.gz 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57379.log 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65086.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77218.log.gz 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58522.log 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65086.log.gz 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81488.log 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48253.log 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58522.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57379.log.gz 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47210.log 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58522.log.gz 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr: 89.7%/var/log/ceph/ceph-client.admin.48253.log: -- replaced with /var/log/ceph/ceph-client.admin.81488.log.gz 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46962.log 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.47210.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48253.log.gz 2024-04-14T08:43:58.813 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57845.log 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77890.log 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46962.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47210.log.gz 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57845.log: gzip -5 --verbose 87.8% -- -- replaced with /var/log/ceph/ceph-client.admin.46962.log.gz /var/log/ceph/ceph-client.admin.57132.log 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49755.log 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57845.log.gz 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77890.log: /var/log/ceph/ceph-client.admin.57132.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77890.log.gz 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47989.log 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43849.log 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57132.log.gz 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49755.log: /var/log/ceph/ceph-client.2.47989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45240.log 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49755.log.gz 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47989.log.gz 2024-04-14T08:43:58.814 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48642.log 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43849.log.gz 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71208.log 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59773.log 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45240.log.gz/var/log/ceph/ceph-client.admin.48642.log: 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64809.log 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48642.log.gz 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60946.log 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59773.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.71208.log.gz 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.64809.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59773.log.gz 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71040.log 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80938.log 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60946.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.64809.log.gz 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72472.log 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60946.log.gz 2024-04-14T08:43:58.815 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71040.log.gz 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.0.80938.log: /var/log/ceph/ceph-client.admin.74668.log 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.80938.log.gz 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51489.log 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57972.log 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74668.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72472.log.gz 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52232.log 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.74668.log.gz 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57972.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51489.log.gzgzip 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64132.log 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57972.log.gz 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.52232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68890.log 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80233.log 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.64132.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52232.log.gz 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51403.log 2024-04-14T08:43:58.816 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64132.log.gz 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55151.log 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4%/var/log/ceph/ceph-client.admin.80233.log: -- replaced with /var/log/ceph/ceph-client.admin.68890.log.gz 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78135.log 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55151.log: gzip -5 --verbose 89.5% -- -- replaced with /var/log/ceph/ceph-client.admin.51403.log.gz /var/log/ceph/ceph-client.admin.68210.log 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.80233.log.gz 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.78135.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55151.log.gz 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69373.log 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57589.log 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68210.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.78135.log.gz 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69373.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49230.log 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57589.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69373.log.gz 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68210.log.gz 2024-04-14T08:43:58.817 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67014.log 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57589.log.gz 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75907.log 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.67014.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49230.log.gz 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70736.log 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67014.log.gz 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61682.log 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54798.log 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.70736.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75907.log.gz 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61682.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.70736.log.gz /var/log/ceph/ceph-client.admin.74949.log 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61682.log.gz 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60676.log 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49104.log 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74949.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54798.log.gz 2024-04-14T08:43:58.818 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45492.log 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50973.log 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.60676.log.gz 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74949.log.gz 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60466.log 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49104.log.gz 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70862.log 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45492.log.gz 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50973.log: /var/log/ceph/ceph-client.admin.60466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43595.log 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50973.log.gz 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.70862.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60466.log.gz 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47993.log 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70862.log.gz 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49923.log 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59994.log 2024-04-14T08:43:58.819 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.2.47993.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43595.log.gz 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47993.log.gz 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46125.log 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49923.log.gz 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50469.log 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46125.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59994.log.gz 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43113.log 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46125.log.gz 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57295.log 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72514.log 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43113.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50469.log.gz 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46500.log 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43113.log.gz 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72514.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.57295.log.gz --verbose 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-client.admin.83186.log 2024-04-14T08:43:58.820 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72514.log.gz 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78756.log 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72430.log 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46500.log.gz 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.83186.log: /var/log/ceph/ceph-client.admin.78756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75739.log 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83186.log.gz 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57048.log 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78756.log.gz 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72430.log.gz 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74766.log 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75739.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.57048.log: /var/log/ceph/ceph-client.admin.56012.log 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7%/var/log/ceph/ceph-client.admin.74766.log: -- replaced with /var/log/ceph/ceph-client.admin.75739.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68080.log 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57048.log.gz 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43447.log 2024-04-14T08:43:58.821 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74766.log.gz 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68080.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.56012.log.gz 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82388.log 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48936.log 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68080.log.gz 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43447.log: /var/log/ceph/ceph-client.admin.82388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45797.log 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43447.log.gz 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82388.log.gz 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49425.log 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66552.log 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45797.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48936.log.gz 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53706.log 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45797.log.gz 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49425.log.gz 2024-04-14T08:43:58.822 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67738.log 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78389.log 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53706.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66552.log.gz 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.67738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54289.log 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53706.log.gz 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.78389.log: 87.6%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.67738.log.gz --verbose 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-client.admin.65745.log 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78389.log.gz 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63958.log/var/log/ceph/ceph-client.admin.54289.log: 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61459.log 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54289.log.gz 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65745.log.gz 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr:gzip/var/log/ceph/ceph-client.admin.63958.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.82976.log 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54331.log 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61459.log.gz 2024-04-14T08:43:58.823 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51615.log 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr: 91.3%/var/log/ceph/ceph-client.admin.82976.log: -- replaced with /var/log/ceph/ceph-client.admin.63958.log.gz 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76756.log 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82976.log.gz 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54331.log.gz 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79856.log 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74366.log 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76756.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51615.log.gz 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.79856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61850.log 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76756.log.gz 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79856.log.gz 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64541.log 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73062.log 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61850.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74366.log.gz 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.64541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79730.log 2024-04-14T08:43:58.824 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61850.log.gz 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.73062.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.64541.log.gz -5 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59233.log 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73062.log.gz 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70304.log 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.79730.log: /var/log/ceph/ceph-client.admin.59233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72090.log 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79730.log.gz 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.70304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56754.log 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72090.log: gzip -5 --verbose 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59233.log.gz 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-client.admin.74907.log 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56754.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72090.log.gz 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.70304.log.gz 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80554.log 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56754.log.gz 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74453.log 2024-04-14T08:43:58.825 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74907.log.gz 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.80554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58438.log 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80554.log.gz 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53429.log 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82682.log 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58438.log.gz 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74453.log.gz 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48389.log 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51361.log 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.53429.log.gz 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48389.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82682.log.gz 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51057.log 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48389.log.gz 2024-04-14T08:43:58.826 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58354.log 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80750.log 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51361.log.gz/var/log/ceph/ceph-client.admin.51057.log: 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76668.log 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51057.log.gz 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58354.log.gz 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.80750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65516.log 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76668.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80750.log.gz 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74027.log 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76668.log.gz 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51883.log 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50301.log 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65516.log.gz 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51883.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74027.log.gz 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54842.log 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51883.log.gz 2024-04-14T08:43:58.827 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67056.log 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50637.log 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54842.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50301.log.gz 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.67056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43194.log 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54842.log.gz 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67056.log.gz 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43891.log 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82934.log 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50637.log.gz 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43194.log: /var/log/ceph/ceph-client.admin.43891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69837.log 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43194.log.gz 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43891.log.gz 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82934.log: /var/log/ceph/ceph-client.admin.69837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54672.log 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82934.log.gz 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.69837.log.gz 2024-04-14T08:43:58.828 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81226.log 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:gzip/var/log/ceph/ceph-client.admin.54672.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55319.log 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81226.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54672.log.gz 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81226.log.gz 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-client.admin.80299.log 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65474.log 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55319.log.gz 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.80299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56964.log 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60634.log 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56964.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65474.log.gz 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83268.log 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.80299.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56964.log.gz 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52316.log 2024-04-14T08:43:58.829 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60634.log: /var/log/ceph/ceph-client.admin.83268.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60634.log.gz 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-client.admin.66337.log 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83268.log.gz 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.52316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48600.log 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52316.log.gz 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73377.log 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66337.log.gz 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72766.log 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48600.log: /var/log/ceph/ceph-client.admin.73377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81968.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48600.log.gz 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73377.log.gz 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75697.log 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72766.log.gz 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54121.log 2024-04-14T08:43:58.830 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81968.log.gz 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49509.log 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75697.log.gz 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73335.log 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54121.log.gz 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48978.log 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49509.log.gz 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:gzip/var/log/ceph/ceph-client.admin.73335.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.72346.log 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48978.log: gzip -5 --verbose 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73335.log.gz 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-client.admin.66510.log 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48978.log.gz 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49062.log 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72346.log.gz 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74626.log 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49062.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66510.log.gz 2024-04-14T08:43:58.831 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50139.log 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49062.log.gz 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62664.log 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50139.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.74626.log.gz 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58228.log 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50139.log.gz 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.62664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63084.log 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58228.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62664.log.gz 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54756.log 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58228.log.gz 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.63084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68716.log 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5/var/log/ceph/ceph-client.admin.54756.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63084.log.gz 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr: --verbose -- /var/log/ceph/ceph-client.admin.67098.log 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.68716.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54756.log.gz 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74195.log 2024-04-14T08:43:58.832 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.67098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53281.log 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68716.log.gz 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67098.log.gz 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48810.log 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74195.log.gz 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71623.log 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53281.log.gz 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58630.log 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48810.log.gz 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81268.log 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71623.log.gz 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46136.log 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58630.log.gz 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr:gzip/var/log/ceph/ceph-client.admin.81268.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.72808.log 2024-04-14T08:43:58.833 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72874.log 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81268.log.gz 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46136.log.gz 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50847.log 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61594.log 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr: 91.1%/var/log/ceph/ceph-client.admin.50847.log: -- replaced with /var/log/ceph/ceph-client.admin.72808.log.gz 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72874.log.gz 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62942.log 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50847.log.gz 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73066.log 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.62942.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61594.log.gz 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77134.log 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62942.log.gz 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr:gzip/var/log/ceph/ceph-client.0.73066.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.0.73066.log.gz 2024-04-14T08:43:58.834 INFO:teuthology.orchestra.run.smithi172.stderr: /var/log/ceph/ceph-client.admin.46332.log 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77134.log: gzip -5 --verbose 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77134.log.gz 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-client.admin.59027.log 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53832.log 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46332.log.gz 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58312.log 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59027.log.gz 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82011.log 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.58312.log: -- 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53832.log.gz 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr: /var/log/ceph/ceph-client.admin.65955.log 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82011.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58312.log.gz 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44994.log 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82011.log.gz 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64240.log 2024-04-14T08:43:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:gzip/var/log/ceph/ceph-client.admin.44994.log: -5 --verbose 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65955.log.gz 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-osd.0.log 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.64240.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44994.log.gz 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79001.log 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.0.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64240.log.gz 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60382.log 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62868.log/var/log/ceph/ceph-client.admin.79001.log: 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60382.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77260.log 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79001.log.gz 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60382.log.gz 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.62868.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.62868.log.gz 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58818.log 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75486.log 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr: 87.5%/var/log/ceph/ceph-client.0.58818.log: -- replaced with /var/log/ceph/ceph-client.admin.77260.log.gz 2024-04-14T08:43:58.836 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58818.log.gz 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60592.log 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75486.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.57337.log 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75486.log.gz 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51976.log 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3%/var/log/ceph/ceph-client.admin.57337.log: -- replaced with /var/log/ceph/ceph-client.admin.60592.log.gz 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53622.log 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57337.log.gz 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75865.log 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51976.log: /var/log/ceph/ceph-client.admin.53622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61766.log 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75865.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51976.log.gz 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53622.log.gz 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75865.log.gz 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82766.log 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5/var/log/ceph/ceph-client.admin.61766.log: --verbose -- /var/log/ceph/ceph-client.admin.50223.log 2024-04-14T08:43:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82766.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61766.log.gz 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52018.log 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82766.log.gz 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58822.log 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50223.log: /var/log/ceph/ceph-client.admin.52018.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50223.log.gz 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52018.log.gz 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45120.log 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.58822.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.0.58822.log.gz 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr: --verbose -- /var/log/ceph/ceph-client.admin.75949.log 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76418.log 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45120.log.gz 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71082.log 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1%/var/log/ceph/ceph-client.admin.76418.log: -- replaced with /var/log/ceph/ceph-client.admin.75949.log.gz 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69638.log 2024-04-14T08:43:58.838 INFO:teuthology.orchestra.run.smithi172.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76418.log.gz 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75529.log 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82155.log 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71082.log.gz 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69638.log: /var/log/ceph/ceph-client.admin.75529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70225.log 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75529.log.gz/var/log/ceph/ceph-client.admin.82155.log: 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72556.log 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82155.log.gz 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.70225.log: 92.3% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69638.log.gz 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45800.log 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.0.70225.log.gz 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5/var/log/ceph/ceph-client.admin.72556.log: --verbose -- /var/log/ceph/ceph-client.admin.50889.log 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.45800.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72556.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.839 INFO:teuthology.orchestra.run.smithi172.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45800.log.gz 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70533.log 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50889.log: /var/log/ceph/ceph-client.admin.45792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55193.log 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.70533.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50889.log.gz 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70533.log.gz 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59707.log 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55193.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73496.log 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59707.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55193.log.gz 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77932.log 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59707.log.gz 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.73496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70904.log 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.77932.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.73496.log.gz 2024-04-14T08:43:58.840 INFO:teuthology.orchestra.run.smithi172.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77932.log.gz 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56880.log 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr: 86.9% -- replaced with /var/log/ceph/ceph.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72216.log 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.70904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81394.log 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8%/var/log/ceph/ceph-client.admin.56880.log: -- replaced with /var/log/ceph/ceph-client.admin.70904.log.gz 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72216.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75781.log 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72216.log.gz 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81394.log: -- replaced with /var/log/ceph/ceph-client.admin.56880.log.gz 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81394.log.gz 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr: -- /var/log/ceph/ceph-client.admin.67780.log 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53664.log 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75781.log.gz 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.67780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65600.log 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67780.log.gz 2024-04-14T08:43:58.841 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81530.log 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53664.log.gz 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65600.log: /var/log/ceph/ceph-client.admin.81530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55959.log 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.2.log: 88.0% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81530.log.gz 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65600.log.gz 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69722.log 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67696.log 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55959.log.gz 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46135.log 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.67696.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67696.log.gz 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64733.log 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46135.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46135.log.gz 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.69722.log.gz 2024-04-14T08:43:58.842 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80051.log 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.64733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64733.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52424.log 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44768.log 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.52424.log: /var/log/ceph/ceph-client.admin.80051.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52424.log.gz 90.2% -- replaced with /var/log/ceph/ceph-client.admin.80051.log.gz 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57764.log 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.44768.log: /var/log/ceph/ceph-mds.c.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44768.log.gz 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46542.log 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57764.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57764.log.gz 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57090.log 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46542.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46542.log.gz 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43723.log 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57090.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57090.log.gz 2024-04-14T08:43:58.843 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50343.log 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43723.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43723.log.gz 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72174.log 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50343.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50343.log.gz 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55361.log 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72174.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72174.log.gz 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52064.log 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55361.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55361.log.gz 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49272.log 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.52064.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52064.log.gz 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79442.log 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49272.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49272.log.gz 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72640.log 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.79442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79442.log.gz 2024-04-14T08:43:58.844 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67654.log 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72640.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.72640.log.gz 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77644.log 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.67654.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67654.log.gz 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65044.log 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.77644.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77644.log.gz 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59515.log 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65044.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65044.log.gz 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49881.log 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.59515.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.59515.log.gz 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42925.log 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49881.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49881.log.gz 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31489.log 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.42925.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42925.log.gz 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50055.log 2024-04-14T08:43:58.845 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph.tmp-client.admin.31489.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31489.log.gz 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81782.log 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50055.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50055.log.gz 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80383.log 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81782.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81782.log.gz 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63126.log 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.80383.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.80383.log.gz 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82053.log 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.63126.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63126.log.gz 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76243.log 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82053.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82053.log.gz 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76584.log 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76243.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76243.log.gz 2024-04-14T08:43:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54247.log 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76584.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76584.log.gz 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63384.log 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54247.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54247.log.gz 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43363.log 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.63384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63384.log.gz 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61510.log 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43363.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43363.log.gz 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76338.log 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.61510.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61510.log.gz 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56534.log 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.76338.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76338.log.gz 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65997.log 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56534.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56534.log.gz 2024-04-14T08:43:58.847 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73519.log 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.65997.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65997.log.gz 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44910.log 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.73519.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73519.log.gz 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56087.log 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.44910.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44910.log.gz 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53580.log 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56087.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56087.log.gz 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46133.log 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53580.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53580.log.gz 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52612.log 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46133.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46133.log.gz 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82514.log 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.52612.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52612.log.gz 2024-04-14T08:43:58.848 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56367.log 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82514.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82514.log.gz 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62168.log 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.56367.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.56367.log.gz 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54468.log 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.62168.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62168.log.gz 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52930.log 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.54468.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54468.log.gz 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57006.log 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.52930.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52930.log.gz 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79142.log 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57006.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57006.log.gz 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51531.log 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.79142.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79142.log.gz 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44684.log 2024-04-14T08:43:58.849 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51531.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51531.log.gz 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69149.log 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.44684.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44684.log.gz 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83310.log 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69149.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69149.log.gz 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72132.log 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.83310.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83310.log.gz 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55916.log 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72132.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72132.log.gz 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66888.log 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55916.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55916.log.gz 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51183.log 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66888.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66888.log.gz 2024-04-14T08:43:58.850 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57887.log 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.51183.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51183.log.gz 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74603.log 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.57887.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57887.log.gz 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66466.log 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.74603.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74603.log.gz 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43975.log 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66466.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66466.log.gz 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48852.log 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43975.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43975.log.gz 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82346.log 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48852.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48852.log.gz 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53392.log 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82346.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.82346.log.gz 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69289.log 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.53392.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.53392.log.gz 2024-04-14T08:43:58.851 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72388.log 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.69289.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69289.log.gz 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60340.log 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.72388.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72388.log.gz 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46626.log 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60340.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60340.log.gz 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62060.log 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.46626.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46626.log.gz 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43321.log 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.62060.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62060.log.gz 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71250.log 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43321.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43321.log.gz 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75205.log 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71250.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71250.log.gz 2024-04-14T08:43:58.852 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60036.log 2024-04-14T08:43:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75205.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.75205.log.gz 2024-04-14T08:43:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58144.log 2024-04-14T08:43:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60036.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60036.log.gz 2024-04-14T08:43:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66380.log 2024-04-14T08:43:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58144.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58144.log.gz 2024-04-14T08:43:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63629.log 2024-04-14T08:43:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.66380.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66380.log.gz 2024-04-14T08:43:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55595.log 2024-04-14T08:43:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.63629.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63629.log.gz 2024-04-14T08:43:58.853 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49467.log 2024-04-14T08:43:58.854 INFO:teuthology.orchestra.run.smithi177.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-14T08:43:58.870 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.55595.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55595.log.gz 2024-04-14T08:43:58.870 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50595.log 2024-04-14T08:43:58.870 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.49467.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49467.log.gz 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79438.log 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50595.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50595.log.gz 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43491.log 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.79438.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79438.log.gz 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75571.log 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.43491.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43491.log.gz 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78714.log 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.75571.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75571.log.gz 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78672.log 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.78714.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78714.log.gz 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.78672.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78672.log.gz 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73454.log 2024-04-14T08:43:58.871 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mds.a.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60172.log 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.73454.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73454.log.gz 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81824.log 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.60172.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60172.log.gz 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80642.log 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.81824.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81824.log.gz 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63671.log 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.80642.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80642.log.gz 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50805.log 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.63671.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63671.log.gz 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71791.log 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.50805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48684.log 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50805.log.gz 2024-04-14T08:43:58.872 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.71791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63192.log 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48684.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71791.log.gz 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70641.log 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48684.log.gz 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.63192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83144.log 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.70641.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.63192.log.gz 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70641.log.gz 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64325.log 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58014.log 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.83144.log: /var/log/ceph/ceph-client.admin.64325.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83144.log.gz 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64325.log.gzgzip 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61892.log 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62356.log 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.58014.log: /var/log/ceph/ceph-client.admin.61892.log: 88.5% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58014.log.gz 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61892.log.gz 2024-04-14T08:43:58.873 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48126.log 2024-04-14T08:43:58.874 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.0.62356.log: /var/log/ceph/ceph-client.admin.82430.log 2024-04-14T08:43:58.874 INFO:teuthology.orchestra.run.smithi172.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62356.log.gz 2024-04-14T08:43:58.874 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.48126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47046.log 2024-04-14T08:43:58.874 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.82430.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48126.log.gz 2024-04-14T08:43:58.874 INFO:teuthology.orchestra.run.smithi172.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82430.log.gz 2024-04-14T08:43:58.874 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68026.log 2024-04-14T08:43:58.874 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.47046.log: /var/log/ceph/ceph-client.admin.68026.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68026.log.gz 2024-04-14T08:43:58.874 INFO:teuthology.orchestra.run.smithi172.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47046.log.gz 2024-04-14T08:43:58.897 INFO:teuthology.orchestra.run.smithi177.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-14T08:43:58.944 INFO:teuthology.orchestra.run.smithi172.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-14T08:43:59.363 INFO:teuthology.orchestra.run.smithi172.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-14T08:43:59.783 INFO:teuthology.orchestra.run.smithi177.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-14T08:43:59.799 INFO:teuthology.orchestra.run.smithi177.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-14T08:43:59.806 INFO:teuthology.orchestra.run.smithi177.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-14T08:43:59.812 INFO:teuthology.orchestra.run.smithi172.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-14T08:43:59.835 INFO:teuthology.orchestra.run.smithi177.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-14T08:43:59.837 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-04-14T08:43:59.837 INFO:teuthology.orchestra.run.smithi177.stderr:real 0m1.343s 2024-04-14T08:43:59.837 INFO:teuthology.orchestra.run.smithi177.stderr:user 0m5.892s 2024-04-14T08:43:59.837 INFO:teuthology.orchestra.run.smithi177.stderr:sys 0m0.227s 2024-04-14T08:43:59.898 INFO:teuthology.orchestra.run.smithi172.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-14T08:43:59.918 INFO:teuthology.orchestra.run.smithi172.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-14T08:44:00.175 INFO:teuthology.orchestra.run.smithi172.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-14T08:44:00.177 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-14T08:44:00.177 INFO:teuthology.orchestra.run.smithi172.stderr:real 0m1.714s 2024-04-14T08:44:00.177 INFO:teuthology.orchestra.run.smithi172.stderr:user 0m6.722s 2024-04-14T08:44:00.177 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m0.625s 2024-04-14T08:44:00.177 INFO:tasks.ceph:Archiving logs... 2024-04-14T08:44:00.178 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369/remote/smithi172/log 2024-04-14T08:44:00.179 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-14T08:44:01.848 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369/remote/smithi177/log 2024-04-14T08:44:01.849 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-14T08:44:02.758 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-14T08:44:02.769 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-14T08:44:02.769 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-14T08:44:02.772 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-14T08:44:02.870 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-14T08:44:02.879 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-14T08:44:02.880 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-14T08:44:02.883 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-14T08:44:02.897 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-04-14T08:44:02.899 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-04-14T08:44:02.903 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-14T08:44:02.903 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-04-14T08:44:02.903 INFO:teuthology.orchestra.run.smithi172.stdout:^* hv01.front.sepia.ceph.com 3 6 377 64 +842us[ +841us] +/- 67ms 2024-04-14T08:44:02.903 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 -547us[ -547us] +/- 72ms 2024-04-14T08:44:02.903 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 -400us[ -400us] +/- 69ms 2024-04-14T08:44:02.903 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T08:44:02.904 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-14T08:44:02.904 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-04-14T08:44:02.904 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 +866us[ +875us] +/- 67ms 2024-04-14T08:44:02.905 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -543us[ -534us] +/- 72ms 2024-04-14T08:44:02.905 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 0 -364us[ -364us] +/- 70ms 2024-04-14T08:44:02.905 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T08:44:02.906 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-14T08:44:02.916 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-14T08:44:02.917 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-14T08:44:02.926 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-14T08:44:03.037 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-14T08:44:03.113 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-14T08:44:03.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:44:03.147 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-14T08:44:03.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:44:03.179 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-14T08:44:03.190 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-14T08:44:03.200 INFO:teuthology.task.internal:Duration was 1134.880904 seconds 2024-04-14T08:44:03.201 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-14T08:44:03.214 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-14T08:44:03.214 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-14T08:44:03.217 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-14T08:44:03.259 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-14T08:44:03.262 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-14T08:44:03.565 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-14T08:44:03.566 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2024-04-14T08:44:03.566 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-14T08:44:03.585 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2024-04-14T08:44:03.586 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-14T08:44:03.612 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-14T08:44:03.612 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-14T08:44:03.628 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-14T08:44:03.681 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-14T08:44:03.681 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-14T08:44:03.696 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-14T08:44:03.831 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-14T08:44:03.842 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-14T08:44:03.842 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-14T08:44:03.869 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-14T08:44:03.897 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-14T08:44:03.907 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-14T08:44:03.912 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-14T08:44:03.935 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2024-04-14T08:44:03.963 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2024-04-14T08:44:03.979 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-14T08:44:04.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:44:04.007 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-14T08:44:04.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T08:44:04.035 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-14T08:44:04.045 INFO:teuthology.task.internal:Transferring archived files... 2024-04-14T08:44:04.046 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369/remote/smithi172 2024-04-14T08:44:04.046 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-14T08:44:04.086 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369/remote/smithi177 2024-04-14T08:44:04.086 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-14T08:44:04.121 INFO:teuthology.task.internal:Removing archive directory... 2024-04-14T08:44:04.121 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-14T08:44:04.124 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-14T08:44:04.170 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-14T08:44:04.182 INFO:teuthology.task.internal:Not uploading archives. 2024-04-14T08:44:04.182 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-14T08:44:04.193 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-14T08:44:04.193 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-14T08:44:04.196 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-14T08:44:04.208 INFO:teuthology.orchestra.run.smithi172.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 14 08:44 /home/ubuntu/cephtest 2024-04-14T08:44:04.225 INFO:teuthology.orchestra.run.smithi177.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 14 08:44 /home/ubuntu/cephtest 2024-04-14T08:44:04.226 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-14T08:44:04.238 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-14T08:44:04.261 INFO:teuthology.nuke:Checking targets against current locks 2024-04-14T08:44:04.288 DEBUG:teuthology.nuke:shortname: smithi172 2024-04-14T08:44:04.289 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-14T08:44:04.312 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-14 08:15:50.089990', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLf2A6OlxKixkzXPIy/I95aPoML/ifPAeBpNg+k0+A/BYkM6XAvMsOlkDx7+/5WvDC3MQjg0rXwnu5LDckBmdq0='} 2024-04-14T08:44:04.316 DEBUG:teuthology.nuke:shortname: smithi177 2024-04-14T08:44:04.316 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-14T08:44:04.339 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638369', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-14 08:15:50.091053', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF3YkGOI+hKcHUDd0JnNpKEdi2eUoiseAifR6sFT8/i8WM/z1HIrPCzlwlYhh05ID4cTD0ylLad0oCa3thhmXfQ='} 2024-04-14T08:44:04.374 INFO:teuthology.orchestra.console.smithi172:Power off 2024-04-14T08:44:04.374 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T08:44:04.404 DEBUG:teuthology.orchestra.console.smithi172:power off output: Chassis Power Control: Down/Off 2024-04-14T08:44:04.404 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T08:44:04.425 INFO:teuthology.orchestra.console.smithi177:Power off 2024-04-14T08:44:04.425 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T08:44:04.437 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2024-04-14T08:44:04.453 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off 2024-04-14T08:44:04.453 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T08:44:04.479 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-04-14T08:44:08.438 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T08:44:08.480 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T08:44:08.548 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2024-04-14T08:44:08.592 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-04-14T08:44:12.552 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T08:44:12.594 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T08:44:12.663 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2024-04-14T08:44:12.706 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-04-14T08:44:12.764 INFO:teuthology.orchestra.console.smithi172:Power off completed 2024-04-14T08:44:16.707 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T08:44:16.819 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2024-04-14T08:44:16.920 INFO:teuthology.orchestra.console.smithi177:Power off completed 2024-04-14T08:44:17.025 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} duration: 1134.8809041976929 failure_reason: '"2024-04-14T08:39:04.994014+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-04-14T08:44:17.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T08:44:17.112 INFO:teuthology.run:FAIL